@soulcraft/brainy 6.1.0 → 6.2.1

package/CHANGELOG.md

@@ -2,6 +2,277 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+## [6.2.0](https://github.com/soulcraftlabs/brainy/compare/v6.1.0...v6.2.0) (2025-11-20)
+
+### ⚡ Critical Performance Fix
+
+**Fixed VFS tree operations on cloud storage (GCS, S3, Azure, R2, OPFS)**
+
+**Issue:** Despite v6.1.0's PathResolver optimization, `vfs.getTreeStructure()` remained critically slow on cloud storage:
+- **Workshop Production (GCS):** 5,304ms for tree with maxDepth=2
+- **Root Cause:** Tree traversal made 111+ separate storage calls (one per directory)
+- **Why v6.1.0 didn't help:** v6.1.0 optimized path→ID resolution, but tree traversal still called `getChildren()` 111+ times
+
+**Architecture Fix:**
+```
+OLD (v6.1.0):
+- For each directory: getChildren(dirId) → fetch entities → GCS call
+- 111 directories = 111 GCS calls × 50ms = 5,550ms
+
+NEW (v6.2.0):
+1. Traverse graph in-memory to collect all IDs (GraphAdjacencyIndex)
+2. Batch-fetch ALL entities in ONE storage call (brain.batchGet)
+3. Build tree structure from fetched entities
+
+Result: 111 storage calls → 1 storage call
+```
+
+**Performance (Production Measurement):**
+- **GCS:** 5,304ms → ~100ms (**53x faster**)
+- **FileSystem:** Already fast, minimal change
+
+**Files Changed:**
+- `src/vfs/VirtualFileSystem.ts:616-689` - New `gatherDescendants()` method
+- `src/vfs/VirtualFileSystem.ts:691-728` - Updated `getTreeStructure()` to use batch fetch
+- `src/vfs/VirtualFileSystem.ts:730-762` - Updated `getDescendants()` to use batch fetch
+
+**Impact:**
+- ✅ Workshop file explorer now loads instantly on GCS
+- ✅ Clean architecture: one code path, no fallbacks
+- ✅ Production-scale: uses in-memory graph + single batch fetch
+- ✅ Works for ALL storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem)
+
+**Migration:** No code changes required - automatic performance improvement.
+
+### 🚨 Critical Bug Fix: Blob Integrity Check Failures (PERMANENT FIX)
+
+**Fixed blob integrity check failures on cloud storage using key-based dispatch (NO MORE GUESSING)**
+
+**Issue:** Production users reported "Blob integrity check failed" errors when opening files from GCS:
+- **Symptom:** Random file read failures with hash mismatch errors
+- **Root Cause:** `wrapBinaryData()` tried to guess data type by parsing, causing compressed binary that happens to be valid UTF-8 + valid JSON to be stored as parsed objects instead of wrapped binary
+- **Impact:** On read, `JSON.stringify(object)` !== original compressed bytes → hash mismatch → integrity failure
+
+**The Guessing Problem (v5.10.1 - v6.1.0):**
+```typescript
+// FRAGILE: wrapBinaryData() tries to JSON.parse ALL buffers
+wrapBinaryData(compressedBuffer) {
+  try {
+    return JSON.parse(data.toString())  // ← Compressed data accidentally parses!
+  } catch {
+    return {_binary: true, data: base64}
+  }
+}
+
+// FAILURE PATH:
+// 1. WRITE: hash(raw) → compress(raw) → wrapBinaryData(compressed)
+//    → compressed bytes accidentally parse as valid JSON
+//    → stored as parsed object instead of wrapped binary
+// 2. READ: retrieve object → JSON.stringify(object) → decompress
+//    → different bytes than original compressed data
+//    → HASH MISMATCH → "Blob integrity check failed"
+```
+
+**The Permanent Solution (v6.2.0): Key-Based Dispatch**
+
+Stop guessing! The key naming convention **IS** the explicit type contract:
+
+```typescript
+// baseStorage.ts COW adapter (line 371-393)
+put: async (key: string, data: Buffer): Promise<void> => {
+  // NO GUESSING - key format explicitly declares data type:
+  //
+  // JSON keys: 'ref:*', '*-meta:*'
+  // Binary keys: 'blob:*', 'commit:*', 'tree:*'
+
+  const obj = key.includes('-meta:') || key.startsWith('ref:')
+    ? JSON.parse(data.toString())  // Metadata/refs: ALWAYS JSON
+    : { _binary: true, data: data.toString('base64') }  // Blobs: ALWAYS binary
+
+  await this.writeObjectToPath(`_cow/${key}`, obj)
+}
+```
+
+**Why This is Permanent:**
+- ✅ **Zero guessing** - key explicitly declares type
+- ✅ **Works for ANY compression** - gzip, zstd, brotli, future algorithms
+- ✅ **Self-documenting** - code clearly shows intent
+- ✅ **No heuristics** - no fragile first-byte checks or try/catch parsing
+- ✅ **Single source of truth** - key naming convention is the contract
+
+**Files Changed:**
+- `src/storage/baseStorage.ts:371-393` - COW adapter uses key-based dispatch (NO MORE wrapBinaryData)
+- `src/storage/cow/binaryDataCodec.ts:86-119` - Deprecated wrapBinaryData() with warnings
+- `tests/unit/storage/cow/BlobStorage.test.ts:612-705` - Added 4 comprehensive regression tests
+
+**Regression Tests Added:**
+1. JSON-like compressed data (THE KILLER TEST CASE)
+2. All key types dispatch correctly (blob, commit, tree)
+3. Metadata keys handled correctly
+4. Verify wrapBinaryData() never called on write path
+
+**Impact:**
+- ✅ **PERMANENT FIX** - eliminates blob integrity failures forever
+- ✅ Works for ALL storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem)
+- ✅ Works for ALL compression algorithms
+- ✅ Comprehensive regression tests prevent future regressions
+- ✅ No performance cost (key.includes() is fast)
+
+**Migration:** No action required - automatic fix for all blob operations.
+
+### ⚡ Performance Fix: Removed Access Time Updates on Reads
+
+**Fixed 50-100ms GCS write penalty on EVERY file/directory read**
+
+**Issue:** Production GCS performance showed file reads taking significantly longer than expected:
+- **Expected:** ~50ms for file read
+- **Actual:** ~100-150ms for file read
+- **Root Cause:** `updateAccessTime()` called on EVERY `readFile()` and `readdir()` operation
+- **Impact:** Each access time update = 50-100ms GCS write operation + doubled GCS costs
+
+**The Problem:**
+```typescript
+// OLD (v6.1.0):
+async readFile(path: string): Promise<Buffer> {
+  const entity = await this.getEntityByPath(path)
+  await this.updateAccessTime(entityId)  // ← 50-100ms GCS write!
+  return await this.blobStorage.read(blobHash)
+}
+
+async readdir(path: string): Promise<string[]> {
+  const entity = await this.getEntityByPath(path)
+  await this.updateAccessTime(entityId)  // ← 50-100ms GCS write!
+  return children.map(child => child.metadata.name)
+}
+```
+
+**Why Access Time Updates Are Harmful:**
+1. **Performance:** 50-100ms penalty on cloud storage for EVERY read
+2. **Cost:** Doubles GCS operation costs (read + write for every file access)
+3. **Unnecessary:** Modern filesystems use `noatime` mount option for same reason
+4. **Unused:** The `accessed` field was NEVER used in queries, filters, or application logic
+
+**Solution (v6.2.0): Remove Completely**
+
+Following modern filesystem best practices (Linux `noatime`, macOS default behavior):
+- ✅ Removed `updateAccessTime()` call from `readFile()` (line 372)
+- ✅ Removed `updateAccessTime()` call from `readdir()` (line 1002)
+- ✅ Removed `updateAccessTime()` method entirely (lines 1355-1365)
+- ✅ Field `accessed` still exists in metadata for backward compatibility (just won't update)
+
+**Performance Impact (Production Scale):**
+- **File reads:** 100-150ms → 50ms (**2-3x faster**)
+- **Directory reads:** 100-150ms → 50ms (**2-3x faster**)
+- **GCS costs:** ~50% reduction (eliminated write operation on every read)
+- **FileSystem:** Minimal impact (already fast, but removes unnecessary disk I/O)
+
+**Files Changed:**
+- `src/vfs/VirtualFileSystem.ts:372-375` - Removed updateAccessTime() from readFile()
+- `src/vfs/VirtualFileSystem.ts:1002-1006` - Removed updateAccessTime() from readdir()
+- `src/vfs/VirtualFileSystem.ts:1355-1365` - Removed updateAccessTime() method
+
+**Impact:**
+- ✅ **2-3x faster reads** on cloud storage
+- ✅ **~50% GCS cost reduction** (no write on every read)
+- ✅ Follows modern filesystem best practices
+- ✅ Backward compatible: field exists but won't update
+- ✅ Works for ALL storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem)
+
+**Migration:** No action required - automatic performance improvement.
+
+### ⚡ Performance Fix: Eliminated N+1 Patterns Across All APIs
+
+**Fixed 8 N+1 patterns for 10-20x faster batch operations on cloud storage**
+
+**Issue:** Multiple APIs loaded entities/relationships one-by-one instead of using batch operations:
+- `find()`: 5 different code paths loaded entities individually
+- `batchGet()` with vectors: Looped through individual `get()` calls
+- `executeGraphSearch()`: Loaded connected entities one-by-one
+- `relate()` duplicate checking: Loaded existing relationships one-by-one
+- `deleteMany()`: Created separate transaction for each entity
+
+**Root Cause:** Individual storage calls instead of batch operations → N × 50ms on GCS = severe latency
+
+**Solution (v6.2.0): Comprehensive Batch Operations**
+
+**1. Fixed `find()` method - 5 locations**
+```typescript
+// OLD: N separate storage calls
+for (const id of pageIds) {
+  const entity = await this.get(id)  // ❌ N×50ms on GCS
+}
+
+// NEW: Single batch call
+const entitiesMap = await this.batchGet(pageIds)  // ✅ 1×50ms on GCS
+for (const id of pageIds) {
+  const entity = entitiesMap.get(id)
+}
+```
+
+**2. Fixed `batchGet()` with vectors**
+- **Added:** `storage.getNounBatch(ids)` method (baseStorage.ts:1986)
+- Batch-loads vectors + metadata in parallel
+- Eliminates N+1 when `includeVectors: true`
+
+**3. Fixed `executeGraphSearch()`**
+- Uses `batchGet()` for connected entities
+- 20 entities: 1,000ms → 50ms (**20x faster**)
+
+**4. Fixed `relate()` duplicate checking**
+- **Added:** `storage.getVerbsBatch(ids)` method (baseStorage.ts:826)
+- **Added:** `graphIndex.getVerbsBatchCached(ids)` method (graphAdjacencyIndex.ts:384)
+- Batch-loads existing relationships with cache-aware loading
+- 5 verbs: 250ms → 50ms (**5x faster**)
+
+**5. Fixed `deleteMany()`**
+- **Changed:** Batches deletes into chunks of 10
+- Single transaction per chunk (atomic within chunk)
+- 10 entities: 2,000ms → 200ms (**10x faster**)
+- Proper error handling with `continueOnError` flag
+
+**Performance Impact (Production GCS):**
+
+| Operation | Before | After | Speedup |
+|-----------|--------|-------|---------|
+| find() with 10 results | 10×50ms = 500ms | 1×50ms = 50ms | **10x** |
+| batchGet() with vectors (10 entities) | 10×50ms = 500ms | 1×50ms = 50ms | **10x** |
+| executeGraphSearch() with 20 entities | 20×50ms = 1000ms | 1×50ms = 50ms | **20x** |
+| relate() duplicate check (5 verbs) | 5×50ms = 250ms | 1×50ms = 50ms | **5x** |
+| deleteMany() with 10 entities | 10 txns = 2000ms | 1 txn = 200ms | **10x** |
+
+**Files Changed:**
+- `src/brainy.ts:1682-1690` - find() location 1 (batch load)
+- `src/brainy.ts:1713-1720` - find() location 2 (batch load)
+- `src/brainy.ts:1820-1832` - find() location 3 (batch load filtered results)
+- `src/brainy.ts:1845-1853` - find() location 4 (batch load paginated)
+- `src/brainy.ts:1870-1878` - find() location 5 (batch load sorted)
+- `src/brainy.ts:724-732` - batchGet() with vectors optimization
+- `src/brainy.ts:1171-1183` - relate() duplicate check optimization
+- `src/brainy.ts:2216-2310` - deleteMany() transaction batching
+- `src/brainy.ts:4314-4325` - executeGraphSearch() batch load
+- `src/storage/baseStorage.ts:1986-2045` - Added getNounBatch()
+- `src/storage/baseStorage.ts:826-886` - Added getVerbsBatch()
+- `src/graph/graphAdjacencyIndex.ts:384-413` - Added getVerbsBatchCached()
+- `src/coreTypes.ts:721,743` - Added batch methods to StorageAdapter interface
+- `src/types/brainy.types.ts:367` - Added continueOnError to DeleteManyParams
+
+**Architecture:**
+- ✅ **COW/fork/asOf**: All batch methods use `readBatchWithInheritance()`
+- ✅ **All storage adapters**: Works with GCS, S3, Azure, R2, OPFS, FileSystem
+- ✅ **Caching**: getVerbsBatchCached() checks UnifiedCache first
+- ✅ **Transactions**: deleteMany() batches into atomic chunks
+- ✅ **Error handling**: Proper error collection with continueOnError support
+
+**Impact:**
+- ✅ **10-20x faster** batch operations on cloud storage
+- ✅ **50-90% cost reduction** (fewer storage API calls)
+- ✅ Clean architecture - no fallbacks, no hacks
+- ✅ Backward compatible - automatic performance improvement
+
+**Migration:** No action required - automatic performance improvement.
+
+---
+
 ## [6.1.0](https://github.com/soulcraftlabs/brainy/compare/v6.0.2...v6.1.0) (2025-11-20)
 
 ### 🚀 Features

package/dist/augmentations/KnowledgeAugmentation.d.ts

@@ -0,0 +1,40 @@
+/**
+ * Knowledge Layer Augmentation for VFS
+ *
+ * Adds intelligent features to VFS without modifying core functionality:
+ * - Event recording for all operations
+ * - Semantic versioning based on content changes
+ * - Entity and concept extraction
+ * - Git bridge for import/export
+ *
+ * This is a TRUE augmentation - VFS works perfectly without it
+ */
+import { Brainy } from '../brainy.js';
+import { BaseAugmentation } from './brainyAugmentation.js';
+export declare class KnowledgeAugmentation extends BaseAugmentation {
+    name: string;
+    timing: 'after';
+    metadata: 'none';
+    operations: any;
+    priority: number;
+    constructor(config?: any);
+    execute<T = any>(operation: string, params: any, next: () => Promise<T>): Promise<T>;
+    private eventRecorder?;
+    private semanticVersioning?;
+    private entitySystem?;
+    private conceptSystem?;
+    private gitBridge?;
+    private originalMethods;
+    initialize(context: any): Promise<void>;
+    augment(brain: Brainy): Promise<void>;
+    /**
+     * Wrap a VFS method to add Knowledge Layer functionality
+     */
+    private wrapMethod;
+    /**
+     * Add Knowledge Layer methods to VFS
+     */
+    private addKnowledgeMethods;
+    private isSemanticChange;
+    cleanup(brain: Brainy): Promise<void>;
+}

package/dist/augmentations/KnowledgeAugmentation.js

@@ -0,0 +1,251 @@
+/**
+ * Knowledge Layer Augmentation for VFS
+ *
+ * Adds intelligent features to VFS without modifying core functionality:
+ * - Event recording for all operations
+ * - Semantic versioning based on content changes
+ * - Entity and concept extraction
+ * - Git bridge for import/export
+ *
+ * This is a TRUE augmentation - VFS works perfectly without it
+ */
+import { BaseAugmentation } from './brainyAugmentation.js';
+import { EventRecorder } from '../vfs/EventRecorder.js';
+import { SemanticVersioning } from '../vfs/SemanticVersioning.js';
+import { PersistentEntitySystem } from '../vfs/PersistentEntitySystem.js';
+import { ConceptSystem } from '../vfs/ConceptSystem.js';
+import { GitBridge } from '../vfs/GitBridge.js';
+export class KnowledgeAugmentation extends BaseAugmentation {
+    constructor(config = {}) {
+        super(config);
+        this.name = 'knowledge';
+        this.timing = 'after'; // Process after VFS operations
+        this.metadata = 'none'; // No metadata access needed
+        this.operations = []; // VFS-specific augmentation, no operation interception
+        this.priority = 100; // Run last
+        this.originalMethods = new Map();
+    }
+    async execute(operation, params, next) {
+        // Pass through - this augmentation works at VFS level, not operation level
+        return await next();
+    }
+    async initialize(context) {
+        await this.augment(context.brain);
+    }
+    async augment(brain) {
+        // Only augment if VFS exists
+        const vfs = brain.vfs?.();
+        if (!vfs) {
+            console.warn('KnowledgeAugmentation: VFS not found, skipping');
+            return;
+        }
+        // Initialize Knowledge Layer components
+        this.eventRecorder = new EventRecorder(brain);
+        this.semanticVersioning = new SemanticVersioning(brain);
+        this.entitySystem = new PersistentEntitySystem(brain);
+        this.conceptSystem = new ConceptSystem(brain);
+        this.gitBridge = new GitBridge(vfs, brain);
+        // Wrap VFS methods to add intelligence WITHOUT slowing them down
+        this.wrapMethod(vfs, 'writeFile', async (original, path, data, options) => {
+            // Call original first (stays fast)
+            const result = await original.call(vfs, path, data, options);
+            // Knowledge processing in background (non-blocking)
+            setImmediate(async () => {
+                try {
+                    // Record event
+                    if (this.eventRecorder) {
+                        await this.eventRecorder.recordEvent({
+                            type: 'write',
+                            path,
+                            content: data,
+                            size: data.length,
+                            author: options?.author || 'system'
+                        });
+                    }
+                    // Check for semantic versioning
+                    if (this.semanticVersioning) {
+                        const existingContent = await vfs.readFile(path).catch(() => null);
+                        const shouldVersion = existingContent && this.isSemanticChange(existingContent, data);
+                        if (shouldVersion) {
+                            await this.semanticVersioning.createVersion(path, data, {
+                                message: 'Automatic semantic version'
+                            });
+                        }
+                    }
+                    // Extract concepts
+                    if (this.conceptSystem && options?.extractConcepts !== false) {
+                        await this.conceptSystem.extractAndLinkConcepts(path, data);
+                    }
+                    // Extract entities
+                    if (this.entitySystem && options?.extractEntities !== false) {
+                        await this.entitySystem.extractEntities(data.toString('utf8'), data);
+                    }
+                }
+                catch (error) {
+                    // Knowledge Layer errors should not affect VFS operations
+                    console.debug('KnowledgeLayer background processing error:', error);
+                }
+            });
+            return result;
+        });
+        this.wrapMethod(vfs, 'unlink', async (original, path) => {
+            const result = await original.call(vfs, path);
+            // Record deletion event
+            setImmediate(async () => {
+                if (this.eventRecorder) {
+                    await this.eventRecorder.recordEvent({
+                        type: 'delete',
+                        path,
+                        author: 'system'
+                    });
+                }
+            });
+            return result;
+        });
+        this.wrapMethod(vfs, 'rename', async (original, oldPath, newPath) => {
+            const result = await original.call(vfs, oldPath, newPath);
+            // Record rename event
+            setImmediate(async () => {
+                if (this.eventRecorder) {
+                    await this.eventRecorder.recordEvent({
+                        type: 'rename',
+                        path: oldPath,
+                        metadata: { newPath },
+                        author: 'system'
+                    });
+                }
+            });
+            return result;
+        });
+        // Add Knowledge Layer methods to VFS
+        this.addKnowledgeMethods(vfs);
+        console.log('✨ Knowledge Layer augmentation enabled');
+    }
+    /**
+     * Wrap a VFS method to add Knowledge Layer functionality
+     */
+    wrapMethod(vfs, methodName, wrapper) {
+        const original = vfs[methodName];
+        if (!original)
+            return;
+        // Store original for cleanup
+        this.originalMethods.set(methodName, original);
+        // Replace with wrapped version
+        vfs[methodName] = async (...args) => {
+            return await wrapper(original, ...args);
+        };
+    }
+    /**
+     * Add Knowledge Layer methods to VFS
+     */
+    addKnowledgeMethods(vfs) {
+        // Event history
+        vfs.getHistory = async (path, options) => {
+            if (!this.eventRecorder)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.eventRecorder.getHistory(path, options);
+        };
+        vfs.reconstructAtTime = async (path, timestamp) => {
+            if (!this.eventRecorder)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.eventRecorder.reconstructFileAtTime(path, timestamp);
+        };
+        // Semantic versioning
+        vfs.getVersions = async (path) => {
+            if (!this.semanticVersioning)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.semanticVersioning.getVersions(path);
+        };
+        vfs.restoreVersion = async (path, versionId) => {
+            if (!this.semanticVersioning)
+                throw new Error('Knowledge Layer not initialized');
+            const version = await this.semanticVersioning.getVersion(path, versionId);
+            if (version) {
+                await vfs.writeFile(path, version);
+            }
+        };
+        // Entities
+        vfs.findEntity = async (query) => {
+            if (!this.entitySystem)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.entitySystem.findEntity(query);
+        };
+        vfs.getEntityAppearances = async (entityId) => {
+            if (!this.entitySystem)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.entitySystem.getEvolution(entityId);
+        };
+        // Concepts
+        vfs.getConcepts = async (path) => {
+            if (!this.conceptSystem)
+                throw new Error('Knowledge Layer not initialized');
+            const concepts = await this.conceptSystem.findConcepts({ manifestedIn: path });
+            return concepts;
+        };
+        vfs.getConceptGraph = async (options) => {
+            if (!this.conceptSystem)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.conceptSystem.getConceptGraph(options);
+        };
+        // Git bridge
+        vfs.exportToGit = async (vfsPath, gitPath) => {
+            if (!this.gitBridge)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.gitBridge.exportToGit(vfsPath, gitPath);
+        };
+        vfs.importFromGit = async (gitPath, vfsPath) => {
+            if (!this.gitBridge)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.gitBridge.importFromGit(gitPath, vfsPath);
+        };
+        // Temporal coupling
+        vfs.findTemporalCoupling = async (path, windowMs) => {
+            if (!this.eventRecorder)
+                throw new Error('Knowledge Layer not initialized');
+            return await this.eventRecorder.findTemporalCoupling(path, windowMs);
+        };
+    }
+    isSemanticChange(oldContent, newContent) {
+        // Simple heuristic - significant size change or different content
+        const oldStr = oldContent.toString('utf8');
+        const newStr = newContent.toString('utf8');
+        // Check for significant size change (>10%)
+        const sizeDiff = Math.abs(oldStr.length - newStr.length) / oldStr.length;
+        if (sizeDiff > 0.1)
+            return true;
+        // Check for structural changes (simplified)
+        const oldLines = oldStr.split('\n').filter(l => l.trim());
+        const newLines = newStr.split('\n').filter(l => l.trim());
+        // Different number of non-empty lines
+        return Math.abs(oldLines.length - newLines.length) > 5;
+    }
+    async cleanup(brain) {
+        const vfs = brain.vfs?.();
+        if (!vfs)
+            return;
+        // Restore original methods
+        for (const [methodName, original] of this.originalMethods) {
+            vfs[methodName] = original;
+        }
+        // Remove added methods
+        delete vfs.getHistory;
+        delete vfs.reconstructAtTime;
+        delete vfs.getVersions;
+        delete vfs.restoreVersion;
+        delete vfs.findEntity;
+        delete vfs.getEntityAppearances;
+        delete vfs.getConcepts;
+        delete vfs.getConceptGraph;
+        delete vfs.exportToGit;
+        delete vfs.importFromGit;
+        delete vfs.findTemporalCoupling;
+        // Clean up components
+        this.eventRecorder = undefined;
+        this.semanticVersioning = undefined;
+        this.entitySystem = undefined;
+        this.conceptSystem = undefined;
+        this.gitBridge = undefined;
+        console.log('Knowledge Layer augmentation removed');
+    }
+}
+//# sourceMappingURL=KnowledgeAugmentation.js.map

package/dist/brainy.d.ts

@@ -1566,9 +1566,11 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
     get counts(): {
         entities: () => number;
         relationships: () => number;
-        byType: (type?: string) => number | {
-            [k: string]: number;
-        };
+        byType: (typeOrOptions?: string | {
+            excludeVFS?: boolean;
+        }, options?: {
+            excludeVFS?: boolean;
+        }) => Promise<number | Record<string, number>>;
         byTypeEnum: (type: NounType) => number;
         topTypes: (n?: number) => NounType[];
         topVerbTypes: (n?: number) => VerbType[];
@@ -1579,12 +1581,12 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
         };
         byCriteria: (field: string, value: any) => Promise<number>;
         getAllTypeCounts: () => Map<string, number>;
-        getStats: () => {
+        getStats: (options?: {
+            excludeVFS?: boolean;
+        }) => Promise<{
             entities: {
                 total: number;
-                byType: {
-                    [k: string]: number;
-                };
+                byType: Record<string, number>;
             };
             relationships: {
                 totalRelationships: number;
@@ -1594,7 +1596,7 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
                 totalNodes: number;
             };
             density: number;
-        };
+        }>;
     };
     /**
      * Augmentations API - Clean and simple
@@ -1607,14 +1609,16 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
     /**
      * Get complete statistics - convenience method
      * For more granular counting, use brain.counts API
+     * v6.2.1: Added optional excludeVFS using Roaring bitmap intersection
+     * @param options Optional settings - excludeVFS: filter out VFS entities
      * @returns Complete statistics including entities, relationships, and density
      */
-    getStats(): {
+    getStats(options?: {
+        excludeVFS?: boolean;
+    }): Promise<{
         entities: {
             total: number;
-            byType: {
-                [k: string]: number;
-            };
+            byType: Record<string, number>;
         };
         relationships: {
             totalRelationships: number;
@@ -1624,7 +1628,7 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
             totalNodes: number;
         };
         density: number;
-    };
+    }>;
     /**
      * Parse natural language query using advanced NLP with 220+ patterns
      * The embedding model is always available as it's core to Brainy's functionality