@soulcraft/brainy 2.8.0 → 2.10.0

package/CHANGELOG.md

@@ -2,6 +2,8 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+## [2.10.0](https://github.com/soulcraftlabs/brainy/compare/v2.9.0...v2.10.0) (2025-08-29)
+
 ## [2.8.0](https://github.com/soulcraftlabs/brainy/compare/v2.7.4...v2.8.0) (2025-08-29)
 
 ## [2.7.4] - 2025-08-29

package/README.md

@@ -29,15 +29,17 @@
 - **Perfect Interoperability**: All tools and AI models speak the same language
 - **Universal Compatibility**: Node.js, Browser, Edge, Workers
 
-## ⚡ Quick Start
+## ⚡ Quick Start - Zero Configuration
 
 ```bash
 npm install @soulcraft/brainy
 ```
 
+### 🎯 **True Zero Configuration**
 ```javascript
-import { BrainyData } from 'brainy'
+import { BrainyData } from '@soulcraft/brainy'
 
+// Just this - auto-detects everything!
 const brain = new BrainyData()
 await brain.init()
 
@@ -109,11 +111,41 @@ await brain.find("Popular JavaScript libraries similar to Vue")
 await brain.find("Documentation about authentication from last month")
 ```
 
-### Zero Configuration Philosophy
-- **No API keys required** - Built-in embedding models
-- **No external dependencies** - Everything included
-- **No complex setup** - Works instantly
-- **Smart defaults** - Optimized out of the box
+### 🎯 Zero Configuration Philosophy
+Brainy 2.9+ automatically configures **everything**:
+
+```javascript
+import { BrainyData, PresetName } from '@soulcraft/brainy'
+
+// 1. Pure zero-config - detects everything
+const brain = new BrainyData()
+
+// 2. Environment presets (strongly typed)
+const devBrain = new BrainyData(PresetName.DEVELOPMENT)     // Memory + verbose
+const prodBrain = new BrainyData(PresetName.PRODUCTION)      // Disk + optimized
+const miniBrain = new BrainyData(PresetName.MINIMAL)         // Q8 + minimal features
+
+// 3. Distributed architecture presets
+const writer = new BrainyData(PresetName.WRITER)             // Write-only instance
+const reader = new BrainyData(PresetName.READER)             // Read-only + caching
+const ingestion = new BrainyData(PresetName.INGESTION_SERVICE) // High-throughput
+const searchAPI = new BrainyData(PresetName.SEARCH_API)      // Low-latency search
+
+// 4. Custom zero-config (type-safe)
+const customBrain = new BrainyData({
+  mode: 'production',
+  model: 'fp32',        // or 'q8', 'auto'
+  storage: 'cloud',     // or 'memory', 'disk', 'auto'
+  features: ['core', 'search', 'cache']
+})
+```
+
+**What's Auto-Detected:**
+- **Storage**: S3/GCS/R2 → Filesystem → Memory (priority order)
+- **Models**: FP32 vs Q8 based on memory/performance  
+- **Features**: Minimal → Default → Full based on environment
+- **Memory**: Optimal cache sizes and batching
+- **Performance**: Threading, chunking, indexing strategies
 
 ### Production Performance
 - **3ms average search** - Lightning fast queries
@@ -121,29 +153,40 @@ await brain.find("Documentation about authentication from last month")
 - **Worker-based embeddings** - Non-blocking operations
 - **Automatic caching** - Intelligent result caching
 
-### Performance Optimization
+### 🎛️ Advanced Configuration (When Needed)
 
-**Q8 Quantized Models** - 75% smaller, faster loading (v2.8.0+)
+Most users **never need this** - zero-config handles everything. For advanced use cases:
 
 ```javascript
-// Default: Full precision (fp32) - maximum compatibility
-const brain = new BrainyData()
+// Model precision control (auto-detected by default)
+const brain = new BrainyData({
+  model: 'fp32'  // Full precision - max compatibility
+})
+const fastBrain = new BrainyData({
+  model: 'q8'    // Quantized - 75% smaller, 99% accuracy  
+})
+
+// Storage control (auto-detected by default)
+const memoryBrain = new BrainyData({ storage: 'memory' })     // RAM only
+const diskBrain = new BrainyData({ storage: 'disk' })         // Local filesystem  
+const cloudBrain = new BrainyData({ storage: 'cloud' })       // S3/GCS/R2
 
-// Optimized: Quantized models (q8) - 75% smaller, 99% accuracy  
-const brainOptimized = new BrainyData({
-  embeddingOptions: { dtype: 'q8' }
+// Legacy full config (still supported)
+const legacyBrain = new BrainyData({
+  storage: { forceMemoryStorage: true },
+  embeddingOptions: { precision: 'fp32' }
 })
 ```
 
 **Model Comparison:**
-- **FP32 (default)**: 90MB, 100% accuracy, maximum compatibility
-- **Q8 (optional)**: 23MB, ~99% accuracy, faster loading
+- **FP32**: 90MB, 100% accuracy, maximum compatibility
+- **Q8**: 23MB, ~99% accuracy, faster loading, smaller memory
 
 **When to use Q8:**
-- ✅ New projects where size/speed matters
 - ✅ Memory-constrained environments  
+- ✅ Faster startup required
 - ✅ Mobile or edge deployments
-- ❌ Existing projects with FP32 data (incompatible embeddings)
+- ❌ Existing FP32 datasets (incompatible embeddings)
 
 **Air-gap deployment:**
 ```bash

package/dist/brainyData.d.ts

@@ -416,6 +416,7 @@ export declare class BrainyData<T = any> implements BrainyDataInterface<T> {
     private allowDirectReads;
     private storageConfig;
     private config;
+    private rawConfig;
     private useOptimizedIndex;
     private _dimensions;
     private loggingConfig;
@@ -472,8 +473,10 @@ export declare class BrainyData<T = any> implements BrainyDataInterface<T> {
     get initialized(): boolean;
     /**
      * Create a new vector database
+     * @param config - Zero-config string ('production', 'development', 'minimal'),
+     *                 simplified config object, or legacy full config
      */
-    constructor(config?: BrainyDataConfig);
+    constructor(config?: BrainyDataConfig | string | any);
     /**
      * Check if the database is in read-only mode and throw an error if it is
      * @throws Error if the database is in read-only mode

package/dist/brainyData.js

@@ -8,6 +8,7 @@ import { HNSWIndexOptimized } from './hnsw/hnswIndexOptimized.js';
 import { cosineDistance, defaultEmbeddingFunction, cleanupWorkerPools, batchEmbed } from './utils/index.js';
 import { getAugmentationVersion } from './utils/version.js';
 import { matchesMetadataFilter } from './utils/metadataFilter.js';
+import { enforceNodeVersion } from './utils/nodeVersionCheck.js';
 import { createNamespacedMetadata, updateNamespacedMetadata, markDeleted, markRestored, isDeleted, getUserMetadata } from './utils/metadataNamespace.js';
 import { PeriodicCleanup } from './utils/periodicCleanup.js';
 import { NounType, VerbType } from './types/graphTypes.js';
@@ -80,6 +81,8 @@ export class BrainyData {
     }
     /**
      * Create a new vector database
+     * @param config - Zero-config string ('production', 'development', 'minimal'),
+     *                 simplified config object, or legacy full config
      */
     constructor(config = {}) {
         this.storage = null;
@@ -123,72 +126,85 @@ export class BrainyData {
         this.partitioner = null;
         this.operationalMode = null;
         this.domainDetector = null;
-        // Store config
-        this.config = config;
+        // Enforce Node.js version requirement for ONNX stability
+        if (typeof process !== 'undefined' && process.version) {
+            enforceNodeVersion();
+        }
+        // Store raw config for processing in init()
+        this.rawConfig = config;
+        // For now, process as legacy config if it's an object
+        // The actual zero-config processing will happen in init() since it's async
+        if (typeof config === 'object') {
+            this.config = config;
+        }
+        else {
+            // String preset or simplified config - use minimal defaults for now
+            this.config = {};
+        }
         // Set dimensions to fixed value of 384 (all-MiniLM-L6-v2 dimension)
         this._dimensions = 384;
         // Set distance function
-        this.distanceFunction = config.distanceFunction || cosineDistance;
+        this.distanceFunction = this.config.distanceFunction || cosineDistance;
         // Always use the optimized HNSW index implementation
         // Configure HNSW with disk-based storage when a storage adapter is provided
-        const hnswConfig = config.hnsw || {};
-        if (config.storageAdapter) {
+        const hnswConfig = this.config.hnsw || {};
+        if (this.config.storageAdapter) {
             hnswConfig.useDiskBasedIndex = true;
         }
         // Temporarily use base HNSW index for metadata filtering
         this.hnswIndex = new HNSWIndex(hnswConfig, this.distanceFunction);
         this.useOptimizedIndex = false;
         // Set storage if provided, otherwise it will be initialized in init()
-        this.storage = config.storageAdapter || null;
+        this.storage = this.config.storageAdapter || null;
         // Store logging configuration
-        if (config.logging !== undefined) {
+        if (this.config.logging !== undefined) {
             this.loggingConfig = {
                 ...this.loggingConfig,
-                ...config.logging
+                ...this.config.logging
             };
         }
         // Set embedding function if provided, otherwise create one with the appropriate verbose setting
-        if (config.embeddingFunction) {
-            this.embeddingFunction = config.embeddingFunction;
+        if (this.config.embeddingFunction) {
+            this.embeddingFunction = this.config.embeddingFunction;
         }
         else {
             this.embeddingFunction = defaultEmbeddingFunction;
         }
         // Set persistent storage request flag
         this.requestPersistentStorage =
-            config.storage?.requestPersistentStorage || false;
+            this.config.storage?.requestPersistentStorage || false;
         // Set read-only flag
-        this.readOnly = config.readOnly || false;
+        this.readOnly = this.config.readOnly || false;
         // Set frozen flag (defaults to false to allow optimizations in readOnly mode)
-        this.frozen = config.frozen || false;
+        this.frozen = this.config.frozen || false;
         // Set lazy loading in read-only mode flag
-        this.lazyLoadInReadOnlyMode = config.lazyLoadInReadOnlyMode || false;
+        this.lazyLoadInReadOnlyMode = this.config.lazyLoadInReadOnlyMode || false;
         // Set write-only flag
-        this.writeOnly = config.writeOnly || false;
+        this.writeOnly = this.config.writeOnly || false;
         // Set allowDirectReads flag
-        this.allowDirectReads = config.allowDirectReads || false;
+        this.allowDirectReads = this.config.allowDirectReads || false;
         // Validate that readOnly and writeOnly are not both true
         if (this.readOnly && this.writeOnly) {
             throw new Error('Database cannot be both read-only and write-only');
         }
         // Set default service name if provided
-        if (config.defaultService) {
-            this.defaultService = config.defaultService;
+        if (this.config.defaultService) {
+            this.defaultService = this.config.defaultService;
         }
         // Store storage configuration for later use in init()
-        this.storageConfig = config.storage || {};
+        this.storageConfig = this.config.storage || {};
         // Store timeout and retry configuration
-        this.timeoutConfig = config.timeouts || {};
-        this.retryConfig = config.retryPolicy || {};
+        this.timeoutConfig = this.config.timeouts || {};
+        this.retryConfig = this.config.retryPolicy || {};
         // Store remote server configuration if provided
-        if (config.remoteServer) {
-            this.remoteServerConfig = config.remoteServer;
+        if (this.config.remoteServer) {
+            this.remoteServerConfig = this.config.remoteServer;
         }
         // Initialize real-time update configuration if provided
-        if (config.realtimeUpdates) {
+        if (this.config.realtimeUpdates) {
             this.realtimeUpdateConfig = {
                 ...this.realtimeUpdateConfig,
-                ...config.realtimeUpdates
+                ...this.config.realtimeUpdates
             };
         }
         // Initialize cache configuration with intelligent defaults
@@ -206,15 +222,15 @@ export class BrainyData {
             }
         };
         // Override defaults with user-provided configuration if available
-        if (config.cache) {
+        if (this.config.cache) {
             this.cacheConfig = {
                 ...this.cacheConfig,
-                ...config.cache
+                ...this.config.cache
             };
         }
         // Store distributed configuration
-        if (config.distributed) {
-            if (typeof config.distributed === 'boolean') {
+        if (this.config.distributed) {
+            if (typeof this.config.distributed === 'boolean') {
                 // Auto-mode enabled
                 this.distributedConfig = {
                     enabled: true
@@ -222,7 +238,7 @@ export class BrainyData {
             }
             else {
                 // Explicit configuration
-                this.distributedConfig = config.distributed;
+                this.distributedConfig = this.config.distributed;
             }
         }
         // Initialize cache auto-configurator first
@@ -350,16 +366,28 @@ export class BrainyData {
                     console.warn('Ignoring s3Storage configuration due to missing required fields');
                 }
             }
-            // Check if specific storage is configured
+            // Check if specific storage is configured (legacy and new formats)
             if (storageOptions.s3Storage || storageOptions.r2Storage ||
                 storageOptions.gcsStorage || storageOptions.forceMemoryStorage ||
-                storageOptions.forceFileSystemStorage) {
-                // Create storage from config
-                const { createStorage } = await import('./storage/storageFactory.js');
-                this.storage = await createStorage(storageOptions);
-                // Wrap in augmentation for consistency
-                const wrapper = new DynamicStorageAugmentation(this.storage);
-                this.augmentations.register(wrapper);
+                storageOptions.forceFileSystemStorage ||
+                typeof storageOptions === 'string') {
+                // Handle string storage types (new zero-config)
+                if (typeof storageOptions === 'string') {
+                    const { createAutoStorageAugmentation } = await import('./augmentations/storageAugmentations.js');
+                    // For now, use auto-detection - TODO: extend to support preferred types
+                    const autoAug = await createAutoStorageAugmentation({
+                        rootDirectory: './brainy-data'
+                    });
+                    this.augmentations.register(autoAug);
+                }
+                else {
+                    // Legacy object config
+                    const { createStorage } = await import('./storage/storageFactory.js');
+                    this.storage = await createStorage(storageOptions);
+                    // Wrap in augmentation for consistency
+                    const wrapper = new DynamicStorageAugmentation(this.storage);
+                    this.augmentations.register(wrapper);
+                }
             }
             else {
                 // Zero-config: auto-select based on environment
@@ -878,6 +906,34 @@ export class BrainyData {
             return;
         }
         this.isInitializing = true;
+        // Process zero-config if needed
+        if (this.rawConfig !== undefined) {
+            try {
+                const { applyZeroConfig } = await import('./config/index.js');
+                const processedConfig = await applyZeroConfig(this.rawConfig);
+                // Apply processed config if it's different from raw
+                if (processedConfig !== this.rawConfig) {
+                    // Log if verbose
+                    if (processedConfig.logging?.verbose) {
+                        console.log('🤖 Zero-config applied successfully');
+                    }
+                    // Update config with processed values
+                    this.config = processedConfig;
+                    // Update relevant properties from processed config
+                    this.storageConfig = processedConfig.storage || {};
+                    this.loggingConfig = processedConfig.logging || { verbose: false };
+                    // Update embedding function if precision was specified
+                    if (processedConfig.embeddingOptions?.precision) {
+                        const { createEmbeddingFunctionWithPrecision } = await import('./config/index.js');
+                        this.embeddingFunction = await createEmbeddingFunctionWithPrecision(processedConfig.embeddingOptions.precision);
+                    }
+                }
+            }
+            catch (error) {
+                console.warn('Zero-config processing failed, using defaults:', error);
+                // Continue with existing config
+            }
+        }
         // CRITICAL: Initialize universal memory manager ONLY for default embedding function
         // This preserves custom embedding functions (like test mocks)
         if (typeof this.embeddingFunction === 'function' && this.embeddingFunction === defaultEmbeddingFunction) {

package/dist/config/distributedPresets-new.d.ts

@@ -0,0 +1,118 @@
+/**
+ * Extended Distributed Configuration Presets
+ * Common patterns for distributed and multi-service architectures
+ * All strongly typed with enums for compile-time safety
+ */
+/**
+ * Strongly typed enum for preset names
+ */
+export declare enum PresetName {
+    PRODUCTION = "production",
+    DEVELOPMENT = "development",
+    MINIMAL = "minimal",
+    ZERO = "zero",
+    WRITER = "writer",
+    READER = "reader",
+    INGESTION_SERVICE = "ingestion-service",
+    SEARCH_API = "search-api",
+    ANALYTICS_SERVICE = "analytics-service",
+    EDGE_CACHE = "edge-cache",
+    BATCH_PROCESSOR = "batch-processor",
+    STREAMING_SERVICE = "streaming-service",
+    ML_TRAINING = "ml-training",
+    SIDECAR = "sidecar"
+}
+/**
+ * Preset categories for organization
+ */
+export declare enum PresetCategory {
+    BASIC = "basic",
+    DISTRIBUTED = "distributed",
+    SERVICE = "service"
+}
+/**
+ * Model precision options
+ */
+export declare enum ModelPrecision {
+    FP32 = "fp32",
+    Q8 = "q8",
+    AUTO = "auto",
+    FAST = "fast",
+    SMALL = "small"
+}
+/**
+ * Storage options
+ */
+export declare enum StorageOption {
+    AUTO = "auto",
+    MEMORY = "memory",
+    DISK = "disk",
+    CLOUD = "cloud"
+}
+/**
+ * Feature set options
+ */
+export declare enum FeatureSet {
+    MINIMAL = "minimal",
+    DEFAULT = "default",
+    FULL = "full",
+    CUSTOM = "custom"
+}
+/**
+ * Distributed role options
+ */
+export declare enum DistributedRole {
+    WRITER = "writer",
+    READER = "reader",
+    HYBRID = "hybrid"
+}
+/**
+ * Preset configuration interface
+ */
+export interface PresetConfig {
+    storage: StorageOption;
+    model: ModelPrecision;
+    features: FeatureSet | string[];
+    distributed: boolean;
+    role?: DistributedRole;
+    readOnly?: boolean;
+    writeOnly?: boolean;
+    allowDirectReads?: boolean;
+    lazyLoadInReadOnlyMode?: boolean;
+    cache?: {
+        hotCacheMaxSize?: number;
+        autoTune?: boolean;
+        batchSize?: number;
+    };
+    verbose?: boolean;
+    description: string;
+    category: PresetCategory;
+}
+/**
+ * Strongly typed preset configurations
+ */
+export declare const PRESET_CONFIGS: Readonly<Record<PresetName, PresetConfig>>;
+/**
+ * Type-safe preset getter
+ */
+export declare function getPreset(name: PresetName): PresetConfig;
+/**
+ * Check if a string is a valid preset name
+ */
+export declare function isValidPreset(name: string): name is PresetName;
+/**
+ * Get presets by category
+ */
+export declare function getPresetsByCategory(category: PresetCategory): PresetName[];
+/**
+ * Get all preset names
+ */
+export declare function getAllPresetNames(): PresetName[];
+/**
+ * Get preset description
+ */
+export declare function getPresetDescription(name: PresetName): string;
+/**
+ * Convert preset config to Brainy config
+ */
+export declare function presetToBrainyConfig(preset: PresetConfig): any;