@soulcraft/brainy 4.1.3 → 4.1.4

package/CHANGELOG.md

@@ -2,6 +2,11 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [4.1.4](https://github.com/soulcraftlabs/brainy/compare/v4.1.3...v4.1.4) (2025-10-21)
+
+- feat: add import API validation and v4.x migration guide (a1a0576)
+
+
 ### [4.1.3](https://github.com/soulcraftlabs/brainy/compare/v4.1.2...v4.1.3) (2025-10-21)
 
 - perf: make getRelations() pagination consistent and efficient (54d819c)
@@ -223,22 +228,110 @@ $ brainy import ./research-papers --extract-concepts --progress
 
 ### ⚠️ Breaking Changes
 
-**NONE** - v4.0.0 is 100% backward compatible!
+#### 💥 Import API Redesign
+
+The import API has been redesigned for clarity and better feature control. **Old v3.x option names are no longer recognized** and will throw errors.
+
+**What Changed:**
+
+| v3.x Option | v4.x Option | Action Required |
+|-------------|-------------|-----------------|
+| `extractRelationships` | `enableRelationshipInference` | **Rename option** |
+| `autoDetect` | *(removed)* | **Delete option** (always enabled) |
+| `createFileStructure` | `vfsPath` | **Replace** with VFS path |
+| `excelSheets` | *(removed)* | **Delete option** (all sheets processed) |
+| `pdfExtractTables` | *(removed)* | **Delete option** (always enabled) |
+| - | `enableNeuralExtraction` | **Add option** (new in v4.x) |
+| - | `enableConceptExtraction` | **Add option** (new in v4.x) |
+| - | `preserveSource` | **Add option** (new in v4.x) |
+
+**Why These Changes?**
+
+1. **Clearer option names**: `enableRelationshipInference` explicitly indicates AI-powered relationship inference
+2. **Separation of concerns**: Neural extraction, relationship inference, and VFS are now separate, explicit options
+3. **Better defaults**: Auto-detection and AI features are enabled by default
+4. **Reduced confusion**: Removed redundant options like `autoDetect` and format-specific options
+
+**Migration Examples:**
+
+<details>
+<summary>Example 1: Basic Excel Import</summary>
+
+```typescript
+// v3.x (OLD - Will throw error)
+await brain.import('./glossary.xlsx', {
+  extractRelationships: true,
+  createFileStructure: true
+})
+
+// v4.x (NEW - Use this)
+await brain.import('./glossary.xlsx', {
+  enableRelationshipInference: true,
+  vfsPath: '/imports/glossary'
+})
+```
+</details>
+
+<details>
+<summary>Example 2: Full-Featured Import</summary>
 
-All v4.0.0 features are:
+```typescript
+// v3.x (OLD - Will throw error)
+await brain.import('./data.xlsx', {
+  extractRelationships: true,
+  autoDetect: true,
+  createFileStructure: true
+})
+
+// v4.x (NEW - Use this)
+await brain.import('./data.xlsx', {
+  enableNeuralExtraction: true,      // Extract entity names
+  enableRelationshipInference: true, // Infer semantic relationships
+  enableConceptExtraction: true,     // Extract entity types
+  vfsPath: '/imports/data',          // VFS directory
+  preserveSource: true               // Save original file
+})
+```
+</details>
+
+**Error Messages:**
+
+If you use old v3.x options, you'll get a clear error message:
+
+```
+❌ Invalid import options detected (Brainy v4.x breaking changes)
+
+The following v3.x options are no longer supported:
+
+  ❌ extractRelationships
+     → Use: enableRelationshipInference
+     → Why: Option renamed for clarity in v4.x
+
+📖 Migration Guide: https://brainy.dev/docs/guides/migrating-to-v4
+```
+
+**Other v4.0.0 Features (Non-Breaking):**
+
+All other v4.0.0 features are:
 - ✅ Opt-in (lifecycle, compression, batch operations)
 - ✅ Additive (new CLI commands, new methods)
 - ✅ Non-breaking (existing code continues to work)
 
 ### 📝 Migration
 
-**No migration required!** All v4.0.0 features are optional enhancements.
+**Import API migration required** if you use `brain.import()` with the old v3.x option names.
 
-To use new features:
+#### Required Changes:
 1. Update to v4.0.0: `npm install @soulcraft/brainy@4.0.0`
-2. Enable lifecycle policies: `brainy storage lifecycle set`
-3. Use batch operations: `brainy storage batch-delete entities.txt`
-4. See `docs/MIGRATION-V3-TO-V4.md` for full feature documentation
+2. Update import calls to use new option names (see table above)
+3. Test your imports - you'll get clear error messages if you use old options
+
+#### Optional Enhancements:
+- Enable lifecycle policies: `brainy storage lifecycle set`
+- Use batch operations: `brainy storage batch-delete entities.txt`
+- See full migration guide: `docs/guides/migrating-to-v4.md`
+
+**Complete Migration Guide:** [docs/guides/migrating-to-v4.md](./docs/guides/migrating-to-v4.md)
 
 ### 🎓 What This Means
 

package/dist/brainy.d.ts

@@ -686,33 +686,91 @@ export declare class Brainy<T = any> implements BrainyInterface<T> {
         limit?: number;
     }): Promise<string[]>;
     /**
-     * Import files with auto-detection and dual storage (VFS + Knowledge Graph)
+     * Import files with intelligent extraction and dual storage (VFS + Knowledge Graph)
      *
      * Unified import system that:
      * - Auto-detects format (Excel, PDF, CSV, JSON, Markdown)
-     * - Extracts entities and relationships
+     * - Extracts entities with AI-powered name/type detection
+     * - Infers semantic relationships from context
      * - Stores in both VFS (organized files) and Knowledge Graph (connected entities)
      * - Links VFS files to graph entities
      *
-     * @example
-     * // Import from file path
-     * const result = await brain.import('/path/to/file.xlsx')
+     * @since 4.0.0
      *
-     * @example
-     * // Import from buffer
+     * @example Quick Start (All AI features enabled by default)
+     * ```typescript
+     * const result = await brain.import('./glossary.xlsx')
+     * // Auto-detects format, extracts entities, infers relationships
+     * ```
+     *
+     * @example Full-Featured Import (v4.x)
+     * ```typescript
+     * const result = await brain.import('./data.xlsx', {
+     *   // AI features
+     *   enableNeuralExtraction: true,      // Extract entity names/metadata
+     *   enableRelationshipInference: true, // Detect semantic relationships
+     *   enableConceptExtraction: true,     // Extract types/concepts
+     *
+     *   // VFS features
+     *   vfsPath: '/imports/my-data',       // Store in VFS directory
+     *   groupBy: 'type',                   // Organize by entity type
+     *   preserveSource: true,              // Keep original file
+     *
+     *   // Progress tracking
+     *   onProgress: (p) => console.log(p.message)
+     * })
+     * ```
+     *
+     * @example Performance Tuning (Large Files)
+     * ```typescript
+     * const result = await brain.import('./huge-file.csv', {
+     *   enableDeduplication: false,  // Skip dedup for speed
+     *   confidenceThreshold: 0.8,    // Higher threshold = fewer entities
+     *   onProgress: (p) => console.log(`${p.processed}/${p.total}`)
+     * })
+     * ```
+     *
+     * @example Import from Buffer or Object
+     * ```typescript
+     * // From buffer
      * const result = await brain.import(buffer, { format: 'pdf' })
      *
-     * @example
-     * // Import JSON object
+     * // From object
      * const result = await brain.import({ entities: [...] })
+     * ```
      *
-     * @example
-     * // Custom VFS path and grouping
-     * const result = await brain.import(buffer, {
-     *   vfsPath: '/my-imports/data',
-     *   groupBy: 'type',
-     *   onProgress: (progress) => console.log(progress.message)
-     * })
+     * @throws {Error} If invalid options are provided (v4.x breaking changes)
+     *
+     * @see {@link https://brainy.dev/docs/api/import API Documentation}
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     *
+     * @remarks
+     * **⚠️ Breaking Changes from v3.x:**
+     *
+     * The import API was redesigned in v4.0.0 for clarity and better feature control.
+     * Old v3.x option names are **no longer recognized** and will throw errors.
+     *
+     * **Option Changes:**
+     * - ❌ `extractRelationships` → ✅ `enableRelationshipInference`
+     * - ❌ `createFileStructure` → ✅ `vfsPath: '/your/path'`
+     * - ❌ `autoDetect` → ✅ *(removed - always enabled)*
+     * - ❌ `excelSheets` → ✅ *(removed - all sheets processed)*
+     * - ❌ `pdfExtractTables` → ✅ *(removed - always enabled)*
+     *
+     * **New Options:**
+     * - ✅ `enableNeuralExtraction` - Extract entity names via AI
+     * - ✅ `enableConceptExtraction` - Extract entity types via AI
+     * - ✅ `preserveSource` - Save original file in VFS
+     *
+     * **If you get an error:**
+     * The error message includes migration instructions and examples.
+     * See the complete migration guide for all details.
+     *
+     * **Why these changes?**
+     * - Clearer option names (explicitly describe what they do)
+     * - Separation of concerns (neural, relationships, VFS are separate)
+     * - Better defaults (AI features enabled by default)
+     * - Reduced confusion (removed redundant options)
      */
     import(source: Buffer | string | object, options?: {
         format?: 'excel' | 'pdf' | 'csv' | 'json' | 'markdown';

package/dist/brainy.js

@@ -1593,33 +1593,91 @@ export class Brainy {
         return options?.limit ? concepts.slice(0, options.limit) : concepts;
     }
     /**
-     * Import files with auto-detection and dual storage (VFS + Knowledge Graph)
+     * Import files with intelligent extraction and dual storage (VFS + Knowledge Graph)
      *
      * Unified import system that:
      * - Auto-detects format (Excel, PDF, CSV, JSON, Markdown)
-     * - Extracts entities and relationships
+     * - Extracts entities with AI-powered name/type detection
+     * - Infers semantic relationships from context
      * - Stores in both VFS (organized files) and Knowledge Graph (connected entities)
      * - Links VFS files to graph entities
      *
-     * @example
-     * // Import from file path
-     * const result = await brain.import('/path/to/file.xlsx')
+     * @since 4.0.0
      *
-     * @example
-     * // Import from buffer
+     * @example Quick Start (All AI features enabled by default)
+     * ```typescript
+     * const result = await brain.import('./glossary.xlsx')
+     * // Auto-detects format, extracts entities, infers relationships
+     * ```
+     *
+     * @example Full-Featured Import (v4.x)
+     * ```typescript
+     * const result = await brain.import('./data.xlsx', {
+     *   // AI features
+     *   enableNeuralExtraction: true,      // Extract entity names/metadata
+     *   enableRelationshipInference: true, // Detect semantic relationships
+     *   enableConceptExtraction: true,     // Extract types/concepts
+     *
+     *   // VFS features
+     *   vfsPath: '/imports/my-data',       // Store in VFS directory
+     *   groupBy: 'type',                   // Organize by entity type
+     *   preserveSource: true,              // Keep original file
+     *
+     *   // Progress tracking
+     *   onProgress: (p) => console.log(p.message)
+     * })
+     * ```
+     *
+     * @example Performance Tuning (Large Files)
+     * ```typescript
+     * const result = await brain.import('./huge-file.csv', {
+     *   enableDeduplication: false,  // Skip dedup for speed
+     *   confidenceThreshold: 0.8,    // Higher threshold = fewer entities
+     *   onProgress: (p) => console.log(`${p.processed}/${p.total}`)
+     * })
+     * ```
+     *
+     * @example Import from Buffer or Object
+     * ```typescript
+     * // From buffer
      * const result = await brain.import(buffer, { format: 'pdf' })
      *
-     * @example
-     * // Import JSON object
+     * // From object
      * const result = await brain.import({ entities: [...] })
+     * ```
      *
-     * @example
-     * // Custom VFS path and grouping
-     * const result = await brain.import(buffer, {
-     *   vfsPath: '/my-imports/data',
-     *   groupBy: 'type',
-     *   onProgress: (progress) => console.log(progress.message)
-     * })
+     * @throws {Error} If invalid options are provided (v4.x breaking changes)
+     *
+     * @see {@link https://brainy.dev/docs/api/import API Documentation}
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     *
+     * @remarks
+     * **⚠️ Breaking Changes from v3.x:**
+     *
+     * The import API was redesigned in v4.0.0 for clarity and better feature control.
+     * Old v3.x option names are **no longer recognized** and will throw errors.
+     *
+     * **Option Changes:**
+     * - ❌ `extractRelationships` → ✅ `enableRelationshipInference`
+     * - ❌ `createFileStructure` → ✅ `vfsPath: '/your/path'`
+     * - ❌ `autoDetect` → ✅ *(removed - always enabled)*
+     * - ❌ `excelSheets` → ✅ *(removed - all sheets processed)*
+     * - ❌ `pdfExtractTables` → ✅ *(removed - always enabled)*
+     *
+     * **New Options:**
+     * - ✅ `enableNeuralExtraction` - Extract entity names via AI
+     * - ✅ `enableConceptExtraction` - Extract entity types via AI
+     * - ✅ `preserveSource` - Save original file in VFS
+     *
+     * **If you get an error:**
+     * The error message includes migration instructions and examples.
+     * See the complete migration guide for all details.
+     *
+     * **Why these changes?**
+     * - Clearer option names (explicitly describe what they do)
+     * - Separation of concerns (neural, relationships, VFS are separate)
+     * - Better defaults (AI features enabled by default)
+     * - Reduced confusion (removed redundant options)
      */
     async import(source, options) {
         // Lazy load ImportCoordinator

package/dist/import/ImportCoordinator.d.ts

@@ -21,7 +21,10 @@ export interface ImportSource {
     /** Optional filename hint */
     filename?: string;
 }
-export interface ImportOptions {
+/**
+ * Valid import options for v4.x
+ */
+export interface ValidImportOptions {
     /** Force specific format (skip auto-detection) */
     format?: SupportedFormat;
     /** VFS root path for imported files */
@@ -55,6 +58,45 @@ export interface ImportOptions {
     /** Progress callback */
     onProgress?: (progress: ImportProgress) => void;
 }
+/**
+ * Deprecated import options from v3.x
+ * Using these will cause TypeScript compile errors
+ *
+ * @deprecated These options are no longer supported in v4.x
+ * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+ */
+export interface DeprecatedImportOptions {
+    /**
+     * @deprecated Use `enableRelationshipInference` instead
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     */
+    extractRelationships?: never;
+    /**
+     * @deprecated Removed in v4.x - auto-detection is now always enabled
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     */
+    autoDetect?: never;
+    /**
+     * @deprecated Use `vfsPath` to specify the directory path instead
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     */
+    createFileStructure?: never;
+    /**
+     * @deprecated Removed in v4.x - all sheets are now processed automatically
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     */
+    excelSheets?: never;
+    /**
+     * @deprecated Removed in v4.x - table extraction is now automatic for PDF imports
+     * @see {@link https://brainy.dev/docs/guides/migrating-to-v4 Migration Guide}
+     */
+    pdfExtractTables?: never;
+}
+/**
+ * Complete import options interface
+ * Combines valid v4.x options with deprecated v3.x options (which cause TypeScript errors)
+ */
+export type ImportOptions = ValidImportOptions & DeprecatedImportOptions;
 export interface ImportProgress {
     stage: 'detecting' | 'extracting' | 'storing-vfs' | 'storing-graph' | 'relationships' | 'complete';
     /** Phase of import - extraction or relationship building (v3.49.0) */
@@ -165,4 +207,14 @@ export declare class ImportCoordinator {
      * Normalize extraction result to unified format (Excel-like structure)
      */
     private normalizeExtractionResult;
+    /**
+     * Validate options and reject deprecated v3.x options (v4.0.0+)
+     * Throws clear errors with migration guidance
+     */
+    private validateOptions;
+    /**
+     * Build detailed error message for invalid options
+     * Respects LOG_LEVEL for verbosity (detailed in dev, concise in prod)
+     */
+    private buildValidationErrorMessage;
 }

package/dist/import/ImportCoordinator.js

@@ -62,6 +62,8 @@ export class ImportCoordinator {
     async import(source, options = {}) {
         const startTime = Date.now();
         const importId = uuidv4();
+        // Validate options (v4.0.0+: Reject deprecated v3.x options)
+        this.validateOptions(options);
         // Normalize source
         const normalizedSource = this.normalizeSource(source, options.format);
         // Report detection stage
@@ -558,5 +560,101 @@ export class ImportCoordinator {
         // Fallback: return as-is
         return result;
     }
+    /**
+     * Validate options and reject deprecated v3.x options (v4.0.0+)
+     * Throws clear errors with migration guidance
+     */
+    validateOptions(options) {
+        const invalidOptions = [];
+        // Check for v3.x deprecated options
+        if ('extractRelationships' in options) {
+            invalidOptions.push({
+                old: 'extractRelationships',
+                new: 'enableRelationshipInference',
+                message: 'Option renamed for clarity in v4.x - explicitly indicates AI-powered relationship inference'
+            });
+        }
+        if ('autoDetect' in options) {
+            invalidOptions.push({
+                old: 'autoDetect',
+                new: '(removed)',
+                message: 'Auto-detection is now always enabled - no need to specify this option'
+            });
+        }
+        if ('createFileStructure' in options) {
+            invalidOptions.push({
+                old: 'createFileStructure',
+                new: 'vfsPath',
+                message: 'Use vfsPath to explicitly specify the virtual filesystem directory path'
+            });
+        }
+        if ('excelSheets' in options) {
+            invalidOptions.push({
+                old: 'excelSheets',
+                new: '(removed)',
+                message: 'All sheets are now processed automatically - no configuration needed'
+            });
+        }
+        if ('pdfExtractTables' in options) {
+            invalidOptions.push({
+                old: 'pdfExtractTables',
+                new: '(removed)',
+                message: 'Table extraction is now automatic for PDF imports'
+            });
+        }
+        // If invalid options found, throw error with detailed message
+        if (invalidOptions.length > 0) {
+            const errorMessage = this.buildValidationErrorMessage(invalidOptions);
+            throw new Error(errorMessage);
+        }
+    }
+    /**
+     * Build detailed error message for invalid options
+     * Respects LOG_LEVEL for verbosity (detailed in dev, concise in prod)
+     */
+    buildValidationErrorMessage(invalidOptions) {
+        // Check environment for verbosity level
+        const verbose = process.env.LOG_LEVEL === 'debug' ||
+            process.env.LOG_LEVEL === 'verbose' ||
+            process.env.NODE_ENV === 'development' ||
+            process.env.NODE_ENV === 'dev';
+        if (verbose) {
+            // DETAILED mode (development)
+            const optionDetails = invalidOptions
+                .map((opt) => `
+  ❌ ${opt.old}
+     → Use: ${opt.new}
+     → Why: ${opt.message}`)
+                .join('\n');
+            return `
+❌ Invalid import options detected (Brainy v4.x breaking changes)
+
+The following v3.x options are no longer supported:
+${optionDetails}
+
+📖 Migration Guide: https://brainy.dev/docs/guides/migrating-to-v4
+💡 Quick Fix Examples:
+
+   Before (v3.x):
+   await brain.import(file, {
+     extractRelationships: true,
+     createFileStructure: true
+   })
+
+   After (v4.x):
+   await brain.import(file, {
+     enableRelationshipInference: true,
+     vfsPath: '/imports/my-data'
+   })
+
+🔗 Full API docs: https://brainy.dev/docs/api/import
+      `.trim();
+        }
+        else {
+            // CONCISE mode (production)
+            const optionsList = invalidOptions.map((o) => `'${o.old}'`).join(', ');
+            return `Invalid import options: ${optionsList}. See https://brainy.dev/docs/guides/migrating-to-v4`;
+        }
+    }
 }
 //# sourceMappingURL=ImportCoordinator.js.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@soulcraft/brainy",
-  "version": "4.1.3",
+  "version": "4.1.4",
   "description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. 31 nouns × 40 verbs for infinite expressiveness.",
   "main": "dist/index.js",
   "module": "dist/index.js",