jexidb 2.0.1 → 2.0.3

package/CHANGELOG.md

@@ -0,0 +1,109 @@
+# Changelog
+
+All notable changes to JexiDB will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [2.0.3] - 2024-12-19
+
+### 🚀 Added
+- **Intelligent Auto-Save System**: Automatic data persistence with configurable thresholds and intervals
+  - `autoSaveThreshold`: Flush buffer when it reaches N records (default: 50)
+  - `autoSaveInterval`: Flush buffer every N milliseconds (default: 5000ms)
+  - `forceSaveOnClose`: Always save when closing database (default: true)
+- **New Public Methods**:
+  - `flush()`: Explicitly flush insertion buffer to disk
+  - `forceSave()`: Force save operation regardless of buffer size
+  - `getBufferStatus()`: Get current buffer state information
+  - `configurePerformance(settings)`: Dynamic performance configuration
+  - `getPerformanceConfig()`: Get current performance settings
+  - `deleteDatabase()`: Permanently delete database file
+  - `removeDatabase()`: Alias for deleteDatabase()
+- **Event-Driven Monitoring**: New events for auto-save operations
+  - `buffer-flush`: Emitted when buffer is flushed
+  - `buffer-full`: Emitted when buffer reaches threshold
+  - `auto-save-timer`: Emitted when time-based auto-save triggers
+  - `save-complete`: Emitted when save operation completes
+  - `close-save-complete`: Emitted when database closes with final save
+  - `performance-configured`: Emitted when performance settings change
+  - `delete-database`: Emitted when database file is deleted
+- **Performance Configuration Options**:
+  - `adaptiveBatchSize`: Adjust batch size based on usage (default: true)
+  - `minBatchSize`: Minimum batch size for flush (default: 10)
+  - `maxBatchSize`: Maximum batch size for performance (default: 200)
+  - `maxMemoryUsage`: Memory usage limits
+  - `maxFlushChunkBytes`: Maximum bytes per flush chunk (default: 8MB)
+
+### 🔧 Changed
+- **API Simplification**: Removed confirmation requirements from `deleteDatabase()` and `removeDatabase()`
+  - Methods now work directly without `force` or `confirm` options
+  - Names are self-explanatory and follow industry standards
+- **Constructor Defaults**: Updated default options for better performance
+  - `batchSize`: Reduced from 100 to 50 for faster response
+  - `autoSave`: Enabled by default (true)
+  - `autoSaveThreshold`: 50 records
+  - `autoSaveInterval`: 5000ms (5 seconds)
+  - `forceSaveOnClose`: Enabled by default (true)
+- **Method Behavior**:
+  - `destroy()`: Now equivalent to `close()` (closes instance, keeps file)
+  - `deleteDatabase()`: Permanently deletes database file
+  - `removeDatabase()`: Alias for `deleteDatabase()`
+
+### 🐛 Fixed
+- **Query Operators**: Fixed issues with `$gt`, `$gte`, `$lt`, `$lte`, `$ne`, `$nin` operators
+- **Data Duplication**: Resolved duplicate results in `find()` operations
+- **Type Preservation**: Fixed numeric values being stored as strings in persistent indexes
+- **Persistence Issues**: Corrected data persistence between database instances
+- **Performance Test Logic**: Fixed test expectations for large dataset operations
+
+### 📚 Documentation
+- **Updated README.md**: Added comprehensive auto-save documentation and examples
+- **Updated API.md**: Added new methods, events, and configuration options
+- **New Examples**: Created `auto-save-example.js` and `close-vs-delete-example.js`
+- **Enhanced Tests**: Added comprehensive test suite for auto-save functionality
+
+### 🧪 Testing
+- **New Test Suite**: Added `tests/auto-save.test.js` with 10 comprehensive tests
+- **All Tests Passing**: 86 tests passing (100% success rate)
+- **Improved Coverage**: Better test coverage for new auto-save features
+
+## [2.0.2] - 2024-12-18
+
+### 🚀 Added
+- **Persistent Indexes**: Indexes are now saved to disk and loaded on startup
+- **Point Reading**: Efficient memory usage - only reads necessary data
+- **Rich Query API**: Support for complex queries with operators, sorting, and pagination
+- **Event-Driven Architecture**: Real-time notifications for all database operations
+- **Automatic Integrity Validation**: Built-in data integrity checking and repair
+- **Legacy Compatibility**: Automatic migration from JexiDB 1.x databases
+- **Pure JavaScript**: No native dependencies, works everywhere
+
+### 🔧 Changed
+- **Performance**: 10-100x faster than JexiDB 1.x
+- **Memory Usage**: 25% less memory consumption
+- **File Format**: Improved JSONL architecture with separate index files
+- **API**: Enhanced query methods with MongoDB-style operators
+
+### 🐛 Fixed
+- **Data Integrity**: Safe truncation and consistent offsets
+- **Test Isolation**: Proper test isolation and cleanup
+- **V8 Dependency**: Removed dependency on V8 engine
+
+## [2.0.1] - 2024-12-17
+
+### 🚀 Added
+- **Initial Release**: First stable version of JexiDB 2.0
+- **JSONL Architecture**: Pure JavaScript JSONL database implementation
+- **Basic CRUD Operations**: Insert, find, update, delete operations
+- **Index Support**: Basic indexing for fast queries
+- **File Management**: Database file creation and management
+
+### 🔧 Changed
+- **Complete Rewrite**: New architecture from ground up
+- **Performance Focus**: Optimized for speed and efficiency
+- **Modern JavaScript**: ES6+ features and async/await support
+
+### 🐛 Fixed
+- **Stability**: Improved error handling and edge cases
+- **Compatibility**: Better Node.js version support

package/README.md

@@ -8,8 +8,10 @@
 - **Persistent Indexes**: Fast searches with disk-persisted indexes that don't need rebuilding
 - **Point Reading**: Efficient memory usage - only reads necessary data
 - **Rich Query API**: Support for complex queries with operators, sorting, and pagination
+- **Intelligent Auto-Save**: Automatic data persistence with configurable thresholds and intervals
+- **Event-Driven Monitoring**: Real-time notifications for all database operations
+- **Performance Optimization**: Adaptive batch sizes and memory management
 - **Automatic Integrity Validation**: Built-in data integrity checking and repair
-- **Event System**: Real-time notifications for database operations
 - **Legacy Compatibility**: Automatic migration from JexiDB 1.x databases
 - **Pure JavaScript**: No native dependencies, works everywhere, easy to deploy
 
@@ -21,22 +23,18 @@ npm install jexidb
 
 ## 🚀 Quick Start
 
+### ESM
+
 ```javascript
-// import { Database } from 'jexidb'
-const { Database } = require('jexidb');
+import Database from 'jexidb';
 
-// Create database with indexes (supports both .jdb and .jsonl)
+// Prefer default import aliased as Database for clarity
 const db = new Database('./users.jdb', {
-  indexes: {
-    id: 'number',
-    email: 'string',
-    age: 'number'
-  },
+  indexes: { id: 'number', email: 'string', age: 'number' },
   autoSave: true,
   validateOnInit: true
 });
 
-// Initialize
 await db.init();
 
 // Event listeners
@@ -45,12 +43,7 @@ db.on('update', (record, index) => console.log(`Record updated at index ${index}
 db.on('save', () => console.log('Changes saved'));
 
 // Insert data
-const user = await db.insert({
-  id: 1,
-  name: 'John Doe',
-  email: 'john@example.com',
-  age: 30
-});
+await db.insert({ id: 1, name: 'John Doe', email: 'john@example.com', age: 30 });
 
 // Search data (both methods work)
 const john = await db.findOne({ id: 1 });
@@ -59,17 +52,29 @@ const youngUsers = await db.find({ age: { '<': 30 } });
 // JexiDB 1.x compatible query
 const results = await db.query({ name: 'john doe' }, { caseInsensitive: true });
 
-// Update data
+// Update / Delete / Save
 await db.update({ id: 1 }, { age: 31 });
-
-// Remove data
 await db.delete({ id: 1 });
-
-// Save changes
 await db.save();
+await db.close();  // or await db.destroy()
+```
+
+### CommonJS
 
-// Destroy database
-await db.destroy();
+```javascript
+const Database = require('jexidb');
+// Alternatively (backward compatible): const { Database } = require('jexidb');
+
+const db = new Database('./users.jdb', {
+  indexes: { id: 'number', email: 'string', age: 'number' }
+});
+
+(async () => {
+  await db.init();
+  await db.insert({ id: 1, name: 'John' });
+  console.log(await db.findOne({ id: 1 }));
+  await db.close();  // or await db.destroy()
+})();
 ```
 
 ## 📚 API Reference
@@ -87,13 +92,73 @@ const db = new Database(filePath, options);
 **Options:**
 ```javascript
 {
+  // Core options
   indexes: {},           // Indexes for fields
-  markDeleted: true,     // Mark as deleted instead of physically removing
-  autoSave: true,        // Automatically save after operations
-  validateOnInit: false  // Validate integrity on initialization
+  create: true,          // Create database if it doesn't exist
+  clear: false,          // Clear database on load
+  
+  // Auto-save configuration
+  autoSave: true,        // Enable intelligent auto-save (default: true)
+  autoSaveThreshold: 50, // Flush buffer when it reaches this many records
+  autoSaveInterval: 5000, // Flush buffer every N milliseconds
+  forceSaveOnClose: true, // Always save when closing database
+  
+  // Performance configuration
+  batchSize: 50,         // Batch size for inserts (reduced for faster response)
+  adaptiveBatchSize: true, // Adjust batch size based on usage
+  minBatchSize: 10,      // Minimum batch size for flush
+  maxBatchSize: 200      // Maximum batch size for performance
 }
 ```
 
+### Auto-Save Intelligence
+
+JexiDB features intelligent auto-save capabilities that automatically manage data persistence without manual intervention.
+
+#### Auto-Save Modes
+
+**Intelligent Auto-Save (Default):**
+- Automatically flushes buffer when it reaches the threshold (default: 50 records)
+- Automatically flushes buffer every N milliseconds (default: 5000ms)
+- Always saves when closing the database
+- Provides real-time feedback through events
+
+**Manual Mode:**
+- Disable auto-save with `autoSave: false`
+- Manually call `flush()` and `save()` when needed
+- Useful for applications requiring precise control over persistence timing
+
+#### Event-Driven Monitoring
+
+```javascript
+// Monitor auto-save operations
+db.on('buffer-flush', (count) => {
+  console.log(`Flushed ${count} records`);
+});
+
+db.on('buffer-full', () => {
+  console.log('Buffer reached threshold');
+});
+
+db.on('auto-save-timer', () => {
+  console.log('Time-based auto-save triggered');
+});
+
+db.on('save-complete', () => {
+  console.log('Database saved successfully');
+});
+```
+
+#### Buffer Status Monitoring
+
+```javascript
+// Check buffer status anytime
+const status = db.getBufferStatus();
+console.log(`Pending: ${status.pendingCount}/${status.bufferSize}`);
+console.log(`Should flush: ${status.shouldFlush}`);
+console.log(`Auto-save enabled: ${status.autoSaveEnabled}`);
+```
+
 ### Main Methods
 
 #### `init()`
@@ -170,8 +235,46 @@ Counts records.
 #### `save()`
 Saves pending changes.
 
+#### `flush()`
+Flushes the insertion buffer to disk immediately.
+
+#### `forceSave()`
+Forces a save operation regardless of buffer size.
+
+#### `getBufferStatus()`
+Gets information about the current buffer state.
+
+#### `configurePerformance(settings)`
+Dynamically configures performance settings.
+
+#### `getPerformanceConfig()`
+Gets current performance configuration.
+
+#### `close()`
+Closes the database instance and saves pending changes.
+
 #### `destroy()`
-Destroys the database.
+Closes the database instance and saves pending changes (equivalent to close()).
+
+```javascript
+await db.destroy()  // Same as: await db.close()
+```
+
+#### `deleteDatabase()`
+**⚠️ WARNING: This permanently deletes the database file!**
+
+Deletes the database file from disk and closes the instance.
+
+```javascript
+await db.deleteDatabase()  // Deletes the database file permanently
+```
+
+#### `removeDatabase()`
+Removes the database file from disk (alias for deleteDatabase).
+
+```javascript
+await db.removeDatabase()  // Same as: await db.deleteDatabase()
+```
 
 #### `validateIntegrity(options)`
 Validates database integrity.
@@ -213,13 +316,25 @@ Index statistics.
 ### Events
 
 ```javascript
+// Core events
 db.on('init', () => console.log('Database initialized'));
 db.on('insert', (record, index) => console.log('Record inserted'));
 db.on('update', (record, index) => console.log('Record updated'));
 db.on('delete', (record, index) => console.log('Record deleted'));
 db.on('before-save', () => console.log('Before save'));
 db.on('save', () => console.log('Save completed'));
+db.on('close', () => console.log('Database closed'));
 db.on('destroy', () => console.log('Database destroyed'));
+db.on('delete-database', () => console.log('Database file deleted'));
+
+// Auto-save events
+db.on('buffer-flush', (count) => console.log(`Buffer flushed: ${count} records`));
+db.on('buffer-full', () => console.log('Buffer reached threshold'));
+db.on('auto-save-timer', () => console.log('Time-based auto-save triggered'));
+db.on('save-complete', () => console.log('Save operation completed'));
+db.on('close-save-complete', () => console.log('Database closed with final save'));
+db.on('close', () => console.log('Database closed'));
+db.on('performance-configured', (config) => console.log('Performance reconfigured'));
 ```
 
 ## 📁 File Structure