jexidb 2.0.2 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/.babelrc +13 -0
  2. package/.gitattributes +2 -0
  3. package/CHANGELOG.md +140 -0
  4. package/LICENSE +21 -21
  5. package/README.md +301 -527
  6. package/babel.config.json +5 -0
  7. package/dist/Database.cjs +3896 -0
  8. package/docs/API.md +1051 -0
  9. package/docs/EXAMPLES.md +701 -0
  10. package/docs/README.md +194 -0
  11. package/examples/iterate-usage-example.js +157 -0
  12. package/examples/simple-iterate-example.js +115 -0
  13. package/jest.config.js +24 -0
  14. package/package.json +63 -51
  15. package/scripts/README.md +47 -0
  16. package/scripts/clean-test-files.js +75 -0
  17. package/scripts/prepare.js +31 -0
  18. package/scripts/run-tests.js +80 -0
  19. package/src/Database.mjs +4130 -0
  20. package/src/FileHandler.mjs +1101 -0
  21. package/src/OperationQueue.mjs +279 -0
  22. package/src/SchemaManager.mjs +268 -0
  23. package/src/Serializer.mjs +511 -0
  24. package/src/managers/ConcurrencyManager.mjs +257 -0
  25. package/src/managers/IndexManager.mjs +1403 -0
  26. package/src/managers/QueryManager.mjs +1273 -0
  27. package/src/managers/StatisticsManager.mjs +262 -0
  28. package/src/managers/StreamingProcessor.mjs +429 -0
  29. package/src/managers/TermManager.mjs +278 -0
  30. package/test/$not-operator-with-and.test.js +282 -0
  31. package/test/README.md +8 -0
  32. package/test/close-init-cycle.test.js +256 -0
  33. package/test/critical-bugs-fixes.test.js +1069 -0
  34. package/test/index-persistence.test.js +306 -0
  35. package/test/index-serialization.test.js +314 -0
  36. package/test/indexed-query-mode.test.js +360 -0
  37. package/test/iterate-method.test.js +272 -0
  38. package/test/query-operators.test.js +238 -0
  39. package/test/regex-array-fields.test.js +129 -0
  40. package/test/score-method.test.js +238 -0
  41. package/test/setup.js +17 -0
  42. package/test/term-mapping-minimal.test.js +154 -0
  43. package/test/term-mapping-simple.test.js +257 -0
  44. package/test/term-mapping.test.js +514 -0
  45. package/test/writebuffer-flush-resilience.test.js +204 -0
  46. package/dist/FileHandler.js +0 -688
  47. package/dist/IndexManager.js +0 -353
  48. package/dist/IntegrityChecker.js +0 -364
  49. package/dist/JSONLDatabase.js +0 -1194
  50. package/dist/index.js +0 -617
  51. package/src/FileHandler.js +0 -674
  52. package/src/IndexManager.js +0 -363
  53. package/src/IntegrityChecker.js +0 -379
  54. package/src/JSONLDatabase.js +0 -1248
  55. package/src/index.js +0 -608
package/README.md CHANGED
@@ -1,527 +1,301 @@
1
- # JexiDB - Pure JavaScript JSONL Database
2
-
3
- **JexiDB** is a lightweight, high-performance JSONL (JSON Lines) database for Node.js built in pure JavaScript that provides fast data storage and retrieval with persistent indexing.
4
-
5
- ## 🚀 Features
6
-
7
- - **JSONL Architecture**: Each database is a single JSONL file for simplicity and portability
8
- - **Persistent Indexes**: Fast searches with disk-persisted indexes that don't need rebuilding
9
- - **Point Reading**: Efficient memory usage - only reads necessary data
10
- - **Rich Query API**: Support for complex queries with operators, sorting, and pagination
11
- - **Automatic Integrity Validation**: Built-in data integrity checking and repair
12
- - **Event System**: Real-time notifications for database operations
13
- - **Legacy Compatibility**: Automatic migration from JexiDB 1.x databases
14
- - **Pure JavaScript**: No native dependencies, works everywhere, easy to deploy
15
-
16
- ## 📦 Installation
17
-
18
- ```bash
19
- npm install jexidb
20
- ```
21
-
22
- ## 🚀 Quick Start
23
-
24
- ### ESM
25
-
26
- ```javascript
27
- import Database from 'jexidb';
28
-
29
- // Prefer default import aliased as Database for clarity
30
- const db = new Database('./users.jdb', {
31
- indexes: { id: 'number', email: 'string', age: 'number' },
32
- autoSave: true,
33
- validateOnInit: true
34
- });
35
-
36
- await db.init();
37
-
38
- // Event listeners
39
- db.on('insert', (record, index) => console.log(`Record inserted at index ${index}`));
40
- db.on('update', (record, index) => console.log(`Record updated at index ${index}`));
41
- db.on('save', () => console.log('Changes saved'));
42
-
43
- // Insert data
44
- await db.insert({ id: 1, name: 'John Doe', email: 'john@example.com', age: 30 });
45
-
46
- // Search data (both methods work)
47
- const john = await db.findOne({ id: 1 });
48
- const youngUsers = await db.find({ age: { '<': 30 } });
49
-
50
- // JexiDB 1.x compatible query
51
- const results = await db.query({ name: 'john doe' }, { caseInsensitive: true });
52
-
53
- // Update / Delete / Save
54
- await db.update({ id: 1 }, { age: 31 });
55
- await db.delete({ id: 1 });
56
- await db.save();
57
- await db.destroy();
58
- ```
59
-
60
- ### CommonJS
61
-
62
- ```javascript
63
- const Database = require('jexidb');
64
- // Alternatively (backward compatible): const { Database } = require('jexidb');
65
-
66
- const db = new Database('./users.jdb', {
67
- indexes: { id: 'number', email: 'string', age: 'number' }
68
- });
69
-
70
- (async () => {
71
- await db.init();
72
- await db.insert({ id: 1, name: 'John' });
73
- console.log(await db.findOne({ id: 1 }));
74
- await db.destroy();
75
- })();
76
- ```
77
-
78
- ## 📚 API Reference
79
-
80
- ### Constructor
81
-
82
- ```javascript
83
- const db = new Database(filePath, options);
84
- ```
85
-
86
- **Parameters:**
87
- - `filePath` (string): Path to the main file (.jdb)
88
- - `options` (object): Configuration options
89
-
90
- **Options:**
91
- ```javascript
92
- {
93
- indexes: {}, // Indexes for fields
94
- markDeleted: true, // Mark as deleted instead of physically removing
95
- autoSave: true, // Automatically save after operations
96
- validateOnInit: false // Validate integrity on initialization
97
- }
98
- ```
99
-
100
- ### Main Methods
101
-
102
- #### `init()`
103
- Initializes the database.
104
-
105
- #### `insert(data)`
106
- Inserts a record.
107
-
108
- #### `insertMany(dataArray)`
109
- Inserts multiple records.
110
-
111
- #### `find(criteria, options)` / `query(criteria, options)`
112
- Searches records with optional criteria. Both methods work identically.
113
-
114
- **Supported operators:**
115
- ```javascript
116
- // Comparison
117
- { age: { '>': 25 } }
118
- { age: { '>=': 25 } }
119
- { age: { '<': 30 } }
120
- { age: { '<=': 30 } }
121
- { age: { '!=': 25 } }
122
-
123
- // Arrays
124
- { tags: { in: ['developer', 'admin'] } }
125
- { tags: { nin: ['designer'] } }
126
-
127
- // Strings
128
- { name: { regex: 'john' } }
129
- { name: { contains: 'john' } }
130
- ```
131
-
132
- **Options:**
133
- ```javascript
134
- {
135
- limit: 10, // Limit results
136
- skip: 5, // Skip records
137
- sort: { age: 1 }, // Sorting (1 = ascending, -1 = descending)
138
- caseInsensitive: false, // Case insensitive matching (query() only)
139
- matchAny: false // OR instead of AND
140
- }
141
- ```
142
-
143
- **JexiDB 1.x Compatibility:**
144
- ```javascript
145
- // Both work identically
146
- const results1 = await db.find({ name: 'John' });
147
- const results2 = await db.query({ name: 'John' });
148
-
149
- // Case insensitive query (JexiDB 1.x style)
150
- const results = await db.query({ name: 'john' }, { caseInsensitive: true });
151
- ```
152
-
153
- #### `findOne(criteria, options)`
154
- Searches for one record.
155
-
156
- #### `update(criteria, updateData, options)`
157
- Updates records.
158
-
159
- #### `delete(criteria, options)`
160
- Removes records.
161
-
162
- **Delete options:**
163
- ```javascript
164
- {
165
- physical: false, // Physically remove instead of marking as deleted
166
- limit: 1 // Limit number of records to delete
167
- }
168
- ```
169
-
170
- #### `count(criteria, options)`
171
- Counts records.
172
-
173
- #### `save()`
174
- Saves pending changes.
175
-
176
- #### `destroy()`
177
- Destroys the database.
178
-
179
- #### `validateIntegrity(options)`
180
- Validates database integrity.
181
-
182
- #### `rebuildIndexes(options)`
183
- Rebuilds indexes.
184
-
185
- #### `getStats()`
186
- Gets detailed statistics.
187
-
188
- ### `walk()` Iterator
189
-
190
- For traversing large volumes of data:
191
-
192
- ```javascript
193
- // Traverse all records
194
- for await (const record of db.walk()) {
195
- console.log(record.name);
196
- }
197
-
198
- // With options
199
- for await (const record of db.walk({
200
- limit: 100,
201
- skip: 50,
202
- includeDeleted: false
203
- })) {
204
- console.log(record.name);
205
- }
206
- ```
207
-
208
- ### Properties
209
-
210
- #### `length`
211
- Total number of records.
212
-
213
- #### `indexStats`
214
- Index statistics.
215
-
216
- ### Events
217
-
218
- ```javascript
219
- db.on('init', () => console.log('Database initialized'));
220
- db.on('insert', (record, index) => console.log('Record inserted'));
221
- db.on('update', (record, index) => console.log('Record updated'));
222
- db.on('delete', (record, index) => console.log('Record deleted'));
223
- db.on('before-save', () => console.log('Before save'));
224
- db.on('save', () => console.log('Save completed'));
225
- db.on('destroy', () => console.log('Database destroyed'));
226
- ```
227
-
228
- ## 📁 File Structure
229
-
230
- For each database, 2 files are created:
231
-
232
- ```
233
- users.jdb # Data (JSON Lines format)
234
- users.idx.jdb # Compressed persistent indexes
235
- ```
236
-
237
- ### 🔄 Legacy Compatibility
238
-
239
- JexiDB automatically detects and migrates JexiDB 1.x files:
240
-
241
- **Legacy Format (JexiDB 1.x):**
242
- ```
243
- users.jsonl # Data + indexes + offsets in single file
244
- ```
245
-
246
- **New Format (JexiDB):**
247
- ```
248
- users.jdb # Data + offsets
249
- users.idx.jdb # Compressed indexes
250
- ```
251
-
252
-
253
-
254
- ### 🚀 Persistent Indexes
255
-
256
- JexiDB implements **persistent indexes** that are saved to disk:
257
-
258
- **Benefits:**
259
- - **Fast startup**: No need to read all data to rebuild indexes
260
- - **Scalable**: Works well with large databases (100k+ records)
261
- - **Consistent**: Indexes synchronized with data
262
- - **Portable**: Only 2 files to manage
263
- - **Compressed**: Indexes compressed using gzip
264
-
265
- **🔧 How it works:**
266
- 1. **First open**: Indexes are built by reading data
267
- 2. **Save**: Indexes are persisted and compressed to `users.idx.jdb`
268
- 3. **Reopen**: Indexes are loaded instantly from disk
269
- 4. **Fallback**: If index file is corrupted, rebuilds automatically
270
-
271
- ### JSONL Format
272
-
273
- Each line is a valid JSON record:
274
-
275
- ```json
276
- {"id":1,"name":"John","email":"john@example.com","_created":"2024-12-19T10:00:00.000Z","_updated":"2024-12-19T10:00:00.000Z"}
277
- {"id":2,"name":"Jane","email":"jane@example.com","_created":"2024-12-19T10:01:00.000Z","_updated":"2024-12-19T10:01:00.000Z"}
278
- ```
279
-
280
- ## 🔍 Advanced Examples
281
-
282
- ### Complex Search
283
-
284
- ```javascript
285
- // Young users from New York who are developers
286
- const users = await db.find({
287
- age: { '<': 30 },
288
- 'profile.city': 'New York',
289
- tags: { in: ['developer'] }
290
- }, {
291
- sort: { age: 1 },
292
- limit: 10
293
- });
294
- ```
295
-
296
- ### Batch Update
297
-
298
- ```javascript
299
- // Update age of all users from a city
300
- const updated = await db.update(
301
- { 'profile.city': 'New York' },
302
- { 'profile.country': 'USA' }
303
- );
304
- ```
305
-
306
- ### Integrity Validation
307
-
308
- ```javascript
309
- // Validate integrity with details
310
- const integrity = await db.validateIntegrity({
311
- checkData: true,
312
- checkIndexes: true,
313
- checkOffsets: true,
314
- verbose: true
315
- });
316
-
317
- if (!integrity.isValid) {
318
- console.log('Errors:', integrity.errors);
319
- console.log('Warnings:', integrity.warnings);
320
- }
321
- ```
322
-
323
- ### Detailed Statistics
324
-
325
- ```javascript
326
- const stats = await db.getStats();
327
- console.log('File size:', stats.file.size);
328
- console.log('Total records:', stats.summary.totalRecords);
329
- console.log('Indexes:', stats.indexes.indexCount);
330
- ```
331
-
332
- ## 🧪 Tests
333
-
334
- ```bash
335
- npm test
336
- ```
337
-
338
- **Automatic Cleanup**: The test script automatically removes all test files after execution to keep the project directory clean.
339
-
340
- **Manual Cleanup**: If you need to clean up test files manually:
341
- ```bash
342
- npm run test:clean
343
- ```
344
-
345
- **Available Test Scripts**:
346
- - `npm test` - Run all tests with automatic cleanup
347
- - `npm run test:watch` - Run tests in watch mode
348
- - `npm run test:clean` - Clean up test files manually
349
- - `npm run test:optimized` - Run optimized performance tests
350
- - `npm run test:parallel` - Run tests in parallel
351
- - `npm run test:fast` - Run fast tests without isolation
352
-
353
- ## 📈 Performance
354
-
355
- ### JSONL Features
356
-
357
- - **Point reading**: Only reads necessary lines
358
- - **In-memory indexes**: Fast search by indexed fields
359
- - **No complete parsing**: Doesn't load entire file into memory
360
- - **Large volume support**: Scales with millions of records
361
-
362
- ### Comparison: JexiDB vs 1.x
363
-
364
- | Feature | JexiDB | JexiDB 1.x |
365
- |---------|---------------|------------|
366
- | Safe truncation | ✅ | ❌ |
367
- | Consistent offsets | ✅ | ❌ |
368
- | Integrity validation | ✅ | ❌ |
369
- | Isolated tests | ✅ | ❌ |
370
- | No V8 dependency | ✅ | ❌ |
371
- | Similar API | ✅ | ✅ |
372
-
373
- ## 🔧 Utilities
374
-
375
- ```javascript
376
- const { utils } = require('jexidb');
377
-
378
- // Validate JSONL file
379
- const validation = await utils.validateJSONLFile('./data.jsonl');
380
-
381
- // Convert JSON to JSONL (basic)
382
- await utils.convertJSONToJSONL('./data.json', './data.jsonl');
383
-
384
- // Convert JSONL to JSON
385
- await utils.convertJSONLToJSON('./data.jsonl', './data.json');
386
-
387
- // Create JexiDB database with automatic indexes
388
- const result = await utils.createDatabaseFromJSON('./users.json', './users.jsonl', {
389
- autoDetectIndexes: true,
390
- autoIndexFields: ['id', 'email', 'name', 'username']
391
- });
392
-
393
- // Analyze JSON and suggest optimal indexes
394
- const analysis = await utils.analyzeJSONForIndexes('./users.json', 100);
395
- console.log('Recommended indexes:', analysis.suggestions.recommended);
396
-
397
- // Migrate from JexiDB 1.x to JexiDB
398
- await utils.migrateFromJexiDB('./jexidb-v1-database', './users.jsonl');
399
- ```
400
-
401
- ### 🔍 **How Utilities Work**
402
-
403
- #### **1. Basic Conversion (No Indexes)**
404
- ```javascript
405
- // Only converts format - DOES NOT add indexes
406
- await utils.convertJSONToJSONL('./data.json', './data.jsonl');
407
- ```
408
- - ✅ Converts JSON to JSONL
409
- - ❌ **DOES NOT create indexes**
410
- - ❌ **DOES NOT create JexiDB database**
411
- - ✅ Pure JSONL file
412
-
413
- #### **2. Database Creation with Automatic Indexes**
414
- ```javascript
415
- // Create complete JexiDB database with indexes
416
- const result = await utils.createDatabaseFromJSON('./users.json', './users.jsonl', {
417
- autoDetectIndexes: true,
418
- autoIndexFields: ['id', 'email', 'name']
419
- });
420
-
421
- console.log(result);
422
- // {
423
- // success: true,
424
- // recordCount: 1000,
425
- // indexes: ['id', 'email', 'name'],
426
- // dbPath: './users.jsonl'
427
- // }
428
- ```
429
- - ✅ Converts JSON to JSONL
430
- - ✅ **Creates indexes automatically**
431
- - ✅ **Creates complete JexiDB database**
432
- - ✅ File ready for use
433
-
434
- #### **3. Intelligent Index Analysis**
435
- ```javascript
436
- // Analyze data and suggest optimal indexes
437
- const analysis = await utils.analyzeJSONForIndexes('./users.json');
438
-
439
- console.log('Recommended:', analysis.suggestions.recommended);
440
- // [
441
- // { field: 'id', type: 'number', coverage: 100, uniqueness: 100 },
442
- // { field: 'email', type: 'string', coverage: 95, uniqueness: 98 }
443
- // ]
444
- ```
445
-
446
-
447
-
448
- ## 🔄 Migration from JexiDB 1.x
449
-
450
- ### Seamless Migration
451
-
452
- JexiDB is **fully backward compatible** with JexiDB 1.x! You can use the same API:
453
-
454
- ```javascript
455
- // JexiDB 1.x code works unchanged in JexiDB
456
- import { Database } from 'jexidb';
457
-
458
- const db = new Database('./database.jdb', {
459
- indexes: { id: 'number', name: 'string' }
460
- });
461
- await db.init();
462
-
463
- // All JexiDB 1.x methods work:
464
- await db.insert({ id: 1, name: 'John Doe' });
465
- const results = await db.query({ name: 'John Doe' }, { caseInsensitive: true });
466
- await db.update({ id: 1 }, { name: 'John Smith' });
467
- await db.delete({ id: 1 });
468
- await db.save();
469
- ```
470
-
471
- ### File Format Support
472
-
473
- JexiDB supports both file formats:
474
- - **`.jdb`** (preferred) - JexiDB's branded extension
475
- - **`.jsonl`** (standard) - JSON Lines format
476
-
477
- ```javascript
478
- // Both work identically:
479
- const db1 = new Database('./users.jdb', { indexes: { id: 'number' } });
480
- const db2 = new Database('./users.jsonl', { indexes: { id: 'number' } });
481
- ```
482
-
483
- ### Key Improvements
484
-
485
- | Feature | JexiDB 1.x | JexiDB |
486
- |---------|------------|--------------|
487
- | **API Compatibility** | Original | ✅ **100% Backward Compatible** |
488
- | **Query Methods** | `db.query()` | ✅ `db.query()` + `db.find()` |
489
- | **File Format** | `.jdb` (proprietary) | ✅ `.jdb` + `.jsonl` support |
490
- | **Performance** | Basic | ✅ **10-100x faster** |
491
- | **Memory Usage** | Higher | ✅ **25% less memory** |
492
- | **Data Integrity** | Basic | ✅ **Advanced validation** |
493
-
494
- ## 📝 Changelog
495
-
496
- See [CHANGELOG.md](CHANGELOG.md) for complete change history.
497
-
498
- ## 🤝 Contributing
499
-
500
- 1. Fork the project
501
- 2. Create a branch for your feature
502
- 3. Commit your changes
503
- 4. Push to the branch
504
- 5. Open a Pull Request
505
-
506
- ## 📄 License
507
-
508
- MIT License - see [LICENSE](LICENSE) for details.
509
-
510
- ---
511
-
512
- ## 🎯 About JexiDB
513
-
514
- JexiDB maintains the original JexiDB philosophy while fixing bugs and implementing a more robust architecture.
515
-
516
- ### 🚀 Performance
517
-
518
- **JexiDB** performance compared to version 1.x:
519
-
520
- - **Find operations**: 103x faster
521
- - **Update operations**: 26x faster
522
- - **Insert operations**: 6-11x faster
523
- - **Memory usage**: 25% less memory
524
-
525
- <p align="center">
526
- <img width="420" src="https://edenware.app/jexidb/images/jexi-mascot.webp" alt="JexiDB mascot" title="JexiDB mascot" />
527
- </p>
1
+ # JexiDB
2
+
3
+ <p align="center">
4
+ <img width="270" src="https://edenware.app/jexidb/images/jexidb-logo-icon.jpg" alt="JexiDB logo" title="JexiDB logo" />
5
+ </p>
6
+
7
+ ## Overview
8
+
9
+ JexiDB is a high-performance, in-memory database with persistence capabilities. It provides advanced indexing, querying, and term mapping features for efficient data operations. Ideal for local Node.js projects as well as apps built with Electron or NW.js. Written in pure JavaScript, it requires no compilation and is compatible with both CommonJS and ESM modules.
10
+
11
+ **Key Features:**
12
+ - In-memory storage with optional persistence
13
+ - Advanced indexing and querying capabilities
14
+ - Term mapping for efficient storage and querying
15
+ - Bulk update operations with `iterate()` method
16
+ - Manual save functionality (auto-save removed for better control)
17
+ - Transaction support with operation queuing
18
+ - Recovery mechanisms for data integrity
19
+
20
+ ## Installation
21
+
22
+ To install JexiDB, you can use npm:
23
+
24
+ ```bash
25
+ npm install EdenwareApps/jexidb
26
+ ```
27
+
28
+ ## Usage
29
+
30
+ ### Creating a Database Instance
31
+
32
+ To create a new instance of the database, you need to provide a file path where the database will be stored and an optional configuration object for indexes.
33
+
34
+ ```javascript
35
+ // const { Database } = require('jexidb'); // commonjs
36
+
37
+ import { Database } from 'jexidb'; // ESM
38
+
39
+ const db = new Database('path/to/database.jdb', {
40
+ create: true, // Create file if doesn't exist (default: true)
41
+ clear: false, // Clear existing files before loading (default: false)
42
+
43
+ // REQUIRED - Define your schema structure
44
+ fields: {
45
+ id: 'number',
46
+ name: 'string',
47
+ email: 'string',
48
+ tags: 'array:string'
49
+ },
50
+
51
+ // OPTIONAL - Performance optimization (only fields you query frequently)
52
+ indexes: {
53
+ name: 'string', // Search by name
54
+ tags: 'array:string' // Search by tags
55
+ }
56
+
57
+ // termMapping is now auto-enabled for array:string fields
58
+ });
59
+ ```
60
+ #### Constructor Options
61
+
62
+ - **create** (boolean, default: `true`): Controls whether the database file should be created if it doesn't exist.
63
+ - `true`: Creates the file if it doesn't exist (default behavior)
64
+ - `false`: Throws an error if the file doesn't exist
65
+
66
+ - **clear** (boolean, default: `false`): Controls whether to clear existing database files before loading.
67
+ - `true`: Deletes both the main database file (.jdb) and index file (.idx.jdb) if they exist, then starts with a clean database
68
+ - `false`: Loads existing data from files (default behavior)
69
+
70
+ You can [learn a bit more about these options at this link](https://github.com/EdenwareApps/jexidb/tree/main/test#readme).
71
+
72
+
73
+ ### Initializing the Database
74
+
75
+ Before using the database, you need to initialize it. This will load the existing data and indexes from the file.
76
+
77
+ ```javascript
78
+ await db.init();
79
+ ```
80
+ Only the values ​​specified as indexes are kept in memory for faster queries. JexiDB will never load the entire file into memory.
81
+
82
+
83
+ ### Inserting Data
84
+
85
+ You can insert data into the database by using the `insert` method. The data should be an object that contains the defined indexes. All object values will be saved into database.
86
+
87
+ ```javascript
88
+ await db.insert({ id: 1, name: 'John Doe' });
89
+ await db.insert({ id: 2, name: 'Jane Doe', anyArbitraryField: '1' });
90
+ ```
91
+
92
+ ### Querying Data
93
+
94
+ The `query` method allows you to retrieve data based on specific criteria. You can specify criteria for multiple fields.
95
+
96
+ ```javascript
97
+ const results = await db.query({ name: 'John Doe' }, { caseInsensitive: true });
98
+ console.log(results); // [{ id: 1, name: 'John Doe' }]
99
+ ```
100
+
101
+ Note: For now the query should be limited to using the fields specified as 'indexes' when instantiating the class.
102
+
103
+ #### Querying with Conditions
104
+
105
+ You can use conditions to perform more complex queries:
106
+
107
+ ```javascript
108
+ const results = await db.query({ id: { '>': 1 } });
109
+ console.log(results); // [{ id: 2, name: 'Jane Doe' }]
110
+ ```
111
+
112
+ ### Updating Data
113
+
114
+ To update existing records, use the `update` method with the criteria to find the records and the new data.
115
+
116
+ ```javascript
117
+ await db.update({ id: 1 }, { name: 'John Smith' });
118
+ ```
119
+
120
+ ### Deleting Data
121
+
122
+ You can delete records that match certain criteria using the `delete` method.
123
+
124
+ ```javascript
125
+ const deletedCount = await db.delete({ name: 'Jane Doe' });
126
+ console.log(`Deleted ${deletedCount} record(s).`);
127
+ ```
128
+
129
+ ### Iterating Through Records
130
+
131
+ You can iterate through records in the database using the `walk` method, which returns an async generator.
132
+
133
+ ```javascript
134
+ for await (const record of db.walk()) {
135
+ console.log(record);
136
+ }
137
+ ```
138
+
139
+ ### Saving Changes
140
+
141
+ After making any changes to the database, you need to save them using the `save` method. This will persist the changes to disk.
142
+
143
+ ```javascript
144
+ await db.save();
145
+ ```
146
+
147
+ ## Testing
148
+
149
+ JexiDB includes a comprehensive test suite built with Jest. To run the tests:
150
+
151
+ ```bash
152
+ # Run all tests
153
+ npm test
154
+
155
+ # Run tests in watch mode (for development)
156
+ npm test:watch
157
+
158
+ # Run tests with coverage report
159
+ npm test:coverage
160
+
161
+ # Run legacy test suite (original Mortal Kombat themed tests)
162
+ npm run test:legacy
163
+ ```
164
+
165
+ The test suite includes:
166
+ - **Database Tests**: Complete CRUD operations, querying, indexing, and persistence
167
+ - **IndexManager Tests**: Index creation, querying with various operators, and data management
168
+ - **Serializer Tests**: JSON serialization/deserialization with various data types
169
+
170
+ All database operations ensure that the `_` property (position index) is always included in returned results.
171
+
172
+ ## Bulk Operations with `iterate()`
173
+
174
+ The `iterate()` method provides high-performance bulk update capabilities with streaming support:
175
+
176
+ ```javascript
177
+ // Basic bulk update
178
+ for await (const entry of db.iterate({ category: 'fruits' })) {
179
+ entry.price = entry.price * 1.1 // 10% price increase
180
+ entry.lastUpdated = new Date().toISOString()
181
+ }
182
+
183
+ // Advanced options with progress tracking
184
+ for await (const entry of db.iterate(
185
+ { status: 'active' },
186
+ {
187
+ chunkSize: 1000,
188
+ progressCallback: (progress) => {
189
+ console.log(`Processed: ${progress.processed}, Modified: ${progress.modified}`)
190
+ }
191
+ }
192
+ )) {
193
+ entry.processed = true
194
+ }
195
+ ```
196
+
197
+ **Key Benefits:**
198
+ - **Streaming Performance**: Process large datasets without loading everything into memory
199
+ - **Bulk Updates**: Modify multiple records in a single operation
200
+ - **Automatic Change Detection**: Automatically detects which records were modified
201
+ - **Progress Tracking**: Optional progress callbacks for long-running operations
202
+
203
+ ## 🔄 Migration Guide (1.x.x → 2.1.0)
204
+
205
+ ### ⚠️ Important: Database Files Are NOT Compatible
206
+
207
+ **Existing `.jdb` files from version 1.x.x will NOT work with version 2.1.0.**
208
+
209
+ ### Step 1: Export Data from 1.x.x
210
+
211
+ ```javascript
212
+ // In your 1.x.x application
213
+ const oldDb = new Database('old-database.jdb', {
214
+ indexes: { name: 'string', tags: 'array:string' }
215
+ })
216
+
217
+ await oldDb.init()
218
+ const allData = await oldDb.find({}) // Export all data
219
+ await oldDb.destroy()
220
+ ```
221
+
222
+ ### Step 2: Update Your Code
223
+
224
+ ```javascript
225
+ // OLD (1.x.x)
226
+ const db = new Database('db.jdb', {
227
+ indexes: { name: 'string', tags: 'array:string' }
228
+ })
229
+
230
+ // NEW (2.1.0)
231
+ const db = new Database('db.jdb', {
232
+ fields: { // REQUIRED - Define schema
233
+ id: 'number',
234
+ name: 'string',
235
+ tags: 'array:string'
236
+ },
237
+ indexes: { // OPTIONAL - Performance optimization
238
+ name: 'string', // ✅ Search by name
239
+ tags: 'array:string' // Search by tags
240
+ }
241
+ })
242
+ ```
243
+
244
+ ### Step 3: Import Data to 2.1.0
245
+
246
+ ```javascript
247
+ // In your 2.1.0 application
248
+ const newDb = new Database('new-database.jdb', {
249
+ fields: { /* your schema */ },
250
+ indexes: { /* your indexes */ }
251
+ })
252
+
253
+ await newDb.init()
254
+
255
+ // Import all data
256
+ for (const record of allData) {
257
+ await newDb.insert(record)
258
+ }
259
+
260
+ await newDb.save()
261
+ ```
262
+
263
+ ### Key Changes Summary
264
+
265
+ | Feature | 1.x.x | 2.1.0 |
266
+ |---------|-------|-------|
267
+ | `fields` | Optional | **MANDATORY** |
268
+ | `termMapping` | `false` (default) | `true` (default) |
269
+ | `termMappingFields` | Manual config | Auto-detected |
270
+ | Database files | Compatible | **NOT compatible** |
271
+ | Performance | Basic | **77% size reduction** |
272
+
273
+ ## 📚 Documentation
274
+
275
+ For comprehensive documentation and examples:
276
+
277
+ - **[📖 Full Documentation](docs/README.md)** - Complete documentation index
278
+ - **[🔧 API Reference](docs/API.md)** - Detailed API documentation
279
+ - **[💡 Examples](docs/EXAMPLES.md)** - Practical examples and use cases
280
+ - **[🚀 Getting Started](docs/README.md#quick-start)** - Quick start guide
281
+
282
+ ### Key Features Documentation
283
+
284
+ - **[Bulk Operations](docs/API.md#bulk-operations)** - High-performance `iterate()` method
285
+ - **[Term Mapping](docs/API.md#term-mapping)** - Optimize storage for repetitive data
286
+ - **[Query Operators](docs/API.md#query-operators)** - Advanced querying capabilities
287
+ - **[Performance Tips](docs/API.md#best-practices)** - Optimization strategies
288
+
289
+ ## Conclusion
290
+
291
+ JexiDB provides a simple yet powerful way to store and manage data in JavaScript applications. With its indexing and querying features, you can build efficient data-driven applications.
292
+
293
+ <p align="center">
294
+ <img width="380" src="https://edenware.app/jexidb/images/jexidb-mascot3.jpg" alt="JexiDB mascot" title="JexiDB mascot" />
295
+ </p>
296
+
297
+ # Contributing
298
+
299
+ Please, feel free to contribute to the project by opening a discussion under Issues section or sending your PR.
300
+
301
+ If you find this library useful, please consider making a donation of any amount via [PayPal by clicking here](https://www.paypal.com/donate/?item_name=megacubo.tv&cmd=_donations&business=efox.web%40gmail.com) to help the developer continue to dedicate himself to the project. ❤