jexidb 2.0.2 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.babelrc +13 -0
- package/.gitattributes +2 -0
- package/CHANGELOG.md +140 -0
- package/LICENSE +21 -21
- package/README.md +301 -527
- package/babel.config.json +5 -0
- package/dist/Database.cjs +3896 -0
- package/docs/API.md +1051 -0
- package/docs/EXAMPLES.md +701 -0
- package/docs/README.md +194 -0
- package/examples/iterate-usage-example.js +157 -0
- package/examples/simple-iterate-example.js +115 -0
- package/jest.config.js +24 -0
- package/package.json +63 -51
- package/scripts/README.md +47 -0
- package/scripts/clean-test-files.js +75 -0
- package/scripts/prepare.js +31 -0
- package/scripts/run-tests.js +80 -0
- package/src/Database.mjs +4130 -0
- package/src/FileHandler.mjs +1101 -0
- package/src/OperationQueue.mjs +279 -0
- package/src/SchemaManager.mjs +268 -0
- package/src/Serializer.mjs +511 -0
- package/src/managers/ConcurrencyManager.mjs +257 -0
- package/src/managers/IndexManager.mjs +1403 -0
- package/src/managers/QueryManager.mjs +1273 -0
- package/src/managers/StatisticsManager.mjs +262 -0
- package/src/managers/StreamingProcessor.mjs +429 -0
- package/src/managers/TermManager.mjs +278 -0
- package/test/$not-operator-with-and.test.js +282 -0
- package/test/README.md +8 -0
- package/test/close-init-cycle.test.js +256 -0
- package/test/critical-bugs-fixes.test.js +1069 -0
- package/test/index-persistence.test.js +306 -0
- package/test/index-serialization.test.js +314 -0
- package/test/indexed-query-mode.test.js +360 -0
- package/test/iterate-method.test.js +272 -0
- package/test/query-operators.test.js +238 -0
- package/test/regex-array-fields.test.js +129 -0
- package/test/score-method.test.js +238 -0
- package/test/setup.js +17 -0
- package/test/term-mapping-minimal.test.js +154 -0
- package/test/term-mapping-simple.test.js +257 -0
- package/test/term-mapping.test.js +514 -0
- package/test/writebuffer-flush-resilience.test.js +204 -0
- package/dist/FileHandler.js +0 -688
- package/dist/IndexManager.js +0 -353
- package/dist/IntegrityChecker.js +0 -364
- package/dist/JSONLDatabase.js +0 -1194
- package/dist/index.js +0 -617
- package/src/FileHandler.js +0 -674
- package/src/IndexManager.js +0 -363
- package/src/IntegrityChecker.js +0 -379
- package/src/JSONLDatabase.js +0 -1248
- package/src/index.js +0 -608
package/README.md
CHANGED
|
@@ -1,527 +1,301 @@
|
|
|
1
|
-
# JexiDB
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
db
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
//
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
//
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
}
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
{
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
```javascript
|
|
134
|
-
{
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
```
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
###
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
{ 'profile.country': 'USA' }
|
|
303
|
-
);
|
|
304
|
-
```
|
|
305
|
-
|
|
306
|
-
### Integrity Validation
|
|
307
|
-
|
|
308
|
-
```javascript
|
|
309
|
-
// Validate integrity with details
|
|
310
|
-
const integrity = await db.validateIntegrity({
|
|
311
|
-
checkData: true,
|
|
312
|
-
checkIndexes: true,
|
|
313
|
-
checkOffsets: true,
|
|
314
|
-
verbose: true
|
|
315
|
-
});
|
|
316
|
-
|
|
317
|
-
if (!integrity.isValid) {
|
|
318
|
-
console.log('Errors:', integrity.errors);
|
|
319
|
-
console.log('Warnings:', integrity.warnings);
|
|
320
|
-
}
|
|
321
|
-
```
|
|
322
|
-
|
|
323
|
-
### Detailed Statistics
|
|
324
|
-
|
|
325
|
-
```javascript
|
|
326
|
-
const stats = await db.getStats();
|
|
327
|
-
console.log('File size:', stats.file.size);
|
|
328
|
-
console.log('Total records:', stats.summary.totalRecords);
|
|
329
|
-
console.log('Indexes:', stats.indexes.indexCount);
|
|
330
|
-
```
|
|
331
|
-
|
|
332
|
-
## 🧪 Tests
|
|
333
|
-
|
|
334
|
-
```bash
|
|
335
|
-
npm test
|
|
336
|
-
```
|
|
337
|
-
|
|
338
|
-
**Automatic Cleanup**: The test script automatically removes all test files after execution to keep the project directory clean.
|
|
339
|
-
|
|
340
|
-
**Manual Cleanup**: If you need to clean up test files manually:
|
|
341
|
-
```bash
|
|
342
|
-
npm run test:clean
|
|
343
|
-
```
|
|
344
|
-
|
|
345
|
-
**Available Test Scripts**:
|
|
346
|
-
- `npm test` - Run all tests with automatic cleanup
|
|
347
|
-
- `npm run test:watch` - Run tests in watch mode
|
|
348
|
-
- `npm run test:clean` - Clean up test files manually
|
|
349
|
-
- `npm run test:optimized` - Run optimized performance tests
|
|
350
|
-
- `npm run test:parallel` - Run tests in parallel
|
|
351
|
-
- `npm run test:fast` - Run fast tests without isolation
|
|
352
|
-
|
|
353
|
-
## 📈 Performance
|
|
354
|
-
|
|
355
|
-
### JSONL Features
|
|
356
|
-
|
|
357
|
-
- **Point reading**: Only reads necessary lines
|
|
358
|
-
- **In-memory indexes**: Fast search by indexed fields
|
|
359
|
-
- **No complete parsing**: Doesn't load entire file into memory
|
|
360
|
-
- **Large volume support**: Scales with millions of records
|
|
361
|
-
|
|
362
|
-
### Comparison: JexiDB vs 1.x
|
|
363
|
-
|
|
364
|
-
| Feature | JexiDB | JexiDB 1.x |
|
|
365
|
-
|---------|---------------|------------|
|
|
366
|
-
| Safe truncation | ✅ | ❌ |
|
|
367
|
-
| Consistent offsets | ✅ | ❌ |
|
|
368
|
-
| Integrity validation | ✅ | ❌ |
|
|
369
|
-
| Isolated tests | ✅ | ❌ |
|
|
370
|
-
| No V8 dependency | ✅ | ❌ |
|
|
371
|
-
| Similar API | ✅ | ✅ |
|
|
372
|
-
|
|
373
|
-
## 🔧 Utilities
|
|
374
|
-
|
|
375
|
-
```javascript
|
|
376
|
-
const { utils } = require('jexidb');
|
|
377
|
-
|
|
378
|
-
// Validate JSONL file
|
|
379
|
-
const validation = await utils.validateJSONLFile('./data.jsonl');
|
|
380
|
-
|
|
381
|
-
// Convert JSON to JSONL (basic)
|
|
382
|
-
await utils.convertJSONToJSONL('./data.json', './data.jsonl');
|
|
383
|
-
|
|
384
|
-
// Convert JSONL to JSON
|
|
385
|
-
await utils.convertJSONLToJSON('./data.jsonl', './data.json');
|
|
386
|
-
|
|
387
|
-
// Create JexiDB database with automatic indexes
|
|
388
|
-
const result = await utils.createDatabaseFromJSON('./users.json', './users.jsonl', {
|
|
389
|
-
autoDetectIndexes: true,
|
|
390
|
-
autoIndexFields: ['id', 'email', 'name', 'username']
|
|
391
|
-
});
|
|
392
|
-
|
|
393
|
-
// Analyze JSON and suggest optimal indexes
|
|
394
|
-
const analysis = await utils.analyzeJSONForIndexes('./users.json', 100);
|
|
395
|
-
console.log('Recommended indexes:', analysis.suggestions.recommended);
|
|
396
|
-
|
|
397
|
-
// Migrate from JexiDB 1.x to JexiDB
|
|
398
|
-
await utils.migrateFromJexiDB('./jexidb-v1-database', './users.jsonl');
|
|
399
|
-
```
|
|
400
|
-
|
|
401
|
-
### 🔍 **How Utilities Work**
|
|
402
|
-
|
|
403
|
-
#### **1. Basic Conversion (No Indexes)**
|
|
404
|
-
```javascript
|
|
405
|
-
// Only converts format - DOES NOT add indexes
|
|
406
|
-
await utils.convertJSONToJSONL('./data.json', './data.jsonl');
|
|
407
|
-
```
|
|
408
|
-
- ✅ Converts JSON to JSONL
|
|
409
|
-
- ❌ **DOES NOT create indexes**
|
|
410
|
-
- ❌ **DOES NOT create JexiDB database**
|
|
411
|
-
- ✅ Pure JSONL file
|
|
412
|
-
|
|
413
|
-
#### **2. Database Creation with Automatic Indexes**
|
|
414
|
-
```javascript
|
|
415
|
-
// Create complete JexiDB database with indexes
|
|
416
|
-
const result = await utils.createDatabaseFromJSON('./users.json', './users.jsonl', {
|
|
417
|
-
autoDetectIndexes: true,
|
|
418
|
-
autoIndexFields: ['id', 'email', 'name']
|
|
419
|
-
});
|
|
420
|
-
|
|
421
|
-
console.log(result);
|
|
422
|
-
// {
|
|
423
|
-
// success: true,
|
|
424
|
-
// recordCount: 1000,
|
|
425
|
-
// indexes: ['id', 'email', 'name'],
|
|
426
|
-
// dbPath: './users.jsonl'
|
|
427
|
-
// }
|
|
428
|
-
```
|
|
429
|
-
- ✅ Converts JSON to JSONL
|
|
430
|
-
- ✅ **Creates indexes automatically**
|
|
431
|
-
- ✅ **Creates complete JexiDB database**
|
|
432
|
-
- ✅ File ready for use
|
|
433
|
-
|
|
434
|
-
#### **3. Intelligent Index Analysis**
|
|
435
|
-
```javascript
|
|
436
|
-
// Analyze data and suggest optimal indexes
|
|
437
|
-
const analysis = await utils.analyzeJSONForIndexes('./users.json');
|
|
438
|
-
|
|
439
|
-
console.log('Recommended:', analysis.suggestions.recommended);
|
|
440
|
-
// [
|
|
441
|
-
// { field: 'id', type: 'number', coverage: 100, uniqueness: 100 },
|
|
442
|
-
// { field: 'email', type: 'string', coverage: 95, uniqueness: 98 }
|
|
443
|
-
// ]
|
|
444
|
-
```
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
## 🔄 Migration from JexiDB 1.x
|
|
449
|
-
|
|
450
|
-
### Seamless Migration
|
|
451
|
-
|
|
452
|
-
JexiDB is **fully backward compatible** with JexiDB 1.x! You can use the same API:
|
|
453
|
-
|
|
454
|
-
```javascript
|
|
455
|
-
// JexiDB 1.x code works unchanged in JexiDB
|
|
456
|
-
import { Database } from 'jexidb';
|
|
457
|
-
|
|
458
|
-
const db = new Database('./database.jdb', {
|
|
459
|
-
indexes: { id: 'number', name: 'string' }
|
|
460
|
-
});
|
|
461
|
-
await db.init();
|
|
462
|
-
|
|
463
|
-
// All JexiDB 1.x methods work:
|
|
464
|
-
await db.insert({ id: 1, name: 'John Doe' });
|
|
465
|
-
const results = await db.query({ name: 'John Doe' }, { caseInsensitive: true });
|
|
466
|
-
await db.update({ id: 1 }, { name: 'John Smith' });
|
|
467
|
-
await db.delete({ id: 1 });
|
|
468
|
-
await db.save();
|
|
469
|
-
```
|
|
470
|
-
|
|
471
|
-
### File Format Support
|
|
472
|
-
|
|
473
|
-
JexiDB supports both file formats:
|
|
474
|
-
- **`.jdb`** (preferred) - JexiDB's branded extension
|
|
475
|
-
- **`.jsonl`** (standard) - JSON Lines format
|
|
476
|
-
|
|
477
|
-
```javascript
|
|
478
|
-
// Both work identically:
|
|
479
|
-
const db1 = new Database('./users.jdb', { indexes: { id: 'number' } });
|
|
480
|
-
const db2 = new Database('./users.jsonl', { indexes: { id: 'number' } });
|
|
481
|
-
```
|
|
482
|
-
|
|
483
|
-
### Key Improvements
|
|
484
|
-
|
|
485
|
-
| Feature | JexiDB 1.x | JexiDB |
|
|
486
|
-
|---------|------------|--------------|
|
|
487
|
-
| **API Compatibility** | Original | ✅ **100% Backward Compatible** |
|
|
488
|
-
| **Query Methods** | `db.query()` | ✅ `db.query()` + `db.find()` |
|
|
489
|
-
| **File Format** | `.jdb` (proprietary) | ✅ `.jdb` + `.jsonl` support |
|
|
490
|
-
| **Performance** | Basic | ✅ **10-100x faster** |
|
|
491
|
-
| **Memory Usage** | Higher | ✅ **25% less memory** |
|
|
492
|
-
| **Data Integrity** | Basic | ✅ **Advanced validation** |
|
|
493
|
-
|
|
494
|
-
## 📝 Changelog
|
|
495
|
-
|
|
496
|
-
See [CHANGELOG.md](CHANGELOG.md) for complete change history.
|
|
497
|
-
|
|
498
|
-
## 🤝 Contributing
|
|
499
|
-
|
|
500
|
-
1. Fork the project
|
|
501
|
-
2. Create a branch for your feature
|
|
502
|
-
3. Commit your changes
|
|
503
|
-
4. Push to the branch
|
|
504
|
-
5. Open a Pull Request
|
|
505
|
-
|
|
506
|
-
## 📄 License
|
|
507
|
-
|
|
508
|
-
MIT License - see [LICENSE](LICENSE) for details.
|
|
509
|
-
|
|
510
|
-
---
|
|
511
|
-
|
|
512
|
-
## 🎯 About JexiDB
|
|
513
|
-
|
|
514
|
-
JexiDB maintains the original JexiDB philosophy while fixing bugs and implementing a more robust architecture.
|
|
515
|
-
|
|
516
|
-
### 🚀 Performance
|
|
517
|
-
|
|
518
|
-
**JexiDB** performance compared to version 1.x:
|
|
519
|
-
|
|
520
|
-
- **Find operations**: 103x faster
|
|
521
|
-
- **Update operations**: 26x faster
|
|
522
|
-
- **Insert operations**: 6-11x faster
|
|
523
|
-
- **Memory usage**: 25% less memory
|
|
524
|
-
|
|
525
|
-
<p align="center">
|
|
526
|
-
<img width="420" src="https://edenware.app/jexidb/images/jexi-mascot.webp" alt="JexiDB mascot" title="JexiDB mascot" />
|
|
527
|
-
</p>
|
|
1
|
+
# JexiDB
|
|
2
|
+
|
|
3
|
+
<p align="center">
|
|
4
|
+
<img width="270" src="https://edenware.app/jexidb/images/jexidb-logo-icon.jpg" alt="JexiDB logo" title="JexiDB logo" />
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
## Overview
|
|
8
|
+
|
|
9
|
+
JexiDB is a high-performance, in-memory database with persistence capabilities. It provides advanced indexing, querying, and term mapping features for efficient data operations. Ideal for local Node.js projects as well as apps built with Electron or NW.js. Written in pure JavaScript, it requires no compilation and is compatible with both CommonJS and ESM modules.
|
|
10
|
+
|
|
11
|
+
**Key Features:**
|
|
12
|
+
- In-memory storage with optional persistence
|
|
13
|
+
- Advanced indexing and querying capabilities
|
|
14
|
+
- Term mapping for efficient storage and querying
|
|
15
|
+
- Bulk update operations with `iterate()` method
|
|
16
|
+
- Manual save functionality (auto-save removed for better control)
|
|
17
|
+
- Transaction support with operation queuing
|
|
18
|
+
- Recovery mechanisms for data integrity
|
|
19
|
+
|
|
20
|
+
## Installation
|
|
21
|
+
|
|
22
|
+
To install JexiDB, you can use npm:
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
npm install EdenwareApps/jexidb
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Usage
|
|
29
|
+
|
|
30
|
+
### Creating a Database Instance
|
|
31
|
+
|
|
32
|
+
To create a new instance of the database, you need to provide a file path where the database will be stored and an optional configuration object for indexes.
|
|
33
|
+
|
|
34
|
+
```javascript
|
|
35
|
+
// const { Database } = require('jexidb'); // commonjs
|
|
36
|
+
|
|
37
|
+
import { Database } from 'jexidb'; // ESM
|
|
38
|
+
|
|
39
|
+
const db = new Database('path/to/database.jdb', {
|
|
40
|
+
create: true, // Create file if doesn't exist (default: true)
|
|
41
|
+
clear: false, // Clear existing files before loading (default: false)
|
|
42
|
+
|
|
43
|
+
// REQUIRED - Define your schema structure
|
|
44
|
+
fields: {
|
|
45
|
+
id: 'number',
|
|
46
|
+
name: 'string',
|
|
47
|
+
email: 'string',
|
|
48
|
+
tags: 'array:string'
|
|
49
|
+
},
|
|
50
|
+
|
|
51
|
+
// OPTIONAL - Performance optimization (only fields you query frequently)
|
|
52
|
+
indexes: {
|
|
53
|
+
name: 'string', // ✅ Search by name
|
|
54
|
+
tags: 'array:string' // ✅ Search by tags
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// termMapping is now auto-enabled for array:string fields
|
|
58
|
+
});
|
|
59
|
+
```
|
|
60
|
+
#### Constructor Options
|
|
61
|
+
|
|
62
|
+
- **create** (boolean, default: `true`): Controls whether the database file should be created if it doesn't exist.
|
|
63
|
+
- `true`: Creates the file if it doesn't exist (default behavior)
|
|
64
|
+
- `false`: Throws an error if the file doesn't exist
|
|
65
|
+
|
|
66
|
+
- **clear** (boolean, default: `false`): Controls whether to clear existing database files before loading.
|
|
67
|
+
- `true`: Deletes both the main database file (.jdb) and index file (.idx.jdb) if they exist, then starts with a clean database
|
|
68
|
+
- `false`: Loads existing data from files (default behavior)
|
|
69
|
+
|
|
70
|
+
You can [learn a bit more about these options at this link](https://github.com/EdenwareApps/jexidb/tree/main/test#readme).
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
### Initializing the Database
|
|
74
|
+
|
|
75
|
+
Before using the database, you need to initialize it. This will load the existing data and indexes from the file.
|
|
76
|
+
|
|
77
|
+
```javascript
|
|
78
|
+
await db.init();
|
|
79
|
+
```
|
|
80
|
+
Only the values specified as indexes are kept in memory for faster queries. JexiDB will never load the entire file into memory.
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
### Inserting Data
|
|
84
|
+
|
|
85
|
+
You can insert data into the database by using the `insert` method. The data should be an object that contains the defined indexes. All object values will be saved into database.
|
|
86
|
+
|
|
87
|
+
```javascript
|
|
88
|
+
await db.insert({ id: 1, name: 'John Doe' });
|
|
89
|
+
await db.insert({ id: 2, name: 'Jane Doe', anyArbitraryField: '1' });
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Querying Data
|
|
93
|
+
|
|
94
|
+
The `query` method allows you to retrieve data based on specific criteria. You can specify criteria for multiple fields.
|
|
95
|
+
|
|
96
|
+
```javascript
|
|
97
|
+
const results = await db.query({ name: 'John Doe' }, { caseInsensitive: true });
|
|
98
|
+
console.log(results); // [{ id: 1, name: 'John Doe' }]
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Note: For now the query should be limited to using the fields specified as 'indexes' when instantiating the class.
|
|
102
|
+
|
|
103
|
+
#### Querying with Conditions
|
|
104
|
+
|
|
105
|
+
You can use conditions to perform more complex queries:
|
|
106
|
+
|
|
107
|
+
```javascript
|
|
108
|
+
const results = await db.query({ id: { '>': 1 } });
|
|
109
|
+
console.log(results); // [{ id: 2, name: 'Jane Doe' }]
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### Updating Data
|
|
113
|
+
|
|
114
|
+
To update existing records, use the `update` method with the criteria to find the records and the new data.
|
|
115
|
+
|
|
116
|
+
```javascript
|
|
117
|
+
await db.update({ id: 1 }, { name: 'John Smith' });
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### Deleting Data
|
|
121
|
+
|
|
122
|
+
You can delete records that match certain criteria using the `delete` method.
|
|
123
|
+
|
|
124
|
+
```javascript
|
|
125
|
+
const deletedCount = await db.delete({ name: 'Jane Doe' });
|
|
126
|
+
console.log(`Deleted ${deletedCount} record(s).`);
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Iterating Through Records
|
|
130
|
+
|
|
131
|
+
You can iterate through records in the database using the `walk` method, which returns an async generator.
|
|
132
|
+
|
|
133
|
+
```javascript
|
|
134
|
+
for await (const record of db.walk()) {
|
|
135
|
+
console.log(record);
|
|
136
|
+
}
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### Saving Changes
|
|
140
|
+
|
|
141
|
+
After making any changes to the database, you need to save them using the `save` method. This will persist the changes to disk.
|
|
142
|
+
|
|
143
|
+
```javascript
|
|
144
|
+
await db.save();
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
## Testing
|
|
148
|
+
|
|
149
|
+
JexiDB includes a comprehensive test suite built with Jest. To run the tests:
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
# Run all tests
|
|
153
|
+
npm test
|
|
154
|
+
|
|
155
|
+
# Run tests in watch mode (for development)
|
|
156
|
+
npm test:watch
|
|
157
|
+
|
|
158
|
+
# Run tests with coverage report
|
|
159
|
+
npm test:coverage
|
|
160
|
+
|
|
161
|
+
# Run legacy test suite (original Mortal Kombat themed tests)
|
|
162
|
+
npm run test:legacy
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
The test suite includes:
|
|
166
|
+
- **Database Tests**: Complete CRUD operations, querying, indexing, and persistence
|
|
167
|
+
- **IndexManager Tests**: Index creation, querying with various operators, and data management
|
|
168
|
+
- **Serializer Tests**: JSON serialization/deserialization with various data types
|
|
169
|
+
|
|
170
|
+
All database operations ensure that the `_` property (position index) is always included in returned results.
|
|
171
|
+
|
|
172
|
+
## Bulk Operations with `iterate()`
|
|
173
|
+
|
|
174
|
+
The `iterate()` method provides high-performance bulk update capabilities with streaming support:
|
|
175
|
+
|
|
176
|
+
```javascript
|
|
177
|
+
// Basic bulk update
|
|
178
|
+
for await (const entry of db.iterate({ category: 'fruits' })) {
|
|
179
|
+
entry.price = entry.price * 1.1 // 10% price increase
|
|
180
|
+
entry.lastUpdated = new Date().toISOString()
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
// Advanced options with progress tracking
|
|
184
|
+
for await (const entry of db.iterate(
|
|
185
|
+
{ status: 'active' },
|
|
186
|
+
{
|
|
187
|
+
chunkSize: 1000,
|
|
188
|
+
progressCallback: (progress) => {
|
|
189
|
+
console.log(`Processed: ${progress.processed}, Modified: ${progress.modified}`)
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
)) {
|
|
193
|
+
entry.processed = true
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
**Key Benefits:**
|
|
198
|
+
- **Streaming Performance**: Process large datasets without loading everything into memory
|
|
199
|
+
- **Bulk Updates**: Modify multiple records in a single operation
|
|
200
|
+
- **Automatic Change Detection**: Automatically detects which records were modified
|
|
201
|
+
- **Progress Tracking**: Optional progress callbacks for long-running operations
|
|
202
|
+
|
|
203
|
+
## 🔄 Migration Guide (1.x.x → 2.1.0)
|
|
204
|
+
|
|
205
|
+
### ⚠️ Important: Database Files Are NOT Compatible
|
|
206
|
+
|
|
207
|
+
**Existing `.jdb` files from version 1.x.x will NOT work with version 2.1.0.**
|
|
208
|
+
|
|
209
|
+
### Step 1: Export Data from 1.x.x
|
|
210
|
+
|
|
211
|
+
```javascript
|
|
212
|
+
// In your 1.x.x application
|
|
213
|
+
const oldDb = new Database('old-database.jdb', {
|
|
214
|
+
indexes: { name: 'string', tags: 'array:string' }
|
|
215
|
+
})
|
|
216
|
+
|
|
217
|
+
await oldDb.init()
|
|
218
|
+
const allData = await oldDb.find({}) // Export all data
|
|
219
|
+
await oldDb.destroy()
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
### Step 2: Update Your Code
|
|
223
|
+
|
|
224
|
+
```javascript
|
|
225
|
+
// ❌ OLD (1.x.x)
|
|
226
|
+
const db = new Database('db.jdb', {
|
|
227
|
+
indexes: { name: 'string', tags: 'array:string' }
|
|
228
|
+
})
|
|
229
|
+
|
|
230
|
+
// ✅ NEW (2.1.0)
|
|
231
|
+
const db = new Database('db.jdb', {
|
|
232
|
+
fields: { // REQUIRED - Define schema
|
|
233
|
+
id: 'number',
|
|
234
|
+
name: 'string',
|
|
235
|
+
tags: 'array:string'
|
|
236
|
+
},
|
|
237
|
+
indexes: { // OPTIONAL - Performance optimization
|
|
238
|
+
name: 'string', // ✅ Search by name
|
|
239
|
+
tags: 'array:string' // ✅ Search by tags
|
|
240
|
+
}
|
|
241
|
+
})
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
### Step 3: Import Data to 2.1.0
|
|
245
|
+
|
|
246
|
+
```javascript
|
|
247
|
+
// In your 2.1.0 application
|
|
248
|
+
const newDb = new Database('new-database.jdb', {
|
|
249
|
+
fields: { /* your schema */ },
|
|
250
|
+
indexes: { /* your indexes */ }
|
|
251
|
+
})
|
|
252
|
+
|
|
253
|
+
await newDb.init()
|
|
254
|
+
|
|
255
|
+
// Import all data
|
|
256
|
+
for (const record of allData) {
|
|
257
|
+
await newDb.insert(record)
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
await newDb.save()
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### Key Changes Summary
|
|
264
|
+
|
|
265
|
+
| Feature | 1.x.x | 2.1.0 |
|
|
266
|
+
|---------|-------|-------|
|
|
267
|
+
| `fields` | Optional | **MANDATORY** |
|
|
268
|
+
| `termMapping` | `false` (default) | `true` (default) |
|
|
269
|
+
| `termMappingFields` | Manual config | Auto-detected |
|
|
270
|
+
| Database files | Compatible | **NOT compatible** |
|
|
271
|
+
| Performance | Basic | **77% size reduction** |
|
|
272
|
+
|
|
273
|
+
## 📚 Documentation
|
|
274
|
+
|
|
275
|
+
For comprehensive documentation and examples:
|
|
276
|
+
|
|
277
|
+
- **[📖 Full Documentation](docs/README.md)** - Complete documentation index
|
|
278
|
+
- **[🔧 API Reference](docs/API.md)** - Detailed API documentation
|
|
279
|
+
- **[💡 Examples](docs/EXAMPLES.md)** - Practical examples and use cases
|
|
280
|
+
- **[🚀 Getting Started](docs/README.md#quick-start)** - Quick start guide
|
|
281
|
+
|
|
282
|
+
### Key Features Documentation
|
|
283
|
+
|
|
284
|
+
- **[Bulk Operations](docs/API.md#bulk-operations)** - High-performance `iterate()` method
|
|
285
|
+
- **[Term Mapping](docs/API.md#term-mapping)** - Optimize storage for repetitive data
|
|
286
|
+
- **[Query Operators](docs/API.md#query-operators)** - Advanced querying capabilities
|
|
287
|
+
- **[Performance Tips](docs/API.md#best-practices)** - Optimization strategies
|
|
288
|
+
|
|
289
|
+
## Conclusion
|
|
290
|
+
|
|
291
|
+
JexiDB provides a simple yet powerful way to store and manage data in JavaScript applications. With its indexing and querying features, you can build efficient data-driven applications.
|
|
292
|
+
|
|
293
|
+
<p align="center">
|
|
294
|
+
<img width="380" src="https://edenware.app/jexidb/images/jexidb-mascot3.jpg" alt="JexiDB mascot" title="JexiDB mascot" />
|
|
295
|
+
</p>
|
|
296
|
+
|
|
297
|
+
# Contributing
|
|
298
|
+
|
|
299
|
+
Please, feel free to contribute to the project by opening a discussion under Issues section or sending your PR.
|
|
300
|
+
|
|
301
|
+
If you find this library useful, please consider making a donation of any amount via [PayPal by clicking here](https://www.paypal.com/donate/?item_name=megacubo.tv&cmd=_donations&business=efox.web%40gmail.com) to help the developer continue to dedicate himself to the project. ❤
|