@harperfast/rocksdb-js 2.13.0 → 2.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +246 -42
- package/dist/index.cjs +307 -124
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +109 -20
- package/dist/index.d.mts +109 -20
- package/dist/index.mjs +306 -125
- package/dist/index.mjs.map +1 -1
- package/docs/performance/transaction-state-2026-11-09.csv +28 -0
- package/docs/performance/transaction-state.md +67 -0
- package/docs/transaction-log.md +34 -3
- package/package.json +16 -16
package/README.md
CHANGED
|
@@ -59,6 +59,19 @@ Creates a new database instance.
|
|
|
59
59
|
used as given, except that RocksDB raises one below 4 KiB to 4 KiB. Like `writeBufferSize`,
|
|
60
60
|
this applies to every column family when a database is first opened in the process and to
|
|
61
61
|
column families created later; a column family that is already open keeps its size.
|
|
62
|
+
- `bloomFilterBitsPerKey: number` The size of the filter written into each SST file, in bits per
|
|
63
|
+
key. A `get()` of a key that is not in a file checks that file's filter and skips its index and
|
|
64
|
+
data blocks, so reads of absent keys, and of keys whose range overlaps several files (keys
|
|
65
|
+
written in random order, or updates still in level 0), touch fewer blocks. Filters cover whole
|
|
66
|
+
keys only, so range reads are unaffected. Defaults to `10` (about 1% false positives); otherwise
|
|
67
|
+
a number from 1 to 100, fractions allowed, or `0` for no filters. Each open SST file holds its
|
|
68
|
+
filter in memory beside its index, outside the block cache (about 1.25 bytes per key at `10`;
|
|
69
|
+
a `secondaryPath` open holds every file open, so the whole database's filters). `0` also stops
|
|
70
|
+
an open from loading the filters files already have, so a reader can trade them for memory. An existing
|
|
71
|
+
database opens with the new setting, but its files gain or lose filters only as compaction
|
|
72
|
+
rewrites them (`compact({ bottommost: true })` rewrites them all). Like `writeBufferSize`, this
|
|
73
|
+
applies to every column family when a database is first opened in the process and to column
|
|
74
|
+
families created later; a column family that is already open keeps its setting.
|
|
62
75
|
- `compression: string | { algorithm: string, level?: number }` The block/blob compression
|
|
63
76
|
algorithm for this column family. Pass an algorithm name — one of `'none'`, `'snappy'`,
|
|
64
77
|
`'zlib'`, `'bzip2'`, `'lz4'`, `'lz4hc'`, or `'zstd'` — or an object with an
|
|
@@ -202,11 +215,14 @@ Creates a new database instance.
|
|
|
202
215
|
nonempty directory that reappears before first use is refused until every handle closes and the
|
|
203
216
|
path reopens with this option. The name must be a single path component that does not start with
|
|
204
217
|
`.`: startup discovery skips such directories, so the log could never be scanned again.
|
|
205
|
-
- `verificationTable: boolean` When `true`, this
|
|
206
|
-
[Verification Table](#verification-table):
|
|
207
|
-
|
|
208
|
-
are cached (e.g. the primary column family of a table). Defaults to `false
|
|
209
|
-
|
|
218
|
+
- `verificationTable: boolean` When `true`, this handle participates in the process-global
|
|
219
|
+
[Verification Table](#verification-table): writes through it invalidate the verification slot
|
|
220
|
+
for each written key, and reads through it may seed slots. Enable this only for column families
|
|
221
|
+
whose records are cached (e.g. the primary column family of a table). Defaults to `false`; a
|
|
222
|
+
handle that leaves it unset joins a column family another handle opened with `true`, and an
|
|
223
|
+
explicit `false` throws while any handle on that column family participates (see
|
|
224
|
+
[Verification Table](#verification-table)). Requires `verificationTableEntries` to be configured
|
|
225
|
+
before the first database is opened.
|
|
210
226
|
- `writeBufferSize: number` The per-column-family memtable size in bytes at which the memtable is
|
|
211
227
|
sealed and flushed to an SST file. Smaller values produce more frequent, faster flushes; larger
|
|
212
228
|
values batch more writes per SST file at the cost of memory. Defaults to `16777216` (16 MB).
|
|
@@ -214,13 +230,16 @@ Creates a new database instance.
|
|
|
214
230
|
### `db.close()`
|
|
215
231
|
|
|
216
232
|
Closes a database. This function can be called multiple times and will only close an opened
|
|
217
|
-
database. A database instance can be reopened once it is closed.
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
233
|
+
database. A database instance can be reopened once it is closed. Closing the last handle flushes the
|
|
234
|
+
memtables and waits for that flush, then cancels background compaction instead of waiting for it:
|
|
235
|
+
a running compaction is abandoned and redone later, so close does not wait for the compaction
|
|
236
|
+
backlog (the flush itself still waits out a write stall). A flush failure leaves the native database quarantined so `shutdown()`
|
|
237
|
+
can retry without losing unflushed data; an explicit `destroy()` can instead delete it. A failure
|
|
238
|
+
from RocksDB's own final close is reported after native teardown completes; the optional
|
|
239
|
+
`compactOnClose` pass still runs to completion before background work is cancelled, but it is
|
|
240
|
+
best-effort and its errors do not fail the close, since a skipped compaction loses no data. All
|
|
241
|
+
native close errors emit `database:closeFailed`. The quarantine applies to both writable and
|
|
242
|
+
read-only opens because both modes share the physical path lifecycle.
|
|
224
243
|
|
|
225
244
|
```typescript
|
|
226
245
|
const db = RocksDatabase.open('foo');
|
|
@@ -290,6 +309,10 @@ Sets global database settings.
|
|
|
290
309
|
callers could already produce it. With `1`, a database's async commits run one at a time in
|
|
291
310
|
the order `commit()` was called. Databases already open keep their limit.
|
|
292
311
|
- `compactOnClose: boolean` When `true`, compacts the database on close. Defaults to `false`.
|
|
312
|
+
- `shutdownOnExit: boolean` When `true`, the main thread runs [`shutdown()`](#shutdown-void)
|
|
313
|
+
from a `process.on('exit')` listener, so `process.exit()` still flushes and closes every
|
|
314
|
+
database. Defaults to `true`. Set to `false` to leave memtables unflushed at a Node
|
|
315
|
+
`process.exit()`, e.g. to exercise crash recovery.
|
|
293
316
|
- `transactionLogAbortMarkers: boolean` When `true`, a transaction that wrote its
|
|
294
317
|
transaction-log batch and then ends without committing (an `abort()`, a commit error it gives
|
|
295
318
|
up on, or a worker torn down with it pending) gets that batch marked aborted in place, and every
|
|
@@ -714,7 +737,11 @@ because releasing the native database beneath an active copy would be unsafe.
|
|
|
714
737
|
A previously opened instance does not need to remain open, which allows an explicit `destroy()`
|
|
715
738
|
retry after failed physical cleanup. A never-opened or read-only instance cannot destroy the
|
|
716
739
|
database. `shutdown()` reports a pending cleanup tombstone but never retries deletion; only an
|
|
717
|
-
explicit `destroy()` can remove the path.
|
|
740
|
+
explicit `destroy()` can remove the path. Destruction flushes the memtables before it closes the
|
|
741
|
+
native database, without waiting out a write stall, so a database whose files outlive the call (the
|
|
742
|
+
deletion fails, or the process dies first) keeps writes made with `disableWAL`. If that flush fails,
|
|
743
|
+
destruction still deletes the database and reports the flush error through `database:closeFailed`;
|
|
744
|
+
only when the deletion then fails too are the unflushed `disableWAL` writes lost.
|
|
718
745
|
|
|
719
746
|
```typescript
|
|
720
747
|
db.destroy();
|
|
@@ -1020,7 +1047,9 @@ again, so a backward step between runs can reissue a timestamp that is already a
|
|
|
1020
1047
|
node's transaction log. Opening with
|
|
1021
1048
|
[`timestampFloorLog`](#new-rocksdatabasepath-options) raises the clock
|
|
1022
1049
|
above every batch key still durable in the named log before the database handle is returned, so no
|
|
1023
|
-
transaction can be constructed below it.
|
|
1050
|
+
transaction can be constructed below it. Keys in segments retention has purged are not durable, and
|
|
1051
|
+
neither is a bound advertised to peers; carry those across a restart yourself with
|
|
1052
|
+
[`raiseTimestampFloor()`](#raisetimestampfloortimestamp-number-boolean).
|
|
1024
1053
|
|
|
1025
1054
|
Name only a log this process **originates**. A log written under a timestamp adopted from another
|
|
1026
1055
|
node — a replication receiver calling [`txn.setTimestamp()`](#txnsettimestampts-number-void) — is
|
|
@@ -1451,6 +1480,11 @@ Type: `number`
|
|
|
1451
1480
|
The transaction ID, a positive integer no greater than `Number.MAX_SAFE_INTEGER`. Transaction IDs
|
|
1452
1481
|
are unique to the RocksDB database path, regardless the database name/column family.
|
|
1453
1482
|
|
|
1483
|
+
A transaction belongs to the thread that created it. Although every `worker_threads` worker that
|
|
1484
|
+
opens the same path shares one id space, using a transaction by id from another worker (for example
|
|
1485
|
+
`{ transaction }` read options or `log.addEntry(data, txnId)`) throws `ERR_TRANSACTION_WRONG_ENV`:
|
|
1486
|
+
RocksDB transactions are not thread-safe.
|
|
1487
|
+
|
|
1454
1488
|
#### `txn.setTimestamp(ts?: number): void`
|
|
1455
1489
|
|
|
1456
1490
|
Overrides the transaction timestamp in milliseconds since the Unix epoch. Replication receivers and
|
|
@@ -2190,10 +2224,12 @@ The freshness check works as follows:
|
|
|
2190
2224
|
2. On a cold read, pass `{ populateVersion: true }` (or call `db.populateVersion()` afterward) to
|
|
2191
2225
|
seed the slot with the version extracted from the value, so subsequent freshness checks hit.
|
|
2192
2226
|
|
|
2193
|
-
|
|
2194
|
-
|
|
2195
|
-
|
|
2196
|
-
|
|
2227
|
+
Writes through a handle opened with `verificationTable: true` invalidate the slot for each written
|
|
2228
|
+
key at write time, so a stale version cannot survive them, and a read seeds a slot only through
|
|
2229
|
+
such a handle. Once one handle opens a column family with the option, the handles on it that left
|
|
2230
|
+
the option unset participate too (see below). Invalidation is also what enables
|
|
2231
|
+
[`coordinatedRetry`](#transactionoptions): a conflicting transaction parks on the slot and retries
|
|
2232
|
+
once the write intent is released.
|
|
2197
2233
|
|
|
2198
2234
|
The table is **process-global** and backed by a single shared structure, so versions populated on
|
|
2199
2235
|
the main thread are visible to `worker_threads` workers and vice versa.
|
|
@@ -2202,7 +2238,8 @@ the main thread are visible to `worker_threads` workers and vice versa.
|
|
|
2202
2238
|
|
|
2203
2239
|
The table must be sized before the first database is opened, via the
|
|
2204
2240
|
[`verificationTableEntries`](#dbconfigoptions) config option (default `131072` slots = 1 MB; set to
|
|
2205
|
-
`0` to disable). Then
|
|
2241
|
+
`0` to disable). Then open each cached column family with the `verificationTable: true` open
|
|
2242
|
+
option:
|
|
2206
2243
|
|
|
2207
2244
|
```typescript
|
|
2208
2245
|
import { RocksDatabase } from '@harperfast/rocksdb-js';
|
|
@@ -2215,6 +2252,18 @@ const db = RocksDatabase.open('path/to/db', { verificationTable: true });
|
|
|
2215
2252
|
Enable `verificationTable` only for column families whose records are cached (e.g. the primary
|
|
2216
2253
|
column family of a table); enabling it adds per-write slot invalidation overhead.
|
|
2217
2254
|
|
|
2255
|
+
Participation applies to the whole column family within the process. Once a handle opens a column
|
|
2256
|
+
family with `verificationTable: true`, every later handle on it participates too, even one opened
|
|
2257
|
+
without the option, so a write through any handle invalidates its key's slot. This lasts until
|
|
2258
|
+
every handle on the database closes, or until the column family is opened with an explicit
|
|
2259
|
+
`verificationTable: false` after every participating handle on it has closed. While any handle on
|
|
2260
|
+
it participates, that explicit `false` throws `ERR_VERIFICATION_TABLE_CONFLICT`. Opening with
|
|
2261
|
+
`verificationTable: true` throws `ERR_VERIFICATION_TABLE_CONFLICT` while a non-participating handle
|
|
2262
|
+
is open on that column family: one opened before any opt-in, or with an explicit `false`. After
|
|
2263
|
+
such a handle closes, the same error is thrown while a transaction that existed before it closed
|
|
2264
|
+
is still pending, or while another operation on the database is running; retry the open once they
|
|
2265
|
+
finish.
|
|
2266
|
+
|
|
2218
2267
|
### `db.verifyVersion(key: Key, version: number): boolean`
|
|
2219
2268
|
|
|
2220
2269
|
Returns `true` when the verification table currently records `version` for `key` (in this database
|
|
@@ -2315,7 +2364,10 @@ every file after it is retained regardless of age; `purge.retainedUnflushedFiles
|
|
|
2315
2364
|
While a backup with `transactionLogs` is copying a store's files, purges of that store are skipped
|
|
2316
2365
|
(`purge.retentionPins` reports it). On a filesystem that rejects a directory fsync, the store's last
|
|
2317
2366
|
file is kept, since `txn.state` could not be made durable before deleting it.
|
|
2318
|
-
|
|
2367
|
+
Before deleting any file, the purge records the largest key it held in the log's purge watermark
|
|
2368
|
+
([`log.getPurgeWatermark()`](#loggetpurgewatermark-transactionlogpurgewatermark)) and makes that
|
|
2369
|
+
durable. Use `destroy: true` only to remove the store itself; a log recreated after it starts a new
|
|
2370
|
+
watermark history.
|
|
2319
2371
|
|
|
2320
2372
|
- `options: object`
|
|
2321
2373
|
- `before?: number` Remove transaction log files older than the specified timestamp, subject to
|
|
@@ -2375,6 +2427,8 @@ to disk right before the transaction is committed. You may add multiple enties p
|
|
|
2375
2427
|
underlying architecture is thread safe.
|
|
2376
2428
|
|
|
2377
2429
|
- `log.addEntry()`
|
|
2430
|
+
- `log.equals()`
|
|
2431
|
+
- `log.getPurgeWatermark()`
|
|
2378
2432
|
- `log.path`
|
|
2379
2433
|
- `log.query()`
|
|
2380
2434
|
|
|
@@ -2407,6 +2461,57 @@ Note that the `TransactionLog` class also has internal methods `_getMemoryMapOfF
|
|
|
2407
2461
|
`_findPosition`, and `_getLastCommittedPosition` that should not be used directly and may change in
|
|
2408
2462
|
any version.
|
|
2409
2463
|
|
|
2464
|
+
#### `log.equals(other): boolean`
|
|
2465
|
+
|
|
2466
|
+
Returns `true` when `other` is a `TransactionLog` for the same log: the same database and the same
|
|
2467
|
+
log name. The database is compared by its resolved filesystem identity, so `db.useLog('foo')`,
|
|
2468
|
+
`txn.useLog('foo')` and a log from another handle opened with a different spelling of the same path
|
|
2469
|
+
all compare equal, and two databases opened through the same relative path from different working
|
|
2470
|
+
directories do not. The comparison still works after either database closes. Any other value
|
|
2471
|
+
compares unequal.
|
|
2472
|
+
|
|
2473
|
+
```typescript
|
|
2474
|
+
const log = db.useLog('foo');
|
|
2475
|
+
log.equals(otherHandle.useLog('foo')); // true: same database and name
|
|
2476
|
+
log.equals(db.useLog('bar')); // false
|
|
2477
|
+
```
|
|
2478
|
+
|
|
2479
|
+
A log destroyed with `purgeLogs({ destroy: true })` and recreated under the same name still compares
|
|
2480
|
+
equal to wrappers of the old one.
|
|
2481
|
+
|
|
2482
|
+
#### `log.getPurgeWatermark(): TransactionLogPurgeWatermark`
|
|
2483
|
+
|
|
2484
|
+
```typescript
|
|
2485
|
+
const log = db.useLog('foo');
|
|
2486
|
+
const entries = Array.from(log.query({ start: resumeKey, exclusiveStart: true }));
|
|
2487
|
+
const watermark = log.getPurgeWatermark();
|
|
2488
|
+
const complete = watermark.known && resumeKey >= watermark.purgedThrough;
|
|
2489
|
+
```
|
|
2490
|
+
|
|
2491
|
+
Returns an upper bound on every entry key that retention or `db.purgeLogs()` has deleted from this
|
|
2492
|
+
log, or reports that no such bound can be given. A reader that needs every entry with a key above
|
|
2493
|
+
some position can be served from the log only when that position is at or above `purgedThrough`.
|
|
2494
|
+
|
|
2495
|
+
- `{ known: true, purgedThrough: number }` `purgedThrough` is at or above every key of every file
|
|
2496
|
+
deleted from the log, in whatever order the keys were written (0 when nothing was deleted). It is
|
|
2497
|
+
made durable before each file is deleted, including by the retention that runs when a database
|
|
2498
|
+
opens.
|
|
2499
|
+
- `{ known: false }` The log cannot vouch for what was deleted. This holds permanently for a log
|
|
2500
|
+
created by a release without the watermark, one that file deletions bypassed (such as an older
|
|
2501
|
+
release's retention), and one whose record cannot be read. It is also returned when the log's
|
|
2502
|
+
store no longer exists, such as after `purgeLogs({ destroy: true })` until the next write.
|
|
2503
|
+
|
|
2504
|
+
The call is synchronous. A read-only or secondary database reads the record from disk on each call.
|
|
2505
|
+
The result is a snapshot: a purge can run right after it. To trust what a read returned, read the log
|
|
2506
|
+
first and check the watermark afterwards. See
|
|
2507
|
+
[the purge watermark](docs/transaction-log.md#purge-watermark) for the rules.
|
|
2508
|
+
|
|
2509
|
+
##### `TransactionLogPurgeWatermark`
|
|
2510
|
+
|
|
2511
|
+
```typescript
|
|
2512
|
+
type TransactionLogPurgeWatermark = { known: true; purgedThrough: number } | { known: false };
|
|
2513
|
+
```
|
|
2514
|
+
|
|
2410
2515
|
#### `log.path: string`
|
|
2411
2516
|
|
|
2412
2517
|
Returns the path to the transaction log store files.
|
|
@@ -2416,7 +2521,7 @@ const log = db.useLog('foo');
|
|
|
2416
2521
|
console.log(log.path);
|
|
2417
2522
|
```
|
|
2418
2523
|
|
|
2419
|
-
#### `log.query(options?):
|
|
2524
|
+
#### `log.query(options?): TransactionLogIterator`
|
|
2420
2525
|
|
|
2421
2526
|
Returns an iterable/iterator that streams all log entries for the given filter.
|
|
2422
2527
|
|
|
@@ -2425,6 +2530,11 @@ Returns an iterable/iterator that streams all log entries for the given filter.
|
|
|
2425
2530
|
- `end?: string` The transction end timestamp.
|
|
2426
2531
|
- `exclusiveStart?: boolean` When `true`, this will only match transactions with timestamps after
|
|
2427
2532
|
the start timestamp.
|
|
2533
|
+
- `until?: TransactionLogIterator` Stops before the unread position of another iterator from a log
|
|
2534
|
+
that `equals()` this one, including an entry it has peeked. The bound is exclusive, compares
|
|
2535
|
+
segment and offset before filtering, and works across rotation, even when this query starts past
|
|
2536
|
+
it. It resumes when `until` advances. A closed iterator no longer bounds the query. An invalid or
|
|
2537
|
+
foreign-log iterator throws `TypeError` at query creation.
|
|
2428
2538
|
- `exactStart?: boolean` When `true`, this will only match and iterate starting from a transaction
|
|
2429
2539
|
with the given start timestamp. Once the specified transaction is found, all subsequent
|
|
2430
2540
|
transactions will be returned (regardless of whether their timestamp comes before the `start`
|
|
@@ -2441,6 +2551,32 @@ Returns an iterable/iterator that streams all log entries for the given filter.
|
|
|
2441
2551
|
have been written to the log but are no longer considered committed if they were not flushed to
|
|
2442
2552
|
disk.
|
|
2443
2553
|
|
|
2554
|
+
An unbounded query whose starting segment is temporarily unmappable reports done and retries
|
|
2555
|
+
its original position on the next pull.
|
|
2556
|
+
|
|
2557
|
+
`iterator.peek()` returns the next entry without consuming it. Repeated peeks return the same
|
|
2558
|
+
result until `next()` consumes it once. A `done` result is not cached, so later appends remain
|
|
2559
|
+
visible. Errors propagate as from `next()`.
|
|
2560
|
+
|
|
2561
|
+
With `until`, `done: true` means the reader has caught up to the bound as of that pull. Running
|
|
2562
|
+
out of readable entries short of the bound throws instead, so an unavailable segment cannot
|
|
2563
|
+
silently trigger a handoff. Exact-start queries take the index's stop-to-tail jump only when
|
|
2564
|
+
its target does not pass the handoff position.
|
|
2565
|
+
|
|
2566
|
+
`iterator.close()` permanently ends the iterator and releases its cached entry and mapping.
|
|
2567
|
+
It is idempotent; later `next()` and `peek()` calls report done. Breaking out of `for...of`
|
|
2568
|
+
keeps the iterator resumable and does not close it. Destroying a log with
|
|
2569
|
+
`purgeLogs({ destroy: true })` invalidates its existing iterators; close them and create new
|
|
2570
|
+
iterators for a recreated log, including any `until` bounds.
|
|
2571
|
+
|
|
2572
|
+
```typescript
|
|
2573
|
+
const live = log.query();
|
|
2574
|
+
const replay = log.query({ start: startTime, until: live });
|
|
2575
|
+
for (const entry of replay) deliver(entry);
|
|
2576
|
+
replay.close();
|
|
2577
|
+
// A later commit belongs to live even if its timestamp is earlier than startTime.
|
|
2578
|
+
```
|
|
2579
|
+
|
|
2444
2580
|
The iterator produces an object with the log entry timestamp and data.
|
|
2445
2581
|
|
|
2446
2582
|
- `object`
|
|
@@ -2448,10 +2584,11 @@ The iterator produces an object with the log entry timestamp and data.
|
|
|
2448
2584
|
- `timestamp: number` The entry timestamp used to collate entries by transaction.
|
|
2449
2585
|
- `endTxn: boolean` This is `true` when the entry is the last entry in a transaction.
|
|
2450
2586
|
|
|
2451
|
-
|
|
2452
|
-
past the problem. Other errors, such as a failure to open a segment, do not move the iterator, so
|
|
2453
|
-
retrying them without a fix fails the same way.
|
|
2587
|
+
These errors leave the iterator usable:
|
|
2454
2588
|
|
|
2589
|
+
- `TransactionLogUntilUnreachedError`: readable input ended before `until`. The cursor stays in
|
|
2590
|
+
place; retry after the input becomes readable or the pending transaction commits. Properties:
|
|
2591
|
+
`logId` and `position`, identifying where the read stopped.
|
|
2455
2592
|
- `TransactionLogGapError`: the iterator reached segments it had not read yet and could not read
|
|
2456
2593
|
them, usually because retention purged them while the reader lagged. Their entries are not in
|
|
2457
2594
|
this log anymore, so a consumer that replicates the log should treat its copy as discontinuous
|
|
@@ -2462,7 +2599,9 @@ retrying them without a fix fails the same way.
|
|
|
2462
2599
|
- `CorruptFrameError` (a `RangeError`): an entry's framing is broken. `resyncPosition` is where
|
|
2463
2600
|
valid framing resumes, or `undefined` when nothing follows the break.
|
|
2464
2601
|
|
|
2465
|
-
|
|
2602
|
+
All three classes are exported from the package. After a gap or corrupt frame, calling `next()`
|
|
2603
|
+
again continues past the problem. Other errors, such as a failure to open a segment, do not move
|
|
2604
|
+
the iterator, so retrying them without a fix fails the same way.
|
|
2466
2605
|
|
|
2467
2606
|
```typescript
|
|
2468
2607
|
const log = db.useLog('foo');
|
|
@@ -2643,6 +2782,55 @@ Contract:
|
|
|
2643
2782
|
- **Cost**: one clock read and a double conversion; no lock, no native allocation, no wall-clock read,
|
|
2644
2783
|
no database or log I/O, no per-worker calibration. Safe from any thread. The runtime may allocate a boxed JS number.
|
|
2645
2784
|
|
|
2785
|
+
### `raiseTimestampFloor(timestamp: number): boolean`
|
|
2786
|
+
|
|
2787
|
+
Raises the process-wide monotonic timestamp clock — the clock behind
|
|
2788
|
+
[`db.getMonotonicTimestamp()`](#dbgetmonotonictimestamp-number), every transaction's initial
|
|
2789
|
+
timestamp and therefore every transaction-log batch key — so that every timestamp the process
|
|
2790
|
+
issues afterwards is strictly greater than `timestamp` (Unix epoch milliseconds). No database
|
|
2791
|
+
handle is needed.
|
|
2792
|
+
|
|
2793
|
+
```typescript
|
|
2794
|
+
import { RocksDatabase, raiseTimestampFloor } from '@harperfast/rocksdb-js';
|
|
2795
|
+
|
|
2796
|
+
// Before anything in this process starts a transaction:
|
|
2797
|
+
raiseTimestampFloor(persistedFloor);
|
|
2798
|
+
const db = RocksDatabase.open('/path/to/db', { timestampFloorLog: 'local' });
|
|
2799
|
+
```
|
|
2800
|
+
|
|
2801
|
+
The clock is monotonic within a process, not across restarts, and
|
|
2802
|
+
[`timestampFloorLog`](#timestamp-floor-at-open) only restores keys still on disk. A bound this node
|
|
2803
|
+
has promised that is not in its log — a progress floor advertised to peers, or the largest key of
|
|
2804
|
+
segments retention has since purged — can only come back through this call. The caller names the
|
|
2805
|
+
value because only the caller knows which bounds it has promised.
|
|
2806
|
+
|
|
2807
|
+
Contract:
|
|
2808
|
+
|
|
2809
|
+
- **Returns** `true` if the clock was raised: `timestamp` was ahead of both the wall clock and every
|
|
2810
|
+
timestamp already issued, as after a backward clock step. `false` if the clock was already at or
|
|
2811
|
+
above it, which is the usual result at a restart. Raising is one-way; a lower value never moves
|
|
2812
|
+
the clock back.
|
|
2813
|
+
- **Throws** a `TypeError` if `timestamp` is not a number, and a `RangeError` if it is not finite, is
|
|
2814
|
+
not greater than `0`, is not less than `8.64e15`, or is more than ten years ahead of the wall
|
|
2815
|
+
clock — the same bound `timestampFloorLog` applies to a durable key. An unusable persisted floor
|
|
2816
|
+
therefore fails loudly rather than being ignored. The implausibility error includes the wall
|
|
2817
|
+
clock it was checked against, since a host that booted with an unsynchronized clock trips it too.
|
|
2818
|
+
- **Process-wide**: it applies to every database and every `worker_threads` worker in the process,
|
|
2819
|
+
but not to other processes. Each new process must call it again.
|
|
2820
|
+
- **Only issued timestamps**: a transaction constructed before the call keeps its timestamp, and
|
|
2821
|
+
[`txn.setTimestamp(ts)`](#txnsettimestampts-number-void) can still adopt any valid timestamp,
|
|
2822
|
+
including one below the floor. `txn.setTimestamp()` with no argument issues a fresh timestamp
|
|
2823
|
+
above it.
|
|
2824
|
+
- **Not durable and not a fence**: the floor lives in memory only, and nothing stops a transaction
|
|
2825
|
+
constructed concurrently with the call. Persist the floor yourself, and restore it before the
|
|
2826
|
+
process starts any transaction it originates. It composes with `timestampFloorLog` (the clock takes
|
|
2827
|
+
the larger of the two), so keep naming the local log unless the persisted floor already covers
|
|
2828
|
+
every key written to it.
|
|
2829
|
+
- **Trusted input only**: the floor is not lowered until the process exits, so a value hours or years
|
|
2830
|
+
ahead holds every timestamp that far ahead of wall time until the clock catches up — and keys
|
|
2831
|
+
written in the meantime raise the next restart's seed with them. Pass only floors this node
|
|
2832
|
+
persisted itself, never an unvalidated value received from another node.
|
|
2833
|
+
|
|
2646
2834
|
### `currentThreadId(): number`
|
|
2647
2835
|
|
|
2648
2836
|
Returns the current thread ID.
|
|
@@ -2676,27 +2864,41 @@ console.log(getRegistryStatus());
|
|
|
2676
2864
|
|
|
2677
2865
|
### `shutdown(): void`
|
|
2678
2866
|
|
|
2679
|
-
The `shutdown()` will flush all in-memory data to disk and
|
|
2680
|
-
|
|
2681
|
-
|
|
2682
|
-
|
|
2683
|
-
|
|
2684
|
-
tombstones without deleting their paths; retry those with an explicit `destroy()`.
|
|
2867
|
+
The `shutdown()` will flush all in-memory data to disk and cancel outstanding background
|
|
2868
|
+
compactions, for all open databases. It throws the first close failure after attempting every
|
|
2869
|
+
claimed database; call it again to retry any descriptor whose native teardown did not complete. It
|
|
2870
|
+
reports pending destroy-cleanup tombstones without deleting their paths; retry those with an
|
|
2871
|
+
explicit `destroy()`.
|
|
2685
2872
|
|
|
2686
|
-
|
|
2687
|
-
|
|
2873
|
+
You do not need to call it at exit. When rocksdb-js loads on the main thread, it registers a
|
|
2874
|
+
`process.on('exit')` listener that runs the same shutdown, so a `process.exit()` (which skips the
|
|
2875
|
+
native cleanup Node runs on a natural exit) still flushes and closes every database, including
|
|
2876
|
+
those opened by workers that are still running. That listener reports failures to stderr instead
|
|
2877
|
+
of throwing, and sets `process.exitCode` to 1 (unless it is already nonzero) when data may be left
|
|
2878
|
+
unflushed: a database another thread was still closing when the shutdown timed out, or a final
|
|
2879
|
+
flush that failed. Afterwards opening a database throws. It
|
|
2880
|
+
still runs when an earlier `exit` listener throws. It does not run when the process is killed by a
|
|
2881
|
+
signal it does not handle.
|
|
2882
|
+
|
|
2883
|
+
Like the cleanup on a natural exit, it waits for native work already in flight (a commit, a flush
|
|
2884
|
+
waiting out a write stall, or a backup) without a time limit, so `process.exit()` returns only once
|
|
2885
|
+
that work finishes.
|
|
2886
|
+
|
|
2887
|
+
Because it is registered when rocksdb-js loads, `exit` listeners added later run after databases
|
|
2888
|
+
are closed. An `exit` listener that still writes to a database must run first:
|
|
2688
2889
|
|
|
2689
2890
|
```typescript
|
|
2690
|
-
|
|
2691
|
-
|
|
2692
|
-
try {
|
|
2693
|
-
shutdown();
|
|
2694
|
-
} catch (error) {
|
|
2695
|
-
console.error('rocksdb-js shutdown failed', error);
|
|
2696
|
-
}
|
|
2891
|
+
process.prependListener('exit', () => {
|
|
2892
|
+
db.putSync('lastSeen', Date.now());
|
|
2697
2893
|
});
|
|
2698
2894
|
```
|
|
2699
2895
|
|
|
2896
|
+
Set `RocksDatabase.config({ shutdownOnExit: false })` to skip it, for example to leave memtables
|
|
2897
|
+
unflushed at `process.exit()` and exercise crash recovery in a test. The listener stays registered
|
|
2898
|
+
and does nothing: on a natural exit the runtime's native cleanup still flushes, and whether Bun's
|
|
2899
|
+
does at `process.exit()` depends on the Bun release.
|
|
2900
|
+
Calling `shutdown()` from your own `exit` listener as well is safe.
|
|
2901
|
+
|
|
2700
2902
|
### `versions: { 'rocksdb': string; 'rocksdb-js': string }`
|
|
2701
2903
|
|
|
2702
2904
|
Returns the `rocksdb-js` and RocksDB version.
|
|
@@ -3021,11 +3223,13 @@ Options for `get()`, `getSync()`, and the `getBinary*` methods.
|
|
|
3021
3223
|
- `expectedVersion: number` When set, the [Verification Table](#verification-table) is checked
|
|
3022
3224
|
before reading. If the slot holds this version, the read is skipped and the
|
|
3023
3225
|
`FRESH_VERSION_FLAG` sentinel is returned. After a database read, the slot is seeded with the
|
|
3024
|
-
version extracted from the value. Requires the
|
|
3226
|
+
version extracted from the value. Requires the handle to be opened with
|
|
3025
3227
|
`verificationTable: true`.
|
|
3026
3228
|
- `populateVersion: boolean` When `true`, after a database read the verification-table slot is
|
|
3027
3229
|
seeded with the version extracted from the value, eliminating the need for a separate
|
|
3028
|
-
`db.populateVersion()` call on cold reads.
|
|
3230
|
+
`db.populateVersion()` call on cold reads. Requires the handle to participate in the
|
|
3231
|
+
[Verification Table](#verification-table); otherwise the read seeds nothing. Defaults to
|
|
3232
|
+
`false`.
|
|
3029
3233
|
- `skipDecode: boolean` When `true`, the value is returned without being decoded. Defaults to
|
|
3030
3234
|
`false`.
|
|
3031
3235
|
|