@harperfast/rocksdb-js 2.9.1 → 2.11.0

package/README.md

@@ -169,10 +169,25 @@ Creates a new database instance.
   - `transactionLogMaxSize: number` The target maximum size of a transaction log file. Transactions
     are never split across files: if the complete transaction does not fit, the log rotates before
     writing it. A transaction written to an empty file may exceed the target. Defaults to 16 MB.
+    `0` is rejected: it formerly meant unlimited segment size, which a reader's fixed-size mapping
+    cannot cover. A store configured with `0` must omit the option or set a size before opening;
+    once it rotates by size, rotated segments older than `transactionLogRetention` are purged, so
+    raise the retention to keep more history.
   - `transactionLogRetention: string | number` The number of minutes to retain transaction logs
     before purging. Defaults to `'3d'` (3 days).
   - `transactionLogsPath: string` The path to store transaction logs. Defaults to
     `"${db.path}/transaction_logs"`.
+  - `timestampFloorLog: string` The name of the transaction log whose batch keys this process
+    originates. At open, the process-wide monotonic clock is raised above every batch key still
+    durable in that log, so a backward wall-clock step between runs cannot reissue a transaction
+    timestamp that is already a key in it. See
+    [Timestamp floor at open](#timestamp-floor-at-open). Unset by default, which leaves the clock
+    alone. Its first use in a process must be on a writable primary open; read-only and secondary
+    handles can reuse an already-established seed for the same log but cannot establish one.
+    If that first seed warned because the log was absent, the log may be created fresh, but a
+    nonempty directory that reappears before first use is refused until every handle closes and the
+    path reopens with this option. The name must be a single path component that does not start with
+    `.`: startup discovery skips such directories, so the log could never be scanned again.
   - `verificationTable: boolean` When `true`, this column family participates in the process-global
     [Verification Table](#verification-table): transaction writes to this column family invalidate
     the verification slot for each written key. Enable this only for column families whose records
@@ -185,7 +200,13 @@ Creates a new database instance.
 ### `db.close()`
 
 Closes a database. This function can be called multiple times and will only close an opened
-database. A database instance can be reopened once its closed.
+database. A database instance can be reopened once it is closed. A flush failure leaves the native
+database quarantined so `shutdown()` can retry without losing unflushed data; an explicit
+`destroy()` can instead delete it. A failure while waiting for compaction to settle is reported
+after native teardown completes; the optional `compactOnClose` pass itself is best-effort and its
+errors do not fail the close, since a skipped compaction loses no data. All native close errors
+emit `database:closeFailed`. The quarantine applies to both writable and read-only opens because
+both modes share the physical path lifecycle.
 
 ```typescript
 const db = RocksDatabase.open('foo');
@@ -212,12 +233,21 @@ when a non-default compression level is set. The database must be open. See
 [Compression](#compression).
 
 ```typescript
-const db = RocksDatabase.open('path/to/db', {
-	compression: { algorithm: 'zstd', level: 3 },
-});
+const db = RocksDatabase.open('path/to/db', { compression: { algorithm: 'zstd', level: 3 } });
 console.log(db.compression); // { algorithm: 'zstd', level: 3 }
 ```
 
+### `db.setCompression(compression)`
+
+Dynamically changes the compression algorithm (and optional level) for this database's column
+family on an already-open database — no close, no reopen. See [Compression](#compression) for full
+semantics and caveats.
+
+```typescript
+const db = RocksDatabase.open('path/to/db', { compression: 'none' });
+db.setCompression({ algorithm: 'zstd', level: 19 });
+```
+
 ### `db.config(options)`
 
 Sets global database settings.
@@ -226,7 +256,65 @@ Sets global database settings.
   - `blockCacheSize: number` The amount of memory in bytes to use to cache uncompressed blocks.
     Defaults to 32MB. Set to `0` (zero) disables block cache for future opened databases. Existing
     block cache for any opened databases is resized immediately. Negative values throw an error.
+  - `commitThreads: number` The maximum number of dedicated threads each database opened after the
+    call uses to run async `transaction.commit()` calls. Defaults to `min(4, cores)`; accepts
+    integers from `1` through `64`. A thread starts only when a commit finds every started thread
+    busy, so a database that is never committed to concurrently owns one. Async commits never use
+    the libuv threadpool, so slow commits cannot delay `fs`, `dns`, `crypto` or async `get()` calls.
+    With more than one thread, RocksDB validates and applies concurrent commits on several cores
+    and can combine them into one write: with four worker threads committing 64-key transactions to
+    one database, this committed about twice as many transactions per second as one thread.
+    Concurrent commits finish, and their promises resolve, in any order, as concurrent
+    `commitSync()` calls always have, and their transaction-log entries are appended in the order
+    the commits reach the log; a log reader never sees an entry before its transaction's data is
+    committed. Log position is not the order in which writes to the same key were applied: a
+    commit that conflicts and is retried keeps its original log position but lands after commits
+    logged behind it. Its transaction timestamp does not advance across a retry either, so
+    replaying entries for one key in timestamp order does not recover true apply order — a
+    consumer that needs one must have the producer embed its own last-writer-wins marker.
+    Concurrent commit threads make this more frequent; a single commit thread and `commitSync()`
+    callers could already produce it. With `1`, a database's async commits run one at a time in
+    the order `commit()` was called. Databases already open keep their limit.
   - `compactOnClose: boolean` When `true`, compacts the database on close. Defaults to `false`.
+  - `occValidation: 'parallel' | 'serial'` How writable optimistic databases opened after the call
+    validate commits for conflicts. Defaults to `'parallel'` (RocksDB's default). Databases already
+    open keep their policy. Pessimistic, read-only and secondary opens are unaffected.
+    - `'parallel'` locks the commit's keys in a per-database pool of `occLockBuckets` mutexes,
+      checks for conflicts, and holds the locks through the write. Commits on different threads
+      that touch different buckets can then be batched into one RocksDB write. Non-transactional
+      writes (`db.putSync()` and friends) take no bucket, so one that lands between a commit's
+      conflict check and its write is overwritten by that commit without an `ERR_BUSY`.
+    - `'serial'` checks for conflicts inside RocksDB's write group instead, and allocates no bucket
+      pool, which saves its memory. Because the check and the write happen in one step, a
+      non-transactional write cannot land between them and be silently overwritten, and a column
+      family cannot be dropped between them. The cost: RocksDB never batches a serial-validated
+      commit with another writer, including the database's other commit threads, so it gives back
+      what `commitThreads` gains. With four worker threads
+      committing 64-key transactions to one database it committed about 4,500 transactions per
+      second against about 9,900 for `'parallel'`, the same as a single commit thread; it came out
+      ahead (about 7%) only at 1,000 keys per transaction with the default bucket count, where
+      raising `occLockBuckets` gains more. Concurrent `commitSync()` calls lose throughput the same
+      way. It suits a database committed to one transaction at a time (`commitThreads: 1`).
+  - `occLockBuckets: number` The number of commit lock buckets each writable optimistic database
+    allocates when it is opened with `occValidation: 'parallel'`. Defaults to `65536` (2^16), not
+    RocksDB's 2^20. Accepts integers from `16` through `16777216` (2^24). Applies to databases
+    opened after the call; databases already open keep their count. Memory is the count times the
+    platform mutex size (40 bytes on Linux x64, 64 bytes on macOS), so the default is about 2.5 MiB
+    per database on Linux, against about 40 MiB at RocksDB's default. Multiple keys can share a
+    bucket, so this is **not a transaction write limit**. A small pool costs throughput only when
+    commits to the same database overlap (its `commitThreads`, `commitSync()` callers, or libuv
+    commits under `ROCKSDB_JS_COMMIT_THREAD=0`) and their keys share a bucket: colliding commits
+    run one at a time and cannot share a RocksDB write. Collisions grow with transaction size. With
+    four worker threads and the default four commit threads, 65536 buckets matched 2^20 and 2^22 at
+    64 keys per transaction (and 4096 gave about a third fewer commits per second), but at 1,000 keys
+    65536 committed 336 transactions per second against 466 at 2^20 and 685 at 2^22, so databases
+    with large transactions should raise the count.
+  - `lifecycleWaitSeconds: number` How long a synchronous open, destroy, or shutdown waits for a
+    _conflicting_ lifecycle operation already in progress on the same path (e.g. another open or
+    close) before throwing a retryable timeout error. It does not bound the separate, intentionally
+    unbounded wait that `destroy()`/`shutdown()` make for in-flight backups, checkpoints, or other
+    async work still using the database — see [`db.destroy()`](#dbdestroy-void). Defaults to `30`
+    seconds and must be a positive integer.
   - `verificationTableEntries: number` The number of slots in the process-global
     [Verification Table](#verification-table). Each slot is 8 bytes, so the default of `131072`
     (128K) slots is 1 MB. Set to `0` to disable the verification table. This must be configured
@@ -254,7 +342,10 @@ Sets global database settings.
 ```typescript
 RocksDatabase.config({
 	blockCacheSize: 100 * 1024 * 1024, // 100MB
+	commitThreads: 4, // per database; applies to subsequent opens
 	compactOnClose: true,
+	occLockBuckets: 1 << 20, // per optimistic database with parallel validation
+	occValidation: 'parallel', // or 'serial'; applies to subsequent optimistic opens
 	writeBufferManagerAllowStall: false,
 	writeBufferManagerCostToCache: false,
 	writeBufferManagerSize: 64 * 1024 * 1024, // 64MB
@@ -574,7 +665,16 @@ db.compactSync({ bottommost: true });
 ### `db.destroy(): void`
 
 Completely removes a database based on the `db` instance's path including all data, column families,
-and files on disk.
+and files on disk. Destruction owns the physical path for the process: it closes every writable and
+read-only handle for that path, waits for registered backups and checkpoints to stop using the
+native database, and prevents another handle from reopening the path until removal finishes. Those
+waits are synchronous and can outlive `lifecycleWaitSeconds` once destruction has claimed the path,
+because releasing the native database beneath an active copy would be unsafe.
+
+A previously opened instance does not need to remain open, which allows an explicit `destroy()`
+retry after failed physical cleanup. A never-opened or read-only instance cannot destroy the
+database. `shutdown()` reports a pending cleanup tombstone but never retries deletion; only an
+explicit `destroy()` can remove the path.
 
 ```typescript
 db.destroy();
@@ -583,8 +683,8 @@ console.log(fs.existsSync(db.path)); // false
 
 ### `db.drop(): Promise<void>`
 
-Removes all entries in the database. If the database was opened with a `name`, the database will be
-deleted on close.
+Drops the column family the database was opened with (`name`). For the default column family this
+clears all entries instead.
 
 ```typescript
 const db = RocksDatabase.open('path/to/db', { name: 'users' });
@@ -592,9 +692,56 @@ await db.drop();
 db.close();
 ```
 
+#### Dropping column families
+
+A drop retires the column family **logically** before it returns: the name is gone from
+`db.columns`, a later `open()` with the same name creates a fresh, empty column family, and any
+transaction that then stages a write to a handle of the dropped family, or commits one it staged
+earlier, is refused whole with `ERR_COLUMN_FAMILY_DROPPED` (`Column family "users" was dropped`).
+That terminal refusal releases the transaction's verification-table intents and bars further
+writes or commit attempts; retained reads continue until the caller aborts the transaction, and
+they still serve that transaction's own staged writes — values no commit will ever produce. A
+caller that catches the refusal instead of letting it propagate must not read a value back through
+the transaction and carry it forward.
+If a staging call fails while the generation is being retired, retirement takes precedence and the
+transaction is refused whole even when the immediate RocksDB failure was a pessimistic lock timeout.
+Handles other threads still hold keep **reading** the dropped data until they close; a
+non-transactional `putSync`/`removeSync` through such a handle is discarded.
+
+The **physical** RocksDB drop is deferred behind commits already admitted when the drop lands: a
+commit claims every column family its batch names before it writes its transaction-log batch and
+releases them after RocksDB has applied it, and the physical drop runs from whichever
+side releases last (or, for a commit a mid-flight `close()` tore out of its pipeline, from the next
+drop, open of that name, or close on the database). With no such commit (the common case) `drop()`/`dropSync()` perform the
+physical drop before returning, exactly as before. This is what keeps a drop racing another
+thread's commit from latching RocksDB's fatal `Invalid column family specified in write batch`
+error on the whole database.
+
+Consequences to know about:
+
+- A commit admitted before the drop completes successfully into the retiring generation, then the
+  physical drop removes that generation. Its caller sees a successful commit, but those writes are
+  intentionally discarded with the rest of the dropped column family; a same-name reopen creates
+  a fresh, empty generation. If the transaction writes to a transaction log, its entries are still
+  published; consumers must order the schema drop after those entries.
+- `open()` of a name whose previous generation is still held by an admitted commit waits for the
+  full admission-to-reclamation interval (bounded by `ROCKSDB_JS_CF_RECLAIM_WAIT_MS`, default
+  `30000`) before creating the fresh column family; if the previous generation's physical drop
+  failed, the open retries it once and throws with that error if it fails again.
+- A physical drop that fails (an I/O error writing the MANIFEST) keeps the name retired, is
+  retried on the next drop on the database, the next `open()` of that name, or close, and is
+  reported through the global `log.warn` event and the `columnFamily.pendingReclaims` stat. When
+  the failing drop was the one `drop()` itself ran, the call rejects with that error as well.
+
+What is **not** guaranteed: a process that exits while a physical drop is still pending, or
+after one failed, leaves the column family on disk under its name, and the next open of the
+database opens it as a live column family. A backup or checkpoint taken inside that window copies
+it. Callers that need a drop to survive a crash record their own durable tombstone before
+acknowledging it.
+
 ### `db.dropSync(): void`
 
-Synchronous version of `db.drop()`.
+Synchronous version of `db.drop()`, with the same deferral contract.
 
 ```typescript
 const db = RocksDatabase.open('path/to/db');
@@ -620,9 +767,9 @@ Flushes all in-memory data to disk asynchronously.
     memtable switch can itself prolong an existing L0 stop-trigger condition rather than clear it.
     That cost is database-wide, covering every column family on the (process-global,
     `worker_threads`-shared) database handle, not just the caller's — and it relocates the hang
-    rather than removing it: a stalled write blocks the database's single commit thread, which
-    dispatches every `Transaction.commit()` in order, so every commit behind it queues up too,
-    including ones from callers that never touched flush. This is a different knob from
+    rather than removing it: a stalled write blocks each of the database's commit threads that
+    reaches it, so every async `Transaction.commit()` behind them queues up too, including ones
+    from callers that never touched flush. This is a different knob from
     `writeBufferManagerAllowStall` (see [`new RocksDatabase()`](#new-rocksdatabasepath-options)
     options), with nearly opposite polarity: that one governs whether the `WriteBufferManager` may
     stall writers at all, this one governs whether one manual flush is willing to cause a stall
@@ -796,13 +943,47 @@ const range = db.getKeysCount({ start: 'a', end: 'z' }); // exact number of keys
 ### `db.getMonotonicTimestamp(): number`
 
 Returns the current timestamp as a monotonically increasing timestamp in milliseconds represented as
-a decimal number. This process-wide clock also supplies each transaction's initial timestamp.
+a decimal number. This process-wide clock supplies each transaction's initial timestamp, which is
+also the key of the transaction-log batch the transaction is written under.
 
 ```typescript
 const ts = db.getMonotonicTimestamp();
 console.log(ts); // 1764307857213.739
 ```
 
+It is a wall-clock (Unix epoch) value made strictly increasing: on a tie or a backward step of the
+host clock it advances by one floating-point ulp per call until the wall clock catches up. That
+keeps transaction timestamps ordered and durable, but it does not measure elapsed time — after a
+backward step, differences between two calls understate real time. Use
+[`steadyClockNow()`](#steadyclocknow-number) for elapsed durations and deadlines.
+
+#### Timestamp floor at open
+
+The clock is monotonic within a process, not across restarts: a new process reads the wall clock
+again, so a backward step between runs can reissue a timestamp that is already a batch key in this
+node's transaction log. Opening with
+[`timestampFloorLog`](#new-rocksdatabasepath-options) raises the clock
+above every batch key still durable in the named log before the database handle is returned, so no
+transaction can be constructed below it.
+
+Name only a log this process **originates**. A log written under a timestamp adopted from another
+node — a replication receiver calling [`txn.setTimestamp()`](#txnsettimestampts-number-void) — is
+keyed by that node's clock; seeding from it would ratchet this process's clock to the fastest of
+those nodes at every restart, and there is no way for the database to tell the two kinds of log
+apart on its own.
+
+The named floor is fail closed: an unreadable segment, incomplete scan, malformed framing, or a key
+more than ten years ahead of the wall clock refuses the opted-in open rather than reissuing a durable
+batch key. The floor is process-wide, so it applies to every database open in the process. The scan
+runs while the process serializes database opens and closes, so increasing its budget can delay
+unrelated opens and closes too.
+
+Naming an absent log warns so a fresh local log can be configured before its first write. The first
+use may create that fresh log, but if a nonempty directory appears after the warning it is refused:
+those keys were not part of the seed, and a transaction timestamp is already chosen before
+`txn.useLog()` could scan it. Close every handle for the path and reopen with `timestampFloorLog` to
+scan the restored log before using it.
+
 ### `db.getOldestSnapshotTimestamp(): number`
 
 Returns a number representing a unix timestamp of the oldest unreleased snapshot.
@@ -928,8 +1109,13 @@ Returns a new `ArrayBuffer` with two additional methods:
 - `cancel()` - Removes the callback; future `notify()` calls do nothing
 
 Note: If a shared buffer already exists for the given `key`, the returned `ArrayBuffer` will
-reference this existing shared buffer. Once all `ArrayBuffer` instances have gone out of scope and
-garbage collected, the underlying memory and notify callback will be freed.
+reference this existing shared buffer. The buffer lives as long as its column family is open (a
+`drop()` discards it with the column family — except the default column family, which `drop()`
+only clears, so its buffers survive): it is process-wide state, so a thread dropping or garbage
+collecting its own view (or exiting) never resets it for the others. Because nothing is ever
+evicted, key the buffer on a small fixed set of names rather than on unbounded data such as
+record ids. The notify callback is removed when the `ArrayBuffer` it was registered with is
+garbage collected, or by `cancel()`.
 
 ```typescript
 const buffer = new Uint8Array(db.getUserSharedBuffer('isDone', new ArrayBuffer(1)));
@@ -1585,9 +1771,7 @@ Set the algorithm per column family with the `compression` option when opening a
 const db = RocksDatabase.open('/path/to/db', { compression: 'zstd' });
 
 // Or an object with an explicit level
-const db2 = RocksDatabase.open('/path/to/db2', {
-	compression: { algorithm: 'zstd', level: 3 },
-});
+const db2 = RocksDatabase.open('/path/to/db2', { compression: { algorithm: 'zstd', level: 3 } });
 
 // Disable compression entirely
 const db3 = RocksDatabase.open('/path/to/db3', { compression: 'none' });
@@ -1604,16 +1788,6 @@ was compiled against, so it varies by build. Always check
 [`supportedCompression`](#supportedcompression) at runtime — opening with an unavailable algorithm
 throws. `'none'` is always available.
 
-**Changing the codec of data already written.** Setting `compression` affects files written from
-that point on. Existing SST and blob files keep the codec they were written with until something
-rewrites them. An ordinary `compact()` can re-encode non-bottommost levels, but RocksDB leaves the
-bottommost level alone unless a compaction filter is installed — and that is where most of the data
-sits, so a plain compaction will not rewrite all existing data. Use
-[`compact({ bottommost: true })`](#dbcompactoptions-promisevoid) to force the rewrite, once per
-column family you want migrated. Note also that omitting `compression` on a column family that
-already exists **inherits** its current codec rather than applying the default — the default
-applies only when the family is created.
-
 **Adopting a codec across a whole database.** Compression is chosen per column family, and RocksDB
 opens every family of a database in one call — so by default the families you did not name keep
 their persisted algorithm. If your first open targets some other family (a catalog, say), the rest
@@ -1643,10 +1817,90 @@ changeable, so an explicit change governs files written afterward while existing
 their original compression until rewritten by compaction. It also applies to blob files (large
 values), whose compression otherwise defaults to none.
 
-Because a column family's compression is fixed while it is open, if the same column family is opened
-a second time in the same process (another `RocksDatabase` on the same path/`name`, including from a
-`worker_thread`) with an **explicitly different** algorithm or level, the second open **throws** — a
-plain reopen (no `compression`) instead inherits the live setting.
+Because an open-time `compression` option applies only when a column family is first opened in a
+process, if the same column family is opened a second time in the same process (another
+`RocksDatabase` on the same path/`name`, including from a `worker_thread`) with an **explicitly
+different** algorithm or level, the second open **throws** — a plain reopen (no `compression`) instead
+inherits the live setting. This is a structural limit of the open-time API: `rocksdb::DB::Open()` opens
+every column family in the database transitively in one call, so by the time a caller can name a
+newly-discovered column family explicitly, RocksDB has already opened it. Use `db.setCompression()`
+below to change the algorithm of an already-open column family without a reopen.
+
+### `db.setCompression(compression: string | { algorithm: string, level?: number }): void`
+
+Dynamically changes the compression algorithm (and optional level) for an **already-open** column
+family, live, with no close/reopen and no conflict with other handles that already have this column
+family open elsewhere in the process. It accepts the same `compression` shape as the open-time option:
+
+```typescript
+const db = RocksDatabase.open('/path/to/db', { compression: 'none' });
+
+// ... database is upgraded/reconfigured to prefer zstd ...
+db.setCompression({ algorithm: 'zstd', level: 19 });
+// new writes are now compressed with zstd/level 19; db is still open and usable.
+```
+
+This is backed by RocksDB's `DB::SetOptions()`, which the RocksDB headers document as dynamically
+mutable for exactly the fields this binding needs: `ColumnFamilyOptions::compression` and
+`AdvancedColumnFamilyOptions::blob_compression_type` are both annotated `// Dynamically changeable
+through SetOptions() API` (`rocksdb/options.h`, `rocksdb/advanced_options.h`). `setCompression` sets
+both together (mirroring the open-time option) so blob-stored large values don't stay uncompressed
+after a live algorithm change. It was chosen over a per-CF options map on the root/default open call
+(the alternative design considered) because that alternative doesn't fit the motivating use case at
+all: a caller that doesn't yet know a column family's desired codec until _after_ reading its own
+catalog — itself a column family opened by that very same call — has no way to supply per-CF options
+up front. `SetOptions()` sidesteps the problem structurally: it operates on a column family that is
+already open, after the caller has had a chance to learn what codec it wants.
+
+**Verified semantics** (RocksDB's own `SetOptions()` doc explicitly limits the guarantee to what's
+listed as mutable — this was checked directly against the linked RocksDB build, not assumed):
+
+- **Governs new writes only, going forward.** The very next flush (memtable → SST) and any future
+  compaction output use the new algorithm; SST/blob files already on disk are untouched by the call
+  itself and keep their old compression until they are naturally rewritten by a later compaction.
+  This exactly matches the open-time `compression` option's existing "governs subsequently written
+  files" semantics — `setCompression` is the live-mutation path to the same effective behavior,
+  without the reopen restriction above. An ordinary `db.compact()`/`db.compactSync()` can leave
+  already-bottommost files untouched because RocksDB skips that level by default when no compaction
+  filter is configured. Use `db.compact({ bottommost: true })` or
+  `db.compactSync({ bottommost: true })` to force those files to be rewritten under the new codec.
+- **The level is a partial update.** The new level is applied via RocksDB's nested-option syntax
+  (`compression_opts={level=N;}`), which — per RocksDB's own documented example for this same API
+  (`db->SetOptions(cfh, {{"block_based_table_factory", "{prepopulate_block_cache=kDisable;}"}})`)
+  — only touches the field(s) named; every other `CompressionOptions` sub-field (window size,
+  strategy, dictionary training, ...) is left at its current live value. Omitting `level` resets it to
+  the algorithm's default (RocksDB's `kDefaultCompressionLevel` sentinel), matching the open-time
+  option's "an explicit algorithm without a level does not inherit a previously-set level" behavior —
+  it is never silently carried over from before the call.
+- **A persisted no-op request is free.** If the requested algorithm, blob algorithm, and effective
+  level already match the live options, `setCompression` skips the RocksDB call entirely rather than
+  paying for a no-op OPTIONS-file write. If a prior attempt applied the setting in memory but failed to
+  persist it, the same request retries the persistence step instead. This matters because RocksDB's
+  own `SetOptions()` doc calls it "a slow call because a new OPTIONS file is serialized and persisted
+  for each call. Use only infrequently" — a caller that reconciles many column families against a
+  catalog on every boot will find most of them already at the desired codec, and this keeps that sweep
+  cheap.
+- **A persistence failure can split live and durable state.** RocksDB applies the new options in
+  memory first, then persists an OPTIONS file reflecting them; the returned status covers the persist
+  step alone. On a failure (e.g. the volume is full or read-only), the column family may already be
+  running the new algorithm in memory while the on-disk OPTIONS file still holds the old one —
+  `setCompression` detects this case and says so explicitly in the thrown error's message, since a
+  later cold reopen (which trusts only the OPTIONS file) will silently revert to the old algorithm.
+- **Rejected on a read-only database.** `setCompression` persists a new OPTIONS file, which a
+  read-only handle (`{ readOnly: true }`) may not do — it throws `ERR_DATABASE_READONLY`, matching
+  every other mutating operation on this binding.
+- **Read your write.** `db.compression` reflects the change immediately (it reads live via
+  `DB::GetOptions()`), and so does the already-open-column-family conflict check described above: once
+  `setCompression` has changed the live algorithm, a second explicit `RocksDatabase.open()` on that
+  column family is compared against the **new** value, not the one it was originally opened with.
+- **Overrides another handle's explicit codec without notice.** The open-time conflict check exists
+  so two handles on the same column family can't disagree about compression — but `setCompression`
+  bypasses it by design: it changes the live setting for every handle on that column family, including
+  one that originally opened with an explicit, different algorithm. There is currently no event or
+  other signal to such a handle when this happens.
+- **Does not change `RocksDatabase.open`'s existing open-time behavior.** `compression` at open time
+  behaves exactly as before for a column family that is not already open in the process; `setCompression`
+  is purely additive for the already-open case.
 
 ### `supportedCompression`
 
@@ -1808,8 +2062,12 @@ intended for read-heavy workloads where records carry a monotonically increasing
 Each record's version is the numeric value stored in the first 8 bytes of its value (interpreted as
 a big-endian float64). The table maps `(database, column family, key)` to a single 8-byte slot that
 holds the last-known version for that key. Because slots are addressed by a hash, distinct keys may
-share a slot; a collision only ever causes a conservative miss (a real read), never a stale value to
-be treated as fresh.
+share a slot. A slot stores the version combined with a tag derived from its key, so a version that
+another key cached in a shared slot — even the same version, as keys written in one transaction
+often carry — answers fresh only if the two keys' version-and-tag combinations coincide; otherwise
+the collision costs only a conservative miss (a real read). With the default 131,072 slots that
+coincidence has a probability of 2^-47 per lookup that finds another key's version in its slot, and
+each doubling of `verificationTableEntries` doubles it.
 
 This first word is the only version the table derives on its own: it is what a read with
 `populateVersion: true` publishes and what a transaction write invalidates against. A producer whose
@@ -1889,13 +2147,18 @@ Transaction log files are read through read-only memory maps. Understanding how
 with system memory helps when interpreting [`log.getStats()`](#loggetstats-transactionlogstats)
 figures and process memory usage:
 
-- **Maps are created lazily and live until the file is closed.** Writing log entries does not map
-  anything; a log file is mapped the first time a `log.query()` reads from it. The native layer
-  holds each file's map for the life of the file — it is released when the log file is purged or
-  the database is closed, not by garbage collection. JS `Buffer` views over the map (including
-  `entry.data`) hold an additional reference, so the underlying memory is never unmapped while a
-  view is still reachable. `stats.memory.activeMaps` counts the maps currently held by the native
-  layer.
+- **Maps are created lazily; only the current (still-being-written) file's map is held natively.**
+  Writing log entries does not map anything; a log file is mapped the first time a `log.query()`
+  reads from it. The native layer holds the current file's map for as long as it stays current —
+  released when the log file rotates, is purged, or the database is closed. On POSIX, once a file
+  rotates out or is purged, the native layer keeps only a weak reference: the mapping survives
+  solely through JS `Buffer` views over it (including `entry.data`) and is released by garbage
+  collection once none remain, independent of purge or close, and `stats.memory.activeMaps` counts
+  only the current file's strongly-held map — a frozen file's map is weakly held and not counted,
+  even while a JS `Buffer` view keeps it alive. **On Windows this weak-ownership optimization does
+  not apply**: every frozen read re-creates the mapping and re-pins it strongly for the rest of the
+  file's life, so `stats.memory.activeMaps`/`mappedBytes` count every frozen map a process has read,
+  not just the current file's.
 - **Mapped bytes are virtual, not resident.** Creating a map reserves address space only. A page
   consumes physical RAM (RSS) when it is first read (demand paging). Querying a multi-gigabyte log
   can show `memory.mappedBytes` in the gigabytes while actual memory usage barely moves.
@@ -1911,9 +2174,10 @@ figures and process memory usage:
 OS-specific differences:
 
 - **POSIX (Linux and macOS):** The active write file is mapped at the full configured
-  `transactionLogMaxSize` (an anonymous reservation with the file's contents overlaid on top), so
-  `memory.mappedBytes` over-reports the active file; `memory.overlayBytes` is the file-backed
-  portion and is the closer proxy for real consumption.
+  `transactionLogMaxSize` (an anonymous reservation with the file's contents overlaid on top), or
+  at its size when a single transaction pushed it past that target, so `memory.mappedBytes`
+  over-reports the active file; `memory.overlayBytes` is the file-backed portion and is the closer
+  proxy for real consumption.
 - **macOS:** Activity Monitor's "Memory" column reports the physical footprint, which excludes
   clean file-backed pages — mapped log data is essentially invisible there even when resident. Use
   process RSS (e.g. `process.memoryUsage().rss`, `ps`, or `vmmap <pid>`) to observe it.
@@ -2212,6 +2476,51 @@ Options:
 Validating a store that is being actively appended to can spuriously report a torn tail for the
 current log file — the tail of an in-flight append is indistinguishable from a crash artifact.
 
+### `steadyClockNow(): number`
+
+Reads the process-wide steady clock. No database handle is needed.
+
+```typescript
+import { steadyClockNow } from '@harperfast/rocksdb-js';
+
+const start = steadyClockNow();
+// ... work, possibly on other worker threads ...
+const elapsedMs = steadyClockNow() - start;
+```
+
+Contract:
+
+- **Units / type**: milliseconds with a fractional part, as a `number`. Read from
+  `std::chrono::steady_clock` (`CLOCK_MONOTONIC` on Linux, `CLOCK_MONOTONIC_RAW` on macOS,
+  `QueryPerformanceCounter` on Windows).
+- **Origin and lifetime**: the origin is unspecified and fixed for the life of the process. Every
+  sample taken in the process is in one domain — the main thread and every `worker_threads` worker,
+  including workers started or restarted at any later time, under Node, Bun and Deno alike. Samples
+  are not meaningful across processes or restarts, and are unrelated to the Unix epoch: never compare
+  them with `Date.now()`, `db.getMonotonicTimestamp()` or transaction timestamps.
+- **Monotonic, not unique**: a sample taken after another (in real time, on any thread) is `>=` it.
+  Two samples can be equal — in the same clock tick, or when two distinct readings round to the same
+  double — so treat `==` as "not later" and pair the sample with a sequence number if strict ordering
+  is required. On Windows, QPC samples from different threads within ±1 native counter tick also
+  have ambiguous ordering; do not use near-equal clock samples alone to prove causality. No
+  wall-clock ratchet is applied.
+- **Precision**: the double's spacing grows with distance from the origin: about 2 ns at 100 days,
+  61 ns at 10 years, 0.49 µs at 100 years. Conversion error is below 0.51 µs per sample through
+  100 years from the native origin. This is representation precision, not clock accuracy. Rounding is
+  monotone, so distinct readings can only collapse to equality, never invert. The clock's own
+  resolution is platform-defined (the C++ duration period is 1 ns on these platforms; actual clock
+  resolution can be coarser). The native signed 64-bit nanosecond range is about 292 years in either
+  direction from its origin; conversion to `number` does not narrow it.
+- **Independent of the wall clock**: `settimeofday`/NTP steps in either direction do not move it. NTP
+  frequency slew is platform-defined (`CLOCK_MONOTONIC` is slewed; `CLOCK_MONOTONIC_RAW` and
+  `QueryPerformanceCounter` are not).
+- **Suspend**: whether time the host spends suspended counts is platform-defined — `CLOCK_MONOTONIC`
+  excludes it on Linux; current macOS `CLOCK_MONOTONIC_RAW` and Windows QPC include it. Do not
+  assume runtime timers (`setTimeout`) use the same suspend policy. Portable suspend-inclusive
+  deadlines are not provided by this API.
+- **Cost**: one clock read and a double conversion; no lock, no native allocation, no wall-clock read,
+  no database or log I/O, no per-worker calibration. Safe from any thread. The runtime may allocate a boxed JS number.
+
 ### `currentThreadId(): number`
 
 Returns the current thread ID.
@@ -2221,33 +2530,49 @@ import { currentThreadId } from '@harperfast/rocksdb-js';
 console.log(currentThreadId());
 ```
 
-### `registryStatus(): RegistryStatus`
+### `getRegistryStatus(): RegistryStatus`
 
 Returns an array containing that status of all active RocksDB instances.
 
 - `path: string` The database path.
+- `closeError?: string` The native lifecycle error retaining this registry entry.
+- `destroyCleanupPending?: boolean` The native database is closed, but physical path cleanup must
+  finish before the next open. Call `destroy()` to retry cleanup.
 - `refCount: number` The number of JavaScript database instances plus the registry's reference.
 - `columnFamiles: object` A map of column family names and their their info.
-  - `userSharedBuffers: number` The count of active user shared buffers.
+  - `userSharedBuffers: number` The count of user shared buffer keys created since the column family
+    was opened; entries are never evicted, so this never decreases.
 - `transactions: number` The count of active transactions.
 - `closables: number` The count of active database, transactions, and iterators.
 - `locks: number` The count of active locks.
 - `listenerCallbacks: number` The count of in-flight callbacks.
 
 ```typescript
-import { registryStatus } from '@harperfast/rocksdb-js';
-console.log(registryStatus());
+import { getRegistryStatus } from '@harperfast/rocksdb-js';
+console.log(getRegistryStatus());
 ```
 
 ### `shutdown(): void`
 
 The `shutdown()` will flush all in-memory data to disk and wait for any outstanding compactions to
 finish, for all open databases. It is highly recommended to call this in a `process` `exit` event
-listener (on the main thread), to ensure that all data is flushed to disk before the process exits:
+listener (on the main thread), to ensure that all data is flushed to disk before the process exits.
+It throws the first close failure after attempting every claimed database; call it again to retry
+any descriptor whose native teardown did not complete. It reports pending destroy-cleanup
+tombstones without deleting their paths; retry those with an explicit `destroy()`.
+
+Wrap the call: an exception thrown from an `exit` listener skips every `exit` listener registered
+after it, and sets the exit code to 1 unless an `uncaughtException` handler is installed.
 
 ```typescript
 import { shutdown } from '@harperfast/rocksdb-js';
-process.on('exit', shutdown);
+process.on('exit', () => {
+	try {
+		shutdown();
+	} catch (error) {
+		console.error('rocksdb-js shutdown failed', error);
+	}
+});
 ```
 
 ### `versions: { 'rocksdb': string; 'rocksdb-js': string }`
@@ -2339,9 +2664,7 @@ live database directory would write backup files on top of RocksDB's own files.
 before anything is written.
 
 ```typescript
-const id = await db.backup('/path/to/backups', {
-	metadata: 'nightly-2026-06-04',
-});
+const id = await db.backup('/path/to/backups', { metadata: 'nightly-2026-06-04' });
 ```
 
 `BackupOptions`:
@@ -2391,9 +2714,7 @@ await db.backup(Writable.toWeb(createWriteStream('/path/to/backup.tar')));
 // Restore: `tar -xf backup.tar -C /restored`, then open '/restored'.
 
 // Or gzip it (`tar -xzf backup.tar.gz` to restore):
-await db.backup(Writable.toWeb(createWriteStream('/path/to/backup.tar.gz')), {
-	gzip: true,
-});
+await db.backup(Writable.toWeb(createWriteStream('/path/to/backup.tar.gz')), { gzip: true });
 ```
 
 `BackupStreamOptions`:
@@ -2671,7 +2992,7 @@ Available commands:
 
 ## Development
 
-This package requires Node.js 18 or higher, pnpm, and a C++ compiler.
+This package requires Node.js `^22.18.0 || >=24.0.0`, pnpm, and a C++ compiler.
 
 > [!TIP]
 > Enable pnpm log streaming to see full build output: