@harperfast/rocksdb-js 2.12.0 → 2.14.0

package/README.md

@@ -46,6 +46,32 @@ Creates a new database instance.
 - `path: string` The path to write the database files to. This path does not need to exist, but the
   parent directories do.
 - `options: object` [optional]
+  - `arenaBlockSize: number` The size in bytes of the blocks a column family's memtable allocates
+    from. Every column family that has been written to holds at least one block, and with a
+    `writeBufferManagerSize` set, those blocks count against that shared budget even when the
+    column family is idle — so with many column families, large blocks can use up most of a small
+    budget and cause constant tiny flushes. Defaults to `0`, which picks a size automatically: with
+    a write buffer manager, `writeBufferManagerSize / 2048`, but at least 64 KiB and at most
+    `writeBufferSize / 8` or 1 MiB, whichever is smaller (a 64 MiB budget gives 64 KiB, 512 MiB
+    gives 256 KiB, 2 GiB or more gives 1 MiB); without one, RocksDB's own default of
+    `writeBufferSize / 8` capped at 1 MiB (1 MiB at the default `writeBufferSize`). The automatic
+    size uses the manager's budget at the time the column family is opened. An explicit value is
+    used as given, except that RocksDB raises one below 4 KiB to 4 KiB. Like `writeBufferSize`,
+    this applies to every column family when a database is first opened in the process and to
+    column families created later; a column family that is already open keeps its size.
+  - `bloomFilterBitsPerKey: number` The size of the filter written into each SST file, in bits per
+    key. A `get()` of a key that is not in a file checks that file's filter and skips its index and
+    data blocks, so reads of absent keys, and of keys whose range overlaps several files (keys
+    written in random order, or updates still in level 0), touch fewer blocks. Filters cover whole
+    keys only, so range reads are unaffected. Defaults to `10` (about 1% false positives); otherwise
+    a number from 1 to 100, fractions allowed, or `0` for no filters. Each open SST file holds its
+    filter in memory beside its index, outside the block cache (about 1.25 bytes per key at `10`;
+    a `secondaryPath` open holds every file open, so the whole database's filters). `0` also stops
+    an open from loading the filters files already have, so a reader can trade them for memory. An existing
+    database opens with the new setting, but its files gain or lose filters only as compaction
+    rewrites them (`compact({ bottommost: true })` rewrites them all). Like `writeBufferSize`, this
+    applies to every column family when a database is first opened in the process and to column
+    families created later; a column family that is already open keeps its setting.
   - `compression: string | { algorithm: string, level?: number }` The block/blob compression
     algorithm for this column family. Pass an algorithm name — one of `'none'`, `'snappy'`,
     `'zlib'`, `'bzip2'`, `'lz4'`, `'lz4hc'`, or `'zstd'` — or an object with an
@@ -119,9 +145,10 @@ Creates a new database instance.
   - `readOnly: boolean` When `true`, the database is opened in read-only mode. Read operations are
     permitted. Write operations will throw an error with code `ERR_DATABASE_READONLY`. Transactions
     are a no-op in read-only mode. [`flush()`](#dbflushoptions-promisevoid),
-    [`flushSync()`](#dbflushsyncoptions-void), [`compact()`](#dbcompactoptions-promisevoid) and
-    [`compactSync()`](#dbcompactsyncoptions-void) are the exception: they have nothing to do when
-    there are no writes, so they succeed as no-ops rather than throwing.
+    [`flushSync()`](#dbflushsyncoptions-void), [`compact()`](#dbcompactoptions-promisevoid),
+    [`compactSync()`](#dbcompactsyncoptions-void) and [`syncWal()`](#dbsyncwal-promisevoid) are the
+    exception: they have nothing to do when there are no writes, so they succeed as no-ops rather
+    than throwing.
 
     A read-only open is a **point-in-time snapshot**: it never sees later writes. It is safe
     against a quiescent database, but **unsafe against a live writer**: the open holds no reference
@@ -188,11 +215,14 @@ Creates a new database instance.
     nonempty directory that reappears before first use is refused until every handle closes and the
     path reopens with this option. The name must be a single path component that does not start with
     `.`: startup discovery skips such directories, so the log could never be scanned again.
-  - `verificationTable: boolean` When `true`, this column family participates in the process-global
-    [Verification Table](#verification-table): transaction writes to this column family invalidate
-    the verification slot for each written key. Enable this only for column families whose records
-    are cached (e.g. the primary column family of a table). Defaults to `false`. Requires
-    `verificationTableEntries` to be configured before the first database is opened.
+  - `verificationTable: boolean` When `true`, this handle participates in the process-global
+    [Verification Table](#verification-table): writes through it invalidate the verification slot
+    for each written key, and reads through it may seed slots. Enable this only for column families
+    whose records are cached (e.g. the primary column family of a table). Defaults to `false`; a
+    handle that leaves it unset joins a column family another handle opened with `true`, and an
+    explicit `false` throws while any handle on that column family participates (see
+    [Verification Table](#verification-table)). Requires `verificationTableEntries` to be configured
+    before the first database is opened.
   - `writeBufferSize: number` The per-column-family memtable size in bytes at which the memtable is
     sealed and flushed to an SST file. Smaller values produce more frequent, faster flushes; larger
     values batch more writes per SST file at the cost of memory. Defaults to `16777216` (16 MB).
@@ -200,13 +230,16 @@ Creates a new database instance.
 ### `db.close()`
 
 Closes a database. This function can be called multiple times and will only close an opened
-database. A database instance can be reopened once it is closed. A flush failure leaves the native
-database quarantined so `shutdown()` can retry without losing unflushed data; an explicit
-`destroy()` can instead delete it. A failure while waiting for compaction to settle is reported
-after native teardown completes; the optional `compactOnClose` pass itself is best-effort and its
-errors do not fail the close, since a skipped compaction loses no data. All native close errors
-emit `database:closeFailed`. The quarantine applies to both writable and read-only opens because
-both modes share the physical path lifecycle.
+database. A database instance can be reopened once it is closed. Closing the last handle flushes the
+memtables and waits for that flush, then cancels background compaction instead of waiting for it:
+a running compaction is abandoned and redone later, so close does not wait for the compaction
+backlog (the flush itself still waits out a write stall). A flush failure leaves the native database quarantined so `shutdown()`
+can retry without losing unflushed data; an explicit `destroy()` can instead delete it. A failure
+from RocksDB's own final close is reported after native teardown completes; the optional
+`compactOnClose` pass still runs to completion before background work is cancelled, but it is
+best-effort and its errors do not fail the close, since a skipped compaction loses no data. All
+native close errors emit `database:closeFailed`. The quarantine applies to both writable and
+read-only opens because both modes share the physical path lifecycle.
 
 ```typescript
 const db = RocksDatabase.open('foo');
@@ -276,6 +309,21 @@ Sets global database settings.
     callers could already produce it. With `1`, a database's async commits run one at a time in
     the order `commit()` was called. Databases already open keep their limit.
   - `compactOnClose: boolean` When `true`, compacts the database on close. Defaults to `false`.
+  - `shutdownOnExit: boolean` When `true`, the main thread runs [`shutdown()`](#shutdown-void)
+    from a `process.on('exit')` listener, so `process.exit()` still flushes and closes every
+    database. Defaults to `true`. Set to `false` to leave memtables unflushed at a Node
+    `process.exit()`, e.g. to exercise crash recovery.
+  - `transactionLogAbortMarkers: boolean` When `true`, a transaction that wrote its
+    transaction-log batch and then ends without committing (an `abort()`, a commit error it gives
+    up on, or a worker torn down with it pending) gets that batch marked aborted in place, and every
+    reader skips it; see [Abort markers](#abort-markers). Defaults to `false`, which behaves as
+    before. Read when a transaction starts and when it is reset for a retry.
+  - `transactionLogWriteIntents: boolean` When `true` with `transactionLogAbortMarkers` on, a
+    conflicting attempt is refused before it writes its batch when an older transaction holds a
+    write intent on one of its keys; see [Abort markers](#abort-markers). This shortens the worst
+    waits of transactions that overlap heavily, but adds retries and costs throughput when
+    transactions are large or contention is low. Defaults to `false`. Needs `verificationTable`.
+    Read with `transactionLogAbortMarkers`.
   - `occValidation: 'parallel' | 'serial'` How writable optimistic databases opened after the call
     validate commits for conflicts. Defaults to `'parallel'` (RocksDB's default). Databases already
     open keep their policy. Pessimistic, read-only and secondary opens are unaffected.
@@ -328,6 +376,19 @@ Sets global database settings.
     [`db.isWriteStalled()`](#dbiswritestalled-boolean) — see
     [`getWriteBufferManagerStats()`](#getwritebuffermanagerstats-writebuffermanagerstats),
     which also describes the watchdog that logs a sustained stall.
+  - `writeBufferManagerFlushPolicy: 'oldest' | 'largest' | 'largestAcrossDBs'` What the write
+    buffer manager flushes when it is over budget. `'oldest'` flushes the oldest memtable of the
+    database that is being written; `'largest'` its largest memtable (databases here are opened with
+    atomic flush, so this currently behaves like `'oldest'`); `'largestAcrossDBs'` has a background
+    thread flush whichever database holds the most memtable memory, whether or not it is being
+    written. When not set, the policy is `'largestAcrossDBs'` while
+    `writeBufferManagerAllowStall` is `true` and `'oldest'` otherwise, and it follows later changes
+    to `writeBufferManagerAllowStall`. With stalls allowed, `'oldest'` can stop writes permanently:
+    a write to one database waits for memory held by other databases that nothing is writing to,
+    so nothing flushes them. Without stalls, `'largestAcrossDBs'` has measured slower. Can be
+    changed while databases are open. Once set, the value stays in effect for the life of the
+    process; it is reported as `flushPolicy` by
+    [`getWriteBufferManagerStats()`](#getwritebuffermanagerstats-writebuffermanagerstats).
   - `writeBufferManagerCostToCache: boolean` When `true`, memtable memory is "charged" against the
     shared block cache so the block cache and write buffers draw from a single pool. During write
     bursts the cache shrinks to make room for memtables; once memtables flush, the cache can grow
@@ -370,6 +431,8 @@ where the call was made.
 - `mutableMemoryUsage: number` The share of `memoryUsage` held by active (mutable) memtables; the
   rest is memtables awaiting flush plus retained write history.
 - `allowStall: boolean`, `costToCache: boolean` The manager's configuration.
+- `flushPolicy: 'oldest' | 'largest' | 'largestAcrossDBs'` The flush policy in effect (see
+  `writeBufferManagerFlushPolicy`); before a manager exists, the policy one would be created with.
 - `stallActive: boolean` Whether the manager is currently stalling writes.
 - `stallActiveMs: number` How long the current stall has been active; `0` when not stalled.
   Sampled once a second by the watchdog, so `0` for a stall's first second and whenever
@@ -388,7 +451,7 @@ where the call was made.
 
 `bufferSize`, `memoryUsage`, `mutableMemoryUsage`, `stallActive` and `stallActiveMs` are also in
 [`db.getStats()`](#dbgetstatsall-boolean-rocksdbstats) and `db.getStat()` under the same
-`writeBufferManager.` prefix, for scraping; `enabled`, `allowStall`, `costToCache`,
+`writeBufferManager.` prefix, for scraping; `enabled`, `allowStall`, `costToCache`, `flushPolicy`,
 `watchdogRunning`, and the column-family inventory are only here, because collecting them either
 walks the database registry or isn't scrape-shaped.
 
@@ -674,7 +737,11 @@ because releasing the native database beneath an active copy would be unsafe.
 A previously opened instance does not need to remain open, which allows an explicit `destroy()`
 retry after failed physical cleanup. A never-opened or read-only instance cannot destroy the
 database. `shutdown()` reports a pending cleanup tombstone but never retries deletion; only an
-explicit `destroy()` can remove the path.
+explicit `destroy()` can remove the path. Destruction flushes the memtables before it closes the
+native database, without waiting out a write stall, so a database whose files outlive the call (the
+deletion fails, or the process dies first) keeps writes made with `disableWAL`. If that flush fails,
+destruction still deletes the database and reports the flush error through `database:closeFailed`;
+only when the deletion then fails too are the unflushed `disableWAL` writes lost.
 
 ```typescript
 db.destroy();
@@ -806,6 +873,22 @@ db.flushSync({ allowWriteStall: true });
 On a [read-only](#new-rocksdatabasepath-options) database it validates its options and then returns
 without flushing, rather than throwing.
 
+### `db.syncWal(): Promise<void>`
+
+Syncs the write-ahead log to stable storage (`DB::SyncWAL`), so every WAL-backed write committed
+before the call survives a power loss afterwards. Unlike [`flush()`](#dbflushoptions-promisevoid) it
+writes no SST file and never stalls writers. Writes that skipped the WAL (`disableWAL`) and
+transaction-log files, which keep their own durability, are not covered. RocksDB serializes
+concurrent syncs of one WAL, so keep one in flight per database rather than queueing them on the
+libuv threadpool.
+
+```typescript
+await db.syncWal();
+```
+
+On a [read-only](#new-rocksdatabasepath-options) database the returned promise resolves without
+syncing.
+
 ### `db.get(key: Key, options?: GetOptions): MaybePromise<any>`
 
 Retreives the value for a given key. If the key does not exist, it will resolve `undefined`.
@@ -964,7 +1047,9 @@ again, so a backward step between runs can reissue a timestamp that is already a
 node's transaction log. Opening with
 [`timestampFloorLog`](#new-rocksdatabasepath-options) raises the clock
 above every batch key still durable in the named log before the database handle is returned, so no
-transaction can be constructed below it.
+transaction can be constructed below it. Keys in segments retention has purged are not durable, and
+neither is a bound advertised to peers; carry those across a restart yourself with
+[`raiseTimestampFloor()`](#raisetimestampfloortimestamp-number-boolean).
 
 Name only a log this process **originates**. A log written under a timestamp adopted from another
 node — a replication receiver calling [`txn.setTimestamp()`](#txnsettimestampts-number-void) — is
@@ -1250,7 +1335,9 @@ The `attempt` parameter is the number of times the transaction has been retried.
 - `disableSnapshot?: boolean` Whether to disable snapshots. Defaults to `false`.
 - `maxRetries?: number` The maximum number of times to retry the transaction. Defaults to `3`.
 - `retryOnBusy?: boolean` Whether to retry the transaction if the commit fails with `IsBusy`.
-  Defaults to `true` when the transaction is bound to a transaction log, otherwise `false`.
+  Defaults to `true` when the transaction wrote a transaction-log batch or was refused before
+  writing one (`refusedBeforeLog`), otherwise `false`. With `false`, `db.transaction()` also
+  commits without `coordinatedRetry`, so a conflict fails at once instead of waiting first.
 
 ### Transaction Retry Logic
 
@@ -1263,7 +1350,45 @@ a bad state and the transaction will need to be retried. If the max retries is r
 transaction is not retried, a `ERR_TRANSACTION_ABANDONED` error will be thrown.
 
 Users should use the `attempt` transaction callback parameter to ensure duplicate transaction log
-entries are not added.
+entries are not added. `txn.logWritten()` reports whether the batch is already in the log: a commit
+that conflicts after writing it keeps it for the retry, while an attempt refused before writing it
+(`err.refusedBeforeLog`, with `transactionLogAbortMarkers`) leaves nothing and must stage it again.
+
+### Abort markers
+
+With `RocksDatabase.config({ transactionLogAbortMarkers: true })`, a transaction's log batch is
+never left live when the transaction does not commit:
+
+- A transaction that wrote its batch and then ends without committing has the batch marked aborted:
+  bit `0x02` (`TRANSACTION_LOG_ENTRY_ABORTED_FLAG`) is set on its first entry, and the change is
+  synced to disk before `abort()` returns or the batch stops holding back committed readers. Every
+  reader skips the batch. `abort()` then no longer throws `ERR_TRANSACTION_ABANDONED`.
+- If the marker cannot be written or synced, `abort()` throws `ERR_TRANSACTION_ABORT_UNRESOLVED`
+  (again on a second call), committed readers stay behind the batch, and the log refuses new batches
+  until a retry on the next batch, database flush or close marks it.
+- Under `occValidation: 'parallel'` (the default), a retried optimistic commit that RocksDB's own
+  conflict check shows would fail is refused before it writes its batch too, with
+  `refusedBeforeLog` set (needs the RocksDB build this package pins). A first attempt skips the
+  check, so commits that do not conflict pay nothing for it, and so does a retry whose batch an
+  earlier attempt already wrote.
+- With `transactionLogWriteIntents: true` as well, a commit that finds an older transaction holding
+  a write intent on one of its keys (with `verificationTable: true`) is refused before writing its
+  batch, with `refusedBeforeLog` set on the
+  error, or, if its batch is already written from an earlier attempt, before committing. With
+  `coordinatedRetry` it waits for that transaction first. `commitSync()` never waits: it throws, and
+  `transactionSync()` retries after a short backoff.
+- `txn.setParkDeadline(epochMs)` bounds those waits by the caller's own deadline.
+- `log.getStats().totals` reports `abortsMarked`, `abortMarkFailures`, and `watermarkPinnedUs` /
+  `pinnedTransactions`: how long logged batches held committed readers back after their first
+  conflict, until they committed or ended.
+- A backup waits for every batch it captures that has neither committed nor been aborted, and is
+  refused if one is still unresolved after `ROCKSDB_JS_BACKUP_UNRESOLVED_WAIT_MS` (default
+  `30000`).
+
+A reader older than the release that added aborted-batch support shows an aborted batch as live, so
+turn this on only once every reader of the logs (replicas, restores, repair tools) has it. A batch
+whose transaction was still unresolved when the process crashed is not marked, and is replayed as
+before.
 
 When `coordinatedRetry: true`, the retry behavior changes for `IsBusy` conflicts: instead of
 rejecting (or retrying immediately and potentially conflicting again), the commit waits until the
@@ -1297,6 +1422,12 @@ operations methods as the `RocksDatabase` instance plus:
 - `txn.commitSync()` Synchronously commits and closes the transaction.
 - `txn.getTimestamp(): number` Retrieves the transaction timestamp in milliseconds as a decimal. It
   defaults to a process-wide monotonic value assigned when the transaction was created.
+- `txn.lastConflictKind(): number` Why the most recent attempt was refused or retried, one of
+  `ConflictKind`: `none` (0), `intentBeforeLog` (1), `intentAfterLog` (2), `atCommit` (3),
+  `checkBeforeLog` (4).
+- `txn.logWritten(): boolean` Whether the transaction's batch is in the transaction log.
+- `txn.setParkDeadline(epochMs: number): void` Past this time a coordinated-retry wait resolves at
+  once. `0` clears it.
 - `txn.id: number` The read-only transaction ID. Transaction IDs are unique to the RocksDB database
   path, regardless the database name/column family.
 - `txn.setTimestamp(ts?: number): void` Overrides the transaction timestamp in milliseconds. If
@@ -1349,6 +1480,11 @@ Type: `number`
 The transaction ID, a positive integer no greater than `Number.MAX_SAFE_INTEGER`. Transaction IDs
 are unique to the RocksDB database path, regardless the database name/column family.
 
+A transaction belongs to the thread that created it. Although every `worker_threads` worker that
+opens the same path shares one id space, using a transaction by id from another worker (for example
+`{ transaction }` read options or `log.addEntry(data, txnId)`) throws `ERR_TRANSACTION_WRONG_ENV`:
+RocksDB transactions are not thread-safe.
+
 #### `txn.setTimestamp(ts?: number): void`
 
 Overrides the transaction timestamp in milliseconds since the Unix epoch. Replication receivers and
@@ -1937,6 +2073,10 @@ Attempts to acquire a lock for a given key. If the lock is available, the functi
 and the optional `onUnlocked` callback is never called. If the lock is not available, the function
 returns `false` and the `onUnlocked` callback is queued until the lock is released.
 
+A queued `onUnlocked` callback keeps the thread's event loop alive until it is called, even when the
+lock is held by another thread. A thread with a callback queued on a lock that is never released
+will not exit on its own; closing the database that queued the callback does not cancel it.
+
 When a database is closed, all locks associated to it will be unlocked.
 
 ```typescript
@@ -1977,7 +2117,9 @@ db.unlock('foo'); // false, already unlocked
 
 ### `db.withLock(key: Key, callback: () => void | Promise<void>): Promise<void>`
 
-Runs a function with guaranteed exclusive access across all threads.
+Runs a function with guaranteed exclusive access across all threads. The returned promise rejects
+with the error if the callback throws or its promise rejects; the lock is released either way. A
+pending `withLock()` keeps the thread's event loop alive until its callback has run.
 
 ```typescript
 await db.withLock('key', async () => {
@@ -2005,9 +2147,7 @@ await Promise.all([
 ]);
 ```
 
-Note: If the `callback` throws an error, Node.js suppress the error. Node.js 18.3.0 introduced a
-`--force-node-api-uncaught-exceptions-policy` flag which will cause errors to emit the
-`'uncaughtException'` event. Future Node.js releases will enable this flag by default.
+Note: Errors thrown by a `tryLock()` `onUnlocked` callback are discarded.
 
 ## File Locking
 
@@ -2084,10 +2224,12 @@ The freshness check works as follows:
 2. On a cold read, pass `{ populateVersion: true }` (or call `db.populateVersion()` afterward) to
    seed the slot with the version extracted from the value, so subsequent freshness checks hit.
 
-Transaction writes to a column family opened with `verificationTable: true` invalidate the slot for
-each written key at write time, so a stale version can never survive a write. This is also what
-enables [`coordinatedRetry`](#transactionoptions): a conflicting transaction parks on the slot and
-retries once the write intent is released.
+Writes through a handle opened with `verificationTable: true` invalidate the slot for each written
+key at write time, so a stale version cannot survive them, and a read seeds a slot only through
+such a handle. Once one handle opens a column family with the option, the handles on it that left
+the option unset participate too (see below). Invalidation is also what enables
+[`coordinatedRetry`](#transactionoptions): a conflicting transaction parks on the slot and retries
+once the write intent is released.
 
 The table is **process-global** and backed by a single shared structure, so versions populated on
 the main thread are visible to `worker_threads` workers and vice versa.
@@ -2096,7 +2238,8 @@ the main thread are visible to `worker_threads` workers and vice versa.
 
 The table must be sized before the first database is opened, via the
 [`verificationTableEntries`](#dbconfigoptions) config option (default `131072` slots = 1 MB; set to
-`0` to disable). Then opt-in per column family with the `verificationTable: true` open option:
+`0` to disable). Then open each cached column family with the `verificationTable: true` open
+option:
 
 ```typescript
 import { RocksDatabase } from '@harperfast/rocksdb-js';
@@ -2109,6 +2252,18 @@ const db = RocksDatabase.open('path/to/db', { verificationTable: true });
 Enable `verificationTable` only for column families whose records are cached (e.g. the primary
 column family of a table); enabling it adds per-write slot invalidation overhead.
 
+Participation applies to the whole column family within the process. Once a handle opens a column
+family with `verificationTable: true`, every later handle on it participates too, even one opened
+without the option, so a write through any handle invalidates its key's slot. This lasts until
+every handle on the database closes, or until the column family is opened with an explicit
+`verificationTable: false` after every participating handle on it has closed. While any handle on
+it participates, that explicit `false` throws `ERR_VERIFICATION_TABLE_CONFLICT`. Opening with
+`verificationTable: true` throws `ERR_VERIFICATION_TABLE_CONFLICT` while a non-participating handle
+is open on that column family: one opened before any opt-in, or with an explicit `false`. After
+such a handle closes, the same error is thrown while a transaction that existed before it closed
+is still pending, or while another operation on the database is running; retry the open once they
+finish.
+
 ### `db.verifyVersion(key: Key, version: number): boolean`
 
 Returns `true` when the verification table currently records `version` for `key` (in this database
@@ -2209,7 +2364,10 @@ every file after it is retained regardless of age; `purge.retainedUnflushedFiles
 While a backup with `transactionLogs` is copying a store's files, purges of that store are skipped
 (`purge.retentionPins` reports it). On a filesystem that rejects a directory fsync, the store's last
 file is kept, since `txn.state` could not be made durable before deleting it.
-Use `destroy: true` only to remove the store itself.
+Before deleting any file, the purge records the largest key it held in the log's purge watermark
+([`log.getPurgeWatermark()`](#loggetpurgewatermark-transactionlogpurgewatermark)) and makes that
+durable. Use `destroy: true` only to remove the store itself; a log recreated after it starts a new
+watermark history.
 
 - `options: object`
   - `before?: number` Remove transaction log files older than the specified timestamp, subject to
@@ -2269,6 +2427,8 @@ to disk right before the transaction is committed. You may add multiple enties p
 underlying architecture is thread safe.
 
 - `log.addEntry()`
+- `log.equals()`
+- `log.getPurgeWatermark()`
 - `log.path`
 - `log.query()`
 
@@ -2301,6 +2461,57 @@ Note that the `TransactionLog` class also has internal methods `_getMemoryMapOfF
 `_findPosition`, and `_getLastCommittedPosition` that should not be used directly and may change in
 any version.
 
+#### `log.equals(other): boolean`
+
+Returns `true` when `other` is a `TransactionLog` for the same log: the same database and the same
+log name. The database is compared by its resolved filesystem identity, so `db.useLog('foo')`,
+`txn.useLog('foo')` and a log from another handle opened with a different spelling of the same path
+all compare equal, and two databases opened through the same relative path from different working
+directories do not. The comparison still works after either database closes. Any other value
+compares unequal.
+
+```typescript
+const log = db.useLog('foo');
+log.equals(otherHandle.useLog('foo')); // true: same database and name
+log.equals(db.useLog('bar')); // false
+```
+
+A log destroyed with `purgeLogs({ destroy: true })` and recreated under the same name still compares
+equal to wrappers of the old one.
+
+#### `log.getPurgeWatermark(): TransactionLogPurgeWatermark`
+
+```typescript
+const log = db.useLog('foo');
+const entries = Array.from(log.query({ start: resumeKey, exclusiveStart: true }));
+const watermark = log.getPurgeWatermark();
+const complete = watermark.known && resumeKey >= watermark.purgedThrough;
+```
+
+Returns an upper bound on every entry key that retention or `db.purgeLogs()` has deleted from this
+log, or reports that no such bound can be given. A reader that needs every entry with a key above
+some position can be served from the log only when that position is at or above `purgedThrough`.
+
+- `{ known: true, purgedThrough: number }` `purgedThrough` is at or above every key of every file
+  deleted from the log, in whatever order the keys were written (0 when nothing was deleted). It is
+  made durable before each file is deleted, including by the retention that runs when a database
+  opens.
+- `{ known: false }` The log cannot vouch for what was deleted. This holds permanently for a log
+  created by a release without the watermark, one that file deletions bypassed (such as an older
+  release's retention), and one whose record cannot be read. It is also returned when the log's
+  store no longer exists, such as after `purgeLogs({ destroy: true })` until the next write.
+
+The call is synchronous. A read-only or secondary database reads the record from disk on each call.
+The result is a snapshot: a purge can run right after it. To trust what a read returned, read the log
+first and check the watermark afterwards. See
+[the purge watermark](docs/transaction-log.md#purge-watermark) for the rules.
+
+##### `TransactionLogPurgeWatermark`
+
+```typescript
+type TransactionLogPurgeWatermark = { known: true; purgedThrough: number } | { known: false };
+```
+
 #### `log.path: string`
 
 Returns the path to the transaction log store files.
@@ -2310,7 +2521,7 @@ const log = db.useLog('foo');
 console.log(log.path);
 ```
 
-#### `log.query(options?): IterableIterator<TransactionLogEntry>`
+#### `log.query(options?): TransactionLogIterator`
 
 Returns an iterable/iterator that streams all log entries for the given filter.
 
@@ -2319,6 +2530,11 @@ Returns an iterable/iterator that streams all log entries for the given filter.
   - `end?: string` The transction end timestamp.
   - `exclusiveStart?: boolean` When `true`, this will only match transactions with timestamps after
     the start timestamp.
+  - `until?: TransactionLogIterator` Stops before the unread position of another iterator from a log
+    that `equals()` this one, including an entry it has peeked. The bound is exclusive, compares
+    segment and offset before filtering, and works across rotation, even when this query starts past
+    it. It resumes when `until` advances. A closed iterator no longer bounds the query. An invalid or
+    foreign-log iterator throws `TypeError` at query creation.
   - `exactStart?: boolean` When `true`, this will only match and iterate starting from a transaction
     with the given start timestamp. Once the specified transaction is found, all subsequent
     transactions will be returned (regardless of whether their timestamp comes before the `start`
@@ -2335,6 +2551,32 @@ Returns an iterable/iterator that streams all log entries for the given filter.
     have been written to the log but are no longer considered committed if they were not flushed to
     disk.
 
+An unbounded query whose starting segment is temporarily unmappable reports done and retries
+its original position on the next pull.
+
+`iterator.peek()` returns the next entry without consuming it. Repeated peeks return the same
+result until `next()` consumes it once. A `done` result is not cached, so later appends remain
+visible. Errors propagate as from `next()`.
+
+With `until`, `done: true` means the reader has caught up to the bound as of that pull. Running
+out of readable entries short of the bound throws instead, so an unavailable segment cannot
+silently trigger a handoff. Exact-start queries take the index's stop-to-tail jump only when
+its target does not pass the handoff position.
+
+`iterator.close()` permanently ends the iterator and releases its cached entry and mapping.
+It is idempotent; later `next()` and `peek()` calls report done. Breaking out of `for...of`
+keeps the iterator resumable and does not close it. Destroying a log with
+`purgeLogs({ destroy: true })` invalidates its existing iterators; close them and create new
+iterators for a recreated log, including any `until` bounds.
+
+```typescript
+const live = log.query();
+const replay = log.query({ start: startTime, until: live });
+for (const entry of replay) deliver(entry);
+replay.close();
+// A later commit belongs to live even if its timestamp is earlier than startTime.
+```
+
 The iterator produces an object with the log entry timestamp and data.
 
 - `object`
@@ -2342,6 +2584,25 @@ The iterator produces an object with the log entry timestamp and data.
   - `timestamp: number` The entry timestamp used to collate entries by transaction.
   - `endTxn: boolean` This is `true` when the entry is the last entry in a transaction.
 
+These errors leave the iterator usable:
+
+- `TransactionLogUntilUnreachedError`: readable input ended before `until`. The cursor stays in
+  place; retry after the input becomes readable or the pending transaction commits. Properties:
+  `logId` and `position`, identifying where the read stopped.
+- `TransactionLogGapError`: the iterator reached segments it had not read yet and could not read
+  them, usually because retention purged them while the reader lagged. Their entries are not in
+  this log anymore, so a consumer that replicates the log should treat its copy as discontinuous
+  (for example, by resyncing from a full copy) rather than continuing. Properties: `logId` (the
+  last segment read), `resumeLogId` (the segment the next `next()` call reads), and
+  `unavailableLogCount`. A new query positioned within the retained segments does not report the
+  segments purged before its starting point.
+- `CorruptFrameError` (a `RangeError`): an entry's framing is broken. `resyncPosition` is where
+  valid framing resumes, or `undefined` when nothing follows the break.
+
+All three classes are exported from the package. After a gap or corrupt frame, calling `next()`
+again continues past the problem. Other errors, such as a failure to open a segment, do not move
+the iterator, so retrying them without a fix fails the same way.
+
 ```typescript
 const log = db.useLog('foo');
 const iter = log.query({});
@@ -2495,7 +2756,7 @@ Contract:
   `QueryPerformanceCounter` on Windows).
 - **Origin and lifetime**: the origin is unspecified and fixed for the life of the process. Every
   sample taken in the process is in one domain — the main thread and every `worker_threads` worker,
-  including workers started or restarted at any later time, under Node, Bun and Deno alike. Samples
+  including workers started or restarted at any later time, under Node and Bun alike. Samples
   are not meaningful across processes or restarts, and are unrelated to the Unix epoch: never compare
   them with `Date.now()`, `db.getMonotonicTimestamp()` or transaction timestamps.
 - **Monotonic, not unique**: a sample taken after another (in real time, on any thread) is `>=` it.
@@ -2521,6 +2782,55 @@ Contract:
 - **Cost**: one clock read and a double conversion; no lock, no native allocation, no wall-clock read,
   no database or log I/O, no per-worker calibration. Safe from any thread. The runtime may allocate a boxed JS number.
 
+### `raiseTimestampFloor(timestamp: number): boolean`
+
+Raises the process-wide monotonic timestamp clock — the clock behind
+[`db.getMonotonicTimestamp()`](#dbgetmonotonictimestamp-number), every transaction's initial
+timestamp and therefore every transaction-log batch key — so that every timestamp the process
+issues afterwards is strictly greater than `timestamp` (Unix epoch milliseconds). No database
+handle is needed.
+
+```typescript
+import { RocksDatabase, raiseTimestampFloor } from '@harperfast/rocksdb-js';
+
+// Before anything in this process starts a transaction:
+raiseTimestampFloor(persistedFloor);
+const db = RocksDatabase.open('/path/to/db', { timestampFloorLog: 'local' });
+```
+
+The clock is monotonic within a process, not across restarts, and
+[`timestampFloorLog`](#timestamp-floor-at-open) only restores keys still on disk. A bound this node
+has promised that is not in its log — a progress floor advertised to peers, or the largest key of
+segments retention has since purged — can only come back through this call. The caller names the
+value because only the caller knows which bounds it has promised.
+
+Contract:
+
+- **Returns** `true` if the clock was raised: `timestamp` was ahead of both the wall clock and every
+  timestamp already issued, as after a backward clock step. `false` if the clock was already at or
+  above it, which is the usual result at a restart. Raising is one-way; a lower value never moves
+  the clock back.
+- **Throws** a `TypeError` if `timestamp` is not a number, and a `RangeError` if it is not finite, is
+  not greater than `0`, is not less than `8.64e15`, or is more than ten years ahead of the wall
+  clock — the same bound `timestampFloorLog` applies to a durable key. An unusable persisted floor
+  therefore fails loudly rather than being ignored. The implausibility error includes the wall
+  clock it was checked against, since a host that booted with an unsynchronized clock trips it too.
+- **Process-wide**: it applies to every database and every `worker_threads` worker in the process,
+  but not to other processes. Each new process must call it again.
+- **Only issued timestamps**: a transaction constructed before the call keeps its timestamp, and
+  [`txn.setTimestamp(ts)`](#txnsettimestampts-number-void) can still adopt any valid timestamp,
+  including one below the floor. `txn.setTimestamp()` with no argument issues a fresh timestamp
+  above it.
+- **Not durable and not a fence**: the floor lives in memory only, and nothing stops a transaction
+  constructed concurrently with the call. Persist the floor yourself, and restore it before the
+  process starts any transaction it originates. It composes with `timestampFloorLog` (the clock takes
+  the larger of the two), so keep naming the local log unless the persisted floor already covers
+  every key written to it.
+- **Trusted input only**: the floor is not lowered until the process exits, so a value hours or years
+  ahead holds every timestamp that far ahead of wall time until the clock catches up — and keys
+  written in the meantime raise the next restart's seed with them. Pass only floors this node
+  persisted itself, never an unvalidated value received from another node.
+
 ### `currentThreadId(): number`
 
 Returns the current thread ID.
@@ -2554,27 +2864,41 @@ console.log(getRegistryStatus());
 
 ### `shutdown(): void`
 
-The `shutdown()` will flush all in-memory data to disk and wait for any outstanding compactions to
-finish, for all open databases. It is highly recommended to call this in a `process` `exit` event
-listener (on the main thread), to ensure that all data is flushed to disk before the process exits.
-It throws the first close failure after attempting every claimed database; call it again to retry
-any descriptor whose native teardown did not complete. It reports pending destroy-cleanup
-tombstones without deleting their paths; retry those with an explicit `destroy()`.
+The `shutdown()` will flush all in-memory data to disk and cancel outstanding background
+compactions, for all open databases. It throws the first close failure after attempting every
+claimed database; call it again to retry any descriptor whose native teardown did not complete. It
+reports pending destroy-cleanup tombstones without deleting their paths; retry those with an
+explicit `destroy()`.
+
+You do not need to call it at exit. When rocksdb-js loads on the main thread, it registers a
+`process.on('exit')` listener that runs the same shutdown, so a `process.exit()` (which skips the
+native cleanup Node runs on a natural exit) still flushes and closes every database, including
+those opened by workers that are still running. That listener reports failures to stderr instead
+of throwing, and sets `process.exitCode` to 1 (unless it is already nonzero) when data may be left
+unflushed: a database another thread was still closing when the shutdown timed out, or a final
+flush that failed. Afterwards opening a database throws. It
+still runs when an earlier `exit` listener throws. It does not run when the process is killed by a
+signal it does not handle.
+
+Like the cleanup on a natural exit, it waits for native work already in flight (a commit, a flush
+waiting out a write stall, or a backup) without a time limit, so `process.exit()` returns only once
+that work finishes.
 
-Wrap the call: an exception thrown from an `exit` listener skips every `exit` listener registered
-after it, and sets the exit code to 1 unless an `uncaughtException` handler is installed.
+Because it is registered when rocksdb-js loads, `exit` listeners added later run after databases
+are closed. An `exit` listener that still writes to a database must run first:
 
 ```typescript
-import { shutdown } from '@harperfast/rocksdb-js';
-process.on('exit', () => {
-	try {
-		shutdown();
-	} catch (error) {
-		console.error('rocksdb-js shutdown failed', error);
-	}
+process.prependListener('exit', () => {
+	db.putSync('lastSeen', Date.now());
 });
 ```
 
+Set `RocksDatabase.config({ shutdownOnExit: false })` to skip it, for example to leave memtables
+unflushed at `process.exit()` and exercise crash recovery in a test. The listener stays registered
+and does nothing: on a natural exit the runtime's native cleanup still flushes, and whether Bun's
+does at `process.exit()` depends on the Bun release.
+Calling `shutdown()` from your own `exit` listener as well is safe.
+
 ### `versions: { 'rocksdb': string; 'rocksdb-js': string }`
 
 Returns the `rocksdb-js` and RocksDB version.
@@ -2899,11 +3223,13 @@ Options for `get()`, `getSync()`, and the `getBinary*` methods.
   - `expectedVersion: number` When set, the [Verification Table](#verification-table) is checked
     before reading. If the slot holds this version, the read is skipped and the
     `FRESH_VERSION_FLAG` sentinel is returned. After a database read, the slot is seeded with the
-    version extracted from the value. Requires the column family to be opened with
+    version extracted from the value. Requires the handle to be opened with
     `verificationTable: true`.
   - `populateVersion: boolean` When `true`, after a database read the verification-table slot is
     seeded with the version extracted from the value, eliminating the need for a separate
-    `db.populateVersion()` call on cold reads. Defaults to `false`.
+    `db.populateVersion()` call on cold reads. Requires the handle to participate in the
+    [Verification Table](#verification-table); otherwise the read seeds nothing. Defaults to
+    `false`.
   - `skipDecode: boolean` When `true`, the value is returned without being decoded. Defaults to
     `false`.