@harperfast/rocksdb-js 2.7.1 → 2.9.0

package/README.md

@@ -11,6 +11,8 @@ A Node.js binding for the RocksDB library.
 - Custom stores provide ability to override default database interactions
 - Efficient binary key and value encoding
 - Configurable block/blob compression (LZ4, Zstd, Zlib, and more)
+- Observable background errors via the `'error'` event or `db.getLastError()`, with in-process
+  recovery (`db.resume()`)
 - Access to internal RocksDB statistics
 - Designed for Node.js and Bun on Linux, macOS, and Windows
 
@@ -53,6 +55,16 @@ Creates a new database instance.
     when linked, else no compression). See [Compression](#compression). Throws if the algorithm is
     not compiled into the native build — check [`supportedCompression`](#supportedcompression) for
     the available list.
+  - `compressionForAllColumnFamilies: boolean` When `true`, applies `compression` to every column
+    family opened for the database rather than only the column family specified by `name`. Requires an explicit
+    `compression`. Defaults to `false`. See [Compression](#compression).
+  - `dbWriteBufferSize: number` The total memtable memory budget in bytes shared across all of the
+    database's column families. When the combined size of all memtables reaches this value, RocksDB
+    flushes the largest one. `0` (the default) disables this global trigger, so per-column-family
+    `writeBufferSize` alone drives flushing. This is distinct from the process-wide
+    [`writeBufferManagerSize`](#dbconfigoptions) config option. Database-wide, so it binds when the
+    path is first opened in this process: a later open of the same path — including from another
+    worker thread — keeps the first opener's value rather than overriding or rejecting it.
   - `disableWAL: boolean` Whether to disable the RocksDB write ahead log. Defaults to `false`.
   - `enableStats: boolean` When `true` and the database is open, RocksDB will captures stats that
     are retrieved by calling `db.getStats()`. Enabling statistics imposes 5-10% in overhead.
@@ -76,6 +88,17 @@ Creates a new database instance.
     compaction falls behind under sustained ingest); a positive `int32` is an explicit cap. Reads
     only pay a reopen cost when the number of live table files exceeds the budget, so raise the
     process fd limit (and with it the derived budget) for very large databases.
+  - `maxWriteBufferNumber: number` The maximum number of memtables that can be queued per column
+    family before writes stall. Higher values absorb write bursts while flushes catch up, at the
+    cost of memory (roughly `maxWriteBufferNumber * writeBufferSize` per column family). Defaults to
+    `16`.
+  - `maxWriteBufferSizeToMaintain: number` The number of bytes of recent memtable history to keep in
+    memory for transaction conflict checking. `-1` (the default) derives the value from
+    `maxWriteBufferNumber * writeBufferSize` (the RocksDB-recommended default for optimistic
+    transactions) — except when a stalling [`writeBufferManager`](#dbconfigoptions) is configured
+    (`writeBufferManagerSize > 0` with `writeBufferManagerAllowStall`), in which case it resolves to
+    `0` to avoid retaining memtable history the manager will never release. An explicit non-negative
+    value is always honored as-is.
   - `name: string` The column family name. Defaults to `"default"`.
   - `noBlockCache: boolean` When `true`, disables the block cache. Block caching is enabled by
     default and the cache is shared across all database instances.
@@ -85,7 +108,44 @@ Creates a new database instance.
     until commit. Defaults to `false`.
   - `readOnly: boolean` When `true`, the database is opened in read-only mode. Read operations are
     permitted. Write operations will throw an error with code `ERR_DATABASE_READONLY`. Transactions
-    are a no-op in read-only mode.
+    are a no-op in read-only mode. [`flush()`](#dbflushoptions-promisevoid),
+    [`flushSync()`](#dbflushsyncoptions-void), [`compact()`](#dbcompactoptions-promisevoid) and
+    [`compactSync()`](#dbcompactsyncoptions-void) are the exception: they have nothing to do when
+    there are no writes, so they succeed as no-ops rather than throwing.
+
+    A read-only open is a **point-in-time snapshot**: it never sees later writes. It is safe
+    against a quiescent database, but **unsafe against a live writer**: the open holds no reference
+    on the files it is about to read, so a concurrent compaction, blob GC, or flush in the writing
+    process can delete one mid-open and the open throws with code `ERR_CONCURRENT_COMPACTION` — the
+    database is not corrupt; the reader lost a race, and retrying can succeed. Even a successful
+    open can fail _later reads_ the same way, because files the snapshot references are opened
+    lazily while the writer keeps deleting obsolete ones. To follow a database another process is
+    actively writing, open a secondary instead (`secondaryPath` below) — that is the supported mode
+    for a live follower.
+
+  - `secondaryPath: string` Opens the database as a **secondary instance**: a read-only follower of
+    a live primary that tolerates the primary deleting files and sees new writes on each
+    [`catchUpWithPrimary()`](#dbcatchupwithprimary-promisevoid) call. This is the supported way to
+    read a database another process is actively writing. The value is the secondary instance's own
+    workspace directory (created if missing) where RocksDB keeps the secondary's private state — it
+    must be outside `path` (enforced) and exclusive to one secondary instance. Exclusivity is
+    enforced: reusing a workspace in-process for a different database is rejected at open (the same
+    database + workspace share the one follower instance), and a kernel advisory lock on
+    `<secondaryPath>/.secondary.lock` (same discipline and caveats as the
+    [backup directory lock](#backups)) excludes other processes — with the same limits: on
+    filesystems without advisory locking (`flock` unsupported — e.g. the FUSE/9p mounts behind
+    Docker Desktop bind mounts) the lock degrades to a no-op, and on network filesystems with
+    node-local `flock` (NFS `local_lock`, CIFS, 9p) it does not exclude across hosts, so workspace
+    exclusivity is the caller's job there. Implies
+    `readOnly: true` (an explicit `readOnly: false` throws), so all read-only behavior above
+    applies. Forces `maxOpenFiles: -1`: every table and blob file is opened and held for the life
+    of each version, which is what makes the primary's deletions safe — budget file descriptors
+    accordingly on large databases (an explicit `maxOpenFiles` other than `-1` is rejected). Column
+    families created by the primary after the secondary opens are invisible until the secondary
+    reopens. Note: RocksDB upstream documents secondary instances as unsupported in combination
+    with integrated BlobDB (which this library enables for values ≥ 2KB); the pinned RocksDB build
+    handles blob files in secondary mode — covered by native regression tests — but upstream does
+    not guarantee the combination.
   - `statsLevel: StatsLevel` Controls which type of statistics to skip and reduce statistic
     overhead. Defaults to `StatsLevel.ExceptDetailedTimers`.
   - `store: Store` A custom store that handles all interaction between the `RocksDatabase` or
@@ -96,10 +156,9 @@ Creates a new database instance.
     modified time to be older than the retention period before it is rotated to the next sequence
     number. Value must be between `0.0` and `1.0`. A threshold of `0.0` means ignore age check.
     Defaults to `0.75`.
-  - `transactionLogMaxSize: number` The maximum size of a transaction log file. If a log file is
-    empty, the first log entry will always be added regardless if it's larger than the max size. If
-    a log file is not empty and the entry is larger than the space available, the log file is
-    rotated to the next sequence number. Defaults to 16 MB.
+  - `transactionLogMaxSize: number` The target maximum size of a transaction log file. Transactions
+    are never split across files: if the complete transaction does not fit, the log rotates before
+    writing it. A transaction written to an empty file may exceed the target. Defaults to 16 MB.
   - `transactionLogRetention: string | number` The number of minutes to retain transaction logs
     before purging. Defaults to `'3d'` (3 days).
   - `transactionLogsPath: string` The path to store transaction logs. Defaults to
@@ -109,6 +168,9 @@ Creates a new database instance.
     the verification slot for each written key. Enable this only for column families whose records
     are cached (e.g. the primary column family of a table). Defaults to `false`. Requires
     `verificationTableEntries` to be configured before the first database is opened.
+  - `writeBufferSize: number` The per-column-family memtable size in bytes at which the memtable is
+    sealed and flushed to an SST file. Smaller values produce more frequent, faster flushes; larger
+    values batch more writes per SST file at the cost of memory. Defaults to `16777216` (16 MB).
 
 ### `db.close()`
 
@@ -128,7 +190,7 @@ Returns the list of column families in the RocksDB database.
 const db = RocksDatabase.open('path/to/db');
 console.log(db.columns); // ['default']
 
-const db2 = new RocksDatabase('path/to/db', { name: 'users' });
+db.use('users');
 console.log(db.columns); // ['default', 'users']
 ```
 
@@ -146,18 +208,6 @@ const db = RocksDatabase.open('path/to/db', {
 console.log(db.compression); // { algorithm: 'zstd', level: 3 }
 ```
 
-### `db.logOptions: { maxLogFileSize: number, infoLogLevel: number }`
-
-Returns the informational-log settings currently in effect for this database, read live from
-RocksDB. These are database-wide settings (not per-column-family). `maxLogFileSize` is the
-per-file size cap for informational log files (`LOG` / `LOG.old.*`); `infoLogLevel` is the logging
-verbosity. The database must be open.
-
-```typescript
-const db = RocksDatabase.open('path/to/db', { maxLogFileSize: 4 * 1024 * 1024 });
-console.log(db.logOptions); // { maxLogFileSize: 4194304, infoLogLevel: 1 }
-```
-
 ### `db.config(options)`
 
 Sets global database settings.
@@ -175,7 +225,11 @@ Sets global database settings.
   - `writeBufferManagerAllowStall: boolean` When `true`, writes are stalled once the manager's
     `buffer_size` is exceeded, providing a hard cap on memtable memory. When `false`, memtables are
     allowed to grow past the limit and flushes are simply scheduled more aggressively. Off by
-    default to favor write throughput over hard memory bounding. Defaults to `false`.
+    default to favor write throughput over hard memory bounding. Defaults to `false`. A stall here
+    is invisible to RocksDB's own stall counters and to
+    [`db.isWriteStalled()`](#dbiswritestalled-boolean) — see
+    [`getWriteBufferManagerStats()`](#getwritebuffermanagerstats-writebuffermanagerstats),
+    which also describes the watchdog that logs a sustained stall.
   - `writeBufferManagerCostToCache: boolean` When `true`, memtable memory is "charged" against the
     shared block cache so the block cache and write buffers draw from a single pool. During write
     bursts the cache shrinks to make room for memtables; once memtables flush, the cache can grow
@@ -195,6 +249,80 @@ RocksDatabase.config({
 });
 ```
 
+### `getWriteBufferManagerStats(): WriteBufferManagerStats`
+
+Reads the live state of the `WriteBufferManager`.
+
+**Process-wide, not per-database.** The manager is a singleton shared by every database opened in
+this process, `worker_threads` included, so these values describe the whole process regardless of
+where the call was made.
+
+- `enabled: boolean` Whether a manager has been created. `bufferSize`, `memoryUsage`,
+  `mutableMemoryUsage`, `stallActive`, `stallActiveMs`, `watchdogRunning` and `columnFamilies` are
+  `0`/`false` when not; `allowStall` and `costToCache` still reflect the configured setting (a
+  manager is only created once `writeBufferManagerSize` is also set), and `inventoryAvailable` is
+  `true` (there is nothing to fail to collect).
+- `bufferSize: number` The budget in bytes (`writeBufferManagerSize`, read live).
+- `memoryUsage: number` Total memtable memory in bytes charged against the manager.
+- `mutableMemoryUsage: number` The share of `memoryUsage` held by active (mutable) memtables; the
+  rest is memtables awaiting flush plus retained write history.
+- `allowStall: boolean`, `costToCache: boolean` The manager's configuration.
+- `stallActive: boolean` Whether the manager is currently stalling writes.
+- `stallActiveMs: number` How long the current stall has been active; `0` when not stalled.
+  Sampled once a second by the watchdog, so `0` for a stall's first second and whenever
+  `watchdogRunning` is `false`.
+- `watchdogRunning: boolean` Whether the stall watchdog thread is running.
+- `columnFamilies: number` Live column families across every database attached to this manager. A
+  dropped column family keeps charging the manager until its last handle closes, so it is counted
+  until then.
+- `inventoryAvailable: boolean` `false` when the inventory could not be collected because the
+  database registry or a column-family inventory was locked. The two inventory fields are then
+  empty and everything else is still live; the call never blocks on database work that may itself
+  be stalled.
+- `maxWriteBufferSizeToMaintain: Record<string, number>` Effective per-column-family retained-history
+  target (as a decimal string) to how many of those column families carry it. Effective, not
+  requested: RocksDB rewrites a requested `0` for a transaction database.
+
+`bufferSize`, `memoryUsage`, `mutableMemoryUsage`, `stallActive` and `stallActiveMs` are also in
+[`db.getStats()`](#dbgetstatsall-boolean-rocksdbstats) and `db.getStat()` under the same
+`writeBufferManager.` prefix, for scraping; `enabled`, `allowStall`, `costToCache`,
+`watchdogRunning`, and the column-family inventory are only here, because collecting them either
+walks the database registry or isn't scrape-shaped.
+
+`stallActive` is the signal that distinguishes a stalled process from an idle one.
+[`db.isWriteStalled()`](#dbiswritestalled-boolean), the [`'writeStall'` event](#event-writestall)
+and `rocksdb.stall.micros` all track RocksDB's `WriteController`, which a `WriteBufferManager`
+stall never goes through.
+
+#### The stall watchdog
+
+While a manager exists with `writeBufferManagerAllowStall`, one low-frequency thread samples the
+manager once a second. When a stall has been continuously active past
+`ROCKSDB_JS_WBM_STALL_WARN_MS` (default `5000`; `0` disables the watchdog) it writes **one** line
+to `stderr` and emits it as a process-wide `'log.warn'` event — once per stall episode, not per
+blocked writer and not per sample:
+
+```
+[rocksdb-js] WriteBufferManager write stall active for 5.0s - no write can complete until memtable
+memory drops below the budget. budget=661.2MB usage=662.2MB (100.2%) mutable=12.4MB (1.9%)
+allowStall=true costToCache=true columnFamilies=28 maxWriteBufferSizeToMaintain={268435456:28}
+```
+
+It needs its own thread because every other candidate is blocked by the very condition it reports:
+a writer is parked inside RocksDB, RocksDB's stall callbacks are `WriteController`-only, and a JS
+timer cannot fire on a thread parked in a synchronous write. The line goes to `stderr` as well as
+the event so it is not lost when nothing has registered a `'log.warn'` listener; route one and
+ignore `stderr` if you would rather have it in your own log.
+
+```typescript
+RocksDatabase.on('log.warn', (message) => logger.warn(message));
+
+const wbm = getWriteBufferManagerStats();
+if (wbm.stallActive) {
+	logger.warn(`writes stalled for ${wbm.stallActiveMs}ms: ${wbm.memoryUsage}/${wbm.bufferSize}`);
+}
+```
+
 ### `db.isOpen(): boolean`
 
 Returns `true` if the database is open, otherwise false.
@@ -203,6 +331,18 @@ Returns `true` if the database is open, otherwise false.
 console.log(db.isOpen()); // true or false
 ```
 
+### `db.logOptions: { maxLogFileSize: number, infoLogLevel: number }`
+
+Returns the informational-log settings currently in effect for this database, read live from
+RocksDB. These are database-wide settings (not per-column-family). `maxLogFileSize` is the
+per-file size cap for informational log files (`LOG` / `LOG.old.*`); `infoLogLevel` is the logging
+verbosity. The database must be open.
+
+```typescript
+const db = RocksDatabase.open('path/to/db', { maxLogFileSize: 4 * 1024 * 1024 });
+console.log(db.logOptions); // { maxLogFileSize: 4194304, infoLogLevel: 1 }
+```
+
 ### `db.name: string`
 
 Returns the database column family's name.
@@ -232,6 +372,20 @@ There's also a static `open()` method for convenience that performs the same thi
 const db = RocksDatabase.open('path/to/db');
 ```
 
+### `db.secondaryPath: string | undefined`
+
+The secondary instance's workspace directory when the database was opened as a
+[secondary](#new-rocksdatabasepath-options) (a read-only follower of a live primary), or
+`undefined` for a regular or plain read-only open.
+
+```typescript
+const follower = RocksDatabase.open('/path/to/database', {
+	secondaryPath: '/path/to/follower-workspace',
+});
+console.log(follower.secondaryPath); // '/path/to/follower-workspace'
+console.log(follower.readOnly); // true
+```
+
 ### `db.status: 'opened' | 'closed'`
 
 Returns a string `'opened'` or `'closed'` indicating if the database is opened or closed.
@@ -240,8 +394,95 @@ Returns a string `'opened'` or `'closed'` indicating if the database is opened o
 console.log(db.status);
 ```
 
+### `db.use(name, options?): RocksDatabase`
+
+Returns a `RocksDatabase` bound to the `name` column family of this same database, opening — and
+creating, if it does not exist — the column family on first use. This is a factory (like
+[`useLog`](#dbuselogname-transactionlog)), not a stateful switch: the returned view is an
+independent instance whose own reads and writes target its column family, while sharing the same
+underlying database. Because a single RocksDB database backs every column family, a transaction,
+backup, or checkpoint still spans all of them.
+
+- `name: string` The column family name.
+- `options?: object` Options for the column family (same shape as the constructor's, minus `name`),
+  overriding the options inherited from this database. Options only take effect when the view is
+  (re)opened.
+
+`use()` is get-or-create, backed by a weak cache: `db.use('events') === db.use('events')` while the
+view is still referenced **and open**, and calling with this database's own column-family name
+returns `this`. A view that has been closed (or garbage-collected — the cache does not pin it) is
+transparently recreated on the next `use()`. Views are independent handles: closing this database
+does not close them (and vice versa); the underlying database stays open until every handle is closed
+or collected.
+
+The view's store is derived by `Store#createColumnFamilyStore(name, options)`, which builds an
+independent store of the same class with **its own codec state** — views never share a mutable
+encoder/decoder. Consequently a database configured with a _pre-constructed_ encoder/decoder instance
+cannot derive views (its `name`/`structures` would be shared and corrupt both column families); use an
+encoder factory (`{ Encoder }`) or a named `encoding` instead. A custom `Store` whose constructor
+can't be recreated from `(path, options)` (e.g. it takes injected dependencies) should override
+`createColumnFamilyStore` to build its views.
+
+```typescript
+const db = RocksDatabase.open('path/to/db');
+
+const events = db.use('events');
+await events.put('e1', payload);
+
+await db.put('k', 'v'); // default column family, unaffected
+console.log(events.get('e1')); // payload
+console.log(db.get('e1')); // undefined — different column family
+
+events.close();
+db.close();
+```
+
 ## Data Operations
 
+### `db.catchUpWithPrimary(): Promise<void>`
+
+Advances a [secondary instance](#new-rocksdatabasepath-options) to the primary's current state by
+tailing and replaying the primary's MANIFEST and WAL. A secondary does not see the primary's
+writes — flushed or not — until it catches up. Reads through the handle remain safe while the
+catch-up runs; catch-ups on the same database are serialized internally.
+
+Throws with code `ERR_NOT_SECONDARY` on a database that was not opened with `secondaryPath`.
+Column families the primary created after the secondary opened stay invisible until the secondary
+reopens; ones the primary dropped remain readable until then. Catch-up advances the **database**
+view only: transaction-log reads through a secondary serve the stores discovered at open, and a
+store the primary creates afterward becomes visible only on reopen. Whether entries appended to an
+already-open store are visible depends on where the writer is: a cross-process primary's appends
+are not (the reader's view of the file extent is fixed at open), while a writer in the _same_
+process shares the store object, so its appends are. Either way a log entry can describe data the
+database view does not have yet — the log write completes before the RocksDB commit for every
+writer, so log-leads-database is the normal direction and a consumer has to tolerate it. Serialize
+catch-up calls per database. Each async call holds a libuv worker for the whole replay, and
+concurrent calls queue on an internal per-database mutex while holding theirs, so a handful of
+overlapping catch-ups on a backlogged follower can exhaust the default four-thread pool and stall
+unrelated `fs`/`dns`/`crypto` work in the process. Await one before starting the next rather than
+firing one per timer tick.
+
+```typescript
+const primary = RocksDatabase.open('/path/to/database');
+const follower = RocksDatabase.open('/path/to/database', {
+	secondaryPath: '/path/to/follower-workspace',
+});
+
+primary.putSync('foo', 'bar');
+console.log(follower.getSync('foo')); // undefined — not caught up yet
+await follower.catchUpWithPrimary();
+console.log(follower.getSync('foo')); // 'bar'
+```
+
+### `db.catchUpWithPrimarySync(): void`
+
+Synchronous version of `catchUpWithPrimary()`. The replay runs on the JS thread, so prefer the
+async form unless the caller is already blocking.
+
+```typescript
+follower.catchUpWithPrimarySync();
+```
+
 ### `db.clear(options?): Promise<number>`
 
 Asychronously removes all data in the current database.
@@ -302,9 +543,13 @@ await db.compact({ start: 'a', end: 'z' });
 await db.compact({ bottommost: true });
 ```
 
+On a [read-only](#new-rocksdatabasepath-options) database this is a no-op: arguments are still
+validated, but the returned promise resolves without compacting rather than rejecting.
+
 ### `db.compactSync(options?): void`
 
-Synchronous version of `compact()`.
+Synchronous version of `compact()`. On a [read-only](#new-rocksdatabasepath-options) database it
+validates its arguments and then returns without compacting, rather than throwing.
 
 ```typescript
 db.compactSync();
@@ -345,23 +590,63 @@ db.dropSync();
 db.close();
 ```
 
-### `db.flush(): Promise<void>`
+### `db.flush(options?): Promise<void>`
 
 Flushes all in-memory data to disk asynchronously.
 
+- `options: object`
+  - `allowWriteStall?: boolean` Whether the flush may proceed even though it will stall writes for
+    its duration. Defaults to `false`, which — despite how that reads — means the flush **waits**
+    until it can run without causing a stall, with no timeout: on a database stuck in a stall
+    condition (immutable-memtable backlog, L0 stop trigger, an exhausted `WriteBufferManager`
+    budget), the returned promise never settles, and since `flush()` runs on the libuv threadpool
+    (default size 4), a handful of concurrently stalled flushes can exhaust the pool and stall
+    every unrelated `fs`/`dns`/`crypto` call and cold-cache `get()` in the process, not only this
+    database's. Pass `true` up front, before issuing a flush you expect might stall, when it's a
+    durability gate you'd rather have stall writers than wait indefinitely — there's no way to
+    cancel an in-flight `flush()`, so this isn't a rescue for one already hung, and forcing the
+    memtable switch can itself prolong an existing L0 stop-trigger condition rather than clear it.
+    That cost is database-wide, covering every column family on the (process-global,
+    `worker_threads`-shared) database handle, not just the caller's — and it relocates the hang
+    rather than removing it: a stalled write blocks the database's single commit thread, which
+    dispatches every `Transaction.commit()` in order, so every commit behind it queues up too,
+    including ones from callers that never touched flush. This is a different knob from
+    `writeBufferManagerAllowStall` (see [`new RocksDatabase()`](#new-rocksdatabasepath-options)
+    options), with nearly opposite polarity: that one governs whether the `WriteBufferManager` may
+    stall writers at all, this one governs whether one manual flush is willing to cause a stall
+    rather than wait one out.
+
 ```typescript
 await db.flush();
+
+// Chosen up front, as a durability gate willing to pay the stall cost
+await db.flush({ allowWriteStall: true });
 ```
 
-### `db.flushSync(): void`
+On a [read-only](#new-rocksdatabasepath-options) database this is a no-op: the options bag is still
+validated, but the returned promise resolves without flushing rather than rejecting.
+
+### `db.flushSync(options?): void`
 
 Flushes all in-memory data to disk synchronously. Note that this can be an expensive operation, so
 it is recommended to use `flush()` if you want to keep the event loop free.
 
+- `options: object` Same as [`flush()`](#dbflushoptions-promisevoid). The default
+  `allowWriteStall: false` wait is taken on the calling thread, which here is the JS thread — so
+  the hazard described there is strictly larger on this entry point, not smaller: instead of
+  parking one libuv pool worker it freezes the event loop outright, and it holds the in-flight
+  operation claim that `close()` waits on, so the database cannot be closed out from under it
+  either. Prefer `flush()` if there is any chance the database is in a stall condition.
+
 ```typescript
 db.flushSync();
+
+db.flushSync({ allowWriteStall: true });
 ```
 
+On a [read-only](#new-rocksdatabasepath-options) database it validates its options and then returns
+without flushing, rather than throwing.
+
 ### `db.get(key: Key, options?: GetOptions): MaybePromise<any>`
 
 Retreives the value for a given key. If the key does not exist, it will resolve `undefined`.
@@ -408,13 +693,75 @@ the `expectedVersion` option is used.
 ### `db.getEstimatedKeyCount(): number`
 
 Retrieves the estimated number of keys in the database. This is an alias for
-`db.getDBIntProperty('rocksdb.estimate-num-keys')`.
+`db.getDBIntProperty('rocksdb.estimate-num-keys')`; use `estimateCount()` for range support and a
+confidence indicator.
 
 ```typescript
 const estimated = db.getEstimatedKeyCount();
 console.log(estimated);
 ```
 
+### `db.estimateCount(options?: CountEstimateOptions): CountEstimate`
+
+Estimates the number of keys in the database, or within a key range, returning
+`{ count, confidence }`. Unlike `getKeysCount()`, this never iterates: the estimate is derived
+from RocksDB statistics (memtable stats plus approximate SST sizes converted through the entry
+density of the SSTs overlapping the range), so its cost scales with the number of SSTs overlapping
+the range rather than the number of keys. Reading cold table properties can do I/O through the
+table cache, so narrow ranges are preferable. A start-only range is computed as the
+whole-database estimate minus the complement, so it does the work of the range _below_ `start`.
+Accuracy improves with range size. Resolution is bounded by SST data-block granularity, so a range
+narrower than a block is unreliable in either direction: it may over-report or report 0 for present
+keys, and its low `confidence` is the signal. Recently deleted or overwritten entries may be counted
+until compaction.
+Estimates always reflect committed state; writes pending in a transaction are not included. Set
+`reverse: true` to use `getRange()`'s reverse convention (`start` is the upper bound and `end` is
+the lower bound). An inverted range (`start` ≥ `end`) returns
+`{ count: 0, confidence: 1 }`.
+
+`confidence` is a heuristic 0–1 indicator of how trustworthy `count` is — exactly 1 only when the
+count is exact. It is derived from the estimate's resolution (data-block/memtable-sampling
+granularity relative to the count), the tombstone fraction of the overlapping SSTs, and — for
+start-only ranges — the error compounded by complement subtraction. Treat it as an ordering
+signal (e.g. when to trust an estimate for query planning vs fall back to a heuristic), not a
+statistical bound.
+
+```typescript
+const { count, confidence } = db.estimateCount({ start: 'a', end: 'z' });
+```
+
+### `db.createCountEstimator(options?: CountEstimatorOptions): CountEstimator`
+
+Creates an estimator that progressively refines a range count estimate while the range is being
+iterated — useful for reporting a total alongside a page of results without scanning the full
+range. Before any traversal, `estimate()` returns the pure statistical estimate (same as
+`estimateCount(range)`). As the caller reports progress with `advance(lastKey, count)` (e.g. once
+per page), `estimate()` returns the exact traversed count plus a statistical estimate of the
+remainder, calibrated by the observed ratio of actual-to-estimated entries over the portion
+already traversed — so the count converges toward the exact total. `confidence` is the
+exactness-weighted blend of the traversed portion and the remainder's confidence, so it approaches 1
+as the exact portion grows, although a checkpoint may decrease when calibration makes a large
+correction.
+Each checkpoint reads committed state, so a traversal performed against a transaction snapshot may
+be calibrated against data committed after that snapshot.
+When traversal completes, call `finish()` and `estimate()` returns the exact count with
+confidence 1. Reverse ranges follow `getRange`: set `start` to the upper bound and `end` to the
+lower bound, then set `reverse: true`. The caller owns the progress contract: cursors must move
+monotonically through the range and each entry must be reported exactly once.
+
+```typescript
+const range = { start: 'a', end: 'z' };
+const estimator = db.createCountEstimator(range);
+let lastKey;
+let pageSize = 0;
+for (const { key } of db.getRange({ ...range, limit: 25 })) {
+	lastKey = key;
+	pageSize++;
+}
+estimator.advance(lastKey, pageSize);
+const { count, confidence } = estimator.estimate();
+```
+
 ### `db.getKeys(options?: IteratorOptions): ExtendedIterable`
 
 Retrieves all keys within a range.
@@ -437,7 +784,7 @@ const range = db.getKeysCount({ start: 'a', end: 'z' }); // exact number of keys
 ### `db.getMonotonicTimestamp(): number`
 
 Returns the current timestamp as a monotonically increasing timestamp in milliseconds represented as
-a decimal number.
+a decimal number. This process-wide clock also supplies each transaction's initial timestamp.
 
 ```typescript
 const ts = db.getMonotonicTimestamp();
@@ -497,6 +844,25 @@ const blobFiles = db.getDBIntProperty('rocksdb.num-blob-files');
 const numKeys = db.getDBIntProperty('rocksdb.estimate-num-keys');
 ```
 
+### `db.isWriteStalled(): boolean`
+
+Whether RocksDB is currently applying write backpressure to this database —
+delaying (rate-limiting) or fully stopping writes. Reads the live
+`rocksdb.is-write-stopped` and `rocksdb.actual-delayed-write-rate` properties.
+
+This is the authoritative, live pull counterpart to the
+[`'writeStall'` event](#event-writestall): the event pushes per-column-family
+entries into a stall (rate-limited), while this reports the current state on
+demand and can never be stale. It is **database-wide** — the write controller is
+shared across every column family, so it answers "are writes stalled anywhere"
+rather than for one column family.
+
+```typescript
+if (db.isWriteStalled()) {
+	console.warn('writes are currently throttled or blocked');
+}
+```
+
 ### `db.getRange(options?: IteratorOptions): ExtendedIterable`
 
 Retrieves a range of keys and their values. Supports both synchronous and asynchronous iteration.
@@ -518,6 +884,23 @@ for (const { key, value } of db.getRange({ start: 'a', end: 'z' })) {
 }
 ```
 
+Pass `transaction: txn` to iterate this column family through that transaction, exactly as
+`get()` does with the same option: the iterator sees the transaction's staged writes and reads on its
+snapshot, and range bounds apply to staged keys too. `getKeys()` and `getKeysCount()` accept it as
+well. On a transaction (`txn.getRange()`) the transaction itself is the context and takes precedence
+over a `transaction` option, as it does for `txn.get()`. Such an iterator is closed when the
+transaction commits or aborts: a later `next()` throws, while `return()` stays a no-op. Opening a
+range or counting through a transaction that has already started committing throws as well.
+
+```typescript
+await db.transaction(async (txn) => {
+	await txn.put('c', 'staged');
+	for (const { key, value } of db.getRange({ start: 'a', end: 'z', transaction: txn })) {
+		console.log({ key, value }); // includes { key: "c", value: "staged" }
+	}
+});
+```
+
 ### `db.getUserSharedBuffer(key: Key, defaultBuffer: ArrayBuffer, options?)`
 
 Creates a new buffer with the contents of `defaultBuffer` that can be accessed across threads. This
@@ -692,6 +1075,15 @@ not committed after the configured number of coordinated retries, the transactio
 an `ERR_TRANSACTION_ABANDONED` error. Coordinated retry requires the column family to be opened with
 `verificationTable: true`.
 
+That wait is bounded so a conflicting transaction that is never committed or aborted cannot block
+the commit forever: if the write intent has not been released after `ROCKSDB_JS_PARK_TIMEOUT_MS`
+(default `5000`), the commit resolves anyway and consumes a retry attempt exactly as a real release
+would. A conflicting transaction held for longer than roughly `maxRetries` times that timeout
+therefore ends in `ERR_TRANSACTION_ABANDONED` rather than waiting indefinitely. Deployments where
+waiting is preferable to failing should raise the timeout; there is no way to disable the bound.
+In particular `0` does not disable it: `0`, negative, and unparseable values all fall back to the
+`5000` default, and a value between `1` and `49` is clamped up to `50`.
+
 ### Class: `Transaction`
 
 The transaction callback is passed in a `Transaction` instance which contains all of the same data
@@ -705,13 +1097,14 @@ operations methods as the `RocksDatabase` instance plus:
   no further transaction operations are permitted. Calling this method multiple times has no effect.
 - `txn.commit(): Promise<void>` Asynchronously commits the transaction and closes the transaction.
 - `txn.commitSync()` Synchronously commits and closes the transaction.
-- `txn.getTimestamp(): number` Retrieves the transaction start timestamp in seconds as a decimal. It
-  defaults to the time at which the transaction was created.
+- `txn.getTimestamp(): number` Retrieves the transaction timestamp in milliseconds as a decimal. It
+  defaults to a process-wide monotonic value assigned when the transaction was created.
 - `txn.id: number` The read-only transaction ID. Transaction IDs are unique to the RocksDB database
   path, regardless the database name/column family.
-- `txn.setTimestamp(ts?: number): void` Overrides the transaction start timestamp. If called without
-  a timestamp, it will set the timestamp to the current time. The value must be in seconds with
-  higher precision in the decimal.
+- `txn.setTimestamp(ts?: number): void` Overrides the transaction timestamp in milliseconds. If
+  called without a timestamp, it claims a fresh monotonic value. It must be called before staging
+  any write or transaction-log entry, and a supplied value must be finite, positive, and below
+  `8.64e15`.
 
 #### `txn.abandonWrites(): void`
 
@@ -747,8 +1140,9 @@ Once called, no further transaction operations are permitted.
 
 #### `txn.getTimestamp(): number`
 
-Retrieves the transaction start timestamp in seconds as a decimal. It defaults to the time at which
-the transaction was created.
+Retrieves the transaction timestamp in milliseconds since the Unix epoch as a decimal. It defaults
+to a process-wide monotonic value assigned when the transaction was created. The transaction log
+uses this timestamp as the batch key; producers may also encode it into their own record format.
 
 #### `txn.id`
 
@@ -757,14 +1151,91 @@ Type: `number`
 The transaction ID represented as a 32-bit unsigned integer. Transaction IDs are unique to the
 RocksDB database path, regardless the database name/column family.
 
-#### `txn.setTimestamp(ts: number?): void`
+#### `txn.setTimestamp(ts?: number): void`
+
+Overrides the transaction timestamp in milliseconds since the Unix epoch. Replication receivers and
+crash replay can use this to adopt an origin transaction's log key. If called without a timestamp,
+it claims a fresh monotonic value.
 
-Overrides the transaction start timestamp. If called without a timestamp, it will set the timestamp
-to the current time. The value must be in seconds with higher precision in the decimal.
+The override must happen while the transaction is pending and before any database write or
+transaction-log entry is staged. A log batch that has already been written keeps its timestamp
+across a coordinated retry; reapplying that same timestamp remains idempotent while pending. A
+supplied value must be finite, positive, and below `8.64e15`.
+
+rocksdb-js does not define a record's value layout or version metadata. A producer that copies the
+transaction timestamp into record bytes is responsible for calling `setTimestamp()` before reading
+and encoding it.
 
 ```typescript
 await db.transaction(async (txn) => {
-	txn.setTimestamp(Date.now() / 1000);
+	txn.setTimestamp(Date.now());
+});
+```
+
+## Error Handling
+
+### `db.getLastError(): BackgroundError | null`
+
+Returns the most recent [`BackgroundError`](#event-error) observed on this database, or `null` when
+none has occurred. This is the **pull** counterpart to the `'error'` [event](#event-error): use it
+for an on-demand check (e.g. a health probe) or to catch an error that fired before a listener was
+attached. The value is historical — it is **not** cleared by a successful
+[`db.resume()`](#dbresume-void); reset it explicitly with
+[`db.setLastError(null)`](#dbsetlasterrorerror-void). When the returned error's `writesDisabled` is
+`true`, writes are stopped until recovery.
+
+```typescript
+const err = db.getLastError();
+if (err?.writesDisabled) {
+	// ...free disk space, then...
+	db.resume();
+}
+```
+
+### `db.setLastError(error?): void`
+
+Sets or clears the last background error, mirroring the Win32 `SetLastError` / `GetLastError` pair.
+
+- **Clear** — pass `null` (or no argument). [`db.getLastError()`](#dbgetlasterror-backgrounderror--null)
+  then returns `null` and no event fires. This is how you reset the error after handling or
+  recovering it (e.g. after [`db.resume()`](#dbresume-void)) — the equivalent of
+  `SetLastError(ERROR_SUCCESS)`.
+- **Set** — pass an object (`{ message, severity?, severityName?, writesDisabled?, reason?, reasonName? }`;
+  `type` defaults to `'background'`). It is stored and the `'error'` [event](#event-error) fires with
+  the reconstructed [`BackgroundError`](#event-error). Useful for surfacing an application-level
+  "this database is unusable" state, and for deterministically exercising the error path in tests.
+
+```typescript
+// reset after recovery
+db.resume();
+db.setLastError(null);
+
+// inject (e.g. in a test)
+db.on('error', (err) => console.error(err.message));
+db.setLastError({
+	message: 'disk quota exceeded',
+	severity: 2,
+	severityName: 'hard',
+	writesDisabled: true,
+});
+```
+
+### `db.resume(): void`
+
+Attempts to recover the database from a background error (see the `'error'` [event](#event-error))
+by calling RocksDB's `DB::Resume()`. Call this after the underlying condition has cleared — e.g.
+once disk space has been freed. On success writes are accepted again (and RocksDB can resume the
+obsolete-file cleanup it was blocking); on failure — the condition has not actually cleared — it
+throws with the RocksDB error and the database stays read-only. A no-op on a healthy database.
+
+Runs synchronously and may briefly block, since recovery can re-flush memtables.
+
+```typescript
+db.on('error', (err) => {
+	if (err.writesDisabled) {
+		// ...free disk space, then...
+		db.resume();
+	}
 });
 ```
 
@@ -794,6 +1265,89 @@ The `'committed'` event is emitted after the transaction has been written. When
 emitted, the transaction is still cleaning up. If you need to know when the transaction is fully
 complete, use the `'aftercommit'` event.
 
+### Event: `'error'`
+
+When a write fails at the filesystem level — a full disk, an exhausted quota — RocksDB records a
+background error and, for a hard-or-worse severity, stops accepting writes. The database emits an
+`'error'` event carrying a `BackgroundError` so a consumer can observe it and recover in-process
+instead of restarting. When `writesDisabled` is `true`, call
+[`db.resume()`](#dbresume-void) after the underlying condition clears. Register the listener with
+`db.on('error', ...)` before the database is put under load — or use the pull-based
+[`db.getLastError()`](#dbgetlasterror-backgrounderror--null) to catch an error that fired earlier.
+
+`BackgroundError` is a real `Error` subclass, exported from the package, so both
+`err instanceof Error` and `err instanceof BackgroundError` hold. It extends `Error` with:
+
+- `message: string` — the RocksDB error string.
+- `name: string` — always `'BackgroundError'`.
+- `type: string` — the error-class discriminator; always `'background'`.
+- `severity: number` — the RocksDB `Status::Severity`: `1` soft, `2` hard, `3` fatal,
+  `4` unrecoverable.
+- `severityName: string` — `'soft'`, `'hard'`, `'fatal'`, or `'unrecoverable'`.
+- `writesDisabled: boolean` — whether RocksDB has disabled writes on the database (`severity >= 2`).
+  Only then is `resume()` warranted; a soft error is auto-recoverable and leaves writes enabled.
+  Distinct from opening the database in read-only mode.
+- `reason?: number` / `reasonName?: string` — the originating `BackgroundErrorReason`
+  (e.g. `'flush'`, `'compaction'`), present when the error came from a reason-bearing callback.
+
+```typescript
+import { BackgroundError } from '@harperfast/rocksdb-js';
+
+db.on('error', (err) => {
+	if (err instanceof BackgroundError && err.writesDisabled) {
+		console.error(`writes disabled (${err.severityName}): ${err.message}`);
+	}
+});
+```
+
+```typescript
+db.on('error', (err) => {
+	if (err.writesDisabled) {
+		// Hard-or-worse: RocksDB has disabled writes on this database. They stay
+		// disabled until you clear the underlying condition and call db.resume().
+		console.error(`database writes disabled (${err.severityName}): ${err.message}`);
+	} else {
+		// Soft error: writes are NOT disabled and RocksDB will auto-recover. This
+		// is typically a transient background hiccup (e.g. a retryable I/O error
+		// during flush or compaction), surfaced for visibility only — no action
+		// is required.
+		console.warn(`recoverable background error (${err.severityName}): ${err.message}`);
+	}
+});
+```
+
+### Event: `'writeStall'`
+
+Emitted when a column family **enters** a RocksDB write-stall — the push signal that writes are being
+throttled or blocked (for example when `dbWriteBufferSize` is too small for the number of column
+families and RocksDB thrashes on premature flushes). Listeners receive three string arguments:
+
+- `columnFamily: string` — the column family that entered the stall (`'default'` for the primary).
+- `previousCondition: string` — `'normal' | 'delayed' | 'stopped'`.
+- `currentCondition: string` — `'delayed'` (writes rate-limited) or `'stopped'` (writes blocked until
+  a flush frees a memtable/L0 slot).
+
+The event is **rising-edge only and rate-limited** per column family. RocksDB oscillates a CF's stall
+condition many times per second during a sustained stall, so the event fires when a CF _enters_ a
+stall and then at most once per `ROCKSDB_JS_WRITE_STALL_DEBOUNCE_MS` (default 1000 ms) while it stays
+stalling — never a flood. **Recovery is not pushed**: reliably distinguishing a brief dip from a real
+recovery isn't possible without a timer, so for current state use the pull below rather than waiting
+for a `'normal'` event.
+
+For "are writes stalled right now?", call [`db.isWriteStalled()`](#dbiswritestalled-boolean) — it
+reads the live write-controller state and is authoritative. Note it is **database-wide** (the write
+controller is shared across all column families), so it answers "is anything stalling writes" while
+the event names _which_ CF; a listener attached mid-stall should consult it rather than expect a
+missed rising edge.
+
+```typescript
+db.on('writeStall', (cf, prev, cur) => {
+	console.warn(`RocksDB write stall entered on ${cf}: ${prev} -> ${cur}`);
+});
+// ...and to check the current state on demand:
+if (db.isWriteStalled()) console.warn('writes are currently throttled or blocked');
+```
+
 ## Event API
 
 `rocksdb-js` provides a EventEmitter-like API that lets you asynchronously notify events to one or
@@ -934,8 +1488,11 @@ Column family and ticker stat values are 64-bit unsigned integers and histogram
 `StatsHistogramData` objects.
 
 The result also always includes a summarized, aggregate set of `txnlog.*` keys covering all of
-the database's transaction logs. These are present regardless of whether statistics are enabled
-and can be fetched individually with `db.getStat('txnlog.…')`. For detailed, per-log statistics —
+the database's transaction logs, and a set of `writeBufferManager.*` keys describing the
+process-wide `WriteBufferManager`
+([`getWriteBufferManagerStats()`](#getwritebuffermanagerstats-writebuffermanagerstats)
+carries the same values plus its column-family inventory). Both sets are present regardless of
+whether statistics are enabled and can be fetched individually with `db.getStat()`. For detailed, per-log statistics —
 including memory-map usage — use [`log.getStats()`](#loggetstats-transactionlogstats). All stat
 names are documented in [docs/stats.md](docs/stats.md).
 
@@ -1242,6 +1799,14 @@ holds the last-known version for that key. Because slots are addressed by a hash
 share a slot; a collision only ever causes a conservative miss (a real read), never a stale value to
 be treated as fresh.
 
+This first word is the only version the table derives on its own: it is what a read with
+`populateVersion: true` publishes and what a transaction write invalidates against. A producer whose
+record format also carries a separately assigned version elsewhere in the value must keep using the
+first word on both sides of the check — as `expectedVersion`, and as the argument to any explicit
+[`db.populateVersion()`](#dbpopulateversionkey-key-version-number-void) call, which publishes
+whatever it is given. Comparing against the other value would miss the fast path, and publishing it
+would make the slot disagree with what the next write invalidates.
+
 The freshness check works as follows:
 
 1. Pass `{ expectedVersion }` to `get()` / `getSync()`. If the slot currently records that version,
@@ -1359,9 +1924,19 @@ const names = db.listLogs();
 ### `db.purgeLogs({ includeEntryCounts: true, ...options }): { path: string; entries: number }[]`
 
 Deletes transaction log files older than the `transactionLogRetention` (defaults to 3 days).
+Ordinary retention keeps the sequence file named by `txn.state` and every newer file as the live
+store's retention floor, or the highest sequence file when there is no persisted flush position. It
+removes only an eligible contiguous prefix below that floor. An idle store can therefore retain one
+file past the cutoff until a later write rotates and flushes it; that extra file is bounded by the
+store's `transactionLogMaxSize` (except when a single transaction exceeds the target). That bound
+covers only the floor file: when the flush position lags — `txn.state` stuck at an old sequence
+because RocksDB flushing is behind — every file above the floor is retained too, which
+`transactionLogMaxSize` does not bound. `purge.retainedUnflushedFiles` reports that case. Use
+`destroy: true` only to remove the store itself.
 
 - `options: object`
-  - `before?: number` Remove all transaction log files older than the specified timestamp.
+  - `before?: number` Remove transaction log files older than the specified timestamp, subject to
+    the retention floor and contiguous-prefix rules above.
   - `destroy?: boolean` When `true`, deletes transaction log stores including all log sequence files
     on disk.
   - `includeEntryCounts?: boolean` When `true`, counts the entries in each deleted log file and
@@ -1535,7 +2110,9 @@ stats.totals.transactionsWritten; // lifetime count of transactions written
 
 The `purge.retainedUnflushedFiles` gauge is useful for diagnosing why logs are not being cleaned
 up: a file can be older than the retention period but still retained because its transactions have
-not yet been flushed to RocksDB (purging it would be unsafe for crash recovery).
+not yet been flushed to RocksDB (purging it would be unsafe for crash recovery). The retention
+floor — the sequence named by `txn.state`, or the highest sequence when there is no persisted flush
+position — is not counted as purgeable even when it is old and fully flushed.
 
 ### Transaction Log Initialization
 
@@ -1705,7 +2282,7 @@ snapshots of a database. A backup covers the **entire database** — every colum
 manifest, and (by default) the write-ahead log — so it is not scoped to an individual `Store`.
 
 A backup can be written to a local **directory** (incremental, with a management API) or streamed to
-a **`WritableStream`** as a tar archive with no intermediate copy on disk. See
+a `WritableStream` as a tar archive with no intermediate copy on disk. See
 [docs/backups.md](docs/backups.md) for a full guide covering both modes, restore, checkpoints, and
 caveats.
 
@@ -1781,10 +2358,10 @@ required bytes — before any files are copied, so a full destination fails fast
 through (a mid-copy failure can leave zero usable backups). The required size is estimated
 conservatively as a full copy of the database's live files, plus the transaction-log snapshot when
 `transactionLogs` is set; since a backup only ever copies files, this can never under-estimate.
-Incremental backups to a right-sized volume may therefore be over-rejected — set `checkDiskSpace:
-false` to skip the check. It also self-disables when free space can't be determined (e.g. some
-network filesystems report `0`), so it never blocks a backup on an untrustworthy number. Only
-directory-target backups are checked; the streaming backup path is unaffected.
+Incremental backups to a right-sized volume may therefore be over-rejected — set
+`checkDiskSpace: false` to skip the check. It also self-disables when free space can't be determined
+(e.g. some network filesystems report `0`), so it never blocks a backup on an untrustworthy number.
+Only directory-target backups are checked; the streaming backup path is unaffected.
 
 ### `db.backup(stream: WritableStream<Uint8Array>, options?: BackupStreamOptions): Promise<void>`
 
@@ -2014,7 +2591,9 @@ Options for `get()`, `getSync()`, and the `getBinary*` methods.
   - `tailing: boolean` When `true`, creates a "tailing iterator" which is a special iterator that
     has a view of the complete database including newly added data and is optimized for sequential
     reads. This will return records that were inserted into the database after the creation of the
-    iterator. Defaults to `false`.
+    iterator. Defaults to `false`. A tailing iterator ignores a transaction's snapshot: through a
+    transaction it still reads the latest committed state, merged with that transaction's staged
+    writes.
 
 ### `RangeOptions`