@harperfast/rocksdb-js 2.9.1 → 2.10.0

package/README.md

@@ -185,7 +185,13 @@ Creates a new database instance.
 ### `db.close()`
 
 Closes a database. This function can be called multiple times and will only close an opened
-database. A database instance can be reopened once its closed.
+database. A database instance can be reopened once it is closed. A flush failure leaves the native
+database quarantined so `shutdown()` can retry without losing unflushed data; an explicit
+`destroy()` can instead delete it. A failure while waiting for compaction to settle is reported
+after native teardown completes; the optional `compactOnClose` pass itself is best-effort and its
+errors do not fail the close, since a skipped compaction loses no data. All native close errors
+emit `database:closeFailed`. The quarantine applies to both writable and read-only opens because
+both modes share the physical path lifecycle.
 
 ```typescript
 const db = RocksDatabase.open('foo');
@@ -227,6 +233,12 @@ Sets global database settings.
     Defaults to 32MB. Set to `0` (zero) disables block cache for future opened databases. Existing
     block cache for any opened databases is resized immediately. Negative values throw an error.
   - `compactOnClose: boolean` When `true`, compacts the database on close. Defaults to `false`.
+  - `lifecycleWaitSeconds: number` How long a synchronous open, destroy, or shutdown waits for a
+    _conflicting_ lifecycle operation already in progress on the same path (e.g. another open or
+    close) before throwing a retryable timeout error. It does not bound the separate, intentionally
+    unbounded wait that `destroy()`/`shutdown()` make for in-flight backups, checkpoints, or other
+    async work still using the database — see [`db.destroy()`](#dbdestroy-void). Defaults to `30`
+    seconds and must be a positive integer.
   - `verificationTableEntries: number` The number of slots in the process-global
     [Verification Table](#verification-table). Each slot is 8 bytes, so the default of `131072`
     (128K) slots is 1 MB. Set to `0` to disable the verification table. This must be configured
@@ -574,7 +586,16 @@ db.compactSync({ bottommost: true });
 ### `db.destroy(): void`
 
 Completely removes a database based on the `db` instance's path including all data, column families,
-and files on disk.
+and files on disk. Destruction owns the physical path for the process: it closes every writable and
+read-only handle for that path, waits for registered backups and checkpoints to stop using the
+native database, and prevents another handle from reopening the path until removal finishes. Those
+waits are synchronous and can outlive `lifecycleWaitSeconds` once destruction has claimed the path,
+because releasing the native database beneath an active copy would be unsafe.
+
+A previously opened instance does not need to remain open, which allows an explicit `destroy()`
+retry after failed physical cleanup. A never-opened or read-only instance cannot destroy the
+database. `shutdown()` reports a pending cleanup tombstone but never retries deletion; only an
+explicit `destroy()` can remove the path.
 
 ```typescript
 db.destroy();
@@ -583,8 +604,8 @@ console.log(fs.existsSync(db.path)); // false
 
 ### `db.drop(): Promise<void>`
 
-Removes all entries in the database. If the database was opened with a `name`, the database will be
-deleted on close.
+Drops the column family the database was opened with (`name`). For the default column family this
+clears all entries instead.
 
 ```typescript
 const db = RocksDatabase.open('path/to/db', { name: 'users' });
@@ -592,9 +613,56 @@ await db.drop();
 db.close();
 ```
 
+#### Dropping column families
+
+A drop retires the column family **logically** before it returns: the name is gone from
+`db.columns`, a later `open()` with the same name creates a fresh, empty column family, and any
+transaction that then stages a write to a handle of the dropped family, or commits one it staged
+earlier, is refused whole with `ERR_COLUMN_FAMILY_DROPPED` (`Column family "users" was dropped`).
+That terminal refusal releases the transaction's verification-table intents and bars further
+writes or commit attempts; retained reads continue until the caller aborts the transaction, and
+they still serve that transaction's own staged writes — values no commit will ever produce. A
+caller that catches the refusal instead of letting it propagate must not read a value back through
+the transaction and carry it forward.
+If a staging call fails while the generation is being retired, retirement takes precedence and the
+transaction is refused whole even when the immediate RocksDB failure was a pessimistic lock timeout.
+Handles other threads still hold keep **reading** the dropped data until they close; a
+non-transactional `putSync`/`removeSync` through such a handle is discarded.
+
+The **physical** RocksDB drop is deferred behind commits already admitted when the drop lands: a
+commit claims every column family its batch names before it writes its transaction-log batch and
+releases them after RocksDB has applied it, and the physical drop runs from whichever
+side releases last (or, for a commit a mid-flight `close()` tore out of its pipeline, from the next
+drop, open of that name, or close on the database). With no such commit (the common case) `drop()`/`dropSync()` perform the
+physical drop before returning, exactly as before. This is what keeps a drop racing another
+thread's commit from latching RocksDB's fatal `Invalid column family specified in write batch`
+error on the whole database.
+
+Consequences to know about:
+
+- A commit admitted before the drop completes successfully into the retiring generation, then the
+  physical drop removes that generation. Its caller sees a successful commit, but those writes are
+  intentionally discarded with the rest of the dropped column family; a same-name reopen creates
+  a fresh, empty generation. If the transaction writes to a transaction log, its entries are still
+  published; consumers must order the schema drop after those entries.
+- `open()` of a name whose previous generation is still held by an admitted commit waits for the
+  full admission-to-reclamation interval (bounded by `ROCKSDB_JS_CF_RECLAIM_WAIT_MS`, default
+  `30000`) before creating the fresh column family; if the previous generation's physical drop
+  failed, the open retries it once and throws with that error if it fails again.
+- A physical drop that fails (an I/O error writing the MANIFEST) keeps the name retired, is
+  retried on the next drop on the database, the next `open()` of that name, or close, and is
+  reported through the global `log.warn` event and the `columnFamily.pendingReclaims` stat. When
+  the failing drop was the one `drop()` itself ran, the call rejects with that error as well.
+
+What is **not** guaranteed: a process that exits while a physical drop is still pending, or
+after one failed, leaves the column family on disk under its name, and the next open of the
+database opens it as a live column family. A backup or checkpoint taken inside that window copies
+it. Callers that need a drop to survive a crash record their own durable tombstone before
+acknowledging it.
+
 ### `db.dropSync(): void`
 
-Synchronous version of `db.drop()`.
+Synchronous version of `db.drop()`, with the same deferral contract.
 
 ```typescript
 const db = RocksDatabase.open('path/to/db');
@@ -803,6 +871,12 @@ const ts = db.getMonotonicTimestamp();
 console.log(ts); // 1764307857213.739
 ```
 
+It is a wall-clock (Unix epoch) value made strictly increasing: on a tie or a backward step of the
+host clock it advances by one floating-point ulp per call until the wall clock catches up. That
+keeps transaction timestamps ordered and durable, but it does not measure elapsed time — after a
+backward step, differences between two calls understate real time. Use
+[`steadyClockNow()`](#steadyclocknow-number) for elapsed durations and deadlines.
+
 ### `db.getOldestSnapshotTimestamp(): number`
 
 Returns a number representing a unix timestamp of the oldest unreleased snapshot.
@@ -928,8 +1002,13 @@ Returns a new `ArrayBuffer` with two additional methods:
 - `cancel()` - Removes the callback; future `notify()` calls do nothing
 
 Note: If a shared buffer already exists for the given `key`, the returned `ArrayBuffer` will
-reference this existing shared buffer. Once all `ArrayBuffer` instances have gone out of scope and
-garbage collected, the underlying memory and notify callback will be freed.
+reference this existing shared buffer. The buffer lives as long as its column family is open (a
+`drop()` discards it with the column family — except the default column family, which `drop()`
+only clears, so its buffers survive): it is process-wide state, so a thread dropping or garbage
+collecting its own view (or exiting) never resets it for the others. Because nothing is ever
+evicted, key the buffer on a small fixed set of names rather than on unbounded data such as
+record ids. The notify callback is removed when the `ArrayBuffer` it was registered with is
+garbage collected, or by `cancel()`.
 
 ```typescript
 const buffer = new Uint8Array(db.getUserSharedBuffer('isDone', new ArrayBuffer(1)));
@@ -2212,6 +2291,51 @@ Options:
 Validating a store that is being actively appended to can spuriously report a torn tail for the
 current log file — the tail of an in-flight append is indistinguishable from a crash artifact.
 
+### `steadyClockNow(): number`
+
+Reads the process-wide steady clock. No database handle is needed.
+
+```typescript
+import { steadyClockNow } from '@harperfast/rocksdb-js';
+
+const start = steadyClockNow();
+// ... work, possibly on other worker threads ...
+const elapsedMs = steadyClockNow() - start;
+```
+
+Contract:
+
+- **Units / type**: milliseconds with a fractional part, as a `number`. Read from
+  `std::chrono::steady_clock` (`CLOCK_MONOTONIC` on Linux, `CLOCK_MONOTONIC_RAW` on macOS,
+  `QueryPerformanceCounter` on Windows).
+- **Origin and lifetime**: the origin is unspecified and fixed for the life of the process. Every
+  sample taken in the process is in one domain — the main thread and every `worker_threads` worker,
+  including workers started or restarted at any later time, under Node, Bun and Deno alike. Samples
+  are not meaningful across processes or restarts, and are unrelated to the Unix epoch: never compare
+  them with `Date.now()`, `db.getMonotonicTimestamp()` or transaction timestamps.
+- **Monotonic, not unique**: a sample taken after another (in real time, on any thread) is `>=` it.
+  Two samples can be equal — in the same clock tick, or when two distinct readings round to the same
+  double — so treat `==` as "not later" and pair the sample with a sequence number if strict ordering
+  is required. On Windows, QPC samples from different threads within ±1 native counter tick also
+  have ambiguous ordering; do not use near-equal clock samples alone to prove causality. No
+  wall-clock ratchet is applied.
+- **Precision**: the double's spacing grows with distance from the origin: about 2 ns at 100 days,
+  61 ns at 10 years, 0.49 µs at 100 years. Conversion error is below 0.51 µs per sample through
+  100 years from the native origin. This is representation precision, not clock accuracy. Rounding is
+  monotone, so distinct readings can only collapse to equality, never invert. The clock's own
+  resolution is platform-defined (the C++ duration period is 1 ns on these platforms; actual clock
+  resolution can be coarser). The native signed 64-bit nanosecond range is about 292 years in either
+  direction from its origin; conversion to `number` does not narrow it.
+- **Independent of the wall clock**: `settimeofday`/NTP steps in either direction do not move it. NTP
+  frequency slew is platform-defined (`CLOCK_MONOTONIC` is slewed; `CLOCK_MONOTONIC_RAW` and
+  `QueryPerformanceCounter` are not).
+- **Suspend**: whether time the host spends suspended counts is platform-defined — `CLOCK_MONOTONIC`
+  excludes it on Linux; current macOS `CLOCK_MONOTONIC_RAW` and Windows QPC include it. Do not
+  assume runtime timers (`setTimeout`) use the same suspend policy. Portable suspend-inclusive
+  deadlines are not provided by this API.
+- **Cost**: one clock read and a double conversion; no lock, no native allocation, no wall-clock read,
+  no database or log I/O, no per-worker calibration. Safe from any thread. The runtime may allocate a boxed JS number.
+
 ### `currentThreadId(): number`
 
 Returns the current thread ID.
@@ -2226,9 +2350,13 @@ console.log(currentThreadId());
 Returns an array containing that status of all active RocksDB instances.
 
 - `path: string` The database path.
+- `closeError?: string` The native lifecycle error retaining this registry entry.
+- `destroyCleanupPending?: boolean` The native database is closed, but physical path cleanup must
+  finish before the next open. Call `destroy()` to retry cleanup.
 - `refCount: number` The number of JavaScript database instances plus the registry's reference.
 - `columnFamiles: object` A map of column family names and their their info.
-  - `userSharedBuffers: number` The count of active user shared buffers.
+  - `userSharedBuffers: number` The count of user shared buffer keys created since the column family
+    was opened; entries are never evicted, so this never decreases.
 - `transactions: number` The count of active transactions.
 - `closables: number` The count of active database, transactions, and iterators.
 - `locks: number` The count of active locks.
@@ -2243,11 +2371,23 @@ console.log(registryStatus());
 
 The `shutdown()` will flush all in-memory data to disk and wait for any outstanding compactions to
 finish, for all open databases. It is highly recommended to call this in a `process` `exit` event
-listener (on the main thread), to ensure that all data is flushed to disk before the process exits:
+listener (on the main thread), to ensure that all data is flushed to disk before the process exits.
+It throws the first close failure after attempting every claimed database; call it again to retry
+any descriptor whose native teardown did not complete. It reports pending destroy-cleanup
+tombstones without deleting their paths; retry those with an explicit `destroy()`.
+
+Wrap the call: an exception thrown from an `exit` listener skips every `exit` listener registered
+after it, and sets the exit code to 1 unless an `uncaughtException` handler is installed.
 
 ```typescript
 import { shutdown } from '@harperfast/rocksdb-js';
-process.on('exit', shutdown);
+process.on('exit', () => {
+	try {
+		shutdown();
+	} catch (error) {
+		console.error('rocksdb-js shutdown failed', error);
+	}
+});
 ```
 
 ### `versions: { 'rocksdb': string; 'rocksdb-js': string }`

package/dist/index.cjs

@@ -131,9 +131,49 @@ const NativeIterator = binding.Iterator;
 const NativeTransaction = binding.Transaction;
 const TransactionLog = binding.TransactionLog;
 const registryStatus = binding.registryStatus;
+/**
+* Flushes every open database and waits for outstanding compactions, then
+* releases the registry.
+*
+* **Throws** the first failure it encounters: a close whose native teardown
+* failed (that descriptor stays quarantined — call `shutdown()` again to retry
+* it), or a `lifecycleWaitSeconds` timeout waiting on a concurrent
+* open/destroy/shutdown. A quarantining close also emits
+* `database:closeFailed`; a timeout has no event, so the throw is its only
+* signal.
+*
+* A `process.on('exit')` listener must therefore wrap it: an exception thrown
+* from an `exit` listener skips every `exit` listener registered after it (and
+* sets exit code 1 unless an `uncaughtException` handler is installed).
+*
+* ```typescript
+* process.on('exit', () => {
+* 	try {
+* 		shutdown();
+* 	} catch (error) {
+* 		console.error('rocksdb-js shutdown failed', error);
+* 	}
+* });
+* ```
+*/
 const shutdown = binding.shutdown;
 const currentThreadId = binding.currentThreadId;
 /**
+* Reads the process-wide steady clock: `std::chrono::steady_clock` as
+* fractional milliseconds from an unspecified origin that is fixed for the life
+* of the process. Every sample in the process — main thread and every
+* `worker_threads` worker, whenever started or restarted, on Node, Bun and Deno
+* — is in one domain, and the difference between two samples is real elapsed
+* time unaffected by wall-clock steps. Non-decreasing but not unique (two
+* samples may be equal; treat `==` as "not later"). Windows QPC samples from
+* different threads within one native tick also have ambiguous ordering.
+* Clock readings alone do not prove causality. Not comparable with
+* `Date.now()`, `db.getMonotonicTimestamp()` or transaction timestamps, and not
+* meaningful across processes or restarts. Time spent in host suspend is
+* platform-defined. No open database is needed.
+*/
+const steadyClockNow = binding.steadyClockNow;
+/**
 * Advises the kernel that the file-backed pages of every mapped transaction log
 * are cold (Linux MADV_COLD), so they are reclaimed first under memory pressure
 * without being freed — useful during replication catch-up, where a full read of
@@ -159,6 +199,21 @@ const transactionLogMapCount = binding.transactionLogMapCount;
 * disarm. Used by the ERR_TRY_AGAIN retry regression test.
 */
 const forceTryAgainForTesting = binding.forceTryAgainForTesting;
+/**
+* Test-only seam: makes the physical column-family drop report failure.
+* `0` = inert, `1` = fail without dropping (the retry performs the drop),
+* `2` = drop for real and then report failure (the retry resolves as
+* "already dropped"). Process-global, like `forceTryAgainForTesting`.
+*/
+const forceDropFailureForTesting = binding.forceDropFailureForTesting;
+/** Parks one selected transaction write after its retirement precheck. */
+const setTransactionStagingDelayForTesting = binding.setTransactionStagingDelayForTesting;
+/** Reports whether the selected transaction write is parked. */
+const isTransactionStagingDelayedForTesting = binding.isTransactionStagingDelayedForTesting;
+/** Reports whether a transaction is parked at the native commit seam. */
+const isTransactionCommitExecuteDelayedForTesting = binding.isTransactionCommitExecuteDelayedForTesting;
+const setTransactionCommitAdmissionDelayForTesting = binding.setTransactionCommitAdmissionDelayForTesting;
+const isTransactionCommitAdmissionDelayedForTesting = binding.isTransactionCommitAdmissionDelayedForTesting;
 /** Delays one selected watchdog join in the concurrent-shutdown regression test. */
 const setWriteBufferManagerJoinDelayForTesting = binding.setWriteBufferManagerJoinDelayForTesting;
 /**
@@ -2119,12 +2174,12 @@ var Store = class {
 		return false;
 	}
 	putSync(context, key, value, options) {
-		if (!this.db.opened) throw new Error("Database not open");
+		if (!this.db.opened) throw new Error(this.db.closing ? "Database is closing" : "Database not open");
 		const valueBuffer = this.encodeValue(value);
 		context.putSync(this.encodeKey(key), valueBuffer, this.getTxnId(options));
 	}
 	removeSync(context, key, options) {
-		if (!this.db.opened) throw new Error("Database not open");
+		if (!this.db.opened) throw new Error(this.db.closing ? "Database is closing" : "Database not open");
 		context.removeSync(this.encodeKey(key), this.getTxnId(options));
 	}
 	/**
@@ -2628,7 +2683,10 @@ var RocksDatabase = class RocksDatabase extends DBI {
 	* native binding use namespaced keys (e.g. `'transactionLog:warning'`).
 	*
 	* Listeners are not tied to any specific database — they fire for every
-	* matching event emitted in this process.
+	* matching event emitted in this process. Native lifecycle failures use
+	* `'database:closeFailed'` with `(path, error)` string arguments. The event
+	* reports both completed and incomplete teardowns; an incomplete close can
+	* be retried by `shutdown()` or explicitly deleted by `destroy()`.
 	*
 	* @example
 	* ```typescript
@@ -2712,7 +2770,7 @@ var RocksDatabase = class RocksDatabase extends DBI {
 		this.store.db.catchUpWithPrimarySync();
 	}
 	destroy() {
-		this.store.db.destroy();
+		this.store.db.destroy(this.store.readOnly);
 	}
 	async drop() {
 		return new Promise((resolve, reject) => {
@@ -3578,8 +3636,8 @@ function frameFits(dataView, pos, dataEnd) {
 * terminator, let a chain "end" anywhere in megabytes of padding. Against the written extent the
 * end is a single offset, so a chain landing on it is ~1/2^32 to be coincidence.
 *
-* The JS counterpart of `validFramingResumes()` in `transaction_log_recovery.cpp`, which answers
-* the same question at open time but keeps only the yes/no, discarding the offset.
+* The JS counterpart of `findFramingResumeOffset()` in `transaction_log_recovery.cpp`, which the
+* open-time scan and the timestamp index use to keep walking past the same break.
 */
 function findResyncPosition(dataView, from, dataEnd) {
 	const lastStart = dataEnd - TRANSACTION_LOG_ENTRY_HEADER_SIZE;
@@ -3803,7 +3861,7 @@ function loadLastPosition(transactionLog, readUncommitted) {
 //#region src/index.ts
 const versions = {
 	rocksdb: version,
-	"rocksdb-js": "2.9.1"
+	"rocksdb-js": "2.10.0"
 };
 
 //#endregion
@@ -3826,6 +3884,7 @@ exports.parseTransactionLog = parseTransactionLog;
 exports.registryStatus = registryStatus;
 exports.shutdown = shutdown;
 exports.stats = stats;
+exports.steadyClockNow = steadyClockNow;
 exports.supportedCompression = supportedCompression;
 exports.tryFileLock = tryFileLock;
 exports.validateTransactionLogStore = validateTransactionLogStore;