@harperfast/rocksdb-js 2.11.0 → 2.12.0

package/README.md

@@ -2199,20 +2199,21 @@ const names = db.listLogs();
 
 ### `db.purgeLogs({ includeEntryCounts: true, ...options }): { path: string; entries: number }[]`
 
-Deletes transaction log files older than the `transactionLogRetention` (defaults to 3 days).
-Ordinary retention keeps the sequence file named by `txn.state` and every newer file as the live
-store's retention floor, or the highest sequence file when there is no persisted flush position. It
-removes only an eligible contiguous prefix below that floor. An idle store can therefore retain one
-file past the cutoff until a later write rotates and flushes it; that extra file is bounded by the
-store's `transactionLogMaxSize` (except when a single transaction exceeds the target). That bound
-covers only the floor file: when the flush position lags — `txn.state` stuck at an old sequence
-because RocksDB flushing is behind — every file above the floor is retained too, which
-`transactionLogMaxSize` does not bound. `purge.retainedUnflushedFiles` reports that case. Use
-`destroy: true` only to remove the store itself.
+Deletes transaction log files older than the `transactionLogRetention` (defaults to 3 days). A file
+is deleted only when it lies entirely at or before the flushed position recorded in `txn.state`, so
+every entry in it is already in RocksDB; that includes the file `txn.state` names and the file
+currently being written, so an idle, fully flushed store is left with no log files. Retention removes
+a contiguous prefix: the first file that is too new or holds an unflushed entry stops the run. When
+the flush position lags — `txn.state` stuck at an old sequence because RocksDB flushing is behind —
+every file after it is retained regardless of age; `purge.retainedUnflushedFiles` reports that case.
+While a backup with `transactionLogs` is copying a store's files, purges of that store are skipped
+(`purge.retentionPins` reports it). On a filesystem that rejects a directory fsync, the store's last
+file is kept, since `txn.state` could not be made durable before deleting it.
+Use `destroy: true` only to remove the store itself.
 
 - `options: object`
   - `before?: number` Remove transaction log files older than the specified timestamp, subject to
-    the retention floor and contiguous-prefix rules above.
+    the flushed-position and contiguous-prefix rules above.
   - `destroy?: boolean` When `true`, deletes transaction log stores including all log sequence files
     on disk.
   - `includeEntryCounts?: boolean` When `true`, counts the entries in each deleted log file and
@@ -2386,9 +2387,8 @@ stats.totals.transactionsWritten; // lifetime count of transactions written
 
 The `purge.retainedUnflushedFiles` gauge is useful for diagnosing why logs are not being cleaned
 up: a file can be older than the retention period but still retained because its transactions have
-not yet been flushed to RocksDB (purging it would be unsafe for crash recovery). The retention
-floor — the sequence named by `txn.state`, or the highest sequence when there is no persisted flush
-position — is not counted as purgeable even when it is old and fully flushed.
+not yet been flushed to RocksDB (purging it would be unsafe for crash recovery). An old, fully
+flushed current file counts as purgeable.
 
 ### Transaction Log Initialization
 
@@ -2789,7 +2789,7 @@ backup references them, so this is not equivalent to deleting files manually.
 
 Deletes all but the newest `keepCount` backups.
 
-### `backups.verify(backupDir: string, backupId: number, options?: { verifyWithChecksum?: boolean; verifyTransactionLogs?: boolean }): Promise<void>`
+### `backups.verify(backupDir: string, backupId: number, options?: { verifyWithChecksum?: boolean; verifyTables?: boolean; verifyTransactionLogs?: boolean }): Promise<void>`
 
 Verifies a backup's file sizes, and optionally their checksums (which requires reading all
 backed-up data). Resolves if the backup is intact and rejects otherwise.
@@ -2800,8 +2800,17 @@ When the backup was created with `transactionLogs: true`, the backup's transacti
 log file's header and entry framing must be intact (snapshots are copied on committed entry
 boundaries, so even a torn tail fails verification). Set `verifyTransactionLogs: false` to skip.
 
+Set `verifyTables: true` (default `false`) to also prove this build can read the backup's table
+files before you restore it. Each SST is opened in turn, one at a time, so file-descriptor use does
+not grow with the number of tables. Verification rejects, naming the file, when this build cannot
+decode a table's footer (for example a `format_version` written by a newer RocksDB) or its
+properties, or when the table records a compression algorithm this build does not support (see
+[`supportedCompression`](#supportedcompression)). Checksums cannot catch either case: the file is
+intact, just unreadable here, and RocksDB would otherwise fail on the first read after a restore.
+Blob files are not checked.
+
 ```typescript
-await backups.verify('/path/to/backups', 1, { verifyWithChecksum: true });
+await backups.verify('/path/to/backups', 1, { verifyWithChecksum: true, verifyTables: true });
 ```
 
 ## Custom Store

package/dist/index.cjs

@@ -550,6 +550,14 @@ const backups = {
 	* Verifies a backup's file sizes, and optionally their checksums (which
 	* requires reading all backed-up data).
 	*
+	* Set `verifyTables: true` to also prove this build can read the backup
+	* before restoring it: every table file is opened, one at a time, and its
+	* footer (including `format_version`) and properties must be readable, and
+	* every compression algorithm it records must be supported by this build.
+	* A checksum-valid table written by a newer or differently built RocksDB
+	* otherwise passes and fails on the first read after a restore. Blob files
+	* are not checked.
+	*
 	* When the backup was created with `transactionLogs: true`, the backup's
 	* transaction log snapshot (`<backupDir>/transaction_logs/<backupId>/`) is
 	* also validated — every log file's header and entry framing must be intact
@@ -557,7 +565,9 @@ const backups = {
 	* is a verification failure). Set `verifyTransactionLogs: false` to skip.
 	*/
 	async verify(backupDir, backupId, options) {
-		await new Promise((resolve, reject) => nativeBackupVerify(resolve, reject, backupDir, backupId, options?.verifyWithChecksum ?? false));
+		const verifyTables = options?.verifyTables;
+		if (verifyTables !== void 0 && typeof verifyTables !== "boolean") throw new TypeError(`verifyTables must be a boolean, received ${verifyTables === null ? "null" : typeof verifyTables}`);
+		await new Promise((resolve, reject) => nativeBackupVerify(resolve, reject, backupDir, backupId, options?.verifyWithChecksum ?? false, verifyTables === true));
 		if (options?.verifyTransactionLogs === false) return;
 		const logsDir = (0, node_path.join)(backupDir, TRANSACTION_LOGS_DIRNAME, String(backupId));
 		if (!await exists(logsDir)) return;
@@ -3975,7 +3985,7 @@ function loadLastPosition(transactionLog, readUncommitted) {
 //#region src/index.ts
 const versions = {
 	rocksdb: version,
-	"rocksdb-js": "2.11.0"
+	"rocksdb-js": "2.12.0"
 };
 
 //#endregion