@harperfast/rocksdb-js 2.9.0 → 2.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +170 -18
- package/dist/index.cjs +85 -12
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +66 -6
- package/dist/index.d.mts +66 -6
- package/dist/index.mjs +85 -13
- package/dist/index.mjs.map +1 -1
- package/docs/backups.md +3 -2
- package/docs/performance/mutex-contention-2026-09-16.json +684 -0
- package/docs/performance/mutex-contention-admission-2026-09-17.json +730 -0
- package/docs/performance/mutex-contention.md +204 -0
- package/docs/stats.md +47 -46
- package/docs/steady-clock-design.md +121 -0
- package/package.json +13 -13
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
# Mutex contention investigation (#462)
|
|
2
|
+
|
|
3
|
+
Measured on 2026-09-16 against main `b4d104562e7d5353b6d7a4412e0a8b48ee805062`:
|
|
4
|
+
macOS 26.6.2 arm64, 14 logical CPUs, 36 GiB RAM, Node 24.16.0, RocksDB 11.8.1,
|
|
5
|
+
Release native binding, local APFS storage. This is a local workload study, not a
|
|
6
|
+
cross-platform capacity claim.
|
|
7
|
+
|
|
8
|
+
## Finding and change
|
|
9
|
+
|
|
10
|
+
The strongest steady-state contention was the descriptor's shared commit-completion
|
|
11
|
+
mutex. It serialized registration, dispatch, and accounting for otherwise independent
|
|
12
|
+
worker environments. Each environment now has a shared completion object with its own
|
|
13
|
+
mutex. A DBHandle caches that object on its owning JS thread; each dispatched commit
|
|
14
|
+
retains it until completion. Reopening the same native handle resets the cache.
|
|
15
|
+
|
|
16
|
+
This preserves mutual exclusion between TSFN calls and env cleanup. Atomics on the
|
|
17
|
+
pending count or TSFN pointer alone cannot keep Node from freeing the TSFN during a
|
|
18
|
+
call. Cold release paths retain the registry lock until the completion is released,
|
|
19
|
+
so a concurrent env-cleanup hook cannot miss an entry being released. The sole nested
|
|
20
|
+
lock order is registry → completion. Commit queue order, operation admission/drain,
|
|
21
|
+
log append serialization, and the legacy libuv mode are unchanged.
|
|
22
|
+
|
|
23
|
+
The first experiment retained a registry lookup for every registration. It improved
|
|
24
|
+
eight-worker log throughput but added a second registration lock and regressed the
|
|
25
|
+
single-worker case. Caching at the existing DBHandle owner removes that extra hot-path
|
|
26
|
+
lock. Fixed hash buckets were considered and rejected after independent planning
|
|
27
|
+
review: collisions leave unrelated environments serialized, and a fixed set of maps
|
|
28
|
+
and mutexes adds memory to every database, including idle databases.
|
|
29
|
+
|
|
30
|
+
## Profiling evidence
|
|
31
|
+
|
|
32
|
+
Apple `sample` collected native stacks at a requested 1 ms interval. The sustained
|
|
33
|
+
baseline driver used four workers, eight outstanding transactions per worker, and
|
|
34
|
+
100-byte log entries, with WAL disabled and no database key writes. It completed
|
|
35
|
+
2,958,027 transactions in 10.0004 seconds; the log's transaction counter matched the
|
|
36
|
+
reported completion count. Its profiled throughput is not the comparison baseline.
|
|
37
|
+
|
|
38
|
+
| Nearest binding ancestor of a kernel mutex wait | Thread samples |
|
|
39
|
+
| ----------------------------------------------- | -------------: |
|
|
40
|
+
| `registerCommitCompletion` | 2,588 |
|
|
41
|
+
| `finishCommitCompletion` | 2,374 |
|
|
42
|
+
| `dispatchCommitCompletion` | 230 |
|
|
43
|
+
| Transaction registry add/get/remove | 2,529 |
|
|
44
|
+
| `TransactionHandle::addLogEntry` | 97 |
|
|
45
|
+
| Commit queue enqueue/run | 174 |
|
|
46
|
+
| Other | 89 |
|
|
47
|
+
| Total | 8,081 |
|
|
48
|
+
|
|
49
|
+
Commit-completion paths account for 64.25% of these mutex-wait samples. This is **not**
|
|
50
|
+
64.25% of elapsed time, CPU usage, or lock acquisitions. Threads are sampled separately;
|
|
51
|
+
condition-variable sleeps, event-loop waits, and `writev` are separate categories.
|
|
52
|
+
No kernel mutex wait was sampled inside the store's `writeBatch` on this workload.
|
|
53
|
+
Short uncontended locks can still have a cost below the sampling resolution.
|
|
54
|
+
|
|
55
|
+
A separate profile of existing worker-put, worker-log, realistic-load and tight-log
|
|
56
|
+
benchmarks showed the same completion/transaction-registry paths, plus startup
|
|
57
|
+
serialization in `DBRegistry::OpenDB`. The full stress profile was dominated by open/close
|
|
58
|
+
registry locking during its many-handle workload. Those setup/teardown waits must not
|
|
59
|
+
be mistaken for steady-state transaction-log append contention.
|
|
60
|
+
|
|
61
|
+
The candidate profile on the same four-worker/8-outstanding, 10-second workload
|
|
62
|
+
completed 2,917,655 transactions. Completion register/dispatch/finish accounted for
|
|
63
|
+
128 of 2,373 kernel mutex-wait thread samples (5.4%), with transaction add/get/remove
|
|
64
|
+
now accounting for 1,935. The baseline and candidate collected 7,620 and 7,428
|
|
65
|
+
main-thread samples respectively; raw totals are not normalized timing estimates.
|
|
66
|
+
This supports the intended removal of cross-env completion contention, while the
|
|
67
|
+
separate unprofiled trials below determine the throughput claim.
|
|
68
|
+
|
|
69
|
+
## Paired throughput comparison
|
|
70
|
+
|
|
71
|
+
Five independent 3-second trials per version/workload; baseline/candidate order alternates by repetition. Each trial counts completed transactions. Runs were unprofiled and did not overlap tests or builds. The baseline is the supplied Release binding from the base checkout; the candidate was rebuilt locally. Binary SHA-256 identities and every trial are retained in [the raw results](mutex-contention-2026-09-16.json), where candidate trials use the label `cached`.
|
|
72
|
+
|
|
73
|
+
The measured candidate source was `b7a23da2`. Through `f9f87aa5`, production changes
|
|
74
|
+
after that measurement were comments and the `completionStateMutex` member rename.
|
|
75
|
+
The subsequent admission-order fix is described below; the table is the original
|
|
76
|
+
measurement, not a fresh measurement of that fix.
|
|
77
|
+
|
|
78
|
+
| Workload | Base median txns/s (range) | Candidate median txns/s (range) | Median change |
|
|
79
|
+
| ---------------- | -------------------------: | ------------------------------: | ------------: |
|
|
80
|
+
| log, 1 worker(s) | 285,814 (268,902–296,796) | 296,388 (287,185–299,151) | +3.7% |
|
|
81
|
+
| log, 4 worker(s) | 339,454 (332,123–350,653) | 329,680 (320,238–346,135) | -2.9% |
|
|
82
|
+
| log, 8 worker(s) | 185,836 (178,970–188,273) | 258,480 (255,425–284,300) | +39.1% |
|
|
83
|
+
| put, 4 worker(s) | 475,821 (463,921–485,965) | 500,384 (478,962–507,948) | +5.2% |
|
|
84
|
+
|
|
85
|
+
The eight-worker log result improves substantially across all five trials. The single-worker and four-worker put results are smaller gains. **Four-worker log throughput is 2.9% lower at the median**, with overlapping ranges; this is not a universal speedup. Treat the small differences as workload/host-sensitive and the eight-worker result as the strongest evidence. These ranges describe the observed trials, not confidence intervals. No p99 latency claim is made.
|
|
86
|
+
|
|
87
|
+
The cost is one cached shared pointer per database handle, one shared pointer per in-flight native commit, and a mutex-bearing object per active descriptor/environment pair. These references do not keep a Node environment alive; TSFN pending accounting retains that responsibility.
|
|
88
|
+
|
|
89
|
+
### Admission-order fix follow-up
|
|
90
|
+
|
|
91
|
+
On 2026-09-17 UTC, the same driver compared locally built Release bindings from
|
|
92
|
+
`f9f87aa5` (before the admission-order fix) and `16a98614` (after it), on the same
|
|
93
|
+
host described above. Five alternating 3-second trials per version/workload used
|
|
94
|
+
eight outstanding transactions per worker, with no overlapping tests or builds.
|
|
95
|
+
[All 40 trials and binary identities](mutex-contention-admission-2026-09-17.json)
|
|
96
|
+
are retained separately from the original main-versus-candidate measurements.
|
|
97
|
+
|
|
98
|
+
| Workload | Before fix median txns/s (range) | After fix median txns/s (range) | Median change |
|
|
99
|
+
| -------------- | -------------------------------: | ------------------------------: | ------------: |
|
|
100
|
+
| log, 1 worker | 303,080 (295,717–309,402) | 302,419 (300,443–303,889) | -0.2% |
|
|
101
|
+
| log, 4 workers | 359,899 (355,036–364,464) | 354,747 (344,170–356,092) | -1.4% |
|
|
102
|
+
| log, 8 workers | 269,078 (257,075–273,362) | 283,706 (276,600–288,097) | +5.4% |
|
|
103
|
+
| put, 4 workers | 509,108 (502,560–547,630) | 526,417 (495,881–547,885) | +3.4% |
|
|
104
|
+
|
|
105
|
+
This measures the incremental fix, including the disabled test seam's relaxed
|
|
106
|
+
atomic load. It does not remeasure the optimization against main. Results are
|
|
107
|
+
mixed, with overlapping ranges except for eight-worker logs; small differences
|
|
108
|
+
remain workload/host-sensitive. No general speedup is attributed to reordering
|
|
109
|
+
admission, and these changes must not be added to the original percentages.
|
|
110
|
+
|
|
111
|
+
## Remaining synchronization decisions
|
|
112
|
+
|
|
113
|
+
| Area | Decision and reason |
|
|
114
|
+
| --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
115
|
+
| `TransactionLogStore::writeMutex` | Retain: append bytes, rotation, failure retirement and log-position publication are one ordered protocol. The default commit lane already serializes async appends; synchronous commits and legacy mode still need exclusion. |
|
|
116
|
+
| Store `dataSetsMutex` | Retain: protects the file map, uncommitted-position collection and coherent current/frozen mapping transition. An atomic sequence number does not make those collections or their lifetimes safe. |
|
|
117
|
+
| Store `transactionBindMutex` | Retain: binding, pending accounting and the closing decision form a multi-field protocol. It was a small share of sampled contention; any combined atomic state needs its own lifecycle proof and measurements. |
|
|
118
|
+
| Store `flushedStateMutex` | Retain: serializes the stream and offset of `txn.state`, including I/O; a scalar atomic cannot replace stream ownership. |
|
|
119
|
+
| Transaction/file registries and `txnsMutex` | Remaining measurable candidate. Maps and shared object acquisition require synchronization. Consider reducing transaction-ID lookups on the log-entry path before attempting lock-free reclamation. |
|
|
120
|
+
| File `fileMutex` / `indexMutex` | Retain: own handles, mappings and index construction. Read/rotate/purge coherence is more than an atomic size field. |
|
|
121
|
+
| VT writer mutex | Retain: protects tracker acquisition and reclamation as well as slot transitions. An atomic slot pointer alone permits a reader to acquire a freed tracker. |
|
|
122
|
+
| Event/lock callback mutexes | Retain: serialize TSFN dispatch with environment cleanup. This is the same lifetime invariant as commit completions. |
|
|
123
|
+
| Commit queues, park timeouts, watchdogs | Already use condition variables with queue/deadline/stop predicates. Sleeping background threads are expected, not evidence for spinning. |
|
|
124
|
+
| Descriptor operation drain | Already uses C++20 atomic wait/notify. |
|
|
125
|
+
| Async-work drain | Uses an atomic count plus timed condition-variable waits. Any polling removal must preserve bounded shutdown and close/admission behavior; the broader lifecycle work is tracked in #784. |
|
|
126
|
+
| Settings, iterator close, backup stream and OS file locks | No measured reason here to change their ownership/exclusion contracts. Kernel backup locks also coordinate processes, which in-process atomics cannot do. |
|
|
127
|
+
|
|
128
|
+
The practical action is to reduce the scope of a contended lock while keeping its
|
|
129
|
+
lifetime guarantee. Broad replacement of mutexes with atomics is not justified by
|
|
130
|
+
these profiles.
|
|
131
|
+
|
|
132
|
+
## Reproducing the throughput workload
|
|
133
|
+
|
|
134
|
+
After building the bundle and Release binding, run from the repository root:
|
|
135
|
+
|
|
136
|
+
```sh
|
|
137
|
+
node benchmark/mutex-contention.mts --workers=4 --seconds=3 --mode=log
|
|
138
|
+
node benchmark/mutex-contention.mts --workers=4 --seconds=3 --mode=put
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
`--concurrency` defaults to 8 per worker. `log` appends one 100-byte entry per transaction;
|
|
142
|
+
`put` updates a distinct key per worker/lane with a 100-byte value. Both use async
|
|
143
|
+
transactions with `disableWAL: true`. The driver awaits every counted transaction,
|
|
144
|
+
checks log totals or final values, and emits JSON with count, elapsed seconds,
|
|
145
|
+
throughput and process CPU time. Worker startup/open and final verification/teardown
|
|
146
|
+
are outside the timed interval; there is no separate JIT warmup. Each trial uses a new
|
|
147
|
+
database on the repository's volume. These workloads do not measure fsync durability
|
|
148
|
+
latency or combined database-write-plus-log transactions.
|
|
149
|
+
|
|
150
|
+
The existing Vitest worker benchmarks use worker messages for iterations, and their
|
|
151
|
+
`concurrent()` wrapper can return with operations outstanding. Their displayed Hz is
|
|
152
|
+
benchmark iterations per second, not directly comparable to the driver's completed
|
|
153
|
+
transactions per second. They remain useful representative smoke/comparison workloads;
|
|
154
|
+
use the driver for explicit completed-operation counts.
|
|
155
|
+
|
|
156
|
+
## Stability scope and follow-up
|
|
157
|
+
|
|
158
|
+
Passing stress tests and a sampled profile do not prove the absence of races or
|
|
159
|
+
deadlocks. The changed protocol still requires its lock-order and env-lifetime proof.
|
|
160
|
+
Linux/Windows/runtime coverage belongs to CI; no ThreadSanitizer result is claimed from
|
|
161
|
+
this macOS investigation.
|
|
162
|
+
|
|
163
|
+
Validation at `16a98614`: `pnpm test` passed 70 files / 1,010 tests (8 skipped);
|
|
164
|
+
`pnpm test:native` passed 221 tests (3 platform-specific skips); `pnpm test:stress`
|
|
165
|
+
passed all 5 files / 9 tests; `pnpm check` passed. Same-environment multi-handle
|
|
166
|
+
accounting and reopen tests ran across all four transaction-option variants.
|
|
167
|
+
The teardown fixture also verified that a live
|
|
168
|
+
environment's commit completes while another worker environment is terminated.
|
|
169
|
+
|
|
170
|
+
A separate native reproduction confirmed [#860](https://github.com/HarperFast/rocksdb-js/issues/860):
|
|
171
|
+
`tryClose()` releases its written-position lock before latching admission closed.
|
|
172
|
+
A transaction can bind, finish `writeBatch()` and reduce the pending count back to zero
|
|
173
|
+
between those checks. A temporary scheduling hook before phase 3 reproduced
|
|
174
|
+
`pending=0`, one real uncommitted position, and `tryClose() == true`. This is a
|
|
175
|
+
pre-existing close-protocol defect, not fixed by the completion-lock change. Its
|
|
176
|
+
production trigger is a concurrent destructive log purge; no customer incident or
|
|
177
|
+
JavaScript-level data loss was reproduced. Keep the binding lock until a separately
|
|
178
|
+
reviewed admission/drain fix resolves that handoff.
|
|
179
|
+
|
|
180
|
+
The investigation also found [#859](https://github.com/HarperFast/rocksdb-js/issues/859):
|
|
181
|
+
the DB-instance stress test awaits an empty promise array in its worker-close phase.
|
|
182
|
+
That test passed locally, but its close acknowledgement gap limits what the pass proves.
|
|
183
|
+
The logged-transaction stress test's title says 10k while its worker count is 1,000;
|
|
184
|
+
results should be interpreted using the executed count (30,000 total).
|
|
185
|
+
|
|
186
|
+
Outside review also identified [#861](https://github.com/HarperFast/rocksdb-js/issues/861):
|
|
187
|
+
the native teardown test's parent does not kill/reap a hung fixture when Vitest times
|
|
188
|
+
out. The successful run here does not exercise that failure path. The broader
|
|
189
|
+
transaction close/admission contract remains [#784](https://github.com/HarperFast/rocksdb-js/issues/784).
|
|
190
|
+
The completion-cache dereference must follow operation admission to avoid a crash
|
|
191
|
+
within that window. A deterministic child/worker regression pauses the commit after capturing its
|
|
192
|
+
descriptor, completes foreign `shutdown()`, then resumes admission. The old PR head
|
|
193
|
+
`f9f87aa5`, with only the test seam added, crashed with SIGSEGV in all four combinations
|
|
194
|
+
of cold/warm cache and single/two-lane mode. Admission now publishes the descriptor
|
|
195
|
+
operation count and checks closing before accessing the cache. A winning shutdown
|
|
196
|
+
therefore rejects before dereferencing the reset handle; a winning commit makes shutdown
|
|
197
|
+
wait. Setup cleanup also retains the descriptor pin until its operation count is released.
|
|
198
|
+
This fixes the new cache access, while earlier transaction-entry reads and the broader
|
|
199
|
+
#784 admission contract remain outside this patch's guarantee.
|
|
200
|
+
|
|
201
|
+
The separate read-heavy work in [#545](https://github.com/HarperFast/rocksdb-js/pull/545)
|
|
202
|
+
and its baseline [#546](https://github.com/HarperFast/rocksdb-js/pull/546) was not evaluated
|
|
203
|
+
by this change. Those subscriber/short-range workloads are a distinct follow-up to the
|
|
204
|
+
commit-side profile here.
|
package/docs/stats.md
CHANGED
|
@@ -32,52 +32,53 @@ const keysWrittenHistogram: StatsHistogramData = db.getStat('rocksdb.db.flush.mi
|
|
|
32
32
|
|
|
33
33
|
By default, `db.getStats()` returns properties only.
|
|
34
34
|
|
|
35
|
-
| Name | Description
|
|
36
|
-
| ------------------------------------------- |
|
|
37
|
-
| `
|
|
38
|
-
| `commitPipeline.
|
|
39
|
-
| `
|
|
40
|
-
| `rocksdb.block-cache-
|
|
41
|
-
| `rocksdb.block-cache-usage`
|
|
42
|
-
| `rocksdb.
|
|
43
|
-
| `rocksdb.
|
|
44
|
-
| `rocksdb.cur-size-
|
|
45
|
-
| `rocksdb.
|
|
46
|
-
| `rocksdb.
|
|
47
|
-
| `rocksdb.estimate-
|
|
48
|
-
| `rocksdb.estimate-
|
|
49
|
-
| `rocksdb.
|
|
50
|
-
| `rocksdb.live-
|
|
51
|
-
| `rocksdb.
|
|
52
|
-
| `rocksdb.
|
|
53
|
-
| `rocksdb.num-
|
|
54
|
-
| `rocksdb.num-
|
|
55
|
-
| `rocksdb.num-
|
|
56
|
-
| `rocksdb.num-immutable-mem-table
|
|
57
|
-
| `rocksdb.num-
|
|
58
|
-
| `rocksdb.num-
|
|
59
|
-
| `rocksdb.num-running-
|
|
60
|
-
| `rocksdb.num-
|
|
61
|
-
| `rocksdb.
|
|
62
|
-
| `rocksdb.
|
|
63
|
-
| `rocksdb.
|
|
64
|
-
| `rocksdb.total-
|
|
65
|
-
| `
|
|
66
|
-
| `txnlog.
|
|
67
|
-
| `txnlog.
|
|
68
|
-
| `txnlog.
|
|
69
|
-
| `txnlog.
|
|
70
|
-
| `txnlog.
|
|
71
|
-
| `txnlog.
|
|
72
|
-
| `txnlog.
|
|
73
|
-
| `txnlog.
|
|
74
|
-
| `txnlog.
|
|
75
|
-
| `txnlog.
|
|
76
|
-
| `
|
|
77
|
-
| `writeBufferManager.
|
|
78
|
-
| `writeBufferManager.
|
|
79
|
-
| `writeBufferManager.
|
|
80
|
-
| `writeBufferManager.
|
|
35
|
+
| Name | Description | Type |
|
|
36
|
+
| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
|
|
37
|
+
| `columnFamily.pendingReclaims` | Number of dropped column families of this database whose physical `DropColumnFamily` has not completed: a commit admitted before the drop still holds it, or the drop failed and is retried on the next drop, open of that name, or close (see "Dropping column families" in the README). | gauge |
|
|
38
|
+
| `commitPipeline.commitQueueDepth` | Number of async commits queued on the database's commit lane but not yet started (in the default single-lane mode this covers the whole commit: log write + RocksDB commit). | gauge |
|
|
39
|
+
| `commitPipeline.logQueueDepth` | Number of async commits queued on the database's transaction-log lane but not yet started (always `0` in the default single-lane mode; see `ROCKSDB_JS_COMMIT_THREAD=2`). | gauge |
|
|
40
|
+
| `rocksdb.block-cache-capacity` | Capacity in bytes of the block cache. | gauge |
|
|
41
|
+
| `rocksdb.block-cache-pinned-usage` | Bytes occupied by pinned block cache entries. | gauge |
|
|
42
|
+
| `rocksdb.block-cache-usage` | Bytes currently used by block cache entries. | gauge |
|
|
43
|
+
| `rocksdb.compaction-pending` | `1` if at least one compaction is pending, else `0`. | gauge |
|
|
44
|
+
| `rocksdb.cur-size-active-mem-table` | Approximate size in bytes of the active memtable. | gauge |
|
|
45
|
+
| `rocksdb.cur-size-all-mem-tables` | Approximate size in bytes of the active and unflushed immutable memtables. | gauge |
|
|
46
|
+
| `rocksdb.current-super-version-number` | Current super-version number, incremented on each LSM change. | gauge |
|
|
47
|
+
| `rocksdb.estimate-live-data-size` | Estimated size in bytes of the live data. | gauge |
|
|
48
|
+
| `rocksdb.estimate-num-keys` | Estimated number of live keys. | gauge |
|
|
49
|
+
| `rocksdb.estimate-pending-compaction-bytes` | Estimated bytes compaction must rewrite to rebalance the LSM tree. | gauge |
|
|
50
|
+
| `rocksdb.live-blob-file-size` | Total size in bytes of blob files in the current version. | gauge |
|
|
51
|
+
| `rocksdb.live-sst-files-size` | Total size in bytes of SST files in the current version. | gauge |
|
|
52
|
+
| `rocksdb.mem-table-flush-pending` | `1` if a memtable flush is pending, else `0`. | gauge |
|
|
53
|
+
| `rocksdb.num-blob-files` | Number of blob files in the current version. | gauge |
|
|
54
|
+
| `rocksdb.num-deletes-active-mem-table` | Number of delete entries in the active memtable. | gauge |
|
|
55
|
+
| `rocksdb.num-entries-active-mem-table` | Number of entries in the active memtable. | gauge |
|
|
56
|
+
| `rocksdb.num-immutable-mem-table` | Number of immutable memtables not yet flushed. | gauge |
|
|
57
|
+
| `rocksdb.num-immutable-mem-table-flushed` | Number of immutable memtables already flushed. | gauge |
|
|
58
|
+
| `rocksdb.num-live-versions` | Number of live LSM versions (high values indicate versions held by iterators/snapshots). | gauge |
|
|
59
|
+
| `rocksdb.num-running-compactions` | Number of compactions currently running. | gauge |
|
|
60
|
+
| `rocksdb.num-running-flushes` | Number of flushes currently running. | gauge |
|
|
61
|
+
| `rocksdb.num-snapshots` | Number of unreleased snapshots. Any nonzero value blocks reclamation of obsolete versions behind the oldest one. | gauge |
|
|
62
|
+
| `rocksdb.oldest-snapshot-time` | Unix timestamp of the oldest unreleased snapshot, or `0`. | gauge |
|
|
63
|
+
| `rocksdb.size-all-mem-tables` | Approximate size in bytes of active, unflushed, and pinned memtables. | gauge |
|
|
64
|
+
| `rocksdb.total-blob-file-size` | Total size in bytes of all blob files across all versions. | gauge |
|
|
65
|
+
| `rocksdb.total-sst-files-size` | Total size in bytes of all SST files across all versions. | gauge |
|
|
66
|
+
| `txnlog.activeMaps` | Number of transaction log files currently memory-mapped, summed across all logs. | gauge |
|
|
67
|
+
| `txnlog.bytesWritten` | Cumulative bytes written to transaction logs (entry payload plus entry header, excluding file headers), summed across all logs. | ticker |
|
|
68
|
+
| `txnlog.fileCount` | Total number of transaction log files on disk, summed across all logs. | gauge |
|
|
69
|
+
| `txnlog.logCount` | Number of transaction log stores attached to the database. | gauge |
|
|
70
|
+
| `txnlog.mappedBytes` | Virtual address space in bytes reserved for memory-mapped log files, summed across all logs (on POSIX the active write file maps the full configured `maxFileSize`, so this is allocated address space, not resident memory). | gauge |
|
|
71
|
+
| `txnlog.overlayBytes` | File-backed overlay portion in bytes (POSIX only; `0` on Windows), summed across all logs — a closer proxy for real memory consumption than `mappedBytes`. | gauge |
|
|
72
|
+
| `txnlog.pendingTransactions` | Number of transactions bound to a log but not yet written via `writeBatch()`, summed across all logs. | gauge |
|
|
73
|
+
| `txnlog.replayGapBytes` | Bytes between the last flushed position and the write head (written but not yet flushed to the database), summed across all logs. | gauge |
|
|
74
|
+
| `txnlog.totalSizeBytes` | Total on-disk size in bytes of all transaction log files, summed across all logs. | gauge |
|
|
75
|
+
| `txnlog.transactionsWritten` | Cumulative number of transactions successfully written to the logs (lifetime total), summed across all logs. | ticker |
|
|
76
|
+
| `txnlog.uncommittedTransactions` | Number of transactions written to a log but not yet committed to RocksDB, summed across all logs. | gauge |
|
|
77
|
+
| `writeBufferManager.bufferSize` | The `WriteBufferManager`'s memtable-memory budget in bytes, read live. `0` when no manager is configured. **Process-wide**, not per-database — see below. | gauge |
|
|
78
|
+
| `writeBufferManager.memoryUsage` | Total memtable memory in bytes charged against the manager. **Process-wide.** | gauge |
|
|
79
|
+
| `writeBufferManager.mutableMemoryUsage` | The share of `writeBufferManager.memoryUsage` held by active (mutable) memtables; the rest is awaiting flush or retained write history. **Process-wide.** | gauge |
|
|
80
|
+
| `writeBufferManager.stallActive` | `1` while the manager is stalling writes, else `0`. **Process-wide.** | gauge |
|
|
81
|
+
| `writeBufferManager.stallActiveMs` | How long the current stall has been active, in milliseconds; `0` when not stalled. Sampled once a second by the stall watchdog. **Process-wide.** | gauge |
|
|
81
82
|
|
|
82
83
|
#### The `writeBufferManager.*` keys are process-wide
|
|
83
84
|
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
# Process-wide steady clock
|
|
2
|
+
|
|
3
|
+
## Intent and invariant
|
|
4
|
+
|
|
5
|
+
HarperFast/harper#2658 needs to compare coverage boundaries between workers and measure elapsed
|
|
6
|
+
lag and deadlines independently of wall-clock steps. Every sample must share one steady temporal
|
|
7
|
+
domain within the process, including workers created later. Suspend behavior is platform-defined;
|
|
8
|
+
this API does not promise a portable suspend-inclusive deadline.
|
|
9
|
+
|
|
10
|
+
Bun 1.3.14/1.4.0 worker-local runtime clocks (`process.hrtime.bigint()`, `process.uptime()`,
|
|
11
|
+
`performance.now()` and `Bun.nanoseconds()`) do not provide this common domain (the motivating
|
|
12
|
+
probe read about 2.1 seconds in the parent and 12 milliseconds in a new worker). The existing
|
|
13
|
+
`getMonotonicTimestamp()` reads `system_clock` and ratchets ties/rollback with `nextafter`
|
|
14
|
+
(`src/binding/core/platform.cpp`). It orders transaction timestamps but can stall after rollback.
|
|
15
|
+
That epoch-based contract and all transaction/log paths must remain unchanged.
|
|
16
|
+
|
|
17
|
+
## Chosen
|
|
18
|
+
|
|
19
|
+
Expose `steadyClockNow(): number` at the module root, requiring no database handle. The Node-free native helpers `getSteadyClockNow()` and `steadyClockMilliseconds(duration)`
|
|
20
|
+
allow deterministic conversion tests to call the production conversion. The former reads `std::chrono::steady_clock::now().time_since_epoch()` and converts its duration to
|
|
21
|
+
`std::chrono::duration<double, std::milli>`. There is no per-worker initialization, shared mutable
|
|
22
|
+
state, addon lock, wall-clock read, database/log I/O or native allocation. The N-API callback checks
|
|
23
|
+
`napi_create_double` with `NAPI_STATUS_THROWS`; engines may allocate a boxed JS number.
|
|
24
|
+
|
|
25
|
+
Values are fractional milliseconds from an unspecified origin fixed throughout the process.
|
|
26
|
+
Comparisons are meaningful across all workers of that process, but not across processes, restarts
|
|
27
|
+
or hosts, nor against epoch/transaction timestamps. Samples are non-decreasing, not unique.
|
|
28
|
+
Equality cannot establish strict ordering: treat equal boundaries as "not later", or use a separate
|
|
29
|
+
sequence number. No atomic ratchet is applied to this clock.
|
|
30
|
+
|
|
31
|
+
### Representation, range and precision
|
|
32
|
+
|
|
33
|
+
`number` matches this package's millisecond duration APIs and avoids bigint arithmetic for the
|
|
34
|
+
consumer's 3000 ms lag bound. Bigint nanoseconds would retain every represented native tick, but
|
|
35
|
+
this API permits equality and does not require nanosecond-exact identities. Neither representation
|
|
36
|
+
is promised allocation-free at the JS boundary.
|
|
37
|
+
|
|
38
|
+
The selected standard libraries use signed 64-bit nanosecond durations (about 292 years in either
|
|
39
|
+
direction from their origin). Conversion to double does not narrow that range. Positive conversion
|
|
40
|
+
and scaling preserve non-decreasing order, though adjacent ticks can collapse to equality.
|
|
41
|
+
The output spacing is about 1.9 ns at 100 days from the origin, 61 ns at 10 years and 0.49 µs at
|
|
42
|
+
100 years. Integer-to-double rounding also contributes conversion error: at 100 years its half-ulp
|
|
43
|
+
is 256 ns, followed by up to 244 ns of millisecond rounding, less than 0.51 µs total per sample.
|
|
44
|
+
This is representation precision, not an accuracy or resolution guarantee. Precision depends on
|
|
45
|
+
distance from the native origin, not time since JS startup.
|
|
46
|
+
|
|
47
|
+
### Supported platforms
|
|
48
|
+
|
|
49
|
+
Node, Bun and Deno all call the same compiled native function on Linux, macOS and Windows.
|
|
50
|
+
The standard library owns the clock source; runtime VM startup does not reset it.
|
|
51
|
+
|
|
52
|
+
- Linux libstdc++ uses `CLOCK_MONOTONIC`, which excludes suspend and can be frequency-slewed by NTP.
|
|
53
|
+
[GCC implementation](https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/src/c%2B%2B11/chrono.cc)
|
|
54
|
+
- Apple libc++ uses `CLOCK_MONOTONIC_RAW`, which includes suspend on macOS. Do not infer Apple's
|
|
55
|
+
semantics from the identically named Linux clock.
|
|
56
|
+
[libc++ implementation](https://github.com/llvm/llvm-project/blob/main/libcxx/src/chrono.cpp)
|
|
57
|
+
- Windows MSVC uses `QueryPerformanceCounter` converted to nanoseconds; the counter frequency
|
|
58
|
+
determines actual resolution (a common 10 MHz counter ticks every 100 ns, but the C++ duration's
|
|
59
|
+
period is 1 ns). QPC includes standby/hibernate and is independent of UTC adjustments.
|
|
60
|
+
[MSVC implementation](https://github.com/microsoft/STL/blob/main/stl/inc/__msvc_chrono.hpp),
|
|
61
|
+
[QPC contract](https://learn.microsoft.com/en-us/windows/win32/sysinfo/acquiring-high-resolution-time-stamps)
|
|
62
|
+
|
|
63
|
+
Windows documents ±1 QPC tick of ordering ambiguity between threads. Near-equal values cannot
|
|
64
|
+
prove causality; strict coverage handoff needs synchronization or a separate sequence, even when
|
|
65
|
+
numeric samples differ. The accessor does not hide hardware uncertainty behind a synthetic ratchet.
|
|
66
|
+
|
|
67
|
+
These sources were inspected on 2026-09-17. None supplies a portable nanosecond accuracy guarantee.
|
|
68
|
+
Wall-clock steps do not affect these sources; frequency adjustment and suspend semantics are
|
|
69
|
+
separate properties. No promise is made that runtime timers use the same suspend policy.
|
|
70
|
+
|
|
71
|
+
## Approaches considered
|
|
72
|
+
|
|
73
|
+
- **Different layer:** repair Bun or calibrate clocks in Harper. A Bun fix is unavailable to
|
|
74
|
+
already-deployed runtimes; wall-time calibration reintroduces step sensitivity and an independent
|
|
75
|
+
cross-worker calibration protocol adds synchronization error to every boundary.
|
|
76
|
+
- **Deeper cause:** make `getMonotonicTimestamp()` steady. Its values are durable epoch transaction
|
|
77
|
+
identities and log keys; changing their domain breaks the persisted/replay contract. Another
|
|
78
|
+
alternative is explicit suspend-inclusive OS clocks, but that changes the requested portable
|
|
79
|
+
`steady_clock` contract and requires separate clock selection on each platform.
|
|
80
|
+
- **Do less:** reuse the wall-clock ratchet or enforce only strict coverage. The former stalls
|
|
81
|
+
elapsed lag after rollback; the latter discards the approved bounded-lag behavior. Runtime-local
|
|
82
|
+
timers alone cannot compare two workers' coverage samples on Bun.
|
|
83
|
+
- **Chosen:** additive native steady accessor. One native domain supplies the missing cross-worker
|
|
84
|
+
elapsed measurement without altering durable identities. Fractional milliseconds preserve far
|
|
85
|
+
more precision than the consumer's millisecond budget; equality is explicitly not strict order.
|
|
86
|
+
|
|
87
|
+
## Verification
|
|
88
|
+
|
|
89
|
+
The end-to-end route is the real N-API export through `src/index.ts`, parent/worker bracketing on
|
|
90
|
+
Node, Bun and Deno, staggered/restarted and concurrent workers (including `workerFirst >= parentBefore + stagger - tolerance`), elapsed
|
|
91
|
+
progression, and built ESM/CJS
|
|
92
|
+
exports. Temporarily substituting a worker-local clock must make those brackets fail, proving
|
|
93
|
+
they reject the motivating defect.
|
|
94
|
+
|
|
95
|
+
GoogleTest covers conversion boundaries (including long durations, adjacent ticks and equal
|
|
96
|
+
rounded samples), native thread brackets and elapsed progression. Native and JS tests interleave
|
|
97
|
+
steady reads with the unchanged transaction timestamp ratchet. Existing transaction and log suites
|
|
98
|
+
cover allocation/adoption, replay and ordering.
|
|
99
|
+
|
|
100
|
+
An opt-in Linux/Node test uses libfaketime in an isolated child, with monotonic clocks left real.
|
|
101
|
+
It steps the child's wall clock backward and forward and observes both `Date.now()` and the native
|
|
102
|
+
transaction timestamp. Steady deltas must still measure the intervening sleep. This exercises
|
|
103
|
+
native clock selection, not a JS Date stub; it never changes the host clock. CI installs libfaketime
|
|
104
|
+
for the Linux Node 24 run so this regression does not silently remain local-only.
|
|
105
|
+
|
|
106
|
+
Run this repository's full `pnpm build`, `pnpm check`, `pnpm test:native`, `pnpm test` and
|
|
107
|
+
`pnpm test:bun` gates. Local Windows/macOS execution is unavailable; their existing CI matrices
|
|
108
|
+
exercise native and runtime worker tests. Record actual results and limitations in the PR.
|
|
109
|
+
|
|
110
|
+
## Planning review and rollout
|
|
111
|
+
|
|
112
|
+
The initial design at `75caf59d` received `Framing-Verdict: chosen-approach-sound` from the prior
|
|
113
|
+
session's planning review. Its allocation, precision, built-export and within-process testing
|
|
114
|
+
findings were adopted. The resumed planning review also cleared `Framing-Verdict: chosen-approach-sound`
|
|
115
|
+
with Gemini and Cursor Composer. Its requests for a tested conversion helper, stagger-magnitude
|
|
116
|
+
assertions, CI wall-step coverage and README/Deno parity are included in this implementation. Source inspection corrected the earlier draft's Windows period and macOS suspend
|
|
117
|
+
claims; the API's platform-defined suspend contract is unchanged.
|
|
118
|
+
|
|
119
|
+
Harper must consume matching JS declarations/bundles and native prebuilds from a release containing
|
|
120
|
+
this change, or build this branch from source. Existing 2.9.1 binaries do not provide the export.
|
|
121
|
+
No package is published here and Harper consumer changes remain in HarperFast/harper#2658.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@harperfast/rocksdb-js",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.10.0",
|
|
4
4
|
"description": "RocksDB binding for Node.js",
|
|
5
5
|
"homepage": "https://github.com/HarperFast/rocksdb-js",
|
|
6
6
|
"bugs": [
|
|
@@ -38,15 +38,15 @@
|
|
|
38
38
|
},
|
|
39
39
|
"devDependencies": {
|
|
40
40
|
"@rollup/plugin-replace": "6.0.3",
|
|
41
|
-
"@types/node": "26.
|
|
41
|
+
"@types/node": "26.6.1",
|
|
42
42
|
"@vitest/coverage-v8": "4.1.11",
|
|
43
43
|
"cross-env": "10.1.0",
|
|
44
44
|
"dotenv": "17.4.2",
|
|
45
|
-
"lefthook": "2.1.
|
|
45
|
+
"lefthook": "2.1.14",
|
|
46
46
|
"lmdb": "3.5.6",
|
|
47
47
|
"node-gyp": "13.0.2",
|
|
48
|
-
"oxfmt": "0.
|
|
49
|
-
"oxlint": "1.
|
|
48
|
+
"oxfmt": "0.68.0",
|
|
49
|
+
"oxlint": "1.83.0",
|
|
50
50
|
"prebuildify": "6.0.1",
|
|
51
51
|
"semver": "7.8.5",
|
|
52
52
|
"tar": "7.5.22",
|
|
@@ -56,14 +56,14 @@
|
|
|
56
56
|
"vitest": "4.1.11"
|
|
57
57
|
},
|
|
58
58
|
"optionalDependencies": {
|
|
59
|
-
"@harperfast/rocksdb-js-darwin-arm64": "2.
|
|
60
|
-
"@harperfast/rocksdb-js-darwin-x64": "2.
|
|
61
|
-
"@harperfast/rocksdb-js-linux-arm64-glibc": "2.
|
|
62
|
-
"@harperfast/rocksdb-js-linux-arm64-musl": "2.
|
|
63
|
-
"@harperfast/rocksdb-js-linux-x64-glibc": "2.
|
|
64
|
-
"@harperfast/rocksdb-js-linux-x64-musl": "2.
|
|
65
|
-
"@harperfast/rocksdb-js-win32-arm64": "2.
|
|
66
|
-
"@harperfast/rocksdb-js-win32-x64": "2.
|
|
59
|
+
"@harperfast/rocksdb-js-darwin-arm64": "2.10.0",
|
|
60
|
+
"@harperfast/rocksdb-js-darwin-x64": "2.10.0",
|
|
61
|
+
"@harperfast/rocksdb-js-linux-arm64-glibc": "2.10.0",
|
|
62
|
+
"@harperfast/rocksdb-js-linux-arm64-musl": "2.10.0",
|
|
63
|
+
"@harperfast/rocksdb-js-linux-x64-glibc": "2.10.0",
|
|
64
|
+
"@harperfast/rocksdb-js-linux-x64-musl": "2.10.0",
|
|
65
|
+
"@harperfast/rocksdb-js-win32-arm64": "2.10.0",
|
|
66
|
+
"@harperfast/rocksdb-js-win32-x64": "2.10.0"
|
|
67
67
|
},
|
|
68
68
|
"engines": {
|
|
69
69
|
"node": "^22.18.0 || >=24.0.0"
|