@nxtedition/cache 2.1.13 → 2.1.15

package/README.md

@@ -1,13 +1,14 @@
 # @nxtedition/cache
 
-A two-tier async cache with SQLite persistence, in-memory pseudo-LRU, stale-while-revalidate, cross-process/thread deduplication, and automatic request coalescing.
+A two-tier async cache with SQLite persistence, in-memory pseudo-LRU, stale-while-revalidate, cross-thread deduplication via `SharedArrayBuffer`, and automatic request coalescing.
 
 ## Features
 
 - **Two-tier storage** — In-memory cache backed by SQLite on disk
+- **File-sharded SQLite** — Keys are hash-routed across N independent SQLite files, bypassing SQLite's per-file writer serialization. ~40% higher throughput under multi-thread write contention (see [Benchmarks](#benchmarks)).
 - **Stale-while-revalidate** — Serve stale data synchronously while refreshing in the background
-- **Request coalescing** — Concurrent fetches for the same key share a single in-flight request
-- **Cross-process/thread locking** — SQLite-based distributed locks with `Atomics`-based wakeup prevent redundant work across processes and worker threads
+- **Request coalescing** — Concurrent fetches for the same key share a single in-flight `Promise`
+- **Cross-thread locking** — `SharedArrayBuffer` + `Atomics.compareExchange` / `Atomics.waitAsync` prevent redundant `valueSelector` calls across worker threads in the same process that share the same `location`
 - **Async value resolution** — Transparently fetches missing values via a user-defined `valueSelector`
 - **Binary support** — Store and retrieve `Buffer` / `Uint8Array` alongside JSON values
 - **Size-bounded storage** — Configurable max database size with automatic eviction of oldest entries
@@ -67,14 +68,14 @@ if (result.async) {
 
 #### `CacheOptions`
 
-| Option       | Type                               | Default                               | Description                                                                    |
-| ------------ | ---------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------ |
-| `ttl`        | `number \| (value, key) => number` | `MAX_SAFE_INTEGER`                    | Time-to-live in milliseconds. After this, the entry is stale.                  |
-| `stale`      | `number \| (value, key) => number` | `MAX_SAFE_INTEGER`                    | Stale-while-revalidate window in ms. After `ttl + stale`, the entry is purged. |
-| `memory`     | `MemoryOptions \| false \| null`   | `{ maxSize: 16MB, maxCount: 16384 }`  | In-memory cache config, or `false`/`null` to disable.                          |
-| `database`   | `DatabaseOptions \| false \| null` | `{ timeout: 20, maxSize: 128MB }`     | SQLite config, or `false`/`null` to disable persistence.                       |
-| `lock`       | `LockOptions \| false \| null`     | `{ minTimeout: 1, maxTimeout: 1000 }` | Cross-process/thread lock config, or `false`/`null` to disable.                |
-| `serializer` | `Serializer<V>`                    | JSON + ArrayBufferView passthrough    | Custom `{ serialize, deserialize }` for value encoding.                        |
+| Option       | Type                               | Default                              | Description                                                                                                                               |
+| ------------ | ---------------------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| `ttl`        | `number \| (value, key) => number` | `MAX_SAFE_INTEGER`                   | Time-to-live in milliseconds. After this, the entry is stale.                                                                             |
+| `stale`      | `number \| (value, key) => number` | `MAX_SAFE_INTEGER`                   | Stale-while-revalidate window in ms. After `ttl + stale`, the entry is purged.                                                            |
+| `memory`     | `MemoryOptions \| false \| null`   | `{ maxSize: 16MB, maxCount: 16384 }` | In-memory cache config, or `false`/`null` to disable.                                                                                     |
+| `database`   | `DatabaseOptions \| false \| null` | `{ timeout: 20, maxSize: 128MB }`    | SQLite config, or `false`/`null` to disable persistence.                                                                                  |
+| `lock`       | `false \| null`                    | enabled                              | Pass `false`/`null` to disable cross-thread SAB locking (see [Cross-Thread Locking](#cross-thread-locking)). Always off for `':memory:'`. |
+| `serializer` | `Serializer<V>`                    | JSON + ArrayBufferView passthrough   | Custom `{ serialize, deserialize }` for value encoding.                                                                                   |
 
 #### `MemoryOptions`
 
@@ -87,19 +88,11 @@ The in-memory tier uses a random-two-choice eviction strategy: when the cache is
 
 #### `DatabaseOptions`
 
-| Option    | Type     | Default                      | Description                                                       |
-| --------- | -------- | ---------------------------- | ----------------------------------------------------------------- |
-| `timeout` | `number` | `20`                         | SQLite busy timeout in milliseconds.                              |
-| `maxSize` | `number` | `128 * 1024 * 1024` (128 MB) | Maximum database file size. Oldest entries are evicted when full. |
-
-#### `LockOptions`
-
-Cross-process/thread locking prevents multiple processes or worker threads from computing the same value simultaneously. The lock timeout is adaptive — it uses an exponential moving average (EMA) of `valueSelector` durations to estimate how long to wait before taking over a lock.
-
-| Option       | Type     | Default | Description                                                                |
-| ------------ | -------- | ------- | -------------------------------------------------------------------------- |
-| `minTimeout` | `number` | `1`     | Minimum lock timeout in ms. Also the starting timeout before EMA warms up. |
-| `maxTimeout` | `number` | `1000`  | Maximum lock timeout in ms. Caps the EMA-derived timeout.                  |
+| Option    | Type     | Default                      | Description                                                                                                               |
+| --------- | -------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
+| `timeout` | `number` | `20`                         | SQLite busy timeout in milliseconds.                                                                                      |
+| `maxSize` | `number` | `128 * 1024 * 1024` (128 MB) | Maximum total database size across all shards. Divided evenly per shard. Oldest entries are evicted when a shard is full. |
+| `shards`  | `number` | `4`                          | Number of SQLite files (shards) the cache is spread across. See [File Sharding](#file-sharding). Use `1` for single-file. |
 
 #### `Serializer<V>`
 
@@ -149,7 +142,7 @@ Remove an entry from both memory and SQLite. Also cancels any in-flight deduplic
 
 #### `cache.purgeStale(): void`
 
-Remove all expired entries (past `ttl + stale`) from both the in-memory cache and SQLite. Also cleans up stale lock rows older than 1 hour and runs `PRAGMA wal_checkpoint(TRUNCATE)` + `PRAGMA optimize`.
+Remove all expired entries (past `ttl + stale`) from both the in-memory cache and SQLite, and run `PRAGMA wal_checkpoint(TRUNCATE)` + `PRAGMA optimize`.
 
 #### `cache.close(): void`
 
@@ -163,7 +156,6 @@ Returns runtime statistics:
 
 ```ts
 {
-  lock: { timeout, mean, stddev } | undefined,
   dedupe: { size },
   memory: { size, maxSize, count, maxCount } | undefined,
   database: { location, size } | undefined,
@@ -193,34 +185,130 @@ Once the stale window expires, the entry is purged entirely and the next `get()`
 
 ```
 |--- ttl ---|--- stale ---|
-     fresh      stale       expired
-       ↓          ↓           ↓
-  sync hit    sync hit     async miss
-              + bg refresh
+    fresh        stale      expired
+      ↓            ↓           ↓
+  sync hit     sync hit    async miss
+             + bg refresh
 ```
 
-## Cross-Process/Thread Locking
+## Cross-Thread Locking
+
+Worker threads in the same process that pass the same `location` to `new Cache(...)` share a `SharedArrayBuffer` (acquired via `@nxtedition/shared`'s `getOrCreate` registry). That buffer is treated as a hash table of per-key `Int32` mutex slots:
+
+- Acquire is `Atomics.compareExchange(slot, 0, 1)`. The winner runs `valueSelector`; losers `Atomics.waitAsync` for a notify (with a 1 s timeout that re-enters `#load` — a guard against missed notifies and holder crashes).
+- On completion the holder does `Atomics.sub(slot, 1)` + `Atomics.notify`, waking any waiters which then re-enter `get()` and read the freshly-cached value.
+- Exception paths (sync `valueSelector` throw, async rejection, serializer/ttl/stale throwing inside `#set`) all run the release under a `try/finally`, so a buggy user callback can never wedge a slot.
+
+When the `location` is `':memory:'`, the cache is inherently per-instance (the SQLite DB isn't shared), so the SAB is skipped entirely and all coordination falls back to the instance-local `#dedupe` Map.
+
+Pass `{ lock: false }` (or `lock: null`) to opt out of the SAB lock on any cache — the instance-local `#dedupe` Map still coalesces concurrent `get()` calls on the same key, but sibling threads hitting the same DB file will each call `valueSelector` independently.
+
+There is **no cross-process coordination** — two separate Node processes pointed at the same SQLite file may both run `valueSelector` for the same key. If you need cross-process dedup, do it at a layer above (e.g. a request-coalescing service).
+
+### When to disable the lock
 
-When multiple processes or worker threads share the same SQLite database, the lock mechanism prevents redundant `valueSelector` calls. Process A acquires a lock, computes the value, and writes it. Process B sees the lock, waits for the estimated completion time (EMA-based), then reads the value from SQLite.
+- `':memory:'` — already disabled for you; nothing to configure.
+- Single-threaded app that doesn't spawn workers — disable to skip the cheap SAB round-trip on every `get()` cache miss.
+- Workload where each worker uses a disjoint keyspace — the SAB adds overhead without any dedupe benefit.
 
-For worker threads sharing the same process, a `SharedArrayBuffer` + `Atomics.waitAsync` mechanism provides efficient cross-thread wakeup — the waiting thread is notified immediately when the lock holder writes the value, rather than polling.
+### Counter overflow
 
-If the lock holder crashes, the lock becomes stale after `3 * lockTimeout` and another process steals it.
+Each slot is an `Int32`. `refresh()` uses `Atomics.add(slot, 1)` rather than a CAS (so concurrent refreshes stack), with an overflow guard at `0xfffffff` — if the counter ever reaches that value the call throws `Error: lock counter overflow` and rolls the increment back, so a wedged slot can't silently wrap. In practice this is unreachable: `0xfffffff` = ~268 M in-flight refreshes, which would OOM long before hitting the guard.
+
+## File Sharding
+
+The cache partitions its SQLite storage across N physical files (default `shards: 4`). Each key is hash-routed (via xxhash32) to one shard, and reads/writes for that key only touch that shard's connection. This is intended to reduce the SQLite writer-serialization ceiling: SQLite allows only one writer at a time per database file, even in WAL mode, so a single file caps concurrent-writer throughput at roughly one thread's worth of work regardless of how many threads the process spawns.
+
+**On-disk layout:**
+
+| `shards`    | Files                                                                |
+| ----------- | -------------------------------------------------------------------- |
+| `1`         | `<location>` (single file, legacy layout — compatible with `stat()`) |
+| `N` (N ≥ 2) | `<location>.0`, `<location>.1`, …, `<location>.{N-1}`                |
+
+With `shards: 1` the cache is a single file at the given `location`, identical to the pre-sharding layout — useful for backup scripts or observability tools that expect one file.
+
+**Cross-instance consistency:** hash routing is deterministic (xxhash32 is pure), so two Cache instances pointing at the same `location` with the same `shards` count will route the same keys to the same shards. Data persists across instance restarts.
+
+**Changing `shards` invalidates data.** If you open a cache with `shards: 2` after previously using `shards: 4`, keys hash-route to different shard files than they were written to. The old data is still on disk but effectively unreachable until you open with the original shard count.
+
+**Per-shard state:** `maxSize` divides evenly across shards (`maxSize / shards` per shard), eviction runs per-shard on `SQLITE_FULL`, and `stats.database.size` is summed across all shards.
 
 ## Batched Writes
 
-SQLite writes are batched using `setImmediate` — multiple `set()` calls within the same microtask turn are coalesced into a single `BEGIN`/`COMMIT` transaction. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a batch exceeds 512 items, it is flushed immediately.
+SQLite writes are batched using `setImmediate` — multiple `set()` calls within the same microtask turn are coalesced into a single `BEGIN`/`COMMIT` transaction per shard. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a per-shard batch exceeds 512 items, it is flushed immediately.
+
+Flushes iterate shards starting at a random index on each call, so no single shard is starved when the 10 ms per-flush time budget is exhausted mid-pass — the next flush picks a different starting shard.
 
-If the database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries and retries up to 3 times. On other errors, the rolled-back items are dropped from the batch and the error is emitted.
+If a shard's database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries in that shard and retries up to 3 times. On other errors, the entire pending batch for that shard is dropped (items remain in the in-memory cache until natural eviction/TTL) and the error is emitted — this prevents error floods when the underlying failure is persistent (e.g. a read-only DB).
 
 ## Error Handling
 
-`Cache` extends `EventEmitter`. Non-fatal errors (SQLite failures, lock contention errors, stale revalidation failures) are emitted as `'error'` events when a listener is attached. If no `'error'` listener is registered, errors are surfaced via `process.emitWarning()` instead, avoiding unhandled crashes.
+`Cache` extends `EventEmitter`. Non-fatal errors (SQLite failures, stale revalidation failures, background-refresh rejections from the SWR fire-and-forget path) are emitted as `'error'` events when a listener is attached. If no `'error'` listener is registered, errors are surfaced via `process.emitWarning()` instead, avoiding unhandled crashes.
 
 ## Off-Peak Purge
 
 All cache instances listen on the `nxt:offPeak` `BroadcastChannel`. When a message is received, `purgeStale()` is called on every active instance, enabling coordinated cleanup during low-traffic periods.
 
+## Benchmarks
+
+Measured on Apple M3 Pro (12 CPUs), Node 25.6.1. Throughput is `ops/sec`; latency is `ns` (median).
+
+### Single-thread hot paths
+
+| Operation                                   | ops/sec | p50    | p99    |
+| ------------------------------------------- | ------- | ------ | ------ |
+| `get()` memory hit (sequential keys)        | 6.87 M  | 125 ns | 333 ns |
+| `get()` memory hit (random keys)            | 7.09 M  | 125 ns | 250 ns |
+| `peek()` memory hit                         | 5.74 M  | 125 ns | 625 ns |
+| `get()` memory miss, DB hit (sequential)    | 386 K   | 2.3 µs | 6.9 µs |
+| `get()` memory miss, DB hit (random)        | 345 K   | 2.3 µs | 9.4 µs |
+| `get()` cold (sync `valueSelector`)         | 284 K   | 2.0 µs | 3.4 µs |
+| `get()` memory-only hit (no DB)             | 8.37 M  | 125 ns | 209 ns |
+| `get()` memory-only cold (no DB)            | 1.73 M  | 500 ns | 1.1 µs |
+| `get()` eviction pressure (`maxCount=1000`) | 1.43 M  | 583 ns | 1.8 µs |
+| `delete()` existing keys                    | 73 K    | 12 µs  | 32 µs  |
+| `purgeStale()` 10 K expired entries         | 7.4 ms  | —      | —      |
+
+### Shard-count comparison
+
+Single-thread overhead of sharding on the hot paths is ≤1 % in either direction. The pay-off is under multi-thread write contention:
+
+**4 threads, partitioned cold writes** (each worker writes unique keys, stressing concurrent writers):
+
+| `shards` | Aggregate throughput | Scaling vs. 1 thread | Δ vs. `shards: 1` |
+| -------- | -------------------- | -------------------- | ----------------- |
+| 1        | 383 K ops/s          | 1.47×                | baseline          |
+| 2        | 491 K ops/s          | 1.85×                | **+28 %**         |
+| 4        | 551 K ops/s          | 2.15×                | **+44 %**         |
+
+**4 threads, shared-keys hot-hit** (mostly memory-resident; sharding shouldn't help much here):
+
+| `shards` | Aggregate throughput |
+| -------- | -------------------- |
+| 1        | 25.3 M ops/s         |
+| 2        | 24.5 M ops/s         |
+| 4        | 24.6 M ops/s         |
+
+So `shards: 4` (the default) is a large win for write-heavy multi-threaded workloads and roughly break-even otherwise. If you run single-threaded or strictly read-heavy, `shards: 1` removes all sharding overhead and restores the single-file on-disk layout.
+
+### Lock option overhead (single-thread)
+
+| Path                              | lock enabled | lock disabled |
+| --------------------------------- | ------------ | ------------- |
+| `get()` cold (sync valueSelector) | 361 K ops/s  | 388 K ops/s   |
+| `get()` memory hit                | 9.23 M ops/s | 8.93 M ops/s  |
+
+Cross-thread SAB locking adds ~7 % on the cold path and is negligible (or slightly faster) on memory hits. It is worth leaving on unless the workload is strictly single-threaded or partitioned across workers.
+
+### Reproducing
+
+```sh
+yarn build
+node scripts/bench.mjs              # full bench suite
+node scripts/bench-shards.mjs       # shard-count comparison (1, 2, 4 shards)
+```
+
 ## Scripts
 
 ```sh

package/lib/index.d.ts

@@ -3,12 +3,9 @@ import { type MemoryOptions } from './memory.ts';
 export type { MemoryOptions } from './memory.ts';
 export interface DatabaseOptions {
     timeout?: number;
+    shards?: number;
     maxSize?: number;
 }
-export interface LockOptions {
-    minTimeout?: number;
-    maxTimeout?: number;
-}
 export interface Serializer<V> {
     serialize: (value: V) => Buffer | Uint8Array | string;
     deserialize: (data: Buffer | string) => V;
@@ -16,8 +13,8 @@ export interface Serializer<V> {
 export interface CacheOptions<V> {
     ttl?: number | ((value: V, key: string) => number);
     stale?: number | ((value: V, key: string) => number);
-    lock?: LockOptions | false | null;
     memory?: MemoryOptions | false | null;
+    lock?: false | null;
     database?: DatabaseOptions | false | null;
     serializer?: Serializer<V>;
 }
@@ -37,11 +34,6 @@ export declare class Cache<V = unknown, A extends unknown[] = [string]> extends
     #private;
     constructor(location: string, valueSelector?: (...args: A) => V | PromiseLike<V>, keySelector?: (...args: A) => string, opts?: CacheOptions<V>);
     get stats(): {
-        lock: {
-            timeout: number;
-            mean: number;
-            stddev: number;
-        } | undefined;
         dedupe: {
             size: number;
         };
@@ -68,3 +60,4 @@ export declare class Cache<V = unknown, A extends unknown[] = [string]> extends
 export declare const AsyncCache: typeof Cache;
 /** @deprecated Use `CacheOptions` instead. */
 export type AsyncCacheOptions<V> = CacheOptions<V>;
+//# sourceMappingURL=index.d.ts.map

package/lib/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAA;AAO1C,OAAO,EAAsC,KAAK,aAAa,EAAE,MAAM,aAAa,CAAA;AAEpF,YAAY,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AA2BhD,MAAM,WAAW,eAAe;IAC9B,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,MAAM,CAAC,EAAE,MAAM,CAAA;IACf,OAAO,CAAC,EAAE,MAAM,CAAA;CACjB;AAED,MAAM,WAAW,UAAU,CAAC,CAAC;IAC3B,SAAS,EAAE,CAAC,KAAK,EAAE,CAAC,KAAK,MAAM,GAAG,UAAU,GAAG,MAAM,CAAA;IACrD,WAAW,EAAE,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,KAAK,CAAC,CAAA;CAC1C;AAaD,MAAM,WAAW,YAAY,CAAC,CAAC;IAC7B,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IAClD,KAAK,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IACpD,MAAM,CAAC,EAAE,aAAa,GAAG,KAAK,GAAG,IAAI,CAAA;IACrC,IAAI,CAAC,EAAE,KAAK,GAAG,IAAI,CAAA;IACnB,QAAQ,CAAC,EAAE,eAAe,GAAG,KAAK,GAAG,IAAI,CAAA;IACzC,UAAU,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,CAAA;CAC3B;AAED,MAAM,MAAM,WAAW,CAAC,CAAC,IACrB;IAAE,KAAK,EAAE,CAAC,GAAG,SAAS,CAAC;IAAC,KAAK,EAAE,KAAK,CAAA;CAAE,GACtC;IAAE,KAAK,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAAC,KAAK,EAAE,IAAI,CAAA;CAAE,CAAA;AAEtC,OAAO,CAAC,MAAM,CAAC;IAEb,IAAI,WAAW,EAAE,OAAO,CAAC;QAAE,KAAK,EAAE,KAAK,CAAC,OAAO,CAAC,CAAA;KAAE,CAAC,EAAE,CAAA;CACtD;AAyFD,qBAAa,KAAK,CAAC,CAAC,GAAG,OAAO,EAAE,CAAC,SAAS,OAAO,EAAE,GAAG,CAAC,MAAM,CAAC,CAAE,SAAQ,YAAY;;gBA2BhF,QAAQ,EAAE,MAAM,EAChB,aAAa,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC,EAClD,WAAW,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,MAAM,EACpC,IAAI,CAAC,EAAE,YAAY,CAAC,CAAC,CAAC;IA+FxB,IAAI,KAAK;;;;;;;;;;;sBACmB,MAAM;kBAAQ,MAAM,GAAG,SAAS;;MAqB3D;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;IAIhB,KAAK,IAAI,IAAI;IAsBb,GAAG,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAO/B,IAAI,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAOhC,OAAO,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IA0BnC,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,IAAI;IAYxB,UAAU,IAAI,IAAI;CAgXnB;AAoBD,uCAAuC;AACvC,eAAO,MAAM,UAAU,cAAQ,CAAA;AAC/B,8CAA8C;AAC9C,MAAM,MAAM,iBAAiB,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,CAAC,CAAA"}