@nxtedition/cache 2.1.14 → 2.1.16

package/README.md

@@ -5,6 +5,7 @@ A two-tier async cache with SQLite persistence, in-memory pseudo-LRU, stale-whil
 ## Features
 
 - **Two-tier storage** — In-memory cache backed by SQLite on disk
+- **File-sharded SQLite** — Keys are hash-routed across N independent SQLite files, bypassing SQLite's per-file writer serialization. ~50% higher throughput under multi-thread write contention (see [Benchmarks](#benchmarks)).
 - **Stale-while-revalidate** — Serve stale data synchronously while refreshing in the background
 - **Request coalescing** — Concurrent fetches for the same key share a single in-flight `Promise`
 - **Cross-thread locking** — `SharedArrayBuffer` + `Atomics.compareExchange` / `Atomics.waitAsync` prevent redundant `valueSelector` calls across worker threads in the same process that share the same `location`
@@ -87,10 +88,11 @@ The in-memory tier uses a random-two-choice eviction strategy: when the cache is
 
 #### `DatabaseOptions`
 
-| Option    | Type     | Default                      | Description                                                       |
-| --------- | -------- | ---------------------------- | ----------------------------------------------------------------- |
-| `timeout` | `number` | `20`                         | SQLite busy timeout in milliseconds.                              |
-| `maxSize` | `number` | `128 * 1024 * 1024` (128 MB) | Maximum database file size. Oldest entries are evicted when full. |
+| Option    | Type     | Default                      | Description                                                                                                               |
+| --------- | -------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
+| `timeout` | `number` | `20`                         | SQLite busy timeout in milliseconds.                                                                                      |
+| `maxSize` | `number` | `128 * 1024 * 1024` (128 MB) | Maximum total database size across all shards. Divided evenly per shard. Oldest entries are evicted when a shard is full. |
+| `shards`  | `number` | `4`                          | Number of SQLite files (shards) the cache is spread across. See [File Sharding](#file-sharding). Use `1` for single-file. |
 
 #### `Serializer<V>`
 
@@ -138,16 +140,22 @@ Forces a new fetch via `valueSelector` regardless of cache state. Unlike `get()`
 
 Remove an entry from both memory and SQLite. Also cancels any in-flight deduplication for that key — a pending fetch will still resolve for its callers, but the result is **not** written to the cache.
 
-#### `cache.purgeStale(): void`
+#### `cache.gc(): void`
 
 Remove all expired entries (past `ttl + stale`) from both the in-memory cache and SQLite, and run `PRAGMA wal_checkpoint(TRUNCATE)` + `PRAGMA optimize`.
 
+#### `cache.flushSync(): void`
+
+Synchronously drain all pending batched writes to SQLite. The cache remains open and usable afterwards. This is useful when you need to guarantee persistence at a specific point without closing the cache (e.g. before handing off to another cache instance that shares the same database).
+
 #### `cache.close(): void`
 
-Flush any pending writes, close the SQLite database, and release resources. Clears all in-flight deduplication. Operations after `close()` throw.
+Calls `flushSync()` to drain pending writes, then closes the SQLite database and releases resources. Clears all in-flight deduplication. Operations after `close()` throw.
 
 Also available as `[Symbol.dispose]()` for use with `using` declarations.
 
+Open caches are automatically closed on the `beforeExit` event, ensuring pending writes are flushed before the process exits.
+
 #### `cache.stats`
 
 Returns runtime statistics:
@@ -183,19 +191,19 @@ Once the stale window expires, the entry is purged entirely and the next `get()`
 
 ```
 |--- ttl ---|--- stale ---|
-     fresh      stale       expired
-       ↓          ↓           ↓
-  sync hit    sync hit     async miss
-              + bg refresh
+    fresh        stale      expired
+      ↓            ↓           ↓
+  sync hit     sync hit    async miss
+             + bg refresh
 ```
 
 ## Cross-Thread Locking
 
-Worker threads in the same process that pass the same `location` to `new Cache(...)` share a `SharedArrayBuffer` (acquired via `@nxtedition/shared`'s `getOrCreate` registry). That buffer is treated as a hash table of per-key `Int32` mutex slots:
+Worker threads in the same process that pass the same `location` to `new Cache(...)` share a 256 KiB `SharedArrayBuffer` (64K `Int32` slots, acquired via `@nxtedition/shared`'s `getOrCreate` registry). That buffer is treated as a hash table of per-key binary mutex slots:
 
-- Acquire is `Atomics.compareExchange(slot, 0, 1)`. The winner runs `valueSelector`; losers `Atomics.waitAsync` for a notify (with a 1 s timeout that re-enters `#load` — a guard against missed notifies and holder crashes).
-- On completion the holder does `Atomics.sub(slot, 1)` + `Atomics.notify`, waking any waiters which then re-enter `get()` and read the freshly-cached value.
-- Exception paths (sync `valueSelector` throw, async rejection, serializer/ttl/stale throwing inside `#set`) all run the release under a `try/finally`, so a buggy user callback can never wedge a slot.
+- **Acquire:** `Atomics.compareExchange(slot, 0, 1)`. The winner runs `valueSelector`; losers call `Atomics.waitAsync` (with a 1 s timeout as a guard against missed notifies and holder crashes). On timeout, the waiter falls back to a lockless refresh.
+- **Release:** The holder does `Atomics.sub(slot, 1)` + `Atomics.notify` inside the batched flush, waking waiters which then read the freshly-cached value from the database.
+- **Exception safety:** All exception paths (sync `valueSelector` throw, async rejection, serializer/ttl/stale throwing inside `#set`) release the slot under a `try/finally`, so a buggy user callback can never wedge a slot.
 
 When the `location` is `':memory:'`, the cache is inherently per-instance (the SQLite DB isn't shared), so the SAB is skipped entirely and all coordination falls back to the instance-local `#dedupe` Map.
 
@@ -209,15 +217,32 @@ There is **no cross-process coordination** — two separate Node processes point
 - Single-threaded app that doesn't spawn workers — disable to skip the cheap SAB round-trip on every `get()` cache miss.
 - Workload where each worker uses a disjoint keyspace — the SAB adds overhead without any dedupe benefit.
 
-### Counter overflow
+## File Sharding
+
+The cache partitions its SQLite storage across N physical files (default `shards: 4`). Each key is hash-routed (via xxhash32) to one shard, and reads/writes for that key only touch that shard's connection. This is intended to reduce the SQLite writer-serialization ceiling: SQLite allows only one writer at a time per database file, even in WAL mode, so a single file caps concurrent-writer throughput at roughly one thread's worth of work regardless of how many threads the process spawns.
+
+**On-disk layout:**
+
+| `shards`    | Files                                                                |
+| ----------- | -------------------------------------------------------------------- |
+| `1`         | `<location>` (single file, legacy layout — compatible with `stat()`) |
+| `N` (N ≥ 2) | `<location>.0`, `<location>.1`, …, `<location>.{N-1}`                |
+
+With `shards: 1` the cache is a single file at the given `location`, identical to the pre-sharding layout — useful for backup scripts or observability tools that expect one file.
 
-Each slot is an `Int32`. `refresh()` uses `Atomics.add(slot, 1)` rather than a CAS (so concurrent refreshes stack), with an overflow guard at `0xfffffff` — if the counter ever reaches that value the call throws `Error: lock counter overflow` and rolls the increment back, so a wedged slot can't silently wrap. In practice this is unreachable: `0xfffffff` = ~268 M in-flight refreshes, which would OOM long before hitting the guard.
+**Cross-instance consistency:** hash routing is deterministic (xxhash32 is pure), so two Cache instances pointing at the same `location` with the same `shards` count will route the same keys to the same shards. Data persists across instance restarts.
+
+**Changing `shards` invalidates data.** If you open a cache with `shards: 2` after previously using `shards: 4`, keys hash-route to different shard files than they were written to. The old data is still on disk but effectively unreachable until you open with the original shard count.
+
+**Per-shard state:** `maxSize` divides evenly across shards (`maxSize / shards` per shard), eviction runs per-shard on `SQLITE_FULL`, and `stats.database.size` is summed across all shards.
 
 ## Batched Writes
 
-SQLite writes are batched using `setImmediate` — multiple `set()` calls within the same microtask turn are coalesced into a single `BEGIN`/`COMMIT` transaction. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a batch exceeds 512 items, it is flushed immediately.
+SQLite writes are batched using `setImmediate` — multiple `set()` calls within the same microtask turn are coalesced into a single `BEGIN`/`COMMIT` transaction per shard. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a per-shard batch exceeds 512 items, it is flushed immediately.
 
-If the database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries and retries up to 3 times. On other errors, the rolled-back items are dropped from the batch and the error is emitted.
+Flushes iterate shards starting at a random index on each call, so no single shard is starved when the 10 ms per-flush time budget is exhausted mid-pass — the next flush picks a different starting shard.
+
+If a shard's database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries in that shard and retries up to 3 times. On other errors, the entire pending batch for that shard is dropped (items remain in the in-memory cache until natural eviction/TTL) and the error is emitted — this prevents error floods when the underlying failure is persistent (e.g. a read-only DB).
 
 ## Error Handling
 
@@ -225,7 +250,68 @@ If the database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries
 
 ## Off-Peak Purge
 
-All cache instances listen on the `nxt:offPeak` `BroadcastChannel`. When a message is received, `purgeStale()` is called on every active instance, enabling coordinated cleanup during low-traffic periods.
+All cache instances listen on the `nxt:offPeak` `BroadcastChannel`. When a message is received, `gc()` is called on every active instance, enabling coordinated cleanup during low-traffic periods.
+
+## Benchmarks
+
+Measured on Apple M3 Pro (12 CPUs), Node 25.6.1. Throughput is `ops/sec`; latency is `ns` (median).
+
+### Single-thread hot paths
+
+| Operation                                   | ops/sec | p50    | p99     |
+| ------------------------------------------- | ------- | ------ | ------- |
+| `get()` memory hit (sequential keys)        | 4.90 M  | 125 ns | 666 ns  |
+| `get()` memory hit (random keys)            | 6.05 M  | 125 ns | 584 ns  |
+| `peek()` memory hit                         | 8.05 M  | 125 ns | 250 ns  |
+| `get()` memory miss, DB hit (sequential)    | 303 K   | 2.3 µs | 10.6 µs |
+| `get()` memory miss, DB hit (random)        | 399 K   | 2.3 µs | 6.4 µs  |
+| `get()` cold (sync `valueSelector`)         | 274 K   | 1.8 µs | 4.5 µs  |
+| `get()` memory-only hit (no DB)             | 5.39 M  | 125 ns | 500 ns  |
+| `get()` memory-only cold (no DB)            | 1.16 M  | 708 ns | 2.3 µs  |
+| `get()` eviction pressure (`maxCount=1000`) | 1.64 M  | 542 ns | 1.5 µs  |
+| `delete()` existing keys                    | 67 K    | 13 µs  | 44 µs   |
+| `gc()` 10 K expired entries                 | 7.2 ms  | —      | —       |
+
+### Shard-count comparison
+
+Single-thread overhead of sharding on the hot paths is ≤5 % in either direction. The pay-off is under multi-thread write contention:
+
+**12 threads, partitioned cold writes** (each worker writes unique keys, stressing concurrent writers):
+
+| `shards` | Aggregate throughput | Scaling vs. 1 thread | Δ vs. `shards: 1` |
+| -------- | -------------------- | -------------------- | ----------------- |
+| 1        | 154 K ops/s          | 0.80×                | baseline          |
+| 2        | 223 K ops/s          | 1.04×                | **+45 %**         |
+| 4        | 237 K ops/s          | 1.37×                | **+54 %**         |
+| 8        | 182 K ops/s          | 1.08×                | **+19 %**         |
+
+**12 threads, shared-keys hot-hit** (mostly memory-resident; sharding shouldn't help much here):
+
+| `shards` | Aggregate throughput |
+| -------- | -------------------- |
+| 1        | 7.69 M ops/s         |
+| 2        | 7.86 M ops/s         |
+| 4        | 7.70 M ops/s         |
+| 8        | 7.51 M ops/s         |
+
+So `shards: 4` (the default) is a large win for write-heavy multi-threaded workloads and roughly break-even otherwise. If you run single-threaded or strictly read-heavy, `shards: 1` removes all sharding overhead and restores the single-file on-disk layout.
+
+### Lock option overhead (single-thread)
+
+| Path                              | lock enabled | lock disabled |
+| --------------------------------- | ------------ | ------------- |
+| `get()` cold (sync valueSelector) | 298 K ops/s  | 306 K ops/s   |
+| `get()` memory hit                | 7.44 M ops/s | 6.19 M ops/s  |
+
+Cross-thread SAB locking adds ~3 % on the cold path and is negligible on memory hits. It is worth leaving on unless the workload is strictly single-threaded or partitioned across workers.
+
+### Reproducing
+
+```sh
+yarn build
+node scripts/bench.mjs              # full bench suite
+node scripts/bench-shards.mjs       # shard-count comparison (1, 2, 4, 8 shards)
+```
 
 ## Scripts
 

package/lib/index.d.ts

@@ -3,6 +3,7 @@ import { type MemoryOptions } from './memory.ts';
 export type { MemoryOptions } from './memory.ts';
 export interface DatabaseOptions {
     timeout?: number;
+    shards?: number;
     maxSize?: number;
 }
 export interface Serializer<V> {
@@ -48,12 +49,13 @@ export declare class Cache<V = unknown, A extends unknown[] = [string]> extends
         } | undefined;
     };
     [Symbol.dispose](): void;
+    flushSync(): void;
     close(): void;
     get(...args: A): CacheResult<V>;
     peek(...args: A): CacheResult<V>;
     refresh(...args: A): CacheResult<V>;
     delete(...args: A): void;
-    purgeStale(): void;
+    gc(): void;
 }
 /** @deprecated Use `Cache` instead. */
 export declare const AsyncCache: typeof Cache;

package/lib/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAA;AAO1C,OAAO,EAAsC,KAAK,aAAa,EAAE,MAAM,aAAa,CAAA;AAEpF,YAAY,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AA2BhD,MAAM,WAAW,eAAe;IAC9B,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,OAAO,CAAC,EAAE,MAAM,CAAA;CACjB;AACD,MAAM,WAAW,UAAU,CAAC,CAAC;IAC3B,SAAS,EAAE,CAAC,KAAK,EAAE,CAAC,KAAK,MAAM,GAAG,UAAU,GAAG,MAAM,CAAA;IACrD,WAAW,EAAE,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,KAAK,CAAC,CAAA;CAC1C;AAaD,MAAM,WAAW,YAAY,CAAC,CAAC;IAC7B,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IAClD,KAAK,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IACpD,MAAM,CAAC,EAAE,aAAa,GAAG,KAAK,GAAG,IAAI,CAAA;IACrC,IAAI,CAAC,EAAE,KAAK,GAAG,IAAI,CAAA;IACnB,QAAQ,CAAC,EAAE,eAAe,GAAG,KAAK,GAAG,IAAI,CAAA;IACzC,UAAU,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,CAAA;CAC3B;AAED,MAAM,MAAM,WAAW,CAAC,CAAC,IACrB;IAAE,KAAK,EAAE,CAAC,GAAG,SAAS,CAAC;IAAC,KAAK,EAAE,KAAK,CAAA;CAAE,GACtC;IAAE,KAAK,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAAC,KAAK,EAAE,IAAI,CAAA;CAAE,CAAA;AAEtC,OAAO,CAAC,MAAM,CAAC;IAEb,IAAI,WAAW,EAAE,OAAO,CAAC;QAAE,KAAK,EAAE,KAAK,CAAC,OAAO,CAAC,CAAA;KAAE,CAAC,EAAE,CAAA;CACtD;AAMD,qBAAa,KAAK,CAAC,CAAC,GAAG,OAAO,EAAE,CAAC,SAAS,OAAO,EAAE,GAAG,CAAC,MAAM,CAAC,CAAE,SAAQ,YAAY;;gBAoChF,QAAQ,EAAE,MAAM,EAChB,aAAa,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC,EAClD,WAAW,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,MAAM,EACpC,IAAI,CAAC,EAAE,YAAY,CAAC,CAAC,CAAC;IAoIxB,IAAI,KAAK;;;;;;;;;;;sBACmB,MAAM;kBAAQ,MAAM,GAAG,SAAS;;MAkB3D;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;IAIhB,KAAK,IAAI,IAAI;IA2Bb,GAAG,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAO/B,IAAI,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAOhC,OAAO,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IA4BnC,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,IAAI;IAOxB,UAAU,IAAI,IAAI;CA+VnB;AAoBD,uCAAuC;AACvC,eAAO,MAAM,UAAU,cAAQ,CAAA;AAC/B,8CAA8C;AAC9C,MAAM,MAAM,iBAAiB,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,CAAC,CAAA"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAA;AAO1C,OAAO,EAAsC,KAAK,aAAa,EAAE,MAAM,aAAa,CAAA;AAEpF,YAAY,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AAiDhD,MAAM,WAAW,eAAe;IAC9B,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,MAAM,CAAC,EAAE,MAAM,CAAA;IACf,OAAO,CAAC,EAAE,MAAM,CAAA;CACjB;AAED,MAAM,WAAW,UAAU,CAAC,CAAC;IAC3B,SAAS,EAAE,CAAC,KAAK,EAAE,CAAC,KAAK,MAAM,GAAG,UAAU,GAAG,MAAM,CAAA;IACrD,WAAW,EAAE,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,KAAK,CAAC,CAAA;CAC1C;AAaD,MAAM,WAAW,YAAY,CAAC,CAAC;IAC7B,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IAClD,KAAK,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,MAAM,KAAK,MAAM,CAAC,CAAA;IACpD,MAAM,CAAC,EAAE,aAAa,GAAG,KAAK,GAAG,IAAI,CAAA;IACrC,IAAI,CAAC,EAAE,KAAK,GAAG,IAAI,CAAA;IACnB,QAAQ,CAAC,EAAE,eAAe,GAAG,KAAK,GAAG,IAAI,CAAA;IACzC,UAAU,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,CAAA;CAC3B;AAED,MAAM,MAAM,WAAW,CAAC,CAAC,IACrB;IAAE,KAAK,EAAE,CAAC,GAAG,SAAS,CAAC;IAAC,KAAK,EAAE,KAAK,CAAA;CAAE,GACtC;IAAE,KAAK,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IAAC,KAAK,EAAE,IAAI,CAAA;CAAE,CAAA;AAEtC,OAAO,CAAC,MAAM,CAAC;IAEb,IAAI,WAAW,EAAE,OAAO,CAAC;QAAE,KAAK,EAAE,KAAK,CAAC,OAAO,CAAC,CAAA;KAAE,CAAC,EAAE,CAAA;CACtD;AA6FD,qBAAa,KAAK,CAAC,CAAC,GAAG,OAAO,EAAE,CAAC,SAAS,OAAO,EAAE,GAAG,CAAC,MAAM,CAAC,CAAE,SAAQ,YAAY;;gBAuChF,QAAQ,EAAE,MAAM,EAChB,aAAa,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC,EAClD,WAAW,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,KAAK,MAAM,EACpC,IAAI,CAAC,EAAE,YAAY,CAAC,CAAC,CAAC;IAuHxB,IAAI,KAAK;;;;;;;;;;;sBACmB,MAAM;kBAAQ,MAAM,GAAG,SAAS;;MAqB3D;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;IAIhB,SAAS;IAiBT,KAAK;IA4BL,GAAG,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAO/B,IAAI,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAOhC,OAAO,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC;IAcnC,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,GAAG,IAAI;IAYxB,EAAE,IAAI,IAAI;CA6bX;AA2BD,uCAAuC;AACvC,eAAO,MAAM,UAAU,cAAQ,CAAA;AAC/B,8CAA8C;AAC9C,MAAM,MAAM,iBAAiB,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,CAAC,CAAA"}