@nxtedition/cache 2.1.9 → 2.1.11

package/README.md

@@ -1,17 +1,19 @@
 # @nxtedition/cache
 
-A two-tier async cache with SQLite persistence, in-memory LRU, stale-while-revalidate, cross-process deduplication, and automatic request deduplication.
+A two-tier async cache with SQLite persistence, in-memory pseudo-LRU, stale-while-revalidate, cross-process/thread deduplication, and automatic request coalescing.
 
 ## Features
 
-- **Two-tier storage** — In-memory LRU cache backed by SQLite on disk
+- **Two-tier storage** — In-memory cache backed by SQLite on disk
 - **Stale-while-revalidate** — Serve stale data synchronously while refreshing in the background
-- **Request deduplication** — Concurrent fetches for the same key share a single in-flight request
-- **Cross-process locking** — SQLite-based distributed locks prevent redundant work across processes/threads
+- **Request coalescing** — Concurrent fetches for the same key share a single in-flight request
+- **Cross-process/thread locking** — SQLite-based distributed locks with `Atomics`-based wakeup prevent redundant work across processes and worker threads
 - **Async value resolution** — Transparently fetches missing values via a user-defined `valueSelector`
 - **Binary support** — Store and retrieve `Buffer` / `Uint8Array` alongside JSON values
 - **Size-bounded storage** — Configurable max database size with automatic eviction of oldest entries
 - **Custom serialization** — Pluggable `serialize`/`deserialize` for non-JSON value types
+- **Batched writes** — SQLite writes are coalesced into transactions via `setImmediate`, reducing I/O
+- **Disposable** — Implements `Symbol.dispose` for use with `using` declarations
 
 ## Usage
 
@@ -42,6 +44,16 @@ if (result.async) {
 }
 ```
 
+### Using `Symbol.dispose`
+
+```ts
+{
+  using cache = new Cache('./tmp.db', fetchItem, (id) => id, { ttl: 5_000 })
+  const result = cache.get('key')
+  // ...
+} // cache.close() called automatically
+```
+
 ## API
 
 ### `new Cache(location, valueSelector?, keySelector?, opts?)`
@@ -61,7 +73,7 @@ if (result.async) {
 | `stale`      | `number \| (value, key) => number` | `MAX_SAFE_INTEGER`                    | Stale-while-revalidate window in ms. After `ttl + stale`, the entry is purged. |
 | `memory`     | `MemoryOptions \| false \| null`   | `{ maxSize: 16MB, maxCount: 16384 }`  | In-memory cache config, or `false`/`null` to disable.                          |
 | `database`   | `DatabaseOptions \| false \| null` | `{ timeout: 20, maxSize: 128MB }`     | SQLite config, or `false`/`null` to disable persistence.                       |
-| `lock`       | `LockOptions \| false \| null`     | `{ minTimeout: 1, maxTimeout: 1000 }` | Cross-process lock config, or `false`/`null` to disable.                       |
+| `lock`       | `LockOptions \| false \| null`     | `{ minTimeout: 1, maxTimeout: 1000 }` | Cross-process/thread lock config, or `false`/`null` to disable.                |
 | `serializer` | `Serializer<V>`                    | JSON + ArrayBufferView passthrough    | Custom `{ serialize, deserialize }` for value encoding.                        |
 
 #### `MemoryOptions`
@@ -71,6 +83,8 @@ if (result.async) {
 | `maxSize`  | `number` | `16 * 1024 * 1024` (16 MB) | Maximum total size in bytes of cached entries. |
 | `maxCount` | `number` | `16 * 1024` (16384)        | Maximum number of entries in memory.           |
 
+The in-memory tier uses a random-two-choice eviction strategy: when the cache is full, two entries are sampled at random and the least recently accessed one is evicted. This provides near-LRU behavior with O(1) eviction cost.
+
 #### `DatabaseOptions`
 
 | Option    | Type     | Default                      | Description                                                       |
@@ -80,7 +94,7 @@ if (result.async) {
 
 #### `LockOptions`
 
-Cross-process locking prevents multiple processes from computing the same value simultaneously. The lock timeout is adaptive — it uses an exponential moving average (EMA) of `valueSelector` durations to estimate how long to wait before taking over a lock.
+Cross-process/thread locking prevents multiple processes or worker threads from computing the same value simultaneously. The lock timeout is adaptive — it uses an exponential moving average (EMA) of `valueSelector` durations to estimate how long to wait before taking over a lock.
 
 | Option       | Type     | Default | Description                                                                |
 | ------------ | -------- | ------- | -------------------------------------------------------------------------- |
@@ -119,7 +133,7 @@ if (result.async) {
 
 #### `cache.get(...args): CacheResult<V>`
 
-Returns a cached value or triggers a fetch on cache miss. If the entry is stale and the `valueSelector` is async, returns the stale value synchronously while a background refresh runs.
+Returns a cached value or triggers a fetch on cache miss. If the entry is stale and the `valueSelector` is async, returns the stale value synchronously (`async: false`) while a background refresh runs. If the `valueSelector` throws during a stale revalidation, the error is emitted and the stale value is preserved.
 
 #### `cache.peek(...args): CacheResult<V>`
 
@@ -139,7 +153,9 @@ Remove all expired entries (past `ttl + stale`) from both the in-memory cache an
 
 #### `cache.close(): void`
 
-Close the SQLite database and release resources. Clears all in-flight deduplication. Operations after `close()` throw.
+Flush any pending writes, close the SQLite database, and release resources. Clears all in-flight deduplication. Operations after `close()` throw.
+
+Also available as `[Symbol.dispose]()` for use with `using` declarations.
 
 #### `cache.stats`
 
@@ -183,11 +199,23 @@ Once the stale window expires, the entry is purged entirely and the next `get()`
               + bg refresh
 ```
 
-## Cross-Process Locking
+## Cross-Process/Thread Locking
+
+When multiple processes or worker threads share the same SQLite database, the lock mechanism prevents redundant `valueSelector` calls. Process A acquires a lock, computes the value, and writes it. Process B sees the lock, waits for the estimated completion time (EMA-based), then reads the value from SQLite.
+
+For worker threads sharing the same process, a `SharedArrayBuffer` + `Atomics.waitAsync` mechanism provides efficient cross-thread wakeup — the waiting thread is notified immediately when the lock holder writes the value, rather than polling.
+
+If the lock holder crashes, the lock becomes stale after `3 * lockTimeout` and another process steals it.
+
+## Batched Writes
+
+SQLite writes are batched using `setImmediate` — multiple `set()` calls within the same microtask turn are coalesced into a single `BEGIN`/`COMMIT` transaction. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a batch exceeds 512 items, it is flushed immediately.
+
+If the database is full (`SQLITE_FULL`), the cache evicts the 256 oldest entries and retries up to 3 times. On other errors, the rolled-back items are dropped from the batch and the error is emitted.
 
-When multiple processes or threads share the same SQLite database, the lock mechanism prevents redundant `valueSelector` calls. Process A acquires a lock, computes the value, and writes it. Process B sees the lock, waits for the estimated completion time (EMA-based), then reads the value from SQLite.
+## Error Handling
 
-If the lock holder crashes, the lock becomes stale after `3 × lockTimeout` and another process steals it.
+`Cache` extends `EventEmitter`. Non-fatal errors (SQLite failures, lock contention errors, stale revalidation failures) are emitted as `'error'` events when a listener is attached. If no `'error'` listener is registered, errors are surfaced via `process.emitWarning()` instead, avoiding unhandled crashes.
 
 ## Off-Peak Purge
 

package/lib/index.js

@@ -2,6 +2,11 @@ import { randomUUID } from 'node:crypto'
 import { EventEmitter } from 'node:events'
 import { DatabaseSync,                                        } from 'node:sqlite'
 import { setTimeout as delay } from 'node:timers/promises'
+
+import xxhash from 'xxhash-wasm'
+
+import { getOrCreate } from '@nxtedition/shared'
+
 import { MemoryCache,                                           } from "./memory.js"
 
                                                 
@@ -79,6 +84,7 @@ const defaultSerializer                  = {
 
 const VERSION = 5
 const MAX_DURATION = 365000000e3
+const HASHER = await xxhash()
 
 export class Cache                                              extends EventEmitter {
   #memory                       
@@ -92,12 +98,16 @@ export class Cache                                              extends EventEmi
   #stale                                 
   #serializer               
 
-  #lockId = randomUUID()
+  #lockOrigin         = 0
+  #lockId                = null
+  #lockArray                    = null
+  #lockSet                    
   #lockVar = 0
   #lockMean         = 5
   #lockTimeout         = 10
   #lockMinTimeout         = 1
   #lockMaxTimeout         = 1_000
+
   #flushHandle                          = null
   #location        
   #databaseTimeout         = 20
@@ -113,9 +123,10 @@ export class Cache                                              extends EventEmi
   #pageCountQuery                       = null
   #pageSizeQuery                       = null
   #setQuery                       = null
-  #setBatch                                                                          = []
+  #setBatch                                                                                        =
+    []
 
-  #emitError(err       )       {
+  #emitError = (err       ) => {
     if (this.listenerCount('error') > 0) {
       this.emit('error', err)
     } else {
@@ -164,27 +175,42 @@ export class Cache                                              extends EventEmi
     }
 
     if (opts?.lock === false || opts?.lock === null) {
+      this.#lockId = null
+      this.#lockArray = null
+      this.#lockSet = null
       this.#lockMinTimeout = -1
       this.#lockTimeout = -1
       this.#lockMean = -1
-    } else if (opts?.lock !== undefined) {
-      if (typeof opts.lock !== 'object' || opts.lock === null) {
-        throw new TypeError('lock must be an object')
-      }
-      if (opts.lock.minTimeout !== undefined) {
-        if (typeof opts.lock.minTimeout !== 'number') {
-          throw new TypeError('lock.minTimeout must be a number')
+    } else {
+      if (opts?.lock !== undefined) {
+        if (typeof opts.lock !== 'object' || opts.lock === null) {
+          throw new TypeError('lock must be an object')
         }
-        this.#lockMinTimeout = Math.max(0, opts.lock.minTimeout)
-        this.#lockTimeout = this.#lockMinTimeout
-        this.#lockMean = this.#lockTimeout / 1.2
-      }
-      if (opts.lock.maxTimeout !== undefined) {
-        if (typeof opts.lock.maxTimeout !== 'number') {
-          throw new TypeError('lock.maxTimeout must be a number')
+        if (opts.lock.minTimeout !== undefined) {
+          if (typeof opts.lock.minTimeout !== 'number') {
+            throw new TypeError('lock.minTimeout must be a number')
+          }
+          this.#lockMinTimeout = Math.max(0, opts.lock.minTimeout)
+          this.#lockTimeout = this.#lockMinTimeout
+          this.#lockMean = this.#lockTimeout / 1.2
+        }
+        if (opts.lock.maxTimeout !== undefined) {
+          if (typeof opts.lock.maxTimeout !== 'number') {
+            throw new TypeError('lock.maxTimeout must be a number')
+          }
+          this.#lockMaxTimeout = Math.max(this.#lockMinTimeout, opts.lock.maxTimeout)
         }
-        this.#lockMaxTimeout = Math.max(this.#lockMinTimeout, opts.lock.maxTimeout)
       }
+
+      const sab = getOrCreate(`__@nxtedition/cache/${location}`, 8 + 1024)
+
+      const originArray = new BigInt64Array(sab, 0, 1)
+      Atomics.compareExchange(originArray, 0, 0n, BigInt(Date.now()))
+
+      this.#lockOrigin = Number(originArray[0])
+      this.#lockId = randomUUID()
+      this.#lockArray = new Int32Array(sab, 8)
+      this.#lockSet = new Set()
     }
 
     if (opts?.serializer !== undefined) {
@@ -227,12 +253,6 @@ export class Cache                                              extends EventEmi
           ) WITHOUT ROWID;
 
           CREATE INDEX IF NOT EXISTS cache_v${VERSION}_stale_idx ON cache_v${VERSION}(stale);
-
-          CREATE TABLE IF NOT EXISTS cache_lock_v${VERSION} (
-            key TEXT PRIMARY KEY NOT NULL,
-            lock_acquired INTEGER NOT NULL,
-            lock_owner TEXT NOT NULL
-          );
         `)
 
         this.#getQuery = this.#database.prepare(
@@ -249,21 +269,32 @@ export class Cache                                              extends EventEmi
           `DELETE FROM cache_v${VERSION} WHERE key IN (SELECT key FROM cache_v${VERSION} ORDER BY stale ASC LIMIT ?)`,
         )
 
-        // ON CONFLICT refreshes lock_acquired when we are the owner,
-        // preventing other processes from stealing our still-active lock.
-        this.#lockAcquireQuery = this.#database.prepare(
-          `INSERT INTO cache_lock_v${VERSION} (key, lock_acquired, lock_owner) VALUES (?, ?, ?) ON CONFLICT(key) DO UPDATE SET lock_acquired = CASE WHEN lock_owner = excluded.lock_owner THEN excluded.lock_acquired ELSE lock_acquired END RETURNING lock_acquired, lock_owner`,
-        )
-        this.#lockStealQuery = this.#database.prepare(
-          `UPDATE cache_lock_v${VERSION} SET lock_acquired = ?, lock_owner = ? WHERE key = ? AND lock_acquired = ?`,
-        )
-        // Read the current lock state after a failed steal to get the winner's timestamp.
-        this.#lockGetQuery = this.#database.prepare(
-          `SELECT lock_acquired FROM cache_lock_v${VERSION} WHERE key = ?`,
-        )
-        this.#lockPurgeQuery = this.#database.prepare(
-          `DELETE FROM cache_lock_v${VERSION} WHERE lock_acquired <= ?`,
-        )
+        if (this.#lockId != null) {
+          this.#database.exec(`
+            CREATE TABLE IF NOT EXISTS cache_lock_v${VERSION} (
+              key TEXT PRIMARY KEY NOT NULL,
+              lock_acquired INTEGER NOT NULL,
+              lock_owner TEXT NOT NULL
+            );
+          `)
+
+          // ON CONFLICT refreshes lock_acquired when we are the owner,
+          // preventing other processes from stealing our still-active lock.
+          this.#lockAcquireQuery = this.#database.prepare(
+            `INSERT INTO cache_lock_v${VERSION} (key, lock_acquired, lock_owner) VALUES (?, ?, ?) ON CONFLICT(key) DO UPDATE SET lock_acquired = CASE WHEN lock_owner = excluded.lock_owner THEN excluded.lock_acquired ELSE lock_acquired END RETURNING lock_acquired, lock_owner`,
+          )
+          this.#lockStealQuery = this.#database.prepare(
+            `UPDATE cache_lock_v${VERSION} SET lock_acquired = ?, lock_owner = ? WHERE key = ? AND lock_acquired = ?`,
+          )
+          // Read the current lock state after a failed steal to get the winner's timestamp.
+          this.#lockGetQuery = this.#database.prepare(
+            `SELECT lock_acquired FROM cache_lock_v${VERSION} WHERE key = ?`,
+          )
+          this.#lockPurgeQuery = this.#database.prepare(
+            `DELETE FROM cache_lock_v${VERSION} WHERE lock_acquired <= ?`,
+          )
+        }
+
         this.#pageCountQuery = this.#database.prepare('PRAGMA page_count')
         this.#pageSizeQuery = this.#database.prepare('PRAGMA page_size')
         break
@@ -497,15 +528,16 @@ export class Cache                                              extends EventEmi
       throw new TypeError('keySelector must return a non-empty string')
     }
 
-    if (this.#lockTimeout >= 0) {
+    if (this.#lockId != null) {
       // Another process holds the lock — wait for the value instead of calling valueSelector.
       // Use deferred pattern so #waitForValue can check promise identity to detect delete+get races.
-      const lockResult = this.#tryAcquireLock(key)
+      const lockIdx = this.#lockArray ? HASHER.h32(key) % this.#lockArray.length : -1
+      const lockResult = this.#tryAcquireLock(key, lockIdx)
       if (typeof lockResult === 'number') {
         const { promise, resolve, reject } = Promise.withResolvers   ()
         promise.catch(noop)
         this.#dedupe.set(key, promise)
-        this.#waitForValue(args, key, lockResult, promise).then(resolve, reject)
+        this.#waitForValue(args, key, lockIdx, lockResult, promise).then(resolve, reject)
         return { async: true, value: promise }
       }
     }
@@ -541,16 +573,26 @@ export class Cache                                              extends EventEmi
     const value = this.#valueSelector(...args)
 
     if (isThenable(value)) {
+      const promise = Promise.resolve(value)
+
+      if (this.#lockId != null) {
+        promise
+          .then(() => {
+            this.#updateLockTimeout(performance.now() - startTime)
+          })
+          .catch(this.#emitError)
+      }
+
       return {
         async: true,
-        value: Promise.resolve(value).then((value) => {
-          this.#updateLockTimeout(performance.now() - startTime)
-          return value
-        }),
+        value: promise,
       }
     }
 
-    this.#updateLockTimeout(performance.now() - startTime)
+    if (this.#lockId != null) {
+      this.#updateLockTimeout(performance.now() - startTime)
+    }
+
     return { async: false, value }
   }
 
@@ -609,7 +651,7 @@ export class Cache                                              extends EventEmi
       this.#flush()
     }
 
-    this.#setBatch.push({ key, data, ttl, stale })
+    this.#setBatch.push({ key, data, ttl, stale, hash: this.#lockArray ? HASHER.h32(key) : -1 })
   }
 
   #flush = () => {
@@ -628,18 +670,31 @@ export class Cache                                              extends EventEmi
         try {
           this.#database.exec('BEGIN')
           while (n < this.#setBatch.length) {
-            const { key, data, ttl, stale } = this.#setBatch[n++]
+            const { key, data, ttl, stale, hash } = this.#setBatch[n++]
             if (data != null) {
               this.#setQuery?.run(key, data, ttl, stale)
             } else {
               this.#delQuery?.run(key)
             }
+
+            if (this.#lockSet && this.#lockArray && hash >= 0) {
+              this.#lockSet.add(hash % this.#lockArray.length)
+            }
+
             if ((n & 0xf) === 0 && performance.now() - startTime > 10) {
               break
             }
           }
           this.#database.exec('COMMIT')
           this.#setBatch.splice(0, n)
+
+          if (this.#lockSet && this.#lockArray) {
+            for (const idx of this.#lockSet) {
+              Atomics.store(this.#lockArray, idx, 0)
+              Atomics.notify(this.#lockArray, idx)
+            }
+            this.#lockSet.clear()
+          }
           break
         } catch (err) {
           // ROLLBACK is required: a failed statement leaves the connection with
@@ -660,6 +715,11 @@ export class Cache                                              extends EventEmi
           ) {
             this.#evictQuery.run(256)
           } else {
+            // Intentional: drop the rolled-back items from the batch and surface
+            // the error via #emitError below. The corresponding entries remain in
+            // #memory until natural eviction/TTL, which is fine for cache semantics
+            // — callers already accept that cache values can disappear at any time.
+            // Do NOT flag this as silent data loss (see closed issue #167).
             this.#setBatch.splice(0, n)
             throw err
           }
@@ -707,8 +767,8 @@ export class Cache                                              extends EventEmi
   }
 
   // Returns lock_acquired timestamp (number) when contended, or a string status.
-  #tryAcquireLock(key        )                                      {
-    if (this.#lockAcquireQuery === null) {
+  #tryAcquireLock(key        , lockIdx        )                                      {
+    if (this.#lockAcquireQuery === null || this.#lockId == null) {
       return 'unavailable'
     }
 
@@ -724,6 +784,9 @@ export class Cache                                              extends EventEmi
       }
 
       if (row.lock_owner === this.#lockId) {
+        if (this.#lockArray && lockIdx >= 0) {
+          Atomics.store(this.#lockArray, lockIdx, now - this.#lockOrigin)
+        }
         return 'acquired'
       }
 
@@ -734,6 +797,9 @@ export class Cache                                              extends EventEmi
         // process wins if multiple try to steal concurrently.
         const stealResult = this.#lockStealQuery .run(now, this.#lockId, key, lockedAt)
         if (stealResult.changes === 1) {
+          if (this.#lockArray && lockIdx >= 0) {
+            Atomics.store(this.#lockArray, lockIdx, now - this.#lockOrigin)
+          }
           return 'acquired'
         }
 
@@ -774,16 +840,33 @@ export class Cache                                              extends EventEmi
   // If no result found, take over by calling valueSelector ourselves.
   // selfPromise is the exact promise stored in #dedupe, used for identity checks
   // to avoid double valueSelector calls after delete()+get() races.
-  async #waitForValue(args   , key        , lockedAt        , selfPromise            )             {
+  async #waitForValue(
+    args   ,
+    key        ,
+    lockIdx        ,
+    lockedAt        ,
+    selfPromise            ,
+  )             {
+    // Wait for estimated completion: locked_at + our EMA-based timeout.
     // Loop: wait for lock holder to write a value, retry if lock was stolen by another process.
     for (let retries = 0; retries < 2 && this.#dedupe.get(key) === selfPromise; retries++) {
-      // Wait for estimated completion: locked_at + our EMA-based timeout.
-      const waitTime = Math.max(0, lockedAt + this.#lockTimeout - Date.now())
-      if (waitTime > 0) {
-        // Add 20% jitter to reduce thundering herd on contention
-        await delay(waitTime + Math.floor(Math.random() * waitTime * 0.2), undefined, {
-          ref: false,
-        })
+      // Add 5% jitter to reduce thundering herd on contention
+      const waitTime = Math.ceil(
+        Math.max(0, lockedAt + this.#lockTimeout - Date.now()) * (1 + Math.random() * 0.05),
+      )
+
+      if (this.#lockArray != null && lockIdx >= 0) {
+        const { async, value } = Atomics.waitAsync(
+          this.#lockArray,
+          lockIdx,
+          lockedAt - this.#lockOrigin,
+          waitTime,
+        )
+        if (async) {
+          await value
+        }
+      } else {
+        await delay(waitTime, undefined, { ref: false })
       }
 
       // Check if the lock holder wrote a value
@@ -802,7 +885,7 @@ export class Cache                                              extends EventEmi
       }
 
       // Try to acquire lock for takeover
-      const lockResult = this.#tryAcquireLock(key)
+      const lockResult = this.#tryAcquireLock(key, lockIdx)
       if (typeof lockResult === 'number') {
         // Another process holds the lock — wait for them
         lockedAt = lockResult

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@nxtedition/cache",
-  "version": "2.1.9",
+  "version": "2.1.11",
   "type": "module",
   "main": "lib/index.js",
   "types": "lib/index.d.ts",
@@ -30,5 +30,9 @@
     "tsd": "^0.33.0",
     "typescript": "^5.9.3"
   },
-  "gitHead": "0bc928c692badb55bdf3c6928eea7742b8e17732"
+  "gitHead": "46cd2017c0b9d238f2509e3612afbdb11fbefb69",
+  "dependencies": {
+    "@nxtedition/shared": "^5.1.10",
+    "xxhash-wasm": "^1.1.0"
+  }
 }