cacheable 1.8.10 → 1.10.0

package/README.md

@@ -29,6 +29,7 @@
 * [Basic Usage](#basic-usage)
 * [Hooks and Events](#hooks-and-events)
 * [Storage Tiering and Caching](#storage-tiering-and-caching)
+* [TTL Propagation and Storage Tiering](#ttl-propagation-and-storage-tiering)
 * [Shorthand for Time to Live (ttl)](#shorthand-for-time-to-live-ttl)
 * [Non-Blocking Operations](#non-blocking-operations)
 * [CacheSync - Distributed Updates](#cachesync---distributed-updates)
@@ -36,10 +37,13 @@
 * [Cacheable Statistics (Instance Only)](#cacheable-statistics-instance-only)
 * [Cacheable - API](#cacheable---api)
 * [CacheableMemory - In-Memory Cache](#cacheablememory---in-memory-cache)
+* [CacheableMemory Store Hashing](#cacheablememory-store-hashing)
+* [CacheableMemory LRU Feature](#cacheablememory-lru-feature)
+* [CacheableMemory Performance](#cacheablememory-performance)
 * [CacheableMemory Options](#cacheablememory-options)
 * [CacheableMemory - API](#cacheablememory---api)
-* [Wrap / Memoization for Sync and Async Functions](#wrap--memoization-for-sync-and-async-functions)
 * [Keyv Storage Adapter - KeyvCacheableMemory](#keyv-storage-adapter---keyvcacheablememory)
+* [Wrap / Memoization for Sync and Async Functions](#wrap--memoization-for-sync-and-async-functions)
 * [How to Contribute](#how-to-contribute)
 * [License and Copyright](#license-and-copyright)
 
@@ -98,6 +102,7 @@ The following hooks are available for you to extend the functionality of `cachea
 * `AFTER_GET`: This is called after the `get()` method is called.
 * `BEFORE_GET_MANY`: This is called before the `getMany()` method is called.
 * `AFTER_GET_MANY`: This is called after the `getMany()` method is called.
+* `BEFORE_SECONDARY_SETS_PRIMARY`: This is called when the secondary store sets the value in the primary store.
 
 An example of how to use these hooks:
 
@@ -110,6 +115,19 @@ cacheable.onHook(CacheableHooks.BEFORE_SET, (data) => {
 });
 ```
 
+Here is an example of how to use `BEFORE_SECONDARY_SETS_PRIMARY` hook:
+
+```javascript
+import { Cacheable, CacheableHooks } from 'cacheable';
+import KeyvRedis from '@keyv/redis';
+const secondary = new KeyvRedis('redis://user:pass@localhost:6379');
+const cache = new Cacheable({secondary});
+cache.onHook(CacheableHooks.BEFORE_SECONDARY_SETS_PRIMARY, (data) => {
+  console.log(`before secondary sets primary: ${data.key} ${data.value} ${data.ttl}`);
+});
+```
+This is called when the secondary store sets the value in the primary store. This is useful if you want to do something before the value is set in the primary store such as manipulating the ttl or the value.
+
 # Storage Tiering and Caching
 
 `cacheable` is built as a layer 1 and layer 2 caching engine by default. The purpose is to have your layer 1 be fast and your layer 2 be more persistent. The primary store is the layer 1 cache and the secondary store is the layer 2 cache. By adding the secondary store you are enabling layer 2 caching. By default the operations are blocking but fault tolerant:
@@ -119,6 +137,48 @@ cacheable.onHook(CacheableHooks.BEFORE_SET, (data) => {
 * `Deleting Data`: Deletes the value from the primary store and secondary store at the same time waiting for both to respond.
 * `Clearing Data`: Clears the primary store and secondary store at the same time waiting for both to respond.
 
+When `Getting Data` if the value does not exist in the primary store it will try to get it from the secondary store. If the secondary store returns the value it will set it in the primary store. Because we use [TTL Propagation](#ttl-propagation-and-storage-tiering) the value will be set in the primary store with the TTL of the secondary store unless the time to live (TTL) is greater than the primary store which will then use the TTL of the primary store. An example of this is:
+
+```javascript
+import { Cacheable } from 'cacheable';
+import KeyvRedis from '@keyv/redis';
+const secondary = new KeyvRedis('redis://user:pass@localhost:6379', { ttl: 1000 });
+const cache = new Cacheable({secondary, ttl: 100});
+
+await cache.set('key', 'value'); // sets the value in the primary store with a ttl of 100 ms and secondary store with a ttl of 1000 ms
+
+await sleep(500); // wait for .5 seconds
+
+const value = await cache.get('key'); // gets the value from the secondary store and now sets the value in the primary store with a ttl of 500 ms which is what is left from the secondary store
+```
+
+In this example the primary store has a ttl of `100 ms` and the secondary store has a ttl of `1000 ms`. Because the ttl is greater in the secondary store it will default to setting ttl value in the primary store.
+
+```javascript
+import { Cacheable } from 'cacheable';
+import {Keyv} from 'keyv';
+import KeyvRedis from '@keyv/redis';
+const primary = new Keyv({ ttl: 200 });
+const secondary = new KeyvRedis('redis://user:pass@localhost:6379', { ttl: 1000 });
+const cache = new Cacheable({primary, secondary});
+
+await cache.set('key', 'value'); // sets the value in the primary store with a ttl of 100 ms and secondary store with a ttl of 1000 ms
+
+await sleep(200); // wait for .2 seconds
+
+const value = await cache.get('key'); // gets the value from the secondary store and now sets the value in the primary store with a ttl of 200 ms which is what the primary store is set with
+```
+
+# TTL Propagation and Storage Tiering
+
+Cacheable TTL propagation is a feature that allows you to set a time to live (TTL) for the cache. By default the TTL is set in the following order:
+
+```
+ttl = set at the function ?? storage adapter ttl ?? cacheable ttl
+```
+
+This means that if you set a TTL at the function level it will override the storage adapter TTL and the cacheable TTL. If you do not set a TTL at the function level it will use the storage adapter TTL and then the cacheable TTL. If you do not set a TTL at all it will use the default TTL of `undefined` which is disabled.
+
 # Shorthand for Time to Live (ttl)
 
 By default `Cacheable` and `CacheableMemory` the `ttl` is in milliseconds but you can use shorthand for the time to live. Here are the following shorthand values:
@@ -157,10 +217,79 @@ cache.ttl = -1; // sets the default ttl to 0 which is disabled
 console.log(cache.ttl); // undefined
 ```
 
+## Retrieving raw cache entries
+
+The `get` and `getMany` methods support a `raw` option, which returns the full stored metadata (`StoredDataRaw<T>`) instead of just the value:
+
+```typescript
+import { Cacheable } from 'cacheable';
+
+const cache = new Cacheable();
+
+// store a value
+await cache.set('user:1', { name: 'Alice' });
+
+// default: only the value
+const user = await cache.get<{ name: string }>('user:1');
+console.log(user); // { name: 'Alice' }
+
+// with raw: full record including expiration
+const raw = await cache.get<{ name: string }>('user:1', { raw: true });
+console.log(raw.value);   // { name: 'Alice' }
+console.log(raw.expires); // e.g. 1677628495000 or null
+```
+
+```typescript
+// getMany with raw option
+await cache.set('a', 1);
+await cache.set('b', 2);
+
+const raws = await cache.getMany<number>(['a', 'b'], { raw: true });
+raws.forEach((entry, idx) => {
+  console.log(`key=${['a','b'][idx]}, value=${entry?.value}, expires=${entry?.expires}`);
+});
+```
+
+
 # Non-Blocking Operations
 
 If you want your layer 2 (secondary) store to be non-blocking you can set the `nonBlocking` property to `true` in the options. This will make the secondary store non-blocking and will not wait for the secondary store to respond on `setting data`, `deleting data`, or `clearing data`. This is useful if you want to have a faster response time and not wait for the secondary store to respond.
 
+# GetOrSet
+
+The `getOrSet` method provides a convenient way to implement the cache-aside pattern. It attempts to retrieve a value
+from cache, and if not found, calls the provided function to compute the value and store it in cache before returning
+it.
+
+```typescript
+import { Cacheable } from 'cacheable';
+
+// Create a new Cacheable instance
+const cache = new Cacheable();
+
+// Use getOrSet to fetch user data
+async function getUserData(userId: string) {
+  return await cache.getOrSet(
+    `user:${userId}`,
+    async () => {
+      // This function only runs if the data isn't in the cache
+      console.log('Fetching user from database...');
+      // Simulate database fetch
+      return { id: userId, name: 'John Doe', email: 'john@example.com' };
+    },
+    { ttl: '30m' } // Cache for 30 minutes
+  );
+}
+
+// First call - will fetch from "database"
+const user1 = await getUserData('123');
+console.log(user1); // { id: '123', name: 'John Doe', email: 'john@example.com' }
+
+// Second call - will retrieve from cache
+const user2 = await getUserData('123');
+console.log(user2); // Same data, but retrieved from cache
+```
+
 ```javascript
 import { Cacheable } from 'cacheable';
 import {KeyvRedis} from '@keyv/redis';
@@ -215,7 +344,9 @@ _This does not enable statistics for your layer 2 cache as that is a distributed
 * `set(key, value, ttl?)`: Sets a value in the cache.
 * `setMany([{key, value, ttl?}])`: Sets multiple values in the cache.
 * `get(key)`: Gets a value from the cache.
+* `get(key, { raw: true })`: Gets a raw value from the cache.
 * `getMany([keys])`: Gets multiple values from the cache.
+* `getMany([keys], { raw: true })`: Gets multiple raw values from the cache.
 * `has(key)`: Checks if a value exists in the cache.
 * `hasMany([keys])`: Checks if multiple values exist in the cache.
 * `take(key)`: Takes a value from the cache and deletes it.
@@ -224,6 +355,7 @@ _This does not enable statistics for your layer 2 cache as that is a distributed
 * `deleteMany([keys])`: Deletes multiple values from the cache.
 * `clear()`: Clears the cache stores. Be careful with this as it will clear both layer 1 and layer 2.
 * `wrap(function, WrapOptions)`: Wraps an `async` function in a cache.
+* `getOrSet(key, valueFunction, ttl?)`: Gets a value from cache or sets it if not found using the provided function.
 * `disconnect()`: Disconnects from the cache stores.
 * `onHook(hook, callback)`: Sets a hook.
 * `removeHook(hook)`: Removes a hook.
@@ -258,12 +390,111 @@ This simple in-memory cache uses multiple Map objects and a with `expiration` an
 
 By default we use lazy expiration deletion which means on `get` and `getMany` type functions we look if it is expired and then delete it. If you want to have a more aggressive expiration policy you can set the `checkInterval` property to a value greater than `0` which will check for expired keys at the interval you set.
 
+Here are some of the main features of `CacheableMemory`:
+* High performance in-memory cache with a robust API and feature set. 🚀
+* Can scale past the `16,777,216 (2^24) keys` limit of a single `Map` via `hashStoreSize`. Default is `16` Map objects.
+* LRU (Least Recently Used) cache feature to limit the number of keys in the cache via `lruSize`. Limit to `16,777,216 (2^24) keys` total.
+* Expiration policy to delete expired keys with lazy deletion or aggressive deletion via `checkInterval`.
+* `Wrap` feature to memoize `sync` and `async` functions with stampede protection.
+* Ability to do many operations at once such as `setMany`, `getMany`, `deleteMany`, and `takeMany`.
+* Supports `raw` data retrieval with `getRaw` and `getManyRaw` methods to get the full metadata of the cache entry.
+
+## CacheableMemory Store Hashing
+
+`CacheableMemory` uses `Map` objects to store the keys and values. To make this scale past the `16,777,216 (2^24) keys` limit of a single `Map` we use a hash to balance the data across multiple `Map` objects. This is done by hashing the key and using the hash to determine which `Map` object to use. The default hashing algorithm is `djb2Hash` but you can change it by setting the `storeHashAlgorithm` property in the options. By default we set the amount of `Map` objects to `16`. 
+
+NOTE: if you are using the LRU cache feature the `lruSize` no matter how many `Map` objects you have it will be limited to the `16,777,216 (2^24) keys` limit of a single `Map` object. This is because we use a double linked list to manage the LRU cache and it is not possible to have more than `16,777,216 (2^24) keys` in a single `Map` object.
+
+Here is an example of how to set the number of `Map` objects and the hashing algorithm:
+
+```javascript
+import { CacheableMemory } from 'cacheable';
+const cache = new CacheableMemory({
+  storeSize: 32, // set the number of Map objects to 32
+});
+cache.set('key', 'value');
+const value = cache.get('key'); // value
+```
+
+Here is an example of how to use the `storeHashAlgorithm` property:
+
+```javascript
+import { CacheableMemory } from 'cacheable';
+const cache = new CacheableMemory({ storeHashAlgorithm: 'sha256' });
+cache.set('key', 'value');
+const value = cache.get('key'); // value
+```
+
+If you want to provide your own hashing function you can set the `storeHashAlgorithm` property to a function that takes an object and returns a `number` that is in the range of the amount of `Map` stores you have.
+
+```javascript
+import { CacheableMemory } from 'cacheable';
+/**
+ * Custom hash function that takes a key and the size of the store
+ * and returns a number between 0 and storeHashSize - 1.
+ * @param {string} key - The key to hash.
+ * @param {number} storeHashSize - The size of the store (number of Map objects).
+ * @returns {number} - A number between 0 and storeHashSize - 1.
+ */
+const customHash = (key, storeHashSize) => {
+  // custom hashing logic
+  return key.length % storeHashSize; // returns a number between 0 and 31 for 32 Map objects
+};
+const cache = new CacheableMemory({ storeHashAlgorithm: customHash, storeSize: 32 });
+cache.set('key', 'value');
+const value = cache.get('key'); // value
+```
+
+## CacheableMemory LRU Feature
+
+You can enable the LRU (Least Recently Used) feature in `CacheableMemory` by setting the `lruSize` property in the options. This will limit the number of keys in the cache to the size you set. When the cache reaches the limit it will remove the least recently used keys from the cache. This is useful if you want to limit the memory usage of the cache.
+
+When you set the `lruSize` we use a double linked list to manage the LRU cache and also set the `hashStoreSize` to `1` which means we will only use a single `Map` object for the LRU cache. This is because the LRU cache is managed by the double linked list and it is not possible to have more than `16,777,216 (2^24) keys` in a single `Map` object.
+
+```javascript
+import { CacheableMemory } from 'cacheable';
+const cache = new CacheableMemory({ lruSize: 1 }); // sets the LRU cache size to 1000 keys and hashStoreSize to 1
+cache.set('key1', 'value1');
+cache.set('key2', 'value2');
+const value1 = cache.get('key1');
+console.log(value1); // undefined if the cache is full and key1 is the least recently used
+const value2 = cache.get('key2');
+console.log(value2); // value2 if key2 is still in the cache
+console.log(cache.size()); // 1
+```
+
+NOTE: if you set the `lruSize` property to `0` after it was enabled it will disable the LRU cache feature and will not limit the number of keys in the cache. This will remove the `16,777,216 (2^24) keys` limit of a single `Map` object and will allow you to store more keys in the cache.
+
+## CacheableMemory Performance
+
+Our goal with `cacheable` and `CacheableMemory` is to provide a high performance caching engine that is simple to use and has a robust API. We test it against other cacheing engines such that are less feature rich to make sure there is little difference. Here are some of the benchmarks we have run:
+
+*Memory Benchmark Results:*
+|                   name                   |  summary  |  ops/sec  |  time/op  |  margin  |  samples  |
+|------------------------------------------|:---------:|----------:|----------:|:--------:|----------:|
+|  Map (v22) - set / get                   |    🥇     |     117K  |      9µs  |  ±1.29%  |     110K  |
+|  Cacheable Memory (v1.10.0) - set / get  |   -1.3%   |     116K  |      9µs  |  ±0.77%  |     110K  |
+|  Node Cache - set / get                  |   -4.1%   |     112K  |      9µs  |  ±1.34%  |     107K  |
+|  bentocache (v1.4.0) - set / get         |   -45%    |      65K  |     17µs  |  ±1.10%  |     100K  |
+
+*Memory LRU Benchmark Results:*
+|                   name                   |  summary  |  ops/sec  |  time/op  |  margin  |  samples  |
+|------------------------------------------|:---------:|----------:|----------:|:--------:|----------:|
+|  quick-lru (v7.0.1) - set / get          |    🥇     |     118K  |      9µs  |  ±0.85%  |     112K  |
+|  Map (v22) - set / get                   |  -0.56%   |     117K  |      9µs  |  ±1.35%  |     110K  |
+|  lru.min (v1.1.2) - set / get            |   -1.7%   |     116K  |      9µs  |  ±0.90%  |     110K  |
+|  Cacheable Memory (v1.10.0) - set / get  |   -3.3%   |     114K  |      9µs  |  ±1.16%  |     108K  |
+
+As you can see from the benchmarks `CacheableMemory` is on par with other caching engines such as `Map`, `Node Cache`, and `bentocache`. We have also tested it against other LRU caching engines such as `quick-lru` and `lru.min` and it performs well against them too.
+
 ## CacheableMemory Options
 
 * `ttl`: The time to live for the cache in milliseconds. Default is `undefined` which is means indefinitely.
 * `useClones`: If the cache should use clones for the values. Default is `true`.
 * `lruSize`: The size of the LRU cache. Default is `0` which is unlimited.
 * `checkInterval`: The interval to check for expired keys in milliseconds. Default is `0` which is disabled.
+* `storeHashSize`: The number of `Map` objects to use for the cache. Default is `16`.
+* `storeHashAlgorithm`: The hashing algorithm to use for the cache. Default is `djb2Hash`.
 
 ## CacheableMemory - API
 
@@ -281,13 +512,33 @@ By default we use lazy expiration deletion which means on `get` and `getMany` ty
 * `takeMany([keys])`: Takes multiple values from the cache and deletes them.
 * `wrap(function, WrapSyncOptions)`: Wraps a `sync` function in a cache.
 * `clear()`: Clears the cache.
-* `size()`: The number of keys in the cache.
-* `keys()`: The keys in the cache.
-* `items()`: The items in the cache as `CacheableStoreItem` example `{ key, value, expires? }`.
+* `ttl`: The default time to live for the cache in milliseconds. Default is `undefined` which is disabled.
+* `useClones`: If the cache should use clones for the values. Default is `true`.
+* `lruSize`: The size of the LRU cache. Default is `0` which is unlimited.
+* `size`: The number of keys in the cache.
+* `checkInterval`: The interval to check for expired keys in milliseconds. Default is `0` which is disabled.
+* `storeHashSize`: The number of `Map` objects to use for the cache. Default is `16`.
+* `storeHashAlgorithm`: The hashing algorithm to use for the cache. Default is `djb2Hash`.
+* `keys`: Get the keys in the cache. Not able to be set.
+* `items`: Get the items in the cache as `CacheableStoreItem` example `{ key, value, expires? }`.
+* `store`: The hash store for the cache which is an array of `Map` objects.
 * `checkExpired()`: Checks for expired keys in the cache. This is used by the `checkInterval` property.
 * `startIntervalCheck()`: Starts the interval check for expired keys if `checkInterval` is above 0 ms.
 * `stopIntervalCheck()`: Stops the interval check for expired keys.
-* `hash(object: any, algorithm = 'sha256'): string`: Hashes an object with the algorithm. Default is `sha256`.
+
+# Keyv Storage Adapter - KeyvCacheableMemory
+
+`cacheable` comes with a built-in storage adapter for Keyv called `KeyvCacheableMemory`. This takes `CacheableMemory` and creates a storage adapter for Keyv. This is useful if you want to use `CacheableMemory` as a storage adapter for Keyv. Here is an example of how to use `KeyvCacheableMemory`:
+
+```javascript
+import { Keyv } from 'keyv';
+import { KeyvCacheableMemory } from 'cacheable';
+
+const keyv = new Keyv({ store: new KeyvCacheableMemory() });
+await keyv.set('foo', 'bar');
+const value = await keyv.get('foo');
+console.log(value); // bar 
+```
 
 # Wrap / Memoization for Sync and Async Functions
 
@@ -363,23 +614,9 @@ console.log(wrappedFunction()); // error
 console.log(wrappedFunction()); // error from cache
 ```
 
-# Keyv Storage Adapter - KeyvCacheableMemory
-
-`cacheable` comes with a built-in storage adapter for Keyv called `KeyvCacheableMemory`. This takes `CacheableMemory` and creates a storage adapter for Keyv. This is useful if you want to use `CacheableMemory` as a storage adapter for Keyv. Here is an example of how to use `KeyvCacheableMemory`:
-
-```javascript
-import { Keyv } from 'keyv';
-import { KeyvCacheableMemory } from 'cacheable';
-
-const keyv = new Keyv({ store: new KeyvCacheableMemory() });
-await keyv.set('foo', 'bar');
-const value = await keyv.get('foo');
-console.log(value); // bar 
-```
-
 # How to Contribute
 
 You can contribute by forking the repo and submitting a pull request. Please make sure to add tests and update the documentation. To learn more about how to contribute go to our main README [https://github.com/jaredwray/cacheable](https://github.com/jaredwray/cacheable). This will talk about how to `Open a Pull Request`, `Ask a Question`, or `Post an Issue`.
 
 # License and Copyright
-[MIT © Jared Wray](./LICENSE)
+[MIT © Jared Wray](./LICENSE)