valkyries 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +120 -0
- package/dist/bloom-filter/batching.d.mts +3 -0
- package/dist/bloom-filter/batching.d.mts.map +1 -0
- package/dist/bloom-filter/batching.mjs +18 -0
- package/dist/bloom-filter/batching.mjs.map +1 -0
- package/dist/bloom-filter/lookups.d.mts +6 -0
- package/dist/bloom-filter/lookups.d.mts.map +1 -0
- package/dist/bloom-filter/lookups.mjs +101 -0
- package/dist/bloom-filter/lookups.mjs.map +1 -0
- package/dist/bloom-filter/rebuilds.d.mts +9 -0
- package/dist/bloom-filter/rebuilds.d.mts.map +1 -0
- package/dist/bloom-filter/rebuilds.mjs +66 -0
- package/dist/bloom-filter/rebuilds.mjs.map +1 -0
- package/dist/bloom-filter/results.d.mts +3 -0
- package/dist/bloom-filter/results.d.mts.map +1 -0
- package/dist/bloom-filter/results.mjs +23 -0
- package/dist/bloom-filter/results.mjs.map +1 -0
- package/dist/bloom-filter/scripts.d.mts +9 -0
- package/dist/bloom-filter/scripts.d.mts.map +1 -0
- package/dist/bloom-filter/scripts.mjs +10 -0
- package/dist/bloom-filter/scripts.mjs.map +1 -0
- package/dist/bloom-filter/types.d.mts +12 -0
- package/dist/bloom-filter/types.d.mts.map +1 -0
- package/dist/bloom-filter/types.mjs +2 -0
- package/dist/bloom-filter/types.mjs.map +1 -0
- package/dist/bloom-filter/writes.d.mts +5 -0
- package/dist/bloom-filter/writes.d.mts.map +1 -0
- package/dist/bloom-filter/writes.mjs +54 -0
- package/dist/bloom-filter/writes.mjs.map +1 -0
- package/dist/bloom-filter.d.mts +90 -0
- package/dist/bloom-filter.d.mts.map +1 -0
- package/dist/bloom-filter.mjs +154 -0
- package/dist/bloom-filter.mjs.map +1 -0
- package/dist/cache/batch-read.d.mts +7 -0
- package/dist/cache/batch-read.d.mts.map +1 -0
- package/dist/cache/batch-read.mjs +142 -0
- package/dist/cache/batch-read.mjs.map +1 -0
- package/dist/cache/constants.d.mts +7 -0
- package/dist/cache/constants.d.mts.map +1 -0
- package/dist/cache/constants.mjs +8 -0
- package/dist/cache/constants.mjs.map +1 -0
- package/dist/cache/core.d.mts +38 -0
- package/dist/cache/core.d.mts.map +1 -0
- package/dist/cache/core.mjs +171 -0
- package/dist/cache/core.mjs.map +1 -0
- package/dist/cache/deletes.d.mts +8 -0
- package/dist/cache/deletes.d.mts.map +1 -0
- package/dist/cache/deletes.mjs +44 -0
- package/dist/cache/deletes.mjs.map +1 -0
- package/dist/cache/mutations.d.mts +25 -0
- package/dist/cache/mutations.d.mts.map +1 -0
- package/dist/cache/mutations.mjs +181 -0
- package/dist/cache/mutations.mjs.map +1 -0
- package/dist/cache/set-if-not-invalidated-events.d.mts +4 -0
- package/dist/cache/set-if-not-invalidated-events.d.mts.map +1 -0
- package/dist/cache/set-if-not-invalidated-events.mjs +21 -0
- package/dist/cache/set-if-not-invalidated-events.mjs.map +1 -0
- package/dist/cache/single-read.d.mts +11 -0
- package/dist/cache/single-read.d.mts.map +1 -0
- package/dist/cache/single-read.mjs +73 -0
- package/dist/cache/single-read.mjs.map +1 -0
- package/dist/cache/stale-refresh.d.mts +15 -0
- package/dist/cache/stale-refresh.d.mts.map +1 -0
- package/dist/cache/stale-refresh.mjs +70 -0
- package/dist/cache/stale-refresh.mjs.map +1 -0
- package/dist/cache-metrics.d.mts +10 -0
- package/dist/cache-metrics.d.mts.map +1 -0
- package/dist/cache-metrics.mjs +12 -0
- package/dist/cache-metrics.mjs.map +1 -0
- package/dist/cache-utils.d.mts +20 -0
- package/dist/cache-utils.d.mts.map +1 -0
- package/dist/cache-utils.mjs +67 -0
- package/dist/cache-utils.mjs.map +1 -0
- package/dist/cache.d.mts +4 -0
- package/dist/cache.d.mts.map +1 -0
- package/dist/cache.mjs +4 -0
- package/dist/cache.mjs.map +1 -0
- package/dist/clients.d.mts +51 -0
- package/dist/clients.d.mts.map +1 -0
- package/dist/clients.mjs +95 -0
- package/dist/clients.mjs.map +1 -0
- package/dist/config.d.mts +6 -0
- package/dist/config.d.mts.map +1 -0
- package/dist/config.mjs +10 -0
- package/dist/config.mjs.map +1 -0
- package/dist/delete.d.mts +3 -0
- package/dist/delete.d.mts.map +1 -0
- package/dist/delete.mjs +23 -0
- package/dist/delete.mjs.map +1 -0
- package/dist/dynamic-config/fields.d.mts +6 -0
- package/dist/dynamic-config/fields.d.mts.map +1 -0
- package/dist/dynamic-config/fields.mjs +50 -0
- package/dist/dynamic-config/fields.mjs.map +1 -0
- package/dist/dynamic-config/storage.d.mts +33 -0
- package/dist/dynamic-config/storage.d.mts.map +1 -0
- package/dist/dynamic-config/storage.mjs +55 -0
- package/dist/dynamic-config/storage.mjs.map +1 -0
- package/dist/dynamic-config.d.mts +34 -0
- package/dist/dynamic-config.d.mts.map +1 -0
- package/dist/dynamic-config.mjs +173 -0
- package/dist/dynamic-config.mjs.map +1 -0
- package/dist/errors.d.mts +4 -0
- package/dist/errors.d.mts.map +1 -0
- package/dist/errors.mjs +15 -0
- package/dist/errors.mjs.map +1 -0
- package/dist/events.d.mts +84 -0
- package/dist/events.d.mts.map +1 -0
- package/dist/events.mjs +18 -0
- package/dist/events.mjs.map +1 -0
- package/dist/index.d.mts +14 -0
- package/dist/index.d.mts.map +1 -0
- package/dist/index.mjs +13 -0
- package/dist/index.mjs.map +1 -0
- package/dist/key-normalization.d.mts +2 -0
- package/dist/key-normalization.d.mts.map +1 -0
- package/dist/key-normalization.mjs +4 -0
- package/dist/key-normalization.mjs.map +1 -0
- package/dist/rate-limiter.d.mts +40 -0
- package/dist/rate-limiter.d.mts.map +1 -0
- package/dist/rate-limiter.mjs +123 -0
- package/dist/rate-limiter.mjs.map +1 -0
- package/dist/scripts/bloom-filter-add.lua +26 -0
- package/dist/scripts/bloom-filter-ensure-exists.lua +13 -0
- package/dist/scripts/bloom-filter-exists-if-ready.lua +14 -0
- package/dist/scripts/bloom-filter-exists.lua +14 -0
- package/dist/scripts/bloom-filter-mexists-if-ready.lua +19 -0
- package/dist/scripts/bloom-filter-mexists.lua +17 -0
- package/dist/scripts/bloom-filter-reserve.lua +11 -0
- package/dist/scripts/cache-delete-with-invalidation.lua +29 -0
- package/dist/scripts/cache-set-if-not-invalidated.lua +47 -0
- package/dist/scripts/dynamic-config-set-fields.lua +23 -0
- package/dist/scripts/get-value-with-ttl.lua +35 -0
- package/dist/scripts/get-values-with-ttl.lua +81 -0
- package/dist/scripts/rate-limiter-add-and-check.lua +34 -0
- package/dist/scripts/rate-limiter-add.lua +26 -0
- package/dist/scripts/rate-limiter-get.lua +22 -0
- package/dist/scripts.d.mts +4 -0
- package/dist/scripts.d.mts.map +1 -0
- package/dist/scripts.mjs +31 -0
- package/dist/scripts.mjs.map +1 -0
- package/dist/shutdown.d.mts +3 -0
- package/dist/shutdown.d.mts.map +1 -0
- package/dist/shutdown.mjs +27 -0
- package/dist/shutdown.mjs.map +1 -0
- package/dist/types.d.mts +76 -0
- package/dist/types.d.mts.map +1 -0
- package/dist/types.mjs +2 -0
- package/dist/types.mjs.map +1 -0
- package/dist/utils.d.mts +16 -0
- package/dist/utils.d.mts.map +1 -0
- package/dist/utils.mjs +35 -0
- package/dist/utils.mjs.map +1 -0
- package/docs/bloom-filters.md +12 -0
- package/docs/cache.md +19 -0
- package/docs/clients.md +13 -0
- package/docs/configuration.md +16 -0
- package/docs/dynamic-config.md +13 -0
- package/docs/events-and-metrics.md +15 -0
- package/docs/lua-scripts.md +7 -0
- package/docs/migration.md +74 -0
- package/docs/rate-limiter.md +17 -0
- package/docs/testing-and-ci.md +21 -0
- package/package.json +80 -0
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
-- Get value with TTL operation (with optional bloom filter check)
|
|
2
|
+
-- This script is read-only and is executed using the read client
|
|
3
|
+
-- KEYS[1] = cache key (format: namespace:prefix:{serializedKey})
|
|
4
|
+
-- ARGV[1] = bloom filter key (optional, empty string if not provided)
|
|
5
|
+
-- Returns: {value, ttl_milliseconds, bloom_miss}
|
|
6
|
+
-- Where ttl_milliseconds is -2 if key doesn't exist, -1 if no expiry, or milliseconds remaining
|
|
7
|
+
-- bloom_miss is 1 if bloom filter says key definitely doesn't exist, 0 otherwise
|
|
8
|
+
if #KEYS == 0 then
|
|
9
|
+
return {}
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
local cacheKey = KEYS[1]
|
|
13
|
+
local bloomFilterKey = ARGV[1]
|
|
14
|
+
-- Check bloom filter if provided and it exists
|
|
15
|
+
if bloomFilterKey ~= '' then
|
|
16
|
+
local filterExists = redis.call('EXISTS', bloomFilterKey)
|
|
17
|
+
if filterExists == 1 then
|
|
18
|
+
-- Extract the entity key from inside {} (e.g., "cache:posts:{my-uuid}" -> "my-uuid")
|
|
19
|
+
local entityKey = cacheKey:match('{(.+)}')
|
|
20
|
+
if entityKey then
|
|
21
|
+
local bloomExists = redis.call('BF.EXISTS', bloomFilterKey, entityKey)
|
|
22
|
+
if bloomExists == 0 then
|
|
23
|
+
-- Bloom filter says this key definitely doesn't exist
|
|
24
|
+
return {false, -2, 1}
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
local value = redis.call('GET', cacheKey)
|
|
31
|
+
if not value then
|
|
32
|
+
return {false, -2, 0}
|
|
33
|
+
end
|
|
34
|
+
local ttl = redis.call('PTTL', cacheKey)
|
|
35
|
+
return {value, ttl, 0}
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
-- Get values with TTL operation (handles multiple keys with optional bloom filter check)
|
|
2
|
+
-- This script is read-only and is executed using the read client
|
|
3
|
+
-- KEYS[1..n] = cache keys (format: namespace:prefix:{serializedKey})
|
|
4
|
+
-- ARGV[1] = bloom filter key (optional, empty string if not provided)
|
|
5
|
+
-- Returns: {value1, ttl1, bloom_miss1, value2, ttl2, bloom_miss2, ...}
|
|
6
|
+
-- Where ttl is in milliseconds (-2 if key doesn't exist, -1 if no expiry)
|
|
7
|
+
-- bloom_miss is 1 if bloom filter says key definitely doesn't exist, 0 otherwise
|
|
8
|
+
if #KEYS == 0 then
|
|
9
|
+
return {}
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
local bloomFilterKey = ARGV[1]
|
|
13
|
+
-- Step 1: Check bloom filter before MGET to avoid fetching known-missing keys.
|
|
14
|
+
-- bloomResults[i] == 0: not in filter = bloom miss
|
|
15
|
+
-- bloomResults[i] == nil: no entity key extracted, skip check = not a bloom miss
|
|
16
|
+
-- bloomResults[i] == 1: in filter = not a bloom miss
|
|
17
|
+
local bloomResults = {}
|
|
18
|
+
if bloomFilterKey ~= '' then
|
|
19
|
+
local filterExists = redis.call('EXISTS', bloomFilterKey)
|
|
20
|
+
if filterExists == 1 then
|
|
21
|
+
local entityKeys = {}
|
|
22
|
+
local entityKeyIndices = {}
|
|
23
|
+
for i, key in ipairs(KEYS) do
|
|
24
|
+
local entityKey = key:match('{(.+)}')
|
|
25
|
+
if entityKey then
|
|
26
|
+
table.insert(entityKeys, entityKey)
|
|
27
|
+
table.insert(entityKeyIndices, i)
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
if #entityKeys > 0 then
|
|
31
|
+
local mexistsResults = redis.call('BF.MEXISTS', bloomFilterKey, unpack(entityKeys))
|
|
32
|
+
for j, result in ipairs(mexistsResults) do
|
|
33
|
+
bloomResults[entityKeyIndices[j]] = result
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
-- Step 2: Collect non-bloom-miss keys to fetch.
|
|
40
|
+
local fetchIndices = {}
|
|
41
|
+
local fetchKeys = {}
|
|
42
|
+
for i, key in ipairs(KEYS) do
|
|
43
|
+
if bloomResults[i] ~= 0 then
|
|
44
|
+
table.insert(fetchIndices, i)
|
|
45
|
+
table.insert(fetchKeys, key)
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
-- Step 3: MGET only the non-bloom-miss keys.
|
|
50
|
+
local fetched = {}
|
|
51
|
+
if #fetchKeys > 0 then
|
|
52
|
+
fetched = redis.call('MGET', unpack(fetchKeys))
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
-- Map fetched values back to original indices.
|
|
56
|
+
local valuesByIndex = {}
|
|
57
|
+
for j, idx in ipairs(fetchIndices) do
|
|
58
|
+
valuesByIndex[idx] = fetched[j]
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
-- Step 4: Build response.
|
|
62
|
+
local response = {}
|
|
63
|
+
for i, key in ipairs(KEYS) do
|
|
64
|
+
if bloomResults[i] == 0 then
|
|
65
|
+
-- Bloom miss: key definitely doesn't exist, no MGET or PTTL needed
|
|
66
|
+
table.insert(response, false)
|
|
67
|
+
table.insert(response, -2)
|
|
68
|
+
table.insert(response, 1)
|
|
69
|
+
else
|
|
70
|
+
local value = valuesByIndex[i]
|
|
71
|
+
local ttl = -2
|
|
72
|
+
if value then
|
|
73
|
+
ttl = redis.call('PTTL', key)
|
|
74
|
+
end
|
|
75
|
+
table.insert(response, value)
|
|
76
|
+
table.insert(response, ttl)
|
|
77
|
+
table.insert(response, 0)
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
return response
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
-- Rate limiter add-and-check operation (atomic add + count in one roundtrip)
|
|
2
|
+
-- KEYS[1..N]: the sorted set keys
|
|
3
|
+
-- ARGV[1]: TTL in seconds
|
|
4
|
+
-- ARGV[2..N+1]: random element strings (one per key)
|
|
5
|
+
-- Returns array of counts after adding (same format as rate-limiter-get.lua)
|
|
6
|
+
-- Uses server time to avoid client clock skew
|
|
7
|
+
|
|
8
|
+
assert(#ARGV == #KEYS + 1, 'ARGV length mismatch: expected #KEYS + 1 (ttl + one random element per key)')
|
|
9
|
+
|
|
10
|
+
local ttl = tonumber(ARGV[1])
|
|
11
|
+
local currentTime = redis.call('TIME')
|
|
12
|
+
local now = tonumber(currentTime[1]) * 1000 + tonumber(currentTime[2]) / 1000
|
|
13
|
+
local minScore = now - (ttl * 1000)
|
|
14
|
+
|
|
15
|
+
local response = {}
|
|
16
|
+
for i, key in ipairs(KEYS) do
|
|
17
|
+
local randomElement = ARGV[1 + i] -- ARGV[2] for KEYS[1], ARGV[3] for KEYS[2], etc.
|
|
18
|
+
|
|
19
|
+
-- Add the current timestamp as the score
|
|
20
|
+
redis.call('ZADD', key, now, randomElement)
|
|
21
|
+
|
|
22
|
+
-- Remove all scores older than the TTL
|
|
23
|
+
redis.call('ZREMRANGEBYSCORE', key, '-inf', minScore)
|
|
24
|
+
|
|
25
|
+
-- Always expire the key after the TTL
|
|
26
|
+
redis.call('EXPIRE', key, ttl)
|
|
27
|
+
|
|
28
|
+
-- Count items in the current window (exclusive lower bound: ZREMRANGEBYSCORE already
|
|
29
|
+
-- removed entries <= minScore, so use '(' to match only entries that survived cleanup)
|
|
30
|
+
local count = redis.call('ZCOUNT', key, '(' .. minScore, now)
|
|
31
|
+
table.insert(response, count)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
return response
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
-- Rate limiter add operation (handles multiple keys)
|
|
2
|
+
-- KEYS[1..N]: the sorted set keys
|
|
3
|
+
-- ARGV[1]: TTL in seconds
|
|
4
|
+
-- ARGV[2..N+1]: random element strings (one per key)
|
|
5
|
+
-- Uses server time to avoid client clock skew
|
|
6
|
+
|
|
7
|
+
local ttl = tonumber(ARGV[1])
|
|
8
|
+
local currentTime = redis.call('TIME')
|
|
9
|
+
local now = tonumber(currentTime[1]) * 1000 + tonumber(currentTime[2]) / 1000
|
|
10
|
+
local minScore = now - (ttl * 1000)
|
|
11
|
+
|
|
12
|
+
for i, key in ipairs(KEYS) do
|
|
13
|
+
local randomElement = ARGV[1 + i] -- ARGV[2] for KEYS[1], ARGV[3] for KEYS[2], etc.
|
|
14
|
+
|
|
15
|
+
-- Add the current timestamp as the score
|
|
16
|
+
redis.call('ZADD', key, now, randomElement)
|
|
17
|
+
|
|
18
|
+
-- Remove all scores older than the TTL (use -inf for clarity and correctness)
|
|
19
|
+
redis.call('ZREMRANGEBYSCORE', key, '-inf', minScore)
|
|
20
|
+
|
|
21
|
+
-- Always expire the key after the TTL
|
|
22
|
+
redis.call('EXPIRE', key, ttl)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
return 1
|
|
26
|
+
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
-- Rate limiter get operation (handles multiple keys)
|
|
2
|
+
-- NOTE: intentionally executed via rateLimiterValkeyClient (primary-only) to avoid
|
|
3
|
+
-- replica lag causing stale counts that could allow rate limit bypasses.
|
|
4
|
+
-- KEYS[1..N]: the sorted set keys
|
|
5
|
+
-- ARGV[1]: TTL in seconds
|
|
6
|
+
-- Uses server time to avoid client clock skew
|
|
7
|
+
|
|
8
|
+
local ttl = tonumber(ARGV[1])
|
|
9
|
+
local currentTime = redis.call('TIME')
|
|
10
|
+
local now = tonumber(currentTime[1]) * 1000 + tonumber(currentTime[2]) / 1000
|
|
11
|
+
local minScore = now - (ttl * 1000)
|
|
12
|
+
local maxScore = now
|
|
13
|
+
|
|
14
|
+
local response = {}
|
|
15
|
+
for _, key in ipairs(KEYS) do
|
|
16
|
+
-- Use exclusive lower bound to match add-and-check.lua behavior at boundary
|
|
17
|
+
local count = redis.call('ZCOUNT', key, '(' .. minScore, maxScore)
|
|
18
|
+
table.insert(response, count)
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
return response
|
|
22
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"scripts.d.mts","sourceRoot":"","sources":["../scripts.mts"],"names":[],"mappings":"AACA,OAAO,EAAE,MAAM,EAAE,MAAM,sBAAsB,CAAC;AAoB9C,wBAAgB,cAAc,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAKnD;AAED,wBAAgB,UAAU,CAAC,YAAY,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,GAAG,GAAG,MAAM,CAG9E"}
|
package/dist/scripts.mjs
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
import { readFileSync } from "node:fs";
|
|
2
|
+
import { Script } from "@valkey/valkey-glide";
|
|
3
|
+
const scriptRegistry = [];
|
|
4
|
+
let releaseHookRegistered = false;
|
|
5
|
+
function registerReleaseHook() {
|
|
6
|
+
if (releaseHookRegistered)
|
|
7
|
+
return;
|
|
8
|
+
releaseHookRegistered = true;
|
|
9
|
+
process.once("exit", () => {
|
|
10
|
+
for (const script of scriptRegistry) {
|
|
11
|
+
try {
|
|
12
|
+
script.release();
|
|
13
|
+
}
|
|
14
|
+
catch (err) {
|
|
15
|
+
// oxlint-disable-next-line no-console -- synchronous; onError queues async Sentry I/O which won't flush in an exit handler
|
|
16
|
+
console.error(err);
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
});
|
|
20
|
+
}
|
|
21
|
+
export function registerScript(code) {
|
|
22
|
+
registerReleaseHook();
|
|
23
|
+
const script = new Script(code);
|
|
24
|
+
scriptRegistry.push(script);
|
|
25
|
+
return script;
|
|
26
|
+
}
|
|
27
|
+
export function loadScript(relativePath, baseUrl) {
|
|
28
|
+
const fileUrl = new URL(`./scripts/${relativePath}`, baseUrl);
|
|
29
|
+
return readFileSync(fileUrl, "utf8");
|
|
30
|
+
}
|
|
31
|
+
//# sourceMappingURL=scripts.mjs.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"scripts.mjs","sourceRoot":"","sources":["../scripts.mts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,MAAM,EAAE,MAAM,sBAAsB,CAAC;AAE9C,MAAM,cAAc,GAAa,EAAE,CAAC;AACpC,IAAI,qBAAqB,GAAG,KAAK,CAAC;AAElC,SAAS,mBAAmB;IAC1B,IAAI,qBAAqB;QAAE,OAAO;IAClC,qBAAqB,GAAG,IAAI,CAAC;IAC7B,OAAO,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,EAAE;QACxB,KAAK,MAAM,MAAM,IAAI,cAAc,EAAE,CAAC;YACpC,IAAI,CAAC;gBACH,MAAM,CAAC,OAAO,EAAE,CAAC;YACnB,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,2HAA2H;gBAC3H,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;YACrB,CAAC;QACH,CAAC;IACH,CAAC,CAAC,CAAC;AACL,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,IAAY;IACzC,mBAAmB,EAAE,CAAC;IACtB,MAAM,MAAM,GAAG,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC;IAChC,cAAc,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC5B,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,MAAM,UAAU,UAAU,CAAC,YAAoB,EAAE,OAAqB;IACpE,MAAM,OAAO,GAAG,IAAI,GAAG,CAAC,aAAa,YAAY,EAAE,EAAE,OAAO,CAAC,CAAC;IAC9D,OAAO,YAAY,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC;AACvC,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"shutdown.d.mts","sourceRoot":"","sources":["../shutdown.mts"],"names":[],"mappings":"AAMA,eAAO,MAAM,kBAAkB,QAAa,OAAO,CAAC,IAAI,CAkBvD,CAAC;AAEF,eAAO,MAAM,kBAAkB,QApBa,OAAO,CAAC,IAAI,CAoBJ,CAAC"}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
import { urlsToClients, closeDynamicConfigValkeySubscriptionClient } from "./clients.mjs";
|
|
2
|
+
import { dynamicConfigs } from "./dynamic-config.mjs";
|
|
3
|
+
import { handleValkeyError } from "./errors.mjs";
|
|
4
|
+
let shutdown = false;
|
|
5
|
+
export const onGracefulShutdown = async () => {
|
|
6
|
+
if (shutdown)
|
|
7
|
+
return;
|
|
8
|
+
shutdown = true;
|
|
9
|
+
const configResults = await Promise.allSettled([...dynamicConfigs].map((config) => Promise.resolve().then(() => config.close())));
|
|
10
|
+
for (const result of configResults) {
|
|
11
|
+
if (result.status === "rejected")
|
|
12
|
+
handleValkeyError(toError(result.reason));
|
|
13
|
+
}
|
|
14
|
+
const results = await Promise.allSettled([
|
|
15
|
+
...[...urlsToClients.values()].map((client) => Promise.resolve().then(() => client.close())),
|
|
16
|
+
closeDynamicConfigValkeySubscriptionClient(),
|
|
17
|
+
]);
|
|
18
|
+
for (const result of results) {
|
|
19
|
+
if (result.status === "rejected")
|
|
20
|
+
handleValkeyError(toError(result.reason));
|
|
21
|
+
}
|
|
22
|
+
};
|
|
23
|
+
export const closeValkeyClients = onGracefulShutdown;
|
|
24
|
+
function toError(error) {
|
|
25
|
+
return error instanceof Error ? error : new Error(String(error));
|
|
26
|
+
}
|
|
27
|
+
//# sourceMappingURL=shutdown.mjs.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"shutdown.mjs","sourceRoot":"","sources":["../shutdown.mts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,0CAA0C,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAAE,iBAAiB,EAAE,MAAM,cAAc,CAAC;AAEjD,IAAI,QAAQ,GAAG,KAAK,CAAC;AAErB,MAAM,CAAC,MAAM,kBAAkB,GAAG,KAAK,IAAmB,EAAE;IAC1D,IAAI,QAAQ;QAAE,OAAO;IACrB,QAAQ,GAAG,IAAI,CAAC;IAEhB,MAAM,aAAa,GAAG,MAAM,OAAO,CAAC,UAAU,CAC5C,CAAC,GAAG,cAAc,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAClF,CAAC;IACF,KAAK,MAAM,MAAM,IAAI,aAAa,EAAE,CAAC;QACnC,IAAI,MAAM,CAAC,MAAM,KAAK,UAAU;YAAE,iBAAiB,CAAC,OAAO,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;IAC9E,CAAC;IAED,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,UAAU,CAAC;QACvC,GAAG,CAAC,GAAG,aAAa,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;QAC5F,0CAA0C,EAAE;KAC7C,CAAC,CAAC;IACH,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAC7B,IAAI,MAAM,CAAC,MAAM,KAAK,UAAU;YAAE,iBAAiB,CAAC,OAAO,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;IAC9E,CAAC;AACH,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,kBAAkB,GAAG,kBAAkB,CAAC;AAErD,SAAS,OAAO,CAAC,KAAc;IAC7B,OAAO,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC;AACnE,CAAC"}
|
package/dist/types.d.mts
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
import type { GlideClient } from "@valkey/valkey-glide";
|
|
2
|
+
export type ValkeyClientsOption = {
|
|
3
|
+
cacheClient?: GlideClient;
|
|
4
|
+
rateLimiterClient?: GlideClient;
|
|
5
|
+
dynamicConfigClient?: GlideClient;
|
|
6
|
+
};
|
|
7
|
+
export type RateLimiterOptions = {
|
|
8
|
+
/** The prefix to use for the rate limiter */
|
|
9
|
+
prefix: string;
|
|
10
|
+
/** The TTL in seconds for the rate limiter */
|
|
11
|
+
ttlSeconds: number;
|
|
12
|
+
/** Optional Valkey client. Defaults to the package rate limiter client. */
|
|
13
|
+
client?: GlideClient;
|
|
14
|
+
};
|
|
15
|
+
export type ValkeyCacheOptions<K = string> = {
|
|
16
|
+
/** The prefix to use for the cache */
|
|
17
|
+
prefix: string;
|
|
18
|
+
/** The TTL in seconds for non-null values */
|
|
19
|
+
ttlSeconds: number;
|
|
20
|
+
/** The TTL in seconds for null values, defaults to ttlSeconds / 60 */
|
|
21
|
+
nullTtlSeconds?: number;
|
|
22
|
+
/** The mode to use: 'json' (default) for JSON serialization, 'text' for raw text storage, 'buffer' for Buffer storage */
|
|
23
|
+
mode?: "json" | "text" | "buffer";
|
|
24
|
+
/** Fraction of ttlSeconds representing the age threshold; values are stale when ttlSecondsRemaining < (1 - staleTtlAge) * ttlSeconds (0-1, defaults to 0.9) */
|
|
25
|
+
staleTtlAge?: number;
|
|
26
|
+
/** Optional bloom filter for negative lookups — skip DB calls for entities that definitely don't exist */
|
|
27
|
+
bloomFilter?: import("./bloom-filter.mts").ValkeyBloomFilter;
|
|
28
|
+
/** Optional runtime feature flag — if provided, called per-request; bloom filter is bypassed when it returns false */
|
|
29
|
+
bloomFilterEnabled?: () => boolean;
|
|
30
|
+
/** Optional Valkey client. Defaults to the package cache client. */
|
|
31
|
+
client?: GlideClient;
|
|
32
|
+
} & (K extends string ? {
|
|
33
|
+
/**
|
|
34
|
+
* Converts a key to its string representation for cache storage.
|
|
35
|
+
* The returned string is automatically lowercased and trimmed before use.
|
|
36
|
+
* Defaults to `String(key)` for string keys.
|
|
37
|
+
*/
|
|
38
|
+
keySerializer?: (key: K) => string;
|
|
39
|
+
} : {
|
|
40
|
+
/**
|
|
41
|
+
* Converts a key to its string representation for cache storage.
|
|
42
|
+
* The returned string is automatically lowercased and trimmed before use.
|
|
43
|
+
*/
|
|
44
|
+
keySerializer: (key: K) => string;
|
|
45
|
+
});
|
|
46
|
+
export type ValkeyCacheMode = "json" | "text" | "buffer";
|
|
47
|
+
export type ValkeyCacheResponse = Promise<string | Buffer | Record<string, unknown> | null>;
|
|
48
|
+
export type DynamicConfigField = string | number | boolean;
|
|
49
|
+
export type DynamicConfigFieldType = "string" | "number" | "boolean";
|
|
50
|
+
export type DynamicConfigOptions = {
|
|
51
|
+
/** The TTL in seconds until the values become stale and values are re-fetched */
|
|
52
|
+
staleTtlSeconds?: number | null;
|
|
53
|
+
/** The key of the hash to use for the distributed configuration */
|
|
54
|
+
key: string;
|
|
55
|
+
/** The fields and types of the hash to use for the distributed configuration */
|
|
56
|
+
fieldTypes: Record<string, DynamicConfigFieldType>;
|
|
57
|
+
/** The default field values if not set */
|
|
58
|
+
defaultFields: Record<string, DynamicConfigField>;
|
|
59
|
+
/** Optional Valkey client. Defaults to the package dynamic config client. */
|
|
60
|
+
client?: GlideClient;
|
|
61
|
+
};
|
|
62
|
+
export type ValkeyBloomFilterOptions = {
|
|
63
|
+
/** The name of the bloom filter (used for key prefixing) */
|
|
64
|
+
name: string;
|
|
65
|
+
/** The expected capacity (number of items) */
|
|
66
|
+
capacity: number;
|
|
67
|
+
/** The target error rate (0-1, e.g., 0.01 for 1%) */
|
|
68
|
+
errorRate: number;
|
|
69
|
+
/** The batch size for bulk operations (optional, defaults to 10,000; Lua paths clamp to 5,000) */
|
|
70
|
+
batchSize?: number;
|
|
71
|
+
/** The expansion rate for auto-growth (optional, defaults to 2) */
|
|
72
|
+
expansionRate?: number;
|
|
73
|
+
/** Optional Valkey client. Defaults to the package cache client. */
|
|
74
|
+
client?: GlideClient;
|
|
75
|
+
};
|
|
76
|
+
//# sourceMappingURL=types.d.mts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types.d.mts","sourceRoot":"","sources":["../types.mts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,sBAAsB,CAAC;AAExD,MAAM,MAAM,mBAAmB,GAAG;IAChC,WAAW,CAAC,EAAE,WAAW,CAAC;IAC1B,iBAAiB,CAAC,EAAE,WAAW,CAAC;IAChC,mBAAmB,CAAC,EAAE,WAAW,CAAC;CACnC,CAAC;AAEF,MAAM,MAAM,kBAAkB,GAAG;IAC/B,6CAA6C;IAC7C,MAAM,EAAE,MAAM,CAAC;IACf,8CAA8C;IAC9C,UAAU,EAAE,MAAM,CAAC;IACnB,2EAA2E;IAC3E,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB,CAAC;AAEF,MAAM,MAAM,kBAAkB,CAAC,CAAC,GAAG,MAAM,IAAI;IAC3C,sCAAsC;IACtC,MAAM,EAAE,MAAM,CAAC;IACf,6CAA6C;IAC7C,UAAU,EAAE,MAAM,CAAC;IACnB,sEAAsE;IACtE,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,yHAAyH;IACzH,IAAI,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,QAAQ,CAAC;IAClC,+JAA+J;IAC/J,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,0GAA0G;IAC1G,WAAW,CAAC,EAAE,OAAO,oBAAoB,EAAE,iBAAiB,CAAC;IAC7D,sHAAsH;IACtH,kBAAkB,CAAC,EAAE,MAAM,OAAO,CAAC;IACnC,oEAAoE;IACpE,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB,GAAG,CAAC,CAAC,SAAS,MAAM,GACjB;IACE;;;;OAIG;IACH,aAAa,CAAC,EAAE,CAAC,GAAG,EAAE,CAAC,KAAK,MAAM,CAAC;CACpC,GACD;IACE;;;OAGG;IACH,aAAa,EAAE,CAAC,GAAG,EAAE,CAAC,KAAK,MAAM,CAAC;CACnC,CAAC,CAAC;AAEP,MAAM,MAAM,eAAe,GAAG,MAAM,GAAG,MAAM,GAAG,QAAQ,CAAC;AACzD,MAAM,MAAM,mBAAmB,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,IAAI,CAAC,CAAC;AAE5F,MAAM,MAAM,kBAAkB,GAAG,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC;AAC3D,MAAM,MAAM,sBAAsB,GAAG,QAAQ,GAAG,QAAQ,GAAG,SAAS,CAAC;AACrE,MAAM,MAAM,oBAAoB,GAAG;IACjC,iFAAiF;IACjF,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,mEAAmE;IACnE,GAAG,EAAE,MAAM,CAAC;IACZ,gFAAgF;IAChF,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;IACnD,0CAA0C;IAC1C,aAAa,EAAE,MAAM,CAAC,MAAM,EAAE,kBAAkB,CAAC,CAAC;IAClD,6EAA6E;IAC7E,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB,CAAC;AAEF,MAAM,MAAM,wBAAwB,GAAG;IACrC,4DAA4D;IAC5D,IAAI,EAAE,MAAM,CAAC;IACb,8CAA8C;IAC9C,QAAQ,EAAE,MAAM,CAAC;IACjB,qDAAqD;IACrD,SAAS,EAAE,MAAM,CAAC;IAClB,kGAAkG;IAClG,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,mEAAmE;IACnE,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,oEAAoE;IACpE,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB,CAAC"}
|
package/dist/types.mjs
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types.mjs","sourceRoot":"","sources":["../types.mts"],"names":[],"mappings":""}
|
package/dist/utils.d.mts
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Utility functions for Valkey operations
|
|
3
|
+
*/
|
|
4
|
+
/**
|
|
5
|
+
* Normalizes TTL result from PTTL command
|
|
6
|
+
* PTTL returns milliseconds: -2 if key doesn't exist, -1 if no expiry, or milliseconds remaining
|
|
7
|
+
* Converts to seconds for internal use, preserving sentinel values
|
|
8
|
+
* valkey-glide returns number or bigint for TTL results
|
|
9
|
+
*/
|
|
10
|
+
export declare function normalizeTtlResult(value: unknown): number | null;
|
|
11
|
+
/**
|
|
12
|
+
* Normalizes count result from Valkey operations (e.g., ZCOUNT)
|
|
13
|
+
* valkey-glide returns number or bigint for count results
|
|
14
|
+
*/
|
|
15
|
+
export declare function normalizeCountResult(result: unknown): number;
|
|
16
|
+
//# sourceMappingURL=utils.d.mts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"utils.d.mts","sourceRoot":"","sources":["../utils.mts"],"names":[],"mappings":"AAAA;;GAEG;AAEH;;;;;GAKG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,OAAO,GAAG,MAAM,GAAG,IAAI,CAWhE;AAED;;;GAGG;AACH,wBAAgB,oBAAoB,CAAC,MAAM,EAAE,OAAO,GAAG,MAAM,CAI5D"}
|
package/dist/utils.mjs
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Utility functions for Valkey operations
|
|
3
|
+
*/
|
|
4
|
+
/**
|
|
5
|
+
* Normalizes TTL result from PTTL command
|
|
6
|
+
* PTTL returns milliseconds: -2 if key doesn't exist, -1 if no expiry, or milliseconds remaining
|
|
7
|
+
* Converts to seconds for internal use, preserving sentinel values
|
|
8
|
+
* valkey-glide returns number or bigint for TTL results
|
|
9
|
+
*/
|
|
10
|
+
export function normalizeTtlResult(value) {
|
|
11
|
+
if (typeof value === "number") {
|
|
12
|
+
if (value === -2 || value === -1)
|
|
13
|
+
return value; // Preserve sentinel values
|
|
14
|
+
return Math.floor(value / 1000); // Convert milliseconds to seconds
|
|
15
|
+
}
|
|
16
|
+
if (typeof value === "bigint") {
|
|
17
|
+
const num = Number(value);
|
|
18
|
+
if (num === -2 || num === -1)
|
|
19
|
+
return num;
|
|
20
|
+
return Math.floor(num / 1000);
|
|
21
|
+
}
|
|
22
|
+
return null;
|
|
23
|
+
}
|
|
24
|
+
/**
|
|
25
|
+
* Normalizes count result from Valkey operations (e.g., ZCOUNT)
|
|
26
|
+
* valkey-glide returns number or bigint for count results
|
|
27
|
+
*/
|
|
28
|
+
export function normalizeCountResult(result) {
|
|
29
|
+
if (typeof result === "number")
|
|
30
|
+
return result;
|
|
31
|
+
if (typeof result === "bigint")
|
|
32
|
+
return Number(result);
|
|
33
|
+
return 0;
|
|
34
|
+
}
|
|
35
|
+
//# sourceMappingURL=utils.mjs.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"utils.mjs","sourceRoot":"","sources":["../utils.mts"],"names":[],"mappings":"AAAA;;GAEG;AAEH;;;;;GAKG;AACH,MAAM,UAAU,kBAAkB,CAAC,KAAc;IAC/C,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;QAC9B,IAAI,KAAK,KAAK,CAAC,CAAC,IAAI,KAAK,KAAK,CAAC,CAAC;YAAE,OAAO,KAAK,CAAC,CAAC,2BAA2B;QAC3E,OAAO,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,IAAI,CAAC,CAAC,CAAC,kCAAkC;IACrE,CAAC;IACD,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;QAC9B,MAAM,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;QAC1B,IAAI,GAAG,KAAK,CAAC,CAAC,IAAI,GAAG,KAAK,CAAC,CAAC;YAAE,OAAO,GAAG,CAAC;QACzC,OAAO,IAAI,CAAC,KAAK,CAAC,GAAG,GAAG,IAAI,CAAC,CAAC;IAChC,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,oBAAoB,CAAC,MAAe;IAClD,IAAI,OAAO,MAAM,KAAK,QAAQ;QAAE,OAAO,MAAM,CAAC;IAC9C,IAAI,OAAO,MAAM,KAAK,QAAQ;QAAE,OAAO,MAAM,CAAC,MAAM,CAAC,CAAC;IACtD,OAAO,CAAC,CAAC;AACX,CAAC"}
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Bloom Filters
|
|
2
|
+
|
|
3
|
+
`ValkeyBloomFilter` wraps `BF.RESERVE`, `BF.MADD`, `BF.EXISTS`, and `BF.MEXISTS` with ready-key and zero-downtime rebuild helpers.
|
|
4
|
+
|
|
5
|
+
Important behavior:
|
|
6
|
+
|
|
7
|
+
- `exists()` and `mexists()` return `null` when the filter key is missing.
|
|
8
|
+
- `existsIfReady()` and `mexistsIfReady()` also require an external ready marker.
|
|
9
|
+
- `add()` writes to the live key and to the building key when a rebuild is active.
|
|
10
|
+
- `rebuildFromStream()` builds under `bloom-filter:{name}:building` and renames it over the live key.
|
|
11
|
+
|
|
12
|
+
Large Lua-backed calls are clamped to `LUA_UNPACK_BATCH_SIZE` to avoid Valkey Lua argument limits.
|
package/docs/cache.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Cache
|
|
2
|
+
|
|
3
|
+
`ValkeyCache` provides read-through helpers around Valkey keys named:
|
|
4
|
+
|
|
5
|
+
```txt
|
|
6
|
+
cache:{prefix}:{serializedKey}
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Supported modes:
|
|
10
|
+
|
|
11
|
+
- `json` stores JSON values and is the default.
|
|
12
|
+
- `text` stores strings only.
|
|
13
|
+
- `buffer` stores `Buffer` values only.
|
|
14
|
+
|
|
15
|
+
Values larger than 2 KiB are gzip-compressed. Null results are cached with `nullTtlSeconds`, defaulting to `ttlSeconds / 60` with a minimum of 1 second.
|
|
16
|
+
|
|
17
|
+
`cacheGetByAny()` and `cacheGetByAnyBatch()` protect against stale miss writes by setting short-lived invalidation markers during delete operations.
|
|
18
|
+
|
|
19
|
+
Optional Bloom filter integration can skip authoritative-store calls for definite misses. If the Bloom filter is unavailable or not ready, cache reads fall back to the normal miss path.
|
package/docs/clients.md
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Clients
|
|
2
|
+
|
|
3
|
+
The package exports default clients for simple applications:
|
|
4
|
+
|
|
5
|
+
- `cacheValkeyClient`
|
|
6
|
+
- `rateLimiterValkeyClient`
|
|
7
|
+
- `dynamicConfigValkeyClient`
|
|
8
|
+
|
|
9
|
+
Connections are deduplicated by URL, read preference, and lazy-connect setting through `upsertValkeyClientByUrl()`.
|
|
10
|
+
|
|
11
|
+
For libraries, tests, and multi-tenant services, pass a `GlideClient` directly to `ValkeyCache`, `ValkeyBloomFilter`, `DynamicConfig`, or `RateLimiter`.
|
|
12
|
+
|
|
13
|
+
Use `closeValkeyClients()` or `onGracefulShutdown()` to close package-managed clients and the dynamic-config pub/sub connection.
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Configuration
|
|
2
|
+
|
|
3
|
+
`VALKEY_URL` is the default URL for all package-managed clients.
|
|
4
|
+
|
|
5
|
+
Specialized URLs override it:
|
|
6
|
+
|
|
7
|
+
- `VALKEY_CACHE_URL`
|
|
8
|
+
- `VALKEY_RATE_LIMITER_URL`
|
|
9
|
+
- `VALKEY_DYNAMIC_CONFIG_URL`
|
|
10
|
+
|
|
11
|
+
Redis-style URLs are supported:
|
|
12
|
+
|
|
13
|
+
- `redis://localhost:6379`
|
|
14
|
+
- `rediss://user:password@example.com:6380`
|
|
15
|
+
|
|
16
|
+
The cache client uses `readFrom: "preferReplica"`. Rate limiter reads use `primary` to avoid replica lag. Dynamic config commands use `preferReplica`; pub/sub uses a dedicated eager connection.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Dynamic Config
|
|
2
|
+
|
|
3
|
+
`DynamicConfig` stores runtime fields in a Valkey hash under:
|
|
4
|
+
|
|
5
|
+
```txt
|
|
6
|
+
dynamic-config:{key}
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Fields are declared with `fieldTypes` and initialized from `defaultFields`. Missing defaults are written during initialization.
|
|
10
|
+
|
|
11
|
+
Use `setField()` or `setFields()` to update values. Updates are written with a Lua script that performs `HSET` and `PUBLISH` in one roundtrip.
|
|
12
|
+
|
|
13
|
+
Each instance subscribes to `dynamic-config:*` and applies matching field updates from pub/sub. Call `close()` in tests and shutdown paths to remove handlers and clear refresh timers.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Events And Metrics
|
|
2
|
+
|
|
3
|
+
Subscribe to `valkeyEvents` for observability.
|
|
4
|
+
|
|
5
|
+
Events include cache hits/misses, cache writes/deletes, Bloom filter checks/writes, rate limiter operations, and `cache:call` metric payloads.
|
|
6
|
+
|
|
7
|
+
Errors from background work are sent to the package error handler. Configure it with:
|
|
8
|
+
|
|
9
|
+
```ts
|
|
10
|
+
import { setValkeyErrorHandler } from "valkyries";
|
|
11
|
+
|
|
12
|
+
setValkeyErrorHandler((error) => {
|
|
13
|
+
logger.error(error);
|
|
14
|
+
});
|
|
15
|
+
```
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
# Lua Scripts
|
|
2
|
+
|
|
3
|
+
Lua scripts live in `scripts/` and are copied to `dist/scripts/` during build.
|
|
4
|
+
|
|
5
|
+
Use `loadScript()` and `registerScript()` for package scripts. Registered scripts are released on process exit.
|
|
6
|
+
|
|
7
|
+
Scripts are used when operations must be atomic or when combining multiple Valkey commands into one roundtrip.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Migration Guide
|
|
2
|
+
|
|
3
|
+
Use this guide when moving existing Valkey-backed cache, Bloom filter, dynamic config, or rate limiter code to `valkyries`.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
pnpm add valkyries @valkey/valkey-glide
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
`valkyries` is ESM-only and requires Node.js 24 or newer.
|
|
12
|
+
|
|
13
|
+
## Update Imports
|
|
14
|
+
|
|
15
|
+
Import public classes and helpers from `valkyries` instead of local application modules:
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
import { ValkeyCache, ValkeyBloomFilter, DynamicConfig, RateLimiter } from "valkyries";
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Subpath imports are also available:
|
|
22
|
+
|
|
23
|
+
```ts
|
|
24
|
+
import { ValkeyCache } from "valkyries/cache";
|
|
25
|
+
import { ValkeyBloomFilter } from "valkyries/bloom-filter";
|
|
26
|
+
import { DynamicConfig } from "valkyries/dynamic-config";
|
|
27
|
+
import { RateLimiter } from "valkyries/rate-limiter";
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Configure Connections
|
|
31
|
+
|
|
32
|
+
The default connection URL is `VALKEY_URL`.
|
|
33
|
+
|
|
34
|
+
Use specialized URLs when you want separate pools or clusters:
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
VALKEY_CACHE_URL=redis://localhost:6379
|
|
38
|
+
VALKEY_RATE_LIMITER_URL=redis://localhost:6379
|
|
39
|
+
VALKEY_DYNAMIC_CONFIG_URL=redis://localhost:6379
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Every primary class also accepts a `client` option for explicit `GlideClient` ownership.
|
|
43
|
+
|
|
44
|
+
## Review Removed Responsibilities
|
|
45
|
+
|
|
46
|
+
`valkyries` contains only Valkey data-store primitives:
|
|
47
|
+
|
|
48
|
+
- Cache helpers
|
|
49
|
+
- Bloom filters
|
|
50
|
+
- Dynamic configuration
|
|
51
|
+
- Rate limiting
|
|
52
|
+
- Shared Valkey clients, events, scripts, and shutdown helpers
|
|
53
|
+
|
|
54
|
+
Queueing, job orchestration, service-specific metrics backends, and application-specific error reporting should stay in the consuming application.
|
|
55
|
+
|
|
56
|
+
## Replace Application-Specific Hooks
|
|
57
|
+
|
|
58
|
+
The package exposes generic hooks for behavior that applications usually customize:
|
|
59
|
+
|
|
60
|
+
- `setValkeyErrorHandler(handler)` for error reporting
|
|
61
|
+
- `valkeyEvents` / `emitValkeyEvent()` for event and metric collection
|
|
62
|
+
- Per-class `client` options for custom connection lifecycle management
|
|
63
|
+
|
|
64
|
+
## Validate
|
|
65
|
+
|
|
66
|
+
Run the package checks locally with a Valkey Bundle instance:
|
|
67
|
+
|
|
68
|
+
```sh
|
|
69
|
+
docker run --rm -p 6379:6379 valkey/valkey-bundle:latest
|
|
70
|
+
pnpm run typecheck
|
|
71
|
+
pnpm run lint
|
|
72
|
+
pnpm run build
|
|
73
|
+
pnpm run test:coverage
|
|
74
|
+
```
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Rate Limiter
|
|
2
|
+
|
|
3
|
+
`RateLimiter` uses sorted sets with server time to implement sliding windows.
|
|
4
|
+
|
|
5
|
+
Key format:
|
|
6
|
+
|
|
7
|
+
```txt
|
|
8
|
+
rate-limiter:{prefix}:{id}
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Use `addAndCheck(ids, threshold)` for request paths. It atomically adds an event, removes expired events, counts the current window, and returns:
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
{ counts: number[]; limited: boolean }
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
A threshold of `N` blocks the Nth request because the count is checked after incrementing.
|