@ultimat3/cache 29.4.0 → 30.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +16 -1
- package/README.md +30 -2
- package/package.json +12 -2
- package/src/fence.ts +8 -0
- package/src/purge-http.ts +30 -17
- package/src/redis-deadline.ts +126 -0
- package/src/redis.ts +42 -24
- package/src/tier-leading.ts +64 -0
- package/src/tiers.ts +17 -1
package/CLAUDE.md
CHANGED
|
@@ -89,6 +89,12 @@ Tier 1. Tagged caching + THE invalidation graph.
|
|
|
89
89
|
`requestTimeoutMs` default, written as a literal because http is tier 2. `loadDeadlineMs`
|
|
90
90
|
overrides it and `schedule` injects the timer. Eviction frees the KEY only; the load runs on. Not
|
|
91
91
|
an `app.config.ts` key on purpose.
|
|
92
|
+
- **A read refuses to join a load a bust has overtaken** (`tier-leading.ts`): a reader's unseen tags are
|
|
93
|
+
covered, then an invalid fence evicts the key and the read leads. A joiner from before the bust keeps
|
|
94
|
+
its load. An entry lives exactly while the flight holds the key (`released`). A joiner asks the fence
|
|
95
|
+
only when the generation moved or it brought a new tag: ~1,600 arrivals must not each walk the marks.
|
|
96
|
+
- **A joiner survives its leader's abort** (`singleFlight`'s `signal`, `tryUseContext()?.signal`): it
|
|
97
|
+
re-leads under its own context. A failure with the leader's signal live is still shared.
|
|
92
98
|
- **A joiner shares the leader's WRITE, so it contributes to it** (`FlightJoin`, `mergeSetOptions`
|
|
93
99
|
in `set-options.ts`): tags union, TTLs take the shortest. `work` reads the merge after the load
|
|
94
100
|
AND once more after the fill (`tagsAddedSince`), re-filling every tier.
|
|
@@ -122,6 +128,13 @@ Tier 1. Tagged caching + THE invalidation graph.
|
|
|
122
128
|
Every claim about what a script DOES belongs in `redis.live.test.ts`.
|
|
123
129
|
- **The Redis namespace carries the build id by default** (`namespaceFor`, `appVersion()`);
|
|
124
130
|
`buildId: null` opts out. Changing the layout is a cold cache.
|
|
131
|
+
- **Redis down is a slow miss, never a hang** (`redis-deadline.ts`). Every value-path command gets
|
|
132
|
+
`commandTimeoutMs` (150 ms); one miss parks the value path `REDIS_PARK_MS` (5 s) — commands refuse
|
|
133
|
+
unsent, one clock read, no state machine. A bust NEVER parks and gets 1,000 ms: a skipped bust is
|
|
134
|
+
stale rows until TTL. A miss or a park is `X_CACHE_DRIVER_UNAVAILABLE`, which `bestEffort` records;
|
|
135
|
+
so is a STOPPED store (Bun's `ERR_REDIS_CONNECTION_CLOSED`, typed in `withinDeadline`, a bust's
|
|
136
|
+
included). A server's own error reply passes through (`redis-outage.test.ts`).
|
|
137
|
+
The client is core's `redisClient()` (no offline queue, endless reconnect); never `Bun.redis`.
|
|
125
138
|
- **Cross-instance invalidation is a SEAM, never a transport.** `registerInvalidationBroadcast` /
|
|
126
139
|
`receiveInvalidationBroadcast`; `cli` owns the bus. The inbound half cannot re-emit (only it passes
|
|
127
140
|
`emit: false`). An undeclared inbound tag goes to `report.errors`, never a throw.
|
|
@@ -161,6 +174,7 @@ Tier 1. Tagged caching + THE invalidation graph.
|
|
|
161
174
|
| `graph.ts` | tag → dependents (cache keys, ISR routes, CDN paths, live queries) |
|
|
162
175
|
| `tiers.ts` | `CacheTier`, `TierLabel`, `assertTtl`, read-through stack |
|
|
163
176
|
| `fence.ts` | the invalidation fence a fill (here or in `query`) checks before it publishes |
|
|
177
|
+
| `tier-leading.ts` | which fence each in-flight load holds, and whether a reader may join it |
|
|
164
178
|
| `tier-fence.ts` | `TierFence`/`FenceVerdict`, and how a fill samples and asks a shared tier's fence |
|
|
165
179
|
| `redis-fence.ts` | the leased generation keys a bust writes and a fill watches |
|
|
166
180
|
| `set-options.ts` | how two callers' `CacheSetOptions` combine, and the `null`-load TTL |
|
|
@@ -168,7 +182,8 @@ Tier 1. Tagged caching + THE invalidation graph.
|
|
|
168
182
|
| `memo.ts` | request memo over the ALS ctx (WeakMap, no lifecycle) |
|
|
169
183
|
| `lru.ts` | byte-budgeted LRU (linked list + map + tag index) |
|
|
170
184
|
| `value-codec.ts` | the one value encoding every tier stores and decodes |
|
|
171
|
-
| `redis.ts` | `Bun.redis`
|
|
185
|
+
| `redis.ts` | the shared tier over core's `redisClient()` (`@ultimat3/core/redis`, never `Bun.redis`), build-namespaced keys, hash-tagged buckets, one script call per tag |
|
|
186
|
+
| `redis-deadline.ts` | `withinDeadline`: a per-command deadline (150 ms, `commandTimeoutMs`), a 5 s park of the value path after one is missed, and the bust view that skips the park on its own 1,000 ms |
|
|
172
187
|
| `cdn.ts` | `Cache-Control`/`Surrogate-Key` emission, `surrogateKeys`, the purge key list, the `PurgeDriver` seam |
|
|
173
188
|
| `purge-http.ts` | the HTTP half both remote drivers share: one POST, batching, key guard, and core's retryable table re-exported |
|
|
174
189
|
| `purge-fastly.ts` | `fastlyPurgeDriver`: surrogate-key batch purge, `purge_all` |
|
package/README.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
Four tiers. **One invalidation graph, not three.**
|
|
4
4
|
|
|
5
|
+
`bun add @ultimat3/cache` · guide: [Caching And Invalidation](https://github.com/developerz-ai/ultimate/wiki/Caching-And-Invalidation)
|
|
6
|
+
|
|
5
7
|
```ts
|
|
6
8
|
cache: { invalidates: [tag.post, tag.feed] }
|
|
7
9
|
```
|
|
@@ -27,7 +29,7 @@ Reads walk down until a hit, then populate every tier they walked past. Writes p
|
|
|
27
29
|
|---|---|---|---|---|
|
|
28
30
|
| 0 | `request-memo` | ALS context (`WeakMap`) | dies with the request | never |
|
|
29
31
|
| 1 | `lru` | in-process, byte-budgeted | tag index | never |
|
|
30
|
-
| 2 | `redis` | `
|
|
32
|
+
| 2 | `redis` | `redisClient()` (`@ultimat3/core/redis`) | tag→keys set, one `EVAL` **per tag** + slot-local `DEL`s | single node |
|
|
31
33
|
| 3 | `cdn` | headers + purge driver | surrogate keys | no CDN |
|
|
32
34
|
|
|
33
35
|
A tier is a `CacheTier` (`get`/`set`/`del`/`invalidateTags`, plus an optional `fence` for a tier
|
|
@@ -266,6 +268,32 @@ what `x cache bust --json` will print once it ships; that command is planned and
|
|
|
266
268
|
A dead tier lands in `errors` and never throws — a Redis outage must not fail the write
|
|
267
269
|
that triggered the bust. Entries there expire by TTL instead.
|
|
268
270
|
|
|
271
|
+
### Redis down is a slow miss, not a hang
|
|
272
|
+
|
|
273
|
+
A refused tier was always a miss; a tier that never ANSWERS was unbounded — measured on Redis 7,
|
|
274
|
+
a paused server hung `stack.read` 45 s and a bust 20 s, on an action's commit path. `As of 2026-11`:
|
|
275
|
+
|
|
276
|
+
| Redis is | A read | A bust |
|
|
277
|
+
|---|---|---|
|
|
278
|
+
| up | answers, single milliseconds | answers |
|
|
279
|
+
| paused (accepts, never answers) | one command misses its `commandTimeoutMs` (default 150 ms) → a miss, the source answers; the value path then **parks** for 5 s and every read there is an immediate miss, nothing sent | its own 1,000 ms deadline, **never parked**: a bust skipped for a stall leaves stale rows until TTL. Past it → `report.errors` |
|
|
280
|
+
| stopped | refused at once (no offline queue) → a miss | refused at once → `report.errors`; retry with the same tags |
|
|
281
|
+
| back | the client reconnects on its own, every pod, no restart | answers |
|
|
282
|
+
|
|
283
|
+
A missed deadline or a parked command is `X_CACHE_DRIVER_UNAVAILABLE` in `recentTierFailures()`,
|
|
284
|
+
the cause naming the deadline or the park left; a stopped server's refusal is recorded with Bun's
|
|
285
|
+
own `ERR_REDIS_CONNECTION_CLOSED` message and no `code`. The client is core's `redisClient()` (`@ultimat3/core/redis`,
|
|
286
|
+
`REDIS_URL`), never `Bun.redis`: its default offline queue made the first read against a stopped
|
|
287
|
+
server wait 31 s, and it stopped reconnecting after that. A `client` handed in keeps its own
|
|
288
|
+
reconnect behaviour and still gets the deadline. Raise the budget only when that latency is the
|
|
289
|
+
store's normal:
|
|
290
|
+
|
|
291
|
+
```ts
|
|
292
|
+
import { redisTier } from '@ultimat3/cache';
|
|
293
|
+
|
|
294
|
+
redisTier({ commandTimeoutMs: 400 }); // anything but whole ms above 0 is X_CACHE_LIMIT_INVALID
|
|
295
|
+
```
|
|
296
|
+
|
|
269
297
|
The fan-out walks the ladder **farthest tier first** — Redis before the LRU before the request memo
|
|
270
298
|
— and reports in read order. Clearing near-to-far leaves the far tier holding the old value after
|
|
271
299
|
the near ones are clear, and a read racing the bust promotes it straight back up into them: every
|
|
@@ -441,7 +469,7 @@ reached the floor unchecked would have passed it, at any floor — `0` included.
|
|
|
441
469
|
|
|
442
470
|
| Code | Cause |
|
|
443
471
|
|---|---|
|
|
444
|
-
| `X_CACHE_DRIVER_UNAVAILABLE` |
|
|
472
|
+
| `X_CACHE_DRIVER_UNAVAILABLE` | a Redis command missed its deadline, or the tier is parked after one; a purge driver built without its token; or a batch size that is not a positive integer |
|
|
445
473
|
| `X_CACHE_JITTER_INVALID` | a tier's `jitterFraction` outside `[0, 1)` |
|
|
446
474
|
| `X_CACHE_PURGE_FAILED` | the CDN refused a purge, or a key it would split on whitespace |
|
|
447
475
|
| `X_CACHE_TAG_UNKNOWN` | a tag no entity declared — usually a typo |
|
package/package.json
CHANGED
|
@@ -1,7 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/cache",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "30.0.0",
|
|
4
4
|
"description": "Tagged caching: request memo, LRU, Redis, CDN — one invalidation graph",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"ultimate",
|
|
7
|
+
"bun",
|
|
8
|
+
"framework",
|
|
9
|
+
"cache",
|
|
10
|
+
"invalidation",
|
|
11
|
+
"redis",
|
|
12
|
+
"cdn"
|
|
13
|
+
],
|
|
14
|
+
"homepage": "https://github.com/developerz-ai/ultimate/wiki/Caching-And-Invalidation",
|
|
5
15
|
"license": "MIT",
|
|
6
16
|
"type": "module",
|
|
7
17
|
"repository": {
|
|
@@ -32,6 +42,6 @@
|
|
|
32
42
|
"test": "bun test"
|
|
33
43
|
},
|
|
34
44
|
"dependencies": {
|
|
35
|
-
"@ultimat3/core": "
|
|
45
|
+
"@ultimat3/core": "30.0.0"
|
|
36
46
|
}
|
|
37
47
|
}
|
package/src/fence.ts
CHANGED
|
@@ -70,6 +70,14 @@ export function markInvalidated(scope: FenceScope): void {
|
|
|
70
70
|
}
|
|
71
71
|
}
|
|
72
72
|
|
|
73
|
+
/**
|
|
74
|
+
* Moves with every invalidation of any kind. Unchanged means a fence that answered valid at this
|
|
75
|
+
* reading still does, so a caller asking many times need only walk the marks once per reading.
|
|
76
|
+
*/
|
|
77
|
+
export function fenceGeneration(): number {
|
|
78
|
+
return generation;
|
|
79
|
+
}
|
|
80
|
+
|
|
73
81
|
/**
|
|
74
82
|
* Everything, invalidated: every fence sampled before this call answers `false`. For a process
|
|
75
83
|
* that cannot name what it missed (`flushProcessTiers`) — the same "older than the ring remembers:
|
package/src/purge-http.ts
CHANGED
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
// judgement, and two copies of it would drift into two answers for the same 429.
|
|
5
5
|
|
|
6
6
|
import { renderFixShellArg, renderThrowable } from '@ultimat3/core';
|
|
7
|
+
import { outboundResponse } from '@ultimat3/core/outbound';
|
|
7
8
|
import { CacheDriverUnavailableError, CachePurgeFailedError } from './errors';
|
|
8
9
|
|
|
9
10
|
/** Just the call. `typeof fetch` also carries `preconnect`, which no test double should have to. */
|
|
@@ -21,10 +22,11 @@ const MAX_DETAIL_LENGTH = 200;
|
|
|
21
22
|
// Both purge drivers import it from core; it is not re-exported here (`X_HELPER_COPY`).
|
|
22
23
|
|
|
23
24
|
/**
|
|
24
|
-
*
|
|
25
|
-
*
|
|
25
|
+
* The platform's `fetch`, closed over so it is detached from any receiver ("Illegal invocation").
|
|
26
|
+
* `outbound()` always hands it the attempt's signal.
|
|
26
27
|
*/
|
|
27
|
-
export const defaultPurgeFetch: PurgeFetch = (input, init) =>
|
|
28
|
+
export const defaultPurgeFetch: PurgeFetch = (input, init) =>
|
|
29
|
+
globalThis.fetch(input, { ...init, signal: init.signal ?? null });
|
|
28
30
|
|
|
29
31
|
/**
|
|
30
32
|
* Providers cap keys per request. A bust of 300 tags is still one purge — several requests.
|
|
@@ -169,30 +171,41 @@ export interface PurgePostInput {
|
|
|
169
171
|
}
|
|
170
172
|
|
|
171
173
|
/**
|
|
172
|
-
* One POST
|
|
173
|
-
*
|
|
174
|
+
* One POST through core's `outbound()`, with the transport failure already translated. A purge is
|
|
175
|
+
* idempotent by nature — purging a key twice clears it once — so a 5xx or a throttle is retried
|
|
176
|
+
* inside the deadline (`timeoutMs`, the WHOLE call), and the final answer comes back as a
|
|
177
|
+
* `Response` for the drivers' own status table. A request that never got a status — DNS, TLS, a
|
|
178
|
+
* reset, the deadline — is retryable by definition: nothing at the edge has seen it.
|
|
174
179
|
*/
|
|
175
180
|
export async function purgePost(input: PurgePostInput): Promise<Response> {
|
|
176
181
|
try {
|
|
177
|
-
return await
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
182
|
+
return await outboundResponse(
|
|
183
|
+
{
|
|
184
|
+
url: input.url,
|
|
185
|
+
vendor: input.driver,
|
|
186
|
+
dial: 'named',
|
|
187
|
+
method: 'POST',
|
|
188
|
+
idempotent: true,
|
|
189
|
+
deadlineMs: input.timeoutMs,
|
|
190
|
+
headers: {
|
|
191
|
+
'Content-Type': 'application/json',
|
|
192
|
+
Accept: 'application/json',
|
|
193
|
+
...input.headers,
|
|
194
|
+
},
|
|
195
|
+
body: JSON.stringify(input.body),
|
|
196
|
+
},
|
|
197
|
+
{ fetch: input.fetch },
|
|
198
|
+
);
|
|
183
199
|
} catch (error) {
|
|
184
|
-
// `renderThrowable
|
|
185
|
-
//
|
|
186
|
-
// throws on a `Proxy` whose `getPrototypeOf` does, which would replace the coded refusal this
|
|
187
|
-
// catch exists to raise with a bare `TypeError` from inside it. Same rule as `invalidate.ts`.
|
|
200
|
+
// `renderThrowable`: an injected fetch rejects with anything, and `outbound()`'s own codes
|
|
201
|
+
// render a cause that names no query string.
|
|
188
202
|
const reason = renderThrowable(error);
|
|
189
203
|
throw new CachePurgeFailedError({
|
|
190
204
|
driver: input.driver,
|
|
191
205
|
detail: `${reason} — nothing left this host for ${input.url} (egress, DNS or TLS)`,
|
|
192
206
|
retryable: true,
|
|
193
207
|
// The endpoint comes from `.env.production` (`purge-env.ts`), so it is operator data in a
|
|
194
|
-
// COMMAND position: `$(…)` and a backtick substitute before `curl` is reached.
|
|
195
|
-
// already in the `detail` above, which is read rather than pasted.
|
|
208
|
+
// COMMAND position: `$(…)` and a backtick substitute before `curl` is reached.
|
|
196
209
|
fix: `curl -sS -m 5 -o /dev/null ${renderFixShellArg(input.url, '<the purge endpoint the cause names>')}`,
|
|
197
210
|
});
|
|
198
211
|
}
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
// A deadline on every command the shared tier sends, and a park after one is missed. A paused
|
|
2
|
+
// Redis accepts a command and never answers it: unbounded, `stack.read` hung 45 s and a bust 20 s
|
|
3
|
+
// on an action's commit path. Bounded, a stalled store is a slow miss, then a free one. A STOPPED
|
|
4
|
+
// store is typed here too: Bun's client rejects every command with a bare connection error.
|
|
5
|
+
|
|
6
|
+
import { type Clock, renderThrowable, stringField } from '@ultimat3/core';
|
|
7
|
+
import type { DeadlineTimer } from '@ultimat3/core/outbound';
|
|
8
|
+
import { platformDeadlineTimer } from '@ultimat3/core/outbound';
|
|
9
|
+
import { CacheDriverUnavailableError } from './errors';
|
|
10
|
+
import type { RedisLike } from './redis';
|
|
11
|
+
|
|
12
|
+
/** A value-path command's budget. A healthy shared tier answers in single milliseconds. */
|
|
13
|
+
export const DEFAULT_REDIS_COMMAND_TIMEOUT_MS = 150;
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* A bust's budget, and it is longer on purpose: a bust skipped for a stall leaves stale rows
|
|
17
|
+
* serving until TTL, where a skipped read only costs a load.
|
|
18
|
+
*/
|
|
19
|
+
export const DEFAULT_REDIS_BUST_TIMEOUT_MS = 1_000;
|
|
20
|
+
|
|
21
|
+
/**
|
|
22
|
+
* How long one missed deadline parks the value path. A read is five commands; at 150 ms each a
|
|
23
|
+
* paused server cost 757 ms per read. Parked, the next reads refuse without sending anything.
|
|
24
|
+
*/
|
|
25
|
+
export const REDIS_PARK_MS = 5_000;
|
|
26
|
+
|
|
27
|
+
export interface DeadlineOptions {
|
|
28
|
+
/** Already screened by the caller (`redisTier`'s `assertFiniteDurationMs`). */
|
|
29
|
+
readonly commandTimeoutMs: number;
|
|
30
|
+
/** `monotonic()` is the park's one clock read. */
|
|
31
|
+
readonly clock: Clock;
|
|
32
|
+
readonly timer?: DeadlineTimer;
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
/** One client, two views: `value` honours the park, `bust` never does. */
|
|
36
|
+
export interface DeadlinedRedis {
|
|
37
|
+
readonly value: RedisLike;
|
|
38
|
+
readonly bust: RedisLike;
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
const FIX =
|
|
42
|
+
'redis-cli -u "$REDIS_URL" --latency — while the store stalls every read is a miss and Postgres answers; raise redisTier({ commandTimeoutMs }) only if that latency is its normal';
|
|
43
|
+
|
|
44
|
+
/** Bun's code for a command refused for want of a connection: nothing reached the server. */
|
|
45
|
+
const CONNECTION_CLOSED = 'ERR_REDIS_CONNECTION_CLOSED';
|
|
46
|
+
|
|
47
|
+
/** A refused connection as the cache contract says it; a server's own error reply passes. */
|
|
48
|
+
const typed = (command: string, error: unknown): unknown =>
|
|
49
|
+
stringField(error, 'code') === CONNECTION_CLOSED
|
|
50
|
+
? new CacheDriverUnavailableError({
|
|
51
|
+
driver: 'redis',
|
|
52
|
+
cause: `${command} was not sent, the connection is down (${CONNECTION_CLOSED}: ${renderThrowable(error)})`,
|
|
53
|
+
fix: 'redis-cli -u "$REDIS_URL" PING — while it is down every read is a miss and Postgres answers; the client reconnects on its own once it answers PONG',
|
|
54
|
+
})
|
|
55
|
+
: error;
|
|
56
|
+
|
|
57
|
+
export function withinDeadline(client: RedisLike, options: DeadlineOptions): DeadlinedRedis {
|
|
58
|
+
const { clock, commandTimeoutMs } = options;
|
|
59
|
+
const timer = options.timer ?? platformDeadlineTimer;
|
|
60
|
+
// One number, no state machine: a missed deadline moves it forward, time moves past it.
|
|
61
|
+
let parkedUntil = Number.NEGATIVE_INFINITY;
|
|
62
|
+
|
|
63
|
+
const bounded = <T>(command: string, ms: number, run: () => Promise<T>): Promise<T> =>
|
|
64
|
+
new Promise<T>((resolve, reject) => {
|
|
65
|
+
const handle = timer.set(() => {
|
|
66
|
+
// Any missed deadline is evidence the store is stalled, a bust's included.
|
|
67
|
+
parkedUntil = clock.monotonic() + REDIS_PARK_MS;
|
|
68
|
+
reject(
|
|
69
|
+
new CacheDriverUnavailableError({
|
|
70
|
+
driver: 'redis',
|
|
71
|
+
cause: `${command} did not answer within ${String(ms)} ms; reads skip the shared tier for ${String(REDIS_PARK_MS)} ms`,
|
|
72
|
+
fix: FIX,
|
|
73
|
+
}),
|
|
74
|
+
);
|
|
75
|
+
}, ms);
|
|
76
|
+
const settle = (): void => timer.clear(handle);
|
|
77
|
+
// Sent now, not a microtask later, so a pipelined pair stays one round trip. A synchronous
|
|
78
|
+
// throw is a refusal like any other. A late answer lands after the refusal and changes
|
|
79
|
+
// nothing: a promise settles once.
|
|
80
|
+
let sent: Promise<T>;
|
|
81
|
+
try {
|
|
82
|
+
sent = run();
|
|
83
|
+
} catch (error) {
|
|
84
|
+
sent = Promise.reject(error);
|
|
85
|
+
}
|
|
86
|
+
sent.then(
|
|
87
|
+
(answer) => {
|
|
88
|
+
settle();
|
|
89
|
+
resolve(answer);
|
|
90
|
+
},
|
|
91
|
+
(error: unknown) => {
|
|
92
|
+
settle();
|
|
93
|
+
reject(typed(command, error));
|
|
94
|
+
},
|
|
95
|
+
);
|
|
96
|
+
});
|
|
97
|
+
|
|
98
|
+
const parked = <T>(command: string, run: () => Promise<T>): Promise<T> => {
|
|
99
|
+
const left = parkedUntil - clock.monotonic();
|
|
100
|
+
if (left > 0) {
|
|
101
|
+
return Promise.reject(
|
|
102
|
+
new CacheDriverUnavailableError({
|
|
103
|
+
driver: 'redis',
|
|
104
|
+
cause: `parked for another ${String(Math.ceil(left))} ms after a command missed its ${String(commandTimeoutMs)} ms deadline; ${command} was not sent`,
|
|
105
|
+
fix: FIX,
|
|
106
|
+
}),
|
|
107
|
+
);
|
|
108
|
+
}
|
|
109
|
+
return bounded(command, commandTimeoutMs, run);
|
|
110
|
+
};
|
|
111
|
+
|
|
112
|
+
return {
|
|
113
|
+
value: {
|
|
114
|
+
get: (key) => parked('GET', () => client.get(key)),
|
|
115
|
+
set: (key, text) => parked('SET', () => client.set(key, text)),
|
|
116
|
+
send: (command, args) => parked(command, () => client.send(command, args)),
|
|
117
|
+
},
|
|
118
|
+
bust: {
|
|
119
|
+
get: (key) => bounded('GET', DEFAULT_REDIS_BUST_TIMEOUT_MS, () => client.get(key)),
|
|
120
|
+
set: (key, text) =>
|
|
121
|
+
bounded('SET', DEFAULT_REDIS_BUST_TIMEOUT_MS, () => client.set(key, text)),
|
|
122
|
+
send: (command, args) =>
|
|
123
|
+
bounded(command, DEFAULT_REDIS_BUST_TIMEOUT_MS, () => client.send(command, args)),
|
|
124
|
+
},
|
|
125
|
+
};
|
|
126
|
+
}
|
package/src/redis.ts
CHANGED
|
@@ -1,12 +1,16 @@
|
|
|
1
|
-
// Tier 2: shared cache over `
|
|
1
|
+
// Tier 2: shared cache over core's `redisClient()` (no client dependency — the runtime ships one).
|
|
2
2
|
// A tag -> keys SET is maintained alongside every write so `invalidateTags` is ONE round
|
|
3
3
|
// trip via a server-side script, not a KEYS scan. KEYS is O(n) and blocks the server; a
|
|
4
4
|
// framework that ships it as the invalidation path is shipping an outage.
|
|
5
5
|
|
|
6
6
|
import type { Clock } from '@ultimat3/core';
|
|
7
7
|
import { appVersion, logger, systemClock } from '@ultimat3/core';
|
|
8
|
+
import type { DeadlineTimer } from '@ultimat3/core/outbound';
|
|
9
|
+
import { redisClient } from '@ultimat3/core/redis';
|
|
8
10
|
import { CacheDriverUnavailableError } from './errors';
|
|
9
11
|
import type { FenceScope } from './fence';
|
|
12
|
+
import type { DeadlinedRedis } from './redis-deadline';
|
|
13
|
+
import { DEFAULT_REDIS_COMMAND_TIMEOUT_MS, withinDeadline } from './redis-deadline';
|
|
10
14
|
import { bumpGenerations, sampleGenerations } from './redis-fence';
|
|
11
15
|
import type { CacheTag } from './tags';
|
|
12
16
|
import { parseTag, serializeTag } from './tags';
|
|
@@ -38,8 +42,20 @@ export interface RedisTierOptions {
|
|
|
38
42
|
*/
|
|
39
43
|
readonly buildId?: string | null;
|
|
40
44
|
readonly defaultTtlMs?: number;
|
|
41
|
-
/**
|
|
45
|
+
/**
|
|
46
|
+
* Injected in tests and for a second server; production takes `redisClient()` from
|
|
47
|
+
* `@ultimat3/core/redis` (`REDIS_URL`, else `VALKEY_URL`; the CLI boot hands in a lazy one). A client handed in keeps its own reconnect behaviour and
|
|
48
|
+
* still gets the deadline.
|
|
49
|
+
*/
|
|
42
50
|
readonly client?: RedisLike;
|
|
51
|
+
/**
|
|
52
|
+
* How long one value-path command may take, in ms. Default `DEFAULT_REDIS_COMMAND_TIMEOUT_MS`
|
|
53
|
+
* (150). A miss is `X_CACHE_DRIVER_UNAVAILABLE` in `recentTierFailures()`, read as a cache miss,
|
|
54
|
+
* and parks the value path for `REDIS_PARK_MS`. A bust has its own, longer, unparked deadline.
|
|
55
|
+
*/
|
|
56
|
+
readonly commandTimeoutMs?: number;
|
|
57
|
+
/** Injected so a test fires the deadline instead of waiting for it. */
|
|
58
|
+
readonly timer?: DeadlineTimer;
|
|
43
59
|
/** Turns `PTTL`'s remaining life into the absolute `expiresAt` a hit reports. */
|
|
44
60
|
readonly clock?: Clock;
|
|
45
61
|
/** TTL spread, in `[0, 1)`. Default `DEFAULT_TTL_JITTER_FRACTION`; `0` disables it. */
|
|
@@ -132,19 +148,6 @@ const DELETE_BATCH = 128;
|
|
|
132
148
|
/** Seconds a tag set outlives its newest member. See `TAG_MEMBER_SCRIPT`. */
|
|
133
149
|
const TAG_TTL_GRACE_SECONDS = 60;
|
|
134
150
|
|
|
135
|
-
function resolveClient(injected: RedisLike | undefined): RedisLike {
|
|
136
|
-
if (injected !== undefined) return injected;
|
|
137
|
-
const candidate = (Bun as unknown as { redis?: RedisLike }).redis;
|
|
138
|
-
if (candidate === undefined || typeof candidate.send !== 'function') {
|
|
139
|
-
throw new CacheDriverUnavailableError({
|
|
140
|
-
driver: 'redis',
|
|
141
|
-
cause: 'Bun.redis is not available (needs bun >= 1.3 and REDIS_URL)',
|
|
142
|
-
fix: 'set REDIS_URL in .env, or drop the redis tier from cache.tiers in app.config.ts',
|
|
143
|
-
});
|
|
144
|
-
}
|
|
145
|
-
return candidate;
|
|
146
|
-
}
|
|
147
|
-
|
|
148
151
|
const toStrings = (value: unknown): string[] =>
|
|
149
152
|
Array.isArray(value) ? value.map((item) => String(item)) : [];
|
|
150
153
|
|
|
@@ -204,12 +207,27 @@ export function redisTier(options: RedisTierOptions = {}): CacheTier {
|
|
|
204
207
|
...(options.jitterFraction === undefined ? {} : { jitterFraction: options.jitterFraction }),
|
|
205
208
|
...(options.rng === undefined ? {} : { rng: options.rng }),
|
|
206
209
|
};
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
210
|
+
const commandTimeoutMs = assertFiniteDurationMs(
|
|
211
|
+
'redis',
|
|
212
|
+
'commandTimeoutMs',
|
|
213
|
+
options.commandTimeoutMs ?? DEFAULT_REDIS_COMMAND_TIMEOUT_MS,
|
|
214
|
+
);
|
|
215
|
+
// Built at the first command, never at construction: a tier selected in config dials nothing
|
|
216
|
+
// until it is used.
|
|
217
|
+
let bounded: DeadlinedRedis | undefined;
|
|
218
|
+
const views = (): DeadlinedRedis => {
|
|
219
|
+
bounded ??= withinDeadline(options.client ?? redisClient(), {
|
|
220
|
+
commandTimeoutMs,
|
|
221
|
+
clock,
|
|
222
|
+
...(options.timer === undefined ? {} : { timer: options.timer }),
|
|
223
|
+
});
|
|
224
|
+
return bounded;
|
|
212
225
|
};
|
|
226
|
+
// A read, a fill and a fence sample: parked after a missed deadline, so a stalled store costs
|
|
227
|
+
// one deadline per `REDIS_PARK_MS`, not five per read.
|
|
228
|
+
const conn = (): RedisLike => views().value;
|
|
229
|
+
// A bust skips the park: skipped for a stall, it leaves stale rows serving until TTL.
|
|
230
|
+
const bustConn = (): RedisLike => views().bust;
|
|
213
231
|
|
|
214
232
|
const valueKey = (key: string): string => `${ns}:c:${key}`;
|
|
215
233
|
// `{entity}` is a Redis Cluster hash tag, not decoration: it is what makes a row's bucket and
|
|
@@ -369,10 +387,10 @@ export function redisTier(options: RedisTierOptions = {}): CacheTier {
|
|
|
369
387
|
// Before the buckets are read, and awaited: see `bumpGenerations` for why the order is the
|
|
370
388
|
// guarantee. A refusal here rejects the bust into `report.errors` with nothing deleted, so
|
|
371
389
|
// the retry it asks for repeats the whole thing.
|
|
372
|
-
await bumpGenerations(
|
|
390
|
+
await bumpGenerations(bustConn(), ns, tags);
|
|
373
391
|
const replies = await Promise.all(
|
|
374
392
|
perTag.map(({ read }) =>
|
|
375
|
-
|
|
393
|
+
bustConn().send('EVAL', [INVALIDATE_SCRIPT, String(read.length), ...read]),
|
|
376
394
|
),
|
|
377
395
|
);
|
|
378
396
|
// A member may sit in two tag sets; deleting it twice is harmless but reporting it twice
|
|
@@ -385,7 +403,7 @@ export function redisTier(options: RedisTierOptions = {}): CacheTier {
|
|
|
385
403
|
// `allSettled`, because which ones died decides what leaves the buckets below.
|
|
386
404
|
const batch = members.slice(start, start + DELETE_BATCH);
|
|
387
405
|
const settled = await Promise.allSettled(
|
|
388
|
-
batch.map((member) =>
|
|
406
|
+
batch.map((member) => bustConn().send('DEL', [member])),
|
|
389
407
|
);
|
|
390
408
|
settled.forEach((result, index) => {
|
|
391
409
|
const member = batch[index];
|
|
@@ -402,7 +420,7 @@ export function redisTier(options: RedisTierOptions = {}): CacheTier {
|
|
|
402
420
|
const gone = [...new Set(toStrings(replies[i]))].filter((member) => deleted.has(member));
|
|
403
421
|
for (const bucket of perTag[i]?.sweep ?? []) {
|
|
404
422
|
for (let start = 0; start < gone.length; start += DELETE_BATCH) {
|
|
405
|
-
await
|
|
423
|
+
await bustConn().send('SREM', [bucket, ...gone.slice(start, start + DELETE_BATCH)]);
|
|
406
424
|
}
|
|
407
425
|
}
|
|
408
426
|
}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
// Which load holds each key in a read-through flight, and whether a bust has overtaken it. A read
|
|
2
|
+
// asks this before joining, and the stampede guard's arrivals are the callers: every one of ~1,600
|
|
3
|
+
// joiners asking the fence walks the marks since its sample, so this asks only when something moved.
|
|
4
|
+
|
|
5
|
+
import type { CacheFence } from './fence';
|
|
6
|
+
import { fenceGeneration } from './fence';
|
|
7
|
+
import type { CacheTag } from './tags';
|
|
8
|
+
import { serializeTag } from './tags';
|
|
9
|
+
|
|
10
|
+
interface Leading {
|
|
11
|
+
readonly fence: CacheFence;
|
|
12
|
+
/** Wire forms already on the fence, so a joiner's repeat of a tag adds no scope to walk. */
|
|
13
|
+
readonly covered: Set<string>;
|
|
14
|
+
/** The fence generation it last answered valid at. */
|
|
15
|
+
checkedAt: number;
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export interface LeadingLoads {
|
|
19
|
+
/** Call as the fence is sampled, in the same tick, so `checkedAt` is the fence's own sample. */
|
|
20
|
+
lead(key: string, fence: CacheFence, tags: readonly CacheTag[] | undefined): void;
|
|
21
|
+
/** Has anything this reader needs to see landed since the load holding `key` began? */
|
|
22
|
+
overtaken(key: string, tags: readonly CacheTag[] | undefined): boolean;
|
|
23
|
+
/** The flight's `released`: an entry exists exactly while the flight holds the key. */
|
|
24
|
+
release(key: string): void;
|
|
25
|
+
/** Keys tracked right now. A number that does not fall back to `0` is a leak. */
|
|
26
|
+
readonly size: number;
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
export function leadingLoads(): LeadingLoads {
|
|
30
|
+
const loads = new Map<string, Leading>();
|
|
31
|
+
return {
|
|
32
|
+
lead(key, fence, tags): void {
|
|
33
|
+
const covered = new Set((tags ?? []).map(serializeTag));
|
|
34
|
+
loads.set(key, { fence, covered, checkedAt: fenceGeneration() });
|
|
35
|
+
},
|
|
36
|
+
|
|
37
|
+
overtaken(key, tags): boolean {
|
|
38
|
+
const held = loads.get(key);
|
|
39
|
+
if (held === undefined) return false;
|
|
40
|
+
const unseen: CacheTag[] = [];
|
|
41
|
+
for (const owned of tags ?? []) {
|
|
42
|
+
const wire = serializeTag(owned);
|
|
43
|
+
if (held.covered.has(wire)) continue;
|
|
44
|
+
held.covered.add(wire);
|
|
45
|
+
unseen.push(owned);
|
|
46
|
+
}
|
|
47
|
+
// Covering reaches back to the sample, so a tag new to this load must be asked about even
|
|
48
|
+
// when nothing has moved since the last answer.
|
|
49
|
+
if (unseen.length > 0) held.fence.cover({ tags: unseen });
|
|
50
|
+
const generation = fenceGeneration();
|
|
51
|
+
if (unseen.length === 0 && held.checkedAt === generation) return false;
|
|
52
|
+
held.checkedAt = generation;
|
|
53
|
+
return !held.fence.isValid();
|
|
54
|
+
},
|
|
55
|
+
|
|
56
|
+
release(key): void {
|
|
57
|
+
loads.delete(key);
|
|
58
|
+
},
|
|
59
|
+
|
|
60
|
+
get size(): number {
|
|
61
|
+
return loads.size;
|
|
62
|
+
},
|
|
63
|
+
};
|
|
64
|
+
}
|
package/src/tiers.ts
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
// sites. Order is data, not control flow.
|
|
5
5
|
|
|
6
6
|
import type { CacheTierName, Clock, Scheduler } from '@ultimat3/core';
|
|
7
|
-
import { CACHE_TIERS, singleFlight, systemClock } from '@ultimat3/core';
|
|
7
|
+
import { CACHE_TIERS, singleFlight, systemClock, tryUseContext } from '@ultimat3/core';
|
|
8
8
|
import { CacheJitterInvalidError, CacheLimitInvalidError, CacheTtlInvalidError } from './errors';
|
|
9
9
|
import type { CacheFence, FenceScope } from './fence';
|
|
10
10
|
import { markInvalidated, sampleFence } from './fence';
|
|
@@ -13,6 +13,7 @@ import type { CacheTag } from './tags';
|
|
|
13
13
|
import { bestEffort } from './tier-failures';
|
|
14
14
|
import type { TierFence, TierFences } from './tier-fence';
|
|
15
15
|
import { sampleTierFences, verdictOf } from './tier-fence';
|
|
16
|
+
import { leadingLoads } from './tier-leading';
|
|
16
17
|
|
|
17
18
|
/**
|
|
18
19
|
* The rungs, spelled in `@ultimat3/core` and nowhere else. Tier 0 owns the NAMES because
|
|
@@ -265,6 +266,9 @@ export function cacheStack(
|
|
|
265
266
|
): CacheStack {
|
|
266
267
|
const ordered = sortTiers(tiers);
|
|
267
268
|
const clock = options.clock ?? systemClock;
|
|
269
|
+
// The fence of each load holding a key, so a read can refuse to join one a bust has overtaken.
|
|
270
|
+
// The fence stops that load's FILL; only this stops a read-your-write being answered by it.
|
|
271
|
+
const leading = leadingLoads();
|
|
268
272
|
// Per stack, not per module: two stacks are two ladders and must not join each other's loads.
|
|
269
273
|
//
|
|
270
274
|
// The deadline frees the KEY and nothing else — `load()` is the app's function and this stack
|
|
@@ -282,6 +286,12 @@ export function cacheStack(
|
|
|
282
286
|
'the loadDeadlineMs argument to cacheStack(...)',
|
|
283
287
|
),
|
|
284
288
|
schedule: options.schedule,
|
|
289
|
+
// A closed tab's aborted load must not fail every healthy reader that joined it: this is the
|
|
290
|
+
// cold-key arrival the flight exists to collapse, so the first of them loads again instead.
|
|
291
|
+
signal: () => tryUseContext()?.signal,
|
|
292
|
+
released: (key) => {
|
|
293
|
+
leading.release(key);
|
|
294
|
+
},
|
|
285
295
|
});
|
|
286
296
|
|
|
287
297
|
/** Take back what a fence refused mid-ladder: half a stale ladder is still a stale read. */
|
|
@@ -397,6 +407,11 @@ export function cacheStack(
|
|
|
397
407
|
// The stampede guard. The homepage feed read 8,000x/s with a 60s lease misses for the whole
|
|
398
408
|
// ~200ms `load()` takes, so ~1,600 identical queries reach Postgres at every TTL boundary
|
|
399
409
|
// unless the arrivals inside that window join the read already running.
|
|
410
|
+
//
|
|
411
|
+
// Unless that read began before a write this one must see. Asked of the fence and not of the
|
|
412
|
+
// key, because a tag bust names no key. This reader's own tags are covered first, back to
|
|
413
|
+
// the sample: a bust of a tag only it declares has overtaken the load just the same.
|
|
414
|
+
if (leading.overtaken(key, setOptions?.tags)) flight.evict(key);
|
|
400
415
|
return await flight.run<T, CacheSetOptions>(
|
|
401
416
|
key,
|
|
402
417
|
async (shared) => {
|
|
@@ -406,6 +421,7 @@ export function cacheStack(
|
|
|
406
421
|
key,
|
|
407
422
|
...(setOptions?.tags === undefined ? {} : { tags: setOptions.tags }),
|
|
408
423
|
});
|
|
424
|
+
leading.lead(key, fence, setOptions?.tags);
|
|
409
425
|
// The same sample, taken in each tier other replicas also write. One round trip, and
|
|
410
426
|
// only when such a tier is on the ladder.
|
|
411
427
|
const fleet = await sampleTierFences(ordered, key, {
|