@ayepi/redis 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/ayepi-redis.md +501 -0
- package/package.json +9 -8
package/README.md
CHANGED
|
@@ -101,7 +101,7 @@ This package ships dense, machine-oriented reference docs written for **AI codin
|
|
|
101
101
|
|
|
102
102
|
- [`ayepi-redis.md`](./ayepi-redis.md)
|
|
103
103
|
|
|
104
|
-
They
|
|
104
|
+
They ship with this package and also live in the [repo](https://github.com/ClickerMonkey/ayepi/tree/main/packages/redis).
|
|
105
105
|
|
|
106
106
|
## License
|
|
107
107
|
|
package/ayepi-redis.md
ADDED
|
@@ -0,0 +1,501 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
ayepi-redis.md — reference for `@ayepi/redis`, written for coding agents.
|
|
3
|
+
|
|
4
|
+
Copy this file into any project that depends on `@ayepi/redis` (e.g. into your repo's
|
|
5
|
+
`docs/` or `.claude/` directory) and reference it from your agents and slash commands.
|
|
6
|
+
It documents the public API, the patterns the package expects, and how it works under the
|
|
7
|
+
hood, with copy-pasteable examples. Keep it in sync with the installed package version.
|
|
8
|
+
-->
|
|
9
|
+
|
|
10
|
+
# `@ayepi/redis`
|
|
11
|
+
|
|
12
|
+
The Redis backends for running ayepi on **more than one instance**, built for
|
|
13
|
+
[ioredis](https://github.com/redis/ioredis). The package ships **four** things:
|
|
14
|
+
|
|
15
|
+
- a pub/sub `Broker` (`redisBroker`) for [`@ayepi/core`](./ayepi-core.md) — an `emit` on one
|
|
16
|
+
instance reaches subscribers on **every** instance (multi-pod / multi-process event fanout,
|
|
17
|
+
no sticky sessions);
|
|
18
|
+
- a [`@ayepi/work`](./ayepi-work.md) `Store` (`redisStore`) — get/set with TTL, the atomic
|
|
19
|
+
`setIfNotExists` claim, and the atomic `increment` counter;
|
|
20
|
+
- a `@ayepi/work` `PubSub` (`redisPubSub`) — the same fanout as the broker, exposed under the
|
|
21
|
+
work-port name so a `{ store, pubsub }` pairing reads cleanly;
|
|
22
|
+
- a [`@ayepi/cache`](./ayepi-cache.md) `CacheStore` (`redisCache`) — a response cache shared
|
|
23
|
+
across instances.
|
|
24
|
+
|
|
25
|
+
Reach for the **broker** whenever you run more than one server instance behind a load
|
|
26
|
+
balancer and need live events (progress, presence, chat fanout) to reach a WebSocket client
|
|
27
|
+
regardless of which instance it is connected to — the in-process default broker (`localBroker`,
|
|
28
|
+
see [ayepi-core.md](./ayepi-core.md)) cannot do this. Reach for the **store/pubsub/cache** when
|
|
29
|
+
you run a distributed `@ayepi/work` engine or a shared response cache. Every store/cache Redis
|
|
30
|
+
call is wrapped in `@ayepi/core`'s `retry` (configurable per backend), so a transient blip or a
|
|
31
|
+
throttled reply is absorbed rather than surfaced; a final failure fires `onError` and propagates.
|
|
32
|
+
|
|
33
|
+
`ioredis` is a **peer dependency** (`^5`) — you install and own the client.
|
|
34
|
+
[`@ayepi/work`](./ayepi-work.md) and [`@ayepi/cache`](./ayepi-cache.md) are **optional,
|
|
35
|
+
type-only peer deps** — only needed if you use those backends:
|
|
36
|
+
|
|
37
|
+
```sh
|
|
38
|
+
pnpm add @ayepi/redis @ayepi/core ioredis
|
|
39
|
+
# only if you use the work backend: pnpm add @ayepi/work
|
|
40
|
+
# only if you use the cache store: pnpm add @ayepi/cache
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Public API
|
|
44
|
+
|
|
45
|
+
The package exports the `redisBroker` factory plus its `RedisBrokerOptions` options type and
|
|
46
|
+
the `RedisLike` structural client interface; the work `Store` factory `redisStore` (with
|
|
47
|
+
`RedisStoreOptions`) and the work `PubSub` factory `redisPubSub`; the `@ayepi/cache`
|
|
48
|
+
`CacheStore` factory `redisCache` (with `RedisCacheOptions`); and the `RedisCommandClient`
|
|
49
|
+
structural interface the store and cache use.
|
|
50
|
+
|
|
51
|
+
### `redisBroker(client, opts?)`
|
|
52
|
+
|
|
53
|
+
```ts
|
|
54
|
+
function redisBroker(client: RedisLike, opts?: RedisBrokerOptions): Broker
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Creates a Redis pub/sub `Broker`. Parameters:
|
|
58
|
+
|
|
59
|
+
- `client` — an ioredis connection used to **publish**. ioredis's `Redis` satisfies
|
|
60
|
+
`RedisLike` structurally, so pass `new Redis(url)` directly. Unless you pass
|
|
61
|
+
`opts.subscriber`, a dedicated **subscriber** connection is derived from it via
|
|
62
|
+
`client.duplicate()`.
|
|
63
|
+
- `opts` — `RedisBrokerOptions` (all fields optional, see below).
|
|
64
|
+
|
|
65
|
+
Returns a `Broker` (the `@ayepi/core` interface):
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
interface Broker {
|
|
69
|
+
publish(message: string): void | Promise<void>;
|
|
70
|
+
subscribe(listener: (message: string) => void): () => void;
|
|
71
|
+
}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
You normally never call `publish`/`subscribe` yourself — you hand the broker to
|
|
75
|
+
`server(api, [impl], { broker })` and the framework drives it. The behavior of the
|
|
76
|
+
returned broker:
|
|
77
|
+
|
|
78
|
+
- `publish(message)` — publishes `message` to the configured channel via `client.publish`.
|
|
79
|
+
Always returns a `Promise<void>` that resolves even on failure; publish errors are routed
|
|
80
|
+
to `onError` (never thrown).
|
|
81
|
+
- `subscribe(listener)` — on the **first** call it lazily wires up the subscriber
|
|
82
|
+
connection (subscribes to the channel, attaches the `message` handler). Adds `listener` to
|
|
83
|
+
an in-memory set; every message delivered on the channel is dispatched to all listeners.
|
|
84
|
+
Returns an **unsubscribe** function that removes just that listener. A throwing listener is
|
|
85
|
+
caught and reported to `onError`; it does not break delivery to the other listeners.
|
|
86
|
+
|
|
87
|
+
### `RedisBrokerOptions`
|
|
88
|
+
|
|
89
|
+
```ts
|
|
90
|
+
interface RedisBrokerOptions {
|
|
91
|
+
/** The pub/sub channel name (default `'ayepi'`). All instances must agree. */
|
|
92
|
+
readonly channel?: string;
|
|
93
|
+
/**
|
|
94
|
+
* The connection to subscribe on. Defaults to `client.duplicate()`. Provide one
|
|
95
|
+
* if you manage connections yourself — it must be dedicated to subscribing.
|
|
96
|
+
*/
|
|
97
|
+
readonly subscriber?: RedisLike;
|
|
98
|
+
/** Notified on publish/subscribe/connection errors. */
|
|
99
|
+
readonly onError?: (error: unknown) => void;
|
|
100
|
+
}
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
- **`channel`** (default `'ayepi'`) — the Redis channel all instances publish to and
|
|
104
|
+
subscribe on. Every instance that should share events **must use the same channel**. Use
|
|
105
|
+
distinct channel names to isolate independent app clusters that share one Redis.
|
|
106
|
+
- **`subscriber`** (default `client.duplicate()`) — the dedicated connection used to receive
|
|
107
|
+
messages. A connection in subscribe mode cannot run other commands, so the broker keeps
|
|
108
|
+
publishing on `client` and subscribing on a *separate* connection. Pass your own when you
|
|
109
|
+
manage connection lifecycles yourself — but it must be dedicated to subscribing.
|
|
110
|
+
- **`onError`** — called for publish errors, subscribe errors, connection (`'error'`) events
|
|
111
|
+
on both connections, and exceptions thrown by your subscribe listeners. If omitted, the
|
|
112
|
+
broker does not attach `'error'` handlers to the connections (so connection errors follow
|
|
113
|
+
ioredis's own defaults) and silently swallows the rest.
|
|
114
|
+
|
|
115
|
+
### `RedisLike`
|
|
116
|
+
|
|
117
|
+
The minimal ioredis surface the broker uses. ioredis's `Redis` satisfies it structurally;
|
|
118
|
+
a compatible client with matching method shapes works too. You rarely reference this type —
|
|
119
|
+
it exists so you can pass mocks or alternative clients.
|
|
120
|
+
|
|
121
|
+
```ts
|
|
122
|
+
interface RedisLike {
|
|
123
|
+
publish(channel: string, message: string): Promise<number> | number;
|
|
124
|
+
subscribe(...channels: string[]): Promise<unknown> | unknown;
|
|
125
|
+
unsubscribe(...channels: string[]): Promise<unknown> | unknown;
|
|
126
|
+
duplicate(): RedisLike;
|
|
127
|
+
on(event: 'message', listener: (channel: string, message: string) => void): unknown;
|
|
128
|
+
on(event: 'error', listener: (error: Error) => void): unknown;
|
|
129
|
+
}
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
### `redisStore(client, opts?)`
|
|
133
|
+
|
|
134
|
+
```ts
|
|
135
|
+
function redisStore(client: RedisCommandClient, opts?: RedisStoreOptions): Store
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
A Redis-backed [`@ayepi/work`](./ayepi-work.md) `Store` (the get/set + compare-and-set port;
|
|
139
|
+
see [ayepi-work-ports.md](./ayepi-work-ports.md)). You normally never call its methods
|
|
140
|
+
yourself — you hand it to `createWork({ store: redisStore(client), … })` and the engine
|
|
141
|
+
drives it. Each operation issues one Redis command (all keys prefixed by `opts.prefix`, all
|
|
142
|
+
wrapped in `retry`):
|
|
143
|
+
|
|
144
|
+
- `get(key)` → `GET` — returns the string, or `undefined` when absent/expired (`null` → `undefined`).
|
|
145
|
+
- `set(key, value, ttl?)` → `SET key value` (no TTL) or `SET key value PX ttl` (TTL in ms).
|
|
146
|
+
- `delete(key)` → `DEL`.
|
|
147
|
+
- `setIfNotExists(key, value, ttl?)` → `SET key value NX` (or `SET … PX ttl NX` with a TTL).
|
|
148
|
+
Returns `true` when this caller won the slot, `false` when the key already held a value
|
|
149
|
+
(Redis returns `null`). **`SET NX` is server-side atomic — this is the fleet-safe CAS atom
|
|
150
|
+
behind every claim/lease/idempotency check.**
|
|
151
|
+
- `increment(key, by, ttl?)` → `INCRBY key by`, returning the new integer; when `ttl` is
|
|
152
|
+
given it then issues `PEXPIRE key ttl`. **`INCRBY` is server-side atomic — the group
|
|
153
|
+
open-work counter is correct under concurrency across the fleet.** Note the optional
|
|
154
|
+
`PEXPIRE` is a *separate* command (the increment and its expiry are not one atomic unit).
|
|
155
|
+
|
|
156
|
+
### `redisPubSub(client, opts?)`
|
|
157
|
+
|
|
158
|
+
```ts
|
|
159
|
+
function redisPubSub(client: RedisLike, opts?: RedisBrokerOptions): PubSub
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The [`@ayepi/work`](./ayepi-work.md) `PubSub` backed by Redis pub/sub. It is **literally
|
|
163
|
+
`redisBroker`** — the `@ayepi/core` `Broker` and the `@ayepi/work` `PubSub` ports are
|
|
164
|
+
identical in shape (`publish(message)` + `subscribe(listener) → unsubscribe`), so this is a
|
|
165
|
+
thin alias exposed under the work-port name so `{ store: redisStore(c), pubsub: redisPubSub(c) }`
|
|
166
|
+
reads cleanly. It takes the same `RedisLike` client and the same `RedisBrokerOptions`
|
|
167
|
+
(`channel`, `subscriber`, `onError`) and has identical behavior and gotchas — see
|
|
168
|
+
[`redisBroker`](#redisbrokerclient-opts) above. Best-effort: it wakes distributed waiters; the
|
|
169
|
+
engine's store-poll fallback covers a silent channel, so a missed message is not a
|
|
170
|
+
correctness bug.
|
|
171
|
+
|
|
172
|
+
### `redisCache(client, opts?)`
|
|
173
|
+
|
|
174
|
+
```ts
|
|
175
|
+
function redisCache(client: RedisCommandClient, opts?: RedisCacheOptions): CacheStore
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
A Redis-backed [`@ayepi/cache`](./ayepi-cache.md) `CacheStore` so a response cache is shared
|
|
179
|
+
across instances. Hand it to `cache.server(def, { store: redisCache(client) })`. Entries are
|
|
180
|
+
stored as JSON; keys are prefixed by `opts.prefix` (default `'ayepi:cache:'`); every call is
|
|
181
|
+
wrapped in `retry`:
|
|
182
|
+
|
|
183
|
+
- `get(key)` → `GET`, `JSON.parse`d back to a `CacheEntry` (or `undefined` when absent).
|
|
184
|
+
- `set(key, entry)` → `SET key <json> PX <ms>`, where `<ms> = max(1, entry.staleUntil - now())`.
|
|
185
|
+
The Redis TTL is derived from the entry's `staleUntil`, so a dead entry self-evicts even if
|
|
186
|
+
nothing reads it (clamped to a minimum of `1` ms so an already-stale entry still `SET`s).
|
|
187
|
+
- `delete(key)` → `DEL`; returns `true` when a key was removed.
|
|
188
|
+
- `clear()` → `SCAN` the whole `prefix*` keyspace (`MATCH prefix* COUNT 256`, paginated until
|
|
189
|
+
the cursor returns to `0`) and `DEL` everything found.
|
|
190
|
+
- `invalidate(pred)` → `SCAN` the `prefix*` keyspace, `GET` + `JSON.parse` each key, run
|
|
191
|
+
`pred(meta)` against its `EntryMeta` (`key`/`method`/`path`/`storedAt`/`expires`/`staleUntil`/
|
|
192
|
+
`bytes`), and `DEL` the matches; returns how many were removed. Keys that vanished between the
|
|
193
|
+
scan and the read (expired in the interim) are skipped.
|
|
194
|
+
|
|
195
|
+
`RedisCacheOptions` adds one field over the shared options: **`now`** (default `Date.now`) —
|
|
196
|
+
the clock used to compute the `PX` TTL from `staleUntil`; inject it in tests.
|
|
197
|
+
|
|
198
|
+
### `RedisCommandClient`
|
|
199
|
+
|
|
200
|
+
The minimal command surface the **store and cache** use (the pub/sub broker uses the
|
|
201
|
+
different, subscribe-mode `RedisLike` surface above). ioredis's `Redis` satisfies it
|
|
202
|
+
structurally — pass `new Redis(url)` directly; a compatible client with matching method
|
|
203
|
+
shapes (or a mock) works too. You rarely reference this type by name.
|
|
204
|
+
|
|
205
|
+
```ts
|
|
206
|
+
interface RedisCommandClient {
|
|
207
|
+
get(key: string): Promise<string | null>;
|
|
208
|
+
/** `set(key, value)` / `set(key, value, 'PX', ttl)` / `set(key, value, 'NX')` — returns `'OK'` or `null`. */
|
|
209
|
+
set(key: string, value: string, ...args: (string | number)[]): Promise<string | null>;
|
|
210
|
+
del(...keys: string[]): Promise<number>;
|
|
211
|
+
incrby(key: string, increment: number): Promise<number>;
|
|
212
|
+
pexpire(key: string, ms: number): Promise<number>;
|
|
213
|
+
scan(cursor: string | number, ...args: (string | number)[]): Promise<[cursor: string, keys: string[]]>;
|
|
214
|
+
}
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
### `RedisStoreOptions` / `RedisCacheOptions` (shared resilience options)
|
|
218
|
+
|
|
219
|
+
Both `redisStore` and `redisCache` share three options; `redisCache` adds `now` (above).
|
|
220
|
+
|
|
221
|
+
```ts
|
|
222
|
+
interface RedisStoreOptions {
|
|
223
|
+
readonly prefix?: string; // key namespace (default: '' for the store)
|
|
224
|
+
readonly retry?: Omit<RetryOptions, 'errorResult'>; // per-call retry policy (core `retry`)
|
|
225
|
+
readonly onError?: (err: unknown) => void; // fired once on final failure, then rethrown
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
interface RedisCacheOptions extends RedisStoreOptions {
|
|
229
|
+
readonly now?: () => number; // clock for the PX TTL (default: Date.now)
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
- **`prefix`** — namespaces every key the backend touches. The store defaults to `''` (no
|
|
234
|
+
prefix); the cache defaults to `'ayepi:cache:'`. Give independent apps/engines distinct
|
|
235
|
+
prefixes to share one Redis without collisions; the cache's `clear`/`invalidate` only ever
|
|
236
|
+
`SCAN`/`DEL` keys under its own prefix.
|
|
237
|
+
- **`retry`** — the per-call retry policy, an `Omit<RetryOptions, 'errorResult'>` passed
|
|
238
|
+
straight to `@ayepi/core`'s `retry` (`attempts`/`base`/`factor`/`max`/`jitter`/`sleep`/…; see
|
|
239
|
+
[ayepi-core.md](./ayepi-core.md)). `errorResult` is reserved by the package (it routes the
|
|
240
|
+
final error to `onError`), so you cannot set it. Defaults are core's `retry` defaults.
|
|
241
|
+
- **`onError`** — called **once**, with the final error, only after retries are exhausted; the
|
|
242
|
+
error then propagates to the caller. Off by default. It is **guarded**: a throwing `onError`
|
|
243
|
+
is swallowed so it can never mask the original Redis error. (This is distinct from the
|
|
244
|
+
broker's `onError`, which is per-event and never rethrows.)
|
|
245
|
+
|
|
246
|
+
How resilience works: each method runs through an internal `makeRun(opts)` helper that calls
|
|
247
|
+
`retry(fn, { ...opts.retry, onError })`. So a transient failure is retried per `opts.retry`;
|
|
248
|
+
if every attempt fails, `onError` is invoked (guarded) and the rejection is rethrown to the
|
|
249
|
+
engine/middleware, which applies its own higher-level handling.
|
|
250
|
+
|
|
251
|
+
## Examples
|
|
252
|
+
|
|
253
|
+
### Basic: create a broker and pass it to `server`
|
|
254
|
+
|
|
255
|
+
```ts
|
|
256
|
+
import Redis from 'ioredis';
|
|
257
|
+
import { redisBroker } from '@ayepi/redis';
|
|
258
|
+
import { implement, server } from '@ayepi/core';
|
|
259
|
+
|
|
260
|
+
const app = server(api, [implement(api).handlers(handlers)], {
|
|
261
|
+
broker: redisBroker(new Redis(process.env.REDIS_URL!)),
|
|
262
|
+
});
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
### Emitting across instances
|
|
266
|
+
|
|
267
|
+
`emit` is the same call you already use with the default broker — the Redis broker just
|
|
268
|
+
changes *where* it fans out. Any instance can emit; every instance's matching WebSocket
|
|
269
|
+
subscribers receive it. (See [ayepi-core.md](./ayepi-core.md) for `emit` / event params.)
|
|
270
|
+
|
|
271
|
+
```ts
|
|
272
|
+
// Instance B — emit a typed event. Reaches subscribers on instance A too.
|
|
273
|
+
appB.emit('progress', { job: 'j1' }, { pct: 77 });
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
Event params are matched per-subscriber by the framework: a client subscribed with
|
|
277
|
+
`{ job: 'j1' }` receives `{ job: 'j1' }` emits but not `{ job: 'other' }` ones — the broker
|
|
278
|
+
just carries the serialized message, `@ayepi/core` does the routing.
|
|
279
|
+
|
|
280
|
+
### Providing your own ioredis clients
|
|
281
|
+
|
|
282
|
+
Give the broker an explicit publisher **and** a dedicated subscriber when you want to own
|
|
283
|
+
connection options (TLS, retry strategy, etc.). Note `maxRetriesPerRequest: null` is a
|
|
284
|
+
common setting for long-lived pub/sub connections.
|
|
285
|
+
|
|
286
|
+
```ts
|
|
287
|
+
import Redis from 'ioredis';
|
|
288
|
+
import { redisBroker } from '@ayepi/redis';
|
|
289
|
+
|
|
290
|
+
const url = process.env.REDIS_URL!;
|
|
291
|
+
const pub = new Redis(url, { maxRetriesPerRequest: null });
|
|
292
|
+
const sub = new Redis(url, { maxRetriesPerRequest: null });
|
|
293
|
+
|
|
294
|
+
const broker = redisBroker(pub, {
|
|
295
|
+
subscriber: sub,
|
|
296
|
+
channel: 'myapp:events',
|
|
297
|
+
onError: (err) => console.error('[broker]', err),
|
|
298
|
+
});
|
|
299
|
+
|
|
300
|
+
const app = server(api, [implement(api).handlers(handlers)], { broker });
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
### Simulating multi-pod fanout (what the integration test proves)
|
|
304
|
+
|
|
305
|
+
```ts
|
|
306
|
+
const impl = implement(api).handlers(handlers);
|
|
307
|
+
const appA = server(api, [impl], { broker: redisBroker(new Redis(url), { subscriber: new Redis(url) }) });
|
|
308
|
+
const appB = server(api, [impl], { broker: redisBroker(new Redis(url), { subscriber: new Redis(url) }) });
|
|
309
|
+
|
|
310
|
+
// A WebSocket client is connected to appA and subscribes to `progress`:
|
|
311
|
+
sdk.on('progress', { job: 'j1' }, (d) => console.log(d.pct));
|
|
312
|
+
|
|
313
|
+
// An emit on the OTHER instance reaches that subscriber:
|
|
314
|
+
appB.emit('progress', { job: 'j1' }, { pct: 77 }); // → client logs 77
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
### Cleanup / shutdown
|
|
318
|
+
|
|
319
|
+
The broker has **no `close`/`disconnect` method of its own** — lifecycle is the ioredis
|
|
320
|
+
clients' lifecycle, which you own. To shut down: drop subscriptions (the unsubscribe
|
|
321
|
+
functions returned by `subscribe`, normally managed by `@ayepi/core`) and disconnect the
|
|
322
|
+
connections you created.
|
|
323
|
+
|
|
324
|
+
```ts
|
|
325
|
+
const pub = new Redis(url);
|
|
326
|
+
const sub = new Redis(url);
|
|
327
|
+
const broker = redisBroker(pub, { subscriber: sub });
|
|
328
|
+
// ... use broker ...
|
|
329
|
+
|
|
330
|
+
// on shutdown — disconnect the connections you own:
|
|
331
|
+
pub.disconnect();
|
|
332
|
+
sub.disconnect();
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
If you let the broker derive its subscriber via `client.duplicate()`, that duplicated
|
|
336
|
+
connection is internal — prefer passing an explicit `subscriber` you can close when you need
|
|
337
|
+
deterministic shutdown (e.g. in tests).
|
|
338
|
+
|
|
339
|
+
### Work backend: `redisStore` + `redisPubSub` in `createWork`
|
|
340
|
+
|
|
341
|
+
The store and pubsub are a drop-in `@ayepi/work` backend pairing. Combine them with a durable
|
|
342
|
+
queue (the queue must persist — pub/sub does not) and pass all three to `createWork`. One
|
|
343
|
+
ioredis connection serves both the store and the (publisher side of the) pubsub:
|
|
344
|
+
|
|
345
|
+
```ts
|
|
346
|
+
import Redis from 'ioredis';
|
|
347
|
+
import { redisStore, redisPubSub } from '@ayepi/redis';
|
|
348
|
+
import { createWork } from '@ayepi/work';
|
|
349
|
+
import { sqsQueue } from '@ayepi/aws'; // any durable Queue implementation
|
|
350
|
+
|
|
351
|
+
const redis = new Redis(process.env.REDIS_URL!, { maxRetriesPerRequest: null });
|
|
352
|
+
|
|
353
|
+
const work = createWork({
|
|
354
|
+
work: workDef,
|
|
355
|
+
store: redisStore(redis, { prefix: 'work:' }),
|
|
356
|
+
pubsub: redisPubSub(redis, { channel: 'work' }),
|
|
357
|
+
queue: sqsQueue({ url: process.env.QUEUE_URL! }),
|
|
358
|
+
});
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
`setIfNotExists` (`SET NX`) and `increment` (`INCRBY`) are atomic Redis-side, so claims and
|
|
362
|
+
the open-work counter stay correct no matter how many instances run. See
|
|
363
|
+
[ayepi-work.md](./ayepi-work.md) and [ayepi-work-ports.md](./ayepi-work-ports.md) for the
|
|
364
|
+
engine and the port contracts.
|
|
365
|
+
|
|
366
|
+
### Cache store: `redisCache` with `@ayepi/cache`
|
|
367
|
+
|
|
368
|
+
Back the response cache with Redis so every instance shares the same cached responses (and a
|
|
369
|
+
mutation on one instance can invalidate them for all):
|
|
370
|
+
|
|
371
|
+
```ts
|
|
372
|
+
import Redis from 'ioredis';
|
|
373
|
+
import { implement } from '@ayepi/core';
|
|
374
|
+
import { cache } from '@ayepi/cache';
|
|
375
|
+
import { cache as cacheServer } from '@ayepi/cache/server';
|
|
376
|
+
import { redisCache } from '@ayepi/redis';
|
|
377
|
+
|
|
378
|
+
const cached = cache(); // the frontend-safe def
|
|
379
|
+
const redis = new Redis(process.env.REDIS_URL!);
|
|
380
|
+
|
|
381
|
+
implement(api).middleware(
|
|
382
|
+
cacheServer.server(cached, {
|
|
383
|
+
store: redisCache(redis, { prefix: 'myapp:cache:' }),
|
|
384
|
+
ttl: 30_000,
|
|
385
|
+
}),
|
|
386
|
+
);
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
Entries carry a Redis `PX` TTL derived from each entry's `staleUntil`, so dead entries
|
|
390
|
+
self-evict; `clear`/`invalidate` `SCAN` the prefix. See [ayepi-cache.md](./ayepi-cache.md) for
|
|
391
|
+
the middleware, `EntryMeta`, and `cacheKey` (target `delete`/`invalidate` after a mutation).
|
|
392
|
+
|
|
393
|
+
### Configuring retry for a flaky Redis
|
|
394
|
+
|
|
395
|
+
The store and cache wrap every call in core's `retry`. Tune it per backend with `retry`, and
|
|
396
|
+
surface exhausted failures with `onError`:
|
|
397
|
+
|
|
398
|
+
```ts
|
|
399
|
+
import { redisStore, redisCache } from '@ayepi/redis';
|
|
400
|
+
|
|
401
|
+
const opts = {
|
|
402
|
+
retry: { attempts: 5, base: 50, factor: 2, max: 1000, jitter: true },
|
|
403
|
+
onError: (err: unknown) => console.error('[redis backend] gave up:', err),
|
|
404
|
+
};
|
|
405
|
+
|
|
406
|
+
const store = redisStore(redis, { prefix: 'work:', ...opts });
|
|
407
|
+
const cacheStore = redisCache(redis, { prefix: 'cache:', ...opts });
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
A transient failure is retried per the policy; only when **all** attempts fail is `onError`
|
|
411
|
+
called (once) and the error rethrown to the engine/middleware. A throwing `onError` is ignored
|
|
412
|
+
so it can never mask the underlying Redis error.
|
|
413
|
+
|
|
414
|
+
## How it works under the hood
|
|
415
|
+
|
|
416
|
+
- **Two connections, one channel.** Redis requires a connection in subscribe mode to do
|
|
417
|
+
nothing but subscribe, so the broker splits roles: `client` only `publish`es; a separate
|
|
418
|
+
connection (`opts.subscriber` or `client.duplicate()`) only subscribes. Both talk to the
|
|
419
|
+
same channel (`opts.channel`, default `'ayepi'`).
|
|
420
|
+
- **Local emit → fanout → back to subscribers.** When any instance emits, `@ayepi/core`
|
|
421
|
+
calls `broker.publish(message)`, which `client.publish(channel, message)`s into Redis.
|
|
422
|
+
Redis delivers that message to every connection subscribed to the channel — including the
|
|
423
|
+
emitting instance's own subscriber. Each instance's subscriber `'message'` handler checks
|
|
424
|
+
the channel matches, then dispatches the string to every registered local listener;
|
|
425
|
+
`@ayepi/core` matches it against locally connected WebSocket subscribers and delivers.
|
|
426
|
+
- **Serialization is opaque.** The broker carries `message` as an **opaque string** — it
|
|
427
|
+
does no JSON parsing, framing, or transformation. `@ayepi/core` is responsible for
|
|
428
|
+
serializing events into that string and deserializing on the receiving side. Channel
|
|
429
|
+
filtering is the only thing the broker inspects.
|
|
430
|
+
- **Lazy, once-only subscribe.** The subscriber is wired on the first `subscribe()` call
|
|
431
|
+
(idempotent — guarded by an internal flag) and the channel is subscribed exactly once.
|
|
432
|
+
ioredis **automatically re-subscribes** to its channels after a reconnect, so a network
|
|
433
|
+
blip doesn't silently stop delivery — the broker relies on this rather than resubscribing
|
|
434
|
+
itself.
|
|
435
|
+
- **Errors never throw.** `publish` swallows rejections into `onError` and resolves; a
|
|
436
|
+
throwing listener is caught per-listener and reported, so one bad listener can't starve the
|
|
437
|
+
others. Connection `'error'` events on both connections are forwarded to `onError` only if
|
|
438
|
+
you provided one.
|
|
439
|
+
|
|
440
|
+
## Gotchas / constraints
|
|
441
|
+
|
|
442
|
+
- **Best-effort, ephemeral delivery.** Redis pub/sub does **not** persist. An instance that
|
|
443
|
+
is down (or a subscriber that connects late) **misses** events published while it was
|
|
444
|
+
offline — there is no replay. This is the right model for live UI events; if you need
|
|
445
|
+
guaranteed delivery use a durable transport (Redis Streams, a queue).
|
|
446
|
+
- **All instances must agree on `channel`.** Mismatched channel names silently partition
|
|
447
|
+
your cluster — publishers and subscribers on different channels simply never see each
|
|
448
|
+
other (the broker filters by exact channel match).
|
|
449
|
+
- **Don't reuse a subscribing connection for commands.** If you pass `opts.subscriber`, give
|
|
450
|
+
it a dedicated connection. Once subscribed, ioredis puts it in subscriber mode where normal
|
|
451
|
+
commands are rejected. Likewise don't pass the same connection as both `client` and
|
|
452
|
+
`subscriber`.
|
|
453
|
+
- **You own the connections.** The broker never closes connections. Disconnect the clients
|
|
454
|
+
you created on shutdown; prefer an explicit `subscriber` over `duplicate()` when you need a
|
|
455
|
+
handle to close.
|
|
456
|
+
- **Silent by default.** Without `onError`, publish/subscribe failures and throwing listeners
|
|
457
|
+
are swallowed, and connection `'error'` handlers aren't attached. Pass `onError` in
|
|
458
|
+
production to surface problems.
|
|
459
|
+
- **No backpressure / dedup.** Every listener is called synchronously for every message; the
|
|
460
|
+
broker does no batching, ordering guarantees beyond Redis's own, or deduplication.
|
|
461
|
+
|
|
462
|
+
### Store & cache backends
|
|
463
|
+
|
|
464
|
+
- **Keys are namespaced by `prefix`.** Choose distinct prefixes per app/engine sharing one
|
|
465
|
+
Redis (store default `''`, cache default `'ayepi:cache:'`). The cache's `clear`/`invalidate`
|
|
466
|
+
only ever touch keys under its own prefix — but they `SCAN` the keyspace, so an overly broad
|
|
467
|
+
prefix means scanning more keys.
|
|
468
|
+
- **`setIfNotExists` and `increment` are atomic Redis-side, so they're fleet-safe.** `SET NX`
|
|
469
|
+
and `INCRBY` are single server-side operations — the claim/lease CAS and the open-work
|
|
470
|
+
counter are correct under concurrency across every instance. (The optional `PEXPIRE` after
|
|
471
|
+
`INCRBY` is a *separate* command, so increment-then-expire is not one atomic unit.)
|
|
472
|
+
- **Cache `clear`/`invalidate` are `SCAN`-based: O(keyspace) and non-atomic.** They walk the
|
|
473
|
+
whole `prefix*` keyspace in pages and delete in a second step, so on a large keyspace they
|
|
474
|
+
are not free and not a point-in-time snapshot — keys can be added, expire, or change between
|
|
475
|
+
the scan and the delete (mid-scan expirations are tolerated and skipped). Fine for occasional
|
|
476
|
+
invalidation; don't call them in a hot path.
|
|
477
|
+
- **TTL comes from `staleUntil`.** A cache entry's Redis `PX` lifetime is
|
|
478
|
+
`max(1, entry.staleUntil - now())`, so an entry self-evicts at its stale boundary regardless
|
|
479
|
+
of reads. The middleware still owns freshness vs `entry.expires`; the store only owns when
|
|
480
|
+
the key disappears.
|
|
481
|
+
- **Resilience is opt-in and silent by default.** Without `retry`, core's defaults apply;
|
|
482
|
+
without `onError`, a final (post-retry) failure simply propagates. Pass `onError` in
|
|
483
|
+
production to surface exhausted failures. `onError` must not rely on throwing — a throwing
|
|
484
|
+
`onError` is swallowed.
|
|
485
|
+
- **`@ayepi/work` and `@ayepi/cache` are optional, type-only peer deps.** They're imported only
|
|
486
|
+
as `import type`, so the package has no runtime dependency on them — install one only if you
|
|
487
|
+
use the corresponding backend. Using just `redisBroker` needs neither.
|
|
488
|
+
|
|
489
|
+
## See also
|
|
490
|
+
|
|
491
|
+
- [ayepi-core.md](./ayepi-core.md) — the `Broker` interface, `localBroker` (in-process
|
|
492
|
+
default), `server(api, [impl], { broker })`, and `app.emit(...)` event semantics; also the
|
|
493
|
+
`retry` / `RetryOptions` the store and cache use.
|
|
494
|
+
- [ayepi-work.md](./ayepi-work.md) — the distributed work engine and `createWork`, the
|
|
495
|
+
consumer of `redisStore` + `redisPubSub`.
|
|
496
|
+
- [ayepi-work-ports.md](./ayepi-work-ports.md) — the `Store` / `PubSub` / `Queue` port
|
|
497
|
+
contracts `redisStore` and `redisPubSub` implement.
|
|
498
|
+
- [ayepi-cache.md](./ayepi-cache.md) — the response-cache middleware, the `CacheStore` /
|
|
499
|
+
`CacheEntry` / `EntryMeta` types `redisCache` implements, and `cache.server(def, { store })`.
|
|
500
|
+
- [ayepi-aws.md](./ayepi-aws.md) — the SQS `Queue` (and S3 file store) that pairs with the
|
|
501
|
+
Redis store/pubsub to complete a distributed `@ayepi/work` backend.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ayepi/redis",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Redis backends for ayepi (ioredis) — pub/sub Broker, @ayepi/work Store + PubSub, and an @ayepi/cache store, all retry-wrapped",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"publishConfig": {
|
|
@@ -18,7 +18,8 @@
|
|
|
18
18
|
"type": "module",
|
|
19
19
|
"sideEffects": false,
|
|
20
20
|
"files": [
|
|
21
|
-
"dist"
|
|
21
|
+
"dist",
|
|
22
|
+
"ayepi-*.md"
|
|
22
23
|
],
|
|
23
24
|
"exports": {
|
|
24
25
|
".": {
|
|
@@ -38,9 +39,9 @@
|
|
|
38
39
|
},
|
|
39
40
|
"peerDependencies": {
|
|
40
41
|
"ioredis": "^5",
|
|
41
|
-
"@ayepi/work": "^0.
|
|
42
|
-
"@ayepi/core": "^0.
|
|
43
|
-
"@ayepi/cache": "^0.
|
|
42
|
+
"@ayepi/work": "^0.2.0",
|
|
43
|
+
"@ayepi/core": "^0.2.0",
|
|
44
|
+
"@ayepi/cache": "^0.2.0"
|
|
44
45
|
},
|
|
45
46
|
"peerDependenciesMeta": {
|
|
46
47
|
"@ayepi/work": {
|
|
@@ -58,9 +59,9 @@
|
|
|
58
59
|
"tsdown": "^0.12.0",
|
|
59
60
|
"vitest": "^2.1.8",
|
|
60
61
|
"zod": "^4.4.3",
|
|
61
|
-
"@ayepi/
|
|
62
|
-
"@ayepi/
|
|
63
|
-
"@ayepi/
|
|
62
|
+
"@ayepi/work": "0.2.0",
|
|
63
|
+
"@ayepi/core": "0.2.0",
|
|
64
|
+
"@ayepi/cache": "0.2.0"
|
|
64
65
|
},
|
|
65
66
|
"keywords": [
|
|
66
67
|
"ayepi",
|