@softistx/janus-webhooks-redis 0.0.0-stage → 0.12.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,339 @@
1
+ # Troubleshooting `@softistx/janus-webhooks-redis`
2
+
3
+ Installing `@softistx/janus*` fails with a 401 or 404? See
4
+ [the install entry](https://github.com/softistx/nxgt-janus/blob/develop/packages/janus/docs/troubleshooting.md#installing-softistx-packages).
5
+
6
+ Each entry is headed by the text you see: a compiler error, a message, or an
7
+ error `code`. Search this page for the words of your message.
8
+
9
+ This adapter **defines no error class**, and writes no warning. It throws two
10
+ kinds of error, both built in or `@softistx/janus`'s:
11
+
12
+ - `@softistx/janus`'s `StoreFailure`, with `operation` the queue method, when
13
+ Redis cannot answer — then `cause` is what Bun's Redis client threw — or
14
+ answers a key this adapter did not write, with no `cause`;
15
+ - a `TypeError`, before anything is sent, for a date, a `limit`, a
16
+ `failed.status` or an `event.data` the queue could not read back.
17
+
18
+ `@softistx/janus-webhooks` turns them into its own warnings —
19
+ `JANUS_WEBHOOK_QUEUE_FAILED`, and `JANUS_EVENT_FAILED` for an insert — which
20
+ are in
21
+ [its troubleshooting](https://github.com/softistx/nxgt-janus/blob/develop/packages/janus-webhooks/docs/troubleshooting.md).
22
+
23
+ ## Index
24
+
25
+ **Types**
26
+ - [`TS2345: Argument of type '…' is not assignable to parameter of type 'RedisConnection'.`](#ts2345-argument-of-type--is-not-assignable-to-parameter-of-type-redisconnection)
27
+ - [`TS2322: Type 'number' is not assignable to type 'string'.`](#ts2322-type-number-is-not-assignable-to-type-string)
28
+
29
+ **Runtime**
30
+ - [`STORE_FAILED` after about 31 seconds, caused by `Max reconnection attempts reached`](#store_failed-after-about-31-seconds-caused-by-max-reconnection-attempts-reached)
31
+ - [`STORE_FAILED`: `webhookQueue.<method>: the queue could not answer`](#store_failed-webhookqueuemethod-the-queue-could-not-answer)
32
+ - [`STORE_FAILED`: `webhookQueue.<method>: a reply that is not … — a key under the prefix this adapter did not write`](#store_failed-webhookqueuemethod-a-reply-that-is-not---a-key-under-the-prefix-this-adapter-did-not-write)
33
+ - [`TypeError: webhookQueue.<method>: <name> is a valid Date`](#typeerror-webhookqueuemethod-name-is-a-valid-date)
34
+ - [`TypeError: webhookQueue.insertDeliveries: event.data is a JSON object`](#typeerror-webhookqueueinsertdeliveries-eventdata-is-a-json-object)
35
+ - [`TypeError: webhookQueue.<method>: limit is a whole number of deliveries, 0 or more`](#typeerror-webhookqueuemethod-limit-is-a-whole-number-of-deliveries-0-or-more)
36
+ - [`TypeError: webhookQueue.scheduleRetry: failed.status is null or a whole number`](#typeerror-webhookqueuescheduleretry-failedstatus-is-null-or-a-whole-number)
37
+ - [Every claim of one endpoint fails: a delivery key that cannot be read](#every-claim-of-one-endpoint-fails-a-delivery-key-that-cannot-be-read)
38
+ - [`CROSSSLOT Keys in request don't hash to the same slot`, or `Script attempted to access a non local key in a cluster node`](#crossslot-keys-in-request-dont-hash-to-the-same-slot-or-script-attempted-to-access-a-non-local-key-in-a-cluster-node)
39
+ - [`TypeError: connectRedis: this URI is already connected with other options.`](#typeerror-connectredis-this-uri-is-already-connected-with-other-options)
40
+ - [Deliveries are lost when Redis restarts](#deliveries-are-lost-when-redis-restarts)
41
+ - [Deliveries disappear while Redis is up, under memory pressure](#deliveries-disappear-while-redis-is-up-under-memory-pressure)
42
+ - [Another application's endpoints are given up as `endpointRemoved`](#another-applications-endpoints-are-given-up-as-endpointremoved)
43
+
44
+ ---
45
+
46
+ ## Types
47
+
48
+ ### `TS2345: Argument of type '…' is not assignable to parameter of type 'RedisConnection'.`
49
+
50
+ `'…'` is `'string'` for a URL, `'RedisClient'` for Bun's client, or
51
+ `'Promise<RedisConnection>'` for `connectRedis(…)` not awaited.
52
+
53
+ **When:** you pass `createRedisWebhookQueue` a URL, a `RedisClient` you
54
+ created yourself, or the promise `connectRedis` answers.
55
+
56
+ **Why:** the adapter takes the connection `@softistx/redis`'s `connectRedis`
57
+ resolves to, and connects to nothing itself.
58
+
59
+ **Fix:**
60
+
61
+ ```ts
62
+ import { connectRedis } from '@softistx/redis';
63
+
64
+ const queue = createRedisWebhookQueue(await connectRedis(process.env.REDIS_URL ?? 'redis://localhost:6379'));
65
+ ```
66
+
67
+ ### `TS2322: Type 'number' is not assignable to type 'string'.`
68
+
69
+ **When:** `createRedisWebhookQueue(redis, { prefix: 1 })`, or a prefix read
70
+ from somewhere typed otherwise.
71
+
72
+ **Why:** the prefix starts every key's name.
73
+
74
+ **Fix:** a string, ending with a separator:
75
+
76
+ ```ts
77
+ createRedisWebhookQueue(redis, { prefix: 'clinic:webhooks:' });
78
+ ```
79
+
80
+ ---
81
+
82
+ ## Runtime
83
+
84
+ ### `STORE_FAILED` after about 31 seconds, caused by `Max reconnection attempts reached`
85
+
86
+ **When:** Redis is unreachable, and every flow that sends a user event —
87
+ `signUp`, `verifyEmail.confirm`, `delete` — hangs for about half a minute.
88
+
89
+ **Why:** the listener awaits the insert, and Bun's client queues commands
90
+ while it reconnects, failing them only once it gives up, after about 31 s
91
+ measured with `@softistx/janus-redis`.
92
+
93
+ **Fix:** turn the queue off for this connection, so an outage fails the
94
+ insert at once; the flow then answers, and `janus` reports the event as
95
+ `JANUS_EVENT_FAILED`:
96
+
97
+ ```ts
98
+ const redis = await connectRedis(process.env.REDIS_URL ?? 'redis://localhost:6379', {
99
+ enableOfflineQueue: false,
100
+ });
101
+ ```
102
+
103
+ ### `STORE_FAILED`: `webhookQueue.<method>: the queue could not answer`
104
+
105
+ `<method>` is one of `insertDeliveries`, `claimDeliveries`,
106
+ `claimOrphanedDeliveries`, `extendLease`, `scheduleRetry`, `deleteDelivery`.
107
+
108
+ **When:** any call, for as long as Redis cannot answer: unreachable, out of
109
+ memory (`OOM`), a replica that refuses writes (`READONLY`), a user without the
110
+ right (`NOPERM`), a key under the prefix of another type (`WRONGTYPE`). With
111
+ `enableOfflineQueue: false`, an outage's `cause` reads `Connection is closed
112
+ and offline queue is disabled`.
113
+
114
+ A script refused **half-way** — `NOPERM` on one key of several, `WRONGTYPE`
115
+ on a key someone else wrote — leaves what Redis already ran, since Redis
116
+ does not roll a script back. Only the insert undoes its own writes: nothing
117
+ of an insert refused is left. For the others:
118
+
119
+ - a claim keeps the deliveries it took before the refusal: each has spent
120
+ an attempt, and is claimed again once its lease ends — or at once, for
121
+ the one whose lease the refusal cut short;
122
+ - `deleteDelivery` removes the hash before its member, so a refusal between
123
+ the two leaves only a member — or, past both, an endpoint with nothing
124
+ due — which the next claim drops;
125
+ - `scheduleRetry` refused between its two writes has released its lease and
126
+ stored its failure, but the delivery is still due when the old lease
127
+ ends: it is retried then — earlier than scheduled.
128
+
129
+ **Why:** the adapter never turns a failure into an absence. An outage is not
130
+ "nothing due", which would hold every delivery back without a word.
131
+
132
+ **Fix:** nothing to do in code: `@softistx/janus-webhooks` warns once per outage
133
+ with `JANUS_WEBHOOK_QUEUE_FAILED`, backs off, and sends what waits once Redis
134
+ answers again. Log `cause` for Redis's reply:
135
+
136
+ ```ts
137
+ import { StoreFailure } from '@softistx/janus';
138
+
139
+ if (error instanceof StoreFailure) console.error(error.operation, error.cause);
140
+ ```
141
+
142
+ A Redis user restricted by ACL needs `+@scripting` and the commands the
143
+ scripts run (`HSET`, `HGET`, `HGETALL`, `HINCRBY`, `EXISTS`, `DEL`, `ZADD`,
144
+ `ZREM`, `ZRANGEBYSCORE`, `ZCARD`, `SADD`, `SREM`, `SISMEMBER`, `SMEMBERS`), on
145
+ the keys `~janus:webhooks:*`, or `~<your prefix>*`.
146
+
147
+ ### `STORE_FAILED`: `webhookQueue.<method>: a reply that is not … — a key under the prefix this adapter did not write`
148
+
149
+ `…` names what was expected: `a list`, `a delivery`, `a delivery claimed`,
150
+ `a hash with \`<field>\``, `a date in \`occurredAt\``, `a count in
151
+ \`attempts\``, `a status in \`status\``, `a user event type`, `0 or 1`, `a
152
+ count`.
153
+
154
+ **When:** Redis answered, but a key under the prefix holds something this
155
+ adapter did not write: a key set by hand, another application using the same
156
+ prefix, or a key written by another version — among them a
157
+ `user.secondFactorEnabled` or `user.secondFactorDisabled` delivery, written
158
+ by 0.2.0, claimed by a 0.1.x process, or a `user.recoveryCodesRegenerated` or
159
+ `user.recoveryCodeUsed` delivery, written by 0.3.0, claimed by a 0.2.x process,
160
+ or a `user.passwordChanged` or `user.emailChanged` delivery, written by 0.4.0,
161
+ claimed by a 0.3.x process, or a `user.newDeviceSignedIn` delivery, written
162
+ by 0.5.0, claimed by a 0.4.x process:
163
+ upgrade every process sharing the queue, with each endpoint's `types` limited
164
+ to the ones the older processes know until they all run the new version.
165
+
166
+ **Why:** a delivery it cannot read is a failure, never an absence — skipping
167
+ it would hold it back for ever, and answering it would send an event that
168
+ never happened. There is no `cause`: Redis did not fail. A claim —
169
+ `claimDeliveries` or `claimOrphanedDeliveries` — that meets such a key fails
170
+ every time it reaches it: see
171
+ [Every claim of one endpoint fails](#every-claim-of-one-endpoint-fails-a-delivery-key-that-cannot-be-read)
172
+ for how to find the key and remove it.
173
+
174
+ **Fix:** give the queue a prefix no other code writes under, and delete the
175
+ foreign keys:
176
+
177
+ ```ts
178
+ createRedisWebhookQueue(redis, { prefix: 'clinic:webhooks:' });
179
+ ```
180
+
181
+ ### `TypeError: webhookQueue.<method>: <name> is a valid Date`
182
+
183
+ `<name>` is `event.occurredAt` or `dueAt` for `insertDeliveries`, `now`,
184
+ `dueBefore` or `leaseUntil` for a claim, `until` for `extendLease`, `dueAt`
185
+ for `scheduleRetry`.
186
+
187
+ **When:** calling the queue yourself — in a test, a script, a wrapper — with
188
+ an Invalid Date: `new Date('soon')`, `new Date(NaN)`. `webhooks()` never
189
+ does: it refuses such an event itself, with `webhooks: an event's occurredAt
190
+ is a valid Date`.
191
+
192
+ **Why:** a date is stored as milliseconds, and an Invalid Date would be
193
+ stored as `NaN`, which no claim could read back: every claim of that
194
+ endpoint would then fail. It is refused before anything is sent. A date
195
+ before 1970 is a negative number, and is kept.
196
+
197
+ **Fix:** pass a valid `Date`:
198
+
199
+ ```ts
200
+ await queue.insertDeliveries(event, ['crm'], new Date());
201
+ ```
202
+
203
+ ### `TypeError: webhookQueue.insertDeliveries: event.data is a JSON object`
204
+
205
+ **When:** inserting a delivery of a generic event — one that carries a
206
+ `data` — through the queue directly, as an adapter's test or your own
207
+ wrapper does, with a `data` that is not a JSON object: an array, `null`, a
208
+ `bigint`, a cycle.
209
+ **Why:** the queue stores `data` as JSON in the delivery's hash and reads it
210
+ back at the claim: what it could not read back it refuses before it writes
211
+ anything. Through `webhooks()` the listener has already checked the event
212
+ (`webhooks: an event's data is an object`, `… is JSON — no BigInt, no cycle`),
213
+ so this reaches you only past it.
214
+ **Fix:** pass a plain JSON object as `data`, as the listener does.
215
+
216
+ ### `TypeError: webhookQueue.<method>: limit is a whole number of deliveries, 0 or more`
217
+
218
+ **When:** `claimDeliveries` or `claimOrphanedDeliveries` called with a
219
+ `limit` of `2.5`, `-1`, `Infinity` or `NaN`.
220
+
221
+ **Why:** a claim answers at most `limit` deliveries; a limit that is not a
222
+ whole number is none: no claim can take 2.5 deliveries. It is refused before
223
+ anything is sent.
224
+
225
+ **Fix:** a whole number: `Math.floor(free)`, where `free` is what you can
226
+ send at once.
227
+
228
+ ### `TypeError: webhookQueue.scheduleRetry: failed.status is null or a whole number`
229
+
230
+ **When:** `scheduleRetry` called with a `failed.status` of `503.5`, `-1` or
231
+ `NaN`.
232
+
233
+ **Why:** the status is stored as digits, and read back as a whole number;
234
+ anything else could not be. It is refused before anything is sent.
235
+
236
+ **Fix:** the response's `status`, or `null` when there was no response.
237
+
238
+ ### Every claim of one endpoint fails: a delivery key that cannot be read
239
+
240
+ The message is `STORE_FAILED`: `webhookQueue.claimDeliveries: a reply that
241
+ is not … — a key under the prefix this adapter did not write` — or
242
+ `webhookQueue.claimOrphanedDeliveries: …` for an endpoint no process is
243
+ configured with — again and again, with `JANUS_WEBHOOK_QUEUE_FAILED` from `@softistx/janus-webhooks`.
244
+
245
+ **When:** a delivery hash under the prefix was changed by hand, or written
246
+ by another version or another application, and a claim reached it. Every
247
+ claim that reaches it fails the same way, so the endpoint's other deliveries
248
+ wait behind it.
249
+
250
+ **Why:** a delivery the adapter cannot read is a failure, never skipped: a
251
+ queue that skipped what it cannot read would hold that delivery back without
252
+ a word. The claim is refused once Redis has answered it, so every delivery
253
+ it took — the broken one included — has spent an attempt, and is claimed
254
+ again once its lease ends: until the key is fixed, each lease period spends
255
+ one attempt of each of the endpoint's first deliveries. After about as many
256
+ lease lengths as the retry schedule has entries — seven by default — a
257
+ healthy delivery behind the broken key is given up as `retriesRanOut`.
258
+ Remove the broken key within a few lease lengths.
259
+
260
+ **Fix:** find the broken key, keep what it holds if you need it, and delete
261
+ it — the next claim drops its member from the due set and goes on:
262
+
263
+ ```sh
264
+ # The endpoint's deliveries, earliest first: the broken one is among the first.
265
+ redis-cli ZRANGE 'janus:webhooks:due:crm' 0 9 WITHSCORES
266
+ # Each is a hash: look for a field missing, or one that is not digits.
267
+ redis-cli HGETALL 'janus:webhooks:delivery:<event id>:<type>:crm'
268
+ redis-cli DEL 'janus:webhooks:delivery:<event id>:<type>:crm'
269
+ ```
270
+
271
+ With your own prefix in place of `janus:webhooks:`. A hash this adapter
272
+ writes holds `eventId`, `type`, `occurredAt` (milliseconds), `userId`,
273
+ `userType` — or, for a generic event, `data`, a JSON object, in their place —
274
+ `endpoint`, `attempts` (digits) and `lease`, and after a failed
275
+ attempt `status` (digits, or empty) and `error`.
276
+
277
+ ### `CROSSSLOT Keys in request don't hash to the same slot`, or `Script attempted to access a non local key in a cluster node`
278
+
279
+ The message is the `cause` of a `STORE_FAILED`.
280
+
281
+ **When:** the queue's Redis is a Redis Cluster.
282
+
283
+ **Why:** a script touches a delivery's key, its endpoint's sorted set and the
284
+ set of endpoints, which live in different hash slots. Cluster is not
285
+ supported.
286
+
287
+ **Fix:** give the queue a single Redis, or a primary with replicas.
288
+
289
+ ### `TypeError: connectRedis: this URI is already connected with other options.`
290
+
291
+ **When:** your application and the queue connect to the same Redis URI, one
292
+ with `enableOfflineQueue: false` and one without.
293
+
294
+ **Why:** `@softistx/redis` shares one client per URI, and one client has one set
295
+ of options.
296
+
297
+ **Fix:** pass the same options everywhere, or give the queue its own URI —
298
+ another database number is enough: `redis://localhost:6379/2`.
299
+
300
+ ### Deliveries are lost when Redis restarts
301
+
302
+ **When:** after a restart or a failover of Redis, endpoints never receive
303
+ some events, and neither `onGivingUp` nor `JANUS_WEBHOOK_GAVE_UP` says so.
304
+
305
+ **Why:** Redis kept them in memory only. The queue is durable only as far as
306
+ Redis is.
307
+
308
+ **Fix:** enable persistence on this Redis — AOF with `appendfsync everysec`,
309
+ which loses at most a second of inserts on a crash, or RDB snapshots, which
310
+ lose what came since the last one.
311
+
312
+ ### Deliveries disappear while Redis is up, under memory pressure
313
+
314
+ **When:** endpoints never receive some events, nothing reports them, and
315
+ Redis's `evicted_keys` (in `INFO stats`) grows.
316
+
317
+ **Why:** Redis evicts keys when it reaches `maxmemory`, under any policy but
318
+ `noeviction`. An evicted delivery hash is a delivery gone.
319
+
320
+ **Fix:** set `maxmemory-policy noeviction` on this Redis, or give the queue a
321
+ Redis of its own. With `noeviction`, a full Redis refuses the insert with
322
+ `OOM`: the event is reported as `JANUS_EVENT_FAILED` instead of lost in
323
+ silence.
324
+
325
+ ### Another application's endpoints are given up as `endpointRemoved`
326
+
327
+ **When:** `onGivingUp` receives deliveries with `reason.why`
328
+ `'endpointRemoved'` and `delivery.url` `null`, for endpoint ids this
329
+ application never had.
330
+
331
+ **Why:** two applications share one Redis and one prefix, so they share one
332
+ queue. Each claims the other's deliveries as orphans — endpoints it is not
333
+ configured with — and gives them up after `orphanGrace`.
334
+
335
+ **Fix:** a prefix per application:
336
+
337
+ ```ts
338
+ createRedisWebhookQueue(redis, { prefix: 'billing:webhooks:' });
339
+ ```
package/package.json CHANGED
@@ -1,6 +1,66 @@
1
1
  {
2
- "name": "@softistx/janus-webhooks-redis",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
2
+ "name": "@softistx/janus-webhooks-redis",
3
+ "version": "0.12.5",
4
+ "description": "The Redis queue for @softistx/janus-webhooks on @softistx/redis: deliveries that outlive the process, claimed atomically under a lease",
5
+ "license": "MIT",
6
+ "type": "module",
7
+ "main": "./dist/index.js",
8
+ "types": "./dist/index.d.ts",
9
+ "files": [
10
+ "dist",
11
+ "docs",
12
+ "README.md",
13
+ "package.json",
14
+ "LICENSE"
15
+ ],
16
+ "exports": {
17
+ ".": {
18
+ "types": "./dist/index.d.ts",
19
+ "import": "./dist/index.js",
20
+ "default": "./dist/index.js"
21
+ },
22
+ "./package.json": "./package.json"
23
+ },
24
+ "keywords": [
25
+ "webhooks",
26
+ "standard-webhooks",
27
+ "queue",
28
+ "redis",
29
+ "janus",
30
+ "typescript"
31
+ ],
32
+ "repository": {
33
+ "type": "git",
34
+ "url": "git+https://github.com/softistx/nxgt-janus.git",
35
+ "directory": "packages/janus-webhooks-redis"
36
+ },
37
+ "publishConfig": {
38
+ "registry": "https://registry.npmjs.org/",
39
+ "access": "public"
40
+ },
41
+ "scripts": {
42
+ "build": "bun run ../../build.ts",
43
+ "test": "bun test src",
44
+ "typecheck": "bun check",
45
+ "typecheck:tsc": "tsc --noEmit"
46
+ },
47
+ "nxgt": {
48
+ "entrypoints": [
49
+ "src/index.ts"
50
+ ]
51
+ },
52
+ "devDependencies": {
53
+ "@softistx/redis": "0.24.0",
54
+ "@softistx/janus": "^0.74.0",
55
+ "@softistx/janus-webhooks": "^0.13.5",
56
+ "@types/bun": "^1.4.4",
57
+ "redis-memory-server": "0.17.2",
58
+ "zod": "^4.6.7"
59
+ },
60
+ "peerDependencies": {
61
+ "@softistx/redis": ">=0.14.0 <1",
62
+ "@softistx/janus": "^0.74.0",
63
+ "@softistx/janus-webhooks": "^0.13.5",
64
+ "typescript": "^6.0.3 || ^7.0.0"
65
+ }
66
+ }