@bumail/queue 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +44 -8
- package/dist/chunks/index-b4xvg3xn.js +23 -0
- package/dist/chunks/index-b4xvg3xn.js.map +10 -0
- package/dist/chunks/index-vg971acr.js.map +2 -2
- package/dist/contract/queue-store.d.ts +8 -2
- package/dist/contract/queue-store.d.ts.map +1 -1
- package/dist/dsn/build.d.ts +7 -3
- package/dist/dsn/build.d.ts.map +1 -1
- package/dist/errors.d.ts +5 -2
- package/dist/errors.d.ts.map +1 -1
- package/dist/index.js +117 -33
- package/dist/index.js.map +8 -7
- package/dist/masked.d.ts +8 -0
- package/dist/masked.d.ts.map +1 -0
- package/dist/postgres/connect.d.ts +6 -0
- package/dist/postgres/connect.d.ts.map +1 -1
- package/dist/postgres/index.js +49 -28
- package/dist/postgres/index.js.map +7 -6
- package/dist/postgres/isolation.d.ts +6 -0
- package/dist/postgres/isolation.d.ts.map +1 -0
- package/dist/postgres/schema.d.ts.map +1 -1
- package/dist/postgres/store.d.ts.map +1 -1
- package/dist/queue/deliver.d.ts +10 -1
- package/dist/queue/deliver.d.ts.map +1 -1
- package/dist/queue/lease.d.ts +41 -0
- package/dist/queue/lease.d.ts.map +1 -0
- package/dist/queue/notify.d.ts +3 -2
- package/dist/queue/notify.d.ts.map +1 -1
- package/dist/queue/worker.d.ts.map +1 -1
- package/dist/redis/connect.d.ts +27 -0
- package/dist/redis/connect.d.ts.map +1 -0
- package/dist/redis/index.d.ts +3 -0
- package/dist/redis/index.d.ts.map +1 -0
- package/dist/redis/index.js +438 -0
- package/dist/redis/index.js.map +14 -0
- package/dist/redis/items.d.ts +13 -0
- package/dist/redis/items.d.ts.map +1 -0
- package/dist/redis/options.d.ts +47 -0
- package/dist/redis/options.d.ts.map +1 -0
- package/dist/redis/run.d.ts +12 -0
- package/dist/redis/run.d.ts.map +1 -0
- package/dist/redis/scripts.d.ts +60 -0
- package/dist/redis/scripts.d.ts.map +1 -0
- package/dist/redis/store.d.ts +30 -0
- package/dist/redis/store.d.ts.map +1 -0
- package/docs/README.md +1 -1
- package/docs/guide.md +271 -16
- package/docs/roadmap.md +49 -7
- package/docs/troubleshooting.md +303 -16
- package/package.json +9 -3
package/docs/troubleshooting.md
CHANGED
|
@@ -34,9 +34,14 @@ parts shown as … vary.
|
|
|
34
34
|
|
|
35
35
|
- [`QueueError: The lease on … was lost before its outcome was recorded; another worker will try it again`](#queueerror-the-lease-on--was-lost-before-its-outcome-was-recorded-another-worker-will-try-it-again)
|
|
36
36
|
- [`QueueError: The lease on … was lost while it was delivered`](#queueerror-the-lease-on--was-lost-while-it-was-delivered)
|
|
37
|
+
- [`QueueError: The lease on … expired before its outcome was recorded, and the item is gone: lost to another worker that finished it, the message then sent twice, or cancelled`](#queueerror-the-lease-on--expired-before-its-outcome-was-recorded-and-the-item-is-gone-lost-to-another-worker-that-finished-it-the-message-then-sent-twice-or-cancelled)
|
|
38
|
+
- [`QueueError: The lease on … was lost while it was delivered, and the item is gone: finished by the worker that took it, the message then sent twice, or cancelled`](#queueerror-the-lease-on--was-lost-while-it-was-delivered-and-the-item-is-gone-finished-by-the-worker-that-took-it-the-message-then-sent-twice-or-cancelled)
|
|
39
|
+
- [`QueueError: The message of … is unreadable: the store holds the item but not its message, so every pending recipient failed`](#queueerror-the-message-of--is-unreadable-the-store-holds-the-item-but-not-its-message-so-every-pending-recipient-failed)
|
|
37
40
|
- [`SmtpError: sendMail(): …` (`INVALID_OPTION`), with recipients deferred as `4.3.5`](#smtperror-sendmail--invalid_option-with-recipients-deferred-as-435)
|
|
38
41
|
- [`SQLiteError: database is locked`, or another store error, during a delivery](#sqliteerror-database-is-locked-or-another-store-error-during-a-delivery)
|
|
39
42
|
- [`PostgresError: …`, or a connection error, during a delivery](#postgreserror--or-a-connection-error-during-a-delivery)
|
|
43
|
+
- [`RedisError: …` during a delivery: `Connection closed`, `OOM command not allowed …`, `READONLY …`](#rediserror--during-a-delivery-connection-closed-oom-command-not-allowed--readonly-)
|
|
44
|
+
- [`PostgresError: could not serialize access due to concurrent update` (SQLSTATE `40001`)](#postgreserror-could-not-serialize-access-due-to-concurrent-update-sqlstate-40001)
|
|
40
45
|
|
|
41
46
|
**The `bun:sqlite` store** (`@bumail/queue/sqlite`)
|
|
42
47
|
|
|
@@ -55,6 +60,16 @@ parts shown as … vary.
|
|
|
55
60
|
- [`QueueError: The PostgreSQL queue cannot be set up: …`](#queueerror-the-postgresql-queue-cannot-be-set-up-)
|
|
56
61
|
- [`QueueError: The database is at schema version …, newer than this store's …`](#queueerror-the-database-is-at-schema-version--newer-than-this-stores-), and [`QueueError: The queue store is closed`](#queueerror-the-queue-store-is-closed), as for `bun:sqlite`
|
|
57
62
|
|
|
63
|
+
**The Redis store** (`@bumail/queue/redis`)
|
|
64
|
+
|
|
65
|
+
- [`QueueError: A Redis queue store needs client, a Bun.RedisClient, or url, a redis:// URL`](#queueerror-a-redis-queue-store-needs-client-a-bunredisclient-or-url-a-redis-url)
|
|
66
|
+
- [`QueueError: A Redis queue store takes client or url, not both`](#queueerror-a-redis-queue-store-takes-client-or-url-not-both)
|
|
67
|
+
- [`QueueError: keyPrefix must be lowercase letters, digits, '_', ':', '.' and '-', starting with a letter, at most 40 characters, not …`](#queueerror-keyprefix-must-be-lowercase-letters-digits-_---and---starting-with-a-letter-at-most-40-characters-not-)
|
|
68
|
+
- [`QueueError: The URL in url cannot be opened: …`](#queueerror-the-url-in-url-cannot-be-opened-)
|
|
69
|
+
- [`QueueError: The Redis queue cannot be set up: …`](#queueerror-the-redis-queue-cannot-be-set-up-)
|
|
70
|
+
- [`QueueError: The key …schema does not hold a layout version: is the prefix another application's?`](#queueerror-the-key-schema-does-not-hold-a-layout-version-is-the-prefix-another-applications)
|
|
71
|
+
- [`QueueError: The database is at schema version …, newer than this store's …`](#queueerror-the-database-is-at-schema-version--newer-than-this-stores-), and [`QueueError: The queue store is closed`](#queueerror-the-queue-store-is-closed), as for `bun:sqlite`
|
|
72
|
+
|
|
58
73
|
**Listing** (`list`)
|
|
59
74
|
|
|
60
75
|
- [`QueueError: limit must be an integer from 1 to 1000, not …`](#queueerror-limit-must-be-an-integer-from-1-to-1000-not-)
|
|
@@ -63,6 +78,10 @@ parts shown as … vary.
|
|
|
63
78
|
**A store's own checks** (a store of your own, called directly)
|
|
64
79
|
|
|
65
80
|
- [`QueueError: … must be a finite number`, `owner must be a non-empty string`, and the others](#a-stores-own-checks)
|
|
81
|
+
- [`QueueError: A reply is an object with a text`](#queueerror-a-reply-is-an-object-with-a-text)
|
|
82
|
+
- [`QueueError: reply.status must be a string`](#queueerror-replystatus-must-be-a-string)
|
|
83
|
+
- [`QueueError: reply.host must be a string`](#queueerror-replyhost-must-be-a-string)
|
|
84
|
+
- [`QueueError: … holds a NUL or a lone surrogate, which a store cannot keep`](#queueerror--holds-a-nul-or-a-lone-surrogate-which-a-store-cannot-keep)
|
|
66
85
|
|
|
67
86
|
**Delivery**
|
|
68
87
|
|
|
@@ -70,6 +89,7 @@ parts shown as … vary.
|
|
|
70
89
|
- [Every delivery fails at once with `5.7.1`, or with a reply naming your IP or EHLO name](#every-delivery-fails-at-once-with-571-or-with-a-reply-naming-your-ip-or-ehlo-name)
|
|
71
90
|
- [A recipient fails with `5.1.3`, `The address is not one SMTP can carry`](#a-recipient-fails-with-513-the-address-is-not-one-smtp-can-carry)
|
|
72
91
|
- [Every recipient fails with `5.1.7`, `The sender's address is not one SMTP can carry`](#every-recipient-fails-with-517-the-senders-address-is-not-one-smtp-can-carry)
|
|
92
|
+
- [Every pending recipient fails with `5.3.0`, `Message unreadable: the queue store holds the item but not its message`](#every-pending-recipient-fails-with-530-message-unreadable-the-queue-store-holds-the-item-but-not-its-message)
|
|
73
93
|
- [`retryNow` returns `false` for an item that is in the queue](#retrynow-returns-false-for-an-item-that-is-in-the-queue)
|
|
74
94
|
|
|
75
95
|
## Enqueuing
|
|
@@ -238,8 +258,9 @@ or `… must be a number from … to …, not …`. A number option out of its r
|
|
|
238
258
|
|
|
239
259
|
### `QueueError: owner must be a non-empty string`
|
|
240
260
|
|
|
241
|
-
**Code:** `INVALID`, also as
|
|
242
|
-
|
|
261
|
+
**Code:** `INVALID`, also as
|
|
262
|
+
[`owner holds a NUL or a lone surrogate, which a store cannot keep`](#queueerror--holds-a-nul-or-a-lone-surrogate-which-a-store-cannot-keep).
|
|
263
|
+
Leave `owner` out for a random one, or give
|
|
243
264
|
each worker its own name.
|
|
244
265
|
|
|
245
266
|
## The `error` event
|
|
@@ -256,8 +277,77 @@ the process stalled longer than that, or the store could not be reached.
|
|
|
256
277
|
|
|
257
278
|
### `QueueError: The lease on … was lost while it was delivered`
|
|
258
279
|
|
|
259
|
-
**Code:** `LEASE_LOST`. A renewal
|
|
260
|
-
|
|
280
|
+
**Code:** `LEASE_LOST`. A renewal was refused while the item was still in
|
|
281
|
+
the store, held by another worker or by none (another worker took it and
|
|
282
|
+
let go of it): the same cause as above, seen sooner. A renewal that finds the item gone
|
|
283
|
+
says nothing: the outcome, once the sessions end, tells a cancel from a
|
|
284
|
+
lease lost (below).
|
|
285
|
+
|
|
286
|
+
### `QueueError: The lease on … expired before its outcome was recorded, and the item is gone: lost to another worker that finished it, the message then sent twice, or cancelled`
|
|
287
|
+
|
|
288
|
+
**Code:** `LEASE_LOST`.
|
|
289
|
+
**When:** a delivery ran past its lease's expiry — no renewal reached the
|
|
290
|
+
store in time — and, when it tried to record its outcome, the item was no
|
|
291
|
+
longer in the store. No `delivered`, `deferred` or `failed` event and no
|
|
292
|
+
DSN come from this attempt.
|
|
293
|
+
**Why:** past its expiry, another worker may claim the item, deliver it
|
|
294
|
+
and drop it; but a `cancel` drops it too, and the store keeps no record of
|
|
295
|
+
which happened. The worker cannot tell them apart, so it says both rather
|
|
296
|
+
than report outcomes nobody recorded. When another worker took it, the
|
|
297
|
+
recipients both reached got the message twice; that worker's events tell
|
|
298
|
+
what it delivered.
|
|
299
|
+
**Fix:** as for the lease lost above: a longer `leaseMs`, or find what
|
|
300
|
+
stalled the process — an event loop blocked, a store out of reach. A
|
|
301
|
+
cancel under a lease that still held is told as before: its outcomes on
|
|
302
|
+
the events, with no error and no DSN.
|
|
303
|
+
|
|
304
|
+
When the instances' clocks are out of step, one whose clock runs ahead
|
|
305
|
+
can claim the item while this worker's own clock says the lease still
|
|
306
|
+
holds. If it finishes and drops the item between two of this worker's
|
|
307
|
+
renewals, no renewal saw it taken and the expiry has not passed by this
|
|
308
|
+
clock: the attempt reads as a cancel, its outcomes told on the events
|
|
309
|
+
with no error. Keep the machines on NTP; the 10-minute `leaseMs` leaves
|
|
310
|
+
room for seconds of skew.
|
|
311
|
+
|
|
312
|
+
### `QueueError: The lease on … was lost while it was delivered, and the item is gone: finished by the worker that took it, the message then sent twice, or cancelled`
|
|
313
|
+
|
|
314
|
+
**Code:** `LEASE_LOST`, after
|
|
315
|
+
[`The lease on … was lost while it was delivered`](#queueerror-the-lease-on--was-lost-while-it-was-delivered).
|
|
316
|
+
**When:** a renewal was refused while the item was still in the store,
|
|
317
|
+
held by another worker or by none, and when this worker tried to record
|
|
318
|
+
its outcome the item was gone. No outcome event and no DSN come from this
|
|
319
|
+
attempt.
|
|
320
|
+
**Why:** another worker claimed the item once this one's lease had
|
|
321
|
+
expired by its own clock — this worker stalled, or the instances' clocks
|
|
322
|
+
disagree (an instance whose clock runs ahead sees leases expire early) —
|
|
323
|
+
and has most likely finished and dropped it, the recipients both reached
|
|
324
|
+
getting the message twice. A `cancel` after the renewal drops it too, and
|
|
325
|
+
the store keeps no record of which happened, so the message names both.
|
|
326
|
+
**Fix:** a longer `leaseMs`, find what stalled the process, and keep the
|
|
327
|
+
machines on NTP.
|
|
328
|
+
|
|
329
|
+
### `QueueError: The message of … is unreadable: the store holds the item but not its message, so every pending recipient failed`
|
|
330
|
+
|
|
331
|
+
**Code:** `MESSAGE_UNREADABLE`, with the item's `id`.
|
|
332
|
+
**When:** a worker claimed an item, the store gave no message for it
|
|
333
|
+
while it still held the item, and the failures were then recorded: it
|
|
334
|
+
comes after the record, before the `failed` events. With the lease lost
|
|
335
|
+
before the record, only `LEASE_LOST` is told, and the next worker tells
|
|
336
|
+
this; with the item cancelled meanwhile, nothing is.
|
|
337
|
+
**Why:** the message is gone and the item is not: on Redis, its
|
|
338
|
+
`<prefix>message:<id>` key evicted under a `maxmemory-policy` other than
|
|
339
|
+
`noeviction`, or deleted; on any store, a message row or key removed by
|
|
340
|
+
hand, or a store of your own that loses it. Nothing can be sent, so
|
|
341
|
+
every recipient still pending fails as `5.3.0` (see
|
|
342
|
+
[below](#every-pending-recipient-fails-with-530-message-unreadable-the-queue-store-holds-the-item-but-not-its-message)),
|
|
343
|
+
the failure DSN goes to the sender without the original, and the item
|
|
344
|
+
leaves the queue. Before this, such an item stayed leased and was
|
|
345
|
+
claimed again at every lease, forever, with no event.
|
|
346
|
+
**Fix:** find what removed the message. On Redis, set
|
|
347
|
+
`maxmemory-policy noeviction` and size `maxmemory` for the queue (the
|
|
348
|
+
guide's [Redis](guide.md#redis)); never delete a queue's keys or rows by
|
|
349
|
+
hand, `cancel(id)` drops an item with its message. The sender has the
|
|
350
|
+
DSN and may send again.
|
|
261
351
|
|
|
262
352
|
### `SmtpError: sendMail(): …` (`INVALID_OPTION`), with recipients deferred as `4.3.5`
|
|
263
353
|
|
|
@@ -298,6 +388,51 @@ its lease expires — its recipients may then get the message twice.
|
|
|
298
388
|
too many connections, give each instance's client a smaller `max` (see
|
|
299
389
|
the guide's Pool size) or raise `max_connections`.
|
|
300
390
|
|
|
391
|
+
### `RedisError: …` during a delivery: `Connection closed`, `OOM command not allowed …`, `READONLY …`
|
|
392
|
+
|
|
393
|
+
**When:** the `error` event, with the item's `id`, while it is delivered,
|
|
394
|
+
on `@bumail/queue/redis`. A call of yours — `enqueue`, `list`, `get`,
|
|
395
|
+
`retryNow`, `cancel` — rejects with the same `RedisError` for the same
|
|
396
|
+
reasons: an `enqueue` refused with `OOM …` kept nothing.
|
|
397
|
+
**Why:** a lease renewal failed. `Connection closed`: Redis restarted,
|
|
398
|
+
or the connection dropped, while the command was on its way; a command
|
|
399
|
+
sent while the client is reconnecting waits instead. `Max reconnection
|
|
400
|
+
attempts reached` or `Connection has failed`: Redis is out of reach, and
|
|
401
|
+
`Bun.RedisClient` gave up reconnecting (20 tries by default). `OOM
|
|
402
|
+
command not allowed when used memory > 'maxmemory'`: Redis is full and
|
|
403
|
+
refuses writes. `READONLY You can't write against a read only replica`:
|
|
404
|
+
the client reaches a replica, or a primary that was demoted by a
|
|
405
|
+
failover. As on PostgreSQL, the next renewal tries again, only a lease
|
|
406
|
+
taken by another instance stops them, a claim that fails rejects
|
|
407
|
+
`deliverDue()` or is told on `error` under `start()`, and an outcome
|
|
408
|
+
that cannot be recorded leaves the item to be claimed again once its
|
|
409
|
+
lease expires — its recipients may then get the message twice.
|
|
410
|
+
**Fix:** check Redis is up and that the URL names the primary (through
|
|
411
|
+
your provider's endpoint or Sentinel's, which follows a failover). For
|
|
412
|
+
`OOM`, give Redis more `maxmemory`, or drain the queue; keep
|
|
413
|
+
`maxmemory-policy noeviction` (the guide's [Redis](guide.md#redis)), or
|
|
414
|
+
Redis drops queue items to make room instead of refusing.
|
|
415
|
+
|
|
416
|
+
### `PostgresError: could not serialize access due to concurrent update` (SQLSTATE `40001`)
|
|
417
|
+
|
|
418
|
+
**When:** on `@bumail/queue/postgres` 0.3.0 or earlier — a claim, an
|
|
419
|
+
outcome, an `enqueue` with `limits.maxItems` or a migration rejecting, or
|
|
420
|
+
told on `error`, with `could not serialize access due to concurrent
|
|
421
|
+
update` or `… due to read/write dependencies among transactions`, while
|
|
422
|
+
several workers run — or `limits.maxItems` letting more items in than
|
|
423
|
+
its limit.
|
|
424
|
+
**Why:** the client's sessions default to `repeatable read` or
|
|
425
|
+
`serializable` (`connection: { default_transaction_isolation }`, or the
|
|
426
|
+
role's or database's own default). The claim's `FOR UPDATE SKIP LOCKED`
|
|
427
|
+
and the `maxItems` advisory lock need `READ COMMITTED`: under a stricter
|
|
428
|
+
level, a row another worker changed since the transaction's snapshot
|
|
429
|
+
fails, and the count taken after the lock reads a snapshot from before
|
|
430
|
+
it.
|
|
431
|
+
**Fix:** upgrade: every transaction the store writes in now starts with
|
|
432
|
+
`SET TRANSACTION ISOLATION LEVEL READ COMMITTED`, whatever the client's
|
|
433
|
+
default. The error from a transaction of your own on the same client is
|
|
434
|
+
yours to retry.
|
|
435
|
+
|
|
301
436
|
## The `bun:sqlite` store
|
|
302
437
|
|
|
303
438
|
### `QueueError: A SQLite queue store needs a directory`
|
|
@@ -315,7 +450,9 @@ should read the queue.
|
|
|
315
450
|
### `QueueError: The database is at schema version …, newer than this store's …`
|
|
316
451
|
|
|
317
452
|
**Code:** `INVALID`. A newer version of the package wrote the database
|
|
318
|
-
(`queue.sqlite`,
|
|
453
|
+
(`queue.sqlite`, on PostgreSQL the prefix's tables, on Redis the
|
|
454
|
+
prefix's keys, whose layout version is `<prefix>schema`): upgrade this
|
|
455
|
+
one.
|
|
319
456
|
|
|
320
457
|
### `QueueError: busyTimeout must be an integer of at least 0, not …`
|
|
321
458
|
|
|
@@ -324,8 +461,8 @@ should read the queue.
|
|
|
324
461
|
### `QueueError: The queue store is closed`
|
|
325
462
|
|
|
326
463
|
**Code:** `CLOSED`. The store was used after `close()`: call
|
|
327
|
-
`queue.stop()` first, then `store.close()`. The PostgreSQL
|
|
328
|
-
same.
|
|
464
|
+
`queue.stop()` first, then `store.close()`. The PostgreSQL and Redis
|
|
465
|
+
stores say the same.
|
|
329
466
|
|
|
330
467
|
## The PostgreSQL store
|
|
331
468
|
|
|
@@ -356,7 +493,8 @@ PostgresQueueStore.open({ sql: new Bun.SQL({ url, max: 10 }) });
|
|
|
356
493
|
**Why:** the prefix is written into every statement, never bound as a
|
|
357
494
|
value, so anything that is not a plain name is refused — and 40
|
|
358
495
|
characters keep the longest name built on it within PostgreSQL's 63.
|
|
359
|
-
**Fix:** `tablePrefix: 'mail_queue_'
|
|
496
|
+
**Fix:** `tablePrefix: 'mail_queue_'`, a prefix of the queue's own: never
|
|
497
|
+
one another package's or application's tables use. To put the tables in another
|
|
360
498
|
schema, set the connection's `search_path` rather than a dotted prefix.
|
|
361
499
|
|
|
362
500
|
### `QueueError: The URL in sql cannot be opened: …`
|
|
@@ -366,7 +504,10 @@ schema, set the connection's `search_path` rather than a dotted prefix.
|
|
|
366
504
|
connecting, such as a `sslmode` it does not know. The rest is Bun's
|
|
367
505
|
reason (`The argument 'sslmode' must be one of: disable, allow, prefer,
|
|
368
506
|
require, verify-ca, verify-full. Received '…'`); the URL is never
|
|
369
|
-
repeated, and the password is masked should the reason name it
|
|
507
|
+
repeated, and the password is masked should the reason name it: always
|
|
508
|
+
where a URL holds it (`:…@`), and alone, elsewhere, when it is 4
|
|
509
|
+
characters or more — masking `x` everywhere would turn "execute" into
|
|
510
|
+
"e…ecute".
|
|
370
511
|
**Fix:** correct the parameter: `?sslmode=require`, or `verify-full` to
|
|
371
512
|
check the server's certificate.
|
|
372
513
|
|
|
@@ -376,7 +517,10 @@ check the server's certificate.
|
|
|
376
517
|
**When:** the first call on a store, or `migrate()`, when the tables
|
|
377
518
|
cannot be made or brought up to date. The rest of the message is the
|
|
378
519
|
database's: `Failed to connect`, `password authentication failed for
|
|
379
|
-
user "…"`, `permission denied for schema public`.
|
|
520
|
+
user "…"`, `permission denied for schema public`. The password — the
|
|
521
|
+
URL's, or a given `Bun.SQL`'s — is masked should the reason name it, as
|
|
522
|
+
for `The URL in sql cannot be opened`: a `…` where you expect a role or
|
|
523
|
+
a database name is one that equals the password.
|
|
380
524
|
**Why:** the database is out of reach, the credentials are wrong, or the
|
|
381
525
|
tables are missing or behind and the role may not create them: a role
|
|
382
526
|
without `CREATE` can use tables that are current, never make them.
|
|
@@ -387,6 +531,77 @@ their narrower role (the guide's [Migrations](guide.md#migrations)); or
|
|
|
387
531
|
give the role `CREATE` on the schema. The next call tries again: nothing is left half made,
|
|
388
532
|
since a migration is one transaction.
|
|
389
533
|
|
|
534
|
+
## The Redis store
|
|
535
|
+
|
|
536
|
+
### `QueueError: A Redis queue store needs client, a Bun.RedisClient, or url, a redis:// URL`
|
|
537
|
+
|
|
538
|
+
**Code:** `INVALID`.
|
|
539
|
+
**When:** `RedisQueueStore.open` with neither `client` nor `url`, a
|
|
540
|
+
`url` that is not a URL of a scheme `Bun.RedisClient` takes (`redis://`,
|
|
541
|
+
`rediss://`, `valkey://`, `valkeys://`, `redis+tls://`, `redis+unix://`,
|
|
542
|
+
`redis+tls+unix://`), or a `client` without `send`, `getBuffer` and
|
|
543
|
+
`close`. The message never repeats the URL, which may hold a password.
|
|
544
|
+
**Fix:**
|
|
545
|
+
|
|
546
|
+
```ts
|
|
547
|
+
RedisQueueStore.open({ url: 'rediss://bumail:secret@cache.internal:6380/0' });
|
|
548
|
+
// or a client of yours, configured as you need:
|
|
549
|
+
RedisQueueStore.open({ client: new Bun.RedisClient(url, { connectionTimeout: 5000 }) });
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
### `QueueError: A Redis queue store takes client or url, not both`
|
|
553
|
+
|
|
554
|
+
**Code:** `INVALID`. Give one: `client` for a client you configure and
|
|
555
|
+
close, `url` for one the store opens and closes.
|
|
556
|
+
|
|
557
|
+
### `QueueError: keyPrefix must be lowercase letters, digits, '_', ':', '.' and '-', starting with a letter, at most 40 characters, not …`
|
|
558
|
+
|
|
559
|
+
**Code:** `INVALID`.
|
|
560
|
+
**Why:** the prefix starts every key the store writes. Anything Redis or
|
|
561
|
+
a `SCAN MATCH` reads specially is refused: `{` (a Cluster hash tag), `*`,
|
|
562
|
+
`?`, `[` and `\` (glob characters), spaces and control characters.
|
|
563
|
+
**Fix:** `keyPrefix: 'mail:queue:'`. Give each queue in one Redis
|
|
564
|
+
database its own prefix.
|
|
565
|
+
|
|
566
|
+
### `QueueError: The URL in url cannot be opened: …`
|
|
567
|
+
|
|
568
|
+
**Code:** `INVALID`.
|
|
569
|
+
**When:** `RedisQueueStore.open` with a URL `Bun.RedisClient` refuses
|
|
570
|
+
before connecting, such as a database that is not a number (`Invalid
|
|
571
|
+
database number in Redis URL: "…"`). The URL is never repeated, and the
|
|
572
|
+
password is masked should the reason name it, as on PostgreSQL: always
|
|
573
|
+
in a URL, and alone from 4 characters.
|
|
574
|
+
**Fix:** correct the URL: the database is the path, `/0` to `/15` on a
|
|
575
|
+
default Redis.
|
|
576
|
+
|
|
577
|
+
### `QueueError: The Redis queue cannot be set up: …`
|
|
578
|
+
|
|
579
|
+
**Code:** `INVALID`.
|
|
580
|
+
**When:** the first call on a store, which reads the layout version of
|
|
581
|
+
its keys (`<prefix>schema`), writing it on a new queue. The rest of the
|
|
582
|
+
message is Redis's or Bun's: `Max reconnection attempts reached` (out of
|
|
583
|
+
reach), `WRONGPASS invalid username-password pair or user is disabled.`,
|
|
584
|
+
`ERR DB index is out of range`, `NOPERM …` (an ACL user without
|
|
585
|
+
`EVALSHA`, `EVAL` or the keys). The URL's password is masked should the
|
|
586
|
+
reason repeat it; a given client's is not known to the store.
|
|
587
|
+
**Why:** the server is out of reach, the credentials or the database are
|
|
588
|
+
wrong, or the user may not run scripts. With `Bun.RedisClient`'s
|
|
589
|
+
defaults an unreachable server takes about half a minute of reconnecting
|
|
590
|
+
before the call fails.
|
|
591
|
+
**Fix:** check the URL and that Redis answers. An ACL user needs the
|
|
592
|
+
commands the store sends and those its scripts call, on the keys under
|
|
593
|
+
its prefix, and nothing else (the guide's [Redis](guide.md#redis) has
|
|
594
|
+
the `ACL SETUSER` line). The next call tries again.
|
|
595
|
+
|
|
596
|
+
### `QueueError: The key …schema does not hold a layout version: is the prefix another application's?`
|
|
597
|
+
|
|
598
|
+
**Code:** `INVALID`.
|
|
599
|
+
**When:** the first call on a store whose `<prefix>schema` key holds
|
|
600
|
+
something other than a whole number of at least 1.
|
|
601
|
+
**Why:** the store keeps its layout version there; anything else means
|
|
602
|
+
another application, or a hand, wrote under the same prefix.
|
|
603
|
+
**Fix:** give the queue a `keyPrefix` of its own. Nothing was written.
|
|
604
|
+
|
|
390
605
|
## Listing
|
|
391
606
|
|
|
392
607
|
### `QueueError: limit must be an integer from 1 to 1000, not …`
|
|
@@ -425,15 +640,71 @@ never gives it. Each message names what is wrong:
|
|
|
425
640
|
- `QueueError: delayNotified must be true or false`
|
|
426
641
|
- `QueueError: recipients must be an array of { address, status:
|
|
427
642
|
delivered, deferred or failed }`
|
|
428
|
-
- `QueueError: A reply is an object with a text
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
`
|
|
433
|
-
them; the queue's own replies never hold one
|
|
643
|
+
- [`QueueError: A reply is an object with a text`](#queueerror-a-reply-is-an-object-with-a-text),
|
|
644
|
+
[`reply.status must be a string`](#queueerror-replystatus-must-be-a-string),
|
|
645
|
+
[`reply.host must be a string`](#queueerror-replyhost-must-be-a-string)
|
|
646
|
+
- [`QueueError: … holds a NUL or a lone surrogate, which a store cannot
|
|
647
|
+
keep`](#queueerror--holds-a-nul-or-a-lone-surrogate-which-a-store-cannot-keep)
|
|
434
648
|
|
|
435
649
|
A list's `offset` and `limit` are checked as under [Listing](#listing).
|
|
436
650
|
|
|
651
|
+
### `QueueError: A reply is an object with a text`
|
|
652
|
+
|
|
653
|
+
**Code:** `INVALID`.
|
|
654
|
+
**When:** `complete` called directly with a recipient whose `reply` is
|
|
655
|
+
not an object, or has no string `text`.
|
|
656
|
+
**Why:** a reply is a `Diagnostic`, `{ code?, status?, text, host? }`:
|
|
657
|
+
`text` is what a DSN and the events show, so every store needs it. The
|
|
658
|
+
queue's own outcomes always have one.
|
|
659
|
+
**Fix:** give the text, even for a reply with no code:
|
|
660
|
+
|
|
661
|
+
```ts
|
|
662
|
+
await store.complete(id, owner, {
|
|
663
|
+
...result,
|
|
664
|
+
recipients: [{ address, status: 'deferred', reply: { status: '4.4.1', text: 'No answer' } }],
|
|
665
|
+
});
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
### `QueueError: reply.status must be a string`
|
|
669
|
+
|
|
670
|
+
**Code:** `INVALID`.
|
|
671
|
+
**When:** `complete` called directly with a `reply.status` that is
|
|
672
|
+
neither left out nor a string — often the reply code given as a number.
|
|
673
|
+
**Why:** `status` is the enhanced status code, `x.y.z` (RFC 3463), kept
|
|
674
|
+
as text; the reply code goes in `code`.
|
|
675
|
+
**Fix:** `reply: { code: 451, status: '4.3.0', text: 'Try later' }`, or
|
|
676
|
+
leave `status` out.
|
|
677
|
+
|
|
678
|
+
### `QueueError: reply.host must be a string`
|
|
679
|
+
|
|
680
|
+
**Code:** `INVALID`.
|
|
681
|
+
**When:** `complete` called directly with a `reply.host` that is neither
|
|
682
|
+
left out nor a string, such as an address object from a socket.
|
|
683
|
+
**Why:** `host` is the name or address the attempt was for, kept as text.
|
|
684
|
+
**Fix:** give it as text — `host: 'mx1.example.com'` or
|
|
685
|
+
`host: '192.0.2.1'` — or leave it out.
|
|
686
|
+
|
|
687
|
+
### `QueueError: … holds a NUL or a lone surrogate, which a store cannot keep`
|
|
688
|
+
|
|
689
|
+
**Code:** `INVALID`. The … is `from`, `A recipient`, `owner`,
|
|
690
|
+
`reply.text`, `reply.status` or `reply.host`.
|
|
691
|
+
**When:** `add`, `complete`, or any call that takes an `owner`, called
|
|
692
|
+
directly with a text holding U+0000 or half of a surrogate pair — a string cut in the
|
|
693
|
+
middle of an emoji, say. `createQueue` refuses such an `owner` the same
|
|
694
|
+
way.
|
|
695
|
+
**Why:** PostgreSQL cannot keep a NUL in text, and no UTF-8 encoding can
|
|
696
|
+
carry a lone surrogate (`Bun.RedisClient` would write one as U+FFFD
|
|
697
|
+
without a word), so every store refuses them alike — whichever
|
|
698
|
+
store you give it, the same text is kept or refused. The queue's own
|
|
699
|
+
replies never hold one: it cuts a reply's text only between whole
|
|
700
|
+
characters.
|
|
701
|
+
**Fix:** drop the NULs and cut text on whole characters before you give
|
|
702
|
+
it to the store:
|
|
703
|
+
|
|
704
|
+
```ts
|
|
705
|
+
const clean = text.replaceAll('\0', '').toWellFormed();
|
|
706
|
+
```
|
|
707
|
+
|
|
437
708
|
## Delivery
|
|
438
709
|
|
|
439
710
|
### A recipient stays `deferred` with `4.4.1` or `4.4.2`
|
|
@@ -481,6 +752,22 @@ other.
|
|
|
481
752
|
`@bumail/smtp/client`'s `isMailbox` (or let it be `''`) before a store of
|
|
482
753
|
your own keeps it.
|
|
483
754
|
|
|
755
|
+
### Every pending recipient fails with `5.3.0`, `Message unreadable: the queue store holds the item but not its message`
|
|
756
|
+
|
|
757
|
+
**When:** the store held the item but gave no message for it, and the
|
|
758
|
+
failures were recorded: the `error` event says
|
|
759
|
+
[`The message of … is unreadable`](#queueerror-the-message-of--is-unreadable-the-store-holds-the-item-but-not-its-message-so-every-pending-recipient-failed)
|
|
760
|
+
for the same item, just before these `failed` events. An item cancelled
|
|
761
|
+
before the record tells no `5.3.0` failure; with the lease lost, this
|
|
762
|
+
attempt tells none and the next worker tells them.
|
|
763
|
+
**Why:** no session can send a message that cannot be read, and an
|
|
764
|
+
attempt left without an outcome would be claimed again at every lease,
|
|
765
|
+
forever, keeping its `maxItems` place. Recipients already delivered keep
|
|
766
|
+
their state; the others fail in one attempt, with one failure DSN —
|
|
767
|
+
none for a message from `<>` — that returns nothing of the original.
|
|
768
|
+
**Fix:** as for that error: find what removed the message, and on Redis
|
|
769
|
+
keep `maxmemory-policy noeviction`.
|
|
770
|
+
|
|
484
771
|
### `retryNow` returns `false` for an item that is in the queue
|
|
485
772
|
|
|
486
773
|
**Why:** a worker is delivering it: its lease is held. That attempt's
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bumail/queue",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "The outbound mail queue of a mail server: every recipient's state, retries with back-off, delivery status notifications (RFC 3464), leases for several workers, and a memory, a bun:sqlite
|
|
3
|
+
"version": "0.4.0",
|
|
4
|
+
"description": "The outbound mail queue of a mail server: every recipient's state, retries with back-off, delivery status notifications (RFC 3464), leases for several workers, and a memory, a bun:sqlite, a PostgreSQL (Bun.sql) and a Redis (Bun.redis) store, delivering through @bumail/smtp/client",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./dist/index.js",
|
|
@@ -34,6 +34,11 @@
|
|
|
34
34
|
"import": "./dist/postgres/index.js",
|
|
35
35
|
"default": "./dist/postgres/index.js"
|
|
36
36
|
},
|
|
37
|
+
"./redis": {
|
|
38
|
+
"types": "./dist/redis/index.d.ts",
|
|
39
|
+
"import": "./dist/redis/index.js",
|
|
40
|
+
"default": "./dist/redis/index.js"
|
|
41
|
+
},
|
|
37
42
|
"./package.json": "./package.json"
|
|
38
43
|
},
|
|
39
44
|
"repository": {
|
|
@@ -55,7 +60,8 @@
|
|
|
55
60
|
"src/index.ts",
|
|
56
61
|
"src/memory/index.ts",
|
|
57
62
|
"src/sqlite/index.ts",
|
|
58
|
-
"src/postgres/index.ts"
|
|
63
|
+
"src/postgres/index.ts",
|
|
64
|
+
"src/redis/index.ts"
|
|
59
65
|
]
|
|
60
66
|
},
|
|
61
67
|
"devDependencies": {
|