queen-mq 1.2.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +71 -18
- package/client-v2/Queen.js +45 -13
- package/client-v2/README.md +26 -11
- package/client-v2/admin/Admin.js +0 -47
- package/client-v2/builders/QueueBuilder.js +33 -10
- package/client-v2/builders/TimerBuilder.js +2 -2
- package/client-v2/builders/TransactionBuilder.js +35 -9
- package/client-v2/consumer/ConsumerManager.js +67 -46
- package/client-v2/ephemeral/Ephemeral.js +7 -9
- package/client-v2/kv/Kv.js +4 -4
- package/client-v2/kv/expiry.js +1 -1
- package/client-v2/streams/Stream.js +8 -3
- package/client-v2/streams/helpers/rateLimiter.js +4 -4
- package/client-v2/streams/operators/GateOperator.js +9 -2
- package/client-v2/streams/operators/ReduceOperator.js +2 -2
- package/client-v2/streams/operators/WindowSessionOperator.js +3 -3
- package/client-v2/streams/runtime/Runner.js +111 -54
- package/client-v2/streams/runtime/cycle.js +4 -4
- package/client-v2/streams/runtime/register.js +1 -1
- package/client-v2/utils/conflation.js +0 -6
- package/client-v2/utils/consumerGroup.js +54 -0
- package/package.json +5 -8
- package/test-v2/_kvtimers.js +12 -13
- package/test-v2/ackwindow.js +12 -192
- package/test-v2/bootstrap.js +2 -2
- package/test-v2/conflation-unit/conflationWire.test.js +0 -12
- package/test-v2/consume.js +41 -1
- package/test-v2/consumer-unit/handlerError.test.js +161 -0
- package/test-v2/docs.js +5 -4
- package/test-v2/http-unit/pushStatus.test.js +142 -0
- package/test-v2/http-unit/renew.test.js +96 -0
- package/test-v2/kv-unit/timerWire.test.js +1 -1
- package/test-v2/kv-unit/txnWire.test.js +101 -2
- package/test-v2/kv.js +6 -5
- package/test-v2/pop.js +28 -1
- package/test-v2/retention.js +28 -7
- package/test-v2/run.js +50 -114
- package/test-v2/runner-unit/fatalExit.test.js +64 -0
- package/test-v2/semantics.js +20 -34
- package/test-v2/stream/_helpers.js +10 -19
- package/test-v2/stream/cron.js +1 -1
- package/test-v2/stream/gate.js +54 -0
- package/test-v2/stream/index.js +2 -0
- package/test-v2/stream/tumbling.js +13 -9
- package/test-v2/streams-unit/ack.test.js +61 -0
- package/test-v2/streams-unit/cycle.test.js +1 -1
- package/test-v2/streams-unit/e2e.test.js +6 -10
- package/test-v2/streams-unit/gate.test.js +189 -0
- package/test-v2/timers.js +6 -5
- package/test-v2/transaction.js +169 -0
- package/test-v2/watermark.js +38 -176
- package/test-v2/maintenance.js +0 -277
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
[](https://www.npmjs.com/package/queen-mq)
|
|
8
8
|
[](LICENSE.md)
|
|
9
|
-
[](https://nodejs.org/)
|
|
10
10
|
|
|
11
11
|
[Quick Start](#quick-start) • [Complete Guide](client-v2/README.md) • [Examples](#examples) • [API Reference](#api-reference)
|
|
12
12
|
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
|
|
17
17
|
## What is Queen MQ?
|
|
18
18
|
|
|
19
|
-
Queen MQ is a
|
|
19
|
+
Queen MQ is a partitioned message queue broker that keeps its state in its own replicated log, with a powerful feature set:
|
|
20
20
|
|
|
21
21
|
- **FIFO Partitions** - Unlimited ordered partitions within queues
|
|
22
22
|
- **Consumer Groups** - Kafka-style consumer groups for scalability
|
|
@@ -40,7 +40,7 @@ This client provides a fluent, promise-based API for Node.js applications.
|
|
|
40
40
|
npm install queen-mq
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-
**Requirements:** Node.js
|
|
43
|
+
**Requirements:** Node.js 24+
|
|
44
44
|
|
|
45
45
|
---
|
|
46
46
|
|
|
@@ -133,20 +133,21 @@ await queen.queue('emails')
|
|
|
133
133
|
|
|
134
134
|
### Subscription Modes
|
|
135
135
|
|
|
136
|
-
Control whether consumer groups process historical messages
|
|
136
|
+
Control whether consumer groups process historical messages. A group's mode is fixed the first
|
|
137
|
+
time the group pops a queue; unset, it is the broker's `DEFAULT_SUBSCRIPTION_MODE`, which is `new`.
|
|
137
138
|
|
|
138
139
|
```javascript
|
|
139
|
-
// Default:
|
|
140
|
-
await queen.queue('events')
|
|
141
|
-
.group('batch-analytics')
|
|
142
|
-
.consume(async (message) => { /* all messages */ })
|
|
143
|
-
|
|
144
|
-
// Skip history, only new messages
|
|
140
|
+
// Default ('new'): start where the group first pops, skip what is already there
|
|
145
141
|
await queen.queue('events')
|
|
146
142
|
.group('realtime-monitor')
|
|
147
|
-
.subscriptionMode('new')
|
|
148
143
|
.consume(async (message) => { /* new only */ })
|
|
149
144
|
|
|
145
|
+
// Process ALL messages, including the backlog
|
|
146
|
+
await queen.queue('events')
|
|
147
|
+
.group('batch-analytics')
|
|
148
|
+
.subscriptionMode('all')
|
|
149
|
+
.consume(async (message) => { /* all messages */ })
|
|
150
|
+
|
|
150
151
|
// Start from specific timestamp
|
|
151
152
|
await queen.queue('events')
|
|
152
153
|
.group('replay')
|
|
@@ -278,6 +279,51 @@ await queen.queue('tasks')
|
|
|
278
279
|
})
|
|
279
280
|
```
|
|
280
281
|
|
|
282
|
+
**When the handler throws**, the consumer nacks what it was given (the message, or the whole batch)
|
|
283
|
+
and keeps consuming. The broker redelivers it, and moves it to the dead letter queue once the queue's
|
|
284
|
+
`retryLimit` is spent. With `.each()`, the rest of the popped batch is dropped after a nack: the
|
|
285
|
+
broker redelivers it too.
|
|
286
|
+
|
|
287
|
+
This is the same with `.autoAck(false)`. That setting hands the *success* path to your handler (it
|
|
288
|
+
acks), not the failure path: a handler that threw never got to settle its messages. A message your
|
|
289
|
+
handler acked before throwing is not affected: it is already settled, so the broker refuses
|
|
290
|
+
its nack.
|
|
291
|
+
|
|
292
|
+
To handle failures yourself, add `.onError(async (message, error) => { ... })`. The error then never
|
|
293
|
+
reaches the consumer, nothing is nacked for you, and the message is yours to ack, nack, or leave
|
|
294
|
+
until its lease expires. To stop consuming on an error, abort the `signal` you passed to
|
|
295
|
+
`consume(handler, { signal })` from inside `onError`.
|
|
296
|
+
|
|
297
|
+
```javascript
|
|
298
|
+
// autoAck(false): the handler acks. A throw before the ack is nacked and retried.
|
|
299
|
+
await queen.queue('tasks')
|
|
300
|
+
.group('workers')
|
|
301
|
+
.autoAck(false)
|
|
302
|
+
.each() // one message per call; without it the handler gets the popped array
|
|
303
|
+
.consume(async (message) => {
|
|
304
|
+
await processTask(message.data)
|
|
305
|
+
await queen.ack(message, true)
|
|
306
|
+
})
|
|
307
|
+
|
|
308
|
+
// Your own failure policy: nack, then stop consuming.
|
|
309
|
+
const stop = new AbortController()
|
|
310
|
+
await queen.queue('tasks')
|
|
311
|
+
.group('workers')
|
|
312
|
+
.autoAck(false)
|
|
313
|
+
.each()
|
|
314
|
+
.consume(async (message) => {
|
|
315
|
+
await processTask(message.data)
|
|
316
|
+
await queen.ack(message, true)
|
|
317
|
+
}, { signal: stop.signal })
|
|
318
|
+
.onError(async (message, error) => {
|
|
319
|
+
await queen.ack(message, false, { error: error.message })
|
|
320
|
+
stop.abort()
|
|
321
|
+
})
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
Earlier versions stopped the consumer on a throw under `.autoAck(false)`: `consume()` rejected after
|
|
325
|
+
the first failure, and the message stayed leased until its lease expired.
|
|
326
|
+
|
|
281
327
|
### Pop Messages (On-Demand Processing)
|
|
282
328
|
|
|
283
329
|
```javascript
|
|
@@ -494,9 +540,9 @@ Both surfaces are **always there**. There is nothing to enable: kv and timers ar
|
|
|
494
540
|
broker the way push and pop are, on every cell that runs it. There is no capability to probe and
|
|
495
541
|
no 404 that means "this cell does not have the feature" — a 404 from these routes is a bug.
|
|
496
542
|
|
|
497
|
-
What an operator can still do is **pause** them, with the runtime kill
|
|
498
|
-
|
|
499
|
-
|
|
543
|
+
What an operator can still do is **pause** them, with the broker's runtime kill switches
|
|
544
|
+
(`kv_enabled`, `timers_schedule_enabled`, `timers_fire_enabled`) — a lever pulled live during an
|
|
545
|
+
incident and expected to be pulled back.
|
|
500
546
|
A paused surface answers `503` with `Retry-After` and `error: 'kv_disabled'` / `'timers_disabled'`,
|
|
501
547
|
which this client retries like any other 5xx. Inside a transaction it is a `403` on the `kv` or
|
|
502
548
|
`timers` rider instead, so a bundle holding messages does not spin forever on a paused cell.
|
|
@@ -581,7 +627,7 @@ the retry budget.
|
|
|
581
627
|
|
|
582
628
|
Durations that can be sub-second are in **milliseconds** (`delayMs`), the ones that cannot are in
|
|
583
629
|
**seconds** (`ttlSeconds`). Only relative delays exist, because there is one clock and it is the
|
|
584
|
-
|
|
630
|
+
broker's. A delay in the past is legal and fires on the first cycle.
|
|
585
631
|
|
|
586
632
|
`deliverAt` is **"not before"**, never "exactly at".
|
|
587
633
|
|
|
@@ -768,6 +814,11 @@ await queen.ack(message, false, { error: 'reason' })
|
|
|
768
814
|
await queen.ack([msg1, msg2], true) // Batch ack
|
|
769
815
|
```
|
|
770
816
|
|
|
817
|
+
An ack is judged against the lease of the consumer group the message was popped under. `queen.ack()`
|
|
818
|
+
and `transaction().ack()` take that group from the message (`message.consumerGroup`, which every pop
|
|
819
|
+
returns) unless you name one: `{ group }` on `queen.ack()`, `{ consumerGroup }` or `{ group }` on a
|
|
820
|
+
transaction. A batch ack carries a single group, so `queen.ack()` throws on a batch that mixes groups.
|
|
821
|
+
|
|
771
822
|
### Transactions
|
|
772
823
|
|
|
773
824
|
```javascript
|
|
@@ -816,11 +867,14 @@ await queen.timer(q).list({ limit: 50, after }) // {rows, trunc
|
|
|
816
867
|
### Lease Renewal
|
|
817
868
|
|
|
818
869
|
```javascript
|
|
819
|
-
await queen.renew(message)
|
|
820
|
-
await queen.renew([msg1, msg2, msg3])
|
|
870
|
+
await queen.renew(message) // {leaseId, success, newExpiresAt, renewed}
|
|
871
|
+
await queen.renew([msg1, msg2, msg3]) // one result per distinct lease
|
|
821
872
|
await queen.queue('q').renewLease(true, 60000).consume(async (msg) => { /* auto-renew */ })
|
|
822
873
|
```
|
|
823
874
|
|
|
875
|
+
`success: false` (with an `error`) means nothing was renewed: the lease expired, an ack or nack
|
|
876
|
+
already released it, or it never existed. The messages it covered may already be redelivered.
|
|
877
|
+
|
|
824
878
|
### Buffering
|
|
825
879
|
|
|
826
880
|
```javascript
|
|
@@ -970,4 +1024,3 @@ const messages: Message<OrderData>[] = await queen.queue('orders').pop()
|
|
|
970
1024
|
## License
|
|
971
1025
|
|
|
972
1026
|
Apache 2.0 - See [LICENSE.md](../LICENSE.md)
|
|
973
|
-
|
package/client-v2/Queen.js
CHANGED
|
@@ -15,6 +15,7 @@ import { StreamBuilder } from './stream/StreamBuilder.js'
|
|
|
15
15
|
import { StreamConsumer } from './stream/StreamConsumer.js'
|
|
16
16
|
import { Admin } from './admin/Admin.js'
|
|
17
17
|
import { popAutopilotDisabledByEnv } from './utils/autopilot.js'
|
|
18
|
+
import { consumerGroupOf, sharedConsumerGroupOf } from './utils/consumerGroup.js'
|
|
18
19
|
import { CLIENT_DEFAULTS } from './utils/defaults.js'
|
|
19
20
|
import { validateUrl, validateUrls } from './utils/validation.js'
|
|
20
21
|
import * as logger from './utils/logger.js'
|
|
@@ -28,9 +29,9 @@ const CLOSE_FLUSH_DEADLINE_MILLIS = 30000
|
|
|
28
29
|
// Both /api/v1/ack and /api/v1/ack/batch respond with a top-level JSON array,
|
|
29
30
|
// one item per acknowledgment in request order:
|
|
30
31
|
// [{index, transactionId, success, error, queueName, partitionName, leaseReleased, dlq}]
|
|
31
|
-
//
|
|
32
|
-
//
|
|
33
|
-
//
|
|
32
|
+
// A rejected ack/nack (e.g. "Invalid or expired lease") still arrives as HTTP
|
|
33
|
+
// 200 with success=false on the item, so the per-item flag is the only signal
|
|
34
|
+
// that the broker accepted it.
|
|
34
35
|
|
|
35
36
|
function normalizeAckItem(item, index) {
|
|
36
37
|
if (item === null || typeof item !== 'object') {
|
|
@@ -61,6 +62,30 @@ function parseAckResults(result, expected) {
|
|
|
61
62
|
throw new Error('Unexpected ack response format: missing per-item result array')
|
|
62
63
|
}
|
|
63
64
|
|
|
65
|
+
// POST /api/v1/lease/:leaseId/extend answers HTTP 200 whether or not anything
|
|
66
|
+
// was renewed, so `success` in the body is the only signal:
|
|
67
|
+
// 1.x and 2.x brokers: {leaseId, success, renewed, newExpiresAt, expiresAt, lease_expires_at}
|
|
68
|
+
// the C++ broker: [{index, leaseId, success, error, expiresAt}]
|
|
69
|
+
// success:false means the lease is gone (expired, released by an ack or nack,
|
|
70
|
+
// or never existed): nothing was extended, and the messages it covered can
|
|
71
|
+
// already be on their way to another consumer.
|
|
72
|
+
function parseRenewResult(result, leaseId) {
|
|
73
|
+
const item = Array.isArray(result) ? result[0] : result
|
|
74
|
+
if (item === null || typeof item !== 'object') {
|
|
75
|
+
return { leaseId, success: false, newExpiresAt: null, error: 'Unexpected lease renewal response' }
|
|
76
|
+
}
|
|
77
|
+
const newExpiresAt = item.newExpiresAt ?? item.expiresAt ?? item.lease_expires_at ?? null
|
|
78
|
+
const success = typeof item.success === 'boolean' ? item.success : newExpiresAt !== null
|
|
79
|
+
const outcome = { leaseId, success, newExpiresAt }
|
|
80
|
+
if (typeof item.renewed === 'number') outcome.renewed = item.renewed
|
|
81
|
+
if (!success) {
|
|
82
|
+
outcome.error = typeof item.error === 'string' && item.error.length > 0
|
|
83
|
+
? item.error
|
|
84
|
+
: 'Lease not renewed: it expired, was released by an ack or nack, or does not exist'
|
|
85
|
+
}
|
|
86
|
+
return outcome
|
|
87
|
+
}
|
|
88
|
+
|
|
64
89
|
export class Queen {
|
|
65
90
|
#httpClient
|
|
66
91
|
#bufferManager
|
|
@@ -438,11 +463,15 @@ export class Queen {
|
|
|
438
463
|
})
|
|
439
464
|
}
|
|
440
465
|
|
|
466
|
+
// The group the messages were popped under unless the caller names one:
|
|
467
|
+
// the lease every ack has to match belongs to it (utils/consumerGroup.js).
|
|
468
|
+
const consumerGroup = context.group || sharedConsumerGroupOf(message)
|
|
469
|
+
|
|
441
470
|
// Call batch ack endpoint
|
|
442
471
|
try {
|
|
443
472
|
const result = await this.#httpClient.post('/api/v1/ack/batch', {
|
|
444
473
|
acknowledgments,
|
|
445
|
-
consumerGroup
|
|
474
|
+
consumerGroup
|
|
446
475
|
})
|
|
447
476
|
|
|
448
477
|
const results = parseAckResults(result, acknowledgments.length)
|
|
@@ -487,7 +516,9 @@ export class Queen {
|
|
|
487
516
|
partitionId,
|
|
488
517
|
status: statusStr,
|
|
489
518
|
error: context.error || null,
|
|
490
|
-
|
|
519
|
+
// The group the message was popped under unless the caller names one
|
|
520
|
+
// (utils/consumerGroup.js).
|
|
521
|
+
consumerGroup: context.group || consumerGroupOf(message)
|
|
491
522
|
}
|
|
492
523
|
|
|
493
524
|
if (leaseId) body.leaseId = leaseId
|
|
@@ -529,8 +560,8 @@ export class Queen {
|
|
|
529
560
|
}
|
|
530
561
|
|
|
531
562
|
// Dedupe: with v4 multi-partition pop, all messages in one batch share
|
|
532
|
-
// the same leaseId (one
|
|
533
|
-
//
|
|
563
|
+
// the same leaseId (one renew call extends the lease of every claimed
|
|
564
|
+
// partition). Without this, callers passing the full
|
|
534
565
|
// messages array would issue N redundant identical HTTP calls.
|
|
535
566
|
leaseIds = [...new Set(leaseIds)]
|
|
536
567
|
|
|
@@ -545,12 +576,13 @@ export class Queen {
|
|
|
545
576
|
for (const leaseId of leaseIds) {
|
|
546
577
|
try {
|
|
547
578
|
const result = await this.#httpClient.post(`/api/v1/lease/${leaseId}/extend`, {})
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
}
|
|
553
|
-
|
|
579
|
+
const outcome = parseRenewResult(result, leaseId)
|
|
580
|
+
results.push(outcome)
|
|
581
|
+
if (outcome.success) {
|
|
582
|
+
logger.log('Queen.renew', { leaseId, success: true, renewed: outcome.renewed })
|
|
583
|
+
} else {
|
|
584
|
+
logger.error('Queen.renew', { leaseId, error: outcome.error })
|
|
585
|
+
}
|
|
554
586
|
} catch (error) {
|
|
555
587
|
results.push({ leaseId, success: false, error: error.message })
|
|
556
588
|
logger.error('Queen.renew', { leaseId, error: error.message })
|
package/client-v2/README.md
CHANGED
|
@@ -60,7 +60,7 @@ When connecting to multiple Queen servers, you can choose how requests are distr
|
|
|
60
60
|
|
|
61
61
|
#### Affinity Mode (Recommended for Production)
|
|
62
62
|
|
|
63
|
-
Uses consistent hashing with virtual nodes to route consumer groups to the same backend server.
|
|
63
|
+
Uses consistent hashing with virtual nodes to route consumer groups to the same backend server.
|
|
64
64
|
|
|
65
65
|
```javascript
|
|
66
66
|
const queen = new Queen({
|
|
@@ -444,28 +444,31 @@ When a consumer group first subscribes to a queue, should it process **all histo
|
|
|
444
444
|
- Join a stream at a specific point in time
|
|
445
445
|
- Skip historical data for new analytics consumers
|
|
446
446
|
|
|
447
|
-
### Default Behavior (
|
|
447
|
+
### Default Behavior (New Messages)
|
|
448
448
|
|
|
449
|
-
By default, consumer
|
|
449
|
+
By default, a consumer group starts where it first pops the queue and skips the messages that were
|
|
450
|
+
already there. That default is the broker's `DEFAULT_SUBSCRIPTION_MODE`, which is `new`:
|
|
450
451
|
|
|
451
452
|
```javascript
|
|
452
|
-
// This consumer group gets
|
|
453
|
+
// This consumer group gets only messages that arrive after it first pops
|
|
453
454
|
await queen
|
|
454
455
|
.queue('events')
|
|
455
456
|
.group('new-analytics')
|
|
457
|
+
.each()
|
|
456
458
|
.consume(async (message) => {
|
|
457
459
|
console.log('Processing:', message.data)
|
|
458
460
|
})
|
|
459
461
|
```
|
|
460
462
|
|
|
461
|
-
|
|
463
|
+
To process the backlog too, ask for it with `.subscriptionMode('all')`, or start the broker with
|
|
464
|
+
`DEFAULT_SUBSCRIPTION_MODE=all` to make that the default for every new group:
|
|
462
465
|
```bash
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
./bin/queen-server
|
|
466
|
+
export DEFAULT_SUBSCRIPTION_MODE="all"
|
|
467
|
+
./bin/queen
|
|
466
468
|
```
|
|
467
469
|
|
|
468
|
-
|
|
470
|
+
The mode is stored with the group the first time it pops; changing the default later does not move
|
|
471
|
+
an existing group.
|
|
469
472
|
|
|
470
473
|
### Subscription Mode: 'new'
|
|
471
474
|
|
|
@@ -1004,8 +1007,9 @@ const message = messages[0]
|
|
|
1004
1007
|
|
|
1005
1008
|
// Start long processing
|
|
1006
1009
|
const timer = setInterval(async () => {
|
|
1007
|
-
await queen.renew(message) // Extend lease
|
|
1008
|
-
console.log('Lease renewed
|
|
1010
|
+
const { success, newExpiresAt, error } = await queen.renew(message) // Extend lease
|
|
1011
|
+
if (success) console.log('Lease renewed until', newExpiresAt)
|
|
1012
|
+
else console.warn('Lease lost:', error) // expired or already released: it will be redelivered
|
|
1009
1013
|
}, 30000) // Every 30 seconds
|
|
1010
1014
|
|
|
1011
1015
|
try {
|
|
@@ -1432,6 +1436,16 @@ await queen.queue('reports').consume(async (msg) => {
|
|
|
1432
1436
|
|
|
1433
1437
|
Sometimes you need more control over what happens when messages succeed or fail.
|
|
1434
1438
|
|
|
1439
|
+
### What a Throw Does
|
|
1440
|
+
|
|
1441
|
+
Without callbacks, a handler that throws gets what it was given nacked (the message with `.each()`,
|
|
1442
|
+
otherwise the whole popped batch), and the consumer keeps going. The broker redelivers it, and moves
|
|
1443
|
+
it to the DLQ once the queue's `retryLimit` is spent. That holds with `.autoAck(false)` too: there
|
|
1444
|
+
your handler acks on success, but a handler that threw never got to settle anything.
|
|
1445
|
+
|
|
1446
|
+
With `.onError()` the failure is yours: nothing is nacked on your behalf. Ack, nack or DLQ the
|
|
1447
|
+
message in the callback, or it stays leased until its lease expires.
|
|
1448
|
+
|
|
1435
1449
|
### Success Callback
|
|
1436
1450
|
|
|
1437
1451
|
```javascript
|
|
@@ -1457,6 +1471,7 @@ await queen
|
|
|
1457
1471
|
.onError(async (message, error) => {
|
|
1458
1472
|
console.error('Failed:', error.message)
|
|
1459
1473
|
// Log to external service, send alert, etc.
|
|
1474
|
+
await queen.ack(message, false) // with onError, nothing is nacked for you
|
|
1460
1475
|
})
|
|
1461
1476
|
```
|
|
1462
1477
|
|
package/client-v2/admin/Admin.js
CHANGED
|
@@ -359,44 +359,6 @@ export class Admin {
|
|
|
359
359
|
return this.#httpClient.get('/metrics')
|
|
360
360
|
}
|
|
361
361
|
|
|
362
|
-
/**
|
|
363
|
-
* Get push maintenance mode status
|
|
364
|
-
* @returns {Promise<object>}
|
|
365
|
-
*/
|
|
366
|
-
async getMaintenanceMode() {
|
|
367
|
-
logger.log('Admin.getMaintenanceMode', {})
|
|
368
|
-
return this.#httpClient.get('/api/v1/system/maintenance')
|
|
369
|
-
}
|
|
370
|
-
|
|
371
|
-
/**
|
|
372
|
-
* Set push maintenance mode
|
|
373
|
-
* @param {boolean} enabled - Enable or disable maintenance mode
|
|
374
|
-
* @returns {Promise<object>}
|
|
375
|
-
*/
|
|
376
|
-
async setMaintenanceMode(enabled) {
|
|
377
|
-
logger.log('Admin.setMaintenanceMode', { enabled })
|
|
378
|
-
return this.#httpClient.post('/api/v1/system/maintenance', { enabled })
|
|
379
|
-
}
|
|
380
|
-
|
|
381
|
-
/**
|
|
382
|
-
* Get pop maintenance mode status
|
|
383
|
-
* @returns {Promise<object>}
|
|
384
|
-
*/
|
|
385
|
-
async getPopMaintenanceMode() {
|
|
386
|
-
logger.log('Admin.getPopMaintenanceMode', {})
|
|
387
|
-
return this.#httpClient.get('/api/v1/system/maintenance/pop')
|
|
388
|
-
}
|
|
389
|
-
|
|
390
|
-
/**
|
|
391
|
-
* Set pop maintenance mode
|
|
392
|
-
* @param {boolean} enabled - Enable or disable pop maintenance mode
|
|
393
|
-
* @returns {Promise<object>}
|
|
394
|
-
*/
|
|
395
|
-
async setPopMaintenanceMode(enabled) {
|
|
396
|
-
logger.log('Admin.setPopMaintenanceMode', { enabled })
|
|
397
|
-
return this.#httpClient.post('/api/v1/system/maintenance/pop', { enabled })
|
|
398
|
-
}
|
|
399
|
-
|
|
400
362
|
/**
|
|
401
363
|
* Get system metrics (CPU, memory, connections, etc.)
|
|
402
364
|
* @param {object} params - Query parameters (from, to, etc.)
|
|
@@ -419,15 +381,6 @@ export class Admin {
|
|
|
419
381
|
return this.#httpClient.get(`/api/v1/analytics/worker-metrics${queryString}`)
|
|
420
382
|
}
|
|
421
383
|
|
|
422
|
-
/**
|
|
423
|
-
* Get PostgreSQL statistics
|
|
424
|
-
* @returns {Promise<object>}
|
|
425
|
-
*/
|
|
426
|
-
async getPostgresStats() {
|
|
427
|
-
logger.log('Admin.getPostgresStats', {})
|
|
428
|
-
return this.#httpClient.get('/api/v1/analytics/postgres-stats')
|
|
429
|
-
}
|
|
430
|
-
|
|
431
384
|
// ===========================
|
|
432
385
|
// Helper Methods
|
|
433
386
|
// ===========================
|
|
@@ -16,6 +16,14 @@ import { checkConflationResponse, CONFLATION_UNSUPPORTED } from '../utils/confla
|
|
|
16
16
|
import { popSizing, parseAutopilotDecision } from '../utils/autopilot.js'
|
|
17
17
|
import * as logger from '../utils/logger.js'
|
|
18
18
|
|
|
19
|
+
// Per-item push statuses that mean the broker took the message: `queued`
|
|
20
|
+
// (stored), and `buffered` (a 1.x broker wrote it to its failover file and
|
|
21
|
+
// replays it into storage itself). Every other status is a rejection: `error`
|
|
22
|
+
// from a 2.x broker, which carries no per-item message, and `failed` from a
|
|
23
|
+
// 1.x broker, which does. Unknown statuses count as rejections too: a status
|
|
24
|
+
// this client cannot read is not a message it can report as stored.
|
|
25
|
+
const ACCEPTED_PUSH_STATUSES = new Set(['queued', 'buffered'])
|
|
26
|
+
|
|
19
27
|
export class QueueBuilder {
|
|
20
28
|
#queen
|
|
21
29
|
#httpClient
|
|
@@ -849,24 +857,40 @@ class PushBuilder {
|
|
|
849
857
|
for (let i = 0; i < results.length; i++) {
|
|
850
858
|
const result = results[i]
|
|
851
859
|
const originalItem = this.#formattedItems[i]
|
|
860
|
+
const status = result && typeof result === 'object' ? result.status : undefined
|
|
852
861
|
|
|
853
|
-
if (
|
|
862
|
+
if (status === 'duplicate') {
|
|
854
863
|
duplicates.push({ ...originalItem, result })
|
|
855
|
-
} else if (
|
|
856
|
-
failed.push({ ...originalItem, result, error: result.error })
|
|
857
|
-
} else if (result.status === 'queued') {
|
|
864
|
+
} else if (ACCEPTED_PUSH_STATUSES.has(status)) {
|
|
858
865
|
successful.push({ ...originalItem, result })
|
|
866
|
+
} else {
|
|
867
|
+
const error = result && typeof result.error === 'string' && result.error.length > 0
|
|
868
|
+
? result.error
|
|
869
|
+
: `push rejected by the broker (item status: ${status === undefined ? 'missing' : JSON.stringify(status)})`
|
|
870
|
+
failed.push({ ...originalItem, result, error })
|
|
859
871
|
}
|
|
860
872
|
}
|
|
861
873
|
|
|
874
|
+
// One error for the whole call, built once so the callback and the
|
|
875
|
+
// throw describe the failure the same way. `results` is the broker's
|
|
876
|
+
// per-item answer in input order: the items that did go through are
|
|
877
|
+
// in it too, which is what a caller needs to retry only the rest.
|
|
878
|
+
let failure = null
|
|
879
|
+
if (failed.length > 0) {
|
|
880
|
+
failure = new Error(failed.length === 1
|
|
881
|
+
? failed[0].error
|
|
882
|
+
: `${failed.length} of ${results.length} pushed items rejected: ${failed[0].error}`)
|
|
883
|
+
failure.results = results
|
|
884
|
+
logger.error('PushBuilder.execute', { status: 'failed', count: failed.length, total: results.length, error: failed[0].error })
|
|
885
|
+
}
|
|
886
|
+
|
|
862
887
|
// Call appropriate callbacks
|
|
863
888
|
if (duplicates.length > 0 && this.#onDuplicateCallback) {
|
|
864
889
|
await this.#onDuplicateCallback(duplicates, new Error('Duplicate transaction IDs detected'))
|
|
865
890
|
}
|
|
866
891
|
|
|
867
|
-
if (
|
|
868
|
-
|
|
869
|
-
await this.#onErrorCallback(failed, error)
|
|
892
|
+
if (failure && this.#onErrorCallback) {
|
|
893
|
+
await this.#onErrorCallback(failed, failure)
|
|
870
894
|
}
|
|
871
895
|
|
|
872
896
|
if (successful.length > 0 && this.#onSuccessCallback) {
|
|
@@ -874,9 +898,8 @@ class PushBuilder {
|
|
|
874
898
|
}
|
|
875
899
|
|
|
876
900
|
// Only throw if no error callback is defined
|
|
877
|
-
if (
|
|
878
|
-
|
|
879
|
-
throw new Error(failed[0].error || 'Push failed')
|
|
901
|
+
if (failure && !this.#onErrorCallback) {
|
|
902
|
+
throw failure
|
|
880
903
|
}
|
|
881
904
|
|
|
882
905
|
logger.log('PushBuilder.execute', { status: 'success', successful: successful.length, duplicates: duplicates.length, failed: failed.length })
|
|
@@ -24,8 +24,8 @@
|
|
|
24
24
|
* bundle's fate; see TransactionBuilder, which says so where it happens.)
|
|
25
25
|
*
|
|
26
26
|
* 2. ONLY RELATIVE DURATIONS, IN MILLISECONDS (§4.2, §20.6). `delayMs`, never
|
|
27
|
-
* `delaySeconds` and never an absolute instant: `deliver_at` is computed
|
|
28
|
-
*
|
|
27
|
+
* `delaySeconds` and never an absolute instant: `deliver_at` is computed on
|
|
28
|
+
* the broker's clock, so there is ONE clock and no client's skew can enter. The rule
|
|
29
29
|
* of the product is "durations that can be sub-second are in milliseconds,
|
|
30
30
|
* the ones that cannot are in seconds" -- a 250 ms retry backoff is a real
|
|
31
31
|
* and central use of timers, which is why this wire is the millisecond one.
|
|
@@ -30,6 +30,7 @@ import { generateUUID } from './QueueBuilder.js'
|
|
|
30
30
|
import { isValidUUID } from '../utils/validation.js'
|
|
31
31
|
import { kvOp, materializeKvOp } from '../kv/Kv.js'
|
|
32
32
|
import { TimerBuilder } from './TimerBuilder.js'
|
|
33
|
+
import { consumerGroupOf } from '../utils/consumerGroup.js'
|
|
33
34
|
|
|
34
35
|
export class TransactionBuilder {
|
|
35
36
|
#httpClient
|
|
@@ -46,10 +47,21 @@ export class TransactionBuilder {
|
|
|
46
47
|
this.#httpClient = httpClient
|
|
47
48
|
}
|
|
48
49
|
|
|
50
|
+
/**
|
|
51
|
+
* Ack popped messages as part of this transaction.
|
|
52
|
+
*
|
|
53
|
+
* The consumer group defaults to the one each message was popped under
|
|
54
|
+
* (`message.consumerGroup`, which every pop answers with), because the lease
|
|
55
|
+
* an ack has to match belongs to that group: an ack that names no group is
|
|
56
|
+
* judged in queue mode, and a message popped by a group then fails with
|
|
57
|
+
* `rejected_ack`. Pass `{ consumerGroup }` (or `{ group }`, the key
|
|
58
|
+
* `queen.ack()` takes) to override it.
|
|
59
|
+
*/
|
|
49
60
|
ack(messages, status = 'completed', context = {}) {
|
|
50
61
|
const msgs = Array.isArray(messages) ? messages : [messages]
|
|
51
|
-
|
|
52
|
-
|
|
62
|
+
const explicitGroup = context.consumerGroup || context.group || null
|
|
63
|
+
|
|
64
|
+
logger.log('TransactionBuilder.ack', { count: msgs.length, status, consumerGroup: explicitGroup })
|
|
53
65
|
|
|
54
66
|
msgs.forEach(msg => {
|
|
55
67
|
const transactionId = typeof msg === 'string' ? msg : (msg.transactionId || msg.id)
|
|
@@ -72,9 +84,18 @@ export class TransactionBuilder {
|
|
|
72
84
|
status
|
|
73
85
|
}
|
|
74
86
|
|
|
75
|
-
|
|
76
|
-
if (
|
|
77
|
-
operation.consumerGroup =
|
|
87
|
+
const consumerGroup = explicitGroup || consumerGroupOf(msg)
|
|
88
|
+
if (consumerGroup) {
|
|
89
|
+
operation.consumerGroup = consumerGroup
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
// Each ack names its own lease. A Queen 2 broker fences an ack with the
|
|
93
|
+
// lease the operation carries, and lends it one from requiredLeases only
|
|
94
|
+
// when the bundle names a single lease: in a bundle spanning two leases,
|
|
95
|
+
// an ack without its own would be applied whoever holds its partition
|
|
96
|
+
// now. 1.x brokers read the operation's lease before requiredLeases too.
|
|
97
|
+
if (leaseId) {
|
|
98
|
+
operation.leaseId = leaseId
|
|
78
99
|
}
|
|
79
100
|
|
|
80
101
|
this.#operations.push(operation)
|
|
@@ -204,7 +225,7 @@ export class TransactionBuilder {
|
|
|
204
225
|
*
|
|
205
226
|
* `putIfAbsent` with `required:true`, so a marker that already exists ABORTS
|
|
206
227
|
* the transaction: the push and the ack roll back together with it. That is
|
|
207
|
-
* what makes "the email is sent exactly once" a property of the
|
|
228
|
+
* what makes "the email is sent exactly once" a property of the broker
|
|
208
229
|
* rather than a hope about redelivery.
|
|
209
230
|
*
|
|
210
231
|
* The verdict is RETURNED by `commit()`, not thrown -- see there.
|
|
@@ -303,9 +324,14 @@ export class TransactionBuilder {
|
|
|
303
324
|
|
|
304
325
|
logger.error('TransactionBuilder.commit', { error: result.error, reason: result.reason })
|
|
305
326
|
const error = new Error(result.error || 'Transaction failed')
|
|
306
|
-
// The
|
|
307
|
-
// to match the message
|
|
308
|
-
//
|
|
327
|
+
// The broker's reason code travels ON the error, so no caller ever has
|
|
328
|
+
// to match the message. The ones worth branching on from a 2.x broker:
|
|
329
|
+
// bad_request, duplicate (a pushed transactionId is already stored),
|
|
330
|
+
// rejected_ack (an acked message is no longer leased by this worker, or
|
|
331
|
+
// was popped under a different consumer group than the ack names), and
|
|
332
|
+
// too_large. A 1.x broker spells the ack one ack_rejected, and also
|
|
333
|
+
// sends timer_horizon_exceeded, payload_too_large, misaligned and
|
|
334
|
+
// db_error.
|
|
309
335
|
if (result.reason) error.reason = result.reason
|
|
310
336
|
error.result = result
|
|
311
337
|
throw error
|