queen-mq 2.0.3 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -636,6 +636,49 @@ Otherwise the lanes do not serialise it for you: use `incr`, or carry `expect`.
636
636
  **`putIfAbsent` plus a TTL is not a distributed lock.** A lock that expires is not revoked: the old
637
637
  holder keeps working, it simply no longer has the row. Carry your `version` as `expect` on every
638
638
  later write so a lapsed holder fails with `reason: 'version'` instead of overwriting the new one.
639
+ `queen.lock()` below is that, done for you.
640
+
641
+ **`check` asserts a version and writes nothing.** `kv.check(ns, key, { expect })` holds while the
642
+ key is at that version (`expect: 0`: while it is absent). With `required: true` in a transaction
643
+ it gates the commit on a key the transaction does not write.
644
+
645
+ ### Locks and semaphores
646
+
647
+ A lock is a lease: one holder at a time, for a lifetime the holder renews, with a token that
648
+ fences a holder that outlived it. `queen.semaphore(name, n, opts)` is the same with `n` permits.
649
+
650
+ ```javascript
651
+ const lock = queen.lock('daily-report', { ttl: '30s' })
652
+ if (!(await lock.acquire({ wait: '5s' }))) return // somebody else holds it
653
+
654
+ try {
655
+ await queen.transaction()
656
+ .guard(lock) // commits only while the lock is ours
657
+ .queue('reports').push([{ data: report }])
658
+ .commit()
659
+ } finally {
660
+ await lock.release()
661
+ }
662
+
663
+ // Or: acquire, run, release.
664
+ const { acquired, value } = await queen.lock('sync:crm', { ttl: '1m' }).run(() => sync())
665
+ ```
666
+
667
+ **It expires, and nobody tells the holder.** A paused or partitioned process carries on past its
668
+ lifetime while somebody else acquires. `.guard(lock)` is what keeps its work out: the transaction
669
+ rolls back with `reason: 'kv_precondition'` (returned from `commit()`, not thrown) when the lock is
670
+ no longer this handle's. Outside Queen, fence with `lock.token`, which only rises on a lock.
671
+
672
+ **The handle renews in the background**, every third of the lifetime (`autoRenew: false` to do it
673
+ yourself with `lock.renew()`), and says when the lock is gone: `lock.signal` aborts and
674
+ `lock.onLost(fn)` runs. Nothing stops your code: pass `lock.signal` to what the work awaits.
675
+
676
+ **The token changes at every renew.** Read `lock.token` and `lock.guard()` when you use them; do
677
+ not keep a copy across an `await`.
678
+
679
+ `queen.locks` is the wire, with no state kept: `acquire`, `renew`, `release`, `get(name)` (who
680
+ holds it, since when, until when) and `batch` for several locks in one call. `queen.close()`
681
+ releases the locks its handles still hold.
639
682
 
640
683
  ### Timers
641
684
 
@@ -1079,3 +1122,60 @@ const messages: Message<OrderData>[] = await queen.queue('orders').pop()
1079
1122
  ## License
1080
1123
 
1081
1124
  Apache 2.0 - See [LICENSE.md](../LICENSE.md)
1125
+
1126
+ ### Optional consumer supervision
1127
+
1128
+ ```js
1129
+ await queen.queue('orders').group('billing')
1130
+ .supervision({ group: 'billing-production' })
1131
+ .concurrency(4).each().consume(async message => { /* process message */ })
1132
+ ```
1133
+
1134
+ Supervision defaults to off; `.supervision(false)` disables it. The group names
1135
+ an application/deployment in the dashboard, independently of the consumer group.
1136
+ Each consume invocation publishes its own instance into the broker's
1137
+ `queen-supervisor` KV namespace every 10 seconds (30-second heartbeat timeout,
1138
+ 60-second TTL), plus a final stopped observation. The credential needs KV write
1139
+ access. Publication is serialized and best effort with a two-second deadline.
1140
+
1141
+ #### Match supervision to the consumer lifetime
1142
+
1143
+ The instance belongs to one **`consume()` invocation**, not to the `Queen`
1144
+ client, queue, hostname, process or publication group. Reusing the same client,
1145
+ builder and group does not reuse the instance ID. Each invocation starts new
1146
+ counters, publishes its initial state, then reports `stopped` when all its
1147
+ workers exit, including a normal exit caused by `.limit(...)`, `.idleMillis(...)`
1148
+ or cancellation.
1149
+
1150
+ | Application pattern | Supervision setup |
1151
+ | --- | --- |
1152
+ | One persistent `consume()` with several concurrent workers | Enable `.supervision({ group })` on that invocation. |
1153
+ | Several persistent `consume()` calls in one process | Enable reporting on each; expect several instances with the same hostname. Budget their combined concurrency. |
1154
+ | An outer scheduler repeatedly calling `.limit(1).consume(...)` or rotating queues | Leave per-call supervision off when you need the scheduler's lifetime status; publish one observation for the persistent scheduler instead. |
1155
+
1156
+ For a persistent consumer, omit limits that deliberately end the invocation.
1157
+ Pass an `AbortSignal` to `consume(handler, { signal })`; on shutdown, abort it,
1158
+ await the consume promise so active handlers drain, then close the client.
1159
+ Do not remove an existing scheduler's limits just to change the dashboard:
1160
+ those limits may enforce queue fairness and the application's worker budget.
1161
+
1162
+ Supervising short calls is valid when you want to observe those calls. A loop
1163
+ that starts a new call after every message or idle poll will produce many
1164
+ recent `stopped` instances, with `running: 0`, until their 60-second TTL expires.
1165
+ Expired records can remain visible until the broker sweeps them.
1166
+ That is the last state of each finished invocation, not evidence that its
1167
+ process or pod has stopped. Compare instance IDs and application logs.
1168
+
1169
+ The JavaScript API has no option to attach several consume invocations to one
1170
+ persistent reporter or supply their instance ID. A custom scheduler can publish
1171
+ the [consumer status contract](https://queenmq.com/reference/supervisor-status/#reporting-an-application-owned-scheduler)
1172
+ with one ID per scheduler lifetime, stable worker pool names and counters
1173
+ across turns. Keep SDK supervision disabled on its short calls and preserve
1174
+ their ACK/NACK, lease renewal, cancellation and scheduling behavior.
1175
+
1176
+ The Supervisors page supporting `queen.consumer.status/v1` shows live async
1177
+ consumer loops, busy handlers, successful/failed handler calls and progress times.
1178
+ A batch is one handler call; completion does not imply ACK success. No payloads or
1179
+ error text are published. Event-loop starvation can stop heartbeats. Reporting
1180
+ does not restart processes or tasks, change ACK/lease policies, or enable remote
1181
+ control. With reporting off there is no additional timer or network traffic.
@@ -10,6 +10,7 @@ import { QueueBuilder } from './builders/QueueBuilder.js'
10
10
  import { TransactionBuilder } from './builders/TransactionBuilder.js'
11
11
  import { TimerBuilder } from './builders/TimerBuilder.js'
12
12
  import { Kv } from './kv/Kv.js'
13
+ import { Locks, Lock } from './locks/Locks.js'
13
14
  import { Ephemeral } from './ephemeral/Ephemeral.js'
14
15
  import { StreamBuilder } from './stream/StreamBuilder.js'
15
16
  import { StreamConsumer } from './stream/StreamConsumer.js'
@@ -93,6 +94,7 @@ export class Queen {
93
94
  #shutdownHandlers = []
94
95
  #admin = null
95
96
  #kv = null
97
+ #locks = null
96
98
  #ephemeral = null
97
99
  // Process-wide kill switch for pop autopilot, read from
98
100
  // QUEEN_SDK_POP_AUTOPILOT once here rather than on every pop: it is a
@@ -298,6 +300,76 @@ export class Queen {
298
300
  return this.#kv
299
301
  }
300
302
 
303
+ // ===========================
304
+ // Locks API Entry Point
305
+ // ===========================
306
+
307
+ /**
308
+ * A lock: one holder at a time, held as a lease, with a fencing token.
309
+ *
310
+ * const lock = queen.lock('daily-report', { ttl: '30s' })
311
+ * if (!(await lock.acquire())) return // somebody else has it
312
+ * try {
313
+ * await queen.transaction().guard(lock)
314
+ * .queue('reports').push([{ data: report }]).commit()
315
+ * } finally {
316
+ * await lock.release()
317
+ * }
318
+ *
319
+ * Creating the handle sends nothing. `acquire()` resolves a boolean,
320
+ * `acquire({ wait: '10s' })` keeps trying, and while it is held the handle
321
+ * renews it in the background and aborts `lock.signal` if it is lost.
322
+ *
323
+ * It is a lease, not a mutex: it expires, and a holder that outlived it
324
+ * keeps running. `.guard(lock)` on a transaction is what makes the WORK
325
+ * exclusive; see `locks/Locks.js` for the whole argument.
326
+ *
327
+ * @param {string} name - no '#', no control characters, at most 256 bytes
328
+ * @param {{ttl?: string, ttlSeconds?: number, owner?: string, autoRenew?: boolean, renewEvery?: string}} opts
329
+ * a lifetime is mandatory: `ttl: '30s'` or `ttlSeconds: 30`
330
+ * @returns {Lock}
331
+ */
332
+ lock(name, opts = {}) {
333
+ if (opts.limit !== undefined && opts.limit !== 1) {
334
+ throw new Error('lock: a lock has one permit — queen.semaphore(name, limit, opts) is the one with more')
335
+ }
336
+ return new Lock(this.locks, name, opts)
337
+ }
338
+
339
+ /**
340
+ * A semaphore: at most `limit` holders at a time. One handle is ONE permit;
341
+ * make a handle per holder.
342
+ *
343
+ * const gpu = queen.semaphore('gpu', 4, { ttl: '2m' })
344
+ * const { acquired } = await gpu.run(() => train(job), { wait: '30s' })
345
+ *
346
+ * Everything a lock's handle does, this one does (it is the same class: a
347
+ * lock is the semaphore of one). Every holder of one name passes the same
348
+ * limit — it is the caller's and is stored nowhere, so while a limit is
349
+ * being changed the larger one rules.
350
+ *
351
+ * @returns {Lock}
352
+ */
353
+ semaphore(name, limit, opts = {}) {
354
+ return new Lock(this.locks, name, { ...opts, limit })
355
+ }
356
+
357
+ /**
358
+ * The four lock operations as the broker speaks them — `acquire`, `renew`,
359
+ * `release`, `get`, and `batch` for several locks in one call — with no
360
+ * state kept: you carry the token. `queen.lock()` is what most code wants;
361
+ * this is for `get` (who holds it?) and for a caller with its own loop.
362
+ *
363
+ * Lazily initialized, singleton, like `kv`.
364
+ * @returns {Locks}
365
+ */
366
+ get locks() {
367
+ if (!this.#locks) {
368
+ this.#locks = new Locks(this.#httpClient)
369
+ }
370
+ return this.#locks
371
+ }
372
+
301
373
  // ===========================
302
374
  // Ephemeral API Entry Point
303
375
  // ===========================
@@ -717,6 +789,18 @@ export class Queen {
717
789
  // Cleanup buffer manager
718
790
  this.#bufferManager.cleanup()
719
791
 
792
+ // Give back the locks this client's handles still hold, so the next
793
+ // holder does not wait out their lifetime. Best effort: one that cannot
794
+ // be released expires by itself.
795
+ if (this.#locks) {
796
+ try {
797
+ const released = await this.#locks.releaseAll()
798
+ if (released > 0) logger.log('Queen.close', { locksReleased: released })
799
+ } catch (error) {
800
+ logger.warn('Queen.close', { error: error.message, phase: 'lock-release' })
801
+ }
802
+ }
803
+
720
804
  // Remove shutdown handlers
721
805
  for (const cleanup of this.#shutdownHandlers) {
722
806
  cleanup()
@@ -42,6 +42,7 @@ export class QueueBuilder {
42
42
  #config = {}
43
43
 
44
44
  // Consume options
45
+ #supervision = false
45
46
  #concurrency = CONSUME_DEFAULTS.concurrency
46
47
  // batch / maxPartitions hold the USER's value, and null means the setter was
47
48
  // never called -- which is the dimension pop autopilot gets to choose. The
@@ -238,6 +239,12 @@ export class QueueBuilder {
238
239
  return this
239
240
  }
240
241
 
242
+ /** Opt-in consumer observations in the broker dashboard; false by default. */
243
+ supervision(config = false) {
244
+ this.#supervision = config
245
+ return this
246
+ }
247
+
241
248
  concurrency(count) {
242
249
  this.#concurrency = Math.max(1, count)
243
250
  return this
@@ -410,6 +417,7 @@ export class QueueBuilder {
410
417
  // has to survive all the way to #buildParams: it is the ONLY record that
411
418
  // the user said nothing about that dimension.
412
419
  autopilot: this.#autopilotEnabled(),
420
+ supervision: options.supervision ?? this.#supervision,
413
421
  signal: options.signal
414
422
  }
415
423
 
@@ -40,6 +40,10 @@ export class TransactionBuilder {
40
40
  // was queued would ship a TTL already stale by however long the bundle took
41
41
  // to assemble. Materialized in commit(), which is send time.
42
42
  #kvEntries = []
43
+ // Lock handles whose permit this bundle commits under (`.guard(lock)`).
44
+ // Kept as handles, not as ops: the token is read at SEND time, because a
45
+ // lock's own renewal changes it.
46
+ #guards = []
43
47
  #timerOps = []
44
48
  #kvApi = null
45
49
 
@@ -202,6 +206,9 @@ export class TransactionBuilder {
202
206
  putIfAbsent: (ns, key, value, opts = {}) => add(kvOp.putIfAbsent(ns, key, value, opts)),
203
207
  delete: (ns, key, opts = {}) => add(kvOp.delete(ns, key, opts)),
204
208
  incr: (ns, key, delta = 1, opts = {}) => add(kvOp.incr(ns, key, delta, opts)),
209
+ // A precondition on a key the bundle does not write. With
210
+ // `required: true` it is the bundle's gate; without, only a look.
211
+ check: (ns, key, opts = {}) => add(kvOp.check(ns, key, opts)),
205
212
  getPrefix: () => {
206
213
  throw new Error(
207
214
  'kv: getPrefix is not available inside a transaction — unbounded read work under the outermost ' +
@@ -236,6 +243,40 @@ export class TransactionBuilder {
236
243
  return this.kv.putIfAbsent(ns, key, value, { ...opts, ...required })
237
244
  }
238
245
 
246
+ /**
247
+ * Commit this bundle only while a lock is held.
248
+ *
249
+ * const lock = queen.lock('daily-report', { ttl: '30s' })
250
+ * if (!(await lock.acquire())) return
251
+ * const res = await queen.transaction()
252
+ * .guard(lock)
253
+ * .queue('reports').push([{ data: report }])
254
+ * .commit()
255
+ * if (res.success === false) return // the lock is somebody else's now
256
+ *
257
+ * The guard is a `check` of the permit's row at the lock's token, with
258
+ * `required: true`: the broker judges it in the same log entry as the acks,
259
+ * pushes, keys and timers beside it. A holder that was paused past its
260
+ * lifetime and replaced commits NOTHING — which the lock by itself cannot
261
+ * promise, since nobody stops an expired holder from running.
262
+ *
263
+ * The token is read when `commit()` sends, and a lock's own background
264
+ * renewal moves it. A guard that lost to this handle's own renewal is sent
265
+ * again with the new token; one that lost to another holder is the verdict,
266
+ * returned like `once`'s (`success: false, reason: 'kv_precondition'`), and
267
+ * the handle then reports the lock lost.
268
+ *
269
+ * Throws at `commit()`, with `.code === 'LOCK_NOT_HELD'`, when the handle
270
+ * holds nothing: a step that asked for a guard must not go out without one.
271
+ */
272
+ guard(lock) {
273
+ if (!lock || typeof lock.guard !== 'function' || typeof lock._settled !== 'function') {
274
+ throw new Error('transaction: guard() takes a lock from queen.lock() or queen.semaphore()')
275
+ }
276
+ this.#guards.push(lock)
277
+ return this
278
+ }
279
+
239
280
  // ===========================
240
281
  // Timer rider (§4, §9.6)
241
282
  // ===========================
@@ -266,20 +307,11 @@ export class TransactionBuilder {
266
307
  })
267
308
  }
268
309
 
269
- async commit() {
270
- const riderCount = this.#kvEntries.length + this.#timerOps.length
271
- if (this.#operations.length === 0 && riderCount === 0) {
272
- logger.error('TransactionBuilder.commit', 'No operations to commit')
273
- throw new Error('Transaction has no operations to commit')
274
- }
275
-
276
- logger.log('TransactionBuilder.commit', {
277
- operationCount: this.#operations.length,
278
- requiredLeases: this.#requiredLeases.length,
279
- kv: this.#kvEntries.length,
280
- timers: this.#timerOps.length
281
- })
282
-
310
+ /**
311
+ * The request body, built at send time. The guards go first in `kv`, so
312
+ * guard `i` is op `i` of the rider.
313
+ */
314
+ #body() {
283
315
  // Byte-identity when the riders are absent (§6.3): the keys are added only
284
316
  // when there is something in them, so a bundle that uses neither feature
285
317
  // produces exactly the body it produced before this feature existed.
@@ -287,16 +319,72 @@ export class TransactionBuilder {
287
319
  operations: this.#operations,
288
320
  requiredLeases: [...new Set(this.#requiredLeases)] // Unique leases
289
321
  }
290
- if (this.#kvEntries.length > 0) {
322
+ if (this.#guards.length + this.#kvEntries.length > 0) {
291
323
  const now = Date.now()
292
- body.kv = this.#kvEntries.map(e => materializeKvOp(e, now))
324
+ body.kv = [
325
+ ...this.#guards.map(lock => lock.guard()),
326
+ ...this.#kvEntries.map(e => materializeKvOp(e, now))
327
+ ]
293
328
  }
294
329
  if (this.#timerOps.length > 0) {
295
330
  body.timers = this.#timerOps
296
331
  }
332
+ return body
333
+ }
334
+
335
+ /**
336
+ * The lock whose guard is the precondition a rolled-back bundle names, if
337
+ * it is one of this bundle's. `failedIndex` is in the FLAT space of
338
+ * `results[]`: every pushed item and every ack first, then the `kv` rider.
339
+ */
340
+ #failedGuard(result) {
341
+ if (this.#guards.length === 0 || !Number.isInteger(result.failedIndex)) return null
342
+ const flatOperations = this.#operations.reduce(
343
+ (n, op) => n + (op.type === 'push' ? op.items.length : 1), 0
344
+ )
345
+ return this.#guards[result.failedIndex - flatOperations] ?? null
346
+ }
347
+
348
+ async commit() {
349
+ const riderCount = this.#guards.length + this.#kvEntries.length + this.#timerOps.length
350
+ if (this.#operations.length === 0 && riderCount === 0) {
351
+ logger.error('TransactionBuilder.commit', 'No operations to commit')
352
+ throw new Error('Transaction has no operations to commit')
353
+ }
354
+
355
+ logger.log('TransactionBuilder.commit', {
356
+ operationCount: this.#operations.length,
357
+ requiredLeases: this.#requiredLeases.length,
358
+ kv: this.#kvEntries.length,
359
+ guards: this.#guards.length,
360
+ timers: this.#timerOps.length
361
+ })
297
362
 
298
363
  try {
299
- const result = await this.#httpClient.post('/api/v1/transaction', body)
364
+ // A bundle is sent again in ONE case: its guard lost to the lock's own
365
+ // background renewal, which moved the token between the moment the body
366
+ // was built and the moment the broker judged it. The row still names
367
+ // this owner, so the lock is held; nothing committed (a lost required
368
+ // precondition rolls the whole bundle back), so sending it again with
369
+ // the new token is the same step, not a second one.
370
+ let result
371
+ for (let attempt = 0; ; attempt++) {
372
+ await Promise.all(this.#guards.map(lock => lock._settled()))
373
+ result = await this.#httpClient.post('/api/v1/transaction', this.#body())
374
+ if (result.success || result.reason !== 'kv_precondition') break
375
+ const lock = this.#failedGuard(result)
376
+ if (!lock) break
377
+ const ownRenewal = result.kvReason === 'version' && result.value?.owner === lock.owner
378
+ if (!ownRenewal) {
379
+ // Expired, released, or another holder's: the lock is gone.
380
+ lock._lost('guard')
381
+ break
382
+ }
383
+ if (attempt >= 3) break
384
+ await lock._settled()
385
+ if (!lock.held) break
386
+ logger.log('TransactionBuilder.commit', { status: 'guard_renewed', lock: lock.name, attempt })
387
+ }
300
388
 
301
389
  if (!result.success) {
302
390
  // THE ONE OUTCOME THAT RETURNS INSTEAD OF THROWING (§8.3).
@@ -3,6 +3,7 @@
3
3
  */
4
4
 
5
5
  import * as logger from '../utils/logger.js'
6
+ import { Supervision } from './Supervision.js'
6
7
  import { checkConflationResponse, CONFLATION_UNSUPPORTED } from '../utils/conflation.js'
7
8
  import { popSizing, parseAutopilotDecision, emptyPollDelayMillis } from '../utils/autopilot.js'
8
9
  import { CONSUME_DEFAULTS } from '../utils/defaults.js'
@@ -92,9 +93,13 @@ export class ConsumerManager {
92
93
  // Generate affinity key for consistent routing to same backend
93
94
  const affinityKey = this.#getAffinityKey(queue, partition, namespace, task, group)
94
95
 
96
+ const supervision = options.supervision ? new Supervision(this.#httpClient, options.supervision, options) : null
97
+ if (supervision) handler = supervision.wrap(handler)
98
+
95
99
  // Start workers
96
100
  const workers = []
97
101
  for (let i = 0; i < concurrency; i++) {
102
+ if (supervision) supervision.running++
98
103
  workers.push(this.#worker(i, handler, path, baseParams, {
99
104
  batch,
100
105
  limit,
@@ -113,9 +118,12 @@ export class ConsumerManager {
113
118
  // the pop target to key the once-per-(queue,group) conflict warning.
114
119
  conflation,
115
120
  conflationCtx: { queue, namespace, task, group }
121
+ }).finally(async () => {
122
+ if (supervision && --supervision.running === 0) await supervision.stop()
116
123
  }))
117
124
  }
118
125
 
126
+ supervision?.start()
119
127
  logger.log('ConsumerManager.start', { status: 'workers-started', count: concurrency })
120
128
 
121
129
  // Wait for all workers to complete
@@ -0,0 +1,78 @@
1
+ import { randomBytes } from 'node:crypto'
2
+ import { hostname } from 'node:os'
3
+ import * as logger from '../utils/logger.js'
4
+
5
+ // One opt-in reporter per consume invocation. Handler calls are not ACKs.
6
+ export class Supervision {
7
+ constructor(http, config, options) {
8
+ if (!config || typeof config !== 'object' || typeof config.group !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9._-]{0,254}$/.test(config.group) || config.group === 'coordination' || /[\r\n]/.test(config.group)) throw new Error('supervision.group must be a valid application/deployment name')
9
+ if (!Number.isInteger(options.concurrency) || options.concurrency < 1 || options.concurrency > 4096) throw new Error('supervision requires concurrency between 1 and 4096')
10
+ this.http = http
11
+ this.options = options
12
+ this.group = config.group
13
+ this.id = randomBytes(16).toString('hex')
14
+ this.started = Date.now()
15
+ this.monotonic = performance.now()
16
+ this.running = this.completed = this.failed = this.sequence = 0
17
+ this.last = this.pending = this.timer = null
18
+ this.active = new Map()
19
+ this.state = 'running'
20
+ this.warned = false
21
+ }
22
+ wrap(handler) {
23
+ return async (...args) => {
24
+ const id = this.sequence++
25
+ this.active.set(id, performance.now())
26
+ try { const result = await handler(...args); this.completed++; return result }
27
+ catch (error) { this.failed++; throw error }
28
+ finally { this.active.delete(id); this.last = Math.floor(Date.now() / 1000) }
29
+ }
30
+ }
31
+ start() {
32
+ this.publish()
33
+ this.timer = setInterval(() => this.publish(), 10_000)
34
+ this.timer.unref()
35
+ }
36
+ document() {
37
+ const now = performance.now()
38
+ return {
39
+ schema: 'queen.consumer.status/v1', instance_id: this.id, engine: 'js', execution_model: 'async-tasks',
40
+ hostname: hostname(), pid: process.pid, state: this.state, updated_at_epoch: Math.floor(Date.now() / 1000),
41
+ started_at_epoch: Math.floor(this.started / 1000), uptime_seconds: Math.floor((now - this.monotonic) / 1000),
42
+ configuration: { heartbeat_timeout: 30 },
43
+ pool_status: [{ name: 'consumer', queue: this.options.queue || null, namespace: this.options.namespace || null,
44
+ task: this.options.task || null, consumer_group: this.options.group || '__QUEUE_MODE__',
45
+ desired: this.options.concurrency, running: this.running, busy: this.active.size,
46
+ completed: this.completed, failed: this.failed, last_completed_at_epoch: this.last,
47
+ oldest_inflight_seconds: this.active.size ? Math.floor((now - Math.min(...this.active.values())) / 1000) : null }],
48
+ }
49
+ }
50
+ publish() {
51
+ if (this.pending) return this.pending
52
+ this.pending = this.send().finally(() => { this.pending = null })
53
+ return this.pending
54
+ }
55
+ async send() {
56
+ try {
57
+ const bytes = Buffer.from(JSON.stringify(this.document()))
58
+ if (bytes.length > 45_000) throw new Error('Consumer status exceeds one chunk')
59
+ const write = randomBytes(16).toString('hex'), slot = `${this.group}/${this.id}`
60
+ const ops = [
61
+ { op: 'put', ns: 'queen-supervisor', key: `${slot}/head`, value: { format: 'queen.supervisor.remote-status/v1', write, chunks: 1, bytes: bytes.length }, ttlSeconds: 60 },
62
+ { op: 'put', ns: 'queen-supervisor', key: `${slot}/chunk/0000`, value: { write, index: 0, data: bytes.toString('base64') }, ttlSeconds: 60 },
63
+ ]
64
+ const result = await this.http.post('/api/v1/kv', { operations: ops }, 2000, null, null, AbortSignal.timeout(2000))
65
+ if (!Array.isArray(result?.results) || result.results.length !== 2 || result.results.some(r => r.applied !== true)) throw new Error('Consumer status publication was not applied')
66
+ this.warned = false
67
+ } catch {
68
+ if (!this.warned) logger.warn('Consumer.supervision', 'Status publication failed; consumption continues')
69
+ this.warned = true
70
+ }
71
+ }
72
+ async stop() {
73
+ clearInterval(this.timer)
74
+ if (this.pending) await this.pending
75
+ this.state = 'stopped'
76
+ await this.publish()
77
+ }
78
+ }
@@ -611,8 +611,8 @@ export class HttpClient {
611
611
  return this.#requestWithFailover('GET', path, null, requestTimeoutMillis, affinityKey, retryKind, signal)
612
612
  }
613
613
 
614
- async post(path, body = null, requestTimeoutMillis = null, affinityKey = null, retryKind = null) {
615
- return this.#requestWithFailover('POST', path, body, requestTimeoutMillis, affinityKey, retryKind)
614
+ async post(path, body = null, requestTimeoutMillis = null, affinityKey = null, retryKind = null, signal = null) {
615
+ return this.#requestWithFailover('POST', path, body, requestTimeoutMillis, affinityKey, retryKind, signal)
616
616
  }
617
617
 
618
618
  async put(path, body = null, requestTimeoutMillis = null, affinityKey = null, retryKind = null) {
@@ -16,6 +16,11 @@
16
16
 
17
17
  export { Queen } from './Queen.js'
18
18
  export { Admin } from './admin/Admin.js'
19
+ // Locks and semaphores. Reached as `queen.lock()`, `queen.semaphore()` and
20
+ // `queen.locks`; the classes are exported for typing. `LOCK_NOT_HELD` is the
21
+ // `.code` of the error a guard throws for a handle that holds nothing, and of
22
+ // the reason `lock.signal` aborts with.
23
+ export { Lock, Locks, LOCK_NOT_HELD } from './locks/Locks.js'
19
24
  // RAM-class queues (EPHEMERAL_QUEUES.md §4). Reached as `queen.ephemeral`; the
20
25
  // class is exported for typing and for embedding it on a client of your own.
21
26
  // The two `.code`s the family's 404s carry, which are NOT the same fact:
@@ -152,6 +152,21 @@ export const kvOp = {
152
152
  if (opts.min !== undefined) base.min = opts.min
153
153
  if (opts.max !== undefined) base.max = opts.max
154
154
  return { base: withRequired(base, opts), expiry: opts }
155
+ },
156
+
157
+ check(ns, key, opts = {}) {
158
+ requireName(ns, 'ns')
159
+ requireName(key, 'key')
160
+ // A check IS its expect. One without says nothing, and the broker's
161
+ // `applied:true` to it would read as a guard that held.
162
+ if (!('expect' in opts)) {
163
+ throw new Error(
164
+ 'kv: check needs `expect` — the version the key must be at, or 0 for a key that must not exist'
165
+ )
166
+ }
167
+ const base = { op: 'check', ns, key }
168
+ // No expiry: it writes nothing.
169
+ return { base: withRequired(withExpect(base, opts), opts), expiry: null }
155
170
  }
156
171
  }
157
172
 
@@ -408,6 +423,32 @@ export class Kv {
408
423
  return res
409
424
  }
410
425
 
426
+ /**
427
+ * Is the key still at this version? Writes nothing.
428
+ *
429
+ * const r = await kv.check('orders', '9137', { expect: row.version })
430
+ * if (!r.applied) console.log(r.reason, r.value) // somebody wrote since
431
+ *
432
+ * `expect: 0` asks the opposite: the key must not exist. A WriteResult like
433
+ * every write's, `applied` being "the precondition held"; when it did not,
434
+ * `reason` (`version`, `absent` or `exists`), `value` and `version` are what
435
+ * a reader would see.
436
+ *
437
+ * On its own it is a linearizable look. Where it earns its place is beside
438
+ * writes, with `required: true`: in a batch, or in a transaction's
439
+ * `.kv.check(...)`, it makes everything else commit only if a key the call
440
+ * does not write is unchanged — which is how a step is tied to a lock
441
+ * (`queen.lock`).
442
+ *
443
+ * Needs a broker at cluster version 5, the first with this operation. While
444
+ * a cluster is being upgraded to it the call answers 503
445
+ * `kv_check_needs_cluster_version_5`; an older broker answers 400
446
+ * `kv_unknown_op`.
447
+ */
448
+ async check(ns, key, opts = {}) {
449
+ return this.#write(kvOp.check(ns, key, opts))
450
+ }
451
+
411
452
  /**
412
453
  * The idempotency marker, written the way it is actually used:
413
454
  *