queen-mq 0.16.0 → 1.0.3

package/README.md

@@ -28,6 +28,7 @@ Queen MQ is a PostgreSQL-backed message queue system with a powerful feature set
 - **Message Tracing** - Debug distributed workflows with trace timelines
 - **Client-Side Buffering** - 10x-100x throughput boost for high-volume pushes
 - **Real-time Streaming** - Windowed aggregation and processing
+- **Key/Value State and Timers** - Transactional state and scheduled messages
 
 This client provides a fluent, promise-based API for Node.js applications.
 
@@ -373,6 +374,145 @@ await queen.queue('orders').consume(async (msg) => {
 
 ---
 
+## Key/Value State and Timers
+
+Both surfaces are **always there**. There is nothing to enable: kv and timers are part of the
+broker the way push and pop are, on every cell that runs it. There is no capability to probe and
+no 404 that means "this cell does not have the feature" — a 404 from these routes is a bug.
+
+What an operator can still do is **pause** them, with the runtime kill switch in
+`queen.system_state` (`kv_enabled`, `timers_schedule_enabled`, `timers_fire_enabled`) — the same
+class of lever as maintenance mode, pulled live during an incident and expected to be pulled back.
+A paused surface answers `503` with `Retry-After` and `error: 'kv_disabled'` / `'timers_disabled'`,
+which this client retries like any other 5xx. Inside a transaction it is a `403` on the `kv` or
+`timers` rider instead, so a bundle holding messages does not spin forever on a paused cell.
+
+```javascript
+try {
+  await queen.kv.put('orders', 'order:9f1', { state: 'held' }, { ttl: '60s' })
+} catch (e) {
+  if (e.code === 'kv_disabled') { /* paused by an operator; it will come back */ }
+  throw e
+}
+```
+
+Branch on `e.code`, never on the message. And write that branch as "temporarily paused", not as
+"this deployment lacks KV": handling the refusal is right, treating it as a configuration to check
+before you use the surface is not.
+
+### Key/Value
+
+```javascript
+// An expiry is MANDATORY on every write: exactly one of ttlSeconds (or the
+// sugar ttl / until) and forever: true. A put never inherits the previous TTL.
+await queen.kv.put('orders', 'order:9f1', { state: 'held' }, { ttl: '60s' })
+
+const row = await queen.kv.get('orders', 'order:9f1')
+if (row.found) console.log(row.value, row.version)   // found is separate: null is a legal value
+
+// "Did I win?" in one call. This is the idempotency marker.
+const { won, value } = await queen.kv.once('dedup', eventId, { ttl: '24h' })
+if (!won) return                                     // somebody already did this
+
+// Optimistic lock. expect: 0 means "must not exist"; expect: N is a pure
+// update that creates nothing when it matches no row.
+const res = await queen.kv.put('orders', 'order:9f1', { state: 'shipped' },
+                               { ttl: '60s', expect: row.version })
+if (!res.applied) console.log(res.reason)            // 'version' | 'exists' | 'absent' | 'limit' | 'type'
+
+// Rate limiting without a CAS loop. With max, `applied` IS the admission
+// decision: nothing saturates, nothing truncates, a refusal spends no budget.
+const hit = await queen.kv.incr('quota', `${customer}:${hour}`, 1, { max: 1000, ttl: '1h' })
+if (!hit.applied) throw new TooManyRequests()
+
+for await (const r of queen.kv.listAll('saga', 'order:9f1:')) { /* follows nextAfter */ }
+```
+
+Seven operations: `get`, `getMany`, `getPrefix`, `put`, `putIfAbsent`, `delete`, `incr`, plus the
+two conveniences this client owns, `once` and `listAll`.
+
+**Every write returns an OBJECT, and objects are always truthy.** `if (await queen.kv.delete(ns,
+key))` is always taken and is a bug. Read `.applied`, or `.won` on `once`, or `.found` on a read.
+That holds for all five writes, and it is the one trap this language cannot defend against
+structurally.
+
+**A write that did not apply is not an error.** `applied: false` answers HTTP 200 with the current
+value and version, so the loser needs no second round trip.
+
+**Read-modify-write across two calls is safe only when the KV key derives from the partition key.**
+Otherwise the lanes do not serialise it for you: use `incr`, or carry `expect`.
+
+**`putIfAbsent` plus a TTL is not a distributed lock.** A lock that expires is not revoked: the old
+holder keeps working, it simply no longer has the row. Carry your `version` as `expect` on every
+later write so a lapsed holder fails with `reason: 'version'` instead of overwriting the new one.
+
+### Timers
+
+```javascript
+// Fire no earlier than 30 minutes from now, into a real queue, through the log.
+const res = await queen.timer('reminders')
+  .key(`order:${orderId}`)
+  .delay('30m')                    // or .delayMs(250)
+  .payload({ orderId })
+  .schedule()                      // status: 'scheduled' | 'rescheduled' | 'too_late'
+
+await queen.timer('reminders').key(`order:${orderId}`).peek()
+await queen.timer('reminders').list({ limit: 50 })
+await queen.timer('reminders').key(`order:${orderId}`).cancel()
+```
+
+Scheduling the same `(queue, timerKey)` again is the same upsert, so a retry after a client crash
+is safe by construction and `status` says which it was. A reschedule mints a new `txn` and resets
+the retry budget.
+
+Durations that can be sub-second are in **milliseconds** (`delayMs`), the ones that cannot are in
+**seconds** (`ttlSeconds`). Only relative delays exist, because there is one clock and it is the
+database's. A delay in the past is legal and fires on the first cycle.
+
+`deliverAt` is **"not before"**, never "exactly at".
+
+**`absent` means "no longer pending" and may mean ALREADY DELIVERED.** There is no tombstone: a
+fired timer has no row left, so `absent` carries `ok: false` and the answer echoes the `txn` so the
+authority, the log, can be consulted without a second API. A saga that cancels a compensation timer
+must have the compensating consumer re-check the saga's KV state before compensating, because the
+cancel may have arrived 5 ms after the fire.
+
+Use `queen.timer(q).key(k).cancel()` rather than a cancel inside a bundle when the cancel must land
+regardless: it takes the DELETE route, the one a quota is forbidden to block. A tenant that cannot
+cancel keeps producing messages it cannot stop.
+
+### Inside a transaction
+
+The transaction is the **primary fence**; `expect` is only the secondary assertion. A state write
+that shares the transaction with its ack is undone when an expired lease makes the ack fail, which
+a compare-and-set cannot do.
+
+```javascript
+const result = await queen.transaction()
+  .ack(message)
+  .queue('emails').push([{ data: mail }])
+  .once('sent', message.transactionId, { ttl: '24h' })   // the gate
+  .timer('reminders').key(orderId).delay('24h').payload({ orderId }).schedule()
+  .commit()
+
+if (result.success === false) {
+  // RETURNED, not thrown: a lost gate is the expected outcome of a legitimate
+  // redelivery, so it stays out of your retry policy and your error metrics.
+  result.reason        // 'kv_precondition'
+  result.failedIndex, result.kvReason, result.version, result.value
+  return
+}
+```
+
+`once` is `putIfAbsent` with `required: true`, and `required` is what makes it a gate: without it a
+lost precondition is only a verdict in the results, and the push and the ack still go through.
+
+`kv.getPrefix` is not available inside a transaction and throws here rather than at the broker: its
+cost is not bounded by the caller. `get` and `getMany` are allowed, because they are. Everything
+other than the lost precondition still throws.
+
+---
+
 ## Examples
 
 ### Complete Pipeline with Consumer Groups
@@ -521,6 +661,41 @@ await queen.transaction()
   .queue('output')
   .push([{ data: { result: 'processed' } }])
   .commit()
+
+// Riders. commit() RETURNS on a lost gate, and throws on everything else.
+await queen.transaction()
+  .ack(message)
+  .kv.put('saga', sagaId, { step: 'charged' }, { ttl: '24h' })
+  .once('dedup', message.transactionId, { ttl: '24h' })
+  .timer('reminders').key(orderId).delay('30m').payload({ orderId }).schedule()
+  .commit()
+```
+
+### Key/Value
+
+```javascript
+await queen.kv.get(ns, key)                      // {found, key, value, version, expiresAt, updatedAt}
+await queen.kv.getMany(ns, [k1, k2])             // {rows, missing, truncated}
+await queen.kv.getPrefix(ns, prefix, { limit: 100, after, keysOnly })
+await queen.kv.put(ns, key, value, { ttl: '24h', expect, required })
+await queen.kv.putIfAbsent(ns, key, value, { ttlSeconds: 86400 })
+await queen.kv.delete(ns, key, { expect })
+await queen.kv.incr(ns, key, 1, { max: 1000, min, ttl: '1h' })
+await queen.kv.once(ns, key, { ttl: '24h' })     // {won, value, version, result}
+for await (const row of queen.kv.listAll(ns, prefix)) { }
+```
+
+### Timers
+
+```javascript
+// Builder steps: .key(timerKey) required, .delayMs(250) or .delay('30m') required,
+// .payload(anyJsonOrBuffer) required to schedule, .partition(name) optional
+// (defaults to 'Default'), .txn(transactionId) optional (minted when absent).
+
+await queen.timer(q).key(k).delay('30m').payload(p).schedule()   // {ok, status, txn, messageId, deliverAt}
+await queen.timer(q).key(k).cancel()                             // {ok, status, txn}
+await queen.timer(q).key(k).peek()                               // {found, ...}
+await queen.timer(q).list({ limit: 50, after })                  // {rows, truncated, nextAfter}
 ```
 
 ### Lease Renewal

package/client-v2/Queen.js

@@ -8,6 +8,8 @@ import { LoadBalancer } from './http/LoadBalancer.js'
 import { BufferManager } from './buffer/BufferManager.js'
 import { QueueBuilder } from './builders/QueueBuilder.js'
 import { TransactionBuilder } from './builders/TransactionBuilder.js'
+import { TimerBuilder } from './builders/TimerBuilder.js'
+import { Kv } from './kv/Kv.js'
 import { StreamBuilder } from './stream/StreamBuilder.js'
 import { StreamConsumer } from './stream/StreamConsumer.js'
 import { Admin } from './admin/Admin.js'
@@ -15,12 +17,49 @@ import { CLIENT_DEFAULTS } from './utils/defaults.js'
 import { validateUrl, validateUrls } from './utils/validation.js'
 import * as logger from './utils/logger.js'
 
+// Both /api/v1/ack and /api/v1/ack/batch respond with a top-level JSON array,
+// one item per acknowledgment in request order:
+//   [{index, transactionId, success, error, queueName, partitionName, leaseReleased, dlq}]
+// (see queen.ack_messages_v2 / routes/ack.cpp). A rejected ack/nack (e.g.
+// "Invalid or expired lease") still arrives as HTTP 200 with success=false on
+// the item, so the per-item flag is the only signal that the broker accepted it.
+
+function normalizeAckItem(item, index) {
+  if (item === null || typeof item !== 'object') {
+    throw new Error(`Unexpected ack result item at index ${index}`)
+  }
+  let success = typeof item.success === 'boolean' ? item.success : true
+  let error = typeof item.error === 'string' && item.error.length > 0 ? item.error : null
+  if (error) success = false
+  if (!success && !error) error = 'Acknowledgment rejected by server'
+  return { ...item, success, error }
+}
+
+// expected is the number of acknowledgments sent, so a truncated or misaligned
+// response fails loudly instead of being misattributed to the wrong message.
+function parseAckResults(result, expected) {
+  if (Array.isArray(result)) {
+    if (result.length !== expected) {
+      throw new Error(`Ack response has ${result.length} results, expected ${expected}`)
+    }
+    return result.map(normalizeAckItem)
+  }
+
+  // Top-level error envelope: the whole request was rejected.
+  if (result && typeof result === 'object' && typeof result.error === 'string' && result.error.length > 0) {
+    return Array.from({ length: expected }, () => ({ success: false, error: result.error }))
+  }
+
+  throw new Error('Unexpected ack response format: missing per-item result array')
+}
+
 export class Queen {
   #httpClient
   #bufferManager
   #config
   #shutdownHandlers = []
   #admin = null
+  #kv = null
 
   constructor(config = {}) {
     // Configure custom logger before anything else.
@@ -87,7 +126,7 @@ export class Queen {
   }
 
   #createHttpClient() {
-    const { urls, timeoutMillis, retryAttempts, retryDelayMillis, loadBalancingStrategy, affinityHashRing, healthRetryAfterMillis, enableFailover, bearerToken, headers } = this.#config
+    const { urls, timeoutMillis, retryAttempts, retryDelayMillis, loadBalancingStrategy, affinityHashRing, healthRetryAfterMillis, enableFailover, bearerToken, headers, hostHeader, retry429 } = this.#config
 
     if (urls.length === 1) {
       // Single server
@@ -97,7 +136,9 @@ export class Queen {
         retryAttempts,
         retryDelayMillis,
         bearerToken,
-        headers
+        headers,
+        hostHeader,
+        retry429
       })
     }
 
@@ -113,7 +154,9 @@ export class Queen {
       retryDelayMillis,
       enableFailover,
       bearerToken,
-      headers
+      headers,
+      hostHeader,
+      retry429
     })
   }
 
@@ -176,6 +219,57 @@ export class Queen {
     return this.#admin
   }
 
+  // ===========================
+  // KV API Entry Point
+  // ===========================
+
+  /**
+   * Transactional key/value state, alongside the log
+   * (PLAN_KV_TIMERS.md §5).
+   *
+   *     const { won } = await queen.kv.once('idem', orderId, { ttl: '24h' })
+   *     const row = await queen.kv.get('saga', sagaId)   // {found, value, version, ...}
+   *
+   * Two things to carry into every use of it:
+   *   * every write returns an OBJECT, and objects are always truthy --
+   *     `if (await queen.kv.delete(ns, key))` is always true. Read `.applied`.
+   *   * an expiry is MANDATORY on every write: exactly one of `ttlSeconds`
+   *     (or the sugar `ttl` / `until`) and `forever: true`. There is no
+   *     default, because a default is how a marker becomes immortal.
+   *
+   * Lazily initialized, singleton, like `admin`.
+   * @returns {Kv}
+   */
+  get kv() {
+    if (!this.#kv) {
+      this.#kv = new Kv(this.#httpClient)
+    }
+    return this.#kv
+  }
+
+  // ===========================
+  // Timers API Entry Point
+  // ===========================
+
+  /**
+   * Scheduled messages for one queue (PLAN_KV_TIMERS.md §4).
+   *
+   *     await queen.timer('orders').key(orderId).delay('30m')
+   *                .payload({ orderId }).schedule()
+   *     await queen.timer('orders').key(orderId).cancel()
+   *
+   * `deliverAt` is "not before", never "exactly at". A cancel that answers
+   * `absent` means "no longer pending" and MAY MEAN ALREADY DELIVERED -- there
+   * is no tombstone, and the authority is the log.
+   *
+   * @param {string} queueName - destination queue (mandatory)
+   * @returns {TimerBuilder}
+   */
+  timer(queueName) {
+    logger.log('Queen.timer', { queue: queueName })
+    return new TimerBuilder(this.#httpClient, queueName)
+  }
+
   // ===========================
   // Transaction API
   // ===========================
@@ -195,7 +289,7 @@ export class Queen {
     // Handle batch acknowledgment
     if (Array.isArray(message)) {
       if (message.length === 0) {
-        return { processed: 0, results: [] }
+        return { success: true, processed: 0, results: [] }
       }
 
       // Check if messages have individual status
@@ -277,13 +371,19 @@ export class Queen {
           consumerGroup: context.group || null
         })
 
-        if (result && result.error) {
-          logger.error('Queen.ack', { type: 'batch', error: result.error })
-          return { success: false, error: result.error }
+        const results = parseAckResults(result, acknowledgments.length)
+        const failed = results.filter(r => !r.success)
+
+        if (failed.length > 0) {
+          const error = failed.length === 1
+            ? failed[0].error
+            : `${failed.length} of ${results.length} acknowledgments rejected: ${failed[0].error}`
+          logger.error('Queen.ack', { type: 'batch', error, failed: failed.length, count: results.length })
+          return { success: false, error, results }
         }
 
-        logger.log('Queen.ack', { type: 'batch', success: true, count: acknowledgments.length })
-        return { success: true, ...result }
+        logger.log('Queen.ack', { type: 'batch', success: true, count: results.length })
+        return { success: true, processed: results.length, results }
       } catch (error) {
         logger.error('Queen.ack', { type: 'batch', error: error.message })
         return { success: false, error: error.message }
@@ -321,13 +421,14 @@ export class Queen {
     try {
       const result = await this.#httpClient.post('/api/v1/ack', body)
 
-      if (result && result.error) {
-        logger.error('Queen.ack', { type: 'single', transactionId, error: result.error })
-        return { success: false, error: result.error }
-      }
+      const [ackResult] = parseAckResults(result, 1)
 
-      logger.log('Queen.ack', { type: 'single', transactionId, success: true })
-      return { success: true, ...result }
+      if (!ackResult.success) {
+        logger.error('Queen.ack', { type: 'single', transactionId, error: ackResult.error })
+      } else {
+        logger.log('Queen.ack', { type: 'single', transactionId, success: true })
+      }
+      return ackResult
     } catch (error) {
       logger.error('Queen.ack', { type: 'single', transactionId, error: error.message })
       return { success: false, error: error.message }
@@ -512,6 +613,15 @@ export class Queen {
     }
     this.#shutdownHandlers = []
 
+    // Release the HTTP agent's keep-alive sockets — without this the Node
+    // event loop stays pinned by open sockets and the process never exits
+    // naturally after close().
+    try {
+      await this.#httpClient.destroy()
+    } catch (error) {
+      logger.warn('Queen.close', { error: error.message, phase: 'http-destroy' })
+    }
+
     logger.log('Queen.close', 'Client closed successfully')
   }
 }

package/client-v2/README.md

@@ -104,6 +104,43 @@ const queen = new Queen({
 })
 ```
 
+### Connecting Through a Hosted Proxy (Cluster Selection)
+
+A Queen proxy deployment routes each request to a tenant cluster using the
+**Host header** (its first DNS label is the cluster slug). Normally you don't
+have to think about it — every cluster has its own hostname, so pointing the
+client at that hostname is all it takes:
+
+```javascript
+const queen = new Queen({
+  url: 'https://acme.eu1.queenmq.cloud',   // Host follows from the URL
+  bearerToken: process.env.QUEEN_API_KEY
+})
+```
+
+When the base URL can't be the cluster's hostname — an IP or a shared cell
+endpoint, a local rig, a staging box, split-horizon DNS — use `hostHeader` to
+advertise the cluster host while the connection still goes to the configured
+address (the same thing `curl --resolve` does; TLS SNI and certificate
+validation follow `hostHeader`, not the address):
+
+```javascript
+const queen = new Queen({
+  url: 'https://10.0.0.7:6711',            // where we connect
+  hostHeader: 'acme.eu1.queenmq.cloud',    // which cluster we ask for
+  bearerToken: process.env.QUEEN_API_KEY
+})
+```
+
+`hostHeader` takes a bare authority (`'acme'`, `'acme.eu1.queenmq.cloud'`,
+`'acme.local:6711'`) — not a URL. It works with `urls: [...]` too: every
+backend is dialed as usual, and all of them get the same Host.
+
+> ⚠️ `headers: { Host: '...' }` **cannot** work in JavaScript: `Host` is a
+> WHATWG forbidden header name and `fetch()` drops it. Rather than let that
+> silently route you to the wrong cluster, the client maps it onto
+> `hostHeader` and warns once. Set `hostHeader` directly.
+
 ---
 
 ## Part 1: Hello Queue!
@@ -1693,6 +1730,38 @@ await queen
   .commit()
 ```
 
+### Key/Value and Timers
+
+Always present on every broker — no flag, nothing to probe. An operator's runtime kill switch can
+pause them (`503` + `Retry-After`, `error: 'kv_disabled'` / `'timers_disabled'`; `403` on a rider
+inside a transaction), which is a pause and not an absence. Full treatment in the
+[client README](../README.md#keyvalue-state-and-timers).
+
+```javascript
+// Every write states its lifetime: exactly one of ttl/ttlSeconds/until and forever.
+await queen.kv.put('orders', 'order:9f1', { state: 'held' }, { ttl: '60s' })
+const row = await queen.kv.get('orders', 'order:9f1')   // {found, value, version, ...}
+
+// Writes return an OBJECT, always truthy. Read .applied, never the result itself.
+const res = await queen.kv.delete('orders', 'order:9f1')
+if (res.applied) { }
+
+const { won } = await queen.kv.once('dedup', eventId, { ttl: '24h' })
+const hit = await queen.kv.incr('quota', key, 1, { max: 1000, ttl: '1h' })  // applied IS admission
+
+await queen.timer('reminders').key(orderId).delay('30m').payload({ orderId }).schedule()
+await queen.timer('reminders').key(orderId).cancel()    // 'absent' may mean already delivered
+
+// The gate: marker, push and ack commit or roll back together.
+const out = await queen
+  .transaction()
+  .ack(message)
+  .queue('emails').push([{ data: mail }])
+  .once('sent', message.transactionId, { ttl: '24h' })
+  .commit()
+if (out.success === false) return   // returned, not thrown: a redelivery already handled
+```
+
 ### Lease Renewal
 
 ```javascript

package/client-v2/builders/QueueBuilder.js

@@ -335,8 +335,10 @@ export class QueueBuilder {
 
       // Generate affinity key for consistent routing to same backend
       const affinityKey = this.#getAffinityKey()
-      
-      const result = await this.#httpClient.get(`${path}?${params}`, this.#timeoutMillis + 5000, affinityKey)
+
+      // wait=true is a long-poll: on 429 it should back off and keep waiting
+      // rather than give up after a handful of tries (retryKind: 'pop').
+      const result = await this.#httpClient.get(`${path}?${params}`, this.#timeoutMillis + 5000, affinityKey, this.#wait ? 'pop' : null)
 
       if (!result || !result.messages) {
         logger.log('QueueBuilder.pop', { status: 'no-messages' })
@@ -347,8 +349,12 @@ export class QueueBuilder {
       logger.log('QueueBuilder.pop', { status: 'success', count: messages.length })
       return messages
     } catch (error) {
-      // Return empty array on error instead of throwing
-      logger.error('QueueBuilder.pop', { error: error.message })
+      // Return empty array on error instead of throwing. This also covers a
+      // 429 whose retry429 policy was exhausted (bounded pop, or an explicit
+      // maxAttempts override) and a terminal 403 (e.g. cluster_suspended) --
+      // both are logged with their `.code` rather than raising, matching this
+      // method's existing swallow-to-[] contract.
+      logger.error('QueueBuilder.pop', { error: error.message, status: error.status, code: error.code })
       return []
     }
   }
@@ -368,22 +374,14 @@ export class QueueBuilder {
     throw new Error('Must specify queue, namespace, or task for pop operation')
   }
 
-  #buildPopParams() {
-    const params = new URLSearchParams({
-      batch: this.#batch.toString(),
-      wait: this.#wait.toString(),
-      timeout: this.#timeoutMillis.toString()  // Server expects 'timeout', not 'timeoutMillis'
-    })
-
-    if (this.#group) params.append('consumerGroup', this.#group)
-    if (this.#namespace) params.append('namespace', this.#namespace)
-    if (this.#task) params.append('task', this.#task)
-    if (this.#autoAck) params.append('autoAck', 'true')
-    if (this.#subscriptionMode) params.append('subscriptionMode', this.#subscriptionMode)
-    if (this.#subscriptionFrom) params.append('subscriptionFrom', this.#subscriptionFrom)
-
-    return params
-  }
+  // NOTE: a second, DEAD copy of the pop parameter builder lived here and was
+  // deleted with the kv/timers work (PLAN_KV_TIMERS.md §10.4). pop() builds its
+  // own params inline, above, because it has to override autoAck with the POP
+  // defaults; the dead copy did not. Anyone adding a parameter by looking for
+  // the method whose name says "build pop params" would have added it to the
+  // copy nobody calls: the pop would keep working and the parameter would
+  // simply never arrive, which reads as a server-side mystery and not as a
+  // client bug.
 
   // ===========================
   // Buffer Management Methods