bunqueue-client 0.1.5 → 0.1.7

package/CHANGELOG.md

@@ -0,0 +1,119 @@
+# Changelog
+
+All notable changes to `bunqueue-client` (TypeScript SDK) are documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.7] - 2026-07-10
+
+Audit fixes: typed worker events, error-path hygiene and two more members of
+the "client drops a wire-supported field" class (#111).
+
+### Added
+
+- **Typed Worker events.** `worker.on('completed', (job, result) => ...)` now
+  gets typed `Job<T>`/`R`/`Error` parameters in strict mode instead of
+  `unknown[]` (TS18046). The new `WorkerEventMap<T, R>` covers `ready`,
+  `active`, `completed`, `failed`, `progress`, `error`, `drained`, `cancelled`
+  and `closed`; unknown event names keep a generic overload, so existing code
+  compiles unchanged. (H1)
+- `"prepublishOnly": "bun run build"` so a publish can never ship a stale
+  `dist/`. (H3)
+
+### Fixed
+
+- **Bunqueue constructor crash vector.** `new Bunqueue(..., { dlq })` fired
+  `setDlqConfig` with no rejection handler: an unreachable server at
+  construction time killed the process with an unhandled rejection. The
+  failure now routes to the worker's `'error'` event (swallowed when no
+  listener is attached, matching `pause()`/`resume()`). (H2)
+- **ACK/completed asymmetry.** In the non-batched path the worker emitted
+  `'completed'` and incremented `processed` even when the ACK never reached
+  the server. Both the ACK and FAIL paths now mirror the batched semantics:
+  on a wire failure only `'error'` fires, with no counter increment. Errors
+  emitted on `'error'` are now always `Error` instances. (M1)
+- **Not-found swallowing.** `getJobScheduler`, `getJob` and
+  `getJobByCustomId` caught every error (including `ConnectionClosedError`
+  and `CommandTimeoutError`) and returned `null`. The catch is narrowed to a
+  `CommandError` matching `/not found/i`; everything else rethrows. (M2)
+- **Scheduler template priority/deduplication dropped.**
+  `upsertJobScheduler` put `priority` inside `jobOptions`, where the server's
+  `CronJobOptions` ignores it, and never sent the template's deduplication.
+  Both now travel as the top-level `priority`/`uniqueKey`/`dedup` Cron fields
+  the handler reads, matching the reference client. (#111 class, F3)
+- **moveJobToFailed lost the stack and the unrecoverable flag.** It sent only
+  `error.message`; when given an `Error` it now sends the leading stack lines
+  and `unrecoverable: true` for `UnrecoverableError`, mirroring the worker
+  FAIL path. (#111 class, F4)
+
+## [0.1.6] - 2026-07-09
+
+Enterprise-grade hardening. All additive and backward-compatible; defaults are
+unchanged (observability is silent, backpressure unbounded, ACK batching off).
+
+### Added
+
+- **Observability.** Every `Connection`/`Queue`/`Worker`/`FlowProducer` accepts
+  an injectable `logger` and an `onTelemetry` sink (zero hard deps — bridge it to
+  OpenTelemetry/Prometheus yourself). `TelemetryEvent` is a typed union covering
+  per-command latency, connect/disconnect/reconnect, auth and backpressure.
+  `Connection` is now an `EventEmitter` emitting `connect` / `disconnect` /
+  `reconnect_scheduled`. Ships `noopLogger` (default) and `consoleLogger`.
+- **Backpressure.** `maxInFlight` bounds concurrent in-flight commands; callers
+  park until a slot frees instead of growing memory unbounded under load.
+- **ACK batching.** Opt-in `Worker({ ackBatch: { enabled: true } })` coalesces
+  completed-job ACKs into `ACKB` round-trips for higher throughput; a job stays
+  active (lock renewed) until its batch is confirmed.
+- **Connection pool.** `Queue({ poolSize: N })` fans producer commands across N
+  round-robin connections (`ConnectionPool`, producer-side; workers stay single-
+  connection by design).
+- **Typed responses.** `call<R>()` is generic over the exported response shapes
+  (`JobResponse`, `PulledJobsResponse`, `JobCountsResponse`, …); internal
+  `as Record<string, unknown>` casts removed across the query/control/flow paths.
+
+### CI
+
+- GitHub Actions runs both SDK suites on every `sdk/`/`src/` change (TypeScript
+  on Bun + Node + Deno, Python 3.10/3.12); an npm release workflow publishes
+  with build provenance, gated on the e2e suite.
+
+## [0.1.5] - 2026-07-08
+
+Protocol-coherence audit against the bunqueue server. Every fix ships with a
+RED→GREEN repro in `tests/e2e-audit-fixes.ts`.
+
+### Fixed
+
+- **addBulk dropped the custom job id.** PUSHB entries are `JobInput`
+  (`customId`), not the single-PUSH `jobId` the server renames — the batch
+  path now renames `jobId`→`customId`, so `getJobByCustomId` and idempotent
+  bulk ingest work. (H1)
+- **Half-open link wedge.** Enable TCP keepalive (~15s idle) and tear down the
+  socket after 3 consecutive command timeouts so the next call reconnects,
+  instead of wedging until the OS abandons the writes. The teardown is
+  generation-guarded so a stale-connection timeout can't abort a fresh
+  reconnect. (H2)
+- **getFlow crashed on a missing job.** A missing root/child now yields `null`
+  and is skipped (partial tree) instead of throwing; the catch is narrowed to
+  `'not found'` so real server errors still surface, and a `visited` set guards
+  against cycles now that depth defaults to unlimited. (H4)
+- **waitForJob returned `undefined` on timeout.** It now rejects on
+  non-completion: a `failed` job throws `CommandError`, otherwise
+  `CommandTimeoutError` — the `completed` flag is no longer ignored. (M1)
+- **getWaitingCount / getWaiting counted prioritized jobs.** Now waiting-only,
+  matching BullMQ and the Python SDK. (M2)
+
+### Changed
+
+- `addJobLog(id, message, level?)` accepts an optional level;
+  `getJobLogs` formats entries as `[level] message` (no longer drops the level).
+- `retryJobs`: the dead `count` field is no longer sent on the wire (the server
+  has no partial RetryDlq; `count` is accepted only for API parity).
+- Worker `FAIL` keeps the leading stack lines (`slice(0, N)`) so the error
+  message is preserved on long stacks.
+
+## [0.1.4] - initial published release
+
+- Cross-runtime (Node/Bun/Deno) TCP client: `Queue`, `Worker`, `FlowProducer`,
+  `Bunqueue` Simple Mode, msgpack wire protocol, TLS, auth, pipelining.

package/README.md

@@ -8,9 +8,9 @@ The bunqueue server runs on Bun, distributed as a binary or a Docker image. This
 
 | Runtime | Status | Notes |
 |---|---|---|
-| Node.js 20 or later | Supported, 81/81 e2e and 8/8 integration tests | ESM. TypeScript files run directly on Node 22 or later via `--experimental-strip-types` |
-| Bun | Supported, 81/81 e2e and 8/8 integration tests | No additional configuration required |
-| Deno 2 or later | Supported, 81/81 e2e and 8/8 integration tests | Uses `node:` builtins and the npm `msgpackr` package |
+| Node.js 20 or later | Supported, 100/100 e2e and 8/8 integration tests | ESM. TypeScript files run directly on Node 22 or later via `--experimental-strip-types` |
+| Bun | Supported, 100/100 e2e and 8/8 integration tests | No additional configuration required |
+| Deno 2 or later | Supported, 100/100 e2e and 8/8 integration tests | Uses `node:` builtins and the npm `msgpackr` package |
 | tsx, ts-node, vitest, jest | Supported | These environments execute on Node.js |
 | Cloudflare Workers | Supported, 16/16 e2e tests inside workerd, including Simple Mode and the full API surface | Requires the `nodejs_compat` compatibility flag. The runtime is request scoped, so long lived worker loops are not available: consume in batches from Cron Triggers or Durable Object alarms, a pattern covered by the test suite. TLS connections require a publicly trusted certificate |
 | Browser | Not supported | Raw TCP sockets are unavailable. Use the server HTTP API instead |
@@ -272,6 +272,49 @@ const queue = new Queue('emails', {
 
 Authentication uses server side tokens (`AUTH_TOKENS`). Transport security uses native TLS, with support for system certificate authorities, a custom CA bundle, or disabled verification for development environments.
 
+## Observability
+
+Inject a logger and a telemetry sink to bridge the client into your stack. There are no hard dependencies — you wire OpenTelemetry, Prometheus or your own logger. Defaults are silent.
+
+```typescript
+import { Queue, consoleLogger, type TelemetryEvent } from 'bunqueue-client';
+
+const queue = new Queue('emails', {
+  logger: consoleLogger('info'), // or your own { debug, info, warn, error }
+  onTelemetry: (e: TelemetryEvent) => {
+    // per-command latency, connect/disconnect/reconnect, auth, backpressure
+    if (e.type === 'command') metrics.observe(e.cmd, e.durationMs, e.ok);
+  },
+});
+
+// Connection is an EventEmitter for lifecycle hooks:
+queue.connection.on('reconnect_scheduled', (i) => log.warn('reconnecting', i));
+queue.connection.on('disconnect', () => log.warn('link down'));
+```
+
+## Throughput and resilience
+
+```typescript
+// Bound in-flight commands (backpressure) — parks callers instead of growing memory:
+const queue = new Queue('emails', { maxInFlight: 10_000 });
+
+// Fan producer commands across N connections (round-robin, producer-side):
+const pooled = new Queue('emails', { poolSize: 4 });
+
+// Batch worker ACKs into ACKB round-trips (opt-in) for high-volume consumers:
+const worker = new Worker('emails', process, {
+  ackBatch: { enabled: true, maxSize: 50, maxDelayMs: 5 },
+});
+```
+
+Always attach a `worker.on('error', …)` listener: per Node `EventEmitter` semantics an unhandled `error` event throws. The worker frees each job's concurrency slot before emitting, so even a throwing listener cannot degrade throughput — but the error itself is yours to observe.
+
+Half-open links are detected via TCP keepalive and a consecutive-timeout teardown, so a silently dropped connection (cloud LB/NAT idle drop) recovers in seconds rather than minutes.
+
+## Typed responses
+
+`connection.call<R>()` and `queue.call<R>()` are generic over the exported response shapes (`JobResponse`, `PulledJobsResponse`, `JobCountsResponse`, `WaitJobResponse`, …), so raw command access is fully typed without casting.
+
 ## API surface
 
 | Area | Capabilities |

package/dist/ack-batcher.d.ts

@@ -0,0 +1,28 @@
+/**
+ * Batches completed-job ACKs into ACKB round-trips for higher throughput.
+ *
+ * Opt-in (see WorkerOptions.ackBatch). Items flush when the buffer reaches
+ * `maxSize` or after `maxDelayMs`, whichever comes first; `flush()` drains the
+ * rest on shutdown. Each item's `onSettled` fires after the batch is acked
+ * (or errored), so the worker keeps a job "active" — and its lock renewed —
+ * until the server has confirmed the ack.
+ */
+import type { Connection } from './connection.js';
+export interface AckItem {
+    id: string;
+    token: string;
+    result: unknown;
+    /** Called once the batch settles: `err` set on failure, undefined on ack. */
+    onSettled: (err?: unknown) => void;
+}
+export declare class AckBatcher {
+    private readonly connection;
+    private readonly maxSize;
+    private readonly maxDelayMs;
+    private buffer;
+    private timer;
+    constructor(connection: Connection, maxSize: number, maxDelayMs: number);
+    add(item: AckItem): void;
+    /** Send the buffered ACKs as one ACKB (no-op when empty). */
+    flush(): Promise<void>;
+}

package/dist/ack-batcher.js

@@ -0,0 +1,67 @@
+/**
+ * Batches completed-job ACKs into ACKB round-trips for higher throughput.
+ *
+ * Opt-in (see WorkerOptions.ackBatch). Items flush when the buffer reaches
+ * `maxSize` or after `maxDelayMs`, whichever comes first; `flush()` drains the
+ * rest on shutdown. Each item's `onSettled` fires after the batch is acked
+ * (or errored), so the worker keeps a job "active" — and its lock renewed —
+ * until the server has confirmed the ack.
+ */
+import { compact } from './frame.js';
+export class AckBatcher {
+    connection;
+    maxSize;
+    maxDelayMs;
+    buffer = [];
+    timer = null;
+    constructor(connection, maxSize, maxDelayMs) {
+        this.connection = connection;
+        this.maxSize = maxSize;
+        this.maxDelayMs = maxDelayMs;
+    }
+    add(item) {
+        this.buffer.push(item);
+        if (this.buffer.length >= this.maxSize) {
+            void this.flush();
+        }
+        else if (!this.timer) {
+            this.timer = setTimeout(() => void this.flush(), this.maxDelayMs);
+            this.timer.unref?.(); // don't keep the process alive for a pending flush
+        }
+    }
+    /** Send the buffered ACKs as one ACKB (no-op when empty). */
+    async flush() {
+        if (this.timer) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        if (this.buffer.length === 0)
+            return;
+        const batch = this.buffer;
+        this.buffer = [];
+        // Capture the wire outcome first; settle callbacks run OUTSIDE the try so
+        // a throwing callback can neither be re-invoked with an error (double
+        // settle) nor starve the remaining items of their callback.
+        let error;
+        try {
+            await this.connection.call(compact({
+                cmd: 'ACKB',
+                ids: batch.map((b) => b.id),
+                tokens: batch.map((b) => b.token),
+                results: batch.map((b) => b.result),
+            }));
+        }
+        catch (err) {
+            error = err ?? new Error('ACKB failed');
+        }
+        for (const item of batch) {
+            try {
+                item.onSettled(error);
+            }
+            catch {
+                // A callback error (e.g. an unhandled 'error' emit in the worker)
+                // must never prevent the other items from settling.
+            }
+        }
+    }
+}

package/dist/backpressure.d.ts

@@ -0,0 +1,22 @@
+/**
+ * Backpressure gate: bounds the number of in-flight commands on a connection.
+ *
+ * When the in-flight count reaches `max`, `acquire()` parks the caller until a
+ * slot frees (a pending command settles). This turns unbounded memory growth
+ * under a fast producer / slow server into cooperative flow control. `max <= 0`
+ * disables the gate (unbounded, the default).
+ */
+export declare class Backpressure {
+    private readonly max;
+    private readonly onWait?;
+    private readonly waiters;
+    constructor(max: number, onWait?: (() => void) | undefined);
+    /** Resolve immediately if under the limit, else park until a slot frees. */
+    acquire(inFlight: number): Promise<void> | void;
+    /** Release one parked caller (call when a pending command settles). */
+    release(): void;
+    /** Release every parked caller (call on teardown so none hang forever). */
+    clear(): void;
+    /** Number of callers currently parked. */
+    get waiting(): number;
+}

package/dist/backpressure.js

@@ -0,0 +1,39 @@
+/**
+ * Backpressure gate: bounds the number of in-flight commands on a connection.
+ *
+ * When the in-flight count reaches `max`, `acquire()` parks the caller until a
+ * slot frees (a pending command settles). This turns unbounded memory growth
+ * under a fast producer / slow server into cooperative flow control. `max <= 0`
+ * disables the gate (unbounded, the default).
+ */
+export class Backpressure {
+    max;
+    onWait;
+    waiters = [];
+    constructor(max, onWait) {
+        this.max = max;
+        this.onWait = onWait;
+    }
+    /** Resolve immediately if under the limit, else park until a slot frees. */
+    acquire(inFlight) {
+        if (this.max <= 0 || inFlight < this.max)
+            return;
+        this.onWait?.();
+        return new Promise((resolve) => {
+            this.waiters.push(resolve);
+        });
+    }
+    /** Release one parked caller (call when a pending command settles). */
+    release() {
+        this.waiters.shift()?.();
+    }
+    /** Release every parked caller (call on teardown so none hang forever). */
+    clear() {
+        while (this.waiters.length > 0)
+            this.waiters.shift()?.();
+    }
+    /** Number of callers currently parked. */
+    get waiting() {
+        return this.waiters.length;
+    }
+}

package/dist/bunqueue/bunqueue.js

@@ -76,8 +76,18 @@ export class Bunqueue {
         });
         // DLQ & rate limit manager
         this.dlqrl = new DlqRateLimitManager(this.queue);
-        if (opts.dlq)
-            void this.dlqrl.setDlqConfig(opts.dlq);
+        // Fire-and-forget config push: without the catch, an unreachable server at
+        // construction time becomes an unhandled rejection that kills the process.
+        // Route the failure to the worker's 'error' event (the channel every other
+        // background command failure uses); with no listener attached, swallow it
+        // like pause()/resume() do — an unlistened 'error' emit would itself throw.
+        if (opts.dlq) {
+            void this.dlqrl.setDlqConfig(opts.dlq).catch((err) => {
+                if (this.worker.listenerCount('error') > 0) {
+                    this.worker.emit('error', err instanceof Error ? err : new Error(String(err)));
+                }
+            });
+        }
         // Subsystems
         this.cb = opts.circuitBreaker
             ? new WorkerCircuitBreaker(opts.circuitBreaker, this.worker)

package/dist/connection-pool.d.ts

@@ -0,0 +1,30 @@
+/**
+ * A round-robin pool of TCP connections for producer-side throughput.
+ *
+ * Each `call()` is dispatched to the next connection, so independent commands
+ * (PUSH, queries, control) fan out across N sockets instead of head-of-lining
+ * on one. Use it for {@link Queue} / {@link FlowProducer}; NOT for a Worker,
+ * whose pull→ack flow is bound to a single connection's lock ownership and
+ * reconnect/registration lifecycle.
+ *
+ * Satisfies {@link ConnectionLike}, so it is a drop-in for a single Connection.
+ * Lifecycle events from every member connection are re-emitted here.
+ */
+import { EventEmitter } from 'node:events';
+import type { Command, ConnectionLike, ConnectionOptions, Response } from './connection-types.js';
+export declare class ConnectionPool extends EventEmitter implements ConnectionLike {
+    private readonly connections;
+    private cursor;
+    constructor(size: number, options?: ConnectionOptions);
+    /** How many connections back this pool. */
+    get size(): number;
+    private next;
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
+    ping(): Promise<boolean>;
+    connect(): Promise<void>;
+    close(): void;
+    /** True while at least one member connection is up. */
+    get isConnected(): boolean;
+    /** Representative generation (first connection); pools are producer-side. */
+    get generation(): number;
+}

package/dist/connection-pool.js

@@ -0,0 +1,60 @@
+/**
+ * A round-robin pool of TCP connections for producer-side throughput.
+ *
+ * Each `call()` is dispatched to the next connection, so independent commands
+ * (PUSH, queries, control) fan out across N sockets instead of head-of-lining
+ * on one. Use it for {@link Queue} / {@link FlowProducer}; NOT for a Worker,
+ * whose pull→ack flow is bound to a single connection's lock ownership and
+ * reconnect/registration lifecycle.
+ *
+ * Satisfies {@link ConnectionLike}, so it is a drop-in for a single Connection.
+ * Lifecycle events from every member connection are re-emitted here.
+ */
+import { EventEmitter } from 'node:events';
+import { Connection } from './connection.js';
+import { LIFECYCLE_EVENTS } from './observability.js';
+export class ConnectionPool extends EventEmitter {
+    connections;
+    cursor = 0;
+    constructor(size, options = {}) {
+        super();
+        const n = Math.max(1, Math.floor(size));
+        this.connections = Array.from({ length: n }, () => {
+            const conn = new Connection(options);
+            for (const event of LIFECYCLE_EVENTS) {
+                conn.on(event, (payload) => this.emit(event, payload));
+            }
+            return conn;
+        });
+    }
+    /** How many connections back this pool. */
+    get size() {
+        return this.connections.length;
+    }
+    next() {
+        const conn = this.connections[this.cursor];
+        this.cursor = (this.cursor + 1) % this.connections.length;
+        return conn;
+    }
+    call(command, timeoutMs) {
+        return this.next().call(command, timeoutMs);
+    }
+    ping() {
+        return this.connections[0].ping();
+    }
+    async connect() {
+        await Promise.all(this.connections.map((conn) => conn.connect()));
+    }
+    close() {
+        for (const conn of this.connections)
+            conn.close();
+    }
+    /** True while at least one member connection is up. */
+    get isConnected() {
+        return this.connections.some((conn) => conn.isConnected);
+    }
+    /** Representative generation (first connection); pools are producer-side. */
+    get generation() {
+        return this.connections[0].generation;
+    }
+}

package/dist/connection-types.d.ts

@@ -1,15 +1,21 @@
 /** Connection option and message types. */
+import type { Observability } from './observability.js';
 export type TlsOption = boolean | {
     caFile?: string;
     rejectUnauthorized?: boolean;
 } | undefined;
-export interface ConnectionOptions {
+export interface ConnectionOptions extends Observability {
     host?: string;
     port?: number;
     token?: string;
     tls?: TlsOption;
     connectTimeoutMs?: number;
     commandTimeoutMs?: number;
+    /**
+     * Max in-flight commands before {@link Connection.call} applies backpressure
+     * (awaits a free slot). 0 / undefined = unbounded. Bounds memory under load.
+     */
+    maxInFlight?: number;
 }
 export type Command = Record<string, unknown> & {
     cmd: string;
@@ -17,6 +23,20 @@ export type Command = Record<string, unknown> & {
 export type Response = Record<string, unknown> & {
     ok: boolean;
 };
+/**
+ * The connection surface consumers (Queue, FlowProducer, Job) depend on.
+ * Both {@link Connection} and a round-robin ConnectionPool satisfy it, so a
+ * pool can be dropped in transparently for producer-side throughput.
+ */
+export interface ConnectionLike {
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
+    ping(): Promise<boolean>;
+    connect(): Promise<void>;
+    close(): void;
+    readonly isConnected: boolean;
+    readonly generation: number;
+    on(event: string, listener: (...args: unknown[]) => void): this;
+}
 export interface Pending {
     resolve: (response: Response) => void;
     reject: (error: Error) => void;

package/dist/connection.d.ts

@@ -5,10 +5,17 @@
  * pipelining (many in-flight commands per socket). Uses only `node:`
  * builtins (net/tls), which Node, Bun and Deno all support.
  */
+import { EventEmitter } from 'node:events';
 import type { Command, ConnectionOptions, Response, TlsOption } from './connection-types.js';
 export type { Command, ConnectionOptions, Response, TlsOption } from './connection-types.js';
-/** A single pipelined TCP connection to a bunqueue server. */
-export declare class Connection {
+/**
+ * A single pipelined TCP connection to a bunqueue server.
+ *
+ * Emits typed lifecycle events for observability: `connect`, `disconnect` and
+ * `reconnect_scheduled` (payloads mirror the telemetry events). Attach a
+ * telemetry sink via the `onTelemetry` option for per-command latency metrics.
+ */
+export declare class Connection extends EventEmitter {
     readonly host: string;
     readonly port: number;
     readonly token: string | undefined;
@@ -27,6 +34,8 @@ export declare class Connection {
     private nextAttemptAt;
     private readonly maxCommandTimeouts;
     private consecutiveTimeouts;
+    private readonly telemetry;
+    private readonly backpressure;
     constructor(options?: ConnectionOptions);
     get isConnected(): boolean;
     /**
@@ -42,7 +51,7 @@ export declare class Connection {
      * Send a command and await its response. Rejects with CommandError when
      * the server answers ok=false. Reconnects lazily if the link was lost.
      */
-    call(command: Command, timeoutMs?: number): Promise<Response>;
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
     /** Ping the server; returns true when it answers pong. */
     ping(): Promise<boolean>;
     /** Protocol negotiation; returns server name/version/protocolVersion. */