bunqueue-client 0.1.4 → 0.1.6

package/CHANGELOG.md

@@ -0,0 +1,77 @@
+# Changelog
+
+All notable changes to `bunqueue-client` (TypeScript SDK) are documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.6] - 2026-07-09
+
+Enterprise-grade hardening. All additive and backward-compatible; defaults are
+unchanged (observability is silent, backpressure unbounded, ACK batching off).
+
+### Added
+
+- **Observability.** Every `Connection`/`Queue`/`Worker`/`FlowProducer` accepts
+  an injectable `logger` and an `onTelemetry` sink (zero hard deps — bridge it to
+  OpenTelemetry/Prometheus yourself). `TelemetryEvent` is a typed union covering
+  per-command latency, connect/disconnect/reconnect, auth and backpressure.
+  `Connection` is now an `EventEmitter` emitting `connect` / `disconnect` /
+  `reconnect_scheduled`. Ships `noopLogger` (default) and `consoleLogger`.
+- **Backpressure.** `maxInFlight` bounds concurrent in-flight commands; callers
+  park until a slot frees instead of growing memory unbounded under load.
+- **ACK batching.** Opt-in `Worker({ ackBatch: { enabled: true } })` coalesces
+  completed-job ACKs into `ACKB` round-trips for higher throughput; a job stays
+  active (lock renewed) until its batch is confirmed.
+- **Connection pool.** `Queue({ poolSize: N })` fans producer commands across N
+  round-robin connections (`ConnectionPool`, producer-side; workers stay single-
+  connection by design).
+- **Typed responses.** `call<R>()` is generic over the exported response shapes
+  (`JobResponse`, `PulledJobsResponse`, `JobCountsResponse`, …); internal
+  `as Record<string, unknown>` casts removed across the query/control/flow paths.
+
+### CI
+
+- GitHub Actions runs both SDK suites on every `sdk/`/`src/` change (TypeScript
+  on Bun + Node, Python 3.10/3.12); an npm release workflow publishes with
+  build provenance.
+
+## [0.1.5] - 2026-07-08
+
+Protocol-coherence audit against the bunqueue server. Every fix ships with a
+RED→GREEN repro in `tests/e2e-audit-fixes.ts`.
+
+### Fixed
+
+- **addBulk dropped the custom job id.** PUSHB entries are `JobInput`
+  (`customId`), not the single-PUSH `jobId` the server renames — the batch
+  path now renames `jobId`→`customId`, so `getJobByCustomId` and idempotent
+  bulk ingest work. (H1)
+- **Half-open link wedge.** Enable TCP keepalive (~15s idle) and tear down the
+  socket after 3 consecutive command timeouts so the next call reconnects,
+  instead of wedging until the OS abandons the writes. The teardown is
+  generation-guarded so a stale-connection timeout can't abort a fresh
+  reconnect. (H2)
+- **getFlow crashed on a missing job.** A missing root/child now yields `null`
+  and is skipped (partial tree) instead of throwing; the catch is narrowed to
+  `'not found'` so real server errors still surface, and a `visited` set guards
+  against cycles now that depth defaults to unlimited. (H4)
+- **waitForJob returned `undefined` on timeout.** It now rejects on
+  non-completion: a `failed` job throws `CommandError`, otherwise
+  `CommandTimeoutError` — the `completed` flag is no longer ignored. (M1)
+- **getWaitingCount / getWaiting counted prioritized jobs.** Now waiting-only,
+  matching BullMQ and the Python SDK. (M2)
+
+### Changed
+
+- `addJobLog(id, message, level?)` accepts an optional level;
+  `getJobLogs` formats entries as `[level] message` (no longer drops the level).
+- `retryJobs`: the dead `count` field is no longer sent on the wire (the server
+  has no partial RetryDlq; `count` is accepted only for API parity).
+- Worker `FAIL` keeps the leading stack lines (`slice(0, N)`) so the error
+  message is preserved on long stacks.
+
+## [0.1.4] - initial published release
+
+- Cross-runtime (Node/Bun/Deno) TCP client: `Queue`, `Worker`, `FlowProducer`,
+  `Bunqueue` Simple Mode, msgpack wire protocol, TLS, auth, pipelining.

package/README.md

@@ -272,12 +272,55 @@ const queue = new Queue('emails', {
 
 Authentication uses server side tokens (`AUTH_TOKENS`). Transport security uses native TLS, with support for system certificate authorities, a custom CA bundle, or disabled verification for development environments.
 
+## Observability
+
+Inject a logger and a telemetry sink to bridge the client into your stack. There are no hard dependencies — you wire OpenTelemetry, Prometheus or your own logger. Defaults are silent.
+
+```typescript
+import { Queue, consoleLogger, type TelemetryEvent } from 'bunqueue-client';
+
+const queue = new Queue('emails', {
+  logger: consoleLogger('info'), // or your own { debug, info, warn, error }
+  onTelemetry: (e: TelemetryEvent) => {
+    // per-command latency, connect/disconnect/reconnect, auth, backpressure
+    if (e.type === 'command') metrics.observe(e.cmd, e.durationMs, e.ok);
+  },
+});
+
+// Connection is an EventEmitter for lifecycle hooks:
+queue.connection.on('reconnect_scheduled', (i) => log.warn('reconnecting', i));
+queue.connection.on('disconnect', () => log.warn('link down'));
+```
+
+## Throughput and resilience
+
+```typescript
+// Bound in-flight commands (backpressure) — parks callers instead of growing memory:
+const queue = new Queue('emails', { maxInFlight: 10_000 });
+
+// Fan producer commands across N connections (round-robin, producer-side):
+const pooled = new Queue('emails', { poolSize: 4 });
+
+// Batch worker ACKs into ACKB round-trips (opt-in) for high-volume consumers:
+const worker = new Worker('emails', process, {
+  ackBatch: { enabled: true, maxSize: 50, maxDelayMs: 5 },
+});
+```
+
+Always attach a `worker.on('error', …)` listener: per Node `EventEmitter` semantics an unhandled `error` event throws. The worker frees each job's concurrency slot before emitting, so even a throwing listener cannot degrade throughput — but the error itself is yours to observe.
+
+Half-open links are detected via TCP keepalive and a consecutive-timeout teardown, so a silently dropped connection (cloud LB/NAT idle drop) recovers in seconds rather than minutes.
+
+## Typed responses
+
+`connection.call<R>()` and `queue.call<R>()` are generic over the exported response shapes (`JobResponse`, `PulledJobsResponse`, `JobCountsResponse`, `WaitJobResponse`, …), so raw command access is fully typed without casting.
+
 ## API surface
 
 | Area | Capabilities |
 |---|---|
 | Queue | `add`, `addBulk`, full `JobOptions`: priority, delay, attempts, backoff, ttl, timeout, jobId, deduplication, dependsOn, tags, groupId, lifo, removeOnComplete, removeOnFail, durable, repeat, debounce |
-| Query | `getJob`, `getJobByCustomId`, `getJobs` with per state helpers, state, result, progress, `waitForJob`, counts, counts per priority, children values, job logs |
+| Query | `getJob`, `getJobByCustomId`, `getJobs` with per state helpers, state, result, progress, `waitForJob` (throws on timeout, BullMQ contract), counts, counts per priority, children values, job logs |
 | Control | pause, resume, drain, obliterate, clean, remove, discard, promote, `retryJob`, `retryJobs`, move to wait or delayed, change priority or delay, update data, extend lock |
 | Dead letter queue | `getDlq`, `retryDlq`, `purgeDlq`, DLQ configuration |
 | Administration | rate limiting, global concurrency, stall configuration, webhooks, stats, metrics, `listQueues`, `getWorkers` |

package/dist/ack-batcher.d.ts

@@ -0,0 +1,28 @@
+/**
+ * Batches completed-job ACKs into ACKB round-trips for higher throughput.
+ *
+ * Opt-in (see WorkerOptions.ackBatch). Items flush when the buffer reaches
+ * `maxSize` or after `maxDelayMs`, whichever comes first; `flush()` drains the
+ * rest on shutdown. Each item's `onSettled` fires after the batch is acked
+ * (or errored), so the worker keeps a job "active" — and its lock renewed —
+ * until the server has confirmed the ack.
+ */
+import type { Connection } from './connection.js';
+export interface AckItem {
+    id: string;
+    token: string;
+    result: unknown;
+    /** Called once the batch settles: `err` set on failure, undefined on ack. */
+    onSettled: (err?: unknown) => void;
+}
+export declare class AckBatcher {
+    private readonly connection;
+    private readonly maxSize;
+    private readonly maxDelayMs;
+    private buffer;
+    private timer;
+    constructor(connection: Connection, maxSize: number, maxDelayMs: number);
+    add(item: AckItem): void;
+    /** Send the buffered ACKs as one ACKB (no-op when empty). */
+    flush(): Promise<void>;
+}

package/dist/ack-batcher.js

@@ -0,0 +1,67 @@
+/**
+ * Batches completed-job ACKs into ACKB round-trips for higher throughput.
+ *
+ * Opt-in (see WorkerOptions.ackBatch). Items flush when the buffer reaches
+ * `maxSize` or after `maxDelayMs`, whichever comes first; `flush()` drains the
+ * rest on shutdown. Each item's `onSettled` fires after the batch is acked
+ * (or errored), so the worker keeps a job "active" — and its lock renewed —
+ * until the server has confirmed the ack.
+ */
+import { compact } from './frame.js';
+export class AckBatcher {
+    connection;
+    maxSize;
+    maxDelayMs;
+    buffer = [];
+    timer = null;
+    constructor(connection, maxSize, maxDelayMs) {
+        this.connection = connection;
+        this.maxSize = maxSize;
+        this.maxDelayMs = maxDelayMs;
+    }
+    add(item) {
+        this.buffer.push(item);
+        if (this.buffer.length >= this.maxSize) {
+            void this.flush();
+        }
+        else if (!this.timer) {
+            this.timer = setTimeout(() => void this.flush(), this.maxDelayMs);
+            this.timer.unref?.(); // don't keep the process alive for a pending flush
+        }
+    }
+    /** Send the buffered ACKs as one ACKB (no-op when empty). */
+    async flush() {
+        if (this.timer) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        if (this.buffer.length === 0)
+            return;
+        const batch = this.buffer;
+        this.buffer = [];
+        // Capture the wire outcome first; settle callbacks run OUTSIDE the try so
+        // a throwing callback can neither be re-invoked with an error (double
+        // settle) nor starve the remaining items of their callback.
+        let error;
+        try {
+            await this.connection.call(compact({
+                cmd: 'ACKB',
+                ids: batch.map((b) => b.id),
+                tokens: batch.map((b) => b.token),
+                results: batch.map((b) => b.result),
+            }));
+        }
+        catch (err) {
+            error = err ?? new Error('ACKB failed');
+        }
+        for (const item of batch) {
+            try {
+                item.onSettled(error);
+            }
+            catch {
+                // A callback error (e.g. an unhandled 'error' emit in the worker)
+                // must never prevent the other items from settling.
+            }
+        }
+    }
+}

package/dist/backpressure.d.ts

@@ -0,0 +1,22 @@
+/**
+ * Backpressure gate: bounds the number of in-flight commands on a connection.
+ *
+ * When the in-flight count reaches `max`, `acquire()` parks the caller until a
+ * slot frees (a pending command settles). This turns unbounded memory growth
+ * under a fast producer / slow server into cooperative flow control. `max <= 0`
+ * disables the gate (unbounded, the default).
+ */
+export declare class Backpressure {
+    private readonly max;
+    private readonly onWait?;
+    private readonly waiters;
+    constructor(max: number, onWait?: (() => void) | undefined);
+    /** Resolve immediately if under the limit, else park until a slot frees. */
+    acquire(inFlight: number): Promise<void> | void;
+    /** Release one parked caller (call when a pending command settles). */
+    release(): void;
+    /** Release every parked caller (call on teardown so none hang forever). */
+    clear(): void;
+    /** Number of callers currently parked. */
+    get waiting(): number;
+}

package/dist/backpressure.js

@@ -0,0 +1,39 @@
+/**
+ * Backpressure gate: bounds the number of in-flight commands on a connection.
+ *
+ * When the in-flight count reaches `max`, `acquire()` parks the caller until a
+ * slot frees (a pending command settles). This turns unbounded memory growth
+ * under a fast producer / slow server into cooperative flow control. `max <= 0`
+ * disables the gate (unbounded, the default).
+ */
+export class Backpressure {
+    max;
+    onWait;
+    waiters = [];
+    constructor(max, onWait) {
+        this.max = max;
+        this.onWait = onWait;
+    }
+    /** Resolve immediately if under the limit, else park until a slot frees. */
+    acquire(inFlight) {
+        if (this.max <= 0 || inFlight < this.max)
+            return;
+        this.onWait?.();
+        return new Promise((resolve) => {
+            this.waiters.push(resolve);
+        });
+    }
+    /** Release one parked caller (call when a pending command settles). */
+    release() {
+        this.waiters.shift()?.();
+    }
+    /** Release every parked caller (call on teardown so none hang forever). */
+    clear() {
+        while (this.waiters.length > 0)
+            this.waiters.shift()?.();
+    }
+    /** Number of callers currently parked. */
+    get waiting() {
+        return this.waiters.length;
+    }
+}

package/dist/connection-pool.d.ts

@@ -0,0 +1,30 @@
+/**
+ * A round-robin pool of TCP connections for producer-side throughput.
+ *
+ * Each `call()` is dispatched to the next connection, so independent commands
+ * (PUSH, queries, control) fan out across N sockets instead of head-of-lining
+ * on one. Use it for {@link Queue} / {@link FlowProducer}; NOT for a Worker,
+ * whose pull→ack flow is bound to a single connection's lock ownership and
+ * reconnect/registration lifecycle.
+ *
+ * Satisfies {@link ConnectionLike}, so it is a drop-in for a single Connection.
+ * Lifecycle events from every member connection are re-emitted here.
+ */
+import { EventEmitter } from 'node:events';
+import type { Command, ConnectionLike, ConnectionOptions, Response } from './connection-types.js';
+export declare class ConnectionPool extends EventEmitter implements ConnectionLike {
+    private readonly connections;
+    private cursor;
+    constructor(size: number, options?: ConnectionOptions);
+    /** How many connections back this pool. */
+    get size(): number;
+    private next;
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
+    ping(): Promise<boolean>;
+    connect(): Promise<void>;
+    close(): void;
+    /** True while at least one member connection is up. */
+    get isConnected(): boolean;
+    /** Representative generation (first connection); pools are producer-side. */
+    get generation(): number;
+}

package/dist/connection-pool.js

@@ -0,0 +1,60 @@
+/**
+ * A round-robin pool of TCP connections for producer-side throughput.
+ *
+ * Each `call()` is dispatched to the next connection, so independent commands
+ * (PUSH, queries, control) fan out across N sockets instead of head-of-lining
+ * on one. Use it for {@link Queue} / {@link FlowProducer}; NOT for a Worker,
+ * whose pull→ack flow is bound to a single connection's lock ownership and
+ * reconnect/registration lifecycle.
+ *
+ * Satisfies {@link ConnectionLike}, so it is a drop-in for a single Connection.
+ * Lifecycle events from every member connection are re-emitted here.
+ */
+import { EventEmitter } from 'node:events';
+import { Connection } from './connection.js';
+import { LIFECYCLE_EVENTS } from './observability.js';
+export class ConnectionPool extends EventEmitter {
+    connections;
+    cursor = 0;
+    constructor(size, options = {}) {
+        super();
+        const n = Math.max(1, Math.floor(size));
+        this.connections = Array.from({ length: n }, () => {
+            const conn = new Connection(options);
+            for (const event of LIFECYCLE_EVENTS) {
+                conn.on(event, (payload) => this.emit(event, payload));
+            }
+            return conn;
+        });
+    }
+    /** How many connections back this pool. */
+    get size() {
+        return this.connections.length;
+    }
+    next() {
+        const conn = this.connections[this.cursor];
+        this.cursor = (this.cursor + 1) % this.connections.length;
+        return conn;
+    }
+    call(command, timeoutMs) {
+        return this.next().call(command, timeoutMs);
+    }
+    ping() {
+        return this.connections[0].ping();
+    }
+    async connect() {
+        await Promise.all(this.connections.map((conn) => conn.connect()));
+    }
+    close() {
+        for (const conn of this.connections)
+            conn.close();
+    }
+    /** True while at least one member connection is up. */
+    get isConnected() {
+        return this.connections.some((conn) => conn.isConnected);
+    }
+    /** Representative generation (first connection); pools are producer-side. */
+    get generation() {
+        return this.connections[0].generation;
+    }
+}

package/dist/connection-types.d.ts

@@ -1,15 +1,21 @@
 /** Connection option and message types. */
+import type { Observability } from './observability.js';
 export type TlsOption = boolean | {
     caFile?: string;
     rejectUnauthorized?: boolean;
 } | undefined;
-export interface ConnectionOptions {
+export interface ConnectionOptions extends Observability {
     host?: string;
     port?: number;
     token?: string;
     tls?: TlsOption;
     connectTimeoutMs?: number;
     commandTimeoutMs?: number;
+    /**
+     * Max in-flight commands before {@link Connection.call} applies backpressure
+     * (awaits a free slot). 0 / undefined = unbounded. Bounds memory under load.
+     */
+    maxInFlight?: number;
 }
 export type Command = Record<string, unknown> & {
     cmd: string;
@@ -17,6 +23,20 @@ export type Command = Record<string, unknown> & {
 export type Response = Record<string, unknown> & {
     ok: boolean;
 };
+/**
+ * The connection surface consumers (Queue, FlowProducer, Job) depend on.
+ * Both {@link Connection} and a round-robin ConnectionPool satisfy it, so a
+ * pool can be dropped in transparently for producer-side throughput.
+ */
+export interface ConnectionLike {
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
+    ping(): Promise<boolean>;
+    connect(): Promise<void>;
+    close(): void;
+    readonly isConnected: boolean;
+    readonly generation: number;
+    on(event: string, listener: (...args: unknown[]) => void): this;
+}
 export interface Pending {
     resolve: (response: Response) => void;
     reject: (error: Error) => void;

package/dist/connection.d.ts

@@ -5,10 +5,17 @@
  * pipelining (many in-flight commands per socket). Uses only `node:`
  * builtins (net/tls), which Node, Bun and Deno all support.
  */
+import { EventEmitter } from 'node:events';
 import type { Command, ConnectionOptions, Response, TlsOption } from './connection-types.js';
 export type { Command, ConnectionOptions, Response, TlsOption } from './connection-types.js';
-/** A single pipelined TCP connection to a bunqueue server. */
-export declare class Connection {
+/**
+ * A single pipelined TCP connection to a bunqueue server.
+ *
+ * Emits typed lifecycle events for observability: `connect`, `disconnect` and
+ * `reconnect_scheduled` (payloads mirror the telemetry events). Attach a
+ * telemetry sink via the `onTelemetry` option for per-command latency metrics.
+ */
+export declare class Connection extends EventEmitter {
     readonly host: string;
     readonly port: number;
     readonly token: string | undefined;
@@ -25,6 +32,10 @@ export declare class Connection {
     private connectGeneration;
     private failedAttempts;
     private nextAttemptAt;
+    private readonly maxCommandTimeouts;
+    private consecutiveTimeouts;
+    private readonly telemetry;
+    private readonly backpressure;
     constructor(options?: ConnectionOptions);
     get isConnected(): boolean;
     /**
@@ -40,7 +51,7 @@ export declare class Connection {
      * Send a command and await its response. Rejects with CommandError when
      * the server answers ok=false. Reconnects lazily if the link was lost.
      */
-    call(command: Command, timeoutMs?: number): Promise<Response>;
+    call<R = Response>(command: Command, timeoutMs?: number): Promise<R>;
     /** Ping the server; returns true when it answers pong. */
     ping(): Promise<boolean>;
     /** Protocol negotiation; returns server name/version/protocolVersion. */
@@ -48,5 +59,11 @@ export declare class Connection {
     /** Close permanently; in-flight commands reject. */
     close(): void;
     private handleData;
+    /**
+     * A dead/half-open link makes every command time out while the socket still
+     * looks connected. After maxCommandTimeouts consecutive timeouts, tear down
+     * so the next call() reconnects instead of wedging (mirrors #94).
+     */
+    private noteTimeout;
     private teardown;
 }

package/dist/connection.js

@@ -5,12 +5,21 @@
  * pipelining (many in-flight commands per socket). Uses only `node:`
  * builtins (net/tls), which Node, Bun and Deno all support.
  */
+import { EventEmitter } from 'node:events';
 import { pack, unpack } from 'msgpackr';
+import { Backpressure } from './backpressure.js';
 import { AuthError, CommandError, CommandTimeoutError, ConnectionClosedError } from './errors.js';
 import { compact, FrameParser, frame, PROTOCOL_VERSION } from './frame.js';
+import { nowMs, Telemetry } from './observability.js';
 import { openSocket } from './socket-factory.js';
-/** A single pipelined TCP connection to a bunqueue server. */
-export class Connection {
+/**
+ * A single pipelined TCP connection to a bunqueue server.
+ *
+ * Emits typed lifecycle events for observability: `connect`, `disconnect` and
+ * `reconnect_scheduled` (payloads mirror the telemetry events). Attach a
+ * telemetry sink via the `onTelemetry` option for per-command latency metrics.
+ */
+export class Connection extends EventEmitter {
     host;
     port;
     token;
@@ -27,13 +36,23 @@ export class Connection {
     connectGeneration = 0;
     failedAttempts = 0;
     nextAttemptAt = 0;
+    // Half-open recovery (#94): after this many consecutive command timeouts the
+    // socket is presumed dead and torn down so the next call reconnects.
+    maxCommandTimeouts = 3;
+    consecutiveTimeouts = 0;
+    telemetry;
+    backpressure;
     constructor(options = {}) {
+        super();
         this.host = options.host ?? 'localhost';
         this.port = options.port ?? 6789;
         this.token = options.token;
         this.tls = options.tls;
         this.connectTimeoutMs = options.connectTimeoutMs ?? 5000;
         this.commandTimeoutMs = options.commandTimeoutMs ?? 10_000;
+        this.telemetry = new Telemetry(options, (event, payload) => this.emit(event, payload));
+        const maxInFlight = options.maxInFlight ?? 0;
+        this.backpressure = new Backpressure(maxInFlight, () => this.telemetry.backpressure(this.pending.size, maxInFlight));
     }
     get isConnected() {
         return this.connected;
@@ -69,6 +88,7 @@ export class Connection {
             this.failedAttempts += 1;
             const backoff = Math.min(500 * 2 ** (this.failedAttempts - 1), 5000);
             this.nextAttemptAt = Date.now() + backoff;
+            this.telemetry.reconnectScheduled(this.host, this.port, this.failedAttempts, backoff);
             throw err;
         })
             .finally(() => {
@@ -77,8 +97,12 @@ export class Connection {
         return this.connecting;
     }
     async doConnect() {
+        const startMs = nowMs();
         const socket = await openSocket(this.host, this.port, this.tls, this.connectTimeoutMs);
         socket.setNoDelay(true);
+        // TCP keepalive (~15s idle) surfaces a half-open link (cloud LB/NAT idle
+        // drop with no FIN/RST) in seconds instead of ~tcp_retries2 minutes.
+        socket.setKeepAlive(true, 15_000);
         this.parser.clear();
         this.socket = socket;
         socket.on('data', (chunk) => this.handleData(chunk));
@@ -86,11 +110,21 @@ export class Connection {
         socket.on('close', () => this.teardown());
         this.connected = true;
         this.connectGeneration += 1;
+        this.consecutiveTimeouts = 0;
+        this.telemetry.connected(this.host, this.port, this.connectGeneration, startMs);
+        // INVARIANT (H3): connected is flipped true before Auth, which is safe
+        // ONLY because call() writes the Auth frame synchronously — there is no
+        // `await` between this line and the Auth socket.write, so no concurrent
+        // call() can interleave a frame ahead of Auth on the wire. Do NOT insert
+        // an await here or before the Auth call, or a command could race ahead of
+        // Auth (the Python SDK guards this with a lock; JS relies on this ordering).
         if (this.token) {
             try {
                 await this.call({ cmd: 'Auth', token: this.token });
+                this.telemetry.auth(true);
             }
             catch (err) {
+                this.telemetry.auth(false);
                 this.teardown();
                 if (err instanceof CommandError)
                     throw new AuthError(err.message);
@@ -105,20 +139,32 @@ export class Connection {
     async call(command, timeoutMs) {
         if (!this.connected)
             await this.connect();
+        // Backpressure: park here if too many commands are already in flight. The
+        // socket may be torn down while parked, so re-check after the gate.
+        const gate = this.backpressure.acquire(this.pending.size);
+        if (gate)
+            await gate;
         const socket = this.socket;
-        if (!socket)
+        if (!this.connected || !socket)
             throw new ConnectionClosedError('not connected');
         this.reqCounter = (this.reqCounter + 1) & 0x7fffffff;
         const reqId = String(this.reqCounter);
         const payload = pack({ ...compact(command), reqId });
+        const gen = this.connectGeneration; // snapshot: a timeout must not tear down a newer conn
+        const startMs = nowMs();
         return new Promise((resolve, reject) => {
             const timer = setTimeout(() => {
                 this.pending.delete(reqId);
+                this.backpressure.release();
+                this.telemetry.timeout(command.cmd, reqId);
+                this.noteTimeout(gen);
                 reject(new CommandTimeoutError(`no response for ${command.cmd} within timeout`));
             }, timeoutMs ?? this.commandTimeoutMs);
             this.pending.set(reqId, {
                 resolve: (response) => {
                     clearTimeout(timer);
+                    this.consecutiveTimeouts = 0; // any reply means the link is alive
+                    this.telemetry.command(command.cmd, reqId, startMs, response.ok);
                     if (!response.ok) {
                         reject(new CommandError(String(response.error ?? 'unknown server error')));
                     }
@@ -136,6 +182,7 @@ export class Connection {
                 if (err) {
                     const entry = this.pending.get(reqId);
                     this.pending.delete(reqId);
+                    this.backpressure.release();
                     entry?.reject(new ConnectionClosedError(`send failed: ${err.message}`));
                     this.teardown();
                 }
@@ -192,11 +239,28 @@ export class Connection {
             const entry = this.pending.get(String(reqId));
             if (entry) {
                 this.pending.delete(String(reqId));
+                this.backpressure.release();
                 entry.resolve(response);
             }
         }
     }
+    /**
+     * A dead/half-open link makes every command time out while the socket still
+     * looks connected. After maxCommandTimeouts consecutive timeouts, tear down
+     * so the next call() reconnects instead of wedging (mirrors #94).
+     */
+    noteTimeout(gen) {
+        // A timeout from an already-replaced connection must not tear down (or
+        // miscount against) the current one.
+        if (gen !== this.connectGeneration)
+            return;
+        this.consecutiveTimeouts += 1;
+        if (this.consecutiveTimeouts >= this.maxCommandTimeouts)
+            this.teardown();
+    }
     teardown() {
+        const wasConnected = this.socket !== null;
+        const gen = this.connectGeneration;
         this.connected = false;
         const socket = this.socket;
         this.socket = null;
@@ -210,5 +274,8 @@ export class Connection {
             clearTimeout(entry.timer);
             entry.reject(new ConnectionClosedError('connection lost'));
         }
+        this.backpressure.clear(); // release parked callers; they re-check + fail/reconnect
+        if (wasConnected)
+            this.telemetry.disconnected(this.host, this.port, gen);
     }
 }

package/dist/flow-types.d.ts

@@ -1,6 +1,7 @@
 /** FlowProducer types. */
 import type { Connection, TlsOption } from './connection.js';
 import type { Job } from './job.js';
+import type { Observability } from './observability.js';
 import type { JobOptions } from './types.js';
 export interface FlowJob<T = unknown> {
     name: string;
@@ -19,7 +20,7 @@ export interface FlowStep<T = unknown> {
     data?: T;
     opts?: JobOptions;
 }
-export interface FlowProducerOptions {
+export interface FlowProducerOptions extends Observability {
     host?: string;
     port?: number;
     token?: string;