bunqueue-client 0.1.1 → 0.1.3

package/README.md

@@ -8,9 +8,9 @@ The bunqueue server runs on Bun, distributed as a binary or a Docker image. This
 
 | Runtime | Status | Notes |
 |---|---|---|
-| Node.js 20 or later | Supported, 58/58 e2e and 8/8 integration tests | ESM. TypeScript files run directly on Node 22 or later via `--experimental-strip-types` |
-| Bun | Supported, 58/58 e2e and 8/8 integration tests | No additional configuration required |
-| Deno 2 or later | Supported, 58/58 e2e and 8/8 integration tests | Uses `node:` builtins and the npm `msgpackr` package |
+| Node.js 20 or later | Supported, 72/72 e2e and 8/8 integration tests | ESM. TypeScript files run directly on Node 22 or later via `--experimental-strip-types` |
+| Bun | Supported, 72/72 e2e and 8/8 integration tests | No additional configuration required |
+| Deno 2 or later | Supported, 72/72 e2e and 8/8 integration tests | Uses `node:` builtins and the npm `msgpackr` package |
 | tsx, ts-node, vitest, jest | Supported | These environments execute on Node.js |
 | Cloudflare Workers | Supported, 11/11 e2e tests inside workerd | Requires the `nodejs_compat` compatibility flag. The runtime is request scoped, so long lived worker loops are not available: consume in batches from Cron Triggers or Durable Object alarms, a pattern covered by the test suite. TLS connections require a publicly trusted certificate |
 | Browser | Not supported | Raw TCP sockets are unavailable. Use the server HTTP API instead |
@@ -24,6 +24,50 @@ npm install bunqueue-client
 # or: bun add bunqueue-client / pnpm add bunqueue-client / deno add npm:bunqueue-client
 ```
 
+## Quick start
+
+Sixty seconds from zero to a working queue. Step 1, start the server (requires [Bun](https://bun.sh), or use the Docker image):
+
+```bash
+bunx bunqueue start
+```
+
+Step 2, create `app.ts`: add a job and process it, in the same file for the sake of the demo:
+
+```typescript
+import { Queue, Worker } from 'bunqueue-client';
+
+const worker = new Worker('hello', async (job) => {
+  console.log('processing', job.data);
+  return { greeted: job.data.name };
+});
+worker.on('completed', (job, result) => {
+  console.log('completed', job.id, result);
+  worker.close();
+});
+
+const queue = new Queue('hello');
+await queue.add('greet', { name: 'world' });
+queue.close();
+```
+
+Step 3, run it with the runtime you already use:
+
+```bash
+node --experimental-strip-types app.ts    # Node 22 or later
+bun app.ts                                # Bun
+deno run -A app.ts                        # Deno 2 or later
+```
+
+Expected output:
+
+```
+processing { name: 'world' }
+completed 019f40a5-... { greeted: 'world' }
+```
+
+That is the whole model: the server owns state, retries, and scheduling, your code only adds and processes. In production the producer and the worker are separate services, often in different languages: the [Python client](https://github.com/egeominotti/bunqueue/tree/main/sdk/python) speaks the same protocol against the same queue. Defaults are `host: 'localhost'`, `port: 6789`, so constructors need no options on a local setup.
+
 ## Producing jobs
 
 ```typescript
@@ -98,6 +142,47 @@ const node = await flow.add({
 });
 ```
 
+## Simple Mode
+
+`Bunqueue` bundles a Queue and a Worker in one object, with routes, onion middleware, in process retry strategies, a circuit breaker, batch accumulation, event triggers, job TTL, priority aging, cooperative cancellation, and dedup or debounce defaults. It is a 1:1 port of the official client's Simple Mode, TCP only (the `embedded` option raises).
+
+```typescript
+import { Bunqueue, type Job } from 'bunqueue-client';
+
+const app = new Bunqueue('notifications', {
+  connection: { host: 'localhost', port: 6789 },
+  concurrency: 10,
+  routes: {
+    'send-email': async (job: Job<{ to: string }>) => ({ channel: 'email' }),
+    'send-sms': async (job: Job<{ to: string }>) => ({ channel: 'sms' }),
+  },
+  retry: { maxAttempts: 5, delay: 1000, strategy: 'jitter' },
+  circuitBreaker: { threshold: 5, resetTimeout: 30_000 },
+  ttl: { perName: { 'verify-otp': 60_000 } },
+  deduplication: { ttl: 5000 },
+});
+
+app.use(async (job, next) => {
+  const start = Date.now();
+  const result = await next();
+  console.log(`${job.name}: ${Date.now() - start}ms`);
+  return result;
+});
+
+app.trigger({
+  on: 'send-email',
+  create: 'send-sms',
+  data: (result, job) => job.data,
+  condition: (result) => (result as { channel: string }).channel === 'email',
+});
+
+await app.cron('daily-digest', '0 9 * * *', { to: 'all' });
+await app.add('send-email', { to: 'alice@example.com' });
+// later: await app.close();
+```
+
+Use `processor` for a single handler, `routes` to dispatch by job name, or `batch` to accumulate N jobs into one call. Exactly one of the three is required.
+
 ## Scheduling
 
 ```typescript
@@ -129,9 +214,12 @@ Authentication uses server side tokens (`AUTH_TOKENS`). Transport security uses
 | Dead letter queue | `getDlq`, `retryDlq`, `purgeDlq`, DLQ configuration |
 | Administration | rate limiting, global concurrency, stall configuration, webhooks, stats, metrics, `listQueues`, `getWorkers` |
 | Worker events | `ready`, `active`, `completed`, `failed`, `progress`, `drained`, `error`, `closed`, with automatic lock heartbeats so that jobs longer than the lock TTL survive |
+| Simple Mode | `Bunqueue`: routes, middleware, in process retry (fixed, exponential, jitter, fibonacci, custom), circuit breaker, batch accumulation, triggers, TTL, priority aging, cancellation via `getSignal`, dedup and debounce defaults, cron shorthands |
 
 The following features require the in process Bun runtime and are intentionally out of scope for this client: embedded mode, sandboxed workers, and `QueueEvents`. Use webhooks or the HTTP SSE and WebSocket endpoints for event streaming.
 
+Note on numeric payloads: JavaScript numbers are IEEE 754 doubles, exact up to 2^53. Pass larger 64 bit identifiers, for example snowflake IDs, as strings to avoid silent precision loss. Never place `BigInt` values in job data.
+
 ## Quality assurance
 
 Every release is validated against a real bunqueue server, spawned fresh for each run, across every supported runtime:

package/dist/bunqueue/aging.d.ts

@@ -0,0 +1,17 @@
+/**
+ * Bunqueue Simple Mode — priority aging.
+ * Port of src/client/bunqueue/aging.ts: automatically boosts the priority of
+ * old waiting jobs. Same semantics; the job scan goes over TCP (GetJobs on
+ * the waiting and prioritized states) instead of the embedded manager.
+ */
+import type { Queue } from '../queue.js';
+import type { PriorityAgingConfig } from './types.js';
+export declare class PriorityAger<T = unknown> {
+    private timer;
+    private readonly config;
+    private readonly queue;
+    constructor(config: PriorityAgingConfig, queue: Queue<T>);
+    start(): void;
+    private tick;
+    destroy(): void;
+}

package/dist/bunqueue/aging.js

@@ -0,0 +1,53 @@
+/**
+ * Bunqueue Simple Mode — priority aging.
+ * Port of src/client/bunqueue/aging.ts: automatically boosts the priority of
+ * old waiting jobs. Same semantics; the job scan goes over TCP (GetJobs on
+ * the waiting and prioritized states) instead of the embedded manager.
+ */
+export class PriorityAger {
+    timer = null;
+    config;
+    queue;
+    constructor(config, queue) {
+        this.config = config;
+        this.queue = queue;
+    }
+    start() {
+        const interval = this.config.interval ?? 60000;
+        this.timer = setInterval(() => {
+            void this.tick();
+        }, interval);
+        this.timer.unref?.();
+    }
+    async tick() {
+        const minAge = this.config.minAge ?? 60000;
+        const boost = this.config.boost ?? 1;
+        const maxPriority = this.config.maxPriority ?? 100;
+        const maxScan = this.config.maxScan ?? 100;
+        // Both waiting and prioritized jobs (priority > 0 means "prioritized")
+        const [waiting, prioritized] = await Promise.all([
+            this.queue.getJobs({ state: 'waiting', start: 0, end: maxScan }),
+            this.queue.getJobs({ state: 'prioritized', start: 0, end: maxScan }),
+        ]);
+        const jobs = [...waiting, ...prioritized];
+        const now = Date.now();
+        for (const job of jobs) {
+            const age = now - job.timestamp;
+            if (age >= minAge && job.priority < maxPriority) {
+                const newPriority = Math.min(job.priority + boost, maxPriority);
+                try {
+                    await this.queue.changeJobPriority(job.id, { priority: newPriority });
+                }
+                catch {
+                    // Best-effort — the job may have been processed meanwhile
+                }
+            }
+        }
+    }
+    destroy() {
+        if (this.timer) {
+            clearInterval(this.timer);
+            this.timer = null;
+        }
+    }
+}

package/dist/bunqueue/batch.d.ts

@@ -0,0 +1,17 @@
+/**
+ * Bunqueue Simple Mode — batch processing.
+ * 1:1 port of src/client/bunqueue/batch.ts: accumulates jobs and processes
+ * them in groups; flushes on size, on timeout, and on destroy (close).
+ */
+import type { Processor } from '../worker-types.js';
+import type { BatchConfig } from './types.js';
+export declare class BatchAccumulator<T = unknown, R = unknown> {
+    private readonly buffer;
+    private timer;
+    private readonly config;
+    constructor(config: BatchConfig<T, R>);
+    /** Build a Processor that buffers jobs into batches. */
+    buildProcessor(): Processor<T, R>;
+    flush(): void;
+    destroy(): void;
+}

package/dist/bunqueue/batch.js

@@ -0,0 +1,60 @@
+/**
+ * Bunqueue Simple Mode — batch processing.
+ * 1:1 port of src/client/bunqueue/batch.ts: accumulates jobs and processes
+ * them in groups; flushes on size, on timeout, and on destroy (close).
+ */
+export class BatchAccumulator {
+    buffer = [];
+    timer = null;
+    config;
+    constructor(config) {
+        this.config = config;
+    }
+    /** Build a Processor that buffers jobs into batches. */
+    buildProcessor() {
+        return (job) => {
+            return new Promise((resolve, reject) => {
+                this.buffer.push({ job, resolve, reject });
+                if (this.buffer.length >= this.config.size) {
+                    this.flush();
+                }
+                else if (!this.timer) {
+                    const timeout = this.config.timeout ?? 5000;
+                    this.timer = setTimeout(() => {
+                        this.flush();
+                    }, timeout);
+                }
+            });
+        };
+    }
+    flush() {
+        if (this.timer) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        const batch = this.buffer.splice(0);
+        if (batch.length === 0)
+            return;
+        const jobs = batch.map((b) => b.job);
+        this.config.processor(jobs).then((results) => {
+            for (let i = 0; i < batch.length; i++) {
+                batch[i].resolve(results[i] ?? undefined);
+            }
+        }, (err) => {
+            const error = err instanceof Error ? err : new Error(String(err));
+            for (const b of batch) {
+                b.reject(error);
+            }
+        });
+    }
+    destroy() {
+        if (this.timer) {
+            clearTimeout(this.timer);
+            this.timer = null;
+        }
+        // Flush remaining
+        if (this.buffer.length > 0) {
+            this.flush();
+        }
+    }
+}

package/dist/bunqueue/bunqueue-api.d.ts

@@ -0,0 +1,87 @@
+/**
+ * Bunqueue Simple Mode — delegation API surface (cron, cancellation, circuit
+ * breaker, TTL, DLQ, rate limit, events, control). Merged onto the Bunqueue
+ * prototype by bunqueue.ts; split out to keep files small.
+ */
+import type { Job } from '../job.js';
+import type { JobOptions } from '../types.js';
+import type { Bunqueue } from './bunqueue.js';
+import type { DlqFilter, DlqStats } from './dlq-rate-limit.js';
+import type { CircuitState, TriggerRule } from './types.js';
+type Raw = Record<string, unknown>;
+type Ctx = Bunqueue<any, any>;
+export declare const bunqueueApi: {
+    cron(this: Ctx, id: string, pattern: string, data?: unknown, opts?: {
+        timezone?: string;
+        jobOpts?: JobOptions;
+    }): Promise<Raw | null>;
+    every(this: Ctx, id: string, intervalMs: number, data?: unknown, opts?: {
+        jobOpts?: JobOptions;
+    }): Promise<Raw | null>;
+    removeCron(this: Ctx, id: string): Promise<void>;
+    listCrons(this: Ctx): Promise<Raw[]>;
+    cancel(this: Ctx, jobId: string, gracePeriodMs?: number): void;
+    isCancelled(this: Ctx, jobId: string): boolean;
+    getSignal(this: Ctx, jobId: string): AbortSignal | null;
+    getCircuitState(this: Ctx): CircuitState;
+    resetCircuit(this: Ctx): void;
+    trigger(this: Ctx, rule: TriggerRule): Ctx;
+    setDefaultTtl(this: Ctx, ttlMs: number): void;
+    setNameTtl(this: Ctx, name: string, ttlMs: number): void;
+    setDlqConfig(this: Ctx, config: Raw): Promise<void>;
+    getDlqConfig(this: Ctx): Promise<Raw>;
+    getDlq(this: Ctx, filter?: DlqFilter): Promise<Raw[]>;
+    getDlqStats(this: Ctx): Promise<DlqStats>;
+    retryDlq(this: Ctx, id?: string): Promise<number>;
+    purgeDlq(this: Ctx): Promise<number>;
+    setGlobalRateLimit(this: Ctx, max: number, duration?: number): Promise<void>;
+    removeGlobalRateLimit(this: Ctx): Promise<void>;
+    on(this: Ctx, event: string, listener: (...args: never[]) => void): Ctx;
+    once(this: Ctx, event: string, listener: (...args: never[]) => void): Ctx;
+    off(this: Ctx, event: string, listener: (...args: never[]) => void): Ctx;
+    pause(this: Ctx): void;
+    resume(this: Ctx): void;
+    close(this: Ctx, force?: boolean): Promise<void>;
+    isRunning(this: Ctx): boolean;
+    isPaused(this: Ctx): boolean;
+    isClosed(this: Ctx): boolean;
+};
+/** Declaration-merged API installed on the Bunqueue prototype. */
+export interface BunqueueApi<T = unknown, R = unknown> {
+    cron(id: string, pattern: string, data?: T, opts?: {
+        timezone?: string;
+        jobOpts?: JobOptions;
+    }): Promise<Raw | null>;
+    every(id: string, intervalMs: number, data?: T, opts?: {
+        jobOpts?: JobOptions;
+    }): Promise<Raw | null>;
+    removeCron(id: string): Promise<void>;
+    listCrons(): Promise<Raw[]>;
+    cancel(jobId: string, gracePeriodMs?: number): void;
+    isCancelled(jobId: string): boolean;
+    getSignal(jobId: string): AbortSignal | null;
+    getCircuitState(): CircuitState;
+    resetCircuit(): void;
+    trigger(rule: TriggerRule<T>): Bunqueue<T, R>;
+    setDefaultTtl(ttlMs: number): void;
+    setNameTtl(name: string, ttlMs: number): void;
+    setDlqConfig(config: Raw): Promise<void>;
+    getDlqConfig(): Promise<Raw>;
+    getDlq(filter?: DlqFilter): Promise<Raw[]>;
+    getDlqStats(): Promise<DlqStats>;
+    retryDlq(id?: string): Promise<number>;
+    purgeDlq(): Promise<number>;
+    setGlobalRateLimit(max: number, duration?: number): Promise<void>;
+    removeGlobalRateLimit(): Promise<void>;
+    on(event: string, listener: (...args: never[]) => void): Bunqueue<T, R>;
+    once(event: string, listener: (...args: never[]) => void): Bunqueue<T, R>;
+    off(event: string, listener: (...args: never[]) => void): Bunqueue<T, R>;
+    pause(): void;
+    resume(): void;
+    close(force?: boolean): Promise<void>;
+    isRunning(): boolean;
+    isPaused(): boolean;
+    isClosed(): boolean;
+    getJob(id: string): Promise<Job<T> | null>;
+}
+export {};

package/dist/bunqueue/bunqueue-api.js

@@ -0,0 +1,116 @@
+/**
+ * Bunqueue Simple Mode — delegation API surface (cron, cancellation, circuit
+ * breaker, TTL, DLQ, rate limit, events, control). Merged onto the Bunqueue
+ * prototype by bunqueue.ts; split out to keep files small.
+ */
+export const bunqueueApi = {
+    // --------------------------------------------------------------------- cron
+    async cron(id, pattern, data, opts) {
+        await this.queue.upsertJobScheduler(id, { pattern, tz: opts?.timezone }, { name: id, data, opts: opts?.jobOpts });
+        return this.queue.getJobScheduler(id);
+    },
+    async every(id, intervalMs, data, opts) {
+        await this.queue.upsertJobScheduler(id, { every: intervalMs }, { name: id, data, opts: opts?.jobOpts });
+        return this.queue.getJobScheduler(id);
+    },
+    removeCron(id) {
+        return this.queue.removeJobScheduler(id);
+    },
+    listCrons() {
+        return this.queue.getJobSchedulers();
+    },
+    // ------------------------------------------------------------- cancellation
+    cancel(jobId, gracePeriodMs = 0) {
+        this.cancellation.cancel(jobId, gracePeriodMs);
+    },
+    isCancelled(jobId) {
+        return this.cancellation.isCancelled(jobId);
+    },
+    getSignal(jobId) {
+        return this.cancellation.getSignal(jobId);
+    },
+    // ---------------------------------------------------------- circuit breaker
+    getCircuitState() {
+        return this.cb?.currentState ?? 'closed';
+    },
+    resetCircuit() {
+        this.cb?.reset();
+    },
+    // ----------------------------------------------------------------- triggers
+    trigger(rule) {
+        this.triggerMgr.add(rule);
+        return this;
+    },
+    // ---------------------------------------------------------------------- ttl
+    setDefaultTtl(ttlMs) {
+        this.ttlChecker?.setDefaultTtl(ttlMs);
+    },
+    setNameTtl(name, ttlMs) {
+        this.ttlChecker?.setNameTtl(name, ttlMs);
+    },
+    // ---------------------------------------------------------------------- dlq
+    setDlqConfig(config) {
+        return this.dlqrl.setDlqConfig(config);
+    },
+    getDlqConfig() {
+        return this.dlqrl.getDlqConfig();
+    },
+    getDlq(filter) {
+        return this.dlqrl.getDlq(filter);
+    },
+    getDlqStats() {
+        return this.dlqrl.getDlqStats();
+    },
+    retryDlq(id) {
+        return this.dlqrl.retryDlq(id);
+    },
+    purgeDlq() {
+        return this.dlqrl.purgeDlq();
+    },
+    // ------------------------------------------------------------ rate limiting
+    setGlobalRateLimit(max, duration) {
+        return this.dlqrl.setGlobalRateLimit(max, duration);
+    },
+    removeGlobalRateLimit() {
+        return this.dlqrl.removeGlobalRateLimit();
+    },
+    // ------------------------------------------------------------------- events
+    on(event, listener) {
+        this.worker.on(event, listener);
+        return this;
+    },
+    once(event, listener) {
+        this.worker.once(event, listener);
+        return this;
+    },
+    off(event, listener) {
+        this.worker.off(event, listener);
+        return this;
+    },
+    // ------------------------------------------------------------------ control
+    pause() {
+        void this.queue.pause().catch(() => { });
+        this.worker.pause();
+    },
+    resume() {
+        void this.queue.resume().catch(() => { });
+        this.worker.resume();
+    },
+    async close(force = false) {
+        this.ager?.destroy();
+        this.cb?.destroy();
+        this.batchAcc?.destroy();
+        this.cancellation.destroyAll();
+        await this.worker.close(force);
+        this.queue.close();
+    },
+    isRunning() {
+        return this.worker.isRunning();
+    },
+    isPaused() {
+        return this.worker.isPaused();
+    },
+    isClosed() {
+        return this.worker.isClosed();
+    },
+};

package/dist/bunqueue/bunqueue.d.ts

@@ -0,0 +1,57 @@
+/**
+ * Bunqueue — simplified all-in-one Queue + Worker (Simple Mode).
+ * 1:1 port of src/client/bunqueue.ts from the official client, TCP mode only
+ * (the embedded mode requires the in-process Bun runtime). The delegation
+ * API (cron, DLQ, events, control, …) lives in bunqueue-api.ts and is merged
+ * onto the prototype below.
+ */
+import type { Job } from '../job.js';
+import { Queue } from '../queue.js';
+import type { JobOptions } from '../types.js';
+import { Worker } from '../worker.js';
+import { PriorityAger } from './aging.js';
+import { BatchAccumulator } from './batch.js';
+import { type BunqueueApi } from './bunqueue-api.js';
+import { CancellationManager } from './cancellation.js';
+import { WorkerCircuitBreaker } from './circuit-breaker.js';
+import { DlqRateLimitManager } from './dlq-rate-limit.js';
+import { TriggerManager } from './triggers.js';
+import { TtlChecker } from './ttl.js';
+import type { BunqueueMiddleware, BunqueueOptions } from './types.js';
+export declare class Bunqueue<T = unknown, R = unknown> {
+    readonly name: string;
+    readonly queue: Queue<T>;
+    readonly worker: Worker<T, R>;
+    /** @internal */ readonly cancellation: CancellationManager;
+    /** @internal */ readonly cb: WorkerCircuitBreaker | null;
+    /** @internal */ readonly triggerMgr: TriggerManager<T, R>;
+    /** @internal */ readonly ager: PriorityAger<T> | null;
+    /** @internal */ readonly ttlChecker: TtlChecker | null;
+    /** @internal */ readonly batchAcc: BatchAccumulator<T, R> | null;
+    /** @internal */ readonly dlqrl: DlqRateLimitManager<T>;
+    private readonly middlewares;
+    private readonly baseProcessor;
+    private readonly retryConfig;
+    private readonly merger;
+    private readonly rateGate;
+    private readonly defaultJobOptions;
+    constructor(name: string, opts: BunqueueOptions<T, R>);
+    private buildRouteProcessor;
+    private buildDefaultJobOptions;
+    private processJob;
+    private runMiddlewareChain;
+    use(middleware: BunqueueMiddleware<T, R>): this;
+    add(name: string, data: T, opts?: JobOptions): Promise<Job<T>>;
+    addBulk(jobs: Array<{
+        name: string;
+        data: T;
+        opts?: JobOptions;
+    }>): Promise<Job<T>[]>;
+    getJob(id: string): Promise<Job<T> | null>;
+    getJobCounts(): Promise<import("../types.js").JobCounts>;
+    getJobCountsAsync(): Promise<import("../types.js").JobCounts>;
+    count(): Promise<number>;
+    countAsync(): Promise<number>;
+}
+export interface Bunqueue<T = unknown, R = unknown> extends BunqueueApi<T, R> {
+}