@monlite/queue 0.2.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,6 +1,8 @@
1
1
  # @monlite/queue
2
2
 
3
- A **durable job queue** for [`@monlite/core`](https://www.npmjs.com/package/@monlite/core), backed by SQLite — retries, backoff, delayed jobs, priorities, and concurrency, with no separate server. Part of the [local AI-agent harness](https://github.com/qataruts/monlite#readme) (the BullMQ/Redis role, locally).
3
+ A durable job queue for [`@monlite/core`](https://www.npmjs.com/package/@monlite/core), backed
4
+ by SQLite — retries, backoff, delayed jobs, priorities, dedupe, and concurrency, with no
5
+ separate server. The BullMQ/Redis role, locally.
4
6
 
5
7
  ```bash
6
8
  npm install @monlite/core @monlite/queue
@@ -15,7 +17,7 @@ import { createQueue } from "@monlite/queue";
15
17
  const db = createDb("app.db");
16
18
  const queue = createQueue(db, { maxAttempts: 3 });
17
19
 
18
- // Worker
20
+ // Worker — processes jobs as they arrive
19
21
  queue.process("email", async (job) => {
20
22
  await sendEmail(job.payload);
21
23
  }, { concurrency: 5 });
@@ -23,51 +25,80 @@ queue.process("email", async (job) => {
23
25
  queue.on("completed", (job) => console.log("sent", job.id));
24
26
  queue.on("failed", (job, err) => console.warn("failed", job.id, err.message));
25
27
 
26
- // Producer
28
+ // Producer — enqueue from anywhere, even a different process
27
29
  queue.add("email", { to: "ali@example.com" });
28
- queue.add("digest", { day: "mon" }, { delay: 60_000, priority: 10 });
30
+ queue.add("digest", { day: "monday" }, { delay: 60_000, priority: 10 });
29
31
  ```
30
32
 
31
33
  ## Features
32
34
 
33
- - **Durable & transactional** — jobs live in SQLite, so nothing is lost on crash.
34
- - **Atomic claim** — workers grab jobs with a single `UPDATE RETURNING`, so it's
35
- **multi-process safe** (run workers in several processes against the same db).
36
- - **Retries with backoff** — failed jobs retry up to `maxAttempts`; backoff is
37
- exponential by default (override with `backoff(attempt) => ms`).
38
- - **Delayed / scheduled** — `{ delay }` or `{ runAt }`.
39
- - **Priorities** higher `priority` runs first.
40
- - **Concurrency** `{ concurrency }` per worker.
41
- - **Dead-letter** — exhausted jobs become `status: "failed"` and are kept for inspection.
35
+ **Durable.** Jobs live in SQLite nothing is lost on crash. On restart, call `queue.recover()`
36
+ to requeue any jobs that were `active` when the process died.
37
+
38
+ **Multi-process safe.** Workers claim jobs with a single `UPDATE RETURNING`. Run the same
39
+ queue in multiple processes against the same `.db`; each job is claimed exactly once.
40
+
41
+ **Retries with backoff.** Failed jobs retry up to `maxAttempts`; backoff is exponential by
42
+ default (capped at 30s). Exhausted jobs become `status: "failed"` and are kept for inspection.
43
+
44
+ **Delayed and scheduled.** Pass `{ delay: ms }` or `{ runAt: Date }` to defer a job.
45
+
46
+ **Deduplication.** Pass `{ jobId: "unique-key" }` — a second `add` with the same `jobId` while
47
+ the first is pending or active returns the existing job without creating a duplicate.
42
48
 
43
49
  ## API
44
50
 
45
51
  ```ts
46
52
  const queue = createQueue(db, {
47
- maxAttempts, // default attempts before dead-lettering (default 1)
48
- backoff, // (attempt) => ms; default exponential capped at 30s
49
- removeOnComplete, // delete finished jobs instead of keeping them (default false)
53
+ maxAttempts: 1, // default attempts before dead-lettering
54
+ backoff: (attempt) => Math.min(1000 * 2 ** attempt, 30_000), // default exponential
55
+ removeOnComplete: false, // delete finished jobs instead of keeping them
50
56
  });
51
57
 
52
- queue.add(name, payload, { delay, runAt, priority, maxAttempts, jobId }); // → Job
53
- // jobId dedupes: a second add with the same jobId (while pending/active) returns the existing job
54
- queue.process(name, handler, { concurrency, pollInterval }); // → Worker
58
+ // Add a job
59
+ queue.add(name, payload, { delay?, runAt?, priority?, maxAttempts?, jobId? }); // Job
60
+
61
+ // Process jobs
62
+ queue.process(name, handler, { concurrency?, pollInterval? }); // → Worker
63
+
64
+ // Events
55
65
  queue.on("completed" | "failed", (job, resultOrError) => {});
56
- queue.getJob(id); // Job | undefined
57
- queue.counts(name?); // { pending, active, done, failed }
58
- queue.recover(olderThanMs); // requeue jobs stuck "active" from a crash
59
- await worker.stop(); // stop one worker (drains in-flight)
60
- await queue.close(); // stop all workers
66
+
67
+ // Inspect
68
+ queue.getJob(id); // Job | undefined
69
+ queue.counts(name?); // { pending, active, done, failed }
70
+
71
+ // Crash recovery — requeue jobs stuck "active" from a crashed worker
72
+ queue.recover(olderThanMs);
73
+
74
+ // Shutdown
75
+ await worker.stop(); // drain in-flight jobs for one worker
76
+ await queue.close(); // stop all workers
61
77
  ```
62
78
 
63
- ## Recovering crashed jobs
79
+ ## Recovering crashed workers
64
80
 
65
- A worker that dies mid-job leaves the job `active`. Call `queue.recover()` on
66
- startup (or periodically) to requeue jobs that have been `active` longer than a
67
- threshold:
81
+ A worker that dies mid-job leaves the job in `active` state. Call `queue.recover()` on startup
82
+ (or periodically) to requeue jobs stuck active for longer than a threshold:
68
83
 
69
84
  ```ts
70
- queue.recover(60_000); // requeue anything stuck > 60s
85
+ queue.recover(60_000); // requeue anything active for > 60s
71
86
  ```
72
87
 
88
+ ## Composing with `@monlite/cron`
89
+
90
+ For **scheduled** durable work, have a cron handler enqueue a job rather than running the work
91
+ inline — the schedule is persisted and the work is retried:
92
+
93
+ ```ts
94
+ import { createCron } from "@monlite/cron";
95
+ const cron = createCron(db);
96
+
97
+ cron.schedule("nightly-report", "0 0 * * *", () => {
98
+ queue.add("report", { day: new Date().toISOString() });
99
+ });
100
+ ```
101
+
102
+ ## License
103
+
73
104
  MIT
package/dist/index.cjs CHANGED
@@ -30,6 +30,13 @@ var WorkerImpl = class {
30
30
  this.handler = handler;
31
31
  this.concurrency = Math.max(1, opts.concurrency ?? 1);
32
32
  this.pollInterval = opts.pollInterval ?? 500;
33
+ this.visibilityTimeout = Math.max(0, opts.visibilityTimeout ?? 0);
34
+ if (this.visibilityTimeout > 0) {
35
+ this.reaper = setInterval(() => {
36
+ if (this.running) this.q.recover(this.visibilityTimeout);
37
+ }, Math.max(1e3, Math.floor(this.visibilityTimeout / 2)));
38
+ this.reaper.unref?.();
39
+ }
33
40
  this.kick();
34
41
  }
35
42
  q;
@@ -41,6 +48,8 @@ var WorkerImpl = class {
41
48
  drainResolve;
42
49
  concurrency;
43
50
  pollInterval;
51
+ visibilityTimeout;
52
+ reaper;
44
53
  /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */
45
54
  kick() {
46
55
  if (!this.running) return;
@@ -52,6 +61,14 @@ var WorkerImpl = class {
52
61
  const job = this.q.claimInternal(this.name);
53
62
  if (!job) break;
54
63
  this.inFlight++;
64
+ let hb;
65
+ if (this.visibilityTimeout > 0) {
66
+ hb = setInterval(
67
+ () => this.q.heartbeatInternal(job.id),
68
+ Math.max(1e3, Math.floor(this.visibilityTimeout / 2))
69
+ );
70
+ hb.unref?.();
71
+ }
55
72
  Promise.resolve().then(() => this.handler(job)).then(
56
73
  (result) => {
57
74
  this.q.completeInternal(job, result);
@@ -62,6 +79,7 @@ var WorkerImpl = class {
62
79
  this.q.emit("failed", job, err);
63
80
  }
64
81
  ).finally(() => {
82
+ if (hb) clearInterval(hb);
65
83
  this.inFlight--;
66
84
  if (this.running) this.kick();
67
85
  else this.checkDrained();
@@ -81,6 +99,7 @@ var WorkerImpl = class {
81
99
  async stop() {
82
100
  this.running = false;
83
101
  if (this.timer) clearTimeout(this.timer);
102
+ if (this.reaper) clearInterval(this.reaper);
84
103
  if (this.inFlight === 0) return;
85
104
  await new Promise((resolve) => {
86
105
  this.drainResolve = resolve;
@@ -100,7 +119,7 @@ var Queue = class extends events.EventEmitter {
100
119
  this.maxAttempts = opts.maxAttempts ?? 1;
101
120
  this.backoff = opts.backoff ?? defaultBackoff;
102
121
  this.removeOnComplete = opts.removeOnComplete ?? false;
103
- this.workerId = opts.workerId ?? `w-${process.pid}`;
122
+ this.workerId = opts.workerId ?? `w-${typeof process !== "undefined" && process.pid ? process.pid : Math.floor(Math.random() * 1e6)}`;
104
123
  if (!ensured.has(db)) {
105
124
  this.driver.exec(
106
125
  `CREATE TABLE IF NOT EXISTS _jobs (
@@ -189,6 +208,10 @@ var Queue = class extends events.EventEmitter {
189
208
  WHERE status='active' AND updated_at < ?`
190
209
  ).run(now(), now() - olderThanMs).changes;
191
210
  }
211
+ /** @internal Extend a running job's visibility timeout (worker heartbeat). */
212
+ heartbeatInternal(id) {
213
+ this.driver.prepare(`UPDATE _jobs SET updated_at=? WHERE id=? AND status='active'`).run(now(), id);
214
+ }
192
215
  /** Stop all workers and wait for in-flight jobs to finish. */
193
216
  async close() {
194
217
  await Promise.all(this.workers.map((w) => w.stop()));
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/index.ts"],"names":["EventEmitter"],"mappings":";;;;;AA6EA,IAAM,OAAA,uBAAc,OAAA,EAAgB;AACpC,IAAM,GAAA,GAAM,MAAM,IAAA,CAAK,GAAA,EAAI;AAC3B,IAAM,cAAA,GAAiB,CAAC,OAAA,KACtB,IAAA,CAAK,IAAI,GAAA,EAAQ,GAAA,GAAO,CAAA,KAAM,OAAA,GAAU,CAAA,CAAE,CAAA;AAE5C,SAAS,YAAY,GAAA,EAAe;AAClC,EAAA,OAAO;AAAA,IACL,IAAI,GAAA,CAAI,EAAA;AAAA,IACR,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,KAAA,EAAO,IAAI,MAAA,IAAU,MAAA;AAAA,IACrB,QAAQ,GAAA,CAAI,MAAA;AAAA,IACZ,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAA,EAAS,IAAA,CAAK,KAAA,CAAM,GAAA,CAAI,OAAO,CAAA;AAAA,IAC/B,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,aAAa,GAAA,CAAI,YAAA;AAAA,IACjB,OAAO,GAAA,CAAI,MAAA;AAAA,IACX,MAAA,EAAQ,IAAI,MAAA,IAAU,IAAA,GAAO,KAAK,KAAA,CAAM,GAAA,CAAI,MAAM,CAAA,GAAI,MAAA;AAAA,IACtD,KAAA,EAAO,IAAI,KAAA,IAAS,MAAA;AAAA,IACpB,WAAW,GAAA,CAAI,UAAA;AAAA,IACf,WAAW,GAAA,CAAI;AAAA,GACjB;AACF;AAEA,IAAM,aAAN,MAAmC;AAAA,EAQjC,WAAA,CACmB,CAAA,EACR,IAAA,EACQ,OAAA,EACjB,IAAA,EACA;AAJiB,IAAA,IAAA,CAAA,CAAA,GAAA,CAAA;AACR,IAAA,IAAA,CAAA,IAAA,GAAA,IAAA;AACQ,IAAA,IAAA,CAAA,OAAA,GAAA,OAAA;AAGjB,IAAA,IAAA,CAAK,cAAc,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,IAAA,CAAK,YAAA,GAAe,KAAK,YAAA,IAAgB,GAAA;AACzC,IAAA,IAAA,CAAK,IAAA,EAAK;AAAA,EACZ;AAAA,EARmB,CAAA;AAAA,EACR,IAAA;AAAA,EACQ,OAAA;AAAA,EAVX,OAAA,GAAU,IAAA;AAAA,EACV,QAAA,GAAW,CAAA;AAAA,EACX,KAAA;AAAA,EACA,YAAA;AAAA,EACS,WAAA;AAAA,EACA,YAAA;AAAA;AAAA,EAcjB,IAAA,GAAa;AACX,IAAA,IAAI,CAAC,KAAK,OAAA,EAAS;AACnB,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,KAAK,KAAK,CAAA;AACvB,MAAA,IAAA,CAAK,KAAA,GAAQ,MAAA;AAAA,IACf;AACA,IAAA,OAAO,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACvD,MAAA,MAAM,GAAA,GAAM,IAAA,CAAK,CAAA,CAAE,aAAA,CAAc,KAAK,IAAI,CAAA;AAC1C,MAAA,IAAI,CAAC,GAAA,EAAK;AACV,MAAA,IAAA,CAAK,QAAA,EAAA;AACL,MAAA,OAAA,CAAQ,OAAA,GACL,IAAA,CAAK,MAAM,KAAK,OAAA,CAAQ,GAAG,CAAC,CAAA,CAC5B,IAAA;AAAA,QACC,CAAC,MAAA,KAAW;AACV,UAAA,IAAA,CAAK,CAAA,CAAE,gBAAA,CAAiB,GAAA,EAAK,MAAM,CAAA;AACnC,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,WAAA,EAAa,GAAA,EAAK,MAAM,CAAA;AAAA,QACtC,CAAA;AAAA,QACA,CAAC,GAAA,KAAQ;AACP,UAAA,IAAA,CAAK,CAAA,CAAE,YAAA,CAAa,GAAA,EAAK,GAAG,CAAA;AAC5B,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,QAAA,EAAU,GAAA,EAAK,GAAG,CAAA;AAAA,QAChC;AAAA,OACF,CACC,QAAQ,MAAM;AACb,QAAA,IAAA,CAAK,QAAA,EAAA;AACL,QAAA,IAAI,IAAA,CAAK,OAAA,EAAS,IAAA,CAAK,IAAA,EAAK;AAAA,kBAClB,YAAA,EAAa;AAAA,MACzB,CAAC,CAAA;AAAA,IACL;AACA,IAAA,IAAI,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACpD,MAAA,IAAA,CAAK,QAAQ,UAAA,CAAW,MAAM,KAAK,IAAA,EAAK,EAAG,KAAK,YAAY,CAAA;AAC5D,MAAA,IAAA,CAAK,MAAM,KAAA,IAAQ;AAAA,IACrB;AAAA,EACF;AAAA,EAEQ,YAAA,GAAqB;AAC3B,IAAA,IAAI,CAAC,IAAA,CAAK,OAAA,IAAW,KAAK,QAAA,KAAa,CAAA,IAAK,KAAK,YAAA,EAAc;AAC7D,MAAA,IAAA,CAAK,YAAA,EAAa;AAClB,MAAA,IAAA,CAAK,YAAA,GAAe,MAAA;AAAA,IACtB;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAsB;AAC1B,IAAA,IAAA,CAAK,OAAA,GAAU,KAAA;AACf,IAAA,IAAI,IAAA,CAAK,KAAA,EAAO,YAAA,CAAa,IAAA,CAAK,KAAK,CAAA;AACvC,IAAA,IAAI,IAAA,CAAK,aAAa,CAAA,EAAG;AACzB,IAAA,MAAM,IAAI,OAAA,CAAc,CAAC,OAAA,KAAY;AACnC,MAAA,IAAA,CAAK,YAAA,GAAe,OAAA;AAAA,IACtB,CAAC,CAAA;AAAA,EACH;AACF,CAAA;AAOO,IAAM,KAAA,GAAN,cAAoBA,mBAAA,CAAa;AAAA,EACrB,MAAA;AAAA,EACA,WAAA;AAAA,EACA,OAAA;AAAA,EACA,gBAAA;AAAA,EACR,QAAA;AAAA,EACQ,UAAwB,EAAC;AAAA,EAE1C,WAAA,CAAY,EAAA,EAAa,IAAA,GAAqB,EAAC,EAAG;AAChD,IAAA,KAAA,EAAM;AACN,IAAA,IAAA,CAAK,SAAS,EAAA,CAAG,MAAA;AACjB,IAAA,IAAA,CAAK,WAAA,GAAc,KAAK,WAAA,IAAe,CAAA;AACvC,IAAA,IAAA,CAAK,OAAA,GAAU,KAAK,OAAA,IAAW,cAAA;AAC/B,IAAA,IAAA,CAAK,gBAAA,GAAmB,KAAK,gBAAA,IAAoB,KAAA;AACjD,IAAA,IAAA,CAAK,QAAA,GAAW,IAAA,CAAK,QAAA,IAAY,CAAA,EAAA,EAAK,QAAQ,GAAG,CAAA,CAAA;AAEjD,IAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,EAAE,CAAA,EAAG;AACpB,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,SAAA;AAAA,OAOF;AAEA,MAAA,IAAI;AACF,QAAA,IAAA,CAAK,MAAA,CAAO,KAAK,CAAA,wCAAA,CAA0C,CAAA;AAAA,MAC7D,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,iFAAA;AAAA,OACF;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,wDAAA;AAAA,OACF;AACA,MAAA,OAAA,CAAQ,IAAI,EAAE,CAAA;AAAA,IAChB;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,GAAA,CAAa,IAAA,EAAc,OAAA,EAAY,IAAA,GAAmB,EAAC,EAAW;AACpE,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,MAAM,QAAA,GAAW,KAAK,MAAA,CACnB,OAAA;AAAA,QACC,CAAA,+EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,KAAK,CAAA;AACjB,MAAA,IAAI,QAAA,EAAU,OAAO,WAAA,CAAY,QAAQ,CAAA;AAAA,IAC3C;AACA,IAAA,MAAM,QAAQ,IAAA,CAAK,KAAA,KAAU,KAAK,KAAA,GAAQ,CAAA,GAAI,KAAK,KAAA,GAAQ,CAAA,CAAA;AAC3D,IAAA,MAAM,IAAA,GAAO,KAAK,MAAA,CACf,OAAA;AAAA,MACC,CAAA;AAAA,sDAAA;AAAA,KAEF,CACC,GAAA;AAAA,MACC,IAAA;AAAA,MACA,KAAK,QAAA,IAAY,CAAA;AAAA,MACjB,KAAA;AAAA,MACA,IAAA,CAAK,eAAe,IAAA,CAAK,WAAA;AAAA,MACzB,IAAA,CAAK,SAAA,CAAU,OAAA,IAAW,IAAI,CAAA;AAAA,MAC9B,KAAK,KAAA,IAAS,IAAA;AAAA,MACd,CAAA;AAAA,MACA;AAAA,KACF;AACF,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,KAAA,MAAW,CAAA,IAAK,KAAK,OAAA,EAAS,IAAI,EAAE,IAAA,KAAS,IAAA,IAAQ,IAAA,EAAK;AAC1D,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA,EAGA,OAAA,CACE,IAAA,EACA,OAAA,EACA,IAAA,GAAuB,EAAC,EAChB;AACR,IAAA,MAAM,IAAI,IAAI,UAAA,CAAW,IAAA,EAAM,IAAA,EAAM,SAAoB,IAAI,CAAA;AAC7D,IAAA,IAAA,CAAK,OAAA,CAAQ,KAAK,CAAC,CAAA;AACnB,IAAA,OAAO,CAAA;AAAA,EACT;AAAA,EAEA,OAAgB,EAAA,EAAgC;AAC9C,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CACd,QAAQ,CAAA,gCAAA,CAAkC,CAAA,CAC1C,IAAI,EAAE,CAAA;AACT,IAAA,OAAO,GAAA,GAAO,WAAA,CAAY,GAAG,CAAA,GAAe,MAAA;AAAA,EAC9C;AAAA;AAAA,EAGA,OAAO,IAAA,EAA0C;AAC/C,IAAA,MAAM,IAAA,GACJ,IAAA,GACI,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,MACC,CAAA,uEAAA;AAAA,KACF,CACC,IAAI,IAAI,CAAA,GACX,KAAK,MAAA,CACF,OAAA,CAAQ,CAAA,uDAAA,CAAyD,CAAA,CACjE,GAAA,EAAI;AAEb,IAAA,MAAM,GAAA,GAAiC;AAAA,MACrC,OAAA,EAAS,CAAA;AAAA,MACT,MAAA,EAAQ,CAAA;AAAA,MACR,IAAA,EAAM,CAAA;AAAA,MACN,MAAA,EAAQ;AAAA,KACV;AACA,IAAA,KAAA,MAAW,KAAK,IAAA,EAAM,GAAA,CAAI,CAAA,CAAE,MAAM,IAAI,CAAA,CAAE,CAAA;AACxC,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,OAAA,CAAQ,cAAc,GAAA,EAAgB;AACpC,IAAA,OAAO,KAAK,MAAA,CACT,OAAA;AAAA,MACC,CAAA;AAAA,iDAAA;AAAA,MAGD,GAAA,CAAI,GAAA,IAAO,GAAA,EAAI,GAAI,WAAW,CAAA,CAAE,OAAA;AAAA,EACrC;AAAA;AAAA,EAGA,MAAM,KAAA,GAAuB;AAC3B,IAAA,MAAM,OAAA,CAAQ,GAAA,CAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,CAAC,CAAA,KAAM,CAAA,CAAE,IAAA,EAAM,CAAC,CAAA;AAAA,EACrD;AAAA;AAAA,EAGA,cAAc,IAAA,EAA0B;AACtC,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,MAAM,GAAA,GAAM,KAAK,MAAA,CACd,OAAA;AAAA,MACC,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,oBAAA;AAAA,MAQD,GAAA,CAAI,IAAA,CAAK,QAAA,EAAU,CAAA,EAAG,MAAM,CAAC,CAAA;AAChC,IAAA,OAAO,GAAA,GAAM,WAAA,CAAY,GAAG,CAAA,GAAI,IAAA;AAAA,EAClC;AAAA;AAAA,EAGA,gBAAA,CAAiB,KAAU,MAAA,EAAuB;AAChD,IAAA,IAAI,KAAK,gBAAA,EAAkB;AACzB,MAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,CAAA,8BAAA,CAAgC,CAAA,CAAE,GAAA,CAAI,IAAI,EAAE,CAAA;AAAA,IAClE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,SAAA,CAAU,MAAA,IAAU,IAAI,CAAA,EAAG,GAAA,EAAI,EAAG,GAAA,CAAI,EAAE,CAAA;AAAA,IACtD;AAAA,EACF;AAAA;AAAA,EAGA,YAAA,CAAa,KAAU,GAAA,EAAoB;AAEzC,IAAA,MAAM,UAAU,GAAA,YAAe,KAAA,GAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAC/D,IAAA,IAAI,GAAA,CAAI,QAAA,GAAW,GAAA,CAAI,WAAA,EAAa;AAClC,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6FAAA;AAAA,OACF,CACC,GAAA,CAAI,GAAA,EAAI,GAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,QAAQ,CAAA,EAAG,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IACnE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,kEAAA;AAAA,QAED,GAAA,CAAI,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IAC/B;AAAA,EACF;AACF;AAGO,SAAS,WAAA,CAAY,IAAa,IAAA,EAA4B;AACnE,EAAA,OAAO,IAAI,KAAA,CAAM,EAAA,EAAI,IAAI,CAAA;AAC3B","file":"index.cjs","sourcesContent":["import { EventEmitter } from \"node:events\";\nimport type { Monlite } from \"@monlite/core\";\n\nexport type JobStatus = \"pending\" | \"active\" | \"done\" | \"failed\";\n\nexport interface Job<T = any> {\n id: number;\n queue: string;\n /** Dedupe key, if the job was added with one. */\n jobId?: string;\n status: JobStatus;\n priority: number;\n payload: T;\n /** Number of attempts already made (0 until the first run). */\n attempts: number;\n maxAttempts: number;\n runAt: number;\n result?: any;\n error?: string;\n createdAt: number;\n updatedAt: number;\n}\n\nexport interface AddOptions {\n /** Dedupe key — skip if a job with this id is already pending/active. */\n jobId?: string;\n /** Delay before the job becomes runnable (ms). */\n delay?: number;\n /** Explicit epoch-ms run time (overrides `delay`). */\n runAt?: number;\n /** Higher runs first. Default 0. */\n priority?: number;\n /** Total attempts before dead-lettering. Default: the queue's `maxAttempts`. */\n maxAttempts?: number;\n}\n\nexport interface ProcessOptions {\n /** Jobs run concurrently per worker. Default 1. */\n concurrency?: number;\n /** How often to poll for due jobs when idle (ms). Default 500. */\n pollInterval?: number;\n}\n\nexport interface QueueOptions {\n /** Default attempts before dead-lettering. Default 1 (no retry). */\n maxAttempts?: number;\n /** Backoff before retry N (ms). Default: exponential, capped at 30s. */\n backoff?: (attempt: number) => number;\n /** Delete jobs once completed instead of keeping them as `done`. Default false. */\n removeOnComplete?: boolean;\n /** Identifies this worker process in the `locked_by` column. */\n workerId?: string;\n}\n\nexport type Handler<T = any, R = any> = (job: Job<T>) => Promise<R> | R;\n\nexport interface Worker {\n /** Stop claiming new jobs and wait for in-flight ones to finish. */\n stop(): Promise<void>;\n}\n\ninterface Row {\n id: number;\n queue: string;\n status: JobStatus;\n priority: number;\n run_at: number;\n attempts: number;\n max_attempts: number;\n payload: string;\n result: string | null;\n error: string | null;\n job_id: string | null;\n created_at: number;\n updated_at: number;\n}\n\nconst ensured = new WeakSet<object>();\nconst now = () => Date.now();\nconst defaultBackoff = (attempt: number) =>\n Math.min(30_000, 1000 * 2 ** (attempt - 1));\n\nfunction deserialize(row: Row): Job {\n return {\n id: row.id,\n queue: row.queue,\n jobId: row.job_id ?? undefined,\n status: row.status,\n priority: row.priority,\n payload: JSON.parse(row.payload),\n attempts: row.attempts,\n maxAttempts: row.max_attempts,\n runAt: row.run_at,\n result: row.result != null ? JSON.parse(row.result) : undefined,\n error: row.error ?? undefined,\n createdAt: row.created_at,\n updatedAt: row.updated_at,\n };\n}\n\nclass WorkerImpl implements Worker {\n private running = true;\n private inFlight = 0;\n private timer: ReturnType<typeof setTimeout> | undefined;\n private drainResolve: (() => void) | undefined;\n private readonly concurrency: number;\n private readonly pollInterval: number;\n\n constructor(\n private readonly q: Queue,\n readonly name: string,\n private readonly handler: Handler,\n opts: ProcessOptions,\n ) {\n this.concurrency = Math.max(1, opts.concurrency ?? 1);\n this.pollInterval = opts.pollInterval ?? 500;\n this.kick();\n }\n\n /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */\n kick(): void {\n if (!this.running) return;\n if (this.timer) {\n clearTimeout(this.timer);\n this.timer = undefined;\n }\n while (this.running && this.inFlight < this.concurrency) {\n const job = this.q.claimInternal(this.name);\n if (!job) break;\n this.inFlight++;\n Promise.resolve()\n .then(() => this.handler(job))\n .then(\n (result) => {\n this.q.completeInternal(job, result);\n this.q.emit(\"completed\", job, result);\n },\n (err) => {\n this.q.failInternal(job, err);\n this.q.emit(\"failed\", job, err);\n },\n )\n .finally(() => {\n this.inFlight--;\n if (this.running) this.kick();\n else this.checkDrained();\n });\n }\n if (this.running && this.inFlight < this.concurrency) {\n this.timer = setTimeout(() => this.kick(), this.pollInterval);\n this.timer.unref?.();\n }\n }\n\n private checkDrained(): void {\n if (!this.running && this.inFlight === 0 && this.drainResolve) {\n this.drainResolve();\n this.drainResolve = undefined;\n }\n }\n\n async stop(): Promise<void> {\n this.running = false;\n if (this.timer) clearTimeout(this.timer);\n if (this.inFlight === 0) return;\n await new Promise<void>((resolve) => {\n this.drainResolve = resolve;\n });\n }\n}\n\n/**\n * A durable, multi-process-safe job queue backed by SQLite. Producers `add`\n * jobs; workers `process` them with retries, backoff, delays, and concurrency.\n * Emits `\"completed\"` (job, result) and `\"failed\"` (job, error).\n */\nexport class Queue extends EventEmitter {\n private readonly driver: Monlite[\"driver\"];\n private readonly maxAttempts: number;\n private readonly backoff: (attempt: number) => number;\n private readonly removeOnComplete: boolean;\n readonly workerId: string;\n private readonly workers: WorkerImpl[] = [];\n\n constructor(db: Monlite, opts: QueueOptions = {}) {\n super();\n this.driver = db.driver;\n this.maxAttempts = opts.maxAttempts ?? 1;\n this.backoff = opts.backoff ?? defaultBackoff;\n this.removeOnComplete = opts.removeOnComplete ?? false;\n this.workerId = opts.workerId ?? `w-${process.pid}`;\n\n if (!ensured.has(db)) {\n this.driver.exec(\n `CREATE TABLE IF NOT EXISTS _jobs (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n queue TEXT NOT NULL, status TEXT NOT NULL, priority INTEGER NOT NULL,\n run_at INTEGER NOT NULL, attempts INTEGER NOT NULL, max_attempts INTEGER NOT NULL,\n payload TEXT NOT NULL, result TEXT, error TEXT, locked_by TEXT, job_id TEXT,\n created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL\n )`,\n );\n // Add job_id to pre-existing tables (idempotent).\n try {\n this.driver.exec(`ALTER TABLE _jobs ADD COLUMN job_id TEXT`);\n } catch {\n /* column already exists */\n }\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_claim ON _jobs (queue, status, priority, run_at)`,\n );\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_jobid ON _jobs (job_id)`,\n );\n ensured.add(db);\n }\n }\n\n /**\n * Enqueue a job. Pass `opts.jobId` to **dedupe**: if a job with that id is\n * already pending or active, the existing job is returned instead of adding a\n * duplicate (idempotent enqueue — e.g. for resume/replay).\n */\n add<T = any>(name: string, payload: T, opts: AddOptions = {}): Job<T> {\n const t = now();\n if (opts.jobId) {\n const existing = this.driver\n .prepare(\n `SELECT * FROM _jobs WHERE job_id = ? AND status IN ('pending','active') LIMIT 1`,\n )\n .get(opts.jobId) as Row | undefined;\n if (existing) return deserialize(existing) as Job<T>;\n }\n const runAt = opts.runAt ?? (opts.delay ? t + opts.delay : t);\n const info = this.driver\n .prepare(\n `INSERT INTO _jobs (queue, status, priority, run_at, attempts, max_attempts, payload, job_id, created_at, updated_at)\n VALUES (?, 'pending', ?, ?, 0, ?, ?, ?, ?, ?)`,\n )\n .run(\n name,\n opts.priority ?? 0,\n runAt,\n opts.maxAttempts ?? this.maxAttempts,\n JSON.stringify(payload ?? null),\n opts.jobId ?? null,\n t,\n t,\n );\n const job = this.getJob(Number(info.lastInsertRowid))!;\n for (const w of this.workers) if (w.name === name) w.kick();\n return job as Job<T>;\n }\n\n /** Register a worker for a queue. Returns a handle with `stop()`. */\n process<T = any, R = any>(\n name: string,\n handler: Handler<T, R>,\n opts: ProcessOptions = {},\n ): Worker {\n const w = new WorkerImpl(this, name, handler as Handler, opts);\n this.workers.push(w);\n return w;\n }\n\n getJob<T = any>(id: number): Job<T> | undefined {\n const row = this.driver\n .prepare(`SELECT * FROM _jobs WHERE id = ?`)\n .get(id) as Row | undefined;\n return row ? (deserialize(row) as Job<T>) : undefined;\n }\n\n /** Count jobs by status (optionally for one queue). */\n counts(name?: string): Record<JobStatus, number> {\n const rows = (\n name\n ? this.driver\n .prepare(\n `SELECT status, COUNT(*) AS n FROM _jobs WHERE queue = ? GROUP BY status`,\n )\n .all(name)\n : this.driver\n .prepare(`SELECT status, COUNT(*) AS n FROM _jobs GROUP BY status`)\n .all()\n ) as Array<{ status: JobStatus; n: number }>;\n const out: Record<JobStatus, number> = {\n pending: 0,\n active: 0,\n done: 0,\n failed: 0,\n };\n for (const r of rows) out[r.status] = r.n;\n return out;\n }\n\n /**\n * Reset jobs stuck in `active` (e.g. from a crashed worker) back to `pending`\n * if they haven't been touched in `olderThanMs`. Returns the count recovered.\n */\n recover(olderThanMs = 60_000): number {\n return this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', locked_by=NULL, updated_at=?\n WHERE status='active' AND updated_at < ?`,\n )\n .run(now(), now() - olderThanMs).changes;\n }\n\n /** Stop all workers and wait for in-flight jobs to finish. */\n async close(): Promise<void> {\n await Promise.all(this.workers.map((w) => w.stop()));\n }\n\n /** @internal Atomically claim the next due job, counting the attempt. */\n claimInternal(name: string): Job | null {\n const t = now();\n const row = this.driver\n .prepare(\n `UPDATE _jobs SET status='active', attempts=attempts+1, locked_by=?, updated_at=?\n WHERE id = (\n SELECT id FROM _jobs\n WHERE queue=? AND status='pending' AND run_at<=?\n ORDER BY priority DESC, id ASC LIMIT 1\n )\n RETURNING *`,\n )\n .get(this.workerId, t, name, t) as Row | undefined;\n return row ? deserialize(row) : null;\n }\n\n /** @internal */\n completeInternal(job: Job, result: unknown): void {\n if (this.removeOnComplete) {\n this.driver.prepare(`DELETE FROM _jobs WHERE id = ?`).run(job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='done', result=?, error=NULL, updated_at=? WHERE id=?`,\n )\n .run(JSON.stringify(result ?? null), now(), job.id);\n }\n }\n\n /** @internal */\n failInternal(job: Job, err: unknown): void {\n // `job.attempts` was already incremented at claim time.\n const message = err instanceof Error ? err.message : String(err);\n if (job.attempts < job.maxAttempts) {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', run_at=?, error=?, locked_by=NULL, updated_at=? WHERE id=?`,\n )\n .run(now() + this.backoff(job.attempts), message, now(), job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='failed', error=?, updated_at=? WHERE id=?`,\n )\n .run(message, now(), job.id);\n }\n }\n}\n\n/** Create a job queue over a monlite database. */\nexport function createQueue(db: Monlite, opts?: QueueOptions): Queue {\n return new Queue(db, opts);\n}\n"]}
1
+ {"version":3,"sources":["../src/index.ts"],"names":["EventEmitter"],"mappings":";;;;;AAoFA,IAAM,OAAA,uBAAc,OAAA,EAAgB;AACpC,IAAM,GAAA,GAAM,MAAM,IAAA,CAAK,GAAA,EAAI;AAC3B,IAAM,cAAA,GAAiB,CAAC,OAAA,KACtB,IAAA,CAAK,IAAI,GAAA,EAAQ,GAAA,GAAO,CAAA,KAAM,OAAA,GAAU,CAAA,CAAE,CAAA;AAE5C,SAAS,YAAY,GAAA,EAAe;AAClC,EAAA,OAAO;AAAA,IACL,IAAI,GAAA,CAAI,EAAA;AAAA,IACR,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,KAAA,EAAO,IAAI,MAAA,IAAU,MAAA;AAAA,IACrB,QAAQ,GAAA,CAAI,MAAA;AAAA,IACZ,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAA,EAAS,IAAA,CAAK,KAAA,CAAM,GAAA,CAAI,OAAO,CAAA;AAAA,IAC/B,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,aAAa,GAAA,CAAI,YAAA;AAAA,IACjB,OAAO,GAAA,CAAI,MAAA;AAAA,IACX,MAAA,EAAQ,IAAI,MAAA,IAAU,IAAA,GAAO,KAAK,KAAA,CAAM,GAAA,CAAI,MAAM,CAAA,GAAI,MAAA;AAAA,IACtD,KAAA,EAAO,IAAI,KAAA,IAAS,MAAA;AAAA,IACpB,WAAW,GAAA,CAAI,UAAA;AAAA,IACf,WAAW,GAAA,CAAI;AAAA,GACjB;AACF;AAEA,IAAM,aAAN,MAAmC;AAAA,EAUjC,WAAA,CACmB,CAAA,EACR,IAAA,EACQ,OAAA,EACjB,IAAA,EACA;AAJiB,IAAA,IAAA,CAAA,CAAA,GAAA,CAAA;AACR,IAAA,IAAA,CAAA,IAAA,GAAA,IAAA;AACQ,IAAA,IAAA,CAAA,OAAA,GAAA,OAAA;AAGjB,IAAA,IAAA,CAAK,cAAc,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,IAAA,CAAK,YAAA,GAAe,KAAK,YAAA,IAAgB,GAAA;AACzC,IAAA,IAAA,CAAK,oBAAoB,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,qBAAqB,CAAC,CAAA;AAChE,IAAA,IAAI,IAAA,CAAK,oBAAoB,CAAA,EAAG;AAC9B,MAAA,IAAA,CAAK,MAAA,GAAS,YAAY,MAAM;AAC9B,QAAA,IAAI,KAAK,OAAA,EAAS,IAAA,CAAK,CAAA,CAAE,OAAA,CAAQ,KAAK,iBAAiB,CAAA;AAAA,MACzD,CAAA,EAAG,IAAA,CAAK,GAAA,CAAI,GAAA,EAAM,IAAA,CAAK,MAAM,IAAA,CAAK,iBAAA,GAAoB,CAAC,CAAC,CAAC,CAAA;AACzD,MAAA,IAAA,CAAK,OAAO,KAAA,IAAQ;AAAA,IACtB;AACA,IAAA,IAAA,CAAK,IAAA,EAAK;AAAA,EACZ;AAAA,EAfmB,CAAA;AAAA,EACR,IAAA;AAAA,EACQ,OAAA;AAAA,EAZX,OAAA,GAAU,IAAA;AAAA,EACV,QAAA,GAAW,CAAA;AAAA,EACX,KAAA;AAAA,EACA,YAAA;AAAA,EACS,WAAA;AAAA,EACA,YAAA;AAAA,EACA,iBAAA;AAAA,EACT,MAAA;AAAA;AAAA,EAqBR,IAAA,GAAa;AACX,IAAA,IAAI,CAAC,KAAK,OAAA,EAAS;AACnB,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,KAAK,KAAK,CAAA;AACvB,MAAA,IAAA,CAAK,KAAA,GAAQ,MAAA;AAAA,IACf;AACA,IAAA,OAAO,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACvD,MAAA,MAAM,GAAA,GAAM,IAAA,CAAK,CAAA,CAAE,aAAA,CAAc,KAAK,IAAI,CAAA;AAC1C,MAAA,IAAI,CAAC,GAAA,EAAK;AACV,MAAA,IAAA,CAAK,QAAA,EAAA;AAGL,MAAA,IAAI,EAAA;AACJ,MAAA,IAAI,IAAA,CAAK,oBAAoB,CAAA,EAAG;AAC9B,QAAA,EAAA,GAAK,WAAA;AAAA,UACH,MAAM,IAAA,CAAK,CAAA,CAAE,iBAAA,CAAkB,IAAI,EAAE,CAAA;AAAA,UACrC,IAAA,CAAK,IAAI,GAAA,EAAM,IAAA,CAAK,MAAM,IAAA,CAAK,iBAAA,GAAoB,CAAC,CAAC;AAAA,SACvD;AACA,QAAA,EAAA,CAAG,KAAA,IAAQ;AAAA,MACb;AACA,MAAA,OAAA,CAAQ,OAAA,GACL,IAAA,CAAK,MAAM,KAAK,OAAA,CAAQ,GAAG,CAAC,CAAA,CAC5B,IAAA;AAAA,QACC,CAAC,MAAA,KAAW;AACV,UAAA,IAAA,CAAK,CAAA,CAAE,gBAAA,CAAiB,GAAA,EAAK,MAAM,CAAA;AACnC,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,WAAA,EAAa,GAAA,EAAK,MAAM,CAAA;AAAA,QACtC,CAAA;AAAA,QACA,CAAC,GAAA,KAAQ;AACP,UAAA,IAAA,CAAK,CAAA,CAAE,YAAA,CAAa,GAAA,EAAK,GAAG,CAAA;AAC5B,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,QAAA,EAAU,GAAA,EAAK,GAAG,CAAA;AAAA,QAChC;AAAA,OACF,CACC,QAAQ,MAAM;AACb,QAAA,IAAI,EAAA,gBAAkB,EAAE,CAAA;AACxB,QAAA,IAAA,CAAK,QAAA,EAAA;AACL,QAAA,IAAI,IAAA,CAAK,OAAA,EAAS,IAAA,CAAK,IAAA,EAAK;AAAA,kBAClB,YAAA,EAAa;AAAA,MACzB,CAAC,CAAA;AAAA,IACL;AACA,IAAA,IAAI,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACpD,MAAA,IAAA,CAAK,QAAQ,UAAA,CAAW,MAAM,KAAK,IAAA,EAAK,EAAG,KAAK,YAAY,CAAA;AAC5D,MAAA,IAAA,CAAK,MAAM,KAAA,IAAQ;AAAA,IACrB;AAAA,EACF;AAAA,EAEQ,YAAA,GAAqB;AAC3B,IAAA,IAAI,CAAC,IAAA,CAAK,OAAA,IAAW,KAAK,QAAA,KAAa,CAAA,IAAK,KAAK,YAAA,EAAc;AAC7D,MAAA,IAAA,CAAK,YAAA,EAAa;AAClB,MAAA,IAAA,CAAK,YAAA,GAAe,MAAA;AAAA,IACtB;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAsB;AAC1B,IAAA,IAAA,CAAK,OAAA,GAAU,KAAA;AACf,IAAA,IAAI,IAAA,CAAK,KAAA,EAAO,YAAA,CAAa,IAAA,CAAK,KAAK,CAAA;AACvC,IAAA,IAAI,IAAA,CAAK,MAAA,EAAQ,aAAA,CAAc,IAAA,CAAK,MAAM,CAAA;AAC1C,IAAA,IAAI,IAAA,CAAK,aAAa,CAAA,EAAG;AACzB,IAAA,MAAM,IAAI,OAAA,CAAc,CAAC,OAAA,KAAY;AACnC,MAAA,IAAA,CAAK,YAAA,GAAe,OAAA;AAAA,IACtB,CAAC,CAAA;AAAA,EACH;AACF,CAAA;AAOO,IAAM,KAAA,GAAN,cAAoBA,mBAAA,CAAa;AAAA,EACrB,MAAA;AAAA,EACA,WAAA;AAAA,EACA,OAAA;AAAA,EACA,gBAAA;AAAA,EACR,QAAA;AAAA,EACQ,UAAwB,EAAC;AAAA,EAE1C,WAAA,CAAY,EAAA,EAAa,IAAA,GAAqB,EAAC,EAAG;AAChD,IAAA,KAAA,EAAM;AACN,IAAA,IAAA,CAAK,SAAS,EAAA,CAAG,MAAA;AACjB,IAAA,IAAA,CAAK,WAAA,GAAc,KAAK,WAAA,IAAe,CAAA;AACvC,IAAA,IAAA,CAAK,OAAA,GAAU,KAAK,OAAA,IAAW,cAAA;AAC/B,IAAA,IAAA,CAAK,gBAAA,GAAmB,KAAK,gBAAA,IAAoB,KAAA;AAEjD,IAAA,IAAA,CAAK,WACH,IAAA,CAAK,QAAA,IACL,CAAA,EAAA,EAAK,OAAO,YAAY,WAAA,IAAe,OAAA,CAAQ,GAAA,GAAM,OAAA,CAAQ,MAAM,IAAA,CAAK,KAAA,CAAM,KAAK,MAAA,EAAO,GAAI,GAAG,CAAC,CAAA,CAAA;AAEpG,IAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,EAAE,CAAA,EAAG;AACpB,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,SAAA;AAAA,OAOF;AAEA,MAAA,IAAI;AACF,QAAA,IAAA,CAAK,MAAA,CAAO,KAAK,CAAA,wCAAA,CAA0C,CAAA;AAAA,MAC7D,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,iFAAA;AAAA,OACF;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,wDAAA;AAAA,OACF;AACA,MAAA,OAAA,CAAQ,IAAI,EAAE,CAAA;AAAA,IAChB;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,GAAA,CAAa,IAAA,EAAc,OAAA,EAAY,IAAA,GAAmB,EAAC,EAAW;AACpE,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,MAAM,QAAA,GAAW,KAAK,MAAA,CACnB,OAAA;AAAA,QACC,CAAA,+EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,KAAK,CAAA;AACjB,MAAA,IAAI,QAAA,EAAU,OAAO,WAAA,CAAY,QAAQ,CAAA;AAAA,IAC3C;AACA,IAAA,MAAM,QAAQ,IAAA,CAAK,KAAA,KAAU,KAAK,KAAA,GAAQ,CAAA,GAAI,KAAK,KAAA,GAAQ,CAAA,CAAA;AAC3D,IAAA,MAAM,IAAA,GAAO,KAAK,MAAA,CACf,OAAA;AAAA,MACC,CAAA;AAAA,sDAAA;AAAA,KAEF,CACC,GAAA;AAAA,MACC,IAAA;AAAA,MACA,KAAK,QAAA,IAAY,CAAA;AAAA,MACjB,KAAA;AAAA,MACA,IAAA,CAAK,eAAe,IAAA,CAAK,WAAA;AAAA,MACzB,IAAA,CAAK,SAAA,CAAU,OAAA,IAAW,IAAI,CAAA;AAAA,MAC9B,KAAK,KAAA,IAAS,IAAA;AAAA,MACd,CAAA;AAAA,MACA;AAAA,KACF;AACF,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,KAAA,MAAW,CAAA,IAAK,KAAK,OAAA,EAAS,IAAI,EAAE,IAAA,KAAS,IAAA,IAAQ,IAAA,EAAK;AAC1D,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA,EAGA,OAAA,CACE,IAAA,EACA,OAAA,EACA,IAAA,GAAuB,EAAC,EAChB;AACR,IAAA,MAAM,IAAI,IAAI,UAAA,CAAW,IAAA,EAAM,IAAA,EAAM,SAAoB,IAAI,CAAA;AAC7D,IAAA,IAAA,CAAK,OAAA,CAAQ,KAAK,CAAC,CAAA;AACnB,IAAA,OAAO,CAAA;AAAA,EACT;AAAA,EAEA,OAAgB,EAAA,EAAgC;AAC9C,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CACd,QAAQ,CAAA,gCAAA,CAAkC,CAAA,CAC1C,IAAI,EAAE,CAAA;AACT,IAAA,OAAO,GAAA,GAAO,WAAA,CAAY,GAAG,CAAA,GAAe,MAAA;AAAA,EAC9C;AAAA;AAAA,EAGA,OAAO,IAAA,EAA0C;AAC/C,IAAA,MAAM,IAAA,GACJ,IAAA,GACI,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,MACC,CAAA,uEAAA;AAAA,KACF,CACC,IAAI,IAAI,CAAA,GACX,KAAK,MAAA,CACF,OAAA,CAAQ,CAAA,uDAAA,CAAyD,CAAA,CACjE,GAAA,EAAI;AAEb,IAAA,MAAM,GAAA,GAAiC;AAAA,MACrC,OAAA,EAAS,CAAA;AAAA,MACT,MAAA,EAAQ,CAAA;AAAA,MACR,IAAA,EAAM,CAAA;AAAA,MACN,MAAA,EAAQ;AAAA,KACV;AACA,IAAA,KAAA,MAAW,KAAK,IAAA,EAAM,GAAA,CAAI,CAAA,CAAE,MAAM,IAAI,CAAA,CAAE,CAAA;AACxC,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,OAAA,CAAQ,cAAc,GAAA,EAAgB;AACpC,IAAA,OAAO,KAAK,MAAA,CACT,OAAA;AAAA,MACC,CAAA;AAAA,iDAAA;AAAA,MAGD,GAAA,CAAI,GAAA,IAAO,GAAA,EAAI,GAAI,WAAW,CAAA,CAAE,OAAA;AAAA,EACrC;AAAA;AAAA,EAGA,kBAAkB,EAAA,EAAkB;AAClC,IAAA,IAAA,CAAK,OACF,OAAA,CAAQ,CAAA,4DAAA,CAA8D,EACtE,GAAA,CAAI,GAAA,IAAO,EAAE,CAAA;AAAA,EAClB;AAAA;AAAA,EAGA,MAAM,KAAA,GAAuB;AAC3B,IAAA,MAAM,OAAA,CAAQ,GAAA,CAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,CAAC,CAAA,KAAM,CAAA,CAAE,IAAA,EAAM,CAAC,CAAA;AAAA,EACrD;AAAA;AAAA,EAGA,cAAc,IAAA,EAA0B;AACtC,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,MAAM,GAAA,GAAM,KAAK,MAAA,CACd,OAAA;AAAA,MACC,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,oBAAA;AAAA,MAQD,GAAA,CAAI,IAAA,CAAK,QAAA,EAAU,CAAA,EAAG,MAAM,CAAC,CAAA;AAChC,IAAA,OAAO,GAAA,GAAM,WAAA,CAAY,GAAG,CAAA,GAAI,IAAA;AAAA,EAClC;AAAA;AAAA,EAGA,gBAAA,CAAiB,KAAU,MAAA,EAAuB;AAChD,IAAA,IAAI,KAAK,gBAAA,EAAkB;AACzB,MAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,CAAA,8BAAA,CAAgC,CAAA,CAAE,GAAA,CAAI,IAAI,EAAE,CAAA;AAAA,IAClE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,SAAA,CAAU,MAAA,IAAU,IAAI,CAAA,EAAG,GAAA,EAAI,EAAG,GAAA,CAAI,EAAE,CAAA;AAAA,IACtD;AAAA,EACF;AAAA;AAAA,EAGA,YAAA,CAAa,KAAU,GAAA,EAAoB;AAEzC,IAAA,MAAM,UAAU,GAAA,YAAe,KAAA,GAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAC/D,IAAA,IAAI,GAAA,CAAI,QAAA,GAAW,GAAA,CAAI,WAAA,EAAa;AAClC,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6FAAA;AAAA,OACF,CACC,GAAA,CAAI,GAAA,EAAI,GAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,QAAQ,CAAA,EAAG,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IACnE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,kEAAA;AAAA,QAED,GAAA,CAAI,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IAC/B;AAAA,EACF;AACF;AAGO,SAAS,WAAA,CAAY,IAAa,IAAA,EAA4B;AACnE,EAAA,OAAO,IAAI,KAAA,CAAM,EAAA,EAAI,IAAI,CAAA;AAC3B","file":"index.cjs","sourcesContent":["import { EventEmitter } from \"node:events\";\nimport type { Monlite } from \"@monlite/core\";\n\nexport type JobStatus = \"pending\" | \"active\" | \"done\" | \"failed\";\n\nexport interface Job<T = any> {\n id: number;\n queue: string;\n /** Dedupe key, if the job was added with one. */\n jobId?: string;\n status: JobStatus;\n priority: number;\n payload: T;\n /** Number of attempts already made (0 until the first run). */\n attempts: number;\n maxAttempts: number;\n runAt: number;\n result?: any;\n error?: string;\n createdAt: number;\n updatedAt: number;\n}\n\nexport interface AddOptions {\n /** Dedupe key — skip if a job with this id is already pending/active. */\n jobId?: string;\n /** Delay before the job becomes runnable (ms). */\n delay?: number;\n /** Explicit epoch-ms run time (overrides `delay`). */\n runAt?: number;\n /** Higher runs first. Default 0. */\n priority?: number;\n /** Total attempts before dead-lettering. Default: the queue's `maxAttempts`. */\n maxAttempts?: number;\n}\n\nexport interface ProcessOptions {\n /** Jobs run concurrently per worker. Default 1. */\n concurrency?: number;\n /** How often to poll for due jobs when idle (ms). Default 500. */\n pollInterval?: number;\n /**\n * Visibility timeout (ms). If set, a crashed worker's job is automatically\n * reclaimed: a job that stays `active` without a heartbeat for this long is\n * returned to `pending`. While a handler runs, its job is heartbeated so a\n * legitimately long job isn't reaped. Off by default (jobs are never reaped).\n */\n visibilityTimeout?: number;\n}\n\nexport interface QueueOptions {\n /** Default attempts before dead-lettering. Default 1 (no retry). */\n maxAttempts?: number;\n /** Backoff before retry N (ms). Default: exponential, capped at 30s. */\n backoff?: (attempt: number) => number;\n /** Delete jobs once completed instead of keeping them as `done`. Default false. */\n removeOnComplete?: boolean;\n /** Identifies this worker process in the `locked_by` column. */\n workerId?: string;\n}\n\nexport type Handler<T = any, R = any> = (job: Job<T>) => Promise<R> | R;\n\nexport interface Worker {\n /** Stop claiming new jobs and wait for in-flight ones to finish. */\n stop(): Promise<void>;\n}\n\ninterface Row {\n id: number;\n queue: string;\n status: JobStatus;\n priority: number;\n run_at: number;\n attempts: number;\n max_attempts: number;\n payload: string;\n result: string | null;\n error: string | null;\n job_id: string | null;\n created_at: number;\n updated_at: number;\n}\n\nconst ensured = new WeakSet<object>();\nconst now = () => Date.now();\nconst defaultBackoff = (attempt: number) =>\n Math.min(30_000, 1000 * 2 ** (attempt - 1));\n\nfunction deserialize(row: Row): Job {\n return {\n id: row.id,\n queue: row.queue,\n jobId: row.job_id ?? undefined,\n status: row.status,\n priority: row.priority,\n payload: JSON.parse(row.payload),\n attempts: row.attempts,\n maxAttempts: row.max_attempts,\n runAt: row.run_at,\n result: row.result != null ? JSON.parse(row.result) : undefined,\n error: row.error ?? undefined,\n createdAt: row.created_at,\n updatedAt: row.updated_at,\n };\n}\n\nclass WorkerImpl implements Worker {\n private running = true;\n private inFlight = 0;\n private timer: ReturnType<typeof setTimeout> | undefined;\n private drainResolve: (() => void) | undefined;\n private readonly concurrency: number;\n private readonly pollInterval: number;\n private readonly visibilityTimeout: number;\n private reaper: ReturnType<typeof setInterval> | undefined;\n\n constructor(\n private readonly q: Queue,\n readonly name: string,\n private readonly handler: Handler,\n opts: ProcessOptions,\n ) {\n this.concurrency = Math.max(1, opts.concurrency ?? 1);\n this.pollInterval = opts.pollInterval ?? 500;\n this.visibilityTimeout = Math.max(0, opts.visibilityTimeout ?? 0);\n if (this.visibilityTimeout > 0) {\n this.reaper = setInterval(() => {\n if (this.running) this.q.recover(this.visibilityTimeout);\n }, Math.max(1000, Math.floor(this.visibilityTimeout / 2)));\n this.reaper.unref?.();\n }\n this.kick();\n }\n\n /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */\n kick(): void {\n if (!this.running) return;\n if (this.timer) {\n clearTimeout(this.timer);\n this.timer = undefined;\n }\n while (this.running && this.inFlight < this.concurrency) {\n const job = this.q.claimInternal(this.name);\n if (!job) break;\n this.inFlight++;\n // Heartbeat the job while it runs so the reaper's visibility timeout won't\n // reclaim a legitimately long-running job.\n let hb: ReturnType<typeof setInterval> | undefined;\n if (this.visibilityTimeout > 0) {\n hb = setInterval(\n () => this.q.heartbeatInternal(job.id),\n Math.max(1000, Math.floor(this.visibilityTimeout / 2)),\n );\n hb.unref?.();\n }\n Promise.resolve()\n .then(() => this.handler(job))\n .then(\n (result) => {\n this.q.completeInternal(job, result);\n this.q.emit(\"completed\", job, result);\n },\n (err) => {\n this.q.failInternal(job, err);\n this.q.emit(\"failed\", job, err);\n },\n )\n .finally(() => {\n if (hb) clearInterval(hb);\n this.inFlight--;\n if (this.running) this.kick();\n else this.checkDrained();\n });\n }\n if (this.running && this.inFlight < this.concurrency) {\n this.timer = setTimeout(() => this.kick(), this.pollInterval);\n this.timer.unref?.();\n }\n }\n\n private checkDrained(): void {\n if (!this.running && this.inFlight === 0 && this.drainResolve) {\n this.drainResolve();\n this.drainResolve = undefined;\n }\n }\n\n async stop(): Promise<void> {\n this.running = false;\n if (this.timer) clearTimeout(this.timer);\n if (this.reaper) clearInterval(this.reaper);\n if (this.inFlight === 0) return;\n await new Promise<void>((resolve) => {\n this.drainResolve = resolve;\n });\n }\n}\n\n/**\n * A durable, multi-process-safe job queue backed by SQLite. Producers `add`\n * jobs; workers `process` them with retries, backoff, delays, and concurrency.\n * Emits `\"completed\"` (job, result) and `\"failed\"` (job, error).\n */\nexport class Queue extends EventEmitter {\n private readonly driver: Monlite[\"driver\"];\n private readonly maxAttempts: number;\n private readonly backoff: (attempt: number) => number;\n private readonly removeOnComplete: boolean;\n readonly workerId: string;\n private readonly workers: WorkerImpl[] = [];\n\n constructor(db: Monlite, opts: QueueOptions = {}) {\n super();\n this.driver = db.driver;\n this.maxAttempts = opts.maxAttempts ?? 1;\n this.backoff = opts.backoff ?? defaultBackoff;\n this.removeOnComplete = opts.removeOnComplete ?? false;\n // `process` is absent in the browser — fall back to a random id there.\n this.workerId =\n opts.workerId ??\n `w-${typeof process !== \"undefined\" && process.pid ? process.pid : Math.floor(Math.random() * 1e6)}`;\n\n if (!ensured.has(db)) {\n this.driver.exec(\n `CREATE TABLE IF NOT EXISTS _jobs (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n queue TEXT NOT NULL, status TEXT NOT NULL, priority INTEGER NOT NULL,\n run_at INTEGER NOT NULL, attempts INTEGER NOT NULL, max_attempts INTEGER NOT NULL,\n payload TEXT NOT NULL, result TEXT, error TEXT, locked_by TEXT, job_id TEXT,\n created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL\n )`,\n );\n // Add job_id to pre-existing tables (idempotent).\n try {\n this.driver.exec(`ALTER TABLE _jobs ADD COLUMN job_id TEXT`);\n } catch {\n /* column already exists */\n }\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_claim ON _jobs (queue, status, priority, run_at)`,\n );\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_jobid ON _jobs (job_id)`,\n );\n ensured.add(db);\n }\n }\n\n /**\n * Enqueue a job. Pass `opts.jobId` to **dedupe**: if a job with that id is\n * already pending or active, the existing job is returned instead of adding a\n * duplicate (idempotent enqueue — e.g. for resume/replay).\n */\n add<T = any>(name: string, payload: T, opts: AddOptions = {}): Job<T> {\n const t = now();\n if (opts.jobId) {\n const existing = this.driver\n .prepare(\n `SELECT * FROM _jobs WHERE job_id = ? AND status IN ('pending','active') LIMIT 1`,\n )\n .get(opts.jobId) as Row | undefined;\n if (existing) return deserialize(existing) as Job<T>;\n }\n const runAt = opts.runAt ?? (opts.delay ? t + opts.delay : t);\n const info = this.driver\n .prepare(\n `INSERT INTO _jobs (queue, status, priority, run_at, attempts, max_attempts, payload, job_id, created_at, updated_at)\n VALUES (?, 'pending', ?, ?, 0, ?, ?, ?, ?, ?)`,\n )\n .run(\n name,\n opts.priority ?? 0,\n runAt,\n opts.maxAttempts ?? this.maxAttempts,\n JSON.stringify(payload ?? null),\n opts.jobId ?? null,\n t,\n t,\n );\n const job = this.getJob(Number(info.lastInsertRowid))!;\n for (const w of this.workers) if (w.name === name) w.kick();\n return job as Job<T>;\n }\n\n /** Register a worker for a queue. Returns a handle with `stop()`. */\n process<T = any, R = any>(\n name: string,\n handler: Handler<T, R>,\n opts: ProcessOptions = {},\n ): Worker {\n const w = new WorkerImpl(this, name, handler as Handler, opts);\n this.workers.push(w);\n return w;\n }\n\n getJob<T = any>(id: number): Job<T> | undefined {\n const row = this.driver\n .prepare(`SELECT * FROM _jobs WHERE id = ?`)\n .get(id) as Row | undefined;\n return row ? (deserialize(row) as Job<T>) : undefined;\n }\n\n /** Count jobs by status (optionally for one queue). */\n counts(name?: string): Record<JobStatus, number> {\n const rows = (\n name\n ? this.driver\n .prepare(\n `SELECT status, COUNT(*) AS n FROM _jobs WHERE queue = ? GROUP BY status`,\n )\n .all(name)\n : this.driver\n .prepare(`SELECT status, COUNT(*) AS n FROM _jobs GROUP BY status`)\n .all()\n ) as Array<{ status: JobStatus; n: number }>;\n const out: Record<JobStatus, number> = {\n pending: 0,\n active: 0,\n done: 0,\n failed: 0,\n };\n for (const r of rows) out[r.status] = r.n;\n return out;\n }\n\n /**\n * Reset jobs stuck in `active` (e.g. from a crashed worker) back to `pending`\n * if they haven't been touched in `olderThanMs`. Returns the count recovered.\n */\n recover(olderThanMs = 60_000): number {\n return this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', locked_by=NULL, updated_at=?\n WHERE status='active' AND updated_at < ?`,\n )\n .run(now(), now() - olderThanMs).changes;\n }\n\n /** @internal Extend a running job's visibility timeout (worker heartbeat). */\n heartbeatInternal(id: number): void {\n this.driver\n .prepare(`UPDATE _jobs SET updated_at=? WHERE id=? AND status='active'`)\n .run(now(), id);\n }\n\n /** Stop all workers and wait for in-flight jobs to finish. */\n async close(): Promise<void> {\n await Promise.all(this.workers.map((w) => w.stop()));\n }\n\n /** @internal Atomically claim the next due job, counting the attempt. */\n claimInternal(name: string): Job | null {\n const t = now();\n const row = this.driver\n .prepare(\n `UPDATE _jobs SET status='active', attempts=attempts+1, locked_by=?, updated_at=?\n WHERE id = (\n SELECT id FROM _jobs\n WHERE queue=? AND status='pending' AND run_at<=?\n ORDER BY priority DESC, id ASC LIMIT 1\n )\n RETURNING *`,\n )\n .get(this.workerId, t, name, t) as Row | undefined;\n return row ? deserialize(row) : null;\n }\n\n /** @internal */\n completeInternal(job: Job, result: unknown): void {\n if (this.removeOnComplete) {\n this.driver.prepare(`DELETE FROM _jobs WHERE id = ?`).run(job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='done', result=?, error=NULL, updated_at=? WHERE id=?`,\n )\n .run(JSON.stringify(result ?? null), now(), job.id);\n }\n }\n\n /** @internal */\n failInternal(job: Job, err: unknown): void {\n // `job.attempts` was already incremented at claim time.\n const message = err instanceof Error ? err.message : String(err);\n if (job.attempts < job.maxAttempts) {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', run_at=?, error=?, locked_by=NULL, updated_at=? WHERE id=?`,\n )\n .run(now() + this.backoff(job.attempts), message, now(), job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='failed', error=?, updated_at=? WHERE id=?`,\n )\n .run(message, now(), job.id);\n }\n }\n}\n\n/** Create a job queue over a monlite database. */\nexport function createQueue(db: Monlite, opts?: QueueOptions): Queue {\n return new Queue(db, opts);\n}\n"]}
package/dist/index.d.cts CHANGED
@@ -36,6 +36,13 @@ interface ProcessOptions {
36
36
  concurrency?: number;
37
37
  /** How often to poll for due jobs when idle (ms). Default 500. */
38
38
  pollInterval?: number;
39
+ /**
40
+ * Visibility timeout (ms). If set, a crashed worker's job is automatically
41
+ * reclaimed: a job that stays `active` without a heartbeat for this long is
42
+ * returned to `pending`. While a handler runs, its job is heartbeated so a
43
+ * legitimately long job isn't reaped. Off by default (jobs are never reaped).
44
+ */
45
+ visibilityTimeout?: number;
39
46
  }
40
47
  interface QueueOptions {
41
48
  /** Default attempts before dead-lettering. Default 1 (no retry). */
@@ -81,6 +88,8 @@ declare class Queue extends EventEmitter {
81
88
  * if they haven't been touched in `olderThanMs`. Returns the count recovered.
82
89
  */
83
90
  recover(olderThanMs?: number): number;
91
+ /** @internal Extend a running job's visibility timeout (worker heartbeat). */
92
+ heartbeatInternal(id: number): void;
84
93
  /** Stop all workers and wait for in-flight jobs to finish. */
85
94
  close(): Promise<void>;
86
95
  /** @internal Atomically claim the next due job, counting the attempt. */
package/dist/index.d.ts CHANGED
@@ -36,6 +36,13 @@ interface ProcessOptions {
36
36
  concurrency?: number;
37
37
  /** How often to poll for due jobs when idle (ms). Default 500. */
38
38
  pollInterval?: number;
39
+ /**
40
+ * Visibility timeout (ms). If set, a crashed worker's job is automatically
41
+ * reclaimed: a job that stays `active` without a heartbeat for this long is
42
+ * returned to `pending`. While a handler runs, its job is heartbeated so a
43
+ * legitimately long job isn't reaped. Off by default (jobs are never reaped).
44
+ */
45
+ visibilityTimeout?: number;
39
46
  }
40
47
  interface QueueOptions {
41
48
  /** Default attempts before dead-lettering. Default 1 (no retry). */
@@ -81,6 +88,8 @@ declare class Queue extends EventEmitter {
81
88
  * if they haven't been touched in `olderThanMs`. Returns the count recovered.
82
89
  */
83
90
  recover(olderThanMs?: number): number;
91
+ /** @internal Extend a running job's visibility timeout (worker heartbeat). */
92
+ heartbeatInternal(id: number): void;
84
93
  /** Stop all workers and wait for in-flight jobs to finish. */
85
94
  close(): Promise<void>;
86
95
  /** @internal Atomically claim the next due job, counting the attempt. */
package/dist/index.js CHANGED
@@ -28,6 +28,13 @@ var WorkerImpl = class {
28
28
  this.handler = handler;
29
29
  this.concurrency = Math.max(1, opts.concurrency ?? 1);
30
30
  this.pollInterval = opts.pollInterval ?? 500;
31
+ this.visibilityTimeout = Math.max(0, opts.visibilityTimeout ?? 0);
32
+ if (this.visibilityTimeout > 0) {
33
+ this.reaper = setInterval(() => {
34
+ if (this.running) this.q.recover(this.visibilityTimeout);
35
+ }, Math.max(1e3, Math.floor(this.visibilityTimeout / 2)));
36
+ this.reaper.unref?.();
37
+ }
31
38
  this.kick();
32
39
  }
33
40
  q;
@@ -39,6 +46,8 @@ var WorkerImpl = class {
39
46
  drainResolve;
40
47
  concurrency;
41
48
  pollInterval;
49
+ visibilityTimeout;
50
+ reaper;
42
51
  /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */
43
52
  kick() {
44
53
  if (!this.running) return;
@@ -50,6 +59,14 @@ var WorkerImpl = class {
50
59
  const job = this.q.claimInternal(this.name);
51
60
  if (!job) break;
52
61
  this.inFlight++;
62
+ let hb;
63
+ if (this.visibilityTimeout > 0) {
64
+ hb = setInterval(
65
+ () => this.q.heartbeatInternal(job.id),
66
+ Math.max(1e3, Math.floor(this.visibilityTimeout / 2))
67
+ );
68
+ hb.unref?.();
69
+ }
53
70
  Promise.resolve().then(() => this.handler(job)).then(
54
71
  (result) => {
55
72
  this.q.completeInternal(job, result);
@@ -60,6 +77,7 @@ var WorkerImpl = class {
60
77
  this.q.emit("failed", job, err);
61
78
  }
62
79
  ).finally(() => {
80
+ if (hb) clearInterval(hb);
63
81
  this.inFlight--;
64
82
  if (this.running) this.kick();
65
83
  else this.checkDrained();
@@ -79,6 +97,7 @@ var WorkerImpl = class {
79
97
  async stop() {
80
98
  this.running = false;
81
99
  if (this.timer) clearTimeout(this.timer);
100
+ if (this.reaper) clearInterval(this.reaper);
82
101
  if (this.inFlight === 0) return;
83
102
  await new Promise((resolve) => {
84
103
  this.drainResolve = resolve;
@@ -98,7 +117,7 @@ var Queue = class extends EventEmitter {
98
117
  this.maxAttempts = opts.maxAttempts ?? 1;
99
118
  this.backoff = opts.backoff ?? defaultBackoff;
100
119
  this.removeOnComplete = opts.removeOnComplete ?? false;
101
- this.workerId = opts.workerId ?? `w-${process.pid}`;
120
+ this.workerId = opts.workerId ?? `w-${typeof process !== "undefined" && process.pid ? process.pid : Math.floor(Math.random() * 1e6)}`;
102
121
  if (!ensured.has(db)) {
103
122
  this.driver.exec(
104
123
  `CREATE TABLE IF NOT EXISTS _jobs (
@@ -187,6 +206,10 @@ var Queue = class extends EventEmitter {
187
206
  WHERE status='active' AND updated_at < ?`
188
207
  ).run(now(), now() - olderThanMs).changes;
189
208
  }
209
+ /** @internal Extend a running job's visibility timeout (worker heartbeat). */
210
+ heartbeatInternal(id) {
211
+ this.driver.prepare(`UPDATE _jobs SET updated_at=? WHERE id=? AND status='active'`).run(now(), id);
212
+ }
190
213
  /** Stop all workers and wait for in-flight jobs to finish. */
191
214
  async close() {
192
215
  await Promise.all(this.workers.map((w) => w.stop()));
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/index.ts"],"names":[],"mappings":";;;AA6EA,IAAM,OAAA,uBAAc,OAAA,EAAgB;AACpC,IAAM,GAAA,GAAM,MAAM,IAAA,CAAK,GAAA,EAAI;AAC3B,IAAM,cAAA,GAAiB,CAAC,OAAA,KACtB,IAAA,CAAK,IAAI,GAAA,EAAQ,GAAA,GAAO,CAAA,KAAM,OAAA,GAAU,CAAA,CAAE,CAAA;AAE5C,SAAS,YAAY,GAAA,EAAe;AAClC,EAAA,OAAO;AAAA,IACL,IAAI,GAAA,CAAI,EAAA;AAAA,IACR,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,KAAA,EAAO,IAAI,MAAA,IAAU,MAAA;AAAA,IACrB,QAAQ,GAAA,CAAI,MAAA;AAAA,IACZ,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAA,EAAS,IAAA,CAAK,KAAA,CAAM,GAAA,CAAI,OAAO,CAAA;AAAA,IAC/B,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,aAAa,GAAA,CAAI,YAAA;AAAA,IACjB,OAAO,GAAA,CAAI,MAAA;AAAA,IACX,MAAA,EAAQ,IAAI,MAAA,IAAU,IAAA,GAAO,KAAK,KAAA,CAAM,GAAA,CAAI,MAAM,CAAA,GAAI,MAAA;AAAA,IACtD,KAAA,EAAO,IAAI,KAAA,IAAS,MAAA;AAAA,IACpB,WAAW,GAAA,CAAI,UAAA;AAAA,IACf,WAAW,GAAA,CAAI;AAAA,GACjB;AACF;AAEA,IAAM,aAAN,MAAmC;AAAA,EAQjC,WAAA,CACmB,CAAA,EACR,IAAA,EACQ,OAAA,EACjB,IAAA,EACA;AAJiB,IAAA,IAAA,CAAA,CAAA,GAAA,CAAA;AACR,IAAA,IAAA,CAAA,IAAA,GAAA,IAAA;AACQ,IAAA,IAAA,CAAA,OAAA,GAAA,OAAA;AAGjB,IAAA,IAAA,CAAK,cAAc,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,IAAA,CAAK,YAAA,GAAe,KAAK,YAAA,IAAgB,GAAA;AACzC,IAAA,IAAA,CAAK,IAAA,EAAK;AAAA,EACZ;AAAA,EARmB,CAAA;AAAA,EACR,IAAA;AAAA,EACQ,OAAA;AAAA,EAVX,OAAA,GAAU,IAAA;AAAA,EACV,QAAA,GAAW,CAAA;AAAA,EACX,KAAA;AAAA,EACA,YAAA;AAAA,EACS,WAAA;AAAA,EACA,YAAA;AAAA;AAAA,EAcjB,IAAA,GAAa;AACX,IAAA,IAAI,CAAC,KAAK,OAAA,EAAS;AACnB,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,KAAK,KAAK,CAAA;AACvB,MAAA,IAAA,CAAK,KAAA,GAAQ,MAAA;AAAA,IACf;AACA,IAAA,OAAO,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACvD,MAAA,MAAM,GAAA,GAAM,IAAA,CAAK,CAAA,CAAE,aAAA,CAAc,KAAK,IAAI,CAAA;AAC1C,MAAA,IAAI,CAAC,GAAA,EAAK;AACV,MAAA,IAAA,CAAK,QAAA,EAAA;AACL,MAAA,OAAA,CAAQ,OAAA,GACL,IAAA,CAAK,MAAM,KAAK,OAAA,CAAQ,GAAG,CAAC,CAAA,CAC5B,IAAA;AAAA,QACC,CAAC,MAAA,KAAW;AACV,UAAA,IAAA,CAAK,CAAA,CAAE,gBAAA,CAAiB,GAAA,EAAK,MAAM,CAAA;AACnC,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,WAAA,EAAa,GAAA,EAAK,MAAM,CAAA;AAAA,QACtC,CAAA;AAAA,QACA,CAAC,GAAA,KAAQ;AACP,UAAA,IAAA,CAAK,CAAA,CAAE,YAAA,CAAa,GAAA,EAAK,GAAG,CAAA;AAC5B,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,QAAA,EAAU,GAAA,EAAK,GAAG,CAAA;AAAA,QAChC;AAAA,OACF,CACC,QAAQ,MAAM;AACb,QAAA,IAAA,CAAK,QAAA,EAAA;AACL,QAAA,IAAI,IAAA,CAAK,OAAA,EAAS,IAAA,CAAK,IAAA,EAAK;AAAA,kBAClB,YAAA,EAAa;AAAA,MACzB,CAAC,CAAA;AAAA,IACL;AACA,IAAA,IAAI,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACpD,MAAA,IAAA,CAAK,QAAQ,UAAA,CAAW,MAAM,KAAK,IAAA,EAAK,EAAG,KAAK,YAAY,CAAA;AAC5D,MAAA,IAAA,CAAK,MAAM,KAAA,IAAQ;AAAA,IACrB;AAAA,EACF;AAAA,EAEQ,YAAA,GAAqB;AAC3B,IAAA,IAAI,CAAC,IAAA,CAAK,OAAA,IAAW,KAAK,QAAA,KAAa,CAAA,IAAK,KAAK,YAAA,EAAc;AAC7D,MAAA,IAAA,CAAK,YAAA,EAAa;AAClB,MAAA,IAAA,CAAK,YAAA,GAAe,MAAA;AAAA,IACtB;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAsB;AAC1B,IAAA,IAAA,CAAK,OAAA,GAAU,KAAA;AACf,IAAA,IAAI,IAAA,CAAK,KAAA,EAAO,YAAA,CAAa,IAAA,CAAK,KAAK,CAAA;AACvC,IAAA,IAAI,IAAA,CAAK,aAAa,CAAA,EAAG;AACzB,IAAA,MAAM,IAAI,OAAA,CAAc,CAAC,OAAA,KAAY;AACnC,MAAA,IAAA,CAAK,YAAA,GAAe,OAAA;AAAA,IACtB,CAAC,CAAA;AAAA,EACH;AACF,CAAA;AAOO,IAAM,KAAA,GAAN,cAAoB,YAAA,CAAa;AAAA,EACrB,MAAA;AAAA,EACA,WAAA;AAAA,EACA,OAAA;AAAA,EACA,gBAAA;AAAA,EACR,QAAA;AAAA,EACQ,UAAwB,EAAC;AAAA,EAE1C,WAAA,CAAY,EAAA,EAAa,IAAA,GAAqB,EAAC,EAAG;AAChD,IAAA,KAAA,EAAM;AACN,IAAA,IAAA,CAAK,SAAS,EAAA,CAAG,MAAA;AACjB,IAAA,IAAA,CAAK,WAAA,GAAc,KAAK,WAAA,IAAe,CAAA;AACvC,IAAA,IAAA,CAAK,OAAA,GAAU,KAAK,OAAA,IAAW,cAAA;AAC/B,IAAA,IAAA,CAAK,gBAAA,GAAmB,KAAK,gBAAA,IAAoB,KAAA;AACjD,IAAA,IAAA,CAAK,QAAA,GAAW,IAAA,CAAK,QAAA,IAAY,CAAA,EAAA,EAAK,QAAQ,GAAG,CAAA,CAAA;AAEjD,IAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,EAAE,CAAA,EAAG;AACpB,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,SAAA;AAAA,OAOF;AAEA,MAAA,IAAI;AACF,QAAA,IAAA,CAAK,MAAA,CAAO,KAAK,CAAA,wCAAA,CAA0C,CAAA;AAAA,MAC7D,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,iFAAA;AAAA,OACF;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,wDAAA;AAAA,OACF;AACA,MAAA,OAAA,CAAQ,IAAI,EAAE,CAAA;AAAA,IAChB;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,GAAA,CAAa,IAAA,EAAc,OAAA,EAAY,IAAA,GAAmB,EAAC,EAAW;AACpE,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,MAAM,QAAA,GAAW,KAAK,MAAA,CACnB,OAAA;AAAA,QACC,CAAA,+EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,KAAK,CAAA;AACjB,MAAA,IAAI,QAAA,EAAU,OAAO,WAAA,CAAY,QAAQ,CAAA;AAAA,IAC3C;AACA,IAAA,MAAM,QAAQ,IAAA,CAAK,KAAA,KAAU,KAAK,KAAA,GAAQ,CAAA,GAAI,KAAK,KAAA,GAAQ,CAAA,CAAA;AAC3D,IAAA,MAAM,IAAA,GAAO,KAAK,MAAA,CACf,OAAA;AAAA,MACC,CAAA;AAAA,sDAAA;AAAA,KAEF,CACC,GAAA;AAAA,MACC,IAAA;AAAA,MACA,KAAK,QAAA,IAAY,CAAA;AAAA,MACjB,KAAA;AAAA,MACA,IAAA,CAAK,eAAe,IAAA,CAAK,WAAA;AAAA,MACzB,IAAA,CAAK,SAAA,CAAU,OAAA,IAAW,IAAI,CAAA;AAAA,MAC9B,KAAK,KAAA,IAAS,IAAA;AAAA,MACd,CAAA;AAAA,MACA;AAAA,KACF;AACF,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,KAAA,MAAW,CAAA,IAAK,KAAK,OAAA,EAAS,IAAI,EAAE,IAAA,KAAS,IAAA,IAAQ,IAAA,EAAK;AAC1D,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA,EAGA,OAAA,CACE,IAAA,EACA,OAAA,EACA,IAAA,GAAuB,EAAC,EAChB;AACR,IAAA,MAAM,IAAI,IAAI,UAAA,CAAW,IAAA,EAAM,IAAA,EAAM,SAAoB,IAAI,CAAA;AAC7D,IAAA,IAAA,CAAK,OAAA,CAAQ,KAAK,CAAC,CAAA;AACnB,IAAA,OAAO,CAAA;AAAA,EACT;AAAA,EAEA,OAAgB,EAAA,EAAgC;AAC9C,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CACd,QAAQ,CAAA,gCAAA,CAAkC,CAAA,CAC1C,IAAI,EAAE,CAAA;AACT,IAAA,OAAO,GAAA,GAAO,WAAA,CAAY,GAAG,CAAA,GAAe,MAAA;AAAA,EAC9C;AAAA;AAAA,EAGA,OAAO,IAAA,EAA0C;AAC/C,IAAA,MAAM,IAAA,GACJ,IAAA,GACI,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,MACC,CAAA,uEAAA;AAAA,KACF,CACC,IAAI,IAAI,CAAA,GACX,KAAK,MAAA,CACF,OAAA,CAAQ,CAAA,uDAAA,CAAyD,CAAA,CACjE,GAAA,EAAI;AAEb,IAAA,MAAM,GAAA,GAAiC;AAAA,MACrC,OAAA,EAAS,CAAA;AAAA,MACT,MAAA,EAAQ,CAAA;AAAA,MACR,IAAA,EAAM,CAAA;AAAA,MACN,MAAA,EAAQ;AAAA,KACV;AACA,IAAA,KAAA,MAAW,KAAK,IAAA,EAAM,GAAA,CAAI,CAAA,CAAE,MAAM,IAAI,CAAA,CAAE,CAAA;AACxC,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,OAAA,CAAQ,cAAc,GAAA,EAAgB;AACpC,IAAA,OAAO,KAAK,MAAA,CACT,OAAA;AAAA,MACC,CAAA;AAAA,iDAAA;AAAA,MAGD,GAAA,CAAI,GAAA,IAAO,GAAA,EAAI,GAAI,WAAW,CAAA,CAAE,OAAA;AAAA,EACrC;AAAA;AAAA,EAGA,MAAM,KAAA,GAAuB;AAC3B,IAAA,MAAM,OAAA,CAAQ,GAAA,CAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,CAAC,CAAA,KAAM,CAAA,CAAE,IAAA,EAAM,CAAC,CAAA;AAAA,EACrD;AAAA;AAAA,EAGA,cAAc,IAAA,EAA0B;AACtC,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,MAAM,GAAA,GAAM,KAAK,MAAA,CACd,OAAA;AAAA,MACC,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,oBAAA;AAAA,MAQD,GAAA,CAAI,IAAA,CAAK,QAAA,EAAU,CAAA,EAAG,MAAM,CAAC,CAAA;AAChC,IAAA,OAAO,GAAA,GAAM,WAAA,CAAY,GAAG,CAAA,GAAI,IAAA;AAAA,EAClC;AAAA;AAAA,EAGA,gBAAA,CAAiB,KAAU,MAAA,EAAuB;AAChD,IAAA,IAAI,KAAK,gBAAA,EAAkB;AACzB,MAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,CAAA,8BAAA,CAAgC,CAAA,CAAE,GAAA,CAAI,IAAI,EAAE,CAAA;AAAA,IAClE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,SAAA,CAAU,MAAA,IAAU,IAAI,CAAA,EAAG,GAAA,EAAI,EAAG,GAAA,CAAI,EAAE,CAAA;AAAA,IACtD;AAAA,EACF;AAAA;AAAA,EAGA,YAAA,CAAa,KAAU,GAAA,EAAoB;AAEzC,IAAA,MAAM,UAAU,GAAA,YAAe,KAAA,GAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAC/D,IAAA,IAAI,GAAA,CAAI,QAAA,GAAW,GAAA,CAAI,WAAA,EAAa;AAClC,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6FAAA;AAAA,OACF,CACC,GAAA,CAAI,GAAA,EAAI,GAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,QAAQ,CAAA,EAAG,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IACnE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,kEAAA;AAAA,QAED,GAAA,CAAI,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IAC/B;AAAA,EACF;AACF;AAGO,SAAS,WAAA,CAAY,IAAa,IAAA,EAA4B;AACnE,EAAA,OAAO,IAAI,KAAA,CAAM,EAAA,EAAI,IAAI,CAAA;AAC3B","file":"index.js","sourcesContent":["import { EventEmitter } from \"node:events\";\nimport type { Monlite } from \"@monlite/core\";\n\nexport type JobStatus = \"pending\" | \"active\" | \"done\" | \"failed\";\n\nexport interface Job<T = any> {\n id: number;\n queue: string;\n /** Dedupe key, if the job was added with one. */\n jobId?: string;\n status: JobStatus;\n priority: number;\n payload: T;\n /** Number of attempts already made (0 until the first run). */\n attempts: number;\n maxAttempts: number;\n runAt: number;\n result?: any;\n error?: string;\n createdAt: number;\n updatedAt: number;\n}\n\nexport interface AddOptions {\n /** Dedupe key — skip if a job with this id is already pending/active. */\n jobId?: string;\n /** Delay before the job becomes runnable (ms). */\n delay?: number;\n /** Explicit epoch-ms run time (overrides `delay`). */\n runAt?: number;\n /** Higher runs first. Default 0. */\n priority?: number;\n /** Total attempts before dead-lettering. Default: the queue's `maxAttempts`. */\n maxAttempts?: number;\n}\n\nexport interface ProcessOptions {\n /** Jobs run concurrently per worker. Default 1. */\n concurrency?: number;\n /** How often to poll for due jobs when idle (ms). Default 500. */\n pollInterval?: number;\n}\n\nexport interface QueueOptions {\n /** Default attempts before dead-lettering. Default 1 (no retry). */\n maxAttempts?: number;\n /** Backoff before retry N (ms). Default: exponential, capped at 30s. */\n backoff?: (attempt: number) => number;\n /** Delete jobs once completed instead of keeping them as `done`. Default false. */\n removeOnComplete?: boolean;\n /** Identifies this worker process in the `locked_by` column. */\n workerId?: string;\n}\n\nexport type Handler<T = any, R = any> = (job: Job<T>) => Promise<R> | R;\n\nexport interface Worker {\n /** Stop claiming new jobs and wait for in-flight ones to finish. */\n stop(): Promise<void>;\n}\n\ninterface Row {\n id: number;\n queue: string;\n status: JobStatus;\n priority: number;\n run_at: number;\n attempts: number;\n max_attempts: number;\n payload: string;\n result: string | null;\n error: string | null;\n job_id: string | null;\n created_at: number;\n updated_at: number;\n}\n\nconst ensured = new WeakSet<object>();\nconst now = () => Date.now();\nconst defaultBackoff = (attempt: number) =>\n Math.min(30_000, 1000 * 2 ** (attempt - 1));\n\nfunction deserialize(row: Row): Job {\n return {\n id: row.id,\n queue: row.queue,\n jobId: row.job_id ?? undefined,\n status: row.status,\n priority: row.priority,\n payload: JSON.parse(row.payload),\n attempts: row.attempts,\n maxAttempts: row.max_attempts,\n runAt: row.run_at,\n result: row.result != null ? JSON.parse(row.result) : undefined,\n error: row.error ?? undefined,\n createdAt: row.created_at,\n updatedAt: row.updated_at,\n };\n}\n\nclass WorkerImpl implements Worker {\n private running = true;\n private inFlight = 0;\n private timer: ReturnType<typeof setTimeout> | undefined;\n private drainResolve: (() => void) | undefined;\n private readonly concurrency: number;\n private readonly pollInterval: number;\n\n constructor(\n private readonly q: Queue,\n readonly name: string,\n private readonly handler: Handler,\n opts: ProcessOptions,\n ) {\n this.concurrency = Math.max(1, opts.concurrency ?? 1);\n this.pollInterval = opts.pollInterval ?? 500;\n this.kick();\n }\n\n /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */\n kick(): void {\n if (!this.running) return;\n if (this.timer) {\n clearTimeout(this.timer);\n this.timer = undefined;\n }\n while (this.running && this.inFlight < this.concurrency) {\n const job = this.q.claimInternal(this.name);\n if (!job) break;\n this.inFlight++;\n Promise.resolve()\n .then(() => this.handler(job))\n .then(\n (result) => {\n this.q.completeInternal(job, result);\n this.q.emit(\"completed\", job, result);\n },\n (err) => {\n this.q.failInternal(job, err);\n this.q.emit(\"failed\", job, err);\n },\n )\n .finally(() => {\n this.inFlight--;\n if (this.running) this.kick();\n else this.checkDrained();\n });\n }\n if (this.running && this.inFlight < this.concurrency) {\n this.timer = setTimeout(() => this.kick(), this.pollInterval);\n this.timer.unref?.();\n }\n }\n\n private checkDrained(): void {\n if (!this.running && this.inFlight === 0 && this.drainResolve) {\n this.drainResolve();\n this.drainResolve = undefined;\n }\n }\n\n async stop(): Promise<void> {\n this.running = false;\n if (this.timer) clearTimeout(this.timer);\n if (this.inFlight === 0) return;\n await new Promise<void>((resolve) => {\n this.drainResolve = resolve;\n });\n }\n}\n\n/**\n * A durable, multi-process-safe job queue backed by SQLite. Producers `add`\n * jobs; workers `process` them with retries, backoff, delays, and concurrency.\n * Emits `\"completed\"` (job, result) and `\"failed\"` (job, error).\n */\nexport class Queue extends EventEmitter {\n private readonly driver: Monlite[\"driver\"];\n private readonly maxAttempts: number;\n private readonly backoff: (attempt: number) => number;\n private readonly removeOnComplete: boolean;\n readonly workerId: string;\n private readonly workers: WorkerImpl[] = [];\n\n constructor(db: Monlite, opts: QueueOptions = {}) {\n super();\n this.driver = db.driver;\n this.maxAttempts = opts.maxAttempts ?? 1;\n this.backoff = opts.backoff ?? defaultBackoff;\n this.removeOnComplete = opts.removeOnComplete ?? false;\n this.workerId = opts.workerId ?? `w-${process.pid}`;\n\n if (!ensured.has(db)) {\n this.driver.exec(\n `CREATE TABLE IF NOT EXISTS _jobs (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n queue TEXT NOT NULL, status TEXT NOT NULL, priority INTEGER NOT NULL,\n run_at INTEGER NOT NULL, attempts INTEGER NOT NULL, max_attempts INTEGER NOT NULL,\n payload TEXT NOT NULL, result TEXT, error TEXT, locked_by TEXT, job_id TEXT,\n created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL\n )`,\n );\n // Add job_id to pre-existing tables (idempotent).\n try {\n this.driver.exec(`ALTER TABLE _jobs ADD COLUMN job_id TEXT`);\n } catch {\n /* column already exists */\n }\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_claim ON _jobs (queue, status, priority, run_at)`,\n );\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_jobid ON _jobs (job_id)`,\n );\n ensured.add(db);\n }\n }\n\n /**\n * Enqueue a job. Pass `opts.jobId` to **dedupe**: if a job with that id is\n * already pending or active, the existing job is returned instead of adding a\n * duplicate (idempotent enqueue — e.g. for resume/replay).\n */\n add<T = any>(name: string, payload: T, opts: AddOptions = {}): Job<T> {\n const t = now();\n if (opts.jobId) {\n const existing = this.driver\n .prepare(\n `SELECT * FROM _jobs WHERE job_id = ? AND status IN ('pending','active') LIMIT 1`,\n )\n .get(opts.jobId) as Row | undefined;\n if (existing) return deserialize(existing) as Job<T>;\n }\n const runAt = opts.runAt ?? (opts.delay ? t + opts.delay : t);\n const info = this.driver\n .prepare(\n `INSERT INTO _jobs (queue, status, priority, run_at, attempts, max_attempts, payload, job_id, created_at, updated_at)\n VALUES (?, 'pending', ?, ?, 0, ?, ?, ?, ?, ?)`,\n )\n .run(\n name,\n opts.priority ?? 0,\n runAt,\n opts.maxAttempts ?? this.maxAttempts,\n JSON.stringify(payload ?? null),\n opts.jobId ?? null,\n t,\n t,\n );\n const job = this.getJob(Number(info.lastInsertRowid))!;\n for (const w of this.workers) if (w.name === name) w.kick();\n return job as Job<T>;\n }\n\n /** Register a worker for a queue. Returns a handle with `stop()`. */\n process<T = any, R = any>(\n name: string,\n handler: Handler<T, R>,\n opts: ProcessOptions = {},\n ): Worker {\n const w = new WorkerImpl(this, name, handler as Handler, opts);\n this.workers.push(w);\n return w;\n }\n\n getJob<T = any>(id: number): Job<T> | undefined {\n const row = this.driver\n .prepare(`SELECT * FROM _jobs WHERE id = ?`)\n .get(id) as Row | undefined;\n return row ? (deserialize(row) as Job<T>) : undefined;\n }\n\n /** Count jobs by status (optionally for one queue). */\n counts(name?: string): Record<JobStatus, number> {\n const rows = (\n name\n ? this.driver\n .prepare(\n `SELECT status, COUNT(*) AS n FROM _jobs WHERE queue = ? GROUP BY status`,\n )\n .all(name)\n : this.driver\n .prepare(`SELECT status, COUNT(*) AS n FROM _jobs GROUP BY status`)\n .all()\n ) as Array<{ status: JobStatus; n: number }>;\n const out: Record<JobStatus, number> = {\n pending: 0,\n active: 0,\n done: 0,\n failed: 0,\n };\n for (const r of rows) out[r.status] = r.n;\n return out;\n }\n\n /**\n * Reset jobs stuck in `active` (e.g. from a crashed worker) back to `pending`\n * if they haven't been touched in `olderThanMs`. Returns the count recovered.\n */\n recover(olderThanMs = 60_000): number {\n return this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', locked_by=NULL, updated_at=?\n WHERE status='active' AND updated_at < ?`,\n )\n .run(now(), now() - olderThanMs).changes;\n }\n\n /** Stop all workers and wait for in-flight jobs to finish. */\n async close(): Promise<void> {\n await Promise.all(this.workers.map((w) => w.stop()));\n }\n\n /** @internal Atomically claim the next due job, counting the attempt. */\n claimInternal(name: string): Job | null {\n const t = now();\n const row = this.driver\n .prepare(\n `UPDATE _jobs SET status='active', attempts=attempts+1, locked_by=?, updated_at=?\n WHERE id = (\n SELECT id FROM _jobs\n WHERE queue=? AND status='pending' AND run_at<=?\n ORDER BY priority DESC, id ASC LIMIT 1\n )\n RETURNING *`,\n )\n .get(this.workerId, t, name, t) as Row | undefined;\n return row ? deserialize(row) : null;\n }\n\n /** @internal */\n completeInternal(job: Job, result: unknown): void {\n if (this.removeOnComplete) {\n this.driver.prepare(`DELETE FROM _jobs WHERE id = ?`).run(job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='done', result=?, error=NULL, updated_at=? WHERE id=?`,\n )\n .run(JSON.stringify(result ?? null), now(), job.id);\n }\n }\n\n /** @internal */\n failInternal(job: Job, err: unknown): void {\n // `job.attempts` was already incremented at claim time.\n const message = err instanceof Error ? err.message : String(err);\n if (job.attempts < job.maxAttempts) {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', run_at=?, error=?, locked_by=NULL, updated_at=? WHERE id=?`,\n )\n .run(now() + this.backoff(job.attempts), message, now(), job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='failed', error=?, updated_at=? WHERE id=?`,\n )\n .run(message, now(), job.id);\n }\n }\n}\n\n/** Create a job queue over a monlite database. */\nexport function createQueue(db: Monlite, opts?: QueueOptions): Queue {\n return new Queue(db, opts);\n}\n"]}
1
+ {"version":3,"sources":["../src/index.ts"],"names":[],"mappings":";;;AAoFA,IAAM,OAAA,uBAAc,OAAA,EAAgB;AACpC,IAAM,GAAA,GAAM,MAAM,IAAA,CAAK,GAAA,EAAI;AAC3B,IAAM,cAAA,GAAiB,CAAC,OAAA,KACtB,IAAA,CAAK,IAAI,GAAA,EAAQ,GAAA,GAAO,CAAA,KAAM,OAAA,GAAU,CAAA,CAAE,CAAA;AAE5C,SAAS,YAAY,GAAA,EAAe;AAClC,EAAA,OAAO;AAAA,IACL,IAAI,GAAA,CAAI,EAAA;AAAA,IACR,OAAO,GAAA,CAAI,KAAA;AAAA,IACX,KAAA,EAAO,IAAI,MAAA,IAAU,MAAA;AAAA,IACrB,QAAQ,GAAA,CAAI,MAAA;AAAA,IACZ,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,OAAA,EAAS,IAAA,CAAK,KAAA,CAAM,GAAA,CAAI,OAAO,CAAA;AAAA,IAC/B,UAAU,GAAA,CAAI,QAAA;AAAA,IACd,aAAa,GAAA,CAAI,YAAA;AAAA,IACjB,OAAO,GAAA,CAAI,MAAA;AAAA,IACX,MAAA,EAAQ,IAAI,MAAA,IAAU,IAAA,GAAO,KAAK,KAAA,CAAM,GAAA,CAAI,MAAM,CAAA,GAAI,MAAA;AAAA,IACtD,KAAA,EAAO,IAAI,KAAA,IAAS,MAAA;AAAA,IACpB,WAAW,GAAA,CAAI,UAAA;AAAA,IACf,WAAW,GAAA,CAAI;AAAA,GACjB;AACF;AAEA,IAAM,aAAN,MAAmC;AAAA,EAUjC,WAAA,CACmB,CAAA,EACR,IAAA,EACQ,OAAA,EACjB,IAAA,EACA;AAJiB,IAAA,IAAA,CAAA,CAAA,GAAA,CAAA;AACR,IAAA,IAAA,CAAA,IAAA,GAAA,IAAA;AACQ,IAAA,IAAA,CAAA,OAAA,GAAA,OAAA;AAGjB,IAAA,IAAA,CAAK,cAAc,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,IAAA,CAAK,YAAA,GAAe,KAAK,YAAA,IAAgB,GAAA;AACzC,IAAA,IAAA,CAAK,oBAAoB,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,qBAAqB,CAAC,CAAA;AAChE,IAAA,IAAI,IAAA,CAAK,oBAAoB,CAAA,EAAG;AAC9B,MAAA,IAAA,CAAK,MAAA,GAAS,YAAY,MAAM;AAC9B,QAAA,IAAI,KAAK,OAAA,EAAS,IAAA,CAAK,CAAA,CAAE,OAAA,CAAQ,KAAK,iBAAiB,CAAA;AAAA,MACzD,CAAA,EAAG,IAAA,CAAK,GAAA,CAAI,GAAA,EAAM,IAAA,CAAK,MAAM,IAAA,CAAK,iBAAA,GAAoB,CAAC,CAAC,CAAC,CAAA;AACzD,MAAA,IAAA,CAAK,OAAO,KAAA,IAAQ;AAAA,IACtB;AACA,IAAA,IAAA,CAAK,IAAA,EAAK;AAAA,EACZ;AAAA,EAfmB,CAAA;AAAA,EACR,IAAA;AAAA,EACQ,OAAA;AAAA,EAZX,OAAA,GAAU,IAAA;AAAA,EACV,QAAA,GAAW,CAAA;AAAA,EACX,KAAA;AAAA,EACA,YAAA;AAAA,EACS,WAAA;AAAA,EACA,YAAA;AAAA,EACA,iBAAA;AAAA,EACT,MAAA;AAAA;AAAA,EAqBR,IAAA,GAAa;AACX,IAAA,IAAI,CAAC,KAAK,OAAA,EAAS;AACnB,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,KAAK,KAAK,CAAA;AACvB,MAAA,IAAA,CAAK,KAAA,GAAQ,MAAA;AAAA,IACf;AACA,IAAA,OAAO,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACvD,MAAA,MAAM,GAAA,GAAM,IAAA,CAAK,CAAA,CAAE,aAAA,CAAc,KAAK,IAAI,CAAA;AAC1C,MAAA,IAAI,CAAC,GAAA,EAAK;AACV,MAAA,IAAA,CAAK,QAAA,EAAA;AAGL,MAAA,IAAI,EAAA;AACJ,MAAA,IAAI,IAAA,CAAK,oBAAoB,CAAA,EAAG;AAC9B,QAAA,EAAA,GAAK,WAAA;AAAA,UACH,MAAM,IAAA,CAAK,CAAA,CAAE,iBAAA,CAAkB,IAAI,EAAE,CAAA;AAAA,UACrC,IAAA,CAAK,IAAI,GAAA,EAAM,IAAA,CAAK,MAAM,IAAA,CAAK,iBAAA,GAAoB,CAAC,CAAC;AAAA,SACvD;AACA,QAAA,EAAA,CAAG,KAAA,IAAQ;AAAA,MACb;AACA,MAAA,OAAA,CAAQ,OAAA,GACL,IAAA,CAAK,MAAM,KAAK,OAAA,CAAQ,GAAG,CAAC,CAAA,CAC5B,IAAA;AAAA,QACC,CAAC,MAAA,KAAW;AACV,UAAA,IAAA,CAAK,CAAA,CAAE,gBAAA,CAAiB,GAAA,EAAK,MAAM,CAAA;AACnC,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,WAAA,EAAa,GAAA,EAAK,MAAM,CAAA;AAAA,QACtC,CAAA;AAAA,QACA,CAAC,GAAA,KAAQ;AACP,UAAA,IAAA,CAAK,CAAA,CAAE,YAAA,CAAa,GAAA,EAAK,GAAG,CAAA;AAC5B,UAAA,IAAA,CAAK,CAAA,CAAE,IAAA,CAAK,QAAA,EAAU,GAAA,EAAK,GAAG,CAAA;AAAA,QAChC;AAAA,OACF,CACC,QAAQ,MAAM;AACb,QAAA,IAAI,EAAA,gBAAkB,EAAE,CAAA;AACxB,QAAA,IAAA,CAAK,QAAA,EAAA;AACL,QAAA,IAAI,IAAA,CAAK,OAAA,EAAS,IAAA,CAAK,IAAA,EAAK;AAAA,kBAClB,YAAA,EAAa;AAAA,MACzB,CAAC,CAAA;AAAA,IACL;AACA,IAAA,IAAI,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,QAAA,GAAW,KAAK,WAAA,EAAa;AACpD,MAAA,IAAA,CAAK,QAAQ,UAAA,CAAW,MAAM,KAAK,IAAA,EAAK,EAAG,KAAK,YAAY,CAAA;AAC5D,MAAA,IAAA,CAAK,MAAM,KAAA,IAAQ;AAAA,IACrB;AAAA,EACF;AAAA,EAEQ,YAAA,GAAqB;AAC3B,IAAA,IAAI,CAAC,IAAA,CAAK,OAAA,IAAW,KAAK,QAAA,KAAa,CAAA,IAAK,KAAK,YAAA,EAAc;AAC7D,MAAA,IAAA,CAAK,YAAA,EAAa;AAClB,MAAA,IAAA,CAAK,YAAA,GAAe,MAAA;AAAA,IACtB;AAAA,EACF;AAAA,EAEA,MAAM,IAAA,GAAsB;AAC1B,IAAA,IAAA,CAAK,OAAA,GAAU,KAAA;AACf,IAAA,IAAI,IAAA,CAAK,KAAA,EAAO,YAAA,CAAa,IAAA,CAAK,KAAK,CAAA;AACvC,IAAA,IAAI,IAAA,CAAK,MAAA,EAAQ,aAAA,CAAc,IAAA,CAAK,MAAM,CAAA;AAC1C,IAAA,IAAI,IAAA,CAAK,aAAa,CAAA,EAAG;AACzB,IAAA,MAAM,IAAI,OAAA,CAAc,CAAC,OAAA,KAAY;AACnC,MAAA,IAAA,CAAK,YAAA,GAAe,OAAA;AAAA,IACtB,CAAC,CAAA;AAAA,EACH;AACF,CAAA;AAOO,IAAM,KAAA,GAAN,cAAoB,YAAA,CAAa;AAAA,EACrB,MAAA;AAAA,EACA,WAAA;AAAA,EACA,OAAA;AAAA,EACA,gBAAA;AAAA,EACR,QAAA;AAAA,EACQ,UAAwB,EAAC;AAAA,EAE1C,WAAA,CAAY,EAAA,EAAa,IAAA,GAAqB,EAAC,EAAG;AAChD,IAAA,KAAA,EAAM;AACN,IAAA,IAAA,CAAK,SAAS,EAAA,CAAG,MAAA;AACjB,IAAA,IAAA,CAAK,WAAA,GAAc,KAAK,WAAA,IAAe,CAAA;AACvC,IAAA,IAAA,CAAK,OAAA,GAAU,KAAK,OAAA,IAAW,cAAA;AAC/B,IAAA,IAAA,CAAK,gBAAA,GAAmB,KAAK,gBAAA,IAAoB,KAAA;AAEjD,IAAA,IAAA,CAAK,WACH,IAAA,CAAK,QAAA,IACL,CAAA,EAAA,EAAK,OAAO,YAAY,WAAA,IAAe,OAAA,CAAQ,GAAA,GAAM,OAAA,CAAQ,MAAM,IAAA,CAAK,KAAA,CAAM,KAAK,MAAA,EAAO,GAAI,GAAG,CAAC,CAAA,CAAA;AAEpG,IAAA,IAAI,CAAC,OAAA,CAAQ,GAAA,CAAI,EAAE,CAAA,EAAG;AACpB,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,SAAA;AAAA,OAOF;AAEA,MAAA,IAAI;AACF,QAAA,IAAA,CAAK,MAAA,CAAO,KAAK,CAAA,wCAAA,CAA0C,CAAA;AAAA,MAC7D,CAAA,CAAA,MAAQ;AAAA,MAER;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,iFAAA;AAAA,OACF;AACA,MAAA,IAAA,CAAK,MAAA,CAAO,IAAA;AAAA,QACV,CAAA,wDAAA;AAAA,OACF;AACA,MAAA,OAAA,CAAQ,IAAI,EAAE,CAAA;AAAA,IAChB;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,GAAA,CAAa,IAAA,EAAc,OAAA,EAAY,IAAA,GAAmB,EAAC,EAAW;AACpE,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,MAAM,QAAA,GAAW,KAAK,MAAA,CACnB,OAAA;AAAA,QACC,CAAA,+EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,KAAK,CAAA;AACjB,MAAA,IAAI,QAAA,EAAU,OAAO,WAAA,CAAY,QAAQ,CAAA;AAAA,IAC3C;AACA,IAAA,MAAM,QAAQ,IAAA,CAAK,KAAA,KAAU,KAAK,KAAA,GAAQ,CAAA,GAAI,KAAK,KAAA,GAAQ,CAAA,CAAA;AAC3D,IAAA,MAAM,IAAA,GAAO,KAAK,MAAA,CACf,OAAA;AAAA,MACC,CAAA;AAAA,sDAAA;AAAA,KAEF,CACC,GAAA;AAAA,MACC,IAAA;AAAA,MACA,KAAK,QAAA,IAAY,CAAA;AAAA,MACjB,KAAA;AAAA,MACA,IAAA,CAAK,eAAe,IAAA,CAAK,WAAA;AAAA,MACzB,IAAA,CAAK,SAAA,CAAU,OAAA,IAAW,IAAI,CAAA;AAAA,MAC9B,KAAK,KAAA,IAAS,IAAA;AAAA,MACd,CAAA;AAAA,MACA;AAAA,KACF;AACF,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CAAO,MAAA,CAAO,IAAA,CAAK,eAAe,CAAC,CAAA;AACpD,IAAA,KAAA,MAAW,CAAA,IAAK,KAAK,OAAA,EAAS,IAAI,EAAE,IAAA,KAAS,IAAA,IAAQ,IAAA,EAAK;AAC1D,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA,EAGA,OAAA,CACE,IAAA,EACA,OAAA,EACA,IAAA,GAAuB,EAAC,EAChB;AACR,IAAA,MAAM,IAAI,IAAI,UAAA,CAAW,IAAA,EAAM,IAAA,EAAM,SAAoB,IAAI,CAAA;AAC7D,IAAA,IAAA,CAAK,OAAA,CAAQ,KAAK,CAAC,CAAA;AACnB,IAAA,OAAO,CAAA;AAAA,EACT;AAAA,EAEA,OAAgB,EAAA,EAAgC;AAC9C,IAAA,MAAM,MAAM,IAAA,CAAK,MAAA,CACd,QAAQ,CAAA,gCAAA,CAAkC,CAAA,CAC1C,IAAI,EAAE,CAAA;AACT,IAAA,OAAO,GAAA,GAAO,WAAA,CAAY,GAAG,CAAA,GAAe,MAAA;AAAA,EAC9C;AAAA;AAAA,EAGA,OAAO,IAAA,EAA0C;AAC/C,IAAA,MAAM,IAAA,GACJ,IAAA,GACI,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,MACC,CAAA,uEAAA;AAAA,KACF,CACC,IAAI,IAAI,CAAA,GACX,KAAK,MAAA,CACF,OAAA,CAAQ,CAAA,uDAAA,CAAyD,CAAA,CACjE,GAAA,EAAI;AAEb,IAAA,MAAM,GAAA,GAAiC;AAAA,MACrC,OAAA,EAAS,CAAA;AAAA,MACT,MAAA,EAAQ,CAAA;AAAA,MACR,IAAA,EAAM,CAAA;AAAA,MACN,MAAA,EAAQ;AAAA,KACV;AACA,IAAA,KAAA,MAAW,KAAK,IAAA,EAAM,GAAA,CAAI,CAAA,CAAE,MAAM,IAAI,CAAA,CAAE,CAAA;AACxC,IAAA,OAAO,GAAA;AAAA,EACT;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,OAAA,CAAQ,cAAc,GAAA,EAAgB;AACpC,IAAA,OAAO,KAAK,MAAA,CACT,OAAA;AAAA,MACC,CAAA;AAAA,iDAAA;AAAA,MAGD,GAAA,CAAI,GAAA,IAAO,GAAA,EAAI,GAAI,WAAW,CAAA,CAAE,OAAA;AAAA,EACrC;AAAA;AAAA,EAGA,kBAAkB,EAAA,EAAkB;AAClC,IAAA,IAAA,CAAK,OACF,OAAA,CAAQ,CAAA,4DAAA,CAA8D,EACtE,GAAA,CAAI,GAAA,IAAO,EAAE,CAAA;AAAA,EAClB;AAAA;AAAA,EAGA,MAAM,KAAA,GAAuB;AAC3B,IAAA,MAAM,OAAA,CAAQ,GAAA,CAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,CAAC,CAAA,KAAM,CAAA,CAAE,IAAA,EAAM,CAAC,CAAA;AAAA,EACrD;AAAA;AAAA,EAGA,cAAc,IAAA,EAA0B;AACtC,IAAA,MAAM,IAAI,GAAA,EAAI;AACd,IAAA,MAAM,GAAA,GAAM,KAAK,MAAA,CACd,OAAA;AAAA,MACC,CAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,oBAAA;AAAA,MAQD,GAAA,CAAI,IAAA,CAAK,QAAA,EAAU,CAAA,EAAG,MAAM,CAAC,CAAA;AAChC,IAAA,OAAO,GAAA,GAAM,WAAA,CAAY,GAAG,CAAA,GAAI,IAAA;AAAA,EAClC;AAAA;AAAA,EAGA,gBAAA,CAAiB,KAAU,MAAA,EAAuB;AAChD,IAAA,IAAI,KAAK,gBAAA,EAAkB;AACzB,MAAA,IAAA,CAAK,OAAO,OAAA,CAAQ,CAAA,8BAAA,CAAgC,CAAA,CAAE,GAAA,CAAI,IAAI,EAAE,CAAA;AAAA,IAClE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6EAAA;AAAA,OACF,CACC,GAAA,CAAI,IAAA,CAAK,SAAA,CAAU,MAAA,IAAU,IAAI,CAAA,EAAG,GAAA,EAAI,EAAG,GAAA,CAAI,EAAE,CAAA;AAAA,IACtD;AAAA,EACF;AAAA;AAAA,EAGA,YAAA,CAAa,KAAU,GAAA,EAAoB;AAEzC,IAAA,MAAM,UAAU,GAAA,YAAe,KAAA,GAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAC/D,IAAA,IAAI,GAAA,CAAI,QAAA,GAAW,GAAA,CAAI,WAAA,EAAa;AAClC,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,6FAAA;AAAA,OACF,CACC,GAAA,CAAI,GAAA,EAAI,GAAI,IAAA,CAAK,OAAA,CAAQ,GAAA,CAAI,QAAQ,CAAA,EAAG,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IACnE,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,MAAA,CACF,OAAA;AAAA,QACC,CAAA,kEAAA;AAAA,QAED,GAAA,CAAI,OAAA,EAAS,GAAA,EAAI,EAAG,IAAI,EAAE,CAAA;AAAA,IAC/B;AAAA,EACF;AACF;AAGO,SAAS,WAAA,CAAY,IAAa,IAAA,EAA4B;AACnE,EAAA,OAAO,IAAI,KAAA,CAAM,EAAA,EAAI,IAAI,CAAA;AAC3B","file":"index.js","sourcesContent":["import { EventEmitter } from \"node:events\";\nimport type { Monlite } from \"@monlite/core\";\n\nexport type JobStatus = \"pending\" | \"active\" | \"done\" | \"failed\";\n\nexport interface Job<T = any> {\n id: number;\n queue: string;\n /** Dedupe key, if the job was added with one. */\n jobId?: string;\n status: JobStatus;\n priority: number;\n payload: T;\n /** Number of attempts already made (0 until the first run). */\n attempts: number;\n maxAttempts: number;\n runAt: number;\n result?: any;\n error?: string;\n createdAt: number;\n updatedAt: number;\n}\n\nexport interface AddOptions {\n /** Dedupe key — skip if a job with this id is already pending/active. */\n jobId?: string;\n /** Delay before the job becomes runnable (ms). */\n delay?: number;\n /** Explicit epoch-ms run time (overrides `delay`). */\n runAt?: number;\n /** Higher runs first. Default 0. */\n priority?: number;\n /** Total attempts before dead-lettering. Default: the queue's `maxAttempts`. */\n maxAttempts?: number;\n}\n\nexport interface ProcessOptions {\n /** Jobs run concurrently per worker. Default 1. */\n concurrency?: number;\n /** How often to poll for due jobs when idle (ms). Default 500. */\n pollInterval?: number;\n /**\n * Visibility timeout (ms). If set, a crashed worker's job is automatically\n * reclaimed: a job that stays `active` without a heartbeat for this long is\n * returned to `pending`. While a handler runs, its job is heartbeated so a\n * legitimately long job isn't reaped. Off by default (jobs are never reaped).\n */\n visibilityTimeout?: number;\n}\n\nexport interface QueueOptions {\n /** Default attempts before dead-lettering. Default 1 (no retry). */\n maxAttempts?: number;\n /** Backoff before retry N (ms). Default: exponential, capped at 30s. */\n backoff?: (attempt: number) => number;\n /** Delete jobs once completed instead of keeping them as `done`. Default false. */\n removeOnComplete?: boolean;\n /** Identifies this worker process in the `locked_by` column. */\n workerId?: string;\n}\n\nexport type Handler<T = any, R = any> = (job: Job<T>) => Promise<R> | R;\n\nexport interface Worker {\n /** Stop claiming new jobs and wait for in-flight ones to finish. */\n stop(): Promise<void>;\n}\n\ninterface Row {\n id: number;\n queue: string;\n status: JobStatus;\n priority: number;\n run_at: number;\n attempts: number;\n max_attempts: number;\n payload: string;\n result: string | null;\n error: string | null;\n job_id: string | null;\n created_at: number;\n updated_at: number;\n}\n\nconst ensured = new WeakSet<object>();\nconst now = () => Date.now();\nconst defaultBackoff = (attempt: number) =>\n Math.min(30_000, 1000 * 2 ** (attempt - 1));\n\nfunction deserialize(row: Row): Job {\n return {\n id: row.id,\n queue: row.queue,\n jobId: row.job_id ?? undefined,\n status: row.status,\n priority: row.priority,\n payload: JSON.parse(row.payload),\n attempts: row.attempts,\n maxAttempts: row.max_attempts,\n runAt: row.run_at,\n result: row.result != null ? JSON.parse(row.result) : undefined,\n error: row.error ?? undefined,\n createdAt: row.created_at,\n updatedAt: row.updated_at,\n };\n}\n\nclass WorkerImpl implements Worker {\n private running = true;\n private inFlight = 0;\n private timer: ReturnType<typeof setTimeout> | undefined;\n private drainResolve: (() => void) | undefined;\n private readonly concurrency: number;\n private readonly pollInterval: number;\n private readonly visibilityTimeout: number;\n private reaper: ReturnType<typeof setInterval> | undefined;\n\n constructor(\n private readonly q: Queue,\n readonly name: string,\n private readonly handler: Handler,\n opts: ProcessOptions,\n ) {\n this.concurrency = Math.max(1, opts.concurrency ?? 1);\n this.pollInterval = opts.pollInterval ?? 500;\n this.visibilityTimeout = Math.max(0, opts.visibilityTimeout ?? 0);\n if (this.visibilityTimeout > 0) {\n this.reaper = setInterval(() => {\n if (this.running) this.q.recover(this.visibilityTimeout);\n }, Math.max(1000, Math.floor(this.visibilityTimeout / 2)));\n this.reaper.unref?.();\n }\n this.kick();\n }\n\n /** Fill spare capacity with claimable jobs; otherwise schedule a poll. */\n kick(): void {\n if (!this.running) return;\n if (this.timer) {\n clearTimeout(this.timer);\n this.timer = undefined;\n }\n while (this.running && this.inFlight < this.concurrency) {\n const job = this.q.claimInternal(this.name);\n if (!job) break;\n this.inFlight++;\n // Heartbeat the job while it runs so the reaper's visibility timeout won't\n // reclaim a legitimately long-running job.\n let hb: ReturnType<typeof setInterval> | undefined;\n if (this.visibilityTimeout > 0) {\n hb = setInterval(\n () => this.q.heartbeatInternal(job.id),\n Math.max(1000, Math.floor(this.visibilityTimeout / 2)),\n );\n hb.unref?.();\n }\n Promise.resolve()\n .then(() => this.handler(job))\n .then(\n (result) => {\n this.q.completeInternal(job, result);\n this.q.emit(\"completed\", job, result);\n },\n (err) => {\n this.q.failInternal(job, err);\n this.q.emit(\"failed\", job, err);\n },\n )\n .finally(() => {\n if (hb) clearInterval(hb);\n this.inFlight--;\n if (this.running) this.kick();\n else this.checkDrained();\n });\n }\n if (this.running && this.inFlight < this.concurrency) {\n this.timer = setTimeout(() => this.kick(), this.pollInterval);\n this.timer.unref?.();\n }\n }\n\n private checkDrained(): void {\n if (!this.running && this.inFlight === 0 && this.drainResolve) {\n this.drainResolve();\n this.drainResolve = undefined;\n }\n }\n\n async stop(): Promise<void> {\n this.running = false;\n if (this.timer) clearTimeout(this.timer);\n if (this.reaper) clearInterval(this.reaper);\n if (this.inFlight === 0) return;\n await new Promise<void>((resolve) => {\n this.drainResolve = resolve;\n });\n }\n}\n\n/**\n * A durable, multi-process-safe job queue backed by SQLite. Producers `add`\n * jobs; workers `process` them with retries, backoff, delays, and concurrency.\n * Emits `\"completed\"` (job, result) and `\"failed\"` (job, error).\n */\nexport class Queue extends EventEmitter {\n private readonly driver: Monlite[\"driver\"];\n private readonly maxAttempts: number;\n private readonly backoff: (attempt: number) => number;\n private readonly removeOnComplete: boolean;\n readonly workerId: string;\n private readonly workers: WorkerImpl[] = [];\n\n constructor(db: Monlite, opts: QueueOptions = {}) {\n super();\n this.driver = db.driver;\n this.maxAttempts = opts.maxAttempts ?? 1;\n this.backoff = opts.backoff ?? defaultBackoff;\n this.removeOnComplete = opts.removeOnComplete ?? false;\n // `process` is absent in the browser — fall back to a random id there.\n this.workerId =\n opts.workerId ??\n `w-${typeof process !== \"undefined\" && process.pid ? process.pid : Math.floor(Math.random() * 1e6)}`;\n\n if (!ensured.has(db)) {\n this.driver.exec(\n `CREATE TABLE IF NOT EXISTS _jobs (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n queue TEXT NOT NULL, status TEXT NOT NULL, priority INTEGER NOT NULL,\n run_at INTEGER NOT NULL, attempts INTEGER NOT NULL, max_attempts INTEGER NOT NULL,\n payload TEXT NOT NULL, result TEXT, error TEXT, locked_by TEXT, job_id TEXT,\n created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL\n )`,\n );\n // Add job_id to pre-existing tables (idempotent).\n try {\n this.driver.exec(`ALTER TABLE _jobs ADD COLUMN job_id TEXT`);\n } catch {\n /* column already exists */\n }\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_claim ON _jobs (queue, status, priority, run_at)`,\n );\n this.driver.exec(\n `CREATE INDEX IF NOT EXISTS _jobs_jobid ON _jobs (job_id)`,\n );\n ensured.add(db);\n }\n }\n\n /**\n * Enqueue a job. Pass `opts.jobId` to **dedupe**: if a job with that id is\n * already pending or active, the existing job is returned instead of adding a\n * duplicate (idempotent enqueue — e.g. for resume/replay).\n */\n add<T = any>(name: string, payload: T, opts: AddOptions = {}): Job<T> {\n const t = now();\n if (opts.jobId) {\n const existing = this.driver\n .prepare(\n `SELECT * FROM _jobs WHERE job_id = ? AND status IN ('pending','active') LIMIT 1`,\n )\n .get(opts.jobId) as Row | undefined;\n if (existing) return deserialize(existing) as Job<T>;\n }\n const runAt = opts.runAt ?? (opts.delay ? t + opts.delay : t);\n const info = this.driver\n .prepare(\n `INSERT INTO _jobs (queue, status, priority, run_at, attempts, max_attempts, payload, job_id, created_at, updated_at)\n VALUES (?, 'pending', ?, ?, 0, ?, ?, ?, ?, ?)`,\n )\n .run(\n name,\n opts.priority ?? 0,\n runAt,\n opts.maxAttempts ?? this.maxAttempts,\n JSON.stringify(payload ?? null),\n opts.jobId ?? null,\n t,\n t,\n );\n const job = this.getJob(Number(info.lastInsertRowid))!;\n for (const w of this.workers) if (w.name === name) w.kick();\n return job as Job<T>;\n }\n\n /** Register a worker for a queue. Returns a handle with `stop()`. */\n process<T = any, R = any>(\n name: string,\n handler: Handler<T, R>,\n opts: ProcessOptions = {},\n ): Worker {\n const w = new WorkerImpl(this, name, handler as Handler, opts);\n this.workers.push(w);\n return w;\n }\n\n getJob<T = any>(id: number): Job<T> | undefined {\n const row = this.driver\n .prepare(`SELECT * FROM _jobs WHERE id = ?`)\n .get(id) as Row | undefined;\n return row ? (deserialize(row) as Job<T>) : undefined;\n }\n\n /** Count jobs by status (optionally for one queue). */\n counts(name?: string): Record<JobStatus, number> {\n const rows = (\n name\n ? this.driver\n .prepare(\n `SELECT status, COUNT(*) AS n FROM _jobs WHERE queue = ? GROUP BY status`,\n )\n .all(name)\n : this.driver\n .prepare(`SELECT status, COUNT(*) AS n FROM _jobs GROUP BY status`)\n .all()\n ) as Array<{ status: JobStatus; n: number }>;\n const out: Record<JobStatus, number> = {\n pending: 0,\n active: 0,\n done: 0,\n failed: 0,\n };\n for (const r of rows) out[r.status] = r.n;\n return out;\n }\n\n /**\n * Reset jobs stuck in `active` (e.g. from a crashed worker) back to `pending`\n * if they haven't been touched in `olderThanMs`. Returns the count recovered.\n */\n recover(olderThanMs = 60_000): number {\n return this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', locked_by=NULL, updated_at=?\n WHERE status='active' AND updated_at < ?`,\n )\n .run(now(), now() - olderThanMs).changes;\n }\n\n /** @internal Extend a running job's visibility timeout (worker heartbeat). */\n heartbeatInternal(id: number): void {\n this.driver\n .prepare(`UPDATE _jobs SET updated_at=? WHERE id=? AND status='active'`)\n .run(now(), id);\n }\n\n /** Stop all workers and wait for in-flight jobs to finish. */\n async close(): Promise<void> {\n await Promise.all(this.workers.map((w) => w.stop()));\n }\n\n /** @internal Atomically claim the next due job, counting the attempt. */\n claimInternal(name: string): Job | null {\n const t = now();\n const row = this.driver\n .prepare(\n `UPDATE _jobs SET status='active', attempts=attempts+1, locked_by=?, updated_at=?\n WHERE id = (\n SELECT id FROM _jobs\n WHERE queue=? AND status='pending' AND run_at<=?\n ORDER BY priority DESC, id ASC LIMIT 1\n )\n RETURNING *`,\n )\n .get(this.workerId, t, name, t) as Row | undefined;\n return row ? deserialize(row) : null;\n }\n\n /** @internal */\n completeInternal(job: Job, result: unknown): void {\n if (this.removeOnComplete) {\n this.driver.prepare(`DELETE FROM _jobs WHERE id = ?`).run(job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='done', result=?, error=NULL, updated_at=? WHERE id=?`,\n )\n .run(JSON.stringify(result ?? null), now(), job.id);\n }\n }\n\n /** @internal */\n failInternal(job: Job, err: unknown): void {\n // `job.attempts` was already incremented at claim time.\n const message = err instanceof Error ? err.message : String(err);\n if (job.attempts < job.maxAttempts) {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='pending', run_at=?, error=?, locked_by=NULL, updated_at=? WHERE id=?`,\n )\n .run(now() + this.backoff(job.attempts), message, now(), job.id);\n } else {\n this.driver\n .prepare(\n `UPDATE _jobs SET status='failed', error=?, updated_at=? WHERE id=?`,\n )\n .run(message, now(), job.id);\n }\n }\n}\n\n/** Create a job queue over a monlite database. */\nexport function createQueue(db: Monlite, opts?: QueueOptions): Queue {\n return new Queue(db, opts);\n}\n"]}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@monlite/queue",
3
- "version": "0.2.0",
3
+ "version": "0.3.1",
4
4
  "description": "Durable job queue for @monlite/core: retries, backoff, delayed jobs, concurrency — on SQLite.",
5
5
  "type": "module",
6
6
  "main": "./dist/index.cjs",
@@ -49,7 +49,7 @@
49
49
  "node": ">=18"
50
50
  },
51
51
  "dependencies": {
52
- "@monlite/core": "^2.4.0"
52
+ "@monlite/core": "^2.6.5"
53
53
  },
54
54
  "devDependencies": {
55
55
  "@types/node": "^22.10.0",