@iskra-bun/worker-kit 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +57 -0
- package/README.md +8 -2
- package/dist/index.d.ts +42 -6
- package/dist/index.js +61 -23
- package/dist/index.js.map +1 -1
- package/package.json +5 -2
- package/src/index.ts +88 -27
- package/src/types.ts +24 -11
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,62 @@
|
|
|
1
1
|
# @iskra-bun/worker-kit
|
|
2
2
|
|
|
3
|
+
## 0.3.1
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [e86ed55]
|
|
8
|
+
- Updated dependencies [ae7c798]
|
|
9
|
+
- Updated dependencies [ef10372]
|
|
10
|
+
- @iskra-bun/core@0.3.0
|
|
11
|
+
|
|
12
|
+
## 0.3.0
|
|
13
|
+
|
|
14
|
+
### Minor Changes
|
|
15
|
+
|
|
16
|
+
- 3579944: **Security:** runtime hardening from the second audit round.
|
|
17
|
+
|
|
18
|
+
- `process-kit` (**breaking**): a child no longer inherits the app's whole environment. It gets the variables programs need and that carry no secrets (`PATH`, `HOME`, `USER`, `SHELL`, `TERM`, the locale, `TZ`, the temp dir, `NODE_ENV`, and the Windows essentials), plus `env`: `DATABASE_URL`, `AUTH_SECRET`, cloud keys and whatever was loaded from `.env` reached every child, third-party code included. The new `inheritEnv` option (validated by the core config schema, as is `maxPendingStdinBytes`) takes more names to pass, or `true` for all of them as before.
|
|
19
|
+
- `process-kit`: `send()` refuses a message, with one warning until the child catches up, when the bytes still waiting for a child that is not reading its stdin would go over `maxPendingStdinBytes` (8 MiB by default). They piled up in the app's memory without a bound: 256 MiB sent to such a child grew RSS by 263 MiB. `send()` now resolves to whether the message was sent (`false` for every refusal).
|
|
20
|
+
- `core`: OpenTelemetry. The options of each entry of `otel.instrumentations` reach the instrumentation (the config schema kept only `enabled`, so hooks and `redactedQueryParams` were dropped). HTTP spans export URLs with the values of secret-looking query parameters (`SECRET_QUERY_PARAMS`: `token`, `access_token`, `api_key`, `key`, `code`, `state`, `sig`, `X-Amz-Signature`…) replaced by `REDACTED`, through instrumentation-http's `redactedQueryParams` / `redactedQueryParamsServer` (which the app can set) and a `requestHook` for releases without them; the app's own `requestHook` still runs. The startup log shows only the endpoint's origin (its path, query or password can be an API key), and a plain `http://` endpoint on a remote host logs a warning. New exports: `SECRET_QUERY_PARAMS`, `redactUrl()`, `autoInstrumentationOptions()`, `describeOtelEndpoint()`.
|
|
21
|
+
- `web-kit`: `OtelTracingFeature` exported `url.full` as requested (@hono/otel sets it to `c.req.url`): the `?token=` of an email verification link, the token of better-auth's `/reset-password/<token>` and `?api_key=` reached the collector. The values of `SECRET_QUERY_PARAMS` (or the new `redactedQueryParams`) and that path token are now `REDACTED`. New `ignoreIncomingTraceContext` option to start a new trace per request instead of continuing the client's `traceparent` (default unchanged). `@opentelemetry/api` is now a direct dependency (it already came with `@hono/otel`).
|
|
22
|
+
- `web-kit`: `OpenAPIFeature`'s `/docs` page loaded `@scalar/api-reference@latest` on the app's origin. It now loads a pinned release (1.68.0) with its SRI hash and `crossorigin`, sends a Content-Security-Policy (scripts from that host only; requests only to the app and the spec's `servers`), turns off Scalar's web fonts and AI agent (which sends the spec to Scalar's servers), and HTML-escapes the title. New options: `docs: false` serves neither `/openapi.json` nor `/docs`; `authorize(c)` gates both (they are registered before middleware added after `initialize()`, so a `basicAuth()` there did not cover them); `scalar: { src, integrity }` or `false`.
|
|
23
|
+
- `web-kit` (**breaking**): `HealthCheckFeature`'s `/health/ready` lists the check names (`checks`, `failed`) and `/health/live` the `uptime` only with `includeDetails: true`, like `/health`; the names of failed readiness checks are logged instead.
|
|
24
|
+
- `worker-kit` (**breaking**): finished jobs are no longer kept in Redis forever with their payloads (BullMQ's default when `removeOnComplete`/`removeOnFail` are unset, which worker-kit never set). The queue keeps the last 1000 completed jobs and the failed ones of the last 7 days (at most 5000); `defaultJobOptions` or a job's options override it (`false` keeps them all), and both options now take BullMQ's `{ age, count }` form. `result()` of a job removed since rejects. The dead-letter example in the docs logged the whole payload with `console.error`, outside the logger's redaction; it logs ids now.
|
|
25
|
+
|
|
26
|
+
- fd36d6d: Job routing and connection fixes.
|
|
27
|
+
|
|
28
|
+
- A job with no registered handler used to be marked **completed** (the worker returned early), so it silently disappeared. It now fails with BullMQ's `UnrecoverableError` (no pointless retries), stays in the failed set, and is dead-lettered when `deadLetter` is on.
|
|
29
|
+
- New `consume: false` option for producer-only processes (e.g. an API node): `start()` creates no `Worker`, and `enqueue` accepts jobs whose handler lives in another process. Previously every producer had to register the handler and therefore also consume jobs.
|
|
30
|
+
- Redis URLs keep the ACL username, percent-decode credentials, and `rediss://` enables TLS.
|
|
31
|
+
- BullMQ queue/worker connection errors go to the app logger instead of the console.
|
|
32
|
+
|
|
33
|
+
### Patch Changes
|
|
34
|
+
|
|
35
|
+
- 840439a: Packages declare the runtime they are tested on: `engines.bun` `>=1.3.0` (the monorepo now builds and tests on Bun 1.3). `create-iskra`, a CLI that also runs under `npm create iskra`, declares `engines.node` `>=18`.
|
|
36
|
+
|
|
37
|
+
Every package is published with an npm provenance attestation (`publishConfig.provenance`), linking each version to the commit and CI run that built it.
|
|
38
|
+
|
|
39
|
+
- cb3ec43: Register what each kit puts on the app with core's new registries: `app.context.get('db' | 'kv' | 'oracle')` returns the kit's driver, and the `process:*`, `socket:connected` / `socket:disconnected` and `worker:dead-letter` events have typed payloads. `ProcessManager.send()` takes `unknown` data.
|
|
40
|
+
- 87f6de2: `KVManager` throws when its constructor gets `adapter`, `driver` or `connection`: the store is chosen by the App config (`kv: { driver, connection }`), and the README's `new KVManager({ adapter: 'redis' })` was silently ignored, leaving the app on per-process memory. Without a `kv` driver it now logs a warning in production instead of an info line. The READMEs of kv-kit, worker-kit (`connection` and `queueName`, not `queue`), db-kit (the App's `db` config) and process-kit (the App's `processes` config) show working examples.
|
|
41
|
+
- 58d4a8f: `concurrency: 0` now means producer-only, like `consume: false`. It used to fall back to a concurrency of 1, so a service that set it to only enqueue (forms-app's forms-api did) also consumed jobs from the queue it had no handler for, and those jobs were lost.
|
|
42
|
+
- ee559ec: Per-job options no longer erase `defaultJobOptions`: every unset option was passed to BullMQ as `undefined`, which overrides the queue default, so a job enqueued with just `{ priority }` or `{ delay }`, and every job from `schedule()`, lost its `attempts`, `backoff` and `removeOnComplete`/`removeOnFail` (no retries, and completed repeat jobs kept in Redis forever). IPv6 Redis URLs (`redis://[::1]:6379`) now connect: the host kept its brackets.
|
|
43
|
+
- fba319a: A job whose handler throws is logged once at error level ("Job failed", now with `attemptsMade`) instead of twice: the processor also logged a `JobError` at error level before BullMQ's `failed` event logged it again. That processor log is now debug.
|
|
44
|
+
- Updated dependencies [620da18]
|
|
45
|
+
- Updated dependencies [b635a2c]
|
|
46
|
+
- Updated dependencies [5b2b0fd]
|
|
47
|
+
- Updated dependencies [58d4a8f]
|
|
48
|
+
- Updated dependencies [5c70c5b]
|
|
49
|
+
- Updated dependencies [ec198d4]
|
|
50
|
+
- Updated dependencies [cb3ec43]
|
|
51
|
+
- Updated dependencies [ef2009b]
|
|
52
|
+
- Updated dependencies [840439a]
|
|
53
|
+
- Updated dependencies [dbf8817]
|
|
54
|
+
- Updated dependencies [3dc5581]
|
|
55
|
+
- Updated dependencies [9872d30]
|
|
56
|
+
- Updated dependencies [f2346f5]
|
|
57
|
+
- Updated dependencies [3579944]
|
|
58
|
+
- @iskra-bun/core@0.2.0
|
|
59
|
+
|
|
3
60
|
## 0.2.0
|
|
4
61
|
|
|
5
62
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -17,14 +17,20 @@ import { App } from '@iskra-bun/core'
|
|
|
17
17
|
import { WorkerManager } from '@iskra-bun/worker-kit'
|
|
18
18
|
|
|
19
19
|
const app = new App({ name: 'mi-worker' })
|
|
20
|
-
app.register(
|
|
20
|
+
app.register(
|
|
21
|
+
new WorkerManager({
|
|
22
|
+
connection: process.env.REDIS_URL ?? 'redis://localhost:6379',
|
|
23
|
+
queueName: 'mis-jobs',
|
|
24
|
+
concurrency: 2,
|
|
25
|
+
}),
|
|
26
|
+
)
|
|
21
27
|
|
|
22
28
|
await app.start()
|
|
23
29
|
```
|
|
24
30
|
|
|
25
31
|
## Documentacion
|
|
26
32
|
|
|
27
|
-
Guia completa: [
|
|
33
|
+
Guia completa: [@iskra-bun/worker-kit](https://iskra-docs.fly.dev/es/packages/worker-kit/)
|
|
28
34
|
|
|
29
35
|
## Licencia
|
|
30
36
|
|
package/dist/index.d.ts
CHANGED
|
@@ -1,14 +1,22 @@
|
|
|
1
1
|
import { IskraError, Driver, App } from '@iskra-bun/core';
|
|
2
|
+
import { KeepJobs } from 'bullmq';
|
|
2
3
|
|
|
3
4
|
interface WorkerManagerOptions {
|
|
4
|
-
/** URL de conexión a Redis (ej: 'redis://localhost:6379') */
|
|
5
|
+
/** URL de conexión a Redis (ej: 'redis://user:pass@localhost:6379/0'; `rediss://` activa TLS) */
|
|
5
6
|
connection: string | {
|
|
6
7
|
host: string;
|
|
7
8
|
port: number;
|
|
9
|
+
username?: string;
|
|
8
10
|
password?: string;
|
|
9
11
|
db?: number;
|
|
12
|
+
tls?: object;
|
|
10
13
|
};
|
|
11
|
-
/**
|
|
14
|
+
/**
|
|
15
|
+
* `false` = solo productor: `start()` no crea un Worker y `enqueue` acepta
|
|
16
|
+
* jobs sin handler local (los procesa otro proceso). Default: true.
|
|
17
|
+
*/
|
|
18
|
+
consume?: boolean;
|
|
19
|
+
/** Cantidad de jobs que se procesan en paralelo (default: 1). `0` = solo productor, como `consume: false`. */
|
|
12
20
|
concurrency?: number;
|
|
13
21
|
/** Nombre de la queue en Redis (default: 'iskra-jobs') */
|
|
14
22
|
queueName?: string;
|
|
@@ -48,10 +56,17 @@ interface JobOptions {
|
|
|
48
56
|
type: 'fixed' | 'exponential';
|
|
49
57
|
delay: number;
|
|
50
58
|
};
|
|
51
|
-
/**
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
59
|
+
/**
|
|
60
|
+
* Jobs completados que quedan en Redis, con sus datos: `true` los borra,
|
|
61
|
+
* un número conserva los últimos N, `{ age, count }` por edad (s) y
|
|
62
|
+
* cantidad. Default `{ count: 1000 }`; `false` los conserva todos.
|
|
63
|
+
*/
|
|
64
|
+
removeOnComplete?: boolean | number | KeepJobs;
|
|
65
|
+
/**
|
|
66
|
+
* Jobs fallidos (sin reintentos pendientes) que quedan en Redis, igual que
|
|
67
|
+
* removeOnComplete. Default `{ age: 7 días, count: 5000 }`; `false` los conserva todos.
|
|
68
|
+
*/
|
|
69
|
+
removeOnFail?: boolean | number | KeepJobs;
|
|
55
70
|
/**
|
|
56
71
|
* Programa el job como repetible (cron o intervalo).
|
|
57
72
|
* Se reenvía a la opción `repeat` de BullMQ.
|
|
@@ -139,9 +154,25 @@ declare class WorkerManager implements Driver {
|
|
|
139
154
|
* @param repeat patrón cron (string) o `{ every: ms }`/`{ pattern: cron }`.
|
|
140
155
|
*/
|
|
141
156
|
schedule<T = unknown, R = unknown>(name: string, data: T, repeat: RepeatSpec, opts?: JobOptions): Promise<JobDescriptor<T, R>>;
|
|
157
|
+
/**
|
|
158
|
+
* `consume: false`, or `concurrency: 0` (which used to fall back to 1, so
|
|
159
|
+
* a service meant to only enqueue consumed jobs it had no handler for).
|
|
160
|
+
*/
|
|
161
|
+
private get producerOnly();
|
|
142
162
|
start(): Promise<void>;
|
|
143
163
|
stop(): Promise<void>;
|
|
164
|
+
private logConnectionError;
|
|
165
|
+
/**
|
|
166
|
+
* Turns a redis:// or rediss:// URL into ioredis options, keeping the ACL
|
|
167
|
+
* username, the percent-decoded password, the db index and TLS (rediss).
|
|
168
|
+
*/
|
|
144
169
|
private parseConnection;
|
|
170
|
+
/**
|
|
171
|
+
* Only the options that were given: BullMQ merges `{ ...defaultJobOptions,
|
|
172
|
+
* ...opts }`, so an explicit `undefined` erased the queue default (a job
|
|
173
|
+
* enqueued with just `{ priority }`, and every scheduled job, lost its
|
|
174
|
+
* attempts, backoff and removeOn* settings).
|
|
175
|
+
*/
|
|
145
176
|
private mapJobOptions;
|
|
146
177
|
/**
|
|
147
178
|
* Normaliza una RepeatSpec a la forma `repeat` de BullMQ:
|
|
@@ -177,5 +208,10 @@ declare class WorkerManager implements Driver {
|
|
|
177
208
|
*/
|
|
178
209
|
private getQueueEvents;
|
|
179
210
|
}
|
|
211
|
+
declare module '@iskra-bun/core' {
|
|
212
|
+
interface AppEvents {
|
|
213
|
+
'worker:dead-letter': DeadLetterPayload;
|
|
214
|
+
}
|
|
215
|
+
}
|
|
180
216
|
|
|
181
217
|
export { type DeadLetterPayload, type JobDescriptor, JobError, type JobHandler, type JobOptions, QueueError, type RepeatSpec, WorkerManager, type WorkerManagerOptions };
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// src/index.ts
|
|
2
|
-
import { Queue, QueueEvents, Worker } from "bullmq";
|
|
2
|
+
import { Queue, QueueEvents, UnrecoverableError, Worker } from "bullmq";
|
|
3
3
|
|
|
4
4
|
// src/errors.ts
|
|
5
5
|
import { IskraError, ErrorCodes } from "@iskra-bun/core";
|
|
@@ -17,6 +17,10 @@ var JobError = class extends IskraError {
|
|
|
17
17
|
};
|
|
18
18
|
|
|
19
19
|
// src/index.ts
|
|
20
|
+
var RETENTION = {
|
|
21
|
+
removeOnComplete: { count: 1e3 },
|
|
22
|
+
removeOnFail: { age: 7 * 24 * 60 * 60, count: 5e3 }
|
|
23
|
+
};
|
|
20
24
|
var WorkerManager = class _WorkerManager {
|
|
21
25
|
name = "WorkerManager";
|
|
22
26
|
app = null;
|
|
@@ -39,7 +43,7 @@ var WorkerManager = class _WorkerManager {
|
|
|
39
43
|
try {
|
|
40
44
|
this.queue = new Queue(this.options.queueName || "iskra-jobs", {
|
|
41
45
|
connection,
|
|
42
|
-
defaultJobOptions: this.mapJobOptions(this.options.defaultJobOptions)
|
|
46
|
+
defaultJobOptions: { ...RETENTION, ...this.mapJobOptions(this.options.defaultJobOptions) }
|
|
43
47
|
});
|
|
44
48
|
} catch (err) {
|
|
45
49
|
throw new QueueError("Failed to initialize BullMQ queue", {
|
|
@@ -47,6 +51,7 @@ var WorkerManager = class _WorkerManager {
|
|
|
47
51
|
context: { queueName: this.options.queueName || "iskra-jobs" }
|
|
48
52
|
});
|
|
49
53
|
}
|
|
54
|
+
this.queue.on("error", (err) => this.logConnectionError("queue", err));
|
|
50
55
|
}
|
|
51
56
|
/**
|
|
52
57
|
* Registra un handler para un tipo de job. El handler puede devolver un
|
|
@@ -81,15 +86,29 @@ var WorkerManager = class _WorkerManager {
|
|
|
81
86
|
async schedule(name, data, repeat, opts) {
|
|
82
87
|
return this.enqueue(name, data, { ...opts, repeat });
|
|
83
88
|
}
|
|
89
|
+
/**
|
|
90
|
+
* `consume: false`, or `concurrency: 0` (which used to fall back to 1, so
|
|
91
|
+
* a service meant to only enqueue consumed jobs it had no handler for).
|
|
92
|
+
*/
|
|
93
|
+
get producerOnly() {
|
|
94
|
+
return this.options.consume === false || this.options.concurrency === 0;
|
|
95
|
+
}
|
|
84
96
|
async start() {
|
|
97
|
+
if (this.producerOnly) {
|
|
98
|
+
this.app?.logger.info(
|
|
99
|
+
{ queue: this.options.queueName || "iskra-jobs" },
|
|
100
|
+
"WorkerManager started in producer-only mode (consume: false / concurrency: 0)"
|
|
101
|
+
);
|
|
102
|
+
return;
|
|
103
|
+
}
|
|
85
104
|
const connection = this.parseConnection();
|
|
86
105
|
this.worker = new Worker(
|
|
87
106
|
this.options.queueName || "iskra-jobs",
|
|
88
107
|
async (job) => {
|
|
89
108
|
const handler = this.handlers.get(job.name);
|
|
90
109
|
if (!handler) {
|
|
91
|
-
this.app?.logger.
|
|
92
|
-
|
|
110
|
+
this.app?.logger.error({ jobName: job.name, jobId: job.id }, "No handler registered for job");
|
|
111
|
+
throw new UnrecoverableError(`No handler registered for job "${job.name}"`);
|
|
93
112
|
}
|
|
94
113
|
try {
|
|
95
114
|
return await handler({
|
|
@@ -103,7 +122,7 @@ var WorkerManager = class _WorkerManager {
|
|
|
103
122
|
cause: err instanceof Error ? err : new Error(String(err)),
|
|
104
123
|
context: { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade }
|
|
105
124
|
});
|
|
106
|
-
this.app?.logger.
|
|
125
|
+
this.app?.logger.debug({ err: jobErr }, jobErr.message);
|
|
107
126
|
throw err;
|
|
108
127
|
}
|
|
109
128
|
},
|
|
@@ -118,10 +137,14 @@ var WorkerManager = class _WorkerManager {
|
|
|
118
137
|
this.worker.on("failed", (job, err) => {
|
|
119
138
|
this.onFailed(job, err);
|
|
120
139
|
});
|
|
121
|
-
this.
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
140
|
+
this.worker.on("error", (err) => this.logConnectionError("worker", err));
|
|
141
|
+
this.app?.logger.info(
|
|
142
|
+
{
|
|
143
|
+
queue: this.options.queueName || "iskra-jobs",
|
|
144
|
+
concurrency: this.options.concurrency || 1
|
|
145
|
+
},
|
|
146
|
+
"WorkerManager started"
|
|
147
|
+
);
|
|
125
148
|
}
|
|
126
149
|
async stop() {
|
|
127
150
|
this.stopped = true;
|
|
@@ -157,28 +180,40 @@ var WorkerManager = class _WorkerManager {
|
|
|
157
180
|
}
|
|
158
181
|
this.app?.logger.info("WorkerManager stopped");
|
|
159
182
|
}
|
|
183
|
+
logConnectionError(source, err) {
|
|
184
|
+
this.app?.logger.error({ err, source }, "WorkerManager: BullMQ connection error");
|
|
185
|
+
}
|
|
186
|
+
/**
|
|
187
|
+
* Turns a redis:// or rediss:// URL into ioredis options, keeping the ACL
|
|
188
|
+
* username, the percent-decoded password, the db index and TLS (rediss).
|
|
189
|
+
*/
|
|
160
190
|
parseConnection() {
|
|
161
191
|
if (typeof this.options.connection === "string") {
|
|
162
192
|
const url = new URL(this.options.connection);
|
|
163
193
|
return {
|
|
164
|
-
host
|
|
194
|
+
// URL keeps the brackets of an IPv6 host ("[::1]"); ioredis wants the bare address.
|
|
195
|
+
host: url.hostname.replace(/^\[(.*)\]$/, "$1"),
|
|
165
196
|
port: Number(url.port) || 6379,
|
|
166
|
-
|
|
167
|
-
|
|
197
|
+
username: url.username ? decodeURIComponent(url.username) : void 0,
|
|
198
|
+
password: url.password ? decodeURIComponent(url.password) : void 0,
|
|
199
|
+
db: url.pathname ? Number(url.pathname.slice(1)) || 0 : 0,
|
|
200
|
+
...url.protocol === "rediss:" ? { tls: {} } : {}
|
|
168
201
|
};
|
|
169
202
|
}
|
|
170
203
|
return this.options.connection;
|
|
171
204
|
}
|
|
205
|
+
/**
|
|
206
|
+
* Only the options that were given: BullMQ merges `{ ...defaultJobOptions,
|
|
207
|
+
* ...opts }`, so an explicit `undefined` erased the queue default (a job
|
|
208
|
+
* enqueued with just `{ priority }`, and every scheduled job, lost its
|
|
209
|
+
* attempts, backoff and removeOn* settings).
|
|
210
|
+
*/
|
|
172
211
|
mapJobOptions(opts) {
|
|
173
212
|
if (!opts) return void 0;
|
|
174
|
-
const mapped = {
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
backoff: opts.backoff,
|
|
179
|
-
removeOnComplete: opts.removeOnComplete,
|
|
180
|
-
removeOnFail: opts.removeOnFail
|
|
181
|
-
};
|
|
213
|
+
const mapped = {};
|
|
214
|
+
for (const key of ["attempts", "delay", "priority", "backoff", "removeOnComplete", "removeOnFail"]) {
|
|
215
|
+
if (opts[key] !== void 0) mapped[key] = opts[key];
|
|
216
|
+
}
|
|
182
217
|
if (opts.repeat !== void 0) {
|
|
183
218
|
mapped.repeat = this.mapRepeat(opts.repeat);
|
|
184
219
|
}
|
|
@@ -202,7 +237,7 @@ var WorkerManager = class _WorkerManager {
|
|
|
202
237
|
* problema; no muta nada.
|
|
203
238
|
*/
|
|
204
239
|
validateEnqueue(name, data, opts) {
|
|
205
|
-
if (!this.handlers.has(name)) {
|
|
240
|
+
if (!this.producerOnly && !this.handlers.has(name)) {
|
|
206
241
|
throw new QueueError(`No handler registered for job "${name}"`, {
|
|
207
242
|
context: { jobName: name }
|
|
208
243
|
});
|
|
@@ -265,10 +300,13 @@ var WorkerManager = class _WorkerManager {
|
|
|
265
300
|
* `worker:dead-letter` en el bus de eventos de la App.
|
|
266
301
|
*/
|
|
267
302
|
onFailed(job, err) {
|
|
268
|
-
this.app?.logger.error(
|
|
303
|
+
this.app?.logger.error(
|
|
304
|
+
{ jobId: job?.id, jobName: job?.name, attemptsMade: job?.attemptsMade, err },
|
|
305
|
+
"Job failed"
|
|
306
|
+
);
|
|
269
307
|
if (!this.options.deadLetter || !job) return;
|
|
270
308
|
const maxAttempts = job.opts?.attempts ?? 1;
|
|
271
|
-
if (job.attemptsMade < maxAttempts) return;
|
|
309
|
+
if (job.attemptsMade < maxAttempts && err?.name !== "UnrecoverableError") return;
|
|
272
310
|
const payload = {
|
|
273
311
|
jobId: job.id,
|
|
274
312
|
name: job.name,
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["../src/index.ts","../src/errors.ts"],"sourcesContent":["import type { Driver, App } from '@iskra-bun/core';\nimport { Queue, QueueEvents, Worker, type Job as BullJob } from 'bullmq';\nimport { QueueError, JobError } from './errors';\nimport type {\n WorkerManagerOptions,\n JobOptions,\n JobHandler,\n RepeatSpec,\n JobDescriptor,\n DeadLetterPayload,\n} from './types';\n\nexport type {\n WorkerManagerOptions,\n JobOptions,\n JobHandler,\n RepeatSpec,\n JobDescriptor,\n DeadLetterPayload,\n} from './types';\nexport * from './errors';\n\nexport class WorkerManager implements Driver {\n name = 'WorkerManager';\n private app: App | null = null;\n // eslint-disable-next-line @typescript-eslint/no-explicit-any\n private handlers: Map<string, JobHandler<any, any>> = new Map();\n private queue: Queue | null = null;\n private worker: Worker | null = null;\n private queueEvents: QueueEvents | null = null;\n private stopped = false;\n private options: WorkerManagerOptions;\n\n /** Tope de tamaño (bytes) del payload serializado de un job. */\n private static readonly MAX_PAYLOAD_BYTES = 1024 * 1024; // 1 MB\n\n constructor(options: WorkerManagerOptions) {\n this.options = options;\n }\n\n async init(app: App) {\n this.app = app;\n\n const connection = this.parseConnection();\n\n try {\n this.queue = new Queue(this.options.queueName || 'iskra-jobs', {\n connection,\n defaultJobOptions: this.mapJobOptions(this.options.defaultJobOptions),\n });\n } catch (err) {\n throw new QueueError('Failed to initialize BullMQ queue', {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n }\n\n /**\n * Registra un handler para un tipo de job. El handler puede devolver un\n * valor `R` que queda disponible como resultado del job.\n */\n register<T = unknown, R = void>(jobName: string, handler: JobHandler<T, R>) {\n this.handlers.set(jobName, handler);\n return this;\n }\n\n /**\n * Encola un job para ser procesado. Devuelve un descriptor que, además de\n * los datos del job, expone `result()` para esperar el valor de retorno del\n * handler.\n */\n async enqueue<T = unknown, R = unknown>(\n name: string,\n data: T,\n opts?: JobOptions,\n ): Promise<JobDescriptor<T, R>> {\n if (!this.queue) {\n throw new QueueError('Queue not initialized. Did you call init()?', {\n context: { jobName: name },\n });\n }\n\n this.validateEnqueue(name, data, opts);\n\n const job = await this.queue.add(name, data, this.mapJobOptions(opts));\n this.app?.logger.debug({ jobId: job.id, jobName: name }, 'Job enqueued');\n return this.buildDescriptor<T, R>(job, name, data);\n }\n\n /**\n * Programa un job repetible (cron o intervalo). Conveniencia sobre\n * `enqueue` con la opción `repeat` ya configurada.\n *\n * @param repeat patrón cron (string) o `{ every: ms }`/`{ pattern: cron }`.\n */\n async schedule<T = unknown, R = unknown>(\n name: string,\n data: T,\n repeat: RepeatSpec,\n opts?: JobOptions,\n ): Promise<JobDescriptor<T, R>> {\n return this.enqueue<T, R>(name, data, { ...opts, repeat });\n }\n\n async start() {\n const connection = this.parseConnection();\n\n this.worker = new Worker(\n this.options.queueName || 'iskra-jobs',\n async (job: BullJob) => {\n const handler = this.handlers.get(job.name);\n if (!handler) {\n this.app?.logger.warn({ jobName: job.name, jobId: job.id }, 'No handler registered for job');\n return;\n }\n\n try {\n return await handler({\n id: job.id!,\n name: job.name,\n data: job.data,\n attemptsMade: job.attemptsMade,\n });\n } catch (err) {\n const jobErr = new JobError(`Job \"${job.name}\" failed`, {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade },\n });\n this.app?.logger.error({ err: jobErr }, jobErr.message);\n throw err; // Re-throw para que BullMQ maneje el retry\n }\n },\n {\n connection,\n concurrency: this.options.concurrency || 1,\n },\n );\n\n this.worker.on('completed', (job) => {\n this.app?.logger.debug({ jobId: job.id, jobName: job.name }, 'Job completed');\n });\n\n this.worker.on('failed', (job, err) => {\n this.onFailed(job, err);\n });\n\n this.app?.logger.info({\n queue: this.options.queueName || 'iskra-jobs',\n concurrency: this.options.concurrency || 1,\n }, 'WorkerManager started');\n }\n\n async stop() {\n // Mark as stopped first so any in-flight result()/getQueueEvents() call\n // throws instead of lazily opening a fresh, never-closed QueueEvents.\n this.stopped = true;\n\n // Close the worker first (without force) so BullMQ waits for any\n // in-flight job to finish before tearing down its Redis connections.\n // Only then close the queue — closing them concurrently can cut the\n // queue connection out from under a still-draining worker.\n if (this.worker) {\n try {\n await this.worker.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing worker',\n );\n }\n }\n\n if (this.queueEvents) {\n try {\n await this.queueEvents.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing queue events',\n );\n }\n }\n\n if (this.queue) {\n try {\n await this.queue.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing queue',\n );\n }\n }\n\n this.app?.logger.info('WorkerManager stopped');\n }\n\n private parseConnection() {\n if (typeof this.options.connection === 'string') {\n const url = new URL(this.options.connection);\n return {\n host: url.hostname,\n port: Number(url.port) || 6379,\n password: url.password || undefined,\n db: url.pathname ? Number(url.pathname.slice(1)) || 0 : 0,\n };\n }\n return this.options.connection;\n }\n\n private mapJobOptions(opts?: JobOptions) {\n if (!opts) return undefined;\n const mapped: Record<string, unknown> = {\n attempts: opts.attempts,\n delay: opts.delay,\n priority: opts.priority,\n backoff: opts.backoff,\n removeOnComplete: opts.removeOnComplete,\n removeOnFail: opts.removeOnFail,\n };\n if (opts.repeat !== undefined) {\n mapped.repeat = this.mapRepeat(opts.repeat);\n }\n return mapped;\n }\n\n /**\n * Normaliza una RepeatSpec a la forma `repeat` de BullMQ:\n * - string → `{ pattern: cron }`\n * - `{ every }` / `{ pattern }` → se reenvían tal cual.\n */\n private mapRepeat(repeat: RepeatSpec) {\n if (typeof repeat === 'string') {\n return { pattern: repeat };\n }\n return { ...repeat };\n }\n\n /**\n * Valida la entrada de `enqueue` ANTES de tocar Redis, para evitar que\n * entrada no confiable inunde la queue, almacene payloads gigantes o\n * programe repeticiones malformadas. Lanza `QueueError` ante cualquier\n * problema; no muta nada.\n */\n private validateEnqueue(name: string, data: unknown, opts?: JobOptions) {\n if (!this.handlers.has(name)) {\n throw new QueueError(`No handler registered for job \"${name}\"`, {\n context: { jobName: name },\n });\n }\n\n this.validatePayloadSize(name, data);\n\n if (opts?.repeat !== undefined) {\n this.validateRepeat(name, opts.repeat);\n }\n }\n\n /** Rechaza payloads cuya serialización JSON excede el tope configurado. */\n private validatePayloadSize(name: string, data: unknown) {\n let serialized: string;\n try {\n serialized = JSON.stringify(data ?? null);\n } catch (err) {\n throw new QueueError(`Job \"${name}\" data is not serializable`, {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { jobName: name },\n });\n }\n\n const size = Buffer.byteLength(serialized, 'utf8');\n if (size > WorkerManager.MAX_PAYLOAD_BYTES) {\n throw new QueueError(\n `Job \"${name}\" payload too large: ${size} bytes (max ${WorkerManager.MAX_PAYLOAD_BYTES})`,\n { context: { jobName: name, size, max: WorkerManager.MAX_PAYLOAD_BYTES } },\n );\n }\n }\n\n /** Rechaza specs de repetición vacías, intervalos no positivos o crons en blanco. */\n private validateRepeat(name: string, repeat: RepeatSpec) {\n if (typeof repeat === 'string') {\n if (repeat.trim().length === 0) {\n throw new QueueError(`Job \"${name}\" has an empty cron repeat pattern`, {\n context: { jobName: name },\n });\n }\n return;\n }\n\n const hasEvery = 'every' in repeat;\n const hasPattern = 'pattern' in repeat;\n if (!hasEvery && !hasPattern) {\n throw new QueueError(`Job \"${name}\" repeat spec must define \"every\" or \"pattern\"`, {\n context: { jobName: name },\n });\n }\n\n if (hasEvery && !(typeof repeat.every === 'number' && repeat.every > 0)) {\n throw new QueueError(`Job \"${name}\" repeat \"every\" must be a positive number`, {\n context: { jobName: name, every: repeat.every },\n });\n }\n\n if (hasPattern && (typeof repeat.pattern !== 'string' || repeat.pattern.trim().length === 0)) {\n throw new QueueError(`Job \"${name}\" repeat \"pattern\" must be a non-empty cron string`, {\n context: { jobName: name },\n });\n }\n }\n\n /**\n * Maneja el evento `failed` del worker. Loggea el fallo y, si el job agotó\n * todos sus reintentos y `deadLetter` está activado, emite\n * `worker:dead-letter` en el bus de eventos de la App.\n */\n private onFailed(job: BullJob | undefined, err: Error) {\n this.app?.logger.error({ jobId: job?.id, jobName: job?.name, err }, 'Job failed');\n\n if (!this.options.deadLetter || !job) return;\n\n // BullMQ default attempts is 1 when unspecified.\n //\n // Assumed BullMQ `attemptsMade` semantics at the `failed` event: on a\n // job's TERMINAL failure (all retries exhausted) BullMQ reports\n // `attemptsMade == opts.attempts`, so `attemptsMade < maxAttempts`\n // identifies a non-terminal failure with a retry still pending. This is\n // verified against bullmq 5.78 (see test/dead-letter-attempts*.test.ts);\n // a future bump that changes `attemptsMade` reporting will fail those\n // tests loudly rather than silently skip dead-lettering.\n const maxAttempts = job.opts?.attempts ?? 1;\n if (job.attemptsMade < maxAttempts) return;\n\n const payload: DeadLetterPayload = {\n jobId: job.id,\n name: job.name,\n data: job.data,\n failedReason: job.failedReason ?? err?.message,\n attemptsMade: job.attemptsMade,\n };\n this.app?.events.emit('worker:dead-letter', payload);\n this.app?.logger.warn(\n { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade },\n 'Job routed to dead-letter',\n );\n }\n\n /**\n * Construye el descriptor de un job, incluyendo el helper `result()` que\n * espera el valor de retorno del handler vía `job.waitUntilFinished`.\n */\n private buildDescriptor<T, R>(job: BullJob, name: string, data: T): JobDescriptor<T, R> {\n return {\n id: job.id!,\n name,\n data,\n // async so a post-stop getQueueEvents() throw surfaces as a rejected\n // promise rather than a synchronous throw.\n result: async (ttlMs?: number): Promise<R> => {\n const queueEvents = this.getQueueEvents();\n return job.waitUntilFinished(queueEvents, ttlMs) as Promise<R>;\n },\n };\n }\n\n /**\n * Devuelve (creando perezosamente) una instancia compartida de QueueEvents\n * usada para esperar resultados de jobs.\n */\n private getQueueEvents(): QueueEvents {\n if (this.stopped) {\n throw new QueueError('WorkerManager is stopped; cannot open QueueEvents', {\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n if (!this.queueEvents) {\n try {\n this.queueEvents = new QueueEvents(this.options.queueName || 'iskra-jobs', {\n connection: this.parseConnection(),\n });\n } catch (err) {\n throw new QueueError('Failed to initialize BullMQ QueueEvents', {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n }\n return this.queueEvents;\n }\n}\n","import { IskraError, ErrorCodes } from '@iskra-bun/core';\n\n// ─── Queue Error ─────────────────────────────────────────────────────────────\n\nexport class QueueError extends IskraError {\n constructor(message: string, options?: { cause?: Error; context?: Record<string, unknown> }) {\n super(message, { code: ErrorCodes.QUEUE_ERROR, ...options });\n this.name = 'QueueError';\n }\n}\n\n// ─── Job Error ───────────────────────────────────────────────────────────────\n\nexport class JobError extends IskraError {\n constructor(message: string, options?: { cause?: Error; context?: Record<string, unknown> }) {\n super(message, { code: ErrorCodes.JOB_ERROR, ...options });\n this.name = 'JobError';\n }\n}\n"],"mappings":";AACA,SAAS,OAAO,aAAa,cAAmC;;;ACDhE,SAAS,YAAY,kBAAkB;AAIhC,IAAM,aAAN,cAAyB,WAAW;AAAA,EACvC,YAAY,SAAiB,SAAgE;AACzF,UAAM,SAAS,EAAE,MAAM,WAAW,aAAa,GAAG,QAAQ,CAAC;AAC3D,SAAK,OAAO;AAAA,EAChB;AACJ;AAIO,IAAM,WAAN,cAAuB,WAAW;AAAA,EACrC,YAAY,SAAiB,SAAgE;AACzF,UAAM,SAAS,EAAE,MAAM,WAAW,WAAW,GAAG,QAAQ,CAAC;AACzD,SAAK,OAAO;AAAA,EAChB;AACJ;;;ADIO,IAAM,gBAAN,MAAM,eAAgC;AAAA,EACzC,OAAO;AAAA,EACC,MAAkB;AAAA;AAAA,EAElB,WAA8C,oBAAI,IAAI;AAAA,EACtD,QAAsB;AAAA,EACtB,SAAwB;AAAA,EACxB,cAAkC;AAAA,EAClC,UAAU;AAAA,EACV;AAAA;AAAA,EAGR,OAAwB,oBAAoB,OAAO;AAAA;AAAA,EAEnD,YAAY,SAA+B;AACvC,SAAK,UAAU;AAAA,EACnB;AAAA,EAEA,MAAM,KAAK,KAAU;AACjB,SAAK,MAAM;AAEX,UAAM,aAAa,KAAK,gBAAgB;AAExC,QAAI;AACA,WAAK,QAAQ,IAAI,MAAM,KAAK,QAAQ,aAAa,cAAc;AAAA,QAC3D;AAAA,QACA,mBAAmB,KAAK,cAAc,KAAK,QAAQ,iBAAiB;AAAA,MACxE,CAAC;AAAA,IACL,SAAS,KAAK;AACV,YAAM,IAAI,WAAW,qCAAqC;AAAA,QACtD,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,QACzD,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,MACjE,CAAC;AAAA,IACL;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,SAAgC,SAAiB,SAA2B;AACxE,SAAK,SAAS,IAAI,SAAS,OAAO;AAClC,WAAO;AAAA,EACX;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,QACF,MACA,MACA,MAC4B;AAC5B,QAAI,CAAC,KAAK,OAAO;AACb,YAAM,IAAI,WAAW,+CAA+C;AAAA,QAChE,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,SAAK,gBAAgB,MAAM,MAAM,IAAI;AAErC,UAAM,MAAM,MAAM,KAAK,MAAM,IAAI,MAAM,MAAM,KAAK,cAAc,IAAI,CAAC;AACrE,SAAK,KAAK,OAAO,MAAM,EAAE,OAAO,IAAI,IAAI,SAAS,KAAK,GAAG,cAAc;AACvE,WAAO,KAAK,gBAAsB,KAAK,MAAM,IAAI;AAAA,EACrD;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQA,MAAM,SACF,MACA,MACA,QACA,MAC4B;AAC5B,WAAO,KAAK,QAAc,MAAM,MAAM,EAAE,GAAG,MAAM,OAAO,CAAC;AAAA,EAC7D;AAAA,EAEA,MAAM,QAAQ;AACV,UAAM,aAAa,KAAK,gBAAgB;AAExC,SAAK,SAAS,IAAI;AAAA,MACd,KAAK,QAAQ,aAAa;AAAA,MAC1B,OAAO,QAAiB;AACpB,cAAM,UAAU,KAAK,SAAS,IAAI,IAAI,IAAI;AAC1C,YAAI,CAAC,SAAS;AACV,eAAK,KAAK,OAAO,KAAK,EAAE,SAAS,IAAI,MAAM,OAAO,IAAI,GAAG,GAAG,+BAA+B;AAC3F;AAAA,QACJ;AAEA,YAAI;AACA,iBAAO,MAAM,QAAQ;AAAA,YACjB,IAAI,IAAI;AAAA,YACR,MAAM,IAAI;AAAA,YACV,MAAM,IAAI;AAAA,YACV,cAAc,IAAI;AAAA,UACtB,CAAC;AAAA,QACL,SAAS,KAAK;AACV,gBAAM,SAAS,IAAI,SAAS,QAAQ,IAAI,IAAI,YAAY;AAAA,YACpD,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,YACzD,SAAS,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,MAAM,cAAc,IAAI,aAAa;AAAA,UAChF,CAAC;AACD,eAAK,KAAK,OAAO,MAAM,EAAE,KAAK,OAAO,GAAG,OAAO,OAAO;AACtD,gBAAM;AAAA,QACV;AAAA,MACJ;AAAA,MACA;AAAA,QACI;AAAA,QACA,aAAa,KAAK,QAAQ,eAAe;AAAA,MAC7C;AAAA,IACJ;AAEA,SAAK,OAAO,GAAG,aAAa,CAAC,QAAQ;AACjC,WAAK,KAAK,OAAO,MAAM,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,KAAK,GAAG,eAAe;AAAA,IAChF,CAAC;AAED,SAAK,OAAO,GAAG,UAAU,CAAC,KAAK,QAAQ;AACnC,WAAK,SAAS,KAAK,GAAG;AAAA,IAC1B,CAAC;AAED,SAAK,KAAK,OAAO,KAAK;AAAA,MAClB,OAAO,KAAK,QAAQ,aAAa;AAAA,MACjC,aAAa,KAAK,QAAQ,eAAe;AAAA,IAC7C,GAAG,uBAAuB;AAAA,EAC9B;AAAA,EAEA,MAAM,OAAO;AAGT,SAAK,UAAU;AAMf,QAAI,KAAK,QAAQ;AACb,UAAI;AACA,cAAM,KAAK,OAAO,MAAM;AAAA,MAC5B,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,QAAI,KAAK,aAAa;AAClB,UAAI;AACA,cAAM,KAAK,YAAY,MAAM;AAAA,MACjC,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,QAAI,KAAK,OAAO;AACZ,UAAI;AACA,cAAM,KAAK,MAAM,MAAM;AAAA,MAC3B,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,SAAK,KAAK,OAAO,KAAK,uBAAuB;AAAA,EACjD;AAAA,EAEQ,kBAAkB;AACtB,QAAI,OAAO,KAAK,QAAQ,eAAe,UAAU;AAC7C,YAAM,MAAM,IAAI,IAAI,KAAK,QAAQ,UAAU;AAC3C,aAAO;AAAA,QACH,MAAM,IAAI;AAAA,QACV,MAAM,OAAO,IAAI,IAAI,KAAK;AAAA,QAC1B,UAAU,IAAI,YAAY;AAAA,QAC1B,IAAI,IAAI,WAAW,OAAO,IAAI,SAAS,MAAM,CAAC,CAAC,KAAK,IAAI;AAAA,MAC5D;AAAA,IACJ;AACA,WAAO,KAAK,QAAQ;AAAA,EACxB;AAAA,EAEQ,cAAc,MAAmB;AACrC,QAAI,CAAC,KAAM,QAAO;AAClB,UAAM,SAAkC;AAAA,MACpC,UAAU,KAAK;AAAA,MACf,OAAO,KAAK;AAAA,MACZ,UAAU,KAAK;AAAA,MACf,SAAS,KAAK;AAAA,MACd,kBAAkB,KAAK;AAAA,MACvB,cAAc,KAAK;AAAA,IACvB;AACA,QAAI,KAAK,WAAW,QAAW;AAC3B,aAAO,SAAS,KAAK,UAAU,KAAK,MAAM;AAAA,IAC9C;AACA,WAAO;AAAA,EACX;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,UAAU,QAAoB;AAClC,QAAI,OAAO,WAAW,UAAU;AAC5B,aAAO,EAAE,SAAS,OAAO;AAAA,IAC7B;AACA,WAAO,EAAE,GAAG,OAAO;AAAA,EACvB;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQQ,gBAAgB,MAAc,MAAe,MAAmB;AACpE,QAAI,CAAC,KAAK,SAAS,IAAI,IAAI,GAAG;AAC1B,YAAM,IAAI,WAAW,kCAAkC,IAAI,KAAK;AAAA,QAC5D,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,SAAK,oBAAoB,MAAM,IAAI;AAEnC,QAAI,MAAM,WAAW,QAAW;AAC5B,WAAK,eAAe,MAAM,KAAK,MAAM;AAAA,IACzC;AAAA,EACJ;AAAA;AAAA,EAGQ,oBAAoB,MAAc,MAAe;AACrD,QAAI;AACJ,QAAI;AACA,mBAAa,KAAK,UAAU,QAAQ,IAAI;AAAA,IAC5C,SAAS,KAAK;AACV,YAAM,IAAI,WAAW,QAAQ,IAAI,8BAA8B;AAAA,QAC3D,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,QACzD,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,UAAM,OAAO,OAAO,WAAW,YAAY,MAAM;AACjD,QAAI,OAAO,eAAc,mBAAmB;AACxC,YAAM,IAAI;AAAA,QACN,QAAQ,IAAI,wBAAwB,IAAI,eAAe,eAAc,iBAAiB;AAAA,QACtF,EAAE,SAAS,EAAE,SAAS,MAAM,MAAM,KAAK,eAAc,kBAAkB,EAAE;AAAA,MAC7E;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA,EAGQ,eAAe,MAAc,QAAoB;AACrD,QAAI,OAAO,WAAW,UAAU;AAC5B,UAAI,OAAO,KAAK,EAAE,WAAW,GAAG;AAC5B,cAAM,IAAI,WAAW,QAAQ,IAAI,sCAAsC;AAAA,UACnE,SAAS,EAAE,SAAS,KAAK;AAAA,QAC7B,CAAC;AAAA,MACL;AACA;AAAA,IACJ;AAEA,UAAM,WAAW,WAAW;AAC5B,UAAM,aAAa,aAAa;AAChC,QAAI,CAAC,YAAY,CAAC,YAAY;AAC1B,YAAM,IAAI,WAAW,QAAQ,IAAI,kDAAkD;AAAA,QAC/E,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,QAAI,YAAY,EAAE,OAAO,OAAO,UAAU,YAAY,OAAO,QAAQ,IAAI;AACrE,YAAM,IAAI,WAAW,QAAQ,IAAI,8CAA8C;AAAA,QAC3E,SAAS,EAAE,SAAS,MAAM,OAAO,OAAO,MAAM;AAAA,MAClD,CAAC;AAAA,IACL;AAEA,QAAI,eAAe,OAAO,OAAO,YAAY,YAAY,OAAO,QAAQ,KAAK,EAAE,WAAW,IAAI;AAC1F,YAAM,IAAI,WAAW,QAAQ,IAAI,sDAAsD;AAAA,QACnF,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,SAAS,KAA0B,KAAY;AACnD,SAAK,KAAK,OAAO,MAAM,EAAE,OAAO,KAAK,IAAI,SAAS,KAAK,MAAM,IAAI,GAAG,YAAY;AAEhF,QAAI,CAAC,KAAK,QAAQ,cAAc,CAAC,IAAK;AAWtC,UAAM,cAAc,IAAI,MAAM,YAAY;AAC1C,QAAI,IAAI,eAAe,YAAa;AAEpC,UAAM,UAA6B;AAAA,MAC/B,OAAO,IAAI;AAAA,MACX,MAAM,IAAI;AAAA,MACV,MAAM,IAAI;AAAA,MACV,cAAc,IAAI,gBAAgB,KAAK;AAAA,MACvC,cAAc,IAAI;AAAA,IACtB;AACA,SAAK,KAAK,OAAO,KAAK,sBAAsB,OAAO;AACnD,SAAK,KAAK,OAAO;AAAA,MACb,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,MAAM,cAAc,IAAI,aAAa;AAAA,MACnE;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA,EAMQ,gBAAsB,KAAc,MAAc,MAA8B;AACpF,WAAO;AAAA,MACH,IAAI,IAAI;AAAA,MACR;AAAA,MACA;AAAA;AAAA;AAAA,MAGA,QAAQ,OAAO,UAA+B;AAC1C,cAAM,cAAc,KAAK,eAAe;AACxC,eAAO,IAAI,kBAAkB,aAAa,KAAK;AAAA,MACnD;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA,EAMQ,iBAA8B;AAClC,QAAI,KAAK,SAAS;AACd,YAAM,IAAI,WAAW,qDAAqD;AAAA,QACtE,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,MACjE,CAAC;AAAA,IACL;AACA,QAAI,CAAC,KAAK,aAAa;AACnB,UAAI;AACA,aAAK,cAAc,IAAI,YAAY,KAAK,QAAQ,aAAa,cAAc;AAAA,UACvE,YAAY,KAAK,gBAAgB;AAAA,QACrC,CAAC;AAAA,MACL,SAAS,KAAK;AACV,cAAM,IAAI,WAAW,2CAA2C;AAAA,UAC5D,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,UACzD,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,QACjE,CAAC;AAAA,MACL;AAAA,IACJ;AACA,WAAO,KAAK;AAAA,EAChB;AACJ;","names":[]}
|
|
1
|
+
{"version":3,"sources":["../src/index.ts","../src/errors.ts"],"sourcesContent":["import type { Driver, App } from '@iskra-bun/core';\nimport { Queue, QueueEvents, UnrecoverableError, Worker, type Job as BullJob } from 'bullmq';\nimport { QueueError, JobError } from './errors';\nimport type {\n WorkerManagerOptions,\n JobOptions,\n JobHandler,\n RepeatSpec,\n JobDescriptor,\n DeadLetterPayload,\n} from './types';\n\nexport type {\n WorkerManagerOptions,\n JobOptions,\n JobHandler,\n RepeatSpec,\n JobDescriptor,\n DeadLetterPayload,\n} from './types';\nexport * from './errors';\n\n/**\n * The finished jobs kept in Redis, payload included, unless a job or\n * `defaultJobOptions` says otherwise: BullMQ's own default is to keep every\n * completed and failed job forever.\n */\nconst RETENTION = {\n removeOnComplete: { count: 1000 },\n removeOnFail: { age: 7 * 24 * 60 * 60, count: 5000 },\n};\n\nexport class WorkerManager implements Driver {\n name = 'WorkerManager';\n private app: App | null = null;\n // eslint-disable-next-line @typescript-eslint/no-explicit-any\n private handlers: Map<string, JobHandler<any, any>> = new Map();\n private queue: Queue | null = null;\n private worker: Worker | null = null;\n private queueEvents: QueueEvents | null = null;\n private stopped = false;\n private options: WorkerManagerOptions;\n\n /** Tope de tamaño (bytes) del payload serializado de un job. */\n private static readonly MAX_PAYLOAD_BYTES = 1024 * 1024; // 1 MB\n\n constructor(options: WorkerManagerOptions) {\n this.options = options;\n }\n\n async init(app: App) {\n this.app = app;\n\n const connection = this.parseConnection();\n\n try {\n this.queue = new Queue(this.options.queueName || 'iskra-jobs', {\n connection,\n defaultJobOptions: { ...RETENTION, ...this.mapJobOptions(this.options.defaultJobOptions) },\n });\n } catch (err) {\n throw new QueueError('Failed to initialize BullMQ queue', {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n // Without an 'error' listener BullMQ prints connection errors to the\n // console; route them through the app logger instead.\n this.queue.on('error', (err) => this.logConnectionError('queue', err));\n }\n\n /**\n * Registra un handler para un tipo de job. El handler puede devolver un\n * valor `R` que queda disponible como resultado del job.\n */\n register<T = unknown, R = void>(jobName: string, handler: JobHandler<T, R>) {\n this.handlers.set(jobName, handler);\n return this;\n }\n\n /**\n * Encola un job para ser procesado. Devuelve un descriptor que, además de\n * los datos del job, expone `result()` para esperar el valor de retorno del\n * handler.\n */\n async enqueue<T = unknown, R = unknown>(name: string, data: T, opts?: JobOptions): Promise<JobDescriptor<T, R>> {\n if (!this.queue) {\n throw new QueueError('Queue not initialized. Did you call init()?', {\n context: { jobName: name },\n });\n }\n\n this.validateEnqueue(name, data, opts);\n\n const job = await this.queue.add(name, data, this.mapJobOptions(opts));\n this.app?.logger.debug({ jobId: job.id, jobName: name }, 'Job enqueued');\n return this.buildDescriptor<T, R>(job, name, data);\n }\n\n /**\n * Programa un job repetible (cron o intervalo). Conveniencia sobre\n * `enqueue` con la opción `repeat` ya configurada.\n *\n * @param repeat patrón cron (string) o `{ every: ms }`/`{ pattern: cron }`.\n */\n async schedule<T = unknown, R = unknown>(\n name: string,\n data: T,\n repeat: RepeatSpec,\n opts?: JobOptions,\n ): Promise<JobDescriptor<T, R>> {\n return this.enqueue<T, R>(name, data, { ...opts, repeat });\n }\n\n /**\n * `consume: false`, or `concurrency: 0` (which used to fall back to 1, so\n * a service meant to only enqueue consumed jobs it had no handler for).\n */\n private get producerOnly(): boolean {\n return this.options.consume === false || this.options.concurrency === 0;\n }\n\n async start() {\n if (this.producerOnly) {\n this.app?.logger.info(\n { queue: this.options.queueName || 'iskra-jobs' },\n 'WorkerManager started in producer-only mode (consume: false / concurrency: 0)',\n );\n return;\n }\n\n const connection = this.parseConnection();\n\n this.worker = new Worker(\n this.options.queueName || 'iskra-jobs',\n async (job: BullJob) => {\n const handler = this.handlers.get(job.name);\n if (!handler) {\n // Returning would mark the job completed and silently drop\n // it. Fail it permanently instead (no retries), so it stays\n // in the failed set and reaches dead-letter handling.\n this.app?.logger.error({ jobName: job.name, jobId: job.id }, 'No handler registered for job');\n throw new UnrecoverableError(`No handler registered for job \"${job.name}\"`);\n }\n\n try {\n return await handler({\n id: job.id!,\n name: job.name,\n data: job.data,\n attemptsMade: job.attemptsMade,\n });\n } catch (err) {\n const jobErr = new JobError(`Job \"${job.name}\" failed`, {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade },\n });\n // Debug only: BullMQ then emits `failed`, and onFailed logs\n // the failure once at error level.\n this.app?.logger.debug({ err: jobErr }, jobErr.message);\n throw err; // Re-throw para que BullMQ maneje el retry\n }\n },\n {\n connection,\n concurrency: this.options.concurrency || 1,\n },\n );\n\n this.worker.on('completed', (job) => {\n this.app?.logger.debug({ jobId: job.id, jobName: job.name }, 'Job completed');\n });\n\n this.worker.on('failed', (job, err) => {\n this.onFailed(job, err);\n });\n\n this.worker.on('error', (err) => this.logConnectionError('worker', err));\n\n this.app?.logger.info(\n {\n queue: this.options.queueName || 'iskra-jobs',\n concurrency: this.options.concurrency || 1,\n },\n 'WorkerManager started',\n );\n }\n\n async stop() {\n // Mark as stopped first so any in-flight result()/getQueueEvents() call\n // throws instead of lazily opening a fresh, never-closed QueueEvents.\n this.stopped = true;\n\n // Close the worker first (without force) so BullMQ waits for any\n // in-flight job to finish before tearing down its Redis connections.\n // Only then close the queue — closing them concurrently can cut the\n // queue connection out from under a still-draining worker.\n if (this.worker) {\n try {\n await this.worker.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing worker',\n );\n }\n }\n\n if (this.queueEvents) {\n try {\n await this.queueEvents.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing queue events',\n );\n }\n }\n\n if (this.queue) {\n try {\n await this.queue.close();\n } catch (err) {\n this.app?.logger.error(\n { err: err instanceof Error ? err : new Error(String(err)) },\n 'WorkerManager: error while closing queue',\n );\n }\n }\n\n this.app?.logger.info('WorkerManager stopped');\n }\n\n private logConnectionError(source: string, err: Error) {\n this.app?.logger.error({ err, source }, 'WorkerManager: BullMQ connection error');\n }\n\n /**\n * Turns a redis:// or rediss:// URL into ioredis options, keeping the ACL\n * username, the percent-decoded password, the db index and TLS (rediss).\n */\n private parseConnection() {\n if (typeof this.options.connection === 'string') {\n const url = new URL(this.options.connection);\n return {\n // URL keeps the brackets of an IPv6 host (\"[::1]\"); ioredis wants the bare address.\n host: url.hostname.replace(/^\\[(.*)\\]$/, '$1'),\n port: Number(url.port) || 6379,\n username: url.username ? decodeURIComponent(url.username) : undefined,\n password: url.password ? decodeURIComponent(url.password) : undefined,\n db: url.pathname ? Number(url.pathname.slice(1)) || 0 : 0,\n ...(url.protocol === 'rediss:' ? { tls: {} } : {}),\n };\n }\n return this.options.connection;\n }\n\n /**\n * Only the options that were given: BullMQ merges `{ ...defaultJobOptions,\n * ...opts }`, so an explicit `undefined` erased the queue default (a job\n * enqueued with just `{ priority }`, and every scheduled job, lost its\n * attempts, backoff and removeOn* settings).\n */\n private mapJobOptions(opts?: JobOptions) {\n if (!opts) return undefined;\n const mapped: Record<string, unknown> = {};\n for (const key of ['attempts', 'delay', 'priority', 'backoff', 'removeOnComplete', 'removeOnFail'] as const) {\n if (opts[key] !== undefined) mapped[key] = opts[key];\n }\n if (opts.repeat !== undefined) {\n mapped.repeat = this.mapRepeat(opts.repeat);\n }\n return mapped;\n }\n\n /**\n * Normaliza una RepeatSpec a la forma `repeat` de BullMQ:\n * - string → `{ pattern: cron }`\n * - `{ every }` / `{ pattern }` → se reenvían tal cual.\n */\n private mapRepeat(repeat: RepeatSpec) {\n if (typeof repeat === 'string') {\n return { pattern: repeat };\n }\n return { ...repeat };\n }\n\n /**\n * Valida la entrada de `enqueue` ANTES de tocar Redis, para evitar que\n * entrada no confiable inunde la queue, almacene payloads gigantes o\n * programe repeticiones malformadas. Lanza `QueueError` ante cualquier\n * problema; no muta nada.\n */\n private validateEnqueue(name: string, data: unknown, opts?: JobOptions) {\n // A consuming instance only accepts jobs it can process itself; a\n // producer-only instance (consume: false) enqueues for other workers.\n if (!this.producerOnly && !this.handlers.has(name)) {\n throw new QueueError(`No handler registered for job \"${name}\"`, {\n context: { jobName: name },\n });\n }\n\n this.validatePayloadSize(name, data);\n\n if (opts?.repeat !== undefined) {\n this.validateRepeat(name, opts.repeat);\n }\n }\n\n /** Rechaza payloads cuya serialización JSON excede el tope configurado. */\n private validatePayloadSize(name: string, data: unknown) {\n let serialized: string;\n try {\n serialized = JSON.stringify(data ?? null);\n } catch (err) {\n throw new QueueError(`Job \"${name}\" data is not serializable`, {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { jobName: name },\n });\n }\n\n const size = Buffer.byteLength(serialized, 'utf8');\n if (size > WorkerManager.MAX_PAYLOAD_BYTES) {\n throw new QueueError(\n `Job \"${name}\" payload too large: ${size} bytes (max ${WorkerManager.MAX_PAYLOAD_BYTES})`,\n { context: { jobName: name, size, max: WorkerManager.MAX_PAYLOAD_BYTES } },\n );\n }\n }\n\n /** Rechaza specs de repetición vacías, intervalos no positivos o crons en blanco. */\n private validateRepeat(name: string, repeat: RepeatSpec) {\n if (typeof repeat === 'string') {\n if (repeat.trim().length === 0) {\n throw new QueueError(`Job \"${name}\" has an empty cron repeat pattern`, {\n context: { jobName: name },\n });\n }\n return;\n }\n\n const hasEvery = 'every' in repeat;\n const hasPattern = 'pattern' in repeat;\n if (!hasEvery && !hasPattern) {\n throw new QueueError(`Job \"${name}\" repeat spec must define \"every\" or \"pattern\"`, {\n context: { jobName: name },\n });\n }\n\n if (hasEvery && !(typeof repeat.every === 'number' && repeat.every > 0)) {\n throw new QueueError(`Job \"${name}\" repeat \"every\" must be a positive number`, {\n context: { jobName: name, every: repeat.every },\n });\n }\n\n if (hasPattern && (typeof repeat.pattern !== 'string' || repeat.pattern.trim().length === 0)) {\n throw new QueueError(`Job \"${name}\" repeat \"pattern\" must be a non-empty cron string`, {\n context: { jobName: name },\n });\n }\n }\n\n /**\n * Maneja el evento `failed` del worker. Loggea el fallo y, si el job agotó\n * todos sus reintentos y `deadLetter` está activado, emite\n * `worker:dead-letter` en el bus de eventos de la App.\n */\n private onFailed(job: BullJob | undefined, err: Error) {\n this.app?.logger.error(\n { jobId: job?.id, jobName: job?.name, attemptsMade: job?.attemptsMade, err },\n 'Job failed',\n );\n\n if (!this.options.deadLetter || !job) return;\n\n // BullMQ default attempts is 1 when unspecified.\n //\n // Assumed BullMQ `attemptsMade` semantics at the `failed` event: on a\n // job's TERMINAL failure (all retries exhausted) BullMQ reports\n // `attemptsMade == opts.attempts`, so `attemptsMade < maxAttempts`\n // identifies a non-terminal failure with a retry still pending. This is\n // verified against bullmq 5.78 (see test/dead-letter-attempts*.test.ts);\n // a future bump that changes `attemptsMade` reporting will fail those\n // tests loudly rather than silently skip dead-lettering.\n const maxAttempts = job.opts?.attempts ?? 1;\n // An UnrecoverableError (e.g. no handler) is terminal regardless of attempts left.\n if (job.attemptsMade < maxAttempts && err?.name !== 'UnrecoverableError') return;\n\n const payload: DeadLetterPayload = {\n jobId: job.id,\n name: job.name,\n data: job.data,\n failedReason: job.failedReason ?? err?.message,\n attemptsMade: job.attemptsMade,\n };\n this.app?.events.emit('worker:dead-letter', payload);\n this.app?.logger.warn(\n { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade },\n 'Job routed to dead-letter',\n );\n }\n\n /**\n * Construye el descriptor de un job, incluyendo el helper `result()` que\n * espera el valor de retorno del handler vía `job.waitUntilFinished`.\n */\n private buildDescriptor<T, R>(job: BullJob, name: string, data: T): JobDescriptor<T, R> {\n return {\n id: job.id!,\n name,\n data,\n // async so a post-stop getQueueEvents() throw surfaces as a rejected\n // promise rather than a synchronous throw.\n result: async (ttlMs?: number): Promise<R> => {\n const queueEvents = this.getQueueEvents();\n return job.waitUntilFinished(queueEvents, ttlMs) as Promise<R>;\n },\n };\n }\n\n /**\n * Devuelve (creando perezosamente) una instancia compartida de QueueEvents\n * usada para esperar resultados de jobs.\n */\n private getQueueEvents(): QueueEvents {\n if (this.stopped) {\n throw new QueueError('WorkerManager is stopped; cannot open QueueEvents', {\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n if (!this.queueEvents) {\n try {\n this.queueEvents = new QueueEvents(this.options.queueName || 'iskra-jobs', {\n connection: this.parseConnection(),\n });\n } catch (err) {\n throw new QueueError('Failed to initialize BullMQ QueueEvents', {\n cause: err instanceof Error ? err : new Error(String(err)),\n context: { queueName: this.options.queueName || 'iskra-jobs' },\n });\n }\n }\n return this.queueEvents;\n }\n}\n\n// A job that failed for good (see WorkerManager's dead-letter handling).\ndeclare module '@iskra-bun/core' {\n interface AppEvents {\n 'worker:dead-letter': DeadLetterPayload;\n }\n}\n","import { IskraError, ErrorCodes } from '@iskra-bun/core';\n\n// ─── Queue Error ─────────────────────────────────────────────────────────────\n\nexport class QueueError extends IskraError {\n constructor(message: string, options?: { cause?: Error; context?: Record<string, unknown> }) {\n super(message, { code: ErrorCodes.QUEUE_ERROR, ...options });\n this.name = 'QueueError';\n }\n}\n\n// ─── Job Error ───────────────────────────────────────────────────────────────\n\nexport class JobError extends IskraError {\n constructor(message: string, options?: { cause?: Error; context?: Record<string, unknown> }) {\n super(message, { code: ErrorCodes.JOB_ERROR, ...options });\n this.name = 'JobError';\n }\n}\n"],"mappings":";AACA,SAAS,OAAO,aAAa,oBAAoB,cAAmC;;;ACDpF,SAAS,YAAY,kBAAkB;AAIhC,IAAM,aAAN,cAAyB,WAAW;AAAA,EACvC,YAAY,SAAiB,SAAgE;AACzF,UAAM,SAAS,EAAE,MAAM,WAAW,aAAa,GAAG,QAAQ,CAAC;AAC3D,SAAK,OAAO;AAAA,EAChB;AACJ;AAIO,IAAM,WAAN,cAAuB,WAAW;AAAA,EACrC,YAAY,SAAiB,SAAgE;AACzF,UAAM,SAAS,EAAE,MAAM,WAAW,WAAW,GAAG,QAAQ,CAAC;AACzD,SAAK,OAAO;AAAA,EAChB;AACJ;;;ADSA,IAAM,YAAY;AAAA,EACd,kBAAkB,EAAE,OAAO,IAAK;AAAA,EAChC,cAAc,EAAE,KAAK,IAAI,KAAK,KAAK,IAAI,OAAO,IAAK;AACvD;AAEO,IAAM,gBAAN,MAAM,eAAgC;AAAA,EACzC,OAAO;AAAA,EACC,MAAkB;AAAA;AAAA,EAElB,WAA8C,oBAAI,IAAI;AAAA,EACtD,QAAsB;AAAA,EACtB,SAAwB;AAAA,EACxB,cAAkC;AAAA,EAClC,UAAU;AAAA,EACV;AAAA;AAAA,EAGR,OAAwB,oBAAoB,OAAO;AAAA;AAAA,EAEnD,YAAY,SAA+B;AACvC,SAAK,UAAU;AAAA,EACnB;AAAA,EAEA,MAAM,KAAK,KAAU;AACjB,SAAK,MAAM;AAEX,UAAM,aAAa,KAAK,gBAAgB;AAExC,QAAI;AACA,WAAK,QAAQ,IAAI,MAAM,KAAK,QAAQ,aAAa,cAAc;AAAA,QAC3D;AAAA,QACA,mBAAmB,EAAE,GAAG,WAAW,GAAG,KAAK,cAAc,KAAK,QAAQ,iBAAiB,EAAE;AAAA,MAC7F,CAAC;AAAA,IACL,SAAS,KAAK;AACV,YAAM,IAAI,WAAW,qCAAqC;AAAA,QACtD,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,QACzD,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,MACjE,CAAC;AAAA,IACL;AAGA,SAAK,MAAM,GAAG,SAAS,CAAC,QAAQ,KAAK,mBAAmB,SAAS,GAAG,CAAC;AAAA,EACzE;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,SAAgC,SAAiB,SAA2B;AACxE,SAAK,SAAS,IAAI,SAAS,OAAO;AAClC,WAAO;AAAA,EACX;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,QAAkC,MAAc,MAAS,MAAiD;AAC5G,QAAI,CAAC,KAAK,OAAO;AACb,YAAM,IAAI,WAAW,+CAA+C;AAAA,QAChE,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,SAAK,gBAAgB,MAAM,MAAM,IAAI;AAErC,UAAM,MAAM,MAAM,KAAK,MAAM,IAAI,MAAM,MAAM,KAAK,cAAc,IAAI,CAAC;AACrE,SAAK,KAAK,OAAO,MAAM,EAAE,OAAO,IAAI,IAAI,SAAS,KAAK,GAAG,cAAc;AACvE,WAAO,KAAK,gBAAsB,KAAK,MAAM,IAAI;AAAA,EACrD;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQA,MAAM,SACF,MACA,MACA,QACA,MAC4B;AAC5B,WAAO,KAAK,QAAc,MAAM,MAAM,EAAE,GAAG,MAAM,OAAO,CAAC;AAAA,EAC7D;AAAA;AAAA;AAAA;AAAA;AAAA,EAMA,IAAY,eAAwB;AAChC,WAAO,KAAK,QAAQ,YAAY,SAAS,KAAK,QAAQ,gBAAgB;AAAA,EAC1E;AAAA,EAEA,MAAM,QAAQ;AACV,QAAI,KAAK,cAAc;AACnB,WAAK,KAAK,OAAO;AAAA,QACb,EAAE,OAAO,KAAK,QAAQ,aAAa,aAAa;AAAA,QAChD;AAAA,MACJ;AACA;AAAA,IACJ;AAEA,UAAM,aAAa,KAAK,gBAAgB;AAExC,SAAK,SAAS,IAAI;AAAA,MACd,KAAK,QAAQ,aAAa;AAAA,MAC1B,OAAO,QAAiB;AACpB,cAAM,UAAU,KAAK,SAAS,IAAI,IAAI,IAAI;AAC1C,YAAI,CAAC,SAAS;AAIV,eAAK,KAAK,OAAO,MAAM,EAAE,SAAS,IAAI,MAAM,OAAO,IAAI,GAAG,GAAG,+BAA+B;AAC5F,gBAAM,IAAI,mBAAmB,kCAAkC,IAAI,IAAI,GAAG;AAAA,QAC9E;AAEA,YAAI;AACA,iBAAO,MAAM,QAAQ;AAAA,YACjB,IAAI,IAAI;AAAA,YACR,MAAM,IAAI;AAAA,YACV,MAAM,IAAI;AAAA,YACV,cAAc,IAAI;AAAA,UACtB,CAAC;AAAA,QACL,SAAS,KAAK;AACV,gBAAM,SAAS,IAAI,SAAS,QAAQ,IAAI,IAAI,YAAY;AAAA,YACpD,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,YACzD,SAAS,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,MAAM,cAAc,IAAI,aAAa;AAAA,UAChF,CAAC;AAGD,eAAK,KAAK,OAAO,MAAM,EAAE,KAAK,OAAO,GAAG,OAAO,OAAO;AACtD,gBAAM;AAAA,QACV;AAAA,MACJ;AAAA,MACA;AAAA,QACI;AAAA,QACA,aAAa,KAAK,QAAQ,eAAe;AAAA,MAC7C;AAAA,IACJ;AAEA,SAAK,OAAO,GAAG,aAAa,CAAC,QAAQ;AACjC,WAAK,KAAK,OAAO,MAAM,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,KAAK,GAAG,eAAe;AAAA,IAChF,CAAC;AAED,SAAK,OAAO,GAAG,UAAU,CAAC,KAAK,QAAQ;AACnC,WAAK,SAAS,KAAK,GAAG;AAAA,IAC1B,CAAC;AAED,SAAK,OAAO,GAAG,SAAS,CAAC,QAAQ,KAAK,mBAAmB,UAAU,GAAG,CAAC;AAEvE,SAAK,KAAK,OAAO;AAAA,MACb;AAAA,QACI,OAAO,KAAK,QAAQ,aAAa;AAAA,QACjC,aAAa,KAAK,QAAQ,eAAe;AAAA,MAC7C;AAAA,MACA;AAAA,IACJ;AAAA,EACJ;AAAA,EAEA,MAAM,OAAO;AAGT,SAAK,UAAU;AAMf,QAAI,KAAK,QAAQ;AACb,UAAI;AACA,cAAM,KAAK,OAAO,MAAM;AAAA,MAC5B,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,QAAI,KAAK,aAAa;AAClB,UAAI;AACA,cAAM,KAAK,YAAY,MAAM;AAAA,MACjC,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,QAAI,KAAK,OAAO;AACZ,UAAI;AACA,cAAM,KAAK,MAAM,MAAM;AAAA,MAC3B,SAAS,KAAK;AACV,aAAK,KAAK,OAAO;AAAA,UACb,EAAE,KAAK,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC,EAAE;AAAA,UAC3D;AAAA,QACJ;AAAA,MACJ;AAAA,IACJ;AAEA,SAAK,KAAK,OAAO,KAAK,uBAAuB;AAAA,EACjD;AAAA,EAEQ,mBAAmB,QAAgB,KAAY;AACnD,SAAK,KAAK,OAAO,MAAM,EAAE,KAAK,OAAO,GAAG,wCAAwC;AAAA,EACpF;AAAA;AAAA;AAAA;AAAA;AAAA,EAMQ,kBAAkB;AACtB,QAAI,OAAO,KAAK,QAAQ,eAAe,UAAU;AAC7C,YAAM,MAAM,IAAI,IAAI,KAAK,QAAQ,UAAU;AAC3C,aAAO;AAAA;AAAA,QAEH,MAAM,IAAI,SAAS,QAAQ,cAAc,IAAI;AAAA,QAC7C,MAAM,OAAO,IAAI,IAAI,KAAK;AAAA,QAC1B,UAAU,IAAI,WAAW,mBAAmB,IAAI,QAAQ,IAAI;AAAA,QAC5D,UAAU,IAAI,WAAW,mBAAmB,IAAI,QAAQ,IAAI;AAAA,QAC5D,IAAI,IAAI,WAAW,OAAO,IAAI,SAAS,MAAM,CAAC,CAAC,KAAK,IAAI;AAAA,QACxD,GAAI,IAAI,aAAa,YAAY,EAAE,KAAK,CAAC,EAAE,IAAI,CAAC;AAAA,MACpD;AAAA,IACJ;AACA,WAAO,KAAK,QAAQ;AAAA,EACxB;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQQ,cAAc,MAAmB;AACrC,QAAI,CAAC,KAAM,QAAO;AAClB,UAAM,SAAkC,CAAC;AACzC,eAAW,OAAO,CAAC,YAAY,SAAS,YAAY,WAAW,oBAAoB,cAAc,GAAY;AACzG,UAAI,KAAK,GAAG,MAAM,OAAW,QAAO,GAAG,IAAI,KAAK,GAAG;AAAA,IACvD;AACA,QAAI,KAAK,WAAW,QAAW;AAC3B,aAAO,SAAS,KAAK,UAAU,KAAK,MAAM;AAAA,IAC9C;AACA,WAAO;AAAA,EACX;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,UAAU,QAAoB;AAClC,QAAI,OAAO,WAAW,UAAU;AAC5B,aAAO,EAAE,SAAS,OAAO;AAAA,IAC7B;AACA,WAAO,EAAE,GAAG,OAAO;AAAA,EACvB;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAQQ,gBAAgB,MAAc,MAAe,MAAmB;AAGpE,QAAI,CAAC,KAAK,gBAAgB,CAAC,KAAK,SAAS,IAAI,IAAI,GAAG;AAChD,YAAM,IAAI,WAAW,kCAAkC,IAAI,KAAK;AAAA,QAC5D,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,SAAK,oBAAoB,MAAM,IAAI;AAEnC,QAAI,MAAM,WAAW,QAAW;AAC5B,WAAK,eAAe,MAAM,KAAK,MAAM;AAAA,IACzC;AAAA,EACJ;AAAA;AAAA,EAGQ,oBAAoB,MAAc,MAAe;AACrD,QAAI;AACJ,QAAI;AACA,mBAAa,KAAK,UAAU,QAAQ,IAAI;AAAA,IAC5C,SAAS,KAAK;AACV,YAAM,IAAI,WAAW,QAAQ,IAAI,8BAA8B;AAAA,QAC3D,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,QACzD,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,UAAM,OAAO,OAAO,WAAW,YAAY,MAAM;AACjD,QAAI,OAAO,eAAc,mBAAmB;AACxC,YAAM,IAAI;AAAA,QACN,QAAQ,IAAI,wBAAwB,IAAI,eAAe,eAAc,iBAAiB;AAAA,QACtF,EAAE,SAAS,EAAE,SAAS,MAAM,MAAM,KAAK,eAAc,kBAAkB,EAAE;AAAA,MAC7E;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA,EAGQ,eAAe,MAAc,QAAoB;AACrD,QAAI,OAAO,WAAW,UAAU;AAC5B,UAAI,OAAO,KAAK,EAAE,WAAW,GAAG;AAC5B,cAAM,IAAI,WAAW,QAAQ,IAAI,sCAAsC;AAAA,UACnE,SAAS,EAAE,SAAS,KAAK;AAAA,QAC7B,CAAC;AAAA,MACL;AACA;AAAA,IACJ;AAEA,UAAM,WAAW,WAAW;AAC5B,UAAM,aAAa,aAAa;AAChC,QAAI,CAAC,YAAY,CAAC,YAAY;AAC1B,YAAM,IAAI,WAAW,QAAQ,IAAI,kDAAkD;AAAA,QAC/E,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAEA,QAAI,YAAY,EAAE,OAAO,OAAO,UAAU,YAAY,OAAO,QAAQ,IAAI;AACrE,YAAM,IAAI,WAAW,QAAQ,IAAI,8CAA8C;AAAA,QAC3E,SAAS,EAAE,SAAS,MAAM,OAAO,OAAO,MAAM;AAAA,MAClD,CAAC;AAAA,IACL;AAEA,QAAI,eAAe,OAAO,OAAO,YAAY,YAAY,OAAO,QAAQ,KAAK,EAAE,WAAW,IAAI;AAC1F,YAAM,IAAI,WAAW,QAAQ,IAAI,sDAAsD;AAAA,QACnF,SAAS,EAAE,SAAS,KAAK;AAAA,MAC7B,CAAC;AAAA,IACL;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,SAAS,KAA0B,KAAY;AACnD,SAAK,KAAK,OAAO;AAAA,MACb,EAAE,OAAO,KAAK,IAAI,SAAS,KAAK,MAAM,cAAc,KAAK,cAAc,IAAI;AAAA,MAC3E;AAAA,IACJ;AAEA,QAAI,CAAC,KAAK,QAAQ,cAAc,CAAC,IAAK;AAWtC,UAAM,cAAc,IAAI,MAAM,YAAY;AAE1C,QAAI,IAAI,eAAe,eAAe,KAAK,SAAS,qBAAsB;AAE1E,UAAM,UAA6B;AAAA,MAC/B,OAAO,IAAI;AAAA,MACX,MAAM,IAAI;AAAA,MACV,MAAM,IAAI;AAAA,MACV,cAAc,IAAI,gBAAgB,KAAK;AAAA,MACvC,cAAc,IAAI;AAAA,IACtB;AACA,SAAK,KAAK,OAAO,KAAK,sBAAsB,OAAO;AACnD,SAAK,KAAK,OAAO;AAAA,MACb,EAAE,OAAO,IAAI,IAAI,SAAS,IAAI,MAAM,cAAc,IAAI,aAAa;AAAA,MACnE;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA,EAMQ,gBAAsB,KAAc,MAAc,MAA8B;AACpF,WAAO;AAAA,MACH,IAAI,IAAI;AAAA,MACR;AAAA,MACA;AAAA;AAAA;AAAA,MAGA,QAAQ,OAAO,UAA+B;AAC1C,cAAM,cAAc,KAAK,eAAe;AACxC,eAAO,IAAI,kBAAkB,aAAa,KAAK;AAAA,MACnD;AAAA,IACJ;AAAA,EACJ;AAAA;AAAA;AAAA;AAAA;AAAA,EAMQ,iBAA8B;AAClC,QAAI,KAAK,SAAS;AACd,YAAM,IAAI,WAAW,qDAAqD;AAAA,QACtE,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,MACjE,CAAC;AAAA,IACL;AACA,QAAI,CAAC,KAAK,aAAa;AACnB,UAAI;AACA,aAAK,cAAc,IAAI,YAAY,KAAK,QAAQ,aAAa,cAAc;AAAA,UACvE,YAAY,KAAK,gBAAgB;AAAA,QACrC,CAAC;AAAA,MACL,SAAS,KAAK;AACV,cAAM,IAAI,WAAW,2CAA2C;AAAA,UAC5D,OAAO,eAAe,QAAQ,MAAM,IAAI,MAAM,OAAO,GAAG,CAAC;AAAA,UACzD,SAAS,EAAE,WAAW,KAAK,QAAQ,aAAa,aAAa;AAAA,QACjE,CAAC;AAAA,MACL;AAAA,IACJ;AACA,WAAO,KAAK;AAAA,EAChB;AACJ;","names":[]}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@iskra-bun/worker-kit",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"description": "Cola de jobs en segundo plano para Iskra con BullMQ.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"iskra",
|
|
@@ -20,6 +20,9 @@
|
|
|
20
20
|
},
|
|
21
21
|
"homepage": "https://github.com/fearful/iskra/tree/main/packages/worker-kit#readme",
|
|
22
22
|
"bugs": "https://github.com/fearful/iskra/issues",
|
|
23
|
+
"engines": {
|
|
24
|
+
"bun": ">=1.3.0"
|
|
25
|
+
},
|
|
23
26
|
"type": "module",
|
|
24
27
|
"main": "./dist/index.js",
|
|
25
28
|
"module": "./dist/index.js",
|
|
@@ -47,7 +50,7 @@
|
|
|
47
50
|
"build": "tsup --config ../../tsup.config.ts"
|
|
48
51
|
},
|
|
49
52
|
"dependencies": {
|
|
50
|
-
"@iskra-bun/core": "0.
|
|
53
|
+
"@iskra-bun/core": "^0.3.0",
|
|
51
54
|
"bullmq": "^5.0.0"
|
|
52
55
|
},
|
|
53
56
|
"devDependencies": {
|
package/src/index.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { Driver, App } from '@iskra-bun/core';
|
|
2
|
-
import { Queue, QueueEvents, Worker, type Job as BullJob } from 'bullmq';
|
|
2
|
+
import { Queue, QueueEvents, UnrecoverableError, Worker, type Job as BullJob } from 'bullmq';
|
|
3
3
|
import { QueueError, JobError } from './errors';
|
|
4
4
|
import type {
|
|
5
5
|
WorkerManagerOptions,
|
|
@@ -20,6 +20,16 @@ export type {
|
|
|
20
20
|
} from './types';
|
|
21
21
|
export * from './errors';
|
|
22
22
|
|
|
23
|
+
/**
|
|
24
|
+
* The finished jobs kept in Redis, payload included, unless a job or
|
|
25
|
+
* `defaultJobOptions` says otherwise: BullMQ's own default is to keep every
|
|
26
|
+
* completed and failed job forever.
|
|
27
|
+
*/
|
|
28
|
+
const RETENTION = {
|
|
29
|
+
removeOnComplete: { count: 1000 },
|
|
30
|
+
removeOnFail: { age: 7 * 24 * 60 * 60, count: 5000 },
|
|
31
|
+
};
|
|
32
|
+
|
|
23
33
|
export class WorkerManager implements Driver {
|
|
24
34
|
name = 'WorkerManager';
|
|
25
35
|
private app: App | null = null;
|
|
@@ -46,7 +56,7 @@ export class WorkerManager implements Driver {
|
|
|
46
56
|
try {
|
|
47
57
|
this.queue = new Queue(this.options.queueName || 'iskra-jobs', {
|
|
48
58
|
connection,
|
|
49
|
-
defaultJobOptions: this.mapJobOptions(this.options.defaultJobOptions),
|
|
59
|
+
defaultJobOptions: { ...RETENTION, ...this.mapJobOptions(this.options.defaultJobOptions) },
|
|
50
60
|
});
|
|
51
61
|
} catch (err) {
|
|
52
62
|
throw new QueueError('Failed to initialize BullMQ queue', {
|
|
@@ -54,6 +64,9 @@ export class WorkerManager implements Driver {
|
|
|
54
64
|
context: { queueName: this.options.queueName || 'iskra-jobs' },
|
|
55
65
|
});
|
|
56
66
|
}
|
|
67
|
+
// Without an 'error' listener BullMQ prints connection errors to the
|
|
68
|
+
// console; route them through the app logger instead.
|
|
69
|
+
this.queue.on('error', (err) => this.logConnectionError('queue', err));
|
|
57
70
|
}
|
|
58
71
|
|
|
59
72
|
/**
|
|
@@ -70,11 +83,7 @@ export class WorkerManager implements Driver {
|
|
|
70
83
|
* los datos del job, expone `result()` para esperar el valor de retorno del
|
|
71
84
|
* handler.
|
|
72
85
|
*/
|
|
73
|
-
async enqueue<T = unknown, R = unknown>(
|
|
74
|
-
name: string,
|
|
75
|
-
data: T,
|
|
76
|
-
opts?: JobOptions,
|
|
77
|
-
): Promise<JobDescriptor<T, R>> {
|
|
86
|
+
async enqueue<T = unknown, R = unknown>(name: string, data: T, opts?: JobOptions): Promise<JobDescriptor<T, R>> {
|
|
78
87
|
if (!this.queue) {
|
|
79
88
|
throw new QueueError('Queue not initialized. Did you call init()?', {
|
|
80
89
|
context: { jobName: name },
|
|
@@ -103,7 +112,23 @@ export class WorkerManager implements Driver {
|
|
|
103
112
|
return this.enqueue<T, R>(name, data, { ...opts, repeat });
|
|
104
113
|
}
|
|
105
114
|
|
|
115
|
+
/**
|
|
116
|
+
* `consume: false`, or `concurrency: 0` (which used to fall back to 1, so
|
|
117
|
+
* a service meant to only enqueue consumed jobs it had no handler for).
|
|
118
|
+
*/
|
|
119
|
+
private get producerOnly(): boolean {
|
|
120
|
+
return this.options.consume === false || this.options.concurrency === 0;
|
|
121
|
+
}
|
|
122
|
+
|
|
106
123
|
async start() {
|
|
124
|
+
if (this.producerOnly) {
|
|
125
|
+
this.app?.logger.info(
|
|
126
|
+
{ queue: this.options.queueName || 'iskra-jobs' },
|
|
127
|
+
'WorkerManager started in producer-only mode (consume: false / concurrency: 0)',
|
|
128
|
+
);
|
|
129
|
+
return;
|
|
130
|
+
}
|
|
131
|
+
|
|
107
132
|
const connection = this.parseConnection();
|
|
108
133
|
|
|
109
134
|
this.worker = new Worker(
|
|
@@ -111,8 +136,11 @@ export class WorkerManager implements Driver {
|
|
|
111
136
|
async (job: BullJob) => {
|
|
112
137
|
const handler = this.handlers.get(job.name);
|
|
113
138
|
if (!handler) {
|
|
114
|
-
|
|
115
|
-
|
|
139
|
+
// Returning would mark the job completed and silently drop
|
|
140
|
+
// it. Fail it permanently instead (no retries), so it stays
|
|
141
|
+
// in the failed set and reaches dead-letter handling.
|
|
142
|
+
this.app?.logger.error({ jobName: job.name, jobId: job.id }, 'No handler registered for job');
|
|
143
|
+
throw new UnrecoverableError(`No handler registered for job "${job.name}"`);
|
|
116
144
|
}
|
|
117
145
|
|
|
118
146
|
try {
|
|
@@ -127,7 +155,9 @@ export class WorkerManager implements Driver {
|
|
|
127
155
|
cause: err instanceof Error ? err : new Error(String(err)),
|
|
128
156
|
context: { jobId: job.id, jobName: job.name, attemptsMade: job.attemptsMade },
|
|
129
157
|
});
|
|
130
|
-
|
|
158
|
+
// Debug only: BullMQ then emits `failed`, and onFailed logs
|
|
159
|
+
// the failure once at error level.
|
|
160
|
+
this.app?.logger.debug({ err: jobErr }, jobErr.message);
|
|
131
161
|
throw err; // Re-throw para que BullMQ maneje el retry
|
|
132
162
|
}
|
|
133
163
|
},
|
|
@@ -145,10 +175,15 @@ export class WorkerManager implements Driver {
|
|
|
145
175
|
this.onFailed(job, err);
|
|
146
176
|
});
|
|
147
177
|
|
|
148
|
-
this.
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
178
|
+
this.worker.on('error', (err) => this.logConnectionError('worker', err));
|
|
179
|
+
|
|
180
|
+
this.app?.logger.info(
|
|
181
|
+
{
|
|
182
|
+
queue: this.options.queueName || 'iskra-jobs',
|
|
183
|
+
concurrency: this.options.concurrency || 1,
|
|
184
|
+
},
|
|
185
|
+
'WorkerManager started',
|
|
186
|
+
);
|
|
152
187
|
}
|
|
153
188
|
|
|
154
189
|
async stop() {
|
|
@@ -196,29 +231,42 @@ export class WorkerManager implements Driver {
|
|
|
196
231
|
this.app?.logger.info('WorkerManager stopped');
|
|
197
232
|
}
|
|
198
233
|
|
|
234
|
+
private logConnectionError(source: string, err: Error) {
|
|
235
|
+
this.app?.logger.error({ err, source }, 'WorkerManager: BullMQ connection error');
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
/**
|
|
239
|
+
* Turns a redis:// or rediss:// URL into ioredis options, keeping the ACL
|
|
240
|
+
* username, the percent-decoded password, the db index and TLS (rediss).
|
|
241
|
+
*/
|
|
199
242
|
private parseConnection() {
|
|
200
243
|
if (typeof this.options.connection === 'string') {
|
|
201
244
|
const url = new URL(this.options.connection);
|
|
202
245
|
return {
|
|
203
|
-
host
|
|
246
|
+
// URL keeps the brackets of an IPv6 host ("[::1]"); ioredis wants the bare address.
|
|
247
|
+
host: url.hostname.replace(/^\[(.*)\]$/, '$1'),
|
|
204
248
|
port: Number(url.port) || 6379,
|
|
205
|
-
|
|
249
|
+
username: url.username ? decodeURIComponent(url.username) : undefined,
|
|
250
|
+
password: url.password ? decodeURIComponent(url.password) : undefined,
|
|
206
251
|
db: url.pathname ? Number(url.pathname.slice(1)) || 0 : 0,
|
|
252
|
+
...(url.protocol === 'rediss:' ? { tls: {} } : {}),
|
|
207
253
|
};
|
|
208
254
|
}
|
|
209
255
|
return this.options.connection;
|
|
210
256
|
}
|
|
211
257
|
|
|
258
|
+
/**
|
|
259
|
+
* Only the options that were given: BullMQ merges `{ ...defaultJobOptions,
|
|
260
|
+
* ...opts }`, so an explicit `undefined` erased the queue default (a job
|
|
261
|
+
* enqueued with just `{ priority }`, and every scheduled job, lost its
|
|
262
|
+
* attempts, backoff and removeOn* settings).
|
|
263
|
+
*/
|
|
212
264
|
private mapJobOptions(opts?: JobOptions) {
|
|
213
265
|
if (!opts) return undefined;
|
|
214
|
-
const mapped: Record<string, unknown> = {
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
backoff: opts.backoff,
|
|
219
|
-
removeOnComplete: opts.removeOnComplete,
|
|
220
|
-
removeOnFail: opts.removeOnFail,
|
|
221
|
-
};
|
|
266
|
+
const mapped: Record<string, unknown> = {};
|
|
267
|
+
for (const key of ['attempts', 'delay', 'priority', 'backoff', 'removeOnComplete', 'removeOnFail'] as const) {
|
|
268
|
+
if (opts[key] !== undefined) mapped[key] = opts[key];
|
|
269
|
+
}
|
|
222
270
|
if (opts.repeat !== undefined) {
|
|
223
271
|
mapped.repeat = this.mapRepeat(opts.repeat);
|
|
224
272
|
}
|
|
@@ -244,7 +292,9 @@ export class WorkerManager implements Driver {
|
|
|
244
292
|
* problema; no muta nada.
|
|
245
293
|
*/
|
|
246
294
|
private validateEnqueue(name: string, data: unknown, opts?: JobOptions) {
|
|
247
|
-
|
|
295
|
+
// A consuming instance only accepts jobs it can process itself; a
|
|
296
|
+
// producer-only instance (consume: false) enqueues for other workers.
|
|
297
|
+
if (!this.producerOnly && !this.handlers.has(name)) {
|
|
248
298
|
throw new QueueError(`No handler registered for job "${name}"`, {
|
|
249
299
|
context: { jobName: name },
|
|
250
300
|
});
|
|
@@ -316,7 +366,10 @@ export class WorkerManager implements Driver {
|
|
|
316
366
|
* `worker:dead-letter` en el bus de eventos de la App.
|
|
317
367
|
*/
|
|
318
368
|
private onFailed(job: BullJob | undefined, err: Error) {
|
|
319
|
-
this.app?.logger.error(
|
|
369
|
+
this.app?.logger.error(
|
|
370
|
+
{ jobId: job?.id, jobName: job?.name, attemptsMade: job?.attemptsMade, err },
|
|
371
|
+
'Job failed',
|
|
372
|
+
);
|
|
320
373
|
|
|
321
374
|
if (!this.options.deadLetter || !job) return;
|
|
322
375
|
|
|
@@ -330,7 +383,8 @@ export class WorkerManager implements Driver {
|
|
|
330
383
|
// a future bump that changes `attemptsMade` reporting will fail those
|
|
331
384
|
// tests loudly rather than silently skip dead-lettering.
|
|
332
385
|
const maxAttempts = job.opts?.attempts ?? 1;
|
|
333
|
-
|
|
386
|
+
// An UnrecoverableError (e.g. no handler) is terminal regardless of attempts left.
|
|
387
|
+
if (job.attemptsMade < maxAttempts && err?.name !== 'UnrecoverableError') return;
|
|
334
388
|
|
|
335
389
|
const payload: DeadLetterPayload = {
|
|
336
390
|
jobId: job.id,
|
|
@@ -389,3 +443,10 @@ export class WorkerManager implements Driver {
|
|
|
389
443
|
return this.queueEvents;
|
|
390
444
|
}
|
|
391
445
|
}
|
|
446
|
+
|
|
447
|
+
// A job that failed for good (see WorkerManager's dead-letter handling).
|
|
448
|
+
declare module '@iskra-bun/core' {
|
|
449
|
+
interface AppEvents {
|
|
450
|
+
'worker:dead-letter': DeadLetterPayload;
|
|
451
|
+
}
|
|
452
|
+
}
|
package/src/types.ts
CHANGED
|
@@ -1,7 +1,16 @@
|
|
|
1
|
+
import type { KeepJobs } from 'bullmq';
|
|
2
|
+
|
|
1
3
|
export interface WorkerManagerOptions {
|
|
2
|
-
/** URL de conexión a Redis (ej: 'redis://localhost:6379') */
|
|
3
|
-
connection:
|
|
4
|
-
|
|
4
|
+
/** URL de conexión a Redis (ej: 'redis://user:pass@localhost:6379/0'; `rediss://` activa TLS) */
|
|
5
|
+
connection:
|
|
6
|
+
| string
|
|
7
|
+
| { host: string; port: number; username?: string; password?: string; db?: number; tls?: object };
|
|
8
|
+
/**
|
|
9
|
+
* `false` = solo productor: `start()` no crea un Worker y `enqueue` acepta
|
|
10
|
+
* jobs sin handler local (los procesa otro proceso). Default: true.
|
|
11
|
+
*/
|
|
12
|
+
consume?: boolean;
|
|
13
|
+
/** Cantidad de jobs que se procesan en paralelo (default: 1). `0` = solo productor, como `consume: false`. */
|
|
5
14
|
concurrency?: number;
|
|
6
15
|
/** Nombre de la queue en Redis (default: 'iskra-jobs') */
|
|
7
16
|
queueName?: string;
|
|
@@ -22,10 +31,7 @@ export interface WorkerManagerOptions {
|
|
|
22
31
|
* - `{ every: ms }` repite cada `ms` milisegundos.
|
|
23
32
|
* - `{ pattern: cron }` repite según el patrón cron, con opciones extra.
|
|
24
33
|
*/
|
|
25
|
-
export type RepeatSpec =
|
|
26
|
-
| string
|
|
27
|
-
| { every: number; limit?: number }
|
|
28
|
-
| { pattern: string; limit?: number; tz?: string };
|
|
34
|
+
export type RepeatSpec = string | { every: number; limit?: number } | { pattern: string; limit?: number; tz?: string };
|
|
29
35
|
|
|
30
36
|
export interface JobOptions {
|
|
31
37
|
/** Reintentos en caso de fallo */
|
|
@@ -39,10 +45,17 @@ export interface JobOptions {
|
|
|
39
45
|
type: 'fixed' | 'exponential';
|
|
40
46
|
delay: number;
|
|
41
47
|
};
|
|
42
|
-
/**
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
48
|
+
/**
|
|
49
|
+
* Jobs completados que quedan en Redis, con sus datos: `true` los borra,
|
|
50
|
+
* un número conserva los últimos N, `{ age, count }` por edad (s) y
|
|
51
|
+
* cantidad. Default `{ count: 1000 }`; `false` los conserva todos.
|
|
52
|
+
*/
|
|
53
|
+
removeOnComplete?: boolean | number | KeepJobs;
|
|
54
|
+
/**
|
|
55
|
+
* Jobs fallidos (sin reintentos pendientes) que quedan en Redis, igual que
|
|
56
|
+
* removeOnComplete. Default `{ age: 7 días, count: 5000 }`; `false` los conserva todos.
|
|
57
|
+
*/
|
|
58
|
+
removeOnFail?: boolean | number | KeepJobs;
|
|
46
59
|
/**
|
|
47
60
|
* Programa el job como repetible (cron o intervalo).
|
|
48
61
|
* Se reenvía a la opción `repeat` de BullMQ.
|