@warlock.js/queue 5.34.0 → 5.34.2

package/CHANGELOG.md

@@ -4,6 +4,12 @@ All notable changes to `@warlock.js/queue` are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
 
+## 5.34.2 - 2026-10-19
+
+### Fixed
+
+- Merge package orientation into the grouped skill index and remove the duplicate overview topic; preserve that orientation in generated LLM projections.
+
 ## 5.34.0 - 2026-10-19
 
 ### Added

package/llms-full.txt

@@ -4,6 +4,47 @@
 
 > Generated artifact. Concatenates every SKILL.md and reference file under `@warlock.js/queue/skills/`. Re-run `node scripts/generate-llms.mjs` after any change.
 
+## Package orientation
+
+---
+description: 'Background jobs on BullMQ + Redis, local SQLite, or memory: retries, delays, priorities, workers, failed-job recovery. Exports `defineJob`, `queueConnector`, `startWorkers`, `closeQueue`, `getQueue`, `failedJobs`, `retryFailedJob`, `mountQueueDashboard`, `QueueConfig`, `QueueJob`. Use for: "run this in the background", "retry a failed job", "delay a job", "survive restarts", "job dashboard". Not this package: cron or recurring schedules → @warlock.js/scheduler; broker pub/sub messaging → @warlock.js/herald; core `Queue` is an in-memory batcher.'
+---
+
+# @warlock.js/queue
+
+Jobs are stored in Redis (default; not bundled), local SQLite without Redis, or memory for tests, retried on failure, and run by workers, in the app process by default. `defineJob({ name, handle })` registers a handler and returns a typed job; `job.dispatch(payload, options)` enqueues it. Any process with the same definition and a worker can run it. Server-only.
+
+## The 80% path
+
+1. Use this index for orientation and choose a task below.
+2. Configure `src/config/queue.ts` and register `queueConnector()`: `configure-queue.md`.
+3. Define and dispatch jobs (attempts, backoff, delay, priority, progress): `define-jobs.md`.
+4. Inspect and recover failures, optionally the bull-board dashboard: `manage-failed-jobs.md`.
+5. Back `@warlock.js/notifications` `.queue()` with BullMQ: `queue-notifications.md`.
+
+## Conventions and pitfalls
+
+- Job names are the routing key; a job with no matching `defineJob` in the worker process cannot run.
+- Handlers must be idempotent: retries re-run them.
+- Shutdown lets active jobs finish up to `workers.shutdownTimeout`, then closes connections.
+- Guard the dashboard with middleware; an unguarded one is refused on purpose.
+- Core `Queue` is an in-memory batcher, lost on exit; use this package when work must survive.
+- Do not value-import from client code.
+
+Use the `local-queues` topic for `driver: "local"` / `"memory"`, Node requirements, leases, crash recovery, and cancellation.
+
+## Mental model
+
+- **Job definition** — `defineJob({ name, handle })` registers a handler by name and returns a typed job.
+- **Dispatch** — `job.dispatch(payload, options)` stores the job in the configured backend. Any process with the same definition and a worker can run it.
+- **Worker** — one per queue name, started by `queueConnector()` (or `startWorkers()`); routes each job to the handler with its name.
+- **Shutdown** — workers stop taking jobs, active jobs finish (up to `workers.shutdownTimeout`), then connections close.
+
+## Not core's `Queue`
+
+`@warlock.js/core` exports `Queue` — an in-memory batcher that flushes items by size or interval inside one process. No storage, no retries, lost on exit. Use `@warlock.js/queue` when work must survive a restart or be retried.
+
+
 ## configure-queue  `@warlock.js/queue/configure-queue/SKILL.md`
 
 ---
@@ -365,37 +406,6 @@ await queueDashboard(getHttpServer(), {
 - It has no authentication of its own — pass `middleware` yourself; this path does not enforce the production guard that the `dashboard` config does.
 
 
-## overview  `@warlock.js/queue/overview/SKILL.md`
-
----
-name: overview
-description: "@warlock.js/queue overview for orientation and choosing the focused skill for the task."
----
-
-# `@warlock.js/queue` — overview
-
-Durable background jobs. Jobs are stored in **Redis** (default), local SQLite without Redis, or memory for tests, retried on failure, and processed by workers — inside the app process by default.
-
-## Mental model
-
-- **Job definition** — `defineJob({ name, handle })` registers a handler by name and returns a typed job.
-- **Dispatch** — `job.dispatch(payload, options)` stores the job in the configured backend. Any process with the same definition and a worker can run it.
-- **Worker** — one per queue name, started by `queueConnector()` (or `startWorkers()`); routes each job to the handler with its name.
-- **Shutdown** — workers stop taking jobs, active jobs finish (up to `workers.shutdownTimeout`), then connections close.
-
-## Not core's `Queue`
-
-`@warlock.js/core` exports `Queue` — an in-memory batcher that flushes items by size or interval inside one process. No storage, no retries, lost on exit. Use `@warlock.js/queue` when work must survive a restart or be retried.
-
-## Skills index
-
-- `local-queues` — SQLite/memory configuration, leases, crash recovery, cancellation.
-- `configure-queue` — `src/config/queue.ts`, `queueConnector()`, workers on/off, shutdown.
-- `define-jobs` — `defineJob`, `dispatch` options, retries, progress, `find`.
-- `manage-failed-jobs` — `failedJobs`, `retryFailedJob`, the bull-board dashboard.
-- `queue-notifications` — `bullmqQueue()` for notifications `.queue()`.
-
-
 ## queue-notifications  `@warlock.js/queue/queue-notifications/SKILL.md`
 
 ---

package/llms.txt

@@ -6,9 +6,9 @@
 
 ## Skills
 
+- [Package orientation](@warlock.js/queue/skills/index.md): Background jobs on BullMQ + Redis, local SQLite, or memory: retries, delays, priorities, workers, failed-job recovery. Exports `defineJob`, `queueConnector`, `startWorkers`, `closeQueue`, `getQueue`, `failedJobs`, `retryFailedJob`, `mountQueueDashboard`, `QueueConfig`, `QueueJob`. Use for: "run this in the background", "retry a failed job", "delay a job", "survive restarts", "job dashboard". Not this package: cron or recurring schedules → @warlock.js/scheduler; broker pub/sub messaging → @warlock.js/herald; core `Queue` is an in-memory batcher.
 - [configure-queue](@warlock.js/queue/configure-queue/SKILL.md): Configure `@warlock.js/queue`: the declarative `src/config/queue.ts` (`QueueConfig` — `connection`, `prefix`, `defaultQueue`, `defaultJobOptions`, `workers: { enabled, concurrency, shutdownTimeout }`), registering `queueConnector()` in `warlock.config.ts > connectors`, running a dispatch-only process, and graceful shutdown. Programmatic `setQueueConfig` / `startWorkers` / `closeQueue` for scripts and tests. Triggers: `QueueConfig`, `queueConnector`, `setQueueConfig`, `startWorkers`, `closeQueue`, `workers.enabled`, `shutdownTimeout`; "configure the queue", "connect BullMQ to Redis", "disable workers in the web process", "graceful shutdown of jobs". Skip: writing jobs — the `define-jobs` topic.
 - [define-jobs](@warlock.js/queue/define-jobs/SKILL.md): Define and dispatch background jobs with `@warlock.js/queue`: `defineJob({ name, queue?, attempts?, backoff?, removeOnComplete?, removeOnFail?, handle(payload, ctx) })` returns a typed job; `job.dispatch(payload, { delay, priority, jobId, attempts, backoff })`; the handler context (`id`, `attempt`, `maxAttempts`, `progress()`, `log()`); reading a job with `job.find(id)`. Triggers: `setDispatchInterceptor`, `queueFake`, `defineJob`, `.dispatch(`, `ctx.progress`, `JobContext`, `DispatchOptions`, `backoff`, `attempts`, `priority`, `jobId`; "run this in the background", "retry with backoff", "delay a job", "job progress", "idempotent dispatch". Skip: config and workers — the `configure-queue` topic; failed jobs — the `manage-failed-jobs` topic.
 - [local-queues](@warlock.js/queue/local-queues/SKILL.md): Run @warlock.js/queue without Redis: driver: "local" for durable single-host SQLite jobs using node:sqlite on Node >= 22.15, or driver: "memory" for real handler execution in tests. Covers local.path, leases, crash recovery, polling, concurrency, cancellation, graceful shutdown, and at-least-once delivery. Use for local/desktop background jobs, testing workers without Redis, or configuring SQLite queues.
 - [manage-failed-jobs](@warlock.js/queue/manage-failed-jobs/SKILL.md): Inspect and retry failed `@warlock.js/queue` jobs: `failedJobs({ queue, start, end })` returns `FailedJob[]` (`id`, `name`, `payload`, `attemptsMade`, `failedReason`, `stacktrace`, `failedAt`, `retry()`), `retryFailedJob(id, { queue })` (throws `FailedJobNotFoundError`), and the bull-board UI — `warlock add bull-board` (requires: ["queue"], so it installs queue first when missing) writes a `dashboard: { enabled, path, middleware }` block to `src/config/queue.ts`; a guard middleware is required in production or boot throws `QueueDashboardUnguardedError`. Manual `queueDashboard(server, { basePath, queues, middleware })` also available — needs `@bull-board/api` + `@bull-board/fastify`, loaded only on call, missing ones throw `QueueDashboardDependencyError`. Triggers: `failedJobs`, `retryFailedJob`, `queueDashboard`, `bull-board`, `dashboard.enabled`, `QueueDashboardUnguardedError`; "list failed jobs", "retry a failed job", "queue dashboard", "job admin UI". Skip: defining retries — the `define-jobs` topic.
-- [overview](@warlock.js/queue/overview/SKILL.md): @warlock.js/queue overview for orientation and choosing the focused skill for the task.
 - [queue-notifications](@warlock.js/queue/queue-notifications/SKILL.md): Send `@warlock.js/notifications` `.queue()` deliveries through BullMQ with `bullmqQueue({ queue?, attempts?, backoff? })` from `@warlock.js/notifications` — put it in the `queue` slot of `src/config/notifications.ts`; the queue workers deliver. Honours `SendOptions.delay` (number = seconds, or "10m"), retries a failing `channel.send`, and fails at once for a channel missing from the worker config. Triggers: `bullmqQueue`, `NotificationConfig.queue`, `QueuePackageNotInstalledError`; "queue notifications with BullMQ", "retry notification delivery", "delayed notification". Skip: the herald backend — the `queue-notifications` topic of the `warlock-js-notifications` skill.

package/package.json

@@ -11,8 +11,8 @@
   "peerDependencies": {
     "@bull-board/api": "^9.0.0",
     "@bull-board/fastify": "^9.0.0",
-    "@warlock.js/core": "5.34.0",
-    "@warlock.js/logger": "5.34.0"
+    "@warlock.js/core": "5.34.2",
+    "@warlock.js/logger": "5.34.2"
   },
   "peerDependenciesMeta": {
     "@bull-board/api": {
@@ -38,7 +38,7 @@
   ],
   "author": "hassanzohdy",
   "license": "MIT",
-  "version": "5.34.0",
+  "version": "5.34.2",
   "main": "./cjs/index.cjs",
   "module": "./esm/index.mjs",
   "types": "./esm/index.d.mts",

package/skills/index.md

@@ -8,7 +8,7 @@ Jobs are stored in Redis (default; not bundled), local SQLite without Redis, or
 
 ## The 80% path
 
-1. Orient with `overview.md`.
+1. Use this index for orientation and choose a task below.
 2. Configure `src/config/queue.ts` and register `queueConnector()`: `configure-queue.md`.
 3. Define and dispatch jobs (attempts, backoff, delay, priority, progress): `define-jobs.md`.
 4. Inspect and recover failures, optionally the bull-board dashboard: `manage-failed-jobs.md`.
@@ -24,3 +24,14 @@ Jobs are stored in Redis (default; not bundled), local SQLite without Redis, or
 - Do not value-import from client code.
 
 Use the `local-queues` topic for `driver: "local"` / `"memory"`, Node requirements, leases, crash recovery, and cancellation.
+
+## Mental model
+
+- **Job definition** — `defineJob({ name, handle })` registers a handler by name and returns a typed job.
+- **Dispatch** — `job.dispatch(payload, options)` stores the job in the configured backend. Any process with the same definition and a worker can run it.
+- **Worker** — one per queue name, started by `queueConnector()` (or `startWorkers()`); routes each job to the handler with its name.
+- **Shutdown** — workers stop taking jobs, active jobs finish (up to `workers.shutdownTimeout`), then connections close.
+
+## Not core's `Queue`
+
+`@warlock.js/core` exports `Queue` — an in-memory batcher that flushes items by size or interval inside one process. No storage, no retries, lost on exit. Use `@warlock.js/queue` when work must survive a restart or be retried.

package/skills/overview/SKILL.md

@@ -1,27 +0,0 @@
----
-name: overview
-description: "@warlock.js/queue overview for orientation and choosing the focused skill for the task."
----
-
-# `@warlock.js/queue` — overview
-
-Durable background jobs. Jobs are stored in **Redis** (default), local SQLite without Redis, or memory for tests, retried on failure, and processed by workers — inside the app process by default.
-
-## Mental model
-
-- **Job definition** — `defineJob({ name, handle })` registers a handler by name and returns a typed job.
-- **Dispatch** — `job.dispatch(payload, options)` stores the job in the configured backend. Any process with the same definition and a worker can run it.
-- **Worker** — one per queue name, started by `queueConnector()` (or `startWorkers()`); routes each job to the handler with its name.
-- **Shutdown** — workers stop taking jobs, active jobs finish (up to `workers.shutdownTimeout`), then connections close.
-
-## Not core's `Queue`
-
-`@warlock.js/core` exports `Queue` — an in-memory batcher that flushes items by size or interval inside one process. No storage, no retries, lost on exit. Use `@warlock.js/queue` when work must survive a restart or be retried.
-
-## Skills index
-
-- `local-queues` — SQLite/memory configuration, leases, crash recovery, cancellation.
-- `configure-queue` — `src/config/queue.ts`, `queueConnector()`, workers on/off, shutdown.
-- `define-jobs` — `defineJob`, `dispatch` options, retries, progress, `find`.
-- `manage-failed-jobs` — `failedJobs`, `retryFailedJob`, the bull-board dashboard.
-- `queue-notifications` — `bullmqQueue()` for notifications `.queue()`.