power-queues 2.0.17 → 2.0.19

package/README.md

@@ -1,151 +1,26 @@
-# power-queues - High‑Performance Redis Streams Queue Engine for Node.js
+# power-queues
 
-Production‑ready, lightweight and highly scalable
-queue engine built directly on **Redis Streams + Lua scripts**.
-It is designed for real‑world distributed systems that require **high
-throughput**, **idempotent task execution**, **automatic recovery**, and
-**predictable performance under heavy load**.
+## A lightweight, scalable, and high-performance queue engine for **Node.js** built on **Redis Streams** + **Lua scripts**.
 
-Unlike traditional Redis‑based queues that rely on lists or heavy
-abstractions, power-queues focuses on **low‑level control**, **atomic
-operations**, and **minimal overhead**, making it ideal for high‑load
-backends, microservices, schedulers, telemetry pipelines, and data
-processing clusters.
+The library is designed for real-world distributed systems that require high throughput, idempotent task execution, automatic recovery, and predictable performance under heavy load.
+
+Unlike traditional Redis-based queues that rely on lists or complex abstractions, **power-queues** focuses on low-level control, atomic operations, and minimal overhead, making it ideal for high-load backends, microservices, schedulers, telemetry pipelines, and data-processing clusters.
+
+Extends **power-redis**.
 
 <p align="center">
-  <img src="https://img.shields.io/badge/redis-streams-red?logo=redis" />
-  <img src="https://img.shields.io/badge/nodejs-queue-green?logo=node.js" />
-  <img src="https://img.shields.io/badge/typescript-ready-blue?logo=typescript" />
-  <img src="https://img.shields.io/badge/license-MIT-lightgrey" />
-  <img src="https://img.shields.io/badge/status-production-success" />
+	<img src="https://img.shields.io/badge/redis-streams-red?logo=redis" />
+	<img src="https://img.shields.io/badge/nodejs-queue-green?logo=node.js" />
+	<img src="https://img.shields.io/badge/typescript-ready-blue?logo=typescript" />
+	<img src="https://img.shields.io/badge/license-MIT-lightgrey" />
+	<img src="https://img.shields.io/badge/status-production-success" />
 </p>
 
----
-
 ## 📚 Documentation
 
 Full documentation is available here:  
 👉 **https://power-queues.docs.ihor.bielchenko.com**
 
----
-
-## 🚀 Key Features
-
-### **1. Ultra‑Fast Bulk XADD (Lua‑Powered)**
-
--   Adds thousands of messages per second using optimized Lua scripts.
--   Minimizes round‑trips to Redis.
--   Supports batching based on:
-    -   number of tasks
-    -   number of Redis arguments (safe upper bound)
--   Outperforms typical list‑based queues and generic abstractions.
-
----
-
-### **2. Built‑in Idempotent Workers**
-
-Every task can carry an `idemKey`, guaranteeing **exactly‑once
-execution** even under: - worker crashes
-- network interruptions
-- duplicate task submissions
-- process restarts
-
-Idempotency includes: - Lock key
-- Start key
-- Done key
-- TTL‑managed execution lock
-- Automatic release on failure
-- Heartbeat mechanism
-- Waiting on TTL for contended executions
-
-This makes the engine ideal for: - payment processing
-- external API calls
-- high‑value jobs
-- distributed pipelines
-
----
-
-### **3. Stuck Task Recovery (Advanced Stream Scanning)**
-
-If a worker crashes mid‑execution, power-queues automatically detects: -
-abandoned tasks
-- stalled locks
-- unfinished start keys
-
-The engine then recovers these tasks back to active processing safely
-and efficiently.
-
----
-
-### **4. High‑Throughput Workers**
-
--   Batch execution support
--   Parallel or sequential processing mode
--   Configurable worker loop interval
--   Individual and batch‑level error hooks
--   Safe retry flow with per‑task attempt counters
-
----
-
-### **5. Native DLQ (Dead‑Letter Queue)**
-
-When retries reach the configured limit: - the task is moved into
-`${stream}:dlq`
-- includes: payload, attempt count, job, timestamp, error text
-- fully JSON‑safe
-
-Perfect for monitoring or later re‑processing.
-
----
-
-### **6. Zero‑Overhead Serialization**
-
-power-queues uses: - safe JSON encoding
-- optional "flat" key/value task format
-- predictable and optimized payload transformation
-
-This keeps Redis memory layout clean and eliminates overhead.
-
----
-
-### **7. Complete Set of Lifecycle Hooks**
-
-You can extend any part of the execution flow:
-
--   `onSelected`
--   `onExecute`
--   `onSuccess`
--   `onError`
--   `onRetry`
--   `onBatchError`
--   `onReady`
-
-This allows full integration with: - monitoring systems
-- logging pipelines
-- external APM tools
-- domain logic
-
----
-
-### **8. Atomic Script Loading + NOSCRIPT Recovery**
-
-Scripts are: - loaded once
-- cached
-- auto‑reloaded if Redis restarts
-- executed safely via SHA‑based calls
-
-Ensures resilience in failover scenarios.
-
----
-
-### **9. Job Progress Tracking**
-
-Optional per‑job counters: - `job:ok` - `job:err` - `job:ready`
-
-Useful for UI dashboards and real‑time job progress visualization.
-
----
-
 ## 📦 Installation
 
 ``` bash
@@ -155,20 +30,20 @@ OR
 ```bash
 yarn add power-queues
 ```
----
 
-## 🧪 Quick Start
+## 🧪 Basic usage
 
 ``` ts
 const queue = new PowerQueues({
-  stream: "email",
-  group: "workers",
+	stream: 'email',
+	group: 'workers',
 });
 
 await queue.loadScripts(true);
 
-await queue.addTasks("email", [
-  { payload: { type: "welcome", userId: 42 } },
+await queue.addTasks('email', [
+	{ payload: { type: 'welcome', userId: 42 } },
+	{ payload: { type: 'hello', userId: 51 } }
 ]);
 ```
 
@@ -176,15 +51,13 @@ Worker:
 
 ``` ts
 class EmailWorker extends PowerQueues {
-  async onExecute(id, payload) {
-    await sendEmail(payload);
-  }
+	async onExecute(id, payload) {
+		await sendEmail(payload);
+	}
 }
 ```
 
----
-
-## ⚙ power-queues vs Existing Solutions
+## ⚖️ power-queues vs Existing Solutions
 
 |Feature               |power-queues    |BullMQ      |Bee-Queue   |Custom Streams|
 |----------------------|----------------|----------- |------------|--------------|
@@ -199,108 +72,128 @@ class EmailWorker extends PowerQueues {
 |Throughput            |🔥 Very high    |High        |Medium      |Depends       |
 |Overhead              |Low             |Medium      |Low         |Very high     |
 
-## 🛠 When to Choose power-queues
-
-Use this engine if you need:
+## 🚀 Key Features & Advantages
 
-### **✔ High performance under load**
+### ✔ Ultra‑Fast Bulk XADD (Lua‑Powered)
+- Adds thousands of messages per second using optimized Lua scripts.
+- Minimizes round‑trips to Redis.
+- Supports batching based on:
+	- number of tasks
+	- number of Redis arguments (safe upper bound)
+- Outperforms typical list‑based queues and generic abstractions.
 
-Millions of tasks per hour? No problem.
+### ✔ Built‑in Idempotent Workers
+Every task can carry an `idemKey`, guaranteeing **exactly‑once execution** even under:
+- worker crashes
+- network interruptions
+- duplicate task submissions
+- process restarts
 
-### **✔ Strong idempotent guarantees**
+Idempotency includes:
+- Lock key
+- Start key
+- Done key
+- TTL‑managed execution lock
+- Automatic release on failure
+- Heartbeat mechanism
+- Waiting on TTL for contended executions
 
-Exactly‑once processing for critical operations.
+This makes the engine ideal for:
+- payment processing
+- external API calls
+- high‑value jobs
+- distributed pipelines
 
-### **✔ Low‑level control without heavy abstractions**
+### ✔ Stuck Task Recovery (Advanced Stream Scanning)
+If a worker crashes mid‑execution, **power-queues** automatically detects:
+- abandoned tasks
+- stalled locks
+- unfinished start keys
 
-No magic, no hidden states - everything is explicit.
+The engine then recovers these tasks back to active processing safely
+and efficiently.
 
-### **✔ Predictable behavior in distributed environments**
+### ✔ High‑Throughput Workers
+- Batch execution support
+- Parallel or sequential processing mode
+- Configurable worker loop interval
+- Individual and batch‑level error hooks
+- Safe retry flow with per‑task attempt counters
 
-Even with frequent worker restarts.
+### ✔ Native DLQ (Dead‑Letter Queue)
+When retries reach the configured limit:
+- the task is moved into `${stream}:dlq`
+- includes: payload, attempt count, job, timestamp, error text
+- fully JSON‑safe
 
-### **✔ Production‑grade reliability**
+Perfect for monitoring or later re‑processing.
 
-Backpressure, recovery, retries, dead-lettering - all included.
+### ✔ Zero‑Overhead Serialization
+**power-queues** uses:
+- safe JSON encoding
+- optional "flat" key/value task format
+- predictable and optimized payload transformation
 
----
+This keeps Redis memory layout clean and eliminates overhead.
 
-## 🏗️ Project Structure & Architecture
+### ✔ Complete Set of Lifecycle Hooks
+You can extend any part of the execution flow:
+- `onSelected`
+- `onExecute`
+- `onSuccess`
+- `onError`
+- `onRetry`
+- `onBatchError`
+- `onReady`
+
+This allows full integration with:
+- monitoring systems
+- logging pipelines
+- external APM tools
+- domain logic
 
--   Redis Streams for messaging
--   Lua scripts for atomic operations
--   JavaScript/TypeScript API
--   Full worker lifecycle management
--   Configurable backpressure & contention handling
--   Optional job‑level progress tracking
+### ✔ Atomic Script Loading + NOSCRIPT Recovery
+Scripts are:
+- loaded once
+- cached
+- auto‑reloaded if Redis restarts
+- executed safely via SHA‑based calls
 
----
+Ensures resilience in failover scenarios.
 
-## 🧩 Extensibility
+### ✔ Job Progress Tracking
+Optional per‑job counters:
+- `job:ok`
+- `job:err`
+- `job:ready`
 
-power-queues is ideal for building:
+Useful for UI dashboards and real‑time job progress visualization.
 
--   task schedulers
--   distributed cron engines
--   ETL pipelines
--   telemetry processors
--   notification workers
--   device monitoring systems
--   AI job pipelines
--   high-frequency background jobs
+## 🧩 Extensibility
 
----
+**power-queues** is ideal for building:
+- task schedulers
+- distributed cron engines
+- ETL pipelines
+- telemetry processors
+- notification workers
+- device monitoring systems
+- AI job pipelines
+- high-frequency background jobs
 
 ## 🧱 Reliability First
 
 Every part of the engine is designed to prevent:
-
--   double execution
--   stuck tasks
--   orphan locks
--   lost messages
--   zombie workers
--   script desynchronization
+- double execution
+- stuck tasks
+- orphan locks
+- lost messages
+- zombie workers
+- script desynchronization
 
 The heartbeat + TTL strategy guarantees that no task is "lost" even in
 chaotic cluster environments.
 
----
-
-## 🏷️ SEO‑Optimized Keywords (Non‑Spam)
-
-power-queues is relevant for:
-
--   Redis Streams queue engine
--   Node.js stream-based queue
--   idempotent task processing
--   high‑performance job queue for Node.js
--   Redis Lua queue
--   distributed worker engine
--   scalable background jobs
--   enterprise-grade Redis queue
--   microservices task runner
--   fault-tolerant queue for Node.js
-
----
-
 ## 📝 License
 
 MIT - free for commercial and private use.
-
----
-
-## ⭐ Why This Project Exists
-
-Most Node.js queue libraries are: - too slow
-- too abstract
-- not idempotent
-- not safe for financial or mission‑critical workloads
-
-power-queues was built to solve real production problems where: -
-*duplicate tasks cost money*,
-- *workers are unstable*,
-- *tasks must survive restarts*,
-- *performance matters at scale*.
-
-If these things matter to you - this engine will feel like home.

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "power-queues",
-	"version": "2.0.17",
+	"version": "2.0.19",
 	"description": "High-performance Redis Streams queue for Node.js with Lua-powered bulk XADD, idempotent workers, heartbeat locks, stuck-task recovery, retries, DLQ, and distributed processing.",
 	"author": "ihor-bielchenko",
 	"license": "MIT",
@@ -82,7 +82,7 @@
 	],
 	"dependencies": {
 		"full-utils": "^2.0.5",
-		"power-redis": "^2.0.8",
+		"power-redis": "^2.0.10",
 		"uuid": "^13.0.0"
 	}
 }