multi-tasks 3.5.0 → 3.5.2

package/README.md

@@ -4,7 +4,7 @@ Run large batches of tasks in parallel across multiple Node.js processes, with z
 
 **New in 3.5.0 — `task_storage: 'memory'`**: keep all task state in the master process's memory instead of on disk. Same API, one config field — and ~17× faster on huge batches of small tasks. The trade-off: no resume. See [Task storage modes](#task-storage-modes).
 
-![task_storage benchmark: file mode 195.1s vs memory mode 11.2s — 17.4x faster](bench-storage.svg)
+![task_storage benchmark: file mode 195.1s vs memory mode 11.2s — 17.4x faster](https://cdn.jsdelivr.net/npm/multi-tasks@latest/bench-storage.svg)
 
 ### Install:
 
@@ -170,34 +170,26 @@ multiTasks({
     taskRootFolder: './my-tasks',
     taskId: 'demo',
     task_storage: 'memory', //'file' (default) | 'memory'
+    onFinish: (report, helper) => {
+        helper.foreachResult((count, task, result, succeeded) => {
+            //collect / batch-insert / write out your results here
+        });
+    },
 });
 ```
 
 - **`file`** (default) — every state change hits disk immediately; resume anytime.
-- **`memory`** — task states, results, logs and user messages live in the master process's memory only; no per-task files are ever written. The task folder is still created with the usual subfolders (`progress`/`results`/`logs` stay empty), holding only `task_config.json` and `.sys/run.log`. Dramatically faster for large batches of small tasks — about 20x faster in benchmarks. Results are still available through `onFinish(report, helper)`. `resume`/`retry_fails`/`restart` are not supported for this mode and throw when called — a killed run is simply gone.
-
-Benchmark — 30,000 tiny tasks × 28 workers, average of 3 runs on the same machine (17–27× across repeated benchmark sessions):
-
-![task_storage benchmark: file mode 195.1s vs memory mode 11.2s — 17.4x faster](bench-storage.svg)
+- **`memory`** — task states, results and logs stay in the master's memory; nothing per-task is written to disk, so `onFinish(report, helper)` is the only exit for your results — collect them with `helper.foreachResult` (or `foreachSuccResult`/`foreachErrorResult`) as above; if you need per-task files, write them in bulk there. `resume`/`retry_fails`/`restart` throw — a killed run is simply gone.
 
-**Getting results out in `memory` mode**: since nothing per-task hits disk, `onFinish(report, helper)` is the one and only exit for your data — iterate everything with `helper.foreachResult` (or `foreachSuccResult`/`foreachErrorResult`) and persist it yourself in one batch if you need it kept:
-
-```javascript
-onFinish: (report, helper) => {
-    helper.foreachResult((count, task, result, succeeded) => {
-        //collect / batch-insert / write out — all results are here in memory
-    });
-},
-```
+Benchmark — 30,000 tiny tasks × 28 workers:
 
-If you need per-task files anyway, write them in bulk from `onFinish` — do **not** `writeFileSync` inside `processTask`: in benchmarks (30,000 tasks × 28 workers) per-task self-written files made the run ~3.3× slower than pure `memory` (27.7s vs 8.4s), because disk writes still dominate even without the library's bookkeeping. (That said, even the slow do-it-yourself variant beats `file` mode's ~195s — its cost is the resumable state machine, not the writes alone.)
+![task_storage benchmark: file mode 195.1s vs memory mode 11.2s — 17.4x faster](https://cdn.jsdelivr.net/npm/multi-tasks@latest/bench-storage.svg)
 
 Caveats of `memory` mode:
 
-- **Monitoring**: [multi-tasks-monitor](#real-time-monitoring) reads the task folder, so it has nothing to show for a `memory` run (no task states/results on disk).
-- **Worker crashes**: without `maxTaskRetries`, a crashed worker's in-flight task is not retried — with `file` it stays in `running` until a later `resume` picks it up; with `memory` it is lost for good.
-- **Dynamic tasks**: with `memory`, `helper.createNewTasks` reaches the master over IPC; if the master is killed before applying them, tasks created but not yet applied are lost. (`file` mode writes them to disk from the worker directly.)
-- **Dynamic tasks with `autoCloseAfterCompletion`**: in `memory` mode the master pops straight from the in-memory queue — there is no on-disk re-read fallback like in `file` mode, and no settle delay before killing an idle worker. A task another worker just sent via `helper.createNewTasks` may still be in IPC transit when the master decides the queue is empty, so it can miss the dispatch window and the workers get closed. If your workers create tasks dynamically, keep `autoCloseAfterCompletion` at its default `false`.
+- **Monitoring**: [multi-tasks-monitor](#real-time-monitoring) reads the task folder, so it has nothing to show for a `memory` run.
+- **Worker crashes**: without `maxTaskRetries`, a crashed worker's in-flight task is lost for good (no `resume` to pick it up).
+- **Dynamic tasks**: tasks sent via `helper.createNewTasks` but not yet applied are lost if the master is killed. And keep `autoCloseAfterCompletion` at its default `false` when workers create tasks dynamically — in-transit tasks can miss the dispatch window and the workers get closed early.
 
 ### Broadcasting events between workers:
 
@@ -297,6 +289,8 @@ npx multi-tasks-monitor --task-dir "D:\tasks\multitasks20260806..." --port 3777
 
 ### Changelog:
 
+- 3.5.2 Fix Readme
+- 3.5.1 Fix bench-storage.svg
 - 3.5.0 Add `task_storage: 'memory'` mode (~17x faster, no resume)
 - 3.4.1 Rewrite README: simpler structure, all examples verified by test cases
 - 3.4.0 Show per-worker load (ELU/CPU/memory) under progressBar; stability fixes

package/bench-storage.svg

@@ -0,0 +1,17 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="720" height="252" viewBox="0 0 720 252" font-family="Segoe UI, Helvetica, Arial, sans-serif">
+  <rect x="1" y="1" width="718" height="250" rx="8" fill="#ffffff" stroke="#dadce0"/>
+  <text x="24" y="38" font-size="17" font-weight="600" fill="#202124">task_storage benchmark — 30,000 tasks × 28 workers</text>
+  <text x="24" y="60" font-size="12" fill="#5f6368">wall-clock time of a full run, average of 3 runs · lower is better</text>
+
+  <text x="112" y="114" font-size="14" fill="#202124" text-anchor="end">file</text>
+  <rect x="120" y="94" width="420" height="34" rx="3" fill="#9aa0a6"/>
+  <text x="548" y="116" font-size="14" fill="#202124">195.1s · 154 tasks/s</text>
+
+  <text x="112" y="172" font-size="14" fill="#202124" text-anchor="end">memory</text>
+  <rect x="120" y="152" width="24" height="34" rx="3" fill="#188038"/>
+  <text x="152" y="174" font-size="14" fill="#202124">11.2s · 2,676 tasks/s</text>
+  <text x="360" y="174" font-size="15" font-weight="700" fill="#188038">17.4× faster</text>
+
+  <line x1="120" y1="208" x2="540" y2="208" stroke="#e8eaed"/>
+  <text x="24" y="232" font-size="11" fill="#5f6368">per-round walls — file: 202.7 / 193.6 / 188.9s · memory: 11.3 / 11.1 / 11.3s · Windows 11, SSD · script: test-manual/bench-storage.js</text>
+</svg>

package/package.json

@@ -1,12 +1,13 @@
 {
   "name": "multi-tasks",
-  "version": "3.5.0",
+  "version": "3.5.2",
   "description": "Multi-process task scheduling based on Node.js cluster, with crash resume and failed-task restart support",
   "main": "index.js",
   "files": [
     "index.js",
     "workers",
-    "utils/*.js"
+    "utils/*.js",
+    "bench-storage.svg"
   ],
   "repository": {
     "type": "git",