multi-tasks 3.5.1 → 3.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -18
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -170,34 +170,26 @@ multiTasks({
|
|
|
170
170
|
taskRootFolder: './my-tasks',
|
|
171
171
|
taskId: 'demo',
|
|
172
172
|
task_storage: 'memory', //'file' (default) | 'memory'
|
|
173
|
+
onFinish: (report, helper) => {
|
|
174
|
+
helper.foreachResult((count, task, result, succeeded) => {
|
|
175
|
+
//collect / batch-insert / write out your results here
|
|
176
|
+
});
|
|
177
|
+
},
|
|
173
178
|
});
|
|
174
179
|
```
|
|
175
180
|
|
|
176
181
|
- **`file`** (default) — every state change hits disk immediately; resume anytime.
|
|
177
|
-
- **`memory`** — task states, results
|
|
182
|
+
- **`memory`** — task states, results and logs stay in the master's memory; nothing per-task is written to disk, so `onFinish(report, helper)` is the only exit for your results — collect them with `helper.foreachResult` (or `foreachSuccResult`/`foreachErrorResult`) as above; if you need per-task files, write them in bulk there. `resume`/`retry_fails`/`restart` throw — a killed run is simply gone.
|
|
178
183
|
|
|
179
|
-
Benchmark — 30,000 tiny tasks × 28 workers
|
|
184
|
+
Benchmark — 30,000 tiny tasks × 28 workers:
|
|
180
185
|
|
|
181
186
|

|
|
182
187
|
|
|
183
|
-
**Getting results out in `memory` mode**: since nothing per-task hits disk, `onFinish(report, helper)` is the one and only exit for your data — iterate everything with `helper.foreachResult` (or `foreachSuccResult`/`foreachErrorResult`) and persist it yourself in one batch if you need it kept:
|
|
184
|
-
|
|
185
|
-
```javascript
|
|
186
|
-
onFinish: (report, helper) => {
|
|
187
|
-
helper.foreachResult((count, task, result, succeeded) => {
|
|
188
|
-
//collect / batch-insert / write out — all results are here in memory
|
|
189
|
-
});
|
|
190
|
-
},
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
If you need per-task files anyway, write them in bulk from `onFinish` — do **not** `writeFileSync` inside `processTask`: in benchmarks (30,000 tasks × 28 workers) per-task self-written files made the run ~3.3× slower than pure `memory` (27.7s vs 8.4s), because disk writes still dominate even without the library's bookkeeping. (That said, even the slow do-it-yourself variant beats `file` mode's ~195s — its cost is the resumable state machine, not the writes alone.)
|
|
194
|
-
|
|
195
188
|
Caveats of `memory` mode:
|
|
196
189
|
|
|
197
|
-
- **Monitoring**: [multi-tasks-monitor](#real-time-monitoring) reads the task folder, so it has nothing to show for a `memory` run
|
|
198
|
-
- **Worker crashes**: without `maxTaskRetries`, a crashed worker's in-flight task is
|
|
199
|
-
- **Dynamic tasks**:
|
|
200
|
-
- **Dynamic tasks with `autoCloseAfterCompletion`**: in `memory` mode the master pops straight from the in-memory queue — there is no on-disk re-read fallback like in `file` mode, and no settle delay before killing an idle worker. A task another worker just sent via `helper.createNewTasks` may still be in IPC transit when the master decides the queue is empty, so it can miss the dispatch window and the workers get closed. If your workers create tasks dynamically, keep `autoCloseAfterCompletion` at its default `false`.
|
|
190
|
+
- **Monitoring**: [multi-tasks-monitor](#real-time-monitoring) reads the task folder, so it has nothing to show for a `memory` run.
|
|
191
|
+
- **Worker crashes**: without `maxTaskRetries`, a crashed worker's in-flight task is lost for good (no `resume` to pick it up).
|
|
192
|
+
- **Dynamic tasks**: tasks sent via `helper.createNewTasks` but not yet applied are lost if the master is killed. And keep `autoCloseAfterCompletion` at its default `false` when workers create tasks dynamically — in-transit tasks can miss the dispatch window and the workers get closed early.
|
|
201
193
|
|
|
202
194
|
### Broadcasting events between workers:
|
|
203
195
|
|
|
@@ -297,6 +289,7 @@ npx multi-tasks-monitor --task-dir "D:\tasks\multitasks20260806..." --port 3777
|
|
|
297
289
|
|
|
298
290
|
### Changelog:
|
|
299
291
|
|
|
292
|
+
- 3.5.2 Fix Readme
|
|
300
293
|
- 3.5.1 Fix bench-storage.svg
|
|
301
294
|
- 3.5.0 Add `task_storage: 'memory'` mode (~17x faster, no resume)
|
|
302
295
|
- 3.4.1 Rewrite README: simpler structure, all examples verified by test cases
|