@travetto/worker 3.0.2 → 3.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -8
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
<!-- This file was generated by @travetto/doc and should not be modified directly -->
|
|
2
|
-
<!-- Please modify https://github.com/travetto/travetto/tree/main/module/worker/DOC.
|
|
2
|
+
<!-- Please modify https://github.com/travetto/travetto/tree/main/module/worker/DOC.tsx and execute "npx trv doc" to rebuild -->
|
|
3
3
|
# Worker
|
|
4
|
+
|
|
4
5
|
## Process management utilities, with a focus on inter-process communication
|
|
5
6
|
|
|
6
7
|
**Install: @travetto/worker**
|
|
@@ -15,9 +16,9 @@ yarn add @travetto/worker
|
|
|
15
16
|
This module provides the necessary primitives for handling dependent workers. A worker can be an individual actor or could be a pool of workers. Node provides ipc (inter-process communication) functionality out of the box. This module builds upon that by providing enhanced event management, richer process management, as well as constructs for orchestrating a conversation between two processes.
|
|
16
17
|
|
|
17
18
|
## Execution Pools
|
|
18
|
-
With respect to managing multiple executions, [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31) is provided to allow for concurrent operation, and processing of jobs concurrently. To manage the flow of jobs, there are various [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) implementation that allow for a wide range of use cases.
|
|
19
|
+
With respect to managing multiple executions, [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31) is provided to allow for concurrent operation, and processing of jobs concurrently. To manage the flow of jobs, there are various [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) implementation that allow for a wide range of use cases.
|
|
19
20
|
|
|
20
|
-
The only provided [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) is the [IterableWorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/iterable.ts#L11) which supports all `Iterable` and `Iterator` sources. Additionally, the module provides [ManualAsyncIterator](https://github.com/travetto/travetto/tree/main/module/worker/src/input/async-iterator.ts#L6) which allows for manual control of iteration, which is useful for event driven work loads.
|
|
21
|
+
The only provided [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) is the [IterableWorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/iterable.ts#L11) which supports all `Iterable` and `Iterator` sources. Additionally, the module provides [ManualAsyncIterator](https://github.com/travetto/travetto/tree/main/module/worker/src/input/async-iterator.ts#L6) which allows for manual control of iteration, which is useful for event driven work loads.
|
|
21
22
|
|
|
22
23
|
Below is a pool that will convert images on demand, while queuing as needed.
|
|
23
24
|
|
|
@@ -71,8 +72,7 @@ export class ImageCompressor extends WorkPool<string> {
|
|
|
71
72
|
Once a pool is constructed, it can be shutdown by calling the `.shutdown()` method, and awaiting the result.
|
|
72
73
|
|
|
73
74
|
## IPC Support
|
|
74
|
-
|
|
75
|
-
Within the `comm` package, there is support for two primary communication elements: [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) and [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10). Usually [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) indicates it is the owner of the sub process. [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) indicates that it has been created/spawned/forked by the parent and will communicate back to it's parent. This generally means that a [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) can be destroyed (i.e. killing the subprocess) where a [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) can only exit the process, but the channel cannot be destroyed.
|
|
75
|
+
Within the `comm` package, there is support for two primary communication elements: [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) and [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10). Usually [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) indicates it is the owner of the sub process. [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) indicates that it has been created/spawned/forked by the parent and will communicate back to it's parent. This generally means that a [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) can be destroyed (i.e. killing the subprocess) where a [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) can only exit the process, but the channel cannot be destroyed.
|
|
76
76
|
|
|
77
77
|
### IPC as a Worker
|
|
78
78
|
A common pattern is to want to model a sub process as a worker, to be a valid candidate in a [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31). The [WorkUtil](https://github.com/travetto/travetto/tree/main/module/worker/src/util.ts#L14) class provides a utility to facilitate this desire.
|
|
@@ -125,7 +125,7 @@ import { WorkPool, WorkUtil, IterableWorkSet } from '@travetto/worker';
|
|
|
125
125
|
|
|
126
126
|
export async function main(): Promise<void> {
|
|
127
127
|
const pool = new WorkPool(() =>
|
|
128
|
-
WorkUtil.spawnedWorker<{ data:
|
|
128
|
+
WorkUtil.spawnedWorker<{ data: number }, number>(
|
|
129
129
|
() => ExecUtil.spawn('trv', ['main', '@travetto/worker/doc/spawned.ts']),
|
|
130
130
|
ch => ch.once('ready'), // Wait for child to indicate it is ready
|
|
131
131
|
async (channel, inp) => {
|
|
@@ -135,9 +135,9 @@ export async function main(): Promise<void> {
|
|
|
135
135
|
const { data } = await res; // Get answer
|
|
136
136
|
console.log('Request complete', { input: inp, output: data });
|
|
137
137
|
|
|
138
|
-
if (!(
|
|
138
|
+
if (!(inp + inp === data)) {
|
|
139
139
|
// Ensure the answer is double the input
|
|
140
|
-
throw new Error(
|
|
140
|
+
throw new Error(`Did not get the double: inp=${inp}, data=${data}`);
|
|
141
141
|
}
|
|
142
142
|
}
|
|
143
143
|
)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@travetto/worker",
|
|
3
|
-
"version": "3.0.
|
|
3
|
+
"version": "3.0.3",
|
|
4
4
|
"description": "Process management utilities, with a focus on inter-process communication",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"exec",
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"directory": "module/worker"
|
|
26
26
|
},
|
|
27
27
|
"dependencies": {
|
|
28
|
-
"@travetto/base": "^3.0.
|
|
28
|
+
"@travetto/base": "^3.0.3",
|
|
29
29
|
"generic-pool": "^3.9.0"
|
|
30
30
|
},
|
|
31
31
|
"travetto": {
|