@travetto/worker 3.0.2 → 3.1.0-rc.0

package/README.md

@@ -1,6 +1,7 @@
 <!-- This file was generated by @travetto/doc and should not be modified directly -->
-<!-- Please modify https://github.com/travetto/travetto/tree/main/module/worker/DOC.ts and execute "npx trv doc" to rebuild -->
+<!-- Please modify https://github.com/travetto/travetto/tree/main/module/worker/DOC.tsx and execute "npx trv doc" to rebuild -->
 # Worker
+
 ## Process management utilities, with a focus on inter-process communication
 
 **Install: @travetto/worker**
@@ -15,9 +16,9 @@ yarn add @travetto/worker
 This module provides the necessary primitives for handling dependent workers.  A worker can be an individual actor or could be a pool of workers. Node provides ipc (inter-process communication) functionality out of the box. This module builds upon that by providing enhanced event management, richer process management, as well as constructs for orchestrating a conversation between two processes.
 
 ## Execution Pools
-With respect to managing multiple executions, [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31) is provided to allow for concurrent operation, and processing of jobs concurrently.  To manage the flow of jobs, there are various [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) implementation that allow for a wide range of use cases.
+With respect to managing multiple executions, [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31) is provided to allow for concurrent operation, and processing of jobs concurrently.  To manage the flow of jobs, there are various [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) implementation that allow for a wide range of use cases. 
 
-The only provided [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) is the [IterableWorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/iterable.ts#L11) which supports all `Iterable` and `Iterator` sources.  Additionally, the module provides [ManualAsyncIterator](https://github.com/travetto/travetto/tree/main/module/worker/src/input/async-iterator.ts#L6) which allows for manual control of iteration, which is useful for event driven work loads.
+The only provided [WorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/types.ts#L4) is the [IterableWorkSet](https://github.com/travetto/travetto/tree/main/module/worker/src/input/iterable.ts#L11) which supports all `Iterable` and `Iterator` sources.  Additionally, the module provides [ManualAsyncIterator](https://github.com/travetto/travetto/tree/main/module/worker/src/input/async-iterator.ts#L6) which allows for manual control of iteration, which is useful for event driven work loads. 
 
 Below is a pool that will convert images on demand, while queuing as needed.
 
@@ -71,8 +72,7 @@ export class ImageCompressor extends WorkPool<string> {
 Once a pool is constructed, it can be shutdown by calling the `.shutdown()` method, and awaiting the result.
 
 ## IPC Support
-
-Within the `comm` package, there is support for two primary communication elements: [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) and [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10).  Usually [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) indicates it is the owner of the sub process.  [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) indicates that it has been created/spawned/forked by the parent and will communicate back to it's parent.  This generally means that a [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) can be destroyed (i.e. killing the subprocess) where a [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) can only exit the process, but the channel cannot be destroyed.
+Within the `comm` package, there is support for two primary communication elements: [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) and [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10).  Usually [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) indicates it is the owner of the sub process. [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) indicates that it has been created/spawned/forked by the parent and will communicate back to it's parent. This generally means that a [ParentCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/parent.ts#L10) can be destroyed (i.e. killing the subprocess) where a [ChildCommChannel](https://github.com/travetto/travetto/tree/main/module/worker/src/comm/child.ts#L6) can only exit the process, but the channel cannot be destroyed.
 
 ### IPC as a Worker
 A common pattern is to want to model a sub process as a worker, to be a valid candidate in a [WorkPool](https://github.com/travetto/travetto/tree/main/module/worker/src/pool.ts#L31).  The [WorkUtil](https://github.com/travetto/travetto/tree/main/module/worker/src/util.ts#L14) class provides a utility to facilitate this desire.
@@ -125,7 +125,7 @@ import { WorkPool, WorkUtil, IterableWorkSet } from '@travetto/worker';
 
 export async function main(): Promise<void> {
   const pool = new WorkPool(() =>
-    WorkUtil.spawnedWorker<{ data: string }, number>(
+    WorkUtil.spawnedWorker<{ data: number }, number>(
       () => ExecUtil.spawn('trv', ['main', '@travetto/worker/doc/spawned.ts']),
       ch => ch.once('ready'), // Wait for child to indicate it is ready
       async (channel, inp) => {
@@ -135,9 +135,9 @@ export async function main(): Promise<void> {
         const { data } = await res; // Get answer
         console.log('Request complete', { input: inp, output: data });
 
-        if (!(`${inp + inp}` === data)) {
+        if (!(inp + inp === data)) {
           // Ensure the answer is double the input
-          throw new Error('Did not get the double');
+          throw new Error(`Did not get the double: inp=${inp}, data=${data}`);
         }
       }
     )
@@ -159,8 +159,9 @@ export async function main(): Promise<void> {
 
   exec.send('ready'); // Indicate the child is ready to receive requests
 
-  const heartbeat = (): void => { timers.setTimeout(5000).then(heartbeat); }; // Keep-alive
-  heartbeat();
+  for await (const _ of timers.setInterval(5000)) {
+    // Keep-alive
+  }
 }
 ```
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@travetto/worker",
-  "version": "3.0.2",
+  "version": "3.1.0-rc.0",
   "description": "Process management utilities, with a focus on inter-process communication",
   "keywords": [
     "exec",
@@ -25,7 +25,7 @@
     "directory": "module/worker"
   },
   "dependencies": {
-    "@travetto/base": "^3.0.2",
+    "@travetto/base": "^3.1.0-rc.0",
     "generic-pool": "^3.9.0"
   },
   "travetto": {

package/src/input/async-iterator.ts

@@ -27,7 +27,7 @@ export class ManualAsyncIterator<X> implements AsyncIterator<X>, AsyncIterable<X
    * Wait for next event to fire
    */
   async next(): Promise<IteratorResult<X>> {
-    if (!this.#done && !this.#queue.length) {
+    while (!this.#done && !this.#queue.length) {
       await this.#ready;
       this.#ready = Util.resolvablePromise();
     }

package/src/pool.ts

@@ -30,7 +30,8 @@ type WorkCompletionEvent<X> = { idx: number, total?: number, value: X };
  */
 export class WorkPool<X> {
 
-  static DEFAULT_SIZE = Math.min(os.cpus().length - 1, 4);
+  static MAX_SIZE = os.cpus().length - 1;
+  static DEFAULT_SIZE = Math.min(WorkPool.MAX_SIZE, 4);
 
   /**
    * Generic-pool pool