@shutterstock/p-map-iterable 1.0.11 → 1.1.1

package/README.md

@@ -1,25 +1,116 @@
-[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) [![API Docs](https://img.shields.io/badge/API%20Docs-View%20Here-blue)](https://tech.shutterstock.com/p-map-iterable/) [![Build - CI](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml) [![Package and Publish](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml) [![Publish Docs](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml)
+[![npm (scoped)](https://img.shields.io/npm/v/%40shutterstock/p-map-iterable)](https://www.npmjs.com/package/@shutterstock/p-map-iterable) [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) [![API Docs](https://img.shields.io/badge/API%20Docs-View%20Here-blue)](https://tech.shutterstock.com/p-map-iterable/) [![Build - CI](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml) [![Package and Publish](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml) [![Publish Docs](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml)
 
 # Overview
 
-`@shutterstock/p-map-iterable` provides several classes that allow processing results of `p-map`-style mapper functions by iterating the results as they are completed, with back pressure to limit the number of items that are processed ahead of the consumer.
+`@shutterstock/p-map-iterable` provides several classes that allow processing results of `p-map`-style mapper functions by iterating the results as they are completed, with backpressure to limit the number of items that are processed ahead of the consumer.
 
 A common use case for `@shutterstock/p-map-iterable` is as a "prefetcher" that will fetch, for example, AWS S3 files in an AWS Lambda function. By prefetching large files the consumer is able to use 100% of the paid-for Lambda CPU time for the JS thread, rather than waiting idle while the next file is fetched. The backpressure (set by `maxUnread`) prevents the prefetcher from consuming unlimited memory or disk space by racing ahead of the consumer.
 
 These classes will typically be helpful in batch or queue consumers, not as much in request/response services.
 
-# Example Usage Scenario
+# Example Usage Scenarios
 
-- AWS Lambda function for Kafka or Kinesis stream processing
-- Each invocation has a batch of, say, 100 records
-- Each record points to a file on S3 that needs to be parsed, processed, then written back to S3
-- The average file size is 50 MB, so these will take a few seconds to fetch and write
-- The Lambda function has 1,769 MB of RAM which gives it 1 vCPU, allowing the JS thread to run at roughly 100% speed of 1 core
-- If the code never waits to fetch or write files then the Lambda function will be able to use 100% of the paid-for CPU time
-- If there is no back pressure then reading 100 * 50 MB files would fill up the default Lambda temp disk space of 512 MB OR would consume 5 GB of memory, which would cause the Lambda function to fail
-- We can use `IterableMapper` to fetch up to 5 of the next files while the current file is being processed, then pause until the current file is processed and the next file is consumed
-- We can also use `IterableQueueMapper.enqueue()` to write the files back to S3 without waiting for them to finish, unless we get more than, say, 3-4 files being uploaded at once, at which point we can pause until the current file is uploaded before we allow queuing another file for upload
-- In the rare case that 3-4 files are uploading, but not yet finished, we would block on `.enqueue()` and not consume much CPU while waiting for at least 1 upload to finish
+## Typical Processing Loop without `IterableMapper`
+
+```typescript
+const source = new SomeSource();
+const sourceIds = [1, 2,... 1000];
+const sink = new SomeSink();
+for (const sourceId of sourceIds) {
+  const item = await source.read(sourceId);     // takes 300 ms of I/O wait, no CPU
+  const outputItem = doSomeOperation(item);     // takes 20 ms of CPU
+  await sink.write(outputItem);                 // takes 500 ms of I/O wait, no CPU
+}
+```
+
+Each iteration takes 820ms total, but we waste time waiting for I/O. We could prefetch the next read (300ms) while processing (20ms) and writing (500ms), without changing the order of reads or writes.
+
+## Using `IterableMapper` as Prefetcher with Blocking Sequential Writes
+
+`concurrency: 1` on the prefetcher preserves the order of the reads and and writes are sequential and blocking (unchanged).
+
+```typescript
+const source = new SomeSource();
+const sourceIds = [1, 2,... 1000];
+// Pre-reads up to 8 items serially and releases in sequential order
+const sourcePrefetcher = new IterableMapper(sourceIds,
+  async (sourceId) => source.read(sourceId),
+  { concurrency: 1, maxUnread: 10 }
+);
+const sink = new SomeSink();
+for await (const item of sourcePrefetcher) {    // may not block for fast sources
+  const outputItem = doSomeOperation(item);     // takes 20 ms of CPU
+  await sink.write(outputItem);                 // takes 500 ms of I/O wait, no CPU
+}
+```
+
+This reduces iteration time to 520ms by overlapping reads with processing/writing.
+
+## Using `IterableMapper` as Prefetcher with Background Sequential Writes with `IterableQueueMapperSimple`
+
+`concurrency: 1` on the prefetcher preserves the order of the reads.
+`concurrency: 1` on the flusher preserves the order of the writes, but allows the loop to iterate while last write is completing.
+
+```typescript
+const source = new SomeSource();
+const sourceIds = [1, 2,... 1000];
+const sourcePrefetcher = new IterableMapper(sourceIds,
+  async (sourceId) => source.read(sourceId),
+  { concurrency: 1, maxUnread: 10 }
+);
+const sink = new SomeSink();
+const flusher = new IterableQueueMapperSimple(
+  async (outputItem) => sink.write(outputItem),
+  { concurrency: 1 }
+);
+for await (const item of sourcePrefetcher) {    // may not block for fast sources
+  const outputItem = doSomeOperation(item);     // takes 20 ms of CPU
+  await flusher.enqueue(outputItem);            // will periodically block for portion of write time
+}
+// Wait for all writes to complete
+await flusher.onIdle();
+// Check for errors
+if (flusher.errors.length > 0) {
+// ...
+}
+```
+
+This reduces iteration time to about `max((max(readTime, writeTime) - cpuOpTime, cpuOpTime))`
+by overlapping reads and writes with the CPU processing step.
+In this contrived example, the loop time is reduced to 500ms - 20ms = 480ms.
+In cases where the CPU usage time is higher, the impact can be greater.
+
+## Using `IterableMapper` as Prefetcher with Out of Order Reads and Background Out of Order Writes with `IterableQueueMapperSimple`
+
+For maximum throughput, allow out of order reads and writes with
+`IterableQueueMapper` (to iterate results with backpressure when too many unread items) or
+`IterableQueueMapperSimple` (to handle errors at end without custom iteration and applying backpressure to block further enqueues when `concurrency` items are in process):
+
+```typescript
+const source = new SomeSource();
+const sourceIds = [1, 2,... 1000];
+const sourcePrefetcher = new IterableMapper(sourceIds,
+  async (sourceId) => source.read(sourceId),
+  { concurrency: 10, maxUnread: 20 }
+);
+const sink = new SomeSink();
+const flusher = new IterableQueueMapperSimple(
+  async (outputItem) => sink.write(outputItem),
+  { concurrency: 10 }
+);
+for await (const item of sourcePrefetcher) {    // typically will not block
+  const outputItem = doSomeOperation(item);     // takes 20 ms of CPU
+  await flusher.enqueue(outputItem);            // typically will not block
+}
+// Wait for all writes to complete
+await flusher.onIdle();
+// Check for errors
+if (flusher.errors.length > 0) {
+ // ...
+}
+```
+
+This reduces iteration time to about 20ms by overlapping reads and writes with the CPU processing step. In this contrived (but common) example we would get a 41x improvement in throughput, removing 97.5% of the time to process each item and fully utilizing the CPU time available in the JS event loop.
 
 # Getting Started
 
@@ -32,7 +123,10 @@ The package is available on npm as [@shutterstock/p-map-iterable](https://www.np
 ## Importing
 
 ```typescript
-import { IterableMapper, IterableQueueMapper, IterableQueueMapperSimple } from '@shutterstock/p-map-iterable';
+import {
+  IterableMapper,
+  IterableQueueMapper,
+  IterableQueueMapperSimple } from '@shutterstock/p-map-iterable';
 ```
 
 ## API Documentation
@@ -63,7 +157,7 @@ These diagrams illustrate the differences in operation betweeen `p-map`, `p-queu
   - User supplied sync or async mapper function
   - Exposes an async iterable interface for consuming mapped items
   - Allows a maximum queue depth of mapped items - if the consumer stops consuming, the queue will fill up, at which point the mapper will stop being invoked until an item is consumed from the queue
-  - This allows mapping with back pressure so that the mapper does not consume unlimited resources (e.g. memory, disk, network, event loop time) by racing ahead of the consumer
+  - This allows mapping with backpressure so that the mapper does not consume unlimited resources (e.g. memory, disk, network, event loop time) by racing ahead of the consumer
 - [IterableQueueMapper](https://tech.shutterstock.com/p-map-iterable/classes/IterableQueueMapper.html)
   - Wraps `IterableMapper`
   - Adds items to the queue via the `enqueue` method
@@ -88,7 +182,7 @@ These diagrams illustrate the differences in operation betweeen `p-map`, `p-queu
 
 See [p-map](https://github.com/sindresorhus/p-map) docs for a good start in understanding what this does.
 
-The key difference between `IterableMapper` and `pMap` are that `IterableMapper` does not return when the entire mapping is done, rather it exposes an iterable that the caller loops through. This enables results to be processed while the mapping is still happening, while optionally allowing for back pressure to slow or stop the mapping if the caller is not consuming items fast enough. Common use cases include `prefetching` items from a remote service - the next set of requests are dispatched asyncronously while the current responses are processed and the prefetch requests will pause when the unread queue fills up.
+The key difference between `IterableMapper` and `pMap` are that `IterableMapper` does not return when the entire mapping is done, rather it exposes an iterable that the caller loops through. This enables results to be processed while the mapping is still happening, while optionally allowing for backpressure to slow or stop the mapping if the caller is not consuming items fast enough. Common use cases include `prefetching` items from a remote service - the next set of requests are dispatched asyncronously while the current responses are processed and the prefetch requests will pause when the unread queue fills up.
 
 See [examples/iterable-mapper.ts](./examples/iterable-mapper.ts) for an example.
 

package/dist/src/blocking-queue.js

@@ -1,7 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.BlockingQueue = void 0;
-const tslib_1 = require("tslib");
 const queue_1 = require("./queue");
 /**
  * `enqueue` blocks when the queue is full, until an item is read with `dequeue`, or done
@@ -72,65 +71,61 @@ class BlockingQueue {
      *
      * @param item Element to add
      */
-    enqueue(item) {
-        return tslib_1.__awaiter(this, void 0, void 0, function* () {
-            if (this._doneAdding) {
-                throw new Error('`enqueue` called after `done` called');
-            }
-            // We always add to the internal queue, we just wait to return if the queue is full
-            this._unreadQueue.enqueue(item);
-            if (this._readersWaiting.length > 0) {
-                // Release a reader if one is waiting for an item
-                // Remove the first waiting reader Promise from the queue
-                const reader = this._readersWaiting.dequeue();
-                if (reader === undefined) {
-                    throw new TypeError('reader queue returned undefined');
-                }
-                // Resolve the Promise for a waiting reader
-                reader(undefined);
-            }
-            if (this._unreadQueue.length > this._options.maxUnread) {
-                // We wait if the queue is at max length
-                yield this.waitForRead();
+    async enqueue(item) {
+        if (this._doneAdding) {
+            throw new Error('`enqueue` called after `done` called');
+        }
+        // We always add to the internal queue, we just wait to return if the queue is full
+        this._unreadQueue.enqueue(item);
+        if (this._readersWaiting.length > 0) {
+            // Release a reader if one is waiting for an item
+            // Remove the first waiting reader Promise from the queue
+            const reader = this._readersWaiting.dequeue();
+            if (reader === undefined) {
+                throw new TypeError('reader queue returned undefined');
             }
-        });
+            // Resolve the Promise for a waiting reader
+            reader(undefined);
+        }
+        if (this._unreadQueue.length > this._options.maxUnread) {
+            // We wait if the queue is at max length
+            await this.waitForRead();
+        }
     }
     /**
      * Gets an element when one is ready, waits if none are ready.
      *
      * @returns Element or undefined if queue is empty and `done` has been called
      */
-    dequeue() {
-        return tslib_1.__awaiter(this, void 0, void 0, function* () {
-            // Bail out and release all waiters if there are no more items coming
-            const done = this.areWeDone();
-            if (done) {
-                return undefined;
-            }
-            // Check if queue has an item
-            let item;
-            // Get the item if one is ready
-            if (this._unreadQueue.length !== 0 && this._readersWaiting.length === 0) {
-                // If there are items and no waiters, pop one and return it
-                item = this._unreadQueue.dequeue();
-            }
-            else {
-                // If there are no items and/or there are already waiters, we have to wait for one to be ready
-                yield this.waitForWrite();
-                item = this._unreadQueue.dequeue();
-            }
-            if (this._writersWaiting.length > 0) {
-                // Release a writer if one is waiting
-                // Remove the first waiting reader Promise from the queue
-                const writer = this._writersWaiting.dequeue();
-                if (writer === undefined) {
-                    throw new TypeError('writer queue returned undefined');
-                }
-                // Resolve the Promise for a waiting writer
-                writer(undefined);
+    async dequeue() {
+        // Bail out and release all waiters if there are no more items coming
+        const done = this.areWeDone();
+        if (done) {
+            return undefined;
+        }
+        // Check if queue has an item
+        let item;
+        // Get the item if one is ready
+        if (this._unreadQueue.length !== 0 && this._readersWaiting.length === 0) {
+            // If there are items and no waiters, pop one and return it
+            item = this._unreadQueue.dequeue();
+        }
+        else {
+            // If there are no items and/or there are already waiters, we have to wait for one to be ready
+            await this.waitForWrite();
+            item = this._unreadQueue.dequeue();
+        }
+        if (this._writersWaiting.length > 0) {
+            // Release a writer if one is waiting
+            // Remove the first waiting reader Promise from the queue
+            const writer = this._writersWaiting.dequeue();
+            if (writer === undefined) {
+                throw new TypeError('writer queue returned undefined');
             }
-            return item;
-        });
+            // Resolve the Promise for a waiting writer
+            writer(undefined);
+        }
+        return item;
     }
     releaseAllReaders() {
         // Release all the waiting readers since we're done
@@ -172,30 +167,26 @@ class BlockingQueue {
     /**
      * Waits for an item to be ready in the queue
      */
-    waitForWrite() {
-        return tslib_1.__awaiter(this, void 0, void 0, function* () {
-            // If there are waiters, create a waiter promise and add to end of list
-            // Note: the background reader removes the promise from the readers waiting queue
-            const itemReady = new Promise((resolve) => {
-                this._readersWaiting.enqueue(resolve);
-            });
-            // Wait for our item to be ready
-            yield itemReady;
+    async waitForWrite() {
+        // If there are waiters, create a waiter promise and add to end of list
+        // Note: the background reader removes the promise from the readers waiting queue
+        const itemReady = new Promise((resolve) => {
+            this._readersWaiting.enqueue(resolve);
         });
+        // Wait for our item to be ready
+        await itemReady;
     }
     /**
      * Waits to be able to add an item to the queue when full
      */
-    waitForRead() {
-        return tslib_1.__awaiter(this, void 0, void 0, function* () {
-            // If there are waiters, create a waiter promise and add to end of list
-            // Note: the background reader removes the promise from the readers waiting queue
-            const writeReady = new Promise((resolve) => {
-                this._writersWaiting.enqueue(resolve);
-            });
-            // Wait for our turn to add to the queue
-            yield writeReady;
+    async waitForRead() {
+        // If there are waiters, create a waiter promise and add to end of list
+        // Note: the background reader removes the promise from the readers waiting queue
+        const writeReady = new Promise((resolve) => {
+            this._writersWaiting.enqueue(resolve);
         });
+        // Wait for our turn to add to the queue
+        await writeReady;
     }
 }
 exports.BlockingQueue = BlockingQueue;

package/dist/src/blocking-queue.js.map

@@ -1 +1 @@
-{"version":3,"file":"blocking-queue.js","sourceRoot":"","sources":["../../src/blocking-queue.ts"],"names":[],"mappings":";;;;AAAA,mCAAgC;AAkBhC;;;;;;;GAOG;AACH,MAAa,aAAa;IAwBxB;;;;OAIG;IACH,YAAY,UAAgC,EAAE;QA1B9C,mFAAmF;QACnF,8EAA8E;QAC9E,yEAAyE;QACxD,iBAAY,GAAmB,IAAI,aAAK,EAAE,CAAC;QAE5D,8EAA8E;QAC9E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAE1E,6EAA6E;QAC7E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAElE,gBAAW,GAAG,KAAK,CAAC;QAQ1B,MAAM,EAAE,SAAS,GAAG,CAAC,EAAE,GAAG,OAAO,CAAC;QAClC,IAAI,CAAC,QAAQ,GAAG,EAAE,SAAS,EAAE,CAAC;QAE9B,oCAAoC;QACpC,IAAI,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,SAAS,EAAE;YACzC,MAAM,IAAI,SAAS,CAAC,yBAAyB,CAAC,CAAC;SAChD;QAED,4BAA4B;QAC5B,IACE,CAAC,CACC,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC;YAC5C,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,MAAM,CAAC,iBAAiB,CAAC;YACvD,IAAI,CAAC,QAAQ,CAAC,SAAS,IAAI,CAAC,CAC7B,EACD;YACA,MAAM,IAAI,SAAS,CACjB,gFAAgF,SAAS,OAAO,OAAO,SAAS,GAAG,CACpH,CAAC;SACH;IACH,CAAC;IAED,IAAW,MAAM;QACf,OAAO,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC;IAClC,CAAC;IAED;;;;OAIG;IACI,IAAI;QACT,mCAAmC;QACnC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC;QAExB,+EAA+E;QAC/E,+DAA+D;QAC/D,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;YACrC,OAAO;SACR;QAED,kEAAkE;QAClE,IAAI,CAAC,iBAAiB,EAAE,CAAC;IAC3B,CAAC;IAED;;;;OAIG;IACU,OAAO,CAAC,IAAa;;YAChC,IAAI,IAAI,CAAC,WAAW,EAAE;gBACpB,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;aACzD;YAED,mFAAmF;YACnF,IAAI,CAAC,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YAEhC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;gBACnC,iDAAiD;gBACjD,yDAAyD;gBACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;gBAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;oBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;iBACxD;gBAED,2CAA2C;gBAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;aACnB;YAED,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAC,SAAS,EAAE;gBACtD,wCAAwC;gBACxC,MAAM,IAAI,CAAC,WAAW,EAAE,CAAC;aAC1B;QACH,CAAC;KAAA;IAED;;;;OAIG;IACU,OAAO;;YAClB,qEAAqE;YACrE,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;YAC9B,IAAI,IAAI,EAAE;gBACR,OAAO,SAAS,CAAC;aAClB;YAED,6BAA6B;YAC7B,IAAI,IAAyB,CAAC;YAE9B,+BAA+B;YAC/B,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;gBACvE,2DAA2D;gBAC3D,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;aACpC;iBAAM;gBACL,8FAA8F;gBAC9F,MAAM,IAAI,CAAC,YAAY,EAAE,CAAC;gBAE1B,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;aACpC;YAED,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;gBACnC,qCAAqC;gBACrC,yDAAyD;gBACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;gBAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;oBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;iBACxD;gBAED,2CAA2C;gBAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;aACnB;YAED,OAAO,IAAI,CAAC;QACd,CAAC;KAAA;IAEO,iBAAiB;QACvB,mDAAmD;QACnD,IAAI,MAAsC,CAAC;QAC3C,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC,KAAK,SAAS,EAAE;YAC9D,MAAM,CAAC,SAAS,CAAC,CAAC;SACnB;IACH,CAAC;IAEO,SAAS;QACf,IAAI,IAAI,CAAC,WAAW,EAAE;YACpB,0CAA0C;YAC1C,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;gBACrC,gDAAgD;gBAChD,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,EAAE;oBAClC,iCAAiC;oBACjC,IAAI,CAAC,iBAAiB,EAAE,CAAC;oBAEzB,OAAO,IAAI,CAAC;iBACb;aACF;SACF;QAED,OAAO,KAAK,CAAC;IACf,CAAC;IAED,MAAM;IACN,gDAAgD;IAChD,wCAAwC;IACxC,MAAM;IACN,gEAAgE;IAChE,4EAA4E;IAC5E,sFAAsF;IACtF,iDAAiD;IACjD,6CAA6C;IAC7C,QAAQ;IAER,qCAAqC;IACrC,qBAAqB;IAErB,8CAA8C;IAE9C,oCAAoC;IACpC,iBAAiB;IACjB,IAAI;IAEJ;;OAEG;IACW,YAAY;;YACxB,uEAAuE;YACvE,iFAAiF;YACjF,MAAM,SAAS,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;gBACxC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;YACxC,CAAC,CAAC,CAAC;YAEH,gCAAgC;YAChC,MAAM,SAAS,CAAC;QAClB,CAAC;KAAA;IAED;;OAEG;IACW,WAAW;;YACvB,uEAAuE;YACvE,iFAAiF;YACjF,MAAM,UAAU,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;gBACzC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;YACxC,CAAC,CAAC,CAAC;YAEH,wCAAwC;YACxC,MAAM,UAAU,CAAC;QACnB,CAAC;KAAA;CACF;AA3ND,sCA2NC"}
+{"version":3,"file":"blocking-queue.js","sourceRoot":"","sources":["../../src/blocking-queue.ts"],"names":[],"mappings":";;;AAAA,mCAAgC;AAkBhC;;;;;;;GAOG;AACH,MAAa,aAAa;IAwBxB;;;;OAIG;IACH,YAAY,UAAgC,EAAE;QA1B9C,mFAAmF;QACnF,8EAA8E;QAC9E,yEAAyE;QACxD,iBAAY,GAAmB,IAAI,aAAK,EAAE,CAAC;QAE5D,8EAA8E;QAC9E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAE1E,6EAA6E;QAC7E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAElE,gBAAW,GAAG,KAAK,CAAC;QAQ1B,MAAM,EAAE,SAAS,GAAG,CAAC,EAAE,GAAG,OAAO,CAAC;QAClC,IAAI,CAAC,QAAQ,GAAG,EAAE,SAAS,EAAE,CAAC;QAE9B,oCAAoC;QACpC,IAAI,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,SAAS,EAAE;YACzC,MAAM,IAAI,SAAS,CAAC,yBAAyB,CAAC,CAAC;SAChD;QAED,4BAA4B;QAC5B,IACE,CAAC,CACC,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC;YAC5C,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,MAAM,CAAC,iBAAiB,CAAC;YACvD,IAAI,CAAC,QAAQ,CAAC,SAAS,IAAI,CAAC,CAC7B,EACD;YACA,MAAM,IAAI,SAAS,CACjB,gFAAgF,SAAS,OAAO,OAAO,SAAS,GAAG,CACpH,CAAC;SACH;IACH,CAAC;IAED,IAAW,MAAM;QACf,OAAO,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC;IAClC,CAAC;IAED;;;;OAIG;IACI,IAAI;QACT,mCAAmC;QACnC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC;QAExB,+EAA+E;QAC/E,+DAA+D;QAC/D,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;YACrC,OAAO;SACR;QAED,kEAAkE;QAClE,IAAI,CAAC,iBAAiB,EAAE,CAAC;IAC3B,CAAC;IAED;;;;OAIG;IACI,KAAK,CAAC,OAAO,CAAC,IAAa;QAChC,IAAI,IAAI,CAAC,WAAW,EAAE;YACpB,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;SACzD;QAED,mFAAmF;QACnF,IAAI,CAAC,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QAEhC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;YACnC,iDAAiD;YACjD,yDAAyD;YACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;YAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;gBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;aACxD;YAED,2CAA2C;YAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;SACnB;QAED,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAC,SAAS,EAAE;YACtD,wCAAwC;YACxC,MAAM,IAAI,CAAC,WAAW,EAAE,CAAC;SAC1B;IACH,CAAC;IAED;;;;OAIG;IACI,KAAK,CAAC,OAAO;QAClB,qEAAqE;QACrE,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC9B,IAAI,IAAI,EAAE;YACR,OAAO,SAAS,CAAC;SAClB;QAED,6BAA6B;QAC7B,IAAI,IAAyB,CAAC;QAE9B,+BAA+B;QAC/B,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;YACvE,2DAA2D;YAC3D,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;SACpC;aAAM;YACL,8FAA8F;YAC9F,MAAM,IAAI,CAAC,YAAY,EAAE,CAAC;YAE1B,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;SACpC;QAED,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;YACnC,qCAAqC;YACrC,yDAAyD;YACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;YAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;gBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;aACxD;YAED,2CAA2C;YAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;SACnB;QAED,OAAO,IAAI,CAAC;IACd,CAAC;IAEO,iBAAiB;QACvB,mDAAmD;QACnD,IAAI,MAAsC,CAAC;QAC3C,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC,KAAK,SAAS,EAAE;YAC9D,MAAM,CAAC,SAAS,CAAC,CAAC;SACnB;IACH,CAAC;IAEO,SAAS;QACf,IAAI,IAAI,CAAC,WAAW,EAAE;YACpB,0CAA0C;YAC1C,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;gBACrC,gDAAgD;gBAChD,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,EAAE;oBAClC,iCAAiC;oBACjC,IAAI,CAAC,iBAAiB,EAAE,CAAC;oBAEzB,OAAO,IAAI,CAAC;iBACb;aACF;SACF;QAED,OAAO,KAAK,CAAC;IACf,CAAC;IAED,MAAM;IACN,gDAAgD;IAChD,wCAAwC;IACxC,MAAM;IACN,gEAAgE;IAChE,4EAA4E;IAC5E,sFAAsF;IACtF,iDAAiD;IACjD,6CAA6C;IAC7C,QAAQ;IAER,qCAAqC;IACrC,qBAAqB;IAErB,8CAA8C;IAE9C,oCAAoC;IACpC,iBAAiB;IACjB,IAAI;IAEJ;;OAEG;IACK,KAAK,CAAC,YAAY;QACxB,uEAAuE;QACvE,iFAAiF;QACjF,MAAM,SAAS,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;YACxC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QACxC,CAAC,CAAC,CAAC;QAEH,gCAAgC;QAChC,MAAM,SAAS,CAAC;IAClB,CAAC;IAED;;OAEG;IACK,KAAK,CAAC,WAAW;QACvB,uEAAuE;QACvE,iFAAiF;QACjF,MAAM,UAAU,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;YACzC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QACxC,CAAC,CAAC,CAAC;QAEH,wCAAwC;QACxC,MAAM,UAAU,CAAC;IACnB,CAAC;CACF;AA3ND,sCA2NC"}

package/dist/src/blocking-queue.test.js

@@ -1,6 +1,5 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-const tslib_1 = require("tslib");
 /// <reference types="jest" />
 const blocking_queue_1 = require("./blocking-queue");
 describe('BlockingQueue', () => {
@@ -11,36 +10,36 @@ describe('BlockingQueue', () => {
         jest.clearAllMocks();
     });
     describe('maxUnread: 0', () => {
-        it('single item enqueue/dequeue works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        it('single item enqueue/dequeue works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 0 });
             setTimeout(() => {
                 void queue.enqueue(1);
             }, 1000);
-            const item = yield queue.dequeue();
+            const item = await queue.dequeue();
             queue.done();
             expect(item).toBe(1);
-        }));
-        it('dequeue after done does not hang', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('dequeue after done does not hang', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 0 });
             setTimeout(() => {
                 void queue.enqueue(1);
             }, 1000);
-            const item = yield queue.dequeue();
+            const item = await queue.dequeue();
             queue.done();
-            yield queue.dequeue();
+            await queue.dequeue();
             expect(item).toBe(1);
-        }));
-        it('enqueue after done throws', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('enqueue after done throws', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 0 });
             setTimeout(() => {
                 void queue.enqueue(1);
             }, 1000);
-            const item = yield queue.dequeue();
+            const item = await queue.dequeue();
             queue.done();
-            yield expect(() => tslib_1.__awaiter(void 0, void 0, void 0, function* () { return queue.enqueue(2); })).rejects.toThrowError('`enqueue` called after `done` called');
+            await expect(async () => queue.enqueue(2)).rejects.toThrowError('`enqueue` called after `done` called');
             expect(item).toBe(1);
-        }));
-        it('balanced enqueue/dequeue works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('balanced enqueue/dequeue works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 0 });
             const writers = [];
             writers.push(queue.enqueue(1));
@@ -48,13 +47,13 @@ describe('BlockingQueue', () => {
             const readers = [];
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
-            yield Promise.all(writers);
+            await Promise.all(writers);
             queue.done();
-            yield Promise.all(readers);
-            expect(yield readers[0]).toBe(1);
-            expect(yield readers[1]).toBe(2);
-        }));
-        it('full queue blocks enqueue until dequeue', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+            await Promise.all(readers);
+            expect(await readers[0]).toBe(1);
+            expect(await readers[1]).toBe(2);
+        });
+        it('full queue blocks enqueue until dequeue', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
             // Add first item
             void queue.enqueue(1);
@@ -63,47 +62,47 @@ describe('BlockingQueue', () => {
                 void queue.enqueue(2);
             }, 2000);
             const startTime = Date.now();
-            expect(yield queue.dequeue()).toBe(1);
+            expect(await queue.dequeue()).toBe(1);
             expect(Date.now() - startTime).toBeLessThan(2000);
-            expect(yield queue.dequeue()).toBe(2);
-            expect(Date.now() - startTime).toBeGreaterThanOrEqual(2000);
+            expect(await queue.dequeue()).toBe(2);
+            expect(Math.ceil(Date.now() - startTime)).toBeGreaterThanOrEqual(2000);
             queue.done();
-        }));
+        });
     });
     describe('maxUnread: 1', () => {
-        it('single item enqueue/dequeue works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        it('single item enqueue/dequeue works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
-            yield queue.enqueue(1);
-            const item = yield queue.dequeue();
+            await queue.enqueue(1);
+            const item = await queue.dequeue();
             queue.done();
             expect(item).toBe(1);
-        }));
-        it('dequeue after done does not hang', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('dequeue after done does not hang', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
-            yield queue.enqueue(1);
-            const item = yield queue.dequeue();
+            await queue.enqueue(1);
+            const item = await queue.dequeue();
             queue.done();
-            yield queue.dequeue();
+            await queue.dequeue();
             expect(item).toBe(1);
-        }));
-        it('dequeue after done and empty does not hang', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('dequeue after done and empty does not hang', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
-            yield queue.enqueue(1);
-            const item = yield queue.dequeue();
+            await queue.enqueue(1);
+            const item = await queue.dequeue();
             queue.done();
-            yield queue.dequeue();
+            await queue.dequeue();
             expect(item).toBe(1);
-            yield queue.dequeue();
-        }));
-        it('enqueue after done throws', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+            await queue.dequeue();
+        });
+        it('enqueue after done throws', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
-            yield queue.enqueue(1);
-            const item = yield queue.dequeue();
+            await queue.enqueue(1);
+            const item = await queue.dequeue();
             queue.done();
-            yield expect(() => tslib_1.__awaiter(void 0, void 0, void 0, function* () { return queue.enqueue(2); })).rejects.toThrowError('`enqueue` called after `done` called');
+            await expect(async () => queue.enqueue(2)).rejects.toThrowError('`enqueue` called after `done` called');
             expect(item).toBe(1);
-        }));
-        it('balanced enqueue/dequeue works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        });
+        it('balanced enqueue/dequeue works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
             const writers = [];
             writers.push(queue.enqueue(1));
@@ -111,13 +110,13 @@ describe('BlockingQueue', () => {
             const readers = [];
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
-            yield Promise.all(writers);
+            await Promise.all(writers);
             queue.done();
-            yield Promise.all(readers);
-            expect(yield readers[0]).toBe(1);
-            expect(yield readers[1]).toBe(2);
-        }));
-        it('more dequeue than enqueue works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+            await Promise.all(readers);
+            expect(await readers[0]).toBe(1);
+            expect(await readers[1]).toBe(2);
+        });
+        it('more dequeue than enqueue works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
             const writers = [];
             writers.push(queue.enqueue(1));
@@ -127,49 +126,49 @@ describe('BlockingQueue', () => {
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
-            yield Promise.all(writers);
+            await Promise.all(writers);
             queue.done();
-            yield Promise.all(readers);
-            expect(yield readers[0]).toBe(1);
-            expect(yield readers[1]).toBe(2);
-            expect(yield readers[2]).toBeUndefined();
-            expect(yield readers[3]).toBeUndefined();
-        }));
-        it('full queue blocks enqueue until dequeue', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+            await Promise.all(readers);
+            expect(await readers[0]).toBe(1);
+            expect(await readers[1]).toBe(2);
+            expect(await readers[2]).toBeUndefined();
+            expect(await readers[3]).toBeUndefined();
+        });
+        it('full queue blocks enqueue until dequeue', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 1 });
             // Wait for first item to add
-            yield queue.enqueue(1);
+            await queue.enqueue(1);
             // Do not wait for second item to add (it will wait until an enqueue completes)
             setTimeout(() => {
                 void queue.enqueue(2);
             }, 2000);
-            expect(yield queue.dequeue()).toBe(1);
+            expect(await queue.dequeue()).toBe(1);
             const startTime = Date.now();
-            expect(yield queue.dequeue()).toBe(2);
-            const duration = Date.now() - startTime;
+            expect(await queue.dequeue()).toBe(2);
+            const duration = Math.ceil(Date.now() - startTime);
             expect(duration).toBeGreaterThanOrEqual(2000);
             queue.done();
-        }));
+        });
     });
     describe('maxUnread: 2', () => {
-        it('no reads until done works', () => tslib_1.__awaiter(void 0, void 0, void 0, function* () {
+        it('no reads until done works', async () => {
             const queue = new blocking_queue_1.BlockingQueue({ maxUnread: 2 });
             const writers = [];
             writers.push(queue.enqueue(1));
             writers.push(queue.enqueue(2));
-            yield Promise.all(writers);
+            await Promise.all(writers);
             queue.done();
             const readers = [];
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
             readers.push(queue.dequeue());
-            yield Promise.all(readers);
-            expect(yield readers[0]).toBe(1);
-            expect(yield readers[1]).toBe(2);
-            expect(yield readers[2]).toBeUndefined();
-            expect(yield readers[3]).toBeUndefined();
-        }));
+            await Promise.all(readers);
+            expect(await readers[0]).toBe(1);
+            expect(await readers[1]).toBe(2);
+            expect(await readers[2]).toBeUndefined();
+            expect(await readers[3]).toBeUndefined();
+        });
     });
 });
 //# sourceMappingURL=blocking-queue.test.js.map