@shutterstock/p-map-iterable 0.0.1 → 1.0.10

package/README.md

@@ -1,29 +1,88 @@
-[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) [![API Docs](https://img.shields.io/badge/API%20Docs-View%20Here-blue)](https://tech.shutterstock.com/p-map-iterable/) [![Build - CI](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/ci.yml) [![Package and Publish](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/publish.yml) [![Publish Docs](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml/badge.svg)](https://github.com/shutterstock/p-map-iterable/actions/workflows/docs.yml)
 
 # Overview
 
-Async iterable that maps an async iterable input with backpressure.
+`@shutterstock/p-map-iterable` provides several classes that allow processing results of `p-map`-style mapper functions by iterating the results as they are completed, with back pressure to limit the number of items that are processed ahead of the consumer.
 
-A common use case for `pMapIterable` is as a "prefetcher" that will fetch, for example, DB or remote REST responses. By prefetching items the consumer is able to use 100% of the JS thread time for processing prefetched items instead of waiting for an item to arrive after each item is processed. This use case is most common in batch or queue consumers, not as much in request/response services.
+A common use case for `@shutterstock/p-map-iterable` is as a "prefetcher" that will fetch, for example, AWS S3 files in an AWS Lambda function. By prefetching large files the consumer is able to use 100% of the paid-for Lambda CPU time for the JS thread, rather than waiting idle while the next file is fetched. The backpressure (set by `maxUnread`) prevents the prefetcher from consuming unlimited memory or disk space by racing ahead of the consumer.
 
-# Installation
+These classes will typically be helpful in batch or queue consumers, not as much in request/response services.
+
+# Example Usage Scenario
+
+- AWS Lambda function for Kafka or Kinesis stream processing
+- Each invocation has a batch of, say, 100 records
+- Each record points to a file on S3 that needs to be parsed, processed, then written back to S3
+- The average file size is 50 MB, so these will take a few seconds to fetch and write
+- The Lambda function has 1,769 MB of RAM which gives it 1 vCPU, allowing the JS thread to run at roughly 100% speed of 1 core
+- If the code never waits to fetch or write files then the Lambda function will be able to use 100% of the paid-for CPU time
+- If there is no back pressure then reading 100 * 50 MB files would fill up the default Lambda temp disk space of 512 MB OR would consume 5 GB of memory, which would cause the Lambda function to fail
+- We can use `IterableMapper` to fetch up to 5 of the next files while the current file is being processed, then pause until the current file is processed and the next file is consumed
+- We can also use `IterableQueueMapper.enqueue()` to write the files back to S3 without waiting for them to finish, unless we get more than, say, 3-4 files being uploaded at once, at which point we can pause until the current file is uploaded before we allow queuing another file for upload
+- In the rare case that 3-4 files are uploading, but not yet finished, we would block on `.enqueue()` and not consume much CPU while waiting for at least 1 upload to finish
+
+# Getting Started
+
+## Installation
+
+The package is available on npm as [@shutterstock/p-map-iterable](https://www.npmjs.com/package/@shutterstock/p-map-iterable)
 
 `npm i @shutterstock/p-map-iterable`
 
+## Importing
+
+```typescript
+import { IterableMapper, IterableQueueMapper, IterableQueueMapperSimple } from '@shutterstock/p-map-iterable';
+```
+
+## API Documentation
+
+After installing the package, you might want to look at our [API Documentation](https://tech.shutterstock.com/p-map-iterable/) to learn about all the features available.
+
+# `p-map-iterable` vs `p-map` vs `p-queue`
+
+These diagrams illustrate the differences in operation betweeen `p-map`, `p-queue`, and `p-map-iterable`.
+
+## `p-map-iterable`
+
+![p-map-iterable operations overview](https://github.com/shutterstock/p-map-iterable/assets/5617868/abdc7079-8c12-4518-8135-867fc5085e60)
+
+## `p-map`
+
+![p-map operations overview](https://github.com/shutterstock/p-map-iterable/assets/5617868/2fd88213-3135-4de8-8ec2-224555c08d65)
+
+## `p-queue`
+
+![p-queue operations overview](https://github.com/shutterstock/p-map-iterable/assets/5617868/88300edb-7bfe-41f0-ae5b-1cd5723bc255)
+
 # Features
 
-- `IterableMapper`
+- [IterableMapper](https://tech.shutterstock.com/p-map-iterable/classes/IterableMapper.html)
   - Interface and concept based on: [p-map](https://github.com/sindresorhus/p-map)
   - Allows a sync or async iterable input
   - User supplied sync or async mapper function
   - Exposes an async iterable interface for consuming mapped items
   - Allows a maximum queue depth of mapped items - if the consumer stops consuming, the queue will fill up, at which point the mapper will stop being invoked until an item is consumed from the queue
   - This allows mapping with back pressure so that the mapper does not consume unlimited resources (e.g. memory, disk, network, event loop time) by racing ahead of the consumer
-- `IterableMapperSimple`
+- [IterableQueueMapper](https://tech.shutterstock.com/p-map-iterable/classes/IterableQueueMapper.html)
   - Wraps `IterableMapper`
+  - Adds items to the queue via the `enqueue` method
+- [IterableQueueMapperSimple](https://tech.shutterstock.com/p-map-iterable/classes/IterableQueueMapperSimple.html)
+  - Wraps `IterableQueueMapper`
   - Discards results as they become available
   - Exposes any accumulated errors through the `errors` property instead of throwing an `AggregateError`
-  - Retains the backpressure functionality of `IterableMapper`'s `enqueue` in that addtional items cannot be added if the max number of items are already queued and their mappers have not yet completed, but does not require that the results be read by the caller before the next item can be mapped since the results are discarded immediately after they are available
+  - Not actually `Iterable` - May rename this before 1.0.0
+
+## Lower Level Utilities
+- [IterableQueue](https://tech.shutterstock.com/p-map-iterable/classes/IterableQueue.html)
+  - Lower level utility class
+  - Wraps `BlockingQueue`
+  - Exposes an async iterable interface for consuming items in the queue
+- [BlockingQueue](https://tech.shutterstock.com/p-map-iterable/classes/BlockingQueue.html)
+  - Lower level utility class
+  - `dequeue` blocks until an item is available or until all items have been removed, then returns `undefined`
+  - `enqueue` blocks if the queue is full
+  - `done` signals that no more items will be added to the queue
 
 # `IterableMapper`
 
@@ -31,75 +90,25 @@ See [p-map](https://github.com/sindresorhus/p-map) docs for a good start in unde
 
 The key difference between `IterableMapper` and `pMap` are that `IterableMapper` does not return when the entire mapping is done, rather it exposes an iterable that the caller loops through. This enables results to be processed while the mapping is still happening, while optionally allowing for back pressure to slow or stop the mapping if the caller is not consuming items fast enough. Common use cases include `prefetching` items from a remote service - the next set of requests are dispatched asyncronously while the current responses are processed and the prefetch requests will pause when the unread queue fills up.
 
-```typescript
-import { IterableMapper } from '@shutterstock/p-map-iterable';
-import { promisify } from 'util';
-const sleep = promisify(setTimeout);
-
-class SleepIterator implements AsyncIterable<number> {
-  private _max: number;
-  private _current = 1;
-
-  constructor(max: number) {
-    this._max = max;
-  }
-
-  async *[Symbol.asyncIterator](): AsyncIterator<number> {
-    for (let i = 0; i < this._max; i++) {
-      await sleep(1 * (i % 10));
-
-      if (this._current <= this._max) {
-        yield this._current;
-      }
-
-      this._current++;
-    }
-  }
-}
-
-const max = 100;
-const iterator = new SleepIterator(max);
-let total = 0;
-let callCount = 0;
-
-// Create an item prefetcher with IterableMapper
-// Use `npm i it-batch`'s `batch` function to batch the source iterable
-// into batches of a specific size, such as 50, if making batch requests
-// to a remote service.
-const prefetcher = new IterableMapper(
-  // Batch example:
-  //   batch(iterator, 50),
-  iterator,
-  // Batch example:
-  //   async (values: number[]): Promise<number> => {
-  async (value: number): Promise<number> => {
-    callCount++;
-    total += value;
-
-    // Simulate fetching an async item with varied delays
-    await sleep(10 * (callCount % 6));
-
-    return total;
-  },
-  { concurrency: 2, maxUnread: 10 },
-);
-
-let lastTotal = 0;
-let loopCount = 0;
-
-// Loop through the prefetched items or batches
-// Will pause and wait for prefetch to complete if none available
-// Will immediately start processing if prefetched item or batch is already available
-// If the `maxUnread` count was hit then no prefetches will be in progress until an item
-// is returned from the iterable here.  Once an item is returned a mapper
-// will be started; this will repeat until `concurrency` mappers are running again.
-for await (const item of prefetcher) {
-  loopCount++;
-  if (item > lastTotal) {
-    lastTotal = item;
-  }
-}
-```
+See [examples/iterable-mapper.ts](./examples/iterable-mapper.ts) for an example.
+
+Run the example with `npm run example:iterable-mapper`
+
+# `IterableQueueMapper`
+
+`IterableQueueMapper` is similar to `IterableMapper` but instead of taking an iterable input it instead adds data via the `enqueue` method which will block if `maxUnread` will be reached by the current number of `mapper`'s running in parallel.
+
+See [examples/iterable-queue-mapper.ts](./examples/iterable-queue-mapper.ts) for an example.
+
+Run the example with `npm run example:iterable-queue-mapper`
+
+# `IterableQueueMapperSimple`
+
+`IterableQueueMapperSimple` is similar to `IterableQueueMapper` but instead exposing the results as an iterable it discards the results as soon as they are ready and exposes any errors through the `errors` property.
+
+See [examples/iterable-queue-mapper-simple.ts](./examples/iterable-queue-mapper-simple.ts) for an example.
+
+Run the example with `npm run example:iterable-queue-mapper-simple`
 
 # Contributing - Setting up Build Environment
 

package/dist/{blocking-queue.d.ts → src/blocking-queue.d.ts}

@@ -1,4 +1,4 @@
-interface BlockingQueueOptions {
+export interface BlockingQueueOptions {
     /**
      * Number of pending unread items.
      *
@@ -8,6 +8,14 @@ interface BlockingQueueOptions {
      */
     readonly maxUnread?: number;
 }
+/**
+ * `enqueue` blocks when the queue is full, until an item is read with `dequeue`, or done
+ * `dequeue` blocks when the queue is empty, until an item is added with `enqueue`, or done
+ * `done` must be called when no more items will be added
+ * `dequeue` returns `undefined` there are no more items and processing is complete
+ *
+ * @category Low-Level
+ */
 export declare class BlockingQueue<Element> {
     private _options;
     private readonly _unreadQueue;
@@ -15,9 +23,11 @@ export declare class BlockingQueue<Element> {
     private readonly _writersWaiting;
     private _doneAdding;
     /**
+     * Create a new `BlockingQueue`
      *
+     * @param options BlockingQueue options
      */
-    constructor({ maxUnread }?: BlockingQueueOptions);
+    constructor(options?: BlockingQueueOptions);
     get length(): number;
     /**
      * Indicate that no more items will be enqueued.
@@ -39,11 +49,6 @@ export declare class BlockingQueue<Element> {
     dequeue(): Promise<Element | undefined>;
     private releaseAllReaders;
     private areWeDone;
-    /**
-     * Waits for an item to be ready in the queue
-     * @returns Item from the ready queue
-     */
-    private readWhenReady;
     /**
      * Waits for an item to be ready in the queue
      */
@@ -53,5 +58,4 @@ export declare class BlockingQueue<Element> {
      */
     private waitForRead;
 }
-export {};
 //# sourceMappingURL=blocking-queue.d.ts.map

package/dist/src/blocking-queue.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"blocking-queue.d.ts","sourceRoot":"","sources":["../../src/blocking-queue.ts"],"names":[],"mappings":"AAEA,MAAM,WAAW,oBAAoB;IACnC;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;CAC7B;AAOD;;;;;;;GAOG;AACH,qBAAa,aAAa,CAAC,OAAO;IAChC,OAAO,CAAC,QAAQ,CAAiC;IAKjD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA+B;IAO5D,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA0C;IAO1E,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA0C;IAE1E,OAAO,CAAC,WAAW,CAAS;IAE5B;;;;OAIG;gBACS,OAAO,GAAE,oBAAyB;IAuB9C,IAAW,MAAM,IAAI,MAAM,CAE1B;IAED;;;;OAIG;IACI,IAAI,IAAI,IAAI;IAcnB;;;;OAIG;IACU,OAAO,CAAC,IAAI,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC;IA0BlD;;;;OAIG;IACU,OAAO,IAAI,OAAO,CAAC,OAAO,GAAG,SAAS,CAAC;IAoCpD,OAAO,CAAC,iBAAiB;IAQzB,OAAO,CAAC,SAAS;IAqCjB;;OAEG;YACW,YAAY;IAW1B;;OAEG;YACW,WAAW;CAU1B"}

package/dist/{blocking-queue.js → src/blocking-queue.js}

@@ -3,11 +3,21 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.BlockingQueue = void 0;
 const tslib_1 = require("tslib");
 const queue_1 = require("./queue");
+/**
+ * `enqueue` blocks when the queue is full, until an item is read with `dequeue`, or done
+ * `dequeue` blocks when the queue is empty, until an item is added with `enqueue`, or done
+ * `done` must be called when no more items will be added
+ * `dequeue` returns `undefined` there are no more items and processing is complete
+ *
+ * @category Low-Level
+ */
 class BlockingQueue {
     /**
+     * Create a new `BlockingQueue`
      *
+     * @param options BlockingQueue options
      */
-    constructor({ maxUnread = 8 } = {}) {
+    constructor(options = {}) {
         // NOTE: unreadQueue has to be a proper FIFO queue with O(1) removal from the front
         // as we may have O(m) unread items, where m may approach n, and Array.shift()
         // would be O(m) (approaching O(n)) in that case as the array gets large.
@@ -25,6 +35,7 @@ class BlockingQueue {
         // is ready.
         this._writersWaiting = new queue_1.Queue();
         this._doneAdding = false;
+        const { maxUnread = 8 } = options;
         this._options = { maxUnread };
         // Avoid undefined errors on options
         if (this._options.maxUnread === undefined) {
@@ -142,24 +153,22 @@ class BlockingQueue {
         }
         return false;
     }
-    /**
-     * Waits for an item to be ready in the queue
-     * @returns Item from the ready queue
-     */
-    readWhenReady() {
-        return tslib_1.__awaiter(this, void 0, void 0, function* () {
-            // If there are waiters, create a waiter promise and add to end of list
-            // Note: the background reader removes the promise from the readers waiting queue
-            const itemReady = new Promise((resolve) => {
-                this._readersWaiting.enqueue(resolve);
-            });
-            // Wait for our item to be ready
-            yield itemReady;
-            const item = this._unreadQueue.dequeue();
-            // Pull the item from the front
-            return item;
-        });
-    }
+    // /**
+    //  * Waits for an item to be ready in the queue
+    //  * @returns Item from the ready queue
+    //  */
+    // private async readWhenReady(): Promise<Element | undefined> {
+    //   // If there are waiters, create a waiter promise and add to end of list
+    //   // Note: the background reader removes the promise from the readers waiting queue
+    //   const itemReady = new Promise((resolve) => {
+    //     this._readersWaiting.enqueue(resolve);
+    //   });
+    //   // Wait for our item to be ready
+    //   await itemReady;
+    //   const item = this._unreadQueue.dequeue();
+    //   // Pull the item from the front
+    //   return item;
+    // }
     /**
      * Waits for an item to be ready in the queue
      */

package/dist/src/blocking-queue.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"blocking-queue.js","sourceRoot":"","sources":["../../src/blocking-queue.ts"],"names":[],"mappings":";;;;AAAA,mCAAgC;AAkBhC;;;;;;;GAOG;AACH,MAAa,aAAa;IAwBxB;;;;OAIG;IACH,YAAY,UAAgC,EAAE;QA1B9C,mFAAmF;QACnF,8EAA8E;QAC9E,yEAAyE;QACxD,iBAAY,GAAmB,IAAI,aAAK,EAAE,CAAC;QAE5D,8EAA8E;QAC9E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAE1E,6EAA6E;QAC7E,+EAA+E;QAC/E,gDAAgD;QAChD,2EAA2E;QAC3E,YAAY;QACK,oBAAe,GAA8B,IAAI,aAAK,EAAE,CAAC;QAElE,gBAAW,GAAG,KAAK,CAAC;QAQ1B,MAAM,EAAE,SAAS,GAAG,CAAC,EAAE,GAAG,OAAO,CAAC;QAClC,IAAI,CAAC,QAAQ,GAAG,EAAE,SAAS,EAAE,CAAC;QAE9B,oCAAoC;QACpC,IAAI,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,SAAS,EAAE;YACzC,MAAM,IAAI,SAAS,CAAC,yBAAyB,CAAC,CAAC;SAChD;QAED,4BAA4B;QAC5B,IACE,CAAC,CACC,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC;YAC5C,IAAI,CAAC,QAAQ,CAAC,SAAS,KAAK,MAAM,CAAC,iBAAiB,CAAC;YACvD,IAAI,CAAC,QAAQ,CAAC,SAAS,IAAI,CAAC,CAC7B,EACD;YACA,MAAM,IAAI,SAAS,CACjB,gFAAgF,SAAS,OAAO,OAAO,SAAS,GAAG,CACpH,CAAC;SACH;IACH,CAAC;IAED,IAAW,MAAM;QACf,OAAO,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC;IAClC,CAAC;IAED;;;;OAIG;IACI,IAAI;QACT,mCAAmC;QACnC,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC;QAExB,+EAA+E;QAC/E,+DAA+D;QAC/D,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;YACrC,OAAO;SACR;QAED,kEAAkE;QAClE,IAAI,CAAC,iBAAiB,EAAE,CAAC;IAC3B,CAAC;IAED;;;;OAIG;IACU,OAAO,CAAC,IAAa;;YAChC,IAAI,IAAI,CAAC,WAAW,EAAE;gBACpB,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;aACzD;YAED,mFAAmF;YACnF,IAAI,CAAC,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YAEhC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;gBACnC,iDAAiD;gBACjD,yDAAyD;gBACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;gBAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;oBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;iBACxD;gBAED,2CAA2C;gBAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;aACnB;YAED,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,CAAC,SAAS,EAAE;gBACtD,wCAAwC;gBACxC,MAAM,IAAI,CAAC,WAAW,EAAE,CAAC;aAC1B;QACH,CAAC;KAAA;IAED;;;;OAIG;IACU,OAAO;;YAClB,qEAAqE;YACrE,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;YAC9B,IAAI,IAAI,EAAE;gBACR,OAAO,SAAS,CAAC;aAClB;YAED,6BAA6B;YAC7B,IAAI,IAAyB,CAAC;YAE9B,+BAA+B;YAC/B,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;gBACvE,2DAA2D;gBAC3D,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;aACpC;iBAAM;gBACL,8FAA8F;gBAC9F,MAAM,IAAI,CAAC,YAAY,EAAE,CAAC;gBAE1B,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,EAAE,CAAC;aACpC;YAED,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE;gBACnC,qCAAqC;gBACrC,yDAAyD;gBACzD,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC;gBAC9C,IAAI,MAAM,KAAK,SAAS,EAAE;oBACxB,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;iBACxD;gBAED,2CAA2C;gBAC3C,MAAM,CAAC,SAAS,CAAC,CAAC;aACnB;YAED,OAAO,IAAI,CAAC;QACd,CAAC;KAAA;IAEO,iBAAiB;QACvB,mDAAmD;QACnD,IAAI,MAAsC,CAAC;QAC3C,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,eAAe,CAAC,OAAO,EAAE,CAAC,KAAK,SAAS,EAAE;YAC9D,MAAM,CAAC,SAAS,CAAC,CAAC;SACnB;IACH,CAAC;IAEO,SAAS;QACf,IAAI,IAAI,CAAC,WAAW,EAAE;YACpB,0CAA0C;YAC1C,IAAI,IAAI,CAAC,eAAe,CAAC,MAAM,KAAK,CAAC,EAAE;gBACrC,gDAAgD;gBAChD,IAAI,IAAI,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,EAAE;oBAClC,iCAAiC;oBACjC,IAAI,CAAC,iBAAiB,EAAE,CAAC;oBAEzB,OAAO,IAAI,CAAC;iBACb;aACF;SACF;QAED,OAAO,KAAK,CAAC;IACf,CAAC;IAED,MAAM;IACN,gDAAgD;IAChD,wCAAwC;IACxC,MAAM;IACN,gEAAgE;IAChE,4EAA4E;IAC5E,sFAAsF;IACtF,iDAAiD;IACjD,6CAA6C;IAC7C,QAAQ;IAER,qCAAqC;IACrC,qBAAqB;IAErB,8CAA8C;IAE9C,oCAAoC;IACpC,iBAAiB;IACjB,IAAI;IAEJ;;OAEG;IACW,YAAY;;YACxB,uEAAuE;YACvE,iFAAiF;YACjF,MAAM,SAAS,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;gBACxC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;YACxC,CAAC,CAAC,CAAC;YAEH,gCAAgC;YAChC,MAAM,SAAS,CAAC;QAClB,CAAC;KAAA;IAED;;OAEG;IACW,WAAW;;YACvB,uEAAuE;YACvE,iFAAiF;YACjF,MAAM,UAAU,GAAG,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;gBACzC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;YACxC,CAAC,CAAC,CAAC;YAEH,wCAAwC;YACxC,MAAM,UAAU,CAAC;QACnB,CAAC;KAAA;CACF;AA3ND,sCA2NC"}

package/dist/src/blocking-queue.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"blocking-queue.test.d.ts","sourceRoot":"","sources":["../../src/blocking-queue.test.ts"],"names":[],"mappings":""}

package/dist/src/blocking-queue.test.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"blocking-queue.test.js","sourceRoot":"","sources":["../../src/blocking-queue.test.ts"],"names":[],"mappings":";;;AAAA,8BAA8B;AAC9B,qDAAiD;AAEjD,QAAQ,CAAC,eAAe,EAAE,GAAG,EAAE;IAC7B,SAAS,CAAC,GAAG,EAAE;QACb,UAAU;IACZ,CAAC,CAAC,CAAC;IAEH,UAAU,CAAC,GAAG,EAAE;QACd,IAAI,CAAC,aAAa,EAAE,CAAC;IACvB,CAAC,CAAC,CAAC;IAEH,QAAQ,CAAC,cAAc,EAAE,GAAG,EAAE;QAC5B,EAAE,CAAC,mCAAmC,EAAE,GAAS,EAAE;YACjD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,UAAU,CAAC,GAAG,EAAE;gBACd,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACxB,CAAC,EAAE,IAAI,CAAC,CAAC;YACT,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,kCAAkC,EAAE,GAAS,EAAE;YAChD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,UAAU,CAAC,GAAG,EAAE;gBACd,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACxB,CAAC,EAAE,IAAI,CAAC,CAAC;YACT,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACtB,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,2BAA2B,EAAE,GAAS,EAAE;YACzC,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,UAAU,CAAC,GAAG,EAAE;gBACd,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACxB,CAAC,EAAE,IAAI,CAAC,CAAC;YACT,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,MAAM,CAAC,GAAS,EAAE,0DAAC,OAAA,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAA,GAAA,CAAC,CAAC,OAAO,CAAC,YAAY,CAC7D,sCAAsC,CACvC,CAAC;YACF,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,gCAAgC,EAAE,GAAS,EAAE;YAC9C,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,OAAO,GAAoB,EAAE,CAAC;YACpC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,OAAO,GAAkC,EAAE,CAAC;YAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAE9B,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,KAAK,CAAC,IAAI,EAAE,CAAC;YAEb,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACnC,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,yCAAyC,EAAE,GAAS,EAAE;YACvD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAE1D,iBAAiB;YACjB,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACtB,+EAA+E;YAC/E,UAAU,CAAC,GAAG,EAAE;gBACd,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACxB,CAAC,EAAE,IAAI,CAAC,CAAC;YAET,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;YAC7B,MAAM,CAAC,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACtC,MAAM,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC;YAClD,MAAM,CAAC,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACtC,MAAM,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC,CAAC,sBAAsB,CAAC,IAAI,CAAC,CAAC;YAE5D,KAAK,CAAC,IAAI,EAAE,CAAC;QACf,CAAC,CAAA,CAAC,CAAC;IACL,CAAC,CAAC,CAAC;IAEH,QAAQ,CAAC,cAAc,EAAE,GAAG,EAAE;QAC5B,EAAE,CAAC,mCAAmC,EAAE,GAAS,EAAE;YACjD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,kCAAkC,EAAE,GAAS,EAAE;YAChD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACtB,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,4CAA4C,EAAE,GAAS,EAAE;YAC1D,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACtB,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YAErB,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;QACxB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,2BAA2B,EAAE,GAAS,EAAE;YACzC,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC;YACnC,KAAK,CAAC,IAAI,EAAE,CAAC;YACb,MAAM,MAAM,CAAC,GAAS,EAAE,0DAAC,OAAA,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAA,GAAA,CAAC,CAAC,OAAO,CAAC,YAAY,CAC7D,sCAAsC,CACvC,CAAC;YACF,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACvB,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,gCAAgC,EAAE,GAAS,EAAE;YAC9C,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,OAAO,GAAoB,EAAE,CAAC;YACpC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,OAAO,GAAkC,EAAE,CAAC;YAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAE9B,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,KAAK,CAAC,IAAI,EAAE,CAAC;YAEb,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACnC,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,iCAAiC,EAAE,GAAS,EAAE;YAC/C,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,OAAO,GAAoB,EAAE,CAAC;YACpC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,OAAO,GAAkC,EAAE,CAAC;YAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAE9B,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,KAAK,CAAC,IAAI,EAAE,CAAC;YAEb,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC;YACzC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC;QAC3C,CAAC,CAAA,CAAC,CAAC;QAEH,EAAE,CAAC,yCAAyC,EAAE,GAAS,EAAE;YACvD,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAE1D,6BAA6B;YAC7B,MAAM,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACvB,+EAA+E;YAC/E,UAAU,CAAC,GAAG,EAAE;gBACd,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;YACxB,CAAC,EAAE,IAAI,CAAC,CAAC;YAET,MAAM,CAAC,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACtC,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;YAC7B,MAAM,CAAC,MAAM,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACtC,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;YACxC,MAAM,CAAC,QAAQ,CAAC,CAAC,sBAAsB,CAAC,IAAI,CAAC,CAAC;YAE9C,KAAK,CAAC,IAAI,EAAE,CAAC;QACf,CAAC,CAAA,CAAC,CAAC;IACL,CAAC,CAAC,CAAC;IAEH,QAAQ,CAAC,cAAc,EAAE,GAAG,EAAE;QAC5B,EAAE,CAAC,2BAA2B,EAAE,GAAS,EAAE;YACzC,MAAM,KAAK,GAAG,IAAI,8BAAa,CAAS,EAAE,SAAS,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1D,MAAM,OAAO,GAAoB,EAAE,CAAC;YACpC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAC/B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;YAE/B,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAC3B,KAAK,CAAC,IAAI,EAAE,CAAC;YAEb,MAAM,OAAO,GAAkC,EAAE,CAAC;YAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAC9B,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC;YAE9B,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YAE3B,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YACjC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC;YACzC,MAAM,CAAC,MAAM,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC;QAC3C,CAAC,CAAA,CAAC,CAAC;IACL,CAAC,CAAC,CAAC;AACL,CAAC,CAAC,CAAC"}

package/dist/src/index.d.ts

@@ -0,0 +1,8 @@
+import { Mapper, IterableMapper, IterableMapperOptions } from './iterable-mapper';
+import { BlockingQueue, BlockingQueueOptions } from './blocking-queue';
+import { IterableQueue, IterableQueueOptions } from './iterable-queue';
+import { IterableQueueMapper, IterableQueueMapperOptions } from './iterable-queue-mapper';
+import { IterableQueueMapperSimple } from './iterable-queue-mapper-simple';
+import { Queue } from './queue';
+export { Mapper, IterableMapper, IterableMapperOptions, IterableQueueMapper, IterableQueueMapperOptions, IterableQueueMapperSimple, BlockingQueue, BlockingQueueOptions, IterableQueue, IterableQueueOptions, Queue, };
+//# sourceMappingURL=index.d.ts.map

package/dist/src/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,EAAE,cAAc,EAAE,qBAAqB,EAAE,MAAM,mBAAmB,CAAC;AAClF,OAAO,EAAE,aAAa,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACvE,OAAO,EAAE,aAAa,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACvE,OAAO,EAAE,mBAAmB,EAAE,0BAA0B,EAAE,MAAM,yBAAyB,CAAC;AAC1F,OAAO,EAAE,yBAAyB,EAAE,MAAM,gCAAgC,CAAC;AAC3E,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAEhC,OAAO,EACL,MAAM,EACN,cAAc,EACd,qBAAqB,EACrB,mBAAmB,EACnB,0BAA0B,EAC1B,yBAAyB,EACzB,aAAa,EACb,oBAAoB,EACpB,aAAa,EACb,oBAAoB,EACpB,KAAK,GACN,CAAC"}

package/dist/src/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":";;;AAAA,uDAAkF;AAShF,+FATe,gCAAc,OASf;AARhB,qDAAuE;AAarE,8FAbO,8BAAa,OAaP;AAZf,qDAAuE;AAcrE,8FAdO,8BAAa,OAcP;AAbf,mEAA0F;AAQxF,oGARO,2CAAmB,OAQP;AAPrB,iFAA2E;AASzE,0GATO,wDAAyB,OASP;AAR3B,mCAAgC;AAa9B,sFAbO,aAAK,OAaP"}

package/dist/src/iterable-mapper.d.ts

@@ -0,0 +1,125 @@
+/**
+ * Options for IterableMapper
+ */
+export interface IterableMapperOptions {
+    /**
+     * Number of concurrently pending promises returned by `mapper`.
+     *
+     * Must be an integer from 1 and up or `Infinity`, must be <= `maxUnread`.
+     *
+     * @default 4
+     */
+    readonly concurrency?: number;
+    /**
+     * Number of pending unread iterable items.
+     *
+     * Must be an integer from 1 and up or `Infinity`, must be >= `concurrency`.
+     *
+     * @default 8
+     */
+    readonly maxUnread?: number;
+    /**
+     * When set to `false`, instead of stopping when a promise rejects, it will wait for all the promises to settle and then reject with an [aggregated error](https://github.com/sindresorhus/aggregate-error) containing all the errors from the rejected promises.
+     *
+     * @default true
+     */
+    readonly stopOnMapperError?: boolean;
+}
+/**
+ * Function which is called for every item in `input`. Expected to return a `Promise` or value.
+ *
+ * @template Element - Source element type
+ * @template NewElement - Element type returned by the mapper
+ * @param element - Iterated element
+ * @param index - Index of the element in the source array
+ */
+export type Mapper<Element = unknown, NewElement = unknown> = (element: Element, index: number) => NewElement | Promise<NewElement>;
+/**
+ * Iterates over a source iterable with specified concurrency,
+ * calling the `mapper` on each iterated item, and storing the
+ * `mapper` result in a queue of specified max size, before
+ * being iterated / read by the caller.
+ *
+ * @remarks
+ *
+ * This allows performing a concurrent mapping with
+ * back pressure (won't iterate all source items if the consumer is
+ * not reading).
+ *
+ * Typical use case is for a `prefetcher` that ensures that items
+ * are always ready for the consumer but that large numbers of items
+ * are not processed before the consumer is ready for them.
+ *
+ * @category Iterable Input
+ */
+export declare class IterableMapper<Element, NewElement> implements AsyncIterable<NewElement> {
+    private _mapper;
+    private _options;
+    private _unreadQueue;
+    private _iterator;
+    private readonly _errors;
+    private _asyncIterator;
+    private _isRejected;
+    private _isIterableDone;
+    private _activeRunners;
+    private _resolvingCount;
+    private _currentIndex;
+    private _initialRunnersCreated;
+    /**
+     * Create a new `IterableMapper`
+     *
+     * @param input Iterated over concurrently in the `mapper` function.
+     * @param mapper Function which is called for every item in `input`.
+     *    Expected to return a `Promise` or value.
+     *
+     *    The `mapper` *should* handle all errors and not allow an error to be thrown
+     *    out of the `mapper` function as this enables the best handling of errors
+     *    closest to the time that they occur.
+     *
+     *    If the `mapper` function does allow an error to be thrown then the
+     *   `stopOnMapperError` option controls the behavior:
+     *      - `stopOnMapperError`: `true` - will throw the error
+     *        out of `next` or the `AsyncIterator` returned from `[Symbol.asyncIterator]`
+     *        and stop processing.
+     *     - `stopOnMapperError`: `false` - will continue processing
+     *        and accumulate the errors to be thrown from `next` or the `AsyncIterator`
+     *        returned from `[Symbol.asyncIterator]` when all items have been processed.
+     * @param options IterableMapper options
+     */
+    constructor(input: AsyncIterable<Element> | Iterable<Element>, mapper: Mapper<Element, NewElement>, options?: IterableMapperOptions);
+    [Symbol.asyncIterator](): AsyncIterator<NewElement>;
+    /**
+     * Used by the iterator returned from [Symbol.asyncIterator]
+     * Called every time an item is needed
+     *
+     * @returns Iterator result
+     */
+    next(): Promise<IteratorResult<NewElement>>;
+    private bubbleUpErrors;
+    private startARunnerIfNeeded;
+    private startAnotherRunner;
+    private areWeDone;
+    /**
+     * Throw an exception if the wrapped NewElement is an Error
+     *
+     * @returns Element if no error
+     */
+    private throwIfError;
+    /**
+     * Get the next item from the source iterable.
+     *
+     * @remarks
+     *
+     * This is called up to `concurrency` times in parallel.
+     *
+     * If the read queue is not full, and there are source items to read,
+     * each instance of this will keep calling a new instance of itself
+     * that detaches and runs asynchronously (keeping the same number
+     * of instances running).
+     *
+     * If the read queue + runners = max read queue length then the runner
+     * will exit and will be restarted when an item is read from the queue.
+     */
+    private sourceNext;
+}
+//# sourceMappingURL=iterable-mapper.d.ts.map

package/dist/src/iterable-mapper.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"iterable-mapper.d.ts","sourceRoot":"","sources":["../../src/iterable-mapper.ts"],"names":[],"mappings":"AAMA;;GAEG;AACH,MAAM,WAAW,qBAAqB;IACpC;;;;;;OAMG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAE9B;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAE5B;;;;OAIG;IACH,QAAQ,CAAC,iBAAiB,CAAC,EAAE,OAAO,CAAC;CACtC;AAED;;;;;;;GAOG;AACH,MAAM,MAAM,MAAM,CAAC,OAAO,GAAG,OAAO,EAAE,UAAU,GAAG,OAAO,IAAI,CAC5D,OAAO,EAAE,OAAO,EAChB,KAAK,EAAE,MAAM,KACV,UAAU,GAAG,OAAO,CAAC,UAAU,CAAC,CAAC;AAUtC;;;;;;;;;;;;;;;;;GAiBG;AACH,qBAAa,cAAc,CAAC,OAAO,EAAE,UAAU,CAAE,YAAW,aAAa,CAAC,UAAU,CAAC;IACnF,OAAO,CAAC,OAAO,CAA8B;IAC7C,OAAO,CAAC,QAAQ,CAAkC;IAElD,OAAO,CAAC,YAAY,CAA+C;IAEnE,OAAO,CAAC,SAAS,CAA6C;IAC9D,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAiB;IACzC,OAAO,CAAC,cAAc,CAAS;IAC/B,OAAO,CAAC,WAAW,CAAS;IAC5B,OAAO,CAAC,eAAe,CAAS;IAChC,OAAO,CAAC,cAAc,CAAK;IAC3B,OAAO,CAAC,eAAe,CAAK;IAC5B,OAAO,CAAC,aAAa,CAAK;IAC1B,OAAO,CAAC,sBAAsB,CAAS;IAEvC;;;;;;;;;;;;;;;;;;;;OAoBG;gBAED,KAAK,EAAE,aAAa,CAAC,OAAO,CAAC,GAAG,QAAQ,CAAC,OAAO,CAAC,EACjD,MAAM,EAAE,MAAM,CAAC,OAAO,EAAE,UAAU,CAAC,EACnC,OAAO,GAAE,qBAA0B;IAwF9B,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,aAAa,CAAC,UAAU,CAAC;IAI1D;;;;;OAKG;IACU,IAAI,IAAI,OAAO,CAAC,cAAc,CAAC,UAAU,CAAC,CAAC;IAyBxD,OAAO,CAAC,cAAc;IAOtB,OAAO,CAAC,oBAAoB;IAuB5B,OAAO,CAAC,kBAAkB;IAwB1B,OAAO,CAAC,SAAS;IAiBjB;;;;OAIG;IACH,OAAO,CAAC,YAAY;IASpB;;;;;;;;;;;;;;OAcG;YACW,UAAU;CAiHzB"}

package/dist/{iterable-mapper.js → src/iterable-mapper.js}

@@ -7,33 +7,47 @@ const tslib_1 = require("tslib");
 //
 const aggregate_error_1 = tslib_1.__importDefault(require("aggregate-error"));
 const iterable_queue_1 = require("./iterable-queue");
+/**
+ * Iterates over a source iterable with specified concurrency,
+ * calling the `mapper` on each iterated item, and storing the
+ * `mapper` result in a queue of specified max size, before
+ * being iterated / read by the caller.
+ *
+ * @remarks
+ *
+ * This allows performing a concurrent mapping with
+ * back pressure (won't iterate all source items if the consumer is
+ * not reading).
+ *
+ * Typical use case is for a `prefetcher` that ensures that items
+ * are always ready for the consumer but that large numbers of items
+ * are not processed before the consumer is ready for them.
+ *
+ * @category Iterable Input
+ */
 class IterableMapper {
     /**
-     * Iterates over a source iterable with specified concurrency,
-     * calling the `mapper` on each iterated item, and storing the
-     * `mapper` result in a queue of specified max size, before
-     * being iterated / read by the caller.
+     * Create a new `IterableMapper`
      *
-     * Essentially - This allows performing a concurrent mapping with
-     * back pressure (won't iterate all source items if the consumer is
-     * not reading).
+     * @param input Iterated over concurrently in the `mapper` function.
+     * @param mapper Function which is called for every item in `input`.
+     *    Expected to return a `Promise` or value.
      *
-     * Typical use case is for a `prefetcher` that ensures that items
-     * are always ready for the consumer but that large numbers of items
-     * are not processed before the consumer is ready for them.
+     *    The `mapper` *should* handle all errors and not allow an error to be thrown
+     *    out of the `mapper` function as this enables the best handling of errors
+     *    closest to the time that they occur.
      *
-     * @param input - Iterated over concurrently in the `mapper` function.
-     * @param mapper - Function which is called for every item in `input`. Expected to return a `Promise` or value.
-     * @param options - { concurrency: max mappers to run in parallel,
-     *                    maxUnread: max unread mapped results - this waits for results to be read,
-     *                    stopOnError: return if a mapper throws - note that any other running mappers will then not be able to return results
-     *                  }
-     * @returns A `Promise` that is fulfilled when all promises in `input`
-     * and ones returned from `mapper` are fulfilled, or rejects if any of the
-     * promises reject. The fulfilled value is an `Array` of the fulfilled
-     * values returned from `mapper` in `input` order.
+     *    If the `mapper` function does allow an error to be thrown then the
+     *   `stopOnMapperError` option controls the behavior:
+     *      - `stopOnMapperError`: `true` - will throw the error
+     *        out of `next` or the `AsyncIterator` returned from `[Symbol.asyncIterator]`
+     *        and stop processing.
+     *     - `stopOnMapperError`: `false` - will continue processing
+     *        and accumulate the errors to be thrown from `next` or the `AsyncIterator`
+     *        returned from `[Symbol.asyncIterator]` when all items have been processed.
+     * @param options IterableMapper options
      */
-    constructor(iterable, mapper, { concurrency = 4, stopOnMapperError = true, maxUnread = 8 } = {}) {
+    constructor(input, mapper, options = {}) {
         this._errors = [];
         this._asyncIterator = false;
         this._isRejected = false;
@@ -42,6 +56,7 @@ class IterableMapper {
         this._resolvingCount = 0;
         this._currentIndex = 0;
         this._initialRunnersCreated = false;
+        const { concurrency = 4, stopOnMapperError = true, maxUnread = 8 } = options;
         this._mapper = mapper;
         this._options = { concurrency, stopOnMapperError, maxUnread };
         if (typeof mapper !== 'function') {
@@ -71,13 +86,13 @@ class IterableMapper {
         }
         this._unreadQueue = new iterable_queue_1.IterableQueue({ maxUnread });
         // Setup the source iterator
-        if (iterable[Symbol.asyncIterator] !== undefined) {
+        if (input[Symbol.asyncIterator] !== undefined) {
             // We've got an async iterable
-            this._iterator = iterable[Symbol.asyncIterator]();
+            this._iterator = input[Symbol.asyncIterator]();
             this._asyncIterator = true;
         }
         else {
-            this._iterator = iterable[Symbol.iterator]();
+            this._iterator = input[Symbol.iterator]();
         }
         // Create the initial concurrent runners in a detached (non-awaited)
         // promise.  We need this so we can await the next() calls
@@ -105,6 +120,7 @@ class IterableMapper {
     /**
      * Used by the iterator returned from [Symbol.asyncIterator]
      * Called every time an item is needed
+     *
      * @returns Iterator result
      */
     next() {
@@ -192,7 +208,8 @@ class IterableMapper {
     }
     /**
      * Throw an exception if the wrapped NewElement is an Error
-     * @returns
+     *
+     * @returns Element if no error
      */
     throwIfError(item) {
         if (item.error !== undefined) {
@@ -206,6 +223,8 @@ class IterableMapper {
     /**
      * Get the next item from the source iterable.
      *
+     * @remarks
+     *
      * This is called up to `concurrency` times in parallel.
      *
      * If the read queue is not full, and there are source items to read,
@@ -215,7 +234,6 @@ class IterableMapper {
      *
      * If the read queue + runners = max read queue length then the runner
      * will exit and will be restarted when an item is read from the queue.
-     * @returns
      */
     sourceNext() {
         return tslib_1.__awaiter(this, void 0, void 0, function* () {