@ahrowe/mongo 0.1.2 → 0.2.0

package/docs/CLAUDE.md

@@ -35,8 +35,8 @@ Every `createService<T>(...)` call returns:
   (per-call, or as a `createService` option for a service-wide default) to enforce
   one. `findOne` throws if more than one document matches.
 - `insert(doc | doc[], options?)` — generates `_id` and `createdOn` if not supplied;
-  runs the schema validator (if any) unless `{ skipValidation: true }`; wraps in a
-  transaction unless `{ session }` is passed. Enqueues a `'created'` outbox row
+  runs the schema validator (if any) unless `{ skipValidation: true }`; runs in a
+  transaction (see "Transactions" below). Enqueues a `'created'` outbox row
   (in the same transaction as the write) for each inserted document.
 - `upsertOne(query, updateBody, options?)` — create-or-update a single document.
   If the query matches, behaves like `updateOne` (enqueues `'updated'`). If not,
@@ -49,13 +49,14 @@ Every `createService<T>(...)` call returns:
   is rejected**; use `upsertOne` for create-or-update, `insert` for explicit creates. Adds `updatedOn: new Date()` to the `$set`
   unless `addUpdatedOnField: false` was passed to `createService`. Enqueues an
   `'updated'` outbox row with `{ prevDoc, doc, meta }` for each document that
-  actually changed. `updateMany` always uses the reliable per-document path (see
-  "Events are delivered via a transactional outbox" below).
+  actually changed. For `updateMany`, see "How updateMany/removeMany work" below.
 - `removeOne(query, options?)` and `removeMany(query, options?)` — `removeMany`
   returns removed docs under `results` unless `{ returnRemoved: false }`. Enqueues
   a `'removed'` outbox row with `{ doc, meta }` for each removed document (no row
-  if nothing matched). `removeMany` always uses the reliable per-document path,
-  same as `updateMany`.
+  if nothing matched). For `removeMany`, see "How updateMany/removeMany work" below.
+- Write methods reject `projection` and `includeResultMetadata` (types omit them,
+  and they throw at runtime): validation and outbox events need the full
+  document. Read the fields you need with `find`/`findOne` afterwards.
 - `createIndex(fields, options?)` — `fields` is `IndexSpec<T>`: any top-level key of
   `T` or a dotted path into it (`'users._id'`), following arrays into their element
   type, mapped to `IndexDirection` (`1 | -1 | 'text' | 'hashed' | '2dsphere' | '2d'`).
@@ -65,12 +66,13 @@ Every `createService<T>(...)` call returns:
   having to enumerate that far.
 - `count`, `exists`, `aggregate`, `distinct`, `dropIndex`, `bulkWrite`
   (requires `{ allowBulkwrite: true }` — guards against accidental unvalidated writes).
-  `bulkWrite` does not enqueue outbox rows. `dropIndex(indexName, options?)` drops
+  `bulkWrite` writes no outbox rows, so it throws on a service that emits them.
+  `dropIndex(indexName, options?)` drops
   a single index by name (not by key spec) and does not enqueue outbox rows.
-- `on(event, cb)` / `onPropertiesUpdated(properties, cb)` — subscribe to the
-  service's own `eventBus` (a plain `node:events` `EventEmitter`). Listeners
-  registered here are dispatched to by `startOutboxRelay`, not by the write call
-  itself.
+- `on(event, cb)` / `onPropertiesUpdated(properties, cb)`: register a listener
+  for `'created' | 'updated' | 'removed'` on this collection. Returns a function
+  that removes it. Listeners are called by `startOutboxRelay` (or
+  `dispatchEvent`), not by the write call itself.
 - `withSession(cb)` / `startSession(options?)` — exported from the module (not
   per-service) for manual multi-document transactions across several services.
 - `disconnect()` — exported from the module; closes both MongoDB clients.
@@ -94,8 +96,9 @@ The contract is validator-agnostic — plug in Zod (`schema.safeParse(entity)`),
 directly — writing to the data collection can't be suppressed per-call, so for
 each document actually affected, one row is written to an internal `outbox`
 collection **in the same transaction** as the data write. A separate process,
-`startOutboxRelay({ instanceId, ... })`, claims pending rows (atomic leased
-`findOneAndUpdate`, safe across multiple pods) and dispatches each to whatever's
+`startOutboxRelay({ instanceId, ... })`, claims pending rows (an atomic
+`findOneAndUpdate` that pushes `nextAttemptAt` past a lease, safe across
+multiple pods) and dispatches each to whatever's
 registered via `.on(event, cb)` on that collection's service, retrying with
 backoff and dead-lettering after `maxAttempts`. This means delivery is durable —
 it survives a crash between the write committing and a listener running — and
@@ -108,8 +111,10 @@ const relay = startOutboxRelay({
   instanceId: process.env.HOSTNAME ?? 'local', // required — identifies this lease holder
   pollIntervalMs: 500,    // idle poll interval when the outbox is empty
   batchSize: 20,          // rows claimed+dispatched per tick
-  leaseMs: 30_000,        // how long a claim is held before another instance may reclaim it
+  leaseMs: 30_000,        // how long a claimed row stays hidden from other claims
   maxAttempts: 10,        // attempts before a row is dead-lettered to 'failed'
+  keepDoneMs: 7 * 24 * 60 * 60 * 1000, // how long 'done' rows are kept; Infinity = forever
+  keepFailedMs: 90 * 24 * 60 * 60 * 1000, // how long 'failed' rows are kept for inspection/replay; Infinity = forever
   // backoffMs: (attempt) => ms,  // override the default capped-exponential backoff
   // autoStart: false,            // don't start the poll loop — drive tick()/drain() manually (useful in tests)
 });
@@ -119,6 +124,15 @@ const relay = startOutboxRelay({
 Run exactly one (or more — it's safe across multiple pods) relay per
 deployment that needs events delivered; nothing dispatches them otherwise.
 
+A relay only claims rows for collections that have a service (`createService`)
+in its own process. Rows for any other collection stay `pending` until a relay
+that has that service picks them up, so a relay process without a given
+service can't swallow its events.
+
+A listener must finish within `leaseMs`. Past that, another instance may
+reclaim the row and run it again; the late holder's outcome is then discarded
+(with a warning) instead of overwriting the newer claim's.
+
 Each listener receives `{ prevDoc, doc, meta, context, eventId }` — not just
 `{ prevDoc, doc, meta }`. `eventId` is the outbox row's `_id`, stable across
 retries of the same row; since delivery is **at-least-once**, use it as an
@@ -134,6 +148,16 @@ Set `{ emitOutboxEvents: false }` on `createService` for infra collections with
 no listeners (sequences, the outbox itself, etc.) — writes then take a leaner
 fast path with no pre-reads and no transaction-wrapped outbox insert.
 
+Set `{ toOutboxDoc: (doc) => ... }` on `createService` to decide what a
+collection's outbox rows hold of a document, `doc` and `prevDoc` alike. Rows
+reach every listener, and whatever those store (an audit log, a broker), so a
+secret in a row leaks there. `({ secret, ...hook }) => hook` leaves it out (a
+misspelled field is a type error), and an update that changes only that field
+records no row. To have the change show without the value, return a hash of it
+instead. The function must be pure and must not modify the document it gets. It
+also gets stored documents that were never validated against `T` (older ones,
+`skipValidation` writes), so handle a missing field: a throw fails the write.
+
 If a listener needs request/actor context (e.g. an audit log), register
 `setOutboxContextProvider(fn)` once at startup. It's called **synchronously at
 write time**, in the same transaction as the data write, and the captured value
@@ -154,15 +178,14 @@ const failed = await getOutboxCollection().find({ status: OutboxStatus.Failed })
 // inspect failed[i].lastError, fix the underlying issue, then requeue:
 await getOutboxCollection().updateOne(
   { _id: failed[0]._id },
-  { $set: { status: OutboxStatus.Pending, nextAttemptAt: new Date(), attempts: 0, lastError: null } },
+  { $set: { status: OutboxStatus.Pending, nextAttemptAt: new Date(), attempts: 0, lastError: null, expireAt: null } },
 );
 ```
 
-`getServiceBus(collectionName)` and `getRegisteredCollections()` are the same
-internals the relay itself uses to route a row to its listeners — reach for
-them only if you're building custom tooling around the outbox (a dashboard, a
-manual replay script); normal listener registration is just `.on(...)` on the
-service.
+`getRegisteredCollections()` lists the collections with a service in this
+process (the ones the relay claims rows for). Reach for it only if you're
+building custom tooling around the outbox; normal listener registration is just
+`.on(...)` on the service.
 
 ### Multiple consumers
 
@@ -179,47 +202,59 @@ move is to bridge: run one relay whose only job is to republish each row to a
 message broker (Kafka, SNS, SQS, etc.), and let downstream services subscribe
 there. The outbox guarantees the write reaches the broker exactly once; the
 broker handles fan-out from there. That bridging code lives in your app — this
-package has no broker dependency by design.
+package has no broker dependency by design. On the consuming side, hand each
+received event to `dispatchEvent(collectionName, event)`: it calls the
+listeners registered via `.on(...)` exactly like the relay does (awaits all of
+them, rejects with the first failure so your consumer can retry).
 
 Until a second independent consumer appears, don't add the broker. Bundle the
 relay and all listeners in the same process.
 
 ### Outbox row retention
 
-The relay creates a TTL index that drops `done` (successfully processed) rows
-**7 days** after `processedOn` is set. `failed` rows keep `processedOn: null`
-and are **never** auto-deleted — they accumulate until manually requeued or
-removed, so a steady stream of permanently-failing events will grow the outbox
-collection unbounded until someone looks at it.
-
-## updateMany/removeMany always use the reliable per-document path
-
-Unlike a single native bulk `updateMany`/`deleteMany` call, both methods process
-every matched document one at a time, sequentially (MongoDB doesn't allow
-concurrent operations on one session), inside a single transaction so the whole
-batch still rolls back together (data **and** outbox rows) on any failure. Every
-document is individually validated, and each outbox row's `{ prevDoc, doc }` pair
-comes from its own atomic `findOneAndUpdate`/`findOneAndDelete` — not a diff
-between two independent before/after scans, which could otherwise miss or
-misattribute documents if the matched set changed mid-operation.
-
-This path has **no size cap by default** — pass `{ maxBatchSize }` (per-call, or
-as a `createService` option) to enforce one and fail fast; a single transaction
-processing a very large matched set risks hitting MongoDB's transaction lifetime
-limit (default 60s). Past that limit it throws — use `bulkWrite` for larger jobs
-(no transaction/validation/event guarantees, but no cap either).
-
-Internally this is powered by `src/batchify.ts` (sequential `concurrency: 1`,
-`throwOnError: true` so a per-document failure aborts the whole transaction) — an
-internal implementation detail, not exported from the package.
+When a row is marked `done` or `failed`, the relay sets its `expireAt` from
+`keepDoneMs` (default **7 days**) or `keepFailedMs` (default **90 days**), and a
+TTL index deletes it after that; `Infinity` keeps rows forever. MongoDB's TTL
+task runs every 60 seconds, so deletion isn't instant. Retention is stored per
+row, so changing it needs no index migration; it applies to rows settled from
+then on. A row no relay processes stays until one does. When requeuing a failed row by hand, clear
+`expireAt` (as in the snippet above), or the TTL index may delete it while it
+waits.
+
+## How updateMany/removeMany work
+
+Both collect the matched `_id`s, then work through them in chunks of 500 inside a
+single transaction: per chunk, one read of the current documents, one native
+`updateMany`/`deleteMany` by `_id`, and (for `updateMany`) one read of the
+results, plus one outbox insert. A failure in any chunk rolls back every chunk,
+data **and** outbox rows. Every updated document is validated, and each outbox
+row's `{ prevDoc, doc }` pair is exact: once the transaction has read a document,
+any concurrent write to it makes the transaction fail with a transient write
+conflict, and the whole call is retried.
+
+With `emitOutboxEvents: false`, `updateMany` still validates every document
+(skipping the `prevDoc` reads and outbox rows); only without a validator, or
+with `{ skipValidation: true }`, does it become one native `updateMany`.
+
+There is **no size cap by default**. Pass `{ maxBatchSize }` (per-call, or as a
+`createService` option) to fail fast: a single transaction processing a very
+large matched set risks MongoDB's transaction lifetime limit (default 60s).
+Past the cap it throws before writing anything; it never processes a partial
+batch, so it can't be used to loop. For more than fits one transaction, split
+the work into several calls (see the README for a rerunnable migration loop).
 
 ## Transactions
 
-`insert`/`updateOne`/`updateMany` each wrap themselves in a transaction automatically
-when no `{ session }` is given (via `client.withSession` + `session.withTransaction`).
-Pass an explicit `session` (from `startSession()` or `withSession()`) to compose
-multiple service calls — even across different `createService(...)` instances/collections
-— into one atomic transaction.
+Write methods always run in a transaction, so a data write and its outbox row
+commit together. If the passed `{ session }` has an active transaction, the call
+joins it; otherwise the call runs in its own (on that session, if one was passed).
+To compose several calls, even across different services/collections, into one
+atomic transaction, pass a session **inside** `session.withTransaction(...)`, as in
+the README. A bare session from `startSession()`/`withSession()` gives each call
+its own transaction, not a shared one.
+
+Exceptions: `bulkWrite`, and `removeOne`/`removeMany` on a service with
+`emitOutboxEvents: false`, don't use a transaction.
 
 ## Gotchas (read before integrating)
 
@@ -239,42 +274,23 @@ multiple service calls — even across different `createService(...)` instances/
   somewhere in the deployment.
 - **Two MongoDB clients per `connect()` call** (primary + secondary-preferred reader).
   `preferReadFromSecondary: true` only affects `find`/`findCursor`/`count`/`aggregate`.
-- **`bulkWrite` requires `{ allowBulkwrite: true }`** — there is no other safety net
-  (no schema validation runs on bulk writes).
-- **`updateOne`'s `prevDoc` can be stale** — and so can `updateMany`/`removeMany`'s
-  reliable path, since it reuses the exact same single-document logic
-  (`performSingleUpdate` in `db.ts`). To populate `{ prevDoc, doc }` on the outbox
-  row, the document is read with a separate `findOne` *before* running the atomic
-  `findOneAndUpdate` — a concurrent write landing in that gap means `prevDoc` may
-  not reflect the actual immediately-prior state. The atomic update itself is
-  unaffected (Mongo still applies it correctly); only the `prevDoc` value in the
-  outbox row can be stale. Treat `prevDoc` as best-effort, not authoritative, for
-  anything safety-critical.
-- **`emitOutboxEvents: false` (no-outbox) writes only validate one sampled
-  document on `updateMany`, not every updated document.** For `$set`-with-literal-values
-  updates this is representative (every matched doc lands on the same value for
-  the touched fields), but for operators whose result depends on a document's
-  prior value (`$inc`, `$mul`, `$min`, `$max`, `$push`/`$addToSet`, `$rename`,
-  positional array filters), the same update can be valid for the sampled
-  document and invalid for another — undetected on that fast path. The default
-  reliable path (`emitOutboxEvents: true`, the default) validates every document
-  individually and doesn't have this gap.
+- **`bulkWrite` requires `{ allowBulkwrite: true }`** and a service with
+  `emitOutboxEvents: false` — no schema validation runs on bulk writes, and no
+  outbox rows are written.
 - **Nothing dispatches outbox events unless a relay is running.** Outbox rows
   accumulate (durably, harmlessly) until some process calls `startOutboxRelay`.
   Make sure at least one instance in your deployment runs it, or `.on(...)`
   listeners will never fire.
-- **`createService` called twice for the same collection name reuses the first
-  call's `eventBus` (and logs a warning).** The registry mapping collection name
-  → `eventBus` is process-wide, so without this, a second call's fresh
-  `EventEmitter` would silently replace the first's, orphaning any listeners
-  already registered on it. Reuse means `.on(...)`/`onPropertiesUpdated(...)`
+- **`createService` called twice for the same collection name shares the first
+  call's listeners (and logs a warning).** Listeners are registered per
+  collection name, process-wide, so `.on(...)`/`onPropertiesUpdated(...)`
   registered on *either* instance still receive dispatched events — but other
   per-instance config is **not** shared, and a mismatch there is easy to miss
-  since the shared bus makes everything else look consistent:
+  since the shared listeners make everything else look consistent:
   - **`emitOutboxEvents`** — if one instance has it `false`, writes through
-    that instance never enqueue an outbox row, so listeners on the shared bus
-    silently never fire for those writes specifically (not a bus problem —
-    no event is generated at all). This can be an intentional choice (e.g. a
+    that instance never enqueue an outbox row, so the shared listeners
+    silently never fire for those writes specifically (no event is generated
+    at all). This can be an intentional choice (e.g. a
     backfill/migration instance skipping outbox writes on purpose), so it's
     not something `createService` should reject — just be deliberate about it.
   - **`validate`** — two instances can enforce different schemas on the same
@@ -284,10 +300,24 @@ multiple service calls — even across different `createService(...)` instances/
   - **`addCreatedOnField`/`addUpdatedOnField`** — documents written through
     one instance may end up with `createdOn`/`updatedOn` and documents
     written through the other may not, for the same collection.
+  - **`toOutboxDoc`** — a field one instance keeps out of the outbox reaches
+    the shared listeners through the other. Give every instance the same one.
   Prefer creating each service once (e.g. in a shared module) and importing
   that instance everywhere it's used, unless one of the above divergences is
-  actually what you want. Because the bus is shared, `.eventBus
-  .removeAllListeners()` (or `.off(...)`/`.removeListener(...)`) called on
-  *either* instance also removes listeners registered via the *other*
-  instance — teardown code needs to account for that instead of assuming it
-  only touches its own listeners.
+  actually what you want.
+
+## Upgrading from 0.1
+
+`CHANGELOG.md` (shipped with the package) lists what changed in 0.2.0. One step
+has no place there: once no 0.1 relay runs any more, run this once per database.
+It requeues rows a 0.1 relay left in status `processing` (a relay of this version
+never claims them), and drops the indexes 0.1 created:
+
+```js
+db.outbox.updateMany({ status: 'processing' }, [
+  { $set: { status: 'pending', leasedBy: null, nextAttemptAt: { $ifNull: ['$leasedUntil', '$$NOW'] } } },
+  { $unset: 'leasedUntil' },
+]);
+db.outbox.dropIndex('status_1_nextAttemptAt_1_leasedUntil_1');
+db.outbox.dropIndex('processedOn_1'); // only if it still exists
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ahrowe/mongo",
-  "version": "0.1.2",
+  "version": "0.2.0",
   "type": "module",
   "license": "MIT",
   "description": "Typed MongoDB CRUD service factory with built-in transactions, event hooks, and optional document validation.",
@@ -23,7 +23,8 @@
   "types": "./dist/index.d.ts",
   "files": [
     "dist",
-    "docs/CLAUDE.md"
+    "docs/CLAUDE.md",
+    "CHANGELOG.md"
   ],
   "sideEffects": false,
   "exports": {
@@ -40,6 +41,7 @@
     "mongodb": "^7.0.0"
   },
   "devDependencies": {
+    "@changesets/cli": "^3.0.3",
     "@eslint/js": "^10.0.1",
     "@faker-js/faker": "^10.1.0",
     "@types/lodash": "^4.17.21",
@@ -56,6 +58,7 @@
     "build": "tsc",
     "lint": "eslint .",
     "test": "vitest",
+    "changeset": "changeset",
     "release": "bash scripts/release.sh"
   }
 }

package/dist/batchify.d.ts

@@ -1,36 +0,0 @@
-interface BatchifyOptions<T> {
-    /**
-     * Returns the next batch of data, or an array containing all data (sliced into
-     * batches automatically). When a function: receives the requested batch size
-     * and the number of items already processed.
-     */
-    dataDelivery?: T[] | ((requestedBatchSize: number, alreadyProcessed: number) => T[] | Promise<T[]>);
-    /** Number of items fetched per `dataDelivery` call. Default 1000. */
-    batchSize?: number;
-    /**
-     * Number of items from the current batch processed concurrently. Defaults to
-     * `batchSize` (the whole batch at once). Set to `1` for strictly sequential
-     * processing — required when items share a MongoDB session/transaction, which
-     * doesn't allow concurrent operations.
-     */
-    concurrency?: number;
-    /** Function to run for each item. Receives the item and its global index. */
-    functionToRun?: (data: T, index: number) => void | Promise<void>;
-    /** Invoked after each batch completes, with the total number of items processed so far. */
-    onProgress?: (processedAmount: number) => void | Promise<void>;
-    /** Called when processing an item fails. Receives the failed item and the error. */
-    onError?: (entry: T, err: unknown) => void | Promise<void>;
-    /** If true, stop fetching further batches once any error has occurred. */
-    returnOnError?: boolean;
-    /**
-     * If true, rethrow the first error out of `batchify()` itself once the batch
-     * it occurred in has settled, instead of only invoking `onError`. Use this when
-     * the caller needs the failure to actually propagate (e.g. to abort a transaction).
-     */
-    throwOnError?: boolean;
-    /** Milliseconds to wait after completing each batch before starting the next. */
-    pauseAfterBatch?: number;
-}
-export declare function batchify<T>({ dataDelivery, batchSize, concurrency, functionToRun, onProgress, onError, returnOnError, throwOnError, pauseAfterBatch, }?: BatchifyOptions<T>): Promise<void>;
-export {};
-//# sourceMappingURL=batchify.d.ts.map

package/dist/batchify.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"batchify.d.ts","sourceRoot":"","sources":["../src/batchify.ts"],"names":[],"mappings":"AAAA,UAAU,eAAe,CAAC,CAAC;IACzB;;;;OAIG;IACH,YAAY,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,CAAC,kBAAkB,EAAE,MAAM,EAAE,gBAAgB,EAAE,MAAM,KAAK,CAAC,EAAE,GAAG,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IACpG,qEAAqE;IACrE,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB;;;;;OAKG;IACH,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,6EAA6E;IAC7E,aAAa,CAAC,EAAE,CAAC,IAAI,EAAE,CAAC,EAAE,KAAK,EAAE,MAAM,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IACjE,2FAA2F;IAC3F,UAAU,CAAC,EAAE,CAAC,eAAe,EAAE,MAAM,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/D,oFAAoF;IACpF,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,EAAE,OAAO,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC3D,0EAA0E;IAC1E,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB;;;;OAIG;IACH,YAAY,CAAC,EAAE,OAAO,CAAC;IACvB,iFAAiF;IACjF,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,wBAAsB,QAAQ,CAAC,CAAC,EAAE,EAChC,YAAuB,EACvB,SAAgB,EAChB,WAAW,EACX,aAA+B,EAC/B,UAA4B,EAC5B,OAAyB,EACzB,aAAqB,EACrB,YAAoB,EACpB,eAAmB,GACpB,GAAE,eAAe,CAAC,CAAC,CAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAuDzC"}

package/dist/batchify.js

@@ -1,57 +0,0 @@
-export async function batchify({ dataDelivery = () => [], batchSize = 1000, concurrency, functionToRun = () => undefined, onProgress = () => undefined, onError = () => undefined, returnOnError = false, throwOnError = false, pauseAfterBatch = 0, } = {}) {
-    const effectiveConcurrency = concurrency ?? batchSize;
-    let iteration = 0;
-    let hasError = false;
-    let firstError;
-    let currentBatch = [];
-    const getNextBatch = async () => {
-        if (typeof dataDelivery === 'function') {
-            currentBatch = await dataDelivery(batchSize, iteration * batchSize);
-        }
-        else {
-            currentBatch = dataDelivery.slice(batchSize * iteration, batchSize + batchSize * iteration);
-        }
-    };
-    const runItem = async (entry, globalIndex) => {
-        try {
-            await functionToRun(entry, globalIndex);
-        }
-        catch (error) {
-            hasError = true;
-            if (throwOnError && !firstError)
-                firstError = { error };
-            await onError(entry, error);
-        }
-    };
-    const processCurrentBatch = async () => {
-        for (let start = 0; start < currentBatch.length; start += effectiveConcurrency) {
-            if (firstError)
-                return;
-            const chunk = currentBatch.slice(start, start + effectiveConcurrency);
-            await Promise.allSettled(chunk.map((entry, i) => runItem(entry, batchSize * iteration + start + i)));
-        }
-    };
-    await getNextBatch();
-    await onProgress(0);
-    while (currentBatch.length) {
-        await processCurrentBatch();
-        if (firstError)
-            throw firstError.error;
-        if (currentBatch.length < batchSize) {
-            await onProgress(batchSize * iteration + currentBatch.length);
-            return;
-        }
-        iteration += 1;
-        await onProgress(batchSize * iteration);
-        if (hasError && returnOnError) {
-            return;
-        }
-        await getNextBatch();
-        if (currentBatch.length && pauseAfterBatch) {
-            await new Promise((resolve) => {
-                setTimeout(resolve, pauseAfterBatch);
-            });
-        }
-    }
-}
-//# sourceMappingURL=batchify.js.map

package/dist/batchify.js.map

@@ -1 +0,0 @@
-{"version":3,"file":"batchify.js","sourceRoot":"","sources":["../src/batchify.ts"],"names":[],"mappings":"AAkCA,MAAM,CAAC,KAAK,UAAU,QAAQ,CAAI,EAChC,YAAY,GAAG,GAAG,EAAE,CAAC,EAAE,EACvB,SAAS,GAAG,IAAI,EAChB,WAAW,EACX,aAAa,GAAG,GAAG,EAAE,CAAC,SAAS,EAC/B,UAAU,GAAG,GAAG,EAAE,CAAC,SAAS,EAC5B,OAAO,GAAG,GAAG,EAAE,CAAC,SAAS,EACzB,aAAa,GAAG,KAAK,EACrB,YAAY,GAAG,KAAK,EACpB,eAAe,GAAG,CAAC,MACG,EAAE;IACxB,MAAM,oBAAoB,GAAG,WAAW,IAAI,SAAS,CAAC;IACtD,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,QAAQ,GAAG,KAAK,CAAC;IACrB,IAAI,UAA0C,CAAC;IAE/C,IAAI,YAAY,GAAQ,EAAE,CAAC;IAE3B,MAAM,YAAY,GAAG,KAAK,IAAI,EAAE;QAC9B,IAAI,OAAO,YAAY,KAAK,UAAU,EAAE,CAAC;YACvC,YAAY,GAAG,MAAM,YAAY,CAAC,SAAS,EAAE,SAAS,GAAG,SAAS,CAAC,CAAC;QACtE,CAAC;aAAM,CAAC;YACN,YAAY,GAAG,YAAY,CAAC,KAAK,CAAC,SAAS,GAAG,SAAS,EAAE,SAAS,GAAG,SAAS,GAAG,SAAS,CAAC,CAAC;QAC9F,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,OAAO,GAAG,KAAK,EAAE,KAAQ,EAAE,WAAmB,EAAE,EAAE;QACtD,IAAI,CAAC;YACH,MAAM,aAAa,CAAC,KAAK,EAAE,WAAW,CAAC,CAAC;QAC1C,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,QAAQ,GAAG,IAAI,CAAC;YAChB,IAAI,YAAY,IAAI,CAAC,UAAU;gBAAE,UAAU,GAAG,EAAE,KAAK,EAAE,CAAC;YACxD,MAAM,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC9B,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,mBAAmB,GAAG,KAAK,IAAI,EAAE;QACrC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,YAAY,CAAC,MAAM,EAAE,KAAK,IAAI,oBAAoB,EAAE,CAAC;YAC/E,IAAI,UAAU;gBAAE,OAAO;YACvB,MAAM,KAAK,GAAG,YAAY,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,GAAG,oBAAoB,CAAC,CAAC;YACtE,MAAM,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,KAAK,EAAE,SAAS,GAAG,SAAS,GAAG,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;QACvG,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,YAAY,EAAE,CAAC;IACrB,MAAM,UAAU,CAAC,CAAC,CAAC,CAAC;IACpB,OAAO,YAAY,CAAC,MAAM,EAAE,CAAC;QAC3B,MAAM,mBAAmB,EAAE,CAAC;QAC5B,IAAI,UAAU;YAAE,MAAM,UAAU,CAAC,KAAK,CAAC;QACvC,IAAI,YAAY,CAAC,MAAM,GAAG,SAAS,EAAE,CAAC;YACpC,MAAM,UAAU,CAAC,SAAS,GAAG,SAAS,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC;YAC9D,OAAO;QACT,CAAC;QACD,SAAS,IAAI,CAAC,CAAC;QACf,MAAM,UAAU,CAAC,SAAS,GAAG,SAAS,CAAC,CAAC;QACxC,IAAI,QAAQ,IAAI,aAAa,EAAE,CAAC;YAC9B,OAAO;QACT,CAAC;QACD,MAAM,YAAY,EAAE,CAAC;QACrB,IAAI,YAAY,CAAC,MAAM,IAAI,eAAe,EAAE,CAAC;YAC3C,MAAM,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE;gBAC5B,UAAU,CAAC,OAAO,EAAE,eAAe,CAAC,CAAC;YACvC,CAAC,CAAC,CAAC;QACL,CAAC;IACH,CAAC;AACH,CAAC"}