@flowscripter/pluggable-io-framework 1.2.7 → 2.0.1

package/README.md

@@ -5,202 +5,71 @@
 [![docs](https://img.shields.io/badge/docs-API-blue)](https://flowscripter.github.io/pluggable-io-framework/index.html)
 [![license: MIT](https://img.shields.io/github/license/flowscripter/pluggable-io-framework)](https://github.com/flowscripter/pluggable-io-framework/blob/main/LICENSE)
 
-> A pluggable source/sink IO framework using https://github.com/flowscripter/dynamic-plugin-framework
+> A pluggable source/sink IO framework using
+> [dynamic-plugin-framework](https://github.com/flowscripter/dynamic-plugin-framework)
 
 ## Key Features
 
-- Discovers and instantiates source/sink provider plugins (implementing the
-  `IOProviderFactory`/`IOProvider` contract from
-  [pluggable-io-framework-api](https://github.com/flowscripter/pluggable-io-framework-api))
-  via
-  [dynamic-plugin-framework](https://github.com/flowscripter/dynamic-plugin-framework),
-  including config validation against each plugin's Zod schema.
-- Copy/move orchestration:
-  - Uses a provider's `directCopy`/`directMove` when
-    `canDirectTransfer` reports the source and sink are the same
-    underlying provider (e.g. same filesystem mount, same object storage
-    bucket).
-  - Otherwise transfers via multipart (when both sides support it and file
-    size crosses a configurable threshold) or plain streaming.
-  - Multipart part size is negotiated between source and sink via
-    `getPartSizeConstraints` (e.g. reconciling S3's minimum part size and
-    10000-part cap on both ends) - see [Part-size negotiation](#part-size-negotiation).
-  - A folder `sourcePath` recurses: a folder-aware `directCopy`/`directMove`
-    when the source declares `supportsRecursiveDirectTransfer`, otherwise a
-    concurrency-bounded listing-driven transfer following `cp -r` target
-    semantics - see [Recursive copy/move](#recursive-copymove).
-  - Concurrent multipart parts and recursive entries are bounded by a shared
-    `ConcurrencyLimiter` - see [Concurrency](#concurrency).
-  - The non-direct transfer path is retried on `TransientIOError` - see
-    [Retries](#retries).
-  - Reports progress via a global `TelemetryHooks` callback, tagged with a
-    per-operation correlation id; multipart parts and recursive entries
-    report their own child stream tagged with `parentOperationId`.
-- Stream decorators:
-  - `seekable` wraps a handle that supports `RangeReadable` with a single
-    logical stream whose read position can be jumped via `seek(offset)`,
-    instead of requiring a fresh stream per range.
-  - `locallyCached` wraps a `StreamHandle` factory so the underlying source
-    is read at most once - later calls replay cached chunks in memory
-    without touching the source again.
-- See
-  [io-plugin-filesystem](https://github.com/flowscripter/io-plugin-filesystem)
-  for a reference local filesystem source/sink plugin.
-
-## Bun Module Usage
-
-Add the module:
-
-`bun add @flowscripter/pluggable-io-framework`
-
-Discover and use a provider:
+- Protocol-aware locations: `file:///data/a.txt`, `s3://bucket/key`,
+  `https://host/a` and composite schemes such as `tams+https:` select the
+  installed provider plugin for that protocol. Bare paths default to `file`.
+- A `ProviderRegistry` keyed by protocol and payload kind, with more than one
+  implementation of a protocol installed side by side (e.g. JS and native
+  `file` providers).
+- Negotiation of the payload kind, memory domain and payload type between
+  source and sink, falling back to the cheapest registered payload converter
+  plugin.
+- `copy`/`move` over explicit entry, container and glob pattern targets,
+  with `cp -r` semantics for containers.
+- Direct provider-to-provider transfers, multipart transfers with negotiated
+  part sizes, zero-copy lease transfers into sink-provided buffers, or plain
+  streaming - chosen automatically.
+- Graceful stop and cancellation, per-part retries, resumable writes and
+  automatic reconnection of live sources with gap reporting.
+- Concurrency-bounded recursive and pattern transfers with progress
+  telemetry.
+- Provider composition: a provider can resolve other installed providers.
+- Stream decorators: `seekable` and `locallyCached`.
+- Bun based, written in TypeScript, based on native JavaScript modules.
+
+## Usage Examples
+
+- [flowscripter-io-cli](https://github.com/flowscripter/flowscripter-io-cli)
+  is a CLI application based on this framework.
+- [io-plugin-filesystem](https://github.com/flowscripter/io-plugin-filesystem)
+  is the reference `file` provider plugin.
 
 ```typescript
 import {
   DefaultPluginManager,
   LocalFolderPluginRepository,
 } from "@flowscripter/dynamic-plugin-framework";
-import { ProviderRegistry, copy } from "@flowscripter/pluggable-io-framework";
+import { copy, ProviderRegistry } from "@flowscripter/pluggable-io-framework";
 
-const pluginManager = new DefaultPluginManager([new LocalFolderPluginRepository("./plugins")]);
-const registry = new ProviderRegistry(pluginManager);
+const registry = new ProviderRegistry(
+  new DefaultPluginManager([new LocalFolderPluginRepository("./plugins")]),
+);
 await registry.discover();
 
-const [extension] = await registry.listAvailableProviders();
-const provider = await registry.createProvider(extension.extensionHandle, { rootPath: "/data" });
+const { source, dest, options } = await registry.createProvidersForTransfer(
+  "file:///data/in",
+  "s3://bucket/out",
+);
 
-await copy(provider, "a.txt", provider, "b.txt", {
+const result = await copy(source.provider, source.target, dest.provider, dest.target, {
+  ...options,
   telemetry: { onProgress: (event) => console.log(event) },
 });
+console.log(result.path);
 ```
 
-## Usage Example
-
-The following example project is available:
-
-- [flowscripter-io-cli](https://github.com/flowscripter/flowscripter-io-cli) is
-  an example CLI application based on this framework.
-
-## Part-size negotiation
-
-Before a multipart transfer, `copy()`/`move()` call `getPartSizeConstraints(totalSize)`
-on both `source` and `sink` (when implemented - a missing implementation is
-treated as unconstrained) and reconcile the two into a single part size:
-the larger of the two minimums, clamped to the smaller of the two maximums,
-bumped up if needed so the total number of parts never exceeds the smaller
-of the two `maxParts` (e.g. S3's 10000-part cap on a very large file). If
-the reconciled bounds are mutually infeasible, the transfer falls back to
-plain streaming rather than failing.
-
-## Recursive copy/move
-
-When `sourcePath` is a folder, `copy()`/`move()` follow `cp -r`/`mv`
-semantics for the destination: a `destPath` that doesn't exist becomes the
-copy itself; an existing folder gets the source nested inside it as
-`destPath/<source-basename>`; an existing file is rejected.
-
-If the source provider declares `supportsRecursiveDirectTransfer: true` and
-is direct-transfer-eligible with the sink, the whole folder is handed to a
-single `directCopy`/`directMove` call. Otherwise the folder is listed
-(`source.list(path, { recursive: true })`) and each entry is transferred
-individually - folders via the optional `createFolder` capability (so empty
-folders are preserved), files via the normal single-file path (which may
-still resolve to a per-file `directCopy`/`directMove`). For a recursive
-move without any direct-transfer capability, every entry is copied first;
-the source folder is deleted as a single recursive `delete()` call only
-after every entry has succeeded.
-
-## Concurrency
-
-Multipart parts and recursive-copy/move entries are bounded by a
-`ConcurrencyLimiter` (`TransferOptions.concurrencyLimiter`), defaulting to
-the exported `defaultConcurrencyLimiter` singleton - so two `copy()`/`move()`
-calls in the same process share one cap unless a caller passes its own
-isolated instance:
-
-```typescript
-import { defaultConcurrencyLimiter } from "@flowscripter/pluggable-io-framework";
-
-defaultConcurrencyLimiter.setMaxConcurrency(8);
-```
-
-## Retries
-
-The non-direct transfer path (plain streaming, multipart, and the
-copy-then-delete in a non-direct `move()`) is retried via `TransferOptions.retry`
-(`{ maxRetries, backoffMs? }`, defaulting to `{ maxRetries: 3 }`) whenever a
-provider throws `TransientIOError` from `pluggable-io-framework-api`. Any
-other error - including a plain, unwrapped `Error` - is treated as
-non-retryable. Direct provider calls (`directCopy`/`directMove`) are never
-retried by the framework, since backends like the AWS SDK already retry
-internally. The underlying `withRetry` utility is exported standalone for
-other call sites.
-
-## Development
-
-Install dependencies:
-
-`bun install`
-
-Build (produces `dist/` for Node.js and TypeScript consumers; Bun uses raw source directly):
-
-`bun run build`
-
-Test:
-
-`bun test`
-
-Format:
-
-`bunx oxfmt`
-
-Lint:
-
-`bunx oxlint index.ts src/ tests/`
-
-Generate HTML API Documentation:
-
-`bunx typedoc index.ts`
-
-## Documentation
-
-### Overview
-
-```mermaid
-sequenceDiagram
-    participant Host
-    participant ProviderRegistry
-    participant PluginManager
-    participant Source as IOProvider (source)
-    participant Sink as IOProvider (sink)
-
-    Host->>ProviderRegistry: discover()
-    ProviderRegistry->>PluginManager: registerExtensions(extensionPoint)
-    Host->>ProviderRegistry: createProvider(handle, config)
-    ProviderRegistry->>PluginManager: instantiate(handle)
-    ProviderRegistry-->>Host: IOProvider
-
-    Host->>Source: copy(source, path, sink, path)
-    alt path is a folder
-        alt supportsRecursiveDirectTransfer
-            Source->>Sink: directCopy(folderPath, folderPath)
-        else
-            Source-->>Sink: list + transfer entries (bounded by ConcurrencyLimiter)
-        end
-    else canDirectTransfer
-        Source->>Sink: directCopy(path, path)
-    else multipart eligible (negotiated part size)
-        Source-->>Sink: transfer Parts concurrently (bounded, retried on TransientIOError)
-    else
-        Source-->>Sink: stream ChunkRefs (retried on TransientIOError)
-    end
-```
-
-### API
-
-Link to auto-generated API docs:
+## Further Details
 
-[API Documentation](https://flowscripter.github.io/pluggable-io-framework/index.html)
+- [Key Concepts](./README/key-concepts.md)
+- [Implementation Details](./README/implementation-details.md)
+- [Plugins](./README/plugins.md)
+- [Development](./README/development.md)
+- [API Documentation](https://flowscripter.github.io/pluggable-io-framework/index.html)
 
 ## License
 

package/dist/index.d.ts

@@ -1,7 +1,14 @@
-export * from "./src/ConcurrencyLimiter.ts";
-export * from "./src/ProviderRegistry.ts";
-export * from "./src/copyMove.ts";
+export * from "./src/concurrency/ConcurrencyLimiter.ts";
+export * from "./src/concurrency/mapAsyncIterableConcurrently.ts";
 export * from "./src/decorators/locallyCached.ts";
 export * from "./src/decorators/seekable.ts";
-export * from "./src/withRetry.ts";
+export * from "./src/registry/detectProtocol.ts";
+export * from "./src/registry/ProviderRegistry.ts";
+export * from "./src/retry/RetryOptions.ts";
+export * from "./src/retry/withRetry.ts";
+export * from "./src/transfer/copyMove.ts";
+export * from "./src/transfer/rangeReadableMultipartReader.ts";
+export * from "./src/transfer/TransferOptions.ts";
+export * from "./src/transfer/TransferResult.ts";
+export * from "./src/util/globToRegex.ts";
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../index.ts"],"names":[],"mappings":"AAAA,cAAc,6BAA6B,CAAC;AAC5C,cAAc,2BAA2B,CAAC;AAC1C,cAAc,mBAAmB,CAAC;AAClC,cAAc,mCAAmC,CAAC;AAClD,cAAc,8BAA8B,CAAC;AAC7C,cAAc,oBAAoB,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../index.ts"],"names":[],"mappings":"AAAA,cAAc,yCAAyC,CAAC;AACxD,cAAc,mDAAmD,CAAC;AAClE,cAAc,mCAAmC,CAAC;AAClD,cAAc,8BAA8B,CAAC;AAC7C,cAAc,kCAAkC,CAAC;AACjD,cAAc,oCAAoC,CAAC;AACnD,cAAc,6BAA6B,CAAC;AAC5C,cAAc,0BAA0B,CAAC;AACzC,cAAc,4BAA4B,CAAC;AAC3C,cAAc,gDAAgD,CAAC;AAC/D,cAAc,mCAAmC,CAAC;AAClD,cAAc,kCAAkC,CAAC;AACjD,cAAc,2BAA2B,CAAC"}

package/dist/index.js

@@ -1,6 +1,13 @@
-export * from "./src/ConcurrencyLimiter.js";
-export * from "./src/ProviderRegistry.js";
-export * from "./src/copyMove.js";
+export * from "./src/concurrency/ConcurrencyLimiter.js";
+export * from "./src/concurrency/mapAsyncIterableConcurrently.js";
 export * from "./src/decorators/locallyCached.js";
 export * from "./src/decorators/seekable.js";
-export * from "./src/withRetry.js";
+export * from "./src/registry/detectProtocol.js";
+export * from "./src/registry/ProviderRegistry.js";
+export * from "./src/retry/RetryOptions.js";
+export * from "./src/retry/withRetry.js";
+export * from "./src/transfer/copyMove.js";
+export * from "./src/transfer/rangeReadableMultipartReader.js";
+export * from "./src/transfer/TransferOptions.js";
+export * from "./src/transfer/TransferResult.js";
+export * from "./src/util/globToRegex.js";

package/dist/src/concurrency/AsyncChannel.d.ts

@@ -0,0 +1,9 @@
+/** Minimal single-consumer async FIFO queue. */
+export declare class AsyncChannel<T> implements AsyncIterable<T> {
+    #private;
+    push(item: T): void;
+    close(): void;
+    fail(error: unknown): void;
+    [Symbol.asyncIterator](): AsyncIterator<T>;
+}
+//# sourceMappingURL=AsyncChannel.d.ts.map

package/dist/src/concurrency/AsyncChannel.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"AsyncChannel.d.ts","sourceRoot":"","sources":["../../../src/concurrency/AsyncChannel.ts"],"names":[],"mappings":"AAAA,gDAAgD;AAChD,qBAAa,YAAY,CAAC,CAAC,CAAE,YAAW,aAAa,CAAC,CAAC,CAAC;;IAQtD,IAAI,CAAC,IAAI,EAAE,CAAC,GAAG,IAAI,CAOlB;IAED,KAAK,IAAI,IAAI,CASZ;IAED,IAAI,CAAC,KAAK,EAAE,OAAO,GAAG,IAAI,CAIzB;IAeD,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,aAAa,CAAC,CAAC,CAAC,CAEzC;CACF"}

package/dist/src/concurrency/AsyncChannel.js

@@ -0,0 +1,48 @@
+/** Minimal single-consumer async FIFO queue. */
+export class AsyncChannel {
+    #items = [];
+    #waiters = [];
+    #closed = false;
+    #hasError = false;
+    #error;
+    push(item) {
+        const waiter = this.#waiters.shift();
+        if (waiter) {
+            waiter.resolve({ value: item, done: false });
+        }
+        else {
+            this.#items.push(item);
+        }
+    }
+    close() {
+        this.#closed = true;
+        for (const waiter of this.#waiters.splice(0)) {
+            if (this.#hasError) {
+                waiter.reject(this.#error);
+            }
+            else {
+                waiter.resolve({ value: undefined, done: true });
+            }
+        }
+    }
+    fail(error) {
+        this.#hasError = true;
+        this.#error = error;
+        this.close();
+    }
+    async #next() {
+        if (this.#items.length > 0) {
+            return { value: this.#items.shift(), done: false };
+        }
+        if (this.#closed) {
+            if (this.#hasError) {
+                throw this.#error;
+            }
+            return { value: undefined, done: true };
+        }
+        return new Promise((resolve, reject) => this.#waiters.push({ resolve, reject }));
+    }
+    [Symbol.asyncIterator]() {
+        return { next: () => this.#next() };
+    }
+}

package/dist/src/{ConcurrencyLimiter.d.ts → concurrency/ConcurrencyLimiter.d.ts}

@@ -16,18 +16,7 @@ export declare class ConcurrencyLimiter {
  * Process-wide default limiter, used by `copy()`/`move()` whenever
  * `TransferOptions.concurrencyLimiter` is omitted - so any two calls in the
  * same process share one cap unless a caller explicitly opts out with its
- * own instance (e.g. for test isolation).
+ * own instance.
  */
 export declare const defaultConcurrencyLimiter: ConcurrencyLimiter;
-/**
- * Pulls items from `source` and runs `fn` on each, bounded by `limiter`:
- * exactly `limiter.maxConcurrency` worker loops each pull-then-process one
- * item at a time through `limiter.run`, so at most that many items are ever
- * pulled/in-flight at once (important for sources like a multipart reader
- * where pulling an item eagerly opens a file/network handle). Results are
- * yielded in completion order, not source order. If `fn` throws, no new
- * items are pulled but already-in-flight ones are allowed to finish before
- * the error is rethrown from the returned iterable.
- */
-export declare function mapAsyncIterableConcurrently<T, R>(source: AsyncIterable<T>, fn: (item: T) => Promise<R>, limiter: ConcurrencyLimiter): AsyncIterable<R>;
 //# sourceMappingURL=ConcurrencyLimiter.d.ts.map

package/dist/src/concurrency/ConcurrencyLimiter.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"ConcurrencyLimiter.d.ts","sourceRoot":"","sources":["../../../src/concurrency/ConcurrencyLimiter.ts"],"names":[],"mappings":"AAEA;;;;;;GAMG;AACH,qBAAa,kBAAkB;;IAK7B,YAAmB,cAAc,EAAE,MAAM,EAGxC;IAED,IAAW,cAAc,IAAI,MAAM,CAElC;IAEM,iBAAiB,CAAC,cAAc,EAAE,MAAM,GAAG,IAAI,CAIrD;IAEY,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC,CAQpD;CA0BF;AAED;;;;;GAKG;AACH,eAAO,MAAM,yBAAyB,oBAAkD,CAAC"}

package/dist/src/concurrency/ConcurrencyLimiter.js

@@ -0,0 +1,64 @@
+const DEFAULT_MAX_CONCURRENCY = 4;
+/**
+ * A semaphore bounding how many `run()` callbacks execute at once. A single
+ * shared instance (see {@link defaultConcurrencyLimiter}) enforces its cap
+ * *globally* across concurrently invoked `copy()`/`move()` calls, not just
+ * within one call - two folder copies started at the same time and sharing
+ * a limiter will never together exceed its `maxConcurrency`.
+ */
+export class ConcurrencyLimiter {
+    #maxConcurrency;
+    #active = 0;
+    #queue = [];
+    constructor(maxConcurrency) {
+        ConcurrencyLimiter.#validate(maxConcurrency);
+        this.#maxConcurrency = maxConcurrency;
+    }
+    get maxConcurrency() {
+        return this.#maxConcurrency;
+    }
+    setMaxConcurrency(maxConcurrency) {
+        ConcurrencyLimiter.#validate(maxConcurrency);
+        this.#maxConcurrency = maxConcurrency;
+        this.#drain();
+    }
+    async run(fn) {
+        await this.#acquire();
+        try {
+            return await fn();
+        }
+        finally {
+            this.#active -= 1;
+            this.#drain();
+        }
+    }
+    #acquire() {
+        if (this.#active < this.#maxConcurrency) {
+            this.#active += 1;
+            return Promise.resolve();
+        }
+        return new Promise((resolve) => {
+            this.#queue.push(() => {
+                this.#active += 1;
+                resolve();
+            });
+        });
+    }
+    #drain() {
+        while (this.#active < this.#maxConcurrency && this.#queue.length > 0) {
+            this.#queue.shift()?.();
+        }
+    }
+    static #validate(maxConcurrency) {
+        if (!Number.isInteger(maxConcurrency) || maxConcurrency < 1) {
+            throw new Error(`maxConcurrency must be a positive integer, got ${maxConcurrency}`);
+        }
+    }
+}
+/**
+ * Process-wide default limiter, used by `copy()`/`move()` whenever
+ * `TransferOptions.concurrencyLimiter` is omitted - so any two calls in the
+ * same process share one cap unless a caller explicitly opts out with its
+ * own instance.
+ */
+export const defaultConcurrencyLimiter = new ConcurrencyLimiter(DEFAULT_MAX_CONCURRENCY);

package/dist/src/concurrency/mapAsyncIterableConcurrently.d.ts

@@ -0,0 +1,13 @@
+import type { ConcurrencyLimiter } from "./ConcurrencyLimiter.ts";
+/**
+ * Pulls items from `source` and runs `fn` on each, bounded by `limiter`:
+ * exactly `limiter.maxConcurrency` worker loops each pull-then-process one
+ * item at a time through `limiter.run`, so at most that many items are ever
+ * pulled/in-flight at once (important for sources like a multipart reader
+ * where pulling an item eagerly opens a file/network handle). Results are
+ * yielded in completion order, not source order. If `fn` throws, no new
+ * items are pulled but already-in-flight ones are allowed to finish before
+ * the error is rethrown from the returned iterable.
+ */
+export declare function mapAsyncIterableConcurrently<T, R>(source: AsyncIterable<T>, fn: (item: T) => Promise<R>, limiter: ConcurrencyLimiter): AsyncIterable<R>;
+//# sourceMappingURL=mapAsyncIterableConcurrently.d.ts.map

package/dist/src/concurrency/mapAsyncIterableConcurrently.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"mapAsyncIterableConcurrently.d.ts","sourceRoot":"","sources":["../../../src/concurrency/mapAsyncIterableConcurrently.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,yBAAyB,CAAC;AAElE;;;;;;;;;GASG;AACH,wBAAgB,4BAA4B,CAAC,CAAC,EAAE,CAAC,EAC/C,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC,EACxB,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,EAC3B,OAAO,EAAE,kBAAkB,GAC1B,aAAa,CAAC,CAAC,CAAC,CAgDlB"}

package/dist/src/concurrency/mapAsyncIterableConcurrently.js

@@ -0,0 +1,61 @@
+import { AsyncChannel } from "./AsyncChannel.js";
+/**
+ * Pulls items from `source` and runs `fn` on each, bounded by `limiter`:
+ * exactly `limiter.maxConcurrency` worker loops each pull-then-process one
+ * item at a time through `limiter.run`, so at most that many items are ever
+ * pulled/in-flight at once (important for sources like a multipart reader
+ * where pulling an item eagerly opens a file/network handle). Results are
+ * yielded in completion order, not source order. If `fn` throws, no new
+ * items are pulled but already-in-flight ones are allowed to finish before
+ * the error is rethrown from the returned iterable.
+ */
+export function mapAsyncIterableConcurrently(source, fn, limiter) {
+    const channel = new AsyncChannel();
+    const iterator = source[Symbol.asyncIterator]();
+    let stopped = false;
+    let firstError;
+    let errorRecorded = false;
+    function recordError(error) {
+        stopped = true;
+        if (!errorRecorded) {
+            errorRecorded = true;
+            firstError = error;
+        }
+    }
+    async function worker() {
+        for (;;) {
+            if (stopped)
+                return;
+            let next;
+            try {
+                next = await iterator.next();
+            }
+            catch (error) {
+                recordError(error);
+                return;
+            }
+            if (next.done)
+                return;
+            const value = next.value;
+            try {
+                const result = await limiter.run(() => fn(value));
+                channel.push(result);
+            }
+            catch (error) {
+                recordError(error);
+                return;
+            }
+        }
+    }
+    void (async () => {
+        const workerCount = Math.max(1, limiter.maxConcurrency);
+        await Promise.all(Array.from({ length: workerCount }, () => worker()));
+        if (errorRecorded) {
+            channel.fail(firstError);
+        }
+        else {
+            channel.close();
+        }
+    })();
+    return channel;
+}

package/dist/src/decorators/locallyCached.d.ts

@@ -1,15 +1,14 @@
-import type { ChunkKind, StreamHandle } from "@flowscripter/pluggable-io-framework-api";
+import type { PayloadKind, StreamHandle } from "@flowscripter/pluggable-io-framework-api";
 /**
- * Wraps a `StreamHandle` factory (e.g. `() => provider.getReadableStream(path)`)
- * so the underlying source is read at most once - the first call drains the
- * source while caching every chunk in memory; every subsequent call replays
- * the cached chunks without touching the source again.
+ * A `StreamOpenerDecorator`: wraps a `StreamHandle` opener so the
+ * underlying source is read at most once - the first call drains the
+ * source while caching every item in memory; every subsequent call replays
+ * the cached items without touching the source again.
  *
  * A plain `StreamHandle` only exposes a single one-shot `ReadableStream`, so
- * caching can't be a `StreamDecorator<K, C>` operating on an already-open
- * handle (there would be nothing left to re-read on a second call) - it has
- * to intercept the *open* operation itself, provider-agnostic regardless of
- * what kind of source is behind it.
+ * caching can't be a `StreamDecorator` operating on an already-open handle
+ * (there would be nothing left to re-read on a second call) - it has to
+ * intercept the *open* operation itself.
  */
-export declare function locallyCached<K extends ChunkKind>(open: () => Promise<StreamHandle<K>>): () => Promise<StreamHandle<K>>;
+export declare function locallyCached<K extends PayloadKind>(open: () => Promise<StreamHandle<K>>): () => Promise<StreamHandle<K>>;
 //# sourceMappingURL=locallyCached.d.ts.map

package/dist/src/decorators/locallyCached.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"locallyCached.d.ts","sourceRoot":"","sources":["../../../src/decorators/locallyCached.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,SAAS,EAET,YAAY,EACb,MAAM,0CAA0C,CAAC;AAElD;;;;;;;;;;;GAWG;AACH,wBAAgB,aAAa,CAAC,CAAC,SAAS,SAAS,EAC/C,IAAI,EAAE,MAAM,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,GACnC,MAAM,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,CAuChC"}
+{"version":3,"file":"locallyCached.d.ts","sourceRoot":"","sources":["../../../src/decorators/locallyCached.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAEV,WAAW,EACX,YAAY,EACb,MAAM,0CAA0C,CAAC;AAElD;;;;;;;;;;GAUG;AACH,wBAAgB,aAAa,CAAC,CAAC,SAAS,WAAW,EACjD,IAAI,EAAE,MAAM,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,GACnC,MAAM,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,CAgDhC"}

package/dist/src/decorators/locallyCached.js

@@ -1,48 +1,55 @@
 /**
- * Wraps a `StreamHandle` factory (e.g. `() => provider.getReadableStream(path)`)
- * so the underlying source is read at most once - the first call drains the
- * source while caching every chunk in memory; every subsequent call replays
- * the cached chunks without touching the source again.
+ * A `StreamOpenerDecorator`: wraps a `StreamHandle` opener so the
+ * underlying source is read at most once - the first call drains the
+ * source while caching every item in memory; every subsequent call replays
+ * the cached items without touching the source again.
  *
  * A plain `StreamHandle` only exposes a single one-shot `ReadableStream`, so
- * caching can't be a `StreamDecorator<K, C>` operating on an already-open
- * handle (there would be nothing left to re-read on a second call) - it has
- * to intercept the *open* operation itself, provider-agnostic regardless of
- * what kind of source is behind it.
+ * caching can't be a `StreamDecorator` operating on an already-open handle
+ * (there would be nothing left to re-read on a second call) - it has to
+ * intercept the *open* operation itself.
  */
 export function locallyCached(open) {
     let cache;
-    function replay(chunks) {
+    function replay(items) {
         return new ReadableStream({
             start(controller) {
-                for (const chunk of chunks)
-                    controller.enqueue(chunk);
+                for (const item of items)
+                    controller.enqueue(item);
                 controller.close();
             },
         });
     }
+    function describe(handle, stream) {
+        return {
+            kind: handle.kind,
+            stream,
+            bounded: handle.bounded,
+            payloadType: handle.payloadType,
+        };
+    }
     return async () => {
         if (cache) {
-            return { kind: cache.kind, stream: replay(cache.chunks) };
+            return describe(cache.handle, replay(cache.items));
         }
         const handle = await open();
         const reader = handle.stream.getReader();
-        const chunks = [];
+        const items = [];
         const stream = new ReadableStream({
             async pull(controller) {
                 const { done, value } = await reader.read();
                 if (done) {
-                    cache = { kind: handle.kind, chunks };
+                    cache = { handle, items };
                     controller.close();
                     return;
                 }
-                chunks.push(value);
+                items.push(value);
                 controller.enqueue(value);
             },
             cancel(reason) {
                 return reader.cancel(reason);
             },
         });
-        return { kind: handle.kind, stream };
+        return describe(handle, stream);
     };
 }

package/dist/src/decorators/seekable.d.ts

@@ -1,7 +1,7 @@
-import type { ChunkKind, RangeReadable, Seekable, StreamHandle } from "@flowscripter/pluggable-io-framework-api";
+import type { PayloadKind, RangeReadable, Seekable, StreamHandle } from "@flowscripter/pluggable-io-framework-api";
 /**
- * Wraps a handle that already supports {@link RangeReadable} (arbitrary
- * byte-range reads) with a {@link Seekable} capability: a single logical
+ * Wraps a handle that already supports `RangeReadable` (arbitrary
+ * byte-range reads) with a `Seekable` capability: a single logical
  * stream whose read position can be jumped via `seek(offset)`, rather than
  * requiring the caller to open a fresh stream per range.
  *
@@ -9,5 +9,5 @@ import type { ChunkKind, RangeReadable, Seekable, StreamHandle } from "@flowscri
  * in flight - like any seekable stream, seeking and reading are sequential,
  * not concurrent, operations on the same handle.
  */
-export declare function seekable<K extends ChunkKind>(handle: StreamHandle<K> & RangeReadable<K>): StreamHandle<K> & Seekable;
+export declare function seekable<K extends PayloadKind>(handle: StreamHandle<K> & RangeReadable<K>): StreamHandle<K> & Seekable;
 //# sourceMappingURL=seekable.d.ts.map