@poe-platform/safe-fs 0.1.728 → 0.1.729

package/README.md

@@ -238,7 +238,7 @@ There are no package environment variables, implicit credentials, or automatic `
 | Memory / read-only | Memory accepts independent optional `maxFileBytes`, `maxRetainedBytes`, `maxMetadataUnits` and `maxBytes` quotas, all unlimited by default. Read-only takes the backing filesystem, without an options object. |
 | Real | Required `root`: existing absolute host directory; the constructor/factory also accepts the root string directly. |
 | Mount | Required `root`: fallback filesystem. `mounts` defaults to `{}` and maps absolute virtual paths to filesystems. |
-| Overlay | Required `upper` and `lower`; `maxBufferBytes` is unlimited unless configured. |
+| Overlay | Required `upper` and `lower`; `maxBufferBytes` is unlimited unless configured; it and per-read `maxBytes` accept explicit `Infinity`. |
 | Quota | `withFileSystemQuota` requires a nonnegative safe-integer `maxBytes`. It serializes mutations and counts files, symlinks, copies, hard links, truncation, and streaming writes. |
 | Node bridge | `cwd` defaults to `/`, must be an absolute virtual path; optional lifetime `signal` and trusted `readFileMaxBytes` cap forwarded to backend reads before copy/decode. |
 | Portable bridge | Same `cwd`, `signal` and `readFileMaxBytes`, plus required `codec` with `isEncoding`, `encode`, and `decode` functions. |
@@ -260,7 +260,7 @@ Every raw filesystem operation accepts an optional `signal`. Additional fields a
 
 `access` takes a separate mode bitmask from `ACCESS_MODES`. `chmod` takes a mode, `utimes` takes millisecond timestamps, and `truncate` takes a byte length (default 0). For conditional chmod, check `capabilitiesFor(path, { conditionalChmod: true })` (or `capabilities`) and require `conditionalChmod: true`; supply `parent`, `expected`, and complete root-to-parent `ancestors` together. An optional mutation-free `commitGuard` must return literal `true` synchronously. Memory validates at its metadata commit; mount and supported Memory overlays preserve wrapper ancestry. Real uses its existing externally isolated host-tree boundary and does not prevent races with other processes. Backend limits still apply. Node-shaped bridge methods translate their own options rather than accepting these raw option objects; see the [bridge signatures](src/bridge/filesystem.ts).
 
-`collectBytes(source, { maxBytes, maxMemoryBytes, signal })` snapshots streamed chunks into one growing buffer. `maxMemoryBytes` limits owned capacity, the current input's full backing buffer, and overlapping allocations during growth; exhaustion throws `EFBIG`. Browser and Worker bundles additionally share a fixed 32 MiB budget across active collectors, even when byte limits are omitted. The returned view may retain geometric spare capacity. This budget covers collection, not caller-retained results, transport buffering, archive decoding, strings, or the rest of the runtime; use streaming APIs and limit concurrent workloads for larger inputs.
+`collectBytes(source, { maxBytes, maxMemoryBytes, signal })` snapshots streamed chunks into one growing buffer. `maxBytes` and `maxMemoryBytes` accept explicit `Infinity`. `maxMemoryBytes` limits owned capacity, the current input's full backing buffer, and overlapping allocations during growth; exhaustion throws `EFBIG`. Browser and Worker bundles additionally share a fixed 32 MiB budget across active collectors, even when byte limits are omitted. The returned view may retain geometric spare capacity. This budget covers collection, not caller-retained results, transport buffering, archive decoding, strings, or the rest of the runtime; use streaming APIs and limit concurrent workloads for larger inputs.
 
 <details>
 <summary>S3 filesystem and HTTP transport options</summary>
@@ -280,12 +280,12 @@ For in-memory S3 simulations, `new MockS3Client({ buckets, pageSize?, now?, auth
 | `maxReadBytes` | Unlimited unless configured |
 | `maxStreamBytes` | Unlimited unless configured |
 | `maxListEntries` | Unlimited unless configured |
-| `removalLimits.maxRequests` | 32 transport calls per `rm`, including lookup, listing, and deletes |
-| `removalLimits.maxListEntries` | 32 returned listing entries in aggregate per `rm`, including lookup |
-| `removalLimits.maxDeleteObjects` | 16 objects per `rm` |
+| `removalLimits.maxRequests` | Unlimited unless configured; transport calls per `rm`, including lookup, listing, and deletes |
+| `removalLimits.maxListEntries` | Unlimited unless configured; returned listing entries in aggregate per `rm`, including lookup |
+| `removalLimits.maxDeleteObjects` | Unlimited unless configured; objects per `rm` |
 | `compareEntry` | Optional trusted backing-identity callback |
 
-Removal limits apply even when shell filesystem-call limits admit a recursive `rm` as one operation. Each limit accepts a positive safe integer. Traversal stops at the listing/request cap and rejects with `EFBIG`; all delete requests must fit the remaining request budget before the first mutation. Nonrecursive removal checks for children using pages of at most two entries. Configure larger `removalLimits` only where the deployment can afford the corresponding work. These limits count adapter transport calls; retries inside a supplied transport need their own limit. Remote failures or concurrent writers can still cause partial deletion after preflight.
+Removal limits apply even when shell filesystem-call limits admit a recursive `rm` as one operation. Removal limits default to unlimited and accept a positive safe integer or `Infinity`. Read, stream, listing, and request limits also accept explicit `Infinity`. Traversal stops at the listing/request cap and rejects with `EFBIG`; all delete requests must fit the remaining request budget before the first mutation. Nonrecursive removal checks for children using pages of at most two entries. Configure finite `removalLimits` to bound the work in deployments that need them. These limits count adapter transport calls; retries inside a supplied transport need their own limit. Remote failures or concurrent writers can still cause partial deletion after preflight.
 
 For larger trees, a trusted integration can process one bounded batch per request/job using its explicitly supplied transport. This example uses at most 17 transport calls and retains at most 16 summaries; repeat in a later job until `done`. The prefix must come from trusted deployment configuration, include the filesystem's configured prefix, and end in `/`. This deliberately bypasses filesystem collision checks and deletes directory markers as well as files; serialize it with writers when complete removal is required.
 
@@ -346,7 +346,7 @@ Each batch lists from the beginning because previous keys have been deleted; it
 
 Known identity Content-Length responses use one result buffer; other responses grow storage up to the configured ceiling and return a view without a final copy. Growth can temporarily retain the old and new buffers (up to three times the response size), plus transport chunks. These per-response defaults leave headroom in Workers; hosts must still budget for metadata parsing, text decoding, transport buffers, and concurrent reads, especially when raising the limits. Use streaming reads for large files.
 
-WebDAV metadata parsing limits the document to 100,000 elements, 100,000 content nodes, 100,000 attributes, and 256 levels of nesting, independently of `maxEntries`. Text is bounded by `maxXmlBytes`. Exceeding a structural budget reports `EFBIG`; parsing yields cooperatively so caller cancellation and `timeoutMs` remain active.
+WebDAV `maxEntries`, `maxResponseBytes`, `maxXmlBytes`, and per-read `maxBytes` accept explicit `Infinity`. WebDAV metadata parsing limits the document to 100,000 elements, 100,000 content nodes, 100,000 attributes, and 256 levels of nesting, independently of `maxEntries`. Text is bounded by `maxXmlBytes`. Exceeding a structural budget reports `EFBIG`; parsing yields cooperatively so caller cancellation and `timeoutMs` remain active.
 
 See the [binding types](src/fs/webdav/webdav.ts) before implementing atomic directory removal. A recursive WebDAV DELETE does not satisfy that contract.
 

package/dist/safe-fs/contracts/io.js

@@ -15,6 +15,40 @@ const EMPTY_BYTE_ITERATOR = Object.freeze({
 const EMPTY_BYTE_SOURCE = Object.freeze({
     [Symbol.asyncIterator]() { return EMPTY_BYTE_ITERATOR; },
 });
+const noop = () => { };
+class SingleChunkByteIterator {
+    constructor(bytes) {
+        this.bytes = bytes;
+        this.consumed = false;
+    }
+    [Symbol.asyncIterator]() {
+        return this;
+    }
+    tryNextSync() {
+        if (this.consumed)
+            return DONE_RESULT;
+        this.consumed = true;
+        return { done: false, value: this.bytes };
+    }
+    next() {
+        if (this.consumed)
+            return RESOLVED_DONE;
+        this.consumed = true;
+        return Promise.resolve({ done: false, value: this.bytes });
+    }
+    return() {
+        this.consumed = true;
+        return RESOLVED_DONE;
+    }
+}
+class SingleChunkByteSource {
+    constructor(bytes) {
+        this.bytes = bytes;
+    }
+    [Symbol.asyncIterator]() {
+        return new SingleChunkByteIterator(this.bytes);
+    }
+}
 export function toByteSource(input) {
     if (typeof input !== "string" && !(input instanceof Uint8Array)) {
         throw new TypeError("Byte source input must be a string or Uint8Array");
@@ -27,38 +61,15 @@ export function toByteSource(input) {
         return EMPTY_BYTE_SOURCE;
     }
     const bytes = typeof input === "string" ? (sharedTextEncoder ??= new TextEncoder()).encode(input) : new Uint8Array(input);
-    return {
-        [Symbol.asyncIterator]() {
-            let consumed = false;
-            return {
-                [Symbol.asyncIterator]() { return this; },
-                tryNextSync() {
-                    if (consumed)
-                        return DONE_RESULT;
-                    consumed = true;
-                    return { done: false, value: bytes };
-                },
-                next() {
-                    if (consumed)
-                        return RESOLVED_DONE;
-                    consumed = true;
-                    return Promise.resolve({ done: false, value: bytes });
-                },
-                return() {
-                    consumed = true;
-                    return RESOLVED_DONE;
-                },
-            };
-        },
-    };
+    return new SingleChunkByteSource(bytes);
 }
 let activeCollectionBytes = 0;
 export async function collectBytes(source, options) {
     if (options.maxBytes !== undefined && options.maxBytes !== Infinity && (!Number.isSafeInteger(options.maxBytes) || options.maxBytes < 0)) {
         throw new RangeError("maxBytes must be a nonnegative safe integer");
     }
-    if (options.maxMemoryBytes !== undefined && (!Number.isSafeInteger(options.maxMemoryBytes) || options.maxMemoryBytes < 0)) {
-        throw new RangeError("maxMemoryBytes must be a nonnegative safe integer");
+    if (options.maxMemoryBytes !== undefined && options.maxMemoryBytes !== Infinity && (!Number.isSafeInteger(options.maxMemoryBytes) || options.maxMemoryBytes < 0)) {
+        throw new RangeError("maxMemoryBytes must be a nonnegative safe integer or Infinity");
     }
     const memoryLimit = options.maxMemoryBytes ?? Infinity;
     let reserved = 0;
@@ -132,76 +143,97 @@ async function abortable(operation, signal) {
         }
     });
 }
-export function readBytes(source, signal) {
-    if (signal !== undefined && source[readBytesSignal] === signal) {
-        return source;
+class ReadBytesGenerator {
+    constructor(source, signal) {
+        this.source = source;
+        this.abortSignal = signal;
+        this.iterator = undefined;
+        this.nativeAbort = false;
+        this.finished = false;
+        this.closing = undefined;
+        this.turn = undefined;
+        this.readingSync = false;
+        this.syncFailure = undefined;
+    }
+    get [readBytesSignal]() {
+        return this.abortSignal;
     }
-    let iterator;
-    let nativeAbort = false;
-    let finished = false;
-    let closing;
-    let turn;
-    let readingSync = false;
-    let syncFailure;
-    const schedule = (action) => {
-        const previous = turn;
+    [Symbol.asyncIterator]() {
+        return this;
+    }
+    _schedule(action) {
+        const previous = this.turn;
         let release;
         const reserved = new Promise(resolve => { release = resolve; });
-        turn = reserved;
-        const result = previous ? previous.then(action) : readingSync ? Promise.resolve().then(action) : action();
-        const finish = () => { if (turn === reserved)
-            turn = undefined; release(); };
+        // Reserve before calling a producer, which may synchronously reenter us.
+        this.turn = reserved;
+        const result = previous
+            ? previous.then(action)
+            : this.readingSync
+                ? Promise.resolve().then(action)
+                : action();
+        const finish = () => {
+            if (this.turn === reserved)
+                this.turn = undefined;
+            release();
+        };
         void result.then(finish, finish);
         return result;
-    };
-    const ensureIterator = () => {
-        if (!iterator) {
+    }
+    _ensureIterator() {
+        let it = this.iterator;
+        if (!it) {
+            const signal = this.abortSignal;
             signal?.throwIfAborted();
-            iterator = source[Symbol.asyncIterator]();
-            nativeAbort = signal !== undefined && iterator.abortSignal === signal;
+            it = this.source[Symbol.asyncIterator]();
+            this.iterator = it;
+            this.nativeAbort = signal !== undefined && it.abortSignal === signal;
         }
-        return iterator;
-    };
-    const cleanupIterator = async (failed) => {
-        if (!finished && iterator?.return) {
-            finished = true;
-            const cleanup = Promise.resolve().then(() => iterator.return());
+        return it;
+    }
+    async _cleanupIterator(failed) {
+        const signal = this.abortSignal;
+        if (!this.finished && this.iterator?.return) {
+            this.finished = true;
+            const it = this.iterator;
+            const cleanup = Promise.resolve().then(() => it.return());
             if (signal?.aborted)
-                void cleanup.catch(() => { });
+                void cleanup.catch(noop);
             else
-                closing = abortable(() => cleanup, signal).then(() => { });
+                this.closing = abortable(() => cleanup, signal).then(noop);
         }
         else {
-            finished = true;
+            this.finished = true;
         }
-        if (closing) {
-            const pending = closing;
+        if (this.closing) {
+            const pending = this.closing;
             try {
                 await finishCleanup(() => pending, failed);
             }
             finally {
-                if (closing === pending)
-                    closing = undefined;
+                if (this.closing === pending)
+                    this.closing = undefined;
             }
         }
-    };
-    const tryNextSync = () => {
-        if (turn || readingSync || syncFailure)
+    }
+    tryNextSync() {
+        if (this.turn || this.readingSync || this.syncFailure)
             return undefined;
-        if (finished)
+        if (this.finished)
             return DONE_RESULT;
-        readingSync = true;
+        this.readingSync = true;
         try {
-            const it = ensureIterator();
+            const it = this._ensureIterator();
             if (typeof it.tryNextSync !== "function")
                 return undefined;
+            const signal = this.abortSignal;
             signal?.throwIfAborted();
             const syncResult = it.tryNextSync();
             signal?.throwIfAborted();
             if (syncResult === undefined)
                 return undefined;
             if (syncResult.done) {
-                finished = true;
+                this.finished = true;
                 return DONE_RESULT;
             }
             if (!(syncResult.value instanceof Uint8Array))
@@ -209,29 +241,30 @@ export function readBytes(source, signal) {
             return syncResult;
         }
         catch (error) {
-            syncFailure = { reason: error };
+            this.syncFailure = { reason: error };
             return undefined;
         }
         finally {
-            readingSync = false;
+            this.readingSync = false;
         }
-    };
-    const runNext = async () => {
-        if (finished)
+    }
+    async _runNext() {
+        if (this.finished)
             return DONE_RESULT;
         try {
-            if (syncFailure) {
-                const { reason } = syncFailure;
-                syncFailure = undefined;
+            if (this.syncFailure) {
+                const { reason } = this.syncFailure;
+                this.syncFailure = undefined;
                 throw reason;
             }
-            const it = ensureIterator();
+            const it = this._ensureIterator();
+            const signal = this.abortSignal;
             signal?.throwIfAborted();
             const syncResult = typeof it.tryNextSync === "function" ? it.tryNextSync() : undefined;
-            const result = syncResult ?? (nativeAbort ? await it.next() : await abortable(() => it.next(), signal));
+            const result = syncResult ?? (this.nativeAbort ? await it.next() : await abortable(() => it.next(), signal));
             signal?.throwIfAborted();
             if (result.done) {
-                finished = true;
+                this.finished = true;
                 return DONE_RESULT;
             }
             if (!(result.value instanceof Uint8Array))
@@ -239,42 +272,35 @@ export function readBytes(source, signal) {
             return result;
         }
         catch (error) {
-            await cleanupIterator(true);
+            await this._cleanupIterator(true);
             throw error;
         }
-    };
-    const gen = {
-        [readBytesSignal]: signal,
-        abortSignal: signal,
-        [Symbol.asyncIterator]() { return this; },
-        tryNextSync,
-        next() {
-            if (!turn && !readingSync && !syncFailure) {
-                if (finished && !closing)
-                    return RESOLVED_DONE;
-                const syncResult = tryNextSync();
-                if (syncResult !== undefined) {
-                    return syncResult.done ? RESOLVED_DONE : Promise.resolve(syncResult);
-                }
-            }
-            return schedule(runNext);
-        },
-        return(value) {
-            if ((finished || !iterator?.return) && !turn && !readingSync && !closing && value === undefined) {
-                finished = true;
-                return RESOLVED_DONE;
-            }
-            return schedule(async () => {
-                await cleanupIterator(false);
-                return { done: true, value: await value };
-            });
-        },
-        throw(error) {
-            return schedule(async () => {
-                await cleanupIterator(true);
-                throw error;
-            });
-        },
-    };
-    return gen;
+    }
+    next() {
+        if (this.finished && !this.turn && !this.readingSync && !this.syncFailure && !this.closing)
+            return RESOLVED_DONE;
+        return this._schedule(() => this._runNext());
+    }
+    return(value) {
+        if ((this.finished || !this.iterator?.return) && !this.turn && !this.readingSync && !this.closing && value === undefined) {
+            this.finished = true;
+            return RESOLVED_DONE;
+        }
+        return this._schedule(async () => {
+            await this._cleanupIterator(false);
+            return { done: true, value: await value };
+        });
+    }
+    throw(error) {
+        return this._schedule(async () => {
+            await this._cleanupIterator(true);
+            throw error;
+        });
+    }
+}
+export function readBytes(source, signal) {
+    if (signal !== undefined && source[readBytesSignal] === signal) {
+        return source;
+    }
+    return new ReadBytesGenerator(source, signal);
 }

package/dist/safe-fs/fs/memory/index.d.ts

@@ -1,7 +1,26 @@
 import type { AppendFileOptions, CopyFileOptions, DirectoryEntry, EntryComparison, FileReadHandle, FileResizeHandle, FileStat, FileSystem, FileSystemCapabilities, FsOptions, ChmodOptions, RenameOptions, MkdirOptions, ReadDirectoryOptions, ReadFileOptions, ReadStreamOptions, RemoveOptions, FileDescriptor, OpenFileOptions, OpenReadFileOptions, OpenResizeFileOptions, WriteFileOptions, ConditionalWriteFileOptions, ConditionalRemoveFileOptions, ConditionalRemoveEntryOptions, ConditionalRemoveEntryReceiptOptions, CreateStagedFileOptions, FileStaging, FileStagingEntry, FileStagingResolution, PublishStagedFileOptions, PrepareDirectoryOptions, StagedFileContent } from "../../contracts/filesystem.js";
 import type { ByteSource } from "../../contracts/io.js";
+import { MemoryAllocation } from "./ledger.js";
 import { type MemoryFileSystemOptions } from "./limits.js";
 export { defaultMemoryFileSystemLimits, type MemoryFileSystemLimits, type MemoryFileSystemOptions } from "./limits.js";
+interface Metadata {
+    revision: number;
+    mode: number;
+    ino: number;
+    nlink: number;
+    atimeMs: number;
+    mtimeMs: number;
+    ctimeMs: number;
+    birthtimeMs: number;
+    references: number;
+}
+interface FileNode extends Metadata {
+    type: "file";
+    byteLength: number;
+    view: Uint8Array | undefined;
+    data: Uint8Array;
+    allocation: MemoryAllocation;
+}
 export interface ConditionalMutationBinding {
     readonly path: string;
     readonly parent: FileStat;
@@ -24,6 +43,7 @@ export declare class MemoryFileSystem implements FileSystem {
     confineExtraction(roots: readonly string[], options?: FsOptions): Promise<FileSystem>;
     private metadata;
     private directory;
+    private addDirectoryNode;
     private addNode;
     private releaseNode;
     private releaseReference;
@@ -50,6 +70,7 @@ export declare class MemoryFileSystem implements FileSystem {
     private writeData;
     private writeAt;
     writeMemoryFileFast(path: string, data: Uint8Array, append: boolean, mode: number): void;
+    writeMemoryFileInDirFast(dirPrefix: string, name: string, data: Uint8Array, append: boolean, mode: number): void;
     readFile(path: string, options?: ReadFileOptions): Promise<Uint8Array>;
     private expectEntry;
     private stagingLocations;
@@ -96,6 +117,18 @@ export declare class MemoryFileSystem implements FileSystem {
     writeStream(path: string, source: ByteSource, options?: WriteFileOptions): Promise<void>;
 }
 export declare function createMemoryFileSystem(options?: MemoryFileSystemOptions | Readonly<Record<string, unknown>>): MemoryFileSystem;
+export declare class MemoryRedirectHandle {
+    readonly fs: MemoryFileSystem;
+    readonly path: string;
+    readonly node: FileNode;
+    readonly append: boolean;
+    position: number;
+    closed: boolean;
+    constructor(fs: MemoryFileSystem, path: string, node: FileNode, append: boolean);
+    writeSync(chunk: Uint8Array, signal?: AbortSignal): void;
+    close(): void;
+}
+export declare function tryOpenMemoryRedirectHandleSync(filesystem: FileSystem, path: string, append: boolean, mode: number, signal?: AbortSignal): MemoryRedirectHandle | undefined;
 export declare function utf8ByteLength(value: string): number;
 export declare function isCleanAbsolutePath(path: string): boolean;
 export declare function tryResolveMemoryDevicePath(filesystem: FileSystem, path: string, resizeCreate?: boolean): string | undefined;
@@ -104,3 +137,6 @@ export declare function tryGetMemoryDirectoryEntryNamesSync(filesystem: FileSyst
     readonly type: "file" | "directory" | "symlink";
 }> | undefined;
 export declare function tryWriteMemoryFileSync(filesystem: FileSystem, path: string, data: Uint8Array, append: boolean, mode: number, signal?: AbortSignal): boolean;
+export declare function tryMkdirMemorySync(filesystem: FileSystem, path: string, recursive: boolean, mode: number, signal?: AbortSignal): boolean;
+export declare function tryRmRfMemorySync(filesystem: FileSystem, path: string, signal?: AbortSignal): boolean;
+export declare function tryWriteMemoryFileInDirSync(filesystem: FileSystem, dirPrefix: string, name: string, data: Uint8Array, append: boolean, mode: number, signal?: AbortSignal): boolean;