@poe-platform/safe-fs 0.1.802 → 0.1.804
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -4
- package/dist/safe-fs/bridge/filesystem.js +1 -1
- package/dist/safe-fs/contracts/errors.js +4 -3
- package/dist/safe-fs/contracts/filesystem.d.ts +3 -2
- package/dist/safe-fs/contracts/io.d.ts +2 -0
- package/dist/safe-fs/contracts/io.js +32 -7
- package/dist/safe-fs/contracts/portable-path.d.ts +8 -0
- package/dist/safe-fs/contracts/portable-path.js +22 -1
- package/dist/safe-fs/core.d.ts +1 -0
- package/dist/safe-fs/core.js +1 -0
- package/dist/safe-fs/fs/devices/index.js +2 -1
- package/dist/safe-fs/fs/directory-admission.js +2 -2
- package/dist/safe-fs/fs/memory/index.d.ts +1 -1
- package/dist/safe-fs/fs/memory/index.js +32 -53
- package/dist/safe-fs/fs/memory/ledger.js +3 -1
- package/dist/safe-fs/fs/object-publication/index.js +2 -1
- package/dist/safe-fs/fs/real/index.js +3 -2
- package/dist/safe-fs/fs/s3/http/request-fetch.d.ts +14 -3
- package/dist/safe-fs/fs/s3/http/request-node.d.ts +2 -2
- package/dist/safe-fs/fs/s3/http/types.d.ts +2 -2
- package/dist/safe-fs/fs/s3/manifest-admission.js +0 -4
- package/dist/safe-fs/fs/s3/namespace.js +2 -4
- package/dist/safe-fs/fs/webdav/webdav.js +4 -28
- package/dist/safe-fs/fs/webdav/xml.js +2 -2
- package/dist/safe-fs/python/emscripten.js +5 -2
- package/dist/safe-fs/python/filesystem.js +2 -2
- package/dist/safe-fs/python/native.js +1 -1
- package/dist/safe-fs/python/stat.js +5 -2
- package/package.json +12 -2
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# safe-fs
|
|
2
2
|
|
|
3
|
-
An asynchronous filesystem interface for memory, rooted host directories, S3, and WebDAV, with mounts, copy-on-write overlays, and Node-style bridges. Portable filesystem types and POSIX helpers compile without Node types.
|
|
3
|
+
An asynchronous filesystem interface for memory, rooted host directories, S3, and WebDAV, with mounts, copy-on-write overlays, and Node-style bridges. Portable filesystem types and POSIX helpers compile without Node types. The `posixPath` export from `@poe-code/safe-fs/core` provides `basename`, `dirname`, `extname`, `join`, `isAbsolute`, `normalize`, `resolve`, `relative`, `sep`, and `delimiter`; `resolve` and `relative` use `/` as their working directory in every runtime.
|
|
4
4
|
|
|
5
5
|
Under `workerd` and `browser` conditions, the root export includes portable filesystems, `S3FileSystem`, and `MockS3Client`. Supply an S3 transport to use object storage without Node globals.
|
|
6
6
|
|
|
@@ -86,14 +86,26 @@ cause. Preserve cancellation/control exceptions separately. Cross-family
|
|
|
86
86
|
|
|
87
87
|
All required methods must exist, but a backend may reject an operation with `FsError`, such as `EROFS` for a write or `ENOTSUP` for unsupported semantics. Check optional methods and `capabilities` rather than assuming every backend behaves like a local disk. Errors expose `code` and may include `syscall`, `path`, `dest`, and `cause`.
|
|
88
88
|
|
|
89
|
+
`readBytes` supports frozen host abort signals. The exported
|
|
90
|
+
`listenForAbort(signal, listener)` returns a cleanup function for custom
|
|
91
|
+
operations; if frozen host bookkeeping prevents registration, callers must
|
|
92
|
+
check cancellation at operation boundaries, as `readBytes` does.
|
|
93
|
+
|
|
89
94
|
`createDeviceFileSystem(fs)` adds portable `/dev/null` whole-file, stream, and descriptor I/O. Descriptor reads return EOF and writes discard bytes; stat remains a zero-size character device and descriptor position remains zero. Truncating opens are accepted, exclusive creation fails with `EEXIST`, and descriptor resizing and synchronization are unsupported. Access modes, cancellation, and closed handles use the normal descriptor checks. With an authoritative object store, use `createDeviceFileSystem(withObjectFileDescriptors(fs, store))` so null-device I/O never acquires or publishes an object version; ordinary files retain conditional publication. The device wrapper must be outermost for this composition.
|
|
90
95
|
|
|
96
|
+
`PythonStatTranslator` and `createPythonRuntimeStatMapper` allocate stable guest device
|
|
97
|
+
identities without a default device-count budget. Their optional `maxDevices` accepts
|
|
98
|
+
`Infinity` or a positive safe integer. Explicit budgets remain enforced; device numbers
|
|
99
|
+
must still fit the disjoint application/runtime ranges of the uint32 Python stat ABI.
|
|
100
|
+
|
|
91
101
|
`withObjectFileDescriptors(fs, store)` supports large shell and Python descriptor
|
|
92
102
|
writes when the host supplies `store.createStaging`: private externally backed
|
|
93
103
|
pages keep working memory bounded without publishing the growing file after
|
|
94
104
|
every write. Conditional publication still occurs at sync/close, and retained
|
|
95
105
|
readers keep their old versions. Without that optional backend primitive, the
|
|
96
|
-
optional dirty-page budget limits unflushed output only when configured.
|
|
106
|
+
optional dirty-page budget limits unflushed output only when configured. Descriptor
|
|
107
|
+
resource limits accept `Infinity` and are unlimited when omitted; `chunkBytes`
|
|
108
|
+
remains a positive finite integer. See the
|
|
97
109
|
[object descriptor and spill contract](src/contracts/object-publication.md) for
|
|
98
110
|
backend methods, failure semantics and qualification; no provider storage is
|
|
99
111
|
configured automatically.
|
|
@@ -270,7 +282,8 @@ Every raw filesystem operation accepts an optional `signal`. Additional fields a
|
|
|
270
282
|
|
|
271
283
|
| Operation | Options |
|
|
272
284
|
| --- | --- |
|
|
273
|
-
| `readFile` | `maxBytes`:
|
|
285
|
+
| `readFile` | `maxBytes`: nonnegative safe-integer byte cap; `Infinity` adds no per-read cap |
|
|
286
|
+
| `readdir` | `maxEntries`: nonnegative safe-integer entry cap; omission or `Infinity` preserves backend limits and ordering |
|
|
274
287
|
| `writeFile`, `writeStream` | `flag`: `w` (default, replace), `wx` (exclusive create), `a` (append), or `ax` (exclusive append/create); optional `mode` |
|
|
275
288
|
| `appendFile` | Optional `mode` |
|
|
276
289
|
| `mkdir` | `recursive` (default false); optional `mode` |
|
|
@@ -278,7 +291,7 @@ Every raw filesystem operation accepts an optional `signal`. Additional fields a
|
|
|
278
291
|
| `copyFile` | `exclusive` (default false) |
|
|
279
292
|
| `readStream` | `start` (default 0), `endExclusive` (default end of file), `chunkSize` (built-in default 64 KiB) |
|
|
280
293
|
|
|
281
|
-
`validatePath(path, { maxPathBytes, maxPathComponents })` accepts optional UTF-8 byte and component quotas; both default to `Infinity`. Memory accepts the same constructor options and counts cumulative symlink expansion. Device operations accept these quotas under `pathLimits`. WebDAV accepts `maxTimestampPropertyBytes` for timestamp metadata, also unlimited by default.
|
|
294
|
+
`validatePath(path, { maxPathBytes, maxPathComponents })` accepts optional UTF-8 byte and component quotas; both default to `Infinity`. Memory accepts the same constructor options and counts cumulative symlink expansion. Device operations and WebDAV stat, access, and writes accept these quotas under `pathLimits`. WebDAV accepts `maxTimestampPropertyBytes` for timestamp metadata, also unlimited by default.
|
|
282
295
|
|
|
283
296
|
`access` takes a separate mode bitmask from `ACCESS_MODES`. `chmod` takes a mode, `utimes` takes millisecond timestamps, and `truncate` takes a byte length (default 0). For conditional chmod, check `capabilitiesFor(path, { conditionalChmod: true })` (or `capabilities`) and require `conditionalChmod: true`; supply `parent`, `expected`, and complete root-to-parent `ancestors` together. An optional mutation-free `commitGuard` must return literal `true` synchronously. Memory validates at its metadata commit; mount and supported Memory overlays preserve wrapper ancestry. Real uses its existing externally isolated host-tree boundary and does not prevent races with other processes. Backend limits still apply. Node-shaped bridge methods translate their own options rather than accepting these raw option objects; see the [bridge signatures](src/bridge/filesystem.ts).
|
|
284
297
|
|
|
@@ -289,6 +302,11 @@ Every raw filesystem operation accepts an optional `signal`. Additional fields a
|
|
|
289
302
|
|
|
290
303
|
`S3FileSystem` requires `transport` and `bucket`. A transport supplies `headObject`, `getObject`, `putObject`, `deleteObject`, `copyObject`, and `listObjectsV2`; see the [transport interface](src/fs/s3/transport.ts).
|
|
291
304
|
|
|
305
|
+
`createS3NamespaceFileSystem` resource limits (`maxBytes`, `maxEntries`,
|
|
306
|
+
`maxManifestBytes`, `maxAttempts`, and inherited descriptor limits) are unlimited
|
|
307
|
+
when omitted or set to `Infinity`; positive safe integers enable finite limits.
|
|
308
|
+
Namespace paths have no implicit length cap; configured manifest budgets still apply.
|
|
309
|
+
|
|
292
310
|
`createS3Transport(client, capabilities?)` wraps a compatible client with explicit `streamingRead`, `streamingWrite`, `conditionalPut`, `conditionalCopy`, and `conditionalDelete` flags (all absent by default). Direct transport requests accept an optional `abortSignal`.
|
|
293
311
|
|
|
294
312
|
For in-memory S3 simulations, `new MockS3Client({ buckets, pageSize?, now?, authorize? })` requires bucket names. `pageSize` defaults to 1,000 (range 1–1,000), `now` defaults to the current date/time, and optional `authorize(request)` can reject requests. It does not verify a real provider.
|
|
@@ -70,7 +70,7 @@ export class FileSystemBridge {
|
|
|
70
70
|
assertBridgePath(this.#root, this.#cwd);
|
|
71
71
|
this.#signal = options.signal;
|
|
72
72
|
const maximum = options.readFileMaxBytes;
|
|
73
|
-
if (maximum !== undefined && (!Number.isSafeInteger(maximum) || maximum < 0))
|
|
73
|
+
if (maximum !== undefined && maximum !== Infinity && (!Number.isSafeInteger(maximum) || maximum < 0))
|
|
74
74
|
throw new TypeError("Invalid readFileMaxBytes");
|
|
75
75
|
this.#readFileMaxBytes = maximum;
|
|
76
76
|
if (options.reserveReadFile !== undefined && typeof options.reserveReadFile !== "function")
|
|
@@ -44,13 +44,14 @@ export class FsError extends Error {
|
|
|
44
44
|
const operation = options.syscall ? `, ${options.syscall}` : "";
|
|
45
45
|
const path = options.path === undefined ? "" : ` '${options.path}'`;
|
|
46
46
|
const destination = options.dest === undefined ? "" : ` -> '${options.dest}'`;
|
|
47
|
-
const
|
|
48
|
-
|
|
47
|
+
const errorConstructor = Error;
|
|
48
|
+
const prevStackLimit = errorConstructor.stackTraceLimit;
|
|
49
|
+
errorConstructor.stackTraceLimit = 0;
|
|
49
50
|
try {
|
|
50
51
|
super(`${code}: ${options.message ?? descriptions[code]}${operation}${path}${destination}`, options);
|
|
51
52
|
}
|
|
52
53
|
finally {
|
|
53
|
-
|
|
54
|
+
errorConstructor.stackTraceLimit = prevStackLimit;
|
|
54
55
|
}
|
|
55
56
|
this.name = "FsError";
|
|
56
57
|
this.code = code;
|
|
@@ -170,13 +170,14 @@ export interface RenameOptions extends FsOptions {
|
|
|
170
170
|
readonly noReplace?: boolean;
|
|
171
171
|
}
|
|
172
172
|
export interface ReadFileOptions extends FsOptions {
|
|
173
|
+
/** Nonnegative safe-integer byte cap; Infinity adds no per-read cap. */
|
|
173
174
|
readonly maxBytes?: number;
|
|
174
175
|
}
|
|
175
176
|
export interface ReadDirectoryOptions extends FsOptions {
|
|
176
177
|
/**
|
|
177
|
-
* Optional per-listing entry admission limit, a nonnegative safe integer.
|
|
178
|
+
* Optional per-listing entry admission limit, a nonnegative safe integer or Infinity.
|
|
178
179
|
* Zero permits an empty listing; overflow rejects with EFBIG, never truncates.
|
|
179
|
-
* Omission preserves the adapter's existing limits and ordering. Composed
|
|
180
|
+
* Omission or Infinity preserves the adapter's existing limits and ordering. Composed
|
|
180
181
|
* adapters may conservatively count distinct candidates before visibility
|
|
181
182
|
* filtering. This is not a global traversal, host-allocation or work quota.
|
|
182
183
|
*/
|
|
@@ -7,4 +7,6 @@ export interface CollectOptions {
|
|
|
7
7
|
}
|
|
8
8
|
export declare function toByteSource(input: string | Uint8Array): ByteSource;
|
|
9
9
|
export declare function collectBytes(source: ByteSource, options: CollectOptions): Promise<Uint8Array>;
|
|
10
|
+
/** Subscribe when supported; callers must still check cancellation at operation boundaries. */
|
|
11
|
+
export declare function listenForAbort(signal: AbortSignal, listener: () => void): () => void;
|
|
10
12
|
export declare function readBytes(source: ByteSource, signal?: AbortSignal): AsyncGenerator<Uint8Array>;
|
|
@@ -108,27 +108,52 @@ export async function collectBytes(source, options) {
|
|
|
108
108
|
options.signal?.throwIfAborted();
|
|
109
109
|
return buffer.subarray(0, size);
|
|
110
110
|
}
|
|
111
|
+
/** Subscribe when supported; callers must still check cancellation at operation boundaries. */
|
|
112
|
+
export function listenForAbort(signal, listener) {
|
|
113
|
+
let callback = listener;
|
|
114
|
+
const onAbort = () => callback?.();
|
|
115
|
+
const dispose = () => {
|
|
116
|
+
// Release operation state even when a frozen host refuses removal.
|
|
117
|
+
callback = undefined;
|
|
118
|
+
try {
|
|
119
|
+
signal.removeEventListener("abort", onAbort);
|
|
120
|
+
}
|
|
121
|
+
catch (error) {
|
|
122
|
+
if (!(error instanceof TypeError) || Object.isExtensible(signal))
|
|
123
|
+
throw error;
|
|
124
|
+
}
|
|
125
|
+
};
|
|
126
|
+
try {
|
|
127
|
+
signal.addEventListener("abort", onAbort, { once: true });
|
|
128
|
+
}
|
|
129
|
+
catch (error) {
|
|
130
|
+
// Frozen hosts may register before failing to update their bookkeeping.
|
|
131
|
+
dispose();
|
|
132
|
+
if (!(error instanceof TypeError) || Object.isExtensible(signal))
|
|
133
|
+
throw error;
|
|
134
|
+
}
|
|
135
|
+
return dispose;
|
|
136
|
+
}
|
|
111
137
|
async function abortable(operation, signal) {
|
|
112
138
|
signal?.throwIfAborted();
|
|
113
139
|
if (!signal)
|
|
114
140
|
return operation();
|
|
115
141
|
return new Promise((resolve, reject) => {
|
|
116
|
-
const
|
|
117
|
-
|
|
142
|
+
const dispose = listenForAbort(signal, () => {
|
|
143
|
+
dispose();
|
|
118
144
|
reject(signal.reason);
|
|
119
|
-
};
|
|
120
|
-
signal.addEventListener("abort", onAbort, { once: true });
|
|
145
|
+
});
|
|
121
146
|
try {
|
|
122
147
|
Promise.resolve(operation()).then((result) => {
|
|
123
|
-
|
|
148
|
+
dispose();
|
|
124
149
|
resolve(result);
|
|
125
150
|
}, (error) => {
|
|
126
|
-
|
|
151
|
+
dispose();
|
|
127
152
|
reject(error);
|
|
128
153
|
});
|
|
129
154
|
}
|
|
130
155
|
catch (error) {
|
|
131
|
-
|
|
156
|
+
dispose();
|
|
132
157
|
reject(error);
|
|
133
158
|
}
|
|
134
159
|
});
|
|
@@ -4,10 +4,18 @@ export declare const dirname: typeof virtualDirname;
|
|
|
4
4
|
export declare function extname(path: string): string;
|
|
5
5
|
export declare function isAbsolutePath(path: string): boolean;
|
|
6
6
|
export declare function joinPath(...paths: string[]): string;
|
|
7
|
+
declare function resolve(...paths: string[]): string;
|
|
8
|
+
declare function relative(from: string, to: string): string;
|
|
7
9
|
export declare const posixPath: Readonly<{
|
|
8
10
|
basename: typeof basename;
|
|
9
11
|
dirname: typeof virtualDirname;
|
|
10
12
|
extname: typeof extname;
|
|
11
13
|
join: typeof joinPath;
|
|
12
14
|
isAbsolute: typeof isAbsolutePath;
|
|
15
|
+
normalize: typeof joinPath;
|
|
16
|
+
resolve: typeof resolve;
|
|
17
|
+
relative: typeof relative;
|
|
18
|
+
sep: "/";
|
|
19
|
+
delimiter: ":";
|
|
13
20
|
}>;
|
|
21
|
+
export {};
|
|
@@ -54,4 +54,25 @@ export function joinPath(...paths) {
|
|
|
54
54
|
const result = `${absolute ? "/" : ""}${components.join("/")}` || ".";
|
|
55
55
|
return joined.endsWith("/") && result !== "/" ? `${result}/` : result;
|
|
56
56
|
}
|
|
57
|
-
|
|
57
|
+
// Portable paths have a stable root instead of a process working directory.
|
|
58
|
+
function resolve(...paths) {
|
|
59
|
+
let joined = "/";
|
|
60
|
+
for (const path of paths) {
|
|
61
|
+
assertString(path);
|
|
62
|
+
joined = path.startsWith("/") ? path : `${joined}/${path}`;
|
|
63
|
+
}
|
|
64
|
+
const normalized = joinPath(joined);
|
|
65
|
+
return normalized.length > 1 && normalized.endsWith("/") ? normalized.slice(0, -1) : normalized;
|
|
66
|
+
}
|
|
67
|
+
function relative(from, to) {
|
|
68
|
+
const source = resolve(from).split("/").filter(Boolean);
|
|
69
|
+
const target = resolve(to).split("/").filter(Boolean);
|
|
70
|
+
let common = 0;
|
|
71
|
+
while (common < source.length && common < target.length && source[common] === target[common])
|
|
72
|
+
common++;
|
|
73
|
+
return [...Array(source.length - common).fill(".."), ...target.slice(common)].join("/");
|
|
74
|
+
}
|
|
75
|
+
export const posixPath = Object.freeze({
|
|
76
|
+
basename, dirname, extname, join: joinPath, isAbsolute: isAbsolutePath,
|
|
77
|
+
normalize: joinPath, resolve, relative, sep: "/", delimiter: ":"
|
|
78
|
+
});
|
package/dist/safe-fs/core.d.ts
CHANGED
|
@@ -25,3 +25,4 @@ export * from "./contracts/object.js";
|
|
|
25
25
|
export { ObjectAuthority } from "./fs/object-authority.js";
|
|
26
26
|
export { bindConditionalMutation, type ConditionalMutationBinding } from "./fs/memory/index.js";
|
|
27
27
|
export * from "./fs/s3/index.js";
|
|
28
|
+
export * from "./fs/s3/http/index.js";
|
package/dist/safe-fs/core.js
CHANGED
|
@@ -309,7 +309,8 @@ export class DeviceFileSystem {
|
|
|
309
309
|
async #readFileSlow(path, options) {
|
|
310
310
|
const resolved = await this.#resolve(path, options);
|
|
311
311
|
if (resolved === nullPath) {
|
|
312
|
-
|
|
312
|
+
if (options.maxBytes !== Infinity)
|
|
313
|
+
nonnegative(options.maxBytes, path);
|
|
313
314
|
return new Uint8Array();
|
|
314
315
|
}
|
|
315
316
|
if (resolved === deviceDirectory)
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
import { FsError } from "../contracts/errors.js";
|
|
2
2
|
export function directoryEntryLimit(options, path) {
|
|
3
3
|
options.signal?.throwIfAborted();
|
|
4
|
-
const limit = options.maxEntries;
|
|
4
|
+
const limit = options.maxEntries === Infinity ? undefined : options.maxEntries;
|
|
5
5
|
if (limit !== undefined && (!Number.isSafeInteger(limit) || limit < 0)) {
|
|
6
|
-
throw new FsError("EINVAL", { syscall: "readdir", path, message: "directory entry limit must be a nonnegative safe integer" });
|
|
6
|
+
throw new FsError("EINVAL", { syscall: "readdir", path, message: "directory entry limit must be a nonnegative safe integer or Infinity" });
|
|
7
7
|
}
|
|
8
8
|
return limit;
|
|
9
9
|
}
|
|
@@ -173,7 +173,7 @@ export declare function utf8ByteLength(value: string): number;
|
|
|
173
173
|
export declare function isCleanAbsolutePath(path: string): boolean;
|
|
174
174
|
export declare function tryResolveMemoryDevicePath(filesystem: FileSystem, path: string, resizeCreate?: boolean): string | undefined;
|
|
175
175
|
export declare function getLastReadMemoryFileSourceRef(view: Uint8Array): Uint8Array | undefined;
|
|
176
|
-
export declare function tryReadMemoryFileViewSync(filesystem: FileSystem, path: string, maxBytes?: number, signal?: AbortSignal, captureSourceRef?: boolean): Uint8Array | undefined;
|
|
176
|
+
export declare function tryReadMemoryFileViewSync(filesystem: FileSystem, path: string, maxBytes?: number, signal?: AbortSignal, captureSourceRef?: boolean, beforeRead?: () => void): Uint8Array | undefined;
|
|
177
177
|
export declare function tryGetMemoryDirectoryEntryNamesSync(filesystem: FileSystem, path: string): ReadonlyMap<string, {
|
|
178
178
|
readonly type: "file" | "directory" | "symlink";
|
|
179
179
|
}> | undefined;
|
|
@@ -46,16 +46,10 @@ Object.assign(MemoryFileNode.prototype, {
|
|
|
46
46
|
sourceRef: undefined,
|
|
47
47
|
});
|
|
48
48
|
const EMPTY_ALLOC_BYTES = new Uint8Array(0);
|
|
49
|
-
const SMALL_ALLOC_SLAB_SIZE = 8192;
|
|
50
|
-
let smallAllocSlab = new Uint8Array(SMALL_ALLOC_SLAB_SIZE);
|
|
51
|
-
let smallAllocOffset = 0;
|
|
52
49
|
const DUMMY_POOL_LEDGER = new MemoryLedger(normalizeMemoryFileSystemLimits({}));
|
|
53
50
|
const DUMMY_POOL_ALLOCATION = new MemoryAllocation(EMPTY_ALLOC_BYTES, DUMMY_POOL_LEDGER);
|
|
54
51
|
const DUMMY_POOL_FILE_NODE = new MemoryFileNode(0, 0, null, 0, DUMMY_POOL_ALLOCATION, DUMMY_POOL_ALLOCATION.data);
|
|
55
52
|
DUMMY_POOL_FILE_NODE.atimeMs = DUMMY_POOL_FILE_NODE.mtimeMs = DUMMY_POOL_FILE_NODE.ctimeMs = DUMMY_POOL_FILE_NODE.birthtimeMs = 1700000000000;
|
|
56
|
-
const defaultDateNow = Date.now;
|
|
57
|
-
let fastWriteCachedNow = Date.now();
|
|
58
|
-
let fastWriteNowTick = 0;
|
|
59
53
|
class FastDirectoryEntriesMap {
|
|
60
54
|
// Keep speculative slots local so a failed write cannot pin a tenant globally.
|
|
61
55
|
_missKey = "";
|
|
@@ -159,6 +153,7 @@ class FastDirectoryEntriesMap {
|
|
|
159
153
|
}
|
|
160
154
|
_rebuild(newCap) {
|
|
161
155
|
this._missKey = "";
|
|
156
|
+
this._missSlot = -1;
|
|
162
157
|
const newTableLen = newCap * 2;
|
|
163
158
|
const newMask = newTableLen - 1;
|
|
164
159
|
const newTable = newCap <= 16384 ? new Int16Array(newTableLen).fill(-1) : new Int32Array(newTableLen).fill(-1);
|
|
@@ -198,16 +193,17 @@ class FastDirectoryEntriesMap {
|
|
|
198
193
|
this._next = writeIdx;
|
|
199
194
|
}
|
|
200
195
|
set(k, v) {
|
|
201
|
-
|
|
196
|
+
const missSlot = k === this._missKey ? this._missSlot : -1;
|
|
197
|
+
this._missKey = "";
|
|
198
|
+
this._missSlot = -1;
|
|
199
|
+
if (missSlot >= 0 && (this._table[missSlot] === -1 || this._table[missSlot] === -2) && this._next < this._keys.length) {
|
|
202
200
|
const entryIdx = this._next++;
|
|
203
|
-
this._table[
|
|
201
|
+
this._table[missSlot] = entryIdx;
|
|
204
202
|
this._keys[entryIdx] = k;
|
|
205
203
|
this._vals[entryIdx] = v;
|
|
206
204
|
this.size++;
|
|
207
|
-
this._missKey = "";
|
|
208
205
|
return this;
|
|
209
206
|
}
|
|
210
|
-
this._missKey = "";
|
|
211
207
|
if (this._next >= this._keys.length) {
|
|
212
208
|
this._rebuild(this.size * 2 <= this._keys.length ? this._keys.length : this._keys.length * 2);
|
|
213
209
|
}
|
|
@@ -240,6 +236,7 @@ class FastDirectoryEntriesMap {
|
|
|
240
236
|
}
|
|
241
237
|
delete(k) {
|
|
242
238
|
this._missKey = "";
|
|
239
|
+
this._missSlot = -1;
|
|
243
240
|
if (this.size === 0)
|
|
244
241
|
return false;
|
|
245
242
|
let slot = this._hash(k);
|
|
@@ -268,6 +265,7 @@ class FastDirectoryEntriesMap {
|
|
|
268
265
|
}
|
|
269
266
|
clear() {
|
|
270
267
|
this._missKey = "";
|
|
268
|
+
this._missSlot = -1;
|
|
271
269
|
if (this._next > 0) {
|
|
272
270
|
this._table.fill(-1);
|
|
273
271
|
for (let i = 0; i < this._next; i++) {
|
|
@@ -346,19 +344,13 @@ function collectRemovedChild(child, name) {
|
|
|
346
344
|
}
|
|
347
345
|
function replenishSharedMemoryPools() {
|
|
348
346
|
while (sharedAllocationPoolLen < 112) {
|
|
349
|
-
if (smallAllocOffset + 64 > SMALL_ALLOC_SLAB_SIZE) {
|
|
350
|
-
smallAllocSlab = new Uint8Array(SMALL_ALLOC_SLAB_SIZE);
|
|
351
|
-
smallAllocOffset = 0;
|
|
352
|
-
}
|
|
353
|
-
const slice = smallAllocSlab.subarray(smallAllocOffset, smallAllocOffset + 64);
|
|
354
|
-
smallAllocOffset += 64;
|
|
355
347
|
DUMMY_POOL_LEDGER.reserve(64, 0, "init", "/");
|
|
356
|
-
const alloc = new MemoryAllocation(
|
|
348
|
+
const alloc = new MemoryAllocation(new Uint8Array(64), DUMMY_POOL_LEDGER);
|
|
357
349
|
alloc.release();
|
|
358
350
|
sharedAllocationPool[sharedAllocationPoolLen++] = alloc;
|
|
359
351
|
}
|
|
360
352
|
while (sharedFileNodePoolLen < 112) {
|
|
361
|
-
sharedFileNodePool[sharedFileNodePoolLen++] = new MemoryFileNode(0, 0,
|
|
353
|
+
sharedFileNodePool[sharedFileNodePoolLen++] = new MemoryFileNode(0, 0, 0, 0, DUMMY_POOL_ALLOCATION, undefined);
|
|
362
354
|
}
|
|
363
355
|
while (sharedLargeAllocationPoolLen < 4) {
|
|
364
356
|
DUMMY_POOL_LEDGER.reserve(65536, 0, "init", "/");
|
|
@@ -373,7 +365,7 @@ function replenishSharedMemoryPools() {
|
|
|
373
365
|
sharedMediumAllocationPool[sharedMediumAllocationPoolLen++] = alloc;
|
|
374
366
|
}
|
|
375
367
|
while (sharedDirectoryNodePoolLen < 16) {
|
|
376
|
-
sharedDirectoryNodePool[sharedDirectoryNodePoolLen++] = new MemoryDirectoryNode(0, 0,
|
|
368
|
+
sharedDirectoryNodePool[sharedDirectoryNodePoolLen++] = new MemoryDirectoryNode(0, 0, 0);
|
|
377
369
|
}
|
|
378
370
|
}
|
|
379
371
|
class MemoryDirectoryNode {
|
|
@@ -921,6 +913,7 @@ export class MemoryFileSystem {
|
|
|
921
913
|
cache.lastFastDirNode = undefined;
|
|
922
914
|
}
|
|
923
915
|
if (node.entries.size === 0) {
|
|
916
|
+
node.entries.clear();
|
|
924
917
|
if (sharedDirectoryNodePoolLen < SHARED_DIR_POOL_CAPACITY) {
|
|
925
918
|
sharedDirectoryNodePool[sharedDirectoryNodePoolLen++] = node;
|
|
926
919
|
}
|
|
@@ -1363,17 +1356,6 @@ export class MemoryFileSystem {
|
|
|
1363
1356
|
}
|
|
1364
1357
|
}
|
|
1365
1358
|
try {
|
|
1366
|
-
// A shared slab's backing buffer exceeds the admitted slice capacity.
|
|
1367
|
-
// Finite file/storage ceilings require independently owned exact buffers.
|
|
1368
|
-
if (length <= 64 && this.ledger.hasInfiniteRetained && this.ledger.hasInfiniteFileBytes) {
|
|
1369
|
-
if (smallAllocOffset + length > SMALL_ALLOC_SLAB_SIZE) {
|
|
1370
|
-
smallAllocSlab = new Uint8Array(SMALL_ALLOC_SLAB_SIZE);
|
|
1371
|
-
smallAllocOffset = 0;
|
|
1372
|
-
}
|
|
1373
|
-
const slice = smallAllocSlab.subarray(smallAllocOffset, smallAllocOffset + length);
|
|
1374
|
-
smallAllocOffset += length;
|
|
1375
|
-
return new MemoryAllocation(slice, this.ledger);
|
|
1376
|
-
}
|
|
1377
1359
|
return new MemoryAllocation(new Uint8Array(length), this.ledger);
|
|
1378
1360
|
}
|
|
1379
1361
|
catch (cause) {
|
|
@@ -1559,11 +1541,12 @@ export class MemoryFileSystem {
|
|
|
1559
1541
|
}
|
|
1560
1542
|
if (position > curLen)
|
|
1561
1543
|
storage.fill(0, curLen, position);
|
|
1544
|
+
const source = data;
|
|
1562
1545
|
if (dataLen === data.byteLength) {
|
|
1563
1546
|
storage.set(data, position);
|
|
1564
1547
|
}
|
|
1565
|
-
else if (typeof
|
|
1566
|
-
|
|
1548
|
+
else if (typeof source.copy === "function") {
|
|
1549
|
+
source.copy(storage, position, 0, dataLen);
|
|
1567
1550
|
}
|
|
1568
1551
|
else {
|
|
1569
1552
|
for (let i = 0; i < dataLen; i++)
|
|
@@ -1762,7 +1745,7 @@ export class MemoryFileSystem {
|
|
|
1762
1745
|
1 + batchCount <= entries._keys.length) {
|
|
1763
1746
|
this.ledger.reserve(totalNameBytes, 2 * batchCount, "writeFile", fileNames[batchCount - 1]);
|
|
1764
1747
|
this.totalBytes += payloadBytesSum;
|
|
1765
|
-
const now =
|
|
1748
|
+
const now = Date.now();
|
|
1766
1749
|
const fileMode = typeModes.file | mode;
|
|
1767
1750
|
let ino = this.nextInode;
|
|
1768
1751
|
this.nextInode = ino + batchCount;
|
|
@@ -1835,7 +1818,7 @@ export class MemoryFileSystem {
|
|
|
1835
1818
|
const existing = nameHash !== undefined ? entries.getForWriteWithHash(name, nameHash) : entries.getForWrite(name);
|
|
1836
1819
|
if (existing === undefined && entries._next < entries._keys.length) {
|
|
1837
1820
|
this.ledger.reserve(name.length * 2, 2, "writeFile", name);
|
|
1838
|
-
const now =
|
|
1821
|
+
const now = Date.now();
|
|
1839
1822
|
const length = data.byteLength;
|
|
1840
1823
|
const node = sharedFileNodePool[--sharedFileNodePoolLen];
|
|
1841
1824
|
sharedFileNodePool[sharedFileNodePoolLen] = DUMMY_POOL_FILE_NODE;
|
|
@@ -1957,7 +1940,7 @@ export class MemoryFileSystem {
|
|
|
1957
1940
|
allocation.data.set(data);
|
|
1958
1941
|
this.ledger.reserve(nameBytes, 2, syscall, name);
|
|
1959
1942
|
try {
|
|
1960
|
-
const now =
|
|
1943
|
+
const now = Date.now();
|
|
1961
1944
|
const fileMode = typeModes.file | mode;
|
|
1962
1945
|
const view = zeroCopyConst ? new Uint8Array(data) : (capacity === length ? allocation.data : undefined);
|
|
1963
1946
|
let node;
|
|
@@ -2038,7 +2021,7 @@ export class MemoryFileSystem {
|
|
|
2038
2021
|
}
|
|
2039
2022
|
async readFile(path, options = {}) {
|
|
2040
2023
|
options.signal?.throwIfAborted();
|
|
2041
|
-
if (options.maxBytes !== undefined)
|
|
2024
|
+
if (options.maxBytes !== undefined && options.maxBytes !== Infinity)
|
|
2042
2025
|
this.integer(options.maxBytes, "readFile", path);
|
|
2043
2026
|
const node = this.file(path, "readFile");
|
|
2044
2027
|
this.permission(node, 4, "readFile", path);
|
|
@@ -3111,7 +3094,7 @@ export class MemoryRedirectHandle {
|
|
|
3111
3094
|
if (len === 0)
|
|
3112
3095
|
return;
|
|
3113
3096
|
const pos = this.append ? this.node.byteLength : this.position;
|
|
3114
|
-
const now = Date.now
|
|
3097
|
+
const now = Date.now();
|
|
3115
3098
|
this.fs.writeAtRange(this.node, src, len, pos, "write", this.path, now);
|
|
3116
3099
|
if (!this.append)
|
|
3117
3100
|
this.position = pos + len;
|
|
@@ -3131,7 +3114,7 @@ export class MemoryRedirectHandle {
|
|
|
3131
3114
|
mem.ledger.hasInfiniteRetained &&
|
|
3132
3115
|
mem.ledger.hasInfiniteFileBytes &&
|
|
3133
3116
|
mem.ledger.limits.maxBytes === undefined) {
|
|
3134
|
-
const now = Date.now
|
|
3117
|
+
const now = Date.now();
|
|
3135
3118
|
this.node.byteLength = len;
|
|
3136
3119
|
this.node.view = immutableData;
|
|
3137
3120
|
this.node.sourceRef = immutableData;
|
|
@@ -3257,7 +3240,7 @@ export function tryOpenMemoryRedirectHandleSync(filesystem, path, append, mode,
|
|
|
3257
3240
|
}
|
|
3258
3241
|
const cache = mem._cache;
|
|
3259
3242
|
cache.clearWrites();
|
|
3260
|
-
const now = Date.now
|
|
3243
|
+
const now = Date.now();
|
|
3261
3244
|
const fileMode = typeModes.file | validMode;
|
|
3262
3245
|
let newNode;
|
|
3263
3246
|
if (sharedFileNodePoolLen > 0) {
|
|
@@ -3458,7 +3441,7 @@ export function getLastReadMemoryFileSourceRef(view) {
|
|
|
3458
3441
|
lastReadViewSourceRef = undefined;
|
|
3459
3442
|
return ref;
|
|
3460
3443
|
}
|
|
3461
|
-
export function tryReadMemoryFileViewSync(filesystem, path, maxBytes, signal, captureSourceRef = false) {
|
|
3444
|
+
export function tryReadMemoryFileViewSync(filesystem, path, maxBytes, signal, captureSourceRef = false, beforeRead) {
|
|
3462
3445
|
// A capture belongs only to the most recent read, including failed/fallback reads.
|
|
3463
3446
|
lastReadViewData = undefined;
|
|
3464
3447
|
lastReadViewSourceRef = undefined;
|
|
@@ -3466,19 +3449,15 @@ export function tryReadMemoryFileViewSync(filesystem, path, maxBytes, signal, ca
|
|
|
3466
3449
|
if (mem._owner === undefined || !isStockMemoryMethods(mem, readFileFastMethodNames, false))
|
|
3467
3450
|
return undefined;
|
|
3468
3451
|
signal?.throwIfAborted();
|
|
3469
|
-
|
|
3452
|
+
beforeRead?.();
|
|
3453
|
+
if (maxBytes !== undefined && maxBytes !== Infinity)
|
|
3470
3454
|
mem.integer(maxBytes, "readFile", path);
|
|
3471
3455
|
const node = mem.file(path, "readFile");
|
|
3472
3456
|
mem.permission(node, 4, "readFile", path);
|
|
3473
3457
|
const data = node.data;
|
|
3474
3458
|
if (maxBytes !== undefined && data.byteLength > maxBytes)
|
|
3475
3459
|
mem.fail("EFBIG", "readFile", path);
|
|
3476
|
-
|
|
3477
|
-
node.atimeMs = Date.now();
|
|
3478
|
-
else if ((++fastWriteNowTick & 63) === 0)
|
|
3479
|
-
node.atimeMs = fastWriteCachedNow = Date.now();
|
|
3480
|
-
else
|
|
3481
|
-
node.atimeMs = fastWriteCachedNow;
|
|
3460
|
+
node.atimeMs = Date.now();
|
|
3482
3461
|
if (captureSourceRef && node.revision === 0) {
|
|
3483
3462
|
lastReadViewData = new WeakRef(data);
|
|
3484
3463
|
lastReadViewSourceRef = node.sourceRef === undefined ? undefined : new WeakRef(node.sourceRef);
|
|
@@ -3561,7 +3540,7 @@ export function tryGetMemoryDirectoryEntryNamesSync(filesystem, path) {
|
|
|
3561
3540
|
validatePath(path, owner.ledger.limits);
|
|
3562
3541
|
const root = mem.root;
|
|
3563
3542
|
if (path === "/") {
|
|
3564
|
-
if (((root.mode >> 6) &
|
|
3543
|
+
if (((root.mode >> 6) & 5) !== 5)
|
|
3565
3544
|
return undefined;
|
|
3566
3545
|
return root.entries;
|
|
3567
3546
|
}
|
|
@@ -3572,7 +3551,7 @@ export function tryGetMemoryDirectoryEntryNamesSync(filesystem, path) {
|
|
|
3572
3551
|
const fastPrefix = cache?.lastFastDirPrefix;
|
|
3573
3552
|
if (fastDir !== undefined &&
|
|
3574
3553
|
fastDir.nlink !== 0 &&
|
|
3575
|
-
((fastDir.mode >> 6) &
|
|
3554
|
+
((fastDir.mode >> 6) & 5) === 5 &&
|
|
3576
3555
|
fastPrefix !== undefined &&
|
|
3577
3556
|
(fastPrefix === path || (fastPrefix.length === path.length + 1 && fastPrefix.charCodeAt(path.length) === 47 && fastPrefix.startsWith(path)))) {
|
|
3578
3557
|
return fastDir.entries;
|
|
@@ -3585,7 +3564,7 @@ export function tryGetMemoryDirectoryEntryNamesSync(filesystem, path) {
|
|
|
3585
3564
|
const slash = path.indexOf("/", start);
|
|
3586
3565
|
if (slash === -1) {
|
|
3587
3566
|
const next = current.entries.getSubstr(path, start, path.length);
|
|
3588
|
-
if (!next || next.type !== "directory" || ((next.mode >> 6) &
|
|
3567
|
+
if (!next || next.type !== "directory" || ((next.mode >> 6) & 5) !== 5)
|
|
3589
3568
|
return undefined;
|
|
3590
3569
|
return next.entries;
|
|
3591
3570
|
}
|
|
@@ -3639,7 +3618,7 @@ export function tryMkdirMemorySync(filesystem, path, recursive, mode, signal, di
|
|
|
3639
3618
|
let next = current.entries.get(seg);
|
|
3640
3619
|
if (!next) {
|
|
3641
3620
|
mem.permission(current, 3, "mkdir", path);
|
|
3642
|
-
next = mem.addDirectoryNode(current, seg, validMode, "mkdir", path,
|
|
3621
|
+
next = mem.addDirectoryNode(current, seg, validMode, "mkdir", path, Date.now());
|
|
3643
3622
|
}
|
|
3644
3623
|
else if (next.type !== "directory") {
|
|
3645
3624
|
mem.fail(slash === -1 ? "EEXIST" : "ENOTDIR", "mkdir", path);
|
|
@@ -3660,7 +3639,7 @@ export function tryMkdirMemorySync(filesystem, path, recursive, mode, signal, di
|
|
|
3660
3639
|
if (location.node)
|
|
3661
3640
|
mem.fail("EEXIST", "mkdir", path);
|
|
3662
3641
|
mem.permission(location.parent, 3, "mkdir", path);
|
|
3663
|
-
mem.addDirectoryNode(location.parent, location.name, validMode, "mkdir", path,
|
|
3642
|
+
mem.addDirectoryNode(location.parent, location.name, validMode, "mkdir", path, Date.now());
|
|
3664
3643
|
return true;
|
|
3665
3644
|
}
|
|
3666
3645
|
export function tryRmRfMemorySync(filesystem, path, signal) {
|
|
@@ -3710,7 +3689,7 @@ export function tryRmRfMemorySync(filesystem, path, signal) {
|
|
|
3710
3689
|
sharedFastLocation.name = targetName;
|
|
3711
3690
|
sharedFastLocation.path = path;
|
|
3712
3691
|
try {
|
|
3713
|
-
mem.removeLocation(sharedFastLocation, path, "rm", true,
|
|
3692
|
+
mem.removeLocation(sharedFastLocation, path, "rm", true, Date.now());
|
|
3714
3693
|
}
|
|
3715
3694
|
finally {
|
|
3716
3695
|
sharedFastLocation.node = undefined;
|
|
@@ -84,8 +84,10 @@ export class MemoryAllocation {
|
|
|
84
84
|
return;
|
|
85
85
|
if (this.references <= 0)
|
|
86
86
|
throw new Error("Memory allocation already released");
|
|
87
|
-
if (--this.references === 0)
|
|
87
|
+
if (--this.references === 0) {
|
|
88
|
+
this.data.fill(0);
|
|
88
89
|
this.ledger.release(this.data.byteLength, 0);
|
|
90
|
+
}
|
|
89
91
|
}
|
|
90
92
|
isReleased64() {
|
|
91
93
|
return this.references === 0 && this.data.byteLength === 64;
|
|
@@ -9,7 +9,8 @@ export function withObjectFileDescriptors(filesystem, store, options = {}) {
|
|
|
9
9
|
const maxStagedPages = options.maxStagedPages ?? Infinity;
|
|
10
10
|
const maxFileBytes = options.maxFileBytes ?? Infinity;
|
|
11
11
|
const maxOpenFiles = options.maxOpenFiles ?? Infinity;
|
|
12
|
-
if (!
|
|
12
|
+
if (!Number.isSafeInteger(chunkBytes) || chunkBytes < 1
|
|
13
|
+
|| ![maxStagedBytes, maxStagedPages, maxFileBytes, maxOpenFiles].every(value => value === Infinity || Number.isSafeInteger(value) && value > 0)
|
|
13
14
|
|| typeof store.acquire !== "function"
|
|
14
15
|
|| store.publish !== undefined && typeof store.publish !== "function"
|
|
15
16
|
|| store.createStaging !== undefined && typeof store.createStaging !== "function")
|
|
@@ -608,8 +608,9 @@ export class RealFileSystem {
|
|
|
608
608
|
}
|
|
609
609
|
async readFile(path, options = {}) {
|
|
610
610
|
return this.operation("readFile", path, options, async () => {
|
|
611
|
-
const maxBytes = options.maxBytes ??
|
|
612
|
-
|
|
611
|
+
const maxBytes = options.maxBytes ?? Infinity;
|
|
612
|
+
if (maxBytes !== Infinity)
|
|
613
|
+
integer(maxBytes);
|
|
613
614
|
return collectBytes(this.readStream(path, options), { maxBytes, ...options });
|
|
614
615
|
});
|
|
615
616
|
}
|
|
@@ -1,4 +1,15 @@
|
|
|
1
|
-
import type { RequestOptions } from "node:http";
|
|
2
1
|
import type { RequestScope, WireResponse } from "./request.js";
|
|
3
|
-
|
|
4
|
-
|
|
2
|
+
export type S3HttpRequestFactory = never;
|
|
3
|
+
interface FetchRequestOptions {
|
|
4
|
+
readonly protocol?: string | undefined;
|
|
5
|
+
readonly hostname?: string | undefined;
|
|
6
|
+
readonly port?: string | number | undefined;
|
|
7
|
+
readonly path?: string | undefined;
|
|
8
|
+
readonly method?: string | undefined;
|
|
9
|
+
readonly headers?: Record<string, string> | undefined;
|
|
10
|
+
readonly agent?: false;
|
|
11
|
+
readonly maxHeaderSize?: number;
|
|
12
|
+
readonly rejectUnauthorized?: true;
|
|
13
|
+
}
|
|
14
|
+
export declare function sendRequest(options: FetchRequestOptions, body: Uint8Array, scope: RequestScope, factory?: S3HttpRequestFactory): Promise<WireResponse>;
|
|
15
|
+
export {};
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import type { RequestOptions } from "node:http";
|
|
2
|
-
import type { S3HttpRequestFactory } from "./types.js";
|
|
1
|
+
import type { ClientRequest, IncomingMessage, RequestOptions } from "node:http";
|
|
3
2
|
import type { RequestScope, WireResponse } from "./request.js";
|
|
3
|
+
export type S3HttpRequestFactory = (options: RequestOptions, onResponse: (response: IncomingMessage) => void) => ClientRequest;
|
|
4
4
|
export declare function sendRequest(options: RequestOptions, body: Uint8Array, scope: RequestScope, factory?: S3HttpRequestFactory): Promise<WireResponse>;
|
|
@@ -1,4 +1,5 @@
|
|
|
1
|
-
import type {
|
|
1
|
+
import type { S3HttpRequestFactory } from "#safe-fs-s3-request";
|
|
2
|
+
export type { S3HttpRequestFactory } from "#safe-fs-s3-request";
|
|
2
3
|
export interface S3HttpCredentials {
|
|
3
4
|
readonly accessKeyId: string;
|
|
4
5
|
readonly secretAccessKey: string;
|
|
@@ -7,7 +8,6 @@ export interface S3HttpCredentials {
|
|
|
7
8
|
export type S3HttpCredentialProvider = (options: {
|
|
8
9
|
readonly signal: AbortSignal;
|
|
9
10
|
}) => Promise<S3HttpCredentials>;
|
|
10
|
-
export type S3HttpRequestFactory = (options: RequestOptions, onResponse: (response: IncomingMessage) => void) => ClientRequest;
|
|
11
11
|
export interface S3HttpTransportOptions {
|
|
12
12
|
readonly endpoint: string;
|
|
13
13
|
readonly region: string;
|
|
@@ -41,8 +41,6 @@ export async function* admitManifest(source, options) {
|
|
|
41
41
|
invalid();
|
|
42
42
|
const key = value;
|
|
43
43
|
if (frame.shape === 'nodes') {
|
|
44
|
-
if (key.length > 4096)
|
|
45
|
-
invalid();
|
|
46
44
|
if (++entries > options.maxEntries)
|
|
47
45
|
throw new FsError('ENOSPC');
|
|
48
46
|
}
|
|
@@ -83,8 +81,6 @@ export async function* admitManifest(source, options) {
|
|
|
83
81
|
if (token) {
|
|
84
82
|
if (quoted) {
|
|
85
83
|
token += character;
|
|
86
|
-
if (token.length > 4096 * 6 + 2)
|
|
87
|
-
invalid();
|
|
88
84
|
if (escaped)
|
|
89
85
|
escaped = false;
|
|
90
86
|
else if (character === '\\')
|
|
@@ -4,7 +4,7 @@ import { dirname, normalizePath, validatePath } from '../../contracts/virtual-pa
|
|
|
4
4
|
import { withObjectFileDescriptors } from '../object-publication/index.js';
|
|
5
5
|
import { admitManifest } from './manifest-admission.js';
|
|
6
6
|
function limit(value) {
|
|
7
|
-
if (value === undefined)
|
|
7
|
+
if (value === undefined || value === Infinity)
|
|
8
8
|
return Infinity;
|
|
9
9
|
if (!Number.isSafeInteger(value) || value < 1)
|
|
10
10
|
throw new RangeError('Invalid S3 namespace limit');
|
|
@@ -18,7 +18,7 @@ export async function createS3NamespaceFileSystem(options) {
|
|
|
18
18
|
throw new TypeError('An explicit S3 bucket and manifest key are required');
|
|
19
19
|
const maxBytes = limit(options.maxBytes);
|
|
20
20
|
const maxEntries = limit(options.maxEntries);
|
|
21
|
-
const maxManifestBytes =
|
|
21
|
+
const maxManifestBytes = limit(options.maxManifestBytes);
|
|
22
22
|
const maxAttempts = limit(options.maxAttempts);
|
|
23
23
|
const descriptorOptions = options;
|
|
24
24
|
for (const key of ['chunkBytes', 'maxStagedBytes', 'maxStagedPages', 'maxFileBytes', 'maxOpenFiles']) {
|
|
@@ -32,8 +32,6 @@ export async function createS3NamespaceFileSystem(options) {
|
|
|
32
32
|
validatePath(value);
|
|
33
33
|
if (!value)
|
|
34
34
|
throw new FsError('ENOENT');
|
|
35
|
-
if (value.length > 4096)
|
|
36
|
-
throw new FsError('ENAMETOOLONG');
|
|
37
35
|
return normalizePath(value);
|
|
38
36
|
};
|
|
39
37
|
const entryPath = (value) => {
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { FsError, isErrnoCode, isFsError } from "../../contracts/errors.js";
|
|
2
2
|
import { composeAbortSignals } from "../../contracts/abort.js";
|
|
3
3
|
import { readBytes } from "../../contracts/io.js";
|
|
4
|
+
import { validatePath } from "../../contracts/virtual-path.js";
|
|
4
5
|
import { davChild, davChildren, parseXmlSteps, scalar, XmlResponseLimitError, XmlResourceLimitError } from "./xml.js";
|
|
5
6
|
import { admitDirectoryEntries, directoryEntryLimit } from "../directory-admission.js";
|
|
6
7
|
import { assertCallbackAuthorityAllowed, compareEntries, registerEntryAuthority } from "../mount/comparison.js";
|
|
@@ -159,28 +160,6 @@ function normalize(path) {
|
|
|
159
160
|
}
|
|
160
161
|
return `/${segments.join("/")}`;
|
|
161
162
|
}
|
|
162
|
-
function validateDirectoryWalkPath(path, syscall = "access") {
|
|
163
|
-
if (typeof path !== "string")
|
|
164
|
-
fail("EINVAL", "resolve", String(path), "invalid WebDAV path");
|
|
165
|
-
let bytes = 0;
|
|
166
|
-
let components = 0;
|
|
167
|
-
let inComponent = false;
|
|
168
|
-
for (let offset = 0; offset < path.length; offset++) {
|
|
169
|
-
const point = path.codePointAt(offset);
|
|
170
|
-
if (point === 47)
|
|
171
|
-
inComponent = false;
|
|
172
|
-
else if (!inComponent) {
|
|
173
|
-
components++;
|
|
174
|
-
inComponent = true;
|
|
175
|
-
}
|
|
176
|
-
bytes += point <= 0x7f ? 1 : point <= 0x7ff ? 2 : point <= 0xffff ? 3 : 4;
|
|
177
|
-
if (point > 0xffff)
|
|
178
|
-
offset++;
|
|
179
|
-
if (bytes > 65_536 || components > 256) {
|
|
180
|
-
fail("ENAMETOOLONG", syscall, path, "directory access exceeds the 64KiB path or 256 component limit");
|
|
181
|
-
}
|
|
182
|
-
}
|
|
183
|
-
}
|
|
184
163
|
function requiresCollection(path) {
|
|
185
164
|
const last = path.split("/").at(-1);
|
|
186
165
|
return last === "" || last === "." || last === "..";
|
|
@@ -826,7 +805,7 @@ export class WebDavFileSystem {
|
|
|
826
805
|
}
|
|
827
806
|
}
|
|
828
807
|
async stat(path, options = {}) {
|
|
829
|
-
|
|
808
|
+
validatePath(path, options.pathLimits);
|
|
830
809
|
return this.statWithAncestors(path, options);
|
|
831
810
|
}
|
|
832
811
|
async statWithAncestors(path, options) {
|
|
@@ -909,7 +888,7 @@ export class WebDavFileSystem {
|
|
|
909
888
|
}.bind(this));
|
|
910
889
|
}
|
|
911
890
|
async prepareWrite(path, options) {
|
|
912
|
-
|
|
891
|
+
validatePath(path, options.pathLimits);
|
|
913
892
|
const normalized = normalize(path);
|
|
914
893
|
if (options.mode !== undefined)
|
|
915
894
|
this.unsupported("writeFile mode", path);
|
|
@@ -1329,10 +1308,7 @@ export class WebDavFileSystem {
|
|
|
1329
1308
|
fail("ECANCELED", "access", path);
|
|
1330
1309
|
if (mode & 2)
|
|
1331
1310
|
this.unsupported("access write/execute permission checks", path);
|
|
1332
|
-
|
|
1333
|
-
validateDirectoryWalkPath(path);
|
|
1334
|
-
// Non-execute probes do not inherit stat's raw-path limits.
|
|
1335
|
-
const stat = await (mode & 1 ? this.stat(path, options) : this.statWithAncestors(path, options));
|
|
1311
|
+
const stat = await this.stat(path, options);
|
|
1336
1312
|
if (options.signal?.aborted)
|
|
1337
1313
|
fail("ECANCELED", "access", path);
|
|
1338
1314
|
if ((mode & 1) && stat.type !== "directory")
|
|
@@ -20,8 +20,8 @@ export function parseXml(input, limits = {}) {
|
|
|
20
20
|
}
|
|
21
21
|
export function* parseXmlSteps(input, limits = {}) {
|
|
22
22
|
const { maxResponses, ...documentLimits } = limits;
|
|
23
|
-
if (maxResponses !== undefined && (!Number.isSafeInteger(maxResponses) || maxResponses < 1)) {
|
|
24
|
-
throw new RangeError("XML limits must be positive integers");
|
|
23
|
+
if (maxResponses !== undefined && maxResponses !== Infinity && (!Number.isSafeInteger(maxResponses) || maxResponses < 1)) {
|
|
24
|
+
throw new RangeError("XML limits must be positive integers or Infinity");
|
|
25
25
|
}
|
|
26
26
|
let responses = 0;
|
|
27
27
|
try {
|
|
@@ -332,8 +332,8 @@ export function writePythonStat(heap, pointer, stat) {
|
|
|
332
332
|
return 0;
|
|
333
333
|
}
|
|
334
334
|
/** Runtime storage and application storage occupy disjoint guest device ranges. */
|
|
335
|
-
export function createPythonRuntimeStatMapper(maxDevices =
|
|
336
|
-
if (!Number.isSafeInteger(maxDevices) || maxDevices < 1
|
|
335
|
+
export function createPythonRuntimeStatMapper(maxDevices = Infinity) {
|
|
336
|
+
if (maxDevices !== Infinity && (!Number.isSafeInteger(maxDevices) || maxDevices < 1))
|
|
337
337
|
throw new RangeError('Invalid runtime device limit');
|
|
338
338
|
const mapped = Symbol('mapped Python runtime device');
|
|
339
339
|
const devices = new Map();
|
|
@@ -346,6 +346,9 @@ export function createPythonRuntimeStatMapper(maxDevices = 1024) {
|
|
|
346
346
|
if (device === undefined) {
|
|
347
347
|
if (devices.size >= maxDevices)
|
|
348
348
|
throw new Error('EOVERFLOW');
|
|
349
|
+
// Preserve the uint32 guest ABI even when the resource budget is unlimited.
|
|
350
|
+
if (devices.size >= 0x80000000)
|
|
351
|
+
throw new Error('EOVERFLOW');
|
|
349
352
|
device = 0x80000000 + devices.size;
|
|
350
353
|
devices.set(stat.dev, device);
|
|
351
354
|
}
|
|
@@ -23,11 +23,11 @@ export class PythonFileSystem {
|
|
|
23
23
|
if (!options.cwd.startsWith("/"))
|
|
24
24
|
throw new FsError("EINVAL", { syscall: "python filesystem", path: options.cwd, message: "cwd must be absolute" });
|
|
25
25
|
this.#cwd = options.cwd;
|
|
26
|
-
this.#transfer = options.maxTransferBytes ??
|
|
26
|
+
this.#transfer = options.maxTransferBytes ?? Infinity;
|
|
27
27
|
this.#limit = options.maxOpenFiles ?? Infinity;
|
|
28
28
|
this.#directoryLimit = options.maxDirectoryEntries ?? Infinity;
|
|
29
29
|
for (const limit of [this.#transfer, options.maxOpenFiles, options.maxDirectoryEntries])
|
|
30
|
-
if (limit !== undefined && (!Number.isSafeInteger(limit) || limit < 1))
|
|
30
|
+
if (limit !== undefined && limit !== Infinity && (!Number.isSafeInteger(limit) || limit < 1))
|
|
31
31
|
throw new FsError("EINVAL", { syscall: "python filesystem" });
|
|
32
32
|
this.#open = options.open;
|
|
33
33
|
this.#scope = composeAbortSignals([...(options.signal === undefined ? [] : [options.signal]), this.#abort.signal]);
|
|
@@ -10,7 +10,7 @@ const errno = {
|
|
|
10
10
|
};
|
|
11
11
|
export function createPythonNativeSyscalls(options) {
|
|
12
12
|
const { runtime, signal, maxTransferBytes, runtimeMount } = options;
|
|
13
|
-
if (!Number.isSafeInteger(maxTransferBytes) || maxTransferBytes < 1)
|
|
13
|
+
if (maxTransferBytes !== Infinity && (!Number.isSafeInteger(maxTransferBytes) || maxTransferBytes < 1))
|
|
14
14
|
throw new RangeError('Invalid native transfer limit');
|
|
15
15
|
const descriptors = new Map();
|
|
16
16
|
const closedStdio = new Set();
|
|
@@ -4,8 +4,8 @@ export class PythonStatTranslator {
|
|
|
4
4
|
#devices = new Map();
|
|
5
5
|
#maxDevices;
|
|
6
6
|
#next = 1;
|
|
7
|
-
constructor(maxDevices =
|
|
8
|
-
if (!Number.isSafeInteger(maxDevices) || maxDevices < 1)
|
|
7
|
+
constructor(maxDevices = Infinity) {
|
|
8
|
+
if (maxDevices !== Infinity && (!Number.isSafeInteger(maxDevices) || maxDevices < 1))
|
|
9
9
|
throw new FsError("EINVAL");
|
|
10
10
|
this.#maxDevices = maxDevices;
|
|
11
11
|
}
|
|
@@ -19,6 +19,9 @@ export class PythonStatTranslator {
|
|
|
19
19
|
if (guestDevice === undefined) {
|
|
20
20
|
if (this.#next > this.#maxDevices)
|
|
21
21
|
throw new FsError("EFBIG", { syscall: "stat", message: "Python device identity limit exceeded" });
|
|
22
|
+
// The upper half of the uint32 guest ABI is reserved for runtime devices.
|
|
23
|
+
if (this.#next >= 0x80000000)
|
|
24
|
+
throw new FsError("EFBIG", { syscall: "stat", message: "Python device identity exceeds guest ABI range" });
|
|
22
25
|
if (!devices) {
|
|
23
26
|
devices = new Map();
|
|
24
27
|
this.#devices.set(identityScope, devices);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@poe-platform/safe-fs",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.804",
|
|
4
4
|
"description": "Composable filesystem with a portable core and explicit Node adapters",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -43,12 +43,14 @@
|
|
|
43
43
|
"types": "./dist/safe-fs/testing/atomic-filesystem.d.ts",
|
|
44
44
|
"workerd": "./dist/safe-fs/testing/atomic-filesystem.js",
|
|
45
45
|
"browser": "./dist/safe-fs/testing/atomic-filesystem.js",
|
|
46
|
+
"worker": "./dist/safe-fs/testing/atomic-filesystem.js",
|
|
46
47
|
"import": "./dist/safe-fs/testing/atomic-filesystem.js"
|
|
47
48
|
},
|
|
48
49
|
"./testing/object-publication": {
|
|
49
50
|
"types": "./dist/safe-fs/testing/object-publication.d.ts",
|
|
50
51
|
"workerd": "./dist/safe-fs/testing/object-publication.js",
|
|
51
52
|
"browser": "./dist/safe-fs/testing/object-publication.js",
|
|
53
|
+
"worker": "./dist/safe-fs/testing/object-publication.js",
|
|
52
54
|
"import": "./dist/safe-fs/testing/object-publication.js"
|
|
53
55
|
},
|
|
54
56
|
"./node": {
|
|
@@ -63,10 +65,12 @@
|
|
|
63
65
|
"types": {
|
|
64
66
|
"workerd": "./dist/safe-fs/node-unavailable.d.ts",
|
|
65
67
|
"browser": "./dist/safe-fs/node-unavailable.d.ts",
|
|
68
|
+
"worker": "./dist/safe-fs/node-unavailable.d.ts",
|
|
66
69
|
"default": "./dist/safe-fs/node/filesystem.d.ts"
|
|
67
70
|
},
|
|
68
71
|
"workerd": null,
|
|
69
72
|
"browser": null,
|
|
73
|
+
"worker": null,
|
|
70
74
|
"import": "./dist/safe-fs/node/filesystem.js"
|
|
71
75
|
},
|
|
72
76
|
"./fs/memory": {
|
|
@@ -138,9 +142,15 @@
|
|
|
138
142
|
},
|
|
139
143
|
"imports": {
|
|
140
144
|
"#safe-fs-s3-request": {
|
|
141
|
-
"types":
|
|
145
|
+
"types": {
|
|
146
|
+
"workerd": "./dist/safe-fs/fs/s3/http/request-fetch.d.ts",
|
|
147
|
+
"browser": "./dist/safe-fs/fs/s3/http/request-fetch.d.ts",
|
|
148
|
+
"worker": "./dist/safe-fs/fs/s3/http/request-fetch.d.ts",
|
|
149
|
+
"default": "./dist/safe-fs/fs/s3/http/request-node.d.ts"
|
|
150
|
+
},
|
|
142
151
|
"workerd": "./dist/safe-fs/fs/s3/http/request-fetch.js",
|
|
143
152
|
"browser": "./dist/safe-fs/fs/s3/http/request-fetch.js",
|
|
153
|
+
"worker": "./dist/safe-fs/fs/s3/http/request-fetch.js",
|
|
144
154
|
"default": "./dist/safe-fs/fs/s3/http/request-node.js"
|
|
145
155
|
},
|
|
146
156
|
"#safe-fs-platform": {
|