@ai-ecoverse/slicc-kernel 1.126.0 → 1.126.1

package/README.md

@@ -648,6 +648,8 @@ The proxy answers `200` with `Content-Type: application/vnd.slicc.raw-fetch`: a
 
 `/` is the OPFS root. Each process mounts the top-level directories that exist when it starts (plus `/usr` and `/bin`), so files in them are shared by all processes and visible through the OPFS API as soon as the process that wrote them has closed them; `run` resolves only after that. `createKernel` creates `/tmp` and `/home` in OPFS, so they are shared too. Files directly in `/`, and directories created there after a process started, are OPFS as well, the same for Emscripten and WASI programs; only `/dev` and `/proc` live in each process's memory. A program's `mkdir` (WASI `path_create_directory`, Emscripten's `mkdir`) is exclusive, as on Linux: it fails with `EEXIST` when the name exists, and of several processes or threads making one name at once exactly one succeeds, so a directory works as a lock. Under a missing directory it fails with `ENOENT`, as on Linux (`mkdir -p` and `os.makedirs` make the parents in the program); a JS program's `ctx.fs.mkdir` makes parents itself. The kernel makes the system `PNPM_HOME` (`/usr/local/share/pnpm`, or the embedder's) when it starts, since pnpm makes only its `bin` directory there. A full origin storage (`QuotaExceededError`, which also comes early in a browser's private or off-the-record contexts, whatever `navigator.storage.estimate()` says) is `ENOSPC`, never retried. A read that meets `NotFoundError` while a writer outside the kernel swaps the file in with `createWritable` (and a `stat`, `lstat` or `utimes` that meets it) looks the name up again, up to three times, before it fails ([#1075](https://github.com/ai-ecoverse/slicc-kernel/issues/1075)), so a program whose package is overwritten in place keeps starting. An OPFS or IndexedDB failure that has no POSIX counterpart is `EIO`; its message names the DOM error and the operation (`lookup`, `read`, `write`, `commit` or `sidecar`), and the kernel worker logs the first of each kind to its console. File contents are buffered per open file and written back on close, `fsync` and exit, so a refused write (`ENOSPC`, `EIO`) surfaces at that close or `fsync`, and a program that ignores close errors doesn't see it (`echo x > f` can exit 0 with `f` empty, as with any buffered write-back). A write-back error a close has already reported to a WASI program isn't raised again when it exits, so the program keeps its own exit status. Metadata operations (`mkdir`, `rename`, `rm`, …) go straight to OPFS. The kernel worker is the only writer.
 
+**Renaming a directory on the OPFS root** ([#1122](https://github.com/ai-ecoverse/slicc-kernel/issues/1122)): Chrome's OPFS can move a file but not a directory. So the kernel renames a directory by creating the new tree and moving each file into it (copying a file only if it can't move, e.g. while it's open elsewhere), then removing the old tree. Within the kernel the rename is atomic to readers: a lookup or listing under the old or the new name, or of either parent, waits until the whole tree has moved. So a stage-then-rename updater (pnpm's, an atomic config write) is never seen half there. Another tab sharing the root reads OPFS directly and can see the tree mid-move. A root whose directories can move (the Node memory root) renames them in one step. **If the page reloads or crashes mid-rename**, some files are already in the new tree and the rest are still in the old one. So before the first move, the kernel writes the rename's intent (`{ from, to }`) to its metadata store, and clears it after the last. At the next boot, when no other tab shares the root (the same rule as for the boot's other repair of stale metadata), the kernel finishes every recorded rename: it moves what's left into the new tree, removes the old name, and clears the record. While another tab shares the root, the split tree stays as it is (both names visible, each with part of the files) until a boot that's alone.
+
 A lookup follows at most 40 symlinks, as Linux does: the 41st is `ELOOP`, for WASI programs, JS programs and the kernel API alike, with or without `..` in a link's target. Emscripten programs resolve paths in their glue, which stops one link earlier, at 40.
 
 Directories are renamed with `FileSystemHandle.move()` where available, else by copy and delete.

package/dist/kernel-worker.js

@@ -1098,6 +1098,10 @@ var STALE = /* @__PURE__ */ new Set(["NotFoundError", "TypeMismatchError"]);
 var REPLACED_ATTEMPTS = 4;
 var RECONCILE_LISTINGS = 8;
 var EXECUTABLE = /\/node_modules\/(?:@[^/]+\/)?[^/]+\/bin\//;
+var RENAMES = "/.slicc-renames";
+function movable(handle) {
+  return typeof handle.move === "function";
+}
 var ERRNO_BY_DOM_ERROR = {
   NotFoundError: "ENOENT",
   TypeMismatchError: "ENOTDIR",
@@ -2271,9 +2275,9 @@ var OpfsFs = class {
     }
     const now = Date.now();
     const moving = await this.movedOwner(source);
-    const carry = (entry) => ({
+    const carry = (entry2) => ({
       ino: inodeOf(source.path),
-      ...occupied(entry),
+      ...occupied(entry2),
       ...moving,
       path: destination,
       ctimeMs: now
@@ -2289,29 +2293,127 @@ var OpfsFs = class {
     const name2 = baseOf(destination);
     const swap = targetStat?.isDirectory === true;
     const staging = swap ? `.${name2}.${crypto.randomUUID()}` : name2;
-    this.forget(destination);
-    const timed2 = await this.explicitTimes(source.path);
-    const moved2 = await this.replacingTarget(
-      targetStat !== null,
-      () => this.place(source.handle, dir, staging)
+    const copying = sourceStat.isDirectory && !movable(source.handle);
+    const staged = resolvePath(parent.path, staging);
+    const place = async () => {
+      this.forget(destination);
+      const timed2 = await this.explicitTimes(source.path);
+      const first = copying ? await this.intend({ from: source.path, to: staged }) : void 0;
+      const second = copying && swap ? await this.intend({ from: staged, to: destination, replace: true }) : void 0;
+      const moved2 = await this.replacingTarget(
+        targetStat !== null,
+        () => this.place(source.handle, dir, staging)
+      ).catch(async (err) => {
+        if (first) await this.rollBack(staged, source.path, [first, second]);
+        throw err;
+      });
+      if (swap) await this.rm(destination, { recursive: true });
+      if (second) await this.intend({ from: staged, to: destination }, second);
+      await this.meta.move(source.path, destination, carry, left);
+      this.movePending(source.path, destination);
+      if (!moved2) {
+        await this.rm(source.path, { recursive: true });
+        await this.meta.update(source.path, () => left);
+      }
+      if (first) await this.meta.update(first, () => void 0);
+      let copied = !moved2;
+      if (swap) {
+        const hidden = await dir.getDirectoryHandle(staging);
+        const placed = await this.place(hidden, dir, name2);
+        if (!placed) await dir.removeEntry(staging, { recursive: true });
+        copied ||= !placed;
+      }
+      if (second) await this.meta.update(second, () => void 0);
+      if (copied) await this.restamp(destination, timed2);
+      this.changed(source.path, destination);
+      this.touch(Date.now(), parentOf3(source.path), parent.path);
+    };
+    if (!copying) return place();
+    const { promise, resolve: resolve2 } = Promise.withResolvers();
+    const entry = { from: source.path, to: destination, done: promise };
+    this.renamingDirs.add(entry);
+    try {
+      await place();
+    } finally {
+      this.renamingDirs.delete(entry);
+      resolve2();
+    }
+  }
+  renamingDirs = /* @__PURE__ */ new Set();
+  intents = 0;
+  async intend(rename, at = `${RENAMES}/${crypto.randomUUID()}`) {
+    const record = { ...rename, at: Date.now() * 1e3 + this.intents++ % 1e3 };
+    await this.meta.update(at, () => ({ path: at, rename: record }));
+    return at;
+  }
+  async rollBack(from, to, intents) {
+    try {
+      if (await this.real(from)) {
+        const moved2 = await this.locate(from, false);
+        const back = await this.locate(parentOf3(to), true);
+        await this.copy(
+          moved2.handle,
+          back.handle,
+          baseOf(to)
+        );
+        const [dir, name2] = await this.parent(from);
+        await this.inDir(dir, (parent) => parent.removeEntry(name2, { recursive: true }));
+      }
+      this.forget(from);
+      this.forget(to);
+      for (const intent of intents) if (intent) await this.meta.update(intent, () => void 0);
+    } catch {
+    }
+  }
+  async finishRenames() {
+    const intents = (await this.meta.under(RENAMES)).filter((entry) => entry.rename !== void 0).sort(
+      (a, b) => a.rename.at - b.rename.at
     );
-    if (swap) await this.rm(destination, { recursive: true });
-    await this.meta.move(source.path, destination, carry, left);
-    this.movePending(source.path, destination);
-    if (!moved2) {
-      await this.rm(source.path, { recursive: true });
-      await this.meta.update(source.path, () => left);
-    }
-    let copied = !moved2;
-    if (swap) {
-      const staged = await dir.getDirectoryHandle(staging);
-      const placed = await this.place(staged, dir, name2);
-      if (!placed) await dir.removeEntry(staging, { recursive: true });
-      copied ||= !placed;
-    }
-    if (copied) await this.restamp(destination, timed2);
-    this.changed(source.path, destination);
-    this.touch(Date.now(), parentOf3(source.path), parent.path);
+    for (const entry of intents) {
+      const { from, to, replace } = entry.rename;
+      const left = await this.real(from);
+      if (left && replace && await this.real(to)) await this.rm(to, { recursive: true });
+      if (left) {
+        const source = await this.locate(from, false);
+        const target = await this.locate(parentOf3(to), true);
+        await this.copy(
+          source.handle,
+          target.handle,
+          baseOf(to)
+        );
+        const [dir, name2] = await this.parent(from);
+        await this.inDir(dir, (parent) => parent.removeEntry(name2, { recursive: true }));
+        this.forget(from);
+        this.forget(to);
+      }
+      if ((await this.meta.under(from)).length > 0) {
+        await this.meta.move(from, to, (moved2) => ({ ...occupied(moved2), path: to }));
+      }
+      await this.meta.update(entry.path, () => void 0);
+    }
+    return intents.length;
+  }
+  async settledRename(path) {
+    if (this.renamingDirs.size === 0 || typeof path !== "string") return;
+    const at = normalizePath(path);
+    for (let real; ; ) {
+      const busy = this.renameAt(at) ?? (real === void 0 ? void 0 : this.renameAt(real));
+      if (busy) {
+        await busy.done;
+        continue;
+      }
+      if (real !== void 0 || at === "/") return;
+      const parent = await this.realpath(parentOf3(at), true).catch(() => parentOf3(at));
+      real = resolvePath(parent, baseOf(at));
+      if (real === at) return;
+    }
+  }
+  renameAt(at) {
+    for (const m of this.renamingDirs) {
+      if (within2(at, m.from) || within2(at, m.to) || at === parentOf3(m.from) || at === parentOf3(m.to))
+        return m;
+    }
+    return void 0;
   }
   async replacingTarget(replaces, op) {
     if (!replaces) return op();
@@ -2353,15 +2455,29 @@ var OpfsFs = class {
       () => false
     );
     if (moved2) return true;
+    const before = this.fileMoves;
     try {
       await this.copy(handle, dir, name2);
     } catch (err) {
-      if (handle.kind === "directory") await dir.removeEntry(name2, { recursive: true });
+      if (handle.kind === "directory" && this.fileMoves === before) {
+        await dir.removeEntry(name2, { recursive: true });
+      }
       throw translate(err, name2);
     }
     return false;
   }
+  fileMoves = 0;
   async copy(handle, dir, name2) {
+    if (handle.kind === "file" && movable(handle)) {
+      const moved2 = await handle.move?.(dir, name2).then(
+        () => true,
+        () => false
+      );
+      if (moved2) {
+        this.fileMoves++;
+        return;
+      }
+    }
     if (handle.kind === "file") {
       const writable2 = await (await dir.getFileHandle(name2, { create: true })).createWritable();
       await writable2.write(await handle.getFile());
@@ -2498,6 +2614,7 @@ var OpfsFs = class {
   }
   async reconcile() {
     if (this.remote && await this.remote.peered()) return 0;
+    await this.finishRenames();
     let removed = 0;
     const entries = await this.meta.all();
     const carried = new Set(entries.filter((e) => !e.vacated).map((e) => e.ino));
@@ -2528,7 +2645,9 @@ var READS = [
   "readdir",
   "readdirStat",
   "readlink",
-  "realpath"
+  "realpath",
+  "version",
+  "readVersioned"
 ];
 var WRITES = [
   "writeFile",
@@ -2554,6 +2673,7 @@ for (const [names2, op] of [
     const inner = proto[name2];
     proto[name2] = async function(...args) {
       try {
+        if (op === "read") await this.settledRename(args[0]);
         return await inner.apply(this, args);
       } catch (err) {
         throw translate(err, String(args[0]), op, false);
@@ -6699,7 +6819,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.126.0" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.126.1" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC