@ai-ecoverse/slicc-kernel 1.117.1 → 1.117.2

package/README.md

@@ -701,7 +701,7 @@ Programs mount and unmount with `mount(2)` and `umount2(2)`, so a `mount`/`umoun
 
 ### Exec
 
-An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux, whether the descriptors live in the kernel or a program opened the file in its own worker: every open, read-only ones included, is known to the kernel, and all holders share the one file (writes through one are seen through the others). The kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves the holders' reads and writes from there (a worker's I/O on the old path while the path is gone). No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Under the Node entry, which has no Web Locks, entries are named after the kernel's process id instead, and an entry is removed when that process is gone (`kill(pid, 0)` fails with `ESRCH`); a reused pid leaves the entry alone. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
+An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux, whether the descriptors live in the kernel or a program opened the file in its own worker: every open, read-only ones included, is known to the kernel, and all holders share the one file (writes through one are seen through the others). The kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves the holders' reads and writes from there (a worker's I/O on the old path while the path is gone). No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. A rename over a held file stays atomic for everyone else, as `rename(2)` is: moving the held file aside and the new one into its place are two steps, so a lookup of that name (`open`, `stat`, a read) that comes while they run waits for both, and one that started before them and missed the name in between looks again (also when it spelled the name through a symlinked parent, which the wait doesn't match by path: a miss is retried whenever any replacement finished meanwhile, up to three tries). Every open finds the old file or the new one, never no file (#903). The OPFS root does the same for its own two steps: a rename onto an existing name replaces the file with `move()` (or, in a browser without it, a copy into the name and a removal of the source), and a lookup that loses that race to a stale handle looks again. A `tmpfs` mount replaces the name in one step; on `fsa` and `hostfs` mounts it is the real file system's `rename`. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Under the Node entry, which has no Web Locks, entries are named after the kernel's process id instead, and an entry is removed when that process is gone (`kill(pid, 0)` fails with `ESRCH`); a reused pid leaves the entry alone. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
 
 - **Emscripten:** `Module.sliccKernel.execve(file, argv, env, cwd)` runs `file` as this process's new image on its fds 0, 1 and 2, waits for it, and returns its wait status, or a negative errno when it cannot start. `env` and `cwd` may be `null` for this process's own. An exec shim detects it with `typeof Module.sliccKernel.execve === 'function'` and exits with the status it returns, as `execve` never returns on success. Older shims spawn the program and then call `Module.sliccKernel.execWait(pid)`; that works the same way: a new program asks the kernel for its pid before `main`, and the kernel answers once its parent has made its next call, so an image an older shim execs also reports the pid it replaced (the wait starts once the program is loaded; should the parent make no call within 100 ms after that, the kernel answers anyway, and the image may then report its own pid; `launcher.identities` counts the programs that asked and those answered by that timeout).
 - **WASIX:** `proc_exec` does this by itself.

package/dist/kernel-worker.js

@@ -998,6 +998,7 @@ var ABSENT_MS = 2;
 var ABSENT_NAMES = 4096;
 var FILE_HANDLES = 4096;
 var STALE = /* @__PURE__ */ new Set(["NotFoundError", "TypeMismatchError"]);
+var REPLACED_ATTEMPTS = 4;
 var EXECUTABLE = /\/node_modules\/(?:@[^/]+\/)?[^/]+\/bin\//;
 var ERRNO_BY_DOM_ERROR = {
   NotFoundError: "ENOENT",
@@ -1132,6 +1133,8 @@ var OpfsFs = class {
   files = new PathMap();
   absent = new PathMap();
   changes = 0;
+  replacing = Promise.resolve();
+  replacements = 0;
   snapshots = /* @__PURE__ */ new WeakMap();
   found = /* @__PURE__ */ new WeakMap();
   checked = /* @__PURE__ */ new WeakMap();
@@ -1466,7 +1469,22 @@ var OpfsFs = class {
     }
     return void 0;
   }
-  async locate(path, follow2) {
+  locate(path, follow2) {
+    return this.unreplaced(() => this.located(path, follow2));
+  }
+  async unreplaced(op) {
+    for (let attempt = 1; ; attempt++) {
+      const seen = this.replacements;
+      try {
+        return await op();
+      } catch (err) {
+        if (attempt >= REPLACED_ATTEMPTS || !missing2(err)) throw err;
+        await this.replacing;
+        if (this.replacements === seen) throw err;
+      }
+    }
+  }
+  async located(path, follow2) {
     let current = path;
     for (let hops = 0; hops <= MAX_LINKS2; hops++) {
       try {
@@ -1664,11 +1682,11 @@ var OpfsFs = class {
     const stat = await this.statOf(found.path, found.handle, meta, touched);
     return { ...stat, ...ownOf(meta) ?? inherit(entries) };
   }
-  async stat(path) {
-    return this.describe(await this.locate(path, true));
+  stat(path) {
+    return this.unreplaced(async () => this.describe(await this.located(path, true)));
   }
-  async lstat(path) {
-    return this.describe(await this.locate(path, false));
+  lstat(path) {
+    return this.unreplaced(async () => this.describe(await this.located(path, false)));
   }
   async exists(path) {
     return this.locate(path, true).then(
@@ -1692,6 +1710,12 @@ var OpfsFs = class {
       }
     }
   }
+  reading(path, op) {
+    return this.unreplaced(async () => {
+      const handle = await this.existing(path);
+      return this.retrying(path, () => op(handle), false);
+    });
+  }
   async existing(path, writing = false) {
     const found = await this.locate(path, true);
     if (writing) this.refuse(found.path);
@@ -1705,52 +1729,35 @@ var OpfsFs = class {
     if (entry?.fifo || entry?.socket) throw fsError("ENXIO", path);
   }
   async readFileBuffer(path) {
-    const handle = await this.existing(path);
-    const bytes = await this.retrying(
+    const bytes = await this.reading(
       path,
-      async () => new Uint8Array(await (await handle.getFile()).arrayBuffer()),
-      false
+      async (handle) => new Uint8Array(await (await handle.getFile()).arrayBuffer())
     );
     await this.refuseNode(path, bytes.length === 0);
     return bytes;
   }
   async readVersioned(path, cached = false) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const stamp = await this.stamp(path, cached);
-        const file = await handle.getFile();
-        const bytes = new Uint8Array(await file.arrayBuffer());
-        await this.refuseNode(path, bytes.length === 0);
-        return { bytes, version: `${file.lastModified}:${file.size}:${stamp}` };
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const stamp = await this.stamp(path, cached);
+      const file = await handle.getFile();
+      const bytes = new Uint8Array(await file.arrayBuffer());
+      await this.refuseNode(path, bytes.length === 0);
+      return { bytes, version: `${file.lastModified}:${file.size}:${stamp}` };
+    });
   }
   async version(path, cached = false) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const stamp = await this.stamp(path, cached);
-        const file = await handle.getFile();
-        return `${file.lastModified}:${file.size}:${stamp}`;
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const stamp = await this.stamp(path, cached);
+      const file = await handle.getFile();
+      return `${file.lastModified}:${file.size}:${stamp}`;
+    });
   }
   async pread(path, offset, length2) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const file = await handle.getFile();
-        await this.refuseNode(path, file.size === 0);
-        return new Uint8Array(await file.slice(offset, offset + length2).arrayBuffer());
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const file = await handle.getFile();
+      await this.refuseNode(path, file.size === 0);
+      return new Uint8Array(await file.slice(offset, offset + length2).arrayBuffer());
+    });
   }
   async pwrite(path, offset, bytes) {
     const [handle, , target] = await this.fileFor(path);
@@ -2063,7 +2070,10 @@ var OpfsFs = class {
     const staging = swap ? `.${name2}.${crypto.randomUUID()}` : name2;
     this.forget(destination);
     const timed2 = await this.explicitTimes(source.path);
-    const moved2 = await this.place(source.handle, dir, staging);
+    const moved2 = await this.replacingTarget(
+      targetStat !== null,
+      () => this.place(source.handle, dir, staging)
+    );
     if (swap) await this.rm(destination, { recursive: true });
     await this.meta.move(source.path, destination, carry);
     this.movePending(source.path, destination);
@@ -2079,6 +2089,21 @@ var OpfsFs = class {
     this.changed(source.path, destination);
     this.touch(Date.now(), parentOf3(source.path), parent.path);
   }
+  async replacingTarget(replaces, op) {
+    if (!replaces) return op();
+    const { promise, resolve: resolve2 } = Promise.withResolvers();
+    const before = this.replacing;
+    this.replacing = Promise.all([before, promise]).then(() => void 0);
+    this.replacements++;
+    this.changes++;
+    try {
+      return await op();
+    } finally {
+      this.replacements++;
+      this.changes++;
+      resolve2();
+    }
+  }
   async modified(path) {
     const handle = await this.handle(path);
     return handle.kind === "file" ? (await handle.getFile()).lastModified : 0;
@@ -6260,7 +6285,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.117.1" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.117.2" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -7089,6 +7114,16 @@ var UNLINKED_DIR = ".slicc-unlinked";
 function webLocks() {
   return globalThis.navigator?.locks;
 }
+var LOOKUP_OPS = [
+  "exists",
+  "readFile",
+  "readFileBuffer",
+  "readVersioned",
+  "version",
+  "pread",
+  "readlink",
+  "realpath"
+];
 var within4 = (path, target) => path === target || target === "/" || path.startsWith(`${target}/`);
 var parentOf5 = (path) => path.slice(0, path.lastIndexOf("/")) || "/";
 var lockName = (instance) => `slicc-unlinked:${instance}`;
@@ -7127,6 +7162,8 @@ var UnlinkedKeeper = class {
   gate = new PathGate();
   queue = Promise.resolve();
   count = 0;
+  replacing = /* @__PURE__ */ new Map();
+  replaced = 0;
   links = /* @__PURE__ */ new Map();
   linked = /* @__PURE__ */ new Map();
   dirOwner;
@@ -7149,6 +7186,44 @@ var UnlinkedKeeper = class {
   }
   wrap(fs = this.fs) {
     const keeper = this;
+    const { replacing } = this;
+    const replace = async (target, op) => {
+      const before = replacing.get(target);
+      const { promise, resolve: resolve2 } = Promise.withResolvers();
+      const mine = {
+        done: Promise.all([before?.done, promise]).then(() => void 0)
+      };
+      replacing.set(target, mine);
+      try {
+        await op();
+      } finally {
+        keeper.replaced += 1;
+        resolve2();
+        if (replacing.get(target) === mine) replacing.delete(target);
+      }
+    };
+    const unreplaced = async (path, op) => {
+      const at = fs.resolvePath("/", path);
+      for (let attempt = 1; ; attempt++) {
+        await replacing.get(at)?.done;
+        const seen = keeper.replaced;
+        try {
+          return await op();
+        } catch (err) {
+          const code = err?.code;
+          if (attempt >= 3 || code !== "ENOENT") throw err;
+          await replacing.get(at)?.done;
+          if (keeper.replaced === seen) throw err;
+        }
+      }
+    };
+    const looking = (name2) => {
+      const call = fs[name2];
+      if (!call) return {};
+      return {
+        [name2]: (path, ...rest2) => unreplaced(path, () => call.call(fs, path, ...rest2))
+      };
+    };
     const visible = (names) => names.filter((name2) => name2 !== UNLINKED_DIR);
     const counted2 = (st) => {
       const hidden = keeper.links.get(`${st.dev ?? 0}:${st.ino}`) ?? 0;
@@ -7156,9 +7231,10 @@ var UnlinkedKeeper = class {
     };
     return {
       ...fs,
+      ...LOOKUP_OPS.reduce((all, name2) => Object.assign(all, looking(name2)), {}),
       links: (path) => mayLink(fs, path),
-      stat: async (path) => counted2(await fs.stat(path)),
-      lstat: async (path) => counted2(await fs.lstat(path)),
+      stat: (path) => unreplaced(path, async () => counted2(await fs.stat(path))),
+      lstat: (path) => unreplaced(path, async () => counted2(await fs.lstat(path))),
       async rm(path, options) {
         const target = fs.resolvePath("/", path);
         const pinned = await keeper.pin(target);
@@ -7168,8 +7244,10 @@ var UnlinkedKeeper = class {
       async rename(from, to) {
         const target = fs.resolvePath("/", to);
         if (target === fs.resolvePath("/", from)) return fs.rename(from, to);
-        const pinned = await keeper.pin(target);
-        await keeper.settle(pinned, () => fs.rename(from, to));
+        await replace(target, async () => {
+          const pinned = await keeper.pin(target);
+          await keeper.settle(pinned, () => fs.rename(from, to));
+        });
       },
       readdir: async (path) => visible(await fs.readdir(path)),
       readdirStat: async (path) => (await fs.readdirStat(path)).filter(([name2]) => name2 !== UNLINKED_DIR).map(([name2, st]) => [name2, st && counted2(st)])