@ai-ecoverse/slicc-kernel 1.39.1 → 1.39.3

package/README.md

@@ -420,7 +420,7 @@ Programs mount and unmount with `mount(2)` and `umount2(2)`, so a `mount`/`umoun
 
 ### Exec
 
-An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux: the kernel keeps one copy for all holders, shared by their descriptors (writes through one are seen through the others), and frees it when the last one closes. That holds for files a program opened in its own worker too: every open, read-only ones included, is known to the kernel. When another process removes such a file, or renames another file over it, the kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves that holder's reads and writes of the old path from there while the path is gone. No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Where Web Locks are missing (the Node entry), nothing is swept. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
+An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux, whether the descriptors live in the kernel or a program opened the file in its own worker: every open, read-only ones included, is known to the kernel, and all holders share the one file (writes through one are seen through the others). The kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves the holders' reads and writes from there (a worker's I/O on the old path while the path is gone). No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Under the Node entry, which has no Web Locks, entries are named after the kernel's process id instead, and an entry is removed when that process is gone (`kill(pid, 0)` fails with `ESRCH`); a reused pid leaves the entry alone. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
 
 - **Emscripten:** `Module.sliccKernel.execve(file, argv, env, cwd)` runs `file` as this process's new image on its fds 0, 1 and 2, waits for it, and returns its wait status, or a negative errno when it cannot start. `env` and `cwd` may be `null` for this process's own. An exec shim detects it with `typeof Module.sliccKernel.execve === 'function'` and exits with the status it returns, as `execve` never returns on success. Older shims spawn the program and then call `Module.sliccKernel.execWait(pid)`; that works the same way: a new program asks the kernel for its pid before `main`, and the kernel answers once its parent has made its next call, so an image an older shim execs also reports the pid it replaced (the wait starts once the program is loaded; should the parent make no call within 100 ms after that, the kernel answers anyway, and the image may then report its own pid; `launcher.identities` counts the programs that asked and those answered by that timeout).
 - **WASIX:** `proc_exec` does this by itself.

package/dist/kernel-worker.js

@@ -5315,16 +5315,20 @@ var UnlinkedKeeper = class {
   locks;
   busy;
   waiting = /* @__PURE__ */ new Set();
-  instance = crypto.randomUUID();
+  owner;
+  instance;
   gate = new PathGate();
   queue = Promise.resolve();
   count = 0;
-  constructor(fs, holders, rootOf, locks, busy = () => false) {
+  constructor(fs, holders, rootOf, options = {}) {
+    const { locks, owner } = options;
     this.fs = fs;
     this.holders = holders;
     this.rootOf = rootOf;
     this.locks = locks;
-    this.busy = busy;
+    this.busy = options.busy ?? (() => false);
+    this.owner = owner;
+    this.instance = owner ? `${owner.id}~${crypto.randomUUID()}` : crypto.randomUUID();
     void locks?.request(lockName(this.instance), () => new Promise(() => void 0));
   }
   serial(op) {
@@ -5332,8 +5336,7 @@ var UnlinkedKeeper = class {
     this.queue = next.catch(() => void 0);
     return next;
   }
-  wrap() {
-    const fs = this.fs;
+  wrap(fs = this.fs) {
     const keeper = this;
     const visible = (names) => names.filter((name2) => name2 !== UNLINKED_DIR);
     return {
@@ -5345,7 +5348,9 @@ var UnlinkedKeeper = class {
         else await keeper.settle(pinned, () => fs.rm(path, options));
       },
       async rename(from, to) {
-        const pinned = await keeper.pin(fs.resolvePath("/", to));
+        const target = fs.resolvePath("/", to);
+        if (target === fs.resolvePath("/", from)) return fs.rename(from, to);
+        const pinned = await keeper.pin(target);
         await keeper.settle(pinned, () => fs.rename(from, to));
       },
       readdir: async (path) => visible(await fs.readdir(path)),
@@ -5365,8 +5370,8 @@ var UnlinkedKeeper = class {
     const held = /* @__PURE__ */ new Map();
     for (const holder of this.holders) {
       if (holder.owns(target)) continue;
-      for (const path of holder.opens.keys()) {
-        if (!within3(path, target) || holder.unlinked.has(path)) continue;
+      for (const path of holder.held()) {
+        if (!within3(path, target) || holder.isKept(path)) continue;
         held.set(path, [...held.get(path) ?? [], holder]);
       }
     }
@@ -5381,21 +5386,25 @@ var UnlinkedKeeper = class {
     const done4 = () => {
       for (const release of releases) release();
     };
+    const files = /* @__PURE__ */ new Map();
+    for (const [path, at] of hidden) {
+      const holders = held.get(path);
+      const file = this.keptFile(at, { refs: holders.length + 1 });
+      files.set(path, file);
+      for (const holder of holders) holder.keep(path, file);
+    }
     return {
       moved: new Set(hidden.keys()),
       commit: () => {
-        for (const [path, at] of hidden) {
-          const holders = held.get(path);
-          const file = this.keptFile(at, { refs: holders.length });
-          for (const holder of holders) holder.unlinked.set(path, file);
-        }
+        for (const file of files.values()) file.release();
         done4();
       },
       rollback: async () => {
-        for (const [path, at] of hidden) {
+        for (const [path, file] of files) {
+          for (const holder of held.get(path)) holder.unkeep(path, file);
           await this.serial(async () => {
-            await Promise.allSettled([this.fs.rename(at, path)]);
-            await this.tidy(parentOf3(at));
+            await Promise.allSettled([this.fs.rename(file.hidden, path)]);
+            await this.tidy(parentOf3(file.hidden));
           });
         }
         done4();
@@ -5441,16 +5450,18 @@ var UnlinkedKeeper = class {
   }
   sweep(root) {
     const query = this.locks?.query?.bind(this.locks);
-    if (!query) return Promise.resolve();
+    const owner = this.owner;
+    if (!query && !owner) return Promise.resolve();
     return this.serial(async () => {
       const dir = dirOf(root);
       const names = await this.fs.readdir(dir).catch(() => void 0);
       if (!names) return;
-      const held = new Set(((await query()).held ?? []).map((lock) => lock.name));
+      const held = query && new Set(((await query()).held ?? []).map((lock) => lock.name));
       for (const name2 of names) {
-        const owner = name2.slice(0, name2.lastIndexOf("."));
-        if (owner === this.instance || held.has(lockName(owner))) continue;
-        await this.fs.rm(`${dir}/${name2}`, { recursive: true }).catch(() => void 0);
+        const id = name2.slice(0, name2.lastIndexOf("."));
+        const at = id.indexOf("~");
+        const dead = id !== this.instance && (at < 0 ? held !== void 0 && !held.has(lockName(id)) : owner?.alive(id.slice(0, at)) === false);
+        if (dead) await this.fs.rm(`${dir}/${name2}`, { recursive: true }).catch(() => void 0);
       }
       await this.tidy(dir);
     });
@@ -5653,6 +5664,7 @@ var ChildTable = class {
     if ("fd" in slot) return this.parentFds.get(slot.fd).retain();
     if ("input" in slot) return bytesSource(slot.input);
     if ("device" in slot) return deviceFile(slot.device, slot.access);
+    if ("dir" in slot) return heldFile({ dir: slot.dir });
     if ("capture" in slot) {
       const chunks = [];
       captured.set(n, chunks);
@@ -5884,6 +5896,18 @@ var HeldPaths = class extends Map {
   holding(path) {
     return this.opens.has(this.keyOf(path));
   }
+  held() {
+    return this.opens.keys();
+  }
+  isKept(path) {
+    return this.unlinked.has(path);
+  }
+  keep(path, file) {
+    this.unlinked.set(path, file);
+  }
+  unkeep(path) {
+    this.unlinked.delete(path);
+  }
   hold(path, on, open = false, real) {
     if (open) {
       const key = real ?? this.keyOf(path);
@@ -6751,6 +6775,9 @@ async function run(entry, path, req) {
     case "symlink":
       await fs.symlink(req.arg2 ?? "", path);
       return done;
+    case "create":
+      await createExclusive(fs, path);
+      return done;
     case "readlink":
       return json2(await fs.readlink(path));
     case "chmod":
@@ -6770,6 +6797,15 @@ async function run(entry, path, req) {
       return { ok: false, errno: "EINVAL", message: `sync-fs: unknown op '${req.op}'` };
   }
 }
+async function createExclusive(fs, path) {
+  if (fs.createExclusive) return fs.createExclusive(path);
+  if (await fs.lstat(path).then(
+    () => true,
+    () => false
+  ))
+    throw syncError("EEXIST", path);
+  await fs.writeFile(path, new Uint8Array(0));
+}
 async function ranged(fs, path, req) {
   const ops = rangedOps(fs);
   if (!ops) throw syncError("ENOSYS", path);
@@ -8384,6 +8420,8 @@ var VfsNode = class {
 };
 var VfsNodes = class {
   byPath = /* @__PURE__ */ new Map();
+  kept = /* @__PURE__ */ new Map();
+  keepsOnDisk = false;
   fs;
   entries = /* @__PURE__ */ new Map();
   onClosed;
@@ -8466,15 +8504,47 @@ var VfsNodes = class {
       return;
     }
     if (this.byPath.get(node.path) === node) this.byPath.delete(node.path);
+    this.kept.get(node)?.release();
+    this.kept.delete(node);
     this.onClosed?.(node.path);
   }
+  held() {
+    return this.byPath.keys();
+  }
+  owns() {
+    return false;
+  }
+  isKept() {
+    return false;
+  }
+  keep(path, file) {
+    const node = this.byPath.get(path);
+    if (!node) {
+      file.release();
+      return;
+    }
+    this.byPath.delete(path);
+    node.renamedTo(file.hidden);
+    this.byPath.set(file.hidden, node);
+    this.kept.set(node, file);
+  }
+  unkeep(path, file) {
+    const node = this.byPath.get(file.hidden);
+    if (!node) return;
+    this.byPath.delete(file.hidden);
+    node.renamedTo(path);
+    this.byPath.set(path, node);
+    this.kept.delete(node);
+  }
   async flush(path) {
     for (const [p, node] of this.byPath) if (within5(p, path)) await node.serial(() => node.flush());
   }
   async unlinking(path) {
     for (const [held, node] of [...this.byPath]) {
       if (!within5(held, path)) continue;
-      await node.serial(async () => void await Promise.all([node.keep(), node.limit()]));
+      await node.serial(async () => {
+        await Promise.all([this.keepsOnDisk || node.keep(), node.limit()]);
+      });
     }
   }
   revoke(prefix) {
@@ -8604,6 +8674,7 @@ function keepingOpen(fs, nodes) {
       });
     },
     writeFile: (path, content) => nodes.onEntry(fs.resolvePath("/", path), () => fs.writeFile(path, content)),
+    createExclusive: async (path) => nodes.createExclusive(await nodes.entryKey(fs.resolvePath("/", path))),
     symlink: (target, path) => nodes.onEntry(fs.resolvePath("/", path), () => fs.symlink(target, path)),
     mkdir: (path, options) => nodes.onEntry(fs.resolvePath("/", path), () => fs.mkdir(path, options))
   };
@@ -11262,15 +11333,20 @@ var Launcher = class {
     enableCdp(this.net, this.cdp);
     const fs = withCommandDirs(this.base, async () => new Set((await this.commands()).keys()));
     this.nodes = new VfsNodes(fs, (path) => this.released(path));
+    this.nodes.keepsOnDisk = true;
     this.openFiles.add(this.nodes);
+    const holders = [this.held, [this.nodes]];
     this.unlinked = new UnlinkedKeeper(
       fs,
-      this.held,
+      { [Symbol.iterator]: () => holders.flatMap((set2) => [...set2]).values() },
       (path) => this.mountRoot(path),
-      webLocks(),
-      (path) => [...this.openFiles].some((nodes) => nodes.holds(path))
+      {
+        locks: webLocks(),
+        busy: (path) => [...this.openFiles].some((nodes) => nodes.holds(path)),
+        owner: options.unlinkOwner
+      }
     );
-    this.fs = keepingOpen(this.unlinked.wrap(), this.nodes);
+    this.fs = this.unlinked.wrap(keepingOpen(fs, this.nodes));
     void this.unlinked.sweep("/");
     this.watchers.watch([this.modulesDir, this.pnpmHome], { recursive: true }, () => {
       this.catalog = void 0;