@ai-ecoverse/slicc-kernel 1.134.1 → 1.134.4

package/README.md

@@ -859,13 +859,14 @@ A process's working directory is the directory it changed into, not just its pat
   - a wasip1 program's `.` (its `.` preopen) can't be opened or listed (`ENOENT`, an empty listing for its preopen descriptor);
   - its relative lookups and creates are `ENOENT`;
   - `..` still reaches the parent.
+  - an Emscripten program (bash, coreutils, …) and its forked subshells and children: relative paths resolve against `<path> (deleted)`, so `ls` fails with `cannot open directory '.'`, a glob matches nothing, and a relative create is `ENOENT`. `getcwd` is `ENOENT`, as on Linux, so a shell started there says `shell-init: error retrieving current directory` and its `pwd` fails, while bash's own `$PWD`, `cd ..` and `cd /abs` keep working.
+  - a JS program: `ctx.fs` and `ctx.sync.fs` with relative paths are `ENOENT`, and `../x` reaches the parent. `ctx.cwd` (and `process.cwd()`) keeps the plain path, as a WASIX program's `getcwd` does. This needs the sync lane, which JS programs have where `SharedArrayBuffer` is available.
 - **Detection costs nothing in the steady state:**
   - each held directory has a counter in memory the kernel shares with the process. It moves only when this kernel, another tab, or a `postFsNotice` reports a change at that directory or above it: the kernel's own watcher paths, and every path and tree of a peer change or notice.
   - a process checks the counter, a memory read, before each relative lookup. Only after it moved does it ask the kernel once (`cwd-check`), and the kernel compares the inode at the path with the one it recorded.
 - **The `rm` record:** so that `rm -r d; mkdir d` can be told apart, removing a directory leaves the same small record a rename leaves at the name it vacates (see [Metadata](#metadata)).
 - **Limits:**
   - a WASIX program's C library keeps the working directory as a string and turns relative paths into absolute ones before they reach the kernel. So a WASIX program still sees the new directory through relative paths, and `getcwd` still returns the path, until wasix-sysroot resolves relative paths against the working directory's descriptor ([#1164](https://github.com/ai-ecoverse/slicc-kernel/issues/1164) stays open for that).
-  - Emscripten and JS programs: [#1164](https://github.com/ai-ecoverse/slicc-kernel/issues/1164), next.
   - a removal between the check and the lookup is a race, as with any path, and the lookup sees the new directory.
   - on a `hostfs` folder the inode is the host's, which a host file system may reuse for the new directory (ext4 can; APFS doesn't). There the replacement isn't noticed.
   - with more than 1023 distinct working directories held at once, the rest are not tracked.
@@ -1096,14 +1097,14 @@ The kernel writes `/etc/passwd` and `/etc/group` when it starts (see [Users](#us
 ### How many processes at once
 
 In a browser every process worker of the page shares one renderer. V8 reserves about 8 GiB of address space for each 32-bit wasm memory, whatever its declared maximum, and its sandbox holds about 1 TiB. So a page runs about 125 wasm processes (or programs that own a wasm memory) at once, counting everything in the renderer (#970). Node has no such ceiling, and neither does Linux. The kernel keeps processes inside that ceiling:
-- **The ceiling:** 112 by default in a browser and none in Node. It's the kernel's count of live wasm processes, with the kernel's other wasm memories in the page left outside it. When an instantiation runs out of memory anyway, the kernel learns the real ceiling at that moment (the live count minus one), and that learned value wins over the default. A learned ceiling is never below 16, since an OOM at a low count means something outside it (another tab in the same renderer) filled the renderer. It's raised by 8 for each minute without another OOM, back to the default.
-- **Admission:** a spawn, fork, exec or `kernel.run` at the ceiling waits for another process to exit. It keeps waiting as long as processes exit or are admitted, and fails with `EAGAIN` only after 10 s with neither. A long `make -j` or `xargs -P` therefore queues, and a pipeline that needs more live processes than the ceiling fails cleanly. bash's own retry on a fork's `EAGAIN` (`fork: retry: Resource temporarily unavailable`) handles the rest.
+- **The ceiling:** 112 by default in a browser and none in Node (`createNodeKernel({ wasmMemories: { ceiling, idleMs } })` sets one in Node; `idleMs`, the quiet spell before `EAGAIN` below, 10 s by default, is there for tests). It's the kernel's count of live wasm processes, with the kernel's other wasm memories in the page left outside it. When an instantiation runs out of memory anyway, the kernel learns the real ceiling at that moment (the live count minus one), and that learned value wins over the default. A learned ceiling is never below 16, since an OOM at a low count means something outside it (another tab in the same renderer) filled the renderer. It's raised by 8 for each minute without another OOM, back to the default.
+- **Admission:** a spawn, fork, exec or `kernel.run` at the ceiling waits for another process to exit. It keeps waiting as long as processes exit, are admitted or finish starting (a process becoming ready counts, so a slow host that is still booting the processes ahead of it doesn't fail the spawn, #1196), and fails with `EAGAIN` only after 10 s with none of these. A long `make -j` or `xargs -P` therefore queues, and a pipeline that needs more live processes than the ceiling fails cleanly. bash's own retry on a fork's `EAGAIN` (`fork: retry: Resource temporarily unavailable`) handles the rest.
 - **Close to the ceiling** (within 16 of it), the kernel waits for each new process to finish instantiating before the spawn returns. If it ran out of memory, the kernel starts it again, quietly, once there's room, so the caller sees no failure.
 - **The rare residual case:** a process that fails to instantiate with nobody waiting on it, which happens only when something outside the count fills the renderer. It ends with status 126 and `<program>: cannot allocate memory for a new process (too many processes at once)` on its stderr, never a JS `RangeError`.
 - **Below a learned ceiling,** a process that still runs out of memory is started again once another process exits, and the spawn fails with `EAGAIN` after 10 s with none.
 - **`/proc/sys/kernel/slicc_wasm_memories`** reads `<live> <ceiling> <source>`. The source is `default`, `learned`, or `none` (with the ceiling 0).
 
-`/proc/sys/kernel/slicc_shared_root` reads `1` while another kernel shares the root (a tab, or a second kernel in the page) and `0` otherwise. A peer whose kernel ends (its tab closes or reloads) stops counting at once: each kernel queues a request for every peer's alive Web Lock, and the grant tells it the peer has gone ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)). `/proc/sys/kernel/slicc_fs_calls` reads the number of path calls (lookups, stats, opens, creates, removals and the like) the kernel has served its programs since it started. `/proc/sys/kernel/slicc_opfs_calls` counts the OPFS calls the kernel has made to look names up and read stats (`getFile`, `getFileHandle`, `getDirectoryHandle`, `resolve`). `/proc/sys/kernel/slicc_opfs_ops` counts every request the root file system makes of OPFS: those lookups, plus creates, writes (one per write, truncate or write session), reads, listings, renames and removals ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). With an idle peer, a tar unpack makes at most 1.5 times the calls it makes alone ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)).
+`/proc/sys/kernel/slicc_shared_root` reads `1` while another kernel shares the root (a tab, or a second kernel in the page) and `0` otherwise. A peer whose kernel ends (its tab closes or reloads) stops counting at once: each kernel queues a request for every peer's alive Web Lock, and the grant tells it the peer has gone ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)). `/proc/sys/kernel/slicc_fs_calls` reads the number of path calls (lookups, stats, opens, creates, removals and the like) the kernel has served its programs since it started. `/proc/sys/kernel/slicc_opfs_calls` counts the OPFS calls the kernel has made to look names up and read stats (`getFile`, `getFileHandle`, `getDirectoryHandle`, `resolve`). `/proc/sys/kernel/slicc_opfs_ops` counts every request the root file system makes of OPFS: those lookups, plus creates, writes (one per write, truncate or write session), reads, listings, renames and removals ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). With an idle peer, a tar unpack makes at most 1.5 times the calls it makes alone ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)). `/proc/sys/kernel/slicc_cwd_checks` counts the times a process asked the kernel whether its [working directory](#the-working-directory) is still the one it holds.
 
 ### Metadata
 

package/dist/kernel-worker.js

@@ -2378,14 +2378,14 @@ var OpfsFs = class {
     const staged = resolvePath(parent.path, staging);
     const place = async () => {
       this.forget(destination);
-      const timed2 = await this.explicitTimes(source.path);
+      const copies = /* @__PURE__ */ new Map();
       const root = { ino: inodeOf(source.path), ...moving };
       const owned4 = lock ? { lock } : {};
       const first = copying ? await this.intend({ from: source.path, to: staged, root, ...owned4 }) : void 0;
       const second = copying && swap ? await this.intend({ from: staged, to: destination, replace: true, root, ...owned4 }) : void 0;
       const moved2 = await this.replacingTarget(
         targetStat !== null,
-        () => this.place(source.handle, dir, staging)
+        () => this.place(source.handle, dir, staging, copies)
       ).catch(async (err) => {
         if (first) await this.rollBack(staged, source.path, [first, second]);
         throw err;
@@ -2399,15 +2399,13 @@ var OpfsFs = class {
         await this.meta.update(source.path, () => left);
       }
       if (first) await this.meta.update(first, () => void 0);
-      let copied = !moved2;
       if (swap) {
         const hidden = await dir.getDirectoryHandle(staging);
-        const placed = await this.place(hidden, dir, name2);
+        const placed = await this.place(hidden, dir, name2, copies);
         if (!placed) await dir.removeEntry(staging, { recursive: true });
-        copied ||= !placed;
       }
       if (second) await this.meta.update(second, () => void 0);
-      if (copied) await this.restamp(destination, timed2);
+      await this.restamp(destination, copies);
       this.changed(source.path, destination);
       this.touch(Date.now(), parentOf3(source.path), parent.path);
     };
@@ -2624,22 +2622,17 @@ var OpfsFs = class {
     const handle = await this.handle(path);
     return handle.kind === "file" ? (await this.snapshot(handle)).lastModified : 0;
   }
-  async explicitTimes(path) {
-    const timed2 = [];
-    for (const entry of await this.meta.under(path)) {
-      if (entry.mtimeFor === void 0 || entry.link !== void 0) continue;
-      const current = await this.modified(entry.path).catch(() => void 0);
-      if (current === entry.mtimeFor) timed2.push(entry.path.slice(path.length));
-    }
-    return timed2;
-  }
-  async restamp(path, timed2) {
-    for (const rest2 of timed2) {
-      const mtimeFor = await this.modified(path + rest2);
+  async restamp(path, copies) {
+    if (copies.size === 0) return;
+    const copied = [...copies];
+    const entries = await this.meta.get(copied.map(([rest2]) => path + rest2));
+    for (const [i, [rest2, { before, made: made2 }]] of copied.entries()) {
+      if (entries[i]?.mtimeFor !== before) continue;
+      const mtimeFor = (await this.io(made2.getFile())).lastModified;
       await this.meta.update(path + rest2, (entry) => entry && { ...entry, mtimeFor });
     }
   }
-  async place(handle, dir, name2) {
+  async place(handle, dir, name2, copies) {
     const move = handle.move;
     if (move) this.others++;
     const moved2 = await move?.call(handle, dir, name2).then(
@@ -2649,7 +2642,7 @@ var OpfsFs = class {
     if (moved2) return true;
     const before = this.fileMoves;
     try {
-      await this.copy(handle, dir, name2);
+      await this.copy(handle, dir, name2, copies);
     } catch (err) {
       if (handle.kind === "directory" && this.fileMoves === before) {
         await this.io(dir.removeEntry(name2, { recursive: true }));
@@ -2659,7 +2652,7 @@ var OpfsFs = class {
     return false;
   }
   fileMoves = 0;
-  async copy(handle, dir, name2) {
+  async copy(handle, dir, name2, copies, rest2 = "") {
     if (handle.kind === "file" && movable(handle)) {
       this.others++;
       const moved2 = await handle.move?.(dir, name2).then(
@@ -2674,13 +2667,17 @@ var OpfsFs = class {
     if (handle.kind === "file") {
       const made2 = await this.io(dir.getFileHandle(name2, { create: true }));
       const writable2 = await this.io(made2.createWritable());
-      await writable2.write(await this.io(handle.getFile()));
+      const file = await this.io(handle.getFile());
+      await writable2.write(file);
       await writable2.close();
+      copies?.set(rest2, { before: copies.get(rest2)?.before ?? file.lastModified, made: made2 });
       return;
     }
     const target = await this.io(dir.getDirectoryHandle(name2, { create: true }));
     this.others++;
-    for await (const child of handle.values()) await this.copy(child, target, child.name);
+    for await (const child of handle.values()) {
+      await this.copy(child, target, child.name, copies, `${rest2}/${child.name}`);
+    }
   }
   async symlink(target, path, owner) {
     const parent = await this.locate(parentOf3(path), true);
@@ -7101,7 +7098,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.134.1" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.134.4" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -13665,7 +13662,7 @@ var AsyncRangedFile = class {
 };
 
 // src/realm/sync-sab-bridge.ts
-var PATH_OPS = new Set(SYNC_FS_OPS);
+var UNCOUNTED_OPS = /* @__PURE__ */ new Set([...SYNC_FS_OPS, "cwd-check", "cwd-chdir"]);
 var SPIN_CORES = 4;
 var spinning = (globalThis.navigator?.hardwareConcurrency ?? 0) >= SPIN_CORES;
 
@@ -13713,6 +13710,8 @@ var SYS_TREE = {
     slicc_opfs_calls: (listing) => `${listing.opfsCalls ?? 0}
 `,
     slicc_opfs_ops: (listing) => `${listing.opfsOps ?? 0}
+`,
+    slicc_cwd_checks: (listing) => `${listing.cwdChecks ?? 0}
 `
   }
 };
@@ -24772,6 +24771,9 @@ var WasmMemories = class {
     const source = this.learned ? "learned" : ceiling === void 0 ? "none" : "default";
     return { live: this.live, ceiling: ceiling ?? 0, source };
   }
+  started() {
+    this.progress();
+  }
   ended() {
     this.live = Math.max(0, this.live - 1);
     this.progress();
@@ -28088,6 +28090,7 @@ var Launcher = class {
       });
       const counted2 = (handle2) => {
         void handle2.exited.finally(() => this.memories.ended());
+        void Promise.resolve(handle2.ready).then(() => this.memories.started());
         return handle2;
       };
       if (!this.memories.near()) {
@@ -28186,7 +28189,8 @@ var Launcher = class {
         peers: Number(this.locks.shared?.() === true),
         fsCalls: servedFsCalls(),
         opfsCalls: this.rootFs.opfsCalls?.() ?? 0,
-        opfsOps: this.rootFs.opfsOps?.() ?? 0
+        opfsOps: this.rootFs.opfsOps?.() ?? 0,
+        cwdChecks: this.cwds.checks
       }),
       sudo: (call, signal) => this.sudo(pid, call, signal),
       crontab: (call) => this.crontab(pid, call),