@ai-ecoverse/slicc-kernel 1.117.0 → 1.117.2

package/README.md

@@ -701,7 +701,7 @@ Programs mount and unmount with `mount(2)` and `umount2(2)`, so a `mount`/`umoun
 
 ### Exec
 
-An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux, whether the descriptors live in the kernel or a program opened the file in its own worker: every open, read-only ones included, is known to the kernel, and all holders share the one file (writes through one are seen through the others). The kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves the holders' reads and writes from there (a worker's I/O on the old path while the path is gone). No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Under the Node entry, which has no Web Locks, entries are named after the kernel's process id instead, and an entry is removed when that process is gone (`kill(pid, 0)` fails with `ESRCH`); a reused pid leaves the entry alone. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
+An exec'd program takes over the pid of the process it replaces, as on Linux: `getpid()` and `$$` in the new image, `/proc`, `ps`, `kill` and the parent's `waitpid` all agree on that pid, and the image's parent pid is the replaced process's. Its own children see that pid as their parent. A file removed or renamed over while programs hold it open keeps its bytes for them, as on Linux, whether the descriptors live in the kernel or a program opened the file in its own worker: every open, read-only ones included, is known to the kernel, and all holders share the one file (writes through one are seen through the others). The kernel does not copy it: as NFS does with its "silly rename", it moves the file into a hidden directory, `.slicc-unlinked/`, at the root of the mount it lives on (or of the root file system), and serves the holders' reads and writes from there (a worker's I/O on the old path while the path is gone). No file data is held in memory for it. The file is deleted when the last holder closes it or exits, and the directory goes as soon as it is empty. A file created at the path afterwards is the new file for everyone, also for the holder; a holder whose file another process renames (rather than removes) still uses the old path. A rename over a held file stays atomic for everyone else, as `rename(2)` is: moving the held file aside and the new one into its place are two steps, so a lookup of that name (`open`, `stat`, a read) that comes while they run waits for both, and one that started before them and missed the name in between looks again (also when it spelled the name through a symlinked parent, which the wait doesn't match by path: a miss is retried whenever any replacement finished meanwhile, up to three tries). Every open finds the old file or the new one, never no file (#903). The OPFS root does the same for its own two steps: a rename onto an existing name replaces the file with `move()` (or, in a browser without it, a copy into the name and a removal of the source), and a lookup that loses that race to a stale handle looks again. A `tmpfs` mount replaces the name in one step; on `fsa` and `hostfs` mounts it is the real file system's `rename`. Listings in the kernel leave `.slicc-unlinked` out, but on an `fsa` or `hostfs` mount it is visible on the real machine while a removed file is held (add `.slicc-unlinked/` to a `.gitignore` there). Each kernel names its entries after itself and holds a Web Lock for that name while it runs; when a mount is set up (and the root at start), the kernel removes the entries whose owner holds no lock, so a tab that died while holding a file leaves nothing behind for long, and entries of other live tabs on the same origin are left alone. Under the Node entry, which has no Web Locks, entries are named after the kernel's process id instead, and an entry is removed when that process is gone (`kill(pid, 0)` fails with `ESRCH`); a reused pid leaves the entry alone. Signals a program ignores stay ignored in the image it execs and in the children it spawns or forks, as POSIX has it, so `nohup cmd` and `trap '' TERM; exec cmd` keep `cmd` safe from them; caught signals go back to their default. The kernel holds what was handed down, so a new image that sets one of those signals back to the default with `signal(SIG_DFL)` still ignores it, until it installs a handler.
 
 - **Emscripten:** `Module.sliccKernel.execve(file, argv, env, cwd)` runs `file` as this process's new image on its fds 0, 1 and 2, waits for it, and returns its wait status, or a negative errno when it cannot start. `env` and `cwd` may be `null` for this process's own. An exec shim detects it with `typeof Module.sliccKernel.execve === 'function'` and exits with the status it returns, as `execve` never returns on success. Older shims spawn the program and then call `Module.sliccKernel.execWait(pid)`; that works the same way: a new program asks the kernel for its pid before `main`, and the kernel answers once its parent has made its next call, so an image an older shim execs also reports the pid it replaced (the wait starts once the program is loaded; should the parent make no call within 100 ms after that, the kernel answers anyway, and the image may then report its own pid; `launcher.identities` counts the programs that asked and those answered by that timeout).
 - **WASIX:** `proc_exec` does this by itself.
@@ -1007,7 +1007,7 @@ The tools in `tools/cert-review/`:
 
 `.github/workflows/cert-review-pr.yml` certifies a pull request before it merges, so `main`, and with it npm's `latest`, only gets certified changes. It reports as the `cert-review` commit status, a required check of `main`'s merge queue. Every same-repo pull request gets one (`not behaviour`, `waits for merge-ok`, the review's verdict, or the self-guard's hold). **A pull request from a fork gets none**, because fork code never reaches the pools or the key, so it can't enter the queue: only Lars merges one, after reading it.
 
-- **Which pull requests:** those that change shipped behaviour, by a conservative path filter (`behaviourFiles` in `lib.mjs`): `src/`, `build.mjs`, `package.json` and its lock, `tsconfig*.json`, `.releaserc*`, `tools/release-recover.mjs`, the workflows that build or publish (`release.yml`, `ci.yml`), and the reviewer's own code (`tools/cert-review/`, `cert-review*.yml`, `.github/runner.mjs`). **Main's code decides:** every job checks out the pull request's base and fetches only the trees under test, so the path filter, pack, suite, findings, the instructions and record all run from `main`. A pull request that changes the reviewer is reviewed by `main`'s reviewer. The workflow file of a `pull_request` run still comes from the pull request, so a change to the reviewer is held by the **certification self-guard**, the shared composite action `slicc-shared-web/.github/actions/cert-self-guard` at a pinned sha. Its light `self-guard` job runs on every pull request event and every merge group. When the diff touches the reviewer (`.github/workflows/cert*.yml`, `tools/cert-review/**`, `.github/runner.mjs` and the guard's own paths), `cert-review` is red with `reviewer change: needs coordinator review (cert-review-approved on this head)` and no review runs. The label is the Kernel Coordinator's alone: they add `cert-review-approved` after reading the change, and the review then starts with `merge-ok`. The approval is bound to the head's arrival on GitHub, and a push removes the label. A merge group that holds an unapproved reviewer change is red too. If the guard doesn't finish, the status is an error that asks to re-run the failed jobs. **The residual:** the pull request's YAML still decides what runs, so the guard catches an accidental edit, not a deliberate rewrite of this workflow. The pull request that adds this workflow can't pass its own check, because its base has no tools yet. Any other pull request gets a green status that says `not behaviour: <files>`, so the skip is recorded.
+- **Which pull requests:** those that change shipped behaviour, by a conservative path filter (`behaviourFiles` in `lib.mjs`): `src/`, `build.mjs`, `package.json` and its lock, `tsconfig*.json`, `.releaserc*`, `tools/release-recover.mjs`, the workflows that build or publish (`release.yml`, `ci.yml`), and the reviewer's own code (`tools/cert-review/`, `cert-review*.yml`, `.github/runner.mjs`). **Main's code decides:** every job checks out the pull request's base and fetches only the trees under test, so the path filter, pack, suite, findings, the instructions and record all run from `main`. The merge group's lookup loads its tools from the protected branch itself (`merge_group.base_ref`), not from the group's `base_sha`, which can be the speculative commit of the group ahead and carry the reviewer changes of the pull requests queued before it; `base_sha` is only the lookup's input. A pull request that changes the reviewer is reviewed by `main`'s reviewer. The workflow file of a `pull_request` run still comes from the pull request, so a change to the reviewer is held by the **certification self-guard**, the shared composite action `slicc-shared-web/.github/actions/cert-self-guard` at a pinned sha. Its light `self-guard` job runs on every pull request event and every merge group. When the diff touches the reviewer (`.github/workflows/cert*.yml`, `tools/cert-review/**`, `.github/runner.mjs` and the guard's own paths), `cert-review` is red with `reviewer change: needs coordinator review (cert-review-approved on this head)` and no review runs. The label is the Kernel Coordinator's alone: they add `cert-review-approved` after reading the change, and the review then starts with `merge-ok`. The approval is bound to the head's arrival on GitHub, and a push removes the label. A merge group that holds an unapproved reviewer change is red too. If the guard doesn't finish, the status is an error that asks to re-run the failed jobs. **The residual:** the pull request's YAML still decides what runs, so the guard catches an accidental edit, not a deliberate rewrite of this workflow. The pull request that adds this workflow can't pass its own check, because its base has no tools yet. Any other pull request gets a green status that says `not behaviour: <files>`, so the skip is recorded.
 - **Decide by what ships:** when a pull request's behaviour files are only `package.json` and/or `package-lock.json` (a Renovate devDependency bump, say), both sides are packed as usual, and then the `same dist` job (`same.mjs`, in code) compares the two tarballs before anything else runs. It checks the same paths, entry types (a symlink isn't a file) and modes. `package.json` must be equal once `devDependencies`, `version` and `gitHead` are removed, so `scripts` (they ship, a consumer's install runs `postinstall`, and `prepack`/`prepublishOnly` can make the published build differ from the packed one), runtime dependencies, `exports`, `bin`, `files` and `engines` must all match. Every other file must be byte-identical once each side's **exact** version string (build.mjs bakes it into `dist`) is replaced by one placeholder. A version string in one more place, or any looser match, is a difference. Source maps are compared as JSON without their `mappings`, whose columns the longer or shorter version string shifts; the code itself is compared byte for byte. If all of that matches, `cert-review` is green with `not behaviour (identical dist): <candidate sha256> = <base sha256> after normalising <v1>/<v2>`, and the suites, kernel-compat, the review and the record are skipped. `same.json` (uploaded as an artifact) names the rule and both tarballs' full sha256. Any difference, or a `same dist` job that doesn't finish, means the full review. The kernel has no runtime `dependencies`, since esbuild bundles everything into `dist`, so a devDependency that changes the product shows up as a byte difference. A test-only devDependency is checked by `ci / test`, on the pull request and on the merge group.
 - **When:** only for a same-repo pull request with `merge-ok`, when the label is added and on every push while it is set. Each pull request has its own concurrency group, and a newer push cancels the older review. A behaviour change without `merge-ok` gets a red status, `waits for merge-ok to be reviewed`. Other labels don't start or cancel anything. Fork code never reaches the pools or the key.
 - **What:** the checkout is the event's default `refs/pull/N/merge`, the pull request merged onto `main` at review time, so a review is of what would land, without pushing to the branch. Both sides are packed in the same run, on a GitHub-hosted runner, with no secrets: the candidate (the merge ref, `x.y.z-pr.N.<run>`) and the base (its first parent, `main`). Then:

package/dist/kernel-worker.js

@@ -998,6 +998,7 @@ var ABSENT_MS = 2;
 var ABSENT_NAMES = 4096;
 var FILE_HANDLES = 4096;
 var STALE = /* @__PURE__ */ new Set(["NotFoundError", "TypeMismatchError"]);
+var REPLACED_ATTEMPTS = 4;
 var EXECUTABLE = /\/node_modules\/(?:@[^/]+\/)?[^/]+\/bin\//;
 var ERRNO_BY_DOM_ERROR = {
   NotFoundError: "ENOENT",
@@ -1132,6 +1133,8 @@ var OpfsFs = class {
   files = new PathMap();
   absent = new PathMap();
   changes = 0;
+  replacing = Promise.resolve();
+  replacements = 0;
   snapshots = /* @__PURE__ */ new WeakMap();
   found = /* @__PURE__ */ new WeakMap();
   checked = /* @__PURE__ */ new WeakMap();
@@ -1466,7 +1469,22 @@ var OpfsFs = class {
     }
     return void 0;
   }
-  async locate(path, follow2) {
+  locate(path, follow2) {
+    return this.unreplaced(() => this.located(path, follow2));
+  }
+  async unreplaced(op) {
+    for (let attempt = 1; ; attempt++) {
+      const seen = this.replacements;
+      try {
+        return await op();
+      } catch (err) {
+        if (attempt >= REPLACED_ATTEMPTS || !missing2(err)) throw err;
+        await this.replacing;
+        if (this.replacements === seen) throw err;
+      }
+    }
+  }
+  async located(path, follow2) {
     let current = path;
     for (let hops = 0; hops <= MAX_LINKS2; hops++) {
       try {
@@ -1664,11 +1682,11 @@ var OpfsFs = class {
     const stat = await this.statOf(found.path, found.handle, meta, touched);
     return { ...stat, ...ownOf(meta) ?? inherit(entries) };
   }
-  async stat(path) {
-    return this.describe(await this.locate(path, true));
+  stat(path) {
+    return this.unreplaced(async () => this.describe(await this.located(path, true)));
   }
-  async lstat(path) {
-    return this.describe(await this.locate(path, false));
+  lstat(path) {
+    return this.unreplaced(async () => this.describe(await this.located(path, false)));
   }
   async exists(path) {
     return this.locate(path, true).then(
@@ -1692,6 +1710,12 @@ var OpfsFs = class {
       }
     }
   }
+  reading(path, op) {
+    return this.unreplaced(async () => {
+      const handle = await this.existing(path);
+      return this.retrying(path, () => op(handle), false);
+    });
+  }
   async existing(path, writing = false) {
     const found = await this.locate(path, true);
     if (writing) this.refuse(found.path);
@@ -1705,52 +1729,35 @@ var OpfsFs = class {
     if (entry?.fifo || entry?.socket) throw fsError("ENXIO", path);
   }
   async readFileBuffer(path) {
-    const handle = await this.existing(path);
-    const bytes = await this.retrying(
+    const bytes = await this.reading(
       path,
-      async () => new Uint8Array(await (await handle.getFile()).arrayBuffer()),
-      false
+      async (handle) => new Uint8Array(await (await handle.getFile()).arrayBuffer())
     );
     await this.refuseNode(path, bytes.length === 0);
     return bytes;
   }
   async readVersioned(path, cached = false) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const stamp = await this.stamp(path, cached);
-        const file = await handle.getFile();
-        const bytes = new Uint8Array(await file.arrayBuffer());
-        await this.refuseNode(path, bytes.length === 0);
-        return { bytes, version: `${file.lastModified}:${file.size}:${stamp}` };
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const stamp = await this.stamp(path, cached);
+      const file = await handle.getFile();
+      const bytes = new Uint8Array(await file.arrayBuffer());
+      await this.refuseNode(path, bytes.length === 0);
+      return { bytes, version: `${file.lastModified}:${file.size}:${stamp}` };
+    });
   }
   async version(path, cached = false) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const stamp = await this.stamp(path, cached);
-        const file = await handle.getFile();
-        return `${file.lastModified}:${file.size}:${stamp}`;
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const stamp = await this.stamp(path, cached);
+      const file = await handle.getFile();
+      return `${file.lastModified}:${file.size}:${stamp}`;
+    });
   }
   async pread(path, offset, length2) {
-    const handle = await this.existing(path);
-    return this.retrying(
-      path,
-      async () => {
-        const file = await handle.getFile();
-        await this.refuseNode(path, file.size === 0);
-        return new Uint8Array(await file.slice(offset, offset + length2).arrayBuffer());
-      },
-      false
-    );
+    return this.reading(path, async (handle) => {
+      const file = await handle.getFile();
+      await this.refuseNode(path, file.size === 0);
+      return new Uint8Array(await file.slice(offset, offset + length2).arrayBuffer());
+    });
   }
   async pwrite(path, offset, bytes) {
     const [handle, , target] = await this.fileFor(path);
@@ -2063,7 +2070,10 @@ var OpfsFs = class {
     const staging = swap ? `.${name2}.${crypto.randomUUID()}` : name2;
     this.forget(destination);
     const timed2 = await this.explicitTimes(source.path);
-    const moved2 = await this.place(source.handle, dir, staging);
+    const moved2 = await this.replacingTarget(
+      targetStat !== null,
+      () => this.place(source.handle, dir, staging)
+    );
     if (swap) await this.rm(destination, { recursive: true });
     await this.meta.move(source.path, destination, carry);
     this.movePending(source.path, destination);
@@ -2079,6 +2089,21 @@ var OpfsFs = class {
     this.changed(source.path, destination);
     this.touch(Date.now(), parentOf3(source.path), parent.path);
   }
+  async replacingTarget(replaces, op) {
+    if (!replaces) return op();
+    const { promise, resolve: resolve2 } = Promise.withResolvers();
+    const before = this.replacing;
+    this.replacing = Promise.all([before, promise]).then(() => void 0);
+    this.replacements++;
+    this.changes++;
+    try {
+      return await op();
+    } finally {
+      this.replacements++;
+      this.changes++;
+      resolve2();
+    }
+  }
   async modified(path) {
     const handle = await this.handle(path);
     return handle.kind === "file" ? (await handle.getFile()).lastModified : 0;
@@ -6260,7 +6285,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.117.0" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.117.2" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -7089,6 +7114,16 @@ var UNLINKED_DIR = ".slicc-unlinked";
 function webLocks() {
   return globalThis.navigator?.locks;
 }
+var LOOKUP_OPS = [
+  "exists",
+  "readFile",
+  "readFileBuffer",
+  "readVersioned",
+  "version",
+  "pread",
+  "readlink",
+  "realpath"
+];
 var within4 = (path, target) => path === target || target === "/" || path.startsWith(`${target}/`);
 var parentOf5 = (path) => path.slice(0, path.lastIndexOf("/")) || "/";
 var lockName = (instance) => `slicc-unlinked:${instance}`;
@@ -7127,6 +7162,8 @@ var UnlinkedKeeper = class {
   gate = new PathGate();
   queue = Promise.resolve();
   count = 0;
+  replacing = /* @__PURE__ */ new Map();
+  replaced = 0;
   links = /* @__PURE__ */ new Map();
   linked = /* @__PURE__ */ new Map();
   dirOwner;
@@ -7149,6 +7186,44 @@ var UnlinkedKeeper = class {
   }
   wrap(fs = this.fs) {
     const keeper = this;
+    const { replacing } = this;
+    const replace = async (target, op) => {
+      const before = replacing.get(target);
+      const { promise, resolve: resolve2 } = Promise.withResolvers();
+      const mine = {
+        done: Promise.all([before?.done, promise]).then(() => void 0)
+      };
+      replacing.set(target, mine);
+      try {
+        await op();
+      } finally {
+        keeper.replaced += 1;
+        resolve2();
+        if (replacing.get(target) === mine) replacing.delete(target);
+      }
+    };
+    const unreplaced = async (path, op) => {
+      const at = fs.resolvePath("/", path);
+      for (let attempt = 1; ; attempt++) {
+        await replacing.get(at)?.done;
+        const seen = keeper.replaced;
+        try {
+          return await op();
+        } catch (err) {
+          const code = err?.code;
+          if (attempt >= 3 || code !== "ENOENT") throw err;
+          await replacing.get(at)?.done;
+          if (keeper.replaced === seen) throw err;
+        }
+      }
+    };
+    const looking = (name2) => {
+      const call = fs[name2];
+      if (!call) return {};
+      return {
+        [name2]: (path, ...rest2) => unreplaced(path, () => call.call(fs, path, ...rest2))
+      };
+    };
     const visible = (names) => names.filter((name2) => name2 !== UNLINKED_DIR);
     const counted2 = (st) => {
       const hidden = keeper.links.get(`${st.dev ?? 0}:${st.ino}`) ?? 0;
@@ -7156,9 +7231,10 @@ var UnlinkedKeeper = class {
     };
     return {
       ...fs,
+      ...LOOKUP_OPS.reduce((all, name2) => Object.assign(all, looking(name2)), {}),
       links: (path) => mayLink(fs, path),
-      stat: async (path) => counted2(await fs.stat(path)),
-      lstat: async (path) => counted2(await fs.lstat(path)),
+      stat: (path) => unreplaced(path, async () => counted2(await fs.stat(path))),
+      lstat: (path) => unreplaced(path, async () => counted2(await fs.lstat(path))),
       async rm(path, options) {
         const target = fs.resolvePath("/", path);
         const pinned = await keeper.pin(target);
@@ -7168,8 +7244,10 @@ var UnlinkedKeeper = class {
       async rename(from, to) {
         const target = fs.resolvePath("/", to);
         if (target === fs.resolvePath("/", from)) return fs.rename(from, to);
-        const pinned = await keeper.pin(target);
-        await keeper.settle(pinned, () => fs.rename(from, to));
+        await replace(target, async () => {
+          const pinned = await keeper.pin(target);
+          await keeper.settle(pinned, () => fs.rename(from, to));
+        });
       },
       readdir: async (path) => visible(await fs.readdir(path)),
       readdirStat: async (path) => (await fs.readdirStat(path)).filter(([name2]) => name2 !== UNLINKED_DIR).map(([name2, st]) => [name2, st && counted2(st)])