@ai-ecoverse/slicc-kernel 1.123.0 → 1.123.2

package/README.md

@@ -933,7 +933,7 @@ OPFS stores names, bytes, sizes and modification times, nothing else. Everything
 
 - mode bits (`chmod`; a `chmod` to the mode a file or directory already has writes nothing and leaves its ctime as it was, which saves a sidecar write for every installer that sets the mode a file was made with);
 - access and change times, and a modification time set explicitly with `utime` (it holds until the file is written again: any write or truncate moves it to the time of the write, also in Chrome, whose OPFS doesn't move a file's `lastModified` for writes through a sync access handle, so the kernel records the write's time itself, and it reaches the metadata store up to 100 ms after the write, so a tab closed inside that window keeps the old mtime on disk; the time follows a rename and is dropped with the file); a directory's times move whenever an entry is created, removed or renamed in it, as POSIX asks, and a time set explicitly after such a change keeps winning, also when the change was made in another kernel and its write lands later; an Emscripten program reads a directory's times again after any call that can wait on another process (a pipe or socket read, `wait`, `poll`, a sleep), so a program that waits for a child or a peer to fill a directory sees the new times (rsync's final directory times rely on it), while a walk without waits reads each directory once; `chown`/`fchown` leave the times alone (Emscripten stamps the setattr of a chown with the current time, which isn't taken as an mtime);
-- inode numbers that stay stable across renames;
+- inode numbers that stay stable across renames. A file with no record has an inode derived from its path, and a rename keeps it on the file; the rename leaves a small record at the name it vacated with a fresh inode, so a file, symlink or directory made there later doesn't share the renamed one's ([#954](https://github.com/ai-ecoverse/slicc-kernel/issues/954)). An exclusive create doesn't count that record as a taken name, and the boot reconcile keeps it only while a live record still carries the inode it guards;
 - symbolic links (`symlink`, `readlink`, `lstat`, and following them in paths), which exist only in the sidecar, with their own times: `touch -h`, `lutimes` and `utimensat` with `AT_SYMLINK_NOFOLLOW` set the link's, never the target's. A link's own mode can't be changed, as on Linux: `chmod -h`, `lchmod` and `fchmodat` with `AT_SYMLINK_NOFOLLOW` fail with `EOPNOTSUPP` (which `chmod -h` and `tar` ignore for links) and never follow it, and `open` with `O_PATH | O_NOFOLLOW` gives a descriptor for the link itself.
 
 OPFS stays authoritative: an entry is consulted only for a path that exists in OPFS (symlinks only for paths that don't), so files written straight to OPFS, for example by the BIOS installer or a page, get the defaults: directories `0755`, files `0644`, and files under `node_modules/*/bin/` `0755`. A file deleted through the OPFS API disappears at once; its leftover entry is ignored and removed when the next kernel starts. Changes go to OPFS first and then to the sidecar, so a crash in between at worst loses the metadata of that one change. Each change is a read-modify-write inside one IndexedDB transaction, so two kernels (two tabs) on the same origin don't lose each other's updates.
@@ -946,6 +946,8 @@ Not covered: hard links (OPFS cannot share contents between names; they work onl
 
 Reads retry briefly when a concurrent write has invalidated the OPFS file snapshot (`NotReadableError`).
 
+At boot, a kernel drops sidecar records whose file is gone (and symlink records whose name became a real file or whose directory is gone), left behind by a tab that closed mid-change. It does this only when it is the only kernel on the origin (the watch channel's join handshake finds no other), and it drops a record only if it is still the one it found: another tab's live changes made while it looked, such as a placeholder tar replaces with a symlink, are kept. Up to 1.119.2, a tab that booted while another extracted symlinks with tar could drop some of them.
+
 A kernel caches the directory handles it has walked and the sidecar entries it has read. Its own changes keep both caches current, and the other kernels on the origin hear of them through a `BroadcastChannel` named after the database (one for the sidecar, one for directories). A cached directory is checked against its path (`resolve()`) at most once a second before a lookup goes through it, so a directory moved, removed or replaced by another writer, such as a page using the OPFS API, is found at its new place or reported missing within a second. It also keeps the file handles it has looked up (up to 4,096, the least recently used dropped first) for a second after each lookup. Before it uses one, it reads the file through it, which is the read a `stat` needs anyway, and that read serves the handle's uses of the next 2 ms. So a file another writer removed is missing, and a rewritten one shows its new contents, within 2 ms. The kernel's own writes, truncates, rewrites and renames drop the read at once, and a read that started before an own change isn't kept, so the kernel's own changes always show at once. One replaced by a directory is seen as a directory within a second. Its own changes, and the other kernels' notices, drop the handles they touch at once. A lookup of a directory it has cached and checked within the second is answered from the cache, so a directory replaced by another writer is seen as replaced within a second, as above. A name it has just found missing stays missing for 2 ms, which serves the several checks a single create makes of a name that isn't there yet (about ten per file in a pnpm install); its own creates, renames and links in that directory end that at once, so only a file another kernel or a page using the OPFS API makes in those 2 ms is seen up to 2 ms late. Both 2 ms windows are off while another kernel shares the root (a second tab on the origin, found through the watch channel's join handshake, and until that handshake is done), so between tabs every look is fresh, as before 1.113.0; only a page writing through the OPFS API itself can still be seen up to 2 ms late ([#945](https://github.com/ai-ecoverse/slicc-kernel/issues/945)). A single tab keeps them.
 
 The filesystem is our own rather than [ZenFS](https://github.com/zen-fs/core) (which SLICC uses), because OPFS stays the single source of truth: other writers, such as the BIOS installing packages or a page writing files, need no index to stay consistent with, and nothing is preloaded into memory at mount. POSIX metadata lives in the IndexedDB sidecar described above, which has per-entry transactions instead of one JSON file rewritten on every change.
@@ -1031,6 +1033,7 @@ The tools in `tools/cert-review/`:
 `.github/workflows/cert-review-pr.yml` certifies a pull request before it merges, so `main`, and with it npm's `latest`, only gets certified changes. It reports as the `cert-review` commit status, a required check of `main`'s merge queue. Every same-repo pull request gets one (`not behaviour`, `waits for merge-ok`, the review's verdict, or the self-guard's hold). **A pull request from a fork gets none**, because fork code never reaches the pools or the key, so it can't enter the queue: only Lars merges one, after reading it.
 
 - **Which pull requests:** those that change shipped behaviour, by a conservative path filter (`behaviourFiles` in `lib.mjs`): `src/`, `build.mjs`, `package.json` and its lock, `tsconfig*.json`, `.releaserc*`, `tools/release-recover.mjs`, the workflows that build or publish (`release.yml`, `ci.yml`), and the reviewer's own code (`tools/cert-review/`, `cert-review*.yml`, `.github/runner.mjs`). **Main's code decides:** every job checks out the pull request's base and fetches only the trees under test, so the path filter, pack, suite, findings, the instructions and record all run from `main`. The merge group's lookup loads its tools from the protected branch itself (`merge_group.base_ref`), not from the group's `base_sha`, which can be the speculative commit of the group ahead and carry the reviewer changes of the pull requests queued before it; `base_sha` is only the lookup's input. A pull request that changes the reviewer is reviewed by `main`'s reviewer. The workflow file of a `pull_request` run still comes from the pull request, so a change to the reviewer is held by the **certification self-guard**, the shared composite action `slicc-shared-web/.github/actions/cert-self-guard` at a pinned sha. Its light `self-guard` job runs on every pull request event and every merge group. When the diff touches the reviewer (`.github/workflows/cert*.yml`, `tools/cert-review/**`, `.github/runner.mjs` and the guard's own paths), `cert-review` is red with `reviewer change: needs coordinator review (cert-review-approved on this head)` and no review runs. The label is the Kernel Coordinator's alone: they add `cert-review-approved` after reading the change, and the review then starts with `merge-ok`. The approval is bound to the head's arrival on GitHub, and a push removes the label. A merge group that holds an unapproved reviewer change is red too. If the guard doesn't finish, the status is an error that asks to re-run the failed jobs. **The residual:** the pull request's YAML still decides what runs, so the guard catches an accidental edit, not a deliberate rewrite of this workflow. The pull request that adds this workflow can't pass its own check, because its base has no tools yet. Any other pull request gets a green status that says `not behaviour: <files>`, so the skip is recorded.
+- **A gate's silence never reads as a pass:** every tool a workflow gates on (`pr.mjs plan`, `same` and `lookup`, `same.mjs`, `pack.mjs`, `delta.mjs`, `suite.mjs`, `compat.mjs`, `findings.mjs`, `record.mjs`, `verify.mjs`) ends with one verdict line, `<tool>: PASS <what>` or `<tool>: FAIL <why>`, from the shared `verdict()` in `lib.mjs`. `tools/cert-review/gate.sh [--either|--evidence] <tool> <command>` runs a tool, passes its output through, and exits 70 with an error when the tool's last line starting `<tool>: ` isn't a PASS or FAIL verdict, so a tool replaced by `true`, an early exit 0 or an empty run fails its step. By default the step passes only on PASS with exit 0; a PASS with a non-zero exit, or a FAIL with exit 0, fails it. `--either` keeps the tool's own exit, for tools whose FAIL is evidence that record judges later (suite, compat). `--evidence` needs only the verdict, for the NEGATIVE run on main's tree, which is expected to fail. `suite: PASS` means exit 0, no failures and every expected file has results; `compat: PASS` means a run turned up with no regressions. `prompt.mjs` writes the review prompt to stdout, which goes into a step output, so it prints no verdict; a missing prompt leaves the review without `verdict.json`, and record marks that INCOMPLETE. The gate reads the tool's stdout, and no tool passes the pull request's output there: suite pipes the tests' output into its log file, and compat prints only its own lines, so a test that prints `suite: PASS` can't stand in for a silent suite. A tool that let such output through would need its verdict on a separate descriptor. Because the workflow of a pull request runs main's tools, the tools gain their verdicts first, and the workflows call them through `gate.sh` in the pull request after that.
 - **Decide by what ships:** when a pull request's behaviour files are only `package.json` and/or `package-lock.json` (a Renovate devDependency bump, say), both sides are packed as usual, and then the `same dist` job (`same.mjs`, in code) compares the two tarballs before anything else runs. It checks the same paths, entry types (a symlink isn't a file) and modes. `package.json` must be equal once `devDependencies`, `version` and `gitHead` are removed, so `scripts` (they ship, a consumer's install runs `postinstall`, and `prepack`/`prepublishOnly` can make the published build differ from the packed one), runtime dependencies, `exports`, `bin`, `files` and `engines` must all match. Every other file must be byte-identical once each side's **exact** version string (build.mjs bakes it into `dist`) is replaced by one placeholder. A version string in one more place, or any looser match, is a difference. Source maps are compared as JSON without their `mappings`, whose columns the longer or shorter version string shifts; the code itself is compared byte for byte. If all of that matches, `cert-review` is green with `not behaviour (identical dist): <candidate sha256> = <base sha256> after normalising <v1>/<v2>`, and the suites, kernel-compat, the review and the record are skipped. `same.json` (uploaded as an artifact) names the rule and both tarballs' full sha256. Any difference, or a `same dist` job that doesn't finish, means the full review. The kernel has no runtime `dependencies`, since esbuild bundles everything into `dist`, so a devDependency that changes the product shows up as a byte difference. A test-only devDependency is checked by `ci / test`, on the pull request and on the merge group.
 - **When:** only for a same-repo pull request with `merge-ok`, when the label is added and on every push while it is set. Each pull request has its own concurrency group, and a newer push cancels the older review. A behaviour change without `merge-ok` gets a red status, `waits for merge-ok to be reviewed`. Other labels don't start or cancel anything. Fork code never reaches the pools or the key.
 - **What:** the checkout is the event's default `refs/pull/N/merge`, the pull request merged onto `main` at review time, so a review is of what would land, without pushing to the branch. Both sides are packed in the same run, on a GitHub-hosted runner, with no secrets: the candidate (the merge ref, `x.y.z-pr.N.<run>`) and the base (its first parent, `main`). Then:

package/dist/kernel-worker.js

@@ -107,7 +107,7 @@ var MemoryMeta = class {
     if (entry) this.entries.set(path, entry);
     else this.entries.delete(path);
   }
-  async move(from, to, root) {
+  async move(from, to, root, left) {
     await this.remove([to]);
     const carried = this.entries.subtree(from).map((path) => this.entries.get(path));
     for (const entry of carried) this.entries.delete(entry.path);
@@ -116,6 +116,7 @@ var MemoryMeta = class {
       if (path !== to) this.entries.set(path, moved(entry, path));
     }
     this.entries.set(to, moved(root(carried.find((entry) => entry.path === from)), to));
+    if (left) this.entries.set(from, left);
   }
   async remove(paths) {
     for (const path of paths) this.entries.deleteTree(path);
@@ -240,7 +241,7 @@ var IndexedDbMeta = class {
     this.channel.close();
     this.db.close();
   }
-  async move(from, to, root) {
+  async move(from, to, root, left) {
     this.settleRemoves();
     const [transaction, store] = this.transaction("readwrite");
     store.delete(to);
@@ -254,6 +255,7 @@ var IndexedDbMeta = class {
         store.put(moved(entry, to + entry.path.slice(from.length)));
       }
       store.put(moved(root(source.result), to));
+      if (left) store.put(left);
     };
     await settle(transaction, () => void 0);
     this.changed({ trees: [from, to] });
@@ -1104,6 +1106,14 @@ function sidecar(meta) {
     });
   return ops;
 }
+function occupied(entry) {
+  if (!entry?.vacated) return entry ?? {};
+  const { vacated: _, ...kept } = entry;
+  return kept;
+}
+function vacatedAt(path) {
+  return { path, ino: inodeOf(`${path}\0${crypto.randomUUID()}`), vacated: true };
+}
 function missing2(err) {
   const code = err?.code;
   return code === "ENOENT" || code === "ENOTDIR";
@@ -1338,7 +1348,7 @@ var OpfsFs = class {
     const kind = directory ? void 0 : owner?.socket ? "socket" : owner?.fifo ? "fifo" : void 0;
     const unique = kind === "fifo" ? { ino: inodeOf(`${path}\0${crypto.randomUUID()}`) } : {};
     const fields = kind ? { ...owned4, ...unique, [kind]: true } : owned4;
-    if (fields) await this.meta.update(path, (entry) => ({ ...entry, path, ...fields }));
+    if (fields) await this.meta.update(path, (entry) => ({ ...occupied(entry), path, ...fields }));
   }
   guarded = [];
   protect(prefixes2, versions = false) {
@@ -2050,8 +2060,9 @@ var OpfsFs = class {
     await this.locked(physical, async () => {
       if (await this.handleExists(dir, name2)) throw fsError("EEXIST", path);
       const freshMeta = this.meta.fresh ? await this.meta.fresh([path]) : await this.meta.get([path]);
-      if (freshMeta[0]?.link !== void 0) throw fsError("EEXIST", path);
-      if (freshMeta[0] !== void 0) await this.meta.remove([path]);
+      const found = freshMeta[0];
+      if (found?.link !== void 0) throw fsError("EEXIST", path);
+      if (found !== void 0 && !found.vacated) await this.meta.remove([path]);
       this.forget(physical);
       this.refuse(physical);
       await dir.getDirectoryHandle(name2, { create: true });
@@ -2071,8 +2082,9 @@ var OpfsFs = class {
     await this.locked(physical, async () => {
       if (await this.handleExists(dir, name2)) throw fsError("EEXIST", path);
       const freshMeta = this.meta.fresh ? await this.meta.fresh([path]) : await this.meta.get([path]);
-      if (freshMeta[0]?.link !== void 0) throw fsError("EEXIST", path);
-      if (freshMeta[0] !== void 0) await this.meta.remove([path]);
+      const found = freshMeta[0];
+      if (found?.link !== void 0) throw fsError("EEXIST", path);
+      if (found !== void 0 && !found.vacated) await this.meta.remove([path]);
       this.forget(physical);
       this.refuse(physical);
       await dir.getFileHandle(name2, { create: true });
@@ -2176,14 +2188,15 @@ var OpfsFs = class {
     const moving = await this.movedOwner(source);
     const carry = (entry) => ({
       ino: inodeOf(source.path),
-      ...entry,
+      ...occupied(entry),
       ...moving,
       path: destination,
       ctimeMs: now
     });
+    const left = vacatedAt(source.path);
     if (source.link) {
       if (target?.handle) await this.rm(destination);
-      await this.meta.move(source.path, destination, carry);
+      await this.meta.move(source.path, destination, carry, left);
       this.touch(Date.now(), parentOf3(source.path), parent.path);
       return;
     }
@@ -2198,9 +2211,12 @@ var OpfsFs = class {
       () => this.place(source.handle, dir, staging)
     );
     if (swap) await this.rm(destination, { recursive: true });
-    await this.meta.move(source.path, destination, carry);
+    await this.meta.move(source.path, destination, carry, left);
     this.movePending(source.path, destination);
-    if (!moved2) await this.rm(source.path, { recursive: true });
+    if (!moved2) {
+      await this.rm(source.path, { recursive: true });
+      await this.meta.update(source.path, () => left);
+    }
     let copied = !moved2;
     if (swap) {
       const staged = await dir.getDirectoryHandle(staging);
@@ -2285,11 +2301,11 @@ var OpfsFs = class {
     const fields = await this.newFields(at, owner && { uid: owner.uid, gid: owner.gid }, false);
     await this.entering(
       at,
-      () => this.meta.update(at, () => ({
+      () => this.meta.update(at, (entry) => ({
         path: at,
         link: target,
         dir: parent.path,
-        ino: inodeOf(at),
+        ino: entry?.vacated && entry.ino !== void 0 ? entry.ino : inodeOf(at),
         mtimeMs: now,
         ctimeMs: now,
         ...fields
@@ -2356,12 +2372,20 @@ var OpfsFs = class {
     }));
   }
   async reconcile() {
+    if (this.remote && await this.remote.peered()) return 0;
     let removed = 0;
-    for (const entry of await this.meta.all()) {
+    const entries = await this.meta.all();
+    const carried = new Set(entries.filter((e) => !e.vacated).map((e) => e.ino));
+    for (const entry of entries) {
+      if (entry.vacated && carried.has(inodeOf(entry.path))) continue;
       const real = await this.real(entry.path);
       const stale = entry.link === void 0 ? !real : real || !await this.real(parentOf3(entry.path));
       if (!stale) continue;
-      await this.meta.update(entry.path, () => void 0);
+      const seen = JSON.stringify(entry);
+      await this.meta.update(
+        entry.path,
+        (now) => now === void 0 || JSON.stringify(now) === seen ? void 0 : now
+      );
       removed++;
     }
     return removed;
@@ -6408,7 +6432,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.123.0" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.123.2" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC