@ai-ecoverse/slicc-kernel 1.147.9 → 1.147.10

package/README.md

@@ -826,7 +826,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
   - **The same file:** a lock is keyed by the file's device and inode, which both kernels read from the shared sidecar, so a rename by either tab keeps it. A file removed and created again at the same path has the same key, as it already has within one kernel.
   - **Versions:** the channel and Web Lock names carry a protocol version (`v1`), and a kernel ignores messages that aren't v1. So kernels on different protocol versions (two tabs across an update) don't share locks, as before #675.
   - **Not across tabs:** the imports module's whole-file `tryLock` (per kernel), and locks on other mounts.
-- **Another tab's writes** ([#959](https://github.com/ai-ecoverse/slicc-kernel/issues/959)): a program's names and stats are cached in its process, and its kernel refreshes them when the kernel itself changes a file. While another kernel shares the root (a second tab), the kernel marks that view foreign. A WASI program then keeps a stat or a missing name it looked up by path for at most 100 ms after the kernel answered it, however often it looks in between ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). Every notice from another kernel, a lock grant, and a kernel joining or leaving the root (also one that ends without a word) drop everything its programs cached. An answer that a notice overtook on its way is not kept. An open never uses what the program cached (close-to-open), and calls on a descriptor (`fstat`, `pread`) go to the kernel, which holds them to its own 2 ms. So a program that keeps checking a file sees what the other tab wrote, synced and closed, once the notice has arrived, and within 100 ms if the notice is late or lost; one that opens the file sees it at once. A kernel alone caches as before. An Emscripten program keeps the names it has looked up, directories and files ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)), until its next call that isn't by path: a read, a write, a poll, a sleep, a wait, a spawn, a look at the clock or a draw of random bytes. Within one run of path-only calls (lstat, chmod, readlink and the like) with none of those in between, an answer the program takes from its own node, such as the node's type, can lag behind the other tab until that next call, as a path call racing a rename does on Linux. A program can only learn that the other tab acted through one of those calls, a new process or a stat, and all of them refresh what it sees. While another kernel shares the root, a kernel also tells it of each file it writes ([#962](https://github.com/ai-ecoverse/slicc-kernel/issues/962)). The other kernel then drops the clean contents of that file, both its own and its programs', so a descriptor tab B already has open reads tab A's new bytes on its next read, as Linux's shared page cache does. The notice is a message, so a read that races it by less than a message's delivery can still see the old bytes, as with any reader that doesn't lock. The whole cross-tab contract is under "What another tab sees, and when".
+- **Another tab's writes** ([#959](https://github.com/ai-ecoverse/slicc-kernel/issues/959)): a program's names and stats are cached in its process, and its kernel refreshes them when the kernel itself changes a file. While another kernel shares the root (a second tab), the kernel marks that view foreign. A WASI program then keeps a stat or a missing name it looked up by path for at most 100 ms after the kernel answered it, however often it looks in between ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). Every notice from another kernel, a lock grant, and a kernel joining or leaving the root (also one that ends without a word) drop everything its programs cached. An answer that a notice overtook on its way is not kept. An open never uses what the program cached (close-to-open), and calls on a descriptor (`fstat`, `pread`) go to the kernel, which holds them to its own window (100 ms at most). So a program that keeps checking a file sees what the other tab wrote, synced and closed, once the notice has arrived, and within 100 ms if the notice is late or lost; one that opens the file sees it at once. A kernel alone caches as before. An Emscripten program keeps the names it has looked up, directories and files ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)), until its next call that isn't by path: a read, a write, a poll, a sleep, a wait, a spawn, a look at the clock or a draw of random bytes. Within one run of path-only calls (lstat, chmod, readlink and the like) with none of those in between, an answer the program takes from its own node, such as the node's type, can lag behind the other tab until that next call, as a path call racing a rename does on Linux. A program can only learn that the other tab acted through one of those calls, a new process or a stat, and all of them refresh what it sees. While another kernel shares the root, a kernel also tells it of each file it writes ([#962](https://github.com/ai-ecoverse/slicc-kernel/issues/962)). The other kernel then drops the clean contents of that file, both its own and its programs', so a descriptor tab B already has open reads tab A's new bytes on its next read, as Linux's shared page cache does. The notice is a message, so a read that races it by less than a message's delivery can still see the old bytes, as with any reader that doesn't lock. The whole cross-tab contract is under "What another tab sees, and when".
 - **`slicc.fcntl_lock(fd, cmd, flock_ptr) -> errno`** ([#722](https://github.com/ai-ecoverse/slicc-kernel/issues/722)): a WASIX program's `fcntl` record locks reach the same table as an Emscripten program's, so the two see each other's locks.
   - **cmd:** `12` `F_GETLK`, `13` `F_SETLK` or `14` `F_SETLKW`. Any other cmd is `EINVAL`.
   - **The struct:** musl's wasm32 `struct flock`, 32 bytes: `l_type` i16 at 0 (`0` read, `1` write, `2` unlock), `l_whence` i16 at 2 (`0` SET, `1` CUR, `2` END, resolved against the descriptor's offset or size), `l_start` i64 at 8, `l_len` i64 at 16 (`0` to EOF and beyond, negative before `l_start`), `l_pid` i32 at 24. The kernel reads and writes only those five fields and never the padding.
@@ -1194,17 +1194,17 @@ Reads retry briefly when a concurrent write has invalidated the OPFS file snapsh
 
 At boot, a kernel drops sidecar records whose file is gone (and symlink records whose name became a real file or whose directory is gone), left behind by a tab that closed mid-change. It does this only when it is the only kernel on the origin (the watch channel's join handshake finds no other), and it drops a record only if it is still the one it found: another tab's live changes made while it looked, such as a placeholder tar replaces with a symlink, are kept. Up to 1.119.2, a tab that booted while another extracted symlinks with tar could drop some of them.
 
-A kernel caches the directory handles it has walked and the sidecar entries it has read. Its own changes keep both caches current, and the other kernels on the origin hear of them through a `BroadcastChannel` named after the database (one for the sidecar, one for directories). A cached directory is checked against its path (`resolve()`) at most once a second before a lookup goes through it, so a directory moved, removed or replaced by another writer, such as a page using the OPFS API, is found at its new place or reported missing within a second. It also keeps the file handles it has looked up (up to 4,096, the least recently used dropped first) for a second after each lookup. Before it uses one, it reads the file through it, which is the read a `stat` needs anyway, and that read serves the handle's uses of the next 2 ms. So a file another writer removed is missing, and a rewritten one shows its new contents, within 2 ms. The kernel's own writes, truncates, rewrites and renames drop the read at once, and a read that started before an own change isn't kept, so the kernel's own changes always show at once. One replaced by a directory is seen as a directory within a second. Its own changes, and the other kernels' notices, drop the handles they touch at once. A lookup of a directory it has cached and checked within the second is answered from the cache, so a directory replaced by another writer is seen as replaced within a second, as above. A name it has just found missing stays missing for 2 ms, which serves the several checks a single create makes of a name that isn't there yet (about ten per file in a pnpm install); its own creates, renames and links in that directory end that at once, so only a file another kernel or a page using the OPFS API makes in those 2 ms is seen up to 2 ms late. While another kernel shares the root, both 2 ms windows follow the contract below. From [#945](https://github.com/ai-ecoverse/slicc-kernel/issues/945) until [#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984) they were off then, which made every stat and every miss an OPFS call.
+A kernel caches the directory handles it has walked and the sidecar entries it has read. Its own changes keep both caches current, and the other kernels on the origin hear of them through a `BroadcastChannel` named after the database (one for the sidecar, one for directories). A cached directory is checked against its path (`resolve()`) at most once a second before a lookup goes through it, so a directory moved, removed or replaced by another writer, such as a page using the OPFS API, is found at its new place or reported missing within a second. It also keeps the file handles it has looked up (up to 4,096, the least recently used dropped first) for a second after each lookup. Before it uses one, it reads the file through it, which is the read a `stat` needs anyway, and that read serves the handle's uses in the kernel's next 64 lookups, for at most 100 ms ([#1425](https://github.com/ai-ecoverse/slicc-kernel/issues/1425)), so the count doesn't depend on how fast the machine is. So a file another writer removed is missing, and a rewritten one shows its new contents, within 100 ms. The kernel's own writes, truncates, rewrites and renames drop the read at once, and a read that started before an own change isn't kept, so the kernel's own changes always show at once. One replaced by a directory is seen as a directory within a second. Its own changes, and the other kernels' notices, drop the handles they touch at once. A lookup of a directory it has cached and checked within the second is answered from the cache, so a directory replaced by another writer is seen as replaced within a second, as above. A name it has just found missing stays missing for the kernel's next 256 lookups, for at most 100 ms, which serves the several checks a single create makes of a name that isn't there yet (about ten per file in a pnpm install, up to a few hundred lookups apart); its own creates, renames and links in that directory end that at once, so only a file another kernel or a page using the OPFS API makes in that window is seen up to 100 ms late. While another kernel shares the root, both short-lived windows follow the contract below. From [#945](https://github.com/ai-ecoverse/slicc-kernel/issues/945) until [#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984) they were off then, which made every stat and every miss an OPFS call.
 
 **What another tab sees, and when** ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)): one rule covers everything a kernel caches about a shared root.
 - **Notices:** while another kernel shares the root, a kernel posts a notice before each changing call returns. A notice names the sidecar records it changed (owners, modes, links, times, write counts), the directories it created, removed or renamed into, and the files it wrote back ([#962](https://github.com/ai-ecoverse/slicc-kernel/issues/962)).
-- **What a notice drops:** the receiving kernel drops at once what the notice names, and everything under it: sidecar entries, directory and file handles, the 2 ms file reads, the 2 ms missing names, and its programs' clean contents.
-- **Unlocked, unsynchronized reads:** a read like this sees another tab's change once that change's notice has arrived. A read that races the notice can get the old answer, by at most a message's delivery. Even with the notice lost, a stat or lookup is at most 2 ms of this kernel's clock behind its own last look at that name.
+- **What a notice drops:** the receiving kernel drops at once what the notice names, and everything under it: sidecar entries, directory and file handles, the short-lived file reads and missing names, and its programs' clean contents.
+- **Unlocked, unsynchronized reads:** a read like this sees another tab's change once that change's notice has arrived. A read that races the notice can get the old answer, by at most a message's delivery. Even with the notice lost, a stat or lookup is at most 100 ms of this kernel's clock behind its own last look at that name.
 - **Exact, whatever the notices:**
   - a lock grant (`flock`, `fcntl`, `lockf`, tryLock) drops every cached file read and missing name, so the holder's first look at each name goes to OPFS ([#949](https://github.com/ai-ecoverse/slicc-kernel/issues/949), [#958](https://github.com/ai-ecoverse/slicc-kernel/issues/958));
   - `fsync`, close and unlock write through and post their notice before they return ([#961](https://github.com/ai-ecoverse/slicc-kernel/issues/961));
   - an exclusive create (`O_EXCL`, `mkdir`, `symlink`, `link`) and a rename's check of its target decide against OPFS itself, never against a cached missing name ([#942](https://github.com/ai-ecoverse/slicc-kernel/issues/942));
-  - the join handshake writes back every open file before a new kernel counts as a peer, and a change in whether the root is shared drops both 2 ms windows.
+  - the join handshake writes back every open file before a new kernel counts as a peer, and a change in whether the root is shared drops both short-lived windows.
 
 So a locked read-modify-write across tabs (sqlite, a JSON counter under `flock`) is exact, and anything else between tabs is as fresh as the last notice. The same holds for what a WASI program caches in its own process: a notice, a lock grant or a peer joining or leaving drops all of it, and a late or lost notice leaves an unlocked path stat at most 100 ms behind the kernel's answer; opens, descriptor calls and anything under a lock are exact ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). Emscripten programs add their own rule on top: see "Another tab's writes" above. `program-cache-tabs.test.mjs` holds the rows for programs; it freezes a program's clock through `page/notice/program-clock-worker.js`. `crowded-lookups.test.mjs` and `notice-caches.test.mjs` hold the rows. The `notice-caches` rows freeze one kernel's clock and drop the other's notices, through the kernel workers in `test/integration/page/notice/`.
 

package/dist/kernel-worker.js

@@ -1277,8 +1277,9 @@ async function swapped(op) {
 }
 var MAX_LINKS2 = 40;
 var CHECK_MS = 1e3;
-var ABSENT_MS = 2;
-var GLANCE_MS = 2;
+var ABSENT_LOOKUPS = 256;
+var GLANCE_LOOKUPS = 64;
+var GLANCE_CAP_MS = 100;
 var ABSENT_NAMES = 4096;
 var FILE_HANDLES = 4096;
 var STALE = /* @__PURE__ */ new Set(["NotFoundError", "TypeMismatchError"]);
@@ -1493,6 +1494,7 @@ var OpfsFs = class {
   dirs = new PathMap();
   files = new PathMap();
   absent = new PathMap();
+  lookups = 0;
   changes = 0;
   replacing = Promise.resolve();
   replacements = 0;
@@ -1851,7 +1853,7 @@ var OpfsFs = class {
   }
   missed(path) {
     if (this.absent.size >= ABSENT_NAMES) this.absent.clear();
-    this.absent.set(path, performance.now());
+    this.absent.set(path, { at: performance.now(), op: this.lookups });
   }
   remember(path, handle, looked = false) {
     if (handle.kind !== "file") {
@@ -1892,7 +1894,11 @@ var OpfsFs = class {
     if (name2 === void 0) throw fsError("EBUSY", path);
     return [`/${parts2.join("/")}`, name2];
   }
+  recent(seen, lookups) {
+    return this.lookups - seen.op < lookups && performance.now() - seen.at < GLANCE_CAP_MS;
+  }
   async handle(path) {
+    this.lookups++;
     if (split(path).length === 0) return this.root;
     const cached = this.files.get(path);
     const fresh2 = cached && performance.now() - (this.found.get(cached) ?? -CHECK_MS) < CHECK_MS;
@@ -1912,7 +1918,7 @@ var OpfsFs = class {
     const kept = this.dirs.get(path);
     if (kept && await this.current(path, kept)) return kept;
     const gone = this.absent.get(path);
-    if (gone !== void 0 && this.exact === 0 && performance.now() - gone < ABSENT_MS) {
+    if (gone !== void 0 && this.exact === 0 && this.recent(gone, ABSENT_LOOKUPS)) {
       throw translate({ name: "NotFoundError" }, path);
     }
     const seen = this.changes;
@@ -2115,15 +2121,16 @@ var OpfsFs = class {
   glanced(handle) {
     if (this.exact > 0) return void 0;
     const seen = this.glances.get(handle);
-    const fresh2 = seen && seen.changes === this.changes && performance.now() - seen.at < GLANCE_MS;
+    const fresh2 = seen && seen.changes === this.changes && this.recent(seen, GLANCE_LOOKUPS);
     return fresh2 ? seen.file : void 0;
   }
   async glance(handle) {
     const at = performance.now();
+    const op = this.lookups;
     const before = this.changes;
     this.calls++;
     const file = await handle.getFile();
-    if (before === this.changes) this.glances.set(handle, { at, file, changes: before });
+    if (before === this.changes) this.glances.set(handle, { at, op, file, changes: before });
     else this.glances.delete(handle);
     return file;
   }
@@ -7819,7 +7826,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.147.9" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.147.10" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC