@ai-ecoverse/slicc-kernel 1.117.3 → 1.117.4

package/README.md

@@ -356,7 +356,7 @@ The module's default export (or its `main` export) is called with a context and
 - `argv` (with `argv[0]`), `env`, `pid` (the pid `ps` shows, also after an `exec`), `ppid()` (asked each time, so an orphan sees 1) and `cwd`, and `resolve(path)` to make a path absolute against `cwd`;
 - `stdin`, `stdout` and `stderr` as web streams, and `read(fd, max?)` (empty at end of file), `write(fd, bytes | string)` and `close(fd)` on any descriptor it inherited. A write waits until the pipe has room, so a slow reader slows the program down;
 - `fdStatus(fd)` (`{ type, tty, seekable }`, `type` being `file`, `pipe`, `tty`, `socket`, `device` or `directory`) and `isatty(fd)`, to tell `< file` from a pipe;
-- `open(path, { read, write, append, create, exclusive, truncate })`, which returns a file with `read(position, length)`, `write(position, bytes)`, `size()`, `truncate(size)`, `sync()` and `close()`. Reads and writes go through the kernel's paged files at those positions, so a large file is never read whole;
+- `open(path, { read, write, append, create, exclusive, truncate, nonblock, signal })`, which returns a file with `read(position, length)`, `write(position, bytes)`, `size()`, `truncate(size)`, `sync()` and `close()`. Reads and writes go through the kernel's paged files at those positions, so a large file is never read whole; **A FIFO** ([#935](https://github.com/ai-ecoverse/slicc-kernel/issues/935)) opens through the kernel's FIFO open, as fifo(7) says. A read-open waits for a writer and a write-open for a reader, unless the open is `read` and `write` both. `nonblock: true` gives a read-open at once and a write-open with no reader `ENXIO`, and `signal` aborts a waiting open with its reason. Its file reads and writes the pipe: `read(0, length)` and `write(0, bytes)`, any other position is `ESPIPE`, `size()` is `0`, and `truncate` is `EINVAL`. A socket node is `ENXIO`.
 - `fs`: `stat`, `lstat`, `exists`, `readdir`, `mkdir` (with parents, or, with `{ recursive: false }`, as Node's `fs.mkdir` without `recursive`: `EEXIST` for an existing entry and `ENOENT` for a missing parent, atomically among the processes of one kernel, so it can serve as a lock there, but not between two kernels on one origin, such as two tabs ([#519](https://github.com/ai-ecoverse/slicc-kernel/issues/519)); `sync.fs.mkdir` takes the same option), `rm` (recursive), `unlink`, `rename`, `symlink`, `readlink`, `statfs`, `chmod(path, mode)`, and `readFile`/`writeFile` for small files. `chmod` takes the permission bits (`0` to `0o7777`, `EINVAL` otherwise) and follows a symlink; as `chmod(2)`, only the file's owner or root may (`EPERM`), so a program can make a script it wrote runnable by path. `sync.fs.chmod` is the same, synchronously. `statfs(path)` answers `{ type, bsize, blocks, bfree, bavail, files, ffree, quota, usage }`: the fields of Node's `StatFs`, plus `quota` and `usage` in bytes (what jsh 0.17 reads). The values come from the same source as an Emscripten program's `statfs`/`df`: the driver of the mount the path is on, or, on the root file system, the origin's storage estimate (`navigator.storage.estimate()`). `bsize` is 4096, `bfree` and `bavail` are what's left of the quota, and `type`, `files` and `ffree` are 0, since the kernel doesn't know them. Where there's no estimate (the Node entry's root), every count is 0. A missing path is `ENOENT`. `sync.fs` has it too. `rm` removes a whole tree in one kernel call (on the OPFS root, one recursive removal and one metadata delete: 50,000 files take about 3 s in Chrome, where `rm -r` takes about 25 s) with the result of a file-by-file removal: symlinks inside the tree are removed, never followed; a tree holding a protected path is refused with `EROFS` and nothing is removed; a mount inside the tree is emptied and its mount point answers `EBUSY`, while the rest is removed; under `users`, what the user may remove is removed and the call then fails with the first `EACCES`/`EPERM`; files a process holds open stay readable to it;
 - `/dev`, in `open`, `fs` and their `sync` forms, as a WASI or Emscripten program sees it, with the same modes and owners as under `users`: `null`, `zero`, `full`, `random`, `urandom` and `tty` (character devices, `0666`, root's), `stdin`, `stdout` and `stderr` (links to `/proc/self/fd/0`, `1`, `2`), `fd` (a link to `/proc/self/fd`, so `/dev/fd/N` is descriptor N) and the kernel's terminals `/dev/ttyN` and `/dev/pts/N` (with users on, owned by the session's user or the master's opener, `0620`). `/dev/shm` is the kernel's shared tmpfs (see [Mounts](#mounts)), read and written like any other mount; `/dev/ptmx` isn't there. `stat` through `/dev/stdin`, `/dev/stdout` or `/dev/fd/N` describes the program's own descriptor (the file it is redirected to, a pipe, a terminal), `readlink` of `/dev/fd/N` names it (`pipe:[N]` and `socket:[N]` for a pipe or a socket), and a closed descriptor is `ENOENT`. Opening a device gives a kernel descriptor on it; opening `/dev/std*` or `/dev/fd/N` opens the file again with the flags asked for when the descriptor is on a file (as Linux reopens `/proc/self/fd/N`, so `append` appends and `truncate` truncates), and otherwise duplicates the descriptor; `/dev/tty` is the program's controlling terminal (`ENXIO` without one); and `/dev/ttyN` and `/dev/pts/N` are opened by name through the kernel, so a terminal that isn't the user's is `EACCES` under `enforce`. A file on anything but a regular file reads and writes as a stream (the position is ignored) and only in the directions it was opened for (`EBADF` otherwise), its `size()` is 0 and `truncate` is `EINVAL`. `readFile` reads a device or a descriptor to its end, up to 64 MiB, and fails with `EFBIG` past that (where Node's `readFile` throws `ERR_FS_FILE_TOO_LARGE`), so `readFile('/dev/zero')` fails instead of hanging, while a bounded `read` of it works. Other paths pay only a prefix check: they make the same kernel calls as before. Removing a `/dev` entry fails (see [`/dev`](#dev-entries)), and so does a new name directly in `/dev` or `/proc`;
 - `/proc`, in `fs`, `open` and their `sync` forms, the same as a WASI program's (see [`/proc`](#proc)): `stat`, `lstat`, `exists`, `readdir`, `readlink` and `readFile` answer it, and `/proc/self` is the program's own pid. It's read-only, so every change in it is `EROFS`: `writeFile`, `mkdir`, `rm`, `unlink`, `chmod`, a `rename`, `link` or `symlink` into or out of it, and an `open` for writing, creating or truncating. A file opened in it is a snapshot read at the open, with no descriptor (`fd` is `-1`): its `read` and `size` answer from the snapshot until it's closed, and `write` and `truncate` are `EBADF`. Other paths pay only a prefix check;
@@ -923,7 +923,7 @@ Not covered: hard links (OPFS cannot share contents between names; they work onl
 
 Reads retry briefly when a concurrent write has invalidated the OPFS file snapshot (`NotReadableError`).
 
-A kernel caches the directory handles it has walked and the sidecar entries it has read. Its own changes keep both caches current, and the other kernels on the origin hear of them through a `BroadcastChannel` named after the database (one for the sidecar, one for directories). A cached directory is checked against its path (`resolve()`) at most once a second before a lookup goes through it, so a directory moved, removed or replaced by another writer, such as a page using the OPFS API, is found at its new place or reported missing within a second. It also keeps the file handles it has looked up (up to 4,096, the least recently used dropped first) for a second after each lookup. Before it uses one, it reads the file through it, which is the read a `stat` needs anyway, so a file another writer removed is missing at once and a rewritten one shows its new contents. One replaced by a directory is seen as a directory within a second. Its own changes, and the other kernels' notices, drop the handles they touch at once. A lookup of a directory it has cached and checked within the second is answered from the cache, so a directory replaced by another writer is seen as replaced within a second, as above. A name it has just found missing stays missing for 2 ms, which serves the several checks a single create makes of a name that isn't there yet (about ten per file in a pnpm install); its own creates, renames and links in that directory end that at once, so only a file another kernel or a page using the OPFS API makes in those 2 ms is seen up to 2 ms late.
+A kernel caches the directory handles it has walked and the sidecar entries it has read. Its own changes keep both caches current, and the other kernels on the origin hear of them through a `BroadcastChannel` named after the database (one for the sidecar, one for directories). A cached directory is checked against its path (`resolve()`) at most once a second before a lookup goes through it, so a directory moved, removed or replaced by another writer, such as a page using the OPFS API, is found at its new place or reported missing within a second. It also keeps the file handles it has looked up (up to 4,096, the least recently used dropped first) for a second after each lookup. Before it uses one, it reads the file through it, which is the read a `stat` needs anyway, and that read serves the handle's uses of the next 2 ms. So a file another writer removed is missing, and a rewritten one shows its new contents, within 2 ms. The kernel's own writes, truncates, rewrites and renames drop the read at once, and a read that started before an own change isn't kept, so the kernel's own changes always show at once. One replaced by a directory is seen as a directory within a second. Its own changes, and the other kernels' notices, drop the handles they touch at once. A lookup of a directory it has cached and checked within the second is answered from the cache, so a directory replaced by another writer is seen as replaced within a second, as above. A name it has just found missing stays missing for 2 ms, which serves the several checks a single create makes of a name that isn't there yet (about ten per file in a pnpm install); its own creates, renames and links in that directory end that at once, so only a file another kernel or a page using the OPFS API makes in those 2 ms is seen up to 2 ms late.
 
 The filesystem is our own rather than [ZenFS](https://github.com/zen-fs/core) (which SLICC uses), because OPFS stays the single source of truth: other writers, such as the BIOS installing packages or a page writing files, need no index to stay consistent with, and nothing is preloaded into memory at mount. POSIX metadata lives in the IndexedDB sidecar described above, which has per-entry transactions instead of one JSON file rewritten on every change.
 

package/dist/kernel-worker.js

@@ -995,6 +995,7 @@ var READ_ATTEMPTS = 3;
 var MAX_LINKS2 = 40;
 var CHECK_MS = 1e3;
 var ABSENT_MS = 2;
+var GLANCE_MS = 2;
 var ABSENT_NAMES = 4096;
 var FILE_HANDLES = 4096;
 var STALE = /* @__PURE__ */ new Set(["NotFoundError", "TypeMismatchError"]);
@@ -1135,7 +1136,7 @@ var OpfsFs = class {
   changes = 0;
   replacing = Promise.resolve();
   replacements = 0;
-  snapshots = /* @__PURE__ */ new WeakMap();
+  glances = /* @__PURE__ */ new WeakMap();
   found = /* @__PURE__ */ new WeakMap();
   checked = /* @__PURE__ */ new WeakMap();
   channel;
@@ -1389,7 +1390,7 @@ var OpfsFs = class {
     if (handle.kind !== "file") return handle;
     if (looked) {
       this.found.set(handle, performance.now());
-      this.snapshots.delete(handle);
+      this.glances.delete(handle);
     }
     this.files.delete(path);
     if (this.files.size >= FILE_HANDLES)
@@ -1423,14 +1424,15 @@ var OpfsFs = class {
     const fresh2 = cached && performance.now() - (this.found.get(cached) ?? -CHECK_MS) < CHECK_MS;
     if (cached && !fresh2) this.files.delete(path);
     if (cached && fresh2) {
+      if (this.glanced(cached)) return this.remember(path, cached);
       try {
-        this.snapshots.set(cached, await cached.getFile());
+        await this.glance(cached);
         return this.remember(path, cached);
       } catch (err) {
         if (!STALE.has(err?.name))
           throw translate(err, path, "read");
         this.files.delete(path);
-        this.snapshots.delete(cached);
+        this.glances.delete(cached);
       }
     }
     const kept = this.dirs.get(path);
@@ -1623,11 +1625,25 @@ var OpfsFs = class {
         if (this.dirTimes.get(path) === ms) this.dirTimes.delete(path);
     }
   }
+  glanced(handle) {
+    const seen = this.glances.get(handle);
+    const fresh2 = seen && seen.changes === this.changes && performance.now() - seen.at < GLANCE_MS;
+    return fresh2 ? seen.file : void 0;
+  }
+  async glance(handle) {
+    const at = performance.now();
+    const before = this.changes;
+    const file = await handle.getFile();
+    if (before === this.changes) this.glances.set(handle, { at, file, changes: before });
+    else this.glances.delete(handle);
+    return file;
+  }
+  wrote(handle) {
+    this.changes++;
+    this.glances.delete(handle);
+  }
   async snapshot(handle) {
-    const taken = this.snapshots.get(handle);
-    if (!taken) return handle.getFile();
-    this.snapshots.delete(handle);
-    return taken;
+    return this.glanced(handle) ?? this.glance(handle);
   }
   async statOf(path, handle, meta, touched) {
     const directory = handle.kind === "directory";
@@ -1762,9 +1778,11 @@ var OpfsFs = class {
   async pwrite(path, offset, bytes) {
     const [handle, , target] = await this.fileFor(path);
     await this.counting(target);
+    this.wrote(handle);
     try {
       await this.retrying(path, () => positioned(handle, (io) => io.write(offset, bytes)), true);
     } finally {
+      this.wrote(handle);
       await this.recounting(handle, target);
     }
     this.written(target);
@@ -1775,9 +1793,11 @@ var OpfsFs = class {
     const resolve2 = this.ranged || this.remote?.watched();
     const target = resolve2 || this.counted.length > 0 ? (await this.locate(path, true)).path : path;
     await this.counting(target);
+    this.wrote(handle);
     try {
       await this.retrying(path, () => positioned(handle, (io) => io.truncate(size)), true);
     } finally {
+      this.wrote(handle);
       await this.recounting(handle, target);
     }
     this.written(target);
@@ -1797,9 +1817,12 @@ var OpfsFs = class {
         throw translate(err, path, "write", false);
       }),
       close: async () => {
+        this.wrote(handle);
         await writable2.close().catch((err) => {
+          this.wrote(handle);
           throw translate(err, path, "commit", false);
         });
+        this.wrote(handle);
         this.committed(target);
       }
     };
@@ -1846,6 +1869,7 @@ var OpfsFs = class {
     if (!existed) await this.stampNew(target, owner, false);
     if (!existed && content.length === 0) return;
     let writable2;
+    this.wrote(file);
     try {
       writable2 = await file.createWritable();
       await writable2.write(content);
@@ -1853,6 +1877,8 @@ var OpfsFs = class {
     } catch (err) {
       await writable2?.abort().catch(() => void 0);
       throw translate(err, path, "commit", false);
+    } finally {
+      this.wrote(file);
     }
     this.committed(target);
   }
@@ -2106,7 +2132,7 @@ var OpfsFs = class {
   }
   async modified(path) {
     const handle = await this.handle(path);
-    return handle.kind === "file" ? (await handle.getFile()).lastModified : 0;
+    return handle.kind === "file" ? (await this.snapshot(handle)).lastModified : 0;
   }
   async explicitTimes(path) {
     const timed2 = [];
@@ -6285,7 +6311,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.117.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.117.4" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC