@ai-ecoverse/slicc-kernel 1.99.0 → 1.99.2

package/README.md

@@ -346,7 +346,7 @@ The module's default export (or its `main` export) is called with a context and
 - `fdStatus(fd)` (`{ type, tty, seekable }`, `type` being `file`, `pipe`, `tty`, `socket`, `device` or `directory`) and `isatty(fd)`, to tell `< file` from a pipe;
 - `open(path, { read, write, append, create, exclusive, truncate })`, which returns a file with `read(position, length)`, `write(position, bytes)`, `size()`, `truncate(size)`, `sync()` and `close()`. Reads and writes go through the kernel's paged files at those positions, so a large file is never read whole;
 - `fs`: `stat`, `lstat`, `exists`, `readdir`, `mkdir` (with parents, or, with `{ recursive: false }`, as Node's `fs.mkdir` without `recursive`: `EEXIST` for an existing entry and `ENOENT` for a missing parent, atomically among the processes of one kernel, so it can serve as a lock there, but not between two kernels on one origin, such as two tabs ([#519](https://github.com/ai-ecoverse/slicc-kernel/issues/519)); `sync.fs.mkdir` takes the same option), `rm` (recursive), `unlink`, `rename`, `symlink`, `readlink`, `statfs`, `chmod(path, mode)`, and `readFile`/`writeFile` for small files. `chmod` takes the permission bits (`0` to `0o7777`, `EINVAL` otherwise) and follows a symlink; as `chmod(2)`, only the file's owner or root may (`EPERM`), so a program can make a script it wrote runnable by path. `sync.fs.chmod` is the same, synchronously. `statfs(path)` answers `{ type, bsize, blocks, bfree, bavail, files, ffree, quota, usage }`: the fields of Node's `StatFs`, plus `quota` and `usage` in bytes (what jsh 0.17 reads). The values come from the same source as an Emscripten program's `statfs`/`df`: the driver of the mount the path is on, or, on the root file system, the origin's storage estimate (`navigator.storage.estimate()`). `bsize` is 4096, `bfree` and `bavail` are what's left of the quota, and `type`, `files` and `ffree` are 0, since the kernel doesn't know them. Where there's no estimate (the Node entry's root), every count is 0. A missing path is `ENOENT`. `sync.fs` has it too. `rm` removes a whole tree in one kernel call (on the OPFS root, one recursive removal and one metadata delete: 50,000 files take about 3 s in Chrome, where `rm -r` takes about 25 s) with the result of a file-by-file removal: symlinks inside the tree are removed, never followed; a tree holding a protected path is refused with `EROFS` and nothing is removed; a mount inside the tree is emptied and its mount point answers `EBUSY`, while the rest is removed; under `users`, what the user may remove is removed and the call then fails with the first `EACCES`/`EPERM`; files a process holds open stay readable to it;
-- `/dev`, in `open`, `fs` and their `sync` forms, as a WASI or Emscripten program sees it, with the same modes and owners as under `users`: `null`, `zero`, `full`, `random`, `urandom` and `tty` (character devices, `0666`, root's), `stdin`, `stdout` and `stderr` (links to `/proc/self/fd/0`, `1`, `2`), `fd` (a link to `/proc/self/fd`, so `/dev/fd/N` is descriptor N) and the kernel's terminals `/dev/ttyN` and `/dev/pts/N` (with users on, owned by the session's user or the master's opener, `0620`). `/dev/shm` and `/dev/ptmx` aren't there. `stat` through `/dev/stdin`, `/dev/stdout` or `/dev/fd/N` describes the program's own descriptor (the file it is redirected to, a pipe, a terminal), `readlink` of `/dev/fd/N` names it (`pipe:[N]` and `socket:[N]` for a pipe or a socket), and a closed descriptor is `ENOENT`. Opening a device gives a kernel descriptor on it; opening `/dev/std*` or `/dev/fd/N` opens the file again with the flags asked for when the descriptor is on a file (as Linux reopens `/proc/self/fd/N`, so `append` appends and `truncate` truncates), and otherwise duplicates the descriptor; `/dev/tty` is the program's controlling terminal (`ENXIO` without one); and `/dev/ttyN` and `/dev/pts/N` are opened by name through the kernel, so a terminal that isn't the user's is `EACCES` under `enforce`. A file on anything but a regular file reads and writes as a stream (the position is ignored) and only in the directions it was opened for (`EBADF` otherwise), its `size()` is 0 and `truncate` is `EINVAL`. `readFile` reads a device or a descriptor to its end, up to 64 MiB, and fails with `EFBIG` past that (where Node's `readFile` throws `ERR_FS_FILE_TOO_LARGE`), so `readFile('/dev/zero')` fails instead of hanging, while a bounded `read` of it works. Other paths pay only a prefix check: they make the same kernel calls as before. Creating, removing or renaming names in `/dev` goes to the root file system as before;
+- `/dev`, in `open`, `fs` and their `sync` forms, as a WASI or Emscripten program sees it, with the same modes and owners as under `users`: `null`, `zero`, `full`, `random`, `urandom` and `tty` (character devices, `0666`, root's), `stdin`, `stdout` and `stderr` (links to `/proc/self/fd/0`, `1`, `2`), `fd` (a link to `/proc/self/fd`, so `/dev/fd/N` is descriptor N) and the kernel's terminals `/dev/ttyN` and `/dev/pts/N` (with users on, owned by the session's user or the master's opener, `0620`). `/dev/shm` is the kernel's shared tmpfs (see [Mounts](#mounts)), read and written like any other mount; `/dev/ptmx` isn't there. `stat` through `/dev/stdin`, `/dev/stdout` or `/dev/fd/N` describes the program's own descriptor (the file it is redirected to, a pipe, a terminal), `readlink` of `/dev/fd/N` names it (`pipe:[N]` and `socket:[N]` for a pipe or a socket), and a closed descriptor is `ENOENT`. Opening a device gives a kernel descriptor on it; opening `/dev/std*` or `/dev/fd/N` opens the file again with the flags asked for when the descriptor is on a file (as Linux reopens `/proc/self/fd/N`, so `append` appends and `truncate` truncates), and otherwise duplicates the descriptor; `/dev/tty` is the program's controlling terminal (`ENXIO` without one); and `/dev/ttyN` and `/dev/pts/N` are opened by name through the kernel, so a terminal that isn't the user's is `EACCES` under `enforce`. A file on anything but a regular file reads and writes as a stream (the position is ignored) and only in the directions it was opened for (`EBADF` otherwise), its `size()` is 0 and `truncate` is `EINVAL`. `readFile` reads a device or a descriptor to its end, up to 64 MiB, and fails with `EFBIG` past that (where Node's `readFile` throws `ERR_FS_FILE_TOO_LARGE`), so `readFile('/dev/zero')` fails instead of hanging, while a bounded `read` of it works. Other paths pay only a prefix check: they make the same kernel calls as before. Creating, removing or renaming names in `/dev` goes to the root file system as before;
 - `/proc`, in `fs`, `open` and their `sync` forms, the same as a WASI program's (see [`/proc`](#proc)): `stat`, `lstat`, `exists`, `readdir`, `readlink` and `readFile` answer it, and `/proc/self` is the program's own pid. It's read-only, so every change in it is `EROFS`: `writeFile`, `mkdir`, `rm`, `unlink`, `chmod`, a `rename`, `link` or `symlink` into or out of it, and an `open` for writing, creating or truncating. A file opened in it is a snapshot read at the open, with no descriptor (`fd` is `-1`): its `read` and `size` answer from the snapshot until it's closed, and `write` and `truncate` are `EBADF`. Other paths pay only a prefix check;
 - `signals`: `on(signal, handler)` catches a signal (by number or as `'SIGINT'`), `ignore(signal)` ignores it and `reset(signal)` restores its default action. A call blocked in a read or write is restarted after a handler runs; signals without a handler keep their default actions, so an uncaught `SIGINT` ends the program with 130 and `SIGKILL` always does;
 - `sync`: `read`, `write`, `close`, `open` (whose files' methods are synchronous too) and `fs`, made synchronously, for code that cannot `await`, such as a compiled program's file system or a `readFileSync`. A synchronous call holds the worker until the kernel answers, so nothing else of the program runs meanwhile, timers and callbacks included; a signal that arrives during a blocked read or write runs its handler, and the call is then restarted. Descriptors are shared with the rest of the context, so a file opened with `sync.open` can be closed with `close`, and the other way around; `sync.run` and `sync.exit` are described next;
@@ -554,6 +554,8 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 
 `kernel.mount({ type, source, target, options })` mounts a file system on an existing directory. Every process sees it, Emscripten and WASI alike, and so does every attached client. `kernel.umount(target)` unmounts it, and fails with `EBUSY` while a process has a file open under it. `kernel.mounts()` lists the table, as does `/proc/mounts`. The Node entry and attached clients have the same three calls (client protocol 1.2).
 
+`/dev/shm` is a tmpfs the kernel mounts at boot (source `shm`), mode `1777` and owned by root, so with `users` the sticky bit keeps one user from removing or renaming another's file there (`EPERM`). It's in `/proc/mounts`, but not in `kernel.mounts()`, which lists the mounts a caller made. Every process sees the same files, Emscripten (whose glue otherwise keeps a private `/dev/shm` in memory), WASI and JS programs alike, and they last until the kernel terminates, which is what musl's `shm_open` and `sem_open` need.
+
 - **Built in:** `tmpfs`, which lives in memory until unmounted, in 1 MiB chunks; `fsa`, a folder the user picks (see [Removable media](#removable-media-fsa)); and `hostfs`, a folder the local proxy exports (see [Host folders](#host-folders-hostfs)).
 - **Package drivers:** a package declares a type in `"slicc": { "filesystems": { "<type>": { "module": "<file>" } } }`, and the kernel starts that module in a worker of its own for each mount. The module is a single self-contained file: it can't import bare specifiers.
   - Its default export receives `{ fetch }`, the kernel's network transport. It returns `{ handlers, capabilities }`.
@@ -883,6 +885,35 @@ The Biome, TypeScript, lefthook, Renovate and CI configuration comes from [slicc
 
 Releases are cut by semantic-release on every push to `main`, in two jobs, and only on `main`. The first job runs on the bench pool, so it starts within a minute of the push, before `main` has moved on. It computes the version, tags the commit and creates the GitHub release, with `npmPublish: false`. A run that starts after `main` has moved skips; the run for the newer commit releases it. The second job runs on a GitHub-hosted runner, because npm's trusted publishing and provenance accept no self-hosted runner. It publishes the tagged commit with `tools/release-recover.mjs`. However long it waits for a runner, it publishes exactly the commit the run started on, so the provenance, the tag and the package's `gitHead` agree. For any version tagged at HEAD, `tools/release-recover.mjs` pushes the tag if origin lacks it, publishes the version if npm lacks it (with retries), and creates its GitHub release if that is missing too. So a failure after the version is tagged (a rejected push, a failed `npm publish`) doesn't strand it, and re-running the Release workflow completes it the same way.
 
+### Certification (`cert-review`)
+
+`.github/workflows/cert-review.yml` certifies every release from CI. It runs when the Release workflow succeeds on `main`. The version is the tag on the released commit. The delta is read from the newest earlier release whose `cert-review` status is `success` or `pending`, skipping failed and unreviewed ones (or `base.json`'s release, or the tag before), so a release whose review was cancelled or failed is covered by the next one, and the line says `covers <first>–<last>`. It also runs on `workflow_dispatch` with a `version`, optionally with `prev` and `base`, or with `planted` (a branch) and `pr` for proofs. A dispatch off `main`, or a planted one, is a dry run. Each release (its commit) has its own concurrency group, so no pending review is cancelled by a newer one. The tools are in `tools/cert-review/`:
+
+- **plan:** works out the version, the previous tag, the commit and the base (`base.mjs`). The base is the newest earlier release whose `cert-review` commit status is `success`, or, until there is one, the release `base.json` names (the cert lane's last line). CI never reads the npm `certified` dist-tag.
+- **delta** (`delta.mjs`): lists the pull requests merged between the two tags (their merge commits, plus `pr` for a planted branch). Each gets its conventional-commit kind (`fix`, `feat`, `perf` and `revert` are behaviour changes) and the integration test files and titles it adds or changes. The titles are read from the pull request's patch: the added `test(…)` lines and the hunk headers git puts above changed lines. Escapes are decoded as JavaScript does, and a title git cut short, or a template literal, matches as a prefix.
+- **suite** (`suite.mjs`, `reporter.mjs`) runs on the bench pool, without secrets. It takes the tag's tree with its own dependencies and Chromium, and replaces `dist/` with the **published npm tarball** (for a planted branch, its own `npm pack`).
+  - The release job runs the release's whole `test/integration/` suite, then the delta's test files again on the **previous release's dist** (the NEGATIVE).
+  - The base job runs the base release's own suite on its own published dist.
+  - Each run uses the tree's own `npm test` command, with a reporter that records every test and subtest with its status and error.
+- **kernel-compat** (`compat.mjs`) supplies the base-set and catalog evidence: homescoop's kernel-compat run for the release.
+  - Release runs share one group and intermediate ones are cancelled. When there is no finished run for the release commit, it dispatches `kernel-compat.yml` for that version (whose run is named `Kernel compat v<version>`) and waits up to 150 minutes.
+  - It reads the run's report artifact, and **re-runs every regressing package once** (a dispatch with that `set`).
+  - A regression that doesn't reproduce is a `compat-flake`; one that does is a `compat-regression`. A crash that reproduces and doesn't happen on the reference kernel is a `compat-crash`, which record FAILs in code.
+- **review:** `findings.mjs` decides in code, per pull request:
+  - `negative` (a fix's test fails on the previous dist and passes on the release) or `passes` (a feature's test passes);
+  - otherwise a finding: `no-test`, `no-negative`, `fails-on-release`, `not-run`, `prev-not-run` or `no-titles`.
+
+  It also diffs the two suites test by test (`regressed`, `newFailing`, `fixed`, `stillFailing`, `removed`), flags a suite run that didn't finish (`suite-incomplete`), and adds the kernel-compat findings. Claude, on Bedrock with the key only in that step, then reads those artifacts and `instructions.txt` (which has the roll-forward FAIL list verbatim) and writes `verdict.json` and `comment.md`. Its tools are reading files, `gh pr view`, and `gh issue list`/`view`. `gh issue create` is allowed only for a release on `main`; otherwise, and for an issue in another repository, it lists the issue under `elsewhere` for the cert lane to file.
+- **record** (`record.mjs`), in code, with read-only permissions:
+  - It re-downloads the tarball and checks `dist.integrity` (sha512) and the sha256, the SLSA v1 provenance (the tarball is its subject; `release.yml`, `refs/heads/main`, a GitHub-hosted builder; commit = `gitHead` = the tag), `npm audit signatures`, and that the commit is on `main`. A problem there, a `compat-crash`, or a planted build is a FAIL whatever the review says.
+  - It adds the **known issues as of the release**: every issue labelled `known-issue` (open or closed) and every one in `known.json`, unless it was fixed before the release. That holds when `known.json`'s `fixedIn` names an earlier or equal version, or the merged pull request that closed it is an ancestor of the tag, or it was closed (with no closing pull request) before the release commit. An issue closed as not planned or as a duplicate is dropped.
+  - The line is `certified @ai-ecoverse/slicc-kernel@<v> sha256=<…> (ci: <run>; known: …)`. Any test gaps from the findings (`no-test`, `no-negative`, `prev-not-run`, `no-titles`, `not-run`) are appended in code as `test gaps: #<pr> <kind>`, so a fix without a NEGATIVE is never certified silently.
+- **publish** runs only for a release on `main`, never for a dry run. It puts the line into a `cert-review` commit status on the tag's commit and into the GitHub release's notes, between `<!-- cert-review -->` markers.
+
+Moving the `certified` dist-tag stays manual. Automating it needs an npm token that may change dist-tags, in a step that re-hashes the tarball and tags only on a `certified` line.
+
+Not part of the run yet: bios's real setup (the base-set sweep in bios, the upgrade boot, the sudo rows under `enforce`). It would plug in as one more job before review that runs slicc-bios's `npm run test:cert` with `CERT_COMMANDS` and the release's tarball, and leaves its results next to the others.
+
 ## Test fixtures
 
 The WASIX fixtures of `test/integration/epoll.test.mjs` (#674), under `test/integration/fixtures/`, are committed with their sources, build scripts and built wasm.

package/dist/kernel-worker.js

@@ -5063,7 +5063,7 @@ function serveClient(port, host) {
       case "umount":
         return { result: await l.umountAs(clientCred(self), req.target).then(() => true) };
       case "mounts":
-        return { result: l.mounts.list() };
+        return { result: l.mounts.listed() };
       case "dial":
         return dialFor(l, req, watches);
       case "serve-cdp":
@@ -8449,6 +8449,18 @@ function withCommandDirs(fs, names, userDir = async () => void 0, owners = false
   };
 }
 
+// src/fs/dev-removal.ts
+var SHM = "/dev/shm";
+function inDevTree(path) {
+  if (path.startsWith(`${SHM}/`)) return false;
+  return path === "/dev" || path.startsWith("/dev/");
+}
+function devRemoval(path, cred) {
+  if (!inDevTree(path)) return void 0;
+  if (cred && cred.euid !== 0) return "EACCES";
+  return path === "/dev" ? "EBUSY" : "EPERM";
+}
+
 // src/fs/fresh-modules.ts
 var MAGIC = [0, 97, 115, 109];
 var MAX_FRESH = 65536;
@@ -9293,12 +9305,18 @@ var DriverConnection = class _DriverConnection {
   timeoutMs;
   nextId = 0;
   onFail;
-  constructor(port, capabilities, waiting, timeoutMs, onFail) {
+  idle;
+  constructor(port, capabilities, waiting, timeoutMs, options) {
     this.port = port;
     this.capabilities = capabilities;
     this.waiting = waiting;
     this.timeoutMs = timeoutMs;
-    this.onFail = onFail;
+    this.onFail = options.onFail;
+    this.idle = options.idle === true;
+    if (this.idle) port.unref?.();
+  }
+  settled() {
+    if (this.idle && this.waiting.size === 0) this.port.unref?.();
   }
   static async open(port, mount, options = {}) {
     const timeoutMs = options.timeoutMs ?? DRIVER_TIMEOUT_MS;
@@ -9314,6 +9332,7 @@ var DriverConnection = class _DriverConnection {
       clearTimeout(call.timer);
       if (data.errno) call.reject(errnoError(data.errno, data.message ?? data.errno));
       else call.resolve(data.result);
+      if (options.idle && waiting.size === 0) port.unref?.();
     });
     port.start?.();
     port.postMessage({ hello: { protocol: DRIVER_PROTOCOL, mount } });
@@ -9332,7 +9351,7 @@ var DriverConnection = class _DriverConnection {
     const refused2 = driverVersionError(hello.protocol);
     if (refused2) throw errnoError("EPROTO", refused2);
     if (hello.error) throw errnoError(hello.errno ?? "EIO", hello.error);
-    return new _DriverConnection(port, hello.capabilities ?? {}, waiting, timeoutMs, options.onFail);
+    return new _DriverConnection(port, hello.capabilities ?? {}, waiting, timeoutMs, options);
   }
   call(call, transfer = []) {
     if (this.failure) return Promise.reject(this.failure);
@@ -9341,6 +9360,7 @@ var DriverConnection = class _DriverConnection {
       const timer = unref(
         setTimeout(() => {
           this.waiting.delete(id2);
+          this.settled();
           const error = errnoError(
             "EIO",
             `the driver did not answer ${call.op} within ${this.timeoutMs} ms`
@@ -9350,6 +9370,7 @@ var DriverConnection = class _DriverConnection {
           this.onFail?.(error);
         }, this.timeoutMs)
       );
+      if (this.idle && this.waiting.size === 0) this.port.ref?.();
       this.waiting.set(id2, { resolve: resolve2, reject, timer });
       this.port.postMessage({ ...call, id: id2 }, transfer);
     });
@@ -9362,6 +9383,7 @@ var DriverConnection = class _DriverConnection {
       call.reject(error);
     }
     this.waiting.clear();
+    this.settled();
   }
   close() {
     this.fail(errnoError("EIO", "the file system is unmounted"));
@@ -9626,6 +9648,7 @@ var MountTable = class {
   devices = 0;
   pending = /* @__PURE__ */ new Set();
   notes = /* @__PURE__ */ new Map();
+  system = /* @__PURE__ */ new Set();
   deps;
   constructor(deps) {
     this.deps = deps;
@@ -9647,6 +9670,13 @@ var MountTable = class {
     const noted = [...this.notes.values()].filter((note) => !this.mounts.has(note.target));
     return [...mounted, ...noted.map((note) => ({ ...note, options: { ...note.options } }))];
   }
+  listed() {
+    return this.list().filter((entry) => !this.system.has(entry.target));
+  }
+  async mountSystem(spec, fs) {
+    await this.mount(spec, fs);
+    this.system.add(normalizePath(spec.target));
+  }
   note(spec, state, error) {
     const target = normalizePath(spec.target);
     this.notes.set(target, {
@@ -9708,6 +9738,7 @@ var MountTable = class {
           { source: spec.source, options },
           {
             ...this.deps.timeoutMs ? { timeoutMs: this.deps.timeoutMs } : {},
+            ...opened2.idle ? { idle: true } : {},
             onInvalidate: (paths) => this.invalidated(target, paths),
             onFail: (error) => {
               if (mounted) this.crashed(mounted, error);
@@ -9753,6 +9784,7 @@ var MountTable = class {
     if (!mount && this.notes.delete(at)) return;
     if (!mount) throw errnoError("EINVAL", `${at} is not mounted`);
     if (!detach && this.deps.busy(at)) throw errnoError("EBUSY", `${at} has open files`);
+    this.system.delete(at);
     this.notes.delete(at);
     this.mounts.delete(at);
     this.deps.reshaped?.();
@@ -10661,16 +10693,6 @@ var MountFs = class _MountFs {
   }
 };
 
-// src/fs/dev-removal.ts
-function inDevTree(path) {
-  return path === "/dev" || path.startsWith("/dev/");
-}
-function devRemoval(path, cred) {
-  if (!inDevTree(path)) return void 0;
-  if (cred && cred.euid !== 0) return "EACCES";
-  return path === "/dev" ? "EBUSY" : "EPERM";
-}
-
 // src/realm/sync-fs-token-registry.ts
 var registry = /* @__PURE__ */ new Map();
 function mintSyncFsToken(entry) {
@@ -12085,7 +12107,7 @@ var PATH_OPS = new Set(SYNC_FS_OPS);
 var EARLY_WHOLE_FILE = 64 * 1024;
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.99.0" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.99.2" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -22021,8 +22043,10 @@ var Launcher = class {
         },
         { owned: true }
       );
+      if (spec.target === SHM) port2.unref?.();
       return {
         port: port1,
+        ...spec.target === SHM ? { idle: true } : {},
         dispose: () => {
           port1.close();
           port2.close();
@@ -22123,6 +22147,11 @@ var Launcher = class {
     this.pnpmStale();
     return entry;
   }
+  async mountShm() {
+    await this.base.mkdir(SHM, { recursive: true });
+    await this.mounts.mountSystem({ type: "tmpfs", source: "shm", target: SHM }, this.fs);
+    await this.fs.chmod(SHM, 1023);
+  }
   async ownTmp() {
     const real = await this.realLayoutPath("/tmp");
     const st = real === void 0 ? void 0 : await this.base.lstat(real).catch(() => void 0);
@@ -22279,6 +22308,7 @@ var Launcher = class {
     await this.resolvePrefixes();
     for (const dir of SHARED_DIRS) await this.base.mkdir(dir, { recursive: true });
     if (this.owners) await this.ownTmp();
+    await this.mountShm();
     await this.pnpmReady();
     await this.users.ensure();
     if (this.owners) await this.stampLayout();
@@ -22827,7 +22857,7 @@ function serveKernel(port, deps) {
     if (req.op === "mount") return l.mount(req.spec);
     if (req.op === "umount") return l.umount(req.target);
     if (req.op === "insert") return l.insert(req.target, req.handle, req.source);
-    return l.mounts.list();
+    return l.mounts.listed();
   }
   const attestOp = attestOps(port);
   async function handle(req) {