@ai-ecoverse/slicc-kernel 1.100.3 → 1.101.0

package/README.md

@@ -76,7 +76,7 @@ Starts `argv` as the session leader on a new terminal and resolves once it is ru
 | `pid` | the session leader's pid |
 | `onData` | the output listener; output that arrives while it is `null` is kept and delivered when one is set |
 | `write(data)` | types a string or bytes into the terminal; with `ISIG` on, `^C`, `^Z` and `^\` signal the foreground process group. A key typed while a child of that group is still setting up is held until the child has finished, then sent to the group in the foreground at that point; this applies only to children of a job-control shell (a process that ignores `SIGTTOU`, as interactive shells do) and to the programs those children exec, so a program that starts a child without job control (Python's `subprocess`) gets the key at once. Setting up means the child's calls before its first other one: `setpgid`, `tcsetpgrp`, signal masks, and starting the program it execs (which then sets itself up too). The hold ends 2 s after the child's first call at the latest. A key typed while the foreground group is busy, not reading the terminal (a shell between reading a command line and starting it), goes to that group and also to the job the group starts next, once that job has taken the terminal; the second copy is dropped when the group reads the terminal again. So `^C` or `^Z` right after a command line reaches the new job, as it does on Linux, where starting a job takes microseconds rather than a worker start. For the same reason, a child that is still setting up cannot give the terminal to its own group once its parent has already given the terminal to that group and taken it back: on Linux the child's `tcsetpgrp` would have landed first, so the parent's later choice stands. Otherwise `tcsetpgrp` behaves as before. A `^C`, `^\` or `^Z` that a program with a handler for it takes between two system calls, rather than while waiting, ends the program's next wait in the kernel that would block (reading nothing yet, `select` or `poll` with a timeout, a blocking write, `waitpid`) with `EINTR`, once, as if the key had come during that wait. A zero-timeout `select` or `poll` does not use it up. Without this, an interactive shell whose `^C` landed while it was taking in the key before it would wait for the next key before showing a new prompt: readline acts on `^C` only when a read ends with `EINTR`. On Linux that window is the microseconds between two reads; here it is a few kernel calls per key. A signal sent with `kill` does not do this. |
-| `resize(cols, rows)` | sets the window size (`TIOCGWINSZ`) and sends `SIGWINCH` to the foreground process group |
+| `resize(cols, rows)` | sets the window size (`TIOCGWINSZ`) and sends `SIGWINCH` to the foreground process group. As on Linux, setting the size the terminal already has, here or with a program's `TIOCSWINSZ`, sends no `SIGWINCH`. A program's `TIOCSWINSZ` that writes back the size it last read with `TIOCGWINSZ` is left out, so it can't undo a resize that came in between. Readline does that write-back every time it starts reading a line; on Linux the gap between the two calls is microseconds, here it's two kernel calls, wide enough for a resize to land in it and the shell to keep the old size (#740). Limit: Linux has the same race and applies the stale write-back, only in a far shorter gap; leaving it out is a deliberate difference, so a program that really means to set the size it just read, after a resize it hasn't seen, keeps the newer size. |
 | `signal(name)` | sends a signal (`'SIGINT'`, `'SIGTSTP'`, `'SIGQUIT'`, `'SIGHUP'`, …) to the foreground process group, regardless of termios |
 | `exited` | resolves with the session leader's exit status (`128 + n` if killed by signal `n`) |
 | `ended` | resolves with `{ status, signal }` at the same time: `status` as `exited`, and `signal` named (`'SIGKILL'`) only when a signal ended it |
@@ -261,7 +261,7 @@ An empty path is `ENOENT` in every path call (`path_open`, `path_filestat_get`,
 On top of preview1 a WASI program gets:
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table, and a file written, created, renamed or removed in one thread shows its new size and metadata to `stat` in every other thread at once, under any of its names (a symlink, a hard link, a new name after a rename in the same process), also when another thread renames a file over that name meanwhile, since the kernel reports which file an open or a change reached; this holds among the kernel's own processes, not against other tabs, `hostfs` writers or programs that aren't WASI; another process's changes show at close and after a `spawn` or `wait`, as with close-to-open caching, so a file another process renames while this one holds it open may show its old size under the new name until then; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
-- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), with the unwound frames kept in the unused part of the program's stack, below its stack pointer (for a position-independent program, the stack the kernel's linker gives it), so a fork from deep in the stack leaves the program's data alone, and a forked child's thread-locals (`errno` included) where the parent left them: the kernel reruns the module's `__wasm_init_tls` at the parent's TLS base (and puts back the block it overwrites), which also sets the internal globals that hold thread-local addresses (a position-independent program's, a side module's, and those of a variable in the `global-dynamic` TLS model), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. A forked child also gets the side modules its parent had loaded, at the same addresses and table slots, with the slot and closure counters and the parent's thread-local blocks: they are linked again in the child before it resumes, without running their constructors again, since their data is already in the copied memory. A side-module function on the stack when `fork` is called (a callback into the main module that forks) is not instrumented by Asyncify, so it runs again when the stack is rewound, in the parent and in the child, and libffi frames there can trap; side modules would need building with Asyncify for that, which the toolchain does not do yet. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`.
+- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), with the unwound frames kept in the unused part of the program's stack, below its stack pointer (for a position-independent program, the stack the kernel's linker gives it), so a fork from deep in the stack leaves the program's data alone, and a forked child's thread-locals (`errno` included) where the parent left them: the kernel reruns the module's `__wasm_init_tls` at the parent's TLS base (and puts back the block it overwrites), which also sets the internal globals that hold thread-local addresses (a position-independent program's, a side module's, and those of a variable in the `global-dynamic` TLS model), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. A forked child also gets the side modules its parent had loaded, at the same addresses and table slots, with the slot and closure counters and the parent's thread-local blocks: they are linked again in the child before it resumes, without running their constructors again, since their data is already in the copied memory. A side-module function on the stack when `fork` is called (a callback into the main module that forks) is not instrumented by Asyncify, so it runs again when the stack is rewound, in the parent and in the child, and libffi frames there can trap; side modules would need building with Asyncify for that, which the toolchain does not do yet. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`. Each WASIX thread has its own Asyncify driver too, so a thread can `longjmp` through Asyncify and `fork` (#710). The unwound frames go at the bottom of the thread's own stack (the `stack` and `stack_size` its libc passes to `wasi_thread_start`), never on the main thread's. A thread whose start args don't look like wasix-libc's (a `stack_size` under 4096 or over the memory, a stack top that isn't 16-aligned or not right above the args) gets no driver, so `fork` there is `ENOSYS` as before, rather than an unwind buffer at a bad address. A child forked from a thread runs as that thread: it's rewound through `wasi_thread_start`, can fork again, and ends when the thread's function returns. A program built with exceptions (every position-independent one) does `setjmp` with exceptions rather than Asyncify, so there only `fork` uses the driver.
 - **epoll and eventfd** (WASIX `epoll_create`, `epoll_ctl`, `epoll_wait` and `fd_event`, which tokio's reactor uses through the WASIX mio fork):
   - **Where they live:** an epoll instance and an eventfd are kernel descriptors, so they survive `dup`, `fork` and `exec` (unless close-on-exec) and are shared by a process's threads.
   - **What epoll watches:** sockets (listening, connecting, connected, loopback and uplink), pipes, terminals, `kernel.run`'s stdin, eventfds and other epoll instances (a cycle of instances is `ELOOP`).
@@ -899,13 +899,14 @@ Releases are cut by semantic-release on every push to `main`, in two jobs, and o
 - **plan:** works out the version, the previous tag, the commit and the base (`base.mjs`). The base is the newest earlier release whose `cert-review` commit status is `success`, or, until there is one, the release `base.json` names (the cert lane's last line). CI never reads the npm `certified` dist-tag.
 - **delta** (`delta.mjs`): lists the pull requests merged between the two tags (their merge commits, plus `pr` for a planted branch). Each gets its conventional-commit kind (`fix`, `feat`, `perf` and `revert` are behaviour changes) and the integration test files and titles it adds or changes. The titles are read from the pull request's patch: the added `test(…)` lines and the hunk headers git puts above changed lines. Escapes are decoded as JavaScript does, and a title git cut short, or a template literal, matches as a prefix.
 - **suite** (`suite.mjs`, `reporter.mjs`) runs on the bench pool, without secrets. It takes the tag's tree with its own dependencies and Chromium, and replaces `dist/` with the **published npm tarball** (for a planted branch, its own `npm pack`). The test command runs in a session of its own (`detached`), and `suite.mjs` notes a `SIGINT` (`ignored` in its summary) instead of dying of it. That way a stray signal to the step's process group can't cut the run short, while a `SIGTERM` (a real cancel) still ends it.
-  - The release job runs the release's whole `test/integration/` suite, then the delta's test files again on the **previous release's dist** (the NEGATIVE).
+  - The release job runs the release's whole `test/integration/` suite, then each fix's test files again on the dist of **the newest published release before the fix was merged** (the NEGATIVE). `delta.mjs` groups the files by that baseline: for most fixes it is the previous release, but a fix for a bug that shipped inside the delta (a regression in v1.99.1 fixed for v1.99.2) is checked against the release that had the bug.
   - The base job runs the base release's own suite on its own published dist.
   - Each run uses the tree's own `npm test` command, with a reporter that records every test and subtest with its status and error.
 - **kernel-compat** (`compat.mjs`) supplies the base-set and catalog evidence: homescoop's kernel-compat run for the release.
   - Release runs share one group and intermediate ones are cancelled. When there is no finished run for the release commit, it dispatches `kernel-compat.yml` for that version (whose run is named `Kernel compat v<version>`) and waits up to 150 minutes.
   - It reads the run's report artifact, and **re-runs every regressing package once** (a dispatch with that `set`).
   - A regression that doesn't reproduce is a `compat-flake`; one that does is a `compat-regression`. A crash that reproduces and doesn't happen on the reference kernel is a `compat-crash`, which record FAILs in code.
+  - A run that failed with no regression in its report is a `compat-incomplete`, missing evidence and not a regression. It names the shards that left no report (for example, a shard that lost its bench runner), and those shards' failed jobs are re-run once first.
 - **review:** `findings.mjs` decides in code, per pull request:
   - `negative` (a fix's test fails on the previous dist and passes on the release) or `passes` (a feature's test passes);
   - otherwise a finding: `no-test`, `no-negative`, `fails-on-release`, `not-run`, `prev-not-run` or `no-titles`.
@@ -1007,6 +1008,14 @@ Two builds of one C program (`forktls.c` with `tlsvars.c`), a position-independe
 
 **Build:** `WASIX_SYSROOT_28=/path/to/wasix-sysroot ./build.sh`, with homescoop's pinned `wasixcc` 0.4.7 and binaryen's `wasm-opt` 133 on `PATH` (the same toolchain rebuilds `bin/piefork.wasm` byte for byte). `bin/forktls-pie.wasm` is 816,066 bytes, sha256 `c5ed6b06036f4d3c42c660fc13ca1a995d647f11021829f2169567574254fd9e`; `bin/forktls-static.wasm` is 45,564 bytes, sha256 `5df019d5f2a4432132c8b7c6f5992d18ecf9d8e4c4be21351acd59c46c3ca48d`; `lib/libtlsside.so` is 736 bytes, sha256 `ecd32e561c051576a99e87caee5e1aca5238c4055e2e551065fa10fe7476d0b5`.
 
+### `threadjmp`
+
+A threaded WASIX C program for `test/integration/thread-asyncify.test.mjs` (#710). A thread longjmps 50 times through a 40-frame recursion and forks. Its child forks again and reports its own child's exit status. Meanwhile the main thread keeps recursing 200 frames deep and checks its frames afterwards (`main damaged`).
+- `bin/threadjmp.wasm`: built without exceptions (`WASIXCC_WASM_EXCEPTIONS=no`), so `setjmp` goes through Asyncify's `stack_checkpoint`/`stack_restore`. wasixcc runs Asyncify itself for that build.
+- `bin/threadjmp-pie.wasm`: position-independent (`WASIXCC_PIC=yes`, legacy exceptions), with Asyncify for `proc_fork` only. It's built with `-DFORK_ONLY`, because Asyncify can't flatten a function that uses exceptions for `setjmp`.
+
+**Build:** `WASIX_SYSROOT_28=/path/to/wasix-sysroot ./build.sh`, with homescoop's pinned `wasixcc` 0.4.7 and binaryen's `wasm-opt` on `PATH`. `bin/threadjmp.wasm` is 76,125 bytes, sha256 `351fe15874125b1115f555ea406af9a6b27700bb8a51eb599baf7529ef631b20`; `bin/threadjmp-pie.wasm` is 815,633 bytes, sha256 `9179896d805b005b8d9569286b6e233ee96f718ae3630708c859be3eee5c1c08`.
+
 ### `stopwait`
 
 A WASIX C program for `test/integration/stop-restart.test.mjs` (#738). `stopwait CALL` forks a child that waits 300 ms in `CALL` (`epoll`, `poll`, `ppoll`, `select`, `pselect`, `nanosleep` or `clock_nanosleep`) on a pipe that never gets data. The parent stops the child 50 ms after it's ready and continues it 200 ms later. The child prints the result, the error and whether the wait ended before the deadline, at it (300–450 ms) or late.

package/dist/kernel-worker.js

@@ -12114,7 +12114,7 @@ var PATH_OPS = new Set(SYNC_FS_OPS);
 var EARLY_WHOLE_FILE = 64 * 1024;
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.100.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.101.0" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -14017,6 +14017,7 @@ var KernelTty = class {
     this.wake();
   }
   resize(cols, rows) {
+    if (cols === this.cols && rows === this.rows) return;
     this.setSize(cols, rows);
     this.signal(SIG.WINCH);
   }
@@ -14391,6 +14392,9 @@ function terminalOf(ctx, fd) {
 }
 var json4 = (value) => ({ ok: true, kind: "json", json: value });
 var done3 = { ok: true, kind: "void" };
+function sizeKey(rows, cols) {
+  return `${rows}x${cols}`;
+}
 function ptySyscall(req, ctx) {
   switch (req.op) {
     case "pty-open": {
@@ -14423,9 +14427,11 @@ function ptySyscall(req, ctx) {
     case "pty-packet":
       masterOf(ctx, req.fd).packet = req.on;
       return done3;
-    case "pty-winsz-set":
-      terminalOf(ctx, req.fd).resize(req.cols, req.rows);
+    case "pty-winsz-set": {
+      const tty = terminalOf(ctx, req.fd);
+      if (ctx.sizesRead?.get(tty) !== sizeKey(req.rows, req.cols)) tty.resize(req.cols, req.rows);
       return done3;
+    }
   }
 }
 
@@ -14589,13 +14595,6 @@ var SEM_NAMES_MAX = 4096;
 var SEM_HANDLES_MAX = 256;
 var NAME_MAX = 251;
 var TIMER_MAX_MS = 2147483647;
-var deadlines = /* @__PURE__ */ new WeakMap();
-function timeLeft(req, timeoutMs) {
-  const now = performance.now();
-  const deadline = deadlines.get(req) ?? now + timeoutMs;
-  deadlines.set(req, deadline);
-  return Math.max(0, deadline - now);
-}
 var SEM_OPS = [
   "sem-open",
   "sem-close",
@@ -14782,9 +14781,8 @@ async function semSyscall(req, proc) {
       const sem = handles.get(req.handle);
       if (sem.take()) return ok2();
       if (req.how === "try") throw new KernelError("EAGAIN");
-      const timeoutMs = req.how === "block" ? -1 : timeLeft(req, req.timeoutMs);
-      if (timeoutMs === 0) throw new KernelError("ETIMEDOUT");
-      const got = await sem.wait(proc.blocking(), timeoutMs);
+      if (req.how === "timed" && req.timeoutMs === 0) throw new KernelError("ETIMEDOUT");
+      const got = await sem.wait(proc.blocking(), req.how === "block" ? -1 : req.timeoutMs);
       if (!got) throw new KernelError("ETIMEDOUT");
       return ok2();
     }
@@ -15807,7 +15805,7 @@ function identityKey(st, path) {
 }
 var DYING = { ok: false, errno: "EINTR", message: "the process is being killed" };
 var HOSTS_ONLY = new Resolver({ routes: new Routes() });
-function timeLeft2(req, started) {
+function timeLeft(req, started) {
   if (!("timeoutMs" in req) || typeof req.timeoutMs !== "number" || req.timeoutMs < 0) return req;
   return { ...req, timeoutMs: Math.max(0, req.timeoutMs - (performance.now() - started)) };
 }
@@ -15969,6 +15967,7 @@ var WasmProcess = class {
     for (const listener of this.stateListeners) listener("continued", SIG.CONT);
   }
   blocks = /* @__PURE__ */ new WeakMap();
+  sizesRead = /* @__PURE__ */ new WeakMap();
   async syscall(req, cancelled, block) {
     this.options.onSyscall?.(req);
     const started = performance.now();
@@ -15976,7 +15975,7 @@ var WasmProcess = class {
     for (; ; ) {
       await this.resumed;
       if (this.dying) return DYING;
-      const current = restarted ? timeLeft2(req, started) : req;
+      const current = restarted ? timeLeft(req, started) : req;
       const stops = this.stops;
       const scope = new AbortController();
       const forward = () => scope.abort();
@@ -16039,7 +16038,17 @@ var WasmProcess = class {
         const owner = this.options.owners ? this.ttyOwner() : void 0;
         const guard2 = this.guarded();
         const { outputFilter } = this.options;
-        const ctx = { pid: this.pid, fds: this.fds, ptys, jobs, guard: guard2, owner, outputFilter };
+        const sizesRead = this.sizesRead;
+        const ctx = {
+          pid: this.pid,
+          fds: this.fds,
+          ptys,
+          jobs,
+          guard: guard2,
+          owner,
+          outputFilter,
+          sizesRead
+        };
         return ptySyscall(req, ctx);
       }
       return await this.procSyscall(req, cancelled);
@@ -16675,8 +16684,11 @@ var WasmProcess = class {
       case "tty-set":
         tty.tcsets(req.termios, req.flush === true);
         return { ok: true, kind: "void" };
-      case "tty-winsz":
-        return { ok: true, kind: "json", json: tty.winsize() };
+      case "tty-winsz": {
+        const size = tty.winsize();
+        this.sizesRead.set(tty, sizeKey(...size));
+        return { ok: true, kind: "json", json: size };
+      }
       case "tty-pgrp-get":
         return { ok: true, kind: "json", json: jobs ? jobs.tcgetpgrp(tty, this.sid()) : this.pid };
       case "tty-pgrp-set":