@ai-ecoverse/slicc-kernel 1.93.1 → 1.93.2

package/README.md

@@ -261,7 +261,7 @@ An empty path is `ENOENT` in every path call (`path_open`, `path_filestat_get`,
 On top of preview1 a WASI program gets:
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table, and a file written, created, renamed or removed in one thread shows its new size and metadata to `stat` in every other thread at once, under any of its names (a symlink, a hard link, a new name after a rename in the same process), also when another thread renames a file over that name meanwhile, since the kernel reports which file an open or a change reached; this holds among the kernel's own processes, not against other tabs, `hostfs` writers or programs that aren't WASI; another process's changes show at close and after a `spawn` or `wait`, as with close-to-open caching, so a file another process renames while this one holds it open may show its old size under the new name until then; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
-- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`.
+- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. A forked child also gets the side modules its parent had loaded, at the same addresses and table slots, with the slot and closure counters and the parent's thread-local blocks: they are linked again in the child before it resumes, without running their constructors again, since their data is already in the copied memory. A side-module function on the stack when `fork` is called (a callback into the main module that forks) is not instrumented by Asyncify, so it runs again when the stack is rewound, in the parent and in the child, and libffi frames there can trap; side modules would need building with Asyncify for that, which the toolchain does not do yet. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`.
 - **Foreign calls** (libffi's WASIX backend, so Python's `ctypes`): `call_dynamic` calls a function through the program's indirect function table (a static module's exported `__indirect_function_table`, or the dynamic linker's), its arguments and results packed in memory one after another (`i32`/`f32` 4 bytes, `i64`/`f64` 8). With `strict` set, the buffers must fit the function exactly (else `EINVAL`, before the function is called); without it, missing arguments are zero (one cut short keeps the bytes given, with zeros above them) and results that do not fit are left out. A trap or exception of the callee goes through to the caller. JS cannot ask a function for its type, so the kernel reads the types from the module files (the program's and each side module's, once, when a program first calls through `call_dynamic`): a table slot that is empty, past the end, holds a function of no module the kernel knows of, or takes or returns a `v128`, gives `EINVAL`. For closures (callbacks such as `CFUNCTYPE`), a dynamically linked program gets table slots from `closure_allocate`, and `closure_free` gives them back for reuse and empties the slot in every thread; a static program gets `ENOTSUP`. `closure_prepare` puts a trampoline with the requested signature (any of `i32`, `i64`, `f32`, `f64`, `v128`) into such a slot, in every thread: each call stores its arguments below the stack pointer and calls the backing function as `backing(values, results, user_data)`, then returns what it wrote to `results`. The trampolines are small wasm modules made and compiled at run time, once per signature in each thread (an instance per closure), so a page whose Content Security Policy forbids compiling WebAssembly (no `'wasm-unsafe-eval'`) cannot run them; it cannot run the program either. A `fork` or a `longjmp` (both through Asyncify) inside a function reached through `call_dynamic` cannot unwind through it.
 - **Job control:** `^Z`, `SIGTSTP`, `SIGTTIN` and `SIGTTOU` stop a WASIX program at once, unless it has a handler for them. A stopped program that is busy computing is stopped for job control (`jobs` shows it, `fg`, `bg` and `SIGCONT` work), but its worker keeps a core busy until its next system call, since a worker's running code cannot be frozen from outside. Agents should `kill` such a job rather than leave it stopped. An Emscripten program stops at its next kernel call as well, and at its next write, read or sleep that its glue answers without the kernel (a write to its own `/dev/null`, as in `yes > /dev/null`), so a stopped one keeps no core busy unless it computes without any of these calls. A stopped program doesn't move on while stopped: a `sleep` (or a blocked read) that would end meanwhile ends after `SIGCONT`. A signal for such a program is also delivered at those glue-answered calls, so a handler runs in a loop that only writes to `/dev/null`. For the same reason, a signal with a handler or a terminating default (`^C`, `SIGTERM`) reaches a program that is busy computing only at its next system call, so a CPU-bound loop may not end on `^C`; `kill -9` always ends it. A pending signal is delivered at the program's next WASI or WASIX call, including the calls its worker answers without the kernel (`random_get`, `args_get`, …), except the exempt ones: `clock_time_get` (a tight loop of it measured up to 30% slower), `proc_exit`, `proc_exit2`, `thread_exit`, `thread_spawn_v2` and `wasi.thread-spawn` (wasi-threads), the calls that install handlers (`callback_signal`, `slicc.sigaction_set`), and calls made while Asyncify unwinds or rewinds the stack for `fork` or `longjmp`. A signal that arrives while a thread computes is thus delivered at its next call; when that call is a wait (`poll_oneoff` with a timeout or a descriptor, so `nanosleep` and `poll`/`select` too, `sigmask_poll`, or a futex wait), the handler has run during the call, so the wait ends at once: `EINTR` for a libc of -22 or later (an older one sees its clocks fire, as for any interruption), and a futex wait returns woken. A thread blocked in the kernel or waiting on a lock (`pthread_cond_wait`, a futex) is woken by a signal for it (a futex wait checks for signals every 50 ms, so each waiting thread wakes briefly that often), and `pthread_kill` runs the handler on the thread it names. A sleep, or a clock in `poll_oneoff`, never ends before the time asked (it can end later, never earlier). A signal that ends a sleep or a `poll`/`select` with a timeout makes the call fail with `EINTR` in a program whose libc says it is wasix-sysroot 2025.9.30-22 or later (its `slicc.libc` custom section). Older libcs turn that answer into `ENOTSUP` from `nanosleep` (up to -18), so for a program without the section the wait ends as if it had timed out, unless it imports `fd_fdflags_set` (then `EINTR`, as before).
 - **Blocked signals** (`sigprocmask`, `pthread_sigmask`, `sigpending`): a program whose libc imports `slicc.sigmask_set(how, set_lo, set_hi, old_ptr)` and `slicc.sigpending_get(ptr)` (wasix-sysroot 2025.9.30-28 and later) has a blocked mask per thread. `how` is musl's `SIG_BLOCK` 0, `SIG_UNBLOCK` 1 or `SIG_SETMASK` 2, the set is the 64-bit `sigset_t` (bit `sig - 1`), and the old mask is written as a u64 at `old_ptr`; any other `how` is `EINVAL`. `SIGKILL` and `SIGSTOP` cannot be blocked. A blocked signal is not delivered, does not interrupt a sleep or a blocked call, and its default action (terminate or stop) waits: it stays pending, `sigpending` shows it (the thread's pending signals and the process's), and it is delivered when it is unblocked, before `sigmask_set` returns. A signal ignored with `SIG_IGN` is discarded even while blocked. A forked child (with the mask of the thread that forked), the program an `exec` starts and a spawned child keep the mask, all 64 bits of it, and so does a new thread; `/proc/<pid>/status` shows the main thread's as `SigBlk`. The mask is kept in a word of the worker's shared header, so changing it costs no kernel call. A program without the import (an older libc) has no mask and gets every signal at once, as before, and one it starts by `exec` with a mask set starts with none. A signal for the whole process goes to the main thread, even if the main thread blocks it and another thread does not. A signal sent to one thread (`pthread_kill`) that it blocks waits for that thread, and if it has no handler, the thread's unblock ends the whole process by the signal, whatever the other threads block. `slicc.sigmask_poll(set_lo, set_hi, in_ptr, out_ptr, nsubscriptions, nevents_ptr)` is `pselect`, `ppoll` and `sigsuspend`: it swaps the thread's mask to the given set for the call and puts the old one back before it returns. It delivers a pending signal the new mask unblocks, without polling, or polls as `poll_oneoff` does, under the new mask, so a signal it unblocks interrupts the wait with no window between the swap and the wait; with no subscriptions it waits until a signal is delivered. An interrupted call returns `EINTR` with no events, after the handler has run.
@@ -807,6 +807,8 @@ Unit tests live in `test/unit/`, which stays out of git. They run real wasm bash
 
 The Biome, TypeScript, lefthook, Renovate and CI configuration comes from [slicc-shared-web](https://github.com/ai-ecoverse/slicc-shared-web), which also provides the `slicc-lint-comments` (no comments anywhere), `slicc-no-unit-tests` (no unit tests in git) and `slicc-diff-cover` (100% coverage of changed lines) commands that `npm run lint` and the pre-commit hook use.
 
+`test/integration/fixtures/dlfork/` is a WASIX PIE program (`dlfork.c`) that `dlopen`s a side module (`side.c`, built as `lib/libdlside.so`), forks, and has the child read the module's data and call through function pointers taken before the fork, for the fork-after-`dlopen` test. It is built with wasix-sysroot 2025.9.30-28's legacy-EH PIC variant and homescoop's pinned wasixcc, then Asyncify for `proc_fork`: `INSTALL_WASIXCC=<homescoop>/scripts/install-wasixcc.sh WASIX_SYSROOT_28=<unpacked -28 package> test/integration/fixtures/dlfork/build.sh`. `bin/dlfork.wasm` is about 800 KB, far bigger than the other fixtures: wasixcc links a position-independent main with `--whole-archive --export-all`, so side modules can find any libc symbol, and neither `--gc-sections` nor a narrower export list (`--no-export-dynamic --export=…`) takes that out.
+
 `tools/big-file.mjs [bytes]` is a manual check, not part of any suite. It writes a file of that size (64 MiB by default) through a tmpfs mount with the unit-test launcher, reads it back, and prints the peak RSS and its ratio to the file size. A 256 MiB file peaks at about 1.8 times its size. Before ranged I/O (#127), a 1.1 GiB file peaked at 18 GB of RSS. **Do not run it past 1 GiB on a workstation.** It refuses anything over 256 MiB unless `SLICC_BIG_FILE_I_HAVE_THE_RAM=1` is set. Measure a small file and extrapolate instead. The unit suite checks a 32 MiB file the same way, and allows a fixed 192 MiB for garbage V8 has yet to collect, which dominates at that size.
 
 Releases are cut by semantic-release on every push to `main`, in two jobs, and only on `main`. The first job runs on the bench pool, so it starts within a minute of the push, before `main` has moved on. It computes the version, tags the commit and creates the GitHub release, with `npmPublish: false`. A run that starts after `main` has moved skips; the run for the newer commit releases it. The second job runs on a GitHub-hosted runner, because npm's trusted publishing and provenance accept no self-hosted runner. It publishes the tagged commit with `tools/release-recover.mjs`. However long it waits for a runner, it publishes exactly the commit the run started on, so the provenance, the tag and the package's `gitHead` agree. For any version tagged at HEAD, `tools/release-recover.mjs` pushes the tag if origin lacks it, publishes the version if npm lacks it (with retries), and creates its GitHub release if that is missing too. So a failure after the version is tagged (a rejected push, a failed `npm publish`) doesn't strand it, and re-running the Release workflow completes it the same way.

package/dist/kernel-worker.js

@@ -11569,7 +11569,7 @@ function mountTable(mounts = []) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.93.1" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.93.2" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -14378,6 +14378,7 @@ var WasmProcess = class {
     this.nodes = options.nodes ?? new VfsNodes(options.fs);
     if (!options.nodes) options.openFiles?.add(this.nodes);
     this.children.onChildState = () => this.signal(SIG.CHLD);
+    this.seedDl(options.dl);
     this.children.onReap = (pid2) => {
       this.deathSignals.delete(pid2);
       options.onReap?.(pid2);
@@ -15212,7 +15213,7 @@ var WasmProcess = class {
     switch (req.op) {
       case "proc-fork":
         this.refuseIfPending(req.restart);
-        return { ok: true, kind: "json", json: await this.children.fork(req.state) };
+        return { ok: true, kind: "json", json: await this.children.fork(this.withDl(req.state)) };
       case "proc-spawn":
         this.refuseIfPending(req.restart);
         return { ok: true, kind: "json", json: await this.spawn(req) };
@@ -15355,6 +15356,20 @@ var WasmProcess = class {
     free: [],
     live: /* @__PURE__ */ new Set()
   };
+  withDl(state) {
+    if (this.dlLog.length === 0) return state;
+    const { next: next2, handle, free, live } = this.closures;
+    const dl = { log: [...this.dlLog], next: next2, handle, free: [...free], live: [...live] };
+    return { ...state, dl };
+  }
+  seedDl(dl) {
+    if (!dl) return;
+    this.dlLog.push(...dl.log);
+    this.closures.next = dl.next;
+    this.closures.handle = dl.handle;
+    this.closures.free.push(...dl.free);
+    for (const index of dl.live) this.closures.live.add(index);
+  }
   allocateClosure(floor) {
     const index = this.closures.free.pop() ?? Math.max(floor, this.closures.next);
     this.closures.next = Math.max(this.closures.next, index + 1);
@@ -15664,6 +15679,7 @@ function processOptions(opts, header3, hooks, created3) {
     ...opts.settling ? { settling: opts.settling } : {},
     spawner: opts.spawner,
     forker: opts.forker,
+    ...opts.fork?.dl ? { dl: opts.fork.dl } : {},
     fs: opts.fs,
     kill: opts.kill,
     ...opts.writesBack ? { writesBack: opts.writesBack } : {},
@@ -19570,8 +19586,17 @@ var Launcher = class {
     }
     if (interpreter(head) !== "node") return void 0;
     const wasm = modulePath(glue);
-    if (!await this.base.exists(wasm)) return void 0;
-    return { abi: "emscripten", glue, wasm, argv0: baseName(argv0 || file), ...asked };
+    if (await this.base.exists(wasm)) {
+      return { abi: "emscripten", glue, wasm, argv0: baseName(argv0 || file), ...asked };
+    }
+    const owner = await this.glueOwner(glue, baseName(file), scope);
+    return owner && { ...targetOf(owner), ...asked };
+  }
+  async glueOwner(glue, name2, scope) {
+    const owners = [...(await this.commands(scope)).values()].filter(
+      (c) => c.abi === "emscripten" && !c.script && c.glue === glue
+    );
+    return owners.find((c) => c.name === name2) ?? owners[0];
   }
   async head(path) {
     try {