@ai-ecoverse/slicc-kernel 1.147.0 → 1.147.1

package/README.md

@@ -275,7 +275,7 @@ An empty path is `ENOENT` in every path call (`path_open`, `path_filestat_get`,
 On top of preview1 a WASI program gets:
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table, and a file written, created, renamed or removed in one thread shows its new size and metadata to `stat` in every other thread at once, under any of its names (a symlink, a hard link, a new name after a rename in the same process), also when another thread renames a file over that name meanwhile, since the kernel reports which file an open or a change reached; this holds among the kernel's own processes, not against other tabs, `hostfs` writers or programs that aren't WASI; another process's changes show at close and after a `spawn` or `wait`, as with close-to-open caching, so a file another process renames while this one holds it open may show its old size under the new name until then; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
-- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), with the unwound frames kept in the unused part of the program's stack, below its stack pointer (for a position-independent program, the stack the kernel's linker gives it), so a fork from deep in the stack leaves the program's data alone, and a forked child's thread-locals (`errno` included) where the parent left them: the kernel reruns the module's `__wasm_init_tls` at the parent's TLS base (and puts back the block it overwrites), which also sets the internal globals that hold thread-local addresses (a position-independent program's, a side module's, and those of a variable in the `global-dynamic` TLS model), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. A forked child also gets the side modules its parent had loaded, at the same addresses and table slots, with the slot and closure counters and the parent's thread-local blocks: they are linked again in the child before it resumes, without running their constructors or their data relocations again, since their data is already in the copied memory. The same goes for the main module of a position-independent program: its data relocations ran once, in the parent, so a pointer the parent changed before the fork (`int *p = &a; p = &b;`) is still changed in the child (#777). A side-module function on the stack when `fork` is called (a callback into the main module that forks) is not instrumented by Asyncify, so it runs again when the stack is rewound, in the parent and in the child, and libffi frames there can trap; side modules would need building with Asyncify for that, which the toolchain does not do yet. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`. Each WASIX thread has its own Asyncify driver too, so a thread can `longjmp` through Asyncify and `fork` (#710). The unwound frames go at the bottom of the thread's own stack (the `stack` and `stack_size` its libc passes to `wasi_thread_start`), never on the main thread's. A thread whose start args don't look like wasix-libc's (a `stack_size` under 4096 or over the memory, a stack top that isn't 16-aligned or not right above the args) gets no driver, so `fork` there is `ENOSYS` as before, rather than an unwind buffer at a bad address. A child forked from a thread runs as that thread: it's rewound through `wasi_thread_start`, can fork again, and ends when the thread's function returns. A program built with exceptions (every position-independent one) does `setjmp` with exceptions rather than Asyncify, so there only `fork` uses the driver. A kernel timer can be longer than `setTimeout`'s 2^31-1 ms (about 24.8 days): an `alarm`, an interval timer, or a `select`, `poll_oneoff` or `epoll_wait` timeout waits its whole time, re-armed in steps of at most 2^31-1 ms, instead of firing at once or spinning (#767). A `poll_oneoff` read event's `nbytes` is the number of bytes a read would get at once, as `FIONREAD` counts them, for a pipe, a FIFO, a socket, a terminal and a pty master; on a hangup it's 0 when nothing is left, so a libc can report `POLLHUP` without `POLLIN`, as Linux does (#1053). Other descriptors, regular files included, answer 1. A WASI or WASIX program that a signal ends (`SIGKILL`, or a signal whose default terminates it) stops at its next kernel call. The kernel first lets the calls it is already serving finish and the program write its files back: it ends the process once the program has made no file call for 2 s, and 5 s after the signal at the latest. A call the program makes after the kill is answered `EINTR` at once and doesn't extend the wait, so a program that swallows the kill and keeps retrying (python writing a fatal error to stderr) still ends (#755).
+- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), with the unwound frames kept in the unused part of the program's stack, below its stack pointer (for a position-independent program, the stack the kernel's linker gives it), so a fork from deep in the stack leaves the program's data alone, and a forked child's thread-locals (`errno` included) where the parent left them: the kernel reruns the module's `__wasm_init_tls` at the parent's TLS base (and puts back the block it overwrites), which also sets the internal globals that hold thread-local addresses (a position-independent program's, a side module's, and those of a variable in the `global-dynamic` TLS model), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. A side module's data (its `dylink.0` memory size) is allocated with the program's own `calloc`, when the program exports one, as Emscripten's loader does, so a `dlopen` doesn't grow memory while the heap has room ([#1408](https://github.com/ai-ecoverse/slicc-kernel/issues/1408)); without `calloc`, or mid-unwind, memory grows as before. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. A forked child also gets the side modules its parent had loaded, at the same addresses and table slots, with the slot and closure counters and the parent's thread-local blocks: they are linked again in the child before it resumes, without running their constructors or their data relocations again, since their data is already in the copied memory. The same goes for the main module of a position-independent program: its data relocations ran once, in the parent, so a pointer the parent changed before the fork (`int *p = &a; p = &b;`) is still changed in the child (#777). A side-module function on the stack when `fork` is called (a callback into the main module that forks) is not instrumented by Asyncify, so it runs again when the stack is rewound, in the parent and in the child, and libffi frames there can trap; side modules would need building with Asyncify for that, which the toolchain does not do yet. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`. Each WASIX thread has its own Asyncify driver too, so a thread can `longjmp` through Asyncify and `fork` (#710). The unwound frames go at the bottom of the thread's own stack (the `stack` and `stack_size` its libc passes to `wasi_thread_start`), never on the main thread's. A thread whose start args don't look like wasix-libc's (a `stack_size` under 4096 or over the memory, a stack top that isn't 16-aligned or not right above the args) gets no driver, so `fork` there is `ENOSYS` as before, rather than an unwind buffer at a bad address. A child forked from a thread runs as that thread: it's rewound through `wasi_thread_start`, can fork again, and ends when the thread's function returns. A program built with exceptions (every position-independent one) does `setjmp` with exceptions rather than Asyncify, so there only `fork` uses the driver. A kernel timer can be longer than `setTimeout`'s 2^31-1 ms (about 24.8 days): an `alarm`, an interval timer, or a `select`, `poll_oneoff` or `epoll_wait` timeout waits its whole time, re-armed in steps of at most 2^31-1 ms, instead of firing at once or spinning (#767). A `poll_oneoff` read event's `nbytes` is the number of bytes a read would get at once, as `FIONREAD` counts them, for a pipe, a FIFO, a socket, a terminal and a pty master; on a hangup it's 0 when nothing is left, so a libc can report `POLLHUP` without `POLLIN`, as Linux does (#1053). Other descriptors, regular files included, answer 1. A WASI or WASIX program that a signal ends (`SIGKILL`, or a signal whose default terminates it) stops at its next kernel call. The kernel first lets the calls it is already serving finish and the program write its files back: it ends the process once the program has made no file call for 2 s, and 5 s after the signal at the latest. A call the program makes after the kill is answered `EINTR` at once and doesn't extend the wait, so a program that swallows the kill and keeps retrying (python writing a fatal error to stderr) still ends (#755).
 - **epoll and eventfd** (WASIX `epoll_create`, `epoll_ctl`, `epoll_wait` and `fd_event`, which tokio's reactor uses through the WASIX mio fork):
   - **Where they live:** an epoll instance and an eventfd are kernel descriptors, so they survive `dup`, `fork` and `exec` (unless close-on-exec) and are shared by a process's threads.
   - **What epoll watches:** sockets (listening, connecting, connected, loopback and uplink), pipes, terminals, `kernel.run`'s stdin, eventfds and other epoll instances (a cycle of instances is `ELOOP`).

package/dist/kernel-worker.js

@@ -7785,7 +7785,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.147.0" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.147.1" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC

package/dist/node-process-worker.js

@@ -7157,7 +7157,7 @@ var KERNEL_VERSION, OS_RELEASE;
 var init_version = __esm({
   "src/version.ts"() {
     "use strict";
-    KERNEL_VERSION = true ? "1.147.0" : "0.0.0-dev";
+    KERNEL_VERSION = true ? "1.147.1" : "0.0.0-dev";
     OS_RELEASE = "6.0.0-slicc";
   }
 });
@@ -17601,9 +17601,20 @@ var init_wasix_linker = __esm({
         this.publisher?.({ kind: "slot", index: index3, handle, name: name2 });
         return index3;
       }
+      heapAlloc(bytes) {
+        const exports = this.main?.exports;
+        const calloc = exports?.calloc;
+        if (typeof calloc !== "function") return void 0;
+        const state = exports?.asyncify_get_state;
+        if (typeof state === "function" && state() !== 0) return void 0;
+        const at = calloc(1, bytes) >>> 0;
+        return at === 0 ? void 0 : at;
+      }
       allocate(size, alignment) {
         if (size === 0) return 0;
         const to = Math.max(alignment, 16);
+        const heap = this.heapAlloc(size + to - 16);
+        if (heap !== void 0) return align(heap, to);
         const pages2 = Math.ceil((size + (to > PAGE2 ? to : 0)) / PAGE2);
         return align(this.memory.grow(pages2) * PAGE2, to);
       }