@ai-ecoverse/slicc-kernel 1.54.3 → 1.54.5

package/README.md

@@ -186,8 +186,8 @@ A descriptor a program inherits at one of those numbers (from bash's `3<file`, o
 On top of preview1 a WASI program gets:
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
-- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`.
-- **Foreign calls** (libffi's WASIX backend, so Python's `ctypes`): `call_dynamic` calls a function through the program's indirect function table (a static module's exported `__indirect_function_table`, or the dynamic linker's), its arguments and results packed in memory one after another (`i32`/`f32` 4 bytes, `i64`/`f64` 8). With `strict` set, the buffers must fit the function exactly (else `EINVAL`); without it, missing arguments are zero and results that do not fit are left out. A trap or exception of the callee goes through to the caller. JS cannot ask a function for its type, so the kernel reads the types from the module files (the program's and each side module's, once, when a program first calls through `call_dynamic`): a table slot that is empty, past the end, holds a function of no module the kernel knows of, or takes or returns a `v128`, gives `EINVAL`. For closures (callbacks such as `CFUNCTYPE`), a dynamically linked program gets table slots from `closure_allocate`, and `closure_free` gives them back for reuse and empties the slot in every thread; a static program gets `ENOTSUP`. `closure_prepare` puts a trampoline with the requested signature (any of `i32`, `i64`, `f32`, `f64`, `v128`) into such a slot, in every thread: each call stores its arguments below the stack pointer and calls the backing function as `backing(values, results, user_data)`, then returns what it wrote to `results`. The trampolines are small wasm modules made and compiled at run time, once per signature in each thread (an instance per closure), so a page whose Content Security Policy forbids compiling WebAssembly (no `'wasm-unsafe-eval'`) cannot run them; it cannot run the program either. A `fork` or a `longjmp` (both through Asyncify) inside a function reached through `call_dynamic` cannot unwind through it.
+- **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), exec, `posix_spawn`, `waitpid` (with `WNOHANG`, and `WUNTRACED` for a child that stops; WASIX has no `WCONTINUED`), pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`. Each thread has its own copy of the function table, so the kernel hands out its slots (a side module's range, a `dlsym`'d function, a closure): threads loading or looking up at the same time never get the same slot. As in glibc, one `dlopen` at a time runs in a process, constructors included: a `dlopen` from a library's constructor goes ahead in its own thread, and another thread's `dlopen` waits, then finds the library loaded and gets the same handle. A thread waiting there still ends or stops with its process, but a signal handler it has runs once that `dlopen` returns. The lock is let go when a `dlopen` fails (an exception, a trap or `pthread_exit` out of a constructor included) and when its thread ends; a forked child starts with it free. In such a program, `dlopen(NULL)`, and `dlopen` of the program's own file (also by a path that resolves to it through symlinks), give a handle for the program that is already running (no second copy is loaded); a program that isn't position-independent gets an error from `dlopen(NULL)`, as from any `dlopen`; `dlsym` on it searches the program, then the loaded libraries, and `dlclose` of it does nothing. A function already in the table (the program's own, or a library's) keeps its slot, so `dlsym(handle, "strlen") == strlen`.
+- **Foreign calls** (libffi's WASIX backend, so Python's `ctypes`): `call_dynamic` calls a function through the program's indirect function table (a static module's exported `__indirect_function_table`, or the dynamic linker's), its arguments and results packed in memory one after another (`i32`/`f32` 4 bytes, `i64`/`f64` 8). With `strict` set, the buffers must fit the function exactly (else `EINVAL`, before the function is called); without it, missing arguments are zero (one cut short keeps the bytes given, with zeros above them) and results that do not fit are left out. A trap or exception of the callee goes through to the caller. JS cannot ask a function for its type, so the kernel reads the types from the module files (the program's and each side module's, once, when a program first calls through `call_dynamic`): a table slot that is empty, past the end, holds a function of no module the kernel knows of, or takes or returns a `v128`, gives `EINVAL`. For closures (callbacks such as `CFUNCTYPE`), a dynamically linked program gets table slots from `closure_allocate`, and `closure_free` gives them back for reuse and empties the slot in every thread; a static program gets `ENOTSUP`. `closure_prepare` puts a trampoline with the requested signature (any of `i32`, `i64`, `f32`, `f64`, `v128`) into such a slot, in every thread: each call stores its arguments below the stack pointer and calls the backing function as `backing(values, results, user_data)`, then returns what it wrote to `results`. The trampolines are small wasm modules made and compiled at run time, once per signature in each thread (an instance per closure), so a page whose Content Security Policy forbids compiling WebAssembly (no `'wasm-unsafe-eval'`) cannot run them; it cannot run the program either. A `fork` or a `longjmp` (both through Asyncify) inside a function reached through `call_dynamic` cannot unwind through it.
 - **Job control:** `^Z`, `SIGTSTP`, `SIGTTIN` and `SIGTTOU` stop a WASIX program at once, unless it has a handler for them. A stopped program that is busy computing is stopped for job control (`jobs` shows it, `fg`, `bg` and `SIGCONT` work), but its worker keeps a core busy until its next system call, since a worker's running code cannot be frozen from outside. Agents should `kill` such a job rather than leave it stopped. For the same reason, a signal with a handler or a terminating default (`^C`, `SIGTERM`) reaches a program that is busy computing only at its next system call, so a CPU-bound loop may not end on `^C`; `kill -9` always ends it. A thread blocked in the kernel or waiting on a lock (`pthread_cond_wait`, a futex) is woken by a signal for it (a futex wait checks for signals every 50 ms, so each waiting thread wakes briefly that often), and `pthread_kill` runs the handler on the thread it names. A signal that ends a sleep or a `poll`/`select` with a timeout makes the call fail with `EINTR` in a program whose libc says it is wasix-sysroot 2025.9.30-22 or later (its `slicc.libc` custom section). Older libcs turn that answer into `ENOTSUP` from `nanosleep` (up to -18), so for a program without the section the wait ends as if it had timed out, unless it imports `fd_fdflags_set` (then `EINTR`, as before).
 - **Pipes** have no file type for WASI and WASIX programs' `fstat` (`st_mode` type 0), because WASI has no FIFO file type. A socket type, as kernels 1.47.3 to 1.48.1 gave them, made Ruby treat a `popen` pipe as a socket (`close_write` then never ends the child's input). The kernel's `slicc_fs.fd_mode` answers `S_IFIFO | 0600` (Linux numbering) for a pipe, and wasix-sysroot 2025.9.30-21 and later take the type from it, so programs built on it see a FIFO (`S_ISFIFO`); older libcs keep only the permission bits. Until then Python's asyncio refuses a pipe for a subprocess (`Pipe transport is only for pipes, sockets and character devices`). They are read and written as pipes; socket calls on them fail with `ENOTSOCK`, and `isatty()` is false. Emscripten programs, bash included, see a FIFO. The same holds for any kernel descriptor that is neither seekable nor a terminal or device, such as captured output.
 - **Sockets**: the WASIX socket calls (`sock_open`, `sock_pair` for an `AF_UNIX` stream pair, `bind`, `listen`, `connect`, `accept`, options, local and peer names) are the kernel's loopback sockets, and preview1's `sock_recv` / `sock_send` / `sock_shutdown` work on them and on inherited ones. Writing to a socket whose peer is gone fails with `EPIPE`, through `fd_write` as through `sock_send`, as if `MSG_NOSIGNAL` were set, since a WASI program cannot ignore `SIGPIPE`; a pipe or terminal whose reader is gone still ends the program with status 141. HTTP and HTTPS leave through the realm proxy that `https_proxy` names, as for any other program; `resolve` answers `/etc/hosts` names and literal addresses, and the [uplink](#uplink)'s names.

package/dist/kernel-worker.js

@@ -8152,12 +8152,14 @@ var AsyncOps = class {
     const at = this.ready.indexOf(id2);
     if (at >= 0) this.ready.splice(at, 1);
   }
-  async wait(timeoutMs, signal) {
+  async wait(timeoutMs, blocking) {
+    let signal;
     for (; ; ) {
       if (this.closed) return -1;
       const id2 = this.ready.shift();
       if (id2 !== void 0) return id2;
       if (timeoutMs !== void 0 && timeoutMs <= 0) return 0;
+      signal ??= blocking();
       const woke = await this.sleep(timeoutMs, signal);
       if (!woke) return 0;
     }
@@ -8218,7 +8220,7 @@ async function hostSyscall(req, ctx) {
       if (!ctx.valid(req.req)) throw new KernelError("EINVAL");
       return json3(ctx.ops.submit(req.req));
     case "async-wait":
-      return json3(await ctx.ops.wait(req.timeoutMs, ctx.blocking()));
+      return json3(await ctx.ops.wait(req.timeoutMs, ctx.blocking));
     case "async-take":
       return ctx.ops.take(req.id);
     case "async-resolve":
@@ -9820,6 +9822,9 @@ var SYSCALL_OPS = /* @__PURE__ */ new Set([
   "dl-closure",
   "dl-closure-free",
   "dl-reserve",
+  "dl-load-reserve",
+  "dl-lock",
+  "dl-unlock",
   "fd-renumber",
   "fd-promote",
   "fd-open-tty",
@@ -9879,7 +9884,7 @@ var WasmProcess = class {
   stopped = 0;
   stops = 0;
   owed = false;
-  waits = 0;
+  waiting = /* @__PURE__ */ new Set();
   resumed = Promise.resolve();
   wake;
   stateListeners = [];
@@ -9935,7 +9940,8 @@ var WasmProcess = class {
       return "terminate";
     }
     this.options.onPending?.(sig);
-    if (this.options.jobs?.keyboard && KEYBOARD.has(sig)) this.owed = true;
+    if (this.options.jobs?.keyboard && KEYBOARD.has(sig) && this.waiting.size === 0)
+      this.owed = true;
     const blocked = this.interrupt;
     this.interrupt = new AbortController();
     blocked.abort();
@@ -9980,10 +9986,15 @@ var WasmProcess = class {
       await this.resumed;
       if (this.dying) return DYING;
       const stops = this.stops;
-      const waits = this.waits;
-      const result = await this.dispatch(req, cancelled);
+      const scope = new AbortController();
+      const forward = () => scope.abort();
+      if (cancelled?.aborted) scope.abort();
+      cancelled?.addEventListener("abort", forward, { once: true });
+      const result = await this.dispatch(req, scope.signal).finally(() => {
+        cancelled?.removeEventListener("abort", forward);
+        this.waiting.delete(scope.signal);
+      });
       if (this.dying) return DYING;
-      if (!result.ok && result.errno === "EINTR" && this.waits !== waits) this.owed = false;
       const restart = !result.ok && result.errno === "EINTR" && this.stops !== stops;
       if (restart && !this.options.hasPending?.()) continue;
       await this.resumed;
@@ -10036,12 +10047,31 @@ var WasmProcess = class {
   refuseIfPending(restart) {
     if (restart && this.options.hasPending?.()) throw new KernelError("EINTR");
   }
+  async procWait(req, cancelled) {
+    const flags = {
+      untraced: req.untraced,
+      continued: req.continued,
+      inGroup: this.waitGroup(req.pid)
+    };
+    if (this.owed && !req.nohang) {
+      const now = await this.children.wait(req.pid, true, this.interrupt.signal, flags);
+      if (now[0] !== 0) return now;
+    }
+    const signal = req.nohang ? this.interrupt.signal : this.blockingSignal(cancelled);
+    const waited = await this.children.wait(req.pid, req.nohang, signal, flags);
+    return waited;
+  }
   blockingSignal(cancelled, owing = true) {
-    if (owing) this.waits++;
-    if (this.options.hasPending?.() || cancelled?.aborted || owing && this.owed) {
+    if (owing && this.owed) {
+      this.owed = false;
       throw new KernelError("EINTR");
     }
-    return cancelled ? AbortSignal.any([this.interrupt.signal, cancelled]) : this.interrupt.signal;
+    if (this.options.hasPending?.() || cancelled.aborted) throw new KernelError("EINTR");
+    if (owing) this.waiting.add(cancelled);
+    return this.cancelSignal(cancelled);
+  }
+  cancelSignal(cancelled) {
+    return AbortSignal.any([this.interrupt.signal, cancelled]);
   }
   async read(req, cancelled) {
     const file = this.fds.get(req.fd).file;
@@ -10065,7 +10095,7 @@ var WasmProcess = class {
     if (nonblock) return file.write(body, AbortSignal.abort());
     return file.write(
       body,
-      this.options.hasPending?.() ? AbortSignal.abort() : this.interrupt.signal
+      this.options.hasPending?.() ? AbortSignal.abort() : this.cancelSignal(cancelled)
     );
   }
   async openVfs(req) {
@@ -10209,6 +10239,11 @@ var WasmProcess = class {
         return { ok: true, kind: "json", json: this.options.jobs?.terminalNames() ?? [] };
       case "fd-select": {
         const { read, write, timeoutMs } = req;
+        if (this.owed && timeoutMs !== 0) {
+          const ready2 = await selectFds(this.fds, read, write, 0, this.interrupt.signal);
+          if (ready2.read.length + ready2.write.length > 0)
+            return { ok: true, kind: "json", json: ready2 };
+        }
         const signal = this.blockingSignal(cancelled, timeoutMs !== 0);
         const selected = await selectFds(this.fds, read, write, timeoutMs, signal);
         return { ok: true, kind: "json", json: selected };
@@ -10406,16 +10441,8 @@ var WasmProcess = class {
       case "proc-spawn":
         this.refuseIfPending(req.restart);
         return { ok: true, kind: "json", json: await this.spawn(req) };
-      case "proc-wait": {
-        const signal = req.nohang ? this.interrupt.signal : this.blockingSignal(cancelled);
-        const flags = {
-          untraced: req.untraced,
-          continued: req.continued,
-          inGroup: this.waitGroup(req.pid)
-        };
-        const waited = await this.children.wait(req.pid, req.nohang, signal, flags);
-        return { ok: true, kind: "json", json: waited };
-      }
+      case "proc-wait":
+        return { ok: true, kind: "json", json: await this.procWait(req, cancelled) };
       case "proc-identity":
         return { ok: true, kind: "json", json: await this.options.identity?.() ?? null };
       case "proc-exec": {
@@ -10447,7 +10474,7 @@ var WasmProcess = class {
         return { ok: true, kind: "json", json: this.options.mounts?.() ?? [] };
       case "mount":
       case "umount":
-        return this.mountSyscall(req);
+        return this.mountSyscall(req, cancelled);
       case "proc-alarm":
         this.setAlarm(req.sig, req.firstMs ?? req.ms, req.repeat ? req.ms : 0, req.timer);
         return { ok: true, kind: "void" };
@@ -10468,6 +10495,18 @@ var WasmProcess = class {
         return { ok: true, kind: "void" };
       case "dl-reserve":
         return { ok: true, kind: "json", json: this.reserveSlots(req.floor, req.size) };
+      case "dl-load-reserve":
+        return {
+          ok: true,
+          kind: "json",
+          json: { base: this.reserveSlots(req.floor, req.size), handle: this.closures.handle++ }
+        };
+      case "dl-lock":
+        await this.lockDl(req.owner);
+        return { ok: true, kind: "void" };
+      case "dl-unlock":
+        this.unlockDl(req.owner);
+        return { ok: true, kind: "void" };
       case "sig-mask":
         this.caught = req.caught;
         this.ignored = req.ignored;
@@ -10482,14 +10521,14 @@ var WasmProcess = class {
         return { ok: true, kind: "bytes", bytes: this.children.captured(req.pid, req.slot) };
     }
   }
-  async mountSyscall(req) {
+  async mountSyscall(req, cancelled) {
     const { mount, umount } = this.options;
     if (!mount || !umount) throw new KernelError("ENOSYS");
     try {
       if (req.op === "umount") await umount(req.target, req.flags);
       else {
         const { source, target, type, flags, data } = req;
-        const signal = this.blockingSignal();
+        const signal = this.blockingSignal(cancelled);
         await Promise.race([mount({ source, target, type, flags, data }, signal), aborted(signal)]);
       }
     } catch (err) {
@@ -10504,7 +10543,12 @@ var WasmProcess = class {
   }
   alarm;
   dlLog = [];
-  closures = { next: 0, free: [], live: /* @__PURE__ */ new Set() };
+  closures = {
+    next: 0,
+    handle: 1,
+    free: [],
+    live: /* @__PURE__ */ new Set()
+  };
   allocateClosure(floor) {
     const index = this.closures.free.pop() ?? Math.max(floor, this.closures.next);
     this.closures.next = Math.max(this.closures.next, index + 1);
@@ -10517,6 +10561,47 @@ var WasmProcess = class {
     this.closures.free.push(index);
     this.dlLog.push({ kind: "closure-free", index });
   }
+  dlLock = {
+    owner: 0,
+    depth: 0,
+    waiters: /* @__PURE__ */ new Set()
+  };
+  async lockDl(owner) {
+    while (this.dlLock.depth > 0 && this.dlLock.owner !== owner) {
+      if (this.dying) throw new KernelError("EINTR");
+      const interrupt = this.interrupt.signal;
+      const gone = await new Promise((resolve) => {
+        const waiter = {
+          owner,
+          wake: (ended) => {
+            this.dlLock.waiters.delete(waiter);
+            interrupt.removeEventListener("abort", recheck);
+            resolve(ended);
+          }
+        };
+        const recheck = () => waiter.wake(false);
+        this.dlLock.waiters.add(waiter);
+        interrupt.addEventListener("abort", recheck, { once: true });
+      });
+      if (gone) throw new KernelError("EINTR");
+    }
+    this.dlLock.owner = owner;
+    this.dlLock.depth++;
+  }
+  threadEnded(tid) {
+    for (const waiter of [...this.dlLock.waiters]) if (waiter.owner === tid) waiter.wake(true);
+    if (this.dlLock.depth === 0 || this.dlLock.owner !== tid) return;
+    this.dlLock.depth = 0;
+    this.wakeDl();
+  }
+  unlockDl(owner) {
+    if (this.dlLock.depth === 0 || this.dlLock.owner !== owner) throw new KernelError("EPERM");
+    this.dlLock.depth--;
+    if (this.dlLock.depth === 0) this.wakeDl();
+  }
+  wakeDl() {
+    for (const waiter of [...this.dlLock.waiters]) waiter.wake(false);
+  }
   reserveSlots(floor, size) {
     const base = Math.max(floor, this.closures.next);
     this.closures.next = base + size;
@@ -10828,6 +10913,7 @@ function spawnWasmProcess(opts) {
   const endThread = (tid) => {
     const end = threads.get(tid);
     threads.delete(tid);
+    process.threadEnded(tid);
     end?.();
   };
   const spawnThread = (thread) => {