@ai-ecoverse/slicc-kernel 1.129.0 → 1.130.1

package/README.md

@@ -388,7 +388,7 @@ The module's default export (or its `main` export) is called with a context and
   - The child is in the program's process group, so `^C` reaches it. The kernel reaps it, so it leaves no zombie, and its CPU time counts as the program's children's. A signal the program catches does not cut the run short; its handler runs when the call returns. A signal that kills the program kills it as usual;
 - `sync.spawn({ argv, argv0, env, cwd, pgid, session, parentDeathSignal, stdin, stdout, stderr, fds })` starts a child as `spawn` below does, with the same options and the same child, but synchronously: it returns `{ pid, stdin, stdout, stderr, wait(), kill(signal) }` at once, so a runtime can set Node's `child.pid` before `spawn()` returns. A spawn error (`ENOENT`, `EACCES`, `ESRCH` or `EPERM` for a group, `EINVAL` for a bad option, `EBADF` for a descriptor the program doesn't have) is thrown at once, with its `code`. The streams, `wait()` and `kill()` are the asynchronous ones, used afterwards from async code; the child doesn't wait for the program to read, since its output goes to a kernel pipe, which holds what it writes until the program reads it (a child that fills the pipe waits, as with `spawn`);
 - `sync.kill(pid, signal)`, a synchronous `kill`. When the signal reaches the program itself (its own pid, `0` for its process group, or `-pgid` of its own group) and its action is to end the program (no handler, not ignored), the call never returns: the program's writes reach their descriptors and its terminal modes are put back, as for `sync.exit`, then the rest of the group is signalled and the program ends by the signal, so a parent's `wait` and `sync.run` report `signal`, as Node does for `process.kill(process.pid)`. A signal with a handler runs it when the call returns, an ignored one does nothing, and a stop signal stops the program. A signal whose default action dumps core (`SIGQUIT`, `SIGABRT`, `SIGSEGV`, …) ends it the same way; the kernel writes no core and sets no core flag in the wait status. `-1` signals every process the program may signal except itself and pid 1, as on Linux, and returns (`ESRCH` when there was none, `EPERM` when there were some but it may signal none of them); a process that exec'd gets the signal once. Without users, every process runs as root, so `kill -9 -1` from any program ends every other process; with users it reaches only what the sender may signal (its own uid's processes, or all of them as root);
-- `sync.exit(status)`, which ends the process at once, as Node's `process.exit()` does: no `catch`, `finally`, microtask, timer or signal handler runs after it, because the call never returns. First the writes the program has made reach their descriptors, in order (a broken pipe drops the rest for that descriptor), and terminal modes the program set with `tty` are put back. Then the process ends the way any exit does: its descriptors are closed and written back, its parent gets `SIGCHLD`, and `wait` reports `status`. Children keep running. A JS program has no threads of its own (`Worker` is withheld), so nothing of it outlives the call. `exit(status)` (below) unwinds instead, so the program's `catch` and `finally` blocks run;
+- `sync.exit(status)`, which ends the process at once, as Node's `process.exit()` does: no `catch`, `finally`, microtask, timer or signal handler runs after it, because the call never returns. First the writes the program has made reach their descriptors, in order (a broken pipe drops the rest for that descriptor), and terminal modes the program set with `tty` are put back. Then the process ends the way any exit does: its descriptors are closed and written back, its parent gets `SIGCHLD`, and `wait` reports `status`. Children keep running. In the main thread it ends the process with all its threads (`Worker` itself is withheld; threads come from `thread.spawn`, below); in a thread it ends that thread only. `exit(status)` (below) unwinds instead, so the program's `catch` and `finally` blocks run;
 - `select({ read?, write?, timeoutMs? }, { signal }?)` waits until a descriptor of either list is readable or writable, and answers `{ read, write }` with the ready ones; `timeoutMs` (default: no limit) bounds the wait, and `0` only checks. After a nonblock write's `EAGAIN`, `select({ write: [fd] })` says when room returns ([#1124](https://github.com/ai-ecoverse/slicc-kernel/issues/1124)).
 - `socketpair()` (and `sync.socketpair()`) answers `[a, b]`, two descriptors of the program on a connected `AF_UNIX` stream pair, and `pipe()` (and `sync.pipe()`) answers `[readFd, writeFd]` ([#1093](https://github.com/ai-ecoverse/slicc-kernel/issues/1093)).
   - **The same descriptors either way:** the async and `sync` forms give the program's own kernel descriptors, which `read`, `write`, `close` and their `sync` forms all take, until the program closes them or ends.
@@ -399,6 +399,24 @@ The module's default export (or its `main` export) is called with a context and
   - **Only the named descriptors reach the child.** Others the program holds, the other end of a pair included, don't, and a grandchild gets fd 3 only if the child names it again.
   - **The parent's copy:** the program keeps `f` and closes it itself when it no longer needs it, so the child sees end of file once it's the only one left.
   - **Why a `Record`:** it can't name a child fd twice, and it reads like Node's `stdio` index, so it's this rather than a list of `{ fd, from }`.
+- **`thread`, kernel threads of the same process** ([#1094](https://github.com/ai-ecoverse/slicc-kernel/issues/1094)), the base for Node's `worker_threads` (tinypool, piscina, vitest's and jest's thread pools, eslint's concurrency). `thread.spawn({ data, transfer })` resolves with `{ tid, port, exited, terminate() }`.
+  - **What a thread is:** a new process worker in the **same process**. It has the same pid, descriptor table, cwd, umask, credentials and lockdown, and a sync lane of its own, so `sync` calls in one thread don't hold another.
+  - **What it runs:** the program's same default export, called with a context whose `thread` is `{ tid, port, data }`. In the main thread `thread.tid` is `0` and `thread.port` is `undefined`. Threads are numbered from 1 per process, and a thread can spawn threads too.
+  - **The port** is a native `MessagePort`, one end of the platform's `MessageChannel`, which the kernel keeps from before the program runs, so a runtime that replaces the global `MessageChannel` (as a Node runtime does) still gets real ports. Messages are structured clones: a `SharedArrayBuffer` is shared (`Atomics.wait` and `Atomics.notify` work across threads), and what is listed in `transfer` (an `ArrayBuffer`, a `MessagePort`) moves and is detached in the sender. `data` is cloned the same way, with `transfer`.
+  - **Ending:**
+
+    | How | `exited` |
+    |---|---|
+    | `exit(n)` or `sync.exit(n)` in the thread (only that thread ends) | `n` |
+    | the thread's function returns `n` | `n` (nothing is `0`) |
+    | an uncaught error (the kernel prints nothing: a runtime reports it, as Node does with the `Worker`'s `error` event) | `1` |
+    | `terminate()`, which resolves once the thread has ended | `1` |
+    | out of memory (`RangeError: Array buffer allocation failed`, or the worker dying of it) | `thread.NO_MEMORY`, `256` |
+    | the worker died otherwise (nothing is printed either; the runtime reports it) | `thread.CRASHED`, `257` |
+
+    The two codes are outside 0–255, so no `exit(n)` gives them. The main thread's exit (any exit, or a signal that ends the process) ends every thread.
+  - **The process:** a thread ending changes nothing for `wait`, `ps` or the process's status. Signals go to the main thread only, so a thread's `signals.on`, `ignore` and `reset` change nothing (as Node, where a worker gets no signal events); the main thread's disposition is the process's. Kernel calls a thread leaves in flight when it ends (a `read` it never awaited, say) are cancelled, so they can't take another thread's data. A stopped process (`^Z`, `SIGSTOP`) stops its threads as well, timers included, until `SIGCONT`.
+  - **The cap:** at most twice `navigator.hardwareConcurrency` threads per process at once, and at least 16 (so piscina's default of 1.5 × the parallelism fits), or what `SLICC_JS_THREADS` in the program's environment says, up to 256. Past it, `spawn` rejects with `EAGAIN`, and an ended thread frees its slot. Each thread is a dedicated worker, like a process worker, so threads count against the browser's worker budget the same way processes do.
 - `spawn({ argv, argv0, env, cwd, pgid, session, parentDeathSignal, stdin, stdout, stderr, fds })` starts a command (found as `kernel.run` finds it) as a child process and resolves with `{ pid, stdin, stdout, stderr, wait(), kill(signal) }`. Each stdio is `'inherit'` (the default: the program's own), `'pipe'` (a web stream on a kernel pipe), `'null'`, or one of the program's descriptors. `env` is the child's whole environment (the program's by default), and `cwd` defaults to the program's. The command run is `argv[0]`; `argv0`, when given, is the `argv[0]` the child sees instead, as given (no basename; an empty string too), as with Node's `options.argv0`: `spawn({ argv: ['/bin/sh', '-c', 'echo $0'], argv0: '/bin/sh' })` prints `/bin/sh`, where without it the child sees the command's own name (`sh`). A command whose package manifest sets its `argv0` (a multi-call program's applets, such as coreutils') keeps that one. An `argv0` that isn't a string is `EINVAL`. The child is in the program's process group and session, as after `fork` (Node's `detached: false`), so the terminal's `^C` and `^Z` reach it along with the program. `pgid: 0` makes it lead a new group in the program's session, set before it runs, so a `kill(-pid)` right after the call reaches it and everything it starts (`npx`, then `node`); on the program's terminal that group is in the background, so a read from the terminal stops it with `SIGTTIN` (`EIO` when it ignores `SIGTTIN`), as on Linux. `pgid: N` joins group N, which must be in the program's session: `EPERM` for another session's group, `ESRCH` when there is none. `session: true` makes it lead a new session and a new group (`setsid`, what Node's `detached: true` does on POSIX): its pid is its `pgid` and its `sid`, it has no controlling terminal (`/dev/tty` is `ENXIO`) and the terminal's keys don't signal it. `session` together with `pgid`, or a `pgid` that isn't a non-negative integer, is `EINVAL`. A command that does not exist rejects with `ENOENT`. `parentDeathSignal` (a number or `'SIGTERM'`, as Linux's `PR_SET_PDEATHSIG`) is sent to the child when the program ends, killed or not: to the child's whole group when it leads one (`pgid: 0` or `session: true`), else to the child alone. It's sent with the program's credentials, so a child the program may no longer signal (one that gained privilege) doesn't get it. If the program has already ended by the time the spawn completes, it's sent at once. A child of its own (not a group leader) that has been waited for gets nothing, and `sync.run` has no such option. A bad signal is `EINVAL`;
 - `wait(pid, { untraced })` resolves with `{ status }` once a child ends, or `{ status: 128 + signal, signal }` when a signal ended it, without blocking the program; with `untraced: true` it also resolves when the child stops, with `{ status: 128 + signal, signal, stopped: true }`, as `waitpid` with `WUNTRACED` (a child's `wait({ untraced })` is the same), and options that aren't an object are `EINVAL`; `kill(pid, signal)` signals any process, by number or as `'SIGTERM'` (the default);
 - `sudo(args)`, what the [`sudo`](#sudo) command calls: `args` are its arguments after `sudo`, and it resolves with `{ pid }`, the command the kernel started as the program's child, to `wait` for; `{ run }`, an argv to run as the caller (with `users: 'off'`); `{ list }`, the text of `sudo -l`; or `{ error, errno }`, the message to print;
@@ -722,7 +740,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
   - **The same file:** a lock is keyed by the file's device and inode, which both kernels read from the shared sidecar, so a rename by either tab keeps it. A file removed and created again at the same path has the same key, as it already has within one kernel.
   - **Versions:** the channel and Web Lock names carry a protocol version (`v1`), and a kernel ignores messages that aren't v1. So kernels on different protocol versions (two tabs across an update) don't share locks, as before #675.
   - **Not across tabs:** the imports module's whole-file `tryLock` (per kernel), and locks on other mounts.
-- **Another tab's writes** ([#959](https://github.com/ai-ecoverse/slicc-kernel/issues/959)): a program's names and stats are cached in its process, and its kernel refreshes them when the kernel itself changes a file. While another kernel shares the root (a second tab), the kernel marks that view foreign, and a program then looks names up and reads stats again on every look. So a program that keeps checking a file sees what the other tab wrote, synced and closed. A kernel alone caches as before. Stats are always read fresh while the root is shared. An Emscripten program keeps the names it has looked up, directories and files ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)), until its next call that isn't by path: a read, a write, a poll, a sleep, a wait, a spawn, a look at the clock or a draw of random bytes. Within one run of path-only calls (lstat, chmod, readlink and the like) with none of those in between, an answer the program takes from its own node, such as the node's type, can lag behind the other tab until that next call, as a path call racing a rename does on Linux. A program can only learn that the other tab acted through one of those calls, a new process or a stat, and all of them refresh what it sees. While another kernel shares the root, a kernel also tells it of each file it writes ([#962](https://github.com/ai-ecoverse/slicc-kernel/issues/962)). The other kernel then drops the clean contents of that file, both its own and its programs', so a descriptor tab B already has open reads tab A's new bytes on its next read, as Linux's shared page cache does. The notice is a message, so a read that races it by less than a message's delivery can still see the old bytes, as with any reader that doesn't lock. The whole cross-tab contract is under "What another tab sees, and when".
+- **Another tab's writes** ([#959](https://github.com/ai-ecoverse/slicc-kernel/issues/959)): a program's names and stats are cached in its process, and its kernel refreshes them when the kernel itself changes a file. While another kernel shares the root (a second tab), the kernel marks that view foreign. A WASI program then keeps a stat or a missing name it looked up by path for at most 100 ms after the kernel answered it, however often it looks in between ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). Every notice from another kernel, a lock grant, and a kernel joining or leaving the root (also one that ends without a word) drop everything its programs cached. An answer that a notice overtook on its way is not kept. An open never uses what the program cached (close-to-open), and calls on a descriptor (`fstat`, `pread`) go to the kernel, which holds them to its own 2 ms. So a program that keeps checking a file sees what the other tab wrote, synced and closed, once the notice has arrived, and within 100 ms if the notice is late or lost; one that opens the file sees it at once. A kernel alone caches as before. An Emscripten program keeps the names it has looked up, directories and files ([#984](https://github.com/ai-ecoverse/slicc-kernel/issues/984)), until its next call that isn't by path: a read, a write, a poll, a sleep, a wait, a spawn, a look at the clock or a draw of random bytes. Within one run of path-only calls (lstat, chmod, readlink and the like) with none of those in between, an answer the program takes from its own node, such as the node's type, can lag behind the other tab until that next call, as a path call racing a rename does on Linux. A program can only learn that the other tab acted through one of those calls, a new process or a stat, and all of them refresh what it sees. While another kernel shares the root, a kernel also tells it of each file it writes ([#962](https://github.com/ai-ecoverse/slicc-kernel/issues/962)). The other kernel then drops the clean contents of that file, both its own and its programs', so a descriptor tab B already has open reads tab A's new bytes on its next read, as Linux's shared page cache does. The notice is a message, so a read that races it by less than a message's delivery can still see the old bytes, as with any reader that doesn't lock. The whole cross-tab contract is under "What another tab sees, and when".
 - **`slicc.fcntl_lock(fd, cmd, flock_ptr) -> errno`** ([#722](https://github.com/ai-ecoverse/slicc-kernel/issues/722)): a WASIX program's `fcntl` record locks reach the same table as an Emscripten program's, so the two see each other's locks.
   - **cmd:** `12` `F_GETLK`, `13` `F_SETLK` or `14` `F_SETLKW`. Any other cmd is `EINVAL`.
   - **The struct:** musl's wasm32 `struct flock`, 32 bytes: `l_type` i16 at 0 (`0` read, `1` write, `2` unlock), `l_whence` i16 at 2 (`0` SET, `1` CUR, `2` END, resolved against the descriptor's offset or size), `l_start` i64 at 8, `l_len` i64 at 16 (`0` to EOF and beyond, negative before `l_start`), `l_pid` i32 at 24. The kernel reads and writes only those five fields and never the padding.
@@ -1079,7 +1097,7 @@ A kernel caches the directory handles it has walked and the sidecar entries it h
   - an exclusive create (`O_EXCL`, `mkdir`, `symlink`, `link`) and a rename's check of its target decide against OPFS itself, never against a cached missing name ([#942](https://github.com/ai-ecoverse/slicc-kernel/issues/942));
   - the join handshake writes back every open file before a new kernel counts as a peer, and a change in whether the root is shared drops both 2 ms windows.
 
-So a locked read-modify-write across tabs (sqlite, a JSON counter under `flock`) is exact, and anything else between tabs is as fresh as the last notice. Programs add their own rule on top: see "Another tab's writes" above. `crowded-lookups.test.mjs` and `notice-caches.test.mjs` hold the rows. The `notice-caches` rows freeze one kernel's clock and drop the other's notices, through the kernel workers in `test/integration/page/notice/`.
+So a locked read-modify-write across tabs (sqlite, a JSON counter under `flock`) is exact, and anything else between tabs is as fresh as the last notice. The same holds for what a WASI program caches in its own process: a notice, a lock grant or a peer joining or leaving drops all of it, and a late or lost notice leaves an unlocked path stat at most 100 ms behind the kernel's answer; opens, descriptor calls and anything under a lock are exact ([#1017](https://github.com/ai-ecoverse/slicc-kernel/issues/1017)). Emscripten programs add their own rule on top: see "Another tab's writes" above. `program-cache-tabs.test.mjs` holds the rows for programs; it freezes a program's clock through `page/notice/program-clock-worker.js`. `crowded-lookups.test.mjs` and `notice-caches.test.mjs` hold the rows. The `notice-caches` rows freeze one kernel's clock and drop the other's notices, through the kernel workers in `test/integration/page/notice/`.
 
 **Within one kernel** ([#1026](https://github.com/ai-ecoverse/slicc-kernel/issues/1026)): a WASI process keeps the stats it has read, and checks each one against the kernel's change ring at every look. So a process that keeps stat-ing a file, even in a loop that never sleeps, sees another process's change to that path at its next stat. Before #1026's fix, from v1.4.0 on, a stat it had looked up lasted until the process slept, polled, waited for a child or wrote the path itself. The ring names the changed paths only, so in a loop with none of those calls, a stat stays stale until the next sleep, poll, wait or own write when the change was named elsewhere:
 - a write through another hard link of the same file. Only tmpfs mounts support `link`; on the OPFS root and every other mount, a file's link count is always 1;