@ai-ecoverse/slicc-kernel 1.159.1 → 1.159.3

package/README.md

@@ -422,6 +422,7 @@ The module's default export (or its `main` export) is called with a context and
 - `asyncContext` (the Node entry only; absent in a browser, so a runtime feature-detects it; #1350, slicc-jsh#375) is `{ run(store, fn, ...args), getStore(), enterWith(store) }`, backed by one host `AsyncLocalStorage` the process worker makes for each program, since a program can't import `node:async_hooks` and the realm has no promise hooks of its own. A store follows the program's native `await` continuations and promise jobs, as an `AsyncLocalStorage` does in Node, so Next.js, OpenTelemetry and request-context loggers keep their context. Each process and each thread has its own. Only these three frozen functions on a frozen null-prototype object reach the program: they hold the host storage in a closure, `run` calls the program's function with its own arguments, and nothing they return or expose is a host object.
 - `sync.runScript(source, { filename })` (#1541, slicc-jsh#977) runs `source` as a classic script in the program's own global scope, synchronously, as Node's `vm.runInThisContext` does: `let`, `const` and `class` land in the global lexical scope, `var` (in strict code too) on the global object, and stack frames are named `filename` (default `[eval]`). On the Node entry it returns `{ value, completion: true }` with the completion value, and an error the script throws is thrown as it is. In a browser, where process workers are module workers with no synchronous classic-script path, it returns `undefined`, so a runtime feature-detects it with `ctx.sync.runScript('1') !== undefined`. A non-string `source` or `filename` is a `TypeError` everywhere. The script sees exactly the globals the program sees, `import()` inside it is refused, and a `var`, function or `let` declaration can't replace a global the program can't assign: a non-configurable one stays as it is, and the only configurable, non-assignable ones (`crypto`, `navigator`) are ones the program can already redefine with `Object.defineProperty`.
 - `sync.promiseState(promise)` (#1531, slicc-jsh#974) tells a native promise's state synchronously, as Node's `util.inspect` does: `{ state: 'pending' | 'fulfilled' | 'rejected', value }`. `value` is the very fulfillment value or rejection reason (`undefined` while pending), and probing doesn't handle a rejection. On the Node entry it reads the promise through an inspector session the process worker opens on first use, one per process and per thread; the probe runs no program code. Anything that isn't a native promise (a thenable, a proxy) is a `TypeError`. It answers `undefined` in a browser, where a worker has no inspector, on a Node built without one, once a program has changed `Object.prototype` or put a `toJSON` on a built-in prototype, and when the global object can't take a new property (`Object.preventExtensions(globalThis)`, or a frozen global).
+- **Kernel frames in a program's stacks** (#1583): on the Node entry, the process worker's bundle names itself `slicc-kernel:/node-process-worker.js` (a `sourceURL`), so the kernel's own frames in an error a program sees (its own errors, a failed `ctx` call, `runScript`) show that name and not the host directory the kernel is installed in. The same holds in `CallSite.toString()` and `getScriptNameOrSourceURL()`, which a program's own `Error.prepareStackTrace` gets; `getFileName()` still returns the bundle's file URL. Line and column numbers are unchanged, so they still point into `dist/node-process-worker.js`. Don't run an embedder with `--enable-source-maps` if program-visible paths matter: Node then maps the kernel's frames back to the absolute paths of its sources. In a browser the frames keep the bundle's URL, because Chrome ignores `sourceURL` in a module worker loaded by URL.
 - `memory()`, synchronous and without a kernel call, measures the calling isolate: each `thread` answers for its own (#1240, slicc-jsh#397). It returns a fresh plain object of numbers in bytes, each field only where it's measured: `heapUsed`, `heapTotal`, `heapLimit` (V8's heap size limit), `external`, `arrayBuffers` and `rss`. On the Node entry they come from V8's heap statistics and Node's memory usage, through functions the worker captured before its lockdown, so nothing reachable from the program leads back to `process` or `v8`. `rss` is the host process's resident set size, as a Node worker thread reports it in `process.memoryUsage().rss`: it isn't per process, but shared by every process and thread of the kernel, and by any other kernel in that host process (#1563, slicc-jsh#539). In a browser `memory()` answers `{}` for now, with no `rss`.
 - **`thread`, kernel threads of the same process** ([#1094](https://github.com/ai-ecoverse/slicc-kernel/issues/1094)), the base for Node's `worker_threads` (tinypool, piscina, vitest's and jest's thread pools, eslint's concurrency). `thread.spawn({ data, transfer })` resolves with `{ tid, port, exited, terminate() }`.
   - **What a thread is:** a new process worker in the **same process**. It has the same pid, descriptor table, cwd, umask, credentials and lockdown, and a sync lane of its own, so `sync` calls in one thread don't hold another.
@@ -462,7 +463,7 @@ The module's default export (or its `main` export) is called with a context and
 - `wait(pid, { untraced })` resolves with `{ status }` once a child ends, or `{ status: 128 + signal, signal }` when a signal ended it, without blocking the program; with `untraced: true` it also resolves when the child stops, with `{ status: 128 + signal, signal, stopped: true }`, as `waitpid` with `WUNTRACED` (a child's `wait({ untraced })` is the same), and options that aren't an object are `EINVAL`; `kill(pid, signal)` signals any process, by number or as `'SIGTERM'` (the default);
 - `sudo(args)`, what the [`sudo`](#sudo) command calls: `args` are its arguments after `sudo`, and it resolves with `{ pid }`, the command the kernel started as the program's child, to `wait` for; `{ run }`, an argv to run as the caller (with `users: 'off'`); `{ list }`, the text of `sudo -l`; or `{ error, errno }`, the message to print;
 - `crontab({ action, user?, text? })`, what the [`crontab`](#cron) command calls: `action` is `'list'`, `'remove'` or `'install'` (with the table as `text`), on the caller's own table or, for an effective uid of 0, on `user`'s. It resolves with `{ text }` (the table for `list`, `''` otherwise) or `{ error }`, the message to print. A malformed call is `EINVAL`;
-- `net`: `listen({ port, host, backlog })` and `connect({ host, port, signal })` (`host` defaults to `127.0.0.1`) on the kernel's loopback, the same sockets WASIX programs use, so a JS server is reached by other processes, by the page as `<port>.kernel.localhost` (`loopbackFetch`, `dial`) and by attached clients. A listener has its `port` (as bound, so port `0` picks a free one), `accept({ signal })`, `close()` and `for await` over its connections. A connection has `local` and `remote` addresses, `readable` and `writable` web streams (closing the writable shuts down the sending side, so the peer reads the end), `close()`, and `reset()`, which closes it with an RST as Node's `resetAndDestroy()` does ([#1530](https://github.com/ai-ecoverse/slicc-kernel/issues/1530)): on the kernel's loopback, the peer's pending and later reads and its writes fail with `ECONNRESET` instead of seeing the end, and bytes still in flight to it are dropped. A TLS connection (`secure()`, a `tls` connect on the loopback) resets the same way. On an uplink connection `reset()` closes at once with a FIN, as the transport can't send an RST: bytes still buffered for the peer are discarded instead of sent. The loopback is IPv4 (`127.0.0.1` by default, or `0.0.0.0`), so an IPv6 host such as `::1` fails with `EADDRNOTAVAIL` for both `listen` and `connect`, and binding and connecting fail as they would for a WASIX program (`EADDRINUSE`, `ECONNREFUSED`, `EADDRNOTAVAIL`). `sync.net.listen` takes the same options and gives the same listener, bound and listening when it returns, so its `port` is known at once, as Node's `listen()` needs ([#1398](https://github.com/ai-ecoverse/slicc-kernel/issues/1398)); it needs the sync lane. A blocked accept, read or write waits without holding anything in the kernel, so aborting it or closing the socket leaves nothing behind. A write to a connection the peer closed fails with `EPIPE` and no `SIGPIPE`;
+- `net`: `listen({ port, host, backlog })` and `connect({ host, port, signal })` (`host` defaults to `127.0.0.1`) on the kernel's loopback, the same sockets WASIX programs use, so a JS server is reached by other processes, by the page as `<port>.kernel.localhost` (`loopbackFetch`, `dial`) and by attached clients. A listener has its `port` (as bound, so port `0` picks a free one), `accept({ signal })`, `close()` and `for await` over its connections. A connection has `local` and `remote` addresses, `readable` and `writable` web streams (closing the writable shuts down the sending side, so the peer reads the end), `close()`, and `reset()`, which closes it with an RST as Node's `resetAndDestroy()` does ([#1530](https://github.com/ai-ecoverse/slicc-kernel/issues/1530)): on the kernel's loopback, the peer first reads the bytes it has already received, as on Linux and BSD ([#1577](https://github.com/ai-ecoverse/slicc-kernel/issues/1577)), and then its reads fail with `ECONNRESET` instead of seeing the end; its writes fail with `ECONNRESET` at once. A TLS connection (`secure()`, a `tls` connect on the loopback) resets the same way. On an uplink connection `reset()` closes at once with a FIN, as the transport can't send an RST: bytes still buffered for the peer are discarded instead of sent. The loopback is IPv4 (`127.0.0.1` by default, or `0.0.0.0`), so an IPv6 host such as `::1` fails with `EADDRNOTAVAIL` for both `listen` and `connect`, and binding and connecting fail as they would for a WASIX program (`EADDRINUSE`, `ECONNREFUSED`, `EADDRNOTAVAIL`). `sync.net.listen` takes the same options and gives the same listener, bound and listening when it returns, so its `port` is known at once, as Node's `listen()` needs ([#1398](https://github.com/ai-ecoverse/slicc-kernel/issues/1398)); it needs the sync lane. A blocked accept, read or write waits without holding anything in the kernel, so aborting it or closing the socket leaves nothing behind. A write to a connection the peer closed fails with `EPIPE` and no `SIGPIPE`;
 - `net.connect({ host, port, tls: { servername, alpn, rejectUnauthorized, ca, path, cert, key, passphrase, sni, checkHostname, keylog } })` takes one of two paths. By default it's **A1, TLS through the transport** ([#914](https://github.com/ai-ecoverse/slicc-kernel/issues/914)), described first below. For a destination the uplink routes it can also be **A2, a TLS handshake in the kernel** ([#922](https://github.com/ai-ecoverse/slicc-kernel/issues/922)), described after it. `conn.tls.path` (`'transport'` or `'kernel'`) and `conn.tls.substitutes` say which one was taken.
   - **What it does:** there's no handshake in the kernel. The connection carries plain HTTP/1.1 (and WebSocket upgrades) to the realm proxy's HTTP path for the origin `https://<servername or host>[:port]`, exactly as a `CONNECT` through `127.0.0.1:3128` does after the kernel terminates its TLS. The transport (the page's `fetch` or WebSocket, or the uplink's) makes the real TLS connection and verifies it. So `https.request` over a socket, keep-alive agents, `agent-base` and `wss://` clients work with the transport's semantics. That includes sealed placeholders in headers, which reach the transport exactly as from `ctx.fetch`.
   - **The `tls` property:** the connection's `tls` is `{ path: 'transport', substitutes: true, protocol: 'transport', alpnProtocol: 'http/1.1', authorized: true, servername }`. `authorized` means the transport verified the server. There's no peer certificate chain and no TLS version of the kernel's own, so don't read it as a real handshake's result.
@@ -844,6 +845,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 - **Exclusive create across tabs** ([#912](https://github.com/ai-ecoverse/slicc-kernel/issues/912)): two kernels on one OPFS root (two tabs, or a tab and a worker page) agree on `O_CREAT|O_EXCL` and `set -C`: exactly one wins a name. While another kernel on the root is present, the create runs under a Web Lock for its path and checks the OPFS entry itself. A kernel alone on its root takes no lock. Each kernel holds an alive Web Lock, granted before it first looks for peers. A kernel that finds peers waits for their answer (at most 500 ms) before its first exclusive create, and a kernel answers a newcomer only once its unlocked creates are done. So of two kernels that boot at the same moment, the later one to look sees the other, and no unlocked create overlaps a locked one.
 - **A name another process or tab just removed** ([#950](https://github.com/ai-ecoverse/slicc-kernel/issues/950)): sqlite3 deletes its `-journal` at every commit and the next writer creates it again, so two writers (two processes in one kernel, or two tabs on one OPFS root) keep removing and re-creating one name. Two things used to turn that into `unable to open database file`. The root removes a file's sidecar record a few milliseconds after the file itself (removals are batched), and an exclusive create read that pending record as the name being taken: a plain record whose file is gone now counts as gone, and the new file gets a record of its own. A symlink, which lives only in the sidecar, still takes its name. And an Emscripten program keeps the nodes it has looked up, so after another tab removed a name it still held the old node, and `open(O_CREAT)` opened the gone file instead of creating one. Now, when the kernel answers `ENOENT` for a cached node, for `open`, `stat`, `lstat`, `readlink`, `truncate`, `unlink` or a rename's source, or the program's own lookup answers `EEXIST` to `open(O_CREAT|O_EXCL)`, the node is dropped and the call runs once more, against what is there now. A node the program has open or has unwritten changes in is kept. A rename over a name another tab removed needs nothing: the kernel's rename replaces whatever is there.
 - **One file, several processes** ([#665](https://github.com/ai-ecoverse/slicc-kernel/issues/665)): a process that keeps a file open sees what another process writes to it. Every change the kernel applies to a file (a write back, a truncate, an append, a rename or a removal) bumps a change ring the kernel shares with all its processes. Before each read, write or seek to the end, a process's open file checks the ring (one atomic load) and, if its file changed, drops its clean pages and reads the size again, keeping its own unwritten changes. A write by one process is visible to another one's next read once it has gone back to the kernel: at the latest at the writer's next `fsync`, close, exit or call into the kernel (see above), so sqlite, which `fsync`s every commit, sees another connection's commit at its next transaction. A write through a file the kernel holds (a JS program's `ctx.open`, a WASI program's file after a fork or spawn) is in the kernel already, so other processes see it at their next read, before it is written back. A file the kernel itself holds open (a JS program's file, a WASI program's file after a fork or spawn) is the one copy: reads, writes, truncates and appends by path from other processes go to it, and their writes go through to storage at once, so a commit is in storage (and in `kernel.readFile`) when the committing process returns from `fsync`. `O_APPEND` writes go to the kernel at once, and the kernel picks the offset, so appenders in several processes keep every line. A file system without ranged I/O (`hostfs` with an older proxy, a root outside OPFS) rereads a file another process changed, but writes it back whole, so two writers there still need locks. The same file under two names is one file too ([#669](https://github.com/ai-ecoverse/slicc-kernel/issues/669)): a write by path through a symlink reaches the file the kernel holds under its real path (resolved without following links in a `sudo` process's no-links view, so an untrusted link is still refused), a WASI program's open file also watches its real path (resolved once, the first time another file changes), and an open file with more than one hard link checks its size, version and mtime when anything changed, so a write through another name is seen. Limits for now: an alias on a mount that has no `realpath` is still only the name it was opened by; another tab's writes reach this tab's processes only as far as the root's change notifications (OPFS's `BroadcastChannel` watch) carry them. A by-path `stat` or `lstat` of a file the kernel holds open has its current size, writes not yet written back included ([#780](https://github.com/ai-ecoverse/slicc-kernel/issues/780)). Its mtime is the file system's until write-back, though, so a tool that compares times, like `make`, may see a just-written file as older than it is for up to the write-back delay.
+- **Appends across tabs** ([#1520](https://github.com/ai-ecoverse/slicc-kernel/issues/1520)): while another kernel shares the OPFS root, an `O_APPEND` write lands at the file's real end under a per-file Web Lock, so two tabs' appends never overwrite each other. A tab that sees another join flushes its buffered appends and waits for the writes it has under way at that moment (later ones don't hold it) before it answers, and the joiner's appends wait for every such answer pending at the time, without a timeout, as long as the other tab lives; an append already under way when a further tab's note arrives isn't held back ([#1546](https://github.com/ai-ecoverse/slicc-kernel/issues/1546)). Alone, an append is buffered as before. **Limit:** a write at the end *without* `O_APPEND` (a seek to the end, then a write) is close-to-open across tabs, as on NFS: it can overwrite another tab's appends it hasn't seen yet ([#1546](https://github.com/ai-ecoverse/slicc-kernel/issues/1546)).
 - **Locks between processes** ([#665](https://github.com/ai-ecoverse/slicc-kernel/issues/665)): an Emscripten program's `fcntl` byte-range locks (`F_SETLK`, `F_SETLKW`, `F_GETLK`) are the kernel's, and so are a WASIX program's through the `slicc.fcntl_lock` import (below), so sqlite3 connections in different processes exclude each other: a second writer gets `database is locked` while the first holds its transaction, or waits with a busy timeout. The kernel keeps one table keyed by file identity (device and inode, so every name of a hard link is the same file), with POSIX semantics: locks belong to the process (an `exec`'d program keeps them under its pid; a forked child doesn't get them), a conflicting `F_SETLK` fails with `EAGAIN`, `F_GETLK` reports the holder's pid and range, `F_SETLKW` waits in the kernel without holding up anything else and a signal ends the wait with `EINTR`, a wait that would close a cycle between waiting processes fails with `EDEADLK`, a write lock on a descriptor not open for writing (or a read lock on one not open for reading) is `EBADF`, closing any descriptor of the file drops the process's locks on it, and exit releases them all. Before an unlock, the process writes back what it changed, so the next holder reads it. An Emscripten program's file goes to a kernel descriptor at its first lock. From then on, its `fstat` size is that descriptor's, so on a file that isn't read and written in pages (a root outside OPFS, or `hostfs` without `ranges`), sqlite sees the pages it wrote and the kernel still holds (#1019). An open file's directory entry is checked against the change ring too: a name another process removed or replaced is looked up again, so sqlite's check for a hot journal sees the other connection's commit. A WASI imports module's `tryLock`/`unlock` (wasi-pnpm's store-operation lock) is a whole-file lock, per process and keyed by path, while the kernel is alone on its root. While another kernel shares the root, it becomes a non-blocking `flock` on the descriptor's open file instead, keyed by the file's identity, so it excludes across kernels and tabs (#983). It's released on `unlock`, on close, and about a second after the holder's tab closes. A tryLock checks both the local table and the flock table, so a lock taken before a peer joins, or while one was there, still excludes after the switch. Limits: locks are per kernel, not across tabs ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)); and a WASIX program built against a wasix-sysroot without the import keeps its libc's local answer (every lock granted), so python's sqlite excludes other processes only once wasix-python is rebuilt on the sysroot that has it ([homescoop#407](https://github.com/ai-ecoverse/homescoop/issues/407)). An Emscripten program's `flock()` reaches the kernel's `flock` table (per open file description) through homescoop's shim, a strong `flock()` that calls the import `env.__syscall_flock(fd, op)`. The kernel wraps that import by name (#947): `LOCK_SH`/`LOCK_EX`/`LOCK_UN` with `LOCK_NB` (`EWOULDBLOCK` for a conflict), 0 or -errno. A glue without the shim keeps Emscripten's stub, which answers 0. A lock belongs to the file its descriptor was opened on: a file renamed over the name later, or the name's removal, doesn't move it.
 - **Locks across tabs** ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)): two kernels on one OPFS root (two tabs, or a tab and a worker page) share one table of `fcntl` record locks and `flock` locks, so a sqlite3 in one tab finds the database locked while a transaction is open in another. **Known issue, data corruption: lock-protected writes across tabs** ([#949](https://github.com/ai-ecoverse/slicc-kernel/issues/949)). The locks are enforced across tabs, but what one tab writes under a lock isn't yet guaranteed to be what the other tab reads once it holds the lock, so a lock-protected read-modify-write across tabs can lose or corrupt data. Don't rely on cross-tab locking until #949 is fixed. Use within a single tab is unaffected.
 - **Locks between processes** ([#665](https://github.com/ai-ecoverse/slicc-kernel/issues/665)): an Emscripten program's `fcntl` byte-range locks (`F_SETLK`, `F_SETLKW`, `F_GETLK`) are the kernel's, and so are a WASIX program's through the `slicc.fcntl_lock` import (below), so sqlite3 connections in different processes exclude each other: a second writer gets `database is locked` while the first holds its transaction, or waits with a busy timeout. The kernel keeps one table keyed by file identity (device and inode, so every name of a hard link is the same file), with POSIX semantics: locks belong to the process (an `exec`'d program keeps them under its pid; a forked child doesn't get them), a conflicting `F_SETLK` fails with `EAGAIN`, `F_GETLK` reports the holder's pid and range, `F_SETLKW` waits in the kernel without holding up anything else and a signal ends the wait with `EINTR`, a wait that would close a cycle between waiting processes fails with `EDEADLK`, a write lock on a descriptor not open for writing (or a read lock on one not open for reading) is `EBADF`, closing any descriptor of the file drops the process's locks on it, and exit releases them all. Before an unlock, the process writes back what it changed, so the next holder reads it. An open file's directory entry is checked against the change ring too: a name another process removed or replaced is looked up again, so sqlite's check for a hot journal sees the other connection's commit. A WASI imports module's `tryLock`/`unlock` keeps its behaviour (whole file, per process, keyed by path). Limits: locks are per kernel, not across tabs ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)); and a WASIX program built against a wasix-sysroot without the import keeps its libc's local answer (every lock granted), so python's sqlite excludes other processes only once wasix-python is rebuilt on the sysroot that has it ([homescoop#407](https://github.com/ai-ecoverse/homescoop/issues/407)). An Emscripten program's `flock()` is a stub compiled into its libc that never reaches the kernel either, until a homescoop shim routes it to a kernel import ([#680](https://github.com/ai-ecoverse/slicc-kernel/issues/680)). A lock belongs to the file its descriptor was opened on: a file renamed over the name later, or the name's removal, doesn't move it.