@ai-ecoverse/slicc-kernel 1.118.1 → 1.119.1

package/README.md

@@ -587,7 +587,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 - **Opening one** works as on Linux (fifo(7)). A read-open waits until a writer opens, and a write-open until a reader does. `O_RDWR` doesn't wait. With `O_NONBLOCK` a read-open returns at once, and a write-open with no reader is `ENXIO`. A waiting open is interrupted by a caught signal (`EINTR`), restarts under `SA_RESTART`, and isn't ended by a stop.
 - **The data path** is the kernel's pipe: the same capacity, `EOF` once the last writer closes, and `SIGPIPE`/`EPIPE` once the last reader closes. The data is dropped when no end is open.
 - **The node's identity, not its name, picks the FIFO:** an open FIFO keeps working after an `unlink` or a `rename`, and a new `mkfifo` at the same name is another FIFO. `/proc/<pid>/fd/N` names the FIFO's path, also in children that inherit the end and in a WASIX program's `readlink("/proc/self/fd/N")` of an end it opened itself ([#909](https://github.com/ai-ecoverse/slicc-kernel/issues/909)).
-- **One kernel per storage root:** the table of open FIFOs is per kernel, so two tabs (two kernels on one origin) opening the same FIFO don't connect, as with record locks ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)).
+- **One kernel per storage root:** the table of open FIFOs is per kernel, so two tabs (two kernels on one origin) opening the same FIFO don't connect. Record locks and `flock` do reach across tabs (see below).
 - GNU make 4.4's `-jN` uses a FIFO jobserver this way (`--jobserver-auth=fifo:PATH`), so sub-makes join it by path.
 - **Below an open** ([#908](https://github.com/ai-ecoverse/slicc-kernel/issues/908)): a FIFO or socket node is never read or written as a plain file. A JS program's `ctx.fs.readFile`/`writeFile` and the kernel API's `readFile`/`writeFile` (and the positioned reads, writes and truncates under them) fail with `ENXIO` on either, because they read or write a whole file rather than open one. Emscripten and WASI programs' opens of a FIFO go through the kernel's FIFO open as above, and a WASI program's open of a socket node is `ENXIO`, as on Linux.
 
@@ -613,6 +613,14 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 - **Exclusive create across tabs** ([#912](https://github.com/ai-ecoverse/slicc-kernel/issues/912)): two kernels on one OPFS root (two tabs, or a tab and a worker page) agree on `O_CREAT|O_EXCL` and `set -C`: exactly one wins a name. While another kernel on the root is present, the create runs under a Web Lock for its path and checks the OPFS entry itself. A kernel alone on its root takes no lock. Each kernel holds an alive Web Lock, granted before it first looks for peers. A kernel that finds peers waits for their answer (at most 500 ms) before its first exclusive create, and a kernel answers a newcomer only once its unlocked creates are done. So of two kernels that boot at the same moment, the later one to look sees the other, and no unlocked create overlaps a locked one.
 - **One file, several processes** ([#665](https://github.com/ai-ecoverse/slicc-kernel/issues/665)): a process that keeps a file open sees what another process writes to it. Every change the kernel applies to a file (a write back, a truncate, an append, a rename or a removal) bumps a change ring the kernel shares with all its processes. Before each read, write or seek to the end, a process's open file checks the ring (one atomic load) and, if its file changed, drops its clean pages and reads the size again, keeping its own unwritten changes. A write by one process is visible to another one's next read once it has gone back to the kernel: at the latest at the writer's next `fsync`, close, exit or call into the kernel (see above), so sqlite, which `fsync`s every commit, sees another connection's commit at its next transaction. A write through a file the kernel holds (a JS program's `ctx.open`, a WASI program's file after a fork or spawn) is in the kernel already, so other processes see it at their next read, before it is written back. A file the kernel itself holds open (a JS program's file, a WASI program's file after a fork or spawn) is the one copy: reads, writes, truncates and appends by path from other processes go to it, and their writes go through to storage at once, so a commit is in storage (and in `kernel.readFile`) when the committing process returns from `fsync`. `O_APPEND` writes go to the kernel at once, and the kernel picks the offset, so appenders in several processes keep every line. A file system without ranged I/O (`hostfs` with an older proxy, a root outside OPFS) rereads a file another process changed, but writes it back whole, so two writers there still need locks. The same file under two names is one file too ([#669](https://github.com/ai-ecoverse/slicc-kernel/issues/669)): a write by path through a symlink reaches the file the kernel holds under its real path (resolved without following links in a `sudo` process's no-links view, so an untrusted link is still refused), a WASI program's open file also watches its real path (resolved once, the first time another file changes), and an open file with more than one hard link checks its size, version and mtime when anything changed, so a write through another name is seen. Limits for now: an alias on a mount that has no `realpath` is still only the name it was opened by; another tab's writes reach this tab's processes only as far as the root's change notifications (OPFS's `BroadcastChannel` watch) carry them. A by-path `stat` or `lstat` of a file the kernel holds open has its current size, writes not yet written back included ([#780](https://github.com/ai-ecoverse/slicc-kernel/issues/780)). Its mtime is the file system's until write-back, though, so a tool that compares times, like `make`, may see a just-written file as older than it is for up to the write-back delay.
 - **Locks between processes** ([#665](https://github.com/ai-ecoverse/slicc-kernel/issues/665)): an Emscripten program's `fcntl` byte-range locks (`F_SETLK`, `F_SETLKW`, `F_GETLK`) are the kernel's, and so are a WASIX program's through the `slicc.fcntl_lock` import (below), so sqlite3 connections in different processes exclude each other: a second writer gets `database is locked` while the first holds its transaction, or waits with a busy timeout. The kernel keeps one table keyed by file identity (device and inode, so every name of a hard link is the same file), with POSIX semantics: locks belong to the process (an `exec`'d program keeps them under its pid; a forked child doesn't get them), a conflicting `F_SETLK` fails with `EAGAIN`, `F_GETLK` reports the holder's pid and range, `F_SETLKW` waits in the kernel without holding up anything else and a signal ends the wait with `EINTR`, a wait that would close a cycle between waiting processes fails with `EDEADLK`, a write lock on a descriptor not open for writing (or a read lock on one not open for reading) is `EBADF`, closing any descriptor of the file drops the process's locks on it, and exit releases them all. Before an unlock, the process writes back what it changed, so the next holder reads it. An open file's directory entry is checked against the change ring too: a name another process removed or replaced is looked up again, so sqlite's check for a hot journal sees the other connection's commit. A WASI imports module's `tryLock`/`unlock` keeps its behaviour (whole file, per process, keyed by path). Limits: locks are per kernel, not across tabs ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)); and a WASIX program built against a wasix-sysroot without the import keeps its libc's local answer (every lock granted), so python's sqlite excludes other processes only once wasix-python is rebuilt on the sysroot that has it ([homescoop#407](https://github.com/ai-ecoverse/homescoop/issues/407)). An Emscripten program's `flock()` is a stub compiled into its libc that never reaches the kernel either, until a homescoop shim routes it to a kernel import ([#680](https://github.com/ai-ecoverse/slicc-kernel/issues/680)). A lock belongs to the file its descriptor was opened on: a file renamed over the name later, or the name's removal, doesn't move it.
+- **Locks across tabs** ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)): two kernels on one OPFS root (two tabs, or a tab and a worker page) share one table of `fcntl` record locks and `flock` locks, so a sqlite3 in one tab finds the database locked while a transaction is open in another.
+  - **One leader:** the first kernel on a root takes a Web Lock and serves the table from its kernel worker. A kernel alone on its root is its own leader, and its lock calls cost what they did. Another kernel's lock calls go to the leader over a BroadcastChannel: about 0.1 ms a call against 0.02 ms on the leader, measured with the leader's tab in the background.
+  - **Blocking and deadlocks:** `F_SETLKW` and a blocking `flock` wait at the leader. A signal ends the wait with `EINTR`. `EDEADLK` sees the whole cycle, whichever tabs its processes run in.
+  - **Kernels that go away:** each kernel holds a Web Lock of its own for as long as it runs, so a closed or crashed tab's locks are dropped and its waiters woken within about a second.
+  - **Leader failover:** when the leader goes, the next kernel takes over. It waits half a second while the others hand it the locks they hold and re-send their pending calls. A re-sent call is answered once.
+  - **The same file:** a lock is keyed by the file's device and inode, which both kernels read from the shared sidecar, so a rename by either tab keeps it. A file removed and created again at the same path has the same key, as it already has within one kernel.
+  - **Versions:** the channel and Web Lock names carry a protocol version (`v1`), and a kernel ignores messages that aren't v1. So kernels on different protocol versions (two tabs across an update) don't share locks, as before #675.
+  - **Not across tabs:** the imports module's whole-file `tryLock` (per kernel), and locks on other mounts.
 - **`slicc.fcntl_lock(fd, cmd, flock_ptr) -> errno`** ([#722](https://github.com/ai-ecoverse/slicc-kernel/issues/722)): a WASIX program's `fcntl` record locks reach the same table as an Emscripten program's, so the two see each other's locks.
   - **cmd:** `12` `F_GETLK`, `13` `F_SETLK` or `14` `F_SETLKW`. Any other cmd is `EINVAL`.
   - **The struct:** musl's wasm32 `struct flock`, 32 bytes: `l_type` i16 at 0 (`0` read, `1` write, `2` unlock), `l_whence` i16 at 2 (`0` SET, `1` CUR, `2` END, resolved against the descriptor's offset or size), `l_start` i64 at 8, `l_len` i64 at 16 (`0` to EOF and beyond, negative before `l_start`), `l_pid` i32 at 24. The kernel reads and writes only those five fields and never the padding.
@@ -625,7 +633,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
   - **Owner:** the open file description, not the process. A `dup`'d descriptor and one a forked child or a spawned program inherits share it, so the child's `LOCK_EX|LOCK_NB` is granted and its `LOCK_UN` releases the lock for the parent too. So children that `flock(STDOUT)` an inherited STDOUT don't exclude each other, on Linux either: for that, each child opens a lock file itself. A second open of the same object is another description and conflicts: the file opened again (by any name of it), `/dev/tty` opened again, the other end of a pipe.
   - **Rules:** shared locks coexist, an exclusive one excludes every other description, and any access mode takes either kind, so `LOCK_EX` on a read-only open excludes (homescoop#758). A conversion drops the old lock before it takes the new one, as on Linux, so a refused `LOCK_NB` conversion leaves no lock. The lock goes with `LOCK_UN` or when the description's last descriptor closes, at exit or kill too. flock locks and `fcntl` record locks are separate and don't block each other.
   - **Errors (WASI numbers):** `EAGAIN` for a conflict with `LOCK_NB`, `EINTR` when a signal with a handler ends a wait (with `SA_RESTART` the handler runs and the wait goes on, [#931](https://github.com/ai-ecoverse/slicc-kernel/issues/931)), `EBADF` for a bad descriptor, `EINVAL` for a bad op. There's no deadlock detection, as on Linux.
-  - **Limits:** locks are per kernel ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)). A file the program buffers itself is handed to the kernel at its first `flock`, with every other descriptor the program has open on it. A libc that finds no import on an older kernel gets `ENOSYS`; wasix-libc's `flock()` then keeps its old answer. Emscripten programs and the imports module's `tryLock` are unchanged.
+  - **Limits:** a file the program buffers itself is handed to the kernel at its first `flock`, with every other descriptor the program has open on it. A libc that finds no import on an older kernel gets `ENOSYS`; wasix-libc's `flock()` then keeps its old answer. Emscripten programs and the imports module's `tryLock` are unchanged.
 - **Rules on a mount:**
   - a rename across mounts is `EXDEV`, so `mv` copies;
   - a symlink on the root file system that leads onto a mount, such as `~/proj` → `/mnt/drive/proj`, or a chain from a mount to the root and back, is followed onto the mount by every operation, in every runtime: reads, writes, `stat`, listings, `mkdir`, removals, renames, `chmod`, `chown` and times. Removing the link itself removes the link. Before [#547](https://github.com/ai-ecoverse/slicc-kernel/issues/547) was fixed, the kernel resolved such a link against the directory under the mount point for JS and WASI programs, the embedder's `fs` and attached clients (Emscripten programs weren't affected): reads failed with `ENOENT`, and writes went into that hidden directory, which shows again once the mount is gone;