@ai-ecoverse/slicc-kernel 1.116.1 → 1.117.0

package/README.md

@@ -589,6 +589,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 - **The node's identity, not its name, picks the FIFO:** an open FIFO keeps working after an `unlink` or a `rename`, and a new `mkfifo` at the same name is another FIFO. `/proc/<pid>/fd/N` names the FIFO's path, also in children that inherit the end.
 - **One kernel per storage root:** the table of open FIFOs is per kernel, so two tabs (two kernels on one origin) opening the same FIFO don't connect, as with record locks ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)).
 - GNU make 4.4's `-jN` uses a FIFO jobserver this way (`--jobserver-auth=fifo:PATH`), so sub-makes join it by path.
+- **Below an open** ([#908](https://github.com/ai-ecoverse/slicc-kernel/issues/908)): a FIFO or socket node is never read or written as a plain file. A JS program's `ctx.fs.readFile`/`writeFile` and the kernel API's `readFile`/`writeFile` (and the positioned reads, writes and truncates under them) fail with `ENXIO` on either, because they read or write a whole file rather than open one. Emscripten and WASI programs' opens of a FIFO go through the kernel's FIFO open as above, and a WASI program's open of a socket node is `ENXIO`, as on Linux.
 
 `mmap` of a regular file works on every file system, for Emscripten programs built with `mmap` (whose glue exports `emscripten_builtin_memalign`) as well as for WASIX ones: the kernel reads the mapped range into fresh memory of the process, as Emscripten's MEMFS does. A `MAP_PRIVATE` mapping may be written, and its changes stay in the process. A writable `MAP_SHARED` mapping of such a file is `ENODEV`; a file in the process's own in-memory file system (MEMFS, as under `/dev`) keeps Emscripten's own mapping, which writes back. A read-only `MAP_SHARED` mapping is a copy taken at `mmap`, so later changes to the file do not show in it.
 
@@ -617,6 +618,13 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
   - **F_GETLK:** with a conflicting lock, it writes that lock back (`l_whence` 0, its absolute start, its length, the holder's pid). Without one, it sets only `l_type` to `2`.
   - **Errors (WASI numbers):** `EAGAIN` for a conflict on `F_SETLK` (never `EACCES`), `EINTR` when a signal with a handler ends an `F_SETLKW` wait, `EDEADLK` for a wait that would close a cycle, `EBADF` for a bad descriptor or a lock type the descriptor's access mode doesn't allow, and `EINVAL` for a bad `l_type` or `l_whence` or a start before 0.
   - **Semantics:** the same as Emscripten's, above. A libc that finds no import on an older kernel gets `ENOSYS` and keeps its local answer.
+  - **Any kind of descriptor** ([#893](https://github.com/ai-ecoverse/slicc-kernel/issues/893)): a pipe, a socket or a terminal takes record locks too, as on Linux, keyed by the object (both ends of a pipe are one object, each socket is its own, a terminal is one object however often it's opened). `l_whence` `CUR` and `END` count from 0 there.
+- **`slicc.flock(fd, op) -> errno`** ([#893](https://github.com/ai-ecoverse/slicc-kernel/issues/893)): BSD `flock(2)` locks, with Linux's semantics, for a WASIX program on any descriptor: a regular file, a directory, a pipe, a socket, a terminal or a `/dev` device.
+  - **op:** Linux's bits, `LOCK_SH` `1`, `LOCK_EX` `2` or `LOCK_UN` `8`, optionally with `LOCK_NB` `4`. Anything else is `EINVAL`.
+  - **Owner:** the open file description, not the process. A `dup`'d descriptor and one a forked child or a spawned program inherits share it, so the child's `LOCK_EX|LOCK_NB` is granted and its `LOCK_UN` releases the lock for the parent too. So children that `flock(STDOUT)` an inherited STDOUT don't exclude each other, on Linux either: for that, each child opens a lock file itself. A second open of the same object is another description and conflicts: the file opened again (by any name of it), `/dev/tty` opened again, the other end of a pipe.
+  - **Rules:** shared locks coexist, an exclusive one excludes every other description, and any access mode takes either kind, so `LOCK_EX` on a read-only open excludes (homescoop#758). A conversion drops the old lock before it takes the new one, as on Linux, so a refused `LOCK_NB` conversion leaves no lock. The lock goes with `LOCK_UN` or when the description's last descriptor closes, at exit or kill too. flock locks and `fcntl` record locks are separate and don't block each other.
+  - **Errors (WASI numbers):** `EAGAIN` for a conflict with `LOCK_NB`, `EINTR` when a signal with a handler ends a wait (as for `F_SETLKW`: a libc that wants `SA_RESTART` asks again), `EBADF` for a bad descriptor, `EINVAL` for a bad op. There's no deadlock detection, as on Linux.
+  - **Limits:** locks are per kernel ([#675](https://github.com/ai-ecoverse/slicc-kernel/issues/675)). A file the program buffers itself is handed to the kernel at its first `flock`, with every other descriptor the program has open on it. A libc that finds no import on an older kernel gets `ENOSYS`; wasix-libc's `flock()` then keeps its old answer. Emscripten programs and the imports module's `tryLock` are unchanged.
 - **Rules on a mount:**
   - a rename across mounts is `EXDEV`, so `mv` copies;
   - a symlink on the root file system that leads onto a mount, such as `~/proj` → `/mnt/drive/proj`, or a chain from a mount to the root and back, is followed onto the mount by every operation, in every runtime: reads, writes, `stat`, listings, `mkdir`, removals, renames, `chmod`, `chown` and times. Removing the link itself removes the link. Before [#547](https://github.com/ai-ecoverse/slicc-kernel/issues/547) was fixed, the kernel resolved such a link against the directory under the mount point for JS and WASI programs, the embedder's `fs` and attached clients (Emscripten programs weren't affected): reads failed with `ENOENT`, and writes went into that hidden directory, which shows again once the mount is gone;
@@ -969,7 +977,7 @@ Releases are cut by semantic-release on every push to `main`, in two jobs, and o
 Every change is certified in its pull request, before it merges (`cert-review-pr.yml`, next section), and `cert-review` is a required check of the merge queue, so every commit on `main`, and every release built from it, was certified first. A release is then **verified on npm** (`release / verify on npm` in `release.yml`, `tools/cert-review/verify.mjs`), in code, for every tag the run made:
 
 - It waits up to an hour for npm to serve the version (an accepted publish can take a while to appear), then downloads the tarball and checks `dist.integrity` (sha512), the SLSA v1 provenance (the tarball's sha512 is its subject; `release.yml` on `refs/heads/main`, a GitHub-hosted builder; `gitCommit` = `gitHead` = the tag, on `main`), and `npm audit signatures`. A version that `release-recover.mjs` skipped because npm's `latest` is newer (#837) has nothing to verify.
-- It then writes `certified @ai-ecoverse/slicc-kernel@<v> sha256=<64 hex> (covers <first>–<v>; ci: <run>; pr-certified: <each merged pull request's cert-review>; known: <open known-issue issues>)` into the GitHub release's notes, between `<!-- cert-verify -->` markers. homescoop's ci reads exactly that prefix and re-hashes the npm tarball against it. `covers` starts after the last version npm serves, so a skipped version is covered by the next. Notes that already carry the identical line stay unchanged.
+- It then writes `certified @ai-ecoverse/slicc-kernel@<v> sha256=<64 hex> (covers <first>–<v>; ci: <run>; pr-certified: <each merged pull request's cert-review at its merge>; known: <open known-issue issues>)` into the GitHub release's notes, between `<!-- cert-verify -->` markers. homescoop's ci reads exactly that prefix and re-hashes the npm tarball against it. `covers` starts after the last version npm serves, so a skipped version is covered by the next. Notes that already carry the identical line stay unchanged. A pull request's `cert-review` is the newest finished one (success, failure or error) on its head at or before its merge, or the newest pending one if none had finished. A re-review that a later `merge-ok` re-label starts on the same head, still pending at the merge or ending after it, doesn't change what the merge queue saw, and a status posted after the merge can't change a release's line.
 - A mismatch writes no line. The job is red with a `supply-chain alert`, and so is a line already in the notes with another sha256.
 
 The tools in `tools/cert-review/`:
@@ -1240,6 +1248,16 @@ A WASIX C program for `test/integration/wasix-locks.test.mjs` (#722). It calls `
 
 **Build:** `INSTALL_WASIXCC=<homescoop>/scripts/install-wasixcc.sh WASIX_SYSROOT_28=<unpacked @ai-ecoverse/wasix-sysroot 2025.9.30-28> ./build.sh`, with homescoop's pinned wasixcc 0.4.7 (LLVM 21.1.206, binaryen 133) and Homebrew's binaryen `wasm-opt` 132 on `PATH` for the Asyncify pass (`proc_fork`). Two builds give the same bytes: `bin/wlocktest.wasm` is 56,958 bytes, sha256 `31308a54e13e7dcac497ad9d8a05799008c8beaa4ae3a36c76e218af7235ec23`.
 
+### `flocktest`
+
+A WASIX C program for `test/integration/flock.test.mjs` (#893). It calls `slicc.flock` and `slicc.fcntl_lock` directly (`import_module("slicc")`). Its arguments are a sequence of steps, run in order:
+- `open SLOT r|w|rw PATH`, `pipe SLOT SLOT`, `pair SLOT SLOT` (a `socketpair`), `dup FROM TO`, `fd SLOT N` (names an inherited descriptor), `close SLOT`;
+- `flock SLOT OP`, where OP is made of `sh`, `ex`, `un` and `nb` (`exnb` is `LOCK_EX|LOCK_NB`) or is a number; `badfd`; `fcntl SLOT r|w|u` (`F_SETLK` on the whole file);
+- `fork … end` (the child runs the steps up to the matching `end`, its lines start with `child `, and the parent waits for it before it goes on after `end`);
+- `touch PATH`, `await PATH`, `sleep MS`, `say TEXT`, `trap` (a `SIGUSR1` handler).
+
+**Build:** as `wlocktest`. Two builds give the same bytes: `bin/flocktest.wasm` is 58,442 bytes, sha256 `687ecf21c836adda4ef2e328b6032eed28ca7d37efc8e88d3aaefb8fc117115a`.
+
 ## License
 
 Apache-2.0

package/dist/kernel-worker.js

@@ -1699,13 +1699,20 @@ var OpfsFs = class {
     if (handle.kind === "directory") throw fsError("EISDIR", path);
     return handle;
   }
+  async refuseNode(path, empty) {
+    if (!empty) return;
+    const [entry] = await this.meta.get([(await this.locate(path, true)).path]);
+    if (entry?.fifo || entry?.socket) throw fsError("ENXIO", path);
+  }
   async readFileBuffer(path) {
     const handle = await this.existing(path);
-    return this.retrying(
+    const bytes = await this.retrying(
       path,
       async () => new Uint8Array(await (await handle.getFile()).arrayBuffer()),
       false
     );
+    await this.refuseNode(path, bytes.length === 0);
+    return bytes;
   }
   async readVersioned(path, cached = false) {
     const handle = await this.existing(path);
@@ -1715,6 +1722,7 @@ var OpfsFs = class {
         const stamp = await this.stamp(path, cached);
         const file = await handle.getFile();
         const bytes = new Uint8Array(await file.arrayBuffer());
+        await this.refuseNode(path, bytes.length === 0);
         return { bytes, version: `${file.lastModified}:${file.size}:${stamp}` };
       },
       false
@@ -1736,7 +1744,11 @@ var OpfsFs = class {
     const handle = await this.existing(path);
     return this.retrying(
       path,
-      async () => new Uint8Array(await (await handle.getFile()).slice(offset, offset + length2).arrayBuffer()),
+      async () => {
+        const file = await handle.getFile();
+        await this.refuseNode(path, file.size === 0);
+        return new Uint8Array(await file.slice(offset, offset + length2).arrayBuffer());
+      },
       false
     );
   }
@@ -1752,6 +1764,7 @@ var OpfsFs = class {
   }
   async truncate(path, size) {
     const handle = await this.existing(path, true);
+    await this.refuseNode(path, (await handle.getFile()).size === 0);
     const resolve2 = this.ranged || this.remote?.watched();
     const target = resolve2 || this.counted.length > 0 ? (await this.locate(path, true)).path : path;
     await this.counting(target);
@@ -1804,6 +1817,10 @@ var OpfsFs = class {
   }
   async fileFor(path) {
     const [target, existed] = await this.writeTarget(path);
+    if (existed) {
+      const [entry] = await this.meta.get([target]);
+      if (entry?.fifo || entry?.socket) throw fsError("ENXIO", path);
+    }
     const [dir, name2] = await this.parent(target);
     try {
       const handle = await this.inDir(
@@ -3780,7 +3797,8 @@ function openPipe(capacity) {
       },
       close: () => pipe.closeRead(),
       poll: () => ({ readable: pipe.readReady, writable: false, hangup: pipe.writersGone }),
-      changed: (signal) => pipe.changed(signal)
+      changed: (signal) => pipe.changed(signal),
+      identity: pipe
     }),
     write: new OpenFile({
       write: async (bytes, signal) => {
@@ -3792,7 +3810,8 @@ function openPipe(capacity) {
       },
       close: () => pipe.closeWrite(),
       poll: () => ({ readable: false, writable: pipe.writeReady, hangup: pipe.readersGone }),
-      changed: (signal) => pipe.changed(signal)
+      changed: (signal) => pipe.changed(signal),
+      identity: pipe
     })
   };
 }
@@ -6241,7 +6260,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.116.1" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.117.0" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -9607,6 +9626,7 @@ var FifoTable = class {
     let open = true;
     return new OpenFile({
       fifo: path,
+      identity: fifo,
       ...reads ? {
         read: async (max, signal) => {
           try {
@@ -13040,7 +13060,71 @@ function lockRange(start2, len) {
   if (from < 0) throw new KernelError("EINVAL");
   return { start: from, end };
 }
+function changedAt(wakers, key, signal) {
+  return new Promise((resolve2, reject) => {
+    const set3 = wakers.get(key) ?? /* @__PURE__ */ new Set();
+    wakers.set(key, set3);
+    const done4 = () => {
+      set3.delete(wake);
+      if (set3.size === 0 && wakers.get(key) === set3) wakers.delete(key);
+      signal.removeEventListener("abort", abort);
+    };
+    const wake = () => {
+      done4();
+      resolve2();
+    };
+    const abort = () => {
+      done4();
+      reject(new KernelError("EINTR"));
+    };
+    set3.add(wake);
+    signal.addEventListener("abort", abort, { once: true });
+  });
+}
+function wakeAt(wakers, key) {
+  for (const wake of [...wakers.get(key) ?? []]) wake();
+}
+var FlockTable = class {
+  held = /* @__PURE__ */ new Map();
+  keys = /* @__PURE__ */ new WeakMap();
+  wakers = /* @__PURE__ */ new Map();
+  free(key, owner, kind) {
+    for (const [other, has] of this.held.get(key) ?? [])
+      if (other !== owner && (kind === "exclusive" || has === "exclusive")) return false;
+    return true;
+  }
+  set(key, owner, kind) {
+    const has = this.held.get(key)?.get(owner);
+    if (has === kind) return true;
+    if (has !== void 0) this.drop(owner);
+    if (kind === "unlock") return true;
+    if (!this.free(key, owner, kind)) return false;
+    const holders = this.held.get(key) ?? /* @__PURE__ */ new Map();
+    holders.set(owner, kind);
+    this.held.set(key, holders);
+    this.keys.set(owner, key);
+    return true;
+  }
+  async wait(key, owner, kind, blocking) {
+    if (this.set(key, owner, kind)) return;
+    const signal = blocking();
+    while (!this.set(key, owner, kind)) {
+      if (signal.aborted) throw new KernelError("EINTR");
+      await changedAt(this.wakers, key, signal);
+    }
+  }
+  drop(owner) {
+    const key = this.keys.get(owner);
+    if (key === void 0) return;
+    this.keys.delete(owner);
+    const holders = this.held.get(key);
+    holders?.delete(owner);
+    if (holders?.size === 0) this.held.delete(key);
+    wakeAt(this.wakers, key);
+  }
+};
 var LockTable = class {
+  flocks = new FlockTable();
   records = /* @__PURE__ */ new Map();
   blocked = /* @__PURE__ */ new Map();
   wakers = /* @__PURE__ */ new Map();
@@ -13098,28 +13182,10 @@ var LockTable = class {
     return false;
   }
   changed(key, signal) {
-    return new Promise((resolve2, reject) => {
-      const set3 = this.wakers.get(key) ?? /* @__PURE__ */ new Set();
-      this.wakers.set(key, set3);
-      const done4 = () => {
-        set3.delete(wake);
-        if (set3.size === 0 && this.wakers.get(key) === set3) this.wakers.delete(key);
-        signal.removeEventListener("abort", abort);
-      };
-      const wake = () => {
-        done4();
-        resolve2();
-      };
-      const abort = () => {
-        done4();
-        reject(new KernelError("EINTR"));
-      };
-      set3.add(wake);
-      signal.addEventListener("abort", abort, { once: true });
-    });
+    return changedAt(this.wakers, key, signal);
   }
   wake(key) {
-    for (const wake of [...this.wakers.get(key) ?? []]) wake();
+    wakeAt(this.wakers, key);
   }
   store(key, locks) {
     if (locks.length === 0) this.records.delete(key);
@@ -13184,6 +13250,7 @@ var HOST_OPS = [
   "lk-set",
   "lk-get",
   "lk-close",
+  "flock",
   "async-submit",
   "async-wait",
   "async-take",
@@ -13336,6 +13403,8 @@ async function hostSyscall(req, ctx) {
     case "lk-get":
     case "lk-close":
       return recordLock2(req, ctx);
+    case "flock":
+      return flock(req, ctx);
     case "async-submit":
       if (!ctx.valid(req.req)) throw new KernelError("EINVAL");
       if (req.ack) ctx.ops.ack(req.ack);
@@ -13358,6 +13427,17 @@ async function hostSyscall(req, ctx) {
       return done2;
   }
 }
+var FLOCK_KINDS = /* @__PURE__ */ new Set(["shared", "exclusive", "unlock"]);
+async function flock(req, ctx) {
+  const { locks, flockOf } = ctx;
+  if (!locks || !flockOf) throw new KernelError("ENOLCK");
+  if (!FLOCK_KINDS.has(req.kind)) throw new KernelError("EINVAL");
+  const { key, owner } = await flockOf(req.fd);
+  if (req.kind !== "unlock" && req.wait)
+    await locks.flocks.wait(key, owner, req.kind, ctx.blocking);
+  else if (!locks.flocks.set(key, owner, req.kind)) throw new KernelError("EAGAIN");
+  return done2;
+}
 async function recordLock2(req, ctx) {
   const { locks, fileKey } = ctx;
   if (!locks || !fileKey) throw new KernelError("ENOLCK");
@@ -17486,6 +17566,18 @@ var fileKeys = /* @__PURE__ */ new WeakMap();
 function identityKey(st, path) {
   return st.ino === void 0 ? path : `${st.dev ?? 0}:${st.ino}`;
 }
+var objectIds = /* @__PURE__ */ new WeakMap();
+var nextObjectId = 1;
+function objectKey(file) {
+  const at = file.tty ?? file.identity ?? file.pty ?? file;
+  let id2 = objectIds.get(at);
+  if (id2 === void 0) {
+    id2 = nextObjectId++;
+    objectIds.set(at, id2);
+  }
+  return `object:${id2}`;
+}
+var flockHooked = /* @__PURE__ */ new WeakSet();
 var DYING = { ok: false, errno: "EINTR", message: "the process is being killed" };
 var HOSTS_ONLY = new Resolver({ routes: new Routes() });
 function timeLeft(req, started) {
@@ -17786,6 +17878,7 @@ var WasmProcess = class {
           pid: this.pid,
           owner: () => this.lockOwner(),
           fileKey: (target, kind, setting) => this.lockKey(target, kind, setting),
+          flockOf: (fd) => this.flockOf(fd),
           locks: this.options.locks,
           ops: this.asyncOps,
           blocking: () => this.blockingSignal(cancelled),
@@ -17854,6 +17947,7 @@ var WasmProcess = class {
       throw new KernelError("EBADF");
     const known = file ? fileKeys.get(file) : this.pathKeys.get(target.path ?? "");
     if (kind === "close" && !file && target.path !== void 0) this.pathKeys.delete(target.path);
+    if (access2 && !access2.stat) return objectKey(access2);
     const want = kind === "read" ? R_OK : kind === "write" ? W_OK : 0;
     const path = await this.lockPath(target, want);
     if (known !== void 0) return known;
@@ -17868,6 +17962,33 @@ var WasmProcess = class {
     else if (this.pathKeys.size < 4096) this.pathKeys.set(path, key);
     return key;
   }
+  async flockOf(fd) {
+    const open = this.fds.get(fd);
+    const { locks } = this.options;
+    if (locks && !flockHooked.has(open)) {
+      flockHooked.add(open);
+      open.onClose(() => locks.flocks.drop(open));
+    }
+    return { key: await this.objectKeyOf(open), owner: open };
+  }
+  async objectKeyOf(open) {
+    const { file } = open;
+    const meta = file.heldMeta;
+    if (meta && "device" in meta) return `device:${meta.device}`;
+    if (meta && "dir" in meta) {
+      const st2 = await this.options.fs.stat?.(meta.dir).catch(() => void 0);
+      return st2 ? identityKey(st2, meta.dir) : objectKey(file);
+    }
+    const known = fileKeys.get(open);
+    if (known !== void 0) return known;
+    const vfs = await file.stat?.().catch(() => void 0);
+    if (!vfs || vfs.orphan) return objectKey(file);
+    const st = await this.options.fs.stat?.(vfs.path).catch(() => void 0);
+    if (!st) return objectKey(file);
+    const key = identityKey(st, vfs.path);
+    fileKeys.set(open, key);
+    return key;
+  }
   async dropRecordLocks(fd) {
     const { locks } = this.options;
     const owner = this.options.jobs?.shown(this.pid) ?? this.pid;
@@ -20148,6 +20269,7 @@ function scrubbed(inner, scrubber, idleMs = SCRUB_IDLE_MS) {
     },
     changed,
     ...file.tty ? { tty: file.tty } : {},
+    identity: file.identity ?? file,
     ...file.clipboard ? { clipboard: file.clipboard } : {},
     ...file.write ? {
       write: async (bytes, signal) => {