@ai-ecoverse/slicc-kernel 1.33.0 → 1.34.1

package/README.md

@@ -65,7 +65,7 @@ Starts `argv` as the session leader on a new terminal and resolves once it is ru
 | --- | --- |
 | `pid` | the session leader's pid |
 | `onData` | the output listener; output that arrives while it is `null` is kept and delivered when one is set |
-| `write(data)` | types a string or bytes into the terminal; with `ISIG` on, `^C`, `^Z` and `^\` signal the foreground process group. A key typed while a child of that group is still setting up is held until the child has finished, then sent to the group in the foreground at that point. Setting up means the child's calls before its first other one: `setpgid`, `tcsetpgrp` and signal masks. The hold ends 250 ms after the child's first call at the latest. So `^Z` right after a command line stops the new job, as it does on Linux, where that setup takes microseconds rather than a worker start. |
+| `write(data)` | types a string or bytes into the terminal; with `ISIG` on, `^C`, `^Z` and `^\` signal the foreground process group. A key typed while a child of that group is still setting up is held until the child has finished, then sent to the group in the foreground at that point. Setting up means the child's calls before its first other one: `setpgid`, `tcsetpgrp`, signal masks, and starting the program it execs (which then sets itself up too). The hold ends 2 s after the child's first call at the latest. A key typed while the foreground group is busy, not reading the terminal (a shell between reading a command line and starting it), goes to that group and also to the job the group starts next, once that job has taken the terminal; the second copy is dropped when the group reads the terminal again. So `^C` or `^Z` right after a command line reaches the new job, as it does on Linux, where starting a job takes microseconds rather than a worker start. |
 | `resize(cols, rows)` | sets the window size (`TIOCGWINSZ`) and sends `SIGWINCH` to the foreground process group |
 | `signal(name)` | sends a signal (`'SIGINT'`, `'SIGTSTP'`, `'SIGQUIT'`, `'SIGHUP'`, …) to the foreground process group, regardless of termios |
 | `exited` | resolves with the session leader's exit status (`128 + n` if killed by signal `n`) |
@@ -334,6 +334,8 @@ The proxy answers `200` with `Content-Type: application/vnd.slicc.raw-fetch`: a
 
 Directories are renamed with `FileSystemHandle.move()` where available, else by copy and delete.
 
+`mmap` of a regular file works on every file system, for Emscripten programs built with `mmap` (whose glue exports `emscripten_builtin_memalign`) as well as for WASIX ones: the kernel reads the mapped range into fresh memory of the process, as Emscripten's MEMFS does. A `MAP_PRIVATE` mapping may be written, and its changes stay in the process. A writable `MAP_SHARED` mapping of such a file is `ENODEV`; a file in the process's own in-memory file system (MEMFS, as under `/dev`) keeps Emscripten's own mapping, which writes back. A read-only `MAP_SHARED` mapping is a copy taken at `mmap`, so later changes to the file do not show in it.
+
 ### Mounts
 
 `kernel.mount({ type, source, target, options })` mounts a file system on an existing directory. Every process sees it, Emscripten and WASI alike, and so does every attached client. `kernel.umount(target)` unmounts it, and fails with `EBUSY` while a process has a file open under it. `kernel.mounts()` lists the table, as does `/proc/mounts`. The Node entry and attached clients have the same three calls (client protocol 1.2).
@@ -347,7 +349,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
   - `@ai-ecoverse/slicc-kernel/driver` exports the types and `fsError`.
 - **Capabilities:** `readonly`, `symlinks`, `chmod`, `ranges`, `maxIo` (the largest read or write per message), `maxFile`, `listingStats`, `linkTimes`, and `attrTtl`/`entryTtl` (in ms, default 1000). `linkTimes` promises that `setattr` with `mtime` on a symlink sets the link's own time. Without it, setting a link's own times (`touch -h`, `utimensat` with `AT_SYMLINK_NOFOLLOW`) fails with `EOPNOTSUPP` and never reaches the target. tmpfs has it; hostfs does not. Setting times through a link follows it in the kernel, so `setattr` gets the target. `ranges` (driver protocol 1.1) promises that a write lands in the file at once, with no copy of the whole file on `open` or `release`, and that `setattr` takes `size`.
 - **Caching:** the kernel caches attributes and listings for those TTLs, and a driver can push `invalidate` for changes made outside.
-- **File contents:** on a mount with `ranges`, as `tmpfs` is, and on the root file system when it is in OPFS (in a browser with sync access handles) or an in-memory root, a program reads and writes a file in 64 KiB pages: reads come from the driver 1 MiB at a time, and changed pages go back in 1 MiB runs once 8 MiB have changed, at `fsync`, and when the file is closed (from the kernel's own descriptors also 250 ms after a write). A file of any size the mount can hold works, with memory to spare for about one more copy of it. In OPFS a write opens a sync access handle for just that write. Elsewhere (`fsa` mounts, `hostfs` with an older proxy, and a root outside OPFS), file contents are read whole when a program opens a file and written back when it closes it. Both give close-to-open consistency, as NFS has. A process that ends without closing its files keeps what it wrote: on exit, on a trap, and when a signal kills it. For a killed WASI program, the kernel lets its worker write its changed pages back before `waitpid` reports the death. The only exception is a worker that makes no call into the kernel for 2 seconds after the signal; it is stopped without that write-back.
+- **File contents:** on a mount with `ranges`, as `tmpfs` is, and on the root file system when it is in OPFS (in a browser with sync access handles) or an in-memory root, a program reads and writes a file in 64 KiB pages: reads come from the driver 1 MiB at a time, and changed pages go back in 1 MiB runs once 8 MiB have changed, at `fsync`, and when the file is closed (from the kernel's own descriptors also 250 ms after a write). A file of any size the mount can hold works, with memory to spare for about one more copy of it. In OPFS a write opens a sync access handle for just that write. Elsewhere (`fsa` mounts, `hostfs` with an older proxy, and a root outside OPFS), file contents are read whole when a program opens a file and written back when it closes it. Before a program's next call into the kernel (any wait, read of a pipe or terminal, sleep or spawn), its changed pages also go back, so other processes and the embedder see a write as soon as the writer does anything but compute: a daemon's pidfile, a log another process follows. On a whole-file mount that holds only for files of up to 64 KiB; a larger one goes back at close, fsync and exit, as before. Beyond that, both give close-to-open consistency, as NFS has. A process that ends without closing its files keeps what it wrote: on exit, on a trap, and when a signal kills it. For a killed WASI program, the kernel lets its worker write its changed pages back before `waitpid` reports the death. The only exception is a worker that makes no call into the kernel for 2 seconds after the signal; it is stopped without that write-back.
 - **Rules on a mount:**
   - a rename across mounts is `EXDEV`, so `mv` copies;
   - `df` reports each mount from its driver's `statfs`;

package/dist/kernel-worker.js

@@ -1144,14 +1144,6 @@ function sinkFile(onData) {
     }
   });
 }
-function nullFile() {
-  return new OpenFile({
-    read: async () => new Uint8Array(0),
-    write: async (bytes) => bytes.length,
-    close: () => {
-    }
-  });
-}
 function written(device, bytes) {
   if (device === "full" && bytes.length > 0) throw new KernelError("ENOSPC");
   return bytes.length;
@@ -5003,7 +4995,7 @@ var ChildTable = class {
       captured.set(n, chunks);
       return sinkFile((bytes) => chunks.push(bytes));
     }
-    return nullFile();
+    return deviceFile("null");
   }
   async wait(pid, nohang, signal, flags = {}) {
     let interrupt;
@@ -6568,6 +6560,8 @@ var KernelTty = class {
   waiters = [];
   hungUp = false;
   name;
+  onRead;
+  readers = 0;
   screen;
   signal;
   constructor(screen, signal) {
@@ -6718,7 +6712,19 @@ var KernelTty = class {
     }
     this.screen.write(Uint8Array.from(out));
   }
+  get reading() {
+    return this.readers > 0;
+  }
   async read(max, signal) {
+    this.onRead?.();
+    this.readers++;
+    try {
+      return await this.take(max, signal);
+    } finally {
+      this.readers--;
+    }
+  }
+  async take(max, signal) {
     while (this.readable.length === 0) {
       if (this.hungUp) return new Uint8Array(0);
       await this.changed(signal);
@@ -6783,13 +6789,14 @@ var PtyPair = class {
   slaveSeen = false;
   index;
   onFree;
-  constructor(index, signal, onFree) {
+  constructor(index, signal, onFree, onRead) {
     this.index = index;
     this.onFree = onFree;
     this.slave = new KernelTty(
       { write: (bytes) => this.fromSlave(bytes) },
       (sig) => signal(this.slave, sig)
     );
+    if (onRead) this.slave.onRead = () => onRead(this.slave);
     this.slave.name = `/dev/pts/${index}`;
   }
   master() {
@@ -6887,13 +6894,15 @@ var PtyPair = class {
 var PtyTable = class {
   pairs = /* @__PURE__ */ new Map();
   signal;
-  constructor(signal) {
+  onRead;
+  constructor(signal, onRead) {
     this.signal = signal;
+    this.onRead = onRead;
   }
   open() {
     let index = 0;
     while (this.pairs.has(index)) index += 1;
-    const pair = new PtyPair(index, this.signal, (p) => this.pairs.delete(p.index));
+    const pair = new PtyPair(index, this.signal, (p) => this.pairs.delete(p.index), this.onRead);
     this.pairs.set(index, pair);
     return { pair, master: pair.master() };
   }
@@ -8446,6 +8455,7 @@ var WasmProcess = class {
   async exit() {
     if (this.exited) return;
     this.exited = true;
+    this.die();
     this.clearAlarm();
     this.interrupt.abort();
     this.options.locks?.release(this.pid);
@@ -8858,13 +8868,16 @@ var SETUP_OPS = /* @__PURE__ */ new Set([
   "fd-info",
   "sig-mask",
   "proc-setpgid",
-  "tty-pgrp-set"
+  "tty-pgrp-set",
+  "proc-exec",
+  "proc-spawn"
 ]);
-var SETTLE_MS = 250;
+var SETTLE_MS = 2e3;
 var KEY_SIGNALS = /* @__PURE__ */ new Set([2, 3, 20]);
 var SettlingChildren = class {
   children = /* @__PURE__ */ new Map();
   held = [];
+  parked = [];
   pgidOf;
   settleMs;
   constructor(pgidOf, settleMs = SETTLE_MS) {
@@ -8872,7 +8885,15 @@ var SettlingChildren = class {
     this.settleMs = settleMs;
   }
   started(pid, ppid) {
-    this.children.set(pid, { ppid });
+    const parent = this.children.get(ppid);
+    this.children.set(pid, parent ? { ppid: parent.ppid, execer: ppid } : { ppid });
+    const group = this.pgidOf(ppid);
+    for (const p of this.parked) if (p.pgid === group) this.held.push({ ...p, copy: true });
+    this.parked = this.parked.filter((p) => p.pgid !== group);
+  }
+  reading(pgid) {
+    this.parked = this.parked.filter((p) => p.pgid !== pgid);
+    this.held = this.held.filter((h) => !(h.copy && h.pgid === pgid));
   }
   syscall(pid, op) {
     const child = this.children.get(pid);
@@ -8888,13 +8909,25 @@ var SettlingChildren = class {
     if (!child) return;
     clearTimeout(child.timer);
     this.children.delete(pid);
+    if (child.execer !== void 0) this.settled(child.execer);
     const ready2 = this.held.filter((h) => !this.blocking(h.pgid));
     this.held = this.held.filter((h) => this.blocking(h.pgid));
     for (const h of ready2) h.send();
   }
-  deliver(sig, pgid, send) {
-    if (KEY_SIGNALS.has(sig) && this.blocking(pgid)) this.held.push({ pgid, send });
-    else send();
+  deliver(sig, pgid, send, foreground) {
+    if (!KEY_SIGNALS.has(sig)) {
+      send();
+    } else if (this.blocking(pgid)) {
+      this.held.push({ pgid, send });
+    } else if (foreground) {
+      send();
+      const moved2 = () => {
+        if (foreground() !== pgid) send();
+      };
+      this.parked.push({ pgid, send: moved2 });
+    } else {
+      send();
+    }
   }
   blocking(pgid) {
     for (const { ppid } of this.children.values()) if (this.pgidOf(ppid) === pgid) return true;
@@ -10164,11 +10197,8 @@ var Launcher = class {
   locks = new LockTable();
   settling = new SettlingChildren((pid) => this.jobs.pgidOf(pid));
   ptys = new PtyTable(
-    (tty, sig) => this.settling.deliver(
-      sig,
-      this.jobs.tcgetpgrp(tty, 0),
-      () => this.jobs.signalOwnedForeground(tty, sig)
-    )
+    (tty, sig) => this.keySignal(tty, 0, sig, () => this.jobs.signalOwnedForeground(tty, sig)),
+    (tty) => this.settling.reading(this.jobs.tcgetpgrp(tty, 0))
   );
   net = new LoopbackNet();
   cdp;
@@ -10584,6 +10614,11 @@ var Launcher = class {
       })
     );
   }
+  keySignal(tty, fallback, sig, send) {
+    const pgid = this.jobs.tcgetpgrp(tty, fallback);
+    const foreground = () => this.jobs.tcgetpgrp(tty, fallback);
+    this.settling.deliver(sig, pgid, send, tty.reading ? void 0 : foreground);
+  }
   ignoredBy(pid, fork = false) {
     return this.processes.get(pid)?.ignoredSignals(fork) ?? 0;
   }
@@ -10779,12 +10814,9 @@ var Launcher = class {
     let leader = 0;
     const tty = new KernelTty(
       { write: (bytes) => options.onData(bytes.slice()) },
-      (sig) => this.settling.deliver(
-        sig,
-        this.jobs.tcgetpgrp(tty, leader),
-        () => this.jobs.signalForeground(tty, leader, sig)
-      )
+      (sig) => this.keySignal(tty, leader, sig, () => this.jobs.signalForeground(tty, leader, sig))
     );
+    tty.onRead = () => this.settling.reading(this.jobs.tcgetpgrp(tty, leader));
     tty.name = `/dev/tty${++this.terminals}`;
     tty.setSize(options.cols ?? 80, options.rows ?? 24);
     const fds = new FdTable();
@@ -10833,7 +10865,7 @@ var Launcher = class {
       return { status: NOT_FOUND, stdout: concat5(out), stderr: concat5(err) };
     }
     const fds = new FdTable();
-    fds.installAt(0, options.stdin ? bytesSource(options.stdin) : nullFile());
+    fds.installAt(0, options.stdin ? bytesSource(options.stdin) : deviceFile("null"));
     fds.installAt(1, sinkFile(stdout));
     fds.installAt(2, sinkFile(stderr));
     const group = options.pgid !== void 0 ? { pgid: options.pgid } : {};