@ai-ecoverse/slicc-kernel 1.26.0 → 1.26.2

package/README.md

@@ -120,7 +120,7 @@ const { status, stdout } = await kernel.run(['bash', '-c', 'echo hi'], { cwd: '/
 kernel.terminate();
 ```
 
-`createNodeKernel({ root, modules, env, network, worker, processMounts, fstabRetries, cdp })` takes the options of `createKernel` except `metadata`, and `fstabRetries`, the delays in ms between tries of a failing `/etc/fstab` line (default `[1000, 4000, 16000]`). `root` is an in-memory directory by default (`memoryRoot()` makes another), which keeps each file in 1 MiB chunks, and POSIX metadata stays in memory. Processes and threads run on `worker_threads`. The kernel has `run`, `openTerminal` and `terminate` as above, plus `root`, `writeFile(path, data)` (creating the parent directories) and `readFile(path)` to put files in place and read results. `nodeTransport()` is `fetchTransport()` with Node's `fetch`, which no CORS binds (`crossOrigin: 'any'`). `connect()` and `attachKernel` work as in the browser: the port is a `worker_threads` `MessagePort`, which a worker thread can attach with, and `terminate()` ends every attached client.
+`createNodeKernel({ root, modules, env, network, worker, processMounts, fstabRetries, cdp, hostfs })` takes the options of `createKernel` except `metadata`, and `fstabRetries`, the delays in ms between tries of a failing `/etc/fstab` line (default `[1000, 4000, 16000]`). `root` is an in-memory directory by default (`memoryRoot()` makes another), which keeps each file in 1 MiB chunks, and POSIX metadata stays in memory. Processes and threads run on `worker_threads`. The kernel has `run`, `openTerminal` and `terminate` as above, plus `root`, `writeFile(path, data)` (creating the parent directories) and `readFile(path)` to put files in place and read results. `nodeTransport()` is `fetchTransport()` with Node's `fetch`, which no CORS binds (`crossOrigin: 'any'`). `connect()` and `attachKernel` work as in the browser: the port is a `worker_threads` `MessagePort`, which a worker thread can attach with, and `terminate()` ends every attached client.
 
 ## Commands
 
@@ -144,7 +144,7 @@ A WASI program runs in a process worker of its own like an Emscripten one, again
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
 - **WASIX** (a module importing `wasix_32v1`): fork and setjmp/longjmp through Asyncify (the module is built with `wasm-opt --asyncify`), exec, `posix_spawn`, `waitpid`, pipes, `dup2`, signal handlers and interval timers, terminal modes, and dynamic linking: a position-independent main module loads side modules with `dlopen` from `LD_LIBRARY_PATH`, its runtime path, `/lib`, `/usr/lib` and `/usr/local/lib`.
-- **Sockets**: the WASIX socket calls (`sock_open`, `bind`, `listen`, `connect`, `accept`, options, local and peer names) are the kernel's loopback sockets, and preview1's `sock_recv` / `sock_send` / `sock_shutdown` work on them and on inherited ones. HTTP and HTTPS leave through the realm proxy that `https_proxy` names, as for any other program; `resolve` answers loopback names and literal addresses.
+- **Sockets**: the WASIX socket calls (`sock_open`, `bind`, `listen`, `connect`, `accept`, options, local and peer names) are the kernel's loopback sockets, and preview1's `sock_recv` / `sock_send` / `sock_shutdown` work on them and on inherited ones. Writing to a socket whose peer is gone fails with `EPIPE`, through `fd_write` as through `sock_send`, as if `MSG_NOSIGNAL` were set, since a WASI program cannot ignore `SIGPIPE`; a pipe or terminal whose reader is gone still ends the program with status 141. HTTP and HTTPS leave through the realm proxy that `https_proxy` names, as for any other program; `resolve` answers loopback names and literal addresses.
 - **Diagnostics**: a trap ends the program with 134 and its message on stderr; with `SLICC_WASM_BACKTRACE=1` the wasm frames follow, named from the module's name section, or from a sidecar for a module shipped without one (`<module>.names` beside it, or the same path in a `<package>-names` package). `SLICC_WASI_STATS=1` makes a program print its calls, counted and timed, as it ends.
 
 ### Program imports
@@ -184,7 +184,7 @@ await kernel.run(['curl', '-sS', 'https://registry.npmjs.org/@ai-ecoverse/wasm-b
 Programs drive a browser with the Chrome DevTools Protocol over a WebSocket, whichever host actually has the browser: slicc-extension's `chrome.debugger`, slicc-node's or slicc-swift's CDP proxy, or a test harness. The kernel relays; it does not launch a browser.
 
 - **Programs** find the endpoint in `SLICC_CDP_URL`, `ws://127.0.0.1:9222/devtools/browser/<id>`, where `<id>` is random for each kernel boot. `http://127.0.0.1:9222/json/version` answers `{ "Browser", "Protocol-Version", "webSocketDebuggerUrl" }` with the same URL (carrying the request's query), and `/json/list` (or `/json`) lists the host's targets, without per-target sockets. Every other path is `404`. The listener starts on the first connection, on the kernel's loopback; a program that binds `9222` first keeps it.
-- **Sessions** are Chrome's flattened ones: `Target.attachToTarget` with `flatten: true`, then commands with a top-level `sessionId`. Messages pass unchanged, as text frames up to 256 MiB (a larger one closes the socket with `1009`). Each program WebSocket gets a host connection of its own: closing the socket (or exiting) closes it, which detaches its sessions, and a host that closes is passed on as a close frame (`1011` with its reason). `terminate()` closes them all.
+- **Sessions** are Chrome's flattened ones: `Target.attachToTarget` with `flatten: true`, then commands with a top-level `sessionId`. Messages pass unchanged, as text frames up to 256 MiB (a larger one closes the socket with `1009`). Each program WebSocket gets a host connection of its own: closing the socket (or exiting) closes it, which detaches its sessions, and a host that closes is passed on as a close frame (`1011` with its reason). The kernel then waits up to a second for the program's close frame before it hangs up, as RFC 6455 has it, so a program that answers the close can still report the code. `terminate()` closes them all.
 - **Runtime**: a program adds `runtime=<name>` to the query of the URL it opens, and the kernel gives it to the host; the kernel does not interpret it.
 - **No host**: `/json/*` and the WebSocket handshake answer `503` with `slicc-kernel: no CDP host is attached (createKernel({ cdp }) or client.serveCdp)`, and a host that refuses (an unknown runtime, say) gives `502` with its message.
 - **Kernel-only**: port `9222` takes connections only from programs in the kernel. `kernel.dial`, a client's `dial` and `loopbackFetch` (and so `9222.kernel.localhost` and slicc-node's tunnel) are refused with `ECONNREFUSED`, since the facade drives the user's real browser.
@@ -282,7 +282,7 @@ A `hostfs` mount is a folder exported by the local proxy ([slicc-node](https://g
 - **Options:** `ro` mounts read-only, and `maxfile` caps file size as on any mount. There is no other size limit: reads and writes go in `maxIo` pieces (16 MiB unless the proxy says otherwise), so a file of any size the host can hold works. A proxy that advertises `ranges` in its grant (slicc-node 2.4, slicc-swift 1.4) makes the mount ranged, so programs read and write its files in pages and truncate through `setattr` `size`. With an older proxy files go whole, and a truncate on a ranged mount whose proxy was replaced by an older one is done by rewriting the file.
 - **Changes on the host** arrive as invalidations on a watch stream (`POST /api/hostfs/watch`), so an edit made outside shows up on the next access without a remount.
 - **Proxy loss:** when the watch stream ends, or is silent for 45 s, the kernel reconnects once at once. If that fails, the mount is `nomedium`, as with [removable media](#removable-media-fsa): operations fail with `ENOMEDIUM` (`ENODEV` for WASI), and the kernel keeps reconnecting with backoff (1 s to 30 s), so the folder comes back by itself when the proxy does. A mount made while the proxy is down starts as `nomedium`, and so does one whose grant or first watch takes more than 15 s; it comes up by itself when they arrive.
-- **Consistency:** a file that changes on the host while a program reads it fails with `ESTALE`, and the kernel restarts the whole read (up to 3 times) instead of stitching two versions together. On a ranged mount an open file keeps the version it was opened at (its ETag) until it writes: a page fetched after the host changed the file fails the program's read with `ESTALE` instead of mixing versions.
+- **Consistency:** a file that changes on the host while a program reads it fails with `ESTALE`, and the kernel restarts the whole read (up to 3 times) instead of stitching two versions together. On a ranged mount an open file keeps the version it was opened at (its ETag) until it writes, also when it is handed to a child process: a page fetched after the host changed the file fails the program's read with `ESTALE` ("Stale file handle") instead of mixing versions.
 
 ### Mounting from a program
 

package/dist/kernel-worker.js

@@ -3749,6 +3749,7 @@ Sec-WebSocket-Accept: ${accept2}\r
 // src/cdp/facade.ts
 var MAX_HEAD = 64 * 1024;
 var LIST_TIMEOUT = 1e4;
+var CLOSE_WAIT_MS = 1e3;
 function respond(status, body, type = "text/plain; charset=utf-8") {
   const bytes = new TextEncoder().encode(body);
   const head = latin1Bytes(
@@ -3894,7 +3895,11 @@ var CdpFacade = class {
   async upgrade(conn, incoming, req, runtime) {
     const accept2 = await acceptKey(req.headers);
     const host = await this.host(runtime);
-    await new Relay(conn, incoming, host, this.options.maxMessage ?? MAX_MESSAGE, this.stop.signal).start(switchingHead(accept2)).finally(() => host.close());
+    const limits = {
+      max: this.options.maxMessage ?? MAX_MESSAGE,
+      closeWaitMs: this.options.closeWaitMs ?? CLOSE_WAIT_MS
+    };
+    await new Relay(conn, incoming, host, limits, this.stop.signal).start(switchingHead(accept2)).finally(() => host.close());
   }
 };
 var Relay = class {
@@ -3904,12 +3909,14 @@ var Relay = class {
   incoming;
   host;
   max;
+  closeWaitMs;
   stop;
-  constructor(conn, incoming, host, max, stop) {
+  constructor(conn, incoming, host, limits, stop) {
     this.conn = conn;
     this.incoming = incoming;
     this.host = host;
-    this.max = max;
+    this.max = limits.max;
+    this.closeWaitMs = limits.closeWaitMs;
     this.stop = stop;
   }
   send(bytes) {
@@ -3935,8 +3942,15 @@ var Relay = class {
         void this.end(CLOSE.tooBig, `a message is over ${this.max} bytes`);
       else void this.send(frame(OP_TEXT, bytes));
     };
-    await Promise.race([this.pump(), hostGone]);
+    const pumping = this.pump();
+    await Promise.race([pumping, hostGone]);
     await this.writes;
+    let timer;
+    const waited = new Promise((resolve) => {
+      timer = setTimeout(resolve, this.closeWaitMs);
+    });
+    await Promise.race([pumping, waited]);
+    clearTimeout(timer);
   }
   async pump() {
     const reader = new WsReader(this.incoming, this.max);
@@ -5089,6 +5103,9 @@ var MountFs = class {
       truncate: false,
       exclusive: false,
       ...version !== void 0 ? { ifMatch: version } : {}
+    }).catch((err) => {
+      const gone = version !== void 0 && err.code === "ENOENT";
+      throw gone ? errnoError("ESTALE", `${real} is gone from the host`) : err;
     });
     const chunk2 = mount.caps.maxIo ?? DEFAULT_IO;
     const chunks = [];
@@ -6689,6 +6706,7 @@ function within3(path, root) {
 var VfsNode = class {
   data;
   ranged;
+  pin;
   length = 0;
   dirty = false;
   queue = Promise.resolve();
@@ -6700,8 +6718,9 @@ var VfsNode = class {
   fs;
   path;
   orphaned;
-  constructor(fs, path, contents2, orphaned = false, dirty = false) {
+  constructor(fs, path, contents2, orphaned = false, dirty = false, pin) {
     this.fs = fs;
+    this.pin = pin;
     this.path = path;
     this.orphaned = orphaned;
     if (contents2 !== void 0) this.data = new Uint8Array(contents2);
@@ -6725,6 +6744,10 @@ var VfsNode = class {
     if (this.revoked) throw new KernelError("EIO");
     if (this.ranged || this.data) return this.ranged;
     const io = rangedOps(this.fs);
+    if (io && this.pin && !this.orphaned) {
+      this.ranged = new AsyncRangedFile(io, this.path, this.pin.size, this.pin.version);
+      return this.ranged;
+    }
     const st = io && !this.orphaned ? await this.fs.stat?.(this.path).catch(() => void 0) : void 0;
     if (io && st?.ranged) {
       this.ranged = new AsyncRangedFile(io, this.path, st.size ?? 0, st.version);
@@ -6864,10 +6887,10 @@ var VfsNodes = class {
   constructor(fs) {
     this.fs = fs;
   }
-  open(path) {
+  open(path, pin) {
     let node = this.byPath.get(path);
     if (!node) {
-      node = new VfsNode(this.fs, path);
+      node = new VfsNode(this.fs, path, void 0, false, false, pin);
       this.byPath.set(path, node);
     }
     node.opens++;
@@ -6988,7 +7011,7 @@ function vfsFile(fs, opts, nodes) {
   const access = opts.flags & O_ACCMODE;
   const readable = access !== O_WRONLY;
   const writable = access === O_WRONLY || access === O_RDWR;
-  const node = !nodes || opts.orphan ? new VfsNode(fs, opts.path, opts.contents, opts.orphan === true, opts.dirty === true) : opts.contents !== void 0 ? nodes.adopt(opts.path, opts.contents, opts.dirty === true) : nodes.open(opts.path);
+  const node = !nodes || opts.orphan ? new VfsNode(fs, opts.path, opts.contents, opts.orphan === true, opts.dirty === true) : opts.contents !== void 0 ? nodes.adopt(opts.path, opts.contents, opts.dirty === true) : nodes.open(opts.path, opts.pin);
   let openError;
   const atOpen = (op) => {
     void node.serial(op).catch((err) => {
@@ -7306,7 +7329,8 @@ var WasmProcess = class {
         ...req.contents !== void 0 ? { contents: req.contents } : {},
         ...req.orphan ? { orphan: true } : {},
         ...req.truncate ? { truncate: true } : {},
-        ...req.create ? { create: true } : {}
+        ...req.create ? { create: true } : {},
+        ...req.pin ? { pin: req.pin } : {}
       },
       this.nodes
     );
@@ -7438,7 +7462,8 @@ var WasmProcess = class {
         position: req.position ?? 0,
         ...req.contents !== void 0 ? { contents: req.contents } : {},
         ...req.orphan ? { orphan: true } : {},
-        ...req.dirty ? { dirty: true } : {}
+        ...req.dirty ? { dirty: true } : {},
+        ...req.pin ? { pin: req.pin } : {}
       },
       this.nodes
     );