@ai-ecoverse/slicc-kernel 1.54.8 → 1.54.9

package/README.md

@@ -429,11 +429,13 @@ A `hostfs` mount is a folder exported by the local proxy ([slicc-node](https://g
 - **Proxy loss:** when the watch stream ends, or is silent for 45 s, the kernel reconnects once at once. If that fails, the mount is `nomedium`, as with [removable media](#removable-media-fsa): operations fail with `ENOMEDIUM` (`ENODEV` for WASI), and the kernel keeps reconnecting with backoff (1 s to 30 s), so the folder comes back by itself when the proxy does. A mount made while the proxy is down starts as `nomedium`, and so does one whose grant or first watch takes more than 15 s; it comes up by itself when they arrive.
 - **Consistency:** a file that changes on the host while a program reads it fails with `ESTALE`, and the kernel restarts the whole read (up to 3 times) instead of stitching two versions together. On a ranged mount an open file keeps the version it was opened at (its ETag) until it writes, also when it is handed to a child process: a page fetched after the host changed the file fails the program's read with `ESTALE` ("Stale file handle") instead of mixing versions. A file deleted on the host while a program holds it open fails the next read that goes to the host with `ENOENT`, not with NFS's `ESTALE`.
 - **Over a port:** the hook may answer `{ port, token?, capabilities? }` instead, for a folder that doesn't come from slicc-node over HTTP: for example, a linked `slicc` CLI's export, which the page reaches over its own link. The page implements a `HostfsTransport` for the folder (`capabilities`, `token`, `maxIo`, `call(op, body)`, `read(fh, offset, size, ifMatch?)`, `write(fh, offset, bytes)`, `watch(signal, onEvent, onOpen)`, with slicc-node's ops and answers) and serves it with `serveHostfsPort(port, transport)`, exported from both entries, which returns a function that stops serving and closes the port.
+  - **Slow links:** each read or write message carries up to `maxIo` bytes and has to finish within the 20 s call bound, so a page that serves a folder over a slow or remote link sets `capabilities.maxIo` to fit it: 4 MiB over 1 Mbit/s takes about 32 s and would fail every time, while 512 KiB takes about 4 s.
   - The kernel moves the port into its worker with the grant. Calls, reads and writes are request and reply messages there, and the watch is a stream of messages that the kernel ends by aborting.
   - Errors keep their `code`. A transport that throws `ENOTCONN` (its link is down) makes the mount `nomedium` and reconnecting, as a lost proxy does: the kernel asks the hook again, and the page answers with a new port.
-  - A port that dies doesn't hang anyone: pending requests fail with `ENOTCONN` when the port closes (its `close` event), when its watch ends or goes silent, and, on engines that don't report a closed peer, when a request gets no answer within 60 s (plus 1 s per MiB of a read or write), which fails it with `ETIMEDOUT`. After any of these the port counts as gone and is closed, files opened through it are `ESTALE`, and the next operation reconnects. `serveHostfsPort` stops serving when its port closes.
+  - A port that dies doesn't hang anyone. Chrome reports no `close` when the peer of a `MessagePort` goes away, so while anything is outstanding (the watch always is), the kernel pings the port every second and `serveHostfsPort` answers at once. No answer within 10 s makes the port gone (a page busy for a few seconds, with a dialog open or paused in devtools, is still alive): pending requests fail with `ENOTCONN`, the mount is `nomedium`, and the kernel asks the hook for a new port. A port whose server never answered a ping (an older `serveHostfsPort`) isn't judged by pings. Pending requests also fail when the port's `close` event fires (Node has it), when its watch ends or goes silent, and, as a last resort, after 60 s without an answer (plus 1 s per MiB of a read or write), with `ETIMEDOUT`. A gone port is closed, and `serveHostfsPort` stops serving when its port closes. A watch started with an already aborted signal also counts the port as gone.
+  - **Hanging folders:** every request to a `hostfs` folder, over a port or over HTTP, gets 20 s: each call, and each read or write of up to `maxIo` bytes. The mount tells the kernel the folder's own `maxIo` (at most 16 MiB), so one file operation never puts more than that on the folder at once. One that takes longer fails with `ENOTCONN`, so the mount goes `nomedium` and reconnects by itself, instead of the driver watchdog (30 s) marking the mount `failed` until a remount. The timeout also retires that connection: its requests in flight are aborted (HTTP) or failed (a port), nothing more is sent on it, its open files are `ESTALE`, and a file whose open answers late is released on the folder. Over a port, the kernel keeps the port for another second for cleanup (a temporary write permission it added to open a read-only file is put back), then tells `serveHostfsPort` it is going (`bye`, which Chrome can't signal by itself). The server then releases every file it opened through that port, including one whose open lands later. A request the host already received may still finish there: an operation that timed out may have happened, so check before retrying it (a retried `mkdir` or create can fail with `EEXIST`, a retried `rename` or `unlink` with `ENOENT`). The kernel itself retries only reads and whole-file rewrites after `ESTALE`, never an exclusive create.
   - A read reply sends a copy of the transport's bytes, so a transport may return a view into storage it keeps (a cache, a pooled or shared buffer).
-  - `token` names the connection (a link id, say). When the next port's token differs, files that were open become `ESTALE`. Without a token, every port is a new connection.
+  - `token` names the connection (a link id, say); without one, every port gets its own name. It doesn't decide whether open files survive a reconnect: a file a program holds open is opened again by path on the new connection, with the ETag it had (`ifMatch`). It goes on if the file is unchanged on the host, and fails with `ESTALE` if the host changed it meanwhile. Files opened through a port that is gone are `ESTALE` until they are opened again.
 - **Node:** a page's requests carry its `Origin`, and slicc-node grants a token to that origin only. Node's `fetch` sends none, so a Node embedder passes `createNodeKernel` the origin its `hostfs` hook asked for the grant with, as `hostfsOrigin`, and the kernel adds it to every hostfs request. Without it the mount stays `nomedium`. `hostfsFetch(url, init)` replaces the `fetch` those requests use (for an agent or a test proxy), and gets the `Origin` too when both are given.
 
 ### Mounting from a program

package/dist/index.js

@@ -1646,11 +1646,38 @@ var errorOf = (err) => ({
 });
 function serveHostfsPort(port, transport) {
   const watches = /* @__PURE__ */ new Map();
+  const handles = /* @__PURE__ */ new Set();
+  let stopped = false;
+  const closeHandles = () => {
+    for (const fh of handles) void transport.call("release", { fh }, true).catch(() => {
+    });
+    handles.clear();
+  };
   const send2 = (reply, transfer = []) => port.postMessage(reply, transfer);
+  const pending = /* @__PURE__ */ new Set();
+  const track = (work) => {
+    pending.add(work);
+    const done = () => pending.delete(work);
+    work.then(done, done);
+    return work;
+  };
+  const answer = async (name, body) => {
+    const result = await track(transport.call(name, body));
+    const opened = result?.fh;
+    if (name === "open" && typeof opened === "number") handles.add(opened);
+    if (name === "release") handles.delete(body?.fh);
+    if (stopped) closeHandles();
+    return result;
+  };
   const listener = async ({
     data
   }) => {
     const { id, op } = data;
+    if (op === "ping") return send2({ id, pong: true });
+    if (op === "bye") return stop();
+    if (op === "cleanup")
+      return void Promise.allSettled([...pending]).then(() => answer(data.name, data.body)).catch(() => {
+      });
     if (op === "unwatch") return watches.get(id)?.abort();
     if (op === "watch") {
       const stop2 = new AbortController();
@@ -1671,32 +1698,35 @@ function serveHostfsPort(port, transport) {
     }
     try {
       if (op === "read") {
-        const read = await transport.read(
-          data.fh,
-          data.offset,
-          data.size,
-          data.ifMatch
+        const read = await track(
+          transport.read(
+            data.fh,
+            data.offset,
+            data.size,
+            data.ifMatch
+          )
         );
         const bytes = read.slice();
         return send2({ id, bytes }, [bytes.buffer]);
       }
       if (op === "write") {
-        await transport.write(data.fh, data.offset, data.bytes);
+        await track(
+          transport.write(data.fh, data.offset, data.bytes)
+        );
         return send2({ id });
       }
       if (op !== "call") throw fsError("EINVAL", `unknown hostfs port op ${op}`);
-      send2({
-        id,
-        result: await transport.call(data.name, data.body)
-      });
+      send2({ id, result: await answer(data.name, data.body) });
     } catch (err) {
       send2({ id, error: errorOf(err) });
     }
   };
   const stop = () => {
+    stopped = true;
     port.removeEventListener("message", listener);
     port.removeEventListener("close", stop);
     for (const watch of watches.values()) watch.abort();
+    closeHandles();
     port.close();
   };
   port.addEventListener("message", listener);