@ai-ecoverse/slicc-kernel 1.51.0 → 1.52.0

package/README.md

@@ -178,7 +178,7 @@ That's for programs that walk to the root, such as Go's (`GOOS=wasip1`) and esbu
 **Working directory.** wasi-libc's own `getcwd()` stays `/` (Rust's `current_dir()`, `canonicalize(".")`) in either layout, because the C library starts with a working directory of `/` and no runtime can change it. A wasi-libc program that needs its real directory takes it from `PWD` (`set_current_dir($PWD)` at start); relative paths are the cwd either way. WASIX programs ask the kernel for their cwd and get the real one.
 
 **Descriptor numbers.** Preopens take consecutive descriptors from 3, since WASI libraries look for them until the first gap:
-- without `preopenRoot`: fds 3 through 3 + *n*, *n* being the number of top-level directories, `/dev`, `/usr` and `/bin` together;
+- without `preopenRoot`: fds 3 through 3 + *n*, *n* being the number of top-level directories, `/dev`, `/proc`, `/usr` and `/bin` together;
 - with it: two more, through 3 + *n* + 2.
 
 A descriptor a program inherits at one of those numbers (from bash's `3<file`, or a `posix_spawn` file action) moves to the first free number above them. One above the range keeps its number. A command that takes descriptors by number from its caller should leave `preopenRoot` off, which keeps the shorter range.
@@ -253,7 +253,7 @@ await kernel.run(['curl', '-sS', 'https://registry.npmjs.org/@ai-ecoverse/wasm-b
 - **The proxy** listens on `127.0.0.1:3128` from the first connection on. It takes absolute-form `http:` requests and `CONNECT` tunnels, which it terminates with a certificate for the requested host, issued by the kernel's own CA, then hands each request to the transport. Programs start with `http_proxy`, `https_proxy` (and the upper-case names) pointing at it, `no_proxy` covering loopback and the [node name](#node-name), and `SSL_CERT_FILE`, `CURL_CA_BUNDLE` and `GIT_SSL_CAINFO` pointing at the CA bundle in `/etc/ssl/certs/slicc-kernel-ca.pem`; variables in `env` override them. The CA's key never leaves WebCrypto; it is kept in IndexedDB, in `<metadata>-ca`, or in `slicc-kernel-ca` when `metadata` is the default or `false`, so kernels on one origin share one CA whether or not they keep metadata. `ca` names another database, and `ca: false` keeps a CA in memory for the kernel's lifetime. Each kernel adds its CA to the bundle instead of replacing it, so every kernel on the origin stays trusted; the bundle keeps the newest 16.
 - **TLS** needs [`@ai-ecoverse/wasm-tls-engine`](https://www.npmjs.com/package/@ai-ecoverse/wasm-tls-engine) installed under `modules`, like a command. Without it, `CONNECT` to port 443 is answered `501` with the reason. A tunnel to another port is accepted, and what the program sends first decides: plain HTTP is served as `http://host:port`, without TLS, and a TLS handshake is terminated as above, or the tunnel closes when it cannot be.
 - **The transport** is an object on the page with `traits` (`manualRedirects`, `encodedBodies`, `maxRequestBody`, and `crossOrigin`: `'cors'` or `'any'`) and `fetch(request)`, which answers `{ status, statusText, headers, body, cancel }` with `body` an async iterable of `Uint8Array`, and optionally `url` and `redirected: true` when it followed redirects itself (`fetchTransport()` does), so a JS program's `fetch` reports where it ended. The kernel pulls the body one chunk at a time, so a slow program slows the download. A rejection with a numeric `status` is answered with that status; any other with `502`. `fetchTransport()` is the plain `fetch` of the page, bound by CORS; its `hint` option is appended to the message of a request it could not make, so a program's 502 can name a way around CORS (`registry.npmjs.org` and jsDelivr allow it); an embedder with a way around CORS passes its own. A response body that delivers no data for `bodyIdleMs` (default 5 minutes; `0` or `Infinity` turns it off) fails with `ETIMEDOUT` instead of waiting forever, which matters because Chrome under heavy load can stop handing a page the rest of a body. A slow body that keeps delivering is never cut off.
-- **Redirect hops** ([#302](https://github.com/ai-ecoverse/slicc-kernel/issues/302)): a transport that follows redirects but can see them (a browser extension's `webRequest.onBeforeRedirect` can) sets `traits.redirectHops` and answers `redirects: [{ status, url, headers }]`, one entry per 3xx it followed, in order (`NetworkRedirectHop`). The kernel then treats it as one that does not follow. Programs get each 3xx as it was answered: through the proxy, curl, requests and `hf` see the real redirect, `X-Repo-Commit` and `ETag` included, and `ctx.fetch` with `redirect: 'manual'` answers it. The rest of the chain is held for the process that asked, at most 30 s and 8 chains per process (64 in all), with the final body unread. A follow-up from that process, with the method the redirect leads to and the same `Range`, `If-None-Match`, `If-Modified-Since`, `If-Range` and `Accept-Encoding` (and an `Authorization` if the first had one), is answered from it without a second request; any other request goes to the transport. A chain that expires or is pushed out is cancelled. The page's own `fetch` cannot show a redirect, so `fetchTransport()` in a page stays a transport that follows, and `redirect: 'manual'` fails there as before. `fetchTransport({ redirect: 'manual' })` hands redirects back unfollowed where the platform's `fetch` shows them (Node; a browser answers an opaque status 0), and `nodeTransport()` does so. A kernel socket also records who connected it (`SO_PEERCRED`): the connecting process's pid, uid and gid on the accepting side, and the listener's on the connecting side, kept when the socket passes to another process. The proxy uses this to keep each process's held chains its own.
+- **Redirect hops** ([#302](https://github.com/ai-ecoverse/slicc-kernel/issues/302)): a transport that follows redirects but can see them (a browser extension's `webRequest.onBeforeRedirect` can) sets `traits.redirectHops` and answers `redirects: [{ status, url, headers }]`, one entry per 3xx it followed, in order (`NetworkRedirectHop`). The kernel then treats it as one that does not follow. Programs get each 3xx as it was answered: through the proxy, curl, requests and `hf` see the real redirect, `X-Repo-Commit` and `ETag` included, and `ctx.fetch` with `redirect: 'manual'` answers it. The rest of the chain is held for the process that asked, at most 30 s and 8 chains per process (64 in all), with the final body unread. A follow-up from that process, with the method the redirect leads to and the same `Range`, `If-None-Match`, `If-Modified-Since`, `If-Range` and `Accept-Encoding` (and an `Authorization` if the first had one), is answered from it without a second request; any other request goes to the transport. A chain that expires or is pushed out is cancelled, and so is one whose follow-up arrives and does not match. A redirect to another origin is usually fetched again rather than replayed: HTTP clients drop `Authorization` when they follow one, so the follow-up no longer matches. The page's own `fetch` cannot show a redirect, so `fetchTransport()` in a page stays a transport that follows, and `redirect: 'manual'` fails there as before. `fetchTransport({ redirect: 'manual' })` hands redirects back unfollowed where the platform's `fetch` shows them (Node; a browser answers an opaque status 0), and `nodeTransport()` does so. A kernel socket also records who connected it (`SO_PEERCRED`): the connecting process's pid, uid and gid on the accepting side, and the listener's on the connecting side, kept when the socket passes to another process. The proxy uses this to keep each process's held chains its own.
 - **WebSockets:** a `GET` with `Upgrade: websocket`, in absolute form or inside a tunnel, is bridged to the transport's optional `websocket({ url, protocols, headers, signal })`. It answers `{ protocol, buffered, send(data), messages, close(code, reason), closed }`, with `messages` an async iterable of strings and `Uint8Array`s and `closed` resolving with `{ code, reason }`; a transport that has it says so with `traits.websocket`. The program gets `101 Switching Protocols` with the far end's protocol, then its frames go out as messages and the far end's come back as frames. Pings are answered by the proxy, and close codes pass both ways (a code that may not go on the wire becomes 1000 or 1011). A message over 64 MiB, or more than 16 MiB of payload from the far end waiting for the program, closes with 1009. The proxy stops reading the program while 1 MiB of its messages wait to be sent. A refused connection is `502` with the reason. A transport without WebSockets answers `501` with a message naming the option below, and the kernel logs a warning once, so a program never hangs. `fetchTransport({ webSocket: true })` opens them with the page's `WebSocket` (or pass a constructor); they are off by default because a browser WebSocket carries the user's cookies for the target site, which `fetch` here never sends (`credentials: 'omit'`), so a program could reach a site as the user. A page also cannot send headers on a WebSocket, and an `https:` page cannot open `ws:` to anything but loopback, which is a `502` that says so. `nodeTransport()` has WebSockets on (Node keeps no cookies) and sends the request's headers. The credential-free path for a page is the coming slicc-node relay (`localProxyTransport()`, phase 2 of [#204](https://github.com/ai-ecoverse/slicc-kernel/issues/204)), which has no WebSockets yet.
 - **`localProxyTransport({ url, key })`** sends every request to a local proxy on loopback, such as [slicc-node](https://github.com/ai-ecoverse/slicc-node) or slicc-swift, which fetches it without CORS. Redirects reach the program unfollowed, with every `Set-Cookie`; bodies arrive decoded. `probeLocalProxy({ url, key })` resolves with the proxy's capabilities, or `null` when it is gone or refuses the key, so a page can fall back to `fetchTransport()`. `checkLocalProxy({ url, key })` says why: it resolves with `{ state }`, which is `ready` (with `probe`), `blocked` (the user denied Chrome's Local Network Access permission, `loopback-network`), `unreachable` (nothing answered; its `permission` is `granted`, `prompt` or `unknown`, and with `prompt` the page cannot tell a dismissed prompt from a proxy that is not running, since Chrome asks only once it has a connection), `refused` (with `status` and `error`, such as a stale key) or `incompatible`. The permission is not consulted when the page itself is on loopback. Both take a `fetch` option, and the transport `maxRequestBody` (default 64 MiB) and `bodyIdleMs` like `fetchTransport`.
 - **The host machine** is `host.slicc.internal` (`SLICC_HOST_LOOPBACK`), as `host.docker.internal` is in Docker, while `localhost` and `127.0.0.1` stay the kernel's own loopback. The proxy forwards `http(s)://host.slicc.internal:<port>/…` to the transport as `http(s)://127.0.0.1:<port>/…`, so it reaches the real machine's loopback through the local proxy or Node's `fetch` (through the page's own `fetch`, a request needs CORS from that server and Chrome's Local Network Access permission); other loopback addresses are still refused. This is HTTP and HTTPS through the proxy only: `/etc/hosts` and WASIX name resolution give the name `10.0.2.2`, QEMU's address for its host, so a program that connects a socket to it directly fails with `ENETUNREACH` instead of reaching the kernel's loopback.
@@ -494,7 +494,7 @@ Programs ask the kernel for their ids:
 
 ### `/proc`
 
-Every process of the kernel has `/proc/<pid>/` with `cmdline`, `comm`, `stat`, `statm` and `status`, in the formats procps reads, and `/proc/self` is a link to the `/proc/<pid>` of the process that reads it, as on Linux (`readlink /proc/self` is its pid, the one it was exec'd under included). They come from the kernel's process table when they are opened, so a terminal sees the processes of every client; an exec'd program shows under the pid its parent knows. `/proc/uptime`, `/proc/loadavg`, `/proc/stat` and `/proc/meminfo` are there too, and `/proc/mounts`. A process with no parent in the kernel (one a page, a client or the Node entry started) has parent 1, as children of init do, in `getppid()`, `/proc/<pid>/stat` and `ps` alike, and so does a process whose parent has ended, waited for or not (`getppid()` asks the kernel each time; a shell's `$PPID` keeps the value it started with, as on Linux). Pids, parents, process groups, sessions, command lines, terminals, start and boot times are real, and so are the credentials in `status` (`Uid:`, `Gid:`, `Groups:`) and each process's memory: the size of its wasm memory (`VSZ` and `RSS` alike, as wasm has no paging), which `/proc/meminfo` counts as used. CPU times, load and `MemTotal` (from `navigator.deviceMemory`, else 4 GiB) are placeholders. A process that isn't root sees only its own user's processes (see [Users](#users)). Only Emscripten processes see this `/proc`.
+Every process of the kernel has `/proc/<pid>/` with `cmdline`, `comm`, `stat`, `statm` and `status`, in the formats procps reads, and `/proc/self` is a link to the `/proc/<pid>` of the process that reads it, as on Linux (`readlink /proc/self` is its pid, the one it was exec'd under included). They come from the kernel's process table when they are opened, so a terminal sees the processes of every client; an exec'd program shows under the pid its parent knows. `/proc/uptime`, `/proc/loadavg`, `/proc/stat` and `/proc/meminfo` are there too, and `/proc/mounts`. A process with no parent in the kernel (one a page, a client or the Node entry started) has parent 1, as children of init do, in `getppid()`, `/proc/<pid>/stat` and `ps` alike, and so does a process whose parent has ended, waited for or not (`getppid()` asks the kernel each time; a shell's `$PPID` keeps the value it started with, as on Linux). Pids, parents, process groups, sessions, command lines, terminals, start and boot times are real, and so are the credentials in `status` (`Uid:`, `Gid:`, `Groups:`) and each process's memory: the size of its wasm memory (`VSZ` and `RSS` alike, as wasm has no paging), which `/proc/meminfo` counts as used. CPU times, load and `MemTotal` (from `navigator.deviceMemory`, else 4 GiB) are placeholders. A process that isn't root sees only its own user's processes (see [Users](#users)). WASI and WASIX programs see the same `/proc`, read-only (opening a file in it for writing fails with `EROFS`), and `/` lists it, so a WASI program gets a preopen for it; `/proc/self` is the program's own pid there too. Neither has `/proc/<pid>/fd` yet.
 
 The kernel writes `/etc/passwd` and `/etc/group` when it starts (see [Users](#users)). While the root has none of them, or no `/etc/mtab`, reading them gets per-process ones: root is uid 0 and the realm user, uid 1000, is named after `USER`, or `web_user` (Emscripten's default) without one, so it matches `$USER`, with `HOME` as its home, so `id`, `whoami`, `ls -l` and `ps` show names. Nothing is written to OPFS, and a real file there wins.
 

package/dist/kernel-worker.js

@@ -6239,6 +6239,7 @@ var DriverConnection = class _DriverConnection {
 // src/mount/mount-fs.ts
 var MAX_LINKS4 = 40;
 var STALE_RETRIES = 3;
+var SETTLE_ROUNDS = 4;
 var DEFAULT_IO = 8 * 1024 * 1024;
 var UNITS = { "": 1, k: 1024, m: 1024 ** 2, g: 1024 ** 3, t: 1024 ** 4 };
 function parseSize(text) {
@@ -6556,6 +6557,7 @@ var MountTable = class {
       maxFile: maxFile ?? caps.maxFile ?? 0,
       attrs: /* @__PURE__ */ new Map(),
       lists: /* @__PURE__ */ new Map(),
+      epoch: 0,
       dispose: () => opened2.dispose(),
       ...opened2.present ? { present: opened2.present } : {}
     };
@@ -6610,6 +6612,7 @@ var MountTable = class {
     const session = this.sessions.get(real);
     if (!session) return this.committing.get(real)?.done;
     this.sessions.delete(real);
+    session.mount.epoch++;
     const done4 = (async () => {
       try {
         await session.mount.conn.call({ op: "release", fh: await session.fh });
@@ -6662,19 +6665,30 @@ var MountTable = class {
     this.deps.changed?.(...full);
   }
   forget(mount, path) {
+    mount.epoch++;
     for (const key of [...mount.attrs.keys()]) if (within4(key, path)) mount.attrs.delete(key);
     mount.lists.delete(path);
     mount.lists.delete(parentOf5(path));
     for (const key of [...mount.lists.keys()]) if (within4(key, path)) mount.lists.delete(key);
   }
-  async getattr(mount, path, rel2) {
-    if (mount.conn.failure) throw mount.conn.failure;
-    const cached = mount.attrs.get(path);
-    if (cached && cached.expires > Date.now()) return cached.value;
-    const attr = await mount.conn.call({ op: "getattr", path: rel2 });
-    const ttl = mount.caps.attrTtl ?? 1e3;
-    if (ttl > 0) mount.attrs.set(path, { value: attr, expires: Date.now() + ttl });
-    return attr;
+  async getattr(mount, path, rel2, settled = false) {
+    for (let round = 1; ; round++) {
+      if (mount.conn.failure) throw mount.conn.failure;
+      if (settled) await this.committing.get(path)?.done.catch(() => void 0);
+      const cached = mount.attrs.get(path);
+      if (cached && cached.expires > Date.now()) return cached.value;
+      const seen = mount.epoch;
+      const session = this.sessions.get(path);
+      const attr = await mount.conn.call({ op: "getattr", path: rel2 });
+      if (mount.epoch !== seen) {
+        const moved2 = this.sessions.get(path) !== session || this.committing.has(path);
+        if (settled && moved2 && round < SETTLE_ROUNDS) continue;
+        return attr;
+      }
+      const ttl = mount.caps.attrTtl ?? 1e3;
+      if (ttl > 0) mount.attrs.set(path, { value: attr, expires: Date.now() + ttl });
+      return attr;
+    }
   }
   async readdir(mount, path, rel2) {
     if (mount.conn.failure) throw mount.conn.failure;
@@ -6955,7 +6969,7 @@ var MountFs = class {
     const real = normalizePath(path);
     if (this.table.writing(real)?.truncated) await this.table.commit(real);
     return this.statWriting(
-      await this.table.getattr(found.mount, real, found.rel),
+      await this.table.getattr(found.mount, real, found.rel, true),
       real,
       found.mount
     );
@@ -10746,6 +10760,7 @@ function replayingHops(inner, heldMs = HELD_MS) {
         clearTimeout(waiting.timer);
         return replay(waiting);
       }
+      if (waiting) drop(key);
       const response = await inner.fetch(request2);
       if (!response.redirects?.length) return response;
       return replay({