@ai-ecoverse/slicc-kernel 1.170.4 → 1.171.0

package/README.md

@@ -179,7 +179,7 @@ Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker
 | `ps()` | the kernel's processes: `{ pid, ppid, pgid, sid, argv, tty, started, state, rss, utime, stime, uid, gid }`, `state` being `'S'`, `'T'` (stopped) or `'Z'`, `rss` the process's resident memory in bytes (the size of its wasm memory, what `/proc/<pid>/status` shows as `VmRSS`; absent from kernels before 1.61), `utime`/`stime` the CPU time it has used, in milliseconds, in its program and in the kernel's calls (see [`/proc`](#proc); absent for a JS program and from kernels before 1.63) and `uid`/`gid` the effective ids (absent from a 1.7 kernel), and `tag` the process's tag (see `kernel.run`), when it has one, with `tags`, all of them in order, when a spawn added one (see `kernel.connect`), on rows the client's user may see as root or as the same uid. A client bound to a user sees only that user's processes |
 | `kill(pid, signal?)` | a process, or a process group with a negative pid; no such process rejects with `code: 'ESRCH'`, and one the client's user may not signal with `code: 'EPERM'` |
 | `fs` | `readFile`, `readText`, `writeFile(path, string \| bytes)`, `stat`, `lstat`, `readdir`, `mkdir` (with parents), `rm(path, { force })` (recursive), `rename`, `realpath`, `symlink(target, path)`, `readlink`, `exists`, on the processes' file system, mounts included (a path under a `tmpfs`, `fsa` or `hostfs` mount reads and writes the mounted file system, as `cat` in a process would); failures reject with `KernelCallError` and a POSIX `code` |
-| `fetch({ url, method, headers, body, signal })` | through the kernel's network transport (the one the page passed to `createKernel`), with a streaming body; `transport` is the same as a `NetworkTransport` |
+| `fetch({ url, method, headers, body, redirect, signal })` | through the kernel's network transport (the one the page passed to `createKernel`), with a streaming body; `redirect` is the per-request policy (see [Network](#network)); `transport` is the same as a `NetworkTransport` |
 | `fs.watch(paths, { recursive }, onChange)` | resolves with `{ close() }`; `onChange` gets `{ paths }`, the changed paths at, below (or, without `recursive`, directly in) the watched ones, batched per task, or `{ overflow: true }` when there were too many to list, to rescan; it covers every change made through this kernel, by its processes and its clients, and a removal or rename that failed or found nothing to do is none |
 | `serveCdp(hook, { runtime })` | offers this client's browser to the kernel's programs (protocol 1.5): `hook` is the `cdp` hook of `createKernel`, asked once per program connection; resolves with `{ close() }`, and closing or detaching takes the offer back and closes its connections (see [Browser automation](#browser-automation-cdp)) |
 | `serveFacade(name, hook) → { close() }` | serves a facade that `createKernel({ facades })` declared, the same way as its `hook` (protocol 1.11); the latest registration wins, over the embedder's hook too, and `close()` hands it back. An undeclared name is `EINVAL` |
@@ -637,6 +637,7 @@ WASIX programs get `AF_INET`/`AF_INET6` `SOCK_DGRAM` sockets, and `AF_UNIX` ones
 - **Streamed request bodies** ([#238](https://github.com/ai-ecoverse/slicc-kernel/issues/238)): a request comes with its body whole, as `body` (a `Uint8Array`), unless the transport sets `traits.streamingBodies`. Then a body the kernel doesn't hold whole comes as `bodyStream`, an async iterable of `Uint8Array`, never together with `body`. That's an upload through the proxy (`Content-Length` or chunked, curl's `-T`, git's push) and a JS program's `fetch` with a stream body. The kernel reads from the program only as fast as the transport pulls, applies no `maxRequestBody` and reserves no memory for it. A transport with a limit of its own rejects with `status: 413`. When the transport answers before it has read the whole body, the proxy closes the program's connection after the response, since the rest of the body is still on it. A transport without the trait keeps getting whole bodies, capped at its `maxRequestBody`, so existing transports need no change. A transport that wraps another (a router, an injector, a logger) and passes on the inner `traits` must forward `bodyStream`, or not claim `streamingBodies`: a wrapper that rebuilds the request from `url`, `method`, `headers`, `body` and `signal` would drop a streamed upload and still answer. `nodeTransport()` streams (undici's `fetch` with `duplex: 'half'`). `fetchTransport({ streamingBodies: true })` turns it on for a page that knows its targets speak HTTP/2 or later, since Chrome streams a `fetch` upload only there, so it's off by default. The trait is also how an embedder's own transport, such as the `slicc` CLI link's exit, takes streamed uploads; across the page and the kernel's worker, and to attached clients, the body is pulled one chunk at a time. `fetchTransport()` is the plain `fetch` of the page, bound by CORS; its `hint` option is appended to the message of a request it could not make, so a program's 502 can name a way around CORS (`registry.npmjs.org` and jsDelivr allow it); an embedder with a way around CORS passes its own. A response body that delivers no data for `bodyIdleMs` (default 5 minutes; `0` or `Infinity` turns it off) fails with `ETIMEDOUT` instead of waiting forever, which matters because Chrome under heavy load can stop handing a page the rest of a body. A slow body that keeps delivering is never cut off.
 - **Traits of the embedder's own:** `traits` may carry more fields than the kernel reads. The kernel keeps them, and attached clients get them as they are (`client.transport.traits`, `onTraits`). In TypeScript, `NetworkTraits` types its own fields and leaves any other as `unknown`. One is typed: `exit`, the default exit an embedder routes through, `{ kind, caps }` (`NetworkExit`). `kind` is `'local'`, `'remote'` or `'tailnet'` (or another string), `caps` lists what the exit carries (`['net', 'http', 'ws']`), and further fields are `unknown`. When traits change later, call `kernel.updateTransportTraits()`.
 - **Redirect hops** ([#302](https://github.com/ai-ecoverse/slicc-kernel/issues/302)): a transport that follows redirects but can see them (a browser extension's `webRequest.onBeforeRedirect` can) sets `traits.redirectHops` and answers `redirects: [{ status, url, headers }]`, one entry per 3xx it followed, in order (`NetworkRedirectHop`). The kernel then treats it as one that does not follow. Programs get each 3xx as it was answered: through the proxy, curl, requests and `hf` see the real redirect, `X-Repo-Commit` and `ETag` included, and `ctx.fetch` with `redirect: 'manual'` answers it. The rest of the chain is held for the process that asked, at most 30 s and 8 chains per process (64 in all), with the final body unread. A follow-up from that process, with the method the redirect leads to and the same `Range`, `If-None-Match`, `If-Modified-Since`, `If-Range` and `Accept-Encoding` (and an `Authorization` if the first had one), is answered from it without a second request; any other request goes to the transport. A chain that expires or is pushed out is cancelled, and so is one whose follow-up arrives and does not match. After a redirect to another origin the follow-up is expected without `Authorization`, which HTTP clients drop when they follow one; a `Location` matches without its `#fragment`. The page's own `fetch` cannot show a redirect, so `fetchTransport()` in a page stays a transport that follows, and `redirect: 'manual'` fails there as before. `fetchTransport({ redirect: 'manual' })` hands redirects back unfollowed where the platform's `fetch` shows them (Node; a browser answers an opaque status 0), and `nodeTransport()` does so. A kernel socket also records who connected it (`SO_PEERCRED`): the connecting process's pid, uid and gid on the accepting side, and the listener's on the connecting side, kept when the socket passes to another process. The proxy uses this to keep each process's held chains its own.
+- **Per-request redirect policy:** `client.fetch` and `client.transport.fetch` take `redirect: 'follow' | 'manual' | 'error'` (`NetworkRedirectPolicy`), and it reaches the embedder's transport as `request.redirect`. Without it a transport does what it always did: `request.redirect ?? options.redirect ?? 'follow'`, so `nodeTransport()` and `fetchTransport({ redirect: 'manual' })` stay manual. The field is the caller's: requests from programs (`ctx.fetch`, the proxy) carry none. A `manual` or `error` request bypasses held redirect chains: it neither fills nor reads them, and its answer carries no `redirects`. An explicit `follow` is treated like no policy there, so a `redirectHops` transport still answers it one hop at a time. `error` turns a redirect into a transport error, as `fetch` does. `nodeTransport()` follows an explicit `follow` itself, hop by hop, as `fetch` would, and each hop passes the same loopback refusal as the first request, on a pooled connection too. A transport sets `traits.redirectPolicy` to say it honors the field, as `fetchTransport()` does. `traits.manualRedirects` keeps its meaning. A `manual` or `error` request to a transport that doesn't advertise it is refused before it is sent (`status` 502, `detail: 'transport'`), and so is one whose answer shows the transport followed anyway. The exception is `manual` on a transport with `manualRedirects` and no `redirectHops`, which never follows. An explicit `follow` to a transport that can't is ignored. Where a platform hides a manual redirect (a browser's opaque response), the response has `opaqueRedirect: true`, status 0 and no headers. It crosses to the client unchanged, and is never turned into a success or a 3xx. A response's `url` and `redirected` also cross as the transport gave them.
 - **WebSockets:** a `GET` with `Upgrade: websocket`, in absolute form or inside a tunnel, is bridged to the transport's optional `websocket({ url, protocols, headers, signal })`. It answers `{ protocol, buffered, send(data), messages, close(code, reason), closed }`, with `messages` an async iterable of strings and `Uint8Array`s and `closed` resolving with `{ code, reason }`; a transport that has it says so with `traits.websocket`. The program gets `101 Switching Protocols` with the far end's protocol, then its frames go out as messages and the far end's come back as frames. Pings are answered by the proxy, and close codes pass both ways (a code that may not go on the wire becomes 1000 or 1011). A message over 64 MiB, or more than 16 MiB of payload from the far end waiting for the program, closes with 1009. The proxy stops reading the program while 1 MiB of its messages wait to be sent. A refused connection is `502` with the reason. A transport without WebSockets answers `501` with a message naming the option below, and the kernel logs a warning once, so a program never hangs. `fetchTransport({ webSocket: true })` opens them with the page's `WebSocket` (or pass a constructor); they are off by default because a browser WebSocket carries the user's cookies for the target site, which `fetch` here never sends (`credentials: 'omit'`), so a program could reach a site as the user. A page also cannot send headers on a WebSocket, and an `https:` page cannot open `ws:` to anything but loopback, which is a `502` that says so. `nodeTransport()` has WebSockets on (Node keeps no cookies) and sends the request's headers. An open fails as soon as Node's `WebSocket` reports an error, also on Node 22, which never follows the error with a close. A transport's `websocket()` may reject with an `Error` whose `code` is an errno, and the kernel hands that on to the program. Node's `WebSocket` doesn't say why an open failed, so when one fails, `nodeTransport()` finds out: it looks up the host's name (`EAI_NONAME` when that fails), then makes one TCP connection to the target, or a TLS one for `wss:`, for at most 3 s, and closes it at once. A refusal there is `ECONNREFUSED`, an unreachable host `EHOSTUNREACH` or `ENETUNREACH`, no answer `ETIMEDOUT`, and a TLS failure (a bad certificate) or a server that takes the connection but refused the upgrade `EPROTO`. A server that refused the upgrade therefore sees a second connection. The diagnosis never runs when an open succeeds, and never in a page (`fetchTransport()`), whose failed opens stay `ECONNREFUSED`. The credential-free path for a page is a transport that opens them outside the page, such as one over a linked slicc CLI (`npx sliccy@next`). After the program sends its close frame, data the far end sends before its own close still reaches the program, and the far end's close frame follows once it has drained, as RFC 6455 §5.5.1 has it; a far end that doesn't close within 5 s gets the program's own close echoed back ([#1471](https://github.com/ai-ecoverse/slicc-kernel/issues/1471)). The same holds when the kernel runs in a worker and the page's transport carries the socket ([#1506](https://github.com/ai-ecoverse/slicc-kernel/issues/1506)).
 - **The host machine** is `host.slicc.internal` (`SLICC_HOST_LOOPBACK`), as `host.docker.internal` is in Docker, while `localhost` and `127.0.0.1` stay the kernel's own loopback. The proxy forwards `http(s)://host.slicc.internal:<port>/…` to the transport as `http(s)://127.0.0.1:<port>/…`, so it reaches the real machine's loopback through Node's `fetch` or a transport that runs on that machine (through the page's own `fetch`, a request needs CORS from that server and Chrome's Local Network Access permission); other loopback addresses are still refused. This is HTTP and HTTPS through the proxy only: `/etc/hosts` and WASIX name resolution give the name `10.0.2.2`, QEMU's address for its host, so a program that connects a socket to it directly fails with `ENETUNREACH` instead of reaching the kernel's loopback.
 - **Without a transport**, every request is answered `502` with `slicc-kernel: no network transport (createKernel({ network: { transport } }))`.

package/dist/driver-worker.js

@@ -302,6 +302,18 @@ var TransportError = class extends Error {
     this.detail = detail === "transport" ? detail : void 0;
   }
 };
+function unfollowed(redirect) {
+  return redirect !== void 0 && redirect !== "follow";
+}
+function honors(traits, redirect) {
+  if (traits.redirectPolicy === true) return true;
+  const hops = traits.redirectHops;
+  const raw = traits.manualRedirects === true && (hops === void 0 || hops === false);
+  return redirect === "manual" && raw;
+}
+function refused(message) {
+  return new TransportError(message, 502, void 0, "transport");
+}
 var RemoteTransport = class {
   traits;
   port;
@@ -387,8 +399,11 @@ var RemoteTransport = class {
   }
   async fetch(request) {
     const nid = ++this.nextId;
-    const { signal } = request;
+    const { signal, redirect } = request;
     signal.throwIfAborted();
+    if (unfollowed(redirect) && !honors(this.traits, redirect)) {
+      throw refused(`this transport cannot honor redirect: ${redirect}`);
+    }
     let finished = false;
     const abort = () => {
       finished = true;
@@ -411,13 +426,14 @@ var RemoteTransport = class {
         url: request.url,
         method: request.method,
         headers: request.headers,
-        ...request.bodyStream ? { streamed: true } : request.body ? { body: request.body } : {}
+        ...request.bodyStream ? { streamed: true } : request.body ? { body: request.body } : {},
+        ...redirect !== void 0 ? { redirect } : {}
       });
     } catch (e) {
       finish();
       throw e;
     }
-    const { status, statusText, headers, url, redirected, redirects } = head;
+    const { status, statusText, headers, url, redirected, redirects, opaqueRedirect } = head;
     const read = async () => {
       if (finished) return void 0;
       const reply = await this.ask({ net: "read", nid }).catch((e) => {
@@ -433,9 +449,15 @@ var RemoteTransport = class {
       finish();
       this.cancel(nid);
     };
+    if (unfollowed(redirect) && (redirected || redirects?.length)) {
+      await cancel();
+      throw refused(`this transport followed a redirect despite redirect: ${redirect}`);
+    }
     return {
-      ...redirected ? { url, redirected } : {},
+      ...url !== void 0 ? { url } : {},
+      ...redirected !== void 0 ? { redirected } : {},
       ...redirects ? { redirects } : {},
+      ...opaqueRedirect ? { opaqueRedirect } : {},
       status,
       statusText,
       headers,
@@ -1194,9 +1216,9 @@ async function greet(channel, lock, timeoutMs, requests) {
   } finally {
     clearTimeout(timer);
   }
-  const refused = hello.error ?? versionError(hello.protocol);
-  if (refused) {
-    const error = new Error(refused);
+  const refused2 = hello.error ?? versionError(hello.protocol);
+  if (refused2) {
+    const error = new Error(refused2);
     channel.end(error);
     throw error;
   }
@@ -1343,9 +1365,9 @@ function serveFilesystem(port, handlers, capabilities = {}, options = {}) {
   let open = false;
   const reply = (message, transfer = []) => port.postMessage(message, transfer);
   async function hello(message) {
-    const refused = driverVersionError(message.protocol);
-    if (refused) {
-      reply({ hello: { protocol: DRIVER_PROTOCOL, error: refused, errno: "EPROTO" } });
+    const refused2 = driverVersionError(message.protocol);
+    if (refused2) {
+      reply({ hello: { protocol: DRIVER_PROTOCOL, error: refused2, errno: "EPROTO" } });
       return;
     }
     try {