@ai-ecoverse/slicc-kernel 1.148.1 → 1.149.1

package/README.md

@@ -454,7 +454,7 @@ The module's default export (or its `main` export) is called with a context and
 - `net.connect({ host, port, tls: { servername, alpn, rejectUnauthorized, ca, path, cert, key, passphrase, sni, checkHostname } })` takes one of two paths. By default it's **A1, TLS through the transport** ([#914](https://github.com/ai-ecoverse/slicc-kernel/issues/914)), described first below. For a destination the uplink routes it can also be **A2, a TLS handshake in the kernel** ([#922](https://github.com/ai-ecoverse/slicc-kernel/issues/922)), described after it. `conn.tls.path` (`'transport'` or `'kernel'`) and `conn.tls.substitutes` say which one was taken.
   - **What it does:** there's no handshake in the kernel. The connection carries plain HTTP/1.1 (and WebSocket upgrades) to the realm proxy's HTTP path for the origin `https://<servername or host>[:port]`, exactly as a `CONNECT` through `127.0.0.1:3128` does after the kernel terminates its TLS. The transport (the page's `fetch` or WebSocket, or the uplink's) makes the real TLS connection and verifies it. So `https.request` over a socket, keep-alive agents, `agent-base` and `wss://` clients work with the transport's semantics. That includes sealed placeholders in headers, which reach the transport exactly as from `ctx.fetch`.
   - **The `tls` property:** the connection's `tls` is `{ path: 'transport', substitutes: true, protocol: 'transport', alpnProtocol: 'http/1.1', authorized: true, servername }`. `authorized` means the transport verified the server. There's no peer certificate chain and no TLS version of the kernel's own, so don't read it as a real handshake's result.
-  - **Options:** the transport always verifies, so `rejectUnauthorized: false` is accepted and changes nothing. Outside the uplink, a custom `ca` is refused (`ENOTSUP`), and so is an `alpn` list without `http/1.1` (`EPROTONOSUPPORT`). The kernel's own loopback (`localhost`, `*.localhost`, `127/8`, `0.0.0.0`, `::1`, as `servername` or as `host`) is `ENOTSUP` at connect, as `ctx.fetch` refuses `https:` and `wss:` there. Connect in plain text to a kernel server. `host.slicc.internal` goes to the transport under its policy, as for `ctx.fetch`.
+  - **Options:** the transport always verifies, so `rejectUnauthorized: false` is accepted and changes nothing. Outside the uplink, a custom `ca` is refused (`ENOTSUP`), and so is an `alpn` list without `http/1.1` (`EPROTONOSUPPORT`). The kernel's own loopback never goes to the transport. A `host` of `localhost` or `127.x` makes the handshake in the kernel instead (Loopback, below), so with nothing listening it's `ECONNREFUSED`. The rest of it (`*.localhost`, `0.0.0.0`, `::1`, and any loopback `servername` for another host) is `ENOTSUP` at connect, as `ctx.fetch` refuses `https:` and `wss:` there. `host.slicc.internal` goes to the transport under its policy, as for `ctx.fetch`.
   - **Non-HTTP protocols aren't carried:**
     - the well-known TLS ports of other protocols (465 SMTPS, 993 IMAPS, 995 POP3S, 636 LDAPS, 853, 8883 MQTT, 6697 IRC, 5671 AMQP and the like) are refused at `connect` with `EPROTONOSUPPORT`;
     - non-HTTP first bytes on any other port get a `400` and a close;
@@ -504,6 +504,15 @@ The module's default export (or its `main` export) is called with a context and
       - **The caller must then check the server's identity itself**, as slicc-jsh does with Node's `checkServerIdentity` and `peerCertificates[0]`; pass it only when the program supplied one.
       - With `rejectUnauthorized: false` it's `EINVAL`.
     - **STARTTLS:** `net.connect({ socket: conn, host, tls })` upgrades an open plain connection to an uplink destination in place, as Node's `tls.connect({ socket })` does. It resolves with a connection whose streams now carry the decrypted bytes and whose `tls` is the session. Don't hold a pending read on the plain connection while it upgrades. Pass `host` (or `tls.servername`): the plain connection only knows the address it reached, so without a name the certificate is checked against that IP, and a certificate for the host name fails.
+    - **Loopback:** `net.connect({ host, port, tls })` to `localhost` or a `127.x` address makes the handshake in the kernel too, over the loopback connection, with the same options as A2 (`ca`, `cert`/`key`, `sni`, `checkHostname`). Nothing on loopback is ever substituted (`substitutes: false`).
+    - **A TLS server** ([#1401](https://github.com/ai-ecoverse/slicc-kernel/issues/1401)), with the program's own certificate and key: `net.listen({ port, host, tls: { cert, key, passphrase, alpn, ca, requestCert, rejectUnauthorized } })` (and `sync.net.listen`) gives a listener whose connections are already secured, and `net.secure(conn, { server: true, cert, key, … })` secures one accepted connection. `net.secure(conn, tls)` without `server: true` is the client upgrade, as `connect({ socket })`.
+      - `cert` and `key` take the same forms as a client certificate, and are refused the same way, at `listen` (or `secure`) before anything is accepted: a key for another certificate is `ERR_OSSL_X509_KEY_VALUES_MISMATCH`, a wrong passphrase `ERR_OSSL_BAD_DECRYPT`.
+      - `alpn` is the server's list in order of preference; the first one the client also offers wins, and a client that offers ALPN but none of these fails the handshake.
+      - `requestCert` needs `ca` (the roots a client certificate must reach; else `EINVAL`). With it the handshake fails without a client certificate that verifies, unless `rejectUnauthorized: false`, which only asks for one and reports the verdict.
+      - A connection's `tls` is `{ path: 'kernel', protocol, alpnProtocol, servername (the SNI the client sent, or ''), authorized (whether a client certificate verified), authorizationError?, peerCertificates (the client's chain) }`.
+      - On a `listen({ tls })` listener, the kernel handshakes accepted connections concurrently, from the first `accept` on, and `accept` (and `for await`) yields each as its handshake finishes, so a client that connects and stays silent doesn't hold up the next. It holds at most `backlog` (up to 64) connections in a handshake or handshaken but not yet taken; beyond that it stops accepting, leaving the rest in the kernel's backlog, until one finishes or is taken. A connection whose handshake fails is closed and never yielded. `secure()` rejects instead.
+      - `handshakeTimeout` (milliseconds, 120000 by default, as Node's) ends a handshake that hasn't finished: the connection is closed with `ERR_TLS_HANDSHAKE_TIMEOUT`. A handshake, server or loopback client, waits without holding the program's other calls, and an aborted `accept({ signal })` or `connect({ signal })` ends it. A signal delivered to the program doesn't interrupt a handshake, as with `SA_RESTART`.
+      - **The key:** the kernel hands it to the engine once per listener (once per `secure()` call) and wipes its own copies; the engine keeps its copy until the listener is closed, and not in errors, logs, `ps`, `/proc` or the page. A forked child that inherits the listener shares that one copy, and it's freed when the last process closes it. **The program's own `key` string or `Buffer` stays in its realm**: JavaScript strings can't be wiped, so the kernel can't erase what the program holds.
     - **Sealed values in an A2 session** ([#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)) are substituted only at a credential position, only in a session the kernel verified itself (by its bundled roots, or by a vault anchor), and only as the vault allows. `substitutes` means the session **may** substitute: a placeholder at a recognized position will be offered to the vault, and whether a value comes back is the vault's and the anchor check's decision. It's true when all of these hold:
       - `verified` is true, or the engine can verify a chain on its own (wasm-tls-engine 4.2.0-5's `tls_chain_verify`) and the server name isn't an IP literal, so a vault anchor may verify the session (below);
       - ALPN didn't agree `h2`;
@@ -593,7 +602,7 @@ WASIX programs get `AF_INET`/`AF_INET6` `SOCK_DGRAM` sockets, and `AF_UNIX` ones
 - **Streamed request bodies** ([#238](https://github.com/ai-ecoverse/slicc-kernel/issues/238)): a request comes with its body whole, as `body` (a `Uint8Array`), unless the transport sets `traits.streamingBodies`. Then a body the kernel doesn't hold whole comes as `bodyStream`, an async iterable of `Uint8Array`, never together with `body`. That's an upload through the proxy (`Content-Length` or chunked, curl's `-T`, git's push) and a JS program's `fetch` with a stream body. The kernel reads from the program only as fast as the transport pulls, applies no `maxRequestBody` and reserves no memory for it. A transport with a limit of its own rejects with `status: 413`. When the transport answers before it has read the whole body, the proxy closes the program's connection after the response, since the rest of the body is still on it. A transport without the trait keeps getting whole bodies, capped at its `maxRequestBody`, so existing transports need no change. A transport that wraps another (a router, an injector, a logger) and passes on the inner `traits` must forward `bodyStream`, or not claim `streamingBodies`: a wrapper that rebuilds the request from `url`, `method`, `headers`, `body` and `signal` would drop a streamed upload and still answer. `nodeTransport()` streams (undici's `fetch` with `duplex: 'half'`). `fetchTransport({ streamingBodies: true })` turns it on for a page that knows its targets speak HTTP/2 or later, since Chrome streams a `fetch` upload only there, so it's off by default. The trait is also how an embedder's own transport, such as the `slicc` CLI link's exit, takes streamed uploads; across the page and the kernel's worker, and to attached clients, the body is pulled one chunk at a time. `fetchTransport()` is the plain `fetch` of the page, bound by CORS; its `hint` option is appended to the message of a request it could not make, so a program's 502 can name a way around CORS (`registry.npmjs.org` and jsDelivr allow it); an embedder with a way around CORS passes its own. A response body that delivers no data for `bodyIdleMs` (default 5 minutes; `0` or `Infinity` turns it off) fails with `ETIMEDOUT` instead of waiting forever, which matters because Chrome under heavy load can stop handing a page the rest of a body. A slow body that keeps delivering is never cut off.
 - **Traits of the embedder's own:** `traits` may carry more fields than the kernel reads. The kernel keeps them, and attached clients get them as they are (`client.transport.traits`, `onTraits`). In TypeScript, `NetworkTraits` types its own fields and leaves any other as `unknown`. One is typed: `exit`, the default exit an embedder routes through, `{ kind, caps }` (`NetworkExit`). `kind` is `'local'`, `'remote'` or `'tailnet'` (or another string), `caps` lists what the exit carries (`['net', 'http', 'ws']`), and further fields are `unknown`. When traits change later, call `kernel.updateTransportTraits()`.
 - **Redirect hops** ([#302](https://github.com/ai-ecoverse/slicc-kernel/issues/302)): a transport that follows redirects but can see them (a browser extension's `webRequest.onBeforeRedirect` can) sets `traits.redirectHops` and answers `redirects: [{ status, url, headers }]`, one entry per 3xx it followed, in order (`NetworkRedirectHop`). The kernel then treats it as one that does not follow. Programs get each 3xx as it was answered: through the proxy, curl, requests and `hf` see the real redirect, `X-Repo-Commit` and `ETag` included, and `ctx.fetch` with `redirect: 'manual'` answers it. The rest of the chain is held for the process that asked, at most 30 s and 8 chains per process (64 in all), with the final body unread. A follow-up from that process, with the method the redirect leads to and the same `Range`, `If-None-Match`, `If-Modified-Since`, `If-Range` and `Accept-Encoding` (and an `Authorization` if the first had one), is answered from it without a second request; any other request goes to the transport. A chain that expires or is pushed out is cancelled, and so is one whose follow-up arrives and does not match. After a redirect to another origin the follow-up is expected without `Authorization`, which HTTP clients drop when they follow one; a `Location` matches without its `#fragment`. The page's own `fetch` cannot show a redirect, so `fetchTransport()` in a page stays a transport that follows, and `redirect: 'manual'` fails there as before. `fetchTransport({ redirect: 'manual' })` hands redirects back unfollowed where the platform's `fetch` shows them (Node; a browser answers an opaque status 0), and `nodeTransport()` does so. A kernel socket also records who connected it (`SO_PEERCRED`): the connecting process's pid, uid and gid on the accepting side, and the listener's on the connecting side, kept when the socket passes to another process. The proxy uses this to keep each process's held chains its own.
-- **WebSockets:** a `GET` with `Upgrade: websocket`, in absolute form or inside a tunnel, is bridged to the transport's optional `websocket({ url, protocols, headers, signal })`. It answers `{ protocol, buffered, send(data), messages, close(code, reason), closed }`, with `messages` an async iterable of strings and `Uint8Array`s and `closed` resolving with `{ code, reason }`; a transport that has it says so with `traits.websocket`. The program gets `101 Switching Protocols` with the far end's protocol, then its frames go out as messages and the far end's come back as frames. Pings are answered by the proxy, and close codes pass both ways (a code that may not go on the wire becomes 1000 or 1011). A message over 64 MiB, or more than 16 MiB of payload from the far end waiting for the program, closes with 1009. The proxy stops reading the program while 1 MiB of its messages wait to be sent. A refused connection is `502` with the reason. A transport without WebSockets answers `501` with a message naming the option below, and the kernel logs a warning once, so a program never hangs. `fetchTransport({ webSocket: true })` opens them with the page's `WebSocket` (or pass a constructor); they are off by default because a browser WebSocket carries the user's cookies for the target site, which `fetch` here never sends (`credentials: 'omit'`), so a program could reach a site as the user. A page also cannot send headers on a WebSocket, and an `https:` page cannot open `ws:` to anything but loopback, which is a `502` that says so. `nodeTransport()` has WebSockets on (Node keeps no cookies) and sends the request's headers. An open fails as soon as Node's `WebSocket` reports an error, also on Node 22, which never follows the error with a close. A transport's `websocket()` may reject with an `Error` whose `code` is an errno, and the kernel hands that on to the program. Node's `WebSocket` doesn't say why an open failed, so when one fails, `nodeTransport()` finds out: it looks up the host's name (`EAI_NONAME` when that fails), then makes one TCP connection to the target, or a TLS one for `wss:`, for at most 3 s, and closes it at once. A refusal there is `ECONNREFUSED`, an unreachable host `EHOSTUNREACH` or `ENETUNREACH`, no answer `ETIMEDOUT`, and a TLS failure (a bad certificate) or a server that takes the connection but refused the upgrade `EPROTO`. A server that refused the upgrade therefore sees a second connection. The diagnosis never runs when an open succeeds, and never in a page (`fetchTransport()`), whose failed opens stay `ECONNREFUSED`. The credential-free path for a page is the coming slicc-node relay (`localProxyTransport()`, phase 2 of [#204](https://github.com/ai-ecoverse/slicc-kernel/issues/204)), which has no WebSockets yet. After the program sends its close frame, data the far end sends before its own close still reaches the program, and the far end's close frame follows once it has drained, as RFC 6455 §5.5.1 has it; a far end that doesn't close within 5 s gets the program's own close echoed back ([#1471](https://github.com/ai-ecoverse/slicc-kernel/issues/1471)).
+- **WebSockets:** a `GET` with `Upgrade: websocket`, in absolute form or inside a tunnel, is bridged to the transport's optional `websocket({ url, protocols, headers, signal })`. It answers `{ protocol, buffered, send(data), messages, close(code, reason), closed }`, with `messages` an async iterable of strings and `Uint8Array`s and `closed` resolving with `{ code, reason }`; a transport that has it says so with `traits.websocket`. The program gets `101 Switching Protocols` with the far end's protocol, then its frames go out as messages and the far end's come back as frames. Pings are answered by the proxy, and close codes pass both ways (a code that may not go on the wire becomes 1000 or 1011). A message over 64 MiB, or more than 16 MiB of payload from the far end waiting for the program, closes with 1009. The proxy stops reading the program while 1 MiB of its messages wait to be sent. A refused connection is `502` with the reason. A transport without WebSockets answers `501` with a message naming the option below, and the kernel logs a warning once, so a program never hangs. `fetchTransport({ webSocket: true })` opens them with the page's `WebSocket` (or pass a constructor); they are off by default because a browser WebSocket carries the user's cookies for the target site, which `fetch` here never sends (`credentials: 'omit'`), so a program could reach a site as the user. A page also cannot send headers on a WebSocket, and an `https:` page cannot open `ws:` to anything but loopback, which is a `502` that says so. `nodeTransport()` has WebSockets on (Node keeps no cookies) and sends the request's headers. An open fails as soon as Node's `WebSocket` reports an error, also on Node 22, which never follows the error with a close. A transport's `websocket()` may reject with an `Error` whose `code` is an errno, and the kernel hands that on to the program. Node's `WebSocket` doesn't say why an open failed, so when one fails, `nodeTransport()` finds out: it looks up the host's name (`EAI_NONAME` when that fails), then makes one TCP connection to the target, or a TLS one for `wss:`, for at most 3 s, and closes it at once. A refusal there is `ECONNREFUSED`, an unreachable host `EHOSTUNREACH` or `ENETUNREACH`, no answer `ETIMEDOUT`, and a TLS failure (a bad certificate) or a server that takes the connection but refused the upgrade `EPROTO`. A server that refused the upgrade therefore sees a second connection. The diagnosis never runs when an open succeeds, and never in a page (`fetchTransport()`), whose failed opens stay `ECONNREFUSED`. The credential-free path for a page is the coming slicc-node relay (`localProxyTransport()`, phase 2 of [#204](https://github.com/ai-ecoverse/slicc-kernel/issues/204)), which has no WebSockets yet. After the program sends its close frame, data the far end sends before its own close still reaches the program, and the far end's close frame follows once it has drained, as RFC 6455 §5.5.1 has it; a far end that doesn't close within 5 s gets the program's own close echoed back ([#1471](https://github.com/ai-ecoverse/slicc-kernel/issues/1471)). The same holds when the kernel runs in a worker and the page's transport carries the socket ([#1506](https://github.com/ai-ecoverse/slicc-kernel/issues/1506)).
 - **`localProxyTransport({ url, key })`** sends every request to a local proxy on loopback, such as [slicc-node](https://github.com/ai-ecoverse/slicc-node) or slicc-swift, which fetches it without CORS. Redirects reach the program unfollowed, with every `Set-Cookie`; bodies arrive decoded. `probeLocalProxy({ url, key })` resolves with the proxy's capabilities, or `null` when it is gone or refuses the key, so a page can fall back to `fetchTransport()`. `checkLocalProxy({ url, key })` says why: it resolves with `{ state }`, which is `ready` (with `probe`), `blocked` (the user denied Chrome's Local Network Access permission, `loopback-network`), `unreachable` (nothing answered; its `permission` is `granted`, `prompt` or `unknown`, and with `prompt` the page cannot tell a dismissed prompt from a proxy that is not running, since Chrome asks only once it has a connection), `refused` (with `status` and `error`, such as a stale key) or `incompatible`. The permission is not consulted when the page itself is on loopback. Both take a `fetch` option, and the transport `maxRequestBody` (default 64 MiB) and `bodyIdleMs` like `fetchTransport`.
 - **The host machine** is `host.slicc.internal` (`SLICC_HOST_LOOPBACK`), as `host.docker.internal` is in Docker, while `localhost` and `127.0.0.1` stay the kernel's own loopback. The proxy forwards `http(s)://host.slicc.internal:<port>/…` to the transport as `http(s)://127.0.0.1:<port>/…`, so it reaches the real machine's loopback through the local proxy or Node's `fetch` (through the page's own `fetch`, a request needs CORS from that server and Chrome's Local Network Access permission); other loopback addresses are still refused. This is HTTP and HTTPS through the proxy only: `/etc/hosts` and WASIX name resolution give the name `10.0.2.2`, QEMU's address for its host, so a program that connects a socket to it directly fails with `ENETUNREACH` instead of reaching the kernel's loopback.
 - **Without a transport**, every request is answered `502` with `slicc-kernel: no network transport (createKernel({ network: { transport } }))`.

package/dist/index.js

@@ -1227,6 +1227,7 @@ function serveTransport(peer, transport, { closeWaitMs = CLOSE_WAIT_MS } = {}) {
     const flow = {
       inflight: 0,
       closing: false,
+      dropped: false,
       deadline: () => AbortSignal.timeout(closeWaitMs).addEventListener("abort", late, { once: true })
     };
     credit.set(nid, flow);
@@ -1238,7 +1239,7 @@ function serveTransport(peer, transport, { closeWaitMs = CLOSE_WAIT_MS } = {}) {
         const next = await Promise.race([messages.next(), unanswered]);
         if (next === "late" || next.done) break;
         const data = next.value;
-        if (flow.closing) continue;
+        if (flow.dropped) continue;
         if (payloadSize(data) > MESSAGE_LIMIT) {
           oversize = true;
           break;
@@ -1292,6 +1293,8 @@ function serveTransport(peer, transport, { closeWaitMs = CLOSE_WAIT_MS } = {}) {
   };
   const drop = (nid) => {
     sockets.get(nid)?.close(1e3, "cancelled");
+    const flow = credit.get(nid);
+    if (flow) flow.dropped = true;
     refill(nid, 0, true);
     const entry = open.get(nid);
     open.delete(nid);