@ai-ecoverse/slicc-kernel 1.170.3 → 1.170.4

package/README.md

@@ -479,7 +479,7 @@ The module's default export (or its `main` export) is called with a context and
 - `crontab({ action, user?, text? })`, what the [`crontab`](#cron) command calls: `action` is `'list'`, `'remove'` or `'install'` (with the table as `text`), on the caller's own table or, for an effective uid of 0, on `user`'s. It resolves with `{ text }` (the table for `list`, `''` otherwise) or `{ error }`, the message to print. A malformed call is `EINVAL`;
 - `net`: `listen({ port, host, backlog })` and `connect({ host, port, signal })` (`host` defaults to `127.0.0.1`) on the kernel's loopback, the same sockets WASIX programs use, so a JS server is reached by other processes, by the page as `<port>.kernel.localhost` (`loopbackFetch`, `dial`) and by attached clients. A listener has its `port` (as bound, so port `0` picks a free one), `accept({ signal })`, `close()` and `for await` over its connections. A connection has `local` and `remote` addresses, `readable` and `writable` web streams (closing the writable shuts down the sending side, so the peer reads the end), `close()`, and `reset()`, which closes it with an RST as Node's `resetAndDestroy()` does ([#1530](https://github.com/ai-ecoverse/slicc-kernel/issues/1530)): on the kernel's loopback, the peer first reads the bytes it has already received, as on Linux and BSD ([#1577](https://github.com/ai-ecoverse/slicc-kernel/issues/1577)), and then its reads fail with `ECONNRESET` instead of seeing the end; its writes fail with `ECONNRESET` at once. A TLS connection (`secure()`, a `tls` connect on the loopback) resets the same way. On an uplink connection `reset()` closes at once with a FIN, as the transport can't send an RST: bytes still buffered for the peer are discarded instead of sent. The loopback is IPv4 (`127.0.0.1` by default, or `0.0.0.0`), so an IPv6 host such as `::1` fails with `EADDRNOTAVAIL` for both `listen` and `connect`, and binding and connecting fail as they would for a WASIX program (`EADDRINUSE`, `ECONNREFUSED`, `EADDRNOTAVAIL`). `sync.net.listen` takes the same options and gives the same listener, bound and listening when it returns, so its `port` is known at once, as Node's `listen()` needs ([#1398](https://github.com/ai-ecoverse/slicc-kernel/issues/1398)); it needs the sync lane. A blocked accept, read or write waits without holding anything in the kernel, so aborting it or closing the socket leaves nothing behind. A write to a connection the peer closed fails with `EPIPE` and no `SIGPIPE`;
 - `net.connect({ host, port, tls: { servername, alpn, rejectUnauthorized, ca, path, cert, key, passphrase, sni, checkHostname, keylog } })` takes one of two paths. By default it's **A1, TLS through the transport** ([#914](https://github.com/ai-ecoverse/slicc-kernel/issues/914)), described first below. For a destination the uplink routes it can also be **A2, a TLS handshake in the kernel** ([#922](https://github.com/ai-ecoverse/slicc-kernel/issues/922)), described after it. `conn.tls.path` (`'transport'` or `'kernel'`) and `conn.tls.substitutes` say which one was taken.
-  - **What it does:** there's no handshake in the kernel. The connection carries plain HTTP/1.1 (and WebSocket upgrades) to the realm proxy's HTTP path for the origin `https://<servername or host>[:port]`, exactly as a `CONNECT` through `127.0.0.1:3128` does after the kernel terminates its TLS. The transport (the page's `fetch` or WebSocket, or the uplink's) makes the real TLS connection and verifies it. So `https.request` over a socket, keep-alive agents, `agent-base` and `wss://` clients work with the transport's semantics. That includes sealed placeholders in headers, which reach the transport exactly as from `ctx.fetch`.
+  - **What it does:** there's no handshake in the kernel. The connection carries plain HTTP/1.1 (and WebSocket upgrades) to the realm proxy's HTTP path for the origin `https://<servername or host>[:port]`, exactly as a `CONNECT` through `127.0.0.1:3128` does after the kernel terminates its TLS. The transport (the page's `fetch` or WebSocket, or the uplink's) makes the real TLS connection and verifies it. So `https.request` over a socket, keep-alive agents, `agent-base` and `wss://` clients work with the transport's semantics. A request with a sealed placeholder is refused there as from `ctx.fetch` (see **The transport** under [`/dev/sealed`](#devsealed)).
   - **The `tls` property:** the connection's `tls` is `{ path: 'transport', substitutes: true, protocol: 'transport', alpnProtocol: 'http/1.1', authorized: true, servername }`. `authorized` means the transport verified the server. There's no peer certificate chain and no TLS version of the kernel's own, so don't read it as a real handshake's result.
   - **Options:** the transport always verifies, so `rejectUnauthorized: false` is accepted and changes nothing. Outside the uplink, a custom `ca` is refused (`ENOTSUP`), and so is an `alpn` list without `http/1.1` (`EPROTONOSUPPORT`). The kernel's own loopback never goes to the transport. A `host` of `localhost` or `127.x` makes the handshake in the kernel instead (Loopback, below), so with nothing listening it's `ECONNREFUSED`. The rest of it (`*.localhost`, `0.0.0.0`, `::1`, and any loopback `servername` for another host) is `ENOTSUP` at connect, as `ctx.fetch` refuses `https:` and `wss:` there. `host.slicc.internal` goes to the transport under its policy, as for `ctx.fetch`.
   - **Non-HTTP protocols aren't carried:**
@@ -487,7 +487,7 @@ The module's default export (or its `main` export) is called with a context and
     - non-HTTP first bytes on any other port get a `400` and a close;
     - a server-speaks-first protocol on another port (the client sends nothing) waits for the proxy's 120 s idle limit.
     - For a destination the uplink routes, these cases take A2 instead (below).
-  - **Which path** (A2 needs a destination the uplink routes, `net.destination(host) === 'uplink'`; an uplink-routed host on 443 with default options stays on A1, so its sealed placeholders keep working):
+  - **Which path** (A2 needs a destination the uplink routes, `net.destination(host) === 'uplink'`; an uplink-routed host on 443 with default options stays on A1, where a request with a sealed placeholder is refused):
 
     | The call | Uplink destination | Any other destination |
     |---|---|---|
@@ -1133,6 +1133,12 @@ The kernel then asks the hook `{ v: 1, op: 'materializeConsume', ticket, stamp,
 
 The kernel never asks for a session that isn't `verified`, whose handshake is more than 60 s old (or in the future), or whose fields aren't of that shape. The reply is `{ values: { [placeholder]: value } }` within 10 s; the kernel takes only the placeholders it asked for, with a non-empty value free of CR, LF and NUL, and anything else (an error such as `{ v: 1, error: { code } }`, a timeout, a malformed reply, no hook) means nothing is substituted: the placeholders go out as written.
 
+**The transport.** Nothing fills a placeholder in a request that leaves through the transport, so the kernel refuses that request itself and never hands it to the transport. This covers the realm proxy (`curl`, `CONNECT`, TLS on A1), `ctx.fetch`, `ctx.websocket` and a client's `fetch` and `transport.websocket`. The check needs no vault and no `sealed` hook: anything of the form `fnox_proxy_<24 hex>` is refused, whether or not the vault knows it.
+
+- **Where it looks:** the URL (host, user info, path, query and fragment, percent-encoded or not), header names and values, WebSocket subprotocols, and `Basic` credentials, decoded for the check (the user, the password, or a credential that is malformed or not UTF-8). A `Basic` credential without a placeholder goes out byte for byte. Bodies aren't read: a placeholder in a body goes out as written.
+- **What a refusal looks like:** the realm proxy answers `403` with the reason in the body, for a WebSocket upgrade too. `ctx.fetch` and `ctx.websocket` reject with a `TypeError` whose `cause.code` is `EACCES` and, for `ctx.fetch`, whose `cause.detail` is the reason. A client's `fetch` and `transport.websocket` reject with `code` `EACCES` and `status` `403`. Nothing is sent, retried or sent another way, and a held redirect isn't replayed for it.
+- **What stays as it was:** the kernel's own loopback, A2 sessions (above) and raw sockets never reach this check.
+
 ### `/dev/clipboard`
 
 A character device (`0666`), there in every `users` mode, through which programs read and write the embedder's clipboard (`createKernel({ clipboard: { read, write } })`; on the page, bios answers with `navigator.clipboard`). Emscripten programs (`cat /dev/clipboard`, `cat > /dev/clipboard`, see it in `/dev`, reach it by any path that leads to it, and see it as a character device after fork and exec), WASI programs (`std::fs`) and JS programs (`ctx.fs.readFile`, `ctx.fs.writeFile`, `ctx.open`) all open it.

package/dist/kernel-worker.js

@@ -8498,7 +8498,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.170.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.170.4" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -10392,6 +10392,78 @@ var CronService = class {
   }
 };
 
+// src/kernel/net/placeholder-floor.ts
+var PLACEHOLDER = /fnox_proxy_[0-9a-f]{24}/;
+var ESCAPE = /%([0-9a-f]{2})/gi;
+var BASIC = /\bbasic\b(.*)/is;
+var PERCENT_LAYERS = 8;
+var BASE64_GROUP = 4;
+var PLACEHOLDER_REFUSED = "refused: this request has a sealed placeholder in its URL, a header or a Basic credential, and nothing fills it, so it was not sent";
+var PlaceholderRefused = class extends Error {
+  status = 403;
+  code = "EACCES";
+  constructor() {
+    super(PLACEHOLDER_REFUSED);
+  }
+};
+function carries(text) {
+  let layer = text;
+  for (let i = 0; i < PERCENT_LAYERS; i++) {
+    if (PLACEHOLDER.test(layer)) return true;
+    const next3 = layer.replace(
+      ESCAPE,
+      (_, hex4) => String.fromCharCode(Number.parseInt(hex4, 16))
+    );
+    if (next3 === layer) return false;
+    layer = next3;
+  }
+  return PLACEHOLDER.test(layer);
+}
+function basicCarries(value2) {
+  const credential = BASIC.exec(value2)?.[1];
+  if (credential === void 0) return false;
+  const letters = credential.replace(/[^A-Za-z0-9+/_-]/g, "").replaceAll("-", "+").replaceAll("_", "/");
+  for (let shift = 0; shift < BASE64_GROUP; shift++) {
+    const part = letters.slice(shift);
+    const whole = part.length % BASE64_GROUP === 1 ? part.slice(0, -1) : part;
+    if (carries(atob(whole))) return true;
+  }
+  return false;
+}
+function headersCarry(headers) {
+  return headers.some(
+    ([name2, value2]) => carries(name2.toLowerCase()) || carries(value2) || basicCarries(value2)
+  );
+}
+function urlCarries(url) {
+  return carries(url) || URL.canParse(url) && carries(new URL(url).href);
+}
+function refuse(request3) {
+  if (urlCarries(request3.url) || headersCarry(request3.headers)) throw new PlaceholderRefused();
+}
+function refuseSocket(request3) {
+  refuse(request3);
+  if (request3.protocols.some(carries)) throw new PlaceholderRefused();
+}
+function placeholderFloor(inner) {
+  const open2 = inner.websocket?.bind(inner);
+  return {
+    get traits() {
+      return inner.traits;
+    },
+    ...open2 ? {
+      websocket: async (request3) => {
+        refuseSocket(request3);
+        return open2(request3);
+      }
+    } : {},
+    async fetch(request3, owner) {
+      refuse(request3);
+      return inner.fetch(request3, owner);
+    }
+  };
+}
+
 // src/kernel/net/proxy-service.ts
 var SEND_BUFFER2 = 1024 * 1024;
 var TLS_HANDSHAKE = 22;
@@ -10898,7 +10970,8 @@ var RealmProxy = class {
       });
     } catch (e) {
       waiting.abort();
-      throw new HttpError(502, e instanceof Error ? e.message : String(e));
+      const status = e instanceof PlaceholderRefused ? e.status : 502;
+      throw new HttpError(status, e instanceof Error ? e.message : String(e));
     }
     ctx.idle.ms = Number.POSITIVE_INFINITY;
     try {
@@ -11112,7 +11185,7 @@ async function relay(socket, conn) {
 // src/fs/attest.ts
 var ATTEST_MAX = 1024 * 1024;
 var UNOWNED = 1e3;
-function refuse(code, path, why) {
+function refuse2(code, path, why) {
   throw Object.assign(new Error(`attest ${path}: ${why}`), { code });
 }
 function chainOf(path) {
@@ -11154,11 +11227,11 @@ async function trustedParents(fs, path, o) {
   const parents = [];
   for (const dir of chainOf(path).slice(0, -1)) {
     const st = await fs.lstat(dir);
-    if (st.isSymbolicLink) refuse("EACCES", path, `${dir} is a symlink`);
-    if (!st.isDirectory) refuse("ENOTDIR", path, `${dir} is not a directory`);
+    if (st.isSymbolicLink) refuse2("EACCES", path, `${dir} is a symlink`);
+    if (!st.isDirectory) refuse2("ENOTDIR", path, `${dir} is not a directory`);
     const perm = owned(dir, st);
     if (o.users !== "off" && (perm.uid !== 0 || (perm.mode & 18) !== 0)) {
-      refuse("EACCES", path, `${dir} is not root-owned or is writable by group or others`);
+      refuse2("EACCES", path, `${dir} is not root-owned or is writable by group or others`);
     }
     parents.push(perm);
   }
@@ -11166,27 +11239,27 @@ async function trustedParents(fs, path, o) {
 }
 async function attest(fs, asked, o) {
   if (typeof asked !== "string" || !asked.startsWith("/")) {
-    refuse("EINVAL", String(asked), "not an absolute path");
+    refuse2("EINVAL", String(asked), "not an absolute path");
   }
   const path = normalizePath(asked);
   const mount = o.mounts.find((target) => {
     const at = normalizePath(target);
     return at === "/" || under2(path, at);
   });
-  if (mount !== void 0) refuse("EACCES", path, `under the mount ${mount}`);
+  if (mount !== void 0) refuse2("EACCES", path, `under the mount ${mount}`);
   const parents = await trustedParents(fs, path, o);
   for (let tries = 0; ; tries++) {
     const before = await fs.lstat(path);
-    if (before.isSymbolicLink) refuse("EACCES", path, "is a symlink");
+    if (before.isSymbolicLink) refuse2("EACCES", path, "is a symlink");
     if (!before.isFile)
-      refuse(before.isDirectory ? "EISDIR" : "EINVAL", path, "is not a regular file");
-    if (before.size > ATTEST_MAX) refuse("EFBIG", path, `is over ${ATTEST_MAX} bytes`);
+      refuse2(before.isDirectory ? "EISDIR" : "EINVAL", path, "is not a regular file");
+    if (before.size > ATTEST_MAX) refuse2("EFBIG", path, `is over ${ATTEST_MAX} bytes`);
     const bytes = (await fs.readFileBuffer(path)).slice();
     const after = await fs.lstat(path);
     if (unchanged(before, after, bytes)) {
       return attested(path, after, bytes, await sha2562(bytes), o, parents);
     }
-    if (tries >= 1) refuse("EAGAIN", path, "changed while it was read");
+    if (tries >= 1) refuse2("EAGAIN", path, "changed while it was read");
   }
 }
 var WATCH_DEBOUNCE_MS = 50;
@@ -15754,7 +15827,7 @@ function encodeSabResult(result) {
           message: result.message,
           ...result.direct ? { direct: true } : {},
           ...result.dev !== void 0 ? { dev: result.dev } : {},
-          ...result.detail === "transport" ? { detail: result.detail } : {}
+          ...result.detail ? { detail: result.detail } : {}
         })
       )
     };
@@ -19216,22 +19289,28 @@ var HttpHandles = class {
       return response.redirected ? { ...head, json: { ...head.json, redirected: true } } : head;
     } catch (e) {
       entry.upload?.cancel();
-      const text = message2(e);
-      if (port !== void 0) return { ok: false, errno: codeOf2(e), message: text };
-      const status = e.status;
-      if (text === NO_TRANSPORT || typeof status !== "number" || status === 502) {
-        const code = codeOf2(e);
-        const errno3 = text === NO_TRANSPORT ? "ENETUNREACH" : code !== "EIO" ? code : "ECONNREFUSED";
-        const failed3 = e.detail === "transport";
-        return { ok: false, errno: errno3, message: text, ...failed3 ? { detail: "transport" } : {} };
-      }
-      entry.body = once(new TextEncoder().encode(text))[Symbol.asyncIterator]();
-      return this.headOf(handle, req.url, {
-        status,
-        statusText: "",
-        headers: [["content-type", "text/plain; charset=utf-8"]]
-      });
+      if (port !== void 0) return { ok: false, errno: codeOf2(e), message: message2(e) };
+      return this.failure(handle, req, e, entry);
+    }
+  }
+  failure(handle, req, e, entry) {
+    const text = message2(e);
+    if (e instanceof PlaceholderRefused) {
+      return { ok: false, errno: e.code, message: text, detail: "placeholder" };
     }
+    const status = e.status;
+    if (text === NO_TRANSPORT || typeof status !== "number" || status === 502) {
+      const code = codeOf2(e);
+      const errno3 = text === NO_TRANSPORT ? "ENETUNREACH" : code !== "EIO" ? code : "ECONNREFUSED";
+      const failed3 = e.detail === "transport";
+      return { ok: false, errno: errno3, message: text, ...failed3 ? { detail: "transport" } : {} };
+    }
+    entry.body = once(new TextEncoder().encode(text))[Symbol.asyncIterator]();
+    return this.headOf(handle, req.url, {
+      status,
+      statusText: "",
+      headers: [["content-type", "text/plain; charset=utf-8"]]
+    });
   }
   headOf(handle, url, response) {
     const { status, statusText, headers } = response;
@@ -22442,7 +22521,7 @@ var HEAD_MAX = 16 * 1024;
 var ANY_PLACEHOLDER = /fnox_proxy_[0-9a-f]{24}/g;
 var REQUEST_LINE2 = /^([!#$%&'*+.^_`|~0-9A-Za-z-]+) ([\x21-\x7e]+) HTTP\/1\.[01]$/;
 var HEADER_LINE = /^([!#$%&'*+.^_`|~0-9A-Za-z-]+):[ \t]*(.*?)[ \t]*$/;
-var BASIC = /^(Basic[ \t]+)([A-Za-z0-9+/]+={0,2})$/i;
+var BASIC2 = /^(Basic[ \t]+)([A-Za-z0-9+/]+={0,2})$/i;
 var encoder8 = new TextEncoder();
 function binary(text) {
   let out = "";
@@ -22458,7 +22537,7 @@ function sniffHttp1(start2) {
   return "no";
 }
 function basicOf(value2) {
-  const m = BASIC.exec(value2);
+  const m = BASIC2.exec(value2);
   if (!m) return void 0;
   try {
     return { scheme: m[1], decoded: atob(m[2]) };
@@ -29031,7 +29110,8 @@ function replayingHops(inner, heldMs = HELD_MS) {
       return { ...inner.traits, manualRedirects: true };
     },
     ...inner.websocket ? { websocket: inner.websocket.bind(inner) } : {},
-    async fetch(request3, owner = KERNEL) {
+    async fetch(request3, caller) {
+      const owner = caller ?? KERNEL;
       const key = keyOf3(owner, request3.method, request3.url);
       const waiting = held.get(key);
       if (waiting && waiting.fingerprint === fingerprint(request3.headers)) {
@@ -29040,7 +29120,7 @@ function replayingHops(inner, heldMs = HELD_MS) {
         return replay(waiting);
       }
       if (waiting) drop(key);
-      const response = await inner.fetch(request3);
+      const response = await inner.fetch(request3, caller);
       if (!response.redirects?.length) return response;
       return replay({
         owner,
@@ -31883,7 +31963,7 @@ var Launcher = class {
       ...options.env
     };
     this.ca = kernelCa(options.caStore ?? memoryCaStore());
-    this.transport = replayingHops(options.transport ?? missingTransport());
+    this.transport = placeholderFloor(replayingHops(options.transport ?? missingTransport()));
     this.announcedTraits = JSON.stringify(this.transport.traits);
     this.uplink = options.uplink;
     this.routes = new Routes(options.uplink?.traits.ipv6 === true);