@ai-ecoverse/slicc-kernel 1.126.0 → 1.127.0

package/README.md

@@ -430,7 +430,7 @@ The module's default export (or its `main` export) is called with a context and
     - **`servername`** must be a host name (letters, digits and `-` in labels of at most 63, at most 253 in all, one trailing dot allowed) or an IP literal, else `EINVAL` before anything is dialled. An IP literal sends no SNI and must match an iPAddress subjectAltName.
     - **The `tls` property** is the handshake's: `{ path: 'kernel', substitutes: false, verified, protocol: 'TLSv1.3' or 'TLSv1.2', alpnProtocol (the agreed protocol, or false), authorized, authorizationError?, servername, peerCertificates, cipher, clientCertificate, sni, hostnameChecked, hostnameMatches? }`.
       - `peerCertificates` is the server's chain as DER, leaf first.
-      - `verified` is the kernel's own verdict on the session, whatever the caller's flags: the chain verifies against the kernel's **bundled roots** (never a `ca` the caller passed, which only decides `authorized`) and the leaf names `servername`, which is a host name, not an IP literal. `rejectUnauthorized: false`, `checkHostname: false` and `sni: false` don't change it; a custom `ca` makes it false even when the chain would pass both. It's what sealed substitution in an A2 session will be gated on ([#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)); until then `substitutes` stays false.
+      - `verified` is the kernel's own verdict on the session, whatever the caller's flags: the chain verifies against the kernel's **bundled roots** (never a `ca` the caller passed, which only decides `authorized`) and the leaf names `servername`, which is a host name, not an IP literal. `rejectUnauthorized: false`, `checkHostname: false` and `sni: false` don't change it; a custom `ca` makes it false even when the chain would pass both. Sealed substitution in an A2 session is gated on it ([#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)).
       - `cipher` is the negotiated suite as Node's `getCipher()` gives it: `{ name, standardName, version }`, where `name` is OpenSSL's (`ECDHE-RSA-AES128-GCM-SHA256`) and `standardName` IANA's. As in Node, `version` is the suite's own version, not the session's: `TLSv1.0` for a CBC-SHA suite on a TLS 1.2 session.
       - `clientCertificate` says whether a client certificate was offered. `sni` says whether the server name was sent: it's false with `sni: false` and for an IP-literal `servername`, which TLS never sends.
       - `hostnameChecked` says whether the kernel checked the leaf against `servername`. `hostnameMatches` says whether it names it.
@@ -454,7 +454,24 @@ The module's default export (or its `main` export) is called with a context and
       - **The caller must then check the server's identity itself**, as slicc-jsh does with Node's `checkServerIdentity` and `peerCertificates[0]`; pass it only when the program supplied one.
       - With `rejectUnauthorized: false` it's `EINVAL`.
     - **STARTTLS:** `net.connect({ socket: conn, host, tls })` upgrades an open plain connection to an uplink destination in place, as Node's `tls.connect({ socket })` does. It resolves with a connection whose streams now carry the decrypted bytes and whose `tls` is the session. Don't hold a pending read on the plain connection while it upgrades. Pass `host` (or `tls.servername`): the plain connection only knows the address it reached, so without a name the certificate is checked against that IP, and a certificate for the host name fails.
-    - **Sealed values aren't substituted.** The kernel encrypts end to end, so a placeholder written into an A2 connection reaches the server as written, unsubstituted (it isn't the secret, so nothing leaks). A caller that relies on substitution checks `conn.tls.substitutes` and uses `fetch`, or an A1 connection, for sealed values.
+    - **Sealed values in an A2 session** ([#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)) are substituted only at a credential position, only in a session the kernel verified itself, and only as the vault allows. `substitutes` is true when all of these hold:
+      - `verified` is true;
+      - ALPN didn't agree `h2`;
+      - a vault is connected;
+      - the uplink said which exit the connection leaves by (`UplinkConn.exit`: `tailscale` or `device:<id>`, the only exits an uplink connection takes). Without one, or with any other value, nothing is substituted.
+
+      Then:
+      - **HTTP/1.1:** if the **first request head** the program writes on the session (it starts with a standard method) has placeholders (`fnox_proxy_<24 hex>`) in exactly one header line, either in the value or in the `user:password` of `Basic` credentials, the kernel asks the vault once (`sessionMaterialize`, below) for that header's position.
+      - **What goes out:** the placeholders the vault answers with a value of at least 8 bytes go out as that value (`Basic` decoded, substituted and re-encoded). Everything else goes out as written:
+        - the request line and the body;
+        - other headers;
+        - a head with placeholders in two header lines;
+        - later requests on the same connection;
+        - anything once the head passes 16 KiB, or once the program pauses for 2 s in the middle of it;
+        - SMTP and pg, which come next.
+      - **What's read back:** once a process has had a value substituted for a host, everything its whole process tree reads from any A2 session to that host is scrubbed: the process itself, its ancestors and every descendant (forked, spawned or exec'd, before or after the substitution; a forked child's own new session included) with [slicc-scrub](https://github.com/ai-ecoverse/slicc-scrub), as S2 does: the value comes back as its placeholder, and a fragment or an encoded form (JSON, percent, HTML, base64, hex) as `[redacted:sealed]`. The scrub stays until the whole tree has ended. A process outside the tree isn't covered. A value shorter than 8 bytes is never substituted, as it couldn't be scrubbed.
+      - **The bound:** the server the value goes to is the one the vault rule names, verified against the kernel's own roots. Substitution trusts that server not to reflect the value back in a form the scrub doesn't know (encrypted under a key the program chose, say). That's the same residual as A1's.
+      - **Elsewhere:** a placeholder written into any other A2 session reaches the server as written. It isn't the secret, so nothing leaks.
     - **Limits:** no revocation checking (neither OCSP nor CRLs, as with Node's default), no session resumption, and the clock is the page's `Date.now()`.
 - `net.resolve(name, { family, signal })` asks the kernel's resolver, which WASIX `resolve` and Emscripten `getaddrinfo` use too: entries in a real `/etc/hosts`, the kernel's own names (`localhost`, the node name, `host.slicc.internal`) and then the uplink's DNS ([#913](https://github.com/ai-ecoverse/slicc-kernel/issues/913)). It answers the addresses, in the resolver's order, for `family` `4`, `6` or `0` (both, IPv4 first; the default). A name with no answer rejects with a `JsCallError` of code `ENOTFOUND`, as Node's `dns.lookup` does, another family with `EAI_FAMILY`, and an aborted `signal` with its reason.
 - `websocket(url, { protocols, headers, signal })`, a WebSocket over the kernel's transport (its `websocket`, so `fetchTransport({ webSocket: true })` in a browser), shaped like `WebSocketStream`. It resolves once the socket is open, with `url`, the `protocol` the far end chose (always one the program offered: a transport that hands back a socket with another fails the open with `EPROTO`, while browsers and Node's `WebSocket`, which `fetchTransport()` uses, fail such a handshake themselves without saying why, so the open fails with `ECONNREFUSED`), a `readable` of messages (`string` for text, `Uint8Array` for binary), a `writable` for sending them, `closed` (the `{ code, reason }` of the close, from either side, whether or not the program reads) and `close({ code, reason })`. An `http(s):` URL is taken as `ws(s):`. Sending waits while more than 1 MiB is buffered. A signal aborts the open, and after it the socket. A transport without WebSockets rejects with a `TypeError` whose `cause.code` is `ENOTSUP`. One that cannot connect rejects with the `code` its error carries (an errno, such as `EAI_NONAME`, `ECONNREFUSED`, `ETIMEDOUT` or `EPROTO`), else `ECONNREFUSED`. `ws://host.slicc.internal:port` reaches the real machine's loopback, as it does through the proxy;
@@ -648,6 +665,8 @@ The proxy answers `200` with `Content-Type: application/vnd.slicc.raw-fetch`: a
 
 `/` is the OPFS root. Each process mounts the top-level directories that exist when it starts (plus `/usr` and `/bin`), so files in them are shared by all processes and visible through the OPFS API as soon as the process that wrote them has closed them; `run` resolves only after that. `createKernel` creates `/tmp` and `/home` in OPFS, so they are shared too. Files directly in `/`, and directories created there after a process started, are OPFS as well, the same for Emscripten and WASI programs; only `/dev` and `/proc` live in each process's memory. A program's `mkdir` (WASI `path_create_directory`, Emscripten's `mkdir`) is exclusive, as on Linux: it fails with `EEXIST` when the name exists, and of several processes or threads making one name at once exactly one succeeds, so a directory works as a lock. Under a missing directory it fails with `ENOENT`, as on Linux (`mkdir -p` and `os.makedirs` make the parents in the program); a JS program's `ctx.fs.mkdir` makes parents itself. The kernel makes the system `PNPM_HOME` (`/usr/local/share/pnpm`, or the embedder's) when it starts, since pnpm makes only its `bin` directory there. A full origin storage (`QuotaExceededError`, which also comes early in a browser's private or off-the-record contexts, whatever `navigator.storage.estimate()` says) is `ENOSPC`, never retried. A read that meets `NotFoundError` while a writer outside the kernel swaps the file in with `createWritable` (and a `stat`, `lstat` or `utimes` that meets it) looks the name up again, up to three times, before it fails ([#1075](https://github.com/ai-ecoverse/slicc-kernel/issues/1075)), so a program whose package is overwritten in place keeps starting. An OPFS or IndexedDB failure that has no POSIX counterpart is `EIO`; its message names the DOM error and the operation (`lookup`, `read`, `write`, `commit` or `sidecar`), and the kernel worker logs the first of each kind to its console. File contents are buffered per open file and written back on close, `fsync` and exit, so a refused write (`ENOSPC`, `EIO`) surfaces at that close or `fsync`, and a program that ignores close errors doesn't see it (`echo x > f` can exit 0 with `f` empty, as with any buffered write-back). A write-back error a close has already reported to a WASI program isn't raised again when it exits, so the program keeps its own exit status. Metadata operations (`mkdir`, `rename`, `rm`, …) go straight to OPFS. The kernel worker is the only writer.
 
+**Renaming a directory on the OPFS root** ([#1122](https://github.com/ai-ecoverse/slicc-kernel/issues/1122)): Chrome's OPFS can move a file but not a directory. So the kernel renames a directory by creating the new tree and moving each file into it (copying a file only if it can't move, e.g. while it's open elsewhere), then removing the old tree. Within the kernel the rename is atomic to readers: a lookup or listing under the old or the new name, or of either parent, waits until the whole tree has moved. So a stage-then-rename updater (pnpm's, an atomic config write) is never seen half there. Another tab sharing the root reads OPFS directly and can see the tree mid-move. A root whose directories can move (the Node memory root) renames them in one step. **If the page reloads or crashes mid-rename**, some files are already in the new tree and the rest are still in the old one. So before the first move, the kernel writes the rename's intent (`{ from, to }`) to its metadata store, and clears it after the last. At the next boot, when no other tab shares the root (the same rule as for the boot's other repair of stale metadata), the kernel finishes every recorded rename: it moves what's left into the new tree, removes the old name, and clears the record. While another tab shares the root, the split tree stays as it is (both names visible, each with part of the files) until a boot that's alone.
+
 A lookup follows at most 40 symlinks, as Linux does: the 41st is `ELOOP`, for WASI programs, JS programs and the kernel API alike, with or without `..` in a link's target. Emscripten programs resolve paths in their glue, which stops one link earlier, at 40.
 
 Directories are renamed with `FileSystemHandle.move()` where available, else by copy and delete.
@@ -938,8 +957,8 @@ The kernel then asks the hook `{ v: 1, op: 'materializeConsume', ticket, stamp,
 
 **The child's output is scrubbed (S2).** The kernel compiles a [slicc-scrub](https://github.com/ai-ecoverse/slicc-scrub) scrubber (1.0.0, bundled; the published package has no runtime dependency) over exactly the values it swapped in, and gives the child new open file descriptions for fd 1 and fd 2, for every other descriptor it holds on the same descriptions (a `dup` of them), and for every descriptor on a terminal (fd 0 of a terminal session is one), that pass every write through it into the originals (one wrapper per original description, so `2>&1` keeps its order). A value appears as its placeholder, and a fragment or encoded form of it (JSON, percent, HTML, base64, hex, wrapped) as `[redacted:NAME]`. Bytes go through at once unless they may still be the start of a match; such a tail is let out after 50 ms without a write when it's shorter than 16 bytes, and otherwise at the next write or at close. The child's descendants inherit the scrubbed descriptions through fork, spawn, exec and `dup`, and a descendant that opens its controlling terminal (`/dev/tty`, or a terminal by name) gets one scrubbed too, as does a pseudo-terminal slave (`/dev/pts/N`) or [`/dev/clipboard`](#devclipboard) it opens, and a `/dev/clipboard` descriptor it inherited is wrapped at the redeem like fd 1 and 2, so the page's clipboard gets placeholders, never the values. Files, sockets and pipes the child or a descendant opens itself are not scrubbed, so `printf %s "$T" > f` writes the value. A scrubbed fd 1 or 2 is a stream: it doesn't seek. Under `O_NONBLOCK` it gives `EAGAIN` when the pipe or terminal under it would. A write that finds room returns at once, taking at most 64 KiB, and the scrubbed bytes the file couldn't take yet go out before anything written later. Until they have, `poll` reports it unwritable and a nonblocking write gets `EAGAIN`, taking nothing. A blocking write that waits for them, or for room, ends with `EINTR` at a signal, taking nothing; one interrupted after it began writing returns its whole count, and the rest follows. A descendant that redeems a ticket of its own adds its values: its output is scrubbed for both sets. A value under 8 bytes is swapped but can't be scrubbed. The scrubber goes when the last process and description holding it do.
 
-**A2 sessions (`sessionMaterialize`, [#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)).** The request is in place but not sent yet: the next #1090 steps send it from the http1, SMTP and pg writers. For a kernel-terminated TLS session, the kernel will ask the hook `{ v: 1, op: 'sessionMaterialize', caller, stamp, session, host, port, protocol, position, verified, handshakeAt, exit, placeholders, head? }` (slicc-vault 1.10.0, Sessions), once per session:
-- `stamp` (also `caller`) is the stamp of the process that owns the socket; `session` the kernel's id for it (16 to 128 of `A-Za-z0-9_-`); `host` the name the leaf was verified against, lower-cased without a trailing dot; `handshakeAt` when the handshake finished (epoch ms); `exit` the exit the session leaves by (`page`, `proxy`, `tailscale` or `device:<name>`);
+**A2 sessions (`sessionMaterialize`, [#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)).** For a kernel-terminated TLS session (see **Sealed values in an A2 session** under JS programs' `ctx.net`), the kernel asks the hook `{ v: 1, op: 'sessionMaterialize', caller, stamp, session, host, port, protocol, position, verified, handshakeAt, exit, placeholders, head? }` (slicc-vault 1.10.0, Sessions), once per session:
+- `stamp` (also `caller`) is the stamp of the process that owns the socket; `session` the kernel's id for it (16 to 128 of `A-Za-z0-9_-`); `host` the name the leaf was verified against, lower-cased without a trailing dot; `handshakeAt` when the handshake finished (epoch ms); `exit` the exit the session leaves by, as the uplink reported it (`tailscale` or `device:<id>`, the id of `[A-Za-z0-9._-]`);
 - `protocol` is `http1`, `smtp` or `pg`, and `position` one of its own: `header:<lower-cased name>`, `auth-plain` or `auth-login`, or `password-cleartext`; `head` (`{ method, path }`) is there for `http1` only, and required there;
 - `placeholders` are the 1 to 16 distinct `fnox_proxy_<24 hex>` placeholders found at that position.