@ai-ecoverse/slicc-kernel 1.130.2 → 1.132.0

package/README.md

@@ -487,13 +487,13 @@ The module's default export (or its `main` export) is called with a context and
         - `AUTH PLAIN <blob>`, or the line after a bare `AUTH PLAIN`: the kernel decodes the blob, and asks the vault (`auth-plain`) only when the placeholders are in the **password field**. A placeholder in the user or authorization identity goes out as written. It substitutes there and re-encodes.
         - `AUTH LOGIN`: the password line (the second line, or the first after `AUTH LOGIN <user>`) is decoded, substituted (`auth-login`) and re-encoded; the user line goes out as written.
         - Every other line goes out as written. After the first `AUTH`, or once `MAIL`, `RCPT`, `DATA` or `BDAT` is seen, so is the rest of the session. A line still unfinished after 2 s, or longer than 4 KiB, is sent as written, and so is the rest.
+      - **pg:** a session that starts with a protocol 3 `StartupMessage` (after `sslmode`'s `SSLRequest` upgrade, or with direct TLS) is followed message by message. The only position is the **`PasswordMessage` answering `AuthenticationCleartextPassword` (`R` 3)**, as the program read it from the server. The password is substituted (`password-cleartext`) and the message is framed again. Anything else goes out as written: the startup, an MD5 (`R` 5) or SCRAM (`R` 10) answer, every later message, and a frame over 16 KiB. MD5 and SCRAM need the kernel to compute the answer from the password, so they aren't supported yet. A server whose `pg_hba` asks for them gets the placeholder, and refuses.
       - **What goes out:** the placeholders the vault answers with a value of at least 8 bytes go out as that value (`Basic` decoded, substituted and re-encoded). Everything else goes out as written:
         - the request line and the body;
         - other headers;
         - a head with placeholders in two header lines;
         - later requests on the same connection;
         - anything once the head passes 16 KiB, or once the program pauses for 2 s in the middle of it;
-        - pg, which comes next.
       - **What's read back:** once a process has had a value substituted for a host, everything its whole process tree reads from any A2 session to that host is scrubbed: the process itself, its ancestors and every descendant (forked, spawned or exec'd, before or after the substitution; a forked child's own new session included) with [slicc-scrub](https://github.com/ai-ecoverse/slicc-scrub), as S2 does: the value comes back as its placeholder, and a fragment or an encoded form (JSON, percent, HTML, base64, hex) as `[redacted:sealed]`. The scrub stays until the whole tree has ended. A process outside the tree isn't covered. A value shorter than 8 bytes is never substituted, as it couldn't be scrubbed.
       - **The bound:** the server the value goes to is the one the vault rule names, verified against the kernel's own roots. Substitution trusts that server not to reflect the value back in a form the scrub doesn't know (encrypted under a key the program chose, say). That's the same residual as A1's.
       - **Elsewhere:** a placeholder written into any other A2 session reaches the server as written. It isn't the secret, so nothing leaks.
@@ -986,7 +986,7 @@ The kernel then asks the hook `{ v: 1, op: 'materializeConsume', ticket, stamp,
 **The child's output is scrubbed (S2).** The kernel compiles a [slicc-scrub](https://github.com/ai-ecoverse/slicc-scrub) scrubber (1.0.0, bundled; the published package has no runtime dependency) over exactly the values it swapped in, and gives the child new open file descriptions for fd 1 and fd 2, for every other descriptor it holds on the same descriptions (a `dup` of them), and for every descriptor on a terminal (fd 0 of a terminal session is one), that pass every write through it into the originals (one wrapper per original description, so `2>&1` keeps its order). A value appears as its placeholder, and a fragment or encoded form of it (JSON, percent, HTML, base64, hex, wrapped) as `[redacted:NAME]`. Bytes go through at once unless they may still be the start of a match; such a tail is let out after 50 ms without a write when it's shorter than 16 bytes, and otherwise at the next write or at close. The child's descendants inherit the scrubbed descriptions through fork, spawn, exec and `dup`, and a descendant that opens its controlling terminal (`/dev/tty`, or a terminal by name) gets one scrubbed too, as does a pseudo-terminal slave (`/dev/pts/N`) or [`/dev/clipboard`](#devclipboard) it opens, and a `/dev/clipboard` descriptor it inherited is wrapped at the redeem like fd 1 and 2, so the page's clipboard gets placeholders, never the values. Files, sockets and pipes the child or a descendant opens itself are not scrubbed, so `printf %s "$T" > f` writes the value. A scrubbed fd 1 or 2 is a stream: it doesn't seek. Under `O_NONBLOCK` it gives `EAGAIN` when the pipe or terminal under it would. A write that finds room returns at once, taking at most 64 KiB, and the scrubbed bytes the file couldn't take yet go out before anything written later. Until they have, `poll` reports it unwritable and a nonblocking write gets `EAGAIN`, taking nothing. A blocking write that waits for them, or for room, ends with `EINTR` at a signal, taking nothing; one interrupted after it began writing returns its whole count, and the rest follows. A descendant that redeems a ticket of its own adds its values: its output is scrubbed for both sets. A value under 8 bytes is swapped but can't be scrubbed. The scrubber goes when the last process and description holding it do.
 
 **A2 sessions (`sessionMaterialize`, [#1090](https://github.com/ai-ecoverse/slicc-kernel/issues/1090)).** For a kernel-terminated TLS session (see **Sealed values in an A2 session** under JS programs' `ctx.net`), the kernel asks the hook `{ v: 1, op: 'sessionMaterialize', caller, stamp, session, host, port, protocol, position, verified, handshakeAt, exit, placeholders, head? }` (slicc-vault 1.10.0, Sessions), once per session:
-- `stamp` (also `caller`) is the stamp of the process that owns the socket; `session` the kernel's id for it (16 to 128 of `A-Za-z0-9_-`); `host` the name the leaf was verified against, lower-cased without a trailing dot; `handshakeAt` when the handshake finished (epoch ms); `exit` the exit the session leaves by, as the uplink reported it (`tailscale` or `device:<id>`, the id of `[A-Za-z0-9._-]`);
+- `stamp` (also `caller`) is the stamp of the process that owns the socket; `session` the kernel's id for it (16 to 128 of `A-Za-z0-9_-`); `host` the name the leaf was verified against, lower-cased without a trailing dot; `handshakeAt` when the handshake finished (epoch ms); `exit` the exit the session leaves by, as the uplink reported it (`tailscale` or `device:<id>`, the id of `[A-Za-z0-9._:-]`);
 - `protocol` is `http1`, `smtp` or `pg`, and `position` one of its own: `header:<lower-cased name>`, `auth-plain` or `auth-login`, or `password-cleartext`; `head` (`{ method, path }`) is there for `http1` only, and required there;
 - `placeholders` are the 1 to 16 distinct `fnox_proxy_<24 hex>` placeholders found at that position.
 
@@ -1077,6 +1077,8 @@ OPFS stays authoritative: an entry is consulted only for a path that exists in O
 
 **Stamping files written to OPFS:** `import { writeMetadata } from '@ai-ecoverse/slicc-kernel/meta'` and `await writeMetadata(root, [{ path, mode?, uid?, gid? }], { metadata?, indexedDB? })` is the supported way for an embedder to give files it wrote straight to OPFS a mode and an owner, also while no kernel runs (an installer before the first boot). `root` is the directory handle that becomes `/`, each `path` is absolute (`EINVAL` otherwise) and must exist there (`ENOENT` otherwise), `mode` is the permission bits (`0` to `0o7777`), and `uid` and `gid` come together or not at all (`EINVAL` otherwise). Entries for the same path (after normalization) are merged in order, a later field winning. All entries are written in one IndexedDB transaction, to the database a kernel on that root reads (`metadata`, default `slicc-kernel`), keeping what else a record holds and setting its change time (`ctime`) to now, and a running kernel on the same origin drops what it cached of them. The record format is the kernel's own and changes with it, so use this rather than writing the database. The kernel gives its layout paths (see [Permissions](#permissions)) an owner and mode only when they have no owner yet, so a path stamped here with `uid` and `gid` keeps them.
 
+**Telling running kernels about files written to OPFS:** a writer outside the kernel (an installer worker, a service worker, the page) that changes files on a root a kernel may be running on calls `import { postFsNotice } from '@ai-ecoverse/slicc-kernel/meta'` and `postFsNotice({ paths?, trees? }, { metadata? })` **after** each write has committed: after `close()` of its writable, `removeEntry` or `move()` has resolved. `paths` are absolute paths written, created or removed. List a created or removed name's parent directory too, so listings refresh. `trees` are absolute directories whose whole subtree changed, for a recursive replace or a sweep. The notice goes on the BroadcastChannel `slicc-kernel-dirs:<metadata>` (default `slicc-kernel-dirs:slicc-kernel`), the one kernels on that root already use between themselves, as `{ paths, trees }`. One notice per batch of writes is enough, if it's posted after the last of them. A kernel that receives it drops its cached lookups, handles and misses for those paths (for a tree, everything under it), tells its open files to reread, and publishes the paths to its programs' caches. A tree marks every path changed for them, so a running program's next look at any path goes to the kernel. **The residual window:** a reader can still see the old state between the write's commit and the notice's delivery (a message, usually well under a millisecond, but not bounded). A writer that runs while no kernel is up needs no notice: a kernel's caches start empty when it boots. **Without a notice** a kernel that is alone on its root keeps what it cached: a running program's stats of a path stay as they were for as long as the program runs, and the kernel's own directory handles for up to a second. So a writer that doesn't post notices must not change files a running kernel uses ([#1156](https://github.com/ai-ecoverse/slicc-kernel/issues/1156)).
+
 The database is `slicc-kernel` (or the `metadata` option), schema version 1, with one object store, `entries`, keyed by absolute path (`keyPath: 'path'`) and an index `dir` on the parent directory of symlinks. A record looks like `{ path, mode?, atimeMs?, mtimeMs?, mtimeFor?, ctimeMs?, ino?, link?, dir?, uid?, gid?, heir?, writes?, epoch? }`, where `mtimeFor` is the OPFS `lastModified` an explicit `mtimeMs` belongs to, `uid`, `gid` and `heir` (`[uid, gid]`) are the owners of [File owners](#file-owners), `writes` counts the writes kernels with `integrity` made to a protected file (see [Protected paths](#protected-paths)), whose `lastModified` OPFS leaves alone for a write through a sync access handle, and `epoch`, on the entry of `/`, counts their renames and symlinks at or above a protected path. A future schema bumps the version and migrates in `onupgradeneeded`; an older kernel then closes its connection. The sidecar is keyed by path from the `root` handle, so use a different `metadata` name for each distinct root.
 
 Not covered: hard links (OPFS cannot share contents between names; they work only within a mount whose driver offers `links`, such as `tmpfs`). File owners are stored only with `users: 'report'` or `'enforce'` (see [File owners](#file-owners)). With `users: 'off'`, an Emscripten process built against the kernel's credential calls (its glue imports `slicc_cred_js`, as homescoop's H1 builds do) sees every file as owned by its own effective uid and gid: root's 0 by default, a user's own ids when it runs as one, and the new ids after a `setuid`. Owner checks such as git's `safe.directory`, ssh's key checks and gnupg's home directory check then pass, as WASIX programs already see through their libc. An older program, whose libc says it is uid 1000 whoever runs it, sees every file as owned by 1000:1000 as before, so its own checks pass too. The kernel decides this once, when it loads the program. Emscripten also doesn't enforce permission bits, so a file runs whether or not its exec bit is set; the bits are what `ls`, `stat` and `test` report. With `users: 'off'` a file or directory with no permission bits at all shows as `0666` or `0777` to Emscripten programs; with owners on they see the real mode (`0`), and the kernel, not Emscripten's own checks, decides who may read, write or enter it (`cd` asks the kernel for `x` on the directory), so root still enters a `000` directory.