@ai-ecoverse/slicc-kernel 1.128.1 → 1.129.0

package/README.md

@@ -225,7 +225,7 @@ A WASI preview1 or WASIX program (Zig, Rust, Go, wasi-libc or wasix-libc C) has
 { "slicc": { "abi": "wasi", "commands": { "rg": { "wasm": "bin/rg.wasm" }, "rustc": { "script": "bin/rustc" } } } }
 ```
 
-A command's name is any single path component without whitespace or control characters, so coreutils' `[` is in `/usr/bin` too; a name that is empty, `.` or `..`, or has a `/`, a space, a tab, a newline, NUL or another control character is skipped. Entries may also set `argv0`, `args` and `env`, and a package can set `slicc.env` for all of its commands. A WASI entry with `"preopenRoot": true` also gets `/` as a preopen (see [WASI programs](#wasi-programs)). An entry with `"argv0Path": true` gets the absolute path it was run by as `argv[0]` when that path is a link outside `/bin` and `/usr/bin` to the command, as a venv's `bin/python` is (the `#!` line of a script naming such a link included), so it can find what sits beside that link; other entries keep their `argv0`. In `env`, `${package}` is the package directory, `${cwd}` the process's working directory, and `${NAME}` the caller's `NAME` when the program starts (a default naming an unset variable is left out); a relative path that exists in the package (`lib/python3.14`, `./`) becomes that absolute path. The caller's environment wins over these defaults; a `null` value removes the variable for the command, after the caller's environment is applied (a host-networked program can drop the kernel's proxy settings that way), and a command entry can set a variable its package's `slicc.env` removes. The catalog reads its roots in order, and the first root that provides a name wins it: a user's own global packages (for that user's processes), `/node_modules`, root's global packages, then each appended root. Within a root, packages are read by name and the first wins (compared as strings by UTF-16 code unit, so a scoped `@scope/name` comes before names that start with a letter, but after those that start with a digit), whatever order the file system lists them in; a `PNPM_HOME`'s install groups count as one root, read group by group (by the group directory's name, effectively arbitrary for pnpm's hashed names), each by package name. A variant can take over from the package it extends with `"slicc": { "replaces": ["@ai-ecoverse/wasm-imagemagick"] }`: when it and a package it names are in the same root, it wins every command name they share, whatever the order; names only the other provides stay the other's, a third package that provides the name and sorts first still wins it, and packages that replace each other fall back to the name order. A replace never reaches into an earlier root, so root's global packages can't take a name from `/node_modules` or a user's own packages. The field must be an array of package names; anything else is ignored. `sh` runs bash as `sh` unless a package provides its own. A package's `engines.slicc-kernel` is a floor the kernel holds it to: when the running kernel's version doesn't satisfy that npm range (comparators, `^`, `~`, x-ranges, hyphen ranges, `||`), or the range can't be read, its commands are refused. They stay in `/usr/bin`, and running one by name, by path or as a `#!` interpreter exits 126 with `<cmd>: <pkg>@<version> needs slicc-kernel <range>; this kernel is <version>` on stderr. Each refused package is logged once to the console. A non-string `engines.slicc-kernel`, or none, is no constraint. A development build reports the newest release tag as `<tag>+dev` and checks against that; a build of unknown version (`0.0.0`) checks nothing. Only commands are refused: a package without commands (a `py-*` library, a side module) isn't, and a refused package that replaces base commands makes those commands fail with 126 rather than fall back to the package it replaces. The installed commands appear as executables in a virtual `/usr/bin` and `/bin`, so `PATH` lookups, `command -v` and `ls /usr/bin` work without writing anything to OPFS. An entry reads as its program, as a PATH search that opens what it finds expects (`ruby -S rake`, `perl -S`, Pod::Usage looking itself up, bundler checking that `/usr/bin/ruby` isn't empty). A script command reads as its script, a JS command as its module, and a WASI or Emscripten command as its `.wasm`, so `cat /usr/bin/hello` prints the script and `head -c4 /usr/bin/cat` gives `\0asm`. The entry is a plain file and no link: it stays a root-owned `0755` regular file with its own inode, `readlink` of it is `EINVAL`, and `which` and `command -v` print `/usr/bin/<name>`. Its size and times are the program's when the caller may read the program (anyone, for a kernel-internal look or with users off, only when everyone may), and 0 and the epoch otherwise. Running it still runs the catalog command, with its `env`, `args` and `argv0`. The read is the caller's: with users on, reading an entry also needs `r` on its program, and that includes a user's own commands in `~/.local/share/pnpm/bin`, so an entry whose program the caller can't read is `EACCES`. A process can also run a program by the path of its glue when the `.wasm` sits next to it, a wasm module by its own path (as a WASI program; when it is the `wasm` of a WASI command in the caller's catalog, the user's own packages included, it runs as that command, with its `env` defaults, `args` and `argv0`, the command whose name matches the file's, without `.wasm`, if several share it, while a copy elsewhere runs bare), and a script through its `#!` line (including `#!/usr/bin/env name`). Executing any other file fails with `ENOEXEC`, so bash runs it as a shell script, as POSIX shells do. A `..` in a path that is run is resolved physically, as the file system resolves it: `pkg/../tools/x` after a symlinked `pkg` is the `tools/x` beside the link's target, and its exec permission is checked there. The walk is the caller's: with users on, each directory it passes needs search (`x`) permission (`EACCES`, 126 in bash, as on Linux), and a sudo process follows only the links it trusts, as for any other path. Commands also come from pnpm's global directory. Every process gets a `PNPM_HOME` and has `$PNPM_HOME/bin` on `PATH`: root's is `/usr/local/share/pnpm` (unless the embedder sets its own), a user's `~/.local/share/pnpm` (see [Users](#users)). The kernel scans each of pnpm's global install groups (`$PNPM_HOME/global/*/<group>/node_modules`) after `/node_modules`, which wins when both provide a command. The catalog is re-read whenever anything under either changes, so `pnpm add -g @ai-ecoverse/wasi-ripgrep` makes `rg` work in the same shell, and `pnpm remove -g` removes it again.
+A command's name is any single path component without whitespace or control characters, so coreutils' `[` is in `/usr/bin` too; a name that is empty, `.` or `..`, or has a `/`, a space, a tab, a newline, NUL or another control character is skipped. Entries may also set `argv0`, `args` and `env`, and a package can set `slicc.env` for all of its commands. A WASI entry with `"preopenRoot": true` also gets `/` as a preopen (see [WASI programs](#wasi-programs)). An entry with `"argv0Path": true` gets the absolute path it was run by as `argv[0]` when that path is a link outside `/bin` and `/usr/bin` to the command, as a venv's `bin/python` is (the `#!` line of a script naming such a link included), so it can find what sits beside that link; other entries keep their `argv0`. In `env`, `${package}` is the package directory, `${cwd}` the process's working directory, and `${NAME}` the caller's `NAME` when the program starts (a default naming an unset variable is left out); a relative path that exists in the package (`lib/python3.14`, `./`) becomes that absolute path. The caller's environment wins over these defaults; a `null` value removes the variable for the command, after the caller's environment is applied (a host-networked program can drop the kernel's proxy settings that way), and a command entry can set a variable its package's `slicc.env` removes. The catalog reads its roots in order, and the first root that provides a name wins it: a user's own global packages (for that user's processes), `/node_modules`, root's global packages, then each appended root. Within a root, packages are read by name and the first wins (compared as strings by UTF-16 code unit, so a scoped `@scope/name` comes before names that start with a letter, but after those that start with a digit), whatever order the file system lists them in; a `PNPM_HOME`'s install groups count as one root, read group by group (by the group directory's name, effectively arbitrary for pnpm's hashed names), each by package name. A variant can take over from the package it extends with `"slicc": { "replaces": ["@ai-ecoverse/wasm-imagemagick"] }`: when it and a package it names are in the same root, it wins every command name they share, whatever the order; names only the other provides stay the other's, a third package that provides the name and sorts first still wins it, and packages that replace each other fall back to the name order. A replace never reaches into an earlier root, so root's global packages can't take a name from `/node_modules` or a user's own packages. The field must be an array of package names; anything else is ignored. `sh` runs bash as `sh` unless a package provides its own. A package's `engines.slicc-kernel` is a floor the kernel holds it to: when the running kernel's version doesn't satisfy that npm range (comparators, `^`, `~`, x-ranges, hyphen ranges, `||`), or the range can't be read, its commands are refused. They stay in `/usr/bin`, and running one by name, by path or as a `#!` interpreter exits 126 with `<cmd>: <pkg>@<version> needs slicc-kernel <range>; this kernel is <version>` on stderr. Each refused package is logged once to the console. A non-string `engines.slicc-kernel`, or none, is no constraint. A development build reports the newest release tag as `<tag>+dev` and checks against that; a build of unknown version (`0.0.0`) checks nothing. Only commands are refused: a package without commands (a `py-*` library, a side module) isn't, and a refused package that replaces base commands makes those commands fail with 126 rather than fall back to the package it replaces. The installed commands appear as executables in a virtual `/usr/bin` and `/bin`, so `PATH` lookups, `command -v` and `ls /usr/bin` work without writing anything to OPFS. An entry reads as its program, as a PATH search that opens what it finds expects (`ruby -S rake`, `perl -S`, Pod::Usage looking itself up, bundler checking that `/usr/bin/ruby` isn't empty). A script command reads as its script, a JS command as its module, and a WASI or Emscripten command as its `.wasm`, so `cat /usr/bin/hello` prints the script and `head -c4 /usr/bin/cat` gives `\0asm`. The entry is a plain file and no link: it stays a root-owned `0755` regular file with its own inode, `readlink` of it is `EINVAL`, and `which` and `command -v` print `/usr/bin/<name>`. Its size and times are the program's when the caller may read the program (anyone, for a kernel-internal look or with users off, only when everyone may), and 0 and the epoch otherwise. Running it still runs the catalog command, with its `env`, `args` and `argv0`. The read is the caller's: with users on, reading an entry also needs `r` on its program, and that includes a user's own commands in `~/.local/share/pnpm/bin`, so an entry whose program the caller can't read is `EACCES`. A process can also run a program by the path of its glue when the `.wasm` sits next to it, a wasm module by its own path (as a WASI program; when it is the `wasm` of a WASI command in the caller's catalog, the user's own packages included, it runs as that command, with its `env` defaults, `args` and `argv0`, the command whose name matches the file's, without `.wasm`, if several share it, while a copy elsewhere runs bare), and a script through its `#!` line (including `#!/usr/bin/env name`). Executing any other file fails with `ENOEXEC`, so bash runs it as a shell script, as POSIX shells do. A spawn by a bare name (no `/`) that the catalog doesn't have, and that didn't ask for no search (a WASIX `posix_spawn` or `execve`, as opposed to its `posix_spawnp`/`execvp`, keeps its bare name), searches the `PATH` of the spawn's own environment, in order, with an empty entry meaning the working directory; each candidate is run exactly as a spawn of that path would be. The first that runs wins; if one was found but refused and nothing else matched, the spawn fails with `EACCES` (126), otherwise `ENOENT` (127). A catalog command still wins over the same name on `PATH`, the reverse of a shell's order. The `PATH` searched is the new process's, from the spawn's environment; POSIX `posix_spawnp` uses the caller's, which is the same unless the caller hands the child another one. This is a stopgap for the Emscripten shim's `posix_spawnp`, which hands the kernel the bare name (`timeout 5 myscript`, [#1165](https://github.com/ai-ecoverse/slicc-kernel/issues/1165)); it goes once the shipped packages search `PATH` themselves ([#1168](https://github.com/ai-ecoverse/slicc-kernel/issues/1168)). A `..` in a path that is run is resolved physically, as the file system resolves it: `pkg/../tools/x` after a symlinked `pkg` is the `tools/x` beside the link's target, and its exec permission is checked there. The walk is the caller's: with users on, each directory it passes needs search (`x`) permission (`EACCES`, 126 in bash, as on Linux), and a sudo process follows only the links it trusts, as for any other path. Commands also come from pnpm's global directory. Every process gets a `PNPM_HOME` and has `$PNPM_HOME/bin` on `PATH`: root's is `/usr/local/share/pnpm` (unless the embedder sets its own), a user's `~/.local/share/pnpm` (see [Users](#users)). The kernel scans each of pnpm's global install groups (`$PNPM_HOME/global/*/<group>/node_modules`) after `/node_modules`, which wins when both provide a command. The catalog is re-read whenever anything under either changes, so `pnpm add -g @ai-ecoverse/wasi-ripgrep` makes `rg` work in the same shell, and `pnpm remove -g` removes it again.
 
 **More roots:** `createKernel({ commands: { append: ['/opt/user/node_modules'] } })` (and `createNodeKernel`) scans more directories, each absolute and normalised (anything else throws a `TypeError`). A name is looked up in a user's own global packages first, then in `modules`, then in root's global packages, then in the appended roots in the order given, and the first match wins, so an appended root never shadows another command (or `sudo`). Their commands are in `/usr/bin` for every user, like root's global packages, and their `binfmt` and file system entries count too. A package in an appended root also gets a command for each `bin` entry of its `package.json` (a string `bin` is named after the package, without its scope), which runs as a `#!` script; a name its `slicc.commands` gives wins. A `bin` entry is skipped when its name isn't a command name (see above), or its file, with symlinks resolved, isn't a regular file inside the package. `/node_modules` and the global packages keep reading only `slicc.commands`. A missing root counts as empty until it appears, and an install or removal in a root is picked up as it happens. With `users: 'enforce'` a root counts only when it and every directory above it are root's and not writable by group or others; another root is skipped without a word. Under `'report'` such a root is still scanned, and reported once to `onDenial` and the report log as `{ pid: 0, uid: 0, op: 'commands', path, errno: 'EACCES' }`. Running a `bin` command needs `r` and `x` on its file under `users` (see [Permissions](#permissions)), checked when it runs, so a `0644` file is `EACCES` under `'enforce'`.
 
@@ -465,13 +465,17 @@ The module's default export (or its `main` export) is called with a context and
 
       Then:
       - **HTTP/1.1:** if the **first request head** the program writes on the session (it starts with a standard method) has placeholders (`fnox_proxy_<24 hex>`) in exactly one header line, either in the value or in the `user:password` of `Basic` credentials, the kernel asks the vault once (`sessionMaterialize`, below) for that header's position.
+      - **SMTP:** a session whose first line is `EHLO` or `HELO` (implicit TLS, or after a STARTTLS upgrade; bytes written before the upgrade are never substituted) is read line by line, and only the `AUTH` exchange is a position:
+        - `AUTH PLAIN <blob>`, or the line after a bare `AUTH PLAIN`: the kernel decodes the blob, and asks the vault (`auth-plain`) only when the placeholders are in the **password field**. A placeholder in the user or authorization identity goes out as written. It substitutes there and re-encodes.
+        - `AUTH LOGIN`: the password line (the second line, or the first after `AUTH LOGIN <user>`) is decoded, substituted (`auth-login`) and re-encoded; the user line goes out as written.
+        - Every other line goes out as written. After the first `AUTH`, or once `MAIL`, `RCPT`, `DATA` or `BDAT` is seen, so is the rest of the session. A line still unfinished after 2 s, or longer than 4 KiB, is sent as written, and so is the rest.
       - **What goes out:** the placeholders the vault answers with a value of at least 8 bytes go out as that value (`Basic` decoded, substituted and re-encoded). Everything else goes out as written:
         - the request line and the body;
         - other headers;
         - a head with placeholders in two header lines;
         - later requests on the same connection;
         - anything once the head passes 16 KiB, or once the program pauses for 2 s in the middle of it;
-        - SMTP and pg, which come next.
+        - pg, which comes next.
       - **What's read back:** once a process has had a value substituted for a host, everything its whole process tree reads from any A2 session to that host is scrubbed: the process itself, its ancestors and every descendant (forked, spawned or exec'd, before or after the substitution; a forked child's own new session included) with [slicc-scrub](https://github.com/ai-ecoverse/slicc-scrub), as S2 does: the value comes back as its placeholder, and a fragment or an encoded form (JSON, percent, HTML, base64, hex) as `[redacted:sealed]`. The scrub stays until the whole tree has ended. A process outside the tree isn't covered. A value shorter than 8 bytes is never substituted, as it couldn't be scrubbed.
       - **The bound:** the server the value goes to is the one the vault rule names, verified against the kernel's own roots. Substitution trusts that server not to reflect the value back in a form the scrub doesn't know (encrypted under a key the program chose, say). That's the same residual as A1's.
       - **Elsewhere:** a placeholder written into any other A2 session reaches the server as written. It isn't the secret, so nothing leaks.
@@ -603,6 +607,8 @@ In the browser, the page serves the uplink to the kernel worker over the same po
 
 `fakeCronClock(start?, utc?)` is a clock for the kernel's cron to run on in tests, so a test moves time instead of waiting for it. It starts at `start` (default 2026-01-01T00:00Z) and reads schedules in UTC unless `utc` is `false`. `now()` is the fake time. `sleep(until, signal)` resolves when `advance(ms)` moves time to `until` or past it, or when `signal` aborts. `set(ms)` sets the wall clock, forward or back, and wakes every sleeper, as a scheduler's next periodic wake would after a real clock change. `sleepers` counts the sleeps still waiting. `flush()` resolves once every sleeper that a time move woke has gone back to sleep for a later time, which is the point where a scheduler has done what was due. It never resolves while nothing sleeps.
 
+`setTestSyscallHook(hook)` is **test-only**: while set, every process kernel call (the `WasmSyscall` requests: process, fd, socket, signal and job calls; not the path-only file system requests such as `read`, `write` or `rename`) of a process in this Node process first awaits `hook({ pid, op })`. `pid` is the kernel's own pid of the process, which for an exec'd image is not the pid the program sees (`getpid()`, `ps` show the pid it took over). A test can then act at an exact point (write a ^C when a shell forks) or hold a call (delay a child's `proc-setpgid`). `setTestSyscallHook(undefined)` clears it. Unset, which is the default, it does nothing.
+
 ### Browser automation (CDP)
 
 Programs drive a browser with the Chrome DevTools Protocol over a WebSocket, whichever host actually has the browser: slicc-extension's `chrome.debugger`, slicc-node's or slicc-swift's CDP proxy, or a test harness. The kernel relays; it does not launch a browser.