@ai-ecoverse/slicc-kernel 1.108.5 → 1.110.0

package/README.md

@@ -227,6 +227,8 @@ A command's name is any single path component without whitespace or control char
 
 **More roots:** `createKernel({ commands: { append: ['/opt/user/node_modules'] } })` (and `createNodeKernel`) scans more directories, each absolute and normalised (anything else throws a `TypeError`). A name is looked up in a user's own global packages first, then in `modules`, then in root's global packages, then in the appended roots in the order given, and the first match wins, so an appended root never shadows another command (or `sudo`). Their commands are in `/usr/bin` for every user, like root's global packages, and their `binfmt` and file system entries count too. A package in an appended root also gets a command for each `bin` entry of its `package.json` (a string `bin` is named after the package, without its scope), which runs as a `#!` script; a name its `slicc.commands` gives wins. A `bin` entry is skipped when its name isn't a command name (see above), or its file, with symlinks resolved, isn't a regular file inside the package. `/node_modules` and the global packages keep reading only `slicc.commands`. A missing root counts as empty until it appears, and an install or removal in a root is picked up as it happens. With `users: 'enforce'` a root counts only when it and every directory above it are root's and not writable by group or others; another root is skipped without a word. Under `'report'` such a root is still scanned, and reported once to `onDenial` and the report log as `{ pid: 0, uid: 0, op: 'commands', path, errno: 'EACCES' }`. Running a `bin` command needs `r` and `x` on its file under `users` (see [Permissions](#permissions)), checked when it runs, so a `0644` file is `EACCES` under `'enforce'`.
 
+**Variants by environment** ([homescoop#357](https://github.com/ai-ecoverse/homescoop/issues/357)): one package can carry more than one build of a wasm command and choose between them by the environment. `"variants": [{ "env": { "PYTHON_FORK": "true" }, "wasm": "bin/python-fork.wasm" }]` on a command entry (Emscripten or WASI) is matched in order against the environment the new process gets, at spawn and at exec. Every key must equal its value exactly, and the first match runs its `wasm`; no match runs the command's own `wasm`. This holds however the kernel reaches the entry: by name, by its path (`/usr/bin/python3`, as `sys.executable` gives it to `subprocess` and `multiprocessing`), or through a symlink to that path (a venv's `bin/python`). So `PYTHON_FORK=true python3`, `PYTHON_FORK=true exec /usr/bin/python3` and a child that inherits `PYTHON_FORK=true` all get the variant, and `PYTHON_FORK=1` doesn't. Commands of one package that share a build (`python`, `python3`, `pip`, `pip3`) each carry the same `variants`. A variant replaces only the module: the `abi`, the glue, `env`, `argv0`, `argv0Path` and the `/usr/bin` entry stay the command's. For an Emscripten command that means a variant swaps only the `.wasm` and keeps the command's JS glue, so it must be built against that same glue; a WASI or WASIX command has no glue, so there's no such constraint. Only the chosen module is read, checked and compiled, and compiled modules are cached by its path. A read or `stat` of `/usr/bin/<name>` still shows the default `wasm`. A `variants` that isn't an array, or has an entry without an `env` object of strings and a `wasm` inside the package, is ignored as a whole, with one warning per command, and the command runs its `wasm`. An older kernel ignores the field.
+
 A package can also name a command that runs files by their suffix, as `binfmt_misc` does: `"slicc": { "binfmt": { ".jsh": "jsh" } }`. Executing a file that has no `#!` line and is not a wasm module, whose name ends in a mapped suffix, runs that command with the file's path as invoked and the arguments after it, as a `#!` line naming the command would; a `#!` line still wins. The longest matching suffix wins, the whole name is never a suffix (`.jsh` alone is not one), and of two packages that map one suffix, the one whose name sorts first wins, as for commands; the kernel warns once (`console.warn`) when another package maps that suffix to a different command. A suffix starts with a dot and the command must be a name the catalog resolves when the file runs.
 
 ### WASI programs
@@ -364,6 +366,7 @@ The module's default export (or its `main` export) is called with a context and
 - `spawn({ argv, argv0, env, cwd, pgid, session, parentDeathSignal, stdin, stdout, stderr })` starts a command (found as `kernel.run` finds it) as a child process and resolves with `{ pid, stdin, stdout, stderr, wait(), kill(signal) }`. Each stdio is `'inherit'` (the default: the program's own), `'pipe'` (a web stream on a kernel pipe), `'null'`, or one of the program's descriptors. `env` is the child's whole environment (the program's by default), and `cwd` defaults to the program's. The command run is `argv[0]`; `argv0`, when given, is the `argv[0]` the child sees instead, as given (no basename; an empty string too), as with Node's `options.argv0`: `spawn({ argv: ['/bin/sh', '-c', 'echo $0'], argv0: '/bin/sh' })` prints `/bin/sh`, where without it the child sees the command's own name (`sh`). A command whose package manifest sets its `argv0` (a multi-call program's applets, such as coreutils') keeps that one. An `argv0` that isn't a string is `EINVAL`. The child is in the program's process group and session, as after `fork` (Node's `detached: false`), so the terminal's `^C` and `^Z` reach it along with the program. `pgid: 0` makes it lead a new group in the program's session, set before it runs, so a `kill(-pid)` right after the call reaches it and everything it starts (`npx`, then `node`); on the program's terminal that group is in the background, so a read from the terminal stops it with `SIGTTIN` (`EIO` when it ignores `SIGTTIN`), as on Linux. `pgid: N` joins group N, which must be in the program's session: `EPERM` for another session's group, `ESRCH` when there is none. `session: true` makes it lead a new session and a new group (`setsid`, what Node's `detached: true` does on POSIX): its pid is its `pgid` and its `sid`, it has no controlling terminal (`/dev/tty` is `ENXIO`) and the terminal's keys don't signal it. `session` together with `pgid`, or a `pgid` that isn't a non-negative integer, is `EINVAL`. A command that does not exist rejects with `ENOENT`. `parentDeathSignal` (a number or `'SIGTERM'`, as Linux's `PR_SET_PDEATHSIG`) is sent to the child when the program ends, killed or not: to the child's whole group when it leads one (`pgid: 0` or `session: true`), else to the child alone. It's sent with the program's credentials, so a child the program may no longer signal (one that gained privilege) doesn't get it. If the program has already ended by the time the spawn completes, it's sent at once. A child of its own (not a group leader) that has been waited for gets nothing, and `sync.run` has no such option. A bad signal is `EINVAL`;
 - `wait(pid, { untraced })` resolves with `{ status }` once a child ends, or `{ status: 128 + signal, signal }` when a signal ended it, without blocking the program; with `untraced: true` it also resolves when the child stops, with `{ status: 128 + signal, signal, stopped: true }`, as `waitpid` with `WUNTRACED` (a child's `wait({ untraced })` is the same), and options that aren't an object are `EINVAL`; `kill(pid, signal)` signals any process, by number or as `'SIGTERM'` (the default);
 - `sudo(args)`, what the [`sudo`](#sudo) command calls: `args` are its arguments after `sudo`, and it resolves with `{ pid }`, the command the kernel started as the program's child, to `wait` for; `{ run }`, an argv to run as the caller (with `users: 'off'`); `{ list }`, the text of `sudo -l`; or `{ error, errno }`, the message to print;
+- `crontab({ action, user?, text? })`, what the [`crontab`](#cron) command calls: `action` is `'list'`, `'remove'` or `'install'` (with the table as `text`), on the caller's own table or, for an effective uid of 0, on `user`'s. It resolves with `{ text }` (the table for `list`, `''` otherwise) or `{ error }`, the message to print. A malformed call is `EINVAL`;
 - `net`: `listen({ port, host, backlog })` and `connect({ host, port, signal })` (`host` defaults to `127.0.0.1`) on the kernel's loopback, the same sockets WASIX programs use, so a JS server is reached by other processes, by the page as `<port>.kernel.localhost` (`loopbackFetch`, `dial`) and by attached clients. A listener has its `port` (as bound, so port `0` picks a free one), `accept({ signal })`, `close()` and `for await` over its connections. A connection has `local` and `remote` addresses, `readable` and `writable` web streams (closing the writable shuts down the sending side, so the peer reads the end), and `close()`. The loopback is IPv4 (`127.0.0.1` by default, or `0.0.0.0`), so an IPv6 host such as `::1` fails with `EADDRNOTAVAIL` for both `listen` and `connect`, and binding and connecting fail as they would for a WASIX program (`EADDRINUSE`, `ECONNREFUSED`, `EADDRNOTAVAIL`). A blocked accept, read or write waits without holding anything in the kernel, so aborting it or closing the socket leaves nothing behind. A write to a connection the peer closed fails with `EPIPE` and no `SIGPIPE`;
 - `websocket(url, { protocols, headers, signal })`, a WebSocket over the kernel's transport (its `websocket`, so `fetchTransport({ webSocket: true })` in a browser), shaped like `WebSocketStream`. It resolves once the socket is open, with `url`, the `protocol` the far end chose (always one the program offered: a transport that hands back a socket with another fails the open with `EPROTO`, while browsers and Node's `WebSocket`, which `fetchTransport()` uses, fail such a handshake themselves without saying why, so the open fails with `ECONNREFUSED`), a `readable` of messages (`string` for text, `Uint8Array` for binary), a `writable` for sending them, `closed` (the `{ code, reason }` of the close, from either side, whether or not the program reads) and `close({ code, reason })`. An `http(s):` URL is taken as `ws(s):`. Sending waits while more than 1 MiB is buffered. A signal aborts the open, and after it the socket. A transport without WebSockets rejects with a `TypeError` whose `cause.code` is `ENOTSUP`. One that cannot connect rejects with the `code` its error carries (an errno, such as `EAI_NONAME`, `ECONNREFUSED`, `ETIMEDOUT` or `EPROTO`), else `ECONNREFUSED`. `ws://host.slicc.internal:port` reaches the real machine's loopback, as it does through the proxy;
 - `fetch(input, init)`, the web's `fetch` over the kernel's transport, the one the proxy uses, so the embedder's transport, its WebSocket and proxy settings and the kernel's routes apply. The response body streams as the program reads it. On a transport with `traits.streamingBodies` (below), a `ReadableStream` body (with `duplex: 'half'`, which `Request` requires) streams to it as the program produces it, with no size limit of the kernel's, and a redirect that would send it again (a 307 or 308) rejects with a `TypeError`, as the fetch standard has it. Other bodies (a string, a buffer, a `Blob`, a `FormData`) can be sent again, so they go whole even there. Otherwise the request body is read whole first, and one larger than the transport's `maxRequestBody` is answered `413`, as through the proxy. A signal aborts the request at any stage. Redirects follow the fetch rules: at most 20, a 301 or 302 after a `POST` and a 303 turn into a `GET` without a body, `Authorization` is dropped across origins, and `redirect: 'manual'` returns the redirect itself. On a transport that follows redirects on its own (`fetchTransport()` in a browser) only `'follow'` works: `'manual'` and `'error'` reject with a `TypeError` whose `cause.code` is `ENOTSUP`, since such a transport never shows the program a redirect. A status the transport chose comes back as a response whose body is the reason. An unreachable host, or no transport at all, rejects with a `TypeError` whose `cause.code` is `ECONNREFUSED` or `ENETUNREACH`, as Node's `fetch` does. A `data:` URL is answered by `fetch` itself, as the fetch standard decodes it, without a transport: for any method, a `HEAD` without a body, and its type parsed and serialized as a MIME type (lowercase, no spaces, invalid parameters dropped), as Node's `fetch` answers; any other scheme than `http:` and `https:` rejects with a `TypeError` whose `cause.code` is `EPROTONOSUPPORT`, at the start as for a redirect target. A URL on one of the fetch standard's bad ports (25, 6000 and the rest) rejects too, as Node's `fetch` does: a `TypeError` `fetch failed` whose `cause` is an `Error` `bad port` (no `code`), at the start and as a redirect target, on the kernel's own loopback too ([#822](https://github.com/ai-ecoverse/slicc-kernel/issues/822)). `redirect: 'error'` rejects with a `TypeError` whose `cause` is an `Error` `unexpected redirect`, and a body whose connection ends early fails its read with `TypeError` `terminated` whose `cause` is undici's: `name` `SocketError`, `message` `other side closed`, `code` `UND_ERR_SOCKET`. The request goes out with its headers in the order the program gave them (an object's or an array's order; a `Headers` object's is sorted), lowercase as `Headers` keeps them. The global `fetch` stays withheld;
@@ -761,6 +764,7 @@ With `users: 'enforce'` the kernel checks every file call that reaches it agains
 
 The kernel runs scheduled jobs itself, with no daemon process: a cron service starts with the kernel and stops with `terminate()`. It reads `/etc/crontab` and the per-user tables in `/var/spool/cron/crontabs/<user>`, and runs each due job as `$SHELL -c '<command>'` (`/bin/bash` unless the table sets `SHELL`) as the table's user, in that user's home (`/` if the home is missing), with no terminal and `/dev/null` as standard input.
 
+- **`crontab`** is a command of the kernel itself, in `/usr/bin` (a package's `crontab` never wins over it), with Vixie's options: `crontab -l` prints the caller's table, `crontab -r` removes it, `crontab <file>` and `crontab -` (standard input) install one, and `crontab -e` copies the table to `/tmp`, runs `$VISUAL` (or `$EDITOR`) on the copy through `/bin/bash -c '<editor> "$1"'`, and installs it if it changed and the editor exited with 0. If the edit doesn't parse, the copy is kept and its path is printed. `-u <user>` works on another user's table and needs an effective uid of 0 (`sudo crontab -u cone -l`). An install is checked first: a table with a line that doesn't parse is refused with every error (`"-":<line>: <message>`), and the old one stays. The table becomes the user's `0600` file in the spool, written to a dotfile of a random name and renamed, so the service, which skips dotfiles, never reads half a table. The command only reads the file it is given and passes it on (`ctx.crontab`, see [JS programs](#js-programs)). The kernel decides whose table it is from the caller's real uid, so a user reaches only their own table, and `/etc/crontab` stays root's to edit.
 - **Format:** Vixie's. Five fields (minute, hour, day of month, month, weekday) with lists, ranges, steps and the names `jan`–`dec` and `sun`–`sat` (7 is Sunday too), or one of `@yearly`/`@annually`, `@monthly`, `@weekly`, `@daily`/`@midnight`, `@hourly` and `@reboot`. As in cronie, when the day of month or the weekday starts with `*`, a day must match both fields; when both are restricted, it may match either. `/etc/crontab` has a user field after the schedule; a user's table doesn't. `NAME=value` lines set the environment for the jobs after them, `#` lines are comments, and `# name: <id>` on the line above a job gives it a stable id (letters, digits, `_`, `.`, `-`), which names its log. Without one, the id is a hash of the table, the user, the schedule and the command. `%` is not special. Times are the kernel's local time.
 - **Environment:** only what a login of that user gets, and nothing from the page or the embedder's `env`. That is `HOME`, `USER`, `LOGNAME`, `PATH`, `PNPM_HOME`, `NPM_CONFIG_PREFIX`, the kernel's network settings (`http_proxy`, `SSL_CERT_FILE`, …), `SHELL`, `PWD`, the table's `NAME=value` lines, and `SLICC_CRON=1`, `SLICC_CRON_JOB=<id>`, `SLICC_CRON_DUE=<ISO time of the due run>` and `SLICC_CRON_MISSED=<runs folded into this one>`. An `agent send` from a job therefore knows it was scheduled.
 - **Missed runs:** when the kernel wasn't running (a closed tab) or time jumped forward, every due run since the last one is folded into **one** run, with `SLICC_CRON_MISSED` saying how many more were due (counted up to 100 000). The next run is then computed from now. A clock set back more than two minutes runs nothing until the next due time after the new time.
@@ -891,6 +895,7 @@ The kernel is ported from SLICC's `packages/webapp/src/kernel/` with the browser
 - `src/launcher.ts`, `src/commands.ts`, `src/serve.ts`, `src/index.ts`, `src/users.ts`: command resolution, the kernel worker protocol, the page API and the user database; `src/node.ts`, `src/node-process-worker.ts` and `src/node/` the headless Node entry. An embedder that uses `Launcher` directly, without `createKernel`, calls `await launcher.prepare()` first: it creates `/tmp` and `/home` and writes the CA certificate.
 - `src/kernel/net/`: the proxy, HTTP/1.1, TLS termination and the local CA, and the bridge to the page's transport; `src/transport.ts` and `src/local-proxy-transport.ts` are the page side.
 - `src/cdp/`: the CDP facade on `127.0.0.1:9222`, its WebSocket framing, the registry of hosts and the `MessagePort` bridge to them.
+- `src/cron/`: the cron service (`service.ts`), the schedule and table parsers, the clock, and the `crontab` command with the kernel side of `ctx.crontab`. An embedder that uses `Launcher` directly passes `cron: { lock }` to share the firing lock among the kernels on one root; the default, `slicc-kernel-cron`, is one lock for the whole origin.
 
 
 ## Installing from git
@@ -952,6 +957,21 @@ Releases are cut by semantic-release on every push to `main`, in two jobs, and o
 
 Moving the `certified` dist-tag stays manual. Automating it needs an npm token that may change dist-tags, in a step that re-hashes the tarball and tags only on a `certified` line.
 
+### Certification in the pull request (`cert-review-pr`)
+
+`.github/workflows/cert-review-pr.yml` certifies a pull request before it merges, so `main`, and with it npm's `latest`, only gets certified changes. For now it reports as the `cert-review-pr` commit status and isn't a required check. It becomes the required `cert-review` once it has proved itself on real pull requests; the post-publish review above stays until then.
+
+- **Which pull requests:** those that change shipped behaviour, by a conservative path filter (`behaviourFiles` in `lib.mjs`): `src/`, `build.mjs`, `package.json` and its lock, `tsconfig*.json`, `.releaserc*`, `tools/release-recover.mjs`, the workflows that build or publish (`release.yml`, `ci.yml`), and the reviewer's own code (`tools/cert-review/`, `cert-review*.yml`, `.github/runner.mjs`). **Main's code decides:** every job checks out the pull request's base and fetches only the trees under test, so the path filter, pack, suite, findings, the instructions and record all run from `main`. A pull request that changes the reviewer is reviewed by `main`'s reviewer. The workflow file of a `pull_request` run still comes from the pull request, so **pull requests that touch the reviewer always go to the Kernel Coordinator before they are queued.** The pull request that adds this workflow can't pass its own check, because its base has no tools yet. Any other pull request gets a green status that says `not behaviour: <files>`, so the skip is recorded.
+- **When:** only for a same-repo pull request with `merge-ok`, when the label is added and on every push while it is set. Each pull request has its own concurrency group, and a newer push cancels the older review. A behaviour change without `merge-ok` gets a red status, `waits for merge-ok to be reviewed`. Other labels don't start or cancel anything. Fork code never reaches the pools or the key.
+- **What:** the checkout is the event's default `refs/pull/N/merge`, the pull request merged onto `main` at review time, so a review is of what would land, without pushing to the branch. Both sides are packed in the same run, on a GitHub-hosted runner, with no secrets: the candidate (the merge ref, `x.y.z-pr.N.<run>`) and the base (its first parent, `main`). Then:
+  - the candidate's whole suite, and the base's;
+  - the pull request's new or changed test files on the base build (the NEGATIVE: a fix's test must fail on `main` and pass on the candidate);
+  - kernel-compat on the candidate's artifact (#812), against npm's `latest`;
+  - the review (Claude through `claude-code-action/base-action`, with the Bedrock key in that step only, no issue filing);
+  - and record, which re-hashes the candidate's artifact against the pack job's sha256.
+- **The verdict** goes into the commit status on the pull request's head and a comment on the pull request. Certified is green. A FAIL is red with the line, and the pull request needs a fix. **INCOMPLETE** (a job that didn't succeed, no verdict, a suite without results) is red with `re-run the failed jobs`, never a silent pass. Main moving on after a review doesn't trigger another one; a push does.
+- **Merge groups only look up.** On `merge_group`, the job lists the group's pull requests (its merge commits) and requires, for each one, a green status on the head being merged, or no behaviour change. Otherwise it's red, naming the pull request. It runs in seconds, with no secrets and no model. The group's `ci / test` covers the combined tree.
+
 Not part of the run yet: bios's real setup (the base-set sweep in bios, the upgrade boot, the sudo rows under `enforce`). It would plug in as one more job before review that runs slicc-bios's `npm run test:cert` with `CERT_COMMANDS` and the release's tarball, and leaves its results next to the others.
 
 ## Test fixtures
@@ -1084,9 +1104,9 @@ A threaded WASIX C program for `test/integration/thread-signal.test.mjs` (#520).
 
 A WASIX C program for `test/integration/long-timers.test.mjs` (#767).
 - `longwait alarm` sets `alarm(3000000)` (about 35 days), sleeps 0.5 s, and prints how many alarms came and how many seconds were left.
-- `longwait select|ppoll|nanosleep` waits 3,000,000 s on a pipe that never gets data, until a 300 ms interval timer interrupts the wait, and prints `not before it` when the wait lasted at least 300 ms (`early` otherwise). It sets no upper bound, so a loaded CI host can't fail the row; a wait that never ends times the test out.
+- `longwait select|ppoll|nanosleep` waits 3,000,000 s on a pipe that never gets data, until a 300 ms interval timer interrupts the wait, and prints `not before it` when the wait lasted at least 300 ms (`early` otherwise), measured from before `setitimer` arms the timer, so the time the arming call takes can't shorten it (#858). It sets no upper bound, so a loaded CI host can't fail the row; a wait that never ends times the test out.
 
-**Build:** `WASIX_SYSROOT_28=/path/to/wasix-sysroot ./build.sh`, with `wasixcc` 0.4.7 and binaryen's `wasm-opt` on `PATH`. `bin/longwait.wasm` is 44,004 bytes, sha256 `e69d03f01479693666078297e781f5373719700c8534d1f2933749f5b3cee903`.
+**Build:** `WASIX_SYSROOT_28=/path/to/wasix-sysroot ./build.sh`, with `wasixcc` 0.4.7 and binaryen's `wasm-opt` on `PATH`. `bin/longwait.wasm` is 44,002 bytes, sha256 `24acd996068ea04c526b74c93b896986eef782ec96230e19fb939d1b00a1bad0`.
 
 ### `epollmask`