@ai-ecoverse/slicc-kernel 1.70.0 → 1.71.1

package/README.md

@@ -43,6 +43,7 @@ const { status, stdout, stderr } = await kernel.run(['bash', '-c', 'echo hi > he
 | `sealed` | none | `(req, { signal }) => Promise<object>`: answers requests written to [`/dev/sealed`](#devsealed). `createNodeKernel` takes it too |
 | `approve` | none | `(req, { signal }) => 'once' \| 'always' \| 'deny'` (or a promise of one): decides a `sudo` request that needs approval (see [Sudo](#sudo)); without it, such a request is denied. `createNodeKernel` takes it too |
 | `cdp` | none | `({ runtime }) => Promise<CdpConnection>`: a browser-level DevTools connection for programs that drive a browser (see [Browser automation](#browser-automation-cdp)) |
+| `protect` | none | `{ paths }`: directories no process may change, root included, in every `users` mode (see [Protected paths](#protected-paths)). `createNodeKernel` takes it too |
 
 ### `kernel.run(argv, options?) → Promise<{ status, stdout, stderr }>`
 
@@ -211,6 +212,8 @@ A descriptor a program inherits at one of those numbers (from bash's `3<file`, o
 
 **Paths with `..`.** A `..` after a symlinked directory goes to the parent of the link's target, as on Linux: with `/a/link -> /x/y`, `/a/link/../f` is `/x/f`, for a WASI program as for bash. A `..` after a file or a missing name fails with `ENOTDIR` or `ENOENT`. Paths without `..` are joined as they are.
 
+An empty path is `ENOENT` in every path call (`path_open`, `path_filestat_get`, `path_create_directory`, `path_rename`, `path_link`, `path_symlink` with an empty name or target, `path_readlink`, the `slicc_fs` path calls), as on Linux without `AT_EMPTY_PATH`: it never names the directory descriptor itself.
+
 On top of preview1 a WASI program gets:
 
 - **Threads**: `wasi.thread-spawn` (wasm32-wasip1-threads) and WASIX `thread_spawn_v2` start a worker per thread on the process's shared memory, sized from the module's own import (a 2 GiB maximum when the engine cannot reserve more). The threads share one descriptor table, and a file written, created, renamed or removed in one thread shows its new size and metadata to `stat` in every other thread at once, under any of its names (a symlink, a hard link, a new name after a rename in the same process), also when another thread renames a file over that name meanwhile, since the kernel reports which file an open or a change reached; this holds among the kernel's own processes, not against other tabs, `hostfs` writers or programs that aren't WASI; another process's changes show at close and after a `spawn` or `wait`, as with close-to-open caching, so a file another process renames while this one holds it open may show its old size under the new name until then; `exit` or a trap in any thread ends the process. At most 64 threads per process; `SLICC_WASM_THREADS` sets another cap, up to 256.
@@ -420,7 +423,7 @@ Directories are renamed with `FileSystemHandle.move()` where available, else by
 - **File contents:** on a mount with `ranges`, as `tmpfs` is, and on the root file system when it is in OPFS (in a browser with sync access handles) or an in-memory root, a program reads and writes a file in 64 KiB pages: reads come from the driver 1 MiB at a time, and changed pages go back in 1 MiB runs once 8 MiB have changed, at `fsync`, and when the file is closed (from the kernel's own descriptors also 250 ms after a write). A file of any size the mount can hold works, with memory to spare for about one more copy of it. In OPFS a write opens a sync access handle for just that write. An `fsa` mount (a picked folder) has no sync access handles, and Chrome copies the whole file into a swap file for every writable stream, so it writes through one stream per open file: the first write back opens it, later runs go into it at their offsets, and it is committed, one copy of the file, when the last process writing the file closes it (or exits), before another process reads what was written, and before a rename, unlink or unmount. Until then other processes see the new size but read the old bytes outside what was written. Elsewhere (`hostfs` with an older proxy, and a root outside OPFS), file contents are read whole when a program opens a file and written back when it closes it. Before a program's next call into the kernel (any wait, read of a pipe or terminal, sleep or spawn), its changed pages also go back, so other processes and the embedder see a write as soon as the writer does anything but compute: a daemon's pidfile, a log another process follows. On a whole-file mount that holds only for files of up to 64 KiB; a larger one goes back at close, fsync and exit, as before. Beyond that, both give close-to-open consistency, as NFS has. A process that ends without closing its files keeps what it wrote: on exit, on a trap, and when a signal kills it. For a killed WASI program, the kernel lets its worker write its changed pages back before `waitpid` reports the death. The only exception is a worker that makes no call into the kernel for 2 seconds after the signal; it is stopped without that write-back.
 - **Rules on a mount:**
   - a rename across mounts is `EXDEV`, so `mv` copies;
-  - a hard link (`link()`, `linkat()`, Python's `os.link`, `ctx.fs.link`) works within a mount whose driver offers `links`, as `tmpfs` does: both names are one file with one inode, `st_nlink` counts them, and the data goes with the last name and the last open descriptor. Across mounts it's `EXDEV`. It's `EPERM` on the root file system (OPFS or memory), on `fsa` folders, on `hostfs` until slicc-node offers links, and for directories. `link()` links a symlink itself and `linkat(…, AT_SYMLINK_FOLLOW)` links its target, as on Linux. Descriptors the kernel holds (JS programs' files, and a WASI program's files once it forks or spawns with them) share one cache for all names of a file, so a write through one name is read at once through another, also after either name is removed or replaced while open; `link()` writes the source's pending changes back first. Within one WASI program, its own descriptors of a file's names share one buffer too. Once it starts a subprocess, the files it had open are kernel descriptors, and a later open of one of them, under that name or another (same inode), goes through the kernel and shares their cache; between programs, a WASI program's own descriptors see the other names' writes at open, as for one name across processes. A WASI program that removes or renames over the name it has open, while another link remains, keeps writing to the file through its descriptor, and the other names see those writes; `linkat(…, AT_SYMLINK_FOLLOW)` writes back the program's pending changes to the symlink's target first. `st_nlink` leaves out the names the kernel keeps in `.slicc-unlinked` for open descriptors, so `fstat` on a descriptor whose name was removed counts the names left: 0 once the last one is gone. The other names of a linked file keep the attributes the kernel has cached for them for up to the mount's `attrTtl` (`tmpfs` caches none). Emscripten programs (bash, coreutils `ln`) get `EMLINK` from their own C library, which never asks the kernel, but they see links other programs made: their open descriptors of one file's names share one buffer, and one that removes the name it has open while another link remains keeps writing to the file;
+  - a hard link (`link()`, `linkat()`, Python's `os.link`, `ctx.fs.link`) works within a mount whose driver offers `links`, as `tmpfs` does: both names are one file with one inode, `st_nlink` counts them, and the data goes with the last name and the last open descriptor. Across mounts it's `EXDEV`. It's `EPERM` on the root file system (OPFS or memory), on `fsa` folders, on `hostfs` until slicc-node offers links, and for directories; on the root the names are checked first, as Linux does: a missing source (or, when following, a dangling one) or target directory is `ENOENT`, a target directory that isn't one is `ENOTDIR`, and a taken name is `EEXIST`. `link()` links a symlink itself and `linkat(…, AT_SYMLINK_FOLLOW)` links its target, as on Linux. Descriptors the kernel holds (JS programs' files, and a WASI program's files once it forks or spawns with them) share one cache for all names of a file, so a write through one name is read at once through another, also after either name is removed or replaced while open; `link()` writes the source's pending changes back first. Within one WASI program, its own descriptors of a file's names share one buffer too. Once it starts a subprocess, the files it had open are kernel descriptors, and a later open of one of them, under that name or another (same inode), goes through the kernel and shares their cache; between programs, a WASI program's own descriptors see the other names' writes at open, as for one name across processes. A WASI program that removes or renames over the name it has open, while another link remains, keeps writing to the file through its descriptor, and the other names see those writes; `linkat(…, AT_SYMLINK_FOLLOW)` writes back the program's pending changes to the symlink's target first. `st_nlink` leaves out the names the kernel keeps in `.slicc-unlinked` for open descriptors, so `fstat` on a descriptor whose name was removed counts the names left: 0 once the last one is gone. The other names of a linked file keep the attributes the kernel has cached for them for up to the mount's `attrTtl` (`tmpfs` caches none). Emscripten programs (bash, coreutils `ln`) get `EMLINK` from their own C library, which never asks the kernel, but they see links other programs made: their open descriptors of one file's names share one buffer, and one that removes the name it has open while another link remains keeps writing to the file;
   - `df` reports each mount from its driver's `statfs`;
   - each mount has its own device number;
   - without `chmod` support, `chmod` succeeds and changes nothing;
@@ -600,6 +603,26 @@ A character device (`0666`, major 10, minor 250) through which a program talks t
 - **Reading:** the reply comes from the same descriptor, as JSON (`{ v: 1, … }`). A read waits for it, drains it, then returns end-of-file; with no request outstanding it returns end-of-file at once. A signal ends the wait with `EINTR` and the request stays outstanding, `O_NONBLOCK` gives `EAGAIN`, and `poll`/`select` report the descriptor readable once the reply is there. A new write drops an unread reply.
 - **The hook:** `signal` aborts with `'timeout'` after 10 s (the reply is then `{ v: 1, error: { code: 'ETIMEDOUT' } }`), `'replaced'` when a new write on the descriptor comes first, and `'withdrawn'` when the writer exits or the descriptor is closed. A throw or a reply that isn't an object becomes an `EIO` error reply, with the first line of the message; no hook is `ENOTCONN`. On the page, `createKernel` relays the request from the kernel worker and back, and the abort too.
 
+### Protected paths
+
+`createKernel({ protect: { paths: ['/node_modules', '/os'] } })` makes those directories read-only for every process, root included, in every `users` mode. They hold code every later process runs, which only the embedder writes, straight to OPFS.
+- **What fails:** any change under a path fails with `EROFS` ("Read-only file system"): a write, a create, a truncate, `mkdir`, removing or renaming (from or to), a hard or symbolic link created there, `chmod`, `chown` and setting times. So does removing or renaming a directory that holds one (`rm -rf /`). A missing working directory for `run` or `openTerminal` isn't created there either. Reads are unaffected. `'/'` protects the whole file system.
+- **Stats and symlinks:** files under a path show as on a read-only mount, so opens for writing fail at once. A write through a symlink, or through a symlinked directory, that leads into a path fails too.
+- **Where it's checked:** the root file system itself checks every change, from the moment the kernel is constructed, inside the change and against the path it really lands on. So a symlink swapped mid-way can't redirect a write. The kernel's own writes there are refused too, so don't protect what the kernel writes after it starts:
+  - `/etc` (`passwd`, `group` and the proxy's CA);
+  - `/home` and `/root` (new users' homes);
+  - `/tmp`;
+  - the system `PNPM_HOME` (`/usr/local/share/pnpm`);
+  - `/.slicc-unlinked` (files removed while open);
+  - `/` itself, which holds all of these.
+  Paths in the layout the kernel stamps with root's owner at boot are skipped when protected.
+- **Symlinks as paths:** a path that is itself a symlink protects the directory it names too, by its real path, once the kernel has prepared.
+- **Mounts:** nothing can be mounted at, under or above a path, not even by the embedder (`kernel.mount`, `/etc/fstab`). So a path always lies on the root file system, where that check holds.
+- **Clients:** a client's (`kernel.connect`) file calls, mounts and unmounts see the paths read-only, as its processes do.
+- **Not affected:** the page's own writes to OPFS.
+- **Writers:** a `writer` that lets chosen processes change the paths needs the kernel-side process tag ([#405](https://github.com/ai-ecoverse/slicc-kernel/pull/405)). Until then, `protect.writer` is refused.
+- **Where it is a boundary:** it's one in a browser, where programs' own code can't reach OPFS around the kernel (see [JS programs](#js-programs)). Under the Node entry it isn't, until [#407](https://github.com/ai-ecoverse/slicc-kernel/issues/407), and there `kernel.writeFile` under a path fails too, so write those files into `root` before creating the kernel.
+
 ### `/proc`
 
 Every process of the kernel has `/proc/<pid>/` with `cmdline`, `comm`, `stat`, `statm` and `status`, in the formats procps reads, and `/proc/self` is a link to the `/proc/<pid>` of the process that reads it, as on Linux (`readlink /proc/self` is its pid, the one it was exec'd under included). They come from the kernel's process table when they are opened, so a terminal sees the processes of every client. A process that exits while a program reads the `/proc` directory doesn't fail or cut short the listing: its pid is still listed, and opening its files fails with `ENOENT`, which `ps`, `pgrep` and `pidof` skip, as on Linux; an exec'd program shows under the pid its parent knows. `/proc/uptime`, `/proc/loadavg`, `/proc/stat` and `/proc/meminfo` are there too, and `/proc/mounts`. A process with no parent in the kernel (one a page, a client or the Node entry started) has parent 1, as children of init do, in `getppid()`, `/proc/<pid>/stat` and `ps` alike, and so does a process whose parent has ended, waited for or not (`getppid()` asks the kernel each time; a shell's `$PPID` keeps the value it started with, as on Linux). Pids, parents, process groups, sessions, command lines, terminals, start and boot times are real, and so are the credentials in `status` (`Uid:`, `Gid:`, `Groups:`) and each process's memory: the size of its wasm memory (`VSZ` and `RSS` alike, as wasm has no paging), which `/proc/meminfo` counts as used. CPU times are measured per process, its threads and the images it exec'd included: time a worker spends waiting for the kernel's answer is system time (at most 20 ms of each call; the rest of a longer one, such as a write into a full pipe, counts as blocked), time it spends blocked (a read, a `poll` or `select` that waits, a sleep, `wait`, a futex, while stopped) is neither (a readiness check that never waits is system time), and the rest is user time; a process's times stop when it ends, a call still in flight then adds nothing, so a loop that never calls the kernel shows as user time while it runs; they show in `/proc/<pid>/stat`, `ps` and `ps()`, and `/proc/stat` counts every process of the kernel, the reaped ones included, and `/proc/loadavg` averages the processes not blocked, sampled every 5 seconds. A JS program's time isn't measured yet. `MemTotal` (from `navigator.deviceMemory`, else 4 GiB) is a placeholder. A process that isn't root sees only its own user's processes (see [Users](#users)). WASI and WASIX programs see the same `/proc`, read-only (opening a file in it for writing, or linking one of its files elsewhere, fails with `EROFS`), from every thread, also through a symlink that points into it (where every change, `mkdir`, `rename`, `unlink`, `chmod`, a hard link from it or into it, and so on, fails too: with `EROFS` when the root file system has a physical `/proc`, which stays hidden and untouched, and otherwise with the `ENOENT` of the missing directory, since that check is only paid for when a physical `/proc` exists; with one, a symlink whose target passes `..` after a file or a missing name, such as `/f/../proc`, fails with `ENOTDIR` or `ENOENT`, as on Linux), and `/` lists it, so a WASI program gets a preopen for it; `/proc/self` is the program's own pid there too (the pid an exec'd program took over). Neither has `/proc/<pid>/fd` yet.

package/dist/index.js

@@ -287,6 +287,9 @@ async function loopbackFetch(dial, input, options) {
   }
 }
 
+// src/fs/protect.ts
+var PROTECT_WRITER = "slicc-kernel: protect.writer needs the process tag (#405), which this kernel does not have yet";
+
 // src/kernel/net/loopback-names.ts
 var HOST_LOOPBACK = "host.slicc.internal";
 var HOST_LOOPBACK_ADDRESS = [10, 0, 2, 2];
@@ -1693,7 +1696,7 @@ function localProxyTransport(options) {
 }
 
 // src/mount/driver.ts
-function fsError(code, message = code) {
+function fsError2(code, message = code) {
   return Object.assign(new Error(message), { code });
 }
 
@@ -1775,7 +1778,7 @@ function serveHostfsPort(port, transport) {
         );
         return send2({ id });
       }
-      if (op !== "call") throw fsError("EINVAL", `unknown hostfs port op ${op}`);
+      if (op !== "call") throw fsError2("EINVAL", `unknown hostfs port op ${op}`);
       send2({ id, result: await answer(data.name, data.body) });
     } catch (err) {
       send2({ id, error: errorOf(err) });
@@ -1842,6 +1845,7 @@ function initCall(options, transport) {
     ...options.onDenial ? { denials: true } : {},
     ...options.sealed ? { sealed: true } : {},
     ...options.approve ? { approve: true } : {},
+    ...options.protect ? { protect: { paths: [...options.protect.paths] } } : {},
     ...options.processMounts !== void 0 ? {
       processMounts: typeof options.processMounts === "function" ? "ask" : options.processMounts
     } : {}
@@ -1936,6 +1940,7 @@ function checkOptions(options) {
 }
 async function createKernel(options = {}) {
   checkOptions(options);
+  if (options.protect?.writer) throw new Error(PROTECT_WRITER);
   const url = options.worker ?? new URL("./kernel-worker.js", import.meta.url);
   const worker = new Worker(url, { type: "module", name: "slicc-kernel" });
   const pending = /* @__PURE__ */ new Map();