@ai-ecoverse/slicc-kernel 1.178.4 → 1.180.0

package/README.md

@@ -57,6 +57,7 @@ Runs `argv[0]` with `argv` as its arguments and resolves when it exits. `stdout`
 | option | |
 | --- | --- |
 | `cwd` | working directory, default `/`; created in OPFS if missing. Under `users: 'enforce'`, a `cwd` the user can't search rejects with `code: 'EACCES'` (`cannot change to <cwd>: Permission denied`) and starts nothing, as a `chdir` there fails on Linux; `openTerminal` does the same. A program's spawn with its own `cwd` (`ctx.spawn`, `ctx.sync.run`, a WASIX `posix_spawn` chdir action) is checked the same way and fails with `EACCES`, while a child that inherits its parent's cwd keeps it unchecked, as on Linux |
+| `cwdMissing` | `'create'` (the default) or `'error'`: what happens when `cwd` doesn't exist. `'create'` creates it, with its parents, as described for `cwd`. `'error'` rejects with `code: 'ENOENT'` (`cannot change to <cwd>: No such file or directory`) and starts nothing, as a `chdir` there fails on Linux. A caller that keeps directories in a list to run commands in, and so may name one that was removed since, should pass `'error'`: with `'create'`, its next command brings the removed directory back, empty ([#1808](https://github.com/ai-ecoverse/slicc-kernel/issues/1808)). Any other value rejects with a `TypeError`; `openTerminal` takes it too |
 | `user` | the user to run as, by name or uid (see [Users](#users)); default `root`. An unknown user rejects with `code: 'ENOENT'` |
 | `tag` | a string the kernel keeps with the process and every process it starts (fork, spawn and exec), and hands to `approve` with a [`sudo`](#sudo) request, so the embedder knows which agent asks; nothing a program does changes it. Anything but a string rejects with a `TypeError` |
 | `env` | extra environment, over the user's defaults: `HOME`, `USER`, `LOGNAME`, `PNPM_HOME` and `PATH=$PNPM_HOME/bin:/usr/local/bin:/usr/bin:/bin` (see [Users](#users)), `TZ=<system time zone>` (see [Time zone](#time-zone)), and `PWD=<cwd>` |
@@ -69,7 +70,7 @@ Starts `argv` as the session leader on a new terminal and resolves once it is ru
 
 | option | |
 | --- | --- |
-| `cwd`, `user`, `tag`, `env` | as for `run`; `TERM=xterm-256color` and `COLORTERM=truecolor` are added |
+| `cwd`, `cwdMissing`, `user`, `tag`, `env` | as for `run`; `TERM=xterm-256color` and `COLORTERM=truecolor` are added |
 | `cols`, `rows` | initial size, default 80 × 24, so `$COLUMNS` and `$LINES` are right before the first prompt |
 | `onData` | called with each chunk of output as raw bytes (escape sequences included) |
 
@@ -94,7 +95,7 @@ Makes a port for another client of the same kernel. Hand it to a dedicated worke
 
 **`requests`** binds the port's [`/dev/embedder`](#devembedder) requests when the port is made: a hook (`{ kinds, handle }`, as `createKernel`'s) answers for every process the port's client starts, in the embedder's own realm, so a worker client (an agent) never answers or registers requests itself. `'attach'` lets the client that attaches to the port bring its own hook (`attachKernel(port, { requests })`, protocol 1.13); without it, an attach with `requests` fails with `EPERM`. A port with neither answers nothing: its processes' requests are `ENOTSUP`.
 
-**`user`** (a name or uid) binds the client to that user; without it the client is root. The binding is a capability: the client's `spawn`, `run` and `openTerminal` run as that user, its `ps` lists only that user's processes, and its `kill` follows POSIX permission. A bound client can't run anything as another user (`code: 'EPERM'`). A root client may pass `user` to `spawn`, `run` or `openTerminal` to run one command as any user. An unknown user rejects with `code: 'ENOENT'`. **`tag`** is the tag (see `kernel.run`) of every process the client starts; its own `spawn`, `run` and `openTerminal` cannot name another. They may add one with their own `tag` option (protocol 1.12; an older kernel answers `ENOSYS`): the process then has both tags, the connection's first, and so does everything it starts. A per-spawn tag only adds; no client, program or child can remove or replace a tag ([#1380](https://github.com/ai-ecoverse/slicc-kernel/issues/1380)).
+**`user`** (a name or uid) binds the client to that user; without it the client is root. The binding is a capability: the client's `spawn`, `run` and `openTerminal` run as that user, its `ps` lists only that user's processes, and its `kill` follows POSIX permission. A bound client can't run anything as another user (`code: 'EPERM'`). A root client may pass `user` to `spawn`, `run` or `openTerminal` to run one command as any user. An unknown user rejects with `code: 'ENOENT'`. **`tag`** is the tag (see `kernel.run`) of every process the client starts; its own `spawn`, `run` and `openTerminal` cannot name another. They may add one with their own `tag` option (protocol 1.12; an older kernel answers `ENOSYS`): the process then has both tags, the connection's first, and so does everything it starts. The option may also be a list of at most 8 tags, added in order (protocol 1.15; against an older kernel the client rejects with `code: 'ENOSYS'` before sending anything), so a caller can mark one call's tree with a tag of its own for [`trace`](#clienttracetag-onevents--limit---promise-close-) beside a tag that means something else. A per-spawn tag only adds; no client, program or child can remove or replace a tag ([#1380](https://github.com/ai-ecoverse/slicc-kernel/issues/1380)).
 
 ### `kernel.users`
 
@@ -176,7 +177,7 @@ Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker
 
 | Client | |
 |---|---|
-| `spawn(argv, { cwd, user, env, stdin, pgid, onStdout, onStderr })` | runs over pipes and resolves with `{ pid, pgid, exited, ended, signal(name) }` once the process has started; `ended` resolves with `{ status, signal }`, `signal` named only when a signal ended the process, as for `kernel.run` (an older kernel never names one); output arrives in chunks, as bytes, and the kernel keeps no copy; every spawn leads its own process group, which `signal` (default `SIGTERM`) signals, unless `pgid` names an existing group to join (protocol 1.3): the process then joins that group and that group's session, as `setpgid` would (a client has no session of its own: each of its spawns and terminals leads one, and `setpgid` cannot cross sessions), and `signal` signals the whole group. The group must have been started by this client (a spawn or a terminal it opened), even if its leader has exited; an unknown group rejects with `code: 'ESRCH'`, another session's with `code: 'EPERM'`, and against a 1.2 kernel the client rejects with `code: 'ENOSYS'` before sending anything, as an older kernel would ignore the option; a program that cannot start rejects with `code: 'ENOENT'`. `user` runs it as another user, which only a root client may (see `kernel.connect`); against a 1.7 kernel the client rejects with `code: 'ENOSYS'` |
+| `spawn(argv, { cwd, cwdMissing, user, env, stdin, pgid, onStdout, onStderr })` | runs over pipes and resolves with `{ pid, pgid, exited, ended, signal(name) }` once the process has started; `ended` resolves with `{ status, signal }`, `signal` named only when a signal ended the process, as for `kernel.run` (an older kernel never names one); output arrives in chunks, as bytes, and the kernel keeps no copy; every spawn leads its own process group, which `signal` (default `SIGTERM`) signals, unless `pgid` names an existing group to join (protocol 1.3): the process then joins that group and that group's session, as `setpgid` would (a client has no session of its own: each of its spawns and terminals leads one, and `setpgid` cannot cross sessions), and `signal` signals the whole group. The group must have been started by this client (a spawn or a terminal it opened), even if its leader has exited; an unknown group rejects with `code: 'ESRCH'`, another session's with `code: 'EPERM'`, and against a 1.2 kernel the client rejects with `code: 'ENOSYS'` before sending anything, as an older kernel would ignore the option; a program that cannot start rejects with `code: 'ENOENT'`. `user` runs it as another user, which only a root client may (see `kernel.connect`); against a 1.7 kernel the client rejects with `code: 'ENOSYS'`. `cwdMissing` is as for `kernel.run` (protocol 1.14); `run` and `openTerminal` take it too, and with `'error'` against a 1.13 kernel, which would create the directory, the client rejects with `code: 'ENOSYS'` before sending anything |
 | `run(argv, options?)` | `spawn` and wait: `{ pid, status, signal, stdout, stderr }` as text, with `onStdout`/`onStderr` streaming text; `signal` as for `kernel.run` |
 | `openTerminal(argv, options?)` | a pty session, as `kernel.openTerminal`, `ended` included |
 | `ps()` | the kernel's processes: `{ pid, ppid, pgid, sid, argv, tty, started, state, rss, utime, stime, uid, gid }`, `state` being `'S'`, `'T'` (stopped) or `'Z'`, `rss` the process's resident memory in bytes (the size of its wasm memory, what `/proc/<pid>/status` shows as `VmRSS`; absent from kernels before 1.61), `utime`/`stime` the CPU time it has used, in milliseconds, in its program and in the kernel's calls (see [`/proc`](#proc); absent for a JS program and from kernels before 1.63) and `uid`/`gid` the effective ids (absent from a 1.7 kernel), and `tag` the process's tag (see `kernel.run`), when it has one, with `tags`, all of them in order, when a spawn added one (see `kernel.connect`), on rows the client's user may see as root or as the same uid. A client bound to a user sees only that user's processes |
@@ -186,12 +187,27 @@ Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker
 | `fs.watch(paths, { recursive }, onChange)` | resolves with `{ close() }`; `onChange` gets `{ paths }`, the changed paths at, below (or, without `recursive`, directly in) the watched ones, batched per task, or `{ overflow: true }` when there were too many to list, to rescan; it covers every change made through this kernel, by its processes and its clients, and a removal or rename that failed or found nothing to do is none. It also covers what another kernel on the same OPFS root changes (another tab, a worker page) and what a writer outside the kernel reports with [`postFsNotice`](#metadata), as those notices name them: a directory another kernel creates, removes or renames, the file it writes back, and the records of its symlinks, modes, owners and times, and a notice's `trees` as their roots. A peer's notices come only once the kernels share the root (`/proc/sys/kernel/slicc_shared_root` reads `1` from the join handshake on) and only with the shared metadata (not with `metadata: false`), and a writer outside the kernel that posts no notice is never heard. So a client that has to catch a page writing OPFS directly needs that page to call `postFsNotice` after each write. Kernels before this one didn't pass peer and outside changes to a client's watch; they reached only `ctx.fs.watch`. A watch is on names, not files: a watched path need not exist, nor its directories, and one under a regular file is a watch too (every kernel with `watch` does this, protocol 1.1 on), so its creation, by any `mkdir -p` above it, a write or a rename to it, is reported as its path, and so are a removal or rename of it or of a directory above it; the watch goes on hearing the name after that. So a client that waits for a directory to appear watches it, rather than polling `stat` and `readdir`. Under `users`, a watched path the client may not read is `EACCES`, and so is a missing one under a directory the client may not search. Changes are reported by the name they were made under (a file written through a symlink also by its real path), so a client that watches a path under a symlink watches its real path too, for a missing path the real path of its nearest existing directory with the rest of the path after it |
 | `serveCdp(hook, { runtime })` | offers this client's browser to the kernel's programs (protocol 1.5): `hook` is the `cdp` hook of `createKernel`, asked once per program connection; resolves with `{ close() }`, and closing or detaching takes the offer back and closes its connections (see [Browser automation](#browser-automation-cdp)) |
 | `serveFacade(name, hook) → { close() }` | serves a facade that `createKernel({ facades })` declared, the same way as its `hook` (protocol 1.11); the latest registration wins, over the embedder's hook too, and `close()` hands it back. An undeclared name is `EINVAL` |
+| `trace(tag, onEvents, { limit }?)` | reports, as events, every program that a process carrying `tag` runs and every such process's end, with nothing missed (protocol 1.15; against an older kernel the client rejects with `code: 'ENOSYS'`); resolves with `{ close() }` (see [below](#clienttracetag-onevents--limit---promise-close-)) |
 | `onTraits(listener) → () => void` | calls `listener(traits)` each time the kernel announces new transport traits (`kernel.updateTransportTraits()`, protocol 1.10), after `client.transport.traits` has been updated; the function it returns removes the listener |
 | `close({ kill })` | detaches: the client's processes keep running unless `kill` is set, which ends their process groups with `SIGKILL`; its terminals are hung up |
 | `closed` | resolves with the error that ended the client |
 | `users` | the kernel's `users` mode, `'off'`, `'report'` or `'enforce'`; `'off'` from kernels that don't say |
 
-When the kernel goes away (the page closed or reloaded, `terminate()`), pending calls and `exited` reject with `KernelGoneError`, and so do later calls. Each side holds a Web Lock and waits on the other's, since a `MessagePort` reports no close in browsers; in Node, the port's `close` event does the same. The first message is a handshake on the protocol version, `1.12` (`1.0` had no `watch`, `1.1` no mounts, `1.2` no `spawn` into a group, `1.3` no `dial` and `1.4` no `serveCdp`, which reject with `code: 'ENOSYS'` on such a kernel; a `1.5` kernel has no [uplink](#uplink), a `1.6` kernel resolves names through one but connects nothing to it, a `1.7` kernel has no users, a `1.8` kernel has no `sudo` and no tags, a `1.9` kernel never announces new transport traits, and a `1.11` kernel has no per-spawn `tag`): a client or kernel of another major version is refused with an error naming both.
+When the kernel goes away (the page closed or reloaded, `terminate()`), pending calls and `exited` reject with `KernelGoneError`, and so do later calls. Each side holds a Web Lock and waits on the other's, since a `MessagePort` reports no close in browsers; in Node, the port's `close` event does the same. The first message is a handshake on the protocol version, `1.15` (`1.0` had no `watch`, `1.1` no mounts, `1.2` no `spawn` into a group, `1.3` no `dial` and `1.4` no `serveCdp`, which reject with `code: 'ENOSYS'` on such a kernel; a `1.5` kernel has no [uplink](#uplink), a `1.6` kernel resolves names through one but connects nothing to it, a `1.7` kernel has no users, a `1.8` kernel has no `sudo` and no tags, a `1.9` kernel never announces new transport traits, a `1.11` kernel has no per-spawn `tag`, a `1.12` kernel takes no `requests` at attach, a `1.13` kernel has no `cwdMissing`, and a `1.14` kernel has no `trace` and takes one tag per spawn, not a list, which the client rejects with `code: 'ENOSYS'` before sending anything): a client or kernel of another major version is refused with an error naming both.
+
+#### `client.trace(tag, onEvents, { limit }?) → Promise<{ close() }>`
+
+Tells a client which programs a command really ran, without polling `ps()`, which misses a `seq` or a `head` that lives for a millisecond. The kernel runs real bash, so builtins and keywords (`cd`, `for`, `printf`, `echo`) never start a program, and what does is what the command was about (`seq`, `sed`, `node`, the program behind `timeout`). Give each call a tag of its own (the spawn's `tag`, which may be a list, see [`kernel.connect`](#kernelconnect-user-tag-requests---promisemessageport)), subscribe to it before the spawn, and `onEvents` gets every process that carries the tag, in its whole tree, background jobs that outlive the command included:
+
+- **`exec`** `{ type, seq, pid, ppid, pgid, argv, exe, uid, at, replaced? }`: a program started, for the first image of a spawn, a child a program spawned (`posix_spawn`, as `timeout` runs its command) and each successful `execve`. `replaced: true` marks an `execve`, whose program keeps the `pid` of the image it replaced (bash runs its last command that way, so `bash -c '…; node x'` ends with an `exec` of `node` under bash's pid). An `execve` that fails (`command not found`) has no event.
+- **`fork`** `{ type, seq, pid, ppid, pgid, argv, exe, uid, at }`: a `fork()`. Its `argv` and `exe` are the parent's, a forked bash for a subshell, a pipeline element or a command it is about to run. A fork that then runs a program has an `exec` with the same `pid` next; one that never has an `exec` ran only builtins, so `type` tells a forked `bash` from a real program.
+- **`exit`** `{ type, seq, pid, status, signal?, utime?, stime?, at }`: a process ended, once per `pid`, after its last image: `status` as `waitpid` would report it (127 for a command that wasn't found, 141 for an unhandled `SIGPIPE`), `signal` the name of the signal that killed it (`'SIGKILL'`), `utime`/`stime` the milliseconds of CPU it used, its images together, as in `ps()` (absent for a JS program).
+
+`pid`, `ppid` and `pgid` are the ids `ps()` shows, `ppid` 1 for a spawn's first process; `uid` is the effective uid; `at` is the time in ms since the epoch. The kernel reports from the path every process takes to start and end, so a process that execs and exits within a millisecond is reported like any other. Events come in order: `seq` is one kernel-wide counter, increasing in the order the kernel saw the events (with gaps for events only other subscriptions got), so it orders the execs and exits across a tree more finely than `at`, and for one `pid` the `fork` or `exec` comes before its `exit`. A process that started before the subscription reports only its exit.
+
+A client bound to a user sees only the processes of that user it would see in `ps()` (its own real or effective uid; root sees all), so another user's processes with the same tag never reach it, and a program that `sudo` runs as root drops out of its view. A tag matches exactly, any of a process's tags.
+
+**Batches and overflow:** `onEvents` gets `{ events, dropped }`, the events since the last batch. The next batch waits until the call returns, or, when it returns a promise, until that settles, so a slow consumer holds the kernel back instead of piling up messages. While it waits, at most `limit` events (4096 by default, at most 65536) wait in the kernel for that subscription; any more are dropped and counted, and the next batch says how many in `dropped` (0 otherwise), so a fork bomb costs a bounded amount of memory, and a consumer that sees `dropped > 0` knows its list is incomplete. `close()` ends the subscription, and so does closing the client. An empty tag or a bad `limit` rejects with `code: 'EINVAL'`.
 
 ### Headless in Node, for tests
 

package/dist/driver-worker.js

@@ -824,7 +824,7 @@ async function loopbackFetch(dial, input, options) {
 }
 
 // src/client/protocol.ts
-var PROTOCOL = [1, 13];
+var PROTOCOL = [1, 15];
 function locksOf(scope = globalThis) {
   const locks = scope.navigator?.locks;
   return typeof locks?.request === "function" ? locks : void 0;
@@ -889,6 +889,7 @@ var Channel = class {
   failure;
   pending = /* @__PURE__ */ new Map();
   watching = /* @__PURE__ */ new Map();
+  tracing = /* @__PURE__ */ new Map();
   cdpHooks = /* @__PURE__ */ new Map();
   cdpOpen = /* @__PURE__ */ new Set();
   facadeHooks = /* @__PURE__ */ new Map();
@@ -956,6 +957,10 @@ var Channel = class {
       this.watching.get(data.watch)?.(data.change);
       return;
     }
+    if (data.trace !== void 0 && data.id === void 0) {
+      this.traced(data.trace, data.events ?? [], data.dropped ?? 0);
+      return;
+    }
     if (data.bye !== void 0) {
       this.end(new KernelGoneError(data.bye));
       return;
@@ -984,6 +989,23 @@ var Channel = class {
     if (reply.error !== void 0) call.reject(new KernelCallError(reply.error, reply.code));
     else call.resolve(reply.result);
   }
+  traced(trace, events, dropped) {
+    const listener = this.tracing.get(trace);
+    if (!listener) return;
+    const ack = () => {
+      if (this.tracing.has(trace)) this.post({ traceAck: trace });
+    };
+    let consumed;
+    try {
+      consumed = listener({ events, dropped });
+    } finally {
+      if (typeof consumed?.then === "function") {
+        void Promise.resolve(consumed).then(ack, ack);
+      } else {
+        ack();
+      }
+    }
+  }
   openCdp({ cdpOpen: { registration, ...request }, port }) {
     const hook = registration === void 0 ? void 0 : this.cdpHooks.get(registration);
     if (!hook || this.failure) {
@@ -1046,6 +1068,29 @@ async function serveCdpOn(channel, hook, options) {
     }
   };
 }
+async function traceOn(channel, tag, onEvents, options) {
+  if (typeof onEvents !== "function") throw new TypeError("trace: onEvents must be a function");
+  await since(channel, 15, "trace", async () => {
+  });
+  const { id, done } = channel.request({
+    op: "trace",
+    tag,
+    ...options.limit !== void 0 ? { limit: options.limit } : {}
+  });
+  channel.tracing.set(id, onEvents);
+  try {
+    await done;
+  } catch (error) {
+    channel.tracing.delete(id);
+    throw error;
+  }
+  return {
+    close: () => {
+      if (!channel.tracing.delete(id)) return;
+      void channel.call({ op: "untrace", trace: id }).catch(() => void 0);
+    }
+  };
+}
 async function dialOn(channel, { port, host }) {
   const req = { op: "dial", port, ...host ? { host } : {} };
   return dialStream(await since(channel, 4, "dial", () => channel.call(req)));
@@ -1059,6 +1104,10 @@ async function spawnOn(channel, argv, opts) {
   });
   if (opts.tag !== void 0) await since(channel, 12, "a tag per spawn", async () => {
   });
+  if (Array.isArray(opts.tag)) await since(channel, 15, "several tags per spawn", async () => {
+  });
+  if (opts.cwdMissing === "error") await since(channel, 14, "cwdMissing: 'error'", async () => {
+  });
   const started = Promise.withResolvers();
   let endedBy;
   const options = { ...rest, ...stdin !== void 0 ? { stdin: bytesOf2(stdin) } : {} };
@@ -1118,6 +1167,12 @@ async function terminalOn(channel, argv, opts) {
   });
   if (opts.tag !== void 0) await since(channel, 12, "a tag per terminal", async () => {
   });
+  if (Array.isArray(opts.tag)) {
+    await since(channel, 15, "several tags per terminal", async () => {
+    });
+  }
+  if (opts.cwdMissing === "error") await since(channel, 14, "cwdMissing: 'error'", async () => {
+  });
   let listener = onData ?? null;
   const backlog = [];
   const started = Promise.withResolvers();
@@ -1275,6 +1330,7 @@ async function attachKernel(port, options = {}) {
       channel.traitsListeners.add(listener);
       return () => channel.traitsListeners.delete(listener);
     },
+    trace: (tag, onEvents, options2 = {}) => traceOn(channel, tag, onEvents, options2),
     async close(o = {}) {
       if (channel.failure) return;
       await channel.call({ op: "detach", ...o.kill ? { kill: true } : {} }).catch(() => void 0);