@ai-ecoverse/slicc-kernel 1.179.0 → 1.180.0

package/README.md

@@ -95,7 +95,7 @@ Makes a port for another client of the same kernel. Hand it to a dedicated worke
 
 **`requests`** binds the port's [`/dev/embedder`](#devembedder) requests when the port is made: a hook (`{ kinds, handle }`, as `createKernel`'s) answers for every process the port's client starts, in the embedder's own realm, so a worker client (an agent) never answers or registers requests itself. `'attach'` lets the client that attaches to the port bring its own hook (`attachKernel(port, { requests })`, protocol 1.13); without it, an attach with `requests` fails with `EPERM`. A port with neither answers nothing: its processes' requests are `ENOTSUP`.
 
-**`user`** (a name or uid) binds the client to that user; without it the client is root. The binding is a capability: the client's `spawn`, `run` and `openTerminal` run as that user, its `ps` lists only that user's processes, and its `kill` follows POSIX permission. A bound client can't run anything as another user (`code: 'EPERM'`). A root client may pass `user` to `spawn`, `run` or `openTerminal` to run one command as any user. An unknown user rejects with `code: 'ENOENT'`. **`tag`** is the tag (see `kernel.run`) of every process the client starts; its own `spawn`, `run` and `openTerminal` cannot name another. They may add one with their own `tag` option (protocol 1.12; an older kernel answers `ENOSYS`): the process then has both tags, the connection's first, and so does everything it starts. A per-spawn tag only adds; no client, program or child can remove or replace a tag ([#1380](https://github.com/ai-ecoverse/slicc-kernel/issues/1380)).
+**`user`** (a name or uid) binds the client to that user; without it the client is root. The binding is a capability: the client's `spawn`, `run` and `openTerminal` run as that user, its `ps` lists only that user's processes, and its `kill` follows POSIX permission. A bound client can't run anything as another user (`code: 'EPERM'`). A root client may pass `user` to `spawn`, `run` or `openTerminal` to run one command as any user. An unknown user rejects with `code: 'ENOENT'`. **`tag`** is the tag (see `kernel.run`) of every process the client starts; its own `spawn`, `run` and `openTerminal` cannot name another. They may add one with their own `tag` option (protocol 1.12; an older kernel answers `ENOSYS`): the process then has both tags, the connection's first, and so does everything it starts. The option may also be a list of at most 8 tags, added in order (protocol 1.15; against an older kernel the client rejects with `code: 'ENOSYS'` before sending anything), so a caller can mark one call's tree with a tag of its own for [`trace`](#clienttracetag-onevents--limit---promise-close-) beside a tag that means something else. A per-spawn tag only adds; no client, program or child can remove or replace a tag ([#1380](https://github.com/ai-ecoverse/slicc-kernel/issues/1380)).
 
 ### `kernel.users`
 
@@ -187,12 +187,27 @@ Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker
 | `fs.watch(paths, { recursive }, onChange)` | resolves with `{ close() }`; `onChange` gets `{ paths }`, the changed paths at, below (or, without `recursive`, directly in) the watched ones, batched per task, or `{ overflow: true }` when there were too many to list, to rescan; it covers every change made through this kernel, by its processes and its clients, and a removal or rename that failed or found nothing to do is none. It also covers what another kernel on the same OPFS root changes (another tab, a worker page) and what a writer outside the kernel reports with [`postFsNotice`](#metadata), as those notices name them: a directory another kernel creates, removes or renames, the file it writes back, and the records of its symlinks, modes, owners and times, and a notice's `trees` as their roots. A peer's notices come only once the kernels share the root (`/proc/sys/kernel/slicc_shared_root` reads `1` from the join handshake on) and only with the shared metadata (not with `metadata: false`), and a writer outside the kernel that posts no notice is never heard. So a client that has to catch a page writing OPFS directly needs that page to call `postFsNotice` after each write. Kernels before this one didn't pass peer and outside changes to a client's watch; they reached only `ctx.fs.watch`. A watch is on names, not files: a watched path need not exist, nor its directories, and one under a regular file is a watch too (every kernel with `watch` does this, protocol 1.1 on), so its creation, by any `mkdir -p` above it, a write or a rename to it, is reported as its path, and so are a removal or rename of it or of a directory above it; the watch goes on hearing the name after that. So a client that waits for a directory to appear watches it, rather than polling `stat` and `readdir`. Under `users`, a watched path the client may not read is `EACCES`, and so is a missing one under a directory the client may not search. Changes are reported by the name they were made under (a file written through a symlink also by its real path), so a client that watches a path under a symlink watches its real path too, for a missing path the real path of its nearest existing directory with the rest of the path after it |
 | `serveCdp(hook, { runtime })` | offers this client's browser to the kernel's programs (protocol 1.5): `hook` is the `cdp` hook of `createKernel`, asked once per program connection; resolves with `{ close() }`, and closing or detaching takes the offer back and closes its connections (see [Browser automation](#browser-automation-cdp)) |
 | `serveFacade(name, hook) → { close() }` | serves a facade that `createKernel({ facades })` declared, the same way as its `hook` (protocol 1.11); the latest registration wins, over the embedder's hook too, and `close()` hands it back. An undeclared name is `EINVAL` |
+| `trace(tag, onEvents, { limit }?)` | reports, as events, every program that a process carrying `tag` runs and every such process's end, with nothing missed (protocol 1.15; against an older kernel the client rejects with `code: 'ENOSYS'`); resolves with `{ close() }` (see [below](#clienttracetag-onevents--limit---promise-close-)) |
 | `onTraits(listener) → () => void` | calls `listener(traits)` each time the kernel announces new transport traits (`kernel.updateTransportTraits()`, protocol 1.10), after `client.transport.traits` has been updated; the function it returns removes the listener |
 | `close({ kill })` | detaches: the client's processes keep running unless `kill` is set, which ends their process groups with `SIGKILL`; its terminals are hung up |
 | `closed` | resolves with the error that ended the client |
 | `users` | the kernel's `users` mode, `'off'`, `'report'` or `'enforce'`; `'off'` from kernels that don't say |
 
-When the kernel goes away (the page closed or reloaded, `terminate()`), pending calls and `exited` reject with `KernelGoneError`, and so do later calls. Each side holds a Web Lock and waits on the other's, since a `MessagePort` reports no close in browsers; in Node, the port's `close` event does the same. The first message is a handshake on the protocol version, `1.12` (`1.0` had no `watch`, `1.1` no mounts, `1.2` no `spawn` into a group, `1.3` no `dial` and `1.4` no `serveCdp`, which reject with `code: 'ENOSYS'` on such a kernel; a `1.5` kernel has no [uplink](#uplink), a `1.6` kernel resolves names through one but connects nothing to it, a `1.7` kernel has no users, a `1.8` kernel has no `sudo` and no tags, a `1.9` kernel never announces new transport traits, and a `1.11` kernel has no per-spawn `tag`): a client or kernel of another major version is refused with an error naming both.
+When the kernel goes away (the page closed or reloaded, `terminate()`), pending calls and `exited` reject with `KernelGoneError`, and so do later calls. Each side holds a Web Lock and waits on the other's, since a `MessagePort` reports no close in browsers; in Node, the port's `close` event does the same. The first message is a handshake on the protocol version, `1.15` (`1.0` had no `watch`, `1.1` no mounts, `1.2` no `spawn` into a group, `1.3` no `dial` and `1.4` no `serveCdp`, which reject with `code: 'ENOSYS'` on such a kernel; a `1.5` kernel has no [uplink](#uplink), a `1.6` kernel resolves names through one but connects nothing to it, a `1.7` kernel has no users, a `1.8` kernel has no `sudo` and no tags, a `1.9` kernel never announces new transport traits, a `1.11` kernel has no per-spawn `tag`, a `1.12` kernel takes no `requests` at attach, a `1.13` kernel has no `cwdMissing`, and a `1.14` kernel has no `trace` and takes one tag per spawn, not a list, which the client rejects with `code: 'ENOSYS'` before sending anything): a client or kernel of another major version is refused with an error naming both.
+
+#### `client.trace(tag, onEvents, { limit }?) → Promise<{ close() }>`
+
+Tells a client which programs a command really ran, without polling `ps()`, which misses a `seq` or a `head` that lives for a millisecond. The kernel runs real bash, so builtins and keywords (`cd`, `for`, `printf`, `echo`) never start a program, and what does is what the command was about (`seq`, `sed`, `node`, the program behind `timeout`). Give each call a tag of its own (the spawn's `tag`, which may be a list, see [`kernel.connect`](#kernelconnect-user-tag-requests---promisemessageport)), subscribe to it before the spawn, and `onEvents` gets every process that carries the tag, in its whole tree, background jobs that outlive the command included:
+
+- **`exec`** `{ type, seq, pid, ppid, pgid, argv, exe, uid, at, replaced? }`: a program started, for the first image of a spawn, a child a program spawned (`posix_spawn`, as `timeout` runs its command) and each successful `execve`. `replaced: true` marks an `execve`, whose program keeps the `pid` of the image it replaced (bash runs its last command that way, so `bash -c '…; node x'` ends with an `exec` of `node` under bash's pid). An `execve` that fails (`command not found`) has no event.
+- **`fork`** `{ type, seq, pid, ppid, pgid, argv, exe, uid, at }`: a `fork()`. Its `argv` and `exe` are the parent's, a forked bash for a subshell, a pipeline element or a command it is about to run. A fork that then runs a program has an `exec` with the same `pid` next; one that never has an `exec` ran only builtins, so `type` tells a forked `bash` from a real program.
+- **`exit`** `{ type, seq, pid, status, signal?, utime?, stime?, at }`: a process ended, once per `pid`, after its last image: `status` as `waitpid` would report it (127 for a command that wasn't found, 141 for an unhandled `SIGPIPE`), `signal` the name of the signal that killed it (`'SIGKILL'`), `utime`/`stime` the milliseconds of CPU it used, its images together, as in `ps()` (absent for a JS program).
+
+`pid`, `ppid` and `pgid` are the ids `ps()` shows, `ppid` 1 for a spawn's first process; `uid` is the effective uid; `at` is the time in ms since the epoch. The kernel reports from the path every process takes to start and end, so a process that execs and exits within a millisecond is reported like any other. Events come in order: `seq` is one kernel-wide counter, increasing in the order the kernel saw the events (with gaps for events only other subscriptions got), so it orders the execs and exits across a tree more finely than `at`, and for one `pid` the `fork` or `exec` comes before its `exit`. A process that started before the subscription reports only its exit.
+
+A client bound to a user sees only the processes of that user it would see in `ps()` (its own real or effective uid; root sees all), so another user's processes with the same tag never reach it, and a program that `sudo` runs as root drops out of its view. A tag matches exactly, any of a process's tags.
+
+**Batches and overflow:** `onEvents` gets `{ events, dropped }`, the events since the last batch. The next batch waits until the call returns, or, when it returns a promise, until that settles, so a slow consumer holds the kernel back instead of piling up messages. While it waits, at most `limit` events (4096 by default, at most 65536) wait in the kernel for that subscription; any more are dropped and counted, and the next batch says how many in `dropped` (0 otherwise), so a fork bomb costs a bounded amount of memory, and a consumer that sees `dropped > 0` knows its list is incomplete. `close()` ends the subscription, and so does closing the client. An empty tag or a bad `limit` rejects with `code: 'EINVAL'`.
 
 ### Headless in Node, for tests
 

package/dist/driver-worker.js

@@ -824,7 +824,7 @@ async function loopbackFetch(dial, input, options) {
 }
 
 // src/client/protocol.ts
-var PROTOCOL = [1, 14];
+var PROTOCOL = [1, 15];
 function locksOf(scope = globalThis) {
   const locks = scope.navigator?.locks;
   return typeof locks?.request === "function" ? locks : void 0;
@@ -889,6 +889,7 @@ var Channel = class {
   failure;
   pending = /* @__PURE__ */ new Map();
   watching = /* @__PURE__ */ new Map();
+  tracing = /* @__PURE__ */ new Map();
   cdpHooks = /* @__PURE__ */ new Map();
   cdpOpen = /* @__PURE__ */ new Set();
   facadeHooks = /* @__PURE__ */ new Map();
@@ -956,6 +957,10 @@ var Channel = class {
       this.watching.get(data.watch)?.(data.change);
       return;
     }
+    if (data.trace !== void 0 && data.id === void 0) {
+      this.traced(data.trace, data.events ?? [], data.dropped ?? 0);
+      return;
+    }
     if (data.bye !== void 0) {
       this.end(new KernelGoneError(data.bye));
       return;
@@ -984,6 +989,23 @@ var Channel = class {
     if (reply.error !== void 0) call.reject(new KernelCallError(reply.error, reply.code));
     else call.resolve(reply.result);
   }
+  traced(trace, events, dropped) {
+    const listener = this.tracing.get(trace);
+    if (!listener) return;
+    const ack = () => {
+      if (this.tracing.has(trace)) this.post({ traceAck: trace });
+    };
+    let consumed;
+    try {
+      consumed = listener({ events, dropped });
+    } finally {
+      if (typeof consumed?.then === "function") {
+        void Promise.resolve(consumed).then(ack, ack);
+      } else {
+        ack();
+      }
+    }
+  }
   openCdp({ cdpOpen: { registration, ...request }, port }) {
     const hook = registration === void 0 ? void 0 : this.cdpHooks.get(registration);
     if (!hook || this.failure) {
@@ -1046,6 +1068,29 @@ async function serveCdpOn(channel, hook, options) {
     }
   };
 }
+async function traceOn(channel, tag, onEvents, options) {
+  if (typeof onEvents !== "function") throw new TypeError("trace: onEvents must be a function");
+  await since(channel, 15, "trace", async () => {
+  });
+  const { id, done } = channel.request({
+    op: "trace",
+    tag,
+    ...options.limit !== void 0 ? { limit: options.limit } : {}
+  });
+  channel.tracing.set(id, onEvents);
+  try {
+    await done;
+  } catch (error) {
+    channel.tracing.delete(id);
+    throw error;
+  }
+  return {
+    close: () => {
+      if (!channel.tracing.delete(id)) return;
+      void channel.call({ op: "untrace", trace: id }).catch(() => void 0);
+    }
+  };
+}
 async function dialOn(channel, { port, host }) {
   const req = { op: "dial", port, ...host ? { host } : {} };
   return dialStream(await since(channel, 4, "dial", () => channel.call(req)));
@@ -1059,6 +1104,8 @@ async function spawnOn(channel, argv, opts) {
   });
   if (opts.tag !== void 0) await since(channel, 12, "a tag per spawn", async () => {
   });
+  if (Array.isArray(opts.tag)) await since(channel, 15, "several tags per spawn", async () => {
+  });
   if (opts.cwdMissing === "error") await since(channel, 14, "cwdMissing: 'error'", async () => {
   });
   const started = Promise.withResolvers();
@@ -1120,6 +1167,10 @@ async function terminalOn(channel, argv, opts) {
   });
   if (opts.tag !== void 0) await since(channel, 12, "a tag per terminal", async () => {
   });
+  if (Array.isArray(opts.tag)) {
+    await since(channel, 15, "several tags per terminal", async () => {
+    });
+  }
   if (opts.cwdMissing === "error") await since(channel, 14, "cwdMissing: 'error'", async () => {
   });
   let listener = onData ?? null;
@@ -1279,6 +1330,7 @@ async function attachKernel(port, options = {}) {
       channel.traitsListeners.add(listener);
       return () => channel.traitsListeners.delete(listener);
     },
+    trace: (tag, onEvents, options2 = {}) => traceOn(channel, tag, onEvents, options2),
     async close(o = {}) {
       if (channel.failure) return;
       await channel.call({ op: "detach", ...o.kill ? { kill: true } : {} }).catch(() => void 0);