@ai-ecoverse/slicc-kernel 1.152.2 → 1.153.0

package/README.md

@@ -43,6 +43,7 @@ TypeScript declarations ship in `dist/types/`, and each entry (`.`, `./node`, `.
 | `onDenial` | none | `({ pid, uid, op, path, errno, tag? }) => void`: called for every permission check that fails, under `'report'` and `'enforce'` (see [Permissions](#permissions)). `createNodeKernel` takes it too |
 | `sealed` | none | `(req, { signal }) => Promise<object>`: answers requests written to [`/dev/sealed`](#devsealed). `createNodeKernel` takes it too |
 | `clipboard` | none | `{ read(stamp) => Promise<string>, write(text, stamp) => Promise<void> }`: the clipboard behind [`/dev/clipboard`](#devclipboard). `createNodeKernel` takes it too |
+| `requests` | none | `{ kinds, handle(request) }`: the page's answers to [`/dev/embedder`](#devembedder) for the processes it starts. `createNodeKernel` takes it too |
 | `approve` | none | `(req, { signal }) => 'once' \| 'always' \| 'deny'` (or a promise of one): decides a `sudo` request that needs approval (see [Sudo](#sudo)); without it, such a request is denied. `createNodeKernel` takes it too |
 | `cdp` | none | `({ runtime }) => Promise<CdpConnection>`: a browser-level DevTools connection for programs that drive a browser (see [Browser automation](#browser-automation-cdp)) |
 | `facades` | none | `{ link: { port, hook? } }`: kernel-only loopback ports whose connections a hook on the page serves, with the connecting program's identity (see [Page facades](#page-facades)) |
@@ -86,10 +87,12 @@ Starts `argv` as the session leader on a new terminal and resolves once it is ru
 
 Stops the kernel worker and every process. Pending and later calls reject.
 
-### `kernel.connect({ user, tag }?) → Promise<MessagePort>`
+### `kernel.connect({ user, tag, requests }?) → Promise<MessagePort>`
 
 Makes a port for another client of the same kernel. Hand it to a dedicated worker, or through a SharedWorker to another tab: it can be transferred any number of times. All clients share one process table, so a terminal on the page can `ps` and `kill` what a worker started.
 
+**`requests`** binds the port's [`/dev/embedder`](#devembedder) requests when the port is made: a hook (`{ kinds, handle }`, as `createKernel`'s) answers for every process the port's client starts, in the embedder's own realm, so a worker client (an agent) never answers or registers requests itself. `'attach'` lets the client that attaches to the port bring its own hook (`attachKernel(port, { requests })`, protocol 1.13); without it, an attach with `requests` fails with `EPERM`. A port with neither answers nothing: its processes' requests are `ENOTSUP`.
+
 **`user`** (a name or uid) binds the client to that user; without it the client is root. The binding is a capability: the client's `spawn`, `run` and `openTerminal` run as that user, its `ps` lists only that user's processes, and its `kill` follows POSIX permission. A bound client can't run anything as another user (`code: 'EPERM'`). A root client may pass `user` to `spawn`, `run` or `openTerminal` to run one command as any user. An unknown user rejects with `code: 'ENOENT'`. **`tag`** is the tag (see `kernel.run`) of every process the client starts; its own `spawn`, `run` and `openTerminal` cannot name another. They may add one with their own `tag` option (protocol 1.12; an older kernel answers `ENOSYS`): the process then has both tags, the connection's first, and so does everything it starts. A per-spawn tag only adds; no client, program or child can remove or replace a tag ([#1380](https://github.com/ai-ecoverse/slicc-kernel/issues/1380)).
 
 ### `kernel.users`
@@ -166,7 +169,7 @@ Page-only: neither is on an attached client (`attachKernel`, `connect()`), so no
 
 ### `attachKernel(port, options?) → Promise<KernelClient>`
 
-Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker, another tab, or Node. `options.timeoutMs` (10 s by default) bounds the handshake: a port with no kernel behind it rejects with `KernelGoneError`.
+Attaches to a kernel over a port from `kernel.connect()`, in any realm: a worker, another tab, or Node. `options.timeoutMs` (10 s by default) bounds the handshake: a port with no kernel behind it rejects with `KernelGoneError`. `options.requests` (`{ kinds, handle }`) answers the [`/dev/embedder`](#devembedder) requests of the client's processes, when the port was made with `connect({ requests: 'attach' })`.
 
 | Client | |
 |---|---|
@@ -1126,6 +1129,27 @@ A character device (`0666`), there in every `users` mode, through which programs
 - **Cost:** other files' reads and closes check one property more, and their writes nothing; an Emscripten close checks one flag.
 - **On the page,** `createKernel` relays the calls from the kernel worker and back, keeping the error's `name` and `message`.
 
+### `/dev/embedder`
+
+A character device (`0666`, every `users` mode) through which a program asks its embedder's page to do what only a page can: open a URL, capture the screen, play or record audio ([#1527](https://github.com/ai-ecoverse/slicc-kernel/issues/1527)). One open file description is one request, and the reply streams back; plain `sh` with coreutils is enough:
+
+```sh
+exec 3<>/dev/embedder
+{ printf '%s\n' '{"kind":"audio.transcribe","language":"en"}'; cat speech.wav; } >&3
+cat <&3 > text.txt
+printf '{"kind":"open.url","url":"%s"}\n' "$url" | cat > /dev/embedder
+```
+
+- **The request:** the first line is a JSON object with a `kind` (`a.b-c`: lower-case words with dots, at most 64 bytes) and options (every other key), at most 64 KiB. Everything written after it is the body, streamed to the page as it is written: a write waits while 1 MiB of it is unread by the page. A bad header fails that `write()` with `EINVAL`, and the description stays failed. The request goes to the page when its header line ends.
+- **The reply:** reads return the page's reply in order, pulled as the program reads, and its end is end-of-file. Opened `O_RDWR`, the first `read()` ends the body. Opened `O_WRONLY` (nobody reads), the last close ends the body and waits for the page to finish, so `close()` returns its error (an Emscripten program's standard output follows `/dev/clipboard`'s exit rule, so `printf … | cat > /dev/embedder` fails with `cat: write error: …`). There is no timeout: a reply may wait for a prompt or a model download; run `timeout` for one.
+- **Cancel:** a signal during a blocked read, write or `O_WRONLY` close fails it with `EINTR`, and the page's `signal` aborts with reason `interrupted`. Closing the description before the end of the reply aborts it with reason `closed`, and that is no error: it is how `hear` stops recording. A process that ends during a request aborts it too.
+- **Errors:** the hook's error gives the errno: `NotAllowedError` or `AbortError` (a dismissed prompt, a refused permission) `EACCES`; `NotSupportedError` `ENOTSUP`; `NotFoundError` `ENOENT`; anything else `EIO`. An Emscripten or WASI program first gets `<argv0>: /dev/embedder: <first line of the message>` on standard error, as for `/dev/clipboard`; a JS program has it in the error. `request.progress(line)` from the page writes `<argv0>: <line>` to standard error the same way, at most 4 lines a second.
+- **Who answers:** the hook of the client the process's session was started through, inherited by its children, forks and `exec`s: `createKernel({ requests })` (and `createNodeKernel`) for the page's own `run` and `openTerminal`, the hook a port was made with (`connect({ requests })`), or the hook its attached client brought (`connect({ requests: 'attach' })`, then `attachKernel(port, { requests })`). Nothing falls back to another client's hook, so a request from another tab's terminal never opens in this one. With no hook, or a kind the hook doesn't list, the request is `ENOTSUP` before the page sees it. A program has no way to register, replace or see a hook.
+- **The allow-list and policy:** `kinds` maps each kind the hook answers to its policy: `{}`, `{ group: 'video' }`, `{ users: ['alice', 1001] }`. With `users: 'report'` or `'enforce'` the kernel lets a request through only from root, the session's user (the user its port was connected as, or the user `run` or `openTerminal` ran as), a member of the kind's `group`, or a listed user: a process that `sudo -u`'d to someone else gets `EACCES`, and the page never sees it. The handler can still prompt or refuse by the stamp.
+- **The hook:** `handle({ kind, options, body, stamp, signal, progress })` gets the body as a `ReadableStream<Uint8Array>` and resolves with the reply: a `ReadableStream<Uint8Array>`, bytes, a string, or nothing (an empty reply). `stamp` is `{ pid, uid, gid, groups, users, kind, tag?, tags? }`: the process that sent the header line, with its effective ids and groups.
+- **Limits:** at most 16 open requests per process (`EMFILE`) and 256 MiB of request windows per kernel, 128 requests at 2 MiB each (`EAGAIN` at `open()` beyond); a closed request frees its share.
+- `/dev/clipboard` stays as it is, text only. Image clipboards are kinds on this channel (`clipboard.read`, `clipboard.write`), answered by the embedder.
+
 ### Protected paths
 
 `createKernel({ protect: { paths: ['/node_modules', '/os'] } })` makes those directories read-only for every process, root included, in every `users` mode. They hold code every later process runs, which only the embedder writes, straight to OPFS.

package/dist/driver-worker.js

@@ -43,6 +43,111 @@ function serveCdpPort(port, hook, request, onDone) {
   };
 }
 
+// src/embedder-relay.ts
+function fresh(bytes) {
+  return bytes.byteOffset === 0 && bytes.byteLength === bytes.buffer.byteLength ? bytes : bytes.slice();
+}
+function errorOf(err) {
+  const name = err?.name;
+  const message = err instanceof Error ? err.message : String(err);
+  return { name: typeof name === "string" ? name : "Error", message };
+}
+function bytesOf(reply) {
+  if (reply instanceof ReadableStream) return reply;
+  const bytes = typeof reply === "string" ? new TextEncoder().encode(reply) : reply instanceof Uint8Array ? reply : new Uint8Array(0);
+  return new ReadableStream({
+    start(controller) {
+      if (bytes.length > 0) controller.enqueue(bytes);
+      controller.close();
+    }
+  });
+}
+function bodyStream(id, state, post) {
+  return new ReadableStream(
+    {
+      pull: async (controller) => {
+        if (state.body.length === 0) post({ id, bodyPull: true });
+        while (state.body.length === 0 && !state.abort.signal.aborted) {
+          await new Promise((resolve) => {
+            state.wakeBody = resolve;
+          });
+        }
+        const next = state.body.shift();
+        if (next) controller.enqueue(next);
+        else controller.close();
+      }
+    },
+    { highWaterMark: 0 }
+  );
+}
+function embedderAnswerer(port, hookFor) {
+  const running = /* @__PURE__ */ new Map();
+  const post = (message, transfer = []) => port.postMessage({ embedderReply: message }, transfer);
+  const fail = (id, err) => {
+    running.delete(id);
+    post({ id, error: errorOf(err) });
+  };
+  const start = async (ask) => {
+    const state = { abort: new AbortController(), body: [], wakeBody: void 0 };
+    running.set(ask.id, state);
+    try {
+      const hook = hookFor(ask.client);
+      if (!hook || !Object.hasOwn(hook.kinds, ask.kind))
+        throw Object.assign(new Error(`this page does not answer ${ask.kind}`), {
+          name: "NotSupportedError"
+        });
+      const reply = await hook.handle({
+        kind: ask.kind,
+        options: ask.options,
+        body: bodyStream(ask.id, state, post),
+        stamp: ask.stamp,
+        signal: state.abort.signal,
+        progress: (line) => post({ id: ask.id, progress: String(line) })
+      });
+      state.reader = bytesOf(reply).getReader();
+      post({ id: ask.id, ready: true });
+    } catch (err) {
+      fail(ask.id, err);
+    }
+  };
+  const pull = async (id, state) => {
+    try {
+      const got = await state.reader.read();
+      if (got.done) {
+        running.delete(id);
+        post({ id, done: true });
+        return;
+      }
+      const bytes = fresh(got.value);
+      post({ id, chunk: bytes }, [bytes.buffer]);
+    } catch (err) {
+      fail(id, err);
+    }
+  };
+  const step = (message) => {
+    const state = running.get(message.id);
+    if (!state) return;
+    if ("body" in message) state.body.push(message.body);
+    else if ("bodyEnd" in message) state.body.push(null);
+    else if ("pull" in message) void pull(message.id, state);
+    else {
+      running.delete(message.id);
+      state.abort.abort(new DOMException(message.cancel, "AbortError"));
+      void state.reader?.cancel(message.cancel).catch(() => void 0);
+    }
+    state.wakeBody?.();
+  };
+  return {
+    handle(data) {
+      const message = data?.embedder;
+      if (!message) return false;
+      if ("ask" in message) void start(message.ask);
+      else step(message);
+      return true;
+    }
+  };
+}
+
 // src/facade/port.ts
 function serveFacadePort(port, hook, { name, stamp }, onDone) {
   let conn;
@@ -93,6 +198,27 @@ function serveFacadePort(port, hook, { name, stamp }, onDone) {
   };
 }
 
+// src/kernel/pipe.ts
+var PIPE_CAPACITY = 64 * 1024;
+
+// src/kernel/embedder.ts
+var EMBEDDER_HEADER_MAX = 64 * 1024;
+var EMBEDDER_WINDOW = 1024 * 1024;
+var EMBEDDER_BUDGET = 256 * 1024 * 1024;
+function kindsOf(hook) {
+  if (typeof hook?.handle !== "function" || typeof hook.kinds !== "object" || !hook.kinds)
+    throw new TypeError("requests needs kinds and handle");
+  return Object.fromEntries(
+    Object.entries(hook.kinds).map(([kind, policy]) => [
+      kind,
+      {
+        ...policy?.group !== void 0 ? { group: String(policy.group) } : {},
+        ...policy?.users ? { users: [...policy.users] } : {}
+      }
+    ])
+  );
+}
+
 // src/kernel/net/ws-queue.ts
 var INBOUND_LIMIT = 16 * 1024 * 1024;
 var MESSAGE_LIMIT = 64 * 1024 * 1024;
@@ -421,7 +547,7 @@ var KERNEL_SIGINFO = Object.freeze({
 });
 
 // src/bytes.ts
-function fresh(bytes, start = 0, end = bytes.length) {
+function fresh2(bytes, start = 0, end = bytes.length) {
   return new Uint8Array(bytes.subarray(start, end));
 }
 
@@ -478,7 +604,7 @@ function dialStream(port, onClose) {
   const writable = new WritableStream({
     write(chunk) {
       if (closed) throw errorWithCode("EPIPE: the connection is closed");
-      const copy = fresh(chunk);
+      const copy = fresh2(chunk);
       const acked = new Promise((resolve, reject) => acks.push({ resolve, reject }));
       port.postMessage(copy, [copy.buffer]);
       return acked;
@@ -665,7 +791,7 @@ async function loopbackFetch(dial, input, options) {
 }
 
 // src/client/protocol.ts
-var PROTOCOL = [1, 12];
+var PROTOCOL = [1, 13];
 function locksOf(scope = globalThis) {
   const locks = scope.navigator?.locks;
   return typeof locks?.request === "function" ? locks : void 0;
@@ -693,7 +819,7 @@ var KernelCallError = class extends Error {
   }
 };
 var encoder2 = new TextEncoder();
-var bytesOf = (data) => typeof data === "string" ? encoder2.encode(data) : data;
+var bytesOf2 = (data) => typeof data === "string" ? encoder2.encode(data) : data;
 var UNAVAILABLE = {
   manualRedirects: false,
   encodedBodies: false,
@@ -777,7 +903,9 @@ var Channel = class {
   call(call) {
     return this.request(call).done;
   }
+  requests;
   receive(data) {
+    if (this.requests?.handle(data)) return;
     if (data.traits && data.id === void 0) {
       if (this.remote) this.remote.traits = data.traits;
       for (const listener of [...this.traitsListeners]) listener(data.traits);
@@ -900,7 +1028,7 @@ async function spawnOn(channel, argv, opts) {
   });
   const started = Promise.withResolvers();
   let endedBy;
-  const options = { ...rest, ...stdin !== void 0 ? { stdin: bytesOf(stdin) } : {} };
+  const options = { ...rest, ...stdin !== void 0 ? { stdin: bytesOf2(stdin) } : {} };
   const { done } = channel.request(
     { op: "spawn", argv, options },
     {
@@ -983,7 +1111,7 @@ async function terminalOn(channel, argv, opts) {
       listener = next;
       if (next) for (const bytes of backlog.splice(0)) next(bytes);
     },
-    write: (data) => send({ action: "write", bytes: bytesOf(data) }),
+    write: (data) => send({ action: "write", bytes: bytesOf2(data) }),
     resize: (cols, rows) => send({ action: "resize", cols, rows }),
     signal: (name) => send({ action: "signal", signal: name }),
     close: () => send({ action: "close" })
@@ -1045,8 +1173,10 @@ function since(channel, minor, what, run) {
   }
   return run();
 }
-async function greet(channel, lock, timeoutMs) {
-  channel.post({ hello: { protocol: PROTOCOL, ...lock ? { lock } : {} } });
+async function greet(channel, lock, timeoutMs, requests) {
+  channel.post({
+    hello: { protocol: PROTOCOL, ...lock ? { lock } : {}, ...requests ? { requests } : {} }
+  });
   const timer = setTimeout(
     () => channel.end(
       new KernelGoneError(`no slicc-kernel answered on this port within ${timeoutMs} ms`)
@@ -1071,7 +1201,14 @@ async function attachKernel(port, options = {}) {
   const locks = options.locks === null ? void 0 : options.locks ?? locksOf();
   const own = await holdLock(locks);
   const channel = new Channel(port, own.release);
-  const hello = await greet(channel, own.name, options.timeoutMs ?? 1e4);
+  const requests = options.requests;
+  if (requests) channel.requests = embedderAnswerer(port, () => requests);
+  const hello = await greet(
+    channel,
+    own.name,
+    options.timeoutMs ?? 1e4,
+    requests ? kindsOf(requests) : void 0
+  );
   channel.protocol = hello.protocol;
   if (hello.lock && locks) void locks.request(hello.lock, () => channel.end(new KernelGoneError()));
   const transport = new RemoteTransport(
@@ -1234,7 +1371,7 @@ function serveFilesystem(port, handlers, capabilities = {}, options = {}) {
       (result) => {
         if (result instanceof Uint8Array) {
           const whole = result.byteOffset === 0 && result.byteLength === result.buffer.byteLength;
-          const bytes = options.owned && whole ? result : fresh(result);
+          const bytes = options.owned && whole ? result : fresh2(result);
           reply({ id: data.id, result: bytes }, [bytes.buffer]);
         } else reply({ id: data.id, result: result ?? null });
       },