@ai-ecoverse/slicc-capture 1.2.0 → 1.3.0

package/README.md

@@ -105,7 +105,7 @@ As `sccapture`, the messages start with `sccapture:`. A line of progress from th
 
 ### The request
 
-The commands run in a kernel worker, which has no `getDisplayMedia`, so they ask seven's page over [`/dev/embedder`](https://github.com/ai-ecoverse/slicc-kernel#devembedder) (slicc-kernel 1.153.0, [#1527](https://github.com/ai-ecoverse/slicc-kernel/issues/1527)): one open file description per request, a JSON header line `{ kind, …options }`, then the body, and the reply read to end-of-file. The page side, the prompt, `getDisplayMedia`, the frame, the crop and the encoding, is a page module that slicc-bios registers for `capture.screen` and wraps with its own prompt, user activation, `selfBrowserSurface: 'exclude'` and a check of the picked surface against protected origins; it answers this contract:
+The commands run in a kernel worker, which has no `getDisplayMedia`, so they ask seven's page over [`/dev/embedder`](https://github.com/ai-ecoverse/slicc-kernel#devembedder) (slicc-kernel 1.153.0, [#1527](https://github.com/ai-ecoverse/slicc-kernel/issues/1527)): one open file description per request, a JSON header line `{ kind, …options }`, then the body, and the reply read to end-of-file. The page side, the prompt, `getDisplayMedia`, the frame, the crop and the encoding, is [a page module](#the-page-module) that slicc-bios registers for `capture.screen` and wraps with its own prompt, user activation, `selfBrowserSurface: 'exclude'` and a check of the picked surface against protected origins; it answers this contract:
 
 - **`capture.screen`:** the header is plain JSON, with no body, and the reply is the encoded image's bytes, nothing else. The command opens the device read-write (`exec 3<>/dev/embedder`), writes the header from a child `sh` that stays until the reply is read (so the page's progress reaches stderr), and reads the reply with a `cat <&3` that it starts in the background, so that a signal to the command can stop it.
 
@@ -122,6 +122,29 @@ The commands run in a kernel worker, which has no `getDisplayMedia`, so they ask
 For example, `screencapture -C -T 2 -R 0,0,800,600 shot.jpg` sends `{"kind":"capture.screen","type":"image/jpeg","quality":0.92,"cursor":"always","delay":2,"rect":{"x":0,"y":0,"width":800,"height":600}}`. The page rejects with a `NotAllowedError` or `AbortError` for **Cancel**, a closed picker, a refused permission or a protected surface (`EACCES`); a `NotSupportedError` without `getDisplayMedia` (`ENOTSUP`); and any other error, such as a `TimeoutError` `timed out waiting for the picker` after 5 minutes, with the message the command prints (`EIO`). It aborts the capture when its `signal` does.
 - **`clipboard.write`:** `-c` sends `{"kind":"clipboard.write","type":"image/png"}` and the PNG as the body, and reads the reply, which is empty, to end-of-file. slicc-bios answers it; with no page that does, the header write fails with `ENOTSUP`. The device is opened read-write and the reply read only after the whole body is written, so a page that answers without reading the body ([slicc-kernel#1536](https://github.com/ai-ecoverse/slicc-kernel/issues/1536)) is the only way a body over 1 MiB could block; the embedder's handler reads it.
 
+### The page module
+
+`@ai-ecoverse/slicc-capture/page` is that page module: one plain ES module (`page/index.mjs`), with no build step and no dependencies, which runs in Chrome and loads in Node. It answers `capture.screen` and nothing else; `clipboard.write` is the embedder's own.
+
+```js
+import capture, { create, handle, kinds } from '@ai-ecoverse/slicc-capture/page';
+
+createKernel({
+  requests: {
+    kinds: capture.kinds,
+    handle: (request) => capture.handle({ ...request, display: guardedGetDisplayMedia }),
+  },
+});
+```
+
+- **The exports:** `kinds` is `{ 'capture.screen': {} }`, with no policy; `handle(request)` is a [`/dev/embedder` hook](https://github.com/ai-ecoverse/slicc-kernel#devembedder)'s `handle`; the default export is `{ kinds, handle }`. `create({ ImageCapture, OffscreenCanvas, document, timeout })` returns another `{ kinds, handle }` that uses those instead of the page's globals, and `timeout` (in ms, 5 minutes by default) for the picker; the tests inject fakes through it.
+- **`display` is required:** the request is the kernel's (`{ kind, options, body, stamp, signal, progress }`) plus `display(constraints)`, the embedder's `getDisplayMedia`, which resolves to a `MediaStream`. The module never calls `navigator.mediaDevices.getDisplayMedia` itself; without `display`, or without `OffscreenCanvas` and a way to grab a frame (`ImageCapture`, else `document`), it rejects with a `NotSupportedError` `screen capture is not supported in this browser` (`ENOTSUP`).
+- **The header:** `type` (PNG by default), `quality`, `cursor`, `surface`, `delay` and `rect` as in [the table above](#the-request); other keys are ignored. A `type` it doesn't encode rejects with a `NotSupportedError` `unsupported type: <type>`, and a value of the wrong shape with a `TypeError` `invalid <field>: <json>` (`EIO`), before `display` is called.
+- **The capture:** it calls `display({ video: { cursor, displaySurface }, audio: false })` (each field only when the header has it: `cursor` as is, `surface` as `displaySurface`) after `progress('waiting for the picker…')`. No stream within the timeout rejects with a `TimeoutError` `timed out waiting for the picker`, one without a video track with a `NotFoundError` `the capture has no video track` (`ENOENT`), and one whose track has ended, or ends before the frame (the user's **Stop sharing**), with an `AbortError` `the capture was stopped` (`EACCES`). With a `delay`, it says `waiting <delay> s…` and waits that long. It grabs one frame with `ImageCapture.grabFrame()`, else from a muted `<video>` once it has data, crops it to `rect` (outside the frame: a `RangeError` `-R: rectangle outside the capture`), draws it on an `OffscreenCanvas` and encodes it with `convertToBlob({ type, quality })`. A browser that encodes another type than asked rejects with a `NotSupportedError` `<type> is not supported by this browser`. It resolves with the bytes, a `Uint8Array`.
+- **Stopping:** every track of the stream is stopped when `handle` settles, whether it resolved, failed or was aborted, and so is a stream that `display` delivers after the timeout or an abort. When `signal` aborts, `handle` rejects with an `AbortError` carrying the reason's message (`interrupted`, `closed`, or the embedder's own).
+
+**What slicc-bios does around it:** bios registers its own `kinds` table, of which the module's is a subset, and ignores the module's policies. It owns the prompt (`screencapture wants to capture your screen`, **Choose…**, **Cancel**), the user activation `getDisplayMedia` needs, the session tag and the protected origins, and calls `handle` only after its checks pass. Its `display` is a guarded `getDisplayMedia`: it adds `selfBrowserSurface: 'exclude'`, checks the picked surface's capture handle against the protected origins, and rejects with a `NotAllowedError` on a match. Its `signal` also aborts when its prompt is withdrawn.
+
 ### Options
 
 Every option of macOS's `screencapture`, and what it does in SLICC and with `sccapture` on each host. On macOS, `sccapture` passes the options through to `screencapture` unchanged, except the ones the macOS column names. "No-op" means the option is accepted and changes nothing. "Not supported" means `<cmd>: <opt>: not supported in SLICC` (or `on Linux`, `on Windows`, `on macOS`) on stderr, exit 2, before anything is captured; "not with `<tool>`" means `sccapture: <opt>: not supported with <tool>`, exit 2, when no tool on `PATH` supports every option given (it names the first one tried).
@@ -225,6 +248,7 @@ npm run test:unit
 - `npm install` also installs the git hooks into this worktree's own `node_modules/` through slicc-shared-web's `slicc-install-hooks` and the committed `.githooks/`.
 - `npm run lint`: Biome, the no-comment check and the other [slicc-shared-web](https://github.com/ai-ecoverse/slicc-shared-web) checks, then `shellcheck -s sh` and `dash -n` on `install.sh` and `bin/*`.
 - `npm test`: the integration tests in `test/integration/`. `commands.test.mjs` runs `screencapture` and `sccapture` on `@ai-ecoverse/slicc-kernel/node` (the version `engines.slicc-kernel` names, pinned) with wasm-bash and wasm-coreutils, and a fake `capture.screen` and `clipboard.write` handler (`createNodeKernel({ requests })`): every option's header, the bytes written unchanged, the errnos and their messages, `-c`, `-v`, and slicc-kernel 1.152.2, which has no `/dev/embedder`. `test/integration/repo.test.mjs` checks the workflow, the release config, the pinned devDependencies and what `npm pack` ships. The host tests run `host/` against fake `screencapture`, `grim`, `slurp`, `gnome-screenshot`, `import`, `scrot`, `wl-copy`, `xclip`, `xdg-open`, `powershell.exe` and `pwsh.exe` on a temporary `PATH`, and `--upgrade` against installers served from a local HTTP server, with the platform forced, so each branch is checked on any machine; only CI touches a real screen. `install.test.mjs` runs `install.sh` on a tarball packed from the tree, with the Node on `PATH` and with a Node download from a fake `nodejs.org`, and `--upgrade` against a release served from a local directory; `install.ps1` is checked statically, and run when `pwsh` is on `PATH`.
+- `page.test.mjs` registers [the page module](#the-page-module) through `createNodeKernel({ requests })`, with a fake `display` and fake `ImageCapture`, `OffscreenCanvas` and `<video>` injected through `create`, and runs the real `screencapture`: the constraints, the crop, the type and quality, the delay and its progress, an interrupt in the picker and in the delay, **Stop sharing**, the picker's timeout, the errnos and their messages end to end, and that every track is stopped. `browser.test.mjs` runs the module in Chrome, through slicc-shared-web's harness, with a `display` that returns a canvas's `captureStream()`: PNG, JPEG and WebP, the crop, the `<video>` fallback, an abort and the stopped track. It runs on CI only, and is skipped without `CI`.
 - `npm run test:unit`: the unit tests in the gitignored `test/unit/`, with coverage in `coverage/unit/lcov.info`. `npm run test:sh` runs the `test/unit/*.sh.mjs` ones, which trace `bin/` with `set -x` in the node kernel, and writes `coverage/sh/lcov.info`. The pre-commit hook wants 100% coverage of the changed lines of both, and runs `diff-cover` through `uvx`, so install [uv](https://docs.astral.sh/uv/) once.
 
 CI is `.github/workflows/ci.yml`, the only workflow, which runs [slicc-shared-web's pipeline](https://github.com/ai-ecoverse/slicc-shared-web#pipeline) at the devDependency's version: `npm run lint`, then `npm test`, then on a pull request the Claude and Codex reviews. The merge queue requires `ci / integration` and `ci / review`. On a push to `main`, semantic-release releases and publishes to npm through trusted publishing, with provenance from `ci.yml`, and attaches the installers.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ai-ecoverse/slicc-capture",
-  "version": "1.2.0",
+  "version": "1.3.0",
   "description": "screencapture for SLICC, and sccapture for SLICC and any host",
   "type": "module",
   "license": "Apache-2.0",
@@ -15,6 +15,10 @@
   "bin": {
     "sccapture": "host/sccapture.mjs"
   },
+  "exports": {
+    "./page": "./page/index.mjs",
+    "./package.json": "./package.json"
+  },
   "fledgling": {
     "provider": "github",
     "workflow": "ci.yml",
@@ -35,12 +39,14 @@
       }
     },
     "unitSources": [
-      "host"
+      "host",
+      "page"
     ]
   },
   "files": [
     "bin",
-    "host"
+    "host",
+    "page"
   ],
   "scripts": {
     "prepare": "slicc-install-hooks",
@@ -55,8 +61,9 @@
     "release": "semantic-release"
   },
   "devDependencies": {
+    "@ai-ecoverse/slicc-cdp": "1.40.0",
     "@ai-ecoverse/slicc-kernel": "1.153.0",
-    "@ai-ecoverse/slicc-shared-web": "2.11.3",
+    "@ai-ecoverse/slicc-shared-web": "2.12.0",
     "@ai-ecoverse/wasm-bash": "5.3.0-23",
     "@ai-ecoverse/wasm-coreutils": "9.12.0-15",
     "@biomejs/biome": "2.5.15",

package/page/index.mjs

@@ -0,0 +1,199 @@
+const types = ['image/png', 'image/jpeg', 'image/webp'];
+const cursors = ['always', 'motion', 'never'];
+const surfaces = ['monitor', 'window', 'browser'];
+const unsupported = 'screen capture is not supported in this browser';
+
+export const kinds = { 'capture.screen': {} };
+
+function fail(name, message) {
+  return new DOMException(message, name);
+}
+
+function aborted(signal) {
+  const reason = signal.reason;
+  if (reason?.name === 'AbortError') return reason;
+  return fail('AbortError', String(reason?.message ?? reason));
+}
+
+function race(promise, signal) {
+  return new Promise((resolve, reject) => {
+    const stop = () => reject(aborted(signal));
+    if (signal.aborted) stop();
+    signal.addEventListener('abort', stop, { once: true });
+    Promise.resolve(promise)
+      .then(resolve, reject)
+      .finally(() => signal.removeEventListener('abort', stop));
+  });
+}
+
+function sleep(ms, signal) {
+  let timer;
+  const wait = new Promise((resolve) => {
+    timer = setTimeout(resolve, ms);
+  });
+  return race(wait, signal).finally(() => clearTimeout(timer));
+}
+
+function stopAll(stream) {
+  for (const track of stream?.getTracks?.() ?? []) track.stop();
+}
+
+function invalid(field, value) {
+  return new TypeError(`invalid ${field}: ${JSON.stringify(value)}`);
+}
+
+const whole = (value, least) => Number.isInteger(value) && value >= least;
+
+function parse(options = {}) {
+  const { type = 'image/png', quality, cursor, surface, delay, rect } = options;
+  if (!types.includes(type)) throw fail('NotSupportedError', `unsupported type: ${type}`);
+  if (quality !== undefined && !(typeof quality === 'number' && quality >= 0 && quality <= 1)) {
+    throw invalid('quality', quality);
+  }
+  if (cursor !== undefined && !cursors.includes(cursor)) throw invalid('cursor', cursor);
+  if (surface !== undefined && !surfaces.includes(surface)) throw invalid('surface', surface);
+  if (delay !== undefined && !(Number.isFinite(delay) && delay >= 0)) throw invalid('delay', delay);
+  if (
+    rect !== undefined &&
+    !(whole(rect?.x, 0) && whole(rect?.y, 0) && whole(rect?.width, 1) && whole(rect?.height, 1))
+  ) {
+    throw invalid('rect', rect);
+  }
+  const video = {};
+  if (cursor !== undefined) video.cursor = cursor;
+  if (surface !== undefined) video.displaySurface = surface;
+  return {
+    constraints: { video, audio: false },
+    type,
+    quality,
+    delay: delay ?? 0,
+    rect,
+  };
+}
+
+async function pick(display, constraints, signal, timeout) {
+  const pending = Promise.resolve().then(() => display(constraints));
+  let timer;
+  const late = new Promise((_, reject) => {
+    timer = setTimeout(
+      () => reject(fail('TimeoutError', 'timed out waiting for the picker')),
+      timeout
+    );
+  });
+  try {
+    return await race(Promise.race([pending, late]), signal);
+  } catch (error) {
+    pending.then(stopAll, () => {});
+    throw error;
+  } finally {
+    clearTimeout(timer);
+  }
+}
+
+async function grab(stream, track, env, signal) {
+  const ImageCapture = env('ImageCapture');
+  if (ImageCapture) {
+    const bitmap = await race(new ImageCapture(track).grabFrame(), signal);
+    return {
+      source: bitmap,
+      width: bitmap.width,
+      height: bitmap.height,
+      close: () => bitmap.close?.(),
+    };
+  }
+  const video = env('document').createElement('video');
+  const close = () => {
+    video.pause();
+    video.srcObject = null;
+  };
+  const ready = new Promise((resolve, reject) => {
+    video.onloadedmetadata = () => video.play().catch(reject);
+    video.onloadeddata = resolve;
+    video.onerror = () => reject(new Error('the capture could not be played'));
+  });
+  video.muted = true;
+  video.playsInline = true;
+  video.srcObject = stream;
+  try {
+    await race(ready, signal);
+  } catch (error) {
+    close();
+    throw error;
+  }
+  return { source: video, width: video.videoWidth, height: video.videoHeight, close };
+}
+
+async function encode(frame, request, env, signal) {
+  const { type, quality, rect } = request;
+  const { x, y, width, height } = rect ?? { x: 0, y: 0, width: frame.width, height: frame.height };
+  if (!(frame.width > 0 && frame.height > 0)) throw new Error('the capture has no frame');
+  if (x + width > frame.width || y + height > frame.height) {
+    throw new RangeError('-R: rectangle outside the capture');
+  }
+  const canvas = new (env('OffscreenCanvas'))(width, height);
+  canvas.getContext('2d').drawImage(frame.source, x, y, width, height, 0, 0, width, height);
+  const blob = await race(
+    canvas.convertToBlob(quality === undefined ? { type } : { type, quality }),
+    signal
+  );
+  if (blob.type !== type)
+    throw fail('NotSupportedError', `${type} is not supported by this browser`);
+  return new Uint8Array(await race(blob.arrayBuffer(), signal));
+}
+
+export function create(overrides = {}) {
+  const env = (name) => (name in overrides ? overrides[name] : globalThis[name]);
+  const timeout = overrides.timeout ?? 300000;
+
+  async function handle({
+    options,
+    signal = new AbortController().signal,
+    progress = () => {},
+    display,
+  }) {
+    const request = parse(options);
+    if (typeof display !== 'function') throw fail('NotSupportedError', unsupported);
+    if (!env('OffscreenCanvas') || !(env('ImageCapture') || env('document'))) {
+      throw fail('NotSupportedError', unsupported);
+    }
+    const local = new AbortController();
+    const forward = () => local.abort(signal.reason);
+    if (signal.aborted) forward();
+    signal.addEventListener('abort', forward, { once: true });
+    let stream;
+    let frame;
+    try {
+      if (local.signal.aborted) throw aborted(local.signal);
+      progress('waiting for the picker…');
+      stream = await pick(display, request.constraints, local.signal, timeout);
+      const track = stream?.getVideoTracks?.()[0];
+      if (!track) throw fail('NotFoundError', 'the capture has no video track');
+      if (track.readyState === 'ended') throw fail('AbortError', 'the capture was stopped');
+      track.addEventListener(
+        'ended',
+        () => local.abort(fail('AbortError', 'the capture was stopped')),
+        {
+          once: true,
+        }
+      );
+      if (request.delay > 0) {
+        progress(`waiting ${request.delay} s…`);
+        await sleep(request.delay * 1000, local.signal);
+      }
+      frame = await grab(stream, track, env, local.signal);
+      return await encode(frame, request, env, local.signal);
+    } finally {
+      signal.removeEventListener('abort', forward);
+      frame?.close();
+      stopAll(stream);
+    }
+  }
+
+  return { kinds, handle };
+}
+
+const page = create();
+
+export const handle = page.handle;
+
+export default page;