@ai-ecoverse/slicc-shared-web 1.10.0 → 1.10.2

package/README.md

@@ -31,7 +31,7 @@ The hooks expect two scripts in each repo: `test:unit`, which writes `coverage/u
 
 ## Integration-test harness
 
-`harness/` drives playwright-core's Chromium over raw CDP, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage and the CPU profiler switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: one CPU profile per target and snapshot, a screenshot per tab and a `console.log` of every target. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`, the 15 frames with the most self time in the repo's own scripts.
+`harness/` drives playwright-core's Chromium over raw CDP on `--remote-debugging-pipe`, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage and the CPU profiler switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: one CPU profile per target and snapshot, a screenshot per tab and a `console.log` of every target. Lines logged before the first test, such as an extension service worker's at startup, go into the first test's `console.log`. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`, the 15 frames with the most self time in the repo's own scripts.
 
 `playwright-core` and `monocart-coverage-reports` are peer dependencies; pin both in the repo's `devDependencies`. A repo describes what to serve and launches the browser once per test file:
 
@@ -60,12 +60,12 @@ test('boots', async (t) => {
 | `coverage` | URL prefixes whose `.js`/`.mjs` scripts count for coverage. `node_modules/` never does. A `.map` next to a file is used | `['/']` |
 | `exits` | `{ urlPath: [regex, ...] }`: a breakpoint on the first line matching each pattern in that worker script, where the harness takes the worker's coverage and profile before it closes itself | none |
 | `intercept` | Remote URL prefixes answered from a disk cache in `node_modules/.cache/`. Registry documents (anything on a `registry.*` host but a `/-/….tgz` tarball: packuments, abbreviated or full, and dist-tags) are fetched fresh once per test process and only served from the disk cache when the registry can't be reached (a `4xx` is passed on) | none |
-| `extensions` | Unpacked extension directories to load. Their scripts are covered and profiled like the repo's own, and tests then run in the default browser context, closing their tabs when they end, because Chrome does not run extensions in the per-test contexts | none |
+| `extensions` | Unpacked extension directories to load. They are loaded with `Extensions.loadUnpacked` (behind `--enable-unsafe-extension-debugging`) once the harness is attached, so their service workers start paused like every other target. Their scripts are covered and profiled like the repo's own, and tests then run in the default browser context, closing their tabs when they end, because Chrome does not run extensions in the per-test contexts | none |
 | `args` | Extra Chromium flags, such as `--host-resolver-rules` or `--ignore-certificate-errors` | none |
 | `timeout` | How long `page.until` polls and `page.evaluate` waits, in ms | `30000` |
 | `stallAfter` | How long one evaluation may run before the harness probes the test's targets, in ms | `30000` |
 
-`chrome.page(t)` opens a tab and returns the page API: `goto(path)`, `reload()`, `evaluate(fn, ...args)`, `until(fn, ...args)`, `press(key, ...modifiers)` (a key name like `Enter` or `F5`, or a single character typed with its US-layout key code), `type(text)`, `insert(text)`, `enter()`, `init(fn)` (runs in every new document), `expose(name, handler)` (a binding the page calls with a JSON string), `screenshot(file)`, `tab()` (another tab in the same context), `close()`, `send(method, params, ms)` (a raw CDP call in the tab's session), `errors`, `responses` and `dir` (the test's artifact directory). `chrome.pid` is the browser's process id. `goto` and `reload` wait until the new document is complete and no longer `about:blank`, and retry the navigation once if that never happens. `until` polls until the function returns a truthy value (not only strict `true`) and resolves with that value; the timeout error names the last result. `within(ms, fn, ...args)` is `until` with its own deadline, for a stage that should take a known time. Both stop at the deadline even if one evaluation never answers. Every CDP call has a deadline: `connect(url).send(method, params, sessionId, ms)` rejects with `<method>: no answer in <ms> ms` (60 s unless given), so a stuck page fails its test instead of hanging the run. A crash fails it at once instead:
+`chrome.page(t)` opens a tab and returns the page API: `goto(path)`, `reload()`, `evaluate(fn, ...args)`, `until(fn, ...args)`, `press(key, ...modifiers)` (a key name like `Enter` or `F5`, or a single character typed with its US-layout key code), `type(text)`, `insert(text)`, `enter()`, `init(fn)` (runs in every new document), `expose(name, handler)` (a binding the page calls with a JSON string), `screenshot(file)`, `tab()` (another tab in the same context), `close()`, `send(method, params, ms)` (a raw CDP call in the tab's session), `errors`, `responses` and `dir` (the test's artifact directory). `chrome.pid` is the browser's process id and `chrome.profile` its `slicc-harness-*` profile directory in `$TMPDIR`. `goto` and `reload` wait until the new document is complete and no longer `about:blank`, and retry the navigation once if that never happens. `until` polls until the function returns a truthy value (not only strict `true`) and resolves with that value; the timeout error names the last result. `within(ms, fn, ...args)` is `until` with its own deadline, for a stage that should take a known time. Both stop at the deadline even if one evaluation never answers. Every CDP call has a deadline: `connect(url).send(method, params, sessionId, ms)` rejects with `<method>: no answer in <ms> ms` (60 s unless given), so a stuck page fails its test instead of hanging the run. A crash fails it at once instead:
 - when the browser dies, every pending and later call rejects with `<method>: browser exited with <signal>` (or `with code <n>`);
 - when a renderer crashes, every pending and later call to that target, and to the dedicated workers it started, rejects with `<method>: page renderer crashed (<status>, code <n>) at <url>` (or `worker`, `shared worker`, `service worker`), and `console.log` gets the same line;
 - `until`, `within`, `goto` and `reload` stop retrying and rethrow it;
@@ -96,6 +96,13 @@ Why it works the way it does:
 - A worker blocked in `Atomics.wait`, or one that is already gone, may never answer; snapshots give up after 3 s, and closing a context or the browser after 10 s. A call to a target that detaches or crashes fails instead of waiting forever, and a new tab that never attaches fails after 10 s.
 - A WebSocket that has closed takes new messages without an error, and a crashed target never answers, so after a crash every call used to wait out its own deadline: a browser that died 4 s in showed up as `no answer` 15 minutes later. The harness now remembers the crash or the exit and fails each call with it.
 
+## Cleanup
+
+`close()` stops Chrome and removes its profile. It asks Chrome to close, waits up to 10 s for the exit, sends `SIGKILL`, waits up to 5 s more and then moves on with a warning, so a browser that ignores `Browser.close` or a missed exit never hangs teardown. A test process that ends without `close()` still leaves nothing running:
+- on `exit` and on `SIGINT`, `SIGTERM` or `SIGHUP`, the harness kills every browser it started in that process with `SIGKILL` and removes its profile and crash dump directory, then re-raises the signal so the process ends the way it would have;
+- when the test process dies of `SIGKILL`, no handler runs. A watchdog, a `sh` that reads a pipe from the test process, then gets end of file and kills Chrome with `SIGKILL` at once. Chrome also shuts itself down when the CDP pipe closes, but that can take seconds. Only the profile stays behind;
+- the first `launch()` in a process removes every `slicc-harness-*` directory in `$TMPDIR` that is older than a day, or older than a minute and no running process uses as its `--user-data-dir`. It never kills a process.
+
 ## Crash dumps
 
 Chrome for Testing runs crashpad by default and ignores `--crash-dumps-dir` and `--enable-crash-reporter`: on Linux it writes every dump to `~/.config/google-chrome-for-testing/Crash Reports/pending/`. `launch` points it at a fresh directory per browser through `BREAKPAD_DUMP_LOCATION`, waits up to 10 s after a crash for crashpad to finish writing, copies the new dumps into the test's artifact directory and deletes the directory on `close()`. Nothing is uploaded.

package/harness/cdp.mjs

@@ -17,12 +17,42 @@ function grace(ms) {
   return new Promise((resolve) => setTimeout(resolve, ms, 'socket closed').unref());
 }
 
-export async function connect(url, { exited, tail = () => [] } = {}) {
+async function websocket(url) {
   const socket = new WebSocket(url);
   await new Promise((resolve, reject) => {
     socket.onopen = resolve;
     socket.onerror = reject;
   });
+  return {
+    send: (text) => socket.send(text),
+    close: () => socket.close(),
+    listen(message, closed) {
+      socket.onmessage = ({ data }) => message(data);
+      socket.onclose = closed;
+    },
+  };
+}
+
+export function pipe(input, output) {
+  input.on('error', () => null);
+  return {
+    send: (text) => input.write(`${text}\0`),
+    close: () => input.end(),
+    listen(message, closed) {
+      let partial = '';
+      output.setEncoding('utf8');
+      output.on('data', (chunk) => {
+        const parts = `${partial}${chunk}`.split('\0');
+        partial = parts.pop();
+        for (const part of parts) message(part);
+      });
+      output.once('close', closed);
+    },
+  };
+}
+
+export async function connect(url, { exited, tail = () => [] } = {}) {
+  const socket = typeof url?.listen === 'function' ? url : await websocket(url);
   const pending = new Map();
   const listeners = new Set();
   const targets = new Map();
@@ -79,16 +109,18 @@ export async function connect(url, { exited, tail = () => [] } = {}) {
     gone = reason;
     drop(() => true, told(reason), fatal);
   };
-  socket.onmessage = ({ data }) => {
-    const message = JSON.parse(data);
-    track(message);
-    const call = pending.get(message.id);
-    pending.delete(message.id);
-    if (!call) notify(message);
-    else if (message.error) call.reject(new Error(`${call.method}: ${message.error.message}`));
-    else call.resolve(message.result);
-  };
-  socket.onclose = async () => die(await Promise.race([exited ?? 'socket closed', grace(2000)]));
+  socket.listen(
+    (data) => {
+      const message = JSON.parse(data);
+      track(message);
+      const call = pending.get(message.id);
+      pending.delete(message.id);
+      if (!call) notify(message);
+      else if (message.error) call.reject(new Error(`${call.method}: ${message.error.message}`));
+      else call.resolve(message.result);
+    },
+    async () => die(await Promise.race([exited ?? 'socket closed', grace(2000)]))
+  );
   void exited?.then(die);
   return {
     send: (method, params = {}, sessionId, ms = SEND_TIMEOUT) =>

package/harness/chrome.mjs

@@ -1,13 +1,15 @@
-import { spawn } from 'node:child_process';
-import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
+import { execFile, spawn } from 'node:child_process';
+import { rmSync } from 'node:fs';
+import { lstat, mkdir, mkdtemp, readdir, rm, writeFile } from 'node:fs/promises';
 import { tmpdir } from 'node:os';
 import { basename, join } from 'node:path';
 import { env as inherited } from 'node:process';
+import { promisify } from 'node:util';
 import { chromium } from 'playwright-core';
 import { cdn } from './cdn.mjs';
-import { CRASHED, connect } from './cdp.mjs';
+import { CRASHED, connect, pipe } from './cdp.mjs';
 import { crashpad } from './dumps.mjs';
-import { extensionSource, unpacked } from './extensions.mjs';
+import { extensionSource, install, unpacked } from './extensions.mjs';
 import { page } from './page.mjs';
 import { breakpoints, recorder } from './recorder.mjs';
 import { serve } from './server.mjs';
@@ -16,6 +18,7 @@ import { artifacts, bounded, ignore, sleep, slug, TIMED_OUT } from './util.mjs';
 export const flags = [
   '--headless',
   '--remote-debugging-port=0',
+  '--remote-debugging-pipe',
   '--no-first-run',
   '--no-default-browser-check',
   '--no-sandbox',
@@ -32,9 +35,7 @@ export const flags = [
 export function commandLine(profile, extensions = [], extra = []) {
   const loaded = extensions.length > 0;
   const base = loaded ? flags.filter((flag) => flag !== '--disable-extensions') : flags;
-  const load = loaded
-    ? [`--disable-extensions-except=${extensions}`, `--load-extension=${extensions}`]
-    : [];
+  const load = loaded ? ['--enable-unsafe-extension-debugging'] : [];
   return [...base, ...load, ...extra, `--user-data-dir=${profile}`, 'about:blank'];
 }
 
@@ -71,7 +72,10 @@ export async function start(
   executable = chromium.executablePath(),
   env = inherited
 ) {
-  const child = spawn(executable, args, { env, stdio: ['ignore', 'ignore', 'pipe'] });
+  const child = spawn(executable, args, {
+    env,
+    stdio: ['ignore', 'ignore', 'pipe', 'pipe', 'pipe'],
+  });
   const stderr = lines(child.stderr);
   const drained = new Promise((resolve) => child.stderr.once('close', resolve));
   const exited = new Promise((resolve) => {
@@ -99,17 +103,102 @@ export function died(child) {
   return (child.exitCode ?? 0) !== 0;
 }
 
-export async function stop(cdp, child) {
+export const WAITS = { close: 10000, exit: 10000, kill: 5000 };
+
+export async function stop(cdp, child, waits = WAITS) {
+  const gone = () => child.exitCode !== null || child.signalCode !== null;
   const exited = new Promise((resolve) => child.once('exit', resolve));
-  await bounded(cdp.send('Browser.close').catch(ignore), 'Browser.close');
+  await bounded(cdp.send('Browser.close').catch(ignore), 'Browser.close', waits.close);
   cdp.close();
-  const running = child.exitCode === null && child.signalCode === null;
-  if (running && (await bounded(exited, 'exit')) === TIMED_OUT) {
-    child.kill('SIGKILL');
-    await exited;
+  if (gone() || (await bounded(exited, 'exit', waits.exit)) !== TIMED_OUT || gone()) return;
+  child.kill('SIGKILL');
+  if (gone() || (await bounded(exited, 'exit after SIGKILL', waits.kill)) !== TIMED_OUT) return;
+  if (!gone()) console.warn(`chrome: pid ${child.pid} still runs after SIGKILL, moving on`);
+}
+
+export const PREFIX = 'slicc-harness-';
+export const DAY = 24 * 60 * 60 * 1000;
+export const GRACE = 60 * 1000;
+
+const signals = ['SIGINT', 'SIGTERM', 'SIGHUP'];
+const running = new Set();
+
+function kill(child) {
+  if (child.exitCode === null && child.signalCode === null) child.kill('SIGKILL');
+}
+
+function reap() {
+  for (const { children, paths } of running) {
+    for (const child of children) kill(child);
+    for (const path of paths) rmSync(path, { recursive: true, force: true });
   }
+  running.clear();
+  unhook();
+}
+
+function reraise(signal) {
+  reap();
+  process.kill(process.pid, signal);
+}
+
+function hook() {
+  process.on('exit', reap);
+  for (const signal of signals) process.on(signal, reraise);
+}
+
+function unhook() {
+  process.off('exit', reap);
+  for (const signal of signals) process.off(signal, reraise);
 }
 
+export function watchdog(pid) {
+  const dog = spawn('sh', ['-c', 'read -r _; kill -9 "$0" 2>/dev/null', String(pid)], {
+    stdio: ['pipe', 'ignore', 'ignore'],
+  });
+  dog.on('error', ignore);
+  dog.stdin.on('error', ignore);
+  return dog;
+}
+
+export function track(child, paths) {
+  const dog = watchdog(child.pid);
+  const entry = { children: [child, dog], paths };
+  if (running.size === 0) hook();
+  running.add(entry);
+  return () => {
+    running.delete(entry);
+    if (running.size === 0) unhook();
+    kill(dog);
+  };
+}
+
+export async function commandLines() {
+  const { stdout } = await promisify(execFile)('ps', ['axww', '-o', 'args='], {
+    maxBuffer: 64 * 1024 * 1024,
+  });
+  return stdout.split('\n').map((line) => `${line} `);
+}
+
+export async function prune(dir = tmpdir(), grace = GRACE) {
+  const lines = await commandLines().catch(ignore);
+  const used = (path) => lines.some((line) => line.includes(`--user-data-dir=${path} `));
+  const names = await readdir(dir).catch(() => []);
+  const now = Date.now();
+  const removing = names
+    .filter((name) => name.startsWith(PREFIX))
+    .map(async (name) => {
+      const path = join(dir, name);
+      const info = await lstat(path).catch(ignore);
+      if (!info?.isDirectory()) return;
+      const age = now - info.mtimeMs;
+      const orphaned = lines !== null && age > grace && !used(path);
+      if (age > DAY || orphaned) await rm(path, { recursive: true, force: true }).catch(ignore);
+    });
+  await Promise.all(removing);
+}
+
+let pruning = null;
+
 export async function launch({
   roots,
   aliases,
@@ -122,21 +211,24 @@ export async function launch({
   timeout,
   stallAfter,
 } = {}) {
+  pruning ??= prune();
+  await pruning;
   const server = await serve({ roots, aliases, isolated });
   const found = await unpacked(extensions);
   const fromExtension = extensionSource(found);
   const files = { ...server, source: (href) => server.source(href) ?? fromExtension(href) };
   const shared = extensions.length > 0;
-  const profile = await mkdtemp(join(tmpdir(), 'slicc-harness-'));
+  const profile = await mkdtemp(join(tmpdir(), PREFIX));
   const loaded = [...found.values()].map((root) => root.slice(0, -1));
   const dumps = await crashpad();
-  const { child, url, exited, stderr } = await start(
+  const { child, exited, stderr } = await start(
     profile,
     commandLine(profile, loaded, args),
     chromium.executablePath(),
     dumps.env
   );
-  const cdp = await connect(url, {
+  const untrack = track(child, [profile, dumps.dir]);
+  const cdp = await connect(pipe(child.stdio[3], child.stdio[4]), {
     exited,
     tail: () => stderr.tail().map((line) => `stderr: ${line}`),
   });
@@ -154,6 +246,7 @@ export async function launch({
     waitForDebuggerOnStart: true,
     flatten: true,
   });
+  await install(cdp, found);
 
   async function tabs() {
     const { targetInfos } = await cdp.send('Target.getTargets');
@@ -189,6 +282,7 @@ export async function launch({
   return {
     url: server.url,
     pid: child.pid,
+    profile,
     cdn: remote.state,
     requests: server.requests,
     overrides: server.overrides,
@@ -223,6 +317,7 @@ export async function launch({
     async close() {
       await stop(cdp, child);
       await rm(profile, { recursive: true, force: true });
+      untrack();
       closed = true;
       if (open === 0) await dumps.remove();
       await server.close();

package/harness/extensions.mjs

@@ -26,3 +26,10 @@ export function extensionSource(found) {
     return root ? join(root, normalize(decodeURIComponent(pathname))) : null;
   };
 }
+
+export async function install(cdp, found) {
+  for (const [expected, root] of [...found]) {
+    const { id } = await cdp.send('Extensions.loadUnpacked', { path: root.slice(0, -1) });
+    if (id !== expected) found.set(id, root);
+  }
+}

package/harness/recorder.mjs

@@ -125,6 +125,7 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
   const pauses = new Map();
   const parents = new Map();
   const tabs = new Map();
+  const early = [];
   const ours = covered(server, coverage);
   let run = null;
 
@@ -204,8 +205,8 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
 
   function log(sessionId, line) {
     const target = sessions.get(sessionId);
-    if (!run || target?.browserContextId !== run.context) return;
-    run.console.push(`${label(target)}: ${line}`);
+    const into = run ? target?.browserContextId === run.context && run.console : target && early;
+    if (into) into.push(`${label(target)}: ${line}`);
   }
 
   async function capture(sessionId, target) {
@@ -274,7 +275,7 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
   }
 
   function begin(context, dir, name) {
-    run = { context, dir, name, scripts: [], console: [], profiles: 0, self: {} };
+    run = { context, dir, name, scripts: [], console: early.splice(0), profiles: 0, self: {} };
   }
 
   async function end() {

package/harness/server.mjs

@@ -78,6 +78,10 @@ export async function serve({
     locate,
     source: (href) =>
       href.startsWith(url) ? lookup(decodeURIComponent(new URL(href).pathname)) : null,
-    close: () => new Promise((done) => server.close(done)),
+    close: () =>
+      new Promise((done) => {
+        server.close(done);
+        server.closeAllConnections();
+      }),
   };
 }

package/harness/util.mjs

@@ -19,7 +19,11 @@ export function sleep(ms) {
 }
 
 export async function bounded(promise, label, ms = 10000) {
-  const result = await Promise.race([promise, sleep(ms).then(() => TIMED_OUT)]);
+  let timer;
+  const expired = new Promise((resolve) => {
+    timer = setTimeout(resolve, ms, TIMED_OUT);
+  });
+  const result = await Promise.race([promise, expired]).finally(() => clearTimeout(timer));
   if (result === TIMED_OUT) console.warn(`chrome: ${label} took over ${ms} ms, moving on`);
   return result;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ai-ecoverse/slicc-shared-web",
-  "version": "1.10.0",
+  "version": "1.10.2",
   "description": "Shared Renovate, Biome, TypeScript, lefthook and GitHub Actions configuration, plus the CDP integration-test harness, for SLICC's web repos",
   "type": "module",
   "license": "Apache-2.0",