@ai-ecoverse/slicc-shared-web 1.10.2 → 1.11.0

package/README.md

@@ -31,7 +31,9 @@ The hooks expect two scripts in each repo: `test:unit`, which writes `coverage/u
 
 ## Integration-test harness
 
-`harness/` drives playwright-core's Chromium over raw CDP on `--remote-debugging-pipe`, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage and the CPU profiler switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: one CPU profile per target and snapshot, a screenshot per tab and a `console.log` of every target. Lines logged before the first test, such as an extension service worker's at startup, go into the first test's `console.log`. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`, the 15 frames with the most self time in the repo's own scripts.
+`harness/` drives playwright-core's Chromium over raw CDP on `--remote-debugging-pipe`, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage (and the CPU profiler, when profiling is on) switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: a screenshot per tab and a `console.log` of every target. Lines logged before the first test, such as an extension service worker's at startup, go into the first test's `console.log`. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`.
+
+CPU profiling is off by default. V8's sampler interrupts each page and worker thread every 100 µs with a signal, and the renderer SIGSEGVs in slicc-bios#114 all fault inside a signal handler walking the interrupted thread's frames, which points at it. Turn it on with `profile: true` or `SLICC_PROFILE=1` to get one CPU profile per target and snapshot in the test's artifact directory, and the 15 frames with the most self time in the repo's own scripts in `hotspots.md`. Without it, `hotspots.md` says that profiling is off.
 
 `playwright-core` and `monocart-coverage-reports` are peer dependencies; pin both in the repo's `devDependencies`. A repo describes what to serve and launches the browser once per test file:
 
@@ -64,6 +66,7 @@ test('boots', async (t) => {
 | `args` | Extra Chromium flags, such as `--host-resolver-rules` or `--ignore-certificate-errors` | none |
 | `timeout` | How long `page.until` polls and `page.evaluate` waits, in ms | `30000` |
 | `stallAfter` | How long one evaluation may run before the harness probes the test's targets, in ms | `30000` |
+| `profile` | Record a CPU profile of every target, sampled every 100 µs | `true` if `SLICC_PROFILE=1`, else `false` |
 
 `chrome.page(t)` opens a tab and returns the page API: `goto(path)`, `reload()`, `evaluate(fn, ...args)`, `until(fn, ...args)`, `press(key, ...modifiers)` (a key name like `Enter` or `F5`, or a single character typed with its US-layout key code), `type(text)`, `insert(text)`, `enter()`, `init(fn)` (runs in every new document), `expose(name, handler)` (a binding the page calls with a JSON string), `screenshot(file)`, `tab()` (another tab in the same context), `close()`, `send(method, params, ms)` (a raw CDP call in the tab's session), `errors`, `responses` and `dir` (the test's artifact directory). `chrome.pid` is the browser's process id and `chrome.profile` its `slicc-harness-*` profile directory in `$TMPDIR`. `goto` and `reload` wait until the new document is complete and no longer `about:blank`, and retry the navigation once if that never happens. `until` polls until the function returns a truthy value (not only strict `true`) and resolves with that value; the timeout error names the last result. `within(ms, fn, ...args)` is `until` with its own deadline, for a stage that should take a known time. Both stop at the deadline even if one evaluation never answers. Every CDP call has a deadline: `connect(url).send(method, params, sessionId, ms)` rejects with `<method>: no answer in <ms> ms` (60 s unless given), so a stuck page fails its test instead of hanging the run. A crash fails it at once instead:
 - when the browser dies, every pending and later call rejects with `<method>: browser exited with <signal>` (or `with code <n>`);
@@ -75,7 +78,7 @@ test('boots', async (t) => {
  When one evaluation runs past `stallAfter`, the harness probes once, without interrupting the wait:
 - it adds a `pending` line to `console.log` for every navigation and request of the page that has no answer yet, with its age. A worker's script loads in the worker's own session, so its request is left out. Chrome holds CDP messages to a page while a navigation is waiting for its response, so an evaluation that hangs while the page is idle usually means one of these;
 - it saves `stall-<n>.png` (a screenshot of the page, if the renderer still answers) in the test's artifact directory;
-- it saves a CPU profile sample of every page and worker in the test, including workers started by other workers, as `stall-<n>-<target>-<session>.cpuprofile`;
+- when profiling is on, it saves a CPU profile sample of every page and worker in the test, including workers started by other workers, as `stall-<n>-<target>-<session>.cpuprofile`;
 - it adds a line per target to `console.log`, saying whether that target is running or paused;
 - on the first stall, it records a 10 s browser trace as `stall-1-trace.json` (open it in DevTools' Performance panel or Perfetto). The browser collects it with V8 CPU samples per thread, task, IPC, Blink and loading events, so it shows what a page's main thread is doing even when that thread no longer answers CDP. It logs each process's CPU time over the window, and for each renderer its main thread's task count, the call chain of its longest or still open task (function, file, line and URL of a request), and where its CPU samples fall. The targets are sampled after the trace starts, so their new profiles carry their call frames into it. A call that started before the trace shows only as a node id, since its frames were named before; a thread that is blocked keeps `Tracing.start` waiting, which the harness logs and traces through;
 - it adds a line for every debugger pause still held in the browser, in any target, with its id, URL, start time, reason and top call frames.
@@ -99,8 +102,8 @@ Why it works the way it does:
 ## Cleanup
 
 `close()` stops Chrome and removes its profile. It asks Chrome to close, waits up to 10 s for the exit, sends `SIGKILL`, waits up to 5 s more and then moves on with a warning, so a browser that ignores `Browser.close` or a missed exit never hangs teardown. A test process that ends without `close()` still leaves nothing running:
-- on `exit` and on `SIGINT`, `SIGTERM` or `SIGHUP`, the harness kills every browser it started in that process with `SIGKILL` and removes its profile and crash dump directory, then re-raises the signal so the process ends the way it would have;
-- when the test process dies of `SIGKILL`, no handler runs. A watchdog, a `sh` that reads a pipe from the test process, then gets end of file and kills Chrome with `SIGKILL` at once. Chrome also shuts itself down when the CDP pipe closes, but that can take seconds. Only the profile stays behind;
+- on `exit` and on `SIGINT`, `SIGTERM` or `SIGHUP`, the harness kills every browser it started in that process, with all its helper processes (Chrome runs in its own process group), with `SIGKILL` and removes its profile and crash dump directory, then re-raises the signal so the process ends the way it would have;
+- when the test process dies of `SIGKILL`, no handler runs. A watchdog, a `sh` in its own process group that reads a pipe from the test process, then gets end of file and kills Chrome's process group with `SIGKILL` at once. Chrome also shuts itself down when the CDP pipe closes, but that can take seconds. Only the profile stays behind;
 - the first `launch()` in a process removes every `slicc-harness-*` directory in `$TMPDIR` that is older than a day, or older than a minute and no running process uses as its `--user-data-dir`. It never kills a process.
 
 ## Crash dumps

package/harness/chrome.mjs

@@ -74,6 +74,7 @@ export async function start(
 ) {
   const child = spawn(executable, args, {
     env,
+    detached: true,
     stdio: ['ignore', 'ignore', 'pipe', 'pipe', 'pipe'],
   });
   const stderr = lines(child.stderr);
@@ -111,7 +112,7 @@ export async function stop(cdp, child, waits = WAITS) {
   await bounded(cdp.send('Browser.close').catch(ignore), 'Browser.close', waits.close);
   cdp.close();
   if (gone() || (await bounded(exited, 'exit', waits.exit)) !== TIMED_OUT || gone()) return;
-  child.kill('SIGKILL');
+  kill(child);
   if (gone() || (await bounded(exited, 'exit after SIGKILL', waits.kill)) !== TIMED_OUT) return;
   if (!gone()) console.warn(`chrome: pid ${child.pid} still runs after SIGKILL, moving on`);
 }
@@ -123,15 +124,27 @@ export const GRACE = 60 * 1000;
 const signals = ['SIGINT', 'SIGTERM', 'SIGHUP'];
 const running = new Set();
 
-function kill(child) {
+export function kill(child) {
+  if (child.pid > 0 && child.spawnargs?.length) {
+    try {
+      process.kill(-child.pid, 'SIGKILL');
+    } catch {}
+  }
   if (child.exitCode === null && child.signalCode === null) child.kill('SIGKILL');
 }
 
-function reap() {
-  for (const { children, paths } of running) {
-    for (const child of children) kill(child);
+function remove(entries) {
+  for (const { paths } of entries) {
     for (const path of paths) rmSync(path, { recursive: true, force: true });
   }
+}
+
+function reap() {
+  const entries = [...running];
+  for (const { children } of entries) children.forEach(kill);
+  remove(entries);
+  Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 200);
+  remove(entries);
   running.clear();
   unhook();
 }
@@ -152,7 +165,8 @@ function unhook() {
 }
 
 export function watchdog(pid) {
-  const dog = spawn('sh', ['-c', 'read -r _; kill -9 "$0" 2>/dev/null', String(pid)], {
+  const dog = spawn('sh', ['-c', 'read -r _; kill -9 -"$0" "$0" 2>/dev/null', String(pid)], {
+    detached: true,
     stdio: ['pipe', 'ignore', 'ignore'],
   });
   dog.on('error', ignore);
@@ -210,6 +224,7 @@ export async function launch({
   args = [],
   timeout,
   stallAfter,
+  profile = inherited.SLICC_PROFILE === '1',
 } = {}) {
   pruning ??= prune();
   await pruning;
@@ -218,16 +233,16 @@ export async function launch({
   const fromExtension = extensionSource(found);
   const files = { ...server, source: (href) => server.source(href) ?? fromExtension(href) };
   const shared = extensions.length > 0;
-  const profile = await mkdtemp(join(tmpdir(), PREFIX));
+  const userData = await mkdtemp(join(tmpdir(), PREFIX));
   const loaded = [...found.values()].map((root) => root.slice(0, -1));
   const dumps = await crashpad();
   const { child, exited, stderr } = await start(
-    profile,
-    commandLine(profile, loaded, args),
+    userData,
+    commandLine(userData, loaded, args),
     chromium.executablePath(),
     dumps.env
   );
-  const untrack = track(child, [profile, dumps.dir]);
+  const untrack = track(child, [userData, dumps.dir]);
   const cdp = await connect(pipe(child.stdio[3], child.stdio[4]), {
     exited,
     tail: () => stderr.tail().map((line) => `stderr: ${line}`),
@@ -238,7 +253,11 @@ export async function launch({
   cdp.on(({ method }) => {
     if (method === CRASHED || method === 'Target.targetCrashed') crashes += 1;
   });
-  const record = recorder(cdp, files, { coverage, exits: await breakpoints(server, exits) });
+  const record = recorder(cdp, files, {
+    coverage,
+    exits: await breakpoints(server, exits),
+    profiling: profile,
+  });
   const remote = await cdn(cdp, intercept);
   await cdp.send('Target.setDiscoverTargets', { discover: true });
   await cdp.send('Target.setAutoAttach', {
@@ -282,7 +301,7 @@ export async function launch({
   return {
     url: server.url,
     pid: child.pid,
-    profile,
+    profile: userData,
     cdn: remote.state,
     requests: server.requests,
     overrides: server.overrides,
@@ -316,7 +335,7 @@ export async function launch({
     },
     async close() {
       await stop(cdp, child);
-      await rm(profile, { recursive: true, force: true });
+      await rm(userData, { recursive: true, force: true });
       untrack();
       closed = true;
       if (open === 0) await dumps.remove();

package/harness/global.mjs

@@ -41,6 +41,9 @@ export function table(runs) {
       self.set(frame, (self.get(frame) ?? 0) + micros);
     }
   }
+  if (profiles === 0) {
+    return '### Hotspots: CPU profiling is off. Set SLICC_PROFILE=1 or pass `profile: true` to `launch()` to record profiles.\n';
+  }
   const top = [...self].sort((a, b) => b[1] - a[1]).slice(0, 15);
   const rows = top.map(([frame, micros]) => `| ${(micros / 1000).toFixed(2)} ms | \`${frame}\` |`);
   return [

package/harness/recorder.mjs

@@ -8,6 +8,7 @@ import { bounded, ignore, raw, samples, sleep, TIMED_OUT } from './util.mjs';
 
 const profiled = new Set(['page', 'worker', 'service_worker', 'shared_worker']);
 const nested = { autoAttach: true, waitForDebuggerOnStart: true, flatten: true };
+const sampling = [['Profiler.setSamplingInterval', { interval: 100 }], ['Profiler.start']];
 const unload = "addEventListener('beforeunload', () => { debugger; })";
 
 function pathOf(url) {
@@ -77,18 +78,28 @@ function frames(params) {
     .join(' < ');
 }
 
-async function sample(cdp, pauses, [sessionId, target], dir, tag) {
+async function sample(cdp, pauses, [sessionId, target], dir, tag, profiling) {
   const send = (method) => cdp.send(method, {}, sessionId, 8000);
-  const stopped = await send('Profiler.stop').catch((error) => ({ error }));
-  send('Profiler.start').catch(ignore);
   const pause = pauses.get(sessionId);
   const state = pause ? `paused (${pause.reason})` : 'running';
+  if (!profiling) return `${label(target)}: ${state}`;
+  const stopped = await send('Profiler.stop').catch((error) => ({ error }));
+  send('Profiler.start').catch(ignore);
   if (!stopped.profile) return `${label(target)}: ${state}, no profile (${stopped.error?.message})`;
   const name = `${tag}-${label(target)}-${sessionId.slice(0, 6)}.cpuprofile`;
   await writeFile(new URL(name, dir), JSON.stringify(stopped.profile)).catch(ignore);
   return `${label(target)}: ${state}, profile ${name}`;
 }
 
+function collect(send, profiling) {
+  const stop = profiling ? send('Profiler.stop') : {};
+  const taken = Promise.all([send('Profiler.takePreciseCoverage'), stop]);
+  return Promise.race([taken, sleep(3000)]).then(
+    (answer) => answer ?? [{}, {}],
+    () => [{}, {}]
+  );
+}
+
 function held(pauses) {
   return [...pauses.values()].map(
     (pause) =>
@@ -120,7 +131,7 @@ function contextOf(sessions, parents, sessionId) {
   return undefined;
 }
 
-export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } = {}) {
+export function recorder(cdp, server, { coverage = ['/'], exits = new Map(), profiling } = {}) {
   const sessions = new Map();
   const pauses = new Map();
   const parents = new Map();
@@ -132,14 +143,11 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
   async function dump(sessionId, target, restart) {
     const current = run;
     const send = (method) => cdp.send(method, {}, sessionId);
-    const taken = Promise.all([send('Profiler.takePreciseCoverage'), send('Profiler.stop')]);
-    const [{ result }, { profile }] = await Promise.race([taken, sleep(3000)]).then(
-      (answer) => answer ?? [{}, {}],
-      () => [{}, {}]
-    );
+    const [{ result }, { profile }] = await collect(send, profiling);
     if (!(result && current)) return;
-    if (restart) send('Profiler.start').catch(ignore);
     current.scripts.push(...result.filter(ours));
+    if (!profile) return;
+    if (restart) send('Profiler.start').catch(ignore);
     current.profiles += 1;
     selfTimes(profile, server, current.self);
     const name = `${String(current.profiles).padStart(3, '0')}-${label(target)}.cpuprofile`;
@@ -168,11 +176,10 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
       list.push(
         ['Runtime.enable'],
         ['Profiler.enable'],
-        ['Profiler.setSamplingInterval', { interval: 100 }],
         ['Profiler.startPreciseCoverage', { callCount: true, detailed: true }],
-        ['Profiler.start'],
         ['Target.setAutoAttach', nested]
       );
+      if (profiling) list.push(...sampling);
     }
     const lines = type === 'worker' ? exits.get(pathOf(url)) : undefined;
     if (lines) {
@@ -230,7 +237,8 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
     const current = [...sessions].filter(
       ([sessionId]) => contextOf(sessions, parents, sessionId) === run?.context
     );
-    const sampled = () => Promise.all(current.map((entry) => sample(cdp, pauses, entry, dir, tag)));
+    const sampled = () =>
+      Promise.all(current.map((entry) => sample(cdp, pauses, entry, dir, tag, profiling)));
     const [lines, traced] = first ? await trace(cdp, dir, tag, sampled) : [await sampled(), []];
     const all = [...pending, ...lines, ...traced, ...held(pauses)];
     for (const line of all) run?.console.push(`${tag}: ${line}`);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ai-ecoverse/slicc-shared-web",
-  "version": "1.10.2",
+  "version": "1.11.0",
   "description": "Shared Renovate, Biome, TypeScript, lefthook and GitHub Actions configuration, plus the CDP integration-test harness, for SLICC's web repos",
   "type": "module",
   "license": "Apache-2.0",
@@ -52,7 +52,7 @@
   "scripts": {
     "lint": "bin/slicc-lint.sh && shellcheck bin/*.sh",
     "pretest": "playwright-core install --no-shell chromium",
-    "test": "node --test --test-global-setup=harness/global.mjs test/integration/*.test.mjs && node test/integration/verify.mjs",
+    "test": "SLICC_PROFILE=1 node --test --test-global-setup=harness/global.mjs test/integration/*.test.mjs && node test/integration/verify.mjs",
     "test:unit": "mcr -c test/unit/mcr.config.mjs node --test 'test/unit/**/*.test.mjs'",
     "trust": "npx fledgling sync -y --skip-publish",
     "trust:dry": "npx fledgling sync --dry-run -y --skip-publish",