@ai-ecoverse/slicc-shared-web 1.10.3 → 1.12.0

package/README.md

@@ -20,7 +20,7 @@ Install it from npm, and pin the reusable workflow to the matching `vX.Y.Z` tag.
 npm install --save-dev --save-exact @ai-ecoverse/slicc-shared-web
 ```
 
-The hooks expect two scripts in each repo: `test:unit`, which writes `coverage/unit/lcov.info` (unit tests stay out of git), and, in TypeScript repos, `typecheck`. The reusable workflow expects `lint`, plus `test` for the integration tests, with a `pretest` that installs Chromium (`playwright-core install --no-shell chromium`). The diff-coverage hook runs `diff-cover` through `uvx`, so install [uv](https://docs.astral.sh/uv/) once.
+The hooks expect two scripts in each repo: `test:unit`, which writes `coverage/unit/lcov.info` (unit tests stay out of git), and, in TypeScript repos, `typecheck`. The reusable workflow expects `lint`, plus `test` for the integration tests, with a `pretest` that installs Chromium (`playwright-core install --no-shell chromium`). Its inputs `lint`, `test`, `chromium` and `node-version` override those, and `timeout-minutes` (20 by default) gives a long job, such as a background-tab measurement, more time. The diff-coverage hook runs `diff-cover` through `uvx`, so install [uv](https://docs.astral.sh/uv/) once.
 
 ## Rules every repo follows
 
@@ -31,7 +31,9 @@ The hooks expect two scripts in each repo: `test:unit`, which writes `coverage/u
 
 ## Integration-test harness
 
-`harness/` drives playwright-core's Chromium over raw CDP on `--remote-debugging-pipe`, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage and the CPU profiler switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: one CPU profile per target and snapshot, a screenshot per tab and a `console.log` of every target. Lines logged before the first test, such as an extension service worker's at startup, go into the first test's `console.log`. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`, the 15 frames with the most self time in the repo's own scripts.
+`harness/` drives playwright-core's Chromium over raw CDP on `--remote-debugging-pipe`, as its only client, from `node --test`. Every page, service worker, shared worker and dedicated worker, nested ones included, starts paused, gets coverage (and the CPU profiler, when profiling is on) switched on in the same tick, and only then runs. Each test gets its own browser context and leaves `artifacts/<suite>/<test>/` behind: a screenshot per tab and a `console.log` of every target. Lines logged before the first test, such as an extension service worker's at startup, go into the first test's `console.log`. The global teardown merges the raw coverage into `coverage/` (console table, lcov, V8 HTML) and writes `artifacts/hotspots.md`.
+
+CPU profiling is off by default. V8's sampler interrupts each page and worker thread every 100 µs with a signal, and the renderer SIGSEGVs in slicc-bios#114 all fault inside a signal handler walking the interrupted thread's frames, which points at it. Turn it on with `profile: true` or `SLICC_PROFILE=1` to get one CPU profile per target and snapshot in the test's artifact directory, and the 15 frames with the most self time in the repo's own scripts in `hotspots.md`. Without it, `hotspots.md` says that profiling is off.
 
 `playwright-core` and `monocart-coverage-reports` are peer dependencies; pin both in the repo's `devDependencies`. A repo describes what to serve and launches the browser once per test file:
 
@@ -64,6 +66,8 @@ test('boots', async (t) => {
 | `args` | Extra Chromium flags, such as `--host-resolver-rules` or `--ignore-certificate-errors` | none |
 | `timeout` | How long `page.until` polls and `page.evaluate` waits, in ms | `30000` |
 | `stallAfter` | How long one evaluation may run before the harness probes the test's targets, in ms | `30000` |
+| `profile` | Record a CPU profile of every target, sampled every 100 µs | `true` if `SLICC_PROFILE=1`, else `false` |
+| `realistic` | Run Chrome as a user would: headed, and without `--disable-background-timer-throttling`, `--disable-backgrounding-occluded-windows` and `--disable-renderer-backgrounding`, so a tab behind another one is hidden and throttled. For measuring background tabs. It needs a display: on CI, run the tests under `xvfb-run -a` | `false` |
 
 `chrome.page(t)` opens a tab and returns the page API: `goto(path)`, `reload()`, `evaluate(fn, ...args)`, `until(fn, ...args)`, `press(key, ...modifiers)` (a key name like `Enter` or `F5`, or a single character typed with its US-layout key code), `type(text)`, `insert(text)`, `enter()`, `init(fn)` (runs in every new document), `expose(name, handler)` (a binding the page calls with a JSON string), `screenshot(file)`, `tab()` (another tab in the same context), `close()`, `send(method, params, ms)` (a raw CDP call in the tab's session), `errors`, `responses` and `dir` (the test's artifact directory). `chrome.pid` is the browser's process id and `chrome.profile` its `slicc-harness-*` profile directory in `$TMPDIR`. `goto` and `reload` wait until the new document is complete and no longer `about:blank`, and retry the navigation once if that never happens. `until` polls until the function returns a truthy value (not only strict `true`) and resolves with that value; the timeout error names the last result. `within(ms, fn, ...args)` is `until` with its own deadline, for a stage that should take a known time. Both stop at the deadline even if one evaluation never answers. Every CDP call has a deadline: `connect(url).send(method, params, sessionId, ms)` rejects with `<method>: no answer in <ms> ms` (60 s unless given), so a stuck page fails its test instead of hanging the run. A crash fails it at once instead:
 - when the browser dies, every pending and later call rejects with `<method>: browser exited with <signal>` (or `with code <n>`);
@@ -75,7 +79,7 @@ test('boots', async (t) => {
  When one evaluation runs past `stallAfter`, the harness probes once, without interrupting the wait:
 - it adds a `pending` line to `console.log` for every navigation and request of the page that has no answer yet, with its age. A worker's script loads in the worker's own session, so its request is left out. Chrome holds CDP messages to a page while a navigation is waiting for its response, so an evaluation that hangs while the page is idle usually means one of these;
 - it saves `stall-<n>.png` (a screenshot of the page, if the renderer still answers) in the test's artifact directory;
-- it saves a CPU profile sample of every page and worker in the test, including workers started by other workers, as `stall-<n>-<target>-<session>.cpuprofile`;
+- when profiling is on, it saves a CPU profile sample of every page and worker in the test, including workers started by other workers, as `stall-<n>-<target>-<session>.cpuprofile`;
 - it adds a line per target to `console.log`, saying whether that target is running or paused;
 - on the first stall, it records a 10 s browser trace as `stall-1-trace.json` (open it in DevTools' Performance panel or Perfetto). The browser collects it with V8 CPU samples per thread, task, IPC, Blink and loading events, so it shows what a page's main thread is doing even when that thread no longer answers CDP. It logs each process's CPU time over the window, and for each renderer its main thread's task count, the call chain of its longest or still open task (function, file, line and URL of a request), and where its CPU samples fall. The targets are sampled after the trace starts, so their new profiles carry their call frames into it. A call that started before the trace shows only as a node id, since its frames were named before; a thread that is blocked keeps `Tracing.start` waiting, which the harness logs and traces through;
 - it adds a line for every debugger pause still held in the browser, in any target, with its id, URL, start time, reason and top call frames.

package/harness/chrome.mjs

@@ -32,9 +32,20 @@ export const flags = [
   '--window-size=1280,800',
 ];
 
-export function commandLine(profile, extensions = [], extra = []) {
+export const unrealistic = [
+  '--headless',
+  '--disable-background-timer-throttling',
+  '--disable-backgrounding-occluded-windows',
+  '--disable-renderer-backgrounding',
+];
+
+export function commandLine(profile, extensions = [], extra = [], realistic = false) {
   const loaded = extensions.length > 0;
-  const base = loaded ? flags.filter((flag) => flag !== '--disable-extensions') : flags;
+  const dropped = new Set([
+    ...(loaded ? ['--disable-extensions'] : []),
+    ...(realistic ? unrealistic : []),
+  ]);
+  const base = flags.filter((flag) => !dropped.has(flag));
   const load = loaded ? ['--enable-unsafe-extension-debugging'] : [];
   return [...base, ...load, ...extra, `--user-data-dir=${profile}`, 'about:blank'];
 }
@@ -224,6 +235,8 @@ export async function launch({
   args = [],
   timeout,
   stallAfter,
+  profile = inherited.SLICC_PROFILE === '1',
+  realistic = false,
 } = {}) {
   pruning ??= prune();
   await pruning;
@@ -232,16 +245,16 @@ export async function launch({
   const fromExtension = extensionSource(found);
   const files = { ...server, source: (href) => server.source(href) ?? fromExtension(href) };
   const shared = extensions.length > 0;
-  const profile = await mkdtemp(join(tmpdir(), PREFIX));
+  const userData = await mkdtemp(join(tmpdir(), PREFIX));
   const loaded = [...found.values()].map((root) => root.slice(0, -1));
   const dumps = await crashpad();
   const { child, exited, stderr } = await start(
-    profile,
-    commandLine(profile, loaded, args),
+    userData,
+    commandLine(userData, loaded, args, realistic),
     chromium.executablePath(),
     dumps.env
   );
-  const untrack = track(child, [profile, dumps.dir]);
+  const untrack = track(child, [userData, dumps.dir]);
   const cdp = await connect(pipe(child.stdio[3], child.stdio[4]), {
     exited,
     tail: () => stderr.tail().map((line) => `stderr: ${line}`),
@@ -252,7 +265,11 @@ export async function launch({
   cdp.on(({ method }) => {
     if (method === CRASHED || method === 'Target.targetCrashed') crashes += 1;
   });
-  const record = recorder(cdp, files, { coverage, exits: await breakpoints(server, exits) });
+  const record = recorder(cdp, files, {
+    coverage,
+    exits: await breakpoints(server, exits),
+    profiling: profile,
+  });
   const remote = await cdn(cdp, intercept);
   await cdp.send('Target.setDiscoverTargets', { discover: true });
   await cdp.send('Target.setAutoAttach', {
@@ -296,7 +313,7 @@ export async function launch({
   return {
     url: server.url,
     pid: child.pid,
-    profile,
+    profile: userData,
     cdn: remote.state,
     requests: server.requests,
     overrides: server.overrides,
@@ -330,7 +347,7 @@ export async function launch({
     },
     async close() {
       await stop(cdp, child);
-      await rm(profile, { recursive: true, force: true });
+      await rm(userData, { recursive: true, force: true });
       untrack();
       closed = true;
       if (open === 0) await dumps.remove();

package/harness/global.mjs

@@ -41,6 +41,9 @@ export function table(runs) {
       self.set(frame, (self.get(frame) ?? 0) + micros);
     }
   }
+  if (profiles === 0) {
+    return '### Hotspots: CPU profiling is off. Set SLICC_PROFILE=1 or pass `profile: true` to `launch()` to record profiles.\n';
+  }
   const top = [...self].sort((a, b) => b[1] - a[1]).slice(0, 15);
   const rows = top.map(([frame, micros]) => `| ${(micros / 1000).toFixed(2)} ms | \`${frame}\` |`);
   return [

package/harness/recorder.mjs

@@ -8,6 +8,7 @@ import { bounded, ignore, raw, samples, sleep, TIMED_OUT } from './util.mjs';
 
 const profiled = new Set(['page', 'worker', 'service_worker', 'shared_worker']);
 const nested = { autoAttach: true, waitForDebuggerOnStart: true, flatten: true };
+const sampling = [['Profiler.setSamplingInterval', { interval: 100 }], ['Profiler.start']];
 const unload = "addEventListener('beforeunload', () => { debugger; })";
 
 function pathOf(url) {
@@ -77,18 +78,28 @@ function frames(params) {
     .join(' < ');
 }
 
-async function sample(cdp, pauses, [sessionId, target], dir, tag) {
+async function sample(cdp, pauses, [sessionId, target], dir, tag, profiling) {
   const send = (method) => cdp.send(method, {}, sessionId, 8000);
-  const stopped = await send('Profiler.stop').catch((error) => ({ error }));
-  send('Profiler.start').catch(ignore);
   const pause = pauses.get(sessionId);
   const state = pause ? `paused (${pause.reason})` : 'running';
+  if (!profiling) return `${label(target)}: ${state}`;
+  const stopped = await send('Profiler.stop').catch((error) => ({ error }));
+  send('Profiler.start').catch(ignore);
   if (!stopped.profile) return `${label(target)}: ${state}, no profile (${stopped.error?.message})`;
   const name = `${tag}-${label(target)}-${sessionId.slice(0, 6)}.cpuprofile`;
   await writeFile(new URL(name, dir), JSON.stringify(stopped.profile)).catch(ignore);
   return `${label(target)}: ${state}, profile ${name}`;
 }
 
+function collect(send, profiling) {
+  const stop = profiling ? send('Profiler.stop') : {};
+  const taken = Promise.all([send('Profiler.takePreciseCoverage'), stop]);
+  return Promise.race([taken, sleep(3000)]).then(
+    (answer) => answer ?? [{}, {}],
+    () => [{}, {}]
+  );
+}
+
 function held(pauses) {
   return [...pauses.values()].map(
     (pause) =>
@@ -120,7 +131,7 @@ function contextOf(sessions, parents, sessionId) {
   return undefined;
 }
 
-export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } = {}) {
+export function recorder(cdp, server, { coverage = ['/'], exits = new Map(), profiling } = {}) {
   const sessions = new Map();
   const pauses = new Map();
   const parents = new Map();
@@ -132,14 +143,11 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
   async function dump(sessionId, target, restart) {
     const current = run;
     const send = (method) => cdp.send(method, {}, sessionId);
-    const taken = Promise.all([send('Profiler.takePreciseCoverage'), send('Profiler.stop')]);
-    const [{ result }, { profile }] = await Promise.race([taken, sleep(3000)]).then(
-      (answer) => answer ?? [{}, {}],
-      () => [{}, {}]
-    );
+    const [{ result }, { profile }] = await collect(send, profiling);
     if (!(result && current)) return;
-    if (restart) send('Profiler.start').catch(ignore);
     current.scripts.push(...result.filter(ours));
+    if (!profile) return;
+    if (restart) send('Profiler.start').catch(ignore);
     current.profiles += 1;
     selfTimes(profile, server, current.self);
     const name = `${String(current.profiles).padStart(3, '0')}-${label(target)}.cpuprofile`;
@@ -168,11 +176,10 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
       list.push(
         ['Runtime.enable'],
         ['Profiler.enable'],
-        ['Profiler.setSamplingInterval', { interval: 100 }],
         ['Profiler.startPreciseCoverage', { callCount: true, detailed: true }],
-        ['Profiler.start'],
         ['Target.setAutoAttach', nested]
       );
+      if (profiling) list.push(...sampling);
     }
     const lines = type === 'worker' ? exits.get(pathOf(url)) : undefined;
     if (lines) {
@@ -230,7 +237,8 @@ export function recorder(cdp, server, { coverage = ['/'], exits = new Map() } =
     const current = [...sessions].filter(
       ([sessionId]) => contextOf(sessions, parents, sessionId) === run?.context
     );
-    const sampled = () => Promise.all(current.map((entry) => sample(cdp, pauses, entry, dir, tag)));
+    const sampled = () =>
+      Promise.all(current.map((entry) => sample(cdp, pauses, entry, dir, tag, profiling)));
     const [lines, traced] = first ? await trace(cdp, dir, tag, sampled) : [await sampled(), []];
     const all = [...pending, ...lines, ...traced, ...held(pauses)];
     for (const line of all) run?.console.push(`${tag}: ${line}`);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ai-ecoverse/slicc-shared-web",
-  "version": "1.10.3",
+  "version": "1.12.0",
   "description": "Shared Renovate, Biome, TypeScript, lefthook and GitHub Actions configuration, plus the CDP integration-test harness, for SLICC's web repos",
   "type": "module",
   "license": "Apache-2.0",
@@ -52,7 +52,7 @@
   "scripts": {
     "lint": "bin/slicc-lint.sh && shellcheck bin/*.sh",
     "pretest": "playwright-core install --no-shell chromium",
-    "test": "node --test --test-global-setup=harness/global.mjs test/integration/*.test.mjs && node test/integration/verify.mjs",
+    "test": "SLICC_PROFILE=1 node --test --test-global-setup=harness/global.mjs test/integration/*.test.mjs && node test/integration/verify.mjs",
     "test:unit": "mcr -c test/unit/mcr.config.mjs node --test 'test/unit/**/*.test.mjs'",
     "trust": "npx fledgling sync -y --skip-publish",
     "trust:dry": "npx fledgling sync --dry-run -y --skip-publish",