webperf-snippets 1.11.0 → 1.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -73,7 +73,7 @@ npx webperf-snippets https://example.com --json
73
73
  The JSON output records how the page was measured in `conditions`, so a later run can tell whether it measured the same way: the viewport, the workflow (or the path of the snippet with `--snippet`), the wait, the version of the CLI and the machine: its CPU model, cores, architecture and memory, as the operating system reports them, and a benchmark index, a short CPU test in Node before Chromium starts, which only informs, since on the same machine it moves by up to 40 % with the load.
74
74
 
75
75
  ```json
76
- "conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.11.0", "machine": { "cpu": "Apple M4", "cores": 10, "arch": "arm64", "memoryGB": 16, "benchmarkIndex": 2371 } }
76
+ "conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.12.0", "machine": { "cpu": "Apple M4", "cores": 10, "arch": "arm64", "memoryGB": 16, "benchmarkIndex": 2371 } }
77
77
  ```
78
78
 
79
79
  Single snippet:
@@ -171,7 +171,7 @@ Without `--report-to` the CLI makes no external calls. With it, the results are
171
171
  "timestamp": "2026-05-06T10:00:00.000Z",
172
172
  "navMs": 1240,
173
173
  "results": [],
174
- "meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.11.0" }
174
+ "meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.12.0" }
175
175
  }
176
176
  ```
177
177
 
@@ -297,7 +297,7 @@ The report adds the comparison after the usual one, and `--markdown` a table for
297
297
  }
298
298
  ```
299
299
 
300
- When both runs were measured differently, with another workflow or snippet, viewport, wait, throttling, number of runs or version of the CLI, on another CPU model or with other cores, or on another path or query, the CLI compares anyway and warns on stderr, in the reports and in `warnings`: whoever reads the pull request comment does not see the stderr of the CI. Another host does not warn, since it is the review app of a pull request against production. A baseline of one run warns that only the thresholds apply.
300
+ When both runs were measured differently, with another workflow or snippet, viewport, wait, throttling, number of runs or version of the CLI, on another CPU model or with other cores, or on another path or query, the CLI compares anyway and warns on stderr, in the reports and in `warnings`: whoever reads the pull request comment does not see the stderr of the CI. Another host does not warn, since it is the review app of a pull request against production. A baseline of one run warns that only the thresholds apply, and any other metric compared without statistics gets a line with the values left on each side, as in `no statistics for LCP: the baseline has 5 values, and this run 4 after 1 failed run; they need 5 a side, so measure both with 5 runs or more`.
301
301
 
302
302
  The comparison only informs, and the exit code is the same as without it, unless `--fail-on-regression`, which exits with `1` when a metric regressed or an error is new, and names them on stderr. A baseline that does not exist or is not an output of the CLI, a baseline of a page for a matrix or of a matrix for a page, or a `--threshold` the CLI does not understand exits with code `2`, before the page loads.
303
303
 
@@ -351,6 +351,7 @@ Each budget compares the value measured, or the median with `--runs`, and fails
351
351
  | `--verbose` | Show all items, including passing checks. |
352
352
  | `--headed` | Show the browser window (debug). |
353
353
  | `-h, --help` | Show help. |
354
+ | `-v, --version` | Show the version. |
354
355
 
355
356
  ### Snippet aliases
356
357
 
@@ -360,6 +361,11 @@ Each budget compares the value measured, or the median with `--runs`, and fails
360
361
  | `CLS` | CoreWebVitals/CLS |
361
362
  | `INP` | CoreWebVitals/INP |
362
363
  | `LCP-Subparts` | CoreWebVitals/LCP-Subparts |
364
+ | `Largest-Contentful-Paint` | CoreWebVitals/LCP |
365
+ | `Cumulative-Layout-Shift` | CoreWebVitals/CLS |
366
+ | `Interaction-to-Next-Paint` | CoreWebVitals/INP |
367
+ | `First-Contentful-Paint` | Loading/FCP |
368
+ | `Time-to-First-Byte` | Loading/TTFB |
363
369
  | `fonts` | Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold |
364
370
  | `render-blocking` | Loading/Find-render-blocking-resources |
365
371
  | `resource-hints` | Loading/Resource-Hints-Validation |
@@ -411,9 +417,11 @@ It needs the Chromium of Playwright (`npx playwright install chromium`); without
411
417
 
412
418
  Each run is a page load plus `waitMs`. With the default wait of 3 seconds, five runs of a workflow take about 20 seconds on a local page, and about a minute on a page that takes a few seconds to load. When the client asks for progress, the server reports the three stages of each run, numbered across the runs. A client sets its own limit on a tool call: the MCP SDK stops waiting after 60 seconds unless it is given a longer `timeout` or `resetTimeoutOnProgress`, and in Claude Code `MCP_TOOL_TIMEOUT` sets the limit. When the client gives up, the run in progress ends and no other starts.
413
419
 
420
+ An agent often sends several calls in one turn. The calls of a connection that make a page work (`run_snippet`, `run_workflow`, `open_page`, `interact`) run one at a time, in the order they arrive, so they do not compete for the CPU and the turn saves no time. A call keeps its turn only while it runs, so a page left open holds up nothing, and `read_results`, `close_page` and the tools that load no page answer at once. Waiting counts towards the limit of the client; when it asks for progress, a waiting call reports how many calls are ahead, when the queue moves and every 5 seconds. A call the client gives up on leaves the queue and its page never loads; if it was the one measuring, its Chromium closes and the next call starts. A page that stops answering fails its call after 2 minutes per run, plus the `wait` steps of `interact`, and closes. Two clients at the same time, or a client and the CLI, have no queue in common.
421
+
414
422
  `run_snippet` and `run_workflow` compare a run with an earlier one, with the verdict of `--baseline`. `baseline` takes the `runId` of an earlier run of the connection, which every run of these two tools returns, or the path of a page the CLI wrote with `--json`, absolute or relative to the working directory of the server. The answer adds a `baseline` field: the one of the JSON of the CLI, with the `runId` or the `file` it compared with, plus `regressions`, the same lines `--fail-on-regression` prints. `thresholds` takes the values of `--threshold`, such as `["LCP=15%", "CLS=0.1"]`. A baseline that cannot be read or a threshold the server does not understand is a tool error, before the page loads, and a baseline of a matrix is refused, since a tool measures one page.
415
423
 
416
- With five runs or more a side, a regression needs a Mann-Whitney U p-value under 0.05 on top of its threshold, as in the CLI, and each metric adds `stats` with the p-value, the shift and its 95 % interval. With fewer, a metric compares by its threshold and, against a baseline of several runs, by the range of those runs. A single run can be off by more than a threshold on a page that did not change, so read `warnings`, and measure both sides with `runs: 5`, or `--runs 5` in the CLI, when the verdict matters. An earlier run of the connection works for a change made in the same session, and a file measured with the CLI for a comparison with `main`.
424
+ With five runs or more a side, a regression needs a Mann-Whitney U p-value under 0.05 on top of its threshold, as in the CLI, and each metric adds `stats` with the p-value, the shift and its 95 % interval. With fewer, a metric compares by its threshold and, against a baseline of several runs, by the range of those runs. A single run can be off by more than a threshold on a page that did not change, so read `warnings`, which also names each metric left without statistics and the values on each side after failed runs, and measure both sides with `runs: 5`, or `--runs 5` in the CLI, when the verdict matters. An earlier run of the connection works for a change made in the same session, and a file measured with the CLI for a comparison with `main`.
417
425
 
418
426
  The server also offers resources, documents a client can attach to the context:
419
427
 
@@ -533,6 +541,7 @@ The `NPM_TOKEN` secret must be set in the repository settings with publish acces
533
541
  - **CLS in headless is conservative**: layout shifts that only happen on scroll are missed unless you script the scroll.
534
542
  - **First navigation only**: each `webperf-snippets` invocation runs one URL. SPAs need the post-route URL passed directly.
535
543
  - **Synthetic INP ≠ field INP**: `--interact-script` measures handler latency for a single scripted event. Real INP reflects the worst interaction across all user sessions; use RUM for field data.
544
+ - **A click that paints nothing**: INP is under 16 ms with the `no-paint-measured` issue, as web-vitals reports it, so it rates `good` and passes `--budget-inp`. Check that the selector reaches an element that responds. `tracking` after a click means that the document recorded no interaction, as with a click inside an iframe: the run exits with code `0` unless it has a budget.
536
545
 
537
546
  ## Roadmap
538
547
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "webperf-snippets",
3
- "version": "1.11.0",
3
+ "version": "1.12.0",
4
4
  "mcpName": "io.github.nucliweb/webperf-snippets",
5
5
  "description": "Run curated WebPerf Snippets headlessly via Playwright. Diagnose Core Web Vitals beyond what Lighthouse exposes.",
6
6
  "type": "module",
@@ -169,9 +169,14 @@
169
169
  observer.observe({
170
170
  type: "event",
171
171
  buffered: true,
172
- durationThreshold: 16, // Only interactions > 16ms (1 frame)
172
+ durationThreshold: 16, // The lowest the browser accepts
173
173
  });
174
174
 
175
+ // With no frame after it, an interaction that paints nothing is measured to the end of its handlers,
176
+ // under the 16 ms of `event` entries. `first-input` has no threshold, so the first one is still read;
177
+ // web-vitals does the same.
178
+ observer.observe({ type: "first-input", buffered: true });
179
+
175
180
  // Log INP summary
176
181
  const logINP = () => {
177
182
  const rating = valueToRating(inpValue);
@@ -231,7 +236,7 @@
231
236
  console.log(` Presentation Delay: ${formatMs(phases.presentationDelay)}`);
232
237
 
233
238
  // Visual bar
234
- const total = inpEntry.duration;
239
+ const total = phases.inputDelay + phases.processingTime + phases.presentationDelay;
235
240
  const barWidth = 40;
236
241
  const inputBar = "▓".repeat(Math.round((phases.inputDelay / total) * barWidth));
237
242
  const processBar = "█".repeat(Math.round((phases.processingTime / total) * barWidth));
@@ -358,15 +363,26 @@
358
363
  details,
359
364
  };
360
365
  }
366
+ const value = Math.round(inpValue);
361
367
  return {
362
368
  script: "INP",
363
369
  status: "ok",
364
370
  metric: "INP",
365
- value: Math.round(inpValue),
371
+ value,
366
372
  unit: "ms",
367
373
  rating,
368
374
  thresholds: { good: 200, needsImprovement: 500 },
369
375
  details,
376
+ // Under one frame there is no paint to measure to: the value is real, and says little about the page
377
+ ...(inpValue < 16 && {
378
+ issues: [
379
+ {
380
+ code: "no-paint-measured",
381
+ severity: "info",
382
+ message: `Worst interaction: ${value} ms, under the 16 ms of a frame, so the page most likely painted nothing after it. If it should respond, check that the click or key press reaches an element that does.`,
383
+ },
384
+ ],
385
+ }),
370
386
  };
371
387
  };
372
388
 
@@ -492,7 +508,7 @@
492
508
  });
493
509
 
494
510
  console.log("%c⚡ INP Tracking Active", "font-weight: bold; font-size: 14px;");
495
- console.log(" Interactions with duration > 16ms will be tracked.");
511
+ console.log(" Interactions of 16ms or more are tracked, and the first one whatever it takes.");
496
512
  console.log(
497
513
  " Call %cgetINP()%c to see current INP value.",
498
514
  "font-family: monospace; background: #f3f4f6; padding: 2px 4px;",
@@ -64,7 +64,12 @@
64
64
  }
65
65
  // @end-shared uniqueSelector
66
66
 
67
+ // The element of an entry is null once its node has left the DOM, as with a loading spinner that
68
+ // an app removes when it renders. The entry keeps its time, size and URL.
69
+ const REMOVED = "(removed from DOM)";
70
+
67
71
  const getElementInfo = (element, entry) => {
72
+ if (!element) return entry.url ? { type: "Image", url: entry.url } : { type: "Removed element" };
68
73
  const tag = element.tagName.toLowerCase();
69
74
  if (tag === "img") return { type: "Image", url: entry.url || element.src };
70
75
  if (tag === "video") return { type: "Video poster", url: entry.url || element.poster };
@@ -72,6 +77,19 @@
72
77
  return { type: tag === "h1" || tag === "p" ? "Text block" : tag };
73
78
  };
74
79
 
80
+ // One candidate per element, or per entry when its element was removed
81
+ const candidateKey = (entry) => entry.element ?? `${entry.startTime}|${entry.size}`;
82
+
83
+ // A data: URL keeps its media type, and another URL its file name
84
+ const shortUrl = (url) => {
85
+ if (url.startsWith("data:")) return `${url.split(",")[0]},…`;
86
+ try {
87
+ const u = new URL(url);
88
+ const file = u.pathname.split("/").pop()?.split("?")[0];
89
+ return u.hostname !== location.hostname ? `${u.hostname}/…/${file}` : file || url;
90
+ } catch { return url; }
91
+ };
92
+
75
93
  const candidates = [];
76
94
 
77
95
  const logTrail = () => {
@@ -89,12 +107,13 @@
89
107
  // Current LCP element attribution
90
108
  console.log("");
91
109
  console.log("%cLCP Element:", "font-weight: bold;");
92
- console.log(` Element: ${current.selector}`, current.element);
110
+ if (current.element) console.log(` Element: ${current.selector}`, current.element);
111
+ else console.log(` Element: ${REMOVED}${current.entry.id ? ` #${current.entry.id}` : ""}`);
93
112
 
94
113
  const { type, url } = getElementInfo(current.element, current.entry);
95
114
  console.log(` Type: ${type}`);
96
- if (url) console.log(` URL: ${url}`);
97
- if (current.element.naturalWidth) {
115
+ if (url) console.log(` URL: ${shortUrl(url)}`);
116
+ if (current.element?.naturalWidth) {
98
117
  console.log(
99
118
  ` Dimensions: ${current.element.naturalWidth}×${current.element.naturalHeight}`
100
119
  );
@@ -111,14 +130,14 @@
111
130
  console.log(
112
131
  `%c ● ${index}. ${selector}`,
113
132
  `color: ${color}; font-weight: ${isCurrent ? "bold" : "normal"};`,
114
- `| ${(time / 1000).toFixed(2)}s — ${name}${isCurrent ? " ← LCP" : ""}`,
115
- element
133
+ `| ${(time / 1000).toFixed(2)}s${element ? ` — ${name}` : ""}${isCurrent ? " ← LCP" : ""}`,
134
+ ...(element ? [element] : [])
116
135
  );
117
136
  });
118
137
 
119
138
  console.log("");
120
139
  console.log(
121
- "%c✓ Each candidate highlighted with a colored dashed outline",
140
+ "%c✓ Each candidate in the DOM highlighted with a colored dashed outline",
122
141
  "color: #22c55e;"
123
142
  );
124
143
  console.groupEnd();
@@ -126,28 +145,30 @@
126
145
 
127
146
  const observer = new PerformanceObserver((list) => {
128
147
  const activationStart = getActivationStart();
129
- const seen = new Set(candidates.map((c) => c.element));
148
+ const seen = new Set(candidates.map((c) => candidateKey(c.entry)));
130
149
 
131
150
  for (const entry of list.getEntries()) {
132
- const { element } = entry;
133
- if (!element || seen.has(element)) continue;
151
+ const key = candidateKey(entry);
152
+ if (seen.has(key)) continue;
153
+ seen.add(key);
134
154
 
155
+ const { element } = entry;
135
156
  const { color, name } = PALETTE[candidates.length % PALETTE.length];
136
157
 
137
- element.style.outline = `3px dashed ${color}`;
138
- element.style.outlineOffset = "2px";
158
+ if (element) {
159
+ element.style.outline = `3px dashed ${color}`;
160
+ element.style.outlineOffset = "2px";
161
+ }
139
162
 
140
163
  candidates.push({
141
164
  index: candidates.length + 1,
142
165
  element,
143
- selector: uniqueSelector(element),
144
- color,
166
+ selector: element ? uniqueSelector(element) : REMOVED,
167
+ color: element ? color : "#9CA3AF",
145
168
  name,
146
169
  time: Math.max(0, entry.startTime - activationStart),
147
170
  entry,
148
171
  });
149
-
150
- seen.add(element);
151
172
  }
152
173
 
153
174
  logTrail();
@@ -171,37 +192,28 @@
171
192
  return { script: "LCP-Trail", status: "error", error: "No LCP entries yet" };
172
193
  }
173
194
  const trailActivationStart = getActivationStart();
174
- const seenEls = new Set();
175
- const syncCandidates = [];
195
+ const byKey = new Map();
176
196
  for (const entry of trailEntries) {
197
+ const key = candidateKey(entry);
198
+ if (byKey.has(key)) continue;
177
199
  const el = entry.element;
178
- if (!el || seenEls.has(el)) continue;
179
- seenEls.add(el);
180
- const selector = uniqueSelector(el);
181
- const time = Math.round(Math.max(0, entry.startTime - trailActivationStart));
182
200
  const { type, url } = getElementInfo(el, entry);
183
- syncCandidates.push({
184
- index: syncCandidates.length + 1,
185
- selector,
186
- time,
201
+ byKey.set(key, {
202
+ index: byKey.size + 1,
203
+ selector: el ? uniqueSelector(el) : null,
204
+ ...(el ? {} : { removed: true }),
205
+ ...(!el && entry.id ? { id: entry.id } : {}),
206
+ time: Math.round(Math.max(0, entry.startTime - trailActivationStart)),
187
207
  elementType: type,
188
- ...(url ? {
189
- url: (() => {
190
- try {
191
- const u = new URL(url);
192
- return u.hostname !== location.hostname
193
- ? `${u.hostname}/…/${u.pathname.split("/").pop()?.split("?")[0]}`
194
- : u.pathname.split("/").pop()?.split("?")[0] || url;
195
- } catch { return url; }
196
- })(),
197
- } : {}),
208
+ sizePixels: entry.size,
209
+ ...(url ? { url: shortUrl(url) } : {}),
198
210
  });
199
211
  }
200
- if (syncCandidates.length === 0) {
201
- return { script: "LCP-Trail", status: "error", error: "No LCP elements in DOM" };
202
- }
203
- const lastCandidate = syncCandidates.at(-1);
204
- const trailValue = lastCandidate.time;
212
+ const syncCandidates = [...byKey.values()];
213
+ // The LCP is the last entry, whether or not its element is still in the DOM
214
+ const lastEntry = trailEntries.at(-1);
215
+ const lastCandidate = byKey.get(candidateKey(lastEntry));
216
+ const trailValue = Math.round(Math.max(0, lastEntry.startTime - trailActivationStart));
205
217
  const trailRating = valueToRating(trailValue);
206
218
  return {
207
219
  script: "LCP-Trail",
@@ -92,6 +92,9 @@
92
92
  const tracker = createInteractionTracker();
93
93
  const observer = new PerformanceObserver((list) => tracker.add(list.getEntries()));
94
94
  observer.observe({ type: "event", durationThreshold: 0, buffered: true });
95
+ // The browser gives no `event` entries under 16 ms, whatever the threshold, and an interaction that
96
+ // paints nothing can take less. `first-input` has no threshold, so the first one is still read.
97
+ observer.observe({ type: "first-input", buffered: true });
95
98
 
96
99
  // Interactions grouped by event type: "click", "keydown", "pointerdown", etc.
97
100
  const groupByEventType = () => {
@@ -173,6 +173,10 @@
173
173
  buffered: true,
174
174
  });
175
175
 
176
+ // The browser gives no `event` entries under 16 ms, whatever the threshold, and an interaction that
177
+ // paints nothing can take less. `first-input` has no threshold, so the first one is still read.
178
+ observer.observe({ type: "first-input", buffered: true });
179
+
176
180
  // Summary function
177
181
  window.getInteractionSummary = () => {
178
182
  const allInteractions = tracker.list().map((i) => ({
@@ -217,6 +217,8 @@ A script wrapped in `void (async () => {...})()` discards its return value. Neve
217
217
 
218
218
  `details.element` is a CSS selector that matches only the LCP element, so `document.querySelector(details.element)` returns it: the id when it is unique, otherwise a path of tag, classes and `:nth-of-type()` from the nearest ancestor with a unique id, such as `section.intro:nth-of-type(2) > p.copy:nth-of-type(2)`. LCP-Subparts (`details.element`) and LCP-Trail (`finalElement` and each `selector`) build it the same way.
219
219
 
220
+ In LCP-Trail, a candidate whose element has left the DOM has `selector: null` and `removed: true`, with its `time`, `sizePixels`, `url` for an image and `id` when the element had one, and `finalElement` is `null` when the LCP element itself was removed. `value` is the time of the last entry either way.
221
+
220
222
  #### CLS
221
223
 
222
224
  Returns buffered CLS immediately and keeps tracking. Always call `getCLS()` after interactions to get an updated value.
@@ -269,6 +271,24 @@ Returns buffered CLS immediately and keeps tracking. Always call `getCLS()` afte
269
271
 
270
272
  If no interactions yet, `getINP()` returns `status: "tracking"` with a `message` and `getDataFn: "getINP"`: interact with the page and call it again.
271
273
 
274
+ An interaction that paints nothing has no frame to be measured to, so the browser measures it to the end of its handlers, often 0 ms, and gives no `event` entry under 16 ms at any `durationThreshold`. The first interaction of a page always has a `first-input` entry, which is how web-vitals reads it too, so `getINP()` returns it as `status: "ok"` with its value, and an `info` issue when the worst value is under 16 ms:
275
+
276
+ ```json
277
+ {
278
+ "script": "INP",
279
+ "status": "ok",
280
+ "metric": "INP",
281
+ "value": 0,
282
+ "unit": "ms",
283
+ "rating": "good",
284
+ "thresholds": { "good": 200, "needsImprovement": 500 },
285
+ "details": { "totalInteractions": 1, "worstEvent": "pointerdown → h1", "phases": { "inputDelay": 0, "processingTime": 0, "presentationDelay": 0 } },
286
+ "issues": [{ "code": "no-paint-measured", "severity": "info", "message": "Worst interaction: 0 ms, under the 16 ms of a frame, so the page most likely painted nothing after it. If it should respond, check that the click or key press reaches an element that does." }]
287
+ }
288
+ ```
289
+
290
+ Later interactions that short have no entry of any kind: they only count in `performance.interactionCount`, and INP does not depend on them. `tracking` after a click means that the document recorded no interaction at all, as with a click inside an iframe, which records its own; the CLI and the MCP server replace the `message` to say that a click or key press was sent.
291
+
272
292
  The phases describe the whole interaction, not one of its events. A tap fires `pointerdown`, `pointerup` and `click`, and a key press `keydown` and `keyup`, with one `interactionId`. INP, Interactions and Input-Latency-Breakdown group the entries by `interactionId`, also when they arrive in separate observer callbacks, and measure the interaction over its entries painted in the frame of the longest one: `inputDelay` runs from the first `startTime` to the first `processingStart`, `processingTime` to the last `processingEnd`, and `presentationDelay` from there to the paint (`startTime + duration`, never below zero). The interaction is reported under the event whose handler ran longest, or under the `click` or `keydown` when no handler took a millisecond more than theirs; `worstEvent`, the Interactions `type` and the Input-Latency-Breakdown event type follow that rule.
273
293
 
274
294
  `getINPDetails()` returns the 15 slowest interactions, slowest first, with the total in `count`. `target` is a CSS selector built like `details.element` of LCP, or `null` when the browser reports no target. Use it when `getINP()` shows poor INP and you need to identify patterns across multiple slow interactions. With no interactions yet it returns `status: "tracking"`, a `message`, `count: 0` and no items.
@@ -853,6 +873,8 @@ With no usage the script returns `count: 0`, no items and an `info` issue.
853
873
 
854
874
  Tracking scripts return `status: "tracking"` first. The function named by `getDataFn` returns the full result. Items are capped at 50 (20 for frames) and sorted by relevance; `count` holds the total.
855
875
 
876
+ Interactions and Input-Latency-Breakdown read `first-input` as INP does, so an interaction that paints nothing counts when it is the first one, and the later ones under 16 ms have no entry.
877
+
856
878
  #### Interactions (tracking)
857
879
 
858
880
  ```json
@@ -1202,6 +1224,12 @@ Every issue has a `code` that says what it is, so a comparison pairs the issues
1202
1224
 
1203
1225
  An issue about one thing among several, one issue per thing, also names it in `target`: the URL of a resource, a CSS selector, or a name such as an event type or a CSS property. Two issues with the same code and different targets are two issues.
1204
1226
 
1227
+ ### INP
1228
+
1229
+ | Code | Severity | When |
1230
+ |------|----------|------|
1231
+ | `no-paint-measured` | info | The worst interaction took under 16 ms, too short for a frame, so the page most likely painted nothing after it |
1232
+
1205
1233
  ### LCP-Image-Entropy
1206
1234
 
1207
1235
  | Code | Severity | When |
package/src/baseline.js CHANGED
@@ -204,8 +204,32 @@ function machineWarning(base, current) {
204
204
  return `the baseline was measured on another machine (${base.cpu}, ${base.cores} cores) than this run (${current.cpu}, ${current.cores} cores); timings can differ for that alone`;
205
205
  }
206
206
 
207
- // What differs in how both were measured. The comparison goes on: the warnings say how far to trust it.
208
- function warningsOf(base, current) {
207
+ const valueCount = (payload, metric) => measured(payload, metric).values?.length ?? 1;
208
+
209
+ // Why five runs can leave four values, when a run failed; a run that missed a metric alone does not count
210
+ function afterFailedRuns(payload) {
211
+ const failed = payload.runs?.failed ?? 0;
212
+ return failed ? ` after ${failed} failed run${failed === 1 ? "" : "s"}` : "";
213
+ }
214
+
215
+ // The metrics compared on both sides that have no statistics, one line for each set of numbers. A
216
+ // metric that one side lacks has no comparison to back, and its verdict already says "not measured".
217
+ function statisticsWarnings(base, current, metrics) {
218
+ const metricsByNumbers = new Map();
219
+ for (const [metric, { stats, verdict }] of Object.entries(metrics)) {
220
+ if (stats || verdict === "not measured") continue;
221
+ const inBase = valueCount(base, metric);
222
+ const numbers = `the baseline has ${inBase} value${inBase === 1 ? "" : "s"}${afterFailedRuns(base)}, and this run ${valueCount(current, metric)}${afterFailedRuns(current)}`;
223
+ metricsByNumbers.set(numbers, [...(metricsByNumbers.get(numbers) ?? []), metric]);
224
+ }
225
+ return [...metricsByNumbers].map(
226
+ ([numbers, names]) => `no statistics for ${names.join(", ")}: ${numbers}; they need ${RUNS_FOR_STATISTICS} a side, so measure both with ${RUNS_FOR_STATISTICS} runs or more`
227
+ );
228
+ }
229
+
230
+ // What differs in how both were measured, and what it cost. The comparison goes on: the warnings say
231
+ // how far to trust it.
232
+ function warningsOf(base, current, metrics) {
209
233
  const warnings = [];
210
234
  const b = base.conditions;
211
235
  const c = current.conditions;
@@ -221,7 +245,10 @@ function warningsOf(base, current) {
221
245
  }
222
246
  // In words for both: the CLI asks for runs with --runs, and the tools of the MCP server with runs
223
247
  if (runCount(base) < 2) warnings.push(`the baseline has 1 run, so only the thresholds apply; measure both with ${RUNS_FOR_STATISTICS} runs or more`);
224
- else if (runCount(base) !== runCount(current)) warnings.push(`the baseline has ${runCount(base)} runs, and this run ${runCount(current)}`);
248
+ else {
249
+ if (runCount(base) !== runCount(current)) warnings.push(`the baseline has ${runCount(base)} runs, and this run ${runCount(current)}`);
250
+ warnings.push(...statisticsWarnings(base, current, metrics));
251
+ }
225
252
  const machine = machineWarning(b?.machine, c?.machine);
226
253
  if (machine) warnings.push(machine);
227
254
  if (b && c && b.cliVersion !== c.cliVersion) {
@@ -235,13 +262,14 @@ function warningsOf(base, current) {
235
262
  // A page against its baseline, as the `baseline` field of its JSON. The runs of the baseline say how
236
263
  // much to trust its values, in the header of the reports.
237
264
  export function compareRun(base, current, { file, thresholds = DEFAULT_THRESHOLDS }) {
265
+ const metrics = compareMetrics(base, current, thresholds);
238
266
  return {
239
267
  file,
240
268
  url: base.url,
241
269
  runs: base.runs ? { count: base.runs.count, medianBy: base.runs.medianBy } : { count: 1 },
242
- metrics: compareMetrics(base, current, thresholds),
270
+ metrics,
243
271
  errors: compareErrors(base, current),
244
- warnings: warningsOf(base, current),
272
+ warnings: warningsOf(base, current, metrics),
245
273
  };
246
274
  }
247
275
 
package/src/bin.js CHANGED
@@ -89,6 +89,7 @@ Options:
89
89
  --verbose Show all items, even for passing checks
90
90
  --headed Show the browser window (debug)
91
91
  -h, --help Show this help
92
+ -v, --version Show the version
92
93
 
93
94
  Examples:
94
95
  npx webperf-snippets https://example.com
@@ -182,6 +183,7 @@ async function main() {
182
183
  verbose: { type: "boolean" },
183
184
  headed: { type: "boolean" },
184
185
  help: { type: "boolean", short: "h" },
186
+ version: { type: "boolean", short: "v" },
185
187
  },
186
188
  allowPositionals: true,
187
189
  });
@@ -196,6 +198,11 @@ async function main() {
196
198
  return;
197
199
  }
198
200
 
201
+ if (values.version) {
202
+ process.stdout.write(`${cliVersion}\n`);
203
+ return;
204
+ }
205
+
199
206
  const url = positionals[0];
200
207
  const matrixPath = values.matrix;
201
208
  if (url && matrixPath) fail("Use a URL or --matrix, not both");
@@ -0,0 +1,72 @@
1
+ // The reason of an abort is whatever the client sent, often a string, or nothing: a caller gets an error
2
+ const gaveUp = () => new Error("The caller gave up while it waited for its turn");
3
+
4
+ // One task at a time, in the order they arrive. A caller that gives up while it waits leaves the line
5
+ // at once, so it never runs and holds up no one. A task that fails passes the turn like any other, and
6
+ // its failure stays with its caller.
7
+ //
8
+ // A waiting caller hears how many tasks are ahead when it starts to wait, whenever the line moves, and
9
+ // every `reportEveryMs` while it does not: a client can restart its timeout on each report, and a task
10
+ // ahead can take minutes.
11
+ export function createQueue({ reportEveryMs = 5000 } = {}) {
12
+ const waiting = [];
13
+ let taken = false;
14
+
15
+ // A report is a courtesy: one that fails, as one to a client that left, changes nothing
16
+ const report = async ({ onWait }, ahead) => {
17
+ try {
18
+ await onWait?.(ahead);
19
+ } catch {
20
+ // nothing to tell, and no one to tell it to
21
+ }
22
+ };
23
+ const reportAll = () => waiting.forEach((waiter, i) => report(waiter, i + 1));
24
+
25
+ function take({ signal, onWait }) {
26
+ if (signal?.aborted) return Promise.reject(gaveUp());
27
+ if (!taken) {
28
+ taken = true;
29
+ return Promise.resolve();
30
+ }
31
+ return new Promise((resolve, reject) => {
32
+ const waiter = { onWait };
33
+ const leave = () => {
34
+ waiting.splice(waiting.indexOf(waiter), 1);
35
+ clearInterval(timer);
36
+ signal?.removeEventListener("abort", cancel);
37
+ };
38
+ const cancel = () => {
39
+ leave();
40
+ reject(gaveUp());
41
+ reportAll();
42
+ };
43
+ waiter.enter = () => {
44
+ leave();
45
+ resolve();
46
+ };
47
+ const timer = setInterval(() => report(waiter, waiting.indexOf(waiter) + 1), reportEveryMs);
48
+ // The timer alone must not keep the process alive
49
+ timer.unref?.();
50
+ signal?.addEventListener("abort", cancel, { once: true });
51
+ waiting.push(waiter);
52
+ report(waiter, waiting.length);
53
+ });
54
+ }
55
+
56
+ function pass() {
57
+ taken = waiting.length > 0;
58
+ waiting[0]?.enter();
59
+ reportAll();
60
+ }
61
+
62
+ async function run(task, options = {}) {
63
+ await take(options);
64
+ try {
65
+ return await task();
66
+ } finally {
67
+ pass();
68
+ }
69
+ }
70
+
71
+ return { run };
72
+ }
package/src/mcp/server.js CHANGED
@@ -10,6 +10,7 @@ import { RULES } from "../decision-tree.js";
10
10
  import { fetchCrux, fetchCruxHistory } from "../crux.js";
11
11
  import { runInteractions, isInputStep, settle } from "../interactions.js";
12
12
  import { createSessions, SessionError } from "./sessions.js";
13
+ import { createQueue } from "./queue.js";
13
14
  import { launchChromium, isMissingBrowser, MISSING_BROWSER } from "../browser.js";
14
15
  import { registerResources } from "./documents.js";
15
16
  import { registerPrompts } from "./prompts.js";
@@ -24,6 +25,10 @@ import { MAX_RUNS, measureRuns } from "../runs.js";
24
25
  const VIEWPORTS = Object.keys(VIEWPORT_PRESETS);
25
26
  const STAGES = 3;
26
27
  const NAV_TIMEOUT_MS = 30000;
28
+ // How long a page load can keep its turn: far beyond the load and the wait, which have their own limits
29
+ // (30 s and 10 s), because a run that reaches this is not slow. A page whose main thread never yields
30
+ // answers no evaluation, and Playwright has no timeout for one, so it would hold up the queue for ever.
31
+ const CALL_LIMIT_MS = 120000;
27
32
 
28
33
  const HttpUrl = z
29
34
  .string()
@@ -111,6 +116,7 @@ const NextSteps = z
111
116
  const DETAIL_NOTE =
112
117
  "Large results come as summaries with a runId for get_details; nextSteps, when present, is the call to make next.";
113
118
  const BASELINE_NOTE = "With baseline, it also compares the run with an earlier one, as --baseline of the CLI, and says what regressed.";
119
+ const QUEUE_NOTE = "Measurements run one at a time, so several in one turn save no time.";
114
120
 
115
121
  // A page can put instructions for the agent in what a snippet returns: an alt text, a class in a
116
122
  // selector, the URL of a script. The tools that return what a page holds say how to read it, and so
@@ -162,12 +168,44 @@ function progressFor(ctx, runs) {
162
168
  });
163
169
  }
164
170
 
171
+ // The report of a call that waits for its turn, for a client that wants progress: how many calls are
172
+ // ahead. Progress has to grow with each report, and the stages of a run start at 1, so it stays below 1.
173
+ function waitProgressFor(ctx, total) {
174
+ const progressToken = ctx.mcpReq._meta?.progressToken;
175
+ if (progressToken === undefined) return undefined;
176
+ let reports = 0;
177
+ return (ahead) => {
178
+ reports++;
179
+ const message = `Waiting for its turn: ${ahead} ${ahead === 1 ? "call" : "calls"} ahead`;
180
+ return ctx.mcpReq.notify({ method: "notifications/progress", params: { progressToken, progress: reports / (reports + 1), ...(total ? { total } : {}), message } });
181
+ };
182
+ }
183
+
184
+ const stoppedAnswering = (limitMs) => `The page did not answer within ${limitMs / 1000} seconds`;
185
+
186
+ // A page that stopped answering is closed, and the runs left of the call would only hold up the queue
187
+ class PageStoppedError extends Error {}
188
+ const endsTheCall = (err) => isMissingBrowser(err) || err instanceof PageStoppedError;
189
+
165
190
  // A page load measured once, or `count` times with the median run as the answer, as --runs of the CLI.
166
- // When the client gives up (a cancel, its timeout, or leaving), the runs left never start.
167
- async function measurePage(measure, count, ctx) {
191
+ // `measure` gets the signal of its run, which closes its Chromium. When the client gives up (a cancel,
192
+ // its timeout, or leaving), the run in progress ends and the runs left never start.
193
+ async function measurePage(measure, count, ctx, limitMs) {
168
194
  const progress = progressFor(ctx, count);
169
- if (count === 1) return measure(progress(1));
170
- const { payload, runs, errors } = await measureRuns((i) => measure(progress(i)), count, { isFatal: isMissingBrowser, signal: ctx.mcpReq.signal });
195
+ const measureRun = async (run) => {
196
+ const limit = AbortSignal.timeout(limitMs);
197
+ let measured;
198
+ try {
199
+ measured = await measure(progress(run), AbortSignal.any([ctx.mcpReq.signal, limit]));
200
+ } catch (err) {
201
+ if (!limit.aborted) throw err;
202
+ }
203
+ // The runner keeps a failed evaluation as a result, and closing Chromium fails every one
204
+ if (limit.aborted) throw new PageStoppedError(stoppedAnswering(limitMs));
205
+ return measured;
206
+ };
207
+ if (count === 1) return measureRun(1);
208
+ const { payload, runs, errors } = await measureRuns(measureRun, count, { isFatal: endsTheCall, signal: ctx.mcpReq.signal });
171
209
  if (!payload) throw new Error(`Every run failed: ${[...new Set(errors.map(firstLine))].join("; ")}`);
172
210
  return { ...payload, runs };
173
211
  }
@@ -224,10 +262,18 @@ function registerRuns(server, runs) {
224
262
 
225
263
  // The MCP server of the CLI: the same snippets and runner, offered as tools an agent calls by name.
226
264
  // serveStdio creates one instance per connection, so the pages it keeps open belong to that client.
227
- export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env, machine = machineOf } = {}) {
265
+ export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env, machine = machineOf, callLimitMs = CALL_LIMIT_MS, reportEveryMs } = {}) {
228
266
  const server = new McpServer({ name: "webperf-snippets", version: cliVersion }, { instructions: INSTRUCTIONS });
229
267
  const sessions = createSessions({ launchBrowser, maxPages, idleMs });
230
268
  const runs = createRuns();
269
+ // An agent sends several calls in one turn and the client runs them at once. The calls that make a page
270
+ // work take turns (run_snippet, run_workflow, open_page, interact): a load or a click competes for the
271
+ // CPU with any other, which inflates what each page reports. A call keeps its turn only while it runs,
272
+ // so it can wait only for something that ends on its own: never for a later call of the agent, and
273
+ // never for a page left open, which keeps no turn. read_results and close_page take no turn: they ask
274
+ // little of Chromium, and they are how an agent gets out of a page.
275
+ const queue = createQueue({ reportEveryMs });
276
+ const inTurn = (ctx, total, task) => queue.run(task, { signal: ctx.mcpReq.signal, onWait: waitProgressFor(ctx, total) });
231
277
  // The client left (end of stdin, or a closed transport): close the pages and Chromium
232
278
  server.server.onclose = () => sessions.closeAll();
233
279
  registerResources(server);
@@ -235,10 +281,13 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
235
281
  registerRuns(server, runs);
236
282
 
237
283
  // The machine goes in the conditions of every run, as in the JSON of the CLI, so a baseline measured
238
- // on another one warns. It is measured once, with the first run, before its Chromium starts.
284
+ // on another one warns. It is measured once, in the turn of the first run, before its Chromium starts.
239
285
  let machineOfServer;
240
286
  const conditionsOf = (conditions) => measurementConditions({ ...conditions, machine: (machineOfServer ??= machine()) });
241
287
 
288
+ const measureInTurn = (conditions, measure, count, ctx) =>
289
+ inTurn(ctx, STAGES * count, async () => ({ conditions: conditionsOf(conditions), run: await measurePage(measure, count, ctx, callLimitMs) }));
290
+
242
291
  // A run of a page load, kept with its page in the shape of the JSON of the CLI: its runId is the
243
292
  // baseline of a later run, and the resources list it
244
293
  const keepPage = (run, results, conditions) => {
@@ -292,7 +341,7 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
292
341
  {
293
342
  title: "Run a snippet",
294
343
  description:
295
- `Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
344
+ `Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${QUEUE_NOTE} ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
296
345
  inputSchema: z.object({
297
346
  url: HttpUrl,
298
347
  snippet: z.string().describe("The snippet: its name (LCP-Subparts), an alias (render-blocking) or Category/Name"),
@@ -337,8 +386,12 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
337
386
  }
338
387
 
339
388
  try {
340
- const conditions = conditionsOf({ viewport, snippet: path, waitMs });
341
- const run = await measurePage((onProgress) => runSnippets({ url, items: [{ id: path, path, source }], waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress }), count, ctx);
389
+ const { conditions, run } = await measureInTurn(
390
+ { viewport, snippet: path, waitMs },
391
+ (onProgress, signal) => runSnippets({ url, items: [{ id: path, path, source }], waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress, signal }),
392
+ count,
393
+ ctx
394
+ );
342
395
  const shaped = shapeResults(run.results.map(({ id, ...result }) => result), detail);
343
396
  const { runId, current } = keepPage(run, run.results, conditions);
344
397
  const id = run.results[0].id;
@@ -369,7 +422,7 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
369
422
  {
370
423
  title: "Run a workflow",
371
424
  description:
372
- `Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
425
+ `Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${QUEUE_NOTE} ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
373
426
  inputSchema: z.object({
374
427
  url: HttpUrl,
375
428
  workflow: z.enum(Object.keys(WORKFLOWS)).default("core-web-vitals"),
@@ -405,9 +458,9 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
405
458
  return toolError(err.message);
406
459
  }
407
460
  try {
408
- const conditions = conditionsOf({ viewport, workflow, waitMs });
409
- const run = await measurePage(
410
- (onProgress) => runMeasurement({ url, workflow: WORKFLOWS[workflow], rules: RULES, waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress }),
461
+ const { conditions, run } = await measureInTurn(
462
+ { viewport, workflow, waitMs },
463
+ (onProgress, signal) => runMeasurement({ url, workflow: WORKFLOWS[workflow], rules: RULES, waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress, signal }),
411
464
  count,
412
465
  ctx
413
466
  );
@@ -537,27 +590,33 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
537
590
  items.push({ id: path, path, source });
538
591
  }
539
592
 
540
- let session;
541
- try {
542
- session = await sessions.create({ viewport: VIEWPORT_PRESETS[viewport], signal: ctx.mcpReq.signal });
543
- } catch (err) {
544
- if (isMissingBrowser(err)) return toolError(MISSING_BROWSER);
545
- throw err;
546
- }
547
- try {
548
- const navStart = Date.now();
549
- await session.page.goto(url, { waitUntil: "load", timeout: NAV_TIMEOUT_MS });
550
- if (waitMs > 0) await session.page.waitForTimeout(waitMs);
551
- const navMs = Date.now() - navStart;
552
- session.tracking = await evaluateItems(session.page, items);
553
- session.stopListening();
554
- const tracking = session.tracking.map(({ id, status }) => ({ snippet: id, status }));
555
- return toolResult({ sessionId: session.id, url, viewport, navMs, tracking });
556
- } catch (err) {
557
- await sessions.close(session.id);
558
- if (/page\.goto/.test(err.message)) return toolError(`Could not load ${url}: ${firstLine(err.message)}`);
559
- return toolError(`Opening the page failed: ${firstLine(err.message)}`);
560
- }
593
+ return inTurn(ctx, undefined, async () => {
594
+ const limit = AbortSignal.timeout(callLimitMs);
595
+ let session;
596
+ try {
597
+ session = await sessions.create({ viewport: VIEWPORT_PRESETS[viewport], signal: AbortSignal.any([ctx.mcpReq.signal, limit]) });
598
+ } catch (err) {
599
+ if (isMissingBrowser(err)) return toolError(MISSING_BROWSER);
600
+ throw err;
601
+ }
602
+ try {
603
+ const navStart = Date.now();
604
+ await session.page.goto(url, { waitUntil: "load", timeout: NAV_TIMEOUT_MS });
605
+ if (waitMs > 0) await session.page.waitForTimeout(waitMs);
606
+ const navMs = Date.now() - navStart;
607
+ session.tracking = await evaluateItems(session.page, items);
608
+ // evaluateItems keeps a failed evaluation as a result, and closing the page fails every one
609
+ if (limit.aborted) return toolError(`Opening the page failed: ${stoppedAnswering(callLimitMs)}`);
610
+ session.stopListening();
611
+ const tracking = session.tracking.map(({ id, status }) => ({ snippet: id, status }));
612
+ return toolResult({ sessionId: session.id, url, viewport, navMs, tracking });
613
+ } catch (err) {
614
+ await sessions.close(session.id);
615
+ if (limit.aborted) return toolError(`Opening the page failed: ${stoppedAnswering(callLimitMs)}`);
616
+ if (/page\.goto/.test(err.message)) return toolError(`Could not load ${url}: ${firstLine(err.message)}`);
617
+ return toolError(`Opening the page failed: ${firstLine(err.message)}`);
618
+ }
619
+ });
561
620
  })
562
621
  );
563
622
 
@@ -575,16 +634,28 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
575
634
  // A click can submit a form or change data on the site, so this tool is not read-only
576
635
  annotations: { readOnlyHint: false, openWorldHint: true },
577
636
  },
578
- withSession(async ({ sessionId, steps }) => {
579
- const session = sessions.get(sessionId);
580
- try {
581
- await runInteractions(session.page, steps);
582
- } catch (err) {
583
- return toolError(`A step failed: ${firstLine(err.message)}`);
584
- }
585
- session.hadInput ||= steps.some(isInputStep);
586
- await settle(session.page);
587
- return toolResult({ sessionId, steps: steps.length, url: session.page.url() });
637
+ withSession(async ({ sessionId, steps }, ctx) => {
638
+ // A page that is not open answers now, not after a turn it does not need
639
+ sessions.get(sessionId);
640
+ return inTurn(ctx, undefined, async () => {
641
+ const session = sessions.get(sessionId);
642
+ // A wait step is time the agent asked for. Closing the page fails whatever it stopped answering.
643
+ const limitMs = callLimitMs + steps.reduce((sum, step) => sum + (step.ms ?? 0), 0);
644
+ const limit = AbortSignal.timeout(limitMs);
645
+ const closePage = () => sessions.close(sessionId);
646
+ limit.addEventListener("abort", closePage, { once: true });
647
+ try {
648
+ await runInteractions(session.page, steps);
649
+ session.hadInput ||= steps.some(isInputStep);
650
+ await settle(session.page);
651
+ if (!limit.aborted) return toolResult({ sessionId, steps: steps.length, url: session.page.url() });
652
+ } catch (err) {
653
+ if (!limit.aborted) return toolError(`A step failed: ${firstLine(err.message)}`);
654
+ } finally {
655
+ limit.removeEventListener("abort", closePage);
656
+ }
657
+ return toolError(`${stoppedAnswering(limitMs)}, and it was closed. Open it again with open_page.`);
658
+ });
588
659
  })
589
660
  );
590
661
 
package/src/runner.js CHANGED
@@ -21,6 +21,12 @@ const DEFAULT_NAV_TIMEOUT = 30000;
21
21
  const LATE_ENTRIES_MS = 2000;
22
22
  const POLL_MS = 100;
23
23
 
24
+ // A snippet still tracking after that wait has seen no interaction at all, and the browser agrees: a
25
+ // click that paints nothing is recorded too. The message of the snippet (click on the page, then call
26
+ // the function again) would ask for what the run has just done.
27
+ const NOTHING_RECORDED =
28
+ "A click or key press was sent, and this document recorded no interaction. Check that it reaches an element of this document: an interaction inside an iframe is recorded by the iframe.";
29
+
24
30
  // Snippets are IIFEs. Playwright evaluates a string as an expression, so we
25
31
  // trim trailing semicolons to keep the IIFE call as a single expression and
26
32
  // recover its return value in Node.
@@ -83,13 +89,26 @@ export async function collectTrackingData(page, result, hasInput) {
83
89
  await page.waitForTimeout(POLL_MS);
84
90
  data = await read();
85
91
  }
86
- if (data && typeof data === "object") return { id: result.id, ...data };
92
+ if (data && typeof data === "object") {
93
+ const sentAndMissed = hasInput && data.status === "tracking";
94
+ return { id: result.id, ...data, ...(sentAndMissed && { message: NOTHING_RECORDED }) };
95
+ }
87
96
  return { id: result.id, status: "error", error: `${result.getDataFn}() did not return an object` };
88
97
  } catch (err) {
89
98
  return { id: result.id, status: "error", error: `${result.getDataFn}() failed: ${err.message}` };
90
99
  }
91
100
  }
92
101
 
102
+ // An aborted signal closes Chromium, which fails whatever the run is waiting for, a page that stopped
103
+ // answering included: Playwright has no timeout for an evaluation. Returns what stops listening.
104
+ function closeOnAbort(browser, signal) {
105
+ if (!signal) return () => {};
106
+ const close = () => browser.close().catch(() => {});
107
+ if (signal.aborted) close();
108
+ signal.addEventListener("abort", close, { once: true });
109
+ return () => signal.removeEventListener("abort", close);
110
+ }
111
+
93
112
  export async function runSnippets({
94
113
  url,
95
114
  items,
@@ -106,9 +125,11 @@ export async function runSnippets({
106
125
  ownDomains,
107
126
  // Called at each of the three stages of a run, for callers that report progress (the MCP server)
108
127
  onProgress = () => {},
128
+ signal,
109
129
  }) {
110
130
  await onProgress(1, "Opening Chromium");
111
131
  const browser = await launchChromium({ headless });
132
+ const stopListening = closeOnAbort(browser, signal);
112
133
  try {
113
134
  const context = await browser.newContext({ viewport, storageState });
114
135
  const page = await context.newPage();
@@ -126,6 +147,7 @@ export async function runSnippets({
126
147
  await onProgress(3, "Snippets evaluated");
127
148
  return { url, navMs, results, pageErrors, throttling: throttlingOf({ cpu, network }) };
128
149
  } finally {
150
+ stopListening();
129
151
  await browser.close();
130
152
  }
131
153
  }
@@ -146,9 +168,11 @@ export async function runMeasurement({
146
168
  network,
147
169
  ownDomains,
148
170
  onProgress = () => {},
171
+ signal,
149
172
  }) {
150
173
  await onProgress(1, "Opening Chromium");
151
174
  const browser = await launchChromium({ headless });
175
+ const stopListening = closeOnAbort(browser, signal);
152
176
  try {
153
177
  const context = await browser.newContext({ viewport, storageState });
154
178
  const page = await context.newPage();
@@ -212,6 +236,7 @@ export async function runMeasurement({
212
236
  await onProgress(3, "Workflow and follow-ups evaluated");
213
237
  return { url, navMs, results: [...initialResults, ...followUpResults], pageErrors, throttling: throttlingOf({ cpu, network }) };
214
238
  } finally {
239
+ stopListening();
215
240
  await browser.close();
216
241
  }
217
242
  }
@@ -11,6 +11,12 @@ const SNIPPET_ALIASES = {
11
11
  CLS: "CoreWebVitals/CLS",
12
12
  INP: "CoreWebVitals/INP",
13
13
  "LCP-Subparts": "CoreWebVitals/LCP-Subparts",
14
+ // The full names of the metrics, in any case: the words someone types who does not know the acronym
15
+ "Largest-Contentful-Paint": "CoreWebVitals/LCP",
16
+ "Cumulative-Layout-Shift": "CoreWebVitals/CLS",
17
+ "Interaction-to-Next-Paint": "CoreWebVitals/INP",
18
+ "First-Contentful-Paint": "Loading/FCP",
19
+ "Time-to-First-Byte": "Loading/TTFB",
14
20
  fonts: "Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold",
15
21
  "Fonts-Preloaded-Loaded-and-used-above-the-fold":
16
22
  "Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold",
@@ -65,34 +71,99 @@ export function listSnippets() {
65
71
  return cache;
66
72
  }
67
73
 
74
+ // The edits between two strings. Swapping two neighbours counts as one: LPC is one edit from LCP.
68
75
  function distance(a, b) {
69
- const row = Array.from({ length: b.length + 1 }, (_, i) => i);
76
+ let twoBack = [];
77
+ let previous = Array.from({ length: b.length + 1 }, (_, i) => i);
70
78
  for (let i = 1; i <= a.length; i++) {
71
- let previous = row[0];
72
- row[0] = i;
79
+ const row = [i];
73
80
  for (let j = 1; j <= b.length; j++) {
74
- const current = row[j];
75
- row[j] = Math.min(row[j] + 1, row[j - 1] + 1, previous + (a[i - 1] === b[j - 1] ? 0 : 1));
76
- previous = current;
81
+ row[j] = Math.min(previous[j] + 1, row[j - 1] + 1, previous[j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1));
82
+ if (i > 1 && j > 1 && a[i - 1] === b[j - 2] && a[i - 2] === b[j - 1]) row[j] = Math.min(row[j], twoBack[j - 2] + 1);
77
83
  }
84
+ twoBack = previous;
85
+ previous = row;
78
86
  }
79
- return row[b.length];
87
+ return previous[b.length];
80
88
  }
81
89
 
82
- // Turns an alias, a bare snippet name or a Category/Name path into a Category/Name path.
83
- // Only a listed snippet resolves, so a name can never point at a file outside the snippets.
84
- export function resolveSnippetName(name) {
90
+ // The words of a name, without their endings: LongTask and long-tasks give the same two.
91
+ function wordsOf(name) {
92
+ return name
93
+ .replace(/([a-z])([A-Z])|([A-Z])([A-Z][a-z])/g, "$1$3-$2$4")
94
+ .toLowerCase()
95
+ .split(/[^a-z0-9]+/)
96
+ .filter(Boolean)
97
+ .map((word) => (word.length > 4 ? word.replace(/(ing|ed)?s?$/, "") : word));
98
+ }
99
+
100
+ // The words of a guess that say nothing of what a snippet looks at. They rank, but do not make a match.
101
+ const GENERIC_WORDS = new Set(wordsOf("and the of with by find audit analysis analyzer info detection validation performance"));
102
+
103
+ // Equal, inside a longer word of the name (font in webfont), or a typo apart. Words of three letters only
104
+ // count when equal, and a typo needs five letters: size and side are one edit apart.
105
+ function isSameWord(guessed, word) {
106
+ if (guessed === word) return true;
107
+ if (Math.min(guessed.length, word.length) < 4) return false;
108
+ return word.includes(guessed) || (Math.max(guessed.length, word.length) > 4 && distance(guessed, word) === 1);
109
+ }
110
+
111
+ // How close a guess is to a name or an alias of a snippet, from 0 to 1: the share of its words the name
112
+ // has, or how alike they are spelled, which catches what splits into other words, such as LCP-Sub-Parts.
113
+ // Under MIN_SPELLING the likeness is chance: any two short names share letters.
114
+ const MIN_SPELLING = 0.65;
115
+ const MIN_SCORE = 0.5;
116
+ const NEAR_THE_FIRST = 0.75;
117
+ const MAX_SUGGESTIONS = 5;
118
+
119
+ function closeness(guess, name) {
120
+ const [a, b] = [guess.toLowerCase(), name.toLowerCase()];
121
+ const spelling = 1 - distance(a, b) / Math.max(a.length, b.length);
122
+ const words = wordsOf(name);
123
+ const guessed = wordsOf(guess);
124
+ const found = guessed.filter((word) => words.some((other) => isSameWord(word, other)));
125
+ const shared = found.some((word) => !GENERIC_WORDS.has(word)) ? found.length / guessed.length : 0;
126
+ return Math.max(spelling >= MIN_SPELLING ? spelling : 0, shared);
127
+ }
128
+
129
+ // The snippets a guess may have meant, the closest first, and on a tie the shortest name, which says the
130
+ // least else: Script-Loading before Inline-Script-Info-and-Size.
131
+ function suggestionsFor(guess) {
132
+ const ranked = listSnippets()
133
+ .map((path) => {
134
+ const name = path.split("/")[1];
135
+ const score = Math.max(...[name, ...aliasesOf(path)].map((n) => closeness(guess, n)));
136
+ return { path, score, length: wordsOf(name).length };
137
+ })
138
+ .filter((snippet) => snippet.score >= MIN_SCORE)
139
+ .sort((a, b) => b.score - a.score || a.length - b.length);
140
+ return ranked
141
+ .filter((snippet) => snippet.score >= ranked[0].score * NEAR_THE_FIRST)
142
+ .slice(0, MAX_SUGGESTIONS)
143
+ .map((snippet) => snippet.path);
144
+ }
145
+
146
+ // An alias as written, then a snippet name in any case, then an alias in any case: TTFB is the snippet,
147
+ // and ttfb the alias of its sub-parts.
148
+ function findSnippet(name) {
85
149
  if (Object.hasOwn(SNIPPET_ALIASES, name)) return SNIPPET_ALIASES[name];
86
- const all = listSnippets();
87
150
  const wanted = name.toLowerCase();
88
- const found = all.find((path) => path.toLowerCase() === wanted || path.split("/")[1].toLowerCase() === wanted);
151
+ const snippet = listSnippets().find((path) => path.toLowerCase() === wanted || path.split("/")[1].toLowerCase() === wanted);
152
+ if (snippet) return snippet;
153
+ const alias = Object.keys(SNIPPET_ALIASES).find((key) => key.toLowerCase() === wanted);
154
+ return alias && SNIPPET_ALIASES[alias];
155
+ }
156
+
157
+ // Turns an alias, a bare snippet name or a Category/Name path into a Category/Name path. The name decides
158
+ // and not the category: no two snippets share a name, so a wrong category can only mean that snippet.
159
+ // Only a listed snippet resolves, so a name can never point at a file outside the snippets.
160
+ export function resolveSnippetName(name) {
161
+ const [category, bare, ...rest] = name.split("/");
162
+ const inCategory = bare && rest.length === 0 && CATEGORIES.some((c) => c.toLowerCase() === category.toLowerCase());
163
+ const found = findSnippet(name) ?? (inCategory ? findSnippet(bare) : undefined);
89
164
  if (found) return found;
90
165
 
91
- const bare = wanted.split("/").pop();
92
- const closest = all
93
- .map((path) => ({ path, score: distance(bare, path.split("/")[1].toLowerCase()) }))
94
- .sort((a, b) => a.score - b.score)
95
- .slice(0, 3)
96
- .map((c) => c.path);
97
- throw new Error(`Unknown snippet "${name}". Closest: ${closest.join(", ")}. Run with a snippet name such as Compression-Audit, or a Category/Name path.`);
166
+ const suggestions = suggestionsFor(name.split("/").pop());
167
+ const answer = suggestions.length > 0 ? `Closest: ${suggestions.join(", ")}.` : "No snippet has a close name.";
168
+ throw new Error(`Unknown snippet "${name}". ${answer} Every name is in the list_snippets tool of the MCP server and in https://webperf-snippets.net/llms.txt.`);
98
169
  }