webperf-snippets 1.11.0 → 1.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -4
- package/package.json +1 -1
- package/snippets/CoreWebVitals/INP.js +20 -4
- package/snippets/CoreWebVitals/LCP-Trail.js +52 -40
- package/snippets/Interaction/Input-Latency-Breakdown.js +3 -0
- package/snippets/Interaction/Interactions.js +4 -0
- package/snippets/SCHEMA.md +28 -0
- package/src/baseline.js +33 -5
- package/src/bin.js +7 -0
- package/src/mcp/queue.js +72 -0
- package/src/mcp/server.js +115 -44
- package/src/runner.js +26 -1
- package/src/snippet-names.js +90 -19
package/README.md
CHANGED
|
@@ -73,7 +73,7 @@ npx webperf-snippets https://example.com --json
|
|
|
73
73
|
The JSON output records how the page was measured in `conditions`, so a later run can tell whether it measured the same way: the viewport, the workflow (or the path of the snippet with `--snippet`), the wait, the version of the CLI and the machine: its CPU model, cores, architecture and memory, as the operating system reports them, and a benchmark index, a short CPU test in Node before Chromium starts, which only informs, since on the same machine it moves by up to 40 % with the load.
|
|
74
74
|
|
|
75
75
|
```json
|
|
76
|
-
"conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.
|
|
76
|
+
"conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.12.0", "machine": { "cpu": "Apple M4", "cores": 10, "arch": "arm64", "memoryGB": 16, "benchmarkIndex": 2371 } }
|
|
77
77
|
```
|
|
78
78
|
|
|
79
79
|
Single snippet:
|
|
@@ -171,7 +171,7 @@ Without `--report-to` the CLI makes no external calls. With it, the results are
|
|
|
171
171
|
"timestamp": "2026-05-06T10:00:00.000Z",
|
|
172
172
|
"navMs": 1240,
|
|
173
173
|
"results": [],
|
|
174
|
-
"meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.
|
|
174
|
+
"meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.12.0" }
|
|
175
175
|
}
|
|
176
176
|
```
|
|
177
177
|
|
|
@@ -297,7 +297,7 @@ The report adds the comparison after the usual one, and `--markdown` a table for
|
|
|
297
297
|
}
|
|
298
298
|
```
|
|
299
299
|
|
|
300
|
-
When both runs were measured differently, with another workflow or snippet, viewport, wait, throttling, number of runs or version of the CLI, on another CPU model or with other cores, or on another path or query, the CLI compares anyway and warns on stderr, in the reports and in `warnings`: whoever reads the pull request comment does not see the stderr of the CI. Another host does not warn, since it is the review app of a pull request against production. A baseline of one run warns that only the thresholds apply
|
|
300
|
+
When both runs were measured differently, with another workflow or snippet, viewport, wait, throttling, number of runs or version of the CLI, on another CPU model or with other cores, or on another path or query, the CLI compares anyway and warns on stderr, in the reports and in `warnings`: whoever reads the pull request comment does not see the stderr of the CI. Another host does not warn, since it is the review app of a pull request against production. A baseline of one run warns that only the thresholds apply, and any other metric compared without statistics gets a line with the values left on each side, as in `no statistics for LCP: the baseline has 5 values, and this run 4 after 1 failed run; they need 5 a side, so measure both with 5 runs or more`.
|
|
301
301
|
|
|
302
302
|
The comparison only informs, and the exit code is the same as without it, unless `--fail-on-regression`, which exits with `1` when a metric regressed or an error is new, and names them on stderr. A baseline that does not exist or is not an output of the CLI, a baseline of a page for a matrix or of a matrix for a page, or a `--threshold` the CLI does not understand exits with code `2`, before the page loads.
|
|
303
303
|
|
|
@@ -351,6 +351,7 @@ Each budget compares the value measured, or the median with `--runs`, and fails
|
|
|
351
351
|
| `--verbose` | Show all items, including passing checks. |
|
|
352
352
|
| `--headed` | Show the browser window (debug). |
|
|
353
353
|
| `-h, --help` | Show help. |
|
|
354
|
+
| `-v, --version` | Show the version. |
|
|
354
355
|
|
|
355
356
|
### Snippet aliases
|
|
356
357
|
|
|
@@ -360,6 +361,11 @@ Each budget compares the value measured, or the median with `--runs`, and fails
|
|
|
360
361
|
| `CLS` | CoreWebVitals/CLS |
|
|
361
362
|
| `INP` | CoreWebVitals/INP |
|
|
362
363
|
| `LCP-Subparts` | CoreWebVitals/LCP-Subparts |
|
|
364
|
+
| `Largest-Contentful-Paint` | CoreWebVitals/LCP |
|
|
365
|
+
| `Cumulative-Layout-Shift` | CoreWebVitals/CLS |
|
|
366
|
+
| `Interaction-to-Next-Paint` | CoreWebVitals/INP |
|
|
367
|
+
| `First-Contentful-Paint` | Loading/FCP |
|
|
368
|
+
| `Time-to-First-Byte` | Loading/TTFB |
|
|
363
369
|
| `fonts` | Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold |
|
|
364
370
|
| `render-blocking` | Loading/Find-render-blocking-resources |
|
|
365
371
|
| `resource-hints` | Loading/Resource-Hints-Validation |
|
|
@@ -411,9 +417,11 @@ It needs the Chromium of Playwright (`npx playwright install chromium`); without
|
|
|
411
417
|
|
|
412
418
|
Each run is a page load plus `waitMs`. With the default wait of 3 seconds, five runs of a workflow take about 20 seconds on a local page, and about a minute on a page that takes a few seconds to load. When the client asks for progress, the server reports the three stages of each run, numbered across the runs. A client sets its own limit on a tool call: the MCP SDK stops waiting after 60 seconds unless it is given a longer `timeout` or `resetTimeoutOnProgress`, and in Claude Code `MCP_TOOL_TIMEOUT` sets the limit. When the client gives up, the run in progress ends and no other starts.
|
|
413
419
|
|
|
420
|
+
An agent often sends several calls in one turn. The calls of a connection that make a page work (`run_snippet`, `run_workflow`, `open_page`, `interact`) run one at a time, in the order they arrive, so they do not compete for the CPU and the turn saves no time. A call keeps its turn only while it runs, so a page left open holds up nothing, and `read_results`, `close_page` and the tools that load no page answer at once. Waiting counts towards the limit of the client; when it asks for progress, a waiting call reports how many calls are ahead, when the queue moves and every 5 seconds. A call the client gives up on leaves the queue and its page never loads; if it was the one measuring, its Chromium closes and the next call starts. A page that stops answering fails its call after 2 minutes per run, plus the `wait` steps of `interact`, and closes. Two clients at the same time, or a client and the CLI, have no queue in common.
|
|
421
|
+
|
|
414
422
|
`run_snippet` and `run_workflow` compare a run with an earlier one, with the verdict of `--baseline`. `baseline` takes the `runId` of an earlier run of the connection, which every run of these two tools returns, or the path of a page the CLI wrote with `--json`, absolute or relative to the working directory of the server. The answer adds a `baseline` field: the one of the JSON of the CLI, with the `runId` or the `file` it compared with, plus `regressions`, the same lines `--fail-on-regression` prints. `thresholds` takes the values of `--threshold`, such as `["LCP=15%", "CLS=0.1"]`. A baseline that cannot be read or a threshold the server does not understand is a tool error, before the page loads, and a baseline of a matrix is refused, since a tool measures one page.
|
|
415
423
|
|
|
416
|
-
With five runs or more a side, a regression needs a Mann-Whitney U p-value under 0.05 on top of its threshold, as in the CLI, and each metric adds `stats` with the p-value, the shift and its 95 % interval. With fewer, a metric compares by its threshold and, against a baseline of several runs, by the range of those runs. A single run can be off by more than a threshold on a page that did not change, so read `warnings`, and measure both sides with `runs: 5`, or `--runs 5` in the CLI, when the verdict matters. An earlier run of the connection works for a change made in the same session, and a file measured with the CLI for a comparison with `main`.
|
|
424
|
+
With five runs or more a side, a regression needs a Mann-Whitney U p-value under 0.05 on top of its threshold, as in the CLI, and each metric adds `stats` with the p-value, the shift and its 95 % interval. With fewer, a metric compares by its threshold and, against a baseline of several runs, by the range of those runs. A single run can be off by more than a threshold on a page that did not change, so read `warnings`, which also names each metric left without statistics and the values on each side after failed runs, and measure both sides with `runs: 5`, or `--runs 5` in the CLI, when the verdict matters. An earlier run of the connection works for a change made in the same session, and a file measured with the CLI for a comparison with `main`.
|
|
417
425
|
|
|
418
426
|
The server also offers resources, documents a client can attach to the context:
|
|
419
427
|
|
|
@@ -533,6 +541,7 @@ The `NPM_TOKEN` secret must be set in the repository settings with publish acces
|
|
|
533
541
|
- **CLS in headless is conservative**: layout shifts that only happen on scroll are missed unless you script the scroll.
|
|
534
542
|
- **First navigation only**: each `webperf-snippets` invocation runs one URL. SPAs need the post-route URL passed directly.
|
|
535
543
|
- **Synthetic INP ≠ field INP**: `--interact-script` measures handler latency for a single scripted event. Real INP reflects the worst interaction across all user sessions; use RUM for field data.
|
|
544
|
+
- **A click that paints nothing**: INP is under 16 ms with the `no-paint-measured` issue, as web-vitals reports it, so it rates `good` and passes `--budget-inp`. Check that the selector reaches an element that responds. `tracking` after a click means that the document recorded no interaction, as with a click inside an iframe: the run exits with code `0` unless it has a budget.
|
|
536
545
|
|
|
537
546
|
## Roadmap
|
|
538
547
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "webperf-snippets",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.12.0",
|
|
4
4
|
"mcpName": "io.github.nucliweb/webperf-snippets",
|
|
5
5
|
"description": "Run curated WebPerf Snippets headlessly via Playwright. Diagnose Core Web Vitals beyond what Lighthouse exposes.",
|
|
6
6
|
"type": "module",
|
|
@@ -169,9 +169,14 @@
|
|
|
169
169
|
observer.observe({
|
|
170
170
|
type: "event",
|
|
171
171
|
buffered: true,
|
|
172
|
-
durationThreshold: 16, //
|
|
172
|
+
durationThreshold: 16, // The lowest the browser accepts
|
|
173
173
|
});
|
|
174
174
|
|
|
175
|
+
// With no frame after it, an interaction that paints nothing is measured to the end of its handlers,
|
|
176
|
+
// under the 16 ms of `event` entries. `first-input` has no threshold, so the first one is still read;
|
|
177
|
+
// web-vitals does the same.
|
|
178
|
+
observer.observe({ type: "first-input", buffered: true });
|
|
179
|
+
|
|
175
180
|
// Log INP summary
|
|
176
181
|
const logINP = () => {
|
|
177
182
|
const rating = valueToRating(inpValue);
|
|
@@ -231,7 +236,7 @@
|
|
|
231
236
|
console.log(` Presentation Delay: ${formatMs(phases.presentationDelay)}`);
|
|
232
237
|
|
|
233
238
|
// Visual bar
|
|
234
|
-
const total =
|
|
239
|
+
const total = phases.inputDelay + phases.processingTime + phases.presentationDelay;
|
|
235
240
|
const barWidth = 40;
|
|
236
241
|
const inputBar = "▓".repeat(Math.round((phases.inputDelay / total) * barWidth));
|
|
237
242
|
const processBar = "█".repeat(Math.round((phases.processingTime / total) * barWidth));
|
|
@@ -358,15 +363,26 @@
|
|
|
358
363
|
details,
|
|
359
364
|
};
|
|
360
365
|
}
|
|
366
|
+
const value = Math.round(inpValue);
|
|
361
367
|
return {
|
|
362
368
|
script: "INP",
|
|
363
369
|
status: "ok",
|
|
364
370
|
metric: "INP",
|
|
365
|
-
value
|
|
371
|
+
value,
|
|
366
372
|
unit: "ms",
|
|
367
373
|
rating,
|
|
368
374
|
thresholds: { good: 200, needsImprovement: 500 },
|
|
369
375
|
details,
|
|
376
|
+
// Under one frame there is no paint to measure to: the value is real, and says little about the page
|
|
377
|
+
...(inpValue < 16 && {
|
|
378
|
+
issues: [
|
|
379
|
+
{
|
|
380
|
+
code: "no-paint-measured",
|
|
381
|
+
severity: "info",
|
|
382
|
+
message: `Worst interaction: ${value} ms, under the 16 ms of a frame, so the page most likely painted nothing after it. If it should respond, check that the click or key press reaches an element that does.`,
|
|
383
|
+
},
|
|
384
|
+
],
|
|
385
|
+
}),
|
|
370
386
|
};
|
|
371
387
|
};
|
|
372
388
|
|
|
@@ -492,7 +508,7 @@
|
|
|
492
508
|
});
|
|
493
509
|
|
|
494
510
|
console.log("%c⚡ INP Tracking Active", "font-weight: bold; font-size: 14px;");
|
|
495
|
-
console.log(" Interactions
|
|
511
|
+
console.log(" Interactions of 16ms or more are tracked, and the first one whatever it takes.");
|
|
496
512
|
console.log(
|
|
497
513
|
" Call %cgetINP()%c to see current INP value.",
|
|
498
514
|
"font-family: monospace; background: #f3f4f6; padding: 2px 4px;",
|
|
@@ -64,7 +64,12 @@
|
|
|
64
64
|
}
|
|
65
65
|
// @end-shared uniqueSelector
|
|
66
66
|
|
|
67
|
+
// The element of an entry is null once its node has left the DOM, as with a loading spinner that
|
|
68
|
+
// an app removes when it renders. The entry keeps its time, size and URL.
|
|
69
|
+
const REMOVED = "(removed from DOM)";
|
|
70
|
+
|
|
67
71
|
const getElementInfo = (element, entry) => {
|
|
72
|
+
if (!element) return entry.url ? { type: "Image", url: entry.url } : { type: "Removed element" };
|
|
68
73
|
const tag = element.tagName.toLowerCase();
|
|
69
74
|
if (tag === "img") return { type: "Image", url: entry.url || element.src };
|
|
70
75
|
if (tag === "video") return { type: "Video poster", url: entry.url || element.poster };
|
|
@@ -72,6 +77,19 @@
|
|
|
72
77
|
return { type: tag === "h1" || tag === "p" ? "Text block" : tag };
|
|
73
78
|
};
|
|
74
79
|
|
|
80
|
+
// One candidate per element, or per entry when its element was removed
|
|
81
|
+
const candidateKey = (entry) => entry.element ?? `${entry.startTime}|${entry.size}`;
|
|
82
|
+
|
|
83
|
+
// A data: URL keeps its media type, and another URL its file name
|
|
84
|
+
const shortUrl = (url) => {
|
|
85
|
+
if (url.startsWith("data:")) return `${url.split(",")[0]},…`;
|
|
86
|
+
try {
|
|
87
|
+
const u = new URL(url);
|
|
88
|
+
const file = u.pathname.split("/").pop()?.split("?")[0];
|
|
89
|
+
return u.hostname !== location.hostname ? `${u.hostname}/…/${file}` : file || url;
|
|
90
|
+
} catch { return url; }
|
|
91
|
+
};
|
|
92
|
+
|
|
75
93
|
const candidates = [];
|
|
76
94
|
|
|
77
95
|
const logTrail = () => {
|
|
@@ -89,12 +107,13 @@
|
|
|
89
107
|
// Current LCP element attribution
|
|
90
108
|
console.log("");
|
|
91
109
|
console.log("%cLCP Element:", "font-weight: bold;");
|
|
92
|
-
console.log(` Element: ${current.selector}`, current.element);
|
|
110
|
+
if (current.element) console.log(` Element: ${current.selector}`, current.element);
|
|
111
|
+
else console.log(` Element: ${REMOVED}${current.entry.id ? ` #${current.entry.id}` : ""}`);
|
|
93
112
|
|
|
94
113
|
const { type, url } = getElementInfo(current.element, current.entry);
|
|
95
114
|
console.log(` Type: ${type}`);
|
|
96
|
-
if (url) console.log(` URL: ${url}`);
|
|
97
|
-
if (current.element
|
|
115
|
+
if (url) console.log(` URL: ${shortUrl(url)}`);
|
|
116
|
+
if (current.element?.naturalWidth) {
|
|
98
117
|
console.log(
|
|
99
118
|
` Dimensions: ${current.element.naturalWidth}×${current.element.naturalHeight}`
|
|
100
119
|
);
|
|
@@ -111,14 +130,14 @@
|
|
|
111
130
|
console.log(
|
|
112
131
|
`%c ● ${index}. ${selector}`,
|
|
113
132
|
`color: ${color}; font-weight: ${isCurrent ? "bold" : "normal"};`,
|
|
114
|
-
`| ${(time / 1000).toFixed(2)}s — ${name}${isCurrent ? " ← LCP" : ""}`,
|
|
115
|
-
element
|
|
133
|
+
`| ${(time / 1000).toFixed(2)}s${element ? ` — ${name}` : ""}${isCurrent ? " ← LCP" : ""}`,
|
|
134
|
+
...(element ? [element] : [])
|
|
116
135
|
);
|
|
117
136
|
});
|
|
118
137
|
|
|
119
138
|
console.log("");
|
|
120
139
|
console.log(
|
|
121
|
-
"%c✓ Each candidate highlighted with a colored dashed outline",
|
|
140
|
+
"%c✓ Each candidate in the DOM highlighted with a colored dashed outline",
|
|
122
141
|
"color: #22c55e;"
|
|
123
142
|
);
|
|
124
143
|
console.groupEnd();
|
|
@@ -126,28 +145,30 @@
|
|
|
126
145
|
|
|
127
146
|
const observer = new PerformanceObserver((list) => {
|
|
128
147
|
const activationStart = getActivationStart();
|
|
129
|
-
const seen = new Set(candidates.map((c) => c.
|
|
148
|
+
const seen = new Set(candidates.map((c) => candidateKey(c.entry)));
|
|
130
149
|
|
|
131
150
|
for (const entry of list.getEntries()) {
|
|
132
|
-
const
|
|
133
|
-
if (
|
|
151
|
+
const key = candidateKey(entry);
|
|
152
|
+
if (seen.has(key)) continue;
|
|
153
|
+
seen.add(key);
|
|
134
154
|
|
|
155
|
+
const { element } = entry;
|
|
135
156
|
const { color, name } = PALETTE[candidates.length % PALETTE.length];
|
|
136
157
|
|
|
137
|
-
element
|
|
138
|
-
|
|
158
|
+
if (element) {
|
|
159
|
+
element.style.outline = `3px dashed ${color}`;
|
|
160
|
+
element.style.outlineOffset = "2px";
|
|
161
|
+
}
|
|
139
162
|
|
|
140
163
|
candidates.push({
|
|
141
164
|
index: candidates.length + 1,
|
|
142
165
|
element,
|
|
143
|
-
selector: uniqueSelector(element),
|
|
144
|
-
color,
|
|
166
|
+
selector: element ? uniqueSelector(element) : REMOVED,
|
|
167
|
+
color: element ? color : "#9CA3AF",
|
|
145
168
|
name,
|
|
146
169
|
time: Math.max(0, entry.startTime - activationStart),
|
|
147
170
|
entry,
|
|
148
171
|
});
|
|
149
|
-
|
|
150
|
-
seen.add(element);
|
|
151
172
|
}
|
|
152
173
|
|
|
153
174
|
logTrail();
|
|
@@ -171,37 +192,28 @@
|
|
|
171
192
|
return { script: "LCP-Trail", status: "error", error: "No LCP entries yet" };
|
|
172
193
|
}
|
|
173
194
|
const trailActivationStart = getActivationStart();
|
|
174
|
-
const
|
|
175
|
-
const syncCandidates = [];
|
|
195
|
+
const byKey = new Map();
|
|
176
196
|
for (const entry of trailEntries) {
|
|
197
|
+
const key = candidateKey(entry);
|
|
198
|
+
if (byKey.has(key)) continue;
|
|
177
199
|
const el = entry.element;
|
|
178
|
-
if (!el || seenEls.has(el)) continue;
|
|
179
|
-
seenEls.add(el);
|
|
180
|
-
const selector = uniqueSelector(el);
|
|
181
|
-
const time = Math.round(Math.max(0, entry.startTime - trailActivationStart));
|
|
182
200
|
const { type, url } = getElementInfo(el, entry);
|
|
183
|
-
|
|
184
|
-
index:
|
|
185
|
-
selector,
|
|
186
|
-
|
|
201
|
+
byKey.set(key, {
|
|
202
|
+
index: byKey.size + 1,
|
|
203
|
+
selector: el ? uniqueSelector(el) : null,
|
|
204
|
+
...(el ? {} : { removed: true }),
|
|
205
|
+
...(!el && entry.id ? { id: entry.id } : {}),
|
|
206
|
+
time: Math.round(Math.max(0, entry.startTime - trailActivationStart)),
|
|
187
207
|
elementType: type,
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
try {
|
|
191
|
-
const u = new URL(url);
|
|
192
|
-
return u.hostname !== location.hostname
|
|
193
|
-
? `${u.hostname}/…/${u.pathname.split("/").pop()?.split("?")[0]}`
|
|
194
|
-
: u.pathname.split("/").pop()?.split("?")[0] || url;
|
|
195
|
-
} catch { return url; }
|
|
196
|
-
})(),
|
|
197
|
-
} : {}),
|
|
208
|
+
sizePixels: entry.size,
|
|
209
|
+
...(url ? { url: shortUrl(url) } : {}),
|
|
198
210
|
});
|
|
199
211
|
}
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
const lastCandidate =
|
|
204
|
-
const trailValue =
|
|
212
|
+
const syncCandidates = [...byKey.values()];
|
|
213
|
+
// The LCP is the last entry, whether or not its element is still in the DOM
|
|
214
|
+
const lastEntry = trailEntries.at(-1);
|
|
215
|
+
const lastCandidate = byKey.get(candidateKey(lastEntry));
|
|
216
|
+
const trailValue = Math.round(Math.max(0, lastEntry.startTime - trailActivationStart));
|
|
205
217
|
const trailRating = valueToRating(trailValue);
|
|
206
218
|
return {
|
|
207
219
|
script: "LCP-Trail",
|
|
@@ -92,6 +92,9 @@
|
|
|
92
92
|
const tracker = createInteractionTracker();
|
|
93
93
|
const observer = new PerformanceObserver((list) => tracker.add(list.getEntries()));
|
|
94
94
|
observer.observe({ type: "event", durationThreshold: 0, buffered: true });
|
|
95
|
+
// The browser gives no `event` entries under 16 ms, whatever the threshold, and an interaction that
|
|
96
|
+
// paints nothing can take less. `first-input` has no threshold, so the first one is still read.
|
|
97
|
+
observer.observe({ type: "first-input", buffered: true });
|
|
95
98
|
|
|
96
99
|
// Interactions grouped by event type: "click", "keydown", "pointerdown", etc.
|
|
97
100
|
const groupByEventType = () => {
|
|
@@ -173,6 +173,10 @@
|
|
|
173
173
|
buffered: true,
|
|
174
174
|
});
|
|
175
175
|
|
|
176
|
+
// The browser gives no `event` entries under 16 ms, whatever the threshold, and an interaction that
|
|
177
|
+
// paints nothing can take less. `first-input` has no threshold, so the first one is still read.
|
|
178
|
+
observer.observe({ type: "first-input", buffered: true });
|
|
179
|
+
|
|
176
180
|
// Summary function
|
|
177
181
|
window.getInteractionSummary = () => {
|
|
178
182
|
const allInteractions = tracker.list().map((i) => ({
|
package/snippets/SCHEMA.md
CHANGED
|
@@ -217,6 +217,8 @@ A script wrapped in `void (async () => {...})()` discards its return value. Neve
|
|
|
217
217
|
|
|
218
218
|
`details.element` is a CSS selector that matches only the LCP element, so `document.querySelector(details.element)` returns it: the id when it is unique, otherwise a path of tag, classes and `:nth-of-type()` from the nearest ancestor with a unique id, such as `section.intro:nth-of-type(2) > p.copy:nth-of-type(2)`. LCP-Subparts (`details.element`) and LCP-Trail (`finalElement` and each `selector`) build it the same way.
|
|
219
219
|
|
|
220
|
+
In LCP-Trail, a candidate whose element has left the DOM has `selector: null` and `removed: true`, with its `time`, `sizePixels`, `url` for an image and `id` when the element had one, and `finalElement` is `null` when the LCP element itself was removed. `value` is the time of the last entry either way.
|
|
221
|
+
|
|
220
222
|
#### CLS
|
|
221
223
|
|
|
222
224
|
Returns buffered CLS immediately and keeps tracking. Always call `getCLS()` after interactions to get an updated value.
|
|
@@ -269,6 +271,24 @@ Returns buffered CLS immediately and keeps tracking. Always call `getCLS()` afte
|
|
|
269
271
|
|
|
270
272
|
If no interactions yet, `getINP()` returns `status: "tracking"` with a `message` and `getDataFn: "getINP"`: interact with the page and call it again.
|
|
271
273
|
|
|
274
|
+
An interaction that paints nothing has no frame to be measured to, so the browser measures it to the end of its handlers, often 0 ms, and gives no `event` entry under 16 ms at any `durationThreshold`. The first interaction of a page always has a `first-input` entry, which is how web-vitals reads it too, so `getINP()` returns it as `status: "ok"` with its value, and an `info` issue when the worst value is under 16 ms:
|
|
275
|
+
|
|
276
|
+
```json
|
|
277
|
+
{
|
|
278
|
+
"script": "INP",
|
|
279
|
+
"status": "ok",
|
|
280
|
+
"metric": "INP",
|
|
281
|
+
"value": 0,
|
|
282
|
+
"unit": "ms",
|
|
283
|
+
"rating": "good",
|
|
284
|
+
"thresholds": { "good": 200, "needsImprovement": 500 },
|
|
285
|
+
"details": { "totalInteractions": 1, "worstEvent": "pointerdown → h1", "phases": { "inputDelay": 0, "processingTime": 0, "presentationDelay": 0 } },
|
|
286
|
+
"issues": [{ "code": "no-paint-measured", "severity": "info", "message": "Worst interaction: 0 ms, under the 16 ms of a frame, so the page most likely painted nothing after it. If it should respond, check that the click or key press reaches an element that does." }]
|
|
287
|
+
}
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Later interactions that short have no entry of any kind: they only count in `performance.interactionCount`, and INP does not depend on them. `tracking` after a click means that the document recorded no interaction at all, as with a click inside an iframe, which records its own; the CLI and the MCP server replace the `message` to say that a click or key press was sent.
|
|
291
|
+
|
|
272
292
|
The phases describe the whole interaction, not one of its events. A tap fires `pointerdown`, `pointerup` and `click`, and a key press `keydown` and `keyup`, with one `interactionId`. INP, Interactions and Input-Latency-Breakdown group the entries by `interactionId`, also when they arrive in separate observer callbacks, and measure the interaction over its entries painted in the frame of the longest one: `inputDelay` runs from the first `startTime` to the first `processingStart`, `processingTime` to the last `processingEnd`, and `presentationDelay` from there to the paint (`startTime + duration`, never below zero). The interaction is reported under the event whose handler ran longest, or under the `click` or `keydown` when no handler took a millisecond more than theirs; `worstEvent`, the Interactions `type` and the Input-Latency-Breakdown event type follow that rule.
|
|
273
293
|
|
|
274
294
|
`getINPDetails()` returns the 15 slowest interactions, slowest first, with the total in `count`. `target` is a CSS selector built like `details.element` of LCP, or `null` when the browser reports no target. Use it when `getINP()` shows poor INP and you need to identify patterns across multiple slow interactions. With no interactions yet it returns `status: "tracking"`, a `message`, `count: 0` and no items.
|
|
@@ -853,6 +873,8 @@ With no usage the script returns `count: 0`, no items and an `info` issue.
|
|
|
853
873
|
|
|
854
874
|
Tracking scripts return `status: "tracking"` first. The function named by `getDataFn` returns the full result. Items are capped at 50 (20 for frames) and sorted by relevance; `count` holds the total.
|
|
855
875
|
|
|
876
|
+
Interactions and Input-Latency-Breakdown read `first-input` as INP does, so an interaction that paints nothing counts when it is the first one, and the later ones under 16 ms have no entry.
|
|
877
|
+
|
|
856
878
|
#### Interactions (tracking)
|
|
857
879
|
|
|
858
880
|
```json
|
|
@@ -1202,6 +1224,12 @@ Every issue has a `code` that says what it is, so a comparison pairs the issues
|
|
|
1202
1224
|
|
|
1203
1225
|
An issue about one thing among several, one issue per thing, also names it in `target`: the URL of a resource, a CSS selector, or a name such as an event type or a CSS property. Two issues with the same code and different targets are two issues.
|
|
1204
1226
|
|
|
1227
|
+
### INP
|
|
1228
|
+
|
|
1229
|
+
| Code | Severity | When |
|
|
1230
|
+
|------|----------|------|
|
|
1231
|
+
| `no-paint-measured` | info | The worst interaction took under 16 ms, too short for a frame, so the page most likely painted nothing after it |
|
|
1232
|
+
|
|
1205
1233
|
### LCP-Image-Entropy
|
|
1206
1234
|
|
|
1207
1235
|
| Code | Severity | When |
|
package/src/baseline.js
CHANGED
|
@@ -204,8 +204,32 @@ function machineWarning(base, current) {
|
|
|
204
204
|
return `the baseline was measured on another machine (${base.cpu}, ${base.cores} cores) than this run (${current.cpu}, ${current.cores} cores); timings can differ for that alone`;
|
|
205
205
|
}
|
|
206
206
|
|
|
207
|
-
|
|
208
|
-
|
|
207
|
+
const valueCount = (payload, metric) => measured(payload, metric).values?.length ?? 1;
|
|
208
|
+
|
|
209
|
+
// Why five runs can leave four values, when a run failed; a run that missed a metric alone does not count
|
|
210
|
+
function afterFailedRuns(payload) {
|
|
211
|
+
const failed = payload.runs?.failed ?? 0;
|
|
212
|
+
return failed ? ` after ${failed} failed run${failed === 1 ? "" : "s"}` : "";
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
// The metrics compared on both sides that have no statistics, one line for each set of numbers. A
|
|
216
|
+
// metric that one side lacks has no comparison to back, and its verdict already says "not measured".
|
|
217
|
+
function statisticsWarnings(base, current, metrics) {
|
|
218
|
+
const metricsByNumbers = new Map();
|
|
219
|
+
for (const [metric, { stats, verdict }] of Object.entries(metrics)) {
|
|
220
|
+
if (stats || verdict === "not measured") continue;
|
|
221
|
+
const inBase = valueCount(base, metric);
|
|
222
|
+
const numbers = `the baseline has ${inBase} value${inBase === 1 ? "" : "s"}${afterFailedRuns(base)}, and this run ${valueCount(current, metric)}${afterFailedRuns(current)}`;
|
|
223
|
+
metricsByNumbers.set(numbers, [...(metricsByNumbers.get(numbers) ?? []), metric]);
|
|
224
|
+
}
|
|
225
|
+
return [...metricsByNumbers].map(
|
|
226
|
+
([numbers, names]) => `no statistics for ${names.join(", ")}: ${numbers}; they need ${RUNS_FOR_STATISTICS} a side, so measure both with ${RUNS_FOR_STATISTICS} runs or more`
|
|
227
|
+
);
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
// What differs in how both were measured, and what it cost. The comparison goes on: the warnings say
|
|
231
|
+
// how far to trust it.
|
|
232
|
+
function warningsOf(base, current, metrics) {
|
|
209
233
|
const warnings = [];
|
|
210
234
|
const b = base.conditions;
|
|
211
235
|
const c = current.conditions;
|
|
@@ -221,7 +245,10 @@ function warningsOf(base, current) {
|
|
|
221
245
|
}
|
|
222
246
|
// In words for both: the CLI asks for runs with --runs, and the tools of the MCP server with runs
|
|
223
247
|
if (runCount(base) < 2) warnings.push(`the baseline has 1 run, so only the thresholds apply; measure both with ${RUNS_FOR_STATISTICS} runs or more`);
|
|
224
|
-
else
|
|
248
|
+
else {
|
|
249
|
+
if (runCount(base) !== runCount(current)) warnings.push(`the baseline has ${runCount(base)} runs, and this run ${runCount(current)}`);
|
|
250
|
+
warnings.push(...statisticsWarnings(base, current, metrics));
|
|
251
|
+
}
|
|
225
252
|
const machine = machineWarning(b?.machine, c?.machine);
|
|
226
253
|
if (machine) warnings.push(machine);
|
|
227
254
|
if (b && c && b.cliVersion !== c.cliVersion) {
|
|
@@ -235,13 +262,14 @@ function warningsOf(base, current) {
|
|
|
235
262
|
// A page against its baseline, as the `baseline` field of its JSON. The runs of the baseline say how
|
|
236
263
|
// much to trust its values, in the header of the reports.
|
|
237
264
|
export function compareRun(base, current, { file, thresholds = DEFAULT_THRESHOLDS }) {
|
|
265
|
+
const metrics = compareMetrics(base, current, thresholds);
|
|
238
266
|
return {
|
|
239
267
|
file,
|
|
240
268
|
url: base.url,
|
|
241
269
|
runs: base.runs ? { count: base.runs.count, medianBy: base.runs.medianBy } : { count: 1 },
|
|
242
|
-
metrics
|
|
270
|
+
metrics,
|
|
243
271
|
errors: compareErrors(base, current),
|
|
244
|
-
warnings: warningsOf(base, current),
|
|
272
|
+
warnings: warningsOf(base, current, metrics),
|
|
245
273
|
};
|
|
246
274
|
}
|
|
247
275
|
|
package/src/bin.js
CHANGED
|
@@ -89,6 +89,7 @@ Options:
|
|
|
89
89
|
--verbose Show all items, even for passing checks
|
|
90
90
|
--headed Show the browser window (debug)
|
|
91
91
|
-h, --help Show this help
|
|
92
|
+
-v, --version Show the version
|
|
92
93
|
|
|
93
94
|
Examples:
|
|
94
95
|
npx webperf-snippets https://example.com
|
|
@@ -182,6 +183,7 @@ async function main() {
|
|
|
182
183
|
verbose: { type: "boolean" },
|
|
183
184
|
headed: { type: "boolean" },
|
|
184
185
|
help: { type: "boolean", short: "h" },
|
|
186
|
+
version: { type: "boolean", short: "v" },
|
|
185
187
|
},
|
|
186
188
|
allowPositionals: true,
|
|
187
189
|
});
|
|
@@ -196,6 +198,11 @@ async function main() {
|
|
|
196
198
|
return;
|
|
197
199
|
}
|
|
198
200
|
|
|
201
|
+
if (values.version) {
|
|
202
|
+
process.stdout.write(`${cliVersion}\n`);
|
|
203
|
+
return;
|
|
204
|
+
}
|
|
205
|
+
|
|
199
206
|
const url = positionals[0];
|
|
200
207
|
const matrixPath = values.matrix;
|
|
201
208
|
if (url && matrixPath) fail("Use a URL or --matrix, not both");
|
package/src/mcp/queue.js
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
// The reason of an abort is whatever the client sent, often a string, or nothing: a caller gets an error
|
|
2
|
+
const gaveUp = () => new Error("The caller gave up while it waited for its turn");
|
|
3
|
+
|
|
4
|
+
// One task at a time, in the order they arrive. A caller that gives up while it waits leaves the line
|
|
5
|
+
// at once, so it never runs and holds up no one. A task that fails passes the turn like any other, and
|
|
6
|
+
// its failure stays with its caller.
|
|
7
|
+
//
|
|
8
|
+
// A waiting caller hears how many tasks are ahead when it starts to wait, whenever the line moves, and
|
|
9
|
+
// every `reportEveryMs` while it does not: a client can restart its timeout on each report, and a task
|
|
10
|
+
// ahead can take minutes.
|
|
11
|
+
export function createQueue({ reportEveryMs = 5000 } = {}) {
|
|
12
|
+
const waiting = [];
|
|
13
|
+
let taken = false;
|
|
14
|
+
|
|
15
|
+
// A report is a courtesy: one that fails, as one to a client that left, changes nothing
|
|
16
|
+
const report = async ({ onWait }, ahead) => {
|
|
17
|
+
try {
|
|
18
|
+
await onWait?.(ahead);
|
|
19
|
+
} catch {
|
|
20
|
+
// nothing to tell, and no one to tell it to
|
|
21
|
+
}
|
|
22
|
+
};
|
|
23
|
+
const reportAll = () => waiting.forEach((waiter, i) => report(waiter, i + 1));
|
|
24
|
+
|
|
25
|
+
function take({ signal, onWait }) {
|
|
26
|
+
if (signal?.aborted) return Promise.reject(gaveUp());
|
|
27
|
+
if (!taken) {
|
|
28
|
+
taken = true;
|
|
29
|
+
return Promise.resolve();
|
|
30
|
+
}
|
|
31
|
+
return new Promise((resolve, reject) => {
|
|
32
|
+
const waiter = { onWait };
|
|
33
|
+
const leave = () => {
|
|
34
|
+
waiting.splice(waiting.indexOf(waiter), 1);
|
|
35
|
+
clearInterval(timer);
|
|
36
|
+
signal?.removeEventListener("abort", cancel);
|
|
37
|
+
};
|
|
38
|
+
const cancel = () => {
|
|
39
|
+
leave();
|
|
40
|
+
reject(gaveUp());
|
|
41
|
+
reportAll();
|
|
42
|
+
};
|
|
43
|
+
waiter.enter = () => {
|
|
44
|
+
leave();
|
|
45
|
+
resolve();
|
|
46
|
+
};
|
|
47
|
+
const timer = setInterval(() => report(waiter, waiting.indexOf(waiter) + 1), reportEveryMs);
|
|
48
|
+
// The timer alone must not keep the process alive
|
|
49
|
+
timer.unref?.();
|
|
50
|
+
signal?.addEventListener("abort", cancel, { once: true });
|
|
51
|
+
waiting.push(waiter);
|
|
52
|
+
report(waiter, waiting.length);
|
|
53
|
+
});
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
function pass() {
|
|
57
|
+
taken = waiting.length > 0;
|
|
58
|
+
waiting[0]?.enter();
|
|
59
|
+
reportAll();
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
async function run(task, options = {}) {
|
|
63
|
+
await take(options);
|
|
64
|
+
try {
|
|
65
|
+
return await task();
|
|
66
|
+
} finally {
|
|
67
|
+
pass();
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
return { run };
|
|
72
|
+
}
|
package/src/mcp/server.js
CHANGED
|
@@ -10,6 +10,7 @@ import { RULES } from "../decision-tree.js";
|
|
|
10
10
|
import { fetchCrux, fetchCruxHistory } from "../crux.js";
|
|
11
11
|
import { runInteractions, isInputStep, settle } from "../interactions.js";
|
|
12
12
|
import { createSessions, SessionError } from "./sessions.js";
|
|
13
|
+
import { createQueue } from "./queue.js";
|
|
13
14
|
import { launchChromium, isMissingBrowser, MISSING_BROWSER } from "../browser.js";
|
|
14
15
|
import { registerResources } from "./documents.js";
|
|
15
16
|
import { registerPrompts } from "./prompts.js";
|
|
@@ -24,6 +25,10 @@ import { MAX_RUNS, measureRuns } from "../runs.js";
|
|
|
24
25
|
const VIEWPORTS = Object.keys(VIEWPORT_PRESETS);
|
|
25
26
|
const STAGES = 3;
|
|
26
27
|
const NAV_TIMEOUT_MS = 30000;
|
|
28
|
+
// How long a page load can keep its turn: far beyond the load and the wait, which have their own limits
|
|
29
|
+
// (30 s and 10 s), because a run that reaches this is not slow. A page whose main thread never yields
|
|
30
|
+
// answers no evaluation, and Playwright has no timeout for one, so it would hold up the queue for ever.
|
|
31
|
+
const CALL_LIMIT_MS = 120000;
|
|
27
32
|
|
|
28
33
|
const HttpUrl = z
|
|
29
34
|
.string()
|
|
@@ -111,6 +116,7 @@ const NextSteps = z
|
|
|
111
116
|
const DETAIL_NOTE =
|
|
112
117
|
"Large results come as summaries with a runId for get_details; nextSteps, when present, is the call to make next.";
|
|
113
118
|
const BASELINE_NOTE = "With baseline, it also compares the run with an earlier one, as --baseline of the CLI, and says what regressed.";
|
|
119
|
+
const QUEUE_NOTE = "Measurements run one at a time, so several in one turn save no time.";
|
|
114
120
|
|
|
115
121
|
// A page can put instructions for the agent in what a snippet returns: an alt text, a class in a
|
|
116
122
|
// selector, the URL of a script. The tools that return what a page holds say how to read it, and so
|
|
@@ -162,12 +168,44 @@ function progressFor(ctx, runs) {
|
|
|
162
168
|
});
|
|
163
169
|
}
|
|
164
170
|
|
|
171
|
+
// The report of a call that waits for its turn, for a client that wants progress: how many calls are
|
|
172
|
+
// ahead. Progress has to grow with each report, and the stages of a run start at 1, so it stays below 1.
|
|
173
|
+
function waitProgressFor(ctx, total) {
|
|
174
|
+
const progressToken = ctx.mcpReq._meta?.progressToken;
|
|
175
|
+
if (progressToken === undefined) return undefined;
|
|
176
|
+
let reports = 0;
|
|
177
|
+
return (ahead) => {
|
|
178
|
+
reports++;
|
|
179
|
+
const message = `Waiting for its turn: ${ahead} ${ahead === 1 ? "call" : "calls"} ahead`;
|
|
180
|
+
return ctx.mcpReq.notify({ method: "notifications/progress", params: { progressToken, progress: reports / (reports + 1), ...(total ? { total } : {}), message } });
|
|
181
|
+
};
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
const stoppedAnswering = (limitMs) => `The page did not answer within ${limitMs / 1000} seconds`;
|
|
185
|
+
|
|
186
|
+
// A page that stopped answering is closed, and the runs left of the call would only hold up the queue
|
|
187
|
+
class PageStoppedError extends Error {}
|
|
188
|
+
const endsTheCall = (err) => isMissingBrowser(err) || err instanceof PageStoppedError;
|
|
189
|
+
|
|
165
190
|
// A page load measured once, or `count` times with the median run as the answer, as --runs of the CLI.
|
|
166
|
-
//
|
|
167
|
-
|
|
191
|
+
// `measure` gets the signal of its run, which closes its Chromium. When the client gives up (a cancel,
|
|
192
|
+
// its timeout, or leaving), the run in progress ends and the runs left never start.
|
|
193
|
+
async function measurePage(measure, count, ctx, limitMs) {
|
|
168
194
|
const progress = progressFor(ctx, count);
|
|
169
|
-
|
|
170
|
-
|
|
195
|
+
const measureRun = async (run) => {
|
|
196
|
+
const limit = AbortSignal.timeout(limitMs);
|
|
197
|
+
let measured;
|
|
198
|
+
try {
|
|
199
|
+
measured = await measure(progress(run), AbortSignal.any([ctx.mcpReq.signal, limit]));
|
|
200
|
+
} catch (err) {
|
|
201
|
+
if (!limit.aborted) throw err;
|
|
202
|
+
}
|
|
203
|
+
// The runner keeps a failed evaluation as a result, and closing Chromium fails every one
|
|
204
|
+
if (limit.aborted) throw new PageStoppedError(stoppedAnswering(limitMs));
|
|
205
|
+
return measured;
|
|
206
|
+
};
|
|
207
|
+
if (count === 1) return measureRun(1);
|
|
208
|
+
const { payload, runs, errors } = await measureRuns(measureRun, count, { isFatal: endsTheCall, signal: ctx.mcpReq.signal });
|
|
171
209
|
if (!payload) throw new Error(`Every run failed: ${[...new Set(errors.map(firstLine))].join("; ")}`);
|
|
172
210
|
return { ...payload, runs };
|
|
173
211
|
}
|
|
@@ -224,10 +262,18 @@ function registerRuns(server, runs) {
|
|
|
224
262
|
|
|
225
263
|
// The MCP server of the CLI: the same snippets and runner, offered as tools an agent calls by name.
|
|
226
264
|
// serveStdio creates one instance per connection, so the pages it keeps open belong to that client.
|
|
227
|
-
export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env, machine = machineOf } = {}) {
|
|
265
|
+
export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env, machine = machineOf, callLimitMs = CALL_LIMIT_MS, reportEveryMs } = {}) {
|
|
228
266
|
const server = new McpServer({ name: "webperf-snippets", version: cliVersion }, { instructions: INSTRUCTIONS });
|
|
229
267
|
const sessions = createSessions({ launchBrowser, maxPages, idleMs });
|
|
230
268
|
const runs = createRuns();
|
|
269
|
+
// An agent sends several calls in one turn and the client runs them at once. The calls that make a page
|
|
270
|
+
// work take turns (run_snippet, run_workflow, open_page, interact): a load or a click competes for the
|
|
271
|
+
// CPU with any other, which inflates what each page reports. A call keeps its turn only while it runs,
|
|
272
|
+
// so it can wait only for something that ends on its own: never for a later call of the agent, and
|
|
273
|
+
// never for a page left open, which keeps no turn. read_results and close_page take no turn: they ask
|
|
274
|
+
// little of Chromium, and they are how an agent gets out of a page.
|
|
275
|
+
const queue = createQueue({ reportEveryMs });
|
|
276
|
+
const inTurn = (ctx, total, task) => queue.run(task, { signal: ctx.mcpReq.signal, onWait: waitProgressFor(ctx, total) });
|
|
231
277
|
// The client left (end of stdin, or a closed transport): close the pages and Chromium
|
|
232
278
|
server.server.onclose = () => sessions.closeAll();
|
|
233
279
|
registerResources(server);
|
|
@@ -235,10 +281,13 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
235
281
|
registerRuns(server, runs);
|
|
236
282
|
|
|
237
283
|
// The machine goes in the conditions of every run, as in the JSON of the CLI, so a baseline measured
|
|
238
|
-
// on another one warns. It is measured once,
|
|
284
|
+
// on another one warns. It is measured once, in the turn of the first run, before its Chromium starts.
|
|
239
285
|
let machineOfServer;
|
|
240
286
|
const conditionsOf = (conditions) => measurementConditions({ ...conditions, machine: (machineOfServer ??= machine()) });
|
|
241
287
|
|
|
288
|
+
const measureInTurn = (conditions, measure, count, ctx) =>
|
|
289
|
+
inTurn(ctx, STAGES * count, async () => ({ conditions: conditionsOf(conditions), run: await measurePage(measure, count, ctx, callLimitMs) }));
|
|
290
|
+
|
|
242
291
|
// A run of a page load, kept with its page in the shape of the JSON of the CLI: its runId is the
|
|
243
292
|
// baseline of a later run, and the resources list it
|
|
244
293
|
const keepPage = (run, results, conditions) => {
|
|
@@ -292,7 +341,7 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
292
341
|
{
|
|
293
342
|
title: "Run a snippet",
|
|
294
343
|
description:
|
|
295
|
-
`Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
|
|
344
|
+
`Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${QUEUE_NOTE} ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
|
|
296
345
|
inputSchema: z.object({
|
|
297
346
|
url: HttpUrl,
|
|
298
347
|
snippet: z.string().describe("The snippet: its name (LCP-Subparts), an alias (render-blocking) or Category/Name"),
|
|
@@ -337,8 +386,12 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
337
386
|
}
|
|
338
387
|
|
|
339
388
|
try {
|
|
340
|
-
const
|
|
341
|
-
|
|
389
|
+
const { conditions, run } = await measureInTurn(
|
|
390
|
+
{ viewport, snippet: path, waitMs },
|
|
391
|
+
(onProgress, signal) => runSnippets({ url, items: [{ id: path, path, source }], waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress, signal }),
|
|
392
|
+
count,
|
|
393
|
+
ctx
|
|
394
|
+
);
|
|
342
395
|
const shaped = shapeResults(run.results.map(({ id, ...result }) => result), detail);
|
|
343
396
|
const { runId, current } = keepPage(run, run.results, conditions);
|
|
344
397
|
const id = run.results[0].id;
|
|
@@ -369,7 +422,7 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
369
422
|
{
|
|
370
423
|
title: "Run a workflow",
|
|
371
424
|
description:
|
|
372
|
-
`Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
|
|
425
|
+
`Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${QUEUE_NOTE} ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
|
|
373
426
|
inputSchema: z.object({
|
|
374
427
|
url: HttpUrl,
|
|
375
428
|
workflow: z.enum(Object.keys(WORKFLOWS)).default("core-web-vitals"),
|
|
@@ -405,9 +458,9 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
405
458
|
return toolError(err.message);
|
|
406
459
|
}
|
|
407
460
|
try {
|
|
408
|
-
const conditions
|
|
409
|
-
|
|
410
|
-
(onProgress) => runMeasurement({ url, workflow: WORKFLOWS[workflow], rules: RULES, waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress }),
|
|
461
|
+
const { conditions, run } = await measureInTurn(
|
|
462
|
+
{ viewport, workflow, waitMs },
|
|
463
|
+
(onProgress, signal) => runMeasurement({ url, workflow: WORKFLOWS[workflow], rules: RULES, waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress, signal }),
|
|
411
464
|
count,
|
|
412
465
|
ctx
|
|
413
466
|
);
|
|
@@ -537,27 +590,33 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
537
590
|
items.push({ id: path, path, source });
|
|
538
591
|
}
|
|
539
592
|
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
session
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
593
|
+
return inTurn(ctx, undefined, async () => {
|
|
594
|
+
const limit = AbortSignal.timeout(callLimitMs);
|
|
595
|
+
let session;
|
|
596
|
+
try {
|
|
597
|
+
session = await sessions.create({ viewport: VIEWPORT_PRESETS[viewport], signal: AbortSignal.any([ctx.mcpReq.signal, limit]) });
|
|
598
|
+
} catch (err) {
|
|
599
|
+
if (isMissingBrowser(err)) return toolError(MISSING_BROWSER);
|
|
600
|
+
throw err;
|
|
601
|
+
}
|
|
602
|
+
try {
|
|
603
|
+
const navStart = Date.now();
|
|
604
|
+
await session.page.goto(url, { waitUntil: "load", timeout: NAV_TIMEOUT_MS });
|
|
605
|
+
if (waitMs > 0) await session.page.waitForTimeout(waitMs);
|
|
606
|
+
const navMs = Date.now() - navStart;
|
|
607
|
+
session.tracking = await evaluateItems(session.page, items);
|
|
608
|
+
// evaluateItems keeps a failed evaluation as a result, and closing the page fails every one
|
|
609
|
+
if (limit.aborted) return toolError(`Opening the page failed: ${stoppedAnswering(callLimitMs)}`);
|
|
610
|
+
session.stopListening();
|
|
611
|
+
const tracking = session.tracking.map(({ id, status }) => ({ snippet: id, status }));
|
|
612
|
+
return toolResult({ sessionId: session.id, url, viewport, navMs, tracking });
|
|
613
|
+
} catch (err) {
|
|
614
|
+
await sessions.close(session.id);
|
|
615
|
+
if (limit.aborted) return toolError(`Opening the page failed: ${stoppedAnswering(callLimitMs)}`);
|
|
616
|
+
if (/page\.goto/.test(err.message)) return toolError(`Could not load ${url}: ${firstLine(err.message)}`);
|
|
617
|
+
return toolError(`Opening the page failed: ${firstLine(err.message)}`);
|
|
618
|
+
}
|
|
619
|
+
});
|
|
561
620
|
})
|
|
562
621
|
);
|
|
563
622
|
|
|
@@ -575,16 +634,28 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
|
|
|
575
634
|
// A click can submit a form or change data on the site, so this tool is not read-only
|
|
576
635
|
annotations: { readOnlyHint: false, openWorldHint: true },
|
|
577
636
|
},
|
|
578
|
-
withSession(async ({ sessionId, steps }) => {
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
637
|
+
withSession(async ({ sessionId, steps }, ctx) => {
|
|
638
|
+
// A page that is not open answers now, not after a turn it does not need
|
|
639
|
+
sessions.get(sessionId);
|
|
640
|
+
return inTurn(ctx, undefined, async () => {
|
|
641
|
+
const session = sessions.get(sessionId);
|
|
642
|
+
// A wait step is time the agent asked for. Closing the page fails whatever it stopped answering.
|
|
643
|
+
const limitMs = callLimitMs + steps.reduce((sum, step) => sum + (step.ms ?? 0), 0);
|
|
644
|
+
const limit = AbortSignal.timeout(limitMs);
|
|
645
|
+
const closePage = () => sessions.close(sessionId);
|
|
646
|
+
limit.addEventListener("abort", closePage, { once: true });
|
|
647
|
+
try {
|
|
648
|
+
await runInteractions(session.page, steps);
|
|
649
|
+
session.hadInput ||= steps.some(isInputStep);
|
|
650
|
+
await settle(session.page);
|
|
651
|
+
if (!limit.aborted) return toolResult({ sessionId, steps: steps.length, url: session.page.url() });
|
|
652
|
+
} catch (err) {
|
|
653
|
+
if (!limit.aborted) return toolError(`A step failed: ${firstLine(err.message)}`);
|
|
654
|
+
} finally {
|
|
655
|
+
limit.removeEventListener("abort", closePage);
|
|
656
|
+
}
|
|
657
|
+
return toolError(`${stoppedAnswering(limitMs)}, and it was closed. Open it again with open_page.`);
|
|
658
|
+
});
|
|
588
659
|
})
|
|
589
660
|
);
|
|
590
661
|
|
package/src/runner.js
CHANGED
|
@@ -21,6 +21,12 @@ const DEFAULT_NAV_TIMEOUT = 30000;
|
|
|
21
21
|
const LATE_ENTRIES_MS = 2000;
|
|
22
22
|
const POLL_MS = 100;
|
|
23
23
|
|
|
24
|
+
// A snippet still tracking after that wait has seen no interaction at all, and the browser agrees: a
|
|
25
|
+
// click that paints nothing is recorded too. The message of the snippet (click on the page, then call
|
|
26
|
+
// the function again) would ask for what the run has just done.
|
|
27
|
+
const NOTHING_RECORDED =
|
|
28
|
+
"A click or key press was sent, and this document recorded no interaction. Check that it reaches an element of this document: an interaction inside an iframe is recorded by the iframe.";
|
|
29
|
+
|
|
24
30
|
// Snippets are IIFEs. Playwright evaluates a string as an expression, so we
|
|
25
31
|
// trim trailing semicolons to keep the IIFE call as a single expression and
|
|
26
32
|
// recover its return value in Node.
|
|
@@ -83,13 +89,26 @@ export async function collectTrackingData(page, result, hasInput) {
|
|
|
83
89
|
await page.waitForTimeout(POLL_MS);
|
|
84
90
|
data = await read();
|
|
85
91
|
}
|
|
86
|
-
if (data && typeof data === "object")
|
|
92
|
+
if (data && typeof data === "object") {
|
|
93
|
+
const sentAndMissed = hasInput && data.status === "tracking";
|
|
94
|
+
return { id: result.id, ...data, ...(sentAndMissed && { message: NOTHING_RECORDED }) };
|
|
95
|
+
}
|
|
87
96
|
return { id: result.id, status: "error", error: `${result.getDataFn}() did not return an object` };
|
|
88
97
|
} catch (err) {
|
|
89
98
|
return { id: result.id, status: "error", error: `${result.getDataFn}() failed: ${err.message}` };
|
|
90
99
|
}
|
|
91
100
|
}
|
|
92
101
|
|
|
102
|
+
// An aborted signal closes Chromium, which fails whatever the run is waiting for, a page that stopped
|
|
103
|
+
// answering included: Playwright has no timeout for an evaluation. Returns what stops listening.
|
|
104
|
+
function closeOnAbort(browser, signal) {
|
|
105
|
+
if (!signal) return () => {};
|
|
106
|
+
const close = () => browser.close().catch(() => {});
|
|
107
|
+
if (signal.aborted) close();
|
|
108
|
+
signal.addEventListener("abort", close, { once: true });
|
|
109
|
+
return () => signal.removeEventListener("abort", close);
|
|
110
|
+
}
|
|
111
|
+
|
|
93
112
|
export async function runSnippets({
|
|
94
113
|
url,
|
|
95
114
|
items,
|
|
@@ -106,9 +125,11 @@ export async function runSnippets({
|
|
|
106
125
|
ownDomains,
|
|
107
126
|
// Called at each of the three stages of a run, for callers that report progress (the MCP server)
|
|
108
127
|
onProgress = () => {},
|
|
128
|
+
signal,
|
|
109
129
|
}) {
|
|
110
130
|
await onProgress(1, "Opening Chromium");
|
|
111
131
|
const browser = await launchChromium({ headless });
|
|
132
|
+
const stopListening = closeOnAbort(browser, signal);
|
|
112
133
|
try {
|
|
113
134
|
const context = await browser.newContext({ viewport, storageState });
|
|
114
135
|
const page = await context.newPage();
|
|
@@ -126,6 +147,7 @@ export async function runSnippets({
|
|
|
126
147
|
await onProgress(3, "Snippets evaluated");
|
|
127
148
|
return { url, navMs, results, pageErrors, throttling: throttlingOf({ cpu, network }) };
|
|
128
149
|
} finally {
|
|
150
|
+
stopListening();
|
|
129
151
|
await browser.close();
|
|
130
152
|
}
|
|
131
153
|
}
|
|
@@ -146,9 +168,11 @@ export async function runMeasurement({
|
|
|
146
168
|
network,
|
|
147
169
|
ownDomains,
|
|
148
170
|
onProgress = () => {},
|
|
171
|
+
signal,
|
|
149
172
|
}) {
|
|
150
173
|
await onProgress(1, "Opening Chromium");
|
|
151
174
|
const browser = await launchChromium({ headless });
|
|
175
|
+
const stopListening = closeOnAbort(browser, signal);
|
|
152
176
|
try {
|
|
153
177
|
const context = await browser.newContext({ viewport, storageState });
|
|
154
178
|
const page = await context.newPage();
|
|
@@ -212,6 +236,7 @@ export async function runMeasurement({
|
|
|
212
236
|
await onProgress(3, "Workflow and follow-ups evaluated");
|
|
213
237
|
return { url, navMs, results: [...initialResults, ...followUpResults], pageErrors, throttling: throttlingOf({ cpu, network }) };
|
|
214
238
|
} finally {
|
|
239
|
+
stopListening();
|
|
215
240
|
await browser.close();
|
|
216
241
|
}
|
|
217
242
|
}
|
package/src/snippet-names.js
CHANGED
|
@@ -11,6 +11,12 @@ const SNIPPET_ALIASES = {
|
|
|
11
11
|
CLS: "CoreWebVitals/CLS",
|
|
12
12
|
INP: "CoreWebVitals/INP",
|
|
13
13
|
"LCP-Subparts": "CoreWebVitals/LCP-Subparts",
|
|
14
|
+
// The full names of the metrics, in any case: the words someone types who does not know the acronym
|
|
15
|
+
"Largest-Contentful-Paint": "CoreWebVitals/LCP",
|
|
16
|
+
"Cumulative-Layout-Shift": "CoreWebVitals/CLS",
|
|
17
|
+
"Interaction-to-Next-Paint": "CoreWebVitals/INP",
|
|
18
|
+
"First-Contentful-Paint": "Loading/FCP",
|
|
19
|
+
"Time-to-First-Byte": "Loading/TTFB",
|
|
14
20
|
fonts: "Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold",
|
|
15
21
|
"Fonts-Preloaded-Loaded-and-used-above-the-fold":
|
|
16
22
|
"Loading/Fonts-Preloaded-Loaded-and-used-above-the-fold",
|
|
@@ -65,34 +71,99 @@ export function listSnippets() {
|
|
|
65
71
|
return cache;
|
|
66
72
|
}
|
|
67
73
|
|
|
74
|
+
// The edits between two strings. Swapping two neighbours counts as one: LPC is one edit from LCP.
|
|
68
75
|
function distance(a, b) {
|
|
69
|
-
|
|
76
|
+
let twoBack = [];
|
|
77
|
+
let previous = Array.from({ length: b.length + 1 }, (_, i) => i);
|
|
70
78
|
for (let i = 1; i <= a.length; i++) {
|
|
71
|
-
|
|
72
|
-
row[0] = i;
|
|
79
|
+
const row = [i];
|
|
73
80
|
for (let j = 1; j <= b.length; j++) {
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
previous = current;
|
|
81
|
+
row[j] = Math.min(previous[j] + 1, row[j - 1] + 1, previous[j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1));
|
|
82
|
+
if (i > 1 && j > 1 && a[i - 1] === b[j - 2] && a[i - 2] === b[j - 1]) row[j] = Math.min(row[j], twoBack[j - 2] + 1);
|
|
77
83
|
}
|
|
84
|
+
twoBack = previous;
|
|
85
|
+
previous = row;
|
|
78
86
|
}
|
|
79
|
-
return
|
|
87
|
+
return previous[b.length];
|
|
80
88
|
}
|
|
81
89
|
|
|
82
|
-
//
|
|
83
|
-
|
|
84
|
-
|
|
90
|
+
// The words of a name, without their endings: LongTask and long-tasks give the same two.
|
|
91
|
+
function wordsOf(name) {
|
|
92
|
+
return name
|
|
93
|
+
.replace(/([a-z])([A-Z])|([A-Z])([A-Z][a-z])/g, "$1$3-$2$4")
|
|
94
|
+
.toLowerCase()
|
|
95
|
+
.split(/[^a-z0-9]+/)
|
|
96
|
+
.filter(Boolean)
|
|
97
|
+
.map((word) => (word.length > 4 ? word.replace(/(ing|ed)?s?$/, "") : word));
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
// The words of a guess that say nothing of what a snippet looks at. They rank, but do not make a match.
|
|
101
|
+
const GENERIC_WORDS = new Set(wordsOf("and the of with by find audit analysis analyzer info detection validation performance"));
|
|
102
|
+
|
|
103
|
+
// Equal, inside a longer word of the name (font in webfont), or a typo apart. Words of three letters only
|
|
104
|
+
// count when equal, and a typo needs five letters: size and side are one edit apart.
|
|
105
|
+
function isSameWord(guessed, word) {
|
|
106
|
+
if (guessed === word) return true;
|
|
107
|
+
if (Math.min(guessed.length, word.length) < 4) return false;
|
|
108
|
+
return word.includes(guessed) || (Math.max(guessed.length, word.length) > 4 && distance(guessed, word) === 1);
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
// How close a guess is to a name or an alias of a snippet, from 0 to 1: the share of its words the name
|
|
112
|
+
// has, or how alike they are spelled, which catches what splits into other words, such as LCP-Sub-Parts.
|
|
113
|
+
// Under MIN_SPELLING the likeness is chance: any two short names share letters.
|
|
114
|
+
const MIN_SPELLING = 0.65;
|
|
115
|
+
const MIN_SCORE = 0.5;
|
|
116
|
+
const NEAR_THE_FIRST = 0.75;
|
|
117
|
+
const MAX_SUGGESTIONS = 5;
|
|
118
|
+
|
|
119
|
+
function closeness(guess, name) {
|
|
120
|
+
const [a, b] = [guess.toLowerCase(), name.toLowerCase()];
|
|
121
|
+
const spelling = 1 - distance(a, b) / Math.max(a.length, b.length);
|
|
122
|
+
const words = wordsOf(name);
|
|
123
|
+
const guessed = wordsOf(guess);
|
|
124
|
+
const found = guessed.filter((word) => words.some((other) => isSameWord(word, other)));
|
|
125
|
+
const shared = found.some((word) => !GENERIC_WORDS.has(word)) ? found.length / guessed.length : 0;
|
|
126
|
+
return Math.max(spelling >= MIN_SPELLING ? spelling : 0, shared);
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
// The snippets a guess may have meant, the closest first, and on a tie the shortest name, which says the
|
|
130
|
+
// least else: Script-Loading before Inline-Script-Info-and-Size.
|
|
131
|
+
function suggestionsFor(guess) {
|
|
132
|
+
const ranked = listSnippets()
|
|
133
|
+
.map((path) => {
|
|
134
|
+
const name = path.split("/")[1];
|
|
135
|
+
const score = Math.max(...[name, ...aliasesOf(path)].map((n) => closeness(guess, n)));
|
|
136
|
+
return { path, score, length: wordsOf(name).length };
|
|
137
|
+
})
|
|
138
|
+
.filter((snippet) => snippet.score >= MIN_SCORE)
|
|
139
|
+
.sort((a, b) => b.score - a.score || a.length - b.length);
|
|
140
|
+
return ranked
|
|
141
|
+
.filter((snippet) => snippet.score >= ranked[0].score * NEAR_THE_FIRST)
|
|
142
|
+
.slice(0, MAX_SUGGESTIONS)
|
|
143
|
+
.map((snippet) => snippet.path);
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
// An alias as written, then a snippet name in any case, then an alias in any case: TTFB is the snippet,
|
|
147
|
+
// and ttfb the alias of its sub-parts.
|
|
148
|
+
function findSnippet(name) {
|
|
85
149
|
if (Object.hasOwn(SNIPPET_ALIASES, name)) return SNIPPET_ALIASES[name];
|
|
86
|
-
const all = listSnippets();
|
|
87
150
|
const wanted = name.toLowerCase();
|
|
88
|
-
const
|
|
151
|
+
const snippet = listSnippets().find((path) => path.toLowerCase() === wanted || path.split("/")[1].toLowerCase() === wanted);
|
|
152
|
+
if (snippet) return snippet;
|
|
153
|
+
const alias = Object.keys(SNIPPET_ALIASES).find((key) => key.toLowerCase() === wanted);
|
|
154
|
+
return alias && SNIPPET_ALIASES[alias];
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
// Turns an alias, a bare snippet name or a Category/Name path into a Category/Name path. The name decides
|
|
158
|
+
// and not the category: no two snippets share a name, so a wrong category can only mean that snippet.
|
|
159
|
+
// Only a listed snippet resolves, so a name can never point at a file outside the snippets.
|
|
160
|
+
export function resolveSnippetName(name) {
|
|
161
|
+
const [category, bare, ...rest] = name.split("/");
|
|
162
|
+
const inCategory = bare && rest.length === 0 && CATEGORIES.some((c) => c.toLowerCase() === category.toLowerCase());
|
|
163
|
+
const found = findSnippet(name) ?? (inCategory ? findSnippet(bare) : undefined);
|
|
89
164
|
if (found) return found;
|
|
90
165
|
|
|
91
|
-
const
|
|
92
|
-
const
|
|
93
|
-
|
|
94
|
-
.sort((a, b) => a.score - b.score)
|
|
95
|
-
.slice(0, 3)
|
|
96
|
-
.map((c) => c.path);
|
|
97
|
-
throw new Error(`Unknown snippet "${name}". Closest: ${closest.join(", ")}. Run with a snippet name such as Compression-Audit, or a Category/Name path.`);
|
|
166
|
+
const suggestions = suggestionsFor(name.split("/").pop());
|
|
167
|
+
const answer = suggestions.length > 0 ? `Closest: ${suggestions.join(", ")}.` : "No snippet has a close name.";
|
|
168
|
+
throw new Error(`Unknown snippet "${name}". ${answer} Every name is in the list_snippets tool of the MCP server and in https://webperf-snippets.net/llms.txt.`);
|
|
98
169
|
}
|