webperf-snippets 1.9.0 → 1.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -73,7 +73,7 @@ npx webperf-snippets https://example.com --json
73
73
  The JSON output records how the page was measured in `conditions`, so a later run can tell whether it measured the same way: the viewport, the workflow (or the path of the snippet with `--snippet`), the wait, the version of the CLI and the machine: its CPU model, cores, architecture and memory, as the operating system reports them, and a benchmark index, a short CPU test in Node before Chromium starts, which only informs, since on the same machine it moves by up to 40 % with the load.
74
74
 
75
75
  ```json
76
- "conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.9.0", "machine": { "cpu": "Apple M4", "cores": 10, "arch": "arm64", "memoryGB": 16, "benchmarkIndex": 2371 } }
76
+ "conditions": { "viewport": "mobile", "workflow": "core-web-vitals", "waitMs": 3000, "cliVersion": "1.11.0", "machine": { "cpu": "Apple M4", "cores": 10, "arch": "arm64", "memoryGB": 16, "benchmarkIndex": 2371 } }
77
77
  ```
78
78
 
79
79
  Single snippet:
@@ -136,6 +136,19 @@ Field data (CrUX p75, mobile)
136
136
 
137
137
  Get a key for the [Chrome UX Report API](https://developer.chrome.com/docs/crux/api) and pass it with `--crux-key` or the `CRUX_API_KEY` environment variable; prefer the variable, because a flag ends up in shell history and CI logs. The section appears in the human and markdown output and under `crux` in the `--json` output.
138
138
 
139
+ With a key, the field data also gives what CrUX knows of the LCP when it is an image: the p75 of its sub-parts, TTFB, load delay, load duration and render delay. When the measurement ran LCP-Subparts, with `--snippet LCP-Subparts` or the `core-web-vitals` workflow over 2.5 s, a table puts each one next to its lab value, with a note where one side is at least 1.5 times the other and 100 ms apart: a slower server or network for users, an image the lab finds sooner, slower connections, slower devices, or a lab slower than users. The field sub-parts describe only the loads whose LCP is an image, and the table says which share that is; when the LCP of the lab is not an image, only TTFB and render delay compare. Each p75 is taken on its own, so the sub-parts do not add up to the p75 of LCP. A line gives the navigation types of the users, by share, and points to Back-Forward-Cache when over 5 % of the visits come back without the back/forward cache. The JSON output carries them in `crux.subParts`, `crux.lcpImageShare`, `crux.roundTripTime` and `crux.navigationTypes`.
140
+
141
+ `--crux-history <weeks>` adds the trend of the field data over the last 1 to 40 weeks, from the [CrUX History API](https://developer.chrome.com/docs/crux/history-api), with the same key. Each week is the p75 of the 28 days before it, so the weeks overlap and the series is smooth. For LCP, CLS and INP the section gives the first and the last week, the change between them and its direction, `stable` within 5 %, and the weeks as a sparkline, where a gap is a week with too little data. The bars span at least the smallest change `--baseline` counts for the metric (100 ms for LCP, 50 ms for INP, 0.05 for CLS), so a series that barely moves draws almost flat:
142
+
143
+ ```
144
+ Field data trend (CrUX p75, 40 weeks, mobile, origin-level)
145
+ LCP 2.88s → 3.10s ▲ +7.7% worsening ▃▃▄▃▃▃▄▆▇██▇▆▄▂▁▂▃▃▄▅▆▆▆▆▆▆
146
+ CLS 0.2600 → 0.2100 ▼ −19.2% improving ▅▆▆▇▇█▇▆▆▅▄▃▃▃▃▃▂▂▁▂▂▂▂▂▂▂
147
+ INP 253ms → 251ms ▼ −0.8% stable ▅▅▅▄▄▄▅▅▆▆▆▅▅▄▄▄▄▅▅▅▅▄▄▄▃▃▄▄▄▅▅
148
+ ```
149
+
150
+ The JSON output carries it in `crux.history`, with the periods, the p75 of each week and the trend of each metric. Like the p75, it uses the page when it has data of its own and its origin otherwise, and says which in `scope`.
151
+
139
152
  - The population follows `--viewport`: `desktop` uses desktop data, `mobile` and `tablet` use phone data. The heading says which one.
140
153
  - When the page itself has no data, the CLI looks the origin up instead and labels the section `origin-level`, so it is not mistaken for page-level data. If the origin has none either, the output says `Field data: not available`.
141
154
  - A CrUX error (quota, network, no answer within 10 seconds) only skips the section. It never changes the exit code, and the key is never printed.
@@ -158,7 +171,7 @@ Without `--report-to` the CLI makes no external calls. With it, the results are
158
171
  "timestamp": "2026-05-06T10:00:00.000Z",
159
172
  "navMs": 1240,
160
173
  "results": [],
161
- "meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.9.0" }
174
+ "meta": { "viewport": "mobile", "waitMs": 3000, "cli_version": "1.11.0" }
162
175
  }
163
176
  ```
164
177
 
@@ -327,6 +340,7 @@ Each budget compares the value measured, or the median with `--runs`, and fails
327
340
  | `--threshold <metric>=<v>` | The threshold of a metric for `--baseline`, such as `LCP=15%`, `LCP=200ms` or `CLS=0.1`. Repeat it for more metrics. |
328
341
  | `--fail-on-regression` | Exit `1` when a metric regressed or an error is new against `--baseline`. |
329
342
  | `--crux-key <key>` | Add CrUX field data (p75) next to the measured values. Prefer `CRUX_API_KEY`. |
343
+ | `--crux-history <weeks>` | Add the trend of the field data over the last 1 to 40 weeks. Needs a CrUX key. |
330
344
  | `--report-to <url>` | POST the results to this `https` URL after the run. No external calls without it. |
331
345
  | `--api-key <key>` | Sent as a Bearer token with `--report-to`. Prefer `PERF_REVIEWS_API_KEY`. |
332
346
  | `--budget-lcp <ms>` | Exit `1` if LCP exceeds this value or was not measured. |
@@ -384,8 +398,8 @@ It needs the Chromium of Playwright (`npx playwright install chromium`); without
384
398
  | Tool | What it does |
385
399
  | --------------- | ---------------------------------------------------------------------------- |
386
400
  | `list_snippets` | Lists the snippets with their category, title, short aliases and whether they track interactions. Takes an optional `category`. |
387
- | `run_snippet` | Opens a URL in a headless Chromium, runs one snippet and returns a summary of its result. Takes `url`, `snippet` (name, alias or `Category/Name`), and optional `viewport` and `waitMs`. The snippets that record interactions run on a page opened with `open_page`. |
388
- | `run_workflow` | Runs a workflow of the CLI (`core-web-vitals`, `audit`, `loading`, `interaction`) and the follow-ups of its decision tree. Takes `url`, `workflow`, and optional `viewport` and `waitMs`. |
401
+ | `run_snippet` | Opens a URL in a headless Chromium, runs one snippet and returns a summary of its result. Takes `url`, `snippet` (name, alias or `Category/Name`), and optional `viewport`, `waitMs`, `runs`, `baseline` and `thresholds`. The snippets that record interactions run on a page opened with `open_page`. |
402
+ | `run_workflow` | Runs a workflow of the CLI (`core-web-vitals`, `audit`, `loading`, `interaction`) and the follow-ups of its decision tree. Takes `url`, `workflow`, and optional `viewport`, `waitMs`, `runs`, `baseline` and `thresholds`. |
389
403
  | `get_crux` | Returns the CrUX p75 of LCP, CLS and INP for a URL, or its origin. Takes `url` and an optional `formFactor` (`phone`, `desktop`, `tablet`). Needs `CRUX_API_KEY` in the environment of the server. |
390
404
  | `open_page` | Opens a URL in a page that stays open between calls and installs tracking snippets (`track`, such as `INP`). Returns a `sessionId`. |
391
405
  | `interact` | Runs steps on an open page: `click`, `type`, `hover`, `scroll` or `wait`. |
@@ -393,6 +407,14 @@ It needs the Chromium of Playwright (`npx playwright install chromium`); without
393
407
  | `get_details` | Returns the whole object of one snippet of a run, by the `runId` that `run_snippet`, `run_workflow` or `read_results` returned, without measuring again. Takes `runId` and `snippet`. |
394
408
  | `close_page` | Closes an open page. |
395
409
 
410
+ `run_snippet` and `run_workflow` take `runs`, from 1 to 9, as `--runs`: the page loads that many times, one after the other, and the answer is the median run plus a `runs` field, the one of the JSON, with the median, min, max, the value of each run and the agreement of each metric. A run that fails counts in `failed` and is left out; when every run fails, the answer is a tool error. The server keeps the results of the median run only, as the JSON does, so `get_details` answers from that run and names it in `medianRun`; a metric other than the one that picked it (`medianBy`) can differ there from its median in `runs`.
411
+
412
+ Each run is a page load plus `waitMs`. With the default wait of 3 seconds, five runs of a workflow take about 20 seconds on a local page, and about a minute on a page that takes a few seconds to load. When the client asks for progress, the server reports the three stages of each run, numbered across the runs. A client sets its own limit on a tool call: the MCP SDK stops waiting after 60 seconds unless it is given a longer `timeout` or `resetTimeoutOnProgress`, and in Claude Code `MCP_TOOL_TIMEOUT` sets the limit. When the client gives up, the run in progress ends and no other starts.
413
+
414
+ `run_snippet` and `run_workflow` compare a run with an earlier one, with the verdict of `--baseline`. `baseline` takes the `runId` of an earlier run of the connection, which every run of these two tools returns, or the path of a page the CLI wrote with `--json`, absolute or relative to the working directory of the server. The answer adds a `baseline` field: the one of the JSON of the CLI, with the `runId` or the `file` it compared with, plus `regressions`, the same lines `--fail-on-regression` prints. `thresholds` takes the values of `--threshold`, such as `["LCP=15%", "CLS=0.1"]`. A baseline that cannot be read or a threshold the server does not understand is a tool error, before the page loads, and a baseline of a matrix is refused, since a tool measures one page.
415
+
416
+ With five runs or more a side, a regression needs a Mann-Whitney U p-value under 0.05 on top of its threshold, as in the CLI, and each metric adds `stats` with the p-value, the shift and its 95 % interval. With fewer, a metric compares by its threshold and, against a baseline of several runs, by the range of those runs. A single run can be off by more than a threshold on a page that did not change, so read `warnings`, and measure both sides with `runs: 5`, or `--runs 5` in the CLI, when the verdict matters. An earlier run of the connection works for a change made in the same session, and a file measured with the CLI for a comparison with `main`.
417
+
396
418
  The server also offers resources, documents a client can attach to the context:
397
419
 
398
420
  | Resource | What it has |
@@ -400,6 +422,7 @@ The server also offers resources, documents a client can attach to the context:
400
422
  | `webperf://schema` | The contract every snippet result follows: base shape, statuses, how to read a result, the rules of the issue codes. |
401
423
  | `webperf://schema/{snippet}` | The fields one snippet returns, with an example, and the codes of its issues. |
402
424
  | `webperf://workflows/{category}` | The workflows and the decision tree of a category. |
425
+ | `webperf://runs/{runId}` | A run of `run_snippet` or `run_workflow` of the connection, in the shape of the JSON of the CLI. The list names each run with its snippet or workflow, its URL, its viewport, how many runs it measured when they are several, and its time. |
403
426
 
404
427
  And prompts, templates that chain the tools (in Claude Code, `/mcp__webperf__audit_lcp`): `audit_lcp`, `debug_inp` and `audit_page`, each with a `url` argument.
405
428
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "webperf-snippets",
3
- "version": "1.9.0",
3
+ "version": "1.11.0",
4
4
  "mcpName": "io.github.nucliweb/webperf-snippets",
5
5
  "description": "Run curated WebPerf Snippets headlessly via Playwright. Diagnose Core Web Vitals beyond what Lighthouse exposes.",
6
6
  "type": "module",
@@ -64,7 +64,7 @@ Every script must return an object matching this shape:
64
64
 
65
65
  // Tracking scripts (status: "tracking")
66
66
  message?: string; // Human-readable status message
67
- getDataFn?: string; // window function to call for data: evaluate_script(`${getDataFn}()`). May be a dotted path such as "loafHelpers.getData"
67
+ getDataFn?: string; // window function to call for data: evaluate_script(`() => ${getDataFn}()`). May be a dotted path such as "loafHelpers.getData"
68
68
 
69
69
  // Error info (status: "error" or "unsupported")
70
70
  error?: string;
@@ -169,7 +169,7 @@ return {
169
169
  ```
170
170
  1. evaluate_script(INP.js) → { status: "tracking", getDataFn: "getINP" }
171
171
  2. (user interacts with the page)
172
- 3. evaluate_script("getINP()") → { script: "INP", status: "ok", value: 350, rating: "needs-improvement", ... }
172
+ 3. evaluate_script("() => getINP()") → { script: "INP", status: "ok", value: 350, rating: "needs-improvement", ... }
173
173
  ```
174
174
 
175
175
  **The window function must also return a structured object** matching the same schema, including when nothing has been recorded yet:
@@ -181,7 +181,7 @@ return {
181
181
 
182
182
  ### Pattern 4: Async scripts
183
183
 
184
- Scripts that use `async/await` or `setTimeout`. The IIFE returns a Promise, which `evaluate_script` can await (Chrome DevTools `awaitPromise`).
184
+ Scripts that use `async/await` or `setTimeout`. The IIFE returns a Promise, which `evaluate_script` awaits with `format: "script"` (Chrome DevTools `awaitPromise`).
185
185
 
186
186
  Keep the existing `async () => {}` wrapper. Add a `return` statement with structured data at the end. The agent receives the resolved value.
187
187
 
@@ -1622,7 +1622,7 @@ Each issue names its video in `target`, except `no-videos`.
1622
1622
  // Read the return value: the skill scripts have no console output
1623
1623
  result = evaluate_script(scriptCode)
1624
1624
  if result.status == "ok" → use result.value, result.rating, result.details, result.items
1625
- if result.status == "tracking" → call evaluate_script(`${result.getDataFn}()`) after user interaction
1625
+ if result.status == "tracking" → call evaluate_script(`() => ${result.getDataFn}()`) after user interaction
1626
1626
  if result.status == "error" → check result.error, the browser may not have loaded the page yet
1627
1627
  if result.status == "unsupported" → browser does not support the required API (check: Chrome 107+?)
1628
1628
  ```
@@ -1637,7 +1637,7 @@ result = evaluate_script(INP_js)
1637
1637
  // 2. Wait for/trigger user interactions
1638
1638
 
1639
1639
  // 3. Collect data
1640
- data = evaluate_script("getINP()")
1640
+ data = evaluate_script("() => getINP()")
1641
1641
  // data = { status: "ok", value: 350, rating: "needs-improvement", ... }
1642
1642
  ```
1643
1643
 
package/src/baseline.js CHANGED
@@ -23,7 +23,8 @@ export const DEFAULT_THRESHOLDS = {
23
23
  // The baseline in a file, as { kind: "page" | "matrix", file, data }. Throws with what to do when
24
24
  // the file is not an output of the CLI.
25
25
  export function readBaseline(text, file) {
26
- const howTo = `save one with webperf-snippets <url> --json > ${file}`;
26
+ // Never this path as the place to save it: the file is something else, and an agent may follow the message
27
+ const howTo = "give a file saved with webperf-snippets <url> --json";
27
28
  let data;
28
29
  try {
29
30
  data = JSON.parse(text);
@@ -49,18 +50,18 @@ export function loadBaseline(path) {
49
50
  // (absolute), CLS=0.1 (CLS has only an absolute part, with no unit)
50
51
  export function parseThreshold(raw) {
51
52
  const match = /^([A-Za-z]+)=(\d+(?:\.\d+)?)(%|ms)?$/.exec(raw.trim());
52
- if (!match) throw new Error(`Invalid --threshold "${raw}": give METRIC=VALUE with its unit, such as LCP=15%, LCP=200ms or CLS=0.1`);
53
+ if (!match) throw new Error(`Invalid threshold "${raw}": give METRIC=VALUE with its unit, such as LCP=15%, LCP=200ms or CLS=0.1`);
53
54
  const [, name, number, unit] = match;
54
55
  const metric = name.toUpperCase();
55
- if (!METRICS.includes(metric)) throw new Error(`Unknown metric "${name}" in --threshold "${raw}". Choose from: ${METRICS.join(", ")}`);
56
+ if (!METRICS.includes(metric)) throw new Error(`Unknown metric "${name}" in threshold "${raw}". Choose from: ${METRICS.join(", ")}`);
56
57
  const value = Number(number);
57
58
  if (metric === "CLS") {
58
- if (unit) throw new Error(`Invalid --threshold "${raw}": CLS is a score, with no unit and no relative part, such as CLS=0.1`);
59
+ if (unit) throw new Error(`Invalid threshold "${raw}": CLS is a score, with no unit and no relative part, such as CLS=0.1`);
59
60
  return { metric, absolute: value };
60
61
  }
61
62
  if (unit === "%") return { metric, relative: value / 100 };
62
63
  if (unit === "ms") return { metric, absolute: value };
63
- throw new Error(`Invalid --threshold "${raw}": give the unit, such as ${metric}=15% or ${metric}=200ms`);
64
+ throw new Error(`Invalid threshold "${raw}": give the unit, such as ${metric}=15% or ${metric}=200ms`);
64
65
  }
65
66
 
66
67
  // The default thresholds with the parts that --threshold changes
@@ -153,8 +154,17 @@ function targetOf(target, pageUrl) {
153
154
  // (69.21 KB)…" and "Render-blocking CSS (71.80 KB)…"
154
155
  const byMessage = (error) => `${error.id}\u0000${error.message.replace(/\d+(?:[.,]\d+)*/g, "#")}`;
155
156
 
157
+ // One snippet measured on both sides is the same result, whatever name it was asked by: the CLI names
158
+ // the result of --snippet as it was typed (an alias, its name or Category/Name), and the MCP server by
159
+ // its path. Its conditions name it by its path on both.
160
+ function baseResultsOf(base, current) {
161
+ const path = base.conditions?.snippet;
162
+ if (!path || path !== current.conditions?.snippet || current.results?.length !== 1) return base.results ?? [];
163
+ return base.results.map((r) => ({ ...r, id: current.results[0].id }));
164
+ }
165
+
156
166
  function compareErrors(base, current) {
157
- const baseErrors = blockingErrors(base.results ?? []);
167
+ const baseErrors = blockingErrors(baseResultsOf(base, current));
158
168
  const currentErrors = blockingErrors(current.results ?? []);
159
169
  // The snippets whose errors on one side all lack a code pair by message on both
160
170
  const uncoded = (errors) => {
@@ -209,7 +219,8 @@ function warningsOf(base, current) {
209
219
  if (describeLoad(base.throttling) !== describeLoad(current.throttling)) {
210
220
  warnings.push(`the baseline was measured ${describeLoad(base.throttling)}, and this run ${describeLoad(current.throttling)}`);
211
221
  }
212
- if (runCount(base) < 2) warnings.push("the baseline has 1 run, so only the thresholds apply; measure both with --runs");
222
+ // In words for both: the CLI asks for runs with --runs, and the tools of the MCP server with runs
223
+ if (runCount(base) < 2) warnings.push(`the baseline has 1 run, so only the thresholds apply; measure both with ${RUNS_FOR_STATISTICS} runs or more`);
213
224
  else if (runCount(base) !== runCount(current)) warnings.push(`the baseline has ${runCount(base)} runs, and this run ${runCount(current)}`);
214
225
  const machine = machineWarning(b?.machine, c?.machine);
215
226
  if (machine) warnings.push(machine);
package/src/bin.js CHANGED
@@ -10,12 +10,12 @@ import { reportHuman, reportMatrixHuman } from "./reporters/human.js";
10
10
  import { reportJson } from "./reporters/json.js";
11
11
  import { reportMarkdown, reportMatrixMarkdown } from "./reporters/markdown.js";
12
12
  import { cliVersion, reportTo, validateReportUrl } from "./report-to.js";
13
- import { fetchCrux, formFactorFor } from "./crux.js";
13
+ import { fetchCrux, fetchCruxHistory, formFactorFor } from "./crux.js";
14
14
  import { MissingBrowserError } from "./browser.js";
15
15
  import { parseHeader } from "./headers.js";
16
16
  import { NETWORK_PRESETS, parseCpuFactor } from "./throttling.js";
17
17
  import { parseOwnDomains } from "./own-domains.js";
18
- import { MEDIAN_METRICS, summarizeRuns } from "./runs.js";
18
+ import { MEDIAN_METRICS, measureRuns } from "./runs.js";
19
19
  import { loadMatrix } from "./matrix.js";
20
20
  import { blockingErrors } from "./blocking.js";
21
21
  import { MEASURED_BY, checkBudgets, parseBudgets, unmeasuredMetrics, violationsOf } from "./budgets.js";
@@ -80,6 +80,7 @@ Options:
80
80
  LCP=200ms or CLS=0.1. Repeat it for more metrics
81
81
  --fail-on-regression Exit 1 when a metric regressed or an error is new against --baseline
82
82
  --crux-key <key> Add CrUX field data (p75) next to the measured values.
83
+ --crux-history <weeks> Add the trend of the field data over the last 1 to 40 weeks (needs a CrUX key).
83
84
  Also read from the CRUX_API_KEY environment variable
84
85
  --report-to <url> POST the results to this https URL after the run
85
86
  (without it the CLI makes no external calls)
@@ -175,6 +176,7 @@ async function main() {
175
176
  threshold: { type: "string", multiple: true },
176
177
  "fail-on-regression": { type: "boolean" },
177
178
  "crux-key": { type: "string" },
179
+ "crux-history": { type: "string" },
178
180
  "report-to": { type: "string" },
179
181
  "api-key": { type: "string" },
180
182
  verbose: { type: "boolean" },
@@ -253,6 +255,19 @@ async function main() {
253
255
  }
254
256
  if (medianBy != null && runCount == null && !matrixPath) fail("--median-by needs --runs");
255
257
 
258
+ // The field data: the p75 of the last 28 days with a key, and its trend week by week with --crux-history
259
+ const cruxKey = values["crux-key"] || process.env.CRUX_API_KEY;
260
+ let cruxWeeks;
261
+ if (values["crux-history"] !== undefined) {
262
+ cruxWeeks = /^\d+$/.test(values["crux-history"]) ? Number(values["crux-history"]) : NaN;
263
+ if (!(cruxWeeks >= 1 && cruxWeeks <= 40)) fail(`--crux-history takes 1 to 40 weeks, not "${values["crux-history"]}"`);
264
+ if (!cruxKey) fail("--crux-history needs a CrUX key: --crux-key or CRUX_API_KEY");
265
+ }
266
+ const fieldDataOf = async (url, formFactor) => {
267
+ const crux = await fetchCrux({ url, apiKey: cruxKey, formFactor });
268
+ return cruxWeeks ? { ...crux, history: await fetchCruxHistory({ url, apiKey: cruxKey, formFactor, weeks: cruxWeeks }) } : crux;
269
+ };
270
+
256
271
  // The baseline is read and checked before the browser opens: a wrong file fails at once
257
272
  let baseline;
258
273
  let thresholds;
@@ -301,7 +316,7 @@ async function main() {
301
316
  } catch (err) {
302
317
  fail(err.message);
303
318
  }
304
- await runMatrix(matrixPath, matrix, measureWith, { values, waitMs, reportUrl, conditionsFor, baseline, thresholds, budgets });
319
+ await runMatrix(matrixPath, matrix, measureWith, { values, waitMs, reportUrl, conditionsFor, baseline, thresholds, budgets, cruxKey, fieldDataOf });
305
320
  return;
306
321
  }
307
322
 
@@ -310,15 +325,13 @@ async function main() {
310
325
  if (runCount == null) {
311
326
  payload = await measureWith(conditions);
312
327
  } else {
313
- const median = await measureRuns(() => measureWith(conditions), runCount, medianBy, (i) => `[${i}/${runCount}] ${url} · run ${i}/${runCount}`);
328
+ const median = await measureRunsWithProgress(() => measureWith(conditions), runCount, medianBy, (i) => `[${i}/${runCount}] ${url} · run ${i}/${runCount}`);
314
329
  if (!median.payload) throw new Error(`Every run failed: ${median.errors.join("; ")}`);
315
330
  payload = { ...median.payload, runs: median.runs };
316
331
  }
317
332
  payload.conditions = conditionsFor(viewportName);
318
333
 
319
- const cruxKey = values["crux-key"] || process.env.CRUX_API_KEY;
320
- const crux = cruxKey ? await fetchCrux({ url, apiKey: cruxKey, formFactor: formFactorFor(viewportName) }) : undefined;
321
- if (crux) payload.crux = crux;
334
+ if (cruxKey) payload.crux = await fieldDataOf(url, formFactorFor(viewportName));
322
335
 
323
336
  if (baseline) {
324
337
  payload.baseline = compareRun(baseline.data, payload, { file: baseline.file, thresholds });
@@ -375,26 +388,20 @@ async function main() {
375
388
  process.exit(blockingErrors(payload.results).length > 0 ? 1 : 0);
376
389
  }
377
390
 
378
- // Measures `count` runs one after the other, never in parallel: they would compete for the CPU and
379
- // skew the timings. Each run is reported on stderr as it starts, so stdout keeps only the report. A
380
- // run that fails is kept as { error } and left out of the median.
381
- async function measureRuns(measure, count, medianBy, progress) {
382
- const runs = [];
383
- for (let i = 1; i <= count; i++) {
384
- process.stderr.write(`${progress(i)}\n`);
385
- try {
386
- runs.push(await measure());
387
- } catch (err) {
388
- if (err instanceof MissingBrowserError) throw err;
389
- runs.push({ error: err.message });
390
- }
391
- }
392
- return summarizeRuns(runs, { medianBy });
393
- }
391
+ // Each run is reported on stderr as it starts, so stdout keeps only the report
392
+ const measureRunsWithProgress = (measure, count, medianBy, progress) =>
393
+ measureRuns(
394
+ (i) => {
395
+ process.stderr.write(`${progress(i)}\n`);
396
+ return measure();
397
+ },
398
+ count,
399
+ { medianBy, isFatal: (err) => err instanceof MissingBrowserError }
400
+ );
394
401
 
395
402
  // Every combination of the matrix, in order, each one `matrix.runs` times. Each combination reports
396
403
  // its median run, and the exit code follows the same rules as one page, for every combination.
397
- async function runMatrix(matrixPath, matrix, measureWith, { values, waitMs, reportUrl, conditionsFor, baseline, thresholds, budgets }) {
404
+ async function runMatrix(matrixPath, matrix, measureWith, { values, waitMs, reportUrl, conditionsFor, baseline, thresholds, budgets, cruxKey, fieldDataOf }) {
398
405
  for (const warning of matrix.warnings) process.stderr.write(`Warning: ${warning}\n`);
399
406
  const viewportOf = new Map(matrix.combinations.map((c) => [c.id, c.viewport]));
400
407
 
@@ -404,7 +411,7 @@ async function runMatrix(matrixPath, matrix, measureWith, { values, waitMs, repo
404
411
  const combinations = [];
405
412
  for (const c of matrix.combinations) {
406
413
  const conditions = { url: c.url, viewport: VIEWPORT_PRESETS[c.viewport], storageState: c.storageState, headers: c.headers, cpu: c.cpu, network: c.network, ownDomains: c.ownDomains };
407
- const median = await measureRuns(() => measureWith(conditions), matrix.runs, matrix.medianBy, (i) => `[${String(++started).padStart(width)}/${total}] ${c.id} · run ${i}/${matrix.runs}`);
414
+ const median = await measureRunsWithProgress(() => measureWith(conditions), matrix.runs, matrix.medianBy, (i) => `[${String(++started).padStart(width)}/${total}] ${c.id} · run ${i}/${matrix.runs}`);
408
415
  const identity = { id: c.id, name: c.name, device: c.device, ...(c.variant ? { variant: c.variant } : {}), url: c.url };
409
416
  combinations.push(
410
417
  median.payload
@@ -414,13 +421,12 @@ async function runMatrix(matrixPath, matrix, measureWith, { values, waitMs, repo
414
421
  }
415
422
 
416
423
  // The field data of a URL depends on the form factor, not on the variant: one query for each pair
417
- const cruxKey = values["crux-key"] || process.env.CRUX_API_KEY;
418
424
  if (cruxKey) {
419
425
  const queries = new Map();
420
426
  for (const c of combinations.filter((c) => !c.error)) {
421
427
  const formFactor = formFactorFor(viewportOf.get(c.id));
422
428
  const key = `${c.url} ${formFactor}`;
423
- if (!queries.has(key)) queries.set(key, fetchCrux({ url: c.url, apiKey: cruxKey, formFactor }));
429
+ if (!queries.has(key)) queries.set(key, fieldDataOf(c.url, formFactor));
424
430
  const crux = await queries.get(key);
425
431
  if (crux) c.crux = crux;
426
432
  }
package/src/crux.js CHANGED
@@ -1,6 +1,9 @@
1
+ import { DEFAULT_THRESHOLDS } from "./baseline.js";
1
2
  import { rate } from "./ratings.js";
2
3
 
3
4
  const DEFAULT_ENDPOINT = "https://chromeuxreport.googleapis.com/v1/records:queryRecord";
5
+ // The History API lives next to the daily one; an endpoint given for tests or a proxy points at both
6
+ const historyEndpointOf = (endpoint) => endpoint.replace(/records:queryRecord$/, "records:queryHistoryRecord");
4
7
  const CRUX_TIMEOUT_MS = 10000;
5
8
 
6
9
  const METRICS = [
@@ -27,6 +30,42 @@ function parseMetrics(record) {
27
30
  return metrics;
28
31
  }
29
32
 
33
+ // The sub-parts of the LCP as CrUX reports them, for the loads whose LCP is an image, by the names of
34
+ // the lab sub-parts of LCP-Subparts
35
+ const SUB_PARTS = [
36
+ { part: "ttfb", label: "TTFB", key: "largest_contentful_paint_image_time_to_first_byte" },
37
+ { part: "resourceLoadDelay", label: "Load delay", key: "largest_contentful_paint_image_resource_load_delay" },
38
+ { part: "resourceLoadTime", label: "Load duration", key: "largest_contentful_paint_image_resource_load_duration" },
39
+ { part: "elementRenderDelay", label: "Render delay", key: "largest_contentful_paint_image_element_render_delay" },
40
+ ];
41
+
42
+ const p75Of = (record, key) => {
43
+ const value = Number(record?.metrics?.[key]?.percentiles?.p75);
44
+ return Number.isFinite(value) ? value : undefined;
45
+ };
46
+
47
+ // What else the record of a page says besides LCP, CLS and INP: the sub-parts of an image LCP, the
48
+ // share of loads whose LCP is an image, the round trip time and the navigation types, by share
49
+ export function parseFieldExtras(record) {
50
+ const extras = {};
51
+ const subParts = {};
52
+ for (const { part, key } of SUB_PARTS) {
53
+ const value = p75Of(record, key);
54
+ if (value !== undefined) subParts[part] = value;
55
+ }
56
+ if (Object.keys(subParts).length) extras.subParts = subParts;
57
+ const imageShare = record?.metrics?.largest_contentful_paint_resource_type?.fractions?.image;
58
+ if (typeof imageShare === "number") extras.lcpImageShare = imageShare;
59
+ const rtt = p75Of(record, "round_trip_time");
60
+ if (rtt !== undefined) extras.roundTripTime = rtt;
61
+ const fractions = record?.metrics?.navigation_types?.fractions;
62
+ if (fractions) {
63
+ const sorted = Object.entries(fractions).filter(([, share]) => share > 0).sort((a, b) => b[1] - a[1]);
64
+ if (sorted.length) extras.navigationTypes = Object.fromEntries(sorted);
65
+ }
66
+ return extras;
67
+ }
68
+
30
69
  async function query({ endpoint, apiKey, body, timeoutMs }) {
31
70
  const res = await fetch(`${endpoint}?key=${encodeURIComponent(apiKey)}`, {
32
71
  method: "POST",
@@ -61,7 +100,83 @@ export async function fetchCrux({
61
100
  if (answer.status) return { unavailable: `the CrUX API answered ${answer.status}`, failed: true };
62
101
  const metrics = parseMetrics(answer.record);
63
102
  if (Object.keys(metrics).length === 0) return { unavailable: "no Core Web Vitals in the CrUX record" };
64
- return { scope, formFactor, metrics };
103
+ return { scope, formFactor, metrics, ...parseFieldExtras(answer.record) };
104
+ } catch (err) {
105
+ const reason = err.name === "TimeoutError" ? `no answer within ${timeoutMs}ms` : (err.cause?.code ?? "request failed");
106
+ return { unavailable: reason, failed: true };
107
+ }
108
+ }
109
+
110
+ // A trend compares the first and the last periods with data: each period already covers 28 days, so the
111
+ // series is smooth enough. Within 5 % it is stable; lower is better for every metric.
112
+ const STABLE_WITHIN = 5;
113
+
114
+ export function trendOf(p75s) {
115
+ const values = p75s.filter((v) => v !== null);
116
+ const first = values[0];
117
+ const last = values.at(-1);
118
+ if (values.length < 2) return { first, last, change: null, trend: null };
119
+ const change = first === 0 ? null : Math.round(((last - first) / first) * 1000) / 10;
120
+ const trend = change === null || Math.abs(change) <= STABLE_WITHIN ? "stable" : change < 0 ? "improving" : "worsening";
121
+ return { first, last, change, trend };
122
+ }
123
+
124
+ const BARS = "▁▂▃▄▅▆▇█";
125
+
126
+ // The series as bars, a space for a period without data. The bars span the range of the series, or
127
+ // `minRange` when the series is narrower, centred in it: a CLS of 0.20 to 0.21 drawn from its lowest
128
+ // to its highest value looked like a big swing. Eight equal steps, so a flat series sits in the middle.
129
+ export function sparkline(values, minRange = 0) {
130
+ const known = values.filter((v) => v !== null);
131
+ const min = Math.min(...known);
132
+ const max = Math.max(...known);
133
+ const range = Math.max(max - min, minRange);
134
+ if (range === 0) return values.map((v) => (v === null ? " " : BARS[4])).join("");
135
+ const low = max - min >= minRange ? min : (min + max) / 2 - range / 2;
136
+ return values.map((v) => (v === null ? " " : BARS[Math.min(7, Math.floor(((v - low) / range) * 8))])).join("");
137
+ }
138
+
139
+ const dateOf = ({ year, month, day }) => `${year}-${String(month).padStart(2, "0")}-${String(day).padStart(2, "0")}`;
140
+
141
+ // The record of the History API as periods, oldest first, and the p75s of each Core Web Vital
142
+ export function parseHistory(record) {
143
+ const periods = (record?.collectionPeriods ?? []).map((p) => ({ start: dateOf(p.firstDate), end: dateOf(p.lastDate) }));
144
+ const metrics = {};
145
+ for (const m of METRICS) {
146
+ const raw = record?.metrics?.[m.key]?.percentilesTimeseries?.p75s;
147
+ if (!raw) continue;
148
+ // CLS comes as strings ("0.21"), and a period with too little data as null
149
+ const p75s = raw.map((v) => (v === null || v === undefined || !Number.isFinite(Number(v)) ? null : Number(v)));
150
+ if (p75s.every((v) => v === null)) continue;
151
+ metrics[m.name] = { unit: m.unit, p75s, ...trendOf(p75s) };
152
+ }
153
+ return { periods, metrics };
154
+ }
155
+
156
+ // The p75 of LCP, CLS and INP week by week over the last `weeks` periods (1 to 40), from the History
157
+ // API: the URL, or its origin when the URL has too little traffic. Never throws, and like fetchCrux
158
+ // never puts the key in a message.
159
+ export async function fetchCruxHistory({
160
+ url,
161
+ apiKey,
162
+ weeks,
163
+ formFactor = "PHONE",
164
+ endpoint = process.env.WEBPERF_CRUX_ENDPOINT ?? DEFAULT_ENDPOINT,
165
+ timeoutMs = CRUX_TIMEOUT_MS,
166
+ }) {
167
+ const history = historyEndpointOf(endpoint);
168
+ try {
169
+ let scope = "url";
170
+ let answer = await query({ endpoint: history, apiKey, body: { url, formFactor, collectionPeriodCount: weeks }, timeoutMs });
171
+ if (answer.notFound) {
172
+ scope = "origin";
173
+ answer = await query({ endpoint: history, apiKey, body: { origin: new URL(url).origin, formFactor, collectionPeriodCount: weeks }, timeoutMs });
174
+ }
175
+ if (answer.notFound) return { unavailable: "not in the CrUX dataset" };
176
+ if (answer.status) return { unavailable: `the CrUX History API answered ${answer.status}`, failed: true };
177
+ const { periods, metrics } = parseHistory(answer.record);
178
+ if (Object.keys(metrics).length === 0) return { unavailable: "no Core Web Vitals in the CrUX history" };
179
+ return { scope, formFactor, weeks: periods.length, periods, metrics };
65
180
  } catch (err) {
66
181
  const reason = err.name === "TimeoutError" ? `no answer within ${timeoutMs}ms` : (err.cause?.code ?? "request failed");
67
182
  return { unavailable: reason, failed: true };
@@ -84,3 +199,80 @@ export function fieldDataView(crux, results) {
84
199
  });
85
200
  return { title, rows };
86
201
  }
202
+
203
+ // The trend of the field data, shared by both reporters: per metric the first and last p75 of the
204
+ // weeks, the change and its direction, and the series as a sparkline
205
+ export function historyView(history) {
206
+ const label = FORM_FACTOR_LABEL[history.formFactor] ?? history.formFactor.toLowerCase();
207
+ const title = `Field data trend (CrUX p75, ${history.weeks} weeks, ${label}${history.scope === "origin" ? ", origin-level" : ""})`;
208
+ const rows = METRICS.filter((m) => history.metrics[m.name]).map((m) => {
209
+ const { p75s, first, last, change, trend } = history.metrics[m.name];
210
+ const changeText = change === null ? "·" : `${change > 0 ? "▲ +" : change < 0 ? "▼ −" : "= "}${Math.abs(change)}%`;
211
+ // The smallest change --baseline counts for the metric is the least the bars span
212
+ return { metric: m.name, unit: m.unit, first, last, change: changeText, trend: trend ?? "·", sparkline: sparkline(p75s, DEFAULT_THRESHOLDS[m.name].absolute) };
213
+ });
214
+ return { title, rows };
215
+ }
216
+
217
+ const share = (fraction) => (fraction < 0.01 ? `${Math.round(fraction * 1000) / 10}%` : `${Math.round(fraction * 100)}%`);
218
+
219
+ // A sub-part counts as different when one side is at least 1.5 times the other and 100 ms apart: less
220
+ // than that is within the noise of the lab and of a percentile
221
+ const differs = (slow, fast) => slow >= fast * 1.5 && slow - fast >= 100;
222
+
223
+ const rttNote = (rtt) => (rtt === undefined ? "" : ` (RTT p75 ${rtt} ms)`);
224
+ const FIELD_SLOWER = {
225
+ ttfb: (rtt) => `real users wait longer for the server: network distance${rttNote(rtt)}, the CDN or cache misses the lab does not see`,
226
+ resourceLoadDelay: () => 'the lab finds the image sooner: preload it or give it fetchpriority="high", or users run more scripts before it',
227
+ resourceLoadTime: (rtt) => `slower connections in the field${rttNote(rtt)}: compress or resize the LCP image`,
228
+ elementRenderDelay: () => "slower devices or more main-thread work in the field",
229
+ };
230
+ const LAB_SLOWER = "the lab is slower than users: check the CPU and network throttling of the lab";
231
+
232
+ // The sub-parts of the field next to those of LCP-Subparts in the lab, when it ran, with a note where
233
+ // they differ, and what the field sub-parts describe. Null without field sub-parts.
234
+ export function subPartsView(extras, results) {
235
+ if (!extras.subParts) return null;
236
+ const labResult = results.find((r) => r.script === "LCP-Subparts" && r.status === "ok");
237
+ const labParts = labResult?.details?.subParts ?? {};
238
+ // The field sub-parts are those of image LCPs: a lab LCP without a resource, such as text, has no load
239
+ // delay or duration to set against them
240
+ const labImage = !labResult || Boolean(labResult.details?.url);
241
+ const rows = SUB_PARTS.filter(({ part }) => extras.subParts[part] !== undefined).map(({ part, label }) => {
242
+ const field = extras.subParts[part];
243
+ const comparable = labImage || part === "ttfb" || part === "elementRenderDelay";
244
+ const lab = comparable ? (labParts[part]?.value ?? null) : null;
245
+ let note = null;
246
+ if (lab !== null && differs(field, lab)) note = FIELD_SLOWER[part](extras.roundTripTime);
247
+ else if (lab !== null && differs(lab, field)) note = LAB_SLOWER;
248
+ return { part, label, field, lab, note };
249
+ });
250
+ const caveats = [
251
+ ...(extras.lcpImageShare !== undefined ? [`The field sub-parts describe the ${share(extras.lcpImageShare)} of loads whose LCP is an image.`] : []),
252
+ ...(labImage ? [] : ["The LCP of the lab is not an image, so it has no load delay or duration to compare."]),
253
+ "Each p75 is taken on its own, so the sub-parts do not add up to the p75 of LCP.",
254
+ ];
255
+ return { rows, caveats };
256
+ }
257
+
258
+ const NAVIGATION_LABELS = {
259
+ navigate: "navigate",
260
+ navigate_cache: "navigate from cache",
261
+ reload: "reload",
262
+ restore: "restore",
263
+ back_forward: "back/forward",
264
+ back_forward_cache: "back/forward cache",
265
+ prerender: "prerender",
266
+ };
267
+
268
+ // How users arrive at the page, by share, and a pointer to Back-Forward-Cache when over 5 % of the
269
+ // visits come back without the back/forward cache, which would have shown the page at once
270
+ export function navigationView(extras) {
271
+ if (!extras.navigationTypes) return null;
272
+ const parts = Object.entries(extras.navigationTypes).map(([type, fraction]) => `${NAVIGATION_LABELS[type] ?? type} ${share(fraction)}`);
273
+ const missed = extras.navigationTypes.back_forward ?? 0;
274
+ return {
275
+ line: `Navigations: ${parts.join(", ")}`,
276
+ hint: missed > 0.05 ? `${share(missed)} of the visits come back without the back/forward cache: run Back-Forward-Cache to see what blocks it.` : null,
277
+ };
278
+ }
@@ -0,0 +1,26 @@
1
+ import { loadBaseline, compareRun, parseThreshold, thresholdsWith } from "../baseline.js";
2
+ import { describeRegressions } from "../reporters/baseline-view.js";
3
+
4
+ // The baseline of run_snippet and run_workflow, as --baseline of the CLI: the runId of an earlier run of
5
+ // the connection, or the path of a page the CLI wrote with --json. The comparison is the one of the CLI
6
+ // (baseline.js), so both give the same verdict for the same two pages.
7
+
8
+ const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
9
+
10
+ // Read and checked before the page loads, as the CLI does before the browser opens: a wrong baseline
11
+ // fails at once. Throws an error whose message says what to do.
12
+ export function prepareBaseline({ baseline, thresholds = [] }, runs) {
13
+ const parsed = thresholdsWith(thresholds.map(parseThreshold));
14
+ // An id of a run the connection no longer keeps is not a file to look for: runs.page says so
15
+ if (runs.has(baseline) || UUID.test(baseline)) return { data: runs.page(baseline), source: { runId: baseline }, thresholds: parsed };
16
+ const { kind, data } = loadBaseline(baseline);
17
+ if (kind === "matrix") throw new Error(`${baseline} is the baseline of a matrix: this server measures one page; compare it with webperf-snippets --matrix <file> --baseline ${baseline}`);
18
+ return { data, source: { file: baseline }, thresholds: parsed };
19
+ }
20
+
21
+ // The page of this run against its baseline: the `baseline` of the JSON of the CLI, named by its runId or
22
+ // its file, with the lines --fail-on-regression would print for it
23
+ export function compareWithBaseline({ data, source, thresholds }, current) {
24
+ const { file, ...comparison } = compareRun(data, current, { file: source.file, thresholds });
25
+ return { ...source, ...comparison, regressions: describeRegressions(comparison) };
26
+ }
package/src/mcp/server.js CHANGED
@@ -1,4 +1,4 @@
1
- import { McpServer } from "@modelcontextprotocol/server";
1
+ import { McpServer, ResourceTemplate, ResourceNotFoundError } from "@modelcontextprotocol/server";
2
2
  import * as z from "zod/v4";
3
3
  import { cliVersion } from "../report-to.js";
4
4
  import { CATEGORIES, aliasesOf, listSnippets, resolveSnippetName } from "../snippet-names.js";
@@ -7,7 +7,7 @@ import { isTrackingSnippet } from "../tracking.js";
7
7
  import { runSnippets, runMeasurement, evaluateItems, collectTrackingData, VIEWPORT_PRESETS } from "../runner.js";
8
8
  import { WORKFLOWS } from "../workflows/index.js";
9
9
  import { RULES } from "../decision-tree.js";
10
- import { fetchCrux } from "../crux.js";
10
+ import { fetchCrux, fetchCruxHistory } from "../crux.js";
11
11
  import { runInteractions, isInputStep, settle } from "../interactions.js";
12
12
  import { createSessions, SessionError } from "./sessions.js";
13
13
  import { launchChromium, isMissingBrowser, MISSING_BROWSER } from "../browser.js";
@@ -16,6 +16,10 @@ import { registerPrompts } from "./prompts.js";
16
16
  import { shapeResults, createRuns, RunError } from "./summary.js";
17
17
  import { followUpCalls, trackingCall } from "./next-steps.js";
18
18
  import { isHttpUrl } from "../url.js";
19
+ import { machineOf } from "../machine.js";
20
+ import { measurementConditions } from "../conditions.js";
21
+ import { prepareBaseline, compareWithBaseline } from "./baseline.js";
22
+ import { MAX_RUNS, measureRuns } from "../runs.js";
19
23
 
20
24
  const VIEWPORTS = Object.keys(VIEWPORT_PRESETS);
21
25
  const STAGES = 3;
@@ -36,8 +40,59 @@ const SnippetResult = z
36
40
  const SessionId = z.string().describe("The id open_page returned");
37
41
 
38
42
  const RunId = z.string().describe("The id of the run, for get_details");
39
- // Only a summary names its run: whole objects leave nothing for get_details to add
43
+ // Only a summary of read_results names its run: whole objects leave nothing for get_details to add
40
44
  const SummaryRunId = RunId.optional().describe("The id of the run, for get_details; only with summaries");
45
+ // A run of a page load is also the baseline of a later one, so it always has its id
46
+ const PageRunId = RunId.describe("The id of the run: the baseline of a later run_snippet or run_workflow; with summaries, also for get_details");
47
+
48
+ const Baseline = z
49
+ .string()
50
+ .min(1)
51
+ .optional()
52
+ .describe(
53
+ "Compare this run with a baseline, as --baseline of the CLI: the runId of an earlier run_snippet or run_workflow of this connection (listed as webperf://runs/{runId}), or the path of a page the CLI wrote with --json, absolute or relative to the working directory of the server"
54
+ );
55
+ const Thresholds = z
56
+ .array(z.string())
57
+ .optional()
58
+ .describe("With baseline, the threshold of a metric, as --threshold of the CLI: LCP=15% (relative), LCP=200ms (absolute), CLS=0.1. The part not given keeps its default");
59
+
60
+ const Runs = z
61
+ .number()
62
+ .int()
63
+ .min(1)
64
+ .max(MAX_RUNS)
65
+ .default(1)
66
+ .describe(
67
+ "Measure the page this many times, one after the other, as --runs of the CLI: the answer is the median run, plus runs with the median, min, max and agreement of each metric. With baseline, 5 or more on both sides give the statistical verdict: use 5 for a comparison, and more only when agreement says noisy. Each run is a full page load and wait, so it takes that much longer"
68
+ );
69
+
70
+ const LooseObject = (shape) => z.looseObject(shape).meta({ additionalProperties: true });
71
+ const RunsSummary = LooseObject({
72
+ count: z.number(),
73
+ failed: z.number().describe("Runs left out because they failed"),
74
+ medianRun: z.number().describe("The run the answer comes from, from 1"),
75
+ medianBy: z.string().describe("The metric that picked it"),
76
+ summary: z.record(z.string(), LooseObject({ median: z.number() })).describe("By metric: median, min, max, values of each run (null for a failed one), unit, cv and agreement (consistent, variable, noisy)"),
77
+ })
78
+ .optional()
79
+ .describe("With several runs, the runs field of the JSON of the CLI");
80
+ const BaselineError = LooseObject({ id: z.string(), message: z.string().optional() });
81
+ const Comparison = z
82
+ .object({
83
+ runId: z.string().optional().describe("The run the baseline was, when it was a runId"),
84
+ file: z.string().optional().describe("The file the baseline was, when it was a path"),
85
+ url: z.string().describe("The URL the baseline measured"),
86
+ runs: LooseObject({ count: z.number() }).describe("The runs of the baseline"),
87
+ metrics: z
88
+ .record(z.string(), LooseObject({ verdict: z.string() }))
89
+ .describe("By metric: baseline, current, delta, deltaPercent, threshold, verdict (regressed, improved, unchanged, not measured) and, with five runs a side, stats (p, shift, ci, method)"),
90
+ errors: z.object({ new: z.array(BaselineError), fixed: z.array(BaselineError) }).describe("The errors only this run has, and the ones only the baseline had"),
91
+ warnings: z.array(z.string()).describe("What differs in how both were measured: how far to trust the comparison"),
92
+ regressions: z.array(z.string()).describe("What got worse, one line each: the metrics that regressed and the new errors"),
93
+ })
94
+ .optional()
95
+ .describe("With baseline, the comparison, as the baseline field of the JSON of the CLI");
41
96
 
42
97
  // auto, the default, returns whole objects when they are small and summaries when they are large: a
43
98
  // summary of a small result costs the agent a turn for the details, which is more than it saves
@@ -55,6 +110,7 @@ const NextSteps = z
55
110
 
56
111
  const DETAIL_NOTE =
57
112
  "Large results come as summaries with a runId for get_details; nextSteps, when present, is the call to make next.";
113
+ const BASELINE_NOTE = "With baseline, it also compares the run with an earlier one, as --baseline of the CLI, and says what regressed.";
58
114
 
59
115
  // A page can put instructions for the agent in what a snippet returns: an alt text, a class in a
60
116
  // selector, the URL of a script. The tools that return what a page holds say how to read it, and so
@@ -92,13 +148,28 @@ const toolError = (text) => ({ isError: true, content: [{ type: "text", text }]
92
148
 
93
149
  const firstLine = (message) => message.split("\n")[0];
94
150
 
95
- // A client that wants progress sends a token with the call; the stages of a run go to it
96
- function progressFor(ctx) {
151
+ // A client that wants progress sends a token with the call; the stages of each run go to it, numbered
152
+ // across the runs. Returns the reporter of a run, by its number.
153
+ function progressFor(ctx, runs) {
97
154
  const progressToken = ctx.mcpReq._meta?.progressToken;
98
- return (progress, message) =>
155
+ const total = STAGES * runs;
156
+ return (run) => (stage, message) =>
99
157
  progressToken === undefined
100
158
  ? undefined
101
- : ctx.mcpReq.notify({ method: "notifications/progress", params: { progressToken, progress, total: STAGES, message } });
159
+ : ctx.mcpReq.notify({
160
+ method: "notifications/progress",
161
+ params: { progressToken, progress: (run - 1) * STAGES + stage, total, message: runs > 1 ? `Run ${run} of ${runs}: ${message}` : message },
162
+ });
163
+ }
164
+
165
+ // A page load measured once, or `count` times with the median run as the answer, as --runs of the CLI.
166
+ // When the client gives up (a cancel, its timeout, or leaving), the runs left never start.
167
+ async function measurePage(measure, count, ctx) {
168
+ const progress = progressFor(ctx, count);
169
+ if (count === 1) return measure(progress(1));
170
+ const { payload, runs, errors } = await measureRuns((i) => measure(progress(i)), count, { isFatal: isMissingBrowser, signal: ctx.mcpReq.signal });
171
+ if (!payload) throw new Error(`Every run failed: ${[...new Set(errors.map(firstLine))].join("; ")}`);
172
+ return { ...payload, runs };
102
173
  }
103
174
 
104
175
  const CRUX_FORM_FACTORS = { phone: "PHONE", desktop: "DESKTOP", tablet: "TABLET" };
@@ -116,9 +187,44 @@ function describeSnippet(name) {
116
187
  };
117
188
  }
118
189
 
190
+ // The runs of the connection that measured a page, as resources: an agent picks the baseline of a run
191
+ // among them. Each is the JSON the CLI prints with --json, so saved to a file it is a baseline of the
192
+ // CLI too.
193
+ function registerRuns(server, runs) {
194
+ const JSON_TYPE = "application/json";
195
+ server.registerResource(
196
+ "runs",
197
+ new ResourceTemplate("webperf://runs/{runId}", {
198
+ list: async () => ({
199
+ resources: runs.pages().map(({ runId, url, conditions, at, runs: count }) => ({
200
+ uri: `webperf://runs/${runId}`,
201
+ name: `${conditions.snippet ?? conditions.workflow} on ${url} (${conditions.viewport})${count ? `, ${count} runs` : ""}`,
202
+ description: `Measured at ${at}`,
203
+ mimeType: JSON_TYPE,
204
+ })),
205
+ }),
206
+ }),
207
+ {
208
+ title: "Runs of this connection",
209
+ description: `The page an earlier run_snippet or run_workflow of this connection measured, with its results and conditions, as the JSON of the CLI (--json). Its runId is the baseline argument of a later run. The server keeps the last 20 runs. ${PAGE_DATA}`,
210
+ mimeType: JSON_TYPE,
211
+ },
212
+ async (uri, { runId }) => {
213
+ let page;
214
+ try {
215
+ page = runs.page(runId);
216
+ } catch (err) {
217
+ if (err instanceof RunError) throw new ResourceNotFoundError(uri.href);
218
+ throw err;
219
+ }
220
+ return { contents: [{ uri: uri.href, mimeType: JSON_TYPE, text: JSON.stringify(page) }] };
221
+ }
222
+ );
223
+ }
224
+
119
225
  // The MCP server of the CLI: the same snippets and runner, offered as tools an agent calls by name.
120
226
  // serveStdio creates one instance per connection, so the pages it keeps open belong to that client.
121
- export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env } = {}) {
227
+ export function createServer({ launchBrowser = () => launchChromium(), maxPages, idleMs, env = process.env, machine = machineOf } = {}) {
122
228
  const server = new McpServer({ name: "webperf-snippets", version: cliVersion }, { instructions: INSTRUCTIONS });
123
229
  const sessions = createSessions({ launchBrowser, maxPages, idleMs });
124
230
  const runs = createRuns();
@@ -126,6 +232,30 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
126
232
  server.server.onclose = () => sessions.closeAll();
127
233
  registerResources(server);
128
234
  registerPrompts(server);
235
+ registerRuns(server, runs);
236
+
237
+ // The machine goes in the conditions of every run, as in the JSON of the CLI, so a baseline measured
238
+ // on another one warns. It is measured once, with the first run, before its Chromium starts.
239
+ let machineOfServer;
240
+ const conditionsOf = (conditions) => measurementConditions({ ...conditions, machine: (machineOfServer ??= machine()) });
241
+
242
+ // A run of a page load, kept with its page in the shape of the JSON of the CLI: its runId is the
243
+ // baseline of a later run, and the resources list it
244
+ const keepPage = (run, results, conditions) => {
245
+ const page = { url: run.url, navMs: run.navMs, pageErrors: run.pageErrors, ...(run.throttling ? { throttling: run.throttling } : {}), ...(run.runs ? { runs: run.runs } : {}), conditions };
246
+ const runId = runs.keep(results, page);
247
+ server.sendResourceListChanged();
248
+ return { runId, current: { ...page, results } };
249
+ };
250
+
251
+ // The baseline of a tool, read and checked before the page loads; a wrong one is a tool error
252
+ const baselineFor = ({ baseline, thresholds }) => {
253
+ if (baseline === undefined) {
254
+ if (thresholds?.length) throw new Error("thresholds need a baseline");
255
+ return undefined;
256
+ }
257
+ return prepareBaseline({ baseline, thresholds }, runs);
258
+ };
129
259
 
130
260
  // Runs a session tool, and turns the errors the model can act on into tool errors
131
261
  const withSession = (handler) => async (args, ctx) => {
@@ -162,28 +292,33 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
162
292
  {
163
293
  title: "Run a snippet",
164
294
  description:
165
- `Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${DETAIL_NOTE} ${PAGE_DATA}`,
295
+ `Opens the URL in a headless Chromium, runs one web performance snippet on the loaded page and returns the object the snippet returns, as defined in snippets/SCHEMA.md. Use list_snippets for the names. The snippets that record interactions (INP, Interactions, Input-Latency-Breakdown) run on a page opened with open_page instead. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
166
296
  inputSchema: z.object({
167
297
  url: HttpUrl,
168
298
  snippet: z.string().describe("The snippet: its name (LCP-Subparts), an alias (render-blocking) or Category/Name"),
169
299
  viewport: z.enum(VIEWPORTS).default("mobile").describe("Device size of the page"),
170
300
  waitMs: z.number().int().min(0).max(10000).default(3000).describe("Wait after the load event before running the snippet, in ms"),
301
+ runs: Runs,
171
302
  detail: Detail,
303
+ baseline: Baseline,
304
+ thresholds: Thresholds,
172
305
  }),
173
306
  outputSchema: z.object({
174
307
  url: z.string(),
175
- runId: SummaryRunId,
308
+ runId: PageRunId,
176
309
  detail: DetailOut,
177
310
  snippet: z.string().describe("Category/Name of the snippet that ran"),
178
311
  viewport: z.enum(VIEWPORTS),
179
312
  navMs: z.number().describe("From the start of the navigation to the end of the wait"),
180
313
  pageErrors: z.array(z.string()).describe("Uncaught errors of the page during the run"),
181
314
  result: SnippetResult,
315
+ runs: RunsSummary,
316
+ baseline: Comparison,
182
317
  nextSteps: NextSteps,
183
318
  }),
184
319
  annotations: { readOnlyHint: true, openWorldHint: true },
185
320
  },
186
- async ({ url, snippet, viewport, waitMs, detail }, ctx) => {
321
+ async ({ url, snippet, viewport, waitMs, runs: count, detail, baseline, thresholds }, ctx) => {
187
322
  let path;
188
323
  try {
189
324
  path = resolveSnippetName(snippet);
@@ -194,22 +329,31 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
194
329
  if (isTrackingSnippet(path, source)) {
195
330
  return toolError(`${path} records interactions: open the page with open_page and this snippet in track, use it with interact, and read it with read_results.`);
196
331
  }
332
+ let base;
333
+ try {
334
+ base = baselineFor({ baseline, thresholds });
335
+ } catch (err) {
336
+ return toolError(err.message);
337
+ }
197
338
 
198
339
  try {
199
- const run = await runSnippets({ url, items: [{ id: path, path, source }], waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress: progressFor(ctx) });
340
+ const conditions = conditionsOf({ viewport, snippet: path, waitMs });
341
+ const run = await measurePage((onProgress) => runSnippets({ url, items: [{ id: path, path, source }], waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress }), count, ctx);
200
342
  const shaped = shapeResults(run.results.map(({ id, ...result }) => result), detail);
201
- const runId = shaped.detail === "summary" ? runs.keep(run.results) : undefined;
343
+ const { runId, current } = keepPage(run, run.results, conditions);
202
344
  const id = run.results[0].id;
203
345
  const nextSteps = followUpCalls({ results: run.results, url, viewport, waitMs, ran: [path] });
204
346
  return toolResult({
205
347
  url,
206
- ...(runId ? { runId } : {}),
348
+ runId,
207
349
  detail: shaped.detail,
208
350
  snippet: id,
209
351
  viewport,
210
352
  navMs: run.navMs,
211
353
  pageErrors: run.pageErrors,
212
354
  result: shaped.results[0],
355
+ ...(run.runs ? { runs: run.runs } : {}),
356
+ ...(base ? { baseline: compareWithBaseline(base, current) } : {}),
213
357
  ...(nextSteps.length ? { nextSteps } : {}),
214
358
  });
215
359
  } catch (err) {
@@ -225,17 +369,20 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
225
369
  {
226
370
  title: "Run a workflow",
227
371
  description:
228
- `Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${DETAIL_NOTE} ${PAGE_DATA}`,
372
+ `Opens the URL in a headless Chromium and runs a workflow: a sequence of snippets plus the follow-ups its decision tree adds from the results (for example LCP-Subparts when LCP is over 2.5 s, with the reason). core-web-vitals: LCP and CLS. audit: render-blocking resources, resource hints, critical CSS, TTFB, scripts, images and more. loading: TTFB, FCP, render-blocking, scripts and fonts. interaction: long tasks, animation frames, scroll and layout shifts while it scrolls the page. ${BASELINE_NOTE} ${DETAIL_NOTE} ${PAGE_DATA}`,
229
373
  inputSchema: z.object({
230
374
  url: HttpUrl,
231
375
  workflow: z.enum(Object.keys(WORKFLOWS)).default("core-web-vitals"),
232
376
  viewport: z.enum(VIEWPORTS).default("mobile").describe("Device size of the page"),
233
377
  waitMs: z.number().int().min(0).max(10000).default(3000).describe("Wait after the load event before running the workflow, in ms"),
378
+ runs: Runs,
234
379
  detail: Detail,
380
+ baseline: Baseline,
381
+ thresholds: Thresholds,
235
382
  }),
236
383
  outputSchema: z.object({
237
384
  url: z.string(),
238
- runId: SummaryRunId,
385
+ runId: PageRunId,
239
386
  detail: DetailOut,
240
387
  workflow: z.string(),
241
388
  viewport: z.enum(VIEWPORTS),
@@ -244,20 +391,26 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
244
391
  results: z
245
392
  .array(z.looseObject({ id: z.string(), status: z.string() }).meta({ additionalProperties: true }))
246
393
  .describe("One object per step, in order, then the follow-ups, each with the reason it was added"),
394
+ runs: RunsSummary,
395
+ baseline: Comparison,
247
396
  nextSteps: NextSteps,
248
397
  }),
249
398
  annotations: { readOnlyHint: true, openWorldHint: true },
250
399
  },
251
- async ({ url, workflow, viewport, waitMs, detail }, ctx) => {
400
+ async ({ url, workflow, viewport, waitMs, runs: count, detail, baseline, thresholds }, ctx) => {
401
+ let base;
252
402
  try {
253
- const run = await runMeasurement({
254
- url,
255
- workflow: WORKFLOWS[workflow],
256
- rules: RULES,
257
- waitMs,
258
- viewport: VIEWPORT_PRESETS[viewport],
259
- onProgress: progressFor(ctx),
260
- });
403
+ base = baselineFor({ baseline, thresholds });
404
+ } catch (err) {
405
+ return toolError(err.message);
406
+ }
407
+ try {
408
+ const conditions = conditionsOf({ viewport, workflow, waitMs });
409
+ const run = await measurePage(
410
+ (onProgress) => runMeasurement({ url, workflow: WORKFLOWS[workflow], rules: RULES, waitMs, viewport: VIEWPORT_PRESETS[viewport], onProgress }),
411
+ count,
412
+ ctx
413
+ );
261
414
  // The runner explains a skipped step with the flag of the CLI; here the tools are what the model has
262
415
  const results = run.results.map((r) =>
263
416
  r.status === "skipped"
@@ -265,7 +418,7 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
265
418
  : r
266
419
  );
267
420
  const shaped = shapeResults(results, detail);
268
- const runId = shaped.detail === "summary" ? runs.keep(results) : undefined;
421
+ const { runId, current } = keepPage(run, results, conditions);
269
422
  // The runner already ran the follow-ups of the decision tree; what is left is the steps it skipped
270
423
  const skipped = results
271
424
  .filter((r) => r.status === "skipped")
@@ -274,12 +427,14 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
274
427
  const nextSteps = skipped.length ? [trackingCall({ paths: skipped, url, viewport })] : [];
275
428
  return toolResult({
276
429
  url,
277
- ...(runId ? { runId } : {}),
430
+ runId,
278
431
  ...shaped,
279
432
  workflow,
280
433
  viewport,
281
434
  navMs: run.navMs,
282
435
  pageErrors: run.pageErrors,
436
+ ...(run.runs ? { runs: run.runs } : {}),
437
+ ...(base ? { baseline: compareWithBaseline(base, current) } : {}),
283
438
  ...(nextSteps.length ? { nextSteps } : {}),
284
439
  });
285
440
  } catch (err) {
@@ -295,10 +450,11 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
295
450
  {
296
451
  title: "Get CrUX field data",
297
452
  description:
298
- "Returns the p75 of LCP, CLS and INP that real Chrome users had on a URL over the last 28 days, from the Chrome UX Report, with their rating. Falls back to the origin when the URL has no data of its own. Compare it with the lab values of run_snippet: field data is what users get.",
453
+ "Returns the p75 of LCP, CLS and INP that real Chrome users had on a URL over the last 28 days, from the Chrome UX Report, with their rating. Falls back to the origin when the URL has no data of its own. Compare it with the lab values of run_snippet: field data is what users get. With weeks, it adds the trend: the p75 of each week, each the 28 days before it, with the change from the first week to the last.",
299
454
  inputSchema: z.object({
300
455
  url: HttpUrl,
301
456
  formFactor: z.enum(Object.keys(CRUX_FORM_FACTORS)).default("phone").describe("The devices the data describes"),
457
+ weeks: z.number().int().min(1).max(40).optional().describe("Add the trend of the last 1 to 40 weeks"),
302
458
  }),
303
459
  outputSchema: z.object({
304
460
  url: z.string(),
@@ -309,21 +465,38 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
309
465
  .optional()
310
466
  .describe("By metric name: LCP, CLS, INP"),
311
467
  unavailable: z.string().optional().describe("Why there is no field data, such as a page with too little traffic"),
468
+ subParts: z
469
+ .record(z.string(), z.number())
470
+ .optional()
471
+ .describe("The p75 of the sub-parts of an image LCP: ttfb, resourceLoadDelay, resourceLoadTime, elementRenderDelay"),
472
+ lcpImageShare: z.number().optional().describe("The share of loads whose LCP is an image, which the sub-parts describe"),
473
+ roundTripTime: z.number().optional().describe("The p75 of the round trip time of the users, in ms"),
474
+ navigationTypes: z.record(z.string(), z.number()).optional().describe("The share of each navigation type, such as navigate, reload or back_forward_cache"),
475
+ history: z
476
+ .looseObject({})
477
+ .meta({ additionalProperties: true })
478
+ .optional()
479
+ .describe("With weeks: scope, weeks, periods, and per metric its p75s, first, last, change in percent and trend"),
312
480
  }),
313
481
  annotations: { readOnlyHint: true, openWorldHint: true },
314
482
  },
315
- async ({ url, formFactor }) => {
483
+ async ({ url, formFactor, weeks }) => {
316
484
  const apiKey = env.CRUX_API_KEY;
317
485
  if (!apiKey) return toolError(CRUX_SETUP);
318
- const crux = await fetchCrux({
486
+ const options = {
319
487
  url,
320
488
  apiKey,
321
489
  formFactor: CRUX_FORM_FACTORS[formFactor],
322
490
  ...(env.WEBPERF_CRUX_ENDPOINT ? { endpoint: env.WEBPERF_CRUX_ENDPOINT } : {}),
323
- });
491
+ };
492
+ const crux = await fetchCrux(options);
324
493
  if (crux.failed) return toolError(`Could not read the CrUX data: ${crux.unavailable}.`);
325
- if (crux.unavailable) return toolResult({ url, formFactor, unavailable: crux.unavailable });
326
- return toolResult({ url, formFactor, scope: crux.scope, metrics: crux.metrics });
494
+ const history = weeks ? await fetchCruxHistory({ ...options, weeks }) : undefined;
495
+ // A history that failed says why in its own unavailable, next to the p75 that did come
496
+ const trend = history ? { history: history.failed ? { unavailable: history.unavailable } : history } : {};
497
+ if (crux.unavailable) return toolResult({ url, formFactor, unavailable: crux.unavailable, ...trend });
498
+ const { scope, metrics, subParts, lcpImageShare, roundTripTime, navigationTypes } = crux;
499
+ return toolResult({ url, formFactor, scope, metrics, subParts, lcpImageShare, roundTripTime, navigationTypes, ...trend });
327
500
  }
328
501
  );
329
502
 
@@ -445,7 +618,12 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
445
618
  runId: RunId,
446
619
  snippet: z.string().describe("The id of the result in the summary, or its script"),
447
620
  }),
448
- outputSchema: z.object({ runId: z.string(), snippet: z.string().describe("The id of the result in the run"), result: SnippetResult }),
621
+ outputSchema: z.object({
622
+ runId: z.string(),
623
+ snippet: z.string().describe("The id of the result in the run"),
624
+ medianRun: z.number().optional().describe("With several runs, the one this result comes from: the median run, the only one kept"),
625
+ result: SnippetResult,
626
+ }),
449
627
  annotations: { readOnlyHint: true, openWorldHint: false },
450
628
  },
451
629
  async ({ runId, snippet }) => {
@@ -464,7 +642,8 @@ export function createServer({ launchBrowser = () => launchChromium(), maxPages,
464
642
  if (!(err instanceof RunError) || !resolved || resolved === snippet) throw err;
465
643
  result = runs.get(runId, resolved);
466
644
  }
467
- return toolResult({ runId, snippet: result.id, result });
645
+ const medianRun = runs.medianRun(runId);
646
+ return toolResult({ runId, snippet: result.id, ...(medianRun ? { medianRun } : {}), result });
468
647
  } catch (err) {
469
648
  if (err instanceof RunError) return toolError(err.message);
470
649
  throw err;
@@ -29,27 +29,49 @@ export function shapeResults(results, detail = "auto", maxFullChars = FULL_UP_TO
29
29
  }
30
30
 
31
31
  // The full results of the last runs of an MCP connection, by run id, so get_details answers with the
32
- // values the summary came from: measuring again would give other ones
32
+ // values the summary came from: measuring again would give other ones. A run of a page load also keeps
33
+ // its page (url, navMs, pageErrors, conditions), so a later run can take it as its baseline.
33
34
  export function createRuns({ max = 20 } = {}) {
34
35
  const runs = new Map();
36
+ const unknown = (runId) =>
37
+ new RunError(`Unknown runId ${runId}: this server keeps the last ${max} runs of the connection. Run the tool again for a new one.`);
35
38
 
36
- function keep(results) {
39
+ function keep(results, page) {
37
40
  const runId = randomUUID();
38
- runs.set(runId, results);
41
+ runs.set(runId, { results, page, at: new Date().toISOString() });
39
42
  // A Map keeps the order of insertion, so the first key is the oldest run
40
43
  if (runs.size > max) runs.delete(runs.keys().next().value);
41
44
  return runId;
42
45
  }
43
46
 
44
47
  function get(runId, snippet) {
45
- const results = runs.get(runId);
46
- if (!results) {
47
- throw new RunError(`Unknown runId ${runId}: this server keeps the last ${max} runs of the connection. Run the tool again for a new one.`);
48
- }
48
+ const run = runs.get(runId);
49
+ if (!run) throw unknown(runId);
50
+ const { results } = run;
49
51
  const result = results.find((r) => r.id === snippet) ?? results.find((r) => r.script === snippet);
50
52
  if (!result) throw new RunError(`${snippet} is not in run ${runId}, which has: ${results.map((r) => r.id).join(", ")}.`);
51
53
  return result;
52
54
  }
53
55
 
54
- return { keep, get };
56
+ // A run as the JSON of the CLI gives a page, which is what a baseline is
57
+ function page(runId) {
58
+ const run = runs.get(runId);
59
+ if (!run) throw unknown(runId);
60
+ if (!run.page) throw new RunError(`Run ${runId} read an open page; a baseline is a run of run_snippet or run_workflow.`);
61
+ return { ...run.page, results: run.results };
62
+ }
63
+
64
+ // The runs that can be a baseline, the oldest first, with the number of measurements of each when it
65
+ // has several: the statistics of a comparison need five a side
66
+ const pages = () =>
67
+ [...runs]
68
+ .filter(([, run]) => run.page)
69
+ .map(([runId, { page, at }]) => ({ runId, url: page.url, conditions: page.conditions, at, ...(page.runs ? { runs: page.runs.count } : {}) }));
70
+
71
+ // A measurement of several runs keeps the results of the median one, as the JSON of the CLI does
72
+ const medianRun = (runId) => runs.get(runId)?.page?.runs?.medianRun;
73
+
74
+ const has = (runId) => runs.has(runId);
75
+
76
+ return { keep, get, page, pages, medianRun, has };
55
77
  }
@@ -1,7 +1,7 @@
1
1
  import { styleText } from "node:util";
2
2
  import { describeResult } from "../view.js";
3
3
  import { isSummaryResult, summaryNumbers } from "./summary-view.js";
4
- import { fieldDataView } from "../crux.js";
4
+ import { fieldDataView, historyView, subPartsView, navigationView } from "../crux.js";
5
5
  import { describeThrottling } from "../throttling.js";
6
6
  import { describeRuns, describeAgreement } from "../runs.js";
7
7
  import { matrixView } from "./matrix-view.js";
@@ -197,7 +197,8 @@ function renderResult(r, verbose) {
197
197
  }
198
198
 
199
199
  function renderFieldData(crux, results) {
200
- if (crux.unavailable) return [styleText("dim", `Field data: not available (${crux.unavailable})`)];
200
+ const trend = crux.history ? ["", ...renderFieldTrend(crux.history)] : [];
201
+ if (crux.unavailable) return [styleText("dim", `Field data: not available (${crux.unavailable})`), ...trend];
201
202
  const { title, rows } = fieldDataView(crux, results);
202
203
  const lines = [styleText("bold", title)];
203
204
  for (const row of rows) {
@@ -207,7 +208,41 @@ function renderFieldData(crux, results) {
207
208
  ` ${icon} ${pad(row.metric, 6)} ${pad(paint(row.rating, formatValue(row.p75, row.unit)), 10)} ${pad(styleText("dim", row.rating), 20)} ${styleText("dim", `(synthetic: ${synthetic})`)}`
208
209
  );
209
210
  }
210
- return lines;
211
+ return [...lines, ...renderFieldSubParts(crux, results), ...renderNavigations(crux), ...trend];
212
+ }
213
+
214
+ // "TTFB 1.53s lab 400ms real users wait longer for the server…"
215
+ function renderFieldSubParts(crux, results) {
216
+ const view = subPartsView(crux, results);
217
+ if (!view) return [];
218
+ return [
219
+ "",
220
+ styleText("bold", "LCP sub-parts (CrUX p75 against the lab)"),
221
+ ...view.rows.map(
222
+ (row) => ` ${pad(row.label, 14)}${pad(formatValue(row.field, "ms"), 10)}${pad(`lab ${row.lab === null ? "n/a" : formatValue(row.lab, "ms")}`, 12)}${row.note ?? ""}`.trimEnd()
223
+ ),
224
+ ...view.caveats.map((caveat) => styleText("dim", ` ${caveat}`)),
225
+ ];
226
+ }
227
+
228
+ function renderNavigations(crux) {
229
+ const view = navigationView(crux);
230
+ if (!view) return [];
231
+ return ["", styleText("dim", view.line), ...(view.hint ? [` ${view.hint}`] : [])];
232
+ }
233
+
234
+ // "LCP 2.88s → 3.10s ▲ +7.7% worsening ▁▅█"
235
+ function renderFieldTrend(history) {
236
+ if (history.unavailable) return [styleText("dim", `Field data trend: not available (${history.unavailable})`)];
237
+ const { title, rows } = historyView(history);
238
+ const style = { improving: "green", worsening: "red" };
239
+ return [
240
+ styleText("bold", title),
241
+ ...rows.map(
242
+ (row) =>
243
+ ` ${pad(row.metric, 6)}${pad(`${formatValue(row.first, row.unit)} → ${formatValue(row.last, row.unit)}`, 20)}${pad(row.change, 10)}${pad(style[row.trend] ? styleText(style[row.trend], row.trend) : row.trend, 12)}${row.sparkline}`
244
+ ),
245
+ ];
211
246
  }
212
247
 
213
248
  // The runs line, and the median, min and max of each metric across the runs, with how much they agree
@@ -1,4 +1,4 @@
1
- import { fieldDataView } from "../crux.js";
1
+ import { fieldDataView, historyView, subPartsView, navigationView } from "../crux.js";
2
2
  import { describeResult } from "../view.js";
3
3
  import { isSummaryResult, summaryNumbers } from "./summary-view.js";
4
4
  import { describeThrottling } from "../throttling.js";
@@ -102,7 +102,8 @@ function renderSection(title, results) {
102
102
  }
103
103
 
104
104
  function renderFieldData(crux, results) {
105
- if (crux.unavailable) return [`> Field data: not available (${crux.unavailable})`];
105
+ const trend = crux.history ? ["", ...renderFieldTrend(crux.history)] : [];
106
+ if (crux.unavailable) return [`> Field data: not available (${crux.unavailable})`, ...trend];
106
107
  const { title, rows } = fieldDataView(crux, results);
107
108
  return [
108
109
  `### ${title}`,
@@ -112,6 +113,40 @@ function renderFieldData(crux, results) {
112
113
  const synthetic = row.synthetic == null ? "n/a" : formatValue(row.synthetic, row.unit);
113
114
  return `| ${row.metric} | ${formatValue(row.p75, row.unit)} | ${RATING_ICON[row.rating] ?? "·"} ${row.rating} | ${synthetic} |`;
114
115
  }),
116
+ ...renderFieldSubParts(crux, results),
117
+ ...renderNavigations(crux),
118
+ ...trend,
119
+ ];
120
+ }
121
+
122
+ function renderFieldSubParts(crux, results) {
123
+ const view = subPartsView(crux, results);
124
+ if (!view) return [];
125
+ return [
126
+ "",
127
+ "### LCP sub-parts (CrUX p75 against the lab)",
128
+ "| Sub-part | Field p75 | Lab | Note |",
129
+ "|----------|-----------|-----|------|",
130
+ ...view.rows.map((row) => `| ${row.label} | ${formatValue(row.field, "ms")} | ${row.lab === null ? "n/a" : formatValue(row.lab, "ms")} | ${row.note ?? ""} |`),
131
+ "",
132
+ ...view.caveats.map((caveat) => `> ${caveat}`),
133
+ ];
134
+ }
135
+
136
+ function renderNavigations(crux) {
137
+ const view = navigationView(crux);
138
+ if (!view) return [];
139
+ return ["", `> ${view.line}`, ...(view.hint ? [">", `> ${view.hint}`] : [])];
140
+ }
141
+
142
+ function renderFieldTrend(history) {
143
+ if (history.unavailable) return [`> Field data trend: not available (${history.unavailable})`];
144
+ const { title, rows } = historyView(history);
145
+ return [
146
+ `### ${title}`,
147
+ "| Metric | First | Last | Change | Trend | Weeks |",
148
+ "|--------|-------|------|--------|-------|-------|",
149
+ ...rows.map((row) => `| ${row.metric} | ${formatValue(row.first, row.unit)} | ${formatValue(row.last, row.unit)} | ${row.change} | ${row.trend} | ${row.sparkline} |`),
115
150
  ];
116
151
  }
117
152
 
package/src/runs.js CHANGED
@@ -4,6 +4,11 @@ import { DEFAULT_THRESHOLDS } from "./baseline.js";
4
4
  // Every result of the report comes from that run, so they agree with each other (the sub-parts of
5
5
  // the LCP add up to its LCP), and a summary gives the median, min and max of each metric.
6
6
 
7
+ // The most runs of a measurement, the same in the tools of the MCP server as in --runs. Mann-Whitney U
8
+ // needs five a side (baseline.js), so nine leave room for failed runs and for a noisy page, and a page
9
+ // that takes a few seconds to load stays under two minutes.
10
+ export const MAX_RUNS = 9;
11
+
7
12
  // The metrics the snippets report, and navMs, which every run has
8
13
  export const MEDIAN_METRICS = ["LCP", "FCP", "TTFB", "CLS", "INP", "navMs"];
9
14
 
@@ -50,6 +55,23 @@ export function describeRuns({ count, failed, medianRun, medianBy, medianByReque
50
55
  return `${count} run${count === 1 ? "" : "s"}, median by ${by}: run ${medianRun}${failed ? ` (${failed} failed)` : ""}`;
51
56
  }
52
57
 
58
+ // Measures `count` runs one after the other, never in parallel: they would compete for the CPU and
59
+ // skew the timings. A run that fails is kept as { error } and left out of the median, unless isFatal
60
+ // says the next run would fail the same way. An aborted signal, a caller that left, starts no more runs.
61
+ export async function measureRuns(measure, count, { medianBy, isFatal = () => false, signal } = {}) {
62
+ const runs = [];
63
+ for (let i = 1; i <= count; i++) {
64
+ signal?.throwIfAborted();
65
+ try {
66
+ runs.push(await measure(i));
67
+ } catch (err) {
68
+ if (isFatal(err)) throw err;
69
+ runs.push({ error: err.message });
70
+ }
71
+ }
72
+ return summarizeRuns(runs, { medianBy });
73
+ }
74
+
53
75
  // The runs as the CLI made them, in order, a run that failed being { error }. Returns the payload of
54
76
  // the median run and the `runs` field of the report, or nulls and the errors when every run failed.
55
77
  export function summarizeRuns(runs, { medianBy: requested } = {}) {