pagespeed-quest 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. package/CHANGELOG.md +24 -0
  2. package/README.ja.md +47 -0
  3. package/README.md +49 -0
  4. package/build/cli.d.ts +8 -0
  5. package/build/cli.js +263 -0
  6. package/build/command.js +3 -229
  7. package/build/lighthouse-summary.d.ts +3 -1
  8. package/build/lighthouse-summary.js +31 -46
  9. package/build/lighthouse.d.ts +0 -1
  10. package/build/lighthouse.js +6 -2
  11. package/build/llm.d.ts +20 -0
  12. package/build/llm.js +42 -0
  13. package/llmdocs/00-guide.md +146 -0
  14. package/llmdocs/10-inventory.md +157 -0
  15. package/llmdocs/20-experiments.md +93 -0
  16. package/llmdocs/30-measuring.md +158 -0
  17. package/llmdocs/90-commands.md +188 -0
  18. package/package.json +13 -7
  19. package/scripts/build-llm-docs.js +79 -0
  20. package/scripts/postinstall.js +2 -3
  21. package/scripts/validate-plugin-skills.js +117 -0
  22. package/build/docs/assets/css/main.css +0 -1
  23. package/build/docs/assets/images/icons.png +0 -0
  24. package/build/docs/assets/images/icons@2x.png +0 -0
  25. package/build/docs/assets/images/widgets.png +0 -0
  26. package/build/docs/assets/images/widgets@2x.png +0 -0
  27. package/build/docs/assets/js/main.js +0 -51
  28. package/build/docs/classes/inventoryrepository.html +0 -305
  29. package/build/docs/classes/playbackproxy.html +0 -524
  30. package/build/docs/classes/proxy.html +0 -474
  31. package/build/docs/classes/recordingproxy.html +0 -532
  32. package/build/docs/classes/testproxy.html +0 -497
  33. package/build/docs/classes/throttle.html +0 -425
  34. package/build/docs/classes/throttlinglog.html +0 -174
  35. package/build/docs/classes/throttlingtransform.html +0 -6297
  36. package/build/docs/globals.html +0 -1131
  37. package/build/docs/index.html +0 -348
  38. package/build/docs/interfaces/beautifyresult.html +0 -174
  39. package/build/docs/interfaces/contentencodingpair.html +0 -174
  40. package/build/docs/interfaces/inventory.html +0 -174
  41. package/build/docs/interfaces/lighthouseoptions.html +0 -230
  42. package/build/docs/interfaces/playbacktransaction.html +0 -272
  43. package/build/docs/interfaces/proxyoptions.html +0 -423
  44. package/build/docs/interfaces/recordingsession.html +0 -174
  45. package/build/docs/interfaces/recordingtransaction.html +0 -286
  46. package/build/docs/interfaces/resource.html +0 -314
  47. package/build/docs/interfaces/transaction.html +0 -258
  48. package/build/docs/interfaces/withproxyoptions.html +0 -418
  49. package/build/logger.d.ts +0 -2
  50. package/build/logger.js +0 -15
  51. package/build/proxy.d.ts +0 -35
  52. package/build/proxy.js +0 -96
  53. package/build/throttling.d.ts +0 -33
  54. package/build/throttling.js +0 -89
@@ -0,0 +1,146 @@
1
+ # pagespeed-quest — reference for AI agents
2
+
3
+ PageSpeed Quest (`psq`) records a real page load through an HTTP proxy, writes
4
+ it to disk as an editable set of files, and replays it with the **original
5
+ timing** reproduced. You then edit those files — the HTML, the CSS, the images,
6
+ even the per-resource latency — replay again, and measure the difference.
7
+
8
+ The point is that you do **not** need the site's development environment, its
9
+ repository, or a deploy. Any public page can be frozen into a local, editable,
10
+ repeatable copy of itself. A hypothesis like "inlining critical CSS would gain
11
+ 0.4s of LCP" becomes something you can answer in minutes, on someone else's
12
+ site, before writing a line of production code.
13
+
14
+ Recording is powered by [rust-http-playback-proxy]. Measurement is delegated to
15
+ [Lighthouse] (scores and metrics), [loadshow] (a video of the load) and
16
+ [static-webshot] (screenshots and visual diffs). `psq` is the layer that makes
17
+ the recording **editable** and the comparison **repeatable**.
18
+
19
+ [rust-http-playback-proxy]: https://github.com/ideamans/rust-http-playback-proxy
20
+ [Lighthouse]: https://developer.chrome.com/docs/lighthouse/overview/
21
+ [loadshow]: https://github.com/ideamans/go-loadshow
22
+ [static-webshot]: https://github.com/ideamans/static-webshot
23
+
24
+ This reference ships inside the package, so `psq llm` always describes the exact
25
+ version installed in the project you are working in.
26
+
27
+ ## Ground rules
28
+
29
+ 1. **Record once. Compare against that recording, never against the live site.**
30
+ The live site changes between runs; the recording does not. That is the whole
31
+ value proposition. A "before" measured against the network and an "after"
32
+ measured against the inventory are not comparable numbers.
33
+
34
+ 2. **Copy the inventory before you edit it.** There is no undo, and the baseline
35
+ is the thing you are comparing to.
36
+
37
+ ```sh
38
+ cp -r inventory baseline
39
+ ```
40
+
41
+ Then measure either side by pointing `-i` at it. Keep `baseline/` untouched
42
+ for the rest of the quest.
43
+
44
+ 3. **`-i/--inventory` is a root option — it goes before the subcommand.**
45
+ `psq -i ./baseline lighthouse playback` works; `psq lighthouse playback -i
46
+ ./baseline` does not. The same applies one level down: `-a`, `-t`, `-l` and
47
+ `-c` belong to the `lighthouse` / `loadshow` group, so they go *before*
48
+ `recording` / `playback`.
49
+
50
+ 4. **`playback` takes no URL.** The entry URL and the device type are stored in
51
+ the inventory at recording time and reused. If you need to change either,
52
+ record again.
53
+
54
+ 5. **Lighthouse reports the load as observed** (`--throttling-method=provided`,
55
+ `--throttling.*=0`, `cpuSlowdownMultiplier=1`). Timing is reproduced by the
56
+ proxy, per resource, so Lighthouse must neither throttle again nor simulate.
57
+ Its default `simulate` method re-derives the metrics from a network model
58
+ that ignores the proxy — psq before 0.12 did that, and its numbers could even
59
+ fall below the document's TTFB. Do not compare scores across that boundary.
60
+
61
+ 6. **Unmatched requests get a 404.** The playback proxy only serves what is in
62
+ the inventory. That is deliberate — falling through to the network would make
63
+ the replay non-deterministic. So a URL you delete from `index.json` becomes a
64
+ failed request, not a request that never happens. To model "this resource is
65
+ gone", also remove the reference to it from the HTML.
66
+
67
+ 7. **Never conclude from one run.** Runs of the same recording are close but
68
+ not identical, and an occasional run is an outlier. Measure each state at
69
+ least three times, compare medians, and treat a difference smaller than the
70
+ spread within each state as no difference.
71
+
72
+ 8. **The recording run is not the baseline.** `psq lighthouse recording` also
73
+ writes a Lighthouse report, but it measured the live site through the
74
+ recording proxy. Send it somewhere out of the way
75
+ (`psq lighthouse -a ./artifacts/recording recording <url>`) and measure the
76
+ baseline by playback.
77
+
78
+ ## The loop
79
+
80
+ ```sh
81
+ # 1. Freeze the page (writes ./inventory)
82
+ npx psq lighthouse -a ./artifacts/recording recording https://example.com/
83
+
84
+ # 2. Keep the untouched copy, and a copy that accumulates adopted changes
85
+ cp -r inventory baseline
86
+ cp -r inventory accepted
87
+
88
+ # 3. Measure the baseline, three runs → artifacts/baseline-N/summary.md, lighthouse.report.json
89
+ for n in 1 2 3; do npx psq -i ./baseline lighthouse -a ./artifacts/baseline-$n playback; done
90
+
91
+ # 4. Each trial: start from accepted, make ONE edit to ./inventory (chapter "Experiments")
92
+ rm -rf inventory && cp -r accepted inventory
93
+
94
+ # 5. Measure the trial, three runs, and check it still renders (chapter "Measuring")
95
+ for n in 1 2 3; do npx psq lighthouse -a ./artifacts/t01-$n playback; done
96
+
97
+ # 6. Better beyond the spread and visually unchanged → adopt it
98
+ rm -rf accepted && cp -r inventory accepted
99
+
100
+ # 7. At the end, measure accepted against baseline from scratch
101
+ for n in 1 2 3; do npx psq -i ./accepted lighthouse -a ./artifacts/final-$n playback; done
102
+ ```
103
+
104
+ `summary.md` is a short Markdown digest (metrics, per-metric scores with their
105
+ Lighthouse weights, traffic by resource type). Medians across runs and the
106
+ evidence for *why* a page is slow come from `lighthouse.report.json` — see
107
+ "Measuring".
108
+
109
+ ## What lives where
110
+
111
+ | Path | What it is |
112
+ | --- | --- |
113
+ | `inventory/index.json` | the resource list and every resource's metadata. This is the file the proxy reads |
114
+ | `baseline/`, `accepted/` | conventional copies: the page as recorded, and the page with the adopted changes |
115
+ | `inventory/contents/<method>/<protocol>/<host>/<path>` | one file per recorded response body; `contentFilePath` gives the exact path |
116
+ | `artifacts/lighthouse.report.html`, `.report.json` | the full Lighthouse report |
117
+ | `artifacts/summary.md` | the digest to diff |
118
+ | `artifacts/lighthouse.digest.png` | score and metrics as an image |
119
+ | `artifacts/loadshow.mp4` | filmstrip video of the load |
120
+ | `artifacts/capture.png`, `capture-diff.png`, `capture-diff.txt` | screenshot and visual diff |
121
+
122
+ ## Failure modes
123
+
124
+ | Symptom | Cause | Fix |
125
+ | --- | --- | --- |
126
+ | `playback` serves nothing, every request 404s | the inventory is empty, or `-i` points at the wrong directory | record first; check `inventory/index.json` exists and `resources` is non-empty |
127
+ | `unknown option '-i'` / the option is ignored | root and group options were placed after the subcommand | `psq -i ./baseline lighthouse -a ./art playback` |
128
+ | the "after" is faster but the page is visibly broken | a referenced resource was deleted from `index.json` but not from the HTML | run `psq capture --compare` and look at the diff before trusting any number |
129
+ | edits to a `.js` / `.css` file seem to have no effect on transfer size | the resource has `"minify": true`, so playback re-minifies before serving | expected — the wire size reflects the minified bytes, which is what the original site shipped |
130
+ | a smaller file loads no faster | the recording has no `formatVersion: 2` (made before psq 0.12) and replays the recorded duration | re-record |
131
+ | the page looks fine but LCP got worse for no reason | an edit changed the byte size of a resource, and transfer time scales with size at the recorded `mbps` | that *is* the simulation working; check `summary.md` traffic figures |
132
+ | recording captured analytics and ad beacons | third parties were not excluded | re-record with `-x 'google-analytics|doubleclick|facebook'` (repeatable) |
133
+ | `loadshow` / `static-webshot` not found | the postinstall step that downloads them did not run | re-run `npm rebuild pagespeed-quest`, or set `LOADSHOW_PATH` / `WEBSHOT_PATH` |
134
+ | Lighthouse cannot be found | it resolves to `./node_modules/.bin/lighthouse` | run inside the quest project, or set `LIGHTHOUSE_PATH` |
135
+ | numbers drift between identical runs | normal measurement noise | three runs or more per state; compare medians against the spread |
136
+ | the baseline came from the recording run | that report measured the live site | measure the baseline with `psq -i ./baseline lighthouse … playback` |
137
+ | trial deltas cannot be attributed | a trial was built on the previous trial, or bundled several edits | start every trial from a fresh copy of `accepted/`, one edit each |
138
+
139
+ ## What this tool will not do
140
+
141
+ - It does not crawl. One entry URL plus whatever that page load requests.
142
+ - It does not modify the live site, and nothing you edit locally reaches it.
143
+ - It does not rewrite your resources for you, and this reference does not
144
+ prescribe optimizations. Deciding what to change is the work; this reference
145
+ covers how an edit reaches the replay and how to measure it.
146
+ - It does not replay user interaction. What is recorded is one page load.
@@ -0,0 +1,157 @@
1
+ # The inventory — what you edit
2
+
3
+ The inventory is the reverse-engineered page. Everything the browser will
4
+ receive during playback comes from here, and nothing else does.
5
+
6
+ ```
7
+ inventory/
8
+ ├── index.json ← the resource list. THIS is what the proxy reads
9
+ ├── originals/ ← the recorded wire bytes of reformatted bodies. Never edit
10
+ └── contents/
11
+ └── get/
12
+ └── https/
13
+ └── example.com/
14
+ ├── index.html
15
+ ├── assets/app.css
16
+ ├── assets/app.js
17
+ └── img/hero.jpg
18
+ ```
19
+
20
+ - **`index.json` is authoritative.** The playback proxy reads exactly that file
21
+ name. If a file called `inventory.json` also exists (older recordings wrote
22
+ both names), `psq` will read *that* one for the entry URL, device type and
23
+ traffic figures while the proxy serves from `index.json` — the two silently
24
+ disagree. Delete `inventory.json`, or keep it byte-identical.
25
+ - **`contents/` holds one file per response body.** Text bodies are stored
26
+ decoded and converted to UTF-8; minified CSS and JavaScript are also
27
+ pretty-printed (and marked `minify: true`), HTML is kept as recorded.
28
+ - **`originals/` keeps the bytes that went over the wire** for every body that
29
+ was stored differently (decompressed, re-encoded or pretty-printed). Playback
30
+ serves them for unedited resources. Do not edit or delete it.
31
+
32
+ ## How a URL becomes a path
33
+
34
+ `contents/<method, lower-case>/<protocol>/<host>/<pathname>`
35
+
36
+ | URL | File |
37
+ | --- | --- |
38
+ | `https://example.com/` | `contents/get/https/example.com/index.html` |
39
+ | `https://example.com/assets/app.css` | `contents/get/https/example.com/assets/app.css` |
40
+ | `https://example.com/app.js?v=1` | `contents/get/https/example.com/app~v%3D1.js` |
41
+
42
+ Rules: a trailing `/` gets `index.html`; the query string is URL-encoded and
43
+ inserted after `~`, before the extension; a query longer than 32 characters is
44
+ cut there and suffixed with `.~<sha1 of the rest>`. Do not guess and do not
45
+ rename these files by hand — `contentFilePath` in `index.json` (relative to the
46
+ inventory directory, so it starts with `contents/`) is the mapping, and the file
47
+ name is only a convenience:
48
+
49
+ ```sh
50
+ jq -r '.resources[] | [.url, .contentFilePath] | @tsv' inventory/index.json
51
+ ```
52
+
53
+ ## `index.json`
54
+
55
+ ```json
56
+ {
57
+ "entryUrl": "https://example.com/",
58
+ "deviceType": "mobile",
59
+ "formatVersion": 2,
60
+ "resources": [ ... ]
61
+ }
62
+ ```
63
+
64
+ `entryUrl` and `deviceType` are what `playback` uses instead of asking you for a
65
+ URL. Changing `deviceType` here changes the Lighthouse form factor and the
66
+ loadshow preset on the next playback — it does **not** re-record.
67
+ `formatVersion` is `2` for recordings made with psq 0.12 or later; without it
68
+ the older timing model applies (see below).
69
+
70
+ ### A resource
71
+
72
+ | Field | Effect at playback time |
73
+ | --- | --- |
74
+ | `method`, `url` | how the request is matched. A request that matches nothing gets a 404. Required |
75
+ | `ttfbMs` | **the proxy sleeps this long after receiving the request, before the first byte.** Per request, not an offset from page start. Required |
76
+ | `durationMs` | recorded transfer duration. Used as-is only by recordings without `formatVersion: 2` |
77
+ | `mbps` | transfer speed for the body, measured at recording. Used for an edited body. Missing → the median of the same host |
78
+ | `statusCode` | the status served |
79
+ | `rawHeaders` | headers served, minus `content-length` and `content-encoding`, which are recomputed |
80
+ | `contentEncoding` | `gzip` / `br` / `deflate` / `zstd` / `compress` / `identity`. The body is **re-compressed with this before it goes on the wire**, so it decides the transferred size |
81
+ | `contentTypeMime`, `contentCharset` | the `Content-Type` served. `contentCharset` re-encodes the body from UTF-8 on the way out |
82
+ | `contentFilePath` | the file under `contents/` to serve |
83
+ | `contentUtf8` / `contentBase64` | inline body, used only when there is no `contentFilePath`. Recordings do not write it |
84
+ | `minify` | `true` → the body is **re-minified before serving**. Set during recording when the original was minified |
85
+ | `httpVersion` | protocol version reported for the response |
86
+ | `original` | written by the recorder: the path of the recorded wire bytes under `originals/`, a SHA-256 of the file in `contents/`, and the `contentEncoding` / `contentCharset` / `minify` at recording time. **Do not edit** |
87
+
88
+ ### The timing model, precisely
89
+
90
+ ```
91
+ unedited body: recorded wire bytes, ttfbMs + the recorded transfer time
92
+ edited body: ttfbMs + (bytes on the wire × 8) / (mbps × 1,000,000)
93
+ ```
94
+
95
+ - A body is **unedited** while its file still matches the hash in `original`
96
+ and `contentEncoding`, `contentCharset` and `minify` are as recorded. It is
97
+ then served byte for byte as the site sent it — same compression, same size,
98
+ same time. Changing any of those makes it edited.
99
+ - `ttfbMs` is a fixed per-request delay. It models server think time plus
100
+ connection latency. Lower it to model a faster origin or a nearer CDN edge.
101
+ - For an edited body the transfer time is recomputed **from the bytes you leave
102
+ in the file**, after minification and after `contentEncoding` compression. So
103
+ shrinking a file makes it arrive sooner, with no other edit. `mbps` was
104
+ measured per resource at recording, so the same bytes take the time they took
105
+ on that connection.
106
+ - **Recordings without `formatVersion: 2`** (psq before 0.12) replay every body
107
+ in its recorded `durationMs` whatever its size, and re-minify text bodies with
108
+ an older minifier. Shrinking a file does not make it faster there. Re-record
109
+ to get the model above.
110
+ - `--full-throttle` (used internally by `psq capture`) skips both. Never compare
111
+ a full-throttle run with a normal one.
112
+
113
+ ### Two traps worth stating plainly
114
+
115
+ **`minify: true` re-minifies your edit.** The pretty-printed file in `contents/`
116
+ is for you; the wire sees the minified form. This keeps the byte count honest —
117
+ the original site shipped minified bytes — but it means whitespace edits have no
118
+ size effect, and it means a syntax error introduced during editing can surface
119
+ as a minifier failure rather than a browser error.
120
+
121
+ **`contentEncoding` decides the transferred size, not the file size.** A 300 KB
122
+ JavaScript file with `"contentEncoding": "br"` is perhaps 60 KB on the wire, and
123
+ that 60 KB is what the timing is computed from. To model "we turned compression
124
+ off", set it to `identity`; to model "we finally enabled Brotli", set it to
125
+ `br`. Do not edit `content-length` — it is recomputed and your value is
126
+ discarded.
127
+
128
+ ## Editing safely
129
+
130
+ 1. Copy `inventory/` to `baseline/` (and to `accepted/`) before the first edit.
131
+ 2. Change **one thing at a time**, then measure. Two edits in one run means you
132
+ cannot attribute the delta. Start each trial from a fresh copy of
133
+ `accepted/` (see "The loop" in the guide).
134
+ 3. After a structural edit (removing a script, changing a URL), run
135
+ `psq capture --compare` and look at the picture. A faster broken page is easy
136
+ to produce and easy to mistake for a win.
137
+ 4. Keep the inventory in version control if the quest lasts more than a day.
138
+ `git diff` over `contents/` is a readable record of the hypothesis.
139
+
140
+ ## Live editing with a browser
141
+
142
+ ```sh
143
+ psq proxy -p 8080
144
+ ```
145
+
146
+ serves the inventory and **watches it**: any change under `inventory/` restarts
147
+ the proxy, so a browser pointed at `http://localhost:8080` picks up your edit on
148
+ the next reload. The proxy uses a dummy certificate, so the browser has to be
149
+ told to accept it:
150
+
151
+ ```sh
152
+ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
153
+ --ignore-certificate-errors --proxy-server=http://localhost:8080
154
+ ```
155
+
156
+ This is the fastest way to iterate on markup edits before spending a Lighthouse
157
+ run on them.
@@ -0,0 +1,93 @@
1
+ # Experiments — how an edit reaches the replay
2
+
3
+ This chapter is about the mechanics of changing a recording so that a replay
4
+ reflects a change you want to test. **It does not prescribe optimizations** —
5
+ deciding what to try is the agent's or the engineer's judgement. What matters
6
+ here is that the edit models what you intend, and nothing else.
7
+
8
+ Paths below are relative to the inventory directory.
9
+
10
+ ## Content edits
11
+
12
+ Edit the file a resource's `contentFilePath` points to. The browser receives and
13
+ executes exactly those bytes, so effects on parsing, script execution, layout
14
+ and paint are **measured, not estimated**, and the transfer time follows the new
15
+ size (see the timing model in "The inventory" — this needs a recording with
16
+ `formatVersion: 2`, i.e. made with psq 0.12 or later).
17
+
18
+ psq-specific rules:
19
+
20
+ - **Changing an image's format** — replace the bytes at the same path, then set
21
+ the served type to match:
22
+
23
+ ```json
24
+ { "contentTypeMime": "image/webp", "rawHeaders": { "content-type": "image/webp" } }
25
+ ```
26
+
27
+ The URL may keep its old extension; browsers go by `Content-Type`. Never edit
28
+ `content-length`.
29
+ - **`minify: true`** — the file is re-minified before serving. Whitespace has no
30
+ effect on the wire; a syntax error may surface as a minifier failure.
31
+ - **Leave `original` and `originals/` alone.** They are how playback recognises
32
+ the resources you did not touch and serves them exactly as recorded.
33
+ - **`ttfbMs` does not move with the content.** A preloaded, reprioritised or
34
+ shrunk resource still waits its recorded `ttfbMs` before its first byte.
35
+
36
+ ## Adding and removing resources
37
+
38
+ - **Removing**: delete the reference from the HTML (or CSS, or script) in the
39
+ same edit. An entry deleted while still referenced becomes a 404 — a failed
40
+ request, which makes the result look worse than the truth. Removing a parent
41
+ tag also removes whatever it would have injected; say so when reporting.
42
+ - **Adding** (a split file, a self-hosted font, a new preload): add an entry to
43
+ `resources`. `method`, `url` and `ttfbMs` are required — an entry missing them
44
+ makes `index.json` unreadable. Set `statusCode`, `contentTypeMime` and
45
+ `contentFilePath`, and copy `ttfbMs` / `mbps` from a comparable resource on
46
+ the same host so the timing is not invented. Any URL the browser requests
47
+ that has no entry gets a 404.
48
+
49
+ ## Metadata edits — what a front-end change cannot do
50
+
51
+ These levers do not exist in a normal development environment, and they are the
52
+ reason for the `index.json` fields.
53
+
54
+ | To model | Edit | Note |
55
+ | --- | --- | --- |
56
+ | a faster or slower origin | lower / raise `ttfbMs`, the document first | separates the back-end share of the load from the front-end share |
57
+ | a faster or slower link, a CDN | `mbps` on every resource of that host | per host, not per file, or the result resembles nothing real |
58
+ | compression on or off | `contentEncoding` (`br`, `gzip`, `identity`) on text resources | the proxy re-compresses; the reverse quantifies what compression already buys |
59
+ | a different form factor | top-level `deviceType` | changes the Lighthouse form factor and the loadshow preset; does not re-record |
60
+
61
+ ## What this cannot model
62
+
63
+ State these plainly when reporting results.
64
+
65
+ - **Caching across page views.** Every playback is a cold load, so
66
+ `cache-insight` and `bf-cache` findings cannot be acted on in a replay.
67
+ - **HTTP/2 prioritisation and connection contention.** Each host is served by
68
+ the local proxy; the real contention on a shared connection is not
69
+ reproduced, and `modern-http-insight` describes the proxy, not the origin.
70
+ - **Server-side changes that alter the HTML.** You can hand-edit the HTML to
71
+ what the server *would* emit, but the edit is your hypothesis, not a
72
+ measurement of it.
73
+ - **Anything after the load.** Interaction, INP, scrolling — a recording is one
74
+ page load.
75
+ - **Variable content.** Personalised or A/B-tested pages are frozen at the
76
+ moment of recording, which is a feature for comparison and a limitation for
77
+ representativeness.
78
+
79
+ ## Reporting a trial
80
+
81
+ A useful line answers four questions: what was changed, what evidence led to it,
82
+ what moved (medians of several runs, reference → trial), and whether the page
83
+ still looks right.
84
+
85
+ ```
86
+ Trial : t01 defer tag manager
87
+ Evidence : render-blocking-insight lists gtm.js
88
+ Edit : contents/get/https/example.com/index.html — added defer to <script src="gtm.js">
89
+ Reference : Score 61 LCP 3.41s TBT 890ms (median of 3, artifacts/baseline-*)
90
+ Trial : Score 68 LCP 2.95s TBT 640ms (median of 3, artifacts/t01-defer-gtm-*)
91
+ Visual : no diff (artifacts/t01-capture/capture-diff.txt)
92
+ Decision : adopted
93
+ ```
@@ -0,0 +1,158 @@
1
+ # Measuring — scores, video, screenshots
2
+
3
+ Three measurement surfaces, all driven through the same playback proxy so they
4
+ see the same page. Choose by the question being asked.
5
+
6
+ | Question | Command | Read |
7
+ | --- | --- | --- |
8
+ | Did the score move? | `psq lighthouse playback` | `artifacts/summary.md` |
9
+ | What does the difference look like? | `psq loadshow playback` | `artifacts/loadshow.mp4` |
10
+ | Did I break the page? | `psq capture --compare <png>` | `artifacts/capture-diff.png` |
11
+
12
+ Keep the two sides in separate artifact directories. `-a` is an option of the
13
+ `lighthouse` / `loadshow` group, so it goes before the sub-subcommand.
14
+
15
+ ## Lighthouse, several runs per state
16
+
17
+ ```sh
18
+ for n in 1 2 3; do psq -i ./baseline lighthouse -a ./artifacts/baseline-$n playback; done
19
+ for n in 1 2 3; do psq -i ./inventory lighthouse -a ./artifacts/t01-$n playback; done
20
+ ```
21
+
22
+ One directory per run — the same `-a` twice overwrites. Name them
23
+ `<label>-<n>`, the label naming the state.
24
+
25
+ Each run writes:
26
+
27
+ | File | Contents |
28
+ | --- | --- |
29
+ | `summary.md` | metrics, per-metric scores with weights, overall score, traffic and resource size by type |
30
+ | `lighthouse.report.html` | the full report (open it when a number needs explaining) |
31
+ | `lighthouse.report.json` | the same, machine-readable — the source for medians and for the diagnosis |
32
+ | `lighthouse.digest.png` | score gauge and metrics as one image — the thing to paste into a report |
33
+ | `lighthouse-0.trace.json`, `*.devtoolslog.json` | saved assets (`--save-assets`) |
34
+
35
+ A playback run takes roughly 20–30 seconds: Lighthouse measures the load as
36
+ observed (`--throttling-method=provided`) and waits for the page to go quiet.
37
+
38
+ Options on the `lighthouse` group: `-a/--artifacts` (default `./artifacts`),
39
+ `-t/--timeout` (ms, default 30000), `-l/--laud` — show the browser window and
40
+ open the report at the end. `--laud` is for watching a run, not for measuring;
41
+ a visible browser is not the same environment as the headless one.
42
+
43
+ ### Medians and spread
44
+
45
+ **Never conclude from one run.** Runs of the same recording are close but not
46
+ identical, and an occasional run is an outlier. Take the median of at least three runs
47
+ per state, and call a difference real only when it is larger than the spread
48
+ (max − min) within each state:
49
+
50
+ ```sh
51
+ jq -s -r '
52
+ def median: sort | if length % 2 == 1 then .[length/2|floor] else (.[length/2-1] + .[length/2]) / 2 end;
53
+ map({Score: (.categories.performance.score*100),
54
+ LCP: .audits["largest-contentful-paint"].numericValue,
55
+ FCP: .audits["first-contentful-paint"].numericValue,
56
+ SI: .audits["speed-index"].numericValue,
57
+ TBT: .audits["total-blocking-time"].numericValue,
58
+ CLS: .audits["cumulative-layout-shift"].numericValue}) as $r
59
+ | ["Score","LCP","FCP","SI","TBT","CLS"][] as $m | [$r[][$m]] as $v
60
+ | "\($m)\t\($v|median)\t\($v|min)\t\($v|max)"' artifacts/baseline-*/lighthouse.report.json
61
+ ```
62
+
63
+ ### Where the time goes
64
+
65
+ `lighthouse.report.json` holds the evidence. With
66
+ `J=artifacts/baseline-1/lighthouse.report.json`:
67
+
68
+ | Question | jq |
69
+ | --- | --- |
70
+ | which audits flag a problem | `.audits[] \| select(.score != null and .score < 1) \| [.id, .displayValue, .metricSavings]` |
71
+ | the LCP element | `.audits["lcp-breakdown-insight"].details.items[] \| select(.type=="node") \| .snippet` |
72
+ | how LCP time splits | `.audits["lcp-breakdown-insight"].details.items[] \| select(.type=="table") \| .items` |
73
+ | the request waterfall | `.audits["network-requests"].details.items` (`networkRequestTime`, `networkEndTime`, `transferSize`, `url`) |
74
+ | script cost per URL | `.audits["bootup-time"].details.items` |
75
+ | third parties | `.audits["third-parties-insight"].details.items` |
76
+
77
+ Lighthouse 13 keeps most of its diagnosis in `*-insight` audits. To act on a
78
+ URL, look up its `contentFilePath` in `index.json`. `cache-insight`, `bf-cache`,
79
+ `modern-http-insight` and interaction metrics describe things a replay cannot
80
+ model (see "Experiments").
81
+
82
+ ## Video of the load, before and after
83
+
84
+ ```sh
85
+ psq -i ./baseline loadshow -a ./artifacts/video-baseline playback
86
+ psq -i ./accepted loadshow -a ./artifacts/video-after playback
87
+ ```
88
+
89
+ Each writes `<artifacts>/loadshow.mp4` — a filmstrip of the load, laid out in
90
+ 3 columns for mobile recordings and 2 for desktop, plus
91
+ `<artifacts>/loadshow/summary.md` and the intermediate frames in
92
+ `<artifacts>/loadshow/`.
93
+
94
+ Use `-c/--credit` to stamp a label into the video, which is what makes a pair of
95
+ videos legible side by side:
96
+
97
+ ```sh
98
+ psq -i ./baseline loadshow -a ./artifacts/video-baseline -c "Before" playback
99
+ psq -i ./accepted loadshow -a ./artifacts/video-after -c "After — deferred GTM" playback
100
+ ```
101
+
102
+ To place them side by side, stack the two MP4s with ffmpeg; `psq` does not
103
+ compose videos:
104
+
105
+ ```sh
106
+ ffmpeg -i artifacts/video-baseline/loadshow.mp4 -i artifacts/video-after/loadshow.mp4 \
107
+ -filter_complex hstack -c:v libx264 before-after.mp4
108
+ ```
109
+
110
+ ### Recording *with* loadshow rather than Lighthouse
111
+
112
+ ```sh
113
+ psq loadshow recording https://example.com/
114
+ ```
115
+
116
+ Lighthouse and loadshow drive the browser slightly differently. If the goal of
117
+ the quest is the video rather than the score, record with `loadshow recording`
118
+ so that the recording and the playback are driven the same way.
119
+
120
+ ## Screenshot and visual diff
121
+
122
+ ```sh
123
+ psq -i ./baseline capture -a ./artifacts/baseline-capture
124
+ psq capture -a ./artifacts/t01-capture --compare ./artifacts/baseline-capture/capture.png \
125
+ --baseline-label "Before" --current-label "After"
126
+ ```
127
+
128
+ `capture` runs the proxy in **full-throttle** mode — no TTFB, no bandwidth
129
+ simulation — because the question is what the page looks like when fully
130
+ loaded, not when. Never quote a timing number from a `capture` run.
131
+
132
+ Output: `capture.png` (rendered at 400×1600, resized to 200×800),
133
+ `capture-diff.png`, and `capture-diff.txt` with a text digest of the difference.
134
+
135
+ Run this after **every** structural edit. A deleted script that also removed a
136
+ banner will make every metric better and the page wrong.
137
+
138
+ ## Reading `summary.md`
139
+
140
+ ```
141
+ ## Metrics
142
+ - LCP 2780 ms
143
+ - CLS 0.02
144
+ ...
145
+ ## Scores
146
+ - LCP 0.71 x0.25
147
+ - TBT 0.88 x0.30
148
+ - **Overall** 0.61
149
+ ## Traffic: Type / Count / Size / Bytes
150
+ - script / 14 / 512.3 KB / 524595
151
+ ```
152
+
153
+ *Traffic* is what the browser received in this run, as Lighthouse measured it
154
+ (after compression, headers included); *Resource* is the decoded body size.
155
+ Both come from the report's `network-requests` audit, so they count every
156
+ request the page made, including 404s. A change that moves resource size but
157
+ not traffic size means you edited something that compresses away — expect
158
+ little timing effect.