pagespeed-quest 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.ja.md +47 -0
- package/README.md +49 -0
- package/build/cli.d.ts +8 -0
- package/build/cli.js +263 -0
- package/build/command.js +3 -229
- package/build/lighthouse-summary.d.ts +3 -1
- package/build/lighthouse-summary.js +31 -46
- package/build/lighthouse.d.ts +0 -1
- package/build/lighthouse.js +6 -2
- package/build/llm.d.ts +20 -0
- package/build/llm.js +42 -0
- package/llmdocs/00-guide.md +146 -0
- package/llmdocs/10-inventory.md +157 -0
- package/llmdocs/20-experiments.md +93 -0
- package/llmdocs/30-measuring.md +158 -0
- package/llmdocs/90-commands.md +188 -0
- package/package.json +13 -7
- package/scripts/build-llm-docs.js +79 -0
- package/scripts/postinstall.js +2 -3
- package/scripts/validate-plugin-skills.js +117 -0
- package/build/docs/assets/css/main.css +0 -1
- package/build/docs/assets/images/icons.png +0 -0
- package/build/docs/assets/images/icons@2x.png +0 -0
- package/build/docs/assets/images/widgets.png +0 -0
- package/build/docs/assets/images/widgets@2x.png +0 -0
- package/build/docs/assets/js/main.js +0 -51
- package/build/docs/classes/inventoryrepository.html +0 -305
- package/build/docs/classes/playbackproxy.html +0 -524
- package/build/docs/classes/proxy.html +0 -474
- package/build/docs/classes/recordingproxy.html +0 -532
- package/build/docs/classes/testproxy.html +0 -497
- package/build/docs/classes/throttle.html +0 -425
- package/build/docs/classes/throttlinglog.html +0 -174
- package/build/docs/classes/throttlingtransform.html +0 -6297
- package/build/docs/globals.html +0 -1131
- package/build/docs/index.html +0 -348
- package/build/docs/interfaces/beautifyresult.html +0 -174
- package/build/docs/interfaces/contentencodingpair.html +0 -174
- package/build/docs/interfaces/inventory.html +0 -174
- package/build/docs/interfaces/lighthouseoptions.html +0 -230
- package/build/docs/interfaces/playbacktransaction.html +0 -272
- package/build/docs/interfaces/proxyoptions.html +0 -423
- package/build/docs/interfaces/recordingsession.html +0 -174
- package/build/docs/interfaces/recordingtransaction.html +0 -286
- package/build/docs/interfaces/resource.html +0 -314
- package/build/docs/interfaces/transaction.html +0 -258
- package/build/docs/interfaces/withproxyoptions.html +0 -418
- package/build/logger.d.ts +0 -2
- package/build/logger.js +0 -15
- package/build/proxy.d.ts +0 -35
- package/build/proxy.js +0 -96
- package/build/throttling.d.ts +0 -33
- package/build/throttling.js +0 -89
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# pagespeed-quest — reference for AI agents
|
|
2
|
+
|
|
3
|
+
PageSpeed Quest (`psq`) records a real page load through an HTTP proxy, writes
|
|
4
|
+
it to disk as an editable set of files, and replays it with the **original
|
|
5
|
+
timing** reproduced. You then edit those files — the HTML, the CSS, the images,
|
|
6
|
+
even the per-resource latency — replay again, and measure the difference.
|
|
7
|
+
|
|
8
|
+
The point is that you do **not** need the site's development environment, its
|
|
9
|
+
repository, or a deploy. Any public page can be frozen into a local, editable,
|
|
10
|
+
repeatable copy of itself. A hypothesis like "inlining critical CSS would gain
|
|
11
|
+
0.4s of LCP" becomes something you can answer in minutes, on someone else's
|
|
12
|
+
site, before writing a line of production code.
|
|
13
|
+
|
|
14
|
+
Recording is powered by [rust-http-playback-proxy]. Measurement is delegated to
|
|
15
|
+
[Lighthouse] (scores and metrics), [loadshow] (a video of the load) and
|
|
16
|
+
[static-webshot] (screenshots and visual diffs). `psq` is the layer that makes
|
|
17
|
+
the recording **editable** and the comparison **repeatable**.
|
|
18
|
+
|
|
19
|
+
[rust-http-playback-proxy]: https://github.com/ideamans/rust-http-playback-proxy
|
|
20
|
+
[Lighthouse]: https://developer.chrome.com/docs/lighthouse/overview/
|
|
21
|
+
[loadshow]: https://github.com/ideamans/go-loadshow
|
|
22
|
+
[static-webshot]: https://github.com/ideamans/static-webshot
|
|
23
|
+
|
|
24
|
+
This reference ships inside the package, so `psq llm` always describes the exact
|
|
25
|
+
version installed in the project you are working in.
|
|
26
|
+
|
|
27
|
+
## Ground rules
|
|
28
|
+
|
|
29
|
+
1. **Record once. Compare against that recording, never against the live site.**
|
|
30
|
+
The live site changes between runs; the recording does not. That is the whole
|
|
31
|
+
value proposition. A "before" measured against the network and an "after"
|
|
32
|
+
measured against the inventory are not comparable numbers.
|
|
33
|
+
|
|
34
|
+
2. **Copy the inventory before you edit it.** There is no undo, and the baseline
|
|
35
|
+
is the thing you are comparing to.
|
|
36
|
+
|
|
37
|
+
```sh
|
|
38
|
+
cp -r inventory baseline
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Then measure either side by pointing `-i` at it. Keep `baseline/` untouched
|
|
42
|
+
for the rest of the quest.
|
|
43
|
+
|
|
44
|
+
3. **`-i/--inventory` is a root option — it goes before the subcommand.**
|
|
45
|
+
`psq -i ./baseline lighthouse playback` works; `psq lighthouse playback -i
|
|
46
|
+
./baseline` does not. The same applies one level down: `-a`, `-t`, `-l` and
|
|
47
|
+
`-c` belong to the `lighthouse` / `loadshow` group, so they go *before*
|
|
48
|
+
`recording` / `playback`.
|
|
49
|
+
|
|
50
|
+
4. **`playback` takes no URL.** The entry URL and the device type are stored in
|
|
51
|
+
the inventory at recording time and reused. If you need to change either,
|
|
52
|
+
record again.
|
|
53
|
+
|
|
54
|
+
5. **Lighthouse reports the load as observed** (`--throttling-method=provided`,
|
|
55
|
+
`--throttling.*=0`, `cpuSlowdownMultiplier=1`). Timing is reproduced by the
|
|
56
|
+
proxy, per resource, so Lighthouse must neither throttle again nor simulate.
|
|
57
|
+
Its default `simulate` method re-derives the metrics from a network model
|
|
58
|
+
that ignores the proxy — psq before 0.12 did that, and its numbers could even
|
|
59
|
+
fall below the document's TTFB. Do not compare scores across that boundary.
|
|
60
|
+
|
|
61
|
+
6. **Unmatched requests get a 404.** The playback proxy only serves what is in
|
|
62
|
+
the inventory. That is deliberate — falling through to the network would make
|
|
63
|
+
the replay non-deterministic. So a URL you delete from `index.json` becomes a
|
|
64
|
+
failed request, not a request that never happens. To model "this resource is
|
|
65
|
+
gone", also remove the reference to it from the HTML.
|
|
66
|
+
|
|
67
|
+
7. **Never conclude from one run.** Runs of the same recording are close but
|
|
68
|
+
not identical, and an occasional run is an outlier. Measure each state at
|
|
69
|
+
least three times, compare medians, and treat a difference smaller than the
|
|
70
|
+
spread within each state as no difference.
|
|
71
|
+
|
|
72
|
+
8. **The recording run is not the baseline.** `psq lighthouse recording` also
|
|
73
|
+
writes a Lighthouse report, but it measured the live site through the
|
|
74
|
+
recording proxy. Send it somewhere out of the way
|
|
75
|
+
(`psq lighthouse -a ./artifacts/recording recording <url>`) and measure the
|
|
76
|
+
baseline by playback.
|
|
77
|
+
|
|
78
|
+
## The loop
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
# 1. Freeze the page (writes ./inventory)
|
|
82
|
+
npx psq lighthouse -a ./artifacts/recording recording https://example.com/
|
|
83
|
+
|
|
84
|
+
# 2. Keep the untouched copy, and a copy that accumulates adopted changes
|
|
85
|
+
cp -r inventory baseline
|
|
86
|
+
cp -r inventory accepted
|
|
87
|
+
|
|
88
|
+
# 3. Measure the baseline, three runs → artifacts/baseline-N/summary.md, lighthouse.report.json
|
|
89
|
+
for n in 1 2 3; do npx psq -i ./baseline lighthouse -a ./artifacts/baseline-$n playback; done
|
|
90
|
+
|
|
91
|
+
# 4. Each trial: start from accepted, make ONE edit to ./inventory (chapter "Experiments")
|
|
92
|
+
rm -rf inventory && cp -r accepted inventory
|
|
93
|
+
|
|
94
|
+
# 5. Measure the trial, three runs, and check it still renders (chapter "Measuring")
|
|
95
|
+
for n in 1 2 3; do npx psq lighthouse -a ./artifacts/t01-$n playback; done
|
|
96
|
+
|
|
97
|
+
# 6. Better beyond the spread and visually unchanged → adopt it
|
|
98
|
+
rm -rf accepted && cp -r inventory accepted
|
|
99
|
+
|
|
100
|
+
# 7. At the end, measure accepted against baseline from scratch
|
|
101
|
+
for n in 1 2 3; do npx psq -i ./accepted lighthouse -a ./artifacts/final-$n playback; done
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
`summary.md` is a short Markdown digest (metrics, per-metric scores with their
|
|
105
|
+
Lighthouse weights, traffic by resource type). Medians across runs and the
|
|
106
|
+
evidence for *why* a page is slow come from `lighthouse.report.json` — see
|
|
107
|
+
"Measuring".
|
|
108
|
+
|
|
109
|
+
## What lives where
|
|
110
|
+
|
|
111
|
+
| Path | What it is |
|
|
112
|
+
| --- | --- |
|
|
113
|
+
| `inventory/index.json` | the resource list and every resource's metadata. This is the file the proxy reads |
|
|
114
|
+
| `baseline/`, `accepted/` | conventional copies: the page as recorded, and the page with the adopted changes |
|
|
115
|
+
| `inventory/contents/<method>/<protocol>/<host>/<path>` | one file per recorded response body; `contentFilePath` gives the exact path |
|
|
116
|
+
| `artifacts/lighthouse.report.html`, `.report.json` | the full Lighthouse report |
|
|
117
|
+
| `artifacts/summary.md` | the digest to diff |
|
|
118
|
+
| `artifacts/lighthouse.digest.png` | score and metrics as an image |
|
|
119
|
+
| `artifacts/loadshow.mp4` | filmstrip video of the load |
|
|
120
|
+
| `artifacts/capture.png`, `capture-diff.png`, `capture-diff.txt` | screenshot and visual diff |
|
|
121
|
+
|
|
122
|
+
## Failure modes
|
|
123
|
+
|
|
124
|
+
| Symptom | Cause | Fix |
|
|
125
|
+
| --- | --- | --- |
|
|
126
|
+
| `playback` serves nothing, every request 404s | the inventory is empty, or `-i` points at the wrong directory | record first; check `inventory/index.json` exists and `resources` is non-empty |
|
|
127
|
+
| `unknown option '-i'` / the option is ignored | root and group options were placed after the subcommand | `psq -i ./baseline lighthouse -a ./art playback` |
|
|
128
|
+
| the "after" is faster but the page is visibly broken | a referenced resource was deleted from `index.json` but not from the HTML | run `psq capture --compare` and look at the diff before trusting any number |
|
|
129
|
+
| edits to a `.js` / `.css` file seem to have no effect on transfer size | the resource has `"minify": true`, so playback re-minifies before serving | expected — the wire size reflects the minified bytes, which is what the original site shipped |
|
|
130
|
+
| a smaller file loads no faster | the recording has no `formatVersion: 2` (made before psq 0.12) and replays the recorded duration | re-record |
|
|
131
|
+
| the page looks fine but LCP got worse for no reason | an edit changed the byte size of a resource, and transfer time scales with size at the recorded `mbps` | that *is* the simulation working; check `summary.md` traffic figures |
|
|
132
|
+
| recording captured analytics and ad beacons | third parties were not excluded | re-record with `-x 'google-analytics|doubleclick|facebook'` (repeatable) |
|
|
133
|
+
| `loadshow` / `static-webshot` not found | the postinstall step that downloads them did not run | re-run `npm rebuild pagespeed-quest`, or set `LOADSHOW_PATH` / `WEBSHOT_PATH` |
|
|
134
|
+
| Lighthouse cannot be found | it resolves to `./node_modules/.bin/lighthouse` | run inside the quest project, or set `LIGHTHOUSE_PATH` |
|
|
135
|
+
| numbers drift between identical runs | normal measurement noise | three runs or more per state; compare medians against the spread |
|
|
136
|
+
| the baseline came from the recording run | that report measured the live site | measure the baseline with `psq -i ./baseline lighthouse … playback` |
|
|
137
|
+
| trial deltas cannot be attributed | a trial was built on the previous trial, or bundled several edits | start every trial from a fresh copy of `accepted/`, one edit each |
|
|
138
|
+
|
|
139
|
+
## What this tool will not do
|
|
140
|
+
|
|
141
|
+
- It does not crawl. One entry URL plus whatever that page load requests.
|
|
142
|
+
- It does not modify the live site, and nothing you edit locally reaches it.
|
|
143
|
+
- It does not rewrite your resources for you, and this reference does not
|
|
144
|
+
prescribe optimizations. Deciding what to change is the work; this reference
|
|
145
|
+
covers how an edit reaches the replay and how to measure it.
|
|
146
|
+
- It does not replay user interaction. What is recorded is one page load.
|
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
# The inventory — what you edit
|
|
2
|
+
|
|
3
|
+
The inventory is the reverse-engineered page. Everything the browser will
|
|
4
|
+
receive during playback comes from here, and nothing else does.
|
|
5
|
+
|
|
6
|
+
```
|
|
7
|
+
inventory/
|
|
8
|
+
├── index.json ← the resource list. THIS is what the proxy reads
|
|
9
|
+
├── originals/ ← the recorded wire bytes of reformatted bodies. Never edit
|
|
10
|
+
└── contents/
|
|
11
|
+
└── get/
|
|
12
|
+
└── https/
|
|
13
|
+
└── example.com/
|
|
14
|
+
├── index.html
|
|
15
|
+
├── assets/app.css
|
|
16
|
+
├── assets/app.js
|
|
17
|
+
└── img/hero.jpg
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
- **`index.json` is authoritative.** The playback proxy reads exactly that file
|
|
21
|
+
name. If a file called `inventory.json` also exists (older recordings wrote
|
|
22
|
+
both names), `psq` will read *that* one for the entry URL, device type and
|
|
23
|
+
traffic figures while the proxy serves from `index.json` — the two silently
|
|
24
|
+
disagree. Delete `inventory.json`, or keep it byte-identical.
|
|
25
|
+
- **`contents/` holds one file per response body.** Text bodies are stored
|
|
26
|
+
decoded and converted to UTF-8; minified CSS and JavaScript are also
|
|
27
|
+
pretty-printed (and marked `minify: true`), HTML is kept as recorded.
|
|
28
|
+
- **`originals/` keeps the bytes that went over the wire** for every body that
|
|
29
|
+
was stored differently (decompressed, re-encoded or pretty-printed). Playback
|
|
30
|
+
serves them for unedited resources. Do not edit or delete it.
|
|
31
|
+
|
|
32
|
+
## How a URL becomes a path
|
|
33
|
+
|
|
34
|
+
`contents/<method, lower-case>/<protocol>/<host>/<pathname>`
|
|
35
|
+
|
|
36
|
+
| URL | File |
|
|
37
|
+
| --- | --- |
|
|
38
|
+
| `https://example.com/` | `contents/get/https/example.com/index.html` |
|
|
39
|
+
| `https://example.com/assets/app.css` | `contents/get/https/example.com/assets/app.css` |
|
|
40
|
+
| `https://example.com/app.js?v=1` | `contents/get/https/example.com/app~v%3D1.js` |
|
|
41
|
+
|
|
42
|
+
Rules: a trailing `/` gets `index.html`; the query string is URL-encoded and
|
|
43
|
+
inserted after `~`, before the extension; a query longer than 32 characters is
|
|
44
|
+
cut there and suffixed with `.~<sha1 of the rest>`. Do not guess and do not
|
|
45
|
+
rename these files by hand — `contentFilePath` in `index.json` (relative to the
|
|
46
|
+
inventory directory, so it starts with `contents/`) is the mapping, and the file
|
|
47
|
+
name is only a convenience:
|
|
48
|
+
|
|
49
|
+
```sh
|
|
50
|
+
jq -r '.resources[] | [.url, .contentFilePath] | @tsv' inventory/index.json
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## `index.json`
|
|
54
|
+
|
|
55
|
+
```json
|
|
56
|
+
{
|
|
57
|
+
"entryUrl": "https://example.com/",
|
|
58
|
+
"deviceType": "mobile",
|
|
59
|
+
"formatVersion": 2,
|
|
60
|
+
"resources": [ ... ]
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`entryUrl` and `deviceType` are what `playback` uses instead of asking you for a
|
|
65
|
+
URL. Changing `deviceType` here changes the Lighthouse form factor and the
|
|
66
|
+
loadshow preset on the next playback — it does **not** re-record.
|
|
67
|
+
`formatVersion` is `2` for recordings made with psq 0.12 or later; without it
|
|
68
|
+
the older timing model applies (see below).
|
|
69
|
+
|
|
70
|
+
### A resource
|
|
71
|
+
|
|
72
|
+
| Field | Effect at playback time |
|
|
73
|
+
| --- | --- |
|
|
74
|
+
| `method`, `url` | how the request is matched. A request that matches nothing gets a 404. Required |
|
|
75
|
+
| `ttfbMs` | **the proxy sleeps this long after receiving the request, before the first byte.** Per request, not an offset from page start. Required |
|
|
76
|
+
| `durationMs` | recorded transfer duration. Used as-is only by recordings without `formatVersion: 2` |
|
|
77
|
+
| `mbps` | transfer speed for the body, measured at recording. Used for an edited body. Missing → the median of the same host |
|
|
78
|
+
| `statusCode` | the status served |
|
|
79
|
+
| `rawHeaders` | headers served, minus `content-length` and `content-encoding`, which are recomputed |
|
|
80
|
+
| `contentEncoding` | `gzip` / `br` / `deflate` / `zstd` / `compress` / `identity`. The body is **re-compressed with this before it goes on the wire**, so it decides the transferred size |
|
|
81
|
+
| `contentTypeMime`, `contentCharset` | the `Content-Type` served. `contentCharset` re-encodes the body from UTF-8 on the way out |
|
|
82
|
+
| `contentFilePath` | the file under `contents/` to serve |
|
|
83
|
+
| `contentUtf8` / `contentBase64` | inline body, used only when there is no `contentFilePath`. Recordings do not write it |
|
|
84
|
+
| `minify` | `true` → the body is **re-minified before serving**. Set during recording when the original was minified |
|
|
85
|
+
| `httpVersion` | protocol version reported for the response |
|
|
86
|
+
| `original` | written by the recorder: the path of the recorded wire bytes under `originals/`, a SHA-256 of the file in `contents/`, and the `contentEncoding` / `contentCharset` / `minify` at recording time. **Do not edit** |
|
|
87
|
+
|
|
88
|
+
### The timing model, precisely
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
unedited body: recorded wire bytes, ttfbMs + the recorded transfer time
|
|
92
|
+
edited body: ttfbMs + (bytes on the wire × 8) / (mbps × 1,000,000)
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
- A body is **unedited** while its file still matches the hash in `original`
|
|
96
|
+
and `contentEncoding`, `contentCharset` and `minify` are as recorded. It is
|
|
97
|
+
then served byte for byte as the site sent it — same compression, same size,
|
|
98
|
+
same time. Changing any of those makes it edited.
|
|
99
|
+
- `ttfbMs` is a fixed per-request delay. It models server think time plus
|
|
100
|
+
connection latency. Lower it to model a faster origin or a nearer CDN edge.
|
|
101
|
+
- For an edited body the transfer time is recomputed **from the bytes you leave
|
|
102
|
+
in the file**, after minification and after `contentEncoding` compression. So
|
|
103
|
+
shrinking a file makes it arrive sooner, with no other edit. `mbps` was
|
|
104
|
+
measured per resource at recording, so the same bytes take the time they took
|
|
105
|
+
on that connection.
|
|
106
|
+
- **Recordings without `formatVersion: 2`** (psq before 0.12) replay every body
|
|
107
|
+
in its recorded `durationMs` whatever its size, and re-minify text bodies with
|
|
108
|
+
an older minifier. Shrinking a file does not make it faster there. Re-record
|
|
109
|
+
to get the model above.
|
|
110
|
+
- `--full-throttle` (used internally by `psq capture`) skips both. Never compare
|
|
111
|
+
a full-throttle run with a normal one.
|
|
112
|
+
|
|
113
|
+
### Two traps worth stating plainly
|
|
114
|
+
|
|
115
|
+
**`minify: true` re-minifies your edit.** The pretty-printed file in `contents/`
|
|
116
|
+
is for you; the wire sees the minified form. This keeps the byte count honest —
|
|
117
|
+
the original site shipped minified bytes — but it means whitespace edits have no
|
|
118
|
+
size effect, and it means a syntax error introduced during editing can surface
|
|
119
|
+
as a minifier failure rather than a browser error.
|
|
120
|
+
|
|
121
|
+
**`contentEncoding` decides the transferred size, not the file size.** A 300 KB
|
|
122
|
+
JavaScript file with `"contentEncoding": "br"` is perhaps 60 KB on the wire, and
|
|
123
|
+
that 60 KB is what the timing is computed from. To model "we turned compression
|
|
124
|
+
off", set it to `identity`; to model "we finally enabled Brotli", set it to
|
|
125
|
+
`br`. Do not edit `content-length` — it is recomputed and your value is
|
|
126
|
+
discarded.
|
|
127
|
+
|
|
128
|
+
## Editing safely
|
|
129
|
+
|
|
130
|
+
1. Copy `inventory/` to `baseline/` (and to `accepted/`) before the first edit.
|
|
131
|
+
2. Change **one thing at a time**, then measure. Two edits in one run means you
|
|
132
|
+
cannot attribute the delta. Start each trial from a fresh copy of
|
|
133
|
+
`accepted/` (see "The loop" in the guide).
|
|
134
|
+
3. After a structural edit (removing a script, changing a URL), run
|
|
135
|
+
`psq capture --compare` and look at the picture. A faster broken page is easy
|
|
136
|
+
to produce and easy to mistake for a win.
|
|
137
|
+
4. Keep the inventory in version control if the quest lasts more than a day.
|
|
138
|
+
`git diff` over `contents/` is a readable record of the hypothesis.
|
|
139
|
+
|
|
140
|
+
## Live editing with a browser
|
|
141
|
+
|
|
142
|
+
```sh
|
|
143
|
+
psq proxy -p 8080
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
serves the inventory and **watches it**: any change under `inventory/` restarts
|
|
147
|
+
the proxy, so a browser pointed at `http://localhost:8080` picks up your edit on
|
|
148
|
+
the next reload. The proxy uses a dummy certificate, so the browser has to be
|
|
149
|
+
told to accept it:
|
|
150
|
+
|
|
151
|
+
```sh
|
|
152
|
+
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
|
|
153
|
+
--ignore-certificate-errors --proxy-server=http://localhost:8080
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
This is the fastest way to iterate on markup edits before spending a Lighthouse
|
|
157
|
+
run on them.
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Experiments — how an edit reaches the replay
|
|
2
|
+
|
|
3
|
+
This chapter is about the mechanics of changing a recording so that a replay
|
|
4
|
+
reflects a change you want to test. **It does not prescribe optimizations** —
|
|
5
|
+
deciding what to try is the agent's or the engineer's judgement. What matters
|
|
6
|
+
here is that the edit models what you intend, and nothing else.
|
|
7
|
+
|
|
8
|
+
Paths below are relative to the inventory directory.
|
|
9
|
+
|
|
10
|
+
## Content edits
|
|
11
|
+
|
|
12
|
+
Edit the file a resource's `contentFilePath` points to. The browser receives and
|
|
13
|
+
executes exactly those bytes, so effects on parsing, script execution, layout
|
|
14
|
+
and paint are **measured, not estimated**, and the transfer time follows the new
|
|
15
|
+
size (see the timing model in "The inventory" — this needs a recording with
|
|
16
|
+
`formatVersion: 2`, i.e. made with psq 0.12 or later).
|
|
17
|
+
|
|
18
|
+
psq-specific rules:
|
|
19
|
+
|
|
20
|
+
- **Changing an image's format** — replace the bytes at the same path, then set
|
|
21
|
+
the served type to match:
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{ "contentTypeMime": "image/webp", "rawHeaders": { "content-type": "image/webp" } }
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
The URL may keep its old extension; browsers go by `Content-Type`. Never edit
|
|
28
|
+
`content-length`.
|
|
29
|
+
- **`minify: true`** — the file is re-minified before serving. Whitespace has no
|
|
30
|
+
effect on the wire; a syntax error may surface as a minifier failure.
|
|
31
|
+
- **Leave `original` and `originals/` alone.** They are how playback recognises
|
|
32
|
+
the resources you did not touch and serves them exactly as recorded.
|
|
33
|
+
- **`ttfbMs` does not move with the content.** A preloaded, reprioritised or
|
|
34
|
+
shrunk resource still waits its recorded `ttfbMs` before its first byte.
|
|
35
|
+
|
|
36
|
+
## Adding and removing resources
|
|
37
|
+
|
|
38
|
+
- **Removing**: delete the reference from the HTML (or CSS, or script) in the
|
|
39
|
+
same edit. An entry deleted while still referenced becomes a 404 — a failed
|
|
40
|
+
request, which makes the result look worse than the truth. Removing a parent
|
|
41
|
+
tag also removes whatever it would have injected; say so when reporting.
|
|
42
|
+
- **Adding** (a split file, a self-hosted font, a new preload): add an entry to
|
|
43
|
+
`resources`. `method`, `url` and `ttfbMs` are required — an entry missing them
|
|
44
|
+
makes `index.json` unreadable. Set `statusCode`, `contentTypeMime` and
|
|
45
|
+
`contentFilePath`, and copy `ttfbMs` / `mbps` from a comparable resource on
|
|
46
|
+
the same host so the timing is not invented. Any URL the browser requests
|
|
47
|
+
that has no entry gets a 404.
|
|
48
|
+
|
|
49
|
+
## Metadata edits — what a front-end change cannot do
|
|
50
|
+
|
|
51
|
+
These levers do not exist in a normal development environment, and they are the
|
|
52
|
+
reason for the `index.json` fields.
|
|
53
|
+
|
|
54
|
+
| To model | Edit | Note |
|
|
55
|
+
| --- | --- | --- |
|
|
56
|
+
| a faster or slower origin | lower / raise `ttfbMs`, the document first | separates the back-end share of the load from the front-end share |
|
|
57
|
+
| a faster or slower link, a CDN | `mbps` on every resource of that host | per host, not per file, or the result resembles nothing real |
|
|
58
|
+
| compression on or off | `contentEncoding` (`br`, `gzip`, `identity`) on text resources | the proxy re-compresses; the reverse quantifies what compression already buys |
|
|
59
|
+
| a different form factor | top-level `deviceType` | changes the Lighthouse form factor and the loadshow preset; does not re-record |
|
|
60
|
+
|
|
61
|
+
## What this cannot model
|
|
62
|
+
|
|
63
|
+
State these plainly when reporting results.
|
|
64
|
+
|
|
65
|
+
- **Caching across page views.** Every playback is a cold load, so
|
|
66
|
+
`cache-insight` and `bf-cache` findings cannot be acted on in a replay.
|
|
67
|
+
- **HTTP/2 prioritisation and connection contention.** Each host is served by
|
|
68
|
+
the local proxy; the real contention on a shared connection is not
|
|
69
|
+
reproduced, and `modern-http-insight` describes the proxy, not the origin.
|
|
70
|
+
- **Server-side changes that alter the HTML.** You can hand-edit the HTML to
|
|
71
|
+
what the server *would* emit, but the edit is your hypothesis, not a
|
|
72
|
+
measurement of it.
|
|
73
|
+
- **Anything after the load.** Interaction, INP, scrolling — a recording is one
|
|
74
|
+
page load.
|
|
75
|
+
- **Variable content.** Personalised or A/B-tested pages are frozen at the
|
|
76
|
+
moment of recording, which is a feature for comparison and a limitation for
|
|
77
|
+
representativeness.
|
|
78
|
+
|
|
79
|
+
## Reporting a trial
|
|
80
|
+
|
|
81
|
+
A useful line answers four questions: what was changed, what evidence led to it,
|
|
82
|
+
what moved (medians of several runs, reference → trial), and whether the page
|
|
83
|
+
still looks right.
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
Trial : t01 defer tag manager
|
|
87
|
+
Evidence : render-blocking-insight lists gtm.js
|
|
88
|
+
Edit : contents/get/https/example.com/index.html — added defer to <script src="gtm.js">
|
|
89
|
+
Reference : Score 61 LCP 3.41s TBT 890ms (median of 3, artifacts/baseline-*)
|
|
90
|
+
Trial : Score 68 LCP 2.95s TBT 640ms (median of 3, artifacts/t01-defer-gtm-*)
|
|
91
|
+
Visual : no diff (artifacts/t01-capture/capture-diff.txt)
|
|
92
|
+
Decision : adopted
|
|
93
|
+
```
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# Measuring — scores, video, screenshots
|
|
2
|
+
|
|
3
|
+
Three measurement surfaces, all driven through the same playback proxy so they
|
|
4
|
+
see the same page. Choose by the question being asked.
|
|
5
|
+
|
|
6
|
+
| Question | Command | Read |
|
|
7
|
+
| --- | --- | --- |
|
|
8
|
+
| Did the score move? | `psq lighthouse playback` | `artifacts/summary.md` |
|
|
9
|
+
| What does the difference look like? | `psq loadshow playback` | `artifacts/loadshow.mp4` |
|
|
10
|
+
| Did I break the page? | `psq capture --compare <png>` | `artifacts/capture-diff.png` |
|
|
11
|
+
|
|
12
|
+
Keep the two sides in separate artifact directories. `-a` is an option of the
|
|
13
|
+
`lighthouse` / `loadshow` group, so it goes before the sub-subcommand.
|
|
14
|
+
|
|
15
|
+
## Lighthouse, several runs per state
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
for n in 1 2 3; do psq -i ./baseline lighthouse -a ./artifacts/baseline-$n playback; done
|
|
19
|
+
for n in 1 2 3; do psq -i ./inventory lighthouse -a ./artifacts/t01-$n playback; done
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
One directory per run — the same `-a` twice overwrites. Name them
|
|
23
|
+
`<label>-<n>`, the label naming the state.
|
|
24
|
+
|
|
25
|
+
Each run writes:
|
|
26
|
+
|
|
27
|
+
| File | Contents |
|
|
28
|
+
| --- | --- |
|
|
29
|
+
| `summary.md` | metrics, per-metric scores with weights, overall score, traffic and resource size by type |
|
|
30
|
+
| `lighthouse.report.html` | the full report (open it when a number needs explaining) |
|
|
31
|
+
| `lighthouse.report.json` | the same, machine-readable — the source for medians and for the diagnosis |
|
|
32
|
+
| `lighthouse.digest.png` | score gauge and metrics as one image — the thing to paste into a report |
|
|
33
|
+
| `lighthouse-0.trace.json`, `*.devtoolslog.json` | saved assets (`--save-assets`) |
|
|
34
|
+
|
|
35
|
+
A playback run takes roughly 20–30 seconds: Lighthouse measures the load as
|
|
36
|
+
observed (`--throttling-method=provided`) and waits for the page to go quiet.
|
|
37
|
+
|
|
38
|
+
Options on the `lighthouse` group: `-a/--artifacts` (default `./artifacts`),
|
|
39
|
+
`-t/--timeout` (ms, default 30000), `-l/--laud` — show the browser window and
|
|
40
|
+
open the report at the end. `--laud` is for watching a run, not for measuring;
|
|
41
|
+
a visible browser is not the same environment as the headless one.
|
|
42
|
+
|
|
43
|
+
### Medians and spread
|
|
44
|
+
|
|
45
|
+
**Never conclude from one run.** Runs of the same recording are close but not
|
|
46
|
+
identical, and an occasional run is an outlier. Take the median of at least three runs
|
|
47
|
+
per state, and call a difference real only when it is larger than the spread
|
|
48
|
+
(max − min) within each state:
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
jq -s -r '
|
|
52
|
+
def median: sort | if length % 2 == 1 then .[length/2|floor] else (.[length/2-1] + .[length/2]) / 2 end;
|
|
53
|
+
map({Score: (.categories.performance.score*100),
|
|
54
|
+
LCP: .audits["largest-contentful-paint"].numericValue,
|
|
55
|
+
FCP: .audits["first-contentful-paint"].numericValue,
|
|
56
|
+
SI: .audits["speed-index"].numericValue,
|
|
57
|
+
TBT: .audits["total-blocking-time"].numericValue,
|
|
58
|
+
CLS: .audits["cumulative-layout-shift"].numericValue}) as $r
|
|
59
|
+
| ["Score","LCP","FCP","SI","TBT","CLS"][] as $m | [$r[][$m]] as $v
|
|
60
|
+
| "\($m)\t\($v|median)\t\($v|min)\t\($v|max)"' artifacts/baseline-*/lighthouse.report.json
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### Where the time goes
|
|
64
|
+
|
|
65
|
+
`lighthouse.report.json` holds the evidence. With
|
|
66
|
+
`J=artifacts/baseline-1/lighthouse.report.json`:
|
|
67
|
+
|
|
68
|
+
| Question | jq |
|
|
69
|
+
| --- | --- |
|
|
70
|
+
| which audits flag a problem | `.audits[] \| select(.score != null and .score < 1) \| [.id, .displayValue, .metricSavings]` |
|
|
71
|
+
| the LCP element | `.audits["lcp-breakdown-insight"].details.items[] \| select(.type=="node") \| .snippet` |
|
|
72
|
+
| how LCP time splits | `.audits["lcp-breakdown-insight"].details.items[] \| select(.type=="table") \| .items` |
|
|
73
|
+
| the request waterfall | `.audits["network-requests"].details.items` (`networkRequestTime`, `networkEndTime`, `transferSize`, `url`) |
|
|
74
|
+
| script cost per URL | `.audits["bootup-time"].details.items` |
|
|
75
|
+
| third parties | `.audits["third-parties-insight"].details.items` |
|
|
76
|
+
|
|
77
|
+
Lighthouse 13 keeps most of its diagnosis in `*-insight` audits. To act on a
|
|
78
|
+
URL, look up its `contentFilePath` in `index.json`. `cache-insight`, `bf-cache`,
|
|
79
|
+
`modern-http-insight` and interaction metrics describe things a replay cannot
|
|
80
|
+
model (see "Experiments").
|
|
81
|
+
|
|
82
|
+
## Video of the load, before and after
|
|
83
|
+
|
|
84
|
+
```sh
|
|
85
|
+
psq -i ./baseline loadshow -a ./artifacts/video-baseline playback
|
|
86
|
+
psq -i ./accepted loadshow -a ./artifacts/video-after playback
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Each writes `<artifacts>/loadshow.mp4` — a filmstrip of the load, laid out in
|
|
90
|
+
3 columns for mobile recordings and 2 for desktop, plus
|
|
91
|
+
`<artifacts>/loadshow/summary.md` and the intermediate frames in
|
|
92
|
+
`<artifacts>/loadshow/`.
|
|
93
|
+
|
|
94
|
+
Use `-c/--credit` to stamp a label into the video, which is what makes a pair of
|
|
95
|
+
videos legible side by side:
|
|
96
|
+
|
|
97
|
+
```sh
|
|
98
|
+
psq -i ./baseline loadshow -a ./artifacts/video-baseline -c "Before" playback
|
|
99
|
+
psq -i ./accepted loadshow -a ./artifacts/video-after -c "After — deferred GTM" playback
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
To place them side by side, stack the two MP4s with ffmpeg; `psq` does not
|
|
103
|
+
compose videos:
|
|
104
|
+
|
|
105
|
+
```sh
|
|
106
|
+
ffmpeg -i artifacts/video-baseline/loadshow.mp4 -i artifacts/video-after/loadshow.mp4 \
|
|
107
|
+
-filter_complex hstack -c:v libx264 before-after.mp4
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
### Recording *with* loadshow rather than Lighthouse
|
|
111
|
+
|
|
112
|
+
```sh
|
|
113
|
+
psq loadshow recording https://example.com/
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Lighthouse and loadshow drive the browser slightly differently. If the goal of
|
|
117
|
+
the quest is the video rather than the score, record with `loadshow recording`
|
|
118
|
+
so that the recording and the playback are driven the same way.
|
|
119
|
+
|
|
120
|
+
## Screenshot and visual diff
|
|
121
|
+
|
|
122
|
+
```sh
|
|
123
|
+
psq -i ./baseline capture -a ./artifacts/baseline-capture
|
|
124
|
+
psq capture -a ./artifacts/t01-capture --compare ./artifacts/baseline-capture/capture.png \
|
|
125
|
+
--baseline-label "Before" --current-label "After"
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
`capture` runs the proxy in **full-throttle** mode — no TTFB, no bandwidth
|
|
129
|
+
simulation — because the question is what the page looks like when fully
|
|
130
|
+
loaded, not when. Never quote a timing number from a `capture` run.
|
|
131
|
+
|
|
132
|
+
Output: `capture.png` (rendered at 400×1600, resized to 200×800),
|
|
133
|
+
`capture-diff.png`, and `capture-diff.txt` with a text digest of the difference.
|
|
134
|
+
|
|
135
|
+
Run this after **every** structural edit. A deleted script that also removed a
|
|
136
|
+
banner will make every metric better and the page wrong.
|
|
137
|
+
|
|
138
|
+
## Reading `summary.md`
|
|
139
|
+
|
|
140
|
+
```
|
|
141
|
+
## Metrics
|
|
142
|
+
- LCP 2780 ms
|
|
143
|
+
- CLS 0.02
|
|
144
|
+
...
|
|
145
|
+
## Scores
|
|
146
|
+
- LCP 0.71 x0.25
|
|
147
|
+
- TBT 0.88 x0.30
|
|
148
|
+
- **Overall** 0.61
|
|
149
|
+
## Traffic: Type / Count / Size / Bytes
|
|
150
|
+
- script / 14 / 512.3 KB / 524595
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
*Traffic* is what the browser received in this run, as Lighthouse measured it
|
|
154
|
+
(after compression, headers included); *Resource* is the decoded body size.
|
|
155
|
+
Both come from the report's `network-requests` audit, so they count every
|
|
156
|
+
request the page made, including 404s. A change that moves resource size but
|
|
157
|
+
not traffic size means you edited something that compresses away — expect
|
|
158
|
+
little timing effect.
|