pagespeed-quest 0.10.0 → 0.11.0

package/llmdocs/20-experiments.md

@@ -0,0 +1,194 @@
+# Experiments — which edit models which optimization
+
+Each recipe is: the hypothesis, the exact edit, what should move, and what to
+watch out for. Make one edit, measure, record the delta, revert or keep. The
+metric names are the Lighthouse ones reported in `artifacts/summary.md`
+(LCP ×0.25, CLS ×0.25, TBT ×0.30, FCP ×0.10, SI ×0.10).
+
+Paths below are relative to the inventory directory.
+
+## Images
+
+### Compress or re-encode an image
+
+**Hypothesis** "the hero image is far heavier than it needs to be."
+
+Replace the file in place, keeping the same path:
+
+```sh
+cwebp -q 75 contents/GET/https/example.com/img/hero.jpg \
+  -o contents/GET/https/example.com/img/hero.jpg
+```
+
+Then, in `index.json`, set the served type to match the new bytes:
+
+```json
+{ "contentTypeMime": "image/webp", "rawHeaders": { "content-type": "image/webp" } }
+```
+
+**Moves** LCP, SI, and total transfer. Transfer time falls in proportion to the
+byte count at the recorded `mbps`.
+
+**Watch out** The URL still ends in `.jpg`; that is fine, browsers go by
+`Content-Type`. Do not touch `content-length`. If the format is not supported by
+the recorded device profile the image simply fails to decode — check
+`psq capture`.
+
+### Serve a correctly sized image
+
+Same edit, but resize rather than re-encode. This is usually the larger win on
+mobile recordings, because the recorded page often ships a desktop-sized asset.
+Compare `summary.md`'s image traffic line before and after.
+
+### Lazy-load below-the-fold images
+
+Edit the document: add `loading="lazy"` to `<img>` elements below the fold.
+
+**Moves** FCP and SI, sometimes LCP indirectly (bandwidth freed for the hero).
+Nothing needs to change in `index.json` — the lazily-loaded requests simply do
+not happen during the measured window.
+
+### Preload the LCP image
+
+```html
+<link rel="preload" as="image" href="/img/hero.webp" fetchpriority="high">
+```
+
+in `<head>`. **Moves** LCP. **Watch out** the recorded `ttfbMs` of that resource
+still applies as a fixed delay, so the win is bounded by it.
+
+## Scripts
+
+### Defer or async a render-blocking script
+
+Edit the document, add `defer` (order preserved) or `async` (order not).
+
+**Moves** FCP, LCP, SI. TBT often moves too, because the work lands after the
+first paint.
+
+### Remove a third-party tag
+
+Delete the `<script>` from the document. Optionally also delete the resource
+from `index.json` — but only after the reference is gone, or the browser will
+record a failed request (see the 404 rule).
+
+**Moves** TBT most visibly, plus whatever the tag was blocking. This is the
+single most informative experiment on most commercial pages: it puts a number on
+"what is the tag manager costing us".
+
+**Watch out** Some tags inject further requests. Removing the parent removes
+those too, which is realistic — but it makes the delta larger than the tag's own
+weight, so say so when reporting.
+
+### Shrink the bundle
+
+Replace the file under `contents/` with a smaller build (drop a polyfill, remove
+a library, run a real minifier). Because the browser actually executes what you
+serve, **TBT and main-thread time respond for real** — this is not a simulated
+number.
+
+## CSS and fonts
+
+### Inline critical CSS, defer the rest
+
+In the document, add a `<style>` block with the above-the-fold rules and switch
+the stylesheet link to a deferred load:
+
+```html
+<link rel="preload" href="/assets/app.css" as="style" onload="this.rel='stylesheet'">
+```
+
+**Moves** FCP, LCP, SI. **Watch out** CLS often gets *worse* if the inlined
+subset is incomplete — check `psq capture --compare` as well as the score.
+
+### Remove unused CSS
+
+Edit the stylesheet under `contents/`. Remember `minify: true` re-minifies, so
+the win shows up as fewer bytes, not as tidier formatting.
+
+### `font-display: swap` and font preloading
+
+Edit the `@font-face` block in the stylesheet, and/or add
+`<link rel="preload" as="font" crossorigin>` to the document.
+
+**Moves** FCP and LCP when text was the largest element; CLS if the swap causes
+reflow. Subsetting the font file itself is the transfer-size half of the same
+experiment.
+
+## Network and server
+
+These are the levers that do not exist in a normal development environment, and
+they are the reason for the `index.json` fields.
+
+### "What if the server responded faster?"
+
+Lower `ttfbMs` on the document resource (often 200–800 ms on a real recording):
+
+```json
+{ "url": "https://example.com/", "ttfbMs": 80 }
+```
+
+**Moves** everything downstream, roughly one-for-one. Run this early — it tells
+you how much of the budget is back-end and therefore *not* fixable in the
+front-end.
+
+### "What if we put it on a CDN / on a faster link?"
+
+Raise `mbps` on the resources served from the slow host. To model a slower
+network instead, lower it. Apply it to a whole host, not a single file, or the
+result will not resemble anything real.
+
+### "What if we finally enabled compression?"
+
+Set `contentEncoding` to `br` (or `gzip`) on the text resources that had
+`identity`. The proxy re-compresses and the transfer time follows.
+
+The reverse — setting `identity` on everything — quantifies what compression is
+already buying you, which is a useful number when someone proposes turning it
+off for CPU reasons.
+
+### "What if there were fewer requests?"
+
+Concatenate two files into one under `contents/`, update the document to
+reference the survivor, and delete the other entry from `index.json`. The win
+here is mostly the removed `ttfbMs`, so it is only worth it where that value is
+large.
+
+## Layout stability
+
+### Reserve space for images and embeds
+
+Add `width` / `height` (or an `aspect-ratio` rule) in the document or the
+stylesheet. **Moves** CLS, which carries the same weight as LCP — a cheap edit
+with a disproportionate score effect.
+
+## What this cannot model
+
+State these plainly when reporting results.
+
+- **Caching across page views.** Every playback is a cold load.
+- **HTTP/2 prioritisation and connection contention.** Each host is served by
+  its own local server; the real contention on a shared connection is not
+  reproduced.
+- **Server-side changes that alter the HTML.** You can hand-edit the HTML to
+  what the server *would* emit, but the edit is your hypothesis, not a
+  measurement of it.
+- **Anything after the load.** Interaction, INP, scrolling — a recording is one
+  page load.
+- **Variable content.** Personalised or A/B-tested pages are frozen at the
+  moment of recording, which is a feature for comparison and a limitation for
+  representativeness.
+
+## Reporting a result
+
+A useful report answers three questions: what was changed, what moved, and
+whether the page still looks right.
+
+```
+Hypothesis : defer the tag manager
+Edit       : removed <script src="gtm.js"> from contents/GET/https/example.com/index.html
+Before     : LCP 3.41s  TBT 890ms  Overall 42   (artifacts-baseline/summary.md)
+After      : LCP 2.78s  TBT 210ms  Overall 61   (artifacts/summary.md)
+Visual     : no diff    (artifacts/capture-diff.txt)
+Caveat     : removing the parent tag also removed 4 requests it injected
+```

package/llmdocs/30-measuring.md

@@ -0,0 +1,117 @@
+# Measuring — scores, video, screenshots
+
+Three measurement surfaces, all driven through the same playback proxy so they
+see the same page. Choose by the question being asked.
+
+| Question | Command | Read |
+| --- | --- | --- |
+| Did the score move? | `psq lighthouse playback` | `artifacts/summary.md` |
+| What does the difference look like? | `psq loadshow playback` | `artifacts/loadshow.mp4` |
+| Did I break the page? | `psq capture --compare <png>` | `artifacts/capture-diff.png` |
+
+Keep the two sides in separate artifact directories. `-a` is an option of the
+`lighthouse` / `loadshow` group, so it goes before the sub-subcommand.
+
+## Lighthouse before / after
+
+```sh
+psq -i ./baseline  lighthouse -a ./artifacts-baseline playback
+psq -i ./inventory lighthouse -a ./artifacts-after    playback
+diff artifacts-baseline/summary.md artifacts-after/summary.md
+```
+
+Each run writes:
+
+| File | Contents |
+| --- | --- |
+| `summary.md` | metrics, per-metric scores with weights, overall score, traffic and resource size by type |
+| `lighthouse.report.html` | the full report (open it when a number needs explaining) |
+| `lighthouse.report.json` | the same, machine-readable |
+| `lighthouse.digest.png` | score gauge and metrics as one image — the thing to paste into a report |
+| `lighthouse-0.trace.json`, `*.devtoolslog.json` | saved assets (`--save-assets`) |
+
+`summary.md` is the file to diff. It is deliberately short and stable, so the
+diff shows the delta and nothing else.
+
+Options on the `lighthouse` group: `-a/--artifacts` (default `./artifacts`),
+`-t/--timeout` (ms, default 30000), `-l/--laud` — show the browser window and
+open the report at the end. `--laud` is for watching a run, not for measuring;
+a visible browser is not the same environment as the headless one.
+
+**Repeat before concluding.** Lighthouse noise on a fixed recording is a few
+points. Three runs per side is usually enough to see whether a delta is real.
+
+## Video of the load, before and after
+
+```sh
+psq -i ./baseline  loadshow -a ./artifacts-baseline playback
+psq -i ./inventory loadshow -a ./artifacts-after    playback
+```
+
+Each writes `<artifacts>/loadshow.mp4` — a filmstrip of the load, laid out in
+3 columns for mobile recordings and 2 for desktop, plus
+`<artifacts>/loadshow/summary.md` and the intermediate frames in
+`<artifacts>/loadshow/`.
+
+Use `-c/--credit` to stamp a label into the video, which is what makes a pair of
+videos legible side by side:
+
+```sh
+psq -i ./baseline  loadshow -a ./artifacts-baseline -c "Before" playback
+psq -i ./inventory loadshow -a ./artifacts-after    -c "After — deferred GTM" playback
+```
+
+To place them side by side, stack the two MP4s with ffmpeg; `psq` does not
+compose videos:
+
+```sh
+ffmpeg -i artifacts-baseline/loadshow.mp4 -i artifacts-after/loadshow.mp4 \
+  -filter_complex hstack -c:v libx264 before-after.mp4
+```
+
+### Recording *with* loadshow rather than Lighthouse
+
+```sh
+psq loadshow recording https://example.com/
+```
+
+Lighthouse and loadshow drive the browser slightly differently. If the goal of
+the quest is the video rather than the score, record with `loadshow recording`
+so that the recording and the playback are driven the same way.
+
+## Screenshot and visual diff
+
+```sh
+psq -i ./baseline capture -a ./artifacts-baseline      # baseline.png
+psq capture --compare ./artifacts-baseline/capture.png \
+    --baseline-label "Before" --current-label "After"
+```
+
+`capture` runs the proxy in **full-throttle** mode — no TTFB, no bandwidth
+simulation — because the question is what the page looks like when fully
+loaded, not when. Never quote a timing number from a `capture` run.
+
+Output: `capture.png` (rendered at 400×1600, resized to 200×800),
+`capture-diff.png`, and `capture-diff.txt` with a text digest of the difference.
+
+Run this after **every** structural edit. A deleted script that also removed a
+banner will make every metric better and the page wrong.
+
+## Reading `summary.md`
+
+```
+## Metrics
+- LCP 2780 ms
+- CLS 0.02
+...
+## Scores
+- LCP 0.71 x0.25
+- TBT 0.88 x0.30
+- **Overall** 0.61
+## Traffic: Type / Count / Size / Bytes
+- script / 14 / 512.3 KB / 524595
+```
+
+*Traffic* is what crossed the wire (after compression); *Resource* is the
+decompressed size. A change that moves resource size but not traffic size means
+you edited something that compresses away — expect little timing effect.

package/llmdocs/90-commands.md

@@ -0,0 +1,181 @@
+# Command catalog
+
+<!-- Generated by scripts/build-llm-docs.js. Do not edit by hand. -->
+
+Options belong to the command they are declared on, so they must appear
+before the next subcommand: `psq -i <dir> lighthouse -a <dir> playback`.
+
+## psq
+
+Record a web page through a proxy, edit the recording, and replay it to measure the difference
+
+```
+Usage: psq [options] [command]
+
+Record a web page through a proxy, edit the recording, and replay it to measure the difference
+
+Options:
+  -v, --version          Output the version number
+  -i, --inventory <dir>  Inventory directory (default: "./inventory")
+  -h, --help             display help for command
+
+Commands:
+  lighthouse [options]   Run Lighthouse (performance category) via a proxy
+  loadshow [options]     Run loadshow via a proxy
+  proxy [options]        Run the playback (or recording) proxy standalone
+  capture [options]      Capture a screenshot via playback proxy and optionally compare with a baseline
+  llm [options]          Print the embedded reference for AI agents
+  help [command]         display help for command
+```
+
+### psq lighthouse
+
+Run Lighthouse (performance category) via a proxy
+
+```
+Usage: psq lighthouse [options] [command]
+
+Run Lighthouse (performance category) via a proxy
+
+Options:
+  -a, --artifacts <dir>      Artifacts directory (default: "./artifacts")
+  -l, --laud                 Show browser and open the report after completion (default: false)
+  -t, --timeout <ms>         Timeout milliseconds (default: "30000")
+  -h, --help                 display help for command
+
+Commands:
+  recording [options] <url>  Record contents by lighthouse
+  playback                   Playback contents for lighthouse
+  help [command]             display help for command
+```
+
+#### psq lighthouse recording
+
+Record contents by lighthouse
+
+```
+Usage: psq lighthouse recording [options] <url>
+
+Record contents by lighthouse
+
+Arguments:
+  url                            Url to measure performance
+
+Options:
+  -d, --device <mobile|desktop>  Device type (default: "mobile")
+  -x, --exclude <pattern>        Regex patterns to exclude URLs from recording (repeatable) (default: [])
+  -h, --help                     display help for command
+```
+
+#### psq lighthouse playback
+
+Playback contents for lighthouse
+
+```
+Usage: psq lighthouse playback [options]
+
+Playback contents for lighthouse
+
+Options:
+  -h, --help  display help for command
+```
+
+### psq loadshow
+
+Run loadshow via a proxy
+
+```
+Usage: psq loadshow [options] [command]
+
+Run loadshow via a proxy
+
+Options:
+  -a, --artifacts <dir>      Artifacts directory (default: "./artifacts")
+  -c, --credit <string>      Credit string
+  -t, --timeout <ms>         Timeout milliseconds (default: "30000")
+  -h, --help                 display help for command
+
+Commands:
+  recording [options] <url>  Record contents by loadshow
+  playback                   Playback contents for loadshow
+  help [command]             display help for command
+```
+
+#### psq loadshow recording
+
+Record contents by loadshow
+
+```
+Usage: psq loadshow recording [options] <url>
+
+Record contents by loadshow
+
+Arguments:
+  url                            Url to measure performance
+
+Options:
+  -d, --device <mobile|desktop>  Device type (default: "mobile")
+  -x, --exclude <pattern>        Regex patterns to exclude URLs from recording (repeatable) (default: [])
+  -h, --help                     display help for command
+```
+
+#### psq loadshow playback
+
+Playback contents for loadshow
+
+```
+Usage: psq loadshow playback [options]
+
+Playback contents for loadshow
+
+Options:
+  -h, --help  display help for command
+```
+
+### psq proxy
+
+Run the playback (or recording) proxy standalone
+
+```
+Usage: psq proxy [options]
+
+Run the playback (or recording) proxy standalone
+
+Options:
+  -p, --port <number>      Proxy port (default: "8080")
+  -r, --record <url>       Recording URL to start the proxy as recording mode (default: "")
+  -x, --exclude <pattern>  Regex patterns to exclude URLs from recording (repeatable) (default: [])
+  -h, --help               display help for command
+```
+
+### psq capture
+
+Capture a screenshot via playback proxy and optionally compare with a baseline
+
+```
+Usage: psq capture [options]
+
+Capture a screenshot via playback proxy and optionally compare with a baseline
+
+Options:
+  -a, --artifacts <dir>     Artifacts directory (default: "./artifacts")
+  -t, --timeout <ms>        Timeout milliseconds (default: "30000")
+  --compare <file>          Baseline PNG file to compare against
+  --baseline-label <label>  Label for the baseline image (default: "前回")
+  --current-label <label>   Label for the current image (default: "今回")
+  -h, --help                display help for command
+```
+
+### psq llm
+
+Print the embedded reference for AI agents
+
+```
+Usage: psq llm [options]
+
+Print the embedded reference for AI agents
+
+Options:
+  -f, --format <markdown|json>  Output format (default: "markdown")
+  -h, --help                    display help for command
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pagespeed-quest",
-  "version": "0.10.0",
+  "version": "0.11.0",
   "type": "module",
   "description": "A framework for efficient web front-end speed improvement",
   "main": "build/index.js",
@@ -17,16 +17,20 @@
     "psq": "build/command.js"
   },
   "scripts": {
-    "build": "run-p build:*",
+    "build": "run-s build:module build:llmdocs",
     "build:module": "tsc -p tsconfig.json",
+    "build:llmdocs": "node scripts/build-llm-docs.js",
+    "validate-plugin-skills": "node scripts/validate-plugin-skills.js",
     "fix": "run-s fix:*",
     "fix:prettier": "prettier \"src/**/*.ts\" --write",
     "fix:lint": "eslint src --ext .ts --fix",
-    "test": "run-s build test:*",
+    "test": "run-s build:module test:*",
     "test:lint": "eslint src --ext .ts",
     "test:prettier": "prettier \"src/**/*.ts\" --list-different",
     "test:spelling": "cspell \"{README.md,.github/*.md,src/**/*.ts}\"",
     "test:unit": "nyc --silent ava",
+    "test:plugins": "node scripts/validate-plugin-skills.js",
+    "test:llmdocs": "node scripts/build-llm-docs.js --check",
     "check-cli": "run-s test diff-integration-tests check-integration-tests",
     "check-integration-tests": "run-s check-integration-test:*",
     "diff-integration-tests": "mkdir -p diff && rm -rf diff/test && cp -r test diff/test && rm -rf diff/test/test-*/.git && cd diff && git init --quiet && git add -A && git commit --quiet --no-verify --allow-empty -m 'WIP' && echo '\\n\\nCommitted most recent integration test output in the \"diff\" directory. Review the changes with \"cd diff && git diff HEAD\" or your preferred git diff viewer.'",
@@ -47,10 +51,11 @@
     "adhoc": "yarn build && node build/adhoc.js",
     "command": "yarn build && node build/command.js",
     "prepublishOnly": "yarn build",
-    "postinstall": "node scripts/postinstall.js"
+    "postinstall": "node scripts/postinstall.js",
+    "ai:regen": "run-s build:module build:llmdocs validate-plugin-skills"
   },
   "engines": {
-    "node": ">=22"
+    "node": ">=22.19"
   },
   "dependencies": {
     "commander": "^11.1.0",
@@ -68,7 +73,7 @@
     "tmp-promise": "^3.0.3"
   },
   "devDependencies": {
-    "@ava/typescript": "^5.0.0",
+    "@ava/typescript": "^7.0.0",
     "@istanbuljs/nyc-config-typescript": "^1.0.1",
     "@types/node": "^20.8.9",
     "@typescript-eslint/eslint-plugin": "^4.0.1",
@@ -94,6 +99,7 @@
   "files": [
     "build",
     "scripts",
+    "llmdocs",
     "!**/*.spec.*",
     "!**/*.json",
     "CHANGELOG.md",

package/scripts/build-llm-docs.js

@@ -0,0 +1,70 @@
+#!/usr/bin/env node
+/**
+ * Generate llmdocs/90-commands.md from the actual `psq` command tree.
+ *
+ * Run through `yarn build` (and `yarn ai:regen`) after `tsc`, because it walks
+ * the compiled command definitions rather than a hand-written list. Never edit
+ * llmdocs/90-commands.md by hand — change the command definitions in
+ * src/cli.ts and run this again.
+ */
+
+import { readFile, writeFile } from 'fs/promises'
+import { dirname, join } from 'path'
+import { fileURLToPath } from 'url'
+
+const __dirname = dirname(fileURLToPath(import.meta.url))
+const projectRoot = join(__dirname, '..')
+const outputPath = join(projectRoot, 'llmdocs', '90-commands.md')
+
+const { createProgram } = await import(join(projectRoot, 'build', 'cli.js'))
+
+function walk(command, path, depth, lines) {
+  const heading = '#'.repeat(Math.min(depth + 1, 6))
+  const title = [...path, command.name()].join(' ')
+
+  lines.push(`${heading} ${title}`)
+  lines.push('')
+  const description = command.description()
+  if (description) {
+    lines.push(description)
+    lines.push('')
+  }
+  lines.push('```')
+  lines.push(command.helpInformation().trimEnd())
+  lines.push('```')
+  lines.push('')
+
+  for (const child of command.commands) {
+    walk(child, [...path, command.name()], depth + 1, lines)
+  }
+}
+
+const program = createProgram()
+
+const lines = [
+  '# Command catalog',
+  '',
+  '<!-- Generated by scripts/build-llm-docs.js. Do not edit by hand. -->',
+  '',
+  'Options belong to the command they are declared on, so they must appear',
+  'before the next subcommand: `psq -i <dir> lighthouse -a <dir> playback`.',
+  '',
+]
+
+walk(program, [], 1, lines)
+
+const generated = `${lines.join('\n').replace(/\n{3,}/g, '\n\n').trimEnd()}\n`
+
+if (process.argv.includes('--check')) {
+  const current = await readFile(outputPath, 'utf8').catch(() => '')
+  if (current !== generated) {
+    process.stderr.write(
+      `llmdocs/90-commands.md is out of date with the command definitions.\nRun \`yarn ai:regen\` and commit the result — do not edit it by hand.\n`
+    )
+    process.exit(1)
+  }
+  process.stdout.write('llmdocs/90-commands.md is up to date\n')
+} else {
+  await writeFile(outputPath, generated)
+  process.stdout.write(`Wrote ${outputPath}\n`)
+}