renderdiff 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,74 +1,79 @@
1
1
  # renderdiff
2
2
 
3
- Screenshot diff between two origins.
3
+ **Did my migration change how the site renders? A visual-parity check for coding agents.**
4
4
 
5
- Give it a list of routes and two base URLs. It screenshots each route on both sides at
6
- several viewport widths, pixel-diffs the pairs, and writes a scored HTML report with
7
- left / right / diff side by side, plus `results.json` and crops of the changes, sized to stay
8
- legible to coding agents.
5
+ renderdiff screenshots the same routes on two running copies of a site (your branch on
6
+ localhost, and production) at five viewport widths and pixel-diffs each pair. For every
7
+ change it tells you where on the page it is, which DOM element and CSS property is behind
8
+ it, and what text differs. It also writes small crops sized for an LLM's context. It refuses
9
+ to score what it can't trust: blank pages, error pages, pages still moving. A clean result
10
+ means the pixels really matched.
9
11
 
10
- Useful for the "did I break anything?" question a test suite can't answer: a framework
11
- migration, a CSS refactor, a dependency bump, a staging-vs-production check.
12
+ ### Who it's for
12
13
 
13
- ```
14
- renderdiff http://localhost:3000 https://example.com / /about /pricing
14
+ Developers who hand a **large website migration** to a coding agent (Claude Code, Codex,
15
+ Cursor), and the agents themselves. The goal of such a migration is *no visible change*, and
16
+ the agent needs a cheap, trustworthy loop: change → `renderdiff` → explain every difference
17
+ → repeat.
18
+
19
+ **Use it for:**
20
+
21
+ - framework migrations (e.g. Gatsby → Next.js, Vue 2 → 3, jQuery templates → React)
22
+ - CSS or design-system refactors (Sass → Tailwind, BEM → CSS modules, a token swap)
23
+ - template or component rewrites, CMS moves, static-site-generator swaps
24
+ - dependency and build upgrades (bundler, PostCSS, font loading)
25
+ - pages that moved during the migration (`/pricing::/plans`), and one component at a time (`--selector`)
26
+
27
+ **Don't use it for:**
28
+
29
+ - intentional redesigns: when everything is meant to change, the score says nothing
30
+ - cross-browser testing (Chromium only) or stored-baseline snapshot tests (both origins must be live)
31
+ - discovering routes: you list them; there is no crawler
32
+ - component libraries in isolation; Storybook-based tools fit that better
33
+
34
+ ### 30-second quickstart
35
+
36
+ ```bash
37
+ npx playwright install chromium # once per machine
38
+ npx renderdiff@latest http://localhost:3000 https://example.com / /about /pricing
15
39
  ```
16
40
 
17
- On a terminal it prints a line per shot as it lands, then the worst scores and the paths
18
- to `report.html` and `results.json`. Piped — to a file, CI or a coding agent — it prints
19
- one summary at the end instead, listing only what differs:
41
+ **Left (first origin) is your change; right (second) is the reference.** Piped, which is
42
+ what an agent sees, renderdiff prints only what differs:
20
43
 
21
44
  ```
22
45
  Routes: 3
23
- clean: /
46
+ clean: / /pricing
24
47
  /about clean @1152,896,704,375
25
48
  @1440 97.41% h-24 band -24@y196 aligned 100.00% text -"Pricing"
26
- /pricing
27
- @1440,1152,896,704,375 99.12%–99.87% 1 reg
28
- renderdiff: 2 of 3 routes differ (6 of 15 shots), 0 errors, worst 97.41% /about@1440 — /Users/you/site/renderdiff-out/results.json
29
- ```
30
-
31
- A row covers the widths that agree on everything but the score (shown as a range): height
32
- delta (`h`), bands (`+` rows only left has, `-` rows only right has, at `y`; `~` when the
33
- widths put it in different places), aligned score, region count, unstable pixels and up to
34
- three text-diff lines (`-"x"` only on the right, `+"x"` only on the left). Errors print in
35
- full, grouped across the widths that share a message. Clean means not one pixel differs
36
- and none went unmeasured as unstable; a route with unstable pixels gets its own row. With
37
- `--threshold`, the verdict line ends `, B below 97.00%`. `--progress` gets the per-shot
38
- output when piped, `--no-progress` the summary on a terminal.
39
-
40
- ## Install
41
-
42
- ```bash
43
- npm install -g renderdiff
44
- npx playwright install chromium # once, if you don't already have it
49
+ dom -nav > a[href="/pricing"] "Pricing"
50
+ crops about.1440.c1.png
51
+ renderdiff: 1 of 3 routes differ (1 of 15 shots), 0 errors, worst 97.41% /about@1440 — ./renderdiff-out/results.json
45
52
  ```
46
53
 
47
- Requires Node >= 22.18.
54
+ Read as: on `/about` at 1440px, your change is 24px shorter (`h-24`) because 24 rows that
55
+ only the reference has (`band -24`) are missing at y=196. With those rows cut out the page
56
+ matches exactly (`aligned 100.00%`). The missing rows are the "Pricing" nav link, which
57
+ exists only on the reference (`-`). Open `renderdiff-out/about.1440.c1.png` to see it.
48
58
 
49
- ### Agent skill
59
+ **Without `--threshold`, a diff still exits 0**, so read the summary.
50
60
 
51
- [`skills/renderdiff/SKILL.md`](./skills/renderdiff/SKILL.md) teaches a coding agent to use
52
- renderdiff without fooling itself: run it with `npx renderdiff@latest`, read the summary
53
- it prints, then the text diff, the bands and the region crops, and give a verdict that says
54
- what it looked at — not just a score. The longer reference (triage, rationalizations,
55
- config, gotchas) sits next to it in `reference.md` and is read only when needed.
61
+ ### For coding agents
56
62
 
57
- In Claude Code, as a plugin:
63
+ Install the skill so the agent runs the loop and reports a verdict, not a bare score:
58
64
 
59
65
  ```
60
- /plugin marketplace add tmashara/renderdiff
66
+ /plugin marketplace add tmashara/renderdiff # Claude Code
61
67
  /plugin install renderdiff@renderdiff
68
+ npx skills add tmashara/renderdiff # Codex, Cursor, others (skills.sh)
62
69
  ```
63
70
 
64
- For other agents (Codex, Cursor and so on), with [skills](https://skills.sh):
71
+ ### Requirements
65
72
 
66
- ```bash
67
- npx skills add tmashara/renderdiff
68
- ```
69
-
70
- Until renderdiff is on npm, `npx renderdiff@latest` won't resolve; install the CLI with
71
- `npm install -g github:tmashara/renderdiff`.
73
+ Node >= 22.18 and Chromium (`npx playwright install chromium`). `npm install -g renderdiff`
74
+ installs it globally. If the bundled Chromium won't launch, renderdiff falls back to an
75
+ installed Google Chrome on both sides and says so; `meta.browser` in `results.json` records
76
+ which ran.
72
77
 
73
78
  ## Usage
74
79
 
@@ -78,214 +83,176 @@ renderdiff <left-origin> <right-origin> <route…> [options]
78
83
 
79
84
  | Argument or flag | |
80
85
  | --- | --- |
81
- | `<origin>` | An argument starting with `http://` or `https://`. The first is left, the second right; a third is an error. |
86
+ | `<origin>` | An argument starting with `http://` or `https://`. The first is left (your change), the second right (the reference); a third is an error. |
82
87
  | `<route>` | A literal path, e.g. `/about`, or a pair `/pricing::/plans` (see below). |
83
88
  | `--left <url>` | Left origin, instead of the first positional one. Giving both is an error. |
84
89
  | `--right <url>` | Right origin, instead of the second positional one. |
85
90
  | `--width <n>` | Viewport width; repeatable. Replaces the defaults. |
86
91
  | `--threshold <0..1>` | Exit non-zero if any comparison scores below this. |
87
92
  | `--out <dir>` | Output directory, wiped at the start of a run. Default `./renderdiff-out`. A directory holding files renderdiff didn't write is refused. |
88
- | `--concurrency <n>` | Parallel workers. Default 2. |
93
+ | `--concurrency <n>` | Parallel workers. Default half the CPU cores, between 2 and 8. Lower it when one side is someone's live site. |
89
94
  | `--config <path>` | Config file. Default: `renderdiff.config.{ts,mts,js,mjs}` in cwd. |
90
95
  | `--selector <sel>` | Compare the one element this Playwright selector matches, on both sides, instead of the whole page. Applies to every route, over the config's per-route `selector`. |
91
96
  | `--left-selector <sel>`, `--right-selector <sel>` | That side's selector, over `--selector`. One without the other side (or `--selector`) is an error. |
97
+ | `--storage-state <file>` | Start both sides with this Playwright storage state (cookies, localStorage), e.g. signed in. Relative to cwd. Over the config's `storageState`. |
98
+ | `--left-storage-state <file>`, `--right-storage-state <file>` | That side's storage state, over `--storage-state`. One side alone is fine. |
99
+ | `--accept <reason>` | Record this run's differences as accepted, with a reason. See **Accepted differences and reruns**. |
100
+ | `--rerun-failed` | Rerun only what the last run in `--out` didn't find clean or accepted, keeping the rest. |
92
101
  | `--dry-run` | Print the URL pairs and exit. No browser. |
93
102
  | `--open` | Open the report when the run finishes. |
94
103
  | `--progress` | A line per shot as it finishes, then the worst scores. The default on a terminal; piped, renderdiff prints one compact summary at the end. |
95
104
  | `--no-progress` | The compact summary even on a terminal. |
96
105
  | `-h, --help` / `-v, --version` | |
97
106
 
98
- Routes are **literal paths** — there is no route discovery and globs are rejected. Each
107
+ Routes are **literal paths**: there is no route discovery and globs are rejected. Each
99
108
  route is loaded on both origins, unless it's a pair.
100
109
 
101
- Default widths are `1440, 1152, 896, 704, 375` — one inside each common responsive band
102
- rather than on a breakpoint boundary, so a layout switch never flaps between captures.
110
+ Default widths are `1440, 1152, 896, 704, 375`: each sits mid-way inside a common breakpoint
111
+ range rather than on a boundary, so a layout switch never flaps between runs.
103
112
 
104
- Exit codes: `0` all good · `1` a route errored or scored below `--threshold` ·
105
- `2` bad arguments, bad config, or an origin that isn't responding.
106
-
107
- ## Comparing different routes or components
113
+ | Exit | Meaning |
114
+ | --- | --- |
115
+ | `0` | No route errored and none scored below `--threshold`. Without one, a diff still exits 0. |
116
+ | `1` | A route scored below `--threshold`. Wins over `3` when both happen. |
117
+ | `2` | Bad arguments, bad config, or an origin that isn't responding. |
118
+ | `3` | A route errored, so it went unmeasured, or the run crashed (e.g. no Chromium). |
108
119
 
109
- When a migration moved a page, give the route as a pair: `left::right`.
120
+ ### Moved pages, components, interactions, sign-in
110
121
 
111
122
  ```
112
- renderdiff http://localhost:3000 https://example.com /pricing::/plans /about::/company/about
123
+ renderdiff http://localhost:3000 https://example.com /pricing::/plans
124
+ renderdiff http://localhost:3000 https://example.com /pricing --selector "[data-test=plans]"
113
125
  ```
114
126
 
115
- `/pricing::/plans` loads `/pricing` on the left and `/plans` on the right. The split is
116
- on the first `::` followed by `/`, so `/q?x=std::vector` and `/wiki/Talk:Foo` stay single
117
- routes, and `pricing::plans` (no slash) is a literal path; `--dry-run` shows how each
118
- route was read. Each side is normalized on its own, a pair with identical sides is the
119
- plain route, and an empty side (`/pricing::`) is an error.
120
-
121
- A pair is named `/pricing::/plans` in stdout, `results.json` (`url`, plus `left` and
122
- `right`) and the report; its images are `pricing~plans.{width}.*.png`. A mask whose
123
- `urlPath` is either side's path is painted on both sides. In a config file, a route can
124
- also be an object, `{ left: "/pricing", right: "/plans" }`, where `right` defaults to
125
- `left`.
126
-
127
- To compare one component instead of the whole page, give a selector:
127
+ A pair `left::right` loads `/pricing` on the left and `/plans` on the right. `--selector`
128
+ compares one element, which must match exactly once per side (`--left-selector` /
129
+ `--right-selector` when the markup differs). A state behind a click takes a config `before`
130
+ hook. For pages behind a login:
128
131
 
129
132
  ```
130
- renderdiff http://localhost:3000 https://example.com /pricing --selector "[data-test=plans]"
131
- renderdiff http://localhost:3000 https://example.com /pricing --left-selector .plans-v2 --right-selector .plans
133
+ npx playwright codegen --save-storage=auth.json https://example.com/login
134
+ renderdiff http://localhost:3000 https://example.com /account --storage-state auth.json
132
135
  ```
133
136
 
134
- Any Playwright selector works, `xpath=` and `>>` chains included, and it's never split on
135
- `::`. It must match **exactly one** element on each side; otherwise the route errors at once
136
- with the count (`selector ".card" matched 6 elements on left (/pricing), expected 1`). The
137
- shot, the score, the regions and the text diff are that element's, in its own pixels; a
138
- mask is painted where it overlaps the element.
139
-
140
- A CLI selector applies to every route and replaces the config's. In a config file a route
141
- object takes `selector: ".plans"` or `selector: { left: ".plans-v2", right: ".plans" }`;
142
- to compare several components of one page, list the route once per selector. Such a route
143
- is `/pricing [.plans]` in the summary and the report, keeps `/pricing @1440` on a
144
- `--progress` row with the selector on the line under it, has `selector` in `results.json`
145
- (`{ left, right }`, or `null` for a whole page), and its images are
146
- `pricing@_plans.{width}.*.png`.
147
-
148
- Things to know:
149
-
150
- - The whole page still loads and settles first, so a component costs about what its page does.
151
- - It's shot at scroll 0 with the rest of the page, so a fixed or sticky header over it is
152
- in the shot. Hide it with `hideSelectors`.
153
- - An element inside a shorter scroll container includes whatever paints below the container.
154
- - A transformed element is shot at its axis-aligned bounding box.
155
- - Pseudo-elements (`::before`) are rejected by Playwright; elements inside iframes can't be
156
- reached. Open shadow roots are pierced.
157
- - A selector that matches an element with no box (`display: none`) or one off the page is
158
- an error. The render gate stays page-level: a textless element on a rendered page is fine.
137
+ `auth.json` holds live session tokens: **never commit it**, and keep the out dir (signed-in
138
+ screenshots and text) out of git too. Details: [docs/scoping.md](https://github.com/tmashara/renderdiff/blob/main/docs/scoping.md).
159
139
 
160
140
  ## Config file
161
141
 
162
- Flags cover a one-off run. A `renderdiff.config.ts` (or `.mjs`) in the working directory
163
- covers a repo you diff repeatedly, and expresses the things flags can't: **masks** for
164
- content that legitimately drifts, extra selectors to hide, extra URLs to block.
142
+ Flags cover a one-off run. A `renderdiff.config.{ts,mts,js,mjs}` in the working directory
143
+ covers a repo you diff repeatedly, and expresses what flags can't: **masks** for content
144
+ that legitimately drifts, extra selectors to hide, extra URLs to block, `before` hooks.
165
145
 
166
146
  ```ts
167
147
  export default {
168
- left: "http://localhost:3000",
169
- right: "https://example.com",
170
- routes: [
171
- "/",
172
- "/about",
173
- "/pricing::/plans",
174
- { left: "/blog", right: "/posts" },
175
- { left: "/pricing", selector: "[data-test=plans]" },
176
- ],
148
+ left: "http://localhost:3000", // your change
149
+ right: "https://example.com", // the reference
150
+ routes: ["/", "/about", "/pricing::/plans", { left: "/pricing", selector: "[data-test=plans]" }],
177
151
  widths: [1440, 896, 375],
178
152
  threshold: 0.97,
179
-
180
153
  hideSelectors: [".site-cookie-banner"],
181
154
  blockUrlPatterns: ["**://*.my-tracker.example/**"],
182
-
183
- masks: [
184
- { selector: "[data-star-count]", reason: "live star count", urlPath: "/pricing" },
185
- ],
155
+ masks: [{ selector: "[data-star-count]", reason: "live star count", urlPath: "/pricing" }],
186
156
  };
187
157
  ```
188
158
 
189
- A mask needs a `reason` and exactly one of `selector` (a Playwright selector) or
190
- `locate: (page) => Locator`, the escape hatch for what a selector can't say. Without
191
- `urlPath` it applies to every route.
192
-
193
- See [`renderdiff.config.example.ts`](./renderdiff.config.example.ts) for every field, annotated.
159
+ **Precedence: CLI flag > config file > built-in default.** Every field is annotated in
160
+ [`renderdiff.config.example.ts`](./renderdiff.config.example.ts). `export default { … }`
161
+ needs no import; `import type { RenderdiffConfig } from "renderdiff"` is for editor types
162
+ only.
194
163
 
195
- **Precedence: CLI flag > config file > built-in default.**
164
+ ## Reading the output
196
165
 
197
- `export default { … }` needs no import. `import type { RenderdiffConfig } from "renderdiff"`
198
- with `satisfies RenderdiffConfig` is for editor types only; Node erases it, so the file loads
199
- without renderdiff installed in that project.
166
+ **Signs always point at your change:** `+` = only in or more in the left (your change);
167
+ `-` = only in or more in the right (the reference).
200
168
 
201
- ## How it reads
202
-
203
- The score is the share of pixels that match, per route per width. Green above 99.5%,
204
- amber to 97%, red below. It is a triage signal, not a verdict — open the report and
205
- look at the diff column.
206
-
207
- Under each score, `--progress` output and the report card say where to look:
169
+ | Token | Meaning |
170
+ | --- | --- |
171
+ | `97.41%` | Share of measured pixels that match. A range (`99.12%–99.87%`) when one row covers several widths. A triage signal, not a verdict. |
172
+ | `@1440,1152` | The widths a row covers. They agree on height and width delta, bands, aligned score and text; score and region count are ranges. |
173
+ | `h-24` / `w+8` | Height / width delta, left − right. The smaller image is padded with magenta, and the padding counts against the score on purpose. |
174
+ | `band -24@y196` | 24 rows only the reference has, at y=196 (`+`: only your change has them). `@y~196`: the row's widths put it at different y; the first width's is shown. |
175
+ | `aligned 100.00%` | The score with the bands cut out. 100% means the band is the whole difference. |
176
+ | `unaligned` | Heights differ but too few rows match to find bands. Use the height delta and regions. |
177
+ | `3 reg` | Regions: connected patches of differing pixels. A missing link barely moves the score, but it is still a region. |
178
+ | `unstable 1,204px` | Pixels that kept changing between shots of the same load (a ticker, a carousel). Excluded from the score, so the route is never clean. |
179
+ | `text -"Pricing"` | A line of visible text only the reference has (`+`: only your change). Up to three, then `(+N more)`. |
180
+ | `dom sel prop a→b` | Element on both sides whose computed `prop` differs, left value first. |
181
+ | `dom +sel` / `-sel` | Element only on the left / only on the right. |
182
+ | `dom sel x+8` | It moved without its own style changing; the neighbour or parent listed after it is the cause. |
183
+ | `dom sel` | On both sides and unchanged in style or position, so its own pixels changed (an image or canvas). |
184
+ | `crops …` | Up to 3 PNGs per comparison, bands first, relative to `results.json`. Left is on top or on the left, and the change is boxed in red. |
185
+ | `ERROR …` | Unmeasured, which is neither a pass nor a fail. Exits 3. |
186
+ | `accepted:` / `stale accepted:` | Shots whose every difference was accepted, and accepted entries the run no longer saw. See below. |
187
+ | `rerun:` | With `--rerun-failed`: how many shots were rerun and how many reused. |
188
+
189
+ Read text first, then `dom`, then the crops for anything `dom` doesn't explain. Full
190
+ detail, including the `--progress` and report forms: [docs/output.md](https://github.com/tmashara/renderdiff/blob/main/docs/output.md).
191
+
192
+ ## Accepted differences and reruns
193
+
194
+ A migration starts with differences that already exist between the untouched local build
195
+ and production. Record them once, then change code and re-check only what's red:
208
196
 
209
197
  ```
210
- [3/8] 97.14% /pricing @1440 (height -24px)
211
- band -24px at y=196 (right)
212
- aligned 100.00%
213
- text: 0 added, 1 removed
198
+ renderdiff http://localhost:3000 https://example.com / /about --accept "differs before the migration"
199
+ # …change code…
200
+ renderdiff http://localhost:3000 https://example.com / /about --rerun-failed
214
201
  ```
215
202
 
216
- - **Regions** are connected patches of differing pixels, on a 16px grid, largest first. On
217
- a long page a missing link barely moves the score; it is still a region. Only the area
218
- both sides cover is searched: padding is reported by the height delta and the bands. With
219
- bands, regions come from the realigned rows, so content a band only pushed down isn't
220
- one (above, the band is the whole difference, so there are none). The
221
- card lists the top five with their pixel counts; up to three crops cover the bands first,
222
- then the largest regions.
223
- - **Bands** are rows only one side has, found by aligning the two images row by row. `-24px
224
- at y=196 (right)` means the right side has 24 rows there that the left lacks — signed like
225
- the height delta. When rows of identical content repeat, the band's `y` can slide along
226
- them, so treat it as approximate. `aligned` is the score over the rows the alignment pairs,
227
- i.e. what the page would score if the bands were cut out; it only shows when there are
228
- bands. Pages of different heights that share too few identical rows say `rows too
229
- different to align`. A horizontal shift is never a band. Alignment works on exact row
230
- matches, so it gives up when the edits total more than 2,000 rows (a single inserted or
231
- removed block aligns at any height), or when sub-threshold shade noise touches every row.
232
- - **Text** is a line diff of each side's `document.body.innerText` (the element's, with a
233
- selector). Left is the changed version: *added* lines are on the left only, *removed* lines on the right only. The card
234
- lists the first 20. Masked elements are left out. Unmasked text that changes per load
235
- (a ticker, a clock) shows up here whenever the two sides differed at capture time, even
236
- when its pixels were painted out as unstable.
237
-
238
- **Magenta means padding.** When one side is taller or wider, the shorter image is padded
239
- with magenta, and that padding counts against the score on purpose: a page 400px shorter
240
- than the one it's compared against has not reached parity. The `height ±Npx` badge on the
241
- row names the cause.
242
-
243
- **Errors are not failures.** A route that threw produced no measurement, so it is
244
- neither a pass nor a fail. It's listed separately in the report — and it still exits 1,
245
- because an unmeasured route is not a verified one.
203
+ `--accept "<reason>"` writes every difference the run saw to `renderdiff.accepted.json` (next
204
+ to the config, or in cwd); commit it. Later, a shot whose every difference matches an entry is
205
+ listed on an `accepted:` line and counts neither as differing nor toward `--threshold`;
206
+ anything new on the same page still shows. `--rerun-failed` reruns only what the last run in
207
+ `--out` didn't find clean or accepted and merges it back; it exits 2 if the run differs from
208
+ the previous one. Matching rules, what can still hide, and stale entries:
209
+ [docs/accepted.md](https://github.com/tmashara/renderdiff/blob/main/docs/accepted.md).
246
210
 
247
211
  ## Output
248
212
 
249
213
  Everything lands in the out dir (`./renderdiff-out` by default):
250
214
 
251
- - `report.html` — the scored report.
252
- - `{slug}.{width}.{left,right,diff}.png` — full-page shots and the diff. `/` slugs to
215
+ - `report.html`: the scored report.
216
+ - `{slug}.{width}.{left,right,diff}.png`: full-page shots and the diff. `/` slugs to
253
217
  `index`, `/blog/foo` to `blog-foo`, other characters to `_`, a pair `/a::/b` to `a~b`; a
254
218
  clash gets a `-2` suffix.
255
- - `{slug}.{width}.c{n}.png` — up to three crops per comparison: bands (rows only one side
256
- has), tallest first, then the largest regions. Each shows left and right at native scale,
257
- padded by 48px and at least 320px wide, with a 1px red box around the change; on the side
258
- that lacks a band's rows, a 2px red seam is inserted where they would be, hiding nothing.
259
- Left is always on top (`stacked`, for wide changes) or on the left (`side`, for tall
260
- ones). A crop stays within 1,568px on its long edge and 1,568 image tokens
261
- (⌈w/28⌉·⌈h/28⌉) and is never scaled below 0.75×; a change too big for that keeps its top
262
- rows (or left columns) and is marked `clipped`. Height padding is never cropped.
263
- - `results.json` — everything above as data, written on every run, errors included, one
219
+ - `{slug}.{width}.c{n}.png`: up to three crops per comparison, left on top or on the left,
220
+ change boxed in red.
221
+ - `results.json`: everything above as data, written on every run, errors included, one
264
222
  result per line, scores rounded down to four decimals (so a shot that differs never
265
- reads as `1`). Its types (`Results` and friends) are exported
266
- from the package.
223
+ reads as `1`). Its types (`Results` and friends) are exported from the package.
267
224
 
268
225
  ```jsonc
269
226
  {
270
227
  "meta": {
271
- "tool": "renderdiff", "version": "0.1.0", "generatedAt": "…",
228
+ "tool": "renderdiff", "version": "0.3.0", "generatedAt": "…",
272
229
  "left": "http://localhost:3000", "right": "https://example.com",
273
230
  "widths": [1440, 375], "threshold": null,
274
231
  "clock": "…", // the instant both sides' clocks start at
275
232
  "blockUrlPatterns": ["…"], "hideSelectors": ["…"],
276
- "masks": [{ "urlPath": "/pricing", "reason": "live star count" }] // no urlPath: every route
233
+ "masks": [{ "urlPath": "/pricing", "reason": "live star count" }], // no urlPath: every route
234
+ "browser": { "name": "chromium", "version": "…" }, // "chrome" when it fell back
235
+ "storageState": { "left": "auth.json", "right": null } // file names only
277
236
  },
278
237
  "results": [
279
238
  {
280
239
  "url": "/about", "left": "/about", "right": "/about", // a pair: "/a::/b", "/a", "/b"
240
+ "selector": null, "hook": false, // hook: a before hook ran
281
241
  "width": 1440, "status": "compared",
282
242
  "score": 0.9714, // share of measured pixels that match
283
243
  "alignedScore": 1, // score over the aligned rows; null unless there are bands
284
244
  "diffPixels": 33120, "totalPixels": 1158000, "unstablePixels": 0,
285
245
  "heightDelta": -24, "widthDelta": 0, // left − right
286
246
  "regions": [], // left-page px; with bands, from the realigned rows
287
- "bands": [{ "side": "right", "y": 196, "h": 24 }], // null: too different to align
288
- "text": { "added": [], "removed": ["Pricing"] },
247
+ "bands": [{ "side": "right", "y": 196, "h": 24, // null: too different to align
248
+ "elements": [{ // regions have them too; absent: not looked at
249
+ "selector": "nav > a[href=\"/pricing\"]", "side": "right", // or "left", "both"
250
+ "left": null, "right": { "x": 256, "y": 192, "w": 928, "h": 24 },
251
+ "changed": {}, // "margin-left": ["24px", "16px"], left first; "text" too
252
+ "text": "Pricing" // "rightSelector" when the right one differs
253
+ }]
254
+ }],
255
+ "text": { "added": [], "removed": ["Pricing"] }, // added: left only; removed: right only
289
256
  "images": { "left": "about.1440.left.png", "right": "about.1440.right.png", "diff": "about.1440.diff.png" },
290
257
  "crops": [{
291
258
  "path": "about.1440.c1.png", "kind": "band", // or "region"
@@ -299,108 +266,32 @@ Everything lands in the out dir (`./renderdiff-out` by default):
299
266
  "summary": {
300
267
  "compared": 1, "errors": 1, "mean": 0.9714,
301
268
  "worst": { "url": "/about", "width": 1440, "score": 0.9714 },
302
- "belowThreshold": 0
269
+ "belowThreshold": 0, // accepted shots aside
270
+ "accepted": 0, // shots whose every difference was accepted
271
+ "stale": [] // accepted entries the run measured but didn't see
303
272
  }
304
273
  }
305
274
  ```
306
275
 
307
276
  Paths are relative to the out dir. Results are in route-then-width order.
308
277
 
309
- ## What it does to make two loads comparable
310
-
311
- - **Refuses to start** unless both origins answer a HEAD request. Two error pages diffed
312
- against each other score ~100%, which is the likeliest way a tool like this lies to you.
313
- - **Refuses a shot** it can't trust: an HTTP status of 400 or above on either side is an
314
- error naming the status. A route where *both* sides look blank — under 200 characters of
315
- text and no visible images, SVG, canvas, video, iframes or form controls — errors instead
316
- of quietly scoring as a match. If only one side is blank, the diff shows it.
317
- - **Blocks analytics, consent and A/B tooling** at the network layer on both sides —
318
- GTM, GA, Segment, Intercom, Hotjar, LinkedIn, Meta, OneTrust, CookieLaw, VWO. Blocked
319
- rather than hidden, because A/B tools rewrite arbitrary DOM and CSS can't pre-empt an
320
- experiment you haven't seen. Blocking must stay symmetric: doing it on one side only
321
- is itself a diff.
322
- - **Pins what varies between loads**, identically on both sides: the clock starts at one
323
- instant per run (floored to the minute) on every load and then runs, `Math.random` is
324
- seeded, timezone (UTC), locale (en-US), colour scheme
325
- (light) and `prefers-reduced-motion` are fixed, and videos are paused at frame 0. A clock
326
- that shows seconds can still tick between the two sides and may need a mask.
327
- - **Kills animation, transitions and caret blink**, and hides common consent banners and
328
- dev toolbars.
329
- - **Settles the page** — makes native lazy images eager, scrolls every viewport-height
330
- slice a frame at a time so script lazy loaders fire, waits for every image to decode,
331
- `document.fonts.ready` and up to 3s of network idle, then re-injects the stylesheet to
332
- catch late-injected chrome.
333
- - **Shoots until stable** — repeats the screenshot until two in a row match (at most 3; 2
334
- for tiled pages). Pixels still changing between the last two shots of the same load are
335
- noise from that origin: they're painted magenta on both sides, reported per comparison
336
- as `unstable`, and excluded from the measurement — the score is over the remaining
337
- pixels. A route more than 25% unstable errors as `page did not settle`.
338
- - **Isolates state** — one browser context per origin, so cookies and localStorage never
339
- cross. Both sides of a route load at the same time.
340
- - **Retries once** per shot; a live site can time out on a cold edge.
341
- - **Stitches tall pages.** Chromium silently truncates a single screenshot past 16,384px,
342
- so taller pages are captured as tiles through CDP and stitched back together. Masks are
343
- painted onto the final image from each element's bounding box, so they apply the same
344
- way on both paths. Past 40,000px it errors — that's a broken layout, not a page.
345
-
346
- Every run's report ends with a "What was neutralized" section listing the actual block
347
- patterns, mask reasons and pinned settings in effect, so the report can't drift from the
348
- config.
349
-
350
- ## Development
351
-
352
- ```bash
353
- npm install # builds via the prepare script
354
- npm run build # tsc → dist/
355
- npm run typecheck # src + tests, no emit
356
- npm install -g . # install this working copy globally
357
- ```
278
+ ## Why a clean result is trustworthy
358
279
 
359
- Source is TypeScript in `src/`, compiled to ESM in `dist/`. Import specifiers keep their
360
- `.ts` extension in source (`rewriteRelativeImportExtensions` rewrites them on emit), so
361
- `node src/cli.ts` also runs directly.
280
+ - **Refuses to start** unless both origins answer: two error pages would score ~100%.
281
+ - **Refuses a shot** with an HTTP status ≥ 400, or where both sides look blank.
282
+ - **Blocks analytics, consent and A/B tooling** at the network layer, on both sides.
283
+ - **Pins what varies between loads**: clock, `Math.random`, timezone, locale, colour scheme, reduced motion, video frames.
284
+ - **Kills animation, transitions and caret blink**, and hides common consent banners and dev toolbars.
285
+ - **Settles the page**: lazy images, fonts and network idle before the shot.
286
+ - **Shoots until stable**, and excludes pixels still moving as `unstable`.
287
+ - **Isolates state** per origin, **retries once** per shot, and **stitches tall pages** past 16,384px.
362
288
 
363
- ## Tests
289
+ Each run's report ends with a "What was neutralized" section listing what was in effect.
290
+ Full detail: [docs/how-it-works.md](https://github.com/tmashara/renderdiff/blob/main/docs/how-it-works.md).
364
291
 
365
- ```bash
366
- npm test # unit tier — no browser, ~4s
367
- npm run test:visual # drives Chromium over the fixture corpus, ~40s
368
- npm run test:all
369
- ```
292
+ ## Contributing
370
293
 
371
- Built on `node:test`, no test dependencies. Running them needs a Chromium for the visual tier:
372
- `npx playwright install chromium`.
373
-
374
- The suite is black-box: it spawns `node src/cli.ts` and reads results back from stdout,
375
- `report.html`, `results.json`, and the written PNGs. Nothing under `src/` is imported at
376
- runtime (only the `results.json` types), so what is asserted is what a user gets. The output
377
- formats it depends on are parsed in exactly one place, `tests/helpers/parse.ts`.
378
-
379
- **`tests/unit/`** — argument parsing, exit codes, route normalization, and config discovery,
380
- validation and precedence. All of it through `--dry-run`, which exits before the browser.
381
-
382
- **`tests/visual/`** — real captures against `tests/fixtures/`, a two-origin site served on
383
- ephemeral ports. Three groups:
384
-
385
- - *Equivalence* — six pairs of pages that are visually identical but built with different CSS:
386
- flex against grid, floats against flex, CSS tables against grid, transform-centring against
387
- `place-items`, inline-block against flex. These assert a **zero-pixel** diff, not a
388
- threshold. They are the corpus's central claim: the score tracks pixels, not markup, so a
389
- refactor that changed nothing visible reads as nothing changed.
390
- - *Controls* — an 8px shift, an extra section, one recoloured box. These must score low.
391
- Without them a tool that returned 100% unconditionally would pass the suite.
392
- - *Guard rails* — one per defence above: the blank-page refusal (and a short sign-in page
393
- that must not be refused), the 18,000px tiled capture with masks in both tiles, animation
394
- stilling, the shared clock start and seeded randomness, the unstable-pixel check, lazy-image
395
- settling (native and script-driven, 4,800px below the fold), consent hiding, masks, and
396
- network blocking. Masks
397
- and blocking each run twice, with and without their config, so the mechanism is shown doing
398
- something rather than a page merely happening to match.
399
-
400
- Fixture authoring rules — integer-pixel geometry above all — are in
401
- [`tests/fixtures/pages/README.md`](./tests/fixtures/pages/README.md), and
402
- `tests/unit/fixtures.test.ts` enforces what can be enforced. Read them before adding a pair:
403
- a fractional box is the difference between a clean comparison and sub-pixel noise.
294
+ Development and tests: [CONTRIBUTING.md](https://github.com/tmashara/renderdiff/blob/main/CONTRIBUTING.md).
404
295
 
405
296
  ## License
406
297
 
@@ -0,0 +1,42 @@
1
+ import type { RouteResult } from "./report.ts";
2
+ export declare const ACCEPTED_FILE = "renderdiff.accepted.json";
3
+ /**
4
+ * One band or region, without its `y`: a change above it moves it down the page without
5
+ * changing it. `hash` and `elements` tell a new change in the same spot from the accepted one.
6
+ */
7
+ export interface Fingerprint {
8
+ kind: "band" | "region";
9
+ side?: "left" | "right";
10
+ /** A region's box; a band's rows are in its hash. */
11
+ x?: number;
12
+ w?: number;
13
+ h: number;
14
+ /** The `dom` of each element under it, sorted; absent when they weren't looked at. */
15
+ elements?: string[];
16
+ /** Both sides' pixels under it, 4 bits a channel. */
17
+ hash?: string;
18
+ }
19
+ /** A difference recorded as already there or intended, so later runs report it as accepted. */
20
+ export interface Accepted {
21
+ /** The route as the summary names it: `/a`, `/a::/b`, `/a [.sel]`. */
22
+ route: string;
23
+ width: number;
24
+ reason: string;
25
+ fingerprint: Fingerprint;
26
+ }
27
+ export declare function readAccepted(file: string): Accepted[];
28
+ /**
29
+ * Mark each change an accepted entry matches, one entry per change, and a result `accepted`
30
+ * when every change matched and it is `acceptable`. Returns the changes of acceptable results
31
+ * nothing matched, and the entries for a measured route and width that matched nothing.
32
+ */
33
+ export declare function applyAccepted(results: RouteResult[], list: Accepted[]): {
34
+ unmatched: Omit<Accepted, "reason">[];
35
+ stale: Accepted[];
36
+ };
37
+ /**
38
+ * Drop `stale` and add `added` to the file as it is now, so a concurrent `--accept` isn't lost,
39
+ * sorted by route then width, through a rename so a crash never leaves it half written.
40
+ * Returns false, writing nothing, when there is nothing to change.
41
+ */
42
+ export declare function updateAccepted(file: string, stale: Accepted[], added: Accepted[]): boolean;