@graphty/visual-review 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Adam Powers
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,7 +1,551 @@
1
1
  # @graphty/visual-review
2
2
 
3
- This version is a placeholder. It exists so that later versions can be published from GitHub
4
- Actions through npm trusted publishing, which requires the package to exist first.
3
+ Visual regression review for Storybook, hosted by nobody. GitHub Actions screenshots every story
4
+ on every pull request and compares each screenshot with an approved baseline PNG kept in your
5
+ repository (in Git LFS). You open a review page on your own machine, see every difference side by
6
+ side, and accept, reject or exclude each one. Accepting commits the new baselines to the pull
7
+ request's branch. A required check, the "Visual gate", keeps a pull request from merging while it
8
+ holds a difference nobody accepted.
5
9
 
6
- The first real release, and its documentation, will be published from
7
- https://github.com/graphty-org/graphty-monorepo (the `visual-review/` package).
10
+ There is no service, account or per-snapshot bill: the captures live in GitHub Actions artifacts
11
+ for 30 days, the baselines live in git, and the review page reads both through the `gh` CLI with
12
+ your login.
13
+
14
+ - [How it works](#how-it-works)
15
+ - [Requirements](#requirements)
16
+ - [Install and set up](#install-and-set-up)
17
+ - [Configuration](#configuration)
18
+ - [The GitHub Actions workflows](#the-github-actions-workflows)
19
+ - [Your first review: seeding baselines](#your-first-review-seeding-baselines)
20
+ - [Opening the review page](#opening-the-review-page), [the screens](#the-screens), [keys](#keys),
21
+ [decisions](#what-each-decision-does), [Finish](#finish)
22
+ - [Seeding one story at a time](#seeding-one-story-at-a-time)
23
+ - [Iterating on a story before a pull request exists](#iterating-on-a-story-before-a-pull-request-exists)
24
+ - [Story parameters](#story-parameters)
25
+ - [What the gate does and does not guarantee](#what-the-gate-does-and-does-not-guarantee)
26
+ - [Troubleshooting](#troubleshooting)
27
+
28
+ ## How it works
29
+
30
+ 1. **Capture (CI).** On every pull request and every push to your default branch, a workflow job
31
+ per Storybook builds it, opens every story in Chromium (1200 x 900 at device scale factor 2,
32
+ a fixed clock, software WebGL, WebGPU removed), screenshots it, and compares the screenshot
33
+ with its baseline. Anything that differs is captured a second time, so a real change, an
34
+ unstable story and a one-off flake are told apart. The results (`results.json` and the PNGs
35
+ worth looking at) are uploaded as an artifact.
36
+ 2. **Review (your machine).** `visual-review serve` lists your open pull requests, downloads their
37
+ captures, and serves a page with a grid of every change and a side-by-side, flash, highlight
38
+ and spotlight view of each. You accept, reject (with a reason) or exclude each item.
39
+ 3. **Finish.** One button applies your decisions: the accepted PNGs and a review record are
40
+ committed and pushed to the pull request's branch, and the rejects are posted as one comment.
41
+ CI captures again, and the accepted items now read `unchanged`.
42
+ 4. **Gate (CI).** The "Visual gate" job fails while a pull request holds a difference nobody
43
+ accepted, or a baseline file changed without a review record naming it.
44
+
45
+ A story with no baseline yet does not block anything until a pull request changes it, so you can
46
+ seed baselines a few stories at a time.
47
+
48
+ ## Requirements
49
+
50
+ - **A git repository on GitHub, with GitHub Actions.** The review page talks to GitHub through
51
+ the [`gh` CLI](https://cli.github.com), logged in (`gh auth login`) as someone who can push to
52
+ the repository's branches.
53
+ - **Node.js 20 or newer**, locally and in CI.
54
+ - **Storybook 7 or newer**, built as a static site (`storybook build`), which writes the
55
+ `index.json` the capture reads.
56
+ - **Playwright**, a peer dependency: install it next to this package. `visual-review
57
+ install-browser` installs the Chromium that version of Playwright drives.
58
+ - **git-lfs**, on every machine that accepts baselines or pushes a branch holding them. GitHub's
59
+ runners have it. Install it with your package manager (`brew install git-lfs`,
60
+ `sudo apt-get install git-lfs`), or without root put the `git-lfs` binary from
61
+ https://github.com/git-lfs/git-lfs/releases on your `PATH`; then run `git lfs install` once,
62
+ and `git lfs pull` in every checkout that already existed.
63
+ - **jq** on the runners, which GitHub's Ubuntu runners have.
64
+
65
+ ## Install and set up
66
+
67
+ ```bash
68
+ npm install --save-dev @graphty/visual-review playwright
69
+ npx visual-review init
70
+ ```
71
+
72
+ (`pnpm add -D` and `yarn add -D` work the same way; `init` notices the lockfile and writes
73
+ workflows for that package manager.)
74
+
75
+ `init` writes, at the root of your repository, and never overwrites a file you already have:
76
+
77
+ | File | What it is |
78
+ | ------------------------------------- | ---------------------------------------------------------------------------------------- |
79
+ | `visual-review.config.json` | Your Storybooks and your repository's settings ([Configuration](#configuration)) |
80
+ | `.gitattributes` | `visual-baselines/**/*.png filter=lfs diff=lfs merge=lfs -text`: baselines go to Git LFS |
81
+ | `.gitignore` | `/.visual-review/`, where the review page downloads captures and keeps its state |
82
+ | `.github/workflows/visual-review.yml` | Captures every pull request and push, and gates pull requests |
83
+ | `.github/workflows/visual-seed.yml` | Captures an older commit on demand, to seed baselines from |
84
+
85
+ Then:
86
+
87
+ 1. Edit `visual-review.config.json`: one entry under `projects` per Storybook, with the directory
88
+ its build writes and the command that builds it.
89
+ 2. Commit everything and open a pull request. Its "Visual review" run captures every story; with
90
+ no baselines yet, nothing blocks.
91
+ 3. Make **Visual gate** a required status check (Settings, then Branches or Rulesets).
92
+ 4. Merge, then seed your first baselines ([below](#your-first-review-seeding-baselines)).
93
+
94
+ `visual-review init --force` rewrites the two workflows when they still start with the line
95
+ `init` writes ("Generated by visual-review init"), to pick up a newer template after an upgrade.
96
+ It never replaces the config or a workflow you wrote yourself.
97
+
98
+ Every command has `--help`; `visual-review --help` lists them.
99
+
100
+ ## Configuration
101
+
102
+ `visual-review.config.json` sits at the root of the repository. Only `projects` is required.
103
+
104
+ ```json
105
+ {
106
+ "defaultBranch": "main",
107
+ "workflow": "visual-review.yml",
108
+ "baselines": "visual-baselines",
109
+ "workDir": ".visual-review",
110
+ "commitPrefix": "test",
111
+ "issueLabels": ["bug"],
112
+ "projects": {
113
+ "web": {
114
+ "storybook": "packages/web/storybook-static",
115
+ "build": "npm run build-storybook --workspace packages/web",
116
+ "workers": 4
117
+ },
118
+ "charts": {
119
+ "storybook": "packages/charts/storybook-static",
120
+ "build": "npm run build-storybook --workspace packages/charts",
121
+ "seedFromDefaultBranch": false,
122
+ "waitFor": { "selector": "my-chart", "method": "whenRendered", "failOnConsole": "render timeout" }
123
+ }
124
+ }
125
+ }
126
+ ```
127
+
128
+ | Key | Default | Meaning |
129
+ | --------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------- |
130
+ | `defaultBranch` | `main` | The branch baselines are seeded from and pull requests merge into |
131
+ | `workflow` | `visual-review.yml` | The workflow file whose runs hold the captures; the review page looks runs up by it |
132
+ | `baselines` | `visual-baselines` | Where baselines live: `<baselines>/<project>/<story id>[.<mode>].png`, and review records in `<baselines>/reviews/` |
133
+ | `workDir` | `.visual-review` | Where `serve` downloads captures and keeps its decisions and session token; keep it out of git |
134
+ | `commitPrefix` | `test` | The conventional-commit type and scope of the commits Finish makes, e.g. `test(ui)` |
135
+ | `issueLabels` | `["bug"]` | Labels of the issue Finish opens for rejects on the default branch; each must exist |
136
+ | `projects` | (required) | One entry per Storybook; the id names its baselines directory, CI job and artifact |
137
+
138
+ Per project:
139
+
140
+ | Key | Default | Meaning |
141
+ | ----------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
142
+ | `storybook` | (required) | The built Storybook's directory, relative to the repository root |
143
+ | `build` | none | The shell command CI runs to build it (from the repository root) |
144
+ | `workers` | `4` | How many browsers capture in parallel |
145
+ | `seedFromDefaultBranch` | `true` | `false`: the project's first baselines are accepted on a pull request, not seeded from the default branch |
146
+ | `waitFor` | none | After a story renders, call `method()` on every element matching `selector` and wait for the promise it returns, for a component that keeps drawing after Storybook says it is done. A console line containing `failOnConsole` fails the story |
147
+
148
+ Project ids are letters, digits, `.`, `_` and `-`. The pull request gate reads the config as it is
149
+ on the base branch, so a pull request cannot move `baselines` out from under it.
150
+
151
+ ## The GitHub Actions workflows
152
+
153
+ **`visual-review.yml`** runs on every pull request and every push to the default branch:
154
+
155
+ - **Plan** reads the projects from the config.
156
+ - **visual (&lt;project&gt;)**, one job per project: fetches that project's baselines from Git LFS
157
+ (cached), installs your dependencies, installs Chromium, runs the project's `build`, downloads
158
+ the default branch's newest capture as a reference (on pull requests), captures, and uploads
159
+ the artifact `visual-<project>-<attempt>` (kept 30 days). It never fails because of a
160
+ difference; `continue-on-error` keeps even a crash of the tool from failing the run.
161
+ - **Visual gate** (pull requests only) downloads every capture of the run and runs
162
+ `visual-review gate` at the version `init` pinned, with `npx`, so a pull request's own
163
+ dependencies cannot change it. Make it a required check.
164
+
165
+ The review page finds captures by the workflow's file name (the config's `workflow`), the jobs by
166
+ their names, `visual (<project>)`, and the artifacts by `visual-<project>-<attempt>`. If you would
167
+ rather capture inside an existing CI workflow (to reuse a Storybook your build job already made),
168
+ copy the `visual` job and the gate's steps into it, keep those names, and set `workflow` to that
169
+ file.
170
+
171
+ **`visual-seed.yml`** is started by hand to capture an older commit with the default branch's
172
+ tool: `gh workflow run visual-seed.yml --ref main -f ref=<sha>`. See
173
+ [Seeding](#seeding-one-story-at-a-time).
174
+
175
+ Both need nothing but the default `GITHUB_TOKEN`: the capture job reads Actions artifacts
176
+ (`actions: read`); nothing in CI writes to the repository.
177
+
178
+ ## Your first review: seeding baselines
179
+
180
+ A project has no baselines until you accept some, and until it has one the gate ignores it. After
181
+ the setup pull request merges, the default branch's push runs the capture:
182
+
183
+ 1. Find that run's id: `gh run list --workflow visual-review.yml --branch main --limit 1`.
184
+ 2. Start the page with `--master-run <run id>` ([Opening the review page](#opening-the-review-page))
185
+ and open "master (seed)" (the page calls the default branch's target "master", whatever its
186
+ name). Every story is `new` there.
187
+ 3. Accept what looks right, reject what does not (with a reason), leave the rest, and press
188
+ Finish. You get a pull request `visual/seed-<date>` holding the accepted baselines, and one
189
+ issue listing the rejects.
190
+ 4. Merge the seed pull request once its own run shows its accepted items `unchanged`. From then
191
+ on, every pull request that changes how a seeded story looks is blocked until you accept it.
192
+
193
+ ## Opening the review page
194
+
195
+ ```bash
196
+ PORT=4800 npx visual-review serve
197
+ ```
198
+
199
+ It prints the address to open, with a session token after `#token=`; open that exact URL. The
200
+ token is kept in the work directory, so the URL stays valid across restarts; delete
201
+ `<workDir>/state/token` to issue a new one. Without the token the page shows "No session token".
202
+
203
+ - Without a certificate it serves plain HTTP, and only on `localhost` (`HOST` defaults to it).
204
+ To open the page from another device (an iPad, say), serve HTTPS: set `HOST` to the machine's
205
+ name and `HTTPS_CERT_PATH` and `HTTPS_KEY_PATH` to a certificate and key for it.
206
+ - `serve` refuses to start without git-lfs: an accept would commit raw PNGs.
207
+ - `--master-run <run id>` also lists the default branch at that run, for seeding.
208
+ - `--results <dir>` serves local captures offline (a directory of `<project>/results.json`), for
209
+ looking at a story before a pull request exists. It is listed as "Local preview" and is look
210
+ only: no Accept, Reject or Exclude, and no Finish. Only CI captures of a pushed commit are
211
+ decided.
212
+
213
+ ### Links to a screen
214
+
215
+ The address always names the screen you are on, after the token: the targets list; a pull
216
+ request (or master) and project with the grid's filter and text; or one story with its view and
217
+ zoom, for example
218
+ `#token=...&target=123&project=web&filter=undecided&item=button--primary.dark.png&view=flash&zoom=2`.
219
+ Opening that address, in another tab or on another device, opens the same screen. **Copy link**
220
+ at the top right copies it. The link carries your session token, so it works on your iPad the way
221
+ the printed URL does; keep it to yourself as you would that URL. All of it sits after `#`, which a
222
+ browser never sends to any server or in a Referer, so the page never hands the token to another
223
+ site. Back and Forward move between the targets list, a grid and a story; moving between stories
224
+ or views of one grid updates the address in place.
225
+
226
+ A link to something that is gone opens the nearest screen that still exists, and the status line
227
+ says why: a story not in the newest CI run opens its grid, and a pull request no longer listed
228
+ (closed, or no CI run) opens the targets list.
229
+
230
+ Finish's commit is signed by the git configuration of the process that runs the server. If
231
+ someone else started it for you (an agent, a service manager), the commit carries their
232
+ identity: the page names the key that will sign before every Finish and prints the command that
233
+ starts the same server from your own shell.
234
+
235
+ ## The screens
236
+
237
+ 1. **Targets.** Each open pull request with a run of the capturing workflow, and the default
238
+ branch (shown as "master (seed)") when started with `--master-run`. Per project: how many items need a decision, how many you decided, and badges:
239
+ - **merge master first**: the default branch has newer baselines for this project than the
240
+ pull request. Merge the default branch into the pull request's branch (by merge, never
241
+ rebase) and wait for CI.
242
+ - **capture failed**: the `visual` job produced no results. Re-run that job in GitHub Actions.
243
+ - **incomplete: N of M stories**: the capture stopped part way. Re-run the job.
244
+ - **not seeded from master**: this project is not reviewed on the default branch
245
+ (`"seedFromDefaultBranch": false` in the config); its first baselines are accepted on a pull
246
+ request.
247
+ 2. **Grid.** It opens on **Needs a decision** (the undecided items, counted on the button); **All**
248
+ and one button per status show the rest, and **Accepted**, **Rejected** and **Excluded** show
249
+ what you decided, each counted, as Chromatic's review does. A line above the grid splits what is shown into
250
+ errors and images to compare, so the counts always add up. At the top, **Errors** lists every failed capture with its reason and,
251
+ under "console and stack", the story's console output and the thrown error's stack (a play
252
+ function's failed `expect` included). An error is never accepted: fix the story, re-run the
253
+ `visual` job for a one-off timeout, or exclude it with a reason. Below it the items are grouped
254
+ by component (the story id before `--`), components with a changed item first, then new,
255
+ unstable and removed ones; each story's modes (light, dark) sit side by side under its name.
256
+ Every tile is numbered, and the number is the story screen's "N of M". A component's
257
+ **Accept N undecided** accepts that component's undecided items without opening them, after
258
+ asking. Under every decided tile (and every decided error) its decision is spelled out:
259
+ "Accepted", "Accepted (not opened)" for one Accept all took, or "Rejected" or "Excluded" with
260
+ the reason. Its **Undo** clears it without opening the story. A component's **Undo N
261
+ decisions**, and **Undo all decisions** beside Accept all for the whole project, clear many at
262
+ once: the first press turns the button into "Confirm: undo N decisions", a second press undoes,
263
+ and Escape or any other change to the grid cancels. Every Undo here is the same request as the
264
+ story screen's U. A reject an earlier Finish already posted says "Posted by Finish: stays" and
265
+ has no Undo on the grid; the bulk Undo buttons leave it too. **Filter by story id** narrows the grid; **Go to** opens item N, or the first item
266
+ whose id contains the text. Coming back from a story, its tile is outlined and scrolled into
267
+ view.
268
+ 3. **Story.** One item, on one screen: the controls on top, then two panes of the same size side
269
+ by side, the baseline on the left and the new capture on the right, filling the rest of the
270
+ window. Images open at **Fit to screen**: both whole images fit their panes, across and down,
271
+ at one scale (never above real size), so two captures of the same size line up pixel for pixel
272
+ and nothing scrolls. With no baseline (a new story, or "no baseline yet") the left pane stays
273
+ as an empty frame labelled "No baseline", so the new image sits exactly where it would beside
274
+ one; a removed or failed story leaves the right pane empty the same way. **Real size (1x)** is
275
+ one CSS pixel of the page for each CSS pixel the story was drawn at (a capture holds two image
276
+ pixels per CSS pixel). **2x**, **4x** and **8x** enlarge it; from 4x pixels are drawn as hard
277
+ squares. Zoomed, the images grow past their panes, which scroll: scrolling one scrolls the
278
+ other to the same place, and **Fit to screen** returns to the whole image. (On an iPad,
279
+ pinching zooms the whole page; use the zoom buttons to zoom the images.) **Next changed box**
280
+ (N) scrolls both panes until the next region of changed pixels is in view and outlines it;
281
+ "box i of k" counts them. The views, each shown in the right pane at the same scale and place:
282
+ **Side by side**; **Flash**, which shows baseline and new one after the other in the same
283
+ place, about 1.5 times a second (the images themselves, not an overlay), keeping the zoom and
284
+ scroll it was opened at; **Highlight**, pixelmatch's changed pixels in red over the dimmed
285
+ baseline; and **Spotlight**, the new image dimmed everywhere except around the changed pixels
286
+ (each grown by 10 image pixels), which finds a one-pixel change. Flash, Highlight and
287
+ Spotlight need two images; on a new or removed story they are off and the page says why
288
+ ("New story, no baseline", "Only one image: this story was removed"). Badges here:
289
+ **size changed** (in image pixels), **flaky** (the two captures differed, then matched), and
290
+ **re-review** (an accept you made was replaced by the default branch's newer baseline).
291
+
292
+ **Next** and **Previous** (J and K) walk one pass: the items the grid showed when you opened
293
+ the story, in the grid's order, frozen until you go back to the grid. Accepting, rejecting or
294
+ excluding an item never drops it from the pass: the decision moves on to the next item, and
295
+ **Previous** comes back to the one just decided, showing its decision and an **Undo** (or U).
296
+ Going back to the grid shows what its filter now selects: under **Needs a decision** the items
297
+ you decided have left it, and the count has gone down; **Accepted**, **Rejected** and
298
+ **Excluded** show them with their decisions.
299
+
300
+ Statuses: `changed` (differs from its baseline), `new` (no baseline, and on a pull request the
301
+ story is new or looks different from the default branch's newest capture of it), `no baseline yet` (status
302
+ `unseeded`: no baseline, and the pull request does not change it), `removed` (a baseline whose
303
+ story no longer exists, lost a mode, or whose story's own parameters now exclude it), `unstable`
304
+ (two captures of the same commit differed), `failed` (did not render, even after one retry).
305
+
306
+ `no baseline yet` items are listed under their own filter in the grid and never need a decision:
307
+ they do not block the pull request, Accept all skips them, and the story screen offers no buttons
308
+ for them. Seed them from the default branch (below), or accept them on the pull request that changes them.
309
+
310
+ ## Keys
311
+
312
+ | Key | Action |
313
+ | ------------ | ------------------------------------------------------------------------------ |
314
+ | J / K | Next / previous item of this pass (decided items stay in it) |
315
+ | A | Accept an undecided item |
316
+ | R | Reject an undecided item (asks for a reason, then Enter) |
317
+ | E | Exclude an undecided item (asks for a reason, then Enter, then a confirmation) |
318
+ | U | Undo the item's decision (on the grid: each tile's Undo button) |
319
+ | F | Flash between baseline and new; F again returns to side by side |
320
+ | H | Highlight changed pixels; H again returns to side by side |
321
+ | S | Spotlight the changes; S again returns to side by side |
322
+ | Z | Next zoom: fit to screen, real size, 2x, 4x, 8x, then fit again |
323
+ | N | Next changed box |
324
+ | Space (hold) | Flash while held |
325
+ | Shift+A | Accept every undecided item of this project without opening it (asks first) |
326
+ | Escape | Back to the grid from a story, wherever the focus is (the reason box included) |
327
+ | Escape | On the grid: cancel an Undo N decisions or Undo all decisions pressed once |
328
+
329
+ No key reverses a decision. A, R and E do nothing on an item that is already decided, and say
330
+ so; to change a decision, press U (or the Undo button) first. The same key twice never undoes.
331
+
332
+ ## What each decision does
333
+
334
+ - **Accept**: the new screenshot becomes the baseline (or, for `removed`, the baseline is
335
+ deleted). Allowed on `changed`, `new` and `removed`.
336
+ - **Reject**: the difference is a regression. It always needs a reason, which is posted to the pull
337
+ request as a comment with a machine-readable block an agent can read. The pull request stays
338
+ blocked until its code changes so the capture matches the baseline again.
339
+ - **Exclude**: stops capturing the story. It needs a reason and writes
340
+ `<baselines>/<project>/<story id>.json` with `disableSnapshot: true`. It drops **every mode
341
+ of the story**, on every later pull request, until that file is deleted. It is the only
342
+ decision for `unstable` and `failed` items; for a one-off `failed` item (a timeout on a busy
343
+ runner), re-run the `visual` job instead, since the newest attempt replaces the old results.
344
+ - **Undo** (U, or a tile's Undo on the grid) clears a decision before Finish; it is the only way
345
+ to change one. The grid also undoes a whole component or project, after a second press.
346
+ Decisions are kept across server restarts.
347
+ - After Finish, accepts and exclusions are cleared; rejects stay, marked as already posted, and
348
+ still show as rejected on the next CI run while the capture is unchanged. Finish does not post
349
+ them twice. They live in the work directory's `state/` (the config's `workDir`), not in the
350
+ repository.
351
+
352
+ ## Finish
353
+
354
+ Finish applies every decision on one target at once:
355
+
356
+ - **A pull request:** one commit holding the accepted PNGs, the exclusion files and one review
357
+ record in `<baselines>/reviews/`, pushed to the pull request's branch, plus one comment
358
+ holding every reject. CI then recaptures, and the accepted items read `unchanged`.
359
+ - **One commit status**, "Visual review", posted once when Finish completes (never per
360
+ decision), on the commit Finish pushed, or on the captured commit when it pushed none. It
361
+ fails when anything was rejected, is pending while items are left undecided, and succeeds
362
+ otherwise; its description counts the accepts, rejects, exclusions and undecided items. It is
363
+ information for the pull request page, not a required check: the merge gate is the "Visual
364
+ gate" job. If posting it fails, the page says so; what was pushed and posted stays.
365
+ - **The default branch (seeding):** a branch `visual/seed-<date>` with the same commit and a pull
366
+ request from it, and one issue holding every reject (labelled with the config's `issueLabels`)
367
+ with the same machine-readable block, for a person or an agent to fix the stories. Rejects alone, with nothing accepted, open only the issue.
368
+
369
+ Finish runs on the server, not in the page. A seed of several hundred images takes minutes,
370
+ most of it uploading the images to Git LFS, which is longer than a browser (Safari on an iPad in
371
+ particular) keeps one request open. So pressing Finish only starts it, and the page then shows
372
+ each step as it happens: checking, writing the files, committing, uploading images to LFS (with a
373
+ count of the images uploaded so far), pushing, opening the pull request or posting the rejects,
374
+ and posting the status. When it ends, the page shows what was pushed and posted, or the error.
375
+ Closing or reloading the page does not stop it: reopen the page and it shows the running Finish
376
+ instead of a Finish button, and after it ends the result stays above the targets until the
377
+ server restarts. Only one Finish runs at a time, and decisions on that target are refused until
378
+ it ends.
379
+
380
+ The commit is signed by whatever git configuration the server process sees: yours when you
381
+ started it, someone else's when they (or an agent working for you) started it. The top of the
382
+ targets screen and Finish's confirmation name the key that will sign, where git found it and the
383
+ committer, and print the exact command that starts the same server from your own shell. If
384
+ Finish fails, your decisions are kept and the page shows git's or GitHub's message:
385
+
386
+ - **capture is stale, wait for CI**: someone pushed to the branch after the capture. Wait for the
387
+ new CI run, then decide again what still differs.
388
+ - **merge master first**: see the badge above.
389
+ - **failed to write commit object** or a signing error: unlock or plug in the signing key, then
390
+ Finish again.
391
+ - **the accepts were pushed ..., but the reject comment failed**: the accepts are done and cleared;
392
+ press Finish again to post the rejects.
393
+
394
+ ## Seeding: one story at a time
395
+
396
+ A story does not have to look right the first time, and nothing has to be seeded in one pass.
397
+ Seeding is per story:
398
+
399
+ 1. The review workflow captures every story on every push to the default branch. Start the
400
+ server with `--master-run <run id>` (that workflow's newest run on the default branch) and open
401
+ "master (seed)". Every story without a baseline is `new` there.
402
+ 2. **Accept** the stories that look right. **Reject** the ones that do not, with a reason saying
403
+ what is wrong. **Leave the rest** undecided; they simply stay without a baseline. Exclude only
404
+ stories that are unstable. Press Finish: the accepts become the seed pull request, and the
405
+ rejects become one issue whose machine-readable block says what to fix.
406
+ 3. Merge the seed pull request once its own capture shows its accepted items `unchanged`.
407
+
408
+ To seed from an older, known-good commit instead of the newest, capture it with the default
409
+ branch's tool: `gh workflow run visual-seed.yml --ref <default branch> -f ref=<sha>`, then start
410
+ the server with `--master-run <that run's id>`. It is listed as "master (seed)"; its results.json
411
+ names the captured commit, so Finish's seed branch starts from that commit.
412
+
413
+ A story with no baseline on the default branch is in the "no baseline yet" state. On every pull
414
+ request, CI compares its capture with the default branch's newest capture of that story:
415
+
416
+ - **The pull request does not change it:** `no baseline yet` (`unseeded`). It is shown, it does
417
+ not block the pull request, and it is never accepted by Accept all.
418
+ - **The pull request adds the story, or changes how it looks** (for example an agent fixing a
419
+ story you rejected): `new`. It blocks that pull request until you decide. Review it there;
420
+ accepting it creates its first baseline in that pull request's accept commit.
421
+
422
+ So seeding never restarts from scratch: each round accepts what now looks right, and the rest
423
+ waits, blocking nothing, until a pull request touches it. A project enters the merge gate when its
424
+ first baseline lands on the default branch; before that the gate ignores it entirely.
425
+
426
+ If the default branch's capture could not be downloaded (its artifacts expired, or no run there has
427
+ finished one), every story without a baseline is `new` on that pull request. Re-run its `visual` job
428
+ once the default branch's run has finished.
429
+
430
+ ## Iterating on a story before a pull request exists
431
+
432
+ To try a story's look quickly, capture it locally and look at it, as a PNG or in the page:
433
+
434
+ ```bash
435
+ npx visual-review install-browser # once per machine
436
+ npm run build-storybook # your project's build command
437
+ npx visual-review capture --project web --out .visual-review/preview/web --stories button--,badge--
438
+ ```
439
+
440
+ `--stories` captures only the story ids that start with one of the given prefixes, in seconds
441
+ rather than minutes, and then reports no baseline as removed. Start the server with
442
+ `--results .visual-review/preview` to see the capture beside its baseline. Capture and look again
443
+ after each change. A local preview is look only: its fonts and graphics stack are not CI's, so
444
+ only a CI capture of a pushed commit becomes a baseline. Push, let CI capture, and accept it on
445
+ the pull request.
446
+
447
+ ## How captures and baselines move
448
+
449
+ - **What a capture is.** Each story and mode is opened in a 1200 x 900 viewport at device scale
450
+ factor 2, as Chromatic captures, so a PNG holds two image pixels per CSS pixel. It is always
451
+ the whole canvas, in every project: the full page of the story iframe,
452
+ which is the whole viewport, or everything a scroll would reach when the story is taller or
453
+ wider. It is never cropped to the content, so a small component sits in the full canvas and
454
+ every capture of a project has the same size unless its story overflows. results.json records the scale as `scale`, and each review record
455
+ copies it into its `subject`.
456
+ - **From GitHub Actions to the page.** Each `visual` job uploads `results.json` and the PNGs to
457
+ review as an artifact `visual-<project>-<attempt>`, kept 30 days. The server lists open pull
458
+ requests with `gh`, finds each one's newest run of the capturing workflow, and downloads those
459
+ artifacts with `gh run download` into the work directory. It downloads nothing from Git LFS: the baselines a
460
+ capture was compared with travel inside the artifact.
461
+ - **What an accept does.** Finish writes the accepted PNGs (as LFS pointers, uploading the images
462
+ with `git lfs push`) and one review record in a throwaway worktree at the captured head,
463
+ commits, and pushes to the pull request's branch. CI then runs again on that branch.
464
+ - **Nothing restarts from scratch.** Every push captures again and compares with the baselines
465
+ the branch holds now, so after an accept the accepted items read `unchanged` and only what is
466
+ still undecided shows. Decisions you made but did not Finish are kept for every image whose
467
+ hash did not change.
468
+
469
+ ## Story parameters
470
+
471
+ Capture reads each story's `parameters.chromatic`, the same keys Chromatic reads, so stories
472
+ written for Chromatic work unchanged:
473
+
474
+ | Parameter | Effect |
475
+ | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
476
+ | `disableSnapshot: true` | The story is not captured (a baseline it still has is reported `removed`) |
477
+ | `diffThreshold` | pixelmatch's per-pixel colour threshold, 0 to 1 (default 0.063) |
478
+ | `diffIncludeAntiAliasing: true` | Count anti-aliased pixels as changes |
479
+ | `delay` | Milliseconds to wait after the render before the screenshot |
480
+ | `modes` | `{ "<name>": { <Storybook globals> } }`: one capture per mode, named `<story id>.<name>.png`; `disable: true` drops a mode |
481
+
482
+ Inside a story, `isChromatic()` from `chromatic/isChromatic` is true during capture (the URL carries
483
+ `chromatic=true`). A settings file `<baselines>/<project>/<story id>.json` overrides the story's
484
+ parameters; the page's Exclude writes one with `disableSnapshot: true` and your reason.
485
+
486
+ ## What the gate does and does not guarantee
487
+
488
+ - A pull request cannot pass the gate while its capture of a seeded project holds anything but
489
+ `unchanged`, `excluded` or `no baseline yet` items, including after "Re-run failed jobs" (the
490
+ highest attempt's artifact counts); a missing, unfinished or invalid capture blocks it too. A
491
+ rejected item stays blocking until a code change makes it match the baseline.
492
+ - `no baseline yet` rests on the default branch's capture being honest and recent: a story is
493
+ `new` (blocking) only when it looks different from the default branch's newest complete capture
494
+ of it. That capture may be a few merges older than the pull request's base; a story changed in
495
+ between shows as `new`, which blocks rather than passes.
496
+ - Every baseline PNG, and every settings file that excludes a story, that the pull request adds,
497
+ changes or deletes must be named with its new hash in a review record the pull request adds
498
+ under `<baselines>/reviews/`. A baseline PNG is a Git LFS pointer in git, and the gate reads the
499
+ image's hash from the pointer, so it never downloads an image. Existing records may not be
500
+ edited or deleted. This stops the shortcut of copying captured PNGs, or an exclusion, straight
501
+ into the baselines directory.
502
+ - **It does not prove a person reviewed anything.** A record is a plain JSON file: anyone who can
503
+ push to the branch can write one that names copied PNGs, and the gate cannot tell it from one
504
+ Finish wrote. Records are marked `"unproven": true` for that reason. What the gate shows is that
505
+ the captures match the pull request's baselines and that each baseline change carries a record.
506
+ - **Only the repository owner should approve.** The page runs on a development machine, where
507
+ anything running as you (an AI coding agent included) has your GitHub login and signing key and
508
+ could press Accept or call the page's API. Nothing technical prevents that today; tell your
509
+ agents not to, and keep the review to yourself.
510
+ - The projects the gate checks are the ones with baselines on the base branch, so removing a
511
+ project from the config does not remove it from the gate.
512
+ - The gate is part of a workflow file, which a pull request can edit, and a pull request can
513
+ loosen a story's own `diffThreshold` or `delay`, or a settings file's non-excluding keys,
514
+ without a review item. Read changes to those in code review.
515
+
516
+ ## Troubleshooting
517
+
518
+ - **"baseline is an LFS pointer; run git lfs pull".** The checkout was made without git-lfs, so
519
+ the baselines are small pointer files. Install git-lfs, run `git lfs install`, then
520
+ `git lfs pull`.
521
+ - **`serve` refuses to start, or Finish says git-lfs is missing or its filter is not
522
+ configured.** Install git-lfs and run `git lfs install` (it sets up the filters in your global
523
+ git configuration).
524
+ - **A push of baselines left CI unable to fetch them.** `git lfs install` adds a pre-push hook
525
+ that uploads the images a push points at. If your repository's hooks belong to husky (or any
526
+ other `core.hooksPath`), that hook is not installed: call `git lfs pre-push "$@"` from your own
527
+ pre-push hook. `git push --no-verify` skips the upload too; after one that carried baselines,
528
+ run `git lfs push origin <branch>`.
529
+ - **capture failed / no capture** on a target. The `visual` job produced no results. Open its
530
+ log from the page and re-run the job. **incomplete: N of M stories**: the job stopped part way
531
+ (a timeout); re-run it.
532
+ - **Every story is `new` on a pull request.** No reference capture of the default branch could be
533
+ downloaded (none finished yet, or its artifacts expired after 30 days). Re-run the `visual` job
534
+ once a run on the default branch has finished.
535
+ - **Finish says "capture is stale, wait for CI".** Someone pushed to the branch after the capture.
536
+ Wait for the new run, then decide again what still differs.
537
+ - **"merge master first".** The default branch has newer baselines for that project than the
538
+ pull request. Merge the default branch into the branch (by merge, never by rebase, so an accept
539
+ commit stays as it was made) and wait for CI.
540
+ - **Finish fails with "failed to write commit object"** or another signing error: unlock or plug
541
+ in your signing key, then press Finish again. Your decisions are kept.
542
+ - **"the accepts were pushed ..., but the reject comment failed".** The accepts are done; press
543
+ Finish again to post the rejects.
544
+ - **Opening the seed issue fails.** Every label in `issueLabels` must exist in the repository.
545
+ - **The pnpm setup step fails in CI.** `pnpm/action-setup` reads the pnpm version from the
546
+ `packageManager` field of your root `package.json`; add one.
547
+ - **A merge conflict under the baselines directory.** Take the default branch's side for every
548
+ file there and let CI capture again; review what still differs.
549
+ - **Captures differ from what you see locally.** Only CI's captures are compared: fonts and the
550
+ graphics stack differ from machine to machine. Look locally with `capture --stories` and
551
+ `serve --results`, but let CI's capture become the baseline.