@graphty/visual-review 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +448 -144
- package/capture/capture.mjs +7 -0
- package/package.json +1 -1
- package/templates/visual-review.yml +4 -2
- package/trusted/cli.mjs +1 -0
- package/trusted/gate.mjs +246 -42
- package/trusted/lib/accept.mjs +474 -116
- package/trusted/lib/approval.mjs +300 -0
- package/trusted/lib/config.mjs +4 -3
- package/trusted/lib/github.mjs +176 -29
- package/trusted/lib/serve.mjs +1237 -154
- package/trusted/lib/thumbs.mjs +149 -0
- package/trusted/page/index.html +20 -6
- package/trusted/page/passkey.js +94 -0
- package/trusted/page/review.css +924 -92
- package/trusted/page/review.js +3238 -991
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@ your login.
|
|
|
18
18
|
- [The GitHub Actions workflows](#the-github-actions-workflows)
|
|
19
19
|
- [Your first review: seeding baselines](#your-first-review-seeding-baselines)
|
|
20
20
|
- [Opening the review page](#opening-the-review-page), [the screens](#the-screens), [keys](#keys),
|
|
21
|
-
[decisions](#what-each-decision-does), [Finish](#finish)
|
|
21
|
+
[decisions](#what-each-decision-does), [Finish](#finish), [the passkey](#approving-with-a-passkey)
|
|
22
22
|
- [Seeding one story at a time](#seeding-one-story-at-a-time)
|
|
23
23
|
- [Iterating on a story before a pull request exists](#iterating-on-a-story-before-a-pull-request-exists)
|
|
24
24
|
- [Story parameters](#story-parameters)
|
|
@@ -147,10 +147,12 @@ Per project:
|
|
|
147
147
|
| `seedFromDefaultBranch` | `true` | `false`: the project's first baselines are accepted on a pull request, not seeded from the default branch |
|
|
148
148
|
| `waitFor` | none | After a story renders, call `method()` on every element matching `selector` and wait for the promise it returns, for a component that keeps drawing after Storybook says it is done. A console line containing `failOnConsole` fails the story |
|
|
149
149
|
|
|
150
|
-
Project ids are letters, digits
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
150
|
+
Project ids are lowercase letters, digits and `-` (results.json allows no others), and so are
|
|
151
|
+
the names of Chromatic modes, which a capture refuses before it starts. Every project is gated;
|
|
152
|
+
there is no setting that turns the gate off, and a config that sets `gate` is refused. The pull
|
|
153
|
+
request gate reads the config as it is on the base branch, so a pull request cannot move
|
|
154
|
+
`baselines` out from under it or drop a project from the gate; a project that a pull request adds
|
|
155
|
+
to its own config is gated too.
|
|
154
156
|
|
|
155
157
|
## The GitHub Actions workflows
|
|
156
158
|
|
|
@@ -164,7 +166,9 @@ drop a project from the gate; a project that a pull request adds to its own conf
|
|
|
164
166
|
difference; `continue-on-error` keeps even a crash of the tool from failing the run.
|
|
165
167
|
- **Visual gate** (pull requests only) downloads every capture of the run and runs
|
|
166
168
|
`visual-review gate` at the version `init` pinned, with `npx`, so a pull request's own
|
|
167
|
-
dependencies cannot change it.
|
|
169
|
+
dependencies cannot change it. It passes `--pr` with the pull request's number, which the
|
|
170
|
+
passkey check needs (see [Approving with a passkey](#approving-with-a-passkey)). Make it a
|
|
171
|
+
required check.
|
|
168
172
|
|
|
169
173
|
The review page finds captures by the workflow's file name (the config's `workflow`), the jobs by
|
|
170
174
|
their names, `visual (<project>)`, and the artifacts by `visual-<project>-<attempt>`. If you would
|
|
@@ -190,8 +194,7 @@ the capture:
|
|
|
190
194
|
|
|
191
195
|
1. Find that run's id: `gh run list --workflow visual-review.yml --branch main --limit 1`.
|
|
192
196
|
2. Start the page with `--master-run <run id>` ([Opening the review page](#opening-the-review-page))
|
|
193
|
-
and open
|
|
194
|
-
name). Every story is `new` there.
|
|
197
|
+
and open `<branch> seed` (for example "main seed"). Every story is `new` there.
|
|
195
198
|
3. Accept what looks right, reject what does not (with a reason), leave the rest, and press
|
|
196
199
|
Finish. You get a pull request `visual/seed-<date>` holding the accepted baselines, and one
|
|
197
200
|
issue listing the rejects.
|
|
@@ -221,19 +224,23 @@ token is kept in the work directory, so the URL stays valid across restarts; del
|
|
|
221
224
|
### Links to a screen
|
|
222
225
|
|
|
223
226
|
The address always names the screen you are on, after the token: the targets list; a pull
|
|
224
|
-
request (or
|
|
225
|
-
zoom,
|
|
226
|
-
`#token=...&target=123&project=web&filter=undecided&item=button--primary.dark.png&view=flash&zoom=2&box=on&blink=off`.
|
|
227
|
-
A link's `box` and `
|
|
228
|
-
|
|
227
|
+
request (or the default branch's seed) and project with the grid's filter and Find text; or one
|
|
228
|
+
story with its pass, view, zoom, outline, blink and Spotlight flash, for example
|
|
229
|
+
`#token=...&target=123&project=web&filter=undecided&item=button--primary.dark.png&pass=undecided&view=flash&zoom=2&box=on&blink=off&flash=off`.
|
|
230
|
+
A link's `box`, `blink` and `flash` apply to the page it opens; the choice this browser remembers
|
|
231
|
+
for B, L and F in Spotlight is left as it was. Flash, Blink and Spotlight flash open stopped from
|
|
232
|
+
a link; the first press of F, L or the button starts them.
|
|
229
233
|
Opening that address, in another tab or on another device, opens the same screen. **Copy link**
|
|
230
234
|
at the top right copies it. The link carries your session token, so it works on your iPad the way
|
|
231
235
|
the printed URL does; keep it to yourself as you would that URL. All of it sits after `#`, which a
|
|
232
236
|
browser never sends to any server or in a Referer, so the page never hands the token to another
|
|
233
|
-
site. Back and Forward move between the targets list, a grid and a story
|
|
234
|
-
or views of one grid
|
|
237
|
+
site. Back and Forward move between the targets list, a grid and a story, at once: they read the
|
|
238
|
+
server's cached list and never wait on GitHub. Moving between stories or views of one grid
|
|
239
|
+
updates the address in place. A reload of a story reopens the same pass at the same place (the
|
|
240
|
+
pass is kept in the tab's session storage; a link opened in a new tab rebuilds it from its
|
|
241
|
+
filter).
|
|
235
242
|
|
|
236
|
-
A link to something that is gone opens the nearest screen that still exists, and the status
|
|
243
|
+
A link to something that is gone opens the nearest screen that still exists, and the status row
|
|
237
244
|
says why: a story not in the newest CI run opens its grid, and a pull request no longer listed
|
|
238
245
|
(closed, or no CI run) opens the targets list.
|
|
239
246
|
|
|
@@ -244,70 +251,156 @@ starts the same server from your own shell.
|
|
|
244
251
|
|
|
245
252
|
## The screens
|
|
246
253
|
|
|
254
|
+
Every screen has the same frame. The header holds **Visual review** (the targets list), the
|
|
255
|
+
**Target** and **Project** menus (on the grid and story screens: jump to any pull request or
|
|
256
|
+
project, each with its count of undecided items), **Finish** with the number of decisions it
|
|
257
|
+
would publish ("Finish #201 (12)"; at 0 it is unavailable and says "Nothing new to finish",
|
|
258
|
+
shortened to "Nothing new" on an iPad; whether a passkey must approve it is said in Finish's
|
|
259
|
+
sheet), **Keys** and **Copy link**. Finish never shrinks: on a narrow window the
|
|
260
|
+
menus give up their width first. Under the header is the screen's own bar, then the status row: the one
|
|
261
|
+
place the page writes messages, one line tall on a wide screen and two on an iPad, so a message
|
|
262
|
+
never moves anything; a longer one shows **More**, which opens the row to its full length.
|
|
263
|
+
Errors are shown there in red. **Keys** (or `?`) lists every key, the last
|
|
264
|
+
20 messages in full, and a switch that turns the single-letter keys off.
|
|
265
|
+
|
|
266
|
+
No wait is silent: anything that takes longer than a third of a second says what it is waiting
|
|
267
|
+
for, with a count or the time spent, and a failed one offers Retry. A wait that keeps you from
|
|
268
|
+
working (the first list after the server starts, opening a project, a project whose captures are
|
|
269
|
+
still downloading, Finish) is a box in the middle of the screen, over the page: what it waits for,
|
|
270
|
+
its progress ("1 of 2 artifacts, 41 MB of 120 MB"), the time spent, GitHub's network retries
|
|
271
|
+
("GitHub did not answer (Could not resolve host: api.github.com). Trying again in 4 s, try 2 of
|
|
272
|
+
4."), and **Cancel** where there is something to go back to. A wait that fails becomes the error
|
|
273
|
+
in the same box, with **Retry**. Work in the background (checking GitHub for new CI runs,
|
|
274
|
+
downloading captures nobody has opened yet) is said in the status row and never blocks.
|
|
275
|
+
|
|
247
276
|
1. **Targets.** Each open pull request with a run of the capturing workflow, and the default
|
|
248
|
-
branch (shown as "master
|
|
277
|
+
branch (shown as `<branch> seed`, for example "master seed") when started with
|
|
278
|
+
`--master-run`. The list is the server's, shown at once with "Updated 40 s ago" and
|
|
279
|
+
**Refresh**; it is checked again with GitHub when you press Refresh or when it is over a minute
|
|
280
|
+
old, and the server checks every two minutes on its own, downloading the captures of every CI
|
|
281
|
+
run that finished, so they are there before you open them. The status row shows a check's step
|
|
282
|
+
("Checking GitHub for new CI runs", then "Finding CI runs: 3 of 5, 12 s"). The server keeps the
|
|
283
|
+
list on disk, so after a restart it shows at once and is checked behind; only the very first
|
|
284
|
+
start waits for GitHub, in the box. Above the cards, one line says whether Finish is approved with your passkey ("Finish is
|
|
285
|
+
approved with your passkey (iPad passkey, 2026-10-01), and the CI gate refuses accepts
|
|
286
|
+
without it."), or that accepts are not yet protected ("No passkey registered: accepts are not
|
|
287
|
+
yet protected. ..."), with **Register passkey**, or **Register another device** once one is
|
|
288
|
+
registered ([the passkey](#approving-with-a-passkey)); a problem reading
|
|
289
|
+
`visual-review/passkeys.json` is named there too. Each card says which commit and CI run it captured, any warning the server
|
|
290
|
+
has (with Retry), how many decisions are not yet finished, how many an earlier Finish already
|
|
291
|
+
put on the branch ("Finished: 266 decisions already on the branch, waiting for the next CI
|
|
292
|
+
run, which no longer shows them."), and a table per project: **Project**,
|
|
293
|
+
**Results** (count per status), **Decided** ("12 of 40") and **Review**. Projects with nothing
|
|
294
|
+
to review are one line ("3 projects unchanged: ..."). Badges:
|
|
249
295
|
- **merge master first**: the default branch has newer baselines for this project than the
|
|
250
296
|
pull request. Merge the default branch into the pull request's branch (by merge, never
|
|
251
297
|
rebase) and wait for CI.
|
|
252
|
-
- **
|
|
298
|
+
- **Downloading...**: the captures are still downloading from GitHub. The card says how many
|
|
299
|
+
artifacts and bytes have landed and for how long ("Downloading: 2 of 5 artifacts, 41 MB of
|
|
300
|
+
120 MB, 14 s"), and each row fills in by itself as its own download lands. Pressing it
|
|
301
|
+
opens the project as soon as it lands: its download goes ahead of the others, and the box
|
|
302
|
+
shows its progress.
|
|
303
|
+
- **capture failed**, **CI still running**, **waiting for CI**: there is nothing to review
|
|
304
|
+
yet. **Job log** opens the capturing job; **Retry** checks GitHub again.
|
|
305
|
+
- **artifact expired**: GitHub deleted the capture after 30 days and it was never
|
|
306
|
+
downloaded here. Re-run the `visual` job.
|
|
253
307
|
- **incomplete: N of M stories**: the capture stopped part way. Re-run the job.
|
|
254
|
-
- **
|
|
255
|
-
(`"seedFromDefaultBranch": false` in the config); its first baselines are accepted on
|
|
256
|
-
request.
|
|
257
|
-
2. **Grid.**
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
**
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
**
|
|
308
|
+
- **Reject only here: accept on a pull request**: this project is not reviewed on the default
|
|
309
|
+
branch (`"seedFromDefaultBranch": false` in the config); its first baselines are accepted on
|
|
310
|
+
a pull request.
|
|
311
|
+
2. **Grid.** A bar that stays at the top: "18 of 170 decided"; **Review 152 undecided**, the main
|
|
312
|
+
way in, which opens the first undecided item and walks every undecided item; **Needs a
|
|
313
|
+
decision** and **All**, each counted, and **More filters** (each status, and what you
|
|
314
|
+
**Accepted**, **Rejected** and **Excluded**, each counted); **Find story**; **Accept all
|
|
315
|
+
undecided (N)**; and **More**, with **Undo all decisions...** and **Copy link to this grid**.
|
|
316
|
+
Failed captures come first as one line, "6 failed captures: only Exclude applies"; opened (the
|
|
317
|
+
page remembers), it lists each with its reason and, under "console and stack", the story's
|
|
318
|
+
console output and the thrown error's stack (a play function's failed `expect` included). An
|
|
319
|
+
error is never accepted: fix the story, re-run the `visual` job for a one-off timeout, or
|
|
320
|
+
exclude it with a reason. Below it the items are grouped by component (the story id before
|
|
321
|
+
`--`), in the order failed, changed, moved, new, unstable, removed, no baseline yet; each
|
|
322
|
+
story's modes (light, dark) sit side by side under its name. Every item has a number that
|
|
323
|
+
stays the same however the grid is filtered or decided (its place under All), and the story
|
|
324
|
+
screen shows it too. Tiles show small copies the server makes once, loaded as they come near
|
|
325
|
+
the screen; one that fails reads "Failed -- tap to retry". A component's **Accept N** accepts
|
|
326
|
+
its undecided items without opening them, and **Undo N** clears its decisions; both ask first,
|
|
327
|
+
naming the count. Under every decided tile its decision is spelled out: "Accepted",
|
|
328
|
+
"Accepted (not opened)" for one Accept all took, or "Rejected" or "Excluded" with the reason,
|
|
329
|
+
with an **Undo** that clears it without opening the story. A reject an earlier Finish already
|
|
330
|
+
posted says "Posted by an earlier Finish: it stays.", and an accept or exclusion it pushed says
|
|
331
|
+
"Finished: it is on the branch, and the next CI run no longer shows it."; neither has an Undo. Many Undos at once show
|
|
332
|
+
their progress, with Stop. **Find story** narrows the grid as you type; Enter opens the first
|
|
333
|
+
match, or the item with that number. Coming back from a story, its tile is outlined and
|
|
334
|
+
scrolled into view.
|
|
335
|
+
3. **Story.** One item, on one screen that never scrolls (only the panes do). From the top:
|
|
336
|
+
- **The decision bar**: **Grid** (Escape), **Prev** (K), "12 of 230 -- 18 left" (in this
|
|
337
|
+
pass), **Next** (J), **Accept** (A), **Reject** (R), **Exclude** (E), **Undo** (U) and the
|
|
338
|
+
**Note** box ("Needed to Reject or Exclude"; a note typed before Accept is published with
|
|
339
|
+
it). Below 1280 px it is two rows, the decisions, then the movement and the note; on an
|
|
340
|
+
iPad held upright and below 900 px (Split View, a zoomed page) three, the note on a row of
|
|
341
|
+
its own, and the bar never runs past the window's edge. While the images load, Accept shows a
|
|
342
|
+
small spinner at its left edge; its label and key stay whole. Every button is always there, in the same place on every item, at every zoom; one
|
|
343
|
+
that does not apply is shown unavailable, the line under it says why, and pressing it says
|
|
344
|
+
why in the status row. Each button shows its key.
|
|
345
|
+
- **The item line**: the item's number, name, status and badges (**moved from ...**, its
|
|
346
|
+
decision, **size changed**, **flaky**, **re-review** when an accept you made was replaced by
|
|
347
|
+
the default branch's newer baseline), then one explanation: what changed ("880 pixels
|
|
348
|
+
changed, in a 40 x 40 area at (160, 80). Threshold 0.063."), what a decision will do
|
|
349
|
+
("Removed from the Storybook: Accept deletes its baseline."), or why one does not apply.
|
|
350
|
+
Tap it to read all of a long line.
|
|
351
|
+
- **The view bar**: **Side by side**, **Flash** (F), **Highlight** (H), **Spotlight** (S);
|
|
352
|
+
**Blink** (L) while Highlight is on and **Spotlight flash** (F) while Spotlight is on;
|
|
353
|
+
**Outline** (B); **Next change** (N) with "1 of 3"; the zoom, **Fit**, **1x**, **2x**,
|
|
354
|
+
**4x**, **8x** (Z cycles it); and **Details** (the threshold, the anti-aliasing setting, the
|
|
355
|
+
capture's scale, and any console output). Below 1280 pixels wide (an iPad either way up) it
|
|
356
|
+
is always two rows, the views on the first, so the zoom is always on screen and the panes
|
|
357
|
+
start at the same height on every item and in every view.
|
|
358
|
+
- **The two panes**, the baseline on the left and the new capture on the right, filling the
|
|
359
|
+
rest of the window. Both are drawn at once with "Loading baseline..." and "Loading new
|
|
360
|
+
image..." in them, so nothing moves when the images arrive; Accept shows a spinner until
|
|
361
|
+
they have. The next two items load in the background. A load that fails says so in its pane,
|
|
362
|
+
with Retry.
|
|
363
|
+
|
|
364
|
+
Images open at **Fit**: both whole images fit their panes, across and down, at one scale
|
|
365
|
+
(never above real size), so two captures of the same size line up pixel for pixel and nothing
|
|
366
|
+
scrolls. With no baseline (a new story, or "no baseline yet") the left pane stays as an empty
|
|
367
|
+
frame labeled "No baseline", so the new image sits exactly where it would beside one; a
|
|
368
|
+
removed story leaves the right pane empty the same way, and a failed one shows its log there.
|
|
369
|
+
**1x** is one CSS pixel of the page for each CSS pixel the story was drawn at (a capture holds
|
|
370
|
+
two image pixels per CSS pixel). **2x**, **4x** and **8x** enlarge it; from 4x pixels are
|
|
371
|
+
drawn as hard squares. Zoomed, the images grow past their panes, which scroll: scrolling one
|
|
372
|
+
scrolls the other to the same place, and **Fit** returns to the whole image. (On an iPad,
|
|
373
|
+
pinching zooms the whole page; use the zoom buttons to zoom the images. Held upright, each
|
|
374
|
+
pane is small: **2x** or Spotlight shows a fine change large.) **Next change** scrolls both
|
|
375
|
+
panes until the next region of changed pixels is in view and outlines it; **Outline** turns
|
|
376
|
+
that purple outline on and off, remembered in this browser. The views, each shown in the right
|
|
377
|
+
pane at the same scale and place: **Side by side**; **Flash**, which shows baseline and new one
|
|
378
|
+
after the other in the same place, about 1.5 times a second (the images themselves, not an
|
|
379
|
+
overlay), keeping the zoom and scroll it was opened at; **Highlight**, the changed pixels in
|
|
380
|
+
solid red laid over both images themselves, in both panes, where **Blink** flashes the red
|
|
381
|
+
pixels on and off at Flash's pace (remembered in this browser); and **Spotlight**, the new
|
|
382
|
+
image dimmed everywhere except around the changed pixels (each grown by 10 image pixels),
|
|
383
|
+
which finds a one-pixel change, where **Spotlight flash** shows the spotlighted baseline and
|
|
384
|
+
the spotlighted new image one after the other at Flash's pace, the pane's label saying which
|
|
385
|
+
(remembered in this browser). Flash, Highlight and Spotlight need two images; on a new or
|
|
386
|
+
removed story pressing them says so.
|
|
387
|
+
|
|
388
|
+
**Next** and **Prev** (J and K) walk one pass: the items the grid showed when you opened
|
|
389
|
+
the story (or every undecided item, from **Review N undecided**), in the grid's order, frozen
|
|
390
|
+
until you go back to the grid. Deciding an item never drops it from the pass: the decision
|
|
391
|
+
moves on to the next item, and **Prev** comes back to the one just decided, showing its
|
|
392
|
+
decision pressed and its **Undo**. Going back to the grid shows what its filter now selects.
|
|
393
|
+
|
|
394
|
+
**The end of a pass.** Next on the last item, or deciding it, does not wrap to the first: it
|
|
395
|
+
shows what is next in place of the panes. "End of graphty-element: 164 of 170 decided, 6
|
|
396
|
+
undecided." **Next project: layout (42 undecided)** comes first (focused, so Enter takes it)
|
|
397
|
+
and opens that project's first undecided item, with no wait for GitHub. **Review the 6
|
|
398
|
+
undecided** walks the ones left here, and comes first when no other project has any; when
|
|
399
|
+
every one left can only be excluded (unstable or failed) it says so: "Review the 6 undecided
|
|
400
|
+
(Exclude only)". Then **Back to the grid** and **Finish #201 (12)**. A project still downloading is listed under them ("layout: downloading
|
|
401
|
+
(2 of 5 projects done)"). When every project of the target is decided it says so, offers
|
|
402
|
+
**Finish** first, and **Next: #202 (340 undecided)**, the next pull request with something to
|
|
403
|
+
review. K comes back to the last item.
|
|
311
404
|
|
|
312
405
|
Statuses: `changed` (differs from its baseline), `moved` (a renamed story that looks exactly as
|
|
313
406
|
its old id's baseline; see [renames](#reorganizing-stories-renames)), `new` (no baseline, and on a pull request the
|
|
@@ -323,33 +416,55 @@ Seed them from the default branch (below), or accept them on the pull request.
|
|
|
323
416
|
|
|
324
417
|
## Keys
|
|
325
418
|
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
|
334
|
-
|
|
|
335
|
-
|
|
|
336
|
-
|
|
|
337
|
-
|
|
|
338
|
-
|
|
|
339
|
-
|
|
|
340
|
-
|
|
|
341
|
-
|
|
|
342
|
-
|
|
|
343
|
-
|
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
419
|
+
Keys work on the screen named, never while a question, Finish's sheet or the key list is open,
|
|
420
|
+
and never in a text box except where listed. **Keys** (or `?`) shows this list, and can turn the
|
|
421
|
+
single-letter keys off. The list opens with focus on itself, so a key pressed as it opens changes
|
|
422
|
+
nothing. Turning the letters off says so in the status row, and so does every letter typed while
|
|
423
|
+
they are off (the switch is remembered in this browser). On a touch screen every control is at
|
|
424
|
+
least 44 px tall.
|
|
425
|
+
|
|
426
|
+
| Key | Action |
|
|
427
|
+
| ---------------- | --------------------------------------------------------------------------------------------- |
|
|
428
|
+
| J / K | Next / previous item of this pass; J on the last item shows what is next |
|
|
429
|
+
| A | Accept, once the images are shown |
|
|
430
|
+
| (type), Esc, A | Accept with a note: type it in the note box, leave the box, accept |
|
|
431
|
+
| R | Reject; with an empty note box, type the reason, then Enter |
|
|
432
|
+
| E | Exclude; with an empty note box, type the reason, then Enter, then confirm |
|
|
433
|
+
| U | Undo the item's decision; you stay on the item |
|
|
434
|
+
| Enter (note box) | Send the Reject or Exclude waiting for its reason; otherwise just leave the box |
|
|
435
|
+
| F | Flash between baseline and new; F again returns to side by side |
|
|
436
|
+
| F | In Spotlight: flash the spotlighted baseline and new, or stop flashing |
|
|
437
|
+
| Space (hold) | Flash while held |
|
|
438
|
+
| H | Highlight changed pixels; H again returns to side by side |
|
|
439
|
+
| L | In Highlight: blink the red changed pixels, or hold them on |
|
|
440
|
+
| S | Spotlight the changes; S again returns to side by side |
|
|
441
|
+
| B | Outline the changed area, or stop outlining it |
|
|
442
|
+
| N | Next change |
|
|
443
|
+
| Z | Next zoom: Fit, 1x, 2x, 4x, 8x, then Fit again |
|
|
444
|
+
| Shift+A | Grid: accept every undecided item of this project without opening it (asks first) |
|
|
445
|
+
| / | Grid: Find story |
|
|
446
|
+
| Enter (end card) | Take the first offer: the next project, the undecided items left here, or Finish |
|
|
447
|
+
| ? | Show or hide the key list |
|
|
448
|
+
| Escape | Story: back to the grid; in the note box, first leaves the box (its text stays with the item) |
|
|
449
|
+
|
|
450
|
+
No key reverses a decision. A, R and E on an item that is already decided say "Already accepted.
|
|
451
|
+
Undo it to change it."; press U (or Undo) first. A held A, R, E or U decides once, and an A, R or
|
|
452
|
+
E that comes within a quarter second of an item's images appearing is ignored and says so, so the
|
|
453
|
+
second tap of a double tap never decides the next item unseen. While a decision is being saved
|
|
454
|
+
the page says "Saving the last decision..." and waits for it before moving on; a save that fails
|
|
455
|
+
leaves the item undecided, with its note.
|
|
456
|
+
|
|
457
|
+
Text typed in the note box belongs to the item on screen: it stays with that item while you move
|
|
458
|
+
away and back, and it is cleared when that item's decision is saved. Undo puts a decision's note
|
|
459
|
+
back in the box, so undoing to fix a typo does not lose it. After any decision, focus
|
|
460
|
+
leaves the note box, so the next A accepts instead of typing an "a".
|
|
347
461
|
|
|
348
462
|
## What each decision does
|
|
349
463
|
|
|
350
464
|
- **Accept**: the new screenshot becomes the baseline (or, for `removed`, the baseline is
|
|
351
|
-
deleted). Allowed on `changed`, `moved`, `new` and `removed`. For a renamed
|
|
352
|
-
is written under the new id and the old id's baseline is deleted, in the
|
|
465
|
+
deleted). Allowed on `changed`, `moved`, `new`, `no baseline yet` and `removed`. For a renamed
|
|
466
|
+
story the baseline is written under the new id and the old id's baseline is deleted, in the
|
|
467
|
+
same commit. A note typed with it is optional; Finish publishes it.
|
|
353
468
|
- **Reject**: the difference is a regression. It always needs a reason, which is posted to the pull
|
|
354
469
|
request as a comment with a machine-readable block an agent can read. The pull request stays
|
|
355
470
|
blocked until its code changes so the capture matches the baseline again.
|
|
@@ -359,54 +474,166 @@ so; to change a decision, press U (or the Undo button) first. The same key twice
|
|
|
359
474
|
decision for `unstable` and `failed` items; for a one-off `failed` item (a timeout on a busy
|
|
360
475
|
runner), re-run the `visual` job instead, since the newest attempt replaces the old results.
|
|
361
476
|
- **Undo** (U, or a tile's Undo on the grid) clears a decision before Finish; it is the only way
|
|
362
|
-
to change one. The grid also undoes a whole component or project
|
|
363
|
-
Decisions are kept across server restarts.
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
477
|
+
to change one. The grid also undoes a whole component (**Undo N**) or project (**More > Undo
|
|
478
|
+
all decisions...**), after asking. Decisions are kept across server restarts. A decision
|
|
479
|
+
applies only to the image it was taken on: when a new run or a re-run attempt captures that
|
|
480
|
+
item differently, it is undecided again (the old decision stays saved, and comes back if the
|
|
481
|
+
image does).
|
|
482
|
+
- After Finish, what it published stays shown, marked as published, and Finish does not publish
|
|
483
|
+
it twice. Accepts and exclusions read "Finished" and count as decided until a new CI run
|
|
484
|
+
replaces the capture (the accepted items are then `unchanged`, the excluded ones not captured);
|
|
485
|
+
rejects still show as rejected on the next CI run while the capture is unchanged. They live in the work directory's `state/` (the config's `workDir`), not in the
|
|
367
486
|
repository.
|
|
368
487
|
|
|
369
488
|
## Finish
|
|
370
489
|
|
|
371
|
-
Finish applies every decision on one target at once:
|
|
490
|
+
Finish applies every decision on one target, across all its projects, at once:
|
|
372
491
|
|
|
373
492
|
- **A pull request:** one commit holding the accepted PNGs, the exclusion files and one review
|
|
374
493
|
record in `<baselines>/reviews/`, pushed to the pull request's branch, plus one comment
|
|
375
|
-
holding every reject
|
|
494
|
+
holding every reject and every accept note (the machine-readable block holds the rejects
|
|
495
|
+
only). CI then recaptures, and the accepted items read `unchanged`.
|
|
496
|
+
- **Your passkey first, once one is known** (see [Approving with a passkey](#approving-with-a-passkey)):
|
|
497
|
+
the Finish sheet says "Your passkey confirms this Finish (Face ID or a security key)." and its
|
|
498
|
+
final button reads **Sign and finish #201**. Pressing it opens the Face ID (or Touch ID)
|
|
499
|
+
prompt at once; your device signs the record, and Finish commits exactly that record, as
|
|
500
|
+
version 2 with the approval in it. The comment names the committed record and its commit. A
|
|
501
|
+
rejects-only Finish commits nothing, so the comment's machine-readable block carries the
|
|
502
|
+
approved record itself. Cancelling the prompt changes nothing: the sheet comes back saying
|
|
503
|
+
"Passkey cancelled: nothing was changed." with its final button focused, so Enter tries again.
|
|
504
|
+
- **Before a passkey is registered, accepts are not yet protected.** Finish commits them with an
|
|
505
|
+
unapproved (version 1) record, and both the targets screen ("No passkey registered: accepts
|
|
506
|
+
are not yet protected.") and the Finish sheet ("Not yet protected: ...") say so. Rejects never
|
|
507
|
+
wait for a passkey to be registered.
|
|
376
508
|
- **One commit status**, "Visual review", posted once when Finish completes (never per
|
|
377
509
|
decision), on the commit Finish pushed, or on the captured commit when it pushed none. It
|
|
378
|
-
fails when anything was rejected, is pending while items are left undecided
|
|
379
|
-
otherwise; its description counts the accepts, rejects, exclusions and
|
|
380
|
-
information for the pull request page, not a required check: the merge
|
|
381
|
-
gate" job. If posting it fails, the page says so; what was pushed and
|
|
510
|
+
fails when anything was rejected, is pending while items are left undecided or a project did
|
|
511
|
+
not load, and succeeds otherwise; its description counts the accepts, rejects, exclusions and
|
|
512
|
+
undecided items. It is information for the pull request page, not a required check: the merge
|
|
513
|
+
gate is the "Visual gate" job. If posting it fails, the page says so; what was pushed and
|
|
514
|
+
posted stays, and the next Finish on that pull request posts a new status.
|
|
382
515
|
- **The default branch (seeding):** a branch `visual/seed-<date>` with the same commit and a pull
|
|
383
|
-
request from it, and one issue holding every reject
|
|
384
|
-
with the same machine-readable block, for a person
|
|
516
|
+
request from it, whose description lists the accept notes, and one issue holding every reject
|
|
517
|
+
(labeled with the config's `issueLabels`) with the same machine-readable block, for a person
|
|
518
|
+
or an agent to fix the stories. Rejects alone, with nothing accepted, open only the issue.
|
|
519
|
+
|
|
520
|
+
Pressing Finish opens a sheet that states exactly what will happen, from the server's own
|
|
521
|
+
counts: what will be committed and where ("Commit 214 accepts and 1 exclusion to feature."),
|
|
522
|
+
what will be posted ("Post 3 rejects and 2 accept notes as a comment on #201."), the status it
|
|
523
|
+
will set ("Then set the commit status 'Visual review' to failure (3 rejected)."), how many were
|
|
524
|
+
accepted without being opened, what is left undecided or was not loaded, every note it will
|
|
525
|
+
publish, and the key that will sign. If any decision changes after the sheet opened (in another
|
|
526
|
+
tab, say), Finish refuses, and the sheet comes back with the new summary. Once a passkey is
|
|
527
|
+
known, the final button reads **Sign and finish #201**, and pressing it asks for your passkey
|
|
528
|
+
(Face ID, Touch ID or a security key) before anything runs; the approval covers the rejects too,
|
|
529
|
+
so their reasons are yours. Cancelling it says "Passkey cancelled: nothing was changed." and the
|
|
530
|
+
sheet comes back with its final button focused, so Enter tries again. Before a passkey is
|
|
531
|
+
registered the sheet says accepts are not yet protected, and a Finish with only rejects never
|
|
532
|
+
waits for one.
|
|
385
533
|
|
|
386
534
|
Finish runs on the server, not in the page. A seed of several hundred images takes minutes,
|
|
387
535
|
most of it uploading the images to Git LFS, which is longer than a browser (Safari on an iPad in
|
|
388
|
-
particular) keeps one request open. So pressing Finish only starts it, and the
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
it
|
|
536
|
+
particular) keeps one request open. So pressing Finish only starts it, and the targets screen
|
|
537
|
+
then lists its steps, each marked done, in progress or waiting, with the count of images
|
|
538
|
+
uploaded and the time spent: confirming with your passkey, checking, writing the files,
|
|
539
|
+
committing, uploading images to LFS,
|
|
540
|
+
pushing, opening the pull request, posting the comment (or opening the issue), and posting the
|
|
541
|
+
status. When it ends, the page shows what was pushed and posted, with links, or the error, and
|
|
542
|
+
offers **Next: #202 (340 undecided)** and **Back to #201**. Closing or reloading the page does
|
|
543
|
+
not stop it: reopen the page and it shows the running Finish instead of a Finish button. Only
|
|
544
|
+
one Finish runs at a time, and decisions on that target are refused until it ends.
|
|
396
545
|
|
|
397
546
|
The commit is signed by whatever git configuration the server process sees: yours when you
|
|
398
547
|
started it, someone else's when they (or an agent working for you) started it. The top of the
|
|
399
|
-
targets screen and Finish's
|
|
548
|
+
targets screen and Finish's sheet name the key that will sign, where git found it and the
|
|
400
549
|
committer, and print the exact command that starts the same server from your own shell. If
|
|
401
|
-
Finish fails, your decisions are kept and the page shows git's or GitHub's
|
|
550
|
+
Finish fails, your decisions not yet published are kept and the page shows git's or GitHub's
|
|
551
|
+
message:
|
|
402
552
|
|
|
403
553
|
- **capture is stale, wait for CI**: someone pushed to the branch after the capture. Wait for the
|
|
404
554
|
new CI run, then decide again what still differs.
|
|
405
555
|
- **merge master first**: see the badge above.
|
|
406
556
|
- **failed to write commit object** or a signing error: unlock or plug in the signing key, then
|
|
407
557
|
Finish again.
|
|
408
|
-
- **the accepts were pushed ..., but the
|
|
409
|
-
press Finish again to post the rejects.
|
|
558
|
+
- **the accepts were pushed ..., but the comment with the rejects failed**: the accepts are done
|
|
559
|
+
and finished; press Finish again to post the rejects. Accept notes that were in that comment are
|
|
560
|
+
not posted again: the message names each one, so you can post them by hand. On a seed it reads
|
|
561
|
+
"the accepts were pushed as ... and opened the seed pull request, but the issue with the
|
|
562
|
+
rejects failed"; the accept notes are already in that pull request's description.
|
|
563
|
+
- **The comment with the accept notes was not posted**: the accepts are done; the message names
|
|
564
|
+
the notes, which are not kept.
|
|
565
|
+
|
|
566
|
+
## Approving with a passkey
|
|
567
|
+
|
|
568
|
+
A passkey (Face ID or Touch ID, kept in iCloud Keychain or another passkey manager) proves that
|
|
569
|
+
an accept came from your own device, for exactly the record Finish commits. Once your passkey is
|
|
570
|
+
in `visual-review/passkeys.json` on the default branch, the gate refuses every review record a
|
|
571
|
+
pull request adds unless it carries such an approval. Until then the gate enforces nothing.
|
|
572
|
+
|
|
573
|
+
### Registering the passkey
|
|
574
|
+
|
|
575
|
+
Do this once, yourself, never through an agent.
|
|
576
|
+
|
|
577
|
+
1. Start the page from your own shell on the host the passkey is for, over HTTPS, for example
|
|
578
|
+
`https://dev.example.com:9443`. The passkey belongs to that host name (its "rpId"), so serve
|
|
579
|
+
the page from the same host every time; the port may change.
|
|
580
|
+
2. On the targets screen press **Register passkey**, then **Create the passkey**, and confirm on
|
|
581
|
+
your device. The key is named by the kind of device that made it and the day ("iPad passkey,
|
|
582
|
+
2026-10-01", or "security key, 2026-10-01"), so every device shows which key is which.
|
|
583
|
+
3. The server opens a pull request adding the key to `visual-review/passkeys.json`, and the page
|
|
584
|
+
names the new key's credential id and that pull request. Check that the pull request names the
|
|
585
|
+
same id, then merge it. From that merge on, approvals are enforced.
|
|
586
|
+
|
|
587
|
+
The first key is trusted because you merged it: the server cannot check that a passkey was made
|
|
588
|
+
on a real device (Apple's passkeys give no attestation), so a key added by anything else that
|
|
589
|
+
can reach the page would look the same. Merge a key's pull request only right after you pressed
|
|
590
|
+
Register yourself and only when the ids match. Until the first key is merged, the server that
|
|
591
|
+
registered it already asks for it at Finish. Once the default branch holds a key, the server
|
|
592
|
+
trusts only the default branch's keys, never one registered since.
|
|
593
|
+
|
|
594
|
+
Until it merges, the targets screen reads "Passkey waiting for #650 to merge: Finish asks for it
|
|
595
|
+
already, but the CI gate checks approvals only once it is merged." Afterwards it reads "Finish is
|
|
596
|
+
approved with your passkey (`<name>`), and the CI gate refuses accepts without it." and offers
|
|
597
|
+
**Register another device**. Before any passkey is registered it reads "No passkey registered:
|
|
598
|
+
accepts are not yet protected.", and Finish commits accepts unapproved, as before passkeys.
|
|
599
|
+
|
|
600
|
+
### Finishing with Face ID
|
|
601
|
+
|
|
602
|
+
Finish asks for your passkey as soon as the server knows of a key. The record is built on the
|
|
603
|
+
server first, its SHA-256 is the challenge your device signs, and Finish refuses to commit a
|
|
604
|
+
record that differs from the one you approved ("the record changed after you approved it; press
|
|
605
|
+
Finish again"). Immediately before committing, Finish checks the approval with the gate's own
|
|
606
|
+
code. On a Mac, Touch ID or your login password takes Face ID's place.
|
|
607
|
+
|
|
608
|
+
### What the gate checks
|
|
609
|
+
|
|
610
|
+
For each review record a pull request adds, with `node:crypto` alone:
|
|
611
|
+
|
|
612
|
+
- it is version 2, names this pull request (or none, for a seed), and is not a copy of a record
|
|
613
|
+
already on the base branch;
|
|
614
|
+
- its approval is by a key in `visual-review/passkeys.json` as the base branch has it, over the
|
|
615
|
+
SHA-256 of exactly this record, made on an HTTPS page whose host is exactly the key's host,
|
|
616
|
+
with user verification (Face ID, Touch ID or the device's passcode).
|
|
617
|
+
|
|
618
|
+
A record that fails counts for nothing. Then every changed baseline PNG, every added or changed
|
|
619
|
+
settings file and any change to `visual-review/passkeys.json` must be accounted for: the records'
|
|
620
|
+
items, oldest first, must take the file from its contents on the base branch to its contents in
|
|
621
|
+
the pull request. A record approved for other contents (an old seed, or a decision you replaced
|
|
622
|
+
later in the same pull request) therefore moves nothing. Records already on the base branch are
|
|
623
|
+
never checked again, so baselines accepted before the passkey are kept as they are.
|
|
624
|
+
|
|
625
|
+
### Replacing the passkey
|
|
626
|
+
|
|
627
|
+
An iCloud Keychain passkey is on every device signed in to your Apple account, so a lost device
|
|
628
|
+
loses nothing. Once the default branch holds a key, the gate fails any pull request that changes
|
|
629
|
+
`visual-review/passkeys.json`, so no pull request can swap in another key. To add or replace one
|
|
630
|
+
anyway (another passkey manager, another host, a lost Apple account), register it on the page;
|
|
631
|
+
the pull request it opens says the gate fails it. Check the credential id, and merge it as an
|
|
632
|
+
administrator past the failing check. Removing every key is refused the same way.
|
|
633
|
+
|
|
634
|
+
`visual-review/passkeys.json` sits at that path in every repository that uses this tool. It
|
|
635
|
+
holds public keys only: `{ "version": 1, "keys": [{ "id", "publicKey", "rpId", "label",
|
|
636
|
+
"registeredAt" }] }`, with the public key as base64url SubjectPublicKeyInfo (P-256).
|
|
410
637
|
|
|
411
638
|
## Seeding: one story at a time
|
|
412
639
|
|
|
@@ -415,7 +642,7 @@ Seeding is per story:
|
|
|
415
642
|
|
|
416
643
|
1. The review workflow captures every story on every push to the default branch. Start the
|
|
417
644
|
server with `--master-run <run id>` (that workflow's newest run on the default branch) and open
|
|
418
|
-
"
|
|
645
|
+
`<branch> seed` (for example "main seed"). Every story without a baseline is `new` there.
|
|
419
646
|
2. **Accept** the stories that look right. **Reject** the ones that do not, with a reason saying
|
|
420
647
|
what is wrong. **Leave the rest** undecided; they simply stay without a baseline. Exclude only
|
|
421
648
|
stories that are unstable. Press Finish: the accepts become the seed pull request, and the
|
|
@@ -424,7 +651,7 @@ Seeding is per story:
|
|
|
424
651
|
|
|
425
652
|
To seed from an older, known-good commit instead of the newest, capture it with the default
|
|
426
653
|
branch's tool: `gh workflow run visual-seed.yml --ref <default branch> -f ref=<sha>`, then start
|
|
427
|
-
the server with `--master-run <that run's id>`. It is listed as "
|
|
654
|
+
the server with `--master-run <that run's id>`. It is listed as `<branch> seed` (for example "main seed"); its results.json
|
|
428
655
|
names the captured commit, so Finish's seed branch starts from that commit.
|
|
429
656
|
|
|
430
657
|
A story with no baseline on the default branch is in the "no baseline yet" state. On every pull
|
|
@@ -507,14 +734,16 @@ written for Chromatic work unchanged:
|
|
|
507
734
|
| Parameter | Effect |
|
|
508
735
|
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
|
509
736
|
| `disableSnapshot: true` | The story is not captured (a baseline it still has is reported `removed`) |
|
|
510
|
-
| `diffThreshold` | pixelmatch's per-pixel
|
|
737
|
+
| `diffThreshold` | pixelmatch's per-pixel color threshold, 0 to 1 (default 0.063); the gate fails a story above 0.8 |
|
|
511
738
|
| `diffIncludeAntiAliasing: true` | Count anti-aliased pixels as changes |
|
|
512
739
|
| `delay` | Milliseconds to wait after the render before the screenshot |
|
|
513
740
|
| `modes` | `{ "<name>": { <Storybook globals> } }`: one capture per mode, named `<story id>.<name>.png`; `disable: true` drops a mode |
|
|
514
741
|
|
|
515
742
|
Inside a story, `isChromatic()` from `chromatic/isChromatic` is true during capture (the URL carries
|
|
516
743
|
`chromatic=true`). A settings file `<baselines>/<project>/<story id>.json` overrides the story's
|
|
517
|
-
parameters; the page's Exclude writes one with `disableSnapshot: true` and your reason.
|
|
744
|
+
parameters; the page's Exclude writes one with `disableSnapshot: true` and your reason. A
|
|
745
|
+
pull request that adds or changes a settings file needs a review record for it, like a baseline,
|
|
746
|
+
so set the other keys in the story's parameters, where code review sees them.
|
|
518
747
|
|
|
519
748
|
## Reorganizing stories: renames
|
|
520
749
|
|
|
@@ -564,26 +793,57 @@ PNGs move: a settings file (`<old id>.json`) is not renamed; rename it in the sa
|
|
|
564
793
|
- A story with no baseline always blocks. `new` and `no baseline yet` only tell the reviewer
|
|
565
794
|
whether the pull request changed it, measured against the default branch's newest complete
|
|
566
795
|
capture, which may be a few merges older than the pull request's base.
|
|
567
|
-
- Every baseline PNG
|
|
568
|
-
|
|
569
|
-
under `<baselines>/reviews
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
796
|
+
- Every baseline PNG the pull request adds, changes or deletes, and every settings file it adds
|
|
797
|
+
or changes, must be taken from its contents on the base branch to its new contents by the
|
|
798
|
+
review records the pull request adds under `<baselines>/reviews/` (each item names a path, its
|
|
799
|
+
`from` hash and its `to` hash). A baseline PNG is a Git LFS pointer in git, and the gate reads
|
|
800
|
+
the image's hash from the pointer, so it never downloads an image. Existing records may not be
|
|
801
|
+
edited or deleted. This stops the shortcut of copying captured PNGs, or a settings file that
|
|
802
|
+
excludes a story or loosens its comparison, straight into the baselines directory. Deleting a
|
|
803
|
+
settings file and editing `renames.json` need no record: the captures they cause are reviewed.
|
|
804
|
+
- A story compared at a `diffThreshold` above 0.8 fails the gate (at 1 nothing ever reads as
|
|
805
|
+
changed), and so does a pull request that moves the baselines directory in its config.
|
|
806
|
+
- **Before a passkey is registered, it does not prove a person reviewed anything.** A record is a
|
|
807
|
+
plain JSON file: anyone who can push to the branch can write one that names copied PNGs, and
|
|
808
|
+
the gate cannot tell it from one Finish wrote. Such records are marked `"unproven": true`.
|
|
809
|
+
- **Once `visual-review/passkeys.json` on the base branch holds a key**, every record the pull
|
|
810
|
+
request adds must be version 2, name this pull request (or none, for a seed), and carry a
|
|
811
|
+
passkey approval over exactly that record by one of the base branch's keys, made on an HTTPS
|
|
812
|
+
page on the key's host, with user verification (Face ID, Touch ID or a PIN). A record that
|
|
813
|
+
fails, an old-format record, or one copied from another pull request counts for nothing, so
|
|
814
|
+
the baselines it names are reported as unreviewed. Keys are read only from the base branch,
|
|
815
|
+
never from the pull request, and a pull request that would leave no key fails.
|
|
816
|
+
- **It does not prove you looked at every image.** It proves your device approved the record,
|
|
817
|
+
which lists every accepted and rejected image by hash. A page altered on your machine could ask
|
|
818
|
+
you to approve something other than what it shows; read the counts in Finish's question.
|
|
819
|
+
- **An approved seed that was never merged can be applied by another pull request.** A seed's
|
|
820
|
+
record names no pull request. Copying one already on the base branch fails, and so does one
|
|
821
|
+
whose `from` hashes are no longer the base branch's; but a seed you approved and then abandoned
|
|
822
|
+
without merging still matches, and would apply exactly the images you approved for it. Delete a
|
|
823
|
+
seed branch you do not want, and close its pull request.
|
|
577
824
|
- **Only the repository owner should approve.** The page runs on a development machine, where
|
|
578
825
|
anything running as you (an AI coding agent included) has your GitHub login and signing key and
|
|
579
|
-
could press Accept or call the page's API.
|
|
580
|
-
|
|
826
|
+
could press Accept or call the page's API. Before a passkey is registered nothing technical
|
|
827
|
+
prevents that; afterwards an agent can still decide, but cannot produce the approval Face ID
|
|
828
|
+
gives. Tell your agents not to register passkeys or use the page.
|
|
581
829
|
- The projects the gate checks are every project in the base branch's config and in the pull
|
|
582
830
|
request's config, seeded or not, plus every project with baselines on the base branch. So
|
|
583
831
|
removing a project from the config does not remove it from the gate.
|
|
584
|
-
-
|
|
585
|
-
|
|
586
|
-
|
|
832
|
+
- In this repository's CI, the gate and the capture run as the base branch has them, never the
|
|
833
|
+
pull request's copy, so a pull request cannot loosen the code that judges it; a change to
|
|
834
|
+
either is first exercised by the pull request after it. The `npx` gate of the workflow `init`
|
|
835
|
+
writes runs a pinned published version, to the same end.
|
|
836
|
+
- **The workflow file itself can be edited by the pull request**, which could drop the gate
|
|
837
|
+
step. Closing that needs a check the pull request cannot edit (a ruleset-required workflow).
|
|
838
|
+
The gate prints a warning when a pull request changes `visual-review/passkeys.json`, the tool's
|
|
839
|
+
trusted code or capture, or the workflow that runs it; read those changes in code review.
|
|
840
|
+
- A pull request can still loosen a story's own `diffThreshold` (up to 0.8) or `delay` in the
|
|
841
|
+
story's source, and a story's code runs in the capture browser, so it could draw anything.
|
|
842
|
+
Read story changes in code review.
|
|
843
|
+
- **Every page on the passkey's host can ask for it.** The rpId is a host name, and every server
|
|
844
|
+
on that host (another dev server, a Storybook on another port) can call the passkey prompt with
|
|
845
|
+
a challenge of its choosing; the prompt names only the host. Approve only from the review page,
|
|
846
|
+
right after pressing Finish. A host that serves nothing but the review page closes this.
|
|
587
847
|
|
|
588
848
|
## Troubleshooting
|
|
589
849
|
|
|
@@ -598,6 +858,21 @@ PNGs move: a settings file (`<old id>.json`) is not renamed; rename it in the sa
|
|
|
598
858
|
other `core.hooksPath`), that hook is not installed: call `git lfs pre-push "$@"` from your own
|
|
599
859
|
pre-push hook. `git push --no-verify` skips the upload too; after one that carried baselines,
|
|
600
860
|
run `git lfs push origin <branch>`.
|
|
861
|
+
- **download failed / failed to load: ...; reload the page to retry.** `serve` starts
|
|
862
|
+
downloading every capture as soon as it starts, and retries a gh call that fails on the network
|
|
863
|
+
(DNS, a dropped connection, a GitHub 5xx) three times over about 20 seconds; it logs each failed
|
|
864
|
+
call and each retry to stderr. A project whose download still fails shows "download failed", a
|
|
865
|
+
pull request (or the default branch's run) GitHub would not answer for shows "failed to load",
|
|
866
|
+
or, when it loaded before, keeps what it showed with "could not refresh", and everything else
|
|
867
|
+
loads as usual. Reload the page to try again; captures already downloaded are kept, and a
|
|
868
|
+
damaged one is downloaded again.
|
|
869
|
+
- **A gh or git call hangs.** Every gh and git call `serve` and Finish make is stopped after 10
|
|
870
|
+
minutes (`VISUAL_REVIEW_TIMEOUT_MS` sets another limit, in milliseconds), and git never waits
|
|
871
|
+
for a credential prompt. A stopped Finish names the step it was on and keeps your decisions.
|
|
872
|
+
- **"the server stopped while this Finish was at ..."** The server restarted during a Finish.
|
|
873
|
+
Look at the branch on origin to see whether its commit was pushed before pressing Finish again.
|
|
874
|
+
- **"... was pushed as ..., but opening its pull request failed"** (the seed). Press Finish
|
|
875
|
+
again: it opens the pull request for the branch already pushed.
|
|
601
876
|
- **capture failed / no capture** on a target. The `visual` job produced no results. Open its
|
|
602
877
|
log from the page and re-run the job. **incomplete: N of M stories**: the job stopped part way
|
|
603
878
|
(a timeout); re-run it.
|
|
@@ -611,8 +886,37 @@ PNGs move: a settings file (`<old id>.json`) is not renamed; rename it in the sa
|
|
|
611
886
|
commit stays as it was made) and wait for CI.
|
|
612
887
|
- **Finish fails with "failed to write commit object"** or another signing error: unlock or plug
|
|
613
888
|
in your signing key, then press Finish again. Your decisions are kept.
|
|
614
|
-
- **"the accepts were pushed ..., but the
|
|
615
|
-
Finish again to post the rejects.
|
|
889
|
+
- **"the accepts were pushed ..., but the comment with the rejects failed".** The accepts are
|
|
890
|
+
done; press Finish again to post the rejects. Accept notes it held are named in the message and
|
|
891
|
+
not posted again.
|
|
892
|
+
- **"Passkey failed (This is an invalid domain.)"** or similar: the page is served from an IP
|
|
893
|
+
address or plain http. Serve it over https from a host name.
|
|
894
|
+
- **"the record changed after you approved it; press Finish again".** The decisions, the capture
|
|
895
|
+
or the default branch changed between your Face ID and the commit. Press Finish again and
|
|
896
|
+
approve the new record. **"the approval is stale"** means the same, from another tab or after
|
|
897
|
+
ten minutes.
|
|
898
|
+
- **"no passkey is registered for `<host>`".** Passkeys belong to the host name the page is served
|
|
899
|
+
from. Serve the page from the host your passkey is for, or register one for this host.
|
|
900
|
+
- **"Not approved (the passkey prompt was cancelled or refused): nothing was changed".** Press
|
|
901
|
+
Finish again.
|
|
902
|
+
- **The gate says "changed with no review record taking it from its base branch contents to
|
|
903
|
+
these".** Either nothing approved the change, or the default branch changed the same file
|
|
904
|
+
after your Finish, so the record starts from contents the base no longer has. Merge the default
|
|
905
|
+
branch into the pull request, let CI capture again, and review the file again.
|
|
906
|
+
- **The gate says a record is "a copy of a record already on the base branch".** An approval
|
|
907
|
+
counts once. Remove the copied record and its files, and review the change on this pull
|
|
908
|
+
request.
|
|
909
|
+
- **The gate says a story is compared at a diffThreshold above 0.8.** Lower it in the story's
|
|
910
|
+
parameters or its settings file.
|
|
911
|
+
- **The gate fails a change to `visual-review/passkeys.json`.** See [Replacing the
|
|
912
|
+
passkey](#replacing-the-passkey).
|
|
913
|
+
- **The gate says "approval is from a key not in passkeys.json on the base branch".** The record
|
|
914
|
+
was approved with a key that the base branch does not hold yet: merge the pull request that
|
|
915
|
+
registers it first, then review again.
|
|
916
|
+
- **The gate says a record has no passkey approval** on a pull request Finished before your
|
|
917
|
+
passkey was registered. Revert its accept commit (which takes the record and the baselines out
|
|
918
|
+
of the diff), let CI capture again, and review it again with Face ID. Never edit a record by
|
|
919
|
+
hand: the gate only accepts what your device approved.
|
|
616
920
|
- **Opening the seed issue fails.** Every label in `issueLabels` must exist in the repository.
|
|
617
921
|
- **The pnpm setup step fails in CI.** `pnpm/action-setup` reads the pnpm version from the
|
|
618
922
|
`packageManager` field of your root `package.json`; add one.
|