@graphty/visual-review 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -22,6 +22,7 @@ your login.
22
22
  - [Seeding one story at a time](#seeding-one-story-at-a-time)
23
23
  - [Iterating on a story before a pull request exists](#iterating-on-a-story-before-a-pull-request-exists)
24
24
  - [Story parameters](#story-parameters)
25
+ - [Reorganizing stories: renames](#reorganizing-stories-renames)
25
26
  - [What the gate does and does not guarantee](#what-the-gate-does-and-does-not-guarantee)
26
27
  - [Troubleshooting](#troubleshooting)
27
28
 
@@ -213,9 +214,11 @@ token is kept in the work directory, so the URL stays valid across restarts; del
213
214
  ### Links to a screen
214
215
 
215
216
  The address always names the screen you are on, after the token: the targets list; a pull
216
- request (or master) and project with the grid's filter and text; or one story with its view and
217
- zoom, for example
218
- `#token=...&target=123&project=web&filter=undecided&item=button--primary.dark.png&view=flash&zoom=2`.
217
+ request (or master) and project with the grid's filter and text; or one story with its view,
218
+ zoom, changed box and blink, for example
219
+ `#token=...&target=123&project=web&filter=undecided&item=button--primary.dark.png&view=flash&zoom=2&box=on&blink=off`.
220
+ A link's `box` and `blink` apply to the page it opens; the choice this browser remembers for B
221
+ and L is left as it was.
219
222
  Opening that address, in another tab or on another device, opens the same screen. **Copy link**
220
223
  at the top right copies it. The link carries your session token, so it works on your iPad the way
221
224
  the printed URL does; keep it to yourself as you would that URL. All of it sits after `#`, which a
@@ -278,11 +281,13 @@ starts the same server from your own shell.
278
281
  other to the same place, and **Fit to screen** returns to the whole image. (On an iPad,
279
282
  pinching zooms the whole page; use the zoom buttons to zoom the images.) **Next changed box**
280
283
  (N) scrolls both panes until the next region of changed pixels is in view and outlines it;
281
- "box i of k" counts them. The views, each shown in the right pane at the same scale and place:
284
+ "box i of k" counts them. **Box** (B) turns that outline on and off; the page remembers the
285
+ choice in this browser. The views, each shown in the right pane at the same scale and place:
282
286
  **Side by side**; **Flash**, which shows baseline and new one after the other in the same
283
287
  place, about 1.5 times a second (the images themselves, not an overlay), keeping the zoom and
284
- scroll it was opened at; **Highlight**, pixelmatch's changed pixels in red over the dimmed
285
- baseline; and **Spotlight**, the new image dimmed everywhere except around the changed pixels
288
+ scroll it was opened at; **Highlight**, the changed pixels in solid red laid over both images
289
+ themselves, in both panes, where **Blink** (L) flashes the red pixels on and off at Flash's
290
+ pace (remembered in this browser); and **Spotlight**, the new image dimmed everywhere except around the changed pixels
286
291
  (each grown by 10 image pixels), which finds a one-pixel change. Flash, Highlight and
287
292
  Spotlight need two images; on a new or removed story they are off and the page says why
288
293
  ("New story, no baseline", "Only one image: this story was removed"). Badges here:
@@ -297,7 +302,8 @@ starts the same server from your own shell.
297
302
  you decided have left it, and the count has gone down; **Accepted**, **Rejected** and
298
303
  **Excluded** show them with their decisions.
299
304
 
300
- Statuses: `changed` (differs from its baseline), `new` (no baseline, and on a pull request the
305
+ Statuses: `changed` (differs from its baseline), `moved` (a renamed story that looks exactly as
306
+ its old id's baseline; see [renames](#reorganizing-stories-renames)), `new` (no baseline, and on a pull request the
301
307
  story is new or looks different from the default branch's newest capture of it), `no baseline yet` (status
302
308
  `unseeded`: no baseline, and the pull request does not change it), `removed` (a baseline whose
303
309
  story no longer exists, lost a mode, or whose story's own parameters now exclude it), `unstable`
@@ -321,6 +327,8 @@ for them. Seed them from the default branch (below), or accept them on the pull
321
327
  | S | Spotlight the changes; S again returns to side by side |
322
328
  | Z | Next zoom: fit to screen, real size, 2x, 4x, 8x, then fit again |
323
329
  | N | Next changed box |
330
+ | B | Outline the changed box, or stop outlining it |
331
+ | L | In Highlight: blink the red changed pixels, or hold them on |
324
332
  | Space (hold) | Flash while held |
325
333
  | Shift+A | Accept every undecided item of this project without opening it (asks first) |
326
334
  | Escape | Back to the grid from a story, wherever the focus is (the reason box included) |
@@ -332,7 +340,8 @@ so; to change a decision, press U (or the Undo button) first. The same key twice
332
340
  ## What each decision does
333
341
 
334
342
  - **Accept**: the new screenshot becomes the baseline (or, for `removed`, the baseline is
335
- deleted). Allowed on `changed`, `new` and `removed`.
343
+ deleted). Allowed on `changed`, `moved`, `new` and `removed`. For a renamed story the baseline
344
+ is written under the new id and the old id's baseline is deleted, in the same commit.
336
345
  - **Reject**: the difference is a regression. It always needs a reason, which is posted to the pull
337
346
  request as a comment with a machine-readable block an agent can read. The pull request stays
338
347
  blocked until its code changes so the capture matches the baseline again.
@@ -483,6 +492,45 @@ Inside a story, `isChromatic()` from `chromatic/isChromatic` is true during capt
483
492
  `chromatic=true`). A settings file `<baselines>/<project>/<story id>.json` overrides the story's
484
493
  parameters; the page's Exclude writes one with `disableSnapshot: true` and your reason.
485
494
 
495
+ ## Reorganizing stories: renames
496
+
497
+ Storybook derives a story's id from its title, so moving stories in the sidebar (a new `title`,
498
+ a new folder) gives every moved story a new id. Without help, each old id's baseline is reported
499
+ `removed` and each new id is `new`, with no before and after to compare. A renames file keeps them
500
+ paired. Add it in the pull request that moves the stories, at `<baselines>/<project>/renames.json`:
501
+
502
+ ```json
503
+ [
504
+ { "from": "building-a-panel-fieldrow--default", "to": "components-panels-and-rows-fieldrow--default" },
505
+ { "from": "compact-theme-mantine-components-badge--dot", "to": "components-display-badge--dot" }
506
+ ]
507
+ ```
508
+
509
+ Each entry is one story: `from` is its old id, `to` its new one. Modes map on their own, so
510
+ `<from>.dark.png` is compared with the capture of `<to>` in the dark mode, and `<from>.light.png`
511
+ with the light one; a mode the new story no longer has is still reported `removed`.
512
+
513
+ - **Capture** compares the new id's capture with the old id's baseline. It reads `moved` when the
514
+ two look the same (only the name changed) and `changed` when they differ; either way the item
515
+ carries `from`, and the old baseline is not reported `removed`.
516
+ - **The review page** shows each one as a pair, labeled "moved from &lt;old id&gt;" on the tile
517
+ and on the story screen, with the old id's baseline on the left. The grid has a **moved** filter,
518
+ and a component's Accept N undecided takes its moved stories too.
519
+ - **Accepting** writes the baseline under the new id and deletes the old id's baseline in the same
520
+ accept commit. For a `moved` item the image is the same, so git sees a rename and the Git LFS
521
+ pointer does not change. The review record names both paths.
522
+ - **The gate** blocks a `moved` item until it is accepted, like any change: a pull request that
523
+ moves stories cannot land without their baselines moving with them.
524
+ - **A rename whose new id is not a story** in the Storybook is an error: capture reports it as a
525
+ `failed` item under the new id, with the reason, and the page lists it under Errors. Fix the
526
+ entry in `renames.json`. A rename whose old id is still a story is not a move, and does
527
+ nothing.
528
+ - **An entry whose move was accepted** does nothing any more: the old baseline is gone and the new
529
+ id has its own. You can delete the file once the pull request has merged, or leave it.
530
+
531
+ `renames.json` is read from the pull request's own checkout, like the baselines. Only baseline
532
+ PNGs move: a settings file (`<old id>.json`) is not renamed; rename it in the same pull request.
533
+
486
534
  ## What the gate does and does not guarantee
487
535
 
488
536
  - A pull request cannot pass the gate while its capture of a seeded project holds anything but
@@ -21,6 +21,12 @@
21
21
  * second capture of an unstable one, and `baselines/<file>`: the baseline each changed, unstable
22
22
  * and removed item was compared with.
23
23
  *
24
+ * A story renamed in the project's `renames.json` (see loadRenames) is compared with the baseline
25
+ * of its old id, mode by mode: `moved` when it looks the same, `changed` otherwise, each carrying
26
+ * `from`; that old baseline is then not reported `removed`. A rename whose new id is not a story
27
+ * is a `failed` item under the new id, so the page lists it; one whose old id is still a story
28
+ * does nothing.
29
+ *
24
30
  * With `reference`, a directory holding the default branch's newest capture of the project (CI
25
31
  * downloads it on pull requests), a story with no baseline whose capture matches it is `unseeded`, not
26
32
  * `new`: seeding is per story, so a story nobody has accepted yet does not block every pull
@@ -153,6 +159,47 @@ export function storySettings(parameters, file) {
153
159
  };
154
160
  }
155
161
 
162
+ /** A project's renames file, in its baselines directory. */
163
+ const RENAMES = "renames.json";
164
+
165
+ const STORY_ID = /^[a-z0-9][a-z0-9-]*$/;
166
+
167
+ /**
168
+ * Reads a project's renames file, `<dir>/renames.json`: `[{ "from": "<old story id>", "to":
169
+ * "<new story id>" }]`, for stories whose id changed while they stayed the same story.
170
+ * @param {string} dir the project's baselines directory
171
+ * @returns {Promise<Map<string, string>>} the old id by new id; empty when there is no file
172
+ */
173
+ async function loadRenames(dir) {
174
+ const path = join(dir, RENAMES);
175
+ let list;
176
+ try {
177
+ list = JSON.parse(await readFile(path, "utf8"));
178
+ } catch (e) {
179
+ if (e.code === "ENOENT") {
180
+ return new Map();
181
+ }
182
+ throw new Error(`${path}: ${e.message}`);
183
+ }
184
+ if (!Array.isArray(list)) {
185
+ throw new Error(`${path}: must be an array of { "from": "<old id>", "to": "<new id>" }`);
186
+ }
187
+ const byTo = new Map();
188
+ const froms = new Set();
189
+ list.forEach((r, i) => {
190
+ const ok = (v) => typeof v === "string" && v.length <= 200 && STORY_ID.test(v);
191
+ if (!ok(r?.from) || !ok(r?.to) || r.from === r.to) {
192
+ throw new Error(`${path}: entry ${i} must be { "from": "<old id>", "to": "<another id>" }`);
193
+ }
194
+ if (byTo.has(r.to) || froms.has(r.from)) {
195
+ throw new Error(`${path}: entry ${i} renames ${r.from} or to ${r.to} a second time`);
196
+ }
197
+ byTo.set(r.to, r.from);
198
+ froms.add(r.from);
199
+ });
200
+ return byTo;
201
+ }
202
+
156
203
  async function pngsIn(dir) {
157
204
  try {
158
205
  return (await readdir(dir)).filter((f) => f.endsWith(".png")).sort();
@@ -485,9 +532,9 @@ export async function capture({
485
532
  const started = Date.now();
486
533
  await mkdir(join(out, "baselines"), { recursive: true });
487
534
  await mkdir(join(out, "second"), { recursive: true });
488
- const ids = storyIds(JSON.parse(await readFile(join(storybook, "index.json"), "utf8"))).filter(
489
- (id) => !stories || stories.some((p) => id.startsWith(p)),
490
- );
535
+ const allIds = storyIds(JSON.parse(await readFile(join(storybook, "index.json"), "utf8")));
536
+ const ids = allIds.filter((id) => !stories || stories.some((p) => id.startsWith(p)));
537
+ const renames = await loadRenames(baselines);
491
538
  const refs = await loadReference(reference);
492
539
  const [server, base] = await serve(storybook);
493
540
  // One browser per worker: every page of a browser shares its one GPU process, so with
@@ -508,6 +555,35 @@ export async function capture({
508
555
  // A story whose own parameters exclude it while it still has a baseline is reported as
509
556
  // removed, so a pull request cannot drop a story from review without the owner seeing it.
510
557
  const newlyExcluded = new Set();
558
+ // Old baselines a rename compares a story with (or reports as a broken rename): not removed.
559
+ const renamed = new Set();
560
+ const renameErrors = [];
561
+ const known = new Set(allIds);
562
+ for (const [to, from] of stories ? [] : renames) {
563
+ if (known.has(to)) {
564
+ continue;
565
+ }
566
+ // Only a rename with an old baseline left to move is an error; once it is accepted the
567
+ // old baselines are gone, and the entry does nothing.
568
+ for (const file of existing) {
569
+ const [id, mode = null] = file.slice(0, -4).split(".");
570
+ if (id === from && BASELINE_NAME.test(file)) {
571
+ renamed.add(file);
572
+ renameErrors.push({
573
+ id: to,
574
+ mode,
575
+ file: fileName(to, mode),
576
+ from,
577
+ threshold: DEFAULT_THRESHOLD,
578
+ includeAA: false,
579
+ ...EMPTY,
580
+ status: "failed",
581
+ reason: `${RENAMES} renames ${from} to ${to}, but the Storybook has no story ${to}: fix ${RENAMES}`,
582
+ });
583
+ }
584
+ }
585
+ }
586
+ items.push(...renameErrors);
511
587
  for (const id of ids) {
512
588
  const s = storySettings(params[id] ?? {}, await loadSettings(baselines, id));
513
589
  for (const mode of s.modes) {
@@ -521,11 +597,22 @@ export async function capture({
521
597
  if (s.disableSnapshot) {
522
598
  items.push({ ...common, ...EMPTY, status: "excluded", reason: s.reason });
523
599
  } else {
524
- jobs.push({ ...common, url: base + storyUrl(id, mode.globals), delay: s.delay });
600
+ // Renamed: compared with the old id's baseline of this mode, while the new id
601
+ // has none of its own (after the accept it does, and the rename is done).
602
+ // A rename from an id that is still a story is not a move: it does nothing.
603
+ const old =
604
+ renames.has(id) && !known.has(renames.get(id)) ? fileName(renames.get(id), mode.name) : null;
605
+ const from = old && !existing.has(file) && existing.has(old) ? renames.get(id) : null;
606
+ if (from) {
607
+ renamed.add(old);
608
+ }
609
+ jobs.push({ ...common, from, url: base + storyUrl(id, mode.globals), delay: s.delay });
525
610
  }
526
611
  }
527
612
  }
528
- const gone = stories ? [] : [...existing].filter((f) => !planned.has(f) && BASELINE_NAME.test(f));
613
+ const gone = stories
614
+ ? []
615
+ : [...existing].filter((f) => !planned.has(f) && !renamed.has(f) && BASELINE_NAME.test(f));
529
616
 
530
617
  const emojiFont = hasEmojiFont();
531
618
  if (emojiFont === false) {
@@ -576,10 +663,12 @@ export async function capture({
576
663
  }
577
664
 
578
665
  const run = async (browser, job) => {
579
- const { url, delay, ...common } = job;
580
- const baseline = await readBaseline(join(baselines, job.file));
666
+ const { url, delay, from, ...rest } = job;
667
+ // `from` only on a renamed story, so results.json of a project with no renames is as before.
668
+ const common = from ? { ...rest, from } : rest;
669
+ const baseline = await readBaseline(join(baselines, from ? fileName(from, job.mode) : job.file));
581
670
  const reference = baseline ? null : (refs.images.get(job.file) ?? null);
582
- const opts = { threshold: job.threshold, includeAA: job.includeAA, reference };
671
+ const opts = { threshold: job.threshold, includeAA: job.includeAA, reference, moved: from !== null };
583
672
  const failed = (shot, prefix = "") => ({
584
673
  ...common,
585
674
  ...EMPTY,
@@ -602,16 +691,22 @@ export async function capture({
602
691
  result = classify({ baseline, first: first.png, second: second.png, ...opts });
603
692
  }
604
693
  const { status } = result;
605
- if (["changed", "new", "unseeded", "unstable"].includes(status)) {
606
- await writeFile(join(out, job.file), first.png);
694
+ // A moved capture is also its baseline when the bytes are the same: the page reads it.
695
+ if (["changed", "moved", "new", "unseeded", "unstable"].includes(status)) {
696
+ // The capture results.json names: the second one when the first was a flake.
697
+ await writeFile(join(out, job.file), result.flaky ? second.png : first.png);
607
698
  }
608
699
  if (status === "unstable") {
609
700
  await writeFile(join(out, "second", job.file), second.png);
610
701
  }
611
- if (baseline && (status === "changed" || status === "unstable")) {
702
+ const ownBaseline = status === "moved" && result.baseline !== result.capture;
703
+ if (baseline && (status === "changed" || status === "unstable" || ownBaseline)) {
612
704
  await writeFile(join(out, "baselines", job.file), baseline);
613
705
  }
614
- const lines = status === "unchanged" ? [] : clip([...first.console, ...(second?.console ?? [])]);
706
+ const lines =
707
+ status === "unchanged" || status === "moved"
708
+ ? []
709
+ : clip([...first.console, ...(second?.console ?? [])]);
615
710
  return { ...common, ...result, reason: null, console: lines };
616
711
  };
617
712
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@graphty/visual-review",
3
- "version": "0.1.0",
3
+ "version": "0.1.2",
4
4
  "description": "Self-hosted visual review of Storybook stories: capture in GitHub Actions, keep the baselines in git (Git LFS), accept or reject each change in a local page, and gate pull requests",
5
5
  "author": "Adam Powers <apowers@ato.ms>",
6
6
  "type": "module",
@@ -26,7 +26,7 @@ import { isLfsPointer } from "./compare.mjs";
26
26
  const sha256 = (bytes) => createHash("sha256").update(bytes).digest("hex");
27
27
 
28
28
  /** The statuses an item can be accepted or rejected in; unstable and failed are only excluded. */
29
- const DECIDABLE = new Set(["changed", "new", "removed"]);
29
+ const DECIDABLE = new Set(["changed", "moved", "new", "removed"]);
30
30
  const EXCLUDABLE = new Set([...DECIDABLE, "unstable", "failed"]);
31
31
 
32
32
  /**
@@ -354,7 +354,7 @@ async function commitAccepts({ repo, target, accepts, first, now, progress, conf
354
354
  const counts = { accept: 0, exclude: 0, remove: 0 };
355
355
  progress(`writing ${writes.length} ${writes.length === 1 ? "file" : "files"}`);
356
356
  for (const w of writes) {
357
- items.push(await write(tree, w, counts, baselines));
357
+ items.push(...(await write(tree, w, counts, baselines)));
358
358
  }
359
359
  const stamp = now.toISOString().replace(/[-:]/g, "").replace(/\.\d+/, "");
360
360
  const record = `${baselines}/reviews/${stamp}-${isMaster ? "master" : `pr${target.pr}`}.json`;
@@ -441,8 +441,9 @@ export async function behindMaster(repo, head, project, { defaultBranch, baselin
441
441
  * @param {object} w the decision, its results item, and for an accept the verified bytes
442
442
  * @param {{ accept: number, exclude: number, remove: number }} counts tallied here
443
443
  * @param {string} baselines the baselines directory
444
- * @returns {Promise<{ path: string, from: string | null, to: string | null, reason: string | null }>}
445
- * the record item
444
+ * @returns {Promise<{ path: string, from: string | null, to: string | null, reason: string | null,
445
+ * movedFrom?: string, movedTo?: string }[]>} its record items: two for a renamed story, whose
446
+ * baseline moves to its new name
446
447
  */
447
448
  async function write(tree, w, counts, baselines) {
448
449
  const dir = `${baselines}/${w.project}`;
@@ -453,17 +454,27 @@ async function write(tree, w, counts, baselines) {
453
454
  const bytes = `${JSON.stringify(settings, null, 2)}\n`;
454
455
  await put(join(tree, path), bytes);
455
456
  counts.exclude++;
456
- return { path, from: old && sha256(old), to: sha256(bytes), reason: `exclude: ${w.reason}` };
457
+ return [{ path, from: old && sha256(old), to: sha256(bytes), reason: `exclude: ${w.reason}` }];
457
458
  }
458
459
  const path = `${dir}/${w.item.file}`;
459
460
  if (w.item.status === "removed") {
460
461
  await rm(join(tree, path), { force: true });
461
462
  counts.remove++;
462
- return { path, from: w.item.baseline, to: null, reason: w.reason };
463
+ return [{ path, from: w.item.baseline, to: null, reason: w.reason }];
463
464
  }
464
465
  await put(join(tree, path), w.bytes);
465
466
  counts.accept++;
466
- return { path, from: w.item.baseline, to: w.item.capture, reason: w.reason };
467
+ if (!w.item.from) {
468
+ return [{ path, from: w.item.baseline, to: w.item.capture, reason: w.reason }];
469
+ }
470
+ // A rename: the old id's baseline (of this mode) goes, the new one takes its place. For a
471
+ // moved item the bytes are the same, so git sees a rename and the LFS pointer is unchanged.
472
+ const oldPath = `${dir}/${w.item.mode === null ? w.item.from : `${w.item.from}.${w.item.mode}`}.png`;
473
+ await rm(join(tree, oldPath), { force: true });
474
+ return [
475
+ { path: oldPath, from: w.item.baseline, to: null, reason: w.reason, movedTo: path },
476
+ { path, from: null, to: w.item.capture, reason: w.reason, movedFrom: oldPath },
477
+ ];
467
478
  }
468
479
 
469
480
  // Two modes of one story excluded together write one settings file: keep one record item.
@@ -154,16 +154,21 @@ function crop(img, w, h) {
154
154
  * A story with no baseline is `unseeded` ("no baseline yet") when its capture matches
155
155
  * `reference`, master's newest capture of it: the pull request did not change it, so it needs no
156
156
  * review here. It is `new` when it differs from master's, or master has none (a new story).
157
+ *
158
+ * `moved` says the baseline is another story id's, named in the project's renames.json: a story
159
+ * that only moved (same image, new id) is `moved` rather than `unchanged`, so it still needs an
160
+ * accept, which writes its baseline under the new name.
157
161
  * @param {{ baseline: Buffer | null, first: Buffer | null, second?: Buffer | null,
158
- * reference?: Buffer | null, threshold: number, includeAA: boolean }} input `first` is null
159
- * when the story is gone
160
- * @returns {{ status: "unchanged" | "changed" | "new" | "unseeded" | "removed" | "unstable", flaky: boolean,
162
+ * reference?: Buffer | null, threshold: number, includeAA: boolean, moved?: boolean }} input
163
+ * `first` is null when the story is gone
164
+ * @returns {{ status: "unchanged" | "moved" | "changed" | "new" | "unseeded" | "removed" | "unstable", flaky: boolean,
161
165
  * baseline: string | null, capture: string | null, size: number[] | null,
162
166
  * baselineSize: number[] | null, changedPixels: number | null, bbox: number[] | null }}
163
167
  * `capture` is the hash of the capture the status describes: the second one when flaky
164
168
  */
165
- export function classify({ baseline, first, second = null, reference = null, threshold, includeAA }) {
169
+ export function classify({ baseline, first, second = null, reference = null, threshold, includeAA, moved = false }) {
166
170
  const options = { threshold, includeAA };
171
+ const matched = (r, flaky) => ({ ...r, status: moved ? "moved" : "unchanged", flaky });
167
172
  const none = { flaky: false, size: null, baselineSize: null, changedPixels: null, bbox: null };
168
173
  if (first === null) {
169
174
  return { status: "removed", ...none, baseline: sha256(baseline), capture: null };
@@ -175,12 +180,15 @@ export function classify({ baseline, first, second = null, reference = null, thr
175
180
  return { status, ...none, baseline: null, capture: sha256(first), size: pngSize(first) };
176
181
  }
177
182
  const vsFirst = compareImages(baseline, first, options);
178
- if (vsFirst.status === "unchanged" || second === null) {
183
+ if (vsFirst.status === "unchanged") {
184
+ return matched(vsFirst, false);
185
+ }
186
+ if (second === null) {
179
187
  return { ...vsFirst, flaky: false };
180
188
  }
181
189
  const vsSecond = compareImages(baseline, second, options);
182
190
  if (vsSecond.status === "unchanged") {
183
- return { ...vsSecond, flaky: true };
191
+ return matched(vsSecond, true);
184
192
  }
185
193
  return { ...vsFirst, status: agree() ? "changed" : "unstable", flaky: false };
186
194
  }
@@ -10,8 +10,8 @@
10
10
  */
11
11
 
12
12
  import { execFile } from "node:child_process";
13
- import { existsSync, readFileSync } from "node:fs";
14
- import { join } from "node:path";
13
+ import { existsSync, mkdirSync, mkdtempSync, readFileSync, renameSync, rmSync } from "node:fs";
14
+ import { dirname, join } from "node:path";
15
15
 
16
16
  import { validateResults } from "./results.mjs";
17
17
 
@@ -114,9 +114,48 @@ export async function visualJobs(gh, run, attempt, projects) {
114
114
  );
115
115
  }
116
116
 
117
+ // Downloads in flight, by target directory: concurrent refreshes of one run await the same one.
118
+ const downloading = new Map();
119
+
120
+ /**
121
+ * Downloads one artifact into `dir`, unless it is already there. It is extracted into a sibling
122
+ * temporary directory and renamed into place only once its results.json is there, so `dir` either
123
+ * does not exist or holds a whole artifact; a failed or interrupted download leaves nothing behind.
124
+ * An artifact without results.json is discarded, and its readers report the capture as failed.
125
+ * @param {Function} gh the gh runner
126
+ * @param {number} runId the run
127
+ * @param {string} name the artifact
128
+ * @param {string} dir where it goes
129
+ * @returns {Promise<void>} settles when `dir` is complete, or the artifact had no results.json
130
+ */
131
+ function download(gh, runId, name, dir) {
132
+ if (existsSync(join(dir, "results.json"))) {
133
+ return Promise.resolve();
134
+ }
135
+ if (!downloading.has(dir)) {
136
+ const done = (async () => {
137
+ // A directory without results.json is left over from before downloads were atomic.
138
+ rmSync(dir, { recursive: true, force: true });
139
+ mkdirSync(dirname(dir), { recursive: true });
140
+ const part = mkdtempSync(`${dir}.part-`);
141
+ try {
142
+ await gh(["run", "download", String(runId), "-n", name, "-D", part]);
143
+ if (existsSync(join(part, "results.json"))) {
144
+ renameSync(part, dir);
145
+ }
146
+ } finally {
147
+ rmSync(part, { recursive: true, force: true });
148
+ }
149
+ })().finally(() => downloading.delete(dir));
150
+ downloading.set(dir, done);
151
+ }
152
+ return downloading.get(dir);
153
+ }
154
+
117
155
  /**
118
156
  * Downloads each project's capture artifact from the highest attempt that uploaded one, into
119
- * `<tmp>/<run>-<attempt>/<project>/`. An artifact already downloaded is not fetched again.
157
+ * `<tmp>/<run>-<attempt>/<project>/`. An artifact already downloaded is not fetched again, and
158
+ * concurrent calls for the same one share a single download.
120
159
  * @param {Function} gh the gh runner
121
160
  * @param {{ id: number }} run the run
122
161
  * @param {string[]} projects project ids
@@ -139,9 +178,7 @@ export async function downloadCaptures(gh, run, projects, tmp) {
139
178
  continue;
140
179
  }
141
180
  const dir = join(tmp, `${run.id}-${newest.attempt}`, project);
142
- if (!existsSync(join(dir, "results.json"))) {
143
- await gh(["run", "download", String(run.id), "-n", newest.name, "-D", dir]);
144
- }
181
+ await download(gh, run.id, newest.name, dir);
145
182
  out[project] = { dir, attempt: newest.attempt };
146
183
  }
147
184
  return out;
@@ -159,12 +196,15 @@ export async function downloadCaptures(gh, run, projects, tmp) {
159
196
  * @returns {Promise<string | null>} the capture's directory, or null when no run has one
160
197
  */
161
198
  export async function newestMasterCapture(gh, project, tmp, { workflow, defaultBranch }) {
162
- const { workflow_runs: runs } = await api(
163
- gh,
164
- `repos/{owner}/{repo}/actions/workflows/${encodeURIComponent(workflow)}/runs?branch=` +
165
- `${encodeURIComponent(defaultBranch)}&event=push&per_page=10`,
166
- );
167
- for (const run of runs.map(toRun)) {
199
+ // The branch's newest commits, then each one's run: GitHub's list of a workflow's runs filtered
200
+ // by branch now and then answers with a stale page (runs from weeks ago), which made a pull
201
+ // request compare with an old capture or none. Commits and a run by head sha answer consistently.
202
+ const commits = await api(gh, `repos/{owner}/{repo}/commits?sha=${encodeURIComponent(defaultBranch)}&per_page=10`);
203
+ for (const { sha } of commits) {
204
+ const run = await newestCiRun(gh, sha, { workflow });
205
+ if (!run) {
206
+ continue;
207
+ }
168
208
  const dir = (await downloadCaptures(gh, run, [project], tmp))[project]?.dir;
169
209
  let results = null;
170
210
  try {
@@ -8,9 +8,11 @@
8
8
 
9
9
  /**
10
10
  * Every status an item can have. `unseeded` ("no baseline yet") is a story with no baseline whose
11
- * capture matches master's newest capture of it, so the pull request did not change it.
11
+ * capture matches master's newest capture of it, so the pull request did not change it. `moved`
12
+ * is a story that looks exactly as the baseline of the id it was renamed from (`from`, named in
13
+ * the project's renames.json); a renamed story that looks different is `changed` with `from`.
12
14
  */
13
- const STATUSES = ["unchanged", "changed", "new", "unseeded", "removed", "unstable", "failed", "excluded"];
15
+ const STATUSES = ["unchanged", "moved", "changed", "new", "unseeded", "removed", "unstable", "failed", "excluded"];
14
16
 
15
17
  /** The most items one file may hold (compact-mantine has about 830 today). */
16
18
  export const MAX_ITEMS = 5000;
@@ -26,6 +28,7 @@ const NAME = /^[a-z0-9][a-z0-9-]*$/;
26
28
  const HASHES = {
27
29
  unchanged: { baseline: true, capture: true },
28
30
  changed: { baseline: true, capture: true },
31
+ moved: { baseline: true, capture: true },
29
32
  new: { baseline: false, capture: true },
30
33
  unseeded: { baseline: false, capture: true },
31
34
  removed: { baseline: true, capture: false },
@@ -113,6 +116,14 @@ export function validateResults(r) {
113
116
  }
114
117
 
115
118
  check(STATUSES.includes(item.status), `${at}.status must be one of ${STATUSES.join(", ")}`);
119
+ // The story id a renamed story's baseline was read under (absent when it was not renamed).
120
+ check(
121
+ item.from === undefined ||
122
+ item.from === null ||
123
+ (typeof item.from === "string" && item.from.length <= 200 && NAME.test(item.from)),
124
+ `${at}.from must be a Storybook story id or null`,
125
+ );
126
+ check(item.status !== "moved" || typeof item.from === "string", `${at}.from is required for a moved item`);
116
127
  check(typeof item.flaky === "boolean", `${at}.flaky must be a boolean`);
117
128
  for (const key of ["baseline", "capture"]) {
118
129
  const v = item[key];
@@ -44,7 +44,7 @@ const HEADERS = {
44
44
  */
45
45
  const componentOf = (id) => id.split("--")[0];
46
46
 
47
- const REVIEWABLE = new Set(["changed", "new", "removed", "unstable", "failed"]);
47
+ const REVIEWABLE = new Set(["changed", "moved", "new", "removed", "unstable", "failed"]);
48
48
  const WRITES = new Set(["decide", "accept-all", "finish"]);
49
49
 
50
50
  /**
@@ -434,9 +434,9 @@ export function createApp({ repo, gh, config, tmp, token, origin, masterRun, res
434
434
  if (!hash) {
435
435
  return [404, { error: "no such image" }];
436
436
  }
437
- const bytes = await readFile(kind === "capture" ? join(p.dir, file) : join(p.dir, "baselines", file)).catch(
438
- () => null,
439
- );
437
+ // A moved item's baseline is its capture's bytes, so the artifact holds only the capture.
438
+ const own = kind === "capture" || item.baseline === item.capture;
439
+ const bytes = await readFile(own ? join(p.dir, file) : join(p.dir, "baselines", file)).catch(() => null);
440
440
  if (!bytes || createHash("sha256").update(bytes).digest("hex") !== hash) {
441
441
  return [409, { error: `${file} does not match results.json` }];
442
442
  }
@@ -187,6 +187,7 @@ td {
187
187
  }
188
188
 
189
189
  .badge.changed,
190
+ .badge.moved,
190
191
  .badge.exclude {
191
192
  border-color: var(--accent);
192
193
  color: var(--accent);
@@ -444,6 +445,12 @@ td {
444
445
  image-rendering: pixelated;
445
446
  }
446
447
 
448
+ /* Highlight's changed pixels, over the image: no checkerboard behind them. */
449
+ .sheet > canvas.diffmark {
450
+ background: none;
451
+ pointer-events: none;
452
+ }
453
+
447
454
  .boxmark {
448
455
  position: absolute;
449
456
  box-sizing: border-box;
@@ -494,3 +501,9 @@ td {
494
501
  background: var(--card);
495
502
  color: var(--fg);
496
503
  }
504
+
505
+ .tile .moved-from {
506
+ font-size: 12px;
507
+ word-break: break-all;
508
+ color: var(--accent);
509
+ }
@@ -11,14 +11,14 @@ const app = document.getElementById("app");
11
11
  const crumbs = document.getElementById("crumbs");
12
12
  const statusLine = document.getElementById("status");
13
13
 
14
- const REVIEWABLE = ["changed", "new", "removed", "unstable", "failed"];
15
- const ACCEPTABLE = ["changed", "new", "removed"];
14
+ const REVIEWABLE = ["changed", "moved", "new", "removed", "unstable", "failed"];
15
+ const ACCEPTABLE = ["changed", "moved", "new", "removed"];
16
16
  // A story with no baseline that this pull request did not change: shown, never a decision here.
17
17
  const UNSEEDED = "unseeded";
18
18
  const NO_BASELINE = "no baseline yet";
19
19
  const statusLabel = (status) => (status === UNSEEDED ? NO_BASELINE : status);
20
- // Errors first (their own list), then what changed, then new, unstable and removed stories.
21
- const RANK = { failed: 0, changed: 1, new: 2, unstable: 3, removed: 4, unseeded: 5 };
20
+ // Errors first (their own list), then what changed, then moved, new, unstable and removed stories.
21
+ const RANK = { failed: 0, changed: 1, moved: 2, new: 3, unstable: 4, removed: 5, unseeded: 6 };
22
22
  const FLASH_MS = 333; // one image each third of a second: about 1.5 full cycles a second
23
23
  // "fit" (the default) shows the whole of both images in their panes, at one scale, never above real
24
24
  // size; 1 is real size: one CSS pixel per CSS pixel the story was drawn at, scrolling when larger.
@@ -29,6 +29,9 @@ const SPOT_ALPHA = 190; // the spotlight's dimming, out of 255, as Chromatic's f
29
29
  const RE_REVIEW = "re-review: your earlier accept was replaced by master's baseline";
30
30
  // The grid's decision filters, and how a decision reads on a tile.
31
31
  const DECISIONS = { accept: "Accepted", reject: "Rejected", exclude: "Excluded" };
32
+ // The reviewer's own display choices (the changed box, blinking the overlay), kept in this browser.
33
+ const OPTIONS_KEY = "visual-review:options";
34
+ const saved = loadOptions();
32
35
 
33
36
  const state = {
34
37
  targets: [],
@@ -45,6 +48,8 @@ const state = {
45
48
  view: "side", // side | flash | highlight | spotlight
46
49
  zoom: "fit",
47
50
  box: 0, // which changed box "next changed box" is on
51
+ showBox: saved.showBox ?? true, // outline the changed box (B)
52
+ blink: saved.blink ?? false, // blink the changed pixels Highlight lays over the images (L)
48
53
  held: null, // the view to return to when Space is released
49
54
  pending: "reject", // what Enter in the reason box does
50
55
  screen: "targets",
@@ -139,6 +144,8 @@ function loaded(url) {
139
144
 
140
145
  const componentOf = (id) => id.split("--")[0];
141
146
  const itemName = (item) => (item.mode ? `${item.id} (${item.mode})` : item.id);
147
+ // A renamed story (renames.json) is compared with its old id's baseline: say which.
148
+ const movedFrom = (item) => (item.from ? `moved from ${item.from}` : "");
142
149
  const short = (sha) => (sha ? sha.slice(0, 10) : "none");
143
150
  const decisionOf = (item) => state.data?.decisions[item.file] ?? null;
144
151
  const isLocal = () => state.target?.local === true;
@@ -213,6 +220,30 @@ function stopFlash() {
213
220
  flashTimer = null;
214
221
  }
215
222
 
223
+ // Local storage can be missing or refuse (a private window, blocked site data): the page then uses
224
+ // the defaults and forgets the choices, and nothing else changes.
225
+ function loadOptions() {
226
+ try {
227
+ return JSON.parse(localStorage.getItem(OPTIONS_KEY) ?? "{}") ?? {};
228
+ } catch {
229
+ return {};
230
+ }
231
+ }
232
+
233
+ function saveOptions() {
234
+ try {
235
+ localStorage.setItem(OPTIONS_KEY, JSON.stringify({ showBox: state.showBox, blink: state.blink }));
236
+ } catch {
237
+ // Not remembered; the choice still holds for this page.
238
+ }
239
+ }
240
+
241
+ function toggleOption(key) {
242
+ state[key] = !state[key];
243
+ saveOptions();
244
+ showStory();
245
+ }
246
+
216
247
  // ---------------------------------------------------------------- screen: targets
217
248
 
218
249
  async function loadTargets() {
@@ -459,6 +490,7 @@ function tile(item, number) {
459
490
  img,
460
491
  el("span", { class: "name" }, el("span", { class: "number" }, `${number}`), " ", item.mode ?? ""),
461
492
  el("span", { class: `badge ${item.status}` }, statusLabel(item.status)),
493
+ item.from ? el("span", { class: "moved-from", title: movedFrom(item) }, movedFrom(item)) : null,
462
494
  item.reReview ? el("span", { class: "badge warn", title: RE_REVIEW }, "re-review") : null,
463
495
  ),
464
496
  decisionLine(item),
@@ -697,7 +729,7 @@ function showGrid() {
697
729
  `Needs a decision (${state.data.items.filter((i) => REVIEWABLE.includes(i.status) && !decisionOf(i)).length})`,
698
730
  ),
699
731
  filterButton("all", `All (${state.data.items.filter((i) => REVIEWABLE.includes(i.status)).length})`),
700
- ["changed", "new", "unstable", "removed", "failed"]
732
+ ["changed", "moved", "new", "unstable", "removed", "failed"]
701
733
  .filter((s) => counts[s])
702
734
  .map((s) => filterButton(s, `${s} (${counts[s]})`)),
703
735
  counts[UNSEEDED] ? filterButton(UNSEEDED, `${NO_BASELINE} (${counts[UNSEEDED]})`) : null,
@@ -941,6 +973,7 @@ function showStory() {
941
973
  itemName(item),
942
974
  " ",
943
975
  el("span", { class: `badge ${item.status}` }, statusLabel(item.status)),
976
+ item.from ? el("span", { class: "badge moved" }, movedFrom(item)) : null,
944
977
  d
945
978
  ? el("span", { class: `badge ${d.decision}` }, `${d.decision}${d.reason ? `: ${d.reason}` : ""}`)
946
979
  : null,
@@ -968,6 +1001,30 @@ function showStory() {
968
1001
  viewButton("flash", "Flash", "F, or hold Space"),
969
1002
  viewButton("highlight", "Highlight", "H"),
970
1003
  viewButton("spotlight", "Spotlight", "S"),
1004
+ el(
1005
+ "button",
1006
+ {
1007
+ type: "button",
1008
+ "aria-pressed": String(state.showBox),
1009
+ title: "B: outline the changed box",
1010
+ onclick: () => toggleOption("showBox"),
1011
+ },
1012
+ "Box",
1013
+ ),
1014
+ el(
1015
+ "button",
1016
+ {
1017
+ type: "button",
1018
+ "aria-pressed": String(state.blink),
1019
+ disabled: view !== "highlight",
1020
+ title:
1021
+ view === "highlight"
1022
+ ? "L: blink the changed pixels"
1023
+ : "Blink flashes the changed pixels that Highlight lays over the images",
1024
+ onclick: () => toggleOption("blink"),
1025
+ },
1026
+ "Blink",
1027
+ ),
971
1028
  note ? el("span", { id: "single-note", class: "meta" }, note) : null,
972
1029
  el("span", { class: "spacer" }),
973
1030
  ZOOMS.map(zoomButton),
@@ -1049,7 +1106,7 @@ async function diffOf(item) {
1049
1106
  diffMask: true,
1050
1107
  });
1051
1108
  const grown = grow(mask, w, h);
1052
- return { w, h, a: pa, b: pb, grown, boxes: regions(grown, w, h) };
1109
+ return { w, h, a: pa, b: pb, mask, grown, boxes: regions(grown, w, h) };
1053
1110
  })(),
1054
1111
  );
1055
1112
  }
@@ -1174,7 +1231,15 @@ async function renderStage(item, view, keep) {
1174
1231
  };
1175
1232
  try {
1176
1233
  const diff = item.baseline && item.capture ? await diffOf(item) : null;
1177
- const left = item.baseline ? pane("Baseline", await imgOf("baseline")) : pane("No baseline");
1234
+ const marked = view === "highlight" && diff !== null;
1235
+ const baseName = item.from ? `Baseline of ${item.from}` : "Baseline";
1236
+ const left = item.baseline
1237
+ ? pane(
1238
+ marked ? `${baseName}, changed pixels in red` : baseName,
1239
+ await imgOf("baseline"),
1240
+ ...(marked ? [overlay(diff)] : []),
1241
+ )
1242
+ : pane("No baseline");
1178
1243
  let right;
1179
1244
  if (!item.capture) {
1180
1245
  right = pane(item.status === "failed" ? "No capture: it failed" : "No capture");
@@ -1195,11 +1260,22 @@ async function renderStage(item, view, keep) {
1195
1260
  tag.textContent = showingNew ? "Flash: new" : "Flash: baseline";
1196
1261
  }, FLASH_MS);
1197
1262
  } else if (view === "highlight") {
1198
- right = pane("Changed pixels in red over the dimmed baseline", highlight(item, diff));
1263
+ right = pane("New, changed pixels in red", await imgOf("capture"), overlay(diff));
1199
1264
  } else {
1200
1265
  right = pane("Spotlight: the new image, dimmed except around each change", spotlight(diff));
1201
1266
  }
1202
1267
  stage.replaceChildren(left, right);
1268
+ if (marked && state.blink) {
1269
+ // Both panes' overlays on and off together, at Flash's pace.
1270
+ const marks = [...stage.querySelectorAll(".diffmark")];
1271
+ let on = true;
1272
+ flashTimer = setInterval(() => {
1273
+ on = !on;
1274
+ for (const m of marks) {
1275
+ m.style.visibility = on ? "visible" : "hidden";
1276
+ }
1277
+ }, FLASH_MS);
1278
+ }
1203
1279
  const frames = [...stage.querySelectorAll(".frame")];
1204
1280
  // Zoomed, scrolling one pane scrolls the other to the same place.
1205
1281
  for (const f of frames) {
@@ -1256,14 +1332,16 @@ function showBox(stage, boxes, jump) {
1256
1332
  const [left, top] = [Math.max(0, x * factor - 2), Math.max(0, y * factor - 2)];
1257
1333
  const [right, bottom] = [Math.min(sw, (x + w) * factor + 2), Math.min(sh, (y + h) * factor + 2)];
1258
1334
  sheet.querySelector(".boxmark")?.remove();
1259
- const mark = el("div", { class: "boxmark" });
1260
- Object.assign(mark.style, {
1261
- left: `${left}px`,
1262
- top: `${top}px`,
1263
- width: `${right - left}px`,
1264
- height: `${bottom - top}px`,
1265
- });
1266
- sheet.append(mark);
1335
+ if (state.showBox) {
1336
+ const mark = el("div", { class: "boxmark" });
1337
+ Object.assign(mark.style, {
1338
+ left: `${left}px`,
1339
+ top: `${top}px`,
1340
+ width: `${right - left}px`,
1341
+ height: `${bottom - top}px`,
1342
+ });
1343
+ sheet.append(mark);
1344
+ }
1267
1345
  if (!jump) {
1268
1346
  frame.scrollLeft = 0;
1269
1347
  frame.scrollTop = 0;
@@ -1298,16 +1376,18 @@ async function nextBox() {
1298
1376
  showBox(document.getElementById("stage"), boxes, true);
1299
1377
  }
1300
1378
 
1301
- // pixelmatch's own picture: the changed pixels in red over the dimmed baseline.
1302
- function highlight(item, diff) {
1303
- const canvas = el("canvas", { width: String(diff.w), height: String(diff.h) });
1379
+ // The changed pixels alone, in solid red, transparent everywhere else: laid over an image in the
1380
+ // same grid cell, so it lines up with the image at every zoom.
1381
+ function overlay(diff) {
1382
+ const canvas = el("canvas", { width: String(diff.w), height: String(diff.h), class: "diffmark" });
1304
1383
  const ctx = canvas.getContext("2d");
1305
1384
  const out = ctx.createImageData(diff.w, diff.h);
1306
- pixelmatch(diff.a, diff.b, out.data, diff.w, diff.h, {
1307
- threshold: item.threshold,
1308
- includeAA: item.includeAA,
1309
- alpha: 0.2,
1310
- });
1385
+ for (let i = 0; i < diff.w * diff.h; i++) {
1386
+ if (diff.mask[i * 4 + 3] !== 0) {
1387
+ out.data[i * 4] = 255;
1388
+ out.data[i * 4 + 3] = 255;
1389
+ }
1390
+ }
1311
1391
  ctx.putImageData(out, 0, 0);
1312
1392
  return canvas;
1313
1393
  }
@@ -1562,7 +1642,7 @@ function finishOutcome() {
1562
1642
  // ---------------------------------------------------------------- the address
1563
1643
 
1564
1644
  // Every screen is in the address, after the session token, so a copied link opens it again:
1565
- // #token=...&target=123&project=p&filter=undecided&q=text&item=file.png&view=side&zoom=fit
1645
+ // #token=...&target=123&project=p&filter=undecided&q=text&item=file.png&view=side&zoom=fit&box=on&blink=off
1566
1646
  // Only the fragment holds it: a browser never sends a fragment to a server or in a Referer.
1567
1647
  function hashFor() {
1568
1648
  const p = new URLSearchParams({ token });
@@ -1578,6 +1658,8 @@ function hashFor() {
1578
1658
  p.set("item", current().file);
1579
1659
  p.set("view", state.held ?? state.view);
1580
1660
  p.set("zoom", String(state.zoom));
1661
+ p.set("box", state.showBox ? "on" : "off");
1662
+ p.set("blink", state.blink ? "on" : "off");
1581
1663
  }
1582
1664
  return `#${p}`;
1583
1665
  }
@@ -1654,6 +1736,13 @@ async function route() {
1654
1736
  state.view = VIEWS.includes(p.get("view")) ? p.get("view") : "side";
1655
1737
  const zoom = p.get("zoom") === "fit" ? "fit" : Number(p.get("zoom"));
1656
1738
  state.zoom = ZOOMS.includes(zoom) ? zoom : "fit";
1739
+ // A link's box and blink apply to this page; the browser's remembered choice is unchanged.
1740
+ if (["on", "off"].includes(p.get("box"))) {
1741
+ state.showBox = p.get("box") === "on";
1742
+ }
1743
+ if (["on", "off"].includes(p.get("blink"))) {
1744
+ state.blink = p.get("blink") === "on";
1745
+ }
1657
1746
  state.box = 0;
1658
1747
  say("");
1659
1748
  showStory();
@@ -1721,6 +1810,12 @@ document.addEventListener("keydown", (e) => {
1721
1810
  f: () => toggleView("flash"),
1722
1811
  h: () => toggleView("highlight"),
1723
1812
  s: () => toggleView("spotlight"),
1813
+ b: () => toggleOption("showBox"),
1814
+ l: () => {
1815
+ if (state.view === "highlight") {
1816
+ toggleOption("blink");
1817
+ }
1818
+ },
1724
1819
  n: nextBox,
1725
1820
  z: () => {
1726
1821
  state.zoom = ZOOMS[(ZOOMS.indexOf(state.zoom) + 1) % ZOOMS.length];