capybara-storyboard 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0f2357cb5edac81b907bc4196d8469d85c5de0636932fa2dbe25d2985edaa6a9
-  data.tar.gz: 2c4dffec621810ef339178aafacefffaefec409c3f6b72ef7c4944d1b770e7c3
+  metadata.gz: 3c4545f0de7b20b1fbfcf71a08ed644553d5445001370ff3d59a4d8c9fc9c88f
+  data.tar.gz: a3aa8204023cab2f89f1799f0fbf7aee7fe6965bbbe00d1944e75c27010dc158
 SHA512:
-  metadata.gz: 57a736051c0a33c64227ae7f576d1c5b4739e8c3e1a9c5e969be1534abf1b1072463845a92bbff69b1396df8dbc0a5fa3fd4211c9412ed69133ca2d81a3cfe95
-  data.tar.gz: c366ae904aada06f091ded0cb7697754506f40a5ceac87f6383a1a83cf0bda2f619e3a4901afc9c53690a399f5ace1a1064fcb6e7bc0b3e006b725e78e770f43
+  metadata.gz: 4aed98cd98f7137740c85d5b9d994adbec0ecd2f9c06c972c7caacc5b4087e7450710855e267d507651ceb1f5d4f4667bfb1bdf8d0a9d0f037c5d39f27bcadb4
+  data.tar.gz: f801d661ab48e2d980fc39aaa7f8d9614a79a6c6b3795c7335f33263658a872d97894c8e7ad76fabf72bd51767d2dfbbc17ad45c056bd86661c4c425e8d1d8b6

data/CHANGELOG.md

@@ -1,5 +1,9 @@
 ## [Unreleased]
 
+## [0.2.0] - 2026-07-22
+
+- Add `visual-regression-test` skill for AI-assisted before/after screenshot comparison
+
 ## [0.1.0] - 2026-07-21
 
 - Initial release

data/README.md

@@ -108,7 +108,9 @@ which test files are captured.
 | Selective capture | `1` | set via `SCREENSHOT_TESTS_FILE` / `SCREENSHOT_TESTS` | Only the listed test files are captured. If the resulting set is empty, zero screenshots are taken (an empty selection does NOT fall back to capturing everything). |
 
 In short: `SCREENSHOTS` arms the mechanism as a whole, and the target list — when present —
-filters down which tests are captured within that armed mechanism.
+filters down which tests are captured within that armed mechanism. The target list affects
+capture only, never test execution: RSpec still runs whatever you tell it to run (see
+[Selecting which tests to capture](#selecting-which-tests-to-capture)).
 
 ## Selecting which tests to capture
 
@@ -126,14 +128,25 @@ If `SCREENSHOT_TESTS_FILE` points to a file that does not exist, `Capybara::Stor
 is raised. (This existence check only runs when `SCREENSHOTS` is enabled; when the mechanism
 is disabled, the target list is never read.)
 
+> [!IMPORTANT]
+> The target list narrows **which tests are captured**, not **which tests are run**. RSpec
+> still executes every test you pass to it; the listed files are merely the ones that get
+> screenshots. To also avoid running the other tests — the usual goal in CI — pass the target
+> files to `rspec` as arguments instead of (or in addition to) using the target list. See the
+> examples below.
+
 Examples:
 
 ```bash
-# Inline list of test files
+# Inline list of test files (all system specs still RUN; only these two are CAPTURED)
 SCREENSHOTS=1 SCREENSHOT_TESTS=spec/system/login_spec.rb,spec/system/signup_spec.rb bundle exec rspec
 
 # List generated into a file (e.g. by CI, listing only changed specs)
 SCREENSHOTS=1 SCREENSHOT_TESTS_FILE=tmp/screenshot_targets.txt bundle exec rspec
+
+# Narrow both execution AND capture: pass the files to rspec as arguments.
+# With SCREENSHOTS=1, every test that actually runs is captured, so no target list is needed.
+SCREENSHOTS=1 bundle exec rspec $(cat tmp/screenshot_targets.txt)
 ```
 
 ## Output layout
@@ -232,16 +245,23 @@ is still taken. The wait is tunable via:
 - Test files are expected to be laid out flat as `spec/system/*_spec.rb`. Nested
   subdirectories may work but are not comprehensively verified in this initial version.
 - The target-list selection matches on test file paths. A PR that changes only views
-  (leaving the system spec file itself unchanged) cannot be picked up by the target-list
-  approach — this is a known, deliberate limitation of the initial version.
+  (leaving the system spec file itself unchanged) cannot be picked up by the basic
+  path-diff recipe in [docs/github-actions.md](docs/github-actions.md) — a known, deliberate
+  limitation of that recipe. The [docs/visual-regression.md](docs/visual-regression.md)
+  workflow lifts it via optional Claude-driven target expansion.
 
 ## Guides
 
 - [docs/github-actions.md](docs/github-actions.md) — a recipe for capturing screenshots on a
   diff basis in GitHub Actions.
+- [docs/visual-regression.md](docs/visual-regression.md) — a reusable GitHub Actions workflow
+  for AI-assisted visual regression testing.
 - [skills/capybara-storyboard](skills/capybara-storyboard) — an agent skill for visually
   verifying captured screenshots (Claude Code, or any agent that can read PNG files from the
   filesystem).
+- [skills/visual-regression-test](skills/visual-regression-test) — an agent skill that
+  compares before/after screenshots with `reg-cli` to catch unintended visual regressions,
+  invoked via `/visual-regression-test [ref]` in Claude Code.
 
 ## Development
 

data/docs/visual-regression.md

@@ -0,0 +1,106 @@
+# Reusable GitHub Actions workflow: AI-assisted visual regression testing
+
+This is a reusable workflow, not a copy-paste recipe: capture screenshots on the PR base and
+head branches in parallel, pixel-diff them with [reg-cli](https://github.com/reg-viz/reg-cli),
+have Claude classify each changed image as intended, a likely regression, or uncertain, and
+report the result as a PR comment and job summary. It never blocks the PR — a human makes the
+final call.
+
+This complements the [`docs/github-actions.md`](github-actions.md) recipe rather than
+replacing it. That recipe only captures screenshots and uploads them for manual review; it
+doesn't detect what changed, and it can't pick up a PR that only touches views (not the spec
+files themselves). This workflow adds automatic change detection on top, and — via optional
+Claude-driven target expansion — can also capture specs affected by a view-only PR that never
+touches a spec file.
+
+## Usage
+
+Call it from a workflow in your application repository. Pin the `uses:` ref to one that
+actually exists — the example below uses `@main`; a release tag such as `@v0.1.0`, or a
+`@v1` major tag once one is published, work too (a ref that doesn't resolve fails with
+"workflow was not found"):
+
+```yaml
+name: Visual regression
+
+on:
+  pull_request:
+
+jobs:
+  visual-regression:
+    # A called workflow's token permissions are the intersection of what it
+    # requests and what the caller grants, so grant pull-requests: write here —
+    # otherwise the PR-comment step is denied (403) on repositories whose default
+    # workflow token is read-only.
+    permissions:
+      contents: read
+      pull-requests: write
+    uses: aki77/capybara-storyboard/.github/workflows/visual-regression.yml@main
+    with:
+      ruby_version: "3.4"
+    secrets:
+      anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+```
+
+### Inputs
+
+| Input | Default | Description |
+|---|---|---|
+| `base_ref` | `${{ github.event.pull_request.base.ref }}` | Base ref to compare against (the expected side). |
+| `ruby_version` | `"3.4"` | Ruby version passed to `ruby/setup-ruby`. |
+| `node_version` | `"20"` | Node.js version for reg-cli (requires Node 20+). |
+| `reg_threshold` | `"0"` | `reg-cli --thresholdRate` (0..1). Fraction of changed pixels tolerated before an image counts as changed. |
+| `enable_claude` | `true` | Have Claude classify the changed diff images (intended / regression / uncertain). |
+| `enable_claude_target_expansion` | `true` | Have Claude expand the target spec list from view-facing changes (view/template/component/CSS). |
+| `model` | `""` | Model to pass to `claude-code-action` (via `--model`). Empty uses the action default. |
+| `spec_paths` | `"system,components"` | Comma-separated spec subdirectories under `spec/` to capture (e.g. `system,components`). |
+
+### Secrets
+
+| Secret | Required | Description |
+|---|---|---|
+| `anthropic_api_key` | No | Anthropic API key for `claude-code-action`. Only needed when Claude features are enabled. |
+
+## How it works
+
+The workflow runs four jobs:
+
+1. **targets** — diffs `base_ref` against head to find changed spec files under `spec_paths`,
+   optionally asks Claude to expand that list from view-facing changes elsewhere in the diff,
+   and uploads the final list as an artifact.
+2. **shoot-base** / **shoot-head** — each checks out its own ref, runs only the target specs
+   with `SCREENSHOTS=1`, and uploads the resulting screenshots. Both depend only on `targets`,
+   so **they run in parallel**.
+3. **compare** — downloads both screenshot sets, pixel-diffs them with reg-cli, has Claude
+   classify the changed images (when enabled), and posts a PR comment and job summary. This
+   check never blocks the PR.
+
+## Reading the artifacts
+
+The compare job uploads two artifacts:
+
+- **`vr-flagged`** — only the diff images Claude classified as `regression` or `uncertain`,
+  uploaded uncompressed so the Actions UI can preview each PNG natively. Start here: it's
+  the small set of images that actually need a human look.
+- **`vr-full`** — `report.html`, `reg.json`, and the full `diff` image set, as a normal zip.
+  Download and open `report.html` locally for the complete reg-cli report, or to inspect a
+  diff image that wasn't flagged. The base and head screenshots themselves are the separate
+  **`vr-shots-base`** / **`vr-shots-head`** artifacts.
+
+## Notes
+
+- Set `enable_claude: false` to disable Claude classification entirely (reg-cli diffing still
+  runs); `anthropic_api_key` is not required in that case.
+- Claude's target-list expansion starts only from view-facing changes (views, templates,
+  components, CSS) — never from model/migration changes, which usually don't affect rendered
+  output and would cause over-selection.
+- A large batch of "added"/"deleted" images clustered under one example usually means a
+  Capybara step was inserted or removed in that example, shifting every later step's `NNN`
+  sequence number — not many independent regressions.
+- Diff images are linked via artifact URLs rather than embedded in the PR comment, because
+  GitHub's Markdown image embedding requires a URL its camo proxy can fetch unauthenticated,
+  and artifact URLs require authentication.
+- If a target spec fails on either branch, the shoot job goes red and the compare job is
+  skipped, so no diff report is produced. Fix the failing spec first — the workflow does not
+  diff against a partial screenshot set. ("Never blocks the PR" refers to the compare job not
+  failing the check once it runs, not to tolerating a broken spec run.)

data/lib/capybara/storyboard/version.rb

@@ -2,6 +2,6 @@
 
 module Capybara
   module Storyboard
-    VERSION = '0.1.0'
+    VERSION = '0.2.0'
   end
 end

data/skills/visual-regression-test/README.md

@@ -0,0 +1,60 @@
+# About the visual-regression-test skill
+
+Compares UI changes (views, components, CSS, system specs) before/after using
+capybara-storyboard screenshots and `reg-cli` image diffing, to catch unintended
+visual regressions. Invoked via `/visual-regression-test [ref]`.
+
+- "After" screenshots are taken from the main repo's working tree (including
+  uncommitted changes).
+- "Before" screenshots are taken from a separate worktree
+  (`.claude/worktrees/vrt-baseline`) checked out at the baseline ref (explicit
+  argument, HEAD, or the base branch — see `scripts/resolve-baseline.sh` for
+  details).
+- The two image sets are diffed with `reg-cli` and rendered as an HTML report.
+  The agent verifies whether each diff is intentional before reporting to the
+  user.
+
+See `SKILL.md` for the full procedure, baseline resolution logic, and target
+spec selection. This README only covers the setup needed to pass step 3b
+(the startup check inside the worktree).
+
+## Setup on worktree creation
+
+In `SKILL.md` step 3b, whether to proceed with capturing the "before" set is
+decided solely by whether `bundle exec rails runner 'exit 0'` (or an
+equivalent startup command) succeeds inside the worktree.
+
+This **assumes the project is configured to automatically provision the
+environment whenever `git worktree add` runs**. Gitignored config files (e.g.
+DB connection settings, credentials) must be copied, and dependencies must be
+installed, by the time the worktree is created — otherwise the startup check
+fails.
+
+How this is achieved doesn't matter. A common approach is a `post-checkout`
+hook. Since `post-checkout` fires on both regular checkouts and
+`git worktree add`, distinguish them using the `$1` argument (PREV_HEAD) Git
+passes in: it's the all-zero SHA only for `git worktree add`. Also note that
+if `core.hooksPath` isn't configured, `post-checkout` won't fire at all, even
+from a custom hooks directory.
+
+```sh
+#!/bin/sh
+# .githooks/post-checkout
+[ "$1" = "0000000000000000000000000000000000000000" ] || exit 0  # worktree add only
+
+main=$(realpath "$(git rev-parse --git-common-dir)/..")
+copy_if_missing() {
+  [ -f "$1" ] && return 0
+  [ -f "$main/$1" ] || return 0
+  mkdir -p "$(dirname "$1")"; cp "$main/$1" "$1"
+}
+copy_if_missing config/database.yml   # add any other required config files similarly
+
+[ -d vendor/bundle ] || bundle install
+[ -d node_modules ] || npm install
+# if the DB is shared with the main repo, don't call db:prepare or similar
+```
+
+Which files to copy and which install commands to run varies per project.
+Identify which gitignored files actually cause the startup command to fail
+inside the worktree, and adapt accordingly.

data/skills/visual-regression-test/SKILL.md

@@ -0,0 +1,304 @@
+---
+name: visual-regression-test
+disable-model-invocation: true
+description: >-
+  Compare before/after UI changes (views, components, CSS, system specs)
+  using capybara-storyboard screenshots and reg-cli image diffing, to catch
+  unintended visual regressions. The comparison baseline is determined
+  automatically from an optional ref argument, the presence of uncommitted
+  changes, and the base branch. Invoke explicitly with
+  /visual-regression-test [ref].
+---
+
+# Visual regression test (before / after screenshot diff)
+
+Catch unintended visual changes introduced by uncommitted edits. Capture capybara-storyboard
+screenshots **twice** — once against the main repo's working tree ("after"), and once against a
+separate worktree checked out at a baseline ref (HEAD, an explicit ref, or the merge-base with
+the base branch — "before") — then diff the two image sets with `reg-cli`. The output is an HTML
+report that highlights which screens changed, so both the agent and the user can confirm whether
+each change was intentional.
+
+This differs from "reviewing a screenshot of the current code as-is." A single-snapshot review
+answers "does this screen look right right now?" This skill answers "which screens changed
+compared to before my edit, and was that change intentional?" The latter is what actually catches
+regressions caused by editing a shared partial or layout or CSS. If there's no baseline to compare
+against and the user just wants to look at the current UI, that's the territory of the plain
+`capybara-storyboard` screenshot review, not this skill.
+
+Because the "before" capture runs in a separate worktree, it never touches the user's original
+working tree. There's no stash-and-restore step, and therefore no structural risk of losing the
+user's uncommitted work by failing to restore it.
+
+## Prerequisites (confirm before starting)
+
+This skill assumes capybara-storyboard is already set up (screenshots are captured with
+`SCREENSHOTS=1` and land under `tmp/screenshots/`). Setting it up is out of scope for this skill.
+
+- **You must be able to determine what to compare against (the baseline).** Baseline resolution
+  is delegated to `scripts/resolve-baseline.sh` (see "Determining the baseline" below and step
+  3a). If a ref argument was passed, that ref is always the baseline; with no argument and
+  uncommitted changes present, HEAD is the baseline; with no argument and a clean tree, the base
+  branch is the baseline. So "the tree is clean" is not itself a reason to abort — no-argument
+  plus clean proceeds to a comparison against the base branch (effectively a three-dot diff
+  against the merge-base). Only abort when the baseline genuinely can't be resolved
+  (`resolve-baseline.sh` exits 2).
+- **reg-cli** runs via `npx reg-cli` (no install needed). Open the report with the platform's
+  `open` command (macOS).
+
+## Determining the baseline (the "before" worktree's starting ref)
+
+"After" is always the main repo's working tree as-is (including any uncommitted changes — the
+after side is never turned into a worktree). Only the ref that the "before" worktree is checked
+out at is decided, using this priority:
+
+1. A ref/branch was passed as an argument → use that ref (regardless of uncommitted changes).
+2. No argument, and uncommitted changes exist (`git status --short` is non-empty) → HEAD (the
+   original behavior).
+3. No argument, and the tree is clean → the base branch.
+
+When the baseline is anything other than HEAD (case 1 or 3), the worktree's actual starting point
+is `git merge-base <baseline> HEAD` (a three-dot diff). For HEAD, no merge-base step is needed —
+use HEAD directly. This logic is centralized in the bundled `scripts/resolve-baseline.sh` (it
+prints the starting ref to stdout on one line, and exits 2 if it can't be resolved). Resolving the
+base branch itself is also handled by that script (roughly in the order
+`github-pr-base-branch` → `vscode-merge-base` → `@{upstream}` → `origin/HEAD`; see
+`resolve-baseline.sh` for the exact order and how each source is handled).
+
+### Interpreting the argument
+
+Treat the first argument to `/visual-regression-test [ref]` as the baseline ref (any git ref: a
+branch name, tag, SHA, `origin/foo`, etc.). Pass it through as the first argument to
+`resolve-baseline.sh` (pass an empty string if no argument was given).
+
+- No argument → baseline is determined automatically (uncommitted changes present = HEAD; clean
+  = base branch).
+- One argument → that ref is fixed as the baseline.
+- Two or more arguments → use only the first as the baseline, and briefly tell the user the rest
+  were ignored (this skill only supports a single baseline comparison; the range's starting point
+  is computed automatically via merge-base, so there's no need to accept both a start and an end).
+- If the given ref can't be resolved, `resolve-baseline.sh` stops with exit 2. Tell the user that
+  ref doesn't exist and abort (don't silently fall back to a different ref — respect the explicit
+  choice).
+
+## Procedure
+
+### 1. Identify the target specs
+
+Reuse the same target-selection logic as the plain `capybara-storyboard` review, narrowing to the
+system specs that exercise the changed screen(s). Never run the entire suite twice — it's very
+slow, and produces far too large a diff to read.
+
+- If `app/views`, `app/components`, or CSS changed, find the system specs that render that
+  screen (search by view/component name, controller action, or route). If the call sites are
+  countable (roughly 10 or fewer), target those specs directly.
+- If the change is to a layout or a widely shared partial/component, and the search above would
+  return most of the suite, don't enumerate every caller. Instead propose a small representative
+  sample (roughly 3-6 specs spanning distinct contexts: a signed-out screen, a standard
+  authenticated page, an admin/dense-UI page, a narrow-viewport spec if one exists), and **confirm
+  with the user** before running.
+- If a system spec itself changed, that spec is the target. Note that if the diff changes the
+  spec's own steps, the before/after screenshot sequences won't correspond 1:1 — reg-cli will
+  report these as added/removed images, which is expected in this case.
+- If only a single example in a file is relevant, it's fine to scope down to that example (`-e` or
+  a line number).
+
+Record the exact `rspec` target arguments (files and/or line numbers), since the same command runs
+twice and both runs must match.
+
+Note that the target spec may not exist, or may differ, at the baseline ref (resolved as the
+"before" starting point in step 3a). In no-argument mode with uncommitted changes (baseline=HEAD),
+this only applies to newly added specs. In ref-specified or base-branch-comparison mode, the
+target spec itself might be absent from the baseline, or have a different signature
+(describe/example names). Handle that per step 3c and the "images present on only one side are
+new/deleted" note in step 4. For target selection itself, just use the after side (the main repo's
+working tree) as the reference.
+
+This target argument works as-is in both the main repo and the worktree described below (the
+worktree is a full checkout of the baseline ref, so its path layout matches the main repo).
+However, if the target includes a spec file that doesn't exist at the baseline ref (a newly added
+spec, or one not yet present when comparing against a base branch), the worktree run will either
+fail to load it or produce an empty "before" set with all images treated as new. This is expected,
+not an error. In base-branch-comparison mode (no argument and clean, or an older explicit ref),
+even when the target spec exists at the baseline its contents may differ, so the screenshot count
+or naming may not match "after". reg-cli will report these as diffs/additions/deletions, which is
+also expected.
+
+### 2. Capture the "after" set (main repo's working tree)
+
+Capture the after state (with the diff applied) first. This only touches the main repo, so you
+can confirm the target specs actually pass before creating a worktree. If this fails, no worktree
+is created and no work is wasted.
+
+```bash
+SCREENSHOTS=1 bundle exec rspec <target specs>
+mkdir -p tmp/vrt && rm -rf tmp/vrt/after
+cp -R tmp/screenshots tmp/vrt/after
+```
+
+If the target spec **fails**, the screenshot set may be incomplete. Report the failure (the
+failure itself is a finding), and confirm with the user whether it's acceptable to use the partial
+capture as the diff baseline.
+
+### 3. Create a worktree at the baseline, verify it boots, and capture "before"
+
+#### 3a. Resolve the baseline and create the worktree there
+
+Determine the worktree's starting ref using the logic in "Determining the baseline". Pass the
+argument received by this invocation (the baseline ref, or empty if none) straight through to
+`resolve-baseline.sh`.
+
+```bash
+WT=.claude/worktrees/vrt-baseline
+# Pass "" if no argument was given. Prints the starting ref/SHA to stdout on one line. Abort on exit 2.
+BEFORE_REF=$(.claude/skills/visual-regression-test/scripts/resolve-baseline.sh "${BASELINE_ARG:-}")
+git worktree add "$WT" "$BEFORE_REF"
+```
+
+`.claude/worktrees/` is an existing empty directory reserved for this skill's use.
+`resolve-baseline.sh` writes the resolved mode (explicit / HEAD / base branch), and — when it's
+not HEAD — the merge-base it applied, to **stderr**. Always mention what was used as the baseline
+in the user-facing report (step 6) (e.g. "compared against the merge-base with the base branch
+origin/release-candidate"), since it changes how the diff should be read.
+
+When the starting point is something other than HEAD (an explicit ref or the base branch), the
+worktree is a full checkout of that SHA/branch. Note that the diff shown as "after minus before"
+then includes both the uncommitted changes in the after-side working tree *and* every commit
+between that starting point and HEAD — not just the uncommitted diff, as it would be when the
+baseline is HEAD.
+
+#### 3b. Detect environment setup (important — this is what makes the skill portable)
+
+Verify whether Rails actually boots inside the worktree by checking boot success itself, rather
+than checking for the presence of any specific hook. Judging by "does it boot" rather than by
+project-specific mechanics keeps this step portable to other projects.
+
+```bash
+( cd "$WT" && bundle exec rails runner 'exit 0' )
+```
+
+If it boots successfully, treat the config files and dependencies as present (either the
+worktree's automatic setup handled it, or none was needed) and proceed to 3c.
+
+If boot fails, abort. Tell the user "this worktree is missing config files/dependencies — you need
+automatic setup on worktree creation (e.g. a post-checkout hook)," and point them at this
+directory's `README.md` (the reference implementation for this skill). Then clean up the worktree
+before finishing.
+
+```bash
+git worktree remove --force "$WT"
+```
+
+#### 3c. Capture "before" inside the worktree
+
+Run the identical rspec command inside the worktree, and collect the results into the main repo's
+`tmp/vrt/before`.
+
+```bash
+( cd "$WT" && SCREENSHOTS=1 bundle exec rspec <target specs> )
+rm -rf tmp/vrt/before
+cp -R "$WT/tmp/screenshots" tmp/vrt/before
+```
+
+Running inside a `( cd "$WT" && ... )` subshell keeps the main repo's current directory clean. If
+the target spec fails, handle it the same way as "after" in step 2 (report the failure, and
+confirm with the user whether to use the partial capture).
+
+### 4. Discard the worktree and diff with reg-cli in the main repo
+
+Once "before" is captured, the worktree is no longer needed — discard it, then diff in the main
+repo.
+
+```bash
+git worktree remove --force "$WT"
+.claude/skills/visual-regression-test/scripts/reg-diff.sh tmp/vrt/after tmp/vrt/before tmp/vrt
+```
+
+`--force` is used because any untracked `tmp/screenshots` left inside the worktree doesn't matter
+by this point (it was already collected into `before` in step 3c) — just remove it.
+
+The bundled helper (`reg-diff.sh`) is equivalent to:
+
+```bash
+npx reg-cli tmp/vrt/after tmp/vrt/before tmp/vrt/diff \
+  --report tmp/vrt/report.html \
+  --json tmp/vrt/report.json
+```
+
+- Argument order matters: **actual (after) first, expected (before) second, diff directory
+  third.** Treating after as actual and before as expected frames the report as "what did my diff
+  do relative to HEAD."
+- Script exit codes: `0` for no visual change, `1` when a diff is detected (expected and
+  **not an error** in this context), `2` for an execution error (bad arguments / missing input
+  directory). `1` means "there's a diff, go verify it," not failure.
+- Since both sets ran the same spec, filenames match across the two sets and reg-cli pairs them
+  automatically by path. Images present on only one side are reported as added/deleted — this
+  happens when the diff changed the spec's own steps.
+
+### 5. Verify the detected diffs
+
+Read `tmp/vrt/report.json` to get the list of changed/added/deleted images, and for each changed
+screen, read the actual **diff image** (`tmp/vrt/diff/**`) along with the corresponding before/
+after PNGs. Use the filename to determine which action/screen it belongs to
+(`tmp/vrt/before/{Group}/{example}/{NNN_action}.png`). All paths are under the main repo's
+`tmp/vrt`.
+
+For each changed image, judge whether the visual change is **intentional**.
+
+- Does the visual change match what the code diff was trying to do? (e.g. a color/spacing/label
+  change the user made → expected.)
+- Or did an unrelated screen shift — a shared partial/layout edit leaking into pages it shouldn't
+  affect, broken layout, overflow, a rendering difference in a component? → report this as a
+  regression.
+- Minor sub-pixel/anti-aliasing noise with no meaningful change → note it as likely noise rather
+  than a regression. If noise dominates, mention that reg-cli's threshold options
+  (`--threshold`, `--enableAntialias`, etc.) can suppress it on a re-run.
+
+### 6. Report, then open the HTML report for the user
+
+Only open reg-cli's HTML report after finishing the verification in step 5. Opening it earlier
+would show the user diffs the agent hasn't judged yet, breaking the "agent verifies first, then
+presents results to the user" order.
+
+```bash
+open tmp/vrt/report.html
+```
+
+Then return a concise written report:
+
+- How many screens changed/were added/were deleted (from `report.json`).
+- For each meaningful change: which screen (spec + action/filename), and your judgment —
+  **intentional** (matches the code change) or **possible regression** (what looks wrong and
+  where).
+- A clear conclusion: either "all detected changes are consistent with the intended edit," or
+  "the following look like unintended regressions: ...".
+- Mention that the HTML report is open in the browser so the user can check it themselves.
+
+### 7. Output and cleanup
+
+By step 4, the worktree is guaranteed to already be removed via `git worktree remove --force`
+(the same applies if step 3b aborted — it's cleaned up before finishing there too). If for some
+reason execution was interrupted and the worktree was left behind, check with
+`git worktree list` for a stray `.claude/worktrees/vrt-baseline` and remove it with
+`git worktree remove --force <path>`.
+
+The worktree shares the same DB (`config/database.yml`) as the main repo. Don't run other tests
+concurrently while this skill is executing (steps 2-4), since DB state can conflict.
+
+When the baseline is a base branch or an older ref, that ref's Rails may be out of sync with the
+current shared DB schema (migrations already applied in the main repo), causing the worktree's
+boot check (step 3b) or rspec run to fail. This is an expected failure caused by an outdated
+baseline, and the boot check is what catches it before it goes further. If this happens
+frequently, suggest the user shift the baseline closer to HEAD (e.g. compare against uncommitted
+changes only).
+
+This skill's final step is "let the user see the HTML report," so **leave the report and the diff
+images it references (`tmp/vrt/`) in place**. Deleting them here would pull the report out from
+under the user while they're looking at it. Don't clean up on the agent's side.
+
+Just mention that these can be removed once no longer needed (leave the decision of when to run
+this to the user):
+
+```bash
+rm -rf tmp/vrt tmp/screenshots
+```

data/skills/visual-regression-test/scripts/reg-diff.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# Diff a before/after set of screenshots using reg-cli.
+# Fixes reg-cli's argument order (actual -> expected -> diff) and report paths
+# to prevent mix-ups on the caller's side.
+#
+# NOTE: does not open the report. The agent verifies report.json / diff images
+#       first, and opening it for the user happens afterward (that step is done
+#       explicitly in SKILL.md step 6).
+#
+# Usage:
+#   scripts/reg-diff.sh <after_dir> <before_dir> [out_dir]
+#     after_dir  : screenshot directory for after (working tree)  = reg-cli's actual
+#     before_dir : screenshot directory for before (HEAD)          = reg-cli's expected
+#     out_dir    : output destination (default tmp/vrt). diff/report.html/report.json written here
+#
+# Exit codes:
+#   0   ... no diff (no visual change)
+#   1   ... diff detected (expected in this skill's flow, not an error)
+#   2   ... execution error such as invalid arguments or missing input directory
+
+set -u
+
+after_dir=${1:-}
+before_dir=${2:-}
+out_dir=${3:-tmp/vrt}
+
+if [[ -z "$after_dir" || -z "$before_dir" ]]; then
+  echo "usage: reg-diff.sh <after_dir> <before_dir> [out_dir]" >&2
+  exit 2
+fi
+for d in "$after_dir" "$before_dir"; do
+  if [[ ! -d "$d" ]]; then
+    echo "error: directory not found: $d" >&2
+    exit 2
+  fi
+done
+
+diff_dir="$out_dir/diff"
+report_html="$out_dir/report.html"
+report_json="$out_dir/report.json"
+mkdir -p "$diff_dir"
+
+# Pass args in the order actual(after) -> expected(before) -> diff.
+npx reg-cli "$after_dir" "$before_dir" "$diff_dir" \
+  --report "$report_html" \
+  --json "$report_json"
+reg_status=$?
+
+# reg-cli exits non-zero (1) when a diff is detected. That's expected, so don't abort here.
+# Don't open the report (opened in step 6, after verification).
+echo "---"
+echo "reg-cli exit=$reg_status  (0=no diff, 1=diff detected)"
+echo "report.html: $report_html   # open this after verification to check"
+echo "report.json: $report_json"
+echo "diff images: $diff_dir"
+
+exit "$reg_status"

data/skills/visual-regression-test/scripts/resolve-baseline.sh

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+# Resolve the starting ref for the "before" worktree according to a priority order.
+#
+# Priority:
+#   1. <ref> argument given         -> use that ref as baseline
+#   2. no argument & uncommitted changes present -> HEAD as baseline (current behavior)
+#   3. no argument & clean tree     -> base branch as baseline
+#
+# If baseline is anything other than HEAD, the worktree starting point is
+# merge-base <baseline> HEAD (equivalent to a three-dot diff).
+# If it's HEAD, no merge-base is needed and "HEAD" is returned as-is.
+#
+# stdout : one line with the ref/SHA to pass to worktree add
+# stderr : human-readable explanation of the resolution
+# exit   : 0=resolved, 2=execution error
+
+set -u
+
+arg_ref=${1:-}
+
+log() { echo "$@" >&2; }
+
+# Check whether a ref actually exists (existence check only, no side effects)
+ref_exists() { git rev-parse --verify --quiet "$1" >/dev/null; }
+
+# Resolve the base branch (order follows github.com/aki77/claude-code-review as reference)
+resolve_base_branch() {
+  local cur cfg branch cand upstream sym
+  cur=$(git rev-parse --abbrev-ref HEAD 2>/dev/null)
+
+  # 1. branch.<cur>.github-pr-base-branch
+  #    NOTE: the value may be in "owner#repo#branch" form (VSCode GitHub PR extension).
+  #          Extract the last '#' segment as the branch name and prefer origin/.
+  #          The raw value can't be passed to rev-parse as-is.
+  cfg=$(git config "branch.$cur.github-pr-base-branch" 2>/dev/null)
+  if [ -n "$cfg" ]; then
+    branch=${cfg##*#}
+    for cand in "origin/$branch" "$branch" "$cfg"; do
+      if ref_exists "$cand"; then echo "$cand"; return 0; fi
+    done
+  fi
+
+  # 2. branch.<cur>.vscode-merge-base (e.g. origin/release-candidate; usable as-is)
+  cfg=$(git config "branch.$cur.vscode-merge-base" 2>/dev/null)
+  if [ -n "$cfg" ] && ref_exists "$cfg"; then
+    echo "$cfg"; return 0
+  fi
+
+  # 3. @{upstream}
+  upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}' 2>/dev/null)
+  if [ -n "$upstream" ]; then
+    echo "$upstream"; return 0
+  fi
+
+  # 4. origin/HEAD (e.g. refs/remotes/origin/release-candidate)
+  sym=$(git symbolic-ref --quiet refs/remotes/origin/HEAD 2>/dev/null)
+  if [ -n "$sym" ]; then
+    echo "${sym#refs/remotes/}"; return 0
+  fi
+
+  return 1
+}
+
+emit_mergebase() {
+  local baseline=$1 mb
+  if ! ref_exists "$baseline"; then
+    log "error: baseline ref not found: $baseline"
+    exit 2
+  fi
+  mb=$(git merge-base "$baseline" HEAD 2>/dev/null)
+  if [ -z "$mb" ]; then
+    log "error: no merge-base between $baseline and HEAD"
+    exit 2
+  fi
+  log "baseline: '$baseline' -> merge-base $mb"
+  echo "$mb"
+}
+
+# --- Priority resolution ---
+if [ -n "$arg_ref" ]; then
+  log "baseline mode: explicit ref '$arg_ref'"
+  emit_mergebase "$arg_ref"
+  exit 0
+fi
+
+if [ -n "$(git status --short)" ]; then
+  log "baseline mode: uncommitted changes present -> HEAD"
+  echo "HEAD"
+  exit 0
+fi
+
+log "baseline mode: clean tree -> base branch"
+base=$(resolve_base_branch) || { log "error: could not resolve base branch"; exit 2; }
+log "resolved base branch: $base"
+emit_mergebase "$base"
+exit 0

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: capybara-storyboard
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - aki
@@ -38,6 +38,7 @@ files:
 - README.md
 - Rakefile
 - docs/github-actions.md
+- docs/visual-regression.md
 - lib/capybara/storyboard.rb
 - lib/capybara/storyboard/configuration.rb
 - lib/capybara/storyboard/context.rb
@@ -49,6 +50,10 @@ files:
 - lib/capybara/storyboard/version.rb
 - sig/capybara/storyboard.rbs
 - skills/capybara-storyboard/SKILL.md
+- skills/visual-regression-test/README.md
+- skills/visual-regression-test/SKILL.md
+- skills/visual-regression-test/scripts/reg-diff.sh
+- skills/visual-regression-test/scripts/resolve-baseline.sh
 homepage: https://github.com/aki77/capybara-storyboard
 licenses:
 - MIT