capybara-screenshot-diff 1.12.0 → 1.13.0

data/docs/ci-integration.md

@@ -13,16 +13,7 @@ This sets up Capybara to serve static files and configures screenshot paths auto
 
 ## .gitignore Setup
 
-Add these patterns to your `.gitignore` — diff artifacts are generated at runtime and should not be committed:
-
-```gitignore
-# Screenshot diff artifacts (generated, not committed)
-*.diff.png
-*.base.png
-*.diff.webp
-*.base.webp
-snap_diff_report.html
-```
+See the [Quick Start section](../README.md#quick-start-5-minutes) in the README for recommended `.gitignore` patterns.
 
 Only commit the baseline screenshots (e.g., `homepage.png`). The `.base.png`, `.diff.png`, `.heatmap.diff.png`, and report files are regenerated on every test run.
 
@@ -36,15 +27,19 @@ Add to your test helper:
 require 'capybara_screenshot_diff/reporters/html'
 ```
 
-### 2. Upload artifacts (manual setup)
+### 2. Reusable composite action (recommended)
 
-This is the full YAML so you understand what each step does:
+The simplest way — one step handles artifact upload, job summary, and PR comments:
 
 ```yaml
 # .github/workflows/test.yml
 jobs:
   test:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write  # Required for PR comments
+
     steps:
       - uses: actions/checkout@v6
 
@@ -52,38 +47,6 @@ jobs:
         with:
           bundler-cache: true
 
-      # Install libvips for the :vips driver (optional — skip if using :chunky_png)
-      - name: Install libvips
-        run: sudo apt-get install -y libvips-dev
-
-      - name: Run tests
-        run: bundle exec rake test
-
-      # Upload HTML report — renders inline in Actions UI (no download needed)
-      - name: Upload screenshot report
-        if: failure()
-        uses: actions/upload-artifact@v7
-        with:
-          name: screenshot-report
-          path: doc/screenshots/snap_diff_report.html
-          archive: false
-          retention-days: 2
-
-      # Upload full report with images (for offline review)
-      - name: Upload full screenshot report
-        if: failure()
-        uses: actions/upload-artifact@v7
-        with:
-          name: screenshot-report-full
-          path: doc/screenshots/
-          retention-days: 2
-```
-
-### 3. Or use the reusable action (one line)
-
-Instead of the manual upload steps above, reference our composite action directly:
-
-```yaml
       - name: Run tests
         run: bundle exec rake test
 
@@ -92,54 +55,123 @@ Instead of the manual upload steps above, reference our composite action directl
         uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
         with:
           name: screenshots
+          pr-comment: 'true'
 ```
 
-This uploads diffs, Capybara failure screenshots, and the HTML report (inline + full) in one step.
+That's it. On failure, this will:
+- Upload diff images + HTML report as artifacts
+- Post a PR comment with links to the inline report and full artifact download
+- Add a job summary with report links (visible in the Actions UI)
 
-**Inputs:**
+#### Inputs
 
 | Input | Default | Description |
 |-------|---------|-------------|
 | `name` | (required) | Artifact name prefix |
 | `report-path` | `doc/screenshots` | Path to HTML report directory |
 | `retention-days` | `2` | Days to retain artifacts |
+| `pr-comment` | `false` | Post PR comment with report link (requires `pull-requests: write`) |
 
-### 4. PR comment with link to report (optional)
+#### Outputs
 
-Automatically comment on the PR pointing to the artifact. Uses `find-comment` to update existing comments instead of creating duplicates:
+| Output | Description |
+|--------|-------------|
+| `report-url` | Direct URL to the inline HTML report artifact |
+| `report-full-url` | Direct URL to the full report artifact (with images) |
 
-```yaml
-      - name: Find existing comment
-        if: failure() && github.event_name == 'pull_request'
-        uses: peter-evans/find-comment@v3
-        id: find-comment
-        with:
-          issue-number: ${{ github.event.pull_request.number }}
-          comment-author: 'github-actions[bot]'
-          body-includes: 'Screenshot diffs detected'
+### 3. Ruby + libvips setup action
+
+For consistent CI environments (libvips, font antialiasing disabled), use the setup action:
 
-      - name: Comment PR with report link
-        if: failure() && github.event_name == 'pull_request'
-        uses: peter-evans/create-or-update-comment@v5
+```yaml
+      - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
         with:
-          comment-id: ${{ steps.find-comment.outputs.comment-id }}
-          issue-number: ${{ github.event.pull_request.number }}
-          edit-mode: replace
-          body: |
-            ### Screenshot diffs detected
-            [View report](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}#artifacts)
+          ruby-version: '4.0'
+          cache-apt-packages: true
 ```
 
-**Required:** Add permissions to the job for PR commenting to work:
+This installs Ruby, libvips (with apt caching), and disables font antialiasing for consistent rendering across CI runs.
+
+#### Inputs
+
+| Input | Default | Description |
+|-------|---------|-------------|
+| `ruby-version` | (required) | Ruby version to install |
+| `cache-apt-packages` | `false` | Cache libvips apt packages for faster runs |
+| `ruby-cache-version` | — | Bundler cache version key |
+
+### 4. Full example with both actions
 
 ```yaml
 jobs:
   test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
     permissions:
       contents: read
       pull-requests: write
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
+        with:
+          ruby-version: '4.0'
+          cache-apt-packages: true
+
+      - run: bundle exec rake test
+
+      - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
+        if: failure()
+        with:
+          name: screenshots
+          pr-comment: 'true'
+```
+
+### 5. Manual setup (without composite actions)
+
+If you prefer full control, here's the expanded YAML:
+
+<details>
+<summary>Expand manual setup</summary>
+
+```yaml
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: ruby/setup-ruby@v1
+        with:
+          bundler-cache: true
+
+      - name: Install libvips
+        run: sudo apt-get install -y libvips-dev
+
+      - name: Run tests
+        run: bundle exec rake test
+
+      - name: Upload screenshot report
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: screenshot-report
+          path: doc/screenshots/snap_diff_report.html
+          archive: false
+          retention-days: 2
+
+      - name: Upload full report with images
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: screenshot-report-full
+          path: doc/screenshots/
+          retention-days: 2
 ```
 
+</details>
+
 ## Update Baselines in CI
 
 When intentional UI changes are made, baselines need to be re-recorded. You can do this locally:
@@ -152,6 +184,9 @@ git commit -m "chore: update screenshot baselines"
 
 Or add a workflow that maintainers can trigger manually:
 
+<details>
+<summary>Expand update-baselines workflow</summary>
+
 ```yaml
 # .github/workflows/update-baselines.yml
 name: Update Screenshot Baselines
@@ -175,12 +210,10 @@ jobs:
         with:
           ref: ${{ inputs.branch }}
 
-      - uses: ruby/setup-ruby@v1
+      - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
         with:
-          bundler-cache: true
-
-      - name: Install libvips
-        run: sudo apt-get install -y libvips-dev
+          ruby-version: '4.0'
+          cache-apt-packages: true
 
       - name: Record new baselines
         run: RECORD_SCREENSHOTS=1 bundle exec rake test
@@ -195,14 +228,11 @@ jobs:
           git push
 ```
 
+</details>
+
 **How it works:**
 1. Go to Actions → "Update Screenshot Baselines" → "Run workflow"
 2. Enter the branch name (e.g. your PR branch)
 3. The workflow records new baselines, commits, and pushes
 
-**Safety:**
-- Only maintainers with write access can trigger `workflow_dispatch`
-- The commit uses `git diff --staged --quiet ||` to skip empty commits
-- `GITHUB_TOKEN` pushes don't trigger subsequent CI runs ([by design](https://docs.github.com/actions/using-workflows/events-that-trigger-workflows))
-
 [← Back to README](../README.md)

data/docs/configuration.md

@@ -165,6 +165,22 @@ that does not have a corresponding previous image to compare against.
 This can be useful in situations where you want to ensure
 that every screenshot taken by your tests corresponds to an expected state of your application.
 
+### Marks new screenshots as pending
+
+To mark tests as pending (skipped) if a new screenshot is taken without a baseline, set:
+
+```ruby
+Capybara::Screenshot::Diff.pending_if_new = true
+# Required in CI, because fail_if_new defaults to true there and raises before
+# the pending marker is applied.
+Capybara::Screenshot::Diff.fail_if_new = false
+```
+
+If `pending_if_new` is set to `true`, the test will be marked as skipped in teardown
+when a new screenshot has no committed baseline to compare against.
+This is complementary to `fail_if_new` (which raises immediately); `fail_if_new` takes precedence since it raises first.
+This option is useful when you want to record new screenshots without blocking CI, but still track them as needing review.
+
 ### Screen shot save path
 
 By default, `Capybara::Screenshot::Diff` saves screenshots to a

data/docs/migration-guide.md

@@ -0,0 +1,286 @@
+# Migration Guide
+
+Migrate your visual regression testing from other tools to `capybara-screenshot-diff`. This guide covers the most common migration paths.
+
+## Why Switch?
+
+| Factor | Percy / Chromatic | BackstopJS | `capybara-screenshot-diff` |
+|--------|-------------------|------------|--------------------------|
+| Pricing | Paid SaaS (snapshot limits) | Free | Free (MIT) |
+| Infrastructure | Cloud service, API tokens | Node + Puppeteer | Ruby gem, no external services |
+| Baselines | Hosted on their servers | Local files | Git (committed to repo) |
+| Review | Web dashboard | HTML report | HTML report + GitHub PR comments |
+| PR integration | GitHub app | Manual CI steps | Reusable GitHub Action |
+| Offline | ❌ Requires internet | ✅ | ✅ |
+| Diff in PRs | Screenshot in comment | Manual | Upload artifact + PR comment |
+
+## From Percy
+
+### Setup changes
+
+**Before (Percy):**
+```ruby
+# Gemfile
+gem 'percy-capybara'
+
+# test helper
+require 'percy/capybara'
+
+# test
+def test_homepage
+  visit '/'
+  Percy::Capybara.screenshot('homepage')
+end
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+# Gemfile
+gem 'capybara-screenshot-diff'
+
+# test helper
+require 'capybara_screenshot_diff/minitest'
+
+# test class
+class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
+  include CapybaraScreenshotDiff::Minitest::Assertions
+
+  test "homepage" do
+    visit '/'
+    screenshot 'homepage'
+  end
+end
+```
+
+### What changes
+
+| Concept | Percy | capybara-screenshot-diff |
+|---------|-------|-------------------------|
+| Baseline storage | Percy cloud | Committed to git (`doc/screenshots/`) |
+| First run | Uploads to Percy | Saves locally, passes automatically |
+| CI setup | `PERCY_TOKEN` env var | GitHub Action (3 lines) |
+| Diff review | Percy dashboard | `snap_diff_report.html` or PR artifacts |
+| Update baselines | Percy's "Approve" button | Delete file, re-run tests, commit |
+| Snapshot limits | Paid plan dependent | Unlimited |
+| Parallel builds | Built-in | Thread-safe with t-locals + mutex |
+
+### CI migration
+
+**Before (Percy GitHub Action):**
+```yaml
+- name: Percy Test
+  run: PERCY_TOKEN=${{ secrets.PERCY_TOKEN }} bundle exec rake test
+```
+
+**After (capybara-screenshot-diff):**
+```yaml
+- uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
+  with:
+    ruby-version: '4.0'
+- run: bundle exec rake test
+- uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
+  if: failure()
+  with:
+    name: screenshots
+    pr-comment: 'true'
+```
+
+### Migration steps
+
+1. **Remove Percy gem and configuration**
+2. **Add `capybara-screenshot-diff`** to your Gemfile
+3. **Replace `Percy::Capybara.screenshot` calls** with `screenshot` (or `match_screenshot` for RSpec)
+4. **Run tests once** to generate baselines
+5. **Commit baselines** (`git add doc/screenshots/`)
+6. **Set up CI** with the GitHub Actions upload step
+7. **Remove Percy integration** from CI
+
+---
+
+## From Chromatic
+
+### Setup changes
+
+**Before (Chromatic + Storybook):**
+```js
+// .storybook/preview.js
+import { withScreenshot } from 'chromatic';
+
+export const decorators = [withScreenshot];
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+# test/system/stories_test.rb
+class StoriesTest < ApplicationSystemTestCase
+  test "landing page story" do
+    visit '/iframe.html?id=pages-landing--default'
+    screenshot 'stories/landing-page'
+  end
+end
+```
+
+### Key differences
+
+| Concept | Chromatic | capybara-screenshot-diff |
+|---------|-----------|-------------------------|
+| Focus | Storybook components | Full-page system tests |
+| Baseline | Chromatic cloud | Git-committed |
+| Review | Chromatic web UI | HTML report + PR artifacts |
+| CI integration | Chromatic GitHub App | GitHub Actions + PR comments |
+| Thresholds | Visual catch (AI) | Configurable tolerance (numeric) |
+
+### Migration approach
+
+Chromatic is primarily for Storybook component testing. If you want to continue testing individual components:
+
+1. **Replace with Capybara system tests** that visit each component's rendered page
+2. **Use `crop:` option** to isolate specific elements: `screenshot 'button', crop: '.my-button'`
+3. **Use `skip_area:` option** to ignore dynamic regions: `screenshot 'dashboard', skip_area: ['.timestamp']`
+
+---
+
+## From BackstopJS
+
+### Setup changes
+
+**Before (BackstopJS):**
+```json
+// backstop.json
+{
+  "id": "homepage",
+  "viewports": [{"width": 1280, "height": 1024}],
+  "scenarios": [{
+    "label": "Homepage",
+    "url": "http://localhost:3000",
+    "referenceUrl": "http://localhost:3000",
+    "selectors": ["document"]
+  }],
+  "paths": {
+    "bitmaps_reference": "backstop_data/bitmaps_reference",
+    "bitmaps_test": "backstop_data/bitmaps_test",
+    "html_report": "backstop_data/html_report"
+  }
+}
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+class HomepageTest < ApplicationSystemTestCase
+  test "homepage" do
+    visit '/'
+    screenshot 'homepage'
+  end
+end
+```
+
+### Key differences
+
+| Concept | BackstopJS | capybara-screenshot-diff |
+|---------|-----------|-------------------------|
+| Language | JavaScript + Node | Ruby (runs in test suite) |
+| Dependencies | Node, Puppeteer/Chromium | Ruby gems + optional libvips |
+| Test runner | Standalone CLI | Minitest, RSpec, Cucumber |
+| Selectors | CSS selectors for scenarios | CSS selectors for crop/skip_area |
+| Viewports | Per-scenario config | Global `window_size` setting |
+| CI report | HTML report | HTML report + GitHub Actions |
+| Stability | `misMatchThreshold` + `delay` | `tolerance` + `stability_time_limit` |
+
+### Configuration mapping
+
+| BackstopJS option | capybara-screenshot-diff equivalent |
+|-------------------|-------------------------------------|
+| `misMatchThreshold` | `tolerance` (0.0-1.0 scale, e.g. `0.01` = 1%) |
+| `delay` | `stability_time_limit` (seconds) |
+| `selectors` | `crop:` option with CSS selector |
+| `hideSelectors` | `skip_area:` option with CSS selectors |
+| `removeSelectors` | N/A — use `skip_area` or modify DOM before screenshot |
+| `waitTimeout` | `wait:` option (defaults to `Capybara.default_max_wait_time`) |
+| `viewports` | `window_size: [width, height]` |
+| `onReadyScript` | Custom setup in your test's `setup` block |
+
+### CI migration
+
+**Before (BackstopJS in CI):**
+```yaml
+- run: npx backstop test --config=backstop.json
+```
+
+**After (capybara-screenshot-diff in CI):**
+```yaml
+- uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
+- run: bundle exec rake test
+- uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
+  if: failure()
+  with:
+    name: screenshots
+    pr-comment: 'true'
+```
+
+### Migration steps
+
+1. **Remove BackstopJS configuration** (`backstop.json`, npm dependencies)
+2. **Convert scenarios to Capybara tests** — each scenario becomes a `screenshot` call
+3. **Map threshold and delay settings** to `tolerance` and `stability_time_limit`
+4. **Run tests** to generate baselines
+5. **Commit baselines** (`git add doc/screenshots/`)
+6. **Update CI** to use the GitHub Actions setup
+
+---
+
+## General Migration Checklist
+
+- [ ] Remove old gem/npm dependencies
+- [ ] Add `capybara-screenshot-diff` to Gemfile
+- [ ] Require the appropriate adapter (`minitest`, `rspec`, or `cucumber`)
+- [ ] Replace screenshot calls with `screenshot` / `match_screenshot`
+- [ ] Configure `window_size` for consistent viewport dimensions
+- [ ] Set `tolerance` or `perceptual_threshold` if your previous tool had a mismatch threshold
+- [ ] Add `.gitignore` patterns for diff artifacts
+- [ ] Run tests to generate baseline screenshots
+- [ ] Commit baselines to git
+- [ ] Set up CI with artifact upload
+- [ ] Optional: add HTML reporter and PR commenting
+
+## Common Gotchas
+
+### "My baselines are on Percy/Chromatic servers"
+
+You'll need to take fresh screenshots. Either:
+- Visit each page and capture manually
+- Run tests with `RECORD_SCREENSHOTS=1` to generate all baselines at once
+
+### "I had hundreds of BackstopJS scenarios"
+
+Start small. Migrate one test file at a time. The `screenshot_group` feature helps organize related screenshots:
+```ruby
+screenshot_group 'checkout'
+screenshot 'step1'
+screenshot 'step2'
+# Produces: doc/screenshots/checkout/00_step1.png, 01_step2.png
+```
+
+### "My tests are slow now"
+
+Use the VIPS driver for ~50ms comparisons per image:
+```ruby
+gem 'ruby-vips'
+Capybara::Screenshot::Diff.driver = :vips
+```
+
+### "The diffs look different from what I'm used to"
+
+Each tool uses different comparison algorithms:
+- **Percy:** Proprietary pixel-level comparison with AI smoothing
+- **Chromatic:** Visual catch algorithm (structure-aware)
+- **BackstopJS:** Resemble.js pixel comparison
+- **capybara-screenshot-diff:** Raw pixel difference with configurable tolerance
+
+Start with default settings, then adjust `tolerance` or `perceptual_threshold` based on your needs.
+
+## Need Help?
+
+- [Architecture Overview](docs/architecture.md) — understanding how comparisons work
+- [Configuration Reference](docs/configuration.md) — all available options
+- [CI Integration](docs/ci-integration.md) — setting up in CI
+- [GitHub Issues](https://github.com/snap-diff/snap_diff-capybara/issues) — ask questions

data/docs/organization.md

@@ -33,29 +33,7 @@ version control system (git).
 
 Screenshots are compared to the previously COMMITTED version of the same screenshot.
 
-### Generated artifacts (do not commit)
-
-When a screenshot differs, the gem generates temporary diff files alongside the baseline:
-
-| Pattern | Description |
-|---------|-------------|
-| `*.base.png` | VCS checkout of the committed baseline |
-| `*.diff.png` | Annotated diff with changes highlighted |
-| `*.base.diff.png` | Annotated baseline with diff region marked |
-| `*.heatmap.diff.png` | Heatmap of pixel differences |
-| `snap_diff_report.html` | Interactive Web UI report |
-
-Add these to `.gitignore`:
-
-```gitignore
-*.diff.png
-*.base.png
-*.diff.webp
-*.base.webp
-snap_diff_report.html
-```
-
-Clean up artifacts with `rake snap_diff:clean`.
+**Note:** When a screenshot differs, diff artifacts (`.diff.png`, `.heatmap.diff.png`, etc.) are generated alongside the baseline. Add `*.diff.png`, `*.base.png`, `*.diff.webp`, `*.base.webp`, and `snap_diff_report.html` to your `.gitignore`. Clean up artifacts with `rake snap_diff:clean`.
 
 ## Screenshot groups
 

data/lib/capybara/screenshot/diff/screenshot_matcher.rb

@@ -32,11 +32,25 @@ module Capybara
 
           # Pre-computation: No need to compare without base screenshot
           # NOTE: Consider to return PreValid Assertion Value Object with hard coded valid result
-          return unless need_to_compare?
+          unless need_to_compare?
+            CapybaraScreenshotDiff.record_new_screenshot(screenshot_full_name)
+            return
+          end
 
           create_screenshot_assertion(skip_stack_frames + 1, comparison_options)
         end
 
+        # Captures a screenshot without comparing it to a baseline.
+        def capture
+          check_window_size!
+          prepare_screenshot_options
+
+          capture_options, comparison_options = extract_capture_and_comparison_options!(driver_options)
+
+          @snapshot.manager.create_output_directory_for(@snapshot.path)
+          capture_screenshot(capture_options, comparison_options)
+        end
+
         private
 
         def need_to_compare?

data/lib/capybara/screenshot/diff/version.rb

@@ -3,7 +3,7 @@
 module Capybara
   module Screenshot
     module Diff
-      VERSION = "1.12.0"
+      VERSION = "1.13.0"
     end
   end
 end

data/lib/capybara_screenshot_diff/cucumber.rb

@@ -9,4 +9,13 @@ Before do
   Capybara::Screenshot::BrowserHelpers.resize_window_if_needed
 end
 
+After do |scenario|
+  if !scenario.failed? && Capybara::Screenshot::Diff.pending_if_new && CapybaraScreenshotDiff.new_screenshots_present?
+    names = CapybaraScreenshotDiff.new_screenshots
+    skip_this_scenario("No baseline for: #{names.join(", ")}. Commit the captured screenshots to record them.")
+  end
+ensure
+  CapybaraScreenshotDiff.reset
+end
+
 AfterAll { CapybaraScreenshotDiff.finalize_reporters! }