snap_diff-capybara 0.0.1 → 2.0.0.beta1

data/docs/migration-guide.md

@@ -0,0 +1,286 @@
+# Migration Guide
+
+Migrate your visual regression testing from other tools to `capybara-screenshot-diff`. This guide covers the most common migration paths.
+
+## Why Switch?
+
+| Factor | Percy / Chromatic | BackstopJS | `capybara-screenshot-diff` |
+|--------|-------------------|------------|--------------------------|
+| Pricing | Paid SaaS (snapshot limits) | Free | Free (MIT) |
+| Infrastructure | Cloud service, API tokens | Node + Puppeteer | Ruby gem, no external services |
+| Baselines | Hosted on their servers | Local files | Git (committed to repo) |
+| Review | Web dashboard | HTML report | HTML report + GitHub PR comments |
+| PR integration | GitHub app | Manual CI steps | Reusable GitHub Action |
+| Offline | ❌ Requires internet | ✅ | ✅ |
+| Diff in PRs | Screenshot in comment | Manual | Upload artifact + PR comment |
+
+## From Percy
+
+### Setup changes
+
+**Before (Percy):**
+```ruby
+# Gemfile
+gem 'percy-capybara'
+
+# test helper
+require 'percy/capybara'
+
+# test
+def test_homepage
+  visit '/'
+  Percy::Capybara.screenshot('homepage')
+end
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+# Gemfile
+gem 'capybara-screenshot-diff'
+
+# test helper
+require 'capybara_screenshot_diff/minitest'
+
+# test class
+class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
+  include CapybaraScreenshotDiff::Minitest::Assertions
+
+  test "homepage" do
+    visit '/'
+    screenshot 'homepage'
+  end
+end
+```
+
+### What changes
+
+| Concept | Percy | capybara-screenshot-diff |
+|---------|-------|-------------------------|
+| Baseline storage | Percy cloud | Committed to git (`doc/screenshots/`) |
+| First run | Uploads to Percy | Saves locally, passes automatically |
+| CI setup | `PERCY_TOKEN` env var | GitHub Action (3 lines) |
+| Diff review | Percy dashboard | `snap_diff_report.html` or PR artifacts |
+| Update baselines | Percy's "Approve" button | Delete file, re-run tests, commit |
+| Snapshot limits | Paid plan dependent | Unlimited |
+| Parallel builds | Built-in | Thread-safe with t-locals + mutex |
+
+### CI migration
+
+**Before (Percy GitHub Action):**
+```yaml
+- name: Percy Test
+  run: PERCY_TOKEN=${{ secrets.PERCY_TOKEN }} bundle exec rake test
+```
+
+**After (capybara-screenshot-diff):**
+```yaml
+- uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
+  with:
+    ruby-version: '4.0'
+- run: bundle exec rake test
+- uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
+  if: failure()
+  with:
+    name: screenshots
+    pr-comment: 'true'
+```
+
+### Migration steps
+
+1. **Remove Percy gem and configuration**
+2. **Add `capybara-screenshot-diff`** to your Gemfile
+3. **Replace `Percy::Capybara.screenshot` calls** with `screenshot` (or `match_screenshot` for RSpec)
+4. **Run tests once** to generate baselines
+5. **Commit baselines** (`git add doc/screenshots/`)
+6. **Set up CI** with the GitHub Actions upload step
+7. **Remove Percy integration** from CI
+
+---
+
+## From Chromatic
+
+### Setup changes
+
+**Before (Chromatic + Storybook):**
+```js
+// .storybook/preview.js
+import { withScreenshot } from 'chromatic';
+
+export const decorators = [withScreenshot];
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+# test/system/stories_test.rb
+class StoriesTest < ApplicationSystemTestCase
+  test "landing page story" do
+    visit '/iframe.html?id=pages-landing--default'
+    screenshot 'stories/landing-page'
+  end
+end
+```
+
+### Key differences
+
+| Concept | Chromatic | capybara-screenshot-diff |
+|---------|-----------|-------------------------|
+| Focus | Storybook components | Full-page system tests |
+| Baseline | Chromatic cloud | Git-committed |
+| Review | Chromatic web UI | HTML report + PR artifacts |
+| CI integration | Chromatic GitHub App | GitHub Actions + PR comments |
+| Thresholds | Visual catch (AI) | Configurable tolerance (numeric) |
+
+### Migration approach
+
+Chromatic is primarily for Storybook component testing. If you want to continue testing individual components:
+
+1. **Replace with Capybara system tests** that visit each component's rendered page
+2. **Use `crop:` option** to isolate specific elements: `screenshot 'button', crop: '.my-button'`
+3. **Use `skip_area:` option** to ignore dynamic regions: `screenshot 'dashboard', skip_area: ['.timestamp']`
+
+---
+
+## From BackstopJS
+
+### Setup changes
+
+**Before (BackstopJS):**
+```json
+// backstop.json
+{
+  "id": "homepage",
+  "viewports": [{"width": 1280, "height": 1024}],
+  "scenarios": [{
+    "label": "Homepage",
+    "url": "http://localhost:3000",
+    "referenceUrl": "http://localhost:3000",
+    "selectors": ["document"]
+  }],
+  "paths": {
+    "bitmaps_reference": "backstop_data/bitmaps_reference",
+    "bitmaps_test": "backstop_data/bitmaps_test",
+    "html_report": "backstop_data/html_report"
+  }
+}
+```
+
+**After (capybara-screenshot-diff):**
+```ruby
+class HomepageTest < ApplicationSystemTestCase
+  test "homepage" do
+    visit '/'
+    screenshot 'homepage'
+  end
+end
+```
+
+### Key differences
+
+| Concept | BackstopJS | capybara-screenshot-diff |
+|---------|-----------|-------------------------|
+| Language | JavaScript + Node | Ruby (runs in test suite) |
+| Dependencies | Node, Puppeteer/Chromium | Ruby gems + optional libvips |
+| Test runner | Standalone CLI | Minitest, RSpec, Cucumber |
+| Selectors | CSS selectors for scenarios | CSS selectors for crop/skip_area |
+| Viewports | Per-scenario config | Global `window_size` setting |
+| CI report | HTML report | HTML report + GitHub Actions |
+| Stability | `misMatchThreshold` + `delay` | `tolerance` + `stability_time_limit` |
+
+### Configuration mapping
+
+| BackstopJS option | capybara-screenshot-diff equivalent |
+|-------------------|-------------------------------------|
+| `misMatchThreshold` | `tolerance` (0.0-1.0 scale, e.g. `0.01` = 1%) |
+| `delay` | `stability_time_limit` (seconds) |
+| `selectors` | `crop:` option with CSS selector |
+| `hideSelectors` | `skip_area:` option with CSS selectors |
+| `removeSelectors` | N/A — use `skip_area` or modify DOM before screenshot |
+| `waitTimeout` | `wait:` option (defaults to `Capybara.default_max_wait_time`) |
+| `viewports` | `window_size: [width, height]` |
+| `onReadyScript` | Custom setup in your test's `setup` block |
+
+### CI migration
+
+**Before (BackstopJS in CI):**
+```yaml
+- run: npx backstop test --config=backstop.json
+```
+
+**After (capybara-screenshot-diff in CI):**
+```yaml
+- uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
+- run: bundle exec rake test
+- uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
+  if: failure()
+  with:
+    name: screenshots
+    pr-comment: 'true'
+```
+
+### Migration steps
+
+1. **Remove BackstopJS configuration** (`backstop.json`, npm dependencies)
+2. **Convert scenarios to Capybara tests** — each scenario becomes a `screenshot` call
+3. **Map threshold and delay settings** to `tolerance` and `stability_time_limit`
+4. **Run tests** to generate baselines
+5. **Commit baselines** (`git add doc/screenshots/`)
+6. **Update CI** to use the GitHub Actions setup
+
+---
+
+## General Migration Checklist
+
+- [ ] Remove old gem/npm dependencies
+- [ ] Add `capybara-screenshot-diff` to Gemfile
+- [ ] Require the appropriate adapter (`minitest`, `rspec`, or `cucumber`)
+- [ ] Replace screenshot calls with `screenshot` / `match_screenshot`
+- [ ] Configure `window_size` for consistent viewport dimensions
+- [ ] Set `tolerance` or `perceptual_threshold` if your previous tool had a mismatch threshold
+- [ ] Add `.gitignore` patterns for diff artifacts
+- [ ] Run tests to generate baseline screenshots
+- [ ] Commit baselines to git
+- [ ] Set up CI with artifact upload
+- [ ] Optional: add HTML reporter and PR commenting
+
+## Common Gotchas
+
+### "My baselines are on Percy/Chromatic servers"
+
+You'll need to take fresh screenshots. Either:
+- Visit each page and capture manually
+- Run tests with `RECORD_SCREENSHOTS=1` to generate all baselines at once
+
+### "I had hundreds of BackstopJS scenarios"
+
+Start small. Migrate one test file at a time. The `screenshot_group` feature helps organize related screenshots:
+```ruby
+screenshot_group 'checkout'
+screenshot 'step1'
+screenshot 'step2'
+# Produces: doc/screenshots/checkout/00_step1.png, 01_step2.png
+```
+
+### "My tests are slow now"
+
+Use the VIPS driver for ~50ms comparisons per image:
+```ruby
+gem 'ruby-vips'
+Capybara::Screenshot::Diff.driver = :vips
+```
+
+### "The diffs look different from what I'm used to"
+
+Each tool uses different comparison algorithms:
+- **Percy:** Proprietary pixel-level comparison with AI smoothing
+- **Chromatic:** Visual catch algorithm (structure-aware)
+- **BackstopJS:** Resemble.js pixel comparison
+- **capybara-screenshot-diff:** Raw pixel difference with configurable tolerance
+
+Start with default settings, then adjust `tolerance` or `perceptual_threshold` based on your needs.
+
+## Need Help?
+
+- [Architecture Overview](docs/architecture.md) — understanding how comparisons work
+- [Configuration Reference](docs/configuration.md) — all available options
+- [CI Integration](docs/ci-integration.md) — setting up in CI
+- [GitHub Issues](https://github.com/snap-diff/snap_diff-capybara/issues) — ask questions

data/docs/organization.md

@@ -0,0 +1,204 @@
+# Screenshot Organization
+
+## Taking screenshots
+
+Add `screenshot '<my_feature>'` to your tests.  The screenshot will be saved in
+the `doc/screenshots` directory.
+
+Change your existing `save_screenshot` calls to `screenshot`
+
+```ruby
+test 'my useful feature' do
+  visit '/'
+  screenshot 'welcome_index'
+  click_button 'Useful feature'
+  screenshot 'feature_index'
+  click_button 'Perform action'
+  screenshot 'action_performed'
+end
+```
+
+This will produce a sequence of images like this
+
+```
+doc
+  screenshots
+    action_performed
+    feature_index
+    welcome_index
+```
+
+To store the screenshot history, add the `doc/screenshots` directory to your
+version control system (git).
+
+Screenshots are compared to the previously COMMITTED version of the same screenshot.
+
+**Note:** When a screenshot differs, diff artifacts (`.diff.png`, `.heatmap.diff.png`, etc.) are generated alongside the baseline. Add `*.diff.png`, `*.base.png`, `*.diff.webp`, `*.base.webp`, and `snap_diff_report.html` to your `.gitignore`. Clean up artifacts with `rake snap_diff:clean`.
+
+## Screenshot groups
+
+Commonly it is useful to group screenshots around a feature, and record them as
+a sequence.  To do this, add a `screenshot_group` call to the start of your
+test.
+
+```ruby
+test 'my useful feature' do
+  screenshot_group 'useful_feature'
+  visit '/'
+  screenshot 'welcome_index'
+  click_button 'Useful feature'
+  screenshot 'feature_index'
+  click_button 'Perform action'
+  screenshot 'action_performed'
+end
+```
+
+This will produce a sequence of images like this
+
+```
+doc
+  screenshots
+    useful_feature
+      00_welcome_index
+      01_feature_index
+      02_action_performed
+```
+
+**Note:** `screenshot_group` sets the group name for organizing screenshots. It does not delete existing files.
+
+
+## Screenshot sections
+
+You can introduce another level above the screenshot group called a
+`screenshot_section`.  The section name is inserted just before the group name
+in the save path.  If called in the setup of the test, all screenshots in
+that test will get the same prefix:
+
+```ruby
+setup do
+  screenshot_section 'my_feature'
+end
+
+test 'my subfeature' do
+  screenshot_group 'subfeature'
+  visit '/feature'
+  click_button 'Interesting button'
+  screenshot 'subfeature_index'
+  click_button 'Perform action'
+  screenshot 'action_performed'
+end
+```
+
+This will produce a sequence of images like this
+
+```
+doc
+  screenshots
+    my_feature
+      subfeature
+        00_subfeature_index
+        01_action_performed
+```
+
+
+## Setting `screenshot_section` and/or `screenshot_group` for all tests
+
+Setting the `screenshot_section` and/or `screenshot_group` for all tests can be
+done in the super class setup:
+
+```ruby
+class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
+  setup do
+    screenshot_section class_name.underscore.sub(/(_feature|_system)?_test$/, '')
+    screenshot_group name[5..-1]
+  end
+end
+```
+
+`screenshot_section` and/or `screenshot_group` can still be overridden in each
+test.
+
+
+## Capturing one area instead of the whole page
+
+You can crop images before comparison to be run, by providing region to crop as `[left, top, right, bottom]` or by css selector like `body .tag`
+
+```ruby
+test 'the cool' do
+  visit '/feature'
+  screenshot 'cool_element', crop: '#my_element'
+end
+```
+
+**Note:** When using a retina device screenshots dimensions might be off. If
+you are using (headless) chrome you can prevent this by setting the
+`force-device-scale-factor` argument to `1`.
+
+For Rails system specs using selenium you can do so for example by using the
+following snippet:
+
+```ruby
+driven_by :selenium, using: :chrome_headless do |options|
+  options.args << '--force-device-scale-factor=1'
+end
+```
+
+## Multiple Capybara drivers
+
+Often it is useful to test your app using different browsers.  To avoid the
+screenshots for different Capybara drivers to overwrite each other, set
+
+```ruby
+Capybara::Screenshot.add_driver_path = true
+```
+
+The example above will then save your screenshots like this
+(for poltergeist and selenium):
+
+```
+doc
+  screenshots
+    poltergeist
+      useful_feature
+        00_welcome_index
+        01_feature_index
+        02_action_performed
+    selenium
+      useful_feature
+        00_welcome_index
+        01_feature_index
+        02_action_performed
+```
+
+## Multiple OSs
+
+If you run your tests on multiple operating systems, you will most likely find
+the screen shots differ.  To avoid the screenshots for different OSs to
+overwrite each other, set
+
+```ruby
+Capybara::Screenshot.add_os_path = true
+```
+
+The example above will then save your screenshots like this
+(for Linux and Windows):
+
+```
+doc
+  screenshots
+    linux
+      useful_feature
+        00_welcome_index
+        01_feature_index
+        02_action_performed
+    windows
+      useful_feature
+        00_welcome_index
+        01_feature_index
+        02_action_performed
+```
+
+If you combine this config with the `add_driver_path` config, the driver will be
+put in front of the OS name.
+
+[← Back to README](../README.md)

data/docs/reporters.md

@@ -0,0 +1,46 @@
+# Reporters
+
+## Web UI for Reviewing Screenshot Changes
+
+Generate an interactive Web UI report of screenshot differences:
+
+```ruby
+# Add to test_helper.rb — one line, that's it
+require 'capybara_screenshot_diff/reporters/html'
+```
+
+After running tests, open the report (generated only when there are failures):
+
+```bash
+open doc/screenshots/snap_diff_report.html
+```
+
+The report includes a sidebar with thumbnails, side-by-side comparison with diff toggle, search, and summary stats. No configuration needed — just require it.
+
+**Note:** The report is not generated when all screenshots match. In parallel test environments, each worker writes to the same file — the last worker's results will be in the report.
+
+## Custom Reporters
+
+Build your own reporter by implementing `record` and `finalize`:
+
+```ruby
+class MyReporter
+  def record(assertions)
+    assertions.each do |assertion|
+      next unless assertion.compare&.difference&.different?
+      # process the failure — send to Slack, write JSON, etc.
+    end
+  end
+
+  def finalize
+    # called once at process exit — write summary, upload report, etc.
+  end
+end
+
+# Register in test_helper.rb
+CapybaraScreenshotDiff.reporters << MyReporter.new
+```
+
+Reporters are notified before assertions are cleared on each test teardown. `finalize` is called via `at_exit`.
+
+[← Back to README](../README.md)

data/docs/thread_safety.md

@@ -0,0 +1,112 @@
+# Thread Safety Guide for Parallel Testing
+
+This document explains how `snap_diff` behaves under Rails parallel tests with the `:thread` strategy.
+
+## Overview
+
+`snap_diff` is thread safe for parallel test execution as long as global configuration is set before tests run. Per-thread state is isolated, and shared state is protected where it matters.
+
+## Architecture Summary
+
+### Per-thread Assertion Registry
+
+Each thread gets its own `AssertionRegistry` stored in thread-local storage:
+
+```ruby
+def registry
+  Thread.current[:capybara_screenshot_diff_registry] ||= AssertionRegistry.new
+end
+```
+
+This prevents cross-thread leakage for assertions and screenshot naming.
+
+### Reporters Snapshot on Notify
+
+Reporters are notified using a snapshot protected by an eagerly initialized mutex:
+
+```ruby
+@reporters_mutex = Mutex.new
+
+def notify_reporters(assertions)
+  reporters_snapshot = reporters_mutex.synchronize { reporters.dup }
+  reporters_snapshot.each { |reporter| reporter.record(assertions) }
+end
+```
+
+This ensures a stable list while notifying without forcing a global lock around reporter work.
+
+### HTML Reporter Internal Lock
+
+The HTML reporter protects `@failures`, `@total`, and `@finalized` with a mutex so `record` and `finalize` can run safely:
+
+```ruby
+@mutex.synchronize do
+  return if @finalized
+  @total += total
+  @failures.concat(failures)
+end
+```
+
+`@finalized` is set only after `write_report` succeeds, so a failed write can be retried.
+
+### Screenshot Naming Isolation
+
+Each thread gets its own `ScreenshotNamer` via the per-thread registry, so counters, sections, and groups do not collide.
+
+### SnapManager Per Call
+
+`SnapManager` returns a new instance for each call, avoiding shared mutable state.
+
+## Global Configuration
+
+Configuration uses `mattr_accessor` and should be set once before tests run. Do not mutate config during parallel execution.
+
+## Parallel Test Lifecycle
+
+- Setup: per-thread registry is created, config is read
+- Execution: assertions are added to the thread-local registry
+- Teardown: `verify` and `reset` operate on the thread-local registry, reporters are notified
+- Exit: reporters finalize once per process (using mutex-protected snapshot)
+
+## Usage Examples
+
+```ruby
+parallelize(workers: :number_of_processors, with: :threads)
+
+Capybara::Screenshot::Diff.configure do |screenshot, diff|
+  screenshot.window_size = [1280, 1024]
+  screenshot.save_path = "doc/screenshots"
+  diff.tolerance = 0.001
+end
+```
+
+## Do and Do Not
+
+Do:
+- Set config once in test helper
+- Pass per-screenshot options in the call
+
+Do not:
+- Change global config inside tests
+- Manually mutate registry internals
+
+## File System Notes
+
+- Paths are unique per screenshot name and counter
+- `FileUtils.mv` is atomic on most file systems
+- Directory creation uses `mkpath`
+
+## Load-time thread safety
+
+Runtime state is thread-local (above), but *loading* the gem is a separate
+concern. The require graph is deliberately acyclic: `lib/snap_diff/*` units
+depend only on the legacy-config leaf
+(`capybara/screenshot/diff/config_legacy`) and specific sibling units, the
+umbrella files depend on the units, and nothing requires back up the chain.
+
+Eager mutual requires between entry points are forbidden, even guarded ones:
+per-thread "loading" flags cannot serialize Ruby's process-global per-file
+require locks, so two threads requiring opposite entry points first can
+deadlock (lock-order inversion — observed deterministically before this
+design). If two files ever need each other, extract the shared piece into a
+leaf both can require instead.

data/gems.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in capybara-screenshot-diff.gemspec
+gemspec path: __dir__
+
+gem "rake"
+
+# Image processing libraries
+gem "chunky_png", ">= 1.3", require: false
+gem "oily_png", platform: :ruby, git: "https://github.com/wvanbergen/oily_png", ref: "44042006e79efd42ce4b52c1d78a4c70f0b4b1b2"
+gem "ruby-vips", require: false
+
+group :test do
+  gem "capybara", ">= 3.26"
+  gem "mutex_m" # Needed for RubyMine debugging.  Try removing it.
+  gem "minitest", "< 6", require: false
+  gem "minitest-mock", require: false
+  gem "minitest-stub-const", require: false
+  gem "simplecov", require: false
+  gem "rspec", require: false
+end
+
+# Capybara Server
+gem "puma", require: false
+gem "rackup", require: false
+
+# Capybara Drivers
+gem "cuprite", require: false
+gem "selenium-webdriver", ">= 4.11", require: false
+
+# Test Frameworks
+# gem "cucumber", require: false
+# gem "cucumber-rails", require: false
+
+group :tools do
+  gem "standard", require: false
+end

data/lib/capybara/screenshot/diff/annotation_service.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+# Forwarder (ADR-004 v2 step 6): Capybara::Screenshot::Diff::AnnotationService
+# now resolves lazily via snap_diff/legacy_shims' const_missing, with a
+# deprecation warning pointing at SnapDiff::AnnotationService.
+require "snap_diff/annotation_service"
+require "snap_diff/legacy_shims"

data/lib/capybara/screenshot/diff/area_calculator.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+# Forwarder (ADR-004 v2 step 6): Capybara::Screenshot::Diff::AreaCalculator
+# now resolves lazily via snap_diff/legacy_shims' const_missing, with a
+# deprecation warning pointing at SnapDiff::AreaCalculator.
+require "snap_diff/area_calculator"
+require "snap_diff/legacy_shims"