snap_diff-capybara 2.0.0.beta3 → 2.0.0.beta4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +438 -0
  3. data/README.md +330 -0
  4. data/docs/UPGRADING.md +232 -34
  5. data/docs/architecture.md +9 -7
  6. data/docs/ci-integration.md +42 -11
  7. data/docs/configuration.md +176 -10
  8. data/docs/drivers.md +48 -5
  9. data/docs/framework-setup.md +10 -3
  10. data/docs/migration-guide.md +24 -13
  11. data/docs/reporters.md +70 -1
  12. data/docs/snapdiff.md +41 -8
  13. data/docs/thread_safety.md +77 -86
  14. data/lib/capybara/screenshot/diff/annotation_service.rb +1 -3
  15. data/lib/capybara/screenshot/diff/area_calculator.rb +1 -3
  16. data/lib/capybara/screenshot/diff/browser_helpers.rb +1 -3
  17. data/lib/capybara/screenshot/diff/config_legacy.rb +15 -60
  18. data/lib/capybara/screenshot/diff/difference.rb +1 -4
  19. data/lib/capybara/screenshot/diff/drivers/chunky_png_driver.rb +3 -3
  20. data/lib/capybara/screenshot/diff/drivers/vips_driver.rb +2 -3
  21. data/lib/capybara/screenshot/diff/drivers.rb +4 -5
  22. data/lib/capybara/screenshot/diff/image_compare.rb +11 -22
  23. data/lib/capybara/screenshot/diff/image_preprocessor.rb +1 -3
  24. data/lib/capybara/screenshot/diff/os.rb +4 -11
  25. data/lib/capybara/screenshot/diff/region.rb +2 -2
  26. data/lib/capybara/screenshot/diff/reporters/default.rb +4 -17
  27. data/lib/capybara/screenshot/diff/screenshot_matcher.rb +1 -3
  28. data/lib/capybara/screenshot/diff/screenshoter.rb +1 -3
  29. data/lib/capybara/screenshot/diff/stable_screenshoter.rb +1 -3
  30. data/lib/capybara/screenshot/diff/utils.rb +1 -3
  31. data/lib/capybara/screenshot/diff/vcs.rb +1 -3
  32. data/lib/capybara/screenshot/diff/version.rb +8 -13
  33. data/lib/capybara-screenshot-diff.rb +10 -1
  34. data/lib/capybara_screenshot_diff/attempts_reporter.rb +1 -3
  35. data/lib/capybara_screenshot_diff/dsl.rb +5 -0
  36. data/lib/capybara_screenshot_diff/error_with_filtered_backtrace.rb +1 -4
  37. data/lib/capybara_screenshot_diff/reporters/html.rb +1 -3
  38. data/lib/capybara_screenshot_diff/screenshot_namer.rb +1 -3
  39. data/lib/capybara_screenshot_diff/snap.rb +1 -3
  40. data/lib/capybara_screenshot_diff/snap_manager.rb +1 -3
  41. data/lib/capybara_screenshot_diff.rb +17 -21
  42. data/lib/snap_diff/browser_helpers.rb +26 -5
  43. data/lib/snap_diff/capture/viewport.rb +2 -5
  44. data/lib/snap_diff/comparison.rb +54 -3
  45. data/lib/snap_diff/comparison_result.rb +3 -1
  46. data/lib/snap_diff/config.rb +189 -93
  47. data/lib/snap_diff/deprecation.rb +89 -28
  48. data/lib/snap_diff/driver.rb +18 -0
  49. data/lib/snap_diff/drivers/vips_driver.rb +12 -6
  50. data/lib/snap_diff/drivers.rb +94 -12
  51. data/lib/snap_diff/dsl.rb +45 -47
  52. data/lib/snap_diff/integrations/cucumber.rb +1 -1
  53. data/lib/snap_diff/integrations/minitest.rb +45 -9
  54. data/lib/snap_diff/integrations/rspec.rb +11 -0
  55. data/lib/snap_diff/legacy_shims.rb +285 -27
  56. data/lib/snap_diff/removal.rb +159 -0
  57. data/lib/snap_diff/reporters/default.rb +86 -23
  58. data/lib/snap_diff/reporters/html.rb +29 -13
  59. data/lib/snap_diff/reporting.rb +329 -3
  60. data/lib/snap_diff/screenshot_assertion.rb +28 -23
  61. data/lib/snap_diff/screenshot_matcher.rb +121 -14
  62. data/lib/snap_diff/screenshot_namer.rb +1 -19
  63. data/lib/snap_diff/screenshoter.rb +5 -7
  64. data/lib/snap_diff/snap.rb +6 -1
  65. data/lib/snap_diff/snap_manager.rb +2 -3
  66. data/lib/snap_diff/stable_screenshoter.rb +2 -2
  67. data/lib/snap_diff/static.rb +1 -1
  68. data/lib/snap_diff/utils.rb +35 -17
  69. data/lib/snap_diff/vcs.rb +40 -7
  70. data/lib/snap_diff/version.rb +1 -1
  71. data/lib/snap_diff-capybara.rb +33 -3
  72. data/lib/snap_diff.rb +27 -37
  73. metadata +12 -10
  74. data/CODE_OF_CONDUCT.md +0 -129
  75. data/Rakefile +0 -65
  76. data/capybara-screenshot-diff.gemspec +0 -29
  77. data/docs/RELEASE_PREP.md +0 -44
  78. data/docs/docker-testing.md +0 -24
  79. data/gems.rb +0 -39
@@ -11,7 +11,6 @@ SnapDiff.configure do |config|
11
11
  config.stability_time_limit = 1
12
12
  config.blur_active_element = true
13
13
  config.hide_caret = true
14
- config.driver = :vips
15
14
  config.tolerance = 0.0005
16
15
  config.color_distance_limit = 15
17
16
  end
@@ -26,12 +25,15 @@ Capybara::Screenshot::Diff.configure do |screenshot, diff|
26
25
  screenshot.stability_time_limit = 1
27
26
  screenshot.blur_active_element = true
28
27
  screenshot.hide_caret = true
29
- diff.driver = :vips
30
28
  diff.tolerance = 0.0005
31
29
  diff.color_distance_limit = 15
32
30
  end
33
31
  ```
34
32
 
33
+ > **`driver:` is deliberately absent from both examples.** The setting is removed in 2.1
34
+ > (`NoMethodError` at config time) and 2.0 cannot warn about it. Add `gem "ruby-vips"` and
35
+ > leave the selection alone — see [Drivers](drivers.md#removed-in-21-everything-on-this-page-except-vips).
36
+
35
37
  `SnapDiff::Config` **is** the storage; the legacy accessors are thin delegators onto it. There is
36
38
  one source of truth, so a write through either surface is visible through the other — mixing them
37
39
  is safe, and you can migrate a suite one line at a time:
@@ -46,9 +48,86 @@ exception: `Capybara::Screenshot.enabled` is `SnapDiff.config.screenshot_enabled
46
48
  `SnapDiff.config.enabled` is taken by `Capybara::Screenshot::Diff.enabled`. See
47
49
  [SnapDiff — the canonical API](snapdiff.md) for the full SnapDiff-native surface.
48
50
 
49
- **Note:** `fail_if_new` defaults to `true` in CI environments (when `ENV['CI']` is set). New screenshots are allowed locally but rejected in CI no configuration needed.
51
+ **Note:** Setting `SnapDiff.config.screenshot_enabled = false` is sufficient to disable all screenshots. There is no need to define no-op modules or monkey-patch the gem.
52
+
53
+ ## Record modes — accepting changes
54
+
55
+ `record` is the single setting for *what happens when a screenshot has no committed baseline, or
56
+ when you want to accept the ones that changed*. It replaces `fail_if_new`, which is removed in 2.1.
57
+
58
+ ```ruby
59
+ SnapDiff.config.record = :once # what a local run already does; see Precedence below for CI
60
+ ```
61
+
62
+ | Mode | Missing baseline | Baseline present | Reach for it when |
63
+ |------|------------------|------------------|-------------------|
64
+ | `:once` | recorded, not compared | compared | the default — you want a normal run |
65
+ | `:none` | **fails** with the `git add` command attached | compared | every screenshot must already be recorded |
66
+ | `:all` | recorded | **re-recorded, not compared** | you changed the UI on purpose and want the new rendering to become the baseline |
67
+
68
+ ### `:all` — the bulk-accept verb
69
+
70
+ After an intentional redesign that changed forty screenshots, re-record them in one run rather
71
+ than accepting them one failure at a time:
72
+
73
+ ```ruby
74
+ # test_helper.rb. There is no CLI flag — the gem has no runner to hang one on —
75
+ # so gate it on an environment variable of your own if you want one:
76
+ SnapDiff.config.record = ENV["ACCEPT_SCREENSHOTS"] ? :all : :once
77
+ ```
78
+
79
+ ```bash
80
+ ACCEPT_SCREENSHOTS=1 bin/rails test:system # with the line above in test_helper.rb
81
+ ```
82
+
83
+ Every screenshot is written to its baseline path and nothing is compared, so `git status` lists
84
+ exactly what changed. Review the images, then commit them — see
85
+ [Accepting an intentional change](../README.md#accepting-an-intentional-change).
86
+
87
+ At the end of the run the gem names what it accepted:
88
+
89
+ ```
90
+ [snap_diff] record: :all re-recorded 3 screenshots WITHOUT comparing: checkout/cart, checkout/payment, checkout/review. Review the result before committing -- an unintended change is accepted just as silently.
91
+ ```
92
+
93
+ > **`:all` refuses to run under CI** (when `ENV['CI']` is set to a non-empty value). It accepts
94
+ > every rendering by design, so a mode left in a committed config file would buy you a build that
95
+ > compares nothing and passes forever, with the "recorded" screenshots discarded when the runner
96
+ > is torn down. Re-record locally, where you can look at the result.
97
+ >
98
+ > A CI job that needs to record screenshots with **no baseline yet** does not need `:all`:
99
+ > `record = :once` records those and still compares everything that has a baseline. See
100
+ > [CI integration](ci-integration.md).
101
+
102
+ ### Per screenshot
103
+
104
+ ```ruby
105
+ assert_matches_screenshot "flaky_widget", record: :none # this one must already exist
106
+ ```
107
+
108
+ ### Precedence
109
+
110
+ **An explicitly set mode outranks `fail_if_new`; `fail_if_new` decides only when no mode was set.**
111
+ That is the same rule `fail_if_new` itself has over the `CI` sniff — explicit outranks implicit, all
112
+ the way down. Per screenshot outranks the config; the config outranks `fail_if_new`.
113
+
114
+ With **nothing** set, `record` reads back as `:none` under CI and `:once` off it — which is exactly
115
+ what `fail_if_new` has always done, so a suite that never mentions `record` behaves as it always
116
+ did. The missing-baseline default is deliberately unchanged: failing only under CI is what Jest,
117
+ AVA, Vitest, testthat and jest-image-snapshot all chose, and a screenshot baseline recorded on your
118
+ laptop is often worthless on another OS. `:none` makes strictness an explicit choice instead.
50
119
 
51
- **Note:** Setting `Capybara::Screenshot.enabled = false` is sufficient to disable all screenshots. There is no need to define no-op modules or monkey-patch the gem.
120
+ | You wrote | `record` reads | Missing baseline |
121
+ |-----------|----------------|------------------|
122
+ | nothing, off CI | `:once` | recorded |
123
+ | nothing, under CI | `:none` | fails |
124
+ | `record = :once` | `:once` | recorded, on CI too |
125
+ | `record = :none` | `:none` | fails, off CI too |
126
+ | `record = :once`, `fail_if_new = true` | `:once` | recorded — the mode wins |
127
+ | `record = nil`, `fail_if_new = true` | `:none` | fails — nil hands it back |
128
+
129
+ A misspelt mode raises `ArgumentError` at the point you set it, rather than reading back as
130
+ "nobody said".
52
131
 
53
132
  ## Recommended tolerance values
54
133
 
@@ -105,7 +184,8 @@ screenshot 'dashboard', color_distance_limit: 15
105
184
  ## Configuration Tiers
106
185
 
107
186
  **Tier 1 — Zero config (works immediately):**
108
- `blur_active_element`, `hide_caret`, and `fail_if_new` (in CI) are enabled by default.
187
+ `blur_active_element` and `hide_caret` are on by default, and `record` behaves as `:none` in CI
188
+ (a missing baseline fails) and `:once` off it.
109
189
  Just `require 'snap_diff/integrations/minitest'` (legacy: `capybara_screenshot_diff/minitest`) and call `screenshot`.
110
190
 
111
191
  **Tier 2 — Set when tests are flaky:**
@@ -122,7 +202,7 @@ Just `require 'snap_diff/integrations/minitest'` (legacy: `capybara_screenshot_d
122
202
  | Setting | When to use |
123
203
  |---------|-------------|
124
204
  | `perceptual_threshold` | Anti-aliasing false positives across OS/browser versions |
125
- | `shift_distance_limit` | Content shifts by a few pixels (ChunkyPNG only) |
205
+ | `shift_distance_limit` | Content shifts by a few pixels (ChunkyPNG only — **removed in 2.1**) |
126
206
  | `area_size_limit` | Allow small diff regions below a pixel count |
127
207
  | `color_distance_limit` | Fine-tune raw RGB channel tolerance |
128
208
  | `median_filter_window_size` | Smooth noise before comparison (VIPS only) |
@@ -147,15 +227,18 @@ unless the desired window size can be achieved.
147
227
  If you want to skip taking screen shots, set
148
228
 
149
229
  ```ruby
150
- Capybara::Screenshot.enabled = false
230
+ SnapDiff.config.screenshot_enabled = false
151
231
  ```
152
232
 
153
233
  You can of course set this by an environment variable
154
234
 
155
235
  ```ruby
156
- Capybara::Screenshot.enabled = ENV['TAKE_SCREENSHOTS']
236
+ SnapDiff.config.screenshot_enabled = ENV['TAKE_SCREENSHOTS']
157
237
  ```
158
238
 
239
+ A disabled screenshot is not an assertion, and Minitest is told so: a test whose only assertion
240
+ was a screenshot reports as missing assertions rather than as a pass over nothing.
241
+
159
242
  ### Disabling diff
160
243
 
161
244
  If you want to skip the assertion for change in the screen shot, set
@@ -172,6 +255,10 @@ Capybara::Screenshot::Diff.enabled = ENV['COMPARE_SCREENSHOTS']
172
255
 
173
256
  ### Tolerate screenshot differences
174
257
 
258
+ > **Removed in 2.1.** A screenshot that differs from its baseline fails — that is what the gem is
259
+ > for. To *accept* a difference, re-record it: [`record = :all`](#record-modes--accepting-changes).
260
+ > It keeps working for the whole 2.x line and warns once per process.
261
+
175
262
  To allow screenshot differences, but still fail on functional errors, you can set the following option:
176
263
 
177
264
  ```ruby
@@ -183,6 +270,11 @@ report while still reporting functional errors.
183
270
 
184
271
  ### Does not tolerate new screenshots
185
272
 
273
+ > **Removed in 2.1, superseded by [`record`](#record-modes--accepting-changes).**
274
+ > `record = :none` is `fail_if_new = true`; `record = :once` is `fail_if_new = false`. Unlike the
275
+ > boolean, a mode means the same thing on CI and off it. It keeps working for the whole 2.x line and
276
+ > warns once per process.
277
+
186
278
  To fail the test if a new screenshot is taken, set the following option:
187
279
 
188
280
  ```ruby
@@ -194,8 +286,17 @@ that does not have a corresponding previous image to compare against.
194
286
  This can be useful in situations where you want to ensure
195
287
  that every screenshot taken by your tests corresponds to an expected state of your application.
196
288
 
289
+ `fail_if_new` defaults to `true` in CI environments (when `ENV['CI']` is set to a non-empty value).
290
+ Setting it yourself outranks the environment: `fail_if_new = false` stays `false` under `CI=true`.
291
+ Assign `nil` to hand it back to the environment. Setting `record` outranks it either way.
292
+
197
293
  ### Marks new screenshots as pending
198
294
 
295
+ > **Removed in 2.1.** It skips the test instead of saying what to do about the missing baseline.
296
+ > [`record = :none`](#record-modes--accepting-changes) fails with the `git add` command attached;
297
+ > `record = :once` records the screenshot and names it in the end-of-run summary. It keeps working
298
+ > for the whole 2.x line and warns once per process.
299
+
199
300
  To mark tests as pending (skipped) if a new screenshot is taken without a baseline, set:
200
301
 
201
302
  ```ruby
@@ -313,6 +414,15 @@ Capybara::Screenshot::Diff.color_distance_limit = 42
313
414
 
314
415
  ### Allowed shift distance
315
416
 
417
+ > **Removed in 2.1.** `shift_distance_limit` is implemented only by the ChunkyPNG driver,
418
+ > and 2.1 removes that driver — libvips becomes the only backend. Setting it anywhere
419
+ > (`SnapDiff.config.shift_distance_limit =`, the legacy
420
+ > `Capybara::Screenshot::Diff.shift_distance_limit =`, or `screenshot 'index',
421
+ > shift_distance_limit: 2`) warns once per process in 2.0. There is no vips equivalent:
422
+ > use `median_filter_window_size` (the faster answer to the same problem — see
423
+ > [Drivers](drivers.md#median-filter-size-vips-only)), `tolerance`, or
424
+ > `color_distance_limit`.
425
+
316
426
  Sometimes you want to allow small movements in the images. For example, jquery-tablesorter
317
427
  renders the same table slightly differently sometimes. You can set set the shift distance
318
428
  threshold for the comparison using the `shift_distance_limit` option to the `screenshot`
@@ -362,8 +472,6 @@ Capybara::Screenshot::Diff.area_size_limit = 42
362
472
  Sometimes you have expected change that you want to ignore.
363
473
  You can use the `skip_area` option with `[left, top, right, bottom]`
364
474
  or css selector like `'#footer'` or `'.container .skipped_element'` to the `screenshot` method to ignore an area.
365
- Be aware that if the selector is not in the page then the library will wait the default wait time for it to appear.
366
- Therefore, it is best to only use css selectors for skip_areas you know will be in the page:
367
475
 
368
476
  ```ruby
369
477
  test 'unstable area' do
@@ -372,6 +480,64 @@ test 'unstable area' do
372
480
  end
373
481
  ```
374
482
 
483
+ **`skip_area` masks what is on the page at assertion time — it does not wait.**
484
+ A selector is resolved against the DOM as it is when the screenshot is taken; one
485
+ that matches nothing masks nothing, immediately. (Until 2.0 it blocked for
486
+ `Capybara.default_max_wait_time` per unmatched selector — 5s each, on every
487
+ screenshot. That wait is gone.)
488
+
489
+ So content that arrives late — lazy-loaded images, JS-injected widgets, anything
490
+ behind an unresolved fetch — has to be settled *before* the assertion, or its
491
+ mask will be empty and the unstable region will be compared. Settle it in the
492
+ readiness block described below.
493
+
494
+ If a selector matched nothing in *every* screenshot of a run, the end-of-run
495
+ summary names it:
496
+
497
+ ```
498
+ [snap_diff] 1 selector never matched anything in this run: "artcile img". A selector that matches nothing masks nothing -- check for a typo or a stale selector.
499
+ ```
500
+
501
+ That is a run-level fact on purpose. Per screenshot the gem cannot tell a typo
502
+ from a page that legitimately has no images, so it says nothing; a selector that
503
+ matched *somewhere* is doing its job and is never mentioned. Nothing is printed
504
+ when every selector matched.
505
+
506
+ ### The readiness block
507
+
508
+ `assert_matches_screenshot` and `capture_screenshot` (and the `screenshot` /
509
+ `assert_no_screenshot_changes` wrappers) take an optional block. It runs once,
510
+ after the enabled check and before the capture:
511
+
512
+ ```ruby
513
+ test 'gallery' do
514
+ visit '/gallery'
515
+ assert_matches_screenshot 'gallery', skip_area: ['article img'] do
516
+ scroll_to :bottom
517
+ assert_text 'End of gallery'
518
+ scroll_to :top
519
+ end
520
+ end
521
+ ```
522
+
523
+ **Why a block rather than the line above it:** work in the block is skipped when
524
+ screenshots are off. Both methods return immediately when
525
+ `SnapDiff.config.enabled` (or `screenshot_enabled`) is false, so a
526
+ `preload_all_images` written on the preceding line still pays for its browser
527
+ round-trips — scroll, wait, scroll back — for a screenshot that is never taken.
528
+ Inside the block it costs nothing, which is what makes turning visual tests off
529
+ actually free.
530
+
531
+ Errors raised in the block are yours and propagate unchanged. It is not a hook:
532
+ there is no configuration-level equivalent, no after-block, and it runs once per
533
+ assertion rather than once per stability retry.
534
+
535
+ In RSpec, call `assert_matches_screenshot` directly rather than through the
536
+ `match_screenshot` matcher — `expect(page).to match_screenshot('x') { ... }`
537
+ binds the block by Ruby's `{}`/`do...end` precedence rather than by intent, so
538
+ the matcher does not take one. In Cucumber the DSL is in the World, so step
539
+ definitions pass a block the same way a Minitest test does.
540
+
375
541
  The arguments are `[left, top, right, bottom]` for the area you want to ignore. You can also set this globally:
376
542
 
377
543
  ```ruby
data/docs/drivers.md CHANGED
@@ -6,6 +6,44 @@
6
6
  > [Custom drivers](snapdiff.md#custom-drivers) for the `SnapDiff::Driver` mixin and how
7
7
  > registration in `SnapDiff::Drivers.loaded` works.
8
8
 
9
+ ## Removed in 2.1: everything on this page except VIPS
10
+
11
+ 2.1 makes **libvips the only backend**. 2.0 is the transitional release — all of the
12
+ following still works. Most of it warns once per process naming 2.1; the rows marked
13
+ *silent* cannot warn, and this table is their notice. Silence the warnings with
14
+ `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`.
15
+
16
+ | Removed in 2.1 | What to do in 2.0 |
17
+ |---|---|
18
+ | the `:chunky_png` driver | add `gem "ruby-vips"` to your Gemfile and drop `driver: :chunky_png` |
19
+ | the `driver:` setting itself — `SnapDiff.config.driver =` and the legacy `Capybara::Screenshot::Diff.driver =` (**silent in 2.0**; on 2.1 they raise `NoMethodError` at config time) | delete the line; one backend needs no selection |
20
+ | the per-screenshot `driver:` override — `screenshot "index", driver: :vips` (**no deprecation warning in 2.0** -- but the key is validated, not inert: an unknown driver raises `RuntimeError: Wrong adapter ...`. 2.1 rejects the key outright) | delete the option, and grep for it — nothing will tell you the line is dead |
21
+ | `driver: :auto` (and the `:auto` default) — warns **only when `:auto` actually falls back to ChunkyPNG**, i.e. when `ruby-vips` is missing; **silent** otherwise | with one backend there is nothing to choose; install `ruby-vips` and the default just works |
22
+ | `shift_distance_limit` | ChunkyPNG-only. Use `median_filter_window_size`, `tolerance` or `color_distance_limit` — see [Configuration](configuration.md#allowed-shift-distance) |
23
+ | `SnapDiff::Driver` (the custom-driver mixin) | nothing — see below |
24
+ | `SnapDiff::Drivers.loaded` (the registry) | nothing — see below |
25
+ | `SnapDiff::Drivers.available` (driver detection) | require `ruby-vips` instead of branching on a detected list |
26
+
27
+ Three related names on the same chopping block stay **silent**, and deliberately so:
28
+ `SnapDiff::Drivers.for` (the gem calls it for every comparison — warning there would fire on
29
+ setups that nothing in this table affects), `SnapDiff::Drivers.detect_available` /
30
+ `SnapDiff::Utils.detect_available_drivers` (run at load, before any user code), and the legacy
31
+ `Capybara::Screenshot::Diff::LOADED_DRIVERS` / `::AVAILABLE_DRIVERS` constant aliases (plain
32
+ constants, nothing to hook). Reach the same values through `.loaded` / `.available` and you
33
+ will hear about them.
34
+
35
+ **libvips becomes a hard requirement.** Install it with your system package manager
36
+ (`brew install vips`, `apt-get install libvips`) and add `gem "ruby-vips"`. A 2.1 process
37
+ without it cannot compare images at all.
38
+
39
+ **Custom drivers: there is no migration path.** The driver abstraction is removed whole —
40
+ the `SnapDiff::Driver` mixin, the `SnapDiff::Drivers.loaded` registry, and driver
41
+ selection by name. Third-party drivers stop working in 2.1 and nothing replaces them
42
+ (the decision was made deliberately: no measurable demand, and one backend is what keeps
43
+ the comparison engine honest). If you maintain one, say so on
44
+ [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1
45
+ ships — that is the only thing that can change this.
46
+
9
47
  ## Perceptual color comparison (VIPS only)
10
48
 
11
49
  By default, color differences are measured using raw RGB channel distance. This can produce
@@ -52,15 +90,20 @@ There are several options to setup active driver: `:auto`, `:chunky_png` and `:v
52
90
  * `:auto` - will try to load `:vips` if there is gem `ruby-vips`, in other cases will load `:chunky_png`
53
91
  * `:chunky_png` and `:vips` will load correspondent driver
54
92
 
93
+ > **2.1 keeps only `:vips`.** `:auto` and `:chunky_png` are removed; `:chunky_png` warns once per
94
+ > process in 2.0. If `:auto` is quietly running you on ChunkyPNG today (no `ruby-vips`
95
+ > installed), the warning says so — that is the setup 2.1 breaks.
96
+
55
97
  ## Enable VIPS image processing
56
98
 
57
99
  [Vips](https://www.rubydoc.info/gems/ruby-vips/Vips/Image) driver provides a faster comparison,
58
- and could be enabled by adding `ruby-vips` to `Gemfile`.
59
-
60
- If need to setup explicitly Vips driver, there are several ways to do this:
100
+ and is enabled by adding `ruby-vips` to your `Gemfile` (plus the libvips system package).
101
+ That is the whole setup — with `ruby-vips` installed the default `:auto` already picks it.
61
102
 
62
- * Globally: `Capybara::Screenshot::Diff.driver = :vips`
63
- * Per screenshot option: `screenshot 'index', driver: :vips`
103
+ **Do not select it explicitly.** Both forms are on the 2.1 removal list at the top of this page:
104
+ `Capybara::Screenshot::Diff.driver = :vips` / `SnapDiff.config.driver = :vips` raises
105
+ `NoMethodError` at config time on 2.1, and the per-screenshot `screenshot 'index', driver: :vips`
106
+ becomes inert. Neither warns in 2.0 — delete the line and grep for the per-screenshot form.
64
107
 
65
108
  With enabled VIPS there are new alternatives to process differences, which are easier to find and support.
66
109
  For example, `shift_distance_limit` is a very heavy operation. Instead, use `median_filter_window_size`.
@@ -26,12 +26,16 @@ For Minitest, need to require `capybara_screenshot_diff/minitest`.
26
26
  In your test class, include the `CapybaraScreenshotDiff::Minitest::Assertions` module:
27
27
 
28
28
  ```ruby
29
+ # test/application_system_test_case.rb
30
+ require 'test_helper'
29
31
  require 'capybara_screenshot_diff/minitest'
30
32
 
31
33
  class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
32
- # Make the Capybara & Capybara Screenshot Diff DSLs available in tests
33
- include CapybaraScreenshotDiff::DSL
34
- # Make `assert_*` methods behave like Minitest assertions
34
+ # Pin the browser: window size and pixel ratio are inputs to every comparison
35
+ driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400]
36
+
37
+ # Make `assert_*` methods behave like Minitest assertions.
38
+ # This already includes CapybaraScreenshotDiff::DSL — a separate include is not needed.
35
39
  include CapybaraScreenshotDiff::Minitest::Assertions
36
40
 
37
41
  def test_my_feature
@@ -41,6 +45,9 @@ class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
41
45
  end
42
46
  ```
43
47
 
48
+ Run it with `bin/rails test:system` — `rake test` / `rails test` skip `test/system/` and
49
+ report `0 runs`.
50
+
44
51
  ## RSpec
45
52
 
46
53
  To use the screenshot capturing and change detection features in your tests,
@@ -35,15 +35,19 @@ end
35
35
 
36
36
  **After (capybara-screenshot-diff):**
37
37
  ```ruby
38
- # Gemfile
39
- gem 'capybara-screenshot-diff'
38
+ # Gemfile — pin the exact prerelease; '~> 2.0' does not resolve until 2.0.0 ships
39
+ gem 'capybara-screenshot-diff', '2.0.0.beta4'
40
40
 
41
41
  # test helper
42
- require 'capybara_screenshot_diff/minitest'
42
+ require "snap_diff/integrations/minitest"
43
+
44
+ # test class — test/application_system_test_case.rb
45
+ require "test_helper"
43
46
 
44
- # test class
45
47
  class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
46
- include CapybaraScreenshotDiff::Minitest::Assertions
48
+ driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400]
49
+
50
+ include SnapDiff::Minitest::Assertions
47
51
 
48
52
  test "homepage" do
49
53
  visit '/'
@@ -60,7 +64,7 @@ end
60
64
  | First run | Uploads to Percy | Saves locally, passes automatically |
61
65
  | CI setup | `PERCY_TOKEN` env var | GitHub Action (3 lines) |
62
66
  | Diff review | Percy dashboard | `snap_diff_report.html` or PR artifacts |
63
- | Update baselines | Percy's "Approve" button | Delete file, re-run tests, commit |
67
+ | Update baselines | Percy's "Approve" button | Re-run tests (the run rewrites the baseline), review, commit — **not** delete: baselines are read from git |
64
68
  | Snapshot limits | Paid plan dependent | Unlimited |
65
69
  | Parallel builds | Built-in | Thread-safe with t-locals + mutex |
66
70
 
@@ -77,7 +81,7 @@ end
77
81
  - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
78
82
  with:
79
83
  ruby-version: '4.0'
80
- - run: bundle exec rake test
84
+ - run: bin/rails test:system # not `rake test` — it skips test/system/
81
85
  - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
82
86
  if: failure()
83
87
  with:
@@ -209,7 +213,7 @@ end
209
213
  **After (capybara-screenshot-diff in CI):**
210
214
  ```yaml
211
215
  - uses: snap-diff/snap_diff-capybara/.github/actions/setup-ruby-and-dependencies@master
212
- - run: bundle exec rake test
216
+ - run: bin/rails test:system # not `rake test` — it skips test/system/
213
217
  - uses: snap-diff/snap_diff-capybara/.github/actions/upload-screenshots@master
214
218
  if: failure()
215
219
  with:
@@ -246,9 +250,13 @@ end
246
250
 
247
251
  ### "My baselines are on Percy/Chromatic servers"
248
252
 
249
- You'll need to take fresh screenshots. Either:
250
- - Visit each page and capture manually
251
- - Run tests with `RECORD_SCREENSHOTS=1` to generate all baselines at once
253
+ You'll need to take fresh screenshots. Run your system tests once — every screenshot
254
+ is written to its baseline path, whether or not it matched — then commit them:
255
+
256
+ ```bash
257
+ bin/rails test:system
258
+ git add doc/screenshots/ && git commit -m "chore: record baselines"
259
+ ```
252
260
 
253
261
  ### "I had hundreds of BackstopJS scenarios"
254
262
 
@@ -262,11 +270,14 @@ screenshot 'step2'
262
270
 
263
271
  ### "My tests are slow now"
264
272
 
265
- Use the VIPS driver for ~50ms comparisons per image:
273
+ Use libvips for ~50ms comparisons per image — installing the gem is the whole setup, and from
274
+ 2.1 it is the only backend:
266
275
  ```ruby
276
+ # Gemfile
267
277
  gem 'ruby-vips'
268
- Capybara::Screenshot::Diff.driver = :vips
269
278
  ```
279
+ Do **not** add `driver = :vips`: that setting is removed in 2.1 and raises `NoMethodError`
280
+ there, silently, with nothing in 2.0 to warn you. See [Drivers](drivers.md).
270
281
 
271
282
  ### "The diffs look different from what I'm used to"
272
283
 
data/docs/reporters.md CHANGED
@@ -18,7 +18,76 @@ open doc/screenshots/snap_diff_report.html
18
18
 
19
19
  The report includes a sidebar with thumbnails, side-by-side comparison with diff toggle, search, and summary stats. No configuration needed — just require it.
20
20
 
21
- **Note:** The report is not generated when all screenshots match. In parallel test environments, each worker writes to the same file — the last worker's results will be in the report.
21
+ **Note:** The report is not generated when all screenshots match.
22
+
23
+ ## The end-of-run summary
24
+
25
+ Every run ends with what it actually did, whether or not you require a reporter:
26
+
27
+ ```
28
+ [snap_diff] 12 verified, 1 changed, 2 new (not verified).
29
+ ```
30
+
31
+ - **verified** — a committed baseline existed and was compared
32
+ - **changed** — of those, the ones that differed
33
+ - **new** — captured but *not* compared, for want of a committed baseline: neither a pass nor a
34
+ failure. Commit the files to turn them into baselines.
35
+
36
+ `0 verified` is printed as `NOTHING WAS VERIFIED`, because it is the only signal for the failures
37
+ no per-assertion rule can see: a `rake test` that ran zero system tests, or an inherited `GIT_DIR`
38
+ sending every baseline lookup to the wrong repository. Both leave a green suite that compared
39
+ nothing.
40
+
41
+ Requiring `snap_diff/reporters/html` adds one more line, naming the file it wrote:
42
+
43
+ ```
44
+ [snap_diff] Report: doc/screenshots/snap_diff_report.html
45
+ ```
46
+
47
+ ## Parallel test runs
48
+
49
+ `finalize` — the hook that writes the report — runs from the framework's end-of-suite hook. Whether
50
+ that hook fires in the process holding the results depends on how your runner parallelizes:
51
+
52
+ | How the suite runs | Report |
53
+ | --- | --- |
54
+ | Serial | Written, complete. |
55
+ | `parallelize(with: :threads)` (also the default on JRuby) | Written, complete — same failures and counts as a serial run; only the order of the entries differs. Verified over repeated runs, provided [screenshot names are unique across tests](thread_safety.md#screenshot-names-must-be-unique-across-tests). |
56
+ | `parallelize(workers: N)` (Rails' default, forks) | Written, complete — one report at the usual path, merged from every worker. No configuration needed. |
57
+ | One process per worker (`parallel_tests`, RSpec, CI sharding) | Written, but only the **last** process to finish is in it; the others are overwritten. |
58
+
59
+ Under Rails' forking parallelism the workers hold the results but never finalize: Minitest skips its
60
+ `after_run` hooks in a forked child (`Minitest.allow_fork` defaults to `false`), and the parent
61
+ process — the one that does finalize — recorded nothing. So each worker now hands its records to the
62
+ parent on the way out (Rails' `run_cleanup` hook runs *inside* the worker), and the parent merges
63
+ them before it writes the report. `N verified, N changed, N new` are the merged totals for the whole
64
+ suite, not one worker's.
65
+
66
+ The handoff goes through a scratch directory under the system temp dir, removed as soon as the merge
67
+ is done. Nothing is written inside your repository, and a worker killed mid-write leaves nothing the
68
+ merge will read.
69
+
70
+ ### Rails versions and older workarounds
71
+
72
+ The merge uses `ActiveSupport::Testing::Parallelization`'s worker hooks and is registered only when
73
+ Rails is present, so a non-Rails Capybara suite is unaffected.
74
+
75
+ If your `test_helper.rb` still carries the old per-worker workaround —
76
+
77
+ ```ruby
78
+ parallelize_teardown { SnapDiff::Reporting.finalize! } # no longer needed
79
+ ```
80
+
81
+ — you can delete it. Left in place it does not corrupt anything: the merged report is written last,
82
+ at the documented path, with every failure in it. It just adds noise — each worker writes its own
83
+ partial report over the same file first, and prints its own partial summary line (`32 verified …`
84
+ four times, then the real one).
85
+
86
+ Do **not** separate workers with a per-worker `save_path`: `save_path` is also where baselines are
87
+ read from (`SnapDiff.config.screenshot_area`), so changing it per worker points the comparison at an
88
+ empty baseline directory and every screenshot is recorded as new.
89
+
90
+ See [Thread safety](thread_safety.md) for the state each mode shares.
22
91
 
23
92
  ## Custom Reporters
24
93
 
data/docs/snapdiff.md CHANGED
@@ -4,9 +4,13 @@ Everything in this gem lives under `SnapDiff` since v2. This page is the SnapDif
4
4
  reference: setup, configuration, the object map, and the extension points — all using canonical
5
5
  names only.
6
6
 
7
- The legacy `Capybara::Screenshot::Diff` / `CapybaraScreenshotDiff` names still work (they resolve
8
- to the same objects, with a one-time deprecation warning per constant), and the rest of the docs
9
- still teach them. Nothing here replaces a working setup it is what you write for **new** code.
7
+ The legacy `Capybara::Screenshot::Diff` / `CapybaraScreenshotDiff` names still work they resolve
8
+ to the same objects and the rest of the docs still teach them. A legacy config accessor,
9
+ `include`, `default_options` call, or lazily shimmed constant prints one migration notice per
10
+ process; *lazily shimmed* constants warn once each on top of that. Other legacy names — the
11
+ integration requires and `CapybaraScreenshotDiff::Minitest::Assertions` among them — are silent
12
+ by design, so a quiet suite is not a migrated one.
13
+ [UPGRADING.md](UPGRADING.md#deprecation-warnings) lists exactly which is which. Nothing here replaces a working setup — it is what you write for **new** code.
10
14
  For migrating an existing suite, see [UPGRADING.md](UPGRADING.md).
11
15
 
12
16
  ## Quick start
@@ -20,12 +24,20 @@ require "snap_diff/integrations/minitest"
20
24
 
21
25
  ```ruby
22
26
  # test/application_system_test_case.rb
27
+ require "test_helper"
28
+
23
29
  class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
30
+ # Window size and pixel ratio are inputs to every comparison — pin them.
31
+ driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400]
32
+
24
33
  include SnapDiff::Minitest::Assertions # brings in SnapDiff::DSL too
25
34
  end
26
35
  ```
27
36
 
28
37
  ```ruby
38
+ # test/system/homepage_test.rb
39
+ require "application_system_test_case"
40
+
29
41
  class HomepageTest < ApplicationSystemTestCase
30
42
  test "homepage" do
31
43
  visit "/"
@@ -34,6 +46,9 @@ class HomepageTest < ApplicationSystemTestCase
34
46
  end
35
47
  ```
36
48
 
49
+ Run it with `bin/rails test:system`. `rake test` and `rails test` skip `test/system/`
50
+ entirely — `0 runs` and no baselines, which reads exactly like a pass.
51
+
37
52
  `SnapDiff::Minitest::Assertions` already includes `SnapDiff::DSL`, so a separate
38
53
  `include SnapDiff::DSL` is not needed (it is harmless if you have it).
39
54
 
@@ -95,14 +110,13 @@ loads the Minitest integration. See
95
110
 
96
111
  ## Configuration
97
112
 
98
- All 27 settings live on one flat object, `SnapDiff.config` (a `SnapDiff::Config`).
113
+ All 28 settings live on one flat object, `SnapDiff.config` (a `SnapDiff::Config`).
99
114
 
100
115
  ```ruby
101
116
  # test_helper.rb / rails_helper.rb
102
117
  SnapDiff.configure do |config|
103
118
  config.window_size = [1280, 1024]
104
119
  config.tolerance = 0.0005
105
- config.driver = :vips
106
120
  config.save_path = "doc/screenshots"
107
121
  end
108
122
 
@@ -159,12 +173,12 @@ integration require; a few objects need their own require, noted below.
159
173
  | `SnapDiff::Region` | Bounding box value object — `from_edge_coordinates`, `to_edge_coordinates` |
160
174
  | `SnapDiff::DSL` | `screenshot`, `assert_matches_screenshot`, `capture_screenshot`, groups/sections |
161
175
  | `SnapDiff::Minitest::Assertions` | Minitest wiring (`snap_diff/integrations/minitest`) |
162
- | `SnapDiff::Error` | Base class for every error this gem raises |
176
+ | `SnapDiff::Error` | Base class for every error this gem *defines* — see the caveat below. Misuse still surfaces as plain Ruby: bad arguments raise `ArgumentError`, and a missing image backend raises `RuntimeError` |
163
177
  | `SnapDiff::ExpectationNotMet` | A screenshot did not match its baseline |
164
178
  | `SnapDiff::UnstableImage` | No stable capture within `stability_time_limit` / `wait` |
165
179
  | `SnapDiff::WindowSizeMismatchError` | Browser window is not the configured `window_size` |
166
- | `SnapDiff::Driver` | Mixin with the shared driver defaults (`require "snap_diff/driver"`) |
167
- | `SnapDiff::Drivers` | Driver factory and registry — `.for`, `.loaded`, `.available` |
180
+ | `SnapDiff::Driver` | Mixin with the shared driver defaults (`require "snap_diff/driver"`) — **removed in 2.1** |
181
+ | `SnapDiff::Drivers` | Driver factory and registry — `.for`, `.loaded`, `.available` — **removed in 2.1** |
168
182
  | `SnapDiff::Reporting` | Process-global reporter lifecycle (`require "snap_diff/reporting"`) |
169
183
  | `SnapDiff::Reporters::HTML` | The interactive HTML report (`require "snap_diff/reporters/html"`) |
170
184
  | `SnapDiff::Reporters::Default` | Builds the annotated diff images and the failure message |
@@ -174,6 +188,17 @@ integration require; a few objects need their own require, noted below.
174
188
  | `SnapDiff::Capture::Viewport` | Per-capture viewport check seam (`require "snap_diff/capture/viewport"`) |
175
189
  | `SnapDiff.serve` | Point Capybara at a static site directory (`require "snap_diff/static"`) |
176
190
 
191
+ > **`rescue SnapDiff::Error` does not catch a failed assertion under the framework
192
+ > integrations.** `SnapDiff::ExpectationNotMet` — the row you most want to rescue — is
193
+ > converted before it reaches your code: the Minitest integration re-raises it as
194
+ > `Minitest::Assertion` ([`integrations/minitest.rb`](../lib/snap_diff/integrations/minitest.rb)),
195
+ > and the RSpec integration as `RSpec::Expectations::ExpectationNotMetError`
196
+ > ([`integrations/rspec.rb`](../lib/snap_diff/integrations/rspec.rb)). That conversion is the
197
+ > point — it is what makes a mismatch a test failure rather than an error. You see a raw
198
+ > `SnapDiff::ExpectationNotMet` only when you drive `SnapDiff::DSL` yourself, with a bare
199
+ > `include SnapDiff::DSL` and no integration. `UnstableImage` and `WindowSizeMismatchError`
200
+ > are not converted and do arrive as `SnapDiff::Error`.
201
+
177
202
  ## Compare two images without a browser
178
203
 
179
204
  Works on anything on disk — rendered PDFs, generated charts, CI artifacts:
@@ -262,6 +287,14 @@ SnapDiff::Reporting.finalize!
262
287
 
263
288
  ## Custom drivers
264
289
 
290
+ > **Removed in 2.1 — no replacement.** libvips becomes the only backend, and the driver
291
+ > abstraction goes with the choice: the `SnapDiff::Driver` mixin, the
292
+ > `SnapDiff::Drivers.loaded` registry, `SnapDiff::Drivers.available`, and selecting a driver
293
+ > by name. In 2.0 all of it still works and warns once per process (silence with
294
+ > `SnapDiff.silence_deprecations = true` or `SNAP_DIFF_SILENCE_DEPRECATIONS=1`). Nothing
295
+ > here migrates to a 2.1 shape — there is no 2.1 shape. If you maintain a driver, say so on
296
+ > [the issue tracker](https://github.com/snap-diff/snap_diff-capybara/issues) before 2.1 ships.
297
+
265
298
  A driver is a plain object that does the image work. Include `SnapDiff::Driver` for the shared
266
299
  defaults, then implement the operations the comparison engine calls:
267
300