snap_diff-capybara 2.0.0.beta3 → 2.0.0.beta4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +438 -0
- data/README.md +330 -0
- data/docs/UPGRADING.md +232 -34
- data/docs/architecture.md +9 -7
- data/docs/ci-integration.md +42 -11
- data/docs/configuration.md +176 -10
- data/docs/drivers.md +48 -5
- data/docs/framework-setup.md +10 -3
- data/docs/migration-guide.md +24 -13
- data/docs/reporters.md +70 -1
- data/docs/snapdiff.md +41 -8
- data/docs/thread_safety.md +77 -86
- data/lib/capybara/screenshot/diff/annotation_service.rb +1 -3
- data/lib/capybara/screenshot/diff/area_calculator.rb +1 -3
- data/lib/capybara/screenshot/diff/browser_helpers.rb +1 -3
- data/lib/capybara/screenshot/diff/config_legacy.rb +15 -60
- data/lib/capybara/screenshot/diff/difference.rb +1 -4
- data/lib/capybara/screenshot/diff/drivers/chunky_png_driver.rb +3 -3
- data/lib/capybara/screenshot/diff/drivers/vips_driver.rb +2 -3
- data/lib/capybara/screenshot/diff/drivers.rb +4 -5
- data/lib/capybara/screenshot/diff/image_compare.rb +11 -22
- data/lib/capybara/screenshot/diff/image_preprocessor.rb +1 -3
- data/lib/capybara/screenshot/diff/os.rb +4 -11
- data/lib/capybara/screenshot/diff/region.rb +2 -2
- data/lib/capybara/screenshot/diff/reporters/default.rb +4 -17
- data/lib/capybara/screenshot/diff/screenshot_matcher.rb +1 -3
- data/lib/capybara/screenshot/diff/screenshoter.rb +1 -3
- data/lib/capybara/screenshot/diff/stable_screenshoter.rb +1 -3
- data/lib/capybara/screenshot/diff/utils.rb +1 -3
- data/lib/capybara/screenshot/diff/vcs.rb +1 -3
- data/lib/capybara/screenshot/diff/version.rb +8 -13
- data/lib/capybara-screenshot-diff.rb +10 -1
- data/lib/capybara_screenshot_diff/attempts_reporter.rb +1 -3
- data/lib/capybara_screenshot_diff/dsl.rb +5 -0
- data/lib/capybara_screenshot_diff/error_with_filtered_backtrace.rb +1 -4
- data/lib/capybara_screenshot_diff/reporters/html.rb +1 -3
- data/lib/capybara_screenshot_diff/screenshot_namer.rb +1 -3
- data/lib/capybara_screenshot_diff/snap.rb +1 -3
- data/lib/capybara_screenshot_diff/snap_manager.rb +1 -3
- data/lib/capybara_screenshot_diff.rb +17 -21
- data/lib/snap_diff/browser_helpers.rb +26 -5
- data/lib/snap_diff/capture/viewport.rb +2 -5
- data/lib/snap_diff/comparison.rb +54 -3
- data/lib/snap_diff/comparison_result.rb +3 -1
- data/lib/snap_diff/config.rb +189 -93
- data/lib/snap_diff/deprecation.rb +89 -28
- data/lib/snap_diff/driver.rb +18 -0
- data/lib/snap_diff/drivers/vips_driver.rb +12 -6
- data/lib/snap_diff/drivers.rb +94 -12
- data/lib/snap_diff/dsl.rb +45 -47
- data/lib/snap_diff/integrations/cucumber.rb +1 -1
- data/lib/snap_diff/integrations/minitest.rb +45 -9
- data/lib/snap_diff/integrations/rspec.rb +11 -0
- data/lib/snap_diff/legacy_shims.rb +285 -27
- data/lib/snap_diff/removal.rb +159 -0
- data/lib/snap_diff/reporters/default.rb +86 -23
- data/lib/snap_diff/reporters/html.rb +29 -13
- data/lib/snap_diff/reporting.rb +329 -3
- data/lib/snap_diff/screenshot_assertion.rb +28 -23
- data/lib/snap_diff/screenshot_matcher.rb +121 -14
- data/lib/snap_diff/screenshot_namer.rb +1 -19
- data/lib/snap_diff/screenshoter.rb +5 -7
- data/lib/snap_diff/snap.rb +6 -1
- data/lib/snap_diff/snap_manager.rb +2 -3
- data/lib/snap_diff/stable_screenshoter.rb +2 -2
- data/lib/snap_diff/static.rb +1 -1
- data/lib/snap_diff/utils.rb +35 -17
- data/lib/snap_diff/vcs.rb +40 -7
- data/lib/snap_diff/version.rb +1 -1
- data/lib/snap_diff-capybara.rb +33 -3
- data/lib/snap_diff.rb +27 -37
- metadata +12 -10
- data/CODE_OF_CONDUCT.md +0 -129
- data/Rakefile +0 -65
- data/capybara-screenshot-diff.gemspec +0 -29
- data/docs/RELEASE_PREP.md +0 -44
- data/docs/docker-testing.md +0 -24
- data/gems.rb +0 -39
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4d0039a411b2d60cbcaf7ebe00ecd935b28791abab87fbe2f32cfec4d710a366
|
|
4
|
+
data.tar.gz: 57b5491a9eb1e5bd882f1ff5fad243abf4f4f1e76c7995a605d05a9d41419f07
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4f5156c0e65942d66453cd31dea66cc45c566be9cf5b72fb0fe87525cc5c83216a60ce6078562f4044a9b660adced5b4527f852e2723b99b97b18c38634c574b
|
|
7
|
+
data.tar.gz: d42ddb6f755fd5d423596da137f24414c3f52344ba854e6cbd33744b1b9bc3abc7f9c63a938cd20f9e3a124bdb48a41a025353c5953a207401e2eabbbcd3101b
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,444 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [v2.0.0] - unreleased
|
|
9
|
+
|
|
10
|
+
**The transitional release.** Everything you run today keeps working. 2.0 makes
|
|
11
|
+
`SnapDiff` the canonical namespace, keeps the entire v1 API alongside it as
|
|
12
|
+
same-object aliases, and warns — once per process — about what **2.1** removes.
|
|
13
|
+
There is no 3.0; 2.1 is the cleanup.
|
|
14
|
+
|
|
15
|
+
The rename is not the reason to upgrade. **Four separate bugs let a 1.15.1 suite pass
|
|
16
|
+
green while comparing nothing**, and all four are fixed here — that is the reason.
|
|
17
|
+
|
|
18
|
+
The sections below the divider are the prerelease notes (alpha1 → beta3) and are
|
|
19
|
+
kept as history. This entry is the one to read if you are coming from **1.15.1**.
|
|
20
|
+
|
|
21
|
+
### Upgrading from 1.15.1: change the version, run your suite
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
gem "capybara-screenshot-diff", "2.0.0.beta4" # current 2.0 prerelease; 2.0.0 final is not out yet
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Pin the exact prerelease until 2.0.0 ships: Bundler never resolves a prerelease from a plain
|
|
28
|
+
requirement, so `"~> 2.0"` fails with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'`.
|
|
29
|
+
From 2.0.0 on, `"~> 2.0"` is the pin.
|
|
30
|
+
|
|
31
|
+
That is the whole migration. `screenshot`, `assert_matches_screenshot` and
|
|
32
|
+
`capture_screenshot` are unchanged. Your `Capybara::Screenshot::Diff.configure`
|
|
33
|
+
block, every `Capybara::Screenshot.*` / `Capybara::Screenshot::Diff.*` setting, and
|
|
34
|
+
every legacy constant still resolve — to the *same objects* the new names resolve
|
|
35
|
+
to. Baselines are unchanged: capture, encoding, file naming and the `png` default are
|
|
36
|
+
the same code as 1.15.1, moved — upgrading does not re-encode or invalidate a baseline you
|
|
37
|
+
already committed, and a matching screenshot stays byte-identical. (Unchanged from 1.x: a
|
|
38
|
+
screenshot that *differs* is written to its baseline path, so a failing run leaves
|
|
39
|
+
`doc/screenshots/` dirty. That is how you accept a change — review the diff and commit.)
|
|
40
|
+
|
|
41
|
+
Rolling back is a Gemfile edit: pin `"~> 1.15"` and `bundle update`.
|
|
42
|
+
|
|
43
|
+
### Why upgrade: 1.15.1 had four ways to pass green while testing nothing
|
|
44
|
+
|
|
45
|
+
Each of these was found and fixed for 2.0. They share a shape — the suite reports
|
|
46
|
+
success, the screenshots are never compared, and nothing in the output says so.
|
|
47
|
+
|
|
48
|
+
1. **A screenshot with no *committed* baseline was not compared at all.** Baselines
|
|
49
|
+
are read from git (`git show HEAD:<path>`), never from disk, and `fail_if_new` is
|
|
50
|
+
false off CI by design — so a new screenshot registered no assertion, passed
|
|
51
|
+
whatever the page looked like, and silently overwrote the PNG on disk. Record a
|
|
52
|
+
page, change it, re-run: `1 runs, 1 assertions, 0 failures`. 2.0 warns once per
|
|
53
|
+
screenshot, names the file, and counts it in the summary line as `new (not
|
|
54
|
+
verified)`
|
|
55
|
+
2. **An inherited `GIT_DIR` redirected every baseline lookup.** `git -C <dir>` sets
|
|
56
|
+
the working directory, but `GIT_DIR` / `GIT_WORK_TREE` override it — and every git
|
|
57
|
+
hook exports both. Every lookup then missed, every screenshot recorded as new, and
|
|
58
|
+
the whole suite passed. Measured on the same changed page: 1.15.1 under a foreign
|
|
59
|
+
`GIT_DIR` reports `1 runs, 1 assertions, 0 failures`; 2.0 reports the failure. 2.0
|
|
60
|
+
scrubs the three inherited git variables
|
|
61
|
+
3. **libvips served stale pixels when a path was rewritten inside the same second.**
|
|
62
|
+
The loader cache is keyed on filename + mtime, and mtime has one-second resolution —
|
|
63
|
+
so a screenshot overwritten and re-read in the same second compared against an image
|
|
64
|
+
that was no longer on disk. 2.0 passes `revalidate: true` at the single load site.
|
|
65
|
+
That keyword is libvips 8.15+, so the fix is guarded on it — **on an older libvips
|
|
66
|
+
the bug is still there**, and upgrading libvips is the only cure
|
|
67
|
+
4. **Under Rails' `parallelize(workers: N)` the HTML report was never written.**
|
|
68
|
+
Minitest skips `after_run` in a forked child, so the workers that hold every record
|
|
69
|
+
never finalized and the parent that finalized recorded nothing. Pass/fail was
|
|
70
|
+
unaffected, which is what made it quiet: a suite lost its report the day someone
|
|
71
|
+
added the 51st test (`ActiveSupport.test_parallelization_threshold` defaults to 50).
|
|
72
|
+
2.0 has each worker hand its records to the parent, which merges them into one
|
|
73
|
+
report at the documented path — no application-side configuration
|
|
74
|
+
|
|
75
|
+
And one message that could not be followed: in CI — the one place `fail_if_new` is on
|
|
76
|
+
by default — the missing-baseline error named a path that had not been written, because
|
|
77
|
+
the raise happened *before* the capture. 2.0 raises after, so `git add <path>` is a
|
|
78
|
+
command you can run on the run that printed it, and a failing CI job leaves the new
|
|
79
|
+
screenshot behind for an artifact upload.
|
|
80
|
+
|
|
81
|
+
Relatedly, `fail_if_new` now lets an explicit setting outrank the environment sniff.
|
|
82
|
+
1.x stored `!ENV["CI"].nil?` at require time, so "the user asked for false" and "CI was
|
|
83
|
+
absent when the gem loaded" were the same `false`. The default is unchanged — failing
|
|
84
|
+
only under CI — but `SnapDiff.config.fail_if_new = false` is now honoured under `CI=true`,
|
|
85
|
+
and assigning `nil` hands the setting back to the environment.
|
|
86
|
+
|
|
87
|
+
### What you get
|
|
88
|
+
|
|
89
|
+
- **A summary line on every run, passing or failing** — printed the way Minitest prints
|
|
90
|
+
its seed, because the information is worthless if it only appears once you already
|
|
91
|
+
know you need it:
|
|
92
|
+
|
|
93
|
+
```text
|
|
94
|
+
[snap_diff] 14 verified, 0 changed, 1 new (not verified). Report: /abs/path.html
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`verified` — a committed baseline existed and was compared. `changed` — of those, the
|
|
98
|
+
ones that differed. `new` — captured but *not* compared: neither a pass nor a failure.
|
|
99
|
+
1.x printed `N screenshots compared, no failures`, which was silent about exactly the
|
|
100
|
+
screenshots it did not compare, and printed nothing at all when it compared nothing.
|
|
101
|
+
When nothing was verified the line says so in as many words. Observability only: no
|
|
102
|
+
exit code and no pass/fail behaviour changed. **The line comes from the HTML reporter**,
|
|
103
|
+
so it appears once you `require "snap_diff/reporters/html"` — the same one-line opt-in
|
|
104
|
+
that produces the report
|
|
105
|
+
|
|
106
|
+
- **A failure message you can act on.** 1.15.1 printed a JSON blob and four unlabelled
|
|
107
|
+
absolute paths — and the one file a reader wants most, the committed baseline, was not
|
|
108
|
+
among them:
|
|
109
|
+
|
|
110
|
+
```text
|
|
111
|
+
Screenshot does not match for 'home': ({"area_size":20000.0,"region":[40.0,40.0,240.0,140.0],"diff_mask":"#<Vips::Image:0x…>"})
|
|
112
|
+
/abs/path/doc/screenshots/home.png
|
|
113
|
+
/abs/path/doc/screenshots/home.base.diff.png
|
|
114
|
+
/abs/path/doc/screenshots/home.diff.png
|
|
115
|
+
/abs/path/doc/screenshots/home.heatmap.diff.png
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
2.0 prints the same comparison as:
|
|
119
|
+
|
|
120
|
+
```text
|
|
121
|
+
Screenshot does not match for 'home': the change spans 20000 of 365600 px (5.47% of the 800x457 image)
|
|
122
|
+
changed region: [40.0,40.0,240.0,140.0] (left,top,right,bottom edges)
|
|
123
|
+
judged against: no tolerance thresholds configured (any difference fails)
|
|
124
|
+
baseline: doc/screenshots/home.base.png
|
|
125
|
+
actual: doc/screenshots/home.png
|
|
126
|
+
baseline annotated: doc/screenshots/home.base.diff.png
|
|
127
|
+
actual annotated: doc/screenshots/home.diff.png
|
|
128
|
+
heatmap: doc/screenshots/home.heatmap.diff.png
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
Baseline first, every artifact labelled, paths relative to `SnapDiff.config.root`, a
|
|
132
|
+
denominator on the pixel count, and a `judged against:` line restating the thresholds
|
|
133
|
+
actually applied. A label is printed only when that file is on disk
|
|
134
|
+
|
|
135
|
+
- **Faster, per matching assertion.** `Comparison#different?` skipped step 1 of the
|
|
136
|
+
layered strategy its own documentation describes: a screenshot byte-identical to its
|
|
137
|
+
baseline — the normal outcome of a passing test — still had both PNGs decoded and
|
|
138
|
+
compared pixel by pixel. And `Vcs.checkout_vcs` spawned `git rev-parse --show-toplevel`
|
|
139
|
+
once per screenshot, 200 processes answering the same question in a 200-screenshot
|
|
140
|
+
suite. Both measured in #250 against the code as it stood before those two commits —
|
|
141
|
+
the same code path 1.15.1 runs — with the browser faked out so only the gem's own code
|
|
142
|
+
is timed:
|
|
143
|
+
|
|
144
|
+
| | before | after |
|
|
145
|
+
|---|---|---|
|
|
146
|
+
| `Comparison#different?`, byte-identical 1440x900 | 11.56 ms | 0.16 ms |
|
|
147
|
+
| `Comparison#different?`, byte-identical 2880x1800 | 25.40 ms | 0.31 ms |
|
|
148
|
+
| `Vcs.checkout_vcs`, per screenshot | 14.27 ms | 9.12 ms |
|
|
149
|
+
| one 1440x900 `assert_matches_screenshot` against a matching baseline, end to end | 63.5 ms | 44.5 ms |
|
|
150
|
+
|
|
151
|
+
The cost is one extra `stat` + `FileUtils.compare_file` (0.16–0.28 ms) on the path
|
|
152
|
+
where screenshots genuinely differ. The repo-root answer is memoised under a lock, so
|
|
153
|
+
threaded suites — the default parallel mode on JRuby — share one spawn instead of one
|
|
154
|
+
per thread (#253)
|
|
155
|
+
|
|
156
|
+
- **The gem loads in a bundle without Rails, and in a bundle without Minitest.** See
|
|
157
|
+
*Changed* below; both were load-time crashes in 1.15.1
|
|
158
|
+
|
|
159
|
+
- **Accepting an intentional change is one setting, not folklore.** `record` replaces
|
|
160
|
+
`fail_if_new` (removed in 2.1): `:once` records a missing baseline and compares the rest —
|
|
161
|
+
what a local run already did; `:none` **fails** on a missing baseline with the `git add`
|
|
162
|
+
command attached, and is what CI gets by default; `:all` re-records every screenshot and
|
|
163
|
+
compares nothing, the bulk-accept verb for the run after an intentional redesign. 1.15.1
|
|
164
|
+
documented `RECORD_SCREENSHOTS=1` in four files and printed it in its own error message,
|
|
165
|
+
and nothing in the gem ever read that variable — accepting a change was answered in issue
|
|
166
|
+
comments for six years and never in the docs. See `docs/configuration.md` (#259)
|
|
167
|
+
|
|
168
|
+
### What you will see in your test output
|
|
169
|
+
|
|
170
|
+
One migration notice, once per process, the first time the process goes through any v1
|
|
171
|
+
door. **Requiring a v1-named file is itself one of those doors** — including the gem name
|
|
172
|
+
`capybara-screenshot-diff`, which `Bundler.require` (the Rails default) requires for you.
|
|
173
|
+
So a suite that is otherwise fully migrated still gets this one line while it stays on the
|
|
174
|
+
v1 gem name:
|
|
175
|
+
|
|
176
|
+
```text
|
|
177
|
+
[snap_diff deprecation] This process uses the v1 `Capybara::Screenshot*` /
|
|
178
|
+
`CapybaraScreenshotDiff*` API. It still works in 2.0 and is REMOVED in 2.1 --
|
|
179
|
+
see docs/UPGRADING.md for the SnapDiff replacements. Silence with
|
|
180
|
+
`SnapDiff.silence_deprecations = true` or SNAP_DIFF_SILENCE_DEPRECATIONS=1.
|
|
181
|
+
(shown once per process)
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
If you `require "capybara/screenshot/diff"`, one more — the 1.x auto-activation notice,
|
|
185
|
+
reworded to name the canonical require (1.15.1 printed its own version of this):
|
|
186
|
+
|
|
187
|
+
```text
|
|
188
|
+
[DEPRECATION] `require "capybara/screenshot/diff"` activates the Minitest assertions
|
|
189
|
+
for you; that will be removed.
|
|
190
|
+
Please `require "snap_diff/integrations/minitest"` explicitly.
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Plus one line per *lazily shimmed* legacy constant you reference, naming your call
|
|
194
|
+
site:
|
|
195
|
+
|
|
196
|
+
```text
|
|
197
|
+
[snap_diff deprecation] `Capybara::Screenshot::Diff::ImageCompare` is deprecated
|
|
198
|
+
(constant); use `SnapDiff::Comparison` instead. (called from test/test_helper.rb:12)
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Plus one line per subject 2.1 removes: the `driver` setting and the per-screenshot
|
|
202
|
+
`driver:` option (value-blind — `driver: :vips` warns too, because the *knob* is what
|
|
203
|
+
goes), `shift_distance_limit`, `SnapDiff::Drivers.loaded` / `.available`, and
|
|
204
|
+
`include SnapDiff::Driver`. And one per unrecognised per-screenshot option key: the
|
|
205
|
+
options hash was frozen but never validated, so `tolerence:` bought a green suite that
|
|
206
|
+
compared nothing. 2.0 warns, 2.1 raises `ArgumentError`.
|
|
207
|
+
|
|
208
|
+
`2.0.0.beta3` shipped this channel incomplete — a v1-only suite produced **zero**
|
|
209
|
+
deprecation output while the docs described three channels — because every existing door
|
|
210
|
+
required the user to *call* something, and a suite that requires the gem and only calls
|
|
211
|
+
`screenshot` calls none of them. That is what the require-door notice above fixes.
|
|
212
|
+
|
|
213
|
+
The DSL (`screenshot`, `assert_matches_screenshot`, `capture_screenshot`), the settings
|
|
214
|
+
accessors, and the eagerly-defined constants — `Capybara::Screenshot::Os`, the top-level
|
|
215
|
+
`Region`, `Capybara::Screenshot::Diff::VERSION` / `::Comparison` / `::Reporters::Default` /
|
|
216
|
+
`::LOADED_DRIVERS` / `::AVAILABLE_DRIVERS`, the `CapybaraScreenshotDiff` error classes, and
|
|
217
|
+
`CapybaraScreenshotDiff::DSL` / `::Minitest::Assertions` — are **silent by design**: they
|
|
218
|
+
are real constants precisely so `const_defined?` and `rescue` keep working, which leaves
|
|
219
|
+
nothing to hook. They are removed in 2.1 all the same, so **do not use warning output as a
|
|
220
|
+
migration checklist** — use [docs/UPGRADING.md](docs/UPGRADING.md#deprecation-warnings),
|
|
221
|
+
which lists exactly which names warn and which cannot.
|
|
222
|
+
|
|
223
|
+
Silence everything with `SnapDiff.silence_deprecations = true` or
|
|
224
|
+
`SNAP_DIFF_SILENCE_DEPRECATIONS=1`. One honest limitation: the accessor cannot silence the
|
|
225
|
+
require-time notice, because setting it needs the require to have happened. Under
|
|
226
|
+
`Bundler.require` the environment variable is the only channel.
|
|
227
|
+
|
|
228
|
+
### The five things that can actually break
|
|
229
|
+
|
|
230
|
+
Everything else is source-compatible. These are not:
|
|
231
|
+
|
|
232
|
+
1. **Error class names printed in output are now `SnapDiff::…`.** The class objects
|
|
233
|
+
are identical, so `rescue CapybaraScreenshotDiff::ExpectationNotMet` still
|
|
234
|
+
catches them — but a CI job that greps the *old* class name out of test output
|
|
235
|
+
needs updating.
|
|
236
|
+
2. **`defined?` / `const_defined?` on lazily shimmed legacy names returns
|
|
237
|
+
`nil`/`false`** — they resolve through `const_missing`, which those checks never
|
|
238
|
+
trigger, so `const_get` and a bare reference still work while `defined?` reports
|
|
239
|
+
nothing. `Capybara::Screenshot::Diff::ImageCompare` and
|
|
240
|
+
`Capybara::Screenshot::Diff::Drivers` are two you may be probing today. Move
|
|
241
|
+
feature detection to the `SnapDiff::` name. The names in the silent-by-design list
|
|
242
|
+
above are real constants and are unaffected — but note the two whose spelling is
|
|
243
|
+
easy to get wrong: it is `Capybara::Screenshot::Os` (not under `::Diff`) and the
|
|
244
|
+
**top-level** `Region` (as in 1.x). `defined?(Capybara::Screenshot::Diff::Os)` and
|
|
245
|
+
`defined?(Capybara::Screenshot::Diff::Region)` are `nil` — those names do not exist,
|
|
246
|
+
and did not in 1.15.1 either.
|
|
247
|
+
3. **Reopening `module Capybara::Screenshot::Diff::Drivers`** (the historical
|
|
248
|
+
custom-driver monkey-patch) defines a fresh, empty module that shadows the shim.
|
|
249
|
+
Define custom drivers under `SnapDiff::Drivers` instead — and `BaseDriver` is a
|
|
250
|
+
mixin now: `class MyDriver < BaseDriver` becomes `include SnapDiff::Driver`.
|
|
251
|
+
4. **Stubbing `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` no longer works.** It
|
|
252
|
+
is an eager alias; the gem reads `SnapDiff::Drivers::AVAILABLE_DRIVERS`. Stubbing
|
|
253
|
+
the legacy name rebinds the alias only, so the test passes for the wrong reason.
|
|
254
|
+
5. **`SnapDiff::Config::MAPPING` is gone**, split into `SnapDiff::Config::SETTINGS`
|
|
255
|
+
(setting names) and the `@api private` `SnapDiff::LegacyShims::CONFIG_MAPPING`.
|
|
256
|
+
|
|
257
|
+
### Act now: what 2.1 removes
|
|
258
|
+
|
|
259
|
+
2.1 deletes the v1 namespace trees, the ChunkyPNG driver, `shift_distance_limit`,
|
|
260
|
+
and the whole driver abstraction — **libvips becomes the only backend**. Removing
|
|
261
|
+
public API in a minor is a deliberate departure from strict semver; publishing the
|
|
262
|
+
contract one release ahead is the mitigation. 2.0 warns once per process for each of:
|
|
263
|
+
|
|
264
|
+
| You will hear about it when you… | Do this in 2.0 |
|
|
265
|
+
|---|---|
|
|
266
|
+
| write the `driver` setting — `SnapDiff.config.driver = …`, `Capybara::Screenshot::Diff.driver = …` — or pass `screenshot "x", driver: …`. **Value-blind: `driver: :vips` warns too**, because the knob is what goes, not the value | delete it; one backend needs no selection |
|
|
267
|
+
| select `driver: :chunky_png` | add `gem "ruby-vips"` and drop the option |
|
|
268
|
+
| run `driver: :auto` **without ruby-vips** — nothing in your setup says `chunky_png`, so this warning is the only sign 2.1 will break the process | install libvips + `ruby-vips` |
|
|
269
|
+
| set `shift_distance_limit` | use `median_filter_window_size`, `tolerance` or `color_distance_limit` |
|
|
270
|
+
| read `SnapDiff::Drivers.loaded` / `.available` | require `ruby-vips` instead of branching on a detected list |
|
|
271
|
+
| `include SnapDiff::Driver` in your own driver | nothing — custom drivers have no migration path |
|
|
272
|
+
|
|
273
|
+
On 2.1, `SnapDiff.config.driver = :vips` and `Capybara::Screenshot::Diff.driver = :vips`
|
|
274
|
+
raise `NoMethodError`, and the per-screenshot `driver:` key is silently ignored. Booting
|
|
275
|
+
stays quiet — `Config#initialize` seeds the default directly, so only *your* write warns.
|
|
276
|
+
|
|
277
|
+
One removal 2.0 cannot warn about, written down instead: the legacy `LOADED_DRIVERS` /
|
|
278
|
+
`AVAILABLE_DRIVERS` constants are plain eager aliases, with nothing to hook.
|
|
279
|
+
|
|
280
|
+
### Added
|
|
281
|
+
- **`SnapDiff` is the canonical namespace** — the implementation lives in
|
|
282
|
+
`lib/snap_diff/`. Configuration (`SnapDiff::Config`, one storage behind every
|
|
283
|
+
settings surface), errors (`SnapDiff::Error` and friends), `SnapDiff::Region`,
|
|
284
|
+
`SnapDiff::Reporters::Default`, `SnapDiff.session`, `SnapDiff::Reporting.register`,
|
|
285
|
+
and integration requires under `snap_diff/integrations/…`. Start here for new
|
|
286
|
+
code: [docs/snapdiff.md](docs/snapdiff.md)
|
|
287
|
+
- **One consolidated config object** — all 27 settings on `SnapDiff.config`, via
|
|
288
|
+
`SnapDiff.configure { |config| … }`. Old and new surfaces share one storage, so a
|
|
289
|
+
write through either is visible through the other
|
|
290
|
+
- **`SnapDiff::Error` is the base class for every error the gem defines** —
|
|
291
|
+
`ExpectationNotMet`, `UnstableImage`, `WindowSizeMismatchError` and
|
|
292
|
+
`DualInstallError` all inherit it, so one `rescue SnapDiff::Error` covers them.
|
|
293
|
+
(Misuse still surfaces as plain Ruby: `ArgumentError` for bad arguments,
|
|
294
|
+
`RuntimeError` when no image backend is installed.)
|
|
295
|
+
- **An end-of-run summary line, on every run** — `[snap_diff] N verified, N changed,
|
|
296
|
+
N new (not verified).`, plus the report path when there are failures. Registered
|
|
297
|
+
with the HTML reporter (`require "snap_diff/reporters/html"`)
|
|
298
|
+
- **A named line for every screenshot that had no committed baseline**, once per
|
|
299
|
+
screenshot as it happens and once more as a roll-up at the end of the run
|
|
300
|
+
- **Deprecation warnings name your call site**, so migration is warning-driven
|
|
301
|
+
rather than grep-driven
|
|
302
|
+
- **Unrecognised per-screenshot options warn instead of doing nothing.** The options
|
|
303
|
+
hash was frozen but never validated. 2.0 warns once per key; 2.1 raises
|
|
304
|
+
`ArgumentError`
|
|
305
|
+
- **Dual-install guard** — installing both `capybara-screenshot-diff` and
|
|
306
|
+
`snap_diff-capybara` raises `SnapDiff::DualInstallError` at require time instead of
|
|
307
|
+
silently loading files from whichever gem activated first
|
|
308
|
+
- **The gem is also published as
|
|
309
|
+
[`snap_diff-capybara`](https://rubygems.org/gems/snap_diff-capybara)** — identical
|
|
310
|
+
content and version, matching this repository. `capybara-screenshot-diff` remains
|
|
311
|
+
the name to install; the mirror reserves the forward-looking one. **Install one,
|
|
312
|
+
never both.**
|
|
313
|
+
- New documentation shipped inside the gem:
|
|
314
|
+
[docs/snapdiff.md](docs/snapdiff.md) (canonical API) and
|
|
315
|
+
[docs/UPGRADING.md](docs/UPGRADING.md) (every renamed constant, which names warn,
|
|
316
|
+
rollback)
|
|
317
|
+
|
|
318
|
+
### Changed
|
|
319
|
+
- **No `activesupport` at runtime.** 1.15.1's `snap_manager.rb` required
|
|
320
|
+
`active_support/core_ext/module/attribute_accessors` while `activesupport` was only a
|
|
321
|
+
*development* dependency, so a bundle without Rails died at
|
|
322
|
+
`require "capybara/screenshot/diff"` with `LoadError: cannot load such file --
|
|
323
|
+
active_support/core_ext/module/attribute_accessors`. 2.0 requires nothing beyond
|
|
324
|
+
`capybara`
|
|
325
|
+
- **The failure message.** Labelled artifact paths (baseline first), relative to
|
|
326
|
+
`SnapDiff.config.root`, a denominator on the pixel count, and the thresholds actually
|
|
327
|
+
applied. If you grep test output for the old wording, update the pattern
|
|
328
|
+
- **The summary line.** `N screenshots compared, no failures` became `N verified,
|
|
329
|
+
N changed, N new (not verified)`. Same reporter, same report path
|
|
330
|
+
- **`fail_if_new` has no stored default.** `nil` means nobody said, and only then is
|
|
331
|
+
`ENV["CI"]` consulted — read live, not at require time. The default behaviour (fail
|
|
332
|
+
only under CI) is unchanged; an explicit setting now wins in both directions
|
|
333
|
+
- The images-holder struct is now `SnapDiff::Comparison::Images`, ending the
|
|
334
|
+
two-classes-one-name collision with the comparator
|
|
335
|
+
- The packaged gem is an explicit allow-list — `lib/`, `docs/`, `README.md`,
|
|
336
|
+
`LICENSE.txt`, `CHANGELOG.md`. 1.x shipped `Rakefile`, `gems.rb` and the gemspec,
|
|
337
|
+
and omitted the README
|
|
338
|
+
|
|
339
|
+
### Fixed
|
|
340
|
+
- **A screenshot with no committed baseline no longer passes silently** — it warns,
|
|
341
|
+
names the file, and is counted as `new (not verified)` rather than as a pass
|
|
342
|
+
- **An inherited `GIT_DIR` / `GIT_WORK_TREE` / `GIT_INDEX_FILE` no longer redirects
|
|
343
|
+
baseline lookups.** All three are scrubbed before `git -C` runs, so a suite launched
|
|
344
|
+
from a git hook reads its own repository
|
|
345
|
+
- **libvips no longer serves a cached image for a path rewritten within the same mtime
|
|
346
|
+
second** — `revalidate: true` at the single load site, guarded on libvips 8.15+
|
|
347
|
+
- **Rails' `parallelize(workers: N)` produces one merged HTML report** and the merged
|
|
348
|
+
summary counts, at the documented path, with no application-side configuration.
|
|
349
|
+
`parallelize(with: :threads)` and serial runs are unchanged
|
|
350
|
+
- **The missing-baseline error in CI names a file that is on disk.** The raise moved
|
|
351
|
+
after the capture, so `git add <path>` works on the run that printed it — and a
|
|
352
|
+
failing CI job leaves the new screenshot behind for an artifact upload
|
|
353
|
+
- **A bundle without Minitest loads.** `Bundler.require` requires the gem's own name,
|
|
354
|
+
and that file hard-required `capybara_screenshot_diff/minitest` → `minitest`, which
|
|
355
|
+
is not a declared runtime dependency: an RSpec-only bundle died at boot with `cannot
|
|
356
|
+
load such file -- minitest`, from a gem that ships a first-class RSpec integration.
|
|
357
|
+
It now feature-detects, and says so in one line naming the integration to require.
|
|
358
|
+
Explicit requires still hard-require and still fail loudly
|
|
359
|
+
- **The Minitest activation warning no longer fires for everyone.** It keyed off the
|
|
360
|
+
gem-*name* file, which `Bundler.require` loads whatever you required explicitly, so
|
|
361
|
+
it shouted at every user with no way to silence it. It is keyed to the v1 namespace
|
|
362
|
+
entry now, and honours `SnapDiff.silence_deprecations` /
|
|
363
|
+
`SNAP_DIFF_SILENCE_DEPRECATIONS` (it was a bare `Kernel#warn`)
|
|
364
|
+
- **The new-screenshot error no longer names things that do not exist** — no
|
|
365
|
+
`RECORD_SCREENSHOTS=1` (nothing in `lib/` has ever read it) and no `<name>.base.png`
|
|
366
|
+
(a generated temp file nobody commits)
|
|
367
|
+
- Annotation color constants resolve under a bare `require "snap_diff"`; a differing
|
|
368
|
+
comparison previously raised `NameError` there
|
|
369
|
+
- `require "snap_diff/integrations/…"` loads the full `SnapDiff` surface, and
|
|
370
|
+
`gem "snap_diff-capybara"` works with `Bundler.require`
|
|
371
|
+
- **Failure messages no longer dump the raw diff mask.** The comparison metadata
|
|
372
|
+
carried the `diff_mask` image into the error text (`"diff_mask":"#<Vips::Image:0x…>"`,
|
|
373
|
+
or a `{"ptr":{}…}` struct depending on the driver); it is excluded now
|
|
374
|
+
- Reporter failure warnings use one brand and name the failing reporter class
|
|
375
|
+
|
|
376
|
+
### Known limitations
|
|
377
|
+
- **One process per worker** (`parallel_tests`, RSpec, CI sharding) still writes one
|
|
378
|
+
report per process to the same path, so only the last to finish survives. Rails'
|
|
379
|
+
`parallelize(workers: N)` and `parallelize(with: :threads)` are both complete —
|
|
380
|
+
see [docs/reporters.md](docs/reporters.md#parallel-test-runs)
|
|
381
|
+
|
|
382
|
+
### Unchanged
|
|
383
|
+
- Ruby 3.2+, Capybara `>= 2, < 4`, the `screenshot` / `assert_matches_screenshot`
|
|
384
|
+
DSL, every capture and comparison option, baseline file names and formats
|
|
385
|
+
|
|
386
|
+
---
|
|
387
|
+
|
|
388
|
+
## [v2.0.0.beta4] - 2026-08-24
|
|
389
|
+
|
|
390
|
+
The prerelease the 2.0.0 entry above describes. beta3 fixed the canonical entry
|
|
391
|
+
points; **beta4 is where the behaviour changes land** — the four green-suite bugs,
|
|
392
|
+
the accept workflow, the failure message, and the deprecation warnings that make
|
|
393
|
+
2.1's removals visible. If you are on beta3, this is not an optional bump: beta3
|
|
394
|
+
still passes green on a screenshot it never compared.
|
|
395
|
+
|
|
396
|
+
Read the `[v2.0.0]` section above for the full story. This section is the delta
|
|
397
|
+
from beta3.
|
|
398
|
+
|
|
399
|
+
### Added
|
|
400
|
+
- **Record modes** — `SnapDiff.config.record = :once | :none | :all` replace
|
|
401
|
+
`fail_if_new` (removed in 2.1). `:all` is the bulk-accept verb after an
|
|
402
|
+
intentional redesign (#259)
|
|
403
|
+
- **A summary line that counts what was verified**, not just what was compared:
|
|
404
|
+
`[snap_diff] 12 verified, 1 changed, 2 new (not verified)`. It comes from the
|
|
405
|
+
HTML reporter (#261, #269)
|
|
406
|
+
- **An optional readiness block** — `assert_matches_screenshot("home") { preload_all_images }`
|
|
407
|
+
runs the block *inside* the assertion, so it is skipped along with the screenshot
|
|
408
|
+
when screenshots are off, and a run-level tally names every `skip_area` selector
|
|
409
|
+
that never matched an element (#277)
|
|
410
|
+
- **Deprecation warnings for everything 2.1 removes**, including `chunky_png`,
|
|
411
|
+
`shift_distance_limit`, the driver abstraction and the whole v1 namespace. Nothing
|
|
412
|
+
2.1 deletes is silent in 2.0 (#246, #263)
|
|
413
|
+
|
|
414
|
+
### Fixed
|
|
415
|
+
- **A screenshot with no committed baseline no longer passes green** (#255)
|
|
416
|
+
- **An inherited `GIT_DIR` no longer redirects every baseline lookup** — every git
|
|
417
|
+
hook exports one, and the whole suite passed under it (#256)
|
|
418
|
+
- **libvips no longer serves stale pixels** when a screenshot is rewritten inside the
|
|
419
|
+
same second; its loader cache keys on filename + mtime at one-second resolution (#254)
|
|
420
|
+
- **The HTML report survives Rails' fork-parallel test runs** — `parallelize(workers:)`
|
|
421
|
+
previously wrote no report and printed no summary line at all (#258)
|
|
422
|
+
- **A baseline that disappears mid-test raises** instead of passing green (#217)
|
|
423
|
+
- **The failure message is legible** — baseline first, every artifact labelled, a
|
|
424
|
+
denominator on the pixel count, and a `judged against:` line (#264)
|
|
425
|
+
- **No printed path or command that is not derived from live state** — the source of
|
|
426
|
+
the `RECORD_SCREENSHOTS` folklore (#260)
|
|
427
|
+
- **JRuby no longer hangs at teardown** joining leftover server and browser threads (#248)
|
|
428
|
+
|
|
429
|
+
### Performance
|
|
430
|
+
- **Matching assertions stop decoding both PNGs**, and the repo root is memoised
|
|
431
|
+
under a lock instead of spawning `git rev-parse` once per screenshot: 63.5 ms →
|
|
432
|
+
44.5 ms end to end for one matching 1440x900 assertion (#250, #253)
|
|
433
|
+
- **`skip_area` no longer pays an implicit stabilization wait.** Measured on a real
|
|
434
|
+
suite: 10.012 s → 0.009 s. **See the caveat below** (#272)
|
|
435
|
+
|
|
436
|
+
### Caveat, and it is the reason #277 shipped in the same beta
|
|
437
|
+
Removing `skip_area`'s implicit wait (#272) means a selector that is not yet in the
|
|
438
|
+
DOM when the assertion runs now yields **no mask**, silently, where it previously
|
|
439
|
+
resolved after the wait. Nothing in 1.15.1 or beta3 told you a mask was missing.
|
|
440
|
+
The run-level tally (#277) is the replacement signal: any `skip_area` selector that
|
|
441
|
+
matched nothing during the run is named at the end. If you relied on the old wait,
|
|
442
|
+
that tally is where you will see it — and the readiness block is where you fix it.
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
8
446
|
## [v2.0.0.beta3] - 2026-08-23
|
|
9
447
|
|
|
10
448
|
Fixes the canonical `SnapDiff` entry points, which were incomplete in beta2.
|