compatriot 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +52 -14
- data/lib/compatriot.rb +11 -6
- data/lib/compatriot/assertions.rb +14 -0
- data/lib/compatriot/version.rb +1 -1
- data/spec/spec_helper.rb +19 -0
- data/spec/unit/assertions_spec.rb +28 -0
- data/spec/unit/compatriot_spec.rb +8 -20
- metadata +5 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b5f3fe79e1df887a64a4199746a116f1b1b21507
|
4
|
+
data.tar.gz: b89826f89e27b3a505601a0d86f13458349b6708
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7fe484047be403873aeaa2b7311fbefb23557f0b492fd15f515e3a7b8522cd0bd9e165ee1b05640e230b6881b3930e2ec71e6a2c6950ce35ab17994d131fda4a
|
7
|
+
data.tar.gz: 73691e6f9298e8ccaba766e5ea11f37793b3a75dea9daab680642c5a7edddcc511bcf7f9ee0546403159db9f3979d93142dd53753a5f3f01a01c2e76924dcbab
|
data/README.md
CHANGED
@@ -13,10 +13,10 @@ What it does now
|
|
13
13
|
----------------
|
14
14
|
|
15
15
|
* [Documentation on RelishApp](https://www.relishapp.com/clnclarinet/compatriot)
|
16
|
-
*
|
17
|
-
*
|
18
|
-
*
|
19
|
-
|
16
|
+
* Adds a `assert_no_ui_changes` method to your tests
|
17
|
+
* When called from your tests, it takes a screenshot and compares it to a control image to get percentage difference
|
18
|
+
* If the difference is greater then the threshold, it fails the tests
|
19
|
+
* Pairs well with [BrowserStacks automation feature](browserstack.com/automate) to run across many environments
|
20
20
|
|
21
21
|
What it will do in the future
|
22
22
|
-----------------------------
|
@@ -26,11 +26,7 @@ What it will do in the future
|
|
26
26
|
* Have more and better tests
|
27
27
|
* Find the largest, darkest contiguous region in the image diff and have a threshold of pass/fail based on that
|
28
28
|
* Perform better on the image processing (by sampling/resizing, using oily_png, etc)
|
29
|
-
* Given a list of URLs/paths to visit, will take a screenshot of each and display which URL it came from in the index
|
30
|
-
* Given a test command that uses capybara, runs those tests and takes screenshots
|
31
29
|
* Automatically compare the screenshots across browsers and flags those that are more than some configurable threshold different
|
32
|
-
* Allow configuration of which browsers to use
|
33
|
-
* Connect to virtual machines so that you don't have to have all the browsers on the machine you're running the tests on
|
34
30
|
* Steal some of VCR's relish rake tasks
|
35
31
|
|
36
32
|
|
@@ -40,12 +36,53 @@ How To Use
|
|
40
36
|
**Requirements**
|
41
37
|
|
42
38
|
* Ruby v2.1.7
|
43
|
-
* [
|
44
|
-
|
39
|
+
* [Capybara](http://jnicklas.github.io/capybara/)
|
40
|
+
|
41
|
+
1. Require `compatriot` in your test helper,
|
42
|
+
2. and include `Compatriot::Assertions` in your test class
|
43
|
+
3. and configure Compatriot
|
44
|
+
|
45
|
+
```ruby
|
46
|
+
require 'compatriot'
|
47
|
+
|
48
|
+
class IntegrationTest
|
49
|
+
include Compatriot::Assertions
|
50
|
+
|
51
|
+
Compatriot.configure do |config|
|
52
|
+
screenshot_directory =
|
53
|
+
'./test/screenshots/' +
|
54
|
+
caps['os'] + '_' + #caps(Capabilities) is how we tell browserstack what environment to use
|
55
|
+
caps['os_version'] + '_' +
|
56
|
+
caps['browser'] + '_' +
|
57
|
+
caps['browser_version']
|
58
|
+
config.screenshot_directory = screenshot_directory.downcase
|
59
|
+
config.ui_difference_threshold = 0.05
|
60
|
+
end
|
61
|
+
...
|
62
|
+
```
|
63
|
+
|
64
|
+
4. Call `assert_no_ui_changes` from your tests
|
65
|
+
|
66
|
+
```ruby
|
67
|
+
it 'does not break the ui' do
|
68
|
+
visit some_page_path
|
69
|
+
assert_no_ui_changes('some page')
|
70
|
+
end
|
71
|
+
```
|
72
|
+
|
73
|
+
The first time through it will create the control image in `#{screenshot_directory}/control`. You should then review the screenshots and check them into source control as your baseline.
|
74
|
+
Every run after that, it will take a variable image screenshot in `#{screenshot_directory}/variable` and a difference image in `#{screenshot_directory}/diffs`.
|
75
|
+
|
76
|
+
`assert_no_ui_changes` takes an optional string parameter to allow you to make multiple assertions in a test and not run into name conflicts.
|
77
|
+
|
78
|
+
To see the % difference for every assertion, and get the path the the difference, set `config.show_diffs = true` when you configure Compatriot.
|
79
|
+
|
45
80
|
|
46
|
-
|
81
|
+
How to run this against multiple environments?
|
82
|
+
----------------------------------------------
|
47
83
|
|
48
|
-
|
84
|
+
The intentions for running this against multiple environments is that you would run your test suite once for every environment you wish to run it against.
|
85
|
+
To accomplish this, you should set your `screenshot_directory` to include the current environment name, this way you will store your control images in separate directories.
|
49
86
|
|
50
87
|
|
51
88
|
What to do to run its tests
|
@@ -93,10 +130,11 @@ Many thanks to the wonderful libraries that make this gem possible:
|
|
93
130
|
|
94
131
|
Contributors
|
95
132
|
------------
|
96
|
-
* Carol Nichols ([twitter](http://twitter.com/carols10cents), [website](http://carol-nichols.com))
|
133
|
+
* [Carol Nichols](https://github.com/carols10cents) ([twitter](http://twitter.com/carols10cents), [website](http://carol-nichols.com))
|
97
134
|
* Andrew Cox ([twitter](https://twitter.com/coxandrew), [website](http://andrewcox.org/))
|
98
135
|
* Kurtis Rainbolt-Greene ([twitter](https://twitter.com/krainboltgreene), [website](http://kurtisrainboltgreene.name/))
|
99
|
-
* Steve Klabnik ([twitter](https://twitter.com/steveklabnik), [website](http://www.steveklabnik.com/))
|
136
|
+
* [Steve Klabnik](Klabnik) ([twitter](https://twitter.com/steveklabnik), [website](http://www.steveklabnik.com/))
|
137
|
+
* [Jeff Koenig](https://github.com/thejefe) ([twitter](http://twitter.com/jeffkoenig)
|
100
138
|
* You???
|
101
139
|
|
102
140
|
|
data/lib/compatriot.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
require "compatriot/assertions"
|
1
2
|
require "compatriot/version"
|
2
3
|
require "compatriot/runner"
|
3
4
|
require "compatriot/browser"
|
@@ -6,7 +7,9 @@ require "compatriot/image_differ/image_differ"
|
|
6
7
|
|
7
8
|
module Compatriot
|
8
9
|
class << self
|
9
|
-
attr_accessor :app, :screenshot_directory
|
10
|
+
attr_accessor :app, :screenshot_directory,
|
11
|
+
:ui_difference_threshold, :framework,
|
12
|
+
:show_diffs
|
10
13
|
|
11
14
|
def configure
|
12
15
|
yield self
|
@@ -16,21 +19,20 @@ module Compatriot
|
|
16
19
|
Compatriot::Runner.start(app, paths)
|
17
20
|
end
|
18
21
|
|
19
|
-
def take_screenshot(
|
22
|
+
def take_screenshot(test, description)
|
20
23
|
filename = filename_for_test(test, description)
|
21
24
|
control_image_path = filepath_for_screenshot('control', filename)
|
22
|
-
variable_image_path = filepath_for_screenshot('variable', filename)
|
23
25
|
|
24
26
|
if File.exist?(control_image_path)
|
25
27
|
screenshot_type = 'variable'
|
26
28
|
else
|
27
29
|
screenshot_type = 'control'
|
28
30
|
end
|
29
|
-
|
31
|
+
framework.current_session.save_screenshot filepath_for_screenshot(screenshot_type, filename)
|
30
32
|
end
|
31
33
|
|
32
|
-
def percentage_changed(
|
33
|
-
variable_img_path = take_screenshot(
|
34
|
+
def percentage_changed(test, description = '')
|
35
|
+
variable_img_path = take_screenshot(test, description)
|
34
36
|
control_img_path = filepath_for_screenshot('control', filename_for_test(test, description))
|
35
37
|
diff = Compatriot::ColorDiffer.diff(variable_img_path, control_img_path, self.screenshot_directory + '/')
|
36
38
|
variable_image = ChunkyPNG::Image.from_file(variable_img_path)
|
@@ -52,5 +54,8 @@ module Compatriot
|
|
52
54
|
|
53
55
|
Compatriot.configure do |config|
|
54
56
|
config.screenshot_directory = './compatriot/screenshots'
|
57
|
+
config.ui_difference_threshold = 0.05
|
58
|
+
config.framework = Capybara #only supporting capybara until someone wants to support something else
|
59
|
+
config.show_diffs = false
|
55
60
|
end
|
56
61
|
end
|
@@ -0,0 +1,14 @@
|
|
1
|
+
module Compatriot::Assertions
|
2
|
+
def assert_no_ui_changes(title = '')
|
3
|
+
diff = Compatriot.percentage_changed(self, title)
|
4
|
+
diff_file = Compatriot.filepath_for_screenshot('diffs', Compatriot.filename_for_test(self, title))
|
5
|
+
puts "% diff is #{diff}. #{diff_file}" if Compatriot.show_diffs
|
6
|
+
pass = diff <= Compatriot.ui_difference_threshold
|
7
|
+
|
8
|
+
unless pass
|
9
|
+
puts "Diff > ThresholdFailed: see #{diff_file}"
|
10
|
+
end
|
11
|
+
|
12
|
+
assert pass, "The difference in the page (#{diff}%) is greater then the threshold #{Compatriot.ui_difference_threshold}"
|
13
|
+
end
|
14
|
+
end
|
data/lib/compatriot/version.rb
CHANGED
data/spec/spec_helper.rb
CHANGED
@@ -22,4 +22,23 @@ end
|
|
22
22
|
|
23
23
|
class Minitest::Spec
|
24
24
|
include TestRunner
|
25
|
+
include Compatriot::Assertions
|
26
|
+
end
|
27
|
+
|
28
|
+
module FakeCapybara
|
29
|
+
def self.current_session
|
30
|
+
Page.new
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
class Page
|
35
|
+
def save_screenshot filepath
|
36
|
+
root_dir = File.join(File.dirname(__FILE__), './')
|
37
|
+
image_name = filepath.include?('control') ? '1' : '2'
|
38
|
+
src_img = root_dir + "/images/#{image_name}.png"
|
39
|
+
filepath_dir = File.dirname(filepath)
|
40
|
+
FileUtils.mkdir_p(filepath_dir) unless File.directory?(filepath_dir)
|
41
|
+
FileUtils.cp(src_img, filepath)
|
42
|
+
filepath
|
43
|
+
end
|
25
44
|
end
|
@@ -0,0 +1,28 @@
|
|
1
|
+
require_relative '../spec_helper'
|
2
|
+
|
3
|
+
describe Compatriot::Assertions do
|
4
|
+
let(:page) { Page.new }
|
5
|
+
|
6
|
+
SCREENSHOTS_DIR = './tmp/test/screenshots'
|
7
|
+
CONTROL_IMG_FILENAME = 'important_test_will_do_something_important_and_has_a_description.png'
|
8
|
+
CONTROL_IMG = "#{SCREENSHOTS_DIR}/control/#{CONTROL_IMG_FILENAME}"
|
9
|
+
CONTROL_IMG2 = "#{SCREENSHOTS_DIR}/control/important_test_will_do_something_important_another.png"
|
10
|
+
VARIABLE_IMG = "#{SCREENSHOTS_DIR}/variable/important_test_will_do_something_important_and_has_a_description.png"
|
11
|
+
DIFF_IMG = "#{SCREENSHOTS_DIR}/diffs/color_variable_vs_control_important_test_will_do_something_important_and_has_a_description.png"
|
12
|
+
|
13
|
+
def setup_control_image
|
14
|
+
Page.new.save_screenshot CONTROL_IMG
|
15
|
+
end
|
16
|
+
|
17
|
+
before do
|
18
|
+
FileUtils.remove_dir(SCREENSHOTS_DIR) if File.directory?(SCREENSHOTS_DIR)
|
19
|
+
Compatriot.configure do |config|
|
20
|
+
config.screenshot_directory = SCREENSHOTS_DIR
|
21
|
+
config.framework = FakeCapybara
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
it 'can assert on assert_no_ui_changes' do
|
26
|
+
assert_no_ui_changes('this test')
|
27
|
+
end
|
28
|
+
end
|
@@ -1,5 +1,4 @@
|
|
1
1
|
require_relative '../spec_helper'
|
2
|
-
require 'pry'
|
3
2
|
|
4
3
|
describe Compatriot do
|
5
4
|
let(:class_stub) { stub_everything('class', name: 'important_test') }
|
@@ -13,18 +12,6 @@ describe Compatriot do
|
|
13
12
|
VARIABLE_IMG = "#{SCREENSHOTS_DIR}/variable/important_test_will_do_something_important_and_has_a_description.png"
|
14
13
|
DIFF_IMG = "#{SCREENSHOTS_DIR}/diffs/color_variable_vs_control_important_test_will_do_something_important_and_has_a_description.png"
|
15
14
|
|
16
|
-
class Page
|
17
|
-
def save_screenshot filepath
|
18
|
-
root_dir = File.join(File.dirname(__FILE__), '../')
|
19
|
-
image_name = filepath.include?('control') ? '1' : '2'
|
20
|
-
src_img = root_dir + "/images/#{image_name}.png"
|
21
|
-
filepath_dir = File.dirname(filepath)
|
22
|
-
FileUtils.mkdir_p(filepath_dir) unless File.directory?(filepath_dir)
|
23
|
-
FileUtils.cp(src_img, filepath)
|
24
|
-
filepath
|
25
|
-
end
|
26
|
-
end
|
27
|
-
|
28
15
|
def setup_control_image
|
29
16
|
Page.new.save_screenshot CONTROL_IMG
|
30
17
|
end
|
@@ -33,13 +20,14 @@ describe Compatriot do
|
|
33
20
|
FileUtils.remove_dir(SCREENSHOTS_DIR) if File.directory?(SCREENSHOTS_DIR)
|
34
21
|
Compatriot.configure do |config|
|
35
22
|
config.screenshot_directory = SCREENSHOTS_DIR
|
23
|
+
config.framework = FakeCapybara
|
36
24
|
end
|
37
25
|
end
|
38
26
|
|
39
27
|
describe 'no control image is found' do
|
40
28
|
it 'it will create one' do
|
41
|
-
Compatriot.take_screenshot(
|
42
|
-
Compatriot.take_screenshot(
|
29
|
+
Compatriot.take_screenshot(stubbed_test, 'and has a description')
|
30
|
+
Compatriot.take_screenshot(stubbed_test, 'another')
|
43
31
|
|
44
32
|
assert File.exist?(CONTROL_IMG), 'control image not found'
|
45
33
|
assert File.exist?(CONTROL_IMG2), 'control image not found'
|
@@ -52,23 +40,23 @@ describe Compatriot do
|
|
52
40
|
end
|
53
41
|
|
54
42
|
it 'stores a variable image' do
|
55
|
-
Compatriot.take_screenshot(
|
43
|
+
Compatriot.take_screenshot(stubbed_test, 'and has a description')
|
56
44
|
assert File.exist?(VARIABLE_IMG)
|
57
45
|
end
|
58
46
|
|
59
47
|
it 'stores the image difference' do
|
60
|
-
Compatriot.percentage_changed(
|
48
|
+
Compatriot.percentage_changed( stubbed_test, 'and has a description')
|
61
49
|
assert File.exist?(DIFF_IMG)
|
62
50
|
end
|
63
51
|
|
64
52
|
it 'returns the percentage difference' do
|
65
|
-
result = Compatriot.percentage_changed(
|
53
|
+
result = Compatriot.percentage_changed( stubbed_test, 'and has a description')
|
66
54
|
assert_equal(0.81, result.round(2))
|
67
55
|
end
|
68
56
|
|
69
57
|
it 'returns 0 % if there is no difference' do
|
70
58
|
Compatriot::ColorDiffer.stubs(:diff).returns([])
|
71
|
-
result = Compatriot.percentage_changed(
|
59
|
+
result = Compatriot.percentage_changed( stubbed_test, 'and has a description')
|
72
60
|
assert_equal(0.0, result.round(2))
|
73
61
|
end
|
74
62
|
end
|
@@ -84,7 +72,7 @@ describe Compatriot do
|
|
84
72
|
end
|
85
73
|
|
86
74
|
it 'can set the screenshot directory' do
|
87
|
-
Compatriot.take_screenshot(
|
75
|
+
Compatriot.take_screenshot( stubbed_test, 'and has a description')
|
88
76
|
assert File.exist?(@control_file), 'control image not found'
|
89
77
|
end
|
90
78
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: compatriot
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Carol (Nichols || Goulding)
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2016-
|
12
|
+
date: 2016-02-05 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: minitest
|
@@ -200,6 +200,7 @@ files:
|
|
200
200
|
- compatriot.gemspec
|
201
201
|
- examples/compatriot_tests.rb
|
202
202
|
- lib/compatriot.rb
|
203
|
+
- lib/compatriot/assertions.rb
|
203
204
|
- lib/compatriot/browser.rb
|
204
205
|
- lib/compatriot/image_differ/color_differ.rb
|
205
206
|
- lib/compatriot/image_differ/image_differ.rb
|
@@ -211,6 +212,7 @@ files:
|
|
211
212
|
- spec/sample_app/public/images/smileyface.jpg
|
212
213
|
- spec/sample_app/sample_app.rb
|
213
214
|
- spec/spec_helper.rb
|
215
|
+
- spec/unit/assertions_spec.rb
|
214
216
|
- spec/unit/browser_spec.rb
|
215
217
|
- spec/unit/compatriot_spec.rb
|
216
218
|
- spec/unit/image_differ/color_differ_spec.rb
|
@@ -248,6 +250,7 @@ test_files:
|
|
248
250
|
- spec/sample_app/public/images/smileyface.jpg
|
249
251
|
- spec/sample_app/sample_app.rb
|
250
252
|
- spec/spec_helper.rb
|
253
|
+
- spec/unit/assertions_spec.rb
|
251
254
|
- spec/unit/browser_spec.rb
|
252
255
|
- spec/unit/compatriot_spec.rb
|
253
256
|
- spec/unit/image_differ/color_differ_spec.rb
|