compatriot 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b42a702d988c48093abc7cd8fe13b09d02f3943b
-  data.tar.gz: a7c9af901dc60c45d98b18a798fc5f2d81ecdac7
+  metadata.gz: b5f3fe79e1df887a64a4199746a116f1b1b21507
+  data.tar.gz: b89826f89e27b3a505601a0d86f13458349b6708
 SHA512:
-  metadata.gz: 5f179d47804b164b070ae0b94e3d7fbe08757e4e9e8b797147e4d6dd2852347b8d26657af4bfe58d33b89577d61c06cadb066e70f34afa50af544f27df5b2d22
-  data.tar.gz: 08f1c1500a97095a027b3330ea2c0329e4f3f55012f6f1cac52549cb9f5248806940ae832751d164e4fc620117ccb014be6773cf2d4bc1e216af9af02decc433
+  metadata.gz: 7fe484047be403873aeaa2b7311fbefb23557f0b492fd15f515e3a7b8522cd0bd9e165ee1b05640e230b6881b3930e2ec71e6a2c6950ce35ab17994d131fda4a
+  data.tar.gz: 73691e6f9298e8ccaba766e5ea11f37793b3a75dea9daab680642c5a7edddcc511bcf7f9ee0546403159db9f3979d93142dd53753a5f3f01a01c2e76924dcbab

data/README.md

@@ -13,10 +13,10 @@ What it does now
 ----------------
 
 * [Documentation on RelishApp](https://www.relishapp.com/clnclarinet/compatriot)
-* In firefox and chrome, visits a list of paths to a Rack app and takes a screenshot on each page.
-* Stores the screenshot in `tmp/results/_timestamp_/_browser_/`
-* Creates `tmp/results/_timestamp_/index.html` that shows thumbnails of each screenshot plus a diff of the two in a table for easy comparison.
-
+* Adds a `assert_no_ui_changes` method to your tests
+* When called from your tests, it takes a screenshot and compares it to a control image to get percentage difference
+* If the difference is greater then the threshold, it fails the tests
+* Pairs well with [BrowserStacks automation feature](browserstack.com/automate) to run across many environments
 
 What it will do in the future
 -----------------------------
@@ -26,11 +26,7 @@ What it will do in the future
 * Have more and better tests
 * Find the largest, darkest contiguous region in the image diff and have a threshold of pass/fail based on that
 * Perform better on the image processing (by sampling/resizing, using oily_png, etc)
-* Given a list of URLs/paths to visit, will take a screenshot of each and display which URL it came from in the index
-* Given a test command that uses capybara, runs those tests and takes screenshots
 * Automatically compare the screenshots across browsers and flags those that are more than some configurable threshold different
-* Allow configuration of which browsers to use
-* Connect to virtual machines so that you don't have to have all the browsers on the machine you're running the tests on
 * Steal some of VCR's relish rake tasks
 
 
@@ -40,12 +36,53 @@ How To Use
 **Requirements**
 
 * Ruby v2.1.7
-* [Firefox](http://getfirefox.net)
-* [chromedriver](http://code.google.com/p/selenium/wiki/ChromeDriver)
+* [Capybara](http://jnicklas.github.io/capybara/)
+
+1. Require `compatriot` in your test helper, 
+2. and include `Compatriot::Assertions` in your test class
+3. and configure Compatriot
+
+    ```ruby
+    require 'compatriot'
+
+    class IntegrationTest
+    include Compatriot::Assertions
+
+      Compatriot.configure do |config|
+        screenshot_directory = 
+          './test/screenshots/' +
+          caps['os'] + '_' +       #caps(Capabilities) is how we tell browserstack what environment to use
+          caps['os_version'] + '_' +
+          caps['browser'] + '_' +
+          caps['browser_version']
+        config.screenshot_directory = screenshot_directory.downcase
+        config.ui_difference_threshold = 0.05
+      end
+      ...
+    ```
+
+4. Call `assert_no_ui_changes` from your tests
+
+    ```ruby
+    it 'does not break the ui' do
+      visit some_page_path
+      assert_no_ui_changes('some page')
+    end
+    ```
+
+The first time through it will create the control image in `#{screenshot_directory}/control`. You should then review the screenshots and check them into source control as your baseline.
+Every run after that, it will take a variable image screenshot in `#{screenshot_directory}/variable` and a difference image in `#{screenshot_directory}/diffs`.
+
+`assert_no_ui_changes` takes an optional string parameter to allow you to make multiple assertions in a test and not run into name conflicts.
+
+To see the % difference for every assertion, and get the path the the difference, set `config.show_diffs = true` when you configure Compatriot.
+
 
-There are setup examples in the examples directory and [documentation on RelishApp](https://www.relishapp.com/clnclarinet/compatriot)
+How to run this against multiple environments?
+----------------------------------------------
 
-When you run a file similar to the examples it will save results in `_current-directory_/tmp/results/_timestamp_/_browser_`
+The intentions for running this against multiple environments is that you would run your test suite once for every environment you wish to run it against.
+To accomplish this, you should set your `screenshot_directory` to include the current environment name, this way you will store your control images in separate directories.
 
 
 What to do to run its tests
@@ -93,10 +130,11 @@ Many thanks to the wonderful libraries that make this gem possible:
 
 Contributors
 ------------
-* Carol Nichols ([twitter](http://twitter.com/carols10cents), [website](http://carol-nichols.com))
+* [Carol Nichols](https://github.com/carols10cents) ([twitter](http://twitter.com/carols10cents), [website](http://carol-nichols.com))
 * Andrew Cox ([twitter](https://twitter.com/coxandrew), [website](http://andrewcox.org/))
 * Kurtis Rainbolt-Greene ([twitter](https://twitter.com/krainboltgreene), [website](http://kurtisrainboltgreene.name/))
-* Steve Klabnik ([twitter](https://twitter.com/steveklabnik), [website](http://www.steveklabnik.com/))
+* [Steve Klabnik](Klabnik) ([twitter](https://twitter.com/steveklabnik), [website](http://www.steveklabnik.com/))
+* [Jeff Koenig](https://github.com/thejefe) ([twitter](http://twitter.com/jeffkoenig)
 * You???
 
 

data/lib/compatriot.rb

@@ -1,3 +1,4 @@
+require "compatriot/assertions"
 require "compatriot/version"
 require "compatriot/runner"
 require "compatriot/browser"
@@ -6,7 +7,9 @@ require "compatriot/image_differ/image_differ"
 
 module Compatriot
   class << self
-    attr_accessor :app, :screenshot_directory
+    attr_accessor :app, :screenshot_directory,
+                  :ui_difference_threshold, :framework,
+                  :show_diffs
 
     def configure
       yield self
@@ -16,21 +19,20 @@ module Compatriot
       Compatriot::Runner.start(app, paths)
     end
 
-    def take_screenshot(page, test, description)
+    def take_screenshot(test, description)
       filename = filename_for_test(test, description)
       control_image_path = filepath_for_screenshot('control', filename)
-      variable_image_path = filepath_for_screenshot('variable', filename)
 
       if File.exist?(control_image_path)
         screenshot_type = 'variable'
       else
         screenshot_type = 'control'
       end
-      temp = page.save_screenshot filepath_for_screenshot(screenshot_type, filename)
+      framework.current_session.save_screenshot filepath_for_screenshot(screenshot_type, filename)
     end
 
-    def percentage_changed(page, test, description = '')
-      variable_img_path = take_screenshot(page, test, description)
+    def percentage_changed(test, description = '')
+      variable_img_path = take_screenshot(test, description)
       control_img_path = filepath_for_screenshot('control', filename_for_test(test, description))
       diff = Compatriot::ColorDiffer.diff(variable_img_path, control_img_path, self.screenshot_directory + '/')
       variable_image = ChunkyPNG::Image.from_file(variable_img_path)
@@ -52,5 +54,8 @@ module Compatriot
 
   Compatriot.configure do |config|
     config.screenshot_directory = './compatriot/screenshots'
+    config.ui_difference_threshold  = 0.05
+    config.framework = Capybara #only supporting capybara until someone wants to support something else
+    config.show_diffs = false
   end
 end

data/lib/compatriot/assertions.rb

@@ -0,0 +1,14 @@
+module Compatriot::Assertions
+  def assert_no_ui_changes(title = '')
+    diff = Compatriot.percentage_changed(self, title)
+    diff_file = Compatriot.filepath_for_screenshot('diffs', Compatriot.filename_for_test(self, title))
+    puts "% diff is #{diff}. #{diff_file}" if Compatriot.show_diffs
+    pass = diff <= Compatriot.ui_difference_threshold
+
+    unless pass
+      puts "Diff > ThresholdFailed: see #{diff_file}"
+    end
+
+    assert pass, "The difference in the page (#{diff}%) is greater then the threshold #{Compatriot.ui_difference_threshold}"
+  end
+end

data/lib/compatriot/version.rb

@@ -1,3 +1,3 @@
 module Compatriot
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

data/spec/spec_helper.rb

@@ -22,4 +22,23 @@ end
 
 class Minitest::Spec
   include TestRunner
+  include Compatriot::Assertions
+end
+
+module FakeCapybara
+  def self.current_session
+    Page.new
+  end
+end
+
+class Page
+  def save_screenshot filepath
+    root_dir = File.join(File.dirname(__FILE__), './')
+    image_name = filepath.include?('control') ? '1' : '2'
+    src_img = root_dir + "/images/#{image_name}.png"
+    filepath_dir = File.dirname(filepath)
+    FileUtils.mkdir_p(filepath_dir) unless File.directory?(filepath_dir)
+    FileUtils.cp(src_img, filepath)
+    filepath
+  end
 end

data/spec/unit/assertions_spec.rb

@@ -0,0 +1,28 @@
+require_relative '../spec_helper'
+
+describe Compatriot::Assertions do
+  let(:page) { Page.new }
+
+  SCREENSHOTS_DIR      = './tmp/test/screenshots'
+  CONTROL_IMG_FILENAME = 'important_test_will_do_something_important_and_has_a_description.png'
+  CONTROL_IMG          = "#{SCREENSHOTS_DIR}/control/#{CONTROL_IMG_FILENAME}"
+  CONTROL_IMG2         = "#{SCREENSHOTS_DIR}/control/important_test_will_do_something_important_another.png"
+  VARIABLE_IMG         = "#{SCREENSHOTS_DIR}/variable/important_test_will_do_something_important_and_has_a_description.png"
+  DIFF_IMG             = "#{SCREENSHOTS_DIR}/diffs/color_variable_vs_control_important_test_will_do_something_important_and_has_a_description.png"
+
+  def setup_control_image
+    Page.new.save_screenshot CONTROL_IMG
+  end
+
+  before do
+    FileUtils.remove_dir(SCREENSHOTS_DIR) if File.directory?(SCREENSHOTS_DIR)
+    Compatriot.configure do |config| 
+      config.screenshot_directory = SCREENSHOTS_DIR
+      config.framework = FakeCapybara
+    end
+  end
+
+  it 'can assert on assert_no_ui_changes' do
+    assert_no_ui_changes('this test')
+  end
+end

data/spec/unit/compatriot_spec.rb

@@ -1,5 +1,4 @@
 require_relative '../spec_helper'
-require 'pry'
 
 describe Compatriot do
   let(:class_stub)   { stub_everything('class', name: 'important_test') }
@@ -13,18 +12,6 @@ describe Compatriot do
   VARIABLE_IMG         = "#{SCREENSHOTS_DIR}/variable/important_test_will_do_something_important_and_has_a_description.png"
   DIFF_IMG             = "#{SCREENSHOTS_DIR}/diffs/color_variable_vs_control_important_test_will_do_something_important_and_has_a_description.png"
 
-  class Page
-    def save_screenshot filepath
-      root_dir = File.join(File.dirname(__FILE__), '../')
-      image_name = filepath.include?('control') ? '1' : '2'
-      src_img = root_dir + "/images/#{image_name}.png"
-      filepath_dir = File.dirname(filepath)
-      FileUtils.mkdir_p(filepath_dir) unless File.directory?(filepath_dir)
-      FileUtils.cp(src_img, filepath)
-      filepath
-    end
-  end
-
   def setup_control_image
     Page.new.save_screenshot CONTROL_IMG
   end
@@ -33,13 +20,14 @@ describe Compatriot do
     FileUtils.remove_dir(SCREENSHOTS_DIR) if File.directory?(SCREENSHOTS_DIR)
     Compatriot.configure do |config| 
       config.screenshot_directory = SCREENSHOTS_DIR
+      config.framework = FakeCapybara
     end
   end
 
   describe 'no control image is found' do
     it 'it will create one' do
-      Compatriot.take_screenshot(page, stubbed_test, 'and has a description')
-      Compatriot.take_screenshot(page, stubbed_test, 'another')
+      Compatriot.take_screenshot(stubbed_test, 'and has a description')
+      Compatriot.take_screenshot(stubbed_test, 'another')
 
       assert File.exist?(CONTROL_IMG), 'control image not found'
       assert File.exist?(CONTROL_IMG2), 'control image not found'
@@ -52,23 +40,23 @@ describe Compatriot do
     end
 
     it 'stores a variable image' do
-      Compatriot.take_screenshot(page, stubbed_test, 'and has a description')
+      Compatriot.take_screenshot(stubbed_test, 'and has a description')
       assert File.exist?(VARIABLE_IMG)
     end
 
     it 'stores the image difference' do
-      Compatriot.percentage_changed(page, stubbed_test, 'and has a description')
+      Compatriot.percentage_changed( stubbed_test, 'and has a description')
       assert File.exist?(DIFF_IMG)
     end
 
     it 'returns the percentage difference' do
-      result = Compatriot.percentage_changed(page, stubbed_test, 'and has a description')
+      result = Compatriot.percentage_changed( stubbed_test, 'and has a description')
       assert_equal(0.81, result.round(2))
     end
 
     it 'returns 0 % if there is no difference' do
       Compatriot::ColorDiffer.stubs(:diff).returns([])
-      result = Compatriot.percentage_changed(page, stubbed_test, 'and has a description')
+      result = Compatriot.percentage_changed( stubbed_test, 'and has a description')
       assert_equal(0.0, result.round(2))
     end
   end
@@ -84,7 +72,7 @@ describe Compatriot do
     end
 
     it 'can set the screenshot directory' do
-      Compatriot.take_screenshot(page, stubbed_test, 'and has a description')
+      Compatriot.take_screenshot( stubbed_test, 'and has a description')
       assert File.exist?(@control_file), 'control image not found'
     end
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: compatriot
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Carol (Nichols || Goulding)
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-01-05 00:00:00.000000000 Z
+date: 2016-02-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -200,6 +200,7 @@ files:
 - compatriot.gemspec
 - examples/compatriot_tests.rb
 - lib/compatriot.rb
+- lib/compatriot/assertions.rb
 - lib/compatriot/browser.rb
 - lib/compatriot/image_differ/color_differ.rb
 - lib/compatriot/image_differ/image_differ.rb
@@ -211,6 +212,7 @@ files:
 - spec/sample_app/public/images/smileyface.jpg
 - spec/sample_app/sample_app.rb
 - spec/spec_helper.rb
+- spec/unit/assertions_spec.rb
 - spec/unit/browser_spec.rb
 - spec/unit/compatriot_spec.rb
 - spec/unit/image_differ/color_differ_spec.rb
@@ -248,6 +250,7 @@ test_files:
 - spec/sample_app/public/images/smileyface.jpg
 - spec/sample_app/sample_app.rb
 - spec/spec_helper.rb
+- spec/unit/assertions_spec.rb
 - spec/unit/browser_spec.rb
 - spec/unit/compatriot_spec.rb
 - spec/unit/image_differ/color_differ_spec.rb