kontrast 0.2.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: ccae0e8fea2906d0f534b11feee35bd3690fc4d2
+  data.tar.gz: 0d94b6a934294b8ffb3e480abceb09f5263e637d
+SHA512:
+  metadata.gz: b36e5e0f5d4edf9945f7fe6183b3a8accb2ddcbf1bbe29ded50ce0f236c7a9735e4e34e72ba0addfeaec46e3ec807f38753920401bcfaaa3be53f660133e8d35
+  data.tar.gz: a011155d29e1817de408fb029941856721be1fc05f2e02e1f0afdffe7b25def4b7bff43d92687f5c8952044d18a7fa4f67272037c399540f9b6def91d99c892a

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2014 Ilya Rubnich
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,262 @@
+# Kontrast
+
+An automated testing tool for comparing visual differences between two versions of a website.
+
+Kontrast lets you build a test suite to run against your test and production websites. It uses [Selenium](http://www.seleniumhq.org/) to take screenshots and [ImageMagick](http://www.imagemagick.org/) to compare them. Kontrast then produces a detailed gallery of its test results.
+
+## Prerequisites
+
+1. Install ImageMagick. You can do this on OS X via brew with:
+
+		$ brew install imagemagick
+
+2. Make sure you have Firefox or a different Selenium-compatible browser installed. By default, Firefox is used.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'kontrast'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install kontrast
+
+Lastly, generate the config file:
+
+	$ kontrast generate_config
+
+If you're in Rails, the config file will be generated in `config/initializers/kontrast.rb`.  
+Otherwise, the config file will be generated in your current directory.
+
+## Basic Configuration
+
+Here's all the config you need to get started:
+
+	Kontrast.configure do |config|
+		# Set your test and production domains
+		config.test_domain = "http://localhost:3000"
+		config.production_domain = "http://www.example.com"
+
+		# Build your test suite
+		# These pages will open in a 1280px-wide browser
+		config.pages(1280) do |page|
+			page.home "/"
+			page.about "/about"
+		end
+
+		# These pages will open in a 320px-wide browser
+		config.pages(320) do |page|
+			page.home "/"
+			page.about "/about"
+		end
+	end
+
+## Basic Usage
+Run Kontrast (use `bundle exec` and omit the --config flag if you're within a Rails app):
+
+	$ kontrast local_run --config ./kontrast_config.rb
+	...
+	...
+	...
+	Kontrast is all done!
+	You can find the gallery at: /tmp/shots/1410309651/gallery/gallery.html
+
+Review the gallery in your Favorite Browser:
+
+	$ open /tmp/shots/1410309651/gallery/gallery.html
+	
+## Parallelized Usage
+We designed Kontrast from the very beginning to work with multiple nodes. At Harry's, we use CircleCI for testing and Kontrast works perfectly with CircleCI's multi-container features.
+
+### Method of Action
+
+Because we ultimately need to generate a gallery with all test results from all given nodes, Kontrast uploads the test images it creates plus a per-node manifest file to S3. After all the tests have run, a single node downloads the manifest files and parses them to create a single gallery.
+
+Here's how to get set up:
+
+### 1. Enable Parallelization
+
+	config.run_parallel = true
+
+### 2. Configure Nodes
+Set how many nodes you have in total and the zero-based index of the current node. Kontrast will automatically split up tests among these nodes.
+
+	config.total_nodes = 6
+    config.current_node = 2
+    
+### 3. Configure Remote Options
+Set your S3 details:
+
+    config.aws_bucket = "kontrast-test-results"
+    config.aws_key = ENV['AWS_KEY']
+    config.aws_secret = ENV['AWS_SECRET']
+    
+Set the **local** path where output images will be stored before they are uploaded to S3. This is also where the gallery will be saved on the node that runs the `make_gallery` command. This path will be created if it doesn't already exist.
+
+    config.local_path = "tmp/kontrast"
+    
+Set the **remote** path relative to your S3 bucket's root where Kontrast's output files will be uploaded to. It should be unique to every test.
+
+    config.remote_path = "artifacts.#{ENV['BUILD_NUMBER']}"
+    
+### 4. Run the Tests
+This command should run in parallel on every node. Use `bundle exec` and omit the --config flag if your app is `bundle`'d along with Rails.
+
+	$ kontrast run_tests --config /path/to/config.rb
+
+### 5. Create the Gallery
+This command should only run on one node after all the other nodes have completed the previous command. Use `bundle exec` and omit the --config flag if your app is `bundle`'d along with Rails.
+
+	$ kontrast make_gallery --config /path/to/config.rb
+	
+### 6. Review Your Results
+At this point, the gallery should be saved to `config.local_path` and uploaded to `config.remote_path`. Check it out in your Favorite Browser.
+
+### Sample circle.yml
+Here's an example of how to run Kontrast within a Rails app using CircleCI:
+
+	test:
+  		post:
+      		- bundle exec rails server:
+          		background: true
+          		parallel: true
+      		- bundle exec kontrast run_tests:
+          		parallel: true
+          	- bundle exec kontrast make_gallery
+
+## Advanced Configuration
+
+### Test Suite
+
+#### fail_build
+If you want Kontrast to exit with an error code if it finds any diffs, use this option:
+
+	config.fail_build = true
+
+### Selenium Driver
+#### browser_driver
+Choose which Selenium driver you'd like to use. Kontrast has only been tested on the default Firefox driver but we would love feedback and/or pull requests for other drivers.
+
+	config.browser_driver = "firefox"
+	
+#### browser_profile
+You may set a driver's profile options in this hash.
+
+	config.browser_profile = {
+        "general.useragent.override" => "Some Cool Kontrast User Agent",
+        "image.animation_mode" => "none"
+    }
+
+### Image Comparisons
+#### distortion_metric
+See [http://www.imagemagick.org/RMagick/doc/constants.html#MetricType]() for available values.
+
+	config.distortion_metric = "MeanAbsoluteErrorMetric"
+	
+#### highlight_color
+The ImageMagick comparison tool emphasizes differences with this color.
+Valid options are an RMagick color name or pixel.
+
+	config.highlight_color = "blue"
+	
+#### lowlight_color
+The ImageMagick comparison tool deemphasizes differences with this color.
+Valid options are an RMagick color name or pixel.
+
+	config.lowlight_color = "rgba(255, 255, 255, 0.3)"
+
+### Hooks
+To make Kontrast even more powerful, we provided a set of hooks that you can use in your configuration.
+
+#### before_run
+Runs before the entire suite.
+
+	config.before_run do
+		WebMock.disable!
+	end
+
+#### after_run
+Runs after the entire suite.
+
+	config.after_run do
+		WebMock.enable!
+	end
+
+#### before_gallery
+Runs before the gallery creation step.
+
+	config.before_gallery do
+		WebMock.disable!
+	end
+
+#### after_gallery
+Runs after the gallery creation step.
+
+	config.after_gallery do |diffs, gallery_path|
+		# diffs is a hash containing all the differences that Kontrast found in your test suite
+		# gallery_path is where Kontrast saved the gallery
+	end
+
+#### before_screenshot
+Runs on every test before Selenium takes a screenshot.
+
+	config.before_screenshot do |test_driver, production_driver, test_info|
+		# test_driver and production_driver are instances of Selenium::WebDriver that you can control
+		# test_info is a hash with the current test's name and width
+	end
+
+#### after_screenshot
+Runs on every test after Selenium takes a screenshot.
+
+	config.after_screenshot do |test_driver, production_driver, test_info|
+		# same variables are available as with before_screenshot
+	end
+	
+## Customizing Kontrast
+Kontrast's hooks allow you to insert custom functionality into many parts of the test suite. Here are some examples of how we use hooks at Harry's:
+
+### Integrating with HipChat
+Once a build finishes, we let HipChat know if Kontrast found any diffs using the `hipchat` gem:
+
+	config.after_gallery do |diffs, gallery_path|
+		hipchat_room = "Kontrast Results"
+        hipchat_user = "KontrastBot"
+
+        if !diffs.empty?
+            msg = "Kontrast Diffs: #{diffs.keys.join(', ')}. Don't push to production without reviewing these. You can find the gallery at #{gallery_path}."
+            client = HipChat::Client.new(ENV["HIPCHAT_TOKEN"])
+            client[hipchat_room].send(hipchat_user, msg, :color => "red")
+        end
+	end
+	
+### Setting Cookies
+Testing our cart page required a bit more setup before we could take a screenshot of it:
+
+	config.before_screenshot do |test_driver, production_driver, test|
+		if test[:name] == "cart"
+			# prepare our cookie value
+       		cookie_value = super_secret_magic_cart_cookie
+       	
+       		# write cookies using Mootools
+       		# http://mootools.net/docs/core/Utilities/Cookie
+        	test_driver.execute_script("Cookie.write('cart', '#{cookie_value}');")
+        	production_driver.execute_script("Cookie.write('cart', '#{cookie_value}');")
+        
+        	# refresh the page
+        	test_driver.navigate.refresh
+        	production_driver.navigate.refresh
+    	end
+    end
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,2 @@
+require "bundler/gem_tasks"
+

data/bin/kontrast

@@ -0,0 +1,111 @@
+#!/usr/bin/env ruby
+
+#######################################
+# This is the entry point to Kontrast #
+#######################################
+
+begin
+    require "rubygems"
+    require "kontrast"
+rescue LoadError => e
+    puts "Could not load Kontrast."
+    raise e
+end
+require "thor"
+
+module Kontrast
+    class CLI < Thor
+        class_option :config
+
+        desc "run_tests", "Run Kontrast test suite"
+        def run_tests
+            load_config(options[:config])
+            Kontrast.run
+        end
+
+        desc "make_gallery --result-path PATH", "Create gallery given an optional local path of test results"
+        option :result_path
+        def make_gallery
+            load_config(options[:config])
+
+            # We're only allowed to give no path in the remote case
+            if options[:result_path].nil? && !Kontrast.configuration.run_parallel
+                raise GalleryException.new("You can't omit --result-path when running in local mode")
+            end
+
+            result = Kontrast.make_gallery(options[:result_path])
+            abort if !result && Kontrast.configuration.fail_build
+        end
+
+        # todo: option to specify an output path
+        desc "local_run", "Run Kontrast locally"
+        def local_run
+            load_config(options[:config])
+
+            # Make sure config run_parallel is set to false
+            if Kontrast.configuration.run_parallel
+                raise ConfigurationException.new("You can't run in parallel locally")
+            end
+
+            # Run Kontrast
+            Kontrast.run
+            result = Kontrast.make_gallery(Kontrast.path)
+
+            # Print the gallery path
+            puts "Kontrast is all done!"
+            puts "You can find the gallery at: #{Kontrast.path}/gallery/gallery.html"
+
+            # Quit with an error code if appropriate
+            abort if !result && Kontrast.configuration.fail_build
+        end
+
+        desc "generate_config", "Generate a Kontrast configuration file"
+        def generate_config
+            template = File.read(Kontrast.root + '/lib/kontrast/config/template.rb')
+
+            if Kontrast.in_rails?
+                target_file = './config/initializers/kontrast.rb'
+            else
+                target_file = './kontrast_config.rb'
+            end
+
+            File.open(target_file, 'w') do |f|
+                f.write(template)
+            end
+
+            puts "Created a Kontrast config file at: #{target_file}"
+        end
+
+        private
+            def load_config(config)
+                # Let's check if we're within Rails.
+                if !Kontrast.in_rails?
+                    begin
+                        require config
+                    rescue TypeError => e
+                        raise ConfigurationException.new("Error parsing the config flag '#{config}'")
+                    rescue LoadError => e
+                        raise ConfigurationException.new("Could not load '#{config}'")
+                    rescue Exception => e
+                        raise ConfigurationException.new("An unexpected error occurred while trying to load the given config file: #{e.inspect}")
+                    end
+                else
+                    # Load Rails environment
+                    # We will assume the config was loaded from an initializer
+                    require './config/environment'
+                end
+
+                # Check that we actually got a configuration block
+                if !Kontrast.configuration
+                    raise ConfigurationException.new("No configuration has been loaded")
+                end
+
+                # Make sure we have the bare minimum configuration to continue
+                Kontrast.configuration.validate
+
+                return true
+            end
+    end
+end
+
+Kontrast::CLI.start(ARGV)

data/lib/kontrast.rb

@@ -0,0 +1,97 @@
+# Dependencies
+require "fog"
+require "bundler"
+
+# Load classes
+require "kontrast/exceptions"
+require "kontrast/configuration"
+require "kontrast/test_builder"
+require "kontrast/selenium_handler"
+require "kontrast/image_handler"
+require "kontrast/gallery_creator"
+require "kontrast/runner"
+
+module Kontrast
+    class << self
+        @@path = nil
+
+        def root
+            File.expand_path('../..', __FILE__)
+        end
+
+        def in_rails?
+            # Logic: Rails uses Bundler, so if the Bundler environment contains Rails, return true.
+            # If there's any error whatsoever, return false.
+            begin
+                Bundler.environment.current_dependencies.each do |dep|
+                    return true if dep.name == "rails"
+                end
+            rescue Exception => e
+                # Quietly ignore any exceptions
+            end
+            return false
+        end
+
+        def path
+            return @@path if @@path
+
+            if Kontrast.configuration.run_parallel
+                @@path = FileUtils.mkdir_p(Kontrast.configuration.local_path).join('')
+            elsif Kontrast.in_rails?
+                @@path = FileUtils.mkdir_p(Rails.root + "tmp/shots/#{Time.now.to_i}").join('')
+            else
+                @@path = FileUtils.mkdir_p("/tmp/shots/#{Time.now.to_i}").join('')
+            end
+
+            return @@path
+        end
+
+        def fog
+            return Fog::Storage.new({
+                :provider                 => 'AWS',
+                :aws_access_key_id        => Kontrast.configuration.aws_key,
+                :aws_secret_access_key    => Kontrast.configuration.aws_secret
+            })
+        end
+
+        def run
+            beginning_time = Time.now
+
+            begin
+                # Call "before" hook
+                Kontrast.configuration.before_run
+
+                runner = Runner.new
+                runner.run
+            ensure
+                # Call "after" hook
+                Kontrast.configuration.after_run
+            end
+
+            end_time = Time.now
+            puts "Time elapsed: #{(end_time - beginning_time)} seconds"
+        end
+
+        def make_gallery(result_path = nil)
+            puts "Creating gallery..."
+            gallery_info = {}
+            begin
+                # Call "before" hook
+                Kontrast.configuration.before_gallery
+
+                gallery_creator = GalleryCreator.new(result_path)
+                if Kontrast.configuration.run_parallel
+                    gallery_info = gallery_creator.create_gallery(Kontrast.configuration.local_path)
+                else
+                    gallery_info = gallery_creator.create_gallery(result_path)
+                end
+            ensure
+                # Call "after" hook
+                Kontrast.configuration.after_gallery(gallery_info[:diffs], gallery_info[:path])
+            end
+
+            # Return based on if we have diffs or not
+            return gallery_info[:diffs].empty?
+        end
+    end
+end