wraith 1.3.7 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4ef3702111da955cdc52b3cafc7393003ada3d73
-  data.tar.gz: 4c0a4caa2ab48825a3551fa574e3a7cb3aa9492d
+  metadata.gz: 1d01c988c9cf4c296a6ed667f355fc88bbed62b5
+  data.tar.gz: 1a6ecf623f3639c980148cc964d413bc15c66716
 SHA512:
-  metadata.gz: e01746b3ad8b4327425c35c3cf49880a538b4380ac573a9aa280273613b26261d574f6fc11f70bbe8a5f8a14e89719205fd206f870d4c48d7cecf87d460cf612
-  data.tar.gz: b6d2f51c5c97eee922adbe00d34922d0cc96a8673f20643e81880e937557e23fe7977b3125fc368ad6710469c99093cd8afeae36d09ec83d87812361968b8d23
+  metadata.gz: 226e733d3f7c7ae0fd19c2292238aa9abedd89bbffb5721c0af1b4c0d970385a154bd0c68a1c65b9e85d65c4c9fb7bd8161fb7f12d6d89d348a745065b42149e
+  data.tar.gz: a7a2328d86e60ce253030698cddcc08c6ccce27007b377437a4878c59bb18759e82f3e9bc07425c3b393f0e4aac98b9a6f8c0f9a7ed0a9317d73359bc180d0d8

data/.gitignore

@@ -1,5 +1,6 @@
 .bundle/*
 shots/
+shots_history/
 data.txt
 .DS_Store
 *.swp

data/README.md

@@ -47,18 +47,29 @@ You can type `wraith` into terminal to bring up the list of commands, but the on
 wraith capture config_name
 ```
 
-This assumes that your snap.js and config.yaml are in the folders that were created on setup. There are other commands also available, these all expect a config_name to be passed as an option. Wraith will look for the config file at `configs/[config_name].yaml`.  
+This assumes that your snap.js and config.yaml are in the folders that were created on setup. To run the setup, create a folder and inside run
 
 ```sh
-  wraith capture config_name             # A full Wraith job
-  wraith compare_images config_name      # compares images to generate diffs
-  wraith crop_images config_name         # crops images to the same height
-  wraith setup_folders config_name       # create folders for images
-  wraith generate_gallery config_name    # create page for viewing images
-  wraith generate_thumbnails config_name # create thumbnails for gallery
-  wraith reset_shots config_name         # removes all the files in the shots folder
-  wraith save_images config_name         # captures screenshots
-  wraith setup                           # creates config folder and default config
+wraith setup
+```
+
+### CLI
+
+There are other commands also available, these all expect a config_name to be passed as an option. Wraith will look for the config file at `configs/[config_name].yaml`.  
+
+```sh
+  wraith capture [config_name]              # A full Wraith job
+  wraith compare_images [config_name]       # compares images to generate diffs
+  wraith crop_images [config_name]          # crops images to the same height
+  wraith generate_gallery [config_name]     # create page for viewing images
+  wraith generate_thumbnails [config_name]  # create thumbnails for gallery
+  wraith history [config_name]              # Setup a baseline set of shots
+  wraith latest [config_name]               # Capture new shots to compare with baseline
+  wraith multi_capture [filelist]           # A Batch of Wraith Jobs
+  wraith reset_shots [config_name]          # removes all the files in the shots folder
+  wraith save_images [config_name]          # captures screenshots
+  wraith setup                              # creates config folder and default config
+  wraith setup_folders [config_name]        # create folders for images
 ```
 
 ## Output
@@ -69,6 +80,25 @@ After each screenshot is captured, the compare task will run, this will output a
 
 A gallery is available to view each of the images and the respective diff images located in the shots folder once all the images have been compared.
 
+## History usage
+
+In Wraith 2.0 we introduced a new way for using Wraith in development and testing, historical shots.  Rather than capture 2 domains, you capture 1 domain, probably your local copy of the latest code, then later on after development is in progress, run Wraith again to compare.  This makes working against an isolated dev environment much easier as you wont need an internet connection.  
+
+The usage is different in that you need 1 domain in your config and you will need to set a historical shots folder.  An example of this can be found on Github.  The way this works is that shots are captured as normal with the history command below.  This will create 2 folders with the labels you have specified in the config file, for example, shots and shots_history.  The original shots will be copied into your 'history' folder, then copied back into the shots folder once you have run your latest job.  This workflow will essentially mean your history folder is your baseline, being copied back into the shots folder every time you run the 'latest command'
+
+```sh
+wraith history history.yaml
+```
+After some development, run the latest command
+```sh
+wraith latest history.yaml
+```
+You will now be able to run the latest command over and over without having to do clear up.
+
+## Changelog - updated 2014-09-25
+Wraith 2.0 with new history support, this is a big change for the usage of Wraith, with less dependency on having an Internet connection and capturing two domains.
+A large change in the way file names are made has been introduced into 2.0, with the label of the engine now being used instead of the engine string itself.  This has been updated in all the included configs for reference, but will mean a change to the filenames and urls.  We have also changed the way that folders are loaded from the config, dropping the use of arrays.  This is backward compatible.  
+
 ## Contributing
 
 If you want to add functionality to this project, pull requests are welcome.
@@ -83,9 +113,6 @@ If you want to add functionality to this project, pull requests are welcome.
 
 **Please raise any issues with this project as a GitHub issue.**
 
-## Changelog - updated 2014-05-14
-We have updated Wraith some bug fixes and more importantly, parallel support.  This will now enable use of all your CPU cores for faster Wraith runs.  There is error handling in place to ensure if an image fails to capture, it will try again 5 times, if that fails, it will put an holding image in place.  This should solve all the issues with images not capturing and failing to crop/compare.  Big thanks to [Matt Senior](https://github.com/mattsenior) for doing the work.
-
 ## License
 
 Wraith is available to everyone under the terms of the Apache 2.0 open source license.
@@ -96,10 +123,10 @@ Take a look at the LICENSE file in the code.
  * [Dave Blooman](http://twitter.com/dblooman)
  * [John Cleveley](http://twitter.com/jcleveley)
  * [Simon Thulbourn](http://twitter.com/sthulbourn)
- 
+
 ## Selenium-Wraith
 
-Anyone interested in integrating selenium capability with Wraith should check out 
+Anyone interested in integrating selenium capability with Wraith should check out
 Selenium-Wraith by Andrew Tekle-Cadman of Future Visible.
 
 Selenium-Wraith was forked from the BBC repo on 16/04/14 and adds the following capabilities to Wraith

data/configs/config.yaml

@@ -1,14 +1,13 @@
 #Headless browser option
 browser:
-  webkit: "phantomjs"
-  # gecko: "slimerjs"
+  phantomjs: "phantomjs"
+  # slimerjs: "slimerjs"
 
 #If you want to have multiple snapping files, set the file name here
 snap_file: "lib/wraith/javascript/snap.js"
 
 # Type the name of the directory that shots will be stored in
-directory:
-  - 'shots'
+directory: 'shots'
 
 # Add only 2 domains, key will act as a label
 domains:
@@ -28,13 +27,6 @@ paths:
   home: /
   uk_index: /uk
 
-# If you don't want to name the paths explicitly you can use a yaml
-# collection as follows, and names will be derived by replacing / with _
-#
-# paths:
-#  - /
-#  - /uk
-
 #Amount of fuzz ImageMagick will use
 fuzz: '20%'
 
@@ -45,7 +37,7 @@ fuzz: '20%'
 spider_days:
   - 10
 
-#A list of URLs to skip when spidering. Ruby regular expressions can be 
+#A list of URLs to skip when spidering. Ruby regular expressions can be
 #used, if prefixed with !ruby/regexp as defined in the YAML Cookbook
 #http://www.yaml.org/YAML_for_ruby.html#regexps
 #

data/configs/config_nojs.yaml

@@ -1,14 +1,13 @@
 #Headless browser option
 browser:
-  webkit: "phantomjs"
-  # gecko: "slimerjs"
+  phantomjs: "phantomjs"
+  # slimerjs: "slimerjs"
 
 #If you want to have multiple snapping files, set the file name here
 snap_file: "lib/wraith/javascript/nojs.js"
 
 # Type the name of the directory that shots will be stored in
-directory:
-  - 'shots_nojs'
+directory: 'shots_nojs'
 
 # Add only 2 domains, key will act as a label
 domains:
@@ -28,13 +27,6 @@ paths:
   home: /
   uk_index: /uk
 
-# If you don't want to name the paths explicitly you can use a yaml
-# collection as follows, and names will be derived by replacing / with _
-#
-# paths:
-#  - /imghp
-#  - /maps
-
 #Amount of fuzz ImageMagick will use
 fuzz: '20%'
 
@@ -42,7 +34,7 @@ fuzz: '20%'
 spider_days:
   - 10
 
-    #A list of URLs to skip when spidering. Ruby regular expressions can be 
+    #A list of URLs to skip when spidering. Ruby regular expressions can be
 #used, if prefixed with !ruby/regexp as defined in the YAML Cookbook
 #http://www.yaml.org/YAML_for_ruby.html#regexps
 #

data/configs/history.yaml

@@ -0,0 +1,52 @@
+#Headless browser option
+browser:
+  phantomjs: "phantomjs"
+  # slimerjs: "slimerjs"
+
+#If you want to have multiple snapping files, set the file name here
+snap_file: "lib/wraith/javascript/snap.js"
+
+# Type the name of the directory that shots will be stored in
+directory: 'shots'
+history_dir: 'shots_history'
+
+# Add only 2 domains, key will act as a label
+domains:
+  news: "http://www.live.bbc.co.uk/news"
+
+#Type screen widths below, here are a couple of examples
+screen_widths:
+  - 320
+  - 600
+  - 768
+  - 1024
+  - 1280
+
+#Type page URL paths below, here are a couple of examples
+paths:
+  home: /
+  uk_index: /uk
+
+#Amount of fuzz ImageMagick will use
+fuzz: '20%'
+
+#Set the filename of the spider file to use, if not specified it will fallback to spider.txt
+# spider_file: bbc_co_uk_spider.txt
+
+#Set the number of days to keep the site spider file
+spider_days:
+  - 10
+
+#A list of URLs to skip when spidering. Ruby regular expressions can be
+#used, if prefixed with !ruby/regexp as defined in the YAML Cookbook
+#http://www.yaml.org/YAML_for_ruby.html#regexps
+#
+# spider_skips:
+#   - /foo/bar.html # Matches /foo/bar.html explcitly
+#   - !ruby/regexp /^\/baz\// # Matches any URLs that start with /baz
+
+#Choose how results are displayed, by default alphanumeric. Different screen widths are always grouped.
+#alphanumeric - all paths (with, and without, a difference) are shown, sorted by path
+#diffs_first - all paths (with, and without, a difference) are shown, sorted by difference size (largest first)
+#diffs_only - only paths with a difference are shown, sorted by difference size (largest first)
+#mode: diffs_first

data/configs/test_config.yaml

@@ -1,14 +1,13 @@
 #Headless browser option
 browser:
-  webkit: "phantomjs"
-  # gecko: "./slimerjs"
+  phantomjs: "phantomjs"
+  # slimerjs: "slimerjs"
 
 #If you want to have multiple snapping files, set the file name here
 snap_file: "lib/wraith/javascript/snap.js"
 
 # Type the name of the directory that shots will be stored in
-directory:
-  - 'shots'
+directory: 'shots'
 
 # Add only 2 domains, key will act as a label
 domains:
@@ -28,26 +27,5 @@ paths:
   home: /
   uk_index: /uk
 
-# If you don't want to name the paths explicitly you can use a yaml
-# collection as follows, and names will be derived by replacing / with _
-#
-# paths:
-#  - /imghp
-#  - /maps
-
 #Amount of fuzz ImageMagick will use
 fuzz: '20%'
-
-#Set the filename of the spider file to use, if not specified it will fallback to spider.txt
-# spider_file: bbc_co_uk_spider.txt
-
-#Set the number of days to keep the site spider file
-spider_days:
-  - 10
-#A list of URLs to skip when spidering. Ruby regular expressions can be 
-#used, if prefixed with !ruby/regexp as defined in the YAML Cookbook
-#http://www.yaml.org/YAML_for_ruby.html#regexps
-#
-# spider_skips:
-#   - /foo/bar.html # Matches /foo/bar.html explcitly
-#   - !ruby/regexp /^\/baz\// # Matches any URLs that start with /baz

data/lib/wraith/cli.rb

@@ -40,11 +40,21 @@ class Wraith::CLI < Thor
       spider = Wraith::Spidering.new(config_name)
       spider.check_for_paths
     end
+
+    def copy_old_shots(config_name)
+      create = Wraith::FolderManager.new(config_name)
+      create.copy_old_shots
+    end
+
+    def restore_shots(config_name)
+      create = Wraith::FolderManager.new(config_name)
+      create.restore_shots
+    end
   end
 
   desc 'save_images [config_name]', 'captures screenshots'
-  def save_images(config_name)
-    save_images = Wraith::SaveImages.new(config_name)
+  def save_images(config_name, history = false)
+    save_images = Wraith::SaveImages.new(config_name, history)
     save_images.save_images
   end
 
@@ -98,11 +108,14 @@ class Wraith::CLI < Thor
     reset_shots(config)
     setup_folders(config)
     save_images(config)
+    copy_old_shots(config)
   end
 
   desc 'latest [config_name]', 'Capture new shots to compare with baseline'
   def latest(config)
-    save_images(config)
+    reset_shots(config)
+    restore_shots(config)
+    save_images(config, true)
     crop_images(config)
     compare_images(config)
     generate_thumbnails(config)

data/lib/wraith/folder.rb

@@ -11,6 +11,10 @@ class Wraith::FolderManager
     wraith.directory
   end
 
+  def history_dir
+    wraith.history_dir
+  end
+
   def paths
     wraith.paths
   end
@@ -29,6 +33,15 @@ class Wraith::FolderManager
     FileUtils.mkdir_p("#{dir}")
   end
 
+  def copy_old_shots
+    FileUtils.cp_r(dir, history_dir)
+  end
+
+  def restore_shots
+    puts 'restoring'
+    FileUtils.cp_r(Dir.glob("#{history_dir}/*"), dir)
+  end
+
   def create_folders
     spider_paths.each do |folder_label, path|
       unless path

data/lib/wraith/gallery.rb

@@ -37,43 +37,68 @@ class Wraith::GalleryGenerator
   def match(categories, dirname)
     categories.each do |category|
       @dirs[category] = {}
+
       Dir.foreach("#{dirname}/#{category}") do |filename|
         match = MATCH_FILENAME.match(filename)
         unless match.nil?
-          size = match[1].to_i
-          group = match[2]
-          filepath = category + '/' + filename
-          thumbnail = "thumbnails/#{category}/#{filename}"
-
-          if @dirs[category][size].nil?
-            @dirs[category][size] = { variants: [] }
-          end
-          size_dict = @dirs[category][size]
-
-          case group
-          when 'diff'
-            size_dict[:diff] = {
-              filename: filepath, thumb: thumbnail
-            }
-          when 'data'
-            size_dict[:data] = File.read("#{dirname}/#{filepath}").to_f
-          else
-            size_dict[:variants] << {
-              name: group,
-              filename: filepath,
-              thumb: thumbnail
-            }
-
-          end
-          size_dict[:variants].sort! { |a, b| a[:name] <=> b[:name] }
+          matcher(match, filename, dirname, category)
         end
       end
     end
     @folder_manager.tidy_shots_folder(@dirs)
+    sorting_dirs(@dirs)
+  end
+
+  def matcher(match, filename, dirname, category)
+    @size = match[1].to_i
+    @group = match[2]
+    @filepath = category + '/' + filename
+    @thumbnail = "thumbnails/#{category}/#{filename}"
+
+    if @dirs[category][@size].nil?
+      @dirs[category][@size] = { variants: [] }
+    end
+
+    size_dict = @dirs[category][@size]
+
+    data_group(@group, size_dict, dirname, @filepath)
+  end
+
+  def data_group(group, size_dict, dirname, filepath)
+    case group
+    when 'diff'
+      diff_check(size_dict, filepath)
+    when 'data'
+      data_check(size_dict, dirname, filepath)
+    else
+      variant_check(size_dict, group)
+    end
+  end
+
+  def variant_check(size_dict, group)
+    size_dict[:variants] << {
+      name: group,
+      filename: @filepath,
+      thumb: @thumbnail
+    }
+    size_dict[:variants].sort! { |a, b| a[:name] <=> b[:name] }
+  end
+
+  def diff_check(size_dict, filepath)
+    size_dict[:diff] = {
+      filename: filepath, thumb: @thumbnail
+    }
+  end
+
+  def data_check(size_dict, dirname, filepath)
+    size_dict[:data] = File.read("#{dirname}/#{filepath}").to_f
+  end
+
+  def sorting_dirs(dirs)
     if %w(diffs_only diffs_first).include?(wraith.mode)
-      @sorted = @dirs.sort_by { |_category, sizes| -1 * sizes.max_by { |_size, dict| dict[:data] }[1][:data] }
+      @sorted = dirs.sort_by { |_category, sizes| -1 * sizes.max_by { |_size, dict| dict[:data] }[1][:data] }
     else
-      @sorted = @dirs.sort_by { |category, _sizes| category }
+      @sorted = dirs.sort_by { |category, _sizes| category }
     end
     # The sort has made this into an enumerable, convert it back to a Hash
     Hash[@sorted]

data/lib/wraith/save_images.rb

@@ -2,10 +2,11 @@ require 'wraith'
 require 'parallel'
 
 class Wraith::SaveImages
-  attr_reader :wraith
+  attr_reader :wraith, :history
 
-  def initialize(config)
+  def initialize(config, history = false)
     @wraith = Wraith::Wraith.new(config)
+    @history = history
   end
 
   def directory
@@ -21,10 +22,18 @@ class Wraith::SaveImages
     end
   end
 
+  def history_label
+    history ? '_latest' : ''
+  end
+
   def engine
     wraith.engine.each { |_label, browser| return browser }
   end
 
+  def engine_label
+    wraith.engine.key(engine)
+  end
+
   def base_urls(path)
     wraith.base_domain + path unless wraith.base_domain.nil?
   end
@@ -34,7 +43,7 @@ class Wraith::SaveImages
   end
 
   def file_names(width, label, domain_label)
-    "#{directory}/#{label}/#{width}_#{engine}_#{domain_label}.png"
+    "#{directory}/#{label}/#{width}_#{engine_label}_#{domain_label}.png"
   end
 
   def attempt_image_capture(width, url, filename, max_attempts)
@@ -61,14 +70,17 @@ class Wraith::SaveImages
       compare_url = compare_urls(path)
 
       wraith.widths.each do |width|
-        base_file_name    = file_names(width, label, wraith.base_domain_label)
-        compare_file_name = file_names(width, label, wraith.comp_domain_label)
+        base_file_name    = file_names(width, label, "#{wraith.base_domain_label}#{history_label}")
+        compare_file_name = file_names(width, label, "#{wraith.comp_domain_label}#{history_label}")
 
         jobs << [label, path, width, base_url,    base_file_name]
         jobs << [label, path, width, compare_url, compare_file_name] unless compare_url.nil?
       end
     end
+    parallel_task(jobs)
+  end
 
+  def parallel_task(jobs)
     Parallel.each(jobs, in_threads: 8) do |_label, _path, width, url, filename|
       begin
         attempt_image_capture(width, url, filename, 5)

data/lib/wraith/thumbnails.rb

@@ -22,7 +22,7 @@ class Wraith::Thumbnails
 
   def thumbnail_image(png_path, output_path)
     unless File.directory?(File.dirname(output_path))
-        FileUtils.mkdir_p(File.dirname(output_path));
+      FileUtils.mkdir_p(File.dirname(output_path))
     end
 
     `convert #{png_path} -thumbnail 200 -crop 200x200+0+0 #{output_path}`

data/lib/wraith/version.rb

@@ -1,3 +1,3 @@
 module Wraith
-  VERSION = '1.3.7'
+  VERSION = '2.0.0'
 end

data/lib/wraith/wraith.rb

@@ -15,7 +15,12 @@ class Wraith::Wraith
   end
 
   def directory
-    @config['directory'].first
+    # Legacy support for those using array configs
+    @config['directory'].is_a?(Array) ? @config['directory'].first : @config['directory']
+  end
+
+  def history_dir
+    @config['history_dir']
   end
 
   def snap_file

data/spec/wraith_spec.rb

@@ -19,7 +19,7 @@ describe Wraith do
 
   context 'When creating a wraith worker' do
 
-    Then { wraith.config.keys.size == 8 }
+    Then { wraith.config.keys.size == 7 }
     Then { wraith.widths == [320, 600, 768, 1024, 1280] }
     Then { wraith.base_domain == 'http://pal.live.bbc.com/news' }
     Then { wraith.comp_domain == 'http://pal.live.bbc.co.uk/russian' }
@@ -60,8 +60,8 @@ describe Wraith do
       end
       Wraith::CropImages.new(config_name).crop_images
       Wraith::CompareImages.new(config_name).compare_task(test_image1, test_image2, diff_image, data_txt)
-      Wraith::Thumbnails.new(config_name).generate_thumbnails()
-	end
-    Then { File.exists?('shots/thumbnails/test/test1.png') && File.exists?('shots/thumbnails/test/test2.png') && File.exists?('shots/thumbnails/test/test_diff.png') }
+      Wraith::Thumbnails.new(config_name).generate_thumbnails
+	   end
+    Then { File.exist?('shots/thumbnails/test/test1.png') && File.exist?('shots/thumbnails/test/test2.png') && File.exist?('shots/thumbnails/test/test_diff.png') }
   end
 end

data/templates/configs/config.yaml

@@ -1,14 +1,13 @@
 #Headless browser option
 browser:
-  webkit: "phantomjs"
-  # gecko: "slimerjs"
+  phantomjs: "phantomjs"
+  # slimerjs: "slimerjs"
 
 #If you want to have multiple snapping files, set the file name here
-snap_file: "javascript/snap.js"
+snap_file: "lib/wraith/javascript/snap.js"
 
 # Type the name of the directory that shots will be stored in
-directory:
-  - 'shots'
+directory: 'shots'
 
 # Add only 2 domains, key will act as a label
 domains:
@@ -28,16 +27,26 @@ paths:
   home: /
   uk_index: /uk
 
-# If you don't want to name the paths explicitly you can use a yaml
-# collection as follows, and names will be derived by replacing / with _
-#
-# paths:
-#  - /imghp
-#  - /maps
-
 #Amount of fuzz ImageMagick will use
 fuzz: '20%'
 
+#Set the filename of the spider file to use, if not specified it will fallback to spider.txt
+# spider_file: bbc_co_uk_spider.txt
+
 #Set the number of days to keep the site spider file
 spider_days:
   - 10
+
+#A list of URLs to skip when spidering. Ruby regular expressions can be
+#used, if prefixed with !ruby/regexp as defined in the YAML Cookbook
+#http://www.yaml.org/YAML_for_ruby.html#regexps
+#
+# spider_skips:
+#   - /foo/bar.html # Matches /foo/bar.html explcitly
+#   - !ruby/regexp /^\/baz\// # Matches any URLs that start with /baz
+
+#Choose how results are displayed, by default alphanumeric. Different screen widths are always grouped.
+#alphanumeric - all paths (with, and without, a difference) are shown, sorted by path
+#diffs_first - all paths (with, and without, a difference) are shown, sorted by difference size (largest first)
+#diffs_only - only paths with a difference are shown, sorted by difference size (largest first)
+#mode: diffs_first

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: wraith
 version: !ruby/object:Gem::Version
-  version: 1.3.7
+  version: 2.0.0
 platform: ruby
 authors:
 - Dave Blooman
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-09-12 00:00:00.000000000 Z
+date: 2014-09-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: pry
@@ -173,6 +173,7 @@ files:
 - bin/wraith
 - configs/config.yaml
 - configs/config_nojs.yaml
+- configs/history.yaml
 - configs/test_config.yaml
 - lib/wraith.rb
 - lib/wraith/cli.rb