chemistrykit 3.9.0.rc3 → 3.9.0

data/CHANGELOG.md

@@ -1,3 +1,18 @@
+#3.9.0 (2013-08-26)
+Improved reporting features and configuration for sauce permissions
+
+- Bumped version to 3.9.0 to prepare for release.
+- updated rake file to override build verify in special cases
+- final fix to a test
+- fixed issues causing tests to fail
+- updated docs to reflect recent changes and future release
+- updated report generation to include dom dumps for all available windows
+- upgraded SC version and added hooks to handle tagging sauce jobs and setting the team permission via a beaker tag, plus tests
+- screenshots front and center, hiding of success
+- now hiding success by default with off switch
+- removed specific log specification from system tests, cleaned up log configuration to just default to evidence and refactoried the cli a small bit to support it.
+- updated bundle and fixed new code quality error
+
 #3.9.0-rc.3 (2013-08-19)
 - added toggle for parts of html reports
 - added a fix to allow jenkins ci server to show the html report images

data/README.md

@@ -1,4 +1,4 @@
-#ChemistryKit 3.9.0.rc3 (2013-08-19)
+#ChemistryKit 3.9.0 (2013-08-26)
 
 [![Gem Version](https://badge.fury.io/rb/chemistrykit.png)](http://badge.fury.io/rb/chemistrykit) [![Build Status](https://travis-ci.org/arrgyle/chemistrykit.png?branch=develop)](https://travis-ci.org/jrobertfox/chef-broiler-platter) [![Code Climate](https://codeclimate.com/github/arrgyle/chemistrykit.png)](https://codeclimate.com/github/arrgyle/chemistrykit) [![Coverage Status](https://coveralls.io/repos/arrgyle/chemistrykit/badge.png?branch=develop)](https://coveralls.io/r/arrgyle/chemistrykit?branch=develop)
 
@@ -74,7 +74,11 @@ To exclude a tag, put a ~ in front of it.
 During development it is often helpful to just run a specific beaker, this can be accomplished with the `--beakers` flag:
 
     ckit brew --beakers=beakers/wip_beaker.rb
+    
+####Special Tags
+There are some tags that can be used to control various aspects of the harness. The following are supported:
 
+- `public: SOME_VALUE` - If you are running your harness against sauce labs, then you can control how the permissions are set for a particular beaker according to the visibility options detailed at the bottom of [this page](https://saucelabs.com/docs/additional-config). For example if you wanted a test to be private you could add `public: 'private'` to your beaker tags.  
 
 ###Formula Loading
 Code in the `formula` directory can be used to build out page objects and helper functions to facilitate your testing. The files are loaded in a particular way:
@@ -162,38 +166,41 @@ end
 
 You can even nest them inside different describe/context blocks and they will get executed from the outside-in.
 
-###Logs and CI Integration
-Each run of Chemistry Kit saves logging and test output to the _evidence_ directory by default. And in there will be the full set of JUnit Ant XML files that may be consumed by your CI.
+###Reporting and CI Integration
+Each run of Chemistry Kit saves logging and test output to the _evidence_ directory. And in there will be the full set of JUnit XML files that may be consumed by your CI as well as a `final_results.html` report file with a full review of all the tests.
 
-Assets generated by selenium (logs, screenshots, etc.) are stored in a subfolder with a name matching the describe block in your beaker, slugifyed like: `my_beaker_name`. This is to provide some organization but also allow the integration with jenkins using [this plugin](https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin).
+Assets generated by Selenium (logs, screenshots, etc.) are stored in a subfolder with a name matching the describe block in your beaker, slugifyed like: `my_beaker_name`. This is to provide some organization but also allow the integration with jenkins using [this plugin](https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin).
 
 ##Configuration
 ChemistryKit is configured by default with a `config.yaml` file that is created for you when you scaffold out a test harness. Relevant configuration options are detailed below:
 
-`base_url:` The base url of your app, stored to the ENV for access in your beakers and formulas.
+###Basic Options
+- `base_url:` The base url of your app, stored to the ENV for access in your beakers and formulas.
+- `retries_on_failure:` Defaults to 1, set the number of times a test should be retried on failure
+- `concurrency:` You may override the default concurrency of 1 to run the tests in parallel
 
-`retries_on_failure:` Defaults to 1, set the number of times a test should be retried on failure
+- `screenshot_on_fail` By default false, set to true to download a screenshot of the failure (supported by sauce labs for now.)
 
-`concurrency:` You may override the default concurrency of 1 to run the tests in parallel
+###Selenium Connect Options
 
-`log: path:` You may override the default log path 'evidence'
+`selenium_connect:` Options under this node override the defaults for the [Selenium Connect](https://github.com/arrgyle/selenium-connect) gem. See that documentation for specifics.
 
-`log: results_file:` You may override the default file name 'results_junit.xml'
+###Basic Auth Options
+`basic_auth:` Options under this node allow you to configure your test harness to authenticate prior to executing a test. This is helpful for testing against restricted development environments. The options include:
 
-`log: format:` You may override the default format 'junit' to an alternative like 'doc' or 'html'
+- `username:` The username to access your site with basic HTTP authentication.
+- `password:` The password to access your site with basic HTTP authentication.
+- `http_path:` An HTTP end-point loaded before each test run to cache the credentials for the test run.
+- `https_path:` An HTTPS end-point loaded before each test run to cache the credentials for the test run.
 
-`selenium_connect:` Options in this node override the defaults for the [Selenium Connect](https://github.com/arrgyle/selenium-connect) gem.
-##Command Line Usage
-
-`screenshot_on_fail` By default false, set to true to download a screenshot of the failure (supported by sauce labs for now.)
+###Split Testing Options
+`split_testing:` Currently ChemistryKit supports opting out of [Optimizely](https://www.optimizely.com/) A/B testing. The available options are:
 
-`basic_auth: username:` The username to access your site with basic HTTP authentication
+- `provider:` Currently only `optimizely` is supported.
+- `opt_out:` A value of `true` will opt you out of the A/B testing
 
-`basic_auth: password:` The password to access your site with basic HTTP authentication
 
-`basic_auth: http_path:` An HTTP end-point loaded before each test run to cache the credentials for the test run
-
-`basic_auth: https_path:` An HTTPS end-point loaded before each test run to cache the credentials for the test run
+##Command Line Usage
 
 ###new
 Creates a new ChemistryKit project.
@@ -215,7 +222,6 @@ Available options for the `brew` command:
 -a, --all                   Run every beaker regardless of tag.
 -b, --beakers [BEAKERS]     Pass a list of beaker paths to be executed.
 -c, --config [PATH]         Pass the path to an alternative config.yaml file.
--r, --results_file [NAME]   Specify the name of your results file.
 --tag [TAGS]                Specify a list of tags to run or exclude.
 --params [HASH]             Send a list of "key:value" parameters to the ENV.
 -x, --retry [INT]           How many times should a failing test be retried.

data/Rakefile

@@ -75,7 +75,7 @@ end
 
 # TODO This could probably be more cleanly automated
 desc 'Start a release (Requires Git Flow)'
-task :release_start, :version do |t, args|
+task :release_start, [:version, :no_verify] do |t, args|
   version = args['version']
 
   # make sure we have the latest stuff
@@ -90,7 +90,7 @@ task :release_start, :version do |t, args|
   system 'git pull --no-edit origin develop'
 
   # next assure all the tests run
-  task(:build_full).invoke
+  task(:build_full).invoke unless !!args['no_verify']
 
   # start the release process
   system "git flow release start #{version}"

data/chemistrykit.gemspec

@@ -2,13 +2,13 @@
 
 Gem::Specification.new do |s|
   s.name          = 'chemistrykit'
-  s.version       = '3.9.0.rc3'
+  s.version       = '3.9.0'
   s.platform      = Gem::Platform::RUBY
   s.authors       = ['Dave Haeffner', 'Jason Fox']
   s.email         = ['dave@arrgyle.com', 'jason@arrgyle.com']
   s.homepage      = 'https://github.com/arrgyle/chemistrykit'
   s.summary       = 'A simple and opinionated web testing framework for Selenium that follows convention over configuration.'
-  s.description   = 'Added basic http authentication support'
+  s.description   = 'Improved reporting features and configuration for sauce permissions'
   s.license       = 'MIT'
 
   s.files         = `git ls-files`.split($/)
@@ -24,7 +24,7 @@ Gem::Specification.new do |s|
   s.add_dependency 'builder', '~> 3.2.2'
   s.add_dependency 'selenium-webdriver', '~> 2.29.0'
   s.add_dependency 'rest-client', '~> 1.6.7'
-  s.add_dependency 'selenium-connect', '~> 3.6.0'
+  s.add_dependency 'selenium-connect', '~> 3.7.0'
   s.add_dependency 'parallel_tests', '~> 0.15.0'
   s.add_dependency 'parallel', '~> 0.7.0'
   s.add_dependency 'rspec-retry', '~> 0.2.1'

data/features/brew.feature

@@ -1,3 +1,4 @@
+@announce
 Feature: Brewing a ChemistryKit project
 
   Running ckit brew runs the suite of tests.
@@ -34,20 +35,18 @@ Feature: Brewing a ChemistryKit project
     Given a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     When I run `ckit brew`
     Then the stdout should contain "1 example, 0 failures"
     And the following files should exist:
-      | evidence/junit.xml |
+      | evidence/junit_0.xml |
       | evidence/bookie/bookie_loads_an_external_web_page/server.log        |
 
   Scenario: Brew a single beaker
     Given a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     And a file named "beakers/other_beaker.rb" with:
@@ -67,7 +66,6 @@ Feature: Brewing a ChemistryKit project
     Given a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     And a file named "beakers/other_beaker.rb" with:
@@ -88,7 +86,6 @@ Feature: Brewing a ChemistryKit project
       """
       screenshot_on_fail: true
       selenium_connect:
-          log: 'evidence'
           host: 'saucelabs'
           browser: 'iexplore'
           os: 'windows 2003'
@@ -117,7 +114,6 @@ Feature: Brewing a ChemistryKit project
       """
       retries_on_failure: 3
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     And a file named "beakers/other_beaker.rb" with:

data/features/catalyst.feature

@@ -81,7 +81,6 @@ Catalyst: n. A pocket of data (consumed from a CSV file) used to drive a test th
     And a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     When I run `ckit brew`

data/features/chemists.feature

@@ -15,7 +15,6 @@ Feature: Brewing a ChemistryKit project
       """
       screenshot_on_fail: true
       selenium_connect:
-          log: 'evidence'
           browser: 'firefox'
       """
     And a file named "formulas/basic_formula.rb" with:

data/features/concurrency.feature

@@ -32,7 +32,6 @@ Feature: Support for concurency
       """
       concurrency: 4
       selenium_connect:
-          log: 'evidence'
           host: 'saucelabs'
           browser: 'firefox'
           sauce_username: 'testing_arrgyle'

data/features/exit_status.feature

@@ -6,7 +6,6 @@ Feature: Exit Status
     And a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
 

data/features/global-config.feature

@@ -19,7 +19,6 @@ Feature: Initialize a global configuration
     """
     base_url: http://www.google.com
     selenium_connect:
-        log: 'evidence'
         host: 'localhost'
     """
     When I run `ckit brew`

data/features/load_page_objects.feature

@@ -7,7 +7,6 @@ Formulas should be loaded in the correct order with thier dependencies
     And a file named "config.yaml" with:
       """
       selenium_connect:
-          log: 'evidence'
           host: 'localhost'
       """
     And a file named "formulas/big.rb" with:

data/features/multi-config.feature

@@ -1,3 +1,4 @@
+@announce
 Feature: Support for multiple configuration files
   In order to quickly change between different configurations
   As a chemistry kit harness developer
@@ -58,13 +59,11 @@ Feature: Support for multiple configuration files
     And a file named "alternate.yaml" with:
       """
       concurrency: 4
-      log:
-          path: 'evidence_alternate'
       selenium_connect:
+          log: 'evidence_alternate'
           host: 'localhost'
       """
       When I run `ckit brew --config alternate.yaml`
       Then the stdout should contain "1 example, 0 failures"
-      And there should be "1" unique results files in the "evidence_alternate" directory
       And the following files should exist:
       | evidence_alternate/cheese/cheese_loads_an_external_web_page/server.log |

data/features/reporting.feature

@@ -1,3 +1,4 @@
+@announce
 Feature: Advanced HTML Reports
   In order to quickly know the status of the application
   As a chemistry kit harness user
@@ -44,7 +45,6 @@ Feature: Advanced HTML Reports
       """
       screenshot_on_fail: true
       selenium_connect:
-          log: 'evidence'
           browser: 'firefox'
       """
 
@@ -60,7 +60,6 @@ Feature: Advanced HTML Reports
       concurrency: 2
       screenshot_on_fail: true
       selenium_connect:
-          log: 'evidence'
           browser: 'chrome'
       """
     When I run `ckit brew`
@@ -74,7 +73,6 @@ Feature: Advanced HTML Reports
       concurrency: 2
       screenshot_on_fail: true
       selenium_connect:
-          log: 'evidence'
           host: 'saucelabs'
           browser: 'firefox'
           sauce_username: 'testing_arrgyle'
@@ -100,3 +98,18 @@ Feature: Advanced HTML Reports
   And the following files should exist:
       | evidence/final_results.html |
 
+Scenario: I capture the dom from all open windows
+Given a file named "beakers/fifth_beaker.rb" with:
+  """
+    describe "Reporting Beaker 5", :depth => 'shallow' do
+      it "loads two windows, from 5" do
+        @driver.get 'http://the-internet.herokuapp.com/windows'
+        @driver.find_element(css: '.example a').click
+        @driver.title.should include("Google")
+      end
+    end
+  """
+  When I run `ckit brew --beakers=beakers/fifth_beaker.rb`
+  And the following files should exist:
+      | evidence/final_results.html |
+

data/features/sauce.feature

@@ -0,0 +1,46 @@
+@announce
+Feature: Sauce specific feature tests
+
+  Background:
+    Given I run `ckit new sauce-test`
+    And I cd to "sauce-test"
+    And a file named "beakers/first_beaker.rb" with:
+    """
+    describe "Sauce Beaker", :depth => 'shallow' do
+      it "test" do
+        @driver.get "http://www.google.com"
+        @driver.title.should include("Google")
+      end
+    end
+    """
+    And I overwrite config.yaml with:
+      """
+      screenshot_on_fail: true
+      selenium_connect:
+          host: 'saucelabs'
+          browser: 'firefox'
+          sauce_username: 'testing_arrgyle'
+          sauce_api_key: 'ab7a6e17-16df-42d2-9ef6-c8d2539cc38a'
+          sauce_opts:
+              public: 'private'
+      """
+
+  Scenario: Default permission config should work
+    When I run `ckit brew`
+    Then the stdout should contain "1 example, 0 failures"
+    And the file "evidence/sauce_beaker/sauce_beaker_test/sauce_job.log" should contain "private"
+
+  Scenario: I can set a specific permission with a tag
+    Given  I overwrite "beakers/first_beaker.rb" with:
+    """
+    describe "Sauce Beaker", :depth => 'shallow', :public => 'share', :crazy => 'test_tag' do
+      it "test" do
+        @driver.get "http://www.google.com"
+        @driver.title.should include("Google")
+      end
+    end
+    """
+    When I run `ckit brew`
+    Then the stdout should contain "1 example, 0 failures"
+    And the file "evidence/sauce_beaker/sauce_beaker_test/sauce_job.log" should contain "share"
+    And the file "evidence/sauce_beaker/sauce_beaker_test/sauce_job.log" should contain "crazy:test_tag"

data/features/split_testing.feature

@@ -17,7 +17,6 @@ Background:
       screenshot_on_fail: true
       base_url: 'http://the-internet.herokuapp.com/abtest'
       selenium_connect:
-          log: 'evidence'
           browser: 'firefox'
       split_testing:
         provider: 'optimizely'

data/lib/chemistrykit/catalyst.rb

@@ -27,7 +27,7 @@ module ChemistryKit
     private
 
       def validate_key(key)
-        raise "Unknown \"#{key}\"" unless @data.has_key?(key.to_sym)
+        raise "Unknown \"#{key}\"" unless @data.key?(key.to_sym)
       end
   end
 end

data/lib/chemistrykit/cli/cli.rb

@@ -50,7 +50,7 @@ module ChemistryKit
           c.add_setting :used_tags
           c.before(:suite) { ::RSpec.configuration.used_tags = [] }
           c.around(:each) do |example|
-            standard_keys = [:example_group, :example_group_block, :description_args, :caller, :execution_result]
+            standard_keys = [:example_group, :example_group_block, :description_args, :caller, :execution_result, :full_description]
             example.metadata.each do |key, value|
               tag = "#{key}:#{value}" unless standard_keys.include?(key)
               ::RSpec.configuration.used_tags.push tag unless ::RSpec.configuration.used_tags.include?(tag) || tag.nil?
@@ -135,7 +135,6 @@ module ChemistryKit
 
       def override_configs(options, config)
         # TODO: expand this to allow for more overrides as needed
-        config.log.results_file = options['results_file'] if options['results_file']
         config.retries_on_failure = options['retry'].to_i if options['retry']
         config
       end
@@ -197,6 +196,23 @@ module ChemistryKit
             test_path = File.join(Dir.getwd, sc_config[:log])
             Dir.mkdir test_path unless File.exists?(test_path)
 
+            # set the tags and permissions if sauce
+            if sc_config[:host] == 'saucelabs'
+              tags = example.metadata.reject do |key, value|
+                [:example_group, :example_group_block, :description_args, :caller, :execution_result, :full_description].include? key
+              end
+              sauce_opts = {}
+              sauce_opts.merge!(public: tags.delete(:public)) if tags.key?(:public)
+              sauce_opts.merge!(tags: tags.map { |key, value| "#{key}:#{value}"}) unless tags.empty?
+
+              if sc_config[:sauce_opts]
+                sc_config[:sauce_opts].merge!(sauce_opts) unless sauce_opts.empty?
+              else
+                sc_config[:sauce_opts] = sauce_opts unless sauce_opts.empty?
+              end
+
+            end
+
             # configure and start sc
             configuration = SeleniumConnect::Configuration.new sc_config
             @sc = SeleniumConnect.start configuration
@@ -237,16 +253,16 @@ module ChemistryKit
 
           html_log_name = options[:parallel] ? "results_#{options[:parallel]}.html" : 'results_0.html'
 
-          c.add_formatter(ChemistryKit::RSpec::HtmlFormatter, File.join(Dir.getwd, config.log.path, html_log_name))
-          # c.add_formatter(::RSpec::Core::Formatters::HtmlFormatter, File.join(Dir.getwd, config.log.path, html_log_name))
+          c.add_formatter(ChemistryKit::RSpec::HtmlFormatter, File.join(Dir.getwd, config.reporting.path, html_log_name))
 
           # for rspec-retry
           c.verbose_retry = true # for rspec-retry
           c.default_retry_count = config.retries_on_failure
 
+          # TODO: this is messy... there should be a cleaner way to hook various reporter things.
           if config.concurrency == 1 || options['parallel']
-            junit_log_name = options[:parallel] ? "junit_#{options[:parallel]}.xml" : 'junit.xml'
-            c.add_formatter(ChemistryKit::RSpec::JUnitFormatter, File.join(Dir.getwd, config.log.path, junit_log_name))
+            junit_log_name = options[:parallel] ? "junit_#{options[:parallel]}.xml" : 'junit_0.xml'
+            c.add_formatter(ChemistryKit::RSpec::JUnitFormatter, File.join(Dir.getwd, config.reporting.path, junit_log_name))
           end
         end
       end

data/lib/chemistrykit/config/reporting.rb

@@ -0,0 +1,14 @@
+# Encoding: utf-8
+
+module ChemistryKit
+  module Config
+    # Simple logging configuration class to encapsulate logging settings in the same pattern
+    # as others.
+    class Reporting
+
+      def path
+        'evidence'
+      end
+    end
+  end
+end

data/lib/chemistrykit/configuration.rb

@@ -5,6 +5,7 @@ require 'ostruct'
 
 require 'chemistrykit/config/basic_auth'
 require 'chemistrykit/config/split_testing'
+require 'chemistrykit/config/reporting'
 
 module ChemistryKit
   # Default configuration class
@@ -15,7 +16,7 @@ module ChemistryKit
                   :screenshot_on_fail,
                   :retries_on_failure
 
-    attr_reader   :log, :basic_auth, :split_testing
+    attr_reader   :reporting, :basic_auth, :split_testing
 
     attr_writer   :selenium_connect
 
@@ -25,10 +26,7 @@ module ChemistryKit
       @retries_on_failure = 1
       @selenium_connect = {}
       @screenshot_on_fail = false
-      @log = OpenStruct.new
-      @log.path = 'evidence'
-      @log.results_file = 'results_junit.xml'
-      @log.format = 'ChemistryKit::RSpec::JUnitFormatter'
+      @reporting = ChemistryKit::Config::Reporting.new
       @basic_auth = nil
       @split_testing = nil
 
@@ -36,13 +34,6 @@ module ChemistryKit
       populate_with_hash hash
     end
 
-    def log=(log_hash)
-      log_hash.each do |key, value|
-        value = 'ChemistryKit::RSpec::JUnitFormatter' if key == :format && value =~ /junit/i
-        @log.send("#{key}=", value) unless value.nil?
-      end
-    end
-
     def basic_auth=(basic_auth_hash)
       @basic_auth = ChemistryKit::Config::BasicAuth.new(basic_auth_hash.merge(base_url: base_url))
     end
@@ -54,7 +45,7 @@ module ChemistryKit
     def selenium_connect
       # return the default log unless the sc log is set
       if @selenium_connect[:log].nil?
-        @selenium_connect[:log] = @log.path
+        @selenium_connect[:log] = @reporting.path
         return @selenium_connect
       end
       @selenium_connect

data/lib/chemistrykit/reporting/html_report_assembler.rb

@@ -115,9 +115,14 @@ module ChemistryKit
                     end
                     doc.div(class: 'large-6 columns') do
                       doc.div(class: "#{type}-switch") do
-                        doc.input(id: "show-#{type}", onclick: "toggle#{type.capitalize}();", name: "switch-show-#{type}", type: 'radio', checked: 'checked')
+                        opts = { onclick: "toggle#{type.capitalize}();", name: "switch-show-#{type}", type: 'radio' }
+                        top = opts.merge(id: "show-#{type}")
+                        bot = opts.merge(id: "show-#{type}1")
+                        top.merge!(checked: 'checked') unless type == 'passing'
+                        bot.merge!(checked: 'checked') if type == 'passing'
+                        doc.input(top)
                         doc.label(for: "show-#{type}") { doc.text 'Hide' }
-                        doc.input(id: "show-#{type}1", onclick: "toggle#{type.capitalize}();", name: "switch-show-#{type}", type: 'radio')
+                        doc.input(bot)
                         doc.label(for: "show-#{type}1") { doc.text 'Show' }
                         doc.span
                       end

data/lib/chemistrykit/rspec/html_formatter.rb

@@ -34,7 +34,8 @@ module ChemistryKit
 
       def example_group_finished(example_group)
         @output_html << build_fragment do |doc|
-          doc.div(class: "row example-group #{@example_group_status}") do
+          show = @example_group_status == 'passing' ? 'show' : ''
+          doc.div(class: "row example-group #{@example_group_status} #{show}") do
             doc.div(class: 'large-12 columns') do
               doc.h3 do
                 doc.i(class: 'icon-beaker')
@@ -99,13 +100,13 @@ module ChemistryKit
           doc.div(class: 'row extra-content') do
             doc.div(class: 'large-12 columns') do
               doc.div(class: 'section-container auto', 'data-section' => '') do
+                doc << render_failshot_if_found(example)
                 doc << render_stack_trace(example)
                 doc << render_log_if_found(example, 'server.log')
                 doc << render_log_if_found(example, 'chromedriver.log')
                 doc << render_log_if_found(example, 'firefox.log')
                 doc << render_log_if_found(example, 'sauce_job.log')
                 doc << render_dom_html_if_found(example)
-                doc << render_failshot_if_found(example)
               end
             end
           end
@@ -116,12 +117,18 @@ module ChemistryKit
         # TODO: pull out the common code for checking if the log file exists
         beaker_folder = slugify(@example_group.description)
         example_folder = slugify(@example_group.description + '_' + example.description)
-        path = File.join(Dir.getwd, 'evidence', beaker_folder, example_folder, 'dom.html')
-        if File.exist?(path)
-          render_section('Dom Html') do |doc|
-            doc << Pygments.highlight(File.read(path), lexer: 'html')
+        paths = Dir.glob(File.join(Dir.getwd, 'evidence', beaker_folder, example_folder, 'dom_*.html'))
+        number = 0
+        sections = ''
+        paths.each do |path|
+          if File.exist?(path)
+            sections << render_section("Dom HTML #{number}") do |doc|
+              doc << Pygments.highlight(File.read(path), lexer: 'html')
+            end
+            number += 1
           end
         end
+        sections
       end
 
       # TODO: replace the section id with a uuid or something....
@@ -190,7 +197,8 @@ module ChemistryKit
 
       def render_example(status, example)
         build_fragment do |doc|
-          doc.div(class: "row example #{status}") do
+          show = status == 'passing' ? 'hide' : ''
+          doc.div(class: "row example #{status} #{show}") do
             doc.div(class: 'large-12 columns') do
               doc.div(class: 'row example-heading') do
                 doc.div(class: 'large-9 columns') do

data/spec/support/config.yaml

@@ -3,10 +3,6 @@
 base_url: http://google.com
 concurrency: 1
 screenshot_on_fail:
-log:
-    path: 'evidence'
-    results_file: 'results_junit.xml'
-    format: 'junit'
 selenium_connect:
     log: 'evidence'
     host: 'localhost'

data/spec/unit/lib/chemistrykit/config/reporting_spec.rb

@@ -0,0 +1,21 @@
+# Encoding: utf-8
+
+require 'spec_helper'
+require 'chemistrykit/config/reporting'
+
+describe ChemistryKit::Config::Reporting do
+
+  VALID_REPORTS_PATH = 'evidence'
+
+  before(:each) do
+    @reporting =  ChemistryKit::Config::Reporting.new
+  end
+
+  it 'should be initialized with no arguments' do
+    @reporting.should be_an_instance_of ChemistryKit::Config::Reporting
+  end
+
+  it 'should return "evidence" for the log path' do
+    @reporting.path.should eq VALID_REPORTS_PATH
+  end
+end

data/spec/unit/lib/chemistrykit/configuration_spec.rb

@@ -9,19 +9,14 @@ describe ChemistryKit::Configuration do
   VALID_CONCURRENCY = 1
   VALID_RETRIES_ON_FAILURE = 1
   VALID_CONFIG_FILE = 'config.yaml'
-  VALID_LOG_PATH = 'evidence'
-  VALID_JUNIT = 'results_junit.xml'
-  VALID_FORMAT_JUNIT = 'junit'
-  VALID_JUNIT_FORMAT_OUT = 'ChemistryKit::RSpec::JUnitFormatter'
+  VALID_REPORTING_PATH = 'evidence'
 
   before(:each) do
     @valid_selenium_connect_hash = { log: 'evidence', host: 'localhost' }
-    @valid_log_hash = { path: VALID_LOG_PATH, results_file: VALID_JUNIT, format: VALID_FORMAT_JUNIT }
     @valid_config_hash = {
       base_url: VALID_BASE_URL,
       concurrency: VALID_CONCURRENCY,
       selenium_connect: @valid_selenium_connect_hash,
-      log: @valid_log_hash,
       retries_on_failure: VALID_RETRIES_ON_FAILURE
     }
   end
@@ -32,10 +27,8 @@ describe ChemistryKit::Configuration do
     config.retries_on_failure.should eq VALID_RETRIES_ON_FAILURE
     config.base_url.should eq VALID_BASE_URL
 
-    # log configurations
-    config.log.path.should eq VALID_LOG_PATH
-    config.log.results_file.should eq VALID_JUNIT
-    config.log.format.should eq VALID_JUNIT_FORMAT_OUT
+    # reporting configurations
+    config.reporting.path.should eq VALID_REPORTING_PATH
 
     # selenium connect configurations
     config.selenium_connect.should eq @valid_selenium_connect_hash
@@ -49,10 +42,8 @@ describe ChemistryKit::Configuration do
     config = ChemistryKit::Configuration.new({})
     config.concurrency.should eq VALID_CONCURRENCY
     config.retries_on_failure.should eq VALID_RETRIES_ON_FAILURE
-    config.log.path.should eq VALID_LOG_PATH
-    config.log.results_file.should eq VALID_JUNIT
-    config.log.format.should eq VALID_JUNIT_FORMAT_OUT
-    config.selenium_connect.should eq({ log: VALID_LOG_PATH })
+    config.reporting.path.should eq VALID_REPORTING_PATH
+    config.selenium_connect.should eq({ log: VALID_REPORTING_PATH })
     config.basic_auth.should be_nil
     config.split_testing.should be_nil
   end
@@ -72,18 +63,13 @@ describe ChemistryKit::Configuration do
     end.to raise_error ArgumentError, 'The config key: "bad" is unknown!'
   end
 
-  it 'should correct the format to JUnit' do
-    config = ChemistryKit::Configuration.new(@valid_config_hash)
-    config.log.format.should eq VALID_JUNIT_FORMAT_OUT
-  end
-
   it 'selenium_connect log should default to the main log' do
-    config =  ChemistryKit::Configuration.new({ log: { path: 'main-path' } })
-    config.selenium_connect.should eq({ log: 'main-path' })
+    config =  ChemistryKit::Configuration.new({})
+    config.selenium_connect.should eq({ log: VALID_REPORTING_PATH })
   end
 
   it 'mainlog should not overide selenium_connect log' do
-    config =  ChemistryKit::Configuration.new({ log: { path: 'main-path' }, selenium_connect: { log: 'sc-log' } })
+    config =  ChemistryKit::Configuration.new selenium_connect: { log: 'sc-log' }
     config.selenium_connect.should eq({ log: 'sc-log' })
   end
 

metadata

@@ -1,8 +1,8 @@
 --- !ruby/object:Gem::Specification
 name: chemistrykit
 version: !ruby/object:Gem::Version
-  version: 3.9.0.rc3
-  prerelease: 6
+  version: 3.9.0
+  prerelease: 
 platform: ruby
 authors:
 - Dave Haeffner
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-08-19 00:00:00.000000000 Z
+date: 2013-08-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -99,7 +99,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 3.6.0
+        version: 3.7.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -107,7 +107,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 3.6.0
+        version: 3.7.0
 - !ruby/object:Gem::Dependency
   name: parallel_tests
   requirement: !ruby/object:Gem::Requirement
@@ -204,7 +204,7 @@ dependencies:
     - - ~>
       - !ruby/object:Gem::Version
         version: 0.5.2
-description: Added basic http authentication support
+description: Improved reporting features and configuration for sauce permissions
 email:
 - dave@arrgyle.com
 - jason@arrgyle.com
@@ -239,10 +239,10 @@ files:
 - features/exit_status.feature
 - features/global-config.feature
 - features/load_page_objects.feature
-- features/logging.feature
 - features/multi-config.feature
 - features/new.feature
 - features/reporting.feature
+- features/sauce.feature
 - features/split_testing.feature
 - features/step_definitions/steps.rb
 - features/support/env.rb
@@ -257,6 +257,7 @@ files:
 - lib/chemistrykit/cli/helpers/formula_loader.rb
 - lib/chemistrykit/cli/new.rb
 - lib/chemistrykit/config/basic_auth.rb
+- lib/chemistrykit/config/reporting.rb
 - lib/chemistrykit/config/split_testing.rb
 - lib/chemistrykit/configuration.rb
 - lib/chemistrykit/formula/base.rb
@@ -323,6 +324,7 @@ files:
 - spec/unit/lib/chemistrykit/chemist_spec.rb
 - spec/unit/lib/chemistrykit/cli/helpers/formula_loader_spec.rb
 - spec/unit/lib/chemistrykit/config/basic_auth_spec.rb
+- spec/unit/lib/chemistrykit/config/reporting_spec.rb
 - spec/unit/lib/chemistrykit/config/split_testing_spec.rb
 - spec/unit/lib/chemistrykit/configuration_spec.rb
 - spec/unit/lib/chemistrykit/formula/base_spec.rb
@@ -347,9 +349,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
-  - - ! '>'
+  - - ! '>='
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
+      segments:
+      - 0
+      hash: -3683118535885590803
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25
@@ -366,10 +371,10 @@ test_files:
 - features/exit_status.feature
 - features/global-config.feature
 - features/load_page_objects.feature
-- features/logging.feature
 - features/multi-config.feature
 - features/new.feature
 - features/reporting.feature
+- features/sauce.feature
 - features/split_testing.feature
 - features/step_definitions/steps.rb
 - features/support/env.rb
@@ -395,6 +400,7 @@ test_files:
 - spec/unit/lib/chemistrykit/chemist_spec.rb
 - spec/unit/lib/chemistrykit/cli/helpers/formula_loader_spec.rb
 - spec/unit/lib/chemistrykit/config/basic_auth_spec.rb
+- spec/unit/lib/chemistrykit/config/reporting_spec.rb
 - spec/unit/lib/chemistrykit/config/split_testing_spec.rb
 - spec/unit/lib/chemistrykit/configuration_spec.rb
 - spec/unit/lib/chemistrykit/formula/base_spec.rb

data/features/logging.feature

@@ -1,98 +0,0 @@
-#logging.feature
-
-## NOTE Pending until after release candidate
-
-# Feature: Log handling
-#   In order to examine the results of my test suite
-#   As a test harness user
-#   I want to see all of the logs in a central location
-
-#   Background: Setup the project
-#     Given I run `ckit new logging-test`
-#     And I cd to "logging-test"
-#     And a file named "formulas/bookie.rb" with:
-#       """
-#       module Formulas
-#         class Bookie < Formula
-#           def open(url)
-#             @driver.get url
-#           end
-#         end
-#       end
-#       """
-#     And a file named "beakers/first_beaker.rb" with:
-#       """
-#       describe "First", :depth => 'shallow' do
-#         let(:book) { Formulas::Bookie.new(@driver) }
-
-#         it "loads an external web page" do
-#           book.open "http://www.google.com"
-#         end
-#       end
-#       """
-#     And a file named "beakers/second_beaker.rb" with:
-#       """
-#       describe "Second", :depth => 'shallow' do
-#         let(:book) { Formulas::Bookie.new(@driver) }
-
-#         it "loads an external web page" do
-#           book.open "http://www.google.com"
-#         end
-#       end
-#       """
-
-#   Scenario: I can output junit xml results by default
-#     When I run `ckit brew`
-#     Then the stdout should contain "2 examples, 0 failures"
-#     And the following files should exist:
-#       | evidence/results_junit.xml  |
-
-#   Scenario: I can output custom junit xml results
-#     Given a file named "config.yaml" with:
-#       """
-#       log:
-#           path: 'my_evidence'
-#           results_file: 'my_results.xml'
-#           format: 'junit'
-#       selenium_connect:
-#           log: 'evidence'
-#           host: 'localhost'
-#       """
-#     When I run `ckit brew`
-#     Then the stdout should contain "2 examples, 0 failures"
-#     And the following files should exist:
-#       | my_evidence/my_results.xml  |
-
-#     Scenario: I can output html results
-#       Given a file named "config.yaml" with:
-#         """
-#         log:
-#             results_file: 'report.html'
-#             format: 'html'
-#         """
-#       And I run `ckit brew`
-#       Then the stdout should contain "2 examples, 0 failures"
-#       And the following files should exist:
-#         | evidence/report.html  |
-
-#     Scenario: I define one log location to rule them all
-#       Given a file named "config.yaml" with:
-#         """
-#           log:
-#               path: 'my_evidence'
-#           selenium_connect:
-#               host: 'localhost'
-#         """
-#       When I run `ckit brew`
-#         Then the stdout should contain "2 examples, 0 failures"
-#         And the following files should exist:
-#           | my_evidence/results_junit.xml |
-#           | my_evidence/first/first_loads_an_external_web_page/server.log  |
-#           | my_evidence/second/second_loads_an_external_web_page/server.log  |
-
-#     Scenario: I can runtime override the results output file name
-#       When I run `ckit brew --results_file results_junit_01.xml`
-#       Then the stdout should contain "2 examples, 0 failures"
-#       And the following files should exist:
-#         | evidence/results_junit_01.xml  |
-