chemistrykit 3.4.2 → 3.5.0

data/CHANGELOG.md

@@ -1,3 +1,10 @@
+#3.5.0 (2013-07-15)
+updated logging to store assets in a per beaker folder
+
+- Bumped version to 3.5.0 to prepare for release.
+- updated docs
+- updated ckit to use latest sc with cleaner log storage and added a flag for the screenshot download
+
 #3.4.2 (2013-07-14)
 fixed typo
 

data/README.md

@@ -1,4 +1,4 @@
-#ChemistryKit 3.4.2 (2013-07-14)
+#ChemistryKit 3.5.0 (2013-07-15)
 
 [![Gem Version](https://badge.fury.io/rb/chemistrykit.png)](http://badge.fury.io/rb/chemistrykit) [![Build Status](https://travis-ci.org/arrgyle/chemistrykit.png?branch=develop)](https://travis-ci.org/jrobertfox/chef-broiler-platter) [![Code Climate](https://codeclimate.com/github/arrgyle/chemistrykit.png)](https://codeclimate.com/github/arrgyle/chemistrykit) [![Coverage Status](https://coveralls.io/repos/arrgyle/chemistrykit/badge.png?branch=develop)](https://coveralls.io/r/arrgyle/chemistrykit?branch=develop)
 
@@ -100,7 +100,7 @@ You can even nest them inside different describe/context blocks and they will ge
 ###Logs and CI Integration
 Each run of Chemistry Kit saves logging and test output to the _evidence_ directory by default. And in there will be the full set of JUnit Ant XML files that may be consumed by your CI.
 
-We also output an attachment message to stdout allowing you to use [this plugin](https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin) for integrating the files with jenkins!
+Assets generated by selenium (logs, screenshots, etc.) are stored in a subfolder with a name matching the describe block in your beaker, slugifyed like: `my_beaker_name`. This is to provide some organization but also allow the integration with jenkins using [this plugin](https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin).
 
 ##Configuration
 ChemistryKit is configured by default with a `config.yaml` file that is created for you when you scaffold out a test harness. Relevant configuration options are detailed below:
@@ -118,6 +118,8 @@ ChemistryKit is configured by default with a `config.yaml` file that is created
 `selenium_connect:` Options in this node override the defaults for the [Selenium Connect](https://github.com/arrgyle/selenium-connect) gem.
 ##Command Line Usage
 
+`screenshot_on_fail` By default false, set to true to download a screenshot of the failure (supported by sauce labs for now.)
+
 ###new
 Creates a new ChemistryKit project.
 

data/chemistrykit.gemspec

@@ -2,13 +2,13 @@
 
 Gem::Specification.new do |s|
   s.name          = 'chemistrykit'
-  s.version       = '3.4.2'
+  s.version       = '3.5.0'
   s.platform      = Gem::Platform::RUBY
   s.authors       = ['Dave Haeffner', 'Jason Fox']
   s.email         = ['dave@arrgyle.com', 'jason@arrgyle.com']
   s.homepage      = 'https://github.com/arrgyle/chemistrykit'
   s.summary       = 'A simple and opinionated web testing framework for Selenium that follows convention over configuration.'
-  s.description   = 'fixed typo'
+  s.description   = 'updated logging to store assets in a per beaker folder'
   s.license       = 'MIT'
 
   s.files         = `git ls-files`.split($/)
@@ -24,7 +24,7 @@ Gem::Specification.new do |s|
   s.add_dependency 'yarjuf', '~> 1.0.5'
   s.add_dependency 'selenium-webdriver', '~> 2.29.0'
   s.add_dependency 'rest-client', '~> 1.6.7'
-  s.add_dependency 'selenium-connect', '~> 3.1.2'
+  s.add_dependency 'selenium-connect', '~> 3.2.0'
   s.add_dependency 'parallel_tests', '~> 0.15.0'
   s.add_dependency 'parallel', '~> 0.7.0'
 

data/features/brew.feature

@@ -40,8 +40,8 @@ Feature: Brewing a ChemistryKit project
     When I run `ckit brew`
     Then the stdout should contain "1 example, 0 failures"
     And the following files should exist:
-      | evidence/results_junit.xml  |
-      | evidence/server.log       |
+      | evidence/results_junit.xml       |
+      | evidence/bookie/server.log       |
 
   Scenario: Saucelabs
     Given a file named "config.yaml" with:
@@ -98,10 +98,10 @@ Feature: Brewing a ChemistryKit project
       """
     When I run `ckit brew --all`
     Then the stdout should contain "2 examples, 0 failures"
-
   Scenario: Saucelabs
     Given a file named "config.yaml" with:
       """
+      screenshot_on_fail: true
       selenium_connect:
           log: 'evidence'
           host: 'saucelabs'
@@ -114,7 +114,7 @@ Feature: Brewing a ChemistryKit project
       """
     And a file named "beakers/failure.rb" with:
     """
-    describe "Failing", :depth => 'shallow' do
+    describe "Failing Beaker", :depth => 'shallow' do
       it "loads an external web page" do
         @driver.get "http://www.google.com"
         @driver.title.should_not include("Google")
@@ -123,7 +123,6 @@ Feature: Brewing a ChemistryKit project
     """
     When I run `ckit brew --beakers=beakers/failure.rb`
     Then the stdout should contain "1 example, 1 failure"
-    And the stdout should contain "[[ATTACHEMENT|/Users/jfox/development/arrgyle/chemistrykit/build/tmp/booker/evidence/"
-    And there should be "1" "failed image" log files in "evidence"
-    And there should be "1" "report" log files in "evidence"
-    And there should be "1" "sauce log" log files in "evidence"
+    And there should be "1" "failed image" log files in "evidence/failing_beaker"
+    And there should be "1" "report" log files in "evidence/failing_beaker"
+    And there should be "1" "sauce log" log files in "evidence/failing_beaker"

data/features/concurrency.feature

@@ -97,5 +97,4 @@ Feature: Support for concurency
     Then the stdout should not contain "All examples were filtered out"
     And the stdout should not contain "0 examples, 0 failures"
     And there should be "4" unique results files in the "evidence" directory
-    And there should be "5" "report" log files in "evidence"
-    And there should be "5" "sauce log" log files in "evidence"
+    And there should be "2" "report" log files in "evidence/cheese"

data/features/logging.feature

@@ -84,7 +84,8 @@ Feature: Log handling
         Then the stdout should contain "2 examples, 0 failures"
         And the following files should exist:
           | my_evidence/results_junit.xml |
-          | my_evidence/server.log        |
+          | my_evidence/first/server.log  |
+          | my_evidence/second/server.log  |
 
     Scenario: I can runtime override the results output file name
       When I run `ckit brew --results_file results_junit_01.xml`

data/features/multi-config.feature

@@ -26,7 +26,7 @@ Feature: Support for multiple configuration files
     When I run `ckit brew`
     Then the stdout should contain "1 example, 0 failures"
     And the following files should exist:
-      | evidence_config/server.log       |
+      | evidence_config/cheese/server.log       |
 
   Scenario: I can specifiy an alternative configuration with --config
     Given a directory named "evidence_alternate"
@@ -39,7 +39,7 @@ Feature: Support for multiple configuration files
       When I run `ckit brew --config alternate.yaml`
       Then the stdout should contain "1 example, 0 failures"
       And the following files should exist:
-        | evidence_alternate/server.log       |
+        | evidence_alternate/cheese/server.log       |
 
   Scenario: I can specifiy an alternative configuration with -c
     Given a directory named "evidence_alternate"
@@ -52,7 +52,7 @@ Feature: Support for multiple configuration files
       When I run `ckit brew -c alternate.yaml`
       Then the stdout should contain "1 example, 0 failures"
       And the following files should exist:
-        | evidence_alternate/server.log       |
+        | evidence_alternate/cheese/server.log       |
 
   Scenario: I can specifiy an alternative configuration with --config with concurrency
     Given a directory named "evidence_alternate"
@@ -68,4 +68,4 @@ Feature: Support for multiple configuration files
       Then the stdout should contain "1 example, 0 failures"
       And there should be "1" unique results files in the "evidence_alternate" directory
       And the following files should exist:
-        | evidence_alternate/server.log       |
+        | evidence_alternate/cheese/server.log       |

data/features/step_definitions/steps.rb

@@ -26,11 +26,11 @@ Then(/^there should be "(.*?)" "(.*?)" log files in "(.*?)"$/) do |number, type,
   files.each do |file|
     case type
     when 'failed image'
-      count += 1 if file =~ /failed_.+\.png/
+      count += 1 if file =~ /.+_failshot_.+\.png/
     when 'report'
-      count += 1 if file =~ /report_.+\.log/
+      count += 1 if file =~ /.+_saucejob_.+\.log/
     when 'sauce log'
-        count += 1 if file =~ /sauce_job_.+\.log/
+        count += 1 if file =~ /.+_serverlog_.+\.log/
     end
   end
   count.should == number.to_i

data/lib/chemistrykit/cli/cli.rb

@@ -163,33 +163,30 @@ module ChemistryKit
           c.before(:all) do
             @config = config # set the config available globaly
             ENV['BASE_URL'] = config.base_url # assign base url to env variable for formulas
-            sc_config = SeleniumConnect::Configuration.new
-            sc_config.populate_with_hash config.selenium_connect
-            @sc = SeleniumConnect.start sc_config # fire up a connection to SC
           end
           c.around(:each) do |example|
+            # create the beaker name from the example data
+            beaker_name = example.metadata[:example_group][:description_args].first.downcase.strip.gsub(' ', '_').gsub(/[^\w-]/, '')
+            # override log path with be beaker sub path
+            sc_config = @config.selenium_connect.dup
+            sc_config[:log] += "/#{beaker_name}"
+            sub_evidence_path = File.join(Dir.getwd, sc_config[:log])
+            Dir.mkdir sub_evidence_path unless File.exists?(sub_evidence_path)
+
+            # configure and start sc
+            configuration = SeleniumConnect::Configuration.new sc_config
+            @sc = SeleniumConnect.start configuration
             @job = @sc.create_job # create a new job
             @driver = @job.start name: example.metadata[:full_description]
             example.run
           end
           c.after(:each) do
             if example.exception != nil
-              report = @job.finish failed: true, failshot: true
+              @job.finish failed: true, failshot: @config.screenshot_on_fail
             else
-              report = @job.finish passed: true
+              @job.finish passed: true
             end
             @sc.finish
-
-            # TODO absctract this out into some report handler class
-            data = report.data
-            unless data.empty?
-              report_file = File.join(Dir.getwd, config.log.path, "report_#{data[:sauce_data][:id]}.log")
-              File.open(report_file, 'w') { |file| file.write(data.to_s) }
-              puts "\n[[ATTACHMENT|#{report_file}]]\n"
-            end
-            puts "\n[[ATTACHMENT|#{File.join(Dir.getwd, config.log.path, data[:failshot])}]]\n" if data[:failshot]
-            puts "\n[[ATTACHMENT|#{File.join(Dir.getwd, config.log.path, data[:server_log])}]]\n" if data[:server_log]
-            ###
           end
           c.order = 'random'
           c.default_path = 'beakers'

data/lib/chemistrykit/configuration.rb

@@ -8,7 +8,8 @@ module ChemistryKit
   class Configuration
 
     attr_accessor :base_url,
-                  :concurrency
+                  :concurrency,
+                  :screenshot_on_fail
 
     attr_reader   :log
 
@@ -18,6 +19,7 @@ module ChemistryKit
       # set defaults
       @concurrency = 1
       @selenium_connect = {}
+      @screenshot_on_fail = false
       @log = OpenStruct.new
       @log.path = 'evidence'
       @log.results_file = 'results_junit.xml'

data/spec/support/config.yaml

@@ -2,6 +2,7 @@
 
 base_url: http://google.com
 concurrency: 1
+screenshot_on_fail:
 log:
     path: 'evidence'
     results_file: 'results_junit.xml'

data/spec/unit/lib/chemistrykit/configuration_spec.rb

@@ -24,6 +24,7 @@ describe ChemistryKit::Configuration do
   end
 
   def validate_config(config)
+    config.screenshot_on_fail.should eq false
     config.concurrency.should eq VALID_CONCURRENCY
     config.base_url.should eq VALID_BASE_URL
     config.log.path.should eq VALID_LOG_PATH

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: chemistrykit
 version: !ruby/object:Gem::Version
-  version: 3.4.2
+  version: 3.5.0
   prerelease: 
 platform: ruby
 authors:
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-07-14 00:00:00.000000000 Z
+date: 2013-07-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -99,7 +99,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 3.1.2
+        version: 3.2.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -107,7 +107,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 3.1.2
+        version: 3.2.0
 - !ruby/object:Gem::Dependency
   name: parallel_tests
   requirement: !ruby/object:Gem::Requirement
@@ -252,7 +252,7 @@ dependencies:
     - - ~>
       - !ruby/object:Gem::Version
         version: 0.6.7
-description: fixed typo
+description: updated logging to store assets in a per beaker folder
 email:
 - dave@arrgyle.com
 - jason@arrgyle.com
@@ -337,7 +337,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2534848085378017120
+      hash: -2897425832114574927
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25