beaker 3.22.0 → 3.23.0

data/docs/how_to/hypervisors/README.md

@@ -32,18 +32,23 @@ that are needed for it. An example `.fog` file is below:
 
 # External Hypervisors
 
-Puppetlabs and its community have made several gems that support different hypervisors with beaker, the reason for this is that we're looking to decrease Beaker's
+Puppet and its community have made several gems that support different
+hypervisors with beaker, the reason for this is that we're looking to decrease Beaker's
 dependency footprint, and hypervisors are one of the places where we can often
 increase the load across all Beaker uses to benefit a small group that uses a
 particular hypervisor. 
 
-In order to offset this, we've made a listing of gems and community-supported forks that support other external hypervisors. Please check them out if you'd like to use those hypervisors, hopefully it'll save you from spending time trying to support a new hypervisor yourself.
+In order to offset this, we've made a listing of gems and community-supported forks
+that support other external hypervisors. Please check them out if you'd like to use
+those hypervisors, hopefully it'll save you from spending time trying to support a
+new hypervisor yourself.
 
-Hypervisor gems made by puppet (pre-included in beaker 3.x):
+### Hypervisor gems made by puppet (pre-included in beaker 3.x):
 
 | Hypervisor               | Fork                                                               |
 | :----------------------: | :---------------------------------------------------------:        |
 | Vmpooler                 | [beaker-vmpooler](https://github.com/puppetlabs/beaker-vmpooler)   |
+| Vcloud                   | [beaker-vcloud](https://github.com/puppetlabs/beaker-vcloud)       |
 | AWS                      | [beaker-aws](https://github.com/puppetlabs/beaker-aws)             |
 | Vagrant                  | [beaker-vagrant](https://github.com/puppetlabs/beaker-vagrant)     |
 | VMware/Vsphere           | [beaker-vmware](https://github.com/puppetlabs/beaker-vmware)       |
@@ -51,7 +56,19 @@ Hypervisor gems made by puppet (pre-included in beaker 3.x):
 | Openstack                | [beaker-openstack](https://github.com/puppetlabs/beaker-openstack) |
 | Google Compute           | [beaker-google](https://github.com/puppetlabs/beaker-google)       |
 
-Hypervisor gems and beaker forks made by community:
+
+### beaker-abs
+
+There is another hypervisor made and used internally by puppet,
+[beaker-abs](https://github.com/puppetlabs/beaker-abs), but it isn't included
+in beaker 3.x. If you'd like to use beaker-abs, you'll have to include it yourself.
+
+You do that by requiring beaker-abs in your Gemfile as a sibling to beaker itself
+and then using `abs` as your hypervisor value in your hosts file. Please check the
+[beaker-abs README](https://github.com/puppetlabs/beaker-abs/blob/master/README.md)
+for more information.
+
+### Hypervisor gems and beaker forks made by community:
 
 | Hypervisor   | Fork                                                                 |
 |:------------:|:--------------------------------------------------------------------:|

data/docs/tutorials/creating_a_test_environment.md

@@ -83,13 +83,4 @@ The platform's format is `/^OSFAMILY-VERSION-ARCH.*$/` where `OSFAMILY` is one o
 
 `VERSION`'s format is not enforced, but should reflect the `OSFAMILY` selected (ie, ubuntu-1204-i386-master, scientific-6-i386-agent, etc).  `ARCH`'s format is also not enforced, but should be appropriate to the `OSFAMILY` selected (ie, ubuntu-1204-i386-master, sles-11-x86_64-master, debian-7-amd64-master, etc).
 
-## Supported Virtualization Providers ##
-* [AWS](../how_to/hypervisors/aws.md)
-* [VMWare Fusion](../how_to/hypervisors/vmware_fusion.md)
-* [EC2](../how_to/hypervisors/ec2.md)
-* [vSphere](../how_to/hypervisors/vsphere.md)
-* [Vagrant](../how_to/hypervisors/vagrant.md)
-* [Google Compute Engine](../how_to/hypervisors/google_compute_engine.md)
-* [Docker Support](../how_to/hypervisors/docker.md)
-* [Openstack](../how_to/hypervisors/openstack.md)
-* [Solaris](../how_to/hypervisors/solaris.md)
+## [Supported Virtualization Providers](../how_to/hypervisors/README.md#external-hypervisors) ##

data/lib/beaker.rb

@@ -1,7 +1,7 @@
 require 'rubygems' unless defined?(Gem)
 module Beaker
 
-  %w( version platform test_suite result command options network_manager cli perf logger_junit subcommand ).each do |lib|
+  %w( version platform test_suite test_suite_result result command options network_manager cli perf logger_junit subcommand ).each do |lib|
     begin
       require "beaker/#{lib}"
     rescue LoadError

data/lib/beaker/dsl/structure.rb

@@ -31,7 +31,7 @@ module Beaker
     #     end
     #
     module Structure
-
+      require 'pry'
       # Provides a method to help structure tests into coherent steps.
       # @param [String] step_name The name of the step to be logged.
       # @param [Proc] block The actions to be performed in this step.
@@ -40,7 +40,16 @@ module Beaker
         set_current_step_name(step_name)
         if block_given?
           logger.step_in()
-          yield
+          begin
+            yield
+          rescue Exception => e
+            if(@options.has_key?(:debug_errors) && @options[:debug_errors] == true)
+              logger.info("Exception raised during step execution and debug-errors option is set, entering pry. Exception was: #{e.inspect}")
+              logger.info("HINT: Use the pry 'backtrace' and 'up' commands to navigate to the test code")
+              binding.pry
+            end
+            raise e
+          end
           logger.step_out()
         end
       end

data/lib/beaker/host/unix/pkg.rb

@@ -74,7 +74,7 @@ module Unix::Pkg
   def install_package(name, cmdline_args = '', version = nil, opts = {})
     case self['platform']
       when /sles-/
-        execute("zypper --non-interactive in #{name}", opts)
+        execute("zypper --non-interactive --gpg-auto-import-keys in #{name}", opts)
       when /el-4/
         @logger.debug("Package installation not supported on rhel4")
       when /fedora-(2[2-9])/

data/lib/beaker/logger_junit.rb

@@ -1,10 +1,11 @@
+require 'rexml/document'
 module Beaker
   # The Beaker JUnit Logger class
   # This module handles message reporting from Beaker to the JUnit format
   #
   # There is a specific pattern for using this class.
   # Here's a list of example usages:
-  # - {Beaker::TestSuite::TestSuiteResult#write_junit_xml}
+  # - {Beaker::TestSuiteResult#write_junit_xml}
   module LoggerJunit
 
     # writes the xml created in the block to the xml file given
@@ -34,14 +35,14 @@ module Beaker
 
     # writes out xml content for a doc
     #
-    # @param [Nokogiri::XML] doc Nokogiri doc containing content to write
+    # @param [REXML::Document] doc doc containing content to write
     # @param [String] xml_file Path to the xml file to write
     #
     # @return nil
     def self.finish(doc, xml_file)
       # junit/name.xml will be created in a directory relative to the CWD
-      # --  JLS 2/12
-      File.open(xml_file, 'w') { |fh| fh.write(doc.to_xml) }
+
+      File.open(xml_file, 'w') { |f| doc.write(f, 2) }
     end
 
     # gets the xml doc & suites in order to build your xml output on top of
@@ -50,8 +51,8 @@ module Beaker
     # @param [String] name Name of the testsuite you're writing
     # @param [String] stylesheet Path to the stylesheet file
     #
-    # @return [Nokogiri::XML] doc to use for your xml content
-    # @return [Nokogiri::XML::Node] suites to add your content to
+    # @return [REXML::Document] doc to use for your xml content
+    # @return [REXML::Element] suites to add your content to
     def self.get_xml_contents(xml_file, name, stylesheet)
       self.copy_stylesheet_into_xml_dir(stylesheet, xml_file)
       xml_file_already_exists = File.file?(xml_file)
@@ -74,24 +75,23 @@ module Beaker
 
     # sets up doc & gives us the suites for the testsuite named
     #
-    # @param [Nokogiri::XML] doc Doc that you're getting suites from
+    # @param [REXML::Document] doc Doc that you're getting suites from
     # @param [String] name Testsuite node name
     # @param [Boolean] already_existed Whether or not the doc already existed
     #
-    # @return [Nokogiri::XML::Node] testsuites
+    # @return [Rexml::Element] testsuites
     def self.get_testsuites_from_doc(doc, name, already_existed)
       #check to see if an output file already exists, if it does add or replace test suite data
       if already_existed
-        suites = doc.at_xpath('testsuites')
+        suites = REXML::XPath.first(doc, "testsuites")
         #remove old data
-        doc.search("//testsuite").each do |node|
-          if node['name'] =~ /#{name}/
-            node.unlink
+        suites.elements.each("testsuite") do |e|
+          if e.name =~ /#{name}/
+            suites.delete_element e
           end
         end
       else
-        suites        = Nokogiri::XML::Node.new('testsuites', doc)
-        suites.parent = doc
+        suites = doc.add_element(REXML::Element.new('testsuites'))
       end
       return suites
     end
@@ -102,16 +102,16 @@ module Beaker
     # @param [String] stylesheet Path to the stylesheet for this doc
     # @param [Boolean] already_exists Whether or not the file already exists
     #
-    # @return [Nokogiri::XML] Doc that you want to write in
+    # @return [REXML::Document] Doc that you want to write in
     def self.get_doc_for_filename(filename, stylesheet, already_exists)
       if already_exists
-        doc           = Nokogiri::XML(File.open(filename, 'r'))
+        doc           = REXML::Document.new File.open(filename)
       else
         #no existing file, create a new one
-        doc           = Nokogiri::XML::Document.new()
-        doc.encoding  = 'UTF-8'
-        pi            = Nokogiri::XML::ProcessingInstruction.new(doc, "xml-stylesheet", "type=\"text/xsl\" href=\"#{File.basename(stylesheet)}\"")
-        pi.parent     = doc
+        doc           = REXML::Document.new
+        doc << REXML::XMLDecl.new(version="1.0", encoding="UTF-8")
+        instruction_content = "type='text/xsl' href='#{File.basename(stylesheet)}'"
+        doc << REXML::Instruction.new(target="xml-stylesheet", content=instruction_content)
       end
       return doc
     end
@@ -154,4 +154,4 @@ module Beaker
     end
 
   end
-end
+end

data/lib/beaker/options/command_line_parser.rb

@@ -88,6 +88,12 @@ module Beaker
             @cmd_options[:preserve_hosts] = mode || 'always'
           end
 
+          opts.on '--debug-errors',
+                  'Enter a pry console if or when a test fails',
+                  '(default: false)' do |bool|
+            @cmd_options[:debug_errors] = bool
+          end
+
           opts.on '--root-keys',
                   'Install puppetlabs pubkeys for superuser',
                   '(default: false)' do |bool|

data/lib/beaker/subcommand.rb

@@ -51,6 +51,7 @@ module Beaker
     class_option :tag, :type => :string, :group => 'Beaker run'
     class_option :'exclude-tags', :type => :string, :group => 'Beaker run'
     class_option :'xml-time-order', :type => :boolean, :group => 'Beaker run'
+    class_option :'debug-errors', :type => :boolean, :group => 'Beaker run'
 
     # The following are listed as deprecated in beaker --help, but needed now for
     # feature parity for beaker 3.x.

data/lib/beaker/test_suite.rb

@@ -1,7 +1,6 @@
 # -*- coding: utf-8 -*-
-require 'nokogiri'
 require 'fileutils'
-[ 'test_case', 'logger' ].each do |lib|
+[ 'test_case', 'logger', 'test_suite_result'].each do |lib|
   require "beaker/#{lib}"
 end
 
@@ -10,264 +9,6 @@ module Beaker
   #Handles executing the set of {TestCase} instances and reporting results as post summary text and JUnit XML.
   class TestSuite
 
-    #Holds the output of a test suite, formats in plain text or xml
-    class TestSuiteResult
-      attr_accessor :start_time, :stop_time, :total_tests
-
-      #Create a {TestSuiteResult} instance.
-      #@param [Hash{Symbol=>String}] options Options for this object
-      #@option options [Logger] :logger The Logger object to report information to
-      #@param [String] name The name of the {TestSuite} that the results are for
-      def initialize( options, name )
-        @options = options
-        @logger = options[:logger]
-        @name = name
-        @test_cases = []
-        #Set some defaults, just in case you attempt to print without including them
-        start_time = Time.at(0)
-        stop_time = Time.at(1)
-      end
-
-      #Add a {TestCase} to this {TestSuiteResult} instance, used in calculating {TestSuiteResult} data.
-      #@param [TestCase] test_case An individual, completed {TestCase} to be included in this set of {TestSuiteResult}.
-      def add_test_case( test_case )
-        @test_cases << test_case
-      end
-
-      #How many {TestCase} instances are in this {TestSuiteResult}
-      def test_count
-        @test_cases.length
-      end
-
-      #How many passed {TestCase} instances are in this {TestSuiteResult}
-      def passed_tests
-        @test_cases.select { |c| c.test_status == :pass }.length
-      end
-
-      #How many errored {TestCase} instances are in this {TestSuiteResult}
-      def errored_tests
-        @test_cases.select { |c| c.test_status == :error }.length
-      end
-
-      #How many failed {TestCase} instances are in this {TestSuiteResult}
-      def failed_tests
-        @test_cases.select { |c| c.test_status == :fail }.length
-      end
-
-      #How many skipped {TestCase} instances are in this {TestSuiteResult}
-      def skipped_tests
-        @test_cases.select { |c| c.test_status == :skip }.length
-      end
-
-      #How many pending {TestCase} instances are in this {TestSuiteResult}
-      def pending_tests
-        @test_cases.select {|c| c.test_status == :pending}.length
-      end
-
-      #How many {TestCase} instances failed in this {TestSuiteResult}
-      def sum_failed
-        failed_tests + errored_tests
-      end
-
-      #Did all the {TestCase} instances in this {TestSuiteResult} pass?
-      def success?
-        sum_failed == 0
-      end
-
-      #Did one or more {TestCase} instances in this {TestSuiteResult} fail?
-      def failed?
-        !success?
-      end
-
-      #The sum of all {TestCase} runtimes in this {TestSuiteResult}
-      def elapsed_time
-        @test_cases.inject(0.0) {|r, t| r + t.runtime.to_f }
-      end
-
-      #Plain text summay of test suite
-      #@param [Logger] summary_logger The logger we will print the summary to
-      def summarize(summary_logger)
-
-        summary_logger.notify <<-HEREDOC
-      Test Suite: #{@name} @ #{start_time}
-
-      - Host Configuration Summary -
-        HEREDOC
-
-        average_test_time = elapsed_time / test_count
-
-        summary_logger.notify %Q[
-
-              - Test Case Summary for suite '#{@name}' -
-       Total Suite Time: %.2f seconds
-      Average Test Time: %.2f seconds
-              Attempted: #{test_count}
-                 Passed: #{passed_tests}
-                 Failed: #{failed_tests}
-                Errored: #{errored_tests}
-                Skipped: #{skipped_tests}
-                Pending: #{pending_tests}
-                  Total: #{@total_tests}
-
-      - Specific Test Case Status -
-        ] % [elapsed_time, average_test_time]
-
-        grouped_summary = @test_cases.group_by{|test_case| test_case.test_status }
-
-        summary_logger.notify "Failed Tests Cases:"
-        (grouped_summary[:fail] || []).each do |test_case|
-          summary_logger.notify print_test_result(test_case)
-        end
-
-        summary_logger.notify "Errored Tests Cases:"
-        (grouped_summary[:error] || []).each do |test_case|
-          summary_logger.notify print_test_result(test_case)
-        end
-
-        summary_logger.notify "Skipped Tests Cases:"
-        (grouped_summary[:skip] || []).each do |test_case|
-          summary_logger.notify print_test_result(test_case)
-        end
-
-        summary_logger.notify "Pending Tests Cases:"
-        (grouped_summary[:pending] || []).each do |test_case|
-          summary_logger.notify print_test_result(test_case)
-        end
-
-        summary_logger.notify("\n\n")
-      end
-
-      #A convenience method for printing the results of a {TestCase}
-      #@param [TestCase] test_case The {TestCase} to examine and print results for
-      def print_test_result(test_case)
-        if test_case.exception
-          test_file_trace = ""
-          test_case.exception.backtrace.each do |line|
-            if line.include?(test_case.path)
-              test_file_trace = "\r\n    Test line: #{line}"
-              break
-            end
-          end if test_case.exception.backtrace && test_case.path
-          test_reported = "reported: #{test_case.exception.inspect}#{test_file_trace}"
-        else
-          test_case.test_status
-        end
-        "  Test Case #{test_case.path} #{test_reported}"
-      end
-
-      # Writes Junit XML of this {TestSuiteResult}
-      #
-      # @param [String] xml_file      Path to the XML file (from Beaker's running directory)
-      # @param [String] file_to_link  Path to the paired file that should be linked
-      #                               from this one (this is relative to the XML
-      #                               file itself, so it would just be the different
-      #                               file name if they're in the same directory)
-      # @param [Boolean] time_sort    Whether the test results should be output in
-      #                               order of time spent in the test, or in the
-      #                               order of test execution (default)
-      #
-      # @return nil
-      # @api private
-      def write_junit_xml(xml_file, file_to_link = nil, time_sort = false)
-        stylesheet = File.join(@options[:project_root], @options[:xml_stylesheet])
-
-        begin
-          LoggerJunit.write_xml(xml_file, stylesheet) do |doc, suites|
-
-            meta_info = Nokogiri::XML::Node.new('meta_test_info', doc)
-            unless file_to_link.nil?
-              meta_info['page_active'] = time_sort ? 'performance' : 'execution'
-              meta_info['link_url'] = file_to_link
-            else
-              meta_info['page_active'] = 'no-links'
-              meta_info['link_url'] = ''
-            end
-            suites.add_child(meta_info)
-
-            suite = Nokogiri::XML::Node.new('testsuite', doc)
-            suite['name']     = @name
-            suite['tests']    = test_count
-            suite['errors']   = errored_tests
-            suite['failures'] = failed_tests
-            suite['skipped']     = skipped_tests
-            suite['pending']  = pending_tests
-            suite['total']    = @total_tests
-            suite['time']     = "%f" % (stop_time - start_time)
-            properties = Nokogiri::XML::Node.new('properties', doc)
-            @options.each_pair do | name, value |
-              property = Nokogiri::XML::Node.new('property', doc)
-              property['name']  = name
-              property['value'] = value
-              properties.add_child(property)
-            end
-            suite.add_child(properties)
-
-            test_cases_to_report = @test_cases
-            test_cases_to_report = @test_cases.sort { |x,y| y.runtime <=> x.runtime } if time_sort
-            test_cases_to_report.each do |test|
-              item = Nokogiri::XML::Node.new('testcase', doc)
-              item['classname'] = File.dirname(test.path)
-              item['name']      = File.basename(test.path)
-              item['time']      = "%f" % test.runtime
-
-              #ugh. nokogiri!!!  item can't take a hash, let alone an array.
-              test.exports.each do |export|
-                export.keys.each do |key|
-                  item[key] = export[key]
-                end
-              end
-
-              # Did we fail?  If so, report that.
-              # We need to remove the escape character from colorized text, the
-              # substitution of other entities is handled well by Rexml
-              if test.test_status == :fail || test.test_status == :error then
-                status = Nokogiri::XML::Node.new('failure', doc)
-                status['type'] =  test.test_status.to_s
-                if test.exception then
-                  status['message'] = test.exception.to_s.gsub(/\e/, '')
-                  data = LoggerJunit.format_cdata(test.exception.backtrace.join('\n'))
-                  status.add_child(status.document.create_cdata(data))
-                end
-                item.add_child(status)
-              end
-
-              if test.test_status == :skip
-                status = Nokogiri::XML::Node.new('skipped', doc)
-                status['type'] =  test.test_status.to_s
-                item.add_child(status)
-              end
-
-              if test.test_status == :pending
-                status = Nokogiri::XML::Node.new('pending', doc)
-                status['type'] =  test.test_status.to_s
-                item.add_child(status)
-              end
-
-              if test.sublog then
-                stdout = Nokogiri::XML::Node.new('system-out', doc)
-                data = LoggerJunit.format_cdata(test.sublog)
-                stdout.add_child(stdout.document.create_cdata(data))
-                item.add_child(stdout)
-              end
-
-              if test.last_result and test.last_result.stderr and not test.last_result.stderr.empty? then
-                stderr = Nokogiri::XML::Node.new('system-err', doc)
-                data = LoggerJunit.format_cdata(test.last_result.stderr)
-                stderr.add_child(stderr.document.create_cdata(data))
-                item.add_child(stderr)
-              end
-
-              suite.add_child(item)
-            end
-            suites.add_child(suite)
-          end
-        rescue Exception => e
-          @logger.error "failure in XML output:\n#{e.to_s}\n" + e.backtrace.join("\n")
-        end
-
-      end
-    end
-
     attr_reader :name, :options, :fail_mode
 
     #Create {TestSuite} instance