beaker 2.47.1 → 2.48.0

data/docs/{How-to-Write-a-Beaker-Test-for-a-Module.md → how_to/write_a_beaker_test_for_a_module.md}

@@ -1,11 +1,11 @@
 #Beaker for Modules
 ##Read the Beaker Docs
 
-[Beaker How To](How-To-Beaker.md)
+[Beaker How To](../tutorials/how_to_beaker.md)
 [Beaker DSL API](http://rubydoc.info/github/puppetlabs/beaker/frames)
 
 ##Understand the Difference Between beaker and beaker-rspec
-[beaker vs. beaker-rspec](beaker-vs.-beaker-rspec.md)
+[beaker vs. beaker-rspec](../concepts/beaker_vs_beaker_rspec.md)
 
 ##beaker-rspec Details
 See the [beaker-rspec README](https://github.com/puppetlabs/beaker-rspec/blob/master/README.md) for details on how to use beaker-rspec with modules.

data/docs/{Creating-A-Test-Environment.md → tutorials/creating_a_test_environment.md}

@@ -1,4 +1,7 @@
-Hosts/Nodes/SUTs are defined in the --hosts (--config) file in Yaml format.  This file defines each node in the test configuration. The file can be saved anywhere and used with `beaker --hosts yourhost.cfg` see [The Command Line](The-Command-Line.md) for more info.
+Hosts/Nodes/SUTs are defined in the --hosts (--config) file in Yaml format.
+This file defines each node in the test configuration. The file can be saved
+anywhere and used with `beaker --hosts yourhost.yaml`
+(see [The Command Line](the_command_line.md) for more info).
 
 Example hosts file:
 
@@ -81,12 +84,12 @@ The platform's format is `/^OSFAMILY-VERSION-ARCH.*$/` where `OSFAMILY` is one o
 `VERSION`'s format is not enforced, but should reflect the `OSFAMILY` selected (ie, ubuntu-1204-i386-master, scientific-6-i386-agent, etc).  `ARCH`'s format is also not enforced, but should be appropriate to the `OSFAMILY` selected (ie, ubuntu-1204-i386-master, sles-11-x86_64-master, debian-7-amd64-master, etc).
 
 ## Supported Virtualization Providers ##
-* [AWS](hypervisors/aws.md)
-* [VMWare Fusion](hypervisors/vmware_fusion.md)
-* [EC2](hypervisors/ec2.md)
-* [vSphere](hypervisors/vsphere.md)
-* [Vagrant](hypervisors/vagrant.md)
-* [Google Compute Engine](hypervisors/google_compute_engine.md)
-* [Docker Support](hypervisors/docker.md)
-* [Openstack](hypervisors/openstack.md)
-* [Solaris](hypervisors/solaris.md)
+* [AWS](../how_to/hypervisors/aws.md)
+* [VMWare Fusion](../how_to/hypervisors/vmware_fusion.md)
+* [EC2](../how_to/hypervisors/ec2.md)
+* [vSphere](../how_to/hypervisors/vsphere.md)
+* [Vagrant](../how_to/hypervisors/vagrant.md)
+* [Google Compute Engine](../how_to/hypervisors/google_compute_engine.md)
+* [Docker Support](../how_to/hypervisors/docker.md)
+* [Openstack](../how_to/hypervisors/openstack.md)
+* [Solaris](../how_to/hypervisors/solaris.md)

data/docs/tutorials/how_to_beaker.md

@@ -0,0 +1,6 @@
+* [Beaker Installation](installation.md)
+* [Creating A Test Environment](creating_a_test_environment.md)
+* [The Command Line](the_command_line.md)
+* [The Beaker DSL](../how_to/the_beaker_dsl.md)
+* [Let's Write a Test!](lets_write_a_test.md)
+* [Access The Live Test Console with Pry](../how_to/access_the_live_test_console_with_pry.md)

data/docs/{Lets-Write-a-Test.md → tutorials/lets_write_a_test.md}

@@ -1,8 +1,8 @@
-###The Task
+##The Task
 
 Consider if mcollectived incorrectly spawned a new process with every puppet agent run on Ubuntu 10.04.  We need an acceptance test to check that a new process is not spawned and to ensure that this issue does not regress in new builds.
 
-###Figure Out Test Steps
+##Figure Out Test Steps
 
 What needs to happen in this test:
 
@@ -10,7 +10,10 @@ What needs to happen in this test:
 * Restart mcollective twice
 * Check to see if more than one mcollective process is running
 
-###Install PE
+##Create a host configuration file
+    $ beaker-hostgenerator redhat7-64ma > redhat7-64ma.yaml
+
+##Install PE
 
 We prefer to install PE once and then run a set of tests, so PE installation should not be part of the actual acceptance test.
 
@@ -18,11 +21,12 @@ We prefer to install PE once and then run a set of tests, so PE installation sho
     $ cd setup
     $ cat > install.rb << RUBY
     test_name "Installing Puppet Enterprise" do
-    install_pe      
+    install_pe
     RUBY
     $ cd ..
 
-XXX - Will beaker just pick this up? Is the setup dir magic?
+This places our install steps in a ruby script (install.rb) which will run on localhost, but the #install_pe method knows where to install puppet enterprise based on the host configuration in use.
+The install.rb script is used in our commandline to beaker, below.
 
 ## Create a test file
 
@@ -38,13 +42,14 @@ Here's our magic command that restarts mcollective:
 ####Check to see if more than one mcollective process is running
 
 Here's our magic command that throws an error if more than one mcollective process is running:
-```
-process_count_check = "bash -c '[[ $(ps auxww | grep [m]collectived | wc -l) -eq 1 ]]'"
-```
+
+    process_count_check = "bash -c '[[ $(ps auxww | grep [m]collectived | wc -l) -eq 1 ]]'"
+
 ###Put it all together
 
 Here's the finished acceptance test.
-```
+
+```ruby
 test_name "/etc/init.d/pe-mcollective restart check"
 
 # Don't run these tests on the following platforms
@@ -66,8 +71,10 @@ hosts.each do |host|
   on(host, process_count_check) { assert_equal(0, exit_code) }
 end
 ```
+
+## Run it!
 You can now run this with
 
-  `beaker --host myhost.cfg --test mytest.rb`
+    beaker --host redhat7-64ma.yaml --pre-suite install.rb --test mytest.rb
 
-Next up you may want to look at the [Beaker test for a module](How-to-Write-a-Beaker-Test-for-a-Module.md) page.
+Next up you may want to look at the [Beaker test for a module](../how_to/write_a_beaker_test_for_a_module.md) page.

data/lib/beaker/dsl/helpers/host_helpers.rb

@@ -224,6 +224,45 @@ module Beaker
           end
         end
 
+        # Copy a remote file to the local system and save it under a directory
+        # meant for storing SUT files to be viewed in the event of test failures.
+        #
+        # Files are stored locally with the following structure:
+        #   ./<archive_root>/<hostname>/<from_path>
+        #
+        # This can be used during the post-suite phase to persist relevant log
+        # files from the SUTs so they can available with the test results
+        # (without having to preserve the SUT host and SSH in after the fact).
+        #
+        # Example
+        #
+        #   Archive the Puppet Server log file from the master ('abc123'),
+        #   and archive the Puppet Agent log file from the agent ('xyz098'):
+        #
+        #     archive_file_from(master, '/var/log/puppetlabs/puppetserver.log')
+        #     archive_file_from(agent, '/var/log/puppetlabs/puppetagent.log')
+        #
+        #   Results in the following files on the test runner:
+        #
+        #     archive/sut-files/abc123/var/log/puppetlabs/puppetserver.log
+        #     archive/sut-files/xyz098/var/log/puppetlabs/puppetagent.log
+        #
+        # @param [Host] host A host object (or some object that can be passed to
+        #                    #scp_from)
+        # @param [String] from_path A remote absolute path on the host to copy.
+        # @!macro common_opts
+        # @option [String] archive_root The local directory to store the copied
+        #                               file under. Defaults to
+        #                               'archive/sut-files'.
+        #
+        # @return [Result] Returns the result of the #scp_from operation.
+        def archive_file_from(host, from_path, opts = {}, archive_root = 'archive/sut-files')
+          filedir = File.dirname(from_path)
+          targetdir = File.join(archive_root, host.hostname, filedir)
+          FileUtils.mkdir_p(targetdir)
+          scp_from(host, from_path, targetdir, opts)
+        end
+
         # Deploy packaging configurations generated by
         # https://github.com/puppetlabs/packaging to a host.
         #

data/lib/beaker/dsl/helpers/puppet_helpers.rb

@@ -505,13 +505,22 @@ module Beaker
         #                            or a role (String or Symbol) that identifies one or more hosts.
         # @param ip_spec [Hash{String=>String}] a hash containing the host to ip
         #   mappings
-        # @example Stub puppetlabs.com on the master to 127.0.0.1
-        #   stub_hosts_on(master, 'puppetlabs.com' => '127.0.0.1')
-        def stub_hosts_on(machine, ip_spec)
+        # @param alias_spec [Hash{String=>Array[String]] an hash containing the host to alias(es) mappings to apply
+        # @example Stub puppetlabs.com on the master to 127.0.0.1 with an alias example.com
+        #   stub_hosts_on(master, {'puppetlabs.com' => '127.0.0.1'}, {'puppetlabs.com' => ['example.com']})
+        def stub_hosts_on(machine, ip_spec, alias_spec={})
           block_on machine do | host |
             ip_spec.each do |address, ip|
+              aliases = alias_spec[address] || []
+              manifest =<<-EOS.gsub /^\s+/, ""
+                host { '#{address}':
+                  \tensure       => present,
+                  \tip           => '#{ip}',
+                  \thost_aliases => #{aliases},
+                }
+              EOS
               logger.notify("Stubbing address #{address} to IP #{ip} on machine #{host}")
-              on( host, puppet('resource', 'host', address, 'ensure=present', "ip=#{ip}") )
+              apply_manifest_on( host, manifest )
             end
 
             teardown do
@@ -530,17 +539,15 @@ module Beaker
         #                            or a role (String or Symbol) that identifies one or more hosts.
         # @param ip_spec [Hash{String=>String}] a hash containing the host to ip
         #   mappings
-        # @example Stub puppetlabs.com on the master to 127.0.0.1
-        #   with_host_stubbed_on(master, 'forgeapi.puppetlabs.com' => '127.0.0.1') do
+        # @param alias_spec [Hash{String=>Array[String]] an hash containing the host to alias(es) mappings to apply
+        # @example Stub forgeapi.puppetlabs.com on the master to 127.0.0.1 with an alias forgeapi.example.com
+        #   with_host_stubbed_on(master, {'forgeapi.puppetlabs.com' => '127.0.0.1'}, {'forgeapi.puppetlabs.com' => ['forgeapi.example.com']}) do
         #     puppet( "module install puppetlabs-stdlib" )
         #   end
-        def with_host_stubbed_on(host, ip_spec, &block)
+        def with_host_stubbed_on(host, ip_spec, alias_spec={}, &block)
           begin
             block_on host do |host|
-              ip_spec.each_pair do |address, ip|
-                logger.notify("Stubbing address #{address} to IP #{ip} on machine #{host}")
-                on( host, puppet('resource', 'host', address, 'ensure=present', "ip=#{ip}") )
-              end
+              stub_hosts_on(host, ip_spec, alias_spec)
             end
 
             block.call
@@ -574,11 +581,11 @@ module Beaker
         #                             global options hash
         def stub_forge_on(machine, forge_host = nil)
           #use global options hash
+          primary_forge_name = 'forge.puppetlabs.com'
           forge_host ||= options[:forge_host]
           @forge_ip ||= Resolv.getaddress(forge_host)
           block_on machine do | host |
-            stub_hosts_on(host, 'forge.puppetlabs.com' => @forge_ip)
-            stub_hosts_on(host, 'forgeapi.puppetlabs.com' => @forge_ip)
+            stub_hosts_on(host, {primary_forge_name => @forge_ip}, {primary_forge_name => ['forge.puppet.com','forgeapi.puppetlabs.com','forgeapi.puppet.com']})
           end
         end
 
@@ -593,12 +600,10 @@ module Beaker
         #                             global options hash
         def with_forge_stubbed_on( host, forge_host = nil, &block )
           #use global options hash
+          primary_forge_name = 'forge.puppetlabs.com'
           forge_host ||= options[:forge_host]
           @forge_ip ||= Resolv.getaddress(forge_host)
-          with_host_stubbed_on( host,
-                                {'forge.puppetlabs.com'  => @forge_ip,
-                               'forgeapi.puppetlabs.com' => @forge_ip},
-                                &block                                    )
+          with_host_stubbed_on( host, {primary_forge_name => @forge_ip}, {primary_forge_name => ['forge.puppet.com','forgeapi.puppetlabs.com','forgeapi.puppet.com']}, &block )
         end
 
         # This wraps `with_forge_stubbed_on` and provides it the default host

data/lib/beaker/dsl/install_utils/foss_utils.rb

@@ -1081,21 +1081,29 @@ module Beaker
 
             case variant
             when /^(fedora|el|centos|debian|ubuntu|cumulus|huaweios|cisco_nexus|cisco_ios_xr)$/
-              sha = opts[:puppet_agent_sha] || opts[:puppet_agent_version]
-              opts[:dev_builds_repos] ||= [ opts[:puppet_collection] ]
-              install_puppetlabs_dev_repo( host, 'puppet-agent', sha, nil, opts )
-              host.install_package('puppet-agent')
-              logger.trace("#install_puppet_agent_dev_repo_on: install_puppetlabs_dev_repo finished")
-              next
+              if arch == 's390x'
+                logger.trace("#install_puppet_agent_dev_repo_on: s390x arch detected for host #{host}. using dev package")
+              else
+                sha = opts[:puppet_agent_sha] || opts[:puppet_agent_version]
+                opts[:dev_builds_repos] ||= [ opts[:puppet_collection] ]
+                install_puppetlabs_dev_repo( host, 'puppet-agent', sha, nil, opts )
+                host.install_package('puppet-agent')
+                logger.trace("#install_puppet_agent_dev_repo_on: install_puppetlabs_dev_repo finished")
+                next
+              end
             when /^(eos|osx|windows|solaris|sles|aix)$/
-              release_path_end, release_file = host.puppet_agent_dev_package_info(
-                opts[:puppet_collection], opts[:puppet_agent_version], opts )
-              release_path << release_path_end
-              logger.trace("#install_puppet_agent_dev_repo_on: dev_package_info, continuing...")
+              # Download installer package file & run install manually.
+              # Done below, so that el hosts with s390x arch can use this
+              # workflow as well
             else
               raise "No repository installation step for #{variant} yet..."
             end
 
+            release_path_end, release_file = host.puppet_agent_dev_package_info(
+              opts[:puppet_collection], opts[:puppet_agent_version], opts )
+            release_path << release_path_end
+            logger.trace("#install_puppet_agent_dev_repo_on: dev_package_info, continuing...")
+
             if host['platform'] =~ /eos/
               host.get_remote_file( "#{release_path}/#{release_file}" )
             else
@@ -1107,7 +1115,7 @@ module Beaker
             case variant
             when /^eos/
               host.install_from_file( release_file )
-            when /^(sles|aix)$/
+            when /^(sles|aix|el)$/
               # NOTE: AIX does not support repo management. This block assumes
               # that the desired rpm has been mirrored to the 'repos' location.
               on host, "rpm -ivh #{onhost_copied_file}"

data/lib/beaker/dsl/install_utils/windows_utils.rb

@@ -152,6 +152,45 @@ exit /B %errorlevel%
           end
         end
 
+        # Installs a specified msi path on given hosts
+        # @param [Host, Array<Host>, String, Symbol] hosts    One or more hosts to act upon,
+        #                            or a role (String or Symbol) that identifies one or more hosts.
+        # @param [String] msi_path The path of the MSI - can be a local Windows style file path like
+        #                   c:\temp\foo.msi OR a url like https://download.com/foo.msi or file://c:\temp\foo.msi
+        # @param  [Hash{String=>String}] msi_opts MSI installer options
+        # @option opts [Boolean] :debug output the MSI installation log when set to true
+        #                 otherwise do not output log (false; default behavior)
+        #
+        # @example
+        #  generic_install_msi_on(hosts, 'https://releases.hashicorp.com/vagrant/1.8.4/vagrant_1.8.4.msi', {}, {:debug => true})
+        #
+        # @api private
+        def generic_install_msi_on(hosts, msi_path, msi_opts = {}, opts = {})
+          block_on hosts do | host |
+            batch_path, log_file = create_install_msi_batch_on(host, msi_path, msi_opts)
+
+            # begin / rescue here so that we can reuse existing error msg propagation
+            begin
+              # 1641 = ERROR_SUCCESS_REBOOT_INITIATED
+              # 3010 = ERROR_SUCCESS_REBOOT_REQUIRED
+              on host, Command.new("\"#{batch_path}\"", [], { :cmdexe => true }), :acceptable_exit_codes => [0, 1641, 3010]
+            rescue
+              on host, Command.new("type \"#{log_file}\"", [], { :cmdexe => true })
+              raise
+            end
+
+            if opts[:debug]
+              on host, Command.new("type \"#{log_file}\"", [], { :cmdexe => true })
+            end
+
+            if !host.is_cygwin?
+              # HACK: for some reason, post install we need to refresh the connection to make puppet available for execution
+              host.close
+            end
+
+          end
+        end
+
       end
     end
   end

data/lib/beaker/host/mac/pkg.rb

@@ -6,8 +6,26 @@ module Mac::Pkg
   end
 
   def install_package(name, cmdline_args = '', version = nil)
-    execute("hdiutil attach #{name}.dmg")
-    execute("installer -pkg /Volumes/#{name}/#{name}.pkg -target /")
+    generic_install_dmg("#{name}.dmg", name, "#{name}.pkg")
+  end
+
+  # Install a package from a specified dmg
+  #
+  # @param [String] dmg_file      The dmg file, including path if not
+  #                               relative. Can be a URL.
+  # @param [String] pkg_base      The base name of the directory that the dmg
+  #                               attaches to under `/Volumes`
+  # @param [String] pkg_name      The name of the package file that should be
+  #                               used by the installer
+  # @example: Install vagrant from URL
+  #   mymachost.generic_install_dmg('https://releases.hashicorp.com/vagrant/1.8.4/vagrant_1.8.4.dmg', 'Vagrant', 'Vagrant.pkg')
+  def generic_install_dmg(dmg_file, pkg_base, pkg_name)
+    execute("test -f #{dmg_file}", :accept_all_exit_codes => true) do |result|
+      execute("curl -O #{dmg_file}") unless result.exit_code == 0
+    end
+    dmg_name = File.basename(dmg_file, '.dmg')
+    execute("hdiutil attach #{dmg_name}.dmg")
+    execute("installer -pkg /Volumes/#{pkg_base}/#{pkg_name} -target /")
   end
 
   def uninstall_package(name, cmdline_args = '')

data/lib/beaker/host/unix/exec.rb

@@ -99,7 +99,7 @@ module Unix::Exec
   #@example
   #  host.add_env_var('PATH', '/usr/bin:PATH')
   def add_env_var key, val
-    key = key.to_s.upcase
+    key = key.to_s
     env_file = self[:ssh_env_file]
     escaped_val = Regexp.escape(val).gsub('/', '\/').gsub(';', '\;')
     #see if the key/value pair already exists
@@ -122,7 +122,7 @@ module Unix::Exec
   #@example
   #  host.delete_env_var('PATH', '/usr/bin:PATH')
   def delete_env_var key, val
-    key = key.to_s.upcase
+    key = key.to_s
     env_file = self[:ssh_env_file]
     val = Regexp.escape(val).gsub('/', '\/').gsub(';', '\;')
     #if the key only has that single value remove the entire line
@@ -141,7 +141,7 @@ module Unix::Exec
   #@example
   #  host.get_env_var('path')
   def get_env_var key
-    key = key.to_s.upcase
+    key = key.to_s
     exec(Beaker::Command.new("env | grep #{key}"), :accept_all_exit_codes => true).stdout.chomp
   end
 
@@ -150,7 +150,7 @@ module Unix::Exec
   #@example
   #  host.clear_env_var('PATH')
   def clear_env_var key
-    key = key.to_s.upcase
+    key = key.to_s
     env_file = self[:ssh_env_file]
     #remove entire line
     exec(Beaker::SedCommand.new(self['platform'], "/#{key}=.*$/d", env_file))

data/lib/beaker/host/unix/pkg.rb

@@ -384,7 +384,7 @@ module Unix::Pkg
     when /^(solaris)$/
       release_path_end, release_file = solaris_puppet_agent_dev_package_info(
         puppet_collection, puppet_agent_version, opts )
-    when /^(sles|aix)$/
+    when /^(sles|aix|el)$/
       arch = 'ppc' if variant == 'aix' && arch == 'power'
       release_path_end = "#{variant}/#{version}/#{puppet_collection}/#{arch}"
       release_file = "puppet-agent-#{puppet_agent_version}-1.#{variant}#{version}.#{arch}.rpm"

data/lib/beaker/test_suite.rb

@@ -140,11 +140,18 @@ module Beaker
       #A convenience method for printing the results of a {TestCase}
       #@param [TestCase] test_case The {TestCase} to examine and print results for
       def print_test_result(test_case)
-        test_reported = if test_case.exception
-                          "reported: #{test_case.exception.inspect}"
-                        else
-                          test_case.test_status
-                        end
+        if test_case.exception
+          test_file_trace = ""
+          test_case.exception.backtrace.each do |line|
+            if line.include?(test_case.path)
+              test_file_trace = "\r\n    Test line: #{line}"
+              break
+            end
+          end if test_case.exception.backtrace && test_case.path
+          test_reported = "reported: #{test_case.exception.inspect}#{test_file_trace}"
+        else
+          test_case.test_status
+        end
         @logger.notify "  Test Case #{test_case.path} #{test_reported}"
       end