beaker 3.37.0 → 4.0.0

data/docs/how_to/upgrade_from_3_to_4.md

@@ -0,0 +1,52 @@
+# How To Upgrade from 3.y to 4.0
+
+This is a guide detailing all the issues to be aware of, and to help people make any changes that you might need to move from beaker 2.y to 3.0. To test out beaker 4.0.0, we recommend implementing the strategy outlined [here](test_arbitrary_beaker_versions.md) to ensure this new major release does not break your existing testing.
+
+## PE Dependency
+
+In Beaker 3.0.0, `beaker-pe` was removed as a dependency. A mechanism to automatically include that module, if available, was added for convenience and to ease the transition. That shim has been removed. As of 4.0, you will have to explicitly require `beaker-pe` alongside `beaker` as a dependency in your project if you need it in your tests. You'll also need to add `require 'beaker-pe'` to any tests that use it.
+
+## `PEDefaults` and `#configure_type_defaults_on`
+
+PEDefaults has been moved to `beaker-pe`. The call to `#configure_type_defaults_on` that was previously made in `#set_env` is no longer made. You will now need to explicitly call `#configure_type_defaults_on` in your tests when needed.
+
+## Puppet Dependency
+
+Just like `beaker-pe` was removed as a dependency in 3.0, we have removed `beaker-puppet` as a dependency in 4.0. This means that you will have to explicitly require `beaker-puppet` alongside `beaker` as a dependency in your project if you need it in your tests. You'll also need to add `require 'beaker-puppet'` to any of your tests that use it.
+
+## Hypervisor Loading
+
+We have also removed the explicit dependency on all previously-included hypervisor libraries. Don't worry, the transition should be easy.
+
+In order to use a specific hypervisor or DSL extension library in your project, you will need to include them alongside Beaker in your Gemfile or project.gemspec. E.g.
+
+~~~ruby
+# Gemfile
+gem 'beaker', '~>4.0'
+gem 'beaker-aws'
+# project.gemspec
+s.add_runtime_dependency 'beaker', '~>4.0'
+s.add_runtime_dependency 'beaker-aws'
+~~~
+
+Beaker will automatically load the appropriate hypervisors for any given hosts file, so as long as your project dependencies are satisfied there's nothing else to do. No need to `require` this library in your tests. Simply specify `hypervisor: hypervisor_name` in your hosts file.
+
+The following hypervisor libraries were removed in 4.0:
+
+- [beaker-abs](github.com/puppetlabs/beaker-abs)
+- [beaker-aws](github.com/puppetlabs/beaker-aws)
+- [beaker-docker](github.com/puppetlabs/beaker-docker)
+- [beaker-google](github.com/puppetlabs/beaker-google)
+- [beaker-openstack](github.com/puppetlabs/beaker-openstack)
+- [beaker-vagrant](github.com/puppetlabs/beaker-vagrant)
+- [beaker-vcloud](github.com/puppetlabs/beaker-vcloud)
+- [beaker-vmpooler](github.com/puppetlabs/beaker-vmpooler)
+- [beaker-vmware](github.com/puppetlabs/beaker-vmware)
+
+For acceptance testing, beaker-vmpooler, beaker-aws, and beaker-abs have been retained as development dependencies. These will be removed as the CI pipelines is upgraded, so *do not rely on them being there for your project*.
+
+## `Host.rsync_to` and Rsync-dependent Methods
+
+The `Host.rsync_to()` method has been overhauled, fixing a number of lingering issues and edge cases including [BKR-462](http://tickets.puppetlabs.com/browse/BKR-462) and [BKR-463](http://tickets.puppetlabs.com/browse/BKR-463). When a call to `rsync_to` fails for any reason, Beaker will throw `CommandFailure` as standard, including an error message from the Rsync library. The two primary causes for previously silent Rsync failures are Rsync not installed on remote host (which caused failures on Windows, as well as hosts without Rsync installed by default) and remote path not existing, both documented in the linked tickets.
+
+However, it is important to note that any error causing Rsync to fail, including SSH transport errors and hostname mismatches, will throw `CommandFailure` for `rsync_to` *and any method that relies on `rsync_to` including `create_remote_file_on()`*. If the remote file is not created for any reason, the command will fail.

data/lib/beaker.rb

@@ -45,14 +45,4 @@ module Beaker
   rescue LoadError
     # do nothing
   end
-
-  # If beaker-pe is available, pull it in. The gem beaker-pe will need to be
-  # specified in the project Gemfile independent of beaker itself. If not available,
-  # catch LoadError and continue.
-  begin
-    require 'beaker-pe'
-  rescue LoadError
-    # do nothing
-  end
-
 end

data/lib/beaker/dsl.rb

@@ -83,6 +83,13 @@ module Beaker
 
     def self.register(helper_module)
       include helper_module
+
+      # Modules added into a module which has previously been included are not
+      # retroactively included in the including class. Do this here so we don't
+      # have to in every DSL extension library.
+      #
+      # https://github.com/adrianomitre/retroactive_module_inclusion
+      Beaker::TestCase.class_eval { include Beaker::DSL }
     end
   end
 end

data/lib/beaker/dsl/helpers.rb

@@ -3,13 +3,13 @@
       require "beaker/dsl/helpers/#{lib}_helpers"
 end
 
-require "beaker-hiera"
-require 'beaker-puppet'
 module Beaker
   module DSL
 
-    # Contains methods to help you manage and configure your SUTs and configure and interact with puppet, facter
-    # and hiera.
+    # Contains methods to help you manage and configure your SUTs.
+
+    # Extensions, available in separate modules, enable you to configure and interact with puppet, facter
+    # and hiera. See [the docs](/docs/how_to/the_beaker_dsl.md).
 
     # To mix this is into a class you need the following:
     # * a method *hosts* that yields any hosts implementing
@@ -28,8 +28,6 @@ module Beaker
       include Beaker::DSL::Helpers::TestHelpers
       include Beaker::DSL::Helpers::WebHelpers
       include Beaker::DSL::Helpers::HoconHelpers
-      include Beaker::DSL::Helpers::Hiera
-      include BeakerPuppet::Helpers
     end
   end
 end

data/lib/beaker/dsl/helpers/host_helpers.rb

@@ -394,6 +394,16 @@ module Beaker
           host.install_package package_name, '', package_version
         end
 
+        # Uninstall a package on a host
+        #
+        # @param [Host] host             A host object
+        # @param [String] package_name   Name of the package to uninstall
+        #
+        # @return [Result]   An object representing the outcome of *uninstall command*.
+        def uninstall_package host, package_name
+          host.uninstall_package package_name
+        end
+
         # Check to see if a package is installed on a remote host
         #
         # @param [Host] host             A host object
@@ -558,40 +568,6 @@ module Beaker
           end
         end
 
-        # Create a temp directory on remote host owned by specified user.
-        #
-        # @param [Host, Array<Host>, String, Symbol] host    One or more hosts to act upon,
-        #                            or a role (String or Symbol) that identifies one or more hosts.
-        # @param [String] path_prefix A remote path prefix for the new temp
-        # directory.
-        # @param [String] user The name of user that should own the temp
-        # directory. If no username is specified defaults to the currently logged in user
-        # per host
-        #
-        # @return [String, Array<String>] Returns the name of the newly-created dir, or an array
-        #                                of names of newly-created dirs per-host
-        def create_tmpdir_on(host, path_prefix = '', user=nil)
-
-          block_on host do | host |
-            # use default user logged into this host
-            if not user
-              user = host['user']
-            end
-
-            if not on(host, "getent passwd #{user}").exit_code == 0
-              raise "User #{user} does not exist on #{host}."
-            end
-
-            if defined? host.tmpdir
-              dir = host.tmpdir(path_prefix)
-              on host, "chown #{user}:#{user} #{dir}"
-              dir
-            else
-              raise "Host platform not supported by `create_tmpdir_on`."
-            end
-          end
-        end
-
         # 'echo' the provided value on the given host(s)
         # @param [Host, Array<Host>, String, Symbol] hosts    One or more hosts to act upon,
         #                            or a role (String or Symbol) that identifies one or more hosts.

data/lib/beaker/dsl/install_utils.rb

@@ -1,12 +1,7 @@
-require "beaker/dsl/install_utils/pe_defaults"
-require 'beaker-puppet'
-
 module Beaker
   module DSL
     # Collection of installation methods and support
     module InstallUtils
-      include DSL::InstallUtils::PEDefaults
-      include BeakerPuppet::InstallUtils
     end
   end
 end

data/lib/beaker/dsl/roles.rb

@@ -139,7 +139,7 @@ module Beaker
           return !version_is_less(version, '4.0') if version && !version.empty?
         end
         return true if host[:roles] && host[:roles].include?('aio')
-        return true if host[:type] && !!(host[:type] =~ /(\A|-)aio(\Z|-)/ )
+        return true if host[:type] && !!(host[:type] =~ /(\A|-)(aio|git)(\Z|-)/ )
         false
       end
 

data/lib/beaker/host.rb

@@ -502,6 +502,8 @@ module Beaker
     # @param to_path [String] The destination path on the host
     # @param opts [Hash{Symbol=>String}] Options to alter execution
     # @option opts [Array<String>] :ignore An array of file/dir paths that will not be copied to the host
+    # @raise [Beaker::Host::CommandFailure] With Rsync error (if available)
+    # @return [Rsync::Result] Rsync result with status code
     def do_rsync_to from_path, to_path, opts = {}
       ssh_opts = self['ssh']
       rsync_args = []
@@ -532,18 +534,15 @@ module Beaker
             ssh_opts.has_key?('auth_methods') and
             ssh_opts['auth_methods'].include?('publickey')
 
-          key = ssh_opts['keys']
-
-          # If an array was set, then we use the first value
-          if key.is_a? Array
-            key = key.first
+          # find the first SSH key that exists
+          key = Array(ssh_opts['keys']).find do |k|
+              File.exist?(k)
           end
 
-          # We need to expand tilde manually as rsync can be
-          # funny sometimes
-          key = File.expand_path(key)
-
-          ssh_args << "-i #{key}"
+          if key
+            # rsync doesn't always play nice with tilde, so be sure to expand first
+            ssh_args << "-i #{File.expand_path(key)}"
+          end
         end
       end
 
@@ -571,7 +570,9 @@ module Beaker
       @logger.notify "rsync: localhost:#{from_path} to #{hostname_with_user}:#{to_path} {:ignore => #{opts[:ignore]}}"
       result = Rsync.run(from_path, to_path, rsync_args)
       @logger.debug("rsync returned #{result.inspect}")
-      result
+
+      return result if result.success?
+      raise Beaker::Host::CommandFailure, result.error
     end
 
   end

data/lib/beaker/host/aix/file.rb

@@ -1,11 +1,11 @@
 module Aix::File
   include Beaker::CommandFactory
 
-  def tmpfile(name)
+  def tmpfile(name = '')
     execute("rndnum=${RANDOM} && touch /tmp/#{name}.${rndnum} && echo /tmp/#{name}.${rndnum}")
   end
 
-  def tmpdir(name)
+  def tmpdir(name = '')
     execute("rndnum=${RANDOM} && mkdir /tmp/#{name}.${rndnum} && echo /tmp/#{name}.${rndnum}")
   end
 

data/lib/beaker/host/aix/group.rb

@@ -14,6 +14,7 @@ module Aix::Group
       fail_test "failed to get group #{name}" unless result.stdout =~ /^#{name} id/
 
       yield result if block_given?
+      result
     end
   end
 

data/lib/beaker/host/aix/user.rb

@@ -19,6 +19,7 @@ module Aix::User
       fail_test "failed to get user #{name}" unless result.stdout =~  /^#{name} id/
 
       yield result if block_given?
+      result
     end
   end
 

data/lib/beaker/host/freebsd/pkg.rb

@@ -25,6 +25,15 @@ module FreeBSD::Pkg
     execute(cmd, opts) { |result| result }
   end
 
+  def uninstall_package(package, cmdline_args = nil, opts = {})
+    cmd = if pkgng_active?
+            "pkg delete #{cmdline_args || '-y'} #{package}"
+          else
+            "pkg_delete #{cmdline_args || '-r'} #{package}"
+          end
+    execute(cmd, opts) { |result| result }
+  end
+
   def check_for_package(package, opts = {})
     opts = {:accept_all_exit_codes => true}.merge(opts)
     cmd = if pkgng_active?

data/lib/beaker/host/mac/group.rb

@@ -39,6 +39,7 @@ module Mac::Group
       answer = "#{gi[:name]}:#{gi[:password]}:#{gi[:gid]}"
 
       yield answer if block_given?
+      answer
     end
   end
 

data/lib/beaker/host/mac/user.rb

@@ -21,28 +21,23 @@ module Mac::User
 
   # Gets the user information in /etc/passwd format
   #
+  # @note Calls POSIX-compliant `$ id -P <user>` to get /etc/passwd-style
+  # output
+  #
   # @param [String] name Name of the user
   # @param [Proc] block Additional actions or insertions
   #
-  # @yield [String] The actual mac dscacheutil output
-  # @return [String] User information in /etc/passwd format
+  # @yield [Result] User information in /etc/passwd format
+  # @return [Result] User information in /etc/passwd format
   # @raise [FailTest] Raises an Assertion failure if it can't find the name
   #                   queried for in the returned block
   def user_get(name, &block)
-    answer = ""
-    execute("dscacheutil -q user -a name #{name}") do |result|
-      fail_test "failed to get user #{name}" unless result.stdout =~  /^name: #{name}/
-      ui = Hash.new  # user info
-      result.stdout.each_line { |line|
-        pieces = line.split(': ')
-        ui[pieces[0].to_sym] = pieces[1].strip if pieces[1] != nil
-      }
-      answer  = "#{ui[:name]}:#{ui[:password]}:#{ui[:uid]}:#{ui[:gid]}:"
-      answer << "#{ui[:name]}:#{ui[:dir]}:#{ui[:shell]}"
+    execute("id -P #{name}") do |result|
+      fail_test "failed to get user #{name}" unless result.stdout =~  /^#{name}:/
 
       yield result if block_given?
+      result
     end
-    answer
   end
 
   # Makes sure the user is present, creating them if necessary

data/lib/beaker/host/pswindows/file.rb

@@ -1,12 +1,12 @@
 module PSWindows::File
   include Beaker::CommandFactory
 
-  def tmpfile(name)
+  def tmpfile(name = '')
     result = exec(powershell('[System.IO.Path]::GetTempFileName()'))
     result.stdout.chomp()
   end
 
-  def tmpdir(name)
+  def tmpdir(name = '')
     result = exec(powershell('[System.IO.Path]::GetTempPath()'))
     result.stdout.chomp()
   end

data/lib/beaker/host/pswindows/group.rb

@@ -19,6 +19,7 @@ module PSWindows::Group
       fail_test "failed to get group #{name}" if result.exit_code != 0
 
       yield result if block_given?
+      result
     end
   end
 

data/lib/beaker/host/pswindows/user.rb

@@ -19,6 +19,7 @@ module PSWindows::User
       fail_test "failed to get user #{name}" if result.exit_code != 0
 
       yield result if block_given?
+      result
     end
   end
 

data/lib/beaker/host/unix/exec.rb

@@ -189,25 +189,25 @@ module Unix::Exec
   def ssh_permit_user_environment
     case self['platform']
     when /debian|ubuntu|cumulus|huaweios|archlinux/
-      directory = create_tmpdir_on(self)
+      directory = tmpdir()
       exec(Beaker::Command.new("echo 'PermitUserEnvironment yes' | cat - /etc/ssh/sshd_config > #{directory}/sshd_config.permit"))
       exec(Beaker::Command.new("mv #{directory}/sshd_config.permit /etc/ssh/sshd_config"))
     when /el-7|centos-7|redhat-7|oracle-7|scientific-7|eos-7/
-      directory = create_tmpdir_on(self)
+      directory = tmpdir()
       exec(Beaker::Command.new("echo 'PermitUserEnvironment yes' | cat - /etc/ssh/sshd_config > #{directory}/sshd_config.permit"))
       exec(Beaker::Command.new("mv #{directory}/sshd_config.permit /etc/ssh/sshd_config"))
     when /el-|centos|fedora|redhat|oracle|scientific|eos/
-      directory = create_tmpdir_on(self)
+      directory = tmpdir()
       exec(Beaker::Command.new("echo 'PermitUserEnvironment yes' | cat - /etc/ssh/sshd_config > #{directory}/sshd_config.permit"))
       exec(Beaker::Command.new("mv #{directory}/sshd_config.permit /etc/ssh/sshd_config"))
     when /sles/
-      directory = create_tmpdir_on(self)
+      directory = tmpdir()
       exec(Beaker::Command.new("echo 'PermitUserEnvironment yes' | cat - /etc/ssh/sshd_config > #{directory}/sshd_config.permit"))
       exec(Beaker::Command.new("mv #{directory}/sshd_config.permit /etc/ssh/sshd_config"))
     when /solaris/
       # kept solaris here because refactoring it into its own Host module
       # conflicts with the solaris hypervisor that already exists
-      directory = create_tmpdir_on(self)
+      directory = tmpdir()
       exec(Beaker::Command.new("echo 'PermitUserEnvironment yes' | cat - /etc/ssh/sshd_config > #{directory}/sshd_config.permit"))
       exec(Beaker::Command.new("mv #{directory}/sshd_config.permit /etc/ssh/sshd_config"))
     when /(free|open)bsd/

data/lib/beaker/host/unix/file.rb

@@ -1,11 +1,11 @@
 module Unix::File
   include Beaker::CommandFactory
 
-  def tmpfile(name)
+  def tmpfile(name = '')
     execute("mktemp -t #{name}.XXXXXX")
   end
 
-  def tmpdir(name)
+  def tmpdir(name = '')
     execute("mktemp -dt #{name}.XXXXXX")
   end
 
@@ -13,6 +13,47 @@ module Unix::File
     '/tmp'
   end
 
+  # Change user ownership of a path
+  #
+  # @see http://pubs.opengroup.org/onlinepubs/9699919799/utilities/chown.html
+  #
+  # @note To maintain argument order consistency with the underlying
+  #   syscall, avoid having to specify nil arguments, and not do
+  #   anything hacky with the arguments list, this method does not
+  #   allow you to modify group ownership. Use Host::chgrp instead.
+  # @param [String] user User to chown to
+  # @param [String] path Path to chown
+  # @param [Boolean] recursive Whether to pass the recursive flag
+  #
+  # @return [Beaker::Result] result of command execution
+  def chown(user, path, recursive=false)
+    execute("chown #{recursive ? '-R ' : ''}#{user} #{path}")
+  end
+
+  # Change group ownership of a path
+  #
+  # @see http://pubs.opengroup.org/onlinepubs/9699919799/utilities/chgrp.html
+  #
+  # @param [String] group Group to chgrp to
+  # @param [String] path Path to chgrp
+  # @param [Boolean] recursive Whether to pass the recursive flag
+  #
+  # @return [Beaker::Result] result of command execution
+  def chgrp(group, path, recursive=false)
+    execute("chgrp #{recursive ? '-R ' : ''}#{group} #{path}")
+  end
+
+  # List long output of a path
+  #
+  # @see http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html
+  #
+  # @param [String] path Path to list properties of
+  #
+  # @return [Beaker::Result] result of command execution
+  def ls_ld(path)
+    execute("ls -ld #{path}")
+  end
+
   # Handles any changes needed in a path for SCP
   #
   # @param [String] path File path to SCP to