taste_tester 0.0.13 → 0.0.14

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 13e9b1fb00c7044fe801ccb1158bbaacbbe30dd9
-  data.tar.gz: 3fcd362ea2bbdfed358847a485a695c9669d44df
+SHA256:
+  metadata.gz: dbe88f9386cbb7c68b721ea057a30881dd98ba374a0e84528e43da739eb88d31
+  data.tar.gz: 642d43ce2aa006331680178e0ab15afba8d88323ab89d2e1cb0321f74c663b07
 SHA512:
-  metadata.gz: f4da6aa14ee5b8c3f732eae1480f9793318b377f1574bf1bb12f2ec8b39c91edd8e8635d9ebd6e9042bf62a767ae55655ee447b0b47bfb83fbba4d1b13832e66
-  data.tar.gz: 67db565bdee6e84b0ad23c73183f88af0873ac12a84ae1451c8587038222634e0bf6888ef1e9a30280631586ad7a350087ad7a2f74f249f3a324f208a3a2395a
+  metadata.gz: 3a256a6d7e67b346aad620a1782032394f68848ef9c085c24854b4d70ffdd718b64d4970c479b7080aba11b468423905759d0ae8f92d305aa38fe654187745b7
+  data.tar.gz: '048097e573d03b748a08d09438820d9ee144c90d0fb43e0375fbd91e1354215d7c82d8a0b39f97452b0752ab8dc79fe5b5863c9108727ba87fc2810271f27ee8'

data/README.md

@@ -37,13 +37,13 @@ taste-tester untest [host]   # Put host back in production
                              #   (optional - will revert itself after 1 hour)
 ```
 
-See the help for futher information.
+See the help for further information.
 
 ## Prerequisites
 
-* Taste Tester assumes that /etc/chef/client.rb and /etc/chef/client.pem on your
-servers is a symlink and that your real config is /etc/chef/client-prod.rb and
-/etc/chef/client-prod.pem, respectively.
+* Taste Tester assumes that `/etc/chef/client.rb` and `/etc/chef/client.pem` on your
+servers is a symlink and that your real config is `/etc/chef/client-prod.rb` and
+`/etc/chef/client-prod.pem`, respectively.
 
 * Taste Tester assumes that it's generally safe to "go back" to production. I.e.
 We set things up so you can set a cronjob to un-taste-test a server after the
@@ -61,6 +61,7 @@ you want to test on, i.e. SSH public/private keys, SSH certificates, Kerberos
 * Mixlib::Config
 * Colorize
 * BetweenMeals
+* Minitar
 
 ## Automatic Untesting
 
@@ -79,15 +80,15 @@ is also killing the ssh tunnel whose PID is in `/etc/chef/test_timestamp`
 ## Config file
 
 The default config file is `/etc/taste-tester-config.rb` but you may use -c to
-specify another. The config file works the same as client.rb does for Chef -
-there are a series of keywords that take an arguement and anything else is just
+specify another. The config file works the same as `client.rb` does for Chef -
+there are a series of keywords that take an argument and anything else is just
 standard Ruby.
 
 All command-line options are available in the config file:
 * debug (bool, default: `false`)
 * timestamp (bool, default: `false`)
 * config_file (string, default: `/etc/taste-tester-config.rb`)
-* plugin_path (string, default: `/etc/taste-tester-plugin.rb`)
+* plugin_path (string, no default)
 * repo (string, default: `#{ENV['HOME']}/ops`)
 * testing_time (int, default: `3600`)
 * chef_client_command (strng, default: `chef-client`)
@@ -110,6 +111,10 @@ The following options are also available:
   `#{ENV['HOME']}/.chef/taste-tester-ref.txt`
 * checksum_dir - The checksum directory to put in knife.conf for users. Default:
   `#{ENV['HOME']}/.chef/checksums`
+* bundle - use a single tar.gz file for transporting cookbooks, roles and
+  databags to clients. Experimental.
+  Default: false
+
 
 ## Plugin
 
@@ -150,3 +155,24 @@ Stuff to do after putting all hosts in test mode.
 * self.repo_checks(dryrun, repo)
 
 Additional checks you want to do on the repo as sanity checks.
+
+**Plugin example**
+
+This is an example `/etc/taste-tester-plugin.rb` to add a user-defined string
+to `client-taste-tester.rb` on the remote system:
+```
+Hooks.class_eval do
+  def self.test_remote_client_rb_extra_code(_hostname)
+    %(
+      # This comment gets added to client-taste-tester.rb
+      # This one too.
+      )
+  end
+end
+```
+Be sure to pass this plugin file with `-p` on the command line or set it as 
+`plugin_path` in your `taste-tester-config.rb` file.
+
+## License
+
+See the `LICENSE` file.

data/bin/taste-tester

@@ -28,17 +28,17 @@ require 'taste_tester/commands'
 require 'taste_tester/hooks'
 require 'taste_tester/exceptions'
 
-include TasteTester::Logging
-
-if ENV['USER'] == 'root'
-  logger.warn('You should not be running as root')
-  exit(1)
-end
-
 # Command line parsing and param descriptions
 module TasteTester
+  extend TasteTester::Logging
+
   verify = 'Verify your changes were actually applied as intended!'.red
 
+  if ENV['USER'] == 'root'
+    logger.warn('You should not be running as root')
+    exit(1)
+  end
+
   # Do an initial read of the config file if it's in the default place, so
   # that if people override chef_client_command the help message is correct.
   if File.exists?(File.expand_path(TasteTester::Config.config_file))
@@ -48,7 +48,7 @@ module TasteTester
   end
 
   cmd = TasteTester::Config.chef_client_command
-  description = <<-EOF
+  description = <<-ENDOFDESCRIPTION
 Welcome to taste-tester!
 
 Usage: taste-tester <mode> [<options>]
@@ -74,6 +74,13 @@ MODES:
     point some production server specified by -s to your virtual chef server for
     testing.  If you have you a plugin that uses the hookpoint, it'll may amend
     your commit message to denote the server you tested.
+    Returns:
+      0 Success.
+      1 Failure, e.g. host not reachable or usage error.
+      2 Partial success, mixture of success and failure due to hosts being taste
+        tested by other users.
+      3 All hosts were not taste-tested because they are being taste-tested by
+        other users.
 
   upload
     Sync your local repo to your virtual Chef server (i.e. just the first step
@@ -83,6 +90,13 @@ MODES:
     cookbooks and roles. It also does a fair amount of sanity checking on
     your repo and you may specify --skip-repo-checks to bypass this.
 
+  impact
+    Examine local repo changes to determine which roles are potentially
+    impacted by the changes. Compares the set of modified cookbooks with the
+    dependency lists of each role and reports any role which depends on a
+    modified cookbook. It is recommended to test your changes on at least one
+    server of each role, potentially also on multiple platforms.
+
   keeptesting
     Extend the testing time on server specified by -s by 1 hour unless
     otherwise specified by -t.
@@ -111,7 +125,7 @@ MODES:
     You probably don't want this. It will restart up the chef-zero server on
     your localhost. taste-tester dynamically starts this if it's down, so there
     should be no need to do this manually.
-  EOF
+  ENDOFDESCRIPTION
 
   mode = ARGV.shift unless !ARGV.empty? && ARGV[0].start_with?('-')
 
@@ -241,7 +255,7 @@ MODES:
     opts.on(
       '-t', '--testing-time TIME',
       'How long should the host remain in testing.' +
-      ' Takes a simple relative time string, such as "45m", "4h" or "2d".'
+      ' Takes a simple relative time string, such as "45m", "4h" or "1d".'
     ) do |time|
       m = time.match(/^(\d+)([d|h|m]+)$/)
       if m
@@ -260,7 +274,7 @@ MODES:
 
     opts.on(
       '-r', '--repo DIR',
-      "Custom repo location, current deafult is #{TasteTester::Config.repo}." +
+      "Custom repo location, current default is #{TasteTester::Config.repo}." +
         ' Works on upload and test.'
     ) do |dir|
       options[:repo] = dir
@@ -273,6 +287,16 @@ MODES:
       options[:repo_type] = type
     end
 
+    opts.on(
+      '--clowntown-no-repo', 'This option enables taste-tester to run ' +
+      'without a SCM repo of any sort. This has negative implications like ' +
+      'always doing a full upload. This option is here for weird corner ' +
+      'cases like having to sync out the export of a repo, but is not ' +
+      'generally a good idea or well supported. Use at your own risk.'
+    ) do
+      options[:no_repo] = true
+    end
+
     opts.on(
       '-R', '--roles ROLE[,ROLE]', Array,
       'Specific roles to upload. Intended mostly for debugging,' +
@@ -311,6 +335,12 @@ MODES:
       options[:ssh_command] = c
     end
 
+    opts.on(
+      '--ssh-connect-timeout TIMEOUT', 'SSH \'-o ConnectTimeout\' value'
+    ) do |c|
+      options[:ssh_connect_timeout] = c
+    end
+
     opts.on('--skip-repo-checks', 'Skip repository sanity checks') do
       options[:skip_repo_checks] = true
     end
@@ -319,6 +349,13 @@ MODES:
       options[:yes] = true
     end
 
+    opts.on(
+      '--json', 'Format output as JSON for programatic consumption.' +
+      ' Default to false. Works on impact.'
+    ) do
+      options[:json] = true
+    end
+
     opts.separator ''
     opts.separator 'Control local hook behavior with these options:'
 
@@ -397,6 +434,8 @@ MODES:
       TasteTester::Commands.runchef
     when :upload
       TasteTester::Commands.upload
+    when :impact
+      TasteTester::Commands.impact
     else
       logger.error("Invalid mode: #{mode}")
       puts parser

data/lib/taste_tester/client.rb

@@ -14,6 +14,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+require 'minitar'
+require 'find'
 require 'taste_tester/logging'
 require 'between_meals/repo'
 require 'between_meals/knife'
@@ -44,16 +46,22 @@ module TasteTester
         :databag_dir => TasteTester::Config.databags,
         :checksum_dir => TasteTester::Config.checksum_dir,
         :role_type => TasteTester::Config.role_type,
+        :config => TasteTester::Config.knife_config,
       )
       @knife.write_user_config
-      @repo = BetweenMeals::Repo.get(
-        TasteTester::Config.repo_type,
-        TasteTester::Config.repo,
-        logger,
-      )
-      unless @repo.exists?
+      if TasteTester::Config.no_repo
+        @repo = nil
+      else
+        @repo = BetweenMeals::Repo.get(
+          TasteTester::Config.repo_type,
+          TasteTester::Config.repo,
+          logger,
+        )
+      end
+      if @repo && !@repo.exists?
         fail "Could not open repo from #{TasteTester::Config.repo}"
       end
+
       @track_symlinks = TasteTester::Config.track_symlinks
     end
 
@@ -64,26 +72,30 @@ module TasteTester
     end
 
     def upload
-      checks unless @skip_checks
-
-      logger.info("Last commit: #{@repo.head_rev} " +
-        "'#{@repo.last_msg.split("\n").first}'" +
-        " by #{@repo.last_author[:email]}")
+      head_rev = nil
+      if @repo
+        head_rev = @repo.head_rev
+        checks unless @skip_checks
+        logger.info("Last commit: #{head_rev} " +
+          "'#{@repo.last_msg.split("\n").first}'" +
+          " by #{@repo.last_author[:email]}")
+      end
 
-      if @force || !@server.latest_uploaded_ref
+      if @force || !@server.latest_uploaded_ref || !@repo
         logger.info('Full upload forced') if @force
+        logger.info('No repo, doing full upload') unless @repo
         unless TasteTester::Config.skip_pre_upload_hook
           TasteTester::Hooks.pre_upload(TasteTester::Config.dryrun,
                                         @repo,
                                         nil,
-                                        @repo.head_rev)
+                                        head_rev)
         end
         time(logger) { full }
         unless TasteTester::Config.skip_post_upload_hook
           TasteTester::Hooks.post_upload(TasteTester::Config.dryrun,
                                          @repo,
                                          nil,
-                                         @repo.head_rev)
+                                         head_rev)
         end
       else
         # Since we also upload the index, we always need to run the
@@ -93,7 +105,7 @@ module TasteTester
           TasteTester::Hooks.pre_upload(TasteTester::Config.dryrun,
                                         @repo,
                                         @server.latest_uploaded_ref,
-                                        @repo.head_rev)
+                                        head_rev)
         end
         begin
           time(logger) { partial }
@@ -105,16 +117,99 @@ module TasteTester
           TasteTester::Hooks.post_upload(TasteTester::Config.dryrun,
                                          @repo,
                                          @server.latest_uploaded_ref,
-                                         @repo.head_rev)
+                                         head_rev)
         end
       end
 
-      @server.latest_uploaded_ref = @repo.head_rev
+      @server.latest_uploaded_ref = head_rev
+      @server.last_upload_time = Time.new.strftime('%Y-%m-%d %H:%M:%S')
     end
 
     private
 
+    def populate(stream, writer, path, destination)
+      full_path = File.join(File.join(TasteTester::Config.repo, path))
+      return unless File.directory?(full_path)
+      # everything is relative to the repo dir. chdir makes handling all the
+      # paths within this simpler
+      Dir.chdir(full_path) do
+        Find.find('.') do |p|
+          # ignore current directory. The File.directory? would also skip it,
+          # but we need to do it early because the string is too short for the
+          # next statement.
+          next if p == '.'
+          # paths are enumerated as relative to the input path '.', so we get
+          # './dir/file'. Stripping off the first two characters gives us a
+          # a cleaner 'dir/file' path.
+          name = File.join(destination, p[2..-1])
+          if File.directory?(p)
+            # skip it. This also handles symlinks to directories which aren't
+            # useful either.
+          elsif File.symlink?(p)
+            # tar handling of filenames > 100 characters gets complex. We'd use
+            # split_name from Minitar, but it's a private method. It's
+            # reasonable to assume that all symlink names in the bundle are
+            # less than 100 characters long. Long term, the version of minitar
+            # in chefdk should be upgraded.
+            fail 'Add support for long symlink paths' if name.size > 100
+            # The version of Minitar included in chefdk does not support
+            # symlinks directly. Therefore we use direct writes to the
+            # underlying stream to reproduce the symlinks
+            symlink = {
+              :name => name,
+              :mode => 0644,
+              :typeflag => '2',
+              :size => 0,
+              :linkname => File.readlink(p),
+              :prefix => '',
+            }
+            stream.write(Minitar::PosixHeader.new(symlink))
+          else
+            File.open(p, 'rb') do |r|
+              writer.add_file_simple(
+                name, :mode => 0644, :size => File.size(r)
+              ) do |d, _opts|
+                IO.copy_stream(r, d)
+              end
+            end
+          end
+        end
+      end
+    end
+
+    def bundle_upload
+      dest = File.join(@server.bundle_dir, 'tt.tgz')
+      begin
+        Tempfile.create(['tt', '.tgz'], @server.bundle_dir) do |tempfile|
+          stream = Zlib::GzipWriter.new(tempfile)
+          Minitar::Writer.open(stream) do |writer|
+            TasteTester::Config.relative_cookbook_dirs.each do |cb_dir|
+              populate(stream, writer, cb_dir, 'cookbooks')
+            end
+            populate(
+              stream, writer, TasteTester::Config.relative_role_dir, 'roles'
+            )
+            populate(
+              stream, writer, TasteTester::Config.relative_databag_dir,
+              'databag'
+            )
+          end
+          stream.close
+          File.rename(tempfile.path, dest)
+        end
+      rescue Errno::ENOENT
+        # Normally the temporary file is renamed to the dest name. If this
+        # happens, then the cleanup of of the temporary file doesn't work,
+        # but this is fine and expected.
+        nil
+      end
+    end
+
     def full
+      if TasteTester::Config.bundle
+        bundle_upload
+        return
+      end
       logger.warn('Doing full upload')
       @knife.cookbook_upload_all
       @knife.role_upload_all
@@ -122,6 +217,11 @@ module TasteTester
     end
 
     def partial
+      if TasteTester::Config.bundle
+        logger.info('No partial support for bundle mode, doing full upload')
+        bundle_upload
+        return
+      end
       logger.info('Doing differential upload from ' +
                    @server.latest_uploaded_ref)
       changeset = BetweenMeals::Changeset.new(