pocketknife 0.1.0 → 0.2.0

data/.travis.yml

@@ -0,0 +1,5 @@
+language: ruby
+rvm:
+  - 1.8.7
+  - 1.9.2
+  - 1.9.3

data/CHANGES.md

@@ -0,0 +1,14 @@
+Changes
+=======
+
+* 0.2.0
+    * [!] Changed default transfer mechanism to `rsync` from `tar` in past versions.
+    * Added transfer mechanisms, allowing the choice of `rsync` and `tar` for uploading files to nodes. The new `rsync` option deals gracefully with symlinks in your sources and is faster for making small changes. See "Transfer mechanisms" section in the README for details. Suggested by Roland Moriz.
+    * Added data bags support, contributed by Richard Livsey.
+    * Added ability to override the runlist. See "Override runlist" in the README for details.
+    * Added shellwords to properly escape strings in commands.
+    * Added clear success message and timer, suggested by Trip Leonard.
+    * Added clearer error message when `nodes` directory is missing.
+
+* 0.1.0
+    * First release

data/Gemfile

@@ -1,13 +1,28 @@
-source "http://rubygems.org"
+source 'http://rubygems.org'
 
-gem "archive-tar-minitar", "~> 0.5.0"
-gem "rye", "~> 0.9.0"
+gem 'archive-tar-minitar', '~> 0.5.0'
+gem 'rye', '~> 0.9.0'
 
 group :development do
-  gem "bluecloth", "~> 2.1.0"
-  gem "rspec", "~> 2.3.0"
-  gem "yard", "~> 0.6.0"
-  gem "bundler", "~> 1.0.0"
-  gem "jeweler", "~> 1.6.0"
-  gem "rcov", ">= 0"
+  gem 'rake'
+
+  gem 'bluecloth', '~> 2.2.0'
+  gem 'rspec', '~> 2.10.0'
+  gem 'yard', '~> 0.8.0'
+  gem 'jeweler', '~> 1.8.0'
+
+  # OPTIONAL LIBRARIES: These libraries upset travis-ci and may cause Ruby or
+  # RVM to hang, so only use them when needed.
+  if ENV['DEBUGGER']
+    platform :mri_18 do
+      gem 'rcov', :require => false
+      gem 'ruby-debug'
+    end
+
+    platform :mri_19 do
+      gem 'simplecov', :require => false
+      gem 'debugger-ruby_core_source'
+      gem 'debugger'
+    end
+  end
 end

data/README.md

@@ -1,3 +1,5 @@
+[![Build Status](https://secure.travis-ci.org/igal/pocketknife.png)](http://travis-ci.org/igal/pocketknife)
+
 pocketknife
 ===========
 
@@ -38,24 +40,6 @@ Go into your new *project* directory:
 
 Create cookbooks in the `cookbooks` directory that describe how your computers should be configured. These are standard `chef` cookbooks, like the [opscode/cookbooks](https://github.com/opscode/cookbooks). For example, download a copy of [opscode/cookbooks/ntp](https://github.com/opscode/cookbooks/tree/master/ntp) as `cookbooks/ntp`.
 
-Override cookbooks in the `site-cookbooks` directory. This has the same structure as `cookbooks`, but any files you put here will override the contents of `cookbooks`. This is useful for storing the original code of a third-party cookbook in `cookbooks` and putting your customizations in `site-cookbooks`.
-
-Optionally define roles in the `roles` directory that describe common behavior and attributes of your computers using JSON syntax using [chef's documentation](http://wiki.opscode.com/display/chef/Roles#Roles-AsJSON). For example, define a role called `ntp_client` by creating a file called `roles/ntp_client.json` with this content:
-
-    {
-      "name": "ntp_client",
-      "chef_type": "role",
-      "json_class": "Chef::Role",
-      "run_list": [
-        "recipe[ntp]"
-      ],
-      "override_attributes": {
-        "ntp": {
-          "servers": ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org", "3.pool.ntp.org"]
-        }
-      }
-    }
-
 Define a new node using the `chef` JSON syntax for [runlist](http://wiki.opscode.com/display/chef/Setting+the+run_list+in+JSON+during+run+time) and [attributes](http://wiki.opscode.com/display/chef/Attributes). For example, to define a node with the hostname `henrietta.swa.gov.it` create the `nodes/henrietta.swa.gov.it.json` file, and add the contents below so it uses the `ntp_client` role and overrides its attributes to use a local NTP server:
 
     {
@@ -79,6 +63,52 @@ When deploying a configuration to a node, `pocketknife` will check whether Chef
 
 To always install Chef and its dependencies when they're needed, without prompts, use the `-i` option, e.g. `pocketknife -i henrietta`. Or to never install Chef and its dependencies, use the `-I` option, which will cause the program to quit with an error rather than prompting if Chef or its dependencies aren't installed.
 
+Override runlist
+----------------
+
+Specify the runlist by using the `-r` option, which will override the one specified in the node, e.g.:
+
+    pocketknife -r mycookbook henrietta
+
+Transfer mechanisms
+-------------------
+
+Files can be uploaded to nodes using different transfer mechanisms:
+
+* `tar` - Uses one connection for execution of commands and uploads, and sends a tarball that's then extracted. Pros: Pure Ruby, reuses connection. Cons: Can't cope with symlinks, inefficient for sending a small change.
+* `rsync` - Uses one connection for execution of commands and then runs the `rsync` command to upload files. Pros: Handles symlinks, and is efficient for sending a small change. Cons: Requires `rsync` command, rather than being pure Ruby, and doesn't reuse the connection.
+
+You can specify the transfer mechanism with the `-t` option and the name of the mechanism, e.g.:
+
+    pocketknife -t rsync henrietta
+
+Override cookbooks
+------------------
+
+Override cookbooks in the `site-cookbooks` directory. This has the same structure as `cookbooks`, but any files you put here will override the contents of `cookbooks`. This is useful for storing the original code of a third-party cookbook in `cookbooks` and putting your customizations in `site-cookbooks`.
+
+Roles
+-----
+
+Optionally define roles in the `roles` directory that describe common behavior and attributes of your computers using JSON syntax using [chef's documentation](http://wiki.opscode.com/display/chef/Roles#Roles-AsJSON). For example, define a role called `ntp_client` by creating a file called `roles/ntp_client.json` with this content:
+
+    {
+      "name": "ntp_client",
+      "chef_type": "role",
+      "json_class": "Chef::Role",
+      "run_list": [
+        "recipe[ntp]"
+      ],
+      "override_attributes": {
+        "ntp": {
+          "servers": ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org", "3.pool.ntp.org"]
+        }
+      }
+    }
+
+Debugging
+---------
+
 If something goes wrong while deploying the configuration, you can display verbose logging from `pocketknife` and Chef by using the `-v` option. For example, deploy the configuration to `henrietta` with verbose logging:
 
     pocketknife -v henrietta

data/Rakefile

@@ -1,5 +1,7 @@
 # encoding: utf-8
 
+task :default => :spec
+
 require 'rubygems'
 require 'bundler'
 begin
@@ -39,6 +41,7 @@ With pocketknife, all of your cookbooks, roles and nodes are stored in easy-to-u
     Gemfile
     LICENSE.txt
     README.md
+    CHANGES.md
     Rakefile
     lib/*
     spec/*
@@ -47,18 +50,55 @@ With pocketknife, all of your cookbooks, roles and nodes are stored in easy-to-u
 end
 Jeweler::RubygemsDotOrgTasks.new
 
+desc "Run coverage report using simplecov."
+task :simplecov do
+  ENV['SIMPLECOV'] = 'true'
+  Rake::Task['spec'].invoke
+end
+
 require 'rspec/core'
 require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec) do |spec|
   spec.pattern = FileList['spec/**/*_spec.rb']
 end
 
-RSpec::Core::RakeTask.new(:rcov) do |spec|
-  spec.pattern = 'spec/**/*_spec.rb'
-  spec.rcov = true
+def rcov(options=[])
+  # None of the official ways to invoke Rcov work right now. Sigh.
+  cmd = "rcov --exclude osx\/objc,gems\/,spec\/,features\/,lib/shellwords.rb,lib/pocketknife/version.rb #{[options].flatten} $(which rspec) spec/*_spec.rb 2>&1"
+  puts cmd
+  output = `#{cmd}`
+  puts output
+  return output
 end
 
-task :default => :spec
+RCOV_DATA = 'coverage/rcov.data'
+RCOV_LOG = 'coverage/rcov.txt'
+
+namespace :rcov do
+  desc "Save rcov information for use with rcov:diff"
+  task :save do
+    rcov "--save=#{RCOV_DATA}"
+  end
+
+  desc "Generate report of what code changed since last rcov:save"
+  task :diff do
+    output = rcov "--no-color --text-coverage-diff=#{RCOV_DATA}"
+    File.open(RCOV_LOG, 'w+') do |h|
+      h.write output
+    end
+    puts "\nSaved coverage report to: #{RCOV_LOG}"
+  end
+end
+
+desc  "Run all specs with rcov"
+task :rcov do
+  rcov
+end
 
 require 'yard'
 YARD::Rake::YardocTask.new
+
+desc "List undocumented code"
+task :undoc do
+  system "yardoc --list-undoc | grep -v 'Unrecognized/invalid option'"
+end

data/lib/pocketknife.rb

@@ -2,6 +2,20 @@
 require "pathname"
 require "fileutils"
 
+begin
+  require "shellwords"
+rescue LoadError
+  require "#{File.dirname(__FILE__)}/shellwords"
+end
+
+# @!visibility private
+class Pathname
+  # @!visibility private
+  def shellescape
+    self.to_s.shellescape
+  end
+end
+
 # Gem libraries
 require "archive/tar/minitar"
 require "rye"
@@ -43,8 +57,11 @@ class Pocketknife
   #   Pocketknife.cli('-h')
   #
   # @param [Array<String>] args A list of arguments from the command-line, which may include options (e.g. <tt>-h</tt>).
+  # @return [void]
+  # @raise [SystemExit] Something catastrophic happened, e.g. user passed invalid options to interpreter.
   def self.cli(args)
     pocketknife = Pocketknife.new
+    timer = Time.now
 
     OptionParser.new do |parser|
       parser.banner = <<-HERE
@@ -96,9 +113,18 @@ OPTIONS:
         pocketknife.can_install = false
       end
 
+      parser.on("-r", "--runlist RUNLIST", "Override runlist with a comma-separated list of recipes and roles") do |v|
+        pocketknife.runlist = v
+      end
+
+      transfer_mechanisms = %w[rsync tar]
+      parser.on("-t", "--transfer MECHANISM", transfer_mechanisms, "Specify transfer mechanism (#{transfer_mechanisms.join(', ')})") do |v|
+        pocketknife.transfer_mechanism = v.to_sym
+      end
+
       begin
         arguments = parser.parse!
-      rescue OptionParser::MissingArgument => e
+      rescue OptionParser::ParseError => e
         puts parser
         puts
         puts "ERROR: #{e}"
@@ -126,44 +152,62 @@ OPTIONS:
         if not options[:upload] and not options[:apply]
           pocketknife.deploy(nodes)
         end
+
+        pocketknife.say("* SUCCESS! Took #{"%0.2f" % [Time.now-timer]} seconds")
       rescue NodeError => e
         puts "! #{e.node}: #{e}"
         exit -1
+      rescue Errno::ENOENT => e
+        puts "! #{e.message}"
+        exit -1
       end
     end
   end
 
   # Returns the software's version.
   #
-  # @return [String] A version string.
+  # @return [String] A version string, e.g. <tt>0.0.1</tt>.
   def self.version
-    return "0.0.1"
+    return Pocketknife::Version::STRING
   end
 
-  # Amount of detail to display? true means verbose, nil means normal, false means quiet.
+  # Amount of detail to display.
+  #
+  # @return [Nil, Boolean] +true+ means verbose, +nil+ means normal, +false+ means quiet.
   attr_accessor :verbosity
 
-  # Can chef and its dependencies be installed automatically if not found? true means perform installation without prompting, false means quit if chef isn't available, and nil means prompt the user for input.
+  # Should Chef and its dependencies be installed automatically if not found on a node?
+  #
+  # @return [Nil, Boolean] +true+ means perform the installation without prompting, +false+ means quit if Chef isn't found, and +nil+ means prompt the user to decide this interactively.
   attr_accessor :can_install
 
-  # {Pocketknife::NodeManager} instance.
+  # @return [Pocketknife::NodeManager] This instance's node manager.
   attr_accessor :node_manager
 
-  # Instantiate a new Pocketknife.
+  # @return [Nil, String] Override runlist with a comma-separated list of recipes and roles.
+  attr_accessor :runlist
+
+  # @return [Symbol] Use :rsync or :tar to transfer files.
+  attr_accessor :transfer_mechanism
+
+  # Instantiates a new Pocketknife.
   #
-  # @option [Boolean] verbosity Amount of detail to display. +true+ means verbose, +nil+ means normal, +false+ means quiet.
-  # @option [Boolean] install Install Chef and its dependencies if needed? +true+ means do so automatically, +false+ means don't, and +nil+ means display a prompt to ask the user what to do.
+  # @option [Nil, Boolean] verbosity Amount of detail to display. +true+ means verbose, +nil+ means normal, +false+ means quiet.
+  # @option [Nil, Boolean] install Install Chef and its dependencies if needed? +true+ means do so automatically, +false+ means don't, and +nil+ means display a prompt to ask the user what to do.
   def initialize(opts={})
     self.verbosity   = opts[:verbosity]
     self.can_install = opts[:install]
+    self.runlist     = opts[:runlist]
+    self.transfer_mechanism = opts[:transfer_mechanism] || :rsync
 
     self.node_manager = NodeManager.new(self)
   end
 
-  # Display a message, but only if it's important enough
+  # Displays a message, but only if it's important enough.
   #
   # @param [String] message The message to display.
-  # @param [Boolean] importance How important is this? +true+ means important, +nil+ means normal, +false+ means unimportant.
+  # @param [Nil, Boolean] importance How important is this? +true+ means important, +nil+ means normal, +false+ means unimportant.
+  # @return [void]
   def say(message, importance=nil)
     display = \
       case self.verbosity
@@ -183,8 +227,9 @@ OPTIONS:
   # Creates a new project directory.
   #
   # @param [String] project The name of the project directory to create.
-  # @yield [path] Yields status information to the optionally supplied block.
+  # @yield status information to the optionally supplied block.
   # @yieldparam [String] path The path of the file or directory created.
+  # @return [void]
   def create(project)
     self.say("* Creating project in directory: #{project}")
 
@@ -206,16 +251,18 @@ OPTIONS:
     return true
   end
 
-  # Returns a Node instance.
+  # Returns a node.
   #
-  # @param[String] name The name of the node.
+  # @param [String] name The name of the node.
+  # @return [Pocketknife::Node]
   def node(name)
     return node_manager.find(name)
   end
 
   # Deploys configuration to the nodes, calls {#upload} and {#apply}.
   #
-  # @params[Array<String>] nodes A list of node names.
+  # @param [Array<String>] nodes A list of node names.
+  # @return [void]
   def deploy(nodes)
     node_manager.assert_known(nodes)
 
@@ -229,6 +276,7 @@ OPTIONS:
   # Uploads configuration information to remote nodes.
   #
   # @param [Array<String>] nodes A list of node names.
+  # @return [void]
   def upload(nodes)
     node_manager.assert_known(nodes)
 
@@ -242,6 +290,7 @@ OPTIONS:
   # Applies configurations to remote nodes.
   #
   # @param [Array<String>] nodes A list of node names.
+  # @return [void]
   def apply(nodes)
     node_manager.assert_known(nodes)
 

data/lib/pocketknife/errors.rb

@@ -1,85 +1,125 @@
 class Pocketknife
-  # == NodeError
+  # == Error
   #
-  # An error with a {Pocketknife::Node}. This is meant to be subclassed by a more specific error.
-  class NodeError < StandardError
-    # The name of the node.
-    attr_accessor :node
-
-    # Instantiate a new exception.
+  # Superclass of all Pocketknife errors.
+  class Error < StandardError
+    # == InvalidTransferMechanism
     #
-    # @param [String] message The message to display.
-    # @param [String] node The name of the unknown node.
-    def initialize(message, node)
-      self.node = node
-      super(message)
+    # Exception raised when given an invalid transfer mechanism, e.g. not :tar or :rsync.
+    class InvalidTransferMechanism < Error
+      # @return [Symbol] Transfer mechanism that failed.
+      attr_accessor :mechanism
+
+      def initialize(mechanism)
+        super("Invalid transfer mechanism: #{mechanism}")
+      end
     end
-  end
 
-  # == NoSuchNode
-  #
-  # Exception raised when asked to perform an operation on an unknown node.
-  class NoSuchNode < NodeError
-  end
+    # == NodeError
+    #
+    # An error with a {Pocketknife::Node}. This is meant to be subclassed by a more specific error.
+    class NodeError < Error
+      # @return [String] The name of the node.
+      attr_accessor :node
 
-  # == UnsupportedInstallationPlatform
-  #
-  # Exception raised when asked to install Chef on a node with an unsupported platform.
-  class UnsupportedInstallationPlatform < NodeError
-  end
+      # Instantiate a new exception.
+      #
+      # @param [String] message The message to display.
+      # @param [String] node The name of the unknown node.
+      def initialize(message, node)
+        self.node = node
+        super(message)
+      end
 
-  # == NotInstalling
-  #
-  # Exception raised when Chef is not available ohn a node, but user asked not to install it.
-  class NotInstalling < NodeError
-  end
+      # == NoSuchNode
+      #
+      # Exception raised when asked to perform an operation on an unknown node.
+      class NoSuchNode < NodeError
+      end
 
-  # == ExecutionError
-  #
-  # Exception raised when something goes wrong executing commands against remote host.
-  class ExecutionError < NodeError
-    # Command that failed.
-    attr_accessor :command
+      # == UnsupportedInstallationPlatform
+      #
+      # Exception raised when asked to install Chef on a node with an unsupported platform.
+      class UnsupportedInstallationPlatform < NodeError
+      end
 
-    # Cause of exception, a {Rye:Err}.
-    attr_accessor :cause
+      # == NotInstalling
+      #
+      # Exception raised when Chef is not available ohn a node, but user asked not to install it.
+      class NotInstalling < NodeError
+      end
 
-    # Was execution's output shown immediately? If so, don't include output in message.
-    attr_accessor :immediate
+      # == RsyncError
+      #
+      # Exception raised if rsync command failed.
+      class RsyncError < NodeError
+        # @return [String] Command that failed.
+        attr_accessor :command
 
-    # Instantiates a new exception.
-    #
-    # @param [String] node The name of the unknown node.
-    # @param [String] command The command that failed.
-    # @param [Rye::Err] cause The actual exception thrown.
-    # @param [Boolean] immediate Was execution's output shown immediately? If so, don't include output in message.
-    def initialize(node, command, cause, immediate)
-      self.command = command
-      self.cause = cause
-      self.immediate = immediate
-
-      message = <<-HERE.chomp
+        def initialize(command, node)
+          super("Failed while rsyncing: #{command}", node)
+        end
+      end
+
+      # == ExecutionError
+      #
+      # Exception raised when something goes wrong executing commands against remote host.
+      class ExecutionError < NodeError
+        # @return [String] Command that failed.
+        attr_accessor :command
+
+        # @return [Rye::Err] Cause of exception, a Rye:Err.
+        attr_accessor :cause
+
+        # @return [Boolean] Was execution's output shown immediately? If so, don't include output in message.
+        attr_accessor :immediate
+
+        # Instantiates a new exception.
+        #
+        # @param [String] node The name of the unknown node.
+        # @param [String] command The command that failed.
+        # @param [Rye::Err] cause The actual exception thrown.
+        # @param [Boolean] immediate Was execution's output shown immediately? If so, don't include output in message.
+        def initialize(node, command, cause, immediate)
+          self.command = command
+          self.cause = cause
+          self.immediate = immediate
+
+          message = <<-HERE.chomp
 Failed while executing commands on node '#{node}'
 - COMMAND: #{command}
 - EXIT STATUS: #{cause.exit_status}
-      HERE
+          HERE
 
-      unless immediate
-        message << <<-HERE.chomp
+          unless immediate
+            message << <<-HERE.chomp
 
 - STDOUT: #{cause.stdout.to_s.strip}
 - STDERR: #{cause.stderr.to_s.strip}
-        HERE
+            HERE
+          end
+
+          super(message, node)
+        end
+
+        # Returns exit status.
+        #
+        # @return [Integer] Exit status from execution.
+        def exit_status
+          return self.cause.exit_status
+        end
+
       end
 
-      super(message, node)
     end
 
-    # Returns exit status.
-    #
-    # @return [Integer] Exit status from execution.
-    def exit_status
-      return self.cause.exit_status
-    end
   end
+
+  InvalidTransferMechanism = Pocketknife::Error::InvalidTransferMechanism
+  NodeError = Pocketknife::Error::NodeError
+  NoSuchNode = Pocketknife::Error::NodeError::NoSuchNode
+  UnsupportedInstallationPlatform = Pocketknife::Error::NodeError::UnsupportedInstallationPlatform
+  NotInstalling = Pocketknife::Error::NodeError::NotInstalling
+  RsyncError = Pocketknife::Error::NodeError::RsyncError
+  ExecutionError = Pocketknife::Error::NodeError::ExecutionError
 end