spiceweasel 0.8.2 → 0.9.0

data/README.md

@@ -8,16 +8,11 @@ The [https://github.com/mattray/ravel-repo](https://github.com/mattray/ravel-rep
 
 The [CHANGELOG.md](https://github.com/mattray/spiceweasel/blob/master/CHANGELOG.md) covers current, previous and future development milestones and contains the features backlog.
 
-
 Requirements
 ============
 Spiceweasel currently depends on `knife` to run commands for it. Infrastructure files must either end in .json or .yml to be processed.
 
-Written initially with Chef 0.9.12 (should still work) and continuing to develop with support for 0.10.
-
-Testing
--------
-Tested with Ubuntu 10.04 and Chef 0.9.16 and 0.10.4.
+Written and tested initially with Chef 0.9.12 (should still work) and continuing development with the 0.10 series.
 
 File Syntax
 ===========
@@ -153,7 +148,7 @@ From the `example.json`:
 
 Cookbooks
 ---------
-The `cookbooks` section of the JSON or YAML file currently supports `knife cookbook upload FOO` where `FOO` is the name of the cookbook in the `cookbooks` directory. If a version is passed, it is validated against an existing cookbook `metadata.rb` (it must match the `metadata.rb` string exactly) and if none is found, the missing cookbook is downloaded (without touching version control) and the command to untar it is provided. You may pass any additional arguments if necessary. The YAML snippet
+The `cookbooks` section of the JSON or YAML file currently supports `knife cookbook upload FOO` where `FOO` is the name of the cookbook in the `cookbooks` directory. The default behavior is to download the cookbook as a tarball, untar it and remove the tarball. The `--siteinstall` option will allow for use of `knife cookbook site install` with the cookbook and the creation of a vendor branch if git is the underlying version control. If a version is passed, it is validated against the existing cookbook `metadata.rb` and it must match the `metadata.rb` string exactly. You may pass any additional arguments if necessary. The YAML snippet
 
 ``` yaml
 cookbooks:
@@ -169,6 +164,7 @@ produces the knife commands
 knife cookbook upload apache2
 knife cookbook site download apt 1.1.0 --file cookbooks/apt.tgz
 tar -C cookbooks/ -xf cookbooks/apt.tgz
+rm -f cookbooks/apt.tgz
 knife cookbook upload apt
 knife cookbook upload mysql
 ```
@@ -294,22 +290,38 @@ spiceweasel path/to/infrastructure.yml
 
 This will generate the knife commands to build the described infrastructure. Infrastructure files must end in either `.json` or `.yml`.
 
---dryrun
---------
-This is the default action, printing the knife commands to be run without executing them.
+-c/--knifeconfig
+----------------
+Specify a knife.rb configuration file to use with the knife commands.
+
+--debug
+-------
+This provides verbose debugging messages.
 
 -d/--delete
 -----------
-The delete command will generate the knife commands to delete the infrastructure described in the file. This includes each cookbook, role, data bag, environment and node listed. Currently all nodes from the system are deleted with `knife node bulk_delete`, specific-node support will be added in a future release.
+The delete command will generate the knife commands to delete the infrastructure described in the file. This includes each cookbook, role, data bag, environment and node listed. Node deletion will specify individual nodes, attempt to pass the list of nodes to the cloud provider for deletion, and finish with `knife node bulk delete`. If you are mixing individual nodes with cloud provider nodes it is possible that nodes may be missed from cloud provider deletion and you should double-check (ie. `knife ec2 server list`).
+
+--dryrun
+--------
+This is the default action, printing the knife commands to be run without executing them.
 
 -h/--help
 ---------
 Print the currently-supported usage options for spiceweasel.
 
+--parallel
+----------
+Use the GNU 'parallel' command to execute 'knife VENDOR server create' commands that may be parallelized.
+
 -r/--rebuild
 ---------
 The rebuild command will generate the knife commands to delete and recreate the infrastructure described in the file. This includes each cookbook, role, data bag, environment and node listed. Currently all nodes from the system are deleted with `knife node bulk_delete`, specific-node support will be added in a future release.
 
+--siteinstall
+-------------
+Use the 'install' command with 'knife cookbook site' instead of the default 'download'.
+
 -v/--version
 ------------
 Print the version of spiceweasel currently installed.

data/bin/spiceweasel

@@ -13,6 +13,8 @@ begin
   cli = Spiceweasel::CLI.new
   cli.parse_options
   DEBUG = cli.config[:debug]
+  PARALLEL = cli.config[:parallel]
+  SITEINSTALL = cli.config[:siteinstall]
 rescue OptionParser::InvalidOption => e
   STDERR.puts e.message
   puts cli.opt_parser.to_s

data/lib/spiceweasel/cli.rb

@@ -8,7 +8,7 @@ class Spiceweasel::CLI
 
   option :debug,
   :long => "--debug",
-  :description => "Verbose debugging messages.",
+  :description => "Verbose debugging messages",
   :boolean => true
 
   option :delete,
@@ -31,12 +31,27 @@ class Spiceweasel::CLI
   :show_options => true,
   :exit => 0
 
+  option :knifeconfig,
+  :short => "-c CONFIG",
+  :long => "--knifeconfig CONFIG",
+  :description => "Specify the knife.rb configuration file to use"
+
+  option :parallel,
+  :long => "--parallel",
+  :description => "Use the GNU 'parallel' command to parallelize 'knife VENDOR server create' commands that are not order-dependent",
+  :boolean => true
+
   option :rebuild,
   :short => "-r",
   :long => "--rebuild",
   :description => "Print the knife commands to be delete and recreate the infrastructure",
   :boolean => true
 
+  option :siteinstall,
+  :long => "--siteinstall",
+  :description => "Use the 'install' command with 'knife cookbook site' instead of the default 'download'",
+  :boolean => true
+
   option :version,
   :short => "-v",
   :long => "--version",
@@ -45,9 +60,4 @@ class Spiceweasel::CLI
   :proc => lambda {|v| puts "Spiceweasel: #{Spiceweasel::VERSION}" },
   :exit => 0
 
-  option :knifeconfig,
-  :short => "-c CONFIG",
-  :long => "--knifeconfig CONFIG",
-  :description => "The knife.rb configuration file to use."
-
 end

data/lib/spiceweasel/cookbook_list.rb

@@ -20,8 +20,13 @@ class Spiceweasel::CookbookList
               exit(-1)
             end
           elsif !File.directory?("cookbooks/#{cb}")
-            @create += "knife cookbook#{options['knife_options']} site download #{cb} #{version} --file cookbooks/#{cb}.tgz #{args}\n"
-            @create += "tar -C cookbooks/ -xf cookbooks/#{cb}.tgz\n"
+            if SITEINSTALL #use knife cookbook site install
+              @create += "knife cookbook#{options['knife_options']} site install #{cb} #{version} #{args}\n"
+            else #use knife cookbook site download, untar and then remove the tarball
+              @create += "knife cookbook#{options['knife_options']} site download #{cb} #{version} --file cookbooks/#{cb}.tgz #{args}\n"
+              @create += "tar -C cookbooks/ -xf cookbooks/#{cb}.tgz\n"
+              @create += "rm -f cookbooks/#{cb}.tgz\n"
+            end
           end
         else
           STDERR.puts "cookbooks directory not found, validation and downloading skipped"

data/lib/spiceweasel/node_list.rb

@@ -3,7 +3,6 @@ class Spiceweasel::NodeList
     nodes ||= []
     @create = @delete = ''
 
-    @delete += "knife node#{options['knife_options']} bulk_delete .* -y\n"
     if nodes
       nodes.each do |node|
         STDOUT.puts "DEBUG: node: #{node.keys[0]}" if DEBUG
@@ -11,29 +10,39 @@ class Spiceweasel::NodeList
         STDOUT.puts "DEBUG: node run_list: #{run_list}" if DEBUG
         Spiceweasel::RunList.new(run_list).validate(cookbook_list, environment_list, role_list)
         #provider support
-        if node.keys[0].start_with?("bluebox","ec2","openstack","rackspace","slicehost","terremark")
+        if node.keys[0].start_with?("bluebox","ec2","gandi","openstack","rackspace","slicehost","terremark","voxel")
           provider = node.keys[0].split()
           count = 1
           if (provider.length == 2)
             count = provider[1]
           end
-          #create the instances
-          count.to_i.times do
+          if PARALLEL
+            @create += "seq #{count} | parallel -j 0 -v \""
             @create += "knife #{provider[0]}#{options['knife_options']} server create #{node[node.keys[0]][1]}"
             if run_list.length > 0
-              @create += " -r '#{node[node.keys[0]][0].gsub(/ /,',')}'\n"
+              @create += " -r '#{node[node.keys[0]][0].gsub(/ /,',')}'\"\n"
+            end
+          else
+            count.to_i.times do
+              @create += "knife #{provider[0]}#{options['knife_options']} server create #{node[node.keys[0]][1]}"
+              if run_list.length > 0
+                @create += " -r '#{node[node.keys[0]][0].gsub(/ /,',')}'\n"
+              end
             end
           end
-        else #multinode support
+          @delete += "knife node#{options['knife_options']} list | xargs knife #{provider[0]} server delete -y\n"
+        else #node bootstrap support
           node.keys[0].split.each do |server|
             @create += "knife bootstrap#{options['knife_options']} #{server} #{node[node.keys[0]][1]}"
             if run_list.length > 0
               @create += " -r '#{node[node.keys[0]][0].gsub(/ /,',')}'\n"
             end
+            @delete += "knife node#{options['knife_options']} delete #{server} -y\n"
           end
         end
       end
     end
+    @delete += "knife node#{options['knife_options']} bulk delete .* -y\n"
   end
 
   attr_reader :create, :delete

data/lib/spiceweasel/version.rb

@@ -1,3 +1,3 @@
 module Spiceweasel
-  VERSION = "0.8.2"
+  VERSION = "0.9.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: spiceweasel
 version: !ruby/object:Gem::Version
-  version: 0.8.2
+  version: 0.9.0
   prerelease: 
 platform: ruby
 authors:
@@ -10,12 +10,12 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-10-28 00:00:00.000000000 -05:00
+date: 2011-11-17 00:00:00.000000000 -06:00
 default_executable: 
 dependencies:
 - !ruby/object:Gem::Dependency
   name: chef
-  requirement: &2160839960 !ruby/object:Gem::Requirement
+  requirement: &2151797000 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -23,10 +23,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160839960
+  version_requirements: *2151797000
 - !ruby/object:Gem::Dependency
   name: json
-  requirement: &2160839480 !ruby/object:Gem::Requirement
+  requirement: &2151795080 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -34,10 +34,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160839480
+  version_requirements: *2151795080
 - !ruby/object:Gem::Dependency
   name: mixlib-cli
-  requirement: &2160839000 !ruby/object:Gem::Requirement
+  requirement: &2151793020 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -45,10 +45,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160839000
+  version_requirements: *2151793020
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &2160838480 !ruby/object:Gem::Requirement
+  requirement: &2151791600 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -56,7 +56,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *2160838480
+  version_requirements: *2151791600
 description: This provides a CLI for generating knife commands to build Chef-managed
   infrastructure from a simple YAML file.
 email: