vagabond 0.1.4 → 0.2.0

data/CHANGELOG.md

@@ -1,3 +1,15 @@
+## v0.2.0
+* Migrate to thor
+* Clean up option usage
+* Add support for custom template builds
+* DRY out some reusable stuffs
+* Change Vagabondfile key :boxes to :nodes
+* Start on defined color codes for actions/states (not implemented)
+* Add some validation and error checking
+* Make server stuff work better
+* Add integrated support for test kitchen 1.0
+* Add new 'cluster' support for test kitchen 1.0
+
 ## v0.1.4
 * Fix color option to do the right thing
 

data/README.md

@@ -14,6 +14,10 @@ easily and most importantly, quickly. It uses Linux containers
 instead of full blown VMs which means things are faster. Lots
 faster.
 
+Vagabond is built for Chef. The tooling within Vagabond is targeted
+at Chef. As the initial development push on Vagabond slows, this
+tooling will become optionally swappable (applicable bits at least).
+
 ## Installation
 
 As a rubygem:
@@ -22,16 +26,16 @@ As a rubygem:
 $ gem install vagabond
 ```
 
-## How it is?
+## How does it work
 
 Currently, this is built to run within a classic Chef repository.
-It requires a Vagabond file, that simply outputs a Hash. The file
+It requires a `Vagabond` file, that simply outputs a Hash. The file
 is Ruby though, so you can do lots of crazy stuff to build the
 Hash you return. Heres a simple example:
 
 ```ruby
 {
-  :boxes => {
+  :nodes => {
     :precise => {
       :template => 'ubuntu_1204',
       :run_list => %w(role[base])
@@ -48,6 +52,12 @@ Hash you return. Heres a simple example:
 }
 ```
 
+Now, to create a node, simply run:
+
+```
+$ vagabond up db
+```
+
 Pretty simple, right?
 
 ### Templates available
@@ -64,49 +74,8 @@ Currently builtin templates:
 
 ## Commands
 
-Lots of commands. What to see them all? Just ask:
-
-```
-$ vagabond --help
-Nodes:
-        vagabond create NODE [options]
-        vagabond destroy NODE [options]
-        vagabond freeze NODE [options]
-        vagabond provision NODE [options]
-        vagabond rebuild NODE [options]
-        vagabond ssh NODE [options]
-        vagabond start NODE [options]
-        vagabond status NODE [options]
-        vagabond thaw NODE [options]
-        vagabond up NODE [options]
-Server:
-        vagabond server auto_upload [options]
-        vagabond server create [options]
-        vagabond server destroy [options]
-        vagabond server freeze [options]
-        vagabond server provision [options]
-        vagabond server rebuild [options]
-        vagabond server ssh [options]
-        vagabond server start [options]
-        vagabond server status [options]
-        vagabond server stop [options]
-        vagabond server thaw [options]
-        vagabond server up [options]
-        vagabond server upload_cookbooks [options]
-        vagabond server upload_databags [options]
-        vagabond server upload_environments [options]
-        vagabond server upload_roles [options]
-Knife:
-        vagabond knife COMMAND [knife_options]
-Options:
-        --color
-        --debug
-        --disable-auto-provision
-        --disable-local-server
-        --disable-configure
-        --force-configure
-    -f, --vagabond-file FILE
-```
+See the `USAGE` file for an overview of available commands and their
+usage.
 
 ## Local chef server?
 
@@ -133,6 +102,35 @@ vagabond knife SOME COOL KNIFE COMMAND
 
 This will just push the command into the local chef server. 
 
+## Test Kitchen
+
+Vagabond provides test kitchen 1.0 support. It will map boxes defined
+within platforms to platform templates available (to the best of its
+ability). No need to spin up vagrant VMs, or use another tool. Point
+vagabond at the cookbook, and let it handle the details.
+
+In the TODO pipeline is allowing platform mapping in the Vagabondfile
+so custom templates (with memory limits for example) can be used
+instead of the base templates.
+
+### Cluster testing
+
+Vagabond adds an extra feature to test kitchen: cluster testing. This
+type of testing uses the local chef server, and provides an extreme
+amount of power to tests. Instead of provisioning a node and simply
+testing it in isolation, cluster testing provides the support to
+provision multiple nodes against the local chef server. Once all
+nodes have been successfully provisioned, vagabond will go back through
+each node and run tests.
+
+Seems simple, right? It is, but it's also extremely powerful. Instead
+of testing things individually and isolated, this allows for real
+integration testing. Tests can be applied to discovery, slaving,
+and all the other fun things nodes may be doing that require a
+chef server. Looking for example? See the `USAGE` file!
+
+Double awesome
+
 ## Important note
 
 Until namespaces hit Linux proper, vagabond `sudo`s its way around. You
@@ -152,8 +150,10 @@ boring old `sudo`, you can do that to:
 
 ## Extra note
 
-This is still very much in alpha testing phase. So if you find bugs, please
-report them!
+This thing is still very new and shiny with lots of sharp edges. They
+are getting sanded down as quickly as possible. If you find bugs, are
+confused about some of the available functionality, or just want to point 
+out some stupidity that I have implemented, please file an issue on github!
 
 ## Contributing
 

data/USAGE.md

@@ -0,0 +1,197 @@
+# Vagabond USAGE
+
+## Requirements
+
+* Chef repository structure
+* Ubuntu >= 12.04
+  * This requirement is soft. It is only here because it is the only place it has currently been tested. Testing on centos, debian, and arch is coming soon.
+
+## Setup
+
+* Install vagabond either in a Gemfile, or via gem:
+
+```
+$ gem install vagabond
+```
+
+Drop a Vagabondfile in the root of your Chef repository. Here is a
+very simple Vagabond file to start with:
+
+```ruby
+{
+  :nodes => {
+    :my_precise_node => {
+      :template => 'ubuntu_1204',
+      :run_list => ['role[base]']
+    }
+  }
+}
+```
+
+This assumes you have a role named base, and that it and its dependencies
+are currently pushed up to your configured chef server. So, we create our
+node:
+
+```
+$ vagabond up my_precise_node
+```
+
+The first time this runs, it will take some time. Vagabond will provision
+your local system (using chef-solo) to install the required LXC tools, and
+to build the templates in use by your Vagabond file. Vagabond will do
+this any time it determines it's required, generally when new templates
+are discovered that does not already have a base container built. So sit
+back, the first build will take a few.
+
+## Custom templates
+
+Custom templates are templates that are based on the builtin templates
+but have some restriction placed on them, for example memory usage. Using
+our current Vagabond file example, lets say we wanted to use a container
+that only had 512MB of memory available to it, and no swap space. We
+would provide the details for a custom template, and set the `my_precise_node`
+to then use that template:
+
+
+```ruby
+{
+  :nodes => {
+    :my_precise_node => {
+      :template => 'custom_1204_512',
+      :run_list => ['role[base]']
+    }
+  },
+  :templates => {
+    :custom_1204_512 => {
+      :base => 'ubuntu_1204',
+      :memory => {
+        :ram => '512M',
+        :swap => 0
+      }
+    }
+  }
+}
+```
+
+## vagabond
+
+The `vagabond` command is used for interaction with nodes. Simply running:
+
+```
+$ vagabond
+```
+
+will provide a list of available actions.
+
+## vagabond ssh
+
+This tool just provides an easy way to SSH into running nodes. Just
+provide the name and it will drop you into a root session:
+
+```
+$ vagabond ssh my_precise_node
+```
+
+## vagabond server
+
+Vagabond will optionally allow the installation of a chef server that is
+localized to the chef repository the Vagabondfile is kept within. This 
+is enabled in the Vagabondfile. To do this, our Vagabondfile would now
+look like this:
+
+```ruby
+{
+  :nodes => {
+    :my_precise_node => {
+      :template => 'ubuntu_1204',
+      :run_list => ['role[base]']
+    }
+  },
+  :local_chef_server => {
+    :enabled => true
+  }
+}
+```
+
+The next command run will trigger vagabond to reprovision and will create
+a server container. The chef server will be auto configured using your
+existing client information. It will seamlessly take over while using
+Vagabond. The server commands are explicitly for the server container.
+The commands are similar to the basic `vagabond` commands, with a few
+extra commands as well.
+
+The `:local_chef_server` hash also has two helper keys for setting up
+the server:
+
+* `:auto_upload` - Uploads all cookbooks, roles, data bags and environments after build
+* `:berkshelf` - Uses berkshelf for cookbook upload instead of knife
+
+## vagabond knife
+
+This tool will let you communicate with the chef server provided by Vagabond.
+Just pass arguments to it like you would knife regularly, and they will 
+be set to the local chef server instead:
+
+```
+$ vagabond knife cookbook list
+```
+
+# Testing
+
+Vagabond has built in support for test-kitchen 1.0. However, it does things
+just a little bit differently. First, it will map the platform defined to an lxc
+template. Second, it will use Librarian to resolve dependencies for the
+cookbook and use those for testing. 
+
+Usage is straightforward:
+
+```
+$ vagabond kitchen test COOKBOOK
+```
+
+You can limit what you test at once by providing `--platform` and/or `--suites`
+options to the test. Tests are run the same was as t-k 1.0. Run lists will be
+merged, attributes pushed to the node, and solo to provision the node.
+
+## Clusters
+
+Vagabond offers an extra type of testing. Instead of simply providing an isolated
+place to test a cookbook (on a single node), Vagabond allows you to run tests against
+a cluster of nodes, with a real chef server. This allows for real integration testing
+within a real isolated environment. Multiple nodes. Real Chef server. A real environment
+that can properly test the behavior of an infrastructure. 
+
+## How it works
+
+Currently, Vagabond adds an extra key the `.kitchen.yml` file of a cookbook. It's
+the `clusters` key. It's a simple hash. The keys are the names of the cluster. The
+values are arrays of strings that identify the suites to be built for the cluster.
+A simple example looks like this:
+
+```
+clusters:
+  default:
+    - cool_suite1
+    - cool_suite2
+```
+
+And now, we can spin this up doing:
+
+```
+$ vagabond kitchen test COOKBOOK --cluster default --platform ubuntu-12.04
+```
+
+First, this will restrict our testing to just the ubuntu-12.04 platform. Next
+it will upload all assets to the local chef server. It then provisions two
+nodes. First it provisions cool_suite1. Next it provisions cool_suite2. These
+nodes are defined by their configuration in the suites. After the nodes
+have successfully provisioned, test kitchen is then run on each of the nodes
+in the cluster sequentially.
+
+Simple, yet so very very awesome. \o/
+
+## More to come
+
+Testing support is still very young, just like Vagabond. But there are plans
+in the works for more features. Take a look at the issues section in github
+with `kitchen` tags. Feel free to add more if you see something missing!

data/bin/vagabond

@@ -1,6 +1,28 @@
 #!/usr/bin/env ruby
 
+Signal.trap('INT'){ exit 255 }
+
 require 'rubygems'
-require 'vagabond/commands'
+require 'vagabond'
 
-Vagabond::Commands.new.run!(ARGV)
+if(ARGV.include?('--version') || ARGV.include?('-v'))
+  require 'vagabond/vagabond'
+  Vagabond::Vagabond.new.send(:version)
+else
+  case arg = ARGV.shift
+  when 'server'
+    require 'vagabond/server'
+    Vagabond::Server
+  when 'knife'
+    require 'vagabond/knife'
+    ARGV.unshift(arg) unless ARGV.empty?
+    Vagabond::Knife
+  when 'test', 'kitchen'
+    require 'vagabond/kitchen'
+    Vagabond::Kitchen
+  else
+    require 'vagabond/vagabond'
+    ARGV.unshift(arg)
+    Vagabond::Vagabond
+  end.start
+end

data/lib/vagabond/actions/create.rb

@@ -1,10 +1,11 @@
 module Vagabond
   module Actions
     module Create
-      def create
+      def _create
+        name_required!
         if(lxc.exists?)
           ui.warn "Node already exists: #{name}" unless name == 'server'
-          start
+          _start
         else
           ui.info "#{ui.color('Vagabond:', :bold)} Creating #{ui.color(name, :green)}"
           do_create
@@ -15,12 +16,20 @@ module Vagabond
       private
 
       def do_create
-        com = "#{sudo}lxc-start-ephemeral -d -o #{config[:template]}"
+        tmpl = config[:template]
+        if(internal_config[:template_mappings].keys.include?(tmpl))
+          tmpl = internal_config[:template_mappings][tmpl]
+        elsif(!BASE_TEMPLATES.include?(tmpl))
+          ui.fatal "Template requested for node does not exist: #{tmpl}"
+          exit EXIT_CODES[:invalid_template]
+        end
+        bind_path = File.expand_path(File.dirname(vagabondfile.path))
+        com = "#{sudo}lxc-start-ephemeral -d -b #{bind_path} -o #{tmpl}"
         debug(com)
-        c = Mixlib::ShellOut.new("#{com} && sleep 3", :live_stream => Config[:debug])
+        c = Mixlib::ShellOut.new("#{com} && sleep 3", :live_stream => options[:debug])
         c.run_command
         e_name = c.stdout.split("\n").last.split(' ').last.strip
-        @internal_config[:mappings][name] = e_name
+        @internal_config[mappings_key][name] = e_name
         @internal_config.save
         @lxc = Lxc.new(e_name)
       end

data/lib/vagabond/actions/destroy.rb

@@ -1,7 +1,9 @@
 module Vagabond
   module Actions
     module Destroy
-      def destroy
+
+      def _destroy
+        name_required!
         if(lxc.exists?)
           ui.info "#{ui.color('Vagabond:', :bold)} Destroying node: #{ui.color(name, :red)}"
           do_destroy
@@ -15,24 +17,24 @@ module Vagabond
 
       def do_destroy
         lxc.stop if lxc.running?
-        com = "#{Config[:sudo]}lxc-destroy -n #{lxc.name}"
+        com = "#{options[:sudo]}lxc-destroy -n #{lxc.name}"
         debug(com)
-        cmd = Mixlib::ShellOut.new(com, :live_stream => Config[:debug])
+        cmd = Mixlib::ShellOut.new(com, :live_stream => options[:debug])
         cmd.run_command
         cmd.error!
         if(cmd.stderr.include?('skipping'))
           ui.info ui.color('  -> Failed to unmount some resources. Forcing manually.', :yellow)
           %w(rootfs ephemeralbind).each do |mnt|
-            com = "#{Config[:sudo]}umount /var/lib/lxc/#{lxc.name}/#{mnt}"
+            com = "#{options[:sudo]}umount /var/lib/lxc/#{lxc.name}/#{mnt}"
             debug(com)
-            cmd = Mixlib::ShellOut.new(com, :live_stream => Config[:debug])
+            cmd = Mixlib::ShellOut.new(com, :live_stream => options[:debug])
             cmd.run_command
-            com = "#{Config[:sudo]}lxc-destroy -n #{lxc.name}"
+            com = "#{options[:sudo]}lxc-destroy -n #{lxc.name}"
             debug(com)
-            cmd = Mixlib::ShellOut.new(com, :live_stream => Config[:debug])
+            cmd = Mixlib::ShellOut.new(com, :live_stream => options[:debug])
             cmd.run_command
             cmd.error!
-            internal_config[:mappings].delete(name)
+            internal_config[mappings_key].delete(name)
           end
           internal_config.save
         end

data/lib/vagabond/actions/freeze.rb

@@ -1,7 +1,8 @@
 module Vagabond
   module Actions
     module Freeze
-      def freeze
+      def _freeze
+        name_required!
         if(lxc.exists?)
           ui.info "#{ui.color('Vagabond:', :bold)} Freezing node: #{ui.color(name, :blue)}"
           if(lxc.running?)

data/lib/vagabond/actions/provision.rb

@@ -1,7 +1,8 @@
 module Vagabond
   module Actions
     module Provision
-      def provision
+      def _provision
+        name_required!
         if(lxc.exists?)
           if(lxc.running?)
             do_provision
@@ -22,8 +23,8 @@ module Vagabond
         if(config[:environment])
           com << "-E #{config[:environment]}"
         end
-        if(Config[:knife_opts])
-          com << Config[:knife_opts]
+        if(options[:knife_opts])
+          com << options[:knife_opts]
         end
         debug(com)
         # Send the live stream out since people will generally want to
@@ -33,8 +34,10 @@ module Vagabond
         # NOTE: cmd.status.success? won't be valid, so check for FATAL
         unless(cmd.stdout.split("\n").last.to_s.include?('FATAL'))
           ui.info ui.color('  -> PROVISIONED', :magenta)
+          true
         else
           ui.info ui.color('  -> PROVISION FAILED', :red)
+          false
         end
       end
 

data/lib/vagabond/actions/rebuild.rb

@@ -1,15 +1,16 @@
 module Vagabond
   module Actions
     module Rebuild
-      def rebuild
+      def _rebuild
+        name_required!
         ui.info "#{ui.color('Vagabond:', :bold)} Rebuilding #{ui.color(name, :blue)}"
-        destroy
+        _destroy
         @lxc = Lxc.new(name)
-        destroy
-        Config[:force_solo] = true
+        _destroy
+        options[:force_solo] = true
         ui.info ui.color('  -> DESTROYED!', :red)
         internal_config.run_solo
-        internal_config[:mappings].delete(name)
+        internal_config[mappings_key].delete(name)
         internal_config.save
         ui.info ui.color('  -> REBUILT!', :green)
       end

data/lib/vagabond/actions/ssh.rb

@@ -1,11 +1,12 @@
 module Vagabond
   module Actions
     module SSH
-      def ssh
+      def _ssh
+        name_required!
         if(lxc.exists?)
           if(lxc.running?)
             ui.info "#{ui.color('Vagabond:', :bold)} SSH connect to: #{ui.color(name, :cyan)}"
-            exec("#{Config[:sudo]}ssh root@#{lxc.container_ip(10, true)} -i /opt/hw-lxc-config/id_rsa -oStrictHostKeyChecking=no")
+            exec("#{options[:sudo]}ssh root@#{lxc.container_ip(10, true)} -i /opt/hw-lxc-config/id_rsa -oStrictHostKeyChecking=no")
           else
             ui.error "Node not running: #{name}"
           end

data/lib/vagabond/actions/start.rb

@@ -2,7 +2,8 @@ module Vagabond
   module Actions
     module Start
 
-      def start
+      def _start
+        name_required!
         ui.info "#{ui.color('Vagabond:', :bold)} Starting node: #{ui.color(name, :green)}"
         do_start
         ui.info ui.color('  -> STARTED', :green)

data/lib/vagabond/actions/status.rb

@@ -1,12 +1,28 @@
 module Vagabond
   module Actions
     module Status
-      def status
+      class << self
+        def included(klass)
+          klass.class_eval do
+            class << self
+              def _status_desc
+                if(defined?(Server) && self == Server)
+                  ['status', 'Status of server']
+                else
+                  ['status [NODE]', 'Status of NODE or all nodes']
+                end
+              end
+            end
+          end
+        end
+      end
+      
+      def _status
         ui.info ui.color("Vagabond node status:\n", :bold)
         if(name)
           status_for(name)
         else
-          (Array(vagabondfile[:boxes].keys) | Array(internal_config[:mappings].keys)).sort.each do |n|
+          (Array(vagabondfile[:boxes].keys) | Array(internal_config[mappings_key].keys)).sort.each do |n|
             status_for(n)
           end
         end
@@ -15,7 +31,7 @@ module Vagabond
       private
 
       def status_for(c_name)
-        m_name = internal_config[:mappings][c_name]
+        m_name = internal_config[mappings_key][c_name]
         state = nil
         status = []
         if(Lxc.exists?(m_name))

data/lib/vagabond/actions/thaw.rb

@@ -1,7 +1,8 @@
 module Vagabond
   module Actions
     module Thaw
-      def thaw
+      def _thaw
+        name_required!
         if(lxc.exists?)
           if(lxc.frozen?)
             ui.info "#{ui.color('Vagabond:', :bold)} Thawing node: #{ui.color(name, :yellow)}"

data/lib/vagabond/actions/up.rb

@@ -1,7 +1,20 @@
 module Vagabond
   module Actions
     module Up
-      def up
+      class << self
+        def included(klass)
+          klass.class_eval do
+            class << self
+              def _up_options
+                [[:auto_provision, :type => :boolean, :default => true]]
+              end
+            end
+          end
+        end
+      end
+
+      def _up
+        name_required!
         if(lxc.exists?)
           if(lxc.running?)
             ui.error "Node already exists and is running: #{name}"
@@ -11,9 +24,9 @@ module Vagabond
             ui.info ui.color('  -> STARTED', :green)
           end
         else
-          create
+          _create
         end
-        do_provision unless Config[:disable_auto_provision]
+        do_provision if options[:auto_provision]
       end
 
     end

data/lib/vagabond/constants.rb

@@ -0,0 +1,36 @@
+require 'chef/mash'
+
+module Vagabond
+  BASE_TEMPLATES = File.readlines(
+    File.join(File.dirname(__FILE__), 'cookbooks/vagabond/attributes/default.rb')
+  ).map do |l|
+    l.scan(%r{bases\]\[:([^\]]+)\]}).flatten.first
+  end.compact.uniq
+
+  COLORS = Mash.new(
+    :success => :green,
+    :create => :green,
+    :setup => :blue,
+    :error => :red,
+    :failed => :red,
+    :verified => :yellow,
+    :converged => :magenta,
+    :destroyed => :red,
+    :kitchen => [:cyan, :bold]
+  )
+  
+  EXIT_CODES = Mash.new(
+    :success => 0,
+    :reserved_name => 1,
+    :invalid_name => 2,
+    :invalid_base_template => 3,
+    :invalid_action => 4,
+    :invalid_template => 5,
+    :kitchen_missing_yml => 6,
+    :kitchen_no_cookbook_arg => 7,
+    :kitchen_too_many_args => 8,
+    :kitchen_invalid_platform => 9,
+    :missing_node_name => 10,
+    :cluster_invalid => 11
+  )
+end