rspec-system 0.1.1 → 0.1.2

data/README.md

@@ -32,7 +32,7 @@ Start by creating a helper file in `spec/spec_helper_system.rb` containing somet
       c.system_setup_block = proc do
         include RSpecSystem::Helpers
         # Insert some setup tasks here
-        run('main', 'yum install -y ntp')
+        system_run('yum install -y ntp')
       end
     end
 
@@ -42,8 +42,8 @@ Create the directory `spec/system` in your project, make sure your unit tests go
 
     describe 'basics' do
       it 'should cat /etc/resolv.conf' do
-        run('main', 'cat /etc/resolv.conf') do |status,stdout,stderr|
-          stdout.should =~ /localhost/
+        system_run('cat /etc/resolv.conf') do |s,o,e|
+          o.should =~ /localhost/
         end
       end
     end
@@ -63,23 +63,38 @@ That will setup the rake task `rake spec:system`.
 
 ### Creating a nodeset file
 
-A nodeset file outlines all the node configurations for your tests. The concept here is to define one or more 'nodesets' each nodeset containing one or more 'nodes'.
+A nodeset file outlines all the node configurations for your tests. The concept here is to define one or more 'nodesets' each nodeset containing one or more 'nodes'. Create the file in your projects root directory as `.nodeset.yml`.
 
     ---
     default_set: 'centos-58-x64'
     sets:
       'centos-58-x64':
         nodes:
-          "main":
+          'main.vm':
             prefab: 'centos-58-x64'
+      'debian-606-x64':
+        nodes:
+          'main.vm':
+            prefab: 'debian-606-x64'
 
 The file must adhere to the Kwalify schema supplied in `resources/kwalify-schemas/nodeset_schema.yml`.
 
+* `sets`: Each set contains a series of nodes, and is given a unique name. You can create sets with only 1 node if you like.
+* `sets -> [setname] -> nodes`: Node definitions for a set. Each node needs a unique name so you can address each one individualy if you like.
+* `sets -> [setname] -> nodes -> [name] -> prefab`: This relates to the prefabricated node template you wish to use. Currently this is the only way to launch a node. Look in `resources/prefabs.yml` for more details.
+* `default_set`: this is the default set to run if none are provided with `rake spec:system`. This should be the most common platform normally.
+
 ### Prefabs
 
-Prefabs are 'pre-rolled' virtual images, for now its the only way to do it.
+Prefabs are 'pre-rolled' virtual images, for now its the only way to specify a template. In the future we will probably allow you to specify your own prefab file, and override prefab settings in a nodeset file as well.
+
+The current built-in prefabs are defined in `resources/prefabs.yml`. The current set are based on boxes hosted on <http://puppet-vagrant-boxes.puppetlabs.com> as they have been built by myself and are generally trusted and have a reproducable build cycle (they aren't just 'golden images'). In the future I'll probably expand that list, but attempt to stick to boxes that we have control over.
+
+Prefabs are designed to be generic across different hosting environments. For example, you should be able to use a prefab string and launch an EC2 or Vagrant image and find that the images are identical (or as much as possible). The goal should be that local Vagrant users should find their own local tests pass, and when submitting code this should not change for EC2.
+
+For this reason there are various `provider_specific` settings that apply to different provider types. For now though, only `vagrant` specific settings are provided.
 
-The current prefabs are defined in `resources/prefabs.yml`.
+`facts` in the prefab are literally dumps of `facter -p` on the host stored in the prefab file so you can look them up without addressing the machine. These are accessed using the `system_node#facts` method on the helper results and can be used in conditional logic during test runs and setup tasks. Not all the facts are supplied, only the more interesting ones.
 
 ### Running tests
 
@@ -89,8 +104,8 @@ Run the system tests with:
 
 Instead of switches, we use a number of environment variables to modify the behaviour of running tests. This is more inline with the way testing frameworks like Jenkins work, and should be pretty easy for command line users as well:
 
-* *RSPEC_VIRTUAL_ENV* - the type of virtual environment to run (currently `vagrant` is the only option)
-* *RSPEC_SET* - the set to use when running tests (defaults to the `default_set` setting in the projects `.nodeset.yml` file)
+* *RSPEC_VIRTUAL_ENV* - the type of virtual environment to run (currently `vagrant` is the only option). I'm undecided about this variable, so assume it might change in the future.
+* *RSPEC_SET* - the set to use when running tests (defaults to the `default_set` setting in the projects `.nodeset.yml` file). This string must align with the entries under `sets` in your `.nodeset.yml`.
 
 So if you wanted to run an alternate nodeset you could use:
 
@@ -102,16 +117,17 @@ In Jenkins you should be able to use RSPEC\_SET in a test matrix, thus obtaining
 
 I want to start an eco-system of plugins for rspec-system, but do it in a sane way. Right now I see the following potential plugin types, if you think you can help please do:
 
-* nodes providers - that is, abstractions around other virtualisation tools. Right now a NodeSet is tied to a virtual type, but I think this isn't granual enough.
+* node providers - that is, abstractions around other virtualisation, cloud or system tools. Right now a NodeSet is tied to a virtual type, but I think this isn't granual enough. Some ideas for future providers are:
     * blimpy - for firing up EC2 and OpenStack nodes, useful for Jenkins integration
-    * vmware - for those who have VMWare virtual 'clouds' or boxen
-    * razor - for launching hardware nodes.
-    * manual - not everything has to be 'launched' I can see a need for defining a static configuration for older machines that can't be poked and peeked.
+    * vmware vsphere - for those who have VMWare vSphere deployed already, this would be an awesome bonus.
+    * razor - for launching bare metail nodes for testing purposes. Could be really useful to have baremetal tests for software that needs it like `facter`.
+    * manual - not everything has to be 'launched' I can see a need for defining a static configuration for older machines that can't be poked and peeked. Of course, we might need to add cleanup tasks for this case.
 * helper libraries - libraries that provide test helpers, and setup helpers for testing development on the software in question.
     * distro - helpers that wrap common linux distro tasks, like package installation.
-    * puppet - helpers around installing different versions of puppet, PE as well - firing up masters. Perfect for testing modules I think.
+    * puppet - helpers around installing different versions of puppet, PE as well - firing up masters. Perfect for testing modules I think. Puppet could be used a provisioner for setting up tests, as using shell commands for this purpose is a bit rough, not very idepotent and is re-inventing the wheel.
     * mcollective - for launching the basics, activemq, broker clusters. Useful for testing mcollective agents.
-    * puppetdb - helpers for setting up puppetdb, probably using the modules.
+    * puppetdb - helpers for setting up puppetdb, probably using the modules we already have.
+    * other config management tools - for the purposes of testing modules against them, or using them for test setup provisioners like I've mentioned before with Puppet.
     * others I'm sure ...
 
 These could be shipped as external gems, and plugged in to the rspec-system framework somehow.

data/lib/rspec-system/formatter.rb

@@ -1,6 +1,16 @@
 require "rspec/core/formatters/base_text_formatter"
 
 module RSpecSystem
+  # This custom formatter is designed for rspec-system test presentation
+  #
+  # Because rspec-system tests are often wordier and require lots of diagnostic
+  # information to be enabled for future debugging, the traditional document
+  # and progress formatters just simply aren't sufficient.
+  #
+  # This formatter instead treats each test as a document section, splitting
+  # up the output with obvious breaks so the user can clearly see when a test
+  # has started and finished. It also attempts to use color for visibility
+  # as well as listing test case information in a more verbose way.
   class Formatter < RSpec::Core::Formatters::BaseTextFormatter
     def initialize(output)
       super(output)

data/lib/rspec-system/helpers.rb

@@ -1,47 +1,86 @@
 # This module contains the main rspec helpers that are to be used within
-# rspec-system tests.
-#
-# The methods here-in are accessible within your rspec tests and can also
-# be used within your setup blocks as well.
+# rspec-system tests. These are the meat-and-potatoes of your system tests,
+# and in theory there shouldn't be anything you can't do without the helpers
+# here.
 #
 # These helpers in particular are core to the framework. You can however
-# combined these helpers to create your own more powerful helpers in rspec
+# combine these helpers to create your own more powerful helpers in rspec
 # if you wish.
 #
+# The helpers themselves are split into two main groups, Queries:
+#
+# * +system_node+ - queries and returns node information
+#
+# And Actions:
+#
+# * +system_run+ - runs a command on a node
+# * +system_rcp+ - remote copies to a node
+#
 # @example Using run within your tests
 #   describe 'test running' do
 #     it 'run cat' do
-#       run 'cat /etc/resolv.conf' do |status, out, err|
-#         status.exitstatus.should == 0
-#         stdout.should =~ /localhost/
+#       system_run 'cat /etc/resolv.conf' do |s, o, e|
+#         s.exitstatus.should == 0
+#         o.should =~ /localhost/
 #       end
 #     end
 #   end
 # @example Using rcp in your tests
 #   describe 'test running' do
 #     it 'copy my files' do
-#       rcp :sp => 'mydata', :dp => '/srv/data'.should be_true
+#       system_rcp :sp => 'mydata', :dp => '/srv/data'.should be_true
 #     end
 #   end
 # @example Using node in your tests
 #   describe 'test running' do
 #     it 'do something if redhat' do
-#       if node.facts[:operatingsystem] == 'RedHat' do
-#         run 'cat /etc/redhat-release'
+#       if system_node.facts['operatingsystem'] == 'RedHat' do
+#         system_run 'cat /etc/redhat-release'
+#       end
+#     end
+#   end
+# @example Make your own helper
+#   describe 'my own helper' do
+#     def install_puppet
+#       facts = system_node.facts
+#
+#       # Grab PL repository and install PL copy of puppet
+#       if facts['osfamily'] == 'RedHat'
+#         system_run('rpm -ivh http://yum.puppetlabs.com/el/5/products/i386/puppetlabs-release-5-6.noarch.rpm')
+#         system_run('yum install -y puppet')
+#       elsif facts['osfamily'] == 'Debian'
+#         system_run("wget http://apt.puppetlabs.com/puppetlabs-release-#{facts['lsbdistcodename']}.deb")
+#         system_run("dpkg -i puppetlabs-release-#{facts['lsbdistcodename']}.deb")
+#         system_run('apt-get update')
+#         system_run('apt-get install -y puppet')
+#       end
+#     end
+#
+#     it 'test installing latest puppet' do
+#       install_puppet
+#       system_run('puppet apply --version') do |s, o, e|
+#         s.exitstatus == 0
+#         o.should =~ /3.1/
+#         e.should == ''
 #       end
 #     end
 #   end
 module RSpecSystem::Helpers
   # @!group Actions
 
-  # Runs a shell command on a test host, returning status, stdout and stderr.
+  # Runs a shell command on a test host.
   #
   # When invoked as a block the status,stdout and stderr are yielded to the
-  # block as parameters.
+  # block as parameters. If not these three variables are returned to the
+  # caller.
   #
   # If you have only provided 1 node in your nodeset, or you have specified a
   # a default you can avoid entering the name of the node if you wish.
   #
+  # The underlying implementation is actually performed by the particular
+  # node provider, however this abstraction should mean you shouldn't need
+  # to worry about that.
+  #
   # @api public
   # @param options [Hash, String] options for command execution, if passed a
   #   string it will just use that for the command instead as a convenience.
@@ -58,15 +97,17 @@ module RSpecSystem::Helpers
   # @yieldparam stderr [String] the standard error of the command result
   # @return [Array<Process::Status,String,String>] returns status, stdout and
   #  stderr when called as a simple method.
-  def run(options)
+  def system_run(options)
     ns = rspec_system_node_set
     dn = ns.default_node
 
-    # Take options as a string instead
+    # If options is a string, turn the string into a command in the normal
+    # options hash.
     if options.is_a?(String)
       options = {:c => options}
     end
 
+    # Defaults etc.
     options = {
       :node => options[:n] || dn,
       :n => options[:node] || dn,
@@ -75,12 +116,12 @@ module RSpecSystem::Helpers
     }.merge(options)
 
     if options[:c].nil?
-      raise "Cannot use run with no :command option"
+      raise "Cannot use system_run with no :command option"
     end
 
-    log.info("run #{options[:c]} on #{options[:n].name} executed")
+    log.info("system_run #{options[:c]} on #{options[:n].name} executed")
     status, stdout, stderr = result = ns.run(options)
-    log.info("run results:\n" +
+    log.info("system_run results:\n" +
       "-----------------------\n" +
       "Exit Status: #{status.exitstatus}\n" +
       "<stdout>#{stdout}</stdout>\n" +
@@ -94,14 +135,15 @@ module RSpecSystem::Helpers
     end
   end
 
-  # Remotely copy files to a test node. This will use the underlying nodes
-  # rcp mechanism to do the transfer for you, so you generally shouldn't
-  # need to consider the implementation.
+  # Remotely copy files to a test node
+  #
+  # Just specify a source path, destination path, and optionally a destination
+  # node (if the default isn't enough) and go.
   #
-  # Just specify a source, and a destination path, and go.
+  # The underlying implementation is actually performed by the particular
+  # node provider, however this abstraction should mean you shouldn't need
+  # to worry about that.
   #
-  # @example Remote copy /srv/data to remote host
-  #   rcp(:dest_path => '/srv/data', :source_path => 'mydata')
   # @param options [Hash] options for command execution
   # @option options [String] :source_path source to copy files from (currently
   #    only locally)
@@ -115,10 +157,7 @@ module RSpecSystem::Helpers
   #   for future use. Patches welcome.
   # @option options [RSpecSystem::Node] :s alias for source_node
   # @return [Bool] returns true if successful
-  # @todo Need to create some helpers for validating input and creating default,
-  #   aliases and bloody yarddocs from some other magic format. Ideas?
-  # @todo Support system to system copy using source_node option.
-  def rcp(options)
+  def system_rcp(options)
     options = {
       :source_path => options[:sp],
       :destination_path => options[:dp],
@@ -126,15 +165,15 @@ module RSpecSystem::Helpers
       :sp => options[:source_path],
       :destination_node => rspec_system_node_set.default_node,
       :d => rspec_system_node_set.default_node,
-      :source_node => '',
-      :s => '',
+      :source_node => nil,
+      :s => nil,
     }.merge(options)
 
     d = options[:d]
     sp = options[:sp]
     dp = options[:dp]
 
-    log.info("rcp from #{sp} to #{d.name}:#{dp} executed")
+    log.info("system_rcp from #{sp} to #{d.name}:#{dp} executed")
     status, stdout, stderr = results = rspec_system_node_set.rcp(options)
     log.info("rcp results:\n" +
       "-----------------------\n" +
@@ -160,16 +199,18 @@ module RSpecSystem::Helpers
   # @param options [Hash] search criteria
   # @option options [String] :name the canonical name of the node
   # @return [RSpecSystem::Node] node object
-  def node(options = {})
+  def system_node(options = {})
     ns = rspec_system_node_set
     options = {
-      :name => ns.default_node,
+      :name => ns.default_node.name,
     }.merge(options)
 
-    if !options[:name].nil?
-      return ns.nodes[options[:name]]
+    name = options[:name]
+
+    if name.nil?
+      raise "No nodes search specified, and no default"
     else
-      raise "No nodes to return"
+      return ns.nodes[name]
     end
   end
 end

data/lib/rspec-system/log.rb

@@ -1,6 +1,10 @@
 require 'logger'
 
+# This log overlay module, provides access to the +log+ method.
 module RSpecSystem::Log
+  # Return the default Logger object.
+  #
+  # @return [Logger] default logger object
   def log
     return @logger if @logger
     @logger = ::Logger.new(STDOUT)

data/rspec-system.gemspec

@@ -2,7 +2,7 @@
 Gem::Specification.new do |s|
   # Metadata
   s.name        = "rspec-system"
-  s.version     = "0.1.1"
+  s.version     = "0.1.2"
   s.authors     = ["Ken Barber"]
   s.email       = ["ken@bob.sh"]
   s.homepage    = "https://github.com/kbarber/rspec-system"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rspec-system
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.2
   prerelease: 
 platform: ruby
 authors: