furnish 0.0.1

data/.gitignore

@@ -0,0 +1,19 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+coverage
+html

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in furnish.gemspec
+gemspec
+
+gem 'guard-rake', :git => "https://github.com/erikh/guard-rake", :branch => "failure_ok"

data/Guardfile

@@ -0,0 +1,10 @@
+# vim: ft=ruby
+guard 'minitest' do
+  # with Minitest::Unit
+  watch(%r!^test/(.*)\/?test_(.*)\.rb!)
+  watch(%r!^test/(?:helper|mt_cases)\.rb!) { "test" }
+end
+
+guard 'rake', :failure_ok => true, :run_on_all => false, :task => 'rdoc_cov' do
+  watch(%r!^lib/(.*)([^/]+)\.rb!)
+end

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Erik Hollensbe
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,110 @@
+# Furnish
+
+Furnish is a scheduler that thinks about dependencies and provisioning. It's
+the core of the provisioning logic in
+[chef-workflow](https://github.com/chef-workflow/chef-workflow).
+
+Provisioners are just a pipeline of actions which raise and lower the existence
+of... something. They encapsulate state, and the actions of dealing with that
+state. While chef-workflow uses this for virtual machine and "cloud" things,
+anything that has on and off state can be managed with Furnish.
+
+Furnish is novel because it lets you walk away from the problem of dealing with
+command pipelines and persistence, in a way that lets you deal with it
+concurrently or serially without caring too much, making testing things that
+use Furnish a lot easier. It has a number of guarantees it makes about this
+stuff. See `Contracts` below.
+
+Outside of that, it's just solving Dining Philosophers with Waiters and
+cheating a little by knowing how MRI's thread scheduler works.
+
+Furnish requires MRI Ruby 1.9.3 at minimum. It probably will explode violently
+on a Ruby implemention that doesn't have a GVL or the I/O based coroutine
+scheduler MRI has. If you don't like that, patches welcome.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'furnish'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install furnish
+
+## Usage
+
+```ruby
+Furnish.init("state.db")
+# set a logger if you want -- See Furnish::Logger for more info
+Furnish.logger = File.open('log', 'w')
+# start a scheduler and start spinning
+scheduler = Furnish::Scheduler.new
+scheduler.run # returns immediately
+
+# or, start it serially
+scheduler.serial = true
+scheduler.run # blocks until provisions finish
+
+# Provision something with a Provisioner -- See Furnish::ProvisionerGroup for
+# how to write them.
+scheduler.schedule_provision('some_name', [MyProvisioner.new])
+
+# depend on other provisions
+scheduler.schedule_provision('some_other_name', [MyProvisioner.new], %w[some_name])
+
+# if you want to block the current thread, you can with the wait_for call
+scheduler.wait_for('some_other_name') # waits until some_other_name provisions successfully.
+
+# in threaded mode (the default), these would have already started. If you're
+# in serial mode, you need to kick the scheduler:
+scheduler.run # blocks until everything finishes
+
+# tell the scheduler to stop -- still finishes what's there, just doesn't do
+# anything new.
+scheduler.stop
+
+# shutdown furnish -- closes state database 
+Furnish.shutdown
+```
+
+## Contracts
+
+Furnish has high level contracts that it guarantees. These are expressed
+liberally in the test suite, and any reported issue that can prove these are
+violated is a blocker.
+
+See Furnish::Scheduler and Furnish::ProvisionerGroup for what "provisioner"
+means in this context.
+
+* Furnish is a singleton and operates on a single database. Only one Furnish
+  instance will exist for any given process.
+* Furnish will never lose track of your state unless it is never given the
+  opportunity to record it (e.g., `kill -9` or a hard power-off).
+* Furnish will never deadlock dealing with state.
+* Furnish, in threaded mode, will never block the provisioning process, and
+  provisioners from one group cannot block another group via Furnish.
+* If a provision crashes or fails:
+  * Furnish will never crash as a result.
+  * Furnish will stop processing new items and raise an exception when
+    Furnish::Scheduler#running? is called.
+  * Currently running items will continue in threaded mode, and their state
+    will be dealt with accordingly.
+  * Furnish will never get into an irrecoverable state -- you can clean up and
+    start the scheduler again if that's what you need to do.
+  * Furnish will never try to "fix" a failed provision. You are responsible for
+    dealing with recovery.
+* Furnish will always come with a serial mode to deal with bad actors (quite
+  literally) in a toolkit-independent way, when possible.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,31 @@
+require "bundler/gem_tasks"
+require 'rake/testtask'
+require 'rdoc/task'
+
+Rake::TestTask.new do |t|
+  t.libs << "test"
+  t.test_files = FileList["test/test_*.rb"]
+  t.verbose = true
+end
+
+RDoc::Task.new do |rdoc|
+  rdoc.title = "Furnish: A novel way to do virtual machine provisioning"
+  rdoc.rdoc_files.include("lib/**/*.rb")
+  rdoc.rdoc_files -= ["lib/furnish/version.rb"]
+  if ENV["RDOC_COVER"]
+    rdoc.options << "-C"
+  end
+end
+
+desc "run tests with coverage report"
+task "test:coverage" do
+  ENV["COVERAGE"] = "1"
+  Rake::Task["test"].invoke
+end
+
+desc "run rdoc with coverage report"
+task :rdoc_cov do
+  # ugh
+  ENV["RDOC_COVER"] = "1"
+  ruby "-S rake rerdoc"
+end

data/furnish.gemspec

@@ -0,0 +1,29 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'furnish/version'
+
+Gem::Specification.new do |gem|
+  gem.name          = "furnish"
+  gem.version       = Furnish::VERSION
+  gem.authors       = ["Erik Hollensbe"]
+  gem.email         = ["erik+github@hollensbe.org"]
+  gem.description   = %q{A novel way to do virtual machine provisioning}
+  gem.summary       = %q{A novel way to do virtual machine provisioning}
+  gem.homepage      = ""
+
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+  gem.add_dependency 'palsy', '~> 0.0.2'
+
+  gem.add_development_dependency 'rake'
+  gem.add_development_dependency 'minitest', '~> 4.5.0'
+  gem.add_development_dependency 'guard-minitest'
+  gem.add_development_dependency 'guard-rake'
+  gem.add_development_dependency 'rdoc'
+  gem.add_development_dependency 'rb-fsevent'
+  gem.add_development_dependency 'simplecov'
+end

data/lib/furnish/logger.rb

@@ -0,0 +1,110 @@
+require 'logger'
+require 'thread'
+
+module Furnish
+  #
+  # Furnish::Logger is a thread safe, auto-flushing, IO-delegating logger with
+  # numeric level control.
+  #
+  # See Furnish::Logger::Mixins for functionality you can add to your
+  # provisioners to deal with loggers easily.
+  #
+  # Example:
+  #
+  #   # debug level is 0
+  #   logger = Furnish::Logger.new($stderr, 0)
+  #   # IO methods are sent straight to the IO object, synchronized by a
+  #   # mutex:
+  #   logger.puts "foo"
+  #   logger.print "foo"
+  #
+  #   # if_debug is a way to scope log writes:
+  #
+  #   # this will never run because debug level is 0
+  #   logger.if_debug(1) do
+  #     # self is the IO object here
+  #     puts "foo"
+  #   end
+  #
+  #   logger.if_debug(0) do # this will run
+  #     puts "foo"
+  #   end
+  #
+  #   logger.debug_level = 2
+  #
+  #   # if_debug's parameter merely must equal or be less than the debug
+  #   # level to process.
+  #   logger.if_debug(1) do # will run
+  #     puts "bar"
+  #   end
+  #
+  class Logger
+
+    #
+    # Intended to be mixed in by other classes, provides an API for dealing
+    # with the standard logger object set as Furnish.logger.
+    #
+    module Mixins
+      #
+      # Delegates to Furnish::Logger#if_debug.
+      #
+      def if_debug(*args, &block)
+        Furnish.logger.if_debug(*args, &block)
+      end
+    end
+
+    #
+    # Set the debug level - adjustable after creation.
+    #
+    attr_accessor :debug_level
+
+    #
+    # The IO object. Probably best to not mess with this attribute directly,
+    # most methods will be proxied to it.
+    #
+    attr_reader   :io
+
+    #
+    # Create a new Furnish::Logger. Takes an IO object and an Integer debug
+    # level. See Furnish::Logger class documentation for more information.
+    #
+    def initialize(logger_io=$stderr, debug_level=0)
+      @write_mutex = Mutex.new
+      @io = logger_io
+      @io.sync = true
+      @debug_level = debug_level
+    end
+
+    #
+    # Runs the block if the level is equal to or lesser than the
+    # Furnish::Logger#debug_level. The default debug level is 1.
+    #
+    # The block runs in the context of the Furnish::Logger#io object, that is,
+    # `self` is the IO object.
+    #
+    # If an additional proc is applied, will run that if the debug block would
+    # *not* fire, effectively creating an else. Generally an anti-pattern, but
+    # is useful in a few situations.
+    #
+    # if_debug is synchronized over the logger's mutex.
+    #
+    def if_debug(level=1, else_block=nil, &block)
+      @write_mutex.synchronize do
+        if debug_level >= level and block
+          io.instance_eval(&block)
+        elsif else_block
+          io.instance_eval(&else_block)
+        end
+      end
+    end
+
+    #
+    # Delegates to the Furnish::Logger#io if possible. If not possible, raises
+    # a NoMethodError. All calls are synchronized over the logger's mutex.
+    #
+    def method_missing(sym, *args)
+      raise NoMethodError, "#{io.inspect} has no method #{sym}" unless io.respond_to?(sym)
+      @write_mutex.synchronize { io.__send__(sym, *args) }
+    end
+  end
+end

data/lib/furnish/provisioner.rb

@@ -0,0 +1,46 @@
+module Furnish
+  #
+  # Furnish provides no Provisioners as a part of its package. To use
+  # pre-packaged provisioners, you must install additional packages.
+  #
+  # Provisioners are *objects* that have a simple API and as a result, there is
+  # no "interface" for them in the classic term. You implement it, and if it
+  # doesn't work, you'll know in a hurry.
+  #
+  # I'm going to say this again -- Furnish does not construct your object.
+  # That's your job.
+  #
+  # Provisioners need 3 methods and one attribute, outside of that, you can do
+  # anything you want.
+  #
+  # * name is an attribute (getter/setter) that holds a string. It is used in
+  #   numerous places, is set by the ProvisionerGroup, and must not be volatile.
+  # * startup(*args) is a method to bring the provisioner "up" that takes an
+  #   arbitrary number of arguments and returns truthy or falsey, and in
+  #   exceptional cases may raise. A falsey return value means that provisioning
+  #   failed and the Scheduler will stop. A truthy value is passed to the next
+  #   startup method in the ProvisionerGroup.
+  # * shutdown is a method to bring the provisioner "down" and takes no
+  #   arguments. Like startup, truthy means success and falsey means failed, and
+  #   exceptions are fine, but return values aren't chained.
+  # * report returns an array of strings, and is used for diagnostic functions.
+  #   You can provide anything that fits that description, such as IP addresses
+  #   or other identifiers.
+  #
+  # Tracking external state is not Furnish's job, that's for your provisioner.
+  # Palsy is a state management system that Furnish links deeply to, so any
+  # state tracking you do in your provisioner, presuming you do it with Palsy,
+  # will be tracked along with Furnish's state information in the same
+  # database. That said, you can do whatever you want. Furnish doesn't try to
+  # think about your provisioner deadlocking itself because it's sharing state
+  # with another provisioner, so be mindful of that.
+  #
+  # Additionally, while recovery of furnish's state is something it will do for
+  # you, managing recovery inside your provisioner (e.g., ensuring that EC2
+  # instance really did come up after the program died in the middle of waiting
+  # for it) is your job. Everything will be brought up as it was and
+  # provisioning will be restarted. Account for that.
+  #
+  module Provisioner
+  end
+end

data/lib/furnish/provisioner_group.rb

@@ -0,0 +1,132 @@
+require 'delegate'
+require 'furnish/logger'
+require 'furnish/provisioner'
+
+module Furnish
+  #
+  # A provisioner group is an array of provisioners. See Furnish::Provisioner
+  # for what the Provisioner API looks like.
+  #
+  # A group has a set of provisioner objects, a name for the group, and a list
+  # of names that count as dependencies. It has methods to operate on the group
+  # as a unit, starting them up as a unit and shutting them down. It is
+  # primarily operated on by Furnish::Scheduler.
+  #
+  # In general, you interact with this class via
+  # Furnish::Scheduler#schedule_provision, but you can also construct groups
+  # yourself and deal with them via
+  # Furnish::Scheduler#schedule_provisioner_group.
+  #
+  # It delegates to Array and can be treated like one via the semantics of
+  # Ruby's DelegateClass.
+  #
+  class ProvisionerGroup < DelegateClass(Array)
+
+    include Furnish::Logger::Mixins
+
+    # The name of the group.
+    attr_reader :name
+    # The list of names the group depends on.
+    attr_reader :dependencies
+
+    #
+    # Create a new Provisioner group.
+    #
+    # * provisioners can be an array of provisioner objects or a single item
+    #   (which will be boxed). This is what the array consists of that this
+    #   object is.
+    # * name is a string. always.
+    # * dependencies can either be passed as an Array or Set, and will be
+    #   converted to a Set if they are not a Set.
+    #
+    def initialize(provisioners, name, dependencies=[])
+      #
+      # FIXME maybe move the naming construct to here instead of populating it
+      #       out to the provisioners
+      #
+
+      provisioners = [provisioners] unless provisioners.kind_of?(Array)
+      provisioners.each do |p|
+        p.name = name
+      end
+
+      @name         = name
+      @dependencies = dependencies.kind_of?(Set) ? dependencies : Set[*dependencies]
+
+      super(provisioners)
+    end
+
+    #
+    # Provision this group.
+    #
+    # Initial arguments go to the first provisioner's startup method, and then
+    # the return values, if truthy, get passed to the next provisioner's
+    # startup method. Any falsey value causes a RuntimeError to be raised and
+    # provisioning halts, effectively creating a chain of responsibility
+    # pattern.
+    #
+    def startup(*args)
+      each do |this_prov|
+        unless args = this_prov.startup(args)
+          if_debug do
+            puts "Could not provision #{this_prov.name} with provisioner #{this_prov.class.name}"
+          end
+
+          raise "Could not provision #{this_prov.name} with provisioner #{this_prov.class.name}"
+        end
+      end
+
+      return true
+    end
+
+    #
+    # Deprovision this group.
+    #
+    # Provisioners are run in reverse order against the shutdown method. No
+    # arguments are seeded as in Furnish::ProvisionerGroup#startup. Raise
+    # semantics are the same as with Furnish::ProvisionerGroup#startup.
+    #
+    # If a true argument is passed to this method, the raise semantics will be
+    # ignored (but still logged), allowing all the provisioners to run their
+    # shutdown routines. See Furnish::Scheduler#force_deprovision for
+    # information on how to use this externally.
+    #
+    def shutdown(force=false)
+      reverse.each do |this_prov|
+        success = false
+
+        begin
+          success = perform_deprovision(this_prov) || force
+        rescue Exception => e
+          if force
+            if_debug do
+              puts "Deprovision #{this_prov.class.name}/#{this_prov.name} had errors:"
+              puts "#{e.message}"
+            end
+          else
+            raise e
+          end
+        end
+
+        unless success or force
+          raise "Could not deprovision #{this_prov.name}/#{this_prov.class.name}"
+        end
+      end
+    end
+
+    protected
+
+    #
+    # Just a way to simplify the deprovisioning logic with some generic logging.
+    #
+    def perform_deprovision(this_prov)
+      result = this_prov.shutdown
+      unless result
+        if_debug do
+          puts "Could not deprovision group #{this_prov.name}."
+        end
+      end
+      return result
+    end
+  end
+end

data/lib/furnish/provisioners/dummy.rb

@@ -0,0 +1,87 @@
+module Furnish
+  module Provisioner
+    #
+    # Primarily for testing, this is a provisioner that has a basic storage
+    # model.
+    #
+    # In short, unless you're writing tests you should probably never use this
+    # code.
+    #
+    class Dummy
+
+      #--
+      # Some dancing around the marshal issues with this provisioner. Note that
+      # after restoration, any delegates you set will no longer exist, so
+      # relying on scheduler persistence is a really bad idea.
+      #++
+
+      # basic Palsy::Object store for stuffing random stuff
+      attr_reader   :store
+      # order tracking via Palsy::List, delegation makes a breadcrumb here
+      # that's ordered between all provisioners.
+      attr_reader   :order
+      # name of the provisioner according to the API
+      attr_accessor :name
+      # arbitrary identifier for Dummy#call_order
+      attr_accessor :id
+
+      #
+      # Construct a Dummy.
+      #
+      def initialize
+        @store = Palsy::Object.new('dummy')
+        @order = Palsy::List.new('dummy_order', 'shared')
+      end
+
+      #
+      # call order is ordering on a per-provisioner group basis, and is used to
+      # validate that groups do indeed execute in the proper order.
+      #
+      def call_order
+        @call_order ||= Palsy::List.new('dummy_order', name)
+      end
+
+      #
+      # report shim
+      #
+      def report
+        do_delegate(__method__) do
+          [name]
+        end
+      end
+
+      #
+      # startup shim
+      #
+      def startup(*args)
+        do_delegate(__method__) do
+          true
+        end
+      end
+
+      #
+      # shutdown shim
+      #
+      def shutdown
+        do_delegate(__method__) do
+          true
+        end
+      end
+
+      #
+      # Helper to trace calls to this provisioner. Pretty much everything we
+      # care about goes through here.
+      #
+      def do_delegate(meth_name)
+        meth_name = meth_name.to_s
+
+        # indicate we actually did something
+        @store[ [name, meth_name].join("-") ] = Time.now.to_i
+        @order.push(name)
+        call_order.push(id || "unknown")
+
+        yield
+      end
+    end
+  end
+end