orchestrated 0.0.1

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in orchestrated.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Paydici Inc.
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.markdown

@@ -0,0 +1,133 @@
+Orchestrated
+============
+
+The [delayed_job](https://github.com/collectiveidea/delayed_job) Ruby Gem provides a job queuing system for Ruby. It implements an elegant API for delaying execution of any object method. Not only is the execution of the method (message delivery) delayed in time, it is potentially shifted in space. By shifting in space, i.e. running in a separate virtual machine, possibly on a separate computer, multiple CPUs can be brought to bear on a computing problem.
+
+By breaking up otherwise serial execution into multiple queued jobs, a program can be made more scalable. Processing of (distributed) queues has a long and successful history in data processing for this reason.
+
+Queuing works well for simple tasks. By simple I mean, the task can be done all at once, in one piece. It has no dependencies on other tasks. This works well for performing a file upload task in the background (to avoid tying up a Ruby virtual machine process/thread). More complex (compound) multi-part tasks, however, do not fit this model. Examples of complex (compound) tasks include:
+
+1. pipelined (multi-step) generation of complex PDF documents
+2. extract/transfer/load (ETL) jobs that may load thousands of database records
+
+If we would like to scale these compound operations, breaking them into smaller parts, and managing the execution of those parts across many computers, we need an "orchestrator". This project implements just such a framework, called "Orchestrated".
+
+Installation
+------------
+
+Add this line to your application's Gemfile:
+
+    gem 'orchestrated'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install orchestrated
+
+The API
+-------
+
+To orchestrate (methods) on your own classes you simply call ```acts_as_orchestrated``` in the class definition like this:
+
+```ruby
+class StatementGenerator
+
+  acts_as_orchestrated
+
+  def generate(statement_id)
+  ...
+  end
+
+  def render(statement_id)
+  ...
+  end
+
+end
+```
+
+Declaring ```acts_as_orchestrated``` on your class gives it two methods:
+
+* ```orchestrated```—call this to specify your workflow prerequisite, and designate a workflow step
+* ```orchestration```—call this in the context of a workflow step (execution) to access orchestration (and prerequisite) context
+
+After that you can orchestrate any method on such a class e.g.
+
+```ruby
+gen = StatementGenerator.new
+gen.orchestrated( orchestrated.generate(stmt_id) ).render(stmt_id)
+```
+
+The next time you process a delayed job, the :generate message will be delivered. The time after that, the :render message will be delivered.
+
+What happened there? The pattern is:
+
+1. create an orchestrated object (instantiate it)
+2. call orchestrated on it: this returns an "orchestration"
+3. send a message to the orchestration (returned in the second step)
+
+Now the messages you can send in (3) are limited to the messages that your object can respond to. The message will be "remembered" by the framework and "replayed" (on a new instance of your object) somewhere on the network (later).
+
+Not accidentally, this is similar to the way [delayed_job](https://github.com/collectiveidea/delayed_job)'s delay method works. Under the covers, orchestrated is conspiring with [delayed_job](https://github.com/collectiveidea/delayed_job) when it comes time to actually execute a workflow step. Before that time though, orchestrated keeps track of everything.
+
+Key Concept: Prerequisites (Completion Expressions)
+---------------------------------------------------
+
+Unlike [delayed_job](https://github.com/collectiveidea/delayed_job) ```delay```, the orchestrated ```orchestrated``` method takes an optional parameter: the prerequisite. The prerequisite determines when your workflow step is ready to run.
+
+The return value from "orchestrate" is itself a ready-to-use prerequisite. You saw this in the statement generation example above. The result of the first ```orchestrated``` call was sent as an argument to the second. In this way, the second workflow step was suspended until after the first one finished. You may have also noticed from that example that if you specify no prerequisite then the step will be ready to run immediately, as was the case for the "generate" call).
+
+There are five kinds of prerequisite in all. Some of them are used for combining others. The prerequisites types, also known as "completion expressions" are:
+
+1. ```OrchestrationCompletion```—returned by "orchestrate", complete when its associated orchestration is complete
+2. ```Complete```—always complete
+3. ```FirstCompletion```—aggregates other completions: complete after the first one completes
+4. ```LastCompletion```—aggregates other completions: complete after all of them are complete
+
+See the completion_spec for examples of how to combine these different prerequisite types into completion expressions.
+
+Key Concept: Orchestration State
+--------------------------------
+
+An orchestration can be in one of six (6) states:
+
+![Alt text](https://github.com/paydici/orchestrated/raw/master/Orchestrated::Orchestration_state.png 'Orchestration States')
+
+You'll never see an orchestration in the "new" state, it's for internal use in the framework. But all the others are interesting.
+
+When you create a new orchestration that is waiting on a prerequisite that is not complete yet, the orchestration will be in the "waiting" state. Some time later, if that prerequisite completes, then your orchestration will become "ready". A "ready" orchestration is automatically queued to run by the framework (via [delayed_job](https://github.com/collectiveidea/delayed_job)).
+
+A "ready" orchestration will use [delayed_job](https://github.com/collectiveidea/delayed_job) to delivery its (delayed) message. In the context of such a message delivery (inside your object method e.g. StatementGenerator#generate or StatementGenerator#render) you can rely on the ability to access the current Orchestration (context) object via the "orchestration" accessor.
+
+After your workflow step executes, the orchestration moves into either the "succeeded" or "failed" state.
+
+When an orchestration is "ready" or "waiting" it may be canceled by sending it the ```cancel!``` message. This moves it to the "canceled" state and prevents delivery of the orchestrated message (in the future).
+
+It is important to understand that both of the states: "succeeded" and "failed" are part of a "super-state": "complete". When an orchestration is in either of those two states, it will return ```true``` in response to the ```complete?``` message.
+
+It is not just successful completion of orchestrated methods that causes dependent ones to run—a "failed" orchestration is complete too! If you have an orchestration that actually requires successful completion of its prerequisite then it can inspect the prerequisite as needed. It's accessible through the ```orchestration`` accessor (on the orchestrated object).
+
+Failure (An Option)
+-------------------
+
+Orchestration is built atop [delayed_job](https://github.com/collectiveidea/delayed_job) and borrows [delayed_job](https://github.com/collectiveidea/delayed_job)'s failure semantics. Neither framework imposes any special constraints on the (delayed or orchestrated) methods. In particular, there are no special return values to signal "failure". Orchestration adopts [delayed_job](https://github.com/collectiveidea/delayed_job)'s semantics for failure detection: a method that raises an exception has failed. After a certain number of retries (configurable in [delayed_job](https://github.com/collectiveidea/delayed_job)) the jobs is deemed permanently failed. When that happens, the corresponding orchestration is marked "failed".
+
+See the failure_spec if you'd like to understand more.
+
+Cancelling an Orchestration
+---------------------------
+
+An orchestration can be canceled by sending the (orchestration completion) the ```cancel!``` message. This will prevent the orchestrated method from running (in the future). It will also cancel dependent workflow steps.
+
+The cancellation_spec spells out more of the details.
+
+Contributing
+------------
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/lib/orchestrated/base.rb

@@ -0,0 +1,41 @@
+module Orchestrated
+
+  class Proxy
+    def initialize(prerequisite, target)
+      @prerequisite = prerequisite
+      @target       = target
+    end
+    def method_missing(sym, *args)
+      raise 'cannot orchestrate with blocks because they are not portable across processes' if block_given?
+      OrchestrationCompletion.new do |completion|
+        completion.orchestration = Orchestration.create( @target, sym, args, @prerequisite)
+      end.tap do |completion|
+        completion.save!
+      end
+    end
+  end
+
+  class << self
+    #snarfed from Ruby On Rails
+    def underscore(camel_cased_word)
+      camel_cased_word.to_s.gsub(/::/, '/').
+        gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
+        tr("-", "_").
+        downcase
+    end
+    def belongs_to clazz
+      # borrowed from Ick
+      method_name = self.underscore(self.name.split('::')[-1])
+      unless clazz.method_defined?(method_name)
+        clazz.class_eval "
+          def #{method_name}(prerequisite=Complete.new)
+            raise 'orchestrate does not take a block' if block_given?
+            raise %[cannot use \#{prerequisite.class.name} as a prerequisite] unless
+              prerequisite.kind_of?(CompletionExpression)
+            Proxy.new(prerequisite, self)
+          end"
+      end
+    end
+  end
+
+end

data/lib/orchestrated/completion.rb

@@ -0,0 +1,72 @@
+require 'active_record'
+
+module Orchestrated
+  # a little ditty to support the completion algebra
+  # a composite!
+  # Completion is used as a prerequisite (prerequisites) for message passing
+  class CompletionExpression < ActiveRecord::Base
+    # I'd like to make this abstract, but Rails gets confused if I do
+    # self.abstract_class = true
+    def complete?; throw 'subclass must override!';end
+    # for static analysis
+    def always_complete?; throw 'subclass must override!';end
+    def never_complete?; throw 'subclass must override!';end
+    def canceled?; throw 'subclass must override!';end
+  end
+  class Complete < CompletionExpression
+    def complete?; true; end
+    def always_complete?; true; end
+    def never_complete?; false; end
+    def canceled?; false; end
+  end
+  # Only known use is in testing the framework
+  class Incomplete < CompletionExpression
+    def complete?; false; end
+    def always_complete?; false; end
+    def never_complete?; true; end
+    def canceled?; false; end
+  end
+  class CompositeCompletion < CompletionExpression
+    # self.abstract_class = true
+    has_many :composited_completions
+    has_many :completion_expressions, :through => :composited_completions, :source => :completion_expression
+    def +(c); self << c; end # synonym
+  end
+  class LastCompletion < CompositeCompletion
+    def complete?; completion_expressions.all?(&:complete?); end
+    def always_complete?; completion_expressions.empty?; end
+    def never_complete?; completion_expressions.any?(&:never_complete?); end
+    def canceled?; completion_expressions.any?(&:canceled?); end
+    def <<(c)
+      completion_expressions << c unless c.always_complete?
+      self
+    end
+  end
+  class FirstCompletion < CompositeCompletion
+    def complete?; completion_expressions.any?(&:complete?); end
+    def always_complete?; completion_expressions.any?(&:always_complete?); end
+    def never_complete?; completion_expressions.empty?; end
+    def canceled?; completion_expressions.all?(&:canceled?); end
+    def <<(c)
+      completion_expressions << c unless c.never_complete?
+      self
+    end
+  end
+  class OrchestrationCompletion < CompletionExpression
+    # Arguably, it is "bad" to make this class derive
+    # from CompletionExpression since doing so introduces
+    # the orchestration_id into the table (that constitutes
+    # denormalization since no other types need that field).
+    # The alternative is that we have to do difficult-to-
+    # understand joins when computing dependents at runtime.
+    belongs_to :orchestration
+    validates_presence_of :orchestration_id
+    delegate :complete?, :canceled?, :cancel!, :to => :orchestration
+    def always_complete?; false; end
+    def never_complete?; false; end
+  end
+  class CompositedCompletion < ActiveRecord::Base
+    belongs_to :composite_completion
+    belongs_to :completion_expression
+  end
+end

data/lib/orchestrated/message_delivery.rb

@@ -0,0 +1,31 @@
+module Orchestrated
+  class MessageDelivery
+    attr_accessor :orchestrated, :method_name, :args, :orchestration_id
+
+    def initialize(orchestrated, method_name, args, orchestration_id)
+      raise 'all arguments to MessageDelivery constructor are required' unless
+        orchestrated and method_name and args and orchestration_id
+      self.orchestrated = orchestrated
+      self.method_name  = method_name
+      self.args         = args
+      self.orchestration_id = orchestration_id
+    end
+
+    def perform
+      orchestration = Orchestration.find(self.orchestration_id)
+
+      orchestrated.orchestration = orchestration
+      orchestrated.send(method_name, *args)
+      orchestrated.orchestration = nil
+
+      orchestration.message_delivery_succeeded
+    end
+
+    # delayed_job hands us this message after max_attempts are exhausted
+    def failure
+      orchestration = Orchestration.find(self.orchestration_id)
+      orchestration.message_delivery_failed
+    end
+
+  end
+end

data/lib/orchestrated/object.rb

@@ -0,0 +1,16 @@
+module Orchestrated
+  module InstanceMethods
+    # set by the framework (Orchestration) before
+    # an orchestrated method is called
+    # cleared (nil) outside such a call
+    attr_accessor :orchestration
+  end
+  class ::Object
+    class << self
+      def acts_as_orchestrated
+        Orchestrated.belongs_to self # define "orchestrated instance method"
+        include InstanceMethods
+      end
+    end
+  end
+end

data/lib/orchestrated/orchestration.rb

@@ -0,0 +1,112 @@
+require 'active_record'
+require 'state_machine'
+require 'delayed_job'
+require 'delayed_job_active_record'
+
+module Orchestrated
+  class Orchestration < ActiveRecord::Base
+
+    Handler = Struct.new('Handler', :value, :sym, :args)
+
+    serialize :handler
+
+    belongs_to :prerequisite, :class_name => 'CompletionExpression'
+    belongs_to :delayed_job, :polymorphic => true # loose-ish coupling with delayed_job
+
+    has_many :orchestration_completions
+
+    complete_states = [:succeeded, :failed]
+    state_machine :initial => :new do
+      state :new
+      state :waiting
+      state :ready
+      state :succeeded
+      state :failed
+      state :canceled
+
+      state all - complete_states do
+        def complete?
+          false
+        end
+      end
+
+      state *complete_states do
+        def complete?
+          true
+        end
+      end
+
+      event :prerequisite_changed do
+        transition [:new, :waiting] => :ready, :if => lambda {|orchestration| orchestration.prerequisite.complete?}
+        transition [:ready, :waiting] => :canceled, :if => lambda {|orchestration| orchestration.prerequisite.canceled?}
+        transition :new => :waiting # otherwise
+      end
+
+      event :message_delivery_succeeded do
+        transition :ready => :succeeded
+      end
+
+      event :message_delivery_failed do
+        transition :ready => :failed
+      end
+
+      event :cancel do
+        transition [:waiting, :ready] => :canceled
+      end
+
+      after_transition any => :ready do |orchestration, transition|
+        orchestration.enqueue
+      end
+
+      after_transition :ready => :canceled do |orchestration, transition|
+        orchestration.dequeue
+      end
+
+      after_transition any => complete_states do |orchestration, transition|
+        # completion may make other orchestrations ready to run…
+        # TODO: this is a prime target for benchmarking
+        (Orchestration.with_state('waiting').all - [orchestration]).each do |other|
+          other.prerequisite_changed
+        end
+      end
+
+      after_transition [:ready, :waiting] => :canceled do |orchestration, transition|
+        # cancellation may cancel other orchestrations
+        (Orchestration.with_states(:ready, :waiting).all - [orchestration]).each do |other|
+          other.prerequisite_changed
+        end
+      end
+
+    end
+
+    def self.create( value, sym, args, prerequisite)
+      # set prerequisite in new call so it is passed to state_machine :initial proc
+      new.tap do |orchestration|
+
+        orchestration.handler = Handler.new( value, sym, args)
+
+        # wee! static analysis FTW!
+        raise 'prerequisite can never be complete' if prerequisite.never_complete?
+
+        # saves object as side effect of this assignment
+        # also moves orchestration to :ready state
+        orchestration.prerequisite  = prerequisite
+     end
+    end
+
+    def enqueue
+      self.delayed_job = Delayed::Job.enqueue( MessageDelivery.new( handler.value, handler.sym, handler.args, self.id) )
+    end
+
+    def dequeue
+      delayed_job.destroy# if DelayedJob.exists?(delayed_job_id)
+    end
+
+    alias_method :prerequisite_old_equals, :prerequisite=
+    def prerequisite=(*args)
+      prerequisite_old_equals(*args)
+      prerequisite_changed
+    end
+
+  end
+end

data/lib/orchestrated/version.rb

@@ -0,0 +1,3 @@
+module Orchestrated
+  VERSION = "0.0.1"
+end

data/lib/orchestrated.rb

@@ -0,0 +1,6 @@
+require "orchestrated/version"
+require 'orchestrated/base'
+require 'orchestrated/completion'
+require 'orchestrated/message_delivery'
+require 'orchestrated/orchestration'
+require 'orchestrated/object'

data/orchestrated.gemspec

@@ -0,0 +1,33 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'orchestrated/version'
+
+Gem::Specification.new do |gem|
+  gem.name          = "orchestrated"
+  gem.version       = Orchestrated::VERSION
+  gem.authors       = ["Bill Burcham"]
+  gem.email         = ["bill@paydici.com"]
+  gem.description   = %q{a workflow orchestration framework running on delayed_job and active_record}
+  gem.summary       = %q{Orchestrated is a workflow orchestration framework running on delayed_job and active_record. In the style of delayed_job's 'delay', Orchestration lets you 'orchestrate' delivery of a message so that it will run only after others have been delivered and processed.}
+  gem.homepage      = "https://github.com/paydici/orchestration"
+  gem.license       = 'MIT'
+
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+  gem.add_runtime_dependency     'delayed_job_active_record', '~> 0.3'
+  gem.add_runtime_dependency     'activerecord', ['>= 3']
+  gem.add_runtime_dependency     'state_machine', ['>= 1']
+
+  gem.add_development_dependency 'rake'
+  gem.add_development_dependency 'rails', ['>= 3'] # for rspec-rails
+  gem.add_development_dependency 'rspec-rails'
+  # I couldn't get rspecs transactional fixtures setting to do savepoints
+  # in this project (which is not _really_ a Rails app). database_cleaner
+  # claims it'll help us clean up the database so let's try it!
+  gem.add_development_dependency 'database_cleaner'
+  gem.add_development_dependency 'sqlite3'
+end

data/spec/database.yml

@@ -0,0 +1,17 @@
+# SQLite version 3.x
+#   gem install sqlite3
+#
+#   Ensure the SQLite 3 gem is defined in your Gemfile
+#   gem 'sqlite3'
+development:
+  adapter: sqlite3
+  database: db/development.sqlite3
+  pool: 5
+  timeout: 5000
+
+# Warning: The database defined as "test" will be erased and
+# re-generated from your development database when you run "rake".
+# Do not set this db to the same as development or production.
+test:
+  adapter: sqlite3
+  database: ":memory:"

data/spec/delayed_job_facade.rb

@@ -0,0 +1,28 @@
+require 'delayed_job_active_record'
+
+# facade for controlling delayed_job
+module DJ
+  module_function
+
+  def job_count
+    Delayed::Job.count
+  end
+  def work(num=100)
+    Delayed::Worker.new.work_off(num)
+  end
+  def work_now(num=100)
+    (1..num).each do
+      first = Delayed::Job.first
+      break unless first.present?
+      first.tap{|job| job.run_at = 1.second.ago; job.save!}
+      DJ.work(1)
+    end
+  end
+  def clear_all_jobs
+    Delayed::Job.delete_all
+  end
+  def max_attempts
+    # configured in initializers/delayed_job_config.rb
+    Delayed::Worker.max_attempts
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,110 @@
+require 'simplecov'
+SimpleCov.start do
+  add_group "Orchestrated", "lib/orchestrated"
+end
+
+# Get Rails environment going. Borrowed from delayed_job_active_record project, then heavily modified
+# ...
+
+$:.unshift(File.join( File.dirname(__FILE__), '../lib'))
+
+require 'rubygems'
+require 'bundler/setup'
+
+require 'rails/all' # rspec/rails needs Rails
+require 'rspec/rails' # we want transactional fixtures!
+
+require 'logger'
+
+require 'delayed_job'
+require 'rails'
+
+Delayed::Worker.logger = Logger.new('/tmp/dj.log')
+ENV['RAILS_ENV'] = 'test'
+
+config = YAML.load(File.read('spec/database.yml'))
+ActiveRecord::Base.establish_connection config['test']
+ActiveRecord::Base.logger = Delayed::Worker.logger
+ActiveRecord::Migration.verbose = false
+
+ActiveRecord::Schema.define do
+    create_table :delayed_jobs, :force => true do |table|
+      table.integer  :priority, :default => 0      # Allows some jobs to jump to the front of the queue
+      table.integer  :attempts, :default => 0      # Provides for retries, but still fail eventually.
+      table.text     :handler                      # YAML-encoded string of the object that will do work
+      table.text     :last_error                   # reason for last failure (See Note below)
+      table.datetime :run_at                       # When to run. Could be Time.zone.now for immediately, or sometime in the future.
+      table.datetime :locked_at                    # Set when a client is working on this object
+      table.datetime :failed_at                    # Set when all retries have failed (actually, by default, the record is deleted instead)
+      table.string   :locked_by                    # Who is working on this object (if locked)
+      table.string   :queue                        # The name of the queue this job is in
+      table.timestamps
+    end
+
+    add_index :delayed_jobs, [:priority, :run_at], :name => 'delayed_jobs_priority'
+
+    create_table :orchestrations do |table|
+      table.string     :state
+      table.text       :handler
+      table.references :prerequisite
+      table.references :delayed_job, :polymorphic => true
+      table.timestamps
+    end
+    create_table :completion_expressions do |table|
+      table.string     :type
+      # only one kind of completion expression needs this
+      # (OrchestrationCompletion) but I didn't want to put
+      # it in a separate table because it would really contort
+      # the Rails model
+      table.references :orchestration
+    end
+    create_table :composited_completions do |table|
+      table.references :composite_completion
+      table.references :completion_expression
+    end
+end
+
+# Add this directory so the ActiveSupport autoloading works
+ActiveSupport::Dependencies.autoload_paths << File.dirname(__FILE__)
+
+# when we run via plain old "ruby" command instead of "rspec", this
+# line tells ruby to run the examples
+require 'rspec/autorun'
+
+# This is the present Ruby Gem: the one we are spec-ing/testing
+require 'orchestrated'
+
+# Requires supporting ruby files with custom matchers and macros, etc,
+# in spec/support/ and its subdirectories.
+Dir[File.join( File.dirname(__FILE__), "support/**/*.rb")].each {|f| require f}
+require 'delayed_job_facade'
+require 'spec_helper_methods'
+
+require 'database_cleaner' # see comments below
+
+RSpec.configure do |config|
+  # This standard Rails approach won't work in this project (which is not
+  # _really_ a Rails app after all.
+  #   config.use_transactional_fixtures = true
+  # So we are trying the database_cleaner gem instead:
+  config.before(:suite) do
+    DatabaseCleaner.strategy = :transaction
+    DatabaseCleaner.clean_with(:truncation)
+  end
+
+  config.before(:each) do
+    DatabaseCleaner.start
+  end
+
+  config.after(:each) do
+    DatabaseCleaner.clean
+  end
+
+  # Run specs in random order to surface order dependencies. If you find an
+  # order dependency and want to debug it, you can fix the order by providing
+  # the seed, which is printed after each run.
+  #     --seed 1234
+  config.order = "random"
+
+  config.include SpecHelperMethods
+end

data/spec/spec_helper_methods.rb

@@ -0,0 +1,2 @@
+module SpecHelperMethods
+end

data/spec/support/sample_classes/failer.rb

@@ -0,0 +1,7 @@
+class Failer
+  acts_as_orchestrated
+
+  def always_fail(something)
+    raise 'I never work'
+  end
+end

data/spec/support/sample_classes/first.rb

@@ -0,0 +1,9 @@
+class First
+
+  acts_as_orchestrated
+
+  def do_first_thing(other_prime)
+    5 * other_prime # 5 is a prime
+  end
+
+end

data/spec/support/sample_classes/second.rb

@@ -0,0 +1,9 @@
+class Second
+
+  acts_as_orchestrated
+
+  def do_second_thing(other_prime)
+    7 * other_prime # 7 is a prime
+  end
+
+end

data/spec/unit/cancellation_spec.rb

@@ -0,0 +1,66 @@
+require 'spec_helper'
+
+require 'orchestrated'
+
+shared_examples_for 'cancellation:' do
+  before(:each) do
+    @prerequisite.cancel!
+  end
+  it 'dependent should be in the "canceled" state' do
+    expect(@dependent.reload.canceled?).to be_true
+  end
+end
+
+shared_examples_for 'cannot cancel:' do
+  it 'dependent should be in the "canceled" state' do
+    expect{@prerequisite.cancel!}.to raise_error(StateMachine::InvalidTransition)
+  end
+end
+
+describe 'cancellation' do
+  context 'directly on an orchestration' do
+    before(:each) do
+      @prerequisite = @dependent = First.new.orchestrated.do_first_thing(1)
+    end
+    context 'that is ready' do
+      it_should_behave_like 'cancellation:'
+      it 'should never subsequently deliver the orchestrated message' do
+        First.any_instance.should_not_receive(:do_first_thing)
+        DJ.work(1)
+      end
+    end
+    context 'that is succeeded' do
+      before(:each) do
+        @prerequisite.orchestration.state = 'succeeded'
+      end
+      it_should_behave_like 'cannot cancel:'
+    end
+    context 'that is failed' do
+      before(:each) do
+        @prerequisite.orchestration.state = 'failed'
+      end
+      it_should_behave_like 'cannot cancel:'
+    end
+    context 'that is canceled' do
+      before(:each) do
+        @prerequisite.orchestration.state = 'canceled'
+      end
+      it_should_behave_like 'cannot cancel:'
+    end
+  end
+  context 'of an orchestration that is depended on directly' do
+    before(:each) do
+      @dependent = Second.new.orchestrated( @prerequisite = First.new.orchestrated.do_first_thing(1)).do_second_thing(2)
+    end
+    it_should_behave_like 'cancellation:'
+  end
+  context 'of an orchestration that is depended on through a LastCompletion' do
+    before(:each) do
+      @dependent = Second.new.orchestrated(
+        Orchestrated::LastCompletion.new <<
+          (@prerequisite = First.new.orchestrated.do_first_thing(1))
+        ).do_second_thing(2)
+    end
+    it_should_behave_like 'cancellation:'
+  end
+end

data/spec/unit/completion_spec.rb

@@ -0,0 +1,97 @@
+require 'spec_helper'
+
+require 'orchestrated'
+
+shared_examples_for 'literally complete' do
+  it 'should immediately enqueue the dependent orchestration' do
+    expect(DJ.job_count).to be(1)
+  end
+  it 'should cause the dependent orchestration to run immediately' do
+    First.any_instance.should_receive(:do_first_thing)
+    DJ.work(1)
+  end
+end
+
+describe Orchestrated::CompletionExpression do
+  context 'Complete' do
+    context 'implicitly specified' do
+      before(:each){ First.new.orchestrated.do_first_thing(12) }
+      it_should_behave_like 'literally complete'
+    end
+    context 'explicitly specified' do
+      before(:each){ First.new.orchestrated(Orchestrated::Complete.new).do_first_thing(12) }
+      it_should_behave_like 'literally complete'
+    end
+  end
+  context 'Incomplete' do
+    it 'should immediately raise an error' do
+      expect{First.new.orchestrated(Orchestrated::Incomplete.new).do_first_thing(12)}.to raise_error
+    end
+  end
+  context 'OrchestrationCompletion' do
+    before(:each){Second.new.orchestrated( First.new.orchestrated.do_first_thing(3)).do_second_thing(4)}
+    it 'should block second orchestration until after first runs' do
+      expect(DJ.job_count).to be(1)
+    end
+    it 'should run second orchestration after first is complete' do
+      Second.any_instance.should_receive(:do_second_thing)
+      DJ.work(2)
+    end
+  end
+  context 'FirstCompletion' do
+    context 'given a (literal) Complete' do
+      before(:each) do
+        Second.new.orchestrated( Orchestrated::FirstCompletion.new <<
+          Orchestrated::Complete.new
+          ).do_second_thing(5)
+      end
+      it 'should immediately enqueue the dependent orchestration' do
+        expect(DJ.job_count).to be(1)
+      end
+    end
+    context 'given two OrchestrationCompletions' do
+      before(:each) do
+        Second.new.orchestrated( Orchestrated::FirstCompletion.new <<
+          First.new.orchestrated.do_first_thing(3) <<
+          First.new.orchestrated.do_first_thing(4)
+          ).do_second_thing(5)
+      end
+      it 'should enqueue the dependent orchestration as soon as the first prerequisite completes' do
+        expect(DJ.job_count).to be(2)
+        DJ.work(1)
+        expect(DJ.job_count).to be(2)
+      end
+      it 'should cause the dependent orchestration to run eventually' do
+        Second.any_instance.should_receive(:do_second_thing).with(5)
+        DJ.work(3)
+      end
+      it 'should skip dependents after the first one runs' do
+        First.any_instance.should_not_receive(:do_first_thing).with(4)
+        DJ.work(3)
+      end
+    end
+  end
+  context 'LastCompletion' do
+    context 'given two OrchestrationCompletions' do
+      before(:each) do
+        Second.new.orchestrated( Orchestrated::LastCompletion.new <<
+          First.new.orchestrated.do_first_thing(3) <<
+          First.new.orchestrated.do_first_thing(4)
+          ).do_second_thing(5)
+      end
+      it 'should not enqueue the dependent orchestration as soon as the first prerequisite completes' do
+        expect(DJ.job_count).to be(2)
+        DJ.work(1)
+        expect(DJ.job_count).to be(1)
+      end
+      it 'should not run the dependent orchestration as soon as the first prerequisite completes' do
+        Second.any_instance.should_not_receive(:do_second_thing)
+        DJ.work(2)
+      end
+      it 'should run the dependent orchestration after all the prerequisite are complete' do
+        Second.any_instance.should_receive(:do_second_thing)
+        DJ.work(3)
+      end
+    end
+  end
+end

data/spec/unit/delayed_job_spec.rb

@@ -0,0 +1,79 @@
+require 'spec_helper'
+
+class TestJob < Struct.new(:name)
+
+  class << self
+    attr_accessor :called
+
+    def reset_called
+      @called = Hash.new { |hash, key| hash[key] = 0 }
+    end
+  end
+
+  reset_called # initialized the class
+
+  # some random method
+  def custom_action
+    self.class.called[:custom_action] += 1
+  end
+
+  # -------------- delayed_job lifecycle callbacks ---------------
+  def enqueue(job)
+  end
+
+  def perform
+    self.class.called[:perform] += 1
+  end
+
+  def before(job)
+  end
+
+  def after(job)
+  end
+
+  def success(job)
+  end
+
+  def error(job, exception)
+  end
+
+  def failure
+  end
+
+end
+
+describe Delayed::Job do
+  context 'with a job' do
+    let(:job) {TestJob.new('fred')}
+    it 'should accept rspec message hooks' do
+      # these hooks aren't as useful as you might think since
+      # the object "waked" by DJ is a different object entirely!
+      job.should_receive(:custom_action).and_call_original
+      job.custom_action
+    end
+    it 'should start with an empty queue' do
+      expect(DJ.job_count).to be(0)
+    end
+    it 'should enqueue a job' do
+      expect {
+        job.delay.custom_action
+      }.to change{DJ.job_count}.by(1)
+    end
+    context 'that is enqueued' do
+      before(:each) do
+        DJ.clear_all_jobs
+        job.delay.custom_action # queue exactly one job
+      end
+      it 'should dequeue the job' do
+        expect {
+          successes, failures = DJ.work
+        }.to change{DJ.job_count}.by(-1)
+      end
+      it 'should deliver a message' do
+        expect {
+          successes, failures = DJ.work
+        }.to change{TestJob.called[:custom_action]}.by(1)
+      end
+    end
+  end
+end

data/spec/unit/failure_spec.rb

@@ -0,0 +1,42 @@
+require 'spec_helper'
+
+require 'orchestrated'
+
+describe 'failure' do
+  context 'orchestrating a method that always fails' do
+    before(:each) do
+      Failer.new.orchestrated.always_fail('important stuff')
+    end
+    context 'after first exception from orchestrated method' do
+      before(:each) do
+        DJ.work(1)
+      end
+      it 'should leave the orchestration in the ready state' do
+        expect(Orchestrated::Orchestration.with_state('ready').count).to be(1)
+      end
+      it 'should leave the orchestration in the run queue' do
+        expect(DJ.job_count).to be(1)
+      end
+      context 'on first retry' do
+        it 'should retry with same arguments' do
+          Failer.any_instance.should_receive(:always_fail).with('important stuff')
+          DJ.work_now(1)
+        end
+      end
+    end
+    context 'after (Delayed::Worker.max_attempts + 1) exceptions from orchestrated method' do
+      before(:each) do
+        DJ.work_now(DJ.max_attempts)
+      end
+      it 'should leave the orchestration in the failed state' do
+        expect(Orchestrated::Orchestration.with_state('failed').count).to be(1)
+      end
+      context 'on first subsequent retry' do
+        it 'should never deliver the orchestrated message again' do
+          Failer.any_instance.should_not_receive(:always_fail)
+          DJ.work_now(1)
+        end
+      end
+    end
+  end
+end

data/spec/unit/orchestrated_spec.rb

@@ -0,0 +1,46 @@
+require 'spec_helper'
+
+require 'orchestrated'
+
+describe Orchestrated do
+  context 'initializing' do
+    it 'should not define orchestrated on Object' do
+      expect(Object.public_method_defined?(:orchestrated)).to be_false
+    end
+    it 'should not define orchestrated on ActiveRecord::Base' do
+      expect(ActiveRecord::Base.public_method_defined?(:orchestrated)).to be_false
+    end
+    it 'should define orchestrated on First' do
+      expect(First.public_method_defined?(:orchestrated)).to be_true
+    end
+  end
+  context 'a new orchestrated object' do
+    let(:f){First.new}
+    context 'responding to messages without orchestration' do
+      let(:result){f.do_first_thing(2)} # 2 is a prime number
+      it 'should immediately invoke a non-orchestrated method and return correct result' do
+        expect(result).to eq(5 * 2)
+      end
+    end
+    context 'orchestrating with no prerequisites' do
+      before(:each){@result = f.orchestrated.do_first_thing(2)}
+      after(:each){DJ.clear_all_jobs}
+      it 'should not immediately invoke an orchestrated method' do
+        First.any_instance.should_not_receive(:do_first_thing)
+      end
+      it 'should return an Orchestration object' do
+        expect(@result).to be_kind_of(Orchestrated::CompletionExpression)
+      end
+    end
+  end
+  context 'invocation' do
+    before(:each) do
+      First.new.orchestrated.do_first_thing(1)
+    end
+    it 'should have access to Orchestration' do
+      First.any_instance.should_receive(:orchestration=).with(kind_of(Orchestrated::Orchestration))
+      First.any_instance.should_receive(:orchestration=).with(nil)
+      DJ.work(1)
+    end
+  end
+end

data/spec/unit/rspec_spec.rb

@@ -0,0 +1,21 @@
+class Foo
+  def bump(x)
+    x+1
+  end
+end
+
+describe 'rspec' do
+  context 'mocking should_receive' do
+    it 'should pass method arguments to my block' do
+      x = 0
+      block = nil
+      Foo.any_instance.should_receive(:bump){|x_arg, &block_arg|
+        x = x_arg
+        block = block_arg
+      }
+      Foo.new.bump(1){|hello| puts hello}
+      expect(x).to be(1)
+      expect(block).to be_kind_of(Proc)
+    end
+  end
+end

data/spec/unit/static_analysis_spec.rb

@@ -0,0 +1,48 @@
+require 'spec_helper'
+
+require 'orchestrated'
+
+describe 'performing static analysis' do
+  context 'on a FirstCompletion' do
+    let(:completion){Orchestrated::FirstCompletion.new}
+    context 'that is empty' do
+      # chose this behavior to align with Ruby Enumerable#any?
+      it 'should raise an error since it can never be complete' do
+        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error
+      end
+    end
+    context 'that contains only (static) Incompletes' do
+      before(:each){completion<<Orchestrated::Incomplete.new}
+      it 'should raise an error since it can never be complete' do
+        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error
+      end
+    end
+    context 'that directly containins a (static) Complete' do
+      before(:each){completion<<Orchestrated::Complete.new}
+      it 'should be complete immediately' do
+        expect{completion.complete?}.to be_true
+      end
+    end
+  end
+  context 'on a LastCompletion' do
+    let(:completion){Orchestrated::LastCompletion.new}
+    context 'that is empty' do
+      # chose this behavior to align with Ruby Enumerable#all?
+      it 'should be complete immediately' do
+        expect{completion.complete?}.to be_true
+      end
+    end
+    context 'that contains only (static) Completes' do
+      before(:each){completion<<Orchestrated::Complete.new}
+      it 'should be complete immediately' do
+        expect{completion.complete?}.to be_true
+      end
+    end
+    context 'that directly contains a (static) Incomplete' do
+      before(:each){completion<<Orchestrated::Incomplete.new}
+      it 'should raise an error since it can never be complete' do
+        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error
+      end
+    end
+  end
+end

metadata

@@ -0,0 +1,219 @@
+--- !ruby/object:Gem::Specification
+name: orchestrated
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Bill Burcham
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-01-03 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: delayed_job_active_record
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.3'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3'
+- !ruby/object:Gem::Dependency
+  name: state_machine
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '1'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '3'
+- !ruby/object:Gem::Dependency
+  name: rspec-rails
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: database_cleaner
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: a workflow orchestration framework running on delayed_job and active_record
+email:
+- bill@paydici.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- Gemfile
+- LICENSE.txt
+- Orchestrated::Orchestration_state.png
+- README.markdown
+- Rakefile
+- lib/orchestrated.rb
+- lib/orchestrated/base.rb
+- lib/orchestrated/completion.rb
+- lib/orchestrated/message_delivery.rb
+- lib/orchestrated/object.rb
+- lib/orchestrated/orchestration.rb
+- lib/orchestrated/version.rb
+- orchestrated.gemspec
+- spec/database.yml
+- spec/delayed_job_facade.rb
+- spec/spec_helper.rb
+- spec/spec_helper_methods.rb
+- spec/support/sample_classes/failer.rb
+- spec/support/sample_classes/first.rb
+- spec/support/sample_classes/second.rb
+- spec/unit/cancellation_spec.rb
+- spec/unit/completion_spec.rb
+- spec/unit/delayed_job_spec.rb
+- spec/unit/failure_spec.rb
+- spec/unit/orchestrated_spec.rb
+- spec/unit/rspec_spec.rb
+- spec/unit/static_analysis_spec.rb
+homepage: https://github.com/paydici/orchestration
+licenses:
+- MIT
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: Orchestrated is a workflow orchestration framework running on delayed_job
+  and active_record. In the style of delayed_job's 'delay', Orchestration lets you
+  'orchestrate' delivery of a message so that it will run only after others have been
+  delivered and processed.
+test_files:
+- spec/database.yml
+- spec/delayed_job_facade.rb
+- spec/spec_helper.rb
+- spec/spec_helper_methods.rb
+- spec/support/sample_classes/failer.rb
+- spec/support/sample_classes/first.rb
+- spec/support/sample_classes/second.rb
+- spec/unit/cancellation_spec.rb
+- spec/unit/completion_spec.rb
+- spec/unit/delayed_job_spec.rb
+- spec/unit/failure_spec.rb
+- spec/unit/orchestrated_spec.rb
+- spec/unit/rspec_spec.rb
+- spec/unit/static_analysis_spec.rb