orchestrated 0.0.7 → 0.0.8

data/README.markdown

@@ -1,5 +1,5 @@
-Orchestrated
-============
+[Orchestrated](https://github.com/paydici/orchestrated)
+=======================================================
 
 The [delayed_job](https://github.com/collectiveidea/delayed_job) Ruby Gem provides a restartable queuing system for Ruby. It implements an elegant API for delaying execution of any Ruby object method invocation. Not only is the message delivery delayed in time, it is potentially shifted in space too. By shifting in space, i.e. running in a different virtual machine, possibly on a separate computer, multiple CPUs can be brought to bear on a computing problem.
 
@@ -10,13 +10,13 @@ Queuing works well for simple, independent tasks. By simple we mean the task can
 1. pipelined (multi-step) generation of a complex PDF document
 2. an extract/transfer/load (ETL) job that must acquire data from source systems, transform it and load it into the target system
 
-If we would like to scale these compound operations, breaking them into smaller parts, and managing the execution of those parts across many computers, we need an "orchestrator". This project implements just such a framework, called "Orchestrated".
+If we would like to scale these compound operations, breaking them into smaller parts, and managing the execution of those parts across many computers, we need an "orchestrator". This project implements just such a framework, called "[Orchestrated](https://github.com/paydici/orchestrated)".
 
-Orchestrated introduces the ```acts_as_orchestrated``` Object class method. When invoked on your class, this will define the ```orchestrated``` instance method. You use ```orchestrated``` in a mannner similar to [delayed_job](https://github.com/collectiveidea/delayed_job)'s ```delay```—the difference being that ```orchestrated``` takes a parameter that lets you specify dependencies between your jobs.
+[Orchestrated](https://github.com/paydici/orchestrated) introduces the ```acts_as_orchestrated``` Object class method. When invoked on your class, this will define the ```orchestrate``` instance method. You use ```orchestrate``` in a mannner similar to [delayed_job](https://github.com/collectiveidea/delayed_job)'s ```delay```—the difference being that ```orchestrate``` takes a parameter that lets you specify dependencies between your jobs.
 
 The reason we refer to [delayed_job](https://github.com/collectiveidea/delayed_job) as a restartable queueing system is because, even if computers (database host, worker hosts) in the cluster crash, the work on the queues progresses. If no worker is servicing a particular queue, then work accumulates there. Once workers are available, they consume the jobs. This is a resilient architecture.
 
-With Orchestrated you can create restartable workflows, a workflow consisting of one or more dependent, queueable, tasks. This means that your workflows will continue to make progress even in the face of database and (queue) worker crashes.
+With [Orchestrated](https://github.com/paydici/orchestrated) you can create restartable workflows, a workflow consisting of one or more dependent, queueable, tasks. This means that your workflows will continue to make progress even in the face of database and (queue) worker crashes.
 
 In summary, orchestrated workflows running atop [active_record](https://github.com/rails/rails/tree/master/activerecord) and [delayed_job](https://github.com/collectiveidea/delayed_job) have these characteristics:
 
@@ -52,13 +52,13 @@ If you do not already have [delayed_job](https://github.com/collectiveidea/delay
 The API
 -------
 
-To orchestrate (methods) on your own classes you simply call ```acts_as_orchestrated``` in the class definition. Declaring ```acts_as_orchestrated``` on your class defines the ```orchestrated``` method:
+To orchestrate (methods) on your own classes you simply call ```acts_as_orchestrated``` in the class definition. Declaring ```acts_as_orchestrated``` on your class defines the ```orchestrate``` method:
 
-* ```orchestrated```—call this to specify your workflow prerequisite, and designate a workflow step
+* ```orchestrate```—call this to specify your workflow prerequisite, and designate a workflow step
 
-Use ```orchestrated``` to orchestrate any method on your class.
+Use ```orchestrate``` to orchestrate any method on your class.
 
-Let's say for example you needed to download a couple files from remote systems (a slow process), merge their content and then load the results into your system. Imagine you have a ```Downloader``` class that knows how to download and an ```Xform``` class that knows how to merge the content and load the results into your system. Your ```Xform``` class might look something like this:
+Let's say for example you needed to download a couple files from remote systems (a slow process), merge their content and then load the results into your system. This sort of workflow is sometimes referred to as extract/transfer/load or ETL. Imagine you have a ```Downloader``` class that knows how to download and an ```Xform``` class that knows how to merge the content and load the results into your system. Your ```Xform``` class might look something like this:
 
 ```ruby
 class Xform
@@ -80,13 +80,13 @@ You might write an orchestration like this:
 
 ```ruby
 xform = Xform.new
-xform.orchestrated(
-  xform.orchestrated(
+xform.orchestrate(
+  xform.orchestrate(
     Orchestrated::LastCompletion.new(
-      Downloader.new.orchestrated.download(
+      Downloader.new.orchestrate.download(
         :from=>'http://fred.com/stuff', :to=>'fred_records'
       ),
-      Downloader.new.orchestrated.download(
+      Downloader.new.orchestrate.download(
         :from=>'http://sally.com/stuff', :to=>'sally_records'
       )
     )
@@ -99,24 +99,26 @@ The next time you process delayed jobs, the ```download``` messages will be deli
 What happened there? The pattern is:
 
 1. create an orchestrated object (instantiate it)
-2. call ```orchestrated``` on it: this returns a *magic proxy* object that can respond to any of the messages your object can respond to
+2. call ```orchestrate``` on it: this returns a *magic proxy* object that can respond to any of the messages your object can respond to
 3. send any message to the *magic proxy* (returned in the second step) and the framework will delay delivery of that message and immediately return a "completion expression" you can use as a prerequisite for other orchestrations
 4. (optionally) use the "completion expression" returned in (3) as a prerequisite for other orchestrations
 
 Now the messages you can send in (3) can be anything that your object can respond to. The message will be remembered by the framework and "replayed" (on a new instance of your object) somewhere on the network (later).
 
-Not accidentally, this is similar to the way [delayed_job](https://github.com/collectiveidea/delayed_job)'s delay method works. Under the covers, orchestrated is conspiring with [delayed_job](https://github.com/collectiveidea/delayed_job) when it comes time to actually execute a workflow step. Before that time though, orchestrated keeps track of everything.
+Not accidentally, this is similar to the way [delayed_job](https://github.com/collectiveidea/delayed_job)'s delay method works. Under the covers, [Orchestrated](https://github.com/paydici/orchestrated) is conspiring with [delayed_job](https://github.com/collectiveidea/delayed_job) when it comes time to actually execute a workflow step. Before that time though, [Orchestrated](https://github.com/paydici/orchestrated) keeps track of everything.
 
 Key Concept: Prerequisites (Completion Expressions)
 ---------------------------------------------------
 
-Unlike [delayed_job](https://github.com/collectiveidea/delayed_job) ```delay```, the orchestrated ```orchestrated``` method takes an optional parameter: the prerequisite. The prerequisite determines when your workflow step is ready to run.
+Unlike [delayed_job](https://github.com/collectiveidea/delayed_job) ```delay```, the orchestrated ```orchestrate``` method takes an optional parameter: the prerequisite. The prerequisite determines when your workflow step is ready to run.
 
-The return value from messaging the *magic proxy* is itself a ready-to-use prerequisite. You saw this in the statement generation example above. The result of the first call to ```orchestrated``` calls (to "download") were sent as an argument to the third ("merge"). In this way, the "merge" workflow step was suspended until after the "download"s finished. You may have also noticed from that example that if you specify no prerequisite then the step will be ready to run immediately, as was the case for the "download" calls).
+The return value from messaging the *magic proxy* is itself a ready-to-use prerequisite. You saw this in the ETL example above. The result of the first call to ```orchestrate``` calls (to ```download```) were sent as an argument to the third (```merge```). In this way, the ```merge``` workflow step was suspended until after the ```download```s finished.
+
+You may have also noticed from that example that if you specify no prerequisite then the step will be ready to run immediately, as was the case for the ```download``` calls). If calling ```orchestrate``` with no parameters makes the step ready to run immediately then why should we bother to call it at all? Why not just call the method directly? The answer is that by calling ```orchestrate``` we are submitting the step to the underlying queueing system, enabling the step to be run on other resources (computers). Had we called the ```download``` directly it would have blocked the Ruby thread and would not have taken advantage of (potentially many) ```delayed_job``` job workers.
 
 Users of the framework deal directly with three kinds of prerequisite or "completion expression":
 
-1. ```OrchestrationCompletion```—returned messages to the *magic proxy*, complete when its associated orchestration is complete
+1. ```OrchestrationCompletion```—returned from any message to a *magic proxy*: complete when its associated orchestration is complete
 2. ```FirstCompletion```—aggregates other completions: complete after the first one completes
 3. ```LastCompletion```—aggregates other completions: complete after all of them are complete
 
@@ -135,7 +137,7 @@ A "ready" orchestration will use [delayed_job](https://github.com/collectiveidea
 
 After your workflow step executes, the orchestration moves into either the "succeeded" or "failed" state.
 
-When an orchestration is "ready" or "waiting" it may be canceled by sending it the ```cancel!``` message (i.e. a ```cancel!``` message to the ```OrchestrationCompletion```. This moves it to the orchestration to the "canceled" state and prevents delivery of the orchestrated message (in the future).
+When an orchestration is "ready" or "waiting" it may be canceled by sending it the ```cancel!``` message (i.e. a ```cancel!``` message to the ```OrchestrationCompletion```). This moves the orchestration to the "canceled" state and prevents subsequent delivery of the orchestrated message.
 
 It is important to understand that both of the states: "succeeded" and "failed" are part of a "super-state": "complete". When an orchestration is in either of those two states, it will return ```true``` in response to the ```complete?``` message.
 
@@ -144,7 +146,7 @@ It is not just successful completion of orchestrated methods that causes depende
 Failure (An Option)
 -------------------
 
-Orchestration is built atop [delayed_job](https://github.com/collectiveidea/delayed_job) and borrows [delayed_job](https://github.com/collectiveidea/delayed_job)'s failure semantics. Neither framework imposes any special constraints on the (delayed or orchestrated) methods. In particular, there are no special return values to signal "failure". Orchestration adopts [delayed_job](https://github.com/collectiveidea/delayed_job)'s semantics for failure detection: a method that raises an exception has failed. After a certain number of retries (configurable in [delayed_job](https://github.com/collectiveidea/delayed_job)) the jobs is deemed permanently failed. When that happens, the corresponding orchestration is marked "failed".
+Since Orchestration is built atop [delayed_job](https://github.com/collectiveidea/delayed_job) and borrows [delayed_job](https://github.com/collectiveidea/delayed_job)'s failure semantics. Neither framework imposes any special constraints on the (delayed or orchestrated) methods. In particular, there are no special return values to signal "failure". Orchestration adopts [delayed_job](https://github.com/collectiveidea/delayed_job)'s semantics for failure detection: a method that raises an exception has failed. After a certain number of retries (configurable in [delayed_job](https://github.com/collectiveidea/delayed_job)) the jobs is deemed permanently failed. When that happens, the corresponding orchestration is marked "failed". Until all the retries have been attempted, the orchestration remains in the "ready" state (as it was before the first failed attempt).
 
 See the failure_spec if you'd like to understand more.
 
@@ -169,7 +171,14 @@ Future Work
 
 Some possible avenues for exploration:
 
-* orchestrated option: :max_attempts to configure max_attempts on underlying delayed job instance
-* orchestrated option: :max_run_time to configure max_run_time on underlying delayed job instance
-* orchestrated options: :queue to specify a particular named queue for the underlying delayed job
+* orchestrate option: :max_attempts to configure max_attempts on underlying delayed job instance
+* orchestrate option: :max_run_time to configure max_run_time on underlying delayed job instance
+* orchestrate options: :queue to specify a particular named queue for the underlying delayed job
 * some way to change the run_at recalculation for failed attempts (f(n) = 5 + n**4 is not always right and what's right varies by job)
+
+License
+-------
+
+Copyright © 2013 Paydici Inc. Distributed under the MIT License. See [LICENSE.txt](https://github.com/paydici/orchestrated/blob/master/LICENSE.txt) for further details.
+
+Contains code originally from [delayed_job](https://github.com/collectiveidea/delayed_job) Copyright © 2005 Tobias Luetke, [Ruby on Rails](https://github.com/rails/rails), and [Ick](https://github.com/raganwald-deprecated/ick); all of which are also under the MIT License.

data/lib/orchestrated/base.rb

@@ -11,25 +11,28 @@ module Orchestrated
     end
   end
 
-  class << self
-    #snarfed from Ruby On Rails
-    def underscore(camel_cased_word)
-      camel_cased_word.to_s.gsub(/::/, '/').
-        gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
-        tr("-", "_").
-        downcase
-    end
-    def belongs_to clazz
-      # borrowed from Ick
-      method_name = self.underscore(self.name.split('::')[-1])
-      unless clazz.method_defined?(method_name)
-        clazz.class_eval "
-          def #{method_name}(prerequisite=Complete.new)
-            raise ArgumentError.new('orchestrate does not take a block') if block_given?
-            raise ArgumentError.new(%[cannot use \#{prerequisite.class.name} as a prerequisite]) unless
-              prerequisite.kind_of?(CompletionExpression)
-            Proxy.new(prerequisite, self)
-          end"
+  # because this class is called "Orchestrate", the method defined (on Object) will be "orchestrate"
+  class Orchestrate
+    class << self
+      #snarfed from Ruby On Rails
+      def underscore(camel_cased_word)
+        camel_cased_word.to_s.gsub(/::/, '/').
+          gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
+          tr("-", "_").
+          downcase
+      end
+      def belongs_to clazz
+        # borrowed from Ick
+        method_name = self.underscore(self.name.split('::')[-1])
+        unless clazz.method_defined?(method_name)
+          clazz.class_eval "
+            def #{method_name}(prerequisite=Complete.new)
+              raise ArgumentError.new('orchestrate does not take a block') if block_given?
+              raise ArgumentError.new(%[cannot use \#{prerequisite.class.name} as a prerequisite]) unless
+                prerequisite.kind_of?(CompletionExpression)
+              Proxy.new(prerequisite, self)
+            end"
+        end
       end
     end
   end

data/lib/orchestrated/object.rb

@@ -8,7 +8,7 @@ module Orchestrated
   class ::Object
     class << self
       def acts_as_orchestrated
-        Orchestrated.belongs_to self # define "orchestrated instance method"
+        Orchestrate.belongs_to self # define "orchestrated instance method"
         include InstanceMethods
       end
     end

data/lib/orchestrated/version.rb

@@ -1,3 +1,3 @@
 module Orchestrated
-  VERSION = "0.0.7"
+  VERSION = "0.0.8"
 end

data/spec/unit/cancellation_spec.rb

@@ -34,7 +34,7 @@ describe 'cancellation' do
   def dependent;@dependent;end
   context 'directly on an orchestration' do
     before(:each) do
-      @first_prerequisite = @dependent = First.new.orchestrated.do_first_thing(1)
+      @first_prerequisite = @dependent = First.new.orchestrate.do_first_thing(1)
     end
     it 'should be ready' do
       expect(@dependent.orchestration.state).to eq('ready')
@@ -71,16 +71,16 @@ describe 'cancellation' do
   end
   context 'of an orchestration that is depended on directly' do
     before(:each) do
-      @dependent = Second.new.orchestrated( @first_prerequisite = First.new.orchestrated.do_first_thing(1)).do_second_thing(2)
+      @dependent = Second.new.orchestrate( @first_prerequisite = First.new.orchestrate.do_first_thing(1)).do_second_thing(2)
     end
     include_context 'cancelling first prerequisite'
     it_should_behave_like 'cancellation:'
   end
   context 'of an orchestration that is depended on through a LastCompletion' do
     before(:each) do
-      @dependent = Second.new.orchestrated(
+      @dependent = Second.new.orchestrate(
         Orchestrated::LastCompletion.new <<
-          (@first_prerequisite = First.new.orchestrated.do_first_thing(1))
+          (@first_prerequisite = First.new.orchestrate.do_first_thing(1))
         ).do_second_thing(2)
     end
     include_context 'cancelling first prerequisite'
@@ -88,10 +88,10 @@ describe 'cancellation' do
   end
   context 'of an orchestration that is depended on through a FirstCompletion with two prerequisites' do
     before(:each) do
-      @dependent = Second.new.orchestrated(
+      @dependent = Second.new.orchestrate(
         Orchestrated::FirstCompletion.new <<
-          (@first_prerequisite = First.new.orchestrated.do_first_thing(3)) <<
-          (@last_prerequisite = First.new.orchestrated.do_first_thing(1))
+          (@first_prerequisite = First.new.orchestrate.do_first_thing(3)) <<
+          (@last_prerequisite = First.new.orchestrate.do_first_thing(1))
         ).do_second_thing(2)
     end
     context 'after first prerequisite is canceled' do

data/spec/unit/completion_spec.rb

@@ -15,21 +15,21 @@ end
 describe Orchestrated::CompletionExpression do
   context 'Complete' do
     context 'implicitly specified' do
-      before(:each){ First.new.orchestrated.do_first_thing(12) }
+      before(:each){ First.new.orchestrate.do_first_thing(12) }
       it_should_behave_like 'literally complete'
     end
     context 'explicitly specified' do
-      before(:each){ First.new.orchestrated(Orchestrated::Complete.new).do_first_thing(12) }
+      before(:each){ First.new.orchestrate(Orchestrated::Complete.new).do_first_thing(12) }
       it_should_behave_like 'literally complete'
     end
   end
   context 'Incomplete' do
     it 'should immediately raise an error' do
-      expect{First.new.orchestrated(Orchestrated::Incomplete.new).do_first_thing(12)}.to raise_error(ArgumentError)
+      expect{First.new.orchestrate(Orchestrated::Incomplete.new).do_first_thing(12)}.to raise_error(ArgumentError)
     end
   end
   context 'OrchestrationCompletion' do
-    before(:each){Second.new.orchestrated( First.new.orchestrated.do_first_thing(3)).do_second_thing(4)}
+    before(:each){Second.new.orchestrate( First.new.orchestrate.do_first_thing(3)).do_second_thing(4)}
     it 'should block second orchestration until after first runs' do
       expect(DJ.job_count).to eq(1)
     end
@@ -41,7 +41,7 @@ describe Orchestrated::CompletionExpression do
   context 'FirstCompletion' do
     context 'given a (literal) Complete' do
       before(:each) do
-        Second.new.orchestrated( Orchestrated::FirstCompletion.new <<
+        Second.new.orchestrate( Orchestrated::FirstCompletion.new <<
           Orchestrated::Complete.new
           ).do_second_thing(5)
       end
@@ -51,9 +51,9 @@ describe Orchestrated::CompletionExpression do
     end
     context 'given two OrchestrationCompletions' do
       before(:each) do
-        Second.new.orchestrated( Orchestrated::FirstCompletion.new <<
-          First.new.orchestrated.do_first_thing(3) <<
-          First.new.orchestrated.do_first_thing(4)
+        Second.new.orchestrate( Orchestrated::FirstCompletion.new <<
+          First.new.orchestrate.do_first_thing(3) <<
+          First.new.orchestrate.do_first_thing(4)
           ).do_second_thing(5)
       end
       it 'should enqueue the dependent orchestration as soon as the first prerequisite completes' do
@@ -74,9 +74,9 @@ describe Orchestrated::CompletionExpression do
   context 'LastCompletion' do
     context 'given two OrchestrationCompletions' do
       before(:each) do
-        Second.new.orchestrated( Orchestrated::LastCompletion.new <<
-          First.new.orchestrated.do_first_thing(3) <<
-          First.new.orchestrated.do_first_thing(4)
+        Second.new.orchestrate( Orchestrated::LastCompletion.new <<
+          First.new.orchestrate.do_first_thing(3) <<
+          First.new.orchestrate.do_first_thing(4)
           ).do_second_thing(5)
       end
       it 'should not enqueue the dependent orchestration as soon as the first prerequisite completes' do

data/spec/unit/failure_spec.rb

@@ -5,7 +5,7 @@ require 'orchestrated'
 describe 'failure' do
   context 'orchestrating a method that always fails' do
     before(:each) do
-      Failer.new.orchestrated.always_fail('important stuff')
+      Failer.new.orchestrate.always_fail('important stuff')
     end
     context 'after first exception from orchestrated method' do
       before(:each) do

data/spec/unit/orchestrated_spec.rb

@@ -14,14 +14,14 @@ end
 
 describe Orchestrated do
   context 'initializing' do
-    it 'should not define orchestrated on Object' do
-      expect(Object.public_method_defined?(:orchestrated)).to be_false
+    it 'should not define orchestrate on Object' do
+      expect(Object.public_method_defined?(:orchestrate)).to be_false
     end
-    it 'should not define orchestrated on ActiveRecord::Base' do
-      expect(ActiveRecord::Base.public_method_defined?(:orchestrated)).to be_false
+    it 'should not define orchestrate on ActiveRecord::Base' do
+      expect(ActiveRecord::Base.public_method_defined?(:orchestrate)).to be_false
     end
-    it 'should define orchestrated on First' do
-      expect(First.public_method_defined?(:orchestrated)).to be_true
+    it 'should define orchestrate on First' do
+      expect(First.public_method_defined?(:orchestrate)).to be_true
     end
   end
   context 'a new object' do
@@ -76,7 +76,7 @@ describe Orchestrated do
       end
     end
     context 'orchestrating with no prerequisites' do
-      before(:each){@result = f.orchestrated.do_first_thing(2)}
+      before(:each){@result = f.orchestrate.do_first_thing(2)}
       after(:each){DJ.clear_all_jobs}
       it 'should not immediately invoke an orchestrated method' do
         First.any_instance.should_not_receive(:do_first_thing)
@@ -91,7 +91,7 @@ describe Orchestrated do
   end
   context 'invocation' do
     before(:each) do
-      First.new.orchestrated.do_first_thing(1)
+      First.new.orchestrate.do_first_thing(1)
     end
     it 'should have access to Orchestration' do
       First.any_instance.should_receive(:orchestration=).with(kind_of(Orchestrated::Orchestration))

data/spec/unit/static_analysis_spec.rb

@@ -8,13 +8,13 @@ describe 'performing static analysis' do
     context 'that is empty' do
       # chose this behavior to align with Ruby Enumerable#any?
       it 'should raise an error since it can never be complete' do
-        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error(ArgumentError)
+        expect{Second.new.orchestrate(completion).do_second_thing(5)}.to raise_error(ArgumentError)
       end
     end
     context 'that contains only (static) Incompletes' do
       before(:each){completion<<Orchestrated::Incomplete.new}
       it 'should raise an error since it can never be complete' do
-        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error(ArgumentError)
+        expect{Second.new.orchestrate(completion).do_second_thing(5)}.to raise_error(ArgumentError)
       end
     end
     context 'that directly containins a (static) Complete' do
@@ -41,7 +41,7 @@ describe 'performing static analysis' do
     context 'that directly contains a (static) Incomplete' do
       before(:each){completion<<Orchestrated::Incomplete.new}
       it 'should raise an error since it can never be complete' do
-        expect{Second.new.orchestrated(completion).do_second_thing(5)}.to raise_error(ArgumentError)
+        expect{Second.new.orchestrate(completion).do_second_thing(5)}.to raise_error(ArgumentError)
       end
     end
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: orchestrated
 version: !ruby/object:Gem::Version
-  version: 0.0.7
+  version: 0.0.8
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-01-07 00:00:00.000000000 Z
+date: 2013-02-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: delayed_job_active_record