orchestrated 0.0.3 → 0.0.4

data/README.markdown

@@ -60,20 +60,25 @@ Declaring ```acts_as_orchestrated``` on your class gives it two methods:
 * ```orchestrated```—call this to specify your workflow prerequisite, and designate a workflow step
 * ```orchestration```—call this in the context of a workflow step (execution) to access orchestration (and prerequisite) context
 
-After that you can orchestrate any method on such a class. Let's say you needed to download files from a remote system (a slow process), transform each one, and then load it into your system. Imagine you had had a Downloader class that knew how to download and an Xform class that knew how to transform the downloaded file and load it into your system. You might write an orchestration like this:
+After that you can orchestrate any method on such a class. Let's say you needed to download a couple files from remote systems (a slow process), merge their content and then load the results into your system. Imagine you had had a Downloader class that knew how to download and an Xform class that knew how to merge the content and load the results into your system. You might write an orchestration like this:
 
 ```ruby
 xform = Xform.new
 xform.orchestrated(
   xform.orchestrated(
-    Downloader.new.orchestrated.download(
-      :from=>'http://foo.bar.com/customer_fred', :to=>'fred_file'
+    Orchestrated::LastCompletion.new(
+      Downloader.new.orchestrated.download(
+        :from=>'http://fred.com/stuff', :to=>'fred_records'
+      ),
+      Downloader.new.orchestrated.download(
+        :from=>'http://sally.com/stuff', :to=>'sally_records'
+      )
     )
-  ).transform('fred_file', 'fred_file_processed')
-).load('fred_file_processed')
+  ).merge(['fred_records', 'sally_records'], 'combined_records')
+).load('combined_records')
 ```
 
-The next time you process a delayed job, the :download message will be delivered to a Downloader. After the download is complete, the next time a delayed job is processed, the :transform message will be delivered to an Xform object.
+The next time you process delayed jobs, the :download messages will be delivered to a couple Downloaders. After the last download completes, the next time a delayed job is processed, the :merge message will be delivered to an Xform object. And on it goes…
 
 What happened there? The pattern is:
 
@@ -108,8 +113,6 @@ An orchestration can be in one of six (6) states:
 
 ![Alt text](https://github.com/paydici/orchestrated/raw/master/Orchestrated::Orchestration_state.png 'Orchestration States')
 
-You'll never see an orchestration in the "new" state, it's for internal use in the framework. But all the others are interesting.
-
 When you create a new orchestration that is waiting on a prerequisite that is not complete yet, the orchestration will be in the "waiting" state. Some time later, if that prerequisite completes, then your orchestration will become "ready". A "ready" orchestration is automatically queued to run by the framework (via [delayed_job](https://github.com/collectiveidea/delayed_job)).
 
 A "ready" orchestration will use [delayed_job](https://github.com/collectiveidea/delayed_job) to delivery its (delayed) message. In the context of such a message delivery (inside your object method e.g. StatementGenerator#generate or StatementGenerator#render) you can rely on the ability to access the current Orchestration (context) object via the "orchestration" accessor.
@@ -144,3 +147,13 @@ Contributing
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)
 5. Create new Pull Request
+
+Future Work
+-----------
+
+Some possible avenues for exploration:
+
+* orchestrated option: :max_attempts to configure max_attempts on underlying delayed job instance
+* orchestrated option: :max_run_time to configure max_run_time on underlying delayed job instance
+* orchestrated options: :queue to specify a particular named queue for the underlying delayed job
+* some way to change the run_at recalculation for failed attempts (f(n) = 5 + n**4 is not always right and what's right varies by job)

data/lib/generators/orchestrated/templates/migration.rb

@@ -15,14 +15,15 @@ class CreateOrchestrated < ActiveRecord::Migration
       # the Rails model
       table.references :orchestration
     end
-    create_table :composited_completions do |table|
-      table.references :composite_completion
-      table.references :completion_expression
+    create_table :orchestration_dependencies do |table|
+      table.string     :state
+      table.references :dependent
+      table.references :prerequisite
     end
   end
 
   def self.down
-    drop_table :composited_completions
+    drop_table :orchestration_dependencies
     drop_table :completion_expressions
     drop_table :orchestrations
   end

data/lib/orchestrated/completion.rb

@@ -7,11 +7,22 @@ module Orchestrated
   class CompletionExpression < ActiveRecord::Base
     # I'd like to make this abstract, but Rails gets confused if I do
     # self.abstract_class = true
+    has_many :dependent_associations, :class_name => 'OrchestrationDependency', :foreign_key => 'prerequisite_id'
+    has_many :dependents, :through => :dependent_associations, :autosave => true
     def complete?; throw 'subclass must override!';end
     # for static analysis
     def always_complete?; throw 'subclass must override!';end
     def never_complete?; throw 'subclass must override!';end
     def canceled?; throw 'subclass must override!';end
+    def prerequisite_complete; throw 'subclass must override!';end
+    def notify_dependents_of_completion
+      # NB: we are notifying the Join Model here (it keeps track of status)
+      dependent_associations.each{|d| d.prerequisite_completed}
+    end
+    def notify_dependents_of_cancellation
+      # NB: we are notifying the Join Model here (it keeps track of status)
+      dependent_associations.each{|d| d.prerequisite_canceled}
+    end
   end
   class Complete < CompletionExpression
     def complete?; true; end
@@ -28,31 +39,43 @@ module Orchestrated
   end
   class CompositeCompletion < CompletionExpression
     # self.abstract_class = true
-    has_many :composited_completions
-    has_many :completion_expressions, :through => :composited_completions, :source => :completion_expression
-    def +(c); self << c; end # synonym
+    has_many :prerequisite_associations, :class_name => 'OrchestrationDependency', :foreign_key => 'dependent_id'
+    has_many :prerequisites, :through => :prerequisite_associations, :autosave => true
+    def +(c); self << c; end # synonymc
   end
   class LastCompletion < CompositeCompletion
-    def complete?; completion_expressions.all?(&:complete?); end
-    def always_complete?; completion_expressions.empty?; end
-    def never_complete?; completion_expressions.any?(&:never_complete?); end
-    def canceled?; completion_expressions.any?(&:canceled?); end
+    def complete?; prerequisites.all?(&:complete?); end
+    def always_complete?; prerequisites.empty?; end
+    def never_complete?; prerequisites.any?(&:never_complete?); end
+    def canceled?; prerequisites.any?(&:canceled?); end
     def <<(c)
-      completion_expressions << c unless c.always_complete?
+      prerequisites << c unless c.always_complete?
       self
     end
+    def prerequisite_complete
+      notify_dependents_of_completion unless prerequisite_associations.without_states('complete').exists?
+    end
+    def prerequisite_canceled
+      notify_dependents_of_cancellation
+    end
   end
   class FirstCompletion < CompositeCompletion
-    def complete?; completion_expressions.any?(&:complete?); end
-    def always_complete?; completion_expressions.any?(&:always_complete?); end
-    def never_complete?; completion_expressions.empty?; end
-    def canceled?; completion_expressions.all?(&:canceled?); end
+    def complete?; prerequisites.any?(&:complete?); end
+    def always_complete?; prerequisites.any?(&:always_complete?); end
+    def never_complete?; prerequisites.empty?; end
+    def canceled?; prerequisites.all?(&:canceled?); end
     def <<(c)
-      completion_expressions << c unless c.never_complete?
+      prerequisites << c unless c.never_complete?
       self
     end
+    def prerequisite_complete
+      notify_dependents_of_completion
+    end
+    def prerequisite_canceled
+      notify_dependents_of_completion unless prerequisite_associations.without_states('canceled').exists?
+    end
   end
-  class OrchestrationCompletion < CompletionExpression
+  class OrchestrationCompletionShim < CompletionExpression
     # Arguably, it is "bad" to make this class derive
     # from CompletionExpression since doing so introduces
     # the orchestration_id into the table (that constitutes
@@ -61,12 +84,28 @@ module Orchestrated
     # understand joins when computing dependents at runtime.
     belongs_to :orchestration
     validates_presence_of :orchestration_id
+  end
+  # wraps an Orchestration and makes it usable as a completion expression
+  class OrchestrationCompletion < OrchestrationCompletionShim
     delegate :complete?, :canceled?, :cancel!, :to => :orchestration
     def always_complete?; false; end
     def never_complete?; false; end
+    def prerequisite_complete
+      notify_dependents_of_completion
+    end
+    def prerequisite_canceled
+      notify_dependents_of_cancellation
+    end
   end
-  class CompositedCompletion < ActiveRecord::Base
-    belongs_to :composite_completion
-    belongs_to :completion_expression
+  # registers an Orchestration's interest in a completion expression
+  class OrchestrationInterest < OrchestrationCompletionShim
+    has_one :prerequisite_association, :class_name => 'OrchestrationDependency', :foreign_key => 'dependent_id'
+    has_one :prerequisite, :through => :prerequisite_association, :autosave => true
+    def prerequisite_complete
+      orchestration.prerequisite_complete
+    end
+    def prerequisite_canceled
+      orchestration.prerequisite_canceled
+    end
   end
 end

data/lib/orchestrated/dependency.rb

@@ -0,0 +1,27 @@
+require 'state_machine'
+
+module Orchestrated
+  class OrchestrationDependency < ActiveRecord::Base
+    # TODO: figure out why Rails thinks I'm mass-assigning this over in Orchestration when I'm not really!
+    attr_accessible :prerequisite_id
+    belongs_to :dependent, :class_name => 'CompletionExpression'
+    belongs_to :prerequisite, :class_name => 'CompletionExpression'
+    state_machine :initial => :incomplete do
+      state :incomplete
+      state :complete
+      state :canceled
+      event :prerequisite_completed do
+        transition :incomplete => :complete
+      end
+      event :prerequisite_canceled do
+        transition :incomplete => :canceled
+      end
+      after_transition any => :complete do |orchestration, transition|
+        orchestration.dependent.prerequisite_complete
+      end
+      after_transition any => :canceled do |orchestration, transition|
+        orchestration.dependent.prerequisite_canceled
+      end
+    end
+  end
+end

data/lib/orchestrated/orchestration.rb

@@ -10,14 +10,13 @@ module Orchestrated
 
     serialize :handler
 
-    belongs_to :prerequisite, :class_name => 'CompletionExpression'
-    belongs_to :delayed_job, :polymorphic => true # loose-ish coupling with delayed_job
+    has_one :prerequisite, :class_name => 'OrchestrationInterest'
+    has_one :dependent, :class_name => 'OrchestrationCompletion'
 
-    has_many :orchestration_completions
+    belongs_to :delayed_job, :polymorphic => true # loose-ish coupling with delayed_job
 
     complete_states = [:succeeded, :failed]
-    state_machine :initial => :new do
-      state :new
+    state_machine :initial => :waiting do
       state :waiting
       state :ready
       state :succeeded
@@ -36,10 +35,12 @@ module Orchestrated
         end
       end
 
-      event :prerequisite_changed do
-        transition [:new, :waiting] => :ready, :if => lambda {|orchestration| orchestration.prerequisite.complete?}
-        transition [:ready, :waiting] => :canceled, :if => lambda {|orchestration| orchestration.prerequisite.canceled?}
-        transition :new => :waiting # otherwise
+      event :prerequisite_complete do
+        transition :waiting => :ready
+      end
+
+      event :prerequisite_canceled do
+        transition [:waiting, :ready] => :canceled
       end
 
       event :message_delivery_succeeded do
@@ -63,18 +64,11 @@ module Orchestrated
       end
 
       after_transition any => complete_states do |orchestration, transition|
-        # completion may make other orchestrations ready to run…
-        # TODO: this is a prime target for benchmarking
-        (Orchestration.with_state('waiting').all - [orchestration]).each do |other|
-          other.prerequisite_changed
-        end
+        orchestration.dependent.prerequisite_complete
       end
 
       after_transition [:ready, :waiting] => :canceled do |orchestration, transition|
-        # cancellation may cancel other orchestrations
-        (Orchestration.with_states(:ready, :waiting).all - [orchestration]).each do |other|
-          other.prerequisite_changed
-        end
+        orchestration.dependent.prerequisite_canceled
       end
 
     end
@@ -88,9 +82,16 @@ module Orchestrated
         # wee! static analysis FTW!
         raise 'prerequisite can never be complete' if prerequisite.never_complete?
 
-        # saves object as side effect of this assignment
-        # also moves orchestration to :ready state
-        orchestration.prerequisite  = prerequisite
+        prerequisite.save!
+        orchestration.save!
+        interest = OrchestrationInterest.new.tap do |interest|
+          interest.prerequisite = prerequisite
+          interest.orchestration = orchestration
+        end
+        interest.save!
+
+        # prime the pump for a constant prerequisite
+        orchestration.prerequisite_complete! if prerequisite.complete?
      end
     end
 
@@ -102,11 +103,5 @@ module Orchestrated
       delayed_job.destroy# if DelayedJob.exists?(delayed_job_id)
     end
 
-    alias_method :prerequisite_old_equals, :prerequisite=
-    def prerequisite=(*args)
-      prerequisite_old_equals(*args)
-      prerequisite_changed
-    end
-
   end
 end

data/lib/orchestrated/version.rb

@@ -1,3 +1,3 @@
 module Orchestrated
-  VERSION = "0.0.3"
+  VERSION = "0.0.4"
 end

data/lib/orchestrated.rb

@@ -1,6 +1,7 @@
 require "orchestrated/version"
 require 'orchestrated/base'
 require 'orchestrated/completion'
+require 'orchestrated/dependency'
 require 'orchestrated/message_delivery'
 require 'orchestrated/orchestration'
 require 'orchestrated/object'

data/spec/spec_helper.rb

@@ -58,9 +58,10 @@ ActiveRecord::Schema.define do
       # the Rails model
       table.references :orchestration
     end
-    create_table :composited_completions do |table|
-      table.references :composite_completion
-      table.references :completion_expression
+    create_table :orchestration_dependencies do |table|
+      table.string     :state
+      table.references :dependent
+      table.references :prerequisite
     end
 end
 

data/spec/unit/completion_spec.rb

@@ -4,7 +4,7 @@ require 'orchestrated'
 
 shared_examples_for 'literally complete' do
   it 'should immediately enqueue the dependent orchestration' do
-    expect(DJ.job_count).to be(1)
+    expect(DJ.job_count).to eq(1)
   end
   it 'should cause the dependent orchestration to run immediately' do
     First.any_instance.should_receive(:do_first_thing)
@@ -31,7 +31,7 @@ describe Orchestrated::CompletionExpression do
   context 'OrchestrationCompletion' do
     before(:each){Second.new.orchestrated( First.new.orchestrated.do_first_thing(3)).do_second_thing(4)}
     it 'should block second orchestration until after first runs' do
-      expect(DJ.job_count).to be(1)
+      expect(DJ.job_count).to eq(1)
     end
     it 'should run second orchestration after first is complete' do
       Second.any_instance.should_receive(:do_second_thing)
@@ -46,7 +46,7 @@ describe Orchestrated::CompletionExpression do
           ).do_second_thing(5)
       end
       it 'should immediately enqueue the dependent orchestration' do
-        expect(DJ.job_count).to be(1)
+        expect(DJ.job_count).to eq(1)
       end
     end
     context 'given two OrchestrationCompletions' do
@@ -57,9 +57,9 @@ describe Orchestrated::CompletionExpression do
           ).do_second_thing(5)
       end
       it 'should enqueue the dependent orchestration as soon as the first prerequisite completes' do
-        expect(DJ.job_count).to be(2)
+        expect(DJ.job_count).to eq(2)
         DJ.work(1)
-        expect(DJ.job_count).to be(2)
+        expect(DJ.job_count).to eq(2)
       end
       it 'should cause the dependent orchestration to run eventually' do
         Second.any_instance.should_receive(:do_second_thing).with(5)
@@ -80,9 +80,9 @@ describe Orchestrated::CompletionExpression do
           ).do_second_thing(5)
       end
       it 'should not enqueue the dependent orchestration as soon as the first prerequisite completes' do
-        expect(DJ.job_count).to be(2)
+        expect(DJ.job_count).to eq(2)
         DJ.work(1)
-        expect(DJ.job_count).to be(1)
+        expect(DJ.job_count).to eq(1)
       end
       it 'should not run the dependent orchestration as soon as the first prerequisite completes' do
         Second.any_instance.should_not_receive(:do_second_thing)

data/spec/unit/delayed_job_spec.rb

@@ -52,7 +52,7 @@ describe Delayed::Job do
       job.custom_action
     end
     it 'should start with an empty queue' do
-      expect(DJ.job_count).to be(0)
+      expect(DJ.job_count).to eq(0)
     end
     it 'should enqueue a job' do
       expect {

data/spec/unit/failure_spec.rb

@@ -12,10 +12,10 @@ describe 'failure' do
         DJ.work(1)
       end
       it 'should leave the orchestration in the ready state' do
-        expect(Orchestrated::Orchestration.with_state('ready').count).to be(1)
+        expect(Orchestrated::Orchestration.with_state('ready').count).to eq(1)
       end
       it 'should leave the orchestration in the run queue' do
-        expect(DJ.job_count).to be(1)
+        expect(DJ.job_count).to eq(1)
       end
       context 'on first retry' do
         it 'should retry with same arguments' do
@@ -29,7 +29,7 @@ describe 'failure' do
         DJ.work_now(DJ.max_attempts)
       end
       it 'should leave the orchestration in the failed state' do
-        expect(Orchestrated::Orchestration.with_state('failed').count).to be(1)
+        expect(Orchestrated::Orchestration.with_state('failed').count).to eq(1)
       end
       context 'on first subsequent retry' do
         it 'should never deliver the orchestrated message again' do

data/spec/unit/rspec_spec.rb

@@ -14,7 +14,7 @@ describe 'rspec' do
         block = block_arg
       }
       Foo.new.bump(1){|hello| puts hello}
-      expect(x).to be(1)
+      expect(x).to eq(1)
       expect(block).to be_kind_of(Proc)
     end
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: orchestrated
 version: !ruby/object:Gem::Version
-  version: 0.0.3
+  version: 0.0.4
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-01-03 00:00:00.000000000 Z
+date: 2013-01-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: delayed_job_active_record
@@ -189,6 +189,7 @@ files:
 - lib/orchestrated.rb
 - lib/orchestrated/base.rb
 - lib/orchestrated/completion.rb
+- lib/orchestrated/dependency.rb
 - lib/orchestrated/message_delivery.rb
 - lib/orchestrated/object.rb
 - lib/orchestrated/orchestration.rb