burstflow 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 435e3f0b3dba1c654d192d3f5f4cb1687bfe55e6c60f03e6fcb4643946603f0a
-  data.tar.gz: e5390d5f58fb4a2e3503516f0c0744bce0cac68411edf56477e82d84a9f9ed87
+  metadata.gz: 4d68ef2fa6c3af3c066b65ae6213d3ac96216ea1eb66b849bb979759ed25c67b
+  data.tar.gz: b7868f4aeca3863996bba8f256fa72b79f244375bc6c82c301386ca88b7f66f8
 SHA512:
-  metadata.gz: fcb37c6b28837184136c25333cfaa3bd403bb953b9aa8dc8fe5971d499013650664444b5415aa50e2530218a88d07107e5032f90cc3ef3cd5114bd8593752669
-  data.tar.gz: ae739a9ab5a48b10512c3461e4720e54e564722192fca3aaf36123a48ed3a1c36de28483d19fd975659b2586637120be3dd32068f62e5bb2a294a1f6c8177e33
+  metadata.gz: 59e06e384d30bec993c880bce050f41dbc017cf1b0814855dca2d538ea270a79d9a9cb6a8f4601b47848f810cc768508e4a0c7aa5eade69717c194dc52b07526
+  data.tar.gz: dfba34ed82a10b598dfac4539be6f0b8d6880061eb61ac85780c9fcd84f5f498e72ce9228c3bb088d83bba32bcda75d4238b03e070ea64eb3b9d334a88718636

data/Gemfile

@@ -7,8 +7,8 @@ end
 
 group :test do
   gem 'activesupport'
+  gem 'awesome_print'
   gem 'database_cleaner'
   gem 'otr-activerecord'
   gem 'pg', '~> 0.21.0'
-  gem 'awesome_print'
 end

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    burstflow (0.2.0)
+    burstflow (0.2.1)
       activejob
       activerecord
 

data/README.md

@@ -2,25 +2,33 @@
 
 <!-- ## [![](http://i.imgur.com/ya8Wnyl.png)](https://chaps.io) proudly made by [Chaps](https://chaps.io) -->
 
-Burst is a parallel workflow runner using [ActiveRecord] and [ActiveJob](https://guides.rubyonrails.org/v4.2/active_job_basics.html) for scheduling and executing jobs.
+[![Build Status](https://travis-ci.com/RnD-Soft/burstflow.svg?branch=master)](https://travis-ci.com/RnD-Soft/burstflow) [![Gem Version](https://badge.fury.io/rb/burstflow.svg)](https://badge.fury.io/rb/burstflow)
 
-This gem is higly inspired by [Gush](https://github.com/chaps-io/gush). But burst not tied to Reddis or Sidekiq.
+Burstflow is a parallel workflow runner using [ActiveRecord] and [ActiveJob](https://guides.rubyonrails.org/v4.2/active_job_basics.html) for scheduling and executing jobs.
 
-The main feature of this runner is availablity to **suspend** job(and whole workflow) and **resume** it in future. For example if your job makes asynchronous request and will receive a response some time later. In this case, the job can send request and suspend until some external event resumes it eventually.
+This gem is higly inspired by [Gush](https://github.com/chaps-io/gush). But not tied to Reddis or Sidekiq like Gush.
+
+ActiveRecord used as persisted store during workflow execution.
+
+Additional features:
+
+* **susped** and **resume** job(and whole workflow). For example if your job makes asynchronous request and will receive a response some time later. In this case, the job can send request and suspend until some external event resumes it eventually.
+* before/after callbacks on jobs level
+* before/after callbacks on workflow level
+* **Dynamic workflows**. Any job can produce another jobs while executing. This jobs has its parent as incomming jobs, and all parents outgoing jobs as own outgoings.
 
-Another difference from Gush is **Dynamic workflows**. Any job can produce another jobs while executing. This jobs has its parent as incomming jobs, and all parents outgoing jobs as own outgoings.
 
 ## Installation
 
 ### 1. Add `burst` to Gemfile
 
 ```ruby
-gem 'burst', '~> 0.1.0'
+gem 'burstflow', '~> 0.2.0'
 ```
 
 ### 2. Run migrations
 
-Under development
+```rake burstflow:install:migrations```
 
 ## Example
 
@@ -29,7 +37,7 @@ Here is a complete example of a workflow you can create:
 
 ```ruby
 # app/workflows/sample_workflow.rb
-class SampleWorkflow < Burst::Workflow
+class SampleWorkflow < Burstflow::Workflow
   configure do |url_to_fetch_from|
     run FetchJob1, params: { url: url_to_fetch_from }
     run FetchJob2, params: { some_flag: true, url: 'http://url.com' }
@@ -56,7 +64,7 @@ and this is how the graph will look like:
 Let's start with the simplest workflow possible, consisting of a single job:
 
 ```ruby
-class SimpleWorkflow < Burst::Workflow
+class SimpleWorkflow < Burstflow::Workflow
   configure do 
     run DownloadJob
   end
@@ -74,12 +82,12 @@ class SimpleWorkflow < Burst::Workflow
 end
 ```
 
-We just told Burst to execute `SaveJob` right after `DownloadJob` finishes **successfully**.
+We just told Burstflow to execute `SaveJob` right after `DownloadJob` finishes **successfully**.
 
 But what if your job must have multiple dependencies? That's easy, just provide an array to the `after` attribute:
 
 ```ruby
-class SimpleWorkflow < Burst::Workflow
+class SimpleWorkflow < Burstflow::Workflow
   configure do
     run FirstDownloadJob
     run SecondDownloadJob
@@ -98,7 +106,7 @@ With this simple syntax you can build any complex workflows you can imagine!
 `run` method also accepts `before:` attribute to define the opposite association. So we can write the same workflow as above, but like this:
 
 ```ruby
-class SimpleWorkflow < Burst::Workflow
+class SimpleWorkflow < Burstflow::Workflow
   configure do
     run FirstDownloadJob, before: SaveJob
     run SecondDownloadJob, before: SaveJob
@@ -118,7 +126,7 @@ Workflows can accept any primitive arguments in their constructor, which then wi
 Let's assume we are writing a book publishing workflow which needs to know where the PDF of the book is and under what ISBN it will be released:
 
 ```ruby
-class PublishBookWorkflow < Burst::Workflow
+class PublishBookWorkflow < Burstflow::Workflow
   configure do |url, isbn|
     run FetchBook, params: { url: url }
     run PublishBook, params: { book_isbn: isbn }, after: FetchBook
@@ -136,7 +144,7 @@ and that's basically it for defining workflows, see below on how to define jobs:
 
 ## Defining jobs
 
-The simplest job is a class inheriting from `Burst::Job` and responding to `perform` and `resume` method. Much like any other ActiveJob class.
+The simplest job is a class inheriting from `Burstflow::Job` and responding to `perform` and `resume` method. Much like any other ActiveJob class.
 
 ```ruby
 class FetchBook < Burst::Job
@@ -189,14 +197,14 @@ flow = PublishBookWorkflow.build("http://url.com/book.pdf", "978-0470081204")
 flow.start!
 ```
 
-Now Burst will start processing jobs in the background using ActiveJob and your chosen backend.
+Now Burstflow will start processing jobs in the background using ActiveJob and your chosen backend.
 
 ### 3. Monitor its progress:
 
 ```ruby
 flow.reload
 flow.status
-#=> :running|:finished|:failed
+#=> :running|:finished|:failed|:suspended
 ```
 
 `reload` is needed to see the latest status, since workflows are updated asynchronously.
@@ -205,7 +213,7 @@ flow.status
 
 ### Pipelining
 
-Burst offers a useful tool to pass results of a job to its dependencies, so they can act differently.
+Burstflow offers a useful tool to pass results of a job to its dependencies, so they can act differently.
 
 **Example:**
 
@@ -228,9 +236,9 @@ end
 Now, since `DownloadVideo` finished and its dependant job `EncodeVideo` started, we can access that payload inside it:
 
 ```ruby
-class EncodeVideo < Burst::Job
+class EncodeVideo < Burstflow::Job
   def perform
-    video_path = payloads.first[:output]
+    video_path = payloads.first[:value]
   end
 end
 ```
@@ -243,7 +251,7 @@ end
   {
     id: "DownloadVideo-41bfb730-b49f-42ac-a808-156327989294" # unique id of the ancestor job
     class: "DownloadVideo",
-    output: "https://s3.amazonaws.com/somebucket/downloaded-file.mp4" #the payload returned by DownloadVideo job using `output()` method
+    value: "https://s3.amazonaws.com/somebucket/downloaded-file.mp4" #the payload returned by DownloadVideo job using `output()` method
   }
 ]
 ```
@@ -259,7 +267,7 @@ As an example, let's write a workflow which accepts an array of users and has to
 
 ```ruby
 
-class ParentJob < Burst::Job
+class ParentJob < Burstflow::Job
   def perform
     configure do 
       params[:user_ids].map do |user_id|
@@ -286,7 +294,7 @@ https://github.com/chaps-io/gush#contributors
 
 ## Contributing
 
-1. Fork it ( https://github.com/RnD-Soft/burst/fork )
+1. Fork it ( https://github.com/RnD-Soft/burstflow/fork )
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)

data/burstflow.gemspec

@@ -1,7 +1,7 @@
-$:.push File.expand_path("lib", __dir__)
+$:.push File.expand_path('lib', __dir__)
 
 # Maintain your gem's version:
-require "burstflow/version"
+require 'burstflow/version'
 
 Gem::Specification.new do |spec|
   spec.name          = 'burstflow'
@@ -19,9 +19,9 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency 'activejob'
   spec.add_dependency 'activerecord'
-  
+
   spec.add_development_dependency 'bundler'
+  spec.add_development_dependency 'generator_spec'
   spec.add_development_dependency 'rake'
   spec.add_development_dependency 'rspec'
-  spec.add_development_dependency  'generator_spec'
 end

data/db/schema.rb

@@ -10,20 +10,18 @@
 #
 # It's strongly recommended that you check this file into your version control system.
 
-ActiveRecord::Schema.define(version: 2018_01_01_000001) do
-
+ActiveRecord::Schema.define(version: 20_180_101_000_001) do
   # These are extensions that must be enabled in order to support this database
-  enable_extension "pgcrypto"
-  enable_extension "plpgsql"
+  enable_extension 'pgcrypto'
+  enable_extension 'plpgsql'
 
-  create_table "burstflow_workflows", id: :uuid, default: -> { "gen_random_uuid()" }, force: :cascade do |t|
-    t.string "type"
-    t.string "status"
-    t.jsonb "flow", default: {}, null: false
-    t.datetime "created_at", null: false
-    t.datetime "updated_at", null: false
-    t.index ["status"], name: "index_burstflow_workflows_on_status"
-    t.index ["type"], name: "index_burstflow_workflows_on_type"
+  create_table 'burstflow_workflows', id: :uuid, default: -> { 'gen_random_uuid()' }, force: :cascade do |t|
+    t.string 'type'
+    t.string 'status'
+    t.jsonb 'flow', default: {}, null: false
+    t.datetime 'created_at', null: false
+    t.datetime 'updated_at', null: false
+    t.index ['status'], name: 'index_burstflow_workflows_on_status'
+    t.index ['type'], name: 'index_burstflow_workflows_on_type'
   end
-
 end

data/lib/burstflow/job.rb

@@ -1,12 +1,13 @@
-require "active_support/rescuable"
-require "active_job/callbacks"
+require 'active_support/rescuable'
+require 'active_job/callbacks'
 
 class Burstflow::Job
-  require "burstflow/job/exception"
-  require "burstflow/job/model"
-  require "burstflow/job/initialization"
-  require "burstflow/job/state"
-  require "burstflow/job/callbacks"
+
+  require 'burstflow/job/exception'
+  require 'burstflow/job/model'
+  require 'burstflow/job/initialization'
+  require 'burstflow/job/state'
+  require 'burstflow/job/callbacks'
 
   include Burstflow::Job::Model
   include Burstflow::Job::Initialization
@@ -30,28 +31,28 @@ class Burstflow::Job
   end
 
   # execute this code by ActiveJob. You may return Burstflow::Job::SUSPEND to suspend job, or call suspend method
-  def perform
-  end
+  def perform; end
 
   def perform_now
     run_callbacks :perform do
       perform
     end
-  rescue => exception
+  rescue StandardError => exception
     rescue_with_handler(exception) || raise
   end
 
   # execute this code when resumes after suspending
   def resume(data)
-    raise InternalError.new(self, "Can't perform resume: not resumed") if !resumed?
+    raise InternalError.new(self, "Can't perform resume: not resumed") unless resumed?
+
     set_output(data)
   end
 
-  def resume_now data
+  def resume_now(data)
     run_callbacks :resume do
       resume(data)
     end
-  rescue => exception
+  rescue StandardError => exception
     rescue_with_handler(exception) || raise
   end
 
@@ -62,13 +63,13 @@ class Burstflow::Job
 
   # mark execution as suspended
   def suspend
-    raise InternalError.new(self, "Can't suspend: not running") if !running?
+    raise InternalError.new(self, "Can't suspend: not running") unless running?
+
     set_output(SUSPEND)
   end
 
-  def configure *args, &block
+  def configure(*args, &block)
     workflow.with_lock do
-
       builder = Burstflow::Workflow::Builder.new(workflow, *args, &block)
       workflow.flow['jobs_config'] = builder.as_json
       workflow.save!
@@ -78,12 +79,12 @@ class Burstflow::Job
 
   def attributes
     {
-      workflow_id: self.workflow_id,
-      id: self.id,
-      klass: self.klass,
+      workflow_id: workflow_id,
+      id: id,
+      klass: klass,
       params: params,
-      incoming: self.incoming,
-      outgoing: self.outgoing,
+      incoming: incoming,
+      outgoing: outgoing,
       output: output,
       started_at: started_at,
       enqueued_at: enqueued_at,
@@ -91,12 +92,8 @@ class Burstflow::Job
       failed_at: failed_at,
       suspended_at: suspended_at,
       resumed_at: resumed_at,
-      failure: self.failure
+      failure: failure
     }
   end
 
-  
-
-
-
 end

data/lib/burstflow/job/callbacks.rb

@@ -1,17 +1,15 @@
 module Burstflow::Job::Callbacks
+
   extend  ActiveSupport::Concern
   include ActiveJob::Callbacks
 
   included do
-
     define_callbacks :suspend
     define_callbacks :resume
     define_callbacks :failure
-
   end
 
   class_methods do
-
     def before_suspend(*filters, &blk)
       set_callback(:suspend, :before, *filters, &blk)
     end
@@ -24,7 +22,6 @@ module Burstflow::Job::Callbacks
       set_callback(:suspend, :around, *filters, &blk)
     end
 
-  
     def before_resume(*filters, &blk)
       set_callback(:resume, :before, *filters, &blk)
     end
@@ -37,19 +34,17 @@ module Burstflow::Job::Callbacks
       set_callback(:resume, :around, *filters, &blk)
     end
 
-  
     def before_failure(*filters, &blk)
       set_callback(:failure, :before, *filters, &blk)
     end
-  
+
     def after_failure(*filters, &blk)
       set_callback(:failure, :after, *filters, &blk)
     end
-    
+
     def around_failure(*filters, &blk)
       set_callback(:failure, :around, *filters, &blk)
     end
-
   end
 
-end
+end

data/lib/burstflow/job/exception.rb

@@ -1,8 +1,10 @@
 class Burstflow::Job::InternalError < ::RuntimeError
+
   attr_accessor :job
 
   def initialize(job, message)
     @job = job
     super(message)
   end
-end
+
+end

data/lib/burstflow/job/initialization.rb

@@ -1,8 +1,8 @@
 module Burstflow::Job::Initialization
-  extend  ActiveSupport::Concern
 
-  included do
+  extend ActiveSupport::Concern
 
+  included do
     def initialize(workflow, job_config = {})
       @workflow = workflow
       assign_default_values(job_config)
@@ -21,15 +21,12 @@ module Burstflow::Job::Initialization
     def reload
       assign_default_values(@workflow.job_hash(self.id))
     end
-
   end
 
   class_methods do
-
     def from_hash(workflow, job_config)
       job_config[:klass].constantize.new(workflow, job_config)
     end
-
   end
 
-end
+end

data/lib/burstflow/job/state.rb

@@ -1,11 +1,12 @@
 module Burstflow::Job::State
-  extend  ActiveSupport::Concern
 
-  included do
+  extend ActiveSupport::Concern
 
+  included do
     # mark job as enqueued when it is scheduled to queue
     def enqueue!
       raise Burstflow::Job::InternalError.new(self, "Can't enqueue: already enqueued") if enqueued?
+
       self.enqueued_at = current_timestamp
       self.started_at = nil
       self.finished_at = nil
@@ -17,22 +18,25 @@ module Burstflow::Job::State
     # mark job as started when it is start performing
     def start!
       raise Burstflow::Job::InternalError.new(self, "Can't start: already started") if started?
-      raise Burstflow::Job::InternalError.new(self, "Can't start: not enqueued") if !enqueued?
+      raise Burstflow::Job::InternalError.new(self, "Can't start: not enqueued") unless enqueued?
+
       self.started_at = current_timestamp
     end
 
     # mark job as finished when it is finish performing
     def finish!
       raise Burstflow::Job::InternalError.new(self, "Can't finish: already finished") if finished?
-      raise Burstflow::Job::InternalError.new(self, "Can't finish: not started") if !started?
+      raise Burstflow::Job::InternalError.new(self, "Can't finish: not started") unless started?
+
       self.finished_at = current_timestamp
     end
 
     # mark job as failed when it is failed
-    def fail! msg_or_exception
-      #raise Burstflow::Job::InternalError.new(self, "Can't fail: already failed") if failed?
-      #raise Burstflow::Job::InternalError.new(self, Can't fail: already finished") if finished?
-      raise Burstflow::Job::InternalError.new(self, "Can't fail: not started") if !started?
+    def fail!(msg_or_exception)
+      # raise Burstflow::Job::InternalError.new(self, "Can't fail: already failed") if failed?
+      # raise Burstflow::Job::InternalError.new(self, Can't fail: already finished") if finished?
+      raise Burstflow::Job::InternalError.new(self, "Can't fail: not started") unless started?
+
       self.finished_at = self.failed_at = current_timestamp
 
       context = {}
@@ -51,14 +55,16 @@ module Burstflow::Job::State
     # mark job as suspended
     def suspend!
       raise Burstflow::Job::InternalError.new(self, "Can't suspend: already suspended") if suspended?
-      raise Burstflow::Job::InternalError.new(self, "Can't suspend: not runnig") if !running?
+      raise Burstflow::Job::InternalError.new(self, "Can't suspend: not runnig") unless running?
+
       self.suspended_at = current_timestamp
     end
 
     # mark job as resumed
     def resume!
       raise Burstflow::Job::InternalError.new(self, "Can't resume: already resumed") if resumed?
-      raise Burstflow::Job::InternalError.new(self, "Can't resume: not suspended") if !suspended?
+      raise Burstflow::Job::InternalError.new(self, "Can't resume: not suspended") unless suspended?
+
       self.resumed_at = current_timestamp
     end
 
@@ -115,11 +121,9 @@ module Burstflow::Job::State
     def current_timestamp
       Time.now.to_i
     end
-
   end
 
   class_methods do
-
   end
 
-end
+end