job_reactor 0.5.0.beta4 → 0.5.0

data/README.markdown

@@ -5,22 +5,33 @@ Now we are in beta (need to complete documentation and fix some bugs)
 
 JobReactor is a library for creating, scheduling and processing background jobs.
 It is asynchronous client-server distributed system based on [EventMachine][0].
-Inspired by [Resque][1], [Stalker][2], [DelayedJob][3], and etc.
+Inspired by [Resque][1], [Beanstalkd][2]([Stalker][3]), [DelayedJob][4], and etc.
 
 JobReactor has not 'rails' integration for the time being.
-But it is very close. We need test the system with different servers (clusters) and automatize initialization and restart processes.
+But it is very close. We need to test the system with different servers (clusters) and automate the initialization and re-start processes.
 Collaborators, you are welcome!
 
-So, read 'features' part and try JobReactor. You can do a lot with it.
+So, read the 'features' section and try JobReactor. You can do a lot with it.
+
+Note
+====
+JobReactor is based on [EventMachine][0]. Jobs are launched in EM reactor loop in one thread.
+There are advantages and disadvantages. The main benefit is fast scheduling, saving and loading.
+The weak point is the processing of heavy background jobs when each job takes minutes and hours.
+They will block the reactor and break normal processing.
+
+If you can't divide 'THE BIG JOB' into 'small pieces' you shouldn't use JobReactor. See alternatives such [DelayedJob][4] or [Resque][1].
+
+__JobReactor is the right solution if you have thousands, millions, and, we hope:), billions relatively small jobs.__
 
 Quick start
 ===========
 Use `gem install job_reactor --pre` to try it.
 
-You need to install [Redis][6] if you want to persist your jobs.
+You need to install [Redis][5] if you want to persist your jobs.
 ``$ sudo apt-get install redis-server ``
 
-In you main application:
+In your main application:
 `application.rb`
 ``` ruby
 require 'job_reactor'
@@ -47,9 +58,14 @@ And the last file - 'the worker code':
 `worker.rb`
 ``` ruby
 require 'job_reactor'
-JR.config[:job_directory] = 'reactor_jobs' #this default config so you can omit this line
+JR.config[:job_directory] = 'reactor_jobs' #this default config, so you can omit this line
 JR.run! do
-  JR.start_node({:storage => 'memory_storage', :name => 'worker_1', :server => ['localhost', 5001], :distributors => [['localhost', 5000]] })
+  JR.start_node({
+  :storage => 'memory_storage',
+  :name => 'worker_1',
+  :server => ['localhost', 5001],
+  :distributors => [['localhost', 5000]]
+  })
 end
 ```
 Run 'application.rb' in one terminal window and 'worker.rb' in another.
@@ -71,74 +87,193 @@ simple insignificant jobs in memory.
 And more: your nodes may create jobs for others nodes and communicate with each other. See page [advanced usage].
 3. Full job control
 -------------------
-You can add callback and errbacks to the job which will be called on the node.
+You can add 'callback' and 'errbacks' to the job which will be called on the node.
 You also can add 'success feedback' and 'error feedback' which will be called in your main application.
 When job is done on remote node, your application will receive the result inside corespondent 'feedback'.
-If error occur in the job you can see it in errbacks and do what you want.
-Inside the job you can get information about when it starts, which node execute job and etc.
-You also can add some arguments to the job on-the-fly which will be used in the subsequent callbacks and errbacks. See [advance usage].
-4. Reliability
+If error occur in the job you can see it in 'errbacks' and the in 'error feedback' and do what you want.
+4. Reflection and modifying
+---------------------------
+Inside the job you can get information about when it starts, when it fails, which node execute job and etc.
+You also can add some arguments to the job on-the-fly which will be used in the subsequent callbacks and errbacks.
+These arguments then can be sent back to the distibutor.
+5. Reliability
 --------------
 You can run additional nodes and stop any nodes on-the-fly.
 Distributor is smart enough to send jobs to another node if someone is stopped or crashed.
 If no nodes are connected to distributor it will keep jobs in memory and send them when nodes start.
 If node is stopped or crashed it will retry stored jobs after start.
-5. EventMachine available
+6. EventMachine available
 -------------------------
 Remember, your jobs will be run inside EventMachine reactor! You can easily use the power of async nature of EventMachine.
-Use asynchronous [em-http-request][4], [em-websocket][5], [etc.], [etc.], and [etc]. See page [advance usage].
-6. Deferred and periodic jobs
+Use asynchronous [em-http-request][6], [em-websocket][7], [etc.], [etc.], and [etc].
+7. Thread safe
+Eventmachine reactor loop runs in one thread. So the code in jobs executed in the given node is absolutely threadsafe.  
+8. Deferred and periodic jobs
 -----------------------------
 You can use deferred jobs which will run 'after' some time or 'run_at' given time.
 You can create periodic jobs which will run every given time period and cancel them on condition.
-7. No polling
+9. No polling
 -------------
 There is no storage polling. Absolutely. When node receives job (no matter instant, periodic or deferred) there will be EventMachine timer created
 which will start job at the right time.
-8. Job retrying
+10. Job retrying
 --------------
 If job fails it will be retried. You can choose global retrying strategy or manage separate jobs.
-9. Predefined nodes
+11. Predefined nodes
 -------------------
 You can specify node for jobs, so they will be executed in that node environment. And you can specify which node is forbidden for the job.
 If no nodes are specified distributor will try to send the job to the first free node.
-10. Node based priorities
+12. Node based priorities
 -----------------------
-There are no priorities like in Delayed::Job or Stalker. Bud there are flexible node-based priorities.
+There are no priorities like in Delayed::Job or Stalker. But there are flexible node-based priorities.
 You can specify the node which should execute the job and the node is forbidden for given job. You can reserve several nodes for high priority jobs.
 
+How it works
+============
+1. You run JobReactor::Distributor in your application initializer
+-----------------------------------------------------
+``` ruby
+JR.run do
+  JR.start_distributor('localhost', 5000)
+end
+```
+This code runs EventMachine reactor loop in the new thread and call the block given.
+JR.start_distributor starts EventMachine TCP server on given host and port.
+And now JobReactor is ready to work.
+
+2. You run JobReactor::Node in the different process or different machine
+------------------------------------------------------------------------
 
+``` ruby
+JR.run! do
+  JR.start_node({
+    :storage => 'redis_storage',
+    :name => 'redis_node1',
+    :server => ['localhost', 5001],
+    :distributors => [['localhost', 5000]] 
+})
+end
+```
 
-The main parts of JobReactor are:
----------------------------------
-JobReactor module for creating jobs.
-Distributor module for 'distributing' jobs between working nodes.
-Node object for job processing.
-#TODO
+This code runs EventMachine reactor loop (in the main thread: this is the difference between `run` and `run!`).
+And start the Node inside the reactor.
+When node starts it:
+* parses the 'reactor jobs' files (recursively parse all files specified in JR.config[:job_directory] directory, default is 'reactor_jobs' directory) and create hash of jobs callbacks and errbacs (see [JobReator jobs]);
+* tries to 'retry' the job (if you use 'redis_storage' and `JR.config[:retry_jobs_at_start]` is true) 
+* starts it's own TCP server;
+* connects to Distributor server and sends the information about needed to establish the connection;
+When distributor receives the credentials it connects to Node server. And now there is a full duplex-connection between Distributor and Node.
+
+3. You enqueue the job in your application
+------------------------------------------
+
+```ruby
+JR.enqueue('my_job',{arg1: 1, arg2: 2}, {after: 20}, success, error)
+```
 
+The first argument is the name of the job, the second is the arguments hash for the job.
+The third is the options hash. If you don't specify any option job will be instant job and will be sent to any free node. You can use the following options:
+* `after: seconds` - node will try run the job after  `seconds` seconds;
+* `run_at: time` - node will try run the job at given time;
+* `period: seconds` - node will run job periodically, each `seconds` seconds;
+You can add `node: 'node_name'` and `not_node: 'node_name'` to the options. This specify the node on which the job should or shouldn't be run. For example:
 
+```ruby
+JR.enqueue('my_job', {arg1: 1}, {period: 100, node: 'my_favourite_node', not_node: 'do_not_use_this_node'})
+```
 
+The rule to use specified node is not strict if `JR.config[:always_use_specified_node]` is false (default).
+This means that distributor will try to send the job to the given node at first. But if the node is `locked` (maybe you have just sent another job to it and it is very busy) distributor will look for other node.
+The last two arguments are optional too. The first is 'success feedback' and the last is 'error feedback'. We use term 'feedback' to distinguish from 'callbacks' and 'errbacks'. 'feedback' is executed on the main application side while 'callbacks' on the node side. 'feedbacks' are the procs which will be called when node sent message that job is completed (successfully or not). The argunments for the 'feedback' are the arguments of the initial job plus all added on the node side.
+Example:
 
+```ruby
+#in your 'job_file'
+job 'my_job' do |args|
+#do smth
+args.merge!(result: 'Yay!')
+#in your application
+#success feedback
+success = proc {|args| puts args}
+#enqueue job
+JR.enqueue('my_job', {arg1: 1}, {}, success)
+```
 
+The 'success' proc args will be {arg1: 1, result: 'Yay!'}.
+The same story is with 'error feedback'. Note that error feedback will be launched after all attempts failed on the node side.
+See config: `JR.config[:max_attempt] = 10` and `JR.config[:retry_multiplier]`
 
+4. You disconnect node (stop it manually or node fails itself)
+--------------------------------------------------------------
+* distributor will send jobs to any other nodes if present
+* distributor will store in memory enqueued jobs if there is no connected node (or specified node)
+* when node starts again, then distributor will send jobs to the node
 
+5. You stop the main application.
+---------------------------------
+* Nodes will continue to work, but you won't be able to receive the results from node when you start the application again because all feedbacks are stored in memory.
 
-How it works
-------------
-#TODO
+Job Storage
+==========
+Now you can store your job in [Redis][5] storage (`'redis_storage`') or in memory (`'memory_storage'`).
+Only the first, of course, 'really' persists the jobs. You can use the last one if you don't want install Redis, don't need retry jobs and need more speed (by the way, the difference in performance is not so great - Redis is very fast).
 
+The default host and port for Redis server are:
 
-License
----------
-The MIT License - Copyright (c) 2012 Anton Mishchuk
+```ruby
+JR.config[:redis_host] = 'localhost'
+JR.config[:redis_port] = 6379
+```
 
+JobReactor works asynchronously with Redis using [em-redis][8] library to increase the speed.
+Several nodes can use one Redis storage.
 
+The informaion about jobs is saved several times during processing. This information includes:
+* id - the unique job id;
+* name - job name which 'defines' the job;
+* args - serialized arguments for the job;
+* run_at - the time when job was launched;
+* failed_at - the time when job was failed;
+* last_error - the error occured;
+* period - period (for periodic jobs);
+* status - job status ('new', 'in progress', 'queued', 'complete', 'error', 'failed', 'cancelled');
+* attempt - the number of attempt;
+* make_after - when to start job again (in seconds after last save);
+* distributor - host and port of distributor server which sent the job (used for 'feedbacks');
+* on_success - the unique id of success feedback on the distributor side;
+* on_error - the unique id of error feedback on the distributor side;
 
+By default JobReactor delete all completed and cancelled jobs, but you can configure it:
+The default options are:
+
+```ruby
+JR.config[:remove_done_jobs] = true
+JR.config[:remove_cancelled_jobs] = true
+JR.config[:remove_failed_jobs] = false
+JR.config[:retry_jobs_at_start] = true
+```
+
+We provide simple `JR::RedisMonitor` module to check the Redis storage from irb console (or from your app).
+See methods:
+
+```ruby
+JR::RedisMonitor.jobs_for(node_name)
+JR::RedisMonitor.load(job_id)
+JR::RedisMonitor.destroy(job_id)
+JR::RedisMonitor.destroy_all_jobs_for(node_name)
+```
+
+
+License
+=======
+The MIT License - Copyright (c) 2012 Anton Mishchuk
 
 [0]: http://rubyeventmachine.com
 [1]: https://github.com/defunkt/resque
-[2]: https://github.com/han/stalker
-[3]: https://github.com/tobi/delayed_job
-[4]: https://github.com/igrigorik/em-http-request
-[5]: https://github.com/igrigorik/em-websocket
-[6]: http://redis.io
+[2]: http://kr.github.com/beanstalkd/
+[3]: https://github.com/han/stalker
+[4]: https://github.com/tobi/delayed_job
+[5]: http://redis.io
+[6]: https://github.com/igrigorik/em-http-request
+[7]: https://github.com/igrigorik/em-websocket
+[8]: https://github.com/madsimian/em-redis

data/lib/job_reactor/job_reactor.rb

@@ -52,11 +52,11 @@ module JobReactor
     end
 
     def succ_feedbacks
-      @@callbacks ||= { }
+      @@succ_feedbacks ||= { }
     end
 
     def err_feedbacks
-      @@errbacks ||= { }
+      @@err_feedbacks ||= { }
     end
 
     # Here is the only method user can call inside the application (excepts start-up methods, of course).
@@ -73,7 +73,7 @@ module JobReactor
     # (Do not use procs, objects with singleton methods, etc ... ).
     #
     # Example:
-    # JR.enqueue 'job', {:arg1 => 'arg1', :arg2 => 'arg2'}
+    # JR.enqueue 'job', {arg1: 'arg1', arg2: 'arg2'}
     #
     # You can add the following options:
     # :run_at - run at given time;
@@ -83,8 +83,8 @@ module JobReactor
     # :not_node - to do not send job to the node;
     #
     # Example:
-    # JR.enqueue 'job', {:arg1 => 'arg1'}, {:period => 100, :node => 'my_favorite_node'}
-    # JR.enqueue 'job', {:arg1 => 'arg1'}, {:after => 10, :not_node => 'some_node'}
+    # JR.enqueue 'job', {arg1: 'arg1'}, {period: 100, node: 'my_favorite_node'}
+    # JR.enqueue 'job', {arg1: 'arg1'}, {after: 10, not_node: 'some_node'}
     #
     # You can add 'success feedback' and 'error feedback'. We use term 'feedback' to distinguish them from callbacks and errbacks which are executed on the node side.
     # These feedbacks are the procs. The first is 'success feedback', the second - 'error feedback'.
@@ -93,7 +93,7 @@ module JobReactor
     # Example:
     # success = proc { |args| result = args }
     # error = proc { |args| result = args }
-    # JR.enqueue 'job', { :arg1 => 'arg1'}, {}, success, error
+    # JR.enqueue 'job', { arg1: 'arg1'}, {}, success, error
     #
     def enqueue(name, args = { }, opts = { }, success_proc = nil, error_proc = nil)
       hash = { 'name' => name, 'args' => args, 'attempt' => 0, 'status' => 'new' }
@@ -107,8 +107,8 @@ module JobReactor
 
       hash.merge!('distributor' => "#{JR::Distributor.host}:#{JR::Distributor.port}")
 
-      add_succ_feedbacks!(hash, success_proc) if success_proc
-      add_err_feedbacks!(hash, error_proc) if error_proc
+      add_succ_feedbacks!(hash, success_proc) if success_proc.is_a? Proc
+      add_err_feedbacks!(hash, error_proc) if error_proc.is_a? Proc
 
       JR::Distributor.send_data_to_node(hash)
     end

data/lib/job_reactor/job_reactor/config.rb

@@ -12,7 +12,7 @@ JR.config[:job_directory] = 'reactor_jobs'
 JR.config[:max_attempt] = 10
 JR.config[:retry_multiplier] = 5
 JR.config[:retry_jobs_at_start] = true
-JR.config[:merge_job_itself_to_args] = true
+JR.config[:merge_job_itself_to_args] = false
 JR.config[:log_job_processing] = true
 JR.config[:always_use_specified_node] = false #will send job to another node if specified node is not available
 JR.config[:remove_done_jobs] = true
@@ -22,6 +22,8 @@ JR.config[:remove_failed_jobs] = false
 JR.config[:redis_host] = 'localhost'
 JR.config[:redis_port] = 6379
 
+JR.config[:logger_method] = :puts
+
 #TODO next releases with rails support
 #JR.config[:active_record_adapter] = 'mysql2'
 #JR.config[:active_record_database] = 'em'

data/lib/job_reactor/job_reactor/exceptions.rb

@@ -2,7 +2,7 @@ module JobReactor
 
   class NoSuchJob < RuntimeError
   end
-  class CancelJob < RuntimeError
+  class CancelJob < Exception
   end
 
 end

data/lib/job_reactor/logger.rb

@@ -1,57 +1,30 @@
 module JobReactor
   module Logger
-################
-#   To set output stream
-    @@logger_method = :puts
+
+    # Sets the output stream
+    #
+    @@logger_method = JR.config[:logger_method]
 
     def self.stdout=(value)
-      if value.is_a?(Symbol) && value == :rails_logger
-        @@stdout        = Rails.logger
-        @@logger_method = :info
-      else
-        @@stdout        = value
-        @@logger_method = :puts
-      end
+      @@stdout = value
     end
 
     def self.stdout
       @@stdout ||= $stdout
     end
 
-#################
-#   Is checked in dev_log
-
-    @@development = false
-
-    def self.development=(value)
-      @@development = !!value
-    end
-
-#################
-
-# Log message to output stream
-#
+    # Logs message to output stream
+    #
     def self.log(msg)
       stdout.public_send(@@logger_method, '-'*100)
       stdout.public_send(@@logger_method, msg)
     end
 
-    # Build string for job event and log it
+    # Builds string for job event and log it
     #
     def self.log_event(event, job)
-      log("Log: #{event} #{job['name']}")
-    end
-
-    # Log if JR::Logger.development is set to true
-    #
-    def self.dev_log(msg)
-      log(msg) if development?
+      log("#{event} #{job['name']}")
     end
 
-    # Is JR::Logger.development set to true?
-    #
-    def self.development?
-      @@development
-    end
   end
 end

data/lib/job_reactor/node.rb

@@ -52,9 +52,13 @@ module JobReactor
     # It makes a job and run do_job.
     #
     def schedule(hash)
-      EM::Timer.new(hash['make_after']) do  #Of course, we can start job immediately (unless it is 'after' job), but we let EM take care about it. Maybe there is another job is ready to start
-        self.storage.load(hash) do |hash|
-          do_job(JR.make(hash))
+      if hash['make_after'].to_i > 0
+        EM::Timer.new(hash['make_after']) do
+          self.storage.load(hash) { |hash| do_job(JR.make(hash)) }
+        end
+      else
+        EM.next_tick do
+          self.storage.load(hash) { |hash| do_job(JR.make(hash)) }
         end
       end
     end
@@ -78,7 +82,7 @@ module JobReactor
           job.succeed(args)
           job['args'] = args
           job_completed(job)
-        rescue JobReactor::CancelJob
+        rescue CancelJob
           cancel_job(job)
         rescue Exception => e
           rescue_job(e, job)
@@ -116,14 +120,14 @@ module JobReactor
         job['status']     = 'error'
         self.storage.save(job) do |job|
           begin
-            args = job['args'].merge!(:error => e).merge(JR.config[:merge_job_itself_to_args] ? { :job_itself => job.dup } : { })
+            args = job['args'].merge!(error: e).merge(JR.config[:merge_job_itself_to_args] ? { job_itself: job.dup } : { })
             job.fail(args) #Fire errbacks. You can access error in you errbacks (args[:error])
             job['args'] = args
             complete_rescue(job)
           rescue JobReactor::CancelJob
-            cancel_job(job) #If it was cancelled we destroy it or set status 'cancelled'
+            cancel_job(job, true) #If it was cancelled we destroy it or set status 'cancelled'
           rescue Exception => e  #Recsue Exceptions in errbacks
-            job['args'].merge!(:errback_error => e) #So in args you now have :error and :errback_error
+            job['args'].merge!(errback_error: e) #So in args you now have :error and :errback_error
             complete_rescue(job)
           end
         end
@@ -148,8 +152,9 @@ module JobReactor
 
     # Cancels job. Remove or set 'cancelled status'
     #
-    def cancel_job(job)
-      report_error(job) if job['on_error']
+    def cancel_job(job, error = false)
+      report_success(job) if !error && job['on_success']
+      report_error(job) if error && job['on_error']
       if JR.config[:remove_cancelled_jobs]
         storage.destroy(job)
       else
@@ -178,7 +183,7 @@ module JobReactor
     #
     def retry_jobs
       storage.jobs_for(name) do |job_to_retry|
-        job_to_retry['args'].merge!(:retrying => true)
+        job_to_retry['args'].merge!(retrying: true)
         try_again(job_to_retry) if job_to_retry
       end
     end
@@ -189,7 +194,7 @@ module JobReactor
       host, port = job['distributor'].split(':')
       port = port.to_i
       distributor = self.connections[[host, port]]
-      data = {:success => { callback_id: job['on_success'], args: job['args']}}
+      data = { :success => { callback_id: job['on_success'], args: job['args']} }
       data[:success].merge!(do_not_delete: true) if job['period'] && job['period'].to_i > 0
       data = Marshal.dump(data)
       send_data_to_distributor(distributor, data)

data/lib/job_reactor/storages/memory_storage.rb

@@ -11,9 +11,11 @@ module JobReactor
 
       def load(hash, &block)
         hash      = storage[hash['id']]
-        hash_copy = { }
-        hash.each { |k, v| hash_copy.merge!(k => v) }
-        block.call(hash_copy) if block_given?
+        if hash
+          hash_copy = { }
+          hash.each { |k, v| hash_copy.merge!(k => v) }
+          block.call(hash_copy) if block_given?
+        end
       end
 
       def save(hash, &block)

data/lib/job_reactor/storages/redis_monitor.rb

@@ -51,7 +51,7 @@ module JobReactor
     #
     def destroy_all_jobs_for(name)
       pattern = "*#{name}_*"
-      storage.del(*storage.keys(pattern))
+      storage.del(*storage.keys(pattern)) unless storage.keys(pattern).empty?
     end
 
   end

data/lib/job_reactor/storages/redis_storage.rb

@@ -17,15 +17,17 @@ module JobReactor
         hash_copy = {'node' => hash['node']} #need new object, because old one has been 'failed'
 
         storage.hmget(key, *ATTRS) do |record|
-          ATTRS.each_with_index do |attr, i|
-            hash_copy[attr] = record[i]
-          end
-          ['attempt', 'period', 'make_after'].each do |attr|
-            hash_copy[attr] = hash_copy[attr].to_i
-          end
-          hash_copy['args'] = Marshal.load(hash_copy['args'])
+          unless record.compact.empty?
+            ATTRS.each_with_index do |attr, i|
+              hash_copy[attr] = record[i]
+            end
+            ['attempt', 'period', 'make_after'].each do |attr|
+              hash_copy[attr] = hash_copy[attr].to_i
+            end
+            hash_copy['args'] = Marshal.load(hash_copy['args'])
 
-          block.call(hash_copy) if block_given?
+            block.call(hash_copy) if block_given?
+          end
         end
       end
 

metadata

@@ -1,8 +1,8 @@
 --- !ruby/object:Gem::Specification
 name: job_reactor
 version: !ruby/object:Gem::Version
-  version: 0.5.0.beta4
-  prerelease: 6
+  version: 0.5.0
+  prerelease: 
 platform: ruby
 authors:
 - Anton Mishchuk
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-06-05 00:00:00.000000000 Z
+date: 2012-06-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: eventmachine
@@ -60,8 +60,9 @@ dependencies:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
-description: ! "    JobReactor is a library for creating and processing background
-  jobs.\n    It is client-server distributed system based on EventMachine.\n"
+description: ! "    JobReactor is a library for creating, scheduling and processing
+  background jobs.\n    It is asynchronous client-server distributed system based
+  on EventMachine.\n    Inspired by DelayedJob, Resque, Beanstalkd, and etc.\n"
 email: anton.mishchuk@gmial.com
 executables: []
 extensions: []
@@ -99,9 +100,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
-  - - ! '>'
+  - - ! '>='
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.24