parallel_minion 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: a490fae075880a809d3d3542381454e2737ee560
-  data.tar.gz: bec1bb591e8d19b490821f5153cb195083e9e58a
+  metadata.gz: ba4bc73da8594cb544d75365ae616634584cb197
+  data.tar.gz: 14a7ccfde7406ba6f41b0c28b133fb989160c67f
 SHA512:
-  metadata.gz: 49c1e58bcb4011633cd0f80d666a6872f8fea1719cb5e2085dc463e6afd0ed71672ee1e49c5846fca668a0704beebdf22f0b816d804a7590f88e02571bf7ecd6
-  data.tar.gz: 383505b36f1990379463117143231f2c069dc3451658a05fcc8d5c303a1394cdf76a232a4439f0d90660e8af3305ddc6445f5c74171f75f70940df7f0f43f534
+  metadata.gz: d9dca76b499fa4484575825bfe0e7a86feb49e3a46f921e1a34211c5db4bca6506ad3456a947b1423a8914fb0f371f29443bbd465d7db5525bae1c4b40f97ca6
+  data.tar.gz: 571076e7b09162becba4c8d1c8a79f3f568722b6f276798cbaf5f97cae8abff608c778ec6d0289e9ecc1c931a17bb8dc788f1170a7191e31377b2d989eb66b9e

data/README.md

@@ -1,49 +1,67 @@
 parallel_minion
 ===============
 
-Parallel Minion supports easily handing work off to Minions (Threads) so that tasks
+Parallel Minion supports easily handing work off to minions (threads) so that tasks
 that would normally be performed sequentially can easily be executed in parallel.
 This allows Ruby and Rails applications to very easily do many tasks at the same
 time so that results are returned more quickly.
 
-Our use-case for Minions is where an application grew to a point where it would
+Our use-case for minions is where an application grew to a point where it would
 be useful to run some of the steps in fulfilling a single request in parallel.
 
 ## Features:
 
 Exceptions
 
-- Any exceptions raised in Minions are captured and propagated back to the
+- Any exceptions raised in minions are captured and propagated back to the
   calling thread when #result is called
-- Makes exception handling simple with a drop-in replacement in existing code
+- Makes exception handling simple with a drop-in replacement for existing code
+- Avoids having to implement more complex actors and supervisors required
+  by some concurrency frameworks
 
 Timeouts
 
-- Timeout when a Minion does not return within a specified time
-- Timeouts are a useful feature when one of the Minions fails to respond in a
+- Timeout when a minion does not return within a specified time
+- Timeouts are a useful feature when one of the minions fails to respond in a
   reasonable amount of time. For example when a call to a remote service hangs
   we can send back a partial response of other work that was completed rather
   than just "hanging" or failing completely.
 
 Logging
 
-- Built-in support to log the duration of all Minion tasks to make future analysis
+- Built-in support to log the duration of all minion tasks to make future analysis
   of performance issues much easier
-- Logs any exceptions thrown to assist fwith problem diagnosis
+- Logs any exceptions thrown to assist with problem diagnosis
+- Logging tags from the current thread are propagated to the minions thread
+- The name of the thread in log entries is set to the description supplied for
+  the minion to make it easy to distinguish log entries by minion / thread
+
+Rails Support
+
+- When used in a Rails environment the current scope of specified models can be
+  propagated to the minions thread
 
 ## Example
 
 Simple example
 
 ```ruby
-ParallelMinion::Minion.new(10.days.ago, description: 'Doing something else in parallel', timeout: 1000) do |date|
+minion = ParallelMinion::Minion.new(10.days.ago, description: 'Doing something else in parallel', timeout: 1000) do |date|
   MyTable.where('created_at <= ?', date).count
 end
+
+# Do other work here...
+
+# Retrieve the result of the minion
+count = minion.result
+
+puts "Found #{count} records"
 ```
 
 ## Example
 
-For example, in the code below there are several steps that are performed sequentially:
+For example, in the code below there are several steps that are performed
+sequentially and does not yet use minions:
 
 ```ruby
 # Contrived example to show how to do parallel code execution
@@ -190,13 +208,39 @@ end
 
 The above #process_request method should now take on average 1,810 milli-seconds
 which is significantly faster than the 3,780 milli-seconds it took to perform
-the exact same request, but using only a single thread
+the exact same request prior to using minions.
 
-The exact breakdown of which calls to do in the main thread versus a Minion is determined
+The exact breakdown of which calls to do in the main thread versus a minion is determined
 through experience and trial and error over time. The key is logging the duration
-of each call which Minion does by default so that the exact processing breakdown
+of each call which minion does by default so that the exact processing breakdown
 can be fine-tuned over time.
 
+## Disabling Minions
+
+In the event that strange problems are occurring in production and no one is
+sure if it is due to running the minion tasks in parallel, it is simple to make
+all minion tasks run in the calling thread.
+
+It may also be useful to disable minions on a single production server to compare
+its performance to that of the servers running with minions active.
+
+To disable minions / make them run in the calling thread, add the following
+lines to config/environments/production.rb:
+
+```ruby
+  # Make minions run immediately in the current thread
+  config.parallel_minion.enabled = false
+```
+
+## Notes:
+
+- When using JRuby it is important to enable it's built-in thread-pooling by
+  adding the following line to .jrubyrc, or setting the appropriate command line option:
+
+```ruby
+thread.pool.enabled=true
+```
+
 Meta
 ----
 

data/lib/parallel_minion/minion.rb

@@ -15,24 +15,23 @@ module ParallelMinion
     # Give an infinite amount of time to wait for a Minion to complete a task
     INFINITE = -1
 
-    # Sets whether to run in Synchronous mode
+    # Sets whether minions are enabled to run in their own threads
     #
-    # By Setting synchronous to true all Minions that have not yet been started
-    # will run in the thread from which they are started and not in their own
-    # threads
+    # By Setting _enabled_ to false all Minions that have not yet been started
+    # will run in the thread from which it is created and not on its own thread
     #
     # This is useful:
     # - to run tests under the Capybara gem
     # - when debugging code so that all code is run sequentially in the current thread
     #
-    # Note: Do not set this setting to true in Production
-    def self.synchronous=(synchronous)
-      @@synchronous = synchronous
+    # Note: Not recommended to set this setting to false in Production
+    def self.enabled=(enabled)
+      @@enabled = enabled
     end
 
-    # Returns whether running in Synchronous mode
-    def self.synchronous?
-      @@synchronous
+    # Returns whether minions are enabled to run in their own threads
+    def self.enabled?
+      @@enabled
     end
 
     # The list of classes for which the current scope must be copied into the
@@ -66,14 +65,14 @@ module ParallelMinion
     #     - :timeout does not affect what happens to the Minion running the
     #       the task, it only affects how long #result will take to return.
     #     - The Minion will continue to run even after the timeout has been exceeded
-    #     - If :synchronous is true, or ParallelMinion::Minion.synchronous is
-    #       set to true, then :timeout is ignored and assumed to be Minion::INFINITE
+    #     - If :enabled is false, or ParallelMinion::Minion.enabled is false,
+    #       then :timeout is ignored and assumed to be Minion::INFINITE
     #       since the code is run in the calling thread when the Minion is created
     #
-    #   :synchronous [Boolean]
-    #     Whether the Minion should run in the current thread
+    #   :enabled [Boolean]
+    #     Whether the minion should run in a separate thread
     #     Not recommended in Production, but is useful for debugging purposes
-    #     Default: false
+    #     Default: ParallelMinion::Minion.enabled?
     #
     #   *args
     #     Any number of arguments can be supplied that are passed into the block
@@ -112,20 +111,24 @@ module ParallelMinion
 
       options = self.class.extract_options!(args).dup
 
-      @timeout  = (options.delete(:timeout) || Minion::INFINITE).to_f
+      @timeout       = (options.delete(:timeout) || Minion::INFINITE).to_f
       @description   = (options.delete(:description) || 'Minion').to_s
       @log_exception = options.delete(:log_exception)
-      @synchronous   = options.delete(:synchronous) || self.class.synchronous?
+      @enabled       = options.delete(:enabled)
+      @enabled       = self.class.enabled? if @enabled.nil?
 
       # Warn about any unknown options.
-      options.each_pair { |key,val| logger.warn "Ignoring unknown option: #{key.inspect} => #{val.inspect}" }
+      options.each_pair do |key,val|
+        logger.warn "Ignoring unknown option: #{key.inspect} => #{val.inspect}"
+        warn "ParallelMinion::Minion Ignoring unknown option: #{key.inspect} => #{val.inspect}"
+      end
 
       # Run the supplied block of code in the current thread for testing or
       # debugging purposes
-      if @synchronous == true
+      if @enabled == false
         begin
-          logger.info("Started synchronously #{@description}")
-          logger.benchmark_info("Completed synchronously #{@description}", log_exception: @log_exception) do
+          logger.info("Started in the current thread: #{@description}")
+          logger.benchmark_info("Completed in the current thread: #{@description}", log_exception: @log_exception) do
             @result = instance_exec(*args, &block)
           end
         rescue Exception => exc
@@ -189,12 +192,12 @@ module ParallelMinion
 
     # Returns [Boolean] whether the minion is still working on the assigned task
     def working?
-      synchronous? ? false : @thread.alive?
+      enabled? ? @thread.alive? : false
     end
 
     # Returns [Boolean] whether the minion has completed working on the task
     def completed?
-      synchronous? ? true : @thread.stop?
+      enabled? ? @thread.stop? : true
     end
 
     # Returns [Boolean] whether the minion failed while performing the assigned task
@@ -211,9 +214,9 @@ module ParallelMinion
       duration <= 0 ? 0 : duration
     end
 
-    # Returns [Boolean] whether synchronous mode has been enabled for this minion instance
-    def synchronous?
-      @synchronous
+    # Returns [Boolean] whether this minion is enabled to run in a separate thread
+    def enabled?
+      @enabled
     end
 
     # Returns the current scopes for each of the models for which scopes will be
@@ -225,7 +228,7 @@ module ParallelMinion
 
     protected
 
-    @@synchronous = false
+    @@enabled = true
     @@scoped_classes = []
 
     # Extract options from a hash.

data/lib/parallel_minion/railtie.rb

@@ -9,7 +9,7 @@ module ParallelMinion #:nodoc:
     #   Rails::Application.configure do
     #
     #     # Run Minions in the current thread to make debugging easier
-    #     config.parallel_minion.synchronous = true
+    #     config.parallel_minion.enabled = false
     #
     #     # Add a model so that its current scope is copied to the Minion
     #     config.parallel_minion.scoped_classes << MyScopedModel

data/lib/parallel_minion/version.rb

@@ -1,3 +1,3 @@
 module ParallelMinion #:nodoc
-  VERSION = "0.0.1"
+  VERSION = "0.1.0"
 end

data/test/minion_scope_test.rb

@@ -34,8 +34,8 @@ end
 class MinionScopeTest < Test::Unit::TestCase
 
   context ParallelMinion::Minion do
-    [false, true].each do |synchronous|
-      context ".new with synchronous: #{synchronous.inspect}" do
+    [false, true].each do |enabled|
+      context ".new with enabled: #{enabled.inspect}" do
         setup do
           Person.create(name: 'Jack', state: 'FL', zip_code: 38729)
           Person.create(name: 'John', state: 'FL', zip_code: 35363)
@@ -45,7 +45,7 @@ class MinionScopeTest < Test::Unit::TestCase
           Person.create(name: 'James', state: 'CA', zip_code: 123123)
           # Instruct Minions to adhere to any dynamic scopes for Person model
           ParallelMinion::Minion.scoped_classes << Person
-          ParallelMinion::Minion.synchronous = synchronous
+          ParallelMinion::Minion.enabled = enabled
         end
 
         teardown do

data/test/minion_test.rb

@@ -16,10 +16,10 @@ class MinionTest < Test::Unit::TestCase
 
   context ParallelMinion::Minion do
 
-    [false, true].each do |synchronous|
-      context ".new with synchronous: #{synchronous.inspect}" do
+    [false, true].each do |enabled|
+      context ".new with enabled: #{enabled.inspect}" do
         setup do
-          ParallelMinion::Minion.synchronous = synchronous
+          ParallelMinion::Minion.enabled = enabled
         end
 
         should 'without parameters' do
@@ -96,8 +96,8 @@ class MinionTest < Test::Unit::TestCase
 
         should 'timeout' do
           minion = ParallelMinion::Minion.new(description: 'Test', timeout: 100) { sleep 1 }
-          # Only Parallel Minions time-out when they exceed timeout
-          unless synchronous
+          # Only Parallel Minions time-out when they exceed the timeout
+          if enabled
             assert_equal nil, minion.result
           end
         end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: parallel_minion
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
 platform: ruby
 authors:
 - Reid Morrison
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-12-03 00:00:00.000000000 Z
+date: 2013-12-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: semantic_logger