in-parallel 0.1.4 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 954d90d6730f964c9b6af33e32f377681d762124
-  data.tar.gz: 5714427d316d51853feb018ee79f6dc9fa1fa687
+  metadata.gz: a79b9831824dee514b121f2911693c825c8bda13
+  data.tar.gz: 630845c4f1c5d2f22b83cb6e2a7467118371bf91
 SHA512:
-  metadata.gz: f8ff13104c07ec1b88d9b08b14f28bc939f61a1d5e26b47d927aab6aadb17ee7dedcf12794cbe62f87db2672bae85a8dc7fbf54ec08ba7a5e36b3374e5498908
-  data.tar.gz: 6c371de92cac2770beca520bc4cb261f85d8d51e0b06c59f0a755cc18c1b2908af1fa59081296ac4b5f14d4108d004cf32e539416cec8cf242c14f4490bea573
+  metadata.gz: 97a010bf1fdd61118c873ef60f807b65da112edb9867aa5c68823713cdd0d7e3bf1dae8cf8dc4fc3cc36d47b2f9f1722048106ed21f3cce486ebc1fdf2274ce8
+  data.tar.gz: 14c0e06a6e09f135e56656cc5496cbd6bb15a7613b3964acd306c33ab8259de8062f125675607b851a1ea4a8323c64fc32c46e940f4405212adca2b6275d1afb

data/MAINTAINERS.md

@@ -0,0 +1,8 @@
+# Reviewers/Maintainers For in-parallel
+
+in-parallel is maintained by Puppet's Quality Assurance (QA) Team. These people
+will be reviewing & merging PRs to the project.
+
+| Name           | Github                                              | Email                       |
+|:--------------:|:---------------------------------------------------:|:---------------------------:|
+| Sam Woods      | [samwoods1](https://github.com/samwoods1)           | <sam.woods@puppet.com>      |

data/README.md

@@ -1,17 +1,19 @@
 # in-parallel
 A lightweight Ruby library with very simple syntax, making use of process.fork for parallelization
 
-The other Ruby librarys that do parallel execution all support one primary use case - crunching through a large queue of small tasks as quickly and efficiently as possible.  This library primarily supports the use case of needing to run a few larger tasks in parallel and managing the stdout to make it easy to understand which processes are logging what. This library was created to be used by the Beaker test framework to enable parallel execution of some of the framework's tasks, and allow people within thier tests to execute code in parallel when wanted.  This solution does not check to see how many processors you have, it just forks as many processes as you ask for.  That means that it will handle a handful of parallel processes well, but could definitely overload your system with ruby processes if you try to spin up a LOT of processes.  If you're looking for something simple and light-weight and on either linux or mac (forking processes is not supported on Windows), then this solution could be what you want.
+Other popular Ruby librarys that do parallel execution support one primary use case - crunching through a large queue of small tasks as quickly and efficiently as possible.  This library primarily supports the use case of executing a few larger tasks in parallel and managing the stdout and return values to make it easy to understand which processes are logging what, and what the outcome of the execution was. This library was created to be used by Puppet's Beaker test framework to enable parallel execution of some of the framework's tasks, and allow people within thier tests to execute code in parallel when wanted.  This solution does not check to see how many processors you have, it just forks as many processes as you ask for.  That means that it will handle a handful of parallel processes well, but could definitely overload your system with ruby processes if you try to spin up a LOT of processes.  If you're looking for something simple and light-weight and on either linux or mac (forking processes is not supported on Windows), then this solution could be what you want.
 
-If you are looking for something a little more production ready, you should take a look at the [parallel](https://github.com/grosser/parallel) project.  In the future this library will extend the in the parallel gem to take advantage of all of it's useful features as well.
+If you are looking for something to support executing a lot of tasks in parallel as efficiently as possible, you should take a look at the [parallel](https://github.com/grosser/parallel) project.
 
 ## Methods:
 
-### InParallel.run_in_parallel(&block)
+### run_in_parallel(&block)
 1. You can put whatever methods you want to execute in parallel into a block, and each method will be executed in parallel (unless the method is defined in kernel). 
   1. Any methods further down the stack won't be affected, only the ones directly within the block.  
 2. You can assign the results to instance variables and it just works, no dealing with an array or map of results.
 3. Log STDOUT and STDERR chunked per process to the console so that it is easy to see what happened in which process.
+4. Waits for each process in realtime and logs immediately upon completion of each process
+5. If an exception is raised by a child process, it will immediately be re-raised in the primary process and kill all other still running child processes
 
 ```ruby
   def method_with_param(name)
@@ -28,8 +30,8 @@ If you are looking for something a little more production ready, you should take
   end
 
   # Example:
-  # will spawn 2 processes, (1 for each method) wait until they both complete, 
-  # and log chunked STDOUT/STDERR for each process:
+  # will spawn 2 processes, (1 for each method) wait until they both complete, log chunked STDOUT/STDERR for
+  # each process and assign the method return values to instance variables:
   InParallel.run_in_parallel {
     @result_1 = method_with_param('world')
     @result_2 = method_without_param
@@ -53,7 +55,7 @@ hello world
 hello world, bar
 ```
 
-### InParallel.run_in_background(ignore_results = true, &block)
+### run_in_background(ignore_results = true, &block)
 1. This does basically the same thing as run_in_parallel, except it does not wait for execution of all processes to complete, it returns immediately.
 2. You can optionally ignore results completely (default) or delay evaluating the results until later
 3. You can run multiple blocks in the background and then at some later point evaluate all of the results
@@ -68,7 +70,7 @@ hello world, bar
   end
   
   # Example 1 - ignore results
-  InParallel.run_in_background{
+  run_in_background{
     create_file_with_delay(TMP_FILE)
   }
   
@@ -79,13 +81,13 @@ hello world, bar
   puts(File.exists?(TMP_FILE)) # true
   
   # Example 2 - delay results
-  InParallel.run_in_background(false){
+  run_in_background(false){
     @result = create_file_with_delay(TMP_FILE)
   }
   
   # Do something else
   
-  InParallel.run_in_background(false){
+  run_in_background(false){
     @result2 = create_file_with_delay('/tmp/someotherfile.txt')
   }
   
@@ -93,7 +95,7 @@ hello world, bar
   puts @result >> "unresolved_parallel_result_0"
   
   # This assigns all instance variables within the block and writes STDOUT and STDERR from the process to console.
-  InParallel.get_background_results
+  wait_for_processes
   puts @result # true
   puts @result2 # true
   
@@ -111,19 +113,19 @@ hello world, bar
 ```
 STDOUT:
 ```
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51600'
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51601'
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51602'
+'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51600'
+'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51601'
+'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51602'
 
------- Begin output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51600
+------ Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51600
 foo
------- Completed output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51600
+------ Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51600
 
------- Begin output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51601
+------ Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51601
 bar
------- Completed output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51601
+------ Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51601
 
------- Begin output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51602
+------ Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51602
 baz
------- Completed output for /Users/samwoods/parallel_test/test/paralell_spec.rb:77:in `block (2 levels) in <top (required)>' - 51602
+------ Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51602
 ```

data/in_parallel/version.rb

@@ -1,3 +1,3 @@
-class InParallel
-  VERSION = Version = '0.1.4'
+module InParallel
+  VERSION = Version = '0.1.5'
 end

data/lib/in_parallel.rb

@@ -1,246 +1,301 @@
 require_relative 'parallel_enumerable'
+module InParallel
+  class InParallelExecutor
+    # How many seconds between outputting to stdout that we are waiting for child processes.
+    # 0 or < 0 means no signaling.
+    @@signal_interval = 30
+    @@process_infos = []
+    @@raise_error = nil
+    def self.process_infos
+      @@process_infos
+    end
 
-class InParallel
-  # How many seconds between outputting to stdout that we are waiting for child processes.
-  # 0 or < 0 means no signaling.
-  @@signal_interval = 30
-  @@process_infos = []
-  @@raise_error = nil
-  def self.process_infos
-    @@process_infos
-  end
+    @@background_objs = []
+    @@result_id = 0
 
-  @@background_objs = []
-  @@result_id = 0
+    @@pids = []
 
-  @@pids = []
+    @@main_pid = Process.pid
 
-  # Example - will spawn 2 processes, (1 for each method) wait until they both complete, and log STDOUT:
-  # InParallel.run_in_parallel {
-  #   @result_1 = on agents[0], 'puppet agent -t'
-  #   @result_2 = on agents[1], 'puppet agent -t'
-  # }
-  # NOTE: Only supports assigning instance variables within the block, not local variables
-  def self.run_in_parallel(&block)
-    if Process.respond_to?(:fork)
-      proxy = BlankBindingParallelProxy.new(self)
-      proxy.instance_eval(&block)
-      results_map = wait_for_processes
-      # pass in the 'self' from the block.binding which is the instance of the class
-      # that contains the initial binding call.
-      # This gives us access to the local and instance variables from that context.
-      return result_lookup(proxy, eval("self", block.binding), results_map)
+    def self.main_pid
+      @@main_pid
     end
-    puts 'Warning: Fork is not supported on this OS, executing block normally'
-    block.call
-  end
-
-  # Private method to lookup results from the results_map and replace the
-  # temp values with actual return values
-  def self.result_lookup(proxy_obj, target_obj, results_map)
-    vars = (proxy_obj.instance_variables)
-    results_map.keys.each { |tmp_result|
-      vars.each {|var|
-        if proxy_obj.instance_variable_get(var) == tmp_result
-          target_obj.instance_variable_set(var, results_map[tmp_result])
-          break
-        end
-      }
-    }
-  end
-  private_class_method :result_lookup
 
-  # Example - Will spawn a process in the background to run puppet agent on two agents and return immediately:
-  # Parallel.run_in_background {
-  #   @result = on agents[0], 'puppet agent -t'
-  #   @result_2 = on agents[1], 'puppet agent -t'
-  # }
-  # # Do something else here before waiting for the process to complete
-  #
-  # # Optionally wait for the processes to complete before continuing.
-  # # Otherwise use run_in_background(true) to clean up the process status and output immediately.
-  # Parrallel.get_background_results(self)
-  # NOTE: must call get_background_results to allow instance variables in calling object to be set,
-  # otherwise @result will evaluate to "unresolved_parallel_result_0"
-  def self.run_in_background(ignore_result = true, &block)
-    if Process.respond_to?(:fork)
-      proxy = BlankBindingParallelProxy.new(self)
-      proxy.instance_eval(&block)
-
-      if ignore_result
-        Process.detach(@@process_infos.last[:pid])
-        @@process_infos.pop
-      else
-        @@background_objs << {:proxy => proxy, :target => eval("self", block.binding)}
-        return process_infos.last[:tmp_result]
+    # Example - will spawn 2 processes, (1 for each method) wait until they both complete, and log STDOUT:
+    # InParallel.run_in_parallel {
+    #   @result_1 = method1
+    #   @result_2 = method2
+    # }
+    # NOTE: Only supports assigning instance variables within the block, not local variables
+    def self.run_in_parallel(&block)
+      if Process.respond_to?(:fork)
+        proxy = BlankBindingParallelProxy.new(block.binding)
+        proxy.instance_eval(&block)
+        return wait_for_processes(proxy, block.binding)
       end
-      return
+      puts 'Warning: Fork is not supported on this OS, executing block normally'
+      block.call
     end
-    puts 'Warning: Fork is not supported on this OS, executing block normally'
-    result = block.call
-    return nil if ignore_result
-    result
-  end
-
-  def self.get_background_results
-   results_map = wait_for_processes
-   # pass in the 'self' from the block.binding which is the instance of the class
-   # that contains the initial binding call.
-   # This gives us access to the instance variables from that context.
-   @@background_objs.each {|obj|
-     return result_lookup(obj[:proxy], obj[:target], results_map)
-   }
-  end
 
-  # Waits for all processes to complete and logs STDOUT and STDERR in chunks from any processes
-  # that were triggered from this Parallel class
-  def self.wait_for_processes
-    trap(:INT) do
-      puts "Warning, recieved interrupt.  Processing child results and exiting."
-      @@process_infos.each { |process_info|
-        # Send INT to each child process so it returns and can print stdout and stderr to console before exiting.
-        Process.kill("INT", process_info[:pid])
-      }
-    end
-    return unless Process.respond_to?(:fork)
-    # Custom process to wait so that we can do things like time out, and kill child processes if
-    # one process returns with an error before the others complete.
-    results_map = {}
-    timer = Time.now
-    while !@@process_infos.empty? do
-      if @@signal_interval > 0 && Time.now > timer + @@signal_interval
-        puts 'Waiting for child processes.'
-        timer = Time.now
-      end
-      @@process_infos.each {|process_info|
-        # wait up to half a second for each thread to see if it is complete, if not, check the next thread.
-        # returns immediately if the process has completed.
-        thr = process_info[:wait_thread].join(0.5)
-        unless thr.nil?
-          # the process completed, get the result and rethrow on error.
-          begin
-            # Print the STDOUT and STDERR for each process with signals for start and end
-            puts "\n------ Begin output for #{process_info[:method_sym]} - #{process_info[:pid]}\n"
-            puts File.new(process_info[:std_out], 'r').readlines
-            puts "------ Completed output for #{process_info[:method_sym]} - #{process_info[:pid]}\n"
-            result = process_info[:result].read
-            results_map[process_info[:tmp_result]] = (result.nil? || result.empty?) ? result : Marshal.load(result)
-            File.delete(process_info[:std_out])
-            # Kill all other processes and let them log their stdout before re-raising
-            # if a child process raised an error.
-            if results_map[process_info[:tmp_result]].is_a?(StandardError)
-              @@process_infos.each{|p_info|
-                begin
-                  Process.kill(0, p_info[:pid]) unless p_info[:pid] == process_info[:pid]
-                rescue StandardError
-                end
-              }
-            end
-          ensure
-            # close the read end pipe
-            process_info[:result].close unless process_info[:result].closed?
-            @@process_infos.delete(process_info)
-            @@raise_error = results_map[process_info[:tmp_result]] if results_map[process_info[:tmp_result]].is_a?(StandardError)
+    # Private method to lookup results from the results_map and replace the
+    # temp values with actual return values
+    def self.result_lookup(proxy_obj, target_obj, results_map)
+      vars = (proxy_obj.instance_variables)
+      results = []
+      results_map.keys.each { |tmp_result|
+        results << results_map[tmp_result]
+        vars.each {|var|
+          if proxy_obj.instance_variable_get(var) == tmp_result
+            target_obj.instance_variable_set(var, results_map[tmp_result])
             break
           end
-        end
+        }
       }
+      results
     end
+    private_class_method :result_lookup
 
-    # Reset the error in case the error is rescued
-    begin
-      raise @@raise_error unless @@raise_error.nil?
-    ensure
-      @@raise_error = nil
-    end
+    # Example - Will spawn a process in the background to run puppet agent on two agents and return immediately:
+    # Parallel.run_in_background {
+    #   @result_1 = method1
+    #   @result_2 = method2
+    # }
+    # # Do something else here before waiting for the process to complete
+    #
+    # # Optionally wait for the processes to complete before continuing.
+    # # Otherwise use run_in_background(true) to clean up the process status and output immediately.
+    # wait_for_processes(self)
+    # NOTE: must call get_background_results to allow instance variables in calling object to be set,
+    # otherwise @result_1 will evaluate to "unresolved_parallel_result_0"
+    def self.run_in_background(ignore_result = true, &block)
+      if Process.respond_to?(:fork)
+        proxy = BlankBindingParallelProxy.new(block.binding)
+        proxy.instance_eval(&block)
 
-    return results_map
-  end
+        if ignore_result
+          Process.detach(@@process_infos.last[:pid])
+          @@process_infos.pop
+        else
+          @@background_objs << {:proxy => proxy, :target => eval("self", block.binding)}
+          return process_infos.last[:tmp_result]
+        end
+        return
+      end
+      puts 'Warning: Fork is not supported on this OS, executing block normally'
+      result = block.call
+      return nil if ignore_result
+      result
+    end
 
-  # private method to execute some code in a separate process and store the STDOUT and STDERR for later retrieval
-  def self._execute_in_parallel(method_sym, obj = self, &block)
-    ret_val = nil
-    # Communicate the return value of the method or block
-    read_result, write_result = IO.pipe
-    pid = fork do
-      exit_status = 0
+    # Waits for all processes to complete and logs STDOUT and STDERR in chunks from any processes
+    # that were triggered from this Parallel class
+    def self.wait_for_processes(proxy = nil, binding = nil)
       trap(:INT) do
-        raise StandardError.new("Warning: Interrupt received; exiting...")
+        puts "Warning, recieved interrupt.  Processing child results and exiting."
+        @@process_infos.each { |process_info|
+          # Send INT to each child process so it returns and can print stdout and stderr to console before exiting.
+          Process.kill("INT", process_info[:pid])
+        }
+      end
+      return unless Process.respond_to?(:fork)
+      # Custom process to wait so that we can do things like time out, and kill child processes if
+      # one process returns with an error before the others complete.
+      results_map = {}
+      timer = Time.now
+      while !@@process_infos.empty? do
+        if @@signal_interval > 0 && Time.now > timer + @@signal_interval
+          puts 'Waiting for child processes.'
+          timer = Time.now
+        end
+        @@process_infos.each {|process_info|
+          # wait up to half a second for each thread to see if it is complete, if not, check the next thread.
+          # returns immediately if the process has completed.
+          thr = process_info[:wait_thread].join(0.5)
+          unless thr.nil?
+            # the process completed, get the result and rethrow on error.
+            begin
+              # Print the STDOUT and STDERR for each process with signals for start and end
+              puts "\n------ Begin output for #{process_info[:method_sym]} - #{process_info[:pid]}\n"
+              puts File.new(process_info[:std_out], 'r').readlines
+              puts "------ Completed output for #{process_info[:method_sym]} - #{process_info[:pid]}\n"
+              result = process_info[:result].read
+              results_map[process_info[:tmp_result]] = (result.nil? || result.empty?) ? result : Marshal.load(result)
+              File.delete(process_info[:std_out])
+              # Kill all other processes and let them log their stdout before re-raising
+              # if a child process raised an error.
+              if results_map[process_info[:tmp_result]].is_a?(Exception)
+                @@process_infos.each{|p_info|
+                  begin
+                    Process.kill('SIGINT', p_info[:pid]) unless p_info[:pid] == process_info[:pid]
+                  rescue StandardError
+                  end
+                }
+              end
+            ensure
+              # close the read end pipe
+              process_info[:result].close unless process_info[:result].closed?
+              @@process_infos.delete(process_info)
+              @@raise_error = results_map[process_info[:tmp_result]] if results_map[process_info[:tmp_result]].is_a?(Exception)
+              break
+            end
+          end
+        }
       end
-      Dir.mkdir('tmp') unless Dir.exists? 'tmp'
-      write_file = File.new("tmp/parallel_process_#{Process.pid}", 'w')
-
-      # IO buffer is 64kb, which isn't much... if debug logging is turned on,
-      # this can be exceeded before a process completes.
-      # Storing output in file rather than using IO.pipe
-      STDOUT.reopen(write_file)
-      STDERR.reopen(write_file)
 
+      # Reset the error in case the error is rescued
       begin
-        # close subprocess's copy of read_result since it only needs to write
-        read_result.close
-        ret_val = obj.instance_eval(&block)
-        # Write the result to the write_result IO stream.
-        # Have to serialize the value so it can be transmitted via IO
-        if(!ret_val.nil? && ret_val.singleton_methods && ret_val.class != TrueClass && ret_val.class != FalseClass && ret_val.class != Fixnum)
-          #in case there are other types that can't be duped
+        raise @@raise_error unless @@raise_error.nil?
+      ensure
+        @@raise_error = nil
+      end
+
+      results = []
+
+      # pass in the 'self' from the block.binding which is the instance of the class
+      # that contains the initial binding call.
+      # This gives us access to the local and instance variables from that context.
+      results = result_lookup(proxy, eval("self", binding), results_map) if proxy && binding
+
+      @@background_objs.each {|obj|
+         results = results.concat result_lookup(obj[:proxy], obj[:target], results_map)
+      }
+
+      return results
+    end
+
+    # private method to execute some code in a separate process and store the STDOUT and STDERR for later retrieval
+    def self._execute_in_parallel(method_sym, obj = self, &block)
+      ret_val = nil
+      # Communicate the return value of the method or block
+      read_result, write_result = IO.pipe
+      pid = fork do
+        exit_status = 0
+        trap(:INT) do
+          puts("Warning: Interrupt received in child process; exiting #{Process.pid}")
+          return
+        end
+        Dir.mkdir('tmp') unless Dir.exists? 'tmp'
+        write_file = File.new("tmp/parallel_process_#{Process.pid}", 'w')
+
+        # IO buffer is 64kb, which isn't much... if debug logging is turned on,
+        # this can be exceeded before a process completes.
+        # Storing output in file rather than using IO.pipe
+        STDOUT.reopen(write_file)
+        STDERR.reopen(write_file)
+
+        begin
+          # close subprocess's copy of read_result since it only needs to write
+          read_result.close
+          ret_val = obj.instance_eval(&block)
+          # Write the result to the write_result IO stream.
+          # Have to serialize the value so it can be transmitted via IO
+          if(!ret_val.nil? && ret_val.singleton_methods && ret_val.class != TrueClass && ret_val.class != FalseClass && ret_val.class != Fixnum)
+            #in case there are other types that can't be duped
+            begin
+              ret_val = ret_val.dup
+            rescue StandardError => err
+              puts "Warning: return value from child process #{ret_val} " +
+                       "could not be transferred to parent process: #{err.message}"
+            end
+          end
+          # In case there are other types that can't be dumped
           begin
-          ret_val = ret_val.dup
+            Marshal.dump(ret_val, write_result) unless ret_val.nil?
           rescue StandardError => err
             puts "Warning: return value from child process #{ret_val} " +
                      "could not be transferred to parent process: #{err.message}"
           end
+        rescue Exception => err
+          puts "Error in process #{pid}: #{err.message}"
+          # Return the error if an error is rescued so we can re-throw in the main process.
+          Marshal.dump(err, write_result)
+          exit_status = 1
+        ensure
+          write_result.close
+          exit exit_status
         end
-        # In case there are other types that can't be dumped
-        begin
-          Marshal.dump(ret_val, write_result) unless ret_val.nil?
-        rescue StandardError => err
-          puts "Warning: return value from child process #{ret_val} " +
-                   "could not be transferred to parent process: #{err.message}"
-        end
-      rescue StandardError => err
-        puts "Error in process #{pid}: #{err.message}"
-        # Return the error if an error is rescued so we can re-throw in the main process.
-        Marshal.dump(err, write_result)
-        exit_status = 1
-      ensure
-        write_result.close
-        exit exit_status
       end
+      write_result.close
+      # Process.detach returns a thread that will be nil if the process is still running and thr if not.
+      # This allows us to check to see if processes have exited without having to call the blocking Process.wait functions.
+      wait_thread = Process.detach(pid)
+      # store the IO object with the STDOUT and waiting thread for each pid
+      process_info = { :wait_thread => wait_thread,
+                       :pid => pid,
+                       :method_sym => method_sym,
+                       :std_out => "tmp/parallel_process_#{pid}",
+                       :result => read_result,
+                       :tmp_result => "unresolved_parallel_result_#{@@result_id}" }
+      @@process_infos.push(process_info)
+      @@result_id += 1
+      process_info
     end
-    write_result.close
-    # Process.detach returns a thread that will be nil if the process is still running and thr if not.
-    # This allows us to check to see if processes have exited without having to call the blocking Process.wait functions.
-    wait_thread = Process.detach(pid)
-    # store the IO object with the STDOUT and waiting thread for each pid
-    process_info = { :wait_thread => wait_thread,
-            :pid => pid,
-            :method_sym => method_sym,
-            :std_out => "tmp/parallel_process_#{pid}",
-            :result => read_result,
-            :tmp_result => "unresolved_parallel_result_#{@@result_id}" }
-    @@process_infos.push(process_info)
-    @@result_id += 1
-    process_info
-  end
 
-  # Proxy class used to wrap each method execution in a block and run it in parallel
-  # A block from Parallel.run_in_parallel is executed with a binding of an instance of this class
-  class BlankBindingParallelProxy < BasicObject
-    # Don't worry about running methods like puts or other basic stuff in parallel
-    include ::Kernel
+    # Proxy class used to wrap each method execution in a block and run it in parallel
+    # A block from Parallel.run_in_parallel is executed with a binding of an instance of this class
+    class BlankBindingParallelProxy < BasicObject
+      # Don't worry about running methods like puts or other basic stuff in parallel
+      include ::Kernel
 
-    def initialize(obj)
-      @object = obj
-      @result_id = 0
-    end
+      def initialize(obj)
+        @object = obj
+        @result_id = 0
+      end
 
-    # All methods within the block should show up as missing (unless defined in :Kernel)
-    def method_missing(method_sym, *args, &block)
-        out = ::InParallel._execute_in_parallel(method_sym) {@object.send(method_sym, *args, &block)}
-        puts "Forked process for '#{method_sym}' - PID = '#{out[:pid]}'\n"
-        out[:tmp_result]
+      # All methods within the block should show up as missing (unless defined in :Kernel)
+      def method_missing(method_sym, *args, &block)
+        if InParallelExecutor.main_pid == ::Process.pid
+          out = InParallelExecutor._execute_in_parallel("'#{method_sym.to_s}' #{caller_locations[0].to_s}", @object.eval('self')) {send(method_sym, *args, &block)}
+          puts "Forked process for '#{method_sym}' - PID = '#{out[:pid]}'\n"
+          out[:tmp_result]
+        end
+      end
     end
+  end
+
+  # Executes each method within a block in a different process
+  # Example - Will spawn a process in the background to execute each method
+  # Parallel.run_in_parallel {
+  #   @result_1 = method1
+  #   @result_2 = method2
+  # }
+  # NOTE - Only instance variables can be assigned the return values of the methods within the block.
+  # Local variables will not be assigned any values.
+  # @param [Block] block This method will yield to a block of code passed by the caller
+  # @return [Array<Result>, Result] the return values of each method within the block
+  def run_in_parallel(&block)
+    InParallelExecutor.run_in_parallel(&block)
+  end
+
+  # Forks a process for each method within a block and returns immediately
+  # Example 1 - Will fork a process in the background to execute each method and return immediately:
+  # Parallel.run_in_background {
+  #   @result_1 = method1
+  #   @result_2 = method2
+  # }
+  #
+  # Example 2 - Will fork a process in the background to execute each method, return immediately, then later
+  # wait for the process to complete, printing it's STDOUT and assigning return values to instance variables:
+  # Parallel.run_in_background(false) {
+  #   @result_1 = method1
+  #   @result_2 = method2
+  # }
+  # # Do something else here before waiting for the process to complete
+  #
+  # wait_for_processes
+  # NOTE: must call wait_for_processes to allow instance variables within the block to be set,
+  # otherwise results will evaluate to "unresolved_parallel_result_X"
+  # @param [Boolean] ignore_result True if you do not care about the STDOUT or return value of the methods executing in the background
+  # @param [Block] block This method will yield to a block of code passed by the caller
+  # @return [Array<Result>, Result] the return values of each method within the block
+  def run_in_background(ignore_result = true, &block)
+    InParallelExecutor.run_in_background(ignore_result, &block)
+  end
 
+  # Waits for all processes started by run_in_background to complete execution, then prints STDOUT
+  # and assigns return values to instance variables.  See :run_in_background
+  # @return [Array<Result>, Result] the temporary return values of each method within the block
+  def wait_for_processes
+    InParallelExecutor.wait_for_processes
   end
 end

data/lib/parallel_enumerable.rb

@@ -8,11 +8,11 @@ module Enumerable
     if Process.respond_to?(:fork) && count > 1
       method_sym ||= "#{caller_locations[0]}"
       each do |item|
-        out = InParallel._execute_in_parallel(method_sym) {block.call(item)}
+        out = InParallelExecutor._execute_in_parallel(method_sym) {block.call(item)}
         puts "'each_in_parallel' forked process for '#{method_sym}' - PID = '#{out[:pid]}'\n"
       end
       # return the array of values, no need to look up from the map.
-      return InParallel.wait_for_processes.values
+      return InParallel.wait_for_processes
     end
     puts 'Warning: Fork is not supported on this OS, executing block normally' unless Process.respond_to? :fork
     block.call

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: in-parallel
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.1.5
 platform: ruby
 authors:
 - samwoods1
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-05-31 00:00:00.000000000 Z
+date: 2016-06-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -59,6 +59,7 @@ extra_rdoc_files: []
 files:
 - Gemfile
 - LICENSE
+- MAINTAINERS.md
 - README.md
 - Rakefile
 - in_parallel.gemspec