in-parallel 0.1.12 → 0.1.15

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 71e83d885d72ee9199c9012d190f11c0e4fb7a23
-  data.tar.gz: 62274a9de6e81cd4772ce775904e49e9340b0346
+  metadata.gz: 0b5dc6a05d28155a49a6c5d07c3be8385583ef58
+  data.tar.gz: 6eb5ddb93219985c1513a3a5f1abe23391d5590b
 SHA512:
-  metadata.gz: a21762fb62f58b8a4b4eda90123bbefb41ed0dc078c07239b18a2ec2ba33b43ca5aadc10d99a8d7034ce31ffa662da081c8dbfc9a5030c45614160433c6a1a0e
-  data.tar.gz: dae9561fa380ebcfcbac4ea677fb44878b30d7248f389180578a089243fa3731e9ed815e666ae64bd4337edbf30fbd1e9475b6ffe52c26dd21ba4bf0922a709d
+  metadata.gz: 5f36a6e1b63a18514c9e17713d95d29725c5b6ddfb7cd91074a8e8b25c3bc8004dbcd272fad00c82ac4e155931673bc494d83497b8310e54dca074dabe384d39
+  data.tar.gz: 2a1d6707489ebe47beec5c6959394b66c57926843f377ff5cf24bad66124935afe35a1f171e84c2266e6ab2c229d2085168c8ceb6244c0846977fb534c11330a

data/CONTRIBUTING.md

@@ -0,0 +1,55 @@
+# How To Contribute To in-parallel
+
+## Getting Started
+
+* Make sure you have a [GitHub account](https://github.com/signup/free)
+* Fork the [in-parallel repository on GitHub](https://github.com/puppetlabs/in-parallel)
+
+## Making Changes
+
+* Create a topic branch from where you want to base your work.
+  * This is the `master` branch in the case of in-parallel
+  * To quickly create a topic branch based on master use `git checkout -b my_contribution master`. Do not work directly on the `master` branch.
+* Make commits of logical _working_ and _functional_ units.
+* Check for unnecessary whitespace with `git diff --check` before committing.
+* Make sure your commit messages are in the proper format.
+
+        (BKR-1234) Make the example in CONTRIBUTING imperative and concrete
+
+        Without this patch applied the example commit message in the CONTRIBUTING
+        document is not a concrete example.  This is a problem because the
+        contributor is left to imagine what the commit message should look like
+        based on a description rather than an example.  This patch fixes the
+        problem by making the example concrete and imperative.
+
+        The first line is a real life imperative statement with a ticket number
+        from our issue tracker.  The body describes the behavior without the patch,
+        why this is a problem, and how the patch fixes the problem when applied.
+
+* Make sure you have added [RSpec](http://rspec.info/) tests that exercise your new code.  These test should be located in the appropriate `in-parallel/spec/` subdirectory.  The addition of new methods/classes or the addition of code paths to existing methods/classes requires additional RSpec coverage.
+  * One should **NOT USE** the deprecated `should`/`stub` methods - **USE** `expect`/`allow`. Use of deprecated RSpec methods will result in your patch being rejected.  See a nice blog post from 2013 on [RSpec's new message expectation syntax](http://teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/).
+* Run the spec unit tests to assure nothing else was accidentally broken, using `rake test`
+  * **Bonus**: if possible ensure that `rake test` runs without failures for additional Ruby versions (1.9, 2.0, etc). in-parallel supports Ruby 1.9+, and breakage of support for other rubies will cause a patch to be rejected.
+* Make sure that if you have added new functionality of sufficiently high risk, and it can not be covered adequately via unit tests (mocking, requires disk, other classes, etc), you also include acceptance tests in your PR.
+* Make sure that you have added documentation using [Yard](http://yardoc.org/), new methods/classes without apporpriate documentation will be rejected.
+  * Run the yardoc tool to ensure that your yard documentation is properly formatted and complete
+  * `[bundle exec] yard doc`
+* Yard docs are great for other developers, but often are difficult to read for users. If your change impacts user-facing functionality, please include changes to the human-readable markdown docs starting at README.md
+* During the time that you are working on your patch the master in-parallel branch may have changed - you'll want to [rebase](http://git-scm.com/book/en/Git-Branching-Rebasing) before you submit your PR with `git rebase master`.  A successful rebase ensures that your patch will cleanly merge into in-parallel.
+* Submitted patches will be smoke tested through a series of acceptance level tests that ensures basic in-parallel functionality - the results of these tests will be evaluated by a in-parallel team member.  Failures associated with the submitted patch will result in the patch being rejected.
+
+## Submitting Changes
+
+* Sign the [Contributor License Agreement](http://links.puppet.com/cla).
+* Push your changes to a topic branch in _your_ fork of the repository.
+* Submit a pull request to [in-parallel](https://github.com/puppetlabs/in-parallel)
+* PRs are reviewed as time permits.
+
+# Additional Resources
+
+* [More information on contributing](http://links.puppet.com/contribute-to-puppet)
+* [Contributor License Agreement](http://links.puppet.com/cla)
+* [General GitHub documentation](http://help.github.com/)
+* [GitHub pull request documentation](http://help.github.com/send-pull-requests/)
+* Questions?  Comments?  Contact the in-parallel team at qa-team@puppet.com
+  * The keyword `in-parallel` is monitored and we'll get back to you as quick as we can.

data/Gemfile

@@ -1,4 +1,15 @@
 source 'https://rubygems.org'
 
+# in the Rakefile, so we require it in all groups
+gem 'rspec'                ,'~> 3.1.0'
+
+group :development do
+  gem 'simplecov'
+  gem 'rake', '>= 0.9.0'
+  #Documentation dependencies
+  gem 'yard'                 ,'~> 0'
+  gem 'markdown'             ,'~> 0'
+end
+
 # Specify your gem's dependencies in in_parallel.gemspec
 gemspec

data/Gemfile.lock

@@ -0,0 +1,68 @@
+PATH
+  remote: .
+  specs:
+    in-parallel (0.1.12)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (4.2.6)
+      i18n (~> 0.7)
+      json (~> 1.7, >= 1.7.7)
+      minitest (~> 5.1)
+      thread_safe (~> 0.3, >= 0.3.4)
+      tzinfo (~> 1.1)
+    diff-lcs (1.2.5)
+    docile (1.1.5)
+    i18n (0.7.0)
+    json (1.8.3)
+    kramdown (1.11.1)
+    logutils (0.6.1)
+    markdown (0.4.0)
+      kramdown (>= 0.13.7)
+      props (>= 0.2.0)
+      textutils (>= 0.2.0)
+    minitest (5.9.0)
+    props (1.1.2)
+    rake (10.5.0)
+    rspec (3.1.0)
+      rspec-core (~> 3.1.0)
+      rspec-expectations (~> 3.1.0)
+      rspec-mocks (~> 3.1.0)
+    rspec-core (3.1.7)
+      rspec-support (~> 3.1.0)
+    rspec-expectations (3.1.2)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.1.0)
+    rspec-mocks (3.1.3)
+      rspec-support (~> 3.1.0)
+    rspec-support (3.1.2)
+    rubyzip (1.2.0)
+    simplecov (0.11.2)
+      docile (~> 1.1.0)
+      json (~> 1.8)
+      simplecov-html (~> 0.10.0)
+    simplecov-html (0.10.0)
+    textutils (1.4.0)
+      activesupport
+      logutils (>= 0.6.1)
+      props (>= 1.1.2)
+      rubyzip (>= 1.0.0)
+    thread_safe (0.3.5)
+    tzinfo (1.2.2)
+      thread_safe (~> 0.1)
+    yard (0.9.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  in-parallel!
+  markdown (~> 0)
+  rake (>= 0.9.0)
+  rspec (~> 3.1.0)
+  simplecov
+  yard (~> 0)
+
+BUNDLED WITH
+   1.12.3

data/HISTORY.md

@@ -0,0 +1,23 @@
+# experimental_in-parallel_bump_and_tag_master - History
+## Tags
+* [LATEST - 8 Aug, 2016 (d026c624)](#LATEST)
+* [0.1.13 - 8 Aug, 2016 (26d19934)](#0.1.13)
+
+## Details
+### <a name = "LATEST">LATEST - 8 Aug, 2016 (d026c624)
+
+* (GEM) update in-parallel version to 0.1.14 (d026c624)
+
+* Merge pull request #14 from samwoods1/add_jjb_pipelines (269bc350)
+
+
+```
+Merge pull request #14 from samwoods1/add_jjb_pipelines
+
+(maint) Consistent naming of in-parallel
+```
+* (maint) Consistent naming of in-parallel (6fbb442d)
+
+### <a name = "0.1.13">0.1.13 - 8 Aug, 2016 (26d19934)
+
+* Initial release.

data/MAINTAINERS.md

@@ -6,3 +6,4 @@ will be reviewing & merging PRs to the project.
 | Name           | Github                                              | Email                       |
 |:--------------:|:---------------------------------------------------:|:---------------------------:|
 | Sam Woods      | [samwoods1](https://github.com/samwoods1)           | <sam.woods@puppet.com>      |
+| Zach Reichert  | [zreichert](https://github.com/zreichert)           | <zach.reichert@puppet.com>  |

data/README.md

@@ -1,20 +1,33 @@
-# in-parallel
+
 A lightweight Ruby library with very simple syntax, making use of Process.fork to execute code in parallel.
 
-Other popular Ruby libraries that do parallel execution support one primary use case - crunching through a large queue of small tasks as quickly and efficiently as possible.  This library primarily supports the use case of executing a few larger tasks in parallel and managing the stdout and return values to make it easy to understand which processes are logging what, and what the outcome of the execution was. This library was created to be used by Puppet's Beaker test framework to enable parallel execution of some of the framework's tasks, and allow users to execute code in parallel within their tests.  This solution does not check to see how many processors you have, it just forks as many processes as you ask for.  That means that it will handle a handful of parallel processes well, but could definitely overload your system with ruby processes if you try to spin up a LOT of processes.  If you're looking for something intuitive and simple, but with a slight memory and processing overhead, and are on either Linux or Mac (forking processes is not supported on Windows), then this solution is  what you want.
+# Use Cases
+Many other Ruby libraries that simplify parallel execution support one primary use case - crunching through a large queue of small, similar tasks as quickly and efficiently as possible.  This library primarily supports the use case of executing a few larger and unrelated tasks in parallel, automatically managing the stdout and passing return values back to the main process. This library was created to be used by Puppet's Beaker test framework to enable parallel execution of some of the framework's tasks, and allow users to execute code in parallel within their tests.
 
 If you are looking for something that excels at executing a large queue of tasks in parallel as efficiently as possible, you should take a look at the [parallel](https://github.com/grosser/parallel) project.
 
-## Methods:
+# Install
+```gem install in-parallel```
+
+# Usage
+```include InParallel``` to use as a mix-in
+
+The methods below allow you to fork processes to execute multiple methods or blocks within an enumerable in parallel.  They all have this common behavior:
 
+1. STDOUT is captured for each forked process and logged all at once when the process completes or is terminated.
+1. By default execution of processes in parallel will wait until execution of all processes are complete before continuing (with the exception of run_in_background).
+    1. You can specify the parameter kill_all_on_error=true if you want to immediately exit all forked processes when an error executing any of the forked processes occurs.
+1. When the forked process raises an exception or exits with a non zero exit code, an exception will be raised in the main process.
+1. Terminating the main process with 'ctrl-c' or killing the process in some other way will immediately cause all forked processes to be killed and log their STDOUT up to that point.
+1. If the result of the method or block can be marshalled, it will be returned as though it was executed within the same process.  If the result cannot be marshalled a warning is produced and the return value will be nil.
+    1. NOTE: results of methods within run_in_parallel can be assigned to instance or class variables, but not local variables.  See examples below.
+1. Will timeout (stop execution and raise an exception) based on a global timeout value, or timeout parameter.
+
+## Methods
 ### run_in_parallel(timeout=nil, kill_all_on_error = false, &block)
 1. Each method in a block will be executed in parallel (unless the method is defined in Kernel or BaseObject).
     1. Any methods further down the stack won't be affected, only the ones directly within the block.
-2. You can assign return values to instance variables and it 'just works'.
-3. Log STDOUT and STDERR chunked per process to the console so that it is easy to see what happened in which process.
-4. Waits for each process in realtime and logs immediately upon completion of each process
-5. If an exception is raised by a child process, it will optionally (kill_all_on_error) be re-raised in the primary process and kill all other still running child processes. The default will wait for all processes to complete execution before re-raising any unhandled exception from the child processes.
-6. Times out by default at 30 minutes. Timeout default can be changed with InParallel::InParallelExecutor.parallel_default_timeout=X, or you can set the timeout param when calling the method
+1. Waits for each process in realtime and logs immediately upon completion of each process
 
 ```ruby
   def method_with_param(name)
@@ -40,8 +53,7 @@ If you are looking for something that excels at executing a large queue of tasks
   
   puts "#{@result_1}, #{@result_2[:foo]}"
 ```
-  
-STDOUT would be:
+stdout:
 ```
 Forked process for 'method_with_param' - PID = '49398'
 Forked process for 'method_without_param' - PID = '49399'
@@ -55,41 +67,19 @@ hello world
 ------ Completed output for method_without_param - 49399
 hello world, bar
 ```
-
 ### Enumerable.each_in_parallel(identifier=nil, timeout=(InParallel::InParallelExecutor.timeout), kill_all_on_error = false, &block)
 1. This is very similar to other solutions, except that it directly extends the Enumerable class with an each_in_parallel method, giving you the ability to pretty simply spawn a process for any item in an array or map.
-2. Identifies the block location (or caller location if the block does not have a source_location) in the console log to make it clear which block is being executed
-3. identifier param is only for logging, otherwise it will use the block source location.
-4. If an exception is raised by a child process, it will optionally (kill_all_on_error) be re-raised in the primary process and kill all other still running child processes. The default will wait for all processes to complete execution before re-raising any unhandled exception from the child processes.
-5. Times out by default at 30 minutes. Timeout default can be changed with InParallel::InParallelExecutor.parallel_default_timeout=X, or you can set the timeout param when calling the method
+1. Identifies the block location (or caller location if the block does not have a source_location) in the console log to make it clear which block is being executed
+1. Identifier param is only for logging, otherwise it will use the block source location.
 
 ```ruby
   ["foo", "bar", "baz"].each_in_parallel { |item| puts item }
-
-```
-STDOUT:
-```
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51600'
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51601'
-'each_in_parallel' spawned process for '/Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>'' - PID = '51602'
-
------- Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51600
-foo
------- Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51600
-
------- Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51601
-bar
------- Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51601
-
------- Begin output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51602
-baz
------- Completed output for /Users/samwoods/parallel_test/test.rb:77:in `block (2 levels) in <top (required)>' - 51602
 ```
 
 ### run_in_background(ignore_results = true, &block)
 1. This does basically the same thing as run_in_parallel, except it does not wait for execution of all processes to complete, it returns immediately.
-2. You can optionally ignore results completely (default) or delay evaluating the results until later
-3. You can run multiple blocks in the background and then at some later point evaluate all of the results
+1. You can optionally ignore results completely (default) or delay evaluating the results until later
+1. You can run multiple blocks in the background and then at some later point evaluate all of the results
 
 ```ruby
   TMP_FILE = '/tmp/test_file.txt'
@@ -108,7 +98,8 @@ baz
   sleep(3)
   # Should exist once the delay from create_file_with_delay is done
   puts(File.exists?(TMP_FILE)) # true
-  
+  ```
+  ```ruby
   # Example 2 - delay results
   run_in_background(false) { @result = create_file_with_delay(TMP_FILE) }
   
@@ -128,5 +119,21 @@ baz
 
 ### wait_for_processes(timeout=nil, kill_all_on_error = false)
 1. Used only after run_in_background with ignore_results=false
-2. Optional args for timeout and kill_all_on_error
-3. See run_in_background for examples
+1. Optional args for timeout and kill_all_on_error
+1. See run_in_background for examples
+
+## Global Options
+You can get or set the following values to set global defaults.  These defaults can also be specified per execution by supplying the values as parameters to the parallel methods.
+```
+  # How many seconds to wait between logging a 'Waiting for child processes.' message. Defaults to 30 seconds
+  parallel_signal_interval
+
+  # How many seconds to wait before timing out a forked child process and raising an exception. Defaults to 30 minutes.
+  parallel_default_timeout
+
+  # The log level to log output.
+  # NOTE: The entire contents of STDOUT for forked processes will be printed to console regardless of
+  # the log level set here.
+  @logger.log_level
+```
+

data/Rakefile

@@ -1,2 +1,10 @@
 require "bundler/gem_tasks"
-task :default => :spec
+require 'rspec/core/rake_task'
+
+task :default => :test
+
+desc "Run spec tests"
+RSpec::Core::RakeTask.new(:test) do |t|
+  t.rspec_opts = ['--color']
+  t.pattern = 'spec/'
+end

data/lib/in-parallel/version.rb

@@ -0,0 +1,3 @@
+module InParallel
+  VERSION = '0.1.15'
+end

data/lib/in_parallel.rb

@@ -313,7 +313,7 @@ module InParallel
       # All methods within the block should show up as missing (unless defined in :Kernel)
       def method_missing(method_sym, *args, &block)
         if InParallelExecutor.main_pid == ::Process.pid
-          out = InParallelExecutor._execute_in_parallel("'#{method_sym.to_s}' #{caller_locations[0].to_s}",
+          out = InParallelExecutor._execute_in_parallel("'#{method_sym.to_s}' #{caller[0].to_s}",
                                                         @object.eval('self')) { send(method_sym, *args, &block) }
           out[:tmp_result]
         end

data/lib/parallel_enumerable.rb

@@ -10,7 +10,7 @@ module Enumerable
   # @return [Array<Object>] results - the return value of each block execution.
   def each_in_parallel(identifier=nil, timeout=(InParallel::InParallelExecutor.parallel_default_timeout), kill_all_on_error = false, &block)
     if InParallel::InParallelExecutor.fork_supported? && count > 1
-      identifier ||= "#{caller_locations[0]}"
+      identifier ||= "#{caller[0]}"
       each do |item|
         InParallel::InParallelExecutor._execute_in_parallel(identifier) { block.call(item) }
       end

data/spec/in-paralell_spec.rb

@@ -0,0 +1,251 @@
+require 'rspec'
+require_relative('../lib/in_parallel')
+include InParallel
+TMP_FILE = Dir.mktmpdir + 'test_file.txt'
+
+class SingletonTest
+  def initialize
+    @test_data = [1, 2, 3]
+  end
+
+  def get_test_data
+    @test_data
+  end
+end
+
+class SingletonWrapper
+  def initialize
+    @instance_var = get_singleton_class
+    singleton_class.class_eval do
+      @@x = "foo"
+      @x = 'bar'
+    end
+  end
+
+  def get_instance_var
+    @instance_var
+  end
+end
+
+def get_wrapper
+  SingletonWrapper.new
+end
+
+def get_singleton_class
+  test = SingletonTest.new
+
+  def test.someval
+    "someval"
+  end
+
+  return test
+end
+
+# Helper functions for the unit tests
+def method_with_param(param)
+  puts "foo"
+  puts "bar + #{param} \n"
+  return "bar + #{param}"
+end
+
+def method_without_param
+  ret_val = { :foo => "bar" }
+  puts ret_val
+  return ret_val
+end
+
+def simple_puts(my_string)
+  puts my_string
+end
+
+def create_file_with_delay(file_path, wait=2)
+  sleep wait
+  File.open(file_path, 'w') { |f| f.write('contents') }
+  return true
+end
+
+def get_pid
+  return Process.pid
+end
+
+def raise_an_error
+  raise StandardError.new('An error occurred')
+end
+
+#Tests
+describe '.run_in_parallel' do
+  before do
+    File.delete(TMP_FILE) if File.exists?(TMP_FILE)
+  end
+
+  it 'should run methods in another process' do
+    run_in_parallel do
+      @result  = get_pid
+      @result2 = get_pid
+    end
+
+    expect(@result).to_not eq(Process.pid)
+    expect(@result2).to_not eq(Process.pid)
+    expect(@result).to_not eq(@result2)
+  end
+  it 'should return correct values' do
+    start_time = Time.now
+
+    run_in_parallel do
+      @result_from_test = method_with_param('blah')
+      @result_2         = method_without_param
+    end
+    # return values for instance variables should be set correctly
+    expect(@result_from_test).to eq 'bar + blah'
+    # should be able to return objects (not just strings)
+    expect(@result_2).to eq({ :foo => "bar" })
+  end
+
+  it "should return a singleton class value" do
+
+    run_in_parallel { @result = get_singleton_class }
+
+    expect(@result.get_test_data).to eq([1, 2, 3])
+  end
+
+  it "should return an object with an instance variable set to an object containing singleton methods" do
+    run_in_parallel { @result = get_wrapper }
+    expect(@result.get_instance_var.get_test_data).to eq([1, 2, 3])
+  end
+
+  it "should raise an exception and return immediately with kill_all_on_error and one of the processes errors." do
+    expect { run_in_parallel(nil, true) do
+      @result   = get_singleton_class
+      @result_2 = raise_an_error
+      @result_3 = create_file_with_delay(TMP_FILE)
+    end }.to raise_error StandardError
+
+    expect(@result_3).to_not eq(true)
+  end
+
+  it "should raise an exception and let all processes complete when one of the processes errors." do
+    expect { run_in_parallel(nil, false) do
+      @result   = get_singleton_class
+      @result_2 = raise_an_error
+      @result_3 = create_file_with_delay(TMP_FILE)
+    end }.to raise_error StandardError
+
+    expect(@result_3).to eq(true)
+  end
+
+  it "should not run in parallel if forking is not supported" do
+    InParallel::InParallelExecutor.class_variable_set(:@@supported, nil)
+    expect(Process).to receive(:respond_to?).with(:fork).and_return(false).once
+    expect(InParallel::InParallelExecutor.logger).to receive(:warn).with("Warning: Fork is not supported on this OS, executing block normally")
+
+    run_in_parallel do
+      @result_from_test = method_with_param('blah')
+      @result_2         = get_pid
+    end
+
+    expect(@result_from_test).to eq 'bar + blah'
+    expect(@result_2).to eq Process.pid
+  end
+
+  # it "should chunk stdout per process" do
+  #   expect {run_in_parallel {
+  #     simple_puts('foobar')
+  #   }}.to output(/------ Begin output for simple_puts.*foobar.*------ Completed output for simple_puts/).to_stdout
+  # end
+end
+
+describe '.run_in_background' do
+  before do
+    File.delete(TMP_FILE) if File.exists?(TMP_FILE)
+  end
+
+  it 'should run in the background' do
+    run_in_background { @result = create_file_with_delay(TMP_FILE) }
+
+    start = Time.now
+    # Should not exist immediately upon block completion
+    expect(File.exists? TMP_FILE).to eq false
+    # Give this some time to complete since it takes longer on the vmpooler vms
+    file_exists = false
+    while Time.now < start + 10 do
+      if File.exists? TMP_FILE
+        file_exists = true
+        break
+      end
+    end
+    # Should exist once the delay in create_file_with_delay is done
+    expect(file_exists).to eq true
+  end
+
+  it 'should allow you to get results if ignore_results is false' do
+    @block_result = run_in_background(false) { @result = create_file_with_delay(TMP_FILE) }
+    wait_for_processes
+    # We should get the correct value assigned for the method result
+    expect(@result).to eq true
+  end
+
+end
+
+describe '.wait_for_processes' do
+  after do
+    InParallel::InParallelExecutor.parallel_default_timeout = 1200
+  end
+  it 'should timeout when the default timeout value is hit' do
+    @block_result                                           = run_in_background(false) do
+      @result = create_file_with_delay(TMP_FILE, 30)
+    end
+    InParallel::InParallelExecutor.parallel_default_timeout = 0.1
+    expect { wait_for_processes }.to raise_error RuntimeError
+  end
+
+  it 'should timeout when a specified timeout value is hit' do
+    @block_result = run_in_background(false) do
+      @result  = create_file_with_delay(TMP_FILE, 30)
+      @result2 = method_without_param
+    end
+    expect { wait_for_processes(0.1) }.to raise_error RuntimeError
+  end
+end
+
+describe '.each_in_parallel' do
+  it 'should run each iteration in a separate process' do
+    pids = [1, 2, 3].each_in_parallel { Process.pid }
+    expect(pids.detect { |pid| pids.count(pid) > 1 }).to be_nil
+  end
+
+  it 'should return correct values' do
+    start_time = Time.now
+    items      = ['foo', 'bar', 'baz', 'blah', 'foobar'].each_in_parallel do |item|
+      sleep(Random.rand(1.0))
+      item
+    end
+    # return values should be an array of the returned items in the last line of the block, in correct order
+    expect(['foo', 'bar', 'baz', 'blah', 'foobar']).to eq(items)
+    # time should be less than combined delay in the 3 block calls
+    expect(expect(Time.now - start_time).to be < 5)
+  end
+
+  it 'should run each iteration of a map in parallel' do
+    items = ['foo', 'bar', 'baz'].map.each_in_parallel do |item|
+      puts item
+      item
+    end
+    # return values should be an array of the returned items in the last line of the block, in correct order
+    expect(items).to eq(['foo', 'bar', 'baz'])
+  end
+
+  it 'should not run in parallel if there is only 1 item in the enumerator' do
+    expect(InParallel::InParallelExecutor.logger).to_not receive(:info).with(/Forked process for/)
+    expect(["foo"].map.each_in_parallel { Process.pid }[0]).to eq(Process.pid)
+  end
+
+  it 'should allow you to specify the method_sym' do
+    allow(InParallel::InParallelExecutor.logger).to receive(:info).with(anything())
+    expect(InParallel::InParallelExecutor.logger).to receive(:info).with(/Forked process for my_method/).exactly(3).times
+
+    [1, 2, 3].each_in_parallel('my_method') { |item|
+      puts item
+    }
+  end
+
+end

metadata

@@ -1,72 +1,42 @@
 --- !ruby/object:Gem::Specification
 name: in-parallel
 version: !ruby/object:Gem::Version
-  version: 0.1.12
+  version: 0.1.15
 platform: ruby
 authors:
 - samwoods1
 autorequire: 
-bindir: exe
+bindir: bin
 cert_chain: []
-date: 2016-07-14 00:00:00.000000000 Z
-dependencies:
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.11'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.11'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-description: The other Ruby librarys that do parallel execution all support one primary
-  use case - crunching through a large queue of small tasks as quickly and efficiently
-  as possible. This library primarily supports the use case of needing to run a few
-  larger tasks in parallel and managing the stdout to make it easy to understand which
-  processes are logging what. This library was created to be used by the Beaker test
-  framework to enable parallel execution of some of the framework's tasks, and allow
-  people within thier tests to execute code in parallel when wanted. This solution
-  does not check to see how many processors you have, it just forks as many processes
-  as you ask for. That means that it will handle a handful of parallel processes well,
-  but could definitely overload your system with ruby processes if you try to spin
-  up a LOT of processes. If you're looking for something simple and light-weight and
-  on either linux or mac (forking processes is not supported on Windows), then this
-  solution could be what you want.
+date: 2016-08-08 00:00:00.000000000 Z
+dependencies: []
+description: Many other Ruby libraries that simplify parallel execution support one
+  primary use case - crunching through a large queue of small, similar tasks as quickly
+  and efficiently as possible.  This library primarily supports the use case of executing
+  a few larger and unrelated tasks in parallel, automatically managing the stdout
+  and passing return values back to the main process. This library was created to
+  be used by Puppet's Beaker test framework to enable parallel execution of some of
+  the framework's tasks, and allow users to execute code in parallel within their
+  tests.
 email:
 - sam.woods@puppetlabs.com
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- CONTRIBUTING.md
 - Gemfile
+- Gemfile.lock
+- HISTORY.md
 - LICENSE
 - MAINTAINERS.md
 - README.md
 - Rakefile
-- in_parallel.gemspec
-- in_parallel/version.rb
+- lib/in-parallel/version.rb
 - lib/in_parallel.rb
 - lib/parallel_enumerable.rb
 - lib/parallel_logger.rb
+- spec/in-paralell_spec.rb
 homepage: https://github.com/puppetlabs/in-parallel
 licenses:
 - MIT

data/in_parallel.gemspec

@@ -1,34 +0,0 @@
-# coding: utf-8
-lib = File.expand_path('../lib', __FILE__)
-$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
-require_relative 'in_parallel/version'
-
-Gem::Specification.new do |spec|
-  spec.name          = "in-parallel"
-  spec.version       = InParallel::VERSION
-  spec.authors       = ["samwoods1"]
-  spec.email         = ["sam.woods@puppetlabs.com"]
-
-  spec.summary       = "A lightweight library to execute a handful of tasks in parallel with simple syntax"
-  spec.description   = "The other Ruby librarys that do parallel execution all support one primary use case " +
-      "- crunching through a large queue of small tasks as quickly and efficiently as possible. This library " +
-      "primarily supports the use case of needing to run a few larger tasks in parallel and managing the " +
-      "stdout to make it easy to understand which processes are logging what. This library was created to be " +
-      "used by the Beaker test framework to enable parallel execution of some of the framework's tasks, and " +
-      "allow people within thier tests to execute code in parallel when wanted. This solution does not check " +
-      "to see how many processors you have, it just forks as many processes as you ask for. That means that it " +
-      "will handle a handful of parallel processes well, but could definitely overload your system with ruby " +
-      "processes if you try to spin up a LOT of processes. If you're looking for something simple and " +
-      "light-weight and on either linux or mac (forking processes is not supported on Windows), then this " +
-      "solution could be what you want."
-  spec.homepage      = "https://github.com/puppetlabs/in-parallel"
-  spec.license       = "MIT"
-
-  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
-  spec.bindir        = "exe"
-  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
-  spec.require_paths = ["lib"]
-
-  spec.add_development_dependency "bundler", "~> 1.11"
-  spec.add_development_dependency "rake", "~> 10.0"
-end

data/in_parallel/version.rb

@@ -1,3 +0,0 @@
-module InParallel
-  VERSION = Version = '0.1.12'
-end