in-parallel 0.1.17 → 1.0.0

checksums.yaml

@@ -1,15 +1,7 @@
 ---
-!binary "U0hBMQ==":
-  metadata.gz: !binary |-
-    NWY0MDhlZmFhMGU4ZjNjY2U0YTAyYTJmOTJkYWExMWRhNTFiNTE2Mw==
-  data.tar.gz: !binary |-
-    NzQ4MzE5Yjk0NDc5NjY3YjUxOWM4ODY0NGE2NTk5ZWJiNmY0NWQwMA==
+SHA256:
+  metadata.gz: 2dc8f03c43e170dbb67bef4ababe42609c769a03202ede4fbb6248a88be11fd6
+  data.tar.gz: 13f4848ebf63e8615d5442d3ace5e17ba7c771a2f14e7a8ac6bb1a107a8999f6
 SHA512:
-  metadata.gz: !binary |-
-    YzMwMWI3MGI2N2FiMDljMDUwM2JkNmVlNzI2ODRmOWM5NmJjYzU3NjBkMDdi
-    MTE5NDI0OGYzMzQ4MzNhNDdkOGYyZDBkYTBmOTgxNDQ2ZWU1Zjc1YmUzNDQy
-    OWU2NTJiODYzODE5NWUxODVlOTQ0MjMzZjM1ZWQ4NGE3YWJjMGE=
-  data.tar.gz: !binary |-
-    Y2UyNDhlZjFjNWQzZjNkMWMyOTVlNjljZjkzOWJhZWQ2ZDM1MmUwNTNlNTJm
-    NDY4MTBhYjVmMGJkMzAzMDU0OWE0NDMwZDA1ODc4MGFkZjNjZWQ5ZDBlOGZm
-    NTYxYjVlZGZkYTFjN2NlNTIxZGJmOGJlYjQ2NTVmNDFhNjc1MDk=
+  metadata.gz: '0920d576695d90a654c77e3d59368783f1b03b8c7cdfd302779b4923ab5fed1f67c8e9dacef05bb39dfedfc13a0c18ec594975bbfcd91b276decd147afe94fd6'
+  data.tar.gz: d56bf699d25ab60872463c6883ba774278de655d1996a1e75f661c52e1faf729bd47101e7316ab1dbbb3705a7c58ce412af630bbcb54a2d2c4c8f70c171ffaeb

data/CHANGELOG.md

@@ -0,0 +1,152 @@
+# Changelog
+
+## [1.0.0](https://github.com/puppetlabs/in-parallel/tree/1.0.0) (2023-03-15)
+
+[Full Changelog](https://github.com/puppetlabs/in-parallel/compare/0.1.17...1.0.0)
+
+**Merged pull requests:**
+
+- \(RE-15226\) Update codeowners and add standard workflows [\#20](https://github.com/puppetlabs/in-parallel/pull/20) ([yachub](https://github.com/yachub))
+- \(maint\) update exists? -\> exist? for ruby 3.2 compatibility [\#19](https://github.com/puppetlabs/in-parallel/pull/19) ([tvpartytonight](https://github.com/tvpartytonight))
+
+## [0.1.17](https://github.com/puppetlabs/in-parallel/tree/0.1.17) (2017-02-07)
+
+[Full Changelog](https://github.com/puppetlabs/in-parallel/compare/0.1.16...0.1.17)
+
+**Merged pull requests:**
+
+- \(maint\) Properly handle non-parallel enumerables [\#17](https://github.com/puppetlabs/in-parallel/pull/17) ([nicklewis](https://github.com/nicklewis))
+
+## [0.1.16](https://github.com/puppetlabs/in-parallel/tree/0.1.16) (2017-02-06)
+
+[Full Changelog](https://github.com/puppetlabs/in-parallel/compare/0.1.15...0.1.16)
+
+## [0.1.15](https://github.com/puppetlabs/in-parallel/tree/0.1.15) (2017-02-03)
+
+[Full Changelog](https://github.com/puppetlabs/in-parallel/compare/0.1.14...0.1.15)
+
+**Merged pull requests:**
+
+- \(maint\) Avoid deadlock with large results [\#16](https://github.com/puppetlabs/in-parallel/pull/16) ([nicklewis](https://github.com/nicklewis))
+- \(maint\) Revert name change of in\_parallel.rb [\#15](https://github.com/puppetlabs/in-parallel/pull/15) ([samwoods1](https://github.com/samwoods1))
+
+## [0.1.14](https://github.com/puppetlabs/in-parallel/tree/0.1.14) (2016-08-08)
+
+[Full Changelog](https://github.com/puppetlabs/in-parallel/compare/0.1.13...0.1.14)
+
+**Merged pull requests:**
+
+- \(maint\) Consistent naming of in-parallel [\#14](https://github.com/puppetlabs/in-parallel/pull/14) ([samwoods1](https://github.com/samwoods1))
+
+# experimental_in-parallel_bump_and_tag_master - History
+## Tags
+* [LATEST - 6 Feb, 2017 (8e97ff25)](#LATEST)
+* [0.1.16 - 6 Feb, 2017 (0d5030c3)](#0.1.16)
+* [0.1.15 - 3 Feb, 2017 (ff16929c)](#0.1.15)
+* [0.1.14 - 8 Aug, 2016 (ce331dbd)](#0.1.14)
+* [0.1.13 - 8 Aug, 2016 (26d19934)](#0.1.13)
+
+## Details
+### <a name = "LATEST">LATEST - 6 Feb, 2017 (8e97ff25)
+
+* (GEM) update in-parallel version to 0.1.17 (8e97ff25)
+
+* Merge pull request #17 from nicklewis/handle-non-parallel-enumerables (dca0b0a5)
+
+
+```
+Merge pull request #17 from nicklewis/handle-non-parallel-enumerables
+
+(maint) Properly handle non-parallel enumerables
+```
+* (maint) Properly handle non-parallel enumerables (b041b864)
+
+
+```
+(maint) Properly handle non-parallel enumerables
+
+For Enumerables containing 0 or 1 items, the #each_in_parallel method
+was improperly calling the block without an argument, then calling it
+again properly but not returning the result of the block.
+
+The #each method returns the Enumerable that was it was called on,
+rather than the value of the block. This needs to be #map instead, to
+actually return an array of the one or zero values. The tests weren't
+catching this because they were effectively passing `identity` as the
+block, nullifying the distinction between #each and #map.
+```
+### <a name = "0.1.16">0.1.16 - 6 Feb, 2017 (0d5030c3)
+
+* (HISTORY) update in-parallel history for gem release 0.1.16 (0d5030c3)
+
+* (GEM) update in-parallel version to 0.1.16 (27b497ea)
+
+### <a name = "0.1.15">0.1.15 - 3 Feb, 2017 (ff16929c)
+
+* (HISTORY) update in-parallel history for gem release 0.1.15 (ff16929c)
+
+* (GEM) update in-parallel version to 0.1.15 (206a62fe)
+
+* Merge pull request #16 from nicklewis/support-large-results (4d644d88)
+
+
+```
+Merge pull request #16 from nicklewis/support-large-results
+
+(maint) Avoid deadlock with large results
+```
+* (maint) Avoid deadlock with large results (a5a9c174)
+
+
+```
+(maint) Avoid deadlock with large results
+
+Previously, if the result of a process was larger than the IO buffer
+size (commonly 64k), execution would deadlock until the timeout.
+
+In this scenario, the writer would fill the buffer and block until the
+reader had cleared the buffer by reading. However, the reader will only
+try to read once (to get the whole result) and will only attempt to read
+after the child process has exited. This causes a deadlock, as the child
+can't exit because it can't finish writing, but the the reader won't
+read because the child hasn't exited.
+
+This commit fixes the watcher loop to instead IO.select() from the
+available result readers and read a partial result into a buffer whenever it's
+available. This ensures that the writer will never remain blocked by a
+full buffer. The reader now uses IO#eof? to determine whether the child
+process has exited, at which point it will process the whole result as
+before.
+```
+* Merge pull request #15 from samwoods1/add_jjb_pipelines (72d32635)
+
+
+```
+Merge pull request #15 from samwoods1/add_jjb_pipelines
+
+(maint) Revert name change of in_parallel.rb
+```
+* (maint) Revert name change of in_parallel.rb (327c8fd5)
+
+### <a name = "0.1.14">0.1.14 - 8 Aug, 2016 (ce331dbd)
+
+* (HISTORY) update in-parallel history for gem release 0.1.14 (ce331dbd)
+
+* (GEM) update in-parallel version to 0.1.14 (d026c624)
+
+* Merge pull request #14 from samwoods1/add_jjb_pipelines (269bc350)
+
+
+```
+Merge pull request #14 from samwoods1/add_jjb_pipelines
+
+(maint) Consistent naming of in-parallel
+```
+* (maint) Consistent naming of in-parallel (6fbb442d)
+
+### <a name = "0.1.13">0.1.13 - 8 Aug, 2016 (26d19934)
+
+* Initial release.
+
+
+\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*

data/CODEOWNERS

@@ -0,0 +1,8 @@
+# This will cause RE to be assigned review of any opened PRs against
+# the branches containing this file.
+# See https://help.github.com/en/articles/about-code-owners for info on how to
+# take ownership of parts of the code base that should be reviewed by another
+# team.
+
+# RE will be the default owners for everything in the repo.
+*                                     @puppetlabs/release-engineering

data/Gemfile

@@ -1,7 +1,7 @@
 source 'https://rubygems.org'
 
 # in the Rakefile, so we require it in all groups
-gem 'rspec'                ,'~> 3.1.0'
+gem 'rspec'                ,'~> 3.1'
 
 group :development do
   gem 'simplecov'

data/Gemfile.lock

@@ -0,0 +1,76 @@
+PATH
+  remote: .
+  specs:
+    in-parallel (1.0.0)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (7.0.4.3)
+      concurrent-ruby (~> 1.0, >= 1.0.2)
+      i18n (>= 1.6, < 2)
+      minitest (>= 5.1)
+      tzinfo (~> 2.0)
+    concurrent-ruby (1.2.2)
+    diff-lcs (1.5.0)
+    docile (1.4.0)
+    i18n (1.12.0)
+      concurrent-ruby (~> 1.0)
+    iniparser (1.0.1)
+    kramdown (2.4.0)
+      rexml
+    logutils (0.6.1)
+    markdown (0.4.0)
+      kramdown (>= 0.13.7)
+      props (>= 0.2.0)
+      textutils (>= 0.2.0)
+    minitest (5.18.0)
+    props (1.2.0)
+      iniparser (>= 0.1.0)
+    rake (13.0.6)
+    rexml (3.2.5)
+    rspec (3.12.0)
+      rspec-core (~> 3.12.0)
+      rspec-expectations (~> 3.12.0)
+      rspec-mocks (~> 3.12.0)
+    rspec-core (3.12.1)
+      rspec-support (~> 3.12.0)
+    rspec-expectations (3.12.2)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.12.0)
+    rspec-mocks (3.12.4)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.12.0)
+    rspec-support (3.12.0)
+    rubyzip (2.3.2)
+    simplecov (0.22.0)
+      docile (~> 1.1)
+      simplecov-html (~> 0.11)
+      simplecov_json_formatter (~> 0.1)
+    simplecov-html (0.12.3)
+    simplecov_json_formatter (0.1.4)
+    textutils (1.4.0)
+      activesupport
+      logutils (>= 0.6.1)
+      props (>= 1.1.2)
+      rubyzip (>= 1.0.0)
+    tzinfo (2.0.6)
+      concurrent-ruby (~> 1.0)
+    webrick (1.7.0)
+    yard (0.9.28)
+      webrick (~> 1.7.0)
+
+PLATFORMS
+  aarch64-linux
+  x86_64-linux
+
+DEPENDENCIES
+  in-parallel!
+  markdown (~> 0)
+  rake (>= 0.9.0)
+  rspec (~> 3.1)
+  simplecov
+  yard (~> 0)
+
+BUNDLED WITH
+   2.4.8

data/README.md

@@ -1,15 +1,31 @@
+# in-parallel
+
+- [in-parallel](#in-parallel)
+  - [Use Cases](#use-cases)
+  - [Install](#install)
+  - [Usage](#usage)
+  - [Methods](#methods)
+    - [run\_in\_parallel(timeout=nil, kill\_all\_on\_error = false, \&block)](#run_in_paralleltimeoutnil-kill_all_on_error--false-block)
+    - [Enumerable.each\_in\_parallel(identifier=nil, timeout=(InParallel::InParallelExecutor.timeout), kill\_all\_on\_error = false, \&block)](#enumerableeach_in_parallelidentifiernil-timeoutinparallelinparallelexecutortimeout-kill_all_on_error--false-block)
+    - [run\_in\_background(ignore\_results = true, \&block)](#run_in_backgroundignore_results--true-block)
+    - [wait\_for\_processes(timeout=nil, kill\_all\_on\_error = false)](#wait_for_processestimeoutnil-kill_all_on_error--false)
+  - [Global Options](#global-options)
+  - [Releasing](#releasing)
 
 A lightweight Ruby library with very simple syntax, making use of Process.fork to execute code in parallel.
 
-# Use Cases
+## Use Cases
+
 Many other Ruby libraries that simplify parallel execution support one primary use case - crunching through a large queue of small, similar tasks as quickly and efficiently as possible.  This library primarily supports the use case of executing a few larger and unrelated tasks in parallel, automatically managing the stdout and passing return values back to the main process. This library was created to be used by Puppet's Beaker test framework to enable parallel execution of some of the framework's tasks, and allow users to execute code in parallel within their tests.
 
 If you are looking for something that excels at executing a large queue of tasks in parallel as efficiently as possible, you should take a look at the [parallel](https://github.com/grosser/parallel) project.
 
-# Install
+## Install
+
 ```gem install in-parallel```
 
-# Usage
+## Usage
+
 ```include InParallel``` to use as a mix-in
 
 The methods below allow you to fork processes to execute multiple methods or blocks within an enumerable in parallel.  They all have this common behavior:
@@ -24,7 +40,9 @@ The methods below allow you to fork processes to execute multiple methods or blo
 1. Will timeout (stop execution and raise an exception) based on a global timeout value, or timeout parameter.
 
 ## Methods
+
 ### run_in_parallel(timeout=nil, kill_all_on_error = false, &block)
+
 1. Each method in a block will be executed in parallel (unless the method is defined in Kernel or BaseObject).
     1. Any methods further down the stack won't be affected, only the ones directly within the block.
 1. Waits for each process in realtime and logs immediately upon completion of each process
@@ -53,8 +71,10 @@ The methods below allow you to fork processes to execute multiple methods or blo
   
   puts "#{@result_1}, #{@result_2[:foo]}"
 ```
+
 stdout:
-```
+
+```shell
 Forked process for 'method_with_param' - PID = '49398'
 Forked process for 'method_without_param' - PID = '49399'
 
@@ -67,7 +87,9 @@ hello world
 ------ Completed output for method_without_param - 49399
 hello world, bar
 ```
+
 ### Enumerable.each_in_parallel(identifier=nil, timeout=(InParallel::InParallelExecutor.timeout), kill_all_on_error = false, &block)
+
 1. This is very similar to other solutions, except that it directly extends the Enumerable class with an each_in_parallel method, giving you the ability to pretty simply spawn a process for any item in an array or map.
 1. Identifies the block location (or caller location if the block does not have a source_location) in the console log to make it clear which block is being executed
 1. Identifier param is only for logging, otherwise it will use the block source location.
@@ -77,6 +99,7 @@ hello world, bar
 ```
 
 ### run_in_background(ignore_results = true, &block)
+
 1. This does basically the same thing as run_in_parallel, except it does not wait for execution of all processes to complete, it returns immediately.
 1. You can optionally ignore results completely (default) or delay evaluating the results until later
 1. You can run multiple blocks in the background and then at some later point evaluate all of the results
@@ -94,11 +117,12 @@ hello world, bar
   run_in_background { create_file_with_delay(TMP_FILE) }
   
   # Should not exist immediately upon block completion
-  puts(File.exists?(TMP_FILE)) # false
+  puts(File.exist?(TMP_FILE)) # false
   sleep(3)
   # Should exist once the delay from create_file_with_delay is done
-  puts(File.exists?(TMP_FILE)) # true
+  puts(File.exist?(TMP_FILE)) # true
   ```
+
   ```ruby
   # Example 2 - delay results
   run_in_background(false) { @result = create_file_with_delay(TMP_FILE) }
@@ -118,13 +142,16 @@ hello world, bar
 ```
 
 ### wait_for_processes(timeout=nil, kill_all_on_error = false)
+
 1. Used only after run_in_background with ignore_results=false
 1. Optional args for timeout and kill_all_on_error
 1. See run_in_background for examples
 
 ## Global Options
+
 You can get or set the following values to set global defaults.  These defaults can also be specified per execution by supplying the values as parameters to the parallel methods.
-```
+
+```ruby
   # How many seconds to wait between logging a 'Waiting for child processes.' message. Defaults to 30 seconds
   parallel_signal_interval
 
@@ -137,3 +164,11 @@ You can get or set the following values to set global defaults.  These defaults
   @logger.log_level
 ```
 
+## Releasing
+
+Follow these steps to publish a new GitHub release, and build and push the gem to <https://rubygems.org>.
+
+1. Bump the "VERSION" in lib/in-parallel/version.rb appropriately based on changes in CHANGELOG.md since the last release.
+2. Run `./release-prep` to update `Gemfile.lock` and `CHANGELOG.md`.
+3. Commit and push changes to a new branch, then open a pull request against main and be sure to add the "maintenance" label.
+4. After the pull request is approved and merged, then navigate to Actions --> Release Gem --> run workflow --> Branch: main --> Run workflow.

data/lib/in-parallel/version.rb

@@ -1,3 +1,3 @@
 module InParallel
-  VERSION = '0.1.17'
+  VERSION = '1.0.0'
 end

data/lib/in_parallel.rb

@@ -152,7 +152,7 @@ module InParallel
                 end
                 results_map[process_info[:index]] = { process_info[:tmp_result] => marshalled_result }
               ensure
-                File.delete(process_info[:std_out]) if File.exists?(process_info[:std_out])
+                File.delete(process_info[:std_out]) if File.exist?(process_info[:std_out])
                 # close the read end pipe
                 process_info[:result].close unless process_info[:result].closed?
                 @@process_infos.delete(process_info)
@@ -185,7 +185,7 @@ module InParallel
       ret_val                   = nil
       # Communicate the return value of the method or block
       read_result, write_result = IO.pipe
-      Dir.mkdir('tmp') unless Dir.exists? 'tmp'
+      Dir.mkdir('tmp') unless Dir.exist? 'tmp'
       pid                       = fork do
         stdout_file = File.new("tmp/pp_#{Process.pid}", 'w')
         exit_status = 0

data/spec/in-paralell_spec.rb

@@ -75,7 +75,7 @@ end
 #Tests
 describe '.run_in_parallel' do
   before do
-    File.delete(TMP_FILE) if File.exists?(TMP_FILE)
+    File.delete(TMP_FILE) if File.exist?(TMP_FILE)
   end
 
   it 'should run methods in another process' do
@@ -169,7 +169,7 @@ end
 
 describe '.run_in_background' do
   before do
-    File.delete(TMP_FILE) if File.exists?(TMP_FILE)
+    File.delete(TMP_FILE) if File.exist?(TMP_FILE)
   end
 
   it 'should run in the background' do
@@ -177,11 +177,11 @@ describe '.run_in_background' do
 
     start = Time.now
     # Should not exist immediately upon block completion
-    expect(File.exists? TMP_FILE).to eq false
+    expect(File.exist? TMP_FILE).to eq false
     # Give this some time to complete since it takes longer on the vmpooler vms
     file_exists = false
     while Time.now < start + 10 do
-      if File.exists? TMP_FILE
+      if File.exist? TMP_FILE
         file_exists = true
         break
       end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: in-parallel
 version: !ruby/object:Gem::Version
-  version: 0.1.17
+  version: 1.0.0
 platform: ruby
 authors:
 - samwoods1
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-02-07 00:00:00.000000000 Z
+date: 2023-03-15 00:00:00.000000000 Z
 dependencies: []
 description: Many other Ruby libraries that simplify parallel execution support one
   primary use case - crunching through a large queue of small, similar tasks as quickly
@@ -24,11 +24,13 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- CHANGELOG.md
+- CODEOWNERS
 - CONTRIBUTING.md
 - Gemfile
+- Gemfile.lock
 - HISTORY.md
 - LICENSE
-- MAINTAINERS.md
 - README.md
 - Rakefile
 - lib/in-parallel/version.rb
@@ -46,17 +48,16 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ! '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.4.6
+rubygems_version: 3.4.6
 signing_key: 
 specification_version: 4
 summary: A lightweight library to execute a handful of tasks in parallel with simple

data/MAINTAINERS.md

@@ -1,9 +0,0 @@
-# Reviewers/Maintainers For in-parallel
-
-in-parallel is maintained by Puppet's Quality Assurance (QA) Team. These people
-will be reviewing & merging PRs to the project.
-
-| Name           | Github                                              | Email                       |
-|:--------------:|:---------------------------------------------------:|:---------------------------:|
-| Sam Woods      | [samwoods1](https://github.com/samwoods1)           | <sam.woods@puppet.com>      |
-| Zach Reichert  | [zreichert](https://github.com/zreichert)           | <zach.reichert@puppet.com>  |