forkner 1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b52b5779f3a60718d88c8211ffef94cd28c6f0c4f2c6b5912e15548b271673b5
+  data.tar.gz: 78ad9cbbe65fc736daef6ab882193f7a6b2ff2bd9ab1e4d9d7d45dce9ed49cf2
+SHA512:
+  metadata.gz: 050ae01b17dd141a6164f0748df4a9f0087b8199cc24291275e69b92664c47b8c53de227a877bdacfce1a915ba2b3c7d287bbc80f6f5d1ddea4790d5f12e8a85
+  data.tar.gz: bd8a8bf7a164c6d1219b20c3b3ef6ba5c9b2e0e16a4ebbfe1779f87e1ad1253a02ac8f23feb7e8c32699cced346959dcc36a0b97babd3dd5357a9c0b6672f54c

data/README.md

@@ -0,0 +1,78 @@
+# Forkner
+
+Forkner helps you manage multiple child processes and getting data back from
+them when they finish.
+
+## Usage
+
+Consider a situation in which you need to run a bunch of parallel processes, but
+no more than five at a time. You might do that like this:
+
+	@@code basic basic
+
+First we load the `forkner` gem. Then we call Forkner's container `method`. All
+child processes within `container` will completed before the method is done.
+`container` yields a Forkner object.
+Inside the `container` block we run whatever commands are necessary to set up
+for running the child processes. In this simple example we simply loop 100
+times.
+Inside the loop, we use the forkner object to fork off into a child process.
+Everything in `child` is run within the child process. At the end of that block
+the child processes exits.
+
+When the loop gets around and calls `child` again, another child process is only
+run when there are fewer than five (the number we passed into `container`) child
+processes. `child` pauses until there is a slot available.
+At the bottom of the `container` block, Forkner waits until any remaining
+child processes have finished.
+
+### Getting information from the child process
+
+Forkner allows you to communicate information form the child process back to the
+parent process. Doing so requires two steps. First, set up a block that
+processes information returned from child processes. Second, in the child
+processes, finish the block with a value that can be stored as JSON.
+
+Consider this example:
+
+	@@code retrieve retrieve
+
+In this example, inside the `container` block we call the Forkner object's
+`reaper` method with a block. In that block we get a single parameter which
+contains information from the child process. In this case we know it's a hash
+and we output two of its values.
+
+Inside the `child` block, the child generates a random value and a timestamp. The
+last line in the block is a hash of those values.
+
+Behind the scenes, that hash is converted to JSON and stored in a temporary
+file. When the `reaper` method is called, the hash is reconstituted from the
+JSON file and returned to the `reaper` block. That's why it's important that the
+child block end with a value that can be stored as JSON.
+
+### Using Forkner without the container block
+
+If you prefer to get a little closer to the metal, you can directly create a
+Forkner object and call its `child` method. Just be sure to call the `wait_all`
+method after all the child processes have been called. The following code does
+exactly the same thing as the previous example.
+
+	@@code no-container no-container
+
+## Install
+
+```
+gem install forkner
+```
+
+## Author
+
+Mike O'Sullivan
+mike@idocs.com
+
+## History
+
+| version | date        | notes                                         |
+|---------|-------------|-----------------------------------------------|
+| 1.0     | Jan 7, 2020 | Initial upload.                               |
+| 1.1     | Jan 7, 2020 | Fixed some typos. No change to functionality. |

data/lib/forkner.rb

@@ -0,0 +1,231 @@
+require 'tempfile'
+require 'json'
+
+
+#===============================================================================
+# Forkner
+#
+class Forkner
+	
+	# The maximum number of child processes to run at once.
+	attr_accessor :max
+	
+	# Temporary directory for storing information from child processes. Defaults
+	# to /tmp.
+	attr_accessor :tmp_dir
+	
+	##
+	# Version 1.1
+	VERSION = '1.1'
+	
+	
+	#---------------------------------------------------------------------------
+	# self.container
+	#
+	
+	# container is probably the easiest way to use Forkner. Run all the code
+	# that will have child processes inside a container block. The single
+	# parameter for container is the maximum number of child processes to allow.
+	# After the container block, Forkner waits for all remaining child processes
+	# to exit.
+	#
+	# So, for example, this code loops 100 times, but will not fork more than
+	# five children at a time:
+	#
+	#    Forkner.container(5) do |forkner|
+	#        100.times do
+	#            forkner.child do
+	#                # do stuff in the child process
+	#            end
+	#        end
+	#    end
+	
+	def self.container(max)
+		forkner = Forkner.new(max)
+		yield forkner
+		forkner.wait_all
+	end
+	#
+	# self.container
+	#---------------------------------------------------------------------------
+	
+	
+	#---------------------------------------------------------------------------
+	# initialize
+	#
+	
+	# Creates a new Forkner object. The single parameter is the maximum number
+	# of child processes to run at once. So, for example, the following code
+	# creates a Forkner object that will allow up to five child processes at
+	# once.
+	#
+	#    forkner = Forkner.new(5)
+	
+	def initialize(max)
+		@max = max
+		@children = {}
+		@tmp_dir = '/tmp'
+		@reaper = nil
+	end
+	#
+	# initialize
+	#---------------------------------------------------------------------------
+	
+	
+	#---------------------------------------------------------------------------
+	# child
+	#
+	
+	# Runs a child process. If there are already the maximum number of children
+	# running then this method waits until one of them exits. The last line of
+	# the block is information that can be stored in JSON, and if you have
+	# defined a reaper block, then that information will be conveyed back to the
+	# parent process.
+	#
+	# For example, this child process generates a random number and a timestamp.
+	# The last line of the block is a hash that will be sent back to the parent
+	# process.
+	#
+	#    forkner.child do
+	#        myrand = rand()
+	#        timestamp = Time.now
+	#        {'myrand'=>myrand, 'timestamp'=>timestamp}
+	#    end
+	
+	def child
+		# init
+		json_path = nil
+		
+		# transfer file
+		if @reaper
+			json_path = Random.rand().to_s
+			json_path = json_path.sub(/\A.*\./mu, '')
+			json_path = @tmp_dir + '/' + json_path
+		end
+		
+		# wait until we have a space for a process
+		waiter @max - 1
+		
+		# parent process
+		if new_child_pid = Process.fork()
+			# set child record
+			child = @children[new_child_pid] = {}
+			
+			# file handle
+			if @reaper
+				child['json_path'] = json_path
+			end
+			
+			# return true
+			return true
+		
+		# child process
+		else
+			# yield if necessary
+			if block_given?
+				rv = yield()
+				
+				# save to json file if necessary
+				if json_path
+					File.write json_path, JSON.generate(rv)
+				end
+				
+				# exit child process
+				exit
+			end
+			
+			# always return false
+			return false
+		end
+	end
+	#
+	# child
+	#---------------------------------------------------------------------------
+	
+	
+	#---------------------------------------------------------------------------
+	# reaper
+	#
+	
+	# Defines the block to run when a child process finishes. The single param
+	# passed to the block is a hash or array of information from the child
+	# process. For example, if the child process returns a hash of information,
+	# you might display it like this:
+	#
+	#    forkner.reaper() do |rv|
+	#        puts '-----'
+	#        puts rv['myrand']
+	#        puts rv['timestamp']
+	#    end
+	
+	def reaper(&block)
+		@reaper = block
+	end
+	#
+	# reaper
+	#---------------------------------------------------------------------------
+	
+	
+	#---------------------------------------------------------------------------
+	# wait_all
+	#
+	
+	##
+	# Waits for all child processes to finish. You don't need to call this
+	# method if you're using Forkner.container.
+	def wait_all
+		waiter 0
+	end
+	
+	##
+	# waitall is just an alias for wait_all because I can never remember
+	# whether or not there's an underscore in the method.
+	alias waitall wait_all
+	
+	#
+	# wait_all
+	#---------------------------------------------------------------------------
+	
+	
+	# private
+	private
+	
+	
+	#---------------------------------------------------------------------------
+	# waiter
+	#
+	def waiter(wait_max)
+		# loop until we have fewer than @max children
+		while @children.length > wait_max
+			begin
+				# wait
+				old_child_pid = Process.wait(-1, Process::WNOHANG)
+				old_child = @children.delete(old_child_pid)
+				
+				# if child
+				if old_child and old_child['json_path']
+					# if reaper
+					if @reaper
+						# slurp in JSON
+						rv = JSON.parse(File.read(old_child['json_path']))
+						
+						# delete transfer file
+						File.unlink old_child['json_path']
+						
+						# call reaper
+						@reaper.call rv
+					end
+				end
+				
+			# TODO: Handle system errors
+			rescue SystemCallError
+			end
+		end
+	end
+	#
+	# waiter
+	#---------------------------------------------------------------------------
+end
+#
+# Forkner
+#===============================================================================

metadata

@@ -0,0 +1,46 @@
+--- !ruby/object:Gem::Specification
+name: forkner
+version: !ruby/object:Gem::Version
+  version: '1.1'
+platform: ruby
+authors:
+- Mike O'Sullivan
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2020-01-17 00:00:00.000000000 Z
+dependencies: []
+description: Manages forking multiple processes and returns results to the parent
+  process
+email: mike@idocs.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/forkner.rb
+homepage: https://rubygems.org/gems/forkner
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.7.6
+signing_key: 
+specification_version: 4
+summary: Fork manager with return values
+test_files: []