jobQueue 1.0.1

data/LICENSE

@@ -0,0 +1,27 @@
+Copyright (c) 2011, Ralf Mueller (stark.dreamdetective@googlemail.com)
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+    * Redistributions of source code must retain the above copyright notice,
+      this list of conditions and the following disclaimer.
+
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+
+    * The names of its contributors may not be used to endorse or promote
+      products derived from this software without specific prior written
+      permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.rdoc

@@ -0,0 +1,85 @@
+= JobQueue - Let run your jobs in parallel
+
+This package contains jobQueue: 
+  * A simple class to parallalize your work on a user definend number of threads
+  * A ruby script, which uses the above class to run each line of a shell script
+
+jobQueue can do the following things:
+
+* Run shell commands
+* Run instance and class methods
+* Run Procs and Lambdas
+* Respect user definded locks
+
+== Installation
+
+=== Gem Installation
+
+Download and install jobQueue with the following.
+
+   gem install jobQueue
+
+=== Requirements
+
+JobQueue requires Ruby only, but versions 1.9.x are needed to make use of system threads.
+
+== Usage
+
+=== Parallelize Ruby's procs, lambdas and things
+
+Create a JobQueue with nThreads worker with:
+
+  jq = JobQueue.new(nThreads)
+
+Use its push method to put in something to do
+
+* For procs and lambdas: 
+    
+    jp.push(myProc,arg0,arg1,...)
+
+* For object methods:
+
+    jq.push([myObject,[:method,arg0,arg1,...])
+
+* Same code can be used for class methods:
+
+    jq.push(myClass,[:myClassMethod,arg0,arg1,...])
+
+To start the workers, call 
+
+  jq.run
+
+That's it. You might have look at tests, that come with the jobQueue gem.
+
+=== Parallelize system commands
+
+Any string is expected to be a valid system command and executed via Ruby's system call, e.g.
+
+  jq.push('find ./src -name "*.rb"','find ./downloads -name "*.rb"')
+
+will start 2 parallel searches for ruby files.
+
+== Support, Issues, Bugs, ...
+
+As long there is no dedicataed project page, please use direct mail or the ruby-lang mailing list.
+
+== Credits
+
+[<b>Robert Klemme</b>] For the first hints: https://www.ruby-forum.com/topic/68001#86298
+
+== License
+
+jobQueue makes use of the BSD License
+
+:include: LICENSE
+
+
+---
+
+= Other stuff
+
+Author::   Ralf Mueller <stark.dreamdetective@gmail.com>
+Requires:: Ruby 1.9 or later
+License::  Copyright 2011 by Ralf Mueller
+           Released under BSD-style license.  See the LICENSE
+           file included in the distribution.

data/bin/prun.rb

@@ -0,0 +1,19 @@
+#!/usr/bin/env ruby
+require 'jobqueue'
+# ==============================================================================
+# Example script:
+# Read in a script and process line in parallel with a given number of threads:
+# Usage: prun.rb 10 myScript.sh
+# Attention: each line of the input script is regarded as a separate command
+# ==============================================================================
+unless ARGV.size == 2
+  warn "provide number of threads and input data"
+  exit
+end
+# read number of threads
+noTh = ARGV[0].to_i
+# read file line per line
+lines = File.open(ARGV[1]).readlines.map(&:chomp)
+q = JobQueue.new(noTh)
+q.push(*lines)
+q.run

data/gemspec

@@ -0,0 +1,19 @@
+require 'rubygems'
+
+spec = Gem::Specification.new do |s|
+  s.name              = "jobQueue"
+  s.version           = '1.0.1'
+  s.platform          = Gem::Platform::RUBY
+  s.bindir            = 'bin'
+  s.files             = ["lib/jobqueue.rb","bin/prun.rb"] + ["gemspec","LICENSE","README.rdoc"]
+  s.executables       << 'prun.rb'
+  s.description       = "Run Shell commands or Ruby methods in parallel"
+  s.summary           = s.description
+  s.author            = "Ralf Mueller"
+  s.email             = "stark.dreamdetective@gmail.com"
+#  s.homepage          = "http://
+  s.extra_rdoc_files          = ["README.rdoc","LICENSE"]
+  s.required_ruby_version = ">= 1.9"
+end
+
+# vim:ft=ruby

data/lib/jobqueue.rb

@@ -0,0 +1,83 @@
+#!/usr/bin/env ruby
+require 'thread'
+require 'pp'
+# ==============================================================================
+# Author: Ralf Mueller, ralf.mueller@zmaw.de
+#         suggestions from Robert Klemme (https://www.ruby-forum.com/topic/68001#86298)
+#
+# ==============================================================================
+# Sized Queue for limiting the number of parallel jobs
+# ==============================================================================
+class JobQueue
+  attr_reader :size, :queue, :threads
+
+  # Create a new queue qith a given number of worker threads
+  def initialize(size)
+    @size  = size
+    @queue = Queue.new
+  end
+
+  # Put jobs into the queue. Use
+  #   strings for system commands
+  #   proc,args for single methods
+  #   [object,[:methods,args]] for sende messages to objects
+  #
+  def push(*items)
+    items.each {|it| @queue << it}
+  end
+
+  # Start workers to run through the queue
+  def run
+    @threads = (1..@size).map {|i|
+      Thread.new(@queue) {|q|
+        until ( q == ( task = q.deq ) )
+          if task.kind_of? String
+            system(task)
+          elsif task.kind_of? Proc
+            task.call
+          elsif task.kind_of? Array
+            if task.size > 1
+              if not task[1].kind_of? Array 
+                # Expects proc/lambda with arguments, e.g. [mysqrt,2.789]
+                task[0].call(*task[1..-1])
+              else
+                # expect an object in task[0] and one of its methods with arguments in task[1] as a symbol
+                # e.g. [a,[:attribute=,1]
+                task[0].send(task[1][0],*task[1][1..-1])
+              end
+            else
+              warn 'provide 2d arrays'
+            end
+          end
+        end
+      }
+    }
+    @threads.size.times { @queue.enq @queue}
+    @threads.each {|t| t.join}
+  end
+
+  # Get the maximum number of parallel runs
+  def number_of_processors
+    if RUBY_PLATFORM =~ /linux/
+      return `cat /proc/cpuinfo | grep processor | wc -l`.to_i
+    elsif RUBY_PLATFORM =~ /darwin/
+      return `sysctl -n hw.logicalcpu`.to_i
+    elsif RUBY_PLATFORM =~ /(win32|mingw|cygwin)/
+      # this works for windows 2000 or greater
+      require 'win32ole'
+      wmi = WIN32OLE.connect("winmgmts://")
+      wmi.ExecQuery("select * from Win32_ComputerSystem").each do |system|
+        begin
+          processors = system.NumberOfLogicalProcessors
+        rescue
+          processors = 0
+        end
+        return [system.NumberOfProcessors, processors].max
+      end
+      elseif RUBY_PLATFORM =~ /java/
+        return Runtime.getRuntime().availableProcessors().to_i
+    end
+    raise "can't determine 'number_of_processors' for '#{RUBY_PLATFORM}'"
+  end
+end
+

metadata

@@ -0,0 +1,52 @@
+--- !ruby/object:Gem::Specification
+name: jobQueue
+version: !ruby/object:Gem::Version
+  version: 1.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Ralf Mueller
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2011-12-12 00:00:00.000000000 Z
+dependencies: []
+description: Run Shell commands or Ruby methods in parallel
+email: stark.dreamdetective@gmail.com
+executables:
+- prun.rb
+extensions: []
+extra_rdoc_files:
+- README.rdoc
+- LICENSE
+files:
+- lib/jobqueue.rb
+- bin/prun.rb
+- gemspec
+- LICENSE
+- README.rdoc
+homepage: 
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '1.9'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.11
+signing_key: 
+specification_version: 3
+summary: Run Shell commands or Ruby methods in parallel
+test_files: []