workers 0.1.0 → 0.1.1

data/README.md

@@ -19,8 +19,71 @@ Or install it yourself as:
 
     $ gem install workers
 
+## Tasks
+
+Tasks and task groups build on top of pools of worker threads.
+They provide a means of parallelizing expensive computations, collecing results, and handling exceptions.
+These are the classes you normally work with in your application level code because they remove boiletplate.
+
+    # Create a task group (it contains a pool of worker threads).
+    group = Workers::TaskGroup.new
+
+    # Add tasks to the group.
+    10.times do |i|
+      10.times do |j|
+        group.add(i, j) do
+          group.synchronize { puts "Computing #{i} * #{j}..." }
+          i * j # Last statement executed is the result of the task.
+        end
+      end
+    end
+
+    # Execute the tasks (blocks until the tasks complete).
+    # Returns true if all tasks succeed.  False if any fail.
+    group.run
+
+    # Return an array of all the tasks.
+    group.tasks
+
+    # Return an array of the successful tasks.
+    group.successes
+
+    # Return an array of the failed tasks (raised an exception).
+    group.failures
+
+    # Review the results.
+    group.tasks.each do |t|
+      t.succeeded? # True or false (false if an exception occurred).
+      t.failed?    # True or false (true if an exception occurred).
+      t.args       # Input arguments (the value of i in this example).
+      t.result     # Output value (the result of i * i in this example).
+      t.exception  # The exception if one exists.
+    end
+
+Note that instances of TaskGroup provide a 'synchronize' method.
+This method uses a mutex so you can serialize portions of your tasks that aren't thread safe.
+
+## Parallel Map
+
+Parallel Map is syntactic sugar for tasks and task groups:
+
+    group = Workers::TaskGroup.new
+    group.map([1,2,3,4,5]) { |i| i * i }
+
+The above syntax is equivalent to the below:
+
+    Workers.map([1, 2, 3, 4, 5]) { |i| i * i }
+
+Note that any task failures (exceptions) will cause an exception to be raised.
+This means that parallel map is "all or nothing" where as tasks and task groups leave error processing up to you.
+
 ## Workers - Basic
 
+There are many cases where tasks, task groups, and parallel map are too high-level.
+Fortunately you can use the lower level Worker class to solve these problems.
+The main purpose of the Worker class is to add an event system on top of the Thread class.
+This greatly simplifies inter-thread communication.
+
     # Initialize a worker pool.
     pool = Workers::Pool.new
 
@@ -44,7 +107,7 @@ Or install it yourself as:
 
 The Worker class is designed to be customized through inheritence and its event system:
 
-    # Create a custom worker class that handles custom commands.
+    # Create a subclass that handles custom events.
     class CustomWorker < Workers::Worker
       private
       def process_event(event)
@@ -75,7 +138,7 @@ The Worker class is designed to be customized through inheritence and its event
 Note that you can use custom workers without a pool.
 This effectively gives you direct access to a single event-driven thread.
 
-## Pools - Advanced
+## Pools
 
 As shown above, pools effectively allow a group of workers to share a work queue.
 They have a few additional methods described below:
@@ -95,59 +158,6 @@ They have a few additional methods described below:
     # Resize the pool size to a specific value.
     pool.resize(20)
 
-## Tasks
-
-Tasks and task groups build on top of worker pools.
-They provide a means of parallelizing expensive computations and collecing the results.
-These are the classes you normally work with in your application level code.
-
-    # Create a task group (it contains a pool of workers).
-    group = Workers::TaskGroup.new
-
-    # Add tasks to the group.
-    10.times do |i|
-      10.times do |j|
-        group.add(i, j) do
-          i * j # Last statement executed is the result of the task.
-        end
-      end
-    end
-
-    # Execute the tasks (blocks until the tasks complete).
-    # Returns true if all tasks succeed.  False if any fail.
-    group.run
-
-    # Return an array of all the tasks.
-    group.tasks
-
-    # Return an array of the successful tasks.
-    group.successes
-
-    # Return an array of the failed tasks.
-    group.failures
-
-    # Review the results.
-    group.tasks.each do |t|
-      t.succeeded? # True or false (false if an exception occurred).
-      t.failed?    # True or false (true if an exception occurred).
-      t.args       # Input arguments (the value of i in this example).
-      t.result     # Output value (the result of i * i in this example).
-      t.exception  # The exception if one exists.
-    end
-
-## Parallel Map
-
-Task groups and tasks are the building blocks of parallel map:
-
-    group = Workers::TaskGroup.new
-    group.map([1,2,3,4,5]) { |i| i * i }
-
-The below syntax is equivalent to the above:
-
-    Workers.map([1, 2, 3, 4, 5]) { |i| i * i }
-
-Note that any failures will cause an exception to be raised.
-
 ## Timers
 
 Timers provide a way to execute code in the future:
@@ -213,7 +223,7 @@ You can create additional or custom ones as necessary:
 
     group = Workers::TaskGroup.new(
       :logger => nil,                   # Ruby logger instance.
-      :pool => Workers::Pool.new        # The workers pool used to execute timer callbacks.
+      :pool => Workers.pool             # The workers pool used to execute timer callbacks.
     )
 
     task = Workers::Task.new(

data/lib/workers/task_group.rb

@@ -10,7 +10,8 @@ module Workers
       @pool = options[:pool] || Workers.pool
       @state = :initialized
       @tasks = []
-      @lock = Mutex.new
+      @internal_lock = Mutex.new
+      @external_lock = Mutex.new
       @finished_count = 0
       @conditional = ConditionVariable.new
 
@@ -35,12 +36,12 @@ module Workers
       @state = :running
       @run_thread = Thread.current
 
-      @lock.synchronize do
+      @internal_lock.synchronize do
         @tasks.each do |task|
           @pool.perform { task.run }
         end
 
-        @conditional.wait(@lock)
+        @conditional.wait(@internal_lock)
       end
 
       return @tasks.all? { |t| t.succeeded? }
@@ -74,6 +75,14 @@ module Workers
       return tasks.map { |t| t.result }
     end
 
+    # Convenient mutex to be used by a users's task code that needs serializing.
+    # This should NEVER be used by TaskGroup code (use the @internal_lock instead);
+    def synchronize(&block)
+      @external_lock.synchronize { block.call }
+
+      return nil
+    end
+
     private
 
     def state!(*args)
@@ -85,7 +94,7 @@ module Workers
     end
 
     def finished(task)
-      @lock.synchronize do
+      @internal_lock.synchronize do
         @finished_count += 1
         @conditional.signal if @finished_count >= @tasks.count
       end

data/lib/workers/version.rb

@@ -1,3 +1,3 @@
 module Workers
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

data/workers.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |gem|
   gem.version       = Workers::VERSION
   gem.authors       = ['Chad Remesch']
   gem.email         = ['chad@remesch.com']
-  gem.description   = %q{Simple Ruby workers for performing work in background threads.}
-  gem.summary       = %q{Workers is a Ruby gem for performing work in background threads. Design goals include high performance, low latency, simple API, customizability, and multi-layered architecture.}
+  gem.description   = %q{Workers is a Ruby gem for performing work in background threads.}
+  gem.summary       = %q{Workers is a Ruby gem for performing work in background threads. Design goals include high performance, low latency, simple API, customizability, and multi-layered architecture. It provides a number of simple to use classes that solve a wide range of concurrency problems.}
   gem.homepage      = 'https://github.com/chadrem/workers'
 
   gem.files         = `git ls-files`.split($/)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: workers
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
   prerelease: 
 platform: ruby
 authors:
@@ -9,9 +9,9 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-05-18 00:00:00.000000000 Z
+date: 2013-05-24 00:00:00.000000000 Z
 dependencies: []
-description: Simple Ruby workers for performing work in background threads.
+description: Workers is a Ruby gem for performing work in background threads.
 email:
 - chad@remesch.com
 executables: []
@@ -61,5 +61,6 @@ signing_key:
 specification_version: 3
 summary: Workers is a Ruby gem for performing work in background threads. Design goals
   include high performance, low latency, simple API, customizability, and multi-layered
-  architecture.
+  architecture. It provides a number of simple to use classes that solve a wide range
+  of concurrency problems.
 test_files: []