concurrent-ruby 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 23428b19ba9bf0dbfb21cc1262c8607bd5b40b8d
-  data.tar.gz: 7757f144dfef3b8823d5dacc6e3b3db6a220a9aa
+  metadata.gz: 3f58685d3bea0c67b29987579c607417ba905790
+  data.tar.gz: 82ab333727880726e30369350eaca336a81c5f2c
 SHA512:
-  metadata.gz: 445c8a14988adb216f4314453381dcc56d27f399444d8cc15f4086151699208a15c7fc14a54abeeccf69bd83440ff3a7d8a7ccf240b1c845d2612d567889ebe4
-  data.tar.gz: f6ddc0ec2cf80080fafc79be80d7c6d5f7e220685266ff0ccc6964a06b70f54ca3f0ac4b721266510c356c580089dc892efd6a27698b39a3921fd92169a835ee
+  metadata.gz: 34d8085f8ca406fd76ecb34d78c84b8faff82f49deb52095c00093f54300beb2a024e5b3dc9bd376aaa317cb7f126bcae2a7a2b87012ef975fa9c4760b23a206
+  data.tar.gz: f707b25361f41d930383a4e12381fbd511966a991cb68c5ae97db9fe9494278b78e165bf9f3aadbd3341c9a728c1478d74ae5a8db1482248ce0ed58d34468e33

data/README.md

@@ -1,4 +1,224 @@
-# Concurrent Ruby
+# Concurrent Ruby [![Build Status](https://secure.travis-ci.org/jdantonio/concurrent-ruby.png)](https://travis-ci.org/jdantonio/concurrent-ruby?branch=master) [![Dependency Status](https://gemnasium.com/jdantonio/concurrent-ruby.png)](https://gemnasium.com/jdantonio/concurrent-ruby)
 
-This gem does nothing. It's a placeholder into which I plan to move the concurrency
-features of my [functional-ruby](https://github.com/jdantonio/functional-ruby) gem.
+Erlang, Clojure, and Go inspired concurrent programming tools for Ruby.
+
+## Introduction
+
+The old-school "lock and synchronize" approach to concurrency is dead. The future of concurrency
+is asynchronous. Send out a bunch of independent [actors](http://en.wikipedia.org/wiki/Actor_model)
+to do your bidding and process the results when you are ready. Although the idea of the concurrent
+actor originated in the early 1970's it has only recently started catching on. Although there is
+no one "true" actor implementation (what *exactly* is "object oriented," what *exactly* is
+"concurrent programming"), many modern programming languages implement variations on the actor
+theme. This library implements a few of the most interesting and useful of those variations.
+
+Remember, *there is no silver bullet in concurrent programming.* Concurrency is hard. Very hard.
+These tools will help ease the burden, but at the end of the day it is essential that you
+*know what you are doing.*
+
+The project is hosted on the following sites:
+
+* [RubyGems project page](https://rubygems.org/gems/concurrent-ruby)
+* [Source code on GitHub](https://github.com/jdantonio/concurrent-ruby)
+* [YARD documentation on RubyDoc.info](http://rubydoc.info/github/jdantonio/concurrent-ruby/frames)
+* [Continuous integration on Travis-CI](https://travis-ci.org/jdantonio/concurrent-ruby)
+* [Dependency tracking on Gemnasium](https://gemnasium.com/jdantonio/concurrent-ruby)
+* [Follow me on Twitter](https://twitter.com/jerrydantonio)
+
+### Goals
+
+My history with high-performance, highly-concurrent programming goes back to my days with C/C++.
+I have the same scars as everyone else doing that kind of work with those languages.
+I'm fascinated by modern concurrency patterns like [Actors](http://en.wikipedia.org/wiki/Actor_model),
+[Agents](http://doc.akka.io/docs/akka/snapshot/java/agents.html), and
+[Promises](http://promises-aplus.github.io/promises-spec/). I'm equally fascinated by languages
+with strong concurrency support like [Erlang](http://www.erlang.org/doc/getting_started/conc_prog.html),
+[Go](http://golang.org/doc/articles/concurrency_patterns.html), and
+[Clojure](http://clojure.org/concurrent_programming). My goal is to implement those patterns in Ruby.
+Specifically:
+
+* Stay true to the spirit of the languages providing inspiration
+* But implement in a way that makes sense for Ruby
+* Keep the semantics as idiomatic Ruby as possible
+* Support features that make sense in Ruby
+* Exclude features that don't make sense in Ruby
+* Keep everything small
+* Be as fast as reasonably possible
+
+## Features (and Documentation)
+
+Several features from Erlang, Go, Clojure, Java, and JavaScript have been implemented thus far:
+
+* Clojure inspired [Agent](https://github.com/jdantonio/concurrent-ruby/blob/master/md/agent.md)
+* EventMachine inspired [Defer](https://github.com/jdantonio/concurrent-ruby/blob/master/md/defer.md)
+* Clojure inspired [Future](https://github.com/jdantonio/concurrent-ruby/blob/master/md/future.md)
+* Go inspired [Goroutine](https://github.com/jdantonio/concurrent-ruby/blob/master/md/goroutine.md)
+* JavaScript inspired [Promise](https://github.com/jdantonio/concurrent-ruby/blob/master/md/promise.md)
+* Java inspired [Thread Pools](https://github.com/jdantonio/concurrent-ruby/blob/master/md/thread_pool.md)
+
+### Is it any good?
+
+[Yes](http://news.ycombinator.com/item?id=3067434)
+
+### Supported Ruby versions
+
+MRI 1.9.2, 1.9.3, and 2.0. This library is pure Ruby and has no gem dependencies. It should be
+fully compatible with any Ruby interpreter that is 1.9.x compliant. I simply don't know enough
+about JRuby, Rubinius, or the others to fully support them. I can promise good karma and
+attribution on this page to anyone wishing to take responsibility for verifying compaitibility
+with any Ruby other than MRI.
+
+### Install
+
+```shell
+gem install concurrent-ruby
+```
+
+or add the following line to Gemfile:
+
+```ruby
+gem 'concurrent-ruby'
+```
+
+and run `bundle install` from your shell.
+
+Once you've installed the gem you must `require` it in your project:
+
+```ruby
+require 'concurrent'
+```
+
+### Examples
+
+For complete examples, see the specific documentation linked above. Below are a few examples to whet your appetite.
+
+#### Goroutine (Go)
+
+```ruby
+require 'concurrent'
+
+@expected = nil
+go(1, 2, 3){|a, b, c| @expected = [c, b, a] }
+sleep(0.1)
+@expected #=> [3, 2, 1]
+```
+
+#### Agent (Clojure)
+
+```ruby
+require 'concurrent'
+
+score = agent(10)
+score.value #=> 10
+
+score << proc{|current| current + 100 }
+sleep(0.1)
+score.value #=> 110
+
+score << proc{|current| current * 2 }
+sleep(0.1)
+deref score #=> 220
+
+score << proc{|current| current - 50 }
+sleep(0.1)
+score.value #=> 170
+```
+
+#### Defer (EventMachine)
+
+```ruby
+require 'concurrent'
+
+Concurrent::Defer.new{ "Jerry D'Antonio" }.
+                  then{|result| puts "Hello, #{result}!" }.
+                  rescue{|ex| puts ex.message }.
+                  go
+
+#=> Hello, Jerry D'Antonio!
+
+operation = proc{ raise StandardError.new('Boom!') }
+callback  = proc{|result| puts result }
+errorback = proc{|ex| puts ex.message }
+defer(operation, callback, errorback)
+sleep(0.1)
+
+#=> "Boom!"
+```
+
+#### Future (Clojure)
+
+```ruby
+require 'concurrent'
+
+count = future{ sleep(1); 10 }
+count.state #=> :pending
+# do stuff...
+count.value #=> 10 (after blocking)
+deref count #=> 10
+```
+
+#### Promise (JavaScript)
+
+```ruby
+require 'concurrent'
+
+p = promise("Jerry", "D'Antonio"){|a, b| "#{a} #{b}" }.
+    then{|result| "Hello #{result}." }.
+    rescue(StandardError){|ex| puts "Boom!" }.
+    then{|result| "#{result} Would you like to play a game?"}
+sleep(1)
+p.value #=> "Hello Jerry D'Antonio. Would you like to play a game?" 
+```
+
+#### Thread Pools
+
+```ruby
+require 'concurrent'
+
+pool = Concurrent::FixedThreadPool.new(10)
+@expected = 0
+pool.post{ sleep(0.5); @expected += 100 }
+pool.post{ sleep(0.5); @expected += 100 }
+pool.post{ sleep(0.5); @expected += 100 }
+@expected #=> nil
+sleep(1)
+@expected #=> 300
+
+pool = Concurrent::CachedThreadPool.new
+@expected = 0
+pool << proc{ sleep(0.5); @expected += 10 }
+pool << proc{ sleep(0.5); @expected += 10 }
+pool << proc{ sleep(0.5); @expected += 10 }
+@expected #=> 0
+sleep(1)
+@expected #=> 30
+```
+
+## Copyright
+
+*Concurrent Ruby* is Copyright &copy; 2013 [Jerry D'Antonio](https://twitter.com/jerrydantonio).
+It is free software and may be redistributed under the terms specified in the LICENSE file.
+
+## License
+
+Released under the MIT license.
+
+http://www.opensource.org/licenses/mit-license.php  
+
+> Permission is hereby granted, free of charge, to any person obtaining a copy  
+> of this software and associated documentation files (the "Software"), to deal  
+> in the Software without restriction, including without limitation the rights  
+> to use, copy, modify, merge, publish, distribute, sublicense, and/or sell  
+> copies of the Software, and to permit persons to whom the Software is  
+> furnished to do so, subject to the following conditions:  
+> 
+> The above copyright notice and this permission notice shall be included in  
+> all copies or substantial portions of the Software.  
+> 
+> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR  
+> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,  
+> FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE  
+> AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER  
+> LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,  
+> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN  
+> THE SOFTWARE.  

data/lib/concurrent.rb

@@ -1 +1,20 @@
+require 'thread'
+
 require 'concurrent/version'
+
+require 'concurrent/event'
+
+require 'concurrent/agent'
+require 'concurrent/defer'
+require 'concurrent/future'
+require 'concurrent/goroutine'
+require 'concurrent/promise'
+require 'concurrent/obligation'
+
+require 'concurrent/thread_pool'
+require 'concurrent/cached_thread_pool'
+require 'concurrent/fixed_thread_pool'
+
+require 'concurrent/global_thread_pool'
+
+require 'concurrent/event_machine_defer_proxy' if defined?(EventMachine)

data/lib/concurrent/agent.rb

@@ -0,0 +1,130 @@
+require 'observer'
+require 'thread'
+
+require 'concurrent/global_thread_pool'
+
+module Concurrent
+
+  # An agent is a single atomic value that represents an identity. The current value
+  # of the agent can be requested at any time (#deref). Each agent has a work queue and operates on
+  # the global thread pool. Consumers can #post code blocks to the agent. The code block (function)
+  # will receive the current value of the agent as its sole parameter. The return value of the block
+  # will become the new value of the agent. Agents support two error handling modes: fail and continue.
+  # A good example of an agent is a shared incrementing counter, such as the score in a video game.
+  class Agent
+    include Observable
+
+    TIMEOUT = 5
+
+    attr_reader :initial
+    attr_reader :timeout
+
+    def initialize(initial, timeout = TIMEOUT)
+      @value = initial
+      @timeout = timeout
+      @rescuers = []
+      @validator = nil
+      @queue = Queue.new
+
+      $GLOBAL_THREAD_POOL << proc{ work }
+    end
+
+    def value(timeout = 0) return @value; end
+    alias_method :deref, :value
+
+    def rescue(clazz = Exception, &block)
+      @rescuers << Rescuer.new(clazz, block) if block_given?
+      return self
+    end
+    alias_method :catch, :rescue
+    alias_method :on_error, :rescue
+
+    def validate(&block)
+      @validator = block if block_given?
+      return self
+    end
+    alias_method :validates, :validate
+    alias_method :validate_with, :validate
+    alias_method :validates_with, :validate
+
+    def post(&block)
+      return @queue.length unless block_given?
+      @queue << block
+      return @queue.length
+    end
+
+    def <<(block)
+      self.post(&block)
+      return self
+    end
+
+    def length
+      @queue.length
+    end
+    alias_method :size, :length
+    alias_method :count, :length
+
+    alias_method :add_watch, :add_observer
+
+    private
+
+    # @private
+    Rescuer = Struct.new(:clazz, :block)
+
+    # @private
+    def try_rescue(ex) # :nodoc:
+      rescuer = @rescuers.find{|r| ex.is_a?(r.clazz) }
+      rescuer.block.call(ex) if rescuer
+    rescue Exception => e
+      # supress
+    end
+
+    # @private
+    def work # :nodoc:
+      loop do
+        Thread.pass
+        handler = @queue.pop
+        begin
+          result = Timeout.timeout(@timeout){
+            handler.call(@value)
+          }
+          if @validator.nil? || @validator.call(result)
+            @value = result
+            changed
+            notify_observers(Time.now, @value)
+          end
+        rescue Exception => ex
+          try_rescue(ex)
+        end
+      end
+    end
+  end
+end
+
+module Kernel
+
+  def agent(initial, timeout = Concurrent::Agent::TIMEOUT)
+    return Concurrent::Agent.new(initial, timeout)
+  end
+  module_function :agent
+
+  def deref(agent, timeout = nil)
+    if agent.respond_to?(:deref)
+      return agent.deref(timeout)
+    elsif agent.respond_to?(:value)
+      return agent.deref(timeout)
+    else
+      return nil
+    end
+  end
+  module_function :deref
+
+  def post(agent, &block)
+    if agent.respond_to?(:post)
+      return agent.post(&block)
+    else
+      return nil
+    end
+  end
+  module_function :deref
+end

data/lib/concurrent/cached_thread_pool.rb

@@ -0,0 +1,122 @@
+require 'thread'
+
+require 'concurrent/thread_pool'
+require 'functional/utilities'
+
+module Concurrent
+
+  def self.new_cached_thread_pool
+    return CachedThreadPool.new
+  end
+
+  class CachedThreadPool < ThreadPool
+    behavior(:thread_pool)
+
+    DEFAULT_GC_INTERVAL = 60
+    DEFAULT_THREAD_IDLETIME = 60
+
+    attr_reader :working
+
+    def initialize(opts = {})
+      @gc_interval = opts[:gc_interval] || DEFAULT_GC_INTERVAL
+      @thread_idletime = opts[:thread_idletime] || DEFAULT_THREAD_IDLETIME
+      super()
+      @working = 0
+      @mutex = Mutex.new
+    end
+
+    def kill
+      @status = :killed
+      @mutex.synchronize do
+        @pool.each{|t| Thread.kill(t.thread) }
+      end
+    end
+
+    def size
+      return @pool.length
+    end
+
+    def post(*args, &block)
+      raise ArgumentError.new('no block given') unless block_given?
+      if running?
+        collect_garbage if @pool.empty?
+        @mutex.synchronize do
+          if @working >= @pool.length
+            create_worker_thread
+          end
+          @queue << [args, block]
+        end
+        return true
+      else
+        return false
+      end
+    end
+
+    # @private
+    def status # :nodoc:
+      @mutex.synchronize do
+        @pool.collect do |worker|
+          [
+            worker.status,
+            worker.status == :idle ? delta(worker.idletime, timestamp) : nil,
+            worker.thread.status
+          ]
+        end
+      end
+    end
+
+    private
+
+    Worker = Struct.new(:status, :idletime, :thread)
+
+    # @private
+    def create_worker_thread # :nodoc:
+      worker = Worker.new(:idle, timestamp, nil)
+
+      worker.thread = Thread.new(worker) do |me|
+
+        loop do
+          task = @queue.pop
+
+          @working += 1
+          me.status = :working
+
+          if task == :stop
+            me.status = :stopping
+            break
+          else
+            task.last.call(*task.first)
+            @working -= 1
+            me.status = :idle
+            me.idletime = timestamp
+          end
+        end
+
+        @pool.delete(me)
+        if @pool.empty?
+          @termination.set
+          @status = :shutdown unless killed?
+        end
+      end
+
+      @pool << worker
+    end
+
+    # @private
+    def collect_garbage # :nodoc:
+      @collector = Thread.new do
+        loop do
+          sleep(@gc_interval)
+          @mutex.synchronize do
+            @pool.reject! do |worker|
+              worker.thread.status.nil? ||
+                (worker.status == :idle && @thread_idletime >= delta(worker.idletime, timestamp))
+            end
+          end
+          @working = @pool.count{|worker| worker.status == :working}
+          break if @pool.empty?
+        end
+      end
+    end
+  end
+end