ey-deploy 0.7.0 → 0.7.1

data/lib/vendor/dataflow/README.textile

@@ -0,0 +1,290 @@
+h1. What's this?
+
+A Ruby library that adds Dataflow variables (inspired by the Oz
+language). Dataflow variables have the property that they can only
+be bound/assigned to once, or have an equivalent value as an existing
+assignment (see "unification").
+
+Dataflow variables must be declared before they are used, and can be
+passed around as data without actually being bound. If the variable
+gets used (in this library this means a method call) while being
+unbound then the currently executing thread will suspend.
+
+h1. What's the point?
+
+Ruby is Object Oriented (with the ability to mutate local, instance,
+class, and global variables, and even constants), and on top of that
+it has powerful reflection and meta-programming abilities. While these
+features are useful for certain problems, they are not within the
+declarative model. Staying in the declarative model gives one 2 advantages:
+# It is easy to reason about what the program does
+# Simple but powerful concurrency is possible
+
+Ruby, like many other OO languages, is facing the hurdles of taking
+advantage of the increase of processor cores within a simple parallel
+programming model. This library lets you program Ruby in the
+declarative concurrent model when you need to take advantage of multiple cores
+(assuming a Ruby implementation that uses native threads in one way or
+another).
+
+The trick to this kind of programming is binding variables from other
+threads. The nice thing is that many existing
+libraries/classes/methods can still be used, just avoid
+side-effects. Use regular Ruby threading to create threads, use
+"local" or "declare" to create new variables, and use "unify" to bind
+variables.
+
+h1. Install
+
+To install the latest release as a gem:
+<pre>sudo gem install dataflow</pre>
+
+h1. IRC
+
+<pre>#dataflow-gem @ freenode.net</pre>
+
+h1. Examples
+
+<pre>
+# Local variables
+include Dataflow
+ 
+local do |x, y, z|
+  # notice how the order automatically gets resolved
+  Thread.new { unify y, x + 2 }
+  Thread.new { unify z, y + 3 }
+  Thread.new { unify x, 1 }
+  z #=> 6
+end
+</pre>
+
+<pre>
+# Module methods version
+ 
+Dataflow.local do |x, y, z|
+  # notice how the order automatically gets resolved
+  Thread.new { Dataflow.unify y, x + 2 }
+  Thread.new { Dataflow.unify z, y + 3 }
+  Thread.new { Dataflow.unify x, 1 }
+  z #=> 6
+end
+
+# Note that a gobal Dataflow.declare is not supported
+</pre>
+
+<pre>
+# Instance variables
+class AnimalHouse
+  include Dataflow
+  declare :small_cat, :big_cat
+
+  def fetch_big_cat
+    Thread.new { unify big_cat, small_cat.upcase }
+    unify small_cat, 'cat'
+    big_cat
+  end
+end
+
+AnimalHouse.new.fetch_big_cat #=> 'CAT'
+</pre>
+
+<pre>
+# Data-driven concurrency
+include Dataflow
+
+local do |stream, doubles, triples, squares|
+  unify stream, Array.new(5) { Dataflow::Variable.new }
+
+  Thread.new { unify doubles, stream.map {|n| n*2 } }
+  Thread.new { unify triples, stream.map {|n| n*3 } }
+  Thread.new { unify squares, stream.map {|n| n**2 } }  
+
+  Thread.new { stream.each {|x| unify x, rand(100) } }
+
+  puts "original: #{stream.inspect}"
+  puts "doubles:  #{doubles.inspect}"
+  puts "triples:  #{triples.inspect}"
+  puts "squares:  #{squares.inspect}"  
+end
+</pre>
+
+<pre>
+# By-need trigger laziness
+include Dataflow
+
+local do |x, y, z|
+  Thread.new { unify y, by_need { 4 } }
+  Thread.new { unify z, x + y }
+  Thread.new { unify x, by_need { 3 } }
+  z #=> 7
+end
+</pre>
+
+<pre>
+# Need-later future expressions
+include Dataflow
+
+local do |x, y, z|
+  unify y, need_later { 4 }
+  unify z, need_later { x + y }
+  unify x, need_later { 3 }
+  z #=> 7
+end        
+</pre>
+
+<pre>
+include Dataflow
+
+# flow without parameters
+local do |x|
+  flow do 
+    # other stuff
+    unify x, 1337
+  end
+  x #=> 1337
+end
+
+# flow with an output parameter
+local do |x|
+  flow(x) do
+    # other stuff
+    1337
+  end
+  x #=> 1337
+end
+</pre>
+
+<pre>
+# barrier
+include Dataflow
+
+local do |lock1, lock2|
+  flow { unify lock1, :unlocked }
+  flow { unify lock2, :unlocked }
+  barrier lock1, lock2
+  puts "Barrier broken!"  
+end
+</pre>
+
+<pre>
+# FutureQueue
+include Dataflow
+
+local do |queue, first, second|
+  unify queue, Dataflow::FutureQueue.new
+  queue.pop first
+  queue.push 1
+  queue.push 2
+  queue.pop second
+  first #=> 1
+  second #=> 2
+end
+</pre>
+
+h1. Anonymous variables
+
+Sometimes you may want to pack a data structure with variables that do not need to be referenced with labels. For those cases anonymous variables are a good choice, here are some options:
+
+<pre>
+include Dataflow
+Array.new(3) { Dataflow::Variable.new }
+Array.new(3) { Dataflow.local }
+Array.new(3) { local }
+# and technically not anonymous
+Array.new(3) { local {|v| v } }
+</pre>
+
+h1. Debugging
+
+If you are having trouble and need to debug dataflow variables, simply call #inspect.
+
+If the variable has already been bound, it call inspect on its bound value like normal.However, if the variable is not bound yet then you will get a special string that contains the proxies #__id__ that you can use to track down which proxy objects are being passed around to which parts of your program:
+<pre>
+include Dataflow
+local do |my_var| 
+  my_var.inspect # => #<Dataflow::Variable:2637860 unbound> 
+end
+</pre>
+
+h1. Fork method customization
+
+By default both #flow and #need_later use Thread.fork as their fork method. Youc an access the fork method via Dataflow.forker.
+
+If you would like to use a custom forker, simple set it to an object that responds to #call and internally calls a block passed to it (for an example of a synchronous forker, see spec/forker_spec.rb):
+<pre>
+Dataflow.forker = MyClass.method(:fork_with_threadpool)
+</pre>
+
+Also note that #flow is used interally by #need_later, in case you want to override that specifically.
+
+h1. Ports using Dataflow
+
+Ports are an extension of the declarative concurrent model to support nondeterministic behavior. They accomplish this through the use of a single state variable. Ports are also inspired by the Oz language.
+
+An Actor class in the style of Erlang message-passing processes is also provided. It makes use of the asynchronous behavior of ports, but otherwise uses no state variables.
+
+h1. Examples using Ports
+
+<pre>
+include Dataflow
+
+local do |port, stream|
+  unify port, Dataflow::Port.new(stream)
+  Thread.new {port.send 2}
+  Thread.new {port.send 8}
+  Thread.new {port.send 1024}
+  stream.take(3).sort #=> [2, 8, 1024]
+end
+</pre>
+
+h1. Examples using Actors
+
+<pre>
+include Dataflow
+
+Ping = Actor.new {
+  3.times {
+    case receive
+    when "Ping"
+      puts "Ping"
+      Pong.send "Pong"
+    end
+  }
+}
+
+Pong = Actor.new {
+  3.times {
+    case receive
+    when "Pong"
+      puts "Pong"
+      Ping.send "Ping"
+    end
+  }
+}
+
+Actor.new { Ping.send "Ping" }
+
+Ping.join
+Pong.join
+</pre>
+
+h1. Equality
+
+Most Ruby implmentations will not use method calls for equality
+operations in base types/classes. This means equality between dataflow
+variables and those base types will not behave as expected. Require
+the following to get equality on base types that uses method calls,
+while still passing rubyspec:
+
+<pre>require "dataflow/equality"</pre>
+
+h1. References
+
+The basis of dataflow variables around a language is not common among
+popular languages and may be confusing to some. For an in-depth
+introduction to the Oz language and the techniques used in this
+library (including by_need triggers, port objects, and comparisons to Erlang message passing) see the book "Concepts, Techniques, and Models of Computer Programming":http://en.wikipedia.org/wiki/Concepts,_Techniques,_and_Models_of_Computer_Programming
+
+h1. Contributors
+
+larrytheliquid, amiller

data/lib/vendor/dataflow/Rakefile

@@ -0,0 +1,36 @@
+require "rubygems"
+require "rake/gempackagetask"
+require "rake/clean"
+require "spec/rake/spectask"
+require File.expand_path("./dataflow")
+
+Spec::Rake::SpecTask.new do |t|
+  t.spec_opts = ['--options', "\"#{File.dirname(__FILE__)}/spec/spec.opts\""]
+end
+
+desc "Run the specs"
+task :default => :spec
+
+spec = Gem::Specification.new do |s|
+  s.name              = "dataflow"
+  s.rubyforge_project = s.name
+  s.version           = Dataflow::VERSION
+  s.author            = "Larry Diehl"
+  s.email             = "larrytheliquid" + "@" + "gmail.com"
+  s.homepage          = "http://github.com/larrytheliquid/dataflow"
+  s.summary           = "Dataflow concurrency for Ruby (inspired by the Oz language)"
+  s.description       = s.summary
+  s.files             = %w[LICENSE HISTORY Rakefile README.textile dataflow.rb] + Dir["dataflow/**/*"] + Dir["examples/**/*"]
+  s.require_path      = '.'
+  s.test_files        = Dir["spec/**/*"]
+end
+
+Rake::GemPackageTask.new(spec) do |package|
+  package.gem_spec = spec
+end
+
+desc 'Install the package as a gem.'
+task :install => [:clean, :package] do
+  gem = Dir['pkg/*.gem'].first
+  sh "sudo gem install --no-rdoc --no-ri --local #{gem}"
+end

data/lib/vendor/dataflow/dataflow.rb

@@ -0,0 +1,120 @@
+require 'monitor'
+
+module Dataflow
+  VERSION = "0.3.1"
+  class << self
+    attr_accessor :forker
+  end
+  self.forker = Thread.method(:fork)
+  
+  def self.included(cls)
+    class << cls
+      def declare(*readers)
+        readers.each do |name|
+          class_eval <<-RUBY
+            def #{name}
+              return @__dataflow_#{name}__ if defined? @__dataflow_#{name}__
+              Variable::LOCK.synchronize { @__dataflow_#{name}__ ||= Variable.new }
+            end
+          RUBY
+        end
+      end
+    end
+  end
+  
+  def local(&block)
+    return Variable.new unless block_given?
+    vars = Array.new(block.arity) { Variable.new }
+    block.call *vars
+  end
+
+  def unify(variable, value)
+    variable.__unify__ value
+  end
+
+  def by_need(&block)
+    Variable.new &block
+  end
+
+  def barrier(*variables)
+    variables.each{|v| v.__wait__ }
+  end
+
+  def flow(output=nil, &block)
+    Dataflow.forker.call do
+      result = block.call
+      unify output, result if output
+    end
+  end
+
+  def need_later(&block)
+    local do |future|
+      flow(future) { block.call }
+      future
+    end
+  end
+
+  extend self
+
+  # Note that this class uses instance variables directly rather than nicely
+  # initialized instance variables in get/set methods for memory and
+  # performance reasons
+  class Variable
+    instance_methods.each { |m| undef_method m unless m =~ /^__/ }
+    LOCK = Monitor.new
+    def initialize(&block) @__trigger__ = block if block_given? end
+    
+    # Lazy-load conditions to be nice on memory usage
+    def __binding_condition__() @__binding_condition__ ||= LOCK.new_cond end
+    
+    def __unify__(value)
+      LOCK.synchronize do
+        __activate_trigger__ if @__trigger__
+        if @__bound__
+          return @__value__.__unify__(value) if @__value__.__dataflow__? rescue nil
+          raise UnificationError, "#{@__value__.inspect} != #{value.inspect}" if self != value
+        else
+          @__value__ = value
+          @__bound__ = true
+          __binding_condition__.broadcast # wakeup all method callers
+          @__binding_condition__ = nil # GC
+        end
+      end
+      @__value__
+    end
+
+    def __activate_trigger__
+      @__value__ = @__trigger__.call
+      @__bound__ = true
+      @__trigger__ = nil # GC
+    end
+
+    def __wait__
+      LOCK.synchronize do
+        unless @__bound__          
+          if @__trigger__
+            __activate_trigger__
+          else
+            __binding_condition__.wait
+          end
+        end
+      end unless @__bound__
+    end
+
+    def method_missing(name, *args, &block)
+      return "#<Dataflow::Variable:#{__id__} unbound>" if !@__bound__ && name == :inspect
+      __wait__
+      @__value__.__send__(name, *args, &block)
+    end
+
+    def __dataflow__?
+      true
+    end
+  end
+
+  UnificationError = Class.new StandardError
+end
+
+require "#{File.dirname(__FILE__)}/dataflow/port"
+require "#{File.dirname(__FILE__)}/dataflow/actor"
+require "#{File.dirname(__FILE__)}/dataflow/future_queue"

data/lib/vendor/dataflow/dataflow/actor.rb

@@ -0,0 +1,22 @@
+module Dataflow
+  class Actor < Thread
+    def initialize(&block)
+      @stream = Variable.new
+      @port = Port.new(@stream)
+      # Run this block in a new thread
+      super { instance_eval &block }
+    end
+
+    def send message
+      @port.send message
+    end
+
+  private
+    
+    def receive
+      result = @stream.head
+      @stream = @stream.tail
+      result
+    end
+  end
+end