engineyard-serverside 1.5.21 → 1.5.23.ruby19

data/lib/vendor/dataflow/README.textile

@@ -1,290 +0,0 @@
-h1. What's this?
-
-A Ruby library that adds Dataflow variables (inspired by the Oz
-language). Dataflow variables have the property that they can only
-be bound/assigned to once, or have an equivalent value as an existing
-assignment (see "unification").
-
-Dataflow variables must be declared before they are used, and can be
-passed around as data without actually being bound. If the variable
-gets used (in this library this means a method call) while being
-unbound then the currently executing thread will suspend.
-
-h1. What's the point?
-
-Ruby is Object Oriented (with the ability to mutate local, instance,
-class, and global variables, and even constants), and on top of that
-it has powerful reflection and meta-programming abilities. While these
-features are useful for certain problems, they are not within the
-declarative model. Staying in the declarative model gives one 2 advantages:
-# It is easy to reason about what the program does
-# Simple but powerful concurrency is possible
-
-Ruby, like many other OO languages, is facing the hurdles of taking
-advantage of the increase of processor cores within a simple parallel
-programming model. This library lets you program Ruby in the
-declarative concurrent model when you need to take advantage of multiple cores
-(assuming a Ruby implementation that uses native threads in one way or
-another).
-
-The trick to this kind of programming is binding variables from other
-threads. The nice thing is that many existing
-libraries/classes/methods can still be used, just avoid
-side-effects. Use regular Ruby threading to create threads, use
-"local" or "declare" to create new variables, and use "unify" to bind
-variables.
-
-h1. Install
-
-To install the latest release as a gem:
-<pre>sudo gem install dataflow</pre>
-
-h1. IRC
-
-<pre>#dataflow-gem @ freenode.net</pre>
-
-h1. Examples
-
-<pre>
-# Local variables
-include Dataflow
- 
-local do |x, y, z|
-  # notice how the order automatically gets resolved
-  Thread.new { unify y, x + 2 }
-  Thread.new { unify z, y + 3 }
-  Thread.new { unify x, 1 }
-  z #=> 6
-end
-</pre>
-
-<pre>
-# Module methods version
- 
-Dataflow.local do |x, y, z|
-  # notice how the order automatically gets resolved
-  Thread.new { Dataflow.unify y, x + 2 }
-  Thread.new { Dataflow.unify z, y + 3 }
-  Thread.new { Dataflow.unify x, 1 }
-  z #=> 6
-end
-
-# Note that a gobal Dataflow.declare is not supported
-</pre>
-
-<pre>
-# Instance variables
-class AnimalHouse
-  include Dataflow
-  declare :small_cat, :big_cat
-
-  def fetch_big_cat
-    Thread.new { unify big_cat, small_cat.upcase }
-    unify small_cat, 'cat'
-    big_cat
-  end
-end
-
-AnimalHouse.new.fetch_big_cat #=> 'CAT'
-</pre>
-
-<pre>
-# Data-driven concurrency
-include Dataflow
-
-local do |stream, doubles, triples, squares|
-  unify stream, Array.new(5) { Dataflow::Variable.new }
-
-  Thread.new { unify doubles, stream.map {|n| n*2 } }
-  Thread.new { unify triples, stream.map {|n| n*3 } }
-  Thread.new { unify squares, stream.map {|n| n**2 } }  
-
-  Thread.new { stream.each {|x| unify x, rand(100) } }
-
-  puts "original: #{stream.inspect}"
-  puts "doubles:  #{doubles.inspect}"
-  puts "triples:  #{triples.inspect}"
-  puts "squares:  #{squares.inspect}"  
-end
-</pre>
-
-<pre>
-# By-need trigger laziness
-include Dataflow
-
-local do |x, y, z|
-  Thread.new { unify y, by_need { 4 } }
-  Thread.new { unify z, x + y }
-  Thread.new { unify x, by_need { 3 } }
-  z #=> 7
-end
-</pre>
-
-<pre>
-# Need-later future expressions
-include Dataflow
-
-local do |x, y, z|
-  unify y, need_later { 4 }
-  unify z, need_later { x + y }
-  unify x, need_later { 3 }
-  z #=> 7
-end        
-</pre>
-
-<pre>
-include Dataflow
-
-# flow without parameters
-local do |x|
-  flow do 
-    # other stuff
-    unify x, 1337
-  end
-  x #=> 1337
-end
-
-# flow with an output parameter
-local do |x|
-  flow(x) do
-    # other stuff
-    1337
-  end
-  x #=> 1337
-end
-</pre>
-
-<pre>
-# barrier
-include Dataflow
-
-local do |lock1, lock2|
-  flow { unify lock1, :unlocked }
-  flow { unify lock2, :unlocked }
-  barrier lock1, lock2
-  puts "Barrier broken!"  
-end
-</pre>
-
-<pre>
-# FutureQueue
-include Dataflow
-
-local do |queue, first, second|
-  unify queue, Dataflow::FutureQueue.new
-  queue.pop first
-  queue.push 1
-  queue.push 2
-  queue.pop second
-  first #=> 1
-  second #=> 2
-end
-</pre>
-
-h1. Anonymous variables
-
-Sometimes you may want to pack a data structure with variables that do not need to be referenced with labels. For those cases anonymous variables are a good choice, here are some options:
-
-<pre>
-include Dataflow
-Array.new(3) { Dataflow::Variable.new }
-Array.new(3) { Dataflow.local }
-Array.new(3) { local }
-# and technically not anonymous
-Array.new(3) { local {|v| v } }
-</pre>
-
-h1. Debugging
-
-If you are having trouble and need to debug dataflow variables, simply call #inspect.
-
-If the variable has already been bound, it call inspect on its bound value like normal.However, if the variable is not bound yet then you will get a special string that contains the proxies #__id__ that you can use to track down which proxy objects are being passed around to which parts of your program:
-<pre>
-include Dataflow
-local do |my_var| 
-  my_var.inspect # => #<Dataflow::Variable:2637860 unbound> 
-end
-</pre>
-
-h1. Fork method customization
-
-By default both #flow and #need_later use Thread.fork as their fork method. Youc an access the fork method via Dataflow.forker.
-
-If you would like to use a custom forker, simple set it to an object that responds to #call and internally calls a block passed to it (for an example of a synchronous forker, see spec/forker_spec.rb):
-<pre>
-Dataflow.forker = MyClass.method(:fork_with_threadpool)
-</pre>
-
-Also note that #flow is used interally by #need_later, in case you want to override that specifically.
-
-h1. Ports using Dataflow
-
-Ports are an extension of the declarative concurrent model to support nondeterministic behavior. They accomplish this through the use of a single state variable. Ports are also inspired by the Oz language.
-
-An Actor class in the style of Erlang message-passing processes is also provided. It makes use of the asynchronous behavior of ports, but otherwise uses no state variables.
-
-h1. Examples using Ports
-
-<pre>
-include Dataflow
-
-local do |port, stream|
-  unify port, Dataflow::Port.new(stream)
-  Thread.new {port.send 2}
-  Thread.new {port.send 8}
-  Thread.new {port.send 1024}
-  stream.take(3).sort #=> [2, 8, 1024]
-end
-</pre>
-
-h1. Examples using Actors
-
-<pre>
-include Dataflow
-
-Ping = Actor.new {
-  3.times {
-    case receive
-    when "Ping"
-      puts "Ping"
-      Pong.send "Pong"
-    end
-  }
-}
-
-Pong = Actor.new {
-  3.times {
-    case receive
-    when "Pong"
-      puts "Pong"
-      Ping.send "Ping"
-    end
-  }
-}
-
-Actor.new { Ping.send "Ping" }
-
-Ping.join
-Pong.join
-</pre>
-
-h1. Equality
-
-Most Ruby implmentations will not use method calls for equality
-operations in base types/classes. This means equality between dataflow
-variables and those base types will not behave as expected. Require
-the following to get equality on base types that uses method calls,
-while still passing rubyspec:
-
-<pre>require "dataflow/equality"</pre>
-
-h1. References
-
-The basis of dataflow variables around a language is not common among
-popular languages and may be confusing to some. For an in-depth
-introduction to the Oz language and the techniques used in this
-library (including by_need triggers, port objects, and comparisons to Erlang message passing) see the book "Concepts, Techniques, and Models of Computer Programming":http://en.wikipedia.org/wiki/Concepts,_Techniques,_and_Models_of_Computer_Programming
-
-h1. Contributors
-
-larrytheliquid, amiller

data/lib/vendor/dataflow/Rakefile

@@ -1,36 +0,0 @@
-require "rubygems"
-require "rake/gempackagetask"
-require "rake/clean"
-require "spec/rake/spectask"
-require File.expand_path("./dataflow")
-
-Spec::Rake::SpecTask.new do |t|
-  t.spec_opts = ['--options', "\"#{File.dirname(__FILE__)}/spec/spec.opts\""]
-end
-
-desc "Run the specs"
-task :default => :spec
-
-spec = Gem::Specification.new do |s|
-  s.name              = "dataflow"
-  s.rubyforge_project = s.name
-  s.version           = Dataflow::VERSION
-  s.author            = "Larry Diehl"
-  s.email             = "larrytheliquid" + "@" + "gmail.com"
-  s.homepage          = "http://github.com/larrytheliquid/dataflow"
-  s.summary           = "Dataflow concurrency for Ruby (inspired by the Oz language)"
-  s.description       = s.summary
-  s.files             = %w[LICENSE HISTORY Rakefile README.textile dataflow.rb] + Dir["dataflow/**/*"] + Dir["examples/**/*"]
-  s.require_path      = '.'
-  s.test_files        = Dir["spec/**/*"]
-end
-
-Rake::GemPackageTask.new(spec) do |package|
-  package.gem_spec = spec
-end
-
-desc 'Install the package as a gem.'
-task :install => [:clean, :package] do
-  gem = Dir['pkg/*.gem'].first
-  sh "sudo gem install --no-rdoc --no-ri --local #{gem}"
-end

data/lib/vendor/dataflow/dataflow/actor.rb

@@ -1,22 +0,0 @@
-module Dataflow
-  class Actor < Thread
-    def initialize(&block)
-      @stream = Variable.new
-      @port = Port.new(@stream)
-      # Run this block in a new thread
-      super { instance_eval &block }
-    end
-
-    def send message
-      @port.send message
-    end
-
-  private
-    
-    def receive
-      result = @stream.head
-      @stream = @stream.tail
-      result
-    end
-  end
-end

data/lib/vendor/dataflow/dataflow/equality.rb

@@ -1,44 +0,0 @@
-# The purpose of this file is to make equality use method calls in as
-# many Ruby implementations as possible. If method calls are used,
-# then equality operations using dataflow variaables becomes seemless
-# I realize overriding core classes is a pretty nasty hack, but if you
-# have a better idea that also passes the equality_specs then I'm all
-# ears. Please run the rubyspec before committing changes to this file.
-
-class Object
-  alias original_equality ==
-
-  def ==(other)
-    object_id == other.object_id
-  end
-end
-
-class Symbol
-  alias original_equality ==
-
-  def ==(other)
-    object_id == other.object_id
-  end
-end
-
-class Regexp
-  alias original_equality ==
-
-  if /lol/.respond_to?(:encoding)
-    def ==(other)
-      other.is_a?(Regexp) &&
-      casefold? == other.casefold? &&
-      encoding == other.encoding &&
-      options == other.options &&
-      source == other.source
-    end
-  else
-    def ==(other)
-      other.is_a?(Regexp) &&
-      casefold? == other.casefold? &&
-      kcode == other.kcode &&
-      options == other.options &&
-      source == other.source
-    end
-  end
-end

data/lib/vendor/dataflow/dataflow/future_queue.rb

@@ -1,24 +0,0 @@
-module Dataflow
-  class FutureQueue
-    include Dataflow
-    declare :push_port, :pop_port
-    
-    def initialize
-      local do |pushed, popped|
-        unify push_port, Dataflow::Port.new(pushed)
-        unify pop_port, Dataflow::Port.new(popped)
-        
-        Thread.new {
-          loop do
-            barrier pushed.head, popped.head
-            unify popped.head, pushed.head
-            pushed, popped = pushed.tail, popped.tail
-          end
-        }
-      end
-    end
-
-    def push(x) push_port.send x end
-    def pop(x) pop_port.send x end
-  end
-end

data/lib/vendor/dataflow/dataflow/port.rb

@@ -1,54 +0,0 @@
-require 'thread'
-
-module Dataflow
-  class Port  
-    include Dataflow
-    LOCK = Mutex.new
-
-    class Stream
-      include Dataflow
-      declare :head, :tail
-    
-      # Defining each allows us to use the enumerable mixin
-      # None of the list can be garbage collected less the head is
-      # garbage collected, so it will grow indefinitely even though
-      # the function isn't recursive.
-      include Enumerable
-      def each
-        s = self
-        loop do
-          yield s.head
-          s = s.tail
-        end
-      end
-
-      # Backported Enumerable#take for any 1.8.6 compatible Ruby
-      unless method_defined?(:take)
-        def take(num)
-          result = []
-          each_with_index do |x, i|
-            return result if num == i
-            result << x
-          end
-        end
-      end
-    end
-  
-    # Create a stream object, bind it to the input variable
-    # Instance variables are necessary because @end is state
-    def initialize(x)
-      @end = Stream.new
-      unify x, @end
-    end
-  
-    # This needs to be synchronized because it uses @end as state
-    def send value
-      LOCK.synchronize do
-        unify @end.head, value
-        unify @end.tail, Stream.new
-        @end = @end.tail
-      end
-    end
-  end
-end
-

data/lib/vendor/dataflow/dataflow.rb

@@ -1,124 +0,0 @@
-require 'monitor'
-
-module Dataflow
-  VERSION = "0.3.1"
-  class << self
-    attr_accessor :forker
-  end
-  self.forker = Thread.method(:fork)
-
-  def self.included(cls)
-    class << cls
-      def declare(*readers)
-        readers.each do |name|
-          class_eval <<-RUBY
-            def #{name}
-              return @__dataflow_#{name}__ if defined? @__dataflow_#{name}__
-              Variable::LOCK.synchronize { @__dataflow_#{name}__ ||= Variable.new }
-            end
-          RUBY
-        end
-      end
-    end
-  end
-
-  def local(&block)
-    return Variable.new unless block_given?
-    vars = Array.new(block.arity) { Variable.new }
-    block.call *vars
-  end
-
-  def unify(variable, value)
-    variable.__unify__ value
-  end
-
-  def by_need(&block)
-    Variable.new &block
-  end
-
-  def barrier(*variables)
-    variables.each{|v| v.__wait__ }
-  end
-
-  def flow(output=nil, &block)
-    Dataflow.forker.call do
-      result = block.call
-      unify output, result if output
-    end
-  end
-
-  def need_later(&block)
-    local do |future|
-      flow(future) { block.call }
-      future
-    end
-  end
-
-  extend self
-
-  # Note that this class uses instance variables directly rather than nicely
-  # initialized instance variables in get/set methods for memory and
-  # performance reasons
-  class Variable
-    # Briefly disable the warning we would get when undefining object_id.
-    # We actually rely on the ability to do that, so...
-    v = $VERBOSE; $VERBOSE = nil
-    instance_methods.each { |m| undef_method(m) unless m =~ /^__|instance_eval/ }
-    $VERBOSE = v # back to sanity
-    LOCK = Monitor.new
-    def initialize(&block) @__trigger__ = block if block_given? end
-
-    # Lazy-load conditions to be nice on memory usage
-    def __binding_condition__() @__binding_condition__ ||= LOCK.new_cond end
-
-    def __unify__(value)
-      LOCK.synchronize do
-        __activate_trigger__ if @__trigger__
-        if @__bound__
-          return @__value__.__unify__(value) if @__value__.__dataflow__? rescue nil
-          raise UnificationError, "#{@__value__.inspect} != #{value.inspect}" if self != value
-        else
-          @__value__ = value
-          @__bound__ = true
-          __binding_condition__.broadcast # wakeup all method callers
-          @__binding_condition__ = nil # GC
-        end
-      end
-      @__value__
-    end
-
-    def __activate_trigger__
-      @__value__ = @__trigger__.call
-      @__bound__ = true
-      @__trigger__ = nil # GC
-    end
-
-    def __wait__
-      LOCK.synchronize do
-        unless @__bound__
-          if @__trigger__
-            __activate_trigger__
-          else
-            __binding_condition__.wait
-          end
-        end
-      end unless @__bound__
-    end
-
-    def method_missing(name, *args, &block)
-      return "#<Dataflow::Variable:#{__id__} unbound>" if !@__bound__ && name == :inspect
-      __wait__
-      @__value__.__send__(name, *args, &block)
-    end
-
-    def __dataflow__?
-      true
-    end
-  end
-
-  UnificationError = Class.new StandardError
-end
-
-require "#{File.dirname(__FILE__)}/dataflow/port"
-require "#{File.dirname(__FILE__)}/dataflow/actor"
-require "#{File.dirname(__FILE__)}/dataflow/future_queue"

data/lib/vendor/dataflow/examples/barrier.rb

@@ -1,9 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-include Dataflow
-
-local do |lock1, lock2|
-  flow { unify lock1, :unlocked }
-  flow { unify lock2, :unlocked }
-  barrier lock1, lock2
-  puts "Barrier broken!"  
-end

data/lib/vendor/dataflow/examples/data_driven.rb

@@ -1,17 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-include Dataflow
-
-local do |stream, doubles, triples, squares|
-  unify stream, Array.new(5) { Dataflow::Variable.new }
-  
-  Thread.new { unify doubles, stream.map {|n| n*2 } }
-  Thread.new { unify triples, stream.map {|n| n*3 } }
-  Thread.new { unify squares, stream.map {|n| n**2 } }  
-
-  Thread.new { stream.each {|x| unify x, rand(100) } }
-
-  puts "original: #{stream.inspect}"
-  puts "doubles:  #{doubles.inspect}"
-  puts "triples:  #{triples.inspect}"
-  puts "squares:  #{squares.inspect}"  
-end

data/lib/vendor/dataflow/examples/dataflow_http_gets.rb

@@ -1,13 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-require 'net/http'
-include Dataflow
-
-# Be gentle running this one
-Thread.abort_on_exception = true
-local do |stream, branding_occurences, number_of_mentions|
-  unify stream, Array.new(10) { Dataflow::Variable.new }
-  stream.each {|s| Thread.new(s) {|v| unify v, Net::HTTP.get_response(URI.parse("http://www.cuil.com/search?q=#{rand(1000)}")).body } }
-  Thread.new { unify branding_occurences, stream.map {|http_body| http_body.scan /cuil/ } }
-  Thread.new { unify number_of_mentions, branding_occurences.map {|occurences| occurences.length } }
-  puts number_of_mentions.inspect
-end

data/lib/vendor/dataflow/examples/flow.rb

@@ -1,20 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-include Dataflow
-
-# flow without parameters
-local do |x|
-  flow do 
-    # other stuff
-    unify x, 1337
-  end
-  puts x
-end
-
-# flow with an output parameter
-local do |x|
-  flow(x) do
-    # other stuff
-    1337
-  end
-  puts x
-end

data/lib/vendor/dataflow/examples/future_http_gets.rb

@@ -1,12 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-require 'net/http'
-include Dataflow
-
-# Be gentle running this one
-Thread.abort_on_exception = true
-local do |stream, branding_occurences, number_of_mentions|
-  unify stream, Array.new(10) { need_later { Net::HTTP.get_response(URI.parse("http://www.cuil.com/search?q=#{rand(1000)}")).body } }
-  unify branding_occurences, need_later { stream.map {|http_body| http_body.scan /cuil/ } }
-  unify number_of_mentions, need_later { branding_occurences.map {|occurences| occurences.length } }
-  puts number_of_mentions.inspect
-end

data/lib/vendor/dataflow/examples/future_queue.rb

@@ -1,11 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-include Dataflow
-
-local do |queue, first, second|
-  unify queue, Dataflow::FutureQueue.new
-  queue.pop first
-  queue.push 1
-  queue.push 2
-  queue.pop second
-  puts "first:  #{first}, second: #{second}"
-end

data/lib/vendor/dataflow/examples/instance_variables.rb

@@ -1,15 +0,0 @@
-require "#{File.dirname(__FILE__)}/../dataflow"
-
-class AnimalHouse
-  include Dataflow
-  declare :small_cat, :big_cat
-
-  def fetch_big_cat
-    Thread.new { unify big_cat, small_cat.upcase }
-    unify small_cat, 'cat'
-    big_cat
-  end
-end
-
-puts AnimalHouse.new.fetch_big_cat
-