coroutines 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 89eb0428ee0c27d848aff55a5cbe954edd6f221b
+  data.tar.gz: 4b34bc8df721b4923ba524174dbe396957d75b8e
+SHA512:
+  metadata.gz: 26b3c54acdf282c3db224f3672092bb1f2dcbd7f65d8ac461fa55daec860002fb90e6438f4b6f7dac1143759888616df3b8b2ffb523bf1cc61c02a0a910020f6
+  data.tar.gz: a1f1e7a915922b2ef9e81bcf84600003679deb879fbe56a9a45142bce74e39c96f2d550e8a65a7c8e59248518d77f3554434d1e2bf3bd4652d6b17954e4489b9

data/README.rdoc

@@ -0,0 +1,150 @@
+= Coroutines
+A library for creating and composing producer/transformer/consumer coroutines.
+Producers are already provided by Ruby's built-in Enumerator class; this
+library provides Transformer and Consumer classes that work analogously. In
+particular, they are also based on Fiber and not on threads (as in some other
+producer/consumer libraries). Also provides a module Sink, which is analogous
+to Enumerable, and Enumerable/Transformer/Sink composition using overloaded
+<= and >= operators.
+
+== Installing
+   gem install coroutines
+
+== Using
+A simple consumer:
+
+   require 'coroutines'
+
+   def counter(start)
+     result = start
+     loop { result += await }
+     "Final value: #{result}"
+   end  
+
+   co = consum_for :counter, 10  # => #<Consumer: main:counter (running)>
+   co << 10 << 1000 << 10000
+   co.close  # => "Final value: 11020"
+
+The call to Consumer#close raises StopIteration at the point at which the consumer
+last executed await. In this case, the StopIteration is caught by loop,
+causing it to terminate.
+
+Note that this is an intentionally simplistic example intended to show the
+basic library API. Of course, a counter could just as easily be implemented
+using a closure; the advantage of a consumer is that the implementation could
+involve arbitrary control structures with multiple calls to await.
+
+A simple transformer:
+
+   require 'coroutines'
+
+   def running_sum(start)
+     result = start
+     loop { result += await; yield result }
+   end  
+   
+   tr = trans_for :running_sum, 3  # => #<Transformer: main:running_sum>
+   sums = (1..10) >= tr  # => #<Enumerator: #<Transformer: main:running_sum> <= 1..10>
+   sums.to_a  # => [4, 6, 9, 13, 18, 24, 31, 39, 48, 58]
+
+   tr = trans_for :running_sum, 0  # => #<Transformer: main:running_sum>
+   collect_sums = tr >= []
+   collect_sums << 1 << 1 << 2 << 3 << 5
+   collect_sums.close  # => [1, 2, 4, 7, 12]
+
+Again, this is just a basic demonstration of the API that could be written
+without resorting to coroutines (though probably not quite as succinctly).
+
+== Sources and sinks
+As you probably know, many Ruby classes use the Enumerable mixin to provide
+common functionality like mapping and filtering of sequences of values. We can
+think of such classes as "sources" of the values yielded by their respective
+each methods. In the same way, several classes use the << operator to
+_accept_ sequences of values:
+
+   $stdout << "a" << "b" << "c"  # prints "abc"
+   [] << "a" << "b" << "c"  # => ["a", "b", "c"]
+   "" << "a" << "b" << "c"  # => "abc"
+
+The +coroutines+ library provides the mixin Sink for such classes. Among
+other methods, this provides an operator <= for "connecting" sources (i.e.
+enumerables) to sinks.
+
+   open("test.txt", "w") <= ["a", "b", "c"]  # write "abc" to test.txt
+   ["a", "b", "c"] <= ("d".."f")  # => ["a", "b", "c", "d", "e", "f"]
+   "abc" <= ("d".."f")  # => "abcdef"
+
+Note that <= closes the sink after having exhausted the enumerable,
+returning whatever #close returns. close defaults to simply returning the
+sink itself, as in the Array and String examples above. For IO, which has
+its own close implementation, this implies that the file descriptor will be
+closed after the <= operation finishes. If this is not what you want, use
+dup:
+
+   $stdout.dup <= ["a", "b", "c"]  # print "abc"
+
+For symmetry, the coroutines library augments Enumerable with an operator
+>= that mirrors <=:
+
+   ("d".."f") >= "abc"  # => "abcdef"
+
+== Pipelines involving transformers
+Re-using the running_sum example from above:
+
+   (1..10) >= trans_for(:running_sum, 0) >= proc{|x| x.to_s + "\n" } >= $stdout.dup
+
+What does this do? Well, it takes the sequences of integers from 1 to 10, then
+computes the running sum, then convers each partial sum to a string, and
+finally prints out each string to $stdout. Except that the "thens" in the
+previous sentence are not entirely correct, since the processing stages run  in
+parallel (using coroutines, so blocking IO in one stage will block all other
+stages). Instead of (1..10), we could have a File and iterate over GBs of data,
+and at no point would we need to have the entire sequence in memory.
+
+Any part of a pipeline can be passed around and stored as an individual object:
+
+  enum = (1..10) >= trans_for(:running_sum, 0) >= proc{|x| x.to_s + "\n" }  # => an Enumerator
+  cons = trans_for(:running_sum, 0) >= proc{|x| x.to_s + "\n" } >= $stdout.dup  # => a Consumer
+  trans = trans_for(:running_sum, 0) >= proc{|x| x.to_s + "\n" }  # => a Transformer
+
+== map/filter equivalent pipelines
+As shown above, a Proc object can be used in place of a Transformer. Connecting
+transformer procs to enums, sinks or other procs is special-cased to minimize
+overhead, so that something like
+
+   (1..9) >= proc{|x| x.to_s if x.even? } >= ""
+
+is just syntactic sugar for
+
+   (1..9).lazy.select{|x| x.even? }.map{|x| x.to_s }.inject(""){|memo, x| memo << x }
+
+(And yes, the latter could also be written more succinctly if Enumerator::Lazy
+would provide operations like filter_map and join.)
+
+== Links and Acknowledgements
+Inspired by {David M. Beazley's Generator Tricks}[http://www.dabeaz.com/generators] (Python)
+and {Michael Snoyman's conduit package}[https://www.fpcomplete.com/user/snoyberg/library-documentation/conduit-overview] (Haskell).
+
+== Compatibility
+Tested with MRI 1.9.3p484, 2.0.0p384 and 2.2.0dev (trunk 47827) on Linux.
+Should work on other moderately recent versions and other operating systems.
+Will probably not work on all alternative Ruby implementations, because it
+depends on the fiber library.
+
+Requiring 'coroutines' does some monkey patching to various core classes, which
+may be a compatibility issue. It is possible to load just the core module
+( Sink) and classes ( Consumer and Transformer) using
+"require 'coroutines/base'". Obviously, you will then have to instantiate
+Consumer and Transformer manually in order to use them; and you'll have to be
+more explicit when constructing certain pipelines.
+
+Patched core modules and classes are:
+Enumerable::        add Enumerable#>= operator
+IO, Array, String:: include Sink mixin
+Hash::              define Hash#<< operator and include Sink mixin
+Method::            define Method#<< operator, define Method#close as an alias
+                    for Method#receiver, and include Sink mixin
+Object::            define Object#await, Object#consum_for and Object#trans_for
+Proc::              define Proc#to_trans, Proc#<= and Proc#>=
+Symbol::            define Symbol#to_trans, Symbol#<= and Symbol#>=
+

data/lib/coroutines/base.rb

@@ -0,0 +1,329 @@
+require 'fiber'
+require 'coroutines/sink'
+
+# A class implementing consumer coroutines
+#
+# A Consumer can be created by the following methods:
+# * Object#consum_for
+# * Object#consumer
+# * Consumer.new
+# * Transformer#>=
+#
+# See Object#consum_for for an explanation of the basic concepts.
+class Consumer
+	class Yielder
+		if Fiber.instance_methods.include? :raise
+			def await
+				Fiber.yield
+			end
+		else
+			@@exception_wrapper = Class.new do
+				define_method :initialize do |*args|
+					@args = args
+				end
+				attr_reader :args
+			end
+
+			def await
+				item = Fiber.yield
+				raise(*item.args) if item.instance_of? @@exception_wrapper
+				return item
+			end
+		end
+	end
+
+	# :call-seq:
+	#   Consumer.new { |yielder| ... } -> consumer
+	#
+	# Creates a new Consumer coroutine, which can be used as a Sink.
+	#
+	# The block is called immediately with a "yielder" object as parameter.
+	# +yielder+ can be used to retrieve a value from the consumption context by
+	# calling its await method.
+	#
+	#   consum = Consumer.new do |y|
+	#     a = y.await
+	#     b = y.await
+	#     "#{a} + #{b} = #{a + b}"
+	#   end
+	#
+	#   consum << 42 << 24
+	#   consum.close  # => "42 + 24 = 66"
+	#
+	def initialize(&block)
+		@fiber = Fiber.new(&block)
+		@result = nil
+
+		self << Yielder.new
+	end
+
+	def inspect
+		status = if @fiber.alive? then "running" else @result.inspect end
+		"#<Consumer: 0x#{object_id.to_s(16)} (#{status})>"
+	end
+
+	# After the consumer has terminated (which may have happened in response to
+	# Consumer#close), result contains the value returned by the consumer.
+	attr_reader :result
+
+	# :call-seq:
+	#   consumer << obj -> consumer
+	#
+	# Feeds +obj+ as an input to the consumer.
+	def <<(obj)
+		raise StopIteration unless @fiber.alive?
+		value = @fiber.resume(obj)
+		@result = value unless @fiber.alive?
+		self
+	end
+
+	if Fiber.instance_methods.include? :raise
+		# :call-seq:
+		#   consumer.close -> obj
+		#
+		# Terminate the consumer by raising StopIteration at the point where it
+		# last requested an input value. +obj+ is the return value of the consumer.
+		def close
+			@result = @fiber.raise StopIteration if @fiber.alive?
+			return @result
+		rescue StopIteration
+			return @result
+		end
+	else
+		def close
+			wrapper = Yielder.class_variable_get(:@@exception_wrapper)
+			@result = @fiber.resume(wrapper.new(StopIteration)) if @fiber.alive?
+			return @result
+		rescue StopIteration
+			return @result
+		end
+	end
+
+	include Sink
+end
+
+# A class implementing transformer coroutines
+#
+# A Transformer can be created by the following methods:
+# * Object#trans_for
+# * Transformer.new
+#
+# Transformers are pieces of code that accept input values and produce output
+# values (without returning from their execution context like in a regular
+# method call). They are used by connecting either their input to an enumerable
+# or their output to a sink (or both, using Sink#<= or Enumerable#>=). An
+# enumerable is any object implementing an iterator method each; a sink is
+# any object implementing the << operator (which is assumed to store or
+# output the supplied values in some form).
+#
+# The input of a transformer is connected to an enumerable +enum+ using
+# trans <= enum, the result of which is a new Enumerator instance. The
+# transformer is started upon iterating over this Enumerator. Whenever the
+# transformer requests a new input value (see Object#trans_for and
+# Transformer#new for how to do this), iteration over +enum+ is resumed. The
+# output values of the transformer (again, see Object#trans_for and
+# Transformer#new) are yielded by the enclosing Enumerator.  When +enum+ is
+# exhausted, StopIteration is raised at the point where the transformer last
+# requested an input value. It is expected that the transformer will then
+# terminate without requesting any more values (though it may execute e.g. some
+# cleanup actions).
+#
+# The output of a transformer is connected to a sink using trans >= sink, the
+# result of which is a new Consumer instance. The transformer starts
+# executing right away; when it requests its first input value, the >=
+# operation returns. Input values supplied using << to the enclosing
+# Consumer are forwarded to the transformer by resuming execution at the
+# point where it last requested an input value. Output values produced by the
+# transformer are fed to sink#<<. After terminating, the result of the new
+# Consumer is the value returned by sink.close (see Consumer#result and
+# Consumer#close).
+#
+# Transformers can also be chained together by connecting the output of one the
+# input the next using trans >= other_trans or trans <= other_trans. See
+# Transformer#>= and Transformer#<= for details.
+#
+class Transformer
+	# :call-seq:
+	#   Transformer.new { |yielder| ... } -> trans
+	#
+	# Creates a new Transformer coroutine defined by the given block.
+	#
+	# The block is called with a "yielder" object as parameter. +yielder+ can
+	# be used to retrieve a value from the consumption context by calling its
+	# await method (as in Consumer.new), and to yield a value by calling
+	# its yield method (as in Enumerator.new).
+	#
+	#   running_sum = Transformer.new do |y|
+	#     result = 0
+	#     loop { result += y.await; y.yield result }
+	#   end
+	# 
+	#   (1..3) >= running_sum >= []  # => [1, 3, 6]
+	#
+	def initialize(&block)
+		@self = block
+	end
+
+	def inspect
+		"#<Transformer: 0x#{object_id.to_s(16)}>"
+	end
+
+	# :call-seq:
+	#   transformer.to_trans  -> transformer
+	#
+	# Returns self.
+	def to_trans
+		self
+	end
+
+	# :call-seq:
+	#   trans <= other_trans  -> new_trans
+	#   trans <= enum         -> new_enum
+	#
+	# In the first form, creates a new Transformer that has the input of
+	# +trans+ connected to the output of +other_trans+.
+	#
+	# In the second form, creates a new Enumerator by connecting +enum+ to
+	# the input of +trans+. See Transformer for details.
+	def <=(source)
+		if not source.respond_to? :each
+			return source.to_trans.transformer_chain self
+		end
+
+		source_enum = source.to_enum
+		enum = Enumerator.new do |y|
+			y.define_singleton_method :await do
+				source_enum.next
+			end
+			@self.call(y)
+		end
+
+		description = "#<Enumerator: #{inspect} <= #{source.inspect}>"
+		enum.define_singleton_method :inspect do
+			description
+		end
+
+		enum
+	end
+
+	# :call-seq:
+	#   trans >= other_trans  -> new_trans
+	#   trans >= sink         -> consum
+	#
+	# In the first form, creates a new Transformer that has the output of
+	# +trans+ connected to the input of +other_trans+.
+	#
+	# In the second form, creates a new Consumer by connecting the output of
+	# +trans+ to +sink+. See Transformer for details.
+	def >=(sink)
+		if not sink.respond_to? :<<
+			return transformer_chain sink.to_trans
+		end
+
+		consum = Consumer.new do |y|
+			y.define_singleton_method :yield do |args|
+				sink << args
+			end
+			y.alias_method :<<, :yield
+			begin
+				@self.call(y)
+			rescue StopIteration
+			end
+			sink.close
+		end
+
+		description = "#<Consumer: #{inspect} >= #{sink.inspect}>"
+		consum.define_singleton_method :inspect do
+			description
+		end
+
+		consum
+	end
+
+	protected
+
+	class FirstYielder < Consumer::Yielder
+		if Fiber.instance_methods.include? :raise
+			def await
+				Fiber.yield(:await)
+			end
+			def yield(*args)
+				Fiber.yield(:yield, *args)
+			end
+			alias_method :<<, :yield
+		else
+			def await
+				item = Fiber.yield(:await)
+				raise(*item.args) if item.instance_of? @@exception_wrapper
+				return item
+			end
+			def yield(*args)
+				item = Fiber.yield(:yield, *args)
+				raise(*item.args) if item.instance_of? @@exception_wrapper
+			end
+			alias_method :<<, :yield
+		end
+	end
+
+	class SecondYielder < Consumer::Yielder
+		def initialize(y, fiber)
+			@y, @fiber = y, fiber
+		end
+
+		if Fiber.instance_methods.include? :raise
+			def await
+				raise StopIteration unless @fiber.alive?
+				tag, result = @fiber.resume
+				while tag == :await do
+					x = nil
+					begin
+						x = @y.await
+					rescue Exception => e
+						tag, result = @fiber.raise(e)
+					end
+					tag, result = @fiber.resume(*x) unless x.nil?
+					raise StopIteration unless @fiber.alive?
+				end
+				result
+			end
+		else
+			def await
+				raise StopIteration unless @fiber.alive?
+				tag, result = @fiber.resume
+				while tag == :await do
+					begin
+						x = @y.await
+					rescue Exception => e
+						x = [@@exception_wrapper.new(e)]
+					end
+					tag, result = @fiber.resume(*x) if @fiber.alive?
+					raise StopIteration unless @fiber.alive?
+				end
+				result
+			end
+		end
+
+		def yield(*args)
+			@y.yield(*args)
+		end
+		alias_method :<<, :yield
+	end
+
+	def transformer_chain(other)
+		first = @self
+		second = other.instance_variable_get(:@self)
+
+		trans = Transformer.new do |y|
+			fib = Fiber.new { first.call FirstYielder.new }
+			second.call(SecondYielder.new y, fib)
+		end
+
+		description = "#{inspect} >= #{other.inspect}"
+		trans.define_singleton_method :inspect do
+			description
+		end
+
+		trans
+	end
+end
+

data/lib/coroutines/sink.rb

@@ -0,0 +1,149 @@
+# The <code>Sink</code> mixin provides classes that can accept streams of
+# values with several utility methods. Classes using this mixin must at least
+# provide an operator << which accepts a value and returns the sink instance
+# (allowing chaining). Optionally, the class may provide a close method that
+# signals end of input and (also optionally) retrives a result value.
+module Sink
+	# May be overriden by classes using the Sink mixin. The default
+	# implementation just returns self.
+	def close
+		self
+	end
+
+	# :call-seq:
+	#   sink <= enum  -> obj
+	#   sink <= trans -> new_consum
+	#
+	# In the first form, iterate over +enum+ and write each result to +sink+
+	# using <<; then return the result of sink.close.
+	#
+	# In the second form, create a new Consumer by connecting the output of
+	# +trans+ (which must be convertible to a Transformer using the
+	# to_trans method) to +sink+.
+	def <=(other)
+		if other.respond_to? :each
+			begin
+				other.each { |x| self << x }
+			rescue StopIteration
+			end
+			close
+		elsif other.respond_to? :to_trans
+			other >= self
+		end
+	end
+
+	# :call-seq:
+	#   sink.input_map{ |obj| block } -> new_sink
+	#
+	# Returns a new sink which supplies each of its input values to the given
+	# block and feeds each result of the block to +sink+.
+	def input_map(&block)
+		InputMapWrapper.new(self, &block)
+	end
+
+	# :call-seq:
+	#   sink.input_select{ |obj| block } -> new_sink
+	#
+	# Returns a new sink which feeds those inputs for which +block+ returns a
+	# true value to +sink+ and discards all others.
+	def input_select
+		InputSelectWrapper.new(self, &block)
+	end
+
+	# :call-seq:
+	#   sink.input_reject{ |obj| block } -> new_sink
+	#
+	# Returns a new sink which feeds those inputs for which +block+ returns a
+	# false value to +sink+ and discards all others.
+	def input_reject(&block)
+		InputRejectWrapper.new(self, &block)
+	end
+
+	# :call-seq:
+	#   sink.input_reduce(initial=nil){ |memo, obj| block } -> new_sink
+	#
+	# Returns a new sink which reduces its input values to a single value, as
+	# in Enumerable#reduce.  When +new_sink+ is closed, the reduced value is
+	# fed to +sink+ and +sink+ is closed as well.
+	def input_reduce(*args, &block)
+		InputReduceWrapper.new(self, *args, &block)
+	end
+
+	# Collects multiple sinks into a multicast group. Every input value of the
+	# multicast group is supplied to each of its members. When the multicast
+	# group is closed, all members are closed as well.
+	class Multicast
+		def initialize(*receivers)
+			@receivers = receivers
+		end
+		def <<(obj)
+			@receivers.each { |r| r << obj }
+			self
+		end
+		def close
+			@receivers.each(&:close)
+		end
+		include Sink
+	end
+
+	######################################################################
+	private
+	######################################################################
+
+	class InputWrapper
+		def initialize(target, &block)
+			@target = target; @block = block
+		end
+		def close
+			@target.close
+		end
+		include Sink
+	end
+
+	class InputMapWrapper < InputWrapper
+		def <<(args)
+			@target << @block.call(args)
+		end
+		def inspect
+			"#<#{@target.inspect}#input_map>"
+		end
+	end
+
+	class InputSelectWrapper < InputWrapper
+		def <<(args)
+			@target << args if @block.call(args)
+		end
+		def inspect
+			"#<#{@target.inspect}#select>"
+		end
+	end
+
+	class InputRejectWrapper < InputWrapper
+		def <<(args)
+			@target << args unless @block.call(args)
+		end
+		def inspect
+			"#<#{@target.inspect}#reject>"
+		end
+	end
+
+	class InputReduceWrapper
+		def initialize(target, initial=nil, &block)
+			@target = target; @block = block
+			@memo = initial
+		end
+		def <<(arg)
+			if @memo.nil?
+				@memo = arg
+			else
+				@memo = @block.call(@memo, arg)
+			end
+			self
+		end
+		def close
+			@target << @memo
+			@target.close
+		end
+		include Sink
+	end
+end

data/lib/coroutines.rb

@@ -0,0 +1,334 @@
+require 'coroutines/base'
+
+#--
+# Most of the Sink methods mirror Enumerable, except for <=
+# for symmetry, also add a >= method to Enumerable
+#++
+module Enumerable
+	# :call-seq:
+	#   enum >= sink  -> obj
+	#   enum >= trans -> new_enum
+	#
+	# In the first form, iterate over +enum+ and write each result to +sink+
+	# using <<; then return the result of sink.close.
+	#
+	# In the second form, create a new Enumerator by connecting +enum+ to the
+	# input of +trans+ (which must be convertible to a Transformer using the
+	# to_trans method).
+	def >=(other)
+		if other.respond_to? :<<
+			begin
+				each { |x| other << x }
+			rescue StopIteration
+			end
+			other.close
+		elsif other.respond_to? :to_trans
+			other <= self
+		end
+	end
+end
+
+# convenience alias for Sink::Multicast
+# (this is not defined when requiring just coroutines/base)
+Multicast = Sink::Multicast
+
+class IO
+	include Sink
+end
+
+class Array
+	include Sink
+end
+
+class String
+	include Sink
+end
+
+class Hash
+	# :call-seq:
+	#   hash << [key, value] -> hash
+	#
+	# Equivalent to hash[key] = value, but allows chaining:
+	#
+	#   {} << [1, "one"] << [2, "two"] << [3, "three"]  #=> {1=>"one", 2=>"two", 3=>"three"}
+	def <<(args)
+		self[args[0]] = args[1]
+		self
+	end
+	include Sink
+end
+
+class Method
+	# :call-seq:
+	#   method << [arg1, arg2, ...] -> method
+	#
+	# Equivalent to method.call(arg1, arg2, ...), but allows chaining:
+	#
+	#   method(:puts) << 1 << 2 << 3  # print each number to stdout
+	def <<(args)
+		call(*args)
+		self
+	end
+	alias_method :close, :receiver
+	include Sink
+end
+
+
+class CoroutineError < StandardError
+end
+
+# :call-seq:
+#   await -> obj
+#
+# Evaluates to the next input value from the associated consumption context. In
+# order to create a consumption context, you have to use Object#consum_for or
+# Object#trans_for on the method calling await.
+#
+# The behavior of await is undefined if the method using it is called without
+# being wrapped by consum_for or trans_for. It should be an error to do so, as
+# it is an error to use yield without an associated block; however, since await
+# is not a language primitive (like yield), it is not always possible to
+# enforce this. This is likely to change in a future version if a better
+# solution can be found.
+def await
+	yielder = Fiber.current.instance_variable_get(:@yielder)
+	raise CoroutineError, "you can't call a consumer" if yielder.nil?
+	yielder.await
+end
+
+class Object
+	# :call-seq:
+	#   consum_for(method, *args) -> consumer
+	#
+	# Creates a new Consumer coroutine from the given method. This is analogous
+	# to using Kernel#enum_for to create an Enumerator instance. The method is
+	# called immediately (with the given +args+). It executes until the first
+	# call to #await, at which point consum_for returns the Consumer
+	# instance. Calling consumer << obj resumes the consumer at the point where
+	# it last executed #await, which evaluates to +obj+.
+	#
+	# Calling consumer.close raises StopIteration at the point where the
+	# consumer last executed #await; it is expected that it will
+	# terminate without executing #await again. Consumer#close evaluates
+	# to the return
+	# value of the method.
+	#
+	# Example:
+	#
+	#   def counter(start)
+	#	 result = start
+	#	 loop { result += await }
+	#	 "Final value: #{result}"
+	#   end
+	#
+	#   co = consum_for :counter, 10  #=> #<Consumer: main:counter (running)>
+	#   co << 10 << 1000 << 10000
+	#   co.close  #=> "Final value: 11020"
+	#
+	def consum_for(meth, *args)
+		cons = Consumer.new do |y|
+			Fiber.current.instance_variable_set(:@yielder, y)
+			send(meth, *args)
+		end
+		description = "#{inspect}:#{meth}"
+		cons.define_singleton_method :inspect do
+			state = if @fiber.alive? then "running" else @result.inspect end
+			"#<Consumer: #{description} (#{state})>"
+		end
+		cons
+	end
+
+	# :call-seq:
+	#   trans_for(method, *args) -> transformer
+	#
+	# Creates a new Transformer instance that wraps the given method. This is
+	# analogous to using Kernel#enum_for to create an Enumerator instance, or
+	# using Object#consum_for to create a Consumer instance. The method is not
+	# executed immediately. The resulting Transformer can be connected to an
+	# enumerable (using transformer <= enum) or to a sink (using
+	# transformer >= sink). The point at which the transformer method gets
+	# started depends on how it is connected; in any case however, the method
+	# will be called with the +args+ given to trans_for. See Transformer
+	# for details.
+	#
+	# Within the transformer method, #await can be used to read the next
+	# input value (as in a Consumer, compare Object#consum_for), and yield can
+	# be used to produce an output value (i.e., when it is called, the method
+	# is given a block which accepts its output).
+	#
+	# Example:
+	#
+	#   def running_sum(start)
+	#     result = start
+	#     loop { result += await; yield result }
+	#   end
+	#
+	#   tr = trans_for :running_sum, 3  #=> #<Transformer: main:running_sum>
+	#   sums = (1..10) >= tr  #=> #<Enumerator: #<Transformer: main:running_sum> <= 1..10>
+	#   sums.to_a  #=> [4, 6, 9, 13, 18, 24, 31, 39, 48, 58]
+	#
+	def trans_for(meth, *args)
+		trans = Transformer.new do |y|
+			Fiber.current.instance_variable_set(:@yielder, y)
+			send(meth, *args, &y.method(:yield))
+		end
+		description ="#{inspect}:#{meth}"
+		trans.define_singleton_method :inspect do
+			"#<Transformer: #{description}>"
+		end
+		trans
+	end
+end
+
+class Symbol
+	# :call-seq:
+	#   sym.to_trans -> transformer
+	#
+	# Allows implicit conversion of Symbol to Transformer. The transformer
+	# accepts any objects as input, calls the method given by +sym+ on each and
+	# outputs all non-nil results of the method. See Proc#to_trans for details.
+	#
+	# example:
+	#
+	#   collector = [] <= :to_s
+	#   collector << 1 << 4 << 9
+	#   collector.close # => ["1", "4", "9"]
+	#
+	def to_trans
+		Transformer.new do |y|
+			loop do
+				value = y.await.send self
+				y.yield value unless value.nil?
+			end
+		end
+	end
+
+	# :call-seq:
+	#   sym <= trans  -> new_trans
+	#   sym <= enum   -> new_enum
+	#
+	# Equivalent to sym.to_trans <= trans/enum, except that it uses a more
+	# efficient implementation.
+	def <=(source)
+		to_proc <= source
+	end
+
+	# :call-seq:
+	#   sym >= trans  -> new_trans
+	#   sym >= sink   -> new_consumer
+	#
+	# Equivalent to sym.to_trans >= trans/sink, except that it uses a more
+	# efficient implementation.
+	def >=(sink)
+		to_proc >= sink
+	end
+end
+
+# Define a poor man's Enumerator::Lazy for Ruby < 2.0
+if defined? Enumerator::Lazy
+	LazyEnumerator = Enumerator::Lazy
+else
+	class LazyEnumerator
+		def initialize(obj, &block)
+			@obj = obj; @block = block
+		end
+
+		class Yielder
+			def initialize(iter_block)
+				@iter_block = iter_block
+			end
+			def yield(*values)
+				@iter_block.call(*values)
+			end
+			alias_method :<<, :yield
+		end
+
+		def each(&iter_block)
+			yielder = Yielder.new(iter_block)
+			@obj.each do |*args|
+				@block.call(yielder, *args)
+			end
+		end
+		include Enumerable
+	end
+end
+
+class Proc
+	# :call-seq:
+	#   proc.to_trans -> transformer
+	#
+	# Allows implicit conversion of Proc to Transformer. The transformer is a
+	# combination of map and filter over its input values: For each input
+	# value, +proc+ is called with the input value as parameter. Every non-nil
+	# value returned by +proc+ is yielded as an output value.
+	#
+	# This is similar to Enumerable#map followed by Array#compact, but without
+	# constructing the intermediate Array (so it's even more similar to
+	# something like enum.lazy.map(&proc).reject(&:nil?), using the lazy
+	# enumerator introduced in Ruby 2.0).
+	#
+	# Example:
+	#
+	#   (1..10) >= proc{|x| x.to_s + ", " if x.even? } >= ""
+	#   # => "2, 4, 6, 8, 10, "
+	#
+	def to_trans
+		Transformer.new do |y|
+			loop do
+				value = self.call(y.await)
+				y.yield value unless value.nil?
+			end
+		end
+	end
+
+	# :call-seq:
+	#   proc <= trans  -> new_trans
+	#   proc <= enum   -> new_enum
+	#
+	# Equivalent to proc.to_trans <= trans/enum, except that it uses a more
+	# efficient implementation.
+	def <=(source)
+		if source.respond_to? :each
+			LazyEnumerator.new(source) do |y, *args|
+				value = call(*args)
+				y << value unless value.nil?
+			end
+		elsif source.respond_to? :to_proc
+			sp = source.to_proc
+			proc do |*args|
+				value = sp.call(*args)
+				if value.nil? then nil else self.call value end
+			end
+		elsif source.respond_to? :to_trans
+			# FIXME: this could be implemented more efficiently; proc doesn't
+			#        need to run in a separate Fiber
+			self.to_trans <= source.to_trans
+		else
+			raise ArgumentError, "#{source.inspect} is neither an enumerable nor a transformer"
+		end
+	end
+
+	# :call-seq:
+	#   proc >= trans  -> new_trans
+	#   proc >= sink   -> new_consumer
+	#
+	# Equivalent to proc.to_trans >= trans/sink, except that it uses a more
+	# efficient implementation.
+	def >=(sink)
+		if sink.respond_to? :input_map
+			sink.input_reject(&:nil?).input_map(&self)
+		elsif sink.respond_to? :to_proc
+			sp = sink.to_proc
+			proc do |*args|
+				value = self.call(*args)
+				if value.nil? then nil else sp.call value end
+			end
+		elsif sink.respond_to? :to_trans
+			# FIXME: this could be implemented more efficiently; proc doesn't
+			#        need to run in a separate Fiber
+			self.to_trans >= sink.to_trans
+		else
+			raise ArgumentError, "#{sink.inspect} is neither a sink nor a transformer"
+		end
+	end
+end

data/tests/test_coroutines.rb

@@ -0,0 +1,154 @@
+require 'coroutines'
+require 'test/unit'
+
+class TestCoroutines < Test::Unit::TestCase
+	def test_sink
+		assert_equal("abcdef", "abc" <= ("d".."f"))
+		assert_equal([1,2,3,4], [1] <= [2,3,4])
+	end
+
+	def test_source
+		assert_equal("abcdef", ("d".."f") >= "abc")
+		assert_equal([1,2,3,4], [2,3,4] >= [1])
+	end
+
+	def test_consumer
+		c = Consumer.new { |y| [ y.await, y.await ] }
+		c << :first << :second
+		assert_equal([:first, :second], c.close)
+	end
+
+	def counter(start)
+		result = start
+		loop { result += await }
+		"Final value: #{result}"
+	end
+	def test_consumer_method
+		c = consum_for :counter, 10
+		c << 10 << 1000 << 10000
+		assert_equal("Final value: 11020", c.close)
+	end
+
+	def test_transformer
+		rs = Transformer.new do |y|
+			result = 0
+			loop { result += y.await; y.yield result }
+		end
+		assert_equal([1, 3, 6], (1..3) >= rs >= [])
+
+		rs = Transformer.new do |y|
+			result = 0
+			loop { result += y.await; y.yield result }
+		end
+		assert_equal([1, 3, 6], [] <= rs <= (1..3))
+	end
+
+	def test_transformer_chaining
+		t1 = Transformer.new{|y| y.yield (y.await + "a")}
+		t2 = Transformer.new{|y| y.yield (y.await + "b")}
+		t = t1 >= t2
+		result = %w{x y z} >= t >= []
+		assert_equal(["xab"], result)
+	end
+
+	def test_associativity
+		s = (1..3)
+		def t1
+			Transformer.new{|y| loop { y.yield y.await.to_s } }
+		end
+		def t2
+			Transformer.new{|y| loop { y.yield (y.await + ",") } }
+		end
+
+		assert_equal("1,2,3,", s >= t1 >= t2 >= "")
+		assert_equal("1,2,3,", s >= t1 >= (t2 >= ""))
+		assert_equal("1,2,3,", s >= (t1 >= t2 >= ""))
+		assert_equal("1,2,3,", s >= (t1 >= t2) >= "")
+		assert_equal("1,2,3,", s >= ((t1 >= t2) >= ""))
+
+		assert_equal("1,2,3,", "" <= t2 <= t1 <= s)
+		assert_equal("1,2,3,", "" <= t2 <= (t1 <= s))
+		assert_equal("1,2,3,", "" <= (t2 <= t1 <= s))
+		assert_equal("1,2,3,", "" <= (t2 <= t1) <= s)
+		assert_equal("1,2,3,", "" <= ((t2 <= t1) <= s))
+	end
+
+	def running_sum(start)
+		result = start
+		loop { result += await; yield result }
+	end
+	def test_transformer_method
+		assert_equal([4, 6, 9], (1..3) >= trans_for(:running_sum, 3) >= [])
+		assert_equal([4, 6, 9], (1..3) >= (trans_for(:running_sum, 3) >= []))
+	end
+
+	def test_stop_iteration
+		consume_three = Consumer.new do |y|
+			[y.await, y.await, y.await]
+		end
+		assert_equal([1,2,3], (1..Float::INFINITY) >= consume_three)
+
+		limit_three = Transformer.new do |y|
+			3.times { y.yield y.await }
+		end
+		assert_equal([1,2,3], (1..Float::INFINITY) >= limit_three >= [])
+	end
+
+	def test_transformer_procs
+		s = (1..3)
+		t1_proc = proc{|x| x.to_s }
+		t2_proc = proc{|x| x + "," }
+		def t1
+			Transformer.new{|y| loop { y.yield y.await.to_s } }
+		end
+		def t2
+			Transformer.new{|y| loop { y.yield (y.await + ",") } }
+		end
+
+		assert_equal("1,2,3,", s >= t1_proc >= t2_proc >= "")
+		assert_equal("1,2,3,", s >= t1_proc >= (t2_proc >= ""))
+		assert_equal("1,2,3,", s >= (t1_proc >= t2_proc >= ""))
+		assert_equal("1,2,3,", s >= (t1_proc >= t2_proc) >= "")
+		assert_equal("1,2,3,", s >= ((t1_proc >= t2_proc) >= ""))
+
+		assert_equal("1,2,3,", "" <= t2_proc <= t1_proc <= s)
+		assert_equal("1,2,3,", "" <= t2_proc <= (t1_proc <= s))
+		assert_equal("1,2,3,", "" <= (t2_proc <= t1_proc <= s))
+		assert_equal("1,2,3,", "" <= (t2_proc <= t1_proc) <= s)
+		assert_equal("1,2,3,", "" <= ((t2_proc <= t1_proc) <= s))
+
+		assert_equal("1,2,3,", s >= t1_proc >= t2 >= "")
+		assert_equal("1,2,3,", s >= t1_proc >= (t2 >= ""))
+		assert_equal("1,2,3,", s >= (t1_proc >= t2 >= ""))
+		assert_equal("1,2,3,", s >= (t1_proc >= t2) >= "")
+		assert_equal("1,2,3,", s >= ((t1_proc >= t2) >= ""))
+
+		assert_equal("1,2,3,", "" <= t2 <= t1_proc <= s)
+		assert_equal("1,2,3,", "" <= t2 <= (t1_proc <= s))
+		assert_equal("1,2,3,", "" <= (t2 <= t1_proc <= s))
+		assert_equal("1,2,3,", "" <= (t2 <= t1_proc) <= s)
+		assert_equal("1,2,3,", "" <= ((t2 <= t1_proc) <= s))
+
+		assert_equal("1,2,3,", s >= t1 >= t2_proc >= "")
+		assert_equal("1,2,3,", s >= t1 >= (t2_proc >= ""))
+		assert_equal("1,2,3,", s >= (t1 >= t2_proc >= ""))
+		assert_equal("1,2,3,", s >= (t1 >= t2_proc) >= "")
+		assert_equal("1,2,3,", s >= ((t1 >= t2_proc) >= ""))
+
+		assert_equal("1,2,3,", "" <= t2_proc <= t1 <= s)
+		assert_equal("1,2,3,", "" <= t2_proc <= (t1 <= s))
+		assert_equal("1,2,3,", "" <= (t2_proc <= t1 <= s))
+		assert_equal("1,2,3,", "" <= (t2_proc <= t1) <= s)
+		assert_equal("1,2,3,", "" <= ((t2_proc <= t1) <= s))
+	end
+
+	def test_transformer_proc_filter
+		assert_equal("2468", (1..9) >= proc{|x| x.to_s if x.even? } >= "")
+		assert_equal("2468", (1..9) >= proc{|x| x if x.even? } >= proc{|x| x.to_s} >= "")
+		assert_equal("56789", (5..15) >= proc{|x| x.to_s } >= proc{|s| s if s.length == 1 } >= "")
+	end
+
+	def test_multiple_args
+		assert_equal([[1,2],[3,4]], [[1,2],[3,4]] >= proc{|a,b| [a,b]} >= [])
+	end
+end

metadata

@@ -0,0 +1,59 @@
+--- !ruby/object:Gem::Specification
+name: coroutines
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Knut Franke
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2014-10-11 00:00:00.000000000 Z
+dependencies: []
+description: |
+  A library for creating and composing producer/transformer/consumer coroutines.
+  Producers are already provided by Ruby's built-in Enumerator class; this
+  library provides Transformer and Consumer classes that work analogously. In
+  particular, they are also based on Fiber and not on threads (as in some other
+  producer/consumer libraries). Also provides a module Sink, which is analogous
+  to Enumerable, and Enumerable/Transformer/Sink composition using overloaded
+  <= and >= operators.
+email: knut.franke@gmx.de
+executables: []
+extensions: []
+extra_rdoc_files:
+- README.rdoc
+files:
+- README.rdoc
+- lib/coroutines.rb
+- lib/coroutines/base.rb
+- lib/coroutines/sink.rb
+- tests/test_coroutines.rb
+homepage: https://github.com/nome/coroutines
+licenses:
+- Ruby
+- BSD-2-Clause
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.2
+signing_key: 
+specification_version: 4
+summary: Library for producer/transformer/consumer coroutines
+test_files:
+- tests/test_coroutines.rb
+has_rdoc: false