stepladder 0.0.1

data/.gitignore

@@ -0,0 +1,3 @@
+./spikes/*
+.rvmrc
+Gemfile.lock

data/Gemfile

@@ -0,0 +1,5 @@
+source 'http://rubygems.org'
+
+gem 'cucumber'
+gem 'aruba'
+gem 'rspec'

data/README.md

@@ -0,0 +1,229 @@
+# The Stepladder Framework
+
+_"How many Ruby fibers does it take to screw in a lightbulb?"_
+
+## Quick Start
+
+### Workers have tasks...
+
+Initialize with a block of code:
+
+```ruby
+source_worker = Stepladder::Worker.new { "hulk" }
+
+source_worker.product #=> "hulk"
+```
+If you supply a worker with another worker as its supplier, then you
+can give it a task which accepts a value:
+
+```ruby
+relay_worker = Stepladder::Worker.new { |name| name.upcase }
+relay_worker.supplier = source_worker
+
+relay_worker.product #=> "HULK"
+```
+
+You can also initialize a worker by passing in a callable object
+as its task:
+
+```ruby
+capitalizer = Proc.new { |name| name.capitalize }
+relay_worker = Stepladder::Worker.new(task: capitalizer, supplier: source_worker)
+
+relay_worker.product #=> 'Hulk'
+```
+
+A worker also has an accessor for its @task:
+
+```ruby
+doofusizer = Proc.new { |name| name.gsub(/u/, 'oo') }
+relay_worker.task = doofusizer
+
+relay_worker.product #=> 'hoolk'
+```
+
+And finally, you can provide a task by directly overriding the
+worker's #task instance method:
+
+```ruby
+def relay_worker.task(name)
+  name.to_sym
+end
+
+relay_worker.product #=> :hulk
+```
+
+Even workers without a task have a task; all workers actually come
+with a default task which simply passes on the received value unchanged:
+
+```ruby
+useless_worker = Stepladder::Worker.new(supplier: source_worker)
+
+useless_worker.product #=> 'hulk'
+```
+
+This turns out to be helpful in implementing filter workers, which are up next.
+
+### Workers can have filters...
+
+Given a source worker which provides integers 1-3:
+
+```ruby
+source = Stepladder::Worker.new do
+  (1..3).each { |number| handoff number }
+end
+```
+
+...we can define a subscribing worker with a filter:
+
+```ruby
+odd_number_filter = Proc.new { |number| number % 2 > 0 }
+filter_worker = Stepladder::Worker.new filter: odd_number_filter
+
+filter_worker.product #=> 1
+filter_worker.product #=> 3
+filter_worker.product #=> nil
+```
+
+### The pipeline DSL
+
+You can stitch your workers together using the vertical pipe ("|") like so:
+
+```ruby
+pipeline = source_worker | filter_worker | relay_worker | another worker
+```
+
+...and then just call on that pipeline (it's actually the last worker in the
+chain):
+
+```ruby
+while next_value = pipeline.product do
+  do_something_with next_value
+  # etc.
+end
+```
+
+## Origins of Stepladder
+
+Stepladder grew out of experimentation with Ruby fibers, after readings
+[Dave Thomas' demo of Ruby fibers](http://pragdave.blogs.pragprog.com/pragdave/2007/12/pipelines-using.html), wherein he created a
+pipeline of fiber processes, emulating the style and syntax of the
+\*nix command line.  I noticed that, courtesy of fibers' extremely
+low surface area, fiber-to-fiber collaborators could operate with
+extremely low coupling.  That was the original motivation for creating
+the framework.
+
+After playing around with the new framework a bit, I began to notice
+other interesting characteristics of this paradigm.
+
+### Escalator vs Elevator
+
+Suppose we are performing several operations on the members of a largish
+collection. If we daisy-chain enumerable operators together (which is so
+easy and fun with Ruby) we will notice that if something goes awry with
+item number two during operation number seven, we nonetheless had to wait
+through a complete run of all items through operations 1 - 6 before we
+receive the bad news early in operation seven.  Imagine if the first six
+operations take a long time to each complete?  Furthermore, what if the
+operations on all incomplete items must be reversed in some way (e.g.
+cleaned up or rolled back)?  It would be far less messy and far more
+expedient if each item could be processed though all operations before
+the next one is begun.
+
+This is the design paradigm which stepladder makes easy.  Although all
+the workers in your assembly line can be coded in the same context (which
+is one of the big selling points of the daisy-chaining of enumerable
+methods,incidentally), you also get the advantage of passing each item
+though the entire op-chain before starting the next.
+
+### Think Locally, Act Globally
+
+Because stepladder workers use fibers as their basic API, the are almost
+unaware of the outside world.  And they can pretty much be written as such.
+At the heart of each Stepladder worker is a task which you provide, which
+is a callable ruby object (such as a Proc or a lambda).
+
+The scope of the work is whatever scope existed in the task when you
+initially created the worker.  And if you want a worker to maintain its
+own internal state, you can simply include a loop within the worker's
+task, and use the `#handoff` method to pass along the worker's product at
+the appropriate point in the loop.
+
+For example:
+
+```ruby
+realestate_maker = Stepladder::Worker.new do
+  oceans = %w[Atlantic Pacific Indiana Arctic]
+  previous_ocean = "Atlantic"
+  while current_ocean = oceans.sample
+    drain current_ocean  #=> let's posit that this is a long-running async process!
+    handoff previous_ocean
+    previous_ocean = current_ocean
+  end
+```
+
+Anything scoped to the outside of that loop but inside the worker's task
+will essentially become that worker's initial state.  This means we often
+don't need to bother with instance variables, accessors, and other
+features of classes which deal with maintaining instances' state.
+
+### The Wormhole Effect
+
+Despite the fact that the work is done in separate threads, the _scope_
+of the work is the scope of the coordinating body of code.  This means
+that even the remaining coupling in such systems --which is primarily
+just _common objects_ coupling-- this little remaining coupling is
+mitigated by the possibility of having coupled code _live together_.
+I call this feature the _folded collaborators_ effect.
+
+Consider the following -code- vaporware:
+
+```ruby
+ME = "joelhelbling"
+
+module Stepladder
+  tweet_getter = Worker.new do
+    twitter_api.fetch_my_tweets
+  end
+
+  about_me_filter      = Proc.new { |tweet| tweet.referenced.include? ME }
+  just_about_me_getter = Worker.new filter: about_me_filter
+
+  tweet_formatter = Worker.new do |tweet|
+    apply_format_to tweet
+  end
+
+  formatted_tweets = tweet_getter | just_about_me_getter | tweet_formatter
+end
+```
+
+None of these tasks have hard coupling with each other.  If we were to
+insert another worker between the filter and the formatter, neither of those
+workers would need changes in their code, assuming the inserted worker plays
+nicely with the objects they're all passing along.
+
+Which brings me to the point: these workers to have a dependency upon the
+objects they're handing off and receiving.  But we have the capability to
+coordinate those workers in a centralized location (such as in this code
+example).
+
+## Ok, but why is it called "Stepladder"?
+
+This framework's name was inspired by a conversation with Tim Wingfield
+in which we joked about the profusion of new frameworks in the Ruby
+community.  We quickly began riffing on a fictional framework called
+"Stepladder" which all the cool kids, we asserted, were (or would soon
+be) using.
+
+I have waited a long time to make that farce a reality, but hey, I take
+joke frameworks very seriously. ;)
+([Really?](http://github.com/joelhelbling/really))
+
+## Roadmap
+
+- add a nicer top-layer to the DSL --no reason we should have to do
+  all that `Worker.new` stuff
+- make this into a gem
+- add support for a collector worker which collects values from its
+  supplier, and then passes them downstream in batches (defined by
+  its task block, of course).

data/lib/stepladder/worker.rb

@@ -0,0 +1,82 @@
+module Stepladder
+  class Worker
+    attr_accessor :supplier
+
+    def initialize(p={}, &block)
+      @supplier = p[:supplier]
+      @filter   = p[:filter] || default_filter
+      @task     = block || p[:task]
+    end
+
+    def product
+      if ready_to_work?
+        work.resume
+      end
+    end
+
+    def ready_to_work?
+      @task ||= default_task
+      if (task_accepts_a_value? && supplier.nil?)
+        raise "This worker's task expects to receive a value from a supplier, but has no supplier."
+      end
+      true
+    end
+
+    def |(subscribing_worker)
+      subscribing_worker.supplier = self
+      subscribing_worker
+    end
+
+    private
+
+    def work
+      @my_little_machine ||= Fiber.new do
+        loop do
+          value = supplier && supplier.product
+          if value.nil? || passes_filter?(value)
+            handoff @task.call(value)
+          end
+        end
+      end
+    end
+
+    def default_task
+      if task_method_exists?
+        if task_method_accepts_a_value?
+          Proc.new { |value| self.task value }
+        else
+          Proc.new { self.task }
+        end
+      else # no task method, so assuming we have supplier...
+        Proc.new { |value| value }
+      end
+    end
+
+    def task_accepts_a_value?
+      @task.arity > 0
+    end
+
+    def task_method_exists?
+      self.methods.include? :task
+    end
+
+    def task_method_accepts_a_value?
+      self.method(:task).arity > 0
+    end
+
+    def default_filter
+      Proc.new do |value|
+        true
+      end
+    end
+
+    def passes_filter?(value)
+      @filter.call value
+    end
+
+  end
+end
+
+def handoff(product)
+  Fiber.yield product
+end

data/spec/lib/stepladder/worker_spec.rb

@@ -0,0 +1,198 @@
+require 'spec_helper'
+require 'stepladder/worker'
+
+module Stepladder
+  describe Worker do
+    it { should respond_to(:product, :supplier, :supplier=) }
+
+    describe "can accept a task" do
+      let(:result) { double }
+      before do
+        result.stub(:copasetic?).and_return(true)
+      end
+
+      context "via the constructor" do
+
+        context "as a block passed to ::new" do
+          subject do
+            Worker.new do
+              result
+            end
+          end
+
+          its(:product) { should be_copasetic }
+        end
+
+        context "as {:task => <proc/lambda>} passed to ::new" do
+          let(:callable_task) { Proc.new { result } }
+
+          subject do
+            Worker.new task: callable_task
+          end
+
+          its(:product) { should be_copasetic }
+        end
+      end
+
+      # Note that tasks defined via instance methods will
+      # only have access to the scope of the worker.  If
+      # you want a worker to have access to a scope outside
+      # the worker, use a Proc or Lambda via the constructor
+      context "via an instance method" do
+        subject { Worker.new }
+
+        context "which accepts an argument" do
+          let(:supplier) { double }
+          before do
+            supplier.stub(:product).and_return(result)
+            subject.supplier = supplier
+            def subject.task(value)
+              handoff value
+            end
+          end
+          its(:product) { should be_copasetic }
+        end
+
+        context "or even one which accepts no arguments" do
+          before do
+            def subject.task
+              :copasetic
+            end
+          end
+          its(:product) { should be :copasetic }
+        end
+      end
+
+      context "However, when a worker's task accepts an argument," do
+        context "but the worker has no supplier," do
+          subject { Worker.new { |value| value.do_whatnot } }
+          specify "#product throws an exception" do
+            expect { subject.product }.to raise_error(/has no supplier/)
+          end
+        end
+      end
+
+    end
+
+    describe "= WORKER TYPES =" do
+
+      let(:source_worker) do
+        Worker.new do
+          numbers = (1..3).to_a
+          until numbers.empty?
+            handoff numbers.shift
+          end
+        end
+      end
+
+      let(:relay_worker) do
+        Worker.new do |number|
+          number && number * 3
+        end
+      end
+
+      let(:filter) do
+        Proc.new do |number|
+          number % 2 > 0
+        end
+      end
+
+      let(:filter_worker) do
+        Worker.new filter: filter
+      end
+
+      let(:collector_worker) { Worker.new }
+
+      describe "The Source Worker" do
+        subject(:the_self_starter) { source_worker }
+
+        it "generates products without a supplier." do
+          the_self_starter.product.should == 1
+          the_self_starter.product.should == 2
+          the_self_starter.product.should == 3
+          the_self_starter.product.should be_nil
+        end
+      end
+
+      describe "The Relay Worker" do
+        before do
+          relay_worker.supplier = source_worker
+        end
+
+        subject(:triplizer) { relay_worker }
+
+        it "operates on values received from its supplier." do
+          triplizer.product.should == 3
+          triplizer.product.should == 6
+          triplizer.product.should == 9
+          triplizer.product.should be_nil
+        end
+      end
+
+      describe "The Filter" do
+        before do
+          filter_worker.supplier = source_worker
+        end
+
+        subject(:oddball) { filter_worker }
+
+        it "passes through only select values." do
+          oddball.product.should == 1
+          oddball.product.should == 3
+          oddball.product.should be_nil
+        end
+      end
+
+      describe "The Collector" do
+        before do
+          def collector_worker.task(value)
+            if value
+              @collection = [value]
+              while @collection.size < 3
+                @collection << supplier.product
+              end
+              @collection
+            end
+          end
+
+          collector_worker.supplier = source_worker
+        end
+
+        subject(:collector) { collector_worker }
+
+        it "collects values in threes" do
+          collector.product.should == [1,2,3]
+          collector.product.should be_nil
+        end
+      end
+
+      describe "Also, there's a pipeline dsl:" do
+        let(:subscribing_worker) { relay_worker }
+        let(:pipeline) { source_worker | subscribing_worker }
+
+        subject { pipeline }
+
+        it "lets you daisy-chain workers using \"|\"" do
+          subject.inspect
+          subscribing_worker.supplier.should == source_worker
+        end
+      end
+
+    end
+
+    describe "#product" do
+      before do
+        supplier.stub(:product).and_return(result)
+        subject.supplier = supplier
+      end
+      let(:result)   { :foo }
+      let(:supplier) { double }
+
+      it "resumes a fiber" do
+        Fiber.any_instance.should_receive(:resume).and_return(result)
+        subject.product.should == result
+      end
+    end
+
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1 @@
+$LOAD_PATH.unshift(File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib')))

metadata

@@ -0,0 +1,133 @@
+--- !ruby/object:Gem::Specification
+name: stepladder
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Joel Helbling
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-01-15 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: cucumber
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec-core
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: aruba
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: ''
+email:
+- joel@joelhelbling.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- Gemfile
+- README.md
+- lib/stepladder/worker.rb
+- spec/lib/stepladder/worker_spec.rb
+- spec/spec_helper.rb
+homepage: http://github.com/joelhelbling/stepladder
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: stepladder
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: ''
+test_files:
+- spec/lib/stepladder/worker_spec.rb
+- spec/spec_helper.rb