stepladder 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,3 @@
1
+ ./spikes/*
2
+ .rvmrc
3
+ Gemfile.lock
data/Gemfile ADDED
@@ -0,0 +1,5 @@
1
+ source 'http://rubygems.org'
2
+
3
+ gem 'cucumber'
4
+ gem 'aruba'
5
+ gem 'rspec'
data/README.md ADDED
@@ -0,0 +1,229 @@
1
+ # The Stepladder Framework
2
+
3
+ _"How many Ruby fibers does it take to screw in a lightbulb?"_
4
+
5
+ ## Quick Start
6
+
7
+ ### Workers have tasks...
8
+
9
+ Initialize with a block of code:
10
+
11
+ ```ruby
12
+ source_worker = Stepladder::Worker.new { "hulk" }
13
+
14
+ source_worker.product #=> "hulk"
15
+ ```
16
+ If you supply a worker with another worker as its supplier, then you
17
+ can give it a task which accepts a value:
18
+
19
+ ```ruby
20
+ relay_worker = Stepladder::Worker.new { |name| name.upcase }
21
+ relay_worker.supplier = source_worker
22
+
23
+ relay_worker.product #=> "HULK"
24
+ ```
25
+
26
+ You can also initialize a worker by passing in a callable object
27
+ as its task:
28
+
29
+ ```ruby
30
+ capitalizer = Proc.new { |name| name.capitalize }
31
+ relay_worker = Stepladder::Worker.new(task: capitalizer, supplier: source_worker)
32
+
33
+ relay_worker.product #=> 'Hulk'
34
+ ```
35
+
36
+ A worker also has an accessor for its @task:
37
+
38
+ ```ruby
39
+ doofusizer = Proc.new { |name| name.gsub(/u/, 'oo') }
40
+ relay_worker.task = doofusizer
41
+
42
+ relay_worker.product #=> 'hoolk'
43
+ ```
44
+
45
+ And finally, you can provide a task by directly overriding the
46
+ worker's #task instance method:
47
+
48
+ ```ruby
49
+ def relay_worker.task(name)
50
+ name.to_sym
51
+ end
52
+
53
+ relay_worker.product #=> :hulk
54
+ ```
55
+
56
+ Even workers without a task have a task; all workers actually come
57
+ with a default task which simply passes on the received value unchanged:
58
+
59
+ ```ruby
60
+ useless_worker = Stepladder::Worker.new(supplier: source_worker)
61
+
62
+ useless_worker.product #=> 'hulk'
63
+ ```
64
+
65
+ This turns out to be helpful in implementing filter workers, which are up next.
66
+
67
+ ### Workers can have filters...
68
+
69
+ Given a source worker which provides integers 1-3:
70
+
71
+ ```ruby
72
+ source = Stepladder::Worker.new do
73
+ (1..3).each { |number| handoff number }
74
+ end
75
+ ```
76
+
77
+ ...we can define a subscribing worker with a filter:
78
+
79
+ ```ruby
80
+ odd_number_filter = Proc.new { |number| number % 2 > 0 }
81
+ filter_worker = Stepladder::Worker.new filter: odd_number_filter
82
+
83
+ filter_worker.product #=> 1
84
+ filter_worker.product #=> 3
85
+ filter_worker.product #=> nil
86
+ ```
87
+
88
+ ### The pipeline DSL
89
+
90
+ You can stitch your workers together using the vertical pipe ("|") like so:
91
+
92
+ ```ruby
93
+ pipeline = source_worker | filter_worker | relay_worker | another worker
94
+ ```
95
+
96
+ ...and then just call on that pipeline (it's actually the last worker in the
97
+ chain):
98
+
99
+ ```ruby
100
+ while next_value = pipeline.product do
101
+ do_something_with next_value
102
+ # etc.
103
+ end
104
+ ```
105
+
106
+ ## Origins of Stepladder
107
+
108
+ Stepladder grew out of experimentation with Ruby fibers, after readings
109
+ [Dave Thomas' demo of Ruby fibers](http://pragdave.blogs.pragprog.com/pragdave/2007/12/pipelines-using.html), wherein he created a
110
+ pipeline of fiber processes, emulating the style and syntax of the
111
+ \*nix command line. I noticed that, courtesy of fibers' extremely
112
+ low surface area, fiber-to-fiber collaborators could operate with
113
+ extremely low coupling. That was the original motivation for creating
114
+ the framework.
115
+
116
+ After playing around with the new framework a bit, I began to notice
117
+ other interesting characteristics of this paradigm.
118
+
119
+ ### Escalator vs Elevator
120
+
121
+ Suppose we are performing several operations on the members of a largish
122
+ collection. If we daisy-chain enumerable operators together (which is so
123
+ easy and fun with Ruby) we will notice that if something goes awry with
124
+ item number two during operation number seven, we nonetheless had to wait
125
+ through a complete run of all items through operations 1 - 6 before we
126
+ receive the bad news early in operation seven. Imagine if the first six
127
+ operations take a long time to each complete? Furthermore, what if the
128
+ operations on all incomplete items must be reversed in some way (e.g.
129
+ cleaned up or rolled back)? It would be far less messy and far more
130
+ expedient if each item could be processed though all operations before
131
+ the next one is begun.
132
+
133
+ This is the design paradigm which stepladder makes easy. Although all
134
+ the workers in your assembly line can be coded in the same context (which
135
+ is one of the big selling points of the daisy-chaining of enumerable
136
+ methods,incidentally), you also get the advantage of passing each item
137
+ though the entire op-chain before starting the next.
138
+
139
+ ### Think Locally, Act Globally
140
+
141
+ Because stepladder workers use fibers as their basic API, the are almost
142
+ unaware of the outside world. And they can pretty much be written as such.
143
+ At the heart of each Stepladder worker is a task which you provide, which
144
+ is a callable ruby object (such as a Proc or a lambda).
145
+
146
+ The scope of the work is whatever scope existed in the task when you
147
+ initially created the worker. And if you want a worker to maintain its
148
+ own internal state, you can simply include a loop within the worker's
149
+ task, and use the `#handoff` method to pass along the worker's product at
150
+ the appropriate point in the loop.
151
+
152
+ For example:
153
+
154
+ ```ruby
155
+ realestate_maker = Stepladder::Worker.new do
156
+ oceans = %w[Atlantic Pacific Indiana Arctic]
157
+ previous_ocean = "Atlantic"
158
+ while current_ocean = oceans.sample
159
+ drain current_ocean #=> let's posit that this is a long-running async process!
160
+ handoff previous_ocean
161
+ previous_ocean = current_ocean
162
+ end
163
+ ```
164
+
165
+ Anything scoped to the outside of that loop but inside the worker's task
166
+ will essentially become that worker's initial state. This means we often
167
+ don't need to bother with instance variables, accessors, and other
168
+ features of classes which deal with maintaining instances' state.
169
+
170
+ ### The Wormhole Effect
171
+
172
+ Despite the fact that the work is done in separate threads, the _scope_
173
+ of the work is the scope of the coordinating body of code. This means
174
+ that even the remaining coupling in such systems --which is primarily
175
+ just _common objects_ coupling-- this little remaining coupling is
176
+ mitigated by the possibility of having coupled code _live together_.
177
+ I call this feature the _folded collaborators_ effect.
178
+
179
+ Consider the following -code- vaporware:
180
+
181
+ ```ruby
182
+ ME = "joelhelbling"
183
+
184
+ module Stepladder
185
+ tweet_getter = Worker.new do
186
+ twitter_api.fetch_my_tweets
187
+ end
188
+
189
+ about_me_filter = Proc.new { |tweet| tweet.referenced.include? ME }
190
+ just_about_me_getter = Worker.new filter: about_me_filter
191
+
192
+ tweet_formatter = Worker.new do |tweet|
193
+ apply_format_to tweet
194
+ end
195
+
196
+ formatted_tweets = tweet_getter | just_about_me_getter | tweet_formatter
197
+ end
198
+ ```
199
+
200
+ None of these tasks have hard coupling with each other. If we were to
201
+ insert another worker between the filter and the formatter, neither of those
202
+ workers would need changes in their code, assuming the inserted worker plays
203
+ nicely with the objects they're all passing along.
204
+
205
+ Which brings me to the point: these workers to have a dependency upon the
206
+ objects they're handing off and receiving. But we have the capability to
207
+ coordinate those workers in a centralized location (such as in this code
208
+ example).
209
+
210
+ ## Ok, but why is it called "Stepladder"?
211
+
212
+ This framework's name was inspired by a conversation with Tim Wingfield
213
+ in which we joked about the profusion of new frameworks in the Ruby
214
+ community. We quickly began riffing on a fictional framework called
215
+ "Stepladder" which all the cool kids, we asserted, were (or would soon
216
+ be) using.
217
+
218
+ I have waited a long time to make that farce a reality, but hey, I take
219
+ joke frameworks very seriously. ;)
220
+ ([Really?](http://github.com/joelhelbling/really))
221
+
222
+ ## Roadmap
223
+
224
+ - add a nicer top-layer to the DSL --no reason we should have to do
225
+ all that `Worker.new` stuff
226
+ - make this into a gem
227
+ - add support for a collector worker which collects values from its
228
+ supplier, and then passes them downstream in batches (defined by
229
+ its task block, of course).
@@ -0,0 +1,82 @@
1
+ module Stepladder
2
+ class Worker
3
+ attr_accessor :supplier
4
+
5
+ def initialize(p={}, &block)
6
+ @supplier = p[:supplier]
7
+ @filter = p[:filter] || default_filter
8
+ @task = block || p[:task]
9
+ end
10
+
11
+ def product
12
+ if ready_to_work?
13
+ work.resume
14
+ end
15
+ end
16
+
17
+ def ready_to_work?
18
+ @task ||= default_task
19
+ if (task_accepts_a_value? && supplier.nil?)
20
+ raise "This worker's task expects to receive a value from a supplier, but has no supplier."
21
+ end
22
+ true
23
+ end
24
+
25
+ def |(subscribing_worker)
26
+ subscribing_worker.supplier = self
27
+ subscribing_worker
28
+ end
29
+
30
+ private
31
+
32
+ def work
33
+ @my_little_machine ||= Fiber.new do
34
+ loop do
35
+ value = supplier && supplier.product
36
+ if value.nil? || passes_filter?(value)
37
+ handoff @task.call(value)
38
+ end
39
+ end
40
+ end
41
+ end
42
+
43
+ def default_task
44
+ if task_method_exists?
45
+ if task_method_accepts_a_value?
46
+ Proc.new { |value| self.task value }
47
+ else
48
+ Proc.new { self.task }
49
+ end
50
+ else # no task method, so assuming we have supplier...
51
+ Proc.new { |value| value }
52
+ end
53
+ end
54
+
55
+ def task_accepts_a_value?
56
+ @task.arity > 0
57
+ end
58
+
59
+ def task_method_exists?
60
+ self.methods.include? :task
61
+ end
62
+
63
+ def task_method_accepts_a_value?
64
+ self.method(:task).arity > 0
65
+ end
66
+
67
+ def default_filter
68
+ Proc.new do |value|
69
+ true
70
+ end
71
+ end
72
+
73
+ def passes_filter?(value)
74
+ @filter.call value
75
+ end
76
+
77
+ end
78
+ end
79
+
80
+ def handoff(product)
81
+ Fiber.yield product
82
+ end
@@ -0,0 +1,198 @@
1
+ require 'spec_helper'
2
+ require 'stepladder/worker'
3
+
4
+ module Stepladder
5
+ describe Worker do
6
+ it { should respond_to(:product, :supplier, :supplier=) }
7
+
8
+ describe "can accept a task" do
9
+ let(:result) { double }
10
+ before do
11
+ result.stub(:copasetic?).and_return(true)
12
+ end
13
+
14
+ context "via the constructor" do
15
+
16
+ context "as a block passed to ::new" do
17
+ subject do
18
+ Worker.new do
19
+ result
20
+ end
21
+ end
22
+
23
+ its(:product) { should be_copasetic }
24
+ end
25
+
26
+ context "as {:task => <proc/lambda>} passed to ::new" do
27
+ let(:callable_task) { Proc.new { result } }
28
+
29
+ subject do
30
+ Worker.new task: callable_task
31
+ end
32
+
33
+ its(:product) { should be_copasetic }
34
+ end
35
+ end
36
+
37
+ # Note that tasks defined via instance methods will
38
+ # only have access to the scope of the worker. If
39
+ # you want a worker to have access to a scope outside
40
+ # the worker, use a Proc or Lambda via the constructor
41
+ context "via an instance method" do
42
+ subject { Worker.new }
43
+
44
+ context "which accepts an argument" do
45
+ let(:supplier) { double }
46
+ before do
47
+ supplier.stub(:product).and_return(result)
48
+ subject.supplier = supplier
49
+ def subject.task(value)
50
+ handoff value
51
+ end
52
+ end
53
+ its(:product) { should be_copasetic }
54
+ end
55
+
56
+ context "or even one which accepts no arguments" do
57
+ before do
58
+ def subject.task
59
+ :copasetic
60
+ end
61
+ end
62
+ its(:product) { should be :copasetic }
63
+ end
64
+ end
65
+
66
+ context "However, when a worker's task accepts an argument," do
67
+ context "but the worker has no supplier," do
68
+ subject { Worker.new { |value| value.do_whatnot } }
69
+ specify "#product throws an exception" do
70
+ expect { subject.product }.to raise_error(/has no supplier/)
71
+ end
72
+ end
73
+ end
74
+
75
+ end
76
+
77
+ describe "= WORKER TYPES =" do
78
+
79
+ let(:source_worker) do
80
+ Worker.new do
81
+ numbers = (1..3).to_a
82
+ until numbers.empty?
83
+ handoff numbers.shift
84
+ end
85
+ end
86
+ end
87
+
88
+ let(:relay_worker) do
89
+ Worker.new do |number|
90
+ number && number * 3
91
+ end
92
+ end
93
+
94
+ let(:filter) do
95
+ Proc.new do |number|
96
+ number % 2 > 0
97
+ end
98
+ end
99
+
100
+ let(:filter_worker) do
101
+ Worker.new filter: filter
102
+ end
103
+
104
+ let(:collector_worker) { Worker.new }
105
+
106
+ describe "The Source Worker" do
107
+ subject(:the_self_starter) { source_worker }
108
+
109
+ it "generates products without a supplier." do
110
+ the_self_starter.product.should == 1
111
+ the_self_starter.product.should == 2
112
+ the_self_starter.product.should == 3
113
+ the_self_starter.product.should be_nil
114
+ end
115
+ end
116
+
117
+ describe "The Relay Worker" do
118
+ before do
119
+ relay_worker.supplier = source_worker
120
+ end
121
+
122
+ subject(:triplizer) { relay_worker }
123
+
124
+ it "operates on values received from its supplier." do
125
+ triplizer.product.should == 3
126
+ triplizer.product.should == 6
127
+ triplizer.product.should == 9
128
+ triplizer.product.should be_nil
129
+ end
130
+ end
131
+
132
+ describe "The Filter" do
133
+ before do
134
+ filter_worker.supplier = source_worker
135
+ end
136
+
137
+ subject(:oddball) { filter_worker }
138
+
139
+ it "passes through only select values." do
140
+ oddball.product.should == 1
141
+ oddball.product.should == 3
142
+ oddball.product.should be_nil
143
+ end
144
+ end
145
+
146
+ describe "The Collector" do
147
+ before do
148
+ def collector_worker.task(value)
149
+ if value
150
+ @collection = [value]
151
+ while @collection.size < 3
152
+ @collection << supplier.product
153
+ end
154
+ @collection
155
+ end
156
+ end
157
+
158
+ collector_worker.supplier = source_worker
159
+ end
160
+
161
+ subject(:collector) { collector_worker }
162
+
163
+ it "collects values in threes" do
164
+ collector.product.should == [1,2,3]
165
+ collector.product.should be_nil
166
+ end
167
+ end
168
+
169
+ describe "Also, there's a pipeline dsl:" do
170
+ let(:subscribing_worker) { relay_worker }
171
+ let(:pipeline) { source_worker | subscribing_worker }
172
+
173
+ subject { pipeline }
174
+
175
+ it "lets you daisy-chain workers using \"|\"" do
176
+ subject.inspect
177
+ subscribing_worker.supplier.should == source_worker
178
+ end
179
+ end
180
+
181
+ end
182
+
183
+ describe "#product" do
184
+ before do
185
+ supplier.stub(:product).and_return(result)
186
+ subject.supplier = supplier
187
+ end
188
+ let(:result) { :foo }
189
+ let(:supplier) { double }
190
+
191
+ it "resumes a fiber" do
192
+ Fiber.any_instance.should_receive(:resume).and_return(result)
193
+ subject.product.should == result
194
+ end
195
+ end
196
+
197
+ end
198
+ end
@@ -0,0 +1 @@
1
+ $LOAD_PATH.unshift(File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib')))
metadata ADDED
@@ -0,0 +1,133 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: stepladder
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Joel Helbling
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-01-15 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: cucumber
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ! '>='
28
+ - !ruby/object:Gem::Version
29
+ version: '0'
30
+ - !ruby/object:Gem::Dependency
31
+ name: rake
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ! '>='
36
+ - !ruby/object:Gem::Version
37
+ version: '0'
38
+ type: :development
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ! '>='
44
+ - !ruby/object:Gem::Version
45
+ version: '0'
46
+ - !ruby/object:Gem::Dependency
47
+ name: rspec
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ! '>='
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :development
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ - !ruby/object:Gem::Dependency
63
+ name: rspec-core
64
+ requirement: !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ! '>='
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ type: :development
71
+ prerelease: false
72
+ version_requirements: !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - ! '>='
76
+ - !ruby/object:Gem::Version
77
+ version: '0'
78
+ - !ruby/object:Gem::Dependency
79
+ name: aruba
80
+ requirement: !ruby/object:Gem::Requirement
81
+ none: false
82
+ requirements:
83
+ - - ! '>='
84
+ - !ruby/object:Gem::Version
85
+ version: '0'
86
+ type: :development
87
+ prerelease: false
88
+ version_requirements: !ruby/object:Gem::Requirement
89
+ none: false
90
+ requirements:
91
+ - - ! '>='
92
+ - !ruby/object:Gem::Version
93
+ version: '0'
94
+ description: ''
95
+ email:
96
+ - joel@joelhelbling.com
97
+ executables: []
98
+ extensions: []
99
+ extra_rdoc_files: []
100
+ files:
101
+ - .gitignore
102
+ - Gemfile
103
+ - README.md
104
+ - lib/stepladder/worker.rb
105
+ - spec/lib/stepladder/worker_spec.rb
106
+ - spec/spec_helper.rb
107
+ homepage: http://github.com/joelhelbling/stepladder
108
+ licenses: []
109
+ post_install_message:
110
+ rdoc_options: []
111
+ require_paths:
112
+ - lib
113
+ required_ruby_version: !ruby/object:Gem::Requirement
114
+ none: false
115
+ requirements:
116
+ - - ! '>='
117
+ - !ruby/object:Gem::Version
118
+ version: '0'
119
+ required_rubygems_version: !ruby/object:Gem::Requirement
120
+ none: false
121
+ requirements:
122
+ - - ! '>='
123
+ - !ruby/object:Gem::Version
124
+ version: '0'
125
+ requirements: []
126
+ rubyforge_project: stepladder
127
+ rubygems_version: 1.8.24
128
+ signing_key:
129
+ specification_version: 3
130
+ summary: ''
131
+ test_files:
132
+ - spec/lib/stepladder/worker_spec.rb
133
+ - spec/spec_helper.rb