tap 0.17.1 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/History CHANGED
@@ -1,3 +1,25 @@
1
+ == 0.18.0 / 2009-06-17
2
+
3
+ Several updates to simplify Tap. Most significantly,
4
+ dependencies have been removed because their general
5
+ utility is quite doubtful, and it it easy to implement
6
+ in situations that require them (ex rap).
7
+
8
+ * bug fix in console (app now refers to Tap::App.instance)
9
+ * bug fix in load for recurrently loading from a file
10
+ * fixed a mistake in run help
11
+ * made load only load once by default, stream loading must
12
+ be enabled in subclasses
13
+ * dump now returns the input, not the io
14
+ * removed dependencies from tap
15
+ * removed intern method on Task (now use app.task)
16
+ * removed Support module (ie refactored Templater to
17
+ Tap::Templater)
18
+ * removed Task.help method
19
+ * added Middleware baseclass
20
+ * Task.parse now yields opts to block for modifications
21
+ * renamed 'tap/constants' file as 'tap/version'
22
+
1
23
  == 0.17.1 / 2009-06-06
2
24
 
3
25
  * documentation and interface updates
data/README CHANGED
@@ -6,9 +6,9 @@ A configurable, distributable workflow framework.
6
6
 
7
7
  == Description
8
8
 
9
- Tap allows the construction of imperative and dependency-based workflows that
10
- may be defined, configured, and run from the command line. The tasks and joins
11
- composing a workflow are easy to test, subclass, and distribute as gems.
9
+ Tap allows the construction of workflows that may be defined, configured, and
10
+ run from the command line. The tasks and joins composing a workflow are easy
11
+ to test, subclass, and distribute as gems.
12
12
 
13
13
  Tap is the core of the
14
14
  {Tap-Suite}[http://tap.rubyforge.org/tap-suite/index.html]
@@ -42,7 +42,7 @@ Tasks are defined as subclasses of Tap::Task.
42
42
  end
43
43
  end
44
44
 
45
- Tap automatically discovers and pulls documentation out to generate manifests:
45
+ Tap automatically discovers tasks.
46
46
 
47
47
  % tap run -T
48
48
  sample:
@@ -51,7 +51,7 @@ Tap automatically discovers and pulls documentation out to generate manifests:
51
51
  dump # the default dump task
52
52
  load # the default load task
53
53
 
54
- And help:
54
+ And generates documentation.
55
55
 
56
56
  % tap run -- goodnight --help
57
57
  Goodnight -- your basic goodnight moon task
@@ -68,44 +68,45 @@ And help:
68
68
  --name NAME Specifies the task name
69
69
  --config FILE Specifies a config file
70
70
 
71
- Tasks are available immediately for use in workflows. The process method takes
72
- inputs from the command line and passes its result to other tasks.
71
+ Tasks are immediately available for use in workflows.
73
72
 
74
73
  % tap run -- goodnight moon --: dump
75
74
  goodnight moon
76
75
 
77
- Tasks may be configured as if they were individual executables:
76
+ Tasks may be configured as if they were individual executables.
78
77
 
79
78
  % tap run -- goodnight world --message hello --: dump
80
79
  hello world
81
80
 
82
81
  === Workflow Syntax
83
82
 
84
- Workflows are specified on the command line using argument vectors separated by option breaks. The argument vectors define the tasks and modifications to the breaks (--: or --[i,j][k]) specify joins.
83
+ Workflows are specified on the command line using argument vectors separated
84
+ by option breaks. The vectors define the tasks and modifications to the breaks
85
+ specify joins.
85
86
 
86
- A simple sequence:
87
+ A simple sequence.
87
88
 
88
89
  % tap run -- load 'goodnight moon' --: dump
89
90
  goodnight moon
90
91
 
91
- A more formal way of specifying a sequence:
92
+ A more formal way of specifying a sequence.
92
93
 
93
94
  % tap run -- load 'goodnight moon' -- dump --[0][1]
94
95
  goodnight moon
95
96
 
96
- A fork:
97
+ A fork.
97
98
 
98
99
  % tap run -- load 'goodnight moon' -- dump -- dump --[0][1,2]
99
100
  goodnight moon
100
101
  goodnight moon
101
102
 
102
- A merge (note that dump receives the inputs in serial):
103
+ A merge (note that dump receives the inputs in serial).
103
104
 
104
105
  % tap run -- load goodnight -- load moon -- dump --[0,1][2]
105
106
  goodnight
106
107
  moon
107
108
 
108
- A synchronized merge where the results of each input are collected into an array before being passed to dump. The printout is: ['goodnight', 'moon'].to_s
109
+ A synchronized merge (the printout is ['goodnight', 'moon'].to_s).
109
110
 
110
111
  % tap run -- load goodnight -- load moon -- dump --[0,1][2].sync
111
112
  goodnightmoon
data/cmd/console.rb CHANGED
@@ -26,7 +26,7 @@ end.parse!(ARGV)
26
26
  require "irb"
27
27
 
28
28
  def app
29
- @app ||= Tap::App.new
29
+ @app ||= Tap::App.instance
30
30
  end
31
31
 
32
32
  def env
data/cmd/manifest.rb CHANGED
@@ -82,14 +82,34 @@ puts summary
82
82
 
83
83
  if ARGV.empty?
84
84
  templaters = []
85
- globals = env.recursive_inject([0, nil]) do |(nesting_depth, last), current|
86
- leader = nesting_depth == 0 ? "" : '| ' * (nesting_depth - 1) + (last == current ? "`- " : "|- ")
87
- templaters << Tap::Support::Templater.new("<%= leader %><%= env_key %> \n",
85
+ visited = []
86
+ globals = env.recursive_inject([nil, nil]) do |(leader, last), current|
87
+ current_leader = if leader
88
+ leader.to_s + (last == current ? "`- " : "|- ")
89
+ else
90
+ ""
91
+ end
92
+
93
+ templaters << Tap::Templater.new("<%= leader %><%= env_key %> \n",
88
94
  :env_key => env_keys[current],
89
- :leader => leader
95
+ :leader => current_leader
90
96
  )
91
97
 
92
- [nesting_depth + 1, current.envs[-1]]
98
+ if leader
99
+ leader += (last == current ? ' ' : '| ')
100
+ else
101
+ leader = ""
102
+ end
103
+
104
+ visited << current
105
+ current.envs.reverse_each do |e|
106
+ unless visited.include?(e)
107
+ last = e
108
+ break
109
+ end
110
+ end
111
+
112
+ [leader, last]
93
113
  end
94
114
 
95
115
  tree = templaters.collect do |templater|
data/cmd/run.rb CHANGED
@@ -2,10 +2,6 @@
2
2
  #
3
3
  # examples:
4
4
  # tap run --help Prints this help
5
- # tap run -s schema.yml Build and run a workflow
6
- # tap run -s schema.yml a b c Same with [a, b, c] ARGV
7
- #
8
- # schema:
9
5
  # tap run -- task --help Prints help for task
10
6
  # tap run -- load hello --: dump Say hello
11
7
  #
@@ -22,8 +18,11 @@ argv = []
22
18
  while !ARGV.empty? && ARGV[0] !~ Tap::Schema::Parser::BREAK
23
19
  argv << ARGV.shift
24
20
  end
21
+ schema = ARGV.empty? ? nil : Tap::Schema.parse(ARGV)
22
+ ARGV.replace(argv)
25
23
 
26
24
  # parse options
25
+ mode = :run
27
26
  ConfigParser.new(app.config) do |opts|
28
27
  opts.separator ""
29
28
  opts.separator "configurations:"
@@ -44,53 +43,89 @@ ConfigParser.new(app.config) do |opts|
44
43
  exit(0)
45
44
  end
46
45
 
47
- opts.on('-T', '--manifest', 'Print a list of available tasks') do
46
+ opts.on('-s', '--schema FILE', 'Use the specifed schema') do |file|
47
+ if schema
48
+ puts "An inline schema cannot be specified with a file schema."
49
+ exit(0)
50
+ end
51
+
52
+ schema = Tap::Schema.load_file(file)
53
+ end
54
+
55
+ opts.on('-p', '--preview', 'Print the schema as YAML') do
56
+ mode = :preview
57
+ end
58
+
59
+ opts.on('-t', '--manifest', 'Print a list of available resources') do
48
60
  tasks = env.manifest(:task)
49
61
  tasks_found = !tasks.all_empty?
50
62
 
63
+ joins = env.manifest(:join)
64
+ joins_found = !joins.all_empty?
65
+
51
66
  middleware = env.manifest(:middleware)
52
67
  middleware_found = !middleware.all_empty?
53
68
 
54
69
  if tasks_found
55
- puts "=== tasks ===" if middleware_found
70
+ puts "=== tasks ===" if middleware_found || joins_found
56
71
  puts tasks.summarize
57
72
  end
58
73
 
74
+ if joins_found
75
+ puts "=== joins ===" if tasks_found || middleware_found
76
+ puts joins.summarize
77
+ end
78
+
59
79
  if middleware_found
60
- puts "=== middleware ===" if tasks_found
80
+ puts "=== middleware ===" if tasks_found || joins_found
61
81
  puts middleware.summarize
62
82
  end
63
83
 
64
84
  exit(0)
65
85
  end
66
86
 
67
- end.parse!(argv, :clear_config => false, :add_defaults => false)
87
+ opts.on('-T', '--tasks', 'Print a list of available tasks') do
88
+ puts env.manifest(:task).summarize
89
+ exit(0)
90
+ end
91
+
92
+ end.parse!(ARGV, :clear_config => false, :add_defaults => false)
68
93
 
69
94
  #
70
95
  # build and run
71
96
  #
72
97
 
73
- begin
74
- if ARGV.empty?
75
- msg = "No schema specified"
76
-
77
- unless argv.empty?
78
- args = argv[0, 3].join(' ') + (argv.length > 3 ? ' ...' : '')
79
- msg = "#{msg} (did you mean 'tap run -- #{args}'?)"
80
- end
81
-
82
- puts msg
83
- exit(0)
98
+ unless schema
99
+ msg = "No schema specified"
100
+
101
+ unless ARGV.empty?
102
+ args = ARGV[0, 3].join(' ') + (argv.length > 3 ? ' ...' : '')
103
+ msg = "#{msg} (did you mean 'tap run -- #{args}'?)"
84
104
  end
85
-
86
- # parse argv schema
87
- schema = Tap::Schema.parse(ARGV)
105
+
106
+ puts msg
107
+ exit(0)
108
+ end
109
+
110
+ begin
88
111
  app.build(schema, :resources => env)
89
-
90
112
  ARGV.replace(argv)
91
- Tap::Exe.set_signals(app)
92
113
 
93
- app.run
114
+ case mode
115
+ when :run
116
+ Tap::Exe.set_signals(app)
117
+ app.run
118
+ when :preview
119
+ app.to_schema do |type, resources|
120
+ resources.each do |resource|
121
+ const_name = resource.delete(:class).to_s
122
+ resource[:id] = env.reverse_seek(type, false) do |const|
123
+ const.const_name == const_name
124
+ end
125
+ end
126
+ end.dump($stdout)
127
+ end
128
+
94
129
  rescue
95
130
  raise if $DEBUG
96
131
  puts $!.message
data/doc/API CHANGED
@@ -4,21 +4,16 @@
4
4
 
5
5
  == Tap::App
6
6
 
7
- Applications require the following API for nodes, joins, and middleware.
7
+ Applications require the following API for nodes, joins, and middleware. Tap
8
+ provides modules or base classes that implement these APIs and may be used as
9
+ the foundation for subclasses.
8
10
 
9
11
  ==== Node
10
12
 
11
- Note the signature for call can be modified as necessary.
12
-
13
13
  call(*inputs) any return is allowed
14
- dependencies() returns an array of dependency nodes
15
14
  joins() returns an array of joins
16
15
 
17
- ==== Dependency
18
-
19
- Same as a node, but call must be able to execute without inputs. For instance
20
- signatures like call(), call(a=:default), or call(*inputs) are allowed, but
21
- call(a, b, c) is not.
16
+ The signature for call can be modified as necessary.
22
17
 
23
18
  ==== Join
24
19
 
@@ -30,55 +25,59 @@ call(a, b, c) is not.
30
25
  call(node, inputs=[]) any return is allowed
31
26
  stack() returns the original stack
32
27
 
33
- Note the middleware API is essentially the same as for {Rack}[http://rack.rubyforge.org/].
28
+ The middleware API is essentially the same as for {Rack}[http://rack.rubyforge.org/].
34
29
 
35
30
  == Tap::Schema
36
31
 
37
- Schema describe workflows as data. To build a workflow from a schema, workflow classes need to instantiate themselves using the schema data. The <tt>parse!</tt> and <tt>instantiate</tt> methods are provided to do so.
32
+ Schema describe workflows as data. To build a workflow from a schema, workflow
33
+ resources like nodes, joins, and middleware need to instantiate themselves
34
+ using the schema data. The <tt>parse!</tt> and <tt>instantiate</tt> methods
35
+ must be provided to do so.
38
36
 
39
- WorkflowClass.parse!(argv=ARGV, app=App.instance)
40
- WorkflowClass.instantiate(argh, app=App.instance)
41
-
42
- As implied in by the inputs, <tt>parse!</tt> instantiates from an array, while <tt>instantiate</tt> instantiates from a hash with symbol keys. If <tt>parse!</tt> receives a string, it must be able to convert it to an array (ex using Shellwords).
37
+ Resource.parse!(argv=ARGV, app=App.instance)
38
+ Resource.instantiate(argh, app=App.instance)
43
39
 
44
- How the class actually performs the instantiation is up to the class but typically parse creates a hash and calls instantiate.
40
+ As implied in by the inputs, <tt>parse!</tt> instantiates from an array, while
41
+ <tt>instantiate</tt> instantiates from a hash with symbol keys. If
42
+ <tt>parse!</tt> receives a string, it must be able to convert it to an array
43
+ (ex using Shellwords).
45
44
 
46
- ==== Tap::Task (and subclasses)
45
+ How the class actually performs the instantiation is up to the class but
46
+ typically parse creates a hash and calls instantiate.
47
47
 
48
- Parse and instantiate must return: [instance, args]
48
+ == Tap::Env
49
49
 
50
- ==== Tap::Join (and subclasses)
50
+ Envs identify resources by resource identifiers (ie constant attributes
51
+ recognized by Lazydoc). This identifies 'Sample' as an 'example' resource.
51
52
 
52
- Parse and instantiate must return: [inputs, outputs, instance]
53
+ [file.rb]
53
54
 
54
- == Tap::Env
55
+ # Sample::example summary
56
+ # description
57
+ class Sample
58
+ end
55
59
 
56
- Envs identify resources by a constant attribute using Tap::Env#constant_manifest. For instance this identifies the Sample node:
60
+ The constant name will be inferred from the path for the file containing the
61
+ resource identifier if no constant name is specified.
57
62
 
58
63
  [sample.rb]
59
64
 
60
- # ::node summary
65
+ # ::example summary
61
66
  # description
62
- class Sample < Tap::Node
67
+ class Sample
63
68
  end
64
69
 
65
- Here the constant name is inferred using the filepath. A more formal declaration allows resources to be placed in files that are not named after the resource:
70
+ Resources may be accessed using Tap::Env#manifest. Resources intended to be
71
+ discovered by Env must map the identifier documentation to the class 'desc'
72
+ method. The easiest way to do so is with a
73
+ {lazy_attr}[link:lazydoc/classes/Lazydoc/Attributes.html].
66
74
 
67
- [alt.rb]
68
-
69
- # Sample::node summary
75
+ # ::example summary
70
76
  # description
71
- class Sample < Tap::Node
77
+ class Sample
78
+ lazy_attr :desc, 'example'
72
79
  end
73
-
74
- Joins are similarly identified by <tt>::join</tt>. Resources intended to be discovered by constant_manifest must map the attribute documentation to the class 'desc' method.
75
-
80
+
76
81
  Sample::desc.summary # => "summary"
77
82
  Sample::desc.to_s # => "description"
78
83
 
79
- The easiest way to do so is with a {lazy_attr}[link:lazydoc/classes/Lazydoc/Attributes.html].
80
-
81
- # ::resource
82
- class Resource
83
- lazy_attr :desc, 'resource'
84
- end
@@ -5,20 +5,16 @@ specifically Tasks, Apps, and Envs.
5
5
 
6
6
  == Tasks
7
7
 
8
- ==== Methods
9
-
10
- http://tap.rubyforge.org/images/Method.png
11
-
12
- Tasks are fundamentally just a method, simply a block of code.
13
-
14
8
  ==== Tap::App::Node
15
9
 
16
10
  http://tap.rubyforge.org/images/Node.png
17
11
 
18
12
  Nodes are the building blocks of workflows. Nodes are essentially a method
19
- wrapped with support for dependencies and joins. Any object responding to
20
- <tt>call</tt> may be turned into a node, so any method may be used in a
21
- workflow. Tasks are constructed so that <tt>call</tt> forwards arguments to
13
+ with an array of joins to call when the method completes. Any object
14
+ responding to <tt>call</tt> may be turned into a node, so any method may be
15
+ used in a workflow.
16
+
17
+ Tasks are constructed so that <tt>call</tt> forwards arguments to
22
18
  <tt>process</tt>. This allows hooks and callbacks to be inserted as needed in
23
19
  subclasses.
24
20
 
@@ -28,8 +24,8 @@ http://tap.rubyforge.org/images/Configurable.png
28
24
 
29
25
  Tap uses the {Configurable}[http://tap.rubyforge.org/configurable/] module to
30
26
  declare class configurations and make them available on the command line.
31
- Configurations are essentially a shorthand to define attributes with a default
32
- value. For instance:
27
+ Configurations provide a shorthand to define attributes with a default value.
28
+ For instance:
33
29
 
34
30
  class ConfigClass
35
31
  include Configurable
@@ -69,17 +65,18 @@ see here:
69
65
  c.key = 'another value'
70
66
  c.config[:key] # => 'ANOTHER VALUE'
71
67
 
72
- This setup is both fast and convenient. See the {Configurable}[http://tap.rubyforge.org/configurable]
73
- documentation for more information.
68
+ This setup is both fast and convenient. See the
69
+ {Configurable}[http://tap.rubyforge.org/configurable] documentation for more
70
+ information.
74
71
 
75
72
  ==== {Configurable::Validation}[http://tap.rubyforge.org/configurable/classes/Configurable/Validation.html]
76
73
 
77
- When configurations are set from the command line, the config writer inevitably
78
- receives a string, even though you may want a non-string input. The
74
+ When configurations are set from the command line, the config writer
75
+ inevitably receives a string, even though you may want a non-string input. The
79
76
  {Validation}[http://tap.rubyforge.org/configurable/classes/Configurable/Validation.html]
80
- module provides standard blocks for validating and transforming inputs, and
77
+ module provides standard blocks to validate and transform inputs. These blocks
81
78
  may be accessed through the shortcut method <tt>c</tt> (ex: <tt>c.integer</tt>
82
- or <tt>c.regexp</tt>). Validation blocks (generally) load string inputs as
79
+ or <tt>c.regexp</tt>). Validation blocks generally load string inputs as
83
80
  YAML and validate that the result is the correct class; non-string inputs are
84
81
  simply validated.
85
82
 
@@ -107,8 +104,8 @@ a config into a flag on the command line.
107
104
  ==== {Lazydoc}[http://tap.rubyforge.org/lazydoc]
108
105
 
109
106
  {Lazydoc}[http://tap.rubyforge.org/lazydoc] fits into the space between live
110
- code and code documentation. Lazydoc can scan a file and pull documentation
111
- into the object space, typically via a syntax like this:
107
+ code and code documentation. Lazydoc scans files to pull documentation into
108
+ the object space, and uses a syntax like this:
112
109
 
113
110
  # Constant::key value
114
111
  # comment
@@ -130,11 +127,11 @@ For example:
130
127
  lazydoc['Const::Name']['key'].value # => "value"
131
128
  lazydoc['Const::Name']['key'].comment # => "This is an extended, multiline comment."
132
129
 
133
- Lazydoc can also register specific lines for documentation, like method
130
+ Lazydoc can also register specific lines for documentation, such as method
134
131
  definitions or configurations. Tap uses Lazydoc to identify files that contain
135
- tasks and generators, to extract config documentation, and to infer the args
136
- arguments a task receives. Here is an example of information gleaned from a
137
- task definition:
132
+ resources like tasks and joins, to extract config documentation, and to infer
133
+ the arguments a task receives from the command line. Here is an example of
134
+ information gleaned from a task definition:
138
135
 
139
136
  # Sample::task a summary of the task
140
137
  class Sample < Tap::Task
@@ -157,53 +154,46 @@ information.
157
154
 
158
155
  http://tap.rubyforge.org/images/Task.png
159
156
 
160
- Tasks are the bread and butter of Tap. Tasks act as the nodes in workflows
161
- and, using the Configurable and Lazydoc metadata, map to the command line as
162
- miniature applications. That said, tasks are perfectly useful as ordinary
163
- objects.
157
+ Tasks are the bread and butter of Tap. Tasks serve as nodes in workflows and
158
+ map to the command line as miniature applications. Tasks are also useful as
159
+ ordinary objects.
164
160
 
165
- When subclassing is too much, tasks may be interned with a block that
166
- effectively replaces <tt>process</tt>:
161
+ Tasks may be dynamically generated on an app using the <tt>task</tt> method. This provides a quick way of sketching out a workflow.
167
162
 
168
- t = Tap::Task.intern {|task| 1 + 2 }
163
+ app = Tap::App.instance
164
+ t = app.task {|task| 1 + 2 }
169
165
  t.process # => 3
170
166
 
171
- t = Tap::Task.intern {|task, x, y| x + y }
167
+ t = app.task {|task, x, y| x + y }
172
168
  t.process(1, 2) # => 3
173
169
 
174
- Tasks can be configured,
170
+ Tasks can be configured and joined into workflows.
175
171
 
176
172
  runlist = []
177
- t1 = Tap::Task.intern(:key => 'one') do |task, input|
173
+ results = []
174
+
175
+ t1 = app.task(:key => 'one') do |task, input|
178
176
  runlist << task
179
177
  "#{input}:#{task.config[:key]}"
180
178
  end
181
179
 
182
- joined into dependency-based workflows,
183
-
184
- t0 = Tap::Task.intern {|task| runlist << task }
185
- t1.depends_on(t0)
186
-
187
- and imperative workflows.
188
-
189
- t2 = Tap::Task.intern do |task, input|
180
+ t2 = app.task do |task, input|
190
181
  runlist << task
191
182
  "#{input}:two"
192
183
  end
193
- t1.sequence(t2)
194
184
 
195
- results = []
196
185
  t2.on_complete do |result|
197
186
  results << result
198
187
  end
199
188
 
189
+ t1.sequence(t2)
190
+
200
191
  Results may be collected by the underlying Tap::App.
201
192
 
202
- app = Tap::App.instance
203
193
  app.enq(t1)
204
194
  app.run
205
195
 
206
- runlist # => [t0, t1, t2]
196
+ runlist # => [t1, t2]
207
197
  results # => ["input:one:two"]
208
198
 
209
199
  == Apps
@@ -212,7 +202,7 @@ Results may be collected by the underlying Tap::App.
212
202
 
213
203
  http://tap.rubyforge.org/images/Queue.png
214
204
 
215
- Apps coordinate the execution of tasks through a queue. The queue is just a
205
+ Apps coordinate the execution of nodes through a queue. The queue is just a
216
206
  stack of nodes and inputs; during a run the nodes are sequentially executed
217
207
  with the inputs.
218
208