tap 0.17.1 → 0.18.0

data/History

@@ -1,3 +1,25 @@
+== 0.18.0 / 2009-06-17
+
+Several updates to simplify Tap.  Most significantly,
+dependencies have been removed because their general
+utility is quite doubtful, and it it easy to implement
+in situations that require them (ex rap).
+
+* bug fix in console (app now refers to Tap::App.instance)
+* bug fix in load for recurrently loading from a file
+* fixed a mistake in run help
+* made load only load once by default, stream loading must
+  be enabled in subclasses
+* dump now returns the input, not the io
+* removed dependencies from tap
+* removed intern method on Task (now use app.task)
+* removed Support module (ie refactored Templater to
+  Tap::Templater)
+* removed Task.help method
+* added Middleware baseclass
+* Task.parse now yields opts to block for modifications
+* renamed 'tap/constants' file as 'tap/version'
+
 == 0.17.1 / 2009-06-06
 
 * documentation and interface updates

data/README

@@ -6,9 +6,9 @@ A configurable, distributable workflow framework.
 
 == Description
 
-Tap allows the construction of imperative and dependency-based workflows that
-may be defined, configured, and run from the command line. The tasks and joins
-composing a workflow are easy to test, subclass, and distribute as gems.
+Tap allows the construction of workflows that may be defined, configured, and
+run from the command line. The tasks and joins composing a workflow are easy
+to test, subclass, and distribute as gems.
 
 Tap is the core of the
 {Tap-Suite}[http://tap.rubyforge.org/tap-suite/index.html] 
@@ -42,7 +42,7 @@ Tasks are defined as subclasses of Tap::Task.
     end
   end
 
-Tap automatically discovers and pulls documentation out to generate manifests:
+Tap automatically discovers tasks.
 
   % tap run -T
     sample:
@@ -51,7 +51,7 @@ Tap automatically discovers and pulls documentation out to generate manifests:
       dump        # the default dump task
       load        # the default load task
       
-And help:
+And generates documentation.
 
   % tap run -- goodnight --help
   Goodnight -- your basic goodnight moon task
@@ -68,44 +68,45 @@ And help:
           --name NAME                  Specifies the task name
           --config FILE                Specifies a config file
 
-Tasks are available immediately for use in workflows. The process method takes
-inputs from the command line and passes its result to other tasks.
+Tasks are immediately available for use in workflows.
 
   % tap run -- goodnight moon --: dump
   goodnight moon
 
-Tasks may be configured as if they were individual executables:
+Tasks may be configured as if they were individual executables.
 
   % tap run -- goodnight world --message hello --: dump
   hello world
 
 === Workflow Syntax
 
-Workflows are specified on the command line using argument vectors separated by option breaks.  The argument vectors define the tasks and modifications to the breaks (--: or --[i,j][k]) specify joins.
+Workflows are specified on the command line using argument vectors separated
+by option breaks. The vectors define the tasks and modifications to the breaks
+specify joins.
 
-A simple sequence:
+A simple sequence.
 
   % tap run -- load 'goodnight moon' --: dump
   goodnight moon
 
-A more formal way of specifying a sequence:
+A more formal way of specifying a sequence.
 
   % tap run -- load 'goodnight moon' -- dump --[0][1]
   goodnight moon
 
-A fork:
+A fork.
 
   % tap run -- load 'goodnight moon' -- dump -- dump --[0][1,2]
   goodnight moon
   goodnight moon
 
-A merge (note that dump receives the inputs in serial):
+A merge (note that dump receives the inputs in serial).
 
   % tap run -- load goodnight -- load moon -- dump --[0,1][2]
   goodnight
   moon
 
-A synchronized merge where the results of each input are collected into an array before being passed to dump. The printout is: ['goodnight', 'moon'].to_s
+A synchronized merge (the printout is ['goodnight', 'moon'].to_s).
 
   % tap run -- load goodnight -- load moon -- dump --[0,1][2].sync
   goodnightmoon

data/cmd/console.rb

@@ -26,7 +26,7 @@ end.parse!(ARGV)
 require "irb"
 
 def app
-  @app ||= Tap::App.new
+  @app ||= Tap::App.instance
 end
 
 def env

data/cmd/manifest.rb

@@ -82,14 +82,34 @@ puts summary
 
 if ARGV.empty?
   templaters = []
-  globals = env.recursive_inject([0, nil]) do |(nesting_depth, last), current|
-    leader = nesting_depth == 0 ? "" : '|   ' * (nesting_depth - 1) + (last == current ? "`- " : "|- ")
-    templaters << Tap::Support::Templater.new("<%= leader %><%= env_key %> \n", 
+  visited = []
+  globals = env.recursive_inject([nil, nil]) do |(leader, last), current|
+    current_leader = if leader
+      leader.to_s + (last == current ? "`- " : "|- ")
+    else
+      ""
+    end
+    
+    templaters << Tap::Templater.new("<%= leader %><%= env_key %> \n", 
       :env_key => env_keys[current],
-      :leader => leader
+      :leader => current_leader
     )
     
-    [nesting_depth + 1, current.envs[-1]]
+    if leader
+      leader += (last == current ? '    ' : '|   ')
+    else
+      leader = ""
+    end
+    
+    visited << current
+    current.envs.reverse_each do |e|
+      unless visited.include?(e)
+        last = e
+        break
+      end
+    end
+    
+    [leader, last]
   end
 
   tree = templaters.collect do |templater|

data/cmd/run.rb

@@ -2,10 +2,6 @@
 #
 # examples:
 #   tap run --help                     Prints this help
-#   tap run -s schema.yml              Build and run a workflow
-#   tap run -s schema.yml a b c        Same with [a, b, c] ARGV
-#
-# schema:
 #   tap run -- task --help             Prints help for task
 #   tap run -- load hello --: dump     Say hello
 #
@@ -22,8 +18,11 @@ argv = []
 while !ARGV.empty? && ARGV[0] !~ Tap::Schema::Parser::BREAK
   argv << ARGV.shift
 end
+schema = ARGV.empty? ? nil : Tap::Schema.parse(ARGV)
+ARGV.replace(argv)
 
 # parse options
+mode = :run
 ConfigParser.new(app.config) do |opts|
   opts.separator ""
   opts.separator "configurations:"
@@ -44,53 +43,89 @@ ConfigParser.new(app.config) do |opts|
     exit(0)
   end
   
-  opts.on('-T', '--manifest', 'Print a list of available tasks') do
+  opts.on('-s', '--schema FILE', 'Use the specifed schema') do |file|
+    if schema
+      puts "An inline schema cannot be specified with a file schema."
+      exit(0)
+    end
+    
+    schema = Tap::Schema.load_file(file)
+  end
+  
+  opts.on('-p', '--preview', 'Print the schema as YAML') do
+    mode = :preview
+  end
+  
+  opts.on('-t', '--manifest', 'Print a list of available resources') do
     tasks = env.manifest(:task)
     tasks_found = !tasks.all_empty?
     
+    joins = env.manifest(:join)
+    joins_found = !joins.all_empty?
+    
     middleware = env.manifest(:middleware)
     middleware_found = !middleware.all_empty?
     
     if tasks_found 
-      puts "=== tasks ===" if middleware_found
+      puts "=== tasks ===" if middleware_found || joins_found
       puts tasks.summarize
     end
 
+    if joins_found
+      puts "=== joins ===" if tasks_found || middleware_found
+      puts joins.summarize
+    end
+    
     if middleware_found
-      puts "=== middleware ===" if tasks_found
+      puts "=== middleware ===" if tasks_found || joins_found
       puts middleware.summarize
     end
     
     exit(0)
   end
   
-end.parse!(argv, :clear_config => false, :add_defaults => false)
+  opts.on('-T', '--tasks', 'Print a list of available tasks') do
+    puts env.manifest(:task).summarize
+    exit(0)
+  end
+  
+end.parse!(ARGV, :clear_config => false, :add_defaults => false)
 
 #
 # build and run
 #
 
-begin
-  if ARGV.empty?
-    msg = "No schema specified"
-    
-    unless argv.empty?
-      args = argv[0, 3].join(' ') + (argv.length > 3 ? ' ...' : '')
-      msg = "#{msg} (did you mean 'tap run -- #{args}'?)"
-    end
-    
-    puts msg
-    exit(0)
+unless schema
+  msg = "No schema specified"
+
+  unless ARGV.empty?
+    args = ARGV[0, 3].join(' ') + (argv.length > 3 ? ' ...' : '')
+    msg = "#{msg} (did you mean 'tap run -- #{args}'?)"
   end
-  
-  # parse argv schema
-  schema = Tap::Schema.parse(ARGV)
+
+  puts msg
+  exit(0)
+end
+
+begin
   app.build(schema, :resources => env)
-  
   ARGV.replace(argv)
-  Tap::Exe.set_signals(app)
   
-  app.run
+  case mode
+  when :run
+    Tap::Exe.set_signals(app)
+    app.run
+  when :preview
+    app.to_schema do |type, resources|
+      resources.each do |resource|
+        const_name = resource.delete(:class).to_s
+        resource[:id] = env.reverse_seek(type, false) do |const|
+          const.const_name == const_name
+        end
+      end
+    end.dump($stdout)
+  end
+  
 rescue
   raise if $DEBUG
   puts $!.message

data/doc/API

@@ -4,21 +4,16 @@
 
 == Tap::App
 
-Applications require the following API for nodes, joins, and middleware.
+Applications require the following API for nodes, joins, and middleware. Tap
+provides modules or base classes that implement these APIs and may be used as
+the foundation for subclasses.
 
 ==== Node
 
-Note the signature for call can be modified as necessary.
-
   call(*inputs)    any return is allowed
-  dependencies()   returns an array of dependency nodes
   joins()          returns an array of joins
 
-==== Dependency
-
-Same as a node, but call must be able to execute without inputs. For instance
-signatures like call(), call(a=:default), or call(*inputs) are allowed, but
-call(a, b, c) is not.
+The signature for call can be modified as necessary.
 
 ==== Join
 
@@ -30,55 +25,59 @@ call(a, b, c) is not.
   call(node, inputs=[])   any return is allowed
   stack()                 returns the original stack
 
-Note the middleware API is essentially the same as for {Rack}[http://rack.rubyforge.org/].
+The middleware API is essentially the same as for {Rack}[http://rack.rubyforge.org/].
 
 == Tap::Schema
 
-Schema describe workflows as data.  To build a workflow from a schema, workflow classes need to instantiate themselves using the schema data. The <tt>parse!</tt> and <tt>instantiate</tt> methods are provided to do so.  
+Schema describe workflows as data. To build a workflow from a schema, workflow
+resources like nodes, joins, and middleware need to instantiate themselves
+using the schema data. The <tt>parse!</tt> and <tt>instantiate</tt> methods
+must be provided to do so.
 
-  WorkflowClass.parse!(argv=ARGV, app=App.instance)
-  WorkflowClass.instantiate(argh, app=App.instance)
-  
-As implied in by the inputs, <tt>parse!</tt> instantiates from an array, while <tt>instantiate</tt> instantiates from a hash with symbol keys.  If <tt>parse!</tt> receives a string, it must be able to convert it to an array (ex using Shellwords).
+  Resource.parse!(argv=ARGV, app=App.instance)
+  Resource.instantiate(argh, app=App.instance)
 
-How the class actually performs the instantiation is up to the class but typically parse creates a hash and calls instantiate.
+As implied in by the inputs, <tt>parse!</tt> instantiates from an array, while
+<tt>instantiate</tt> instantiates from a hash with symbol keys. If
+<tt>parse!</tt> receives a string, it must be able to convert it to an array
+(ex using Shellwords).
 
-==== Tap::Task (and subclasses)
+How the class actually performs the instantiation is up to the class but
+typically parse creates a hash and calls instantiate.
 
-Parse and instantiate must return: [instance, args]
+== Tap::Env
 
-==== Tap::Join (and subclasses)
+Envs identify resources by resource identifiers (ie constant attributes
+recognized by Lazydoc). This identifies 'Sample' as an 'example' resource.
 
-Parse and instantiate must return: [inputs, outputs, instance]
+  [file.rb]
 
-== Tap::Env
+  # Sample::example summary
+  # description
+  class Sample
+  end
 
-Envs identify resources by a constant attribute using Tap::Env#constant_manifest.  For instance this identifies the Sample node:
+The constant name will be inferred from the path for the file containing the
+resource identifier if no constant name is specified.
 
   [sample.rb]
   
-  # ::node summary
+  # ::example summary
   # description
-  class Sample < Tap::Node
+  class Sample
   end
 
-Here the constant name is inferred using the filepath.  A more formal declaration allows resources to be placed in files that are not named after the resource:
+Resources may be accessed using Tap::Env#manifest. Resources intended to be
+discovered by Env must map the identifier documentation to the class 'desc'
+method. The easiest way to do so is with a
+{lazy_attr}[link:lazydoc/classes/Lazydoc/Attributes.html].
 
-  [alt.rb]
-  
-  # Sample::node summary
+  # ::example summary
   # description
-  class Sample < Tap::Node
+  class Sample
+    lazy_attr :desc, 'example'
   end
-
-Joins are similarly identified by <tt>::join</tt>.  Resources intended to be discovered by constant_manifest must map the attribute documentation to the class 'desc' method.  
-
+  
   Sample::desc.summary         # => "summary"
   Sample::desc.to_s            # => "description"
   
-The easiest way to do so is with a {lazy_attr}[link:lazydoc/classes/Lazydoc/Attributes.html].
-
-  # ::resource
-  class Resource
-    lazy_attr :desc, 'resource'
-  end

data/doc/Class Reference

@@ -5,20 +5,16 @@ specifically Tasks, Apps, and Envs.
 
 == Tasks
 
-==== Methods
-
-http://tap.rubyforge.org/images/Method.png
-
-Tasks are fundamentally just a method, simply a block of code.
-
 ==== Tap::App::Node
 
 http://tap.rubyforge.org/images/Node.png
 
 Nodes are the building blocks of workflows. Nodes are essentially a method
-wrapped with support for dependencies and joins. Any object responding to
-<tt>call</tt> may be turned into a node, so any method may be used in a
-workflow. Tasks are constructed so that <tt>call</tt> forwards arguments to
+with an array of joins to call when the method completes. Any object
+responding to <tt>call</tt> may be turned into a node, so any method may be
+used in a workflow. 
+
+Tasks are constructed so that <tt>call</tt> forwards arguments to
 <tt>process</tt>. This allows hooks and callbacks to be inserted as needed in
 subclasses.
 
@@ -28,8 +24,8 @@ http://tap.rubyforge.org/images/Configurable.png
 
 Tap uses the {Configurable}[http://tap.rubyforge.org/configurable/] module to
 declare class configurations and make them available on the command line.
-Configurations are essentially a shorthand to define attributes with a default
-value.  For instance:
+Configurations provide a shorthand to define attributes with a default value.
+For instance:
   
   class ConfigClass
     include Configurable
@@ -69,17 +65,18 @@ see here:
   c.key = 'another value'
   c.config[:key]              # => 'ANOTHER VALUE'
 
-This setup is both fast and convenient. See the {Configurable}[http://tap.rubyforge.org/configurable]
-documentation for more information.
+This setup is both fast and convenient. See the
+{Configurable}[http://tap.rubyforge.org/configurable] documentation for more
+information.
 
 ==== {Configurable::Validation}[http://tap.rubyforge.org/configurable/classes/Configurable/Validation.html]
 
-When configurations are set from the command line, the config writer inevitably
-receives a string, even though you may want a non-string input.  The
+When configurations are set from the command line, the config writer
+inevitably receives a string, even though you may want a non-string input. The
 {Validation}[http://tap.rubyforge.org/configurable/classes/Configurable/Validation.html]
-module provides standard blocks for validating and transforming inputs, and
+module provides standard blocks to validate and transform inputs. These blocks
 may be accessed through the shortcut method <tt>c</tt> (ex: <tt>c.integer</tt>
-or <tt>c.regexp</tt>).  Validation blocks (generally) load string inputs as
+or <tt>c.regexp</tt>). Validation blocks generally load string inputs as
 YAML and validate that the result is the correct class; non-string inputs are
 simply validated.
 
@@ -107,8 +104,8 @@ a config into a flag on the command line.
 ==== {Lazydoc}[http://tap.rubyforge.org/lazydoc]
 
 {Lazydoc}[http://tap.rubyforge.org/lazydoc] fits into the space between live
-code and code documentation.  Lazydoc can scan a file and pull documentation
-into the object space, typically via a syntax like this:
+code and code documentation. Lazydoc scans files to pull documentation into
+the object space, and uses a syntax like this:
 
   # Constant::key value
   # comment
@@ -130,11 +127,11 @@ For example:
   lazydoc['Const::Name']['key'].value          # => "value"
   lazydoc['Const::Name']['key'].comment        # => "This is an extended, multiline comment."
 
-Lazydoc can also register specific lines for documentation, like method
+Lazydoc can also register specific lines for documentation, such as method
 definitions or configurations. Tap uses Lazydoc to identify files that contain
-tasks and generators, to extract config documentation, and to infer the args
-arguments a task receives. Here is an example of information gleaned from a
-task definition:
+resources like tasks and joins, to extract config documentation, and to infer
+the arguments a task receives from the command line. Here is an example of
+information gleaned from a task definition:
 
   # Sample::task a summary of the task
   class Sample < Tap::Task
@@ -157,53 +154,46 @@ information.
 
 http://tap.rubyforge.org/images/Task.png
 
-Tasks are the bread and butter of Tap. Tasks act as the nodes in workflows
-and, using the Configurable and Lazydoc metadata, map to the command line as
-miniature applications. That said, tasks are perfectly useful as ordinary
-objects.
+Tasks are the bread and butter of Tap. Tasks serve as nodes in workflows and
+map to the command line as miniature applications. Tasks are also useful as
+ordinary objects.
 
-When subclassing is too much, tasks may be interned with a block that
-effectively replaces <tt>process</tt>:
+Tasks may be dynamically generated on an app using the <tt>task</tt> method.   This provides a quick way of sketching out a workflow.
 
-  t = Tap::Task.intern {|task| 1 + 2 }
+  app = Tap::App.instance
+  t = app.task {|task| 1 + 2 }
   t.process                # => 3
 
-  t = Tap::Task.intern {|task, x, y| x + y }
+  t = app.task {|task, x, y| x + y }
   t.process(1, 2)          # => 3
 
-Tasks can be configured,
+Tasks can be configured and joined into workflows.
 
   runlist = []
-  t1 = Tap::Task.intern(:key => 'one') do |task, input| 
+  results = []
+  
+  t1 = app.task(:key => 'one') do |task, input| 
     runlist << task
     "#{input}:#{task.config[:key]}"
   end
 
-joined into dependency-based workflows,
-
-  t0 = Tap::Task.intern {|task| runlist << task }
-  t1.depends_on(t0)
-
-and imperative workflows.
-
-  t2 = Tap::Task.intern do |task, input|
+  t2 = app.task do |task, input|
     runlist << task
     "#{input}:two"
   end
-  t1.sequence(t2)
   
-  results = []
   t2.on_complete do |result|
     results << result
   end
   
+  t1.sequence(t2)
+
 Results may be collected by the underlying Tap::App.
 
-  app = Tap::App.instance
   app.enq(t1)
   app.run
   
-  runlist                  # => [t0, t1, t2]
+  runlist                  # => [t1, t2]
   results                  # => ["input:one:two"]
 
 == Apps
@@ -212,7 +202,7 @@ Results may be collected by the underlying Tap::App.
 
 http://tap.rubyforge.org/images/Queue.png
 
-Apps coordinate the execution of tasks through a queue. The queue is just a
+Apps coordinate the execution of nodes through a queue. The queue is just a
 stack of nodes and inputs; during a run the nodes are sequentially executed
 with the inputs.