tap 0.12.4 → 0.17.0

data/History

@@ -1,3 +1,37 @@
+== 0.17.0 / 2009-05-25
+
+Significant reorganization and update to Tap internals.
+The workflow syntax for joins has changed in incompatible
+ways, and several key methods have been renamed.  
+Obviously not backwards compatible but most tasks will
+work without alteration.
+
+Upgrade Notes:
+
+* Tasks now are identified by "::task" rather than "::manifest"
+  The manifest identifier still works for now but will be
+  removed by 1.0.
+* Root#filepath and Root#relative_filepath have been renamed
+  Root#path and Root#relative_path
+* Audits have been removed
+* Generators, FileTask, and test modules have been removed
+  to separate modules (tap-gen, tap-tasks, tap-test). Install
+  tap-suite to get them all back.
+* switched const_name, name on Constant
+* refactored on_complete_block to join
+* callbacks in Task have been removed
+* schema syntax no longer supports rounds, globals
+
+Changes:
+
+* added application middleware
+* reworked dump to use an io target config
+* added unloading to Constant
+* reworked App to use aggregators that respond_to? call
+* reworked Join so that it may be dumped as YAML
+* reworked Schema syntax
+* etc
+
 == 0.12.4 / 2009-03-23
 
 * fixed bug in App#inspect

data/README

@@ -2,51 +2,47 @@
 
   tap n. to draw a supply from a resource
 
-A framework for creating configurable, distributable tasks and workflows.
+A configurable, distributable workflow framework.
 
 == Description
 
-Tap tasks are designed to be easy to test, subclass, use in scripts, and run 
-from the command line.  Tap provides methods to join tasks into imperative 
-and dependecy-based workflows, utilize static config files, as well as dump 
-and reuse results.  Task libraries are readily shared as gems.  
-
-Check out these links for tutorials, development, and bug tracking.
-
-* {Tutorial}[link:files/doc/Tutorial.html], {Class}[link:files/doc/Class%20Reference.html], {Command}[link:files/doc/Command%20Reference.html], and {Syntax}[link:files/doc/Syntax%20Reference.html] References
+Tap allows the construction of imperative and dependency-based workflows that
+may be defined, configured, and run from the command line. The tasks and joins
+composing a workflow are easy to test, subclass, and distribute as gems.
+
+Tap is the core of the
+{Tap-Suite}[http://tap.rubyforge.org/tap-suite/index.html] 
+which provides a standard library of 
+{tasks}[http://tap.rubyforge.org/tap-tasks/index.html],
+{generators}[http://tap.rubyforge.org/tap-gen/index.html], 
+{test utilities}[http://tap.rubyforge.org/tap-test/index.html], 
+and a
+{server}[http://tap.rubyforge.org/tap-server/index.html] 
+to execute workflows via HTTP.
+
+* {Tutorial}[link:files/doc/Tutorial.html], {API}[link:files/doc/API.html], {Structure}[link:files/doc/Class%20Reference.html]
 * Website[http://tap.rubyforge.org]
 * Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
 * Github[http://github.com/bahuvrihi/tap/tree/master]
 * {Google Group}[http://groups.google.com/group/ruby-on-tap]
 
-=== Features
+== Usage
 
-- Tap and Rake[http://rake.rubyforge.org/] are targeted at different problems, 
-  but play well together. See the {Tutorial}[link:files/doc/Tutorial.html].
-- Tap is tested on MRI (the standard Ruby interpreter, versions 1.8.6 and 
-  1.9.0) and JRuby[http://jruby.codehaus.org/].
-- Tap is the core of the {Tap-Suite}[http://tap.rubyforge.org/tap-suite], a
-  collection of several modules that simplify development.
-  
-=== Usage
-
-A simple task illustrates the usage of tap:
+Tasks are defined as subclasses of Tap::Task.
 
   [lib/goodnight.rb]
   
-  # Goodnight::manifest your basic goodnight moon task
+  # Goodnight::task your basic goodnight moon task
   # Says goodnight with a configurable message.
   class Goodnight < Tap::Task
-
     config :message, 'goodnight'           # a goodnight message
-
+    
     def process(name)
-      log(message, name)
       "#{message} #{name}"
     end
   end
 
-Tap pulls documentation out of task classes to generate manifests:
+Tap automatically discovers and pulls documentation out to generate manifests:
 
   % tap run -T
     sample:
@@ -71,26 +67,52 @@ And help:
           --help                       Print this help
           --name NAME                  Specifies the task name
           --config FILE                Specifies a config file
-          --use FILE                   Loads inputs to ARGV
 
-Tasks are immediately available to run through the tap executable:
+Tasks are available immediately for use in workflows. The process method takes
+inputs from the command line and passes its result to other tasks.
+
+  % tap run -- goodnight moon --: dump
+  goodnight moon
+
+Tasks may be configured as if they were individual executables:
+
+  % tap run -- goodnight world --message hello --: dump
+  hello world
+
+=== Workflow Syntax
 
-  % tap run -- goodnight moon
-    I[00:09:55]          goodnight moon
+Workflows are specified on the command line using argument vectors separated by option breaks.  The argument vectors define the tasks and modifications to the breaks (--: or --[i,j][k]) specify joins.
 
-Tap comes with generators. To get started:
+A simple sequence:
 
-  % tap generate root sample
-  % cd sample
-  % tap generate task goodnight
-  % tap run -- goodnight moon
+  % tap run -- load 'goodnight moon' --: dump
+  goodnight moon
 
-=== Bugs/Known Issues
+A more formal way of specifying a sequence:
 
-- Some inconsequential tests on JRuby fail due to bugs in JRuby itself.
-- Joins, Parser, and Schema require additional documentation and may be
-  reworked in a future release.  The workflow syntax should remain the same.
-- Audit.dump only works for certain cases.  Merges frequently cause trouble.
+  % tap run -- load 'goodnight moon' -- dump --[0][1]
+  goodnight moon
+
+A fork:
+
+  % tap run -- load 'goodnight moon' -- dump -- dump --[0][1,2]
+  goodnight moon
+  goodnight moon
+
+A merge (note that dump receives the inputs in serial):
+
+  % tap run -- load goodnight -- load moon -- dump --[0,1][2]
+  goodnight
+  moon
+
+A synchronized merge where the results of each input are collected into an array before being passed to dump. The printout is: ['goodnight', 'moon'].to_s
+
+  % tap run -- load goodnight -- load moon -- dump --[0,1][2].sync
+  goodnightmoon
+
+== Known Issues
+
+Tap::Env and Tap::Schema are still in flux and have poor documentation.  Neither should be treated as stable.
 
 == Installation
 
@@ -109,5 +131,4 @@ Tap requires an updated version of RubyGems[http://docs.rubygems.org/]
 Copyright (c) 2006-2009, Regents of the University of Colorado.
 Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com], {Biomolecular Structure Program}[http://biomol.uchsc.edu/], {Hansen Lab}[http://hsc-proteomics.uchsc.edu/hansenlab/] 
 Support:: CU Denver School of Medicine Deans Academic Enrichment Fund
-License:: {MIT-Style}[link:files/MIT-LICENSE.html]
-
+License:: {MIT-Style}[link:files/MIT-LICENSE.html]

data/bin/tap

@@ -1,7 +1,7 @@
 #!/usr/bin/env ruby
-# usage: tap <command> {options} [args] [-d-]
+# usage: tap <command> {options} [args]
 #
-# Launches a tap command. To enable debugging, append the -d- superflag.
+# Launches a tap command.
 #
 # examples:
 #   tap generate root .                  # generates a root dir
@@ -14,9 +14,12 @@
 
 require "#{File.dirname(__FILE__)}/../lib/tap.rb"
 
+#
 # setup the environment
+#
+
 begin
-  env = Tap::Exe.setup(ARGV)
+  exe = Tap::Exe.setup
 rescue(Tap::Env::ConfigError)
   # catch errors and exit gracefully
   # (errors usu from gem loading errors)
@@ -25,47 +28,40 @@ rescue(Tap::Env::ConfigError)
 end
 
 #
-# setup after script
+# launch tap
 #
 
-at_exit do
-  begin
-    eval(env.after) if env.after != nil
-  rescue(Exception)
-    puts "Error in after script."
-    env.handle_error($!)
-    exit(1)
-  end
-end
-
-#
-# run before script
-#
+exe.activate
+exe.launch(ARGV) do
+  template = %Q{<% unless entries.empty? || count <= 1 %>
+<%= env_key %>:
+<% end %>
+<% entries.each do |key, path| %>
+  <%= key.ljust(width) %>
+<% end %>}
 
-begin
-  eval(env.before) if env.before != nil
-rescue(Exception)
-  puts "Error in before script."
-  env.handle_error($!)
-  exit(1)
-end 
-
-#
-# run tap
-#
-
-begin
-  env.activate
-  env.execute(ARGV) do
-    puts Lazydoc.usage(__FILE__)
-    puts
-    puts "available commands:"
-    puts env.summarize(:commands)
-    puts
-    puts "version #{Tap::VERSION} -- #{Tap::WEBSITE}"
+  commands = exe.manifest(:command).inspect(template,
+    :count => 0,
+    :width => 10
+  ) do |templater, globals|
+    entries = templater.manifest.minimap
+    templater.entries = entries
+    
+    # determine width, count
+    width = globals[:width]
+    entries.each do |key, path|
+      width = key.length if width < key.length
+    end
+    globals[:width] = width
+    globals[:count] += 1 unless entries.empty?
   end
-rescue
-  env.handle_error($!)
+  
+  puts Lazydoc.usage(__FILE__)
+  puts
+  puts "available commands:"
+  puts commands
+  puts
+  puts "version #{Tap::VERSION} -- #{Tap::WEBSITE}"
 end
 
 exit(0)

data/cmd/console.rb

@@ -1,13 +1,14 @@
 # usage: tap console [options]
 #
 # Opens up an IRB session with the Tap environment initialized as specified
-# in tap.yml. Access the Tap::App.instance through 'app', and
-# Tap::Env.instance through 'env'.  For example:
+# in tap.yml. Access a Tap::App instance through 'app' and the execution
+# environment through 'env'.  For example:
 #
 #   % tap console
-#   >> env.tasks.search('tap:dump').constantize=> Tap::Tasks::Dump
+#   >> env[:task][:dump]
+#   => Tap::Tasks::Dump
 #   >> app.info
-#   => "state: 0 (READY) queue: 0 results: 0"
+#   => "state: 0 (READY) queue: 0"
 #   >> 
 #
 
@@ -25,11 +26,18 @@ end.parse!(ARGV)
 require "irb"
 
 def app
-  Tap::App.instance
+  @app ||= Tap::App.new
 end
 
 def env
-  Tap::Env.instance
+  @env ||= Tap::Env.instance
+end
+
+def run(cmd, reset=true)
+  app.reset if reset
+  schema = Tap::Schema.parse(cmd)
+  env.run(schema, app)
+  nil
 end
 
 IRB.start

data/cmd/manifest.rb

@@ -1,7 +1,11 @@
-# usage: tap manifest [options]
+# usage: tap manifest [options] [filter]
+#
+# Prints a manifest of all resources in the current tap environment.  Env
+# keys may be provided to select a specific set of environments to list.
+#
+#   % tap manifest
+#   % tap manifest tap-tasks
 #
-# Prints a manifest of all resources in the current tap environment.
-
 options = {}
 ConfigParser.new do |opts|
   
@@ -13,85 +17,85 @@ ConfigParser.new do |opts|
     puts opts
     exit
   end
-  
-  # opts.on("-t", "--tree", "Just print the env tree.") do
-  #   options[:tree] = true
-  # end
-  
-  # opts.on("-r", "--require FILEPATH", "Require the specified file") do |value|
-  #   require value
-  # end
-  
 end.parse!(ARGV)
 
-env = Tap::Env.instance
-env_names = {}
-env.minimap.each do |env_name, environment|
-  env_names[environment] = env_name
-end
+template = %Q{<% unless manifests.empty? %>
+#{'-' * 80}
+<%= (env_key + ':').ljust(width) %> (<%= env.root.root %>)
+<% manifests.each do |type, entries| %>
+  <%= type %>
+<%   entries.each do |key, value| %>
+    <%= key.ljust(width-4) %> (<%= value %>)
+<%   end %>
+<% end %>
+<% end %>
+}
 
-filter = case
-when ARGV.empty? then env_names.keys
+# filter envs to manifest
+env = Tap::Env.instance
+env_keys = env.minihash(true)
+filter = if ARGV.empty?
+  env_keys.keys
 else
   ARGV.collect do |name| 
-    unless entry = env.find(:envs, name, false) 
-      raise "could not find an env matching: #{name}"
-    end
-
-    entry[1]
+    env.minimatch(name) or raise "could not find an env matching: #{name}"
   end
 end
 
-template = %Q{#{'-' * 80}
-<%= (env_name + ':').ljust(width) %> (<%= env.root.root %>)
-<% manifests.each do |manifest_name, entries| %>
-  <%= manifest_name %>
-<%   entries.each do |name, path| %>
-    <%= name.ljust(width-4) %> (<%= path %>)
-<%   end %>
-<% end %>
-}
-
-width = 10
-summary = env.inspect(template) do |templater, share|
+# build the summary
+summary = env.inspect(template, :width => 10) do |templater, globals|
   current = templater.env
-  next unless filter.include?(current)
-
   manifests = []
-  [:commands, :generators, :tasks].each do |name|
-    manifest = current.send(name)
-    next if manifest.build.empty?
+  templater.manifests = manifests
+  next unless filter.include?(current)
+  
+  # determine the width of the keys
+  width = globals[:width]
+  env_key = env_keys[current]
+  templater.env_key = env_key
+  width = env_key.length if width < env_key.length
+  
+  # build up the entries for each type of resource
+  current.registry(true).to_a.sort_by do |(type, entries)|
+    type.to_s
+  end.each do |type, entries|
+    next if entries.empty?
     
-    entries = manifest.minimap.collect do |(entry, path)|
-      path = case path
-      when Tap::Support::Constant then path.require_path
-      else path
+    entries.extend(Tap::Env::Minimap)
+    entries = entries.minimap.collect do |key, entry|
+      path = if entry.kind_of?(Tap::Env::Constant)
+        entry.require_path
+      else
+        entry
       end
   
-      width = entry.length if width < entry.length
-      [entry, current.root.relative_filepath(:root, path) || path]
+      width = key.length if width < key.length
+      [key, current.root.relative_path(:root, path) || path]
     end
     
-    manifests << [name, entries]
+    manifests << [type, entries]
   end
-  templater.manifests = manifests.compact
-  templater.env_name = env_names[current]
   
-  width = templater.env_name.length if width < templater.env_name.length
-  share[:width] = width + 2
+  globals[:width] = width
 end
 puts summary
 
 if ARGV.empty?
-  tree = env.recursive_inspect("<%= leader %><%= env_name %> \n", 0, nil) do |templater, share, nesting_depth, last|
-    current = templater.env
-    
-    templater.leader = nesting_depth == 0 ? "" : '|   ' * (nesting_depth - 1) + (last == current ? "`- " : "|- ")
-    templater.env_name = env_names[current]
+  templaters = []
+  globals = env.recursive_inject([0, nil]) do |(nesting_depth, last), current|
+    leader = nesting_depth == 0 ? "" : '|   ' * (nesting_depth - 1) + (last == current ? "`- " : "|- ")
+    templaters << Tap::Support::Templater.new("<%= leader %><%= env_key %> \n", 
+      :env_key => env_keys[current],
+      :leader => leader
+    )
     
     [nesting_depth + 1, current.envs[-1]]
   end
-  
+
+  tree = templaters.collect do |templater|
+    templater.build
+  end.join
+
   puts '-' * 80
   puts
   puts tree