tap 0.10.1 → 0.11.0

data/History

@@ -1,3 +1,15 @@
+== 0.11.0 / 2008-10-20
+
+Significant update to Tap with many internal reworks.
+Major changes include:
+
+* Addition of Parser/Schema
+* Addition of rap and task declarations
+* Removal of Workflow in preference of workflow
+  definitions within Task
+* Refactoring of Test modules
+* Expanded/updated documentation
+
 == 0.10.1 / 2008-08-21
 
 Update of Tap with a few improvements to manifests

data/MIT-LICENSE

@@ -1,6 +1,4 @@
 Copyright (c) 2006-2008, Regents of the University of Colorado.
-Developer:: Simon Chiang, Biomolecular Structure Program, Hansen Lab
-Support:: CU Denver School of Medicine Deans Academic Enrichment Fund
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this
 software and associated documentation files (the "Software"), to deal in the Software

data/README

@@ -5,21 +5,22 @@ A framework for creating configurable, distributable tasks and workflows.
 == Description
 
 Tap tasks are designed to be easy to test, subclass, use in scripts, and run 
-from the command line.  Tap provides methods to generate and utilize config 
-files, join tasks into workflows, as well as dump and reuse results.  Task 
-libraries are readily shared as gems.  Check out the links for tutorials, 
-development, and bug tracking.
+from the command line.  Tap provides methods to join tasks into imperative 
+and dependecy-based workflows, utilize static config files, as well as dump 
+and reuse results.  Task libraries are readily shared as gems.  
 
-* {Tutorial}[link:files/doc/Tutorial.html], {Class}[link:files/doc/Class%20Reference.html] and {Command}[link:files/doc/Command%20Reference.html] Reference
+Check out these links for tutorials, development, and bug tracking.
+
+* {Tutorial}[link:files/doc/Tutorial.html], {Class}[link:files/doc/Class%20Reference.html], {Command}[link:files/doc/Command%20Reference.html], and {Syntax}[link:files/doc/Syntax%20Reference.html] References
 * Website[http://tap.rubyforge.org]
-* Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/overview]
+* Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
 * Github[http://github.com/bahuvrihi/tap/tree/master]
-* Google Group[http://groups.google.com/group/ruby-on-tap]
+* {Google Group}[http://groups.google.com/group/ruby-on-tap]
 
-=== Additional Notes:
+=== Features
 
 - Tap and Rake[http://rake.rubyforge.org/] are targeted at different problems, 
-  but play well together. See Tap::Tasks::Rake to use rake tasks in tap.
+  but play well together. See the {Tutorial}[link:files/doc/Tutorial.html].
 - Tap is tested on MRI (the standard Ruby interpreter, versions 1.8.6 and 
   1.9.0) and JRuby[http://jruby.codehaus.org/].
 
@@ -27,6 +28,8 @@ development, and bug tracking.
 
 A simple task illustrates the usage of tap:
 
+  [lib/goodnight.rb]
+  
   # Goodnight::manifest your basic goodnight moon task
   # Says goodnight with a configurable message.
   class Goodnight < Tap::Task
@@ -46,6 +49,7 @@ Tap pulls documentation out of task classes to generate manifests:
       goodnight   # your basic goodnight moon task
     tap:
       dump        # the default dump task
+      load        # the default load task
       rake        # run rake tasks  
 
 And help:
@@ -66,39 +70,26 @@ And help:
           --use FILE                   Loads inputs from file
 
 
-Tasks are immediately available to run:
+Tasks are immediately available to run through tap or rap:
 
   % tap run -- goodnight moon
     I[00:09:55]          goodnight moon
   
-
-  % tap run -- goodnight moon --message hello
+  % rap goodnight moon --message hello
     I[00:10:01]              hello moon
-  
-Tap comes with generators and allows distribution of task libraries as gems.  
-This can be illustrated by installing the 
-{sample_tasks}[http://tap.rubyforge.org/sample_tasks] gem:
 
-  % gem install sample_tasks
-  % tap run -T
-  sample:
-    goodnight   # your basic goodnight moon task
-  sample_tasks:
-    concat      # concatenate files with formatting
-    copy        # copies files
-    grep        # search for lines matching a pattern
-    print_tree  # print a directory tree
-  tap:
-    dump        # the default dump task
-    rake        # run rake tasks
+Tap comes with generators. To get started:
+
+  % tap generate root sample
+  % cd sample
+  % tap generate task goodnight
+  % rap goodnight moon
 
 === Bugs/Known Issues
 
 - Some inconsequential tests on JRuby fail due to bugs in JRuby itself.
-- Several patches are required so that Tap runs properly on MRI 1.9.0.  These   
-  will likely resolve themselves as 1.9.0 becomes stable.
-- Despite the talk of workflows, the Workflow class should still be considered
-  in progress.  Tasks themselves are stable.
+- Joins, Parser, and Schema require additional documentation and may be
+  internally reworked.  The workflow syntax should remain the same.
 
 == Installation
 

data/bin/rap

@@ -0,0 +1,116 @@
+#!/usr/bin/env ruby
+# usage: rap taskname {options} [args]
+
+$:.unshift "#{File.dirname(__FILE__)}/../lib"
+require 'tap/declarations'
+
+# setup the environment
+begin
+  
+  # handle super options
+  $DEBUG = true if ARGV.delete('-d-')
+  env = Tap::Exe.instantiate
+  env.unshift(Tap::Declarations.env)
+
+  task_file = File.expand_path('Tapfile')
+  if File.exists?(task_file)
+    env.loads.unshift(task_file)
+    env.tasks.register(Dir.pwd, 'Tapfile')
+  end
+  
+rescue(Tap::Env::ConfigError)
+  # catch errors and exit gracefully
+  # (errors usu from gem loading errors)
+  puts $!.message
+  exit(1)
+end
+
+#
+# setup after script
+#
+
+at_exit do
+  begin
+    eval(env.after) if env.after != nil
+  rescue(Exception)
+    puts "Error in after script."
+    env.handle_error($!)
+    exit(1)
+  end
+end
+
+#
+# run before script
+#
+
+begin
+  eval(env.before) if env.before != nil
+rescue(Exception)
+  puts "Error in before script."
+  env.handle_error($!)
+  exit(1)
+end 
+
+#
+# run rap
+#
+
+module Tap
+  module Tasks
+    autoload(:Rake, 'tap/tasks/rake')
+  end
+end
+
+begin
+  env.activate
+  
+  case ARGV[0]
+  when '--help', nil, '-T'
+    rap_template = %Q{<% if count > 1 && !entries.empty? %>
+<%= env_name %>:
+<% end %>
+<% entries.each do |name, const| %>
+<%   desc = if const.require_path == nil # should be a declaration %>
+<%     manifest = const.constantize.lazydoc[const.name]['manifest'] %>
+<%     next if manifest == nil || manifest.subject.empty? %>
+<%     manifest.subject %>
+<%   else %>
+<%     const.document[const.name]['manifest'] %>
+<%   end %>
+  <%= name.ljust(width) %><%= desc.empty? ? '' : '  # ' %><%= desc %>
+<% end %>}
+
+    puts Tap::Support::Lazydoc.usage(__FILE__)
+    puts
+    puts "===  tap tasks ==="
+    puts env.summarize(:tasks, rap_template)
+    puts
+    puts "=== rake tasks ==="
+    Tap::Tasks::Rake.new.enq('-T')
+    Tap::App.instance.run
+    
+    # this is not currently reached as rake exits on -T
+    puts
+    puts "version #{Tap::VERSION} -- #{Tap::WEBSITE}"
+    exit
+  else
+    queues = env.build(ARGV) do |args|
+      warn "warning: implict rake for [#{args.join(' ')}]"
+      Tap::Tasks::Rake
+    end
+    ARGV.clear
+
+    if queues.empty?
+      puts "no task specified"
+      exit
+    end
+  
+    env.set_signals
+    env.run(queues)
+  end
+  
+rescue
+  env.handle_error($!)
+end
+
+exit(0)

data/bin/tap

@@ -1,4 +1,4 @@
-#!/usr/local/bin/ruby
+#!/usr/bin/env ruby
 # usage: tap <command> {options} [args]
 #
 # examples:
@@ -6,19 +6,19 @@
 #   tap run taskname --option input      # runs the 'taskname' task
 #
 # help:
-#   tap help                             # prints this help
+#   tap --help                           # prints this help
 #   tap command --help                   # prints help for 'command'
 #
 
 require "#{File.dirname(__FILE__)}/../lib/tap.rb"
-require 'tap/exe'
 
 # setup the environment
 begin
   
+  # handle super options
   $DEBUG = true if ARGV.delete('-d-')
   env = Tap::Exe.instantiate
-
+  
 rescue(Tap::Env::ConfigError)
   # catch errors and exit gracefully
   # (errors usu from gem loading errors)
@@ -58,11 +58,8 @@ end
 
 begin
   env.activate
-  env.run do
-    # give some help
-    require 'tap/support/command_line'
-    
-    puts Tap::Support::CommandLine.usage(__FILE__)
+  env.execute(ARGV) do
+    puts Tap::Support::Lazydoc.usage(__FILE__)
     puts
     puts "available commands:"
     puts env.summarize(:commands)

data/cgi/run.rb

@@ -0,0 +1,67 @@
+# ::summary
+# runs a task
+#
+# ::description
+# 
+############################
+require 'cgi'
+require "#{File.dirname(__FILE__)}/../vendor/url_encoded_pair_parser"
+
+env = Tap::Env.instance
+
+module Tap
+  module Support
+    module Server
+      module_function
+      
+      def pair_parse(params)
+        pairs = {}
+        params.each_pair do |key, values|
+          key = key.chomp("%w") if key =~ /%w$/
+
+          slot = pairs[key] ||= []
+          values.each do |value|
+            value = value.respond_to?(:read) ? value.read : value
+            if $~ 
+              slot.concat(Shellwords.shellwords(value))
+            else 
+              slot << value
+            end
+          end
+        end
+
+        UrlEncodedPairParser.new(pairs).result   
+      end
+    end
+  end
+end
+
+cgi = CGI.new("html3")  # add HTML generation methods
+cgi.out() do
+  case cgi.request_method
+  when /GET/i
+    env.render('run.erb', :env => env, :tascs => [] )
+    
+  when /POST/i
+    action = cgi.params['action'][0]
+    case action
+    when 'add'
+      index = cgi.params['index'][0].to_i - 1
+      cgi.params['selected_tasks'].collect do |task|
+        index += 1
+        tasc = env.tasks.search(task).constantize
+        env.render('run/task.erb', :tasc => tasc, :index => index )
+      end.join("\n")
+      
+    when 'remove'
+    when 'update'
+    else
+      cgi.pre do
+        argh = Tap::Support::Server.pair_parse(cgi.params)
+        Tap::Support::Schema.parse(argh['schema']).dump.to_yaml
+      end
+    end
+  else 
+    raise ArgumentError, "unhandled request method: #{cgi.request_method}"
+  end
+end

data/cmd/console.rb

@@ -12,7 +12,7 @@ OptionParser.new do |opts|
   opts.separator "options:"
 
   opts.on("-h", "--help", "Show this message") do
-    opts.banner = Tap::Support::CommandLine.usage(__FILE__)
+    opts.banner = Tap::Support::Lazydoc.usage(__FILE__)
     puts opts
     exit
   end

data/cmd/destroy.rb

@@ -3,14 +3,14 @@ require 'tap/generator/destroy'
 
 env = Tap::Env.instance
 
-if ARGV.empty? || ARGV == ['-T']
-  puts env.summarize(:generators) {|const| const.document[const.name]['generator'] }
+if ARGV.empty? || ARGV == ['--help']
+  puts env.summarize(:generators)
   exit
 end
 
 name = ARGV.shift
-const = env.search(:generators, name) or raise "unknown generator: #{name}"
+const = env.generators.search(name) or raise "unknown generator: #{name}"
 
 generator_class = const.constantize
-generator, argv = generator_class.instantiate(ARGV)
+generator, argv = generator_class.parse(ARGV)
 generator.extend(Tap::Generator::Destroy).process(*argv)

data/cmd/generate.rb

@@ -3,14 +3,14 @@ require 'tap/generator/generate'
 
 env = Tap::Env.instance
 
-if ARGV.empty? || ARGV == ['-T']
-  puts env.summarize(:generators) {|const| const.document[const.name]['generator'] }
+if ARGV.empty? || ARGV == ['--help']
+  puts env.summarize(:generators)
   exit
 end
 
 name = ARGV.shift
-const = env.search(:generators, name) or raise "unknown generator: #{name}"
+const = env.generators.search(name) or raise "unknown generator: #{name}"
 
 generator_class = const.constantize
-generator, argv = generator_class.instantiate(ARGV)
+generator, argv = generator_class.parse(ARGV)
 generator.extend(Tap::Generator::Generate).process(*argv)

data/cmd/manifest.rb

@@ -1,6 +1,6 @@
 # tap manifest
 #
-# Prints information about each env.
+# Prints information about the current tap environment.
 #
 
 options = {}
@@ -10,85 +10,93 @@ OptionParser.new do |opts|
   opts.separator "options:"
 
   opts.on("-h", "--help", "Show this message") do
-    opts.banner = cmdline.usage(__FILE__)
+    opts.banner = Tap::Support::Lazydoc.usage(__FILE__)
     puts opts
     exit
   end
   
-  opts.on("-e", "--envs_only", "Only list environments") do
-    options[:envs_only] = true
-  end
+  # opts.on("-t", "--tree", "Just print the env tree.") do
+  #   options[:tree] = true
+  # end
   
-  opts.on("-r", "--require FILEPATH", "Require the specified file") do |value|
-    require value
-  end
+  # opts.on("-r", "--require FILEPATH", "Require the specified file") do |value|
+  #   require value
+  # end
   
 end.parse!(ARGV)
 
-# Simply a method to collect and format paths for
-# the specified manifest.
-def collect_map(env, manifest)
-  width = 10
-  map = manifest.minimize.collect do |(key, path)|
-    path = case path
-    when Tap::Support::Constant then path.require_path
-    else path
-    end
-    
-    width = key.length if width < key.length
-    [key, env.root.relative_filepath(:root, path) || path]
-  end.collect do |args|
-    "%-#{width}s (%s)" % args
-  end
-  
-  map.unshift("") unless map.empty?
-  map
+env = Tap::Env.instance
+env_names = {}
+env.manifest(:envs, true).minimize.each do |name, environment|
+  env_names[environment] = name
 end
 
-# Collect remaining args as 
-env = Tap::Env.instance
-envs_manifest = if ARGV.empty?
-  env.manifest(:envs, true).minimize
+filter = case
+when ARGV.empty? then env_names.keys
 else
   ARGV.collect do |name| 
-    entry = env.find(:envs, name, false)
-    raise "could not find an env matching: #{name}" if entry == nil
-    entry
+    unless entry = env.find(:envs, name, false) 
+      raise "could not find an env matching: #{name}"
+    end
+
+    entry[1]
   end
 end
 
-width = 10
-envs_manifest.each {|(env_name, e)| width = env_name.length if width < env_name.length}
-width += 2
+template = %Q{#{'-' * 80}
+<%= (env_name + ':').ljust(width) %> (<%= env.root.root %>)
+<% manifests.each do |manifest_name, entries| %>
+  <%= manifest_name %>
+<%   entries.each do |name, path| %>
+    <%= name.ljust(width-4) %> (<%= path %>)
+<%   end %>
+<% end %>
+}
 
-env.each do |current|
-  env_name, current = envs_manifest.find {|(env_name, e)| e == current }
-  next if env_name == nil
-  
-  puts '-' * 80 unless options[:envs_only]
-  puts "%-#{width}s (%s)" % [env_name + ':', current.root.root]
-  
-  next if options[:envs_only]
+width = 10
+summary = env.inspect(template) do |templater, share|
+  current = templater.env
+  next unless filter.include?(current)
   
   manifest_keys = (Tap::Env.manifests.keys + current.manifests.keys).uniq 
-  manifest_keys.each do |name|
+  manifests = manifest_keys.collect do |name|
     next if name == :envs
+    
     manifest = current.manifest(name, true)
     next if manifest.empty?
     
-    puts "  %-10s %s" % [name, collect_map(current, manifest).join("\n    ")]
+    entries = manifest.minimize.collect do |(entry, path)|
+      path = case path
+      when Tap::Support::Constant then path.require_path
+      else path
+      end
+  
+      width = entry.length if width < entry.length
+      [entry, current.root.relative_filepath(:root, path) || path]
+    end
+    
+    [name, entries]
   end
+  templater.manifests = manifests.compact
+  templater.env_name = env_names[current]
+  
+  width = templater.env_name.length if width < templater.env_name.length
+  share[:width] = width + 2
 end
+puts summary
 
 if ARGV.empty?
-  puts '-' * 80
-  puts
-  env.recursive_each(0, nil) do |current, nesting_depth, last_env|
-    env_name, current = envs_manifest.find {|(env_name, e)| e == current }
-
-    leader = nesting_depth == 0 ? "" : '|   ' * (nesting_depth - 1) + (last_env == current ? "`- " : "|- ")
-    puts("#{leader}#{env_name}")
+  tree = env.recursive_inspect("<%= leader %><%= env_name %> \n", 0, nil) do |templater, share, nesting_depth, last|
+    current = templater.env
+    
+    templater.leader = nesting_depth == 0 ? "" : '|   ' * (nesting_depth - 1) + (last == current ? "`- " : "|- ")
+    templater.env_name = env_names[current]
+    
     [nesting_depth + 1, current.envs[-1]]
   end
+  
+  puts '-' * 80
+  puts
+  puts tree
   puts
 end

data/cmd/run.rb

@@ -7,7 +7,6 @@
 
 env = Tap::Env.instance
 app = Tap::App.instance
-cmdline = Tap::Support::CommandLine
 
 #
 # handle options
@@ -15,14 +14,13 @@ cmdline = Tap::Support::CommandLine
 
 dump = false
 OptionParser.new do |opts|
-  
   opts.separator ""
   opts.separator "configurations:"
   
   Tap::App.configurations.each do |receiver, key, config|
     next if receiver == Tap::Root
     
-    opts.on(*cmdline.configv(config)) do |value|
+    opts.on(*config.to_optparse_argv) do |value|
       app.send(config.writer, value)
     end
   end
@@ -31,96 +29,31 @@ OptionParser.new do |opts|
   opts.separator "options:"
 
   opts.on("-h", "--help", "Show this message") do
-    opts.banner = cmdline.usage(__FILE__)
+    opts.banner = Tap::Support::Lazydoc.usage(__FILE__)
     Tap::App.lazydoc.resolve
     puts opts
     exit
   end
   
   opts.on('-T', '--manifest', 'Print a list of available tasks') do |v|
-    puts env.summarize(:tasks) {|const| const.document[const.name]['manifest'] }
+    puts env.summarize(:tasks)
     exit
   end
   
 end.parse!(ARGV)
 
 #
-# handle options for each specified task
+# build and run the argv
 #
 
-rounds = cmdline.split(ARGV).collect do |argvs|
-  argvs.each do |argv|
-    ARGV.clear  
-    ARGV.concat(argv)
-   
-    unless td = cmdline.shift(ARGV)
-      # warn nil?
-      next
-    end
-
-    # attempt lookup the task class
-    const = env.search(:tasks, td) or raise "unknown task: #{td}"
-    task_class = const.constantize or raise "unknown task: #{td}"
-    
-    # now let the class handle the argv
-    task, argv = task_class.instantiate(ARGV, app)
-    task.enq *argv.collect! {|str| cmdline.parse_yaml(str) }
-  end
-
-  app.queue.clear
-end
+queues = env.build(ARGV)
 ARGV.clear
 
-rounds.delete_if {|round| round.empty? }
-if rounds.empty?
+if queues.empty?
   puts "no task specified"
   exit
 end
 
-#
-# set signals 
-#
-
-# info signal -- Note: some systems do 
-# not support the INFO signal 
-# (windows, fedora, at least)
-signals = Signal.list.keys
-if signals.include?("INFO")
-  Signal.trap("INFO") do
-    puts app.info
-  end
-end
-
-# interuption signal
-if signals.include?("INT")
-  Signal.trap("INT") do
-    puts " interrupted!"
-    # prompt for decision
-    while true
-      print "stop, terminate, exit, or resume? (s/t/e/r):"
-      case gets.strip
-      when /s(top)?/i 
-        app.stop
-        break
-      when /t(erminate)?/i 
-        app.terminate
-        break
-      when /e(xit)?/i 
-        exit
-      when /r(esume)?/i 
-        break
-      else
-        puts "unexpected response..."
-      end
-    end
-  end
-end
-
-#
-# enque tasks and run!
-#
+env.set_signals
+env.run(queues)
 
-rounds.each_with_index do |queue, i|
-  app.queue.concat(queue)
-  app.run
-end