tap 0.9.1 → 0.10.0

data/doc/Command Reference

@@ -0,0 +1,153 @@
+= Command Reference
+
+The <tt>tap</tt> executable is the gateway for the execution of tasks.  To get help for tap, type:
+
+  % tap --help
+  
+<tt>tap</tt> sets up the execution environment from <tt>tap.yml</tt>, if it exists, and passes control to the specified command.  Command help can be obtained using:
+
+  % tap [command] --help
+
+This reference covers the commands: run, console, generate, destroy
+
+=== tap run
+
+Run configures, enqueues, and executes tasks.  Run has a rich syntax allowing the specification of any number of tasks with configurations and inputs, but simplifies under most circumstances.  Several examples illustrate the key points:
+
+  % tap run sample/task 
+  % tap run -- sample/task --key=value input_one -- another/task input_two
+  
+The second statement specifies two tasks with inputs, and specifies a configuration for sample/task.  As can be seen, run separates tasks using a double-dash, the standard option break.  Options for the run command can be specified before the first option break.
+
+  % tap run --debug -- sample/task --key=value
+  
+Here run receives the <tt>--debug</tt> option and sample/task receives the <tt>--key=value</tt> option.  NOTE: it's always a good idea to include the first option break to formally signify when configuration of run stops.  Otherwise you may be confused when the following commands both produce the help for run:
+
+  % tap run --help
+  % tap run sample/task --help
+
+Inputs work the same way.  For example:
+
+  % tap run -- sample/task --key=value one -- another/task two three
+  
+Specifies the following:
+
+  t1 = Sample::Task.new(:key => 'value')
+  t1.enq('one')
+  
+  t2 = Another::Task.new
+  t2.enq('two', 'three') 
+  
+Any number of tasks, configurations, and inputs may be specified in this way.
+
+==== Task Lookup
+
+Tap can find and run tasks from multiple environments, for instance from multiple gems.  Tap provides a compact way to specify which task to run in the event of a name conflict.  The actual process involves minimizing the path to the environment and the relative path to the task file, but from the command line all of the details are hidden:
+
+  % tap run -T
+  one:
+    sample/task      # some sample task
+    another/task     # another task
+  two:
+    sample/task      # a conflicting sample task
+
+In this situation, these commands run the 'one' sample/task:
+
+  % tap run -- sample/task
+  % tap run -- one:sample/task
+
+While this runs the 'two' sample/task:
+
+  % tap run -- two::sample/task
+
+Additionally, base-fragments of the minimized paths can be specified (if laziness strikes); they will be resolved in order from top to bottom within the specified environment.
+
+Runs the 'one' sample/task:
+
+  % tap run -- task
+
+Runs the 'two' sample/task:
+
+  % tap run -- two::task
+
+The full minimized path must be specified for another/task:
+
+  % tap run -- another/task
+  % tap run -- one:another/task
+
+==== Rounds
+
+Run allows specification of a number of rounds of tasks.  All tasks in a round are run to completion before the next round begins.  Rounds are specified by adding '+' characters after the double-dash task break.
+
+  % tap run -- round_one_task --+ round_two_task
+
+Tasks may be added to rounds in any order.  This is equivalent to the last:
+
+  % tap run --+ round_two_task -- round_one_task 
+
+Rounds are particularly useful for dump tasks; add a dump task as a final round to capture all results from previous rounds:
+
+  % tap run -- task -- task --+ dump
+
+==== YAML inputs
+
+Non-string inputs can be provided through run.  If an input begins with "---\n" then it is loaded as YAML into an object before being passed to a task. The syntax can be a lot to type, but is very handy to have around.  The following enques sample/task with a hash input, {'number' => 1}.  
+
+On *nix, just hit enter to get the next line:
+
+  % tap run -- sample/task '---
+  > number: 1'
+
+On Windows, you need to pull some tricks to get newlines into your argument. Cleverly use a caret to ignore the next line feed:
+
+  % tap run -- sample/task '---^
+  More?
+  More? number: 1'
+  
+Notice that pressing enter <em>every other line</em> is what actually puts the "\n" into the  parameter.  Keep using carets to enter more lines. The syntax on Windows isn't exactly pretty, and it only works with a caveat: the latest execution script generated by rubygems (tap.bat) re-processes inputs to tap before executing the command for real.  I haven't found a workaround short of invoking tap from ruby itself:
+
+  % ruby C:/ruby/bin/tap run -- sample/task '---^
+  More?
+  More? number: 1'
+
+In these cases, consider putting the inputs into a file and load them to the command line with the <tt>--use</tt> option:
+
+  [inputs.yml]
+  number: 1
+
+  % tap run -- sample/task --use inputs.yml
+
+=== tap console
+
+Console opens an irb session with Tap loaded and configured using <tt>tap.yml</tt>.  Console defines variables 'app' and 'env' referencing Tap::App.instance and Tap::Env.instance, for easy access.
+
+  % tap console
+  irb(main):001:0> app.log(:hello)
+    I[17:18:53]              hello
+  => true
+  irb(main):002:0>
+
+=== tap generate/destroy
+
+Generate and destory launch generator scripts, similar to those in Rails.  By default Tap provides generators for:
+
+{root}[link:classes/Tap/Generator/Generators/RootGenerator.html]:: the basic Tap directory structure
+{task}[link:classes/Tap/Generator/Generators/TaskGenerator.html]:: a Tap::Task and test
+{file_task}[link:classes/Tap/Generator/Generators/FileTaskGenerator.html]:: a Tap::FileTask and test
+{config}[link:classes/Tap/Generator/Generators/ConfigGenerator.html]:: a yaml config file for the specified task 
+{command}[link:classes/Tap/Generator/Generators/CommandGenerator.html]:: a new command
+
+--
+{generator}[link:classes/Tap/Generator/GeneratorseneratorGenerator.html]:: a new generator
+{workflow}[link:classes/Tap/Generator/Generators/Workflow/WorkflowGenerator.html]:: a Tap::Workflow and test
+++
+
+For example:
+
+  % tap generate root .
+  % tap generate task sample_task
+  % tap generate config sample_task
+  
+  % tap destroy config sample_task
+  % tap destroy task sample_task
+  % tap destroy root .

data/doc/Tutorial

@@ -0,0 +1,237 @@
+= Quick Start Tutorial
+
+=== Basic Usage
+
+Begin by creating a tap directory structure:
+
+  % tap generate root sample
+  % cd sample
+  
+Comes with a task:
+
+  % tap run -T
+  sample:
+    goodnight   # your basic goodnight moon task
+  tap:
+    dump        # the default dump task
+    rake        # run rake tasks
+
+Test the task:
+ 
+  % rake test 
+  
+Get help for the task:
+
+  % tap run -- goodnight --help
+  Goodnight -- your basic goodnight moon task
+  --------------------------------------------------------------------------------
+    Prints the input with a configurable message.
+  --------------------------------------------------------------------------------
+  usage: tap run -- goodnight INPUT
+
+  configurations:
+          --message MESSAGE
+
+  options:
+      -h, --help                       Print this help
+          --name NAME                  Specify a name
+          --use FILE                   Loads inputs from file
+
+Run the task:
+
+  % tap run -- goodnight moon
+    I[23:22:19]          goodnight moon
+    
+Run the task, setting the 'message' configuration:
+  
+  % tap run -- goodnight moon --message hello
+    I[23:22:46]              hello moon
+  
+Run multiple tasks, or in this case the same task twice:
+
+  % tap run -- goodnight moon -- goodnight opus
+    I[23:23:06]          goodnight moon
+    I[23:23:06]          goodnight opus
+
+Same as above, but now dump the results to a file:
+
+  % tap run -- goodnight moon -- goodnight opus --+ dump output.yml
+    I[23:23:26]          goodnight moon
+    I[23:23:26]          goodnight opus
+    I[23:23:26]               dump output.yml
+
+The dump file contents look like this:
+
+  # audit:
+  # o-[] "opus"
+  # o-[goodnight] "goodnight opus"
+  # 
+  # o-[] "moon"
+  # o-[goodnight] "goodnight moon"
+  # 
+  # date: 2008-08-05 23:23:26
+  --- 
+  goodnight (2769410): 
+  - goodnight opus
+  goodnight (2780180): 
+  - goodnight moon
+
+The comments at the beginning are an audit trace of the run.  In this case two 
+separate tasks were run sequentially, hence you can see each task, the task inputs, 
+and the task results as separate units.  A YAML hash follows the audit with the 
+aggregated task results, keyed by the task name and object id.  Since the results 
+are represented as a hash, the order of the tasks sometimes gets scrambled, as in 
+this case.
+
+=== Task Declaration
+
+Tap provides a declaration syntax a-la rake, accessible through the Tap module to 
+prevent conflicts with rake.  Declarations can get put in any <tt>.rb</tt> file 
+under the lib directory or in <tt>tapfile.rb</tt>.
+
+  [tapfile.rb]
+  # Goodnight::manifest your basic goodnight moon task
+  # Prints the input with a configurable message.
+
+  Tap.task('goodnight', :message => 'goodnight') do |task, name|
+     task.log task.message, name
+     "#{task.message} #{name}"
+  end
+
+The declaration makes a task class based on the name (ie namespaces are naturally
+supported by names like <tt>'nested/task'</tt>).  The classes are ready for use in 
+scripts:
+
+  require 'tapfile'
+  Goodnight.new.process('moon')       # => 'goodnight moon'
+
+And from the command line, as above.
+
+=== Task Definition
+
+Sometimes you need more than a block to define a task.  Generate a task:
+
+  % tap generate task hello
+
+Navigate to and open the <tt>lib/hello.rb</tt> file.  Inside you can see the class 
+definition.  Notice configurations are mapped to methods, and the task documentation 
+is located in the comments.  Let's change it up a bit:
+
+  [lib/hello.rb]
+  # Hello::manifest your basic hello world task
+  #
+  # Prints hello to a number of things with a configurable, 
+  # reversible message.
+  #
+  class Hello < Tap::Task
+
+    config :message, 'hello'               # a greeting
+    config :reverse, false, &c.flag        # reverses the message
+
+    def process(*names)
+      names.collect do |name|
+        log(reverse ? message.reverse : message, name)
+        "#{message} #{name}"
+      end
+    end
+  end
+
+The new configurations and documentation are immediately available:
+
+  % tap run -- hello --help
+  Hello -- your basic hello world task
+  --------------------------------------------------------------------------------
+    Prints hello to a number of things with a configurable, reversible message.
+  --------------------------------------------------------------------------------
+  usage: tap run -- hello NAMES...
+
+  configurations:
+          --message MESSAGE            a greeting
+          --reverse                    reverses the message
+
+  options:
+      -h, --help                       Print this help
+          --name NAME                  Specify a name
+          --use FILE                   Loads inputs from file
+
+And the task is ready to go:
+
+  % tap run -- hello moon lamp 'little toy boat'
+    I[23:29:26]              hello moon
+    I[23:29:26]              hello lamp
+    I[23:29:26]              hello little toy boat
+
+  % tap run -- hello mittens --reverse
+    I[23:29:53]              olleh mittens
+
+Now lets use the previous results; they get loaded and added to the end of the inputs:
+
+  % tap run -- hello --use output.yml
+    I[23:31:32]              hello goodnight moon
+    I[23:31:32]              hello goodnight opus
+  
+=== Config Files
+
+So say you wanted static configs for a task.  Make a configuration file:
+
+  % tap generate config goodnight
+        
+Set the configurations here and they get used by the task:
+
+  [config/goodnight.yml]
+  ###############################################################################
+  # Goodnight configuration
+  ###############################################################################
+
+  message: good evening
+  
+As can be seen here:
+
+  % tap run -- goodnight moon
+    I[23:40:39]       good evening moon
+
+If you need to run a task with multiple sets of configurations, simply define an 
+array of configurations in the config file:
+  
+  [config/goodnight.yml]
+  - message: good afternoon
+  - message: good evening
+  - message: goodnight
+
+  % tap run -- goodnight moon
+    I[23:42:46]     good afternoon moon
+    I[23:42:46]       good evening moon
+    I[23:42:46]          goodnight moon
+
+The --name option sets the config file used:
+
+  % tap run -- goodnight moon --name no_config_file
+    I[23:43:20]          goodnight moon
+
+=== Tap Configuration
+
+Tap itself is highly configurable.  Say you think the run syntax is unnecessarily 
+verbose; you can make command aliases to shorten it.  Open the <tt>tap.yml</tt> 
+file in your root directory and set the following:
+
+  [tap.yml]
+  alias:
+    --: [run, --]
+    -T: [run, -T]
+
+Now:
+
+  % tap -- hello world
+    I[23:43:59]              hello world
+
+  % tap -T
+  sample:
+    goodnight   # your basic goodnight moon task
+    hello       # your basic hello world task
+  tap:
+    dump        # the default dump task
+    rake        # run rake tasks
+
+Global configurations can go in the <tt>~/.tap.yml</tt> file.  Using configurations,
+you can specify directory aliases, options, gems, and even additional paths to load 
+as if they were gems.

data/lib/tap.rb

@@ -1,69 +1,30 @@
 require 'rubygems'
 
 require 'yaml'                   # expensive to load
-require 'logger'
-require 'ostruct'
 require 'thread'
-require 'erb'
-
-autoload(:GetoptLong, 'getoptlong')
-autoload(:PP, "pp")
 
 # Apply version-specific patches
 case RUBY_VERSION
 when /^1.9/
-  $: << File.dirname(__FILE__) + "/tap/patches/ruby19"
+  $: << File.expand_path(File.dirname(__FILE__) + "/tap/patches/ruby19")
   
   # suppresses TDoc warnings
   $DEBUG_RDOC ||= nil 
 end
 
-# Loading activesupport piecemeal like this cuts the tap load time in half.
-gem 'activesupport'
-
-require 'active_support/core_ext/array/extract_options.rb'
-class Array #:nodoc:
-  include ActiveSupport::CoreExtensions::Array::ExtractOptions
-end
-require 'active_support/core_ext/class.rb'
-require 'active_support/core_ext/module.rb'
-require 'active_support/core_ext/symbol.rb'
-require 'active_support/core_ext/string.rb'
-require 'active_support/core_ext/blank.rb'
-require 'active_support/core_ext/hash/keys.rb'
-require 'active_support/dependencies'
-require 'active_support/clean_logger'
-class Hash #:nodoc:
-  include ActiveSupport::CoreExtensions::Hash::Keys
-end
-
-$:.unshift File.dirname(__FILE__)
+$:.unshift File.expand_path(File.dirname(__FILE__))
 
 require 'tap/constants'
-class String # :nodoc:
-  include Tap::Constants
-end
 
-require 'tap/support/aggregator'
-require 'tap/support/audit'
-require 'tap/support/batchable_methods'
-require 'tap/support/batchable'
-require 'tap/support/class_configuration'
-require 'tap/support/command_line_methods'
-require 'tap/support/configurable'
-require 'tap/support/configurable_methods'
-require 'tap/support/executable'
-require 'tap/support/executable_queue'
-require 'tap/support/logger'
-require 'tap/support/run_error'
-require 'tap/support/shell_utils'
-require 'tap/support/validation'
+# require in order...
 require 'tap/env'
 require 'tap/app'
 require 'tap/task'
 require 'tap/file_task'
 require 'tap/workflow'
-require 'tap/dump'
+
+require 'tap/support/declarations'
+Tap.extend Tap::Support::Declarations
 
 # Apply platform-specific patches
 # case RUBY_PLATFORM