tap 0.10.1 → 0.11.0

data/doc/Syntax Reference

@@ -0,0 +1,290 @@
+= Syntax Reference
+
+Tap uses several domain-specific languages to declare tasks and workflows.  This is a reference for:
+
+* task declarations
+* class definitions
+* workflows
+
+== Task Declarations
+
+A task declaration:
+
+  # name::           the name of the task, either alone or as the
+  #                  key of a {key => dependencies} hash
+  # dependencies::   an array of task names (optional)
+  # arg_names::      an array of argument names (optional)
+  # configurations:: a hash of {key => default} pairs (optional)
+
+  task({<name> => [<dependencies...>]}, <arg_names...>, {<configs>}) do |task, args|
+    # arguments are available through args:
+    args.arg_name
+    
+    # configurations are available through task
+    task.key
+  end
+
+A namespace declaration:
+
+  namespace(<name>) { task ... }
+ 
+Simple documentation:
+ 
+  desc "description"
+  task ...
+  
+Extended documentation:
+  
+  # ::desc description
+  # Extended documentation may span multiple lines, and 
+  # supports
+  #
+  #   code indentation
+  #   like this.
+  #
+  # Lines are justified and wrapped on the command line.
+  task ...
+
+Pains were taken to make task declarations for rap work the similar to rake tasks.  In both cases (noting that in general, beyond rap, task classes are not limited in these ways):
+
+* tasks are singleton instances that may be extended across multiple declarations
+* tasks do not pass inputs from one task to the next
+* tasks only execute once
+
+A few syntactical differences between rake and rap must to be noted, as they can cause errors if you try to migrate rake tasks to tap.  Tap declarations.
+
+==== no needs
+
+Tasks do not support :needs as a way to specify dependencies.  For instance, this will declare a ':needs' configuration with the default value ':another', not a task which depends on another:
+
+  Tap.task :name, :needs => :another
+
+==== immediate namespace lookup
+
+Within a namespace, task will look for a literal match to the dependency first.  If it doesn't find one, it will declare it within the namespace; one way or the other the dependency is resolved immediately.  Hence, in this example the nested task depends on the non-nested task.  
+
+  task :outer { print 'non-nested' }
+  namespace :nest do
+    task :inner => :outer { puts 'was executed' }
+    task :outer { print 'nested' }
+  end
+  
+By contrast, rake waits to resolve dependencies and so will always use a match within a namespace in preference to a literal match.  For these tasks rap and rake produce different results:
+
+  % rap nest/inner
+  non-nested was executed
+  % rake nest:inner
+  nested was executed
+
+== Task Classes
+
+This is a verbose prototype for Tap::Task subclasses:
+
+  # <ClassName>::manifest summary description
+  #
+  # Extended documentation...
+  #
+  class ClassName < Tap::Task
+    
+    # Sets up a configuration and makes the 'key' and 'key=' accessors.
+    # 
+    # This documentation appears in static config files, and in RDoc.
+    #
+    config :key, 'default value' do |value|  # config summary
+      "the config is set to the return value"
+    end 
+    
+    # Subclasses BaseClass with the specified configs, using the block
+    # to override process.
+    #
+    # Also creates the methods 'name', 'name_config', 'name_config='.  
+    # The first will access an instance-specific instance of the 
+    # subclass, the other two are accessors for the instance configs.
+    # 
+    # Primarily used in conjunction with workflow.
+    #
+    define :name, <BaseClass>, {<configs>} do |*args|
+      # this is the process block
+    end
+    
+    # Causes each instance to depend on DependencyClass.instance.
+    #
+    # Also defines a reader 'name' which will access the results
+    # of the dependency.
+    depends_on :name, <DependencyClass>
+    
+    def process(*args)
+      # the method defining what this task does 
+    end
+    
+    protected
+
+    def workflow
+      # a hook to setup joins between various tasks used
+      # by the current instance.  workflow is called
+      # during initialize
+    end
+
+    def before_execute() 
+      # a hook to execute code before process
+      # (only works in workflows)
+    end
+  
+    def after_execute()
+      # a hook to execute code after process
+      # (only works in workflows)
+    end
+
+    def on_execute_error(err)
+      # a hook to handle errors during process
+      # (only works in workflows)
+    end
+  end
+
+== Workflows
+
+The tap workflow syntax is designed to specify an arbitrary number of tasks, inputs, configurations, and joins.  The syntax works both as text and as YAML; 'tap run' and rap both use it to specify which tasks to execute.  Basically, the syntax uses double-dash delimiters to separate task vectors and modified delimiters to specify joins for the tasks.  
+
+These both specify three tasks (x,y,z) joined in a sequence:
+
+  % rap x --: y --: z
+  % rap x -- y -- z --0:1:2
+  
+The modified delimiters use numbers to indicate which tasks participate in a join and punctuation to indicate the join type.  In the first example, the numbers are implicitly added for the preceding and following task.  A variety of joins are supported:
+
+  delimiter  function         syntax                 example        meaning 
+  --         delimiter                               a -- b -- c    enques/configures tasks a, b, c
+  --*        dependency                              --* a          enques dependency instance
+  --+        round                                   --+ a --++ b   enques a to round one, b to round two
+  --+[]      round shorthand  +round[targets]        --+2[0,1]      enques (a,b) to round two
+  
+  --:        sequence         source:target          --0:1:2        sequence a, b, c
+  --[]       fork             source[targets...]     --0[1,2]       fork a to (b,c)
+  --{}       merge            target{sources...}     --2{0,1}       merge (a,b) to c
+  --()       sync_merge       target(sources...)     --2(0,1)       synchronize merge (a,b) to c
+
+Inputs may be specified between delimiters:
+
+  % rap x alpha beta --: y gamma --: z delta
+
+As may be configurations, in a variety of formats:
+
+  % rap x alpha beta -k --: y gamma --key value --: z delta --key=value
+  
+These are the corresponding task vectors:
+
+  ['x', 'alpha', 'beta', '-k']
+  ['y', 'gamma', '--key', 'value']
+  ['z', 'delta', '--key=value']
+
+This is how it would look as YAML:
+
+  - - x
+    - alpha
+    - beta
+    - -k
+  - - y
+    - gamma
+    - key: value
+  - - z
+    - delta
+    - --key=value
+  - 0:1:2
+
+The task vectors are converted into a task and an argument vector by first looking up a task class and then calling TaskClass.parse.  The parse method returns a configured instance of the task and an argv to be executed by the task.  Once the task are instantiated, instances are joined and enqued to Tap::App for execution (see Tap::Support::Schema#build).
+
+==== Class Lookup
+
+Tap looks up classes via Tap::Env.  Tap can find tasks from multiple environments; by adding in environments for gems, Tap can find tasks within a gem.
+
+During lookup, classes are treated like filepaths which match from the basename up:
+
+  class             lookup              matched by
+  Sample::Task      sample/task         task, sample/task
+  A::Nested::Task   a/nested/task       task, nested/task, a/nested/task
+
+In the event of a name conflict, the path of the environment may also be specified.  It seems like this could get confusing, but the tap executable produces manifests that specify the minimal path required to uniquely identify a class.  Fragments of the minimized paths will be resolved in order from top to bottom within the specified environment.  For example: 
+
+  % tap run -T
+  one:
+    sample/task      # some sample task
+    another/task     # another task
+  two:
+    sample/task      # a conflicting sample task
+
+Runs the 'one' sample/task:
+
+  % tap run -- sample/task
+  % tap run -- one:sample/task
+
+Runs the 'two' sample/task:
+
+  % tap run -- two:sample/task
+
+Runs the 'one' sample/task:
+
+  % tap run -- task
+
+Runs the 'two' sample/task:
+
+  % tap run -- two:task
+
+Runs another/task:
+
+  % tap run -- another/task
+  % tap run -- one:another/task
+
+Notice that the full minimized path ('another/task') is required because simply using 'task' will be matched to sample/task.  The order of tasks is obviously not alphabetical -- rather it corresponds to the order in which Tap::Env discovers the tasks.
+
+==== Dependencies (--*)
+
+Normally each task vector specifies a new instance of a task.  This specifies three instance of the 'a' task.
+
+  % rap -- a -- a -- a
+  
+When you want to specify a 'singleton' instance of a task, the instance specified by TaskClass.instance, use the dependency delimiter '--*'.  
+
+  % rap --* a
+  
+The dependency instance will be configured and enqued with the arguments.  Dependency instances will be executed first, even if they are specified at the end of a workflow, they cannot participate in workflows, and any single dependency may only be specified once in a workflow.  These are equivalent:
+
+  % rap --* a -- b -- c
+  % rap b -- c --* a
+
+While these raise errors:
+
+  % rap --* a --* a
+  % rap --* a --: b
+
+==== Rounds (--+)
+
+The workflow syntax allows multiple execution rounds to be specified  All tasks in a round are run to completion before the next round begins.  Rounds are specified by adding '+' characters after the double-dash break.
+
+  % tap run -- round_one_task --+ round_two_task
+
+Tasks may be added to rounds in any order and may use the shorthand for multiple tasks.  These are equivalents:
+
+  % tap run -- a --+ b --+ c --++ d
+  % tap run --+ b --++ d -- a --+ c
+  % tap run -- a -- b -- c -- d --+1[1,2] --+2[3]
+
+Rounds are particularly useful for dump tasks; add a dump task as a final round to capture all results from previous rounds:
+
+  % tap run -- task -- task --+ dump
+
+==== Notes for rap
+
+Specifying tasks for rap is no different than 'tap run'.  The only exception occurs when the task can't be found.  Unknown tasks are implicitly run as a Tap::Tasks::Rake task, which causes the task to be run as if by rake.  These are the same:
+
+Run a rake task with inputs and an ENV config:
+
+  % rake task_name[1,2,3] key=value
+
+Explicitly run a Tap::Tasks::Rake task using the same inputs:
+
+  % rap rake task_name[1,2,3] key=value
+  
+Implicitly run the Tap::Tasks::Rake task:
+
+  % rap task_name[1,2,3] key=value
+