tap 0.18.0 → 0.19.0

data/History

@@ -1,3 +1,24 @@
+== 0.19.0 / 2009-12-05
+
+Significant rework of Tap internals.
+
+* extracted task/join/middleware code to make Api base class
+* some changes to semantics of Minimap; entry_to_minikey is
+  now entry_to_path.  Likewise entries now specify their own
+  path using entry.path rather than entry.minikey
+* added Signals implemented with the Dsl pattern
+
+== 0.18.1
+
+* updated API to allow nodes that do not respond to joins
+* fixed bug in reassigning Sync joins
+* added quick-queue option to run
+* added short options to app configs
+* added blocks to app.log
+* added --help flag for joins, middleware
+* removed recurrent loading hooks from load
+  (see tap-tasks:tap/tasks/stream)
+  
 == 0.18.0 / 2009-06-17
 
 Several updates to simplify Tap.  Most significantly,

data/MIT-LICENSE

@@ -1,19 +1,21 @@
 Copyright (c) 2006-2009, Regents of the University of Colorado.
 
-Permission is hereby granted, free of charge, to any person obtaining a copy of this
-software and associated documentation files (the "Software"), to deal in the Software
-without restriction, including without limitation the rights to use, copy, modify, merge,
-publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
-to whom the Software is furnished to do so, subject to the following conditions:
+Copyright (c) 2009, Simon Chiang.
 
-The above copyright notice and this permission notice shall be included in all copies or
-substantial portions of the Software.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
-HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
-WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
-OTHER DEALINGS IN THE SOFTWARE.
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README

@@ -14,15 +14,10 @@ Tap is the core of the
 {Tap-Suite}[http://tap.rubyforge.org/tap-suite/index.html] 
 which provides a standard library of 
 {tasks}[http://tap.rubyforge.org/tap-tasks/index.html],
-{generators}[http://tap.rubyforge.org/tap-gen/index.html], 
-{test utilities}[http://tap.rubyforge.org/tap-test/index.html], 
-and a
-{server}[http://tap.rubyforge.org/tap-server/index.html] 
-to execute workflows via HTTP.
+{generators}[http://tap.rubyforge.org/tap-gen/index.html], and 
+{test utilities}[http://tap.rubyforge.org/tap-test/index.html].
 
-* {Tutorial}[http://tap.rubyforge.org/tap-suite/files/doc/Tutorial.html], {API}[link:files/doc/API.html], {Structure}[link:files/doc/Class%20Reference.html]
 * Website[http://tap.rubyforge.org]
-* Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
 * Github[http://github.com/bahuvrihi/tap/tree/master]
 * {Google Group}[http://groups.google.com/group/ruby-on-tap]
 
@@ -51,7 +46,7 @@ Tap automatically discovers tasks.
       dump        # the default dump task
       load        # the default load task
       
-And generates documentation.
+And generates command line documentation.
 
   % tap run -- goodnight --help
   Goodnight -- your basic goodnight moon task
@@ -65,24 +60,24 @@ And generates documentation.
 
   options:
           --help                       Print this help
-          --name NAME                  Specifies the task name
+          --enque                      Manually enques self
           --config FILE                Specifies a config file
 
-Tasks are immediately available for use in workflows.
+Tasks are immediately available for use in workflows, and may be configured as
+if they were individual executables. This workflow joins the goodnight task to
+a dump task, which prints the message to stdout.
 
   % tap run -- goodnight moon --: dump
   goodnight moon
 
-Tasks may be configured as if they were individual executables.
-
   % tap run -- goodnight world --message hello --: dump
   hello world
 
 === Workflow Syntax
 
 Workflows are specified on the command line using argument vectors separated
-by option breaks. The vectors define the tasks and modifications to the breaks
-specify joins.
+by option breaks (ie '--'). The argument vectors define the tasks while
+modifications to the breaks specify joins.
 
 A simple sequence.
 
@@ -106,30 +101,18 @@ A merge (note that dump receives the inputs in serial).
   goodnight
   moon
 
-A synchronized merge (the printout is ['goodnight', 'moon'].to_s).
+A synchronized merge (the printout is <tt>['goodnight', 'moon'].to_s</tt>).
 
   % tap run -- load goodnight -- load moon -- dump --[0,1][2].sync
   goodnightmoon
 
-== Known Issues
-
-Tap::Env and Tap::Schema are still in flux and have poor documentation.  Neither should be treated as stable.
-
 == Installation
 
-Tap is available as a gem on RubyForge[http://rubyforge.org/projects/tap].  Use:
+Tap is available as a gem on Gemcutter[http://gemcutter.org/gems/tap].
 
   % gem install tap
 
-Tap requires an updated version of RubyGems[http://docs.rubygems.org/] 
-(>= 1.2.0).  To check the version and update RubyGems:
-
-  % gem --version
-  % gem --update system
-
 == Info 
 
-Copyright (c) 2006-2009, Regents of the University of Colorado.
-Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com], {Biomolecular Structure Program}[http://biomol.uchsc.edu/], {Hansen Lab}[http://hsc-proteomics.uchsc.edu/hansenlab/] 
-Support:: CU Denver School of Medicine Deans Academic Enrichment Fund
-License:: {MIT-Style}[link:files/MIT-LICENSE.html]
+Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com]
+License:: {MIT-Style}[link:files/MIT-LICENSE.html]

data/bin/tap

@@ -15,11 +15,11 @@
 require "#{File.dirname(__FILE__)}/../lib/tap.rb"
 
 #
-# setup the environment
+# setup the application
 #
 
 begin
-  exe = Tap::Exe.setup
+  app = Tap::App.setup
 rescue(Tap::Env::ConfigError)
   # catch errors and exit gracefully
   # (errors usu from gem loading errors)
@@ -28,40 +28,35 @@ rescue(Tap::Env::ConfigError)
 end
 
 #
-# launch tap
+# launch the command
 #
 
-exe.activate
-exe.launch(ARGV) do
-  template = %Q{<% unless entries.empty? || count <= 1 %>
+commands = app.env.manifest do |env|
+  env.root.glob(:cmd, "**/*.rb")
+end
+
+case command = ARGV.shift.to_s  
+when '', '--help'
+  template = %Q{<% unless minimap.empty? || count <= 1 %>
 <%= env_key %>:
 <% end %>
-<% entries.each do |key, path| %>
+<% minimap.each do |key, path| %>
   <%= key.ljust(width) %>
 <% end %>}
 
-  commands = exe.manifest(:command).inspect(template,
-    :count => 0,
-    :width => 10
-  ) do |templater, globals|
-    entries = templater.manifest.minimap
-    templater.entries = entries
-    
-    # determine width, count
-    width = globals[:width]
-    entries.each do |key, path|
-      width = key.length if width < key.length
-    end
-    globals[:width] = width
-    globals[:count] += 1 unless entries.empty?
-  end
-  
   puts Lazydoc.usage(__FILE__)
   puts
   puts "available commands:"
-  puts commands
+  puts commands.summarize(template)
   puts
   puts "version #{Tap::VERSION} -- #{Tap::WEBSITE}"
+else
+  if path = commands.seek(command)
+    load path
+  else
+    puts "Unknown command: '#{command}'"
+    puts "Type 'tap --help' for usage information."
+  end
 end
 
 exit(0)

data/cmd/console.rb

@@ -5,7 +5,7 @@
 # environment through 'env'.  For example:
 #
 #   % tap console
-#   >> env[:task][:dump]
+#   >> app.env[:dump]
 #   => Tap::Tasks::Dump
 #   >> app.info
 #   => "state: 0 (READY) queue: 0"
@@ -29,17 +29,6 @@ def app
   @app ||= Tap::App.instance
 end
 
-def env
-  @env ||= Tap::Env.instance
-end
-
-def run(cmd, reset=true)
-  app.reset if reset
-  schema = Tap::Schema.parse(cmd)
-  env.run(schema, app)
-  nil
-end
-
 IRB.start
 
 # Handles a bug in IRB that causes exit to throw :IRB_EXIT

data/cmd/manifest.rb

@@ -24,15 +24,15 @@ template = %Q{<% unless manifests.empty? %>
 <%= (env_key + ':').ljust(width) %> (<%= env.root.root %>)
 <% manifests.each do |type, entries| %>
   <%= type %>
-<%   entries.each do |key, value| %>
-    <%= key.ljust(width-4) %> (<%= value %>)
+<%   entries.each do |key, paths| %>
+    <%= key.ljust(width) %> (<%= paths.join(', ') %>)
 <%   end %>
 <% end %>
 <% end %>
 }
 
 # filter envs to manifest
-env = Tap::Env.instance
+env = Tap::App.instance.env
 env_keys = env.minihash(true)
 filter = if ARGV.empty?
   env_keys.keys
@@ -43,6 +43,7 @@ else
 end
 
 # build the summary
+constants = env.constants
 summary = env.inspect(template, :width => 10) do |templater, globals|
   current = templater.env
   manifests = []
@@ -56,26 +57,20 @@ summary = env.inspect(template, :width => 10) do |templater, globals|
   width = env_key.length if width < env_key.length
   
   # build up the entries for each type of resource
-  current.registry(true).to_a.sort_by do |(type, entries)|
-    type.to_s
-  end.each do |type, entries|
-    next if entries.empty?
+  types = {}
+  constants.entries(current).minimap.each do |key, const|
+    paths = const.require_paths.collect do |path|
+      current.root.relative_path(:root, path) || path
+    end.uniq
     
-    entries.extend(Tap::Env::Minimap)
-    entries = entries.minimap.collect do |key, entry|
-      path = if entry.kind_of?(Tap::Env::Constant)
-        entry.require_path
-      else
-        entry
-      end
-  
-      width = key.length if width < key.length
-      [key, current.root.relative_path(:root, path) || path]
-    end
+    width = key.length if width < key.length
     
-    manifests << [type, entries]
+    const.types.keys.each do |type|
+      (types[type] ||= []) << [key, paths]
+    end
   end
   
+  manifests.concat types.to_a.sort_by {|type, minimap| type }
   globals[:width] = width
 end
 puts summary

data/cmd/run.rb

@@ -6,128 +6,138 @@
 #   tap run -- load hello --: dump     Say hello
 #
 
-env = Tap::Env.instance
-app = Tap::App.new
+app = Tap::App.instance
 
-#
-# parse argv
-#
-
-# separate out argv schema
-argv = []
-while !ARGV.empty? && ARGV[0] !~ Tap::Schema::Parser::BREAK
-  argv << ARGV.shift
-end
-schema = ARGV.empty? ? nil : Tap::Schema.parse(ARGV)
-ARGV.replace(argv)
-
-# parse options
-mode = :run
-ConfigParser.new(app.config) do |opts|
-  opts.separator ""
-  opts.separator "configurations:"
+opts = {}
+parser = ConfigParser.bind(app.config) do |psr|
+  psr.separator ""
+  psr.separator "configurations:"
   
-  root_keys = Tap::Root.configurations.keys
-  Tap::App.configurations.each_pair do |key, config|
-    next if root_keys.include?(key)
-    opts.define(key, config.default, config.attributes)
-  end
- 
-  opts.separator ""
-  opts.separator "options:"
+  psr.add Tap::App.configurations
+  
+  psr.separator ""
+  psr.separator "options:"
   
-  opts.on("-h", "--help", "Show this message") do
-    Tap::App.lazydoc.resolve
+  psr.on("-h", "--help", "Show this message") do
     puts Lazydoc.usage(__FILE__)
-    puts opts
+    puts psr
     exit(0)
   end
   
-  opts.on('-s', '--schema FILE', 'Use the specifed schema') do |file|
-    if schema
-      puts "An inline schema cannot be specified with a file schema."
-      exit(0)
-    end
-    
-    schema = Tap::Schema.load_file(file)
+  psr.on('-s', '--serialize', 'Serialize the workflow') do
+    opts[:serialize] = true
   end
   
-  opts.on('-p', '--preview', 'Print the schema as YAML') do
-    mode = :preview
-  end
-  
-  opts.on('-t', '--manifest', 'Print a list of available resources') do
-    tasks = env.manifest(:task)
-    tasks_found = !tasks.all_empty?
+  psr.on('-t', '--manifest', 'Print a list of available resources') do
+    constants = app.env.constants
     
-    joins = env.manifest(:join)
-    joins_found = !joins.all_empty?
-    
-    middleware = env.manifest(:middleware)
-    middleware_found = !middleware.all_empty?
+    tasks, joins, middleware = %w{task join middleware}.collect do |type|
+      constants.summarize do |constant|
+        constant.types[type]
+      end
+    end
     
-    if tasks_found 
-      puts "=== tasks ===" if middleware_found || joins_found
-      puts tasks.summarize
+    unless tasks.empty? 
+      puts "=== tasks ===" unless middleware.empty? && joins.empty?
+      puts tasks
     end
 
-    if joins_found
-      puts "=== joins ===" if tasks_found || middleware_found
-      puts joins.summarize
+    unless joins.empty?
+      puts "=== joins ===" unless tasks.empty? && middleware.empty?
+      puts joins
     end
     
-    if middleware_found
-      puts "=== middleware ===" if tasks_found || joins_found
-      puts middleware.summarize
+    unless middleware.empty?
+      puts "=== middleware ===" unless tasks.empty? && joins.empty?
+      puts middleware
     end
     
     exit(0)
   end
   
-  opts.on('-T', '--tasks', 'Print a list of available tasks') do
-    puts env.manifest(:task).summarize
+  psr.on('-T', '--tasks', 'Print a list of available tasks') do
+    constants = app.env.constants
+    tasks = constants.summarize do |constant|
+      constant.types['task']
+    end
+    
+    puts tasks
     exit(0)
   end
   
-end.parse!(ARGV, :clear_config => false, :add_defaults => false)
+  psr.on('-u', '--quick-queue', 'Removes thread-safety from queue') do
+    mod = Module.new do
+      def synchronize
+        yield
+      end
+    end
+    app.queue.extend(mod)
+  end
+end
 
 #
 # build and run
 #
 
-unless schema
-  msg = "No schema specified"
-
-  unless ARGV.empty?
-    args = ARGV[0, 3].join(' ') + (argv.length > 3 ? ' ...' : '')
-    msg = "#{msg} (did you mean 'tap run -- #{args}'?)"
+# Traps interrupt the normal flow of the program and so I assume thread safety
+# is an issue (ex if the INT occurs during an enque and a signal specifies
+# another enque). A safer way to go is to enque the prompt... when the prompt
+# is executed the app won't be be doing anything else so thread safety
+# shouldn't be an issue.
+Signal.trap('INT') do
+  puts
+  puts "Interrupt!  Signals from an interruption are not thread-safe."
+  
+  require 'tap/prompt'
+  prompt = Tap::Prompt.new
+  call_prompt = true
+  3.times do
+    print "Wait for thread-safe break? (y/n): "
+    
+    case gets.strip
+    when /^y(es)?$/i
+      puts "waiting for break..."
+      app.queue.unshift(prompt, [])
+      call_prompt = false
+      break
+      
+    when /^no?$/i
+      break
+    end
+  end
+  
+  if call_prompt
+    prompt.call
   end
-
-  puts msg
-  exit(0)
 end
 
 begin
-  app.build(schema, :resources => env)
-  ARGV.replace(argv)
-  
-  case mode
-  when :run
-    Tap::Exe.set_signals(app)
-    app.run
-  when :preview
-    app.to_schema do |type, resources|
-      resources.each do |resource|
-        const_name = resource.delete(:class).to_s
-        resource[:id] = env.reverse_seek(type, false) do |const|
-          const.const_name == const_name
-        end
+  loop do
+    break if ARGV.empty?
+    parser.scan(ARGV, 
+      :option_break => Tap::Parser::BREAK,
+      :keep_break => true
+    ) do |path|
+      YAML.load_file(path).each do |spec|
+        app.call(spec)
       end
-    end.dump($stdout)
+    end
+
+    break if ARGV.empty?
+    ARGV.replace app.call('sig' => 'parse', 'args' => ARGV)
   end
   
+  if opts[:serialize]
+    YAML.dump(app.serialize, $stdout)
+    exit(0)
+  end
+  
+  opts = nil
+  parser = nil
+  app.run
+  
 rescue
-  raise if $DEBUG
+  raise if app.debug?
   puts $!.message
   exit(1)
 end