t2-server 0.9.3 → 1.0.0

data/CHANGES.rdoc

@@ -1,5 +1,84 @@
 = Changes log for the T2 Ruby Gem
 
+== Version 1.0.0
+
+* Remove deprecated methods.
+* Add read streaming capability to the underlying connection.
+* Add a read method to stream data directly to a file.
+* Stream zip data from server.
+* Update t2-run-workflow so that debug is optional.
+* Stream baclava data from server.
+* Expose streaming capability to output ports.
+* Bring port error handling inline with normal values.
+* Tidy up usage of a mime-type for empty ports.
+* Add methods to test for empty ports or port values.
+* Add output port data streaming functionality.
+* Stream output data directly to a file.
+* Document the credentials classes.
+* Add general stream-to-stream data downloads.
+* Expose stream-to-stream methods in the Run class.
+* Add upload streaming to the underlying connection.
+* Stream uploads of workflows and data.
+* Add streaming examples to the README.
+* Make XPath searching robust to nodes not being found.
+* Read interaction feed URI and query support with API.
+* Read notifications from the interaction feed.
+* Document the Interaction Service additions in the README.
+* Add a bit more compatibility info to the Readme.
+* Clean up run caching code in the Server class.
+* Clean up variables and comments for run creation.
+* Code clarity improvements.
+* Remove duplicated download/streaming code in Run.
+* Remove duplicate URI stripping code from Server.
+* Improve readability of the XML fragments.
+* Add a global complied XPath cache and use for Server.
+* Use global XPath cache for Run class.
+* Add Ruby 2.0.0 as known to work in README.
+* t2-delete-runs: Minor code clean-up.
+* t2-delete-runs: Add a flag to only delete finished runs.
+* t2-run-workflow: Minor code clean-up.
+* t2-run-workflow: Add flag for service credentials.
+* Server now returns 503 when at capacity.
+* Server now has a limit on concurrently running runs.
+* Test for max concurrently running runs.
+* t2-run-workflow: Cope with hitting max running runs.
+* t2-server-stress: Cope with hitting max running runs.
+* Ensure all unit tests delete runs that they create.
+* Mark deleted runs as such in their Run object.
+* [TAVSERV-308] Run#permissions returns nil, instead of :none
+* Add a function to download a single port as a zip file.
+* Added ssl_version to connection parameters
+* Put a guard in to check that the SSL version is sensible.
+* Add SSL version support to the command line tools.
+* Add a new example connection parameter class for SSL version 3.
+* Describe the connection parameters in more detail.
+* Add Run#error? to check all output ports for errors.
+* UnexpectedServerResponse now provides more information.
+* Remove the port value caching code.
+* Simplify the HTTP methods and clean up error handling.
+* Split up the CLI options code.
+* Improve version string parsing.
+* Handle server returning Accepted on run start.
+* Get the run name from the server.
+* t2-server-info: Display run names in list view.
+* Set the run name on the server.
+* t2-run-workflow: Set name of run on server.
+* t2-get-output: Get name of run from server.
+* OutputPort#zip now returns nil for singletons.
+* Fix well-hidden HTTPS connection setup bug.
+* Use a RunCache class instead of a Hash to store local run objects.
+* t2-server-info: Configurable run detail in list view.
+* [TAVSERV-324] Uninitialized connection parameters.
+* Move interaction code out of Server, into Run.
+* Simplifications for per-run feeds.
+* Get different types of notifications for a Run.
+* Expose the run notification/interaction locations.
+* Add Run#log to get run logs.
+* Add connection timeout parameters.
+* Enable interaction reply publishing.
+* Read the interaction serial number from the notification.
+* Enable retrieval of an interaction's input data.
+
 == Version 0.9.3
 
 * Correctly handle missing values in output ports.

data/LICENCE.rdoc

@@ -1,4 +1,4 @@
-Copyright (c) 2010-2012 The University of Manchester, UK.
+Copyright (c) 2010-2013 The University of Manchester, UK.
 
 All rights reserved.
 

data/README.rdoc

@@ -4,8 +4,9 @@ Authors::     Robert Haines
 Contact::     mailto:support@mygrid.org.uk
 URL::         http://www.taverna.org.uk/
 Licence::     BSD (See LICENCE or http://www.opensource.org/licenses/bsd-license.php)
-Copyright::   (c) 2010-2012 The University of Manchester, UK
+Copyright::   (c) 2010-2013 The University of Manchester, UK
 
+{<img src="https://codeclimate.com/github/myGrid/t2-server-gem.png" />}[https://codeclimate.com/github/myGrid/t2-server-gem]
 
 == Synopsis
 
@@ -25,28 +26,41 @@ https://rubygems.org/gems/t2-server
 You can also download the source code from here:
 https://github.com/myGrid/t2-server-gem
 
-== Compatibility with Taverna Server versions
+== Compatibility
 
-From version 0.9.0 this library is compatible with Taverna Server 2.3 onwards.
+After the release of version 1.0.0 this gem will follow the principles of
+{Semantic Versioning}[http://semver.org/]. Until that point there may be
+breaking changes to the API, although these will be kept to a minimum.
+
+=== Taverna Server
+
+Versions 0.9.x of this library are compatible with Taverna Server 2.3 and 2.4.
 It is not compatible with any earlier version of Taverna Server due to breaking
-changes in its REST interface. We strongly encourage all users to upgrade to
-the current version of Taverna Server, but if that is not possible right now
-then please continue to use version 0.6.1 of this library for the time being.
+changes in its REST interface.
+
+From version 1.0.0 this library is not guaranteed to be compatible with
+Taverna Server 2.3. It might work but it is not supported, and may stop working
+at any time.
 
-This version of the library should be compatible with any Ruby code that you
-may have already written against earlier versions but you may see some warnings
-about API deprecations. These are clearly marked [DEPRECATION] and will appear
-on your console output. Anything marked as deprecated will be removed in
-version 1.0.0 (and are not guaranteed to continue working even until then) so
-you are advised to update your code at your earliest convenience.
+Version 1.0.0 of this library saw the removal of older methods that were
+previously deprecated. If your code no longer works with this version then
+please re-test it with version 0.9.3 and check for deprecation messages before
+reporting bugs.
 
-== Compatibility with Ruby versions
+We strongly encourage all users to upgrade to the current version of Taverna
+Server, but if that is not possible right now then these are the recommended
+version pairings:
+* pre Taverna Server 2.3, use version 0.6.1 of the gem
+* 2.3, use version 0.9.3
+* 2.4 and up, use version 1.0.0
+
+=== Ruby
 
 This library is known to work with the following versions of Ruby:
-* 1.8.7 *
+* 1.8.7
 * 1.9.2
 * 1.9.3 *
-* jruby 1.6.4 (in Ruby 1.8 mode)
+* 2.0.0
 
 Those marked with an asterisk (*) are supported and bugs found against them
 will be fixed. Other versions may work but are not tested or supported.
@@ -54,8 +68,8 @@ will be fixed. Other versions may work but are not tested or supported.
 == Usage
 
 There are two entry points for the T2Server API:
-* <tt>T2Server::Run</tt> - Use this for running single jobs on a server.
-* <tt>T2Server::Server</tt> - Use this if you are providing a web interface to
+* T2Server::Run - Use this for running single jobs on a server.
+* T2Server::Server - Use this if you are providing a web interface to
   one or more Taverna 2 Server instances.
 
 In both cases the gem should be initialized by requiring the top level ruby
@@ -76,6 +90,9 @@ Connection configuration settings are passed in to various methods using the
 * <tt>:verify_peer</tt> - Use a CA to verify that the Taverna Server you are connecting to has a valid server certificate and that it is the correct one.
 * <tt>:client_certificate</tt> - A certificate to use for client certificate authentication.
 * <tt>:client_password</tt> - The password to unlock the private key of the client certificate.
+* <tt>:ssl_version</tt> - The TLS/SSL version to use (<tt>:TLSv1</tt>, <tt>:SSLv23</tt> or <tt>:SSLv3</tt>)
+* <tt>:open_timeout</tt> - The number of seconds to wait while opening a connection.
+* <tt>:read_timeout</tt> - The number of seconds to wait while reading from a connection.
 
 And can be set like this for a standard https connection:
 
@@ -91,9 +108,10 @@ automatically.
 For convenience a number of standard sets of parameters have been defined. The
 above example is available as +DefaultConnectionParameters+. Others available
 are:
-* +InsecureSSLConnectionParameters+ - to ignore SSL checks.
+* +InsecureSSLConnectionParameters+ - to ignore SSL checks. <b>Not recommended!</b>
 * +CustomCASSLConnectionParameters+ - for custom (self-signed) CAs.
 * +ClientAuthSSLConnectionParameters+ - for client certificate authentication.
+* +SSL3ConnectionParameters+ - uses SSLv3 as the default transport layer.
 
 See the rdoc for more details on these classes.
 
@@ -106,7 +124,7 @@ These are simple to set up:
 
 === Server API example
 
-The Server constructor can yield the newly created object. Simple supply a URI
+The Server constructor can yield the newly created object. Simply supply a URI
 and a set of connection parameters to connect to a server:
 
   T2Server::Server.new(uri, conn_params) do |server|
@@ -124,8 +142,11 @@ credentials:
     ...
   end
 
-An individual run can be deleted with its own <tt>delete</tt> method (see below)
-but all runs on a server can be deleted in one go:
+The +workflow+ parameter can be the workflow itself, a filename or a File or
+IO object.
+
+An individual run can be deleted with its own <tt>delete</tt> method (see
+below) but all runs on a server can be deleted in one go:
 
   server.delete_all_runs(credentials)
 
@@ -141,6 +162,9 @@ a couple of runs directly:
     ...
   end
 
+As before, the +workflow+ parameter can be the workflow itself, a filename or
+a File or IO object.
+
 Setting an input port to a run is very easy:
 
   run.input_port("port_name").value = 1
@@ -164,25 +188,69 @@ Or just wait until the run has finished:
 
   run.wait
 
-Then the outputs can be collected:
+While a workflow is running it might produce notifications via the
+{Interaction Service}[http://dev.mygrid.org.uk/wiki/display/taverna/Interaction+service].
+These can be polled with:
+
+  run.notifications
+
+If there are any new notifications that have not been responded to they will
+be returned in a list to be processed. Notifications take the form of a Web
+page to be displayed and the notification objects returned from the above call
+hold a URI to that page. The following code prints those URIs to the console:
+
+  run.notifications.each do |i|
+    puts i.uri
+  end
+
+When the workflow has finished executing then the outputs can be collected
+into memory or saved to a file:
 
   result = run.output_port("port_name").value
+  run.output_port("port_name").write_value_to_file("/path/to/value.txt")
+
+Outputs can be queried as to their type, size (in bytes) or if they contain an
+error message:
+
+  run.output_port("port_name").type
+  run.output_port("port_name").size
+  run.output_port("port_name").error?
 
-If you have a lot of output you can grab the whole lot as a zip file:
+If the output does hold an error then it can be found in the value of the
+output as normal.
 
-  zip_data = run.zip_output
+If you have a lot of output you can grab the whole lot in a zip archive. This
+can be downloaded into memory or saved directly to a file.
+
+  zip_data = run.zip_output              # download to memory
+  run.zip_output("/path/to/output.zip")  # save to a file
 
 Using baclava documents for setting inputs and collecting outputs is also
 supported:
 
   run.baclava_input = filename
 
-But make sure you request baclava output *before* starting the run:
+But make sure you request baclava output *before* starting the run. Baclava
+output can be downloaded into memory or saved directly to a file.
 
   run.request_baclava_output
   run.start
   run.wait
-  output = run.baclava_output
+  output = run.baclava_output                    # download to memory
+  run.baclava_output("/path/to/output.baclava")  # save to a file
+
+The log from Taverna Server can be downloaded in a similar way to zip files or
+Baclava documents.
+
+  log = run.log                # download to memory
+  run.log("/path/to/log.txt")  # save to a file
+
+When downloading outputs the underlying stream can be accessed by supplying a
+block to the value, zip_output, baclava_output or log methods:
+
+  run.output_port("port_name").value do |data|
+    print data
+  end
 
 A run can be deleted when no longer needed, like so:
 

data/Rakefile

@@ -1,4 +1,4 @@
-# Copyright (c) 2010-2012 The University of Manchester, UK.
+# Copyright (c) 2010-2013 The University of Manchester, UK.
 #
 # All rights reserved.
 #
@@ -46,8 +46,8 @@ task :default => [:test]
 Jeweler::Tasks.new do |s|
   s.name             = "t2-server"
   s.version          = T2Server::Version::STRING
-  s.authors          = ["Robert Haines"]
-  s.email            = ["rhaines@manchester.ac.uk"]
+  s.authors          = ["Robert Haines", "Finn Bacall"]
+  s.email            = ["support@mygrid.org.uk"]
   s.homepage         = "http://www.taverna.org.uk/"
   s.platform         = Gem::Platform::RUBY
   s.summary          = "Support for interacting with Taverna 2 Server."
@@ -66,8 +66,10 @@ Jeweler::Tasks.new do |s|
   s.add_development_dependency('nokogiri', '>= 1.5.0')
   s.add_development_dependency('rdoc', '>= 3.9.4')
   s.add_development_dependency('jeweler', '~> 1.8.3')
+  s.add_development_dependency('launchy', '~> 2.2.0')
   s.add_runtime_dependency('net-http-persistent', '~> 2.6')
   s.add_runtime_dependency('taverna-baclava', '~> 1.0.0')
+  s.add_runtime_dependency('ratom', '~> 0.8.2')
   s.add_runtime_dependency('hirb', '>= 0.4.0')
 end
 
@@ -95,10 +97,9 @@ end
 RDoc::Task.new do |r|
   r.main = "README.rdoc"
   lib = Dir.glob("lib/**/*.rb").delete_if do |item|
-    item.include?("t2server.rb") or
     item.include?("/xml/") or
+    item.include?("run-cache.rb") or
     item.include?("connection.rb") or
-    item.include?("credentials.rb") or
     item.include?("t2-server-cli.rb")
   end
   r.rdoc_files.include("README.rdoc", "LICENCE.rdoc", "CHANGES.rdoc", lib)

data/bin/t2-delete-runs

@@ -38,16 +38,20 @@ include T2Server::CLI
 
 # set up options
 options = {}
-conn_params, creds = register_options("Usage: t2-delete-runs [options] " +
+conn_params, creds = register_options("Usage: t2-delete-runs [options] "\
     "server-address [run-ids...]") do |opt|
-  opt.separator "  Where server-address is the full URI of the server to " +
+  opt.separator "  Where server-address is the full URI of the server to "\
     "connect to,"
-  opt.separator "  e.g.: http://example.com:8080/taverna, run-ids are the " +
+  opt.separator "  e.g.: http://example.com:8080/taverna, run-ids are the "\
     "id numbers"
   opt.separator "  of the runs you want to delete and [options] can be:"
   opt.on("--all", "Delete all runs on the server") do
     options[:all] = true
   end
+  opt.on("-f", "--finished", "Delete only finished runs. Combine with --all "\
+    "to delete all finished runs") do
+      options[:finished] = true
+  end
 end
 
 # get runs and server address from the arguments
@@ -71,29 +75,27 @@ rescue RuntimeError => e
   exit 1
 end
 
+# ...get the runs...
+server_runs = server.runs(creds)
+
 # ...and delete them!
-if options[:all]
+for run in server_runs
   begin
-    server.delete_all_runs(creds)
+    if options[:all] || runs.include?(run.id)
+      if options[:finished]
+        run.delete if run.finished?
+      else
+        run.delete
+      end
+    end
+  rescue T2Server::RunNotFoundError => rnf
+    puts "Run '#{run.id}' not found - skipping."
+    next
   rescue T2Server::AuthorizationError => ae
-    puts "You are not authorized to delete runs on this server."
+    puts "You are not authorized to delete run '#{run.id}' - skipping."
+    next
   rescue T2Server::T2ServerError => e
-    puts "There was a problem while deleting runs. " +
-      "Some may remain on the server."
-  end
-else
-  for run in runs
-    begin
-      server.delete_run(run, creds)
-    rescue T2Server::RunNotFoundError => rnf
-      puts "Run '#{run}' not found - skipping."
-      next
-    rescue T2Server::AuthorizationError => ae
-      puts "You are not authorized to delete run '#{run}' - skipping."
-      next
-    rescue T2Server::T2ServerError => e
-      puts "There was a problem while deleting run '#{run}' - skipping."
-      next
-    end
+    puts "There was a problem while deleting run '#{run.id}' - skipping."
+    next
   end
 end

data/bin/t2-get-output

@@ -1,5 +1,5 @@
 #!/usr/bin/env ruby
-# Copyright (c) 2010-2012 The University of Manchester, UK.
+# Copyright (c) 2010-2013 The University of Manchester, UK.
 #
 # All rights reserved.
 #
@@ -53,6 +53,10 @@ conn_params, creds = register_options("Usage: t2-get-output [options] " +
     "are specified then list all the output ports.") do |val|
     ports << val.chomp
   end
+  opt.on("-N", "--name", "Return the name of the run. Supported in Taverna "\
+    "Server versions 2.5.0 and up.") do
+      options[:name] = true
+  end
   opt.on("-x", "--exitcode", "Return the exitcode of the run.") do
     options[:exitcode] = true
   end
@@ -95,6 +99,8 @@ begin
   server = T2Server::Server.new(uri, conn_params)
   run = server.run(run_id, creds)
 
+  puts "Run name: #{run.name}" if options[:name]
+
   ports = run.output_ports.keys if ports.empty?
   ports.each do |p|
     port = run.output_port(p)
@@ -112,22 +118,14 @@ begin
       p port.type
     end
 
-    if options[:tsize]
-      puts "  Total data size: #{port.total_size}"
-    end
+    puts "  Total data size: #{port.total_size}" if options[:tsize]
   end
 
-  if options[:exitcode]
-    puts "Exitcode: #{run.exitcode}"
-  end
+  puts "Exitcode: #{run.exitcode}" if options[:exitcode]
 
-  if options[:stdout]
-    puts "Stdout: #{run.stdout}"
-  end
+  puts "Stdout: #{run.stdout}" if options[:stdout]
 
-  if options[:stderr]
-    puts "Stderr: #{run.stderr}"
-  end
+  puts "Stderr: #{run.stderr}" if options[:stderr]
 rescue RuntimeError => e
   puts e
   exit 1