RubyGems - aspera-cli - Versions diffs - 4.2.1 → 4.2.2 - Mend

aspera-cli 4.2.1 → 4.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (22) hide show

checksums.yaml +4 -4
data/README.md +187 -127
data/docs/Makefile +4 -4
data/docs/README.erb.md +206 -38
data/docs/test_env.conf +2 -1
data/examples/faspex4.rb +28 -17
data/lib/aspera/cli/main.rb +47 -19
data/lib/aspera/cli/plugins/aoc.rb +5 -13
data/lib/aspera/cli/plugins/config.rb +47 -30
data/lib/aspera/cli/plugins/faspex.rb +90 -52
data/lib/aspera/cli/plugins/faspex5.rb +8 -7
data/lib/aspera/cli/plugins/preview.rb +29 -25
data/lib/aspera/cli/transfer_agent.rb +71 -44
data/lib/aspera/cli/version.rb +1 -1
data/lib/aspera/command_line_builder.rb +49 -31
data/lib/aspera/fasp/parameters.rb +57 -87
data/lib/aspera/fasp/parameters.yaml +531 -0
data/lib/aspera/fasp/uri.rb +1 -1
data/lib/aspera/oauth.rb +4 -4
data/lib/aspera/sync.rb +40 -35
metadata +17 -3
data/docs/transfer_spec.html +0 -99

data/docs/README.erb.md CHANGED Viewed

@@ -1,7 +1,7 @@
 [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
 <%
  # check that required env vars exist, and files
-%w{EXENAME GEMSPEC INCL_USAGE INCL_COMMANDS INCL_ASESSION INCL_TRSPEC}.each do |e|
+%w{EXENAME GEMSPEC INCL_USAGE INCL_COMMANDS INCL_ASESSION INCL_DIR_GEM}.each do |e|
   raise "missing env var #{e}" unless ENV.has_key?(e)
   raise "missing file #{ENV[e]}" unless File.exist?(ENV[e]) or !e.start_with?('INCL_') #_
 end
@@ -14,6 +14,29 @@ prsts='['+opprst+'s](#lprt)'
 prstt=opprst.capitalize # in title
 gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
 geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
+$LOAD_PATH.unshift(ENV["INCL_DIR_GEM"])
+require 'aspera/fasp/parameters'
+def spec_table
+    r='<table><tr><th>Field</th><th>Type</th>'
+    Aspera::Fasp::Parameters::SUPPORTED_AGENTS_SHORT.each do |c|
+      r << '<th>'<<c.to_s.upcase<<'</th>'
+    end
+    r << '<th>Description</th></tr>'
+    Aspera::Fasp::Parameters.man_table.each do |p|
+      p[:description] << (p[:description].empty? ? '' : "\n") << "(" << p[:cli] << ")" unless p[:cli].to_s.empty?
+      p.delete(:cli)
+      p.keys.each{|c|p[c]='&nbsp;' if p[c].to_s.empty?}
+      p[:description].gsub!("\n",'<br/>')
+      p[:type].gsub!(',','<br/>')
+      r << '<tr><td>'<<p[:name]<<'</td><td>'<<p[:type]<<'</td>'
+      Aspera::Fasp::Parameters::SUPPORTED_AGENTS_SHORT.each do |c|
+        r << '<td>'<<p[c]<<'</td>'
+      end
+      r << '<td>'<<p[:description]<<'</td></tr>'
+    end
+    r << '</table>'
+    return r
+end
 -%>
 # <%=tool%> : Command Line Interface for IBM Aspera products
@@ -1283,11 +1306,31 @@ A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the co
 ## <a name="transferparams"></a>Transfer Parameters
-All standard _transfer-spec_ parameters can be overloaded. To display parameters,
-run in debug mode (--log-level=debug). [_transfer-spec_](#transferspec) can
-also be saved/overridden in the config file.
+All standard _transfer-spec_ parameters can be speficied.
+[_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
+References:
+* [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22)&rarr;/opt/transfers
+* [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22)&rarr;Guides&rarr;API Ref&rarr;Transfer Spec V1
+Parameters can be displayed with commands:
+```
+$ <%=cmd%> config ascp spec
+$ <%=cmd%> config ascp spec --select=@json:'{"f":"Y"}' --fields=-f,n,c
+```
+Columns:
+* D=Direct (local `ascp` execution)
+* N=Node API
+* C=Connect Client
+* arg=`ascp` argument or environment variable
-<%= File.read(ENV['INCL_TRSPEC']).gsub(/.*<body>(.*)<\/body>.*/m,'\1') %>
+Fields with EX_ prefix are extensions to transfer agent `direct`. (only in <%=tool%>).
+<%= spec_table %>
 ### Destination folder for transfers
@@ -1369,7 +1412,7 @@ Source files are located on "Aspera on cloud", when :
 ### <a name="multisession"></a>Support of multi-session
-Multi session, i.e. starting a transfer of a file set using multiple sessions is supported on "direct" and "node" agents, not yet on connect.
+Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
 * when agent=node :
@@ -1390,6 +1433,7 @@ shall be preferred.
 Multi-session spawn is done by <%=tool%>.
+When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
 ### Examples
@@ -1665,7 +1709,7 @@ To activate default use of JWT authentication for <%=tool%> using the <%=prst%>,
 * change auth method to JWT
 * provide location of private key
-* provide username to login as (OAuthg "subject")
+* provide username to login as (OAuth "subject")
 Execute:
@@ -2076,19 +2120,40 @@ The activity app can be queried with:
 $ <%=cmd%> aoc admin analytics transfers
 ```
-It can also support filters and send notification email with a template:
+It can also support filters and send notification using option `notif_to`. a template is defined using option `notif_template` :
+`mytemplate.erb`:
+```
+From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
+To: <<%='<'%>%=ev['user_email']%>>
+Subject: <%='<'%>%=ev['files_completed']%> files received
+Dear <%='<'%>%=ev[:user_email.to_s]%>,
+We received <%='<'%>%=ev['files_completed']%> files for a total of <%='<'%>%=ev['transferred_bytes']%> bytes, starting with file:
+<%='<'%>%=ev['content']%>
+Thank you.
+```
+The environment provided contains the following additional variable:
+* ev : all details on the transfer event
+Example:
 ```
-$ <%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=123455 \
+$ <%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=12345 \
 --query=@json:'{"status":"completed","direction":"receive"}' \
---notify=@json:'{"to":"<''%=transfer[:user_email.to_s]%>","subject":"<''%=transfer[:files_completed.to_s]%> files received","body":"Dear <''%=transfer[:user_email.to_s]%>\nWe received <''%=transfer[:files_completed.to_s]%> files for a total of <''%=transfer[:transferred_bytes.to_s]%> bytes, starting with file:\n<''%=transfer[:content.to_s]%>\n\nThank you."}'
+--notif-to=active --notif-template=@file:mytemplate.erb
 ```
+Options:
 * `once_only` keep track of last date it was called, so next call will get only new events
 * `query` filter (on API call)
 * `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
-Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. here the period is [date of previous execution]..[now].
+Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
 ## Using specific transfer ports
@@ -2339,11 +2404,15 @@ $ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":
 3 authentication methods are supported:
-* boot
-* web
 * jwt
+* web
+* boot
-For boot method:
+For JWT, create an API client in Faspex with jwt support, and use: `--auth=jwt`.
+For web method, create an API client in Faspex, and use: --auth=web
+For boot method: (will be removed in future)
 * open a browser
 * start developer mode
@@ -2356,25 +2425,57 @@ Use it as password and use `--auth=boot`.
 $ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
 ```
-For web method, create an API client in Faspex, and use: --auth=web
-For JWT, create an API client in Faspex with jwt supporot, and use: --auth=jwt
-as of beta£3 this does not allow regular users.
 Ready to use Faspex5 with CLI.
-Once the graphical registration form exist, ther bootstrap method can be removed.
 # Plugin: IBM Aspera Faspex (4.x)
 Notes:
-* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
-* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+* The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+* For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
+## Listing Packages
+Command: `faspex package list`
+### Option `box`
+By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
+### Option `recipient`
+A user can receive a package because the recipient is:
+* the user himself (default)
+* the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
+### Option `query`
+As inboxes may be large, it is possible to use the following query parameters:
+* `count` : (native) number items in one API call (default=0, equivalent to 10)
+* `page` : (native) id of page in call (default=0)
+* `startIndex` : (native) index of item to start, default=0, oldest index=0
+* `max` : maximum number of items
+* `pmax` : maximum number of pages
+(SQL query is `LIMIT <startIndex>, <count>`)
+The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "Services (API v.3)".
+If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`). By default page is `0` (`10`), it can be increased to have less calls.
+### Example
+```
+$ <%=cmd%> faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
+```
+List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
 ## Receiving a Package
-The command is `package recv`, possible methosd are:
+The command is `package recv`, possible methods are:
 * provide a package id with option `id`
 * provide a public link with option `link`
@@ -2391,7 +2492,7 @@ If the package is in a specific dropbox, add option `recipient` for both the `li
 $ <%=cmd%> faspex package list --recipient='*thedropboxname'
 ```
+if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
 ## Sending a Package
@@ -2410,7 +2511,19 @@ Additional optional parameters in `delivery_info`:
 * Package Note: : `"note":"note this and that"`
 * Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
-## operation on dropboxes
+## Email notification on transfer
+Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
+Example:
+```
+$ <%=cmd%> faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>\nTo: <<%='<'%>%=to%>>\nSubject: Package sent: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
+```
+In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
+## Operation on dropboxes
 Example:
@@ -2828,14 +2941,14 @@ Aspera CLI can send email, for that setup SMTP configuration. This is done with
 The `smtp` option is a hash table (extended value) with the following fields:
 <table>
 <tr><th>field</th><th>default</th><th>example</th><th>description</th></tr>
-<tr><td>server</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
-<tr><td>tls</td><td>true</td><td>false</td><td>use of TLS</td></tr>
-<tr><td>port</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
-<tr><td>domain</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
-<tr><td>username</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
-<tr><td>password</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
-<tr><td>from\_email</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
-<tr><td>from\_name</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
+<tr><td>`server`</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
+<tr><td>`tls`</td><td>true</td><td>false</td><td>use of TLS</td></tr>
+<tr><td>`port`</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
+<tr><td>`domain`</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
+<tr><td>`username`</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
+<tr><td>`password`</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
+<tr><td>`from_email`</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
+<tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
 </table>
 ## Example of configuration:
@@ -2865,13 +2978,52 @@ $ <%=cmd%> config id cli_default set smtp @val:@preset:smtp_google
 $ <%=cmd%> config id default set config cli_default
 ```
+## Email templates
+Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
+The template is the full SMTP message, including headers.
+The following variables are defined by default:
+* from_name
+* from_email
+* to
+Other variables are defined depending on context.
 ## Test
 Check settings with `smtp_settings` command. Send test email with `email_test`.
 ```
 $ <%=cmd%> config --smtp=@preset:smtp_google smtp
-$ <%=cmd%> config --smtp=@preset:smtp_google email sample.dest@example.com
+$ <%=cmd%> config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
+```
+## Notifications for transfer status
+An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
+To activate, use option `notif_to`.
+A default e-mail template is used, but it can be overriden with option `notif_template`.
+The environment provided contains the following additional variables:
+* subject
+* body
+* global_transfer_status
+* ts
+Example of template:
+```
+From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
+To: <<%='<'%>%=to%>>
+Subject: <%='<'%>%=subject%>
+Transfer is: <%='<'%>%=global_transfer_status%>
 ```
 # Tool: `asession`
@@ -2982,7 +3134,7 @@ Interesting ascp features are found in its arguments: (see ascp manual):
 Note that:
 * <%=tool%> takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
-* not all native ascp arguments are available as standard transfer_spec parameters
+* most, but not all native ascp arguments are available as standard transfer_spec parameters
 * native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the "local" transfer agent (not connect or node)
 ### server side and configuration
@@ -3081,6 +3233,18 @@ So, it evolved into <%=tool%>:
 * <%= gemspec.version.to_s %>
+	* new: `faspex package list` retrieves the whole list, not just first page
+	* new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
+	* new: the error "Remote host is not who we expected" displays a special remediation message
+	* new: `conf ascp spec` displays supported transfer spec
+	* new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
+	* fix: space character in `faspe:` url are precent encoded if needed
+	* fix: `preview scan`: if file_id is unknown, ignore and continue scan
+	* change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then <%=tool%> exits with code 1 (instead of zero=success)
+	* change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
+* 4.2.1
 	* new: command `faspex package recv` supports link of type: `faspe:`
 	* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
@@ -3503,13 +3667,17 @@ You may either install the suggested Gems, or remove your ed25519 key from your
 ## Error "Remote host is not who we expected"
-`ascp` version 4.x changed the algorithm used to check the SSH server certificate. To ignore the certificate (SSH fingerprint) add option on client side:
+Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
+Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
 ```
 --ts=@json:'{"sshfp":null}'
 ```
-Refer to ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server): if you have access to server side, basically disable other SSH host keys than RSA.
+Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
+References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
 ## Miscelaneous

data/docs/test_env.conf CHANGED Viewed

@@ -5,7 +5,7 @@ default:
   config: cli_default
   aoc: tst_aoc1
   faspex: tst_faspex
-  faspex5: tst_faspex5_web
+  faspex5: tst_faspex5
   shares: tst_shares_1
   shares2: tst_shares2
   node: tst_node
@@ -65,6 +65,7 @@ tst_faspex5:
   client_secret: your value here
   private_key: your value here
   username: your value here
+  password: your value here
 tst_shares:
   url: your value here
   username: your value here

data/examples/faspex4.rb CHANGED Viewed

@@ -1,29 +1,37 @@
 #!/usr/bin/env ruby
+# find Faspex API here: https://developer.ibm.com/apis/catalog/?search=faspex
+# this example makes use of class Aspera::Rest for REST calls, alternatively class RestClient of gem rest-client could be used
+# this example makes use of class Aspera::Fasp::Local for transfers, alternatively the official "Transfer SDK" could be used
+# Aspera SDK can be downloaded with: `ascli conf ascp install` , it installs in $HOME/.aspera/ascli/sdk
 require 'aspera/rest'
 require 'aspera/log'
 require 'aspera/fasp/local'
-require 'aspera/cli/listener/line_dump'
-require 'aspera/cli/extended_value'
-# set high log level for the example
+tmpdir=ENV['tmp']||Dir.tmpdir || '.'
+# Set high log level for the example, decrease to :warn usually
 Aspera::Log.instance.level=:debug
-# set folder where SDK is installed
+# Set folder where SDK is installed (mandatory)
 # (if ascp is not there, the lib will try to find in usual locations)
-Aspera::Fasp::Installation.instance.folder='.'
+# (if data files are not there, they will be created)
+Aspera::Fasp::Installation.instance.folder = tmpdir
 if ! ARGV.length.eql?(3)
-  Aspera::Log.log.error("wrong number of args: #{ARGV.length}")
+  Aspera::Log.log.error("Wrong number of args: #{ARGV.length}")
   Aspera::Log.log.error("Usage: #{$0} <faspex URL> <faspex username> <faspex password>")
   Aspera::Log.log.error("Example: #{$0} https://faspex.com/aspera/faspex john p@sSw0rd")
   Process.exit(1)
 end
-faspex_url=ARGV[0]
+faspex_url=ARGV[0] # typically: https://faspex.example.com/aspera/faspex
 faspex_user=ARGV[1]
 faspex_pass=ARGV[2]
-# 1: demo API v3
+# comment out this if certificate is valid, keep line to ignore certificate
+Aspera::Rest.insecure=true
+# 1: Faspex 4 API v3
 #---------------
 # create REST API object
@@ -35,34 +43,36 @@ api_v3=Aspera::Rest.new({
   :password => faspex_pass
   }})
+# very simple api call
 api_v3.read('me')
 # 2: send a package
 #---------------
 # create a sample file to send
-file_to_send='./myfile.bin'
+file_to_send=File.join(tmpdir,'myfile.bin')
 File.open(file_to_send, "w") {|f| f.write("sample data") }
 # package creation parameters
 package_create_params={'delivery'=>{'title'=>'test package','recipients'=>['aspera.user1@gmail.com'],'sources'=>[{'paths'=>[file_to_send]}]}}
 pkg_created=api_v3.create('send',package_create_params)[:data]
-# get transfer specification
+# get transfer specification (normally: only one)
 transfer_spec=pkg_created['xfer_sessions'].first
 # set paths of files to send
-transfer_spec['paths']=['source'=>file_to_send]
+transfer_spec['paths']=[{'source'=>file_to_send}]
 # get the local agent (i.e. ascp)
-client=Aspera::Fasp::Local.new
-# disable ascp output on stdout to not mix with JSON events
-client.quiet=true
+transfer_client=Aspera::Fasp::Local.new
+# disable ascp output on stdout (optional)
+transfer_client.quiet=true
 # start transfer (asynchronous)
-job_id=client.start_transfer(transfer_spec)
-result=client.wait_for_transfers_completion
+job_id=transfer_client.start_transfer(transfer_spec)
+# wait for all transfer completion (for the example)
+result=transfer_client.wait_for_transfers_completion
 #  notify of any transfer error
 result.select{|i|!i.eql?(:success)}.each do |e|
   Aspera::Log.log.error("A transfer error occured: #{e.message}")
 end
-# 1: demo API v4
+# 3: Faspex 4 API v4
 #---------------
 api_v4=Aspera::Rest.new({
   :base_url  => faspex_url+'/api',
@@ -75,4 +85,5 @@ api_v4=Aspera::Rest.new({
   :scope     => 'admin'
   }})
+# Use it. Note that Faspex 4 API v4 is totally different from Faspex 4 v3 APIs, see ref on line 2
 Aspera::Log.dump('users',api_v4.read('users')[:data])

data/lib/aspera/cli/main.rb CHANGED Viewed

@@ -21,15 +21,17 @@ module Aspera
       private
       # name of application, also foldername where config is stored
       PROGRAM_NAME = 'ascli'
+      # name of the containing gem, same as in <gem name>.gemspec
       GEM_NAME = 'aspera-cli'
-      VERBOSE_LEVELS=[:normal,:minimal,:quiet]
-      private_constant :PROGRAM_NAME,:GEM_NAME,:VERBOSE_LEVELS
+      HELP_URL = "http://www.rubydoc.info/gems/#{GEM_NAME}"
+      GEM_URL  = "https://rubygems.org/gems/#{GEM_NAME}"
+      SRC_URL  = "https://github.com/IBM/aspera-cli"
+      STATUS_FIELD = 'status'
+      private_constant :PROGRAM_NAME,:GEM_NAME,:HELP_URL,:GEM_URL
       # =============================================================
       # Parameter handlers
       #
       def option_insecure; Rest.insecure ; end
       def option_insecure=(value); Rest.insecure = value; end
@@ -40,20 +42,19 @@ module Aspera
       # minimum initialization
       def initialize(argv)
-        # first thing : manage debug level (allows debugging or option parser)
+        # first thing : manage debug level (allows debugging of option parser)
         early_debug_setup(argv)
+        # compare $0 with expected name
         current_prog_name=File.basename($PROGRAM_NAME)
         unless current_prog_name.eql?(PROGRAM_NAME)
           @plugin_env[:formater].display_message(:error,"#{"WARNING".bg_red.blink.gray} Please use '#{PROGRAM_NAME}' instead of '#{current_prog_name}', '#{current_prog_name}' will be removed in a future version")
         end
-        # overriding parameters on transfer spec
         @option_help=false
         @bash_completion=false
         @option_show_config=false
+        # environment provided to plugin for various capabilities
         @plugin_env={}
-        @help_url='http://www.rubydoc.info/gems/'+GEM_NAME
-        @gem_url='https://rubygems.org/gems/'+GEM_NAME
-        # give command line arguments to option manager (no parsing)
+        # find out application main folder
         app_main_folder=ENV[conf_dir_env_var]
         # if env var undefined or empty
         if app_main_folder.nil? or app_main_folder.empty?
@@ -61,20 +62,21 @@ module Aspera
           raise CliError,"Home folder does not exist: #{user_home_folder}. Check your user environment or use #{conf_dir_env_var}." unless Dir.exist?(user_home_folder)
           app_main_folder=File.join(user_home_folder,Plugins::Config::ASPERA_HOME_FOLDER_NAME,PROGRAM_NAME)
         end
+        # give command line arguments to option manager (no parsing)
         @plugin_env[:options]=@opt_mgr=Manager.new(PROGRAM_NAME,argv,app_banner())
         @plugin_env[:formater]=Formater.new(@plugin_env[:options])
         Rest.user_agent=PROGRAM_NAME
-        # must override help methods before parser called (in other constructors)
+        # declare and parse global options
         init_global_options()
         # secret manager
         @plugin_env[:secret]=Aspera::Secrets.new
         # the Config plugin adds the @preset parser
-        @plugin_env[:config]=Plugins::Config.new(@plugin_env,PROGRAM_NAME,@help_url,Aspera::Cli::VERSION,app_main_folder)
+        @plugin_env[:config]=Plugins::Config.new(@plugin_env,PROGRAM_NAME,HELP_URL,Aspera::Cli::VERSION,app_main_folder)
         # the TransferAgent plugin may use the @preset parser
         @plugin_env[:transfer]=TransferAgent.new(@plugin_env)
-        Log.log.debug('created plugin env'.red)
         # set application folder for modules
         @plugin_env[:persistency]=PersistencyFolder.new(File.join(@plugin_env[:config].main_folder,'persist_store'))
+        Log.log.debug('plugin env created'.red)
         Oauth.persist_mgr=@plugin_env[:persistency]
         Fasp::Parameters.file_list_folder=File.join(@plugin_env[:config].main_folder,'filelists')
         Aspera::RestErrorAnalyzer.instance.log_file=File.join(@plugin_env[:config].main_folder,'rest_exceptions.log')
@@ -88,9 +90,9 @@ module Aspera
         banner << "\t#{PROGRAM_NAME} COMMANDS [OPTIONS] [ARGS]\n"
         banner << "\nDESCRIPTION\n"
         banner << "\tUse Aspera application to perform operations on command line.\n"
-        banner << "\tDocumentation and examples: #{@gem_url}\n"
+        banner << "\tDocumentation and examples: #{GEM_URL}\n"
         banner << "\texecute: #{PROGRAM_NAME} conf doc\n"
-        banner << "\tor visit: #{@help_url}\n"
+        banner << "\tor visit: #{HELP_URL}\n"
         banner << "\nENVIRONMENT VARIABLES\n"
         banner << "\t#{conf_dir_env_var}  config folder, default: $HOME/#{Plugins::Config::ASPERA_HOME_FOLDER_NAME}/#{PROGRAM_NAME}\n"
         banner << "\t#any option can be set as an environment variable, refer to the manual\n"
@@ -188,6 +190,8 @@ module Aspera
       protected
+      # env var name to override the app's main folder
+      # default main folder is $HOME/<vendor main app folder>/<program name>
       def conf_dir_env_var
         return "#{PROGRAM_NAME}_home".upcase
       end
@@ -219,10 +223,28 @@ module Aspera
         return Main.result_nothing
       end
+      # used when one command executes several transfer jobs (each job being possibly multi session)
+      # @param status_table [Array] [{STATUS_FIELD=>[status array],...},...]
+      # each element has a key STATUS_FIELD which contains the result of possibly mulmtiple sessions
+      def self.result_transfer_multiple(status_table)
+        global_status=:success
+        # transform status into string and find if there was problem
+        status_table.each do |item|
+          worst=TransferAgent.session_status(item[STATUS_FIELD])
+          global_status=worst unless worst.eql?(:success)
+          item[STATUS_FIELD]=item[STATUS_FIELD].map{|i|i.to_s}.join(',')
+        end
+        raise global_status unless global_status.eql?(:success)
+        return {:type=>:object_list,:data=>status_table}
+      end
       # this is the main function called by initial script just after constructor
       def process_command_line
         Log.log.debug('process_command_line')
+        # catch exception information , if any
         exception_info=nil
+        # false if command shall not be executed ("once_only")
+        execute_command=true
         begin
           # find plugins, shall be after parse! ?
           @plugin_env[:config].add_plugins_from_lookup_folders
@@ -264,11 +286,12 @@ module Aspera
               Log.log.debug("Opening lock port #{lock_port.to_i}")
               @tcp_server=TCPServer.new('127.0.0.1',lock_port.to_i)
             rescue => e
-              raise CliError,"Another instance is already running (#{e.message})."
+              execute_command=false
+              Log.log.warn("Another instance is already running (#{e.message}).")
             end
           end
-          # execute and display
-          @plugin_env[:formater].display_results(command_plugin.execute_action)
+          # execute and display (if not exclusive execution)
+          @plugin_env[:formater].display_results(command_plugin.execute_action) if execute_command
           # finish
           @plugin_env[:transfer].shutdown
         rescue CliBadArgument => e;          exception_info=[e,'Argument',:usage]
@@ -286,10 +309,15 @@ module Aspera
         unless exception_info.nil?
           @plugin_env[:formater].display_message(:error,"ERROR:".bg_red.gray.blink+" "+exception_info[1]+": "+exception_info[0].message)
           @plugin_env[:formater].display_message(:error,"Use '-h' option to get help.") if exception_info[2].eql?(:usage)
+          if exception_info.first.is_a?(Fasp::Error) and exception_info.first.message.eql?('Remote host is not who we expected')
+            @plugin_env[:formater].display_message(:error,"For this specific error, refer to:\n#{SRC_URL}#error-remote-host-is-not-who-we-expected\nAdd this to arguments:\n--ts=@json:'{\"sshfp\":null}'")
+          end
         end
         # 2- processing of command not processed (due to exception or bad command line)
-        @opt_mgr.final_errors.each do |msg|
-          @plugin_env[:formater].display_message(:error,"ERROR:".bg_red.gray.blink+" Argument: "+msg)
+        if execute_command
+          @opt_mgr.final_errors.each do |msg|
+            @plugin_env[:formater].display_message(:error,"ERROR:".bg_red.gray.blink+" Argument: "+msg)
+          end
         end
         # 3- in case of error, fail the process status
         unless exception_info.nil?