aspera-cli 4.2.0 → 4.4.0

data/docs/README.erb.md

@@ -1,35 +1,20 @@
 [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
-<%
- # check that required env vars exist, and files
-%w{EXENAME GEMSPEC INCL_USAGE INCL_COMMANDS INCL_ASESSION INCL_TRSPEC}.each do |e|
-  raise "missing env var #{e}" unless ENV.has_key?(e)
-  raise "missing file #{ENV[e]}" unless File.exist?(ENV[e]) or !e.start_with?('INCL_') #_
-end
-cmd=ENV["EXENAME"] # just command name
-tool='`'+cmd+'`'   # used in text with formatting of command
-evp=cmd.upcase+'_' # prefix for env vars
-opprst='option preset' # just the name for "option preset"
-prst='['+opprst+'](#lprt)'
-prsts='['+opprst+'s](#lprt)'
-prstt=opprst.capitalize # in title
-gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
-geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
--%>
-# <%=tool%> : Command Line Interface for IBM Aspera products
-
-Version : <%= gemspec.version.to_s %>
+<%load File.join(File.dirname(__FILE__),'doc_tools.rb')-%>
+<font size="+12"><center><%=tool%> : Command Line Interface for IBM Aspera products</center></font>
+
+Version : <%=gemspec.version.to_s%>
 
 _Laurent/2016-<%=Time.new.year%>_
 
-This gem provides <%=tool%>: a command line interface to Aspera Applications.
+This gem provides the <%=tool%> Command Line Interface to IBM Aspera software.
 
 <%=tool%> is a also great tool to learn Aspera APIs.
 
-Ruby Gem: [<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
+Ruby Gem: [<%=gemspec.metadata['rubygems_uri']%>](<%=gemspec.metadata['rubygems_uri']%>)
 
-Ruby Doc: [<%= gemspec.metadata['documentation_uri'] %>](<%= gemspec.metadata['documentation_uri'] %>)
+Ruby Doc: [<%=gemspec.metadata['documentation_uri']%>](<%=gemspec.metadata['documentation_uri']%>)
 
-Ruby version must be >= <%= gemspec.required_ruby_version %>
+Required Ruby version: <%=gemspec.required_ruby_version%>
 
 # <a name="when_to_use"></a>When to use and when not to use
 
@@ -45,7 +30,8 @@ So it is designed for:
 
 <%=tool%> can be seen as a command line tool integrating:
 
-* a configuration file (config.yaml) and advanced command line options
+* a configuration file (config.yaml)
+* advanced command line options
 * cURL (for REST calls)
 * Aspera transfer (ascp)
 
@@ -72,7 +58,7 @@ Command line parameters in examples beginning with `my_`, like `my_param_value`
 
 On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before <%=tool%> (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to <%=tool%>. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
 
-On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
+On Windows, `cmd.exe` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
 
 * [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
 * [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
@@ -97,7 +83,7 @@ Once the gem is installed, <%=tool%> shall be accessible:
 
 ```
 $ <%=cmd%> --version
-<%= gemspec.version.to_s %>
+<%=gemspec.version.to_s%>
 ```
 
 ## First use
@@ -124,7 +110,7 @@ $ <%=cmd%> server browse /
 If you want to use <%=tool%> with another server, and in order to make further calls more convenient, it is advised to define a <%=prst%> for the server's authentication options. The following example will:
 
 * create a <%=prst%>
-* define it as default for "server" plugin
+* define it as default for `server` plugin
 * list files in a folder
 * download a file
 
@@ -167,8 +153,8 @@ It is possible to install *either* directly on the host operating system (Linux,
 
 The direct installation is recommended and consists in installing:
 
-* [Ruby](#ruby) version >= <%= gemspec.required_ruby_version %>
-* [<%= gemspec.name %>](#the_gem)
+* [Ruby](#ruby) version <%=gemspec.required_ruby_version%>
+* [<%=gemspec.name%>](#the_gem)
 * [Aspera SDK (ascp)](#fasp_prot)
 
 The following sections provide information on the various installation methods.
@@ -181,6 +167,8 @@ Use this method only if you know what you do, else use the standard recommended
 
 This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
 
+The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli)
+
 Ensure that you have Docker installed.
 
 ```
@@ -215,7 +203,7 @@ Use this method to install on the native host.
 
 A ruby interpreter is required to run the tool or to use the gem and tool.
 
-Ruby minimum version: <%= gemspec.required_ruby_version %>. Ruby version 3 is also supported.
+Required Ruby version: <%=gemspec.required_ruby_version%>. Ruby version 3 is also supported.
 
 *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
 
@@ -223,7 +211,7 @@ Refer to the following sections for a proposed method for specific operating sys
 
 The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
 If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
-If you have a simpler better way to install Ruby version >= <%= gemspec.required_ruby_version %> : use it !
+If you have a simpler better way to install Ruby version <%=gemspec.required_ruby_version%> : use it !
 
 ### Generic: RVM: single user installation (not root)
 
@@ -249,7 +237,7 @@ If you keep the same terminal (ont needed if re-login):
 $ source ~/.rvm/scripts/rvm
 ```
 
-It is advised to get one of the pre-compiled ruby version, you can list with: 
+It is advised to get one of the pre-compiled ruby version, you can list with:
 
 ```
 $ rvm list --remote
@@ -293,10 +281,10 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
 
 ### macOS: pre-installed or `brew`
 
-MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%= gemspec.name %> using `sudo` :
+MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%=gemspec.name%> using `sudo` :
 
 ```
-$ sudo gem install <%= gemspec.name %><%=geminstadd%>
+$ sudo gem install <%=gemspec.name%><%=geminstadd%>
 ```
 
 Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
@@ -384,18 +372,18 @@ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
 
 or restore the `$HOME/.aspera` folder for the user.
 
-## <a name="the_gem"></a>`<%= gemspec.name %>` gem
+## <a name="the_gem"></a>`<%=gemspec.name%>` gem
 
 Once you have Ruby and rights to install gems: Install the gem and its dependencies:
 
 ```
-# gem install <%= gemspec.name %><%=geminstadd%>
+# gem install <%=gemspec.name%><%=geminstadd%>
 ```
 
 To upgrade to the latest version:
 
 ```
-# gem update <%= gemspec.name %>
+# gem update <%=gemspec.name%>
 ```
 
 <%=tool%> checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
@@ -445,8 +433,8 @@ by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/
 <%=tool%> will detect most of Aspera transfer products in standard locations and use the first one found.
 Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
 
-Several methods are provided on how to start a transfer. Use of a local client is one of them, but
-other methods are available. Refer to section: [Transfer Agents](#agents)
+Several methods are provided to start a transfer.
+Use of a local client ([`direct`](#direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
 
 ## <a name="offline_install"></a>Offline Installation (without internet)
 
@@ -477,7 +465,7 @@ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
 
 # <a name="cli"></a>Command Line Interface: <%=tool%>
 
-The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
+The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
 
 * IBM Aspera High Speed Transfer Server (FASP and Node)
 * IBM Aspera on Cloud (including ATS)
@@ -492,7 +480,7 @@ The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which inte
 * Supports most Aspera server products (on-premise and SaaS)
 * Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
 * Supports Commands, Option values and Parameters shortcuts
-* FASP [Transfer Agents](#agents) can be: FaspManager (local ascp), or Connect Client, or any transfer node
+* FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
 * Transfer parameters can be altered by modification of _transfer-spec_, this includes requiring multi-session
 * Allows transfers from products to products, essentially at node level (using the node transfer agent)
 * Supports FaspStream creation (using Node API)
@@ -616,12 +604,12 @@ The style of output can be set using the `format` parameter, supporting:
 * `yaml` : YAML
 * `csv` : Comma Separated Values
 
-### Filtering columns for `object_list`
+### <a name="option_select"></a>Option: `select`: Filter on columns values for `object_list`
 
 Table output can be filtered using the `select` parameter. Example:
 
 ```
-$ <%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"per_page":1000,"page":1,"sort":"name"}' --select=@json:'{"ats_admin":true}'
+$ <%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
 :...............................:..................................:...........:
 :             name              :              email               : ats_admin :
 :...............................:..................................:...........:
@@ -768,19 +756,15 @@ C:\Users\Kenji\.aspera\<%=cmd%>
 On the first execution of <%=tool%>, an empty configuration file is created in the configuration folder.
 Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
 
-Although the file is a standard YAML file, <%=tool%> provides commands to read and modify it
-using the `config` command.
+Although the file is a standard YAML file, <%=tool%> provides commands to read and modify it using the `config` command.
 
-All options for <%=tool%> commands can be set on command line, or by env vars, or using <%=prsts%> in the configuratin file.
+All options for <%=tool%> can be set on command line, or by env vars, or using <%=prsts%> in the configuratin file.
 
-A configuration file provides a way to define default values, especially
-for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
+A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
 
-The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml`
-(this can be overriden with option `--config-file=path` or equivalent env var).
+The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml` (this can be overriden with option `--config-file=path` or equivalent env var).
 
-So, finally, the configuration file is simply a catalog of pre-defined lists of options,
-called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
+The configuration file is simply a catalog of pre-defined lists of options, called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
 
 ### <a name="lprt"></a><%=prstt%>
 
@@ -880,7 +864,7 @@ demo_server:
 We can see here:
 
 * The configuration was created with CLI version 0.3.7
-* the default <%=prst%> to load for plugin "server" is : `demo_server`
+* the default <%=prst%> to load for `server` plugin is : `demo_server`
 * the <%=prst%> `demo_server` defines some parameters: the URL and credentials
 * the default <%=prst%> to load in any case is : `cli_default`
 
@@ -911,26 +895,26 @@ my_aoc_org:
 
 So, the key file will be read only at execution time, but not be embedded in the configuration file.
 
+### Options evaluation order
+
+Some options are global, some options are available only for some plugins. (the plugin is the first level command).
+
 Options are loaded using this algorithm:
 
-* if option '--preset=xxxx' is specified (or -Pxxxx), this reads the <%=prst%> specified from the configuration file.
-    * else if option --no-default (or -N) is specified, then dont load default
-    * else it looks for the name of the default <%=prst%> in section "default" and loads it
-* environment variables are evaluated
-* command line options are evaluated
+* If option `--no-default` (or `-N`) is specified, then no default value is loaded is loaded for the plugin
+* else it looks for the name of the plugin as key in section `default`, the value is the name of the default <%=prst%> for it, and loads it.
+* If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the <%=prst%> specified from the configuration file, or of the value is a Hash, it uses it as options values.
+* Environment variables are evaluated
+* Command line options are evaluated
 
 Parameters are evaluated in the order of command line.
 
-To avoid loading the default <%=prst%> for a plugin, just specify a non existing configuration: `-Pnone`
+To avoid loading the default <%=prst%> for a plugin, use: `-N`
 
 On command line, words in parameter names are separated by a dash, in configuration file, separator
 is an underscore. E.g. --xxx-yyy  on command line gives xxx_yyy in configuration file.
 
-Note: before version 0.4.5, some keys could be ruby symbols, from 0.4.5 all keys are strings. To
-convert olver versions, remove the leading ":" in front of keys.
-
-The main plugin name is *config*, so it is possible to define a default <%=prst%> for
-the main plugin with:
+The main plugin name is `config`, so it is possible to define a default <%=prst%> for the main plugin with:
 
 ```
 $ <%=cmd%> config id cli_default set interactive no
@@ -943,6 +927,18 @@ A <%=prst%> value can be removed with `unset`:
 $ <%=cmd%> config id cli_default unset interactive
 ```
 
+Example: Define options using command line:
+
+```
+$ <%=cmd%> -N --url=x --password=y --username=y node --show-config
+```
+
+Example: Define options using a hash:
+
+```
+$ <%=cmd%> -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
+```
+
 ### Examples
 
 For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
@@ -1071,7 +1067,7 @@ $ <%=cmd%> config ascp info
 ...
 ```
 
-### Selection of local `ascp`
+### Selection of `ascp` location for [`direct`](#direct) agent
 
 By default, <%=tool%> uses any found local product with ascp, including SDK.
 
@@ -1115,7 +1111,7 @@ $ <%=cmd%> config ascp products list
 :.........................................:................................................:
 ```
 
-### Selection of local client
+### Selection of local client for `ascp` for [`direct`](#direct) agent
 
 If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
 
@@ -1170,7 +1166,7 @@ This [_transfer-spec_](#transferspec) will be executed by a transfer client, her
 
 There are currently 3 agents:
 
-* `direct` : a local execution of `ascp`
+* [`direct`](#direct) : a local execution of `ascp`
 * `connect` : use of a local Connect Client
 * `node` : use of an Aspera Transfer Node (potentially _remote_).
 * `httpgw` : use of an Aspera HTTP Gateway
@@ -1182,7 +1178,7 @@ will effectively push files to the related server from the agent node.
 <%=tool%> standadizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
 
 
-### <a name="direct"></a>Direct (local ascp using FASPManager API)
+### <a name="direct"></a>Direct (local ascp execution)
 
 By default <%=tool%> uses a local ascp, equivalent to specifying `--transfer=direct`.
 <%=tool%> will detect locally installed Aspera products.
@@ -1201,11 +1197,19 @@ The `transfer-info` accepts the following optional parameters:
 <tr><td>spawn_timeout_sec</td><td>Float</td><td>3</td><td>Multi session</td><td>Verification time that ascp is running</td></tr>
 <tr><td>spawn_delay_sec</td><td>Float</td><td>2</td><td>Multi session</td><td>Delay between startup of sessions</td></tr>
 <tr><td>wss</td><td>Bool</td><td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
+<tr><td>multi_incr_udp</td><td>Bool</td><td>true</td><td>Multi Session</td><td>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)</td></tr>
 <tr><td>resume</td><td>Hash</td><td>nil</td><td>Resumer parameters</td><td>See below</td></tr>
 </table>
 
 Resume parameters:
 
+In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
+Sleep between iteration is:
+
+```
+max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
+```
+
 <table>
 <tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
 <tr><td>iter_max</td><td>int</td><td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
@@ -1218,7 +1222,7 @@ Examples:
 
 ```
 $ <%=cmd%> ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
-$ <%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5}'
+$ <%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
 ```
 
 ### IBM Aspera Connect Client GUI
@@ -1272,7 +1276,7 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
 
 If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
 
-It is possible to specify ascp options when the `transfer` option is set to `direct` using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
+It is possible to specify ascp options when the `transfer` option is set to [`direct`](#direct) using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
 
 The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
 
@@ -1283,11 +1287,31 @@ A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the co
 
 ## <a name="transferparams"></a>Transfer Parameters
 
-All standard _transfer-spec_ parameters can be overloaded. To display parameters,
-run in debug mode (--log-level=debug). [_transfer-spec_](#transferspec) can
-also be saved/overridden in the config file.
+All standard _transfer-spec_ parameters can be speficied.
+[_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
 
-<%= File.read(ENV['INCL_TRSPEC']).gsub(/.*<body>(.*)<\/body>.*/m,'\1') %>
+References:
+
+* [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22)&rarr;/opt/transfers
+* [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22)&rarr;Guides&rarr;API Ref&rarr;Transfer Spec V1
+
+Parameters can be displayed with commands:
+
+```
+$ <%=cmd%> config ascp spec
+$ <%=cmd%> config ascp spec --select=@json:'{"f":"Y"}' --fields=-f,n,c
+```
+
+Columns:
+
+* D=Direct (local `ascp` execution)
+* N=Node API
+* C=Connect Client
+* arg=`ascp` argument or environment variable
+
+Fields with EX_ prefix are extensions to transfer agent [`direct`](#direct). (only in <%=tool%>).
+
+<%=spec_table%>
 
 ### Destination folder for transfers
 
@@ -1303,8 +1327,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
 
 ### List of files for transfers
 
-When uploading, downloading or sending files, the user must specify the list of files to transfer.
-Most of the time, the list of files to transfer will be simply specified on the command line:
+When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files (Extensed value) is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
+So, by default, the list of files to transfer will be simply specified on the command line:
 
 ```
 $ <%=cmd%> server upload ~/mysample.file secondfile
@@ -1316,7 +1340,7 @@ This is equivalent to:
 $ <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
 ```
 
-More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
+More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
 
 Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
 all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
@@ -1339,13 +1363,15 @@ For ease of use and flexibility, the list of files to transfer is specified by t
 --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
 ```
 
-* Although not recommended, because it applies *only* to the `local` transfer agent (i.e. bare ascp), it is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`. In that case, one must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overriden by the bare ascp command line provided.
+* Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
 
 ```
 --sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
 ```
 
-In case the file list is provided on the command line (i.e. using `--sources=@args` or `--sources=<Array>`, but not `--sources=@ts`), the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
+This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overriden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
+
+In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
 
 * `list` : (default) the path of destination is the same as source
 * `pair` : in that case, the first element is the first source, the second element is the first destination, and so on.
@@ -1356,6 +1382,8 @@ Example:
 $ <%=cmd%> server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
 ```
 
+Internally, when transfer agent [`direct`](#direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pait-list` is provided in `ts` in `EX_ascp_args`.
+
 Note the special case when the source files are located on "Aspera on Cloud", i.e. using access keys and the `file id` API:
 
 * All files must be in the same source folder.
@@ -1369,7 +1397,7 @@ Source files are located on "Aspera on cloud", when :
 
 ### <a name="multisession"></a>Support of multi-session
 
-Multi session, i.e. starting a transfer of a file set using multiple sessions is supported on "direct" and "node" agents, not yet on connect.
+Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
 
 * when agent=node :
 
@@ -1390,6 +1418,7 @@ shall be preferred.
 
 Multi-session spawn is done by <%=tool%>.
 
+When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
 
 ### Examples
 
@@ -1419,22 +1448,161 @@ Multi-session spawn is done by <%=tool%>.
 
 
 
-## <a name="scheduling"></a>Scheduling an exclusive execution
+## <a name="scheduling"></a>Lock for exclusive execution
+
+In some conditions, it may be desirable to ensure that <%=tool%> is not executed several times in parallel.
+
+For instance when <%=tool%> is executed automatically on a schedule basis, one generally desire that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
+
+* Executing instances may pile-up and kill the system
+* The same file may be transfered by multiple instances at the same time.
+* `preview` may generate the same files in multiple instances.
+
+Usually the OS native scheduler already provides some sort of protection against parallel execution:
+
+* The Windows scheduler does this by default
+* Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
+
+```
+/usr/bin/flock -w 0 /var/cron.lock ascli ...
+```
+
+<%=tool%> natively supports a locking mechanism with option `lock_port`.
+(Technically, this opens a local TCP server port, and fails if this port is already used, providing a local lock. Lock is released when process exits).
+
+Example:
+
+Run this same command in two separate terminals within less than 30 seconds:
+
+```
+ascli config echo @ruby:'sleep(30)' --lock-port=12345
+```
+
+The first instance will sleep 30 seconds, the second one will immediately exit like this:
+
+```
+WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
+```
+
+## "Proven&ccedil;ale"
+
+`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
+, some are provided om shared libraries and must be activated. (e.g. using `trapd`)
+
+The list of supported *PVCL* adapters can be retried with command:
+
+```
+$ <%=cmd%> conf ascp info
++--------------------+-----------------------------------------------------------+
+| key                | value                                                     |
++--------------------+-----------------------------------------------------------+
+-----8<----snip---------
+| product_name       | IBM Aspera SDK                                            |
+| product_version    | 4.0.1.182389                                              |
+| process            | pvcl                                                      |
+| shares             | pvcl                                                      |
+| noded              | pvcl                                                      |
+| faux               | pvcl                                                      |
+| file               | pvcl                                                      |
+| stdio              | pvcl                                                      |
+| stdio-tar          | pvcl                                                      |
++--------------------+-----------------------------------------------------------+
+```
+
+Here we can see the adapters: `process`, `shares`, `noded`, `faux`, `file`, `stdio`, `stdio-tar`.
+
+Those adapters can be used wherever a file path is used in `ascp` including configuration. They act as a pseudo "drive".
+
+The simplified format is:
+
+```
+<adapter>:///<sub file path>?<arg1>=<val1>&...
+```
+
+One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
+
+## <a name="faux_testing"></a>`faux:` for testing
+
+This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not <%=tool%>
+
+This adapter can be used to simulate a file or a directory.
+
+To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form `faux:///fname?fsize` where:
+
+* `fname` is the name that will be assigned to the file on the destination
+* `fsize` is the number of bytes that will be sent (in decimal).
+
+Note that the character `?` is a special shell character (wildcard), so `faux` file specification on command line shall be protected (using `\?` and `\&` or using quotes). If not, the shell may give error: `no matches found` or equivalent.
+
+For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2^20, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2^60. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of system memory unless `faux://`.
+
+To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
+
+```
+faux:///dirname?<arg1>=<val1>&...
+```
+
+`dirname` is the folder name and can contain `/` to specify a subfolder.
+
+Supported arguments are:
+
+<table>
+<tr><th>name</th><th>type</th><th>default</th><th>description</th></tr>
+<tr><td>count</td><td>int</td><td>mandatory</td><td>number of files</td></tr>
+<tr><td>file</td><td>string</td><td>file</td><td>basename for files</td></tr>
+<tr><td>size</td><td>int</td><td>0</td><td>size of first file.</td></tr>
+<tr><td>inc</td><td>int</td><td>0</td><td>increment applied to determine next file size</td></tr>
+<tr><td>seq</td><td>sequential<br/>random</td><td>sequential</td><td>sequence in determining next file size</td></tr>
+<tr><td>buf_init</td><td>none<br/>zero<br/>random</td><td>zero</td><td>how source data initialized.<br/>Option 'none' is not allowed for downloads.</td></tr>
+</table>
+
 
-It is possible to ensure that a given command is only run once at a time with parameter: `--lock-port=nnnn`. This is especially usefull when scheduling a command on a regular basis, for instance involving transfers, and a transfer may last longer than the execution period.
+The sequence parameter is applied as follows:
 
-This opens a local TCP server port, and fails if this port is already used, providing a local lock.
+* If `seq` is `random` then each file size is:
 
-This option is used when the tools is executed automatically, for instance with "preview" generation.
+  * size +/- (inc * rand())
+  * Where rand is a random number between 0 and 1
+  * Note that file size must not be negative, inc will be set to size if it is greater than size
+  * Similarly, overall file size must be less than 8 * 2^60. If size + inc is greater, inc will be reduced to limit size + inc to 7 * 2^60.
 
-Usually the OS native scheduler shall already provide some sort of such protection (windows scheduler has it natively, linux cron can leverage `flock`).
+* If `seq` is `sequential` then each file size is:
+
+  * size + ((fileindex - 1) * inc)
+  * Where first file is index 1
+  * So file1 is size bytes, file2 is size + inc bytes, file3 is size + inc * 2 bytes, etc.
+  * As with random, inc will be adjusted if size + (count * inc) is not less then 8 ^ 2^60.
+
+Filenames generated are of the form: `<file>_<00000 . . . count>_<filesize>`
+
+To discard data at the destination, the destination argument is set to `faux://` .
+
+Examples:
+
+* Upload 20 gigabytes of random data to file myfile to directory /Upload
+
+```
+$ <%=cmd%> server upload faux:///myfile\?20g --to-folder=/Upload
+```
+
+* Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
+
+```
+$ <%=cmd%> server upload /tmp/sample --to-folder=faux://
+```
+
+* Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 M - 2 bytes, with the basename of each file being `testfile` to /Upload
+
+```
+$ <%=cmd%> server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
+```
 
 ## <a name="commands"></a>Sample Commands
 
 A non complete list of commands used in unit tests:
 
 ```
-<%= File.read(ENV["INCL_COMMANDS"]) %>
+<%=File.read(ENV["INCL_COMMANDS"])%>
 ...and more
 ```
 
@@ -1442,7 +1610,7 @@ A non complete list of commands used in unit tests:
 
 ```
 $ <%=cmd%> -h
-<%= File.read(ENV["INCL_USAGE"]) %>
+<%=File.read(ENV["INCL_USAGE"])%>
 
 ```
 
@@ -1499,7 +1667,7 @@ For this, specify the option: `--use-generic-client=no`.
 
 This will guide you through the steps to create.
 
-## <a name="aocwizard"></a>Configuration: using manual setup
+## <a name="aocmanual"></a>Configuration: using manual setup
 
 If you used the wizard (recommended): skip this section.
 
@@ -1665,7 +1833,7 @@ To activate default use of JWT authentication for <%=tool%> using the <%=prst%>,
 
 * change auth method to JWT
 * provide location of private key
-* provide username to login as (OAuthg "subject")
+* provide username to login as (OAuth "subject")
 
 Execute:
 
@@ -1697,19 +1865,55 @@ It allows actions (create, update, delete) on "resources": users, group, nodes,
 
 Bulk operations are possible using option `bulk` (yes,no(default)): currently: create only. In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
 
-To get more resources when doing request add:
+#### Listing resources
+
+The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
+
+The option `query` can be optionally used. It expects a Hash using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`. Values are directly sent to the API call and used as a filter on server side.
+
+The following parameters are supported:
+
+* `q` : a filter on name of resource (case insensitive, matches if value is contained in name)
+* `sort`: name of fields to sort results, prefix with `-` for reverse order.
+* `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
+* `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
+* `page` : native api parameter, in general do not use (added by 
+* `per_page` : native api parameter, number of items par api call, in general do not use
+* Other specific parameters depending on resource type.
+
+Both `max` and `pmax` are processed internally in <%=tool%>, not included in actual API call and limit the number of successive pages requested to API. <%=tool%> will return all values using paging if not provided.
+
+Other parameters are directly sent as parameters to the GET request on API.
+
+`page` and `per_page` are normally added by <%=tool%> to build successive API calls to get all values if there are more than 1000. (AoC allows a maximum page size of 1000).
+
+`q` and `sort` are available on most resrouce types.
+
+Other parameters depend on the type of entity (refer to AoC API).
+
+Examples:
+
+* List users with `laurent` in name:
+
+```
+<%=cmd%> aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
+```
+
+* List users who logded-in before a date:
 
 ```
---query=@json:'{"per_page":10000}'
+<%=cmd%> aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
 ```
 
-other query parameters can be used:
+* List external users and sort in reverse alphabetical order using name:
+
 ```
---query=@json:'{"member_of_any_workspace":true}'
---query=@json:'{"q":"laurent"}'
+<%=cmd%> aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
 ```
 
-Refer to the AoC API for full list of query parameters.
+Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
+
+Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
 
 #### Access Key secrets
 
@@ -1763,10 +1967,10 @@ $ <%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id
 : 98398 : dummyuser1@example.com :
 : 98399 : dummyuser2@example.com :
 :.......:........................:
-$ thelist=$(echo $(<%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email --field=id --format=csv)|tr ' ' ,)
+$ thelist=$(<%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
 $ echo $thelist
-98398,98399
-$ <%=cmd%> aoc admin res user --bulk=yes --id=@json:[$thelist] delete
+["113501","354061"]
+$ <%=cmd%> aoc admin res user --bulk=yes --id=@json:"$thelist" delete
 :.......:.........:
 :  id   : status  :
 :.......:.........:
@@ -1775,6 +1979,14 @@ $ <%=cmd%> aoc admin res user --bulk=yes --id=@json:[$thelist] delete
 :.......:.........:
 ```
 
+* <a name="deactuser"></a>Find deactivated users since more than 2 years
+
+```
+ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
+```
+
+To delete them use the same method as before
+
 * Display current user's workspaces
 
 ```
@@ -1802,17 +2014,20 @@ $ <%=cmd%> aoc admin resource node --name=_node_name_ --secret=_secret_ v4 acces
 $ <%=cmd%> aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
 ```
 
-              # page=1&per_page=10&q=type:(file_upload+OR+file_delete+OR+file_download+OR+file_rename+OR+folder_create+OR+folder_delete+OR+folder_share+OR+folder_share_via_public_link)&sort=-date
-              #events=@api_files.read('events',{'q'=>'type:(file_upload OR file_download)'})[:data]
-              # can add filters: tag=aspera.files.package_id%3DLA8OU3p8w
-              #'tag'=>'aspera.files.package_id%3DJvbl0w-5A'
+Examples of query (TODO: cleanup):
+
+```
+{"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
+
+{"tag":"aspera.files.package_id=LA8OU3p8w"}
+
               # filter= 'id', 'short_summary', or 'summary'
               # count=nnn
               # tag=x.y.z%3Dvalue
               # iteration_token=nnn
               # after_time=2016-05-01T23:53:09Z
               # active_only=true|false
-
+```
 
 * Display node events (events)
 
@@ -1823,7 +2038,7 @@ $ <%=cmd%> aoc admin res node --secret=_secret_ v3 events
 * display members of a workspace
 
 ```
-$ <%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
+$ <%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
 :.............:.........:..................................:
 : member_type : manager :           member.email           :
 :.............:.........:..................................:
@@ -1844,27 +2059,27 @@ other query parameters:
 
 * <a name="aoc_sample_member"></a>add all members of a workspace to another workspace
 
-a- get id of first workspace
+a- Get id of first workspace
 
 ```
 WS1='First Workspace'
 WS1ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
 ```
 
-b- get id of second workspace
+b- Get id of second workspace
 
 ```
 WS2='Second Workspace'
 WS2ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
 ```
 
-c- extract membership information and change workspace id
+c- Extract membership information
 
 ```
-$ <%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"per_page":10000,"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
+$ <%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
 ```
 
-d- convert to creation data for second workspace:
+d- Convert to creation data for second workspace:
 
 ```
 grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
@@ -1876,16 +2091,16 @@ or, using jq:
 jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
 ```
 
-e- add members to second workspace
+e- Add members to second workspace
 
 ```
 $ <%=cmd%> aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
 ```
 
-* get users who did not log since a date
+* Get users who did not log since a date
 
 ```
-$ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"per_page":10000,"q":"last_login_at:<2018-05-28"}'
+$ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
 :...............................:
 :             email             :
 :...............................:
@@ -1894,10 +2109,10 @@ $ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"per_page":100
 :...............................:
 ```
 
-* list "Limited" users
+* List "Limited" users
 
 ```
-$ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"per_page":10000}' --select=@json:'{"member_of_any_workspace":false}'
+$ <%=cmd%> aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
 ```
 
 * Perform a multi Gbps transfer between two remote shared folders
@@ -1940,6 +2155,7 @@ $ <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folde
 ```
 
 * create registration key to register a node
+
 ```
 $ <%=cmd%> aoc admin res admin/client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
 jfqslfdjlfdjfhdjklqfhdkl
@@ -1959,6 +2175,19 @@ $ <%=cmd%> aoc admin res admin/client list --fields=id --format=csv|<%=cmd%> aoc
 +-----+---------+
 ```
 
+* List packages in a given shared inbox
+
+First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
+(To find out available filters, consult the API definition, or use the web interface in developer mode).
+
+Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
+
+```
+shbxid=$(ascli aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
+
+ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
+```
+
 ## Shared folders
 
 * list shared folders in node
@@ -1991,7 +2220,7 @@ Notes:
 
 * the `value` parameter can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package to find attributes.
 * to provide the list of recipients, use fields: "recipients" and/or "bcc_recipients". <%=cmd%> will resolve the list of email addresses to expected user ids.
-* a recipîent can be a shared inbox, in this case just use the name of the shared inbox as recipient.
+* a recipient can be a shared inbox, in this case just use the name of the shared inbox as recipient.
 * If a recipient is not already registered and the workspace allows external users, then the package is sent to an external user, and
   * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account.
   * if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
@@ -2000,7 +2229,13 @@ Examples:
 
 ```
 $ <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
+```
+
+```
 $ <%=cmd%> aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
+```
+
+```
 $ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
 ```
 
@@ -2016,7 +2251,7 @@ $ <%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
 * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
 * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
 
-Typically, one would regularly execute this command on a regular basis, using the method of your choice:
+Typically, one would execute this command on a regular basis, using the method of your choice:
 
 * Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
 * Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
@@ -2024,45 +2259,45 @@ Typically, one would regularly execute this command on a regular basis, using th
 
 ## Download Files
 
-Download of files is straightforward with a specific syntax for the `aspera files download` action: Like other commands the source file list is provided as  a list with the `sources` option. Nevertheless, consider this:
+Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as  a list with the `sources` option. Nevertheless, consider this:
 
 * if only one source is provided, it is downloaded
 * if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
 
 ## Find Files
 
-The command `aspera files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
+The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
 
 The expression can be of 3 formats:
 
-* empty (default) : all files, equivalent to: `exec:true`
-* not starting with `exec:` : the expression is a regular expression, using ruby regex syntax. equivalent to: `exec:f['name'].match(/expression/)`
+* empty (default) : all files, equivalent to value: `exec:true`
+* not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
 
 For instance, to find files with a special extension, use `--value='\.myext$'`
 
-* starting with `exec:` : the ruby code after the prefix is executed for each entry found. the entry variable name is `f`. the file is displayed if the result is true;
+* starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
 
-Examples of expressions: (think to prefix with `exec:` and put in single quotes using bash)
+Examples of expressions: (using like this: `--value=exec:'<expression>'`)
 
-* find files more recent than 100 days
+* Find files more recent than 100 days
 
 ```
 f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
 ```
 
-* expression to find files older than 1 year on a given node and store in file list
+* Find files older than 1 year on a given node and store in file list
 
 ```
 $ <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
 ```
 
-* delete the files, one by one
+* Delete the files, one by one
 
 ```
 $ cat my_file_list.txt|while read path;do echo <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
 ```
 
-* delete the files in bulk
+* Delete the files in bulk
 
 ```
 cat my_file_list.txt | <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
@@ -2076,19 +2311,40 @@ The activity app can be queried with:
 $ <%=cmd%> aoc admin analytics transfers
 ```
 
-It can also support filters and send notification email with a template:
+It can also support filters and send notification using option `notif_to`. a template is defined using option `notif_template` :
+
+`mytemplate.erb`:
+
+```
+From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
+To: <<%='<'%>%=ev['user_email']%>>
+Subject: <%='<'%>%=ev['files_completed']%> files received
+
+Dear <%='<'%>%=ev[:user_email.to_s]%>,
+We received <%='<'%>%=ev['files_completed']%> files for a total of <%='<'%>%=ev['transferred_bytes']%> bytes, starting with file:
+<%='<'%>%=ev['content']%>
 
+Thank you.
 ```
-$ <%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=123455 \
+The environment provided contains the following additional variable:
+
+* ev : all details on the transfer event
+
+Example:
+
+```
+$ <%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=12345 \
 --query=@json:'{"status":"completed","direction":"receive"}' \
---notify=@json:'{"to":"<''%=transfer[:user_email.to_s]%>","subject":"<''%=transfer[:files_completed.to_s]%> files received","body":"Dear <''%=transfer[:user_email.to_s]%>\nWe received <''%=transfer[:files_completed.to_s]%> files for a total of <''%=transfer[:transferred_bytes.to_s]%> bytes, starting with file:\n<''%=transfer[:content.to_s]%>\n\nThank you."}'
+--notif-to=active --notif-template=@file:mytemplate.erb
 ```
 
+Options:
+
 * `once_only` keep track of last date it was called, so next call will get only new events
 * `query` filter (on API call)
 * `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
 
-Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. here the period is [date of previous execution]..[now].
+Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
 
 ## Using specific transfer ports
 
@@ -2149,7 +2405,37 @@ $ <%=cmd%> ats api_key create
 +--------+----------------------------------------------+
 $ <%=cmd%> config id my_ibm_ats update --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
 ```
+## Cross Organization transfers
+
+It is possible to transfer files directly between organizations without having to first download locally and then upload...
+
+Although optional, the creation of <%=prst%> is recommended to avoid placing all parameters in the command line.
+
+Procedure to send a file from org1 to org2:
+
+* Get access to Organization 1 and create a <%=prst%>: e.g. `org1`, for instance, use the [Wizard](#aocwizard)
+* Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
+* Get access to Organization 2 and create a <%=prst%>: e.g. `org2`
+* Check that access works and locate the destination folder `mydestfolder`
+* execute the following:
+
+```
+$ <%=cmd%> -Porg1 aoc files node_info /mydestfolder --format=json --display=data | <%=cmd%> -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
+```
 
+Explanation:
+
+* `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
+* `files node_info /mydestfolder` generate transfer information including node api credential and root id, suitable for the next command
+* `--format=json` format the output in JSON (instead of default text table)
+* `--display=data` display only the result, and remove other information, such as workspace name
+* `|` the standard output of the first command is fed into the second one
+* `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
+* `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
+* `--transfer=node` use transfer agent type `node` instead of default [`direct`](#direct)
+* `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
+
+Note that when using a POSIX shell, another possibility to write `cmd1 | cmd2 --transfer-info=@json:stdin:` is `cmd2 --transfer-info=@json:$(cmd1)` instead of ``
 ## Examples
 
 Example: create access key on softlayer:
@@ -2195,6 +2481,14 @@ This plugin works at FASP level (SSH/ascp/ascmd) and does not use the node API.
 
 Both password and SSH keys auth are supported.
 
+If not username is provided, the default transfer user `xfer` is used.
+
+If no ssh password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used.
+
+```
+$ <%=cmd%> server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
+```
+
 Multiple SSH key paths can be provided. The value of the parameter `ssh_keys` can be a single value or an array. Each value is a path to a private key and is expanded ("~" is replaced with the user's home folder).
 
 Examples:
@@ -2232,7 +2526,7 @@ This means that you dont have such an ssh agent running:
 $ <%=cmd%> server --ssh-options=@ruby:'{use_agent: false}' ...
 ```
 
-This can also be set as default using a preset
+This can also be set as default using a preset.
 
 ## Example
 
@@ -2252,18 +2546,36 @@ This creates a <%=prst%> "aspera_demo_server" and set it as default for applicat
 
 This plugin gives access to capabilities provided by HSTS node API.
 
-## Simple Operations
+## File Operations
 
 It is possible to:
 * browse
 * transfer (upload / download)
 * ...
 
+For transfers, it is possible to control how transfer is authorized using option: `token_type`:
+
+* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
+* `basic` : transfer spec is created like this:
+
+```
+{
+  "remote_host": address of node url,
+  "remote_user": "xfer",
+  "ssh_port": 33001,
+  "token": "Basic <base 64 encoded user/pass>",
+  "direction": send/recv
+}
+```
+
+* `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
+
 ## Central
 
 The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transfered files.
 
 Filtering can be applied:
+
 ```
 $ <%=cmd%> node central file list
 ```
@@ -2289,7 +2601,6 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
 * Start watchd and watchfolderd services running as a system user having access to files
 * configure a watchfolder to define automated transfers
 
-
 ```
 $ <%=cmd%> node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
 $ <%=cmd%> node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
@@ -2335,16 +2646,19 @@ to download files.
 $ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
 ```
 
-# Plugin: IBM Aspera Faspex
+# Plugin: IBM Aspera Faspex5
 
-Notes:
+3 authentication methods are supported:
 
-* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
-* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+* jwt
+* web
+* boot
 
-## Faspex 5 Beta1
+For JWT, create an API client in Faspex with jwt support, and use: `--auth=jwt`.
 
-As the web UI does not yet allow adding API client yet, the way to use CLI is:
+For web method, create an API client in Faspex, and use: --auth=web
+
+For boot method: (will be removed in future)
 
 * open a browser
 * start developer mode
@@ -2357,30 +2671,74 @@ Use it as password and use `--auth=boot`.
 $ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
 ```
 
-An JWT client can then be created with a private key:
+Ready to use Faspex5 with CLI.
+
+# Plugin: IBM Aspera Faspex (4.x)
+
+Notes:
+
+* The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+* For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
+
+## Listing Packages
+
+Command: `faspex package list`
+
+### Option `box`
+
+By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
+
+### Option `recipient`
+
+A user can receive a package because the recipient is:
+
+* the user himself (default)
+* the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
+
+### Option `query`
+
+As inboxes may be large, it is possible to use the following query parameters:
+
+* `count` : (native) number items in one API call (default=0, equivalent to 10)
+* `page` : (native) id of page in call (default=0)
+* `startIndex` : (native) index of item to start, default=0, oldest index=0
+* `max` : maximum number of items
+* `pmax` : maximum number of pages
+
+(SQL query is `LIMIT <startIndex>, <count>`)
+
+The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "Services (API v.3)".
+
+If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`). By default page is `0` (`10`), it can be increased to have less calls.
+
+### Example
 
 ```
-$ jsonk=$(openssl rsa -in  ~/.aspera/ascli/aspera_on_cloud_key -pubout 2> /dev/null | sed -e :a -e N -e '$!ba' -e 's/\n/\\n/g')
-$ <%=cmd%> faspex5 -Pf5boot auth_client create --value=@json:'{"name":"hello","client_type":"public","redirect_uris":["https://localhost:12345"],"allow_jwt_grant":true,"public_key":"'$jsonk'"}'
+$ <%=cmd%> faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
 ```
 
-or deleted by name:
+List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
+
+## Receiving a Package
+
+The command is `package recv`, possible methods are:
+
+* provide a package id with option `id`
+* provide a public link with option `link`
+* provide a `faspe:` URI with option `link`
 
 ```
-$ id=$(<%=cmd%> faspex5 auth_client list --select=@json:'{"name":"hello"}' --fields=client_id --format=csv)
-$ <%=cmd%> faspex5 auth_client delete --id=$id
+$ <%=cmd%> faspex package recv --id=12345
+$ <%=cmd%> faspex package recv --link=faspe://...
 ```
 
-Once the API client is created with a client_id and secret (result of create command), create a configuration:
+If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
 
 ```
-$ <%=cmd%> conf id f5 update --url=https://localhost/aspera/faspex --auth=jwt --client-id=abcd --client-secret=def --username=pierre@example.com --private-key=@val:@file:~/.aspera/ascli/aspera_on_cloud_key
-$ <%=cmd%> conf id default set faspex5 f5
-``` 
-
-Ready to use Faspex5 with CLI.
+$ <%=cmd%> faspex package list --recipient='*thedropboxname'
+```
 
-Once the graphical registration form exist, ther bootstrap method can be removed.
+if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
 
 ## Sending a Package
 
@@ -2399,7 +2757,19 @@ Additional optional parameters in `delivery_info`:
 * Package Note: : `"note":"note this and that"`
 * Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
 
-## operation on dropboxes
+## Email notification on transfer
+
+Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
+
+Example:
+
+```
+$ <%=cmd%> faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>\nTo: <<%='<'%>%=to%>>\nSubject: Package sent: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
+```
+
+In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
+
+## Operation on dropboxes
 
 Example:
 
@@ -2409,7 +2779,7 @@ $ <%=cmd%> faspex v4 dropbox list
 $ <%=cmd%> faspex v4 dropbox delete --id=36
 ```
 
-## remote sources
+## Remote sources
 
 Faspex lacks an API to list the contents of a remote source (available in web UI). To workaround this,
 the node API is used, for this it is required to add a section ":storage" that links
@@ -2460,29 +2830,38 @@ $ for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p
 # Plugin: IBM Cloud Object Storage
 
 The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
-It uses the same transfer service as Aspera on Cloud.
-see [https://status.aspera.io](https://status.aspera.io)
+It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
+Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
+
+There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the forst method. If you dont have credentials but have access to the IBM Cloud console, then use the second method.
 
-Required options are either:
+## Using endpoint, apikey and Ressource Instance ID (CRN)
+
+If you have those parameters already, then following options shall be provided:
 
 * `bucket` bucket name
 * `endpoint` storage endpoint url, e.g. https://s3.hkg02.cloud-object-storage.appdomain.cloud
 * `apikey` API Key
 * `crn` resource instance id
 
-or:
+For example, let us create a default configuration:
 
-* `bucket` bucket name
-* `region` bucket region, e.g. eu-de
-* `service_credentials` see below
+```
+$ <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
+$ <%=cmd%> conf id default set cos mycos
+```
 
-Service credentials are directly created using the IBM cloud web ui. Navigate to:
+Then, jump to the transfer example.
+
+## Using service credential file
+
+If you are the COS administrator and dont have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
 
 Navigation Menu &rarr; Resource List &rarr; Storage &rarr; Cloud Object Storage &rarr; Service Credentials &rarr; &lt;select or create credentials&gt; &rarr; view credentials &rarr; copy
 
 Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
 
-or using the CLI:
+or using the IBM Cloud CLI:
 
 ```
 $ ibmcloud resource service-keys
@@ -2513,33 +2892,38 @@ The field `resource_instance_id` is for option `crn`
 
 The field `apikey` is for option `apikey`
 
-Endpoints for regions can be found by querying the `endpoints` URL.
+(If needed: endpoints for regions can be found by querying the `endpoints` URL.)
+
+The required options for this method are:
+
+* `bucket` bucket name
+* `region` bucket region, e.g. eu-de
+* `service_credentials` see below
 
-For convenience, let us create a default configuration, for example:
+For example, let us create a default configuration:
 
 ```
 $ <%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
 $ <%=cmd%> conf id default set cos mycos
 ```
 
-or using direct parameters:
+## Operations, transfers
 
-```
-$ <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
-$ <%=cmd%> conf id default set cos mycos
-```
+Let's assume you created a default configuration from once of the two previous steps (else specify the access options on command lines).
 
-Now, Ready to do operations, a subset of "node" plugin operations are supported, basically node API:
+A subset of `node` plugin operations are supported, basically node API:
 
 ```
-$ <%=cmd%> cos node browse /
-$ <%=cmd%> cos node upload myfile.txt
+$ <%=cmd%> cos node info
+$ <%=cmd%> cos node upload 'faux:///sample1G?1g'
 ```
 
+Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
+
 # Plugin: IBM Aspera Sync
 
-A basic plugin to start an "async" using <%=tool%>. The main advantage is the possibility
-to start from ma configuration file, using <%=tool%> standard options.
+A basic plugin to start an "async" using <%=tool%>.
+The main advantage is the possibility to start from ma configuration file, using <%=tool%> standard options.
 
 # Plugin: Preview
 
@@ -2631,8 +3015,10 @@ yum install -y ImageMagick optipng
 
 ### Video: FFmpeg
 
+The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
+
 ```
-curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && tar xJvf - && rm -f /opt/ffmpeg /usr/bin/{ffmpeg,ffprobe} && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
+curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
 ```
 
 ### Office: Unoconv and Libreoffice
@@ -2817,14 +3203,14 @@ Aspera CLI can send email, for that setup SMTP configuration. This is done with
 The `smtp` option is a hash table (extended value) with the following fields:
 <table>
 <tr><th>field</th><th>default</th><th>example</th><th>description</th></tr>
-<tr><td>server</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
-<tr><td>tls</td><td>true</td><td>false</td><td>use of TLS</td></tr>
-<tr><td>port</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
-<tr><td>domain</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
-<tr><td>username</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
-<tr><td>password</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
-<tr><td>from\_email</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
-<tr><td>from\_name</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
+<tr><td>`server`</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
+<tr><td>`tls`</td><td>true</td><td>false</td><td>use of TLS</td></tr>
+<tr><td>`port`</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
+<tr><td>`domain`</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
+<tr><td>`username`</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
+<tr><td>`password`</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
+<tr><td>`from_email`</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
+<tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
 </table>
 
 ## Example of configuration:
@@ -2854,13 +3240,52 @@ $ <%=cmd%> config id cli_default set smtp @val:@preset:smtp_google
 $ <%=cmd%> config id default set config cli_default
 ```
 
+## Email templates
+
+Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
+
+The template is the full SMTP message, including headers.
+
+The following variables are defined by default:
+
+* from_name
+* from_email
+* to
+
+Other variables are defined depending on context.
+
 ## Test
 
 Check settings with `smtp_settings` command. Send test email with `email_test`.
 
 ```
 $ <%=cmd%> config --smtp=@preset:smtp_google smtp
-$ <%=cmd%> config --smtp=@preset:smtp_google email sample.dest@example.com
+$ <%=cmd%> config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
+```
+
+## Notifications for transfer status
+
+An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
+
+To activate, use option `notif_to`.
+
+A default e-mail template is used, but it can be overriden with option `notif_template`.
+
+The environment provided contains the following additional variables:
+
+* subject
+* body
+* global_transfer_status
+* ts
+
+Example of template:
+
+```
+From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
+To: <<%='<'%>%=to%>>
+Subject: <%='<'%>%=subject%>
+
+Transfer is: <%='<'%>%=global_transfer_status%>
 ```
 
 # Tool: `asession`
@@ -2891,7 +3316,7 @@ Note that there are special "extended" [_transfer-spec_](#transferspec) paramete
   * `EX_loglevel` to change log level of the tool
   * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
 
-Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the "local" transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
+Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
 
 ## Comparison of interfaces
 
@@ -2934,7 +3359,7 @@ Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/asp
 
 ```
 $ asession -h
-<%= File.read(ENV["INCL_ASESSION"]) %>
+<%=File.read(ENV["INCL_ASESSION"])%>
 ```
 
 # Hot folder
@@ -2971,8 +3396,8 @@ Interesting ascp features are found in its arguments: (see ascp manual):
 Note that:
 
 * <%=tool%> takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
-* not all native ascp arguments are available as standard transfer_spec parameters
-* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the "local" transfer agent (not connect or node)
+* most, but not all native ascp arguments are available as standard transfer_spec parameters
+* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#direct) transfer agent (not connect or node)
 
 ### server side and configuration
 
@@ -2991,7 +3416,7 @@ $ <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=1
 
 ```
 
-The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
+The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
 
 # Aspera Health check and Nagios
 
@@ -3068,25 +3493,56 @@ So, it evolved into <%=tool%>:
 
 # Changes (Release notes)
 
-* <%= gemspec.version.to_s %>
+* <%=gemspec.version.to_s%>
+
+	* new: `aoc packages list` add possibility to add filter with option `query`
+	* new: `aoc admin res xxx list` now get all items by default #50
+	* new: `preset` option can specify name or hash value
+	* new: `node` plugin accepts bearer token and access key as credential
+	* new: `node` option `token_type` allows using basic token in addition to aspera type.
+	* change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default. 
+	* change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
+
+* 4.3.0
+
+	* new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#direct) transfer agent.
+	* new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
+
+* 4.2.2
+
+	* new: `faspex package list` retrieves the whole list, not just first page
+	* new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
+	* new: the error "Remote host is not who we expected" displays a special remediation message
+	* new: `conf ascp spec` displays supported transfer spec
+	* new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
+	* fix: space character in `faspe:` url are precent encoded if needed
+	* fix: `preview scan`: if file_id is unknown, ignore and continue scan
+	* change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then <%=tool%> exits with code 1 (instead of zero=success)
+	* change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
+
+* 4.2.1
+
+	* new: command `faspex package recv` supports link of type: `faspe:`
+	* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
+
+* 4.2.0
 
 	* new: command `aoc remind` to receive organization membership by email
 	* new: in `preview` option `value` to filter out on file name
 	* new: `initdemo` to initialize for demo server
-	* new: `direct` transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
+	* new: [`direct`](#direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
 	* fix: on Windows `conf ascp use` expects ascp.exe
 	* fix: (break) multi_session_threshold is Integer, not String
 	* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
 	* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
 	* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
-	* change: (break) parameters for resume in `transfer-info` for `direct` are now in sub-key `"resume"`
+	* change: (break) parameters for resume in `transfer-info` for [`direct`](#direct) are now in sub-key `"resume"`
 
 * 4.1.0
 
-  	* fix: remove keys from transfer spec and command line when not needed
-  	* fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist 
-  	* new: update documentation with regard to offline and docker installation
- 	* new: renamed command `nagios_check` to `health`
+	* fix: remove keys from transfer spec and command line when not needed 	* fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
+	* new: update documentation with regard to offline and docker installation
+	* new: renamed command `nagios_check` to `health`
 	* new: agent `http_gw` now supports upload
 	* new: added option `sdk_url` to install SDK from local file for offline install
 	* new: check new gem version periodically
@@ -3099,7 +3555,7 @@ So, it evolved into <%=tool%>:
 
 * 4.0.0
 
-	* now available as open source at [<%= gemspec.homepage %>](<%= gemspec.homepage %>) with general cleanup
+	* now available as open source at [<%=gemspec.homepage%>](<%=gemspec.homepage%>) with general cleanup
 	* changed default tool name from `mlia` to `ascli`
 	* changed `aspera` command to `aoc`
 	* changed gem name from `asperalm` to `aspera-cli`
@@ -3137,8 +3593,8 @@ So, it evolved into <%=tool%>:
 	* allow bulk delete in `aspera files` with option `bulk=yes`
 	* fix getting connect versions
 	* added section for Aix
-	* support all ciphers for `local`ascp (incl. gcm, etc..)
-	* added transfer spec param `apply_local_docroot` for `local`
+	* support all ciphers for [`direct`](#direct) agent (including gcm, etc..)
+	* added transfer spec param `apply_local_docroot` for [`direct`](#direct)
 
 * 0.11.4
 
@@ -3193,7 +3649,7 @@ So, it evolved into <%=tool%>:
 * 0.10.12
 
 	* added support for AoC node registration keys
-	* replaced option : `local_resume` with `transfer_info` for agent `direct`
+	* replaced option : `local_resume` with `transfer_info` for agent [`direct`](#direct)
 	* Transfer agent is no more a Singleton instance, but only one is used in CLI
 	* `@incps` : new extended value modifier
 	* ATS: no more provides access keys secrets: now user must provide it
@@ -3243,9 +3699,9 @@ So, it evolved into <%=tool%>:
 
 * 0.10.2
 
- 	* updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
- 	* added doc for "cargo" like actions
- 	* added doc for multi-session
+	* updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
+	* added doc for "cargo" like actions
+	* added doc for multi-session
 
 * 0.10.1
 
@@ -3273,22 +3729,22 @@ So, it evolved into <%=tool%>:
 
 * 0.9.33
 
-   * new command to display basic token of node
-   * new command to display bearer token of node in AoC
-   * the --fields= option, support +_fieldname_ to add a field to default fields
-   * many small changes
+	* new command to display basic token of node
+	* new command to display bearer token of node in AoC
+	* the --fields= option, support +_fieldname_ to add a field to default fields
+	* many small changes
 
 * 0.9.32
 
-   * all Faspex public links are now supported
-   * removed faspex operation recv_publink
-   * replaced with option `link` (consistent with AoC)
+	* all Faspex public links are now supported
+	* removed faspex operation recv_publink
+	* replaced with option `link` (consistent with AoC)
 
 * 0.9.31
 
-   * added more support for public link: receive and send package, to user or dropbox and files view.
-   * delete expired file lists
-   * changed text table gem from text-table to terminal-table because it supports multiline values
+	* added more support for public link: receive and send package, to user or dropbox and files view.
+	* delete expired file lists
+	* changed text table gem from text-table to terminal-table because it supports multiline values
 
 * 0.9.27
 
@@ -3308,149 +3764,149 @@ So, it evolved into <%=tool%>:
 
 * 0.9.24
 
-  * fix bug where AoC node to node transfer did not work
-  * fix bug on error if ED25519 private key is defined in .ssh
+	* fix bug where AoC node to node transfer did not work
+	* fix bug on error if ED25519 private key is defined in .ssh
 
 * 0.9.23
 
-  * defined REST error handlers, more error conditions detected
-  * commands to select specific ascp location
+	* defined REST error handlers, more error conditions detected
+	* commands to select specific ascp location
 
 * 0.9.21
 
-  * supports simplified wizard using global client
-  * only ascp binary is required, other SDK (keys) files are now generated
+	* supports simplified wizard using global client
+	* only ascp binary is required, other SDK (keys) files are now generated
 
 * 0.9.20
 
-  * improved wizard (prepare for AoC global client id)
-  * preview generator: addedoption : --skip-format=&lt;png,mp4&gt;
-  * removed outdated pictures from this doc
+	* improved wizard (prepare for AoC global client id)
+	* preview generator: addedoption : --skip-format=&lt;png,mp4&gt;
+	* removed outdated pictures from this doc
 
 * 0.9.19
 
-  * added command aspera bearer --scope=xx
+	* added command aspera bearer --scope=xx
 
 * 0.9.18
 
-  * enhanced aspera admin events to support query
+	* enhanced aspera admin events to support query
 
 * 0.9.16
 
-  * AoC transfers are now reported in activity app
-  * new interface for Rest class authentication (keep backward compatibility)
+	* AoC transfers are now reported in activity app
+	* new interface for Rest class authentication (keep backward compatibility)
 
 * 0.9.15
 
-  * new feature: "find" command in aspera files
-  * sample code for transfer API
+	* new feature: "find" command in aspera files
+	* sample code for transfer API
 
 * 0.9.12
 
-  * add nagios commands
-  * support of ATS for IBM Cloud, removed old version based on aspera id
+	* add nagios commands
+	* support of ATS for IBM Cloud, removed old version based on aspera id
 
 * 0.9.11
 
-  * Breaking change: @stdin is now @stdin:
-  * support of ATS for IBM Cloud, removed old version based on aspera id
+	* Breaking change: @stdin is now @stdin:
+	* support of ATS for IBM Cloud, removed old version based on aspera id
 
 
 * 0.9.10
 
-  * Breaking change: parameter transfer-node becomes more generic: transfer-info
-  * Display SaaS storage usage with command: aspera admin res node --id=nn info
-  * cleaner way of specifying source file list for transfers
-  * Breaking change: replaced download_mode option with http_download action
+	* Breaking change: parameter transfer-node becomes more generic: transfer-info
+	* Display SaaS storage usage with command: aspera admin res node --id=nn info
+	* cleaner way of specifying source file list for transfers
+	* Breaking change: replaced download_mode option with http_download action
 
 * 0.9.9
 
-  * Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
-  * Now supports "cargo" for Aspera on Cloud (automatic package download)
+	* Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
+	* Now supports "cargo" for Aspera on Cloud (automatic package download)
 
 * 0.9.8
 
-  * Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
-  * AoC: share to share transfer with command "transfer"
+	* Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
+	* AoC: share to share transfer with command "transfer"
 
 * 0.9.7
 
-  * homogeneous [_transfer-spec_](#transferspec) for node and local
-  * preview persistency goes to unique file by default
-  * catch mxf extension in preview as video
-  * Faspex: possibility to download all paclages by specifying id=ALL
-  * Faspex: to come: cargo-like function to download only new packages with id=NEW
+	* homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#direct) transfer agents
+	* preview persistency goes to unique file by default
+	* catch mxf extension in preview as video
+	* Faspex: possibility to download all paclages by specifying id=ALL
+	* Faspex: to come: cargo-like function to download only new packages with id=NEW
 
 * 0.9.6
 
-  * Breaking change: `@param:`is now `@preset:` and is generic
-  * AoC: added command to display current workspace information
+	* Breaking change: `@param:`is now `@preset:` and is generic
+	* AoC: added command to display current workspace information
 
 * 0.9.5
 
-  * new parameter: new_user_option used to choose between public_link and invite of external users.
-  * fixed bug in wizard, and wizard uses now product detection
+	* new parameter: new_user_option used to choose between public_link and invite of external users.
+	* fixed bug in wizard, and wizard uses now product detection
 
 * 0.9.4
 
-  * Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
-  * AoC Package send supports external users
-  * new command to export AoC config to Aspera CLI config
+	* Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
+	* AoC Package send supports external users
+	* new command to export AoC config to Aspera CLI config
 
 * 0.9.3
 
-  * REST error message show host and code
-  * option for quiet display
-  * modified transfer interface and allow token re-generation on error
-  * async add admin command
-  * async add db parameters
-  * Breaking change: new option "sources" to specify files to transfer
+	* REST error message show host and code
+	* option for quiet display
+	* modified transfer interface and allow token re-generation on error
+	* async add admin command
+	* async add db parameters
+	* Breaking change: new option "sources" to specify files to transfer
 
 * 0.9.2
 
-  * Breaking change: changed AoC package creation to match API, see AoC section
+	* Breaking change: changed AoC package creation to match API, see AoC section
 
 * 0.9.1
 
-  * Breaking change: changed faspex package creation to match API, see Faspex section
+	* Breaking change: changed faspex package creation to match API, see Faspex section
 
 * 0.9
 
-  * Renamed the CLI from aslmcli to <%=tool%>
-  * Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
+	* Renamed the CLI from aslmcli to <%=tool%>
+	* Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
 
 * 0.7.6
 
-  * add "sync" plugin
+	* add "sync" plugin
 
 * 0.7
 
-  * Breaking change: AoC package recv take option if for package instead of argument.
-  * Breaking change: Rest class and Oauth class changed init parameters
-  * AoC: receive package from public link
-  * select by col value on output
-  * added rename (AoC, node)
+	* Breaking change: AoC package recv take option if for package instead of argument.
+	* Breaking change: Rest class and Oauth class changed init parameters
+	* AoC: receive package from public link
+	* select by col value on output
+	* added rename (AoC, node)
 
 * 0.6.19
 
 Breaking change:
 
-  * ats server list provisioned &rarr; ats cluster list
-  * ats server list clouds &rarr; ats cluster clouds
-  * ats server list instance --cloud=x --region=y &rarr; ats cluster show --cloud=x --region=y
-  * ats server id xxx &rarr; ats cluster show --id=xxx
-  * ats subscriptions &rarr; ats credential subscriptions
-  * ats api_key repository list &rarr; ats credential cache list
-  * ats api_key list &rarr; ats credential list
-  * ats access_key id xxx &rarr; ats access_key --id=xxx
+	* ats server list provisioned &rarr; ats cluster list
+	* ats server list clouds &rarr; ats cluster clouds
+	* ats server list instance --cloud=x --region=y &rarr; ats cluster show --cloud=x --region=y
+	* ats server id xxx &rarr; ats cluster show --id=xxx
+	* ats subscriptions &rarr; ats credential subscriptions
+	* ats api_key repository list &rarr; ats credential cache list
+	* ats api_key list &rarr; ats credential list
+	* ats access_key id xxx &rarr; ats access_key --id=xxx
 
 * 0.6.18
 
-  * some commands take now --id option instead of id command.
+	* some commands take now --id option instead of id command.
 
 * 0.6.15
 
-  * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
+	* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
 
 # BUGS, FEATURES, CONTRIBUTION
 
@@ -3487,20 +3943,24 @@ You may either install the suggested Gems, or remove your ed25519 key from your
 
 ## Error "Remote host is not who we expected"
 
-`ascp` version 4.x changed the algorithm used to check the SSH server certificate. To ignore the certificate (SSH fingerprint) add option on client side:
+Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
+
+Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
 
 ```
 --ts=@json:'{"sshfp":null}'
 ```
 
-Refer to ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server): if you have access to server side, basically disable other SSH host keys than RSA.
+Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
+
+References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
 
 ## Miscelaneous
 
 * remove rest and oauth classes and use ruby standard gems:
 
-  * oauth
-  * https://github.com/rest-client/rest-client
+	* oauth
+	* https://github.com/rest-client/rest-client
 
 * use Thor or any standard Ruby CLI manager