aspera-cli 4.0.0.pre2 → 4.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 36a82a69216429313771832febaa8a3a84b02aa0c6f10c7ccdb5e637c768046e
-  data.tar.gz: 73c2b08c40ed75fbe99164cbdbaf236309d82bd25bf8dd5ce92081ed09faecda
+  metadata.gz: 1cc7508185a5c551f30a30e54a0cfa2d0ac7931a1bbb565bc7b15e98cca31479
+  data.tar.gz: ca9aae212a8411ddcfa4d1db4aa842fb273bf56fe247490b31037dcb0d628c94
 SHA512:
-  metadata.gz: c04ef4969a3cac1435c85b7533b84756fbdf312ee692d79d249839d1cd0b7cd55a8f6f7436e01f8242414218321e81d350f8f1f446f7bb6c3de52f9771bb7612
-  data.tar.gz: e036d3c9ebc46356b99f7fd4a56e7586d992485628add560360d4fda9bd58bd655ba26a4bc1a45b4157597d62e8d250e7da7d44c808644c3cfbf54d3512234f6
+  metadata.gz: 929bcc3dee9158eb60c810796ed8dc88cfd52d187abb47cb6733adb09971e67849ce1b0bf5cff7daf3dc1f7af327490b574ab8351c94435c6ac83f0674423cd8
+  data.tar.gz: 5c3cba44333dfca675a3cc3b0b77bb6d16654f322b580c936c911551dc85bcc55b16b01649faac2a5ae1f0805b3a2a83315c648ea6c1fb7987c569dc431887b4

data/README.md

@@ -1,30 +1,75 @@
 [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
-# `ascli` : a Command Line for IBM Aspera products
+# `ascli` : Command Line Interface for IBM Aspera products
 
-Version : 4.0.0.pre2
+Version : 4.2.0
 
 _Laurent/2016-2021_
 
-This gem provides a command line interface to Aspera Applications.
+This gem provides `ascli`: a command line interface to Aspera Applications.
 
-Location (once released):
-[https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/aspera-cli)
+`ascli` is a also great tool to learn Aspera APIs.
 
-Disclaimers:
+Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/aspera-cli)
 
-* This has not yet been officially released so things may change
+Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
 
-That being said, `ascli` is very powerful and gets things done, it's also a great tool to learn Aspera APIs.
+Ruby version must be >= > 2.4
 
-This manual addresses three parts:
+# <a name="when_to_use"></a>When to use and when not to use
 
-* `ascli` : The command line tool
-* `asession` : starting a FASP Session with JSON parameters
-* `Aspera` : includes a Ruby "FASPManager"
+`ascli` is designed to be used as a command line tool to:
+
+* execute commands on Aspera products
+* transfer to/from Aspera products
+
+So it is designed for:
+
+* Interactive operations on a text terminal (typically, VT100 compatible)
+* Batch operations in (shell) scripts (e.g. cron job)
+
+`ascli` can be seen as a command line tool integrating:
+
+* a configuration file (config.yaml) and advanced command line options
+* cURL (for REST calls)
+* Aspera transfer (ascp)
+
+One might be tempted to use it as an integration element, e.g. by building a command line programmatically, and then executing it. It is generally not a good idea.
+For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
+
+* Product APIs (REST) : e.g. AoC, Faspex, node
+* Transfer SDK : with gRPC interface and laguage stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
+
+Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
+
+For scripting and ad'hoc command line operations, `ascli` is perfect.
+
+# Notations
 
 In examples, command line operations (starting with `$`) are shown using a standard shell: `bash` or `zsh`.
+Prompt `# ` refers to user `root`, prompt `xfer$ ` refer to user `xfer`.
+
+Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
+
+# <a name="parsing"></a>Shell and Command line parsing
+
+`ascli` is typically executed in a shell, either interactively or in a script. `ascli` receives its arguments from this shell.
+
+On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before `ascli` (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to `ascli`. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
+
+On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
+
+* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
+* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
+
+In case of doubt of argument values after parsing test like this:
 
-Command line parameters in example beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
+```
+$ ascli conf echo "Hello World" arg2 3
+"Hello World"
+ERROR: Argument: unprocessed values: ["arg2", "3"]
+```
+
+`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (ascli), next command line arguments are shown in the error message.
 
 # Quick Start
 
@@ -36,7 +81,7 @@ Once the gem is installed, `ascli` shall be accessible:
 
 ```
 $ ascli --version
-4.0.0.pre2
+4.2.0
 ```
 
 ## First use
@@ -45,9 +90,10 @@ Once installation is completed, you can proceed to the first use with a demo ser
 
 If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
 
-If you want to test with Aspera demo transfer server, a default configuration is created on first use:
+To test with Aspera demo transfer server, setup the environment and then test:
 
 ```
+$ ascli config initdemo
 $ ascli server browse /
 :............:...........:......:........:...........................:.......................:
 :   zmode    :   zuid    : zgid :  size  :           mtime           :         name          :
@@ -67,7 +113,7 @@ If you want to use `ascli` with another server, and in order to make further cal
 * download a file
 
 ```
-$ ascli config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera
+$ ascli config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
 updated: myserver
 $ ascli config id default set server myserver
 updated: default&rarr;server to myserver
@@ -101,78 +147,159 @@ Then, follow the section relative to the product you want to interact with ( Asp
 
 # <a name="installation"></a>Installation
 
-In order to use the tool or the gem, it is necessary to install those components:
+It is possible to install *either* directly on the host operating system (Linux, Windows, Macos) or as a docker container.
 
-* [Ruby](#ruby)
+The direct installation is recommended and consists in installing:
+
+* [Ruby](#ruby) version >= > 2.4
 * [aspera-cli](#the_gem)
-* [FASP](#fasp_prot)
+* [Aspera SDK (ascp)](#fasp_prot)
+
+The following sections provide information on the various installation methods.
+
+An internet connection is required for the installation. If you dont have internet for the installation, refer to section [Installation without internet access](#offline_install).
+
+## Docker container
+
+Use this method only if you know what you do, else use the standard recommended method as described here above.
+
+This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
+
+Ensure that you have Docker installed.
+
+```
+$ docker --version
+```
+
+Download the wrapping script:
+
+```
+$ curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
+$ chmod a+x ascli
+```
 
-The following sections provide information on the installation.
+Install the container image:
+
+```
+$ ./ascli install
+```
+
+Start using it !
+
+Note that the tool is run in the container, so transfers are also executed in the container, not calling host.
+
+The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/ascli` on host.
+
+To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
+To add local storage as a volume edit the script: ascli and add a `--volume` stanza.
 
 ## <a name="ruby"></a>Ruby
 
+Use this method to install on the native host.
+
 A ruby interpreter is required to run the tool or to use the gem and tool.
-The Ruby version shall be > 2.4.
-Any type of Ruby installation can be used.
+
+Ruby minimum version: > 2.4. Ruby version 3 is also supported.
+
+*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
 
 Refer to the following sections for a proposed method for specific operating systems.
 
-### macOS
+The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
+If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
+If you have a simpler better way to install Ruby version >= > 2.4 : use it !
+
+### Generic: RVM: single user installation (not root)
 
+Use this method which provides more flexibility.
 
-MacOS 10.13+ (High Sierra) comes with a recent Ruby, so you can use it directly, you will need to install aspera-cli using `sudo` :
+Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
+
+Install the 2 keys
 
 ```
-$ sudo gem install aspera-cli --pre
+$ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
 ```
 
-Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
+Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
 
 ```
-$ brew install ruby
+$ \curl -sSL https://get.rvm.io | bash -s stable
 ```
 
-### Windows
+If you keep the same terminal (ont needed if re-login):
 
-Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/).
+```
+$ source ~/.rvm/scripts/rvm
+```
 
-Go to "Downloads".
+It is advised to get one of the pre-compiled ruby version, you can list with: 
 
-Select the version "without devkit", x64 corresponding to the one recommended "with devkit".
+```
+$ rvm list --remote
+```
+
+Install the chosen pre-compiled Ruby version:
+
+```
+$ rvm install 2.7.2 --binary
+```
+
+Ruby is now installed for the user, go on to Gem installation.
 
-During installation, skip the installation of "MSys2".
+### Generic: RVM: global installation (as root)
 
-### Linux
+Follow the same method as single user install, but execute as "root".
 
-Install Latest Ruby using "rvm" [https://rvm.io/](https://rvm.io/) .
-It installs by default in /usr/local/rvm , but you can install in another location:
+As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
+One can install in another location with :
 
 ```
-curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
+# curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
 ```
 
-Once installed, you can install latest ruby:
+As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
+If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
+To activate ruby (and ascli) later, source it:
 
 ```
-# rvm install ruby
+# source /etc/profile.d/rvm.sh.ok
+# rvm version
 ```
 
-If you dont want all users to have ruby by default,
-rename the file: `/etc/profile.d/rvm.sh` with another extension, and source it to get rvm.
+### Windows: Installer
+
+Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
+
+* Go to "Downloads".
+* Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
+* At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
 
-Alternatively, only if you know what you do, on RPM based systems (CentOs, Redhat), install the ruby provided by yum which may be 2.0.
+### macOS: pre-installed or `brew`
+
+MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
 
 ```
-# yum install -y ruby rubygems ruby-json
+$ sudo gem install aspera-cli
 ```
 
-If necessary, pre-install jwt for older Ruby < 2.1.
+Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
 
 ```
-# gem install jwt -v 1.5.6
+$ brew install ruby
 ```
 
-You can cleanup your whole yum-installed ruby environment like this to uninstall:
+### Linux: package
+
+If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
+
+Example:
+
+```
+# yum install -y ruby rubygems ruby-json
+```
+
+One can cleanup the whole yum-installed ruby environment like this to uninstall:
 
 ```
 gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
@@ -181,7 +308,7 @@ yum remove -y ruby ruby-libs
 
 ### Other Unixes: Aix, etc...
 
-If your unix do not provide a pre-built ruby, you can get using one of those
+If your unix do not provide a pre-built ruby, you can get it using one of those
 [methods](https://www.ruby-lang.org/en/documentation/installation/)
 
 For instance to build from source, and install in `/opt/ruby` :
@@ -197,12 +324,56 @@ For instance to build from source, and install in `/opt/ruby` :
 # make install
 ```
 
+### <a name="offline_install"></a>Installation without internet access
+
+Note that currently no pre-packaged version exist yet.
+A method to build one provided here:
+
+On a server with the same OS version and with internet access follow the "Generic single user installation" method.
+
+Then create an archive:
+
+```
+$ cd
+$ tar zcvf rvm-ascli.tgz .rvm
+```
+
+Get the Aspera SDK. Execute:
+
+```
+$ ascli conf --show-config|grep sdk_url
+```
+
+Then download the SDK archive from that URL.
+
+Another method for the SDK is to install the SDK (`ascli conf ascp install`) on the first system, and archive `$HOME/.aspera`.
+
+Transfer those 2 archives to the target system without internet access.
+
+On the target system:
+
+* Extract the RVM archive either in a global location, or in a user's home folder : `path_to_rvm_root`
+* in the user's `.profile` add this line: (replace `path_to_rvm_root` with the actual location)
+
+```
+source path_to_rvm_root/scripts/rvm
+rvm use 2.7.2
+```
+
+For the SDK, either install from archive:
+
+```
+$ ascli conf ascp install --sdk-url=file:///SDK.zip
+```
+
+or restore the `$HOME/.aspera` folder for the user.
+
 ## <a name="the_gem"></a>`aspera-cli` gem
 
 Once you have Ruby and rights to install gems: Install the gem and its dependencies:
 
 ```
-# gem install aspera-cli --pre
+# gem install aspera-cli
 ```
 
 To upgrade to the latest version:
@@ -211,21 +382,40 @@ To upgrade to the latest version:
 # gem update aspera-cli
 ```
 
+`ascli` checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
+
+To check manually:
+
+```
+# ascli conf check_update
+```
+
+
+
 ## <a name="fasp_prot"></a>FASP Protocol
 
-Most file transfers will be done using the FASP protocol. Only two additional files are required to perform
-an Aspera Transfer:
+Most file transfers will be done using the FASP protocol, using `ascp`.
+Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
 
 * ascp
 * aspera-license (in same folder, or ../etc)
 
-This can be installed directly with
+This can be installed either be installing an Aspera transfer sofware, or using an embedded command:
 
 ```
 $ ascli conf ascp install
 ```
 
-Those can be found in one of IBM Aspera transfer server or client with its license file (some are free):
+If a local SDK installation is prefered instead of fetching from internet: one can specify the location of the SDK file:
+
+```
+$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
+$ ascli conf ascp install --sdk-url=file:///SDK.zip
+```
+
+The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
+
+If the embedded method is not used, the following packages are also suitable:
 
 * IBM Aspera Connect Client (Free)
 * IBM Aspera Desktop Client (Free)
@@ -234,7 +424,7 @@ Those can be found in one of IBM Aspera transfer server or client with its licen
 * IBM Aspera High Speed Transfer EndPoint (Licensed)
 
 For instance, Aspera Connect Client can be installed
-by visiting the page: [http://downloads.asperasoft.com/connect2/](http://downloads.asperasoft.com/connect2/).
+by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
 
 `ascli` will detect most of Aspera transfer products in standard locations and use the first one found.
 Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
@@ -242,6 +432,33 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
 Several methods are provided on how to start a transfer. Use of a local client is one of them, but
 other methods are available. Refer to section: [Transfer Agents](#agents)
 
+## <a name="offline_install"></a>Offline Installation (without internet)
+
+The procedure consists in:
+
+* Follow the non-root installation procedure with RVM, including gem
+* archive (zip, tar) the main RVM folder (includes ascli):
+
+```
+$ cd ~
+$ tar zcvf rvm_ascli.tgz .rvm
+```
+
+* retrieve the SDK:
+
+```
+$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
+```
+
+* on the system without internet access:
+
+```
+$ cd ~
+$ tar zxvf rvm_ascli.tgz
+$ source ~/.rvm/scripts/rvm
+$ ascli conf ascp install --sdk-url=file:///SDK.zip
+```
+
 # <a name="cli"></a>Command Line Interface: `ascli`
 
 The `aspera-cli` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
@@ -422,6 +639,7 @@ By default, a table output will display one line per entry, and columns for each
 * a,b,c : the list of attributes specified by the comma separated list
 * Array extended value: for instance, @json:'["a","b","c"]' same as above
 * +a,b,c : add selected properties to the default selection.
+* -a,b,c : remove selected properties from the default selection.
 
 ## <a name="extended"></a>Extended Value Syntax
 
@@ -435,25 +653,25 @@ The extended value syntax is:
 
 The difference between reader and decoder is order and ordinality. Both act like a function of value on right hand side. Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
 
-The following "readers" are supported:
+The following "readers" are supported (returns value in []):
 
-* @val:VALUE , prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
-* @file:PATH , read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
-* @path:PATH , performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
-* @env:ENVVAR , read from a named env var, e.g.--password=@env:MYPASSVAR
-* @stdin: , read from stdin (no value on right)
-* @preset:NAME , get whole option preset value by name
+* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
+* @file:PATH : [String] read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
+* @path:PATH : [String] performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
+* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
+* @stdin: : [String] read from stdin (no value on right)
+* @preset:NAME : [Hash] get whole option preset value by name
 
 In addition it is possible to decode a value, using one or multiple decoders :
 
-* @base64: decode a base64 encoded string
-* @json: decode JSON values (convenient to provide complex structures)
-* @zlib: uncompress data
-* @ruby: execute ruby code
-* @csvt: decode a titled CSV value
-* @lines: split a string in multiple lines and return an array
-* @list: split a string in multiple items taking first character as separator and return an array
-* @incps: include values of presets specified by key include_presets in hash
+* @base64: [String] decode a base64 encoded string
+* @json: [any] decode JSON values (convenient to provide complex structures)
+* @zlib: [String] uncompress data
+* @ruby: [any] execute ruby code
+* @csvt: [Array] decode a titled CSV value
+* @lines: [Array] split a string in multiple lines and return an array
+* @list: [Array] split a string in multiple items taking first character as separator and return an array
+* @incps: [Hash] include values of presets specified by key `incps` in input hash
 
 To display the result of an extended value, use the `config echo` command.
 
@@ -485,7 +703,7 @@ $ ascli config echo @csvt:@file:test.csv
 :......:.....................:
 ```
 
-Example: create a hash and include values from preset named "config" of config file
+Example: create a hash and include values from preset named "config" of config file in this hash
 
 ```
 $ ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
@@ -507,14 +725,28 @@ It is also possible to provide a _Structured Value_ in a file using `@json:@file
 
 ## <a name="conffolder"></a>Configuration and Persistency Folder
 
-`ascli` configuration and other runtime files (token cache, file lists, persistency files)
-are stored in folder `$HOME/.aspera/ascli`. The folder can be displayed using :
+`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/ascli`.
+
+Note: `[User's home folder]` is found using ruby's `Dir.home` (`rb_w32_home_dir`).
+It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
+
+The main folder can be displayed using :
 
 ```
 $ ascli config folder
 /Users/kenji/.aspera/ascli
 ```
 
+It can be overriden using the envinonment variable `ASCLI_HOME`.
+
+Example (Windows):
+
+```
+$ set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli
+$ ascli config folder
+C:\Users\Kenji\.aspera\ascli
+```
+
 ## <a name="configfile"></a>Configuration file
 
 On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
@@ -547,7 +779,7 @@ $ ascli config id <option preset> set|delete|show|initialize|update
 The command `update` allows the easy creation of [option preset](#lprt) by simply providing the options in their command line format, e.g. :
 
 ```
-$ ascli config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera --ts=@json:'{"precalculate_job_size":true}'
+$ ascli config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
 ```
 
 * This creates a [option preset](#lprt) `demo_server` with all provided options.
@@ -555,13 +787,13 @@ $ ascli config id demo_server update --url=ssh://demo.asperasoft.com:33001 --use
 The command `set` allows setting individual options in a [option preset](#lprt).
 
 ```
-$ ascli config id demo_server set password demoaspera
+$ ascli config id demo_server set password _demo_pass_
 ```
 
 The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
 
 ```
-$ ascli config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"demoaspera","ts":{"precalculate_job_size":true}}'
+$ ascli config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
 ```
 
 A good practice is to not manually edit the configuration file and use modification commands instead.
@@ -598,11 +830,19 @@ Note that special plugin name: `config` can be associated with a preset that is
 Operations on this preset are done using regular `config` operations:
 
 ```
-$ ascli config id default set _plugin_name_ _defauklt_preset_for_plugin_
+$ ascli config id default set _plugin_name_ _default_preset_for_plugin_
 $ ascli config id default get _plugin_name_
-"_defauklt_preset_for_plugin_"
+"_default_preset_for_plugin_"
 ```
 
+### <a name="lprtdef"></a>Special Plugin: config
+
+Plugin `config` (not to be confused with Option preset config) is used to configure `ascli` but it also contains global options.
+
+When `ascli` starts, it lookjs for the `default` Option preset and if there is a value for `config`, if so, it loads the option values for any plugin used.
+
+If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
+
 ### Format of file
 
 The configuration file is a hash in a YAML file. Example:
@@ -618,7 +858,7 @@ cli_default:
 demo_server:
   url: ssh://demo.asperasoft.com:33001
   username: asperaweb
-  password: demoaspera
+  password: _demo_pass_
 ```
 
 We can see here:
@@ -687,7 +927,6 @@ A [option preset](#lprt) value can be removed with `unset`:
 $ ascli config id cli_default unset interactive
 ```
 
-
 ### Examples
 
 For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
@@ -751,6 +990,8 @@ EOF
 
 The gem is equipped with traces. By default logging level is "warn". To increase debug level, use parameter `log_level`, so either command line `--log-level=xx` or env var `ASCLI_LOG_LEVEL`.
 
+It is also possible to activate traces before initialisation using env var `AS_LOG_LEVEL`.
+
 ## Learning Aspera Product APIs (REST)
 
 This CLI uses REST APIs.
@@ -805,21 +1046,42 @@ The `config` plugin also allows specification for the use of a local FASP client
 
 ```
 $ ascli config ascp show
-/Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
+/Users/laurent/.aspera/ascli/sdk/ascp
+$ ascli config ascp info
++--------------------+-----------------------------------------------------------+
+| key                | value                                                     |
++--------------------+-----------------------------------------------------------+
+| ascp               | /Users/laurent/.aspera/ascli/sdk/ascp                     |
+...
 ```
 
 ### Selection of local `ascp`
 
+By default, `ascli` uses any found local product with ascp, including SDK.
+
 To temporarily use an alternate ascp path use option `ascp_path` (`--ascp-path=`)
 
-To permanently use another ascp:
+For a permanent change, the command `config ascp use` sets the same parameter for the global default.
+
+Using a POSIX shell:
 
 ```
 $ ascli config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
-saved to default global preset /Users/laurent/Applications/Aspera CLI/bin/ascp
+ascp version: 4.0.0.182279
+Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
+Saved to default global preset global_common_defaults
+```
+
+Windows:
+
+```
+$ ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
+ascp version: 4.0.0.182279
+Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
+Saved to default global preset global_common_defaults
 ```
 
-This sets up a global default.
+If the path has spaces, read section: [Shell and Command line parsing](#parsing).
 
 ### List locally installed Aspera Transfer products
 
@@ -906,7 +1168,7 @@ will effectively push files to the related server from the agent node.
 
 ### <a name="direct"></a>Direct (local ascp using FASPManager API)
 
-By default the CLI will use a local FASP protocol, equivalent to specifying `--transfer=direct`.
+By default `ascli` uses a local ascp, equivalent to specifying `--transfer=direct`.
 `ascli` will detect locally installed Aspera products.
 Refer to section [FASP](#client).
 
@@ -916,17 +1178,33 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
 * `EX_http_proxy_url` (proxy for legacy http fallback)
 * `EX_ascp_args`
 
-The `transfer-info` optionally provides the following auto resume parameters:
+The `transfer-info` accepts the following optional parameters:
 
 <table>
-<tr><th>Name</th><th>Default</th><th>Feature</th><th>Description</th></tr>
-<tr><td>iter_max</td>.    <td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
-<tr><td>sleep_initial</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
-<tr><td>sleep_factor</td> <td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
-<tr><td>sleep_max</td>.   <td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
-<tr><td>wss</td>          <td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
+<tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
+<tr><td>spawn_timeout_sec</td><td>Float</td><td>3</td><td>Multi session</td><td>Verification time that ascp is running</td></tr>
+<tr><td>spawn_delay_sec</td><td>Float</td><td>2</td><td>Multi session</td><td>Delay between startup of sessions</td></tr>
+<tr><td>wss</td><td>Bool</td><td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
+<tr><td>resume</td><td>Hash</td><td>nil</td><td>Resumer parameters</td><td>See below</td></tr>
 </table>
 
+Resume parameters:
+
+<table>
+<tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
+<tr><td>iter_max</td><td>int</td><td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
+<tr><td>sleep_initial</td><td>int</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
+<tr><td>sleep_factor</td><td>int</td><td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
+<tr><td>sleep_max</td><td>int</td><td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
+</table>
+
+Examples:
+
+```
+$ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
+$ ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5}'
+```
+
 ### IBM Aspera Connect Client GUI
 
 By specifying option: `--transfer=connect`, `ascli` will start transfers
@@ -958,9 +1236,11 @@ If it possible to send using a HTTP gateway, in case FASP is not allowed.
 Example:
 
 ```
-$ ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://eudemo.asperademo.com:9443/aspera/http-gwy/v1"}'
+$ ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
 ```
 
+Note that the gateway only supports transfers authorized with a token.
+
 ## <a name="transferspec"></a>Transfer Specification
 
 Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
@@ -974,7 +1254,7 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
 
 `ascli` builds a default _transfer-spec_ internally, so it is not necessary to provide additional parameters on the command line for this transfer.
 
-If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters.
+If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
 
 It is possible to specify ascp options when the `transfer` option is set to `direct` using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
 
@@ -1011,7 +1291,7 @@ arg: related ascp argument or env var suffix (PASS for ASPERA_SCP_PASS)
 </p>
 <p>
 UNDER CONSTRUCTION<br/>
-<a href="https://developer.ibm.com/api/view/aspera-prod:ibm-aspera:title-IBM_Aspera#id90944">Documentation&rarr;Node API&rarr;/opt/transfers</a><br/>
+<a href="https://developer.ibm.com/apis/catalog/?search=aspera">Aspera API Documentation</a>&rarr;Node API&rarr;/opt/transfers<br/>
 </p>
 
 <table>
@@ -1093,8 +1373,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
 
 ### List of files for transfers
 
-When uploading, downloading or sending files, the user must specify
-the list of files to transfer. Most of the time, the list of files to transfer will be simply specified on the command line:
+When uploading, downloading or sending files, the user must specify the list of files to transfer.
+Most of the time, the list of files to transfer will be simply specified on the command line:
 
 ```
 $ ascli server upload ~/mysample.file secondfile
@@ -1108,7 +1388,8 @@ $ ascli server upload --sources=@args ~/mysample.file secondfile
 
 More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
 
-Note that this is different from the "ascp" command line. The paradigm used by `ascli` is: all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
+Note that this is different from the "ascp" command line. The paradigm used by `ascli` is:
+all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
 
 For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
 
@@ -1225,6 +1506,8 @@ A non complete list of commands used in unit tests:
 ```
 ascli
 ascli -h
+ascli aoc -N remind --username=my_aoc_user_email
+ascli aoc -N servers
 ascli aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}'
 ascli aoc admin ats access_key --id=akibmcloud --secret=somesecret node browse /
 ascli aoc admin ats access_key --id=akibmcloud delete
@@ -1235,6 +1518,26 @@ ascli aoc admin ats cluster clouds
 ascli aoc admin ats cluster list
 ascli aoc admin ats cluster show --cloud=aws --region=eu-west-1
 ascli aoc admin ats cluster show --id=1f412ae7-869a-445c-9c05-02ad16813be2
+ascli aoc admin res apps_new list
+ascli aoc admin res client list
+ascli aoc admin res client_access_key list
+ascli aoc admin res client_registration_token --id=my_clt_reg_id delete
+ascli aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
+ascli aoc admin res client_registration_token list
+ascli aoc admin res contact list
+ascli aoc admin res dropbox list
+ascli aoc admin res dropbox_membership list
+ascli aoc admin res group list
+ascli aoc admin res kms_profile list
+ascli aoc admin res node list
+ascli aoc admin res operation list
+ascli aoc admin res organization show
+ascli aoc admin res package list
+ascli aoc admin res saml_configuration list
+ascli aoc admin res self show
+ascli aoc admin res short_link list
+ascli aoc admin res user list
+ascli aoc admin res workspace_membership list
 ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
 ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 access_key delete --id=testsub1
 ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 events
@@ -1243,7 +1546,6 @@ ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v4
 ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v4 mkdir /folder1
 ascli aoc admin resource workspace list
 ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
-ascli aoc apiinfo
 ascli aoc automation workflow --id="my_wf_id" action create --value=@json:'{"name":"toto"}' | tee action.info
 ascli aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
 ascli aoc automation workflow delete --id="my_wf_id"
@@ -1257,7 +1559,7 @@ ascli aoc files browse /
 ascli aoc files browse / -N --link=my_aoc_publink_folder
 ascli aoc files delete /testsrc
 ascli aoc files download --transfer=connect /200KB.1
-ascli aoc files file 18891
+ascli aoc files file --id=my_file_id show
 ascli aoc files find / --value='\.partial$'
 ascli aoc files http_node_download --to-folder=. /200KB.1
 ascli aoc files mkdir /testsrc
@@ -1278,7 +1580,7 @@ ascli aoc packages recv --id=ALL --to-folder=. --once-only=yes --lock-port=12345
 ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["external.user@example.com"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
 ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["internal.user@example.com"],"note":"my note"}' testfile.bin
 ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
-ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user
+ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
 ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
 ascli aoc user info modify @json:'{"name":"dummy change"}'
 ascli aoc user info show
@@ -1298,8 +1600,8 @@ ascli ats cluster list
 ascli ats cluster show --cloud=aws --region=eu-west-1
 ascli ats cluster show --id=1f412ae7-869a-445c-9c05-02ad16813be2
 ascli conf flush_tokens
-ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user --test-mode=yes
-ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user --test-mode=yes --use-generic-client=yes
+ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes
+ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
 ascli config ascp connect id 'Aspera Connect for Windows' info
 ascli config ascp connect id 'Aspera Connect for Windows' links id 'Windows Installer' download --to-folder=.
 ascli config ascp connect id 'Aspera Connect for Windows' links list
@@ -1308,6 +1610,9 @@ ascli config ascp info
 ascli config ascp install
 ascli config ascp products list
 ascli config ascp show
+ascli config check_update
+ascli config doc
+ascli config doc transfer-parameters
 ascli config email_test aspera.user1@gmail.com
 ascli config export
 ascli config genkey mykey
@@ -1315,28 +1620,29 @@ ascli config plugins
 ascli config proxy_check --fpac=file:///examples/proxy.pac https://eudemo.asperademo.com
 ascli console transfer current list 
 ascli console transfer smart list 
-ascli console transfer smart sub 112 @json:'{"source":{"paths":["10MB.1"]},"source_type":"user_selected"}'
+ascli console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
 ascli cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
 ascli cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
 ascli cos node access_key --id=self show
 ascli cos node download testfile.bin --to-folder=.
 ascli cos node info
 ascli cos node upload testfile.bin
-ascli faspex nagios_check
+ascli faspex health
 ascli faspex package list
 ascli faspex package list --box=sent --fields=package_id --format=csv --display=data|tail -n 1);\
-ascli faspex package recv --box=sent --to-folder=. --id="my_package_id"
+ascli faspex package list --fields=package_id --format=csv --display=data|tail -n 1);\
+ascli faspex package recv --to-folder=. --box=sent --id="my_package_id"
 ascli faspex package recv --to-folder=. --id="my_package_id"
 ascli faspex package recv --to-folder=. --id=ALL --once-only=yes
 ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
-ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["internal.user@example.com"]}' testfile.bin
+ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["internal.user@example.com","FASPEX_USERNAME"]}' testfile.bin
 ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
 ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
 ascli faspex source name "Server Files" node br /
 ascli faspex5 node list --value=@json:'{"type":"received","subtype":"mypackages"}'
-ascli faspex5 package list --value=@json:'{"state":["released"]}'
+ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
 ascli faspex5 package receive --id="my_package_id" --to-folder=.
-ascli faspex5 package send --value=@json:'{"title":"test title","recipients":["admin"]}' testfile.bin
+ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"${f5_user}"}]}' testfile.bin
 ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
 ascli node -N -Ptst_node_preview access_key delete --id=aoc_1
 ascli node async --id=1 bandwidth 
@@ -1350,8 +1656,8 @@ ascli node browse / -r
 ascli node delete folder_1/10MB.1
 ascli node delete folder_1/testfile.bin
 ascli node download --to-folder=. folder_1/testfile.bin
+ascli node health
 ascli node info
-ascli node nagios_check
 ascli node search / --value=@json:'{"sort":"mtime"}'
 ascli node service --id=service1 delete
 ascli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
@@ -1381,7 +1687,8 @@ ascli preview test --case=test png "TSTFILE_MXF" --video-png-conv=fixed --log-le
 ascli preview test --case=test png "TSTFILE_PDF" --log-level=debug
 ascli preview trevents --once-only=yes --skip-types=office --log-level=info
 ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user ctl all:status
-ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user nagios app_services --format=nagios
+ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user health app_services --format=nagios
+ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user health asctlstatus --format=nagios --cmd-prefix='sudo '
 ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user nodeadmin -- -l
 ascli server -N -Ptst_server_bykey -Plocal_user br /
 ascli server browse /
@@ -1391,24 +1698,27 @@ ascli server delete NEW_SERVER_FOLDER
 ascli server delete folder_1/target_hot
 ascli server delete folder_1/to.delete
 ascli server df
-ascli server download NEW_SERVER_FOLDER/testfile.bin --to-folder=.
+ascli server download NEW_SERVER_FOLDER/testfile.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
 ascli server download NEW_SERVER_FOLDER/testfile.bin --to-folder=folder_1 --transfer=node
 ascli server du /
+ascli server health transfer --to-folder=folder_1 --format=nagios 
 ascli server info
 ascli server md5sum NEW_SERVER_FOLDER/testfile.bin
 ascli server mkdir NEW_SERVER_FOLDER --logger=stdout
 ascli server mkdir folder_1/target_hot
 ascli server mv folder_1/200KB.2 folder_1/to.delete
-ascli server nagios transfer --to-folder=folder_1 --format=nagios 
 ascli server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
 ascli server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
 ascli server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername
 ascli server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
-ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER
+ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
+ascli shares admin share list
 ascli shares repository browse /
 ascli shares repository delete /SHARES_UPLOAD/testfile.bin
 ascli shares repository download --to-folder=. /SHARES_UPLOAD/testfile.bin
+ascli shares repository download --to-folder=. /SHARES_UPLOAD/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://HTTP_GW_FQDN/aspera/http-gwy/v1"}'
 ascli shares repository upload --to-folder=/SHARES_UPLOAD testfile.bin
+ascli shares repository upload --to-folder=/SHARES_UPLOAD testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://HTTP_GW_FQDN/aspera/http-gwy/v1"}'
 ascli shares2 appinfo
 ascli shares2 organization list
 ascli shares2 project list --organization=Sport
@@ -1424,7 +1734,7 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
 ```
 $ ascli -h
 NAME
-	ascli -- a command line tool for Aspera Applications (v4.0.0.pre2)
+	ascli -- a command line tool for Aspera Applications (v4.2.0)
 
 SYNOPSIS
 	ascli COMMANDS [OPTIONS] [ARGS]
@@ -1435,6 +1745,10 @@ DESCRIPTION
 	execute: ascli conf doc
 	or visit: http://www.rubydoc.info/gems/aspera-cli
 
+ENVIRONMENT VARIABLES
+	ASCLI_HOME  config folder, default: $HOME/.aspera/ascli
+	#any option can be set as an environment variable, refer to the manual
+
 COMMANDS
 	To list first level commands, execute: ascli
 	Note that commands can be written shortened (provided it is unique).
@@ -1448,38 +1762,38 @@ ARGS
 	Some commands require mandatory arguments, e.g. a path.
 
 OPTIONS: global
-        --interactive=ENUM           use interactive input of missing params: yes, no
-        --ask-options=ENUM           ask even optional options: yes, no
+        --interactive=ENUM           use interactive input of missing params: yes, no
+        --ask-options=ENUM           ask even optional options: yes, no
         --format=ENUM                output format: table, ruby, json, jsonpp, yaml, csv, nagios
         --display=ENUM               output only some information: info, data, error
         --fields=VALUE               comma separated list of fields, or ALL, or DEF
         --select=VALUE               select only some items in lists, extended value: hash (column, value)
         --table-style=VALUE          table display style
-        --flat-hash=ENUM             display hash values as additional keys: yes, no
+        --flat-hash=ENUM             display hash values as additional keys: yes, no
     -h, --help                       Show this message.
         --bash-comp                  generate bash completion for command
         --show-config                Display parameters used for the provided action.
     -r, --rest-debug                 more debug for HTTP calls
     -v, --version                    display version
     -w, --warnings                   check for language warnings
-        --ui=ENUM                    method to start browser: text, graphical
-        --log-level=ENUM             Log level: info, warn, error, fatal, unknown, debug
-        --logger=ENUM                log method: stderr, stdout, syslog
+        --ui=ENUM                    method to start browser: text, graphical
+        --log-level=ENUM             Log level: debug, info, warn, error, fatal, unknown
+        --logger=ENUM                log method: stderr, stdout, syslog
         --lock-port=VALUE            prevent dual execution of a command, e.g. in cron
         --query=VALUE                additional filter for API calls (extended value) (some commands)
-        --insecure=ENUM              do not validate HTTPS certificate: yes, no
-        --once-only=ENUM             process only new items (some commands): yes, no
+        --insecure=ENUM              do not validate HTTPS certificate: yes, no
+        --once-only=ENUM             process only new items (some commands): yes, no
 
 COMMAND: config
-SUBCOMMANDS: gem_path genkey plugins flush_tokens list overview open echo id documentation wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file
+SUBCOMMANDS: gem_path genkey plugins flush_tokens list overview open echo id documentation wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo
 OPTIONS:
         --value=VALUE                extended value for create, update, list filter
         --property=VALUE             name of property to set
         --id=VALUE                   resource identifier (modify,delete,show)
-        --config-file=VALUE          read parameters from file in YAML format, current=/Users/dwosk/.aspera/ascli/config.yaml
-        --override=ENUM              override existing value: yes, no
+        --config-file=VALUE          read parameters from file in YAML format, current=/Users/FooBar/.aspera/ascli/config.yaml
+        --override=ENUM              override existing value: yes, no
     -N, --no-default                 do not load default configuration for plugin
-        --use-generic-client=ENUM    wizard: AoC: use global or org specific jwt client id: yes, no
+        --use-generic-client=ENUM    wizard: AoC: use global or org specific jwt client id: yes, no
         --pkeypath=VALUE             path to private key for JWT (wizard)
         --ascp-path=VALUE            path to ascp
         --use-product=VALUE          use ascp from specified product
@@ -1487,10 +1801,13 @@ OPTIONS:
         --fpac=VALUE                 proxy auto configuration URL
     -P, --presetVALUE                load the named option preset from current config file
         --default=VALUE              set as default configuration for specified plugin
-        --secret=VALUE               access key secret for node
-        --secrets=VALUE              access key secret for node
-        --test-mode=ENUM             skip user validation in wizard mode: yes, no
-        --ts=VALUE                   override transfer spec values (Hash, use @json: prefix), current={}
+        --secret=VALUE               default secret
+        --secrets=VALUE              secret repository (Hash)
+        --sdk-url=VALUE              URL to get SDK
+        --sdk-folder=VALUE           SDK folder location
+        --test-mode=ENUM             skip user validation in wizard mode: yes, no
+        --version-check-days=VALUE   period to check neew version in days (zero to disable)
+        --ts=VALUE                   override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
         --local-resume=VALUE         set resume policy (Hash, use @json: prefix), current=
         --to-folder=VALUE            destination folder for downloaded files
         --sources=VALUE              list of source files (see doc)
@@ -1509,7 +1826,7 @@ OPTIONS:
 
 
 COMMAND: node
-SUBCOMMANDS: postprocess stream transfer cleanup forward access_key watch_folder service async central asperabrowser basic_token browse upload download api_details nagios_check events space info license mkdir mklink mkfile rename delete search
+SUBCOMMANDS: postprocess stream transfer cleanup forward access_key watch_folder service async central asperabrowser basic_token browse upload download api_details health events space info license mkdir mklink mkfile rename delete search
 OPTIONS:
         --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
         --username=VALUE             username to log in
@@ -1528,7 +1845,7 @@ OPTIONS:
         --password=VALUE             user's password
         --params=VALUE               parameters hash table, use @json:{"param":"value"}
         --result=VALUE               specify result value as: 'work step:parameter'
-        --synchronous=ENUM           work step:parameter expected as result: yes, no
+        --synchronous=ENUM           work step:parameter expected as result: yes, no
         --ret-style=ENUM             how return type is requested in api: header, arg, ext
         --auth-style=ENUM            authentication type: arg_pass, head_basic, apikey
 
@@ -1562,11 +1879,16 @@ OPTIONS:
 
 
 COMMAND: faspex5
-SUBCOMMANDS: node package
+SUBCOMMANDS: node package auth_client
 OPTIONS:
         --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
         --username=VALUE             username to log in
         --password=VALUE             user's password
+        --client-id=VALUE            API client identifier in application
+        --client-secret=VALUE        API client secret in application
+        --redirect-uri=VALUE         API client redirect URI
+        --auth=ENUM                  type of Oauth authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey, boot
+        --private-key=VALUE          RSA private key PEM value for JWT (prefix file path with @val:@file:)
 
 
 COMMAND: cos
@@ -1581,7 +1903,7 @@ OPTIONS:
 
 
 COMMAND: faspex
-SUBCOMMANDS: nagios_check package source me dropbox v4 address_book login_methods
+SUBCOMMANDS: health package source me dropbox v4 address_book login_methods
 OPTIONS:
         --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
         --username=VALUE             username to log in
@@ -1611,7 +1933,7 @@ OPTIONS:
         --username=VALUE             username to log in
         --password=VALUE             user's password
         --skip-format=ENUM           skip this preview format (multiple possible): png, mp4
-        --folder-reset-cache=ENUM    force detection of generated preview by refresh cache: no, header, read
+        --folder-reset-cache=ENUM    force detection of generated preview by refresh cache: no, header, read
         --skip-types=VALUE           skip types in comma separated list
         --previews-folder=VALUE      preview folder in storage root
         --temp-folder=VALUE          path to temp folder
@@ -1619,14 +1941,15 @@ OPTIONS:
         --case=VALUE                 basename of output for for test
         --scan-path=VALUE            subpath in folder id to start scan in (default=/)
         --scan-id=VALUE              forder id in storage to start scan in, default is access key main folder id
-        --overwrite=ENUM             when to overwrite result file: always, never, mtime
-        --file-access=ENUM           how to read and write files in repository: local, remote
+        --mimemagic=ENUM             use Mime type detection of gem mimemagic: yes, no
+        --overwrite=ENUM             when to overwrite result file: always, never, mtime
+        --file-access=ENUM           how to read and write files in repository: local, remote
         --max-size=VALUE             maximum size (in bytes) of preview file
         --thumb-vid-scale=VALUE      png: video: size (ffmpeg scale argument)
         --thumb-vid-fraction=VALUE   png: video: position of snapshot
         --thumb-img-size=VALUE       png: non-video: height (and width)
-        --video-conversion=ENUM      mp4: method for preview generation: reencode, blend, clips
-        --video-png-conv=ENUM        mp4: method for thumbnail generation: fixed, animated
+        --video-conversion=ENUM      mp4: method for preview generation: reencode, blend, clips
+        --video-png-conv=ENUM        mp4: method for thumbnail generation: fixed, animated
         --video-start-sec=VALUE      mp4: start offset (seconds) of video preview
         --video-scale=VALUE          mp4: video scale (ffmpeg)
         --blend-keyframes=VALUE      mp4: blend: # key frames
@@ -1642,10 +1965,51 @@ SUBCOMMANDS: start admin
 OPTIONS:
         --parameters=VALUE           extended value for session set definition
         --session-name=VALUE         name of session to use for admin commands, by default first one
-ERROR: Other: uninitialized constant Aspera::Cli::Plugins::Aoc
-Did you mean?  Aspera::AoC
-ERROR: Argument: unprocessed options: ["-Cnone"]
-Use '--log-level=debug' to get more details.
+
+
+COMMAND: aoc
+SUBCOMMANDS: reminder bearer_token organization tier_restrictions user workspace packages files gateway admin automation servers
+OPTIONS:
+        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
+        --username=VALUE             username to log in
+        --password=VALUE             user's password
+        --auth=ENUM                  OAuth type of authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey
+        --operation=ENUM             client operation for transfers: push, pull
+        --client-id=VALUE            OAuth API client identifier in application
+        --client-secret=VALUE        OAuth API client passcode
+        --redirect-uri=VALUE         OAuth API client redirect URI
+        --private-key=VALUE          OAuth JWT RSA private key PEM value (prefix file path with @val:@file:)
+        --workspace=VALUE            name of workspace
+        --name=VALUE                 resource name
+        --path=VALUE                 file or folder path
+        --link=VALUE                 public link to shared resource
+        --new-user-option=VALUE      new user creation option
+        --from-folder=VALUE          share to share source folder
+        --scope=VALUE                OAuth scope for AoC API calls
+        --notify=VALUE               notify users that file was received
+        --bulk=ENUM                  bulk operation: yes, no
+        --default-ports=ENUM         use standard FASP ports or get from node api: yes, no
+
+
+COMMAND: server
+SUBCOMMANDS: health nodeadmin userdata configurator ctl download upload browse delete rename ls rm mv du info mkdir cp df md5sum
+OPTIONS:
+        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
+        --username=VALUE             username to log in
+        --password=VALUE             user's password
+        --ssh-keys=VALUE             ssh key path list (Array or single)
+        --ssh-options=VALUE          ssh options (Hash)
+        --cmd-prefix=VALUE           prefix to add for as cmd execution, e.g. sudo or /opt/aspera/bin 
+
+
+COMMAND: console
+SUBCOMMANDS: transfer health
+OPTIONS:
+        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
+        --username=VALUE             username to log in
+        --password=VALUE             user's password
+        --filter-from=DATE           only after date
+        --filter-to=DATE             only before date
 
 
 ```
@@ -1766,7 +2130,7 @@ updated: my_aoc_org
 Define this [option preset](#lprt) as default configuration for the `aspera` plugin:
 
 ```
-$ ascli config id default set aspera my_aoc_org
+$ ascli config id default set aoc my_aoc_org
 ```
 
 Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
@@ -2140,7 +2504,7 @@ Then, create two shared folders located in two regions, in your files home, in a
 Then, transfer between those:
 
 ```
-$ ascli -Paoc_show aspera files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
+$ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
 ```
 
 * create registration key to register a node
@@ -2188,17 +2552,27 @@ $ ascli aoc admin res node --id=8669 v4 perm 82 show
 Send a package:
 
 ```
-$ ascli aoc packages send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
+$ ascli aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
 ```
 
 Notes:
 
-* the `value` parameter can contain any supported package creation parameter. Refer to the API, or display an existing package.
-* to list recipients use fields: "recipients" and/or "bcc_recipients". ascli will resolve the list of email addresses to expected user ids. If a recipient is not already registered and the workspace allows external users, then the package is sent to an external user, and
+* the `value` parameter can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package to find attributes.
+* to provide the list of recipients, use fields: "recipients" and/or "bcc_recipients". ascli will resolve the list of email addresses to expected user ids.
+* a recipîent can be a shared inbox, in this case just use the name of the shared inbox as recipient.
+* If a recipient is not already registered and the workspace allows external users, then the package is sent to an external user, and
   * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account.
   * if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
 
-## <a name="aoccargo"></a>Receive only new packages
+Examples:
+
+```
+$ ascli aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
+$ ascli aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
+$ ascli aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
+```
+
+## <a name="aoccargo"></a>Receive new packages only
 
 It is possible to automatically download new packages, like using Aspera Cargo:
 
@@ -2210,10 +2584,10 @@ $ ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
 * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
 * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
 
-Typically, one would regularly execute this command on a regular basis, using the method oif your choice:
+Typically, one would regularly execute this command on a regular basis, using the method of your choice:
 
-* Windows scheduler
-* cron
+* Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
+* Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
 * etc...
 
 ## Download Files
@@ -2433,7 +2807,7 @@ This can also be set as default using a preset
 One can test the "server" application using the well known demo server:
 
 ```
-$ ascli config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera
+$ ascli config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
 $ ascli config id default set server aspera_demo_server
 $ ascli server browse /aspera-test-dir-large
 $ ascli server download /aspera-test-dir-large/200MB
@@ -2531,23 +2905,67 @@ $ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"my
 
 # Plugin: IBM Aspera Faspex
 
-Note that the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+Notes:
 
-## Sending a Package
+* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+
+## Faspex 5 Beta1
+
+As the web UI does not yet allow adding API client yet, the way to use CLI is:
+
+* open a browser
+* start developer mode
+* login to faspex 5
+* find the first API call with `Authorization` token, and copy it (kind of base64 long string)
+
+Use it as password and use `--auth=boot`.
+
+```
+$ ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
+```
 
-Provide delivery info in JSON, example:
+An JWT client can then be created with a private key:
 
 ```
---delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}'
+$ jsonk=$(openssl rsa -in  ~/.aspera/ascli/aspera_on_cloud_key -pubout 2> /dev/null | sed -e :a -e N -e '$!ba' -e 's/\n/\\n/g')
+$ ascli faspex5 -Pf5boot auth_client create --value=@json:'{"name":"hello","client_type":"public","redirect_uris":["https://localhost:12345"],"allow_jwt_grant":true,"public_key":"'$jsonk'"}'
 ```
 
-a note can be added: `"note":"Please ..."`
+or deleted by name:
+
+```
+$ id=$(ascli faspex5 auth_client list --select=@json:'{"name":"hello"}' --fields=client_id --format=csv)
+$ ascli faspex5 auth_client delete --id=$id
+```
+
+Once the API client is created with a client_id and secret (result of create command), create a configuration:
+
+```
+$ ascli conf id f5 update --url=https://localhost/aspera/faspex --auth=jwt --client-id=abcd --client-secret=def --username=pierre@example.com --private-key=@val:@file:~/.aspera/ascli/aspera_on_cloud_key
+$ ascli conf id default set faspex5 f5
+``` 
+
+Ready to use Faspex5 with CLI.
+
+Once the graphical registration form exist, ther bootstrap method can be removed.
+
+## Sending a Package
+
+The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
+
+Example:
+
+```
+$ ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
+```
 
-metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
+If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
 
+Additional optional parameters in `delivery_info`:
 
-Note for full details, refer to:
-[Reference on Developer Site](https://developer.asperasoft.com/web/faspex/sending)
+* Package Note: : `"note":"note this and that"`
+* Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
 
 ## operation on dropboxes
 
@@ -2610,6 +3028,8 @@ $ for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}'
 # Plugin: IBM Cloud Object Storage
 
 The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
+It uses the same transfer service as Aspera on Cloud.
+see [https://status.aspera.io](https://status.aspera.io)
 
 Required options are either:
 
@@ -2666,7 +3086,14 @@ Endpoints for regions can be found by querying the `endpoints` URL.
 For convenience, let us create a default configuration, for example:
 
 ```
-$ ascli conf id mycos update --service-credentials=@val:@json:@file:$HOME/service_creds.json  --region=us-south --bucket=laurent
+$ ascli conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
+$ ascli conf id default set cos mycos
+```
+
+or using direct parameters:
+
+```
+$ ascli conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
 $ ascli conf id default set cos mycos
 ```
 
@@ -2684,7 +3111,7 @@ to start from ma configuration file, using `ascli` standard options.
 
 # Plugin: Preview
 
-The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on an Aspera HSTS for use primarily in the Aspera on Cloud application.
+The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
 This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
 Several parameters can be used to tune several aspects:
 
@@ -2732,6 +3159,7 @@ The tool requires the following external tools available in the `PATH`:
 * OptiPNG : `optipng`
 * FFmpeg : `ffmpeg` `ffprobe`
 * Libreoffice : `libreoffice`
+* ruby gem `mimemagic`
 
 Here shown on Redhat/CentOS.
 
@@ -2743,6 +3171,26 @@ To check if all tools are found properly, execute:
 $ ascli preview check
 ```
 
+### mimemagic
+
+To benefit from extra mime type detection install gem mimemagic:
+
+```
+# gem install mimemagic
+```
+
+or to install an earlier version if any problem:
+
+```
+# gem install mimemagic -v '~> 0.3.0'
+```
+
+To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
+
+If not used, Mime type used for conversion is the one provided by the node API.
+
+If used, it the `preview` command will first analyse the file content using mimemagic, and if no match, will try by extension.
+
 ### Image: Imagemagick and optipng
 
 ```
@@ -2767,90 +3215,111 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
 # dnf install unoconv
 ```
 
+* Amazon Linux
+
+```
+# amazon-linux-extras enable libreoffice
+# yum clean metadata
+# yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
+# wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
+# mv unoconv /usr/bin
+# chmod a+x /usr/bin/unoconv
+```
 
 ## Configuration
 
-Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt). Note that if you use the `ascli` run as `xfer` user, like here, the configuration file must be created as the same user. Example using a [option preset](#lprt) named `my_preset_name` (choose any name relevant to you, e.g. the AoC node name, and replace in the following lines):
+The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
+
+Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt).  The configuration file must be created with the same user used to run so that it is properly used on runtime.
+
+Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
 
 ```
 # su -s /bin/bash - xfer
-$ ascli config id my_preset_name update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
-$ ascli config id default set preview my_preset_name
+$ ascli config id previewconf update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
+$ ascli config id default set preview previewconf
 ```
 
-Here we assume that Office file generation is disabled, else remove the option. For the `lock_port` option refer to a previous section in thsi manual.
+Here we assume that Office file generation is disabled, else remove this option.
+`lock_port` prevents concurrent execution of generation when using a scheduler.
 
-Once can check if the access key is well configured using:
+One can check if the access key is well configured using:
 
 ```
-$ ascli -Pmy_preset_name node browse /
+$ ascli -Ppreviewconf node browse /
 ```
 
 This shall list the contents of the storage root of the access key.
 
 ## Execution
 
-The tool intentionally supports only a "one shot" mode in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
-It needs to be run regularly to create or update preview files. For that use your best
+The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
+It needs to be run on a regular basis to create or update preview files. For that use your best
 reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
 
-Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consiustent have generate the appropriate access rights, the generation process
-should be run as user `xfer`.
+Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
 
 Lets do a one shot test, using the configuration previously created:
 
 ```
 # su -s /bin/bash - xfer
-$ ascli preview scan --overwrite=always
+xfer$ ascli preview scan --overwrite=always
 ```
 
 When the preview generator is first executed it will create a file: `.aspera_access_key`
-which contains the access key used.
+in the previews folder which contains the access key used.
 On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
 
 ## Configuration for Execution in scheduler
 
-Here is an example of configuration for use with cron on Linux. Adapt the scripts to your own preference.
+Here is an example of configuration for use with cron on Linux.
+Adapt the scripts to your own preference.
 
 We assume here that a configuration preset was created as shown previously.
 
-Here the cronjob is created for `root`, and changes the user to `xfer`, also overriding the shell which should be `aspshell`. (adapt the command below, as it would override existing crontab). It is also up to you to use directly the `xfer` user's crontab. This is an example only.
+Lets first setup a script that will be used in the sceduler and sets up the environment.
+
+Example of startup script `cron_ascli`, which sets the Ruby environment and adds some timeout protection:
+
+```
+#!/bin/bash
+# set a timeout protection, just in case
+case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
+. /etc/profile.d/rvm.sh
+rvm use 2.6 --quiet
+exec timeout ${tmout} ascli "${@}"
+```
+
+Here the cronjob is created for user `xfer`.
 
 ```
-# crontab<<EOF
-2-59 * * * * su -s /bin/bash - xfer -c 'nice +10 timeout 10m ascli preview event --log-level=info --logger=syslog --iteration-file=/tmp/preview_restart.txt'
-0 * * * *    su -s /bin/bash - xfer -c 'nice +10 timeout 30m ascli preview scan  --log-level=info --logger=syslog'
+xfer$ crontab<<EOF
+0    * * * *  /home/xfer/cron_ascli preview scan --logger=syslog --display=error
+2-59 * * * *  /home/xfer/cron_ascli preview trev --logger=syslog --display=error
 EOF
 ```
 
-Nopte that the options here may be located in the config preset, but it was left on the command line to keep stdout for command line execution of preview.
+Note that the loging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
 
 ## Candidate detection for creation or update (or deletion)
 
-The tool will find candidates for preview generation using three commands:
+The tool generates preview files using those commands:
 
 * `trevents` : only recently uploaded files will be tested (transfer events)
 * `events` : only recently uploaded files will be tested (file events: not working)
-* `scan` : deeply scan all files under the access key&apos;s "storage root"
-* `folder` : same as `scan`, but only on the specified folder&apos;s "file identifier"
-* `file` : for an individual file generation
-
-Note that for the `event`, the option `iteration_file` should be specified so that
-successive calls only process new events. This file will hold an identifier
-telling from where to get new events.
-
-It is also possible to test a local file, using the `test` command.
+* `scan` : recursively scan all files under the access key&apos;s "storage root"
+* `test` : test using a local file
 
 Once candidate are selected, once candidates are selected,
 a preview is always generated if it does not exist already,
 else if a preview already exist, it will be generated
-using one of three overwrite method:
+using one of three values for the `overwrite` option:
 
 * `always` : preview is always generated, even if it already exists and is newer than original
 * `never` : preview is generated only if it does not exist already
 * `mtime` : preview is generated only if the original file is newer than the existing
 
-Deletion of preview for deleted source files: not implemented yet.
+Deletion of preview for deleted source files: not implemented yet (TODO).
 
 If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
 
@@ -2860,6 +3329,14 @@ $ ascli preview scan --skip-folders=@json:'["/not_here"]'
 
 The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
 
+When scanning the option `value` has the same behaviour as for the `node find` command.
+
+For instance to filter out files beginning with `._` do:
+
+```
+... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
+```
+
 ## Preview File types
 
 Two types of preview can be generated:
@@ -2997,7 +3474,7 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
 ## Simple session
 
 ```
-MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
+MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
 
 echo "${MY_TSPEC}"|asession
 ```
@@ -3010,7 +3487,7 @@ This is particularly useful for a persistent session ( with the [_transfer-spec_
 
 ```
 $ asession
-{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"demoaspera","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
+{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
 {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
 {"type":"DONE"}
 ```
@@ -3104,6 +3581,34 @@ $ ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=1234
 
 The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
 
+# Aspera Health check and Nagios
+
+Each plugin provide a `health` command that will check the health status of the application. Example:
+
+```
+$ ascli console health
++--------+-------------+------------+
+| status | component   | message    |
++--------+-------------+------------+
+| ok     | console api | accessible |
++--------+-------------+------------+
+```
+
+Typically, the health check uses the REST API of the application with the following exception: the `server` plugin allows checking health by:
+
+* issuing a transfer to the server
+* checking web app status with `asctl all:status`
+* checking daemons process status
+
+`ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
+
+```
+$ ascli server health transfer --to-folder=/Upload --format=nagios --progress=none
+OK - [transfer:ok]
+$ ascli server health asctlstatus --cmd_prefix='sudo ' --format=nagios
+OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
+```
+
 # Module: `Aspera`
 
 Main components:
@@ -3123,6 +3628,16 @@ This sample code shows some example of use of the API as well as
 REST API.
 Note: although nice, it's probably a good idea to use RestClient for REST.
 
+Example of use of the API of Aspera on Cloud:
+
+```
+require 'aspera/aoc'
+
+aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/ascli/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
+
+aoc.read('self')
+```
+
 # History
 
 When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
@@ -3139,11 +3654,38 @@ So, it evolved into `ascli`:
 * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
 * `ruby` is consistent with other Aspera products
 
-
-
-# Release Notes
-
-* 4.0.0.pre2
+# Changes (Release notes)
+
+* 4.2.0
+
+	* new: command `aoc remind` to receive organization membership by email
+	* new: in `preview` option `value` to filter out on file name
+	* new: `initdemo` to initialize for demo server
+	* new: `direct` transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
+	* fix: on Windows `conf ascp use` expects ascp.exe
+	* fix: (break) multi_session_threshold is Integer, not String
+	* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
+	* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
+	* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
+	* change: (break) parameters for resume in `transfer-info` for `direct` are now in sub-key `"resume"`
+
+* 4.1.0
+
+  	* fix: remove keys from transfer spec and command line when not needed
+  	* fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist 
+  	* new: update documentation with regard to offline and docker installation
+ 	* new: renamed command `nagios_check` to `health`
+	* new: agent `http_gw` now supports upload
+	* new: added option `sdk_url` to install SDK from local file for offline install
+	* new: check new gem version periodically
+	* new: the --fields= option, support -_fieldname_ to remove a field from default fields
+	* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
+	* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
+	* new: AoC a password can be provided for a public link
+	* new: `conf doc` take an optional parameter to go to a section
+	* new: initial support for Faspex 5 Beta 1
+
+* 4.0.0
 
 	* now available as open source at [https://github.com/IBM/aspera-cli](https://github.com/IBM/aspera-cli) with general cleanup
 	* changed default tool name from `mlia` to `ascli`
@@ -3271,7 +3813,7 @@ So, it evolved into `ascli`:
 
 * 0.10.6
 
-	* FaspManager: transfer spec `authentication` no more needed for local tranfer to use aspera public keys. public keys will be used if there is a token and no key or password is provided.
+	* FaspManager: transfer spec `authentication` no more needed for local tranfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
 	* gem version requirements made more open
 
 * 0.10.5
@@ -3498,11 +4040,15 @@ Breaking change:
 
   * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in [option preset](#lprt) "default".
 
-# BUGS
+# BUGS, FEATURES, CONTRIBUTION
 
-* This is best effort code without official support, dont expect full capabilities. This code is not supported by IBM/Aspera. You can contact the author for bugs or features.
+For issues or feature requests use the Github repository and issues.
 
-## only one value for any option
+You can also contribute to this open source project.
+
+One can also create one's own command nplugin.
+
+## Only one value for any option
 
 Some commands and sub commands may ask for the same option name.
 Currently, since option definition is position independant (last one wins), it is not possible
@@ -3516,7 +4062,8 @@ This happens typically for the `node` sub command, e.g. identify the node by nam
 
 ## ED255519 key not supported
 
-ED255519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
+ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
+
 Without this deactivation, if such key was present the following error was generated:
 
 ```
@@ -3526,7 +4073,17 @@ OpenSSH keys only supported if ED25519 is available
 Which meant that you do not have ruby support for ED25519 SSH keys.
 You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
 
-# TODO
+## Error "Remote host is not who we expected"
+
+`ascp` version 4.x changed the algorithm used to check the SSH server certificate. To ignore the certificate (SSH fingerprint) add option on client side:
+
+```
+--ts=@json:'{"sshfp":null}'
+```
+
+Refer to ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server): if you have access to server side, basically disable other SSH host keys than RSA.
+
+## Miscelaneous
 
 * remove rest and oauth classes and use ruby standard gems:
 
@@ -3542,9 +4099,3 @@ You may either install the suggested Gems, or remove your ed25519 key from your
 * Going through proxy: use env var http_proxy and https_proxy, no_proxy
 
 * easier use with https://github.com/pmq20/ruby-packer
-
-# Contribution
-
-Send comments !
-
-Create your own plugin !