aspera-cli 4.8.0 → 4.9.0

data/README.md

@@ -2,7 +2,7 @@
 
 [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
 
-Version : 4.8.0
+Version : 4.9.0
 
 Laurent/2016-2022
 
@@ -58,33 +58,12 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
 
 For scripting and ad'hoc command line operations, `ascli` is perfect.
 
-## <a id="parsing"></a>Notations, Shell and Command line parsing
+## Notations, Shell, Examples
 
 In examples, command line operations are shown using a shell such: `bash` or `zsh`.
 
 Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
 
-`ascli` is typically executed in a shell, either interactively or in a script. `ascli` receives its arguments from this shell.
-
-On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before `ascli` (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to `ascli`. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
-
-On Windows, `cmd.exe` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
-
-* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
-* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
-
-In case of doubt of argument values after parsing test like this:
-
-```bash
-ascli conf echo "Hello World" arg2 3
-```
-
-```bash
-"Hello World"
-ERROR: Argument: unprocessed values: ["arg2", "3"]
-```
-
-`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (`ascli`), next command line arguments are shown in the error message.
 
 ## Quick Start
 
@@ -99,7 +78,7 @@ ascli --version
 ```
 
 ```bash
-4.8.0
+4.9.0
 ```
 
 ### First use
@@ -574,6 +553,191 @@ Using `ascli` with plugin `server` for command line gives advantages over ascp:
 
 Moreover all `ascp` options are supported either through transfer spec parameters and with the possibility to provide `ascp` arguments directly when the `direct` agent is used (`EX_ascp_args`).
 
+### <a id="parsing"></a>Command line parsing, Special Characters
+
+`ascli` is typically executed in a shell, either interactively or in a script.
+`ascli` receives its arguments from this shell (through Operating System).
+
+#### Shell parsing for Linux, Unix, Macos
+
+On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
+In this environment the shell parses the command line, possibly replacing variables, etc...
+see [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
+Then it builds a list of arguments and then `ascli` (Ruby) is executed.
+Ruby receives a list parameters from shell and gives it to `ascli`.
+So special character handling (quotes, spaces, env vars, ...) is first done in the shell.
+
+#### Shell parsing for Windows
+
+On Windows, `cmd.exe` is typically used.
+Windows process creation does not receive the list of arguments but just the whole line.
+It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
+
+* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
+* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
+
+#### Extended Values (JSON, Ruby, ...)
+
+Some of the CLI parameters are expected to be [Extended Values](#extended), i.e. not a simple strings, but a complex structure (Hash, Array).
+Typically, the `@json:` modifier is used, it expects a JSON string. JSON itself has some special syntax: for example `"` is used to denote strings.
+
+#### Testing Extended Values
+
+In case of doubt of argument values after parsing, one can test using command `config echo`. `config echo` takes exactly **one** argument which can use the [Extended Value](#extended) syntax. Unprocessed command line arguments are shown in the error message.
+
+Example: The shell parses three arguments (as strings: `1`, `2` and `3`), so the additional two arguments are not processed by the `echo` command.
+
+```bash
+ascli conf echo 1 2 3
+```
+
+```bash
+"1"
+ERROR: Argument: unprocessed values: ["2", "3"]
+```
+
+`config echo` displays the value of the first argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
+Note that it gets its value after shell command line parsing and `ascli` extended value parsing.
+
+In the following examples (using a POSIX shell, such as `bash`), several sample commands are provided when equivalent.
+For all example, most of special character handling is not specific to `ascli`: It depoends on the underlying syntax: shell , JSON, etc...
+Depending on the case, a different `format` is used to display the actual value.
+
+For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
+
+Double quotes are processed by the shell to create a single string argument.
+For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash.
+
+```bash
+ascli conf echo "Hello World" --format=text
+ascli conf echo 'Hello World' --format=text
+ascli conf echo Hello\ World --format=text
+```
+
+```output
+Hello World
+```
+
+#### Using a shell variable, parsed by shell, in an extended value
+
+To be evaluated by shell, the shell variable must not be in single quotes.
+Note we use a simple variable here: the variable is not necessarily an environment variable.
+Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell. 
+
+```bash
+MYVAR="Hello World"
+ascli conf echo @json:'{"title":"'$MYVAR'"}' --format=json
+ascli conf echo @json:{\"title\":\"$MYVAR\"} --format=json
+```
+
+```json
+{"title":"Hello World"}
+```
+
+#### Double quote in strings in command line
+
+Double quote is a shell special character.
+Like any shell special character, it can be protected either by preceding with a backslash or by enclosing in a single quote.
+
+```bash
+ascli conf echo \"
+ascli conf echo '"'
+```
+
+```output
+"
+```
+
+Double quote in JSON is a little tricky because `"` is special both for the shell and JSON. Both shell and JSON syntax allow to protect `"`, but only the shell allows protection using single quote.
+
+```bash
+ascli conf echo @json:'"\""' --format=text
+ascli conf echo @json:\"\\\"\" --format=text
+ascli conf echo @ruby:\'\"\' --format=text
+```
+
+```output
+"
+```
+
+Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional `\` is added to protect the `"` for JSON. But as `\` is also shell special, then it is protected by another `\`.
+  
+#### Shell and JSON or Ruby special characters in extended value
+
+Construction of values with special characters is done like this:
+
+* First select a syntax to represent the extended value, e.g. JSON or Ruby
+
+* Write the expression using this syntax, for example, using JSON:
+
+```json
+{"title":"Test \" ' & \\"}
+```
+
+or using Ruby:
+
+```ruby
+{"title"=>"Test \" ' & \\"}
+{'title'=>%q{Test " ' & \\}}
+```
+
+Both `"` and `\` are special characters for JSON and Ruby and can be protected with `\` (unless Ruby's extended single quote notation `%q` is used).
+  
+* Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding `\` for each character to protect, or by enclosing in single quote:
+
+```bash
+ascli conf echo @json:{\"title\":\"Test\ \\\"\ \'\ \&\ \\\\\"} --format=json
+ascli conf echo @json:'{"title":"Test \" '\'' & \\"}' --format=json
+ascli conf echo @ruby:"{'title'=>%q{Test \" ' & \\\\}}" --format=json
+```
+
+```json
+{"title":"Test \" ' & \\"}
+```
+
+#### Reading special characters interractively
+
+If `ascli` is used interractively (a user typing on terminal), it is easy to require the user to type values:
+
+```bash
+ascli conf echo @ruby:"{'title'=>gets.chomp}" --format=json
+```
+
+`gets` is Ruby's method of terminal input (terminated by `\n`), and `chomp` removes the trailing `\n`.
+
+#### Extended value using special characters read from environmental variables or files
+
+Using a text editor or shell: create a file `title.txt` (and env var) that contains exactly the text required: `Test " ' & \` :
+
+```bash
+export MYTITLE='Test " '\'' & \'
+echo -n $MYTITLE > title.txt
+```
+
+Using those values will not require any escaping of characters since values do not go through shell or JSON parsing.
+
+If the value is to be assigned directly to an option of ascli, then you can directly use the content of the file or env var using the `@file:` or `@env:` readers:
+
+```bash
+ascli conf echo @file:title.txt --format=text
+ascli conf echo @env:MYTITLE --format=text
+```
+
+```output
+Test " ' & \
+```
+
+If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any ruby code in expression, including reading from file or env var. In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.
+
+```bash
+ascli conf echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
+ascli conf echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
+```
+
+```json
+{"title":"Test \" ' & \\"}
+```
+
 ### Arguments : Commands and options
 
 Arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
@@ -677,6 +841,7 @@ If transposition of single object is not desired, use option: `transpose_single`
 
 The style of output can be set using the `format` parameter, supporting:
 
+* `text` : Value as String
 * `table` : Text table
 * `ruby` : Ruby code
 * `json` : JSON code
@@ -1692,6 +1857,28 @@ For example the option `--ts=@json:'{"EX_ascp_args":["-DDL-"]}'` will activate d
 This is useful if the transfer fails.
 To store ascp logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
 
+> Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
+
+In addition to standard methods described in section [File List](#file_list), it is possible to specify the list of file using those additional methods:
+
+* Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_file_list`
+
+```javascript
+--sources=@ts --ts=@json:'{"EX_file_list":"filelist.txt"}'
+```
+
+* Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`
+
+```javascript
+--sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","myfilelist"]}'
+```
+
+> File lists is shown here, there are also similar options for file pair lists.
+
+> Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
+
+> Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
+
 #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
 
 By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
@@ -1702,6 +1889,7 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
 Transfer Server using the Node API, either on a local or remote node.
 Parameters provided in option `transfer_info` are:
 
+<table>
 <tr><th>Name</th><th>Type</th><th>Description</th></tr>
 <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
 <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
@@ -1802,7 +1990,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 
 <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr>
 <tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr>
-<tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Passphrase used for at rest encryption or decryption. Prefer to use standard: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
+<tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
 <tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr>
 <tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr>
 <tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr>
@@ -1810,7 +1998,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 <tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr>
 <tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr>
 <tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr>
-<tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr>
+<tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr>
 <tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr>
 <tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr>
 <tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr>
@@ -1820,7 +2008,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 <tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
 <tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr>
 <tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr>
-<tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source. The source and destination arguments must be directories that have matching names. Objects on the destination that have the same name but different type or size as objects on the source are not deleted.<br/>(--delete-before-transfer)</td></tr>
+<tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer)</td></tr>
 <tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
 <tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr>
 <tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr>
@@ -1839,10 +2027,11 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 <tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
 <tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
 <tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr>
-<tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr>
-<tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr>
-<tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value. (0=no file is split)<br/>(--multi-session-threshold)</td></tr>
+<tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The relative path to which the files will be moved after the transfer at the source side.<br/>(--move-after-transfer)</td></tr>
+<tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/></td></tr>
+<tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold)</td></tr>
 <tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr>
+<tr><td>password</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.</td></tr>
 <tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr>
 <tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr>
 <tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr>
@@ -1854,7 +2043,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 <tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr>
 <tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr>
 <tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr>
-<tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr>
+<tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr>
 <tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr>
 <tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr>
 <tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr>
@@ -1866,10 +2055,11 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
 <tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr>
 <tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr>
 <tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr>
-<tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr>
+<tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr>
 <tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr>
+<tr><td>src_base</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64)</td></tr>
 <tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr>
-<tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr>
+<tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr>
 <tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr>
 <tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr>
 <tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr>
@@ -1896,89 +2086,98 @@ As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path
 The option `to_folder` provides an equivalent and convenient way to change this parameter:
 `--to-folder=<path>` .
 
-#### List of files for transfers
+#### <a id="file_list"></a>List of files for transfers
 
-When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
-So, by default, the list of files to transfer will be simply specified on the command line:
+When uploading, downloading or sending files, the user must specify the list of files to transfer.
 
-```bash
-ascli server upload ~/mysample.file secondfile
-```
+By default the list of files to transfer is simply provided on the command line.
 
-This is equivalent to:
+The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
+The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
 
-```bash
-ascli server upload --sources=@args ~/mysample.file secondfile
-```
+In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
+So, eventually, the list of files to transfer is provided to the transfer agent using the [*transfer-spec*](#transferspec) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
+The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
 
-More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [*transfer-spec*](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
+Possible values for option `sources` are:
 
-Note that this is different from the "ascp" command line. The paradigm used by `ascli` is:
-all transfer parameters are kept in [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
+* `@args` : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with `-`) are considered as source files.
+So, by default, the list of files to transfer will be simply specified on the command line. Example:
 
-For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
+  ```bash
+  ascli server upload ~/first.file secondfile
+  ```
 
-* `@args` : (default value) the list of files is directly provided at the end of the command line (see at the beginning of this section).
+  This is the same as (with default values):
 
-* an [Extended Value](#extended) holding an *Array of String*. Examples:
+  ```bash
+  ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile
+  ```
 
-```javascript
---sources=@json:'["file1","file2"]'
-```
+* an [Extended Value](#extended) with type **Array of String**
 
-```bash
---sources=@lines:@stdin:
-```
+  > Note: extended values can be tested with the command `conf echo`
 
-```ruby
---sources=@ruby:'File.read("myfilelist").split("\n")'
-```
+  Examples:
 
-* `@ts` : the user provides the list of files directly in the `ts` option, in its `paths` field. Example:
+  * Using extended value
 
-```javascript
---sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
-```
+    Create the file list:
 
-providing a file list directly to ascp:
+    ```bash
+    echo ~/mysample.file > myfilelist.txt
+    echo secondfile >> myfilelist.txt
+    ```
 
-```javascript
-... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
-```
+    Use the file list: one path per line:
 
-* Not recommended: It is possible to specify bare ascp arguments using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`.
+    ```ruby
+    --sources=@lines:@file:myfilelist.txt
+    ```
 
-```javascript
---sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
-```
+  * Using JSON array
 
-This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
-One must specify a dummy list in the [*transfer-spec*](#transferspec), which will be overridden by the `ascp` command line provided.
-(TODO) In next version, dummy source paths can be removed.
+    ```javascript
+    --sources=@json:'["file1","file2"]'
+    ```
 
-In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
+  * Using STDIN, one path per line
 
-* `list` : (default) the path of destination is the same as source
-* `pair` : in that case, the first element is the first source, the second element is the first destination, and so on.
+    ```bash
+    --sources=@lines:@stdin:
+    ```
 
-Example:
+  * Using ruby code (one path per line in file)
 
-```bash
-ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
-```
+    ```ruby
+    --sources=@ruby:'File.read("myfilelist.txt").split("\n")'
+    ```
 
-Internally, when transfer agent [`direct`](#agt_direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pair-list` is provided in `ts` in `EX_ascp_args`.
+* `@ts` : the user provides the list of files directly in the `paths` field of transfer spec (option `ts`).
+Examples:
 
-Note the special case when the source files are located on "Aspera on Cloud", i.e. using access keys and the `file id` API:
+  * Using transfer spec
 
-* All files must be in the same source folder.
-* If there is a single file : specify the full path
-* For multiple files, specify the source folder as first item in the list followed by the list of file names.
+  ```javascript
+  --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
+  ```
 
-Source files are located on "Aspera on cloud", when :
+The option `src_type` allows specifying if the list specified in option `sources` is a simple file list or if it is a file pair list.
+
+> Note: Option `src_type` is not used if option `sources` is set to `@ts`
+
+Supported values for `src_type` are:
 
-* the server is Aspera on Cloud, and making a download / recv
-* the agent is Aspera on Cloud, and making an upload / send
+* `list` : (default) the path of destination is the same as source and each entry is a source file path
+* `pair` : the first element is the first source, the second element is the first destination, and so on.
+
+Example: Source file `200KB.1` is renamed `sample1` on destination:
+
+```bash
+ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
+```
+
+> Note there are some specific rules to specify file list when using "Aspera on Cloud", refer to the AoC plugin section.
 
 #### <a id="multisession"></a>Support of multi-session
 
@@ -2018,13 +2217,13 @@ activating CSEAR consists in using transfer spec parameters:
 
 Example: parameter to download a faspex package and decrypt on the fly
 
-```json
+```javascript
 --ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
 ```
 
 Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
 
-```json
+```javascript
 --ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
 ```
 
@@ -2216,7 +2415,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
 ```text
 ascli -h
 NAME
-        ascli -- a command line tool for Aspera Applications (v4.8.0)
+        ascli -- a command line tool for Aspera Applications (v4.9.0)
 
 SYNOPSIS
         ascli COMMANDS [OPTIONS] [ARGS]
@@ -2248,7 +2447,7 @@ ARGS
 OPTIONS: global
         --interactive=ENUM           use interactive input of missing params: yes, [no]
         --ask-options=ENUM           ask even optional options: yes, [no]
-        --format=ENUM                output format: [table], ruby, json, jsonpp, yaml, csv, nagios
+        --format=ENUM                output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
         --display=ENUM               output only some information: [info], data, error
         --fields=VALUE               comma separated list of fields, or ALL, or DEF
         --select=VALUE               select only some items in lists, extended value: hash (column, value)
@@ -2279,6 +2478,7 @@ OPTIONS:
         --value=VALUE                extended value for create, update, list filter
         --property=VALUE             name of property to set
         --id=VALUE                   resource identifier (modify,delete,show)
+        --bulk=ENUM                  Bulk operation (only some): yes, [no]
         --config-file=VALUE          read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
     -N, --no-default                 do not load default configuration for plugin
         --override=ENUM              Wizard: override existing value: yes, [no]
@@ -2301,11 +2501,11 @@ OPTIONS:
         --plugin-folder=VALUE        folder where to find additional plugins
         --ts=VALUE                   override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
         --local-resume=VALUE         set resume policy (Hash, use @json: prefix), current=
-        --to-folder=VALUE            destination folder for downloaded files
-        --sources=VALUE              list of source files (see doc)
-        --transfer-info=VALUE        parameters for transfer agent
+        --to-folder=VALUE            destination folder for transfered files
+        --sources=VALUE              how list of transfered files is provided (@args,@ts,Array)
         --src-type=ENUM              type of file list: list, pair
         --transfer=ENUM              type of transfer agent: direct, node, connect, httpgw, trsdk
+        --transfer-info=VALUE        parameters for transfer agent
         --progress=ENUM              type of progress bar: none, native, multi
 
 
@@ -2469,23 +2669,21 @@ OPTIONS:
         --name=VALUE                 resource name
         --path=VALUE                 file or folder path
         --link=VALUE                 public link to shared resource
-        --new-user-option=VALUE      new user creation option
+        --new-user-option=VALUE      new user creation option for unknown package recipients
         --from-folder=VALUE          share to share source folder
         --scope=VALUE                OAuth scope for AoC API calls
-        --bulk=ENUM                  bulk operation: yes, [no]
         --default-ports=ENUM         use standard FASP ports or get from node api: yes, [no]
         --validate-metadata=ENUM     validate shared inbox metadata: yes, [no]
 
 
 COMMAND: server
-SUBCOMMANDS: health nodeadmin userdata configurator ctl download upload browse delete rename ls rm mv du info mkdir cp df md5sum
+SUBCOMMANDS: health download upload browse delete rename ls rm mv du info mkdir cp df md5sum
 OPTIONS:
         --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
         --username=VALUE             username to log in
         --password=VALUE             user's password
-        --ssh-keys=VALUE             ssh key path list (Array or single)
-        --ssh-options=VALUE          ssh options (Hash)
-        --cmd-prefix=VALUE           prefix to add for as cmd execution, e.g. sudo or /opt/aspera/bin 
+        --ssh-keys=VALUE             SSH key path list (Array or single)
+        --ssh-options=VALUE          SSH options (Hash)
 
 
 COMMAND: console
@@ -2502,6 +2700,12 @@ OPTIONS:
 
 Note that actions and parameter values can be written in short form.
 
+### Bulk creation and deletion of resources
+
+Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
+In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
+This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
+
 ## <a id="aoc"></a>Plugin: Aspera on Cloud
 
 Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
@@ -2545,7 +2749,7 @@ ascli config wizard --value=aoc
 
 ### <a id="aocmanual"></a>Configuration: using manual setup
 
-If you used the wizard (recommended): skip this section.
+> If you used the wizard (recommended): skip this section.
 
 #### Configuration details
 
@@ -2706,7 +2910,7 @@ Execute:
 ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=laurent.martin.aspera@fr.ibm.com
 ```
 
-Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.
+Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the `@file:` prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the `@val:` prefix is added.
 
 After this last step, commands do not require web login anymore.
 
@@ -2750,11 +2954,6 @@ The `admin` command allows several administrative tasks (and require admin privi
 
 It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
 
-#### Bulk creation and deletion of resource
-
-Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
-In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
-
 #### Listing resources
 
 The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
@@ -3053,7 +3252,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
 :.............:.........:..................................:
 ```
 
-other query parameters:
+Other query parameters:
 
 ```javascript
 {"workspace_membership_through":true,"include_indirect":true}
@@ -3227,40 +3426,64 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
 +-----+---------+
 ```
 
-#### Example: Create a node
+#### Example: Create a Node
 
 AoC nodes as actually composed with two related entities:
 
 * An access key created on the Transfer Server (HSTS/ATS)
 * a `node` resource in the AoC application.
 
-The web UI allows creation of both entities in one shot but not the CLI for more flexibility.
-Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation.
+The web UI allows creation of both entities in one shot.
+For more flexibility, `ascli` allows this in two separate steps.
+Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
 
 So, for example, the creation of a node using ATS in IBM Cloud looks like (see other example in this manual):
 
-* create the access key on ATS
+* Create the access key on ATS
 
-```javascript
-ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
-```
+  The creation options are the ones of ATS API, refer to the [section on ATS](#ats_params) for more details and examples.
 
-Take a note of the randomly generated `id` and `secret`.
+  ```javascript
+  ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
+  ```
 
-* Retrieve the ATS node address
+  Once executed, the access key `id` and `secret`, randomly generated by the node api, is displayed: take note of it as the secrets will not be disclosed again.
 
-```bash
-ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
-```
+* Create the AoC node entity
 
-* Create the node entity
+  First, Retrieve the ATS node address
 
-```javascript
-ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
-```
+  ```bash
+  ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
+  ```
+
+  Then use the returned address for the `url` key to actually create the AoC Node entity:
+
+  ```javascript
+  ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
+  ```
 
 Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
 
+### List of files to transfer
+
+Source files are provided as a list with the `sources` option. Refer to section [File list](#file_list)
+
+Note the special case when the source files are located on "Aspera on Cloud" (i.e. using access keys and the `file id` API).
+
+Source files are located on "Aspera on cloud", when :
+
+* the server is Aspera on Cloud, and executing a download or recv
+* the agent is Aspera on Cloud, and executing an upload or send
+
+In this case:
+
+* If there is a single file : specify the full path
+* Else, if there are multiple files:
+  * All source files must be in the same source folder
+  * Specify the source folder as first item in the list
+  * followed by the list of file names.
+
 ### Packages
 
 The webmail-like application.
@@ -3328,6 +3551,31 @@ shbxid=$(ascli aoc packages shared_inboxes show name 'My Shared Inbox' --format=
 ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
 ```
 
+#### Example: Send a package with files from the Files app
+
+Find files in Files app:
+
+```bash
+ascli aoc files browse /src_folder
+```
+
+```bash
++------------------------------+--------+----------------+--------------+----------------------+--------------+
+| name                         | type   | recursive_size | size         | modified_time        | access_level |
++------------------------------+--------+----------------+--------------+----------------------+--------------+
+| sample_video                 | link   |                |              | 2020-11-29T22:49:09Z | edit         |
+| 100G                         | file   |                | 107374182400 | 2021-04-21T18:19:25Z | edit         |
+| 10M.dat                      | file   |                | 10485760     | 2021-05-18T08:22:39Z | edit         |
+| Test.pdf                     | file   |                | 1265103      | 2022-06-16T12:49:55Z | edit         |
++------------------------------+--------+----------------+--------------+----------------------+--------------+
+```
+
+Let's send a package with the file `10M.dat` from subfolder /src_folder in a package:
+
+```bash
+ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc package send --value=@json:'{"name":"test","recipients":["laurent.martin.aspera@fr.ibm.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
+```
+
 #### <a id="aoccargo"></a>Receive new packages only (Cargo)
 
 It is possible to automatically download new packages, like using Aspera Cargo:
@@ -3352,26 +3600,47 @@ Folder sharing app.
 
 #### Download Files
 
-Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as  a list with the `sources` option. Nevertheless, consider this:
-
-* if only one source is provided, it is downloaded
-* if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
+Download of files is straightforward using command: `aoc files download`
 
 #### Shared folders
 
-* list shared folders in node
+Shared folder by users are managed through **permissions**.
+For creation, parameters are the same as for node api [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
+`ascli` expects the same payload for creation, but it will automatically populated required tags if needed.
+Also, the pseudo key `with` is added: it will lookup the name in the contacts and fill the proper type and id.
+The pseudo parameter `link_name` allows changing default "shared as" name.
+
+* List permissions on a shared folder as user
+
+```bash
+ascli aoc files file --path=/shared_folder_test1 perm list
+```
+
+* Share a personal folder with other users
+
+```bash
+ascli aoc files file --path=/shared_folder_test1 perm create @json:'{"with":"laurent"}'
+```
+
+* Revoke shared access
+
+```bash
+ascli aoc files file --path=/shared_folder_test1 perm delete 6161
+```
+
+* List shared folders in node
 
 ```bash
 ascli aoc admin res node --id=8669 shared_folders
 ```
 
-* list shared folders in workspace
+* List shared folders in workspace
 
 ```bash
 ascli aoc admin res workspace --id=10818 shared_folders
 ```
 
-* list members of shared folder
+* List members of shared folder
 
 ```bash
 ascli aoc admin res node --id=8669 v4 perm 82 show
@@ -3503,6 +3772,8 @@ aoc files browse /
 aoc files browse / -N --link=my_aoc_publink_folder
 aoc files delete /testsrc
 aoc files download --transfer=connect /200KB.1
+aoc files file permission --path=/ascli_test list
+aoc files file show --path=/200KB.1
 aoc files file show my_file_id
 aoc files find / --value='\.partial$'
 aoc files http_node_download --to-folder=. /200KB.1
@@ -3518,6 +3789,7 @@ aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
 aoc files v3 info
 aoc org -N --link=my_aoc_publink_recv_from_aocuser
 aoc organization
+aoc packages browse "my_package_id" /contents
 aoc packages list
 aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
 aoc packages recv "my_package_id" --to-folder=.
@@ -3614,6 +3886,10 @@ ascli ats api_key create
 ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
 ```
 
+### <a id="ats_params"></a>ATS Access key creation parameters
+
+When creating an ATS access key, the option `params` must contain an extended value with the creation parameters. Those asre directly the parameters expected by the [ATS API](https://developer.ibm.com/apis/catalog?search=%22Aspera%20ATS%20API%22).
+
 ### Misc. Examples
 
 Example: create access key on IBM Cloud (softlayer):
@@ -3678,11 +3954,6 @@ Modern mode is to use the node API and transfer tokens.
 ### Server sample commands
 
 ```bash
-server -N -Ptst_hstsfaspex_ssh -Plocal_user configurator get_node_data
-server -N -Ptst_hstsfaspex_ssh -Plocal_user ctl all:status
-server -N -Ptst_hstsfaspex_ssh -Plocal_user health app_services --format=nagios
-server -N -Ptst_hstsfaspex_ssh -Plocal_user health asctlstatus --format=nagios --cmd-prefix='sudo '
-server -N -Ptst_hstsfaspex_ssh -Plocal_user nodeadmin -- -l
 server -N -Ptst_server_bykey -Plocal_user br /
 server browse /
 server browse folder_1/target_hot
@@ -3700,6 +3971,10 @@ server md5sum NEW_SERVER_FOLDER/testfile.bin
 server mkdir NEW_SERVER_FOLDER --logger=stdout
 server mkdir folder_1/target_hot
 server mv folder_1/200KB.2 folder_1/to.delete
+server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER 
+server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
+server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
+server upload --sources=@ts --ts=@json:'{"EX_file_pair_list":"'"filepairlist.txt"'"}'
 server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
 server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
 server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
@@ -3708,7 +3983,7 @@ server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX
 server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
 ```
 
-### Authentication
+### Authentication on Server with SSH session
 
 If SSH is the session control protocol (i.e. not WSS), then following session authentication methods are supported:
 
@@ -3723,8 +3998,10 @@ If no SSH password or key is provided and a transfer token is provided in transf
 ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
 ```
 
-The value of the option `ssh_keys` can be a single value or an array.
-Each value is a path to a private key and is expanded (`~` is replaced with the user's home folder).
+> Note: If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
+
+The value of the `ssh_keys` option can be a single value or an array.
+Each value is a **path** to a private key and is expanded (`~` is replaced with the user's home folder).
 
 Examples:
 
@@ -3734,8 +4011,10 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
 ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
 ```
 
-The underlying ssh library `net::ssh` provides several options that may be used depending on environment.
-By default the ssh library expect that an ssh-agent is running.
+For non-transfer related command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
+For a list of SSH client options, refer to the ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html).
+
+By default the SSH library expect that a local ssh-agent is running.
 
 On Linux, if you get an error message such as:
 
@@ -3749,12 +4028,12 @@ or on Windows:
 ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
 ```
 
-This means that you don't have such an ssh agent running, then:
+This means that you don't have such an SSH agent running, then:
 
 * check env var: `SSH_AGENT_SOCK`
-* check if the ssh key is protected with a passphrase
+* check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
 * [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
-* To disable use of `ssh-agent`, use the option `ssh_option` like this:
+* To disable the use of `ssh-agent`, use the option `ssh_options` like this:
 
 ```bash
 ascli server --ssh-options=@ruby:'{use_agent: false}' ...
@@ -3891,7 +4170,7 @@ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"myst
 ```bash
 node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
 node -N -Ptst_node_preview access_key delete aoc_1
-node access_key do my_aoc_ak_name br
+node access_key do my_aoc_ak_name br /
 node access_key list
 node api_details
 node async bandwidth 1
@@ -4022,19 +4301,19 @@ faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":
 
 * List all shared inboxes
 
-```json
+```javascript
 ascli faspex5 admin res shared list --value=@json:'{"all":true}' --fields=id,name
 ```
 
 * Create Metadata profile
 
-```json
+```javascript
 ascli faspex5 admin res metadata_profiles create --value=@json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
 ```
 
 * Create a Shared inbox with specific metadata profile
 
-```json
+```javascript
 ascli faspex5 admin res shared create --value=@json:'{"name":"the shared inbox","metadata_profile_id":1}'
 ```
 
@@ -4108,7 +4387,8 @@ if `id` is set to `ALL`, then all packages are downloaded, and if option `once_o
 
 ### Sending a Package
 
-The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
+The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`.
+The contents of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages), consult it for extended supported parameters.
 
 Example:
 
@@ -4795,8 +5075,7 @@ Transfer is: <%=global_transfer_status%>
 
 ## Tool: `asession`
 
-This gem comes with a second executable tool providing a simplified standardized interface
-to start a FASP session: `asession`.
+This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
 
 It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transferspec) is:
 
@@ -4810,7 +5089,7 @@ This makes it easy to integrate with any language provided that one can spawn a
 
 The tool expect one single argument: a [*transfer-spec*](#transferspec).
 
-If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
+If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
 
 Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
 
@@ -4827,18 +5106,24 @@ Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters ar
 
 <table>
 <tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
-<tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>any</td></tr>
-<tr><td>additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
+<tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>many</td></tr>
+<tr><td>required additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
 <tr><td>startup</td><td>JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn)</td><td>command line arguments</td><td>API</td><td>daemon</td></tr>
 <tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
-<tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp</td><td>any with ascp</td><td>any with ascp and transferdaemon</td></tr></table>
+<tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp (and SDK if compiled)</td><td>any with ascp</td><td>any with ascp and transfer daemon</td></tr></table>
 
 ### Simple session
 
-```javascript
-MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
+Create a file `session.json` with:
+
+```json
+{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}
+````
+
+Then start the session:
 
-echo "${MY_TSPEC}"|asession
+```
+asession < session.json
 ```
 
 ### Asynchronous commands and Persistent session
@@ -5043,7 +5328,7 @@ Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including EC
 
 Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
 
-```json
+```javascript
 --ts=@json:'{"sshfp":null}'
 ```