aspera-cli 4.9.0 → 4.11.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (95) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/BUGS.md +20 -0
  4. data/CHANGELOG.md +509 -0
  5. data/CONTRIBUTING.md +118 -0
  6. data/README.md +1241 -916
  7. data/bin/ascli +4 -4
  8. data/bin/asession +11 -11
  9. data/docs/test_env.conf +32 -21
  10. data/examples/aoc.rb +4 -4
  11. data/examples/dascli +16 -9
  12. data/examples/faspex4.rb +8 -8
  13. data/examples/node.rb +12 -12
  14. data/examples/server.rb +10 -10
  15. data/lib/aspera/aoc.rb +273 -266
  16. data/lib/aspera/ascmd.rb +56 -54
  17. data/lib/aspera/ats_api.rb +4 -4
  18. data/lib/aspera/cli/basic_auth_plugin.rb +15 -12
  19. data/lib/aspera/cli/extended_value.rb +5 -5
  20. data/lib/aspera/cli/formater.rb +64 -64
  21. data/lib/aspera/cli/info.rb +2 -2
  22. data/lib/aspera/cli/listener/line_dump.rb +1 -1
  23. data/lib/aspera/cli/listener/logger.rb +1 -1
  24. data/lib/aspera/cli/listener/progress.rb +5 -6
  25. data/lib/aspera/cli/listener/progress_multi.rb +14 -19
  26. data/lib/aspera/cli/main.rb +66 -67
  27. data/lib/aspera/cli/manager.rb +112 -110
  28. data/lib/aspera/cli/plugin.rb +57 -36
  29. data/lib/aspera/cli/plugins/alee.rb +4 -4
  30. data/lib/aspera/cli/plugins/aoc.rb +309 -670
  31. data/lib/aspera/cli/plugins/ats.rb +44 -46
  32. data/lib/aspera/cli/plugins/bss.rb +10 -10
  33. data/lib/aspera/cli/plugins/config.rb +497 -378
  34. data/lib/aspera/cli/plugins/console.rb +12 -12
  35. data/lib/aspera/cli/plugins/cos.rb +18 -20
  36. data/lib/aspera/cli/plugins/faspex.rb +112 -114
  37. data/lib/aspera/cli/plugins/faspex5.rb +71 -46
  38. data/lib/aspera/cli/plugins/node.rb +379 -283
  39. data/lib/aspera/cli/plugins/orchestrator.rb +46 -46
  40. data/lib/aspera/cli/plugins/preview.rb +122 -114
  41. data/lib/aspera/cli/plugins/server.rb +137 -83
  42. data/lib/aspera/cli/plugins/shares.rb +30 -29
  43. data/lib/aspera/cli/plugins/sync.rb +13 -33
  44. data/lib/aspera/cli/transfer_agent.rb +60 -59
  45. data/lib/aspera/cli/version.rb +1 -1
  46. data/lib/aspera/colors.rb +3 -3
  47. data/lib/aspera/command_line_builder.rb +27 -27
  48. data/lib/aspera/cos_node.rb +22 -20
  49. data/lib/aspera/data_repository.rb +1 -1
  50. data/lib/aspera/environment.rb +35 -15
  51. data/lib/aspera/fasp/agent_base.rb +15 -15
  52. data/lib/aspera/fasp/agent_connect.rb +23 -21
  53. data/lib/aspera/fasp/agent_direct.rb +66 -64
  54. data/lib/aspera/fasp/agent_httpgw.rb +141 -78
  55. data/lib/aspera/fasp/agent_node.rb +23 -21
  56. data/lib/aspera/fasp/agent_trsdk.rb +20 -20
  57. data/lib/aspera/fasp/error.rb +3 -2
  58. data/lib/aspera/fasp/error_info.rb +11 -8
  59. data/lib/aspera/fasp/installation.rb +79 -79
  60. data/lib/aspera/fasp/listener.rb +1 -1
  61. data/lib/aspera/fasp/parameters.rb +86 -71
  62. data/lib/aspera/fasp/parameters.yaml +7 -4
  63. data/lib/aspera/fasp/resume_policy.rb +8 -8
  64. data/lib/aspera/fasp/transfer_spec.rb +35 -2
  65. data/lib/aspera/fasp/uri.rb +7 -7
  66. data/lib/aspera/faspex_gw.rb +7 -5
  67. data/lib/aspera/hash_ext.rb +3 -3
  68. data/lib/aspera/id_generator.rb +5 -5
  69. data/lib/aspera/keychain/encrypted_hash.rb +38 -105
  70. data/lib/aspera/keychain/macos_security.rb +128 -57
  71. data/lib/aspera/log.rb +7 -7
  72. data/lib/aspera/nagios.rb +19 -18
  73. data/lib/aspera/node.rb +209 -35
  74. data/lib/aspera/oauth.rb +37 -36
  75. data/lib/aspera/open_application.rb +19 -11
  76. data/lib/aspera/persistency_action_once.rb +4 -4
  77. data/lib/aspera/persistency_folder.rb +16 -15
  78. data/lib/aspera/preview/file_types.rb +8 -8
  79. data/lib/aspera/preview/generator.rb +67 -67
  80. data/lib/aspera/preview/utils.rb +27 -27
  81. data/lib/aspera/proxy_auto_config.js +41 -41
  82. data/lib/aspera/proxy_auto_config.rb +21 -14
  83. data/lib/aspera/rest.rb +72 -67
  84. data/lib/aspera/rest_call_error.rb +2 -1
  85. data/lib/aspera/rest_error_analyzer.rb +18 -17
  86. data/lib/aspera/rest_errors_aspera.rb +16 -16
  87. data/lib/aspera/secret_hider.rb +15 -13
  88. data/lib/aspera/ssh.rb +11 -10
  89. data/lib/aspera/sync.rb +158 -44
  90. data/lib/aspera/temp_file_manager.rb +2 -2
  91. data/lib/aspera/uri_reader.rb +4 -4
  92. data/lib/aspera/web_auth.rb +14 -13
  93. data.tar.gz.sig +0 -0
  94. metadata +11 -36
  95. metadata.gz.sig +0 -0
data/README.md CHANGED
@@ -1,10 +1,13 @@
1
1
  # Command Line Interface for IBM Aspera products
2
+ <!-- markdownlint-disable MD033 MD003 MD053 -->
2
3
 
3
4
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
4
5
 
5
- Version : 4.9.0
6
+ ##
6
7
 
7
- Laurent/2016-2022
8
+ Version : 4.11.0
9
+
10
+ Laurent/2016-2023
8
11
 
9
12
  This gem provides the `ascli` Command Line Interface to IBM Aspera software.
10
13
 
@@ -14,9 +17,11 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
14
17
 
15
18
  Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
16
19
 
17
- Minimum required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
20
+ Minimum required Ruby version: >= 2.4.
21
+
22
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
18
23
 
19
- [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
24
+ [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs&sort=title_asc)
20
25
  [Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
21
26
 
22
27
  Release notes: see [CHANGELOG.md](CHANGELOG.md)
@@ -33,26 +38,25 @@ One can also [create one's own plugin](#createownplugin).
33
38
 
34
39
  `ascli` is designed to be used as a command line tool to:
35
40
 
36
- * execute commands on Aspera products
37
- * transfer to/from Aspera products
41
+ - Execute commands remotely on Aspera products
42
+ - Transfer to/from Aspera products
38
43
 
39
44
  So it is designed for:
40
45
 
41
- * Interactive operations on a text terminal (typically, VT100 compatible)
42
- * Batch operations in (shell) scripts (e.g. cron job)
46
+ - Interactive operations on a text terminal (typically, VT100 compatible), e.g. for maintenance
47
+ - Scripting, e.g. batch operations in (shell) scripts (e.g. cron job)
43
48
 
44
49
  `ascli` can be seen as a command line tool integrating:
45
50
 
46
- * a configuration file (config.yaml)
47
- * advanced command line options
48
- * cURL (for REST calls)
49
- * Aspera transfer (ascp)
51
+ - A configuration file (config.yaml)
52
+ - Advanced command line options
53
+ - cURL (for REST calls)
54
+ - Aspera transfer (`ascp`)
50
55
 
51
- One might be tempted to use it as an integration element, e.g. by building a command line programmatically, and then executing it. It is generally not a good idea.
52
- For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
56
+ If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
53
57
 
54
- * Product APIs (REST) : e.g. AoC, Faspex, node
55
- * Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
58
+ - Product APIs (REST) : e.g. AoC, Faspex, node
59
+ - Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
56
60
 
57
61
  Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
58
62
 
@@ -64,7 +68,6 @@ In examples, command line operations are shown using a shell such: `bash` or `zs
64
68
 
65
69
  Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
66
70
 
67
-
68
71
  ## Quick Start
69
72
 
70
73
  This section guides you from installation, first use and advanced use.
@@ -78,7 +81,7 @@ ascli --version
78
81
  ```
79
82
 
80
83
  ```bash
81
- 4.9.0
84
+ 4.11.0
82
85
  ```
83
86
 
84
87
  ### First use
@@ -110,10 +113,10 @@ ascli server browse /
110
113
 
111
114
  If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [option preset](#lprt) for the server's authentication options. The following example will:
112
115
 
113
- * create a [option preset](#lprt)
114
- * define it as default for `server` plugin
115
- * list files in a folder
116
- * download a file
116
+ - create a [option preset](#lprt)
117
+ - define it as default for `server` plugin
118
+ - list files in a folder
119
+ - download a file
117
120
 
118
121
  ```bash
119
122
  ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_
@@ -128,7 +131,7 @@ ascli config preset set default server myserver
128
131
  ```
129
132
 
130
133
  ```output
131
- updated: default&rarr;server to myserver
134
+ updated: default &rarr; server to myserver
132
135
  ```
133
136
 
134
137
  ```bash
@@ -159,7 +162,7 @@ ascli server download /aspera-test-dir-large/200MB
159
162
  ```
160
163
 
161
164
  ```output
162
- Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
165
+ Time: 00:00:02 =========================================================== 100% 100 Mbps Time: 00:00:00
163
166
  complete
164
167
  ```
165
168
 
@@ -175,9 +178,13 @@ It is possible to install *either* directly on the host operating system (Linux,
175
178
 
176
179
  The direct installation is recommended and consists in installing:
177
180
 
178
- * [Ruby](#ruby) (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
179
- * [aspera-cli](#the_gem)
180
- * [Aspera SDK (ascp)](#fasp_prot)
181
+ - [Ruby](#ruby)
182
+ - [aspera-cli](#the_gem)
183
+ - [Aspera SDK (`ascp`)](#fasp_prot)
184
+
185
+ Ruby version: >= 2.4.
186
+
187
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
181
188
 
182
189
  The following sections provide information on the various installation methods.
183
190
 
@@ -185,45 +192,136 @@ An internet connection is required for the installation. If you don't have inter
185
192
 
186
193
  ### Docker container
187
194
 
188
- This method installs a docker image that contains: Ruby, `ascli` and the FASP SDK.
195
+ The image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
196
+ The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
197
+ To use the container, ensure that you have `docker` (or `podman`) installed.
189
198
 
190
- The image is: <https://hub.docker.com/r/martinlaurent/ascli>. It is built from this [Dockerfile](Dockerfile).
199
+ ```bash
200
+ docker --version
201
+ ```
191
202
 
192
- Ensure that you have Docker installed.
203
+ **Wanna start quickly ?** With an interactive shell ? Execute this:
193
204
 
194
205
  ```bash
195
- docker --version
206
+ docker run --tty --interactive --entrypoint bash martinlaurent/ascli:latest
196
207
  ```
197
208
 
198
- An example of wrapping script is provided: `dascli`. If you have installed `ascli`, the script `dascli` can be found:
209
+ Then, execute individual `ascli` commands such as:
199
210
 
200
211
  ```bash
201
- cp $(ascli conf gem path)/../examples/dascli ascli
212
+ ascli conf init
213
+ ascli conf preset overview
214
+ ascli conf ascp info
215
+ ascli server ls /
202
216
  ```
203
217
 
204
- Alternatively [download from the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
218
+ That is simple, but there are limitations:
219
+
220
+ - Everything happens in the container
221
+ - Any generated file in the container will be lost on container (shell) exit. Including configuration files and downloaded files.
222
+ - No possibility to upload files located on the host system
223
+
224
+ The container image is built from this [Dockerfile](Dockerfile): the entry point is `ascli` and the default command is `help`.
225
+
226
+ The container can also be execute for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
205
227
 
206
228
  ```bash
207
- curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli
229
+ docker run --rm --tty --interactive martinlaurent/ascli:latest
208
230
  ```
209
231
 
232
+ For more convenience, you may define a shell alias:
233
+
210
234
  ```bash
211
- chmod a+x ascli
235
+ alias ascli='docker run --rm --tty --interactive martinlaurent/ascli:latest'
236
+ ```
237
+
238
+ Then, you can execute the container like a local command:
239
+
240
+ ```bash
241
+ ascli -v
242
+ ```
243
+
244
+ ```text
245
+ 4.11.0
246
+ ```
247
+
248
+ In order to keep persistency of configuration on the host,
249
+ you should mount your user's config folder to the container.
250
+ To enable write access, a possibility is to run as `root` in the container (and set the default configuration folder to `/home/cliuser/.aspera/ascli`).
251
+ Add options:
252
+
253
+ ```bash
254
+ --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli
255
+ ```
256
+
257
+ > **Note:** if you are using a `podman machine`, e.g. on Macos , make sure that the folder is also shared between the VM and the host, so that sharing is: container &rarr; VM &rarr; Host: `podman machine init ... --volume="/Users:/Users"`
258
+
259
+ As shown in the quick start, if you prefer to keep a running container with a shell and `ascli` available,
260
+ you can change the entry point, add option:
261
+
262
+ ```bash
263
+ --entrypoint bash
212
264
  ```
213
265
 
214
- Install the container image:
266
+ You may also probably want that files downloaded in the container are in fact placed on the host.
267
+ In this case you need also to mount the shared transfer folder:
215
268
 
216
269
  ```bash
217
- ./ascli install
270
+ --volume $HOME/xferdir:/xferfiles
218
271
  ```
219
272
 
220
- Note that ascli is run in the container, so transfers are also executed in the container (not calling host, like for the regular ascli).
273
+ > **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
221
274
 
222
- The wrapping script maps the folder `/usr/src/app/config` in the container to configuration folder `$HOME/.aspera/ascli` on host.
223
- This allows having persistent configuration.
275
+ And if you want all the above, simply use all the options:
276
+
277
+ ```bash
278
+ alias asclish="docker run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
279
+ ```
280
+
281
+ ```bash
282
+ export xferdir=$HOME/xferdir
283
+ mkdir -p $xferdir
284
+ chmod -R 777 $xferdir
285
+ mkdir -p $HOME/.aspera/ascli
286
+ asclish
287
+ ```
224
288
 
225
- To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
226
- To add local storage as a volume edit the script: `ascli` and add a `--volume` stanza near the existing one.
289
+ A convenience sample script is also provided: download the script [`dascli`](../examples/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
290
+
291
+ > **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli conf gem path)/../examples/dascli ascli`
292
+
293
+ Some environment variables can be set for this script to adapt its behaviour:
294
+
295
+ | env var | description | default | example |
296
+ |--------------|------------------------------------|--------------------------|--------------------------|
297
+ | ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascliconfig` |
298
+ | docker_args | additional options to `docker` | &lt;empty&gt; | `--volume /Users:/Users` |
299
+ | image | container image name | martinlaurent/ascli | |
300
+ | version | container image version | latest | `4.8.0.pre` |
301
+
302
+ The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
303
+ (value expected in the container).
304
+ This allows having persistent configuration on the host.
305
+
306
+ To add local storage as a volume, you can use the env var `docker_args`:
307
+
308
+ Example of use:
309
+
310
+ ```bash
311
+ curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli
312
+ chmod a+x ascli
313
+ export xferdir=$HOME/xferdir
314
+ mkdir -p $xferdir
315
+ chmod -R 777 $xferdir
316
+ export docker_args="--volume $xferdir:/xferfiles"
317
+
318
+ ./ascli conf init
319
+
320
+ echo 'Local file to transfer' > $xferdir/samplefile.txt
321
+ ./ascli server upload /xferfiles/samplefile.txt --to-folder=/Upload
322
+ ```
323
+
324
+ > **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
227
325
 
228
326
  ### <a id="ruby"></a>Ruby
229
327
 
@@ -231,7 +329,9 @@ Use this method to install on the native host.
231
329
 
232
330
  A ruby interpreter is required to run the tool or to use the gem and tool.
233
331
 
234
- Required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
332
+ Required Ruby version: >= 2.4.
333
+
334
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
235
335
 
236
336
  *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
237
337
 
@@ -239,7 +339,7 @@ Refer to the following sections for a proposed method for specific operating sys
239
339
 
240
340
  The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, macOS, Windows with cygwin, etc...).
241
341
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
242
- If you have a simpler better way to install Ruby : use it ! (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
342
+ If you have a simpler better way to install Ruby : use it !
243
343
 
244
344
  #### Generic: RVM: single user installation (not root)
245
345
 
@@ -247,18 +347,14 @@ Use this method which provides more flexibility.
247
347
 
248
348
  Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
249
349
 
250
- Install the 2 keys
251
-
252
- ```bash
253
- gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
254
- ```
255
-
256
350
  Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
257
351
 
258
352
  ```bash
259
353
  \curl -sSL https://get.rvm.io | bash -s stable
260
354
  ```
261
355
 
356
+ Follow on-screen instructions to install keys, and then re-execute the command.
357
+
262
358
  If you keep the same terminal (not needed if re-login):
263
359
 
264
360
  ```bash
@@ -306,11 +402,10 @@ rvm version
306
402
 
307
403
  Install Latest stable Ruby:
308
404
 
309
- * Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) &rarr; **Downloads**.
310
- * Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
311
- * Execute the installer which installs by default in: `RubyVV-x64` (VV is the version number)
312
- * At the end of the installation procedure execute the Msys2 installer (ridk install) and select option 3 (msys and mingw)
313
- * Install the "mime info" file as specified in [this section](#mimeinfo).
405
+ - Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) &rarr; **Downloads**.
406
+ - Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
407
+ - Execute the installer which installs by default in: `C:\RubyVV-x64` (VV is the version number)
408
+ - At the end of the installation procedure, the Msys2 installer is automatically executed, select option 3 (msys and mingw)
314
409
 
315
410
  #### macOS: pre-installed or `brew`
316
411
 
@@ -330,15 +425,20 @@ brew install ruby
330
425
 
331
426
  If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
332
427
 
333
- Example: Centos 8 Stream
428
+ Example: RHEL 8 and 9: basic installation
429
+
430
+ ```bash
431
+ yum module install ruby:3.1
432
+ ```
433
+
434
+ Example: RHEL 8, Centos 8 Stream: with extensions to compile native gems
334
435
 
335
436
  ```bash
336
437
  yum install make automake gcc gcc-c++ kernel-devel
337
438
  yum install redhat-rpm-config
338
439
  dnf module reset ruby
339
- dnf module enable ruby:3.0
340
- dnf module -y install ruby:3.0/common
341
- gem install aspera-cli
440
+ dnf module enable ruby:3.1
441
+ dnf module -y install ruby:3.1/common
342
442
  ```
343
443
 
344
444
  Other examples:
@@ -391,7 +491,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
391
491
 
392
492
  <https://www.jruby.org/download>
393
493
 
394
- Note that using jruby the startup time is longer than the native ruby, but transfer speed is not impacted (executed by `ascp` binary).
494
+ > **Note:** Using jruby the startup time is longer than the native ruby, but the transfer speed is not impacted (executed by `ascp` binary).
395
495
 
396
496
  ### <a id="the_gem"></a>`aspera-cli` gem
397
497
 
@@ -420,8 +520,8 @@ ascli conf check_update
420
520
  Most file transfers will be done using the FASP protocol, using `ascp`.
421
521
  Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
422
522
 
423
- * ascp
424
- * aspera-license (in same folder, or ../etc)
523
+ - `ascp`
524
+ - aspera-license (in same folder, or ../etc)
425
525
 
426
526
  This can be installed either be installing an Aspera transfer software, or using an embedded command:
427
527
 
@@ -443,11 +543,11 @@ The format is: `file:///<path>`, where `<path>` can be either a relative path (n
443
543
 
444
544
  If the embedded method is not used, the following packages are also suitable:
445
545
 
446
- * IBM Aspera Connect Client (Free)
447
- * IBM Aspera Desktop Client (Free)
448
- * IBM Aspera CLI (Free)
449
- * IBM Aspera High Speed Transfer Server (Licensed)
450
- * IBM Aspera High Speed Transfer EndPoint (Licensed)
546
+ - IBM Aspera Connect Client (Free)
547
+ - IBM Aspera Desktop Client (Free)
548
+ - IBM Aspera CLI (Free)
549
+ - IBM Aspera High Speed Transfer Server (Licensed)
550
+ - IBM Aspera High Speed Transfer EndPoint (Licensed)
451
551
 
452
552
  For instance, Aspera Connect Client can be installed
453
553
  by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
@@ -460,33 +560,34 @@ Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, b
460
560
 
461
561
  ### <a id="offline_install"></a>Installation in air gapped environment
462
562
 
463
- Note that currently no pre-packaged version exist yet.
563
+ > **Note:** no pre-packaged version is provided.
564
+
464
565
  A method to build one is provided here:
465
566
 
466
567
  The procedure:
467
568
 
468
- * Follow the non-root installation procedure with RVM, including gem
469
- * Archive (zip, tar) the main RVM folder (includes ascli):
569
+ - Follow the non-root installation procedure with RVM, including gem
570
+ - Archive (zip, tar) the main RVM folder (includes ascli):
470
571
 
471
572
  ```bash
472
573
  cd $HOME && tar zcvf rvm-ascli.tgz .rvm
473
574
  ```
474
575
 
475
- * Get the Aspera SDK.
576
+ - Get the Aspera SDK.
476
577
 
477
578
  ```bash
478
579
  ascli conf --show-config --fields=sdk_url
479
580
  ```
480
581
 
481
- * Download the SDK archive from that URL.
582
+ - Download the SDK archive from that URL.
482
583
 
483
584
  ```bash
484
585
  curl -Lso SDK.zip https://ibm.biz/aspera_sdk
485
586
  ```
486
587
 
487
- * Transfer those 2 files to the target system
588
+ - Transfer those 2 files to the target system
488
589
 
489
- * On target system
590
+ - On target system
490
591
 
491
592
  ```bash
492
593
  cd $HOME
@@ -498,7 +599,7 @@ source ~/.rvm/scripts/rvm
498
599
  ascli conf ascp install --sdk-url=file:///SDK.zip
499
600
  ```
500
601
 
501
- * Add those lines to shell init (`.profile`)
602
+ - Add those lines to shell init (`.profile`)
502
603
 
503
604
  ```bash
504
605
  source ~/.rvm/scripts/rvm
@@ -508,27 +609,27 @@ source ~/.rvm/scripts/rvm
508
609
 
509
610
  The `aspera-cli` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
510
611
 
511
- * IBM Aspera High Speed Transfer Server (FASP and Node)
512
- * IBM Aspera on Cloud (including ATS)
513
- * IBM Aspera Faspex
514
- * IBM Aspera Shares
515
- * IBM Aspera Console
516
- * IBM Aspera Orchestrator
517
- * and more...
612
+ - IBM Aspera High Speed Transfer Server (FASP and Node)
613
+ - IBM Aspera on Cloud (including ATS)
614
+ - IBM Aspera Faspex
615
+ - IBM Aspera Shares
616
+ - IBM Aspera Console
617
+ - IBM Aspera Orchestrator
618
+ - and more...
518
619
 
519
620
  `ascli` provides the following features:
520
621
 
521
- * Supports most Aspera server products (on-premise and SaaS)
522
- * Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
523
- * Supports Commands, Option values and Parameters shortcuts
524
- * FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
525
- * Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
526
- * Allows transfers from products to products, essentially at node level (using the node transfer agent)
527
- * Supports FaspStream creation (using Node API)
528
- * Supports Watchfolder creation (using Node API)
529
- * Additional command plugins can be written by the user
530
- * Supports download of faspex and Aspera on Cloud "external" links
531
- * Supports "legacy" ssh based FASP transfers and remote commands (ascmd)
622
+ - Supports most Aspera server products (on-premise and SaaS)
623
+ - Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
624
+ - Supports Commands, Option values and Parameters shortcuts
625
+ - FASP [Transfer Agents](#agents) can be: local `ascp`, or Connect Client, or any transfer node
626
+ - Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
627
+ - Allows transfers from products to products, essentially at node level (using the node transfer agent)
628
+ - Supports FaspStream creation (using Node API)
629
+ - Supports Watchfolder creation (using Node API)
630
+ - Additional command plugins can be written by the user
631
+ - Supports download of faspex and Aspera on Cloud "external" links
632
+ - Supports "legacy" ssh based FASP transfers and remote commands (ascmd)
532
633
 
533
634
  Basic usage is displayed by executing:
534
635
 
@@ -544,12 +645,12 @@ Not all `ascli` features are fully documented here, the user may explore command
544
645
 
545
646
  If you want to use `ascp` directly as a command line, refer to IBM Aspera documentation of either [Desktop Client](https://www.ibm.com/docs/en/asdc), [Endpoint](https://www.ibm.com/docs/en/ahte) or [Transfer Server](https://www.ibm.com/docs/en/ahts) where [a section on `ascp` can be found](https://www.ibm.com/docs/en/ahts/4.4?topic=linux-ascp-transferring-from-command-line).
546
647
 
547
- Using `ascli` with plugin `server` for command line gives advantages over ascp:
648
+ Using `ascli` with plugin `server` for command line gives advantages over `ascp`:
548
649
 
549
- * automatic resume on error
550
- * configuration file
551
- * choice of transfer agents
552
- * integrated support of multi-session
650
+ - automatic resume on error
651
+ - configuration file
652
+ - choice of transfer agents
653
+ - integrated support of multi-session
553
654
 
554
655
  Moreover all `ascp` options are supported either through transfer spec parameters and with the possibility to provide `ascp` arguments directly when the `direct` agent is used (`EX_ascp_args`).
555
656
 
@@ -573,8 +674,8 @@ On Windows, `cmd.exe` is typically used.
573
674
  Windows process creation does not receive the list of arguments but just the whole line.
574
675
  It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
575
676
 
576
- * [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
577
- * [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
677
+ - [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
678
+ - [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
578
679
 
579
680
  #### Extended Values (JSON, Ruby, ...)
580
681
 
@@ -597,7 +698,8 @@ ERROR: Argument: unprocessed values: ["2", "3"]
597
698
  ```
598
699
 
599
700
  `config echo` displays the value of the first argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
600
- Note that it gets its value after shell command line parsing and `ascli` extended value parsing.
701
+
702
+ > **Note:** It gets its value after shell command line parsing and `ascli` extended value parsing.
601
703
 
602
704
  In the following examples (using a POSIX shell, such as `bash`), several sample commands are provided when equivalent.
603
705
  For all example, most of special character handling is not specific to `ascli`: It depoends on the underlying syntax: shell , JSON, etc...
@@ -606,7 +708,7 @@ Depending on the case, a different `format` is used to display the actual value.
606
708
  For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
607
709
 
608
710
  Double quotes are processed by the shell to create a single string argument.
609
- For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash.
711
+ For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash. <!-- markdownlint-disable-line -->
610
712
 
611
713
  ```bash
612
714
  ascli conf echo "Hello World" --format=text
@@ -621,8 +723,9 @@ Hello World
621
723
  #### Using a shell variable, parsed by shell, in an extended value
622
724
 
623
725
  To be evaluated by shell, the shell variable must not be in single quotes.
624
- Note we use a simple variable here: the variable is not necessarily an environment variable.
625
- Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
726
+ Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
727
+
728
+ > **Note:** we use a simple variable here: the variable is not necessarily an environment variable.
626
729
 
627
730
  ```bash
628
731
  MYVAR="Hello World"
@@ -666,9 +769,9 @@ Here a single quote or a backslash protects the double quote to avoid shell proc
666
769
 
667
770
  Construction of values with special characters is done like this:
668
771
 
669
- * First select a syntax to represent the extended value, e.g. JSON or Ruby
772
+ - First select a syntax to represent the extended value, e.g. JSON or Ruby
670
773
 
671
- * Write the expression using this syntax, for example, using JSON:
774
+ - Write the expression using this syntax, for example, using JSON:
672
775
 
673
776
  ```json
674
777
  {"title":"Test \" ' & \\"}
@@ -683,7 +786,7 @@ or using Ruby:
683
786
 
684
787
  Both `"` and `\` are special characters for JSON and Ruby and can be protected with `\` (unless Ruby's extended single quote notation `%q` is used).
685
788
 
686
- * Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding `\` for each character to protect, or by enclosing in single quote:
789
+ - Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding `\` for each character to protect, or by enclosing in single quote:
687
790
 
688
791
  ```bash
689
792
  ascli conf echo @json:{\"title\":\"Test\ \\\"\ \'\ \&\ \\\\\"} --format=json
@@ -748,10 +851,10 @@ There are two types of command line arguments: Commands and Options. Example :
748
851
  ascli command subcommand --option-name=VAL1 VAL2
749
852
  ```
750
853
 
751
- * executes *command*: `command subcommand`
752
- * with one *option*: `option_name`
753
- * this option is given a *value* of: `VAL1`
754
- * the command has one additional *argument*: `VAL2`
854
+ - executes *command*: `command subcommand`
855
+ - with one *option*: `option_name`
856
+ - this option is given a *value* of: `VAL1`
857
+ - the command has one additional *argument*: `VAL2`
755
858
 
756
859
  When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `ascli conf ov` is the same as `ascli config overview`.
757
860
 
@@ -761,16 +864,16 @@ The value of options and arguments is evaluated with the [Extended Value Syntax]
761
864
 
762
865
  All options, e.g. `--log-level=debug`, are command line arguments that:
763
866
 
764
- * start with `--`
765
- * have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
766
- * have a value, separated from name with a `=`
767
- * can be used by prefix, provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
867
+ - start with `--`
868
+ - have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
869
+ - have a value, separated from name with a `=`
870
+ - can be used by prefix, provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
768
871
 
769
872
  Exceptions:
770
873
 
771
- * some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
772
- * some options (flags) don't take a value, e.g. `-r`
773
- * the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
874
+ - some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
875
+ - some options (flags) don't take a value, e.g. `-r`
876
+ - the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
774
877
 
775
878
  ```bash
776
879
  ascli config echo -- --sample
@@ -780,15 +883,15 @@ ascli config echo -- --sample
780
883
  "--sample"
781
884
  ```
782
885
 
783
- Note that here, `--sample` is taken as an argument, and not as an option, due to `--`.
886
+ > **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
784
887
 
785
888
  Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
786
889
 
787
890
  The value for *any* options can come from the following locations (in this order, last value evaluated overrides previous value):
788
891
 
789
- * [Configuration file](#configfile).
790
- * Environment variable
791
- * Command line
892
+ - [Configuration file](#configfile).
893
+ - Environment variable
894
+ - Command line
792
895
 
793
896
  Environment variable starting with prefix: ASCLI_ are taken as option values, e.g. `ASCLI_OPTION_NAME` is for `--option-name`.
794
897
 
@@ -804,11 +907,11 @@ Some options and parameters are mandatory and other optional. By default, the to
804
907
 
805
908
  The behavior can be controlled with:
806
909
 
807
- * --interactive=&lt;yes|no&gt; (default=yes if STDIN is a terminal, else no)
808
- * yes : missing mandatory parameters/options are asked to the user
809
- * no : missing mandatory parameters/options raise an error message
810
- * --ask-options=&lt;yes|no&gt; (default=no)
811
- * optional parameters/options are asked to user
910
+ - --interactive=&lt;yes|no&gt; (default=yes if STDIN is a terminal, else no)
911
+ - yes : missing mandatory parameters/options are asked to the user
912
+ - no : missing mandatory parameters/options raise an error message
913
+ - --ask-options=&lt;yes|no&gt; (default=no)
914
+ - optional parameters/options are asked to user
812
915
 
813
916
  ### Output
814
917
 
@@ -819,12 +922,12 @@ The information displayed depends on the action.
819
922
 
820
923
  Depending on action, the output will contain:
821
924
 
822
- * `single_object` : displayed as a 2 dimensional table: one line per attribute, first column is attribute name, and second is attribute value. Nested hashes are collapsed.
823
- * `object_list` : displayed as a 2 dimensional table: one line per item, one column per attribute.
824
- * `value_list` : a table with one column.
825
- * `empty` : nothing
826
- * `status` : a message
827
- * `other_struct` : a complex structure that cannot be displayed as an array
925
+ - `single_object` : displayed as a 2 dimensional table: one line per attribute, first column is attribute name, and second is attribute value. Nested hashes are collapsed.
926
+ - `object_list` : displayed as a 2 dimensional table: one line per item, one column per attribute.
927
+ - `value_list` : a table with one column.
928
+ - `empty` : nothing
929
+ - `status` : a message
930
+ - `other_struct` : a complex structure that cannot be displayed as an array
828
931
 
829
932
  #### Format of output
830
933
 
@@ -841,13 +944,13 @@ If transposition of single object is not desired, use option: `transpose_single`
841
944
 
842
945
  The style of output can be set using the `format` parameter, supporting:
843
946
 
844
- * `text` : Value as String
845
- * `table` : Text table
846
- * `ruby` : Ruby code
847
- * `json` : JSON code
848
- * `jsonpp` : JSON pretty printed
849
- * `yaml` : YAML
850
- * `csv` : Comma Separated Values
947
+ - `text` : Value as String
948
+ - `table` : Text table
949
+ - `ruby` : Ruby code
950
+ - `json` : JSON code
951
+ - `jsonpp` : JSON pretty printed
952
+ - `yaml` : YAML
953
+ - `csv` : Comma Separated Values
851
954
 
852
955
  #### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
853
956
 
@@ -866,21 +969,21 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
866
969
  :...............................:..................................:...........:
867
970
  ```
868
971
 
869
- Note that `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
972
+ > **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
870
973
 
871
974
  #### Verbosity of output
872
975
 
873
976
  Output messages are categorized in 3 types:
874
977
 
875
- * `info` output contain additional information, such as number of elements in a table
876
- * `data` output contain the actual output of the command (object, or list of objects)
877
- * `error`output contain error messages
978
+ - `info` output contain additional information, such as number of elements in a table
979
+ - `data` output contain the actual output of the command (object, or list of objects)
980
+ - `error`output contain error messages
878
981
 
879
982
  The option `display` controls the level of output:
880
983
 
881
- * `info` displays all messages: `info`, `data`, and `error`
882
- * `data` display `data` and `error` messages
883
- * `error` display only error messages.
984
+ - `info` displays all messages: `info`, `data`, and `error`
985
+ - `data` display `data` and `error` messages
986
+ - `error` display only error messages.
884
987
 
885
988
  By default, secrets are removed from output: option `show_secrets` defaults to `no`, unless `display` is `data`, to allows piping results.
886
989
  To hide secrets from output, set option `show_secrets` to `no`.
@@ -889,12 +992,12 @@ To hide secrets from output, set option `show_secrets` to `no`.
889
992
 
890
993
  By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
891
994
 
892
- * DEF : default display of columns (that's the default, when not set)
893
- * ALL : all columns available
894
- * a,b,c : the list of attributes specified by the comma separated list
895
- * Array extended value: for instance, @json:'["a","b","c"]' same as above
896
- * +a,b,c : add selected properties to the default selection.
897
- * -a,b,c : remove selected properties from the default selection.
995
+ - DEF : default display of columns (that's the default, when not set)
996
+ - ALL : all columns available
997
+ - a,b,c : the list of attributes specified by the comma separated list
998
+ - Array extended value: for instance, @json:'["a","b","c"]' same as above
999
+ - +a,b,c : add selected properties to the default selection.
1000
+ - -a,b,c : remove selected properties from the default selection.
898
1001
 
899
1002
  ### <a id="extended"></a>Extended Value Syntax
900
1003
 
@@ -910,24 +1013,24 @@ The difference between reader and decoder is order and ordinality. Both act like
910
1013
 
911
1014
  The following "readers" are supported (returns value in []):
912
1015
 
913
- * @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
914
- * @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
915
- * @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
916
- * @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
917
- * @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
918
- * @stdin: : [String] read from stdin (no value on right)
919
- * @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
1016
+ - @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
1017
+ - @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
1018
+ - @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
1019
+ - @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
1020
+ - @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
1021
+ - @stdin: : [String] read from stdin (no value on right)
1022
+ - @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
920
1023
 
921
1024
  In addition it is possible to decode a value, using one or multiple decoders :
922
1025
 
923
- * @base64: [String] decode a base64 encoded string
924
- * @json: [any] decode JSON values (convenient to provide complex structures)
925
- * @zlib: [String] uncompress data
926
- * @ruby: [any] execute ruby code
927
- * @csvt: [Array] decode a titled CSV value
928
- * @lines: [Array] split a string in multiple lines and return an array
929
- * @list: [Array] split a string in multiple items taking first character as separator and return an array
930
- * @incps: [Hash] include values of presets specified by key `incps` in input hash
1026
+ - @base64: [String] decode a base64 encoded string
1027
+ - @json: [any] decode JSON values (convenient to provide complex structures)
1028
+ - @zlib: [String] uncompress data
1029
+ - @ruby: [any] execute ruby code
1030
+ - @csvt: [Array] decode a titled CSV value
1031
+ - @lines: [Array] split a string in multiple lines and return an array
1032
+ - @list: [Array] split a string in multiple items taking first character as separator and return an array
1033
+ - @incps: [Hash] include values of presets specified by key `incps` in input hash
931
1034
 
932
1035
  To display the result of an extended value, use the `config echo` command.
933
1036
 
@@ -978,7 +1081,7 @@ ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
978
1081
  {"version"=>"0.9", "hello"=>true}
979
1082
  ```
980
1083
 
981
- Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
1084
+ > **Note:** `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` are equivalent to: `@preset:config`
982
1085
 
983
1086
  ### <a id="native"></a>Structured Value
984
1087
 
@@ -1057,7 +1160,7 @@ The command `update` allows the easy creation of [option preset](#lprt) by simpl
1057
1160
  ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_ --ts=@json:'{"precalculate_job_size":true}'
1058
1161
  ```
1059
1162
 
1060
- * This creates a [option preset](#lprt) `demo_server` with all provided options.
1163
+ - This creates a [option preset](#lprt) `demo_server` with all provided options.
1061
1164
 
1062
1165
  The command `set` allows setting individual options in a [option preset](#lprt).
1063
1166
 
@@ -1090,6 +1193,8 @@ If necessary, the configuration file can opened in a text editor with:
1090
1193
  ascli config open
1091
1194
  ```
1092
1195
 
1196
+ > **Note:** this starts the editor specified by env var `EDITOR` if defined.
1197
+
1093
1198
  Older format for commands are still supported:
1094
1199
 
1095
1200
  ```bash
@@ -1108,7 +1213,7 @@ This preset name is reserved and contains an array of key-value , where the key
1108
1213
 
1109
1214
  When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option --no-default (or -N) is used.
1110
1215
 
1111
- Note that special plugin name: `config` can be associated with a preset that is loaded initially, typically used for default values.
1216
+ > **Note:** Special plugin name: `config` can be associated with a preset that is loaded initially, typically used for default values.
1112
1217
 
1113
1218
  Operations on this preset are done using regular `config` operations:
1114
1219
 
@@ -1124,9 +1229,9 @@ ascli config preset get default _plugin_name_
1124
1229
  "_default_preset_for_plugin_"
1125
1230
  ```
1126
1231
 
1127
- #### <a id="lplugconf"></a>Special Plugin: config
1232
+ #### <a id="config"></a>Plugin: `config`: CLI Configuration
1128
1233
 
1129
- Plugin `config` (not to be confused with Option preset config) is used to configure `ascli` but it also contains global options.
1234
+ Plugin `config` is used to configure `ascli` and also contains global options.
1130
1235
 
1131
1236
  When `ascli` starts, it looks for the `default` Option preset and if there is a value for `config`, if so, it loads the option values for any plugin used.
1132
1237
 
@@ -1195,16 +1300,16 @@ demo_server:
1195
1300
 
1196
1301
  We can see here:
1197
1302
 
1198
- * The configuration was created with CLI version 0.3.7
1199
- * the default [option preset](#lprt) to load for `server` plugin is : `demo_server`
1200
- * the [option preset](#lprt) `demo_server` defines some parameters: the URL and credentials
1201
- * the default [option preset](#lprt) to load in any case is : `cli_default`
1303
+ - The configuration was created with CLI version 0.3.7
1304
+ - the default [option preset](#lprt) to load for `server` plugin is : `demo_server`
1305
+ - the [option preset](#lprt) `demo_server` defines some parameters: the URL and credentials
1306
+ - the default [option preset](#lprt) to load in any case is : `cli_default`
1202
1307
 
1203
1308
  Two [option presets](#lprt) are reserved:
1204
1309
 
1205
- * `config` contains a single value: `version` showing the CLI
1310
+ - `config` contains a single value: `version` showing the CLI
1206
1311
  version used to create the configuration file. It is used to check compatibility.
1207
- * `default` is reserved to define the default [option preset](#lprt) name used for known plugins.
1312
+ - `default` is reserved to define the default [option preset](#lprt) name used for known plugins.
1208
1313
 
1209
1314
  The user may create as many [option presets](#lprt) as needed. For instance, a particular [option preset](#lprt) can be created for a particular application instance and contain URL and credentials.
1210
1315
 
@@ -1233,11 +1338,11 @@ Some options are global, some options are available only for some plugins. (the
1233
1338
 
1234
1339
  Options are loaded using this algorithm:
1235
1340
 
1236
- * If option `--no-default` (or `-N`) is specified, then no default value is loaded is loaded for the plugin
1237
- * else it looks for the name of the plugin as key in section `default`, the value is the name of the default [option preset](#lprt) for it, and loads it.
1238
- * If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or of the value is a Hash, it uses it as options values.
1239
- * Environment variables are evaluated
1240
- * Command line options are evaluated
1341
+ - If option `--no-default` (or `-N`) is specified, then no default value is loaded is loaded for the plugin
1342
+ - else it looks for the name of the plugin as key in section `default`, the value is the name of the default [option preset](#lprt) for it, and loads it.
1343
+ - If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or of the value is a Hash, it uses it as options values.
1344
+ - Environment variables are evaluated
1345
+ - Command line options are evaluated
1241
1346
 
1242
1347
  Parameters are evaluated in the order of command line.
1243
1348
 
@@ -1286,7 +1391,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=_p
1286
1391
 
1287
1392
  This can also be provisioned in a config file:
1288
1393
 
1289
- * Build [option preset](#lprt)
1394
+ - Build [option preset](#lprt)
1290
1395
 
1291
1396
  ```bash
1292
1397
  ascli config preset set shares06 url https://10.25.0.6
@@ -1306,19 +1411,19 @@ or
1306
1411
  ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=_pass_here_
1307
1412
  ```
1308
1413
 
1309
- * Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
1414
+ - Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
1310
1415
 
1311
1416
  ```bash
1312
1417
  ascli config preset set default shares shares06
1313
1418
  ```
1314
1419
 
1315
- * Display the content of configuration file in table format
1420
+ - Display the content of configuration file in table format
1316
1421
 
1317
1422
  ```bash
1318
1423
  ascli config overview
1319
1424
  ```
1320
1425
 
1321
- * Execute a command on the shares application using default parameters
1426
+ - Execute a command on the shares application using default parameters
1322
1427
 
1323
1428
  ```bash
1324
1429
  ascli shares repo browse /
@@ -1326,67 +1431,86 @@ ascli shares repo browse /
1326
1431
 
1327
1432
  ### <a id="vault"></a>Secret Vault
1328
1433
 
1329
- When a secret or password is needed, it is possible to store in the secret vault.
1434
+ Password and secrets are command options.
1435
+ They can be provided on command line, env vars, files etc.
1436
+ A more secure option is to retrieve values from a secret vault.
1330
1437
 
1331
- By default the vault is defined using option `secrets`, which can be stored in the configuration file.
1438
+ The vault is used with options `vault` and `vault_password`.
1332
1439
 
1333
- #### Using system keychain
1440
+ `vault` defines the vault to be used and shall be a Hash, example:
1334
1441
 
1335
- Only on macOS.
1442
+ ```json
1443
+ {"type":"system","name":"ascli"}
1444
+ ```
1336
1445
 
1337
- It is possible to store secrets in macOS keychain (only read supported currently).
1446
+ `vault_password` specifies the password for the vault.
1447
+ Although it can be specified on command line, for security reason you can hide the value.
1448
+ For example it can be securely specified on command line like this:
1338
1449
 
1339
- Set option `secrets` to value `system` to use the default keychain or use value `system:[name]` to use a custom keychain.
1450
+ ```bash
1451
+ export ASCLI_VAULT_PASSWORD
1452
+ read -s ASCLI_VAULT_PASSWORD
1453
+ ```
1340
1454
 
1341
- #### Modern config file format: encrypted in config file
1455
+ #### Vault: System keychain
1342
1456
 
1343
- It is possible to store and use secrets encrypted.
1344
- For this use the `config vault` command.
1457
+ > **Note:** **macOS only**
1345
1458
 
1346
- The vault can be initialized with `config vault init`
1459
+ It is possible to manage secrets in macOS keychain (only read supported currently).
1347
1460
 
1348
- Then secrets can be manipulated using commands:
1461
+ ```json
1462
+ --vault=@json:'{"type":"system","name":"ascli"}'
1463
+ ```
1349
1464
 
1350
- * `set`
1351
- * `get`
1352
- * `list`
1353
- * `delete`
1465
+ #### Vault: Encrypted file
1354
1466
 
1355
- Secrets must be uniquely identified by `url` and `username`. An optional description can be provided using option `value`.
1467
+ It is possible to store and use secrets encrypted in a file.
1356
1468
 
1357
- #### Legacy config file format
1469
+ ```json
1470
+ --vault=@json:'{"type":"file","name":"vault.bin"}'
1471
+ ```
1472
+
1473
+ `name` is the file path, absolute or relative to the config folder `ASCLI_HOME`.
1358
1474
 
1359
- THIS FORMAT WILL BE DEPRECATED
1475
+ #### Vault: Operations
1360
1476
 
1361
- The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
1477
+ For this use the `config vault` command.
1478
+
1479
+ Then secrets can be manipulated using commands:
1362
1480
 
1363
- For example, choose a repository name, for example `my_secrets`, and populate it like this:
1481
+ - `create`
1482
+ - `show`
1483
+ - `list`
1484
+ - `delete`
1364
1485
 
1365
1486
  ```bash
1366
- ascli conf id my_secrets set 'access_key1' 'secret1'
1487
+ ascli conf vault create mylabel @json:'{"password":"__value_here__","description":"for this account"}'
1488
+ ```
1367
1489
 
1368
- ascli conf id my_secrets set 'access_key2' 'secret2'
1490
+ #### <a id="config_finder"></a>Configuration Finder
1369
1491
 
1370
- ascli conf id default get config
1492
+ When a secret is needed by a sub command, the command can search for existing configurations in the config file.
1371
1493
 
1372
- cli_default
1373
- ```
1494
+ The lookup is done by comparing the service URL and username (or access key).
1495
+
1496
+ #### Securing passwords and secrets
1374
1497
 
1375
- Here above, one has already set a `config` global preset to preset `cli_default` (refer to earlier in documentation).
1376
- So the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
1498
+ A passwords can be saved in clear in a [option preset](#lprt) together with other account information (URL, username, etc...).
1499
+ Example:
1377
1500
 
1378
1501
  ```bash
1379
- ascli conf id cli_default set secrets @val:@preset:my_secrets
1502
+ `ascli` conf preset update myconf --url=... --username=... --password=...
1380
1503
  ```
1381
1504
 
1382
- A secret repository can always be selected at runtime using `--secrets=@preset:xxxx`, or `--secrets=@json:'{"accesskey1":"secret1"}'`
1383
-
1384
- To test if a secret can be found use:
1505
+ For a more secure storage one can do:
1385
1506
 
1386
1507
  ```bash
1387
- ascli conf vault get --username=access_key1
1508
+ `ascli` conf preset update myconf --url=... --username=... --password=@val:@vault:myconf.password
1509
+ `ascli` conf vault create myconf @json:'{"password":"__value_here__"}'
1388
1510
  ```
1389
1511
 
1512
+ > **Note:** use `@val:` in front of `@vault:` so that the extended value is not evaluated.
1513
+
1390
1514
  ### <a id="private_key"></a>Private Key
1391
1515
 
1392
1516
  Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5, ...).
@@ -1397,15 +1521,17 @@ The private key can be protected by a passphrase or not.
1397
1521
  If the key is protected by a passphrase, then it will be prompted.
1398
1522
  (some plugins support option `passphrase`)
1399
1523
 
1400
- Several methods can be used to generate a key pair:
1401
-
1402
- The following commands use this variable:
1524
+ The following commands use the shell variable `PRIVKEYFILE`.
1525
+ Set it to the desired safe location of the private key.
1526
+ Typically, in `$HOME/.ssh` or `$HOME/.aspera/ascli`:
1403
1527
 
1404
1528
  ```bash
1405
1529
  PRIVKEYFILE=~/.aspera/ascli/my_private_key
1406
1530
  ```
1407
1531
 
1408
- * `ascli`
1532
+ Several methods can be used to generate a key pair:
1533
+
1534
+ - `ascli`
1409
1535
 
1410
1536
  The generated key is of type RSA, by default: 4096 bit.
1411
1537
  For convenience, the public key is also extracted with extension `.pub`.
@@ -1415,23 +1541,30 @@ The key is not passphrase protected.
1415
1541
  ascli config genkey ${PRIVKEYFILE} 4096
1416
1542
  ```
1417
1543
 
1418
- * `ssh-keygen`
1544
+ - `ssh-keygen`
1419
1545
 
1420
1546
  Both private and public keys are generated, option `-N` is for passphrase.
1421
1547
 
1422
1548
  ```bash
1423
- ssh-keygen -t rsa -b 4096 -N '' -f ${PRIVKEYFILE}
1549
+ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
1424
1550
  ```
1425
1551
 
1426
- * `openssl`
1552
+ - `openssl`
1427
1553
 
1428
- openssl is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
1429
- To generate a private key pair without passphrase the following can be used on any system:
1554
+ To generate a private key pair with a passphrase the following can be used on any system:
1430
1555
 
1431
1556
  ```bash
1432
- openssl genrsa -passout pass:dummypassword -out ${PRIVKEYFILE}.protected 4096
1433
- openssl rsa -passin pass:dummypassword -in ${PRIVKEYFILE}.protected -out ${PRIVKEYFILE}
1557
+ openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE}.protected 4096
1434
1558
  openssl rsa -pubout -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.pub
1559
+ ```
1560
+
1561
+ `openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
1562
+ In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` to generate a key without passphrase.
1563
+
1564
+ If option `-nodes` is not available, the passphrase can be removed using this method:
1565
+
1566
+ ```bash
1567
+ openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE}.protected -out ${PRIVKEYFILE}
1435
1568
  rm -f ${PRIVKEYFILE}.protected
1436
1569
  ```
1437
1570
 
@@ -1505,9 +1638,9 @@ REST APIs of Aspera legacy applications (Aspera Node, Faspex, Shares, Console, O
1505
1638
 
1506
1639
  Those are using options:
1507
1640
 
1508
- * url
1509
- * username
1510
- * password
1641
+ - url
1642
+ - username
1643
+ - password
1511
1644
 
1512
1645
  Those can be provided using command line, parameter set, env var, see section above.
1513
1646
 
@@ -1528,17 +1661,17 @@ Available loggers: `stdout`, `stderr`, `syslog`.
1528
1661
 
1529
1662
  Available levels: `debug`, `info`, `warn`, `error`.
1530
1663
 
1531
- Note that when using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` option `EX_ascp_args`, see [`direct`](#agt_direct).
1664
+ > **Note:** When using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` option `EX_ascp_args`, see [`direct`](#agt_direct).
1532
1665
 
1533
1666
  Examples:
1534
1667
 
1535
- * display debugging log on `stdout`:
1668
+ - display debugging log on `stdout`:
1536
1669
 
1537
1670
  ```bash
1538
1671
  ascli conf over --log-level=debug --logger=stdout
1539
1672
  ```
1540
1673
 
1541
- * log errors to `syslog`:
1674
+ - log errors to `syslog`:
1542
1675
 
1543
1676
  ```bash
1544
1677
  ascli conf over --log-level=error --logger=syslog
@@ -1582,76 +1715,100 @@ ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
1582
1715
 
1583
1716
  Some actions may require the use of a graphical tool:
1584
1717
 
1585
- * a browser for Aspera on Cloud authentication (web auth method)
1586
- * a text editor for configuration file edition
1718
+ - a browser for Aspera on Cloud authentication (web auth method)
1719
+ - a text editor for configuration file edition
1587
1720
 
1588
1721
  By default the CLI will assume that a graphical environment is available on windows, and on other systems, rely on the presence of the "DISPLAY" environment variable.
1589
1722
  It is also possible to force the graphical mode with option --ui :
1590
1723
 
1591
- * `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1592
- * `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1724
+ - `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1725
+ - `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1593
1726
 
1594
1727
  ### Proxy
1595
1728
 
1596
- There are several types of network connections, each of them use a different mechanism to define a *proxy*.
1729
+ There are several types of network connections, each of them use a different mechanism to define a (forward) **proxy**:
1597
1730
 
1598
- #### HTTP proxy for REST calls and transfers using HTTP gateway
1599
-
1600
- To specify a HTTP proxy when ruby HTTP is used, set the `http_proxy` environment variable (lower case, preferred, or upper case).
1601
- See [Ruby findproxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
1602
-
1603
- ```bash
1604
- export http_proxy=http://myproxy.org.net:3128
1605
- ```
1731
+ - Ruby HTTP: REST and HTTPGW client
1732
+ - Legacy Aspera HTTP/S Fallback
1733
+ - Aspera FASP
1606
1734
 
1607
- Note that ruby expects a URL and `myproxy.org.net:3128` alone is not accepted.
1735
+ Refer to the following sections.
1608
1736
 
1609
- #### FASP proxy (forward) for transfers
1737
+ ### Proxy for REST and HTTPGW
1610
1738
 
1611
- To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `EX_fasp_proxy_url` (only supported with the `direct` agent).
1739
+ There are two possibilities to define an HTTP proxy to be used when Ruby HTTP is used.
1612
1740
 
1613
- #### HTTP proxy legacy Aspera HTTP fallback transfers
1741
+ The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy, e.g. `http://myproxy.org.net:3128`.
1742
+ Refer to [Ruby findproxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
1614
1743
 
1615
- To specify a proxy for legacy HTTP fallback, set the [*transfer-spec*](#transferspec) parameter: `EX_http_proxy_url` (only supported with the `direct` agent).
1616
- (It is also possible to use `EX_ascp_args` and native options in `direct`)
1744
+ > **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
1617
1745
 
1618
- #### Proxy auto config
1746
+ ```bash
1747
+ export http_proxy=http://proxy.example.com:3128
1748
+ ```
1619
1749
 
1620
- The `fpac` option allows use of a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) script defined as javascript value.
1621
- To read the script from a URL (`http:`, `https:` and `file:`), use: `@uri:`.
1622
- A minimal script can be specified, for example like this, to define the use of a local proxy:
1750
+ The `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) javascript value.
1751
+ To read the script from a URL (`http:`, `https:` and `file:`), use prefix: `@uri:`.
1752
+ A minimal script can be specified to define the use of a local proxy:
1623
1753
 
1624
1754
  ```bash
1625
- export ASCLI_FPAC='function FindProxyForURL(url, host){return "PROXY localhost:3128"}'
1755
+ ascli --fpac='function FindProxyForURL(url, host){return "PROXY localhost:3128"}' ...
1626
1756
  ```
1627
1757
 
1628
- The PAC file will be used for any HTTP/HTTPS/REST connection, but not other (e.g. FASP, HTTP fallback, HTTPGW)
1629
-
1630
- The PAC file can be tested with command: `config proxy_check`. Example, using command line option:
1758
+ The result of a PAC file can be tested with command: `config proxy_check`.
1759
+ Example, using command line option:
1631
1760
 
1632
1761
  ```bash
1633
- ascli conf proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:1234;DIRECT";}' http://example.com
1762
+ ascli conf proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:3128;DIRECT";}' http://example.com
1763
+ ```
1764
+
1765
+ ```text
1634
1766
  PROXY proxy.example.com:1234;DIRECT
1635
1767
  ```
1636
1768
 
1637
1769
  ```bash
1638
1770
  ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
1771
+ ```
1772
+
1773
+ ```text
1639
1774
  PROXY proxy.example.com:8080
1640
1775
  ```
1641
1776
 
1642
1777
  ```bash
1643
1778
  ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.com
1779
+ ```
1780
+
1781
+ ```text
1644
1782
  PROXY proxy.example.com:8080
1645
1783
  ```
1646
1784
 
1785
+ If the proxy requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
1786
+
1787
+ ```bash
1788
+ ascli --proxy-credentials=@json:'["__username_here__","__password_here__"]' ...
1789
+ ```
1790
+
1791
+ ```bash
1792
+ ascli --proxy-credentials=@list::__username_here__:__password_here__ ...
1793
+ ```
1794
+
1795
+ ### Proxy for Legacy Aspera HTTP/S Fallback
1796
+
1797
+ To specify a proxy for legacy HTTP fallback, set the [*transfer-spec*](#transferspec) parameter: `EX_http_proxy_url` (only supported with the `direct` agent).
1798
+ (It is also possible to use `EX_ascp_args` and native options in `direct`)
1799
+
1800
+ ### FASP proxy (forward) for transfers
1801
+
1802
+ To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `EX_fasp_proxy_url` (only supported with the `direct` agent).
1803
+
1647
1804
  ### <a id="client"></a>FASP configuration
1648
1805
 
1649
1806
  The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
1650
1807
 
1651
- * `show` : shows the path of ascp used
1652
- * `use` : list,download connect client versions available on internet
1653
- * `products` : list Aspera transfer products available locally
1654
- * `connect` : list,download connect client versions available on internet
1808
+ - `show` : shows the path of `ascp` used
1809
+ - `use` : list,download connect client versions available on internet
1810
+ - `products` : list Aspera transfer products available locally
1811
+ - `connect` : list,download connect client versions available on internet
1655
1812
 
1656
1813
  #### Show path of currently used `ascp`
1657
1814
 
@@ -1677,9 +1834,9 @@ ascli config ascp info
1677
1834
 
1678
1835
  #### Selection of `ascp` location for [`direct`](#agt_direct) agent
1679
1836
 
1680
- By default, `ascli` uses any found local product with ascp, including SDK.
1837
+ By default, `ascli` uses any found local product with `ascp`, including SDK.
1681
1838
 
1682
- To temporarily use an alternate ascp path use option `ascp_path` (`--ascp-path=`)
1839
+ To temporarily use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
1683
1840
 
1684
1841
  For a permanent change, the command `config ascp use` sets the same parameter for the global default.
1685
1842
 
@@ -1730,11 +1887,11 @@ ascli config ascp products list
1730
1887
 
1731
1888
  #### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
1732
1889
 
1733
- If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
1890
+ If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
1734
1891
 
1735
- Using the option use_product finds the ascp binary of the selected product.
1892
+ Using the option use_product finds the `ascp` binary of the selected product.
1736
1893
 
1737
- To permanently use the ascp of a product:
1894
+ To permanently use the `ascp` of a product:
1738
1895
 
1739
1896
  ```bash
1740
1897
  ascli config ascp products use 'Aspera Connect'
@@ -1780,7 +1937,7 @@ ascli config ascp connect version 'Aspera Connect for Mac Intel' download enclos
1780
1937
  ```
1781
1938
 
1782
1939
  ```output
1783
- Time: 00:00:02 ======================================================================= 100% 27766 KB/sec Time: 00:00:02
1940
+ Time: 00:00:02 =========================================================== 100% 27766 KB/sec Time: 00:00:02
1784
1941
  Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
1785
1942
  ```
1786
1943
 
@@ -1793,21 +1950,21 @@ This [*transfer-spec*](#transferspec) will be executed by a transfer client, her
1793
1950
 
1794
1951
  There are currently 3 agents:
1795
1952
 
1796
- * [`direct`](#agt_direct) : a local execution of `ascp`
1797
- * [`connect`](#agt_connect) : use of a local Connect Client
1798
- * [`node`](#agt_node) : use of an Aspera Transfer Node (potentially *remote*).
1799
- * [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
1800
- * [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
1953
+ - [`direct`](#agt_direct) : a local execution of `ascp`
1954
+ - [`connect`](#agt_connect) : use of a local Connect Client
1955
+ - [`node`](#agt_node) : use of an Aspera Transfer Node (potentially *remote*).
1956
+ - [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
1957
+ - [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
1801
1958
 
1802
- Note that all transfer operation are seen from the point of view of the agent.
1803
- For instance, a node agent making an "upload", or "package send" operation,
1959
+ > **Note:** All transfer operations are seen from the point of view of the agent.
1960
+ For example, a node agent executing an "upload", or "package send" operation
1804
1961
  will effectively push files to the related server from the agent node.
1805
1962
 
1806
- `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *raw* ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1963
+ `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *native* `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1807
1964
 
1808
1965
  #### <a id="agt_direct"></a>Direct
1809
1966
 
1810
- The `direct` agent directly executes a local ascp.
1967
+ The `direct` agent directly executes a local `ascp`.
1811
1968
  This is the default agent for `ascli`.
1812
1969
  This is equivalent to option `--transfer=direct`.
1813
1970
  `ascli` will detect locally installed Aspera products, including SDK, and use `ascp` from that component.
@@ -1815,18 +1972,17 @@ Refer to section [FASP](#client).
1815
1972
 
1816
1973
  The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
1817
1974
 
1818
- <table>
1819
- <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1820
- <tr><td>wss</td><td>Bool</td><td>Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: false</td></tr>
1821
- <tr><td>spawn_timeout_sec</td><td>Float</td><td>Multi session<br/>Verification time that ascp is running<br/>Default: 3</td></tr>
1822
- <tr><td>spawn_delay_sec</td><td>Float</td><td>Multi session<br/>Delay between startup of sessions<br/>Default: 2</td></tr>
1823
- <tr><td>multi_incr_udp</td><td>Bool</td><td>Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true</td></tr>
1824
- <tr><td>resume</td><td>Hash</td><td>Resume<br/>parameters<br/>See below</td></tr>
1825
- <tr><td>resume.iter_max</td><td>int</td><td>Resume<br/>Max number of retry on error<br/>Default: 7</td></tr>
1826
- <tr><td>resume.sleep_initial</td><td>int</td><td>Resume<br/>First Sleep before retry<br/>Default: 2</td></tr>
1827
- <tr><td>resume.sleep_factor</td><td>int</td><td>Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2</td></tr>
1828
- <tr><td>resume.sleep_max</td><td>int</td><td>Resume<br/>Default: 60</td></tr>
1829
- </table>
1975
+ | Name | Type | Description |
1976
+ |----------------------|-------|-------------|
1977
+ | wss | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
1978
+ | spawn_timeout_sec | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
1979
+ | spawn_delay_sec | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
1980
+ | multi_incr_udp | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
1981
+ | resume | Hash | Resume<br/>parameters<br/>See below |
1982
+ | resume.iter_max | int | Resume<br/>Max number of retry on error<br/>Default: 7 |
1983
+ | resume.sleep_initial | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
1984
+ | resume.sleep_factor | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
1985
+ | resume.sleep_max | int | Resume<br/>Default: 60 |
1830
1986
 
1831
1987
  In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
1832
1988
  Sleep between iterations is:
@@ -1849,35 +2005,46 @@ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":20}}'
1849
2005
  ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1850
2006
  ```
1851
2007
 
1852
- Note that the `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
2008
+ > **Note:** The `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
1853
2009
  In particular the field, `EX_ascp_args` which is a list of additional command line options to `ascp`.
1854
2010
 
1855
- This can be useful to activate logging using option `-L` of ascp.
2011
+ This can be useful to activate logging using option `-L` of `ascp`.
1856
2012
  For example the option `--ts=@json:'{"EX_ascp_args":["-DDL-"]}'` will activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`).
1857
2013
  This is useful if the transfer fails.
1858
- To store ascp logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
2014
+ To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
1859
2015
 
1860
- > Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
2016
+ > **Note:** Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
1861
2017
 
1862
2018
  In addition to standard methods described in section [File List](#file_list), it is possible to specify the list of file using those additional methods:
1863
2019
 
1864
- * Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_file_list`
2020
+ - Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_file_list`
1865
2021
 
1866
2022
  ```javascript
1867
2023
  --sources=@ts --ts=@json:'{"EX_file_list":"filelist.txt"}'
1868
2024
  ```
1869
2025
 
1870
- * Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`
2026
+ - Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`
1871
2027
 
1872
2028
  ```javascript
1873
2029
  --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","myfilelist"]}'
1874
2030
  ```
1875
2031
 
1876
- > File lists is shown here, there are also similar options for file pair lists.
2032
+ > **Note:** File lists is shown here, there are also similar options for file pair lists.
2033
+ >
2034
+ > **Note:** Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
2035
+ >
2036
+ > **Note:** Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
1877
2037
 
1878
- > Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
2038
+ In addition to special transfer spec parameter `EX_ascp_args`, it is possible to provide the same `ascp` options using option `ascp_opts` of `ascli`.
2039
+ This option expects an `Array`, which can be conveniently provided with extended syntax `@list:`
1879
2040
 
1880
- > Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
2041
+ ```bash
2042
+ ascli server download /aspera-test-dir-large/200MB --ascp-opts=@list:' -l 10m -k 3'
2043
+ ```
2044
+
2045
+ > **Note:** Using `@list:`, the use of quotes and leading character "space" here is important: The `@list:` expects a single string which must not be parsed by the shell (so protected with quotes) and the leading space is the separator character.
2046
+ >
2047
+ > **Note:** The option `ascp_opts` are appended to `EX_ascp_args` if present.
1881
2048
 
1882
2049
  #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1883
2050
 
@@ -1889,13 +2056,12 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
1889
2056
  Transfer Server using the Node API, either on a local or remote node.
1890
2057
  Parameters provided in option `transfer_info` are:
1891
2058
 
1892
- <table>
1893
- <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1894
- <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
1895
- <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
1896
- <tr><td>password</td><td>string</td><td>password, secret or bearer token</br>Mandatory</td></tr>
1897
- <tr><td>root_id</td><td>string</td><td>password or secret</br>Mandatory only for bearer token</td></tr>
1898
- </table>
2059
+ | Name | Type | Description |
2060
+ |------|------|-------------|
2061
+ | url | string | URL of the node API</br>Mandatory |
2062
+ | username | string | node api user or access key</br>Mandatory |
2063
+ | password | string | password, secret or bearer token</br>Mandatory |
2064
+ | root_id | string | password or secret</br>Mandatory only for bearer token |
1899
2065
 
1900
2066
  Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
1901
2067
  `--transfer-info=@preset:<psetname>` or be specified using the extended value syntax :
@@ -1907,7 +2073,15 @@ If the `password` value begins with `Bearer` then the `username` is expected to
1907
2073
 
1908
2074
  #### <a id="agt_httpgw"></a>HTTP Gateway
1909
2075
 
1910
- If it possible to send using a HTTP gateway, in case FASP is not allowed. `transfer_info` shall have a single mandatory parameter: `url`.
2076
+ If it possible to send using a HTTP gateway, in case FASP is not allowed.
2077
+
2078
+ Parameters provided in option `transfer_info` are:
2079
+
2080
+ | Name | Type | Description |
2081
+ |------|------|-------------|
2082
+ | url | string | URL of the HTTP GW</br>Mandatory |
2083
+ | upload_bar_refresh_sec | float | Refresh rate for upload progress bar |
2084
+ | upload_chunksize | int | Size in bytes of chunks for upload |
1911
2085
 
1912
2086
  Example:
1913
2087
 
@@ -1915,7 +2089,7 @@ Example:
1915
2089
  ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
1916
2090
  ```
1917
2091
 
1918
- Note that the gateway only supports transfers authorized with a token.
2092
+ > **Note:** The gateway only supports transfers authorized with a token.
1919
2093
 
1920
2094
  #### <a id="agt_trsdk"></a>Transfer SDK
1921
2095
 
@@ -1923,7 +2097,8 @@ Another possibility is to use the Transfer SDK daemon (asperatransferd).
1923
2097
 
1924
2098
  By default it will listen on local port `55002` on `127.0.0.1`.
1925
2099
 
1926
- The gem `grpc` was removed from dependencies, as it requires compilation of a native part. So, to use the Transfer SDK you should install this gem:
2100
+ The gem `grpc` was removed from dependencies, as it requires compilation of a native part.
2101
+ So, to use the Transfer SDK you should install this gem:
1927
2102
 
1928
2103
  ```bash
1929
2104
  gem install grpc
@@ -1931,9 +2106,9 @@ gem install grpc
1931
2106
 
1932
2107
  On Windows the compilation may fail for various reasons (3.1.1):
1933
2108
 
1934
- * `cannot find -lx64-ucrt-ruby310`
2109
+ - `cannot find -lx64-ucrt-ruby310`
1935
2110
  &rarr; copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the dll extension)
1936
- * `conflicting types for 'gettimeofday'`
2111
+ - `conflicting types for 'gettimeofday'`
1937
2112
  &rarr; edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)` ,i.e. change `struct timezone` to `void`
1938
2113
 
1939
2114
  ### <a id="transferspec"></a>Transfer Specification
@@ -1941,22 +2116,22 @@ On Windows the compilation may fail for various reasons (3.1.1):
1941
2116
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
1942
2117
  is described in a [*transfer-spec*](#transferspec) (Transfer Specification), such as:
1943
2118
 
1944
- * server address
1945
- * transfer user name
1946
- * credentials
1947
- * file list
1948
- * etc...
2119
+ - server address
2120
+ - transfer user name
2121
+ - credentials
2122
+ - file list
2123
+ - etc...
1949
2124
 
1950
2125
  `ascli` builds a default [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
1951
2126
 
1952
2127
  If needed, it is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters. Multiple `ts` options on command line are cumulative.
1953
2128
 
1954
- It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
2129
+ It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for `ascp` command line parameters not supported yet in the transfer spec.
1955
2130
 
1956
2131
  The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has the advantage of:
1957
2132
 
1958
- * common to all [Transfer Agent](#agents)
1959
- * not dependent on command line limitations (special characters...)
2133
+ - common to all [Transfer Agent](#agents)
2134
+ - not dependent on command line limitations (special characters...)
1960
2135
 
1961
2136
  A [*transfer-spec*](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
1962
2137
 
@@ -1967,9 +2142,9 @@ All standard [*transfer-spec*](#transferspec) parameters can be specified.
1967
2142
 
1968
2143
  References:
1969
2144
 
1970
- * [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22)&rarr;/opt/transfers
1971
- * [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22)&rarr;Guides&rarr;API Ref&rarr;Transfer Spec V1
1972
- * [Aspera Connect SDK](https://d3gcli72yxqn2z.cloudfront.net/connect/v4/asperaweb-4.js) &rarr; search `The parameters for starting a transfer.`
2145
+ - [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22) &rarr; /opt/transfers
2146
+ - [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22) &rarr; Guides &rarr; API Ref &rarr; Transfer Spec V1
2147
+ - [Aspera Connect SDK](https://d3gcli72yxqn2z.cloudfront.net/connect/v4/asperaweb-4.js) &rarr; search `The parameters for starting a transfer.`
1973
2148
 
1974
2149
  Parameters can be displayed with commands:
1975
2150
 
@@ -1980,106 +2155,106 @@ ascli config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
1980
2155
 
1981
2156
  Columns:
1982
2157
 
1983
- * D=Direct (local `ascp` execution)
1984
- * N=Node API
1985
- * C=Connect Client
2158
+ - D=Direct (local `ascp` execution)
2159
+ - N=Node API
2160
+ - C=Connect Client
1986
2161
 
1987
2162
  `ascp` argument or environment variable is provided in description.
1988
2163
 
1989
2164
  Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
1990
2165
 
1991
- <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr>
1992
- <tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr>
1993
- <tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
1994
- <tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr>
1995
- <tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr>
1996
- <tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr>
1997
- <tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr>
1998
- <tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr>
1999
- <tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr>
2000
- <tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr>
2001
- <tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr>
2002
- <tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr>
2003
- <tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr>
2004
- <tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr>
2005
- <tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c)</td></tr>
2006
- <tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr>
2007
- <tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt)</td></tr>
2008
- <tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
2009
- <tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr>
2010
- <tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr>
2011
- <tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer)</td></tr>
2012
- <tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2013
- <tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr>
2014
- <tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr>
2015
- <tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr>
2016
- <tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr>
2017
- <tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr>
2018
- <tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr>
2019
- <tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr>
2020
- <tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y)</td></tr>
2021
- <tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port when no cipher is used<br/>(-t)</td></tr>
2022
- <tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t)</td></tr>
2023
- <tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2024
- <tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2025
- <tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2026
- <tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2027
- <tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2028
- <tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2029
- <tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr>
2030
- <tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The relative path to which the files will be moved after the transfer at the source side.<br/>(--move-after-transfer)</td></tr>
2031
- <tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/></td></tr>
2032
- <tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold)</td></tr>
2033
- <tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr>
2034
- <tr><td>password</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.</td></tr>
2035
- <tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr>
2036
- <tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr>
2037
- <tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr>
2038
- <tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls)</td></tr>
2039
- <tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr>
2040
- <tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr>
2041
- <tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr>
2042
- <tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr>
2043
- <tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr>
2044
- <tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr>
2045
- <tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr>
2046
- <tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr>
2047
- <tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr>
2048
- <tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr>
2049
- <tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr>
2050
- <tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr>
2051
- <tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr>
2052
- <tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2053
- <tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr>
2054
- <tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr>
2055
- <tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr>
2056
- <tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr>
2057
- <tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr>
2058
- <tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr>
2059
- <tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr>
2060
- <tr><td>src_base</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64)</td></tr>
2061
- <tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr>
2062
- <tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr>
2063
- <tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr>
2064
- <tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr>
2065
- <tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr>
2066
- <tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64)</td></tr>
2067
- <tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr>
2068
- <tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr>
2069
- <tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2070
- <tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr>
2071
- <tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr>
2072
- <tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr>
2073
- <tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2074
- <tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr>
2075
- </table>
2166
+ | Field | Type | D | N | C | Description |
2167
+ |-------|------|---|---|---|-------------|
2168
+ | EX_ascp_args | array | Y | &nbsp; | &nbsp; | Add native command line arguments to ascp |
2169
+ | EX_at_rest_password | string | Y | &nbsp; | &nbsp; | DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS) |
2170
+ | EX_file_list | string | Y | &nbsp; | &nbsp; | source file list |
2171
+ | EX_file_pair_list | string | Y | &nbsp; | &nbsp; | source file pair list |
2172
+ | EX_http_proxy_url | string | Y | &nbsp; | &nbsp; | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
2173
+ | EX_http_transfer_jpeg | int | Y | &nbsp; | &nbsp; | HTTP transfers as JPEG file<br/>(-j {int}) |
2174
+ | EX_license_text | string | Y | &nbsp; | &nbsp; | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
2175
+ | EX_no_read | bool | Y | &nbsp; | &nbsp; | no read source<br/>(--no-read) |
2176
+ | EX_no_write | bool | Y | &nbsp; | &nbsp; | no write on destination<br/>(--no-write) |
2177
+ | EX_proxy_password | string | Y | &nbsp; | &nbsp; | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS) |
2178
+ | EX_ssh_key_paths | array | Y | &nbsp; | &nbsp; | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
2179
+ | apply_local_docroot | bool | Y | &nbsp; | &nbsp; | (--apply-local-docroot) |
2180
+ | authentication | string | &nbsp; | &nbsp; | Y | value=token for SSH bypass keys, else password asked if not provided. |
2181
+ | cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
2182
+ | cipher_allowed | string | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none". |
2183
+ | content_protection | string | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
2184
+ | content_protection_password | string | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
2185
+ | cookie | string | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
2186
+ | create_dir | bool | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
2187
+ | delete_before_transfer | bool | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
2188
+ | delete_source | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2189
+ | destination_root | string | Y | Y | Y | Destination root directory. |
2190
+ | dgram_size | int | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
2191
+ | direction | string | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
2192
+ | exclude_newer_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
2193
+ | exclude_older_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
2194
+ | fasp_port | int | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
2195
+ | file_checksum | string | Y | Y | &nbsp; | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none |
2196
+ | http_fallback | bool<br/>string | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
2197
+ | http_fallback_port | int | Y | &nbsp; | &nbsp; | Specifies http port when no cipher is used<br/>(-t {int}) |
2198
+ | https_fallback_port | int | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
2199
+ | lock_min_rate | bool | Y | Y | Y | &nbsp; |
2200
+ | lock_min_rate_kbps | bool | Y | Y | Y | &nbsp; |
2201
+ | lock_rate_policy | bool | Y | Y | Y | &nbsp; |
2202
+ | lock_target_rate | bool | Y | Y | Y | &nbsp; |
2203
+ | lock_target_rate_kbps | bool | Y | Y | Y | &nbsp; |
2204
+ | min_rate_cap_kbps | int | Y | Y | Y | &nbsp; |
2205
+ | min_rate_kbps | int | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
2206
+ | move_after_transfer | string | Y | Y | &nbsp; | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
2207
+ | multi_session | int | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/> |
2208
+ | multi_session_threshold | int | Y | Y | &nbsp; | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
2209
+ | overwrite | string | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
2210
+ | password | string | &nbsp; | Y | &nbsp; | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only. |
2211
+ | paths | array | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional). |
2212
+ | precalculate_job_size | bool | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
2213
+ | preserve_access_time | bool | Y | Y | Y | (--preserve-access-time) |
2214
+ | preserve_acls | string | Y | &nbsp; | &nbsp; | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
2215
+ | preserve_creation_time | bool | Y | Y | Y | (--preserve-creation-time) |
2216
+ | preserve_file_owner_gid | bool | Y | &nbsp; | &nbsp; | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
2217
+ | preserve_file_owner_uid | bool | Y | &nbsp; | &nbsp; | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
2218
+ | preserve_modification_time | bool | Y | Y | Y | (--preserve-modification-time) |
2219
+ | preserve_remote_acls | string | Y | &nbsp; | &nbsp; | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum}) |
2220
+ | preserve_source_access_time | bool | Y | &nbsp; | &nbsp; | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
2221
+ | preserve_times | bool | Y | Y | Y | (--preserve-times) |
2222
+ | proxy | string | Y | &nbsp; | &nbsp; | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
2223
+ | rate_policy | string | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
2224
+ | rate_policy_allowed | string | &nbsp; | &nbsp; | Y | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed |
2225
+ | remote_host | string | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
2226
+ | remote_password | string | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
2227
+ | remote_user | string | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
2228
+ | remove_after_transfer | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2229
+ | remove_empty_directories | bool | Y | Y | &nbsp; | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
2230
+ | remove_empty_source_directory | bool | Y | &nbsp; | &nbsp; | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
2231
+ | remove_skipped | bool | Y | Y | Y | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
2232
+ | resume_policy | string | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
2233
+ | retry_duration | string<br/>int | &nbsp; | Y | Y | Specifies how long to wait before retrying transfer. (e.g. "5min") |
2234
+ | source_root | string | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
2235
+ | source_root_id | string | &nbsp; | Y | &nbsp; | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
2236
+ | src_base | string | Y | Y | &nbsp; | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
2237
+ | ssh_port | int | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
2238
+ | ssh_private_key | string | Y | &nbsp; | &nbsp; | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----&sol;nMII...<br/>Note the JSON encoding: &sol;n for newlines.<br/>(env:ASPERA_SCP_KEY) |
2239
+ | ssh_private_key_passphrase | string | Y | &nbsp; | &nbsp; | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
2240
+ | sshfp | string | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
2241
+ | symlink_policy | string | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
2242
+ | tags | hash | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
2243
+ | target_rate_cap_kbps | int | &nbsp; | &nbsp; | Y | Returned by upload/download_setup node API. |
2244
+ | target_rate_kbps | int | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
2245
+ | target_rate_percentage | string | Y | Y | Y | &nbsp; |
2246
+ | title | string | &nbsp; | Y | Y | Title of the transfer |
2247
+ | token | string | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
2248
+ | use_ascp4 | bool | Y | Y | &nbsp; | specify version of protocol |
2249
+ | wss_enabled | bool | Y | Y | Y | &nbsp; |
2250
+ | wss_port | int | Y | Y | Y | TCP port used for websocket service feed. |
2076
2251
 
2077
2252
  #### Destination folder for transfers
2078
2253
 
2079
2254
  The destination folder is set by `ascli` by default to:
2080
2255
 
2081
- * `.` for downloads
2082
- * `/` for uploads
2256
+ - `.` for downloads
2257
+ - `/` for uploads
2083
2258
 
2084
2259
  It is specified by the [*transfer-spec*](#transferspec) parameter `destination_root`.
2085
2260
  As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
@@ -2101,7 +2276,7 @@ The `sources` and `src_type` options provide convenient ways to populate the tra
2101
2276
 
2102
2277
  Possible values for option `sources` are:
2103
2278
 
2104
- * `@args` : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with `-`) are considered as source files.
2279
+ - `@args` : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with `-`) are considered as source files.
2105
2280
  So, by default, the list of files to transfer will be simply specified on the command line. Example:
2106
2281
 
2107
2282
  ```bash
@@ -2114,13 +2289,13 @@ So, by default, the list of files to transfer will be simply specified on the co
2114
2289
  ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile
2115
2290
  ```
2116
2291
 
2117
- * an [Extended Value](#extended) with type **Array of String**
2292
+ - an [Extended Value](#extended) with type **Array of String**
2118
2293
 
2119
- > Note: extended values can be tested with the command `conf echo`
2294
+ > **Note:** extended values can be tested with the command `conf echo`
2120
2295
 
2121
2296
  Examples:
2122
2297
 
2123
- * Using extended value
2298
+ - Using extended value
2124
2299
 
2125
2300
  Create the file list:
2126
2301
 
@@ -2135,28 +2310,28 @@ So, by default, the list of files to transfer will be simply specified on the co
2135
2310
  --sources=@lines:@file:myfilelist.txt
2136
2311
  ```
2137
2312
 
2138
- * Using JSON array
2313
+ - Using JSON array
2139
2314
 
2140
2315
  ```javascript
2141
2316
  --sources=@json:'["file1","file2"]'
2142
2317
  ```
2143
2318
 
2144
- * Using STDIN, one path per line
2319
+ - Using STDIN, one path per line
2145
2320
 
2146
2321
  ```bash
2147
2322
  --sources=@lines:@stdin:
2148
2323
  ```
2149
2324
 
2150
- * Using ruby code (one path per line in file)
2325
+ - Using ruby code (one path per line in file)
2151
2326
 
2152
2327
  ```ruby
2153
2328
  --sources=@ruby:'File.read("myfilelist.txt").split("\n")'
2154
2329
  ```
2155
2330
 
2156
- * `@ts` : the user provides the list of files directly in the `paths` field of transfer spec (option `ts`).
2331
+ - `@ts` : the user provides the list of files directly in the `paths` field of transfer spec (option `ts`).
2157
2332
  Examples:
2158
2333
 
2159
- * Using transfer spec
2334
+ - Using transfer spec
2160
2335
 
2161
2336
  ```javascript
2162
2337
  --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
@@ -2164,12 +2339,12 @@ Examples:
2164
2339
 
2165
2340
  The option `src_type` allows specifying if the list specified in option `sources` is a simple file list or if it is a file pair list.
2166
2341
 
2167
- > Note: Option `src_type` is not used if option `sources` is set to `@ts`
2342
+ > **Note:** Option `src_type` is not used if option `sources` is set to `@ts`
2168
2343
 
2169
2344
  Supported values for `src_type` are:
2170
2345
 
2171
- * `list` : (default) the path of destination is the same as source and each entry is a source file path
2172
- * `pair` : the first element is the first source, the second element is the first destination, and so on.
2346
+ - `list` : (default) the path of destination is the same as source and each entry is a source file path
2347
+ - `pair` : the first element is the first source, the second element is the first destination, and so on.
2173
2348
 
2174
2349
  Example: Source file `200KB.1` is renamed `sample1` on destination:
2175
2350
 
@@ -2177,13 +2352,13 @@ Example: Source file `200KB.1` is renamed `sample1` on destination:
2177
2352
  ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
2178
2353
  ```
2179
2354
 
2180
- > Note there are some specific rules to specify file list when using "Aspera on Cloud", refer to the AoC plugin section.
2355
+ > **Note:** There are some specific rules to specify a file list when using **Aspera on Cloud**, refer to the AoC plugin section.
2181
2356
 
2182
2357
  #### <a id="multisession"></a>Support of multi-session
2183
2358
 
2184
- Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
2359
+ Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on "direct" and "node" agents, not yet on connect.
2185
2360
 
2186
- * when agent=node :
2361
+ - when agent=node :
2187
2362
 
2188
2363
  ```javascript
2189
2364
  --ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
@@ -2191,16 +2366,15 @@ Multi session, i.e. starting a transfer of a file set using multiple sessions (o
2191
2366
 
2192
2367
  Multi-session is directly supported by the node daemon.
2193
2368
 
2194
- * when agent=direct :
2369
+ - when agent=direct :
2195
2370
 
2196
2371
  ```javascript
2197
2372
  --ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
2198
2373
  ```
2199
2374
 
2200
- Note: resume policy of "attr" may cause problems. "none" or "sparse_csum"
2201
- shall be preferred.
2375
+ Note: `resume_policy` set to `attr` may cause problems: `none` or `sparse_csum` shall be preferred.
2202
2376
 
2203
- Multi-session spawn is done by `ascli`.
2377
+ `ascli` starts multiple `ascp` for Multi-session using `direct` agent.
2204
2378
 
2205
2379
  When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
2206
2380
 
@@ -2212,8 +2386,8 @@ using a passphrase, only known by users sharing files. Files stay encrypted on s
2212
2386
 
2213
2387
  activating CSEAR consists in using transfer spec parameters:
2214
2388
 
2215
- * `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
2216
- * `content_protection_password` : the passphrase to be used.
2389
+ - `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
2390
+ - `content_protection_password` : the passphrase to be used.
2217
2391
 
2218
2392
  Example: parameter to download a faspex package and decrypt on the fly
2219
2393
 
@@ -2221,7 +2395,7 @@ Example: parameter to download a faspex package and decrypt on the fly
2221
2395
  --ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
2222
2396
  ```
2223
2397
 
2224
- Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
2398
+ > **Note:** Up to version `ascli` 4.6.0, the following parameters should be used for agent `direct`:
2225
2399
 
2226
2400
  ```javascript
2227
2401
  --ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
@@ -2229,44 +2403,48 @@ Note that up to version 4.6.0, the following parameters should be used for agent
2229
2403
 
2230
2404
  #### Transfer Spec Examples
2231
2405
 
2232
- * Change target rate
2406
+ - Change target rate
2233
2407
 
2234
2408
  ```javascript
2235
2409
  --ts=@json:'{"target_rate_kbps":500000}'
2236
2410
  ```
2237
2411
 
2238
- * Override the FASP SSH port to a specific TCP port:
2412
+ - Override the FASP SSH port to a specific TCP port:
2239
2413
 
2240
2414
  ```javascript
2241
2415
  --ts=@json:'{"ssh_port":33002}'
2242
2416
  ```
2243
2417
 
2244
- * Force http fallback mode:
2418
+ - Force http fallback mode:
2245
2419
 
2246
2420
  ```javascript
2247
2421
  --ts=@json:'{"http_fallback":"force"}'
2248
2422
  ```
2249
2423
 
2250
- * Activate progress when not activated by default on server
2424
+ - Activate progress when not activated by default on server
2251
2425
 
2252
2426
  ```javascript
2253
2427
  --ts=@json:'{"precalculate_job_size":true}'
2254
2428
  ```
2255
2429
 
2256
- ### <a id="scheduling"></a>Lock for exclusive execution
2430
+ ### <a id="scheduling"></a>Scheduling
2431
+
2432
+ It is useful to configure automated scheduled execution.
2433
+
2434
+ #### <a id="locking"></a>Locking for exclusive execution
2257
2435
 
2258
- In some conditions, it may be desirable to ensure that `ascli` is not executed several times in parallel.
2436
+ It is also useful to ensure that `ascli` is not executed several times in parallel.
2259
2437
 
2260
2438
  For instance when `ascli` is executed automatically on a schedule basis, one generally desire that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
2261
2439
 
2262
- * Executing instances may pile-up and kill the system
2263
- * The same file may be transferred by multiple instances at the same time.
2264
- * `preview` may generate the same files in multiple instances.
2440
+ - Executing instances may pile-up and kill the system
2441
+ - The same file may be transferred by multiple instances at the same time.
2442
+ - `preview` may generate the same files in multiple instances.
2265
2443
 
2266
2444
  Usually the OS native scheduler already provides some sort of protection against parallel execution:
2267
2445
 
2268
- * The Windows scheduler does this by default
2269
- * Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
2446
+ - The Windows scheduler does this by default
2447
+ - Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
2270
2448
 
2271
2449
  ```bash
2272
2450
  /usr/bin/flock -w 0 /var/cron.lock ascli ...
@@ -2289,6 +2467,18 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
2289
2467
  WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
2290
2468
  ```
2291
2469
 
2470
+ #### <a id="scheduler"></a>Scheduler
2471
+
2472
+ `ascli` does not provide an internal scheduler.
2473
+
2474
+ Instead, use the service provided by the Operating system:
2475
+
2476
+ - Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
2477
+ - Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
2478
+ - etc...
2479
+
2480
+ Linux also provides `anacron`, if tasks are hourly or daily.
2481
+
2292
2482
  ### "Proven&ccedil;ale"
2293
2483
 
2294
2484
  `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
@@ -2343,8 +2533,8 @@ faux:///filename?filesize
2343
2533
 
2344
2534
  where:
2345
2535
 
2346
- * `filename` is the name that will be assigned to the file on the destination
2347
- * `filesize` is the number of bytes that will be sent (in decimal).
2536
+ - `filename` is the name that will be assigned to the file on the destination
2537
+ - `filesize` is the number of bytes that will be sent (in decimal).
2348
2538
 
2349
2539
  Note: characters `?` and `&` are shell special characters (wildcard and backround), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
2350
2540
 
@@ -2358,8 +2548,8 @@ faux:///dirname?<arg1>=<val1>&...
2358
2548
 
2359
2549
  where:
2360
2550
 
2361
- * `dirname` is the folder name and can contain `/` to specify a subfolder.
2362
- * supported arguments are:
2551
+ - `dirname` is the folder name and can contain `/` to specify a subfolder.
2552
+ - supported arguments are:
2363
2553
 
2364
2554
  | Name | Type | Description |
2365
2555
  |--------|------|-------------|
@@ -2372,19 +2562,19 @@ where:
2372
2562
 
2373
2563
  The sequence parameter is applied as follows:
2374
2564
 
2375
- * If `seq` is `random` then each file size is:
2565
+ - If `seq` is `random` then each file size is:
2376
2566
 
2377
- * size +/- (inc * rand())
2378
- * Where rand is a random number between 0 and 1
2379
- * Note that file size must not be negative, inc will be set to size if it is greater than size
2380
- * Similarly, overall file size must be less than 8*2<sup>60</sup>. If size + inc is greater, inc will be reduced to limit size + inc to 7*2<sup>60</sup>.
2567
+ - size +/- (inc * rand())
2568
+ - Where rand is a random number between 0 and 1
2569
+ - Note that file size must not be negative, inc will be set to size if it is greater than size
2570
+ - Similarly, overall file size must be less than 8*2<sup>60</sup>. If size + inc is greater, inc will be reduced to limit size + inc to 7*2<sup>60</sup>.
2381
2571
 
2382
- * If `seq` is `sequential` then each file size is:
2572
+ - If `seq` is `sequential` then each file size is:
2383
2573
 
2384
- * `size + ((fileindex - 1) * inc)`
2385
- * Where first file is index 1
2386
- * So file1 is `size` bytes, file2 is `size + inc` bytes, file3 is `size + inc * 2` bytes, etc.
2387
- * As with `random`, `inc` will be adjusted if `size + (count * inc)` is not less then 8*2<sup>60</sup>.
2574
+ - `size + ((fileindex - 1) * inc)`
2575
+ - Where first file is index 1
2576
+ - So file1 is `size` bytes, file2 is `size + inc` bytes, file3 is `size + inc * 2` bytes, etc.
2577
+ - As with `random`, `inc` will be adjusted if `size + (count * inc)` is not less then 8*2<sup>60</sup>.
2388
2578
 
2389
2579
  Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
2390
2580
 
@@ -2392,19 +2582,19 @@ To discard data at the destination, the destination argument is set to `faux://`
2392
2582
 
2393
2583
  Examples:
2394
2584
 
2395
- * Upload 20 gibibytes of random data to file myfile to directory /Upload
2585
+ - Upload 20 gibibytes of random data to file myfile to directory /Upload
2396
2586
 
2397
2587
  ```bash
2398
2588
  ascli server upload faux:///myfile\?20g --to-folder=/Upload
2399
2589
  ```
2400
2590
 
2401
- * Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
2591
+ - Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
2402
2592
 
2403
2593
  ```bash
2404
2594
  ascli server upload /tmp/sample --to-folder=faux://
2405
2595
  ```
2406
2596
 
2407
- * Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
2597
+ - Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
2408
2598
 
2409
2599
  ```bash
2410
2600
  ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
@@ -2415,7 +2605,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
2415
2605
  ```text
2416
2606
  ascli -h
2417
2607
  NAME
2418
- ascli -- a command line tool for Aspera Applications (v4.9.0)
2608
+ ascli -- a command line tool for Aspera Applications (v4.11.0)
2419
2609
 
2420
2610
  SYNOPSIS
2421
2611
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -2438,23 +2628,23 @@ COMMANDS
2438
2628
  OPTIONS
2439
2629
  Options begin with a '-' (minus), and value is provided on command line.
2440
2630
  Special values are supported beginning with special prefix @pfx:, where pfx is one of:
2441
- base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, uri, stdin, preset
2631
+ base64, json, zlib, ruby, csvt, lines, list, incps, vault, val, file, path, env, uri, stdin, preset
2442
2632
  Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
2443
2633
 
2444
2634
  ARGS
2445
2635
  Some commands require mandatory arguments, e.g. a path.
2446
2636
 
2447
2637
  OPTIONS: global
2448
- --interactive=ENUM use interactive input of missing params: yes, [no]
2449
- --ask-options=ENUM ask even optional options: yes, [no]
2638
+ --interactive=ENUM use interactive input of missing params: [no], yes
2639
+ --ask-options=ENUM ask even optional options: [no], yes
2450
2640
  --format=ENUM output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
2451
2641
  --display=ENUM output only some information: [info], data, error
2452
2642
  --fields=VALUE comma separated list of fields, or ALL, or DEF
2453
2643
  --select=VALUE select only some items in lists, extended value: hash (column, value)
2454
2644
  --table-style=VALUE table display style
2455
- --flat-hash=ENUM display hash values as additional keys: [yes], no
2456
- --transpose-single=ENUM single object fields output vertically: [yes], no
2457
- --show-secrets=ENUM show secrets on command output: yes, [no]
2645
+ --flat-hash=ENUM display hash values as additional keys: no, [yes]
2646
+ --transpose-single=ENUM single object fields output vertically: no, [yes]
2647
+ --show-secrets=ENUM show secrets on command output: [no], yes
2458
2648
  -h, --help Show this message.
2459
2649
  --bash-comp generate bash completion for command
2460
2650
  --show-config Display parameters used for the provided action.
@@ -2465,52 +2655,55 @@ OPTIONS: global
2465
2655
  --log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
2466
2656
  --logger=ENUM log method: [stderr], stdout, syslog
2467
2657
  --lock-port=VALUE prevent dual execution of a command, e.g. in cron
2468
- --query=VALUE additional filter for API calls (extended value) (some commands)
2469
2658
  --http-options=VALUE options for http socket (extended value)
2470
- --insecure=ENUM do not validate HTTPS certificate: [yes], no
2471
- --once-only=ENUM process only new items (some commands): yes, [no]
2472
- --log-secrets=ENUM show passwords in logs: yes, [no]
2473
- --cache-tokens=ENUM save and reuse Oauth tokens: [yes], no
2659
+ --insecure=ENUM do not validate HTTPS certificate: no, [yes]
2660
+ --once-only=ENUM process only new items (some commands): [no], yes
2661
+ --log-secrets=ENUM show passwords in logs: [no], yes
2662
+ --cache-tokens=ENUM save and reuse Oauth tokens: no, [yes]
2474
2663
 
2475
2664
  COMMAND: config
2476
- SUBCOMMANDS: list overview id preset open documentation genkey gem plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
2665
+ SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test export_to_cli file flush_tokens folder gem genkey id initdemo list lookup open overview plugin preset proxy_check secure smtp_settings vault wizard
2477
2666
  OPTIONS:
2667
+ --query=VALUE additional filter for API calls (extended value) (some commands)
2478
2668
  --value=VALUE extended value for create, update, list filter
2479
2669
  --property=VALUE name of property to set
2480
2670
  --id=VALUE resource identifier (modify,delete,show)
2481
- --bulk=ENUM Bulk operation (only some): yes, [no]
2671
+ --bulk=ENUM Bulk operation (only some): [no], yes
2672
+ --bfail=ENUM Bulk operation error handling: [no], yes
2482
2673
  --config-file=VALUE read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
2483
2674
  -N, --no-default do not load default configuration for plugin
2484
- --override=ENUM Wizard: override existing value: yes, [no]
2485
- --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, [no]
2486
- --default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, [no]
2487
- --test-mode=ENUM Wizard: skip private key check step: yes, [no]
2675
+ --override=ENUM Wizard: override existing value: [no], yes
2676
+ --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: [no], yes
2677
+ --default=ENUM Wizard: set as default configuration for specified plugin (also: update): [no], yes
2678
+ --test-mode=ENUM Wizard: skip private key check step: [no], yes
2488
2679
  -P, --presetVALUE load the named option preset from current config file
2489
2680
  --pkeypath=VALUE Wizard: path to private key for JWT
2490
- --ascp-path=VALUE path to ascp
2491
- --use-product=VALUE use ascp from specified product
2492
- --smtp=VALUE smtp configuration (extended value: hash)
2493
- --fpac=VALUE proxy auto configuration script
2494
- --secret=VALUE default secret
2495
- --secrets=VALUE secret vault
2681
+ --ascp-path=VALUE Path to ascp
2682
+ --use-product=VALUE Use ascp from specified product
2683
+ --smtp=VALUE SMTP configuration (extended value: hash)
2684
+ --fpac=VALUE Proxy auto configuration script
2685
+ --proxy-credentials=VALUE HTTP proxy credentials (Array with user and password)
2686
+ --secret=VALUE Secret for access keys
2687
+ --vault=VALUE Vault for secrets
2688
+ --vault-password=VALUE Vault password
2496
2689
  --sdk-url=VALUE URL to get SDK
2497
2690
  --sdk-folder=VALUE SDK folder path
2498
- --notif-to=VALUE email recipient for notification of transfers
2499
- --notif-template=VALUE email ERB template for notification of transfers
2500
- --version-check-days=VALUE period in days to check new version (zero to disable)
2501
- --plugin-folder=VALUE folder where to find additional plugins
2502
- --ts=VALUE override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
2503
- --local-resume=VALUE set resume policy (Hash, use @json: prefix), current=
2504
- --to-folder=VALUE destination folder for transfered files
2505
- --sources=VALUE how list of transfered files is provided (@args,@ts,Array)
2506
- --src-type=ENUM type of file list: list, pair
2507
- --transfer=ENUM type of transfer agent: direct, node, connect, httpgw, trsdk
2508
- --transfer-info=VALUE parameters for transfer agent
2509
- --progress=ENUM type of progress bar: none, native, multi
2691
+ --notif-to=VALUE Email recipient for notification of transfers
2692
+ --notif-template=VALUE Email ERB template for notification of transfers
2693
+ --version-check-days=VALUE Period in days to check new version (zero to disable)
2694
+ --plugin-folder=VALUE Folder where to find additional plugins
2695
+ --ts=VALUE Override transfer spec values (Hash, e.g. use @json: prefix), current={"create_dir"=>true}
2696
+ --to-folder=VALUE Destination folder for transfered files
2697
+ --sources=VALUE How list of transfered files is provided (@args,@ts,Array)
2698
+ --ascp-opts=VALUE Options for ascp in its native format
2699
+ --src-type=ENUM Type of file list: list, pair
2700
+ --transfer=ENUM Type of transfer agent: direct, node, connect, httpgw, trsdk
2701
+ --transfer-info=VALUE Parameters for transfer agent
2702
+ --progress=ENUM Type of progress bar: none, native, multi
2510
2703
 
2511
2704
 
2512
2705
  COMMAND: shares
2513
- SUBCOMMANDS: health repository admin
2706
+ SUBCOMMANDS: admin health repository
2514
2707
  OPTIONS:
2515
2708
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2516
2709
  --username=VALUE username to log in
@@ -2518,7 +2711,7 @@ OPTIONS:
2518
2711
 
2519
2712
 
2520
2713
  COMMAND: node
2521
- SUBCOMMANDS: postprocess stream transfer cleanup forward access_key watch_folder service async central asperabrowser basic_token browse upload download api_details health events space info license mkdir mklink mkfile rename delete search
2714
+ SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
2522
2715
  OPTIONS:
2523
2716
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2524
2717
  --username=VALUE username to log in
@@ -2526,18 +2719,20 @@ OPTIONS:
2526
2719
  --validator=VALUE identifier of validator (optional for central)
2527
2720
  --asperabrowserurl=VALUE URL for simple aspera web ui
2528
2721
  --sync-name=VALUE sync name
2722
+ --path=VALUE file or folder path for gen4 operation "file"
2529
2723
  --token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
2724
+ --default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
2530
2725
 
2531
2726
 
2532
2727
  COMMAND: orchestrator
2533
- SUBCOMMANDS: health info workflow plugins processes
2728
+ SUBCOMMANDS: health info plugins processes workflow
2534
2729
  OPTIONS:
2535
2730
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2536
2731
  --username=VALUE username to log in
2537
2732
  --password=VALUE user's password
2538
2733
  --params=VALUE parameters hash table, use @json:{"param":"value"}
2539
2734
  --result=VALUE specify result value as: 'work step:parameter'
2540
- --synchronous=ENUM work step:parameter expected as result: yes, [no]
2735
+ --synchronous=ENUM work step:parameter expected as result: [no], yes
2541
2736
  --ret-style=ENUM how return type is requested in api: header, arg, ext
2542
2737
  --auth-style=ENUM authentication type: arg_pass, head_basic, apikey
2543
2738
 
@@ -2559,7 +2754,7 @@ OPTIONS:
2559
2754
 
2560
2755
 
2561
2756
  COMMAND: ats
2562
- SUBCOMMANDS: cluster access_key api_key aws_trust_policy
2757
+ SUBCOMMANDS: access_key api_key aws_trust_policy cluster
2563
2758
  OPTIONS:
2564
2759
  --ibm-api-key=VALUE IBM API key, see https://cloud.ibm.com/iam/apikeys
2565
2760
  --instance=VALUE ATS instance in ibm cloud
@@ -2571,7 +2766,7 @@ OPTIONS:
2571
2766
 
2572
2767
 
2573
2768
  COMMAND: faspex5
2574
- SUBCOMMANDS: health package admin user
2769
+ SUBCOMMANDS: admin bearer_token health package user version
2575
2770
  OPTIONS:
2576
2771
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2577
2772
  --username=VALUE username to log in
@@ -2579,7 +2774,7 @@ OPTIONS:
2579
2774
  --client-id=VALUE OAuth client identifier
2580
2775
  --client-secret=VALUE OAuth client secret
2581
2776
  --redirect-uri=VALUE OAuth redirect URI for web authentication
2582
- --auth=ENUM OAuth type of authentication: web, jwt, boot
2777
+ --auth=ENUM OAuth type of authentication: boot, web, jwt
2583
2778
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2584
2779
  --passphrase=VALUE RSA private key passphrase
2585
2780
 
@@ -2597,7 +2792,7 @@ OPTIONS:
2597
2792
 
2598
2793
 
2599
2794
  COMMAND: faspex
2600
- SUBCOMMANDS: health package source me dropbox v4 address_book login_methods
2795
+ SUBCOMMANDS: address_book dropbox health login_methods me package source v4
2601
2796
  OPTIONS:
2602
2797
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2603
2798
  --username=VALUE username to log in
@@ -2611,7 +2806,7 @@ OPTIONS:
2611
2806
 
2612
2807
 
2613
2808
  COMMAND: preview
2614
- SUBCOMMANDS: scan events trevents check test
2809
+ SUBCOMMANDS: check events scan test trevents
2615
2810
  OPTIONS:
2616
2811
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2617
2812
  --username=VALUE username to log in
@@ -2625,7 +2820,7 @@ OPTIONS:
2625
2820
  --case=VALUE basename of output for for test
2626
2821
  --scan-path=VALUE subpath in folder id to start scan in (default=/)
2627
2822
  --scan-id=VALUE forder id in storage to start scan in, default is access key main folder id
2628
- --mimemagic=ENUM use Mime type detection of gem mimemagic: yes, [no]
2823
+ --mimemagic=ENUM use Mime type detection of gem mimemagic: [no], yes
2629
2824
  --overwrite=ENUM when to overwrite result file: always, never, [mtime]
2630
2825
  --file-access=ENUM how to read and write files in repository: [local], remote
2631
2826
  --max-size=VALUE maximum size (in bytes) of preview file
@@ -2646,14 +2841,14 @@ OPTIONS:
2646
2841
 
2647
2842
 
2648
2843
  COMMAND: sync
2649
- SUBCOMMANDS: start admin
2844
+ SUBCOMMANDS: admin start
2650
2845
  OPTIONS:
2651
- --parameters=VALUE extended value for session set definition
2652
- --session-name=VALUE name of session to use for admin commands, by default first one
2846
+ --sync-info=VALUE Information for sync instance and sessions (Hash)
2847
+ --sync-session=VALUE Name of session to use for admin commands. default: first in parameters
2653
2848
 
2654
2849
 
2655
2850
  COMMAND: aoc
2656
- SUBCOMMANDS: reminder servers bearer_token organization tier_restrictions user packages files admin automation gateway
2851
+ SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
2657
2852
  OPTIONS:
2658
2853
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2659
2854
  --username=VALUE username to log in
@@ -2664,20 +2859,28 @@ OPTIONS:
2664
2859
  --client-secret=VALUE OAuth API client passcode
2665
2860
  --redirect-uri=VALUE OAuth API client redirect URI
2666
2861
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2667
- --passphrase=VALUE RSA private key passphrase
2668
- --workspace=VALUE name of workspace
2669
- --name=VALUE resource name
2670
- --path=VALUE file or folder path
2671
- --link=VALUE public link to shared resource
2672
- --new-user-option=VALUE new user creation option for unknown package recipients
2673
- --from-folder=VALUE share to share source folder
2674
2862
  --scope=VALUE OAuth scope for AoC API calls
2675
- --default-ports=ENUM use standard FASP ports or get from node api: yes, [no]
2676
- --validate-metadata=ENUM validate shared inbox metadata: yes, [no]
2863
+ --passphrase=VALUE RSA private key passphrase
2864
+ --workspace=VALUE Name of workspace
2865
+ --name=VALUE Resource name (prefer to use keyword name)
2866
+ --link=VALUE Public link to shared resource
2867
+ --new-user-option=VALUE New user creation option for unknown package recipients
2868
+ --from-folder=VALUE Source folder for Folder-to-Folder transfer
2869
+ --validate-metadata=ENUM Validate shared inbox metadata: [no], yes
2870
+
2871
+ COMMAND: node
2872
+ SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
2873
+ OPTIONS:
2874
+ --validator=VALUE identifier of validator (optional for central)
2875
+ --asperabrowserurl=VALUE URL for simple aspera web ui
2876
+ --sync-name=VALUE sync name
2877
+ --path=VALUE file or folder path for gen4 operation "file"
2878
+ --token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
2879
+ --default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
2677
2880
 
2678
2881
 
2679
2882
  COMMAND: server
2680
- SUBCOMMANDS: health download upload browse delete rename ls rm mv du info mkdir cp df md5sum
2883
+ SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
2681
2884
  OPTIONS:
2682
2885
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2683
2886
  --username=VALUE username to log in
@@ -2687,7 +2890,7 @@ OPTIONS:
2687
2890
 
2688
2891
 
2689
2892
  COMMAND: console
2690
- SUBCOMMANDS: transfer health
2893
+ SUBCOMMANDS: health transfer
2691
2894
  OPTIONS:
2692
2895
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2693
2896
  --username=VALUE username to log in
@@ -2698,7 +2901,7 @@ OPTIONS:
2698
2901
 
2699
2902
  ```
2700
2903
 
2701
- Note that actions and parameter values can be written in short form.
2904
+ > **Note:** commands and parameter values can be written in short form.
2702
2905
 
2703
2906
  ### Bulk creation and deletion of resources
2704
2907
 
@@ -2706,7 +2909,7 @@ Bulk creation and deletion of resources are possible using option `bulk` (yes,no
2706
2909
  In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2707
2910
  This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
2708
2911
 
2709
- ## <a id="aoc"></a>Plugin: Aspera on Cloud
2912
+ ## <a id="aoc"></a>Plugin: `aoc`: IBM Aspera on Cloud
2710
2913
 
2711
2914
  Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
2712
2915
 
@@ -2749,15 +2952,15 @@ ascli config wizard --value=aoc
2749
2952
 
2750
2953
  ### <a id="aocmanual"></a>Configuration: using manual setup
2751
2954
 
2752
- > If you used the wizard (recommended): skip this section.
2955
+ > **Note:** If you used the wizard (recommended): skip this section.
2753
2956
 
2754
2957
  #### Configuration details
2755
2958
 
2756
2959
  Several types of OAuth authentication are supported:
2757
2960
 
2758
- * JSON Web Token (JWT) : authentication is secured by a private key (recommended for CLI)
2759
- * Web based authentication : authentication is made by user using a browser
2760
- * URL Token : external users authentication with url tokens (public links)
2961
+ - JSON Web Token (JWT) : authentication is secured by a private key (recommended for CLI)
2962
+ - Web based authentication : authentication is made by user using a browser
2963
+ - URL Token : external users authentication with url tokens (public links)
2761
2964
 
2762
2965
  The authentication method is controlled by option `auth`.
2763
2966
 
@@ -2773,19 +2976,19 @@ If you use the built-in client_id and client_secret, skip this and do not set th
2773
2976
 
2774
2977
  Else you can use a specific OAuth API client_id, the first step is to declare `ascli` in Aspera on Cloud using the admin interface.
2775
2978
 
2776
- (official documentation: <https://ibmaspera.com/help/admin/organization/registering_an_api_client> ).
2979
+ ([AoC documentation: Registering an API Client](https://ibmaspera.com/help/admin/organization/registering_an_api_client) ).
2777
2980
 
2778
2981
  Let's start by a registration with web based authentication (auth=web):
2779
2982
 
2780
- * Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
2781
- * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
2782
- * Click "Create New"
2783
- * Client Name: `ascli`
2784
- * Redirect URIs: `http://localhost:12345`
2785
- * Origins: `localhost`
2786
- * uncheck "Prompt users to allow client to access"
2787
- * leave the JWT part for now
2788
- * Save
2983
+ - Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
2984
+ - Go to Apps &rarr; Admin &rarr; Organization &rarr; Integrations
2985
+ - Click "Create New"
2986
+ - Client Name: `ascli`
2987
+ - Redirect URIs: `http://localhost:12345`
2988
+ - Origins: `localhost`
2989
+ - uncheck "Prompt users to allow client to access"
2990
+ - leave the JWT part for now
2991
+ - Save
2789
2992
 
2790
2993
  Note: for web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
2791
2994
 
@@ -2826,16 +3029,16 @@ a [private/public key pair](#private_key) must be used.
2826
3029
 
2827
3030
  If you are not using the built-in client_id and secret, JWT needs to be authorized in Aspera on Cloud. This can be done in two manners:
2828
3031
 
2829
- * Graphically
3032
+ - Graphically
2830
3033
 
2831
- * Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
2832
- * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
2833
- * Click on the previously created application
2834
- * select tab : "JSON Web Token Auth"
2835
- * Modify options if necessary, for instance: activate both options in section "Settings"
2836
- * Click "Save"
3034
+ - Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
3035
+ - Go to Apps &rarr; Admin &rarr; Organization &rarr; Integrations
3036
+ - Click on the previously created application
3037
+ - select tab : "JSON Web Token Auth"
3038
+ - Modify options if necessary, for instance: activate both options in section "Settings"
3039
+ - Click "Save"
2837
3040
 
2838
- * Using command line
3041
+ - Using command line
2839
3042
 
2840
3043
  ```bash
2841
3044
  ascli aoc admin res client list
@@ -2865,11 +3068,11 @@ The public key must be assigned to your user. This can be done in two manners:
2865
3068
 
2866
3069
  Open the previously generated public key located here: `$HOME/.aspera/ascli/my_private_key.pub`
2867
3070
 
2868
- * Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
2869
- * Click on the user's icon (top right)
2870
- * Select "Account Settings"
2871
- * Paste the *Public Key* in the "Public Key" section
2872
- * Click on "Submit"
3071
+ - Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
3072
+ - Click on the user's icon (top right)
3073
+ - Select "Account Settings"
3074
+ - Paste the *Public Key* in the "Public Key" section
3075
+ - Click on "Submit"
2873
3076
 
2874
3077
  ##### Using command line
2875
3078
 
@@ -2900,9 +3103,9 @@ Note: the `aspera user info show` command can be used to verify modifications.
2900
3103
 
2901
3104
  To activate default use of JWT authentication for `ascli` using the [option preset](#lprt), do the following:
2902
3105
 
2903
- * change auth method to JWT
2904
- * provide location of private key
2905
- * provide username to login as (OAuth "subject")
3106
+ - change auth method to JWT
3107
+ - provide location of private key
3108
+ - provide username to login as (OAuth "subject")
2906
3109
 
2907
3110
  Execute:
2908
3111
 
@@ -2962,13 +3165,13 @@ The option `query` can be optionally used. It expects a Hash using [Extended Val
2962
3165
 
2963
3166
  The following parameters are supported:
2964
3167
 
2965
- * `q` : a filter on name of resource (case insensitive, matches if value is contained in name)
2966
- * `sort`: name of fields to sort results, prefix with `-` for reverse order.
2967
- * `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
2968
- * `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
2969
- * `page` : native api parameter, in general do not use (added by
2970
- * `per_page` : native api parameter, number of items par api call, in general do not use
2971
- * Other specific parameters depending on resource type.
3168
+ - `q` : a filter on name of resource (case insensitive, matches if value is contained in name)
3169
+ - `sort`: name of fields to sort results, prefix with `-` for reverse order.
3170
+ - `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
3171
+ - `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
3172
+ - `page` : native api parameter, in general do not use (added by
3173
+ - `per_page` : native api parameter, number of items par api call, in general do not use
3174
+ - Other specific parameters depending on resource type.
2972
3175
 
2973
3176
  Both `max` and `pmax` are processed internally in `ascli`, not included in actual API call and limit the number of successive pages requested to API. `ascli` will return all values using paging if not provided.
2974
3177
 
@@ -2982,19 +3185,19 @@ Other parameters depend on the type of entity (refer to AoC API).
2982
3185
 
2983
3186
  Examples:
2984
3187
 
2985
- * List users with `laurent` in name:
3188
+ - List users with `laurent` in name:
2986
3189
 
2987
3190
  ```javascript
2988
3191
  ascli aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
2989
3192
  ```
2990
3193
 
2991
- * List users who logged-in before a date:
3194
+ - List users who logged-in before a date:
2992
3195
 
2993
3196
  ```javascript
2994
3197
  ascli aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
2995
3198
  ```
2996
3199
 
2997
- * List external users and sort in reverse alphabetical order using name:
3200
+ - List external users and sort in reverse alphabetical order using name:
2998
3201
 
2999
3202
  ```javascript
3000
3203
  ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
@@ -3002,7 +3205,7 @@ ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"s
3002
3205
 
3003
3206
  Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
3004
3207
 
3005
- Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
3208
+ > **Note:** The option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
3006
3209
 
3007
3210
  #### <a id="res_select"></a>Selecting a resource
3008
3211
 
@@ -3010,10 +3213,10 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
3010
3213
 
3011
3214
  To execute an action on a specific resource, select it using one of those methods:
3012
3215
 
3013
- * *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
3014
- * give name on command line *after the action*: `aoc admin res node show name abc`
3015
- * provide option `id` : `aoc admin res node show --id=123`
3016
- * provide option `name` : `aoc admin res node show --name=abc`
3216
+ - *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
3217
+ - give name on command line *after the action*: `aoc admin res node show name abc`
3218
+ - provide option `id` : `aoc admin res node show --id=123`
3219
+ - provide option `name` : `aoc admin res node show --name=abc`
3017
3220
 
3018
3221
  #### <a id="res_create"></a>Creating a resource
3019
3222
 
@@ -3054,22 +3257,23 @@ If the command returns an error, example:
3054
3257
  | | request_id: b0f45d5b-c00a-4711-acef-72b633f8a6ea |
3055
3258
  | | api.ibmaspera.com 422 Unprocessable Entity |
3056
3259
  +----+-----------------------------------------------------------------------------------+```
3260
+ ```
3057
3261
 
3058
3262
  Well, remove the offending parameters and try again.
3059
3263
 
3060
- Note that some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
3264
+ > **Note:** Some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
3061
3265
 
3062
3266
  #### Access Key secrets
3063
3267
 
3064
- In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
3065
- It is usually provided using the `secret` option.
3268
+ In order to access some administrative actions on **nodes** (in fact, access keys), the associated secret is required.
3269
+ The secret is provided using the `secret` option.
3066
3270
  For example in a command like:
3067
3271
 
3068
3272
  ```bash
3069
3273
  ascli aoc admin res node --id=123 --secret="secret1" v3 info
3070
3274
  ```
3071
3275
 
3072
- It is also possible to provide a set of secrets used on a regular basis using the [secret vault](#vault).
3276
+ It is also possible to store secrets in the [secret vault](#vault) and then automatically find the related secret using the [config finder](#config_finder).
3073
3277
 
3074
3278
  #### Activity
3075
3279
 
@@ -3097,7 +3301,7 @@ Thank you.
3097
3301
 
3098
3302
  The environment provided contains the following additional variable:
3099
3303
 
3100
- * ev : all details on the transfer event
3304
+ - ev : all details on the transfer event
3101
3305
 
3102
3306
  Example:
3103
3307
 
@@ -3109,11 +3313,11 @@ ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 \
3109
3313
 
3110
3314
  Options:
3111
3315
 
3112
- * `once_only` keep track of last date it was called, so next call will get only new events
3113
- * `query` filter (on API call)
3114
- * `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
3316
+ - `once_only` keep track of last date it was called, so next call will get only new events
3317
+ - `query` filter (on API call)
3318
+ - `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
3115
3319
 
3116
- Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
3320
+ > **Note:** This must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
3117
3321
 
3118
3322
  #### Transfer: Using specific transfer ports
3119
3323
 
@@ -3321,7 +3525,7 @@ ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_wor
3321
3525
 
3322
3526
  #### Example: create a group, add to workspace and add user to group
3323
3527
 
3324
- * Create the group and take note of `id`
3528
+ - Create the group and take note of `id`
3325
3529
 
3326
3530
  ```bash
3327
3531
  ascli aoc admin res group create @json:'{"name":"group 1","description":"my super group"}'
@@ -3329,7 +3533,7 @@ ascli aoc admin res group create @json:'{"name":"group 1","description":"my supe
3329
3533
 
3330
3534
  Group: `11111`
3331
3535
 
3332
- * Get the workspace id
3536
+ - Get the workspace id
3333
3537
 
3334
3538
  ```bash
3335
3539
  ascli aoc admin res workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
@@ -3337,13 +3541,13 @@ ascli aoc admin res workspace list --query=@json:'{"q":"myworkspace"}' --fields=
3337
3541
 
3338
3542
  Workspace: 22222
3339
3543
 
3340
- * Add group to workspace
3544
+ - Add group to workspace
3341
3545
 
3342
3546
  ```bash
3343
3547
  ascli aoc admin res workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
3344
3548
  ```
3345
3549
 
3346
- * Get a user's id
3550
+ - Get a user's id
3347
3551
 
3348
3552
  ```bash
3349
3553
  ascli aoc admin res user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
@@ -3351,7 +3555,7 @@ ascli aoc admin res user list --query=@json:'{"q":"manu.macron@example.com"}' --
3351
3555
 
3352
3556
  User: 33333
3353
3557
 
3354
- * Add user to group
3558
+ - Add user to group
3355
3559
 
3356
3560
  ```bash
3357
3561
  ascli aoc admin res group_membership create @json:'{"group_id":11111,"member_type":"user","member_id":33333}'
@@ -3430,16 +3634,17 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
3430
3634
 
3431
3635
  AoC nodes as actually composed with two related entities:
3432
3636
 
3433
- * An access key created on the Transfer Server (HSTS/ATS)
3434
- * a `node` resource in the AoC application.
3637
+ - An access key created on the Transfer Server (HSTS/ATS)
3638
+ - a `node` resource in the AoC application.
3435
3639
 
3436
3640
  The web UI allows creation of both entities in one shot.
3437
3641
  For more flexibility, `ascli` allows this in two separate steps.
3438
- Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
3642
+
3643
+ > **Note:** When selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
3439
3644
 
3440
3645
  So, for example, the creation of a node using ATS in IBM Cloud looks like (see other example in this manual):
3441
3646
 
3442
- * Create the access key on ATS
3647
+ - Create the access key on ATS
3443
3648
 
3444
3649
  The creation options are the ones of ATS API, refer to the [section on ATS](#ats_params) for more details and examples.
3445
3650
 
@@ -3447,9 +3652,11 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
3447
3652
  ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
3448
3653
  ```
3449
3654
 
3450
- Once executed, the access key `id` and `secret`, randomly generated by the node api, is displayed: take note of it as the secrets will not be disclosed again.
3655
+ Once executed, the access key `id` and `secret`, randomly generated by the node api, is displayed.
3656
+
3657
+ > **Note:** Once returned by the API, the secret will not be available anymore, so store this preciously. ATS secrets can only be reset by asking to IBM support.
3451
3658
 
3452
- * Create the AoC node entity
3659
+ - Create the AoC node entity
3453
3660
 
3454
3661
  First, Retrieve the ATS node address
3455
3662
 
@@ -3467,22 +3674,23 @@ Creation of a node with a self-managed node is similar, but the command `aoc adm
3467
3674
 
3468
3675
  ### List of files to transfer
3469
3676
 
3470
- Source files are provided as a list with the `sources` option. Refer to section [File list](#file_list)
3677
+ Source files are provided as a list with the `sources` option.
3678
+ Refer to section [File list](#file_list)
3471
3679
 
3472
- Note the special case when the source files are located on "Aspera on Cloud" (i.e. using access keys and the `file id` API).
3680
+ > **Note:** A special case is when the source files are located on **Aspera on Cloud** (i.e. using access keys and the `file id` API).
3473
3681
 
3474
3682
  Source files are located on "Aspera on cloud", when :
3475
3683
 
3476
- * the server is Aspera on Cloud, and executing a download or recv
3477
- * the agent is Aspera on Cloud, and executing an upload or send
3684
+ - the server is Aspera on Cloud, and executing a download or recv
3685
+ - the agent is Aspera on Cloud, and executing an upload or send
3478
3686
 
3479
3687
  In this case:
3480
3688
 
3481
- * If there is a single file : specify the full path
3482
- * Else, if there are multiple files:
3483
- * All source files must be in the same source folder
3484
- * Specify the source folder as first item in the list
3485
- * followed by the list of file names.
3689
+ - If there is a single file : specify the full path
3690
+ - Else, if there are multiple files:
3691
+ - All source files must be in the same source folder
3692
+ - Specify the source folder as first item in the list
3693
+ - followed by the list of file names.
3486
3694
 
3487
3695
  ### Packages
3488
3696
 
@@ -3498,14 +3706,14 @@ ascli aoc packages send --value=[package extended value] [other parameters such
3498
3706
 
3499
3707
  Notes:
3500
3708
 
3501
- * The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
3502
- * List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
3503
- * Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
3504
- * Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
3505
- * or just names: `"recipients":[{"The Dest"}]` . ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
3506
- * If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
3507
- * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
3508
- * if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
3709
+ - The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
3710
+ - List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
3711
+ - Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
3712
+ - Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
3713
+ - or just names: `"recipients":[{"The Dest"}]` . ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
3714
+ - If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
3715
+ - if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
3716
+ - if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
3509
3717
 
3510
3718
  #### Example: Send a package with one file to two users, using their email
3511
3719
 
@@ -3584,23 +3792,32 @@ It is possible to automatically download new packages, like using Aspera Cargo:
3584
3792
  ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
3585
3793
  ```
3586
3794
 
3587
- * `--id=ALL` (case sensitive) will download all packages
3588
- * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
3589
- * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
3795
+ - `--id=ALL` (case sensitive) will download all packages
3796
+ - `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
3797
+ - `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
3590
3798
 
3591
- Typically, one would execute this command on a regular basis, using the method of your choice:
3592
-
3593
- * Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
3594
- * Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
3595
- * etc...
3799
+ Typically, one would execute this command on a regular basis, using the method of your choice: see [Scheduler](#scheduler).
3596
3800
 
3597
3801
  ### Files
3598
3802
 
3599
- Folder sharing app.
3803
+ The Files application presents a **Home** folder to users in a given workspace.
3804
+ Files located here are either user's files, or shared folders.
3600
3805
 
3601
3806
  #### Download Files
3602
3807
 
3603
- Download of files is straightforward using command: `aoc files download`
3808
+ The general download command is:
3809
+
3810
+ ```bash
3811
+ ascli aoc files download <source folder path> <source filename 1> ...
3812
+ ```
3813
+
3814
+ I.e. the first argument is the source folder, and the following arguments are the source file names in this folder.
3815
+
3816
+ If a single file or folder is to be downloaded, then a single argument can be provided.
3817
+
3818
+ ```bash
3819
+ ascli aoc files download <single file path>
3820
+ ```
3604
3821
 
3605
3822
  #### Shared folders
3606
3823
 
@@ -3610,37 +3827,37 @@ For creation, parameters are the same as for node api [permissions](https://deve
3610
3827
  Also, the pseudo key `with` is added: it will lookup the name in the contacts and fill the proper type and id.
3611
3828
  The pseudo parameter `link_name` allows changing default "shared as" name.
3612
3829
 
3613
- * List permissions on a shared folder as user
3830
+ - List permissions on a shared folder as user
3614
3831
 
3615
3832
  ```bash
3616
3833
  ascli aoc files file --path=/shared_folder_test1 perm list
3617
3834
  ```
3618
3835
 
3619
- * Share a personal folder with other users
3836
+ - Share a personal folder with other users
3620
3837
 
3621
3838
  ```bash
3622
3839
  ascli aoc files file --path=/shared_folder_test1 perm create @json:'{"with":"laurent"}'
3623
3840
  ```
3624
3841
 
3625
- * Revoke shared access
3842
+ - Revoke shared access
3626
3843
 
3627
3844
  ```bash
3628
3845
  ascli aoc files file --path=/shared_folder_test1 perm delete 6161
3629
3846
  ```
3630
3847
 
3631
- * List shared folders in node
3848
+ - List shared folders in node
3632
3849
 
3633
3850
  ```bash
3634
3851
  ascli aoc admin res node --id=8669 shared_folders
3635
3852
  ```
3636
3853
 
3637
- * List shared folders in workspace
3854
+ - List shared folders in workspace
3638
3855
 
3639
3856
  ```bash
3640
3857
  ascli aoc admin res workspace --id=10818 shared_folders
3641
3858
  ```
3642
3859
 
3643
- * List members of shared folder
3860
+ - List members of shared folder
3644
3861
 
3645
3862
  ```bash
3646
3863
  ascli aoc admin res node --id=8669 v4 perm 82 show
@@ -3654,11 +3871,11 @@ Although optional, the creation of [option preset](#lprt) is recommended to avoi
3654
3871
 
3655
3872
  Procedure to send a file from org1 to org2:
3656
3873
 
3657
- * Get access to Organization 1 and create a [option preset](#lprt): e.g. `org1`, for instance, use the [Wizard](#aocwizard)
3658
- * Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
3659
- * Get access to Organization 2 and create a [option preset](#lprt): e.g. `org2`
3660
- * Check that access works and locate the destination folder `mydestfolder`
3661
- * execute the following:
3874
+ - Get access to Organization 1 and create a [option preset](#lprt): e.g. `org1`, for instance, use the [Wizard](#aocwizard)
3875
+ - Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
3876
+ - Get access to Organization 2 and create a [option preset](#lprt): e.g. `org2`
3877
+ - Check that access works and locate the destination folder `mydestfolder`
3878
+ - execute the following:
3662
3879
 
3663
3880
  ```bash
3664
3881
  ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | ascli -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
@@ -3666,15 +3883,15 @@ ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | as
3666
3883
 
3667
3884
  Explanation:
3668
3885
 
3669
- * `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
3670
- * `files node_info /mydestfolder` generate transfer information including node api credential and root id, suitable for the next command
3671
- * `--format=json` format the output in JSON (instead of default text table)
3672
- * `--display=data` display only the result, and remove other information, such as workspace name
3673
- * `|` the standard output of the first command is fed into the second one
3674
- * `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
3675
- * `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
3676
- * `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
3677
- * `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
3886
+ - `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
3887
+ - `files node_info /mydestfolder` generate transfer information including node api credential and root id, suitable for the next command
3888
+ - `--format=json` format the output in JSON (instead of default text table)
3889
+ - `--display=data` display only the result, and remove other information, such as workspace name
3890
+ - `|` the standard output of the first command is fed into the second one
3891
+ - `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
3892
+ - `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
3893
+ - `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
3894
+ - `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
3678
3895
 
3679
3896
  #### Find Files
3680
3897
 
@@ -3682,34 +3899,34 @@ The command `aoc files find [--value=expression]` will recursively scan storage
3682
3899
 
3683
3900
  The expression can be of 3 formats:
3684
3901
 
3685
- * empty (default) : all files, equivalent to value: `exec:true`
3686
- * not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
3902
+ - empty (default) : all files, equivalent to value: `exec:true`
3903
+ - not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
3687
3904
 
3688
3905
  For instance, to find files with a special extension, use `--value='\.myext$'`
3689
3906
 
3690
- * starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
3907
+ - starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
3691
3908
 
3692
3909
  Examples of expressions: (using like this: `--value=exec:'<expression>'`)
3693
3910
 
3694
- * Find files more recent than 100 days
3911
+ - Find files more recent than 100 days
3695
3912
 
3696
3913
  ```bash
3697
3914
  f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
3698
3915
  ```
3699
3916
 
3700
- * Find files older than 1 year on a given node and store in file list
3917
+ - Find files older than 1 year on a given node and store in file list
3701
3918
 
3702
3919
  ```bash
3703
3920
  ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
3704
3921
  ```
3705
3922
 
3706
- * Delete the files, one by one
3923
+ - Delete the files, one by one
3707
3924
 
3708
3925
  ```bash
3709
3926
  cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 delete "$path" ;done
3710
3927
  ```
3711
3928
 
3712
- * Delete the files in bulk
3929
+ - Delete the files in bulk
3713
3930
 
3714
3931
  ```bash
3715
3932
  cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='_secret_here_' v3 delete @lines:@stdin:
@@ -3718,18 +3935,17 @@ cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='
3718
3935
  ### AoC sample commands
3719
3936
 
3720
3937
  ```bash
3721
- aoc -N remind --username=my_aoc_user_email
3722
- aoc -N servers
3723
3938
  aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
3724
3939
  aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3725
- aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3726
- aoc admin ats access_key delete akibmcloud
3940
+ aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak1ibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3941
+ aoc admin ats access_key delete ak1ibmcloud
3727
3942
  aoc admin ats access_key list --fields=name,id
3728
- aoc admin ats access_key node akibmcloud --secret=somesecret browse /
3943
+ aoc admin ats access_key node ak1ibmcloud --secret=somesecret browse /
3729
3944
  aoc admin ats cluster clouds
3730
3945
  aoc admin ats cluster list
3731
3946
  aoc admin ats cluster show --cloud=aws --region=eu-west-1
3732
3947
  aoc admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
3948
+ aoc admin auth_providers list
3733
3949
  aoc admin res application list
3734
3950
  aoc admin res client list
3735
3951
  aoc admin res client_access_key list
@@ -3750,14 +3966,15 @@ aoc admin res self show
3750
3966
  aoc admin res short_link list
3751
3967
  aoc admin res user list
3752
3968
  aoc admin res workspace_membership list
3753
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
3754
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 events
3755
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 browse /
3756
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 delete /folder1
3757
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 mkdir /folder1
3758
- aoc admin resource node v3 name my_aoc_ak_name --secret=my_aoc_ak_secret access_key delete testsub1
3969
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do browse /
3970
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do delete /folder1
3971
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do mkdir /folder1
3972
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
3973
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 events
3974
+ aoc admin resource node do name my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key delete testsub1
3759
3975
  aoc admin resource workspace list
3760
3976
  aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
3977
+ aoc admin subscription
3761
3978
  aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
3762
3979
  aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
3763
3980
  aoc automation workflow delete my_wf_id
@@ -3769,10 +3986,11 @@ aoc faspex
3769
3986
  aoc files bearer /
3770
3987
  aoc files bearer_token_node / --cache-tokens=no
3771
3988
  aoc files browse /
3772
- aoc files browse / -N --link=my_aoc_publink_folder
3989
+ aoc files browse / --link=my_aoc_publink_folder
3773
3990
  aoc files delete /testsrc
3774
3991
  aoc files download --transfer=connect /200KB.1
3775
- aoc files file permission --path=/ascli_test list
3992
+ aoc files file modify --path=my_aoc_test_folder
3993
+ aoc files file permission --path=my_aoc_test_folder list
3776
3994
  aoc files file show --path=/200KB.1
3777
3995
  aoc files file show my_file_id
3778
3996
  aoc files find / --value='\.partial$'
@@ -3782,12 +4000,14 @@ aoc files rename /somefolder testdst
3782
4000
  aoc files short_link create --to-folder=/testdst --value=private
3783
4001
  aoc files short_link create --to-folder=/testdst --value=public
3784
4002
  aoc files short_link list --value=@json:'{"purpose":"shared_folder_auth_link"}'
4003
+ aoc files sync admin status --to-folder=/testdst --sync-info=@json:'{"sessions":[{"name":"s2","direction":"pull","local_dir":"syncdir","reset":true}]}'
4004
+ aoc files sync start --to-folder=/testdst --sync-info=@json:'{"sessions":[{"name":"s2","direction":"pull","local_dir":"syncdir","reset":true}]}'
3785
4005
  aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
4006
+ aoc files upload --to-folder=/ testfile.bin --link=my_aoc_publink_folder
3786
4007
  aoc files upload --to-folder=/testsrc testfile.bin
3787
- aoc files upload -N --to-folder=/ testfile.bin --link=my_aoc_publink_folder
3788
4008
  aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
3789
4009
  aoc files v3 info
3790
- aoc org -N --link=my_aoc_publink_recv_from_aocuser
4010
+ aoc org --link=my_aoc_publink_recv_from_aocuser
3791
4011
  aoc organization
3792
4012
  aoc packages browse "my_package_id" /contents
3793
4013
  aoc packages list
@@ -3796,25 +4016,27 @@ aoc packages recv "my_package_id" --to-folder=.
3796
4016
  aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
3797
4017
  aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
3798
4018
  aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
4019
+ aoc packages send --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
4020
+ aoc packages send --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
3799
4021
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
3800
4022
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
3801
4023
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
3802
- aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
3803
- aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
3804
4024
  aoc packages shared_inboxes list
4025
+ aoc remind --username=my_aoc_user_email
4026
+ aoc servers
3805
4027
  aoc user profile modify @json:'{"name":"dummy change"}'
3806
4028
  aoc user profile show
3807
4029
  aoc user workspaces current
3808
4030
  aoc user workspaces list
3809
4031
  ```
3810
4032
 
3811
- ## <a id="ats"></a>Plugin: Aspera Transfer Service
4033
+ ## <a id="ats"></a>Plugin: `ats`: IBM Aspera Transfer Service
3812
4034
 
3813
4035
  ATS is usable either :
3814
4036
 
3815
- * from an AoC subscription : ascli aoc admin ats : use AoC authentication
4037
+ - from an AoC subscription : ascli aoc admin ats : use AoC authentication
3816
4038
 
3817
- * or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
4039
+ - or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
3818
4040
 
3819
4041
  ### IBM Cloud ATS : creation of api key
3820
4042
 
@@ -3843,8 +4065,8 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
3843
4065
 
3844
4066
  References:
3845
4067
 
3846
- * [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
3847
- * [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
4068
+ - [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
4069
+ - [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
3848
4070
 
3849
4071
  Then, to register the key by default for the ats plugin, create a preset. Execute:
3850
4072
 
@@ -3929,13 +4151,15 @@ The parameters provided to ATS for access key creation are the ones of [ATS API]
3929
4151
  ### ATS sample commands
3930
4152
 
3931
4153
  ```bash
3932
- ats access_key cluster akibmcloud --secret=somesecret
4154
+ ats access_key cluster ak2ibmcloud --secret=somesecret
3933
4155
  ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3934
- ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4156
+ ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak2ibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4157
+ ats access_key delete ak2ibmcloud
3935
4158
  ats access_key delete ak_aws
3936
- ats access_key delete akibmcloud
4159
+ ats access_key entitlement ak2ibmcloud
3937
4160
  ats access_key list --fields=name,id
3938
- ats access_key node akibmcloud browse / --secret=somesecret
4161
+ ats access_key node ak2ibmcloud browse / --secret=somesecret
4162
+ ats access_key show ak2ibmcloud
3939
4163
  ats api_key create
3940
4164
  ats api_key instances
3941
4165
  ats api_key list
@@ -3945,17 +4169,21 @@ ats cluster show --cloud=aws --region=eu-west-1
3945
4169
  ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
3946
4170
  ```
3947
4171
 
3948
- ## Plugin: IBM Aspera High Speed Transfer Server (transfer)
4172
+ ## <a id="server"></a>Plugin: `server`: IBM Aspera High Speed Transfer Server (SSH)
3949
4173
 
3950
- This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
4174
+ The `server` plugin is used for operations on Aspera HSTS using SSH authentication.
3951
4175
  It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
3952
- Modern mode is to use the node API and transfer tokens.
4176
+ An SSH session is established, authenticated with either a password or an SSH private key,
4177
+ then commands `ascp` (for transfers) and `ascmd` (for file operations) are executed.
4178
+
4179
+ > **Note:** The URL to be provided is usually: `ssh://_server_address_:33001`
3953
4180
 
3954
4181
  ### Server sample commands
3955
4182
 
3956
4183
  ```bash
3957
- server -N -Ptst_server_bykey -Plocal_user br /
4184
+ server br /
3958
4185
  server browse /
4186
+ server browse NEW_SERVER_FOLDER/testfile.bin
3959
4187
  server browse folder_1/target_hot
3960
4188
  server cp NEW_SERVER_FOLDER/testfile.bin folder_1/200KB.2
3961
4189
  server delete NEW_SERVER_FOLDER
@@ -3971,34 +4199,39 @@ server md5sum NEW_SERVER_FOLDER/testfile.bin
3971
4199
  server mkdir NEW_SERVER_FOLDER --logger=stdout
3972
4200
  server mkdir folder_1/target_hot
3973
4201
  server mv folder_1/200KB.2 folder_1/to.delete
4202
+ server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"syncdir"}}'
4203
+ server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"syncdir"}]}'
4204
+ server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"syncdir"},"remote":{"path":"'"NEW_SERVER_FOLDER"'"},"reset":true,"quiet":false}'
4205
+ server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"syncdir","reset":true}]}'
3974
4206
  server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
3975
4207
  server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
3976
4208
  server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
3977
4209
  server upload --sources=@ts --ts=@json:'{"EX_file_pair_list":"'"filepairlist.txt"'"}'
3978
4210
  server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
3979
4211
  server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
3980
- server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
4212
+ server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email --ascp-opts=@list:' -l 10m'
3981
4213
  server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
3982
4214
  server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
3983
- server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
4215
+ server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress=multi
3984
4216
  ```
3985
4217
 
3986
4218
  ### Authentication on Server with SSH session
3987
4219
 
3988
- If SSH is the session control protocol (i.e. not WSS), then following session authentication methods are supported:
4220
+ If SSH is the session protocol (by default i.e. not WSS), then following session authentication methods are supported:
3989
4221
 
3990
- * SSH password
3991
- * SSH keys (Multiple SSH key paths can be provided.)
4222
+ - `password`: SSH password
4223
+ - `ssh_keys`: SSH keys (Multiple SSH key paths can be provided.)
3992
4224
 
3993
- If username is not provided, the default transfer user `xfer` is used.
4225
+ If `username` is not provided then the default transfer user `xfer` is used.
3994
4226
 
3995
- If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used. Example:
4227
+ If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used.
4228
+ Example:
3996
4229
 
3997
- ```javascript
3998
- ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
4230
+ ```bash
4231
+ ascli server --url=ssh://_server_address_:33001 ... --ts=@json:'{"token":"Basic abc123"}'
3999
4232
  ```
4000
4233
 
4001
- > Note: If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
4234
+ > **Note:** If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
4002
4235
 
4003
4236
  The value of the `ssh_keys` option can be a single value or an array.
4004
4237
  Each value is a **path** to a private key and is expanded (`~` is replaced with the user's home folder).
@@ -4011,7 +4244,7 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
4011
4244
  ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
4012
4245
  ```
4013
4246
 
4014
- For non-transfer related command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
4247
+ For file operation command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
4015
4248
  For a list of SSH client options, refer to the ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html).
4016
4249
 
4017
4250
  By default the SSH library expect that a local ssh-agent is running.
@@ -4030,10 +4263,10 @@ ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant p
4030
4263
 
4031
4264
  This means that you don't have such an SSH agent running, then:
4032
4265
 
4033
- * check env var: `SSH_AGENT_SOCK`
4034
- * check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
4035
- * [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
4036
- * To disable the use of `ssh-agent`, use the option `ssh_options` like this:
4266
+ - Check env var: `SSH_AGENT_SOCK`
4267
+ - Check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
4268
+ - [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
4269
+ - To disable the use of `ssh-agent`, use the option `ssh_options` like this:
4037
4270
 
4038
4271
  ```bash
4039
4272
  ascli server --ssh-options=@ruby:'{use_agent: false}' ...
@@ -4041,7 +4274,17 @@ ascli server --ssh-options=@ruby:'{use_agent: false}' ...
4041
4274
 
4042
4275
  This can also be set as default using a global preset.
4043
4276
 
4044
- ### Example
4277
+ ### Other session channels for `server`
4278
+
4279
+ URL schemes `local` and `https` are also supported, mainly for testing purpose.
4280
+ (`--url=local:` , `--url=https://...`)
4281
+
4282
+ - `local` will execute `ascmd` locally, instead of using a SSH cnnection.
4283
+ - `https` will use Web Socket Session: This requires the use of a transfer token. For example a `Basic` token can be used.
4284
+
4285
+ As, most of the time, SSH is used, if an `http` scheme is provided without token, the plugin will fallback to SSH and port 33001.
4286
+
4287
+ ### Examples: `server`
4045
4288
 
4046
4289
  One can test the `server` application using the well known demo server:
4047
4290
 
@@ -4053,7 +4296,7 @@ ascli server download /aspera-test-dir-large/200MB
4053
4296
 
4054
4297
  `initdemo` creates a [option preset](#lprt) `demoserver` and set it as default for plugin `server`.
4055
4298
 
4056
- ## Plugin: IBM Aspera High Speed Transfer Server (node)
4299
+ ## <a id="node"></a>Plugin: `node`: IBM Aspera High Speed Transfer Server Node
4057
4300
 
4058
4301
  This plugin gives access to capabilities provided by HSTS node API.
4059
4302
 
@@ -4061,27 +4304,27 @@ This plugin gives access to capabilities provided by HSTS node API.
4061
4304
 
4062
4305
  It is possible to:
4063
4306
 
4064
- * browse
4065
- * transfer (upload / download)
4066
- * ...
4307
+ - browse
4308
+ - transfer (upload / download)
4309
+ - ...
4067
4310
 
4068
4311
  For transfers, it is possible to control how transfer is authorized using option: `token_type`:
4069
4312
 
4070
- * `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token, used as is.
4071
- * `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
4072
- * `basic` : transfer spec is created like this:
4313
+ - `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token, used as is.
4314
+ - `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
4315
+ - `basic` : transfer spec is created like this:
4073
4316
 
4074
4317
  ```javascript
4075
4318
  {
4076
- "remote_host": address of node url,
4319
+ "remote_host": "<address of node url>",
4077
4320
  "remote_user": "xfer",
4078
4321
  "ssh_port": 33001,
4079
4322
  "token": "Basic <base 64 encoded user/pass>",
4080
- "direction": send/receive
4323
+ "direction": "[send|receive]"
4081
4324
  }
4082
4325
  ```
4083
4326
 
4084
- Note that the port is assumed to be the default SSH port `33001` and transfer user is assumed to be `xfer`.
4327
+ > **Note:** the port is assumed to be the default Aspera SSH port `33001` and transfer user is assumed to be `xfer`.
4085
4328
 
4086
4329
  ### Central
4087
4330
 
@@ -4111,8 +4354,8 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
4111
4354
 
4112
4355
  `ascli` supports remote operations through the node API. Operations are:
4113
4356
 
4114
- * Start watchd and watchfolderd services running as a system user having access to files
4115
- * configure a watchfolder to define automated transfers
4357
+ - Start watchd and watchfolderd services running as a system user having access to files
4358
+ - configure a watchfolder to define automated transfers
4116
4359
 
4117
4360
  ```javascript
4118
4361
  ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
@@ -4168,9 +4411,19 @@ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"myst
4168
4411
  ### Node sample commands
4169
4412
 
4170
4413
  ```bash
4171
- node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
4172
- node -N -Ptst_node_preview access_key delete aoc_1
4173
- node access_key do my_aoc_ak_name br /
4414
+ node access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
4415
+ node access_key delete aoc_1
4416
+ node access_key do my_aoc_ak_name browse /
4417
+ node access_key do my_aoc_ak_name delete /folder2
4418
+ node access_key do my_aoc_ak_name delete testfile1
4419
+ node access_key do my_aoc_ak_name download testfile1 --to-folder=.
4420
+ node access_key do my_aoc_ak_name file show --path=/testfile1
4421
+ node access_key do my_aoc_ak_name file show 1
4422
+ node access_key do my_aoc_ak_name find /
4423
+ node access_key do my_aoc_ak_name mkdir /folder1
4424
+ node access_key do my_aoc_ak_name node_info /
4425
+ node access_key do my_aoc_ak_name rename /folder1 folder2
4426
+ node access_key do my_aoc_ak_name upload 'faux:///testfile1?1k' --default_ports=no
4174
4427
  node access_key list
4175
4428
  node api_details
4176
4429
  node async bandwidth 1
@@ -4182,7 +4435,7 @@ node async show ALL
4182
4435
  node basic_token
4183
4436
  node browse / -r
4184
4437
  node delete /todelete
4185
- node delete @list:,/todelete,/tdlink,/delfile
4438
+ node delete @list:,folder_1/todelete,folder_1/tdlink,folder_1/delfile
4186
4439
  node delete folder_1/10MB.1
4187
4440
  node delete testfile.bin
4188
4441
  node download testfile.bin --to-folder=.
@@ -4190,15 +4443,26 @@ node download testfile.bin --to-folder=. --token-type=hybrid
4190
4443
  node health
4191
4444
  node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
4192
4445
  node license
4193
- node mkdir /todelete
4194
- node mkfile /delfile1 "hello world"
4195
- node mklink /todelete /tdlink
4196
- node rename / delfile1 delfile
4446
+ node mkdir folder_1/todelete
4447
+ node mkfile folder_1/delfile1 "hello world"
4448
+ node mklink folder_1/todelete folder_1/tdlink
4449
+ node rename folder_1 delfile1 delfile
4197
4450
  node search / --value=@json:'{"sort":"mtime"}'
4198
4451
  node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
4199
4452
  node service delete service1
4200
4453
  node service list
4201
4454
  node space /
4455
+ node sync bandwidth my_syncid
4456
+ node sync counters my_syncid
4457
+ node sync create --value=@json:'{"configuration":{"name":"sync1","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password","path":"my_remote_path"}}}'
4458
+ node sync delete my_syncid
4459
+ node sync files my_syncid
4460
+ node sync list
4461
+ node sync show my_syncid
4462
+ node sync start my_syncid
4463
+ node sync state my_syncid
4464
+ node sync stop my_syncid
4465
+ node sync summary my_syncid
4202
4466
  node transfer list --value=@json:'{"active_only":true}'
4203
4467
  node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
4204
4468
  node upload --username=my_aoc_ak_name --password=my_aoc_ak_secret testfile.bin --token-type=basic
@@ -4206,123 +4470,130 @@ node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps
4206
4470
  node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
4207
4471
  ```
4208
4472
 
4209
- ## Plugin: IBM Aspera Faspex5
4210
-
4211
- This is currently in beta, limited operations are supported.
4473
+ ## <a id="faspex5"></a>Plugin: `faspex5`: IBM Aspera Faspex v5
4212
4474
 
4213
- This was tested with version Beta 5.
4214
-
4215
- The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "IBM Aspera Faspex API".
4216
-
4217
- ### Faspex 5 authentication
4475
+ IBM Aspera's newer self-managed application.
4218
4476
 
4219
4477
  3 authentication methods are supported:
4220
4478
 
4221
- * jwt
4222
- * web
4223
- * boot
4479
+ - jwt
4480
+ - web
4481
+ - boot
4224
4482
 
4225
- #### Faspex 5 using JWT
4483
+ ### Faspex 5 JWT authentication
4226
4484
 
4227
- This is the preferred method to use.
4485
+ This is the **recomended** method to use.
4228
4486
 
4229
4487
  For `jwt`, create an API client in Faspex with JWT support:
4230
4488
 
4231
- * Select a private key file: if you don't have any refer to section [Private Key](#private_key)
4232
- * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
4233
- * Activate JWT
4234
- * Paste public key in the appropriate section
4235
- * Click on Create Button
4236
- * Take note of Client Id (and Client Secret, but not used in current version)
4489
+ - Select a private key file: if you don't have any refer to section [Private Key](#private_key)
4490
+ - Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
4491
+ - Activate JWT
4492
+ - Paste **public** key in the appropriate section
4493
+ - Click on Create Button
4494
+ - Take note of Client Id (and Client Secret, but not used in current version)
4237
4495
 
4238
4496
  Then use these options:
4239
4497
 
4240
4498
  ```text
4241
4499
  --auth=jwt
4242
- --client-id=xxx
4243
- --client-secret=xxx
4244
- --username=xxx
4500
+ --client-id=_client_id_here_
4501
+ --client-secret=_secret_here_
4502
+ --username=_username_here_
4245
4503
  --private-key=@file:.../path/to/key.pem
4246
4504
  ```
4247
4505
 
4248
- #### Faspex 5 using web browser
4506
+ > **Note:** The `private_key` option must contain the PEM value of the private key which can be read from a file using the modifier: `@file:`, e.g. `@file:/path/to/key.pem`.
4507
+
4508
+ ### Faspex 5 web authentication
4249
4509
 
4250
4510
  For `web` method, create an API client in Faspex without JWT:
4251
4511
 
4252
- * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
4253
- * Do not Activate JWT
4254
- * enter `https://127.0.0.1:8888` in the redirect URI
4255
- * Click on Create Button
4256
- * Take note of Client Id (and Client Secret, but not used in current version)
4512
+ - Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
4513
+ - Do not Activate JWT
4514
+ - Set **Redirect URI** to `https://127.0.0.1:8888`
4515
+ - Click on Create Button
4516
+ - Take note of Client Id (and Client Secret, but not used in current version)
4257
4517
 
4258
4518
  Then use options:
4259
4519
 
4260
4520
  ```text
4261
4521
  --auth=web
4262
- --client-id=xxx
4263
- --client-secret=xxx
4522
+ --client-id=_client_id_here_
4523
+ --client-secret=_secret_here_
4264
4524
  --redirect-uri=https://127.0.0.1:8888
4265
4525
  ```
4266
4526
 
4267
- #### Faspex 5 using bootstrap
4527
+ ### Faspex 5 bootstrap authentication
4268
4528
 
4269
4529
  For `boot` method: (will be removed in future)
4270
4530
 
4271
- * open a browser
4272
- * start developer mode
4273
- * login to faspex 5
4274
- * find the first API call with `Authorization` token, and copy it (kind of base64 long string)
4531
+ - Open a Web Browser
4532
+ - Start developer mode
4533
+ - Login to Faspex 5
4534
+ - Find the first API call with `Authorization` header, and copy the value of the token (series of base64 values with dots)
4275
4535
 
4276
- Use it as password and use `--auth=boot`.
4536
+ Use this token as password and use `--auth=boot`.
4277
4537
 
4278
4538
  ```bash
4279
- ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
4539
+ ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=_token_here_
4280
4540
  ```
4281
4541
 
4282
4542
  ### Faspex 5 sample commands
4283
4543
 
4544
+ Most commands are directly REST API calls.
4545
+ Parameters to commandsa are carried through option `value`, as extended value.
4546
+ Usually using JSON format with prefix `@json:`.
4547
+
4548
+ > **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
4549
+
4284
4550
  ```bash
4285
4551
  faspex5 admin res accounts list
4286
4552
  faspex5 admin res contacts list
4287
4553
  faspex5 admin res jobs list
4554
+ faspex5 admin res metadata_profiles list
4288
4555
  faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
4289
4556
  faspex5 admin res oauth_clients list
4290
4557
  faspex5 admin res registrations list
4291
4558
  faspex5 admin res saml_configs list
4292
4559
  faspex5 admin res shared_inboxes list
4293
4560
  faspex5 admin res workgroups list
4561
+ faspex5 bearer_token
4294
4562
  faspex5 health
4295
4563
  faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
4296
4564
  faspex5 package receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
4297
4565
  faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"_content_prot_here_"}'
4566
+ faspex5 package show "my_package_id"
4567
+ faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
4568
+ faspex5 user profile show
4298
4569
  ```
4299
4570
 
4300
- ### Faspex 5 other examples
4571
+ Other examples:
4301
4572
 
4302
- * List all shared inboxes
4573
+ - List all shared inboxes
4303
4574
 
4304
4575
  ```javascript
4305
4576
  ascli faspex5 admin res shared list --value=@json:'{"all":true}' --fields=id,name
4306
4577
  ```
4307
4578
 
4308
- * Create Metadata profile
4579
+ - Create Metadata profile
4309
4580
 
4310
4581
  ```javascript
4311
4582
  ascli faspex5 admin res metadata_profiles create --value=@json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
4312
4583
  ```
4313
4584
 
4314
- * Create a Shared inbox with specific metadata profile
4585
+ - Create a Shared inbox with specific metadata profile
4315
4586
 
4316
4587
  ```javascript
4317
4588
  ascli faspex5 admin res shared create --value=@json:'{"name":"the shared inbox","metadata_profile_id":1}'
4318
4589
  ```
4319
4590
 
4320
- ## Plugin: IBM Aspera Faspex (4.x)
4591
+ ## <a id="faspex"></a>Plugin: `faspex`: IBM Aspera Faspex v4
4321
4592
 
4322
4593
  Notes:
4323
4594
 
4324
- * The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
4325
- * For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
4595
+ - The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
4596
+ - For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
4326
4597
 
4327
4598
  ### Listing Packages
4328
4599
 
@@ -4336,18 +4607,18 @@ By default it looks in box `inbox`, but the following boxes are also supported:
4336
4607
 
4337
4608
  A user can receive a package because the recipient is:
4338
4609
 
4339
- * the user himself (default)
4340
- * the user is member of a dropbox/workgroup: filter using option `recipient` set with value `*<name of dropbox/workgroup>`
4610
+ - the user himself (default)
4611
+ - the user is member of a dropbox/workgroup: filter using option `recipient` set with value `*<name of dropbox/workgroup>`
4341
4612
 
4342
4613
  #### Option `query`
4343
4614
 
4344
4615
  As inboxes may be large, it is possible to use the following query parameters:
4345
4616
 
4346
- * `count` : (native) number items in one API call (default=0, equivalent to 10)
4347
- * `page` : (native) id of page in call (default=0)
4348
- * `startIndex` : (native) index of item to start, default=0, oldest index=0
4349
- * `max` : maximum number of items
4350
- * `pmax` : maximum number of pages
4617
+ - `count` : (native) number items in one API call (default=0, equivalent to 10)
4618
+ - `page` : (native) id of page in call (default=0)
4619
+ - `startIndex` : (native) index of item to start, default=0, oldest index=0
4620
+ - `max` : maximum number of items
4621
+ - `pmax` : maximum number of pages
4351
4622
 
4352
4623
  (SQL query is `LIMIT <startIndex>, <count>`)
4353
4624
 
@@ -4367,9 +4638,9 @@ List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in recei
4367
4638
 
4368
4639
  The command is `package recv`, possible methods are:
4369
4640
 
4370
- * provide a package id with option `id`
4371
- * provide a public link with option `link`
4372
- * provide a `faspe:` URI with option `link`
4641
+ - provide a package id with option `id`
4642
+ - provide a public link with option `link`
4643
+ - provide a `faspe:` URI with option `link`
4373
4644
 
4374
4645
  ```bash
4375
4646
  ascli faspex package recv --id=12345
@@ -4401,8 +4672,8 @@ If the recipient is a dropbox or workgroup: provide the name of the dropbox or w
4401
4672
 
4402
4673
  Additional optional parameters in `delivery_info`:
4403
4674
 
4404
- * Package Note: : `"note":"note this and that"`
4405
- * Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
4675
+ - Package Note: : `"note":"note this and that"`
4676
+ - Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
4406
4677
 
4407
4678
  ### Email notification on transfer
4408
4679
 
@@ -4500,7 +4771,7 @@ faspex v4 wmembership list
4500
4771
  faspex v4 workgroup list
4501
4772
  ```
4502
4773
 
4503
- ## Plugin: IBM Aspera Shares
4774
+ ## <a id="shares"></a>Plugin: `shares`: IBM Aspera Shares v1
4504
4775
 
4505
4776
  Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
4506
4777
 
@@ -4508,33 +4779,39 @@ Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and
4508
4779
 
4509
4780
  ```bash
4510
4781
  shares admin share list
4511
- shares admin share user_permissions 9
4782
+ shares admin share list --fields=-status,status_message
4783
+ shares admin share user_permissions 1
4512
4784
  shares admin user app_authorizations 1
4785
+ shares admin user ldap_import --value=the_name
4513
4786
  shares admin user list
4787
+ shares admin user saml_import --value=@json:'{"id":"the_id","name_id":"the_name"}'
4514
4788
  shares admin user share_permissions 1
4789
+ shares health
4515
4790
  shares repository browse /
4516
4791
  shares repository delete my_shares_upload/testfile.bin
4517
4792
  shares repository download --to-folder=. my_shares_upload/testfile.bin
4518
- shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
4793
+ shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
4519
4794
  shares repository upload --to-folder=my_shares_upload testfile.bin
4520
- shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
4795
+ shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
4521
4796
  ```
4522
4797
 
4523
- ## Plugin: Console
4798
+ ## <a id="console"></a>Plugin: `console`: IBM Aspera Console
4524
4799
 
4525
4800
  ### Console sample commands
4526
4801
 
4527
4802
  ```bash
4528
- console transfer current list
4529
- console transfer smart list
4803
+ console health
4804
+ console transfer current list
4805
+ console transfer smart list
4530
4806
  console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
4531
4807
  ```
4532
4808
 
4533
- ## Plugin: Orchestrator
4809
+ ## <a id="orchestrator"></a>Plugin: `orchestrator`:IBM Aspera Orchestrator
4534
4810
 
4535
4811
  ### Orchestrator sample commands
4536
4812
 
4537
4813
  ```bash
4814
+ orchestrator health
4538
4815
  orchestrator info
4539
4816
  orchestrator plugins
4540
4817
  orchestrator processes
@@ -4548,22 +4825,24 @@ orchestrator workflow status ALL
4548
4825
  orchestrator workflow status my_orch_workflow_id
4549
4826
  ```
4550
4827
 
4551
- ## Plugin: IBM Cloud Object Storage
4828
+ ## <a id="cos"></a>Plugin: `cos`: IBM Cloud Object Storage
4552
4829
 
4553
4830
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
4554
4831
  It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
4555
4832
  Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
4556
4833
 
4557
- There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
4834
+ There are two possibilities to provide credentials.
4835
+ If you already have the endpoint, apikey and CRN, use the first method.
4836
+ If you don't have credentials but have access to the IBM Cloud console, then use the second method.
4558
4837
 
4559
4838
  ### Using endpoint, apikey and Resource Instance ID (CRN)
4560
4839
 
4561
4840
  If you have those parameters already, then following options shall be provided:
4562
4841
 
4563
- * `bucket` bucket name
4564
- * `endpoint` storage endpoint url, e.g. `https://s3.hkg02.cloud-object-storage.appdomain.cloud`
4565
- * `apikey` API Key
4566
- * `crn` resource instance id
4842
+ - `bucket` bucket name
4843
+ - `endpoint` storage endpoint url, e.g. `https://s3.hkg02.cloud-object-storage.appdomain.cloud`
4844
+ - `apikey` API Key
4845
+ - `crn` resource instance id
4567
4846
 
4568
4847
  For example, let us create a default configuration:
4569
4848
 
@@ -4576,9 +4855,17 @@ Then, jump to the transfer example.
4576
4855
 
4577
4856
  ### Using service credential file
4578
4857
 
4579
- If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
4858
+ If you are the COS administrator and don't have yet the credential:
4859
+ Service credentials are directly created using the IBM cloud Console (web UI).
4860
+ Navigate to:
4580
4861
 
4581
- Navigation Menu &rarr; Resource List &rarr; Storage &rarr; Cloud Object Storage &rarr; Service Credentials &rarr; &lt;select or create credentials&gt; &rarr; view credentials &rarr; copy
4862
+ - &rarr; Navigation Menu
4863
+ - &rarr; [Resource List](https://cloud.ibm.com/resources)
4864
+ - &rarr; [Storage](https://cloud.ibm.com/objectstorage)
4865
+ - &rarr; Select your storage instance
4866
+ - &rarr; Service Credentials
4867
+ - &rarr; New credentials (Leave default role: Writer, no special options)
4868
+ - &rarr; Copy to clipboard
4582
4869
 
4583
4870
  Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
4584
4871
 
@@ -4617,9 +4904,9 @@ The field `apikey` is for option `apikey`
4617
4904
 
4618
4905
  The required options for this method are:
4619
4906
 
4620
- * `bucket` bucket name
4621
- * `region` bucket region, e.g. eu-de
4622
- * `service_credentials` see below
4907
+ - `bucket` bucket name
4908
+ - `region` bucket region, e.g. eu-de
4909
+ - `service_credentials` see below
4623
4910
 
4624
4911
  For example, let us create a default configuration:
4625
4912
 
@@ -4639,39 +4926,51 @@ ascli cos node info
4639
4926
  ascli cos node upload 'faux:///sample1G?1g'
4640
4927
  ```
4641
4928
 
4642
- Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
4929
+ Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man `ascp` and section above), but you can of course send a real file by specifying a real file instead.
4643
4930
 
4644
4931
  ### COS sample commands
4645
4932
 
4646
4933
  ```bash
4647
- cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
4648
- cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
4934
+ cos --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
4935
+ cos --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
4649
4936
  cos node access_key show self
4650
4937
  cos node download testfile.bin --to-folder=.
4651
4938
  cos node info
4652
4939
  cos node upload testfile.bin
4653
4940
  ```
4654
4941
 
4655
- ## Plugin: IBM Aspera Sync
4942
+ ## <a id="async"></a>Plugin: `async`: IBM Aspera Sync
4656
4943
 
4657
4944
  A basic plugin to start an "async" using `ascli`.
4658
- The main advantage is the possibility to start from ma configuration file, using `ascli` standard options.
4945
+ The main advantage over bare `async` command line is the possibility to use a configuration file, using `ascli` standard options.
4946
+
4947
+ Also, the `sync` command is also made available through the `server sync` and `aoc files sync` commands.
4948
+ In this case, some of the `sync` parameters are fill from parameters of the related plugin.
4949
+
4950
+ > **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
4951
+ >
4952
+ > **Note:** Two JSON syntax are supported for option `sync_info`.
4953
+ > The first is same sync payload as specified on the `async` option `--conf` or in the latest node API, this is the prefered syntax and allows a single session definition.
4954
+ > The second (legacy) is specific to `ascli` and allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
4955
+
4956
+ Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
4659
4957
 
4660
4958
  ### Sync sample commands
4661
4959
 
4662
4960
  ```bash
4663
- sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
4961
+ sync admin status --sync-info=@json:'{"sessions":[{"name":"test","local_dir":"contents"}]}'
4962
+ sync start --sync-info=@json:'{"instance":{"quiet":true},"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_paths":["my_local_user_key"]}]}'
4664
4963
  ```
4665
4964
 
4666
- ## Plugin: Preview
4965
+ ## <a id="preview"></a>Plugin: `preview`: Preview generator for AoC
4667
4966
 
4668
4967
  The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
4669
4968
  It uses the **node API** of Aspera HSTS and requires use of Access Keys and it's **storage root**.
4670
4969
  Several parameters can be used to tune several aspects:
4671
4970
 
4672
- * Methods for detection of new files needing generation
4673
- * Methods for generation of video preview
4674
- * Parameters for video handling
4971
+ - Methods for detection of new files needing generation
4972
+ - Methods for generation of video preview
4973
+ - Parameters for video handling
4675
4974
 
4676
4975
  ### Aspera Server configuration
4677
4976
 
@@ -4713,10 +5012,10 @@ Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* whi
4713
5012
 
4714
5013
  The tool requires the following external tools available in the `PATH`:
4715
5014
 
4716
- * ImageMagick : `convert` `composite`
4717
- * OptiPNG : `optipng`
4718
- * FFmpeg : `ffmpeg` `ffprobe`
4719
- * Libreoffice : `libreoffice`
5015
+ - ImageMagick : `convert` `composite`
5016
+ - OptiPNG : `optipng`
5017
+ - FFmpeg : `ffmpeg` `ffprobe`
5018
+ - Libreoffice : `libreoffice`
4720
5019
 
4721
5020
  Here shown on Redhat/CentOS.
4722
5021
 
@@ -4752,13 +5051,13 @@ If you don't want to have preview for office documents or if it is too complex y
4752
5051
 
4753
5052
  The generation of preview in based on the use of `unoconv` and `libreoffice`
4754
5053
 
4755
- * CentOS 8
5054
+ - CentOS 8
4756
5055
 
4757
5056
  ```bash
4758
5057
  dnf install unoconv
4759
5058
  ```
4760
5059
 
4761
- * Amazon Linux
5060
+ - Amazon Linux
4762
5061
 
4763
5062
  ```bash
4764
5063
  amazon-linux-extras enable libreoffice
@@ -4775,7 +5074,7 @@ The preview generator is run as a user, preferably a regular user (not root). Wh
4775
5074
 
4776
5075
  Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt). The configuration file must be created with the same user used to run so that it is properly used on runtime.
4777
5076
 
4778
- Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
5077
+ The `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
4779
5078
 
4780
5079
  ```bash
4781
5080
  su -s /bin/bash - xfer
@@ -4798,11 +5097,11 @@ This shall list the contents of the storage root of the access key.
4798
5097
 
4799
5098
  ### Execution
4800
5099
 
4801
- The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
4802
- It needs to be run on a regular basis to create or update preview files. For that use your best
4803
- reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
5100
+ The tool intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
5101
+ It needs to be run on a regular basis to create or update preview files.
5102
+ For that use your best reliable scheduler, see [Scheduling](#scheduling).
4804
5103
 
4805
- Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
5104
+ Typically, for **Access key** access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
4806
5105
 
4807
5106
  Lets do a one shot test, using the configuration previously created:
4808
5107
 
@@ -4818,12 +5117,12 @@ On subsequent run it reads this file and check that previews are generated for t
4818
5117
 
4819
5118
  ### Configuration for Execution in scheduler
4820
5119
 
4821
- Here is an example of configuration for use with cron on Linux.
4822
- Adapt the scripts to your own preference.
5120
+ Here is an example of configuration for use with `cron` on Linux.
5121
+ Adapt the scripts to your own needs.
4823
5122
 
4824
5123
  We assume here that a configuration preset was created as shown previously.
4825
5124
 
4826
- Lets first setup a script that will be used in the scheduler and sets up the environment.
5125
+ Lets first setup a script that will be used in the scheduler and set up the environment.
4827
5126
 
4828
5127
  Example of startup script `cron_ascli`, which sets the Ruby environment and adds some timeout protection:
4829
5128
 
@@ -4845,25 +5144,25 @@ crontab<<EOF
4845
5144
  EOF
4846
5145
  ```
4847
5146
 
4848
- Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
5147
+ > **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
4849
5148
 
4850
5149
  ### Candidate detection for creation or update (or deletion)
4851
5150
 
4852
5151
  The tool generates preview files using those commands:
4853
5152
 
4854
- * `trevents` : only recently uploaded files will be tested (transfer events)
4855
- * `events` : only recently uploaded files will be tested (file events: not working)
4856
- * `scan` : recursively scan all files under the access key&apos;s "storage root"
4857
- * `test` : test using a local file
5153
+ - `trevents` : only recently uploaded files will be tested (transfer events)
5154
+ - `events` : only recently uploaded files will be tested (file events: not working)
5155
+ - `scan` : recursively scan all files under the access key&apos;s "storage root"
5156
+ - `test` : test using a local file
4858
5157
 
4859
5158
  Once candidate are selected, once candidates are selected,
4860
5159
  a preview is always generated if it does not exist already,
4861
5160
  else if a preview already exist, it will be generated
4862
5161
  using one of three values for the `overwrite` option:
4863
5162
 
4864
- * `always` : preview is always generated, even if it already exists and is newer than original
4865
- * `never` : preview is generated only if it does not exist already
4866
- * `mtime` : preview is generated only if the original file is newer than the existing
5163
+ - `always` : preview is always generated, even if it already exists and is newer than original
5164
+ - `never` : preview is generated only if it does not exist already
5165
+ - `mtime` : preview is generated only if the original file is newer than the existing
4867
5166
 
4868
5167
  Deletion of preview for deleted source files: not implemented yet (TODO).
4869
5168
 
@@ -4887,8 +5186,8 @@ For instance to filter out files beginning with `._` do:
4887
5186
 
4888
5187
  Two types of preview can be generated:
4889
5188
 
4890
- * png: thumbnail
4891
- * mp4: video preview (only for video)
5189
+ - png: thumbnail
5190
+ - mp4: video preview (only for video)
4892
5191
 
4893
5192
  Use option `skip_format` to skip generation of a format.
4894
5193
 
@@ -4896,11 +5195,11 @@ Use option `skip_format` to skip generation of a format.
4896
5195
 
4897
5196
  The preview generator supports rendering of those file categories:
4898
5197
 
4899
- * image
4900
- * pdf
4901
- * plaintext
4902
- * office
4903
- * video
5198
+ - image
5199
+ - pdf
5200
+ - plaintext
5201
+ - office
5202
+ - video
4904
5203
 
4905
5204
  To avoid generation for some categories, specify a list using option `skip_types`.
4906
5205
 
@@ -4923,31 +5222,31 @@ In this case the `preview` command will first analyze the file content using mim
4923
5222
 
4924
5223
  If the `mimemagic` gem complains about missing mime info file:
4925
5224
 
4926
- * any OS:
5225
+ - any OS:
4927
5226
 
4928
- * Examine the error message
4929
- * Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
4930
- * move and rename this file to one of the locations expected by mimemagic as specified in the error message
5227
+ - Examine the error message
5228
+ - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5229
+ - move and rename this file to one of the locations expected by mimemagic as specified in the error message
4931
5230
 
4932
- * Windows:
5231
+ - Windows:
4933
5232
 
4934
- * Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
4935
- * Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
4936
- * Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
5233
+ - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5234
+ - Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
5235
+ - Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
4937
5236
 
4938
5237
  ```cmd
4939
5238
  SETX FREEDESKTOP_MIME_TYPES_PATH C:\RubyVV-x64\freedesktop.org.xml.in
4940
5239
  ```
4941
5240
 
4942
- * Close the `cmd` and restart a new one if needed to get refreshed env vars
5241
+ - Close the `cmd` and restart a new one if needed to get refreshed env vars
4943
5242
 
4944
- * Linux:
5243
+ - Linux:
4945
5244
 
4946
5245
  ```bash
4947
5246
  yum install shared-mime-info
4948
5247
  ```
4949
5248
 
4950
- * macOS:
5249
+ - macOS:
4951
5250
 
4952
5251
  ```bash
4953
5252
  brew install shared-mime-info
@@ -4986,17 +5285,17 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
4986
5285
  Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
4987
5286
 
4988
5287
  The `smtp` option is a hash table (extended value) with the following fields:
4989
- <table>
4990
- <tr><th>field</th><th>default</th><th>example</th><th>description</th></tr>
4991
- <tr><td>`server`</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
4992
- <tr><td>`tls`</td><td>true</td><td>false</td><td>use of TLS</td></tr>
4993
- <tr><td>`port`</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
4994
- <tr><td>`domain`</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
4995
- <tr><td>`username`</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
4996
- <tr><td>`password`</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
4997
- <tr><td>`from_email`</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
4998
- <tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
4999
- </table>
5288
+
5289
+ | field | default | example | description |
5290
+ |--------------|---------------------|----------------------------|----------------------------------|
5291
+ | `server` | - | smtp.gmail.com | SMTP server address |
5292
+ | `tls` | true | false | use of TLS |
5293
+ | `port` | TLS: 587<br/>25 | 587 | port for service |
5294
+ | `domain` | domain of server | gmail.com | email domain of user |
5295
+ | `username` | - | john@example.com | user to authenticate on SMTP server, leave empty for open auth. |
5296
+ | `password` | - | my_password_here | password for above username |
5297
+ | `from_email` | username if defined | johnny@example.com | address used if receiver replies |
5298
+ | `from_name` | same as email | John Wayne | display name of sender |
5000
5299
 
5001
5300
  ### Example of configuration
5002
5301
 
@@ -5033,9 +5332,9 @@ The template is the full SMTP message, including headers.
5033
5332
 
5034
5333
  The following variables are defined by default:
5035
5334
 
5036
- * from_name
5037
- * from_email
5038
- * to
5335
+ - `from_name`
5336
+ - `from_email`
5337
+ - `to`
5039
5338
 
5040
5339
  Other variables are defined depending on context.
5041
5340
 
@@ -5058,10 +5357,10 @@ A default e-mail template is used, but it can be overridden with option `notif_t
5058
5357
 
5059
5358
  The environment provided contains the following additional variables:
5060
5359
 
5061
- * subject
5062
- * body
5063
- * global_transfer_status
5064
- * ts
5360
+ - subject
5361
+ - body
5362
+ - global_transfer_status
5363
+ - ts
5065
5364
 
5066
5365
  Example of template:
5067
5366
 
@@ -5079,9 +5378,9 @@ This gem comes with a second executable tool providing a simplified standardized
5079
5378
 
5080
5379
  It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transferspec) is:
5081
5380
 
5082
- * common to Aspera Node API (HTTP POST /ops/transfer)
5083
- * common to Aspera Connect API (browser javascript startTransfer)
5084
- * easy to generate by using any third party language specific JSON library
5381
+ - common to Aspera Node API (HTTP POST /ops/transfer)
5382
+ - common to Aspera Connect API (browser javascript startTransfer)
5383
+ - easy to generate by using any third party language specific JSON library
5085
5384
 
5086
5385
  Hopefully, IBM integrates this diectly in `ascp`, and this tool is made redundant.
5087
5386
 
@@ -5091,26 +5390,26 @@ The tool expect one single argument: a [*transfer-spec*](#transferspec).
5091
5390
 
5092
5391
  If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
5093
5392
 
5094
- Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
5393
+ > **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the hash using JSON syntax.
5095
5394
 
5096
5395
  During execution, it generates all low level events, one per line, in JSON format on stdout.
5097
5396
 
5098
- Note that there are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
5397
+ There are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
5099
5398
 
5100
- * `EX_loglevel` to change log level of the tool
5101
- * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
5399
+ - `EX_loglevel` to change log level of the tool
5400
+ - `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
5102
5401
 
5103
- Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
5402
+ > **Note:** In addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
5104
5403
 
5105
5404
  ### Comparison of interfaces
5106
5405
 
5107
- <table>
5108
- <tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
5109
- <tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>many</td></tr>
5110
- <tr><td>required additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
5111
- <tr><td>startup</td><td>JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn)</td><td>command line arguments</td><td>API</td><td>daemon</td></tr>
5112
- <tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
5113
- <tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp (and SDK if compiled)</td><td>any with ascp</td><td>any with ascp and transfer daemon</td></tr></table>
5406
+ | feature/tool | asession | `ascp` | FaspManager | Transfer SDK |
5407
+ |--------------|----------|--------|-------------|--------------|
5408
+ | language integration | any | any | C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/> | many |
5409
+ | required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
5410
+ | startup | JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn) | command line arguments | API | daemon |
5411
+ | events | JSON on stdout | none by default<br/>or need to open management port<br/>and proprietary text syntax | callback | callback |
5412
+ | platforms | any with ruby and `ascp` | any with `ascp` (and SDK if compiled) | any with `ascp` | any with `ascp` and transfer daemon |
5114
5413
 
5115
5414
  ### Simple session
5116
5415
 
@@ -5122,13 +5421,13 @@ Create a file `session.json` with:
5122
5421
 
5123
5422
  Then start the session:
5124
5423
 
5125
- ```
5424
+ ```bash
5126
5425
  asession < session.json
5127
5426
  ```
5128
5427
 
5129
5428
  ### Asynchronous commands and Persistent session
5130
5429
 
5131
- `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
5430
+ `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
5132
5431
 
5133
5432
  This is particularly useful for a persistent session ( with the [*transfer-spec*](#transferspec) parameter: `"keepalive":true` )
5134
5433
 
@@ -5176,13 +5475,14 @@ EXAMPLES
5176
5475
 
5177
5476
  ### Requirements
5178
5477
 
5179
- `ascli` maybe used as a simple hot folder engine. A hot folder being defined as a tool that:
5478
+ `ascli` maybe used as a simple hot folder engine.
5479
+ A hot folder being defined as a tool that:
5180
5480
 
5181
- * locally (or remotely) detects new files in a top folder
5182
- * send detected files to a remote (respectively, local) repository
5183
- * only sends new files, do not re-send already sent files
5184
- * optionally: sends only files that are not still "growing"
5185
- * optionally: after transfer of files, deletes or moves to an archive
5481
+ - locally (or remotely) detects new files in a top folder
5482
+ - send detected files to a remote (respectively, local) repository
5483
+ - only sends new files, do not re-send already sent files
5484
+ - optionally: sends only files that are not still "growing"
5485
+ - optionally: after transfer of files, deletes or moves to an archive
5186
5486
 
5187
5487
  In addition: the detection should be made "continuously" or on specific time/date.
5188
5488
 
@@ -5190,42 +5490,67 @@ In addition: the detection should be made "continuously" or on specific time/dat
5190
5490
 
5191
5491
  The general idea is to rely on :
5192
5492
 
5193
- * existing `ascp` features for detection and transfer
5194
- * take advantage of `ascli` configuration capabilities and server side knowledge
5195
- * the OS scheduler for reliability and continuous operation
5493
+ - existing `ascp` features for detection and transfer
5494
+ - take advantage of `ascli` configuration capabilities and server side knowledge
5495
+ - the OS scheduler for reliability and continuous operation
5196
5496
 
5197
- #### ascp features
5497
+ #### `ascp` features
5198
5498
 
5199
- Interesting ascp features are found in its arguments: (see ascp manual):
5499
+ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
5200
5500
 
5201
- * `ascp` already takes care of sending only "new" files: option `-k 1,2,3`, or transfer_spec: `resume_policy`
5202
- * `ascp` has some options to remove or move files after transfer: `--remove-after-transfer`, `--move-after-transfer`, `--remove-empty-directories`
5203
- * `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than` (--exclude-older-than)
5204
- * `--src-base` if top level folder name shall not be created on destination
5501
+ - `ascp` already takes care of sending only "new" files: option `-k 1,2,3` (`resume_policy`)
5502
+ - `ascp` has some options to remove or move files after transfer: `--remove-after-transfer`, `--move-after-transfer`, `--remove-empty-directories` (`remove_after_transfer`, `move_after_transfer`, `remove_empty_directories`)
5503
+ - `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
5504
+ - `--src-base` (`src_base`) if top level folder name shall not be created on destination
5205
5505
 
5206
5506
  Note that:
5207
5507
 
5208
- * `ascli` takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
5209
- * most, but not all native ascp arguments are available as standard transfer_spec parameters
5210
- * native ascp arguments can be provided with the [*transfer-spec*](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
5508
+ - `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transferspec), with `--ts` parameter.
5509
+ - most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transferspec) parameters
5510
+ - native `ascp` arguments can be provided with the [*transfer-spec*](#transferspec) parameter: `EX_ascp_args` (array), only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node)
5211
5511
 
5212
5512
  #### server side and configuration
5213
5513
 
5214
- Virtually any transfer on a "repository" on a regular basis might emulate a hot folder. Note that file detection is not based on events (inotify, etc...), but on a stateless scan on source side.
5514
+ Virtually any transfer on a "repository" on a regular basis might emulate a hot folder.
5215
5515
 
5216
- Note: parameters may be saved in a [option preset](#lprt) and used with `-P`.
5516
+ > **Note:** file detection is not based on events (inotify, etc...), but on a simple folder scan on source side.
5517
+ >
5518
+ > **Note:** parameters may be saved in a [option preset](#lprt) and used with `-P`.
5217
5519
 
5218
5520
  #### Scheduling
5219
5521
 
5220
- Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [Scheduling](#scheduling).
5522
+ Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
5523
+ Refer to section [Scheduling](#scheduling). (on use of option `lock_port`)
5221
5524
 
5222
- ### Example: upload folder
5525
+ ### Example: upload hot folder
5223
5526
 
5224
- ```javascript
5225
- ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
5527
+ ```bash
5528
+ ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"remove_after_transfer":true,"remove_empty_directories":true,"exclude_newer_than:-8,"src_base":"source_hot"}'
5529
+ ```
5530
+
5531
+ The local folder (here, relative path: `source_hot`) is sent (upload) to an aspera server.
5532
+ Source files are deleted after transfer.
5533
+ Growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period).
5534
+ If a transfer takes more than the execution period, then the subsequent execution is skipped (`lock_port`) preventing multiple concurrent runs.
5535
+
5536
+ ### Example: unidirectional synchronization (upload) to server
5537
+
5538
+ ```bash
5539
+ ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=12345 --ts=@json:'{"resume_policy":"sparse_csum","exclude_newer_than":-8,"src_base":"source_sync"}'
5540
+ ```
5541
+
5542
+ This can also be used with other folder-based applications: Aspera on Cloud, Shares, Node:
5543
+
5544
+ ### Example: unidirectional synchronization (download) from Aspera on Cloud Files
5545
+
5546
+ ```bash
5547
+ ascli aoc files download . --to-folder=. --lock-port=12345 --progress=none --display=data \
5548
+ --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
5226
5549
  ```
5227
5550
 
5228
- The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
5551
+ > **Note:** option `delete_before_transfer` will delete files locally, if they are not present on remote side.
5552
+ >
5553
+ > **Note:** options `progress` and `display` limit output for headless operation (e.g. cron job)
5229
5554
 
5230
5555
  ## Health check and Nagios
5231
5556
 
@@ -5245,9 +5570,9 @@ ascli console health
5245
5570
 
5246
5571
  Typically, the health check uses the REST API of the application with the following exception: the `server` plugin allows checking health by:
5247
5572
 
5248
- * issuing a transfer to the server
5249
- * checking web app status with `asctl all:status`
5250
- * checking daemons process status
5573
+ - issuing a transfer to the server
5574
+ - checking web app status with `asctl all:status`
5575
+ - checking daemons process status
5251
5576
 
5252
5577
  `ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
5253
5578
 
@@ -5271,9 +5596,9 @@ OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:runnin
5271
5596
 
5272
5597
  Main components:
5273
5598
 
5274
- * `Aspera` generic classes for REST and OAuth
5275
- * `Aspera::Fasp`: starting and monitoring transfers. It can be considered as a FASPManager class for Ruby.
5276
- * `Aspera::Cli`: `ascli`.
5599
+ - `Aspera` generic classes for REST and OAuth
5600
+ - `Aspera::Fasp`: starting and monitoring transfers. It can be considered as a FASPManager class for Ruby.
5601
+ - `Aspera::Cli`: `ascli`.
5277
5602
 
5278
5603
  A working example can be found in the gem, example:
5279
5604
 
@@ -5302,23 +5627,23 @@ aoc.read('self')
5302
5627
 
5303
5628
  ## Changes (Release notes)
5304
5629
 
5305
- See <CHANGELOG.md>
5630
+ See [CHANGELOG.md](CHANGELOG.md)
5306
5631
 
5307
5632
  ## History
5308
5633
 
5309
- When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
5634
+ When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (`ascp`), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
5310
5635
 
5311
5636
  There were a few pitfalls:
5312
5637
 
5313
- * The tool was written in the aging `perl` language while most Aspera application products (but the Transfer Server) are written in `ruby`.
5314
- * The tool was only for transfers, but not able to call other products APIs
5638
+ - The tool was written in the aging `perl` language while most Aspera application products (but the Transfer Server) are written in `ruby`.
5639
+ - The tool was only for transfers, but not able to call other products APIs
5315
5640
 
5316
5641
  So, it evolved into `ascli`:
5317
5642
 
5318
- * portable: works on platforms supporting `ruby` (and `ascp`)
5319
- * easy to install with the `gem` utility
5320
- * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
5321
- * `ruby` is consistent with other Aspera products
5643
+ - portable: works on platforms supporting `ruby` (and `ascp`)
5644
+ - easy to install with the `gem` utility
5645
+ - supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from `ascp` command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
5646
+ - `ruby` is consistent with other Aspera products
5322
5647
 
5323
5648
  ## Common problems
5324
5649