aspera-cli 4.10.0 → 4.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (94) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/BUGS.md +20 -0
  4. data/CHANGELOG.md +509 -0
  5. data/CONTRIBUTING.md +118 -0
  6. data/README.md +621 -378
  7. data/bin/ascli +4 -4
  8. data/bin/asession +11 -11
  9. data/docs/test_env.conf +28 -19
  10. data/examples/aoc.rb +4 -4
  11. data/examples/dascli +11 -9
  12. data/examples/faspex4.rb +8 -8
  13. data/examples/node.rb +11 -11
  14. data/examples/server.rb +9 -9
  15. data/lib/aspera/aoc.rb +273 -266
  16. data/lib/aspera/ascmd.rb +56 -54
  17. data/lib/aspera/ats_api.rb +4 -4
  18. data/lib/aspera/cli/basic_auth_plugin.rb +15 -12
  19. data/lib/aspera/cli/extended_value.rb +5 -5
  20. data/lib/aspera/cli/formater.rb +64 -64
  21. data/lib/aspera/cli/listener/line_dump.rb +1 -1
  22. data/lib/aspera/cli/listener/logger.rb +1 -1
  23. data/lib/aspera/cli/listener/progress.rb +5 -6
  24. data/lib/aspera/cli/listener/progress_multi.rb +14 -19
  25. data/lib/aspera/cli/main.rb +66 -67
  26. data/lib/aspera/cli/manager.rb +110 -110
  27. data/lib/aspera/cli/plugin.rb +54 -37
  28. data/lib/aspera/cli/plugins/alee.rb +4 -4
  29. data/lib/aspera/cli/plugins/aoc.rb +308 -669
  30. data/lib/aspera/cli/plugins/ats.rb +44 -46
  31. data/lib/aspera/cli/plugins/bss.rb +10 -10
  32. data/lib/aspera/cli/plugins/config.rb +447 -344
  33. data/lib/aspera/cli/plugins/console.rb +12 -12
  34. data/lib/aspera/cli/plugins/cos.rb +18 -20
  35. data/lib/aspera/cli/plugins/faspex.rb +110 -112
  36. data/lib/aspera/cli/plugins/faspex5.rb +67 -46
  37. data/lib/aspera/cli/plugins/node.rb +364 -288
  38. data/lib/aspera/cli/plugins/orchestrator.rb +46 -46
  39. data/lib/aspera/cli/plugins/preview.rb +122 -114
  40. data/lib/aspera/cli/plugins/server.rb +137 -83
  41. data/lib/aspera/cli/plugins/shares.rb +30 -29
  42. data/lib/aspera/cli/plugins/sync.rb +13 -33
  43. data/lib/aspera/cli/transfer_agent.rb +57 -57
  44. data/lib/aspera/cli/version.rb +1 -1
  45. data/lib/aspera/colors.rb +3 -3
  46. data/lib/aspera/command_line_builder.rb +27 -27
  47. data/lib/aspera/cos_node.rb +22 -20
  48. data/lib/aspera/data_repository.rb +1 -1
  49. data/lib/aspera/environment.rb +30 -28
  50. data/lib/aspera/fasp/agent_base.rb +15 -15
  51. data/lib/aspera/fasp/agent_connect.rb +23 -21
  52. data/lib/aspera/fasp/agent_direct.rb +65 -67
  53. data/lib/aspera/fasp/agent_httpgw.rb +72 -68
  54. data/lib/aspera/fasp/agent_node.rb +23 -21
  55. data/lib/aspera/fasp/agent_trsdk.rb +20 -20
  56. data/lib/aspera/fasp/error.rb +3 -2
  57. data/lib/aspera/fasp/error_info.rb +11 -8
  58. data/lib/aspera/fasp/installation.rb +78 -78
  59. data/lib/aspera/fasp/listener.rb +1 -1
  60. data/lib/aspera/fasp/parameters.rb +75 -72
  61. data/lib/aspera/fasp/parameters.yaml +2 -2
  62. data/lib/aspera/fasp/resume_policy.rb +8 -8
  63. data/lib/aspera/fasp/transfer_spec.rb +35 -2
  64. data/lib/aspera/fasp/uri.rb +7 -7
  65. data/lib/aspera/faspex_gw.rb +7 -5
  66. data/lib/aspera/hash_ext.rb +3 -3
  67. data/lib/aspera/id_generator.rb +5 -5
  68. data/lib/aspera/keychain/encrypted_hash.rb +23 -28
  69. data/lib/aspera/keychain/macos_security.rb +21 -20
  70. data/lib/aspera/log.rb +7 -7
  71. data/lib/aspera/nagios.rb +19 -18
  72. data/lib/aspera/node.rb +209 -35
  73. data/lib/aspera/oauth.rb +37 -36
  74. data/lib/aspera/open_application.rb +19 -11
  75. data/lib/aspera/persistency_action_once.rb +4 -4
  76. data/lib/aspera/persistency_folder.rb +13 -13
  77. data/lib/aspera/preview/file_types.rb +8 -8
  78. data/lib/aspera/preview/generator.rb +67 -67
  79. data/lib/aspera/preview/utils.rb +27 -27
  80. data/lib/aspera/proxy_auto_config.js +41 -41
  81. data/lib/aspera/proxy_auto_config.rb +16 -16
  82. data/lib/aspera/rest.rb +56 -60
  83. data/lib/aspera/rest_call_error.rb +2 -1
  84. data/lib/aspera/rest_error_analyzer.rb +18 -17
  85. data/lib/aspera/rest_errors_aspera.rb +16 -16
  86. data/lib/aspera/secret_hider.rb +15 -13
  87. data/lib/aspera/ssh.rb +11 -10
  88. data/lib/aspera/sync.rb +158 -44
  89. data/lib/aspera/temp_file_manager.rb +2 -2
  90. data/lib/aspera/uri_reader.rb +4 -4
  91. data/lib/aspera/web_auth.rb +14 -13
  92. data.tar.gz.sig +0 -0
  93. metadata +8 -5
  94. metadata.gz.sig +0 -0
data/README.md CHANGED
@@ -1,10 +1,13 @@
1
1
  # Command Line Interface for IBM Aspera products
2
+ <!-- markdownlint-disable MD033 MD003 MD053 -->
2
3
 
3
4
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
4
5
 
5
- Version : 4.10.0
6
+ ##
6
7
 
7
- Laurent/2016-2022
8
+ Version : 4.11.0
9
+
10
+ Laurent/2016-2023
8
11
 
9
12
  This gem provides the `ascli` Command Line Interface to IBM Aspera software.
10
13
 
@@ -14,9 +17,11 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
14
17
 
15
18
  Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
16
19
 
17
- Minimum required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.7 in a future version.
20
+ Minimum required Ruby version: >= 2.4.
21
+
22
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
18
23
 
19
- [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
24
+ [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs&sort=title_asc)
20
25
  [Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
21
26
 
22
27
  Release notes: see [CHANGELOG.md](CHANGELOG.md)
@@ -46,7 +51,7 @@ So it is designed for:
46
51
  - A configuration file (config.yaml)
47
52
  - Advanced command line options
48
53
  - cURL (for REST calls)
49
- - Aspera transfer (ascp)
54
+ - Aspera transfer (`ascp`)
50
55
 
51
56
  If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
52
57
 
@@ -63,7 +68,6 @@ In examples, command line operations are shown using a shell such: `bash` or `zs
63
68
 
64
69
  Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
65
70
 
66
-
67
71
  ## Quick Start
68
72
 
69
73
  This section guides you from installation, first use and advanced use.
@@ -77,7 +81,7 @@ ascli --version
77
81
  ```
78
82
 
79
83
  ```bash
80
- 4.10.0
84
+ 4.11.0
81
85
  ```
82
86
 
83
87
  ### First use
@@ -127,7 +131,7 @@ ascli config preset set default server myserver
127
131
  ```
128
132
 
129
133
  ```output
130
- updated: default&rarr;server to myserver
134
+ updated: default &rarr; server to myserver
131
135
  ```
132
136
 
133
137
  ```bash
@@ -158,7 +162,7 @@ ascli server download /aspera-test-dir-large/200MB
158
162
  ```
159
163
 
160
164
  ```output
161
- Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
165
+ Time: 00:00:02 =========================================================== 100% 100 Mbps Time: 00:00:00
162
166
  complete
163
167
  ```
164
168
 
@@ -174,9 +178,13 @@ It is possible to install *either* directly on the host operating system (Linux,
174
178
 
175
179
  The direct installation is recommended and consists in installing:
176
180
 
177
- - [Ruby](#ruby) (version: >= 2.4. Deprecation notice: the minimum will be 2.7 in a future version)
181
+ - [Ruby](#ruby)
178
182
  - [aspera-cli](#the_gem)
179
- - [Aspera SDK (ascp)](#fasp_prot)
183
+ - [Aspera SDK (`ascp`)](#fasp_prot)
184
+
185
+ Ruby version: >= 2.4.
186
+
187
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
180
188
 
181
189
  The following sections provide information on the various installation methods.
182
190
 
@@ -184,33 +192,114 @@ An internet connection is required for the installation. If you don't have inter
184
192
 
185
193
  ### Docker container
186
194
 
187
- This method installs a docker image that contains: Ruby, `ascli` and the FASP SDK.
195
+ The image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
196
+ The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
197
+ To use the container, ensure that you have `docker` (or `podman`) installed.
188
198
 
189
- The image is: <https://hub.docker.com/r/martinlaurent/ascli>.
190
- It is built from this [Dockerfile](Dockerfile).
199
+ ```bash
200
+ docker --version
201
+ ```
191
202
 
192
- Ensure that you have `docker` (or `podman`) installed.
203
+ **Wanna start quickly ?** With an interactive shell ? Execute this:
193
204
 
194
205
  ```bash
195
- docker --version
206
+ docker run --tty --interactive --entrypoint bash martinlaurent/ascli:latest
207
+ ```
208
+
209
+ Then, execute individual `ascli` commands such as:
210
+
211
+ ```bash
212
+ ascli conf init
213
+ ascli conf preset overview
214
+ ascli conf ascp info
215
+ ascli server ls /
216
+ ```
217
+
218
+ That is simple, but there are limitations:
219
+
220
+ - Everything happens in the container
221
+ - Any generated file in the container will be lost on container (shell) exit. Including configuration files and downloaded files.
222
+ - No possibility to upload files located on the host system
223
+
224
+ The container image is built from this [Dockerfile](Dockerfile): the entry point is `ascli` and the default command is `help`.
225
+
226
+ The container can also be execute for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
227
+
228
+ ```bash
229
+ docker run --rm --tty --interactive martinlaurent/ascli:latest
230
+ ```
231
+
232
+ For more convenience, you may define a shell alias:
233
+
234
+ ```bash
235
+ alias ascli='docker run --rm --tty --interactive martinlaurent/ascli:latest'
236
+ ```
237
+
238
+ Then, you can execute the container like a local command:
239
+
240
+ ```bash
241
+ ascli -v
242
+ ```
243
+
244
+ ```text
245
+ 4.11.0
246
+ ```
247
+
248
+ In order to keep persistency of configuration on the host,
249
+ you should mount your user's config folder to the container.
250
+ To enable write access, a possibility is to run as `root` in the container (and set the default configuration folder to `/home/cliuser/.aspera/ascli`).
251
+ Add options:
252
+
253
+ ```bash
254
+ --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli
255
+ ```
256
+
257
+ > **Note:** if you are using a `podman machine`, e.g. on Macos , make sure that the folder is also shared between the VM and the host, so that sharing is: container &rarr; VM &rarr; Host: `podman machine init ... --volume="/Users:/Users"`
258
+
259
+ As shown in the quick start, if you prefer to keep a running container with a shell and `ascli` available,
260
+ you can change the entry point, add option:
261
+
262
+ ```bash
263
+ --entrypoint bash
264
+ ```
265
+
266
+ You may also probably want that files downloaded in the container are in fact placed on the host.
267
+ In this case you need also to mount the shared transfer folder:
268
+
269
+ ```bash
270
+ --volume $HOME/xferdir:/xferfiles
196
271
  ```
197
272
 
198
- Download the script [`dascli`](../examples/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
273
+ > **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
199
274
 
200
- > If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli conf gem path)/../examples/dascli ascli`
275
+ And if you want all the above, simply use all the options:
201
276
 
202
- Note that ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
277
+ ```bash
278
+ alias asclish="docker run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
279
+ ```
280
+
281
+ ```bash
282
+ export xferdir=$HOME/xferdir
283
+ mkdir -p $xferdir
284
+ chmod -R 777 $xferdir
285
+ mkdir -p $HOME/.aspera/ascli
286
+ asclish
287
+ ```
203
288
 
204
- Some environment variables can be set to alter the behaviour of the script:
289
+ A convenience sample script is also provided: download the script [`dascli`](../examples/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
205
290
 
206
- | env var | description | default | example |
207
- |-------------|------------------------------------|--------------------------|--------------------------|
208
- | ASCLI__HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascliconfig` |
209
- | docker_args | additional options to `docker` | &lt;empty&gt; | `--volume /Users:/Users` |
210
- | image | container image name | martinlaurent/ascli | |
211
- | version | container image version | latest | `4.8.0.pre` |
291
+ > **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli conf gem path)/../examples/dascli ascli`
212
292
 
213
- The wrapping script maps the folder `$ASCLI__HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
293
+ Some environment variables can be set for this script to adapt its behaviour:
294
+
295
+ | env var | description | default | example |
296
+ |--------------|------------------------------------|--------------------------|--------------------------|
297
+ | ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascliconfig` |
298
+ | docker_args | additional options to `docker` | &lt;empty&gt; | `--volume /Users:/Users` |
299
+ | image | container image name | martinlaurent/ascli | |
300
+ | version | container image version | latest | `4.8.0.pre` |
301
+
302
+ The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
214
303
  (value expected in the container).
215
304
  This allows having persistent configuration on the host.
216
305
 
@@ -221,21 +310,18 @@ Example of use:
221
310
  ```bash
222
311
  curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli
223
312
  chmod a+x ascli
224
- export ASCLI__HOME=$HOME/.ascliconf
225
- mkdir -p $ASCLI__HOME
226
- chmod -R 777 $ASCLI__HOME
227
313
  export xferdir=$HOME/xferdir
228
314
  mkdir -p $xferdir
229
315
  chmod -R 777 $xferdir
230
- export docker_args="--volume $xferdir:/xferfolder"
316
+ export docker_args="--volume $xferdir:/xferfiles"
231
317
 
232
318
  ./ascli conf init
233
319
 
234
- touch $xferdir/samplefile
235
- ./ascli server upload /xferfolder/samplefile --to-folder=/Upload
320
+ echo 'Local file to transfer' > $xferdir/samplefile.txt
321
+ ./ascli server upload /xferfiles/samplefile.txt --to-folder=/Upload
236
322
  ```
237
323
 
238
- > The local file (`samplefile`) is specified relative to storage view from container (`/xferfolder`) mapped to the host folder `$HOME/xferdir`
324
+ > **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
239
325
 
240
326
  ### <a id="ruby"></a>Ruby
241
327
 
@@ -243,7 +329,9 @@ Use this method to install on the native host.
243
329
 
244
330
  A ruby interpreter is required to run the tool or to use the gem and tool.
245
331
 
246
- Required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.7 in a future version.
332
+ Required Ruby version: >= 2.4.
333
+
334
+ > **Deprecation notice**: the minimum Ruby version will be 2.7 in a future version.
247
335
 
248
336
  *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
249
337
 
@@ -251,7 +339,7 @@ Refer to the following sections for a proposed method for specific operating sys
251
339
 
252
340
  The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, macOS, Windows with cygwin, etc...).
253
341
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
254
- If you have a simpler better way to install Ruby : use it ! (version: >= 2.4. Deprecation notice: the minimum will be 2.7 in a future version)
342
+ If you have a simpler better way to install Ruby : use it !
255
343
 
256
344
  #### Generic: RVM: single user installation (not root)
257
345
 
@@ -318,7 +406,7 @@ Install Latest stable Ruby:
318
406
  - Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
319
407
  - Execute the installer which installs by default in: `C:\RubyVV-x64` (VV is the version number)
320
408
  - At the end of the installation procedure, the Msys2 installer is automatically executed, select option 3 (msys and mingw)
321
- - for the installation of
409
+
322
410
  #### macOS: pre-installed or `brew`
323
411
 
324
412
  macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
@@ -337,15 +425,20 @@ brew install ruby
337
425
 
338
426
  If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
339
427
 
340
- Example: Centos 8 Stream
428
+ Example: RHEL 8 and 9: basic installation
429
+
430
+ ```bash
431
+ yum module install ruby:3.1
432
+ ```
433
+
434
+ Example: RHEL 8, Centos 8 Stream: with extensions to compile native gems
341
435
 
342
436
  ```bash
343
437
  yum install make automake gcc gcc-c++ kernel-devel
344
438
  yum install redhat-rpm-config
345
439
  dnf module reset ruby
346
- dnf module enable ruby:3.0
347
- dnf module -y install ruby:3.0/common
348
- gem install aspera-cli
440
+ dnf module enable ruby:3.1
441
+ dnf module -y install ruby:3.1/common
349
442
  ```
350
443
 
351
444
  Other examples:
@@ -398,7 +491,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
398
491
 
399
492
  <https://www.jruby.org/download>
400
493
 
401
- Note that using jruby the startup time is longer than the native ruby, but transfer speed is not impacted (executed by `ascp` binary).
494
+ > **Note:** Using jruby the startup time is longer than the native ruby, but the transfer speed is not impacted (executed by `ascp` binary).
402
495
 
403
496
  ### <a id="the_gem"></a>`aspera-cli` gem
404
497
 
@@ -427,7 +520,7 @@ ascli conf check_update
427
520
  Most file transfers will be done using the FASP protocol, using `ascp`.
428
521
  Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
429
522
 
430
- - ascp
523
+ - `ascp`
431
524
  - aspera-license (in same folder, or ../etc)
432
525
 
433
526
  This can be installed either be installing an Aspera transfer software, or using an embedded command:
@@ -467,7 +560,8 @@ Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, b
467
560
 
468
561
  ### <a id="offline_install"></a>Installation in air gapped environment
469
562
 
470
- Note that currently no pre-packaged version exist yet.
563
+ > **Note:** no pre-packaged version is provided.
564
+
471
565
  A method to build one is provided here:
472
566
 
473
567
  The procedure:
@@ -528,7 +622,7 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
528
622
  - Supports most Aspera server products (on-premise and SaaS)
529
623
  - Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
530
624
  - Supports Commands, Option values and Parameters shortcuts
531
- - FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
625
+ - FASP [Transfer Agents](#agents) can be: local `ascp`, or Connect Client, or any transfer node
532
626
  - Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
533
627
  - Allows transfers from products to products, essentially at node level (using the node transfer agent)
534
628
  - Supports FaspStream creation (using Node API)
@@ -551,7 +645,7 @@ Not all `ascli` features are fully documented here, the user may explore command
551
645
 
552
646
  If you want to use `ascp` directly as a command line, refer to IBM Aspera documentation of either [Desktop Client](https://www.ibm.com/docs/en/asdc), [Endpoint](https://www.ibm.com/docs/en/ahte) or [Transfer Server](https://www.ibm.com/docs/en/ahts) where [a section on `ascp` can be found](https://www.ibm.com/docs/en/ahts/4.4?topic=linux-ascp-transferring-from-command-line).
553
647
 
554
- Using `ascli` with plugin `server` for command line gives advantages over ascp:
648
+ Using `ascli` with plugin `server` for command line gives advantages over `ascp`:
555
649
 
556
650
  - automatic resume on error
557
651
  - configuration file
@@ -604,7 +698,8 @@ ERROR: Argument: unprocessed values: ["2", "3"]
604
698
  ```
605
699
 
606
700
  `config echo` displays the value of the first argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
607
- Note that it gets its value after shell command line parsing and `ascli` extended value parsing.
701
+
702
+ > **Note:** It gets its value after shell command line parsing and `ascli` extended value parsing.
608
703
 
609
704
  In the following examples (using a POSIX shell, such as `bash`), several sample commands are provided when equivalent.
610
705
  For all example, most of special character handling is not specific to `ascli`: It depoends on the underlying syntax: shell , JSON, etc...
@@ -613,7 +708,7 @@ Depending on the case, a different `format` is used to display the actual value.
613
708
  For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
614
709
 
615
710
  Double quotes are processed by the shell to create a single string argument.
616
- For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash.
711
+ For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash. <!-- markdownlint-disable-line -->
617
712
 
618
713
  ```bash
619
714
  ascli conf echo "Hello World" --format=text
@@ -628,8 +723,9 @@ Hello World
628
723
  #### Using a shell variable, parsed by shell, in an extended value
629
724
 
630
725
  To be evaluated by shell, the shell variable must not be in single quotes.
631
- Note we use a simple variable here: the variable is not necessarily an environment variable.
632
- Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
726
+ Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
727
+
728
+ > **Note:** we use a simple variable here: the variable is not necessarily an environment variable.
633
729
 
634
730
  ```bash
635
731
  MYVAR="Hello World"
@@ -787,7 +883,7 @@ ascli config echo -- --sample
787
883
  "--sample"
788
884
  ```
789
885
 
790
- Note that here, `--sample` is taken as an argument, and not as an option, due to `--`.
886
+ > **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
791
887
 
792
888
  Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
793
889
 
@@ -873,7 +969,7 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
873
969
  :...............................:..................................:...........:
874
970
  ```
875
971
 
876
- Note that `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
972
+ > **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
877
973
 
878
974
  #### Verbosity of output
879
975
 
@@ -923,7 +1019,7 @@ The following "readers" are supported (returns value in []):
923
1019
  - @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
924
1020
  - @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
925
1021
  - @stdin: : [String] read from stdin (no value on right)
926
- - @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
1022
+ - @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
927
1023
 
928
1024
  In addition it is possible to decode a value, using one or multiple decoders :
929
1025
 
@@ -985,7 +1081,7 @@ ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
985
1081
  {"version"=>"0.9", "hello"=>true}
986
1082
  ```
987
1083
 
988
- Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
1084
+ > **Note:** `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` are equivalent to: `@preset:config`
989
1085
 
990
1086
  ### <a id="native"></a>Structured Value
991
1087
 
@@ -1097,6 +1193,8 @@ If necessary, the configuration file can opened in a text editor with:
1097
1193
  ascli config open
1098
1194
  ```
1099
1195
 
1196
+ > **Note:** this starts the editor specified by env var `EDITOR` if defined.
1197
+
1100
1198
  Older format for commands are still supported:
1101
1199
 
1102
1200
  ```bash
@@ -1115,7 +1213,7 @@ This preset name is reserved and contains an array of key-value , where the key
1115
1213
 
1116
1214
  When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option --no-default (or -N) is used.
1117
1215
 
1118
- Note that special plugin name: `config` can be associated with a preset that is loaded initially, typically used for default values.
1216
+ > **Note:** Special plugin name: `config` can be associated with a preset that is loaded initially, typically used for default values.
1119
1217
 
1120
1218
  Operations on this preset are done using regular `config` operations:
1121
1219
 
@@ -1131,9 +1229,9 @@ ascli config preset get default _plugin_name_
1131
1229
  "_default_preset_for_plugin_"
1132
1230
  ```
1133
1231
 
1134
- #### <a id="lplugconf"></a>Special Plugin: config
1232
+ #### <a id="config"></a>Plugin: `config`: CLI Configuration
1135
1233
 
1136
- Plugin `config` (not to be confused with Option preset config) is used to configure `ascli` but it also contains global options.
1234
+ Plugin `config` is used to configure `ascli` and also contains global options.
1137
1235
 
1138
1236
  When `ascli` starts, it looks for the `default` Option preset and if there is a value for `config`, if so, it loads the option values for any plugin used.
1139
1237
 
@@ -1347,7 +1445,7 @@ The vault is used with options `vault` and `vault_password`.
1347
1445
 
1348
1446
  `vault_password` specifies the password for the vault.
1349
1447
  Although it can be specified on command line, for security reason you can hide the value.
1350
- For example it can be specified on command line like this:
1448
+ For example it can be securely specified on command line like this:
1351
1449
 
1352
1450
  ```bash
1353
1451
  export ASCLI_VAULT_PASSWORD
@@ -1356,7 +1454,7 @@ read -s ASCLI_VAULT_PASSWORD
1356
1454
 
1357
1455
  #### Vault: System keychain
1358
1456
 
1359
- > **macOS only**
1457
+ > **Note:** **macOS only**
1360
1458
 
1361
1459
  It is possible to manage secrets in macOS keychain (only read supported currently).
1362
1460
 
@@ -1372,7 +1470,7 @@ It is possible to store and use secrets encrypted in a file.
1372
1470
  --vault=@json:'{"type":"file","name":"vault.bin"}'
1373
1471
  ```
1374
1472
 
1375
- `name` is the file path, absolute or relative to the config folder `ASCLI__HOME`.
1473
+ `name` is the file path, absolute or relative to the config folder `ASCLI_HOME`.
1376
1474
 
1377
1475
  #### Vault: Operations
1378
1476
 
@@ -1395,6 +1493,24 @@ When a secret is needed by a sub command, the command can search for existing co
1395
1493
 
1396
1494
  The lookup is done by comparing the service URL and username (or access key).
1397
1495
 
1496
+ #### Securing passwords and secrets
1497
+
1498
+ A passwords can be saved in clear in a [option preset](#lprt) together with other account information (URL, username, etc...).
1499
+ Example:
1500
+
1501
+ ```bash
1502
+ `ascli` conf preset update myconf --url=... --username=... --password=...
1503
+ ```
1504
+
1505
+ For a more secure storage one can do:
1506
+
1507
+ ```bash
1508
+ `ascli` conf preset update myconf --url=... --username=... --password=@val:@vault:myconf.password
1509
+ `ascli` conf vault create myconf @json:'{"password":"__value_here__"}'
1510
+ ```
1511
+
1512
+ > **Note:** use `@val:` in front of `@vault:` so that the extended value is not evaluated.
1513
+
1398
1514
  ### <a id="private_key"></a>Private Key
1399
1515
 
1400
1516
  Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5, ...).
@@ -1545,7 +1661,7 @@ Available loggers: `stdout`, `stderr`, `syslog`.
1545
1661
 
1546
1662
  Available levels: `debug`, `info`, `warn`, `error`.
1547
1663
 
1548
- Note that when using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` option `EX_ascp_args`, see [`direct`](#agt_direct).
1664
+ > **Note:** When using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` option `EX_ascp_args`, see [`direct`](#agt_direct).
1549
1665
 
1550
1666
  Examples:
1551
1667
 
@@ -1625,7 +1741,7 @@ There are two possibilities to define an HTTP proxy to be used when Ruby HTTP is
1625
1741
  The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy, e.g. `http://myproxy.org.net:3128`.
1626
1742
  Refer to [Ruby findproxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
1627
1743
 
1628
- > Note: Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
1744
+ > **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
1629
1745
 
1630
1746
  ```bash
1631
1747
  export http_proxy=http://proxy.example.com:3128
@@ -1689,7 +1805,7 @@ To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) para
1689
1805
 
1690
1806
  The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
1691
1807
 
1692
- - `show` : shows the path of ascp used
1808
+ - `show` : shows the path of `ascp` used
1693
1809
  - `use` : list,download connect client versions available on internet
1694
1810
  - `products` : list Aspera transfer products available locally
1695
1811
  - `connect` : list,download connect client versions available on internet
@@ -1718,9 +1834,9 @@ ascli config ascp info
1718
1834
 
1719
1835
  #### Selection of `ascp` location for [`direct`](#agt_direct) agent
1720
1836
 
1721
- By default, `ascli` uses any found local product with ascp, including SDK.
1837
+ By default, `ascli` uses any found local product with `ascp`, including SDK.
1722
1838
 
1723
- To temporarily use an alternate ascp path use option `ascp_path` (`--ascp-path=`)
1839
+ To temporarily use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
1724
1840
 
1725
1841
  For a permanent change, the command `config ascp use` sets the same parameter for the global default.
1726
1842
 
@@ -1771,11 +1887,11 @@ ascli config ascp products list
1771
1887
 
1772
1888
  #### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
1773
1889
 
1774
- If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
1890
+ If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
1775
1891
 
1776
- Using the option use_product finds the ascp binary of the selected product.
1892
+ Using the option use_product finds the `ascp` binary of the selected product.
1777
1893
 
1778
- To permanently use the ascp of a product:
1894
+ To permanently use the `ascp` of a product:
1779
1895
 
1780
1896
  ```bash
1781
1897
  ascli config ascp products use 'Aspera Connect'
@@ -1821,7 +1937,7 @@ ascli config ascp connect version 'Aspera Connect for Mac Intel' download enclos
1821
1937
  ```
1822
1938
 
1823
1939
  ```output
1824
- Time: 00:00:02 ======================================================================= 100% 27766 KB/sec Time: 00:00:02
1940
+ Time: 00:00:02 =========================================================== 100% 27766 KB/sec Time: 00:00:02
1825
1941
  Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
1826
1942
  ```
1827
1943
 
@@ -1840,15 +1956,15 @@ There are currently 3 agents:
1840
1956
  - [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
1841
1957
  - [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
1842
1958
 
1843
- Note that all transfer operation are seen from the point of view of the agent.
1844
- For instance, a node agent making an "upload", or "package send" operation,
1959
+ > **Note:** All transfer operations are seen from the point of view of the agent.
1960
+ For example, a node agent executing an "upload", or "package send" operation
1845
1961
  will effectively push files to the related server from the agent node.
1846
1962
 
1847
- `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *raw* ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1963
+ `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *native* `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1848
1964
 
1849
1965
  #### <a id="agt_direct"></a>Direct
1850
1966
 
1851
- The `direct` agent directly executes a local ascp.
1967
+ The `direct` agent directly executes a local `ascp`.
1852
1968
  This is the default agent for `ascli`.
1853
1969
  This is equivalent to option `--transfer=direct`.
1854
1970
  `ascli` will detect locally installed Aspera products, including SDK, and use `ascp` from that component.
@@ -1856,18 +1972,17 @@ Refer to section [FASP](#client).
1856
1972
 
1857
1973
  The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
1858
1974
 
1859
- <table>
1860
- <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1861
- <tr><td>wss</td><td>Bool</td><td>Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: false</td></tr>
1862
- <tr><td>spawn_timeout_sec</td><td>Float</td><td>Multi session<br/>Verification time that ascp is running<br/>Default: 3</td></tr>
1863
- <tr><td>spawn_delay_sec</td><td>Float</td><td>Multi session<br/>Delay between startup of sessions<br/>Default: 2</td></tr>
1864
- <tr><td>multi_incr_udp</td><td>Bool</td><td>Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true</td></tr>
1865
- <tr><td>resume</td><td>Hash</td><td>Resume<br/>parameters<br/>See below</td></tr>
1866
- <tr><td>resume.iter_max</td><td>int</td><td>Resume<br/>Max number of retry on error<br/>Default: 7</td></tr>
1867
- <tr><td>resume.sleep_initial</td><td>int</td><td>Resume<br/>First Sleep before retry<br/>Default: 2</td></tr>
1868
- <tr><td>resume.sleep_factor</td><td>int</td><td>Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2</td></tr>
1869
- <tr><td>resume.sleep_max</td><td>int</td><td>Resume<br/>Default: 60</td></tr>
1870
- </table>
1975
+ | Name | Type | Description |
1976
+ |----------------------|-------|-------------|
1977
+ | wss | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
1978
+ | spawn_timeout_sec | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
1979
+ | spawn_delay_sec | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
1980
+ | multi_incr_udp | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
1981
+ | resume | Hash | Resume<br/>parameters<br/>See below |
1982
+ | resume.iter_max | int | Resume<br/>Max number of retry on error<br/>Default: 7 |
1983
+ | resume.sleep_initial | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
1984
+ | resume.sleep_factor | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
1985
+ | resume.sleep_max | int | Resume<br/>Default: 60 |
1871
1986
 
1872
1987
  In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
1873
1988
  Sleep between iterations is:
@@ -1890,15 +2005,15 @@ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":20}}'
1890
2005
  ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1891
2006
  ```
1892
2007
 
1893
- Note that the `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
2008
+ > **Note:** The `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
1894
2009
  In particular the field, `EX_ascp_args` which is a list of additional command line options to `ascp`.
1895
2010
 
1896
- This can be useful to activate logging using option `-L` of ascp.
2011
+ This can be useful to activate logging using option `-L` of `ascp`.
1897
2012
  For example the option `--ts=@json:'{"EX_ascp_args":["-DDL-"]}'` will activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`).
1898
2013
  This is useful if the transfer fails.
1899
- To store ascp logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
2014
+ To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
1900
2015
 
1901
- > Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
2016
+ > **Note:** Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
1902
2017
 
1903
2018
  In addition to standard methods described in section [File List](#file_list), it is possible to specify the list of file using those additional methods:
1904
2019
 
@@ -1914,11 +2029,22 @@ In addition to standard methods described in section [File List](#file_list), it
1914
2029
  --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","myfilelist"]}'
1915
2030
  ```
1916
2031
 
1917
- > File lists is shown here, there are also similar options for file pair lists.
2032
+ > **Note:** File lists is shown here, there are also similar options for file pair lists.
2033
+ >
2034
+ > **Note:** Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
2035
+ >
2036
+ > **Note:** Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
1918
2037
 
1919
- > Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
2038
+ In addition to special transfer spec parameter `EX_ascp_args`, it is possible to provide the same `ascp` options using option `ascp_opts` of `ascli`.
2039
+ This option expects an `Array`, which can be conveniently provided with extended syntax `@list:`
2040
+
2041
+ ```bash
2042
+ ascli server download /aspera-test-dir-large/200MB --ascp-opts=@list:' -l 10m -k 3'
2043
+ ```
1920
2044
 
1921
- > Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
2045
+ > **Note:** Using `@list:`, the use of quotes and leading character "space" here is important: The `@list:` expects a single string which must not be parsed by the shell (so protected with quotes) and the leading space is the separator character.
2046
+ >
2047
+ > **Note:** The option `ascp_opts` are appended to `EX_ascp_args` if present.
1922
2048
 
1923
2049
  #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1924
2050
 
@@ -1930,13 +2056,12 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
1930
2056
  Transfer Server using the Node API, either on a local or remote node.
1931
2057
  Parameters provided in option `transfer_info` are:
1932
2058
 
1933
- <table>
1934
- <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1935
- <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
1936
- <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
1937
- <tr><td>password</td><td>string</td><td>password, secret or bearer token</br>Mandatory</td></tr>
1938
- <tr><td>root_id</td><td>string</td><td>password or secret</br>Mandatory only for bearer token</td></tr>
1939
- </table>
2059
+ | Name | Type | Description |
2060
+ |------|------|-------------|
2061
+ | url | string | URL of the node API</br>Mandatory |
2062
+ | username | string | node api user or access key</br>Mandatory |
2063
+ | password | string | password, secret or bearer token</br>Mandatory |
2064
+ | root_id | string | password or secret</br>Mandatory only for bearer token |
1940
2065
 
1941
2066
  Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
1942
2067
  `--transfer-info=@preset:<psetname>` or be specified using the extended value syntax :
@@ -1952,12 +2077,11 @@ If it possible to send using a HTTP gateway, in case FASP is not allowed.
1952
2077
 
1953
2078
  Parameters provided in option `transfer_info` are:
1954
2079
 
1955
- <table>
1956
- <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1957
- <tr><td>url</td><td>string</td><td>URL of the HTTP GW</br>Mandatory</td></tr>
1958
- <tr><td>upload_bar_refresh_sec</td><td>float</td><td>Refresh rate for upload progress bar</td></tr>
1959
- <tr><td>upload_chunksize</td><td>int</td><td>Size in bytes of chunks for upload</td></tr>
1960
- </table>
2080
+ | Name | Type | Description |
2081
+ |------|------|-------------|
2082
+ | url | string | URL of the HTTP GW</br>Mandatory |
2083
+ | upload_bar_refresh_sec | float | Refresh rate for upload progress bar |
2084
+ | upload_chunksize | int | Size in bytes of chunks for upload |
1961
2085
 
1962
2086
  Example:
1963
2087
 
@@ -1965,7 +2089,7 @@ Example:
1965
2089
  ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
1966
2090
  ```
1967
2091
 
1968
- > The gateway only supports transfers authorized with a token.
2092
+ > **Note:** The gateway only supports transfers authorized with a token.
1969
2093
 
1970
2094
  #### <a id="agt_trsdk"></a>Transfer SDK
1971
2095
 
@@ -2002,7 +2126,7 @@ is described in a [*transfer-spec*](#transferspec) (Transfer Specification), suc
2002
2126
 
2003
2127
  If needed, it is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters. Multiple `ts` options on command line are cumulative.
2004
2128
 
2005
- It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
2129
+ It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for `ascp` command line parameters not supported yet in the transfer spec.
2006
2130
 
2007
2131
  The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has the advantage of:
2008
2132
 
@@ -2018,8 +2142,8 @@ All standard [*transfer-spec*](#transferspec) parameters can be specified.
2018
2142
 
2019
2143
  References:
2020
2144
 
2021
- - [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22)&rarr;/opt/transfers
2022
- - [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22)&rarr;Guides&rarr;API Ref&rarr;Transfer Spec V1
2145
+ - [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22) &rarr; /opt/transfers
2146
+ - [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22) &rarr; Guides &rarr; API Ref &rarr; Transfer Spec V1
2023
2147
  - [Aspera Connect SDK](https://d3gcli72yxqn2z.cloudfront.net/connect/v4/asperaweb-4.js) &rarr; search `The parameters for starting a transfer.`
2024
2148
 
2025
2149
  Parameters can be displayed with commands:
@@ -2039,91 +2163,91 @@ Columns:
2039
2163
 
2040
2164
  Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
2041
2165
 
2042
- <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr>
2043
- <tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr>
2044
- <tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
2045
- <tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr>
2046
- <tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr>
2047
- <tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x {string})</td></tr>
2048
- <tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j {int})</td></tr>
2049
- <tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr>
2050
- <tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr>
2051
- <tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr>
2052
- <tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr>
2053
- <tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i {array})</td></tr>
2054
- <tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr>
2055
- <tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr>
2056
- <tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum})</td></tr>
2057
- <tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr>
2058
- <tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum})</td></tr>
2059
- <tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
2060
- <tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr>
2061
- <tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr>
2062
- <tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer)</td></tr>
2063
- <tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2064
- <tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr>
2065
- <tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z {int})</td></tr>
2066
- <tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum})</td></tr>
2067
- <tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than {int})</td></tr>
2068
- <tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than {int})</td></tr>
2069
- <tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O {int})</td></tr>
2070
- <tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr>
2071
- <tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string})</td></tr>
2072
- <tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port when no cipher is used<br/>(-t {int})</td></tr>
2073
- <tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t {int})</td></tr>
2074
- <tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2075
- <tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2076
- <tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2077
- <tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2078
- <tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2079
- <tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2080
- <tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m {int})</td></tr>
2081
- <tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string})</td></tr>
2082
- <tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/></td></tr>
2083
- <tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int})</td></tr>
2084
- <tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum})</td></tr>
2085
- <tr><td>password</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.</td></tr>
2086
- <tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr>
2087
- <tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr>
2088
- <tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr>
2089
- <tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum})</td></tr>
2090
- <tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr>
2091
- <tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr>
2092
- <tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr>
2093
- <tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr>
2094
- <tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum})</td></tr>
2095
- <tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr>
2096
- <tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr>
2097
- <tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string})</td></tr>
2098
- <tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum})</td></tr>
2099
- <tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr>
2100
- <tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host {string})</td></tr>
2101
- <tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr>
2102
- <tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user {string})</td></tr>
2103
- <tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2104
- <tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr>
2105
- <tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr>
2106
- <tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr>
2107
- <tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum})</td></tr>
2108
- <tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr>
2109
- <tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string})</td></tr>
2110
- <tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr>
2111
- <tr><td>src_base</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string})</td></tr>
2112
- <tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int})</td></tr>
2113
- <tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr>
2114
- <tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr>
2115
- <tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp {string})</td></tr>
2116
- <tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum})</td></tr>
2117
- <tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64 (conversion){hash})</td></tr>
2118
- <tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr>
2119
- <tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l {int})</td></tr>
2120
- <tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2121
- <tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr>
2122
- <tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr>
2123
- <tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr>
2124
- <tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2125
- <tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr>
2126
- </table>
2166
+ | Field | Type | D | N | C | Description |
2167
+ |-------|------|---|---|---|-------------|
2168
+ | EX_ascp_args | array | Y | &nbsp; | &nbsp; | Add native command line arguments to ascp |
2169
+ | EX_at_rest_password | string | Y | &nbsp; | &nbsp; | DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS) |
2170
+ | EX_file_list | string | Y | &nbsp; | &nbsp; | source file list |
2171
+ | EX_file_pair_list | string | Y | &nbsp; | &nbsp; | source file pair list |
2172
+ | EX_http_proxy_url | string | Y | &nbsp; | &nbsp; | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
2173
+ | EX_http_transfer_jpeg | int | Y | &nbsp; | &nbsp; | HTTP transfers as JPEG file<br/>(-j {int}) |
2174
+ | EX_license_text | string | Y | &nbsp; | &nbsp; | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
2175
+ | EX_no_read | bool | Y | &nbsp; | &nbsp; | no read source<br/>(--no-read) |
2176
+ | EX_no_write | bool | Y | &nbsp; | &nbsp; | no write on destination<br/>(--no-write) |
2177
+ | EX_proxy_password | string | Y | &nbsp; | &nbsp; | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS) |
2178
+ | EX_ssh_key_paths | array | Y | &nbsp; | &nbsp; | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
2179
+ | apply_local_docroot | bool | Y | &nbsp; | &nbsp; | (--apply-local-docroot) |
2180
+ | authentication | string | &nbsp; | &nbsp; | Y | value=token for SSH bypass keys, else password asked if not provided. |
2181
+ | cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
2182
+ | cipher_allowed | string | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none". |
2183
+ | content_protection | string | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
2184
+ | content_protection_password | string | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
2185
+ | cookie | string | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
2186
+ | create_dir | bool | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
2187
+ | delete_before_transfer | bool | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
2188
+ | delete_source | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2189
+ | destination_root | string | Y | Y | Y | Destination root directory. |
2190
+ | dgram_size | int | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
2191
+ | direction | string | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
2192
+ | exclude_newer_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
2193
+ | exclude_older_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
2194
+ | fasp_port | int | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
2195
+ | file_checksum | string | Y | Y | &nbsp; | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none |
2196
+ | http_fallback | bool<br/>string | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
2197
+ | http_fallback_port | int | Y | &nbsp; | &nbsp; | Specifies http port when no cipher is used<br/>(-t {int}) |
2198
+ | https_fallback_port | int | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
2199
+ | lock_min_rate | bool | Y | Y | Y | &nbsp; |
2200
+ | lock_min_rate_kbps | bool | Y | Y | Y | &nbsp; |
2201
+ | lock_rate_policy | bool | Y | Y | Y | &nbsp; |
2202
+ | lock_target_rate | bool | Y | Y | Y | &nbsp; |
2203
+ | lock_target_rate_kbps | bool | Y | Y | Y | &nbsp; |
2204
+ | min_rate_cap_kbps | int | Y | Y | Y | &nbsp; |
2205
+ | min_rate_kbps | int | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
2206
+ | move_after_transfer | string | Y | Y | &nbsp; | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
2207
+ | multi_session | int | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/> |
2208
+ | multi_session_threshold | int | Y | Y | &nbsp; | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
2209
+ | overwrite | string | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
2210
+ | password | string | &nbsp; | Y | &nbsp; | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only. |
2211
+ | paths | array | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional). |
2212
+ | precalculate_job_size | bool | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
2213
+ | preserve_access_time | bool | Y | Y | Y | (--preserve-access-time) |
2214
+ | preserve_acls | string | Y | &nbsp; | &nbsp; | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
2215
+ | preserve_creation_time | bool | Y | Y | Y | (--preserve-creation-time) |
2216
+ | preserve_file_owner_gid | bool | Y | &nbsp; | &nbsp; | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
2217
+ | preserve_file_owner_uid | bool | Y | &nbsp; | &nbsp; | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
2218
+ | preserve_modification_time | bool | Y | Y | Y | (--preserve-modification-time) |
2219
+ | preserve_remote_acls | string | Y | &nbsp; | &nbsp; | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum}) |
2220
+ | preserve_source_access_time | bool | Y | &nbsp; | &nbsp; | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
2221
+ | preserve_times | bool | Y | Y | Y | (--preserve-times) |
2222
+ | proxy | string | Y | &nbsp; | &nbsp; | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
2223
+ | rate_policy | string | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
2224
+ | rate_policy_allowed | string | &nbsp; | &nbsp; | Y | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed |
2225
+ | remote_host | string | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
2226
+ | remote_password | string | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
2227
+ | remote_user | string | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
2228
+ | remove_after_transfer | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2229
+ | remove_empty_directories | bool | Y | Y | &nbsp; | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
2230
+ | remove_empty_source_directory | bool | Y | &nbsp; | &nbsp; | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
2231
+ | remove_skipped | bool | Y | Y | Y | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
2232
+ | resume_policy | string | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
2233
+ | retry_duration | string<br/>int | &nbsp; | Y | Y | Specifies how long to wait before retrying transfer. (e.g. "5min") |
2234
+ | source_root | string | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
2235
+ | source_root_id | string | &nbsp; | Y | &nbsp; | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
2236
+ | src_base | string | Y | Y | &nbsp; | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
2237
+ | ssh_port | int | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
2238
+ | ssh_private_key | string | Y | &nbsp; | &nbsp; | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----&sol;nMII...<br/>Note the JSON encoding: &sol;n for newlines.<br/>(env:ASPERA_SCP_KEY) |
2239
+ | ssh_private_key_passphrase | string | Y | &nbsp; | &nbsp; | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
2240
+ | sshfp | string | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
2241
+ | symlink_policy | string | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
2242
+ | tags | hash | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
2243
+ | target_rate_cap_kbps | int | &nbsp; | &nbsp; | Y | Returned by upload/download_setup node API. |
2244
+ | target_rate_kbps | int | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
2245
+ | target_rate_percentage | string | Y | Y | Y | &nbsp; |
2246
+ | title | string | &nbsp; | Y | Y | Title of the transfer |
2247
+ | token | string | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
2248
+ | use_ascp4 | bool | Y | Y | &nbsp; | specify version of protocol |
2249
+ | wss_enabled | bool | Y | Y | Y | &nbsp; |
2250
+ | wss_port | int | Y | Y | Y | TCP port used for websocket service feed. |
2127
2251
 
2128
2252
  #### Destination folder for transfers
2129
2253
 
@@ -2167,7 +2291,7 @@ So, by default, the list of files to transfer will be simply specified on the co
2167
2291
 
2168
2292
  - an [Extended Value](#extended) with type **Array of String**
2169
2293
 
2170
- > Note: extended values can be tested with the command `conf echo`
2294
+ > **Note:** extended values can be tested with the command `conf echo`
2171
2295
 
2172
2296
  Examples:
2173
2297
 
@@ -2215,7 +2339,7 @@ Examples:
2215
2339
 
2216
2340
  The option `src_type` allows specifying if the list specified in option `sources` is a simple file list or if it is a file pair list.
2217
2341
 
2218
- > Note: Option `src_type` is not used if option `sources` is set to `@ts`
2342
+ > **Note:** Option `src_type` is not used if option `sources` is set to `@ts`
2219
2343
 
2220
2344
  Supported values for `src_type` are:
2221
2345
 
@@ -2228,11 +2352,11 @@ Example: Source file `200KB.1` is renamed `sample1` on destination:
2228
2352
  ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
2229
2353
  ```
2230
2354
 
2231
- > Note there are some specific rules to specify file list when using "Aspera on Cloud", refer to the AoC plugin section.
2355
+ > **Note:** There are some specific rules to specify a file list when using **Aspera on Cloud**, refer to the AoC plugin section.
2232
2356
 
2233
2357
  #### <a id="multisession"></a>Support of multi-session
2234
2358
 
2235
- Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
2359
+ Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on "direct" and "node" agents, not yet on connect.
2236
2360
 
2237
2361
  - when agent=node :
2238
2362
 
@@ -2271,7 +2395,7 @@ Example: parameter to download a faspex package and decrypt on the fly
2271
2395
  --ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
2272
2396
  ```
2273
2397
 
2274
- Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
2398
+ > **Note:** Up to version `ascli` 4.6.0, the following parameters should be used for agent `direct`:
2275
2399
 
2276
2400
  ```javascript
2277
2401
  --ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
@@ -2303,9 +2427,13 @@ Note that up to version 4.6.0, the following parameters should be used for agent
2303
2427
  --ts=@json:'{"precalculate_job_size":true}'
2304
2428
  ```
2305
2429
 
2306
- ### <a id="scheduling"></a>Lock for exclusive execution
2430
+ ### <a id="scheduling"></a>Scheduling
2431
+
2432
+ It is useful to configure automated scheduled execution.
2433
+
2434
+ #### <a id="locking"></a>Locking for exclusive execution
2307
2435
 
2308
- In some conditions, it may be desirable to ensure that `ascli` is not executed several times in parallel.
2436
+ It is also useful to ensure that `ascli` is not executed several times in parallel.
2309
2437
 
2310
2438
  For instance when `ascli` is executed automatically on a schedule basis, one generally desire that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
2311
2439
 
@@ -2339,6 +2467,18 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
2339
2467
  WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
2340
2468
  ```
2341
2469
 
2470
+ #### <a id="scheduler"></a>Scheduler
2471
+
2472
+ `ascli` does not provide an internal scheduler.
2473
+
2474
+ Instead, use the service provided by the Operating system:
2475
+
2476
+ - Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
2477
+ - Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
2478
+ - etc...
2479
+
2480
+ Linux also provides `anacron`, if tasks are hourly or daily.
2481
+
2342
2482
  ### "Proven&ccedil;ale"
2343
2483
 
2344
2484
  `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
@@ -2465,7 +2605,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
2465
2605
  ```text
2466
2606
  ascli -h
2467
2607
  NAME
2468
- ascli -- a command line tool for Aspera Applications (v4.10.0)
2608
+ ascli -- a command line tool for Aspera Applications (v4.11.0)
2469
2609
 
2470
2610
  SYNOPSIS
2471
2611
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -2515,7 +2655,6 @@ OPTIONS: global
2515
2655
  --log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
2516
2656
  --logger=ENUM log method: [stderr], stdout, syslog
2517
2657
  --lock-port=VALUE prevent dual execution of a command, e.g. in cron
2518
- --query=VALUE additional filter for API calls (extended value) (some commands)
2519
2658
  --http-options=VALUE options for http socket (extended value)
2520
2659
  --insecure=ENUM do not validate HTTPS certificate: no, [yes]
2521
2660
  --once-only=ENUM process only new items (some commands): [no], yes
@@ -2523,8 +2662,9 @@ OPTIONS: global
2523
2662
  --cache-tokens=ENUM save and reuse Oauth tokens: no, [yes]
2524
2663
 
2525
2664
  COMMAND: config
2526
- SUBCOMMANDS: list overview id preset open documentation genkey gem plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
2665
+ SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test export_to_cli file flush_tokens folder gem genkey id initdemo list lookup open overview plugin preset proxy_check secure smtp_settings vault wizard
2527
2666
  OPTIONS:
2667
+ --query=VALUE additional filter for API calls (extended value) (some commands)
2528
2668
  --value=VALUE extended value for create, update, list filter
2529
2669
  --property=VALUE name of property to set
2530
2670
  --id=VALUE resource identifier (modify,delete,show)
@@ -2552,18 +2692,18 @@ OPTIONS:
2552
2692
  --notif-template=VALUE Email ERB template for notification of transfers
2553
2693
  --version-check-days=VALUE Period in days to check new version (zero to disable)
2554
2694
  --plugin-folder=VALUE Folder where to find additional plugins
2555
- --ts=VALUE override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
2556
- --local-resume=VALUE set resume policy (Hash, use @json: prefix), current=
2557
- --to-folder=VALUE destination folder for transfered files
2558
- --sources=VALUE how list of transfered files is provided (@args,@ts,Array)
2559
- --src-type=ENUM type of file list: list, pair
2560
- --transfer=ENUM type of transfer agent: direct, node, connect, httpgw, trsdk
2561
- --transfer-info=VALUE parameters for transfer agent
2562
- --progress=ENUM type of progress bar: none, native, multi
2695
+ --ts=VALUE Override transfer spec values (Hash, e.g. use @json: prefix), current={"create_dir"=>true}
2696
+ --to-folder=VALUE Destination folder for transfered files
2697
+ --sources=VALUE How list of transfered files is provided (@args,@ts,Array)
2698
+ --ascp-opts=VALUE Options for ascp in its native format
2699
+ --src-type=ENUM Type of file list: list, pair
2700
+ --transfer=ENUM Type of transfer agent: direct, node, connect, httpgw, trsdk
2701
+ --transfer-info=VALUE Parameters for transfer agent
2702
+ --progress=ENUM Type of progress bar: none, native, multi
2563
2703
 
2564
2704
 
2565
2705
  COMMAND: shares
2566
- SUBCOMMANDS: health repository admin
2706
+ SUBCOMMANDS: admin health repository
2567
2707
  OPTIONS:
2568
2708
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2569
2709
  --username=VALUE username to log in
@@ -2571,7 +2711,7 @@ OPTIONS:
2571
2711
 
2572
2712
 
2573
2713
  COMMAND: node
2574
- SUBCOMMANDS: postprocess stream transfer cleanup forward access_key watch_folder service async sync central asperabrowser basic_token browse upload download api_details health events space info license mkdir mklink mkfile rename delete search
2714
+ SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
2575
2715
  OPTIONS:
2576
2716
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2577
2717
  --username=VALUE username to log in
@@ -2579,11 +2719,13 @@ OPTIONS:
2579
2719
  --validator=VALUE identifier of validator (optional for central)
2580
2720
  --asperabrowserurl=VALUE URL for simple aspera web ui
2581
2721
  --sync-name=VALUE sync name
2722
+ --path=VALUE file or folder path for gen4 operation "file"
2582
2723
  --token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
2724
+ --default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
2583
2725
 
2584
2726
 
2585
2727
  COMMAND: orchestrator
2586
- SUBCOMMANDS: health info workflow plugins processes
2728
+ SUBCOMMANDS: health info plugins processes workflow
2587
2729
  OPTIONS:
2588
2730
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2589
2731
  --username=VALUE username to log in
@@ -2612,7 +2754,7 @@ OPTIONS:
2612
2754
 
2613
2755
 
2614
2756
  COMMAND: ats
2615
- SUBCOMMANDS: cluster access_key api_key aws_trust_policy
2757
+ SUBCOMMANDS: access_key api_key aws_trust_policy cluster
2616
2758
  OPTIONS:
2617
2759
  --ibm-api-key=VALUE IBM API key, see https://cloud.ibm.com/iam/apikeys
2618
2760
  --instance=VALUE ATS instance in ibm cloud
@@ -2624,7 +2766,7 @@ OPTIONS:
2624
2766
 
2625
2767
 
2626
2768
  COMMAND: faspex5
2627
- SUBCOMMANDS: health version user bearer_token package admin
2769
+ SUBCOMMANDS: admin bearer_token health package user version
2628
2770
  OPTIONS:
2629
2771
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2630
2772
  --username=VALUE username to log in
@@ -2632,7 +2774,7 @@ OPTIONS:
2632
2774
  --client-id=VALUE OAuth client identifier
2633
2775
  --client-secret=VALUE OAuth client secret
2634
2776
  --redirect-uri=VALUE OAuth redirect URI for web authentication
2635
- --auth=ENUM OAuth type of authentication: web, jwt, boot
2777
+ --auth=ENUM OAuth type of authentication: boot, web, jwt
2636
2778
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2637
2779
  --passphrase=VALUE RSA private key passphrase
2638
2780
 
@@ -2650,7 +2792,7 @@ OPTIONS:
2650
2792
 
2651
2793
 
2652
2794
  COMMAND: faspex
2653
- SUBCOMMANDS: health package source me dropbox v4 address_book login_methods
2795
+ SUBCOMMANDS: address_book dropbox health login_methods me package source v4
2654
2796
  OPTIONS:
2655
2797
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2656
2798
  --username=VALUE username to log in
@@ -2664,7 +2806,7 @@ OPTIONS:
2664
2806
 
2665
2807
 
2666
2808
  COMMAND: preview
2667
- SUBCOMMANDS: scan events trevents check test
2809
+ SUBCOMMANDS: check events scan test trevents
2668
2810
  OPTIONS:
2669
2811
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2670
2812
  --username=VALUE username to log in
@@ -2699,14 +2841,14 @@ OPTIONS:
2699
2841
 
2700
2842
 
2701
2843
  COMMAND: sync
2702
- SUBCOMMANDS: start admin
2844
+ SUBCOMMANDS: admin start
2703
2845
  OPTIONS:
2704
- --parameters=VALUE extended value for session set definition
2705
- --session-name=VALUE name of session to use for admin commands, by default first one
2846
+ --sync-info=VALUE Information for sync instance and sessions (Hash)
2847
+ --sync-session=VALUE Name of session to use for admin commands. default: first in parameters
2706
2848
 
2707
2849
 
2708
2850
  COMMAND: aoc
2709
- SUBCOMMANDS: reminder servers bearer_token organization tier_restrictions user packages files admin automation gateway
2851
+ SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
2710
2852
  OPTIONS:
2711
2853
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2712
2854
  --username=VALUE username to log in
@@ -2717,20 +2859,28 @@ OPTIONS:
2717
2859
  --client-secret=VALUE OAuth API client passcode
2718
2860
  --redirect-uri=VALUE OAuth API client redirect URI
2719
2861
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2720
- --passphrase=VALUE RSA private key passphrase
2721
- --workspace=VALUE name of workspace
2722
- --name=VALUE resource name
2723
- --path=VALUE file or folder path
2724
- --link=VALUE public link to shared resource
2725
- --new-user-option=VALUE new user creation option for unknown package recipients
2726
- --from-folder=VALUE share to share source folder
2727
2862
  --scope=VALUE OAuth scope for AoC API calls
2728
- --default-ports=ENUM use standard FASP ports or get from node api: [no], yes
2729
- --validate-metadata=ENUM validate shared inbox metadata: [no], yes
2863
+ --passphrase=VALUE RSA private key passphrase
2864
+ --workspace=VALUE Name of workspace
2865
+ --name=VALUE Resource name (prefer to use keyword name)
2866
+ --link=VALUE Public link to shared resource
2867
+ --new-user-option=VALUE New user creation option for unknown package recipients
2868
+ --from-folder=VALUE Source folder for Folder-to-Folder transfer
2869
+ --validate-metadata=ENUM Validate shared inbox metadata: [no], yes
2870
+
2871
+ COMMAND: node
2872
+ SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
2873
+ OPTIONS:
2874
+ --validator=VALUE identifier of validator (optional for central)
2875
+ --asperabrowserurl=VALUE URL for simple aspera web ui
2876
+ --sync-name=VALUE sync name
2877
+ --path=VALUE file or folder path for gen4 operation "file"
2878
+ --token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
2879
+ --default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
2730
2880
 
2731
2881
 
2732
2882
  COMMAND: server
2733
- SUBCOMMANDS: health download upload browse delete rename ls rm mv du info mkdir cp df md5sum
2883
+ SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
2734
2884
  OPTIONS:
2735
2885
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2736
2886
  --username=VALUE username to log in
@@ -2740,7 +2890,7 @@ OPTIONS:
2740
2890
 
2741
2891
 
2742
2892
  COMMAND: console
2743
- SUBCOMMANDS: transfer health
2893
+ SUBCOMMANDS: health transfer
2744
2894
  OPTIONS:
2745
2895
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2746
2896
  --username=VALUE username to log in
@@ -2751,7 +2901,7 @@ OPTIONS:
2751
2901
 
2752
2902
  ```
2753
2903
 
2754
- Note that actions and parameter values can be written in short form.
2904
+ > **Note:** commands and parameter values can be written in short form.
2755
2905
 
2756
2906
  ### Bulk creation and deletion of resources
2757
2907
 
@@ -2759,7 +2909,7 @@ Bulk creation and deletion of resources are possible using option `bulk` (yes,no
2759
2909
  In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2760
2910
  This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
2761
2911
 
2762
- ## <a id="aoc"></a>Plugin: Aspera on Cloud
2912
+ ## <a id="aoc"></a>Plugin: `aoc`: IBM Aspera on Cloud
2763
2913
 
2764
2914
  Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
2765
2915
 
@@ -2802,7 +2952,7 @@ ascli config wizard --value=aoc
2802
2952
 
2803
2953
  ### <a id="aocmanual"></a>Configuration: using manual setup
2804
2954
 
2805
- > If you used the wizard (recommended): skip this section.
2955
+ > **Note:** If you used the wizard (recommended): skip this section.
2806
2956
 
2807
2957
  #### Configuration details
2808
2958
 
@@ -2826,12 +2976,12 @@ If you use the built-in client_id and client_secret, skip this and do not set th
2826
2976
 
2827
2977
  Else you can use a specific OAuth API client_id, the first step is to declare `ascli` in Aspera on Cloud using the admin interface.
2828
2978
 
2829
- (official documentation: <https://ibmaspera.com/help/admin/organization/registering_an_api_client> ).
2979
+ ([AoC documentation: Registering an API Client](https://ibmaspera.com/help/admin/organization/registering_an_api_client) ).
2830
2980
 
2831
2981
  Let's start by a registration with web based authentication (auth=web):
2832
2982
 
2833
2983
  - Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
2834
- - Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
2984
+ - Go to Apps &rarr; Admin &rarr; Organization &rarr; Integrations
2835
2985
  - Click "Create New"
2836
2986
  - Client Name: `ascli`
2837
2987
  - Redirect URIs: `http://localhost:12345`
@@ -2882,7 +3032,7 @@ If you are not using the built-in client_id and secret, JWT needs to be authoriz
2882
3032
  - Graphically
2883
3033
 
2884
3034
  - Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
2885
- - Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
3035
+ - Go to Apps &rarr; Admin &rarr; Organization &rarr; Integrations
2886
3036
  - Click on the previously created application
2887
3037
  - select tab : "JSON Web Token Auth"
2888
3038
  - Modify options if necessary, for instance: activate both options in section "Settings"
@@ -3055,7 +3205,7 @@ ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"s
3055
3205
 
3056
3206
  Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
3057
3207
 
3058
- Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
3208
+ > **Note:** The option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
3059
3209
 
3060
3210
  #### <a id="res_select"></a>Selecting a resource
3061
3211
 
@@ -3111,7 +3261,7 @@ If the command returns an error, example:
3111
3261
 
3112
3262
  Well, remove the offending parameters and try again.
3113
3263
 
3114
- Note that some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
3264
+ > **Note:** Some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
3115
3265
 
3116
3266
  #### Access Key secrets
3117
3267
 
@@ -3167,7 +3317,7 @@ Options:
3167
3317
  - `query` filter (on API call)
3168
3318
  - `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
3169
3319
 
3170
- Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
3320
+ > **Note:** This must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
3171
3321
 
3172
3322
  #### Transfer: Using specific transfer ports
3173
3323
 
@@ -3489,7 +3639,8 @@ AoC nodes as actually composed with two related entities:
3489
3639
 
3490
3640
  The web UI allows creation of both entities in one shot.
3491
3641
  For more flexibility, `ascli` allows this in two separate steps.
3492
- Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
3642
+
3643
+ > **Note:** When selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
3493
3644
 
3494
3645
  So, for example, the creation of a node using ATS in IBM Cloud looks like (see other example in this manual):
3495
3646
 
@@ -3503,7 +3654,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
3503
3654
 
3504
3655
  Once executed, the access key `id` and `secret`, randomly generated by the node api, is displayed.
3505
3656
 
3506
- > Note: Once returned by the API, the secret will not be available anymore, so store this preciously. ATS secrets can only be reset by asking to IBM support.
3657
+ > **Note:** Once returned by the API, the secret will not be available anymore, so store this preciously. ATS secrets can only be reset by asking to IBM support.
3507
3658
 
3508
3659
  - Create the AoC node entity
3509
3660
 
@@ -3523,9 +3674,10 @@ Creation of a node with a self-managed node is similar, but the command `aoc adm
3523
3674
 
3524
3675
  ### List of files to transfer
3525
3676
 
3526
- Source files are provided as a list with the `sources` option. Refer to section [File list](#file_list)
3677
+ Source files are provided as a list with the `sources` option.
3678
+ Refer to section [File list](#file_list)
3527
3679
 
3528
- Note the special case when the source files are located on "Aspera on Cloud" (i.e. using access keys and the `file id` API).
3680
+ > **Note:** A special case is when the source files are located on **Aspera on Cloud** (i.e. using access keys and the `file id` API).
3529
3681
 
3530
3682
  Source files are located on "Aspera on cloud", when :
3531
3683
 
@@ -3644,19 +3796,28 @@ ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
3644
3796
  - `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
3645
3797
  - `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
3646
3798
 
3647
- Typically, one would execute this command on a regular basis, using the method of your choice:
3648
-
3649
- - Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
3650
- - Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
3651
- - etc...
3799
+ Typically, one would execute this command on a regular basis, using the method of your choice: see [Scheduler](#scheduler).
3652
3800
 
3653
3801
  ### Files
3654
3802
 
3655
- Folder sharing app.
3803
+ The Files application presents a **Home** folder to users in a given workspace.
3804
+ Files located here are either user's files, or shared folders.
3656
3805
 
3657
3806
  #### Download Files
3658
3807
 
3659
- Download of files is straightforward using command: `aoc files download`
3808
+ The general download command is:
3809
+
3810
+ ```bash
3811
+ ascli aoc files download <source folder path> <source filename 1> ...
3812
+ ```
3813
+
3814
+ I.e. the first argument is the source folder, and the following arguments are the source file names in this folder.
3815
+
3816
+ If a single file or folder is to be downloaded, then a single argument can be provided.
3817
+
3818
+ ```bash
3819
+ ascli aoc files download <single file path>
3820
+ ```
3660
3821
 
3661
3822
  #### Shared folders
3662
3823
 
@@ -3774,18 +3935,17 @@ cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='
3774
3935
  ### AoC sample commands
3775
3936
 
3776
3937
  ```bash
3777
- aoc -N remind --username=my_aoc_user_email
3778
- aoc -N servers
3779
3938
  aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
3780
3939
  aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3781
- aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3782
- aoc admin ats access_key delete akibmcloud
3940
+ aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak1ibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3941
+ aoc admin ats access_key delete ak1ibmcloud
3783
3942
  aoc admin ats access_key list --fields=name,id
3784
- aoc admin ats access_key node akibmcloud --secret=somesecret browse /
3943
+ aoc admin ats access_key node ak1ibmcloud --secret=somesecret browse /
3785
3944
  aoc admin ats cluster clouds
3786
3945
  aoc admin ats cluster list
3787
3946
  aoc admin ats cluster show --cloud=aws --region=eu-west-1
3788
3947
  aoc admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
3948
+ aoc admin auth_providers list
3789
3949
  aoc admin res application list
3790
3950
  aoc admin res client list
3791
3951
  aoc admin res client_access_key list
@@ -3806,14 +3966,15 @@ aoc admin res self show
3806
3966
  aoc admin res short_link list
3807
3967
  aoc admin res user list
3808
3968
  aoc admin res workspace_membership list
3809
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
3810
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 events
3811
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 browse /
3812
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 delete /folder1
3813
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 mkdir /folder1
3814
- aoc admin resource node v3 name my_aoc_ak_name --secret=my_aoc_ak_secret access_key delete testsub1
3969
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do browse /
3970
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do delete /folder1
3971
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do mkdir /folder1
3972
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
3973
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 events
3974
+ aoc admin resource node do name my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key delete testsub1
3815
3975
  aoc admin resource workspace list
3816
3976
  aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
3977
+ aoc admin subscription
3817
3978
  aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
3818
3979
  aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
3819
3980
  aoc automation workflow delete my_wf_id
@@ -3825,10 +3986,11 @@ aoc faspex
3825
3986
  aoc files bearer /
3826
3987
  aoc files bearer_token_node / --cache-tokens=no
3827
3988
  aoc files browse /
3828
- aoc files browse / -N --link=my_aoc_publink_folder
3989
+ aoc files browse / --link=my_aoc_publink_folder
3829
3990
  aoc files delete /testsrc
3830
3991
  aoc files download --transfer=connect /200KB.1
3831
- aoc files file permission --path=/ascli_test list
3992
+ aoc files file modify --path=my_aoc_test_folder
3993
+ aoc files file permission --path=my_aoc_test_folder list
3832
3994
  aoc files file show --path=/200KB.1
3833
3995
  aoc files file show my_file_id
3834
3996
  aoc files find / --value='\.partial$'
@@ -3838,12 +4000,14 @@ aoc files rename /somefolder testdst
3838
4000
  aoc files short_link create --to-folder=/testdst --value=private
3839
4001
  aoc files short_link create --to-folder=/testdst --value=public
3840
4002
  aoc files short_link list --value=@json:'{"purpose":"shared_folder_auth_link"}'
4003
+ aoc files sync admin status --to-folder=/testdst --sync-info=@json:'{"sessions":[{"name":"s2","direction":"pull","local_dir":"syncdir","reset":true}]}'
4004
+ aoc files sync start --to-folder=/testdst --sync-info=@json:'{"sessions":[{"name":"s2","direction":"pull","local_dir":"syncdir","reset":true}]}'
3841
4005
  aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
4006
+ aoc files upload --to-folder=/ testfile.bin --link=my_aoc_publink_folder
3842
4007
  aoc files upload --to-folder=/testsrc testfile.bin
3843
- aoc files upload -N --to-folder=/ testfile.bin --link=my_aoc_publink_folder
3844
4008
  aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
3845
4009
  aoc files v3 info
3846
- aoc org -N --link=my_aoc_publink_recv_from_aocuser
4010
+ aoc org --link=my_aoc_publink_recv_from_aocuser
3847
4011
  aoc organization
3848
4012
  aoc packages browse "my_package_id" /contents
3849
4013
  aoc packages list
@@ -3852,19 +4016,21 @@ aoc packages recv "my_package_id" --to-folder=.
3852
4016
  aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
3853
4017
  aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
3854
4018
  aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
4019
+ aoc packages send --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
4020
+ aoc packages send --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
3855
4021
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
3856
4022
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
3857
4023
  aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
3858
- aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
3859
- aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
3860
4024
  aoc packages shared_inboxes list
4025
+ aoc remind --username=my_aoc_user_email
4026
+ aoc servers
3861
4027
  aoc user profile modify @json:'{"name":"dummy change"}'
3862
4028
  aoc user profile show
3863
4029
  aoc user workspaces current
3864
4030
  aoc user workspaces list
3865
4031
  ```
3866
4032
 
3867
- ## <a id="ats"></a>Plugin: Aspera Transfer Service
4033
+ ## <a id="ats"></a>Plugin: `ats`: IBM Aspera Transfer Service
3868
4034
 
3869
4035
  ATS is usable either :
3870
4036
 
@@ -3985,13 +4151,15 @@ The parameters provided to ATS for access key creation are the ones of [ATS API]
3985
4151
  ### ATS sample commands
3986
4152
 
3987
4153
  ```bash
3988
- ats access_key cluster akibmcloud --secret=somesecret
4154
+ ats access_key cluster ak2ibmcloud --secret=somesecret
3989
4155
  ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3990
- ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4156
+ ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak2ibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4157
+ ats access_key delete ak2ibmcloud
3991
4158
  ats access_key delete ak_aws
3992
- ats access_key delete akibmcloud
4159
+ ats access_key entitlement ak2ibmcloud
3993
4160
  ats access_key list --fields=name,id
3994
- ats access_key node akibmcloud browse / --secret=somesecret
4161
+ ats access_key node ak2ibmcloud browse / --secret=somesecret
4162
+ ats access_key show ak2ibmcloud
3995
4163
  ats api_key create
3996
4164
  ats api_key instances
3997
4165
  ats api_key list
@@ -4001,17 +4169,21 @@ ats cluster show --cloud=aws --region=eu-west-1
4001
4169
  ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
4002
4170
  ```
4003
4171
 
4004
- ## Plugin: IBM Aspera High Speed Transfer Server (transfer)
4172
+ ## <a id="server"></a>Plugin: `server`: IBM Aspera High Speed Transfer Server (SSH)
4005
4173
 
4006
- This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
4174
+ The `server` plugin is used for operations on Aspera HSTS using SSH authentication.
4007
4175
  It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
4008
- Modern mode is to use the node API and transfer tokens.
4176
+ An SSH session is established, authenticated with either a password or an SSH private key,
4177
+ then commands `ascp` (for transfers) and `ascmd` (for file operations) are executed.
4178
+
4179
+ > **Note:** The URL to be provided is usually: `ssh://_server_address_:33001`
4009
4180
 
4010
4181
  ### Server sample commands
4011
4182
 
4012
4183
  ```bash
4013
- server -N -Ptst_server_bykey -Plocal_user br /
4184
+ server br /
4014
4185
  server browse /
4186
+ server browse NEW_SERVER_FOLDER/testfile.bin
4015
4187
  server browse folder_1/target_hot
4016
4188
  server cp NEW_SERVER_FOLDER/testfile.bin folder_1/200KB.2
4017
4189
  server delete NEW_SERVER_FOLDER
@@ -4027,34 +4199,39 @@ server md5sum NEW_SERVER_FOLDER/testfile.bin
4027
4199
  server mkdir NEW_SERVER_FOLDER --logger=stdout
4028
4200
  server mkdir folder_1/target_hot
4029
4201
  server mv folder_1/200KB.2 folder_1/to.delete
4202
+ server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"syncdir"}}'
4203
+ server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"syncdir"}]}'
4204
+ server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"syncdir"},"remote":{"path":"'"NEW_SERVER_FOLDER"'"},"reset":true,"quiet":false}'
4205
+ server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"syncdir","reset":true}]}'
4030
4206
  server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
4031
4207
  server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
4032
4208
  server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
4033
4209
  server upload --sources=@ts --ts=@json:'{"EX_file_pair_list":"'"filepairlist.txt"'"}'
4034
4210
  server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
4035
4211
  server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
4036
- server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
4212
+ server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email --ascp-opts=@list:' -l 10m'
4037
4213
  server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
4038
4214
  server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
4039
- server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
4215
+ server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress=multi
4040
4216
  ```
4041
4217
 
4042
4218
  ### Authentication on Server with SSH session
4043
4219
 
4044
- If SSH is the session control protocol (i.e. not WSS), then following session authentication methods are supported:
4220
+ If SSH is the session protocol (by default i.e. not WSS), then following session authentication methods are supported:
4045
4221
 
4046
- - SSH password
4047
- - SSH keys (Multiple SSH key paths can be provided.)
4222
+ - `password`: SSH password
4223
+ - `ssh_keys`: SSH keys (Multiple SSH key paths can be provided.)
4048
4224
 
4049
- If username is not provided, the default transfer user `xfer` is used.
4225
+ If `username` is not provided then the default transfer user `xfer` is used.
4050
4226
 
4051
- If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used. Example:
4227
+ If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used.
4228
+ Example:
4052
4229
 
4053
- ```javascript
4054
- ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
4230
+ ```bash
4231
+ ascli server --url=ssh://_server_address_:33001 ... --ts=@json:'{"token":"Basic abc123"}'
4055
4232
  ```
4056
4233
 
4057
- > Note: If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
4234
+ > **Note:** If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
4058
4235
 
4059
4236
  The value of the `ssh_keys` option can be a single value or an array.
4060
4237
  Each value is a **path** to a private key and is expanded (`~` is replaced with the user's home folder).
@@ -4067,7 +4244,7 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
4067
4244
  ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
4068
4245
  ```
4069
4246
 
4070
- For non-transfer related command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
4247
+ For file operation command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
4071
4248
  For a list of SSH client options, refer to the ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html).
4072
4249
 
4073
4250
  By default the SSH library expect that a local ssh-agent is running.
@@ -4086,8 +4263,8 @@ ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant p
4086
4263
 
4087
4264
  This means that you don't have such an SSH agent running, then:
4088
4265
 
4089
- - check env var: `SSH_AGENT_SOCK`
4090
- - check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
4266
+ - Check env var: `SSH_AGENT_SOCK`
4267
+ - Check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
4091
4268
  - [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
4092
4269
  - To disable the use of `ssh-agent`, use the option `ssh_options` like this:
4093
4270
 
@@ -4097,7 +4274,17 @@ ascli server --ssh-options=@ruby:'{use_agent: false}' ...
4097
4274
 
4098
4275
  This can also be set as default using a global preset.
4099
4276
 
4100
- ### Example
4277
+ ### Other session channels for `server`
4278
+
4279
+ URL schemes `local` and `https` are also supported, mainly for testing purpose.
4280
+ (`--url=local:` , `--url=https://...`)
4281
+
4282
+ - `local` will execute `ascmd` locally, instead of using a SSH cnnection.
4283
+ - `https` will use Web Socket Session: This requires the use of a transfer token. For example a `Basic` token can be used.
4284
+
4285
+ As, most of the time, SSH is used, if an `http` scheme is provided without token, the plugin will fallback to SSH and port 33001.
4286
+
4287
+ ### Examples: `server`
4101
4288
 
4102
4289
  One can test the `server` application using the well known demo server:
4103
4290
 
@@ -4109,7 +4296,7 @@ ascli server download /aspera-test-dir-large/200MB
4109
4296
 
4110
4297
  `initdemo` creates a [option preset](#lprt) `demoserver` and set it as default for plugin `server`.
4111
4298
 
4112
- ## Plugin: IBM Aspera High Speed Transfer Server (node)
4299
+ ## <a id="node"></a>Plugin: `node`: IBM Aspera High Speed Transfer Server Node
4113
4300
 
4114
4301
  This plugin gives access to capabilities provided by HSTS node API.
4115
4302
 
@@ -4129,15 +4316,15 @@ For transfers, it is possible to control how transfer is authorized using option
4129
4316
 
4130
4317
  ```javascript
4131
4318
  {
4132
- "remote_host": address of node url,
4319
+ "remote_host": "<address of node url>",
4133
4320
  "remote_user": "xfer",
4134
4321
  "ssh_port": 33001,
4135
4322
  "token": "Basic <base 64 encoded user/pass>",
4136
- "direction": send/receive
4323
+ "direction": "[send|receive]"
4137
4324
  }
4138
4325
  ```
4139
4326
 
4140
- Note that the port is assumed to be the default SSH port `33001` and transfer user is assumed to be `xfer`.
4327
+ > **Note:** the port is assumed to be the default Aspera SSH port `33001` and transfer user is assumed to be `xfer`.
4141
4328
 
4142
4329
  ### Central
4143
4330
 
@@ -4224,13 +4411,20 @@ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"myst
4224
4411
  ### Node sample commands
4225
4412
 
4226
4413
  ```bash
4227
- node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
4228
- node -N -Ptst_node_preview access_key delete aoc_1
4229
- node -Pnode_srv access_key do my_aoc_ak_name br /
4230
- node -Pnode_srv access_key list
4231
- node -Pnode_srv service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
4232
- node -Pnode_srv service delete service1
4233
- node -Pnode_srv service list
4414
+ node access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
4415
+ node access_key delete aoc_1
4416
+ node access_key do my_aoc_ak_name browse /
4417
+ node access_key do my_aoc_ak_name delete /folder2
4418
+ node access_key do my_aoc_ak_name delete testfile1
4419
+ node access_key do my_aoc_ak_name download testfile1 --to-folder=.
4420
+ node access_key do my_aoc_ak_name file show --path=/testfile1
4421
+ node access_key do my_aoc_ak_name file show 1
4422
+ node access_key do my_aoc_ak_name find /
4423
+ node access_key do my_aoc_ak_name mkdir /folder1
4424
+ node access_key do my_aoc_ak_name node_info /
4425
+ node access_key do my_aoc_ak_name rename /folder1 folder2
4426
+ node access_key do my_aoc_ak_name upload 'faux:///testfile1?1k' --default_ports=no
4427
+ node access_key list
4234
4428
  node api_details
4235
4429
  node async bandwidth 1
4236
4430
  node async counters 1
@@ -4241,7 +4435,7 @@ node async show ALL
4241
4435
  node basic_token
4242
4436
  node browse / -r
4243
4437
  node delete /todelete
4244
- node delete @list:,/todelete,/tdlink,/delfile
4438
+ node delete @list:,folder_1/todelete,folder_1/tdlink,folder_1/delfile
4245
4439
  node delete folder_1/10MB.1
4246
4440
  node delete testfile.bin
4247
4441
  node download testfile.bin --to-folder=.
@@ -4249,11 +4443,14 @@ node download testfile.bin --to-folder=. --token-type=hybrid
4249
4443
  node health
4250
4444
  node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
4251
4445
  node license
4252
- node mkdir /todelete
4253
- node mkfile /delfile1 "hello world"
4254
- node mklink /todelete /tdlink
4255
- node rename / delfile1 delfile
4446
+ node mkdir folder_1/todelete
4447
+ node mkfile folder_1/delfile1 "hello world"
4448
+ node mklink folder_1/todelete folder_1/tdlink
4449
+ node rename folder_1 delfile1 delfile
4256
4450
  node search / --value=@json:'{"sort":"mtime"}'
4451
+ node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
4452
+ node service delete service1
4453
+ node service list
4257
4454
  node space /
4258
4455
  node sync bandwidth my_syncid
4259
4456
  node sync counters my_syncid
@@ -4273,7 +4470,7 @@ node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps
4273
4470
  node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
4274
4471
  ```
4275
4472
 
4276
- ## Plugin: IBM Aspera Faspex5
4473
+ ## <a id="faspex5"></a>Plugin: `faspex5`: IBM Aspera Faspex v5
4277
4474
 
4278
4475
  IBM Aspera's newer self-managed application.
4279
4476
 
@@ -4306,7 +4503,7 @@ Then use these options:
4306
4503
  --private-key=@file:.../path/to/key.pem
4307
4504
  ```
4308
4505
 
4309
- > The `private_key` option must contain the PEM value of the private key which can be read from a file using the modifier: `@file:`, e.g. `@file:/path/to/key.pem`.
4506
+ > **Note:** The `private_key` option must contain the PEM value of the private key which can be read from a file using the modifier: `@file:`, e.g. `@file:/path/to/key.pem`.
4310
4507
 
4311
4508
  ### Faspex 5 web authentication
4312
4509
 
@@ -4348,12 +4545,13 @@ Most commands are directly REST API calls.
4348
4545
  Parameters to commandsa are carried through option `value`, as extended value.
4349
4546
  Usually using JSON format with prefix `@json:`.
4350
4547
 
4351
- > The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
4548
+ > **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
4352
4549
 
4353
4550
  ```bash
4354
4551
  faspex5 admin res accounts list
4355
4552
  faspex5 admin res contacts list
4356
4553
  faspex5 admin res jobs list
4554
+ faspex5 admin res metadata_profiles list
4357
4555
  faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
4358
4556
  faspex5 admin res oauth_clients list
4359
4557
  faspex5 admin res registrations list
@@ -4365,6 +4563,9 @@ faspex5 health
4365
4563
  faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
4366
4564
  faspex5 package receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
4367
4565
  faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"_content_prot_here_"}'
4566
+ faspex5 package show "my_package_id"
4567
+ faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
4568
+ faspex5 user profile show
4368
4569
  ```
4369
4570
 
4370
4571
  Other examples:
@@ -4387,7 +4588,7 @@ ascli faspex5 admin res metadata_profiles create --value=@json:'{"name":"the pro
4387
4588
  ascli faspex5 admin res shared create --value=@json:'{"name":"the shared inbox","metadata_profile_id":1}'
4388
4589
  ```
4389
4590
 
4390
- ## Plugin: IBM Aspera Faspex (4.x)
4591
+ ## <a id="faspex"></a>Plugin: `faspex`: IBM Aspera Faspex v4
4391
4592
 
4392
4593
  Notes:
4393
4594
 
@@ -4570,7 +4771,7 @@ faspex v4 wmembership list
4570
4771
  faspex v4 workgroup list
4571
4772
  ```
4572
4773
 
4573
- ## Plugin: IBM Aspera Shares
4774
+ ## <a id="shares"></a>Plugin: `shares`: IBM Aspera Shares v1
4574
4775
 
4575
4776
  Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
4576
4777
 
@@ -4578,10 +4779,14 @@ Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and
4578
4779
 
4579
4780
  ```bash
4580
4781
  shares admin share list
4782
+ shares admin share list --fields=-status,status_message
4581
4783
  shares admin share user_permissions 1
4582
4784
  shares admin user app_authorizations 1
4785
+ shares admin user ldap_import --value=the_name
4583
4786
  shares admin user list
4787
+ shares admin user saml_import --value=@json:'{"id":"the_id","name_id":"the_name"}'
4584
4788
  shares admin user share_permissions 1
4789
+ shares health
4585
4790
  shares repository browse /
4586
4791
  shares repository delete my_shares_upload/testfile.bin
4587
4792
  shares repository download --to-folder=. my_shares_upload/testfile.bin
@@ -4590,7 +4795,7 @@ shares repository upload --to-folder=my_shares_upload testfile.bin
4590
4795
  shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
4591
4796
  ```
4592
4797
 
4593
- ## Plugin: Console
4798
+ ## <a id="console"></a>Plugin: `console`: IBM Aspera Console
4594
4799
 
4595
4800
  ### Console sample commands
4596
4801
 
@@ -4601,11 +4806,12 @@ console transfer smart list
4601
4806
  console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
4602
4807
  ```
4603
4808
 
4604
- ## Plugin: Orchestrator
4809
+ ## <a id="orchestrator"></a>Plugin: `orchestrator`:IBM Aspera Orchestrator
4605
4810
 
4606
4811
  ### Orchestrator sample commands
4607
4812
 
4608
4813
  ```bash
4814
+ orchestrator health
4609
4815
  orchestrator info
4610
4816
  orchestrator plugins
4611
4817
  orchestrator processes
@@ -4619,13 +4825,15 @@ orchestrator workflow status ALL
4619
4825
  orchestrator workflow status my_orch_workflow_id
4620
4826
  ```
4621
4827
 
4622
- ## Plugin: IBM Cloud Object Storage
4828
+ ## <a id="cos"></a>Plugin: `cos`: IBM Cloud Object Storage
4623
4829
 
4624
4830
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
4625
4831
  It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
4626
4832
  Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
4627
4833
 
4628
- There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
4834
+ There are two possibilities to provide credentials.
4835
+ If you already have the endpoint, apikey and CRN, use the first method.
4836
+ If you don't have credentials but have access to the IBM Cloud console, then use the second method.
4629
4837
 
4630
4838
  ### Using endpoint, apikey and Resource Instance ID (CRN)
4631
4839
 
@@ -4647,9 +4855,17 @@ Then, jump to the transfer example.
4647
4855
 
4648
4856
  ### Using service credential file
4649
4857
 
4650
- If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
4858
+ If you are the COS administrator and don't have yet the credential:
4859
+ Service credentials are directly created using the IBM cloud Console (web UI).
4860
+ Navigate to:
4651
4861
 
4652
- Navigation Menu &rarr; Resource List &rarr; Storage &rarr; Cloud Object Storage &rarr; Service Credentials &rarr; &lt;select or create credentials&gt; &rarr; view credentials &rarr; copy
4862
+ - &rarr; Navigation Menu
4863
+ - &rarr; [Resource List](https://cloud.ibm.com/resources)
4864
+ - &rarr; [Storage](https://cloud.ibm.com/objectstorage)
4865
+ - &rarr; Select your storage instance
4866
+ - &rarr; Service Credentials
4867
+ - &rarr; New credentials (Leave default role: Writer, no special options)
4868
+ - &rarr; Copy to clipboard
4653
4869
 
4654
4870
  Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
4655
4871
 
@@ -4710,31 +4926,43 @@ ascli cos node info
4710
4926
  ascli cos node upload 'faux:///sample1G?1g'
4711
4927
  ```
4712
4928
 
4713
- Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
4929
+ Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man `ascp` and section above), but you can of course send a real file by specifying a real file instead.
4714
4930
 
4715
4931
  ### COS sample commands
4716
4932
 
4717
4933
  ```bash
4718
- cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
4719
- cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
4934
+ cos --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
4935
+ cos --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
4720
4936
  cos node access_key show self
4721
4937
  cos node download testfile.bin --to-folder=.
4722
4938
  cos node info
4723
4939
  cos node upload testfile.bin
4724
4940
  ```
4725
4941
 
4726
- ## Plugin: IBM Aspera Sync
4942
+ ## <a id="async"></a>Plugin: `async`: IBM Aspera Sync
4727
4943
 
4728
4944
  A basic plugin to start an "async" using `ascli`.
4729
- The main advantage is the possibility to start from ma configuration file, using `ascli` standard options.
4945
+ The main advantage over bare `async` command line is the possibility to use a configuration file, using `ascli` standard options.
4946
+
4947
+ Also, the `sync` command is also made available through the `server sync` and `aoc files sync` commands.
4948
+ In this case, some of the `sync` parameters are fill from parameters of the related plugin.
4949
+
4950
+ > **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
4951
+ >
4952
+ > **Note:** Two JSON syntax are supported for option `sync_info`.
4953
+ > The first is same sync payload as specified on the `async` option `--conf` or in the latest node API, this is the prefered syntax and allows a single session definition.
4954
+ > The second (legacy) is specific to `ascli` and allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
4955
+
4956
+ Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
4730
4957
 
4731
4958
  ### Sync sample commands
4732
4959
 
4733
4960
  ```bash
4734
- sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
4961
+ sync admin status --sync-info=@json:'{"sessions":[{"name":"test","local_dir":"contents"}]}'
4962
+ sync start --sync-info=@json:'{"instance":{"quiet":true},"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_paths":["my_local_user_key"]}]}'
4735
4963
  ```
4736
4964
 
4737
- ## Plugin: Preview
4965
+ ## <a id="preview"></a>Plugin: `preview`: Preview generator for AoC
4738
4966
 
4739
4967
  The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
4740
4968
  It uses the **node API** of Aspera HSTS and requires use of Access Keys and it's **storage root**.
@@ -4846,7 +5074,7 @@ The preview generator is run as a user, preferably a regular user (not root). Wh
4846
5074
 
4847
5075
  Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt). The configuration file must be created with the same user used to run so that it is properly used on runtime.
4848
5076
 
4849
- Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
5077
+ The `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
4850
5078
 
4851
5079
  ```bash
4852
5080
  su -s /bin/bash - xfer
@@ -4869,11 +5097,11 @@ This shall list the contents of the storage root of the access key.
4869
5097
 
4870
5098
  ### Execution
4871
5099
 
4872
- The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
4873
- It needs to be run on a regular basis to create or update preview files. For that use your best
4874
- reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
5100
+ The tool intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
5101
+ It needs to be run on a regular basis to create or update preview files.
5102
+ For that use your best reliable scheduler, see [Scheduling](#scheduling).
4875
5103
 
4876
- Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
5104
+ Typically, for **Access key** access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
4877
5105
 
4878
5106
  Lets do a one shot test, using the configuration previously created:
4879
5107
 
@@ -4889,12 +5117,12 @@ On subsequent run it reads this file and check that previews are generated for t
4889
5117
 
4890
5118
  ### Configuration for Execution in scheduler
4891
5119
 
4892
- Here is an example of configuration for use with cron on Linux.
4893
- Adapt the scripts to your own preference.
5120
+ Here is an example of configuration for use with `cron` on Linux.
5121
+ Adapt the scripts to your own needs.
4894
5122
 
4895
5123
  We assume here that a configuration preset was created as shown previously.
4896
5124
 
4897
- Lets first setup a script that will be used in the scheduler and sets up the environment.
5125
+ Lets first setup a script that will be used in the scheduler and set up the environment.
4898
5126
 
4899
5127
  Example of startup script `cron_ascli`, which sets the Ruby environment and adds some timeout protection:
4900
5128
 
@@ -4916,7 +5144,7 @@ crontab<<EOF
4916
5144
  EOF
4917
5145
  ```
4918
5146
 
4919
- Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
5147
+ > **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
4920
5148
 
4921
5149
  ### Candidate detection for creation or update (or deletion)
4922
5150
 
@@ -4997,12 +5225,12 @@ If the `mimemagic` gem complains about missing mime info file:
4997
5225
  - any OS:
4998
5226
 
4999
5227
  - Examine the error message
5000
- - Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
5228
+ - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5001
5229
  - move and rename this file to one of the locations expected by mimemagic as specified in the error message
5002
5230
 
5003
5231
  - Windows:
5004
5232
 
5005
- - Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
5233
+ - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5006
5234
  - Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
5007
5235
  - Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
5008
5236
 
@@ -5057,17 +5285,17 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
5057
5285
  Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
5058
5286
 
5059
5287
  The `smtp` option is a hash table (extended value) with the following fields:
5060
- <table>
5061
- <tr><th>field</th><th>default</th><th>example</th><th>description</th></tr>
5062
- <tr><td>`server`</td><td>-</td><td>smtp.gmail.com</td><td>SMTP server address</td></tr>
5063
- <tr><td>`tls`</td><td>true</td><td>false</td><td>use of TLS</td></tr>
5064
- <tr><td>`port`</td><td>587 for tls<br/>25 else</td><td>587</td><td>port for service</td></tr>
5065
- <tr><td>`domain`</td><td>domain of server</td><td>gmail.com</td><td>email domain of user</td></tr>
5066
- <tr><td>`username`</td><td>-</td><td>john@example.com</td><td>user to authenticate on SMTP server, leave empty for open auth.</td></tr>
5067
- <tr><td>`password`</td><td>-</td><td>MyP@ssword</td><td>password for above username</td></tr>
5068
- <tr><td>`from_email`</td><td>username if defined</td><td>laurent.martin.l@gmail.com</td><td>address used if received replies</td></tr>
5069
- <tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
5070
- </table>
5288
+
5289
+ | field | default | example | description |
5290
+ |--------------|---------------------|----------------------------|----------------------------------|
5291
+ | `server` | - | smtp.gmail.com | SMTP server address |
5292
+ | `tls` | true | false | use of TLS |
5293
+ | `port` | TLS: 587<br/>25 | 587 | port for service |
5294
+ | `domain` | domain of server | gmail.com | email domain of user |
5295
+ | `username` | - | john@example.com | user to authenticate on SMTP server, leave empty for open auth. |
5296
+ | `password` | - | my_password_here | password for above username |
5297
+ | `from_email` | username if defined | johnny@example.com | address used if receiver replies |
5298
+ | `from_name` | same as email | John Wayne | display name of sender |
5071
5299
 
5072
5300
  ### Example of configuration
5073
5301
 
@@ -5104,9 +5332,9 @@ The template is the full SMTP message, including headers.
5104
5332
 
5105
5333
  The following variables are defined by default:
5106
5334
 
5107
- - from_name
5108
- - from_email
5109
- - to
5335
+ - `from_name`
5336
+ - `from_email`
5337
+ - `to`
5110
5338
 
5111
5339
  Other variables are defined depending on context.
5112
5340
 
@@ -5162,26 +5390,26 @@ The tool expect one single argument: a [*transfer-spec*](#transferspec).
5162
5390
 
5163
5391
  If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
5164
5392
 
5165
- Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
5393
+ > **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the hash using JSON syntax.
5166
5394
 
5167
5395
  During execution, it generates all low level events, one per line, in JSON format on stdout.
5168
5396
 
5169
- Note that there are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
5397
+ There are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
5170
5398
 
5171
5399
  - `EX_loglevel` to change log level of the tool
5172
5400
  - `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
5173
5401
 
5174
- Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
5402
+ > **Note:** In addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
5175
5403
 
5176
5404
  ### Comparison of interfaces
5177
5405
 
5178
- <table>
5179
- <tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
5180
- <tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>many</td></tr>
5181
- <tr><td>required additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
5182
- <tr><td>startup</td><td>JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn)</td><td>command line arguments</td><td>API</td><td>daemon</td></tr>
5183
- <tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
5184
- <tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp (and SDK if compiled)</td><td>any with ascp</td><td>any with ascp and transfer daemon</td></tr></table>
5406
+ | feature/tool | asession | `ascp` | FaspManager | Transfer SDK |
5407
+ |--------------|----------|--------|-------------|--------------|
5408
+ | language integration | any | any | C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/> | many |
5409
+ | required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
5410
+ | startup | JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn) | command line arguments | API | daemon |
5411
+ | events | JSON on stdout | none by default<br/>or need to open management port<br/>and proprietary text syntax | callback | callback |
5412
+ | platforms | any with ruby and `ascp` | any with `ascp` (and SDK if compiled) | any with `ascp` | any with `ascp` and transfer daemon |
5185
5413
 
5186
5414
  ### Simple session
5187
5415
 
@@ -5193,13 +5421,13 @@ Create a file `session.json` with:
5193
5421
 
5194
5422
  Then start the session:
5195
5423
 
5196
- ```
5424
+ ```bash
5197
5425
  asession < session.json
5198
5426
  ```
5199
5427
 
5200
5428
  ### Asynchronous commands and Persistent session
5201
5429
 
5202
- `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
5430
+ `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
5203
5431
 
5204
5432
  This is particularly useful for a persistent session ( with the [*transfer-spec*](#transferspec) parameter: `"keepalive":true` )
5205
5433
 
@@ -5266,9 +5494,9 @@ The general idea is to rely on :
5266
5494
  - take advantage of `ascli` configuration capabilities and server side knowledge
5267
5495
  - the OS scheduler for reliability and continuous operation
5268
5496
 
5269
- #### ascp features
5497
+ #### `ascp` features
5270
5498
 
5271
- Interesting ascp features are found in its arguments: (see ascp manual):
5499
+ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
5272
5500
 
5273
5501
  - `ascp` already takes care of sending only "new" files: option `-k 1,2,3` (`resume_policy`)
5274
5502
  - `ascp` has some options to remove or move files after transfer: `--remove-after-transfer`, `--move-after-transfer`, `--remove-empty-directories` (`remove_after_transfer`, `move_after_transfer`, `remove_empty_directories`)
@@ -5279,14 +5507,15 @@ Note that:
5279
5507
 
5280
5508
  - `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transferspec), with `--ts` parameter.
5281
5509
  - most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transferspec) parameters
5282
- - native ascp arguments can be provided with the [*transfer-spec*](#transferspec) parameter: `EX_ascp_args` (array), only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node)
5510
+ - native `ascp` arguments can be provided with the [*transfer-spec*](#transferspec) parameter: `EX_ascp_args` (array), only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node)
5283
5511
 
5284
5512
  #### server side and configuration
5285
5513
 
5286
5514
  Virtually any transfer on a "repository" on a regular basis might emulate a hot folder.
5287
- > file detection is not based on events (inotify, etc...), but on a simple folder scan on source side.
5288
5515
 
5289
- > parameters may be saved in a [option preset](#lprt) and used with `-P`.
5516
+ > **Note:** file detection is not based on events (inotify, etc...), but on a simple folder scan on source side.
5517
+ >
5518
+ > **Note:** parameters may be saved in a [option preset](#lprt) and used with `-P`.
5290
5519
 
5291
5520
  #### Scheduling
5292
5521
 
@@ -5295,7 +5524,7 @@ Refer to section [Scheduling](#scheduling). (on use of option `lock_port`)
5295
5524
 
5296
5525
  ### Example: upload hot folder
5297
5526
 
5298
- ```javascript
5527
+ ```bash
5299
5528
  ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"remove_after_transfer":true,"remove_empty_directories":true,"exclude_newer_than:-8,"src_base":"source_hot"}'
5300
5529
  ```
5301
5530
 
@@ -5304,11 +5533,25 @@ Source files are deleted after transfer.
5304
5533
  Growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period).
5305
5534
  If a transfer takes more than the execution period, then the subsequent execution is skipped (`lock_port`) preventing multiple concurrent runs.
5306
5535
 
5307
- ### Example: unidirectional synchronization
5536
+ ### Example: unidirectional synchronization (upload) to server
5308
5537
 
5309
- ```javascript
5310
- ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=12345 --ts=@json:'{"resume_policy":"sparse_csum","exclude_newer_than:-8,"src_base":"source_sync"}'
5538
+ ```bash
5539
+ ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=12345 --ts=@json:'{"resume_policy":"sparse_csum","exclude_newer_than":-8,"src_base":"source_sync"}'
5540
+ ```
5541
+
5542
+ This can also be used with other folder-based applications: Aspera on Cloud, Shares, Node:
5543
+
5544
+ ### Example: unidirectional synchronization (download) from Aspera on Cloud Files
5545
+
5546
+ ```bash
5547
+ ascli aoc files download . --to-folder=. --lock-port=12345 --progress=none --display=data \
5548
+ --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
5311
5549
  ```
5550
+
5551
+ > **Note:** option `delete_before_transfer` will delete files locally, if they are not present on remote side.
5552
+ >
5553
+ > **Note:** options `progress` and `display` limit output for headless operation (e.g. cron job)
5554
+
5312
5555
  ## Health check and Nagios
5313
5556
 
5314
5557
  Most plugin provide a `health` command that will check the health status of the application. Example:
@@ -5384,11 +5627,11 @@ aoc.read('self')
5384
5627
 
5385
5628
  ## Changes (Release notes)
5386
5629
 
5387
- See <CHANGELOG.md>
5630
+ See [CHANGELOG.md](CHANGELOG.md)
5388
5631
 
5389
5632
  ## History
5390
5633
 
5391
- When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
5634
+ When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (`ascp`), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
5392
5635
 
5393
5636
  There were a few pitfalls:
5394
5637
 
@@ -5399,7 +5642,7 @@ So, it evolved into `ascli`:
5399
5642
 
5400
5643
  - portable: works on platforms supporting `ruby` (and `ascp`)
5401
5644
  - easy to install with the `gem` utility
5402
- - supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
5645
+ - supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from `ascp` command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
5403
5646
  - `ruby` is consistent with other Aspera products
5404
5647
 
5405
5648
  ## Common problems