aspera-cli 4.4.0 → 4.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +1042 -787
  3. data/bin/ascli +1 -1
  4. data/bin/asession +3 -5
  5. data/docs/Makefile +4 -7
  6. data/docs/README.erb.md +988 -740
  7. data/examples/faspex4.rb +4 -6
  8. data/examples/transfer.rb +2 -2
  9. data/lib/aspera/aoc.rb +139 -118
  10. data/lib/aspera/cli/listener/progress_multi.rb +5 -5
  11. data/lib/aspera/cli/main.rb +64 -34
  12. data/lib/aspera/cli/manager.rb +19 -20
  13. data/lib/aspera/cli/plugin.rb +9 -1
  14. data/lib/aspera/cli/plugins/aoc.rb +156 -143
  15. data/lib/aspera/cli/plugins/ats.rb +11 -10
  16. data/lib/aspera/cli/plugins/bss.rb +2 -2
  17. data/lib/aspera/cli/plugins/config.rb +236 -112
  18. data/lib/aspera/cli/plugins/faspex.rb +29 -7
  19. data/lib/aspera/cli/plugins/faspex5.rb +21 -8
  20. data/lib/aspera/cli/plugins/node.rb +21 -9
  21. data/lib/aspera/cli/plugins/orchestrator.rb +5 -3
  22. data/lib/aspera/cli/plugins/preview.rb +2 -2
  23. data/lib/aspera/cli/plugins/server.rb +3 -3
  24. data/lib/aspera/cli/plugins/shares.rb +17 -0
  25. data/lib/aspera/cli/transfer_agent.rb +47 -85
  26. data/lib/aspera/cli/version.rb +1 -1
  27. data/lib/aspera/environment.rb +4 -4
  28. data/lib/aspera/fasp/{manager.rb → agent_base.rb} +7 -6
  29. data/lib/aspera/fasp/{connect.rb → agent_connect.rb} +46 -39
  30. data/lib/aspera/fasp/{local.rb → agent_direct.rb} +14 -17
  31. data/lib/aspera/fasp/{http_gw.rb → agent_httpgw.rb} +4 -4
  32. data/lib/aspera/fasp/{node.rb → agent_node.rb} +25 -8
  33. data/lib/aspera/fasp/agent_trsdk.rb +106 -0
  34. data/lib/aspera/fasp/default.rb +17 -0
  35. data/lib/aspera/fasp/installation.rb +64 -48
  36. data/lib/aspera/fasp/parameters.rb +7 -3
  37. data/lib/aspera/faspex_gw.rb +6 -6
  38. data/lib/aspera/keychain/encrypted_hash.rb +120 -0
  39. data/lib/aspera/keychain/macos_security.rb +94 -0
  40. data/lib/aspera/log.rb +45 -32
  41. data/lib/aspera/node.rb +3 -6
  42. data/lib/aspera/rest.rb +65 -49
  43. metadata +68 -27
  44. data/lib/aspera/api_detector.rb +0 -60
  45. data/lib/aspera/secrets.rb +0 -20
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
2
2
  <font size="+12"><center>`ascli` : Command Line Interface for IBM Aspera products</center></font>
3
3
 
4
- Version : 4.4.0
4
+ Version : 4.5.0
5
5
 
6
6
  _Laurent/2016-2021_
7
7
 
@@ -15,7 +15,9 @@ Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/ge
15
15
 
16
16
  Required Ruby version: > 2.4
17
17
 
18
- # <a name="when_to_use"></a>When to use and when not to use
18
+ [Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
19
+
20
+ # <a id="when_to_use"></a>When to use and when not to use
19
21
 
20
22
  `ascli` is designed to be used as a command line tool to:
21
23
 
@@ -38,21 +40,18 @@ One might be tempted to use it as an integration element, e.g. by building a com
38
40
  For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
39
41
 
40
42
  * Product APIs (REST) : e.g. AoC, Faspex, node
41
- * Transfer SDK : with gRPC interface and laguage stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
43
+ * Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
42
44
 
43
45
  Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
44
46
 
45
47
  For scripting and ad'hoc command line operations, `ascli` is perfect.
46
48
 
47
- # Notations
49
+ # <a id="parsing"></a>Notations, Shell and Command line parsing
48
50
 
49
- In examples, command line operations (starting with `$`) are shown using a standard shell: `bash` or `zsh`.
50
- Prompt `# ` refers to user `root`, prompt `xfer$ ` refer to user `xfer`.
51
+ In examples, command line operations are shown using a shell such: `bash` or `zsh`.
51
52
 
52
53
  Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
53
54
 
54
- # <a name="parsing"></a>Shell and Command line parsing
55
-
56
55
  `ascli` is typically executed in a shell, either interactively or in a script. `ascli` receives its arguments from this shell.
57
56
 
58
57
  On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before `ascli` (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to `ascli`. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
@@ -65,12 +64,15 @@ On Windows, `cmd.exe` is typically used. Windows process creation does not recei
65
64
  In case of doubt of argument values after parsing test like this:
66
65
 
67
66
  ```
68
- $ ascli conf echo "Hello World" arg2 3
67
+ ascli conf echo "Hello World" arg2 3
68
+ ```
69
+
70
+ ```
69
71
  "Hello World"
70
72
  ERROR: Argument: unprocessed values: ["arg2", "3"]
71
73
  ```
72
74
 
73
- `echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (ascli), next command line arguments are shown in the error message.
75
+ `echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (`ascli`), next command line arguments are shown in the error message.
74
76
 
75
77
  # Quick Start
76
78
 
@@ -81,8 +83,11 @@ First, follow the section: [Installation](#installation) (Ruby, Gem, FASP) to st
81
83
  Once the gem is installed, `ascli` shall be accessible:
82
84
 
83
85
  ```
84
- $ ascli --version
85
- 4.4.0
86
+ ascli --version
87
+ ```
88
+
89
+ ```
90
+ 4.5.0
86
91
  ```
87
92
 
88
93
  ## First use
@@ -94,8 +99,14 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
94
99
  To test with Aspera demo transfer server, setup the environment and then test:
95
100
 
96
101
  ```
97
- $ ascli config initdemo
98
- $ ascli server browse /
102
+ ascli config initdemo
103
+ ```
104
+
105
+ ```
106
+ ascli server browse /
107
+ ```
108
+
109
+ ```
99
110
  :............:...........:......:........:...........................:.......................:
100
111
  : zmode : zuid : zgid : size : mtime : name :
101
112
  :............:...........:......:........:...........................:.......................:
@@ -114,11 +125,26 @@ If you want to use `ascli` with another server, and in order to make further cal
114
125
  * download a file
115
126
 
116
127
  ```
117
- $ ascli config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
128
+ ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
129
+ ```
130
+
131
+ ```
118
132
  updated: myserver
119
- $ ascli config id default set server myserver
133
+ ```
134
+
135
+ ```
136
+ ascli config preset set default server myserver
137
+ ```
138
+
139
+ ```
120
140
  updated: default&rarr;server to myserver
121
- $ ascli server browse /aspera-test-dir-large
141
+ ```
142
+
143
+ ```
144
+ ascli server browse /aspera-test-dir-large
145
+ ```
146
+
147
+ ```
122
148
  :............:...........:......:..............:...........................:............................:
123
149
  : zmode : zuid : zgid : size : mtime : name :
124
150
  :............:...........:......:..............:...........................:............................:
@@ -135,7 +161,13 @@ $ ascli server browse /aspera-test-dir-large
135
161
  : -rw-r--r-- : asperaweb : fasp : 104857600 : 2014-04-10 19:49:29 +0200 : 100MB :
136
162
  : -rw-r--r-- : asperaweb : fasp : 10737418240 : 2014-04-10 19:49:04 +0200 : 10GB :
137
163
  :............:...........:......:..............:...........................:............................:
138
- $ ascli server download /aspera-test-dir-large/200MB
164
+ ```
165
+
166
+ ```
167
+ ascli server download /aspera-test-dir-large/200MB
168
+ ```
169
+
170
+ ```
139
171
  Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
140
172
  complete
141
173
  ```
@@ -146,9 +178,9 @@ Get familiar with configuration, options, commands : [Command Line Interface](#c
146
178
 
147
179
  Then, follow the section relative to the product you want to interact with ( Aspera on Cloud, Faspex, ...) : [Application Plugins](plugins)
148
180
 
149
- # <a name="installation"></a>Installation
181
+ # <a id="installation"></a>Installation
150
182
 
151
- It is possible to install *either* directly on the host operating system (Linux, Windows, Macos) or as a docker container.
183
+ It is possible to install *either* directly on the host operating system (Linux, Windows, macOS) or as a docker container.
152
184
 
153
185
  The direct installation is recommended and consists in installing:
154
186
 
@@ -158,33 +190,36 @@ The direct installation is recommended and consists in installing:
158
190
 
159
191
  The following sections provide information on the various installation methods.
160
192
 
161
- An internet connection is required for the installation. If you dont have internet for the installation, refer to section [Installation without internet access](#offline_install).
193
+ An internet connection is required for the installation. If you don't have internet for the installation, refer to section [Installation without internet access](#offline_install).
162
194
 
163
195
  ## Docker container
164
196
 
165
197
  Use this method only if you know what you do, else use the standard recommended method as described here above.
166
198
 
167
- This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
199
+ This method installs a docker image that contains: Ruby, `ascli` and the FASP sdk.
168
200
 
169
201
  The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli)
170
202
 
171
203
  Ensure that you have Docker installed.
172
204
 
173
205
  ```
174
- $ docker --version
206
+ docker --version
175
207
  ```
176
208
 
177
209
  Download the wrapping script:
178
210
 
179
211
  ```
180
- $ curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
181
- $ chmod a+x ascli
212
+ curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
213
+ ```
214
+
215
+ ```
216
+ chmod a+x ascli
182
217
  ```
183
218
 
184
219
  Install the container image:
185
220
 
186
221
  ```
187
- $ ./ascli install
222
+ ./ascli install
188
223
  ```
189
224
 
190
225
  Start using it !
@@ -194,9 +229,9 @@ Note that the tool is run in the container, so transfers are also executed in th
194
229
  The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/ascli` on host.
195
230
 
196
231
  To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
197
- To add local storage as a volume edit the script: ascli and add a `--volume` stanza.
232
+ To add local storage as a volume edit the script: `ascli` and add a `--volume` stanza.
198
233
 
199
- ## <a name="ruby"></a>Ruby
234
+ ## <a id="ruby"></a>Ruby
200
235
 
201
236
  Use this method to install on the native host.
202
237
 
@@ -208,7 +243,7 @@ Required Ruby version: > 2.4. Ruby version 3 is also supported.
208
243
 
209
244
  Refer to the following sections for a proposed method for specific operating systems.
210
245
 
211
- The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
246
+ The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, macOS, Windows with cygwin, etc...).
212
247
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
213
248
  If you have a simpler better way to install Ruby version > 2.4 : use it !
214
249
 
@@ -221,31 +256,31 @@ Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
221
256
  Install the 2 keys
222
257
 
223
258
  ```
224
- $ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
259
+ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
225
260
  ```
226
261
 
227
262
  Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
228
263
 
229
264
  ```
230
- $ \curl -sSL https://get.rvm.io | bash -s stable
265
+ \curl -sSL https://get.rvm.io | bash -s stable
231
266
  ```
232
267
 
233
- If you keep the same terminal (ont needed if re-login):
268
+ If you keep the same terminal (not needed if re-login):
234
269
 
235
270
  ```
236
- $ source ~/.rvm/scripts/rvm
271
+ source ~/.rvm/scripts/rvm
237
272
  ```
238
273
 
239
274
  It is advised to get one of the pre-compiled ruby version, you can list with:
240
275
 
241
276
  ```
242
- $ rvm list --remote
277
+ rvm list --remote
243
278
  ```
244
279
 
245
280
  Install the chosen pre-compiled Ruby version:
246
281
 
247
282
  ```
248
- $ rvm install 2.7.2 --binary
283
+ rvm install 2.7.2 --binary
249
284
  ```
250
285
 
251
286
  Ruby is now installed for the user, go on to Gem installation.
@@ -258,7 +293,7 @@ As root, it installs by default in /usr/local/rvm for all users and creates `/et
258
293
  One can install in another location with :
259
294
 
260
295
  ```
261
- # curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
296
+ curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
262
297
  ```
263
298
 
264
299
  As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
@@ -266,8 +301,11 @@ If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d
266
301
  To activate ruby (and ascli) later, source it:
267
302
 
268
303
  ```
269
- # source /etc/profile.d/rvm.sh.ok
270
- # rvm version
304
+ source /etc/profile.d/rvm.sh.ok
305
+ ```
306
+
307
+ ```
308
+ rvm version
271
309
  ```
272
310
 
273
311
  ### Windows: Installer
@@ -280,16 +318,16 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
280
318
 
281
319
  ### macOS: pre-installed or `brew`
282
320
 
283
- MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
321
+ macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
284
322
 
285
323
  ```
286
- $ sudo gem install aspera-cli
324
+ sudo gem install aspera-cli
287
325
  ```
288
326
 
289
327
  Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
290
328
 
291
329
  ```
292
- $ brew install ruby
330
+ brew install ruby
293
331
  ```
294
332
 
295
333
  ### Linux: package
@@ -299,90 +337,56 @@ If your Linux distribution provides a standard ruby package, you can use it prov
299
337
  Example:
300
338
 
301
339
  ```
302
- # yum install -y ruby rubygems ruby-json
340
+ yum install -y ruby rubygems ruby-json
303
341
  ```
304
342
 
305
343
  One can cleanup the whole yum-installed ruby environment like this to uninstall:
306
344
 
307
345
  ```
308
346
  gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
347
+ ```
348
+
349
+ ```
309
350
  yum remove -y ruby ruby-libs
310
351
  ```
311
352
 
312
353
  ### Other Unixes: Aix, etc...
313
354
 
314
- If your unix do not provide a pre-built ruby, you can get it using one of those
355
+ If your Unix does not provide a pre-built ruby, you can get it using one of those
315
356
  [methods](https://www.ruby-lang.org/en/documentation/installation/)
316
357
 
317
358
  For instance to build from source, and install in `/opt/ruby` :
318
359
 
319
360
  ```
320
- # wget https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.gz
321
- # gzip -d ruby-2.7.2.tar.gz
322
- # tar xvf ruby-2.7.2.tar
323
- # cd ruby-2.7.2
324
- # ./configure --prefix=/opt/ruby
325
- # make ruby.imp
326
- # make
327
- # make install
328
- ```
329
-
330
- ### <a name="offline_install"></a>Installation without internet access
331
-
332
- Note that currently no pre-packaged version exist yet.
333
- A method to build one provided here:
334
-
335
- On a server with the same OS version and with internet access follow the "Generic single user installation" method.
336
-
337
- Then create an archive:
338
-
339
- ```
340
- $ cd
341
- $ tar zcvf rvm-ascli.tgz .rvm
342
- ```
343
-
344
- Get the Aspera SDK. Execute:
345
-
346
- ```
347
- $ ascli conf --show-config|grep sdk_url
348
- ```
349
-
350
- Then download the SDK archive from that URL.
361
+ wget https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.gz
351
362
 
352
- Another method for the SDK is to install the SDK (`ascli conf ascp install`) on the first system, and archive `$HOME/.aspera`.
363
+ gzip -d ruby-2.7.2.tar.gz
353
364
 
354
- Transfer those 2 archives to the target system without internet access.
365
+ tar xvf ruby-2.7.2.tar
355
366
 
356
- On the target system:
367
+ cd ruby-2.7.2
357
368
 
358
- * Extract the RVM archive either in a global location, or in a user's home folder : `path_to_rvm_root`
359
- * in the user's `.profile` add this line: (replace `path_to_rvm_root` with the actual location)
369
+ ./configure --prefix=/opt/ruby
360
370
 
361
- ```
362
- source path_to_rvm_root/scripts/rvm
363
- rvm use 2.7.2
364
- ```
371
+ make ruby.imp
365
372
 
366
- For the SDK, either install from archive:
373
+ make
367
374
 
375
+ make install
368
376
  ```
369
- $ ascli conf ascp install --sdk-url=file:///SDK.zip
370
- ```
371
-
372
- or restore the `$HOME/.aspera` folder for the user.
373
377
 
374
- ## <a name="the_gem"></a>`aspera-cli` gem
378
+ ## <a id="the_gem"></a>`aspera-cli` gem
375
379
 
376
380
  Once you have Ruby and rights to install gems: Install the gem and its dependencies:
377
381
 
378
382
  ```
379
- # gem install aspera-cli
383
+ gem install aspera-cli
380
384
  ```
381
385
 
382
386
  To upgrade to the latest version:
383
387
 
384
388
  ```
385
- # gem update aspera-cli
389
+ gem update aspera-cli
386
390
  ```
387
391
 
388
392
  `ascli` checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
@@ -390,12 +394,10 @@ To upgrade to the latest version:
390
394
  To check manually:
391
395
 
392
396
  ```
393
- # ascli conf check_update
397
+ ascli conf check_update
394
398
  ```
395
399
 
396
-
397
-
398
- ## <a name="fasp_prot"></a>FASP Protocol
400
+ ## <a id="fasp_prot"></a>FASP Protocol
399
401
 
400
402
  Most file transfers will be done using the FASP protocol, using `ascp`.
401
403
  Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
@@ -403,17 +405,20 @@ Only two additional files are required to perform an Aspera Transfer, which are
403
405
  * ascp
404
406
  * aspera-license (in same folder, or ../etc)
405
407
 
406
- This can be installed either be installing an Aspera transfer sofware, or using an embedded command:
408
+ This can be installed either be installing an Aspera transfer software, or using an embedded command:
407
409
 
408
410
  ```
409
- $ ascli conf ascp install
411
+ ascli conf ascp install
410
412
  ```
411
413
 
412
- If a local SDK installation is prefered instead of fetching from internet: one can specify the location of the SDK file:
414
+ If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
413
415
 
414
416
  ```
415
- $ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
416
- $ ascli conf ascp install --sdk-url=file:///SDK.zip
417
+ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
418
+ ```
419
+
420
+ ```
421
+ ascli conf ascp install --sdk-url=file:///SDK.zip
417
422
  ```
418
423
 
419
424
  The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
@@ -433,36 +438,55 @@ by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/
433
438
  Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
434
439
 
435
440
  Several methods are provided to start a transfer.
436
- Use of a local client ([`direct`](#direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
441
+ Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
437
442
 
438
- ## <a name="offline_install"></a>Offline Installation (without internet)
443
+ ## <a id="offline_install"></a>Installation in air gapped environment
439
444
 
440
- The procedure consists in:
445
+ Note that currently no pre-packaged version exist yet.
446
+ A method to build one is provided here:
447
+
448
+ The procedure:
441
449
 
442
450
  * Follow the non-root installation procedure with RVM, including gem
443
- * archive (zip, tar) the main RVM folder (includes ascli):
451
+ * Archive (zip, tar) the main RVM folder (includes ascli):
444
452
 
445
453
  ```
446
- $ cd ~
447
- $ tar zcvf rvm_ascli.tgz .rvm
454
+ cd $HOME && tar zcvf rvm-ascli.tgz .rvm
448
455
  ```
449
456
 
450
- * retrieve the SDK:
457
+ * Get the Aspera SDK.
451
458
 
452
459
  ```
453
- $ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
460
+ ascli conf --show-config --fields=sdk_url
454
461
  ```
455
462
 
456
- * on the system without internet access:
463
+ * Download the SDK archive from that URL.
457
464
 
458
465
  ```
459
- $ cd ~
460
- $ tar zxvf rvm_ascli.tgz
461
- $ source ~/.rvm/scripts/rvm
462
- $ ascli conf ascp install --sdk-url=file:///SDK.zip
466
+ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
463
467
  ```
464
468
 
465
- # <a name="cli"></a>Command Line Interface: `ascli`
469
+ * Transfer those 2 files to the target system
470
+
471
+ * On target system
472
+
473
+ ```
474
+ cd $HOME
475
+
476
+ tar zxvf rvm-ascli.tgz
477
+
478
+ source ~/.rvm/scripts/rvm
479
+
480
+ ascli conf ascp install --sdk-url=file:///SDK.zip
481
+ ```
482
+
483
+ * Add those lines to shell init (`.profile`)
484
+
485
+ ```
486
+ source ~/.rvm/scripts/rvm
487
+ ```
488
+
489
+ # <a id="cli"></a>Command Line Interface: `ascli`
466
490
 
467
491
  The `aspera-cli` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
468
492
 
@@ -491,7 +515,7 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
491
515
  Basic usage is displayed by executing:
492
516
 
493
517
  ```
494
- $ ascli -h
518
+ ascli -h
495
519
  ```
496
520
 
497
521
  Refer to sections: [Usage](#usage) and [Sample Commands](#commands).
@@ -505,7 +529,7 @@ Arguments are the units of command line, as parsed by the shell, typically separ
505
529
  There are two types of arguments: Commands and Options. Example :
506
530
 
507
531
  ```
508
- $ ascli command --option-name=VAL1 VAL2
532
+ ascli command --option-name=VAL1 VAL2
509
533
  ```
510
534
 
511
535
  * executes _command_: `command`
@@ -533,13 +557,16 @@ Exceptions:
533
557
  * the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
534
558
 
535
559
  ```
536
- $ ascli config echo -- --sample
560
+ ascli config echo -- --sample
561
+ ```
562
+
563
+ ```
537
564
  "--sample"
538
565
  ```
539
566
 
540
567
  Note that `--sample` is taken as an argument, and not option.
541
568
 
542
- Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on comand line and evaluated in order.
569
+ Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
543
570
 
544
571
  The value for _any_ options can come from the following locations (in this order, last value evaluated overrides previous value):
545
572
 
@@ -560,7 +587,7 @@ Command line arguments that are not options are either commands or arguments. If
560
587
 
561
588
  Some options and parameters are mandatory and other optional. By default, the tool will ask for missing mandatory options or parameters for interactive execution.
562
589
 
563
- The behaviour can be controlled with:
590
+ The behavior can be controlled with:
564
591
 
565
592
  * --interactive=&lt;yes|no&gt; (default=yes if STDIN is a terminal, else no)
566
593
  * yes : missing mandatory parameters/options are asked to the user
@@ -577,8 +604,8 @@ The information displayed depends on the action.
577
604
 
578
605
  Depending on action, the output will contain:
579
606
 
580
- * `single_object` : displayed as a 2 dimensional table: one line per attribute, first column is attribute name, and second is atteribute value. Nested hashes are collapsed.
581
- * `object_list` : displayed as a 2 dimensional table: one line per item, one colum per attribute.
607
+ * `single_object` : displayed as a 2 dimensional table: one line per attribute, first column is attribute name, and second is attribute value. Nested hashes are collapsed.
608
+ * `object_list` : displayed as a 2 dimensional table: one line per item, one column per attribute.
582
609
  * `value_list` : a table with one column.
583
610
  * `empty` : nothing
584
611
  * `status` : a message
@@ -603,12 +630,15 @@ The style of output can be set using the `format` parameter, supporting:
603
630
  * `yaml` : YAML
604
631
  * `csv` : Comma Separated Values
605
632
 
606
- ### <a name="option_select"></a>Option: `select`: Filter on columns values for `object_list`
633
+ ### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
607
634
 
608
635
  Table output can be filtered using the `select` parameter. Example:
609
636
 
610
637
  ```
611
- $ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
638
+ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
639
+ ```
640
+
641
+ ```
612
642
  :...............................:..................................:...........:
613
643
  : name : email : ats_admin :
614
644
  :...............................:..................................:...........:
@@ -621,7 +651,7 @@ Note that `select` filters selected elements from the result of API calls, while
621
651
 
622
652
  ### Verbosity of output
623
653
 
624
- Outpout messages are categorized in 3 types:
654
+ Output messages are categorized in 3 types:
625
655
 
626
656
  * `info` output contain additional information, such as number of elements in a table
627
657
  * `data` output contain the actual output of the command (object, or list of objects)
@@ -635,7 +665,7 @@ The option `display` controls the level of output:
635
665
 
636
666
  ### Selection of output object properties
637
667
 
638
- By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific colums to be displayed, by setting the `fields` option to one of the following value:
668
+ By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
639
669
 
640
670
  * DEF : default display of columns (that's the default, when not set)
641
671
  * ALL : all columns available
@@ -644,7 +674,7 @@ By default, a table output will display one line per entry, and columns for each
644
674
  * +a,b,c : add selected properties to the default selection.
645
675
  * -a,b,c : remove selected properties from the default selection.
646
676
 
647
- ## <a name="extended"></a>Extended Value Syntax
677
+ ## <a id="extended"></a>Extended Value Syntax
648
678
 
649
679
  Usually, values of options and arguments are specified by a simple string. But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
650
680
 
@@ -681,23 +711,32 @@ To display the result of an extended value, use the `config echo` command.
681
711
  Example: read the content of the specified file, then, base64 decode, then unzip:
682
712
 
683
713
  ```
684
- $ ascli config echo @zlib:@base64:@file:myfile.dat
714
+ ascli config echo @zlib:@base64:@file:myfile.dat
685
715
  ```
686
716
 
687
717
  Example: create a value as a hash, with one key and the value is read from a file:
688
718
 
689
719
  ```
690
- $ ascli config echo @ruby:'{"token_verification_key"=>File.read("pubkey.txt")}'
720
+ ascli config echo @ruby:'{"token_verification_key"=>File.read("pubkey.txt")}'
691
721
  ```
692
722
 
693
723
  Example: read a csv file and create a list of hash for bulk provisioning:
694
724
 
695
725
  ```
696
- $ cat test.csv
726
+ cat test.csv
727
+ ```
728
+
729
+ ```
697
730
  name,email
698
731
  lolo,laurent@example.com
699
732
  toto,titi@tutu.tata
700
- $ ascli config echo @csvt:@file:test.csv
733
+ ```
734
+
735
+ ```
736
+ ascli config echo @csvt:@file:test.csv
737
+ ```
738
+
739
+ ```
701
740
  :......:.....................:
702
741
  : name : email :
703
742
  :......:.....................:
@@ -709,13 +748,16 @@ $ ascli config echo @csvt:@file:test.csv
709
748
  Example: create a hash and include values from preset named "config" of config file in this hash
710
749
 
711
750
  ```
712
- $ ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
751
+ ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
752
+ ```
753
+
754
+ ```
713
755
  {"version"=>"0.9", "hello"=>true}
714
756
  ```
715
757
 
716
758
  Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
717
759
 
718
- ## <a name="native"></a>Structured Value
760
+ ## <a id="native"></a>Structured Value
719
761
 
720
762
  Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
721
763
 
@@ -726,7 +768,7 @@ A convenient way to specify a _Structured Value_ is to use the `@json:` decoder,
726
768
 
727
769
  It is also possible to provide a _Structured Value_ in a file using `@json:@file:<path>`
728
770
 
729
- ## <a name="conffolder"></a>Configuration and Persistency Folder
771
+ ## <a id="conffolder"></a>Configuration and Persistency Folder
730
772
 
731
773
  `ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/ascli`.
732
774
 
@@ -736,49 +778,52 @@ It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMED
736
778
  The main folder can be displayed using :
737
779
 
738
780
  ```
739
- $ ascli config folder
781
+ ascli config folder
782
+ ```
783
+
784
+ ```
740
785
  /Users/kenji/.aspera/ascli
741
786
  ```
742
787
 
743
- It can be overriden using the envinonment variable `ASCLI_HOME`.
788
+ It can be overridden using the environment variable `ASCLI_HOME`.
744
789
 
745
790
  Example (Windows):
746
791
 
747
792
  ```
748
- $ set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli
749
- $ ascli config folder
793
+ set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli
794
+ ascli config folder
750
795
  C:\Users\Kenji\.aspera\ascli
751
796
  ```
752
797
 
753
- ## <a name="configfile"></a>Configuration file
798
+ ## <a id="configfile"></a>Configuration file
754
799
 
755
800
  On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
756
801
  Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
757
802
 
758
803
  Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
759
804
 
760
- All options for `ascli` can be set on command line, or by env vars, or using [option presets](#lprt) in the configuratin file.
805
+ All options for `ascli` can be set on command line, or by env vars, or using [option presets](#lprt) in the configuration file.
761
806
 
762
807
  A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
763
808
 
764
- The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overriden with option `--config-file=path` or equivalent env var).
809
+ The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or equivalent env var).
765
810
 
766
811
  The configuration file is simply a catalog of pre-defined lists of options, called: [option presets](#lprt). Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [option preset](#lprt) (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
767
812
 
768
- ### <a name="lprt"></a>Option preset
813
+ ### <a id="lprt"></a>Option preset
769
814
 
770
815
  A [option preset](#lprt) is simply a collection of parameters and their associated values in a named section in the configuration file.
771
816
 
772
817
  A named [option preset](#lprt) can be modified directly using `ascli`, which will update the configuration file :
773
818
 
774
819
  ```
775
- $ ascli config id <option preset> set|delete|show|initialize|update
820
+ ascli config preset set|delete|show|initialize|update <option preset>
776
821
  ```
777
822
 
778
823
  The command `update` allows the easy creation of [option preset](#lprt) by simply providing the options in their command line format, e.g. :
779
824
 
780
825
  ```
781
- $ ascli config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
826
+ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
782
827
  ```
783
828
 
784
829
  * This creates a [option preset](#lprt) `demo_server` with all provided options.
@@ -786,39 +831,48 @@ $ ascli config id demo_server update --url=ssh://demo.asperasoft.com:33001 --use
786
831
  The command `set` allows setting individual options in a [option preset](#lprt).
787
832
 
788
833
  ```
789
- $ ascli config id demo_server set password _demo_pass_
834
+ ascli config preset set demo_server password _demo_pass_
790
835
  ```
791
836
 
792
837
  The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
793
838
 
794
839
  ```
795
- $ ascli config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
840
+ ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
796
841
  ```
797
842
 
798
- A good practice is to not manually edit the configuration file and use modification commands instead.
799
- If necessary, the configuration file can be edited (or simply consulted) with:
843
+ A full terminal based overview of the configuration can be displayed using:
800
844
 
801
845
  ```
802
- $ ascli config open
846
+ ascli config preset over
803
847
  ```
804
848
 
805
- A full terminal based overview of the configuration can be displayed using:
849
+ A list of [option preset](#lprt) can be displayed using:
806
850
 
807
851
  ```
808
- $ ascli config over
852
+ ascli config preset list
809
853
  ```
810
854
 
811
- A list of [option preset](#lprt) can be displayed using:
855
+ A good practice is to not manually edit the configuration file and use modification commands instead.
856
+ If necessary, the configuration file can opened in a text editor with:
857
+
858
+ ```
859
+ ascli config open
860
+ ```
861
+
862
+ Older format for commands are still supported:
812
863
 
813
864
  ```
814
- $ ascli config list
865
+ ascli config id <name> set|delete|show|initialize|update
866
+ ascli config over
867
+ ascli config list
815
868
  ```
816
869
 
817
- ### <a name="lprtconf"></a>Special Option preset: config
870
+
871
+ ### <a id="lprtconf"></a>Special Option preset: config
818
872
 
819
873
  This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
820
874
 
821
- ### <a name="lprtdef"></a>Special Option preset: default
875
+ ### <a id="lprtdef"></a>Special Option preset: default
822
876
 
823
877
  This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
824
878
 
@@ -829,16 +883,16 @@ Note that special plugin name: `config` can be associated with a preset that is
829
883
  Operations on this preset are done using regular `config` operations:
830
884
 
831
885
  ```
832
- $ ascli config id default set _plugin_name_ _default_preset_for_plugin_
833
- $ ascli config id default get _plugin_name_
886
+ ascli config preset set default _plugin_name_ _default_preset_for_plugin_
887
+ ascli config preset get default _plugin_name_
834
888
  "_default_preset_for_plugin_"
835
889
  ```
836
890
 
837
- ### <a name="lprtdef"></a>Special Plugin: config
891
+ ### <a id="lplugconf"></a>Special Plugin: config
838
892
 
839
893
  Plugin `config` (not to be confused with Option preset config) is used to configure `ascli` but it also contains global options.
840
894
 
841
- When `ascli` starts, it lookjs for the `default` Option preset and if there is a value for `config`, if so, it loads the option values for any plugin used.
895
+ When `ascli` starts, it looks for the `default` Option preset and if there is a value for `config`, if so, it loads the option values for any plugin used.
842
896
 
843
897
  If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
844
898
 
@@ -877,10 +931,10 @@ The user may create as many [option presets](#lprt) as needed. For instance, a p
877
931
 
878
932
  Values in the configuration also follow the [Extended Value Syntax](#extended).
879
933
 
880
- Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config id update` command, the user shall use the `@val:` prefix. Example:
934
+ Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
881
935
 
882
936
  ```
883
- $ ascli config id my_aoc_org set private_key @val:@file:"$HOME/.aspera/ascli/aocapikey"
937
+ ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/aocapikey"
884
938
  ```
885
939
 
886
940
  This creates the [option preset](#lprt):
@@ -916,26 +970,29 @@ is an underscore. E.g. --xxx-yyy on command line gives xxx_yyy in configuration
916
970
  The main plugin name is `config`, so it is possible to define a default [option preset](#lprt) for the main plugin with:
917
971
 
918
972
  ```
919
- $ ascli config id cli_default set interactive no
920
- $ ascli config id default set config cli_default
973
+ ascli config preset set cli_default interactive no
974
+ ```
975
+
976
+ ```
977
+ ascli config preset set default config cli_default
921
978
  ```
922
979
 
923
980
  A [option preset](#lprt) value can be removed with `unset`:
924
981
 
925
982
  ```
926
- $ ascli config id cli_default unset interactive
983
+ ascli config preset unset cli_default interactive
927
984
  ```
928
985
 
929
986
  Example: Define options using command line:
930
987
 
931
988
  ```
932
- $ ascli -N --url=x --password=y --username=y node --show-config
989
+ ascli -N --url=x --password=y --username=y node --show-config
933
990
  ```
934
991
 
935
992
  Example: Define options using a hash:
936
993
 
937
994
  ```
938
- $ ascli -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
995
+ ascli -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
939
996
  ```
940
997
 
941
998
  ### Examples
@@ -945,44 +1002,120 @@ only username/password and url are required (either on command line, or from con
945
1002
  Those can usually be provided on the command line:
946
1003
 
947
1004
  ```
948
- $ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
1005
+ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
949
1006
  ```
950
1007
 
951
1008
  This can also be provisioned in a config file:
952
1009
 
1010
+ * Build [option preset](#lprt)
1011
+
953
1012
  ```
954
- 1$ ascli config id shares06 set url https://10.25.0.6
955
- 2$ ascli config id shares06 set username john
956
- 3$ ascli config id shares06 set password 4sp3ra
957
- 4$ ascli config id default set shares shares06
958
- 5$ ascli config overview
959
- 6$ ascli shares repo browse /
1013
+ ascli config preset set shares06 url https://10.25.0.6
1014
+ ascli config preset set shares06 username john
1015
+ ascli config preset set shares06 password 4sp3ra
960
1016
  ```
961
1017
 
962
- The three first commands build a [option preset](#lprt).
963
1018
  Note that this can also be done with one single command:
964
1019
 
965
1020
  ```
966
- $ ascli config id shares06 init @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
1021
+ ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
967
1022
  ```
968
1023
 
969
- The fourth command defines this [option preset](#lprt) as the default [option preset](#lprt) for the
970
- specified application ("shares"). The 5th command displays the content of configuration file in table format.
971
- Alternative [option presets](#lprt) can be used with option "-P&lt;[option preset](#lprt)&gt;"
972
- (or --preset=&lt;[option preset](#lprt)&gt;)
1024
+ or
1025
+
1026
+ ```
1027
+ ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=4sp3ra
1028
+ ```
1029
+
1030
+ * Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
1031
+
1032
+ ```
1033
+ ascli config preset set default shares shares06
1034
+ ```
1035
+
1036
+ * Display the content of configuration file in table format
1037
+
1038
+ ```
1039
+ ascli config overview
1040
+ ```
1041
+
1042
+ * Execute a command on the shares application using default parameters
1043
+
1044
+ ```
1045
+ ascli shares repo browse /
1046
+ ```
1047
+
1048
+ ## <a id="vault"></a>Secret Vault
1049
+
1050
+ When a secret or password is needed, it is possible to store in the secret vault.
1051
+
1052
+ By default the vault is defined using option `secrets`.
1053
+
1054
+ ### Using system keychain
1055
+
1056
+ Only on macOS.
1057
+
1058
+ It is possible to store secrets in macOS keychain (only read supported currently).
1059
+
1060
+ Set option `secrets` to value `system` to use the default keychain or use value `system:[name]` to use a custom keychain.
1061
+
1062
+ ### Modern config file format: encrypted in config file
973
1063
 
974
- Eventually, the last command shows a call to the shares application using default parameters.
1064
+ It is possible to store and use secrets encrypted.
1065
+ For this use the `config vault` command.
975
1066
 
1067
+ The vault can be initialized with `config vault init`
1068
+
1069
+ Then secrets can be manipulated using commands:
1070
+
1071
+ * `set`
1072
+ * `get`
1073
+ * `list`
1074
+ * `delete`
1075
+
1076
+ Secrets must be uniquely identified by `url` and `username`. An optional description can be provided using option `value`.
1077
+
1078
+ ### Legacy config file format
1079
+
1080
+ The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
1081
+
1082
+ For example, choose a repository name, for example `my_secrets`, and populate it like this:
1083
+
1084
+ ```
1085
+ ascli conf id my_secrets set 'access_key1' 'secret1'
1086
+
1087
+ ascli conf id my_secrets set 'access_key2' 'secret2'
1088
+
1089
+ ascli conf id default get config
1090
+
1091
+ cli_default
1092
+ ```
1093
+
1094
+ Here above, one has already set a `config` global preset to preset `cli_default` (refer to earlier in documentation).
1095
+ So the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
1096
+
1097
+ ```
1098
+ ascli conf id cli_default set secrets @val:@preset:my_secrets
1099
+ ```
1100
+
1101
+ A secret repository can always be selected at runtime using `--secrets=@preset:xxxx`, or `--secrets=@json:'{"accesskey1":"secret1"}'`
1102
+
1103
+ To test if a secret can be found use:
1104
+
1105
+ ```
1106
+ ascli conf vault get --username=access_key1
1107
+ ```
976
1108
 
977
1109
  ## Plugins
978
1110
 
979
1111
  The CLI tool uses a plugin mechanism. The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command. Each plugin usually represent commands sent to a specific application.
980
1112
  For instance, the plugin "faspex" allows operations on the application "Aspera Faspex".
981
1113
 
982
- ### Create your own plugin
1114
+ ### <a id="createownplugin"></a>Create your own plugin
1115
+
983
1116
  ```
984
- $ mkdir -p ~/.aspera/ascli/plugins
985
- $ cat<<EOF>~/.aspera/ascli/plugins/test.rb
1117
+ mkdir -p ~/.aspera/ascli/plugins
1118
+ cat<<EOF>~/.aspera/ascli/plugins/test.rb
986
1119
  require 'aspera/cli/plugin'
987
1120
  module Aspera
988
1121
  module Cli
@@ -997,39 +1130,78 @@ end # Aspera
997
1130
  EOF
998
1131
  ```
999
1132
 
1000
- ## Debugging
1133
+ ### <a id="plugins"></a>Plugins: Application URL and Authentication
1001
1134
 
1002
- The gem is equipped with traces. By default logging level is "warn". To increase debug level, use parameter `log_level`, so either command line `--log-level=xx` or env var `ASCLI_LOG_LEVEL`.
1135
+ `ascli` comes with several Aspera application plugins.
1003
1136
 
1004
- It is also possible to activate traces before initialisation using env var `AS_LOG_LEVEL`.
1137
+ REST APIs of Aspera legacy applications (Aspera Node, Faspex, Shares, Console, Orchestrator, Server) use simple username/password authentication: HTTP Basic Authentication.
1138
+
1139
+ Those are using options:
1140
+
1141
+ * url
1142
+ * username
1143
+ * password
1144
+
1145
+ Those can be provided using command line, parameter set, env var, see section above.
1146
+
1147
+ Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
1148
+
1149
+ ## Logging, Debugging
1150
+
1151
+ The gem is equipped with traces. By default logging level is `warn`.
1152
+ To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or parameter in con file).
1153
+
1154
+ It is also possible to activate traces before initialization using env var `AS_LOG_LEVEL`.
1155
+
1156
+ By default passwords and secrets are removed from logs.
1157
+ Use option `log_passwords` to change this behaviour.
1005
1158
 
1006
1159
  ## Learning Aspera Product APIs (REST)
1007
1160
 
1008
1161
  This CLI uses REST APIs.
1009
- To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display
1010
- exact content or HTTP requests and responses.
1162
+ To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
1011
1163
 
1012
1164
  In order to get traces of execution, use argument : `--log-level=debug`
1013
1165
 
1014
- ## <a name="graphical"></a>Graphical Interactions: Browser and Text Editor
1166
+ ## <a id="http_options"></a>HTTP socket parameters
1167
+
1168
+ If the server does not provide a valid certificate, use parameter: `--insecure=yes`.
1169
+
1170
+ Some of HTTP socket parameters can be adjusted, those are the parameters of Ruby [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html), for example:
1171
+
1172
+ * `read_timeout` 60 sec
1173
+ * `write_timeout` 60 sec
1174
+
1175
+ Default values are the ones of Ruby.
1176
+
1177
+ Example:
1178
+
1179
+ ```
1180
+ ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
1181
+ ```
1182
+
1183
+ ## <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
1015
1184
 
1016
1185
  Some actions may require the use of a graphical tool:
1017
1186
 
1018
1187
  * a browser for Aspera on Cloud authentication (web auth method)
1019
1188
  * a text editor for configuration file edition
1020
1189
 
1021
- By default the CLI will assume that a graphical environment is available on windows,
1022
- and on other systems, rely on the presence of the "DISPLAY" environment variable.
1190
+ By default the CLI will assume that a graphical environment is available on windows, and on other systems, rely on the presence of the "DISPLAY" environment variable.
1023
1191
  It is also possible to force the graphical mode with option --ui :
1024
1192
 
1025
- * `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or
1026
- a text editor for file edition.
1027
- * `--ui=text` forces a text environment, the URL or file path to open is displayed on
1028
- terminal.
1193
+ * `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1194
+ * `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1029
1195
 
1030
1196
  ## HTTP proxy for REST
1031
1197
 
1032
- To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honoured by Ruby when calling REST APIs.
1198
+ To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honored by Ruby when calling REST APIs.
1199
+
1200
+ ## <a id="certificates"></a>SSL CA certificate bundle
1201
+
1202
+ `ascli` uses ruby `openssl` gem, which uses the `openssl` library, so certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..). The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
1203
+
1204
+ `ascp` also needs to validate certificates when using WSS. By default, `ascp` uses primarily certificates from hard coded path (e.g. on macOS: `/Library/Aspera/ssl`). `ascli` overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch. So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
1033
1205
 
1034
1206
  ## Proxy auto config
1035
1207
 
@@ -1038,13 +1210,13 @@ The `fpac` option allows specification of a Proxy Auto Configuration (PAC) file,
1038
1210
  The PAC file can be tested with command: `config proxy_check` , example:
1039
1211
 
1040
1212
  ```
1041
- $ ascli config proxy_check --fpac=file:///./proxy.pac http://www.example.com
1213
+ ascli config proxy_check --fpac=file:///./proxy.pac http://www.example.com
1042
1214
  PROXY proxy.example.com:8080
1043
1215
  ```
1044
1216
 
1045
1217
  This is not yet implemented to specify http proxy, so use `http_proxy` env vars.
1046
1218
 
1047
- ## <a name="client"></a>FASP configuration
1219
+ ## <a id="client"></a>FASP configuration
1048
1220
 
1049
1221
  The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
1050
1222
 
@@ -1056,9 +1228,9 @@ The `config` plugin also allows specification for the use of a local FASP client
1056
1228
  ### Show path of currently used `ascp`
1057
1229
 
1058
1230
  ```
1059
- $ ascli config ascp show
1231
+ ascli config ascp show
1060
1232
  /Users/laurent/.aspera/ascli/sdk/ascp
1061
- $ ascli config ascp info
1233
+ ascli config ascp info
1062
1234
  +--------------------+-----------------------------------------------------------+
1063
1235
  | key | value |
1064
1236
  +--------------------+-----------------------------------------------------------+
@@ -1066,7 +1238,7 @@ $ ascli config ascp info
1066
1238
  ...
1067
1239
  ```
1068
1240
 
1069
- ### Selection of `ascp` location for [`direct`](#direct) agent
1241
+ ### Selection of `ascp` location for [`direct`](#agt_direct) agent
1070
1242
 
1071
1243
  By default, `ascli` uses any found local product with ascp, including SDK.
1072
1244
 
@@ -1077,7 +1249,7 @@ For a permanent change, the command `config ascp use` sets the same parameter fo
1077
1249
  Using a POSIX shell:
1078
1250
 
1079
1251
  ```
1080
- $ ascli config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
1252
+ ascli config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
1081
1253
  ascp version: 4.0.0.182279
1082
1254
  Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
1083
1255
  Saved to default global preset global_common_defaults
@@ -1086,7 +1258,7 @@ Saved to default global preset global_common_defaults
1086
1258
  Windows:
1087
1259
 
1088
1260
  ```
1089
- $ ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1261
+ ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1090
1262
  ascp version: 4.0.0.182279
1091
1263
  Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1092
1264
  Saved to default global preset global_common_defaults
@@ -1099,7 +1271,7 @@ If the path has spaces, read section: [Shell and Command line parsing](#parsing)
1099
1271
  Locally installed Aspera products can be listed with:
1100
1272
 
1101
1273
  ```
1102
- $ ascli config ascp products list
1274
+ ascli config ascp products list
1103
1275
  :.........................................:................................................:
1104
1276
  : name : app_root :
1105
1277
  :.........................................:................................................:
@@ -1110,7 +1282,7 @@ $ ascli config ascp products list
1110
1282
  :.........................................:................................................:
1111
1283
  ```
1112
1284
 
1113
- ### Selection of local client for `ascp` for [`direct`](#direct) agent
1285
+ ### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
1114
1286
 
1115
1287
  If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
1116
1288
 
@@ -1119,14 +1291,14 @@ Using the option use_product finds the ascp binary of the selected product.
1119
1291
  To permanently use the ascp of a product:
1120
1292
 
1121
1293
  ```
1122
- $ ascli config ascp products use 'Aspera Connect'
1294
+ ascli config ascp products use 'Aspera Connect'
1123
1295
  saved to default global preset /Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
1124
1296
  ```
1125
1297
 
1126
1298
  ### Installation of Connect Client on command line
1127
1299
 
1128
1300
  ```
1129
- $ ascli config ascp connect list
1301
+ ascli config ascp connect list
1130
1302
  :...............................................:......................................:..............:
1131
1303
  : id : title : version :
1132
1304
  :...............................................:......................................:..............:
@@ -1139,7 +1311,7 @@ $ ascli config ascp connect list
1139
1311
  : urn:uuid:213C9370-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 32 : 3.6.2.117442 :
1140
1312
  : urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 64 : 3.7.2.141527 :
1141
1313
  :...............................................:......................................:..............:
1142
- $ ascli config ascp connect id 'Aspera Connect for Mac Intel 10.6' links list
1314
+ ascli config ascp connect id 'Aspera Connect for Mac Intel 10.6' links list
1143
1315
  :.............................................:..........................:.......................................................................:..........:...............:
1144
1316
  : title : type : href : hreflang : rel :
1145
1317
  :.............................................:..........................:.......................................................................:..........:...............:
@@ -1152,11 +1324,11 @@ $ ascli config ascp connect id 'Aspera Connect for Mac Intel 10.6' links list
1152
1324
  : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/zh-cn/pdf/Connect_User_3.7.0_OSX_zh-cn.pdf : zh-cn : documentation :
1153
1325
  : Aspera Connect for Mac Release Notes : text/html : http://www.asperasoft.com/en/release_notes/default_1/release_notes_54 : en : release-notes :
1154
1326
  :.............................................:..........................:.......................................................................:..........:...............:
1155
- $ ascli config ascp connect id 'Aspera Connect for Mac Intel 10.6' links id 'Mac Intel Installer' download --to-folder=.
1327
+ ascli config ascp connect id 'Aspera Connect for Mac Intel 10.6' links id 'Mac Intel Installer' download --to-folder=.
1156
1328
  downloaded: AsperaConnect-3.6.1.111259-mac-intel-10.6.dmg
1157
1329
  ```
1158
1330
 
1159
- ## <a name="agents"></a>Transfer Agents
1331
+ ## <a id="agents"></a>Transfer Agents
1160
1332
 
1161
1333
  Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
1162
1334
 
@@ -1165,102 +1337,109 @@ This [_transfer-spec_](#transferspec) will be executed by a transfer client, her
1165
1337
 
1166
1338
  There are currently 3 agents:
1167
1339
 
1168
- * [`direct`](#direct) : a local execution of `ascp`
1169
- * `connect` : use of a local Connect Client
1170
- * `node` : use of an Aspera Transfer Node (potentially _remote_).
1171
- * `httpgw` : use of an Aspera HTTP Gateway
1340
+ * [`direct`](#agt_direct) : a local execution of `ascp`
1341
+ * [`connect`](#agt_connect) : use of a local Connect Client
1342
+ * [`node`](#agt_node) : use of an Aspera Transfer Node (potentially _remote_).
1343
+ * [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
1344
+ * [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
1172
1345
 
1173
1346
  Note that all transfer operation are seen from the point of view of the agent.
1174
1347
  For instance, a node agent making an "upload", or "package send" operation,
1175
1348
  will effectively push files to the related server from the agent node.
1176
1349
 
1177
- `ascli` standadizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1350
+ `ascli` standardizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1178
1351
 
1179
1352
 
1180
- ### <a name="direct"></a>Direct (local ascp execution)
1353
+ ### <a id="agt_direct"></a>Direct
1181
1354
 
1182
- By default `ascli` uses a local ascp, equivalent to specifying `--transfer=direct`.
1183
- `ascli` will detect locally installed Aspera products.
1355
+ The `direct` agent directly executes a local ascp.
1356
+ This is the default for `ascli`.
1357
+ This is equivalent to specifying `--transfer=direct`.
1358
+ `ascli` will detect locally installed Aspera products, including SDK.
1184
1359
  Refer to section [FASP](#client).
1185
1360
 
1186
- To specify a FASP proxy (only supported with the `direct` agent), set the appropriate [_transfer-spec_](#transferspec) parameter:
1187
-
1188
- * `EX_fasp_proxy_url`
1189
- * `EX_http_proxy_url` (proxy for legacy http fallback)
1190
- * `EX_ascp_args`
1191
-
1192
- The `transfer-info` accepts the following optional parameters:
1361
+ The `transfer-info` accepts the following optional parameters to control multi-session, WSS
1193
1362
 
1194
1363
  <table>
1195
- <tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
1196
- <tr><td>spawn_timeout_sec</td><td>Float</td><td>3</td><td>Multi session</td><td>Verification time that ascp is running</td></tr>
1197
- <tr><td>spawn_delay_sec</td><td>Float</td><td>2</td><td>Multi session</td><td>Delay between startup of sessions</td></tr>
1198
- <tr><td>wss</td><td>Bool</td><td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
1199
- <tr><td>multi_incr_udp</td><td>Bool</td><td>true</td><td>Multi Session</td><td>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)</td></tr>
1200
- <tr><td>resume</td><td>Hash</td><td>nil</td><td>Resumer parameters</td><td>See below</td></tr>
1364
+ <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1365
+ <tr><td>wss</td><td>Bool</td><td>Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: false</td></tr>
1366
+ <tr><td>spawn_timeout_sec</td><td>Float</td><td>Multi session<br/>Verification time that ascp is running<br/>Default: 3</td></tr>
1367
+ <tr><td>spawn_delay_sec</td><td>Float</td><td>Multi session<br/>Delay between startup of sessions<br/>Default: 2</td></tr>
1368
+ <tr><td>multi_incr_udp</td><td>Bool</td><td>Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true</td></tr>
1369
+ <tr><td>resume</td><td>Hash</td><td>Resume<br/>parameters<br/>See below</td></tr>
1370
+ <tr><td>resume.iter_max</td><td>int</td><td>Resume<br/>Max number of retry on error<br/>Default: 7</td></tr>
1371
+ <tr><td>resume.sleep_initial</td><td>int</td><td>Resume<br/>First Sleep before retry<br/>Default: 2</td></tr>
1372
+ <tr><td>resume.sleep_factor</td><td>int</td><td>Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2</td></tr>
1373
+ <tr><td>resume.sleep_max</td><td>int</td><td>Resume<br/>Default: 60</td></tr>
1201
1374
  </table>
1202
1375
 
1203
- Resume parameters:
1204
-
1205
- In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
1206
- Sleep between iteration is:
1376
+ Resume: In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
1377
+ Sleep between iterations is:
1207
1378
 
1208
1379
  ```
1209
1380
  max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
1210
1381
  ```
1211
1382
 
1212
- <table>
1213
- <tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
1214
- <tr><td>iter_max</td><td>int</td><td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
1215
- <tr><td>sleep_initial</td><td>int</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
1216
- <tr><td>sleep_factor</td><td>int</td><td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
1217
- <tr><td>sleep_max</td><td>int</td><td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
1218
- </table>
1383
+ Some transfer errors are considered "retryable" (e.g. timeout) and some other not (e.g. wrong password).
1219
1384
 
1220
1385
  Examples:
1221
1386
 
1222
1387
  ```
1223
- $ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1224
- $ ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1388
+ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1389
+ ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1225
1390
  ```
1226
1391
 
1227
- ### IBM Aspera Connect Client GUI
1392
+ To specify a FASP proxy (only supported with the `direct` agent), set the appropriate [_transfer-spec_](#transferspec) parameter:
1393
+
1394
+ * `EX_fasp_proxy_url`
1395
+ * `EX_http_proxy_url` (proxy for legacy http fallback)
1396
+ * `EX_ascp_args`
1397
+
1398
+ ### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1228
1399
 
1229
- By specifying option: `--transfer=connect`, `ascli` will start transfers
1230
- using the locally installed Aspera Connect Client.
1400
+ By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
1231
1401
 
1232
- ### Aspera Node API : Node to node transfers
1402
+ ### <a id="agt_node"></a>Aspera Node API : Node to node transfers
1233
1403
 
1234
1404
  By specifying option: `--transfer=node`, the CLI will start transfers in an Aspera
1235
1405
  Transfer Server using the Node API, either on a local or remote node.
1406
+ Parameters provided in option `transfer_info` are:
1236
1407
 
1237
- If a default node has been configured
1238
- in the configuration file, then this node is used by default else the parameter
1239
- `--transfer-info` is required. The node specification shall be a hash table with
1240
- three keys: url, username and password, corresponding to the URL of the node API
1241
- and associated credentials (node user or access key).
1408
+ <table>
1409
+ <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1410
+ <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
1411
+ <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
1412
+ <tr><td>password</td><td>string</td><td>password, secret or bearer token</br>Mandatory</td></tr>
1413
+ <tr><td>root_id</td><td>string</td><td>password or secret</br>Mandatory only for bearer token</td></tr>
1414
+ </table>
1242
1415
 
1243
- The `--transfer-info` parameter can directly specify a pre-configured [option preset](#lprt) :
1244
- `--transfer-info=@preset:<psetname>` or specified using the option syntax :
1416
+ Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
1417
+ `--transfer-info=@preset:<psetname>` or be specified using the extended value syntax :
1245
1418
  `--transfer-info=@json:'{"url":"https://...","username":"theuser","password":"thepass"}'`
1246
1419
 
1247
- ### <a name="trinfoaoc"></a>Aspera on cloud
1420
+ If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
1248
1421
 
1249
- By specifying option: `--transfer=aoc`, WORK IN PROGRESS
1422
+ If the `password` value begins with `Bearer ` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
1250
1423
 
1251
- ### <a name="httpgw"></a>HTTP Gateway
1424
+ ### <a id="agt_httpgw"></a>HTTP Gateway
1252
1425
 
1253
- If it possible to send using a HTTP gateway, in case FASP is not allowed.
1426
+ If it possible to send using a HTTP gateway, in case FASP is not allowed. `transfer_info` shall have a single mandatory parameter: `url`.
1254
1427
 
1255
1428
  Example:
1256
1429
 
1257
1430
  ```
1258
- $ ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
1431
+ ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
1259
1432
  ```
1260
1433
 
1261
1434
  Note that the gateway only supports transfers authorized with a token.
1262
1435
 
1263
- ## <a name="transferspec"></a>Transfer Specification
1436
+ ### <a id="agt_trsdk"></a>Transfer SDK
1437
+
1438
+ Another possibility is to use the Transfer SDK daemon (asperatransferd).
1439
+
1440
+ By default it will listen on local port `55002` on `127.0.0.1`.
1441
+
1442
+ ## <a id="transferspec"></a>Transfer Specification
1264
1443
 
1265
1444
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
1266
1445
  is described in a _transfer-spec_ (Transfer Specification), such as:
@@ -1273,9 +1452,9 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
1273
1452
 
1274
1453
  `ascli` builds a default _transfer-spec_ internally, so it is not necessary to provide additional parameters on the command line for this transfer.
1275
1454
 
1276
- If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
1455
+ If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cumulative.
1277
1456
 
1278
- It is possible to specify ascp options when the `transfer` option is set to [`direct`](#direct) using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
1457
+ It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
1279
1458
 
1280
1459
  The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
1281
1460
 
@@ -1284,21 +1463,22 @@ The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
1284
1463
 
1285
1464
  A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
1286
1465
 
1287
- ## <a name="transferparams"></a>Transfer Parameters
1466
+ ## <a id="transferparams"></a>Transfer Parameters
1288
1467
 
1289
- All standard _transfer-spec_ parameters can be speficied.
1468
+ All standard _transfer-spec_ parameters can be specified.
1290
1469
  [_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
1291
1470
 
1292
1471
  References:
1293
1472
 
1294
1473
  * [Aspera Node API Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20node%20api%22)&rarr;/opt/transfers
1295
1474
  * [Aspera Transfer SDK Documentation](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20sdk%22)&rarr;Guides&rarr;API Ref&rarr;Transfer Spec V1
1475
+ * [Aspera Connect SDK](https://d3gcli72yxqn2z.cloudfront.net/connect/v4/asperaweb-4.js) &rarr; search `The parameters for starting a transfer.`
1296
1476
 
1297
1477
  Parameters can be displayed with commands:
1298
1478
 
1299
1479
  ```
1300
- $ ascli config ascp spec
1301
- $ ascli config ascp spec --select=@json:'{"f":"Y"}' --fields=-f,n,c
1480
+ ascli config ascp spec
1481
+ ascli config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
1302
1482
  ```
1303
1483
 
1304
1484
  Columns:
@@ -1306,9 +1486,10 @@ Columns:
1306
1486
  * D=Direct (local `ascp` execution)
1307
1487
  * N=Node API
1308
1488
  * C=Connect Client
1309
- * arg=`ascp` argument or environment variable
1310
1489
 
1311
- Fields with EX_ prefix are extensions to transfer agent [`direct`](#direct). (only in `ascli`).
1490
+ `ascp` argument or environment variable is provided in description.
1491
+
1492
+ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
1312
1493
 
1313
1494
  <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>none, aes-128, aes-256<br/>Allowed values: aes128, aes192, aes256, aes128cfb, aes192cfb, aes256cfb, aes128gcm, aes192gcm, aes256gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side content protection. (encryption at rest)<br/>Allowed values: encrypt, decrypt</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password.</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer (older,string)<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>http_fallback</td><td>string<br/>bool</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a fasp transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port.<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port.</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>in bytes<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Required. Contains a path to the source (required) and a path to the destination.</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_access_key</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Node only?</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies ssh (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIVATE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links by:<br/> following the link (follow),<br/> copying the link itself (copy),<br/> skipping (skip),<br/> or forcibly copying the link itself (copy+force).<br/> Default: follow<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node api.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowable values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>TODO: check node, sdk<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Passphrase used for at rest encryption or decryption<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/> Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr></table>
1314
1495
 
@@ -1326,17 +1507,17 @@ The option `to_folder` provides an equivalent and convenient way to change this
1326
1507
 
1327
1508
  ### List of files for transfers
1328
1509
 
1329
- When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files (Extensed value) is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
1510
+ When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
1330
1511
  So, by default, the list of files to transfer will be simply specified on the command line:
1331
1512
 
1332
1513
  ```
1333
- $ ascli server upload ~/mysample.file secondfile
1514
+ ascli server upload ~/mysample.file secondfile
1334
1515
  ```
1335
1516
 
1336
1517
  This is equivalent to:
1337
1518
 
1338
1519
  ```
1339
- $ ascli server upload --sources=@args ~/mysample.file secondfile
1520
+ ascli server upload --sources=@args ~/mysample.file secondfile
1340
1521
  ```
1341
1522
 
1342
1523
  More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
@@ -1362,13 +1543,19 @@ For ease of use and flexibility, the list of files to transfer is specified by t
1362
1543
  --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
1363
1544
  ```
1364
1545
 
1546
+ providing a file list directly to ascp:
1547
+
1548
+ ```
1549
+ ... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
1550
+ ```
1551
+
1365
1552
  * Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
1366
1553
 
1367
1554
  ```
1368
1555
  --sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
1369
1556
  ```
1370
1557
 
1371
- This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overriden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
1558
+ This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overridden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
1372
1559
 
1373
1560
  In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
1374
1561
 
@@ -1378,10 +1565,10 @@ In case the file list is provided on the command line i.e. using `--sources=@arg
1378
1565
  Example:
1379
1566
 
1380
1567
  ```
1381
- $ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
1568
+ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
1382
1569
  ```
1383
1570
 
1384
- Internally, when transfer agent [`direct`](#direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pait-list` is provided in `ts` in `EX_ascp_args`.
1571
+ Internally, when transfer agent [`direct`](#agt_direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pair-list` is provided in `ts` in `EX_ascp_args`.
1385
1572
 
1386
1573
  Note the special case when the source files are located on "Aspera on Cloud", i.e. using access keys and the `file id` API:
1387
1574
 
@@ -1394,7 +1581,7 @@ Source files are located on "Aspera on cloud", when :
1394
1581
  * the server is Aspera on Cloud, and making a download / recv
1395
1582
  * the agent is Aspera on Cloud, and making an upload / send
1396
1583
 
1397
- ### <a name="multisession"></a>Support of multi-session
1584
+ ### <a id="multisession"></a>Support of multi-session
1398
1585
 
1399
1586
  Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
1400
1587
 
@@ -1445,16 +1632,14 @@ When multi-session is used, one separate UDP port is used per session (refer to
1445
1632
  --ts=@json:'{"precalculate_job_size":true}'
1446
1633
  ```
1447
1634
 
1448
-
1449
-
1450
- ## <a name="scheduling"></a>Lock for exclusive execution
1635
+ ## <a id="scheduling"></a>Lock for exclusive execution
1451
1636
 
1452
1637
  In some conditions, it may be desirable to ensure that `ascli` is not executed several times in parallel.
1453
1638
 
1454
1639
  For instance when `ascli` is executed automatically on a schedule basis, one generally desire that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
1455
1640
 
1456
1641
  * Executing instances may pile-up and kill the system
1457
- * The same file may be transfered by multiple instances at the same time.
1642
+ * The same file may be transferred by multiple instances at the same time.
1458
1643
  * `preview` may generate the same files in multiple instances.
1459
1644
 
1460
1645
  Usually the OS native scheduler already provides some sort of protection against parallel execution:
@@ -1488,14 +1673,14 @@ WARN -- : Another instance is already running (Address already in use - bind(2)
1488
1673
  `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
1489
1674
  , some are provided om shared libraries and must be activated. (e.g. using `trapd`)
1490
1675
 
1491
- The list of supported *PVCL* adapters can be retried with command:
1676
+ The list of supported *PVCL* adapters can be retrieved with command:
1492
1677
 
1493
1678
  ```
1494
- $ ascli conf ascp info
1679
+ ascli conf ascp info
1495
1680
  +--------------------+-----------------------------------------------------------+
1496
1681
  | key | value |
1497
1682
  +--------------------+-----------------------------------------------------------+
1498
- -----8<----snip---------
1683
+ -----8<-----snip-----8<-----
1499
1684
  | product_name | IBM Aspera SDK |
1500
1685
  | product_version | 4.0.1.182389 |
1501
1686
  | process | pvcl |
@@ -1520,20 +1705,26 @@ The simplified format is:
1520
1705
 
1521
1706
  One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
1522
1707
 
1523
- ## <a name="faux_testing"></a>`faux:` for testing
1708
+ ## <a id="faux_testing"></a>`faux:` for testing
1524
1709
 
1525
- This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not `ascli`
1710
+ This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not `ascli`.
1526
1711
 
1527
1712
  This adapter can be used to simulate a file or a directory.
1528
1713
 
1529
- To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form `faux:///fname?fsize` where:
1714
+ To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
1530
1715
 
1531
- * `fname` is the name that will be assigned to the file on the destination
1532
- * `fsize` is the number of bytes that will be sent (in decimal).
1716
+ ```
1717
+ faux:///filename?filesize
1718
+ ```
1719
+
1720
+ where:
1721
+
1722
+ * `filename` is the name that will be assigned to the file on the destination
1723
+ * `filesize` is the number of bytes that will be sent (in decimal).
1533
1724
 
1534
- Note that the character `?` is a special shell character (wildcard), so `faux` file specification on command line shall be protected (using `\?` and `\&` or using quotes). If not, the shell may give error: `no matches found` or equivalent.
1725
+ Note: characters `?` and `&` are shell special characters (wildcard and backround), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
1535
1726
 
1536
- For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2^20, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2^60. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of system memory unless `faux://`.
1727
+ For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2<sup>20</sup>, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2<sup>60</sup>. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is `faux://`.
1537
1728
 
1538
1729
  To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
1539
1730
 
@@ -1541,21 +1732,21 @@ To send uninitialized data in place of a source directory, the source argument i
1541
1732
  faux:///dirname?<arg1>=<val1>&...
1542
1733
  ```
1543
1734
 
1544
- `dirname` is the folder name and can contain `/` to specify a subfolder.
1735
+ where:
1545
1736
 
1546
- Supported arguments are:
1737
+ * `dirname` is the folder name and can contain `/` to specify a subfolder.
1738
+ * supported arguments are:
1547
1739
 
1548
1740
  <table>
1549
1741
  <tr><th>name</th><th>type</th><th>default</th><th>description</th></tr>
1550
- <tr><td>count</td><td>int</td><td>mandatory</td><td>number of files</td></tr>
1551
- <tr><td>file</td><td>string</td><td>file</td><td>basename for files</td></tr>
1552
- <tr><td>size</td><td>int</td><td>0</td><td>size of first file.</td></tr>
1553
- <tr><td>inc</td><td>int</td><td>0</td><td>increment applied to determine next file size</td></tr>
1554
- <tr><td>seq</td><td>sequential<br/>random</td><td>sequential</td><td>sequence in determining next file size</td></tr>
1555
- <tr><td>buf_init</td><td>none<br/>zero<br/>random</td><td>zero</td><td>how source data initialized.<br/>Option 'none' is not allowed for downloads.</td></tr>
1742
+ <tr><td>count</td><td>int</td><td>mandatory</td><td>Number of files</td></tr>
1743
+ <tr><td>file</td><td>string</td><td>file</td><td>Basename for files</td></tr>
1744
+ <tr><td>size</td><td>int</td><td>0</td><td>Size of first file.</td></tr>
1745
+ <tr><td>inc</td><td>int</td><td>0</td><td>Increment applied to determine next file size</td></tr>
1746
+ <tr><td>seq</td><td>sequential<br/>random</td><td>sequential</td><td>Sequence in determining next file size</td></tr>
1747
+ <tr><td>buf_init</td><td>none<br/>zero<br/>random</td><td>zero</td><td>How source data is initialized<br/>Option 'none' is not allowed for downloads.</td></tr>
1556
1748
  </table>
1557
1749
 
1558
-
1559
1750
  The sequence parameter is applied as follows:
1560
1751
 
1561
1752
  * If `seq` is `random` then each file size is:
@@ -1563,40 +1754,40 @@ The sequence parameter is applied as follows:
1563
1754
  * size +/- (inc * rand())
1564
1755
  * Where rand is a random number between 0 and 1
1565
1756
  * Note that file size must not be negative, inc will be set to size if it is greater than size
1566
- * Similarly, overall file size must be less than 8 * 2^60. If size + inc is greater, inc will be reduced to limit size + inc to 7 * 2^60.
1757
+ * Similarly, overall file size must be less than 8*2<sup>60</sup>. If size + inc is greater, inc will be reduced to limit size + inc to 7*2<sup>60</sup>.
1567
1758
 
1568
1759
  * If `seq` is `sequential` then each file size is:
1569
1760
 
1570
- * size + ((fileindex - 1) * inc)
1761
+ * `size + ((fileindex - 1) * inc)`
1571
1762
  * Where first file is index 1
1572
- * So file1 is size bytes, file2 is size + inc bytes, file3 is size + inc * 2 bytes, etc.
1573
- * As with random, inc will be adjusted if size + (count * inc) is not less then 8 ^ 2^60.
1763
+ * So file1 is `size` bytes, file2 is `size + inc` bytes, file3 is `size + inc * 2` bytes, etc.
1764
+ * As with `random`, `inc` will be adjusted if `size + (count * inc)` is not less then 8*2<sup>60</sup>.
1574
1765
 
1575
- Filenames generated are of the form: `<file>_<00000 . . . count>_<filesize>`
1766
+ Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
1576
1767
 
1577
1768
  To discard data at the destination, the destination argument is set to `faux://` .
1578
1769
 
1579
1770
  Examples:
1580
1771
 
1581
- * Upload 20 gigabytes of random data to file myfile to directory /Upload
1772
+ * Upload 20 gibibytes of random data to file myfile to directory /Upload
1582
1773
 
1583
1774
  ```
1584
- $ ascli server upload faux:///myfile\?20g --to-folder=/Upload
1775
+ ascli server upload faux:///myfile\?20g --to-folder=/Upload
1585
1776
  ```
1586
1777
 
1587
1778
  * Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
1588
1779
 
1589
1780
  ```
1590
- $ ascli server upload /tmp/sample --to-folder=faux://
1781
+ ascli server upload /tmp/sample --to-folder=faux://
1591
1782
  ```
1592
1783
 
1593
- * Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 M - 2 bytes, with the basename of each file being `testfile` to /Upload
1784
+ * Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
1594
1785
 
1595
1786
  ```
1596
- $ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
1787
+ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
1597
1788
  ```
1598
1789
 
1599
- ## <a name="commands"></a>Sample Commands
1790
+ ## <a id="commands"></a>Sample Commands
1600
1791
 
1601
1792
  A non complete list of commands used in unit tests:
1602
1793
 
@@ -1606,20 +1797,20 @@ ascli -h
1606
1797
  ascli aoc -N remind --username=my_aoc_user_email
1607
1798
  ascli aoc -N servers
1608
1799
  ascli aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
1609
- ascli aoc admin ats access_key --id=akibmcloud --secret=somesecret node browse /
1610
- ascli aoc admin ats access_key --id=akibmcloud delete
1611
1800
  ascli aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
1612
1801
  ascli aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
1802
+ ascli aoc admin ats access_key delete akibmcloud
1613
1803
  ascli aoc admin ats access_key list --fields=name,id
1804
+ ascli aoc admin ats access_key node akibmcloud --secret=somesecret browse /
1614
1805
  ascli aoc admin ats cluster clouds
1615
1806
  ascli aoc admin ats cluster list
1616
1807
  ascli aoc admin ats cluster show --cloud=aws --region=eu-west-1
1617
- ascli aoc admin ats cluster show --id=1f412ae7-869a-445c-9c05-02ad16813be2
1808
+ ascli aoc admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
1618
1809
  ascli aoc admin res application list
1619
1810
  ascli aoc admin res client list
1620
1811
  ascli aoc admin res client_access_key list
1621
- ascli aoc admin res client_registration_token --id=my_clt_reg_id delete
1622
1812
  ascli aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
1813
+ ascli aoc admin res client_registration_token delete my_clt_reg_id
1623
1814
  ascli aoc admin res client_registration_token list
1624
1815
  ascli aoc admin res contact list
1625
1816
  ascli aoc admin res dropbox list
@@ -1629,23 +1820,23 @@ ascli aoc admin res kms_profile list
1629
1820
  ascli aoc admin res node list
1630
1821
  ascli aoc admin res operation list
1631
1822
  ascli aoc admin res organization show
1632
- ascli aoc admin res package list
1823
+ ascli aoc admin res package list --http-options=@json:'{"read_timeout":120.0}'
1633
1824
  ascli aoc admin res saml_configuration list
1634
1825
  ascli aoc admin res self show
1635
1826
  ascli aoc admin res short_link list
1636
1827
  ascli aoc admin res user list
1637
1828
  ascli aoc admin res workspace_membership list
1638
1829
  ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
1639
- ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 access_key delete --id=testsub1
1640
1830
  ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v3 events
1641
1831
  ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v4 browse /
1642
1832
  ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v4 delete /folder1
1643
1833
  ascli aoc admin resource node --name=AOC_NODE1_NAME --secret=AOC_NODE1_SECRET v4 mkdir /folder1
1834
+ ascli aoc admin resource node v3 name AOC_NODE1_NAME --secret=AOC_NODE1_SECRET access_key delete testsub1
1644
1835
  ascli aoc admin resource workspace list
1645
1836
  ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
1646
- ascli aoc automation workflow --id="my_wf_id" action create --value=@json:'{"name":"toto"}' | tee action.info
1837
+ ascli aoc automation workflow "my_wf_id" action create --value=@json:'{"name":"toto"}' | tee action.info
1647
1838
  ascli aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
1648
- ascli aoc automation workflow delete --id="my_wf_id"
1839
+ ascli aoc automation workflow delete "my_wf_id"
1649
1840
  ascli aoc automation workflow list
1650
1841
  ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data > test
1651
1842
  ascli aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
@@ -1657,7 +1848,7 @@ ascli aoc files browse /
1657
1848
  ascli aoc files browse / -N --link=my_aoc_publink_folder
1658
1849
  ascli aoc files delete /testsrc
1659
1850
  ascli aoc files download --transfer=connect /200KB.1
1660
- ascli aoc files file --id=my_file_id show
1851
+ ascli aoc files file show my_file_id
1661
1852
  ascli aoc files find / --value='\.partial$'
1662
1853
  ascli aoc files http_node_download --to-folder=. /200KB.1
1663
1854
  ascli aoc files mkdir /testsrc
@@ -1674,8 +1865,8 @@ ascli aoc org -N --link=my_aoc_publink_recv_from_aocuser
1674
1865
  ascli aoc organization
1675
1866
  ascli aoc packages list
1676
1867
  ascli aoc packages list --query=@json:'{"dropbox_id":"my_shbxid","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
1677
- ascli aoc packages recv --id="my_package_id" --to-folder=.
1678
- ascli aoc packages recv --id=ALL --to-folder=. --once-only=yes --lock-port=12345
1868
+ ascli aoc packages recv "my_package_id" --to-folder=.
1869
+ ascli aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
1679
1870
  ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["external.user@example.com"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
1680
1871
  ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["internal.user@example.com"],"note":"my note"}' testfile.bin
1681
1872
  ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
@@ -1686,20 +1877,20 @@ ascli aoc user info show
1686
1877
  ascli aoc user shared_inboxes
1687
1878
  ascli aoc user workspaces
1688
1879
  ascli aoc workspace
1689
- ascli ats access_key --id=ak_aws delete
1690
- ascli ats access_key --id=akibmcloud --secret=somesecret cluster
1691
- ascli ats access_key --id=akibmcloud --secret=somesecret node browse /
1692
- ascli ats access_key --id=akibmcloud delete
1880
+ ascli ats access_key cluster akibmcloud --secret=somesecret
1693
1881
  ascli ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
1694
1882
  ascli ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
1883
+ ascli ats access_key delete ak_aws
1884
+ ascli ats access_key delete akibmcloud
1695
1885
  ascli ats access_key list --fields=name,id
1886
+ ascli ats access_key node akibmcloud browse / --secret=somesecret
1696
1887
  ascli ats api_key create
1697
1888
  ascli ats api_key instances
1698
1889
  ascli ats api_key list
1699
1890
  ascli ats cluster clouds
1700
1891
  ascli ats cluster list
1701
1892
  ascli ats cluster show --cloud=aws --region=eu-west-1
1702
- ascli ats cluster show --id=1f412ae7-869a-445c-9c05-02ad16813be2
1893
+ ascli ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
1703
1894
  ascli conf flush_tokens
1704
1895
  ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes
1705
1896
  ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
@@ -1713,6 +1904,8 @@ ascli config ascp products list
1713
1904
  ascli config ascp show
1714
1905
  ascli config ascp spec
1715
1906
  ascli config check_update
1907
+ ascli config detect --url=https://my_aoc_org.ibmaspera.com;\
1908
+ ascli config detect --url=my_faspex_url
1716
1909
  ascli config doc
1717
1910
  ascli config doc transfer-parameters
1718
1911
  ascli config email_test --notif-to=my_recipient_email
@@ -1725,34 +1918,35 @@ ascli console transfer smart list
1725
1918
  ascli console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
1726
1919
  ascli cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
1727
1920
  ascli cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
1728
- ascli cos node access_key --id=self show
1921
+ ascli cos node access_key show self
1729
1922
  ascli cos node download testfile.bin --to-folder=.
1730
1923
  ascli cos node info
1731
1924
  ascli cos node upload testfile.bin
1732
1925
  ascli faspex health
1733
1926
  ascli faspex package list
1734
- ascli faspex package list --box=sent --fields=package_id --format=csv --display=data|tail -n 1);\
1735
- ascli faspex package list --fields=package_id --format=csv --display=data|tail -n 1);\
1736
- ascli faspex package recv --to-folder=. --box=sent --id="my_package_id"
1737
- ascli faspex package recv --to-folder=. --id="my_package_id"
1738
- ascli faspex package recv --to-folder=. --id=ALL --once-only=yes
1927
+ ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}');\
1928
+ ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}');\
1929
+ ascli faspex package recv "my_package_id" --to-folder=.
1930
+ ascli faspex package recv "my_package_id" --to-folder=. --box=sent
1931
+ ascli faspex package recv --to-folder=. "my_package_id"
1739
1932
  ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
1933
+ ascli faspex package recv ALL --to-folder=. --once-only=yes
1740
1934
  ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["internal.user@example.com","FASPEX_USERNAME"]}' testfile.bin
1741
1935
  ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
1742
1936
  ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
1743
1937
  ascli faspex source name "Server Files" node br /
1744
1938
  ascli faspex5 node list --value=@json:'{"type":"received","subtype":"mypackages"}'
1745
1939
  ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
1746
- ascli faspex5 package receive --id="my_package_id" --to-folder=.
1940
+ ascli faspex5 package receive "my_package_id" --to-folder=.
1747
1941
  ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"${f5_user}"}]}' testfile.bin
1748
1942
  ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
1749
- ascli node -N -Ptst_node_preview access_key delete --id=aoc_1
1750
- ascli node async --id=1 bandwidth
1751
- ascli node async --id=1 counters
1752
- ascli node async --id=1 files
1943
+ ascli node -N -Ptst_node_preview access_key delete aoc_1
1944
+ ascli node async bandwidth 1
1945
+ ascli node async counters 1
1946
+ ascli node async files 1
1753
1947
  ascli node async list
1754
- ascli node async show --id=1
1755
- ascli node async show --id=ALL
1948
+ ascli node async show 1
1949
+ ascli node async show ALL
1756
1950
  ascli node basic_token
1757
1951
  ascli node browse / -r
1758
1952
  ascli node delete folder_1/10MB.1
@@ -1761,8 +1955,8 @@ ascli node download --to-folder=. folder_1/testfile.bin
1761
1955
  ascli node health
1762
1956
  ascli node info
1763
1957
  ascli node search / --value=@json:'{"sort":"mtime"}'
1764
- ascli node service --id=service1 delete
1765
1958
  ascli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
1959
+ ascli node service delete service1
1766
1960
  ascli node service list
1767
1961
  ascli node transfer list --value=@json:'{"active_only":true}'
1768
1962
  ascli node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
@@ -1770,12 +1964,12 @@ ascli node upload --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000
1770
1964
  ascli orchestrator info
1771
1965
  ascli orchestrator plugins
1772
1966
  ascli orchestrator processes
1773
- ascli orchestrator workflow --id=ORCH_WORKFLOW_ID inputs
1774
- ascli orchestrator workflow --id=ORCH_WORKFLOW_ID start --params=@json:'{"Param":"world !"}'
1775
- ascli orchestrator workflow --id=ORCH_WORKFLOW_ID start --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
1776
- ascli orchestrator workflow --id=ORCH_WORKFLOW_ID status
1967
+ ascli orchestrator workflow inputs ORCH_WORKFLOW_ID
1777
1968
  ascli orchestrator workflow list
1778
- ascli orchestrator workflow status
1969
+ ascli orchestrator workflow start ORCH_WORKFLOW_ID --params=@json:'{"Param":"world !"}'
1970
+ ascli orchestrator workflow start ORCH_WORKFLOW_ID --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
1971
+ ascli orchestrator workflow status ALL
1972
+ ascli orchestrator workflow status ORCH_WORKFLOW_ID
1779
1973
  ascli preview check --skip-types=office
1780
1974
  ascli preview folder 1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
1781
1975
  ascli preview scan --skip-types=office --log-level=info
@@ -1831,12 +2025,12 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
1831
2025
  ...and more
1832
2026
  ```
1833
2027
 
1834
- ## <a name="usage"></a>Usage
2028
+ ## <a id="usage"></a>Usage
1835
2029
 
1836
2030
  ```
1837
- $ ascli -h
2031
+ ascli -h
1838
2032
  NAME
1839
- ascli -- a command line tool for Aspera Applications (v4.4.0)
2033
+ ascli -- a command line tool for Aspera Applications (v4.5.0)
1840
2034
 
1841
2035
  SYNOPSIS
1842
2036
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -1846,10 +2040,11 @@ DESCRIPTION
1846
2040
  Documentation and examples: https://rubygems.org/gems/aspera-cli
1847
2041
  execute: ascli conf doc
1848
2042
  or visit: http://www.rubydoc.info/gems/aspera-cli
2043
+ source repo: https://github.com/IBM/aspera-cli
1849
2044
 
1850
2045
  ENVIRONMENT VARIABLES
1851
2046
  ASCLI_HOME config folder, default: $HOME/.aspera/ascli
1852
- #any option can be set as an environment variable, refer to the manual
2047
+ any option can be set as an environment variable, refer to the manual
1853
2048
 
1854
2049
  COMMANDS
1855
2050
  To list first level commands, execute: ascli
@@ -1857,7 +2052,8 @@ COMMANDS
1857
2052
 
1858
2053
  OPTIONS
1859
2054
  Options begin with a '-' (minus), and value is provided on command line.
1860
- Special values are supported beginning with special prefix, like: @base64: @json: @zlib: @ruby: @csvt: @lines: @list: @val: @file: @path: @env: @stdin:.
2055
+ Special values are supported beginning with special prefix @pfx:, where pfx is one of:
2056
+ base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, stdin, preset
1861
2057
  Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
1862
2058
 
1863
2059
  ARGS
@@ -1883,11 +2079,13 @@ OPTIONS: global
1883
2079
  --logger=ENUM log method: stderr, stdout, syslog
1884
2080
  --lock-port=VALUE prevent dual execution of a command, e.g. in cron
1885
2081
  --query=VALUE additional filter for API calls (extended value) (some commands)
2082
+ --http-options=VALUE options for http socket (extended value)
1886
2083
  --insecure=ENUM do not validate HTTPS certificate: yes, no
1887
2084
  --once-only=ENUM process only new items (some commands): yes, no
2085
+ --log-passwords=ENUM show passwords in logs: yes, no
1888
2086
 
1889
2087
  COMMAND: config
1890
- SUBCOMMANDS: gem_path genkey plugins flush_tokens list overview open echo id documentation wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo
2088
+ SUBCOMMANDS: list overview id preset open documentation genkey gem_path plugins flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
1891
2089
  OPTIONS:
1892
2090
  --value=VALUE extended value for create, update, list filter
1893
2091
  --property=VALUE name of property to set
@@ -1898,14 +2096,14 @@ OPTIONS:
1898
2096
  --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, no
1899
2097
  --default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, no
1900
2098
  --test-mode=ENUM Wizard: skip private key check step: yes, no
2099
+ -P, --presetVALUE load the named option preset from current config file
1901
2100
  --pkeypath=VALUE Wizard: path to private key for JWT
1902
2101
  --ascp-path=VALUE path to ascp
1903
2102
  --use-product=VALUE use ascp from specified product
1904
2103
  --smtp=VALUE smtp configuration (extended value: hash)
1905
2104
  --fpac=VALUE proxy auto configuration URL
1906
- -P, --presetVALUE load the named option preset from current config file
1907
2105
  --secret=VALUE default secret
1908
- --secrets=VALUE secret repository (Hash)
2106
+ --secrets=VALUE secret vault
1909
2107
  --sdk-url=VALUE URL to get SDK
1910
2108
  --sdk-folder=VALUE SDK folder path
1911
2109
  --notif-to=VALUE email recipient for notification of transfers
@@ -1917,7 +2115,7 @@ OPTIONS:
1917
2115
  --sources=VALUE list of source files (see doc)
1918
2116
  --transfer-info=VALUE parameters for transfer agent
1919
2117
  --src-type=ENUM type of file list: list, pair
1920
- --transfer=ENUM type of transfer agent: direct, httpgw, connect, node
2118
+ --transfer=ENUM type of transfer agent: direct, node, connect, httpgw, trsdk
1921
2119
  --progress=ENUM type of progress bar: none, native, multi
1922
2120
 
1923
2121
 
@@ -1988,11 +2186,11 @@ OPTIONS:
1988
2186
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
1989
2187
  --username=VALUE username to log in
1990
2188
  --password=VALUE user's password
1991
- --client-id=VALUE API client identifier in application
1992
- --client-secret=VALUE API client secret in application
1993
- --redirect-uri=VALUE API client redirect URI
1994
- --auth=ENUM type of Oauth authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey, boot
1995
- --private-key=VALUE RSA private key PEM value for JWT (prefix file path with @val:@file:)
2189
+ --client-id=VALUE OAuth client identifier
2190
+ --client-secret=VALUE OAuth client secret
2191
+ --redirect-uri=VALUE OAuth redirect URI
2192
+ --auth=ENUM OAuth type of authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey, boot
2193
+ --private-key=VALUE Oauth RSA private key PEM value for JWT (prefix file path with @val:@file:)
1996
2194
 
1997
2195
 
1998
2196
  COMMAND: cos
@@ -2120,34 +2318,18 @@ OPTIONS:
2120
2318
 
2121
2319
  Note that actions and parameter values can be written in short form.
2122
2320
 
2123
- # <a name="plugins"></a>Plugins: Application URL and Authentication
2124
-
2125
- `ascli` comes with several Aspera application plugins.
2126
-
2127
- REST APIs of Aspera legacy applications (Aspera Node, Faspex, Shares, Console, Orchestrator, Server) use simple username/password authentication: HTTP Basic Authentication.
2128
-
2129
- Those are using options:
2130
-
2131
- * url
2132
- * username
2133
- * password
2134
-
2135
- Those can be provided using command line, parameter set, env var, see section above.
2136
-
2137
- Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
2138
-
2139
- # <a name="aoc"></a>Plugin: Aspera on Cloud
2321
+ # <a id="aoc"></a>Plugin: Aspera on Cloud
2140
2322
 
2141
2323
  Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
2142
2324
 
2143
2325
  It is recommended to use the wizard to set it up, but manual configuration is also possible.
2144
2326
 
2145
- ## <a name="aocwizard"></a>Configuration: using Wizard
2327
+ ## <a id="aocwizard"></a>Configuration: using Wizard
2146
2328
 
2147
2329
  `ascli` provides a configuration wizard. Here is a sample invocation :
2148
2330
 
2149
2331
  ```
2150
- $ ascli config wizard
2332
+ ascli config wizard
2151
2333
  option: url> https://myorg.ibmaspera.com
2152
2334
  Detected: Aspera on Cloud
2153
2335
  Preparing preset: aoc_myorg
@@ -2163,7 +2345,7 @@ Setting config preset as default for aspera
2163
2345
  saving config file
2164
2346
  Done.
2165
2347
  You can test with:
2166
- $ ascli aoc user info show
2348
+ ascli aoc user info show
2167
2349
  ```
2168
2350
 
2169
2351
  Optionally, it is possible to create a new organization-specific "integration".
@@ -2171,7 +2353,7 @@ For this, specify the option: `--use-generic-client=no`.
2171
2353
 
2172
2354
  This will guide you through the steps to create.
2173
2355
 
2174
- ## <a name="aocmanual"></a>Configuration: using manual setup
2356
+ ## <a id="aocmanual"></a>Configuration: using manual setup
2175
2357
 
2176
2358
  If you used the wizard (recommended): skip this section.
2177
2359
 
@@ -2183,7 +2365,7 @@ Several types of OAuth authentication are supported:
2183
2365
  * Web based authentication : authentication is made by user using a browser
2184
2366
  * URL Token : external users authentication with url tokens (public links)
2185
2367
 
2186
- The authentication method is controled by option `auth`.
2368
+ The authentication method is controlled by option `auth`.
2187
2369
 
2188
2370
  For a _quick start_, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
2189
2371
 
@@ -2191,7 +2373,7 @@ For a more convenient, browser-less, experience follow the [JWT](#jwt) section (
2191
2373
 
2192
2374
  In Oauth, a "Bearer" token are generated to authenticate REST calls. Bearer tokens are valid for a period of time.`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
2193
2375
 
2194
- ### <a name="clientreg"></a>Optional: API Client Registration
2376
+ ### <a id="clientreg"></a>Optional: API Client Registration
2195
2377
 
2196
2378
  If you use the built-in client_id and client_secret, skip this and do not set them in next section.
2197
2379
 
@@ -2215,31 +2397,31 @@ Note: for web based authentication, `ascli` listens on a local port (e.g. specif
2215
2397
 
2216
2398
  Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
2217
2399
 
2218
- ### <a name="aocpreset"></a>[option preset](#lprt) for Aspera on Cloud
2400
+ ### <a id="aocpreset"></a>[option preset](#lprt) for Aspera on Cloud
2219
2401
 
2220
2402
  If you did not use the wizard, you can also manually create a [option preset](#lprt) for `ascli` in its configuration file.
2221
2403
 
2222
2404
  Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interactive input (client info from previous step):
2223
2405
 
2224
2406
  ```
2225
- $ ascli config id my_aoc_org ask url client_id client_secret
2407
+ ascli config preset ask my_aoc_org url client_id client_secret
2226
2408
  option: url> https://myorg.ibmaspera.com/
2227
2409
  option: client_id> BJLPObQiFw
2228
2410
  option: client_secret> yFS1mu-crbKuQhGFtfhYuoRW...
2229
2411
  updated: my_aoc_org
2230
2412
  ```
2231
2413
 
2232
- (This can also be done in one line using the command `config id my_aoc_org update --url=...`)
2414
+ (This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
2233
2415
 
2234
2416
  Define this [option preset](#lprt) as default configuration for the `aspera` plugin:
2235
2417
 
2236
2418
  ```
2237
- $ ascli config id default set aoc my_aoc_org
2419
+ ascli config preset set default aoc my_aoc_org
2238
2420
  ```
2239
2421
 
2240
2422
  Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
2241
2423
 
2242
- ### <a name="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
2424
+ ### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
2243
2425
 
2244
2426
  For a Browser-less, Private Key-based authentication, use the following steps.
2245
2427
 
@@ -2254,13 +2436,13 @@ This can be done using any of the following method:
2254
2436
  * using the CLI:
2255
2437
 
2256
2438
  ```
2257
- $ ascli config genkey ~/.aspera/ascli/aocapikey
2439
+ ascli config genkey ~/.aspera/ascli/aocapikey
2258
2440
  ```
2259
2441
 
2260
2442
  * `ssh-keygen`:
2261
2443
 
2262
2444
  ```
2263
- $ ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
2445
+ ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
2264
2446
  ```
2265
2447
 
2266
2448
  * `openssl`
@@ -2268,11 +2450,11 @@ $ ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
2268
2450
  (on some openssl implementation (mac) there is option: -nodes (no DES))
2269
2451
 
2270
2452
  ```
2271
- $ APIKEY=~/.aspera/ascli/aocapikey
2272
- $ openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
2273
- $ openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
2274
- $ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
2275
- $ rm -f ${APIKEY}.protected
2453
+ APIKEY=~/.aspera/ascli/aocapikey
2454
+ openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
2455
+ openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
2456
+ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
2457
+ rm -f ${APIKEY}.protected
2276
2458
  ```
2277
2459
 
2278
2460
  #### API Client JWT activation
@@ -2291,13 +2473,13 @@ If you are not using the built-in client_id and secret, JWT needs to be authoriz
2291
2473
  * Using command line
2292
2474
 
2293
2475
  ```
2294
- $ ascli aoc admin res client list
2476
+ ascli aoc admin res client list
2295
2477
  :............:.........:
2296
2478
  : id : name :
2297
2479
  :............:.........:
2298
2480
  : BJLPObQiFw : ascli :
2299
2481
  :............:.........:
2300
- $ ascli aoc admin res client --id=BJLPObQiFw modify @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2482
+ ascli aoc admin res client modify --id=BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2301
2483
  modified
2302
2484
  ```
2303
2485
 
@@ -2318,14 +2500,14 @@ open the previously generated public key located here: `$HOME/.aspera/ascli/aoca
2318
2500
  * Using command line
2319
2501
 
2320
2502
  ```
2321
- $ ascli aoc admin res user list
2503
+ ascli aoc admin res user list
2322
2504
  :........:................:
2323
2505
  : id : name :
2324
2506
  :........:................:
2325
2507
  : 109952 : Tech Support :
2326
2508
  : 109951 : LAURENT MARTIN :
2327
2509
  :........:................:
2328
- $ ascli aoc user info modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/aocapikey.pub"))}'
2510
+ ascli aoc user info modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/aocapikey.pub"))}'
2329
2511
  modified
2330
2512
  ```
2331
2513
 
@@ -2333,7 +2515,7 @@ Note: the `aspera user info show` command can be used to verify modifications.
2333
2515
 
2334
2516
  ### [option preset](#lprt) modification for JWT
2335
2517
 
2336
- To activate default use of JWT authentication for `ascli` using the [option preset](#lprt), do the folowing:
2518
+ To activate default use of JWT authentication for `ascli` using the [option preset](#lprt), do the following:
2337
2519
 
2338
2520
  * change auth method to JWT
2339
2521
  * provide location of private key
@@ -2342,7 +2524,7 @@ To activate default use of JWT authentication for `ascli` using the [option pres
2342
2524
  Execute:
2343
2525
 
2344
2526
  ```
2345
- $ ascli config id my_aoc_org update --auth=jwt --private-key=@val:@file:~/.aspera/ascli/aocapikey --username=laurent.martin.aspera@fr.ibm.com
2527
+ ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/aocapikey --username=laurent.martin.aspera@fr.ibm.com
2346
2528
  ```
2347
2529
 
2348
2530
  Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.
@@ -2350,26 +2532,29 @@ Note: the private key argument represents the actual PEM string. In order to rea
2350
2532
  After this last step, commands do not require web login anymore.
2351
2533
 
2352
2534
 
2353
- ### <a name="aocfirst"></a>First Use
2535
+ ### <a id="aocfirst"></a>First Use
2354
2536
 
2355
2537
  Once client has been registered and [option preset](#lprt) created: `ascli` can be used:
2356
2538
 
2357
2539
  ```
2358
- $ ascli aoc files br /
2540
+ ascli aoc files br /
2359
2541
  Current Workspace: Default Workspace (default)
2360
2542
  empty
2361
2543
  ```
2362
2544
 
2363
2545
 
2364
- ### Administration
2546
+ ## Administration
2365
2547
 
2366
2548
  The `admin` command allows several administrative tasks (and require admin privilege).
2367
2549
 
2368
2550
  It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
2369
2551
 
2370
- Bulk operations are possible using option `bulk` (yes,no(default)): currently: create only. In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2552
+ ### Bulk creation and deletion of resource
2371
2553
 
2372
- #### Listing resources
2554
+ Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
2555
+ In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2556
+
2557
+ ### Listing resources
2373
2558
 
2374
2559
  The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
2375
2560
 
@@ -2391,7 +2576,7 @@ Other parameters are directly sent as parameters to the GET request on API.
2391
2576
 
2392
2577
  `page` and `per_page` are normally added by `ascli` to build successive API calls to get all values if there are more than 1000. (AoC allows a maximum page size of 1000).
2393
2578
 
2394
- `q` and `sort` are available on most resrouce types.
2579
+ `q` and `sort` are available on most resource types.
2395
2580
 
2396
2581
  Other parameters depend on the type of entity (refer to AoC API).
2397
2582
 
@@ -2403,7 +2588,7 @@ Examples:
2403
2588
  ascli aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
2404
2589
  ```
2405
2590
 
2406
- * List users who logded-in before a date:
2591
+ * List users who logged-in before a date:
2407
2592
 
2408
2593
  ```
2409
2594
  ascli aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
@@ -2419,40 +2604,85 @@ Refer to the AoC API for full list of query parameters, or use the browser in de
2419
2604
 
2420
2605
  Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
2421
2606
 
2422
- #### Access Key secrets
2607
+ ### <a id="res_select"></a>Selecting a resource
2608
+
2609
+ Resources are identified by a unique `id`, as well as a unique `name` (case insensitive).
2610
+
2611
+ To execute an action on a specific resource, select it using one of those methods:
2423
2612
 
2424
- In order to access some administrative actions on "nodes" (in fact, access keys), the associated
2425
- secret is required, it is usually provided using the `secret` option. For example in a command like:
2613
+ * *recommended:* give id directly on command line *after the action*: `aoc admin res node show 123`
2614
+ * give name on command line *after the action*: `aoc admin res node show name abc`
2615
+ * provide option `id` : `aoc admin res node show --id=123`
2616
+ * provide option `name` : `aoc admin res node show --name=abc`
2617
+
2618
+ ### Access Key secrets
2619
+
2620
+ In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
2621
+ It is usually provided using the `secret` option.
2622
+ For example in a command like:
2426
2623
 
2427
2624
  ```
2428
- $ ascli aoc admin res node --id="access_key1" --secret="secret1" v3 info
2625
+ ascli aoc admin res node --id=123 --secret="secret1" v3 info
2429
2626
  ```
2430
2627
 
2431
- It is also possible to provide a set of secrets used on a regular basis. This can be done using the `secrets` option. The value provided shall be a Hash, where keys are access key ids, and values are the associated secrets.
2628
+ It is also possible to provide a set of secrets used on a regular basis using the [secret vault](#vault).
2432
2629
 
2433
- First choose a repository name, for example `my_secrets`, and populate it like this:
2630
+ ### Activity
2631
+
2632
+ The activity app can be queried with:
2434
2633
 
2435
2634
  ```
2436
- $ ascli conf id my_secrets set 'access_key1' 'secret1'
2437
- $ ascli conf id my_secrets set 'access_key2' 'secret2'
2438
- $ ascli conf id default get config
2439
- "cli_default"
2635
+ ascli aoc admin analytics transfers
2440
2636
  ```
2441
2637
 
2442
- Here above, one already has set a `config` global preset to preset `cli_default` (refer to earlier in documentation), then the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
2638
+ It can also support filters and send notification using option `notif_to`. a template is defined using option `notif_template` :
2639
+
2640
+ `mytemplate.erb`:
2443
2641
 
2444
2642
  ```
2445
- $ ascli conf id cli_default set secrets @val:@preset:my_secrets
2643
+ From: <%=from_name%> <<%=from_email%>>
2644
+ To: <<%=ev['user_email']%>>
2645
+ Subject: <%=ev['files_completed']%> files received
2646
+
2647
+ Dear <%=ev[:user_email.to_s]%>,
2648
+ We received <%=ev['files_completed']%> files for a total of <%=ev['transferred_bytes']%> bytes, starting with file:
2649
+ <%=ev['content']%>
2650
+
2651
+ Thank you.
2446
2652
  ```
2653
+ The environment provided contains the following additional variable:
2447
2654
 
2448
- A secret repository can always be selected at runtime using `--secrets=@preset:xxxx`, or `--secrets=@json:'{"accesskey1":"secret1"}'`
2655
+ * ev : all details on the transfer event
2656
+
2657
+ Example:
2449
2658
 
2450
- #### Examples
2659
+ ```
2660
+ ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 \
2661
+ --query=@json:'{"status":"completed","direction":"receive"}' \
2662
+ --notif-to=active --notif-template=@file:mytemplate.erb
2663
+ ```
2451
2664
 
2452
- * Bulk creation
2665
+ Options:
2666
+
2667
+ * `once_only` keep track of last date it was called, so next call will get only new events
2668
+ * `query` filter (on API call)
2669
+ * `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
2670
+
2671
+ Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
2672
+
2673
+ ### Transfer: Using specific transfer ports
2674
+
2675
+ By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
2676
+ The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
2677
+
2678
+ ### Using ATS
2679
+
2680
+ Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
2681
+
2682
+ ### Example: Bulk creation of users
2453
2683
 
2454
2684
  ```
2455
- $ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
2685
+ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
2456
2686
  :.......:.........:
2457
2687
  : id : status :
2458
2688
  :.......:.........:
@@ -2461,20 +2691,20 @@ $ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@exampl
2461
2691
  :.......:.........:
2462
2692
  ```
2463
2693
 
2464
- * Find with filter and delete
2694
+ ### Example: Find with filter and delete
2465
2695
 
2466
2696
  ```
2467
- $ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
2697
+ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
2468
2698
  :.......:........................:
2469
2699
  : id : email :
2470
2700
  :.......:........................:
2471
2701
  : 98398 : dummyuser1@example.com :
2472
2702
  : 98399 : dummyuser2@example.com :
2473
2703
  :.......:........................:
2474
- $ thelist=$(ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
2475
- $ echo $thelist
2704
+ thelist=$(ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
2705
+ echo $thelist
2476
2706
  ["113501","354061"]
2477
- $ ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
2707
+ ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
2478
2708
  :.......:.........:
2479
2709
  : id : status :
2480
2710
  :.......:.........:
@@ -2483,7 +2713,7 @@ $ ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
2483
2713
  :.......:.........:
2484
2714
  ```
2485
2715
 
2486
- * <a name="deactuser"></a>Find deactivated users since more than 2 years
2716
+ ### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
2487
2717
 
2488
2718
  ```
2489
2719
  ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
@@ -2491,10 +2721,10 @@ ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_log
2491
2721
 
2492
2722
  To delete them use the same method as before
2493
2723
 
2494
- * Display current user's workspaces
2724
+ ### Example: Display current user's workspaces
2495
2725
 
2496
2726
  ```
2497
- $ ascli aoc user workspaces
2727
+ ascli aoc user workspaces
2498
2728
  :......:............................:
2499
2729
  : id : name :
2500
2730
  :......:............................:
@@ -2504,18 +2734,18 @@ $ ascli aoc user workspaces
2504
2734
  :......:............................:
2505
2735
  ```
2506
2736
 
2507
- * Create a sub access key in a "node"
2737
+ ### Example: Create a sub access key in a "node"
2508
2738
 
2509
2739
  Creation of a sub-access key is like creation of access key with the following difference: authentication to node API is made with accesskey (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
2510
2740
 
2511
2741
  ```
2512
- $ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create --value=@json:'{"storage":{"path":"/folder1"}}'
2742
+ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create --value=@json:'{"storage":{"path":"/folder1"}}'
2513
2743
  ```
2514
2744
 
2515
- * Display transfer events (ops/transfer)
2745
+ ### Example: Display transfer events (ops/transfer)
2516
2746
 
2517
2747
  ```
2518
- $ ascli aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
2748
+ ascli aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
2519
2749
  ```
2520
2750
 
2521
2751
  Examples of query (TODO: cleanup):
@@ -2533,16 +2763,16 @@ Examples of query (TODO: cleanup):
2533
2763
  # active_only=true|false
2534
2764
  ```
2535
2765
 
2536
- * Display node events (events)
2766
+ ### Example: Display node events (events)
2537
2767
 
2538
2768
  ```
2539
- $ ascli aoc admin res node --secret=_secret_ v3 events
2769
+ ascli aoc admin res node --secret=_secret_ v3 events
2540
2770
  ```
2541
2771
 
2542
- * display members of a workspace
2772
+ ### Example: Display members of a workspace
2543
2773
 
2544
2774
  ```
2545
- $ ascli aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
2775
+ ascli aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
2546
2776
  :.............:.........:..................................:
2547
2777
  : member_type : manager : member.email :
2548
2778
  :.............:.........:..................................:
@@ -2561,7 +2791,7 @@ other query parameters:
2561
2791
  {"workspace_membership_through":true,"include_indirect":true}
2562
2792
  ```
2563
2793
 
2564
- * <a name="aoc_sample_member"></a>add all members of a workspace to another workspace
2794
+ ### Example: <a id="aoc_sample_member"></a>add all members of a workspace to another workspace
2565
2795
 
2566
2796
  a- Get id of first workspace
2567
2797
 
@@ -2580,7 +2810,7 @@ WS2ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --se
2580
2810
  c- Extract membership information
2581
2811
 
2582
2812
  ```
2583
- $ ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
2813
+ ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
2584
2814
  ```
2585
2815
 
2586
2816
  d- Convert to creation data for second workspace:
@@ -2598,13 +2828,13 @@ jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"
2598
2828
  e- Add members to second workspace
2599
2829
 
2600
2830
  ```
2601
- $ ascli aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
2831
+ ascli aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
2602
2832
  ```
2603
2833
 
2604
- * Get users who did not log since a date
2834
+ ### Example: Get users who did not log since a date
2605
2835
 
2606
2836
  ```
2607
- $ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
2837
+ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
2608
2838
  :...............................:
2609
2839
  : email :
2610
2840
  :...............................:
@@ -2613,20 +2843,20 @@ $ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_a
2613
2843
  :...............................:
2614
2844
  ```
2615
2845
 
2616
- * List "Limited" users
2846
+ ### Example: List "Limited" users
2617
2847
 
2618
2848
  ```
2619
- $ ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
2849
+ ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
2620
2850
  ```
2621
2851
 
2622
- * Perform a multi Gbps transfer between two remote shared folders
2852
+ ### Example: Perform a multi Gbps transfer between two remote shared folders
2623
2853
 
2624
- In this example, a user has access to a workspace where two shared folders are located on differente sites, e.g. different cloud regions.
2854
+ In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
2625
2855
 
2626
2856
  First, setup the environment (skip if already done)
2627
2857
 
2628
2858
  ```
2629
- $ ascli conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
2859
+ ascli conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
2630
2860
  Detected: Aspera on Cloud
2631
2861
  Preparing preset: aoc_sedemo
2632
2862
  Using existing key:
@@ -2645,7 +2875,7 @@ Setting config preset as default for aspera
2645
2875
  saving config file
2646
2876
  Done.
2647
2877
  You can test with:
2648
- $ ascli aoc user info show
2878
+ ascli aoc user info show
2649
2879
  ```
2650
2880
 
2651
2881
  This creates the option preset "aoc_&lt;org name&gt;" to allow seamless command line access and sets it as default for aspera on cloud.
@@ -2655,20 +2885,20 @@ Then, create two shared folders located in two regions, in your files home, in a
2655
2885
  Then, transfer between those:
2656
2886
 
2657
2887
  ```
2658
- $ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
2888
+ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
2659
2889
  ```
2660
2890
 
2661
- * create registration key to register a node
2891
+ ### Example: create registration key to register a node
2662
2892
 
2663
2893
  ```
2664
- $ ascli aoc admin res admin/client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
2894
+ ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
2665
2895
  jfqslfdjlfdjfhdjklqfhdkl
2666
2896
  ```
2667
2897
 
2668
- * delete all registration keys
2898
+ ### Example: delete all registration keys
2669
2899
 
2670
2900
  ```
2671
- $ ascli aoc admin res admin/client list --fields=id --format=csv|ascli aoc admin res admin/client delete --bulk=yes --id=@lines:@stdin:
2901
+ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete --bulk=yes --id=@lines:@stdin:
2672
2902
  +-----+---------+
2673
2903
  | id | status |
2674
2904
  +-----+---------+
@@ -2679,81 +2909,107 @@ $ ascli aoc admin res admin/client list --fields=id --format=csv|ascli aoc admin
2679
2909
  +-----+---------+
2680
2910
  ```
2681
2911
 
2682
- * List packages in a given shared inbox
2912
+ ### Example: Create a node
2683
2913
 
2684
- First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
2685
- (To find out available filters, consult the API definition, or use the web interface in developer mode).
2914
+ AoC nodes as actually composed with two related entities:
2686
2915
 
2687
- Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
2916
+ * An access key created on the Transfer Server (HSTS/ATS)
2917
+ * a `node` resource in the AoC application.
2688
2918
 
2689
- ```
2690
- shbxid=$(ascli aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
2919
+ The web UI allows creation of both entities in one shot but not the CLI for more flexibility.
2920
+ Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation.
2691
2921
 
2692
- ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
2922
+ So, for example, the creation of a node using ATS in IBM Cloud looks like (see other example in this manual):
2923
+
2924
+ * create the access key on ATS
2925
+
2926
+ ```
2927
+ ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
2693
2928
  ```
2694
2929
 
2695
- ## Shared folders
2930
+ Take a note of the randomly generated `id` and `secret`.
2696
2931
 
2697
- * list shared folders in node
2932
+ * Retrieve the ATS node address
2698
2933
 
2699
2934
  ```
2700
- $ ascli aoc admin res node --id=8669 shared_folders
2935
+ ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv|cut -f2 -d,
2701
2936
  ```
2702
2937
 
2703
- * list shared folders in workspace
2938
+ * Create the node entity
2704
2939
 
2705
2940
  ```
2706
- $ ascli aoc admin res workspace --id=10818 shared_folders
2941
+ ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
2707
2942
  ```
2708
2943
 
2709
- * list members of shared folder
2944
+ Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
2945
+
2946
+ ### Example: List packages in a given shared inbox
2947
+
2948
+ First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
2949
+ (To find out available filters, consult the API definition, or use the web interface in developer mode).
2950
+
2951
+ Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
2710
2952
 
2711
2953
  ```
2712
- $ ascli aoc admin res node --id=8669 v4 perm 82 show
2954
+ shbxid=$(ascli aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
2955
+
2956
+ ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
2713
2957
  ```
2714
2958
 
2715
- ## Send a Package
2959
+ ## Packages
2960
+
2961
+ The webmail-like application.
2962
+
2963
+ ### Send a Package
2716
2964
 
2717
2965
  Send a package:
2718
2966
 
2719
2967
  ```
2720
- $ ascli aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
2968
+ ascli aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
2721
2969
  ```
2722
2970
 
2723
2971
  Notes:
2724
2972
 
2725
- * the `value` parameter can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package to find attributes.
2726
- * to provide the list of recipients, use fields: "recipients" and/or "bcc_recipients". ascli will resolve the list of email addresses to expected user ids.
2727
- * a recipient can be a shared inbox, in this case just use the name of the shared inbox as recipient.
2728
- * If a recipient is not already registered and the workspace allows external users, then the package is sent to an external user, and
2729
- * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account.
2973
+ * The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
2974
+ * List allowed shared inbox destinations with: `ascli aoc user shared_inboxes`
2975
+ * Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
2976
+ * Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
2977
+ * or just names: `"recipients":[{"The Dest"}]` . ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
2978
+ * If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
2979
+ * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
2730
2980
  * if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
2731
2981
 
2732
2982
  Examples:
2733
2983
 
2984
+ * Send a package with one file to two users, using their email
2985
+
2734
2986
  ```
2735
- $ ascli aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
2987
+ ascli aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
2736
2988
  ```
2737
2989
 
2990
+ * Send a package with one file to a shared inbox, using internal identifier, with specific transfer parameters
2991
+
2738
2992
  ```
2739
- $ ascli aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
2993
+ ascli aoc package send --value=@json:'{"name":"my delivery","recipients":[{"type":"dropbox","id":"12345"}]}' --ts=@json:'{"target_rate_kbps":100000}' my_file.dat
2740
2994
  ```
2741
2995
 
2996
+ * Send a package with one file to a shared inbox (by name) with metadata
2997
+
2742
2998
  ```
2743
- $ ascli aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
2999
+ ascli aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
2744
3000
  ```
2745
3001
 
2746
- ## <a name="aoccargo"></a>Receive new packages only
3002
+ ### <a id="aoccargo"></a>Receive new packages only (Cargo)
2747
3003
 
2748
3004
  It is possible to automatically download new packages, like using Aspera Cargo:
2749
3005
 
2750
3006
  ```
2751
- $ ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
3007
+ ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
2752
3008
  ```
2753
3009
 
2754
3010
  * `--id=ALL` (case sensitive) will download all packages
2755
- * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
2756
- * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
3011
+ * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
3012
+ * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
2757
3013
 
2758
3014
  Typically, one would execute this command on a regular basis, using the method of your choice:
2759
3015
 
@@ -2761,101 +3017,107 @@ Typically, one would execute this command on a regular basis, using the method o
2761
3017
  * Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
2762
3018
  * etc...
2763
3019
 
2764
- ## Download Files
3020
+ ## Files
3021
+
3022
+ Folder sharing app.
3023
+
3024
+ ### Download Files
2765
3025
 
2766
3026
  Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
2767
3027
 
2768
3028
  * if only one source is provided, it is downloaded
2769
3029
  * if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
2770
3030
 
2771
- ## Find Files
2772
-
2773
- The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
2774
-
2775
- The expression can be of 3 formats:
2776
-
2777
- * empty (default) : all files, equivalent to value: `exec:true`
2778
- * not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
2779
-
2780
- For instance, to find files with a special extension, use `--value='\.myext$'`
2781
-
2782
- * starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
2783
-
2784
- Examples of expressions: (using like this: `--value=exec:'<expression>'`)
3031
+ ### Shared folders
2785
3032
 
2786
- * Find files more recent than 100 days
3033
+ * list shared folders in node
2787
3034
 
2788
3035
  ```
2789
- f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
3036
+ ascli aoc admin res node --id=8669 shared_folders
2790
3037
  ```
2791
3038
 
2792
- * Find files older than 1 year on a given node and store in file list
3039
+ * list shared folders in workspace
2793
3040
 
2794
3041
  ```
2795
- $ ascli aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
3042
+ ascli aoc admin res workspace --id=10818 shared_folders
2796
3043
  ```
2797
3044
 
2798
- * Delete the files, one by one
3045
+ * list members of shared folder
2799
3046
 
2800
3047
  ```
2801
- $ cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
3048
+ ascli aoc admin res node --id=8669 v4 perm 82 show
2802
3049
  ```
2803
3050
 
2804
- * Delete the files in bulk
3051
+ ### Cross Organization transfers
2805
3052
 
2806
- ```
2807
- cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
2808
- ```
3053
+ It is possible to transfer files directly between organizations without having to first download locally and then upload...
2809
3054
 
2810
- ## Activity
3055
+ Although optional, the creation of [option preset](#lprt) is recommended to avoid placing all parameters in the command line.
2811
3056
 
2812
- The activity app can be queried with:
3057
+ Procedure to send a file from org1 to org2:
3058
+
3059
+ * Get access to Organization 1 and create a [option preset](#lprt): e.g. `org1`, for instance, use the [Wizard](#aocwizard)
3060
+ * Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
3061
+ * Get access to Organization 2 and create a [option preset](#lprt): e.g. `org2`
3062
+ * Check that access works and locate the destination folder `mydestfolder`
3063
+ * execute the following:
2813
3064
 
2814
3065
  ```
2815
- $ ascli aoc admin analytics transfers
3066
+ ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | ascli -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
2816
3067
  ```
2817
3068
 
2818
- It can also support filters and send notification using option `notif_to`. a template is defined using option `notif_template` :
3069
+ Explanation:
2819
3070
 
2820
- `mytemplate.erb`:
3071
+ * `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
3072
+ * `files node_info /mydestfolder` generate transfer information including node api credential and root id, suitable for the next command
3073
+ * `--format=json` format the output in JSON (instead of default text table)
3074
+ * `--display=data` display only the result, and remove other information, such as workspace name
3075
+ * `|` the standard output of the first command is fed into the second one
3076
+ * `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
3077
+ * `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
3078
+ * `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
3079
+ * `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
2821
3080
 
2822
- ```
2823
- From: <%=from_name%> <<%=from_email%>>
2824
- To: <<%=ev['user_email']%>>
2825
- Subject: <%=ev['files_completed']%> files received
3081
+ ### Find Files
2826
3082
 
2827
- Dear <%=ev[:user_email.to_s]%>,
2828
- We received <%=ev['files_completed']%> files for a total of <%=ev['transferred_bytes']%> bytes, starting with file:
2829
- <%=ev['content']%>
3083
+ The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
2830
3084
 
2831
- Thank you.
2832
- ```
2833
- The environment provided contains the following additional variable:
3085
+ The expression can be of 3 formats:
2834
3086
 
2835
- * ev : all details on the transfer event
3087
+ * empty (default) : all files, equivalent to value: `exec:true`
3088
+ * not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
2836
3089
 
2837
- Example:
3090
+ For instance, to find files with a special extension, use `--value='\.myext$'`
3091
+
3092
+ * starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
3093
+
3094
+ Examples of expressions: (using like this: `--value=exec:'<expression>'`)
3095
+
3096
+ * Find files more recent than 100 days
2838
3097
 
2839
3098
  ```
2840
- $ ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 \
2841
- --query=@json:'{"status":"completed","direction":"receive"}' \
2842
- --notif-to=active --notif-template=@file:mytemplate.erb
3099
+ f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
2843
3100
  ```
2844
3101
 
2845
- Options:
3102
+ * Find files older than 1 year on a given node and store in file list
2846
3103
 
2847
- * `once_only` keep track of last date it was called, so next call will get only new events
2848
- * `query` filter (on API call)
2849
- * `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
3104
+ ```
3105
+ ascli aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
3106
+ ```
2850
3107
 
2851
- Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
3108
+ * Delete the files, one by one
2852
3109
 
2853
- ## Using specific transfer ports
3110
+ ```
3111
+ cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
3112
+ ```
2854
3113
 
2855
- By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that. The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
3114
+ * Delete the files in bulk
2856
3115
 
3116
+ ```
3117
+ cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
3118
+ ```
2857
3119
 
2858
- # Plugin: Aspera Transfer Service
3120
+ # <a id="ats"></a>Plugin: Aspera Transfer Service
2859
3121
 
2860
3122
  ATS is usable either :
2861
3123
 
@@ -2865,10 +3127,13 @@ ATS is usable either :
2865
3127
 
2866
3128
  ## IBM Cloud ATS : creation of api key
2867
3129
 
3130
+ This section is about using ATS with an IBM cloud subscription.
3131
+ If you are using ATS as part of AoC, then authentication is thropugh AoC, not IBM Cloud.
3132
+
2868
3133
  First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
2869
3134
 
2870
3135
  ```
2871
- $ ibmcloud iam api-key-create mykeyname -d 'my sample key'
3136
+ ibmcloud iam api-key-create mykeyname -d 'my sample key'
2872
3137
  OK
2873
3138
  API key mykeyname was created
2874
3139
 
@@ -2887,78 +3152,47 @@ References:
2887
3152
  * [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
2888
3153
  * [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
2889
3154
 
2890
-
2891
3155
  Then, to register the key by default for the ats plugin, create a preset. Execute:
2892
3156
 
2893
3157
  ```
2894
- $ ascli config id my_ibm_ats update --ibm-api-key=my_secret_api_key_here_8f8d9fdakjhfsashjk678
2895
- $ ascli config id default set ats my_ibm_ats
2896
- $ ascli ats api_key instances
3158
+ ascli config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here_8f8d9fdakjhfsashjk678
3159
+ ascli config preset set default ats my_ibm_ats
3160
+ ascli ats api_key instances
2897
3161
  +--------------------------------------+
2898
3162
  | instance |
2899
3163
  +--------------------------------------+
2900
3164
  | aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee |
2901
3165
  +--------------------------------------+
2902
- $ ascli config id my_ibm_ats update --instance=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
2903
- $ ascli ats api_key create
3166
+ ascli config preset update my_ibm_ats --instance=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
3167
+ ascli ats api_key create
2904
3168
  +--------+----------------------------------------------+
2905
3169
  | key | value |
2906
3170
  +--------+----------------------------------------------+
2907
3171
  | id | ats_XXXXXXXXXXXXXXXXXXXXXXXX |
2908
3172
  | secret | YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY |
2909
3173
  +--------+----------------------------------------------+
2910
- $ ascli config id my_ibm_ats update --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
2911
- ```
2912
- ## Cross Organization transfers
2913
-
2914
- It is possible to transfer files directly between organizations without having to first download locally and then upload...
2915
-
2916
- Although optional, the creation of [option preset](#lprt) is recommended to avoid placing all parameters in the command line.
2917
-
2918
- Procedure to send a file from org1 to org2:
2919
-
2920
- * Get access to Organization 1 and create a [option preset](#lprt): e.g. `org1`, for instance, use the [Wizard](#aocwizard)
2921
- * Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
2922
- * Get access to Organization 2 and create a [option preset](#lprt): e.g. `org2`
2923
- * Check that access works and locate the destination folder `mydestfolder`
2924
- * execute the following:
2925
-
3174
+ ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
2926
3175
  ```
2927
- $ ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | ascli -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
2928
- ```
2929
-
2930
- Explanation:
2931
-
2932
- * `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
2933
- * `files node_info /mydestfolder` generate transfer information including node api credential and root id, suitable for the next command
2934
- * `--format=json` format the output in JSON (instead of default text table)
2935
- * `--display=data` display only the result, and remove other information, such as workspace name
2936
- * `|` the standard output of the first command is fed into the second one
2937
- * `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
2938
- * `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
2939
- * `--transfer=node` use transfer agent type `node` instead of default [`direct`](#direct)
2940
- * `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
2941
3176
 
2942
- Note that when using a POSIX shell, another possibility to write `cmd1 | cmd2 --transfer-info=@json:stdin:` is `cmd2 --transfer-info=@json:$(cmd1)` instead of ``
2943
3177
  ## Examples
2944
3178
 
2945
- Example: create access key on softlayer:
3179
+ Example: create access key on IBM Cloud (softlayer):
2946
3180
 
2947
3181
  ```
2948
- $ ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
3182
+ ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
2949
3183
  ```
2950
3184
 
2951
3185
  Example: create access key on AWS:
2952
3186
 
2953
3187
  ```
2954
- $ ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
3188
+ ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
2955
3189
 
2956
3190
  ```
2957
3191
 
2958
3192
  Example: create access key on Azure SAS:
2959
3193
 
2960
3194
  ```
2961
- $ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
3195
+ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
2962
3196
 
2963
3197
  ```
2964
3198
 
@@ -2967,7 +3201,7 @@ $ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id
2967
3201
  Example: create access key on Azure:
2968
3202
 
2969
3203
  ```
2970
- $ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'
3204
+ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'
2971
3205
 
2972
3206
  ```
2973
3207
 
@@ -2977,74 +3211,77 @@ delete all my access keys:
2977
3211
  for k in $(ascli ats access_key list --field=id --format=csv);do ascli ats access_key id $k delete;done
2978
3212
  ```
2979
3213
 
3214
+ The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
3215
+
2980
3216
  # Plugin: IBM Aspera High Speed Transfer Server (transfer)
2981
3217
 
2982
- This plugin works at FASP level (SSH/ascp/ascmd) and does not use the node API.
3218
+ This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
3219
+ It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
3220
+ Modern mode is to use the node API and transfer tokens.
2983
3221
 
2984
3222
  ## Authentication
2985
3223
 
2986
3224
  Both password and SSH keys auth are supported.
2987
3225
 
2988
- If not username is provided, the default transfer user `xfer` is used.
3226
+ If username is not provided, the default transfer user `xfer` is used.
2989
3227
 
2990
- If no ssh password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used.
3228
+ If no SSH password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used:
2991
3229
 
2992
3230
  ```
2993
- $ ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
3231
+ ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
2994
3232
  ```
2995
3233
 
2996
- Multiple SSH key paths can be provided. The value of the parameter `ssh_keys` can be a single value or an array. Each value is a path to a private key and is expanded ("~" is replaced with the user's home folder).
3234
+ Multiple SSH key paths can be provided.
3235
+ The value of the parameter `ssh_keys` can be a single value or an array.
3236
+ Each value is a path to a private key and is expanded (`~` is replaced with the user's home folder).
2997
3237
 
2998
3238
  Examples:
2999
3239
 
3000
3240
  ```
3001
- $ ascli server --ssh-keys=~/.ssh/id_rsa
3002
- $ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
3003
- $ ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
3241
+ ascli server --ssh-keys=~/.ssh/id_rsa
3242
+ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
3243
+ ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
3004
3244
  ```
3005
3245
 
3006
- The underlying ssh library `net::ssh` provides several options that may be used depending on environment. By default the ssh library expect that an ssh-agent is running.
3246
+ The underlying ssh library `net::ssh` provides several options that may be used depending on environment.
3247
+ By default the ssh library expect that an ssh-agent is running.
3007
3248
 
3008
- If you get an error message such as:
3249
+ On Linux, if you get an error message such as:
3009
3250
 
3010
3251
  ```
3011
- [Linux]
3012
3252
  ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: Agent not configured
3013
3253
  ```
3014
3254
 
3015
- or
3255
+ or on Windows:
3016
3256
 
3017
3257
  ```
3018
- [Windows]
3019
3258
  ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
3020
3259
  ```
3021
3260
 
3022
- This means that you dont have such an ssh agent running:
3261
+ This means that you don't have such an ssh agent running, then:
3023
3262
 
3024
3263
  * check env var: `SSH_AGENT_SOCK`
3025
- * check if the key is protected with a passphrase
3264
+ * check if the ssh key is protected with a passphrase
3026
3265
  * [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
3027
- * To diable use of `ssh-agent`, use the option `ssh_option` like this (or set in preset):
3266
+ * To disable use of `ssh-agent`, use the option `ssh_option` like this:
3028
3267
 
3029
3268
  ```
3030
- $ ascli server --ssh-options=@ruby:'{use_agent: false}' ...
3269
+ ascli server --ssh-options=@ruby:'{use_agent: false}' ...
3031
3270
  ```
3032
3271
 
3033
- This can also be set as default using a preset.
3272
+ This can also be set as default using a global preset.
3034
3273
 
3035
3274
  ## Example
3036
3275
 
3037
- One can test the "server" application using the well known demo server:
3276
+ One can test the `server` application using the well known demo server:
3038
3277
 
3039
3278
  ```
3040
- $ ascli config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
3041
- $ ascli config id default set server aspera_demo_server
3042
- $ ascli server browse /aspera-test-dir-large
3043
- $ ascli server download /aspera-test-dir-large/200MB
3279
+ ascli config initdemo
3280
+ ascli server browse /aspera-test-dir-large
3281
+ ascli server download /aspera-test-dir-large/200MB
3044
3282
  ```
3045
3283
 
3046
- This creates a [option preset](#lprt) "aspera_demo_server" and set it as default for application "server"
3047
-
3284
+ `initdemo` creates a [option preset](#lprt) `demoserver` and set it as default for plugin `server`.
3048
3285
 
3049
3286
  # Plugin: IBM Aspera High Speed Transfer Server (node)
3050
3287
 
@@ -3076,12 +3313,12 @@ For transfers, it is possible to control how transfer is authorized using option
3076
3313
 
3077
3314
  ## Central
3078
3315
 
3079
- The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transfered files.
3316
+ The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transferred files.
3080
3317
 
3081
3318
  Filtering can be applied:
3082
3319
 
3083
3320
  ```
3084
- $ ascli node central file list
3321
+ ascli node central file list
3085
3322
  ```
3086
3323
 
3087
3324
  by providing the `validator` option, offline transfer validation can be done.
@@ -3093,7 +3330,7 @@ It is possible to start a FASPStream session using the node API:
3093
3330
  Use the "node stream create" command, then arguments are provided as a [_transfer-spec_](#transferspec).
3094
3331
 
3095
3332
  ```
3096
- $ ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
3333
+ ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
3097
3334
  ```
3098
3335
 
3099
3336
  ## Watchfolder
@@ -3106,9 +3343,9 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
3106
3343
  * configure a watchfolder to define automated transfers
3107
3344
 
3108
3345
  ```
3109
- $ ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
3110
- $ ascli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
3111
- $ ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
3346
+ ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
3347
+ ascli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
3348
+ ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
3112
3349
  ```
3113
3350
 
3114
3351
  ## Out of Transfer File Validation
@@ -3116,13 +3353,13 @@ $ ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1"
3116
3353
  Follow the Aspera Transfer Server configuration to activate this feature.
3117
3354
 
3118
3355
  ```
3119
- $ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_filter":{"max_result":1}}'
3356
+ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_filter":{"max_result":1}}'
3120
3357
  :..............:..............:............:......................................:
3121
3358
  : session_uuid : file_id : status : path :
3122
3359
  :..............:..............:............:......................................:
3123
3360
  : 1a74444c-... : 084fb181-... : validating : /home/xfer.../PKG - my title/200KB.1 :
3124
3361
  :..............:..............:............:......................................:
3125
- $ ascli node central file update --validator=ascli --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
3362
+ ascli node central file update --validator=ascli --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
3126
3363
  updated
3127
3364
  ```
3128
3365
 
@@ -3138,7 +3375,7 @@ Create another configuration for the Azure ATS instance: in section "node", name
3138
3375
  Then execute the following command:
3139
3376
 
3140
3377
  ```
3141
- $ ascli node download /share/sourcefile --to-folder=/destinationfolder --preset=awsshod --transfer=node --transfer-info=@preset:azureats
3378
+ ascli node download /share/sourcefile --to-folder=/destinationfolder --preset=awsshod --transfer=node --transfer-info=@preset:azureats
3142
3379
  ```
3143
3380
 
3144
3381
  This will get transfer information from the SHOD instance and tell the Azure ATS instance
@@ -3147,7 +3384,7 @@ to download files.
3147
3384
  ## Create access key
3148
3385
 
3149
3386
  ```
3150
- $ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
3387
+ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
3151
3388
  ```
3152
3389
 
3153
3390
  # Plugin: IBM Aspera Faspex5
@@ -3172,7 +3409,7 @@ For boot method: (will be removed in future)
3172
3409
  Use it as password and use `--auth=boot`.
3173
3410
 
3174
3411
  ```
3175
- $ ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
3412
+ ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
3176
3413
  ```
3177
3414
 
3178
3415
  Ready to use Faspex5 with CLI.
@@ -3218,7 +3455,7 @@ If no parameter `max` or `pmax` is provided, then all packages will be listed in
3218
3455
  ### Example
3219
3456
 
3220
3457
  ```
3221
- $ ascli faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
3458
+ ascli faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
3222
3459
  ```
3223
3460
 
3224
3461
  List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
@@ -3232,14 +3469,14 @@ The command is `package recv`, possible methods are:
3232
3469
  * provide a `faspe:` URI with option `link`
3233
3470
 
3234
3471
  ```
3235
- $ ascli faspex package recv --id=12345
3236
- $ ascli faspex package recv --link=faspe://...
3472
+ ascli faspex package recv --id=12345
3473
+ ascli faspex package recv --link=faspe://...
3237
3474
  ```
3238
3475
 
3239
3476
  If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
3240
3477
 
3241
3478
  ```
3242
- $ ascli faspex package list --recipient='*thedropboxname'
3479
+ ascli faspex package list --recipient='*thedropboxname'
3243
3480
  ```
3244
3481
 
3245
3482
  if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
@@ -3251,7 +3488,7 @@ The command is `faspex package send`. Package information (title, note, metadata
3251
3488
  Example:
3252
3489
 
3253
3490
  ```
3254
- $ ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
3491
+ ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
3255
3492
  ```
3256
3493
 
3257
3494
  If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
@@ -3268,7 +3505,7 @@ Like for any transfer, a notification can be sent by email using parameters: `no
3268
3505
  Example:
3269
3506
 
3270
3507
  ```
3271
- $ ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
3508
+ ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
3272
3509
  ```
3273
3510
 
3274
3511
  In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
@@ -3278,9 +3515,9 @@ In this example the notification template is directly provided on command line.
3278
3515
  Example:
3279
3516
 
3280
3517
  ```
3281
- $ ascli faspex v4 dropbox create --value=@json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
3282
- $ ascli faspex v4 dropbox list
3283
- $ ascli faspex v4 dropbox delete --id=36
3518
+ ascli faspex v4 dropbox create --value=@json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
3519
+ ascli faspex v4 dropbox list
3520
+ ascli faspex v4 dropbox delete --id=36
3284
3521
  ```
3285
3522
 
3286
3523
  ## Remote sources
@@ -3318,7 +3555,7 @@ It is possible to tell `ascli` to download newly received packages, much like th
3318
3555
  cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on Cloud plugin:
3319
3556
 
3320
3557
  ```
3321
- $ ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
3558
+ ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
3322
3559
  ```
3323
3560
 
3324
3561
  # Plugin: IBM Aspera Shares
@@ -3328,7 +3565,7 @@ Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and
3328
3565
  In Shares2, users, groups listing are paged, to display sequential pages:
3329
3566
 
3330
3567
  ```
3331
- $ for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}';done
3568
+ for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}';done
3332
3569
  ```
3333
3570
 
3334
3571
  # Plugin: IBM Cloud Object Storage
@@ -3337,9 +3574,9 @@ The IBM Cloud Object Storage provides the possibility to execute transfers using
3337
3574
  It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
3338
3575
  Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
3339
3576
 
3340
- There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the forst method. If you dont have credentials but have access to the IBM Cloud console, then use the second method.
3577
+ There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
3341
3578
 
3342
- ## Using endpoint, apikey and Ressource Instance ID (CRN)
3579
+ ## Using endpoint, apikey and Resource Instance ID (CRN)
3343
3580
 
3344
3581
  If you have those parameters already, then following options shall be provided:
3345
3582
 
@@ -3351,15 +3588,15 @@ If you have those parameters already, then following options shall be provided:
3351
3588
  For example, let us create a default configuration:
3352
3589
 
3353
3590
  ```
3354
- $ ascli conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
3355
- $ ascli conf id default set cos mycos
3591
+ ascli conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
3592
+ ascli conf id default set cos mycos
3356
3593
  ```
3357
3594
 
3358
3595
  Then, jump to the transfer example.
3359
3596
 
3360
3597
  ## Using service credential file
3361
3598
 
3362
- If you are the COS administrator and dont have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
3599
+ If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
3363
3600
 
3364
3601
  Navigation Menu &rarr; Resource List &rarr; Storage &rarr; Cloud Object Storage &rarr; Service Credentials &rarr; &lt;select or create credentials&gt; &rarr; view credentials &rarr; copy
3365
3602
 
@@ -3368,11 +3605,11 @@ Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
3368
3605
  or using the IBM Cloud CLI:
3369
3606
 
3370
3607
  ```
3371
- $ ibmcloud resource service-keys
3372
- $ ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HOME/service_creds.json
3608
+ ibmcloud resource service-keys
3609
+ ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HOME/service_creds.json
3373
3610
  ```
3374
3611
 
3375
- (if you dont have `jq` installed, extract the structure as follows)
3612
+ (if you don't have `jq` installed, extract the structure as follows)
3376
3613
 
3377
3614
  It consists in the following structure:
3378
3615
 
@@ -3407,8 +3644,8 @@ The required options for this method are:
3407
3644
  For example, let us create a default configuration:
3408
3645
 
3409
3646
  ```
3410
- $ ascli conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
3411
- $ ascli conf id default set cos mycos
3647
+ ascli conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
3648
+ ascli conf id default set cos mycos
3412
3649
  ```
3413
3650
 
3414
3651
  ## Operations, transfers
@@ -3418,8 +3655,8 @@ Let's assume you created a default configuration from once of the two previous s
3418
3655
  A subset of `node` plugin operations are supported, basically node API:
3419
3656
 
3420
3657
  ```
3421
- $ ascli cos node info
3422
- $ ascli cos node upload 'faux:///sample1G?1g'
3658
+ ascli cos node info
3659
+ ascli cos node upload 'faux:///sample1G?1g'
3423
3660
  ```
3424
3661
 
3425
3662
  Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
@@ -3448,8 +3685,10 @@ Specify the previews folder as shown in:
3448
3685
  By default, the `preview` plugin expects previews to be generated in a folder named `previews` located in the storage root. On the transfer server execute:
3449
3686
 
3450
3687
  ```
3451
- # /opt/aspera/bin/asconfigurator -x "server;preview_dir,previews"
3452
- # /opt/aspera/bin/asnodeadmin --reload
3688
+ PATH=/opt/aspera/bin:$PATH
3689
+
3690
+ asconfigurator -x "server;preview_dir,previews"
3691
+ asnodeadmin --reload
3453
3692
  ```
3454
3693
 
3455
3694
  Note: the configuration `preview_dir` is *relative* to the storage root, no need leading or trailing `/`. In general just set the value to `previews`
@@ -3462,16 +3701,18 @@ This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KBytes.
3462
3701
  To change this parameter in `aspera.conf`, use `asconfigurator`. To display the value, use `asuserdata`:
3463
3702
 
3464
3703
  ```
3465
- # /opt/aspera/bin/asuserdata -a | grep max_request_file_create_size_kb
3704
+ asuserdata -a | grep max_request_file_create_size_kb
3705
+
3466
3706
  max_request_file_create_size_kb: "1024"
3467
- # /opt/aspera/bin/asconfigurator -x "server; max_request_file_create_size_kb,16384"
3707
+
3708
+ asconfigurator -x "server; max_request_file_create_size_kb,16384"
3468
3709
  ```
3469
3710
 
3470
3711
  If you use a value different than 16777216, then specify it using option `max_size`.
3471
3712
 
3472
3713
  Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* while the generator parameter is in *Bytes* (factor of 1024).
3473
3714
 
3474
- ## <a name="prev_ext"></a>External tools: Linux
3715
+ ## <a id="prev_ext"></a>External tools: Linux
3475
3716
 
3476
3717
  The tool requires the following external tools available in the `PATH`:
3477
3718
 
@@ -3488,7 +3729,7 @@ Other OSes should work as well, but are note tested.
3488
3729
  To check if all tools are found properly, execute:
3489
3730
 
3490
3731
  ```
3491
- $ ascli preview check
3732
+ ascli preview check
3492
3733
  ```
3493
3734
 
3494
3735
  ### mimemagic
@@ -3496,22 +3737,22 @@ $ ascli preview check
3496
3737
  To benefit from extra mime type detection install gem mimemagic:
3497
3738
 
3498
3739
  ```
3499
- # gem install mimemagic
3740
+ gem install mimemagic
3500
3741
  ```
3501
3742
 
3502
3743
  or to install an earlier version if any problem:
3503
3744
 
3504
3745
  ```
3505
- # gem install mimemagic -v '~> 0.3.0'
3746
+ gem install mimemagic -v '~> 0.3.0'
3506
3747
  ```
3507
3748
 
3508
3749
  To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
3509
3750
 
3510
3751
  If not used, Mime type used for conversion is the one provided by the node API.
3511
3752
 
3512
- If used, it the `preview` command will first analyse the file content using mimemagic, and if no match, will try by extension.
3753
+ If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
3513
3754
 
3514
- ### Image: Imagemagick and optipng
3755
+ ### Image: ImageMagick and optipng
3515
3756
 
3516
3757
  ```
3517
3758
  yum install -y ImageMagick optipng
@@ -3527,25 +3768,25 @@ curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.ta
3527
3768
 
3528
3769
  ### Office: Unoconv and Libreoffice
3529
3770
 
3530
- If you dont want to have preview for office dpcuments or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
3771
+ If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
3531
3772
 
3532
3773
  The generation of preview in based on the use of `unoconv` and `libreoffice`
3533
3774
 
3534
- * Centos 8
3775
+ * CentOS 8
3535
3776
 
3536
3777
  ```
3537
- # dnf install unoconv
3778
+ dnf install unoconv
3538
3779
  ```
3539
3780
 
3540
3781
  * Amazon Linux
3541
3782
 
3542
3783
  ```
3543
- # amazon-linux-extras enable libreoffice
3544
- # yum clean metadata
3545
- # yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
3546
- # wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
3547
- # mv unoconv /usr/bin
3548
- # chmod a+x /usr/bin/unoconv
3784
+ amazon-linux-extras enable libreoffice
3785
+ yum clean metadata
3786
+ yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
3787
+ wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
3788
+ mv unoconv /usr/bin
3789
+ chmod a+x /usr/bin/unoconv
3549
3790
  ```
3550
3791
 
3551
3792
  ## Configuration
@@ -3557,9 +3798,11 @@ Like any `ascli` commands, parameters can be passed on command line or using a c
3557
3798
  Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
3558
3799
 
3559
3800
  ```
3560
- # su -s /bin/bash - xfer
3561
- $ ascli config id previewconf update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
3562
- $ ascli config id default set preview previewconf
3801
+ su -s /bin/bash - xfer
3802
+
3803
+ ascli config preset update previewconf --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
3804
+
3805
+ ascli config preset set default preview previewconf
3563
3806
  ```
3564
3807
 
3565
3808
  Here we assume that Office file generation is disabled, else remove this option.
@@ -3568,7 +3811,7 @@ Here we assume that Office file generation is disabled, else remove this option.
3568
3811
  One can check if the access key is well configured using:
3569
3812
 
3570
3813
  ```
3571
- $ ascli -Ppreviewconf node browse /
3814
+ ascli -Ppreviewconf node browse /
3572
3815
  ```
3573
3816
 
3574
3817
  This shall list the contents of the storage root of the access key.
@@ -3584,8 +3827,9 @@ Typically, for "Access key" access, the system/transfer is `xfer`. So, in order
3584
3827
  Lets do a one shot test, using the configuration previously created:
3585
3828
 
3586
3829
  ```
3587
- # su -s /bin/bash - xfer
3588
- xfer$ ascli preview scan --overwrite=always
3830
+ su -s /bin/bash - xfer
3831
+
3832
+ ascli preview scan --overwrite=always
3589
3833
  ```
3590
3834
 
3591
3835
  When the preview generator is first executed it will create a file: `.aspera_access_key`
@@ -3599,13 +3843,13 @@ Adapt the scripts to your own preference.
3599
3843
 
3600
3844
  We assume here that a configuration preset was created as shown previously.
3601
3845
 
3602
- Lets first setup a script that will be used in the sceduler and sets up the environment.
3846
+ Lets first setup a script that will be used in the scheduler and sets up the environment.
3603
3847
 
3604
3848
  Example of startup script `cron_ascli`, which sets the Ruby environment and adds some timeout protection:
3605
3849
 
3606
3850
  ```
3607
- #!/bin/bash
3608
- # set a timeout protection, just in case
3851
+ #!/bin/bash
3852
+ # set a timeout protection, just in case
3609
3853
  case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
3610
3854
  . /etc/profile.d/rvm.sh
3611
3855
  rvm use 2.6 --quiet
@@ -3615,13 +3859,13 @@ exec timeout ${tmout} ascli "${@}"
3615
3859
  Here the cronjob is created for user `xfer`.
3616
3860
 
3617
3861
  ```
3618
- xfer$ crontab<<EOF
3862
+ crontab<<EOF
3619
3863
  0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
3620
3864
  2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
3621
3865
  EOF
3622
3866
  ```
3623
3867
 
3624
- Note that the loging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
3868
+ Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
3625
3869
 
3626
3870
  ## Candidate detection for creation or update (or deletion)
3627
3871
 
@@ -3646,12 +3890,12 @@ Deletion of preview for deleted source files: not implemented yet (TODO).
3646
3890
  If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
3647
3891
 
3648
3892
  ```
3649
- $ ascli preview scan --skip-folders=@json:'["/not_here"]'
3893
+ ascli preview scan --skip-folders=@json:'["/not_here"]'
3650
3894
  ```
3651
3895
 
3652
3896
  The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
3653
3897
 
3654
- When scanning the option `value` has the same behaviour as for the `node find` command.
3898
+ When scanning the option `value` has the same behavior as for the `node find` command.
3655
3899
 
3656
3900
  For instance to filter out files beginning with `._` do:
3657
3901
 
@@ -3670,7 +3914,7 @@ Use option `skip_format` to skip generation of a format.
3670
3914
 
3671
3915
  ## Supported input Files types
3672
3916
 
3673
- The preview generator supports redering of those file categories:
3917
+ The preview generator supports rendering of those file categories:
3674
3918
 
3675
3919
  * image
3676
3920
  * pdf
@@ -3690,7 +3934,7 @@ The tool can also locally detect the mime type using gem `mimemagic`.
3690
3934
 
3691
3935
  ## Access to original files and preview creation
3692
3936
 
3693
- Standard open source tools are used to create thumnails and video previews.
3937
+ Standard open source tools are used to create thumbnails and video previews.
3694
3938
  Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
3695
3939
  The tool provides 2 ways to read and write files with the option: `file_access`
3696
3940
 
@@ -3720,28 +3964,28 @@ The `smtp` option is a hash table (extended value) with the following fields:
3720
3964
  ## Example of configuration:
3721
3965
 
3722
3966
  ```
3723
- $ ascli config id smtp_google set server smtp.google.com
3724
- $ ascli config id smtp_google set username john@gmail.com
3725
- $ ascli config id smtp_google set password P@ssw0rd
3967
+ ascli config preset set smtp_google server smtp.google.com
3968
+ ascli config preset set smtp_google username john@gmail.com
3969
+ ascli config preset set smtp_google password P@ssw0rd
3726
3970
  ```
3727
3971
 
3728
3972
  or
3729
3973
 
3730
3974
  ```
3731
- $ ascli config id smtp_google init @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
3975
+ ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
3732
3976
  ```
3733
3977
 
3734
3978
  or
3735
3979
 
3736
3980
  ```
3737
- $ ascli config id smtp_google update --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
3981
+ ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
3738
3982
  ```
3739
3983
 
3740
- Set this configation as global default, for instance:
3984
+ Set this configuration as global default, for instance:
3741
3985
 
3742
3986
  ```
3743
- $ ascli config id cli_default set smtp @val:@preset:smtp_google
3744
- $ ascli config id default set config cli_default
3987
+ ascli config preset set cli_default smtp @val:@preset:smtp_google
3988
+ ascli config preset set default config cli_default
3745
3989
  ```
3746
3990
 
3747
3991
  ## Email templates
@@ -3763,8 +4007,8 @@ Other variables are defined depending on context.
3763
4007
  Check settings with `smtp_settings` command. Send test email with `email_test`.
3764
4008
 
3765
4009
  ```
3766
- $ ascli config --smtp=@preset:smtp_google smtp
3767
- $ ascli config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
4010
+ ascli config --smtp=@preset:smtp_google smtp
4011
+ ascli config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
3768
4012
  ```
3769
4013
 
3770
4014
  ## Notifications for transfer status
@@ -3773,7 +4017,7 @@ An e-mail notification can be sent upon transfer success and failure (one email
3773
4017
 
3774
4018
  To activate, use option `notif_to`.
3775
4019
 
3776
- A default e-mail template is used, but it can be overriden with option `notif_template`.
4020
+ A default e-mail template is used, but it can be overridden with option `notif_template`.
3777
4021
 
3778
4022
  The environment provided contains the following additional variables:
3779
4023
 
@@ -3797,7 +4041,7 @@ Transfer is: <%=global_transfer_status%>
3797
4041
  This gem comes with a second executable tool providing a simplified standardized interface
3798
4042
  to start a FASP session: `asession`.
3799
4043
 
3800
- It aims at simplifying the startup of a FASP session from a programmatic stand point as formating a [_transfer-spec_](#transferspec) is:
4044
+ It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [_transfer-spec_](#transferspec) is:
3801
4045
 
3802
4046
  * common to Aspera Node API (HTTP POST /ops/transfer)
3803
4047
  * common to Aspera Connect API (browser javascript startTransfer)
@@ -3809,7 +4053,7 @@ This makes it easy to integrate with any language provided that one can spawn a
3809
4053
 
3810
4054
  The tool expect one single argument: a [_transfer-spec_](#transferspec).
3811
4055
 
3812
- If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formated [_transfer-spec_](#transferspec) on stdin.
4056
+ If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [_transfer-spec_](#transferspec) on stdin.
3813
4057
 
3814
4058
  Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
3815
4059
 
@@ -3820,7 +4064,7 @@ Note that there are special "extended" [_transfer-spec_](#transferspec) paramete
3820
4064
  * `EX_loglevel` to change log level of the tool
3821
4065
  * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
3822
4066
 
3823
- Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
4067
+ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
3824
4068
 
3825
4069
  ## Comparison of interfaces
3826
4070
 
@@ -3847,7 +4091,7 @@ echo "${MY_TSPEC}"|asession
3847
4091
  This is particularly useful for a persistent session ( with the [_transfer-spec_](#transferspec) parameter: `"keepalive":true` )
3848
4092
 
3849
4093
  ```
3850
- $ asession
4094
+ asession
3851
4095
  {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
3852
4096
  {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
3853
4097
  {"type":"DONE"}
@@ -3862,7 +4106,7 @@ Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/asp
3862
4106
  ## Help
3863
4107
 
3864
4108
  ```
3865
- $ asession -h
4109
+ asession -h
3866
4110
  USAGE
3867
4111
  asession
3868
4112
  asession -h|--help
@@ -3921,7 +4165,7 @@ Note that:
3921
4165
 
3922
4166
  * `ascli` takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
3923
4167
  * most, but not all native ascp arguments are available as standard transfer_spec parameters
3924
- * native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#direct) transfer agent (not connect or node)
4168
+ * native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
3925
4169
 
3926
4170
  ### server side and configuration
3927
4171
 
@@ -3936,18 +4180,18 @@ Once `ascli` parameters are defined, run the command using the OS native schedul
3936
4180
  ## Example
3937
4181
 
3938
4182
  ```
3939
- $ ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
4183
+ ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
3940
4184
 
3941
4185
  ```
3942
4186
 
3943
- The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
4187
+ The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
3944
4188
 
3945
- # Aspera Health check and Nagios
4189
+ # Health check and Nagios
3946
4190
 
3947
- Each plugin provide a `health` command that will check the health status of the application. Example:
4191
+ Most plugin provide a `health` command that will check the health status of the application. Example:
3948
4192
 
3949
4193
  ```
3950
- $ ascli console health
4194
+ ascli console health
3951
4195
  +--------+-------------+------------+
3952
4196
  | status | component | message |
3953
4197
  +--------+-------------+------------+
@@ -3964,13 +4208,13 @@ Typically, the health check uses the REST API of the application with the follow
3964
4208
  `ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
3965
4209
 
3966
4210
  ```
3967
- $ ascli server health transfer --to-folder=/Upload --format=nagios --progress=none
4211
+ ascli server health transfer --to-folder=/Upload --format=nagios --progress=none
3968
4212
  OK - [transfer:ok]
3969
- $ ascli server health asctlstatus --cmd_prefix='sudo ' --format=nagios
4213
+ ascli server health asctlstatus --cmd_prefix='sudo ' --format=nagios
3970
4214
  OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
3971
4215
  ```
3972
4216
 
3973
- # Module: `Aspera`
4217
+ # Ruby Module: `Aspera`
3974
4218
 
3975
4219
  Main components:
3976
4220
 
@@ -3981,8 +4225,8 @@ Main components:
3981
4225
  A working example can be found in the gem, example:
3982
4226
 
3983
4227
  ```
3984
- $ ascli config gem_path
3985
- $ cat $(ascli config gem_path)/../examples/transfer.rb
4228
+ ascli config gem_path
4229
+ cat $(ascli config gem_path)/../examples/transfer.rb
3986
4230
  ```
3987
4231
 
3988
4232
  This sample code shows some example of use of the API as well as
@@ -4017,132 +4261,143 @@ So, it evolved into `ascli`:
4017
4261
 
4018
4262
  # Changes (Release notes)
4019
4263
 
4264
+ * 4.5.0
4265
+
4266
+ * new: support transfer agent: [Transfer SDK](#agt_trsdk)
4267
+ * new: support [http socket options](#http_options)
4268
+ * new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
4269
+ * new: `config vault` supports encrypted passwords, also macos keychain
4270
+ * new: `config preset` command for consistency with id
4271
+ * new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
4272
+ * change: when using wss, use [ruby's CA certs](#certificates)
4273
+ * change: unexpected parameter makes exit code not zero
4274
+ * change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
4275
+ * change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
4276
+ * change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
4277
+ * fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
4278
+
4020
4279
  * 4.4.0
4021
4280
 
4022
- * new: `aoc packages list` add possibility to add filter with option `query`
4023
- * new: `aoc admin res xxx list` now get all items by default #50
4024
- * new: `preset` option can specify name or hash value
4025
- * new: `node` plugin accepts bearer token and access key as credential
4026
- * new: `node` option `token_type` allows using basic token in addition to aspera type.
4027
- * change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
4028
- * change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
4281
+ * new: `aoc packages list` add possibility to add filter with option `query`
4282
+ * new: `aoc admin res xxx list` now get all items by default #50
4283
+ * new: `preset` option can specify name or hash value
4284
+ * new: `node` plugin accepts bearer token and access key as credential
4285
+ * new: `node` option `token_type` allows using basic token in addition to aspera type.
4286
+ * change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
4287
+ * change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
4029
4288
 
4030
4289
  * 4.3.0
4031
4290
 
4032
- * new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#direct) transfer agent.
4033
- * new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
4291
+ * new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
4292
+ * new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
4034
4293
 
4035
4294
  * 4.2.2
4036
4295
 
4037
- * new: `faspex package list` retrieves the whole list, not just first page
4038
- * new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
4039
- * new: the error "Remote host is not who we expected" displays a special remediation message
4040
- * new: `conf ascp spec` displays supported transfer spec
4041
- * new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
4042
- * fix: space character in `faspe:` url are precent encoded if needed
4043
- * fix: `preview scan`: if file_id is unknown, ignore and continue scan
4044
- * change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then `ascli` exits with code 1 (instead of zero=success)
4045
- * change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
4296
+ * new: `faspex package list` retrieves the whole list, not just first page
4297
+ * new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
4298
+ * new: the error "Remote host is not who we expected" displays a special remediation message
4299
+ * new: `conf ascp spec` displays supported transfer spec
4300
+ * new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
4301
+ * fix: space character in `faspe:` url are precent encoded if needed
4302
+ * fix: `preview scan`: if file_id is unknown, ignore and continue scan
4303
+ * change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then `ascli` exits with code 1 (instead of zero=success)
4304
+ * change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
4046
4305
 
4047
4306
  * 4.2.1
4048
4307
 
4049
- * new: command `faspex package recv` supports link of type: `faspe:`
4050
- * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
4308
+ * new: command `faspex package recv` supports link of type: `faspe:`
4309
+ * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
4051
4310
 
4052
4311
  * 4.2.0
4053
4312
 
4054
- * new: command `aoc remind` to receive organization membership by email
4055
- * new: in `preview` option `value` to filter out on file name
4056
- * new: `initdemo` to initialize for demo server
4057
- * new: [`direct`](#direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
4058
- * fix: on Windows `conf ascp use` expects ascp.exe
4059
- * fix: (break) multi_session_threshold is Integer, not String
4060
- * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
4061
- * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
4062
- * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
4063
- * change: (break) parameters for resume in `transfer-info` for [`direct`](#direct) are now in sub-key `"resume"`
4313
+ * new: command `aoc remind` to receive organization membership by email
4314
+ * new: in `preview` option `value` to filter out on file name
4315
+ * new: `initdemo` to initialize for demo server
4316
+ * new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
4317
+ * fix: on Windows `conf ascp use` expects ascp.exe
4318
+ * fix: (break) multi_session_threshold is Integer, not String
4319
+ * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
4320
+ * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
4321
+ * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
4322
+ * change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
4064
4323
 
4065
4324
  * 4.1.0
4066
4325
 
4067
- * fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
4068
- * new: update documentation with regard to offline and docker installation
4069
- * new: renamed command `nagios_check` to `health`
4070
- * new: agent `http_gw` now supports upload
4071
- * new: added option `sdk_url` to install SDK from local file for offline install
4072
- * new: check new gem version periodically
4073
- * new: the --fields= option, support -_fieldname_ to remove a field from default fields
4074
- * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
4075
- * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
4076
- * new: AoC a password can be provided for a public link
4077
- * new: `conf doc` take an optional parameter to go to a section
4078
- * new: initial support for Faspex 5 Beta 1
4326
+ * fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
4327
+ * new: update documentation with regard to offline and docker installation
4328
+ * new: renamed command `nagios_check` to `health`
4329
+ * new: agent `http_gw` now supports upload
4330
+ * new: added option `sdk_url` to install SDK from local file for offline install
4331
+ * new: check new gem version periodically
4332
+ * new: the --fields= option, support -_fieldname_ to remove a field from default fields
4333
+ * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
4334
+ * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
4335
+ * new: AoC a password can be provided for a public link
4336
+ * new: `conf doc` take an optional parameter to go to a section
4337
+ * new: initial support for Faspex 5 Beta 1
4079
4338
 
4080
4339
  * 4.0.0
4081
4340
 
4082
- * now available as open source at [https://github.com/IBM/aspera-cli](https://github.com/IBM/aspera-cli) with general cleanup
4083
- * changed default tool name from `mlia` to `ascli`
4084
- * changed `aspera` command to `aoc`
4085
- * changed gem name from `asperalm` to `aspera-cli`
4086
- * changed module name from `Asperalm` to `Aspera`
4087
- * removed command `folder` in `preview`, merged to `scan`
4088
- * persistency files go to sub folder instead of main folder
4089
- * added possibility to install SDK: `config ascp install`
4341
+ * now available as open source at [https://github.com/IBM/aspera-cli](https://github.com/IBM/aspera-cli) with general cleanup
4342
+ * changed default tool name from `mlia` to `ascli`
4343
+ * changed `aspera` command to `aoc`
4344
+ * changed gem name from `asperalm` to `aspera-cli`
4345
+ * changed module name from `Asperalm` to `Aspera`
4346
+ * removed command `folder` in `preview`, merged to `scan`
4347
+ * persistency files go to sub folder instead of main folder
4348
+ * added possibility to install SDK: `config ascp install`
4090
4349
 
4091
4350
  * 0.11.8
4092
4351
 
4093
- * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
4352
+ * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
4094
4353
 
4095
4354
  * 0.11.7
4096
4355
 
4097
- * rework on rest call error handling
4098
- * use option `display` with value `data` to remove out of extraneous information
4099
- * fixed option `lock_port` not working
4100
- * generate special icon if preview failed
4101
- * possibility to choose transfer progress bar type with option `progress`
4102
- * AoC package creation now output package id
4356
+ * rework on rest call error handling
4357
+ * use option `display` with value `data` to remove out of extraneous information
4358
+ * fixed option `lock_port` not working
4359
+ * generate special icon if preview failed
4360
+ * possibility to choose transfer progress bar type with option `progress`
4361
+ * AoC package creation now output package id
4103
4362
 
4104
4363
  * 0.11.6
4105
4364
 
4106
- * orchestrator : added more choice in auth type
4107
- * preview: cleanup in generator (removed and renamed parameters)
4108
- * preview: better documentation
4109
- * preview: animated thumbnails for video (option: `video_png_conv=animated`)
4110
- * preview: new event trigger: `trevents` (`events` seems broken)
4111
- * preview: unique tmp folder to avoid clash of multiple instances
4112
- * repo: added template for secrets used for testing
4365
+ * orchestrator : added more choice in auth type
4366
+ * preview: cleanup in generator (removed and renamed parameters)
4367
+ * preview: better documentation
4368
+ * preview: animated thumbnails for video (option: `video_png_conv=animated`)
4369
+ * preview: new event trigger: `trevents` (`events` seems broken)
4370
+ * preview: unique tmp folder to avoid clash of multiple instances
4371
+ * repo: added template for secrets used for testing
4113
4372
 
4114
4373
  * 0.11.5
4115
4374
 
4116
- * added option `default_ports` for AoC (see manual)
4117
- * allow bulk delete in `aspera files` with option `bulk=yes`
4118
- * fix getting connect versions
4119
- * added section for Aix
4120
- * support all ciphers for [`direct`](#direct) agent (including gcm, etc..)
4121
- * added transfer spec param `apply_local_docroot` for [`direct`](#direct)
4375
+ * added option `default_ports` for AoC (see manual)
4376
+ * allow bulk delete in `aspera files` with option `bulk=yes`
4377
+ * fix getting connect versions
4378
+ * added section for Aix
4379
+ * support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
4380
+ * added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
4122
4381
 
4123
4382
  * 0.11.4
4124
4383
 
4125
- * possibility to give shared inbox name when sending a package (else use id and type)
4384
+ * possibility to give shared inbox name when sending a package (else use id and type)
4126
4385
 
4127
4386
  * 0.11.3
4128
4387
 
4129
- * minor fixes on multi-session: avoid exception on progress bar
4388
+ * minor fixes on multi-session: avoid exception on progress bar
4130
4389
 
4131
4390
  * 0.11.2
4132
4391
 
4133
- * fixes on multi-session: progress bat and transfer spec param for "direct"
4392
+ * fixes on multi-session: progress bat and transfer spec param for "direct"
4134
4393
 
4135
4394
  * 0.11.1
4136
4395
 
4137
- * enhanced short_link creation commands (see examples)
4396
+ * enhanced short_link creation commands (see examples)
4138
4397
 
4139
4398
  * 0.11
4140
4399
 
4141
- * add option to provide file list directly to ascp like this (only for direct transfer agent):
4142
-
4143
- ```
4144
- ... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist"}'
4145
- ```
4400
+ * add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
4146
4401
 
4147
4402
  * 0.10.18
4148
4403
 
@@ -4151,7 +4406,7 @@ So, it evolved into `ascli`:
4151
4406
  * 0.10.17
4152
4407
 
4153
4408
  * fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
4154
- * new modifier: `@list:<saparator>val1<separator>...`
4409
+ * new modifier: `@list:<separator>val1<separator>...`
4155
4410
 
4156
4411
  * 0.10.16
4157
4412
 
@@ -4173,7 +4428,7 @@ So, it evolved into `ascli`:
4173
4428
  * 0.10.12
4174
4429
 
4175
4430
  * added support for AoC node registration keys
4176
- * replaced option : `local_resume` with `transfer_info` for agent [`direct`](#direct)
4431
+ * replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
4177
4432
  * Transfer agent is no more a Singleton instance, but only one is used in CLI
4178
4433
  * `@incps` : new extended value modifier
4179
4434
  * ATS: no more provides access keys secrets: now user must provide it
@@ -4205,7 +4460,7 @@ So, it evolved into `ascli`:
4205
4460
 
4206
4461
  * 0.10.6
4207
4462
 
4208
- * FaspManager: transfer spec `authentication` no more needed for local tranfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
4463
+ * FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
4209
4464
  * gem version requirements made more open
4210
4465
 
4211
4466
  * 0.10.5
@@ -4355,10 +4610,10 @@ So, it evolved into `ascli`:
4355
4610
 
4356
4611
  * 0.9.7
4357
4612
 
4358
- * homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#direct) transfer agents
4613
+ * homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4359
4614
  * preview persistency goes to unique file by default
4360
4615
  * catch mxf extension in preview as video
4361
- * Faspex: possibility to download all paclages by specifying id=ALL
4616
+ * Faspex: possibility to download all packages by specifying id=ALL
4362
4617
  * Faspex: to come: cargo-like function to download only new packages with id=NEW
4363
4618
 
4364
4619
  * 0.9.6
@@ -4438,16 +4693,16 @@ For issues or feature requests use the Github repository and issues.
4438
4693
 
4439
4694
  You can also contribute to this open source project.
4440
4695
 
4441
- One can also create one's own command nplugin.
4696
+ One can also [create one's own plugin](#createownplugin).
4442
4697
 
4443
4698
  ## Only one value for any option
4444
4699
 
4445
4700
  Some commands and sub commands may ask for the same option name.
4446
- Currently, since option definition is position independant (last one wins), it is not possible
4701
+ Currently, since option definition is position independent (last one wins), it is not possible
4447
4702
  to give an option to a command and the same option with different value to a sub command.
4448
4703
 
4449
- For instance, if an entity is identified by the option `id` but later on the command line another `id` option is required, the later will override the earlier one, and both entity will use the same id.
4450
- As a workaround use another option, if available, to identify the entity.
4704
+ For instance, if an entity is identified by the option `id` but later on the command line another `id` option is required, then the later will override the earlier one, and both entity will use the same id.
4705
+ As a solution, use the position specific notation for selection, i.e. provide the identified just after command and do not use option `id`.
4451
4706
 
4452
4707
  This happens typically for the `node` sub command, e.g. identify the node by name instead of id.
4453
4708
 
@@ -4479,7 +4734,7 @@ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or
4479
4734
 
4480
4735
  References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
4481
4736
 
4482
- ## Miscelaneous
4737
+ ## Miscellaneous
4483
4738
 
4484
4739
  * remove rest and oauth classes and use ruby standard gems:
4485
4740