aspera-cli 4.5.0 → 4.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/docs/README.erb.md CHANGED
@@ -1,10 +1,11 @@
1
+ # Command Line Interface for IBM Aspera products
2
+
1
3
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
2
4
  <%load File.join(File.dirname(__FILE__),'doc_tools.rb')-%>
3
- <font size="+12"><center><%=tool%> : Command Line Interface for IBM Aspera products</center></font>
4
5
 
5
6
  Version : <%=gemspec.version.to_s%>
6
7
 
7
- _Laurent/2016-<%=Time.new.year%>_
8
+ Laurent/2016-<%=Time.new.year%>
8
9
 
9
10
  This gem provides the <%=tool%> Command Line Interface to IBM Aspera software.
10
11
 
@@ -18,7 +19,7 @@ Required Ruby version: <%=gemspec.required_ruby_version%>
18
19
 
19
20
  [Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
20
21
 
21
- # <a id="when_to_use"></a>When to use and when not to use
22
+ ## <a id="when_to_use"></a>When to use and when not to use
22
23
 
23
24
  <%=tool%> is designed to be used as a command line tool to:
24
25
 
@@ -47,7 +48,7 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
47
48
 
48
49
  For scripting and ad'hoc command line operations, <%=tool%> is perfect.
49
50
 
50
- # <a id="parsing"></a>Notations, Shell and Command line parsing
51
+ ## <a id="parsing"></a>Notations, Shell and Command line parsing
51
52
 
52
53
  In examples, command line operations are shown using a shell such: `bash` or `zsh`.
53
54
 
@@ -64,18 +65,18 @@ On Windows, `cmd.exe` is typically used. Windows process creation does not recei
64
65
 
65
66
  In case of doubt of argument values after parsing test like this:
66
67
 
67
- ```
68
+ ```bash
68
69
  <%=cmd%> conf echo "Hello World" arg2 3
69
70
  ```
70
71
 
71
- ```
72
+ ```bash
72
73
  "Hello World"
73
74
  ERROR: Argument: unprocessed values: ["arg2", "3"]
74
75
  ```
75
76
 
76
77
  `echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (<%=tool%>), next command line arguments are shown in the error message.
77
78
 
78
- # Quick Start
79
+ ## Quick Start
79
80
 
80
81
  This section guides you from installation, first use and advanced use.
81
82
 
@@ -83,15 +84,15 @@ First, follow the section: [Installation](#installation) (Ruby, Gem, FASP) to st
83
84
 
84
85
  Once the gem is installed, <%=tool%> shall be accessible:
85
86
 
86
- ```
87
+ ```bash
87
88
  <%=cmd%> --version
88
89
  ```
89
90
 
90
- ```
91
+ ```bash
91
92
  <%=gemspec.version.to_s%>
92
93
  ```
93
94
 
94
- ## First use
95
+ ### First use
95
96
 
96
97
  Once installation is completed, you can proceed to the first use with a demo server:
97
98
 
@@ -99,15 +100,15 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
99
100
 
100
101
  To test with Aspera demo transfer server, setup the environment and then test:
101
102
 
102
- ```
103
+ ```bash
103
104
  <%=cmd%> config initdemo
104
105
  ```
105
106
 
106
- ```
107
+ ```bash
107
108
  <%=cmd%> server browse /
108
109
  ```
109
110
 
110
- ```
111
+ ```output
111
112
  :............:...........:......:........:...........................:.......................:
112
113
  : zmode : zuid : zgid : size : mtime : name :
113
114
  :............:...........:......:........:...........................:.......................:
@@ -125,27 +126,27 @@ If you want to use <%=tool%> with another server, and in order to make further c
125
126
  * list files in a folder
126
127
  * download a file
127
128
 
128
- ```
129
+ ```bash
129
130
  <%=cmd%> config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
130
131
  ```
131
132
 
132
- ```
133
+ ```output
133
134
  updated: myserver
134
135
  ```
135
136
 
136
- ```
137
+ ```bash
137
138
  <%=cmd%> config preset set default server myserver
138
139
  ```
139
140
 
140
- ```
141
+ ```output
141
142
  updated: default&rarr;server to myserver
142
143
  ```
143
144
 
144
- ```
145
+ ```bash
145
146
  <%=cmd%> server browse /aspera-test-dir-large
146
147
  ```
147
148
 
148
- ```
149
+ ```output
149
150
  :............:...........:......:..............:...........................:............................:
150
151
  : zmode : zuid : zgid : size : mtime : name :
151
152
  :............:...........:......:..............:...........................:............................:
@@ -164,22 +165,22 @@ updated: default&rarr;server to myserver
164
165
  :............:...........:......:..............:...........................:............................:
165
166
  ```
166
167
 
167
- ```
168
+ ```bash
168
169
  <%=cmd%> server download /aspera-test-dir-large/200MB
169
170
  ```
170
171
 
171
- ```
172
+ ```output
172
173
  Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
173
174
  complete
174
175
  ```
175
176
 
176
- ## Going further
177
+ ### Going further
177
178
 
178
179
  Get familiar with configuration, options, commands : [Command Line Interface](#cli).
179
180
 
180
181
  Then, follow the section relative to the product you want to interact with ( Aspera on Cloud, Faspex, ...) : [Application Plugins](plugins)
181
182
 
182
- # <a id="installation"></a>Installation
183
+ ## <a id="installation"></a>Installation
183
184
 
184
185
  It is possible to install *either* directly on the host operating system (Linux, Windows, macOS) or as a docker container.
185
186
 
@@ -193,7 +194,7 @@ The following sections provide information on the various installation methods.
193
194
 
194
195
  An internet connection is required for the installation. If you don't have internet for the installation, refer to section [Installation without internet access](#offline_install).
195
196
 
196
- ## Docker container
197
+ ### Docker container
197
198
 
198
199
  Use this method only if you know what you do, else use the standard recommended method as described here above.
199
200
 
@@ -203,23 +204,23 @@ The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.
203
204
 
204
205
  Ensure that you have Docker installed.
205
206
 
206
- ```
207
+ ```bash
207
208
  docker --version
208
209
  ```
209
210
 
210
211
  Download the wrapping script:
211
212
 
212
- ```
213
+ ```bash
213
214
  curl -o <%=cmd%> https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
214
215
  ```
215
216
 
216
- ```
217
+ ```bash
217
218
  chmod a+x <%=cmd%>
218
219
  ```
219
220
 
220
221
  Install the container image:
221
222
 
222
- ```
223
+ ```bash
223
224
  ./<%=cmd%> install
224
225
  ```
225
226
 
@@ -232,7 +233,7 @@ The wrapping script maps the container folder `/usr/src/app/config` to configura
232
233
  To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
233
234
  To add local storage as a volume edit the script: <%=tool%> and add a `--volume` stanza.
234
235
 
235
- ## <a id="ruby"></a>Ruby
236
+ ### <a id="ruby"></a>Ruby
236
237
 
237
238
  Use this method to install on the native host.
238
239
 
@@ -248,7 +249,7 @@ The recommended installation method is `rvm` for systems with "bash-like" shell
248
249
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
249
250
  If you have a simpler better way to install Ruby version <%=gemspec.required_ruby_version%> : use it !
250
251
 
251
- ### Generic: RVM: single user installation (not root)
252
+ #### Generic: RVM: single user installation (not root)
252
253
 
253
254
  Use this method which provides more flexibility.
254
255
 
@@ -256,44 +257,44 @@ Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
256
257
 
257
258
  Install the 2 keys
258
259
 
259
- ```
260
+ ```bash
260
261
  gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
261
262
  ```
262
263
 
263
264
  Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
264
265
 
265
- ```
266
+ ```bash
266
267
  \curl -sSL https://get.rvm.io | bash -s stable
267
268
  ```
268
269
 
269
270
  If you keep the same terminal (not needed if re-login):
270
271
 
271
- ```
272
+ ```bash
272
273
  source ~/.rvm/scripts/rvm
273
274
  ```
274
275
 
275
276
  It is advised to get one of the pre-compiled ruby version, you can list with:
276
277
 
277
- ```
278
+ ```bash
278
279
  rvm list --remote
279
280
  ```
280
281
 
281
282
  Install the chosen pre-compiled Ruby version:
282
283
 
283
- ```
284
+ ```bash
284
285
  rvm install 2.7.2 --binary
285
286
  ```
286
287
 
287
288
  Ruby is now installed for the user, go on to Gem installation.
288
289
 
289
- ### Generic: RVM: global installation (as root)
290
+ #### Generic: RVM: global installation (as root)
290
291
 
291
292
  Follow the same method as single user install, but execute as "root".
292
293
 
293
294
  As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
294
295
  One can install in another location with :
295
296
 
296
- ```
297
+ ```bash
297
298
  curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
298
299
  ```
299
300
 
@@ -301,15 +302,15 @@ As root, make sure this will not collide with other application using Ruby (e.g.
301
302
  If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
302
303
  To activate ruby (and <%=cmd%>) later, source it:
303
304
 
304
- ```
305
+ ```bash
305
306
  source /etc/profile.d/rvm.sh.ok
306
307
  ```
307
308
 
308
- ```
309
+ ```bash
309
310
  rvm version
310
311
  ```
311
312
 
312
- ### Windows: Installer
313
+ #### Windows: Installer
313
314
 
314
315
  Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
315
316
 
@@ -317,48 +318,53 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
317
318
  * Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
318
319
  * At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
319
320
 
320
- ### macOS: pre-installed or `brew`
321
+ #### macOS: pre-installed or `brew`
321
322
 
322
323
  macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%=gemspec.name%> using `sudo` :
323
324
 
324
- ```
325
+ ```bash
325
326
  sudo gem install <%=gemspec.name%><%=geminstadd%>
326
327
  ```
327
328
 
328
329
  Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
329
330
 
330
- ```
331
+ ```bash
331
332
  brew install ruby
332
333
  ```
333
334
 
334
- ### Linux: package
335
+ #### Linux: package
335
336
 
336
337
  If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
337
338
 
338
339
  Example:
339
340
 
340
- ```
341
+ ```bash
341
342
  yum install -y ruby rubygems ruby-json
342
343
  ```
343
344
 
344
345
  One can cleanup the whole yum-installed ruby environment like this to uninstall:
345
346
 
346
- ```
347
+ ```bash
347
348
  gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
348
349
  ```
349
350
 
350
- ```
351
+ ```bash
351
352
  yum remove -y ruby ruby-libs
352
353
  ```
353
354
 
354
- ### Other Unixes: Aix, etc...
355
+ #### Other Unixes (AIX)
356
+
357
+ Ruby is sometimes made available as installable package through third party providers.
358
+ For example for AIX, one can look at:
359
+
360
+ <https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha#R>
355
361
 
356
362
  If your Unix does not provide a pre-built ruby, you can get it using one of those
357
- [methods](https://www.ruby-lang.org/en/documentation/installation/)
363
+ [methods](https://www.ruby-lang.org/en/documentation/installation/).
358
364
 
359
365
  For instance to build from source, and install in `/opt/ruby` :
360
366
 
361
- ```
367
+ ```bash
362
368
  wget https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.gz
363
369
 
364
370
  gzip -d ruby-2.7.2.tar.gz
@@ -376,17 +382,23 @@ make
376
382
  make install
377
383
  ```
378
384
 
379
- ## <a id="the_gem"></a>`<%=gemspec.name%>` gem
385
+ If you already have a Java JVM on your system (`java`), it is possible to use `jruby`:
386
+
387
+ <https://www.jruby.org/download>
388
+
389
+ Note that using jruby the startup time is longer than the native ruby, but transfer speed is not impacted (executed by `ascp` binary).
390
+
391
+ ### <a id="the_gem"></a>`<%=gemspec.name%>` gem
380
392
 
381
393
  Once you have Ruby and rights to install gems: Install the gem and its dependencies:
382
394
 
383
- ```
395
+ ```bash
384
396
  gem install <%=gemspec.name%><%=geminstadd%>
385
397
  ```
386
398
 
387
399
  To upgrade to the latest version:
388
400
 
389
- ```
401
+ ```bash
390
402
  gem update <%=gemspec.name%>
391
403
  ```
392
404
 
@@ -394,11 +406,11 @@ gem update <%=gemspec.name%>
394
406
 
395
407
  To check manually:
396
408
 
397
- ```
409
+ ```bash
398
410
  <%=cmd%> conf check_update
399
411
  ```
400
412
 
401
- ## <a id="fasp_prot"></a>FASP Protocol
413
+ ### <a id="fasp_prot"></a>FASP Protocol
402
414
 
403
415
  Most file transfers will be done using the FASP protocol, using `ascp`.
404
416
  Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
@@ -408,17 +420,17 @@ Only two additional files are required to perform an Aspera Transfer, which are
408
420
 
409
421
  This can be installed either be installing an Aspera transfer software, or using an embedded command:
410
422
 
411
- ```
423
+ ```bash
412
424
  <%=cmd%> conf ascp install
413
425
  ```
414
426
 
415
427
  If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
416
428
 
417
- ```
429
+ ```bash
418
430
  curl -Lso SDK.zip https://ibm.biz/aspera_sdk
419
431
  ```
420
432
 
421
- ```
433
+ ```bash
422
434
  <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
423
435
  ```
424
436
 
@@ -441,7 +453,7 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
441
453
  Several methods are provided to start a transfer.
442
454
  Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
443
455
 
444
- ## <a id="offline_install"></a>Installation in air gapped environment
456
+ ### <a id="offline_install"></a>Installation in air gapped environment
445
457
 
446
458
  Note that currently no pre-packaged version exist yet.
447
459
  A method to build one is provided here:
@@ -451,19 +463,19 @@ The procedure:
451
463
  * Follow the non-root installation procedure with RVM, including gem
452
464
  * Archive (zip, tar) the main RVM folder (includes <%=cmd%>):
453
465
 
454
- ```
466
+ ```bash
455
467
  cd $HOME && tar zcvf rvm-<%=cmd%>.tgz .rvm
456
468
  ```
457
469
 
458
470
  * Get the Aspera SDK.
459
471
 
460
- ```
472
+ ```bash
461
473
  <%=cmd%> conf --show-config --fields=sdk_url
462
474
  ```
463
475
 
464
476
  * Download the SDK archive from that URL.
465
477
 
466
- ```
478
+ ```bash
467
479
  curl -Lso SDK.zip https://ibm.biz/aspera_sdk
468
480
  ```
469
481
 
@@ -471,7 +483,7 @@ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
471
483
 
472
484
  * On target system
473
485
 
474
- ```
486
+ ```bash
475
487
  cd $HOME
476
488
 
477
489
  tar zxvf rvm-<%=cmd%>.tgz
@@ -483,11 +495,11 @@ source ~/.rvm/scripts/rvm
483
495
 
484
496
  * Add those lines to shell init (`.profile`)
485
497
 
486
- ```
498
+ ```bash
487
499
  source ~/.rvm/scripts/rvm
488
500
  ```
489
501
 
490
- # <a id="cli"></a>Command Line Interface: <%=tool%>
502
+ ## <a id="cli"></a>Command Line Interface: <%=tool%>
491
503
 
492
504
  The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
493
505
 
@@ -515,7 +527,7 @@ The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which intera
515
527
 
516
528
  Basic usage is displayed by executing:
517
529
 
518
- ```
530
+ ```bash
519
531
  <%=cmd%> -h
520
532
  ```
521
533
 
@@ -523,26 +535,26 @@ Refer to sections: [Usage](#usage) and [Sample Commands](#commands).
523
535
 
524
536
  Not all <%=tool%> features are fully documented here, the user may explore commands on the command line.
525
537
 
526
- ## Arguments : Commands and options
538
+ ### Arguments : Commands and options
527
539
 
528
540
  Arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
529
541
 
530
- There are two types of arguments: Commands and Options. Example :
542
+ There are two types of command line arguments: Commands and Options. Example :
531
543
 
532
- ```
533
- <%=cmd%> command --option-name=VAL1 VAL2
544
+ ```bash
545
+ <%=cmd%> command subcommand --option-name=VAL1 VAL2
534
546
  ```
535
547
 
536
- * executes _command_: `command`
548
+ * executes _command_: `command subcommand`
537
549
  * with one _option_: `option_name`
538
- * this option has a _value_ of: `VAL1`
550
+ * this option is given a _value_ of: `VAL1`
539
551
  * the command has one additional _argument_: `VAL2`
540
552
 
541
553
  When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `<%=cmd%> conf ov` is the same as `<%=cmd%> config overview`.
542
554
 
543
555
  The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
544
556
 
545
- ### Options
557
+ #### Options
546
558
 
547
559
  All options, e.g. `--log-level=debug`, are command line arguments that:
548
560
 
@@ -557,15 +569,15 @@ Exceptions:
557
569
  * some options (flags) don't take a value, e.g. `-r`
558
570
  * the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
559
571
 
560
- ```
572
+ ```bash
561
573
  <%=cmd%> config echo -- --sample
562
574
  ```
563
575
 
564
- ```
576
+ ```bash
565
577
  "--sample"
566
578
  ```
567
579
 
568
- Note that `--sample` is taken as an argument, and not option.
580
+ Note that here, `--sample` is taken as an argument, and not as an option, due to `--`.
569
581
 
570
582
  Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
571
583
 
@@ -575,33 +587,32 @@ The value for _any_ options can come from the following locations (in this order
575
587
  * Environment variable
576
588
  * Command line
577
589
 
578
- Environment variable starting with prefix: <%=evp%> are taken as option values,
579
- e.g. `<%=evp%>OPTION_NAME` is for `--option-name`.
590
+ Environment variable starting with prefix: <%=evp%> are taken as option values, e.g. `<%=evp%>OPTION_NAME` is for `--option-name`.
580
591
 
581
592
  Options values can be displayed for a given command by providing the `--show-config` option: `<%=cmd%> node --show-config`
582
593
 
583
- ### Commands and Arguments
594
+ #### Commands and Arguments
584
595
 
585
596
  Command line arguments that are not options are either commands or arguments. If an argument must begin with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
586
597
 
587
- ## Interactive Input
598
+ ### Interactive Input
588
599
 
589
600
  Some options and parameters are mandatory and other optional. By default, the tool will ask for missing mandatory options or parameters for interactive execution.
590
601
 
591
602
  The behavior can be controlled with:
592
603
 
593
604
  * --interactive=&lt;yes|no&gt; (default=yes if STDIN is a terminal, else no)
594
- * yes : missing mandatory parameters/options are asked to the user
595
- * no : missing mandatory parameters/options raise an error message
605
+ * yes : missing mandatory parameters/options are asked to the user
606
+ * no : missing mandatory parameters/options raise an error message
596
607
  * --ask-options=&lt;yes|no&gt; (default=no)
597
- * optional parameters/options are asked to user
608
+ * optional parameters/options are asked to user
598
609
 
599
- ## Output
610
+ ### Output
600
611
 
601
612
  Command execution will result in output (terminal, stdout/stderr).
602
613
  The information displayed depends on the action.
603
614
 
604
- ### Types of output data
615
+ #### Types of output data
605
616
 
606
617
  Depending on action, the output will contain:
607
618
 
@@ -612,15 +623,18 @@ Depending on action, the output will contain:
612
623
  * `status` : a message
613
624
  * `other_struct` : a complex structure that cannot be displayed as an array
614
625
 
615
- ### Format of output
626
+ #### Format of output
616
627
 
617
628
  By default, result of type single_object and object_list are displayed using format `table`.
618
629
  The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
619
630
 
620
631
  In a table format, when displaying "objects" (single, or list), by default, sub object are
621
- flatten (option flat_hash). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name. Setting `flat_hash` to `false` will only display one
622
- field: "user" and value is the sub hash table. When in flatten mode, it is possible to
623
- filter fields by "dotted" field name.
632
+ flattened (option `flat_hash`). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name.
633
+ Setting `flat_hash` to `false` will only display one field: "user" and value is the sub hash table.
634
+ When in flatten mode, it is possible to filter fields by "dotted" field name.
635
+
636
+ Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
637
+ If transposition of single object is not desired, use option: `transpose_single` set to `no`.
624
638
 
625
639
  The style of output can be set using the `format` parameter, supporting:
626
640
 
@@ -631,26 +645,26 @@ The style of output can be set using the `format` parameter, supporting:
631
645
  * `yaml` : YAML
632
646
  * `csv` : Comma Separated Values
633
647
 
634
- ### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
648
+ #### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
635
649
 
636
650
  Table output can be filtered using the `select` parameter. Example:
637
651
 
638
- ```
652
+ ```javascript
639
653
  <%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
640
654
  ```
641
655
 
642
- ```
656
+ ```output
643
657
  :...............................:..................................:...........:
644
658
  : name : email : ats_admin :
645
659
  :...............................:..................................:...........:
646
- : John Custis : john@example.com : true :
660
+ : John Curtis : john@example.com : true :
647
661
  : Laurent Martin : laurent@example.com : true :
648
662
  :...............................:..................................:...........:
649
663
  ```
650
664
 
651
665
  Note that `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
652
666
 
653
- ### Verbosity of output
667
+ #### Verbosity of output
654
668
 
655
669
  Output messages are categorized in 3 types:
656
670
 
@@ -664,7 +678,7 @@ The option `display` controls the level of output:
664
678
  * `data` display `data` and `error` messages
665
679
  * `error` display only error messages.
666
680
 
667
- ### Selection of output object properties
681
+ #### Selection of output object properties
668
682
 
669
683
  By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
670
684
 
@@ -675,13 +689,13 @@ By default, a table output will display one line per entry, and columns for each
675
689
  * +a,b,c : add selected properties to the default selection.
676
690
  * -a,b,c : remove selected properties from the default selection.
677
691
 
678
- ## <a id="extended"></a>Extended Value Syntax
692
+ ### <a id="extended"></a>Extended Value Syntax
679
693
 
680
694
  Usually, values of options and arguments are specified by a simple string. But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
681
695
 
682
696
  The extended value syntax is:
683
697
 
684
- ```
698
+ ```bash
685
699
  <0 or more decoders><0 or 1 reader><nothing or some text value>
686
700
  ```
687
701
 
@@ -690,11 +704,11 @@ The difference between reader and decoder is order and ordinality. Both act like
690
704
  The following "readers" are supported (returns value in []):
691
705
 
692
706
  * @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
693
- * @file:PATH : [String] read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
694
- * @path:PATH : [String] performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
707
+ * @file:PATH : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
708
+ * @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
695
709
  * @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
696
710
  * @stdin: : [String] read from stdin (no value on right)
697
- * @preset:NAME : [Hash] get whole <%=opprst%> value by name
711
+ * @preset:NAME : [Hash] get whole <%=opprst%> value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
698
712
 
699
713
  In addition it is possible to decode a value, using one or multiple decoders :
700
714
 
@@ -711,33 +725,33 @@ To display the result of an extended value, use the `config echo` command.
711
725
 
712
726
  Example: read the content of the specified file, then, base64 decode, then unzip:
713
727
 
714
- ```
728
+ ```bash
715
729
  <%=cmd%> config echo @zlib:@base64:@file:myfile.dat
716
730
  ```
717
731
 
718
732
  Example: create a value as a hash, with one key and the value is read from a file:
719
733
 
720
- ```
734
+ ```bash
721
735
  <%=cmd%> config echo @ruby:'{"token_verification_key"=>File.read("pubkey.txt")}'
722
736
  ```
723
737
 
724
738
  Example: read a csv file and create a list of hash for bulk provisioning:
725
739
 
726
- ```
740
+ ```bash
727
741
  cat test.csv
728
742
  ```
729
743
 
730
- ```
744
+ ```bash
731
745
  name,email
732
746
  lolo,laurent@example.com
733
747
  toto,titi@tutu.tata
734
748
  ```
735
749
 
736
- ```
750
+ ```bash
737
751
  <%=cmd%> config echo @csvt:@file:test.csv
738
752
  ```
739
753
 
740
- ```
754
+ ```output
741
755
  :......:.....................:
742
756
  : name : email :
743
757
  :......:.....................:
@@ -748,17 +762,17 @@ toto,titi@tutu.tata
748
762
 
749
763
  Example: create a hash and include values from preset named "config" of config file in this hash
750
764
 
751
- ```
765
+ ```javascript
752
766
  <%=cmd%> config echo @incps:@json:'{"hello":true,"incps":["config"]}'
753
767
  ```
754
768
 
755
- ```
769
+ ```bash
756
770
  {"version"=>"0.9", "hello"=>true}
757
771
  ```
758
772
 
759
773
  Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
760
774
 
761
- ## <a id="native"></a>Structured Value
775
+ ### <a id="native"></a>Structured Value
762
776
 
763
777
  Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
764
778
 
@@ -769,7 +783,7 @@ A convenient way to specify a _Structured Value_ is to use the `@json:` decoder,
769
783
 
770
784
  It is also possible to provide a _Structured Value_ in a file using `@json:@file:<path>`
771
785
 
772
- ## <a id="conffolder"></a>Configuration and Persistency Folder
786
+ ### <a id="conffolder"></a>Configuration and Persistency Folder
773
787
 
774
788
  <%=tool%> configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/<%=cmd%>`.
775
789
 
@@ -778,11 +792,11 @@ It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMED
778
792
 
779
793
  The main folder can be displayed using :
780
794
 
781
- ```
795
+ ```bash
782
796
  <%=cmd%> config folder
783
797
  ```
784
798
 
785
- ```
799
+ ```bash
786
800
  /Users/kenji/.aspera/<%=cmd%>
787
801
  ```
788
802
 
@@ -790,13 +804,15 @@ It can be overridden using the environment variable `<%=evp%>HOME`.
790
804
 
791
805
  Example (Windows):
792
806
 
793
- ```
807
+ ```output
794
808
  set <%=evp%>HOME=C:\Users\Kenji\.aspera\<%=cmd%>
809
+
795
810
  <%=cmd%> config folder
811
+
796
812
  C:\Users\Kenji\.aspera\<%=cmd%>
797
813
  ```
798
814
 
799
- ## <a id="configfile"></a>Configuration file
815
+ ### <a id="configfile"></a>Configuration file
800
816
 
801
817
  On the first execution of <%=tool%>, an empty configuration file is created in the configuration folder.
802
818
  Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
@@ -811,19 +827,19 @@ The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml` (this ca
811
827
 
812
828
  The configuration file is simply a catalog of pre-defined lists of options, called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
813
829
 
814
- ### <a id="lprt"></a><%=prstt%>
830
+ #### <a id="lprt"></a><%=prstt%>
815
831
 
816
832
  A <%=prst%> is simply a collection of parameters and their associated values in a named section in the configuration file.
817
833
 
818
834
  A named <%=prst%> can be modified directly using <%=tool%>, which will update the configuration file :
819
835
 
820
- ```
836
+ ```bash
821
837
  <%=cmd%> config preset set|delete|show|initialize|update <<%=opprst%>>
822
838
  ```
823
839
 
824
840
  The command `update` allows the easy creation of <%=prst%> by simply providing the options in their command line format, e.g. :
825
841
 
826
- ```
842
+ ```bash
827
843
  <%=cmd%> config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
828
844
  ```
829
845
 
@@ -831,49 +847,49 @@ The command `update` allows the easy creation of <%=prst%> by simply providing t
831
847
 
832
848
  The command `set` allows setting individual options in a <%=prst%>.
833
849
 
834
- ```
850
+ ```bash
835
851
  <%=cmd%> config preset set demo_server password _demo_pass_
836
852
  ```
837
853
 
838
854
  The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
839
855
 
840
- ```
856
+ ```javascript
841
857
  <%=cmd%> config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
842
858
  ```
843
859
 
844
860
  A full terminal based overview of the configuration can be displayed using:
845
861
 
846
- ```
862
+ ```bash
847
863
  <%=cmd%> config preset over
848
864
  ```
849
865
 
850
866
  A list of <%=prst%> can be displayed using:
851
867
 
852
- ```
868
+ ```bash
853
869
  <%=cmd%> config preset list
854
870
  ```
855
871
 
856
872
  A good practice is to not manually edit the configuration file and use modification commands instead.
857
873
  If necessary, the configuration file can opened in a text editor with:
858
874
 
859
- ```
875
+ ```bash
860
876
  <%=cmd%> config open
861
877
  ```
862
878
 
863
879
  Older format for commands are still supported:
864
880
 
865
- ```
881
+ ```bash
866
882
  <%=cmd%> config id <name> set|delete|show|initialize|update
867
883
  <%=cmd%> config over
868
884
  <%=cmd%> config list
869
885
  ```
870
886
 
871
887
 
872
- ### <a id="lprtconf"></a>Special <%=prstt%>: config
888
+ #### <a id="lprtconf"></a>Special <%=prstt%>: config
873
889
 
874
890
  This preset name is reserved and contains a single key: `version`. This is the version of <%=tool%> which created the file.
875
891
 
876
- ### <a id="lprtdef"></a>Special <%=prstt%>: default
892
+ #### <a id="lprtdef"></a>Special <%=prstt%>: default
877
893
 
878
894
  This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
879
895
 
@@ -883,13 +899,19 @@ Note that special plugin name: `config` can be associated with a preset that is
883
899
 
884
900
  Operations on this preset are done using regular `config` operations:
885
901
 
886
- ```
902
+ ```bash
887
903
  <%=cmd%> config preset set default _plugin_name_ _default_preset_for_plugin_
904
+ ```
905
+
906
+ ```bash
888
907
  <%=cmd%> config preset get default _plugin_name_
908
+ ```
909
+
910
+ ```javascript
889
911
  "_default_preset_for_plugin_"
890
912
  ```
891
913
 
892
- ### <a id="lplugconf"></a>Special Plugin: config
914
+ #### <a id="lplugconf"></a>Special Plugin: config
893
915
 
894
916
  Plugin `config` (not to be confused with <%=prstt%> config) is used to configure <%=tool%> but it also contains global options.
895
917
 
@@ -897,7 +919,7 @@ When <%=tool%> starts, it looks for the `default` <%=prstt%> and if there is a v
897
919
 
898
920
  If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
899
921
 
900
- ### Format of file
922
+ #### Format of file
901
923
 
902
924
  The configuration file is a hash in a YAML file. Example:
903
925
 
@@ -934,13 +956,13 @@ Values in the configuration also follow the [Extended Value Syntax](#extended).
934
956
 
935
957
  Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
936
958
 
937
- ```
959
+ ```bash
938
960
  <%=cmd%> config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/<%=cmd%>/aocapikey"
939
961
  ```
940
962
 
941
963
  This creates the <%=prst%>:
942
964
 
943
- ```
965
+ ```yaml
944
966
  ...
945
967
  my_aoc_org:
946
968
  private_key: @file:"/Users/laurent/.aspera/<%=cmd%>/aocapikey"
@@ -949,7 +971,7 @@ my_aoc_org:
949
971
 
950
972
  So, the key file will be read only at execution time, but not be embedded in the configuration file.
951
973
 
952
- ### Options evaluation order
974
+ #### Options evaluation order
953
975
 
954
976
  Some options are global, some options are available only for some plugins. (the plugin is the first level command).
955
977
 
@@ -970,39 +992,39 @@ is an underscore. E.g. --xxx-yyy on command line gives xxx_yyy in configuration
970
992
 
971
993
  The main plugin name is `config`, so it is possible to define a default <%=prst%> for the main plugin with:
972
994
 
973
- ```
995
+ ```bash
974
996
  <%=cmd%> config preset set cli_default interactive no
975
997
  ```
976
998
 
977
- ```
999
+ ```bash
978
1000
  <%=cmd%> config preset set default config cli_default
979
1001
  ```
980
1002
 
981
1003
  A <%=prst%> value can be removed with `unset`:
982
1004
 
983
- ```
1005
+ ```bash
984
1006
  <%=cmd%> config preset unset cli_default interactive
985
1007
  ```
986
1008
 
987
1009
  Example: Define options using command line:
988
1010
 
989
- ```
1011
+ ```bash
990
1012
  <%=cmd%> -N --url=x --password=y --username=y node --show-config
991
1013
  ```
992
1014
 
993
1015
  Example: Define options using a hash:
994
1016
 
995
- ```
1017
+ ```javascript
996
1018
  <%=cmd%> -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
997
1019
  ```
998
1020
 
999
- ### Examples
1021
+ #### Shares Examples
1000
1022
 
1001
1023
  For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
1002
1024
  only username/password and url are required (either on command line, or from config file).
1003
1025
  Those can usually be provided on the command line:
1004
1026
 
1005
- ```
1027
+ ```bash
1006
1028
  <%=cmd%> shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
1007
1029
  ```
1008
1030
 
@@ -1010,49 +1032,49 @@ This can also be provisioned in a config file:
1010
1032
 
1011
1033
  * Build <%=prst%>
1012
1034
 
1013
- ```
1035
+ ```bash
1014
1036
  <%=cmd%> config preset set shares06 url https://10.25.0.6
1015
1037
  <%=cmd%> config preset set shares06 username john
1016
1038
  <%=cmd%> config preset set shares06 password 4sp3ra
1017
1039
  ```
1018
1040
 
1019
- Note that this can also be done with one single command:
1041
+ This can also be done with one single command:
1020
1042
 
1021
- ```
1043
+ ```javascript
1022
1044
  <%=cmd%> config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
1023
1045
  ```
1024
1046
 
1025
1047
  or
1026
1048
 
1027
- ```
1049
+ ```bash
1028
1050
  <%=cmd%> config preset update shares06 --url=https://10.25.0.6 --username=john --password=4sp3ra
1029
1051
  ```
1030
1052
 
1031
1053
  * Define this <%=prst%> as the default <%=prst%> for the specified plugin (`shares`)
1032
1054
 
1033
- ```
1055
+ ```bash
1034
1056
  <%=cmd%> config preset set default shares shares06
1035
1057
  ```
1036
1058
 
1037
1059
  * Display the content of configuration file in table format
1038
1060
 
1039
- ```
1061
+ ```bash
1040
1062
  <%=cmd%> config overview
1041
1063
  ```
1042
1064
 
1043
1065
  * Execute a command on the shares application using default parameters
1044
1066
 
1045
- ```
1067
+ ```bash
1046
1068
  <%=cmd%> shares repo browse /
1047
1069
  ```
1048
1070
 
1049
- ## <a id="vault"></a>Secret Vault
1071
+ ### <a id="vault"></a>Secret Vault
1050
1072
 
1051
1073
  When a secret or password is needed, it is possible to store in the secret vault.
1052
1074
 
1053
- By default the vault is defined using option `secrets`.
1075
+ By default the vault is defined using option `secrets`, which can be stored in the configuration file.
1054
1076
 
1055
- ### Using system keychain
1077
+ #### Using system keychain
1056
1078
 
1057
1079
  Only on macOS.
1058
1080
 
@@ -1060,7 +1082,7 @@ It is possible to store secrets in macOS keychain (only read supported currently
1060
1082
 
1061
1083
  Set option `secrets` to value `system` to use the default keychain or use value `system:[name]` to use a custom keychain.
1062
1084
 
1063
- ### Modern config file format: encrypted in config file
1085
+ #### Modern config file format: encrypted in config file
1064
1086
 
1065
1087
  It is possible to store and use secrets encrypted.
1066
1088
  For this use the `config vault` command.
@@ -1076,13 +1098,13 @@ Then secrets can be manipulated using commands:
1076
1098
 
1077
1099
  Secrets must be uniquely identified by `url` and `username`. An optional description can be provided using option `value`.
1078
1100
 
1079
- ### Legacy config file format
1101
+ #### Legacy config file format
1080
1102
 
1081
1103
  The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
1082
1104
 
1083
1105
  For example, choose a repository name, for example `my_secrets`, and populate it like this:
1084
1106
 
1085
- ```
1107
+ ```bash
1086
1108
  <%=cmd%> conf id my_secrets set 'access_key1' 'secret1'
1087
1109
 
1088
1110
  <%=cmd%> conf id my_secrets set 'access_key2' 'secret2'
@@ -1095,7 +1117,7 @@ cli_default
1095
1117
  Here above, one has already set a `config` global preset to preset `cli_default` (refer to earlier in documentation).
1096
1118
  So the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
1097
1119
 
1098
- ```
1120
+ ```bash
1099
1121
  <%=cmd%> conf id cli_default set secrets @val:@preset:my_secrets
1100
1122
  ```
1101
1123
 
@@ -1103,35 +1125,56 @@ A secret repository can always be selected at runtime using `--secrets=@preset:x
1103
1125
 
1104
1126
  To test if a secret can be found use:
1105
1127
 
1106
- ```
1128
+ ```bash
1107
1129
  <%=cmd%> conf vault get --username=access_key1
1108
1130
  ```
1109
1131
 
1110
- ## Plugins
1132
+ ### Plugins
1111
1133
 
1112
- The CLI tool uses a plugin mechanism. The first level command (just after <%=tool%> on the command line) is the name of the concerned plugin which will execute the command. Each plugin usually represent commands sent to a specific application.
1113
- For instance, the plugin "faspex" allows operations on the application "Aspera Faspex".
1134
+ The CLI tool uses a plugin mechanism.
1135
+ The first level command (just after <%=tool%> on the command line) is the name of the concerned plugin which will execute the command.
1136
+ Each plugin usually represents commands sent to a specific application.
1137
+ For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
1114
1138
 
1115
- ### <a id="createownplugin"></a>Create your own plugin
1139
+ Available plugins can be found using command:
1116
1140
 
1141
+ ```bash
1142
+ <%=cmd%> conf plugin list
1117
1143
  ```
1118
- mkdir -p ~/.aspera/<%=cmd%>/plugins
1119
- cat<<EOF>~/.aspera/<%=cmd%>/plugins/test.rb
1120
- require 'aspera/cli/plugin'
1121
- module Aspera
1122
- module Cli
1123
- module Plugins
1124
- class Test < Plugin
1125
- ACTIONS=[]
1126
- def execute_action; puts "Hello World!"; end
1127
- end # Test
1128
- end # Plugins
1129
- end # Cli
1130
- end # Aspera
1131
- EOF
1144
+
1145
+ ```output
1146
+ +--------------+-----------------------------------------------------------------------------------+
1147
+ | plugin | path |
1148
+ +--------------+-----------------------------------------------------------------------------------+
1149
+ | shares | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/shares.rb |
1150
+ | node | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/node.rb |
1151
+ ...
1152
+ +--------------+-----------------------------------------------------------------------------------+
1153
+ ```
1154
+
1155
+ #### <a id="createownplugin"></a>Create your own plugin
1156
+
1157
+ By default plugins are looked-up in folders specifed by (multi-value) option `plugin_folder`:
1158
+
1159
+ ```javascript
1160
+ ascli --show-config --select=@json:'{"key":"plugin_folder"}'
1161
+ ```
1162
+
1163
+ You can create the skeleton of a new plugin like this:
1164
+
1165
+ ```bash
1166
+ <%=cmd%> conf plugin create foo .
1167
+ ```
1168
+
1169
+ ```output
1170
+ Created ./foo.rb
1171
+ ```
1172
+
1173
+ ```bash
1174
+ <%=cmd%> --plugin-folder=. foo
1132
1175
  ```
1133
1176
 
1134
- ### <a id="plugins"></a>Plugins: Application URL and Authentication
1177
+ #### <a id="plugins"></a>Plugins: Application URL and Authentication
1135
1178
 
1136
1179
  <%=tool%> comes with several Aspera application plugins.
1137
1180
 
@@ -1147,7 +1190,7 @@ Those can be provided using command line, parameter set, env var, see section ab
1147
1190
 
1148
1191
  Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
1149
1192
 
1150
- ## Logging, Debugging
1193
+ ### Logging, Debugging
1151
1194
 
1152
1195
  The gem is equipped with traces. By default logging level is `warn`.
1153
1196
  To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `<%=evp%>LOG_LEVEL`, or parameter in con file).
@@ -1157,31 +1200,39 @@ It is also possible to activate traces before initialization using env var `AS_L
1157
1200
  By default passwords and secrets are removed from logs.
1158
1201
  Use option `log_passwords` to change this behaviour.
1159
1202
 
1160
- ## Learning Aspera Product APIs (REST)
1203
+ ### Learning Aspera Product APIs (REST)
1161
1204
 
1162
1205
  This CLI uses REST APIs.
1163
1206
  To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
1164
1207
 
1165
1208
  In order to get traces of execution, use argument : `--log-level=debug`
1166
1209
 
1167
- ## <a id="http_options"></a>HTTP socket parameters
1210
+ ### <a id="http_options"></a>HTTP socket parameters
1168
1211
 
1169
- If the server does not provide a valid certificate, use parameter: `--insecure=yes`.
1212
+ If the server does not provide a valid certificate, use option: `--insecure=yes`.
1170
1213
 
1171
- Some of HTTP socket parameters can be adjusted, those are the parameters of Ruby [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html), for example:
1214
+ Ruby HTTP socket parameters can be adjusted.
1172
1215
 
1173
- * `read_timeout` 60 sec
1174
- * `write_timeout` 60 sec
1216
+ | parameter | default |
1217
+ |----------------------|---------|
1218
+ | `read_timeout` | 60 |
1219
+ | `write_timeout` | 60 |
1220
+ | `open_timeout` | 60 |
1221
+ | `keep_alive_timeout` | 2 |
1175
1222
 
1223
+ Values are in set **seconds** and can be of type either integer or float.
1176
1224
  Default values are the ones of Ruby.
1225
+ For details refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
1226
+
1227
+ Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
1177
1228
 
1178
1229
  Example:
1179
1230
 
1180
- ```
1231
+ ```javascript
1181
1232
  <%=cmd%> aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
1182
1233
  ```
1183
1234
 
1184
- ## <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
1235
+ ### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
1185
1236
 
1186
1237
  Some actions may require the use of a graphical tool:
1187
1238
 
@@ -1194,30 +1245,30 @@ It is also possible to force the graphical mode with option --ui :
1194
1245
  * `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1195
1246
  * `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1196
1247
 
1197
- ## HTTP proxy for REST
1248
+ ### HTTP proxy for REST
1198
1249
 
1199
1250
  To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honored by Ruby when calling REST APIs.
1200
1251
 
1201
- ## <a id="certificates"></a>SSL CA certificate bundle
1252
+ ### <a id="certificates"></a>SSL CA certificate bundle
1202
1253
 
1203
1254
  <%=tool%> uses ruby `openssl` gem, which uses the `openssl` library, so certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..). The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
1204
1255
 
1205
1256
  `ascp` also needs to validate certificates when using WSS. By default, `ascp` uses primarily certificates from hard coded path (e.g. on macOS: `/Library/Aspera/ssl`). <%=tool%> overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch. So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
1206
1257
 
1207
- ## Proxy auto config
1258
+ ### Proxy auto config
1208
1259
 
1209
1260
  The `fpac` option allows specification of a Proxy Auto Configuration (PAC) file, by its URL for local FASP agent. Supported schemes are : http:, https: and file:.
1210
1261
 
1211
1262
  The PAC file can be tested with command: `config proxy_check` , example:
1212
1263
 
1213
- ```
1264
+ ```bash
1214
1265
  <%=cmd%> config proxy_check --fpac=file:///./proxy.pac http://www.example.com
1215
1266
  PROXY proxy.example.com:8080
1216
1267
  ```
1217
1268
 
1218
1269
  This is not yet implemented to specify http proxy, so use `http_proxy` env vars.
1219
1270
 
1220
- ## <a id="client"></a>FASP configuration
1271
+ ### <a id="client"></a>FASP configuration
1221
1272
 
1222
1273
  The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
1223
1274
 
@@ -1226,12 +1277,21 @@ The `config` plugin also allows specification for the use of a local FASP client
1226
1277
  * `products` : list Aspera transfer products available locally
1227
1278
  * `connect` : list,download connect client versions available on internet
1228
1279
 
1229
- ### Show path of currently used `ascp`
1280
+ #### Show path of currently used `ascp`
1230
1281
 
1231
- ```
1282
+ ```bash
1232
1283
  <%=cmd%> config ascp show
1284
+ ```
1285
+
1286
+ ```output
1233
1287
  /Users/laurent/.aspera/ascli/sdk/ascp
1288
+ ```
1289
+
1290
+ ```bash
1234
1291
  <%=cmd%> config ascp info
1292
+ ```
1293
+
1294
+ ```output
1235
1295
  +--------------------+-----------------------------------------------------------+
1236
1296
  | key | value |
1237
1297
  +--------------------+-----------------------------------------------------------+
@@ -1239,7 +1299,7 @@ The `config` plugin also allows specification for the use of a local FASP client
1239
1299
  ...
1240
1300
  ```
1241
1301
 
1242
- ### Selection of `ascp` location for [`direct`](#agt_direct) agent
1302
+ #### Selection of `ascp` location for [`direct`](#agt_direct) agent
1243
1303
 
1244
1304
  By default, <%=tool%> uses any found local product with ascp, including SDK.
1245
1305
 
@@ -1249,8 +1309,11 @@ For a permanent change, the command `config ascp use` sets the same parameter fo
1249
1309
 
1250
1310
  Using a POSIX shell:
1251
1311
 
1312
+ ```bash
1313
+ <%=cmd%> config ascp use @path:'~/Applications/Aspera CLI/bin/ascp'
1252
1314
  ```
1253
- <%=cmd%> config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
1315
+
1316
+ ```output
1254
1317
  ascp version: 4.0.0.182279
1255
1318
  Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
1256
1319
  Saved to default global preset global_common_defaults
@@ -1258,8 +1321,11 @@ Saved to default global preset global_common_defaults
1258
1321
 
1259
1322
  Windows:
1260
1323
 
1261
- ```
1324
+ ```bash
1262
1325
  <%=cmd%> config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1326
+ ```
1327
+
1328
+ ```output
1263
1329
  ascp version: 4.0.0.182279
1264
1330
  Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1265
1331
  Saved to default global preset global_common_defaults
@@ -1267,12 +1333,15 @@ Saved to default global preset global_common_defaults
1267
1333
 
1268
1334
  If the path has spaces, read section: [Shell and Command line parsing](#parsing).
1269
1335
 
1270
- ### List locally installed Aspera Transfer products
1336
+ #### List locally installed Aspera Transfer products
1271
1337
 
1272
1338
  Locally installed Aspera products can be listed with:
1273
1339
 
1274
- ```
1340
+ ```bash
1275
1341
  <%=cmd%> config ascp products list
1342
+ ```
1343
+
1344
+ ```output
1276
1345
  :.........................................:................................................:
1277
1346
  : name : app_root :
1278
1347
  :.........................................:................................................:
@@ -1283,7 +1352,7 @@ Locally installed Aspera products can be listed with:
1283
1352
  :.........................................:................................................:
1284
1353
  ```
1285
1354
 
1286
- ### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
1355
+ #### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
1287
1356
 
1288
1357
  If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
1289
1358
 
@@ -1291,45 +1360,55 @@ Using the option use_product finds the ascp binary of the selected product.
1291
1360
 
1292
1361
  To permanently use the ascp of a product:
1293
1362
 
1294
- ```
1363
+ ```bash
1295
1364
  <%=cmd%> config ascp products use 'Aspera Connect'
1296
1365
  saved to default global preset /Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
1297
1366
  ```
1298
1367
 
1299
- ### Installation of Connect Client on command line
1368
+ #### Installation of Connect Client on command line
1300
1369
 
1301
- ```
1370
+ ```bash
1302
1371
  <%=cmd%> config ascp connect list
1303
- :...............................................:......................................:..............:
1304
- : id : title : version :
1305
- :...............................................:......................................:..............:
1306
- : urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E : Aspera Connect for Windows : 3.7.0.138427 :
1307
- : urn:uuid:A3820D20-083E-11E2-892E-0800200C9A66 : Aspera Connect for Windows 64-bit : 3.7.0.138427 :
1308
- : urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E : Aspera Connect for Windows XP : 3.7.0.138427 :
1309
- : urn:uuid:55425020-083E-11E2-892E-0800200C9A66 : Aspera Connect for Windows XP 64-bit : 3.7.0.138427 :
1310
- : urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF : Aspera Connect for Mac Intel 10.6 : 3.6.1.111259 :
1311
- : urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF : Aspera Connect for Mac Intel : 3.7.0.138427 :
1312
- : urn:uuid:213C9370-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 32 : 3.6.2.117442 :
1313
- : urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 64 : 3.7.2.141527 :
1314
- :...............................................:......................................:..............:
1315
- <%=cmd%> config ascp connect id 'Aspera Connect for Mac Intel 10.6' links list
1316
- :.............................................:..........................:.......................................................................:..........:...............:
1317
- : title : type : href : hreflang : rel :
1318
- :.............................................:..........................:.......................................................................:..........:...............:
1319
- : Mac Intel Installer : application/octet-stream : bin/AsperaConnect-3.6.1.111259-mac-intel-10.6.dmg : en : enclosure :
1320
- : Aspera Connect for Mac HTML Documentation : text/html : : en : documentation :
1321
- : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/ja-jp/pdf/Connect_User_3.7.0_OSX_ja-jp.pdf : ja-jp : documentation :
1322
- : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/en/pdf/Connect_User_3.7.0_OSX.pdf : en : documentation :
1323
- : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/es-es/pdf/Connect_User_3.7.0_OSX_es-es.pdf : es-es : documentation :
1324
- : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/fr-fr/pdf/Connect_User_3.7.0_OSX_fr-fr.pdf : fr-fr : documentation :
1325
- : Aspera Connect PDF Documentation for Mac OS : application/pdf : docs/user/osx/zh-cn/pdf/Connect_User_3.7.0_OSX_zh-cn.pdf : zh-cn : documentation :
1326
- : Aspera Connect for Mac Release Notes : text/html : http://www.asperasoft.com/en/release_notes/default_1/release_notes_54 : en : release-notes :
1327
- :.............................................:..........................:.......................................................................:..........:...............:
1328
- <%=cmd%> config ascp connect id 'Aspera Connect for Mac Intel 10.6' links id 'Mac Intel Installer' download --to-folder=.
1329
- downloaded: AsperaConnect-3.6.1.111259-mac-intel-10.6.dmg
1330
- ```
1331
-
1332
- ## <a id="agents"></a>Transfer Agents
1372
+ ```
1373
+
1374
+ ```output
1375
+ +-----------------------------------------------+--------------------------------------+-----------+
1376
+ | id | title | version |
1377
+ +-----------------------------------------------+--------------------------------------+-----------+
1378
+ | urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows | 3.11.2.63 |
1379
+ | urn:uuid:A3820D20-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows 64-bit | 3.11.2.63 |
1380
+ | urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows XP | 3.11.2.63 |
1381
+ | urn:uuid:55425020-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows XP 64-bit | 3.11.2.63 |
1382
+ | urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF | Aspera Connect for Mac Intel | 3.11.2.63 |
1383
+ | urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 | Aspera Connect for Linux 64 | 3.11.2.63 |
1384
+ +-----------------------------------------------+--------------------------------------+-----------+
1385
+ ```
1386
+
1387
+ ```bash
1388
+ <%=cmd%> config ascp connect version 'Aspera Connect for Mac Intel' list
1389
+ ```
1390
+
1391
+ ```output
1392
+ +-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
1393
+ | title | type | href | hreflang | rel |
1394
+ +-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
1395
+ | Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstaller-3.11.2.63.dmg | en | enclosure |
1396
+ | Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstallerOneClick-3.11.2.63.dmg | en | enclosure-one-click |
1397
+ | Aspera Connect for Mac HTML Documentation | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=aspera-connect-user-guide-macos | en | documentation |
1398
+ | Aspera Connect for Mac Release Notes | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=notes-release-aspera-connect-3112 | en | release-notes |
1399
+ +-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
1400
+ ```
1401
+
1402
+ ```bash
1403
+ <%=cmd%> config ascp connect version 'Aspera Connect for Mac Intel' download enclosure --to-folder=.
1404
+ ```
1405
+
1406
+ ```output
1407
+ Time: 00:00:02 ======================================================================= 100% 27766 KB/sec Time: 00:00:02
1408
+ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
1409
+ ```
1410
+
1411
+ ### <a id="agents"></a>Transfer Agents
1333
1412
 
1334
1413
  Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
1335
1414
 
@@ -1350,8 +1429,7 @@ will effectively push files to the related server from the agent node.
1350
1429
 
1351
1430
  <%=tool%> standardizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1352
1431
 
1353
-
1354
- ### <a id="agt_direct"></a>Direct
1432
+ #### <a id="agt_direct"></a>Direct
1355
1433
 
1356
1434
  The `direct` agent directly executes a local ascp.
1357
1435
  This is the default for <%=tool%>.
@@ -1377,7 +1455,7 @@ The `transfer-info` accepts the following optional parameters to control multi-s
1377
1455
  Resume: In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
1378
1456
  Sleep between iterations is:
1379
1457
 
1380
- ```
1458
+ ```bash
1381
1459
  max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
1382
1460
  ```
1383
1461
 
@@ -1385,7 +1463,7 @@ Some transfer errors are considered "retryable" (e.g. timeout) and some other no
1385
1463
 
1386
1464
  Examples:
1387
1465
 
1388
- ```
1466
+ ```javascript
1389
1467
  <%=cmd%> ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1390
1468
  <%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1391
1469
  ```
@@ -1396,11 +1474,11 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
1396
1474
  * `EX_http_proxy_url` (proxy for legacy http fallback)
1397
1475
  * `EX_ascp_args`
1398
1476
 
1399
- ### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1477
+ #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1400
1478
 
1401
1479
  By specifying option: `--transfer=connect`, <%=tool%> will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
1402
1480
 
1403
- ### <a id="agt_node"></a>Aspera Node API : Node to node transfers
1481
+ #### <a id="agt_node"></a>Aspera Node API : Node to node transfers
1404
1482
 
1405
1483
  By specifying option: `--transfer=node`, the CLI will start transfers in an Aspera
1406
1484
  Transfer Server using the Node API, either on a local or remote node.
@@ -1422,25 +1500,25 @@ If `transfer_info` is not specified and a default node has been configured (name
1422
1500
 
1423
1501
  If the `password` value begins with `Bearer ` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
1424
1502
 
1425
- ### <a id="agt_httpgw"></a>HTTP Gateway
1503
+ #### <a id="agt_httpgw"></a>HTTP Gateway
1426
1504
 
1427
1505
  If it possible to send using a HTTP gateway, in case FASP is not allowed. `transfer_info` shall have a single mandatory parameter: `url`.
1428
1506
 
1429
1507
  Example:
1430
1508
 
1431
- ```
1509
+ ```javascript
1432
1510
  <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
1433
1511
  ```
1434
1512
 
1435
1513
  Note that the gateway only supports transfers authorized with a token.
1436
1514
 
1437
- ### <a id="agt_trsdk"></a>Transfer SDK
1515
+ #### <a id="agt_trsdk"></a>Transfer SDK
1438
1516
 
1439
1517
  Another possibility is to use the Transfer SDK daemon (asperatransferd).
1440
1518
 
1441
1519
  By default it will listen on local port `55002` on `127.0.0.1`.
1442
1520
 
1443
- ## <a id="transferspec"></a>Transfer Specification
1521
+ ### <a id="transferspec"></a>Transfer Specification
1444
1522
 
1445
1523
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
1446
1524
  is described in a _transfer-spec_ (Transfer Specification), such as:
@@ -1464,7 +1542,7 @@ The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
1464
1542
 
1465
1543
  A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
1466
1544
 
1467
- ## <a id="transferparams"></a>Transfer Parameters
1545
+ ### <a id="transferparams"></a>Transfer Parameters
1468
1546
 
1469
1547
  All standard _transfer-spec_ parameters can be specified.
1470
1548
  [_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
@@ -1477,7 +1555,7 @@ References:
1477
1555
 
1478
1556
  Parameters can be displayed with commands:
1479
1557
 
1480
- ```
1558
+ ```javascript
1481
1559
  <%=cmd%> config ascp spec
1482
1560
  <%=cmd%> config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
1483
1561
  ```
@@ -1494,7 +1572,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
1494
1572
 
1495
1573
  <%=spec_table%>
1496
1574
 
1497
- ### Destination folder for transfers
1575
+ #### Destination folder for transfers
1498
1576
 
1499
1577
  The destination folder is set by <%=tool%> by default to:
1500
1578
 
@@ -1506,18 +1584,18 @@ As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path
1506
1584
  The option `to_folder` provides an equivalent and convenient way to change this parameter:
1507
1585
  `--to-folder=<path>` .
1508
1586
 
1509
- ### List of files for transfers
1587
+ #### List of files for transfers
1510
1588
 
1511
1589
  When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
1512
1590
  So, by default, the list of files to transfer will be simply specified on the command line:
1513
1591
 
1514
- ```
1592
+ ```bash
1515
1593
  <%=cmd%> server upload ~/mysample.file secondfile
1516
1594
  ```
1517
1595
 
1518
1596
  This is equivalent to:
1519
1597
 
1520
- ```
1598
+ ```bash
1521
1599
  <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
1522
1600
  ```
1523
1601
 
@@ -1532,27 +1610,33 @@ For ease of use and flexibility, the list of files to transfer is specified by t
1532
1610
 
1533
1611
  * an [Extended Value](#extended) holding an *Array of String*. Examples:
1534
1612
 
1535
- ```
1613
+ ```javascript
1536
1614
  --sources=@json:'["file1","file2"]'
1615
+ ```
1616
+
1617
+ ```bash
1537
1618
  --sources=@lines:@stdin:
1619
+ ```
1620
+
1621
+ ```ruby
1538
1622
  --sources=@ruby:'File.read("myfilelist").split("\n")'
1539
1623
  ```
1540
1624
 
1541
1625
  * `@ts` : the user provides the list of files directly in the `ts` option, in its `paths` field. Example:
1542
1626
 
1543
- ```
1627
+ ```javascript
1544
1628
  --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
1545
1629
  ```
1546
1630
 
1547
1631
  providing a file list directly to ascp:
1548
1632
 
1549
- ```
1633
+ ```javascript
1550
1634
  ... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
1551
1635
  ```
1552
1636
 
1553
1637
  * Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
1554
1638
 
1555
- ```
1639
+ ```javascript
1556
1640
  --sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
1557
1641
  ```
1558
1642
 
@@ -1565,7 +1649,7 @@ In case the file list is provided on the command line i.e. using `--sources=@arg
1565
1649
 
1566
1650
  Example:
1567
1651
 
1568
- ```
1652
+ ```bash
1569
1653
  <%=cmd%> server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
1570
1654
  ```
1571
1655
 
@@ -1582,13 +1666,13 @@ Source files are located on "Aspera on cloud", when :
1582
1666
  * the server is Aspera on Cloud, and making a download / recv
1583
1667
  * the agent is Aspera on Cloud, and making an upload / send
1584
1668
 
1585
- ### <a id="multisession"></a>Support of multi-session
1669
+ #### <a id="multisession"></a>Support of multi-session
1586
1670
 
1587
1671
  Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
1588
1672
 
1589
1673
  * when agent=node :
1590
1674
 
1591
- ```
1675
+ ```javascript
1592
1676
  --ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
1593
1677
  ```
1594
1678
 
@@ -1596,7 +1680,7 @@ Multi-session is directly supported by the node daemon.
1596
1680
 
1597
1681
  * when agent=direct :
1598
1682
 
1599
- ```
1683
+ ```javascript
1600
1684
  --ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
1601
1685
  ```
1602
1686
 
@@ -1607,33 +1691,33 @@ Multi-session spawn is done by <%=tool%>.
1607
1691
 
1608
1692
  When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
1609
1693
 
1610
- ### Examples
1694
+ #### Transfer Spec Examples
1611
1695
 
1612
1696
  * Change target rate
1613
1697
 
1614
- ```
1698
+ ```javascript
1615
1699
  --ts=@json:'{"target_rate_kbps":500000}'
1616
1700
  ```
1617
1701
 
1618
1702
  * Override the FASP SSH port to a specific TCP port:
1619
1703
 
1620
- ```
1704
+ ```javascript
1621
1705
  --ts=@json:'{"ssh_port":33002}'
1622
1706
  ```
1623
1707
 
1624
1708
  * Force http fallback mode:
1625
1709
 
1626
- ```
1710
+ ```javascript
1627
1711
  --ts=@json:'{"http_fallback":"force"}'
1628
1712
  ```
1629
1713
 
1630
1714
  * Activate progress when not activated by default on server
1631
1715
 
1632
- ```
1716
+ ```javascript
1633
1717
  --ts=@json:'{"precalculate_job_size":true}'
1634
1718
  ```
1635
1719
 
1636
- ## <a id="scheduling"></a>Lock for exclusive execution
1720
+ ### <a id="scheduling"></a>Lock for exclusive execution
1637
1721
 
1638
1722
  In some conditions, it may be desirable to ensure that <%=tool%> is not executed several times in parallel.
1639
1723
 
@@ -1648,7 +1732,7 @@ Usually the OS native scheduler already provides some sort of protection against
1648
1732
  * The Windows scheduler does this by default
1649
1733
  * Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
1650
1734
 
1651
- ```
1735
+ ```bash
1652
1736
  /usr/bin/flock -w 0 /var/cron.lock ascli ...
1653
1737
  ```
1654
1738
 
@@ -1659,25 +1743,28 @@ Example:
1659
1743
 
1660
1744
  Run this same command in two separate terminals within less than 30 seconds:
1661
1745
 
1662
- ```
1746
+ ```bash
1663
1747
  <%=cmd%> config echo @ruby:'sleep(30)' --lock-port=12345
1664
1748
  ```
1665
1749
 
1666
1750
  The first instance will sleep 30 seconds, the second one will immediately exit like this:
1667
1751
 
1668
- ```
1752
+ ```bash
1669
1753
  WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
1670
1754
  ```
1671
1755
 
1672
- ## "Proven&ccedil;ale"
1756
+ ### "Proven&ccedil;ale"
1673
1757
 
1674
1758
  `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
1675
1759
  , some are provided om shared libraries and must be activated. (e.g. using `trapd`)
1676
1760
 
1677
1761
  The list of supported *PVCL* adapters can be retrieved with command:
1678
1762
 
1679
- ```
1763
+ ```bash
1680
1764
  <%=cmd%> conf ascp info
1765
+ ```
1766
+
1767
+ ```output
1681
1768
  +--------------------+-----------------------------------------------------------+
1682
1769
  | key | value |
1683
1770
  +--------------------+-----------------------------------------------------------+
@@ -1700,13 +1787,13 @@ Those adapters can be used wherever a file path is used in `ascp` including conf
1700
1787
 
1701
1788
  The simplified format is:
1702
1789
 
1703
- ```
1790
+ ```bash
1704
1791
  <adapter>:///<sub file path>?<arg1>=<val1>&...
1705
1792
  ```
1706
1793
 
1707
1794
  One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
1708
1795
 
1709
- ## <a id="faux_testing"></a>`faux:` for testing
1796
+ ### <a id="faux_testing"></a>`faux:` for testing
1710
1797
 
1711
1798
  This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not <%=tool%>.
1712
1799
 
@@ -1714,7 +1801,7 @@ This adapter can be used to simulate a file or a directory.
1714
1801
 
1715
1802
  To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
1716
1803
 
1717
- ```
1804
+ ```bash
1718
1805
  faux:///filename?filesize
1719
1806
  ```
1720
1807
 
@@ -1729,7 +1816,7 @@ For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e
1729
1816
 
1730
1817
  To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
1731
1818
 
1732
- ```
1819
+ ```bash
1733
1820
  faux:///dirname?<arg1>=<val1>&...
1734
1821
  ```
1735
1822
 
@@ -1772,34 +1859,34 @@ Examples:
1772
1859
 
1773
1860
  * Upload 20 gibibytes of random data to file myfile to directory /Upload
1774
1861
 
1775
- ```
1862
+ ```bash
1776
1863
  <%=cmd%> server upload faux:///myfile\?20g --to-folder=/Upload
1777
1864
  ```
1778
1865
 
1779
1866
  * Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
1780
1867
 
1781
- ```
1868
+ ```bash
1782
1869
  <%=cmd%> server upload /tmp/sample --to-folder=faux://
1783
1870
  ```
1784
1871
 
1785
1872
  * Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
1786
1873
 
1787
- ```
1874
+ ```bash
1788
1875
  <%=cmd%> server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
1789
1876
  ```
1790
1877
 
1791
- ## <a id="commands"></a>Sample Commands
1878
+ ### <a id="commands"></a>Sample Commands
1792
1879
 
1793
1880
  A non complete list of commands used in unit tests:
1794
1881
 
1795
- ```
1882
+ ```bash
1796
1883
  <%=File.read(ENV["INCL_COMMANDS"])%>
1797
1884
  ...and more
1798
1885
  ```
1799
1886
 
1800
- ## <a id="usage"></a>Usage
1887
+ ### <a id="usage"></a>Usage
1801
1888
 
1802
- ```
1889
+ ```bash
1803
1890
  <%=cmd%> -h
1804
1891
  <%=File.read(ENV["INCL_USAGE"])%>
1805
1892
 
@@ -1807,17 +1894,17 @@ A non complete list of commands used in unit tests:
1807
1894
 
1808
1895
  Note that actions and parameter values can be written in short form.
1809
1896
 
1810
- # <a id="aoc"></a>Plugin: Aspera on Cloud
1897
+ ## <a id="aoc"></a>Plugin: Aspera on Cloud
1811
1898
 
1812
1899
  Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
1813
1900
 
1814
1901
  It is recommended to use the wizard to set it up, but manual configuration is also possible.
1815
1902
 
1816
- ## <a id="aocwizard"></a>Configuration: using Wizard
1903
+ ### <a id="aocwizard"></a>Configuration: using Wizard
1817
1904
 
1818
1905
  <%=tool%> provides a configuration wizard. Here is a sample invocation :
1819
1906
 
1820
- ```
1907
+ ```bash
1821
1908
  <%=cmd%> config wizard
1822
1909
  option: url> https://myorg.ibmaspera.com
1823
1910
  Detected: Aspera on Cloud
@@ -1834,7 +1921,7 @@ Setting config preset as default for aspera
1834
1921
  saving config file
1835
1922
  Done.
1836
1923
  You can test with:
1837
- <%=cmd%> aoc user info show
1924
+ <%=cmd%> aoc user profile show
1838
1925
  ```
1839
1926
 
1840
1927
  Optionally, it is possible to create a new organization-specific "integration".
@@ -1842,11 +1929,11 @@ For this, specify the option: `--use-generic-client=no`.
1842
1929
 
1843
1930
  This will guide you through the steps to create.
1844
1931
 
1845
- ## <a id="aocmanual"></a>Configuration: using manual setup
1932
+ ### <a id="aocmanual"></a>Configuration: using manual setup
1846
1933
 
1847
1934
  If you used the wizard (recommended): skip this section.
1848
1935
 
1849
- ### Configuration details
1936
+ #### Configuration details
1850
1937
 
1851
1938
  Several types of OAuth authentication are supported:
1852
1939
 
@@ -1862,7 +1949,7 @@ For a more convenient, browser-less, experience follow the [JWT](#jwt) section (
1862
1949
 
1863
1950
  In Oauth, a "Bearer" token are generated to authenticate REST calls. Bearer tokens are valid for a period of time.<%=tool%> saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
1864
1951
 
1865
- ### <a id="clientreg"></a>Optional: API Client Registration
1952
+ #### <a id="clientreg"></a>Optional: API Client Registration
1866
1953
 
1867
1954
  If you use the built-in client_id and client_secret, skip this and do not set them in next section.
1868
1955
 
@@ -1875,24 +1962,24 @@ Let's start by a registration with web based authentication (auth=web):
1875
1962
  * Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
1876
1963
  * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
1877
1964
  * Click "Create New"
1878
- * Client Name: <%=tool%>
1879
- * Redirect URIs: `http://localhost:12345`
1880
- * Origins: `localhost`
1881
- * uncheck "Prompt users to allow client to access"
1882
- * leave the JWT part for now
1965
+ * Client Name: <%=tool%>
1966
+ * Redirect URIs: `http://localhost:12345`
1967
+ * Origins: `localhost`
1968
+ * uncheck "Prompt users to allow client to access"
1969
+ * leave the JWT part for now
1883
1970
  * Save
1884
1971
 
1885
1972
  Note: for web based authentication, <%=tool%> listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For `<%=tool%>, HTTP is required, and 12345 is the default port.
1886
1973
 
1887
1974
  Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
1888
1975
 
1889
- ### <a id="aocpreset"></a><%=prst%> for Aspera on Cloud
1976
+ #### <a id="aocpreset"></a><%=prst%> for Aspera on Cloud
1890
1977
 
1891
1978
  If you did not use the wizard, you can also manually create a <%=prst%> for <%=tool%> in its configuration file.
1892
1979
 
1893
1980
  Lets create an <%=prst%> called: `my_aoc_org` using `ask` interactive input (client info from previous step):
1894
1981
 
1895
- ```
1982
+ ```bash
1896
1983
  <%=cmd%> config preset ask my_aoc_org url client_id client_secret
1897
1984
  option: url> https://myorg.ibmaspera.com/
1898
1985
  option: client_id> BJLPObQiFw
@@ -1904,17 +1991,17 @@ updated: my_aoc_org
1904
1991
 
1905
1992
  Define this <%=prst%> as default configuration for the `aspera` plugin:
1906
1993
 
1907
- ```
1994
+ ```bash
1908
1995
  <%=cmd%> config preset set default aoc my_aoc_org
1909
1996
  ```
1910
1997
 
1911
1998
  Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
1912
1999
 
1913
- ### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
2000
+ #### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
1914
2001
 
1915
2002
  For a Browser-less, Private Key-based authentication, use the following steps.
1916
2003
 
1917
- #### Key Pair Generation
2004
+ ##### Key Pair Generation
1918
2005
 
1919
2006
  In order to use JWT for Aspera on Cloud API client authentication,
1920
2007
  a private/public key pair must be generated (without passphrase)
@@ -1924,13 +2011,13 @@ This can be done using any of the following method:
1924
2011
 
1925
2012
  * using the CLI:
1926
2013
 
1927
- ```
2014
+ ```bash
1928
2015
  <%=cmd%> config genkey ~/.aspera/<%=cmd%>/aocapikey
1929
2016
  ```
1930
2017
 
1931
2018
  * `ssh-keygen`:
1932
2019
 
1933
- ```
2020
+ ```bash
1934
2021
  ssh-keygen -t rsa -f ~/.aspera/<%=cmd%>/aocapikey -N ''
1935
2022
  ```
1936
2023
 
@@ -1938,7 +2025,7 @@ ssh-keygen -t rsa -f ~/.aspera/<%=cmd%>/aocapikey -N ''
1938
2025
 
1939
2026
  (on some openssl implementation (mac) there is option: -nodes (no DES))
1940
2027
 
1941
- ```
2028
+ ```bash
1942
2029
  APIKEY=~/.aspera/<%=cmd%>/aocapikey
1943
2030
  openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
1944
2031
  openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
@@ -1946,63 +2033,81 @@ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
1946
2033
  rm -f ${APIKEY}.protected
1947
2034
  ```
1948
2035
 
1949
- #### API Client JWT activation
2036
+ ##### API Client JWT activation
1950
2037
 
1951
2038
  If you are not using the built-in client_id and secret, JWT needs to be authorized in Aspera on Cloud. This can be done in two manners:
1952
2039
 
1953
2040
  * Graphically
1954
2041
 
1955
- * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
1956
- * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
1957
- * Click on the previously created application
1958
- * select tab : "JSON Web Token Auth"
1959
- * Modify options if necessary, for instance: activate both options in section "Settings"
1960
- * Click "Save"
2042
+ * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
2043
+ * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
2044
+ * Click on the previously created application
2045
+ * select tab : "JSON Web Token Auth"
2046
+ * Modify options if necessary, for instance: activate both options in section "Settings"
2047
+ * Click "Save"
1961
2048
 
1962
2049
  * Using command line
1963
2050
 
1964
- ```
2051
+ ```bash
1965
2052
  <%=cmd%> aoc admin res client list
1966
- :............:.........:
1967
- : id : name :
1968
- :............:.........:
1969
- : BJLPObQiFw : <%=cmd%> :
1970
- :............:.........:
1971
- <%=cmd%> aoc admin res client modify --id=BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2053
+ ```
2054
+
2055
+ ```output
2056
+ :............:...............:
2057
+ : id : name :
2058
+ :............:...............:
2059
+ : BJLPObQiFw : my-client-app :
2060
+ :............:...............:
2061
+ ```
2062
+
2063
+ ```javascript
2064
+ <%=cmd%> aoc admin res client modify BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2065
+ ```
2066
+
2067
+ ```output
1972
2068
  modified
1973
2069
  ```
1974
2070
 
1975
- ### User key registration
2071
+ #### User key registration
1976
2072
 
1977
2073
  The public key must be assigned to your user. This can be done in two manners:
1978
2074
 
1979
- * Graphically
2075
+ ##### Graphically
1980
2076
 
1981
2077
  open the previously generated public key located here: `$HOME/.aspera/<%=cmd%>/aocapikey.pub`
1982
2078
 
1983
- * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
1984
- * Click on the user's icon (top right)
1985
- * Select "Account Settings"
1986
- * Paste the _Public Key_ in the "Public Key" section
1987
- * Click on "Submit"
2079
+ * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
2080
+ * Click on the user's icon (top right)
2081
+ * Select "Account Settings"
2082
+ * Paste the _Public Key_ in the "Public Key" section
2083
+ * Click on "Submit"
1988
2084
 
1989
- * Using command line
2085
+ ##### Using command line
1990
2086
 
1991
- ```
2087
+ ```bash
1992
2088
  <%=cmd%> aoc admin res user list
2089
+ ```
2090
+
2091
+ ```output
1993
2092
  :........:................:
1994
2093
  : id : name :
1995
2094
  :........:................:
1996
2095
  : 109952 : Tech Support :
1997
2096
  : 109951 : LAURENT MARTIN :
1998
2097
  :........:................:
1999
- <%=cmd%> aoc user info modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/<%=cmd%>/aocapikey.pub"))}'
2098
+ ```
2099
+
2100
+ ```ruby
2101
+ <%=cmd%> aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/<%=cmd%>/aocapikey.pub"))}'
2102
+ ```
2103
+
2104
+ ```output
2000
2105
  modified
2001
2106
  ```
2002
2107
 
2003
2108
  Note: the `aspera user info show` command can be used to verify modifications.
2004
2109
 
2005
- ### <%=prst%> modification for JWT
2110
+ #### <%=prst%> modification for JWT
2006
2111
 
2007
2112
  To activate default use of JWT authentication for <%=tool%> using the <%=prst%>, do the following:
2008
2113
 
@@ -2012,7 +2117,7 @@ To activate default use of JWT authentication for <%=tool%> using the <%=prst%>,
2012
2117
 
2013
2118
  Execute:
2014
2119
 
2015
- ```
2120
+ ```bash
2016
2121
  <%=cmd%> config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/<%=cmd%>/aocapikey --username=laurent.martin.aspera@fr.ibm.com
2017
2122
  ```
2018
2123
 
@@ -2020,30 +2125,31 @@ Note: the private key argument represents the actual PEM string. In order to rea
2020
2125
 
2021
2126
  After this last step, commands do not require web login anymore.
2022
2127
 
2023
-
2024
- ### <a id="aocfirst"></a>First Use
2128
+ #### <a id="aocfirst"></a>First Use
2025
2129
 
2026
2130
  Once client has been registered and <%=prst%> created: <%=tool%> can be used:
2027
2131
 
2028
- ```
2132
+ ```bash
2029
2133
  <%=cmd%> aoc files br /
2134
+ ```
2135
+
2136
+ ```output
2030
2137
  Current Workspace: Default Workspace (default)
2031
2138
  empty
2032
2139
  ```
2033
2140
 
2034
-
2035
- ## Administration
2141
+ ### Administration
2036
2142
 
2037
2143
  The `admin` command allows several administrative tasks (and require admin privilege).
2038
2144
 
2039
2145
  It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
2040
2146
 
2041
- ### Bulk creation and deletion of resource
2147
+ #### Bulk creation and deletion of resource
2042
2148
 
2043
2149
  Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
2044
2150
  In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2045
2151
 
2046
- ### Listing resources
2152
+ #### Listing resources
2047
2153
 
2048
2154
  The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
2049
2155
 
@@ -2055,7 +2161,7 @@ The following parameters are supported:
2055
2161
  * `sort`: name of fields to sort results, prefix with `-` for reverse order.
2056
2162
  * `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
2057
2163
  * `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
2058
- * `page` : native api parameter, in general do not use (added by
2164
+ * `page` : native api parameter, in general do not use (added by
2059
2165
  * `per_page` : native api parameter, number of items par api call, in general do not use
2060
2166
  * Other specific parameters depending on resource type.
2061
2167
 
@@ -2073,19 +2179,19 @@ Examples:
2073
2179
 
2074
2180
  * List users with `laurent` in name:
2075
2181
 
2076
- ```
2182
+ ```javascript
2077
2183
  <%=cmd%> aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
2078
2184
  ```
2079
2185
 
2080
2186
  * List users who logged-in before a date:
2081
2187
 
2082
- ```
2188
+ ```javascript
2083
2189
  <%=cmd%> aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
2084
2190
  ```
2085
2191
 
2086
2192
  * List external users and sort in reverse alphabetical order using name:
2087
2193
 
2088
- ```
2194
+ ```javascript
2089
2195
  <%=cmd%> aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
2090
2196
  ```
2091
2197
 
@@ -2093,34 +2199,34 @@ Refer to the AoC API for full list of query parameters, or use the browser in de
2093
2199
 
2094
2200
  Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
2095
2201
 
2096
- ### <a id="res_select"></a>Selecting a resource
2202
+ #### <a id="res_select"></a>Selecting a resource
2097
2203
 
2098
2204
  Resources are identified by a unique `id`, as well as a unique `name` (case insensitive).
2099
2205
 
2100
2206
  To execute an action on a specific resource, select it using one of those methods:
2101
2207
 
2102
- * *recommended:* give id directly on command line *after the action*: `aoc admin res node show 123`
2208
+ * **recommended:** give id directly on command line *after the action*: `aoc admin res node show 123`
2103
2209
  * give name on command line *after the action*: `aoc admin res node show name abc`
2104
2210
  * provide option `id` : `aoc admin res node show --id=123`
2105
2211
  * provide option `name` : `aoc admin res node show --name=abc`
2106
2212
 
2107
- ### Access Key secrets
2213
+ #### Access Key secrets
2108
2214
 
2109
2215
  In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
2110
2216
  It is usually provided using the `secret` option.
2111
2217
  For example in a command like:
2112
2218
 
2113
- ```
2219
+ ```bash
2114
2220
  <%=cmd%> aoc admin res node --id=123 --secret="secret1" v3 info
2115
2221
  ```
2116
2222
 
2117
2223
  It is also possible to provide a set of secrets used on a regular basis using the [secret vault](#vault).
2118
2224
 
2119
- ### Activity
2225
+ #### Activity
2120
2226
 
2121
2227
  The activity app can be queried with:
2122
2228
 
2123
- ```
2229
+ ```bash
2124
2230
  <%=cmd%> aoc admin analytics transfers
2125
2231
  ```
2126
2232
 
@@ -2128,7 +2234,7 @@ It can also support filters and send notification using option `notif_to`. a tem
2128
2234
 
2129
2235
  `mytemplate.erb`:
2130
2236
 
2131
- ```
2237
+ ```bash
2132
2238
  From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
2133
2239
  To: <<%='<'%>%=ev['user_email']%>>
2134
2240
  Subject: <%='<'%>%=ev['files_completed']%> files received
@@ -2139,13 +2245,14 @@ We received <%='<'%>%=ev['files_completed']%> files for a total of <%='<'%>%=ev[
2139
2245
 
2140
2246
  Thank you.
2141
2247
  ```
2248
+
2142
2249
  The environment provided contains the following additional variable:
2143
2250
 
2144
2251
  * ev : all details on the transfer event
2145
2252
 
2146
2253
  Example:
2147
2254
 
2148
- ```
2255
+ ```javascript
2149
2256
  <%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=12345 \
2150
2257
  --query=@json:'{"status":"completed","direction":"receive"}' \
2151
2258
  --notif-to=active --notif-template=@file:mytemplate.erb
@@ -2159,19 +2266,22 @@ Options:
2159
2266
 
2160
2267
  Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
2161
2268
 
2162
- ### Transfer: Using specific transfer ports
2269
+ #### Transfer: Using specific transfer ports
2163
2270
 
2164
2271
  By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
2165
2272
  The option `default_ports` ([yes]/no) allows <%=cmd%> to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
2166
2273
 
2167
- ### Using ATS
2274
+ #### Using ATS
2168
2275
 
2169
2276
  Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
2170
2277
 
2171
- ### Example: Bulk creation of users
2278
+ #### Example: Bulk creation of users
2172
2279
 
2173
- ```
2280
+ ```javascript
2174
2281
  <%=cmd%> aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
2282
+ ```
2283
+
2284
+ ```output
2175
2285
  :.......:.........:
2176
2286
  : id : status :
2177
2287
  :.......:.........:
@@ -2180,20 +2290,38 @@ Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `ao
2180
2290
  :.......:.........:
2181
2291
  ```
2182
2292
 
2183
- ### Example: Find with filter and delete
2293
+ #### Example: Find with filter and delete
2184
2294
 
2185
- ```
2295
+ ```javascript
2186
2296
  <%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
2297
+ ```
2298
+
2299
+ ```output
2187
2300
  :.......:........................:
2188
2301
  : id : email :
2189
2302
  :.......:........................:
2190
2303
  : 98398 : dummyuser1@example.com :
2191
2304
  : 98399 : dummyuser2@example.com :
2192
2305
  :.......:........................:
2306
+ ```
2307
+
2308
+ ```bash
2193
2309
  thelist=$(<%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
2310
+ ```
2311
+
2312
+ ```bash
2194
2313
  echo $thelist
2314
+ ```
2315
+
2316
+ ```javascript
2195
2317
  ["113501","354061"]
2318
+ ```
2319
+
2320
+ ```bash
2196
2321
  <%=cmd%> aoc admin res user --bulk=yes --id=@json:"$thelist" delete
2322
+ ```
2323
+
2324
+ ```output
2197
2325
  :.......:.........:
2198
2326
  : id : status :
2199
2327
  :.......:.........:
@@ -2202,18 +2330,21 @@ echo $thelist
2202
2330
  :.......:.........:
2203
2331
  ```
2204
2332
 
2205
- ### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
2333
+ #### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
2206
2334
 
2207
- ```
2335
+ ```ruby
2208
2336
  <%=cmd%> aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
2209
2337
  ```
2210
2338
 
2211
2339
  To delete them use the same method as before
2212
2340
 
2213
- ### Example: Display current user's workspaces
2341
+ #### Example: Display current user's workspaces
2214
2342
 
2343
+ ```bash
2344
+ <%=cmd%> aoc user workspaces list
2215
2345
  ```
2216
- <%=cmd%> aoc user workspaces
2346
+
2347
+ ```output
2217
2348
  :......:............................:
2218
2349
  : id : name :
2219
2350
  :......:............................:
@@ -2223,45 +2354,43 @@ To delete them use the same method as before
2223
2354
  :......:............................:
2224
2355
  ```
2225
2356
 
2226
- ### Example: Create a sub access key in a "node"
2357
+ #### Example: Create a sub access key in a "node"
2227
2358
 
2228
2359
  Creation of a sub-access key is like creation of access key with the following difference: authentication to node API is made with accesskey (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
2229
2360
 
2230
- ```
2361
+ ```bash
2231
2362
  <%=cmd%> aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create --value=@json:'{"storage":{"path":"/folder1"}}'
2232
2363
  ```
2233
2364
 
2234
- ### Example: Display transfer events (ops/transfer)
2365
+ #### Example: Display transfer events (ops/transfer)
2235
2366
 
2236
- ```
2367
+ ```bash
2237
2368
  <%=cmd%> aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
2238
2369
  ```
2239
2370
 
2240
2371
  Examples of query (TODO: cleanup):
2241
2372
 
2242
- ```
2373
+ ```javascript
2243
2374
  {"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
2375
+ ```
2244
2376
 
2377
+ ```javascript
2245
2378
  {"tag":"aspera.files.package_id=LA8OU3p8w"}
2246
-
2247
- # filter= 'id', 'short_summary', or 'summary'
2248
- # count=nnn
2249
- # tag=x.y.z%3Dvalue
2250
- # iteration_token=nnn
2251
- # after_time=2016-05-01T23:53:09Z
2252
- # active_only=true|false
2253
2379
  ```
2254
2380
 
2255
- ### Example: Display node events (events)
2381
+ #### Example: Display node events (events)
2256
2382
 
2257
- ```
2383
+ ```bash
2258
2384
  <%=cmd%> aoc admin res node --secret=_secret_ v3 events
2259
2385
  ```
2260
2386
 
2261
- ### Example: Display members of a workspace
2387
+ #### Example: Display members of a workspace
2262
2388
 
2263
- ```
2389
+ ```javascript
2264
2390
  <%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
2391
+ ```
2392
+
2393
+ ```output
2265
2394
  :.............:.........:..................................:
2266
2395
  : member_type : manager : member.email :
2267
2396
  :.............:.........:..................................:
@@ -2276,54 +2405,57 @@ Examples of query (TODO: cleanup):
2276
2405
 
2277
2406
  other query parameters:
2278
2407
 
2279
- ```
2408
+ ```javascript
2280
2409
  {"workspace_membership_through":true,"include_indirect":true}
2281
2410
  ```
2282
2411
 
2283
- ### Example: <a id="aoc_sample_member"></a>add all members of a workspace to another workspace
2412
+ #### Example: <a id="aoc_sample_member"></a>add all members of a workspace to another workspace
2284
2413
 
2285
2414
  a- Get id of first workspace
2286
2415
 
2287
- ```
2416
+ ```bash
2288
2417
  WS1='First Workspace'
2289
2418
  WS1ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
2290
2419
  ```
2291
2420
 
2292
2421
  b- Get id of second workspace
2293
2422
 
2294
- ```
2423
+ ```bash
2295
2424
  WS2='Second Workspace'
2296
2425
  WS2ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
2297
2426
  ```
2298
2427
 
2299
2428
  c- Extract membership information
2300
2429
 
2301
- ```
2430
+ ```bash
2302
2431
  <%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
2303
2432
  ```
2304
2433
 
2305
2434
  d- Convert to creation data for second workspace:
2306
2435
 
2307
- ```
2436
+ ```bash
2308
2437
  grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
2309
2438
  ```
2310
2439
 
2311
2440
  or, using jq:
2312
2441
 
2313
- ```
2442
+ ```bash
2314
2443
  jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
2315
2444
  ```
2316
2445
 
2317
2446
  e- Add members to second workspace
2318
2447
 
2319
- ```
2448
+ ```bash
2320
2449
  <%=cmd%> aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
2321
2450
  ```
2322
2451
 
2323
- ### Example: Get users who did not log since a date
2452
+ #### Example: Get users who did not log since a date
2324
2453
 
2325
- ```
2454
+ ```javascript
2326
2455
  <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
2456
+ ```
2457
+
2458
+ ```output
2327
2459
  :...............................:
2328
2460
  : email :
2329
2461
  :...............................:
@@ -2332,20 +2464,23 @@ e- Add members to second workspace
2332
2464
  :...............................:
2333
2465
  ```
2334
2466
 
2335
- ### Example: List "Limited" users
2467
+ #### Example: List "Limited" users
2336
2468
 
2337
- ```
2469
+ ```javascript
2338
2470
  <%=cmd%> aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
2339
2471
  ```
2340
2472
 
2341
- ### Example: Perform a multi Gbps transfer between two remote shared folders
2473
+ #### Example: Perform a multi Gbps transfer between two remote shared folders
2342
2474
 
2343
2475
  In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
2344
2476
 
2345
2477
  First, setup the environment (skip if already done)
2346
2478
 
2347
- ```
2479
+ ```bash
2348
2480
  <%=cmd%> conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
2481
+ ```
2482
+
2483
+ ```output
2349
2484
  Detected: Aspera on Cloud
2350
2485
  Preparing preset: aoc_sedemo
2351
2486
  Using existing key:
@@ -2364,7 +2499,7 @@ Setting config preset as default for aspera
2364
2499
  saving config file
2365
2500
  Done.
2366
2501
  You can test with:
2367
- <%=cmd%> aoc user info show
2502
+ <%=cmd%> aoc user profile show
2368
2503
  ```
2369
2504
 
2370
2505
  This creates the option preset "aoc_&lt;org name&gt;" to allow seamless command line access and sets it as default for aspera on cloud.
@@ -2373,21 +2508,27 @@ Then, create two shared folders located in two regions, in your files home, in a
2373
2508
 
2374
2509
  Then, transfer between those:
2375
2510
 
2376
- ```
2511
+ ```javascript
2377
2512
  <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
2378
2513
  ```
2379
2514
 
2380
- ### Example: create registration key to register a node
2515
+ #### Example: create registration key to register a node
2381
2516
 
2382
- ```
2517
+ ```javascript
2383
2518
  <%=cmd%> aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
2519
+ ```
2520
+
2521
+ ```output
2384
2522
  jfqslfdjlfdjfhdjklqfhdkl
2385
2523
  ```
2386
2524
 
2387
- ### Example: delete all registration keys
2525
+ #### Example: delete all registration keys
2388
2526
 
2389
- ```
2527
+ ```bash
2390
2528
  <%=cmd%> aoc admin res client list --fields=id --format=csv|<%=cmd%> aoc admin res client delete --bulk=yes --id=@lines:@stdin:
2529
+ ```
2530
+
2531
+ ```output
2391
2532
  +-----+---------+
2392
2533
  | id | status |
2393
2534
  +-----+---------+
@@ -2398,7 +2539,7 @@ jfqslfdjlfdjfhdjklqfhdkl
2398
2539
  +-----+---------+
2399
2540
  ```
2400
2541
 
2401
- ### Example: Create a node
2542
+ #### Example: Create a node
2402
2543
 
2403
2544
  AoC nodes as actually composed with two related entities:
2404
2545
 
@@ -2412,7 +2553,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
2412
2553
 
2413
2554
  * create the access key on ATS
2414
2555
 
2415
- ```
2556
+ ```javascript
2416
2557
  <%=cmd%> aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
2417
2558
  ```
2418
2559
 
@@ -2420,79 +2561,90 @@ Take a note of the randomly generated `id` and `secret`.
2420
2561
 
2421
2562
  * Retrieve the ATS node address
2422
2563
 
2423
- ```
2424
- <%=cmd%> aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv|cut -f2 -d,
2564
+ ```bash
2565
+ <%=cmd%> aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
2425
2566
  ```
2426
2567
 
2427
2568
  * Create the node entity
2428
2569
 
2429
- ```
2570
+ ```javascript
2430
2571
  <%=cmd%> aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
2431
2572
  ```
2432
2573
 
2433
2574
  Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
2434
2575
 
2435
- ### Example: List packages in a given shared inbox
2436
-
2437
- First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
2438
- (To find out available filters, consult the API definition, or use the web interface in developer mode).
2439
-
2440
- Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
2441
-
2442
- ```
2443
- shbxid=$(<%=cmd%> aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
2444
-
2445
- <%=cmd%> aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
2446
- ```
2447
-
2448
- ## Packages
2576
+ ### Packages
2449
2577
 
2450
2578
  The webmail-like application.
2451
2579
 
2452
- ### Send a Package
2580
+ #### Send a Package
2453
2581
 
2454
- Send a package:
2582
+ General syntax:
2455
2583
 
2456
- ```
2584
+ ```bash
2457
2585
  <%=cmd%> aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
2458
2586
  ```
2459
2587
 
2460
2588
  Notes:
2461
2589
 
2462
2590
  * The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
2463
- * List allowed shared inbox destinations with: `<%=cmd%> aoc user shared_inboxes`
2464
- * Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
2591
+ * List allowed shared inbox destinations with: `<%=cmd%> aoc packages shared_inboxes list`
2592
+ * Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
2465
2593
  * Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
2466
2594
  * or just names: `"recipients":[{"The Dest"}]` . <%=cmd%> will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
2467
2595
  * If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
2468
2596
  * if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
2469
2597
  * if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
2470
2598
 
2471
- Examples:
2599
+ #### Example: Send a package with one file to two users, using their email
2472
2600
 
2473
- * Send a package with one file to two users, using their email
2601
+ ```javascript
2602
+ <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
2603
+ ```
2604
+
2605
+ #### Example: Send a package to a shared inbox with metadata
2474
2606
 
2607
+ ```javascript
2608
+ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
2475
2609
  ```
2476
- <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
2610
+
2611
+ It is also possible to use identifiers and API parameters:
2612
+
2613
+ ```javascript
2614
+ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
2477
2615
  ```
2478
2616
 
2479
- * Send a package with one file to a shared inbox, using internal identifier, with specific transfer parameters
2617
+ #### Example: List packages in a given shared inbox
2618
+
2619
+ When user packages are listed, the following query is used:
2480
2620
 
2621
+ ```javascript
2622
+ {"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
2481
2623
  ```
2482
- <%=cmd%> aoc package send --value=@json:'{"name":"my delivery","recipients":[{"type":"dropbox","id":"12345"}]}' --ts=@json:'{"target_rate_kbps":100000}' my_file.dat
2624
+
2625
+ To list packages in a shared inbox, the query has to be specified with withe the shared inbox by name or its identifier. Additionnal parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode). The current workspace is added unless specified in the query.
2626
+
2627
+ Using shared inbox name:
2628
+
2629
+ ```javascript
2630
+ <%=cmd%> aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
2483
2631
  ```
2484
2632
 
2485
- * Send a package with one file to a shared inbox (by name) with metadata
2633
+ Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
2486
2634
 
2635
+ ```bash
2636
+ shbxid=$(<%=cmd%> aoc packages shared_inboxes show name 'My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
2487
2637
  ```
2488
- <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
2638
+
2639
+ ```javascript
2640
+ <%=cmd%> aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
2489
2641
  ```
2490
2642
 
2491
- ### <a id="aoccargo"></a>Receive new packages only (Cargo)
2643
+ #### <a id="aoccargo"></a>Receive new packages only (Cargo)
2492
2644
 
2493
2645
  It is possible to automatically download new packages, like using Aspera Cargo:
2494
2646
 
2495
- ```
2647
+ ```bash
2496
2648
  <%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
2497
2649
  ```
2498
2650
 
@@ -2506,38 +2658,38 @@ Typically, one would execute this command on a regular basis, using the method o
2506
2658
  * Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
2507
2659
  * etc...
2508
2660
 
2509
- ## Files
2661
+ ### Files
2510
2662
 
2511
2663
  Folder sharing app.
2512
2664
 
2513
- ### Download Files
2665
+ #### Download Files
2514
2666
 
2515
2667
  Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
2516
2668
 
2517
2669
  * if only one source is provided, it is downloaded
2518
2670
  * if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
2519
2671
 
2520
- ### Shared folders
2672
+ #### Shared folders
2521
2673
 
2522
2674
  * list shared folders in node
2523
2675
 
2524
- ```
2676
+ ```bash
2525
2677
  <%=cmd%> aoc admin res node --id=8669 shared_folders
2526
2678
  ```
2527
2679
 
2528
2680
  * list shared folders in workspace
2529
2681
 
2530
- ```
2682
+ ```bash
2531
2683
  <%=cmd%> aoc admin res workspace --id=10818 shared_folders
2532
2684
  ```
2533
2685
 
2534
2686
  * list members of shared folder
2535
2687
 
2536
- ```
2688
+ ```bash
2537
2689
  <%=cmd%> aoc admin res node --id=8669 v4 perm 82 show
2538
2690
  ```
2539
2691
 
2540
- ### Cross Organization transfers
2692
+ #### Cross Organization transfers
2541
2693
 
2542
2694
  It is possible to transfer files directly between organizations without having to first download locally and then upload...
2543
2695
 
@@ -2551,7 +2703,7 @@ Procedure to send a file from org1 to org2:
2551
2703
  * Check that access works and locate the destination folder `mydestfolder`
2552
2704
  * execute the following:
2553
2705
 
2554
- ```
2706
+ ```bash
2555
2707
  <%=cmd%> -Porg1 aoc files node_info /mydestfolder --format=json --display=data | <%=cmd%> -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
2556
2708
  ```
2557
2709
 
@@ -2567,7 +2719,7 @@ Explanation:
2567
2719
  * `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
2568
2720
  * `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
2569
2721
 
2570
- ### Find Files
2722
+ #### Find Files
2571
2723
 
2572
2724
  The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
2573
2725
 
@@ -2584,29 +2736,29 @@ Examples of expressions: (using like this: `--value=exec:'<expression>'`)
2584
2736
 
2585
2737
  * Find files more recent than 100 days
2586
2738
 
2587
- ```
2739
+ ```bash
2588
2740
  f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
2589
2741
  ```
2590
2742
 
2591
2743
  * Find files older than 1 year on a given node and store in file list
2592
2744
 
2593
- ```
2745
+ ```bash
2594
2746
  <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
2595
2747
  ```
2596
2748
 
2597
2749
  * Delete the files, one by one
2598
2750
 
2599
- ```
2751
+ ```bash
2600
2752
  cat my_file_list.txt|while read path;do echo <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
2601
2753
  ```
2602
2754
 
2603
2755
  * Delete the files in bulk
2604
2756
 
2605
- ```
2757
+ ```bash
2606
2758
  cat my_file_list.txt | <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
2607
2759
  ```
2608
2760
 
2609
- # <a id="ats"></a>Plugin: Aspera Transfer Service
2761
+ ## <a id="ats"></a>Plugin: Aspera Transfer Service
2610
2762
 
2611
2763
  ATS is usable either :
2612
2764
 
@@ -2614,15 +2766,18 @@ ATS is usable either :
2614
2766
 
2615
2767
  * or from an IBM Cloud subscription : <%=cmd%> ats : use IBM Cloud API key authentication
2616
2768
 
2617
- ## IBM Cloud ATS : creation of api key
2769
+ ### IBM Cloud ATS : creation of api key
2618
2770
 
2619
2771
  This section is about using ATS with an IBM cloud subscription.
2620
2772
  If you are using ATS as part of AoC, then authentication is thropugh AoC, not IBM Cloud.
2621
2773
 
2622
2774
  First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
2623
2775
 
2624
- ```
2776
+ ```bash
2625
2777
  ibmcloud iam api-key-create mykeyname -d 'my sample key'
2778
+ ```
2779
+
2780
+ ```output
2626
2781
  OK
2627
2782
  API key mykeyname was created
2628
2783
 
@@ -2638,22 +2793,40 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
2638
2793
 
2639
2794
  References:
2640
2795
 
2641
- * [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
2642
- * [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
2796
+ * [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
2797
+ * [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
2643
2798
 
2644
2799
  Then, to register the key by default for the ats plugin, create a preset. Execute:
2645
2800
 
2646
- ```
2801
+ ```bash
2647
2802
  <%=cmd%> config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here_8f8d9fdakjhfsashjk678
2803
+ ```
2804
+
2805
+ ```bash
2648
2806
  <%=cmd%> config preset set default ats my_ibm_ats
2807
+ ```
2808
+
2809
+ ```bash
2649
2810
  <%=cmd%> ats api_key instances
2811
+ ```
2812
+
2813
+ ```output
2650
2814
  +--------------------------------------+
2651
2815
  | instance |
2652
2816
  +--------------------------------------+
2653
2817
  | aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee |
2654
2818
  +--------------------------------------+
2819
+ ```
2820
+
2821
+ ```bash
2655
2822
  <%=cmd%> config preset update my_ibm_ats --instance=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
2823
+ ```
2824
+
2825
+ ```bash
2656
2826
  <%=cmd%> ats api_key create
2827
+ ```
2828
+
2829
+ ```output
2657
2830
  +--------+----------------------------------------------+
2658
2831
  | key | value |
2659
2832
  +--------+----------------------------------------------+
@@ -2663,52 +2836,49 @@ Then, to register the key by default for the ats plugin, create a preset. Execut
2663
2836
  <%=cmd%> config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
2664
2837
  ```
2665
2838
 
2666
- ## Examples
2839
+ ### Misc. Examples
2667
2840
 
2668
2841
  Example: create access key on IBM Cloud (softlayer):
2669
2842
 
2670
- ```
2843
+ ```javascript
2671
2844
  <%=cmd%> ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
2672
2845
  ```
2673
2846
 
2674
2847
  Example: create access key on AWS:
2675
2848
 
2676
- ```
2849
+ ```javascript
2677
2850
  <%=cmd%> ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
2678
-
2679
2851
  ```
2680
2852
 
2681
2853
  Example: create access key on Azure SAS:
2682
2854
 
2683
- ```
2855
+ ```javascript
2684
2856
  <%=cmd%> ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
2685
-
2686
2857
  ```
2687
2858
 
2688
2859
  (Note that the blob name is mandatory after server address and before parameters. and that parameter sr=c is mandatory.)
2689
2860
 
2690
2861
  Example: create access key on Azure:
2691
2862
 
2692
- ```
2863
+ ```javascript
2693
2864
  <%=cmd%> ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'
2694
-
2695
2865
  ```
2696
2866
 
2697
2867
  delete all my access keys:
2698
2868
 
2699
- ```
2869
+ ```bash
2700
2870
  for k in $(<%=cmd%> ats access_key list --field=id --format=csv);do <%=cmd%> ats access_key id $k delete;done
2701
2871
  ```
2702
2872
 
2703
2873
  The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
2704
2874
 
2705
- # Plugin: IBM Aspera High Speed Transfer Server (transfer)
2875
+ ## Plugin: IBM Aspera High Speed Transfer Server (transfer)
2706
2876
 
2707
2877
  This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
2708
2878
  It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
2709
2879
  Modern mode is to use the node API and transfer tokens.
2710
2880
 
2711
- ## Authentication
2881
+ ### Authentication
2712
2882
 
2713
2883
  Both password and SSH keys auth are supported.
2714
2884
 
@@ -2716,7 +2886,7 @@ If username is not provided, the default transfer user `xfer` is used.
2716
2886
 
2717
2887
  If no SSH password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used:
2718
2888
 
2719
- ```
2889
+ ```javascript
2720
2890
  <%=cmd%> server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
2721
2891
  ```
2722
2892
 
@@ -2726,7 +2896,7 @@ Each value is a path to a private key and is expanded (`~` is replaced with the
2726
2896
 
2727
2897
  Examples:
2728
2898
 
2729
- ```
2899
+ ```bash
2730
2900
  <%=cmd%> server --ssh-keys=~/.ssh/id_rsa
2731
2901
  <%=cmd%> server --ssh-keys=@list:,~/.ssh/id_rsa
2732
2902
  <%=cmd%> server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
@@ -2737,13 +2907,13 @@ By default the ssh library expect that an ssh-agent is running.
2737
2907
 
2738
2908
  On Linux, if you get an error message such as:
2739
2909
 
2740
- ```
2910
+ ```bash
2741
2911
  ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: Agent not configured
2742
2912
  ```
2743
2913
 
2744
2914
  or on Windows:
2745
2915
 
2746
- ```
2916
+ ```bash
2747
2917
  ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
2748
2918
  ```
2749
2919
 
@@ -2754,17 +2924,17 @@ This means that you don't have such an ssh agent running, then:
2754
2924
  * [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
2755
2925
  * To disable use of `ssh-agent`, use the option `ssh_option` like this:
2756
2926
 
2757
- ```
2927
+ ```bash
2758
2928
  <%=cmd%> server --ssh-options=@ruby:'{use_agent: false}' ...
2759
2929
  ```
2760
2930
 
2761
2931
  This can also be set as default using a global preset.
2762
2932
 
2763
- ## Example
2933
+ ### Example
2764
2934
 
2765
2935
  One can test the `server` application using the well known demo server:
2766
2936
 
2767
- ```
2937
+ ```bash
2768
2938
  <%=cmd%> config initdemo
2769
2939
  <%=cmd%> server browse /aspera-test-dir-large
2770
2940
  <%=cmd%> server download /aspera-test-dir-large/200MB
@@ -2772,13 +2942,14 @@ One can test the `server` application using the well known demo server:
2772
2942
 
2773
2943
  `initdemo` creates a <%=prst%> `demoserver` and set it as default for plugin `server`.
2774
2944
 
2775
- # Plugin: IBM Aspera High Speed Transfer Server (node)
2945
+ ## Plugin: IBM Aspera High Speed Transfer Server (node)
2776
2946
 
2777
2947
  This plugin gives access to capabilities provided by HSTS node API.
2778
2948
 
2779
- ## File Operations
2949
+ ### File Operations
2780
2950
 
2781
2951
  It is possible to:
2952
+
2782
2953
  * browse
2783
2954
  * transfer (upload / download)
2784
2955
  * ...
@@ -2788,7 +2959,7 @@ For transfers, it is possible to control how transfer is authorized using option
2788
2959
  * `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
2789
2960
  * `basic` : transfer spec is created like this:
2790
2961
 
2791
- ```
2962
+ ```javascript
2792
2963
  {
2793
2964
  "remote_host": address of node url,
2794
2965
  "remote_user": "xfer",
@@ -2800,29 +2971,29 @@ For transfers, it is possible to control how transfer is authorized using option
2800
2971
 
2801
2972
  * `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
2802
2973
 
2803
- ## Central
2974
+ ### Central
2804
2975
 
2805
2976
  The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transferred files.
2806
2977
 
2807
2978
  Filtering can be applied:
2808
2979
 
2809
- ```
2980
+ ```bash
2810
2981
  <%=cmd%> node central file list
2811
2982
  ```
2812
2983
 
2813
2984
  by providing the `validator` option, offline transfer validation can be done.
2814
2985
 
2815
- ## FASP Stream
2986
+ ### FASP Stream
2816
2987
 
2817
2988
  It is possible to start a FASPStream session using the node API:
2818
2989
 
2819
2990
  Use the "node stream create" command, then arguments are provided as a [_transfer-spec_](#transferspec).
2820
2991
 
2821
- ```
2992
+ ```javascript
2822
2993
  <%=cmd%> node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
2823
2994
  ```
2824
2995
 
2825
- ## Watchfolder
2996
+ ### Watchfolder
2826
2997
 
2827
2998
  Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
2828
2999
 
@@ -2831,52 +3002,58 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
2831
3002
  * Start watchd and watchfolderd services running as a system user having access to files
2832
3003
  * configure a watchfolder to define automated transfers
2833
3004
 
2834
- ```
3005
+ ```javascript
2835
3006
  <%=cmd%> node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
2836
3007
  <%=cmd%> node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
2837
3008
  <%=cmd%> node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
2838
3009
  ```
2839
3010
 
2840
- ## Out of Transfer File Validation
3011
+ ### Out of Transfer File Validation
2841
3012
 
2842
3013
  Follow the Aspera Transfer Server configuration to activate this feature.
2843
3014
 
2844
- ```
3015
+ ```javascript
2845
3016
  <%=cmd%> node central file list --validator=<%=cmd%> --data=@json:'{"file_transfer_filter":{"max_result":1}}'
3017
+ ```
3018
+
3019
+ ```output
2846
3020
  :..............:..............:............:......................................:
2847
3021
  : session_uuid : file_id : status : path :
2848
3022
  :..............:..............:............:......................................:
2849
3023
  : 1a74444c-... : 084fb181-... : validating : /home/xfer.../PKG - my title/200KB.1 :
2850
3024
  :..............:..............:............:......................................:
3025
+ ```
3026
+
3027
+ ```javascript
2851
3028
  <%=cmd%> node central file update --validator=<%=cmd%> --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
3029
+ ```
3030
+
3031
+ ```output
2852
3032
  updated
2853
3033
  ```
2854
3034
 
2855
- ## Example: SHOD to ATS
3035
+ ### Example: SHOD to ATS
2856
3036
 
2857
- Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner. And we need to
2858
- transfer files from this third party SHOD instance into our Azure BLOB storage.
2859
- Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which
2860
- provides access to the node API.
2861
- Then create a configuration for the "SHOD" instance in the configuration file: in section
2862
- "shares", a configuration named: awsshod.
3037
+ Scenario: Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
3038
+ We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
3039
+ Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which provides access to the node API.
3040
+ Then create a configuration for the "SHOD" instance in the configuration file: in section "shares", a configuration named: awsshod.
2863
3041
  Create another configuration for the Azure ATS instance: in section "node", named azureats.
2864
3042
  Then execute the following command:
2865
3043
 
2866
- ```
3044
+ ```bash
2867
3045
  <%=cmd%> node download /share/sourcefile --to-folder=/destinationfolder --preset=awsshod --transfer=node --transfer-info=@preset:azureats
2868
3046
  ```
2869
3047
 
2870
- This will get transfer information from the SHOD instance and tell the Azure ATS instance
2871
- to download files.
3048
+ This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
2872
3049
 
2873
- ## Create access key
3050
+ ### Create access key
2874
3051
 
2875
- ```
3052
+ ```javascript
2876
3053
  <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
2877
3054
  ```
2878
3055
 
2879
- # Plugin: IBM Aspera Faspex5
3056
+ ## Plugin: IBM Aspera Faspex5
2880
3057
 
2881
3058
  3 authentication methods are supported:
2882
3059
 
@@ -2897,35 +3074,35 @@ For boot method: (will be removed in future)
2897
3074
 
2898
3075
  Use it as password and use `--auth=boot`.
2899
3076
 
2900
- ```
3077
+ ```bash
2901
3078
  <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
2902
3079
  ```
2903
3080
 
2904
3081
  Ready to use Faspex5 with CLI.
2905
3082
 
2906
- # Plugin: IBM Aspera Faspex (4.x)
3083
+ ## Plugin: IBM Aspera Faspex (4.x)
2907
3084
 
2908
3085
  Notes:
2909
3086
 
2910
3087
  * The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
2911
3088
  * For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
2912
3089
 
2913
- ## Listing Packages
3090
+ ### Listing Packages
2914
3091
 
2915
3092
  Command: `faspex package list`
2916
3093
 
2917
- ### Option `box`
3094
+ #### Option `box`
2918
3095
 
2919
3096
  By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
2920
3097
 
2921
- ### Option `recipient`
3098
+ #### Option `recipient`
2922
3099
 
2923
3100
  A user can receive a package because the recipient is:
2924
3101
 
2925
3102
  * the user himself (default)
2926
3103
  * the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
2927
3104
 
2928
- ### Option `query`
3105
+ #### Option `query`
2929
3106
 
2930
3107
  As inboxes may be large, it is possible to use the following query parameters:
2931
3108
 
@@ -2941,15 +3118,15 @@ The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/cat
2941
3118
 
2942
3119
  If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`). By default page is `0` (`10`), it can be increased to have less calls.
2943
3120
 
2944
- ### Example
3121
+ #### Example: list packages in dropbox
2945
3122
 
2946
- ```
3123
+ ```javascript
2947
3124
  <%=cmd%> faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
2948
3125
  ```
2949
3126
 
2950
3127
  List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
2951
3128
 
2952
- ## Receiving a Package
3129
+ ### Receiving a Package
2953
3130
 
2954
3131
  The command is `package recv`, possible methods are:
2955
3132
 
@@ -2957,26 +3134,26 @@ The command is `package recv`, possible methods are:
2957
3134
  * provide a public link with option `link`
2958
3135
  * provide a `faspe:` URI with option `link`
2959
3136
 
2960
- ```
3137
+ ```bash
2961
3138
  <%=cmd%> faspex package recv --id=12345
2962
3139
  <%=cmd%> faspex package recv --link=faspe://...
2963
3140
  ```
2964
3141
 
2965
3142
  If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
2966
3143
 
2967
- ```
3144
+ ```bash
2968
3145
  <%=cmd%> faspex package list --recipient='*thedropboxname'
2969
3146
  ```
2970
3147
 
2971
3148
  if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
2972
3149
 
2973
- ## Sending a Package
3150
+ ### Sending a Package
2974
3151
 
2975
3152
  The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
2976
3153
 
2977
3154
  Example:
2978
3155
 
2979
- ```
3156
+ ```javascript
2980
3157
  <%=cmd%> faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
2981
3158
  ```
2982
3159
 
@@ -2987,29 +3164,29 @@ Additional optional parameters in `delivery_info`:
2987
3164
  * Package Note: : `"note":"note this and that"`
2988
3165
  * Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
2989
3166
 
2990
- ## Email notification on transfer
3167
+ ### Email notification on transfer
2991
3168
 
2992
3169
  Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
2993
3170
 
2994
3171
  Example:
2995
3172
 
2996
- ```
3173
+ ```javascript
2997
3174
  <%=cmd%> faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>\nTo: <<%='<'%>%=to%>>\nSubject: Package sent: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
2998
3175
  ```
2999
3176
 
3000
3177
  In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
3001
3178
 
3002
- ## Operation on dropboxes
3179
+ ### Operation on dropboxes
3003
3180
 
3004
3181
  Example:
3005
3182
 
3006
- ```
3183
+ ```javascript
3007
3184
  <%=cmd%> faspex v4 dropbox create --value=@json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
3008
3185
  <%=cmd%> faspex v4 dropbox list
3009
3186
  <%=cmd%> faspex v4 dropbox delete --id=36
3010
3187
  ```
3011
3188
 
3012
- ## Remote sources
3189
+ ### Remote sources
3013
3190
 
3014
3191
  Faspex lacks an API to list the contents of a remote source (available in web UI). To workaround this,
3015
3192
  the node API is used, for this it is required to add a section ":storage" that links
@@ -3038,26 +3215,26 @@ The node configuration name is "my_faspex_node" here.
3038
3215
 
3039
3216
  Note: the v4 API provide an API for nodes and shares.
3040
3217
 
3041
- ## Automated package download (cargo)
3218
+ ### Automated package download (cargo)
3042
3219
 
3043
3220
  It is possible to tell <%=tool%> to download newly received packages, much like the official
3044
3221
  cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on Cloud plugin:
3045
3222
 
3046
- ```
3223
+ ```bash
3047
3224
  <%=cmd%> faspex packages recv --id=ALL --once-only=yes --lock-port=12345
3048
3225
  ```
3049
3226
 
3050
- # Plugin: IBM Aspera Shares
3227
+ ## Plugin: IBM Aspera Shares
3051
3228
 
3052
3229
  Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
3053
3230
 
3054
3231
  In Shares2, users, groups listing are paged, to display sequential pages:
3055
3232
 
3056
- ```
3233
+ ```bash
3057
3234
  for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p'}';done
3058
3235
  ```
3059
3236
 
3060
- # Plugin: IBM Cloud Object Storage
3237
+ ## Plugin: IBM Cloud Object Storage
3061
3238
 
3062
3239
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
3063
3240
  It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
@@ -3065,7 +3242,7 @@ Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
3065
3242
 
3066
3243
  There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
3067
3244
 
3068
- ## Using endpoint, apikey and Resource Instance ID (CRN)
3245
+ ### Using endpoint, apikey and Resource Instance ID (CRN)
3069
3246
 
3070
3247
  If you have those parameters already, then following options shall be provided:
3071
3248
 
@@ -3076,14 +3253,14 @@ If you have those parameters already, then following options shall be provided:
3076
3253
 
3077
3254
  For example, let us create a default configuration:
3078
3255
 
3079
- ```
3256
+ ```bash
3080
3257
  <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
3081
3258
  <%=cmd%> conf id default set cos mycos
3082
3259
  ```
3083
3260
 
3084
3261
  Then, jump to the transfer example.
3085
3262
 
3086
- ## Using service credential file
3263
+ ### Using service credential file
3087
3264
 
3088
3265
  If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
3089
3266
 
@@ -3093,7 +3270,7 @@ Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
3093
3270
 
3094
3271
  or using the IBM Cloud CLI:
3095
3272
 
3096
- ```
3273
+ ```bash
3097
3274
  ibmcloud resource service-keys
3098
3275
  ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HOME/service_creds.json
3099
3276
  ```
@@ -3102,7 +3279,7 @@ ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HO
3102
3279
 
3103
3280
  It consists in the following structure:
3104
3281
 
3105
- ```
3282
+ ```javascript
3106
3283
  {
3107
3284
  "apikey": "xxxxxxx.....",
3108
3285
  "cos_hmac_keys": {
@@ -3132,40 +3309,40 @@ The required options for this method are:
3132
3309
 
3133
3310
  For example, let us create a default configuration:
3134
3311
 
3135
- ```
3312
+ ```bash
3136
3313
  <%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
3137
3314
  <%=cmd%> conf id default set cos mycos
3138
3315
  ```
3139
3316
 
3140
- ## Operations, transfers
3317
+ ### Operations, transfers
3141
3318
 
3142
3319
  Let's assume you created a default configuration from once of the two previous steps (else specify the access options on command lines).
3143
3320
 
3144
3321
  A subset of `node` plugin operations are supported, basically node API:
3145
3322
 
3146
- ```
3323
+ ```bash
3147
3324
  <%=cmd%> cos node info
3148
3325
  <%=cmd%> cos node upload 'faux:///sample1G?1g'
3149
3326
  ```
3150
3327
 
3151
3328
  Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
3152
3329
 
3153
- # Plugin: IBM Aspera Sync
3330
+ ## Plugin: IBM Aspera Sync
3154
3331
 
3155
3332
  A basic plugin to start an "async" using <%=tool%>.
3156
3333
  The main advantage is the possibility to start from ma configuration file, using <%=tool%> standard options.
3157
3334
 
3158
- # Plugin: Preview
3335
+ ## Plugin: Preview
3159
3336
 
3160
3337
  The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
3161
3338
  This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
3162
3339
  Several parameters can be used to tune several aspects:
3163
3340
 
3164
- * methods for detection of new files needing generation
3165
- * methods for generation of video preview
3166
- * parameters for video handling
3341
+ * methods for detection of new files needing generation
3342
+ * methods for generation of video preview
3343
+ * parameters for video handling
3167
3344
 
3168
- ## Aspera Server configuration
3345
+ ### Aspera Server configuration
3169
3346
 
3170
3347
  Specify the previews folder as shown in:
3171
3348
 
@@ -3173,7 +3350,7 @@ Specify the previews folder as shown in:
3173
3350
 
3174
3351
  By default, the `preview` plugin expects previews to be generated in a folder named `previews` located in the storage root. On the transfer server execute:
3175
3352
 
3176
- ```
3353
+ ```bash
3177
3354
  PATH=/opt/aspera/bin:$PATH
3178
3355
 
3179
3356
  asconfigurator -x "server;preview_dir,previews"
@@ -3189,7 +3366,7 @@ This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KBytes.
3189
3366
 
3190
3367
  To change this parameter in `aspera.conf`, use `asconfigurator`. To display the value, use `asuserdata`:
3191
3368
 
3192
- ```
3369
+ ```bash
3193
3370
  asuserdata -a | grep max_request_file_create_size_kb
3194
3371
 
3195
3372
  max_request_file_create_size_kb: "1024"
@@ -3201,7 +3378,7 @@ If you use a value different than 16777216, then specify it using option `max_si
3201
3378
 
3202
3379
  Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* while the generator parameter is in *Bytes* (factor of 1024).
3203
3380
 
3204
- ## <a id="prev_ext"></a>External tools: Linux
3381
+ ### <a id="prev_ext"></a>External tools: Linux
3205
3382
 
3206
3383
  The tool requires the following external tools available in the `PATH`:
3207
3384
 
@@ -3217,21 +3394,21 @@ Other OSes should work as well, but are note tested.
3217
3394
 
3218
3395
  To check if all tools are found properly, execute:
3219
3396
 
3220
- ```
3397
+ ```bash
3221
3398
  <%=cmd%> preview check
3222
3399
  ```
3223
3400
 
3224
- ### mimemagic
3401
+ #### mimemagic
3225
3402
 
3226
3403
  To benefit from extra mime type detection install gem mimemagic:
3227
3404
 
3228
- ```
3405
+ ```bash
3229
3406
  gem install mimemagic
3230
3407
  ```
3231
3408
 
3232
3409
  or to install an earlier version if any problem:
3233
3410
 
3234
- ```
3411
+ ```bash
3235
3412
  gem install mimemagic -v '~> 0.3.0'
3236
3413
  ```
3237
3414
 
@@ -3241,21 +3418,21 @@ If not used, Mime type used for conversion is the one provided by the node API.
3241
3418
 
3242
3419
  If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
3243
3420
 
3244
- ### Image: ImageMagick and optipng
3421
+ #### Image: ImageMagick and optipng
3245
3422
 
3246
- ```
3423
+ ```bash
3247
3424
  yum install -y ImageMagick optipng
3248
3425
  ```
3249
3426
 
3250
- ### Video: FFmpeg
3427
+ #### Video: FFmpeg
3251
3428
 
3252
3429
  The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
3253
3430
 
3254
- ```
3431
+ ```bash
3255
3432
  curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
3256
3433
  ```
3257
3434
 
3258
- ### Office: Unoconv and Libreoffice
3435
+ #### Office: Unoconv and Libreoffice
3259
3436
 
3260
3437
  If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
3261
3438
 
@@ -3263,13 +3440,13 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
3263
3440
 
3264
3441
  * CentOS 8
3265
3442
 
3266
- ```
3443
+ ```bash
3267
3444
  dnf install unoconv
3268
3445
  ```
3269
3446
 
3270
3447
  * Amazon Linux
3271
3448
 
3272
- ```
3449
+ ```bash
3273
3450
  amazon-linux-extras enable libreoffice
3274
3451
  yum clean metadata
3275
3452
  yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
@@ -3278,7 +3455,7 @@ mv unoconv /usr/bin
3278
3455
  chmod a+x /usr/bin/unoconv
3279
3456
  ```
3280
3457
 
3281
- ## Configuration
3458
+ ### Configuration
3282
3459
 
3283
3460
  The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
3284
3461
 
@@ -3286,7 +3463,7 @@ Like any <%=tool%> commands, parameters can be passed on command line or using a
3286
3463
 
3287
3464
  Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
3288
3465
 
3289
- ```
3466
+ ```bash
3290
3467
  su -s /bin/bash - xfer
3291
3468
 
3292
3469
  <%=cmd%> config preset update previewconf --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
@@ -3299,13 +3476,13 @@ Here we assume that Office file generation is disabled, else remove this option.
3299
3476
 
3300
3477
  One can check if the access key is well configured using:
3301
3478
 
3302
- ```
3479
+ ```bash
3303
3480
  <%=cmd%> -Ppreviewconf node browse /
3304
3481
  ```
3305
3482
 
3306
3483
  This shall list the contents of the storage root of the access key.
3307
3484
 
3308
- ## Execution
3485
+ ### Execution
3309
3486
 
3310
3487
  The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
3311
3488
  It needs to be run on a regular basis to create or update preview files. For that use your best
@@ -3315,7 +3492,7 @@ Typically, for "Access key" access, the system/transfer is `xfer`. So, in order
3315
3492
 
3316
3493
  Lets do a one shot test, using the configuration previously created:
3317
3494
 
3318
- ```
3495
+ ```bash
3319
3496
  su -s /bin/bash - xfer
3320
3497
 
3321
3498
  <%=cmd%> preview scan --overwrite=always
@@ -3325,7 +3502,7 @@ When the preview generator is first executed it will create a file: `.aspera_acc
3325
3502
  in the previews folder which contains the access key used.
3326
3503
  On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
3327
3504
 
3328
- ## Configuration for Execution in scheduler
3505
+ ### Configuration for Execution in scheduler
3329
3506
 
3330
3507
  Here is an example of configuration for use with cron on Linux.
3331
3508
  Adapt the scripts to your own preference.
@@ -3336,7 +3513,7 @@ Lets first setup a script that will be used in the scheduler and sets up the env
3336
3513
 
3337
3514
  Example of startup script `cron_<%=cmd%>`, which sets the Ruby environment and adds some timeout protection:
3338
3515
 
3339
- ```
3516
+ ```bash
3340
3517
  #!/bin/bash
3341
3518
  # set a timeout protection, just in case
3342
3519
  case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
@@ -3347,7 +3524,7 @@ exec timeout ${tmout} <%=cmd%> "${@}"
3347
3524
 
3348
3525
  Here the cronjob is created for user `xfer`.
3349
3526
 
3350
- ```
3527
+ ```bash
3351
3528
  crontab<<EOF
3352
3529
  0 * * * * /home/xfer/cron_<%=cmd%> preview scan --logger=syslog --display=error
3353
3530
  2-59 * * * * /home/xfer/cron_<%=cmd%> preview trev --logger=syslog --display=error
@@ -3356,7 +3533,7 @@ EOF
3356
3533
 
3357
3534
  Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
3358
3535
 
3359
- ## Candidate detection for creation or update (or deletion)
3536
+ ### Candidate detection for creation or update (or deletion)
3360
3537
 
3361
3538
  The tool generates preview files using those commands:
3362
3539
 
@@ -3378,7 +3555,7 @@ Deletion of preview for deleted source files: not implemented yet (TODO).
3378
3555
 
3379
3556
  If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
3380
3557
 
3381
- ```
3558
+ ```bash
3382
3559
  <%=cmd%> preview scan --skip-folders=@json:'["/not_here"]'
3383
3560
  ```
3384
3561
 
@@ -3388,20 +3565,20 @@ When scanning the option `value` has the same behavior as for the `node find` co
3388
3565
 
3389
3566
  For instance to filter out files beginning with `._` do:
3390
3567
 
3391
- ```
3568
+ ```bash
3392
3569
  ... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
3393
3570
  ```
3394
3571
 
3395
- ## Preview File types
3572
+ ### Preview File types
3396
3573
 
3397
3574
  Two types of preview can be generated:
3398
3575
 
3399
- * png: thumbnail
3400
- * mp4: video preview (only for video)
3576
+ * png: thumbnail
3577
+ * mp4: video preview (only for video)
3401
3578
 
3402
3579
  Use option `skip_format` to skip generation of a format.
3403
3580
 
3404
- ## Supported input Files types
3581
+ ### Supported input Files types
3405
3582
 
3406
3583
  The preview generator supports rendering of those file categories:
3407
3584
 
@@ -3421,7 +3598,7 @@ File type is primarily based on file extension detected by the node API and tran
3421
3598
 
3422
3599
  The tool can also locally detect the mime type using gem `mimemagic`.
3423
3600
 
3424
- ## Access to original files and preview creation
3601
+ ### Access to original files and preview creation
3425
3602
 
3426
3603
  Standard open source tools are used to create thumbnails and video previews.
3427
3604
  Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
@@ -3432,8 +3609,7 @@ are directly written to the storage.
3432
3609
 
3433
3610
  If the preview generator does not have access to files on the file system (it is remote, no mount, or is an object storage), then the original file is first downloaded, then the result is uploaded, use method `remote`.
3434
3611
 
3435
-
3436
- # SMTP for email notifications
3612
+ ## SMTP for email notifications
3437
3613
 
3438
3614
  Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
3439
3615
 
@@ -3450,9 +3626,9 @@ The `smtp` option is a hash table (extended value) with the following fields:
3450
3626
  <tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
3451
3627
  </table>
3452
3628
 
3453
- ## Example of configuration:
3629
+ ### Example of configuration:
3454
3630
 
3455
- ```
3631
+ ```bash
3456
3632
  <%=cmd%> config preset set smtp_google server smtp.google.com
3457
3633
  <%=cmd%> config preset set smtp_google username john@gmail.com
3458
3634
  <%=cmd%> config preset set smtp_google password P@ssw0rd
@@ -3460,24 +3636,24 @@ The `smtp` option is a hash table (extended value) with the following fields:
3460
3636
 
3461
3637
  or
3462
3638
 
3463
- ```
3639
+ ```javascript
3464
3640
  <%=cmd%> config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
3465
3641
  ```
3466
3642
 
3467
3643
  or
3468
3644
 
3469
- ```
3645
+ ```bash
3470
3646
  <%=cmd%> config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
3471
3647
  ```
3472
3648
 
3473
3649
  Set this configuration as global default, for instance:
3474
3650
 
3475
- ```
3651
+ ```bash
3476
3652
  <%=cmd%> config preset set cli_default smtp @val:@preset:smtp_google
3477
3653
  <%=cmd%> config preset set default config cli_default
3478
3654
  ```
3479
3655
 
3480
- ## Email templates
3656
+ ### Email templates
3481
3657
 
3482
3658
  Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
3483
3659
 
@@ -3491,16 +3667,16 @@ The following variables are defined by default:
3491
3667
 
3492
3668
  Other variables are defined depending on context.
3493
3669
 
3494
- ## Test
3670
+ ### Test
3495
3671
 
3496
3672
  Check settings with `smtp_settings` command. Send test email with `email_test`.
3497
3673
 
3498
- ```
3674
+ ```bash
3499
3675
  <%=cmd%> config --smtp=@preset:smtp_google smtp
3500
3676
  <%=cmd%> config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
3501
3677
  ```
3502
3678
 
3503
- ## Notifications for transfer status
3679
+ ### Notifications for transfer status
3504
3680
 
3505
3681
  An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
3506
3682
 
@@ -3517,7 +3693,7 @@ The environment provided contains the following additional variables:
3517
3693
 
3518
3694
  Example of template:
3519
3695
 
3520
- ```
3696
+ ```text
3521
3697
  From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
3522
3698
  To: <<%='<'%>%=to%>>
3523
3699
  Subject: <%='<'%>%=subject%>
@@ -3525,7 +3701,7 @@ Subject: <%='<'%>%=subject%>
3525
3701
  Transfer is: <%='<'%>%=global_transfer_status%>
3526
3702
  ```
3527
3703
 
3528
- # Tool: `asession`
3704
+ ## Tool: `asession`
3529
3705
 
3530
3706
  This gem comes with a second executable tool providing a simplified standardized interface
3531
3707
  to start a FASP session: `asession`.
@@ -3550,12 +3726,12 @@ During execution, it generates all low level events, one per line, in JSON forma
3550
3726
 
3551
3727
  Note that there are special "extended" [_transfer-spec_](#transferspec) parameters supported by `asession`:
3552
3728
 
3553
- * `EX_loglevel` to change log level of the tool
3554
- * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
3729
+ * `EX_loglevel` to change log level of the tool
3730
+ * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
3555
3731
 
3556
3732
  Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
3557
3733
 
3558
- ## Comparison of interfaces
3734
+ ### Comparison of interfaces
3559
3735
 
3560
3736
  <table>
3561
3737
  <tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
@@ -3565,21 +3741,21 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
3565
3741
  <tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
3566
3742
  <tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp</td><td>any with ascp</td><td>any with ascp and transferdaemon</td></tr></table>
3567
3743
 
3568
- ## Simple session
3744
+ ### Simple session
3569
3745
 
3570
- ```
3746
+ ```javascript
3571
3747
  MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
3572
3748
 
3573
3749
  echo "${MY_TSPEC}"|asession
3574
3750
  ```
3575
3751
 
3576
- ## Asynchronous commands and Persistent session
3752
+ ### Asynchronous commands and Persistent session
3577
3753
 
3578
3754
  `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
3579
3755
 
3580
3756
  This is particularly useful for a persistent session ( with the [_transfer-spec_](#transferspec) parameter: `"keepalive":true` )
3581
3757
 
3582
- ```
3758
+ ```javascript
3583
3759
  asession
3584
3760
  {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
3585
3761
  {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
@@ -3588,20 +3764,20 @@ asession
3588
3764
 
3589
3765
  (events from FASP are not shown in above example. They would appear after each command)
3590
3766
 
3591
- ## Example of language wrapper
3767
+ ### Example of language wrapper
3592
3768
 
3593
3769
  Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/aspera)
3594
3770
 
3595
- ## Help
3771
+ ### Help
3596
3772
 
3597
- ```
3773
+ ```bash
3598
3774
  asession -h
3599
3775
  <%=File.read(ENV["INCL_ASESSION"])%>
3600
3776
  ```
3601
3777
 
3602
- # Hot folder
3778
+ ## Hot folder
3603
3779
 
3604
- ## Requirements
3780
+ ### Requirements
3605
3781
 
3606
3782
  <%=tool%> maybe used as a simple hot folder engine. A hot folder being defined as a tool that:
3607
3783
 
@@ -3613,7 +3789,7 @@ asession -h
3613
3789
 
3614
3790
  In addition: the detection should be made "continuously" or on specific time/date.
3615
3791
 
3616
- ## Setup procedure
3792
+ ### Setup procedure
3617
3793
 
3618
3794
  The general idea is to rely on :
3619
3795
 
@@ -3621,7 +3797,7 @@ The general idea is to rely on :
3621
3797
  * take advantage of <%=tool%> configuration capabilities and server side knowledge
3622
3798
  * the OS scheduler for reliability and continuous operation
3623
3799
 
3624
- ### ascp features
3800
+ #### ascp features
3625
3801
 
3626
3802
  Interesting ascp features are found in its arguments: (see ascp manual):
3627
3803
 
@@ -3636,31 +3812,33 @@ Note that:
3636
3812
  * most, but not all native ascp arguments are available as standard transfer_spec parameters
3637
3813
  * native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
3638
3814
 
3639
- ### server side and configuration
3815
+ #### server side and configuration
3640
3816
 
3641
3817
  Virtually any transfer on a "repository" on a regular basis might emulate a hot folder. Note that file detection is not based on events (inotify, etc...), but on a stateless scan on source side.
3642
3818
 
3643
3819
  Note: parameters may be saved in a <%=prst%> and used with `-P`.
3644
3820
 
3645
- ### Scheduling
3821
+ #### Scheduling
3646
3822
 
3647
3823
  Once <%=tool%> parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [_Scheduling_](#_scheduling_).
3648
3824
 
3649
- ## Example
3825
+ ### Example: upload folder
3650
3826
 
3651
- ```
3827
+ ```javascript
3652
3828
  <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
3653
-
3654
3829
  ```
3655
3830
 
3656
3831
  The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
3657
3832
 
3658
- # Health check and Nagios
3833
+ ## Health check and Nagios
3659
3834
 
3660
3835
  Most plugin provide a `health` command that will check the health status of the application. Example:
3661
3836
 
3662
- ```
3837
+ ```bash
3663
3838
  <%=cmd%> console health
3839
+ ```
3840
+
3841
+ ```output
3664
3842
  +--------+-------------+------------+
3665
3843
  | status | component | message |
3666
3844
  +--------+-------------+------------+
@@ -3676,14 +3854,23 @@ Typically, the health check uses the REST API of the application with the follow
3676
3854
 
3677
3855
  <%=tool%> can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
3678
3856
 
3679
- ```
3857
+ ```bash
3680
3858
  <%=cmd%> server health transfer --to-folder=/Upload --format=nagios --progress=none
3859
+ ```
3860
+
3861
+ ```output
3681
3862
  OK - [transfer:ok]
3863
+ ```
3864
+
3865
+ ```bash
3682
3866
  <%=cmd%> server health asctlstatus --cmd_prefix='sudo ' --format=nagios
3867
+ ```
3868
+
3869
+ ```output
3683
3870
  OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
3684
3871
  ```
3685
3872
 
3686
- # Ruby Module: `Aspera`
3873
+ ## Ruby Module: `Aspera`
3687
3874
 
3688
3875
  Main components:
3689
3876
 
@@ -3693,18 +3880,20 @@ Main components:
3693
3880
 
3694
3881
  A working example can be found in the gem, example:
3695
3882
 
3696
- ```
3883
+ ```bash
3697
3884
  <%=cmd%> config gem_path
3885
+ ```
3886
+
3887
+ ```bash
3698
3888
  cat $(<%=cmd%> config gem_path)/../examples/transfer.rb
3699
3889
  ```
3700
3890
 
3701
- This sample code shows some example of use of the API as well as
3702
- REST API.
3891
+ This sample code shows some example of use of the API as well as REST API.
3703
3892
  Note: although nice, it's probably a good idea to use RestClient for REST.
3704
3893
 
3705
3894
  Example of use of the API of Aspera on Cloud:
3706
3895
 
3707
- ```
3896
+ ```ruby
3708
3897
  require 'aspera/aoc'
3709
3898
 
3710
3899
  aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/<%=cmd%>/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
@@ -3712,7 +3901,9 @@ aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user
3712
3901
  aoc.read('self')
3713
3902
  ```
3714
3903
 
3715
- # History
3904
+ <https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
3905
+
3906
+ ## History
3716
3907
 
3717
3908
  When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
3718
3909
 
@@ -3728,213 +3919,228 @@ So, it evolved into <%=tool%>:
3728
3919
  * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
3729
3920
  * `ruby` is consistent with other Aspera products
3730
3921
 
3731
- # Changes (Release notes)
3922
+ ## Changes (Release notes)
3732
3923
 
3733
3924
  * <%=gemspec.version.to_s%>
3734
3925
 
3735
- * new: support transfer agent: [Transfer SDK](#agt_trsdk)
3736
- * new: support [http socket options](#http_options)
3737
- * new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
3738
- * new: `config vault` supports encrypted passwords, also macos keychain
3739
- * new: `config preset` command for consistency with id
3740
- * new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
3741
- * change: when using wss, use [ruby's CA certs](#certificates)
3742
- * change: unexpected parameter makes exit code not zero
3743
- * change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
3744
- * change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
3745
- * change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
3746
- * fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
3926
+ * new: command `conf plugin create`
3927
+ * new: global option `plugin_folder`
3928
+ * new: global option `transpose_single`
3929
+ * new: simplified metadata passing for shared inbox package creation in AoC
3930
+ * change: (break) command `aoc packages shared_inboxes list` replaces `aoc user shared_inboxes`
3931
+ * change: (break) command `aoc user profile` replaces `aoc user info`
3932
+ * change: (break) command `aoc user workspaces list` replaces `aoc user workspaces`
3933
+ * change: (break) command `aoc user workspaces current` replaces `aoc workspace`
3934
+ * change: (break) command `conf plugin list` replaces `conf plugins`
3935
+ * change: (break) command `conf connect` simplified
3936
+ * fix: #60 ascli executable was not installed by default in 4.5.0
3937
+ * fix: add password hiding case in logs
3938
+
3939
+ * 4.5.0
3940
+
3941
+ * new: support transfer agent: [Transfer SDK](#agt_trsdk)
3942
+ * new: support [http socket options](#http_options)
3943
+ * new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
3944
+ * new: `config vault` supports encrypted passwords, also macos keychain
3945
+ * new: `config preset` command for consistency with id
3946
+ * new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
3947
+ * change: when using wss, use [ruby's CA certs](#certificates)
3948
+ * change: unexpected parameter makes exit code not zero
3949
+ * change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
3950
+ * change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
3951
+ * change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
3952
+ * fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
3747
3953
 
3748
3954
  * 4.4.0
3749
3955
 
3750
- * new: `aoc packages list` add possibility to add filter with option `query`
3751
- * new: `aoc admin res xxx list` now get all items by default #50
3752
- * new: `preset` option can specify name or hash value
3753
- * new: `node` plugin accepts bearer token and access key as credential
3754
- * new: `node` option `token_type` allows using basic token in addition to aspera type.
3755
- * change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
3756
- * change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
3956
+ * new: `aoc packages list` add possibility to add filter with option `query`
3957
+ * new: `aoc admin res xxx list` now get all items by default #50
3958
+ * new: `preset` option can specify name or hash value
3959
+ * new: `node` plugin accepts bearer token and access key as credential
3960
+ * new: `node` option `token_type` allows using basic token in addition to aspera type.
3961
+ * change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
3962
+ * change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
3757
3963
 
3758
3964
  * 4.3.0
3759
3965
 
3760
- * new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
3761
- * new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
3966
+ * new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
3967
+ * new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
3762
3968
 
3763
3969
  * 4.2.2
3764
3970
 
3765
- * new: `faspex package list` retrieves the whole list, not just first page
3766
- * new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
3767
- * new: the error "Remote host is not who we expected" displays a special remediation message
3768
- * new: `conf ascp spec` displays supported transfer spec
3769
- * new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
3770
- * fix: space character in `faspe:` url are precent encoded if needed
3771
- * fix: `preview scan`: if file_id is unknown, ignore and continue scan
3772
- * change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then <%=tool%> exits with code 1 (instead of zero=success)
3773
- * change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
3971
+ * new: `faspex package list` retrieves the whole list, not just first page
3972
+ * new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
3973
+ * new: the error "Remote host is not who we expected" displays a special remediation message
3974
+ * new: `conf ascp spec` displays supported transfer spec
3975
+ * new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
3976
+ * fix: space character in `faspe:` url are precent encoded if needed
3977
+ * fix: `preview scan`: if file_id is unknown, ignore and continue scan
3978
+ * change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then <%=tool%> exits with code 1 (instead of zero=success)
3979
+ * change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
3774
3980
 
3775
3981
  * 4.2.1
3776
3982
 
3777
- * new: command `faspex package recv` supports link of type: `faspe:`
3778
- * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
3983
+ * new: command `faspex package recv` supports link of type: `faspe:`
3984
+ * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
3779
3985
 
3780
3986
  * 4.2.0
3781
3987
 
3782
- * new: command `aoc remind` to receive organization membership by email
3783
- * new: in `preview` option `value` to filter out on file name
3784
- * new: `initdemo` to initialize for demo server
3785
- * new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
3786
- * fix: on Windows `conf ascp use` expects ascp.exe
3787
- * fix: (break) multi_session_threshold is Integer, not String
3788
- * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
3789
- * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
3790
- * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
3791
- * change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
3988
+ * new: command `aoc remind` to receive organization membership by email
3989
+ * new: in `preview` option `value` to filter out on file name
3990
+ * new: `initdemo` to initialize for demo server
3991
+ * new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
3992
+ * fix: on Windows `conf ascp use` expects ascp.exe
3993
+ * fix: (break) multi_session_threshold is Integer, not String
3994
+ * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
3995
+ * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
3996
+ * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
3997
+ * change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
3792
3998
 
3793
3999
  * 4.1.0
3794
4000
 
3795
- * fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
3796
- * new: update documentation with regard to offline and docker installation
3797
- * new: renamed command `nagios_check` to `health`
3798
- * new: agent `http_gw` now supports upload
3799
- * new: added option `sdk_url` to install SDK from local file for offline install
3800
- * new: check new gem version periodically
3801
- * new: the --fields= option, support -_fieldname_ to remove a field from default fields
3802
- * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
3803
- * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
3804
- * new: AoC a password can be provided for a public link
3805
- * new: `conf doc` take an optional parameter to go to a section
3806
- * new: initial support for Faspex 5 Beta 1
4001
+ * fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
4002
+ * new: update documentation with regard to offline and docker installation
4003
+ * new: renamed command `nagios_check` to `health`
4004
+ * new: agent `http_gw` now supports upload
4005
+ * new: added option `sdk_url` to install SDK from local file for offline install
4006
+ * new: check new gem version periodically
4007
+ * new: the --fields= option, support -_fieldname_ to remove a field from default fields
4008
+ * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
4009
+ * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
4010
+ * new: AoC a password can be provided for a public link
4011
+ * new: `conf doc` take an optional parameter to go to a section
4012
+ * new: initial support for Faspex 5 Beta 1
3807
4013
 
3808
4014
  * 4.0.0
3809
4015
 
3810
- * now available as open source at [<%=gemspec.homepage%>](<%=gemspec.homepage%>) with general cleanup
3811
- * changed default tool name from `mlia` to `ascli`
3812
- * changed `aspera` command to `aoc`
3813
- * changed gem name from `asperalm` to `aspera-cli`
3814
- * changed module name from `Asperalm` to `Aspera`
3815
- * removed command `folder` in `preview`, merged to `scan`
3816
- * persistency files go to sub folder instead of main folder
3817
- * added possibility to install SDK: `config ascp install`
4016
+ * now available as open source at [<%=gemspec.homepage%>](<%=gemspec.homepage%>) with general cleanup
4017
+ * changed default tool name from `mlia` to `ascli`
4018
+ * changed `aspera` command to `aoc`
4019
+ * changed gem name from `asperalm` to `aspera-cli`
4020
+ * changed module name from `Asperalm` to `Aspera`
4021
+ * removed command `folder` in `preview`, merged to `scan`
4022
+ * persistency files go to sub folder instead of main folder
4023
+ * added possibility to install SDK: `config ascp install`
3818
4024
 
3819
4025
  * 0.11.8
3820
4026
 
3821
- * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
4027
+ * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
3822
4028
 
3823
4029
  * 0.11.7
3824
4030
 
3825
- * rework on rest call error handling
3826
- * use option `display` with value `data` to remove out of extraneous information
3827
- * fixed option `lock_port` not working
3828
- * generate special icon if preview failed
3829
- * possibility to choose transfer progress bar type with option `progress`
3830
- * AoC package creation now output package id
4031
+ * rework on rest call error handling
4032
+ * use option `display` with value `data` to remove out of extraneous information
4033
+ * fixed option `lock_port` not working
4034
+ * generate special icon if preview failed
4035
+ * possibility to choose transfer progress bar type with option `progress`
4036
+ * AoC package creation now output package id
3831
4037
 
3832
4038
  * 0.11.6
3833
4039
 
3834
- * orchestrator : added more choice in auth type
3835
- * preview: cleanup in generator (removed and renamed parameters)
3836
- * preview: better documentation
3837
- * preview: animated thumbnails for video (option: `video_png_conv=animated`)
3838
- * preview: new event trigger: `trevents` (`events` seems broken)
3839
- * preview: unique tmp folder to avoid clash of multiple instances
3840
- * repo: added template for secrets used for testing
4040
+ * orchestrator : added more choice in auth type
4041
+ * preview: cleanup in generator (removed and renamed parameters)
4042
+ * preview: better documentation
4043
+ * preview: animated thumbnails for video (option: `video_png_conv=animated`)
4044
+ * preview: new event trigger: `trevents` (`events` seems broken)
4045
+ * preview: unique tmp folder to avoid clash of multiple instances
4046
+ * repo: added template for secrets used for testing
3841
4047
 
3842
4048
  * 0.11.5
3843
4049
 
3844
- * added option `default_ports` for AoC (see manual)
3845
- * allow bulk delete in `aspera files` with option `bulk=yes`
3846
- * fix getting connect versions
3847
- * added section for Aix
3848
- * support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
3849
- * added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
4050
+ * added option `default_ports` for AoC (see manual)
4051
+ * allow bulk delete in `aspera files` with option `bulk=yes`
4052
+ * fix getting connect versions
4053
+ * added section for Aix
4054
+ * support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
4055
+ * added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
3850
4056
 
3851
4057
  * 0.11.4
3852
4058
 
3853
- * possibility to give shared inbox name when sending a package (else use id and type)
4059
+ * possibility to give shared inbox name when sending a package (else use id and type)
3854
4060
 
3855
4061
  * 0.11.3
3856
4062
 
3857
- * minor fixes on multi-session: avoid exception on progress bar
4063
+ * minor fixes on multi-session: avoid exception on progress bar
3858
4064
 
3859
4065
  * 0.11.2
3860
4066
 
3861
- * fixes on multi-session: progress bat and transfer spec param for "direct"
4067
+ * fixes on multi-session: progress bat and transfer spec param for "direct"
3862
4068
 
3863
4069
  * 0.11.1
3864
4070
 
3865
- * enhanced short_link creation commands (see examples)
4071
+ * enhanced short_link creation commands (see examples)
3866
4072
 
3867
4073
  * 0.11
3868
4074
 
3869
- * add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
4075
+ * add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
3870
4076
 
3871
4077
  * 0.10.18
3872
4078
 
3873
- * new option in. `server` : `ssh_options`
4079
+ * new option in. `server` : `ssh_options`
3874
4080
 
3875
4081
  * 0.10.17
3876
4082
 
3877
- * fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
3878
- * new modifier: `@list:<separator>val1<separator>...`
4083
+ * fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
4084
+ * new modifier: `@list:<separator>val1<separator>...`
3879
4085
 
3880
4086
  * 0.10.16
3881
4087
 
3882
- * added list of shared inboxes in workspace (or global), use `--query=@json:'{}'`
4088
+ * added list of shared inboxes in workspace (or global), use `--query=@json:'{}'`
3883
4089
 
3884
4090
  * 0.10.15
3885
4091
 
3886
- * in case of command line error, display the error cause first, and non-parsed argument second
3887
- * AoC : Activity / Analytics
4092
+ * in case of command line error, display the error cause first, and non-parsed argument second
4093
+ * AoC : Activity / Analytics
3888
4094
 
3889
4095
  * 0.10.14
3890
4096
 
3891
- * added missing bss plugin
4097
+ * added missing bss plugin
3892
4098
 
3893
4099
  * 0.10.13
3894
4100
 
3895
- * added Faspex5 (use option `value` to give API arguments)
4101
+ * added Faspex5 (use option `value` to give API arguments)
3896
4102
 
3897
4103
  * 0.10.12
3898
4104
 
3899
- * added support for AoC node registration keys
3900
- * replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
3901
- * Transfer agent is no more a Singleton instance, but only one is used in CLI
3902
- * `@incps` : new extended value modifier
3903
- * ATS: no more provides access keys secrets: now user must provide it
3904
- * begin work on "aoc" transfer agent
4105
+ * added support for AoC node registration keys
4106
+ * replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
4107
+ * Transfer agent is no more a Singleton instance, but only one is used in CLI
4108
+ * `@incps` : new extended value modifier
4109
+ * ATS: no more provides access keys secrets: now user must provide it
4110
+ * begin work on "aoc" transfer agent
3905
4111
 
3906
4112
  * 0.10.11
3907
4113
 
3908
- * minor refactor and fixes
4114
+ * minor refactor and fixes
3909
4115
 
3910
4116
  * 0.10.10
3911
4117
 
3912
- * fix on documentation
4118
+ * fix on documentation
3913
4119
 
3914
4120
  * 0.10.9.1
3915
4121
 
3916
- * add total number of items for AoC resource list
3917
- * better gem version dependency (and fixes to support Ruby 2.0.0)
3918
- * removed aoc search_nodes
4122
+ * add total number of items for AoC resource list
4123
+ * better gem version dependency (and fixes to support Ruby 2.0.0)
4124
+ * removed aoc search_nodes
3919
4125
 
3920
4126
  * 0.10.8
3921
4127
 
3922
- * removed option: `fasp_proxy`, use pseudo transfer spec parameter: `EX_fasp_proxy_url`
3923
- * removed option: `http_proxy`, use pseudo transfer spec parameter: `EX_http_proxy_url`
3924
- * several other changes..
4128
+ * removed option: `fasp_proxy`, use pseudo transfer spec parameter: `EX_fasp_proxy_url`
4129
+ * removed option: `http_proxy`, use pseudo transfer spec parameter: `EX_http_proxy_url`
4130
+ * several other changes..
3925
4131
 
3926
4132
  * 0.10.7
3927
4133
 
3928
- * fix: <%=cmd%> fails when username cannot be computed on Linux.
4134
+ * fix: <%=cmd%> fails when username cannot be computed on Linux.
3929
4135
 
3930
4136
  * 0.10.6
3931
4137
 
3932
- * FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
3933
- * gem version requirements made more open
4138
+ * FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
4139
+ * gem version requirements made more open
3934
4140
 
3935
4141
  * 0.10.5
3936
4142
 
3937
- * fix faspex package receive command not working
4143
+ * fix faspex package receive command not working
3938
4144
 
3939
4145
  * 0.10.4
3940
4146
 
@@ -3943,220 +4149,218 @@ So, it evolved into <%=tool%>:
3943
4149
 
3944
4150
  * 0.10.3
3945
4151
 
3946
- * included user name in oauth bearer token cache for AoC when JWT is used.
4152
+ * included user name in oauth bearer token cache for AoC when JWT is used.
3947
4153
 
3948
4154
  * 0.10.2
3949
4155
 
3950
- * updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
3951
- * added doc for "cargo" like actions
3952
- * added doc for multi-session
4156
+ * updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
4157
+ * added doc for "cargo" like actions
4158
+ * added doc for multi-session
3953
4159
 
3954
4160
  * 0.10.1
3955
4161
 
3956
- * AoC and node v4 "browse" works now on non-folder items: file, link
3957
- * initial support for AoC automation (do not use yet)
4162
+ * AoC and node v4 "browse" works now on non-folder items: file, link
4163
+ * initial support for AoC automation (do not use yet)
3958
4164
 
3959
4165
  * 0.10
3960
4166
 
3961
- * support for transfer using IBM Cloud Object Storage
3962
- * improved `find` action using arbitrary expressions
4167
+ * support for transfer using IBM Cloud Object Storage
4168
+ * improved `find` action using arbitrary expressions
3963
4169
 
3964
4170
  * 0.9.36
3965
4171
 
3966
- * added option to specify file pair lists
4172
+ * added option to specify file pair lists
3967
4173
 
3968
4174
  * 0.9.35
3969
4175
 
3970
- * updated plugin `preview` , changed parameter names, added documentation
3971
- * fix in `ats` plugin : instance id needed in request header
4176
+ * updated plugin `preview` , changed parameter names, added documentation
4177
+ * fix in `ats` plugin : instance id needed in request header
3972
4178
 
3973
4179
  * 0.9.34
3974
4180
 
3975
- * parser "@preset" can be used again in option "transfer_info"
3976
- * some documentation re-organizing
4181
+ * parser "@preset" can be used again in option "transfer_info"
4182
+ * some documentation re-organizing
3977
4183
 
3978
4184
  * 0.9.33
3979
4185
 
3980
- * new command to display basic token of node
3981
- * new command to display bearer token of node in AoC
3982
- * the --fields= option, support +_fieldname_ to add a field to default fields
3983
- * many small changes
4186
+ * new command to display basic token of node
4187
+ * new command to display bearer token of node in AoC
4188
+ * the --fields= option, support +_fieldname_ to add a field to default fields
4189
+ * many small changes
3984
4190
 
3985
4191
  * 0.9.32
3986
4192
 
3987
- * all Faspex public links are now supported
3988
- * removed faspex operation recv_publink
3989
- * replaced with option `link` (consistent with AoC)
4193
+ * all Faspex public links are now supported
4194
+ * removed faspex operation recv_publink
4195
+ * replaced with option `link` (consistent with AoC)
3990
4196
 
3991
4197
  * 0.9.31
3992
4198
 
3993
- * added more support for public link: receive and send package, to user or dropbox and files view.
3994
- * delete expired file lists
3995
- * changed text table gem from text-table to terminal-table because it supports multiline values
4199
+ * added more support for public link: receive and send package, to user or dropbox and files view.
4200
+ * delete expired file lists
4201
+ * changed text table gem from text-table to terminal-table because it supports multiline values
3996
4202
 
3997
4203
  * 0.9.27
3998
4204
 
3999
- * basic email support with SMTP
4000
- * basic proxy auto config support
4205
+ * basic email support with SMTP
4206
+ * basic proxy auto config support
4001
4207
 
4002
4208
  * 0.9.26
4003
4209
 
4004
- * table display with --fields=ALL now includes all column names from all lines, not only first one
4005
- * unprocessed argument shows error even if there is an error beforehand
4210
+ * table display with --fields=ALL now includes all column names from all lines, not only first one
4211
+ * unprocessed argument shows error even if there is an error beforehand
4006
4212
 
4007
4213
  * 0.9.25
4008
4214
 
4009
- * the option `value` of command `find`, to filter on name, is not optional
4010
- * `find` now also reports all types (file, folder, link)
4011
- * `find` now is able to report all fields (type, size, etc...)
4215
+ * the option `value` of command `find`, to filter on name, is not optional
4216
+ * `find` now also reports all types (file, folder, link)
4217
+ * `find` now is able to report all fields (type, size, etc...)
4012
4218
 
4013
4219
  * 0.9.24
4014
4220
 
4015
- * fix bug where AoC node to node transfer did not work
4016
- * fix bug on error if ED25519 private key is defined in .ssh
4221
+ * fix bug where AoC node to node transfer did not work
4222
+ * fix bug on error if ED25519 private key is defined in .ssh
4017
4223
 
4018
4224
  * 0.9.23
4019
4225
 
4020
- * defined REST error handlers, more error conditions detected
4021
- * commands to select specific ascp location
4226
+ * defined REST error handlers, more error conditions detected
4227
+ * commands to select specific ascp location
4022
4228
 
4023
4229
  * 0.9.21
4024
4230
 
4025
- * supports simplified wizard using global client
4026
- * only ascp binary is required, other SDK (keys) files are now generated
4231
+ * supports simplified wizard using global client
4232
+ * only ascp binary is required, other SDK (keys) files are now generated
4027
4233
 
4028
4234
  * 0.9.20
4029
4235
 
4030
- * improved wizard (prepare for AoC global client id)
4031
- * preview generator: addedoption : --skip-format=&lt;png,mp4&gt;
4032
- * removed outdated pictures from this doc
4236
+ * improved wizard (prepare for AoC global client id)
4237
+ * preview generator: addedoption : --skip-format=&lt;png,mp4&gt;
4238
+ * removed outdated pictures from this doc
4033
4239
 
4034
4240
  * 0.9.19
4035
4241
 
4036
- * added command aspera bearer --scope=xx
4242
+ * added command aspera bearer --scope=xx
4037
4243
 
4038
4244
  * 0.9.18
4039
4245
 
4040
- * enhanced aspera admin events to support query
4246
+ * enhanced aspera admin events to support query
4041
4247
 
4042
4248
  * 0.9.16
4043
4249
 
4044
- * AoC transfers are now reported in activity app
4045
- * new interface for Rest class authentication (keep backward compatibility)
4250
+ * AoC transfers are now reported in activity app
4251
+ * new interface for Rest class authentication (keep backward compatibility)
4046
4252
 
4047
4253
  * 0.9.15
4048
4254
 
4049
- * new feature: "find" command in aspera files
4050
- * sample code for transfer API
4255
+ * new feature: "find" command in aspera files
4256
+ * sample code for transfer API
4051
4257
 
4052
4258
  * 0.9.12
4053
4259
 
4054
- * add nagios commands
4055
- * support of ATS for IBM Cloud, removed old version based on aspera id
4260
+ * add nagios commands
4261
+ * support of ATS for IBM Cloud, removed old version based on aspera id
4056
4262
 
4057
4263
  * 0.9.11
4058
4264
 
4059
- * Breaking change: @stdin is now @stdin:
4060
- * support of ATS for IBM Cloud, removed old version based on aspera id
4265
+ * Breaking change: @stdin is now @stdin:
4266
+ * support of ATS for IBM Cloud, removed old version based on aspera id
4061
4267
 
4062
4268
 
4063
4269
  * 0.9.10
4064
4270
 
4065
- * Breaking change: parameter transfer-node becomes more generic: transfer-info
4066
- * Display SaaS storage usage with command: aspera admin res node --id=nn info
4067
- * cleaner way of specifying source file list for transfers
4068
- * Breaking change: replaced download_mode option with http_download action
4271
+ * Breaking change: parameter transfer-node becomes more generic: transfer-info
4272
+ * Display SaaS storage usage with command: aspera admin res node --id=nn info
4273
+ * cleaner way of specifying source file list for transfers
4274
+ * Breaking change: replaced download_mode option with http_download action
4069
4275
 
4070
4276
  * 0.9.9
4071
4277
 
4072
- * Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
4073
- * Now supports "cargo" for Aspera on Cloud (automatic package download)
4278
+ * Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
4279
+ * Now supports "cargo" for Aspera on Cloud (automatic package download)
4074
4280
 
4075
4281
  * 0.9.8
4076
4282
 
4077
- * Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
4078
- * AoC: share to share transfer with command "transfer"
4283
+ * Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
4284
+ * AoC: share to share transfer with command "transfer"
4079
4285
 
4080
4286
  * 0.9.7
4081
4287
 
4082
- * homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4083
- * preview persistency goes to unique file by default
4084
- * catch mxf extension in preview as video
4085
- * Faspex: possibility to download all packages by specifying id=ALL
4086
- * Faspex: to come: cargo-like function to download only new packages with id=NEW
4288
+ * homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4289
+ * preview persistency goes to unique file by default
4290
+ * catch mxf extension in preview as video
4291
+ * Faspex: possibility to download all packages by specifying id=ALL
4292
+ * Faspex: to come: cargo-like function to download only new packages with id=NEW
4087
4293
 
4088
4294
  * 0.9.6
4089
4295
 
4090
- * Breaking change: `@param:`is now `@preset:` and is generic
4091
- * AoC: added command to display current workspace information
4296
+ * Breaking change: `@param:`is now `@preset:` and is generic
4297
+ * AoC: added command to display current workspace information
4092
4298
 
4093
4299
  * 0.9.5
4094
4300
 
4095
- * new parameter: new_user_option used to choose between public_link and invite of external users.
4096
- * fixed bug in wizard, and wizard uses now product detection
4301
+ * new parameter: new_user_option used to choose between public_link and invite of external users.
4302
+ * fixed bug in wizard, and wizard uses now product detection
4097
4303
 
4098
4304
  * 0.9.4
4099
4305
 
4100
- * Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
4101
- * AoC Package send supports external users
4102
- * new command to export AoC config to Aspera CLI config
4306
+ * Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
4307
+ * AoC Package send supports external users
4308
+ * new command to export AoC config to Aspera CLI config
4103
4309
 
4104
4310
  * 0.9.3
4105
4311
 
4106
- * REST error message show host and code
4107
- * option for quiet display
4108
- * modified transfer interface and allow token re-generation on error
4109
- * async add admin command
4110
- * async add db parameters
4111
- * Breaking change: new option "sources" to specify files to transfer
4312
+ * REST error message show host and code
4313
+ * option for quiet display
4314
+ * modified transfer interface and allow token re-generation on error
4315
+ * async add admin command
4316
+ * async add db parameters
4317
+ * Breaking change: new option "sources" to specify files to transfer
4112
4318
 
4113
4319
  * 0.9.2
4114
4320
 
4115
- * Breaking change: changed AoC package creation to match API, see AoC section
4321
+ * Breaking change: changed AoC package creation to match API, see AoC section
4116
4322
 
4117
4323
  * 0.9.1
4118
4324
 
4119
- * Breaking change: changed faspex package creation to match API, see Faspex section
4325
+ * Breaking change: changed faspex package creation to match API, see Faspex section
4120
4326
 
4121
4327
  * 0.9
4122
4328
 
4123
- * Renamed the CLI from aslmcli to <%=tool%>
4124
- * Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
4329
+ * Renamed the CLI from aslmcli to <%=tool%>
4330
+ * Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
4125
4331
 
4126
4332
  * 0.7.6
4127
4333
 
4128
- * add "sync" plugin
4334
+ * add "sync" plugin
4129
4335
 
4130
4336
  * 0.7
4131
4337
 
4132
- * Breaking change: AoC package recv take option if for package instead of argument.
4133
- * Breaking change: Rest class and Oauth class changed init parameters
4134
- * AoC: receive package from public link
4135
- * select by col value on output
4136
- * added rename (AoC, node)
4338
+ * Breaking change: AoC package recv take option if for package instead of argument.
4339
+ * Breaking change: Rest class and Oauth class changed init parameters
4340
+ * AoC: receive package from public link
4341
+ * select by col value on output
4342
+ * added rename (AoC, node)
4137
4343
 
4138
4344
  * 0.6.19
4139
4345
 
4140
- Breaking change:
4141
-
4142
- * ats server list provisioned &rarr; ats cluster list
4143
- * ats server list clouds &rarr; ats cluster clouds
4144
- * ats server list instance --cloud=x --region=y &rarr; ats cluster show --cloud=x --region=y
4145
- * ats server id xxx &rarr; ats cluster show --id=xxx
4146
- * ats subscriptions &rarr; ats credential subscriptions
4147
- * ats api_key repository list &rarr; ats credential cache list
4148
- * ats api_key list &rarr; ats credential list
4149
- * ats access_key id xxx &rarr; ats access_key --id=xxx
4346
+ * change: (break) ats server list provisioned &rarr; ats cluster list
4347
+ * change: (break) ats server list clouds &rarr; ats cluster clouds
4348
+ * change: (break) ats server list instance --cloud=x --region=y &rarr; ats cluster show --cloud=x --region=y
4349
+ * change: (break) ats server id xxx &rarr; ats cluster show --id=xxx
4350
+ * change: (break) ats subscriptions &rarr; ats credential subscriptions
4351
+ * change: (break) ats api_key repository list &rarr; ats credential cache list
4352
+ * change: (break) ats api_key list &rarr; ats credential list
4353
+ * change: (break) ats access_key id xxx &rarr; ats access_key --id=xxx
4150
4354
 
4151
4355
  * 0.6.18
4152
4356
 
4153
- * some commands take now --id option instead of id command.
4357
+ * some commands take now --id option instead of id command.
4154
4358
 
4155
4359
  * 0.6.15
4156
4360
 
4157
- * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
4361
+ * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
4158
4362
 
4159
- # BUGS, FEATURES, CONTRIBUTION
4363
+ ## BUGS, FEATURES, CONTRIBUTION
4160
4364
 
4161
4365
  For issues or feature requests use the Github repository and issues.
4162
4366
 
@@ -4164,7 +4368,7 @@ You can also contribute to this open source project.
4164
4368
 
4165
4369
  One can also [create one's own plugin](#createownplugin).
4166
4370
 
4167
- ## Only one value for any option
4371
+ ### Only one value for any option
4168
4372
 
4169
4373
  Some commands and sub commands may ask for the same option name.
4170
4374
  Currently, since option definition is position independent (last one wins), it is not possible
@@ -4175,27 +4379,26 @@ As a solution, use the position specific notation for selection, i.e. provide th
4175
4379
 
4176
4380
  This happens typically for the `node` sub command, e.g. identify the node by name instead of id.
4177
4381
 
4178
-
4179
- ## ED255519 key not supported
4382
+ ### ED255519 key not supported
4180
4383
 
4181
4384
  ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
4182
4385
 
4183
4386
  Without this deactivation, if such key was present the following error was generated:
4184
4387
 
4185
- ```
4388
+ ```output
4186
4389
  OpenSSH keys only supported if ED25519 is available
4187
4390
  ```
4188
4391
 
4189
4392
  Which meant that you do not have ruby support for ED25519 SSH keys.
4190
4393
  You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
4191
4394
 
4192
- ## Error "Remote host is not who we expected"
4395
+ ### Error "Remote host is not who we expected"
4193
4396
 
4194
4397
  Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
4195
4398
 
4196
4399
  Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
4197
4400
 
4198
- ```
4401
+ ```bash
4199
4402
  --ts=@json:'{"sshfp":null}'
4200
4403
  ```
4201
4404
 
@@ -4203,12 +4406,12 @@ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or
4203
4406
 
4204
4407
  References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
4205
4408
 
4206
- ## Miscellaneous
4409
+ ### Miscellaneous
4207
4410
 
4208
4411
  * remove rest and oauth classes and use ruby standard gems:
4209
4412
 
4210
- * oauth
4211
- * https://github.com/rest-client/rest-client
4413
+ * oauth
4414
+ * <https://github.com/rest-client/rest-client>
4212
4415
 
4213
4416
  * use Thor or any standard Ruby CLI manager
4214
4417
 
@@ -4218,4 +4421,4 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
4218
4421
 
4219
4422
  * Going through proxy: use env var http_proxy and https_proxy, no_proxy
4220
4423
 
4221
- * easier use with https://github.com/pmq20/ruby-packer
4424
+ * easier use with <https://github.com/pmq20/ruby-packer>