aspera-cli 4.14.0 → 4.15.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (90) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/CHANGELOG.md +54 -3
  4. data/CONTRIBUTING.md +7 -7
  5. data/README.md +1457 -880
  6. data/bin/ascli +18 -9
  7. data/bin/asession +12 -14
  8. data/examples/proxy.pac +1 -1
  9. data/lib/aspera/aoc.rb +198 -127
  10. data/lib/aspera/ascmd.rb +24 -14
  11. data/lib/aspera/cli/basic_auth_plugin.rb +9 -6
  12. data/lib/aspera/cli/error.rb +17 -0
  13. data/lib/aspera/cli/extended_value.rb +47 -12
  14. data/lib/aspera/cli/formatter.rb +260 -171
  15. data/lib/aspera/cli/hints.rb +80 -0
  16. data/lib/aspera/cli/main.rb +101 -147
  17. data/lib/aspera/cli/manager.rb +160 -124
  18. data/lib/aspera/cli/plugin.rb +70 -59
  19. data/lib/aspera/cli/plugins/alee.rb +0 -1
  20. data/lib/aspera/cli/plugins/aoc.rb +239 -273
  21. data/lib/aspera/cli/plugins/ats.rb +8 -5
  22. data/lib/aspera/cli/plugins/bss.rb +2 -2
  23. data/lib/aspera/cli/plugins/config.rb +516 -375
  24. data/lib/aspera/cli/plugins/console.rb +40 -0
  25. data/lib/aspera/cli/plugins/cos.rb +4 -5
  26. data/lib/aspera/cli/plugins/faspex.rb +99 -84
  27. data/lib/aspera/cli/plugins/faspex5.rb +179 -148
  28. data/lib/aspera/cli/plugins/node.rb +219 -153
  29. data/lib/aspera/cli/plugins/orchestrator.rb +52 -17
  30. data/lib/aspera/cli/plugins/preview.rb +46 -32
  31. data/lib/aspera/cli/plugins/server.rb +57 -17
  32. data/lib/aspera/cli/plugins/shares.rb +34 -12
  33. data/lib/aspera/cli/sync_actions.rb +68 -0
  34. data/lib/aspera/cli/transfer_agent.rb +45 -55
  35. data/lib/aspera/cli/transfer_progress.rb +74 -0
  36. data/lib/aspera/cli/version.rb +1 -1
  37. data/lib/aspera/colors.rb +3 -1
  38. data/lib/aspera/command_line_builder.rb +14 -11
  39. data/lib/aspera/cos_node.rb +3 -2
  40. data/lib/aspera/environment.rb +17 -6
  41. data/lib/aspera/fasp/agent_aspera.rb +126 -0
  42. data/lib/aspera/fasp/agent_base.rb +31 -77
  43. data/lib/aspera/fasp/agent_connect.rb +21 -22
  44. data/lib/aspera/fasp/agent_direct.rb +88 -102
  45. data/lib/aspera/fasp/agent_httpgw.rb +196 -192
  46. data/lib/aspera/fasp/agent_node.rb +41 -34
  47. data/lib/aspera/fasp/agent_trsdk.rb +75 -34
  48. data/lib/aspera/fasp/error_info.rb +2 -2
  49. data/lib/aspera/fasp/faux_file.rb +52 -0
  50. data/lib/aspera/fasp/installation.rb +43 -184
  51. data/lib/aspera/fasp/management.rb +244 -0
  52. data/lib/aspera/fasp/parameters.rb +59 -26
  53. data/lib/aspera/fasp/parameters.yaml +75 -8
  54. data/lib/aspera/fasp/products.rb +162 -0
  55. data/lib/aspera/fasp/transfer_spec.rb +1 -1
  56. data/lib/aspera/fasp/uri.rb +4 -4
  57. data/lib/aspera/faspex_gw.rb +2 -2
  58. data/lib/aspera/faspex_postproc.rb +2 -2
  59. data/lib/aspera/hash_ext.rb +2 -2
  60. data/lib/aspera/json_rpc.rb +49 -0
  61. data/lib/aspera/line_logger.rb +23 -0
  62. data/lib/aspera/log.rb +57 -16
  63. data/lib/aspera/node.rb +97 -14
  64. data/lib/aspera/oauth.rb +36 -18
  65. data/lib/aspera/open_application.rb +4 -4
  66. data/lib/aspera/persistency_folder.rb +2 -2
  67. data/lib/aspera/preview/file_types.rb +4 -2
  68. data/lib/aspera/preview/generator.rb +22 -35
  69. data/lib/aspera/preview/options.rb +2 -0
  70. data/lib/aspera/preview/terminal.rb +24 -13
  71. data/lib/aspera/preview/utils.rb +19 -26
  72. data/lib/aspera/rest.rb +103 -72
  73. data/lib/aspera/rest_call_error.rb +1 -1
  74. data/lib/aspera/rest_error_analyzer.rb +15 -14
  75. data/lib/aspera/rest_errors_aspera.rb +37 -34
  76. data/lib/aspera/secret_hider.rb +14 -16
  77. data/lib/aspera/ssh.rb +4 -1
  78. data/lib/aspera/sync.rb +128 -122
  79. data/lib/aspera/temp_file_manager.rb +10 -3
  80. data/lib/aspera/web_auth.rb +10 -7
  81. data/lib/aspera/web_server_simple.rb +9 -4
  82. data.tar.gz.sig +0 -0
  83. metadata +33 -15
  84. metadata.gz.sig +0 -0
  85. data/lib/aspera/cli/listener/line_dump.rb +0 -19
  86. data/lib/aspera/cli/listener/logger.rb +0 -22
  87. data/lib/aspera/cli/listener/progress.rb +0 -50
  88. data/lib/aspera/cli/listener/progress_multi.rb +0 -84
  89. data/lib/aspera/cli/plugins/sync.rb +0 -44
  90. data/lib/aspera/fasp/listener.rb +0 -13
data/README.md CHANGED
@@ -1,16 +1,20 @@
1
1
  # Command Line Interface for IBM Aspera products
2
2
  <!-- markdownlint-disable MD033 MD003 MD053 -->
3
- <!-- cspell:ignore devkit zcvf zxvf noded secondfile filesize sedemo eudemo webmail csum eascp loglevel cronfile magick keepalive inotify eastus bluemix trev sshfp struct genkey passout ibmaspera unpermitted schtasks taskschd dascli -->
3
+ <!-- cspell:ignore Serban Antipolis -->
4
4
 
5
5
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
6
6
 
7
- ##
7
+ [![Gem Version](https://badge.fury.io/rb/aspera-cli.svg)](https://badge.fury.io/rb/aspera-cli)
8
+ [![unit tests](https://github.com/IBM/aspera-cli/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/IBM/aspera-cli/actions)
9
+ [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5861/badge)](https://bestpractices.coreinfrastructure.org/projects/5861)
10
+
11
+ ## Introduction
8
12
 
9
- Version : 4.14.0
13
+ Version : 4.15.0
10
14
 
11
15
  Laurent/2016-2023
12
16
 
13
- This gem provides the `ascli` Command Line Interface to IBM Aspera software.
17
+ This gem provides the `ascli` CLI (Command Line Interface) to IBM Aspera software.
14
18
 
15
19
  `ascli` is a also great tool to learn Aspera APIs.
16
20
 
@@ -27,15 +31,13 @@ Minimum required Ruby version: >= 2.6.
27
31
 
28
32
  Release notes: see [CHANGELOG.md](CHANGELOG.md)
29
33
 
30
- [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5861/badge)](https://bestpractices.coreinfrastructure.org/projects/5861)
34
+ A PDF version of this documentation is available here: [docs/Manual.pdf](docs/Manual.pdf).
31
35
 
32
- ## BUGS, FEATURES, CONTRIBUTION
36
+ ### BUGS, FEATURES, CONTRIBUTION
33
37
 
34
38
  Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
35
39
 
36
- One can also [create one's own plugin](#createownplugin).
37
-
38
- ## <a id="when_to_use"></a>When to use and when not to use
40
+ ### <a id="when_to_use"></a>When to use and when not to use
39
41
 
40
42
  `ascli` is designed to be used as a command line tool to:
41
43
 
@@ -63,7 +65,7 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
63
65
 
64
66
  For scripting and ad'hoc command line operations, `ascli` is perfect.
65
67
 
66
- ## Notations, Shell, Examples
68
+ ### Notations, Shell, Examples
67
69
 
68
70
  Command line operations examples are shown using a shell such: `bash` or `zsh`.
69
71
 
@@ -92,7 +94,7 @@ ascli --version
92
94
  ```
93
95
 
94
96
  ```bash
95
- 4.14.0
97
+ 4.15.0
96
98
  ```
97
99
 
98
100
  ### First use
@@ -112,14 +114,14 @@ ascli server browse /
112
114
  ```
113
115
 
114
116
  ```output
115
- +------------+-----------+-----------+-------+---------------------------+-----------------------+
116
- | zmode | zuid | zgid | size | mtime | name |
117
- +------------+-----------+-----------+-------+---------------------------+-----------------------+
118
- | drwxr-xr-x | asperaweb | asperaweb | 90112 | 2023-04-05 15:31:21 +0200 | Upload |
119
- | dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:16 +0200 | aspera-test-dir-large |
120
- | dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-small |
121
- | dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-tiny |
122
- +------------+-----------+-----------+-------+---------------------------+-----------------------+
117
+ +------------+--------+-----------+-------+---------------------------+-----------------------+
118
+ | zmode | zuid | zgid | size | mtime | name |
119
+ +------------+--------+-----------+-------+---------------------------+-----------------------+
120
+ | drwxr-xr-x | aspera | asperaweb | 90112 | 2023-04-05 15:31:21 +0200 | Upload |
121
+ | dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:16 +0200 | aspera-test-dir-large |
122
+ | dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-small |
123
+ | dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-tiny |
124
+ +------------+--------+-----------+-------+---------------------------+-----------------------+
123
125
  ```
124
126
 
125
127
  If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [option preset](#lprt) for the server's authentication options. The following example will:
@@ -168,7 +170,7 @@ ascli server download /aspera-test-dir-large/200MB
168
170
  ```
169
171
 
170
172
  ```output
171
- Time: 00:00:02 =========================================================== 100% 100 Mbps Time: 00:00:00
173
+ Time: 00:00:02 ============================================= 100% 100 Mbps Time: 00:00:00
172
174
  complete
173
175
  ```
174
176
 
@@ -198,7 +200,7 @@ An internet connection is required for the installation. If you don't have inter
198
200
 
199
201
  ### Container
200
202
 
201
- The container image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
203
+ The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
202
204
  The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
203
205
  To use the container, ensure that you have `podman` (or `docker`) installed.
204
206
 
@@ -206,7 +208,7 @@ To use the container, ensure that you have `podman` (or `docker`) installed.
206
208
  podman --version
207
209
  ```
208
210
 
209
- #### Container: quick start
211
+ #### Container: Quick start
210
212
 
211
213
  **Wanna start quickly ?** With an interactive shell ? Execute this:
212
214
 
@@ -254,7 +256,7 @@ ascli -v
254
256
  ```
255
257
 
256
258
  ```text
257
- 4.14.0
259
+ 4.15.0
258
260
  ```
259
261
 
260
262
  In order to keep persistency of configuration on the host,
@@ -306,12 +308,12 @@ A convenience sample script is also provided: download the script [`dascli`](../
306
308
 
307
309
  Some environment variables can be set for this script to adapt its behavior:
308
310
 
309
- | env var | description | default | example |
310
- |--------------|------------------------------------|--------------------------|--------------------------|
311
- | ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
312
- | docker_args | additional options to `podman` | &lt;empty&gt; | `--volume /Users:/Users` |
313
- | image | container image name | martinlaurent/ascli | |
314
- | version | container image version | latest | `4.8.0.pre` |
311
+ | env var | description | default | example |
312
+ |----------------|------------------------------------|--------------------------|--------------------------|
313
+ | `ASCLI_HOME` | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
314
+ | `docker_args` | additional options to `podman` | &lt;empty&gt; | `--volume /Users:/Users` |
315
+ | `image` | container image name | `martinlaurent/ascli` | |
316
+ | `version` | container image version | latest | `4.8.0.pre` |
315
317
 
316
318
  The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
317
319
  (value expected in the container).
@@ -337,7 +339,7 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
337
339
 
338
340
  > **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
339
341
  >
340
- > **Note:** Do not use too many volumes, as the AUFS limits the number.
342
+ > **Note:** Do not use too many volumes, as the legacy `aufs` limits the number. (anyway, prefer to use `overlay2`)
341
343
 
342
344
  #### Container: Offline installation
343
345
 
@@ -364,7 +366,7 @@ If one wants to change the content, it is possible to tell `ascp` to use another
364
366
  echo '<CONF/>' > $HOME/.aspera/ascli/aspera.conf
365
367
  ```
366
368
 
367
- Then, tell `ascp` to use that other conf file:
369
+ Then, tell `ascp` to use that other configuration file:
368
370
 
369
371
  ```bash
370
372
  --transfer-info=@json:'{"ascp_args":["-f","/home/cliuser/.aspera/ascli/aspera.conf"]}'
@@ -376,7 +378,7 @@ Singularity is another type of use of container.
376
378
 
377
379
  On Linux install:
378
380
 
379
- ```console
381
+ ```bash
380
382
  dnf install singularity-ce
381
383
  ```
382
384
 
@@ -386,13 +388,13 @@ Build an image like this:
386
388
  singularity build ascli.sif docker://martinlaurent/ascli
387
389
  ```
388
390
 
389
- The use like this:
391
+ Then, start `ascli` like this:
390
392
 
391
393
  ```bash
392
394
  singularity run ascli.sif
393
395
  ```
394
396
 
395
- Or get a shell with access to the tool like this:
397
+ Or get a shell with access to `ascli` like this:
396
398
 
397
399
  ```bash
398
400
  singularity shell ascli.sif
@@ -402,13 +404,13 @@ singularity shell ascli.sif
402
404
 
403
405
  Use this method to install on the native host.
404
406
 
405
- A Ruby interpreter is required to run the tool or to use the gem and tool.
407
+ A Ruby interpreter is required to run `ascli` or to use the gem and tool.
406
408
 
407
409
  Required Ruby version: >= 2.6.
408
410
 
409
411
  > **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
410
412
 
411
- *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
413
+ **Ruby can be installed using any method** : rpm, yum, dnf, rvm, brew, Windows installer, ... .
412
414
 
413
415
  In priority, refer to the official Ruby documentation:
414
416
 
@@ -421,7 +423,7 @@ The recommended installation method is `rvm` for Unix-like systems (Linux, AIX,
421
423
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
422
424
  If you have a simpler better way to install Ruby : use it !
423
425
 
424
- #### Generic: RVM: single user installation (not root)
426
+ #### Generic: RVM: Single user installation (not root)
425
427
 
426
428
  Use this method which provides more flexibility.
427
429
 
@@ -455,7 +457,7 @@ rvm install 3.2.2
455
457
 
456
458
  Ruby is now installed for the user, go to [Gem installation](#the_gem).
457
459
 
458
- #### Generic: RVM: global installation (as root)
460
+ #### Generic: RVM: Global installation (as root)
459
461
 
460
462
  Follow the same method as single user install, but execute as "root".
461
463
 
@@ -467,7 +469,12 @@ curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
467
469
  ```
468
470
 
469
471
  As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
470
- If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
472
+ If so, one can rename the login script:
473
+
474
+ ```bash
475
+ mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok
476
+ ```
477
+
471
478
  To activate Ruby (and ascli) later, source it:
472
479
 
473
480
  ```bash
@@ -480,16 +487,69 @@ rvm version
480
487
 
481
488
  #### Windows: Installer
482
489
 
483
- Install Latest stable Ruby:
490
+ Manual installation:
484
491
 
485
492
  - Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) &rarr; **Downloads**.
486
- - Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
493
+ - Download the latest Ruby installer **"with devkit"**. (`Msys2` is needed to install some native extensions, such as `grpc`)
487
494
  - Execute the installer which installs by default in: `C:\RubyVV-x64` (VV is the version number)
488
- - At the end of the installation procedure, the Msys2 installer is automatically executed, select option 3 (msys and mingw)
495
+ - At the end of the installation procedure, the `Msys2` installer is automatically executed, select option 3 (`Msys2` and mingw)
496
+ - then install the aspera-cli gem and Aspera SDK (see next sections)
497
+
498
+ Automated installation, with internet access:
499
+
500
+ The ruby installer supports silent installation, to see the options, execute it with `/help`, or refer to the [Ruby Installer FAQ](https://github.com/oneclick/rubyinstaller2/wiki/FAQ)
501
+
502
+ Download the ruby installer executable from <https://rubyinstaller.org/downloads/> and then install:
503
+
504
+ ```bat
505
+ rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
506
+ ```
507
+
508
+ Installation without network:
509
+
510
+ It is essentially the same procedure, but instead of retrieving files from internet, one copies the files from a machine with internet access, and then install from those archives:
511
+
512
+ - Download the `exe` ruby installer from <https://rubyinstaller.org/downloads/>
513
+ - Create an archive with necessary gems: <https://help.rubygems.org/kb/rubygems/installing-gems-with-no-network>
514
+
515
+ ```bat
516
+ gem install aspera-cli -N -i my_gems
517
+ ```
518
+
519
+ Zip the files `*.gem` from folder `repo/my_gems`
520
+
521
+ - Download the SDK from: <https://ibm.biz/aspera_sdk>
522
+
523
+ Create a Zip with all those files, and transfer to the target system.
524
+
525
+ Then, on the target system:
526
+
527
+ - Unzip the archive
528
+ - Execute the installer:
529
+
530
+ ```bat
531
+ rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
532
+ ```
533
+
534
+ - Install the gems:
535
+
536
+ ```bat
537
+ gem install --force --local *.gem
538
+ ```
539
+
540
+ - install the SDK
541
+
542
+ ```bash
543
+ ascli conf ascp install --sdk-url=file:///sdk.zip
544
+ ```
489
545
 
490
- #### macOS: pre-installed or `brew`
546
+ > **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
491
547
 
492
- macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
548
+ #### macOS: Pre-installed or `brew`
549
+
550
+ macOS 10.13+ (High Sierra) comes with a recent Ruby.
551
+ So you can use it directly.
552
+ You will need to install aspera-cli using `sudo` :
493
553
 
494
554
  ```bash
495
555
  sudo gem install aspera-cli
@@ -501,7 +561,7 @@ Alternatively, if you use [Homebrew](https://brew.sh/) already you can install R
501
561
  brew install ruby
502
562
  ```
503
563
 
504
- #### Linux: package
564
+ #### Linux: Package
505
565
 
506
566
  If your Linux distribution provides a standard Ruby package, you can use it provided that the version supported.
507
567
 
@@ -521,9 +581,9 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
521
581
 
522
582
  - Install packages needed to build native gems:
523
583
 
524
- ```bash
525
- dnf install -y make automake gcc gcc-c++ kernel-devel
526
- ```
584
+ ```bash
585
+ dnf install -y make automake gcc gcc-c++ kernel-devel
586
+ ```
527
587
 
528
588
  - Enable the Ruby version you want:
529
589
 
@@ -545,7 +605,7 @@ apt install -y ruby ruby-dev rubygems ruby-json
545
605
  One can cleanup the whole yum-installed Ruby environment like this to uninstall:
546
606
 
547
607
  ```bash
548
- gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
608
+ gem uninstall -axI $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
549
609
  ```
550
610
 
551
611
  #### Other Unixes (AIX)
@@ -582,7 +642,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
582
642
 
583
643
  <https://www.jruby.org/download>
584
644
 
585
- > **Note:** Using jruby the startup time is longer than the native Ruby, but the transfer speed is not impacted (executed by `ascp` binary).
645
+ > **Note:** Using `jruby`, the startup time is longer than the native Ruby, but transfer speed is not impacted (executed by `ascp` binary).
586
646
 
587
647
  ### <a id="the_gem"></a>`aspera-cli` gem
588
648
 
@@ -624,11 +684,11 @@ ascli conf ascp install
624
684
  If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
625
685
 
626
686
  ```bash
627
- curl -Lso SDK.zip https://ibm.biz/aspera_sdk
687
+ curl -Lso sdk.zip https://ibm.biz/aspera_sdk
628
688
  ```
629
689
 
630
690
  ```bash
631
- ascli conf ascp install --sdk-url=file:///SDK.zip
691
+ ascli conf ascp install --sdk-url=file:///sdk.zip
632
692
  ```
633
693
 
634
694
  The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
@@ -673,7 +733,7 @@ ascli conf --show-config --fields=sdk_url
673
733
  - Download the SDK archive from that URL.
674
734
 
675
735
  ```bash
676
- curl -Lso SDK.zip https://ibm.biz/aspera_sdk
736
+ curl -Lso sdk.zip https://ibm.biz/aspera_sdk
677
737
  ```
678
738
 
679
739
  - Transfer those 2 files to the target system
@@ -687,7 +747,7 @@ tar zxvf rvm-ascli.tgz
687
747
 
688
748
  source ~/.rvm/scripts/rvm
689
749
 
690
- ascli conf ascp install --sdk-url=file:///SDK.zip
750
+ ascli conf ascp install --sdk-url=file:///sdk.zip
691
751
  ```
692
752
 
693
753
  - Add those lines to shell init (`.profile`)
@@ -717,10 +777,10 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
717
777
  - Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
718
778
  - Allows transfers from products to products, essentially at node level (using the node transfer agent)
719
779
  - Supports FaspStream creation (using Node API)
720
- - Supports Watchfolder creation (using Node API)
780
+ - Supports "Watchfolder" creation (using Node API)
721
781
  - Additional command plugins can be written by the user
722
782
  - Supports download of faspex and Aspera on Cloud "external" links
723
- - Supports "legacy" ssh based FASP transfers and remote commands (ascmd)
783
+ - Supports "legacy" ssh based FASP transfers and remote commands (`ascmd`)
724
784
 
725
785
  Basic usage is displayed by executing:
726
786
 
@@ -763,11 +823,13 @@ So special character handling (quotes, spaces, env vars, ...) is handled by the
763
823
 
764
824
  #### Shell parsing for Windows
765
825
 
766
- MS Windows command line parsing is not easy: It is not hasndled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the application (here Ruby).
826
+ MS Windows command line parsing is not handled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the executable.
827
+ Typically, Windows executables use the [microsoft library for this parsing](https://learn.microsoft.com/en-us/cpp/cpp/main-function-command-line-args).
767
828
 
768
- As far as `ascli` is concerned: it is close to a Linux shell parsing.
829
+ As far as `ascli` is concerned: the executable is Ruby.
830
+ It has its own parsing algorithm, close to a Linux shell parsing.
769
831
 
770
- Thanksfully, `ascli` provides a command to check the value of an argument after parsing: `config echo`.
832
+ Thankfully, `ascli` provides a command to check the value of an argument after parsing: `config echo`.
771
833
  One can also run `ascli` with option `--log-level=debug` to display the command line after parsing.
772
834
 
773
835
  The following examples give the same result on Windows:
@@ -790,9 +852,9 @@ The following examples give the same result on Windows:
790
852
  conf echo @json:"{\"url\":\"https://...\"}"
791
853
  ```
792
854
 
793
- On Windows, `cmd.exe` is typically used to start .
855
+ More details: on Windows, `cmd.exe` is typically used to start .
794
856
  `cmd.exe` handles some special characters: `^"<>|%&`.
795
- Basically it handles I/O redirections (`<>|`), shell variables (`%`), multiple commands (`&`) and handles those special characters from the command line.
857
+ Basically it handles I/O redirection (`<>|`), shell variables (`%`), multiple commands (`&`) and handles those special characters from the command line.
796
858
  Eventually, all those special characters are removed from the command line unless escaped with `^` or `"`.
797
859
  `"` are kept and given to the program.
798
860
 
@@ -809,20 +871,21 @@ Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
809
871
 
810
872
  - space characters: split arguments (space, tab, newline)
811
873
  - backslash: `\` escape single special character
812
- - globbing characters: `*?[]{}` for file globbing
874
+ - globing characters: `*?[]{}` for file globing
813
875
  - double quotes: `"`
814
876
  - single quotes: `'`
815
877
 
816
878
  #### Extended Values (JSON, Ruby, ...)
817
879
 
818
- Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple strings, but a complex structure (Hash, Array).
819
- Typically, the `@json:` modifier is used, it expects a JSON string. JSON itself has some special syntax: for example `"` is used to denote strings.
880
+ Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple `String`, but a complex structure (`Hash`, `Array`).
881
+ Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
882
+ JSON itself has some special syntax: for example `"` is used to enclose a `String`.
820
883
 
821
884
  #### Testing Extended Values
822
885
 
823
886
  In case of doubt of argument values after parsing, one can test using command `config echo`. `config echo` takes exactly **one** argument which can use the [Extended Value](#extended) syntax. Unprocessed command line arguments are shown in the error message.
824
887
 
825
- Example: The shell parses three arguments (as strings: `1`, `2` and `3`), so the additional two arguments are not processed by the `echo` command.
888
+ Example: The shell parses three arguments (as `String`: `1`, `2` and `3`), so the additional two arguments are not processed by the `echo` command.
826
889
 
827
890
  ```bash
828
891
  ascli conf echo 1 2 3
@@ -993,27 +1056,29 @@ ascli conf echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
993
1056
  {"title":"Test \" ' & \\"}
994
1057
  ```
995
1058
 
996
- ### Commands, Options, Positional Values
1059
+ ### Commands, Options, Positional Arguments
997
1060
 
998
- Command line arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
1061
+ Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
1062
+ The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#parsing).
999
1063
 
1000
1064
  `ascli` considers three types of command line arguments:
1001
1065
 
1002
1066
  - Commands
1003
1067
  - Options
1004
- - Positional Values
1068
+ - Positional Arguments
1069
+
1070
+ For example:
1005
1071
 
1006
1072
  ```bash
1007
1073
  ascli command subcommand --option-name=VAL1 VAL2
1008
1074
  ```
1009
1075
 
1010
- - executes *command*: `command subcommand`
1011
- - with one *option*: `option_name`and its *value*: `VAL1`
1012
- - the command has one additional mandatory *argument*: `VAL2`
1076
+ - executes **command**: `command subcommand`
1077
+ - with one **option**: `option_name` and its **value**: `VAL1`
1078
+ - the command has one additional **positional argument**: `VAL2`
1013
1079
 
1014
- When the value of a command, option or argument is constrained by a fixed list of values.
1015
- It is possible to use the first letters of the value only, provided that it uniquely identifies a value.
1016
- For example `ascli conf ov` is the same as `ascli config overview`.
1080
+ If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
1081
+ For example `ascli conf pre ov` is the same as `ascli config preset overview`.
1017
1082
 
1018
1083
  The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
1019
1084
 
@@ -1032,10 +1097,12 @@ ascli conf ascp info
1032
1097
  - `ascp` is the second level command, and is also the name of the component (singleton)
1033
1098
  - `info` is the third level command, and is also the action to be performed
1034
1099
 
1035
- Typically, commands are located at the beginning of the command line.
1100
+ Typically, commands are located at the **beginning** of the command line.
1036
1101
  Order is significant.
1037
1102
  The provided command must match one of the supported commands in the given context.
1038
- If a wrong , or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
1103
+ If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
1104
+
1105
+ Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
1039
1106
 
1040
1107
  #### Options
1041
1108
 
@@ -1049,16 +1116,16 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
1049
1116
  Exceptions:
1050
1117
 
1051
1118
  - some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
1052
- - some options (flags) don't take a value, e.g. `-r`
1119
+ - some options (flags) don't take a value, e.g. `-N`
1053
1120
  - the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
1054
1121
 
1055
- ```bash
1056
- ascli config echo -- --sample
1057
- ```
1122
+ ```bash
1123
+ ascli config echo -- --sample
1124
+ ```
1058
1125
 
1059
- ```bash
1060
- "--sample"
1061
- ```
1126
+ ```bash
1127
+ "--sample"
1128
+ ```
1062
1129
 
1063
1130
  > **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
1064
1131
 
@@ -1066,14 +1133,14 @@ Options may have an (hardcoded) default value.
1066
1133
 
1067
1134
  Options can be placed anywhere on command line and evaluated in order.
1068
1135
 
1069
- Options are typically either:
1136
+ Options are typically:
1070
1137
 
1071
- - optional : typically to change the default behavior
1072
- - mandatory : typically, connection information are options that are mandatory (so they can be placed in a config file)
1138
+ - optional : to change the default behavior
1139
+ - mandatory : so they can be placed in a config file, for example: connection information
1073
1140
 
1074
- The value for *any* options can come from the following locations (in this order, last value evaluated overrides previous value):
1141
+ The value for **any** options can come from the following locations (in this order, last value evaluated overrides previous value):
1075
1142
 
1076
- - [Configuration file](#configfile).
1143
+ - [Configuration file](#configfile)
1077
1144
  - Environment variable
1078
1145
  - Command line
1079
1146
 
@@ -1081,27 +1148,37 @@ Environment variable starting with prefix: ASCLI_ are taken as option values, e.
1081
1148
 
1082
1149
  Options values can be displayed for a given command by providing the `--show-config` option: `ascli node --show-config`
1083
1150
 
1084
- #### Positional Values
1151
+ Parameters are typically designed as options if:
1152
+
1153
+ - it is a mandatory parameters that would benefit from being set in a config file or environment variable
1154
+ - it is optional
1085
1155
 
1086
- Positional Values are typically mandatory values for a command, such as entity creation data.
1156
+ #### Positional Arguments
1087
1157
 
1088
- If a Positional Values begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
1158
+ Positional Arguments are typically mandatory values for a command, such as entity creation data.
1089
1159
 
1090
- The advantages of using a positional value instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
1160
+ It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
1161
+
1162
+ The advantages of using a positional argument instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
1091
1163
 
1092
1164
  The disadvantage is that it is not possible to define a default value in a config file or environment variable like for options.
1093
1165
  Nevertheless, [Extended Values](#extended) syntax is supported, so it is possible to retrieve a value from the config file or environment variable.
1094
1166
 
1167
+ If a Positional Arguments begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
1168
+
1169
+ Very few positional arguments are optional, they are located at the end of the command line.
1170
+
1095
1171
  ### Interactive Input
1096
1172
 
1097
- Some options and parameters are mandatory and other optional. By default, the tool will ask for missing mandatory options or parameters for interactive execution.
1173
+ Some options and parameters are mandatory and other optional.
1174
+ By default, `ascli` will ask for missing mandatory options or parameters for interactive execution.
1098
1175
 
1099
1176
  The behavior can be controlled with:
1100
1177
 
1101
- - --interactive=&lt;yes|no&gt; (default=yes if STDIN is a terminal, else no)
1178
+ - `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
1102
1179
  - yes : missing mandatory parameters/options are asked to the user
1103
1180
  - no : missing mandatory parameters/options raise an error message
1104
- - --ask-options=&lt;yes|no&gt; (default=no)
1181
+ - `--ask-options=<yes|no>` (default=no)
1105
1182
  - optional parameters/options are asked to user
1106
1183
 
1107
1184
  ### Output
@@ -1126,8 +1203,8 @@ By default, result of type single_object and object_list are displayed using for
1126
1203
  The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
1127
1204
 
1128
1205
  In a table format, when displaying "objects" (single, or list), by default, sub object are
1129
- flattened (option `flat_hash`). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name.
1130
- Setting `flat_hash` to `false` will only display one field: "user" and value is the sub hash table.
1206
+ flattened (option `flat_hash`). So, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
1207
+ Setting `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
1131
1208
  When in flatten mode, it is possible to filter fields by "dotted" field name.
1132
1209
 
1133
1210
  Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
@@ -1143,32 +1220,14 @@ The style of output can be set using the `format` parameter, supporting:
1143
1220
  - `yaml` : YAML
1144
1221
  - `csv` : Comma Separated Values
1145
1222
 
1146
- #### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
1147
-
1148
- Table output can be filtered using the `select` parameter. Example:
1149
-
1150
- ```bash
1151
- ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
1152
- ```
1153
-
1154
- ```output
1155
- +-------------------------------+----------------------------------+-----------+
1156
- | name | email | ats_admin |
1157
- +-------------------------------+----------------------------------+-----------+
1158
- | John Curtis | john@example.com | true |
1159
- | Laurent Martin | laurent@example.com | true |
1160
- +-------------------------------+----------------------------------+-----------+
1161
- ```
1162
-
1163
- > **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
1164
-
1165
- #### entity identifier
1223
+ #### Entity identifier
1166
1224
 
1167
- When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command: e.g. `ascli aoc admin res user show 1234` where `1234` is the user identifier.
1225
+ When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command: e.g. `ascli aoc admin res user show 1234` where `1234` is the user's identifier.
1168
1226
 
1169
1227
  > **Note:** The legacy option `id` is deprecated: `--id=1234` as it does not provide the possibility to have sub-entities.
1170
1228
 
1171
- Only some commands provide the following capability: If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin res user show %name:john` where `john` is the user name.
1229
+ Only some commands provide the following capability:
1230
+ If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin res user show %name:john` where `john` is the user name.
1172
1231
 
1173
1232
  Syntax: `%<field>:<value>`
1174
1233
 
@@ -1189,67 +1248,122 @@ The option `display` controls the level of output:
1189
1248
  By default, secrets are removed from output: option `show_secrets` defaults to `no`, unless `display` is `data`, to allows piping results.
1190
1249
  To hide secrets from output, set option `show_secrets` to `no`.
1191
1250
 
1192
- #### Selection of output object properties
1251
+ #### Option: `fields`: Selection of output object properties
1252
+
1253
+ By default, a table output will display one line per entry, and columns for each properties.
1254
+ Depending on the command, columns may include by default all properties, or only some selected properties.
1255
+ It is possible to define specific columns to be displayed, by setting the `fields` option.
1256
+
1257
+ The `fields` option can be either a comma separated list, or an extended value array.
1258
+
1259
+ Elements of the list can be:
1260
+
1261
+ - `DEF` : default display of columns (that's the default, when not set)
1262
+ - `ALL` : all columns available
1263
+ - -property : remove property from the current list
1264
+ - property : add property to the current list
1265
+ - a ruby `RegEx` : using `@ruby:'/.../'`
1266
+
1267
+ Examples:
1268
+
1269
+ - `a,b,c` : the list of attributes specified as a comma separated list
1270
+ - `@json:'["a","b","c"]'` : Array extended value: same as above
1271
+ - `DEF,-a,b` : default property list, remove `a` and add `b`
1272
+ - `@ruby:'/^server/'` : Display all properties whose name begin with `server`
1273
+
1274
+ #### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
1275
+
1276
+ Table output can be filtered using option `select`.
1277
+ This parameter is either a `Hash` or `Proc`.
1278
+ The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
1279
+
1280
+ Example:
1281
+
1282
+ ```bash
1283
+ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
1284
+ ```
1285
+
1286
+ ```output
1287
+ +-------------------------------+----------------------------------+-----------+
1288
+ | name | email | ats_admin |
1289
+ +-------------------------------+----------------------------------+-----------+
1290
+ | John Curtis | john@example.com | true |
1291
+ | Laurent Martin | laurent@example.com | true |
1292
+ +-------------------------------+----------------------------------+-----------+
1293
+ ```
1294
+
1295
+ > **Note:** `select` filters elements from the result of command, while the `query` parameters gives filtering parameters to the API when listing elements.
1193
1296
 
1194
- By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
1297
+ In above example, the same result is obtained with option:
1195
1298
 
1196
- - DEF : default display of columns (that's the default, when not set)
1197
- - ALL : all columns available
1198
- - a,b,c : the list of attributes specified by the comma separated list
1199
- - Array extended value: for instance, @json:'["a","b","c"]' same as above
1200
- - +a,b,c : add selected properties to the default selection.
1201
- - -a,b,c : remove selected properties from the default selection.
1299
+ ```bash
1300
+ --select=@ruby:'->(i){i["ats_admin"]}'
1301
+ ```
1202
1302
 
1203
1303
  ### <a id="extended"></a>Extended Value Syntax
1204
1304
 
1205
- Some options and arguments are specified by a simple string.
1206
- But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
1305
+ Most options and arguments are specified by a simple string (e.g. username or url).
1306
+ Sometime it is convenient to read a value from a file: for example read the PEM value of a private key, or a list of files.
1307
+ Some options expect a more complex value such as `Hash` or `Array`.
1207
1308
 
1208
- The extended value syntax is:
1309
+ The **Extended Value** Syntax allows to specify such values and even read values from other sources than the command line itself.
1310
+
1311
+ The syntax is:
1209
1312
 
1210
1313
  ```bash
1211
- <0 or more decoders><nothing or some text value>
1314
+ <0 or more decoders><some text value or nothing>
1212
1315
  ```
1213
1316
 
1214
- Decoders act like a function of value on right hand side.
1215
- Decoders are recognized by the prefix: `@` and suffix `:`
1317
+ Decoders act like a function with its parameter on right hand side and are recognized by the prefix: `@` and suffix `:`
1216
1318
 
1217
1319
  The following decoders are supported:
1218
1320
 
1219
1321
  | decoder | parameter | returns | description |
1220
1322
  |---------|-----------|---------|-------------|
1221
1323
  | base64 | String | String | decode a base64 encoded string
1222
- | csvt | String | Array | decode a titled CSV value
1223
- | env | String | String | read from a named env var name, e.g.--password=@env:MYPASSVAR
1224
- | file | String | String | read value from specified file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey` |
1225
- | incps | Hash | Hash | include values of presets specified by key `incps` in input hash
1226
- | json | String | any | decode JSON values (convenient to provide complex structures)
1227
- | lines | String | Array | split a string in multiple lines and return an array
1228
- | list | String | Array | split a string in multiple items taking first character as separator and return an array
1229
- | path | String | String | performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml` |
1324
+ | `csvt` | String | Array | decode a titled CSV value
1325
+ | `env` | String | String | read from a named env var name, e.g. `--password=@env:MYPASSVAR`
1326
+ | `file` | String | String | read value from specified file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
1327
+ | `json` | String | Any | decode JSON values (convenient to provide complex structures)
1328
+ | `lines` | String | Array | split a string in multiple lines and return an array
1329
+ | `list` | String | Array | split a string in multiple items taking first character as separator and return an array
1330
+ | `none` | None | Nil | A null value
1331
+ | `path` | String | String | performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
1230
1332
  | preset | String | Hash | get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
1231
- | ruby | String | any | execute specified Ruby code
1333
+ | extend | String | String | evaluates embedded extended value syntax in string
1334
+ | `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`)
1335
+ | `ruby` | String | Any | execute specified Ruby code
1232
1336
  | secret | None | String | Ask password interactively (hides input)
1233
- | stdin | None | String | read from stdin (no value on right)
1234
- | uri | String | String | read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
1235
- | val | String | String | prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
1236
- | zlib | String | String | un-compress data
1337
+ | `stdin` | None | String | read from stdin (no value on right)
1338
+ | `uri` | String | String | read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac`
1339
+ | `val` | String | String | prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`.
1340
+ | `yaml` | String | Any | decode YAML
1341
+ | `zlib` | String | String | un-compress data
1237
1342
 
1238
1343
  To display the result of an extended value, use the `config echo` command.
1239
1344
 
1345
+ The `extend` decoder is useful to evaluate embedded extended value syntax in a string.
1346
+ It expects a `@` to close the embedded extended value syntax.
1347
+
1348
+ Example: Create a `Hash` value with the convenient `@json:` decoder:
1349
+
1350
+ ```bash
1351
+ ascli config echo @json:'{"key1":"value1","key2":"value2"}'
1352
+ ```
1353
+
1240
1354
  Example: read the content of the specified file, then, base64 decode, then unzip:
1241
1355
 
1242
1356
  ```bash
1243
1357
  ascli config echo @zlib:@base64:@file:myfile.dat
1244
1358
  ```
1245
1359
 
1246
- Example: Create a value as a hash, with one key and the value is read from a file:
1360
+ Example: Create a `Hash` value with one key and the value is read from a file:
1247
1361
 
1248
1362
  ```bash
1249
1363
  ascli config echo @ruby:'{"token_verification_key"=>File.read("mykey.txt")}'
1250
1364
  ```
1251
1365
 
1252
- Example: read a csv file and create a list of hash for bulk provisioning:
1366
+ Example: read a csv file and create an `Array` of `Hash` for bulk provisioning:
1253
1367
 
1254
1368
  ```bash
1255
1369
  cat test.csv
@@ -1274,37 +1388,49 @@ ascli config echo @csvt:@file:test.csv
1274
1388
  +------+---------------------+
1275
1389
  ```
1276
1390
 
1277
- Example: create a hash and include values from preset named "config" of config file in this hash
1391
+ Example: create a `Hash` with values coming from a preset named `config`
1278
1392
 
1279
1393
  ```bash
1280
- ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
1394
+ ascli config echo @json:@extend:'{"hello":true,"version":"@preset:config.version@"}'
1281
1395
  ```
1282
1396
 
1283
- ```bash
1284
- {"version"=>"0.9", "hello"=>true}
1397
+ ```output
1398
+ +---------+-----------+
1399
+ | key | value |
1400
+ +---------+-----------+
1401
+ | hello | true |
1402
+ | version | 4.14.0 |
1403
+ +---------+-----------+
1285
1404
  ```
1286
1405
 
1287
- > **Note:** `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` are equivalent to: `@preset:config`
1288
-
1289
- ### <a id="native"></a>Structured Value
1290
-
1291
- Some options and parameters expect a [Extended Value](#extended), i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
1406
+ Example: Create a `Hash` from YAML provided as **heredoc**:
1292
1407
 
1293
- For instance, a [*transfer-spec*](#transferspec) is expected to be a [Extended Value](#extended).
1294
-
1295
- Structured values shall be described using the [Extended Value Syntax](#extended).
1296
- A convenient way to specify a [Extended Value](#extended) is to use the `@json:` decoder, and describe the value in JSON format. The `@ruby:` decoder can also be used. For an array of hash tables, the `@csvt:` decoder can be used.
1408
+ ```bash
1409
+ ascli conf echo @yaml:@stdin: --format=json<<EOF
1410
+ key1: value1
1411
+ key2:
1412
+ - item1
1413
+ - item2
1414
+ key3:
1415
+ key4: value4
1416
+ key5: value5
1417
+ EOF
1418
+ ```
1297
1419
 
1298
- It is also possible to provide a [Extended Value](#extended) in a file using `@json:@file:<path>`
1420
+ ```json
1421
+ {"key1":"value1","key2":["item1","item2"],"key3":{"key4":"value4","key5":"value5"}}
1422
+ ```
1299
1423
 
1300
1424
  ### <a id="conffolder"></a>Configuration and Persistency Folder
1301
1425
 
1302
- `ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored `[config folder]`: `[User's home folder]/.aspera/ascli`.
1426
+ `ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored by default in `[User's home folder]/.aspera/ascli`.
1303
1427
 
1304
- Note: `[User's home folder]` is found using Ruby's `Dir.home` (`rb_w32_home_dir`).
1305
- It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
1428
+ > **Note:** `[User's home folder]` is found using Ruby's `Dir.home` (`rb_w32_home_dir`).
1429
+ It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`.
1430
+ `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists.
1431
+ So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
1306
1432
 
1307
- The [config folder] can be displayed using :
1433
+ The configuration folder can be displayed using :
1308
1434
 
1309
1435
  ```bash
1310
1436
  ascli config folder
@@ -1314,7 +1440,7 @@ ascli config folder
1314
1440
  /Users/kenji/.aspera/ascli
1315
1441
  ```
1316
1442
 
1317
- It can be overridden using the environment variable `ASCLI_HOME`.
1443
+ It can be overridden using option `home`.
1318
1444
 
1319
1445
  Example (Windows):
1320
1446
 
@@ -1326,11 +1452,17 @@ ascli config folder
1326
1452
  C:\Users\Kenji\.aspera\ascli
1327
1453
  ```
1328
1454
 
1329
- When OAuth is used (AoC, Faspex4 api v4, Faspex5) `ascli` keeps a cache of generated bearer tokens in `[config folder]/persist_store` by default.
1455
+ When OAuth is used (AoC, Faspex4 api v4, Faspex5) `ascli` keeps a cache of generated bearer tokens in folder `persist_store` in configuration folder by default.
1330
1456
  Option `cache_tokens` (**yes**/no) allows to control if Oauth tokens are cached on file system, or generated for each request.
1331
- The command `config flush_tokens` deletes all existing tokens.
1457
+ The command `config flush_tokens` clears that cache.
1332
1458
  Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
1333
- Tokens that can be refreshed are refreshed. Else tokens are re-generated if expired.
1459
+ When a token has expired, then a new token is generated, either using a refresh_token if it is available, or by the default method.
1460
+
1461
+ ### Temporary files
1462
+
1463
+ Some temporary files may be needed during runtime.
1464
+ The temporary folder may be specified with option: `temp_folder`.
1465
+ Temporary files are deleted at the end of execution unless option: `clean_temp` is set to `no`.
1334
1466
 
1335
1467
  ### <a id="configfile"></a>Configuration file
1336
1468
 
@@ -1371,7 +1503,7 @@ The command `set` allows setting individual options in a [option preset](#lprt).
1371
1503
  ascli config preset set demo_server password my_password_here
1372
1504
  ```
1373
1505
 
1374
- The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [*Structured Value*](#native).
1506
+ The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended).
1375
1507
 
1376
1508
  ```bash
1377
1509
  ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
@@ -1406,11 +1538,11 @@ ascli config preset over
1406
1538
  ascli config preset list
1407
1539
  ```
1408
1540
 
1409
- #### <a id="lprtconf"></a>Special Option preset: config
1541
+ #### <a id="lprtconf"></a>Special Option preset: `config`
1410
1542
 
1411
1543
  This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
1412
1544
 
1413
- #### <a id="lprtdef"></a>Special Option preset: default
1545
+ #### <a id="lprtdef"></a>Special Option preset: `default`
1414
1546
 
1415
1547
  This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
1416
1548
 
@@ -1439,13 +1571,15 @@ Plugin `config` provides general commands for `ascli`:
1439
1571
  - Option preset, config file operations
1440
1572
  - wizard
1441
1573
  - vault
1442
- - ascp
1574
+ - `ascp`
1443
1575
 
1444
- The default configuration for `config` is read for any plugin invocation, this allows setting global options, such as `--log-level` or `--interactive`.
1576
+ The default preset for `config` is read for any plugin invocation, this allows setting global options, such as `--log-level` or `--interactive`.
1445
1577
  When `ascli` starts, it looks for the `default` Option preset and checks the value for `config`.
1446
- If set, it loads the option values for any plugin used.
1578
+ If set, it loads the options independently of the plugin used.
1447
1579
 
1448
- > **Note:** If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
1580
+ > **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
1581
+ >
1582
+ > **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
1449
1583
 
1450
1584
  Show current default (global) Option preset (`config` plugin):
1451
1585
 
@@ -1455,13 +1589,14 @@ global_common_defaults
1455
1589
  ```
1456
1590
 
1457
1591
  ```bash
1458
- ascli conf preset set global_common_defaults version_check_days 0
1592
+ ascli conf preset set GLOBAL version_check_days 0
1459
1593
  ```
1460
1594
 
1461
- If the default global Option preset is not set:
1595
+ If the default global Option preset is not set, and you want to use a different name:
1462
1596
 
1463
1597
  ```bash
1464
- ascli conf preset set default config global_common_defaults
1598
+ ascli conf preset set my_common_defaults version_check_days 0
1599
+ ascli conf preset set default config my_common_defaults
1465
1600
  ```
1466
1601
 
1467
1602
  #### Config sample commands
@@ -1473,45 +1608,76 @@ config ascp connect version 'Aspera Connect for Windows' download 'Windows Insta
1473
1608
  config ascp connect version 'Aspera Connect for Windows' list
1474
1609
  config ascp connect version 'Aspera Connect for Windows' open documentation
1475
1610
  config ascp errors
1476
- config ascp info --sdk-folder=Tsdk_test_dir
1477
- config ascp install --sdk-folder=Tsdk_test_dir
1611
+ config ascp info --sdk-folder=sdk_test_dir
1612
+ config ascp install
1613
+ config ascp install --sdk-folder=sdk_test_dir
1478
1614
  config ascp products list
1615
+ config ascp products use 'IBM Aspera Connect'
1479
1616
  config ascp show
1480
1617
  config ascp spec
1618
+ config ascp use /usr/bin/ascp
1481
1619
  config check_update
1482
1620
  config coffee
1483
1621
  config coffee --ui=text
1484
- config detect --url=https://faspex4.example.com/path
1485
- config detect --url=https://my_aoc_org.ibmaspera.com
1486
- config detect --url=https://node_simple.example.com/path
1622
+ config detect https://faspex4.example.com/path
1623
+ config detect https://faspex5.example.com/path
1624
+ config detect https://node.example.com/path
1625
+ config detect https://shares.example.com/path shares
1626
+ config detect my_org aoc
1487
1627
  config doc
1488
1628
  config doc transfer-parameters
1489
- config echo 'hello'
1490
1629
  config echo @base64:SGVsbG8gV29ybGQK
1491
1630
  config echo @csvt:@stdin:
1492
1631
  config echo @env:USER
1493
1632
  config echo @lines:@stdin:
1494
1633
  config echo @list:,1,2,3
1634
+ config echo @secret:
1495
1635
  config echo @uri:/etc/hosts
1496
1636
  config echo @uri:file:/etc/hosts
1497
- config echo @uri:http://www.ibm.com
1498
- config echo @uri:https://www.ibm.com
1499
- config echo @val:@file:no_such_file
1637
+ config echo @uri:http://ifconfig.me
1638
+ config echo @uri:https://ifconfig.me
1639
+ config echo @vault:my_preset.password
1500
1640
  config echo @zlib:@stdin:
1501
- config email_test --notif-to=my_recipient_email
1502
- config export
1641
+ config echo hello
1642
+ config email_test --notify-to=my_email_external
1503
1643
  config flush_tokens
1504
- config genkey mykey
1505
- config plugin create mycommand T
1644
+ config folder
1645
+ config gem name
1646
+ config gem path
1647
+ config gem version
1648
+ config genkey my_key
1649
+ config genkey my_key 4096
1650
+ config initdemo
1651
+ config open
1652
+ config plugin create my_command
1506
1653
  config plugin list
1507
- config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com
1508
- config wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes
1509
- config wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
1654
+ config preset delete conf_name
1655
+ config preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
1656
+ config preset list
1657
+ config preset overview
1658
+ config preset set conf_name param value
1659
+ config preset set default shares conf_name
1660
+ config preset show conf_name
1661
+ config preset unset conf_name param
1662
+ config preset update conf_name --p1=v1 --p2=v2
1663
+ config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
1664
+ config vault create my_label @json:'{"password":"my_password_here","description":"my secret"}'
1665
+ config vault delete my_label
1666
+ config vault list
1667
+ config vault show my_label
1668
+ config wizard https://console.example.com/path console
1669
+ config wizard https://faspex4.example.com/path faspex --username=test --password=test
1670
+ config wizard https://faspex5.example.com/path faspex5 --key-path=my_private_key
1671
+ config wizard https://node.example.com/path node --username=test --password=test
1672
+ config wizard https://orch.example.com/path orchestrator --username=test --password=test
1673
+ config wizard https://shares.example.com/path shares --username=test --password=test
1674
+ config wizard my_org aoc --key-path= --username=my_user_email
1675
+ config wizard my_org aoc --key-path= --username=my_user_email --use-generic-client=yes
1510
1676
  ```
1511
1677
 
1512
1678
  #### Format of file
1513
1679
 
1514
- The configuration file is a hash in a YAML file. Example:
1680
+ The configuration file is a `Hash` in a YAML file. Example:
1515
1681
 
1516
1682
  ```yaml
1517
1683
  config:
@@ -1544,7 +1710,7 @@ The user may create as many [option presets](#lprt) as needed. For instance, a p
1544
1710
 
1545
1711
  Values in the configuration also follow the [Extended Value Syntax](#extended).
1546
1712
 
1547
- Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
1713
+ > **Note:** if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
1548
1714
 
1549
1715
  ```bash
1550
1716
  ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
@@ -1561,7 +1727,7 @@ my_aoc_org:
1561
1727
 
1562
1728
  So, the key file will be read only at execution time, but not be embedded in the configuration file.
1563
1729
 
1564
- #### Options evaluation order
1730
+ #### Evaluation order of options
1565
1731
 
1566
1732
  Some options are global, some options are available only for some plugins. (the plugin is the first level command).
1567
1733
 
@@ -1569,7 +1735,7 @@ Options are loaded using this algorithm:
1569
1735
 
1570
1736
  - If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
1571
1737
  - else it looks for the name of the plugin as key in section `default`, the value is the name of the default [option preset](#lprt) for it, and loads it.
1572
- - If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or of the value is a Hash, it uses it as options values.
1738
+ - If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
1573
1739
  - Environment variables are evaluated
1574
1740
  - Command line options are evaluated
1575
1741
 
@@ -1602,13 +1768,25 @@ Example: Define options using command line:
1602
1768
  ascli -N --url=_url_here_ --password=my_password_here --username=_name_here_ node --show-config
1603
1769
  ```
1604
1770
 
1605
- Example: Define options using a hash:
1771
+ Example: Define options using a `Hash`:
1606
1772
 
1607
1773
  ```bash
1608
1774
  ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","username":"_name_here_"}' node --show-config
1609
1775
  ```
1610
1776
 
1611
- #### Shares Examples
1777
+ #### Wizard
1778
+
1779
+ The wizard is a command that asks the user for information and creates a [option preset](#lprt) with the provided information.
1780
+
1781
+ It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
1782
+
1783
+ The simplest invocation is:
1784
+
1785
+ ```bash
1786
+ ascli config wizard
1787
+ ```
1788
+
1789
+ #### Example of configuration for a plugin
1612
1790
 
1613
1791
  For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
1614
1792
  only username/password and url are required (either on command line, or from config file).
@@ -1666,7 +1844,7 @@ A more secure option is to retrieve values from a secret vault.
1666
1844
 
1667
1845
  The vault is used with options `vault` and `vault_password`.
1668
1846
 
1669
- `vault` defines the vault to be used and shall be a Hash, example:
1847
+ `vault` defines the vault to be used and shall be a `Hash`, example:
1670
1848
 
1671
1849
  ```json
1672
1850
  {"type":"system","name":"ascli"}
@@ -1728,14 +1906,17 @@ A passwords can be saved in clear in a [option preset](#lprt) together with othe
1728
1906
  Example:
1729
1907
 
1730
1908
  ```bash
1731
- `ascli` conf preset update myconf --url=... --username=... --password=...
1909
+ ascli conf preset update myconf --url=... --username=... --password=...
1732
1910
  ```
1733
1911
 
1734
1912
  For a more secure storage one can do:
1735
1913
 
1736
1914
  ```bash
1737
- `ascli` conf preset update myconf --url=... --username=... --password=@val:@vault:myconf.password
1738
- `ascli` conf vault create myconf @json:'{"password":"my_password_here"}'
1915
+ ascli conf preset update myconf --url=... --username=... --password=@val:@vault:myconf.password
1916
+ ```
1917
+
1918
+ ```bash
1919
+ ascli conf vault create myconf @json:'{"password":"my_password_here"}'
1739
1920
  ```
1740
1921
 
1741
1922
  > **Note:** use `@val:` in front of `@vault:` so that the extended value is not evaluated.
@@ -1764,7 +1945,7 @@ The format expected for private keys is [PEM](https://en.wikipedia.org/wiki/Priv
1764
1945
 
1765
1946
  #### `ascli` for key generation
1766
1947
 
1767
- The generated key is of type RSA, by default: 4096 bit.
1948
+ The generated key is of type `RSA`, by default: **4096** bit.
1768
1949
  For convenience, the public key is also extracted with extension `.pub`.
1769
1950
  The key is not passphrase protected.
1770
1951
 
@@ -1772,6 +1953,14 @@ The key is not passphrase protected.
1772
1953
  ascli config genkey ${PRIVKEYFILE} 4096
1773
1954
  ```
1774
1955
 
1956
+ > **Note:** `ascli` uses the `openssl` library.
1957
+
1958
+ To display the version of **openssl** used in `ascli`:
1959
+
1960
+ ```bash
1961
+ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION
1962
+ ```
1963
+
1775
1964
  #### `ssh-keygen`
1776
1965
 
1777
1966
  Both private and public keys are generated, option `-N` is for passphrase.
@@ -1782,7 +1971,7 @@ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
1782
1971
 
1783
1972
  #### `openssl`
1784
1973
 
1785
- To generate a private key pair with a passphrase the following can be used on any system:
1974
+ To generate a private key with a passphrase the following can be used on any system:
1786
1975
 
1787
1976
  ```bash
1788
1977
  openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE} 4096
@@ -1808,83 +1997,99 @@ mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
1808
1997
 
1809
1998
  ### <a id="certificates"></a>SSL CA certificate bundle
1810
1999
 
1811
- `ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
1812
- Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
1813
-
1814
- To display the current root certificate store locations:
2000
+ To display trusted certificate store locations:
1815
2001
 
1816
2002
  ```bash
1817
- ascli conf echo @ruby:'[OpenSSL::X509::DEFAULT_CERT_FILE,OpenSSL::X509::DEFAULT_CERT_DIR]'
2003
+ ascli --show-config --fields=cert_stores
1818
2004
  ```
1819
2005
 
1820
- Ruby's default values can be overridden by env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
2006
+ To modify the locations of certificate store, use option `cert_stores`.
2007
+ If you use this option, then default locations are removed, but they can be added using special value `DEF`.
2008
+ The value can be either an `Array`, or successive options.
1821
2009
 
1822
- `ascp` also needs to validate certificates when using **WSS**.
1823
- By default, `ascp` uses primarily certificates from hard-coded path (e.g. on macOS: `/Library/Aspera/ssl`) for WSS.
1824
- `ascli` overrides and sets the default Ruby certificate path as well for `ascp` using `-i` switch.
2010
+ `ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
2011
+ Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
2012
+ Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
1825
2013
 
1826
- To update `ascli` trusted root certificates, just update your system's root certificates or use env vars specified here above.
2014
+ > **Note:** One can display those values like this:
1827
2015
 
1828
- ### Plugins
2016
+ ```bash
2017
+ ascli conf echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
2018
+ ascli conf echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
2019
+ ```
1829
2020
 
1830
- `ascli` uses a plugin mechanism.
1831
- The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
1832
- Each plugin usually represents commands sent to a specific application.
1833
- For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
2021
+ `ascp` also needs to validate certificates when using **WSS**.
1834
2022
 
1835
- Available plugins can be found using command:
2023
+ > **Note:** `ascli` overrides the default hardcoded location used by `ascp` for WSS (e.g. on macOS: `/Library/Aspera/ssl`) and uses the same locations as specified in `cert_stores` (using `-i` switch of `ascp`). Hardcoded locations can be found using:
1836
2024
 
1837
2025
  ```bash
1838
- ascli conf plugin list
2026
+ strings $(ascli conf ascp info --fields=ascp)|grep -w OPENSSLDIR
1839
2027
  ```
1840
2028
 
1841
- ```output
1842
- +--------------+--------------------------------------------------------+
1843
- | plugin | path |
1844
- +--------------+--------------------------------------------------------+
1845
- | shares | ..../aspera-cli/lib/aspera/cli/plugins/shares.rb |
1846
- | node | ..../aspera-cli/lib/aspera/cli/plugins/node.rb |
1847
- ...
1848
- +--------------+--------------------------------------------------------+
2029
+ or
2030
+
2031
+ ```bash
2032
+ ascli conf ascp info --fields=openssldir
1849
2033
  ```
1850
2034
 
1851
- #### <a id="createownplugin"></a>Create your own plugin
2035
+ To update trusted root certificates for `ascli`:
2036
+ Display the trusted certificate store locations used by `ascli`.
2037
+ Typically done by updating the system's root certificate store.
1852
2038
 
1853
- By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
2039
+ An up-to-date version of the certificate bundle can be retrieved with:
1854
2040
 
1855
2041
  ```bash
1856
- ascli --show-config --select=@json:'{"key":"plugin_folder"}'
2042
+ ascli conf echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
1857
2043
  ```
1858
2044
 
1859
- You can create the skeleton of a new plugin like this:
2045
+ To download that certificate store:
1860
2046
 
1861
2047
  ```bash
1862
- ascli conf plugin create foo .
2048
+ ascli conf echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text > /tmp/cacert.pem
1863
2049
  ```
1864
2050
 
1865
- ```output
1866
- Created ./foo.rb
2051
+ Then, use this store by setting the option `` or env var export SSL_CERT_FILE
2052
+
2053
+ To trust a certificate (e.g. self-signed), provided that the `CN` is correct, save the certificate to a file:
2054
+
2055
+ ```bash
2056
+ ascli conf remote_certificate https://localhost:9092 > myserver.pem
1867
2057
  ```
1868
2058
 
2059
+ > **Note:** the saved certificate shows the CN as first line.
2060
+
2061
+ Then, use this file as certificate store (e.g. here, Node API):
2062
+
1869
2063
  ```bash
1870
- ascli --plugin-folder=. foo
2064
+ ascli conf echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
1871
2065
  ```
1872
2066
 
1873
- #### <a id="plugins"></a>Plugins: Application URL and Authentication
2067
+ ### Image and video thumbnails
2068
+
2069
+ `ascli` can display thumbnails for images and videos in the terminal.
2070
+ This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
2071
+ It's also available when using the `show` command of `preview` plugin.
1874
2072
 
1875
- `ascli` comes with several Aspera application plugins.
2073
+ The following options can be specified in the option `query`:
1876
2074
 
1877
- REST APIs of Aspera legacy applications (Aspera Node, Faspex, Shares, Console, Orchestrator, Server) use simple username/password authentication: HTTP Basic Authentication.
2075
+ | option | description |
2076
+ |------------|-------------|
2077
+ | text | display text instead of image (Bool) |
2078
+ | double | display double text resolution (half characters) (Bool) |
2079
+ | font_ratio | Font height/width ratio in terminal (Float) |
1878
2080
 
1879
- Those are using options:
2081
+ ### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
2082
+
2083
+ Some actions may require the use of a graphical tool:
1880
2084
 
1881
- - url
1882
- - username
1883
- - password
2085
+ - a browser for Aspera on Cloud authentication (web auth method)
2086
+ - a text editor for configuration file edition
1884
2087
 
1885
- Those can be provided using command line, parameter set, env var, see section above.
2088
+ By default `ascli` assumes that a graphical environment is available on Windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
2089
+ It is also possible to force the graphical mode with option --ui :
1886
2090
 
1887
- Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
2091
+ - `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
2092
+ - `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1888
2093
 
1889
2094
  ### Logging, Debugging
1890
2095
 
@@ -1921,16 +2126,21 @@ When `ascli` is used interactively in a shell, the shell itself will usually log
1921
2126
 
1922
2127
  ### Learning Aspera Product APIs (REST)
1923
2128
 
1924
- `ascli` uses mainly Aspera applications REST APIs.
1925
- To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
2129
+ `ascli` uses mainly REST APIs to interact with Aspera applications.
1926
2130
 
1927
- In order to get traces of execution, use argument : `--log-level=debug`
2131
+ To get traces of execution, with dump of API calls, use argument : `--log-level=debug`.
2132
+
2133
+ To display HTTP/S traffic set option `log_level` to `trace2`: `--log-level=trace2`.
2134
+ It will display the exact content of HTTP requests and responses.
1928
2135
 
1929
2136
  ### <a id="http_options"></a>HTTP socket parameters
1930
2137
 
1931
- If the server does not provide a valid certificate, use option: `--insecure=yes`.
2138
+ To ignore SSL certificate for any address/port, use option: `insecure`, i.e. `--insecure=yes`.
2139
+ To ignore SSL certificate for specific address/port, use option `ignore_certificate`, set to an `Array` of URL for which certificate will be ignored (only the address and port are matched), e.g. `--ignore-certificate=@list:,https://127.0.0.1:9092`
2140
+
2141
+ > **Note:** Ignoring certificate also applies to `ascp`'s wss.
1932
2142
 
1933
- HTTP socket parameters can be adjusted using option `http_options`:
2143
+ HTTP connection parameters (not `ascp` wss) can be adjusted using option `http_options`:
1934
2144
 
1935
2145
  | parameter | default |
1936
2146
  |----------------------|---------|
@@ -1939,9 +2149,9 @@ HTTP socket parameters can be adjusted using option `http_options`:
1939
2149
  | `open_timeout` | 60 |
1940
2150
  | `keep_alive_timeout` | 2 |
1941
2151
 
1942
- Values are in set *seconds* and can be of type either integer or float.
2152
+ Values are in set **seconds** and can be of type either integer or float.
1943
2153
  Default values are the ones of Ruby:
1944
- refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
2154
+ For a full list, refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
1945
2155
 
1946
2156
  Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
1947
2157
 
@@ -1951,34 +2161,22 @@ Example:
1951
2161
  ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
1952
2162
  ```
1953
2163
 
1954
- ### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
1955
-
1956
- Some actions may require the use of a graphical tool:
1957
-
1958
- - a browser for Aspera on Cloud authentication (web auth method)
1959
- - a text editor for configuration file edition
1960
-
1961
- By default `ascli` assumes that a graphical environment is available on windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
1962
- It is also possible to force the graphical mode with option --ui :
1963
-
1964
- - `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1965
- - `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1966
-
1967
2164
  ### Proxy
1968
2165
 
1969
2166
  There are several types of network connections, each of them use a different mechanism to define a (forward) **proxy**:
1970
2167
 
1971
- - Ruby HTTP: REST and HTTPGW client
1972
- - Legacy Aspera HTTP/S Fallback
2168
+ - Ruby HTTP: REST and HTTP Gateway client
2169
+ - Legacy Aspera HTTP/S Fallback and `ascp` wss
1973
2170
  - Aspera FASP
1974
2171
 
1975
2172
  Refer to the following sections.
1976
2173
 
1977
- ### Proxy for REST and HTTPGW
2174
+ ### Proxy for REST and HTTP Gateway
1978
2175
 
1979
2176
  There are two possibilities to define an HTTP proxy to be used when Ruby HTTP is used.
1980
2177
 
1981
- The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy, e.g. `http://myproxy.org.net:3128`.
2178
+ The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy.
2179
+ E.g. `http://myproxy.org.net:3128`.
1982
2180
  Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
1983
2181
 
1984
2182
  > **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
@@ -2177,7 +2375,7 @@ ascli config ascp connect version 'Aspera Connect for Mac Intel' download enclos
2177
2375
  ```
2178
2376
 
2179
2377
  ```output
2180
- Time: 00:00:02 =========================================================== 100% 27766 KB/sec Time: 00:00:02
2378
+ Time: 00:00:02 ============================================= 100% 27766 KB/sec Time: 00:00:02
2181
2379
  Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
2182
2380
  ```
2183
2381
 
@@ -2192,7 +2390,7 @@ There are currently 3 agents, set with option `transfer`:
2192
2390
 
2193
2391
  - [`direct`](#agt_direct) : a local execution of `ascp`
2194
2392
  - [`connect`](#agt_connect) : use of a local Connect Client
2195
- - [`node`](#agt_node) : use of an Aspera Transfer Node (potentially *remote*).
2393
+ - [`node`](#agt_node) : use of an Aspera Transfer Node (potentially **remote**).
2196
2394
  - [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
2197
2395
  - [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
2198
2396
 
@@ -2200,7 +2398,7 @@ There are currently 3 agents, set with option `transfer`:
2200
2398
  For example, a node agent executing an "upload", or "package send" operation
2201
2399
  will effectively push files to the related server from the agent node.
2202
2400
 
2203
- `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *native* `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
2401
+ `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
2204
2402
 
2205
2403
  Specific options for agents are provided with option `transfer_info`, cumulatively.
2206
2404
 
@@ -2209,23 +2407,23 @@ Specific options for agents are provided with option `transfer_info`, cumulative
2209
2407
  The `direct` agent directly executes a local `ascp`.
2210
2408
  This is the default agent for `ascli`.
2211
2409
  This is equivalent to option `--transfer=direct`.
2212
- `ascli` will detect locally installed Aspera products, including SDK, and use `ascp` from that component.
2410
+ `ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
2213
2411
  Refer to section [FASP](#client).
2214
2412
 
2215
2413
  The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
2216
2414
 
2217
2415
  | Name | Type | Description |
2218
2416
  |----------------------|-------|-------------|
2219
- | wss | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
2220
- | ascp_args | Array | Array of strings with native ascp arguments<br/>Use this instead of deprecated `EX_ascp_args`.<br/>Default: [] |
2221
- | spawn_timeout_sec | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
2222
- | spawn_delay_sec | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
2223
- | multi_incr_udp | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
2224
- | resume | Hash | Resume<br/>parameters<br/>See below |
2225
- | resume.iter_max | int | Resume<br/>Max number of retry on error<br/>Default: 7 |
2226
- | resume.sleep_initial | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
2227
- | resume.sleep_factor | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
2228
- | resume.sleep_max | int | Resume<br/>Default: 60 |
2417
+ | `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
2418
+ | `ascp_args` | Array | Array of strings with native `ascp` arguments<br/>Use this instead of deprecated `EX_ascp_args`.<br/>Default: [] |
2419
+ | `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
2420
+ | `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
2421
+ | `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
2422
+ | `resume` | Hash | Resume<br/>parameters<br/>See below |
2423
+ | `resume.iter_max` | int | Resume<br/>Max number of retry on error<br/>Default: 7 |
2424
+ | `resume.sleep_initial` | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
2425
+ | `resume.sleep_factor` | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
2426
+ | `resume.sleep_max` | int | Resume<br/>Default: 60 |
2229
2427
 
2230
2428
  In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
2231
2429
  Sleep between iterations is:
@@ -2234,7 +2432,7 @@ Sleep between iterations is:
2234
2432
  max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
2235
2433
  ```
2236
2434
 
2237
- Some transfer errors are considered "retryable" (e.g. timeout) and some other not (e.g. wrong password).
2435
+ Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
2238
2436
  The list of known protocol errors and retry level can be listed:
2239
2437
 
2240
2438
  ```bash
@@ -2252,8 +2450,14 @@ ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
2252
2450
  But it is preferred to use the option `transfer_info` with parameter `ascp_args`.
2253
2451
 
2254
2452
  This can be useful to activate logging using option `-L` of `ascp`.
2255
- For example the option `--transfer-info=@json:'{"ascp_args":["-DDL-"]}'` will activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`).
2256
- This is useful if the transfer fails.
2453
+ For example, to activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`):
2454
+
2455
+ ```bash
2456
+ --transfer-info=@json:'{"ascp_args":["-DDL-"]}'
2457
+ ```
2458
+
2459
+ This is useful to debug if a transfer fails.
2460
+
2257
2461
  To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--transfer-info=@json:'{"ascp_args":["-L","/path/to/folder"]}'`.
2258
2462
 
2259
2463
  > **Note:** When transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided using `transfer_info` parameter `ascp_args`).
@@ -2356,26 +2560,34 @@ Parameters provided in option `transfer_info` are:
2356
2560
  | root_id | string | password or secret</br>Mandatory only for bearer token |
2357
2561
 
2358
2562
  Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
2359
- `--transfer-info=@preset:_name_here_` or be specified using the extended value syntax :
2360
- `--transfer-info=@json:'{"url":"https://...","username":"_user_here_","password":"my_password_here"}'`
2563
+
2564
+ ```bash
2565
+ --transfer-info=@preset:_name_here_
2566
+ ```
2567
+
2568
+ or be specified using the extended value syntax :
2569
+
2570
+ ```bash
2571
+ --transfer-info=@json:'{"url":"https://...","username":"_user_here_","password":"my_password_here"}'
2572
+ ```
2361
2573
 
2362
2574
  If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
2363
2575
 
2364
- If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
2576
+ If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node.
2577
+ It can be either the access key's root file id, or any authorized file id underneath it.
2365
2578
 
2366
2579
  #### <a id="agt_httpgw"></a>HTTP Gateway
2367
2580
 
2368
- If it possible to send using a HTTP gateway, in case FASP is not allowed.
2581
+ If it possible to send using a HTTP gateway, in case use of FASP is not allowed.
2369
2582
 
2370
2583
  Parameters provided in option `transfer_info` are:
2371
2584
 
2372
2585
  | Name | Type | Description |
2373
2586
  |------------------------|--------|---------------------------------------|
2374
2587
  | url | string | URL of the HTTP GW</br>Mandatory |
2375
- | upload_bar_refresh_sec | float | Refresh rate for upload progress bar |
2376
- | upload_chunk_size | int | Size in bytes of chunks for upload |
2377
- | api_version | string | v1 or v2, for force use of version |
2378
- | synchronous | bool | wait for each message acknowledgment |
2588
+ | upload_chunk_size | int | Size in bytes of chunks for upload<br/>Default: 64000 |
2589
+ | api_version | string | v1 or v2, for force use of version<br/>Default: v2 |
2590
+ | synchronous | bool | wait for each message acknowledgment<br/>Default: false |
2379
2591
 
2380
2592
  Example:
2381
2593
 
@@ -2388,8 +2600,17 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
2388
2600
  #### <a id="agt_trsdk"></a>Transfer SDK
2389
2601
 
2390
2602
  Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
2603
+ Set option `transfer` to `trsdk`.
2604
+
2605
+ Options for `transfer_info` are:
2606
+
2607
+ | Name | Type | Description |
2608
+ |----------|--------|-------------|
2609
+ | url | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: grpc://127.0.0.1:0 |
2610
+ | external | bool | Use external daemon, do not start<br/>Default: false |
2611
+ | keep | bool | Keep the daemon running after exiting `ascli`<br/>Default: false |
2391
2612
 
2392
- By default it will listen on local port `55002` on `127.0.0.1`.
2613
+ > **Note:** if port zero is specified in the URL, then the daemon will listen on a random available port.
2393
2614
 
2394
2615
  The gem `grpc` was removed from dependencies, as it requires compilation of a native part.
2395
2616
  So, to use the Transfer SDK you should install this gem:
@@ -2399,11 +2620,17 @@ gem install grpc
2399
2620
  ```
2400
2621
 
2401
2622
  On Windows the compilation may fail for various reasons (3.1.1):
2623
+
2402
2624
  <!-- spellchecker: disable -->
2625
+
2403
2626
  - `cannot find -lx64-ucrt-ruby310`
2627
+
2404
2628
  &rarr; copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the dll extension)
2629
+
2405
2630
  - `conflicting types for 'gettimeofday'`
2631
+
2406
2632
  &rarr; edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)` ,i.e. change `struct timezone` to `void`
2633
+
2407
2634
  <!-- spellchecker: enable -->
2408
2635
 
2409
2636
  ### <a id="transferspec"></a>Transfer Specification
@@ -2417,13 +2644,12 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
2417
2644
  - file list
2418
2645
  - etc...
2419
2646
 
2420
- `ascli` builds the [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
2421
-
2422
- The [*transfer-spec*](#transferspec) is a Hash (dictionary), so it is described on the command line with the [Extended Value Syntax](#extended).
2647
+ `ascli` builds the [*transfer-spec*](#transferspec) internally as a `Hash`.
2648
+ It is not necessary to provide additional parameters on the command line for this transfer.
2423
2649
 
2424
2650
  It is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option.
2425
- The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters in a `Hash`.
2426
- Multiple `ts` options on command line are cumulative, and Hash is deeply merged.
2651
+ The `ts` option accepts a [`Hash` Extended Value](#extended) containing one or several [*transfer-spec*](#transferspec) parameters.
2652
+ Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
2427
2653
  To remove a (deep) key from transfer spec, set the value to `null`.
2428
2654
 
2429
2655
  > **Note:** Default transfer spec values can be displayed with command: `config ascp info --flat-hash=no` under field `ts`.
@@ -2460,96 +2686,109 @@ Columns:
2460
2686
  - D=Direct (local `ascp` execution)
2461
2687
  - N=Node API
2462
2688
  - C=Connect Client
2689
+ - T=Transfer SDK
2690
+ - H=HTTP Gateway
2463
2691
 
2464
2692
  `ascp` argument or environment variable is provided in description.
2465
2693
 
2466
2694
  Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
2467
2695
 
2468
- | Field | Type | D | N | C | Description |
2469
- | ----- | ---- | - | - | - | ----------- |
2470
- | apply_local_docroot | bool | Y | &nbsp; | &nbsp; | (--apply-local-docroot) |
2471
- | authentication | string | &nbsp; | &nbsp; | Y | value=token for SSH bypass keys, else password asked if not provided. |
2472
- | cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
2473
- | cipher_allowed | string | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none". |
2474
- | content_protection | string | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
2475
- | content_protection_password | string | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
2476
- | cookie | string | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
2477
- | create_dir | bool | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
2478
- | delete_before_transfer | bool | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
2479
- | delete_source | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2480
- | destination_root | string | Y | Y | Y | Destination root directory. |
2481
- | dgram_size | int | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
2482
- | direction | string | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
2483
- | exclude_newer_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
2484
- | exclude_older_than | int | Y | &nbsp; | &nbsp; | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
2485
- | fasp_port | int | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
2486
- | file_checksum | string | Y | Y | &nbsp; | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none |
2487
- | http_fallback | bool<br/>string | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
2488
- | http_fallback_port | int | Y | &nbsp; | &nbsp; | Specifies http port when no cipher is used<br/>(-t {int}) |
2489
- | https_fallback_port | int | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
2490
- | lock_min_rate | bool | Y | Y | Y | &nbsp; |
2491
- | lock_min_rate_kbps | bool | Y | Y | Y | &nbsp; |
2492
- | lock_rate_policy | bool | Y | Y | Y | &nbsp; |
2493
- | lock_target_rate | bool | Y | Y | Y | &nbsp; |
2494
- | lock_target_rate_kbps | bool | Y | Y | Y | &nbsp; |
2495
- | min_rate_cap_kbps | int | Y | Y | Y | &nbsp; |
2496
- | min_rate_kbps | int | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
2497
- | move_after_transfer | string | Y | Y | &nbsp; | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
2498
- | multi_session | int | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/> |
2499
- | multi_session_threshold | int | Y | Y | &nbsp; | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
2500
- | overwrite | string | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
2501
- | password | string | &nbsp; | Y | &nbsp; | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only. |
2502
- | paths | array | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional). |
2503
- | precalculate_job_size | bool | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
2504
- | preserve_access_time | bool | Y | Y | Y | (--preserve-access-time) |
2505
- | preserve_acls | string | Y | &nbsp; | &nbsp; | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
2506
- | preserve_creation_time | bool | Y | Y | Y | (--preserve-creation-time) |
2507
- | preserve_file_owner_gid | bool | Y | &nbsp; | &nbsp; | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
2508
- | preserve_file_owner_uid | bool | Y | &nbsp; | &nbsp; | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
2509
- | preserve_modification_time | bool | Y | Y | Y | (--preserve-modification-time) |
2510
- | preserve_remote_acls | string | Y | &nbsp; | &nbsp; | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum}) |
2511
- | preserve_source_access_time | bool | Y | &nbsp; | &nbsp; | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
2512
- | preserve_times | bool | Y | Y | Y | (--preserve-times) |
2513
- | proxy | string | Y | &nbsp; | &nbsp; | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
2514
- | rate_policy | string | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
2515
- | rate_policy_allowed | string | &nbsp; | &nbsp; | Y | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed |
2516
- | remote_host | string | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
2517
- | remote_password | string | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
2518
- | remote_user | string | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
2519
- | remove_after_transfer | bool | Y | Y | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2520
- | remove_empty_directories | bool | Y | Y | &nbsp; | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
2521
- | remove_empty_source_directory | bool | Y | &nbsp; | &nbsp; | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
2522
- | remove_skipped | bool | Y | Y | Y | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
2523
- | resume_policy | string | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
2524
- | retry_duration | string<br/>int | &nbsp; | Y | Y | Specifies how long to wait before retrying transfer. (e.g. "5min") |
2525
- | source_root | string | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
2526
- | source_root_id | string | &nbsp; | Y | &nbsp; | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
2527
- | src_base | string | Y | Y | &nbsp; | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
2528
- | ssh_port | int | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
2529
- | ssh_private_key | string | Y | &nbsp; | &nbsp; | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
2530
- | ssh_private_key_passphrase | string | Y | &nbsp; | &nbsp; | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
2531
- | sshfp | string | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
2532
- | symlink_policy | string | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
2533
- | tags | hash | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
2534
- | target_rate_cap_kbps | int | &nbsp; | &nbsp; | Y | Returned by upload/download_setup node API. |
2535
- | target_rate_kbps | int | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
2536
- | target_rate_percentage | string | Y | Y | Y | &nbsp; |
2537
- | title | string | &nbsp; | Y | Y | Title of the transfer |
2538
- | token | string | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
2539
- | use_ascp4 | bool | Y | Y | &nbsp; | specify version of protocol |
2540
- | wss_enabled | bool | Y | Y | Y | Server has Web Socket service enabled |
2541
- | wss_port | int | Y | Y | Y | TCP port used for websocket service feed |
2542
- | EX_ascp_args | array | Y | &nbsp; | &nbsp; | DEPRECATED: Use parameter ascp_args in option transfer_info<br/>Add native command line arguments to ascp |
2543
- | EX_at_rest_password | string | Y | &nbsp; | &nbsp; | DEPRECATED: Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
2544
- | EX_file_list | string | Y | &nbsp; | &nbsp; | source file list |
2545
- | EX_file_pair_list | string | Y | &nbsp; | &nbsp; | source file pair list |
2546
- | EX_http_proxy_url | string | Y | &nbsp; | &nbsp; | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
2547
- | EX_http_transfer_jpeg | int | Y | &nbsp; | &nbsp; | HTTP transfers as JPEG file<br/>(-j {int}) |
2548
- | EX_license_text | string | Y | &nbsp; | &nbsp; | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
2549
- | EX_no_read | bool | Y | &nbsp; | &nbsp; | no read source<br/>(--no-read) |
2550
- | EX_no_write | bool | Y | &nbsp; | &nbsp; | no write on destination<br/>(--no-write) |
2551
- | EX_proxy_password | string | Y | &nbsp; | &nbsp; | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL provided in parameter: proxy.<br/>(env:ASPERA_PROXY_PASS) |
2552
- | EX_ssh_key_paths | array | Y | &nbsp; | &nbsp; | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
2696
+ | Field | Type | D | N | C | T | H | Description |
2697
+ | ----- | ---- | - | - | - | - | - | ----------- |
2698
+ | apply_local_docroot | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Apply local docroot to source paths.<br/>(--apply-local-docroot) |
2699
+ | authentication | string | &nbsp; | &nbsp; | Y | &nbsp; | &nbsp; | value=token for SSH bypass keys, else password asked if not provided.<br/>(<ignored>) |
2700
+ | cipher | string | Y | Y | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
2701
+ | cipher_allowed | string | Y | Y | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none".<br/>(<ignored>) |
2702
+ | compression | int | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | ascp4 only, 0 / 1?<br/>(<ignored>) |
2703
+ | content_protection | string | Y | Y | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
2704
+ | content_protection_password | string | Y | Y | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
2705
+ | cookie | string | Y | Y | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
2706
+ | create_dir | bool | Y | Y | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
2707
+ | delete_before_transfer | bool | Y | Y | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
2708
+ | delete_source | bool | Y | Y | &nbsp; | &nbsp; | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2709
+ | destination_root | string | Y | Y | Y | Y | Y | Destination root directory.<br/>(<special>) |
2710
+ | destination_root_id | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | The file ID of the destination root directory.<br/>Required when using Bearer token auth for the destination node.<br/>(<ignored>) |
2711
+ | dgram_size | int | Y | Y | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
2712
+ | direction | string | Y | Y | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
2713
+ | exclude_newer_than | int | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
2714
+ | exclude_older_than | int | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
2715
+ | fasp_port | int | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
2716
+ | fasp_url | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Only used in Faspex.<br/>(<ignored>) |
2717
+ | file_checksum | string | Y | Y | &nbsp; | &nbsp; | &nbsp; | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(<ignored>) |
2718
+ | http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
2719
+ | http_fallback_port | int | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Specifies http port when no cipher is used<br/>(-t {int}) |
2720
+ | https_fallback_port | int | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
2721
+ | keepalive | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | The session is running in persistent session mode.<br/>(--keepalive) |
2722
+ | lock_min_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
2723
+ | lock_min_rate_kbps | bool | &nbsp; | &nbsp; | Y | &nbsp; | &nbsp; | If true, lock the minimum transfer rate to the value set for min_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
2724
+ | lock_rate_policy | bool | &nbsp; | &nbsp; | Y | &nbsp; | &nbsp; | If true, lock the rate policy to the default value.<br/>(<ignored>) |
2725
+ | lock_target_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
2726
+ | lock_target_rate_kbps | bool | Y | Y | Y | Y | Y | If true, lock the target transfer rate to the default value set for target_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
2727
+ | min_rate_cap_kbps | int | Y | Y | Y | Y | Y | The highest minimum rate that an incoming transfer can request, in kilobits per second.<br/>Client minimum rate requests that exceed the minimum rate cap are ignored.<br/>The default value of unlimited applies no cap to the minimum rate. (Default: 0)<br/>(<ignored>) |
2728
+ | min_rate_kbps | int | Y | Y | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
2729
+ | move_after_transfer | string | Y | Y | &nbsp; | &nbsp; | &nbsp; | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
2730
+ | multi_session | int | Y | Y | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/><br/>(<special>) |
2731
+ | multi_session_threshold | int | Y | Y | &nbsp; | &nbsp; | &nbsp; | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
2732
+ | obfuscate_file_names | bool | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Y | HTTP Gateway obfuscates file names when set to true.<br/>(<ignored>) |
2733
+ | overwrite | string | Y | Y | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
2734
+ | password | string | &nbsp; | Y | &nbsp; | &nbsp; | &nbsp; | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.<br/>(<ignored>) |
2735
+ | paths | array | Y | Y | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional).<br/>(<special>) |
2736
+ | precalculate_job_size | bool | Y | Y | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
2737
+ | preserve_access_time | bool | Y | Y | Y | Y | Y | Preserve the source-file access timestamps at the destination.<br/>Because source access times are updated by the transfer operation,<br/>the timestamp that is preserved is the one just before to the transfer.<br/>(--preserve-access-time) |
2738
+ | preserve_acls | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
2739
+ | preserve_creation_time | bool | Y | Y | Y | Y | Y | (Windows only) Preserve source-file creation timestamps at the destination.<br/>Only Windows systems retain information about creation time.<br/>If the destination is not a Windows computer, this option is ignored.<br/>(--preserve-creation-time) |
2740
+ | preserve_extended_attrs | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve the extended attributes.<br/>Allowed values: none, native, metafile<br/>(--preserve-xattrs {enum}) |
2741
+ | preserve_file_owner_gid | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
2742
+ | preserve_file_owner_uid | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
2743
+ | preserve_modification_time | bool | Y | Y | Y | Y | Y | Set the modification time, the last time a file or directory was modified (written), of a transferred file<br/>to the modification of the source file or directory.<br/>Preserve source-file modification timestamps at the destination.<br/>(--preserve-modification-time) |
2744
+ | preserve_remote_acls | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum}) |
2745
+ | preserve_source_access_time | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
2746
+ | preserve_times | bool | &nbsp; | Y | &nbsp; | &nbsp; | &nbsp; | Preserve file timestamps.<br/>(--preserve-times) |
2747
+ | proxy | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
2748
+ | rate_policy | string | Y | Y | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
2749
+ | rate_policy_allowed | string | &nbsp; | &nbsp; | Y | &nbsp; | &nbsp; | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed<br/>(<ignored>) |
2750
+ | read_threads | int | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | ascp4 only<br/>(<ignored>) |
2751
+ | remote_access_key | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node.<br/>(<ignored>) |
2752
+ | remote_host | string | Y | Y | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
2753
+ | remote_password | string | Y | Y | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
2754
+ | remote_user | string | Y | Y | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
2755
+ | remove_after_transfer | bool | Y | Y | &nbsp; | &nbsp; | &nbsp; | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
2756
+ | remove_empty_directories | bool | Y | Y | &nbsp; | &nbsp; | &nbsp; | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
2757
+ | remove_empty_source_directory | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
2758
+ | remove_skipped | bool | Y | Y | Y | &nbsp; | &nbsp; | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
2759
+ | resume_policy | string | Y | Y | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
2760
+ | retry_duration | string<br/>int | &nbsp; | Y | Y | &nbsp; | &nbsp; | Specifies how long to wait before retrying transfer. (e.g. "5min")<br/>(<ignored>) |
2761
+ | source_root | string | Y | Y | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
2762
+ | source_root_id | string | &nbsp; | Y | &nbsp; | &nbsp; | &nbsp; | The file ID of the source root directory. Required when using Bearer token auth for the source node.<br/>(<ignored>) |
2763
+ | src_base | string | Y | Y | &nbsp; | &nbsp; | &nbsp; | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
2764
+ | ssh_args | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Array of arguments to pass to SSH. Use with caution.<br/>(<ignored>) |
2765
+ | ssh_port | int | Y | Y | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
2766
+ | ssh_private_key | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
2767
+ | ssh_private_key_passphrase | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
2768
+ | sshfp | string | Y | Y | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
2769
+ | symlink_policy | string | Y | Y | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
2770
+ | tags | hash | Y | Y | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
2771
+ | target_rate_cap_kbps | int | &nbsp; | &nbsp; | Y | &nbsp; | &nbsp; | Returned by upload/download_setup node API.<br/>(<ignored>) |
2772
+ | target_rate_kbps | int | Y | Y | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
2773
+ | target_rate_percentage | string | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
2774
+ | title | string | &nbsp; | Y | Y | &nbsp; | &nbsp; | Title of the transfer<br/>(<ignored>) |
2775
+ | token | string | Y | Y | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
2776
+ | use_ascp4 | bool | Y | Y | &nbsp; | &nbsp; | &nbsp; | specify version of protocol<br/>(<special>) |
2777
+ | use_system_ssh | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | TODO, comment...<br/>(<ignored>) |
2778
+ | write_threads | int | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | ascp4 only<br/>(<ignored>) |
2779
+ | wss_enabled | bool | Y | Y | Y | Y | Y | Server has Web Socket service enabled<br/>(<special>) |
2780
+ | wss_port | int | Y | Y | Y | Y | Y | TCP port used for websocket service feed<br/>(<special>) |
2781
+ | EX_ascp_args | array | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.13) Use option transfer_info.ascp_args<br/>Add native command line arguments to ascp<br/>(<special>) |
2782
+ | EX_at_rest_password | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.13) Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
2783
+ | EX_file_list | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use command line file list, or option transfer_info.ascp_args<br/>source file list<br/>(<special>) |
2784
+ | EX_file_pair_list | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use command line file pair list, or option transfer_info.ascp_args<br/>source file pair list<br/>(<special>) |
2785
+ | EX_http_proxy_url | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) TODO, use proxy option ?<br/>Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
2786
+ | EX_http_transfer_jpeg | int | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>HTTP transfers as JPEG file<br/>(-j {int}) |
2787
+ | EX_license_text | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use env var ASPERA_SCP_LICENSE<br/>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
2788
+ | EX_no_read | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no read source<br/>(--no-read) |
2789
+ | EX_no_write | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no write on destination<br/>(--no-write) |
2790
+ | EX_proxy_password | string | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use env var ASPERA_PROXY_PASS<br/>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL provided in parameter: proxy.<br/>(env:ASPERA_PROXY_PASS) |
2791
+ | EX_ssh_key_paths | array | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
2553
2792
 
2554
2793
  #### Destination folder for transfers
2555
2794
 
@@ -2658,7 +2897,7 @@ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
2658
2897
 
2659
2898
  #### Source directory structure on destination
2660
2899
 
2661
- This section is not specific to `ascli`, it is `ascp` behaviour.
2900
+ This section is not specific to `ascli` it is `ascp` behavior.
2662
2901
 
2663
2902
  The transfer destination is normally expected to designate a destination folder.
2664
2903
 
@@ -2669,7 +2908,7 @@ But there is one exception: The destination specifies the new item name when the
2669
2908
  - destination is not an existing folder
2670
2909
  - the `dirname` of destination is an existing folder
2671
2910
 
2672
- For this reason it is recommended to set `create_dir` to `true` for consistent behaviour between single and multiple items transfer, this is the default in `ascli`.
2911
+ For this reason it is recommended to set `create_dir` to `true` for consistent behavior between single and multiple items transfer, this is the default in `ascli`.
2673
2912
 
2674
2913
  If a simple source file list is provided (no `destination` in `paths`, i.e. no `file_pair_list` provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.
2675
2914
 
@@ -2702,9 +2941,15 @@ If transfer spec has a `src_base`, it has the side effect that the simple source
2702
2941
 
2703
2942
  Advanced Example: Send files `./file1` and `./folder2/files2` to server (e.g. `/Upload`) and keep the original file names and folders, i.e. send `file1` to `/Upload/file1` and `files2` to `/Upload/folder2/files2`.
2704
2943
 
2705
- - If files are specified as `./file1 ./folder2/files2`, then destination will be: `/Upload/file1 /Upload/files2`
2944
+ - If files are specified as `./file1 ./folder2/files2`,
2945
+
2946
+ then destination will be: `/Upload/file1 /Upload/files2`
2947
+
2706
2948
  - One possibility is to specify a file pair list: `--src-type=pair file1 file1 folder2/files2 folder2/files2`
2707
- - Another possibility is to specify a source base: `--src-base=$PWD $PWD/file1 $PWD/folder2/files2` (note that `.` cannot be used as source base)
2949
+ - Another possibility is to specify a source base: `--src-base=$PWD $PWD/file1 $PWD/folder2/files2`
2950
+
2951
+ (note that `.` cannot be used as source base)
2952
+
2708
2953
  - Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--src-base=tmp_base tmp_base/file1 tmp_base/folder2/files2`
2709
2954
  - One can also similarly use `--sources=@ts` and specify the list of files in the `paths` field of transfer spec with both `source` and `destination` for each file.
2710
2955
 
@@ -2726,7 +2971,7 @@ Multi-session is directly supported by the node daemon.
2726
2971
  --ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
2727
2972
  ```
2728
2973
 
2729
- Note: `resume_policy` set to `attr` may cause problems: `none` or `sparse_csum` shall be preferred.
2974
+ > **Note:** `resume_policy` set to `attr` may cause problems: `none` or `sparse_csum` shall be preferred.
2730
2975
 
2731
2976
  `ascli` starts multiple `ascp` for Multi-session using `direct` agent.
2732
2977
 
@@ -2774,6 +3019,14 @@ Example: parameter to download a faspex package and decrypt on the fly
2774
3019
  --ts=@json:'{"precalculate_job_size":true}'
2775
3020
  ```
2776
3021
 
3022
+ ### Transfer progress bar
3023
+
3024
+ File transfer operations are monitored and a progress bar is displayed on the terminal if option `progress_bar` (`Bool`) is set to `yes` (default if the output is a terminal).
3025
+
3026
+ The same progress bar is used for any type of transfer, using `ascp`, server to server, using HTTPS, etc...
3027
+
3028
+ To display the native progress bar of `ascp`, use `--progress-bar=no --transfer-info=@json:'{"quiet":false}'`.
3029
+
2777
3030
  ### <a id="scheduler"></a>Scheduler
2778
3031
 
2779
3032
  It is useful to configure automated scheduled execution.
@@ -2787,7 +3040,7 @@ It can be configured:
2787
3040
 
2788
3041
  - Using utility [`schtasks.exe`](https://learn.microsoft.com/fr-fr/windows-server/administration/windows-commands/schtasks-create)
2789
3042
 
2790
- - Using powershell function [scheduletasks](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
3043
+ - Using powershell function [`scheduletasks`](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
2791
3044
 
2792
3045
  - Using `taskschd.msc` (UI)
2793
3046
 
@@ -2819,7 +3072,7 @@ crontab<<EOF
2819
3072
  EOF
2820
3073
  ```
2821
3074
 
2822
- > **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
3075
+ > **Note:** The logging options are kept here in the cron file instead of conf file to allow execution on command line with output on command line.
2823
3076
 
2824
3077
  ### <a id="locking"></a>Locking for exclusive execution
2825
3078
 
@@ -2857,37 +3110,32 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
2857
3110
  WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
2858
3111
  ```
2859
3112
 
2860
- ### "Proven&ccedil;ale"
3113
+ ### "Proven&ccedil;al"
2861
3114
 
2862
- `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
2863
- , some are provided om shared libraries and must be activated. (e.g. using `trapd`)
3115
+ `ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others.
3116
+ This mechanism is called **PVCL** (from **Proven&ccedil;al**, a restaurant located in Sophia Antipolis).
3117
+ Several **PVCL** adapters are available, one is embedded in `ascp`, the others are provided in shared libraries and must be activated.
2864
3118
 
2865
- The list of supported *PVCL* adapters can be retrieved with command:
3119
+ The list of supported **PVCL** adapters can be retrieved with command:
2866
3120
 
2867
3121
  ```bash
2868
- ascli conf ascp info
3122
+ ascli conf ascp info --fields=@re:'^pvcl'
2869
3123
  ```
2870
3124
 
2871
3125
  ```output
2872
- +--------------------+-----------------------------------------------------------+
2873
- | key | value |
2874
- +--------------------+-----------------------------------------------------------+
2875
- -----8<-----snip-----8<-----
2876
- | product_name | IBM Aspera SDK |
2877
- | product_version | 4.0.1.182389 |
2878
- | process | pvcl |
2879
- | shares | pvcl |
2880
- | noded | pvcl |
2881
- | faux | pvcl |
2882
- | file | pvcl |
2883
- | stdio | pvcl |
2884
- | stdio-tar | pvcl |
2885
- +--------------------+-----------------------------------------------------------+
3126
+ process v1
3127
+ shares v1
3128
+ noded v1
3129
+ faux v1
3130
+ file v1
3131
+ stdio v1
3132
+ stdio-tar v1
2886
3133
  ```
2887
3134
 
2888
3135
  Here we can see the adapters: `process`, `shares`, `noded`, `faux`, `file`, `stdio`, `stdio-tar`.
2889
3136
 
2890
- Those adapters can be used wherever a file path is used in `ascp` including configuration. They act as a pseudo "drive".
3137
+ Those adapters can be used wherever a file path is used in `ascp` including configuration.
3138
+ They act as a pseudo "drive".
2891
3139
 
2892
3140
  The simplified format is:
2893
3141
 
@@ -2895,7 +3143,8 @@ The simplified format is:
2895
3143
  <adapter>:///<sub file path>?<arg1>=<val1>&...
2896
3144
  ```
2897
3145
 
2898
- One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
3146
+ One of the adapters, used in this manual, for testing, is `faux`.
3147
+ It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
2899
3148
 
2900
3149
  ### <a id="faux_testing"></a>`faux:` for testing
2901
3150
 
@@ -2917,7 +3166,7 @@ where:
2917
3166
  - `filename` is the name that will be assigned to the file on the destination
2918
3167
  - `filesize` is the number of bytes that will be sent (in decimal).
2919
3168
 
2920
- Note: characters `?` and `&` are shell special characters (wildcard and background), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
3169
+ > **Note:** characters `?` and `&` are shell special characters (wildcard and background), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
2921
3170
 
2922
3171
  For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2<sup>20</sup>, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2<sup>60</sup>. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is `faux://`.
2923
3172
 
@@ -2961,7 +3210,7 @@ Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
2961
3210
 
2962
3211
  Examples:
2963
3212
 
2964
- - Upload 20 gibibytes of random data to file myfile to directory /Upload
3213
+ - Upload 20 gibibyte of random data to file `myfile` to directory /Upload
2965
3214
 
2966
3215
  ```bash
2967
3216
  ascli server upload faux:///myfile\?20g --to-folder=/Upload
@@ -2984,7 +3233,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
2984
3233
  ```text
2985
3234
  ascli -h
2986
3235
  NAME
2987
- ascli -- a command line tool for Aspera Applications (v4.14.0)
3236
+ ascli -- a command line tool for Aspera Applications (v4.15.0)
2988
3237
 
2989
3238
  SYNOPSIS
2990
3239
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -2997,7 +3246,6 @@ DESCRIPTION
2997
3246
  source repo: https://github.com/IBM/aspera-cli
2998
3247
 
2999
3248
  ENVIRONMENT VARIABLES
3000
- ASCLI_HOME config folder, default: $HOME/.aspera/ascli
3001
3249
  Any option can be set as an environment variable, refer to the manual
3002
3250
 
3003
3251
  COMMANDS
@@ -3007,7 +3255,7 @@ COMMANDS
3007
3255
  OPTIONS
3008
3256
  Options begin with a '-' (minus), and value is provided on command line.
3009
3257
  Special values are supported beginning with special prefix @pfx:, where pfx is one of:
3010
- base64, csvt, env, file, json, lines, list, path, ruby, secret, stdin, uri, val, zlib, preset, incps, vault
3258
+ val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
3011
3259
  Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
3012
3260
 
3013
3261
  ARGS
@@ -3018,8 +3266,8 @@ OPTIONS: global
3018
3266
  --ask-options=ENUM Ask even optional options: [no], yes
3019
3267
  --format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
3020
3268
  --display=ENUM Output only some information: [info], data, error
3021
- --fields=VALUE Comma separated list of fields, or ALL, or DEF
3022
- --select=VALUE Select only some items in lists: column, value (Hash)
3269
+ --fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
3270
+ --select=VALUE Select only some items in lists: column, value (Hash, Proc)
3023
3271
  --table-style=VALUE Table display style
3024
3272
  --flat-hash=ENUM Display deep values as additional keys: no, [yes]
3025
3273
  --transpose-single=ENUM Single object fields output vertically: no, [yes]
@@ -3027,89 +3275,93 @@ OPTIONS: global
3027
3275
  -h, --help Show this message
3028
3276
  --bash-comp Generate bash completion for command
3029
3277
  --show-config Display parameters used for the provided action
3030
- -r, --rest-debug More debug for HTTP calls (REST)
3031
3278
  -v, --version Display version
3032
3279
  -w, --warnings Check for language warnings
3033
3280
  --ui=ENUM Method to start browser: text, [graphical]
3034
- --log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
3281
+ --log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
3035
3282
  --logger=ENUM Logging method: [stderr], stdout, syslog
3036
- --lock-port=VALUE Prevent dual execution of a command, e.g. in cron
3037
- --http-options=VALUE Options for http socket (Hash)
3038
- --insecure=ENUM Do not validate HTTPS certificate: [no], yes
3283
+ --lock-port=VALUE Prevent dual execution of a command, e.g. in cron (Integer)
3039
3284
  --once-only=ENUM Process only new items (some commands): [no], yes
3040
3285
  --log-secrets=ENUM Show passwords in logs: [no], yes
3041
- --cache-tokens=ENUM Save and reuse Oauth tokens: no, [yes]
3286
+ --clean-temp=ENUM Cleanup temporary files on exit: no, [yes]
3287
+ --pid-file=VALUE Write process identifier to file, delete on exit (String)
3042
3288
 
3043
3289
  COMMAND: config
3044
- SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugin preset proxy_check smtp_settings vault wizard
3290
+ SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check remote_certificate smtp_settings throw vault wizard
3045
3291
  OPTIONS:
3292
+ --home=VALUE Home folder for tool (String)
3293
+ --config-file=VALUE Path to YAML file with preset configuration
3294
+ --secret=VALUE Secret for access keys
3295
+ --vault=VALUE Vault for secrets (Hash)
3296
+ --vault-password=VALUE Vault password
3046
3297
  --query=VALUE Additional filter for for some commands (list/delete) (Hash)
3047
- --value=VALUE Value for create, update, list filter (Hash) (deprecated: Use positional value for create/modify or option: query for list/delete)
3298
+ --value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
3048
3299
  --property=VALUE Name of property to set (modify operation)
3049
- --id=VALUE Resource identifier (deprecated: Use identifier after verb (modify,delete,show))
3300
+ --id=VALUE Resource identifier (deprecated: (4.14) Use positional identifier after verb (modify,delete,show))
3050
3301
  --bulk=ENUM Bulk operation (only some): [no], yes
3051
3302
  --bfail=ENUM Bulk operation error handling: no, [yes]
3052
- --config-file=VALUE Read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
3053
3303
  -N, --no-default Do not load default configuration for plugin
3304
+ -P, --presetVALUE Load the named option preset from current config file
3305
+ --version-check-days=VALUE Period in days to check new version (zero to disable)
3306
+ --plugin-folder=VALUE Folder where to find additional plugins
3054
3307
  --override=ENUM Wizard: override existing value: [no], yes
3055
- --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: no, [yes]
3056
3308
  --default=ENUM Wizard: set as default configuration for specified plugin (also: update): no, [yes]
3057
3309
  --test-mode=ENUM Wizard: skip private key check step: [no], yes
3058
- -P, --presetVALUE Load the named option preset from current config file
3059
- --pkeypath=VALUE Wizard: path to private key for JWT
3310
+ --key-path=VALUE Wizard: path to private key for JWT
3060
3311
  --ascp-path=VALUE Path to ascp
3061
3312
  --use-product=VALUE Use ascp from specified product
3062
- --smtp=VALUE SMTP configuration (Hash)
3063
- --fpac=VALUE Proxy auto configuration script
3064
- --proxy-credentials=VALUE HTTP proxy credentials (Array with user and password)
3065
- --secret=VALUE Secret for access keys
3066
- --vault=VALUE Vault for secrets
3067
- --vault-password=VALUE Vault password
3068
3313
  --sdk-url=VALUE URL to get SDK
3069
3314
  --sdk-folder=VALUE SDK folder path
3070
- --notif-to=VALUE Email recipient for notification of transfers
3071
- --notif-template=VALUE Email ERB template for notification of transfers
3072
- --version-check-days=VALUE Period in days to check new version (zero to disable)
3073
- --plugin-folder=VALUE Folder where to find additional plugins
3315
+ --progress-bar=ENUM Display progress bar: [no], yes
3316
+ --smtp=VALUE SMTP configuration (Hash)
3317
+ --notify-to=VALUE Email recipient for notification of transfers
3318
+ --notify-template=VALUE Email ERB template for notification of transfers
3319
+ --insecure=ENUM Do not validate any HTTPS certificate: [no], yes
3320
+ --ignore-certificate=VALUE List of HTTPS url where to no validate certificate (Array)
3321
+ --cert-stores=VALUE List of folder with trusted certificates (Array, String)
3322
+ --http-options=VALUE Options for HTTP/S socket (Hash)
3323
+ --cache-tokens=ENUM Save and reuse Oauth tokens: no, [yes]
3324
+ --fpac=VALUE Proxy auto configuration script
3325
+ --proxy-credentials=VALUE HTTP proxy credentials (user and password) (Array)
3074
3326
  --ts=VALUE Override transfer spec values (Hash)
3075
3327
  --to-folder=VALUE Destination folder for transferred files
3076
3328
  --sources=VALUE How list of transferred files is provided (@args,@ts,Array)
3077
3329
  --src-type=ENUM Type of file list: [list], pair
3078
3330
  --transfer=ENUM Type of transfer agent: [direct], node, connect, httpgw, trsdk
3079
3331
  --transfer-info=VALUE Parameters for transfer agent (Hash)
3080
- --progress=ENUM Type of progress bar: none, [native], multi
3081
3332
 
3082
3333
 
3083
3334
  COMMAND: shares
3084
3335
  SUBCOMMANDS: admin files health
3085
3336
  OPTIONS:
3086
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3337
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3087
3338
  --username=VALUE Username to log in
3088
3339
  --password=VALUE User's password
3089
3340
  --type=ENUM Type of user/group for operations: [any], local, ldap, saml
3090
3341
 
3091
3342
 
3092
3343
  COMMAND: node
3093
- SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
3344
+ SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service slash space ssync stream sync transfer upload watch_folder
3094
3345
  OPTIONS:
3095
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3346
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3096
3347
  --username=VALUE Username to log in
3097
3348
  --password=VALUE User's password
3098
3349
  --validator=VALUE Identifier of validator (optional for central)
3099
3350
  --asperabrowserurl=VALUE URL for simple aspera web ui
3100
3351
  --sync-name=VALUE Sync name
3101
3352
  --default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
3353
+ --root-id=VALUE File id of top folder if using bearer tokens
3354
+ --sync-info=VALUE Information for sync instance and sessions (Hash)
3102
3355
 
3103
3356
 
3104
3357
  COMMAND: orchestrator
3105
3358
  SUBCOMMANDS: health info plugins processes workflow
3106
3359
  OPTIONS:
3107
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3360
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3108
3361
  --username=VALUE Username to log in
3109
3362
  --password=VALUE User's password
3110
- --params=VALUE Start parameters (Hash)
3111
- --result=VALUE Specify result value as: 'work step:parameter'
3112
- --synchronous=ENUM Work step:parameter expected as result: [no], yes
3363
+ --result=VALUE Specify result value as: 'work_step:parameter'
3364
+ --synchronous=ENUM Wait for completion: [no], yes
3113
3365
  --ret-style=ENUM How return type is requested in api: header, [arg], ext
3114
3366
  --auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
3115
3367
 
@@ -3117,7 +3369,7 @@ OPTIONS:
3117
3369
  COMMAND: bss
3118
3370
  SUBCOMMANDS: subscription
3119
3371
  OPTIONS:
3120
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3372
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3121
3373
  --username=VALUE Username to log in
3122
3374
  --password=VALUE User's password
3123
3375
 
@@ -3125,7 +3377,7 @@ OPTIONS:
3125
3377
  COMMAND: alee
3126
3378
  SUBCOMMANDS: entitlement
3127
3379
  OPTIONS:
3128
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3380
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3129
3381
  --username=VALUE Username to log in
3130
3382
  --password=VALUE User's password
3131
3383
 
@@ -3145,19 +3397,18 @@ OPTIONS:
3145
3397
  COMMAND: faspex5
3146
3398
  SUBCOMMANDS: admin bearer_token gateway health packages postprocessing shared_folders user version
3147
3399
  OPTIONS:
3148
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3400
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3149
3401
  --username=VALUE Username to log in
3150
3402
  --password=VALUE User's password
3151
3403
  --client-id=VALUE OAuth client identifier
3152
3404
  --client-secret=VALUE OAuth client secret
3153
3405
  --redirect-uri=VALUE OAuth redirect URI for web authentication
3154
- --auth=ENUM OAuth type of authentication: boot, link, web, [jwt]
3406
+ --auth=ENUM OAuth type of authentication: boot, web, [jwt]
3155
3407
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
3156
3408
  --passphrase=VALUE OAuth JWT RSA private key passphrase
3157
- --link=VALUE Public link authorization (specific operations)
3158
3409
  --box=VALUE Package inbox, either shared inbox name or one of ["inbox", "inbox_history", "inbox_all", "inbox_all_history", "outbox", "outbox_history", "pending", "pending_history", "all"] or ALL
3159
3410
  --shared-folder=VALUE Send package with files from shared folder
3160
- --group-type=ENUM Shared inbox or workgroup: [shared_inboxes], workgroups
3411
+ --group-type=ENUM Type of shared box: [shared_inboxes], workgroups
3161
3412
 
3162
3413
 
3163
3414
  COMMAND: cos
@@ -3175,21 +3426,21 @@ OPTIONS:
3175
3426
  COMMAND: faspex
3176
3427
  SUBCOMMANDS: address_book dropbox health login_methods me package source v4
3177
3428
  OPTIONS:
3178
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3429
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3179
3430
  --username=VALUE Username to log in
3180
3431
  --password=VALUE User's password
3181
3432
  --link=VALUE Public link for specific operation
3182
3433
  --delivery-info=VALUE Package delivery information (Hash)
3183
- --source-name=VALUE Create package from remote source (by name)
3184
- --storage=VALUE Faspex local storage definition
3434
+ --remote-source=VALUE Remote source for package send (id or %name:)
3435
+ --storage=VALUE Faspex local storage definition (for browsing source)
3185
3436
  --recipient=VALUE Use if recipient is a dropbox (with *)
3186
3437
  --box=ENUM Package box: [inbox], archive, sent
3187
3438
 
3188
3439
 
3189
3440
  COMMAND: preview
3190
- SUBCOMMANDS: check events scan test trevents
3441
+ SUBCOMMANDS: check events scan show test trevents
3191
3442
  OPTIONS:
3192
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3443
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3193
3444
  --username=VALUE Username to log in
3194
3445
  --password=VALUE User's password
3195
3446
  --skip-format=ENUM Skip this preview format (multiple possible): png, mp4
@@ -3198,7 +3449,7 @@ OPTIONS:
3198
3449
  --previews-folder=VALUE Preview folder in storage root
3199
3450
  --temp-folder=VALUE Path to temp folder
3200
3451
  --skip-folders=VALUE List of folder to skip
3201
- --case=VALUE Basename of output for for test
3452
+ --base=VALUE Basename of output for for test
3202
3453
  --scan-path=VALUE Subpath in folder id to start scan in (default=/)
3203
3454
  --scan-id=VALUE Folder id in storage to start scan in, default is access key main folder id
3204
3455
  --mimemagic=ENUM Use Mime type detection of gem mimemagic: [no], yes
@@ -3222,58 +3473,40 @@ OPTIONS:
3222
3473
  --clips-length=VALUE Mp4: clips: length in seconds of each clips
3223
3474
 
3224
3475
 
3225
- COMMAND: sync
3226
- SUBCOMMANDS: admin start
3227
- OPTIONS:
3228
- --sync-info=VALUE Information for sync instance and sessions (Hash)
3229
- --sync-session=VALUE Name of session to use for admin commands. default: first in parameters
3230
-
3231
-
3232
3476
  COMMAND: aoc
3233
3477
  SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
3234
3478
  OPTIONS:
3235
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3479
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3236
3480
  --username=VALUE Username to log in
3237
3481
  --password=VALUE User's password
3238
3482
  --auth=ENUM OAuth type of authentication: web, [jwt]
3239
- --operation=ENUM Client operation for transfers: [push], pull
3240
3483
  --client-id=VALUE OAuth API client identifier
3241
3484
  --client-secret=VALUE OAuth API client secret
3485
+ --scope=VALUE OAuth scope for AoC API calls
3242
3486
  --redirect-uri=VALUE OAuth API client redirect URI
3243
3487
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
3244
- --scope=VALUE OAuth scope for AoC API calls
3245
3488
  --passphrase=VALUE RSA private key passphrase
3246
- --workspace=VALUE Name of workspace
3247
- --name=VALUE Resource name (prefer to use keyword name)
3248
- --link=VALUE Public link to shared resource
3489
+ --workspace=VALUE Name of workspace (String, NilClass)
3249
3490
  --new-user-option=VALUE New user creation option for unknown package recipients
3250
- --from-folder=VALUE Source folder for Folder-to-Folder transfer
3251
3491
  --validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
3252
3492
 
3253
- COMMAND: node
3254
- SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
3255
- OPTIONS:
3256
- --validator=VALUE Identifier of validator (optional for central)
3257
- --asperabrowserurl=VALUE URL for simple aspera web ui
3258
- --sync-name=VALUE Sync name
3259
- --default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
3260
-
3261
3493
 
3262
3494
  COMMAND: server
3263
3495
  SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
3264
3496
  OPTIONS:
3265
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3497
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3266
3498
  --username=VALUE Username to log in
3267
3499
  --password=VALUE User's password
3268
3500
  --ssh-keys=VALUE SSH key path list (Array or single)
3269
3501
  --passphrase=VALUE SSH private key passphrase
3270
3502
  --ssh-options=VALUE SSH options (Hash)
3503
+ --sync-info=VALUE Information for sync instance and sessions (Hash)
3271
3504
 
3272
3505
 
3273
3506
  COMMAND: console
3274
3507
  SUBCOMMANDS: health transfer
3275
3508
  OPTIONS:
3276
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
3509
+ --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3277
3510
  --username=VALUE Username to log in
3278
3511
  --password=VALUE User's password
3279
3512
  --filter-from=DATE Only after date
@@ -3286,19 +3519,74 @@ OPTIONS:
3286
3519
 
3287
3520
  ### Bulk creation and deletion of resources
3288
3521
 
3289
- Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
3290
- In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
3522
+ Bulk creation and deletion of resources are possible using option `bulk` (`yes`,`no`(default)).
3523
+ In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended).
3291
3524
  This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
3292
3525
 
3526
+ ### Plugins
3527
+
3528
+ `ascli` uses a plugin mechanism.
3529
+ The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
3530
+ Each plugin usually represents commands sent to a specific application.
3531
+ For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
3532
+
3533
+ Available plugins can be found using command:
3534
+
3535
+ ```bash
3536
+ ascli conf plugin list
3537
+ ```
3538
+
3539
+ ```output
3540
+ +--------------+--------+--------+-------------------------------------------------------+
3541
+ | plugin | detect | wizard | path |
3542
+ +--------------+--------+--------+-------------------------------------------------------+
3543
+ | shares | Y | Y | .../aspera-cli/lib/aspera/cli/plugins/shares.rb |
3544
+ | node | Y | Y | .../aspera-cli/lib/aspera/cli/plugins/node.rb |
3545
+ ...
3546
+ +--------------+--------+--------+-------------------------------------------------------+
3547
+ ```
3548
+
3549
+ Most plugins will take the URL option: `url` to identify their location.
3550
+
3551
+ REST APIs of Aspera legacy applications (Aspera Node, Faspex 4, Shares, Console, Orchestrator) use simple username/password authentication: HTTP Basic Authentication using options: `username` and `password`.
3552
+
3553
+ Aspera on Cloud and Faspex 5 rely on Oauth.
3554
+
3555
+ By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
3556
+
3557
+ ```bash
3558
+ ascli --show-config --select=@json:'{"key":"plugin_folder"}'
3559
+ ```
3560
+
3561
+ You can create the skeleton of a new plugin like this:
3562
+
3563
+ ```bash
3564
+ ascli conf plugin create foo .
3565
+ ```
3566
+
3567
+ ```output
3568
+ Created ./foo.rb
3569
+ ```
3570
+
3571
+ ```bash
3572
+ ascli --plugin-folder=. foo
3573
+ ```
3574
+
3293
3575
  ## <a id="aoc"></a>Plugin: `aoc`: IBM Aspera on Cloud
3294
3576
 
3295
- Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
3577
+ Aspera on Cloud API requires the use of Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
3578
+
3579
+ It is recommended to use the wizard to set it up, although manual configuration is also possible.
3580
+
3581
+ ### <a id="aocwizard"></a>Configuration: Using Wizard
3582
+
3583
+ `ascli` provides a configuration wizard.
3296
3584
 
3297
- It is recommended to use the wizard to set it up, but manual configuration is also possible.
3585
+ The wizard guides you through the steps to create a new configuration preset for Aspera on Cloud.
3298
3586
 
3299
- ### <a id="aocwizard"></a>Configuration: using Wizard
3587
+ The first
3300
3588
 
3301
- `ascli` provides a configuration wizard. Here is a sample invocation :
3589
+ Here is a sample invocation :
3302
3590
 
3303
3591
  ```text
3304
3592
  ascli config wizard
@@ -3306,7 +3594,7 @@ option: url> https://myorg.ibmaspera.com
3306
3594
  Detected: Aspera on Cloud
3307
3595
  Preparing preset: aoc_myorg
3308
3596
  Please provide path to your private RSA key, or empty to generate one:
3309
- option: pkeypath>
3597
+ option: key_path>
3310
3598
  using existing key:
3311
3599
  /Users/myself/.aspera/ascli/aspera_aoc_key
3312
3600
  Using global client_id.
@@ -3323,15 +3611,13 @@ ascli aoc user profile show
3323
3611
  Optionally, it is possible to create a new organization-specific "integration", i.e. client application identification.
3324
3612
  For this, specify the option: `--use-generic-client=no`.
3325
3613
 
3326
- This will guide you through the steps to create.
3327
-
3328
- If the wizard does not detect the application but you know the application, you can force it using option `query`:
3614
+ If you already know the application, and want to limit the detection to it, provide url and plugin name:
3329
3615
 
3330
3616
  ```bash
3331
- ascli config wizard --query=aoc
3617
+ ascli config wizard myorg aoc
3332
3618
  ```
3333
3619
 
3334
- ### <a id="aocmanual"></a>Configuration: using manual setup
3620
+ ### <a id="aocmanual"></a>Configuration: Using manual setup
3335
3621
 
3336
3622
  > **Note:** If you used the wizard (recommended): skip this section.
3337
3623
 
@@ -3345,11 +3631,13 @@ Several types of OAuth authentication are supported:
3345
3631
 
3346
3632
  The authentication method is controlled by option `auth`.
3347
3633
 
3348
- For a *quick start*, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
3634
+ For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
3349
3635
 
3350
3636
  For a more convenient, browser-less, experience follow the [JWT](#jwt) section (auth=jwt) in addition to Client Registration.
3351
3637
 
3352
- In Oauth, a "Bearer" token are generated to authenticate REST calls. Bearer tokens are valid for a period of time.`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
3638
+ In Oauth, a "Bearer" token is generated to authenticate REST calls.
3639
+ Bearer tokens are valid for a period of time defined (by the AoC app, configurable by admin) at its creation.
3640
+ `ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
3353
3641
 
3354
3642
  #### <a id="clientreg"></a>Optional: API Client Registration
3355
3643
 
@@ -3371,7 +3659,7 @@ Let's start by a registration with web based authentication (auth=web):
3371
3659
  - leave the JWT part for now
3372
3660
  - Save
3373
3661
 
3374
- Note: for web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
3662
+ > **Note:** for web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
3375
3663
 
3376
3664
  Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
3377
3665
 
@@ -3397,7 +3685,7 @@ Define this [option preset](#lprt) as default configuration for the `aspera` plu
3397
3685
  ascli config preset set default aoc my_aoc_org
3398
3686
  ```
3399
3687
 
3400
- Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
3688
+ > **Note:** Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
3401
3689
 
3402
3690
  #### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
3403
3691
 
@@ -3452,7 +3740,7 @@ Open the previously generated public key located here: `$HOME/.aspera/ascli/my_p
3452
3740
  - Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
3453
3741
  - Click on the user's icon (top right)
3454
3742
  - Select "Account Settings"
3455
- - Paste the *Public Key* in the "Public Key" section
3743
+ - Paste the **Public Key** in the "Public Key" section
3456
3744
  - Click on "Submit"
3457
3745
 
3458
3746
  ##### Using command line
@@ -3478,7 +3766,7 @@ ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("
3478
3766
  modified
3479
3767
  ```
3480
3768
 
3481
- Note: the `aspera user info show` command can be used to verify modifications.
3769
+ > **Note:** the `aspera user info show` command can be used to verify modifications.
3482
3770
 
3483
3771
  #### [option preset](#lprt) modification for JWT
3484
3772
 
@@ -3491,13 +3779,26 @@ To activate default use of JWT authentication for `ascli` using the [option pres
3491
3779
  Execute:
3492
3780
 
3493
3781
  ```bash
3494
- ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=laurent.martin.aspera@fr.ibm.com
3782
+ ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=someuser@example.com
3495
3783
  ```
3496
3784
 
3497
- Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the `@file:` prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the `@val:` prefix is added.
3785
+ > **Note:** the private key argument represents the actual PEM string. In order to read the content from a file, use the `@file:` prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the `@val:` prefix is added.
3498
3786
 
3499
3787
  After this last step, commands do not require web login anymore.
3500
3788
 
3789
+ #### Public and private links
3790
+
3791
+ AoC gives the possibility to generate public links for both the `Files` and `Packages` modules.
3792
+ Public links embed the authorization of access.
3793
+ Provide the public link using option `url` alone.
3794
+
3795
+ In addition, the `Files` application supports private links.
3796
+ Private links require the user to authenticate.
3797
+ So, provide the same options as for regular authentication, and provide the private link using option `url`.
3798
+
3799
+ A user may not be part of any workspace, but still have access to shared folders (using private links).
3800
+ In that case, it is possible to list those shared folder by using a value for option `workspace` equal to `@none:` or `@json:null` or `@ruby:nil`.
3801
+
3501
3802
  #### <a id="aocfirst"></a>First Use
3502
3803
 
3503
3804
  Once client has been registered and [option preset](#lprt) created: `ascli` can be used:
@@ -3542,7 +3843,9 @@ It allows actions (create, update, delete) on "resources": users, group, nodes,
3542
3843
 
3543
3844
  The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
3544
3845
 
3545
- The option `query` can be optionally used. It expects a Hash using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`. Values are directly sent to the API call and used as a filter on server side.
3846
+ The option `query` can be optionally used.
3847
+ It expects a `Hash` using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`.
3848
+ Values are directly sent to the API call and used as a filter on server side.
3546
3849
 
3547
3850
  The following parameters are supported:
3548
3851
 
@@ -3569,7 +3872,7 @@ Examples:
3569
3872
  - List users with `laurent` in name:
3570
3873
 
3571
3874
  ```bash
3572
- ascli aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
3875
+ ascli aoc admin res user list --query=@json:'{"q":"laurent"}'
3573
3876
  ```
3574
3877
 
3575
3878
  - List users who logged-in before a date:
@@ -3594,8 +3897,8 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
3594
3897
 
3595
3898
  To execute an action on a specific resource, select it using one of those methods:
3596
3899
 
3597
- - *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
3598
- - give name on command line *after the action*: `aoc admin res node show name abc`
3900
+ - **recommended**: give id directly on command line **after the action**: `aoc admin res node show 123`
3901
+ - give name on command line **after the action**: `aoc admin res node show name abc`
3599
3902
  - provide option `id` : `aoc admin res node show 123`
3600
3903
  - provide option `name` : `aoc admin res node show --name=abc`
3601
3904
 
@@ -3664,7 +3967,7 @@ The activity app can be queried with:
3664
3967
  ascli aoc admin analytics transfers
3665
3968
  ```
3666
3969
 
3667
- It can also support filters and send notification using option `notif_to`. a template is defined using option `notif_template` :
3970
+ It can also support filters and send notification using option `notify_to`. a template is defined using option `notify_template` :
3668
3971
 
3669
3972
  `mytemplate.erb`:
3670
3973
 
@@ -3687,9 +3990,7 @@ The environment provided contains the following additional variable:
3687
3990
  Example:
3688
3991
 
3689
3992
  ```bash
3690
- ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 \
3691
- --query=@json:'{"status":"completed","direction":"receive"}' \
3692
- --notif-to=active --notif-template=@file:mytemplate.erb
3993
+ ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=active --notify-template=@file:mytemplate.erb
3693
3994
  ```
3694
3995
 
3695
3996
  Options:
@@ -3770,19 +4071,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,emai
3770
4071
  ```
3771
4072
 
3772
4073
  ```bash
3773
- thelist=$(ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
3774
- ```
3775
-
3776
- ```bash
3777
- echo $thelist
3778
- ```
3779
-
3780
- ```json
3781
- ["113501","354061"]
3782
- ```
3783
-
3784
- ```bash
3785
- ascli aoc admin res user delete @json:"$thelist" --bulk=yes
4074
+ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --display=data --format=csv | ascli aoc admin res user delete @lines:@stdin: --bulk=yes
3786
4075
  ```
3787
4076
 
3788
4077
  ```output
@@ -3832,7 +4121,7 @@ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key
3832
4121
  ascli aoc admin res node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
3833
4122
  ```
3834
4123
 
3835
- Examples of query (TODO: cleanup):
4124
+ Examples of query:
3836
4125
 
3837
4126
  ```json
3838
4127
  {"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
@@ -3859,7 +4148,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
3859
4148
  | member_type | manager | member.email |
3860
4149
  +-------------+---------+----------------------------------+
3861
4150
  | user | true | john.curtis@email.com |
3862
- | user | false | laurent.martin.aspera@fr.ibm.com |
4151
+ | user | false | someuser@example.com |
3863
4152
  | user | false | jean.dupont@me.com |
3864
4153
  | user | false | another.user@example.com |
3865
4154
  | group | false | |
@@ -3934,7 +4223,7 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
3934
4223
  ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
3935
4224
  ```
3936
4225
 
3937
- #### Example: create a group, add to workspace and add user to group
4226
+ #### Example: Create a group, add to workspace and add user to group
3938
4227
 
3939
4228
  - Create the group and take note of `id`
3940
4229
 
@@ -3979,7 +4268,7 @@ In this example, a user has access to a workspace where two shared folders are l
3979
4268
  First, setup the environment (skip if already done)
3980
4269
 
3981
4270
  ```bash
3982
- ascli conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
4271
+ ascli conf wizard --url=https://sedemo.ibmaspera.com --username=someuser@example.com
3983
4272
  ```
3984
4273
 
3985
4274
  ```output
@@ -4014,7 +4303,7 @@ Then, transfer between those:
4014
4303
  ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
4015
4304
  ```
4016
4305
 
4017
- #### Example: create registration key to register a node
4306
+ #### Example: Create registration key to register a node
4018
4307
 
4019
4308
  ```bash
4020
4309
  ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
@@ -4024,7 +4313,7 @@ ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_s
4024
4313
  jfqslfdjlfdjfhdjklqfhdkl
4025
4314
  ```
4026
4315
 
4027
- #### Example: delete all registration keys
4316
+ #### Example: Delete all registration keys
4028
4317
 
4029
4318
  ```bash
4030
4319
  ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete @lines:@stdin: --bulk=yes
@@ -4105,7 +4394,7 @@ In this case:
4105
4394
 
4106
4395
  ### Packages
4107
4396
 
4108
- The webmail-like application.
4397
+ The web-mail-like application.
4109
4398
 
4110
4399
  #### Send a Package
4111
4400
 
@@ -4117,7 +4406,7 @@ ascli aoc packages send [package extended value] [other parameters such as file
4117
4406
 
4118
4407
  Notes:
4119
4408
 
4120
- - Package creation parameter are sent as positional mandatory parameter.
4409
+ - Package creation parameter are sent as positional argument.
4121
4410
  Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
4122
4411
  - List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
4123
4412
  - Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
@@ -4130,7 +4419,7 @@ Notes:
4130
4419
  #### Example: Send a package with one file to two users, using their email
4131
4420
 
4132
4421
  ```bash
4133
- ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
4422
+ ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["someuser@example.com","other@example.com"]}' my_file.dat
4134
4423
  ```
4135
4424
 
4136
4425
  #### Example: Send a package to a shared inbox with metadata
@@ -4166,7 +4455,7 @@ ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archiv
4166
4455
  Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
4167
4456
 
4168
4457
  ```bash
4169
- shared_box_id=$(ascli aoc packages shared_inboxes show name 'My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
4458
+ shared_box_id=$(ascli aoc packages shared_inboxes show --name='My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
4170
4459
  ```
4171
4460
 
4172
4461
  ```bash
@@ -4187,21 +4476,21 @@ Find files in Files app:
4187
4476
  ascli aoc files browse /src_folder
4188
4477
  ```
4189
4478
 
4190
- ```bash
4191
- +------------------------------+--------+----------------+--------------+----------------------+--------------+
4192
- | name | type | recursive_size | size | modified_time | access_level |
4193
- +------------------------------+--------+----------------+--------------+----------------------+--------------+
4194
- | sample_video | link | | | 2020-11-29T22:49:09Z | edit |
4195
- | 100G | file | | 107374182400 | 2021-04-21T18:19:25Z | edit |
4196
- | 10M.dat | file | | 10485760 | 2021-05-18T08:22:39Z | edit |
4197
- | Test.pdf | file | | 1265103 | 2022-06-16T12:49:55Z | edit |
4198
- +------------------------------+--------+----------------+--------------+----------------------+--------------+
4479
+ ```text
4480
+ +---------------+--------+----------------+--------------+----------------------+--------------+
4481
+ | name | type | recursive_size | size | modified_time | access_level |
4482
+ +---------------+--------+----------------+--------------+----------------------+--------------+
4483
+ | sample_video | link | | | 2020-11-29T22:49:09Z | edit |
4484
+ | 100G | file | | 107374182400 | 2021-04-21T18:19:25Z | edit |
4485
+ | 10M.dat | file | | 10485760 | 2021-05-18T08:22:39Z | edit |
4486
+ | Test.pdf | file | | 1265103 | 2022-06-16T12:49:55Z | edit |
4487
+ +---------------+--------+----------------+--------------+----------------------+--------------+
4199
4488
  ```
4200
4489
 
4201
4490
  Let's send a package with the file `10M.dat` from subfolder /src_folder in a package:
4202
4491
 
4203
4492
  ```bash
4204
- ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["laurent.martin.aspera@fr.ibm.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
4493
+ ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
4205
4494
  ```
4206
4495
 
4207
4496
  #### <a id="aoccargo"></a>Receive new packages only (Cargo)
@@ -4223,6 +4512,8 @@ Typically, one would execute this command on a regular basis, using the method o
4223
4512
  The Files application presents a **Home** folder to users in a given workspace.
4224
4513
  Files located here are either user's files, or shared folders.
4225
4514
 
4515
+ > **Note:** All commands under `files` are the same as under `access_keys do self` for plugin `node`, i.e. **gen4/access key** operations.
4516
+
4226
4517
  #### Download Files
4227
4518
 
4228
4519
  The general download command is:
@@ -4242,8 +4533,10 @@ ascli aoc files download <single file path>
4242
4533
  #### Shared folders
4243
4534
 
4244
4535
  Shared folder created by users are managed through **permissions**.
4245
- For creation, parameters are the same as for node api [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
4536
+
4537
+ For creation, parameters are the same as for node API [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
4246
4538
  `ascli` expects the same payload for creation, but it will automatically populate required tags if needed.
4539
+
4247
4540
  Also, the pseudo key `with` is available: it will lookup the name in the contacts and fill the proper type and id.
4248
4541
  The pseudo parameter `link_name` allows changing default "shared as" name.
4249
4542
 
@@ -4265,6 +4558,15 @@ ascli aoc files perm /shared_folder_test1 create @json:'{"with":"laurent"}'
4265
4558
  ascli aoc files perm /shared_folder_test1 delete 6161
4266
4559
  ```
4267
4560
 
4561
+ Public and Private short links can be managed with command:
4562
+
4563
+ ```bash
4564
+ ascli aoc files short_link private create _path_here_
4565
+ ascli aoc files short_link private list _path_here_
4566
+ ascli aoc files short_link public list _path_here_
4567
+ ascli aoc files short_link public delete _id_
4568
+ ```
4569
+
4268
4570
  #### Cross Organization transfers
4269
4571
 
4270
4572
  It is possible to transfer files directly between organizations without having to first download locally and then upload...
@@ -4297,59 +4599,22 @@ Explanation:
4297
4599
 
4298
4600
  #### Find Files
4299
4601
 
4300
- The command `aoc files find [--query=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
4301
-
4302
- The expression can be of 3 formats:
4303
-
4304
- - empty (default) : all files, equivalent to value: `exec:true`
4305
- - not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax, equivalent to value: `exec:f['name'].match(/expression/)`
4306
-
4307
- For instance, to find files with a special extension, use `--query='\.myext$'`
4308
-
4309
- - starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true.
4310
-
4311
- Examples of expressions: (using like this: `--query=exec:'<expression>'`)
4312
-
4313
- - Find files more recent than 100 days
4314
-
4315
- ```ruby
4316
- f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
4317
- ```
4318
-
4319
- - Find files older than 1 year on a given node and store in file list
4320
-
4321
- ```ruby
4322
- f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
4323
- ```
4324
-
4325
- ```bash
4326
- ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find / --fields=path --query='exec:<above expression here>' --format=csv > my_file_list.txt
4327
- ```
4328
-
4329
- - Find files larger than 1MB
4330
-
4331
- ```ruby
4332
- f["type"].eql?("file") and f["size"].to_i>1000000
4333
- ```
4334
-
4335
- - Delete the files, one by one
4602
+ The command `aoc files find` allows to search for files in a given workspace.
4336
4603
 
4337
- ```bash
4338
- cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 delete "$path" ;done
4339
- ```
4604
+ It works also on `node` resource using the `v4` command:
4340
4605
 
4341
- - Delete the files in bulk
4606
+ ```bash
4607
+ ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find ...
4608
+ ```
4342
4609
 
4343
- ```bash
4344
- cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my_secret_here' v3 delete @lines:@stdin:
4345
- ```
4610
+ For instructions, refer to section `find` for plugin `node`.
4346
4611
 
4347
4612
  ### AoC sample commands
4348
4613
 
4349
4614
  ```bash
4350
- aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
4351
- aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
4352
- aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4615
+ aoc admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
4616
+ aoc admin ats access_key create --cloud=aws --region=my_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
4617
+ aoc admin ats access_key create --cloud=softlayer --region=my_bucket_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
4353
4618
  aoc admin ats access_key delete ak1ibmcloud
4354
4619
  aoc admin ats access_key list --fields=name,id
4355
4620
  aoc admin ats access_key node ak1ibmcloud --secret=my_secret_here browse /
@@ -4362,7 +4627,7 @@ aoc admin res application list
4362
4627
  aoc admin res client list
4363
4628
  aoc admin res client_access_key list
4364
4629
  aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
4365
- aoc admin res client_registration_token delete my_clt_reg_id
4630
+ aoc admin res client_registration_token delete client_reg_id
4366
4631
  aoc admin res client_registration_token list
4367
4632
  aoc admin res contact list
4368
4633
  aoc admin res dropbox list
@@ -4377,69 +4642,73 @@ aoc admin res saml_configuration list
4377
4642
  aoc admin res self show
4378
4643
  aoc admin res short_link list
4379
4644
  aoc admin res user list
4645
+ aoc admin res user modify %name:my_user_email @json:'{"deactivated":false}'
4380
4646
  aoc admin res workspace_membership list
4381
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do browse /
4382
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do delete /folder1
4383
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do mkdir /folder1
4384
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
4385
- aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 events
4386
- aoc admin resource node do name my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key delete testsub1
4647
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret browse /
4648
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret delete /folder1
4649
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
4650
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
4651
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
4652
+ aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
4387
4653
  aoc admin resource workspace list
4388
4654
  aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
4389
4655
  aoc admin subscription
4390
- aoc automation workflow action my_wf_id create @json:'{"name":"toto"}' \
4656
+ aoc automation workflow action wf_id create @json:'{"name":"toto"}' \
4391
4657
  aoc automation workflow create @json:'{"name":"test_workflow"}'
4392
- aoc automation workflow delete my_wf_id
4658
+ aoc automation workflow delete wf_id
4393
4659
  aoc automation workflow list
4394
4660
  aoc automation workflow list --query=@json:'{"show_org_workflows":"true"}' --scope=admin:all
4395
4661
  aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
4396
4662
  aoc bearer_token --display=data --scope=user:all
4397
- aoc faspex
4398
4663
  aoc files bearer /
4399
4664
  aoc files bearer_token_node / --cache-tokens=no
4400
4665
  aoc files browse /
4401
- aoc files browse / --link=my_aoc_publink_folder
4666
+ aoc files browse / --url=my_private_link
4667
+ aoc files browse / --url=my_public_link_folder_no_pass
4668
+ aoc files browse / --url=my_public_link_folder_pass --password=my_public_link_password
4402
4669
  aoc files delete /testsrc
4403
- aoc files download --transfer=connect /200KB.1
4404
- aoc files find / --query='\.partial$'
4405
- aoc files http_node_download --to-folder=. /200KB.1
4670
+ aoc files download --transfer=connect testdst/test_file.bin
4671
+ aoc files find / '\.partial$'
4672
+ aoc files http_node_download --to-folder=. testdst/test_file.bin
4406
4673
  aoc files mkdir /testsrc
4407
- aoc files modify my_aoc_test_folder
4408
- aoc files permission my_aoc_test_folder list
4409
- aoc files rename /somefolder testdst
4410
- aoc files short_link create /testdst private
4411
- aoc files short_link create testdst public
4412
- aoc files short_link list /testdst private
4413
- aoc files show %id:my_file_id
4414
- aoc files show /200KB.1
4415
- aoc files sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
4416
- aoc files sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
4417
- aoc files sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
4418
- aoc files sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
4419
- aoc files thumbnail my_aoc_media_file
4420
- aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
4421
- aoc files upload --to-folder=/ testfile.bin --link=my_aoc_publink_folder
4422
- aoc files upload --to-folder=/testsrc testfile.bin
4423
- aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
4674
+ aoc files modify my_test_folder
4675
+ aoc files permission my_test_folder list
4676
+ aoc files rename /some_folder testdst
4677
+ aoc files short_link private create /testdst
4678
+ aoc files short_link private list /testdst
4679
+ aoc files short_link public create testdst
4680
+ aoc files show %id:aoc_file_id
4681
+ aoc files show /
4682
+ aoc files show testdst/test_file.bin
4683
+ aoc files sync admin status --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
4684
+ aoc files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
4685
+ aoc files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
4686
+ aoc files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
4687
+ aoc files thumbnail my_test_folder/video_file.mpg
4688
+ aoc files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
4689
+ aoc files transfer push /testsrc --to-folder=/testdst test_file.bin
4690
+ aoc files upload --to-folder=/ test_file.bin --url=my_public_link_folder_no_pass
4691
+ aoc files upload --to-folder=/testsrc test_file.bin
4692
+ aoc files upload --workspace=my_other_workspace --to-folder=my_other_folder test_file.bin --transfer=node --transfer-info=@json:@stdin:
4424
4693
  aoc files v3 info
4425
- aoc gateway https://localhost:12345/aspera/faspex
4426
- aoc org --link=my_aoc_publink_recv_from_aocuser
4694
+ aoc gateway --pid-file=server_pid https://localhost:12345/aspera/faspex
4695
+ aoc org --url=my_public_link_recv_from_aoc_user
4427
4696
  aoc organization
4428
- aoc packages browse "my_package_id" /contents
4697
+ aoc packages browse package_id3 /contents
4429
4698
  aoc packages list
4430
- aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
4431
- aoc packages recv "my_package_id" --to-folder=.
4699
+ aoc packages list --query=@json:'{"dropbox_name":"my_shared_inbox_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
4432
4700
  aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
4433
- aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"my_aoc_shbx_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
4434
- aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
4435
- aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
4436
- aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
4437
- aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
4438
- aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
4439
- aoc packages send @json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
4440
- aoc packages send @json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
4701
+ aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"my_shared_inbox_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
4702
+ aoc packages recv package_id3 --to-folder=.
4703
+ aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' test_file.bin
4704
+ aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
4705
+ aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"]}' test_file.bin
4706
+ aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_external"]}' --new-user-option=@json:'{"package_contact":true}' test_file.bin
4707
+ aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_internal"],"note":"my note"}' test_file.bin
4708
+ aoc packages send @json:'{"name":"Important files delivery"}' test_file.bin --url=my_public_link_send_aoc_user --password=my_public_link_send_use_pass
4709
+ aoc packages send @json:'{"name":"Important files delivery"}' test_file.bin --url=my_public_link_send_shared_inbox
4441
4710
  aoc packages shared_inboxes list
4442
- aoc remind --username=my_aoc_user_email
4711
+ aoc remind --username=my_user_email
4443
4712
  aoc servers
4444
4713
  aoc user profile modify @json:'{"name":"dummy change"}'
4445
4714
  aoc user profile show
@@ -4455,7 +4724,7 @@ ATS is usable either :
4455
4724
 
4456
4725
  - or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
4457
4726
 
4458
- ### IBM Cloud ATS : creation of api key
4727
+ ### IBM Cloud ATS : Creation of api key
4459
4728
 
4460
4729
  This section is about using ATS with an IBM cloud subscription.
4461
4730
  If you are using ATS as part of AoC, then authentication is through AoC, not IBM Cloud.
@@ -4573,8 +4842,8 @@ The parameters provided to ATS for access key creation are the ones of [ATS API]
4573
4842
 
4574
4843
  ```bash
4575
4844
  ats access_key cluster ak2ibmcloud --secret=my_secret_here
4576
- ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
4577
- ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"ak2ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
4845
+ ats access_key create --cloud=aws --region=my_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
4846
+ ats access_key create --cloud=softlayer --region=my_bucket_region --params=@json:'{"id":"ak2ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
4578
4847
  ats access_key delete ak2ibmcloud
4579
4848
  ats access_key delete ak_aws
4580
4849
  ats access_key entitlement ak2ibmcloud
@@ -4603,36 +4872,36 @@ then commands `ascp` (for transfers) and `ascmd` (for file operations) are execu
4603
4872
 
4604
4873
  ```bash
4605
4874
  server browse /
4606
- server browse NEW_SERVER_FOLDER/testfile.bin
4607
- server browse folder_1/target_hot
4608
- server cp NEW_SERVER_FOLDER/testfile.bin folder_1/200KB.2
4609
- server delete NEW_SERVER_FOLDER
4610
- server delete folder_1/target_hot
4611
- server delete folder_1/to.delete
4875
+ server browse my_inside_folder/test_file.bin
4876
+ server browse my_upload_folder/target_hot
4877
+ server cp my_inside_folder/test_file.bin my_upload_folder/200KB.2
4878
+ server delete my_inside_folder
4879
+ server delete my_upload_folder/to.delete
4612
4880
  server df
4613
- server download NEW_SERVER_FOLDER/testfile.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
4614
- server download NEW_SERVER_FOLDER/testfile.bin --to-folder=folder_1 --transfer=node
4881
+ server download my_inside_folder/test_file.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
4882
+ server download my_inside_folder/test_file.bin --to-folder=my_upload_folder --transfer=node
4615
4883
  server du /
4616
- server health transfer --to-folder=folder_1 --format=nagios
4884
+ server health transfer --to-folder=my_upload_folder --format=nagios
4617
4885
  server info
4618
- server md5sum NEW_SERVER_FOLDER/testfile.bin
4619
- server mkdir NEW_SERVER_FOLDER --logger=stdout
4620
- server mkdir folder_1/target_hot
4621
- server mv folder_1/200KB.2 folder_1/to.delete
4622
- server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"}}'
4623
- server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"my_local_sync_dir"}]}'
4624
- server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"},"remote":{"path":"'"NEW_SERVER_FOLDER"'"},"reset":true,"quiet":false}'
4625
- server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"my_local_sync_dir","reset":true}]}'
4626
- server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
4627
- server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
4628
- server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
4629
- server upload --sources=@ts --ts=@json:'{"EX_file_pair_list":"'"filepairlist.txt"'"}'
4630
- server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
4631
- server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
4632
- server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email --transfer-info=@json:'{"ascp_args":["-l","10m"]}'
4633
- server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"my_secret_here","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
4634
- server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
4635
- server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress=multi
4886
+ server md5sum my_inside_folder/test_file.bin
4887
+ server mkdir my_inside_folder --logger=stdout
4888
+ server mkdir my_upload_folder/target_hot
4889
+ server mv my_upload_folder/200KB.2 my_upload_folder/to.delete
4890
+ server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
4891
+ server sync admin status --sync-info=@json:'{"name":"sync2"}'
4892
+ server sync admin status my_sync --sync-info=@json:'{"sessions":[{"name":"my_sync","local_dir":"/data/local_sync"}]}'
4893
+ server sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
4894
+ server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
4895
+ server upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
4896
+ server upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
4897
+ server upload 'test_file.bin' --to-folder=my_inside_folder --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":100000}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress-bar=yes
4898
+ server upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-list","filelist.txt"]}' --to-folder=my_inside_folder
4899
+ server upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-pair-list","file_pair_list.txt"]}'
4900
+ server upload --sources=@ts --ts=@json:'{"paths":[{"source":"test_file.bin","destination":"my_inside_folder/other_name_4"}]}' --transfer=trsdk
4901
+ server upload --src-type=pair 'test_file.bin' my_inside_folder/other_name_2 --notify-to=my_email_external --transfer-info=@json:'{"ascp_args":["-l","100m"]}'
4902
+ server upload --src-type=pair --sources=@json:'["test_file.bin","my_inside_folder/other_name_3"]' --transfer-info=@json:'{"quiet":false}' --progress=no
4903
+ server upload --src-type=pair test_file.bin my_upload_folder/other_name_5 --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"my_secret_here","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
4904
+ server upload --to-folder=my_upload_folder/target_hot --lock-port=12345 --transfer-info=@json:'{"ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
4636
4905
  ```
4637
4906
 
4638
4907
  ### Authentication on Server with SSH session
@@ -4735,21 +5004,106 @@ ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ss
4735
5004
 
4736
5005
  ## <a id="node"></a>Plugin: `node`: IBM Aspera High Speed Transfer Server Node
4737
5006
 
4738
- This plugin gives access to capabilities provided by HSTS node API.
5007
+ This plugin gives access to capabilities provided by the HSTS node API.
4739
5008
 
4740
- > **Note:** capabilities of this plugin are used in other plugins which access to the node API, such as `aoc`.
5009
+ The authentication is `username` and `password` or `access_key` and `secret` through options: `username` and `password`.
5010
+
5011
+ > **Note:** Capabilities of this plugin are used in other plugins which access to the node API, such as `aoc`, `ats`, `shares`.
4741
5012
 
4742
5013
  ### File Operations
4743
5014
 
4744
- It is possible to:
5015
+ It is possible to do **gen3/node user** operations:
4745
5016
 
4746
5017
  - browse
4747
- - transfer (upload / download)
5018
+ - transfer (upload / download / sync)
5019
+ - delete
4748
5020
  - ...
4749
5021
 
5022
+ When using an access key, so called **gen4/access key** API is also supported through sub commands using `access_keys do self`.
5023
+
5024
+ Example:
5025
+
5026
+ - `ascli node browse /` : list files with **gen3/node user** API
5027
+ - `ascli node access_key do self browse /` : list files with **gen4/access key** API
5028
+
5029
+ ### Operation `find` on **gen4/access key**
5030
+
5031
+ The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
5032
+
5033
+ The argument `<folder>` is mandatory and is the root from which search is performed.
5034
+ The argument `[filter_expr]` is optional and represent the matching criteria.
5035
+
5036
+ It recursively scans storage to find files/folders matching a criteria and then returns a list of matching entries.
5037
+
5038
+ `[filter_expr]` is either:
5039
+
5040
+ - Optional (default) : all files and folder are selected
5041
+ - type `String` : the expression is similar to shell globing, refer to **Ruby** function: [`File.fnmatch`](https://ruby-doc.org/3.2.2/File.html#method-c-fnmatch)
5042
+ - Type `Proc` : the expression is a Ruby lambda that takes one argument: a `Hash` that contains the current folder entry to test. Refer to the following examples.
5043
+
5044
+ Examples of expressions:
5045
+
5046
+ - Find all files and folders under `/`
5047
+
5048
+ ```bash
5049
+ ascli node access_keys do self find
5050
+ ```
5051
+
5052
+ - Find all text files `/Documents`
5053
+
5054
+ ```bash
5055
+ ascli node access_keys do self find /Documents '*.txt'
5056
+ ```
5057
+
5058
+ The following are examples of `ruby_lambda` to be provided in the following template command:
5059
+
5060
+ ```bash
5061
+ ascli node access_keys do self find / @ruby:'ruby_lambda'
5062
+ ```
5063
+
5064
+ > **Note:** Single quotes are used here above to protect the whole **Ruby** expression from the shell. Then double quotes are used for strings in the **Ruby** expression to not mix with the shell.
5065
+
5066
+ - Find files more recent than 100 days
5067
+
5068
+ ```ruby
5069
+ ->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100}
5070
+ ```
5071
+
5072
+ - Find files older than 1 year
5073
+
5074
+ ```ruby
5075
+ ->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))>365}
5076
+ ```
5077
+
5078
+ - Find files larger than 1MB
5079
+
5080
+ ```ruby
5081
+ ->(f){f["type"].eql?("file") and f["size"].to_i>1000000}
5082
+ ```
5083
+
5084
+ - Filter out files beginning with `._` or named `.DS_Store`:
5085
+
5086
+ ```ruby
5087
+ ->(f){!(f["name"].start_with?("._") or f["name"].eql?(".DS_Store"))}
5088
+ ```
5089
+
5090
+ - Match files using a [Ruby Regex](https://ruby-doc.org/core/Regexp.html) : `\.gif$`
5091
+
5092
+ ```ruby
5093
+ ->(f){f["name"].match?(/\.gif$/)}
5094
+ ```
5095
+
5096
+ `ascli` commands can be piped in order to combine operations, such as "find and delete":
5097
+
5098
+ ```bash
5099
+ ascli node access_keys do self find / @ruby:'->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))>365}' --fields=path --format=csv | ascli node --bulk=yes delete @lines:@stdin:
5100
+ ```
5101
+
5102
+ > **Note:** the pipe `|` character on the last line.
5103
+
4750
5104
  ### Central
4751
5105
 
4752
- The central subcommand uses the "reliable query" API (session and file).
5106
+ The central subcommand uses the **reliable query** API (session and file).
4753
5107
  It allows listing transfer sessions and transferred files.
4754
5108
 
4755
5109
  Filtering can be applied:
@@ -4758,26 +5112,36 @@ Filtering can be applied:
4758
5112
  ascli node central file list
4759
5113
  ```
4760
5114
 
4761
- by providing the `validator` option, offline transfer validation can be done.
5115
+ By providing the `validator` option, offline transfer validation can be done.
5116
+
5117
+ > **Note:** This API will probably be deprecated in the future.
5118
+
5119
+ ### Sync
5120
+
5121
+ There are three sync types of operations:
5122
+
5123
+ - `sync`: perform a local sync, by executing `async` locally
5124
+ - `async`: calls legacy async API on node : `/async`
5125
+ - `ssync` : calls newer async API on node : `/asyncs`
4762
5126
 
4763
5127
  ### FASP Stream
4764
5128
 
4765
5129
  It is possible to start a FASPStream session using the node API:
4766
5130
 
4767
- Use the "node stream create" command, then arguments are provided as a [*transfer-spec*](#transferspec).
5131
+ Use the command `ascli node stream create --ts=@json:<value>`, with [*transfer-spec*](#transferspec):
4768
5132
 
4769
- ```bash
4770
- ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}' --preset=stream
5133
+ ```json
5134
+ {"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
4771
5135
  ```
4772
5136
 
4773
- ### Watchfolder
5137
+ ### "Watchfolder"
4774
5138
 
4775
5139
  Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
4776
5140
 
4777
5141
  `ascli` supports remote operations through the node API. Operations are:
4778
5142
 
4779
- - Start watchd and watchfolderd services running as a system user having access to files
4780
- - configure a watchfolder to define automated transfers
5143
+ - Start `watchd` and `watchfolderd` services running as a system user having access to files
5144
+ - configure a "watchfolder" to define automated transfers
4781
5145
 
4782
5146
  ```bash
4783
5147
  ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
@@ -4835,21 +5199,21 @@ dnf install -y ImageMagick-devel
4835
5199
  gem install rmagick rainbow
4836
5200
  ```
4837
5201
 
4838
- For example it is possible to display the preview of a file, if it exists, using:
5202
+ For example, it is possible to display the preview of a file, if it exists, using an access key on node:
4839
5203
 
4840
5204
  ```bash
4841
- ascli aoc files thumbnail /preview_samples/Aspera.mpg
5205
+ ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
4842
5206
  ```
4843
5207
 
4844
- Using direct node access and an access key , one can do:
5208
+ Previews are mainly used in AoC, this also works with AoC:
4845
5209
 
4846
5210
  ```bash
4847
- ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
5211
+ ascli aoc files thumbnail /preview_samples/Aspera.mpg
4848
5212
  ```
4849
5213
 
4850
5214
  > **Note:** To specify the file by its file id, use the selector syntax: `%id:_file_id_here_`
4851
5215
  >
4852
- > **Note:** To force textual display of the preview on iTerm, prefix command with: `env -u TERM_PROGRAM -u LC_TERMINAL`
5216
+ > **Note:** To force textual display of the preview on **iTerm**, prefix command with: `env -u TERM_PROGRAM -u LC_TERMINAL`
4853
5217
 
4854
5218
  ### Create access key
4855
5219
 
@@ -4857,24 +5221,163 @@ ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
4857
5221
  ascli node access_key create @json:'{"id":"myaccesskey","secret":"my_secret_here","storage":{"type":"local","path":"/data/mydir"}}'
4858
5222
  ```
4859
5223
 
5224
+ > **Note:** The `id` and `secret` are optional.
5225
+ > If not provided, they will be generated and returned in the result.
5226
+
5227
+ Access keys support extra overriding parameters using parameter: `configuration` and sub keys `transfer` and `server`. For example, an access key can be modified or created with the following options:
5228
+
5229
+ ```json
5230
+ {"configuration":{"transfer":{"target_rate_cap_kbps":500000}}}
5231
+ ```
5232
+
5233
+ The list of supported options can be displayed using command:
5234
+
5235
+ ```bash
5236
+ ascli node info --field=@ruby:'/^access_key_configuration_capabilities.*/'
5237
+ ```
5238
+
5239
+ ### Generate and use bearer token
5240
+
5241
+ Bearer tokens are part of the **gen4/access key** API.
5242
+ It follows the model of OAuth 2.
5243
+ For example they are used in Aspera on Cloud.
5244
+ This is also available for developers for any application integrating Aspera.
5245
+ In this API, files, users and groups are identified by an id (a String, e.g. `"125"`, not necessarily numerical).
5246
+
5247
+ Bearer tokens are typically generated by the authentication application, and then recognized by the node API.
5248
+ A bearer token is authorized on the node by creating `permissions` on a **folder**.
5249
+
5250
+ Bearer tokens can be generated using command `bearer_token`: it takes two arguments:
5251
+
5252
+ - the private key used to sign the token
5253
+ - the token information, which is a `Hash` containing the following elements:
5254
+
5255
+ | parameter | Default | type | description |
5256
+ | ------------------------ |-----------------------------|-----------|----------------------------------------------------------|
5257
+ | user_id | - | Mandatory | Identifier of user |
5258
+ | scope | `node.<access_key>:<_scope>`| Mandatory | API scope, e.g. `node.<access_key>:<node scope>` |
5259
+ | expires_at | `now+<_validity>` | Mandatory | Format: `%Y-%m-%dT%H:%M:%SZ` e.g. `2021-12-31T23:59:59Z` |
5260
+ | auth_type | `access_key` | Optional | `access_key`, `node_user` |
5261
+ | group_ids | - | Optional | List of group ids |
5262
+ | organization_id | - | Optional | Organization id |
5263
+ | watermarking_json_base64 | - | Optional | Watermarking information (not used) |
5264
+ | _scope | `user:all` | Special | Either `user:all` or `admin:all` |
5265
+ | _validity | 86400 | Special | Validity in seconds from now. |
5266
+
5267
+ > **Note:** For convenience, `ascli` provides additional parameters `_scope` and `_validity`.
5268
+ > They are not part of the API and are removed from the final payload.
5269
+ > They are used respectively to build the default value of `scope` and `expires_at`.
5270
+
5271
+ #### Bearer token: Environment
5272
+
5273
+ - If a self-managed Aspera node is used, then a "node user admin" must be created:
5274
+ It has no `docroot` but has at least one file restriction (for testing, one can use `*` to accept creation of an access key with any storage root path).
5275
+ Refer to the Aspera HSTS documentation.
5276
+
5277
+ - If Cloud Pak for integration is used, then the node admin is created automatically.
5278
+
5279
+ - If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
5280
+
5281
+ - An access key shall be created to grant access for transfers to its storage.
5282
+ The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
5283
+
5284
+ #### Bearer token: Preparation
5285
+
5286
+ Let's assume that the access key was created, and a default configuration is set to use this **access key**.
5287
+
5288
+ - Create a private key (organization key) that will be used to sign bearer tokens:
5289
+
5290
+ ```bash
5291
+ my_private_pem=./myorgkey.pem
5292
+ ascli conf genkey $my_private_pem
5293
+ ```
5294
+
5295
+ > **Note:** This key is not used for authentication, it is used to sign bearer tokens.
5296
+ > Refer to section [private key](#private_key) for more details on generation.
5297
+
5298
+ - The corresponding public key shall be placed as an attribute of the **access key**:
5299
+
5300
+ ```bash
5301
+ ascli node access_key set_bearer_key self @file:$my_private_pem
5302
+ ```
5303
+
5304
+ > **Note:** Either the public or private key can be provided, and only the public key is used.
5305
+ > This will enable to check the signature of the bearer token.
5306
+
5307
+ Alternatively, use the following equivalent command, as `ascli` kindly extracts the public key with extension `.pub`:
5308
+
5309
+ ```bash
5310
+ ascli node access_key modify %id:self @ruby:'{token_verification_key: File.read("'$my_private_pem'.pub")}'
5311
+ ```
5312
+
5313
+ #### Bearer token: Configuration for user
5314
+
5315
+ - Select a folder for which we want to grant access to a user, and get its identifier:
5316
+
5317
+ ```bash
5318
+ my_folder_id=$(ascli node access_key do self show / --fields=id)
5319
+ ```
5320
+
5321
+ > **Note:** Here we simply select `/`, but any folder can be selected in the access key storage.
5322
+
5323
+ - Let's designate a user by its id:
5324
+
5325
+ ```bash
5326
+ my_user_id=777
5327
+ ```
5328
+
5329
+ > **Note:** This is an arbitrary identifier, typically managed by the web application.
5330
+ > Not related to Linux user ids or anything else.
5331
+
5332
+ - Grant this user access to the selected folder:
5333
+
5334
+ ```bash
5335
+ ascli node access_key do self permission %id:$my_folder_id create @json:'{"access_type":"user","access_id":"'$my_user_id'"}'
5336
+ ```
5337
+
5338
+ - Create a Bearer token for the user:
5339
+
5340
+ ```bash
5341
+ ascli node bearer_token @file:./myorgkey.pem @json:'{"user_id":"'$my_user_id'","_validity":3600}' > bearer.txt
5342
+ ```
5343
+
5344
+ #### Bearer token: User side
5345
+
5346
+ Now, let's assume we are the user, the only information received are:
5347
+
5348
+ - the url of the node API
5349
+ - a Bearer token
5350
+ - a file `id` for which we have access
5351
+
5352
+ Let's use it:
5353
+
5354
+ ```bash
5355
+ ascli node -N --url=... --password="Bearer $(cat bearer.txt)" --root-id=$my_folder_id access_key do self br /
5356
+ ```
5357
+
4860
5358
  ### Node sample commands
4861
5359
 
4862
5360
  ```bash
4863
- node access_key create @json:'{"id":"testingAK1","storage":{"type":"local","path":"/"}}'
4864
- node access_key delete testingAK1
4865
- node access_key do my_aoc_ak_name browse /
4866
- node access_key do my_aoc_ak_name delete /folder2
4867
- node access_key do my_aoc_ak_name delete testfile1
4868
- node access_key do my_aoc_ak_name download testfile1 --to-folder=.
4869
- node access_key do my_aoc_ak_name find /
4870
- node access_key do my_aoc_ak_name mkdir /folder1
4871
- node access_key do my_aoc_ak_name node_info /
4872
- node access_key do my_aoc_ak_name rename /folder1 folder2
4873
- node access_key do my_aoc_ak_name show %id:1
4874
- node access_key do my_aoc_ak_name show /testfile1
4875
- node access_key do my_aoc_ak_name thumbnail /testfile1
4876
- node access_key do my_aoc_ak_name upload 'faux:///testfile1?1k' --default_ports=no
5361
+ node --url=https://tst.example.com/path --password="Bearer bearer_666" --root-id=root_id access_key do self br /
5362
+ node access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
5363
+ node access_key delete my_username
5364
+ node access_key do my_ak_name browse /
5365
+ node access_key do my_ak_name delete /folder2
5366
+ node access_key do my_ak_name delete testfile1
5367
+ node access_key do my_ak_name download testfile1 --to-folder=.
5368
+ node access_key do my_ak_name find my_test_folder
5369
+ node access_key do my_ak_name find my_test_folder @ruby:'->(f){f["name"].end_with?(".jpg")}'
5370
+ node access_key do my_ak_name find my_test_folder exec:'f["name"].end_with?(".jpg")'
5371
+ node access_key do my_ak_name mkdir /folder1
5372
+ node access_key do my_ak_name node_info /
5373
+ node access_key do my_ak_name rename /folder1 folder2
5374
+ node access_key do my_ak_name show %id:1
5375
+ node access_key do my_ak_name show /testfile1
5376
+ node access_key do my_ak_name upload 'faux:///testfile1?1k' --default_ports=no
5377
+ node access_key do self permission %id:root_id create @json:'{"access_type":"user","access_id":"666"}'
5378
+ node access_key do self show / --fields=id
4877
5379
  node access_key list
5380
+ node access_key set_bearer_key self @file:my_private_key
4878
5381
  node api_details
4879
5382
  node async bandwidth 1
4880
5383
  node async counters 1
@@ -4883,55 +5386,107 @@ node async list
4883
5386
  node async show 1
4884
5387
  node async show ALL
4885
5388
  node basic_token
4886
- node browse / -r
4887
- node delete /todelete
4888
- node delete @list:,folder_1/todelete,folder_1/tdlink,folder_1/delfile
4889
- node delete folder_1/10MB.2
4890
- node delete testfile.bin
4891
- node download testfile.bin --to-folder=.
5389
+ node bearer_token @file:my_private_key @json:'{"user_id":"666"}'
5390
+ node browse / --log-level=trace2
5391
+ node delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
5392
+ node delete my_upload_folder/test_file.bin
5393
+ node download my_upload_folder/test_file.bin --to-folder=.
4892
5394
  node health
4893
5395
  node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
4894
5396
  node license
4895
- node mkdir folder_1/todelete
4896
- node mkfile folder_1/delfile1 "hello world"
4897
- node mklink folder_1/todelete folder_1/tdlink
4898
- node rename folder_1 delfile1 delfile
5397
+ node mkdir my_upload_folder/a_folder
5398
+ node mkfile my_upload_folder/a_file1 "hello world"
5399
+ node mklink my_upload_folder/a_folder my_upload_folder/tdlink
5400
+ node rename my_upload_folder a_file1 a_file
4899
5401
  node search / --query=@json:'{"sort":"mtime"}'
4900
5402
  node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
4901
5403
  node service delete service1
4902
5404
  node service list
4903
5405
  node space /
4904
- node ssync bandwidth my_syncid
4905
- node ssync counters my_syncid
4906
- node ssync create @json:'{"configuration":{"name":"sync1","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password","path":"my_remote_path"}}}'
4907
- node ssync delete my_syncid
4908
- node ssync files my_syncid
5406
+ node ssync bandwidth %name:my_node_sync
5407
+ node ssync counters %name:my_node_sync
5408
+ node ssync create @json:'{"configuration":{"name":"my_node_sync","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password_here","path":"my_remote_path"}}}'
5409
+ node ssync delete %name:my_node_sync
5410
+ node ssync files %name:my_node_sync
4909
5411
  node ssync list
4910
- node ssync show my_syncid
4911
- node ssync start my_syncid
4912
- node ssync state my_syncid
4913
- node ssync stop my_syncid
4914
- node ssync summary my_syncid
4915
- node sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
4916
- node sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
4917
- node sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
4918
- node sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
5412
+ node ssync show %name:my_node_sync
5413
+ node ssync start %name:my_node_sync
5414
+ node ssync state %name:my_node_sync
5415
+ node ssync stop %name:my_node_sync
5416
+ node ssync summary %name:my_node_sync
5417
+ node sync admin status --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
5418
+ node sync admin status --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
5419
+ node sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
5420
+ node sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
4919
5421
  node transfer list --query=@json:'{"active_only":true}'
4920
- node upload --to-folder=folder_1 --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass_here"}'
4921
- node upload --username=my_aoc_ak_name --password=my_aoc_ak_secret testfile.bin
4922
- node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
5422
+ node transfer sessions
5423
+ node upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
5424
+ node upload --username=my_ak_name --password=my_ak_secret test_file.bin
5425
+ node upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
4923
5426
  ```
4924
5427
 
4925
5428
  ## <a id="faspex5"></a>Plugin: `faspex5`: IBM Aspera Faspex v5
4926
5429
 
4927
5430
  IBM Aspera's newer self-managed application.
4928
5431
 
4929
- 3 authentication methods are supported:
5432
+ 3 authentication methods are supported (option `auth`):
5433
+
5434
+ | Method | Description |
5435
+ | ------------ | ------------------------------------------------------------------- |
5436
+ | `jwt` | General purpose, private-key based authentication |
5437
+ | `web` | Requires authentication with web browser |
5438
+ | `boot` | Use authentication token copied from browser (experimental) |
5439
+ | `public_link`| Public link authentication (set when option `url` is a public link) |
5440
+
5441
+ > **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
5442
+
5443
+ For a quick start, one can use the wizard, which will help creating a [option preset](#lprt):
4930
5444
 
4931
- - jwt : general purpose, private-key based authentication
4932
- - link : public link authentication
4933
- - web : requires authentication with web browser
4934
- - boot : use authentication token copied from browser (experimental)
5445
+ ```bash
5446
+ ascli config wizard
5447
+ ```
5448
+
5449
+ ```text
5450
+ argument: url> faspex5.example.com
5451
+ Multiple applications detected:
5452
+ +---------+-------------------------------------------+-------------+
5453
+ | product | url | version |
5454
+ +---------+-------------------------------------------+-------------+
5455
+ | faspex5 | https://faspex5.example.com/aspera/faspex | F5.0.6 |
5456
+ | server | ssh://faspex5.example.com:22 | OpenSSH_8.3 |
5457
+ +---------+-------------------------------------------+-------------+
5458
+ product> faspex5
5459
+ Using: Faspex at https://faspex5.example.com/aspera/faspex
5460
+ Please provide the path to your private RSA key, or nothing to generate one:
5461
+ option: key_path>
5462
+ Using existing key:
5463
+ /Users/someuser/.aspera/ascli/my_key
5464
+ option: username> someuser@example.com
5465
+ Ask the ascli client id and secret to your Administrator.
5466
+ Admin should login to: https://faspex5.example.com/aspera/faspex
5467
+ Navigate to: :: → Admin → Configurations → API clients
5468
+ Create an API client with:
5469
+ - name: ascli
5470
+ - JWT: enabled
5471
+ Then, logged in as someuser@example.com go to your profile:
5472
+ () → Account Settings → Preferences -> Public Key in PEM:
5473
+ -----BEGIN PUBLIC KEY-----
5474
+ redacted
5475
+ -----END PUBLIC KEY-----
5476
+ Once set, fill in the parameters:
5477
+ option: client_id> _my_key_here_
5478
+ option: client_secret> ****
5479
+ Preparing preset: faspex5_example_com_user
5480
+ Setting config preset as default for faspex5
5481
+ Done.
5482
+ You can test with:
5483
+ ascli faspex5 user profile show
5484
+ Saving config file.
5485
+ ```
5486
+
5487
+ > **Note:** Include the public key `BEGIN` and `END` lines when pasting in the user profile.
5488
+
5489
+ For details on the JWT method, see the following section.
4935
5490
 
4936
5491
  ### Faspex 5 JWT authentication
4937
5492
 
@@ -4939,7 +5494,7 @@ This is the general purpose and **recommended** method to use.
4939
5494
 
4940
5495
  Activation is in two steps:
4941
5496
 
4942
- - The admninistrator must create an API client in Faspex with JWT support
5497
+ - The administrator must create an API client in Faspex with JWT support
4943
5498
 
4944
5499
  This operation is generally done only once:
4945
5500
 
@@ -4986,7 +5541,7 @@ ascli faspex5 user profile show
4986
5541
 
4987
5542
  ### Faspex 5 web authentication
4988
5543
 
4989
- The admninistrator must create an API client in Faspex for an external web app support:
5544
+ The administrator must create an API client in Faspex for an external web app support:
4990
5545
 
4991
5546
  - As Admin, Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
4992
5547
  - Do not Activate JWT
@@ -5021,7 +5576,7 @@ ascli conf preset update f5boot --url=https://localhost/aspera/faspex --auth=boo
5021
5576
  ### Faspex 5 sample commands
5022
5577
 
5023
5578
  Most commands are directly REST API calls.
5024
- Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional parameter for creation.
5579
+ Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional argument for creation.
5025
5580
  One can conveniently use the JSON format with prefix `@json:`.
5026
5581
 
5027
5582
  > **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
@@ -5031,42 +5586,48 @@ faspex5 admin res accounts list
5031
5586
  faspex5 admin res contacts list
5032
5587
  faspex5 admin res jobs list
5033
5588
  faspex5 admin res metadata_profiles list
5034
- faspex5 admin res node list
5589
+ faspex5 admin res node list
5035
5590
  faspex5 admin res oauth_clients list
5036
5591
  faspex5 admin res registrations list
5037
5592
  faspex5 admin res saml_configs list
5038
- faspex5 admin res shared_inboxes invite %name:'ascli shinbox' johnny@example.com
5593
+ faspex5 admin res shared_inboxes invite %name:my_shared_inbox_name johnny@example.com
5039
5594
  faspex5 admin res shared_inboxes list
5040
- faspex5 admin res shared_inboxes members %name:'ascli shinbox' create %name:john@example.com
5041
- faspex5 admin res shared_inboxes members %name:'ascli shinbox' delete %name:john@example.com
5042
- faspex5 admin res shared_inboxes members %name:'ascli shinbox' delete %name:johnny@example.com
5043
- faspex5 admin res shared_inboxes members %name:'ascli shinbox' list
5595
+ faspex5 admin res shared_inboxes members %name:my_shared_inbox_name create %name:john@example.com
5596
+ faspex5 admin res shared_inboxes members %name:my_shared_inbox_name delete %name:john@example.com
5597
+ faspex5 admin res shared_inboxes members %name:my_shared_inbox_name delete %name:johnny@example.com
5598
+ faspex5 admin res shared_inboxes members %name:my_shared_inbox_name list
5044
5599
  faspex5 admin res workgroups list
5045
5600
  faspex5 admin smtp show
5046
5601
  faspex5 admin smtp test my_email_external
5047
5602
  faspex5 bearer_token
5048
- faspex5 gateway https://localhost:12345/aspera/faspex
5603
+ faspex5 gateway --pid-file=server_pid https://localhost:12345/aspera/faspex
5049
5604
  faspex5 health
5050
- faspex5 packages list --box=my_faspex5_shinbox
5051
- faspex5 packages list --box=my_faspex5_workgroup --group-type=workgroups
5605
+ faspex5 packages list --box=my_shared_inbox_name
5606
+ faspex5 packages list --box=my_workgroup --group-type=workgroups
5052
5607
  faspex5 packages list --query=@json:'{"mailbox":"inbox","state":["released"]}'
5053
- faspex5 packages receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
5054
- faspex5 packages receive --box=my_faspex5_shinbox "my_package_id" --to-folder=.
5055
- faspex5 packages receive --box=my_faspex5_workgroup --group-type=workgroups "my_package_id" --to-folder=.
5608
+ faspex5 packages receive --box=my_shared_inbox_name package_box_id1 --to-folder=.
5609
+ faspex5 packages receive --box=my_workgroup --group-type=workgroups workgroup_package_id1 --to-folder=.
5056
5610
  faspex5 packages receive ALL --once-only=yes --to-folder=.
5057
5611
  faspex5 packages receive INIT --once-only=yes
5058
- faspex5 packages send @json:'{"title":"test title","recipients":["my_shinbox"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' testfile.bin
5059
- faspex5 packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' testfile.bin
5060
- faspex5 packages send @json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
5061
- faspex5 packages show "my_package_id"
5062
- faspex5 packages show --box=my_faspex5_shinbox "my_package_id"
5063
- faspex5 packages show --box=my_faspex5_workgroup --group-type=workgroups "my_package_id"
5064
- faspex5 postprocessing @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":"tests"},"certificate":{"key":"../local/k","cert":"../local/c","chain":"../local/ch"}}'
5612
+ faspex5 packages receive f5_p31 --to-folder=. --ts=@json:'{"content_protection_password":"my_secret_here"}'
5613
+ faspex5 packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file
5614
+ faspex5 packages send @json:'{"title":"test title","recipients":["my_shared_inbox_name"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' test_file.bin
5615
+ faspex5 packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' test_file.bin
5616
+ faspex5 packages send @json:'{"title":"test title","recipients":[{"name":"my_username"}]my_meta}' test_file.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
5617
+ faspex5 packages show --box=my_shared_inbox_name package_box_id1
5618
+ faspex5 packages show --box=my_workgroup --group-type=workgroups workgroup_package_id1
5619
+ faspex5 packages show f5_p31
5620
+ faspex5 packages status f5_p31
5621
+ faspex5 postprocessing --pid-file=server_pid @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":"tests"}}'
5622
+ faspex5 shared browse %name:my_src
5623
+ faspex5 shared list
5624
+ faspex5 shared_folders browse %name:my_shared_folder_name
5625
+ faspex5 shared_folders list
5065
5626
  faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
5066
5627
  faspex5 user profile show
5067
5628
  ```
5068
5629
 
5069
- ### Faspex 5: inbox selection
5630
+ ### Faspex 5: Inbox selection
5070
5631
 
5071
5632
  By default, package operations (send, receive, list) are done on the user's inbox.
5072
5633
 
@@ -5150,7 +5711,15 @@ The following parameters in option `query` are supported:
5150
5711
 
5151
5712
  Admin only: If the value `ALL` is provided to option `box`, then all packages are selected.
5152
5713
 
5153
- ### Faspex 5: List all shared inboxes
5714
+ ### Faspex 5: List all shared inboxes and work groups
5715
+
5716
+ If you are a regular user, to list work groups you belong to:
5717
+
5718
+ ```bash
5719
+ ascli faspex5 admin res workgroup list
5720
+ ```
5721
+
5722
+ If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
5154
5723
 
5155
5724
  ```bash
5156
5725
  ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
@@ -5159,16 +5728,16 @@ ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,nam
5159
5728
  Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
5160
5729
 
5161
5730
  ```bash
5162
- ascli faspex5 admin res shared_inboxes invite '%name:ascli shinbox' john@example.com
5731
+ ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' john@example.com
5163
5732
  ```
5164
5733
 
5165
5734
  It is equivalent to:
5166
5735
 
5167
5736
  ```bash
5168
- ascli faspex5 admin res shared_inboxes invite '%name:ascli shinbox' @json:'{"email_address":"john@example.com"}'
5737
+ ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
5169
5738
  ```
5170
5739
 
5171
- Other payload parameters are possible in Hash format:
5740
+ Other payload parameters are possible the `Hash` value:
5172
5741
 
5173
5742
  ```json
5174
5743
  {"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
@@ -5210,7 +5779,7 @@ ascli faspex5 packages send @json:'{"title":"hello","recipients":[{"name":"_reci
5210
5779
 
5211
5780
  > **Note:** The shared folder can be identified by its numerical `id` or by name using percent selector: `%<field>:<value>`. e.g. `--shared-folder=3`
5212
5781
 
5213
- ### Faspex 5: receive all packages (cargo)
5782
+ ### Faspex 5: Receive all packages (cargo)
5214
5783
 
5215
5784
  To receive all packages, only once, through persistency of already received packages:
5216
5785
 
@@ -5310,7 +5879,7 @@ The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/cat
5310
5879
  If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
5311
5880
  By default `count` is `0` (`10`), it can be increased to issue less HTTP calls.
5312
5881
 
5313
- #### Example: list packages in dropbox
5882
+ #### Example: List packages in dropbox
5314
5883
 
5315
5884
  ```bash
5316
5885
  ascli faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
@@ -5342,13 +5911,14 @@ if `id` is set to `ALL`, then all packages are downloaded, and if option `once_o
5342
5911
 
5343
5912
  ### Sending a Package
5344
5913
 
5345
- The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`.
5346
- The contents of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages), consult it for extended supported parameters.
5914
+ The command is `faspex package send`.
5915
+ Package information (title, note, metadata, options) is provided in option `delivery_info`.
5916
+ The contents of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages).
5347
5917
 
5348
5918
  Example:
5349
5919
 
5350
5920
  ```bash
5351
- ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
5921
+ ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
5352
5922
  ```
5353
5923
 
5354
5924
  If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
@@ -5359,19 +5929,29 @@ Additional optional parameters in `delivery_info`:
5359
5929
  - Package Note: : `"note":"note this and that"`
5360
5930
  - Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
5361
5931
 
5932
+ It is possible to send from a remote source using option `remote_source`, providing either the numerical id, or the name of the remote source using percent selector: `%name:<name>`.
5933
+
5934
+ Remote source can be browsed if option `storage` is provided.
5935
+ `storage` is a `Hash` extended value.
5936
+ The key is the storage name, as listed in `source list` command.
5937
+ The value is a `Hash` with the following keys:
5938
+
5939
+ - `node` is a `Hash` with keys: `url`, `username`, `password`
5940
+ - `path` is the subpath inside the node, as configured in Faspex
5941
+
5362
5942
  ### Email notification on transfer
5363
5943
 
5364
- Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
5944
+ Like for any transfer, a notification can be sent by email using parameters: `notify_to` and `notify_template` .
5365
5945
 
5366
5946
  Example:
5367
5947
 
5368
5948
  ```bash
5369
- ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
5949
+ ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notify-to=aspera.user1@gmail.com --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
5370
5950
  ```
5371
5951
 
5372
5952
  In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
5373
5953
 
5374
- ### Operation on dropboxes
5954
+ ### Operation on "dropboxes"
5375
5955
 
5376
5956
  Example:
5377
5957
 
@@ -5406,9 +5986,9 @@ my_faspex_node:
5406
5986
 
5407
5987
  In this example, a faspex storage named `my_storage` exists in Faspex, and is located
5408
5988
  under the docroot in `/mydir` (this must be the same as configured in Faspex).
5409
- The node configuration name is "my_faspex_node" here.
5989
+ The node configuration name is `my_faspex_node` here.
5410
5990
 
5411
- Note: the v4 API provides an API for nodes and shares.
5991
+ > **Note:** the v4 API provides an API for nodes and shares.
5412
5992
 
5413
5993
  ### Automated package download (cargo)
5414
5994
 
@@ -5423,30 +6003,30 @@ ascli faspex packages recv ALL --once-only=yes --lock-port=12345
5423
6003
 
5424
6004
  ```bash
5425
6005
  faspex address_book
5426
- faspex dropbox list --recipient="*my_faspex_dbx"
5427
- faspex dropbox list --recipient="*my_faspex_wkg"
6006
+ faspex dropbox list --recipient="*my_dbx"
5428
6007
  faspex health
5429
6008
  faspex login_methods
5430
6009
  faspex me
5431
- faspex package list
5432
6010
  faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
5433
6011
  faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
5434
- faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
5435
- faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
5436
- faspex package recv "my_package_id" --to-folder=.
5437
- faspex package recv "my_package_id" --to-folder=. --box=sent
6012
+ faspex package list --query=@json:'{"max":5}'
6013
+ faspex package list --recipient="*my_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
6014
+ faspex package list --recipient="*my_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
5438
6015
  faspex package recv --to-folder=. --link=https://app.example.com/recv_from_user_path
5439
- faspex package recv ALL --to-folder=. --once-only=yes
5440
- faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
5441
- faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
5442
- faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
5443
- faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
5444
- faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
5445
- faspex package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
5446
- faspex package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
6016
+ faspex package recv ALL --to-folder=. --once-only=yes --query=@json:'{"max":10}'
6017
+ faspex package recv f4_db_id1 --recipient="*my_dbx" --to-folder=.
6018
+ faspex package recv f4_db_id2 --recipient="*my_wkg" --to-folder=.
6019
+ faspex package recv f4_pri1 --to-folder=.
6020
+ faspex package recv f4_prs2 --to-folder=. --box=sent
6021
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_dbx"]}' test_file.bin
6022
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_wkg"]}' test_file.bin
6023
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal","my_username"]}' test_file.bin
6024
+ faspex package send --delivery-info=@json:'{"title":"TIMESTAMP package remote","recipients":["my_email_internal"]}' --remote_source=%name:my_src sample_source.txt
6025
+ faspex package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"Important files delivery"}' test_file.bin
6026
+ faspex package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"Important files delivery"}' test_file.bin
6027
+ faspex source info %name:my_src --storage=@preset:faspex4_storage
5447
6028
  faspex source list
5448
- faspex source name my_faspex_src info
5449
- faspex source name my_faspex_src node br /
6029
+ faspex source node %name:my_src br / --storage=@preset:faspex4_storage
5450
6030
  faspex v4 dmembership list
5451
6031
  faspex v4 dropbox list
5452
6032
  faspex v4 metadata_profile list
@@ -5464,21 +6044,23 @@ Aspera Shares supports the "node API" for the file transfer part.
5464
6044
  ```bash
5465
6045
  shares admin group list
5466
6046
  shares admin node list
5467
- shares admin share list --fields=-status,status_message
6047
+ shares admin share list --fields=DEF,-status,status_message
5468
6048
  shares admin share user_permissions 1 list
5469
6049
  shares admin user add --type=ldap the_name
5470
6050
  shares admin user app_authorizations 1 modify @json:'{"app_login":true}'
5471
6051
  shares admin user app_authorizations 1 show
5472
6052
  shares admin user import --type=saml @json:'{"id":"the_id","name_id":"the_name"}'
5473
6053
  shares admin user list
6054
+ shares admin user list --type=local
5474
6055
  shares admin user share_permissions 1 list
5475
6056
  shares admin user share_permissions 1 show 1
5476
6057
  shares files browse /
5477
- shares files delete my_shares_upload/testfile.bin
5478
- shares files download --to-folder=. my_shares_upload/testfile.bin
5479
- shares files download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
5480
- shares files upload --to-folder=my_shares_upload testfile.bin
5481
- shares files upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
6058
+ shares files delete my_share1/test_file.bin
6059
+ shares files download --to-folder=. my_share1/test_file.bin
6060
+ shares files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
6061
+ shares files upload --to-folder=my_share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
6062
+ shares files upload --to-folder=my_share1 test_file.bin
6063
+ shares files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
5482
6064
  shares health
5483
6065
  ```
5484
6066
 
@@ -5490,7 +6072,7 @@ shares health
5490
6072
  console health
5491
6073
  console transfer current list
5492
6074
  console transfer smart list
5493
- console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
6075
+ console transfer smart sub my_smart_id @json:'{"source":{"paths":["my_smart_file"]},"source_type":"user_selected"}'
5494
6076
  ```
5495
6077
 
5496
6078
  ## <a id="orchestrator"></a>Plugin: `orchestrator`:IBM Aspera Orchestrator
@@ -5502,14 +6084,14 @@ orchestrator health
5502
6084
  orchestrator info
5503
6085
  orchestrator plugins
5504
6086
  orchestrator processes
5505
- orchestrator workflow details my_orch_workflow_id
5506
- orchestrator workflow export my_orch_workflow_id
5507
- orchestrator workflow inputs my_orch_workflow_id
6087
+ orchestrator workflow details my_workflow_id
6088
+ orchestrator workflow export my_workflow_id
6089
+ orchestrator workflow inputs my_workflow_id
5508
6090
  orchestrator workflow list
5509
- orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}'
5510
- orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
6091
+ orchestrator workflow start my_workflow_id @json:'{"Param":"world !"}'
6092
+ orchestrator workflow start my_workflow_id @json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
5511
6093
  orchestrator workflow status ALL
5512
- orchestrator workflow status my_orch_workflow_id
6094
+ orchestrator workflow status my_workflow_id
5513
6095
  ```
5514
6096
 
5515
6097
  ## <a id="cos"></a>Plugin: `cos`: IBM Cloud Object Storage
@@ -5519,10 +6101,10 @@ It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Ser
5519
6101
  Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
5520
6102
 
5521
6103
  There are two possibilities to provide credentials.
5522
- If you already have the endpoint, apikey and CRN, use the first method.
6104
+ If you already have the endpoint, API key and CRN, use the first method.
5523
6105
  If you don't have credentials but have access to the IBM Cloud console, then use the second method.
5524
6106
 
5525
- ### Using endpoint, apikey and Resource Instance ID (CRN)
6107
+ ### Using endpoint, API key and Resource Instance ID (CRN)
5526
6108
 
5527
6109
  If you have those parameters already, then following options shall be provided:
5528
6110
 
@@ -5613,56 +6195,54 @@ ascli cos node info
5613
6195
  ascli cos node upload 'faux:///sample1G?1g'
5614
6196
  ```
5615
6197
 
5616
- Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man `ascp` and section above), but you can of course send a real file by specifying a real file instead.
6198
+ > **Note:** we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man `ascp` and section above), but you can of course send a real file by specifying a real file instead.
5617
6199
 
5618
6200
  ### COS sample commands
5619
6201
 
5620
6202
  ```bash
5621
- cos --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
5622
- cos --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
6203
+ cos --bucket=my_bucket_name --endpoint=my_bucket_endpoint --apikey=my_bucket_apikey --crn=my_resource_instance_id node info
6204
+ cos --bucket=my_bucket_name --region=my_bucket_region --service-credentials=@json:@file:service_creds.json node info
5623
6205
  cos node access_key show self
5624
- cos node download testfile.bin --to-folder=.
5625
- cos node info
5626
- cos node upload testfile.bin
6206
+ cos node download test_file.bin --to-folder=.
6207
+ cos node info --log-level=trace2
6208
+ cos node upload test_file.bin
5627
6209
  ```
5628
6210
 
5629
- ## <a id="async"></a>Plugin: `async`: IBM Aspera Sync
6211
+ ## <a id="async"></a>IBM Aspera Sync
6212
+
6213
+ An interface for the `async` utility is provided in the following plugins:
6214
+
6215
+ - server sync
6216
+ - node sync
6217
+ - aoc files sync (uses node)
6218
+ - shares files sync (uses node)
5630
6219
 
5631
- A basic plugin to start an `async` using `ascli`.
5632
- The main advantage over bare `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
6220
+ The main advantage over the `async` command line when using `server` is the possibility to use a configuration file, using standard options of `ascli`.
5633
6221
 
5634
- The `sync` command is also made available through the `server sync`, `aoc files sync` and `node sync` commands.
5635
- In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (including token).
6222
+ In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
5636
6223
 
5637
6224
  > **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
5638
6225
 
5639
6226
  Two JSON syntax are supported for option `sync_info`.
5640
6227
 
5641
- ### async native JSON
6228
+ ### async JSON: API format
5642
6229
 
5643
- It is the same payload as specified on the `async` option `--conf` or in the latest node API.
6230
+ It is the same payload as specified on the option `--conf` of `async` or in node API `/asyncs`.
5644
6231
  This is the preferred syntax and allows a single session definition.
5645
6232
  But there is no progress output nor error messages.
5646
6233
 
5647
6234
  Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
5648
6235
 
5649
- ### async options as JSON
6236
+ ### async JSON: Options mapping
5650
6237
 
5651
- This is specific to `ascli`.
6238
+ `ascli` defines a JSON equivalent to regular `async`options.
5652
6239
  It is based on a JSON representation of `async` command line options.
5653
6240
  It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
5654
6241
 
5655
- ### Sync sample commands
5656
-
5657
- ```bash
5658
- sync admin status --sync-info=@json:'{"sessions":[{"name":"test","local_dir":"contents"}]}'
5659
- sync start --sync-info=@json:'{"instance":{"quiet":true},"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_paths":["my_local_user_key"]}]}'
5660
- ```
5661
-
5662
6242
  ## <a id="preview"></a>Plugin: `preview`: Preview generator for AoC
5663
6243
 
5664
6244
  The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
5665
- It uses the **node API** of Aspera HSTS and requires use of Access Keys and it's **storage root**.
6245
+ It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
5666
6246
  Several parameters can be used to tune several aspects:
5667
6247
 
5668
6248
  - Methods for detection of new files needing generation
@@ -5684,7 +6264,7 @@ asconfigurator -x "server;preview_dir,previews"
5684
6264
  asnodeadmin --reload
5685
6265
  ```
5686
6266
 
5687
- Note: the configuration `preview_dir` is *relative* to the storage root, no need leading or trailing `/`. In general just set the value to `previews`
6267
+ > **Note:** the configuration `preview_dir` is **relative** to the storage root, no need leading or trailing `/`. In general just set the value to `previews`
5688
6268
 
5689
6269
  If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
5690
6270
 
@@ -5703,16 +6283,16 @@ asconfigurator -x "server; max_request_file_create_size_kb,16384"
5703
6283
 
5704
6284
  If you use a value different than 16777216, then specify it using option `max_size`.
5705
6285
 
5706
- Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* while the generator parameter is in *Bytes* (factor of 1024).
6286
+ > **Note:** the HSTS parameter (`max_request_file_create_size_kb`) is in **kiloBytes** while the generator parameter is in **Bytes** (factor of 1024).
5707
6287
 
5708
6288
  ### <a id="prev_ext"></a>External tools: Linux
5709
6289
 
5710
- The tool requires the following external tools available in the `PATH`:
6290
+ `ascli` requires the following external tools available in the `PATH`:
5711
6291
 
5712
6292
  - ImageMagick : `convert` `composite`
5713
- - OptiPNG : `optipng`
5714
- - FFmpeg : `ffmpeg` `ffprobe`
5715
- - Libreoffice : `libreoffice`
6293
+ - "OptiPNG" : `optipng`
6294
+ - "FFmpeg" : `ffmpeg` `ffprobe`
6295
+ - "Libreoffice" : `libreoffice`
5716
6296
 
5717
6297
  Here shown on Redhat/CentOS.
5718
6298
 
@@ -5724,7 +6304,7 @@ To check if all tools are found properly, execute:
5724
6304
  ascli preview check
5725
6305
  ```
5726
6306
 
5727
- #### Image: ImageMagick and optipng
6307
+ #### Image: ImageMagick and `optipng`
5728
6308
 
5729
6309
  ```bash
5730
6310
  dnf install -y ImageMagick optipng
@@ -5734,6 +6314,8 @@ You may also install `ghostscript` which adds fonts to ImageMagick.
5734
6314
  Available fonts, used to generate png for text, can be listed with `magick identify -list font`.
5735
6315
  Prefer ImageMagick version >=7.
5736
6316
 
6317
+ More info on ImageMagick at <https://imagemagick.org/>
6318
+
5737
6319
  #### Video: FFmpeg
5738
6320
 
5739
6321
  The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
@@ -5742,7 +6324,7 @@ The easiest method is to download and install the latest released version of ffm
5742
6324
  curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
5743
6325
  ```
5744
6326
 
5745
- #### Office: Unoconv and Libreoffice
6327
+ #### Office: `unoconv` and Libreoffice
5746
6328
 
5747
6329
  If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
5748
6330
 
@@ -5799,11 +6381,11 @@ This shall list the contents of the storage root of the access key.
5799
6381
 
5800
6382
  When generating preview files, some options are provided by default.
5801
6383
  Some values for the options can be modified on command line.
5802
- For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a Hash with two keys: `in` and `out`, each is an array of strings with the native options to `ffmpeg`.
6384
+ For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a `Hash` with two keys: `in` and `out`, each is an `Array` of strings with the native options to `ffmpeg`.
5803
6385
 
5804
6386
  ### Execution
5805
6387
 
5806
- The tool intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
6388
+ `ascli` intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
5807
6389
  It needs to be run on a regular basis to create or update preview files.
5808
6390
  For that use your best reliable scheduler, see [Scheduler](#scheduler).
5809
6391
 
@@ -5835,7 +6417,7 @@ case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
5835
6417
 
5836
6418
  ### Candidate detection for creation or update (or deletion)
5837
6419
 
5838
- The tool generates preview files using those commands:
6420
+ `ascli` generates preview files using those commands:
5839
6421
 
5840
6422
  - `trevents` : only recently uploaded files will be tested (transfer events)
5841
6423
  - `events` : only recently uploaded files will be tested (file events: not working)
@@ -5861,13 +6443,9 @@ ascli preview scan --skip-folders=@json:'["/not_here"]'
5861
6443
 
5862
6444
  The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
5863
6445
 
5864
- When scanning the option `query` has the same behavior as for the `node find` command.
5865
-
5866
- For instance to filter out files beginning with `._` do:
6446
+ When scanning the option `query` has the same behavior as for the `node access_keys do self find` command.
5867
6447
 
5868
- ```bash
5869
- --query='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
5870
- ```
6448
+ Refer to that section for details.
5871
6449
 
5872
6450
  ### Preview File types
5873
6451
 
@@ -5896,28 +6474,28 @@ The mp4 video preview file is only for category `video`
5896
6474
 
5897
6475
  File type is primarily based on file extension detected by the node API and translated info a mime type returned by the node API.
5898
6476
 
5899
- ### mimemagic
6477
+ ### `mimemagic`
5900
6478
 
5901
6479
  By default, the Mime type used for conversion is the one returned by the node API, based on file name extension.
5902
6480
 
5903
6481
  It is also possible to detect the mime type using option `mimemagic`.
5904
6482
  To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`.
5905
6483
 
5906
- This requires to manually install the mimemagic gem: `gem install mimemagic`.
6484
+ This requires to manually install the `mimemagic` gem: `gem install mimemagic`.
5907
6485
 
5908
- In this case the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
6486
+ In this case the `preview` command will first analyze the file content using `mimemagic`, and if no match, will try by extension.
5909
6487
 
5910
6488
  If the `mimemagic` gem complains about missing mime info file:
5911
6489
 
5912
6490
  - any OS:
5913
6491
 
5914
6492
  - Examine the error message
5915
- - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5916
- - move and rename this file to one of the locations expected by mimemagic as specified in the error message
6493
+ - Download the file: [`freedesktop.org.xml.in`](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
6494
+ - move and rename this file to one of the locations expected by `mimemagic` as specified in the error message
5917
6495
 
5918
6496
  - Windows:
5919
6497
 
5920
- - Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
6498
+ - Download the file: [`freedesktop.org.xml.in`](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
5921
6499
  - Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
5922
6500
  - Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
5923
6501
 
@@ -5943,7 +6521,7 @@ brew install shared-mime-info
5943
6521
 
5944
6522
  Standard open source tools are used to create thumbnails and video previews.
5945
6523
  Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
5946
- The tool provides 2 ways to read and write files with the option: `file_access`
6524
+ `ascli` provides 2 ways to read and write files with the option: `file_access`
5947
6525
 
5948
6526
  If the preview generator is run on a system that has direct access to the file system, then the value `local` can be used. In this case, no transfer happen, source files are directly read from the storage, and preview files
5949
6527
  are directly written to the storage.
@@ -5956,14 +6534,14 @@ If the preview generator does not have access to files on the file system (it is
5956
6534
  preview check --skip-types=office
5957
6535
  preview scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
5958
6536
  preview scan --skip-types=office --log-level=info
5959
- preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
5960
- preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
5961
- preview test --case=test mp4 my_file_mxf --video-conversion=reencode --log-level=debug
5962
- preview test --case=test png my_file_dcm --log-level=debug
5963
- preview test --case=test png my_file_docx --log-level=debug
5964
- preview test --case=test png my_file_mxf --video-png-conv=animated --log-level=debug
5965
- preview test --case=test png my_file_mxf --video-png-conv=fixed --log-level=debug
5966
- preview test --case=test png my_file_pdf --log-level=debug
6537
+ preview show --base=test my_docx
6538
+ preview show --base=test my_mpg --video-png-conv=animated
6539
+ preview show --base=test my_mpg --video-png-conv=fixed
6540
+ preview show --base=test my_mpg mp4 --video-conversion=clips
6541
+ preview show --base=test my_mpg mp4 --video-conversion=reencode
6542
+ preview show --base=test my_pdf
6543
+ preview test --base=test my_dcm
6544
+ preview test --base=test my_mxf mp4 --video-conversion=blend --query=@json:'{"text":true,"double":true}'
5967
6545
  preview trevents --once-only=yes --skip-types=office --log-level=info
5968
6546
  ```
5969
6547
 
@@ -5971,7 +6549,7 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
5971
6549
 
5972
6550
  `ascli` can send email, for that setup SMTP configuration. This is done with option `smtp`.
5973
6551
 
5974
- The `smtp` option is a hash table (extended value) with the following fields:
6552
+ The `smtp` option is a `Hash` (extended value) with the following fields:
5975
6553
 
5976
6554
  <!-- markdownlint-disable MD034 -->
5977
6555
  | field | default | example | description |
@@ -6034,16 +6612,16 @@ Check settings with `smtp_settings` command. Send test email with `email_test`.
6034
6612
 
6035
6613
  ```bash
6036
6614
  ascli config --smtp=@preset:smtp_google smtp
6037
- ascli config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
6615
+ ascli config --smtp=@preset:smtp_google email --notify-to=sample.dest@example.com
6038
6616
  ```
6039
6617
 
6040
6618
  ### Notifications for transfer status
6041
6619
 
6042
6620
  An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
6043
6621
 
6044
- To activate, use option `notif_to`.
6622
+ To activate, use option `notify_to`.
6045
6623
 
6046
- A default e-mail template is used, but it can be overridden with option `notif_template`.
6624
+ A default e-mail template is used, but it can be overridden with option `notify_template`.
6047
6625
 
6048
6626
  The environment provided contains the following additional variables:
6049
6627
 
@@ -6076,24 +6654,24 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
6076
6654
 
6077
6655
  This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
6078
6656
 
6079
- The tool expect one single argument: a [*transfer-spec*](#transferspec).
6657
+ `ascli` expect one single argument: a [*transfer-spec*](#transferspec).
6080
6658
 
6081
6659
  If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
6082
6660
 
6083
- > **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the hash using JSON syntax.
6661
+ > **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the `Hash` using JSON syntax.
6084
6662
 
6085
6663
  During execution, it generates all low level events, one per line, in JSON format on stdout.
6086
6664
 
6087
6665
  There are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
6088
6666
 
6089
- - `EX_loglevel` to change log level of the tool
6667
+ - `EX_loglevel` to change log level of `ascli`
6090
6668
  - `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
6091
6669
 
6092
6670
  > **Note:** In addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
6093
6671
 
6094
6672
  ### Comparison of interfaces
6095
6673
 
6096
- | feature/tool | asession | `ascp` | FaspManager | Transfer SDK |
6674
+ | feature/tool | `asession` | `ascp` | FaspManager | Transfer SDK |
6097
6675
  |--------------|----------|--------|-------------|--------------|
6098
6676
  | language integration | any | any | C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/> | many |
6099
6677
  | required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
@@ -6203,7 +6781,7 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
6203
6781
 
6204
6782
  Virtually any transfer on a "repository" on a regular basis might emulate a hot folder.
6205
6783
 
6206
- > **Note:** file detection is not based on events (inotify, etc...), but on a simple folder scan on source side.
6784
+ > **Note:** file detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
6207
6785
  >
6208
6786
  > **Note:** parameters may be saved in a [option preset](#lprt) and used with `-P`.
6209
6787
 
@@ -6212,7 +6790,7 @@ Virtually any transfer on a "repository" on a regular basis might emulate a hot
6212
6790
  Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
6213
6791
  Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
6214
6792
 
6215
- ### Example: upload hot folder
6793
+ ### Example: Upload hot folder
6216
6794
 
6217
6795
  ```bash
6218
6796
  ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"remove_after_transfer":true,"remove_empty_directories":true,"exclude_newer_than:-8,"src_base":"source_hot"}'
@@ -6223,7 +6801,7 @@ Source files are deleted after transfer.
6223
6801
  Growing files will be sent only once they don't grow anymore (based on an 8-second cool-off period).
6224
6802
  If a transfer takes more than the execution period, then the subsequent execution is skipped (`lock_port`) preventing multiple concurrent runs.
6225
6803
 
6226
- ### Example: unidirectional synchronization (upload) to server
6804
+ ### Example: Unidirectional synchronization (upload) to server
6227
6805
 
6228
6806
  ```bash
6229
6807
  ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=12345 --ts=@json:'{"resume_policy":"sparse_csum","exclude_newer_than":-8,"src_base":"source_sync"}'
@@ -6231,11 +6809,10 @@ ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=1234
6231
6809
 
6232
6810
  This can also be used with other folder-based applications: Aspera on Cloud, Shares, Node:
6233
6811
 
6234
- ### Example: unidirectional synchronization (download) from Aspera on Cloud Files
6812
+ ### Example: Unidirectional synchronization (download) from Aspera on Cloud Files
6235
6813
 
6236
6814
  ```bash
6237
- ascli aoc files download . --to-folder=. --lock-port=12345 --progress=none --display=data \
6238
- --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
6815
+ ascli aoc files download . --to-folder=. --lock-port=12345 --progress-bar=no --display=data --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
6239
6816
  ```
6240
6817
 
6241
6818
  > **Note:** option `delete_before_transfer` will delete files locally, if they are not present on remote side.
@@ -6267,7 +6844,7 @@ Typically, the health check uses the REST API of the application with the follow
6267
6844
  `ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
6268
6845
 
6269
6846
  ```bash
6270
- ascli server health transfer --to-folder=/Upload --format=nagios --progress=none
6847
+ ascli server health transfer --to-folder=/Upload --format=nagios --progress-bar=no
6271
6848
  ```
6272
6849
 
6273
6850
  ```output
@@ -6305,8 +6882,8 @@ Also, because there was already the `ascp` tool, I thought of an extended tool :
6305
6882
 
6306
6883
  There were a few pitfalls:
6307
6884
 
6308
- - The tool was written in the aging `perl` language while most Aspera web application products (but the Transfer Server) are written in `ruby`.
6309
- - The tool was only for transfers, but not able to call other products APIs
6885
+ - `ascli` was written in the aging `perl` language while most Aspera web application products (but the Transfer Server) are written in `ruby`.
6886
+ - `ascli` was only for transfers, but not able to call other products APIs
6310
6887
 
6311
6888
  So, it evolved into `ascli`:
6312
6889
 
@@ -6346,7 +6923,7 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
6346
6923
  Some Ruby gems dependencies require compilation of native parts (C).
6347
6924
  This also requires Ruby header files.
6348
6925
  If Ruby was installed as a Linux Packages, then also install Ruby development package:
6349
- `ruby-dev` ir `ruby-devel`, depending on distribution.
6926
+ `ruby-dev` or `ruby-devel`, depending on distribution.
6350
6927
 
6351
6928
  ### ED255519 key not supported
6352
6929