aspera-cli 4.17.0 → 4.18.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +3 -4
  3. data/CHANGELOG.md +33 -0
  4. data/CONTRIBUTING.md +15 -1
  5. data/README.md +711 -432
  6. data/bin/ascli +5 -0
  7. data/bin/asession +2 -2
  8. data/examples/build_package.sh +28 -0
  9. data/lib/aspera/agent/alpha.rb +10 -8
  10. data/lib/aspera/agent/base.rb +9 -6
  11. data/lib/aspera/agent/connect.rb +7 -8
  12. data/lib/aspera/agent/direct.rb +56 -37
  13. data/lib/aspera/agent/httpgw.rb +23 -324
  14. data/lib/aspera/agent/node.rb +19 -20
  15. data/lib/aspera/agent/trsdk.rb +19 -20
  16. data/lib/aspera/api/aoc.rb +17 -14
  17. data/lib/aspera/api/cos_node.rb +4 -4
  18. data/lib/aspera/api/httpgw.rb +342 -0
  19. data/lib/aspera/api/node.rb +135 -89
  20. data/lib/aspera/ascmd.rb +4 -3
  21. data/lib/aspera/ascp/installation.rb +15 -7
  22. data/lib/aspera/ascp/management.rb +2 -2
  23. data/lib/aspera/ascp/products.rb +1 -1
  24. data/lib/aspera/cli/basic_auth_plugin.rb +5 -9
  25. data/lib/aspera/cli/extended_value.rb +35 -16
  26. data/lib/aspera/cli/formatter.rb +161 -70
  27. data/lib/aspera/cli/hints.rb +18 -0
  28. data/lib/aspera/cli/main.rb +32 -39
  29. data/lib/aspera/cli/manager.rb +151 -119
  30. data/lib/aspera/cli/plugin.rb +27 -21
  31. data/lib/aspera/cli/plugin_factory.rb +31 -20
  32. data/lib/aspera/cli/plugins/alee.rb +14 -2
  33. data/lib/aspera/cli/plugins/aoc.rb +152 -141
  34. data/lib/aspera/cli/plugins/ats.rb +1 -1
  35. data/lib/aspera/cli/plugins/config.rb +72 -65
  36. data/lib/aspera/cli/plugins/console.rb +8 -5
  37. data/lib/aspera/cli/plugins/faspex.rb +32 -23
  38. data/lib/aspera/cli/plugins/faspex5.rb +232 -156
  39. data/lib/aspera/cli/plugins/faspio.rb +85 -0
  40. data/lib/aspera/cli/plugins/httpgw.rb +55 -0
  41. data/lib/aspera/cli/plugins/node.rb +129 -64
  42. data/lib/aspera/cli/plugins/orchestrator.rb +33 -30
  43. data/lib/aspera/cli/plugins/preview.rb +7 -3
  44. data/lib/aspera/cli/plugins/server.rb +6 -6
  45. data/lib/aspera/cli/plugins/shares.rb +16 -14
  46. data/lib/aspera/cli/special_values.rb +13 -0
  47. data/lib/aspera/cli/sync_actions.rb +10 -10
  48. data/lib/aspera/cli/transfer_agent.rb +7 -6
  49. data/lib/aspera/cli/version.rb +1 -1
  50. data/lib/aspera/environment.rb +70 -9
  51. data/lib/aspera/faspex_gw.rb +5 -4
  52. data/lib/aspera/faspex_postproc.rb +2 -2
  53. data/lib/aspera/log.rb +6 -3
  54. data/lib/aspera/node_simulator.rb +2 -2
  55. data/lib/aspera/oauth/base.rb +31 -19
  56. data/lib/aspera/oauth/factory.rb +12 -13
  57. data/lib/aspera/oauth/generic.rb +1 -0
  58. data/lib/aspera/oauth/jwt.rb +18 -15
  59. data/lib/aspera/oauth/url_json.rb +8 -6
  60. data/lib/aspera/oauth/web.rb +2 -2
  61. data/lib/aspera/persistency_folder.rb +2 -2
  62. data/lib/aspera/preview/generator.rb +3 -3
  63. data/lib/aspera/preview/options.rb +3 -3
  64. data/lib/aspera/preview/terminal.rb +4 -4
  65. data/lib/aspera/preview/utils.rb +3 -3
  66. data/lib/aspera/proxy_auto_config.rb +5 -1
  67. data/lib/aspera/rest.rb +105 -88
  68. data/lib/aspera/rest_call_error.rb +1 -1
  69. data/lib/aspera/rest_error_analyzer.rb +2 -2
  70. data/lib/aspera/rest_errors_aspera.rb +1 -1
  71. data/lib/aspera/resumer.rb +1 -1
  72. data/lib/aspera/secret_hider.rb +2 -4
  73. data/lib/aspera/ssh.rb +1 -1
  74. data/lib/aspera/transfer/parameters.rb +39 -36
  75. data/lib/aspera/transfer/spec.rb +2 -0
  76. data/lib/aspera/transfer/sync.rb +2 -1
  77. data/lib/aspera/transfer/uri.rb +1 -1
  78. data/lib/aspera/uri_reader.rb +5 -4
  79. data/lib/aspera/web_auth.rb +1 -1
  80. data/lib/aspera/web_server_simple.rb +4 -3
  81. data.tar.gz.sig +0 -0
  82. metadata +7 -4
  83. metadata.gz.sig +0 -0
  84. data/lib/aspera/cli/plugins/bss.rb +0 -71
  85. data/lib/aspera/open_application.rb +0 -71
data/README.md CHANGED
@@ -10,7 +10,7 @@
10
10
 
11
11
  ## Introduction
12
12
 
13
- Version : 4.17.0
13
+ Version : 4.18.1
14
14
 
15
15
  Laurent/2016-2024
16
16
 
@@ -70,7 +70,7 @@ For scripting and ad'hoc command line operations, `ascli` is perfect.
70
70
 
71
71
  Command line operations examples are shown using a shell such as: `bash` or `zsh`.
72
72
 
73
- Command line parameters in examples beginning with `my_`, e.g. `my_param_value`, are user-provided value and not fixed value commands.
73
+ Command line arguments beginning with `my_` in examples, e.g. `my_param_value`, are user-provided value and not fixed value commands.
74
74
 
75
75
  `ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
76
76
 
@@ -92,7 +92,7 @@ Once the gem is installed, `ascli` shall be accessible:
92
92
 
93
93
  ```console
94
94
  $ ascli --version
95
- 4.17.0
95
+ 4.18.1
96
96
  ```
97
97
 
98
98
  ### First use
@@ -122,10 +122,10 @@ ascli server browse /
122
122
  +------------+--------+-----------+-------+---------------------------+-----------------------+
123
123
  ```
124
124
 
125
- If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset)' for the server's authentication options.
125
+ If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset) for the server's authentication options.
126
126
  The following example will:
127
127
 
128
- - Create a [Option Preset](#option-preset)'
128
+ - Create a [Option Preset](#option-preset)
129
129
  - Define it as default for the `server` plugin
130
130
  - List files in a folder
131
131
  - Download a file
@@ -188,8 +188,8 @@ It is possible to install **either** directly on the host operating system (Linu
188
188
  The direct installation is recommended and consists in installing:
189
189
 
190
190
  - [Ruby](#ruby)
191
- - [aspera-cli](#ruby-gem)
192
- - [Aspera SDK (`ascp`)](#fasp-protocol)
191
+ - [aspera-cli](#ruby-gem-aspera-cli) <!-- markdownlint-disable-line -->
192
+ - [Aspera SDK (`ascp`)](#fasp-protocol-ascp)
193
193
 
194
194
  Ruby version: >= 2.6.
195
195
 
@@ -253,7 +253,7 @@ Install the chosen pre-compiled Ruby version:
253
253
  rvm install 3.2.2
254
254
  ```
255
255
 
256
- Ruby is now installed for the user, go to [Gem installation](#ruby-gem).
256
+ Ruby is now installed for the user, go to [Gem installation](#ruby-gem-aspera-cli). <!-- markdownlint-disable-line -->
257
257
 
258
258
  Alternatively RVM can be installed system-wide, for this execute as `root`.
259
259
  It then installs by default in `/usr/local/rvm` for all users and creates `/etc/profile.d/rvm.sh`.
@@ -300,52 +300,6 @@ Download the Ruby installer executable from <https://rubyinstaller.org/downloads
300
300
  rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
301
301
  ```
302
302
 
303
- Installation without network:
304
-
305
- It is essentially the same procedure, but instead of retrieving files from internet, copy the files from a machine with internet access, and then install from those archives:
306
-
307
- - Download the `exe` Ruby installer from <https://rubyinstaller.org/downloads/>
308
-
309
- ```bash
310
- v=$(curl -s https://rubyinstaller.org/downloads/|sed -nEe 's|.*(https://.*/releases/download/.*exe).*|\1|p'|head -n 1)
311
- curl -o ${v##*/} $v
312
- ```
313
-
314
- - Create an archive with necessary gems: <https://help.rubygems.org/kb/rubygems/installing-gems-with-no-network>
315
-
316
- ```bat
317
- gem install aspera-cli -N -i my_gems
318
- ```
319
-
320
- Zip the files `*.gem` from folder `repo/my_gems`
321
-
322
- - Download the SDK from: <https://ibm.biz/aspera_sdk>
323
-
324
- Create a Zip with all those files, and transfer to the target system.
325
-
326
- Then, on the target system:
327
-
328
- - Unzip the archive
329
- - Execute the installer:
330
-
331
- ```bat
332
- rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
333
- ```
334
-
335
- - Install the gems:
336
-
337
- ```bat
338
- gem install --force --local *.gem
339
- ```
340
-
341
- - Install the SDK
342
-
343
- ```bash
344
- ascli config ascp install --sdk-url=file:///sdk.zip
345
- ```
346
-
347
- > **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
348
-
349
303
  #### macOS: `brew`
350
304
 
351
305
  **macOS** come with Ruby.
@@ -388,6 +342,12 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
388
342
  dnf install -y ruby-devel
389
343
  ```
390
344
 
345
+ **Example:** Ubuntu
346
+
347
+ ```bash
348
+ apt-get install -y ruby-full
349
+ ```
350
+
391
351
  **Other examples:**
392
352
 
393
353
  ```bash
@@ -404,6 +364,26 @@ One can remove all installed gems, for example to start fresh:
404
364
  gem uninstall -axI $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
405
365
  ```
406
366
 
367
+ #### Linux as simple user
368
+
369
+ If you don't have root access, you can install Ruby in your home directory using `rbenv` see [rbenv-installer](https://github.com/rbenv/rbenv-installer#rbenv-installer):
370
+
371
+ ```bash
372
+ curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-installer | bash
373
+ ```
374
+
375
+ Then open a new terminal, or "source" the shell initialization script:
376
+
377
+ ```bash
378
+ source ~/.bashrc
379
+ ```
380
+
381
+ Then install Ruby:
382
+
383
+ ```bash
384
+ rbenv install 3.2.2
385
+ ```
386
+
407
387
  #### Other Unixes (AIX)
408
388
 
409
389
  Ruby is sometimes made available as an installable package through third party providers.
@@ -479,7 +459,7 @@ gem install rmagick grpc mimemagic
479
459
 
480
460
  > **Note:** Those are not installed as part of dependencies because they involve compilation of native code.
481
461
 
482
- ### Ruby Gem
462
+ ### Ruby Gem: `aspera-cli`
483
463
 
484
464
  Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
485
465
 
@@ -502,7 +482,26 @@ To check if a new version is available (independently of `version_check_days`):
502
482
  ascli config check_update
503
483
  ```
504
484
 
505
- ### FASP Protocol
485
+ #### Gem installation with signature verification
486
+
487
+ The gem is signed with a private key, and the public certificate is available in the github repository (`certs/aspera-cli-public-cert.pem`).
488
+ When installing the gem, the signature can be optionally verified.
489
+
490
+ For [secure installation](https://guides.rubygems.org/security/#using-gems), one can install the gem with the public key:
491
+
492
+ - import the verification certificate:
493
+
494
+ ```bash
495
+ gem cert --add <(curl -Ls https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem)
496
+ ```
497
+
498
+ - Then, install the gem:
499
+
500
+ ```bash
501
+ gem install -P MediumSecurity aspera-cli
502
+ ```
503
+
504
+ ### FASP Protocol: `ascp`
506
505
 
507
506
  Most file transfers will be executed using the **FASP** protocol, using `ascp`.
508
507
  Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
@@ -519,7 +518,7 @@ ascli config ascp install
519
518
  If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
520
519
 
521
520
  ```bash
522
- curl -Lso sdk.zip https://ibm.biz/aspera_sdk
521
+ curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
523
522
  ```
524
523
 
525
524
  ```bash
@@ -549,6 +548,18 @@ Refer to section: [Transfer Agents](#transfer-clients-agents)
549
548
 
550
549
  > **Note:** No pre-packaged version is provided yet.
551
550
 
551
+ #### Gem files and dependencies
552
+
553
+ The sample script: [examples/build_package.sh](examples/build_package.sh) can be used to download all necessary gems and dependencies in a tar gz.
554
+
555
+ ```console
556
+ $ ./build_package.sh aspera-cli 4.18.0
557
+
558
+ Archive: aspera-cli-4.18.0-gems.tgz
559
+ ```
560
+
561
+ #### Unix-like
562
+
552
563
  A method to build one is provided here:
553
564
 
554
565
  The procedure:
@@ -570,7 +581,7 @@ ascli --show-config --fields=sdk_url
570
581
  - Download the SDK archive from that URL
571
582
 
572
583
  ```bash
573
- curl -Lso sdk.zip https://ibm.biz/aspera_sdk
584
+ curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
574
585
  ```
575
586
 
576
587
  - Transfer those 2 files to the target system
@@ -593,15 +604,51 @@ ascli config ascp install --sdk-url=file:///sdk.zip
593
604
  source ~/.rvm/scripts/rvm
594
605
  ```
595
606
 
596
- > **Note:** Alternatively, to download all necessary gems in folder `my_gems`, execute:
607
+ #### Windows
608
+
609
+ Installation without network:
610
+
611
+ It is essentially the same procedure as installation for Windows with internet, but instead of retrieving files from internet, copy the files from a machine with internet access, and then install from those archives:
612
+
613
+ - Download the `exe` Ruby installer from <https://rubyinstaller.org/downloads/>
614
+
615
+ ```bash
616
+ v=$(curl -s https://rubyinstaller.org/downloads/|sed -nEe 's|.*(https://.*/releases/download/.*exe).*|\1|p'|head -n 1)
617
+ curl -o ${v##*/} $v
618
+ ```
619
+
620
+ - Create an archive with necessary gems like in previous section
621
+
622
+ - Download the SDK from: <https://ibm.biz/aspera_transfer_sdk>
623
+
624
+ - Create a Zip with all those files, and transfer to the target system.
625
+
626
+ Then, on the target system:
627
+
628
+ - Unzip the archive
629
+ - Execute the installer:
630
+
631
+ ```bat
632
+ rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
633
+ ```
634
+
635
+ - Install the gems: Extract the gem archive, and then:
636
+
637
+ ```bat
638
+ gem install --force --local *.gem
639
+ ```
640
+
641
+ - Install the SDK
597
642
 
598
643
  ```bash
599
- gem install aspera-cli -N -i my_gems
644
+ ascli config ascp install --sdk-url=file:///sdk.zip
600
645
  ```
601
646
 
647
+ > **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
648
+
602
649
  ### Container
603
650
 
604
- The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
651
+ The container image is: [`docker.io/martinlaurent/ascli`](https://hub.docker.com/r/docker.io/martinlaurent/ascli).
605
652
  The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
606
653
  To use the container, ensure that you have `podman` (or `docker`) installed.
607
654
 
@@ -615,7 +662,7 @@ podman --version
615
662
  Execute this:
616
663
 
617
664
  ```bash
618
- podman run --rm --tty --interactive --entrypoint bash martinlaurent/ascli:latest
665
+ podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/ascli:latest
619
666
  ```
620
667
 
621
668
  > **Note:** This command changes the entrypoint to an interactive shell instead of direct execution of `ascli`.
@@ -643,13 +690,13 @@ The entry point is `ascli` and the default command is `help`.
643
690
  The container can be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
644
691
 
645
692
  ```bash
646
- podman run --rm --tty --interactive martinlaurent/ascli:latest
693
+ podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest
647
694
  ```
648
695
 
649
696
  For more convenience, you may define a shell alias:
650
697
 
651
698
  ```bash
652
- alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
699
+ alias ascli='podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest'
653
700
  ```
654
701
 
655
702
  Then, you can execute the container like a local command:
@@ -659,7 +706,7 @@ ascli -v
659
706
  ```
660
707
 
661
708
  ```text
662
- 4.17.0
709
+ 4.18.1
663
710
  ```
664
711
 
665
712
  In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
@@ -686,12 +733,12 @@ In this case you need also to specify the shared transfer folder as a volume:
686
733
  --volume $HOME/xferdir:/xferfiles
687
734
  ```
688
735
 
689
- > **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
736
+ > **Note:** `ascli` is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
690
737
 
691
738
  And if you want all the above, simply use all the options:
692
739
 
693
740
  ```bash
694
- alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
741
+ alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash docker.io/martinlaurent/ascli:latest"
695
742
  ```
696
743
 
697
744
  ```bash
@@ -714,7 +761,7 @@ Some environment variables can be set for this script to adapt its behavior:
714
761
  |----------------|------------------------------------|--------------------------|--------------------------|
715
762
  | `ASCLI_HOME` | Configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
716
763
  | `docker_args` | Additional options to `podman` | &lt;empty&gt; | `--volume /Users:/Users` |
717
- | `image` | Container image name | `martinlaurent/ascli` | |
764
+ | `image` | Container image name | `docker.io/martinlaurent/ascli` | |
718
765
  | `version` | Container image version | Latest | `4.8.0.pre` |
719
766
 
720
767
  The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
@@ -748,8 +795,8 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
748
795
  - First create the image archive:
749
796
 
750
797
  ```bash
751
- podman pull martinlaurent/ascli
752
- podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
798
+ podman pull docker.io/martinlaurent/ascli
799
+ podman save docker.io/martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
753
800
  ```
754
801
 
755
802
  - Then, on air-gapped system:
@@ -787,7 +834,7 @@ dnf install singularity-ce
787
834
  Build an image like this:
788
835
 
789
836
  ```bash
790
- singularity build ascli.sif docker://martinlaurent/ascli
837
+ singularity build ascli.sif docker://docker.io/martinlaurent/ascli
791
838
  ```
792
839
 
793
840
  Then, start `ascli` like this:
@@ -802,6 +849,124 @@ Or get a shell with access to `ascli` like this:
802
849
  singularity shell ascli.sif
803
850
  ```
804
851
 
852
+ ### SSL library
853
+
854
+ `ascli` uses the Ruby `openssl` gem which uses by default the system's `openssl` library and its CA certificate bundle.
855
+
856
+ To display the version of **openssl** used in `ascli`:
857
+
858
+ ```bash
859
+ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
860
+ ```
861
+
862
+ It is possible to specify to use another SSL library or version by executing:
863
+
864
+ ```bash
865
+ gem install openssl -- --with-openssl-dir=[openssl library folder]
866
+ ```
867
+
868
+ Where `[openssl library folder]` is the path to the folder containing the `lib` and `include` folders of the `openssl` library.
869
+
870
+ For example, on macOS, to use the `openssl@3` library installed with `brew`:
871
+
872
+ ```bash
873
+ $ openssl version -e|sed -n 's|ENGINESDIR: "\(.*\)/lib[^/]*/.*|\1|p'
874
+ /opt/homebrew/Cellar/openssl@3/3.3.0
875
+ $ gem install openssl -- --with-openssl-dir=/opt/homebrew/Cellar/openssl@3/3.3.0
876
+ ```
877
+
878
+ ### SSL CA certificate bundle
879
+
880
+ SSL certificates are validated using a certificate store, by default it is the one of the system's `openssl` library.
881
+
882
+ To display trusted certificate store locations:
883
+
884
+ ```bash
885
+ ascli --show-config --fields=cert_stores
886
+ ```
887
+
888
+ Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
889
+ Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
890
+
891
+ One can display those default values:
892
+
893
+ ```bash
894
+ ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
895
+ ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
896
+ ```
897
+
898
+ In order to get certificate validation, the CA certificate bundle must be up-to-date.
899
+ Check this repository on how to update the system's CA certificate bundle: [https://github.com/millermatt/osca](https://github.com/millermatt/osca).
900
+
901
+ For example on RHEL/Rocky Linux:
902
+
903
+ ```bash
904
+ dnf install -y ca-certificates
905
+ update-ca-trust extract
906
+ ```
907
+
908
+ The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders.
909
+ By default it uses Ruby's default certificate store.
910
+
911
+ If you use this option, then default locations are not used.
912
+ Default locations can be added using special value `DEF`.
913
+ The value can be either an `Array` or `String` (path).
914
+ Successive options add paths incrementally.
915
+ All files of a folders are added.
916
+
917
+ JRuby uses its own implementation and CA bundles.
918
+
919
+ For example, on Linux to force the use the system's certificate store:
920
+
921
+ ```bash
922
+ --cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
923
+ ```
924
+
925
+ `ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
926
+
927
+ By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
928
+ Original `ascp`'s hardcoded locations can be found using:
929
+
930
+ ```bash
931
+ ascli config ascp info --fields=openssldir
932
+ ```
933
+
934
+ E.g. on macOS: `/Library/Aspera/ssl`.
935
+ Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
936
+ `ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
937
+
938
+ To update trusted root certificates for `ascli`:
939
+ Display the trusted certificate store locations used by `ascli`.
940
+ Typically done by updating the system's root certificate store.
941
+
942
+ An up-to-date version of the certificate bundle can also be retrieved with:
943
+
944
+ ```bash
945
+ ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
946
+ ```
947
+
948
+ To download that certificate store:
949
+
950
+ ```bash
951
+ ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
952
+ ```
953
+
954
+ Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
955
+
956
+ To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
957
+
958
+ ```bash
959
+ ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
960
+ ```
961
+
962
+ > **Note:** Use command `name` to display the remote common name of the remote certificate.
963
+
964
+ Then, use this file as certificate store (e.g. here, Node API):
965
+
966
+ ```bash
967
+ ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
968
+ ```
969
+
805
970
  ## Command Line Interface
806
971
 
807
972
  The command line tool is: ``ascli``
@@ -820,9 +985,9 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
820
985
 
821
986
  - Supports commands to Aspera server products (on-premise and SaaS)
822
987
  - Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
823
- - Supports Commands, Option values and Parameters shortcuts
988
+ - Supports Commands, Options, and Option values shortcuts
824
989
  - FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
825
- - Transfer parameters can be altered by modification of [*transfer-spec*](#transfer-specification), this includes requiring multi-session
990
+ - Transfer parameters can be altered by modification of [**transfer-spec**](#transfer-specification), this includes requiring multi-session
826
991
  - Allows transfers from products to products, essentially at node level (using the node transfer agent)
827
992
  - Supports FaspStream creation (using Node API)
828
993
  - Supports **Watchfolder** creation (using Node API)
@@ -856,19 +1021,20 @@ Moreover all `ascp` options are supported either through transfer spec parameter
856
1021
  ### Command line parsing, Special Characters
857
1022
 
858
1023
  `ascli` is typically executed in a shell, either interactively or in a script.
859
- `ascli` receives its arguments from this shell (through the Operating System).
1024
+ `ascli` receives its arguments on the command line.
1025
+ The way arguments are parsed and provided to `ascli` depend on the Operating System and shell.
860
1026
 
861
1027
  #### Shell parsing for Unix-like systems: Linux, macOS, AIX
862
1028
 
863
- Linux command line parsing is easy:
1029
+ Linux command line parsing is well defined:
864
1030
  It is fully documented in the shell's documentation.
865
1031
 
866
- On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
1032
+ On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh).
867
1033
  A c-shell (`csh`, `tcsh`) or other shell can also be used.
868
1034
  In this environment the shell parses the command line, possibly replacing variables, etc...
869
1035
  See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
870
1036
  The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
871
- Ruby receives a list parameters from shell and gives it to `ascli`.
1037
+ Ruby receives a list command line arguments from shell and gives it to `ascli`.
872
1038
  Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
873
1039
 
874
1040
  #### Shell parsing for Windows
@@ -960,16 +1126,16 @@ Major Minor Build Revision
960
1126
  ----- ----- ----- --------
961
1127
  5 1 19041 4046
962
1128
 
963
- PS C:\> ascli conf echo --% @json:'{"k":"v","x":"y"}'
1129
+ PS C:\> ascli config echo --% @json:'{"k":"v","x":"y"}'
964
1130
 
965
- PS C:\> ascli conf echo @json:'{"""k""":"""v""","""x""":"""y"""}'
1131
+ PS C:\> ascli config echo @json:'{"""k""":"""v""","""x""":"""y"""}'
966
1132
  ```
967
1133
 
968
1134
  > **Note:** The special powershell argument `--%` places powershell in legacy parsing mode.
969
1135
 
970
1136
  #### Extended Values (JSON, Ruby, ...)
971
1137
 
972
- Some of the `ascli` parameters are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
1138
+ Some of the values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
973
1139
  Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
974
1140
  JSON itself has some special syntax: for example `"` is used to enclose a `String`.
975
1141
 
@@ -1154,16 +1320,15 @@ ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
1154
1320
  {"title":"Test \" ' & \\"}
1155
1321
  ```
1156
1322
 
1157
- ### Commands, Options, Positional Arguments
1323
+ ### Positional Arguments and Options
1158
1324
 
1159
1325
  Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
1160
1326
  The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#command-line-parsing-special-characters).
1161
1327
 
1162
- `ascli` handles three types of command line arguments:
1328
+ `ascli` handles two types of command line arguments:
1163
1329
 
1164
- - Commands
1165
- - Positional Arguments
1166
- - Options
1330
+ - **Positional Arguments** : position is significant
1331
+ - **Options** : only order is significant, but not position
1167
1332
 
1168
1333
  For example:
1169
1334
 
@@ -1171,18 +1336,24 @@ For example:
1171
1336
  ascli command subcommand --option-name=VAL1 VAL2
1172
1337
  ```
1173
1338
 
1174
- - Executes **command**: `command subcommand`
1175
- - With one **option**: `option_name` and its **value**: `VAL1`
1176
- - The command has one additional **positional argument**: `VAL2`
1339
+ - Executes **Command** and its **Command Parameters** (**Positional Arguments**): `command subcommand VAL2`
1340
+ - With one **Option**: `option_name` and its **value**: `VAL1`
1177
1341
 
1178
1342
  If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
1179
1343
  For example `ascli config pre ov` is the same as `ascli config preset overview`.
1180
1344
 
1181
- The value of options and arguments is evaluated with the [Extended Value Syntax](#extended-value-syntax).
1345
+ The value of **Options** and **Positional Arguments** is evaluated with the [Extended Value Syntax](#extended-value-syntax).
1346
+
1347
+ #### Positional Arguments
1348
+
1349
+ **Positional Arguments** are either:
1182
1350
 
1183
- #### Commands
1351
+ - **Commands**, typically at the beginning
1352
+ - **Command Parameters** , e.g. creation data or entity identifier
1184
1353
 
1185
- Commands are typically entity types or verbs to act on those entities.
1354
+ When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
1355
+
1356
+ **Commands** are typically entity types or verbs to act on those entities.
1186
1357
  Its value is a `String` that must belong to a fixed list of values in a given context.
1187
1358
 
1188
1359
  Example:
@@ -1196,27 +1367,24 @@ ascli config ascp info
1196
1367
  - `ascp` is the second level command: name of the component (singleton)
1197
1368
  - `info` is the third level command: action to be performed
1198
1369
 
1199
- Typically, commands are located at the **beginning** of the command line.
1370
+ Typically, **Commands** are located at the **beginning** of the command line.
1200
1371
  Order is significant.
1201
1372
  The provided command must match one of the supported commands in the given context.
1202
1373
  If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
1203
1374
 
1204
- Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
1205
-
1206
- #### Positional Arguments
1207
-
1208
- Positional Arguments are typically mandatory values for a command, such as entity creation data.
1209
-
1210
- It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
1375
+ Some sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
1211
1376
 
1212
- The advantages of using a positional argument instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
1377
+ **Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
1213
1378
 
1379
+ > **Note:** It could also have been designed as an option.
1380
+ But since it is mandatory and typically these data need **not** be set in a configuration file, then it is better designed as a **Command Parameter**, and not designed an additional specific option.
1381
+ The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position) and the value is clearly mandatory.
1214
1382
  The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
1215
- Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@preset:`).
1383
+ Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
1216
1384
 
1217
- If a Positional Arguments begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
1385
+ If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
1218
1386
 
1219
- A few positional arguments are optional, they are located at the end of the command line.
1387
+ A few **Command Parameters** are optional, they are located at the end of the command line.
1220
1388
 
1221
1389
  #### Options
1222
1390
 
@@ -1225,13 +1393,15 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
1225
1393
  - Start with `--`
1226
1394
  - Have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
1227
1395
  - Have a value, separated from name with a `=`
1228
- - Can be used by prefix (avoid), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
1396
+ - Can be used by prefix (not recommended), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
1397
+ - Is optional on command line (it has a default value or no value)
1398
+ - whose position is not significant, but order is significant
1229
1399
 
1230
1400
  Exceptions:
1231
1401
 
1232
1402
  - Some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
1233
1403
  - Some options (flags) don't take a value, e.g. `-N`
1234
- - The special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`.
1404
+ - The special option `--` stops option processing and is ignored, following command line arguments are taken as **Positional Arguments**, including the ones starting with a `-`.
1235
1405
 
1236
1406
  Example:
1237
1407
 
@@ -1248,6 +1418,7 @@ ascli config echo -- --sample
1248
1418
  Options may have a (hardcoded) default value.
1249
1419
 
1250
1420
  Options can be placed anywhere on command line and are evaluated in order.
1421
+ Usually the last value evaluated overrides previous values, but some options are cumulative, e.g. `ts`.
1251
1422
 
1252
1423
  Options are typically optional: to change the default behavior.
1253
1424
  But some are mandatory, so they can be placed in a configuration file, for example: connection information.
@@ -1264,23 +1435,23 @@ Option `show_config` dry runs the configuration, and then returns currently set
1264
1435
  `ascli --show-config` outputs global options only, and `ascli [plugin] --show-config` outputs global and plugin default options.
1265
1436
  In addition, option `--show-config` can be added at the end of any full command line, this displays the options that would be used for the command.
1266
1437
 
1267
- A parameter is typically designed as option if:
1438
+ A command line argument is typically designed as option if:
1268
1439
 
1269
1440
  - It is optional, or
1270
- - It is a mandatory parameter that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
1441
+ - It is a mandatory parameter with a default value that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
1271
1442
 
1272
1443
  ### Interactive Input
1273
1444
 
1274
- Some options and parameters are mandatory and other optional.
1275
- By default, `ascli` will ask for missing mandatory options or parameters for interactive execution.
1445
+ Some options and **Command Parameters** are mandatory and other optional.
1446
+ By default, `ascli` will ask for missing mandatory options or **Command Parameters** for interactive execution.
1276
1447
 
1277
1448
  The behavior can be controlled with:
1278
1449
 
1279
1450
  - `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
1280
- - yes : missing mandatory parameters/options are asked to the user
1281
- - no : missing mandatory parameters/options raise an error message
1451
+ - yes : missing mandatory parameters/arguments are asked to the user
1452
+ - no : missing mandatory parameters/arguments raise an error message
1282
1453
  - `--ask-options=<yes|no>` (default=no)
1283
- - optional parameters/options are asked to user
1454
+ - optional parameters/arguments are asked to user
1284
1455
 
1285
1456
  ### Output
1286
1457
 
@@ -1303,25 +1474,36 @@ Depending on action, the output will contain:
1303
1474
  #### Format of output
1304
1475
 
1305
1476
  By default, result of type single_object and object_list are displayed using format `table`.
1306
- The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
1307
1477
 
1308
- In a table format, when displaying **Objects** (single, or list), by default, sub object are
1309
- flattened (option `flat_hash`). So, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
1310
- Setting `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
1311
- When in flatten mode, it is possible to filter fields by compound field name using dot as separator.
1478
+ The table style can be customized with option: `table_style` which expects a `Hash`, options are the ones described in gem [`terminal-table`](https://github.com/tj/terminal-table).
1479
+
1480
+ For example, to display a table with thick unicode borders:
1312
1481
 
1313
- Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
1482
+ ```bash
1483
+ ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
1484
+ ```
1485
+
1486
+ > **Note:** Other border styles exist, not limited to: `:unicode`, `:unicode_round`.
1487
+
1488
+ In a table format, when displaying **Objects** (single, or list), by default, sub object are flattened (option `flat_hash`).
1489
+ For example, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
1490
+ Setting option `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
1491
+ When in flatten mode, it is possible to filter fields using the option `fields` using the compound field name using `.` (dot) as separator.
1492
+
1493
+ Object lists are displayed one per line, with attributes as columns.
1494
+ Single objects (or tables with a single result) are transposed: one attribute per line.
1314
1495
  If transposition of single object is not desired, use option: `transpose_single` set to `no`.
1315
1496
 
1316
- The style of output can be set using the `format` parameter, supporting:
1497
+ The style of output can be set using the `format` option, supporting:
1317
1498
 
1499
+ - `table` : Text table (default)
1318
1500
  - `text` : Value as String
1319
- - `table` : Text table
1320
1501
  - `ruby` : Ruby code
1321
1502
  - `json` : JSON code
1322
1503
  - `jsonpp` : JSON pretty printed
1323
1504
  - `yaml` : YAML
1324
1505
  - `csv` : Comma Separated Values
1506
+ - `image` : Image URL or Image data
1325
1507
 
1326
1508
  #### Verbosity of output
1327
1509
 
@@ -1342,31 +1524,31 @@ To hide secrets from output, set option `show_secrets` to `no`.
1342
1524
 
1343
1525
  #### Option: `fields`: Selection of output object properties
1344
1526
 
1345
- By default, a table output will display one line per entry, and columns for each properties.
1346
- Depending on the command, columns may include by default all properties, or only some selected properties.
1527
+ By default, a `table` output format will display one line per entry, and columns for each fields (attributes, properties).
1528
+ Depending on the command, columns may include by default all fields, or only some selected fields.
1347
1529
  It is possible to define specific columns to be displayed, by setting the `fields` option.
1348
1530
 
1349
- The `fields` option can be either a comma separated list, or an extended value array.
1531
+ The `fields` option is a list that can be either a comma separated list or an extended value `Array`.
1350
1532
 
1351
- Elements of the list can be:
1533
+ Individual elements of the list can be:
1352
1534
 
1353
- - `DEF` : default display of columns (that's the default, when not set)
1354
- - `ALL` : all columns available
1355
- - `-`**property** : remove property from the current list
1356
1535
  - **property** : add property to the current list
1357
- - A Ruby `RegEx` : using `@ruby:'/.../'`
1536
+ - `-`**property** : remove property from the current list
1537
+ - `DEF` : default list of properties (that's the default, when not set)
1538
+ - `ALL` : all properties
1539
+ - A Ruby `RegEx` : using `@ruby:'/.../'`, or `@re:...` add those matching to the list
1358
1540
 
1359
1541
  Examples:
1360
1542
 
1361
- - `a,b,c` : the list of attributes specified as a comma separated list
1362
- - `@json:'["a","b","c"]'` : Array extended value: same as above
1363
- - `DEF,-a,b` : default property list, remove `a` and add `b`
1543
+ - `a,b,c` : the list of attributes specified as a comma separated list (overrides the all default)
1544
+ - `@json:'["a","b","c"]'` : `Array` extended value: same as above
1545
+ - `b,DEF,-a` : default property list, remove `a` and add `b` in first position
1364
1546
  - `@ruby:'/^server/'` : Display all properties whose name begin with `server`
1365
1547
 
1366
1548
  #### Option: `select`
1367
1549
 
1368
1550
  Table output (`object_list`) can be filtered using option `select`.
1369
- This parameter is either a `Hash` or `Proc`.
1551
+ This option is either a `Hash` or `Proc`.
1370
1552
  The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
1371
1553
 
1372
1554
  Example:
@@ -1384,7 +1566,7 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
1384
1566
  +-------------------------------+----------------------------------+-----------+
1385
1567
  ```
1386
1568
 
1387
- > **Note:** `select` filters elements from the result of command, while the `query` parameters gives filtering parameters to the API when listing elements.
1569
+ > **Note:** Option `select` filters elements from the result of command, while the `query` option gives filtering parameters to the API when listing elements.
1388
1570
 
1389
1571
  In above example, the same result is obtained with option:
1390
1572
 
@@ -1440,7 +1622,8 @@ The following decoders are supported:
1440
1622
  | `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`) |
1441
1623
  | `ruby` | String | Any | Execute specified Ruby code |
1442
1624
  | `secret` | None | String | Ask password interactively (hides input) |
1443
- | `stdin` | None | String | Read from stdin (no value on right) |
1625
+ | `stdin` | None | String | Read from stdin in text mode (no value on right) |
1626
+ | `stdbin` | None | String | Read from stdin in binary mode (no value on right) |
1444
1627
  | `uri` | String | String | Read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
1445
1628
  | `val` | String | String | Prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
1446
1629
  | `yaml` | String | Any | Decode YAML |
@@ -1575,44 +1758,46 @@ Temporary files are deleted at the end of execution unless option: `clean_temp`
1575
1758
 
1576
1759
  ### Configuration file
1577
1760
 
1578
- On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
1579
- Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
1761
+ On the first execution of `ascli`, an empty configuration file is created in the configuration folder (`ascli config folder`).
1762
+ There is no mandatory information required in this file.
1763
+ The use of it is optional as any option can be provided on the command line.
1580
1764
 
1581
- Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
1765
+ Although the file is a standard `YAML` file, `ascli` provides commands to read and modify it using the `config` command.
1582
1766
 
1583
- All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset)' in the configuration file.
1767
+ All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset) in the configuration file.
1584
1768
 
1585
- A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
1769
+ A configuration file provides a way to define default values, especially for authentication options, thus avoiding to always having to specify those options on the command line.
1586
1770
 
1587
- The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or equivalent env var).
1771
+ The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or its env var).
1588
1772
 
1589
- The configuration file is simply a catalog of pre-defined lists of options, called: [Option Preset](#option-preset)'. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset)' (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
1773
+ The configuration file is a catalog of named lists of options, called: [Option Preset](#option-preset).
1774
+ Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset) (e.g. `mypreset`) using the option `preset`: `--preset=mypreset` or its shortcut: `-Pmypreset`.
1590
1775
 
1591
1776
  #### Option Preset
1592
1777
 
1593
- A [Option Preset](#option-preset)' is simply a collection of parameters and their associated values in a named section in the configuration file.
1778
+ A [Option Preset](#option-preset) is a collection of options and their associated values in a named section in the configuration file.
1594
1779
 
1595
- A named [Option Preset](#option-preset)' can be modified directly using `ascli`, which will update the configuration file :
1780
+ A named [Option Preset](#option-preset) can be modified directly using `ascli`, which will update the configuration file :
1596
1781
 
1597
1782
  ```bash
1598
1783
  ascli config preset set|delete|show|initialize|update <option preset>
1599
1784
  ```
1600
1785
 
1601
- The command `update` allows the easy creation of [Option Preset](#option-preset)' by simply providing the options in their command line format, e.g. :
1786
+ The command `update` allows the easy creation of [Option Preset](#option-preset) by simply providing the options in their command line format, e.g. :
1602
1787
 
1603
1788
  ```bash
1604
1789
  ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
1605
1790
  ```
1606
1791
 
1607
- - This creates a [Option Preset](#option-preset)' `demo_server` with all provided options.
1792
+ - This creates a [Option Preset](#option-preset) `demo_server` with all provided options.
1608
1793
 
1609
- The command `set` allows setting individual options in a [Option Preset](#option-preset)'.
1794
+ The command `set` allows setting individual options in a [Option Preset](#option-preset).
1610
1795
 
1611
1796
  ```bash
1612
1797
  ascli config preset set demo_server password my_password_here
1613
1798
  ```
1614
1799
 
1615
- The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
1800
+ The command `initialize`, like `update` allows to set several options at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
1616
1801
 
1617
1802
  ```bash
1618
1803
  ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
@@ -1624,7 +1809,7 @@ A full terminal based overview of the configuration can be displayed using:
1624
1809
  ascli config preset over
1625
1810
  ```
1626
1811
 
1627
- A list of [Option Preset](#option-preset)' can be displayed using:
1812
+ A list of [Option Preset](#option-preset) can be displayed using:
1628
1813
 
1629
1814
  ```bash
1630
1815
  ascli config preset list
@@ -1647,6 +1832,15 @@ ascli config preset over
1647
1832
  ascli config preset list
1648
1833
  ```
1649
1834
 
1835
+ It is possible to load a [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
1836
+ For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
1837
+
1838
+ ```bash
1839
+ ascli config preset set pspecific preset pcommon
1840
+ ```
1841
+
1842
+ When `pspecific` is loaded, then cumulative option `preset` will be set and it will also load `pcommon`.
1843
+
1650
1844
  #### Special Option Preset: `config`
1651
1845
 
1652
1846
  This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
@@ -1686,7 +1880,7 @@ The default preset for `config` is read for any plugin invocation, this allows s
1686
1880
  When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
1687
1881
  If set, it loads the options independently of the plugin used.
1688
1882
 
1689
- > **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global parameters (e.g. `config ascp use`)
1883
+ > **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global options (e.g. `config ascp use`)
1690
1884
  >
1691
1885
  > **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
1692
1886
 
@@ -1735,6 +1929,8 @@ detect https://faspex4.example.com/path
1735
1929
  detect https://faspex5.example.com/path
1736
1930
  detect https://node.example.com/path
1737
1931
  detect https://shares.example.com/path shares
1932
+ detect https://tst.example.com/path faspio
1933
+ detect https://tst.example.com/path httpgw
1738
1934
  detect my_org aoc
1739
1935
  doc
1740
1936
  doc transfer-parameters
@@ -1760,10 +1956,11 @@ gem path
1760
1956
  gem version
1761
1957
  genkey my_key
1762
1958
  genkey my_key 4096
1959
+ image https://eudemo.asperademo.com/wallpaper.jpg
1763
1960
  initdemo
1764
1961
  open
1765
- plugin create my_command
1766
- plugin list
1962
+ plugins create my_command
1963
+ plugins list
1767
1964
  preset delete conf_name
1768
1965
  preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
1769
1966
  preset list
@@ -1813,17 +2010,17 @@ demo_server:
1813
2010
  We can see here:
1814
2011
 
1815
2012
  - The configuration was created with `ascli` version 0.3.7
1816
- - The default [Option Preset](#option-preset)' to load for `server` plugin is : `demo_server`
1817
- - The [Option Preset](#option-preset)' `demo_server` defines some parameters: the URL and credentials
1818
- - The default [Option Preset](#option-preset)' to load in any case is : `cli_default`
2013
+ - The default [Option Preset](#option-preset) to load for `server` plugin is : `demo_server`
2014
+ - The [Option Preset](#option-preset) `demo_server` defines some options: the URL and credentials
2015
+ - The default [Option Preset](#option-preset) to load in any case is : `cli_default`
1819
2016
 
1820
- Two [Option Preset](#option-preset)' are reserved:
2017
+ Two [Option Preset](#option-preset) are reserved:
1821
2018
 
1822
2019
  - `config` contains a single value: `version` showing the version used to create the configuration file.
1823
2020
  It is used to check compatibility.
1824
- - `default` is reserved to define the default [Option Preset](#option-preset)' name used for known plugins.
2021
+ - `default` is reserved to define the default [Option Preset](#option-preset) name used for known plugins.
1825
2022
 
1826
- The user may create as many [Option Preset](#option-preset)' as needed. For instance, a particular [Option Preset](#option-preset)' can be created for a particular application instance and contain URL and credentials.
2023
+ The user may create as many [Option Preset](#option-preset) as needed. For instance, a particular [Option Preset](#option-preset) can be created for a particular application instance and contain URL and credentials.
1827
2024
 
1828
2025
  Values in the configuration also follow the [Extended Value Syntax](#extended-value-syntax).
1829
2026
 
@@ -1833,7 +2030,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended-va
1833
2030
  ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
1834
2031
  ```
1835
2032
 
1836
- This creates the [Option Preset](#option-preset)':
2033
+ This creates the [Option Preset](#option-preset):
1837
2034
 
1838
2035
  ```yaml
1839
2036
  my_aoc_org:
@@ -1849,20 +2046,20 @@ Some options are global, some options are available only for some plugins. (the
1849
2046
  Options are loaded using this algorithm:
1850
2047
 
1851
2048
  - If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
1852
- - Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset)' for it, and loads it.
1853
- - If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset)' specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
2049
+ - Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset) for it, and loads it.
2050
+ - If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
1854
2051
  - Environment variables are evaluated
1855
2052
  - Command line options are evaluated
1856
2053
 
1857
- Parameters are evaluated in the order of command line.
2054
+ Options are evaluated in the order of command line.
1858
2055
 
1859
- To avoid loading the default [Option Preset](#option-preset)' for a plugin, use: `-N`
2056
+ To avoid loading the default [Option Preset](#option-preset) for a plugin, use: `-N`
1860
2057
 
1861
- On command line, words in parameter names are separated by a dash (`-`).
2058
+ On command line, words in option names are separated by a dash (`-`).
1862
2059
  In configuration file, separator is an underscore.
1863
2060
  E.g. `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
1864
2061
 
1865
- The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset)' for the main plugin with:
2062
+ The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset) for the main plugin with:
1866
2063
 
1867
2064
  ```bash
1868
2065
  ascli config preset set cli_default interactive no
@@ -1872,7 +2069,7 @@ ascli config preset set cli_default interactive no
1872
2069
  ascli config preset set default config cli_default
1873
2070
  ```
1874
2071
 
1875
- A [Option Preset](#option-preset)' value can be removed with `unset`:
2072
+ A [Option Preset](#option-preset) value can be removed with `unset`:
1876
2073
 
1877
2074
  ```bash
1878
2075
  ascli config preset unset cli_default interactive
@@ -1892,7 +2089,7 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
1892
2089
 
1893
2090
  #### Wizard
1894
2091
 
1895
- The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset)' with the provided information.
2092
+ The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset) with the provided information.
1896
2093
 
1897
2094
  It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
1898
2095
 
@@ -1914,7 +2111,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my
1914
2111
 
1915
2112
  This can also be provisioned in a configuration file:
1916
2113
 
1917
- - Build [Option Preset](#option-preset)'
2114
+ - Build [Option Preset](#option-preset)
1918
2115
 
1919
2116
  ```bash
1920
2117
  ascli config preset set shares06 url https://10.25.0.6
@@ -1934,7 +2131,7 @@ or
1934
2131
  ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=my_password_here
1935
2132
  ```
1936
2133
 
1937
- - Define this [Option Preset](#option-preset)' as the default [Option Preset](#option-preset)' for the specified plugin (`shares`)
2134
+ - Define this [Option Preset](#option-preset) as the default [Option Preset](#option-preset) for the specified plugin (`shares`)
1938
2135
 
1939
2136
  ```bash
1940
2137
  ascli config preset set default shares shares06
@@ -1946,7 +2143,7 @@ ascli config preset set default shares shares06
1946
2143
  ascli config preset overview
1947
2144
  ```
1948
2145
 
1949
- - Execute a command on the shares application using default parameters
2146
+ - Execute a command on the shares application using default options
1950
2147
 
1951
2148
  ```bash
1952
2149
  ascli shares repo browse /
@@ -2039,7 +2236,7 @@ The lookup is done by comparing the service URL and username (or access key).
2039
2236
 
2040
2237
  #### Securing passwords and secrets
2041
2238
 
2042
- A passwords can be saved in clear in a [Option Preset](#option-preset)' together with other account information (URL, username, etc...).
2239
+ A passwords can be saved in clear in a [Option Preset](#option-preset) together with other account information (URL, username, etc...).
2043
2240
  Example:
2044
2241
 
2045
2242
  ```bash
@@ -2060,25 +2257,31 @@ ascli config vault create myconf @json:'{"password":"my_password_here"}'
2060
2257
 
2061
2258
  ### Private Key
2062
2259
 
2063
- Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5, ...).
2064
- It consists in using a pair of keys: the private key and its associated public key.
2065
- The same key can be used for multiple applications.
2066
- Technically, a private key contains the public key, which can be extracted from it.
2067
- The file containing the private key can optionally be protected by a passphrase.
2260
+ Some Aspera applications allow the user to be authenticated using [Public Key Cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography):
2261
+
2262
+ - for SSH: Server
2263
+ - for OAuth JWT: AoC, Faspex5, Faspex, Shares
2264
+
2265
+ It consists in using a pair of associated keys: a private key and a public key.
2266
+ The same pair can be used for multiple applications.
2267
+ The file containing the private key (key pair) can optionally be protected by a passphrase.
2068
2268
  If the key is protected by a passphrase, then it will be prompted when used.
2069
2269
  (some plugins support option `passphrase`)
2070
2270
 
2071
- The following commands use the shell variable `PRIVKEYFILE`.
2271
+ The following sample commands use the shell variable `KEY_PAIR_PATH`.
2072
2272
  Set it to the desired safe location of the private key.
2073
- Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`:
2273
+ Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`.
2274
+ For example:
2074
2275
 
2075
2276
  ```bash
2076
- PRIVKEYFILE=~/.aspera/ascli/my_private_key
2277
+ KEY_PAIR_PATH=~/.aspera/ascli/my_private_key
2077
2278
  ```
2078
2279
 
2079
2280
  Several methods can be used to generate a key pair.
2080
2281
 
2081
- The format expected for private keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
2282
+ The format expected for keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
2283
+
2284
+ If another format is used, such as `DER`, it can be converted to `PEM`, e.g. using `openssl`.
2082
2285
 
2083
2286
  #### `ascli` for key generation
2084
2287
 
@@ -2087,21 +2290,13 @@ For convenience, the public key is also extracted with extension `.pub`.
2087
2290
  The key is not passphrase protected.
2088
2291
 
2089
2292
  ```bash
2090
- ascli config genkey ${PRIVKEYFILE} 4096
2293
+ ascli config genkey ${KEY_PAIR_PATH} 4096
2091
2294
  ```
2092
2295
 
2093
- > **Note:** `ascli` uses the `openssl` library.
2094
-
2095
2296
  To display the public key of a private key:
2096
2297
 
2097
2298
  ```bash
2098
- ascli config pubkey @file:${PRIVKEYFILE}
2099
- ```
2100
-
2101
- To display the version of **openssl** used in `ascli`:
2102
-
2103
- ```bash
2104
- ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
2299
+ ascli config pubkey @file:${KEY_PAIR_PATH}
2105
2300
  ```
2106
2301
 
2107
2302
  #### `ssh-keygen`
@@ -2109,16 +2304,16 @@ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
2109
2304
  Both private and public keys are generated, option `-N` is for passphrase.
2110
2305
 
2111
2306
  ```bash
2112
- ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
2307
+ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
2113
2308
  ```
2114
2309
 
2115
2310
  #### `openssl`
2116
2311
 
2117
- To generate a private key with a passphrase the following can be used on any system:
2312
+ To generate a key pair with a passphrase the following can be used on any system:
2118
2313
 
2119
2314
  ```bash
2120
- openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE} 4096
2121
- openssl rsa -pubout -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.pub
2315
+ openssl genrsa -passout pass:_passphrase_here_ -out ${KEY_PAIR_PATH} 4096
2316
+ openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub
2122
2317
  ```
2123
2318
 
2124
2319
  `openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
@@ -2127,112 +2322,55 @@ In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` t
2127
2322
  If option `-nodes` is not available, the passphrase can be removed using this method:
2128
2323
 
2129
2324
  ```bash
2130
- openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.no_des
2131
- mv ${PRIVKEYFILE}.no_des ${PRIVKEYFILE}
2325
+ openssl rsa -passin pass:_passphrase_here_ -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
2326
+ mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
2132
2327
  ```
2133
2328
 
2134
2329
  To change (or add) the passphrase for a key do:
2135
2330
 
2136
2331
  ```bash
2137
- openssl rsa -des3 -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.with_des
2138
- mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
2332
+ openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
2333
+ mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}
2139
2334
  ```
2140
2335
 
2141
- ### SSL CA certificate bundle
2142
-
2143
- The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders, by default it uses Ruby's default certificate store.
2144
-
2145
- To display trusted certificate store locations:
2146
-
2147
- ```bash
2148
- ascli --show-config --fields=cert_stores
2149
- ```
2150
-
2151
- Use option `cert_stores` to modify the locations of certificate stores (files or folders).
2152
- If you use this option, then default locations are not used.
2153
- Default locations can be added using special value `DEF`.
2154
- The value can be either an `Array` or `String` (path).
2155
- Successive options add paths incrementally.
2156
- All files of a folders are added.
2336
+ #### Using an application to generate a key pair
2157
2337
 
2158
- `ascli` uses the Ruby `openssl` gem.
2159
- By default it uses the system's `openssl` library, but JRuby uses its own implementation.
2338
+ Many applications are available, including on internet, to generate key pairs.
2339
+ For example: <https://cryptotools.net/rsagen>
2160
2340
 
2161
- For example, on Linux to use the system's certificate store:
2341
+ > **Note:** Be careful that private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
2162
2342
 
2163
- ```bash
2164
- --cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
2165
- ```
2166
-
2167
- Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
2168
- Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
2169
-
2170
- One can display those default values:
2171
-
2172
- ```bash
2173
- ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
2174
- ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
2175
- ```
2176
-
2177
- `ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of SSH).
2178
-
2179
- By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
2180
- Original `ascp`'s hardcoded locations can be found using:
2181
-
2182
- ```bash
2183
- ascli config ascp info --fields=openssldir
2184
- ```
2185
-
2186
- E.g. on macOS: `/Library/Aspera/ssl`.
2187
- Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
2188
- `ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
2343
+ ### Image and video thumbnails
2189
2344
 
2190
- To update trusted root certificates for `ascli`:
2191
- Display the trusted certificate store locations used by `ascli`.
2192
- Typically done by updating the system's root certificate store.
2345
+ `ascli` can display thumbnails for images and videos in the terminal.
2346
+ This is available:
2193
2347
 
2194
- An up-to-date version of the certificate bundle can also be retrieved with:
2348
+ - in the `thumbnail` command of `node` when using **gen4/access key** API.
2349
+ - when using the `show` command of `preview` plugin.
2350
+ - `coffee` and `image` commands of `config` plugin.
2351
+ - any displayed value which is an url to image can be displayed with option `format` set to `image`
2195
2352
 
2196
- ```bash
2197
- ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
2198
- ```
2353
+ The following options can be specified in the option `image`:
2199
2354
 
2200
- To download that certificate store:
2355
+ | Option | Type | Description |
2356
+ |------------|---------|-------------|
2357
+ | reserve | Integer | Lines reserved to display a status |
2358
+ | text | Bool | Display text instead of image|
2359
+ | double | Bool | Display double text resolution (half characters) |
2360
+ | font_ratio | Float | Font height/width ratio in terminal |
2201
2361
 
2202
2362
  ```bash
2203
- ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
2363
+ ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --image=@json:'{"text":true}'
2204
2364
  ```
2205
2365
 
2206
- Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
2207
-
2208
- To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
2209
-
2210
2366
  ```bash
2211
- ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
2367
+ curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdbin:
2212
2368
  ```
2213
2369
 
2214
- > **Note:** Use command `name` to display the remote common name of the remote certificate.
2215
-
2216
- Then, use this file as certificate store (e.g. here, Node API):
2217
-
2218
2370
  ```bash
2219
- ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
2371
+ echo -n https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @uri:@stdin:
2220
2372
  ```
2221
2373
 
2222
- ### Image and video thumbnails
2223
-
2224
- `ascli` can display thumbnails for images and videos in the terminal.
2225
- This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
2226
- It's also available when using the `show` command of `preview` plugin.
2227
-
2228
- The following options can be specified in the option `query`:
2229
-
2230
- | Option | Description |
2231
- |------------|-------------|
2232
- | text | Display text instead of image (Bool) |
2233
- | double | Display double text resolution (half characters) (Bool) |
2234
- | font_ratio | Font height/width ratio in terminal (Float) |
2235
-
2236
2374
  ### Graphical Interactions: Browser and Text Editor
2237
2375
 
2238
2376
  Some actions may require the use of a graphical tool:
@@ -2250,9 +2388,7 @@ It is also possible to force the graphical mode with option --ui :
2250
2388
 
2251
2389
  The gem is equipped with traces, mainly for debugging and learning APIs.
2252
2390
  By default logging level is `warn` and the output channel is `stderr`.
2253
- To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or a parameter in the configuration file).
2254
-
2255
- It is also possible to activate traces before log facility initialization using env var `ASCLI_LOG_LEVEL`.
2391
+ To increase debug level, use option `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or an [Option Preset](#option-preset)).
2256
2392
 
2257
2393
  By default passwords and secrets are removed from logs.
2258
2394
  Use option `log_secrets` set to `yes` to reveal secrets in logs.
@@ -2332,21 +2468,33 @@ Refer to the following sections.
2332
2468
 
2333
2469
  #### Proxy for REST and HTTP Gateway
2334
2470
 
2335
- REST API calls and transfers based on HTTP Gateway both use Ruby `Net::HTTP` gem.
2471
+ REST API calls and transfers based on HTTP Gateway both use Ruby's `Net::HTTP` class.
2472
+ Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
2336
2473
 
2337
- There are two possibilities to define an HTTP proxy to be used when Ruby HTTP is used.
2474
+ When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used .
2338
2475
 
2339
- The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy.
2476
+ The `http_proxy` environment variable (**lower case**) can be set to the **URL** of the proxy (with optional credentials).
2477
+ Syntax is: `(http|https)://[user:password@]host:port`.
2340
2478
  E.g. `http://myproxy.org.net:3128`.
2341
- Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
2342
2479
 
2343
- > **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
2480
+ > **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** valid.
2481
+
2482
+ Credentials for proxy are optional but can also be specified:
2344
2483
 
2345
2484
  ```bash
2346
- export http_proxy=http://proxy.example.com:3128
2485
+ export http_proxy=http://user:password@proxy.example.com:3128
2486
+ ```
2487
+
2488
+ Option `http_proxy` does the same (set env var) but on command line:
2489
+
2490
+ ```bash
2491
+ ascli --http-proxy=http://username:password@host:port ...
2347
2492
  ```
2348
2493
 
2349
2494
  Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) javascript value.
2495
+
2496
+ Note that proxy credentials are not supported in PAC files.
2497
+
2350
2498
  To read the script from a URL (`http:`, `https:` and `file:`), use prefix: `@uri:`.
2351
2499
  A minimal script can be specified to define the use of a local proxy:
2352
2500
 
@@ -2381,7 +2529,7 @@ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.
2381
2529
  PROXY proxy.example.com:8080
2382
2530
  ```
2383
2531
 
2384
- If the proxy requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
2532
+ If the proxy found with the PAC requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
2385
2533
 
2386
2534
  ```bash
2387
2535
  ascli --proxy-credentials=@json:'["__username_here__","__password_here__"]' ...
@@ -2397,7 +2545,7 @@ Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallb
2397
2545
 
2398
2546
  #### FASP proxy (forward) for transfers
2399
2547
 
2400
- To specify a FASP proxy (forward), set the [*transfer-spec*](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
2548
+ To specify a FASP proxy (forward), set the [**transfer-spec**](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
2401
2549
 
2402
2550
  For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
2403
2551
 
@@ -2413,7 +2561,20 @@ Or, alternatively, (prefer transfer spec like above, generally):
2413
2561
 
2414
2562
  ### FASP configuration
2415
2563
 
2416
- The `config` plugin also allows specification for the use of a local FASP **client**.
2564
+ `ascli` uses one of the transfer agents to execute transfers.
2565
+
2566
+ By default it uses the `direct` agent, which is basically a local `ascp`.
2567
+ Nevertheless, `ascli` does not come with `ascp` installed.
2568
+ This is the reason why it is advised to install the Aspera Transfer SDK during installation (`ascli config ascp install`).
2569
+
2570
+ By default, `ascli` uses the `ascp` binary found in **well known locations**, i.e. typical Aspera product installation paths.
2571
+
2572
+ The way to specify the location of `ascp` is to use either options:
2573
+
2574
+ - `ascp_path`
2575
+ - `use_product`
2576
+
2577
+ The `config` plugin allows to find and specify the location of `ascp`.
2417
2578
  It provides the following commands for `ascp` subcommand:
2418
2579
 
2419
2580
  - `show` : shows the path of `ascp` used
@@ -2447,9 +2608,9 @@ ascli config ascp info
2447
2608
 
2448
2609
  By default, `ascli` uses any found local product with `ascp`, including Transfer SDK.
2449
2610
 
2450
- To temporarily use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
2611
+ To override and use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
2451
2612
 
2452
- For a permanent change, the command `config ascp use` sets the same parameter for the global default.
2613
+ For a permanent change, the command `config ascp use` sets the same option for the global default.
2453
2614
 
2454
2615
  Using a POSIX shell:
2455
2616
 
@@ -2477,7 +2638,10 @@ Saved to default global preset global_common_defaults
2477
2638
 
2478
2639
  If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
2479
2640
 
2480
- #### List locally installed Aspera Transfer products
2641
+ If option `ascp_path` is not set, then the product identified with option `use_product` is used.
2642
+
2643
+ If `use_product` is not set, then the first product found is used,
2644
+ this is equivalent to using option: `--use-product=FIRST`.
2481
2645
 
2482
2646
  Locally installed Aspera products can be listed with:
2483
2647
 
@@ -2496,10 +2660,6 @@ ascli config ascp products list
2496
2660
  +---------------------------------------+----------------------------------------+
2497
2661
  ```
2498
2662
 
2499
- #### Selection of local client for `ascp` for [`direct`](#agent-direct) agent
2500
-
2501
- If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
2502
-
2503
2663
  Using the option `use_product` finds the `ascp` binary of the selected product.
2504
2664
 
2505
2665
  To permanently use the `ascp` of a product:
@@ -2556,12 +2716,12 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
2556
2716
 
2557
2717
  Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
2558
2718
 
2559
- When a transfer needs to be started, a [*transfer-spec*](#transfer-specification) has been internally prepared.
2560
- This [*transfer-spec*](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
2719
+ When a transfer needs to be started, a [**transfer-spec**](#transfer-specification) has been internally prepared.
2720
+ This [**transfer-spec**](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
2561
2721
 
2562
- There are currently 3 agents, set with option `transfer`:
2722
+ The following agents are supported and selected with option `transfer`:
2563
2723
 
2564
- - [`direct`](#agent-direct) : execution of `ascp`
2724
+ - [`direct`](#agent-direct) : direct execution of `ascp` (local)
2565
2725
  - [`trsdk`](#agent-transfer-sdk) : use of Aspera Transfer SDK (local)
2566
2726
  - [`connect`](#agent-connect-client) : use Connect Client (local)
2567
2727
  - [`alpha`](#agent-desktop-client) : use the new Desktop Client (local)
@@ -2572,7 +2732,7 @@ There are currently 3 agents, set with option `transfer`:
2572
2732
  For example, a node agent executing an **upload**, or **package send** operation
2573
2733
  will effectively push files to the related server from the agent node.
2574
2734
 
2575
- `ascli` standardizes on the use of a [*transfer-spec*](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
2735
+ `ascli` standardizes on the use of a [**transfer-spec**](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
2576
2736
 
2577
2737
  Specific options for agents are provided with option `transfer_info`, cumulatively.
2578
2738
 
@@ -2583,7 +2743,7 @@ This is the default agent for `ascli` (option `--transfer=direct`).
2583
2743
  `ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
2584
2744
  Refer to section [FASP](#fasp-configuration).
2585
2745
 
2586
- The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
2746
+ The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
2587
2747
 
2588
2748
  | Name | Type | Description |
2589
2749
  |------------------------|-------|-------------|
@@ -2607,7 +2767,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
2607
2767
  ```
2608
2768
 
2609
2769
  By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
2610
- In order to use a custom certificate store, use the `trusted_certs` parameter.
2770
+ In order to use a custom certificate store, use the `trusted_certs` option.
2611
2771
  To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
2612
2772
 
2613
2773
  Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
@@ -2726,6 +2886,7 @@ There are no option for `transfer_info`.
2726
2886
 
2727
2887
  By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
2728
2888
  This is especially useful for direct node-to-node transfers.
2889
+
2729
2890
  Parameters provided in option `transfer_info` are:
2730
2891
 
2731
2892
  | Name | Type | Description |
@@ -2735,7 +2896,7 @@ Parameters provided in option `transfer_info` are:
2735
2896
  | password | string | Password, secret or bearer token</br>Mandatory |
2736
2897
  | root_id | string | Root file id</br>Mandatory only for bearer token |
2737
2898
 
2738
- Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset)' :
2899
+ Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset) :
2739
2900
 
2740
2901
  ```bash
2741
2902
  --transfer-info=@preset:_name_here_
@@ -2749,12 +2910,14 @@ or be specified using the extended value syntax :
2749
2910
 
2750
2911
  If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
2751
2912
 
2752
- If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node.
2913
+ If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file id of the top folder to use on the node using this access key.
2753
2914
  It can be either the access key's root file id, or any authorized file id underneath it.
2754
2915
 
2755
2916
  #### Agent: HTTP Gateway
2756
2917
 
2757
- If it possible to send using a HTTP gateway, in case use of FASP is not allowed.
2918
+ The Aspera HTTP Gateway is a service that allows to send and receive files using HTTPS.
2919
+
2920
+ By specifying option: `--transfer=httpgw`, `ascli` will start transfers using the Aspera HTTP Gateway.
2758
2921
 
2759
2922
  Parameters provided in option `transfer_info` are:
2760
2923
 
@@ -2773,6 +2936,8 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
2773
2936
 
2774
2937
  > **Note:** The gateway only supports transfers authorized with a token.
2775
2938
 
2939
+ If the application, e.g. AoC or Faspex 5, is configured to use the HTTP Gateway, then `ascli` will automatically use the gateway URL if `--transfer=httpgw` is specified, so `transfer_info` becomes optional.
2940
+
2776
2941
  #### Agent: Transfer SDK
2777
2942
 
2778
2943
  Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
@@ -2783,7 +2948,7 @@ Options for `transfer_info` are:
2783
2948
  | Name | Type | Description |
2784
2949
  |----------|--------|-------------|
2785
2950
  | `url` | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: `:0` |
2786
- | `external` | bool | Use external daemon, do not start<br/>Default: `false` |
2951
+ | `external` | bool | Use external daemon, do not start one.<br/>Default: `false` |
2787
2952
  | `keep` | bool | Keep the daemon running after exiting `ascli`<br/>Default: `false` |
2788
2953
 
2789
2954
  > **Note:** If port zero is specified in the URL, then the daemon will listen on a random available port. If no address is specified, then `127.0.0.1` is used.
@@ -2812,7 +2977,7 @@ On Windows the compilation may fail for various reasons (3.1.1):
2812
2977
  ### Transfer Specification
2813
2978
 
2814
2979
  Some commands lead to file transfer (upload/download).
2815
- All parameters necessary for this transfer are described in a [*transfer-spec*](#transfer-specification) (Transfer Specification), such as:
2980
+ All parameters necessary for this transfer are described in a [**transfer-spec**](#transfer-specification) (Transfer Specification), such as:
2816
2981
 
2817
2982
  - Server address
2818
2983
  - Transfer user name
@@ -2820,11 +2985,11 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
2820
2985
  - File list
2821
2986
  - Etc...
2822
2987
 
2823
- `ascli` builds the [*transfer-spec*](#transfer-specification) internally as a `Hash`.
2988
+ `ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
2824
2989
  It is not necessary to provide additional parameters on the command line for this transfer.
2825
2990
 
2826
- It is possible to modify or add any of the supported [*transfer-spec*](#transfer-specification) parameter using the `ts` option.
2827
- The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [*transfer-spec*](#transfer-specification) parameters.
2991
+ It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameter using the `ts` option.
2992
+ The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
2828
2993
  Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
2829
2994
  To remove a (deep) key from transfer spec, set the value to `null`.
2830
2995
 
@@ -2834,15 +2999,15 @@ It is possible to specify `ascp` options when the `transfer` option is set to [`
2834
2999
  Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
2835
3000
  This is especially useful for `ascp` command line parameters not supported in the transfer spec.
2836
3001
 
2837
- The use of a [*transfer-spec*](#transfer-specification) instead of `ascp` parameters has the advantage of:
3002
+ The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the advantage of:
2838
3003
 
2839
3004
  - Common to all [Transfer Agent](#transfer-clients-agents)
2840
3005
  - Not dependent on command line limitations (special characters...)
2841
3006
 
2842
3007
  ### Transfer Parameters
2843
3008
 
2844
- All standard [*transfer-spec*](#transfer-specification) parameters can be specified.
2845
- [*transfer-spec*](#transfer-specification) can also be saved/overridden in the configuration file.
3009
+ All standard [**transfer-spec**](#transfer-specification) parameters can be specified.
3010
+ [**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
2846
3011
 
2847
3012
  References:
2848
3013
 
@@ -2889,7 +3054,7 @@ Columns:
2889
3054
  | fasp_port | int | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
2890
3055
  | fasp_url | string | &nbsp; | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Only used in Faspex.<br/>(&lt;ignored&gt;) |
2891
3056
  | file_checksum | string | Y | Y | &nbsp; | &nbsp; | &nbsp; | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(&lt;ignored&gt;) |
2892
- | http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
3057
+ | http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}\|{string}) |
2893
3058
  | http_fallback_port | int | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | Specifies http port when no cipher is used<br/>(-t {int}) |
2894
3059
  | https_fallback_port | int | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
2895
3060
  | keepalive | bool | Y | &nbsp; | &nbsp; | &nbsp; | &nbsp; | The session is running in persistent session mode.<br/>(--keepalive) |
@@ -2960,7 +3125,7 @@ The destination folder is set by `ascli` by default to:
2960
3125
  - `.` for downloads
2961
3126
  - `/` for uploads
2962
3127
 
2963
- It is specified by the [*transfer-spec*](#transfer-specification) parameter `destination_root`.
3128
+ It is specified by the [**transfer-spec**](#transfer-specification) parameter `destination_root`.
2964
3129
  As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
2965
3130
  The option `to_folder` provides an equivalent and convenient way to change this parameter:
2966
3131
  `--to-folder=<path>` .
@@ -2974,8 +3139,8 @@ By default the list of files to transfer is simply provided on the command line.
2974
3139
  The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
2975
3140
  The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
2976
3141
 
2977
- In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [*transfer-spec*](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
2978
- So, eventually, the list of files to transfer is provided to the transfer agent using the [*transfer-spec*](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
3142
+ In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
3143
+ So, eventually, the list of files to transfer is provided to the transfer agent using the [**transfer-spec**](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
2979
3144
  The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
2980
3145
 
2981
3146
  Possible values for option `sources` are:
@@ -3396,7 +3561,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
3396
3561
  ```text
3397
3562
  ascli -h
3398
3563
  NAME
3399
- ascli -- a command line tool for Aspera Applications (v4.17.0)
3564
+ ascli -- a command line tool for Aspera Applications (v4.18.1)
3400
3565
 
3401
3566
  SYNOPSIS
3402
3567
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -3418,7 +3583,7 @@ COMMANDS
3418
3583
  OPTIONS
3419
3584
  Options begin with a '-' (minus), and value is provided on command line.
3420
3585
  Special values are supported beginning with special prefix @pfx:, where pfx is one of:
3421
- val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
3586
+ val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, stdbin, yaml, zlib, extend, preset, vault
3422
3587
  Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
3423
3588
 
3424
3589
  ARGS
@@ -3427,20 +3592,21 @@ ARGS
3427
3592
  OPTIONS: global
3428
3593
  --interactive=ENUM Use interactive input of missing params: [no], yes
3429
3594
  --ask-options=ENUM Ask even optional options: [no], yes
3430
- --format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
3595
+ --struct-parser=ENUM Default parser when expected value is a struct: json, ruby
3596
+ --format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv, image
3431
3597
  --output=VALUE Destination for results (String)
3432
3598
  --display=ENUM Output only some information: [info], data, error
3433
3599
  --fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
3434
3600
  --select=VALUE Select only some items in lists: column, value (Hash, Proc)
3435
- --table-style=VALUE Table display style
3601
+ --table-style=VALUE Table display style (Hash)
3436
3602
  --flat-hash=ENUM Display deep values as additional keys: no, [yes]
3437
3603
  --transpose-single=ENUM Single object fields output vertically: no, [yes]
3438
3604
  --show-secrets=ENUM Show secrets on command output: [no], yes
3605
+ --image=VALUE Options for image display (Hash)
3439
3606
  -h, --help Show this message
3440
3607
  --bash-comp Generate bash completion for command
3441
3608
  --show-config Display parameters used for the provided action
3442
3609
  -v, --version Display version
3443
- -w, --warnings Check for language warnings
3444
3610
  --ui=ENUM Method to start browser: text, [graphical]
3445
3611
  --log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
3446
3612
  --logger=ENUM Logging method: [stderr], stdout, syslog
@@ -3451,7 +3617,7 @@ OPTIONS: global
3451
3617
  --pid-file=VALUE Write process identifier to file, delete on exit (String)
3452
3618
 
3453
3619
  COMMAND: config
3454
- SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
3620
+ SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
3455
3621
  OPTIONS:
3456
3622
  --home=VALUE Home folder for tool (String)
3457
3623
  --config-file=VALUE Path to YAML file with preset configuration
@@ -3461,7 +3627,6 @@ OPTIONS:
3461
3627
  --query=VALUE Additional filter for for some commands (list/delete) (Hash)
3462
3628
  --value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
3463
3629
  --property=VALUE Name of property to set (modify operation)
3464
- --id=VALUE Resource identifier (deprecated: (4.14) Use positional identifier after verb (modify,delete,show))
3465
3630
  --bulk=ENUM Bulk operation (only some): [no], yes
3466
3631
  --bfail=ENUM Bulk operation error handling: no, [yes]
3467
3632
  -N, --no-default Do not load default configuration for plugin
@@ -3485,9 +3650,10 @@ OPTIONS:
3485
3650
  --silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
3486
3651
  --cert-stores=VALUE List of folder with trusted certificates (Array, String)
3487
3652
  --http-options=VALUE Options for HTTP/S socket (Hash)
3653
+ --http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
3488
3654
  --cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
3489
3655
  --fpac=VALUE Proxy auto configuration script
3490
- --proxy-credentials=VALUE HTTP proxy credentials (user and password) (Array)
3656
+ --proxy-credentials=VALUE HTTP proxy credentials for fpac. Array: user,password (Array)
3491
3657
  --ts=VALUE Override transfer spec values (Hash)
3492
3658
  --to-folder=VALUE Destination folder for transferred files
3493
3659
  --sources=VALUE How list of transferred files is provided (@args,@ts,Array)
@@ -3505,7 +3671,7 @@ OPTIONS:
3505
3671
 
3506
3672
 
3507
3673
  COMMAND: node
3508
- SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer upload watch_folder
3674
+ SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer transport upload watch_folder
3509
3675
  OPTIONS:
3510
3676
  --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3511
3677
  --username=VALUE User's name to log in
@@ -3518,28 +3684,32 @@ OPTIONS:
3518
3684
  --sync-info=VALUE Information for sync instance and sessions (Hash)
3519
3685
 
3520
3686
 
3521
- COMMAND: orchestrator
3522
- SUBCOMMANDS: health info plugins processes workflow
3687
+ COMMAND: faspio
3688
+ SUBCOMMANDS: bridges health
3523
3689
  OPTIONS:
3524
3690
  --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3525
3691
  --username=VALUE User's name to log in
3526
3692
  --password=VALUE User's password
3527
- --result=VALUE Specify result value as: 'work_step:parameter'
3528
- --synchronous=ENUM Wait for completion: [no], yes
3529
- --ret-style=ENUM How return type is requested in api: header, [arg], ext
3530
- --auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
3693
+ --auth=ENUM OAuth type of authentication: jwt, basic
3694
+ --client-id=VALUE OAuth client identifier
3695
+ --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
3696
+ --passphrase=VALUE OAuth JWT RSA private key passphrase
3531
3697
 
3532
3698
 
3533
- COMMAND: bss
3534
- SUBCOMMANDS: subscription
3699
+ COMMAND: orchestrator
3700
+ SUBCOMMANDS: health info plugins processes workflow
3535
3701
  OPTIONS:
3536
3702
  --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3537
3703
  --username=VALUE User's name to log in
3538
3704
  --password=VALUE User's password
3705
+ --result=VALUE Specify result value as: 'work_step:parameter'
3706
+ --synchronous=ENUM Wait for completion: [no], yes
3707
+ --ret-style=ENUM How return type is requested in api: header, [arg], ext
3708
+ --auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
3539
3709
 
3540
3710
 
3541
3711
  COMMAND: alee
3542
- SUBCOMMANDS: entitlement
3712
+ SUBCOMMANDS: entitlement health
3543
3713
  OPTIONS:
3544
3714
  --url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
3545
3715
  --username=VALUE User's name to log in
@@ -3587,6 +3757,12 @@ OPTIONS:
3587
3757
  --identity=VALUE Authentication URL (https://iam.cloud.ibm.com/identity)
3588
3758
 
3589
3759
 
3760
+ COMMAND: httpgw
3761
+ SUBCOMMANDS: health info
3762
+ OPTIONS:
3763
+ --url=VALUE URL of application, e.g. https://app.example.com/aspera/app
3764
+
3765
+
3590
3766
  COMMAND: faspex
3591
3767
  SUBCOMMANDS: address_book dropbox health login_methods me package source v4
3592
3768
  OPTIONS:
@@ -3649,9 +3825,9 @@ OPTIONS:
3649
3825
  --scope=VALUE OAuth scope for AoC API calls
3650
3826
  --redirect-uri=VALUE OAuth API client redirect URI
3651
3827
  --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
3652
- --passphrase=VALUE RSA private key passphrase
3828
+ --passphrase=VALUE RSA private key passphrase (String)
3653
3829
  --workspace=VALUE Name of workspace (String, NilClass)
3654
- --new-user-option=VALUE New user creation option for unknown package recipients
3830
+ --new-user-option=VALUE New user creation option for unknown package recipients (Hash)
3655
3831
  --validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
3656
3832
  --validator=VALUE Identifier of validator (optional for central)
3657
3833
  --asperabrowserurl=VALUE URL for simple aspera web ui
@@ -3725,7 +3901,7 @@ Aspera on Cloud and Faspex 5 rely on Oauth.
3725
3901
  By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
3726
3902
 
3727
3903
  ```bash
3728
- ascli --show-config --select=@json:'{"key":"plugin_folder"}'
3904
+ ascli --show-config --fields=plugin_folder
3729
3905
  ```
3730
3906
 
3731
3907
  You can create the skeleton of a new plugin like this:
@@ -3805,7 +3981,7 @@ Several types of OAuth authentication are supported:
3805
3981
 
3806
3982
  The authentication method is controlled by option `auth`.
3807
3983
 
3808
- For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset)' for Aspera on Cloud](#configuration-for-aspera-on-cloud).
3984
+ For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset) for Aspera on Cloud](#configuration-for-aspera-on-cloud).
3809
3985
 
3810
3986
  For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
3811
3987
 
@@ -3842,9 +4018,9 @@ Once the client is registered, a **Client ID** and **Secret** are created, these
3842
4018
 
3843
4019
  #### Configuration for Aspera on Cloud
3844
4020
 
3845
- If you did not use the wizard, you can also manually create a [Option Preset](#option-preset)' for `ascli` in its configuration file.
4021
+ If you did not use the wizard, you can also manually create a [Option Preset](#option-preset) for `ascli` in its configuration file.
3846
4022
 
3847
- Lets create a [Option Preset](#option-preset)' called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
4023
+ Lets create a [Option Preset](#option-preset) called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
3848
4024
 
3849
4025
  ```bash
3850
4026
  ascli config preset ask my_aoc_org url client_id client_secret
@@ -3858,7 +4034,7 @@ updated: my_aoc_org
3858
4034
 
3859
4035
  (This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
3860
4036
 
3861
- Define this [Option Preset](#option-preset)' as default configuration for the `aspera` plugin:
4037
+ Define this [Option Preset](#option-preset) as default configuration for the `aspera` plugin:
3862
4038
 
3863
4039
  ```bash
3864
4040
  ascli config preset set default aoc my_aoc_org
@@ -3949,9 +4125,9 @@ modified
3949
4125
 
3950
4126
  > **Note:** The `aspera user info show` command can be used to verify modifications.
3951
4127
 
3952
- #### [Option Preset](#option-preset)' modification for JWT
4128
+ #### [Option Preset](#option-preset) modification for JWT
3953
4129
 
3954
- To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset)', do the following:
4130
+ To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset), do the following:
3955
4131
 
3956
4132
  - Change auth method to JWT
3957
4133
  - Provide location of private key
@@ -3985,7 +4161,7 @@ In that case, it is possible to list those shared folder by using a value for op
3985
4161
 
3986
4162
  #### AoC: First Use
3987
4163
 
3988
- Once client has been registered and [Option Preset](#option-preset)' created: `ascli` can be used:
4164
+ Once client has been registered and [Option Preset](#option-preset) created: `ascli` can be used:
3989
4165
 
3990
4166
  ```bash
3991
4167
  ascli aoc files br /
@@ -4188,7 +4364,7 @@ Options:
4188
4364
  #### Transfer: Using specific transfer ports
4189
4365
 
4190
4366
  By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
4191
- The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
4367
+ The option `default_ports` ([yes]/no) allows `ascli` to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
4192
4368
 
4193
4369
  #### Using ATS
4194
4370
 
@@ -4588,7 +4764,7 @@ General syntax:
4588
4764
  ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
4589
4765
  ```
4590
4766
 
4591
- Package creation parameter are sent as positional argument.
4767
+ Package creation parameter are sent as **Command Parameter**.
4592
4768
  Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
4593
4769
 
4594
4770
  List allowed shared inbox destinations with:
@@ -4766,13 +4942,13 @@ ascli aoc files short_link public delete _id_
4766
4942
 
4767
4943
  It is possible to transfer files directly between organizations without having to first download locally and then upload...
4768
4944
 
4769
- Although optional, the creation of [Option Preset](#option-preset)' is recommended to avoid placing all parameters in the command line.
4945
+ Although optional, the creation of [Option Preset](#option-preset) is recommended to avoid placing all parameters in the command line.
4770
4946
 
4771
4947
  Procedure to send a file from org1 to org2:
4772
4948
 
4773
- - Get access to Organization 1 and create a [Option Preset](#option-preset)': e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
4949
+ - Get access to Organization 1 and create a [Option Preset](#option-preset): e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
4774
4950
  - Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
4775
- - Get access to Organization 2 and create a [Option Preset](#option-preset)': e.g. `org2`
4951
+ - Get access to Organization 2 and create a [Option Preset](#option-preset): e.g. `org2`
4776
4952
  - Check that access works and locate the destination folder `mydestfolder`
4777
4953
  - Execute the following:
4778
4954
 
@@ -4812,6 +4988,7 @@ For instructions, refer to section `find` for plugin `node`.
4812
4988
  admin analytics transfers nodes
4813
4989
  admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
4814
4990
  admin analytics transfers users --once_only=yes
4991
+ admin application list
4815
4992
  admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
4816
4993
  admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
4817
4994
  admin ats access_key delete ak1ibmcloud
@@ -4822,36 +4999,35 @@ admin ats cluster list
4822
4999
  admin ats cluster show --cloud=aws --region=eu-west-1
4823
5000
  admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
4824
5001
  admin auth_providers list
4825
- admin res application list
4826
- admin res client list
4827
- admin res client_access_key list
4828
- admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
4829
- admin res client_registration_token delete client_reg_id
4830
- admin res client_registration_token list
4831
- admin res contact list
4832
- admin res dropbox list
4833
- admin res dropbox_membership list
4834
- admin res group list
4835
- admin res kms_profile list
4836
- admin res node list
4837
- admin res operation list
4838
- admin res organization show
4839
- admin res package list --http-options=@json:'{"read_timeout":120.0}'
4840
- admin res saml_configuration list
4841
- admin res self show
4842
- admin res short_link list
4843
- admin res user list
4844
- admin res user modify %name:my_user_email @json:'{"deactivated":false}'
4845
- admin res workspace_membership list
4846
- admin resource node do %name:my_ak_name --secret=my_ak_secret browse /
4847
- admin resource node do %name:my_ak_name --secret=my_ak_secret delete /folder1
4848
- admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
4849
- admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
4850
- admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
4851
- admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
4852
- admin resource workspace list
4853
- admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
5002
+ admin client list
5003
+ admin client_access_key list
5004
+ admin client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
5005
+ admin client_registration_token delete client_reg_id
5006
+ admin client_registration_token list
5007
+ admin contact list
5008
+ admin dropbox list
5009
+ admin dropbox_membership list
5010
+ admin group list
5011
+ admin kms_profile list
5012
+ admin node do %name:my_ak_name --secret=my_ak_secret browse /
5013
+ admin node do %name:my_ak_name --secret=my_ak_secret delete /folder1
5014
+ admin node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
5015
+ admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
5016
+ admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
5017
+ admin node do %name:my_ak_name --secret=my_ak_secret v3 events
5018
+ admin node list
5019
+ admin operation list
5020
+ admin organization show
5021
+ admin package list --http-options=@json:'{"read_timeout":120.0}'
5022
+ admin saml_configuration list
5023
+ admin self show
5024
+ admin short_link list
4854
5025
  admin subscription
5026
+ admin user list
5027
+ admin user modify %name:my_user_email @json:'{"deactivated":false}'
5028
+ admin workspace list
5029
+ admin workspace_membership list
5030
+ admin workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
4855
5031
  automation workflow action wf_id create @json:'{"name":"toto"}' \
4856
5032
  automation workflow create @json:'{"name":"test_workflow"}'
4857
5033
  automation workflow delete wf_id
@@ -4865,6 +5041,9 @@ files browse /
4865
5041
  files browse / --url=my_private_link
4866
5042
  files browse / --url=my_public_link_folder_no_pass
4867
5043
  files browse / --url=my_public_link_folder_pass --password=my_public_link_password
5044
+ files browse my_remote_file
5045
+ files browse my_remote_folder
5046
+ files browse my_remote_folder/
4868
5047
  files delete /testsrc
4869
5048
  files download --transfer=alpha testdst/test_file.bin
4870
5049
  files download --transfer=connect testdst/test_file.bin
@@ -4924,9 +5103,9 @@ user workspaces list
4924
5103
 
4925
5104
  ATS is usable either :
4926
5105
 
4927
- - From an AoC subscription : ascli aoc admin ats : use AoC authentication
5106
+ - From an AoC subscription : `ascli aoc admin ats` : use AoC authentication
4928
5107
 
4929
- - Or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
5108
+ - Or from an IBM Cloud subscription : `ascli ats` : use IBM Cloud API key authentication
4930
5109
 
4931
5110
  ### IBM Cloud ATS : Creation of api key
4932
5111
 
@@ -5098,8 +5277,8 @@ mkdir my_upload_folder/target_hot
5098
5277
  mv my_upload_folder/200KB.2 my_upload_folder/to.delete
5099
5278
  sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
5100
5279
  sync admin status --sync-info=@json:'{"name":"sync2"}'
5101
- sync admin status my_sync --sync-info=@json:'{"sessions":[{"name":"my_sync","local_dir":"/data/local_sync"}]}'
5102
- sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
5280
+ sync admin status my_srv_sync --sync-info=@json:'{"sessions":[{"name":"my_srv_sync","local_dir":"/data/local_sync"}]}'
5281
+ sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_srv_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
5103
5282
  sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
5104
5283
  upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
5105
5284
  upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
@@ -5203,7 +5382,7 @@ ascli server browse /aspera-test-dir-large
5203
5382
  ascli server download /aspera-test-dir-large/200MB
5204
5383
  ```
5205
5384
 
5206
- `initdemo` creates a [Option Preset](#option-preset)' `demoserver` and set it as default for plugin `server`.
5385
+ `initdemo` creates a [Option Preset](#option-preset) `demoserver` and set it as default for plugin `server`.
5207
5386
 
5208
5387
  If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
5209
5388
 
@@ -5235,6 +5414,18 @@ Example:
5235
5414
  - `ascli node browse /` : list files with **gen3/node user** API
5236
5415
  - `ascli node access_key do self browse /` : list files with **gen4/access key** API
5237
5416
 
5417
+ #### Browse
5418
+
5419
+ Native API parameters can be placed in option `query`.
5420
+
5421
+ Special parameters can be placed in option `query` for "gen3" browse:
5422
+
5423
+ | Parameter | Description |
5424
+ |-----------|-------------|
5425
+ | `recursive` | Recursively list files |
5426
+ | `max` | Maximum number of files to list |
5427
+ | `self` | Offset in the list |
5428
+
5238
5429
  ### Operation `find` on **gen4/access key**
5239
5430
 
5240
5431
  The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
@@ -5337,7 +5528,7 @@ There are three sync types of operations:
5337
5528
 
5338
5529
  It is possible to start a FASPStream session using the node API:
5339
5530
 
5340
- Use the command `ascli node stream create --ts=@json:<value>`, with [*transfer-spec*](#transfer-specification):
5531
+ Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-spec**](#transfer-specification):
5341
5532
 
5342
5533
  ```json
5343
5534
  {"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
@@ -5643,6 +5834,7 @@ sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"
5643
5834
  sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
5644
5835
  transfer list --query=@json:'{"active_only":true}'
5645
5836
  transfer sessions
5837
+ transport
5646
5838
  upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
5647
5839
  upload --username=my_ak_name --password=my_ak_secret test_file.bin
5648
5840
  upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
@@ -5664,7 +5856,7 @@ IBM Aspera's newer self-managed application.
5664
5856
 
5665
5857
  > **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
5666
5858
 
5667
- For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset)':
5859
+ For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset):
5668
5860
 
5669
5861
  ```bash
5670
5862
  ascli config wizard
@@ -5802,24 +5994,30 @@ ascli config preset update f5boot --url=https://localhost/aspera/faspex --auth=b
5802
5994
  > **Note:** Add `ascli faspex5` in front of the commands:
5803
5995
 
5804
5996
  ```bash
5805
- admin res accounts list
5806
- admin res contacts list
5807
- admin res jobs list
5808
- admin res metadata_profiles list
5809
- admin res node list
5810
- admin res oauth_clients list
5811
- admin res registrations list
5812
- admin res saml_configs list
5813
- admin res shared_inboxes invite %name:my_shared_box_name johnny@example.com
5814
- admin res shared_inboxes list
5815
- admin res shared_inboxes list --query=@json:'{"all":true}'
5816
- admin res shared_inboxes members %name:my_shared_box_name create %name:john@example.com
5817
- admin res shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
5818
- admin res shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
5819
- admin res shared_inboxes members %name:my_shared_box_name list
5820
- admin res workgroups list
5997
+ admin accounts list
5998
+ admin clean_deleted
5999
+ admin contacts list
6000
+ admin distribution_lists create @json:'{"name":"test4","contacts":[{"name":"john@example.com"}]}'
6001
+ admin distribution_lists delete %name:test4
6002
+ admin distribution_lists list --query=@json:'{"type":"global"}'
6003
+ admin event app
6004
+ admin event web
6005
+ admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
6006
+ admin metadata_profiles list
6007
+ admin node list
6008
+ admin oauth_clients list
6009
+ admin registrations list
6010
+ admin saml_configs list
6011
+ admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
6012
+ admin shared_inboxes list
6013
+ admin shared_inboxes list --query=@json:'{"all":true}'
6014
+ admin shared_inboxes members %name:my_shared_box_name create %name:john@example.com
6015
+ admin shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
6016
+ admin shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
6017
+ admin shared_inboxes members %name:my_shared_box_name list
5821
6018
  admin smtp show
5822
6019
  admin smtp test my_email_external
6020
+ admin workgroups list
5823
6021
  bearer_token
5824
6022
  gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
5825
6023
  health
@@ -5853,7 +6051,7 @@ user profile show
5853
6051
  ```
5854
6052
 
5855
6053
  Most commands are directly REST API calls.
5856
- Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional argument for creation.
6054
+ Parameters to commands are carried through option `query`, as extended value, for `list`, or through **Command Parameter** for creation.
5857
6055
  One can conveniently use the JSON format with prefix `@json:`.
5858
6056
 
5859
6057
  > **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
@@ -5880,7 +6078,7 @@ To select another inbox, use option `box` with one of the following values:
5880
6078
 
5881
6079
  ### Faspex 5: Send a package
5882
6080
 
5883
- The `Hash` creation parameter provided to command `faspex5 packages send [extended value: Hash with package info ] [files...]` corresponds to the Faspex 5 API: `POST /packages`.
6081
+ The `Hash` creation **Command Parameter** provided to command `faspex5 packages send [extended value: Hash with package info ] [files...]` corresponds to the Faspex 5 API: `POST /packages`.
5884
6082
 
5885
6083
  The interface is the one of the API (Refer to Faspex5 API documentation, or look at request in browser).
5886
6084
 
@@ -5931,7 +6129,7 @@ Basically, add the field `metadata`, with one key per metadata and the value is
5931
6129
 
5932
6130
  ### Faspex 5: List packages
5933
6131
 
5934
- Option `box` can be used to list packages from a specific box (see **Inbox selection** above).
6132
+ Option `box` can be used to list packages from a specific box (see [Inbox Selection](#faspex-5-inbox-selection) above).
5935
6133
 
5936
6134
  Option `query` can be used to filter the list of packages, based on native API parameters, directly sent to [Faspex 5 API `GET /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#getAllPackages).
5937
6135
 
@@ -5944,7 +6142,7 @@ Option `query` can be used to filter the list of packages, based on native API p
5944
6142
  | `max` | Special | Maximum number of items to retrieve (stop pages when the maximum is passed) |
5945
6143
  | `pmax` | Special | Maximum number of pages to request (stop pages when the maximum is passed) |
5946
6144
 
5947
- A positional parameter in last position, of type `Proc`, can be used to filter the list of packages.
6145
+ A **Command Parameter** in last position, of type `Proc`, can be used to filter the list of packages.
5948
6146
  This advantage of this method is that the expression can be any test, even complex, as it is Ruby code.
5949
6147
  But the disadvantage is that the filtering is done in `ascli` and not in Faspex 5, so it is less efficient.
5950
6148
 
@@ -5972,7 +6170,7 @@ Option `query` is available.
5972
6170
 
5973
6171
  To list recursively add option `--query=@json:{"recursive":true}`.
5974
6172
 
5975
- > **Note:** Option `recirsive` makes recursive API calls, so it can take a long time on large packages.
6173
+ > **Note:** Option `recursive` makes recursive API calls, so it can take a long time on large packages.
5976
6174
 
5977
6175
  To limit the number of items retrieved, use option `--query=@json:{"max":10}`.
5978
6176
 
@@ -5982,7 +6180,7 @@ To receive one, or several packages at once, use command `faspex5 packages recei
5982
6180
  Provide either a single package id, or an extended value `Array` of package ids, e.g. `@list:,1,2,3` as argument.
5983
6181
 
5984
6182
  The same options as for `faspex5 packages list` can be used to select the box and filter the packages to download.
5985
- I.e. options `box` and `query`, as well as last positional parameter `Proc` (filter).
6183
+ I.e. options `box` and `query`, as well as last **Command Parameter** `Proc` (filter).
5986
6184
 
5987
6185
  Option `--once-only=yes` can be used, for "cargo-like" behavior.
5988
6186
  Special package id `INIT` initializes the persistency of already received packages when option `--once-only=yes` is used.
@@ -6004,6 +6202,7 @@ ascli faspex5 admin res workgroup list
6004
6202
  ```
6005
6203
 
6006
6204
  If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
6205
+ Example:
6007
6206
 
6008
6207
  ```bash
6009
6208
  ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
@@ -6021,10 +6220,10 @@ It is equivalent to:
6021
6220
  ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
6022
6221
  ```
6023
6222
 
6024
- Other payload parameters are possible the `Hash` value:
6223
+ Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
6025
6224
 
6026
6225
  ```json
6027
- {"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
6226
+ {"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3}
6028
6227
  ```
6029
6228
 
6030
6229
  ### Faspex 5: Create Metadata profile
@@ -6085,6 +6284,37 @@ Public invitations are for external users, provide just the email address.
6085
6284
 
6086
6285
  Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
6087
6286
 
6287
+ ### Faspex 5: Cleanup packages
6288
+
6289
+ > **Note:** Operation requires admin level.
6290
+
6291
+ Automated cleanup period can be displayed with:
6292
+
6293
+ ```bash
6294
+ ascli faspex5 admin configuration show --fields=days_before_deleting_package_records
6295
+ ```
6296
+
6297
+ This parameter can also be modified, for example:
6298
+
6299
+ ```bash
6300
+ ascli faspex5 admin configuration modify @json:'{"days_before_deleting_package_records":30}'
6301
+ ```
6302
+
6303
+ To start package purge, i.e. permanently remove packages marked for deletion older than `days_before_deleting_package_records`, use command:
6304
+
6305
+ ```bash
6306
+ ascli faspex5 admin clean_deleted
6307
+ ```
6308
+
6309
+ To delete all packages, one can use the following command:
6310
+
6311
+ ```bash
6312
+ ascli faspex5 packages list --box=ALL --format=yaml --fields=id | ascli faspex5 packages delete @yaml:@stdin:
6313
+ ```
6314
+
6315
+ > **Note:** Above command will mark all packages for deletion, and will be permanently removed after the configured period (`clean_deleted` command).
6316
+ > It is possible to add a filter to the list command to only delete packages matching some criteria, e.g. using `--select=@ruby:`.
6317
+
6088
6318
  ### Faspex 5: Faspex 4-style postprocessing
6089
6319
 
6090
6320
  `ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
@@ -6132,6 +6362,14 @@ Then, the postprocessing script executed will be `script1.sh`.
6132
6362
 
6133
6363
  Environment variables at set to the values provided by the web hook which are the same as Faspex 4 postprocessing.
6134
6364
 
6365
+ ### Faspex 5: Missing commands
6366
+
6367
+ If a command is missing, then it is still possible to execute command by calling directly the API on the command line using `curl`:
6368
+
6369
+ ```bash
6370
+ curl -H "Authorization: $(ascli ascli bearer)" https://faspex5.example.com/aspera/faspex/api/v5/api_endpoint_here
6371
+ ```
6372
+
6135
6373
  ## Plugin: `faspex`: IBM Aspera Faspex v4
6136
6374
 
6137
6375
  > **Note:** For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
@@ -6210,13 +6448,13 @@ The contents of `delivery_info` is directly the contents of the `send` v3 [API o
6210
6448
  Example:
6211
6449
 
6212
6450
  ```bash
6213
- ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
6451
+ ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' /tmp/file1 /home/bar/file2
6214
6452
  ```
6215
6453
 
6216
6454
  If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
6217
6455
  `"recipients":["*MyDropboxName"]`
6218
6456
 
6219
- Additional optional parameters in `delivery_info`:
6457
+ Additional optional parameters in mandatory option `delivery_info`:
6220
6458
 
6221
6459
  - Package Note: : `"note":"note this and that"`
6222
6460
  - Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
@@ -6233,7 +6471,7 @@ The value is a `Hash` with the following keys:
6233
6471
 
6234
6472
  ### Email notification on transfer
6235
6473
 
6236
- Like for any transfer, a notification can be sent by email using parameters: `notify_to` and `notify_template` .
6474
+ Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template` .
6237
6475
 
6238
6476
  Example:
6239
6477
 
@@ -6368,12 +6606,14 @@ admin user ldap add the_name
6368
6606
  admin user local list
6369
6607
  admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
6370
6608
  files browse /
6609
+ files delete my_share1/new_folder
6371
6610
  files delete my_share1/test_file.bin
6372
6611
  files download --to-folder=. my_share1/test_file.bin
6373
- files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
6374
- files upload --to-folder=my_share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
6612
+ files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
6613
+ files mkdir my_share1/new_folder
6614
+ files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
6375
6615
  files upload --to-folder=my_share1 test_file.bin
6376
- files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
6616
+ files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
6377
6617
  health
6378
6618
  ```
6379
6619
 
@@ -6527,11 +6767,46 @@ node info --log-level=trace2
6527
6767
  node upload test_file.bin
6528
6768
  ```
6529
6769
 
6770
+ ## Plugin: `httpgw`: HTTP Gateway
6771
+
6772
+ ### Httpgw sample commands
6773
+
6774
+ > **Note:** Add `ascli httpgw` in front of the commands:
6775
+
6776
+ ```bash
6777
+ health
6778
+ ```
6779
+
6780
+ ## Plugin: `faspio`: Faspio Gateway
6781
+
6782
+ ### Faspio sample commands
6783
+
6784
+ > **Note:** Add `ascli faspio` in front of the commands:
6785
+
6786
+ ```bash
6787
+ bridges create @json:'{"name":"test1","local":{"protocol":"tcp","tls_enabled":false,"port":"3000","bind_address":"127.0.0.1"},"forward":{"protocol":"fasp","tls_enabled":false,"port":"3994","bind_address":"127.0.0.1","host":["10.0.0.1"]}}'
6788
+ bridges delete --bulk=yes @json:@stdin:
6789
+ bridges list
6790
+ health
6791
+ ```
6792
+
6793
+ ## Plugin: `alee`: Aspera License Entitlement Engine
6794
+
6795
+ Retrieve information on subscription.
6796
+
6797
+ ### Alee sample commands
6798
+
6799
+ > **Note:** Add `ascli alee` in front of the commands:
6800
+
6801
+ ```bash
6802
+ health
6803
+ ```
6804
+
6530
6805
  ## Plugin: `preview`: Preview generator for AoC
6531
6806
 
6532
6807
  The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
6533
6808
  It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
6534
- Several parameters can be used to tune several aspects:
6809
+ Several options can be used to tune several aspects:
6535
6810
 
6536
6811
  - Methods for detection of new files needing generation
6537
6812
  - Methods for generation of video preview
@@ -6558,10 +6833,11 @@ asnodeadmin --reload
6558
6833
 
6559
6834
  If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
6560
6835
 
6561
- The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 bytes).
6562
- This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KBytes.
6836
+ The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 Bytes).
6837
+ This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KB, i.e. 16MB.
6563
6838
 
6564
- To change this parameter in `aspera.conf`, use `asconfigurator`. To display the value, use `asuserdata`:
6839
+ To change this parameter in `aspera.conf`, use `asconfigurator`.
6840
+ To display the value, use `asuserdata`:
6565
6841
 
6566
6842
  ```bash
6567
6843
  asuserdata -a | grep max_request_file_create_size_kb
@@ -6642,7 +6918,7 @@ chmod a+x /usr/bin/unoconv
6642
6918
  The preview generator should be executed as a non-user.
6643
6919
  When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
6644
6920
 
6645
- Like any `ascli` commands, parameters can be passed on command line or using a configuration [Option Preset](#option-preset)'.
6921
+ Like any `ascli` commands, options can be passed on command line or using a configuration [Option Preset](#option-preset).
6646
6922
  The configuration file must be created with the same user used to run so that it is properly used on runtime.
6647
6923
 
6648
6924
  The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
@@ -6902,9 +7178,9 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
6902
7178
  - `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
6903
7179
  - `--src-base` (`src_base`) if top level folder name shall not be created on destination
6904
7180
 
6905
- > **Note:** `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transfer-specification), with `ts` option.
7181
+ > **Note:** `ascli` takes transfer parameters exclusively as a [**transfer-spec**](#transfer-specification), with `ts` option.
6906
7182
  >
6907
- > **Note:** Most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transfer-specification) parameters.
7183
+ > **Note:** Most, but not all, native `ascp` arguments are available as standard [**transfer-spec**](#transfer-specification) parameters.
6908
7184
  >
6909
7185
  > **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
6910
7186
 
@@ -6914,11 +7190,11 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
6914
7190
 
6915
7191
  > **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
6916
7192
  >
6917
- > **Note:** Parameters may be saved in a [Option Preset](#option-preset)' and used with `-P`.
7193
+ > **Note:** Options may be saved in a [Option Preset](#option-preset) and used with `-P`.
6918
7194
 
6919
7195
  #### Scheduling
6920
7196
 
6921
- Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
7197
+ Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
6922
7198
  Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
6923
7199
 
6924
7200
  ### Example: Upload hot folder
@@ -7081,7 +7357,7 @@ Transfer is: <%=global_transfer_status%>
7081
7357
 
7082
7358
  This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
7083
7359
 
7084
- It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transfer-specification) is:
7360
+ It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [**transfer-spec**](#transfer-specification) is:
7085
7361
 
7086
7362
  - Common to Aspera Node API (HTTP POST /ops/transfer)
7087
7363
  - Common to Aspera Connect API (browser javascript startTransfer)
@@ -7091,7 +7367,7 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
7091
7367
 
7092
7368
  This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
7093
7369
 
7094
- `ascli` expect one single argument: a session specification that contains parameters and a [*transfer-spec*](#transfer-specification).
7370
+ `ascli` expect one single argument: a session specification that contains parameters and a [**transfer-spec**](#transfer-specification).
7095
7371
 
7096
7372
  If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted on stdin.
7097
7373
 
@@ -7103,7 +7379,7 @@ Top level parameters supported by `asession`:
7103
7379
 
7104
7380
  | parameter | description |
7105
7381
  |-----------|-------------|
7106
- | `spec` | the [*transfer-spec*](#transfer-specification) |
7382
+ | `spec` | the [**transfer-spec**](#transfer-specification) |
7107
7383
  | `agent` | same parameters as transfer-info for agent `direct` |
7108
7384
  | `loglevel` | log level of `asession` |
7109
7385
  | `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
@@ -7136,7 +7412,7 @@ asession < session.json
7136
7412
 
7137
7413
  `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` -&gt; `long_parameter`
7138
7414
 
7139
- This is particularly useful for a persistent session ( with the [*transfer-spec*](#transfer-specification) parameter: `"keepalive":true` )
7415
+ This is particularly useful for a persistent session ( with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true` )
7140
7416
 
7141
7417
  ```json
7142
7418
  asession
@@ -7205,7 +7481,7 @@ So, it evolved into `ascli`:
7205
7481
 
7206
7482
  - Portable: works on platforms supporting `ruby` (and `ascp`)
7207
7483
  - Easy to install with the `gem` utility
7208
- - Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that&apos;s why transfer parameters moved from `ascp` command line to [*transfer-spec*](#transfer-specification) (more reliable , more standard)
7484
+ - Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that&apos;s why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable , more standard)
7209
7485
  - `ruby` is consistent with other Aspera products
7210
7486
 
7211
7487
  Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
@@ -7215,7 +7491,7 @@ Enjoy a coffee on me:
7215
7491
 
7216
7492
  ```bash
7217
7493
  ascli config coffee --ui=text
7218
- ascli config coffee --ui=text --query=@json:'{"text":"true"}'
7494
+ ascli config coffee --ui=text --image=@json:'{"text":true}'
7219
7495
  ascli config coffee
7220
7496
  ```
7221
7497
 
@@ -7244,11 +7520,13 @@ This also requires Ruby header files.
7244
7520
  If Ruby was installed as a Linux Packages, then also install Ruby development package:
7245
7521
  `ruby-dev` or `ruby-devel`, depending on distribution.
7246
7522
 
7247
- ### ED255519 key not supported
7523
+ ### ED25519 key not supported
7248
7524
 
7249
7525
  ED25519 keys are deactivated since `ascli` version 0.9.24 as it requires additional gems that require native compilation and thus caused problems.
7250
7526
  This type of key will just be ignored.
7251
7527
 
7528
+ To re-activate, set env var `ASCLI_ENABLE_ED25519` to `true`.
7529
+
7252
7530
  Without this deactivation, if such key was present in user's `.ssh` folder then the following error was generated:
7253
7531
 
7254
7532
  ```output
@@ -7258,4 +7536,5 @@ OpenSSH keys only supported if ED25519 is available
7258
7536
  Which meant that you do not have Ruby support for ED25519 SSH keys.
7259
7537
  You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
7260
7538
 
7261
- To re-activate, set env var `ASCLI_ENABLE_ED25519` to `true`.
7539
+ In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also de-activated by default.
7540
+ To re-activate, set env var `ASCLI_ENABLE_ECDSHA2` to `true`.