aspera-cli 4.17.0 → 4.18.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +2 -4
- data/CHANGELOG.md +23 -0
- data/CONTRIBUTING.md +15 -1
- data/README.md +620 -378
- data/bin/ascli +5 -0
- data/bin/asession +2 -2
- data/lib/aspera/agent/alpha.rb +6 -4
- data/lib/aspera/agent/base.rb +9 -6
- data/lib/aspera/agent/connect.rb +4 -4
- data/lib/aspera/agent/direct.rb +56 -37
- data/lib/aspera/agent/httpgw.rb +23 -324
- data/lib/aspera/agent/node.rb +19 -20
- data/lib/aspera/agent/trsdk.rb +19 -20
- data/lib/aspera/api/aoc.rb +17 -14
- data/lib/aspera/api/cos_node.rb +4 -4
- data/lib/aspera/api/httpgw.rb +339 -0
- data/lib/aspera/api/node.rb +34 -21
- data/lib/aspera/ascmd.rb +4 -3
- data/lib/aspera/ascp/installation.rb +15 -7
- data/lib/aspera/ascp/management.rb +2 -2
- data/lib/aspera/cli/basic_auth_plugin.rb +5 -9
- data/lib/aspera/cli/extended_value.rb +12 -6
- data/lib/aspera/cli/formatter.rb +155 -65
- data/lib/aspera/cli/hints.rb +18 -0
- data/lib/aspera/cli/main.rb +22 -29
- data/lib/aspera/cli/manager.rb +53 -36
- data/lib/aspera/cli/plugin.rb +26 -17
- data/lib/aspera/cli/plugin_factory.rb +31 -20
- data/lib/aspera/cli/plugins/alee.rb +14 -2
- data/lib/aspera/cli/plugins/aoc.rb +141 -131
- data/lib/aspera/cli/plugins/ats.rb +1 -1
- data/lib/aspera/cli/plugins/config.rb +52 -46
- data/lib/aspera/cli/plugins/console.rb +8 -5
- data/lib/aspera/cli/plugins/faspex.rb +27 -19
- data/lib/aspera/cli/plugins/faspex5.rb +222 -149
- data/lib/aspera/cli/plugins/faspio.rb +85 -0
- data/lib/aspera/cli/plugins/httpgw.rb +55 -0
- data/lib/aspera/cli/plugins/node.rb +86 -29
- data/lib/aspera/cli/plugins/orchestrator.rb +31 -29
- data/lib/aspera/cli/plugins/preview.rb +6 -2
- data/lib/aspera/cli/plugins/server.rb +5 -5
- data/lib/aspera/cli/plugins/shares.rb +16 -14
- data/lib/aspera/cli/sync_actions.rb +6 -6
- data/lib/aspera/cli/transfer_agent.rb +5 -4
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/environment.rb +7 -6
- data/lib/aspera/faspex_gw.rb +5 -4
- data/lib/aspera/faspex_postproc.rb +2 -2
- data/lib/aspera/log.rb +6 -3
- data/lib/aspera/node_simulator.rb +2 -2
- data/lib/aspera/oauth/base.rb +31 -19
- data/lib/aspera/oauth/factory.rb +12 -13
- data/lib/aspera/oauth/generic.rb +1 -0
- data/lib/aspera/oauth/jwt.rb +18 -15
- data/lib/aspera/oauth/url_json.rb +8 -6
- data/lib/aspera/open_application.rb +5 -7
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/generator.rb +3 -3
- data/lib/aspera/preview/options.rb +3 -3
- data/lib/aspera/preview/terminal.rb +4 -4
- data/lib/aspera/preview/utils.rb +3 -3
- data/lib/aspera/proxy_auto_config.rb +5 -1
- data/lib/aspera/rest.rb +60 -74
- data/lib/aspera/rest_call_error.rb +1 -1
- data/lib/aspera/rest_error_analyzer.rb +2 -2
- data/lib/aspera/rest_errors_aspera.rb +1 -1
- data/lib/aspera/resumer.rb +1 -1
- data/lib/aspera/secret_hider.rb +2 -4
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/transfer/parameters.rb +39 -36
- data/lib/aspera/transfer/spec.rb +2 -0
- data/lib/aspera/transfer/sync.rb +2 -1
- data/lib/aspera/transfer/uri.rb +1 -1
- data/lib/aspera/uri_reader.rb +5 -4
- data/lib/aspera/web_auth.rb +1 -1
- data/lib/aspera/web_server_simple.rb +4 -3
- data.tar.gz.sig +0 -0
- metadata +5 -3
- metadata.gz.sig +0 -0
- data/lib/aspera/cli/plugins/bss.rb +0 -71
data/README.md
CHANGED
@@ -10,7 +10,7 @@
|
|
10
10
|
|
11
11
|
## Introduction
|
12
12
|
|
13
|
-
Version : 4.
|
13
|
+
Version : 4.18.0
|
14
14
|
|
15
15
|
Laurent/2016-2024
|
16
16
|
|
@@ -70,7 +70,7 @@ For scripting and ad'hoc command line operations, `ascli` is perfect.
|
|
70
70
|
|
71
71
|
Command line operations examples are shown using a shell such as: `bash` or `zsh`.
|
72
72
|
|
73
|
-
Command line
|
73
|
+
Command line arguments beginning with `my_` in examples, e.g. `my_param_value`, are user-provided value and not fixed value commands.
|
74
74
|
|
75
75
|
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
|
76
76
|
|
@@ -92,7 +92,7 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
92
92
|
|
93
93
|
```console
|
94
94
|
$ ascli --version
|
95
|
-
4.
|
95
|
+
4.18.0
|
96
96
|
```
|
97
97
|
|
98
98
|
### First use
|
@@ -122,10 +122,10 @@ ascli server browse /
|
|
122
122
|
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
123
123
|
```
|
124
124
|
|
125
|
-
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset)
|
125
|
+
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset) for the server's authentication options.
|
126
126
|
The following example will:
|
127
127
|
|
128
|
-
- Create a [Option Preset](#option-preset)
|
128
|
+
- Create a [Option Preset](#option-preset)
|
129
129
|
- Define it as default for the `server` plugin
|
130
130
|
- List files in a folder
|
131
131
|
- Download a file
|
@@ -188,8 +188,8 @@ It is possible to install **either** directly on the host operating system (Linu
|
|
188
188
|
The direct installation is recommended and consists in installing:
|
189
189
|
|
190
190
|
- [Ruby](#ruby)
|
191
|
-
- [aspera-cli](#ruby-gem)
|
192
|
-
- [Aspera SDK (`ascp`)](#fasp-protocol)
|
191
|
+
- [aspera-cli](#ruby-gem-aspera-cli) <!-- markdownlint-disable-line -->
|
192
|
+
- [Aspera SDK (`ascp`)](#fasp-protocol-ascp)
|
193
193
|
|
194
194
|
Ruby version: >= 2.6.
|
195
195
|
|
@@ -253,7 +253,7 @@ Install the chosen pre-compiled Ruby version:
|
|
253
253
|
rvm install 3.2.2
|
254
254
|
```
|
255
255
|
|
256
|
-
Ruby is now installed for the user, go to [Gem installation](#ruby-gem).
|
256
|
+
Ruby is now installed for the user, go to [Gem installation](#ruby-gem-aspera-cli). <!-- markdownlint-disable-line -->
|
257
257
|
|
258
258
|
Alternatively RVM can be installed system-wide, for this execute as `root`.
|
259
259
|
It then installs by default in `/usr/local/rvm` for all users and creates `/etc/profile.d/rvm.sh`.
|
@@ -388,6 +388,12 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
|
|
388
388
|
dnf install -y ruby-devel
|
389
389
|
```
|
390
390
|
|
391
|
+
**Example:** Ubuntu
|
392
|
+
|
393
|
+
```bash
|
394
|
+
apt-get install -y ruby-full
|
395
|
+
```
|
396
|
+
|
391
397
|
**Other examples:**
|
392
398
|
|
393
399
|
```bash
|
@@ -479,7 +485,7 @@ gem install rmagick grpc mimemagic
|
|
479
485
|
|
480
486
|
> **Note:** Those are not installed as part of dependencies because they involve compilation of native code.
|
481
487
|
|
482
|
-
### Ruby Gem
|
488
|
+
### Ruby Gem: `aspera-cli`
|
483
489
|
|
484
490
|
Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
|
485
491
|
|
@@ -502,7 +508,26 @@ To check if a new version is available (independently of `version_check_days`):
|
|
502
508
|
ascli config check_update
|
503
509
|
```
|
504
510
|
|
505
|
-
|
511
|
+
#### Gem installation with signature verification
|
512
|
+
|
513
|
+
The gem is signed with a private key, and the public certificate is available in the github repository (`certs/aspera-cli-public-cert.pem`).
|
514
|
+
When installing the gem, the signature can be optionally verified.
|
515
|
+
|
516
|
+
For [secure installation](https://guides.rubygems.org/security/#using-gems), one can install the gem with the public key:
|
517
|
+
|
518
|
+
- import the verification certificate:
|
519
|
+
|
520
|
+
```bash
|
521
|
+
gem cert --add <(curl -Ls https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem)
|
522
|
+
```
|
523
|
+
|
524
|
+
- Then, install the gem:
|
525
|
+
|
526
|
+
```bash
|
527
|
+
gem install -P MediumSecurity aspera-cli
|
528
|
+
```
|
529
|
+
|
530
|
+
### FASP Protocol: `ascp`
|
506
531
|
|
507
532
|
Most file transfers will be executed using the **FASP** protocol, using `ascp`.
|
508
533
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
@@ -601,7 +626,7 @@ gem install aspera-cli -N -i my_gems
|
|
601
626
|
|
602
627
|
### Container
|
603
628
|
|
604
|
-
The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
|
629
|
+
The container image is: [`docker.io/martinlaurent/ascli`](https://hub.docker.com/r/docker.io/martinlaurent/ascli).
|
605
630
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
606
631
|
To use the container, ensure that you have `podman` (or `docker`) installed.
|
607
632
|
|
@@ -615,7 +640,7 @@ podman --version
|
|
615
640
|
Execute this:
|
616
641
|
|
617
642
|
```bash
|
618
|
-
podman run --rm --tty --interactive --entrypoint bash martinlaurent/ascli:latest
|
643
|
+
podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/ascli:latest
|
619
644
|
```
|
620
645
|
|
621
646
|
> **Note:** This command changes the entrypoint to an interactive shell instead of direct execution of `ascli`.
|
@@ -643,13 +668,13 @@ The entry point is `ascli` and the default command is `help`.
|
|
643
668
|
The container can be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
644
669
|
|
645
670
|
```bash
|
646
|
-
podman run --rm --tty --interactive martinlaurent/ascli:latest
|
671
|
+
podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest
|
647
672
|
```
|
648
673
|
|
649
674
|
For more convenience, you may define a shell alias:
|
650
675
|
|
651
676
|
```bash
|
652
|
-
alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
|
677
|
+
alias ascli='podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest'
|
653
678
|
```
|
654
679
|
|
655
680
|
Then, you can execute the container like a local command:
|
@@ -659,7 +684,7 @@ ascli -v
|
|
659
684
|
```
|
660
685
|
|
661
686
|
```text
|
662
|
-
4.
|
687
|
+
4.18.0
|
663
688
|
```
|
664
689
|
|
665
690
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
@@ -686,12 +711,12 @@ In this case you need also to specify the shared transfer folder as a volume:
|
|
686
711
|
--volume $HOME/xferdir:/xferfiles
|
687
712
|
```
|
688
713
|
|
689
|
-
> **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
714
|
+
> **Note:** `ascli` is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
690
715
|
|
691
716
|
And if you want all the above, simply use all the options:
|
692
717
|
|
693
718
|
```bash
|
694
|
-
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
|
719
|
+
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash docker.io/martinlaurent/ascli:latest"
|
695
720
|
```
|
696
721
|
|
697
722
|
```bash
|
@@ -714,7 +739,7 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
714
739
|
|----------------|------------------------------------|--------------------------|--------------------------|
|
715
740
|
| `ASCLI_HOME` | Configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
716
741
|
| `docker_args` | Additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
717
|
-
| `image` | Container image name | `martinlaurent/ascli` | |
|
742
|
+
| `image` | Container image name | `docker.io/martinlaurent/ascli` | |
|
718
743
|
| `version` | Container image version | Latest | `4.8.0.pre` |
|
719
744
|
|
720
745
|
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
@@ -748,8 +773,8 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
748
773
|
- First create the image archive:
|
749
774
|
|
750
775
|
```bash
|
751
|
-
podman pull martinlaurent/ascli
|
752
|
-
podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
776
|
+
podman pull docker.io/martinlaurent/ascli
|
777
|
+
podman save docker.io/martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
753
778
|
```
|
754
779
|
|
755
780
|
- Then, on air-gapped system:
|
@@ -787,7 +812,7 @@ dnf install singularity-ce
|
|
787
812
|
Build an image like this:
|
788
813
|
|
789
814
|
```bash
|
790
|
-
singularity build ascli.sif docker://martinlaurent/ascli
|
815
|
+
singularity build ascli.sif docker://docker.io/martinlaurent/ascli
|
791
816
|
```
|
792
817
|
|
793
818
|
Then, start `ascli` like this:
|
@@ -802,6 +827,124 @@ Or get a shell with access to `ascli` like this:
|
|
802
827
|
singularity shell ascli.sif
|
803
828
|
```
|
804
829
|
|
830
|
+
### SSL library
|
831
|
+
|
832
|
+
`ascli` uses the Ruby `openssl` gem which uses by default the system's `openssl` library and its CA certificate bundle.
|
833
|
+
|
834
|
+
To display the version of **openssl** used in `ascli`:
|
835
|
+
|
836
|
+
```bash
|
837
|
+
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
838
|
+
```
|
839
|
+
|
840
|
+
It is possible to specify to use another SSL library or version by executing:
|
841
|
+
|
842
|
+
```bash
|
843
|
+
gem install openssl -- --with-openssl-dir=[openssl library folder]
|
844
|
+
```
|
845
|
+
|
846
|
+
Where `[openssl library folder]` is the path to the folder containing the `lib` and `include` folders of the `openssl` library.
|
847
|
+
|
848
|
+
For example, on macOS, to use the `openssl@3` library installed with `brew`:
|
849
|
+
|
850
|
+
```bash
|
851
|
+
$ openssl version -e|sed -n 's|ENGINESDIR: "\(.*\)/lib[^/]*/.*|\1|p'
|
852
|
+
/opt/homebrew/Cellar/openssl@3/3.3.0
|
853
|
+
$ gem install openssl -- --with-openssl-dir=/opt/homebrew/Cellar/openssl@3/3.3.0
|
854
|
+
```
|
855
|
+
|
856
|
+
### SSL CA certificate bundle
|
857
|
+
|
858
|
+
SSL certificates are validated using a certificate store, by default it is the one of the system's `openssl` library.
|
859
|
+
|
860
|
+
To display trusted certificate store locations:
|
861
|
+
|
862
|
+
```bash
|
863
|
+
ascli --show-config --fields=cert_stores
|
864
|
+
```
|
865
|
+
|
866
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
867
|
+
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
868
|
+
|
869
|
+
One can display those default values:
|
870
|
+
|
871
|
+
```bash
|
872
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
873
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
874
|
+
```
|
875
|
+
|
876
|
+
In order to get certificate validation, the CA certificate bundle must be up-to-date.
|
877
|
+
Check this repository on how to update the system's CA certificate bundle: [https://github.com/millermatt/osca](https://github.com/millermatt/osca).
|
878
|
+
|
879
|
+
For example on RHEL/Rocky Linux:
|
880
|
+
|
881
|
+
```bash
|
882
|
+
dnf install -y ca-certificates
|
883
|
+
update-ca-trust extract
|
884
|
+
```
|
885
|
+
|
886
|
+
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders.
|
887
|
+
By default it uses Ruby's default certificate store.
|
888
|
+
|
889
|
+
If you use this option, then default locations are not used.
|
890
|
+
Default locations can be added using special value `DEF`.
|
891
|
+
The value can be either an `Array` or `String` (path).
|
892
|
+
Successive options add paths incrementally.
|
893
|
+
All files of a folders are added.
|
894
|
+
|
895
|
+
JRuby uses its own implementation and CA bundles.
|
896
|
+
|
897
|
+
For example, on Linux to force the use the system's certificate store:
|
898
|
+
|
899
|
+
```bash
|
900
|
+
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
901
|
+
```
|
902
|
+
|
903
|
+
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
|
904
|
+
|
905
|
+
By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
|
906
|
+
Original `ascp`'s hardcoded locations can be found using:
|
907
|
+
|
908
|
+
```bash
|
909
|
+
ascli config ascp info --fields=openssldir
|
910
|
+
```
|
911
|
+
|
912
|
+
E.g. on macOS: `/Library/Aspera/ssl`.
|
913
|
+
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
914
|
+
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
915
|
+
|
916
|
+
To update trusted root certificates for `ascli`:
|
917
|
+
Display the trusted certificate store locations used by `ascli`.
|
918
|
+
Typically done by updating the system's root certificate store.
|
919
|
+
|
920
|
+
An up-to-date version of the certificate bundle can also be retrieved with:
|
921
|
+
|
922
|
+
```bash
|
923
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
924
|
+
```
|
925
|
+
|
926
|
+
To download that certificate store:
|
927
|
+
|
928
|
+
```bash
|
929
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
|
930
|
+
```
|
931
|
+
|
932
|
+
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
933
|
+
|
934
|
+
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
935
|
+
|
936
|
+
```bash
|
937
|
+
ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
|
938
|
+
```
|
939
|
+
|
940
|
+
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
941
|
+
|
942
|
+
Then, use this file as certificate store (e.g. here, Node API):
|
943
|
+
|
944
|
+
```bash
|
945
|
+
ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
|
946
|
+
```
|
947
|
+
|
805
948
|
## Command Line Interface
|
806
949
|
|
807
950
|
The command line tool is: ``ascli``
|
@@ -820,9 +963,9 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
|
|
820
963
|
|
821
964
|
- Supports commands to Aspera server products (on-premise and SaaS)
|
822
965
|
- Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
|
823
|
-
- Supports Commands, Option values
|
966
|
+
- Supports Commands, Options, and Option values shortcuts
|
824
967
|
- FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
|
825
|
-
- Transfer parameters can be altered by modification of [
|
968
|
+
- Transfer parameters can be altered by modification of [**transfer-spec**](#transfer-specification), this includes requiring multi-session
|
826
969
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
827
970
|
- Supports FaspStream creation (using Node API)
|
828
971
|
- Supports **Watchfolder** creation (using Node API)
|
@@ -856,19 +999,20 @@ Moreover all `ascp` options are supported either through transfer spec parameter
|
|
856
999
|
### Command line parsing, Special Characters
|
857
1000
|
|
858
1001
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
859
|
-
`ascli` receives its arguments
|
1002
|
+
`ascli` receives its arguments on the command line.
|
1003
|
+
The way arguments are parsed and provided to `ascli` depend on the Operating System and shell.
|
860
1004
|
|
861
1005
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
862
1006
|
|
863
|
-
Linux command line parsing is
|
1007
|
+
Linux command line parsing is well defined:
|
864
1008
|
It is fully documented in the shell's documentation.
|
865
1009
|
|
866
|
-
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
1010
|
+
On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh).
|
867
1011
|
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
868
1012
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
869
1013
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
870
1014
|
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
871
|
-
Ruby receives a list
|
1015
|
+
Ruby receives a list command line arguments from shell and gives it to `ascli`.
|
872
1016
|
Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
|
873
1017
|
|
874
1018
|
#### Shell parsing for Windows
|
@@ -960,16 +1104,16 @@ Major Minor Build Revision
|
|
960
1104
|
----- ----- ----- --------
|
961
1105
|
5 1 19041 4046
|
962
1106
|
|
963
|
-
PS C:\> ascli
|
1107
|
+
PS C:\> ascli config echo --% @json:'{"k":"v","x":"y"}'
|
964
1108
|
|
965
|
-
PS C:\> ascli
|
1109
|
+
PS C:\> ascli config echo @json:'{"""k""":"""v""","""x""":"""y"""}'
|
966
1110
|
```
|
967
1111
|
|
968
1112
|
> **Note:** The special powershell argument `--%` places powershell in legacy parsing mode.
|
969
1113
|
|
970
1114
|
#### Extended Values (JSON, Ruby, ...)
|
971
1115
|
|
972
|
-
Some of the `ascli`
|
1116
|
+
Some of the values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
973
1117
|
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
974
1118
|
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
975
1119
|
|
@@ -1154,16 +1298,15 @@ ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
1154
1298
|
{"title":"Test \" ' & \\"}
|
1155
1299
|
```
|
1156
1300
|
|
1157
|
-
###
|
1301
|
+
### Positional Arguments and Options
|
1158
1302
|
|
1159
1303
|
Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
|
1160
1304
|
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#command-line-parsing-special-characters).
|
1161
1305
|
|
1162
|
-
`ascli` handles
|
1306
|
+
`ascli` handles two types of command line arguments:
|
1163
1307
|
|
1164
|
-
-
|
1165
|
-
-
|
1166
|
-
- Options
|
1308
|
+
- **Positional Arguments** : position is significant
|
1309
|
+
- **Options** : only order is significant, but not position
|
1167
1310
|
|
1168
1311
|
For example:
|
1169
1312
|
|
@@ -1171,18 +1314,24 @@ For example:
|
|
1171
1314
|
ascli command subcommand --option-name=VAL1 VAL2
|
1172
1315
|
```
|
1173
1316
|
|
1174
|
-
- Executes **
|
1175
|
-
- With one **
|
1176
|
-
- The command has one additional **positional argument**: `VAL2`
|
1317
|
+
- Executes **Command** and its **Command Parameters** (**Positional Arguments**): `command subcommand VAL2`
|
1318
|
+
- With one **Option**: `option_name` and its **value**: `VAL1`
|
1177
1319
|
|
1178
1320
|
If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
|
1179
1321
|
For example `ascli config pre ov` is the same as `ascli config preset overview`.
|
1180
1322
|
|
1181
|
-
The value of
|
1323
|
+
The value of **Options** and **Positional Arguments** is evaluated with the [Extended Value Syntax](#extended-value-syntax).
|
1182
1324
|
|
1183
|
-
####
|
1325
|
+
#### Positional Arguments
|
1184
1326
|
|
1185
|
-
|
1327
|
+
**Positional Arguments** are either:
|
1328
|
+
|
1329
|
+
- **Commands**, typically at the beginning
|
1330
|
+
- **Command Parameters** , e.g. creation data or entity identifier
|
1331
|
+
|
1332
|
+
When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
|
1333
|
+
|
1334
|
+
**Commands** are typically entity types or verbs to act on those entities.
|
1186
1335
|
Its value is a `String` that must belong to a fixed list of values in a given context.
|
1187
1336
|
|
1188
1337
|
Example:
|
@@ -1196,27 +1345,24 @@ ascli config ascp info
|
|
1196
1345
|
- `ascp` is the second level command: name of the component (singleton)
|
1197
1346
|
- `info` is the third level command: action to be performed
|
1198
1347
|
|
1199
|
-
Typically,
|
1348
|
+
Typically, **Commands** are located at the **beginning** of the command line.
|
1200
1349
|
Order is significant.
|
1201
1350
|
The provided command must match one of the supported commands in the given context.
|
1202
1351
|
If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
1203
1352
|
|
1204
|
-
Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
1205
|
-
|
1206
|
-
#### Positional Arguments
|
1207
|
-
|
1208
|
-
Positional Arguments are typically mandatory values for a command, such as entity creation data.
|
1209
|
-
|
1210
|
-
It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
|
1353
|
+
Some sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
1211
1354
|
|
1212
|
-
|
1355
|
+
**Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
|
1213
1356
|
|
1357
|
+
> **Note:** It could also have been designed as an option.
|
1358
|
+
But since it is mandatory and typically these data need **not** be set in a configuration file, then it is better designed as a **Command Parameter**, and not designed an additional specific option.
|
1359
|
+
The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position) and the value is clearly mandatory.
|
1214
1360
|
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
|
1215
|
-
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@
|
1361
|
+
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
|
1216
1362
|
|
1217
|
-
If a
|
1363
|
+
If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
1218
1364
|
|
1219
|
-
A few
|
1365
|
+
A few **Command Parameters** are optional, they are located at the end of the command line.
|
1220
1366
|
|
1221
1367
|
#### Options
|
1222
1368
|
|
@@ -1225,13 +1371,15 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
|
|
1225
1371
|
- Start with `--`
|
1226
1372
|
- Have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
|
1227
1373
|
- Have a value, separated from name with a `=`
|
1228
|
-
- Can be used by prefix (
|
1374
|
+
- Can be used by prefix (not recommended), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
|
1375
|
+
- Is optional on command line (it has a default value or no value)
|
1376
|
+
- whose position is not significant, but order is significant
|
1229
1377
|
|
1230
1378
|
Exceptions:
|
1231
1379
|
|
1232
1380
|
- Some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
|
1233
1381
|
- Some options (flags) don't take a value, e.g. `-N`
|
1234
|
-
- The special option `--` stops option processing and is ignored, following command line arguments are taken as
|
1382
|
+
- The special option `--` stops option processing and is ignored, following command line arguments are taken as **Positional Arguments**, including the ones starting with a `-`.
|
1235
1383
|
|
1236
1384
|
Example:
|
1237
1385
|
|
@@ -1248,6 +1396,7 @@ ascli config echo -- --sample
|
|
1248
1396
|
Options may have a (hardcoded) default value.
|
1249
1397
|
|
1250
1398
|
Options can be placed anywhere on command line and are evaluated in order.
|
1399
|
+
Usually the last value evaluated overrides previous values, but some options are cumulative, e.g. `ts`.
|
1251
1400
|
|
1252
1401
|
Options are typically optional: to change the default behavior.
|
1253
1402
|
But some are mandatory, so they can be placed in a configuration file, for example: connection information.
|
@@ -1264,23 +1413,23 @@ Option `show_config` dry runs the configuration, and then returns currently set
|
|
1264
1413
|
`ascli --show-config` outputs global options only, and `ascli [plugin] --show-config` outputs global and plugin default options.
|
1265
1414
|
In addition, option `--show-config` can be added at the end of any full command line, this displays the options that would be used for the command.
|
1266
1415
|
|
1267
|
-
A
|
1416
|
+
A command line argument is typically designed as option if:
|
1268
1417
|
|
1269
1418
|
- It is optional, or
|
1270
|
-
- It is a mandatory parameter that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
1419
|
+
- It is a mandatory parameter with a default value that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
1271
1420
|
|
1272
1421
|
### Interactive Input
|
1273
1422
|
|
1274
|
-
Some options and
|
1275
|
-
By default, `ascli` will ask for missing mandatory options or
|
1423
|
+
Some options and **Command Parameters** are mandatory and other optional.
|
1424
|
+
By default, `ascli` will ask for missing mandatory options or **Command Parameters** for interactive execution.
|
1276
1425
|
|
1277
1426
|
The behavior can be controlled with:
|
1278
1427
|
|
1279
1428
|
- `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
|
1280
|
-
- yes : missing mandatory parameters/
|
1281
|
-
- no : missing mandatory parameters/
|
1429
|
+
- yes : missing mandatory parameters/arguments are asked to the user
|
1430
|
+
- no : missing mandatory parameters/arguments raise an error message
|
1282
1431
|
- `--ask-options=<yes|no>` (default=no)
|
1283
|
-
- optional parameters/
|
1432
|
+
- optional parameters/arguments are asked to user
|
1284
1433
|
|
1285
1434
|
### Output
|
1286
1435
|
|
@@ -1303,25 +1452,36 @@ Depending on action, the output will contain:
|
|
1303
1452
|
#### Format of output
|
1304
1453
|
|
1305
1454
|
By default, result of type single_object and object_list are displayed using format `table`.
|
1306
|
-
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
1307
1455
|
|
1308
|
-
|
1309
|
-
|
1310
|
-
|
1311
|
-
When in flatten mode, it is possible to filter fields by compound field name using dot as separator.
|
1456
|
+
The table style can be customized with option: `table_style` which expects a `Hash`, options are the ones described in gem [`terminal-table`](https://github.com/tj/terminal-table).
|
1457
|
+
|
1458
|
+
For example, to display a table with thick unicode borders:
|
1312
1459
|
|
1313
|
-
|
1460
|
+
```bash
|
1461
|
+
ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
|
1462
|
+
```
|
1463
|
+
|
1464
|
+
> **Note:** Other border styles exist, not limited to: `:unicode`, `:unicode_round`.
|
1465
|
+
|
1466
|
+
In a table format, when displaying **Objects** (single, or list), by default, sub object are flattened (option `flat_hash`).
|
1467
|
+
For example, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
|
1468
|
+
Setting option `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
|
1469
|
+
When in flatten mode, it is possible to filter fields using the option `fields` using the compound field name using `.` (dot) as separator.
|
1470
|
+
|
1471
|
+
Object lists are displayed one per line, with attributes as columns.
|
1472
|
+
Single objects (or tables with a single result) are transposed: one attribute per line.
|
1314
1473
|
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
1315
1474
|
|
1316
|
-
The style of output can be set using the `format`
|
1475
|
+
The style of output can be set using the `format` option, supporting:
|
1317
1476
|
|
1477
|
+
- `table` : Text table (default)
|
1318
1478
|
- `text` : Value as String
|
1319
|
-
- `table` : Text table
|
1320
1479
|
- `ruby` : Ruby code
|
1321
1480
|
- `json` : JSON code
|
1322
1481
|
- `jsonpp` : JSON pretty printed
|
1323
1482
|
- `yaml` : YAML
|
1324
1483
|
- `csv` : Comma Separated Values
|
1484
|
+
- `image` : Image URL or Image data
|
1325
1485
|
|
1326
1486
|
#### Verbosity of output
|
1327
1487
|
|
@@ -1342,31 +1502,31 @@ To hide secrets from output, set option `show_secrets` to `no`.
|
|
1342
1502
|
|
1343
1503
|
#### Option: `fields`: Selection of output object properties
|
1344
1504
|
|
1345
|
-
By default, a table output will display one line per entry, and columns for each properties.
|
1346
|
-
Depending on the command, columns may include by default all
|
1505
|
+
By default, a `table` output format will display one line per entry, and columns for each fields (attributes, properties).
|
1506
|
+
Depending on the command, columns may include by default all fields, or only some selected fields.
|
1347
1507
|
It is possible to define specific columns to be displayed, by setting the `fields` option.
|
1348
1508
|
|
1349
|
-
The `fields` option can be either a comma separated list
|
1509
|
+
The `fields` option is a list that can be either a comma separated list or an extended value `Array`.
|
1350
1510
|
|
1351
|
-
|
1511
|
+
Individual elements of the list can be:
|
1352
1512
|
|
1353
|
-
- `DEF` : default display of columns (that's the default, when not set)
|
1354
|
-
- `ALL` : all columns available
|
1355
|
-
- `-`**property** : remove property from the current list
|
1356
1513
|
- **property** : add property to the current list
|
1357
|
-
-
|
1514
|
+
- `-`**property** : remove property from the current list
|
1515
|
+
- `DEF` : default list of properties (that's the default, when not set)
|
1516
|
+
- `ALL` : all properties
|
1517
|
+
- A Ruby `RegEx` : using `@ruby:'/.../'`, or `@re:...` add those matching to the list
|
1358
1518
|
|
1359
1519
|
Examples:
|
1360
1520
|
|
1361
|
-
- `a,b,c` : the list of attributes specified as a comma separated list
|
1362
|
-
- `@json:'["a","b","c"]'` : Array extended value: same as above
|
1363
|
-
- `DEF,-a
|
1521
|
+
- `a,b,c` : the list of attributes specified as a comma separated list (overrides the all default)
|
1522
|
+
- `@json:'["a","b","c"]'` : `Array` extended value: same as above
|
1523
|
+
- `b,DEF,-a` : default property list, remove `a` and add `b` in first position
|
1364
1524
|
- `@ruby:'/^server/'` : Display all properties whose name begin with `server`
|
1365
1525
|
|
1366
1526
|
#### Option: `select`
|
1367
1527
|
|
1368
1528
|
Table output (`object_list`) can be filtered using option `select`.
|
1369
|
-
This
|
1529
|
+
This option is either a `Hash` or `Proc`.
|
1370
1530
|
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
|
1371
1531
|
|
1372
1532
|
Example:
|
@@ -1384,7 +1544,7 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
|
|
1384
1544
|
+-------------------------------+----------------------------------+-----------+
|
1385
1545
|
```
|
1386
1546
|
|
1387
|
-
> **Note:** `select` filters elements from the result of command, while the `query`
|
1547
|
+
> **Note:** Option `select` filters elements from the result of command, while the `query` option gives filtering parameters to the API when listing elements.
|
1388
1548
|
|
1389
1549
|
In above example, the same result is obtained with option:
|
1390
1550
|
|
@@ -1440,7 +1600,8 @@ The following decoders are supported:
|
|
1440
1600
|
| `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`) |
|
1441
1601
|
| `ruby` | String | Any | Execute specified Ruby code |
|
1442
1602
|
| `secret` | None | String | Ask password interactively (hides input) |
|
1443
|
-
| `stdin` | None | String | Read from stdin (no value on right) |
|
1603
|
+
| `stdin` | None | String | Read from stdin in text mode (no value on right) |
|
1604
|
+
| `stdbin` | None | String | Read from stdin in binary mode (no value on right) |
|
1444
1605
|
| `uri` | String | String | Read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
1445
1606
|
| `val` | String | String | Prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
1446
1607
|
| `yaml` | String | Any | Decode YAML |
|
@@ -1575,44 +1736,46 @@ Temporary files are deleted at the end of execution unless option: `clean_temp`
|
|
1575
1736
|
|
1576
1737
|
### Configuration file
|
1577
1738
|
|
1578
|
-
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
1579
|
-
|
1739
|
+
On the first execution of `ascli`, an empty configuration file is created in the configuration folder (`ascli config folder`).
|
1740
|
+
There is no mandatory information required in this file.
|
1741
|
+
The use of it is optional as any option can be provided on the command line.
|
1580
1742
|
|
1581
|
-
Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
|
1743
|
+
Although the file is a standard `YAML` file, `ascli` provides commands to read and modify it using the `config` command.
|
1582
1744
|
|
1583
|
-
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset)
|
1745
|
+
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset) in the configuration file.
|
1584
1746
|
|
1585
|
-
A configuration file provides a way to define default values, especially for authentication
|
1747
|
+
A configuration file provides a way to define default values, especially for authentication options, thus avoiding to always having to specify those options on the command line.
|
1586
1748
|
|
1587
|
-
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or
|
1749
|
+
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or its env var).
|
1588
1750
|
|
1589
|
-
The configuration file is
|
1751
|
+
The configuration file is a catalog of named lists of options, called: [Option Preset](#option-preset).
|
1752
|
+
Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset) (e.g. `mypreset`) using the option `preset`: `--preset=mypreset` or its shortcut: `-Pmypreset`.
|
1590
1753
|
|
1591
1754
|
#### Option Preset
|
1592
1755
|
|
1593
|
-
A [Option Preset](#option-preset)
|
1756
|
+
A [Option Preset](#option-preset) is a collection of options and their associated values in a named section in the configuration file.
|
1594
1757
|
|
1595
|
-
A named [Option Preset](#option-preset)
|
1758
|
+
A named [Option Preset](#option-preset) can be modified directly using `ascli`, which will update the configuration file :
|
1596
1759
|
|
1597
1760
|
```bash
|
1598
1761
|
ascli config preset set|delete|show|initialize|update <option preset>
|
1599
1762
|
```
|
1600
1763
|
|
1601
|
-
The command `update` allows the easy creation of [Option Preset](#option-preset)
|
1764
|
+
The command `update` allows the easy creation of [Option Preset](#option-preset) by simply providing the options in their command line format, e.g. :
|
1602
1765
|
|
1603
1766
|
```bash
|
1604
1767
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
|
1605
1768
|
```
|
1606
1769
|
|
1607
|
-
- This creates a [Option Preset](#option-preset)
|
1770
|
+
- This creates a [Option Preset](#option-preset) `demo_server` with all provided options.
|
1608
1771
|
|
1609
|
-
The command `set` allows setting individual options in a [Option Preset](#option-preset)
|
1772
|
+
The command `set` allows setting individual options in a [Option Preset](#option-preset).
|
1610
1773
|
|
1611
1774
|
```bash
|
1612
1775
|
ascli config preset set demo_server password my_password_here
|
1613
1776
|
```
|
1614
1777
|
|
1615
|
-
The command `initialize`, like `update` allows to set several
|
1778
|
+
The command `initialize`, like `update` allows to set several options at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
|
1616
1779
|
|
1617
1780
|
```bash
|
1618
1781
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
|
@@ -1624,7 +1787,7 @@ A full terminal based overview of the configuration can be displayed using:
|
|
1624
1787
|
ascli config preset over
|
1625
1788
|
```
|
1626
1789
|
|
1627
|
-
A list of [Option Preset](#option-preset)
|
1790
|
+
A list of [Option Preset](#option-preset) can be displayed using:
|
1628
1791
|
|
1629
1792
|
```bash
|
1630
1793
|
ascli config preset list
|
@@ -1647,6 +1810,15 @@ ascli config preset over
|
|
1647
1810
|
ascli config preset list
|
1648
1811
|
```
|
1649
1812
|
|
1813
|
+
It is possible to load a [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
|
1814
|
+
For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
|
1815
|
+
|
1816
|
+
```bash
|
1817
|
+
ascli config preset set pspecific preset pcommon
|
1818
|
+
```
|
1819
|
+
|
1820
|
+
When `pspecific` is loaded, then cumulative option `preset` will be set and it will also load `pcommon`.
|
1821
|
+
|
1650
1822
|
#### Special Option Preset: `config`
|
1651
1823
|
|
1652
1824
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
@@ -1686,7 +1858,7 @@ The default preset for `config` is read for any plugin invocation, this allows s
|
|
1686
1858
|
When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
|
1687
1859
|
If set, it loads the options independently of the plugin used.
|
1688
1860
|
|
1689
|
-
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global
|
1861
|
+
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global options (e.g. `config ascp use`)
|
1690
1862
|
>
|
1691
1863
|
> **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
|
1692
1864
|
|
@@ -1735,6 +1907,8 @@ detect https://faspex4.example.com/path
|
|
1735
1907
|
detect https://faspex5.example.com/path
|
1736
1908
|
detect https://node.example.com/path
|
1737
1909
|
detect https://shares.example.com/path shares
|
1910
|
+
detect https://tst.example.com/path faspio
|
1911
|
+
detect https://tst.example.com/path httpgw
|
1738
1912
|
detect my_org aoc
|
1739
1913
|
doc
|
1740
1914
|
doc transfer-parameters
|
@@ -1760,6 +1934,7 @@ gem path
|
|
1760
1934
|
gem version
|
1761
1935
|
genkey my_key
|
1762
1936
|
genkey my_key 4096
|
1937
|
+
image https://eudemo.asperademo.com/wallpaper.jpg
|
1763
1938
|
initdemo
|
1764
1939
|
open
|
1765
1940
|
plugin create my_command
|
@@ -1813,17 +1988,17 @@ demo_server:
|
|
1813
1988
|
We can see here:
|
1814
1989
|
|
1815
1990
|
- The configuration was created with `ascli` version 0.3.7
|
1816
|
-
- The default [Option Preset](#option-preset)
|
1817
|
-
- The [Option Preset](#option-preset)
|
1818
|
-
- The default [Option Preset](#option-preset)
|
1991
|
+
- The default [Option Preset](#option-preset) to load for `server` plugin is : `demo_server`
|
1992
|
+
- The [Option Preset](#option-preset) `demo_server` defines some options: the URL and credentials
|
1993
|
+
- The default [Option Preset](#option-preset) to load in any case is : `cli_default`
|
1819
1994
|
|
1820
|
-
Two [Option Preset](#option-preset)
|
1995
|
+
Two [Option Preset](#option-preset) are reserved:
|
1821
1996
|
|
1822
1997
|
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
1823
1998
|
It is used to check compatibility.
|
1824
|
-
- `default` is reserved to define the default [Option Preset](#option-preset)
|
1999
|
+
- `default` is reserved to define the default [Option Preset](#option-preset) name used for known plugins.
|
1825
2000
|
|
1826
|
-
The user may create as many [Option Preset](#option-preset)
|
2001
|
+
The user may create as many [Option Preset](#option-preset) as needed. For instance, a particular [Option Preset](#option-preset) can be created for a particular application instance and contain URL and credentials.
|
1827
2002
|
|
1828
2003
|
Values in the configuration also follow the [Extended Value Syntax](#extended-value-syntax).
|
1829
2004
|
|
@@ -1833,7 +2008,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended-va
|
|
1833
2008
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
1834
2009
|
```
|
1835
2010
|
|
1836
|
-
This creates the [Option Preset](#option-preset)
|
2011
|
+
This creates the [Option Preset](#option-preset):
|
1837
2012
|
|
1838
2013
|
```yaml
|
1839
2014
|
my_aoc_org:
|
@@ -1849,20 +2024,20 @@ Some options are global, some options are available only for some plugins. (the
|
|
1849
2024
|
Options are loaded using this algorithm:
|
1850
2025
|
|
1851
2026
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
1852
|
-
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset)
|
1853
|
-
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset)
|
2027
|
+
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset) for it, and loads it.
|
2028
|
+
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
|
1854
2029
|
- Environment variables are evaluated
|
1855
2030
|
- Command line options are evaluated
|
1856
2031
|
|
1857
|
-
|
2032
|
+
Options are evaluated in the order of command line.
|
1858
2033
|
|
1859
|
-
To avoid loading the default [Option Preset](#option-preset)
|
2034
|
+
To avoid loading the default [Option Preset](#option-preset) for a plugin, use: `-N`
|
1860
2035
|
|
1861
|
-
On command line, words in
|
2036
|
+
On command line, words in option names are separated by a dash (`-`).
|
1862
2037
|
In configuration file, separator is an underscore.
|
1863
2038
|
E.g. `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
|
1864
2039
|
|
1865
|
-
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset)
|
2040
|
+
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset) for the main plugin with:
|
1866
2041
|
|
1867
2042
|
```bash
|
1868
2043
|
ascli config preset set cli_default interactive no
|
@@ -1872,7 +2047,7 @@ ascli config preset set cli_default interactive no
|
|
1872
2047
|
ascli config preset set default config cli_default
|
1873
2048
|
```
|
1874
2049
|
|
1875
|
-
A [Option Preset](#option-preset)
|
2050
|
+
A [Option Preset](#option-preset) value can be removed with `unset`:
|
1876
2051
|
|
1877
2052
|
```bash
|
1878
2053
|
ascli config preset unset cli_default interactive
|
@@ -1892,7 +2067,7 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
|
|
1892
2067
|
|
1893
2068
|
#### Wizard
|
1894
2069
|
|
1895
|
-
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset)
|
2070
|
+
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset) with the provided information.
|
1896
2071
|
|
1897
2072
|
It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
|
1898
2073
|
|
@@ -1914,7 +2089,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my
|
|
1914
2089
|
|
1915
2090
|
This can also be provisioned in a configuration file:
|
1916
2091
|
|
1917
|
-
- Build [Option Preset](#option-preset)
|
2092
|
+
- Build [Option Preset](#option-preset)
|
1918
2093
|
|
1919
2094
|
```bash
|
1920
2095
|
ascli config preset set shares06 url https://10.25.0.6
|
@@ -1934,7 +2109,7 @@ or
|
|
1934
2109
|
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=my_password_here
|
1935
2110
|
```
|
1936
2111
|
|
1937
|
-
- Define this [Option Preset](#option-preset)
|
2112
|
+
- Define this [Option Preset](#option-preset) as the default [Option Preset](#option-preset) for the specified plugin (`shares`)
|
1938
2113
|
|
1939
2114
|
```bash
|
1940
2115
|
ascli config preset set default shares shares06
|
@@ -1946,7 +2121,7 @@ ascli config preset set default shares shares06
|
|
1946
2121
|
ascli config preset overview
|
1947
2122
|
```
|
1948
2123
|
|
1949
|
-
- Execute a command on the shares application using default
|
2124
|
+
- Execute a command on the shares application using default options
|
1950
2125
|
|
1951
2126
|
```bash
|
1952
2127
|
ascli shares repo browse /
|
@@ -2039,7 +2214,7 @@ The lookup is done by comparing the service URL and username (or access key).
|
|
2039
2214
|
|
2040
2215
|
#### Securing passwords and secrets
|
2041
2216
|
|
2042
|
-
A passwords can be saved in clear in a [Option Preset](#option-preset)
|
2217
|
+
A passwords can be saved in clear in a [Option Preset](#option-preset) together with other account information (URL, username, etc...).
|
2043
2218
|
Example:
|
2044
2219
|
|
2045
2220
|
```bash
|
@@ -2060,25 +2235,31 @@ ascli config vault create myconf @json:'{"password":"my_password_here"}'
|
|
2060
2235
|
|
2061
2236
|
### Private Key
|
2062
2237
|
|
2063
|
-
Some applications allow the user to be authenticated using
|
2064
|
-
|
2065
|
-
|
2066
|
-
|
2067
|
-
|
2238
|
+
Some Aspera applications allow the user to be authenticated using [Public Key Cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography):
|
2239
|
+
|
2240
|
+
- for SSH: Server
|
2241
|
+
- for OAuth JWT: AoC, Faspex5, Faspex, Shares
|
2242
|
+
|
2243
|
+
It consists in using a pair of associated keys: a private key and a public key.
|
2244
|
+
The same pair can be used for multiple applications.
|
2245
|
+
The file containing the private key (key pair) can optionally be protected by a passphrase.
|
2068
2246
|
If the key is protected by a passphrase, then it will be prompted when used.
|
2069
2247
|
(some plugins support option `passphrase`)
|
2070
2248
|
|
2071
|
-
The following commands use the shell variable `
|
2249
|
+
The following sample commands use the shell variable `KEY_PAIR_PATH`.
|
2072
2250
|
Set it to the desired safe location of the private key.
|
2073
|
-
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli
|
2251
|
+
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`.
|
2252
|
+
For example:
|
2074
2253
|
|
2075
2254
|
```bash
|
2076
|
-
|
2255
|
+
KEY_PAIR_PATH=~/.aspera/ascli/my_private_key
|
2077
2256
|
```
|
2078
2257
|
|
2079
2258
|
Several methods can be used to generate a key pair.
|
2080
2259
|
|
2081
|
-
The format expected for
|
2260
|
+
The format expected for keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
|
2261
|
+
|
2262
|
+
If another format is used, such as `DER`, it can be converted to `PEM`, e.g. using `openssl`.
|
2082
2263
|
|
2083
2264
|
#### `ascli` for key generation
|
2084
2265
|
|
@@ -2087,21 +2268,13 @@ For convenience, the public key is also extracted with extension `.pub`.
|
|
2087
2268
|
The key is not passphrase protected.
|
2088
2269
|
|
2089
2270
|
```bash
|
2090
|
-
ascli config genkey ${
|
2271
|
+
ascli config genkey ${KEY_PAIR_PATH} 4096
|
2091
2272
|
```
|
2092
2273
|
|
2093
|
-
> **Note:** `ascli` uses the `openssl` library.
|
2094
|
-
|
2095
2274
|
To display the public key of a private key:
|
2096
2275
|
|
2097
2276
|
```bash
|
2098
|
-
ascli config pubkey @file:${
|
2099
|
-
```
|
2100
|
-
|
2101
|
-
To display the version of **openssl** used in `ascli`:
|
2102
|
-
|
2103
|
-
```bash
|
2104
|
-
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
2277
|
+
ascli config pubkey @file:${KEY_PAIR_PATH}
|
2105
2278
|
```
|
2106
2279
|
|
2107
2280
|
#### `ssh-keygen`
|
@@ -2109,16 +2282,16 @@ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
2109
2282
|
Both private and public keys are generated, option `-N` is for passphrase.
|
2110
2283
|
|
2111
2284
|
```bash
|
2112
|
-
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${
|
2285
|
+
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
|
2113
2286
|
```
|
2114
2287
|
|
2115
2288
|
#### `openssl`
|
2116
2289
|
|
2117
|
-
To generate a
|
2290
|
+
To generate a key pair with a passphrase the following can be used on any system:
|
2118
2291
|
|
2119
2292
|
```bash
|
2120
|
-
openssl genrsa -passout pass:_passphrase_here_ -out ${
|
2121
|
-
openssl rsa -pubout -in ${
|
2293
|
+
openssl genrsa -passout pass:_passphrase_here_ -out ${KEY_PAIR_PATH} 4096
|
2294
|
+
openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub
|
2122
2295
|
```
|
2123
2296
|
|
2124
2297
|
`openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
|
@@ -2127,112 +2300,55 @@ In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` t
|
|
2127
2300
|
If option `-nodes` is not available, the passphrase can be removed using this method:
|
2128
2301
|
|
2129
2302
|
```bash
|
2130
|
-
openssl rsa -passin pass:_passphrase_here_ -in ${
|
2131
|
-
mv ${
|
2303
|
+
openssl rsa -passin pass:_passphrase_here_ -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
|
2304
|
+
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
|
2132
2305
|
```
|
2133
2306
|
|
2134
2307
|
To change (or add) the passphrase for a key do:
|
2135
2308
|
|
2136
2309
|
```bash
|
2137
|
-
openssl rsa -des3 -in ${
|
2138
|
-
mv ${
|
2139
|
-
```
|
2140
|
-
|
2141
|
-
### SSL CA certificate bundle
|
2142
|
-
|
2143
|
-
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders, by default it uses Ruby's default certificate store.
|
2144
|
-
|
2145
|
-
To display trusted certificate store locations:
|
2146
|
-
|
2147
|
-
```bash
|
2148
|
-
ascli --show-config --fields=cert_stores
|
2149
|
-
```
|
2150
|
-
|
2151
|
-
Use option `cert_stores` to modify the locations of certificate stores (files or folders).
|
2152
|
-
If you use this option, then default locations are not used.
|
2153
|
-
Default locations can be added using special value `DEF`.
|
2154
|
-
The value can be either an `Array` or `String` (path).
|
2155
|
-
Successive options add paths incrementally.
|
2156
|
-
All files of a folders are added.
|
2157
|
-
|
2158
|
-
`ascli` uses the Ruby `openssl` gem.
|
2159
|
-
By default it uses the system's `openssl` library, but JRuby uses its own implementation.
|
2160
|
-
|
2161
|
-
For example, on Linux to use the system's certificate store:
|
2162
|
-
|
2163
|
-
```bash
|
2164
|
-
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
2165
|
-
```
|
2166
|
-
|
2167
|
-
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
2168
|
-
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
2169
|
-
|
2170
|
-
One can display those default values:
|
2171
|
-
|
2172
|
-
```bash
|
2173
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
2174
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
2310
|
+
openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
|
2311
|
+
mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}
|
2175
2312
|
```
|
2176
2313
|
|
2177
|
-
|
2314
|
+
#### Using an application to generate a key pair
|
2178
2315
|
|
2179
|
-
|
2180
|
-
|
2316
|
+
Many applications are available, including on internet, to generate key pairs.
|
2317
|
+
For example: <https://cryptotools.net/rsagen>
|
2181
2318
|
|
2182
|
-
|
2183
|
-
ascli config ascp info --fields=openssldir
|
2184
|
-
```
|
2319
|
+
> **Note:** Be careful that private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
|
2185
2320
|
|
2186
|
-
|
2187
|
-
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
2188
|
-
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
2321
|
+
### Image and video thumbnails
|
2189
2322
|
|
2190
|
-
|
2191
|
-
|
2192
|
-
Typically done by updating the system's root certificate store.
|
2323
|
+
`ascli` can display thumbnails for images and videos in the terminal.
|
2324
|
+
This is available:
|
2193
2325
|
|
2194
|
-
|
2326
|
+
- in the `thumbnail` command of `node` when using **gen4/access key** API.
|
2327
|
+
- when using the `show` command of `preview` plugin.
|
2328
|
+
- `coffee` and `image` commands of `config` plugin.
|
2329
|
+
- any displayed value which is an url to image can be displayed with option `format` set to `image`
|
2195
2330
|
|
2196
|
-
|
2197
|
-
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
2198
|
-
```
|
2331
|
+
The following options can be specified in the option `image`:
|
2199
2332
|
|
2200
|
-
|
2333
|
+
| Option | Type | Description |
|
2334
|
+
|------------|---------|-------------|
|
2335
|
+
| reserve | Integer | Lines reserved to display a status |
|
2336
|
+
| text | Bool | Display text instead of image|
|
2337
|
+
| double | Bool | Display double text resolution (half characters) |
|
2338
|
+
| font_ratio | Float | Font height/width ratio in terminal |
|
2201
2339
|
|
2202
2340
|
```bash
|
2203
|
-
ascli config
|
2341
|
+
ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --image=@json:'{"text":true}'
|
2204
2342
|
```
|
2205
2343
|
|
2206
|
-
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
2207
|
-
|
2208
|
-
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
2209
|
-
|
2210
2344
|
```bash
|
2211
|
-
|
2345
|
+
curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdbin:
|
2212
2346
|
```
|
2213
2347
|
|
2214
|
-
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
2215
|
-
|
2216
|
-
Then, use this file as certificate store (e.g. here, Node API):
|
2217
|
-
|
2218
2348
|
```bash
|
2219
|
-
ascli config
|
2349
|
+
echo -n https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @uri:@stdin:
|
2220
2350
|
```
|
2221
2351
|
|
2222
|
-
### Image and video thumbnails
|
2223
|
-
|
2224
|
-
`ascli` can display thumbnails for images and videos in the terminal.
|
2225
|
-
This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
|
2226
|
-
It's also available when using the `show` command of `preview` plugin.
|
2227
|
-
|
2228
|
-
The following options can be specified in the option `query`:
|
2229
|
-
|
2230
|
-
| Option | Description |
|
2231
|
-
|------------|-------------|
|
2232
|
-
| text | Display text instead of image (Bool) |
|
2233
|
-
| double | Display double text resolution (half characters) (Bool) |
|
2234
|
-
| font_ratio | Font height/width ratio in terminal (Float) |
|
2235
|
-
|
2236
2352
|
### Graphical Interactions: Browser and Text Editor
|
2237
2353
|
|
2238
2354
|
Some actions may require the use of a graphical tool:
|
@@ -2250,9 +2366,7 @@ It is also possible to force the graphical mode with option --ui :
|
|
2250
2366
|
|
2251
2367
|
The gem is equipped with traces, mainly for debugging and learning APIs.
|
2252
2368
|
By default logging level is `warn` and the output channel is `stderr`.
|
2253
|
-
To increase debug level, use
|
2254
|
-
|
2255
|
-
It is also possible to activate traces before log facility initialization using env var `ASCLI_LOG_LEVEL`.
|
2369
|
+
To increase debug level, use option `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or an [Option Preset](#option-preset)).
|
2256
2370
|
|
2257
2371
|
By default passwords and secrets are removed from logs.
|
2258
2372
|
Use option `log_secrets` set to `yes` to reveal secrets in logs.
|
@@ -2332,21 +2446,33 @@ Refer to the following sections.
|
|
2332
2446
|
|
2333
2447
|
#### Proxy for REST and HTTP Gateway
|
2334
2448
|
|
2335
|
-
REST API calls and transfers based on HTTP Gateway both use Ruby `Net::HTTP`
|
2449
|
+
REST API calls and transfers based on HTTP Gateway both use Ruby's `Net::HTTP` class.
|
2450
|
+
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
2336
2451
|
|
2337
|
-
|
2452
|
+
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used .
|
2338
2453
|
|
2339
|
-
The `http_proxy` environment variable (**lower case
|
2454
|
+
The `http_proxy` environment variable (**lower case**) can be set to the **URL** of the proxy (with optional credentials).
|
2455
|
+
Syntax is: `(http|https)://[user:password@]host:port`.
|
2340
2456
|
E.g. `http://myproxy.org.net:3128`.
|
2341
|
-
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
2342
2457
|
|
2343
|
-
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not**
|
2458
|
+
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** valid.
|
2459
|
+
|
2460
|
+
Credentials for proxy are optional but can also be specified:
|
2344
2461
|
|
2345
2462
|
```bash
|
2346
|
-
export http_proxy=http://proxy.example.com:3128
|
2463
|
+
export http_proxy=http://user:password@proxy.example.com:3128
|
2464
|
+
```
|
2465
|
+
|
2466
|
+
Option `http_proxy` does the same (set env var) but on command line:
|
2467
|
+
|
2468
|
+
```bash
|
2469
|
+
ascli --http-proxy=http://username:password@host:port ...
|
2347
2470
|
```
|
2348
2471
|
|
2349
2472
|
Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) javascript value.
|
2473
|
+
|
2474
|
+
Note that proxy credentials are not supported in PAC files.
|
2475
|
+
|
2350
2476
|
To read the script from a URL (`http:`, `https:` and `file:`), use prefix: `@uri:`.
|
2351
2477
|
A minimal script can be specified to define the use of a local proxy:
|
2352
2478
|
|
@@ -2381,7 +2507,7 @@ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.
|
|
2381
2507
|
PROXY proxy.example.com:8080
|
2382
2508
|
```
|
2383
2509
|
|
2384
|
-
If the proxy requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
2510
|
+
If the proxy found with the PAC requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
2385
2511
|
|
2386
2512
|
```bash
|
2387
2513
|
ascli --proxy-credentials=@json:'["__username_here__","__password_here__"]' ...
|
@@ -2397,7 +2523,7 @@ Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallb
|
|
2397
2523
|
|
2398
2524
|
#### FASP proxy (forward) for transfers
|
2399
2525
|
|
2400
|
-
To specify a FASP proxy (forward), set the [
|
2526
|
+
To specify a FASP proxy (forward), set the [**transfer-spec**](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
|
2401
2527
|
|
2402
2528
|
For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
|
2403
2529
|
|
@@ -2413,7 +2539,20 @@ Or, alternatively, (prefer transfer spec like above, generally):
|
|
2413
2539
|
|
2414
2540
|
### FASP configuration
|
2415
2541
|
|
2416
|
-
|
2542
|
+
`ascli` uses one of the transfer agents to execute transfers.
|
2543
|
+
|
2544
|
+
By default it uses the `direct` agent, which is basically a local `ascp`.
|
2545
|
+
Nevertheless, `ascli` does not come with `ascp` installed.
|
2546
|
+
This is the reason why it is advised to install the Aspera Transfer SDK during installation (`ascli config ascp install`).
|
2547
|
+
|
2548
|
+
By default, `ascli` uses the `ascp` binary found in **well known locations**, i.e. typical Aspera product installation paths.
|
2549
|
+
|
2550
|
+
The way to specify the location of `ascp` is to use either options:
|
2551
|
+
|
2552
|
+
- `ascp_path`
|
2553
|
+
- `use_product`
|
2554
|
+
|
2555
|
+
The `config` plugin allows to find and specify the location of `ascp`.
|
2417
2556
|
It provides the following commands for `ascp` subcommand:
|
2418
2557
|
|
2419
2558
|
- `show` : shows the path of `ascp` used
|
@@ -2447,9 +2586,9 @@ ascli config ascp info
|
|
2447
2586
|
|
2448
2587
|
By default, `ascli` uses any found local product with `ascp`, including Transfer SDK.
|
2449
2588
|
|
2450
|
-
To
|
2589
|
+
To override and use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
|
2451
2590
|
|
2452
|
-
For a permanent change, the command `config ascp use` sets the same
|
2591
|
+
For a permanent change, the command `config ascp use` sets the same option for the global default.
|
2453
2592
|
|
2454
2593
|
Using a POSIX shell:
|
2455
2594
|
|
@@ -2477,7 +2616,10 @@ Saved to default global preset global_common_defaults
|
|
2477
2616
|
|
2478
2617
|
If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
|
2479
2618
|
|
2480
|
-
|
2619
|
+
If option `ascp_path` is not set, then the product identified with option `use_product` is used.
|
2620
|
+
|
2621
|
+
If `use_product` is not set, then the first product found is used,
|
2622
|
+
this is equivalent to using option: `--use-product=FIRST`.
|
2481
2623
|
|
2482
2624
|
Locally installed Aspera products can be listed with:
|
2483
2625
|
|
@@ -2496,10 +2638,6 @@ ascli config ascp products list
|
|
2496
2638
|
+---------------------------------------+----------------------------------------+
|
2497
2639
|
```
|
2498
2640
|
|
2499
|
-
#### Selection of local client for `ascp` for [`direct`](#agent-direct) agent
|
2500
|
-
|
2501
|
-
If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
|
2502
|
-
|
2503
2641
|
Using the option `use_product` finds the `ascp` binary of the selected product.
|
2504
2642
|
|
2505
2643
|
To permanently use the `ascp` of a product:
|
@@ -2556,12 +2694,12 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
|
2556
2694
|
|
2557
2695
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
2558
2696
|
|
2559
|
-
When a transfer needs to be started, a [
|
2560
|
-
This [
|
2697
|
+
When a transfer needs to be started, a [**transfer-spec**](#transfer-specification) has been internally prepared.
|
2698
|
+
This [**transfer-spec**](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
|
2561
2699
|
|
2562
|
-
|
2700
|
+
The following agents are supported and selected with option `transfer`:
|
2563
2701
|
|
2564
|
-
- [`direct`](#agent-direct) : execution of `ascp`
|
2702
|
+
- [`direct`](#agent-direct) : direct execution of `ascp` (local)
|
2565
2703
|
- [`trsdk`](#agent-transfer-sdk) : use of Aspera Transfer SDK (local)
|
2566
2704
|
- [`connect`](#agent-connect-client) : use Connect Client (local)
|
2567
2705
|
- [`alpha`](#agent-desktop-client) : use the new Desktop Client (local)
|
@@ -2572,7 +2710,7 @@ There are currently 3 agents, set with option `transfer`:
|
|
2572
2710
|
For example, a node agent executing an **upload**, or **package send** operation
|
2573
2711
|
will effectively push files to the related server from the agent node.
|
2574
2712
|
|
2575
|
-
`ascli` standardizes on the use of a [
|
2713
|
+
`ascli` standardizes on the use of a [**transfer-spec**](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
2576
2714
|
|
2577
2715
|
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2578
2716
|
|
@@ -2583,7 +2721,7 @@ This is the default agent for `ascli` (option `--transfer=direct`).
|
|
2583
2721
|
`ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
|
2584
2722
|
Refer to section [FASP](#fasp-configuration).
|
2585
2723
|
|
2586
|
-
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and
|
2724
|
+
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
|
2587
2725
|
|
2588
2726
|
| Name | Type | Description |
|
2589
2727
|
|------------------------|-------|-------------|
|
@@ -2607,7 +2745,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
|
|
2607
2745
|
```
|
2608
2746
|
|
2609
2747
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
2610
|
-
In order to use a custom certificate store, use the `trusted_certs`
|
2748
|
+
In order to use a custom certificate store, use the `trusted_certs` option.
|
2611
2749
|
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
2612
2750
|
|
2613
2751
|
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
@@ -2726,6 +2864,7 @@ There are no option for `transfer_info`.
|
|
2726
2864
|
|
2727
2865
|
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
2728
2866
|
This is especially useful for direct node-to-node transfers.
|
2867
|
+
|
2729
2868
|
Parameters provided in option `transfer_info` are:
|
2730
2869
|
|
2731
2870
|
| Name | Type | Description |
|
@@ -2735,7 +2874,7 @@ Parameters provided in option `transfer_info` are:
|
|
2735
2874
|
| password | string | Password, secret or bearer token</br>Mandatory |
|
2736
2875
|
| root_id | string | Root file id</br>Mandatory only for bearer token |
|
2737
2876
|
|
2738
|
-
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset)
|
2877
|
+
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset) :
|
2739
2878
|
|
2740
2879
|
```bash
|
2741
2880
|
--transfer-info=@preset:_name_here_
|
@@ -2749,12 +2888,14 @@ or be specified using the extended value syntax :
|
|
2749
2888
|
|
2750
2889
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
2751
2890
|
|
2752
|
-
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the
|
2891
|
+
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file id of the top folder to use on the node using this access key.
|
2753
2892
|
It can be either the access key's root file id, or any authorized file id underneath it.
|
2754
2893
|
|
2755
2894
|
#### Agent: HTTP Gateway
|
2756
2895
|
|
2757
|
-
|
2896
|
+
The Aspera HTTP Gateway is a service that allows to send and receive files using HTTPS.
|
2897
|
+
|
2898
|
+
By specifying option: `--transfer=httpgw`, `ascli` will start transfers using the Aspera HTTP Gateway.
|
2758
2899
|
|
2759
2900
|
Parameters provided in option `transfer_info` are:
|
2760
2901
|
|
@@ -2773,6 +2914,8 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
|
|
2773
2914
|
|
2774
2915
|
> **Note:** The gateway only supports transfers authorized with a token.
|
2775
2916
|
|
2917
|
+
If the application, e.g. AoC or Faspex 5, is configured to use the HTTP Gateway, then `ascli` will automatically use the gateway URL if `--transfer=httpgw` is specified, so `transfer_info` becomes optional.
|
2918
|
+
|
2776
2919
|
#### Agent: Transfer SDK
|
2777
2920
|
|
2778
2921
|
Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
|
@@ -2783,7 +2926,7 @@ Options for `transfer_info` are:
|
|
2783
2926
|
| Name | Type | Description |
|
2784
2927
|
|----------|--------|-------------|
|
2785
2928
|
| `url` | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: `:0` |
|
2786
|
-
| `external` | bool | Use external daemon, do not start
|
2929
|
+
| `external` | bool | Use external daemon, do not start one.<br/>Default: `false` |
|
2787
2930
|
| `keep` | bool | Keep the daemon running after exiting `ascli`<br/>Default: `false` |
|
2788
2931
|
|
2789
2932
|
> **Note:** If port zero is specified in the URL, then the daemon will listen on a random available port. If no address is specified, then `127.0.0.1` is used.
|
@@ -2812,7 +2955,7 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
2812
2955
|
### Transfer Specification
|
2813
2956
|
|
2814
2957
|
Some commands lead to file transfer (upload/download).
|
2815
|
-
All parameters necessary for this transfer are described in a [
|
2958
|
+
All parameters necessary for this transfer are described in a [**transfer-spec**](#transfer-specification) (Transfer Specification), such as:
|
2816
2959
|
|
2817
2960
|
- Server address
|
2818
2961
|
- Transfer user name
|
@@ -2820,11 +2963,11 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
|
|
2820
2963
|
- File list
|
2821
2964
|
- Etc...
|
2822
2965
|
|
2823
|
-
`ascli` builds the [
|
2966
|
+
`ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
|
2824
2967
|
It is not necessary to provide additional parameters on the command line for this transfer.
|
2825
2968
|
|
2826
|
-
It is possible to modify or add any of the supported [
|
2827
|
-
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [
|
2969
|
+
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameter using the `ts` option.
|
2970
|
+
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
|
2828
2971
|
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
2829
2972
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
2830
2973
|
|
@@ -2834,15 +2977,15 @@ It is possible to specify `ascp` options when the `transfer` option is set to [`
|
|
2834
2977
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
2835
2978
|
This is especially useful for `ascp` command line parameters not supported in the transfer spec.
|
2836
2979
|
|
2837
|
-
The use of a [
|
2980
|
+
The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the advantage of:
|
2838
2981
|
|
2839
2982
|
- Common to all [Transfer Agent](#transfer-clients-agents)
|
2840
2983
|
- Not dependent on command line limitations (special characters...)
|
2841
2984
|
|
2842
2985
|
### Transfer Parameters
|
2843
2986
|
|
2844
|
-
All standard [
|
2845
|
-
[
|
2987
|
+
All standard [**transfer-spec**](#transfer-specification) parameters can be specified.
|
2988
|
+
[**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
|
2846
2989
|
|
2847
2990
|
References:
|
2848
2991
|
|
@@ -2960,7 +3103,7 @@ The destination folder is set by `ascli` by default to:
|
|
2960
3103
|
- `.` for downloads
|
2961
3104
|
- `/` for uploads
|
2962
3105
|
|
2963
|
-
It is specified by the [
|
3106
|
+
It is specified by the [**transfer-spec**](#transfer-specification) parameter `destination_root`.
|
2964
3107
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
2965
3108
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
2966
3109
|
`--to-folder=<path>` .
|
@@ -2974,8 +3117,8 @@ By default the list of files to transfer is simply provided on the command line.
|
|
2974
3117
|
The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
|
2975
3118
|
The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
|
2976
3119
|
|
2977
|
-
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [
|
2978
|
-
So, eventually, the list of files to transfer is provided to the transfer agent using the [
|
3120
|
+
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
|
3121
|
+
So, eventually, the list of files to transfer is provided to the transfer agent using the [**transfer-spec**](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
|
2979
3122
|
The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
|
2980
3123
|
|
2981
3124
|
Possible values for option `sources` are:
|
@@ -3396,7 +3539,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
3396
3539
|
```text
|
3397
3540
|
ascli -h
|
3398
3541
|
NAME
|
3399
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3542
|
+
ascli -- a command line tool for Aspera Applications (v4.18.0)
|
3400
3543
|
|
3401
3544
|
SYNOPSIS
|
3402
3545
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -3418,7 +3561,7 @@ COMMANDS
|
|
3418
3561
|
OPTIONS
|
3419
3562
|
Options begin with a '-' (minus), and value is provided on command line.
|
3420
3563
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
3421
|
-
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
|
3564
|
+
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, stdbin, yaml, zlib, extend, preset, vault
|
3422
3565
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
3423
3566
|
|
3424
3567
|
ARGS
|
@@ -3427,20 +3570,20 @@ ARGS
|
|
3427
3570
|
OPTIONS: global
|
3428
3571
|
--interactive=ENUM Use interactive input of missing params: [no], yes
|
3429
3572
|
--ask-options=ENUM Ask even optional options: [no], yes
|
3430
|
-
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
|
3573
|
+
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv, image
|
3431
3574
|
--output=VALUE Destination for results (String)
|
3432
3575
|
--display=ENUM Output only some information: [info], data, error
|
3433
3576
|
--fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
|
3434
3577
|
--select=VALUE Select only some items in lists: column, value (Hash, Proc)
|
3435
|
-
--table-style=VALUE Table display style
|
3578
|
+
--table-style=VALUE Table display style (Hash)
|
3436
3579
|
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
3437
3580
|
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
3438
3581
|
--show-secrets=ENUM Show secrets on command output: [no], yes
|
3582
|
+
--image=VALUE Options for image display (Hash)
|
3439
3583
|
-h, --help Show this message
|
3440
3584
|
--bash-comp Generate bash completion for command
|
3441
3585
|
--show-config Display parameters used for the provided action
|
3442
3586
|
-v, --version Display version
|
3443
|
-
-w, --warnings Check for language warnings
|
3444
3587
|
--ui=ENUM Method to start browser: text, [graphical]
|
3445
3588
|
--log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
|
3446
3589
|
--logger=ENUM Logging method: [stderr], stdout, syslog
|
@@ -3451,7 +3594,7 @@ OPTIONS: global
|
|
3451
3594
|
--pid-file=VALUE Write process identifier to file, delete on exit (String)
|
3452
3595
|
|
3453
3596
|
COMMAND: config
|
3454
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
3597
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
3455
3598
|
OPTIONS:
|
3456
3599
|
--home=VALUE Home folder for tool (String)
|
3457
3600
|
--config-file=VALUE Path to YAML file with preset configuration
|
@@ -3485,9 +3628,10 @@ OPTIONS:
|
|
3485
3628
|
--silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
|
3486
3629
|
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
3487
3630
|
--http-options=VALUE Options for HTTP/S socket (Hash)
|
3631
|
+
--http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
|
3488
3632
|
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
3489
3633
|
--fpac=VALUE Proxy auto configuration script
|
3490
|
-
--proxy-credentials=VALUE HTTP proxy credentials
|
3634
|
+
--proxy-credentials=VALUE HTTP proxy credentials for fpac. Array: user,password (Array)
|
3491
3635
|
--ts=VALUE Override transfer spec values (Hash)
|
3492
3636
|
--to-folder=VALUE Destination folder for transferred files
|
3493
3637
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
@@ -3505,7 +3649,7 @@ OPTIONS:
|
|
3505
3649
|
|
3506
3650
|
|
3507
3651
|
COMMAND: node
|
3508
|
-
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer upload watch_folder
|
3652
|
+
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer transport upload watch_folder
|
3509
3653
|
OPTIONS:
|
3510
3654
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3511
3655
|
--username=VALUE User's name to log in
|
@@ -3518,28 +3662,32 @@ OPTIONS:
|
|
3518
3662
|
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3519
3663
|
|
3520
3664
|
|
3521
|
-
COMMAND:
|
3522
|
-
SUBCOMMANDS: health
|
3665
|
+
COMMAND: faspio
|
3666
|
+
SUBCOMMANDS: bridges health
|
3523
3667
|
OPTIONS:
|
3524
3668
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3525
3669
|
--username=VALUE User's name to log in
|
3526
3670
|
--password=VALUE User's password
|
3527
|
-
--
|
3528
|
-
--
|
3529
|
-
--
|
3530
|
-
--
|
3671
|
+
--auth=ENUM OAuth type of authentication: jwt, basic
|
3672
|
+
--client-id=VALUE OAuth client identifier
|
3673
|
+
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3674
|
+
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
3531
3675
|
|
3532
3676
|
|
3533
|
-
COMMAND:
|
3534
|
-
SUBCOMMANDS:
|
3677
|
+
COMMAND: orchestrator
|
3678
|
+
SUBCOMMANDS: health info plugins processes workflow
|
3535
3679
|
OPTIONS:
|
3536
3680
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3537
3681
|
--username=VALUE User's name to log in
|
3538
3682
|
--password=VALUE User's password
|
3683
|
+
--result=VALUE Specify result value as: 'work_step:parameter'
|
3684
|
+
--synchronous=ENUM Wait for completion: [no], yes
|
3685
|
+
--ret-style=ENUM How return type is requested in api: header, [arg], ext
|
3686
|
+
--auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
|
3539
3687
|
|
3540
3688
|
|
3541
3689
|
COMMAND: alee
|
3542
|
-
SUBCOMMANDS: entitlement
|
3690
|
+
SUBCOMMANDS: entitlement health
|
3543
3691
|
OPTIONS:
|
3544
3692
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3545
3693
|
--username=VALUE User's name to log in
|
@@ -3587,6 +3735,12 @@ OPTIONS:
|
|
3587
3735
|
--identity=VALUE Authentication URL (https://iam.cloud.ibm.com/identity)
|
3588
3736
|
|
3589
3737
|
|
3738
|
+
COMMAND: httpgw
|
3739
|
+
SUBCOMMANDS: health info
|
3740
|
+
OPTIONS:
|
3741
|
+
--url=VALUE URL of application, e.g. https://app.example.com/aspera/app
|
3742
|
+
|
3743
|
+
|
3590
3744
|
COMMAND: faspex
|
3591
3745
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
3592
3746
|
OPTIONS:
|
@@ -3649,9 +3803,9 @@ OPTIONS:
|
|
3649
3803
|
--scope=VALUE OAuth scope for AoC API calls
|
3650
3804
|
--redirect-uri=VALUE OAuth API client redirect URI
|
3651
3805
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3652
|
-
--passphrase=VALUE RSA private key passphrase
|
3806
|
+
--passphrase=VALUE RSA private key passphrase (String)
|
3653
3807
|
--workspace=VALUE Name of workspace (String, NilClass)
|
3654
|
-
--new-user-option=VALUE New user creation option for unknown package recipients
|
3808
|
+
--new-user-option=VALUE New user creation option for unknown package recipients (Hash)
|
3655
3809
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
3656
3810
|
--validator=VALUE Identifier of validator (optional for central)
|
3657
3811
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
@@ -3725,7 +3879,7 @@ Aspera on Cloud and Faspex 5 rely on Oauth.
|
|
3725
3879
|
By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
3726
3880
|
|
3727
3881
|
```bash
|
3728
|
-
ascli --show-config --
|
3882
|
+
ascli --show-config --fields=plugin_folder
|
3729
3883
|
```
|
3730
3884
|
|
3731
3885
|
You can create the skeleton of a new plugin like this:
|
@@ -3805,7 +3959,7 @@ Several types of OAuth authentication are supported:
|
|
3805
3959
|
|
3806
3960
|
The authentication method is controlled by option `auth`.
|
3807
3961
|
|
3808
|
-
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset)
|
3962
|
+
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset) for Aspera on Cloud](#configuration-for-aspera-on-cloud).
|
3809
3963
|
|
3810
3964
|
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
|
3811
3965
|
|
@@ -3842,9 +3996,9 @@ Once the client is registered, a **Client ID** and **Secret** are created, these
|
|
3842
3996
|
|
3843
3997
|
#### Configuration for Aspera on Cloud
|
3844
3998
|
|
3845
|
-
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset)
|
3999
|
+
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset) for `ascli` in its configuration file.
|
3846
4000
|
|
3847
|
-
Lets create a [Option Preset](#option-preset)
|
4001
|
+
Lets create a [Option Preset](#option-preset) called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
|
3848
4002
|
|
3849
4003
|
```bash
|
3850
4004
|
ascli config preset ask my_aoc_org url client_id client_secret
|
@@ -3858,7 +4012,7 @@ updated: my_aoc_org
|
|
3858
4012
|
|
3859
4013
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
3860
4014
|
|
3861
|
-
Define this [Option Preset](#option-preset)
|
4015
|
+
Define this [Option Preset](#option-preset) as default configuration for the `aspera` plugin:
|
3862
4016
|
|
3863
4017
|
```bash
|
3864
4018
|
ascli config preset set default aoc my_aoc_org
|
@@ -3949,9 +4103,9 @@ modified
|
|
3949
4103
|
|
3950
4104
|
> **Note:** The `aspera user info show` command can be used to verify modifications.
|
3951
4105
|
|
3952
|
-
#### [Option Preset](#option-preset)
|
4106
|
+
#### [Option Preset](#option-preset) modification for JWT
|
3953
4107
|
|
3954
|
-
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset)
|
4108
|
+
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset), do the following:
|
3955
4109
|
|
3956
4110
|
- Change auth method to JWT
|
3957
4111
|
- Provide location of private key
|
@@ -3985,7 +4139,7 @@ In that case, it is possible to list those shared folder by using a value for op
|
|
3985
4139
|
|
3986
4140
|
#### AoC: First Use
|
3987
4141
|
|
3988
|
-
Once client has been registered and [Option Preset](#option-preset)
|
4142
|
+
Once client has been registered and [Option Preset](#option-preset) created: `ascli` can be used:
|
3989
4143
|
|
3990
4144
|
```bash
|
3991
4145
|
ascli aoc files br /
|
@@ -4188,7 +4342,7 @@ Options:
|
|
4188
4342
|
#### Transfer: Using specific transfer ports
|
4189
4343
|
|
4190
4344
|
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
4191
|
-
The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
4345
|
+
The option `default_ports` ([yes]/no) allows `ascli` to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
4192
4346
|
|
4193
4347
|
#### Using ATS
|
4194
4348
|
|
@@ -4588,7 +4742,7 @@ General syntax:
|
|
4588
4742
|
ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
|
4589
4743
|
```
|
4590
4744
|
|
4591
|
-
Package creation parameter are sent as
|
4745
|
+
Package creation parameter are sent as **Command Parameter**.
|
4592
4746
|
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
4593
4747
|
|
4594
4748
|
List allowed shared inbox destinations with:
|
@@ -4766,13 +4920,13 @@ ascli aoc files short_link public delete _id_
|
|
4766
4920
|
|
4767
4921
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
4768
4922
|
|
4769
|
-
Although optional, the creation of [Option Preset](#option-preset)
|
4923
|
+
Although optional, the creation of [Option Preset](#option-preset) is recommended to avoid placing all parameters in the command line.
|
4770
4924
|
|
4771
4925
|
Procedure to send a file from org1 to org2:
|
4772
4926
|
|
4773
|
-
- Get access to Organization 1 and create a [Option Preset](#option-preset)
|
4927
|
+
- Get access to Organization 1 and create a [Option Preset](#option-preset): e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
|
4774
4928
|
- Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
|
4775
|
-
- Get access to Organization 2 and create a [Option Preset](#option-preset)
|
4929
|
+
- Get access to Organization 2 and create a [Option Preset](#option-preset): e.g. `org2`
|
4776
4930
|
- Check that access works and locate the destination folder `mydestfolder`
|
4777
4931
|
- Execute the following:
|
4778
4932
|
|
@@ -4812,6 +4966,7 @@ For instructions, refer to section `find` for plugin `node`.
|
|
4812
4966
|
admin analytics transfers nodes
|
4813
4967
|
admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
4814
4968
|
admin analytics transfers users --once_only=yes
|
4969
|
+
admin application list
|
4815
4970
|
admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4816
4971
|
admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4817
4972
|
admin ats access_key delete ak1ibmcloud
|
@@ -4822,36 +4977,35 @@ admin ats cluster list
|
|
4822
4977
|
admin ats cluster show --cloud=aws --region=eu-west-1
|
4823
4978
|
admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
4824
4979
|
admin auth_providers list
|
4825
|
-
admin
|
4826
|
-
admin
|
4827
|
-
admin
|
4828
|
-
admin
|
4829
|
-
admin
|
4830
|
-
admin
|
4831
|
-
admin
|
4832
|
-
admin
|
4833
|
-
admin
|
4834
|
-
admin
|
4835
|
-
admin
|
4836
|
-
admin
|
4837
|
-
admin
|
4838
|
-
admin
|
4839
|
-
admin
|
4840
|
-
admin
|
4841
|
-
admin
|
4842
|
-
admin
|
4843
|
-
admin
|
4844
|
-
admin
|
4845
|
-
admin
|
4846
|
-
admin
|
4847
|
-
admin
|
4848
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4849
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4850
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4851
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4852
|
-
admin resource workspace list
|
4853
|
-
admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4980
|
+
admin client list
|
4981
|
+
admin client_access_key list
|
4982
|
+
admin client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
|
4983
|
+
admin client_registration_token delete client_reg_id
|
4984
|
+
admin client_registration_token list
|
4985
|
+
admin contact list
|
4986
|
+
admin dropbox list
|
4987
|
+
admin dropbox_membership list
|
4988
|
+
admin group list
|
4989
|
+
admin kms_profile list
|
4990
|
+
admin node do %name:my_ak_name --secret=my_ak_secret browse /
|
4991
|
+
admin node do %name:my_ak_name --secret=my_ak_secret delete /folder1
|
4992
|
+
admin node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4993
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4994
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4995
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4996
|
+
admin node list
|
4997
|
+
admin operation list
|
4998
|
+
admin organization show
|
4999
|
+
admin package list --http-options=@json:'{"read_timeout":120.0}'
|
5000
|
+
admin saml_configuration list
|
5001
|
+
admin self show
|
5002
|
+
admin short_link list
|
4854
5003
|
admin subscription
|
5004
|
+
admin user list
|
5005
|
+
admin user modify %name:my_user_email @json:'{"deactivated":false}'
|
5006
|
+
admin workspace list
|
5007
|
+
admin workspace_membership list
|
5008
|
+
admin workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4855
5009
|
automation workflow action wf_id create @json:'{"name":"toto"}' \
|
4856
5010
|
automation workflow create @json:'{"name":"test_workflow"}'
|
4857
5011
|
automation workflow delete wf_id
|
@@ -4924,9 +5078,9 @@ user workspaces list
|
|
4924
5078
|
|
4925
5079
|
ATS is usable either :
|
4926
5080
|
|
4927
|
-
- From an AoC subscription : ascli aoc admin ats : use AoC authentication
|
5081
|
+
- From an AoC subscription : `ascli aoc admin ats` : use AoC authentication
|
4928
5082
|
|
4929
|
-
- Or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
|
5083
|
+
- Or from an IBM Cloud subscription : `ascli ats` : use IBM Cloud API key authentication
|
4930
5084
|
|
4931
5085
|
### IBM Cloud ATS : Creation of api key
|
4932
5086
|
|
@@ -5098,8 +5252,8 @@ mkdir my_upload_folder/target_hot
|
|
5098
5252
|
mv my_upload_folder/200KB.2 my_upload_folder/to.delete
|
5099
5253
|
sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
|
5100
5254
|
sync admin status --sync-info=@json:'{"name":"sync2"}'
|
5101
|
-
sync admin status
|
5102
|
-
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"
|
5255
|
+
sync admin status my_srv_sync --sync-info=@json:'{"sessions":[{"name":"my_srv_sync","local_dir":"/data/local_sync"}]}'
|
5256
|
+
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_srv_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
|
5103
5257
|
sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
|
5104
5258
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
|
5105
5259
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
|
@@ -5203,7 +5357,7 @@ ascli server browse /aspera-test-dir-large
|
|
5203
5357
|
ascli server download /aspera-test-dir-large/200MB
|
5204
5358
|
```
|
5205
5359
|
|
5206
|
-
`initdemo` creates a [Option Preset](#option-preset)
|
5360
|
+
`initdemo` creates a [Option Preset](#option-preset) `demoserver` and set it as default for plugin `server`.
|
5207
5361
|
|
5208
5362
|
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
|
5209
5363
|
|
@@ -5235,6 +5389,18 @@ Example:
|
|
5235
5389
|
- `ascli node browse /` : list files with **gen3/node user** API
|
5236
5390
|
- `ascli node access_key do self browse /` : list files with **gen4/access key** API
|
5237
5391
|
|
5392
|
+
## Browse
|
5393
|
+
|
5394
|
+
Native API parameters can be placed in option `query`.
|
5395
|
+
|
5396
|
+
Special parameters can be placed in option `query` for "gen3" browse:
|
5397
|
+
|
5398
|
+
| Parameter | Description |
|
5399
|
+
|-----------|-------------|
|
5400
|
+
| `recursive` | Recursively list files |
|
5401
|
+
| `max` | Maximum number of files to list |
|
5402
|
+
| `self` | Offset in the list |
|
5403
|
+
|
5238
5404
|
### Operation `find` on **gen4/access key**
|
5239
5405
|
|
5240
5406
|
The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
|
@@ -5337,7 +5503,7 @@ There are three sync types of operations:
|
|
5337
5503
|
|
5338
5504
|
It is possible to start a FASPStream session using the node API:
|
5339
5505
|
|
5340
|
-
Use the command `ascli node stream create --ts=@json:<value>`, with [
|
5506
|
+
Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-spec**](#transfer-specification):
|
5341
5507
|
|
5342
5508
|
```json
|
5343
5509
|
{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
|
@@ -5643,6 +5809,7 @@ sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"
|
|
5643
5809
|
sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5644
5810
|
transfer list --query=@json:'{"active_only":true}'
|
5645
5811
|
transfer sessions
|
5812
|
+
transport
|
5646
5813
|
upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
5647
5814
|
upload --username=my_ak_name --password=my_ak_secret test_file.bin
|
5648
5815
|
upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
|
@@ -5664,7 +5831,7 @@ IBM Aspera's newer self-managed application.
|
|
5664
5831
|
|
5665
5832
|
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
5666
5833
|
|
5667
|
-
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset)
|
5834
|
+
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset):
|
5668
5835
|
|
5669
5836
|
```bash
|
5670
5837
|
ascli config wizard
|
@@ -5802,24 +5969,30 @@ ascli config preset update f5boot --url=https://localhost/aspera/faspex --auth=b
|
|
5802
5969
|
> **Note:** Add `ascli faspex5` in front of the commands:
|
5803
5970
|
|
5804
5971
|
```bash
|
5805
|
-
admin
|
5806
|
-
admin
|
5807
|
-
admin
|
5808
|
-
admin
|
5809
|
-
admin
|
5810
|
-
admin
|
5811
|
-
admin
|
5812
|
-
admin
|
5813
|
-
admin
|
5814
|
-
admin
|
5815
|
-
admin
|
5816
|
-
admin
|
5817
|
-
admin
|
5818
|
-
admin
|
5819
|
-
admin
|
5820
|
-
admin
|
5972
|
+
admin accounts list
|
5973
|
+
admin clean_deleted
|
5974
|
+
admin contacts list
|
5975
|
+
admin distribution_lists create @json:'{"name":"test4","contacts":[{"name":"john@example.com"}]}'
|
5976
|
+
admin distribution_lists delete %name:test4
|
5977
|
+
admin distribution_lists list --query=@json:'{"type":"global"}'
|
5978
|
+
admin event app
|
5979
|
+
admin event web
|
5980
|
+
admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
|
5981
|
+
admin metadata_profiles list
|
5982
|
+
admin node list
|
5983
|
+
admin oauth_clients list
|
5984
|
+
admin registrations list
|
5985
|
+
admin saml_configs list
|
5986
|
+
admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
5987
|
+
admin shared_inboxes list
|
5988
|
+
admin shared_inboxes list --query=@json:'{"all":true}'
|
5989
|
+
admin shared_inboxes members %name:my_shared_box_name create %name:john@example.com
|
5990
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
|
5991
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
|
5992
|
+
admin shared_inboxes members %name:my_shared_box_name list
|
5821
5993
|
admin smtp show
|
5822
5994
|
admin smtp test my_email_external
|
5995
|
+
admin workgroups list
|
5823
5996
|
bearer_token
|
5824
5997
|
gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
5825
5998
|
health
|
@@ -5853,7 +6026,7 @@ user profile show
|
|
5853
6026
|
```
|
5854
6027
|
|
5855
6028
|
Most commands are directly REST API calls.
|
5856
|
-
Parameters to commands are carried through option `query`, as extended value, for `list`, or through
|
6029
|
+
Parameters to commands are carried through option `query`, as extended value, for `list`, or through **Command Parameter** for creation.
|
5857
6030
|
One can conveniently use the JSON format with prefix `@json:`.
|
5858
6031
|
|
5859
6032
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
@@ -5880,7 +6053,7 @@ To select another inbox, use option `box` with one of the following values:
|
|
5880
6053
|
|
5881
6054
|
### Faspex 5: Send a package
|
5882
6055
|
|
5883
|
-
The `Hash` creation
|
6056
|
+
The `Hash` creation **Command Parameter** provided to command `faspex5 packages send [extended value: Hash with package info ] [files...]` corresponds to the Faspex 5 API: `POST /packages`.
|
5884
6057
|
|
5885
6058
|
The interface is the one of the API (Refer to Faspex5 API documentation, or look at request in browser).
|
5886
6059
|
|
@@ -5931,7 +6104,7 @@ Basically, add the field `metadata`, with one key per metadata and the value is
|
|
5931
6104
|
|
5932
6105
|
### Faspex 5: List packages
|
5933
6106
|
|
5934
|
-
Option `box` can be used to list packages from a specific box (see
|
6107
|
+
Option `box` can be used to list packages from a specific box (see [Inbox Selection](#faspex-5-inbox-selection) above).
|
5935
6108
|
|
5936
6109
|
Option `query` can be used to filter the list of packages, based on native API parameters, directly sent to [Faspex 5 API `GET /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#getAllPackages).
|
5937
6110
|
|
@@ -5944,7 +6117,7 @@ Option `query` can be used to filter the list of packages, based on native API p
|
|
5944
6117
|
| `max` | Special | Maximum number of items to retrieve (stop pages when the maximum is passed) |
|
5945
6118
|
| `pmax` | Special | Maximum number of pages to request (stop pages when the maximum is passed) |
|
5946
6119
|
|
5947
|
-
A
|
6120
|
+
A **Command Parameter** in last position, of type `Proc`, can be used to filter the list of packages.
|
5948
6121
|
This advantage of this method is that the expression can be any test, even complex, as it is Ruby code.
|
5949
6122
|
But the disadvantage is that the filtering is done in `ascli` and not in Faspex 5, so it is less efficient.
|
5950
6123
|
|
@@ -5972,7 +6145,7 @@ Option `query` is available.
|
|
5972
6145
|
|
5973
6146
|
To list recursively add option `--query=@json:{"recursive":true}`.
|
5974
6147
|
|
5975
|
-
> **Note:** Option `
|
6148
|
+
> **Note:** Option `recursive` makes recursive API calls, so it can take a long time on large packages.
|
5976
6149
|
|
5977
6150
|
To limit the number of items retrieved, use option `--query=@json:{"max":10}`.
|
5978
6151
|
|
@@ -5982,7 +6155,7 @@ To receive one, or several packages at once, use command `faspex5 packages recei
|
|
5982
6155
|
Provide either a single package id, or an extended value `Array` of package ids, e.g. `@list:,1,2,3` as argument.
|
5983
6156
|
|
5984
6157
|
The same options as for `faspex5 packages list` can be used to select the box and filter the packages to download.
|
5985
|
-
I.e. options `box` and `query`, as well as last
|
6158
|
+
I.e. options `box` and `query`, as well as last **Command Parameter** `Proc` (filter).
|
5986
6159
|
|
5987
6160
|
Option `--once-only=yes` can be used, for "cargo-like" behavior.
|
5988
6161
|
Special package id `INIT` initializes the persistency of already received packages when option `--once-only=yes` is used.
|
@@ -6004,6 +6177,7 @@ ascli faspex5 admin res workgroup list
|
|
6004
6177
|
```
|
6005
6178
|
|
6006
6179
|
If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
|
6180
|
+
Example:
|
6007
6181
|
|
6008
6182
|
```bash
|
6009
6183
|
ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
|
@@ -6021,10 +6195,10 @@ It is equivalent to:
|
|
6021
6195
|
ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
6022
6196
|
```
|
6023
6197
|
|
6024
|
-
Other payload parameters are possible
|
6198
|
+
Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
|
6025
6199
|
|
6026
6200
|
```json
|
6027
|
-
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
|
6201
|
+
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3}
|
6028
6202
|
```
|
6029
6203
|
|
6030
6204
|
### Faspex 5: Create Metadata profile
|
@@ -6085,6 +6259,37 @@ Public invitations are for external users, provide just the email address.
|
|
6085
6259
|
|
6086
6260
|
Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
|
6087
6261
|
|
6262
|
+
### Faspex 5: Cleanup packages
|
6263
|
+
|
6264
|
+
> **Note:** Operation requires admin level.
|
6265
|
+
|
6266
|
+
Automated cleanup period can be displayed with:
|
6267
|
+
|
6268
|
+
```bash
|
6269
|
+
ascli faspex5 admin configuration show --fields=days_before_deleting_package_records
|
6270
|
+
```
|
6271
|
+
|
6272
|
+
This parameter can also be modified, for example:
|
6273
|
+
|
6274
|
+
```bash
|
6275
|
+
ascli faspex5 admin configuration modify @json:'{"days_before_deleting_package_records":30}'
|
6276
|
+
```
|
6277
|
+
|
6278
|
+
To start package purge, i.e. permanently remove packages marked for deletion older than `days_before_deleting_package_records`, use command:
|
6279
|
+
|
6280
|
+
```bash
|
6281
|
+
ascli faspex5 admin clean_deleted
|
6282
|
+
```
|
6283
|
+
|
6284
|
+
To delete all packages, one can use the following command:
|
6285
|
+
|
6286
|
+
```bash
|
6287
|
+
ascli faspex5 packages list --box=ALL --format=yaml --fields=id | ascli faspex5 packages delete @yaml:@stdin:
|
6288
|
+
```
|
6289
|
+
|
6290
|
+
> **Note:** Above command will mark all packages for deletion, and will be permanently removed after the configured period (`clean_deleted` command).
|
6291
|
+
> It is possible to add a filter to the list command to only delete packages matching some criteria, e.g. using `--select=@ruby:`.
|
6292
|
+
|
6088
6293
|
### Faspex 5: Faspex 4-style postprocessing
|
6089
6294
|
|
6090
6295
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
@@ -6210,13 +6415,13 @@ The contents of `delivery_info` is directly the contents of the `send` v3 [API o
|
|
6210
6415
|
Example:
|
6211
6416
|
|
6212
6417
|
```bash
|
6213
|
-
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}'
|
6418
|
+
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' /tmp/file1 /home/bar/file2
|
6214
6419
|
```
|
6215
6420
|
|
6216
6421
|
If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
|
6217
6422
|
`"recipients":["*MyDropboxName"]`
|
6218
6423
|
|
6219
|
-
Additional optional parameters in `delivery_info`:
|
6424
|
+
Additional optional parameters in mandatory option `delivery_info`:
|
6220
6425
|
|
6221
6426
|
- Package Note: : `"note":"note this and that"`
|
6222
6427
|
- Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
@@ -6233,7 +6438,7 @@ The value is a `Hash` with the following keys:
|
|
6233
6438
|
|
6234
6439
|
### Email notification on transfer
|
6235
6440
|
|
6236
|
-
Like for any transfer, a notification can be sent by email using
|
6441
|
+
Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template` .
|
6237
6442
|
|
6238
6443
|
Example:
|
6239
6444
|
|
@@ -6370,10 +6575,10 @@ admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
|
|
6370
6575
|
files browse /
|
6371
6576
|
files delete my_share1/test_file.bin
|
6372
6577
|
files download --to-folder=. my_share1/test_file.bin
|
6373
|
-
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
6374
|
-
files upload --to-folder=
|
6578
|
+
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6579
|
+
files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6375
6580
|
files upload --to-folder=my_share1 test_file.bin
|
6376
|
-
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
6581
|
+
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6377
6582
|
health
|
6378
6583
|
```
|
6379
6584
|
|
@@ -6527,11 +6732,44 @@ node info --log-level=trace2
|
|
6527
6732
|
node upload test_file.bin
|
6528
6733
|
```
|
6529
6734
|
|
6735
|
+
## Plugin: `httpgw`: HTTP Gateway
|
6736
|
+
|
6737
|
+
### Httpgw sample commands
|
6738
|
+
|
6739
|
+
> **Note:** Add `ascli httpgw` in front of the commands:
|
6740
|
+
|
6741
|
+
```bash
|
6742
|
+
health
|
6743
|
+
```
|
6744
|
+
|
6745
|
+
## Plugin: `faspio`: Faspio Gateway
|
6746
|
+
|
6747
|
+
### Faspio sample commands
|
6748
|
+
|
6749
|
+
> **Note:** Add `ascli faspio` in front of the commands:
|
6750
|
+
|
6751
|
+
```bash
|
6752
|
+
bridges list
|
6753
|
+
health
|
6754
|
+
```
|
6755
|
+
|
6756
|
+
## Plugin: `alee`: Aspera License Entitlement Engine
|
6757
|
+
|
6758
|
+
Retrieve information on subscription.
|
6759
|
+
|
6760
|
+
### Alee sample commands
|
6761
|
+
|
6762
|
+
> **Note:** Add `ascli alee` in front of the commands:
|
6763
|
+
|
6764
|
+
```bash
|
6765
|
+
health
|
6766
|
+
```
|
6767
|
+
|
6530
6768
|
## Plugin: `preview`: Preview generator for AoC
|
6531
6769
|
|
6532
6770
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
6533
6771
|
It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
6534
|
-
Several
|
6772
|
+
Several options can be used to tune several aspects:
|
6535
6773
|
|
6536
6774
|
- Methods for detection of new files needing generation
|
6537
6775
|
- Methods for generation of video preview
|
@@ -6558,10 +6796,11 @@ asnodeadmin --reload
|
|
6558
6796
|
|
6559
6797
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
6560
6798
|
|
6561
|
-
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024
|
6562
|
-
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384
|
6799
|
+
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 Bytes).
|
6800
|
+
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KB, i.e. 16MB.
|
6563
6801
|
|
6564
|
-
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
6802
|
+
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
6803
|
+
To display the value, use `asuserdata`:
|
6565
6804
|
|
6566
6805
|
```bash
|
6567
6806
|
asuserdata -a | grep max_request_file_create_size_kb
|
@@ -6642,7 +6881,7 @@ chmod a+x /usr/bin/unoconv
|
|
6642
6881
|
The preview generator should be executed as a non-user.
|
6643
6882
|
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
6644
6883
|
|
6645
|
-
Like any `ascli` commands,
|
6884
|
+
Like any `ascli` commands, options can be passed on command line or using a configuration [Option Preset](#option-preset).
|
6646
6885
|
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
6647
6886
|
|
6648
6887
|
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
@@ -6902,9 +7141,9 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
6902
7141
|
- `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
|
6903
7142
|
- `--src-base` (`src_base`) if top level folder name shall not be created on destination
|
6904
7143
|
|
6905
|
-
> **Note:** `ascli` takes transfer parameters exclusively as a [
|
7144
|
+
> **Note:** `ascli` takes transfer parameters exclusively as a [**transfer-spec**](#transfer-specification), with `ts` option.
|
6906
7145
|
>
|
6907
|
-
> **Note:** Most, but not all, native `ascp` arguments are available as standard [
|
7146
|
+
> **Note:** Most, but not all, native `ascp` arguments are available as standard [**transfer-spec**](#transfer-specification) parameters.
|
6908
7147
|
>
|
6909
7148
|
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
6910
7149
|
|
@@ -6914,11 +7153,11 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
6914
7153
|
|
6915
7154
|
> **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
6916
7155
|
>
|
6917
|
-
> **Note:**
|
7156
|
+
> **Note:** Options may be saved in a [Option Preset](#option-preset) and used with `-P`.
|
6918
7157
|
|
6919
7158
|
#### Scheduling
|
6920
7159
|
|
6921
|
-
Once `ascli`
|
7160
|
+
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
6922
7161
|
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
6923
7162
|
|
6924
7163
|
### Example: Upload hot folder
|
@@ -7081,7 +7320,7 @@ Transfer is: <%=global_transfer_status%>
|
|
7081
7320
|
|
7082
7321
|
This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
|
7083
7322
|
|
7084
|
-
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [
|
7323
|
+
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [**transfer-spec**](#transfer-specification) is:
|
7085
7324
|
|
7086
7325
|
- Common to Aspera Node API (HTTP POST /ops/transfer)
|
7087
7326
|
- Common to Aspera Connect API (browser javascript startTransfer)
|
@@ -7091,7 +7330,7 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
|
|
7091
7330
|
|
7092
7331
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
7093
7332
|
|
7094
|
-
`ascli` expect one single argument: a session specification that contains parameters and a [
|
7333
|
+
`ascli` expect one single argument: a session specification that contains parameters and a [**transfer-spec**](#transfer-specification).
|
7095
7334
|
|
7096
7335
|
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted on stdin.
|
7097
7336
|
|
@@ -7103,7 +7342,7 @@ Top level parameters supported by `asession`:
|
|
7103
7342
|
|
7104
7343
|
| parameter | description |
|
7105
7344
|
|-----------|-------------|
|
7106
|
-
| `spec` | the [
|
7345
|
+
| `spec` | the [**transfer-spec**](#transfer-specification) |
|
7107
7346
|
| `agent` | same parameters as transfer-info for agent `direct` |
|
7108
7347
|
| `loglevel` | log level of `asession` |
|
7109
7348
|
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
|
@@ -7136,7 +7375,7 @@ asession < session.json
|
|
7136
7375
|
|
7137
7376
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` -> `long_parameter`
|
7138
7377
|
|
7139
|
-
This is particularly useful for a persistent session ( with the [
|
7378
|
+
This is particularly useful for a persistent session ( with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true` )
|
7140
7379
|
|
7141
7380
|
```json
|
7142
7381
|
asession
|
@@ -7205,7 +7444,7 @@ So, it evolved into `ascli`:
|
|
7205
7444
|
|
7206
7445
|
- Portable: works on platforms supporting `ruby` (and `ascp`)
|
7207
7446
|
- Easy to install with the `gem` utility
|
7208
|
-
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [
|
7447
|
+
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable , more standard)
|
7209
7448
|
- `ruby` is consistent with other Aspera products
|
7210
7449
|
|
7211
7450
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
@@ -7215,7 +7454,7 @@ Enjoy a coffee on me:
|
|
7215
7454
|
|
7216
7455
|
```bash
|
7217
7456
|
ascli config coffee --ui=text
|
7218
|
-
ascli config coffee --ui=text --
|
7457
|
+
ascli config coffee --ui=text --image=@json:'{"text":true}'
|
7219
7458
|
ascli config coffee
|
7220
7459
|
```
|
7221
7460
|
|
@@ -7244,11 +7483,13 @@ This also requires Ruby header files.
|
|
7244
7483
|
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
7245
7484
|
`ruby-dev` or `ruby-devel`, depending on distribution.
|
7246
7485
|
|
7247
|
-
###
|
7486
|
+
### ED25519 key not supported
|
7248
7487
|
|
7249
7488
|
ED25519 keys are deactivated since `ascli` version 0.9.24 as it requires additional gems that require native compilation and thus caused problems.
|
7250
7489
|
This type of key will just be ignored.
|
7251
7490
|
|
7491
|
+
To re-activate, set env var `ASCLI_ENABLE_ED25519` to `true`.
|
7492
|
+
|
7252
7493
|
Without this deactivation, if such key was present in user's `.ssh` folder then the following error was generated:
|
7253
7494
|
|
7254
7495
|
```output
|
@@ -7258,4 +7499,5 @@ OpenSSH keys only supported if ED25519 is available
|
|
7258
7499
|
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
7259
7500
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
7260
7501
|
|
7261
|
-
|
7502
|
+
In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also de-activated by default.
|
7503
|
+
To re-activate, set env var `ASCLI_ENABLE_ECDSHA2` to `true`.
|