aspera-cli 4.17.0 → 4.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +2 -4
- data/CHANGELOG.md +23 -0
- data/CONTRIBUTING.md +15 -1
- data/README.md +620 -378
- data/bin/ascli +5 -0
- data/bin/asession +2 -2
- data/lib/aspera/agent/alpha.rb +6 -4
- data/lib/aspera/agent/base.rb +9 -6
- data/lib/aspera/agent/connect.rb +4 -4
- data/lib/aspera/agent/direct.rb +56 -37
- data/lib/aspera/agent/httpgw.rb +23 -324
- data/lib/aspera/agent/node.rb +19 -20
- data/lib/aspera/agent/trsdk.rb +19 -20
- data/lib/aspera/api/aoc.rb +17 -14
- data/lib/aspera/api/cos_node.rb +4 -4
- data/lib/aspera/api/httpgw.rb +339 -0
- data/lib/aspera/api/node.rb +34 -21
- data/lib/aspera/ascmd.rb +4 -3
- data/lib/aspera/ascp/installation.rb +15 -7
- data/lib/aspera/ascp/management.rb +2 -2
- data/lib/aspera/cli/basic_auth_plugin.rb +5 -9
- data/lib/aspera/cli/extended_value.rb +12 -6
- data/lib/aspera/cli/formatter.rb +155 -65
- data/lib/aspera/cli/hints.rb +18 -0
- data/lib/aspera/cli/main.rb +22 -29
- data/lib/aspera/cli/manager.rb +53 -36
- data/lib/aspera/cli/plugin.rb +26 -17
- data/lib/aspera/cli/plugin_factory.rb +31 -20
- data/lib/aspera/cli/plugins/alee.rb +14 -2
- data/lib/aspera/cli/plugins/aoc.rb +141 -131
- data/lib/aspera/cli/plugins/ats.rb +1 -1
- data/lib/aspera/cli/plugins/config.rb +52 -46
- data/lib/aspera/cli/plugins/console.rb +8 -5
- data/lib/aspera/cli/plugins/faspex.rb +27 -19
- data/lib/aspera/cli/plugins/faspex5.rb +222 -149
- data/lib/aspera/cli/plugins/faspio.rb +85 -0
- data/lib/aspera/cli/plugins/httpgw.rb +55 -0
- data/lib/aspera/cli/plugins/node.rb +86 -29
- data/lib/aspera/cli/plugins/orchestrator.rb +31 -29
- data/lib/aspera/cli/plugins/preview.rb +6 -2
- data/lib/aspera/cli/plugins/server.rb +5 -5
- data/lib/aspera/cli/plugins/shares.rb +16 -14
- data/lib/aspera/cli/sync_actions.rb +6 -6
- data/lib/aspera/cli/transfer_agent.rb +5 -4
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/environment.rb +7 -6
- data/lib/aspera/faspex_gw.rb +5 -4
- data/lib/aspera/faspex_postproc.rb +2 -2
- data/lib/aspera/log.rb +6 -3
- data/lib/aspera/node_simulator.rb +2 -2
- data/lib/aspera/oauth/base.rb +31 -19
- data/lib/aspera/oauth/factory.rb +12 -13
- data/lib/aspera/oauth/generic.rb +1 -0
- data/lib/aspera/oauth/jwt.rb +18 -15
- data/lib/aspera/oauth/url_json.rb +8 -6
- data/lib/aspera/open_application.rb +5 -7
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/generator.rb +3 -3
- data/lib/aspera/preview/options.rb +3 -3
- data/lib/aspera/preview/terminal.rb +4 -4
- data/lib/aspera/preview/utils.rb +3 -3
- data/lib/aspera/proxy_auto_config.rb +5 -1
- data/lib/aspera/rest.rb +60 -74
- data/lib/aspera/rest_call_error.rb +1 -1
- data/lib/aspera/rest_error_analyzer.rb +2 -2
- data/lib/aspera/rest_errors_aspera.rb +1 -1
- data/lib/aspera/resumer.rb +1 -1
- data/lib/aspera/secret_hider.rb +2 -4
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/transfer/parameters.rb +39 -36
- data/lib/aspera/transfer/spec.rb +2 -0
- data/lib/aspera/transfer/sync.rb +2 -1
- data/lib/aspera/transfer/uri.rb +1 -1
- data/lib/aspera/uri_reader.rb +5 -4
- data/lib/aspera/web_auth.rb +1 -1
- data/lib/aspera/web_server_simple.rb +4 -3
- data.tar.gz.sig +0 -0
- metadata +5 -3
- metadata.gz.sig +0 -0
- data/lib/aspera/cli/plugins/bss.rb +0 -71
data/README.md
CHANGED
@@ -10,7 +10,7 @@
|
|
10
10
|
|
11
11
|
## Introduction
|
12
12
|
|
13
|
-
Version : 4.
|
13
|
+
Version : 4.18.0
|
14
14
|
|
15
15
|
Laurent/2016-2024
|
16
16
|
|
@@ -70,7 +70,7 @@ For scripting and ad'hoc command line operations, `ascli` is perfect.
|
|
70
70
|
|
71
71
|
Command line operations examples are shown using a shell such as: `bash` or `zsh`.
|
72
72
|
|
73
|
-
Command line
|
73
|
+
Command line arguments beginning with `my_` in examples, e.g. `my_param_value`, are user-provided value and not fixed value commands.
|
74
74
|
|
75
75
|
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
|
76
76
|
|
@@ -92,7 +92,7 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
92
92
|
|
93
93
|
```console
|
94
94
|
$ ascli --version
|
95
|
-
4.
|
95
|
+
4.18.0
|
96
96
|
```
|
97
97
|
|
98
98
|
### First use
|
@@ -122,10 +122,10 @@ ascli server browse /
|
|
122
122
|
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
123
123
|
```
|
124
124
|
|
125
|
-
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset)
|
125
|
+
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset) for the server's authentication options.
|
126
126
|
The following example will:
|
127
127
|
|
128
|
-
- Create a [Option Preset](#option-preset)
|
128
|
+
- Create a [Option Preset](#option-preset)
|
129
129
|
- Define it as default for the `server` plugin
|
130
130
|
- List files in a folder
|
131
131
|
- Download a file
|
@@ -188,8 +188,8 @@ It is possible to install **either** directly on the host operating system (Linu
|
|
188
188
|
The direct installation is recommended and consists in installing:
|
189
189
|
|
190
190
|
- [Ruby](#ruby)
|
191
|
-
- [aspera-cli](#ruby-gem)
|
192
|
-
- [Aspera SDK (`ascp`)](#fasp-protocol)
|
191
|
+
- [aspera-cli](#ruby-gem-aspera-cli) <!-- markdownlint-disable-line -->
|
192
|
+
- [Aspera SDK (`ascp`)](#fasp-protocol-ascp)
|
193
193
|
|
194
194
|
Ruby version: >= 2.6.
|
195
195
|
|
@@ -253,7 +253,7 @@ Install the chosen pre-compiled Ruby version:
|
|
253
253
|
rvm install 3.2.2
|
254
254
|
```
|
255
255
|
|
256
|
-
Ruby is now installed for the user, go to [Gem installation](#ruby-gem).
|
256
|
+
Ruby is now installed for the user, go to [Gem installation](#ruby-gem-aspera-cli). <!-- markdownlint-disable-line -->
|
257
257
|
|
258
258
|
Alternatively RVM can be installed system-wide, for this execute as `root`.
|
259
259
|
It then installs by default in `/usr/local/rvm` for all users and creates `/etc/profile.d/rvm.sh`.
|
@@ -388,6 +388,12 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
|
|
388
388
|
dnf install -y ruby-devel
|
389
389
|
```
|
390
390
|
|
391
|
+
**Example:** Ubuntu
|
392
|
+
|
393
|
+
```bash
|
394
|
+
apt-get install -y ruby-full
|
395
|
+
```
|
396
|
+
|
391
397
|
**Other examples:**
|
392
398
|
|
393
399
|
```bash
|
@@ -479,7 +485,7 @@ gem install rmagick grpc mimemagic
|
|
479
485
|
|
480
486
|
> **Note:** Those are not installed as part of dependencies because they involve compilation of native code.
|
481
487
|
|
482
|
-
### Ruby Gem
|
488
|
+
### Ruby Gem: `aspera-cli`
|
483
489
|
|
484
490
|
Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
|
485
491
|
|
@@ -502,7 +508,26 @@ To check if a new version is available (independently of `version_check_days`):
|
|
502
508
|
ascli config check_update
|
503
509
|
```
|
504
510
|
|
505
|
-
|
511
|
+
#### Gem installation with signature verification
|
512
|
+
|
513
|
+
The gem is signed with a private key, and the public certificate is available in the github repository (`certs/aspera-cli-public-cert.pem`).
|
514
|
+
When installing the gem, the signature can be optionally verified.
|
515
|
+
|
516
|
+
For [secure installation](https://guides.rubygems.org/security/#using-gems), one can install the gem with the public key:
|
517
|
+
|
518
|
+
- import the verification certificate:
|
519
|
+
|
520
|
+
```bash
|
521
|
+
gem cert --add <(curl -Ls https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem)
|
522
|
+
```
|
523
|
+
|
524
|
+
- Then, install the gem:
|
525
|
+
|
526
|
+
```bash
|
527
|
+
gem install -P MediumSecurity aspera-cli
|
528
|
+
```
|
529
|
+
|
530
|
+
### FASP Protocol: `ascp`
|
506
531
|
|
507
532
|
Most file transfers will be executed using the **FASP** protocol, using `ascp`.
|
508
533
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
@@ -601,7 +626,7 @@ gem install aspera-cli -N -i my_gems
|
|
601
626
|
|
602
627
|
### Container
|
603
628
|
|
604
|
-
The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
|
629
|
+
The container image is: [`docker.io/martinlaurent/ascli`](https://hub.docker.com/r/docker.io/martinlaurent/ascli).
|
605
630
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
606
631
|
To use the container, ensure that you have `podman` (or `docker`) installed.
|
607
632
|
|
@@ -615,7 +640,7 @@ podman --version
|
|
615
640
|
Execute this:
|
616
641
|
|
617
642
|
```bash
|
618
|
-
podman run --rm --tty --interactive --entrypoint bash martinlaurent/ascli:latest
|
643
|
+
podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/ascli:latest
|
619
644
|
```
|
620
645
|
|
621
646
|
> **Note:** This command changes the entrypoint to an interactive shell instead of direct execution of `ascli`.
|
@@ -643,13 +668,13 @@ The entry point is `ascli` and the default command is `help`.
|
|
643
668
|
The container can be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
644
669
|
|
645
670
|
```bash
|
646
|
-
podman run --rm --tty --interactive martinlaurent/ascli:latest
|
671
|
+
podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest
|
647
672
|
```
|
648
673
|
|
649
674
|
For more convenience, you may define a shell alias:
|
650
675
|
|
651
676
|
```bash
|
652
|
-
alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
|
677
|
+
alias ascli='podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest'
|
653
678
|
```
|
654
679
|
|
655
680
|
Then, you can execute the container like a local command:
|
@@ -659,7 +684,7 @@ ascli -v
|
|
659
684
|
```
|
660
685
|
|
661
686
|
```text
|
662
|
-
4.
|
687
|
+
4.18.0
|
663
688
|
```
|
664
689
|
|
665
690
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
@@ -686,12 +711,12 @@ In this case you need also to specify the shared transfer folder as a volume:
|
|
686
711
|
--volume $HOME/xferdir:/xferfiles
|
687
712
|
```
|
688
713
|
|
689
|
-
> **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
714
|
+
> **Note:** `ascli` is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
690
715
|
|
691
716
|
And if you want all the above, simply use all the options:
|
692
717
|
|
693
718
|
```bash
|
694
|
-
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
|
719
|
+
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash docker.io/martinlaurent/ascli:latest"
|
695
720
|
```
|
696
721
|
|
697
722
|
```bash
|
@@ -714,7 +739,7 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
714
739
|
|----------------|------------------------------------|--------------------------|--------------------------|
|
715
740
|
| `ASCLI_HOME` | Configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
716
741
|
| `docker_args` | Additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
717
|
-
| `image` | Container image name | `martinlaurent/ascli` | |
|
742
|
+
| `image` | Container image name | `docker.io/martinlaurent/ascli` | |
|
718
743
|
| `version` | Container image version | Latest | `4.8.0.pre` |
|
719
744
|
|
720
745
|
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
@@ -748,8 +773,8 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
748
773
|
- First create the image archive:
|
749
774
|
|
750
775
|
```bash
|
751
|
-
podman pull martinlaurent/ascli
|
752
|
-
podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
776
|
+
podman pull docker.io/martinlaurent/ascli
|
777
|
+
podman save docker.io/martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
753
778
|
```
|
754
779
|
|
755
780
|
- Then, on air-gapped system:
|
@@ -787,7 +812,7 @@ dnf install singularity-ce
|
|
787
812
|
Build an image like this:
|
788
813
|
|
789
814
|
```bash
|
790
|
-
singularity build ascli.sif docker://martinlaurent/ascli
|
815
|
+
singularity build ascli.sif docker://docker.io/martinlaurent/ascli
|
791
816
|
```
|
792
817
|
|
793
818
|
Then, start `ascli` like this:
|
@@ -802,6 +827,124 @@ Or get a shell with access to `ascli` like this:
|
|
802
827
|
singularity shell ascli.sif
|
803
828
|
```
|
804
829
|
|
830
|
+
### SSL library
|
831
|
+
|
832
|
+
`ascli` uses the Ruby `openssl` gem which uses by default the system's `openssl` library and its CA certificate bundle.
|
833
|
+
|
834
|
+
To display the version of **openssl** used in `ascli`:
|
835
|
+
|
836
|
+
```bash
|
837
|
+
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
838
|
+
```
|
839
|
+
|
840
|
+
It is possible to specify to use another SSL library or version by executing:
|
841
|
+
|
842
|
+
```bash
|
843
|
+
gem install openssl -- --with-openssl-dir=[openssl library folder]
|
844
|
+
```
|
845
|
+
|
846
|
+
Where `[openssl library folder]` is the path to the folder containing the `lib` and `include` folders of the `openssl` library.
|
847
|
+
|
848
|
+
For example, on macOS, to use the `openssl@3` library installed with `brew`:
|
849
|
+
|
850
|
+
```bash
|
851
|
+
$ openssl version -e|sed -n 's|ENGINESDIR: "\(.*\)/lib[^/]*/.*|\1|p'
|
852
|
+
/opt/homebrew/Cellar/openssl@3/3.3.0
|
853
|
+
$ gem install openssl -- --with-openssl-dir=/opt/homebrew/Cellar/openssl@3/3.3.0
|
854
|
+
```
|
855
|
+
|
856
|
+
### SSL CA certificate bundle
|
857
|
+
|
858
|
+
SSL certificates are validated using a certificate store, by default it is the one of the system's `openssl` library.
|
859
|
+
|
860
|
+
To display trusted certificate store locations:
|
861
|
+
|
862
|
+
```bash
|
863
|
+
ascli --show-config --fields=cert_stores
|
864
|
+
```
|
865
|
+
|
866
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
867
|
+
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
868
|
+
|
869
|
+
One can display those default values:
|
870
|
+
|
871
|
+
```bash
|
872
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
873
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
874
|
+
```
|
875
|
+
|
876
|
+
In order to get certificate validation, the CA certificate bundle must be up-to-date.
|
877
|
+
Check this repository on how to update the system's CA certificate bundle: [https://github.com/millermatt/osca](https://github.com/millermatt/osca).
|
878
|
+
|
879
|
+
For example on RHEL/Rocky Linux:
|
880
|
+
|
881
|
+
```bash
|
882
|
+
dnf install -y ca-certificates
|
883
|
+
update-ca-trust extract
|
884
|
+
```
|
885
|
+
|
886
|
+
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders.
|
887
|
+
By default it uses Ruby's default certificate store.
|
888
|
+
|
889
|
+
If you use this option, then default locations are not used.
|
890
|
+
Default locations can be added using special value `DEF`.
|
891
|
+
The value can be either an `Array` or `String` (path).
|
892
|
+
Successive options add paths incrementally.
|
893
|
+
All files of a folders are added.
|
894
|
+
|
895
|
+
JRuby uses its own implementation and CA bundles.
|
896
|
+
|
897
|
+
For example, on Linux to force the use the system's certificate store:
|
898
|
+
|
899
|
+
```bash
|
900
|
+
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
901
|
+
```
|
902
|
+
|
903
|
+
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
|
904
|
+
|
905
|
+
By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
|
906
|
+
Original `ascp`'s hardcoded locations can be found using:
|
907
|
+
|
908
|
+
```bash
|
909
|
+
ascli config ascp info --fields=openssldir
|
910
|
+
```
|
911
|
+
|
912
|
+
E.g. on macOS: `/Library/Aspera/ssl`.
|
913
|
+
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
914
|
+
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
915
|
+
|
916
|
+
To update trusted root certificates for `ascli`:
|
917
|
+
Display the trusted certificate store locations used by `ascli`.
|
918
|
+
Typically done by updating the system's root certificate store.
|
919
|
+
|
920
|
+
An up-to-date version of the certificate bundle can also be retrieved with:
|
921
|
+
|
922
|
+
```bash
|
923
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
924
|
+
```
|
925
|
+
|
926
|
+
To download that certificate store:
|
927
|
+
|
928
|
+
```bash
|
929
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
|
930
|
+
```
|
931
|
+
|
932
|
+
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
933
|
+
|
934
|
+
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
935
|
+
|
936
|
+
```bash
|
937
|
+
ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
|
938
|
+
```
|
939
|
+
|
940
|
+
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
941
|
+
|
942
|
+
Then, use this file as certificate store (e.g. here, Node API):
|
943
|
+
|
944
|
+
```bash
|
945
|
+
ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
|
946
|
+
```
|
947
|
+
|
805
948
|
## Command Line Interface
|
806
949
|
|
807
950
|
The command line tool is: ``ascli``
|
@@ -820,9 +963,9 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
|
|
820
963
|
|
821
964
|
- Supports commands to Aspera server products (on-premise and SaaS)
|
822
965
|
- Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
|
823
|
-
- Supports Commands, Option values
|
966
|
+
- Supports Commands, Options, and Option values shortcuts
|
824
967
|
- FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
|
825
|
-
- Transfer parameters can be altered by modification of [
|
968
|
+
- Transfer parameters can be altered by modification of [**transfer-spec**](#transfer-specification), this includes requiring multi-session
|
826
969
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
827
970
|
- Supports FaspStream creation (using Node API)
|
828
971
|
- Supports **Watchfolder** creation (using Node API)
|
@@ -856,19 +999,20 @@ Moreover all `ascp` options are supported either through transfer spec parameter
|
|
856
999
|
### Command line parsing, Special Characters
|
857
1000
|
|
858
1001
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
859
|
-
`ascli` receives its arguments
|
1002
|
+
`ascli` receives its arguments on the command line.
|
1003
|
+
The way arguments are parsed and provided to `ascli` depend on the Operating System and shell.
|
860
1004
|
|
861
1005
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
862
1006
|
|
863
|
-
Linux command line parsing is
|
1007
|
+
Linux command line parsing is well defined:
|
864
1008
|
It is fully documented in the shell's documentation.
|
865
1009
|
|
866
|
-
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
1010
|
+
On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh).
|
867
1011
|
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
868
1012
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
869
1013
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
870
1014
|
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
871
|
-
Ruby receives a list
|
1015
|
+
Ruby receives a list command line arguments from shell and gives it to `ascli`.
|
872
1016
|
Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
|
873
1017
|
|
874
1018
|
#### Shell parsing for Windows
|
@@ -960,16 +1104,16 @@ Major Minor Build Revision
|
|
960
1104
|
----- ----- ----- --------
|
961
1105
|
5 1 19041 4046
|
962
1106
|
|
963
|
-
PS C:\> ascli
|
1107
|
+
PS C:\> ascli config echo --% @json:'{"k":"v","x":"y"}'
|
964
1108
|
|
965
|
-
PS C:\> ascli
|
1109
|
+
PS C:\> ascli config echo @json:'{"""k""":"""v""","""x""":"""y"""}'
|
966
1110
|
```
|
967
1111
|
|
968
1112
|
> **Note:** The special powershell argument `--%` places powershell in legacy parsing mode.
|
969
1113
|
|
970
1114
|
#### Extended Values (JSON, Ruby, ...)
|
971
1115
|
|
972
|
-
Some of the `ascli`
|
1116
|
+
Some of the values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
973
1117
|
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
974
1118
|
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
975
1119
|
|
@@ -1154,16 +1298,15 @@ ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
1154
1298
|
{"title":"Test \" ' & \\"}
|
1155
1299
|
```
|
1156
1300
|
|
1157
|
-
###
|
1301
|
+
### Positional Arguments and Options
|
1158
1302
|
|
1159
1303
|
Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
|
1160
1304
|
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#command-line-parsing-special-characters).
|
1161
1305
|
|
1162
|
-
`ascli` handles
|
1306
|
+
`ascli` handles two types of command line arguments:
|
1163
1307
|
|
1164
|
-
-
|
1165
|
-
-
|
1166
|
-
- Options
|
1308
|
+
- **Positional Arguments** : position is significant
|
1309
|
+
- **Options** : only order is significant, but not position
|
1167
1310
|
|
1168
1311
|
For example:
|
1169
1312
|
|
@@ -1171,18 +1314,24 @@ For example:
|
|
1171
1314
|
ascli command subcommand --option-name=VAL1 VAL2
|
1172
1315
|
```
|
1173
1316
|
|
1174
|
-
- Executes **
|
1175
|
-
- With one **
|
1176
|
-
- The command has one additional **positional argument**: `VAL2`
|
1317
|
+
- Executes **Command** and its **Command Parameters** (**Positional Arguments**): `command subcommand VAL2`
|
1318
|
+
- With one **Option**: `option_name` and its **value**: `VAL1`
|
1177
1319
|
|
1178
1320
|
If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
|
1179
1321
|
For example `ascli config pre ov` is the same as `ascli config preset overview`.
|
1180
1322
|
|
1181
|
-
The value of
|
1323
|
+
The value of **Options** and **Positional Arguments** is evaluated with the [Extended Value Syntax](#extended-value-syntax).
|
1182
1324
|
|
1183
|
-
####
|
1325
|
+
#### Positional Arguments
|
1184
1326
|
|
1185
|
-
|
1327
|
+
**Positional Arguments** are either:
|
1328
|
+
|
1329
|
+
- **Commands**, typically at the beginning
|
1330
|
+
- **Command Parameters** , e.g. creation data or entity identifier
|
1331
|
+
|
1332
|
+
When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
|
1333
|
+
|
1334
|
+
**Commands** are typically entity types or verbs to act on those entities.
|
1186
1335
|
Its value is a `String` that must belong to a fixed list of values in a given context.
|
1187
1336
|
|
1188
1337
|
Example:
|
@@ -1196,27 +1345,24 @@ ascli config ascp info
|
|
1196
1345
|
- `ascp` is the second level command: name of the component (singleton)
|
1197
1346
|
- `info` is the third level command: action to be performed
|
1198
1347
|
|
1199
|
-
Typically,
|
1348
|
+
Typically, **Commands** are located at the **beginning** of the command line.
|
1200
1349
|
Order is significant.
|
1201
1350
|
The provided command must match one of the supported commands in the given context.
|
1202
1351
|
If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
1203
1352
|
|
1204
|
-
Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
1205
|
-
|
1206
|
-
#### Positional Arguments
|
1207
|
-
|
1208
|
-
Positional Arguments are typically mandatory values for a command, such as entity creation data.
|
1209
|
-
|
1210
|
-
It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
|
1353
|
+
Some sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
1211
1354
|
|
1212
|
-
|
1355
|
+
**Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
|
1213
1356
|
|
1357
|
+
> **Note:** It could also have been designed as an option.
|
1358
|
+
But since it is mandatory and typically these data need **not** be set in a configuration file, then it is better designed as a **Command Parameter**, and not designed an additional specific option.
|
1359
|
+
The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position) and the value is clearly mandatory.
|
1214
1360
|
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
|
1215
|
-
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@
|
1361
|
+
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
|
1216
1362
|
|
1217
|
-
If a
|
1363
|
+
If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
1218
1364
|
|
1219
|
-
A few
|
1365
|
+
A few **Command Parameters** are optional, they are located at the end of the command line.
|
1220
1366
|
|
1221
1367
|
#### Options
|
1222
1368
|
|
@@ -1225,13 +1371,15 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
|
|
1225
1371
|
- Start with `--`
|
1226
1372
|
- Have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
|
1227
1373
|
- Have a value, separated from name with a `=`
|
1228
|
-
- Can be used by prefix (
|
1374
|
+
- Can be used by prefix (not recommended), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
|
1375
|
+
- Is optional on command line (it has a default value or no value)
|
1376
|
+
- whose position is not significant, but order is significant
|
1229
1377
|
|
1230
1378
|
Exceptions:
|
1231
1379
|
|
1232
1380
|
- Some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
|
1233
1381
|
- Some options (flags) don't take a value, e.g. `-N`
|
1234
|
-
- The special option `--` stops option processing and is ignored, following command line arguments are taken as
|
1382
|
+
- The special option `--` stops option processing and is ignored, following command line arguments are taken as **Positional Arguments**, including the ones starting with a `-`.
|
1235
1383
|
|
1236
1384
|
Example:
|
1237
1385
|
|
@@ -1248,6 +1396,7 @@ ascli config echo -- --sample
|
|
1248
1396
|
Options may have a (hardcoded) default value.
|
1249
1397
|
|
1250
1398
|
Options can be placed anywhere on command line and are evaluated in order.
|
1399
|
+
Usually the last value evaluated overrides previous values, but some options are cumulative, e.g. `ts`.
|
1251
1400
|
|
1252
1401
|
Options are typically optional: to change the default behavior.
|
1253
1402
|
But some are mandatory, so they can be placed in a configuration file, for example: connection information.
|
@@ -1264,23 +1413,23 @@ Option `show_config` dry runs the configuration, and then returns currently set
|
|
1264
1413
|
`ascli --show-config` outputs global options only, and `ascli [plugin] --show-config` outputs global and plugin default options.
|
1265
1414
|
In addition, option `--show-config` can be added at the end of any full command line, this displays the options that would be used for the command.
|
1266
1415
|
|
1267
|
-
A
|
1416
|
+
A command line argument is typically designed as option if:
|
1268
1417
|
|
1269
1418
|
- It is optional, or
|
1270
|
-
- It is a mandatory parameter that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
1419
|
+
- It is a mandatory parameter with a default value that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
1271
1420
|
|
1272
1421
|
### Interactive Input
|
1273
1422
|
|
1274
|
-
Some options and
|
1275
|
-
By default, `ascli` will ask for missing mandatory options or
|
1423
|
+
Some options and **Command Parameters** are mandatory and other optional.
|
1424
|
+
By default, `ascli` will ask for missing mandatory options or **Command Parameters** for interactive execution.
|
1276
1425
|
|
1277
1426
|
The behavior can be controlled with:
|
1278
1427
|
|
1279
1428
|
- `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
|
1280
|
-
- yes : missing mandatory parameters/
|
1281
|
-
- no : missing mandatory parameters/
|
1429
|
+
- yes : missing mandatory parameters/arguments are asked to the user
|
1430
|
+
- no : missing mandatory parameters/arguments raise an error message
|
1282
1431
|
- `--ask-options=<yes|no>` (default=no)
|
1283
|
-
- optional parameters/
|
1432
|
+
- optional parameters/arguments are asked to user
|
1284
1433
|
|
1285
1434
|
### Output
|
1286
1435
|
|
@@ -1303,25 +1452,36 @@ Depending on action, the output will contain:
|
|
1303
1452
|
#### Format of output
|
1304
1453
|
|
1305
1454
|
By default, result of type single_object and object_list are displayed using format `table`.
|
1306
|
-
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
1307
1455
|
|
1308
|
-
|
1309
|
-
|
1310
|
-
|
1311
|
-
When in flatten mode, it is possible to filter fields by compound field name using dot as separator.
|
1456
|
+
The table style can be customized with option: `table_style` which expects a `Hash`, options are the ones described in gem [`terminal-table`](https://github.com/tj/terminal-table).
|
1457
|
+
|
1458
|
+
For example, to display a table with thick unicode borders:
|
1312
1459
|
|
1313
|
-
|
1460
|
+
```bash
|
1461
|
+
ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
|
1462
|
+
```
|
1463
|
+
|
1464
|
+
> **Note:** Other border styles exist, not limited to: `:unicode`, `:unicode_round`.
|
1465
|
+
|
1466
|
+
In a table format, when displaying **Objects** (single, or list), by default, sub object are flattened (option `flat_hash`).
|
1467
|
+
For example, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
|
1468
|
+
Setting option `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
|
1469
|
+
When in flatten mode, it is possible to filter fields using the option `fields` using the compound field name using `.` (dot) as separator.
|
1470
|
+
|
1471
|
+
Object lists are displayed one per line, with attributes as columns.
|
1472
|
+
Single objects (or tables with a single result) are transposed: one attribute per line.
|
1314
1473
|
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
1315
1474
|
|
1316
|
-
The style of output can be set using the `format`
|
1475
|
+
The style of output can be set using the `format` option, supporting:
|
1317
1476
|
|
1477
|
+
- `table` : Text table (default)
|
1318
1478
|
- `text` : Value as String
|
1319
|
-
- `table` : Text table
|
1320
1479
|
- `ruby` : Ruby code
|
1321
1480
|
- `json` : JSON code
|
1322
1481
|
- `jsonpp` : JSON pretty printed
|
1323
1482
|
- `yaml` : YAML
|
1324
1483
|
- `csv` : Comma Separated Values
|
1484
|
+
- `image` : Image URL or Image data
|
1325
1485
|
|
1326
1486
|
#### Verbosity of output
|
1327
1487
|
|
@@ -1342,31 +1502,31 @@ To hide secrets from output, set option `show_secrets` to `no`.
|
|
1342
1502
|
|
1343
1503
|
#### Option: `fields`: Selection of output object properties
|
1344
1504
|
|
1345
|
-
By default, a table output will display one line per entry, and columns for each properties.
|
1346
|
-
Depending on the command, columns may include by default all
|
1505
|
+
By default, a `table` output format will display one line per entry, and columns for each fields (attributes, properties).
|
1506
|
+
Depending on the command, columns may include by default all fields, or only some selected fields.
|
1347
1507
|
It is possible to define specific columns to be displayed, by setting the `fields` option.
|
1348
1508
|
|
1349
|
-
The `fields` option can be either a comma separated list
|
1509
|
+
The `fields` option is a list that can be either a comma separated list or an extended value `Array`.
|
1350
1510
|
|
1351
|
-
|
1511
|
+
Individual elements of the list can be:
|
1352
1512
|
|
1353
|
-
- `DEF` : default display of columns (that's the default, when not set)
|
1354
|
-
- `ALL` : all columns available
|
1355
|
-
- `-`**property** : remove property from the current list
|
1356
1513
|
- **property** : add property to the current list
|
1357
|
-
-
|
1514
|
+
- `-`**property** : remove property from the current list
|
1515
|
+
- `DEF` : default list of properties (that's the default, when not set)
|
1516
|
+
- `ALL` : all properties
|
1517
|
+
- A Ruby `RegEx` : using `@ruby:'/.../'`, or `@re:...` add those matching to the list
|
1358
1518
|
|
1359
1519
|
Examples:
|
1360
1520
|
|
1361
|
-
- `a,b,c` : the list of attributes specified as a comma separated list
|
1362
|
-
- `@json:'["a","b","c"]'` : Array extended value: same as above
|
1363
|
-
- `DEF,-a
|
1521
|
+
- `a,b,c` : the list of attributes specified as a comma separated list (overrides the all default)
|
1522
|
+
- `@json:'["a","b","c"]'` : `Array` extended value: same as above
|
1523
|
+
- `b,DEF,-a` : default property list, remove `a` and add `b` in first position
|
1364
1524
|
- `@ruby:'/^server/'` : Display all properties whose name begin with `server`
|
1365
1525
|
|
1366
1526
|
#### Option: `select`
|
1367
1527
|
|
1368
1528
|
Table output (`object_list`) can be filtered using option `select`.
|
1369
|
-
This
|
1529
|
+
This option is either a `Hash` or `Proc`.
|
1370
1530
|
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
|
1371
1531
|
|
1372
1532
|
Example:
|
@@ -1384,7 +1544,7 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
|
|
1384
1544
|
+-------------------------------+----------------------------------+-----------+
|
1385
1545
|
```
|
1386
1546
|
|
1387
|
-
> **Note:** `select` filters elements from the result of command, while the `query`
|
1547
|
+
> **Note:** Option `select` filters elements from the result of command, while the `query` option gives filtering parameters to the API when listing elements.
|
1388
1548
|
|
1389
1549
|
In above example, the same result is obtained with option:
|
1390
1550
|
|
@@ -1440,7 +1600,8 @@ The following decoders are supported:
|
|
1440
1600
|
| `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`) |
|
1441
1601
|
| `ruby` | String | Any | Execute specified Ruby code |
|
1442
1602
|
| `secret` | None | String | Ask password interactively (hides input) |
|
1443
|
-
| `stdin` | None | String | Read from stdin (no value on right) |
|
1603
|
+
| `stdin` | None | String | Read from stdin in text mode (no value on right) |
|
1604
|
+
| `stdbin` | None | String | Read from stdin in binary mode (no value on right) |
|
1444
1605
|
| `uri` | String | String | Read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
1445
1606
|
| `val` | String | String | Prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
1446
1607
|
| `yaml` | String | Any | Decode YAML |
|
@@ -1575,44 +1736,46 @@ Temporary files are deleted at the end of execution unless option: `clean_temp`
|
|
1575
1736
|
|
1576
1737
|
### Configuration file
|
1577
1738
|
|
1578
|
-
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
1579
|
-
|
1739
|
+
On the first execution of `ascli`, an empty configuration file is created in the configuration folder (`ascli config folder`).
|
1740
|
+
There is no mandatory information required in this file.
|
1741
|
+
The use of it is optional as any option can be provided on the command line.
|
1580
1742
|
|
1581
|
-
Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
|
1743
|
+
Although the file is a standard `YAML` file, `ascli` provides commands to read and modify it using the `config` command.
|
1582
1744
|
|
1583
|
-
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset)
|
1745
|
+
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset) in the configuration file.
|
1584
1746
|
|
1585
|
-
A configuration file provides a way to define default values, especially for authentication
|
1747
|
+
A configuration file provides a way to define default values, especially for authentication options, thus avoiding to always having to specify those options on the command line.
|
1586
1748
|
|
1587
|
-
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or
|
1749
|
+
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or its env var).
|
1588
1750
|
|
1589
|
-
The configuration file is
|
1751
|
+
The configuration file is a catalog of named lists of options, called: [Option Preset](#option-preset).
|
1752
|
+
Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset) (e.g. `mypreset`) using the option `preset`: `--preset=mypreset` or its shortcut: `-Pmypreset`.
|
1590
1753
|
|
1591
1754
|
#### Option Preset
|
1592
1755
|
|
1593
|
-
A [Option Preset](#option-preset)
|
1756
|
+
A [Option Preset](#option-preset) is a collection of options and their associated values in a named section in the configuration file.
|
1594
1757
|
|
1595
|
-
A named [Option Preset](#option-preset)
|
1758
|
+
A named [Option Preset](#option-preset) can be modified directly using `ascli`, which will update the configuration file :
|
1596
1759
|
|
1597
1760
|
```bash
|
1598
1761
|
ascli config preset set|delete|show|initialize|update <option preset>
|
1599
1762
|
```
|
1600
1763
|
|
1601
|
-
The command `update` allows the easy creation of [Option Preset](#option-preset)
|
1764
|
+
The command `update` allows the easy creation of [Option Preset](#option-preset) by simply providing the options in their command line format, e.g. :
|
1602
1765
|
|
1603
1766
|
```bash
|
1604
1767
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
|
1605
1768
|
```
|
1606
1769
|
|
1607
|
-
- This creates a [Option Preset](#option-preset)
|
1770
|
+
- This creates a [Option Preset](#option-preset) `demo_server` with all provided options.
|
1608
1771
|
|
1609
|
-
The command `set` allows setting individual options in a [Option Preset](#option-preset)
|
1772
|
+
The command `set` allows setting individual options in a [Option Preset](#option-preset).
|
1610
1773
|
|
1611
1774
|
```bash
|
1612
1775
|
ascli config preset set demo_server password my_password_here
|
1613
1776
|
```
|
1614
1777
|
|
1615
|
-
The command `initialize`, like `update` allows to set several
|
1778
|
+
The command `initialize`, like `update` allows to set several options at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
|
1616
1779
|
|
1617
1780
|
```bash
|
1618
1781
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
|
@@ -1624,7 +1787,7 @@ A full terminal based overview of the configuration can be displayed using:
|
|
1624
1787
|
ascli config preset over
|
1625
1788
|
```
|
1626
1789
|
|
1627
|
-
A list of [Option Preset](#option-preset)
|
1790
|
+
A list of [Option Preset](#option-preset) can be displayed using:
|
1628
1791
|
|
1629
1792
|
```bash
|
1630
1793
|
ascli config preset list
|
@@ -1647,6 +1810,15 @@ ascli config preset over
|
|
1647
1810
|
ascli config preset list
|
1648
1811
|
```
|
1649
1812
|
|
1813
|
+
It is possible to load a [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
|
1814
|
+
For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
|
1815
|
+
|
1816
|
+
```bash
|
1817
|
+
ascli config preset set pspecific preset pcommon
|
1818
|
+
```
|
1819
|
+
|
1820
|
+
When `pspecific` is loaded, then cumulative option `preset` will be set and it will also load `pcommon`.
|
1821
|
+
|
1650
1822
|
#### Special Option Preset: `config`
|
1651
1823
|
|
1652
1824
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
@@ -1686,7 +1858,7 @@ The default preset for `config` is read for any plugin invocation, this allows s
|
|
1686
1858
|
When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
|
1687
1859
|
If set, it loads the options independently of the plugin used.
|
1688
1860
|
|
1689
|
-
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global
|
1861
|
+
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global options (e.g. `config ascp use`)
|
1690
1862
|
>
|
1691
1863
|
> **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
|
1692
1864
|
|
@@ -1735,6 +1907,8 @@ detect https://faspex4.example.com/path
|
|
1735
1907
|
detect https://faspex5.example.com/path
|
1736
1908
|
detect https://node.example.com/path
|
1737
1909
|
detect https://shares.example.com/path shares
|
1910
|
+
detect https://tst.example.com/path faspio
|
1911
|
+
detect https://tst.example.com/path httpgw
|
1738
1912
|
detect my_org aoc
|
1739
1913
|
doc
|
1740
1914
|
doc transfer-parameters
|
@@ -1760,6 +1934,7 @@ gem path
|
|
1760
1934
|
gem version
|
1761
1935
|
genkey my_key
|
1762
1936
|
genkey my_key 4096
|
1937
|
+
image https://eudemo.asperademo.com/wallpaper.jpg
|
1763
1938
|
initdemo
|
1764
1939
|
open
|
1765
1940
|
plugin create my_command
|
@@ -1813,17 +1988,17 @@ demo_server:
|
|
1813
1988
|
We can see here:
|
1814
1989
|
|
1815
1990
|
- The configuration was created with `ascli` version 0.3.7
|
1816
|
-
- The default [Option Preset](#option-preset)
|
1817
|
-
- The [Option Preset](#option-preset)
|
1818
|
-
- The default [Option Preset](#option-preset)
|
1991
|
+
- The default [Option Preset](#option-preset) to load for `server` plugin is : `demo_server`
|
1992
|
+
- The [Option Preset](#option-preset) `demo_server` defines some options: the URL and credentials
|
1993
|
+
- The default [Option Preset](#option-preset) to load in any case is : `cli_default`
|
1819
1994
|
|
1820
|
-
Two [Option Preset](#option-preset)
|
1995
|
+
Two [Option Preset](#option-preset) are reserved:
|
1821
1996
|
|
1822
1997
|
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
1823
1998
|
It is used to check compatibility.
|
1824
|
-
- `default` is reserved to define the default [Option Preset](#option-preset)
|
1999
|
+
- `default` is reserved to define the default [Option Preset](#option-preset) name used for known plugins.
|
1825
2000
|
|
1826
|
-
The user may create as many [Option Preset](#option-preset)
|
2001
|
+
The user may create as many [Option Preset](#option-preset) as needed. For instance, a particular [Option Preset](#option-preset) can be created for a particular application instance and contain URL and credentials.
|
1827
2002
|
|
1828
2003
|
Values in the configuration also follow the [Extended Value Syntax](#extended-value-syntax).
|
1829
2004
|
|
@@ -1833,7 +2008,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended-va
|
|
1833
2008
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
1834
2009
|
```
|
1835
2010
|
|
1836
|
-
This creates the [Option Preset](#option-preset)
|
2011
|
+
This creates the [Option Preset](#option-preset):
|
1837
2012
|
|
1838
2013
|
```yaml
|
1839
2014
|
my_aoc_org:
|
@@ -1849,20 +2024,20 @@ Some options are global, some options are available only for some plugins. (the
|
|
1849
2024
|
Options are loaded using this algorithm:
|
1850
2025
|
|
1851
2026
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
1852
|
-
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset)
|
1853
|
-
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset)
|
2027
|
+
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset) for it, and loads it.
|
2028
|
+
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
|
1854
2029
|
- Environment variables are evaluated
|
1855
2030
|
- Command line options are evaluated
|
1856
2031
|
|
1857
|
-
|
2032
|
+
Options are evaluated in the order of command line.
|
1858
2033
|
|
1859
|
-
To avoid loading the default [Option Preset](#option-preset)
|
2034
|
+
To avoid loading the default [Option Preset](#option-preset) for a plugin, use: `-N`
|
1860
2035
|
|
1861
|
-
On command line, words in
|
2036
|
+
On command line, words in option names are separated by a dash (`-`).
|
1862
2037
|
In configuration file, separator is an underscore.
|
1863
2038
|
E.g. `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
|
1864
2039
|
|
1865
|
-
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset)
|
2040
|
+
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset) for the main plugin with:
|
1866
2041
|
|
1867
2042
|
```bash
|
1868
2043
|
ascli config preset set cli_default interactive no
|
@@ -1872,7 +2047,7 @@ ascli config preset set cli_default interactive no
|
|
1872
2047
|
ascli config preset set default config cli_default
|
1873
2048
|
```
|
1874
2049
|
|
1875
|
-
A [Option Preset](#option-preset)
|
2050
|
+
A [Option Preset](#option-preset) value can be removed with `unset`:
|
1876
2051
|
|
1877
2052
|
```bash
|
1878
2053
|
ascli config preset unset cli_default interactive
|
@@ -1892,7 +2067,7 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
|
|
1892
2067
|
|
1893
2068
|
#### Wizard
|
1894
2069
|
|
1895
|
-
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset)
|
2070
|
+
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset) with the provided information.
|
1896
2071
|
|
1897
2072
|
It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
|
1898
2073
|
|
@@ -1914,7 +2089,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my
|
|
1914
2089
|
|
1915
2090
|
This can also be provisioned in a configuration file:
|
1916
2091
|
|
1917
|
-
- Build [Option Preset](#option-preset)
|
2092
|
+
- Build [Option Preset](#option-preset)
|
1918
2093
|
|
1919
2094
|
```bash
|
1920
2095
|
ascli config preset set shares06 url https://10.25.0.6
|
@@ -1934,7 +2109,7 @@ or
|
|
1934
2109
|
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=my_password_here
|
1935
2110
|
```
|
1936
2111
|
|
1937
|
-
- Define this [Option Preset](#option-preset)
|
2112
|
+
- Define this [Option Preset](#option-preset) as the default [Option Preset](#option-preset) for the specified plugin (`shares`)
|
1938
2113
|
|
1939
2114
|
```bash
|
1940
2115
|
ascli config preset set default shares shares06
|
@@ -1946,7 +2121,7 @@ ascli config preset set default shares shares06
|
|
1946
2121
|
ascli config preset overview
|
1947
2122
|
```
|
1948
2123
|
|
1949
|
-
- Execute a command on the shares application using default
|
2124
|
+
- Execute a command on the shares application using default options
|
1950
2125
|
|
1951
2126
|
```bash
|
1952
2127
|
ascli shares repo browse /
|
@@ -2039,7 +2214,7 @@ The lookup is done by comparing the service URL and username (or access key).
|
|
2039
2214
|
|
2040
2215
|
#### Securing passwords and secrets
|
2041
2216
|
|
2042
|
-
A passwords can be saved in clear in a [Option Preset](#option-preset)
|
2217
|
+
A passwords can be saved in clear in a [Option Preset](#option-preset) together with other account information (URL, username, etc...).
|
2043
2218
|
Example:
|
2044
2219
|
|
2045
2220
|
```bash
|
@@ -2060,25 +2235,31 @@ ascli config vault create myconf @json:'{"password":"my_password_here"}'
|
|
2060
2235
|
|
2061
2236
|
### Private Key
|
2062
2237
|
|
2063
|
-
Some applications allow the user to be authenticated using
|
2064
|
-
|
2065
|
-
|
2066
|
-
|
2067
|
-
|
2238
|
+
Some Aspera applications allow the user to be authenticated using [Public Key Cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography):
|
2239
|
+
|
2240
|
+
- for SSH: Server
|
2241
|
+
- for OAuth JWT: AoC, Faspex5, Faspex, Shares
|
2242
|
+
|
2243
|
+
It consists in using a pair of associated keys: a private key and a public key.
|
2244
|
+
The same pair can be used for multiple applications.
|
2245
|
+
The file containing the private key (key pair) can optionally be protected by a passphrase.
|
2068
2246
|
If the key is protected by a passphrase, then it will be prompted when used.
|
2069
2247
|
(some plugins support option `passphrase`)
|
2070
2248
|
|
2071
|
-
The following commands use the shell variable `
|
2249
|
+
The following sample commands use the shell variable `KEY_PAIR_PATH`.
|
2072
2250
|
Set it to the desired safe location of the private key.
|
2073
|
-
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli
|
2251
|
+
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`.
|
2252
|
+
For example:
|
2074
2253
|
|
2075
2254
|
```bash
|
2076
|
-
|
2255
|
+
KEY_PAIR_PATH=~/.aspera/ascli/my_private_key
|
2077
2256
|
```
|
2078
2257
|
|
2079
2258
|
Several methods can be used to generate a key pair.
|
2080
2259
|
|
2081
|
-
The format expected for
|
2260
|
+
The format expected for keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
|
2261
|
+
|
2262
|
+
If another format is used, such as `DER`, it can be converted to `PEM`, e.g. using `openssl`.
|
2082
2263
|
|
2083
2264
|
#### `ascli` for key generation
|
2084
2265
|
|
@@ -2087,21 +2268,13 @@ For convenience, the public key is also extracted with extension `.pub`.
|
|
2087
2268
|
The key is not passphrase protected.
|
2088
2269
|
|
2089
2270
|
```bash
|
2090
|
-
ascli config genkey ${
|
2271
|
+
ascli config genkey ${KEY_PAIR_PATH} 4096
|
2091
2272
|
```
|
2092
2273
|
|
2093
|
-
> **Note:** `ascli` uses the `openssl` library.
|
2094
|
-
|
2095
2274
|
To display the public key of a private key:
|
2096
2275
|
|
2097
2276
|
```bash
|
2098
|
-
ascli config pubkey @file:${
|
2099
|
-
```
|
2100
|
-
|
2101
|
-
To display the version of **openssl** used in `ascli`:
|
2102
|
-
|
2103
|
-
```bash
|
2104
|
-
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
2277
|
+
ascli config pubkey @file:${KEY_PAIR_PATH}
|
2105
2278
|
```
|
2106
2279
|
|
2107
2280
|
#### `ssh-keygen`
|
@@ -2109,16 +2282,16 @@ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
2109
2282
|
Both private and public keys are generated, option `-N` is for passphrase.
|
2110
2283
|
|
2111
2284
|
```bash
|
2112
|
-
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${
|
2285
|
+
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
|
2113
2286
|
```
|
2114
2287
|
|
2115
2288
|
#### `openssl`
|
2116
2289
|
|
2117
|
-
To generate a
|
2290
|
+
To generate a key pair with a passphrase the following can be used on any system:
|
2118
2291
|
|
2119
2292
|
```bash
|
2120
|
-
openssl genrsa -passout pass:_passphrase_here_ -out ${
|
2121
|
-
openssl rsa -pubout -in ${
|
2293
|
+
openssl genrsa -passout pass:_passphrase_here_ -out ${KEY_PAIR_PATH} 4096
|
2294
|
+
openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub
|
2122
2295
|
```
|
2123
2296
|
|
2124
2297
|
`openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
|
@@ -2127,112 +2300,55 @@ In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` t
|
|
2127
2300
|
If option `-nodes` is not available, the passphrase can be removed using this method:
|
2128
2301
|
|
2129
2302
|
```bash
|
2130
|
-
openssl rsa -passin pass:_passphrase_here_ -in ${
|
2131
|
-
mv ${
|
2303
|
+
openssl rsa -passin pass:_passphrase_here_ -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
|
2304
|
+
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
|
2132
2305
|
```
|
2133
2306
|
|
2134
2307
|
To change (or add) the passphrase for a key do:
|
2135
2308
|
|
2136
2309
|
```bash
|
2137
|
-
openssl rsa -des3 -in ${
|
2138
|
-
mv ${
|
2139
|
-
```
|
2140
|
-
|
2141
|
-
### SSL CA certificate bundle
|
2142
|
-
|
2143
|
-
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders, by default it uses Ruby's default certificate store.
|
2144
|
-
|
2145
|
-
To display trusted certificate store locations:
|
2146
|
-
|
2147
|
-
```bash
|
2148
|
-
ascli --show-config --fields=cert_stores
|
2149
|
-
```
|
2150
|
-
|
2151
|
-
Use option `cert_stores` to modify the locations of certificate stores (files or folders).
|
2152
|
-
If you use this option, then default locations are not used.
|
2153
|
-
Default locations can be added using special value `DEF`.
|
2154
|
-
The value can be either an `Array` or `String` (path).
|
2155
|
-
Successive options add paths incrementally.
|
2156
|
-
All files of a folders are added.
|
2157
|
-
|
2158
|
-
`ascli` uses the Ruby `openssl` gem.
|
2159
|
-
By default it uses the system's `openssl` library, but JRuby uses its own implementation.
|
2160
|
-
|
2161
|
-
For example, on Linux to use the system's certificate store:
|
2162
|
-
|
2163
|
-
```bash
|
2164
|
-
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
2165
|
-
```
|
2166
|
-
|
2167
|
-
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
2168
|
-
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
2169
|
-
|
2170
|
-
One can display those default values:
|
2171
|
-
|
2172
|
-
```bash
|
2173
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
2174
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
2310
|
+
openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
|
2311
|
+
mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}
|
2175
2312
|
```
|
2176
2313
|
|
2177
|
-
|
2314
|
+
#### Using an application to generate a key pair
|
2178
2315
|
|
2179
|
-
|
2180
|
-
|
2316
|
+
Many applications are available, including on internet, to generate key pairs.
|
2317
|
+
For example: <https://cryptotools.net/rsagen>
|
2181
2318
|
|
2182
|
-
|
2183
|
-
ascli config ascp info --fields=openssldir
|
2184
|
-
```
|
2319
|
+
> **Note:** Be careful that private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
|
2185
2320
|
|
2186
|
-
|
2187
|
-
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
2188
|
-
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
2321
|
+
### Image and video thumbnails
|
2189
2322
|
|
2190
|
-
|
2191
|
-
|
2192
|
-
Typically done by updating the system's root certificate store.
|
2323
|
+
`ascli` can display thumbnails for images and videos in the terminal.
|
2324
|
+
This is available:
|
2193
2325
|
|
2194
|
-
|
2326
|
+
- in the `thumbnail` command of `node` when using **gen4/access key** API.
|
2327
|
+
- when using the `show` command of `preview` plugin.
|
2328
|
+
- `coffee` and `image` commands of `config` plugin.
|
2329
|
+
- any displayed value which is an url to image can be displayed with option `format` set to `image`
|
2195
2330
|
|
2196
|
-
|
2197
|
-
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
2198
|
-
```
|
2331
|
+
The following options can be specified in the option `image`:
|
2199
2332
|
|
2200
|
-
|
2333
|
+
| Option | Type | Description |
|
2334
|
+
|------------|---------|-------------|
|
2335
|
+
| reserve | Integer | Lines reserved to display a status |
|
2336
|
+
| text | Bool | Display text instead of image|
|
2337
|
+
| double | Bool | Display double text resolution (half characters) |
|
2338
|
+
| font_ratio | Float | Font height/width ratio in terminal |
|
2201
2339
|
|
2202
2340
|
```bash
|
2203
|
-
ascli config
|
2341
|
+
ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --image=@json:'{"text":true}'
|
2204
2342
|
```
|
2205
2343
|
|
2206
|
-
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
2207
|
-
|
2208
|
-
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
2209
|
-
|
2210
2344
|
```bash
|
2211
|
-
|
2345
|
+
curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdbin:
|
2212
2346
|
```
|
2213
2347
|
|
2214
|
-
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
2215
|
-
|
2216
|
-
Then, use this file as certificate store (e.g. here, Node API):
|
2217
|
-
|
2218
2348
|
```bash
|
2219
|
-
ascli config
|
2349
|
+
echo -n https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @uri:@stdin:
|
2220
2350
|
```
|
2221
2351
|
|
2222
|
-
### Image and video thumbnails
|
2223
|
-
|
2224
|
-
`ascli` can display thumbnails for images and videos in the terminal.
|
2225
|
-
This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
|
2226
|
-
It's also available when using the `show` command of `preview` plugin.
|
2227
|
-
|
2228
|
-
The following options can be specified in the option `query`:
|
2229
|
-
|
2230
|
-
| Option | Description |
|
2231
|
-
|------------|-------------|
|
2232
|
-
| text | Display text instead of image (Bool) |
|
2233
|
-
| double | Display double text resolution (half characters) (Bool) |
|
2234
|
-
| font_ratio | Font height/width ratio in terminal (Float) |
|
2235
|
-
|
2236
2352
|
### Graphical Interactions: Browser and Text Editor
|
2237
2353
|
|
2238
2354
|
Some actions may require the use of a graphical tool:
|
@@ -2250,9 +2366,7 @@ It is also possible to force the graphical mode with option --ui :
|
|
2250
2366
|
|
2251
2367
|
The gem is equipped with traces, mainly for debugging and learning APIs.
|
2252
2368
|
By default logging level is `warn` and the output channel is `stderr`.
|
2253
|
-
To increase debug level, use
|
2254
|
-
|
2255
|
-
It is also possible to activate traces before log facility initialization using env var `ASCLI_LOG_LEVEL`.
|
2369
|
+
To increase debug level, use option `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or an [Option Preset](#option-preset)).
|
2256
2370
|
|
2257
2371
|
By default passwords and secrets are removed from logs.
|
2258
2372
|
Use option `log_secrets` set to `yes` to reveal secrets in logs.
|
@@ -2332,21 +2446,33 @@ Refer to the following sections.
|
|
2332
2446
|
|
2333
2447
|
#### Proxy for REST and HTTP Gateway
|
2334
2448
|
|
2335
|
-
REST API calls and transfers based on HTTP Gateway both use Ruby `Net::HTTP`
|
2449
|
+
REST API calls and transfers based on HTTP Gateway both use Ruby's `Net::HTTP` class.
|
2450
|
+
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
2336
2451
|
|
2337
|
-
|
2452
|
+
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used .
|
2338
2453
|
|
2339
|
-
The `http_proxy` environment variable (**lower case
|
2454
|
+
The `http_proxy` environment variable (**lower case**) can be set to the **URL** of the proxy (with optional credentials).
|
2455
|
+
Syntax is: `(http|https)://[user:password@]host:port`.
|
2340
2456
|
E.g. `http://myproxy.org.net:3128`.
|
2341
|
-
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
2342
2457
|
|
2343
|
-
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not**
|
2458
|
+
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** valid.
|
2459
|
+
|
2460
|
+
Credentials for proxy are optional but can also be specified:
|
2344
2461
|
|
2345
2462
|
```bash
|
2346
|
-
export http_proxy=http://proxy.example.com:3128
|
2463
|
+
export http_proxy=http://user:password@proxy.example.com:3128
|
2464
|
+
```
|
2465
|
+
|
2466
|
+
Option `http_proxy` does the same (set env var) but on command line:
|
2467
|
+
|
2468
|
+
```bash
|
2469
|
+
ascli --http-proxy=http://username:password@host:port ...
|
2347
2470
|
```
|
2348
2471
|
|
2349
2472
|
Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) javascript value.
|
2473
|
+
|
2474
|
+
Note that proxy credentials are not supported in PAC files.
|
2475
|
+
|
2350
2476
|
To read the script from a URL (`http:`, `https:` and `file:`), use prefix: `@uri:`.
|
2351
2477
|
A minimal script can be specified to define the use of a local proxy:
|
2352
2478
|
|
@@ -2381,7 +2507,7 @@ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.
|
|
2381
2507
|
PROXY proxy.example.com:8080
|
2382
2508
|
```
|
2383
2509
|
|
2384
|
-
If the proxy requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
2510
|
+
If the proxy found with the PAC requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
2385
2511
|
|
2386
2512
|
```bash
|
2387
2513
|
ascli --proxy-credentials=@json:'["__username_here__","__password_here__"]' ...
|
@@ -2397,7 +2523,7 @@ Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallb
|
|
2397
2523
|
|
2398
2524
|
#### FASP proxy (forward) for transfers
|
2399
2525
|
|
2400
|
-
To specify a FASP proxy (forward), set the [
|
2526
|
+
To specify a FASP proxy (forward), set the [**transfer-spec**](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
|
2401
2527
|
|
2402
2528
|
For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
|
2403
2529
|
|
@@ -2413,7 +2539,20 @@ Or, alternatively, (prefer transfer spec like above, generally):
|
|
2413
2539
|
|
2414
2540
|
### FASP configuration
|
2415
2541
|
|
2416
|
-
|
2542
|
+
`ascli` uses one of the transfer agents to execute transfers.
|
2543
|
+
|
2544
|
+
By default it uses the `direct` agent, which is basically a local `ascp`.
|
2545
|
+
Nevertheless, `ascli` does not come with `ascp` installed.
|
2546
|
+
This is the reason why it is advised to install the Aspera Transfer SDK during installation (`ascli config ascp install`).
|
2547
|
+
|
2548
|
+
By default, `ascli` uses the `ascp` binary found in **well known locations**, i.e. typical Aspera product installation paths.
|
2549
|
+
|
2550
|
+
The way to specify the location of `ascp` is to use either options:
|
2551
|
+
|
2552
|
+
- `ascp_path`
|
2553
|
+
- `use_product`
|
2554
|
+
|
2555
|
+
The `config` plugin allows to find and specify the location of `ascp`.
|
2417
2556
|
It provides the following commands for `ascp` subcommand:
|
2418
2557
|
|
2419
2558
|
- `show` : shows the path of `ascp` used
|
@@ -2447,9 +2586,9 @@ ascli config ascp info
|
|
2447
2586
|
|
2448
2587
|
By default, `ascli` uses any found local product with `ascp`, including Transfer SDK.
|
2449
2588
|
|
2450
|
-
To
|
2589
|
+
To override and use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
|
2451
2590
|
|
2452
|
-
For a permanent change, the command `config ascp use` sets the same
|
2591
|
+
For a permanent change, the command `config ascp use` sets the same option for the global default.
|
2453
2592
|
|
2454
2593
|
Using a POSIX shell:
|
2455
2594
|
|
@@ -2477,7 +2616,10 @@ Saved to default global preset global_common_defaults
|
|
2477
2616
|
|
2478
2617
|
If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
|
2479
2618
|
|
2480
|
-
|
2619
|
+
If option `ascp_path` is not set, then the product identified with option `use_product` is used.
|
2620
|
+
|
2621
|
+
If `use_product` is not set, then the first product found is used,
|
2622
|
+
this is equivalent to using option: `--use-product=FIRST`.
|
2481
2623
|
|
2482
2624
|
Locally installed Aspera products can be listed with:
|
2483
2625
|
|
@@ -2496,10 +2638,6 @@ ascli config ascp products list
|
|
2496
2638
|
+---------------------------------------+----------------------------------------+
|
2497
2639
|
```
|
2498
2640
|
|
2499
|
-
#### Selection of local client for `ascp` for [`direct`](#agent-direct) agent
|
2500
|
-
|
2501
|
-
If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
|
2502
|
-
|
2503
2641
|
Using the option `use_product` finds the `ascp` binary of the selected product.
|
2504
2642
|
|
2505
2643
|
To permanently use the `ascp` of a product:
|
@@ -2556,12 +2694,12 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
|
2556
2694
|
|
2557
2695
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
2558
2696
|
|
2559
|
-
When a transfer needs to be started, a [
|
2560
|
-
This [
|
2697
|
+
When a transfer needs to be started, a [**transfer-spec**](#transfer-specification) has been internally prepared.
|
2698
|
+
This [**transfer-spec**](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
|
2561
2699
|
|
2562
|
-
|
2700
|
+
The following agents are supported and selected with option `transfer`:
|
2563
2701
|
|
2564
|
-
- [`direct`](#agent-direct) : execution of `ascp`
|
2702
|
+
- [`direct`](#agent-direct) : direct execution of `ascp` (local)
|
2565
2703
|
- [`trsdk`](#agent-transfer-sdk) : use of Aspera Transfer SDK (local)
|
2566
2704
|
- [`connect`](#agent-connect-client) : use Connect Client (local)
|
2567
2705
|
- [`alpha`](#agent-desktop-client) : use the new Desktop Client (local)
|
@@ -2572,7 +2710,7 @@ There are currently 3 agents, set with option `transfer`:
|
|
2572
2710
|
For example, a node agent executing an **upload**, or **package send** operation
|
2573
2711
|
will effectively push files to the related server from the agent node.
|
2574
2712
|
|
2575
|
-
`ascli` standardizes on the use of a [
|
2713
|
+
`ascli` standardizes on the use of a [**transfer-spec**](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
2576
2714
|
|
2577
2715
|
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2578
2716
|
|
@@ -2583,7 +2721,7 @@ This is the default agent for `ascli` (option `--transfer=direct`).
|
|
2583
2721
|
`ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
|
2584
2722
|
Refer to section [FASP](#fasp-configuration).
|
2585
2723
|
|
2586
|
-
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and
|
2724
|
+
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
|
2587
2725
|
|
2588
2726
|
| Name | Type | Description |
|
2589
2727
|
|------------------------|-------|-------------|
|
@@ -2607,7 +2745,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
|
|
2607
2745
|
```
|
2608
2746
|
|
2609
2747
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
2610
|
-
In order to use a custom certificate store, use the `trusted_certs`
|
2748
|
+
In order to use a custom certificate store, use the `trusted_certs` option.
|
2611
2749
|
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
2612
2750
|
|
2613
2751
|
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
@@ -2726,6 +2864,7 @@ There are no option for `transfer_info`.
|
|
2726
2864
|
|
2727
2865
|
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
2728
2866
|
This is especially useful for direct node-to-node transfers.
|
2867
|
+
|
2729
2868
|
Parameters provided in option `transfer_info` are:
|
2730
2869
|
|
2731
2870
|
| Name | Type | Description |
|
@@ -2735,7 +2874,7 @@ Parameters provided in option `transfer_info` are:
|
|
2735
2874
|
| password | string | Password, secret or bearer token</br>Mandatory |
|
2736
2875
|
| root_id | string | Root file id</br>Mandatory only for bearer token |
|
2737
2876
|
|
2738
|
-
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset)
|
2877
|
+
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset) :
|
2739
2878
|
|
2740
2879
|
```bash
|
2741
2880
|
--transfer-info=@preset:_name_here_
|
@@ -2749,12 +2888,14 @@ or be specified using the extended value syntax :
|
|
2749
2888
|
|
2750
2889
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
2751
2890
|
|
2752
|
-
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the
|
2891
|
+
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file id of the top folder to use on the node using this access key.
|
2753
2892
|
It can be either the access key's root file id, or any authorized file id underneath it.
|
2754
2893
|
|
2755
2894
|
#### Agent: HTTP Gateway
|
2756
2895
|
|
2757
|
-
|
2896
|
+
The Aspera HTTP Gateway is a service that allows to send and receive files using HTTPS.
|
2897
|
+
|
2898
|
+
By specifying option: `--transfer=httpgw`, `ascli` will start transfers using the Aspera HTTP Gateway.
|
2758
2899
|
|
2759
2900
|
Parameters provided in option `transfer_info` are:
|
2760
2901
|
|
@@ -2773,6 +2914,8 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
|
|
2773
2914
|
|
2774
2915
|
> **Note:** The gateway only supports transfers authorized with a token.
|
2775
2916
|
|
2917
|
+
If the application, e.g. AoC or Faspex 5, is configured to use the HTTP Gateway, then `ascli` will automatically use the gateway URL if `--transfer=httpgw` is specified, so `transfer_info` becomes optional.
|
2918
|
+
|
2776
2919
|
#### Agent: Transfer SDK
|
2777
2920
|
|
2778
2921
|
Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
|
@@ -2783,7 +2926,7 @@ Options for `transfer_info` are:
|
|
2783
2926
|
| Name | Type | Description |
|
2784
2927
|
|----------|--------|-------------|
|
2785
2928
|
| `url` | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: `:0` |
|
2786
|
-
| `external` | bool | Use external daemon, do not start
|
2929
|
+
| `external` | bool | Use external daemon, do not start one.<br/>Default: `false` |
|
2787
2930
|
| `keep` | bool | Keep the daemon running after exiting `ascli`<br/>Default: `false` |
|
2788
2931
|
|
2789
2932
|
> **Note:** If port zero is specified in the URL, then the daemon will listen on a random available port. If no address is specified, then `127.0.0.1` is used.
|
@@ -2812,7 +2955,7 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
2812
2955
|
### Transfer Specification
|
2813
2956
|
|
2814
2957
|
Some commands lead to file transfer (upload/download).
|
2815
|
-
All parameters necessary for this transfer are described in a [
|
2958
|
+
All parameters necessary for this transfer are described in a [**transfer-spec**](#transfer-specification) (Transfer Specification), such as:
|
2816
2959
|
|
2817
2960
|
- Server address
|
2818
2961
|
- Transfer user name
|
@@ -2820,11 +2963,11 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
|
|
2820
2963
|
- File list
|
2821
2964
|
- Etc...
|
2822
2965
|
|
2823
|
-
`ascli` builds the [
|
2966
|
+
`ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
|
2824
2967
|
It is not necessary to provide additional parameters on the command line for this transfer.
|
2825
2968
|
|
2826
|
-
It is possible to modify or add any of the supported [
|
2827
|
-
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [
|
2969
|
+
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameter using the `ts` option.
|
2970
|
+
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
|
2828
2971
|
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
2829
2972
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
2830
2973
|
|
@@ -2834,15 +2977,15 @@ It is possible to specify `ascp` options when the `transfer` option is set to [`
|
|
2834
2977
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
2835
2978
|
This is especially useful for `ascp` command line parameters not supported in the transfer spec.
|
2836
2979
|
|
2837
|
-
The use of a [
|
2980
|
+
The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the advantage of:
|
2838
2981
|
|
2839
2982
|
- Common to all [Transfer Agent](#transfer-clients-agents)
|
2840
2983
|
- Not dependent on command line limitations (special characters...)
|
2841
2984
|
|
2842
2985
|
### Transfer Parameters
|
2843
2986
|
|
2844
|
-
All standard [
|
2845
|
-
[
|
2987
|
+
All standard [**transfer-spec**](#transfer-specification) parameters can be specified.
|
2988
|
+
[**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
|
2846
2989
|
|
2847
2990
|
References:
|
2848
2991
|
|
@@ -2960,7 +3103,7 @@ The destination folder is set by `ascli` by default to:
|
|
2960
3103
|
- `.` for downloads
|
2961
3104
|
- `/` for uploads
|
2962
3105
|
|
2963
|
-
It is specified by the [
|
3106
|
+
It is specified by the [**transfer-spec**](#transfer-specification) parameter `destination_root`.
|
2964
3107
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
2965
3108
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
2966
3109
|
`--to-folder=<path>` .
|
@@ -2974,8 +3117,8 @@ By default the list of files to transfer is simply provided on the command line.
|
|
2974
3117
|
The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
|
2975
3118
|
The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
|
2976
3119
|
|
2977
|
-
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [
|
2978
|
-
So, eventually, the list of files to transfer is provided to the transfer agent using the [
|
3120
|
+
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
|
3121
|
+
So, eventually, the list of files to transfer is provided to the transfer agent using the [**transfer-spec**](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
|
2979
3122
|
The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
|
2980
3123
|
|
2981
3124
|
Possible values for option `sources` are:
|
@@ -3396,7 +3539,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
3396
3539
|
```text
|
3397
3540
|
ascli -h
|
3398
3541
|
NAME
|
3399
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3542
|
+
ascli -- a command line tool for Aspera Applications (v4.18.0)
|
3400
3543
|
|
3401
3544
|
SYNOPSIS
|
3402
3545
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -3418,7 +3561,7 @@ COMMANDS
|
|
3418
3561
|
OPTIONS
|
3419
3562
|
Options begin with a '-' (minus), and value is provided on command line.
|
3420
3563
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
3421
|
-
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
|
3564
|
+
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, stdbin, yaml, zlib, extend, preset, vault
|
3422
3565
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
3423
3566
|
|
3424
3567
|
ARGS
|
@@ -3427,20 +3570,20 @@ ARGS
|
|
3427
3570
|
OPTIONS: global
|
3428
3571
|
--interactive=ENUM Use interactive input of missing params: [no], yes
|
3429
3572
|
--ask-options=ENUM Ask even optional options: [no], yes
|
3430
|
-
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
|
3573
|
+
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv, image
|
3431
3574
|
--output=VALUE Destination for results (String)
|
3432
3575
|
--display=ENUM Output only some information: [info], data, error
|
3433
3576
|
--fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
|
3434
3577
|
--select=VALUE Select only some items in lists: column, value (Hash, Proc)
|
3435
|
-
--table-style=VALUE Table display style
|
3578
|
+
--table-style=VALUE Table display style (Hash)
|
3436
3579
|
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
3437
3580
|
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
3438
3581
|
--show-secrets=ENUM Show secrets on command output: [no], yes
|
3582
|
+
--image=VALUE Options for image display (Hash)
|
3439
3583
|
-h, --help Show this message
|
3440
3584
|
--bash-comp Generate bash completion for command
|
3441
3585
|
--show-config Display parameters used for the provided action
|
3442
3586
|
-v, --version Display version
|
3443
|
-
-w, --warnings Check for language warnings
|
3444
3587
|
--ui=ENUM Method to start browser: text, [graphical]
|
3445
3588
|
--log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
|
3446
3589
|
--logger=ENUM Logging method: [stderr], stdout, syslog
|
@@ -3451,7 +3594,7 @@ OPTIONS: global
|
|
3451
3594
|
--pid-file=VALUE Write process identifier to file, delete on exit (String)
|
3452
3595
|
|
3453
3596
|
COMMAND: config
|
3454
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
3597
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
3455
3598
|
OPTIONS:
|
3456
3599
|
--home=VALUE Home folder for tool (String)
|
3457
3600
|
--config-file=VALUE Path to YAML file with preset configuration
|
@@ -3485,9 +3628,10 @@ OPTIONS:
|
|
3485
3628
|
--silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
|
3486
3629
|
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
3487
3630
|
--http-options=VALUE Options for HTTP/S socket (Hash)
|
3631
|
+
--http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
|
3488
3632
|
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
3489
3633
|
--fpac=VALUE Proxy auto configuration script
|
3490
|
-
--proxy-credentials=VALUE HTTP proxy credentials
|
3634
|
+
--proxy-credentials=VALUE HTTP proxy credentials for fpac. Array: user,password (Array)
|
3491
3635
|
--ts=VALUE Override transfer spec values (Hash)
|
3492
3636
|
--to-folder=VALUE Destination folder for transferred files
|
3493
3637
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
@@ -3505,7 +3649,7 @@ OPTIONS:
|
|
3505
3649
|
|
3506
3650
|
|
3507
3651
|
COMMAND: node
|
3508
|
-
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer upload watch_folder
|
3652
|
+
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer transport upload watch_folder
|
3509
3653
|
OPTIONS:
|
3510
3654
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3511
3655
|
--username=VALUE User's name to log in
|
@@ -3518,28 +3662,32 @@ OPTIONS:
|
|
3518
3662
|
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3519
3663
|
|
3520
3664
|
|
3521
|
-
COMMAND:
|
3522
|
-
SUBCOMMANDS: health
|
3665
|
+
COMMAND: faspio
|
3666
|
+
SUBCOMMANDS: bridges health
|
3523
3667
|
OPTIONS:
|
3524
3668
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3525
3669
|
--username=VALUE User's name to log in
|
3526
3670
|
--password=VALUE User's password
|
3527
|
-
--
|
3528
|
-
--
|
3529
|
-
--
|
3530
|
-
--
|
3671
|
+
--auth=ENUM OAuth type of authentication: jwt, basic
|
3672
|
+
--client-id=VALUE OAuth client identifier
|
3673
|
+
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3674
|
+
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
3531
3675
|
|
3532
3676
|
|
3533
|
-
COMMAND:
|
3534
|
-
SUBCOMMANDS:
|
3677
|
+
COMMAND: orchestrator
|
3678
|
+
SUBCOMMANDS: health info plugins processes workflow
|
3535
3679
|
OPTIONS:
|
3536
3680
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3537
3681
|
--username=VALUE User's name to log in
|
3538
3682
|
--password=VALUE User's password
|
3683
|
+
--result=VALUE Specify result value as: 'work_step:parameter'
|
3684
|
+
--synchronous=ENUM Wait for completion: [no], yes
|
3685
|
+
--ret-style=ENUM How return type is requested in api: header, [arg], ext
|
3686
|
+
--auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
|
3539
3687
|
|
3540
3688
|
|
3541
3689
|
COMMAND: alee
|
3542
|
-
SUBCOMMANDS: entitlement
|
3690
|
+
SUBCOMMANDS: entitlement health
|
3543
3691
|
OPTIONS:
|
3544
3692
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3545
3693
|
--username=VALUE User's name to log in
|
@@ -3587,6 +3735,12 @@ OPTIONS:
|
|
3587
3735
|
--identity=VALUE Authentication URL (https://iam.cloud.ibm.com/identity)
|
3588
3736
|
|
3589
3737
|
|
3738
|
+
COMMAND: httpgw
|
3739
|
+
SUBCOMMANDS: health info
|
3740
|
+
OPTIONS:
|
3741
|
+
--url=VALUE URL of application, e.g. https://app.example.com/aspera/app
|
3742
|
+
|
3743
|
+
|
3590
3744
|
COMMAND: faspex
|
3591
3745
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
3592
3746
|
OPTIONS:
|
@@ -3649,9 +3803,9 @@ OPTIONS:
|
|
3649
3803
|
--scope=VALUE OAuth scope for AoC API calls
|
3650
3804
|
--redirect-uri=VALUE OAuth API client redirect URI
|
3651
3805
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3652
|
-
--passphrase=VALUE RSA private key passphrase
|
3806
|
+
--passphrase=VALUE RSA private key passphrase (String)
|
3653
3807
|
--workspace=VALUE Name of workspace (String, NilClass)
|
3654
|
-
--new-user-option=VALUE New user creation option for unknown package recipients
|
3808
|
+
--new-user-option=VALUE New user creation option for unknown package recipients (Hash)
|
3655
3809
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
3656
3810
|
--validator=VALUE Identifier of validator (optional for central)
|
3657
3811
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
@@ -3725,7 +3879,7 @@ Aspera on Cloud and Faspex 5 rely on Oauth.
|
|
3725
3879
|
By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
3726
3880
|
|
3727
3881
|
```bash
|
3728
|
-
ascli --show-config --
|
3882
|
+
ascli --show-config --fields=plugin_folder
|
3729
3883
|
```
|
3730
3884
|
|
3731
3885
|
You can create the skeleton of a new plugin like this:
|
@@ -3805,7 +3959,7 @@ Several types of OAuth authentication are supported:
|
|
3805
3959
|
|
3806
3960
|
The authentication method is controlled by option `auth`.
|
3807
3961
|
|
3808
|
-
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset)
|
3962
|
+
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset) for Aspera on Cloud](#configuration-for-aspera-on-cloud).
|
3809
3963
|
|
3810
3964
|
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
|
3811
3965
|
|
@@ -3842,9 +3996,9 @@ Once the client is registered, a **Client ID** and **Secret** are created, these
|
|
3842
3996
|
|
3843
3997
|
#### Configuration for Aspera on Cloud
|
3844
3998
|
|
3845
|
-
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset)
|
3999
|
+
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset) for `ascli` in its configuration file.
|
3846
4000
|
|
3847
|
-
Lets create a [Option Preset](#option-preset)
|
4001
|
+
Lets create a [Option Preset](#option-preset) called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
|
3848
4002
|
|
3849
4003
|
```bash
|
3850
4004
|
ascli config preset ask my_aoc_org url client_id client_secret
|
@@ -3858,7 +4012,7 @@ updated: my_aoc_org
|
|
3858
4012
|
|
3859
4013
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
3860
4014
|
|
3861
|
-
Define this [Option Preset](#option-preset)
|
4015
|
+
Define this [Option Preset](#option-preset) as default configuration for the `aspera` plugin:
|
3862
4016
|
|
3863
4017
|
```bash
|
3864
4018
|
ascli config preset set default aoc my_aoc_org
|
@@ -3949,9 +4103,9 @@ modified
|
|
3949
4103
|
|
3950
4104
|
> **Note:** The `aspera user info show` command can be used to verify modifications.
|
3951
4105
|
|
3952
|
-
#### [Option Preset](#option-preset)
|
4106
|
+
#### [Option Preset](#option-preset) modification for JWT
|
3953
4107
|
|
3954
|
-
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset)
|
4108
|
+
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset), do the following:
|
3955
4109
|
|
3956
4110
|
- Change auth method to JWT
|
3957
4111
|
- Provide location of private key
|
@@ -3985,7 +4139,7 @@ In that case, it is possible to list those shared folder by using a value for op
|
|
3985
4139
|
|
3986
4140
|
#### AoC: First Use
|
3987
4141
|
|
3988
|
-
Once client has been registered and [Option Preset](#option-preset)
|
4142
|
+
Once client has been registered and [Option Preset](#option-preset) created: `ascli` can be used:
|
3989
4143
|
|
3990
4144
|
```bash
|
3991
4145
|
ascli aoc files br /
|
@@ -4188,7 +4342,7 @@ Options:
|
|
4188
4342
|
#### Transfer: Using specific transfer ports
|
4189
4343
|
|
4190
4344
|
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
4191
|
-
The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
4345
|
+
The option `default_ports` ([yes]/no) allows `ascli` to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
4192
4346
|
|
4193
4347
|
#### Using ATS
|
4194
4348
|
|
@@ -4588,7 +4742,7 @@ General syntax:
|
|
4588
4742
|
ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
|
4589
4743
|
```
|
4590
4744
|
|
4591
|
-
Package creation parameter are sent as
|
4745
|
+
Package creation parameter are sent as **Command Parameter**.
|
4592
4746
|
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
4593
4747
|
|
4594
4748
|
List allowed shared inbox destinations with:
|
@@ -4766,13 +4920,13 @@ ascli aoc files short_link public delete _id_
|
|
4766
4920
|
|
4767
4921
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
4768
4922
|
|
4769
|
-
Although optional, the creation of [Option Preset](#option-preset)
|
4923
|
+
Although optional, the creation of [Option Preset](#option-preset) is recommended to avoid placing all parameters in the command line.
|
4770
4924
|
|
4771
4925
|
Procedure to send a file from org1 to org2:
|
4772
4926
|
|
4773
|
-
- Get access to Organization 1 and create a [Option Preset](#option-preset)
|
4927
|
+
- Get access to Organization 1 and create a [Option Preset](#option-preset): e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
|
4774
4928
|
- Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
|
4775
|
-
- Get access to Organization 2 and create a [Option Preset](#option-preset)
|
4929
|
+
- Get access to Organization 2 and create a [Option Preset](#option-preset): e.g. `org2`
|
4776
4930
|
- Check that access works and locate the destination folder `mydestfolder`
|
4777
4931
|
- Execute the following:
|
4778
4932
|
|
@@ -4812,6 +4966,7 @@ For instructions, refer to section `find` for plugin `node`.
|
|
4812
4966
|
admin analytics transfers nodes
|
4813
4967
|
admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
4814
4968
|
admin analytics transfers users --once_only=yes
|
4969
|
+
admin application list
|
4815
4970
|
admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4816
4971
|
admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4817
4972
|
admin ats access_key delete ak1ibmcloud
|
@@ -4822,36 +4977,35 @@ admin ats cluster list
|
|
4822
4977
|
admin ats cluster show --cloud=aws --region=eu-west-1
|
4823
4978
|
admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
4824
4979
|
admin auth_providers list
|
4825
|
-
admin
|
4826
|
-
admin
|
4827
|
-
admin
|
4828
|
-
admin
|
4829
|
-
admin
|
4830
|
-
admin
|
4831
|
-
admin
|
4832
|
-
admin
|
4833
|
-
admin
|
4834
|
-
admin
|
4835
|
-
admin
|
4836
|
-
admin
|
4837
|
-
admin
|
4838
|
-
admin
|
4839
|
-
admin
|
4840
|
-
admin
|
4841
|
-
admin
|
4842
|
-
admin
|
4843
|
-
admin
|
4844
|
-
admin
|
4845
|
-
admin
|
4846
|
-
admin
|
4847
|
-
admin
|
4848
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4849
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4850
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4851
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4852
|
-
admin resource workspace list
|
4853
|
-
admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4980
|
+
admin client list
|
4981
|
+
admin client_access_key list
|
4982
|
+
admin client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
|
4983
|
+
admin client_registration_token delete client_reg_id
|
4984
|
+
admin client_registration_token list
|
4985
|
+
admin contact list
|
4986
|
+
admin dropbox list
|
4987
|
+
admin dropbox_membership list
|
4988
|
+
admin group list
|
4989
|
+
admin kms_profile list
|
4990
|
+
admin node do %name:my_ak_name --secret=my_ak_secret browse /
|
4991
|
+
admin node do %name:my_ak_name --secret=my_ak_secret delete /folder1
|
4992
|
+
admin node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4993
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4994
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4995
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4996
|
+
admin node list
|
4997
|
+
admin operation list
|
4998
|
+
admin organization show
|
4999
|
+
admin package list --http-options=@json:'{"read_timeout":120.0}'
|
5000
|
+
admin saml_configuration list
|
5001
|
+
admin self show
|
5002
|
+
admin short_link list
|
4854
5003
|
admin subscription
|
5004
|
+
admin user list
|
5005
|
+
admin user modify %name:my_user_email @json:'{"deactivated":false}'
|
5006
|
+
admin workspace list
|
5007
|
+
admin workspace_membership list
|
5008
|
+
admin workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4855
5009
|
automation workflow action wf_id create @json:'{"name":"toto"}' \
|
4856
5010
|
automation workflow create @json:'{"name":"test_workflow"}'
|
4857
5011
|
automation workflow delete wf_id
|
@@ -4924,9 +5078,9 @@ user workspaces list
|
|
4924
5078
|
|
4925
5079
|
ATS is usable either :
|
4926
5080
|
|
4927
|
-
- From an AoC subscription : ascli aoc admin ats : use AoC authentication
|
5081
|
+
- From an AoC subscription : `ascli aoc admin ats` : use AoC authentication
|
4928
5082
|
|
4929
|
-
- Or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
|
5083
|
+
- Or from an IBM Cloud subscription : `ascli ats` : use IBM Cloud API key authentication
|
4930
5084
|
|
4931
5085
|
### IBM Cloud ATS : Creation of api key
|
4932
5086
|
|
@@ -5098,8 +5252,8 @@ mkdir my_upload_folder/target_hot
|
|
5098
5252
|
mv my_upload_folder/200KB.2 my_upload_folder/to.delete
|
5099
5253
|
sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
|
5100
5254
|
sync admin status --sync-info=@json:'{"name":"sync2"}'
|
5101
|
-
sync admin status
|
5102
|
-
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"
|
5255
|
+
sync admin status my_srv_sync --sync-info=@json:'{"sessions":[{"name":"my_srv_sync","local_dir":"/data/local_sync"}]}'
|
5256
|
+
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_srv_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
|
5103
5257
|
sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
|
5104
5258
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
|
5105
5259
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
|
@@ -5203,7 +5357,7 @@ ascli server browse /aspera-test-dir-large
|
|
5203
5357
|
ascli server download /aspera-test-dir-large/200MB
|
5204
5358
|
```
|
5205
5359
|
|
5206
|
-
`initdemo` creates a [Option Preset](#option-preset)
|
5360
|
+
`initdemo` creates a [Option Preset](#option-preset) `demoserver` and set it as default for plugin `server`.
|
5207
5361
|
|
5208
5362
|
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
|
5209
5363
|
|
@@ -5235,6 +5389,18 @@ Example:
|
|
5235
5389
|
- `ascli node browse /` : list files with **gen3/node user** API
|
5236
5390
|
- `ascli node access_key do self browse /` : list files with **gen4/access key** API
|
5237
5391
|
|
5392
|
+
## Browse
|
5393
|
+
|
5394
|
+
Native API parameters can be placed in option `query`.
|
5395
|
+
|
5396
|
+
Special parameters can be placed in option `query` for "gen3" browse:
|
5397
|
+
|
5398
|
+
| Parameter | Description |
|
5399
|
+
|-----------|-------------|
|
5400
|
+
| `recursive` | Recursively list files |
|
5401
|
+
| `max` | Maximum number of files to list |
|
5402
|
+
| `self` | Offset in the list |
|
5403
|
+
|
5238
5404
|
### Operation `find` on **gen4/access key**
|
5239
5405
|
|
5240
5406
|
The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
|
@@ -5337,7 +5503,7 @@ There are three sync types of operations:
|
|
5337
5503
|
|
5338
5504
|
It is possible to start a FASPStream session using the node API:
|
5339
5505
|
|
5340
|
-
Use the command `ascli node stream create --ts=@json:<value>`, with [
|
5506
|
+
Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-spec**](#transfer-specification):
|
5341
5507
|
|
5342
5508
|
```json
|
5343
5509
|
{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
|
@@ -5643,6 +5809,7 @@ sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"
|
|
5643
5809
|
sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5644
5810
|
transfer list --query=@json:'{"active_only":true}'
|
5645
5811
|
transfer sessions
|
5812
|
+
transport
|
5646
5813
|
upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
5647
5814
|
upload --username=my_ak_name --password=my_ak_secret test_file.bin
|
5648
5815
|
upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
|
@@ -5664,7 +5831,7 @@ IBM Aspera's newer self-managed application.
|
|
5664
5831
|
|
5665
5832
|
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
5666
5833
|
|
5667
|
-
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset)
|
5834
|
+
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset):
|
5668
5835
|
|
5669
5836
|
```bash
|
5670
5837
|
ascli config wizard
|
@@ -5802,24 +5969,30 @@ ascli config preset update f5boot --url=https://localhost/aspera/faspex --auth=b
|
|
5802
5969
|
> **Note:** Add `ascli faspex5` in front of the commands:
|
5803
5970
|
|
5804
5971
|
```bash
|
5805
|
-
admin
|
5806
|
-
admin
|
5807
|
-
admin
|
5808
|
-
admin
|
5809
|
-
admin
|
5810
|
-
admin
|
5811
|
-
admin
|
5812
|
-
admin
|
5813
|
-
admin
|
5814
|
-
admin
|
5815
|
-
admin
|
5816
|
-
admin
|
5817
|
-
admin
|
5818
|
-
admin
|
5819
|
-
admin
|
5820
|
-
admin
|
5972
|
+
admin accounts list
|
5973
|
+
admin clean_deleted
|
5974
|
+
admin contacts list
|
5975
|
+
admin distribution_lists create @json:'{"name":"test4","contacts":[{"name":"john@example.com"}]}'
|
5976
|
+
admin distribution_lists delete %name:test4
|
5977
|
+
admin distribution_lists list --query=@json:'{"type":"global"}'
|
5978
|
+
admin event app
|
5979
|
+
admin event web
|
5980
|
+
admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
|
5981
|
+
admin metadata_profiles list
|
5982
|
+
admin node list
|
5983
|
+
admin oauth_clients list
|
5984
|
+
admin registrations list
|
5985
|
+
admin saml_configs list
|
5986
|
+
admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
5987
|
+
admin shared_inboxes list
|
5988
|
+
admin shared_inboxes list --query=@json:'{"all":true}'
|
5989
|
+
admin shared_inboxes members %name:my_shared_box_name create %name:john@example.com
|
5990
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
|
5991
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
|
5992
|
+
admin shared_inboxes members %name:my_shared_box_name list
|
5821
5993
|
admin smtp show
|
5822
5994
|
admin smtp test my_email_external
|
5995
|
+
admin workgroups list
|
5823
5996
|
bearer_token
|
5824
5997
|
gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
5825
5998
|
health
|
@@ -5853,7 +6026,7 @@ user profile show
|
|
5853
6026
|
```
|
5854
6027
|
|
5855
6028
|
Most commands are directly REST API calls.
|
5856
|
-
Parameters to commands are carried through option `query`, as extended value, for `list`, or through
|
6029
|
+
Parameters to commands are carried through option `query`, as extended value, for `list`, or through **Command Parameter** for creation.
|
5857
6030
|
One can conveniently use the JSON format with prefix `@json:`.
|
5858
6031
|
|
5859
6032
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
@@ -5880,7 +6053,7 @@ To select another inbox, use option `box` with one of the following values:
|
|
5880
6053
|
|
5881
6054
|
### Faspex 5: Send a package
|
5882
6055
|
|
5883
|
-
The `Hash` creation
|
6056
|
+
The `Hash` creation **Command Parameter** provided to command `faspex5 packages send [extended value: Hash with package info ] [files...]` corresponds to the Faspex 5 API: `POST /packages`.
|
5884
6057
|
|
5885
6058
|
The interface is the one of the API (Refer to Faspex5 API documentation, or look at request in browser).
|
5886
6059
|
|
@@ -5931,7 +6104,7 @@ Basically, add the field `metadata`, with one key per metadata and the value is
|
|
5931
6104
|
|
5932
6105
|
### Faspex 5: List packages
|
5933
6106
|
|
5934
|
-
Option `box` can be used to list packages from a specific box (see
|
6107
|
+
Option `box` can be used to list packages from a specific box (see [Inbox Selection](#faspex-5-inbox-selection) above).
|
5935
6108
|
|
5936
6109
|
Option `query` can be used to filter the list of packages, based on native API parameters, directly sent to [Faspex 5 API `GET /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#getAllPackages).
|
5937
6110
|
|
@@ -5944,7 +6117,7 @@ Option `query` can be used to filter the list of packages, based on native API p
|
|
5944
6117
|
| `max` | Special | Maximum number of items to retrieve (stop pages when the maximum is passed) |
|
5945
6118
|
| `pmax` | Special | Maximum number of pages to request (stop pages when the maximum is passed) |
|
5946
6119
|
|
5947
|
-
A
|
6120
|
+
A **Command Parameter** in last position, of type `Proc`, can be used to filter the list of packages.
|
5948
6121
|
This advantage of this method is that the expression can be any test, even complex, as it is Ruby code.
|
5949
6122
|
But the disadvantage is that the filtering is done in `ascli` and not in Faspex 5, so it is less efficient.
|
5950
6123
|
|
@@ -5972,7 +6145,7 @@ Option `query` is available.
|
|
5972
6145
|
|
5973
6146
|
To list recursively add option `--query=@json:{"recursive":true}`.
|
5974
6147
|
|
5975
|
-
> **Note:** Option `
|
6148
|
+
> **Note:** Option `recursive` makes recursive API calls, so it can take a long time on large packages.
|
5976
6149
|
|
5977
6150
|
To limit the number of items retrieved, use option `--query=@json:{"max":10}`.
|
5978
6151
|
|
@@ -5982,7 +6155,7 @@ To receive one, or several packages at once, use command `faspex5 packages recei
|
|
5982
6155
|
Provide either a single package id, or an extended value `Array` of package ids, e.g. `@list:,1,2,3` as argument.
|
5983
6156
|
|
5984
6157
|
The same options as for `faspex5 packages list` can be used to select the box and filter the packages to download.
|
5985
|
-
I.e. options `box` and `query`, as well as last
|
6158
|
+
I.e. options `box` and `query`, as well as last **Command Parameter** `Proc` (filter).
|
5986
6159
|
|
5987
6160
|
Option `--once-only=yes` can be used, for "cargo-like" behavior.
|
5988
6161
|
Special package id `INIT` initializes the persistency of already received packages when option `--once-only=yes` is used.
|
@@ -6004,6 +6177,7 @@ ascli faspex5 admin res workgroup list
|
|
6004
6177
|
```
|
6005
6178
|
|
6006
6179
|
If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
|
6180
|
+
Example:
|
6007
6181
|
|
6008
6182
|
```bash
|
6009
6183
|
ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
|
@@ -6021,10 +6195,10 @@ It is equivalent to:
|
|
6021
6195
|
ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
6022
6196
|
```
|
6023
6197
|
|
6024
|
-
Other payload parameters are possible
|
6198
|
+
Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
|
6025
6199
|
|
6026
6200
|
```json
|
6027
|
-
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
|
6201
|
+
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3}
|
6028
6202
|
```
|
6029
6203
|
|
6030
6204
|
### Faspex 5: Create Metadata profile
|
@@ -6085,6 +6259,37 @@ Public invitations are for external users, provide just the email address.
|
|
6085
6259
|
|
6086
6260
|
Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
|
6087
6261
|
|
6262
|
+
### Faspex 5: Cleanup packages
|
6263
|
+
|
6264
|
+
> **Note:** Operation requires admin level.
|
6265
|
+
|
6266
|
+
Automated cleanup period can be displayed with:
|
6267
|
+
|
6268
|
+
```bash
|
6269
|
+
ascli faspex5 admin configuration show --fields=days_before_deleting_package_records
|
6270
|
+
```
|
6271
|
+
|
6272
|
+
This parameter can also be modified, for example:
|
6273
|
+
|
6274
|
+
```bash
|
6275
|
+
ascli faspex5 admin configuration modify @json:'{"days_before_deleting_package_records":30}'
|
6276
|
+
```
|
6277
|
+
|
6278
|
+
To start package purge, i.e. permanently remove packages marked for deletion older than `days_before_deleting_package_records`, use command:
|
6279
|
+
|
6280
|
+
```bash
|
6281
|
+
ascli faspex5 admin clean_deleted
|
6282
|
+
```
|
6283
|
+
|
6284
|
+
To delete all packages, one can use the following command:
|
6285
|
+
|
6286
|
+
```bash
|
6287
|
+
ascli faspex5 packages list --box=ALL --format=yaml --fields=id | ascli faspex5 packages delete @yaml:@stdin:
|
6288
|
+
```
|
6289
|
+
|
6290
|
+
> **Note:** Above command will mark all packages for deletion, and will be permanently removed after the configured period (`clean_deleted` command).
|
6291
|
+
> It is possible to add a filter to the list command to only delete packages matching some criteria, e.g. using `--select=@ruby:`.
|
6292
|
+
|
6088
6293
|
### Faspex 5: Faspex 4-style postprocessing
|
6089
6294
|
|
6090
6295
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
@@ -6210,13 +6415,13 @@ The contents of `delivery_info` is directly the contents of the `send` v3 [API o
|
|
6210
6415
|
Example:
|
6211
6416
|
|
6212
6417
|
```bash
|
6213
|
-
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}'
|
6418
|
+
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' /tmp/file1 /home/bar/file2
|
6214
6419
|
```
|
6215
6420
|
|
6216
6421
|
If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
|
6217
6422
|
`"recipients":["*MyDropboxName"]`
|
6218
6423
|
|
6219
|
-
Additional optional parameters in `delivery_info`:
|
6424
|
+
Additional optional parameters in mandatory option `delivery_info`:
|
6220
6425
|
|
6221
6426
|
- Package Note: : `"note":"note this and that"`
|
6222
6427
|
- Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
@@ -6233,7 +6438,7 @@ The value is a `Hash` with the following keys:
|
|
6233
6438
|
|
6234
6439
|
### Email notification on transfer
|
6235
6440
|
|
6236
|
-
Like for any transfer, a notification can be sent by email using
|
6441
|
+
Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template` .
|
6237
6442
|
|
6238
6443
|
Example:
|
6239
6444
|
|
@@ -6370,10 +6575,10 @@ admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
|
|
6370
6575
|
files browse /
|
6371
6576
|
files delete my_share1/test_file.bin
|
6372
6577
|
files download --to-folder=. my_share1/test_file.bin
|
6373
|
-
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
6374
|
-
files upload --to-folder=
|
6578
|
+
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6579
|
+
files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6375
6580
|
files upload --to-folder=my_share1 test_file.bin
|
6376
|
-
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
6581
|
+
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6377
6582
|
health
|
6378
6583
|
```
|
6379
6584
|
|
@@ -6527,11 +6732,44 @@ node info --log-level=trace2
|
|
6527
6732
|
node upload test_file.bin
|
6528
6733
|
```
|
6529
6734
|
|
6735
|
+
## Plugin: `httpgw`: HTTP Gateway
|
6736
|
+
|
6737
|
+
### Httpgw sample commands
|
6738
|
+
|
6739
|
+
> **Note:** Add `ascli httpgw` in front of the commands:
|
6740
|
+
|
6741
|
+
```bash
|
6742
|
+
health
|
6743
|
+
```
|
6744
|
+
|
6745
|
+
## Plugin: `faspio`: Faspio Gateway
|
6746
|
+
|
6747
|
+
### Faspio sample commands
|
6748
|
+
|
6749
|
+
> **Note:** Add `ascli faspio` in front of the commands:
|
6750
|
+
|
6751
|
+
```bash
|
6752
|
+
bridges list
|
6753
|
+
health
|
6754
|
+
```
|
6755
|
+
|
6756
|
+
## Plugin: `alee`: Aspera License Entitlement Engine
|
6757
|
+
|
6758
|
+
Retrieve information on subscription.
|
6759
|
+
|
6760
|
+
### Alee sample commands
|
6761
|
+
|
6762
|
+
> **Note:** Add `ascli alee` in front of the commands:
|
6763
|
+
|
6764
|
+
```bash
|
6765
|
+
health
|
6766
|
+
```
|
6767
|
+
|
6530
6768
|
## Plugin: `preview`: Preview generator for AoC
|
6531
6769
|
|
6532
6770
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
6533
6771
|
It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
6534
|
-
Several
|
6772
|
+
Several options can be used to tune several aspects:
|
6535
6773
|
|
6536
6774
|
- Methods for detection of new files needing generation
|
6537
6775
|
- Methods for generation of video preview
|
@@ -6558,10 +6796,11 @@ asnodeadmin --reload
|
|
6558
6796
|
|
6559
6797
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
6560
6798
|
|
6561
|
-
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024
|
6562
|
-
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384
|
6799
|
+
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 Bytes).
|
6800
|
+
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KB, i.e. 16MB.
|
6563
6801
|
|
6564
|
-
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
6802
|
+
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
6803
|
+
To display the value, use `asuserdata`:
|
6565
6804
|
|
6566
6805
|
```bash
|
6567
6806
|
asuserdata -a | grep max_request_file_create_size_kb
|
@@ -6642,7 +6881,7 @@ chmod a+x /usr/bin/unoconv
|
|
6642
6881
|
The preview generator should be executed as a non-user.
|
6643
6882
|
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
6644
6883
|
|
6645
|
-
Like any `ascli` commands,
|
6884
|
+
Like any `ascli` commands, options can be passed on command line or using a configuration [Option Preset](#option-preset).
|
6646
6885
|
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
6647
6886
|
|
6648
6887
|
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
@@ -6902,9 +7141,9 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
6902
7141
|
- `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
|
6903
7142
|
- `--src-base` (`src_base`) if top level folder name shall not be created on destination
|
6904
7143
|
|
6905
|
-
> **Note:** `ascli` takes transfer parameters exclusively as a [
|
7144
|
+
> **Note:** `ascli` takes transfer parameters exclusively as a [**transfer-spec**](#transfer-specification), with `ts` option.
|
6906
7145
|
>
|
6907
|
-
> **Note:** Most, but not all, native `ascp` arguments are available as standard [
|
7146
|
+
> **Note:** Most, but not all, native `ascp` arguments are available as standard [**transfer-spec**](#transfer-specification) parameters.
|
6908
7147
|
>
|
6909
7148
|
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
6910
7149
|
|
@@ -6914,11 +7153,11 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
6914
7153
|
|
6915
7154
|
> **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
6916
7155
|
>
|
6917
|
-
> **Note:**
|
7156
|
+
> **Note:** Options may be saved in a [Option Preset](#option-preset) and used with `-P`.
|
6918
7157
|
|
6919
7158
|
#### Scheduling
|
6920
7159
|
|
6921
|
-
Once `ascli`
|
7160
|
+
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
6922
7161
|
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
6923
7162
|
|
6924
7163
|
### Example: Upload hot folder
|
@@ -7081,7 +7320,7 @@ Transfer is: <%=global_transfer_status%>
|
|
7081
7320
|
|
7082
7321
|
This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
|
7083
7322
|
|
7084
|
-
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [
|
7323
|
+
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [**transfer-spec**](#transfer-specification) is:
|
7085
7324
|
|
7086
7325
|
- Common to Aspera Node API (HTTP POST /ops/transfer)
|
7087
7326
|
- Common to Aspera Connect API (browser javascript startTransfer)
|
@@ -7091,7 +7330,7 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
|
|
7091
7330
|
|
7092
7331
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
7093
7332
|
|
7094
|
-
`ascli` expect one single argument: a session specification that contains parameters and a [
|
7333
|
+
`ascli` expect one single argument: a session specification that contains parameters and a [**transfer-spec**](#transfer-specification).
|
7095
7334
|
|
7096
7335
|
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted on stdin.
|
7097
7336
|
|
@@ -7103,7 +7342,7 @@ Top level parameters supported by `asession`:
|
|
7103
7342
|
|
7104
7343
|
| parameter | description |
|
7105
7344
|
|-----------|-------------|
|
7106
|
-
| `spec` | the [
|
7345
|
+
| `spec` | the [**transfer-spec**](#transfer-specification) |
|
7107
7346
|
| `agent` | same parameters as transfer-info for agent `direct` |
|
7108
7347
|
| `loglevel` | log level of `asession` |
|
7109
7348
|
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
|
@@ -7136,7 +7375,7 @@ asession < session.json
|
|
7136
7375
|
|
7137
7376
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` -> `long_parameter`
|
7138
7377
|
|
7139
|
-
This is particularly useful for a persistent session ( with the [
|
7378
|
+
This is particularly useful for a persistent session ( with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true` )
|
7140
7379
|
|
7141
7380
|
```json
|
7142
7381
|
asession
|
@@ -7205,7 +7444,7 @@ So, it evolved into `ascli`:
|
|
7205
7444
|
|
7206
7445
|
- Portable: works on platforms supporting `ruby` (and `ascp`)
|
7207
7446
|
- Easy to install with the `gem` utility
|
7208
|
-
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [
|
7447
|
+
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable , more standard)
|
7209
7448
|
- `ruby` is consistent with other Aspera products
|
7210
7449
|
|
7211
7450
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
@@ -7215,7 +7454,7 @@ Enjoy a coffee on me:
|
|
7215
7454
|
|
7216
7455
|
```bash
|
7217
7456
|
ascli config coffee --ui=text
|
7218
|
-
ascli config coffee --ui=text --
|
7457
|
+
ascli config coffee --ui=text --image=@json:'{"text":true}'
|
7219
7458
|
ascli config coffee
|
7220
7459
|
```
|
7221
7460
|
|
@@ -7244,11 +7483,13 @@ This also requires Ruby header files.
|
|
7244
7483
|
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
7245
7484
|
`ruby-dev` or `ruby-devel`, depending on distribution.
|
7246
7485
|
|
7247
|
-
###
|
7486
|
+
### ED25519 key not supported
|
7248
7487
|
|
7249
7488
|
ED25519 keys are deactivated since `ascli` version 0.9.24 as it requires additional gems that require native compilation and thus caused problems.
|
7250
7489
|
This type of key will just be ignored.
|
7251
7490
|
|
7491
|
+
To re-activate, set env var `ASCLI_ENABLE_ED25519` to `true`.
|
7492
|
+
|
7252
7493
|
Without this deactivation, if such key was present in user's `.ssh` folder then the following error was generated:
|
7253
7494
|
|
7254
7495
|
```output
|
@@ -7258,4 +7499,5 @@ OpenSSH keys only supported if ED25519 is available
|
|
7258
7499
|
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
7259
7500
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
7260
7501
|
|
7261
|
-
|
7502
|
+
In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also de-activated by default.
|
7503
|
+
To re-activate, set env var `ASCLI_ENABLE_ECDSHA2` to `true`.
|