aspera-cli 4.6.0 → 4.7.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +427 -300
- data/bin/ascli +2 -1
- data/bin/asession +1 -0
- data/docs/test_env.conf +2 -0
- data/examples/aoc.rb +4 -3
- data/examples/faspex4.rb +21 -19
- data/examples/proxy.pac +1 -1
- data/examples/transfer.rb +15 -15
- data/lib/aspera/aoc.rb +135 -124
- data/lib/aspera/ascmd.rb +85 -75
- data/lib/aspera/ats_api.rb +11 -10
- data/lib/aspera/cli/basic_auth_plugin.rb +13 -14
- data/lib/aspera/cli/extended_value.rb +42 -33
- data/lib/aspera/cli/formater.rb +138 -111
- data/lib/aspera/cli/info.rb +17 -0
- data/lib/aspera/cli/listener/line_dump.rb +3 -2
- data/lib/aspera/cli/listener/logger.rb +2 -1
- data/lib/aspera/cli/listener/progress.rb +16 -18
- data/lib/aspera/cli/listener/progress_multi.rb +13 -16
- data/lib/aspera/cli/main.rb +122 -130
- data/lib/aspera/cli/manager.rb +146 -154
- data/lib/aspera/cli/plugin.rb +38 -34
- data/lib/aspera/cli/plugins/alee.rb +6 -6
- data/lib/aspera/cli/plugins/aoc.rb +273 -276
- data/lib/aspera/cli/plugins/ats.rb +82 -76
- data/lib/aspera/cli/plugins/bss.rb +14 -16
- data/lib/aspera/cli/plugins/config.rb +350 -306
- data/lib/aspera/cli/plugins/console.rb +23 -19
- data/lib/aspera/cli/plugins/cos.rb +18 -18
- data/lib/aspera/cli/plugins/faspex.rb +180 -159
- data/lib/aspera/cli/plugins/faspex5.rb +64 -54
- data/lib/aspera/cli/plugins/node.rb +147 -140
- data/lib/aspera/cli/plugins/orchestrator.rb +68 -66
- data/lib/aspera/cli/plugins/preview.rb +92 -96
- data/lib/aspera/cli/plugins/server.rb +79 -75
- data/lib/aspera/cli/plugins/shares.rb +23 -24
- data/lib/aspera/cli/plugins/sync.rb +20 -22
- data/lib/aspera/cli/transfer_agent.rb +40 -39
- data/lib/aspera/cli/version.rb +2 -1
- data/lib/aspera/colors.rb +35 -27
- data/lib/aspera/command_line_builder.rb +48 -34
- data/lib/aspera/cos_node.rb +29 -21
- data/lib/aspera/data_repository.rb +3 -2
- data/lib/aspera/environment.rb +50 -45
- data/lib/aspera/fasp/agent_base.rb +22 -20
- data/lib/aspera/fasp/agent_connect.rb +13 -11
- data/lib/aspera/fasp/agent_direct.rb +48 -59
- data/lib/aspera/fasp/agent_httpgw.rb +33 -39
- data/lib/aspera/fasp/agent_node.rb +15 -13
- data/lib/aspera/fasp/agent_trsdk.rb +12 -14
- data/lib/aspera/fasp/error.rb +2 -1
- data/lib/aspera/fasp/error_info.rb +68 -52
- data/lib/aspera/fasp/installation.rb +106 -94
- data/lib/aspera/fasp/listener.rb +1 -0
- data/lib/aspera/fasp/parameters.rb +83 -92
- data/lib/aspera/fasp/parameters.yaml +305 -249
- data/lib/aspera/fasp/resume_policy.rb +11 -14
- data/lib/aspera/fasp/transfer_spec.rb +26 -0
- data/lib/aspera/fasp/uri.rb +22 -21
- data/lib/aspera/faspex_gw.rb +55 -90
- data/lib/aspera/hash_ext.rb +4 -3
- data/lib/aspera/id_generator.rb +8 -7
- data/lib/aspera/keychain/encrypted_hash.rb +17 -16
- data/lib/aspera/keychain/macos_security.rb +6 -10
- data/lib/aspera/log.rb +25 -20
- data/lib/aspera/nagios.rb +13 -12
- data/lib/aspera/node.rb +30 -22
- data/lib/aspera/oauth.rb +175 -226
- data/lib/aspera/open_application.rb +4 -3
- data/lib/aspera/persistency_action_once.rb +6 -6
- data/lib/aspera/persistency_folder.rb +5 -9
- data/lib/aspera/preview/file_types.rb +6 -5
- data/lib/aspera/preview/generator.rb +25 -24
- data/lib/aspera/preview/options.rb +16 -14
- data/lib/aspera/preview/utils.rb +98 -98
- data/lib/aspera/{proxy_auto_config.erb.js → proxy_auto_config.js} +23 -31
- data/lib/aspera/proxy_auto_config.rb +111 -20
- data/lib/aspera/rest.rb +115 -113
- data/lib/aspera/rest_call_error.rb +2 -2
- data/lib/aspera/rest_error_analyzer.rb +23 -25
- data/lib/aspera/rest_errors_aspera.rb +15 -14
- data/lib/aspera/ssh.rb +12 -10
- data/lib/aspera/sync.rb +42 -41
- data/lib/aspera/temp_file_manager.rb +18 -14
- data/lib/aspera/timer_limiter.rb +2 -1
- data/lib/aspera/uri_reader.rb +7 -5
- data/lib/aspera/web_auth.rb +79 -76
- metadata +64 -21
- data/docs/Makefile +0 -65
- data/docs/README.erb.md +0 -4424
- data/docs/README.md +0 -13
- data/docs/diagrams.txt +0 -49
- data/docs/doc_tools.rb +0 -58
- data/lib/aspera/cli/plugins/shares2.rb +0 -114
- data/lib/aspera/fasp/default.rb +0 -17
data/README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
4
4
|
|
5
|
-
Version : 4.
|
5
|
+
Version : 4.7.0
|
6
6
|
|
7
7
|
Laurent/2016-2022
|
8
8
|
|
@@ -14,9 +14,10 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
|
|
14
14
|
|
15
15
|
Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
|
16
16
|
|
17
|
-
|
17
|
+
Minimum required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
|
18
18
|
|
19
|
-
[Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
|
19
|
+
[Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
|
20
|
+
[Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
|
20
21
|
|
21
22
|
## <a id="when_to_use"></a>When to use and when not to use
|
22
23
|
|
@@ -88,7 +89,7 @@ ascli --version
|
|
88
89
|
```
|
89
90
|
|
90
91
|
```bash
|
91
|
-
4.
|
92
|
+
4.7.0
|
92
93
|
```
|
93
94
|
|
94
95
|
### First use
|
@@ -126,7 +127,7 @@ If you want to use `ascli` with another server, and in order to make further cal
|
|
126
127
|
* download a file
|
127
128
|
|
128
129
|
```bash
|
129
|
-
ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=
|
130
|
+
ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_
|
130
131
|
```
|
131
132
|
|
132
133
|
```output
|
@@ -185,7 +186,7 @@ It is possible to install *either* directly on the host operating system (Linux,
|
|
185
186
|
|
186
187
|
The direct installation is recommended and consists in installing:
|
187
188
|
|
188
|
-
* [Ruby](#ruby) version
|
189
|
+
* [Ruby](#ruby) (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
|
189
190
|
* [aspera-cli](#the_gem)
|
190
191
|
* [Aspera SDK (ascp)](#fasp_prot)
|
191
192
|
|
@@ -238,7 +239,7 @@ Use this method to install on the native host.
|
|
238
239
|
|
239
240
|
A ruby interpreter is required to run the tool or to use the gem and tool.
|
240
241
|
|
241
|
-
Required Ruby version:
|
242
|
+
Required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
|
242
243
|
|
243
244
|
*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
244
245
|
|
@@ -246,7 +247,7 @@ Refer to the following sections for a proposed method for specific operating sys
|
|
246
247
|
|
247
248
|
The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, macOS, Windows with cygwin, etc...).
|
248
249
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
249
|
-
If you have a simpler better way to install Ruby version
|
250
|
+
If you have a simpler better way to install Ruby : use it ! (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
|
250
251
|
|
251
252
|
#### Generic: RVM: single user installation (not root)
|
252
253
|
|
@@ -516,7 +517,7 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
|
|
516
517
|
* Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
|
517
518
|
* Supports Commands, Option values and Parameters shortcuts
|
518
519
|
* FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
|
519
|
-
* Transfer parameters can be altered by modification of
|
520
|
+
* Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
|
520
521
|
* Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
521
522
|
* Supports FaspStream creation (using Node API)
|
522
523
|
* Supports Watchfolder creation (using Node API)
|
@@ -544,10 +545,10 @@ There are two types of command line arguments: Commands and Options. Example :
|
|
544
545
|
ascli command subcommand --option-name=VAL1 VAL2
|
545
546
|
```
|
546
547
|
|
547
|
-
* executes
|
548
|
-
* with one
|
549
|
-
* this option is given a
|
550
|
-
* the command has one additional
|
548
|
+
* executes *command*: `command subcommand`
|
549
|
+
* with one *option*: `option_name`
|
550
|
+
* this option is given a *value* of: `VAL1`
|
551
|
+
* the command has one additional *argument*: `VAL2`
|
551
552
|
|
552
553
|
When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `ascli conf ov` is the same as `ascli config overview`.
|
553
554
|
|
@@ -580,7 +581,7 @@ Note that here, `--sample` is taken as an argument, and not as an option, due to
|
|
580
581
|
|
581
582
|
Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
|
582
583
|
|
583
|
-
The value for
|
584
|
+
The value for *any* options can come from the following locations (in this order, last value evaluated overrides previous value):
|
584
585
|
|
585
586
|
* [Configuration file](#configfile).
|
586
587
|
* Environment variable
|
@@ -677,6 +678,9 @@ The option `display` controls the level of output:
|
|
677
678
|
* `data` display `data` and `error` messages
|
678
679
|
* `error` display only error messages.
|
679
680
|
|
681
|
+
By default, secrets are shown on output.
|
682
|
+
To hide secrets from results, set option `show_secrets` to `no`.
|
683
|
+
|
680
684
|
#### Selection of output object properties
|
681
685
|
|
682
686
|
By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
|
@@ -702,11 +706,12 @@ The difference between reader and decoder is order and ordinality. Both act like
|
|
702
706
|
|
703
707
|
The following "readers" are supported (returns value in []):
|
704
708
|
|
705
|
-
* @val:VALUE
|
706
|
-
* @file:PATH
|
707
|
-
* @
|
708
|
-
* @
|
709
|
-
* @
|
709
|
+
* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
|
710
|
+
* @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
|
711
|
+
* @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
712
|
+
* @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
713
|
+
* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
714
|
+
* @stdin: : [String] read from stdin (no value on right)
|
710
715
|
* @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
|
711
716
|
|
712
717
|
In addition it is possible to decode a value, using one or multiple decoders :
|
@@ -775,7 +780,7 @@ Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["co
|
|
775
780
|
|
776
781
|
Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
|
777
782
|
|
778
|
-
For instance, a [
|
783
|
+
For instance, a [*transfer-spec*](#transferspec) is expected to be a _Structured Value_.
|
779
784
|
|
780
785
|
Structured values shall be described using the [Extended Value Syntax](#extended).
|
781
786
|
A convenient way to specify a _Structured Value_ is to use the `@json:` decoder, and describe the value in JSON format. The `@ruby:` decoder can also be used. For an array of hash tables, the `@csvt:` decoder can be used.
|
@@ -784,12 +789,12 @@ It is also possible to provide a _Structured Value_ in a file using `@json:@file
|
|
784
789
|
|
785
790
|
### <a id="conffolder"></a>Configuration and Persistency Folder
|
786
791
|
|
787
|
-
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored
|
792
|
+
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored `[config folder]`: `[User's home folder]/.aspera/ascli`.
|
788
793
|
|
789
794
|
Note: `[User's home folder]` is found using ruby's `Dir.home` (`rb_w32_home_dir`).
|
790
795
|
It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
|
791
796
|
|
792
|
-
The
|
797
|
+
The [config folder] can be displayed using :
|
793
798
|
|
794
799
|
```bash
|
795
800
|
ascli config folder
|
@@ -811,6 +816,12 @@ ascli config folder
|
|
811
816
|
C:\Users\Kenji\.aspera\ascli
|
812
817
|
```
|
813
818
|
|
819
|
+
When OAuth is used (AoC, Faspex4 apiv4, Faspex5) `ascli` keeps a cache of generated bearer tokens in `[config folder]/persist_store` by default.
|
820
|
+
Option `cache_tokens` (**yes**/no) allows to control if Oauth tokens are cached on file system, or generated for each request.
|
821
|
+
The command `config flush_tokens` deletes all existing tokens.
|
822
|
+
Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
|
823
|
+
Tokens that can be refreshed are refreshed. Else tokens are re-generated if expired.
|
824
|
+
|
814
825
|
### <a id="configfile"></a>Configuration file
|
815
826
|
|
816
827
|
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
@@ -839,7 +850,7 @@ ascli config preset set|delete|show|initialize|update <option preset>
|
|
839
850
|
The command `update` allows the easy creation of [option preset](#lprt) by simply providing the options in their command line format, e.g. :
|
840
851
|
|
841
852
|
```bash
|
842
|
-
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=
|
853
|
+
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_ --ts=@json:'{"precalculate_job_size":true}'
|
843
854
|
```
|
844
855
|
|
845
856
|
* This creates a [option preset](#lprt) `demo_server` with all provided options.
|
@@ -847,13 +858,13 @@ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --u
|
|
847
858
|
The command `set` allows setting individual options in a [option preset](#lprt).
|
848
859
|
|
849
860
|
```bash
|
850
|
-
ascli config preset set demo_server password
|
861
|
+
ascli config preset set demo_server password _pass_here_
|
851
862
|
```
|
852
863
|
|
853
|
-
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a
|
864
|
+
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [*Structured Value*](#native).
|
854
865
|
|
855
866
|
```javascript
|
856
|
-
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"
|
867
|
+
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_pass_here_","ts":{"precalculate_job_size":true}}'
|
857
868
|
```
|
858
869
|
|
859
870
|
A full terminal based overview of the configuration can be displayed using:
|
@@ -933,7 +944,7 @@ cli_default:
|
|
933
944
|
demo_server:
|
934
945
|
url: ssh://demo.asperasoft.com:33001
|
935
946
|
username: asperaweb
|
936
|
-
password:
|
947
|
+
password: _pass_here_
|
937
948
|
```
|
938
949
|
|
939
950
|
We can see here:
|
@@ -956,7 +967,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended).
|
|
956
967
|
Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
957
968
|
|
958
969
|
```bash
|
959
|
-
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/
|
970
|
+
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
960
971
|
```
|
961
972
|
|
962
973
|
This creates the [option preset](#lprt):
|
@@ -964,7 +975,7 @@ This creates the [option preset](#lprt):
|
|
964
975
|
```yaml
|
965
976
|
...
|
966
977
|
my_aoc_org:
|
967
|
-
private_key: @file:"/Users/laurent/.aspera/ascli/
|
978
|
+
private_key: @file:"/Users/laurent/.aspera/ascli/my_private_key"
|
968
979
|
...
|
969
980
|
```
|
970
981
|
|
@@ -1008,13 +1019,13 @@ ascli config preset unset cli_default interactive
|
|
1008
1019
|
Example: Define options using command line:
|
1009
1020
|
|
1010
1021
|
```bash
|
1011
|
-
ascli -N --url=
|
1022
|
+
ascli -N --url=_url_here_ --password=_pass_here_ --username=_name_here_ node --show-config
|
1012
1023
|
```
|
1013
1024
|
|
1014
1025
|
Example: Define options using a hash:
|
1015
1026
|
|
1016
1027
|
```javascript
|
1017
|
-
ascli -N --preset=@json:'{"url":"
|
1028
|
+
ascli -N --preset=@json:'{"url":"_url_here_","password":"_pass_here_","username":"_name_here_"}' node --show-config
|
1018
1029
|
```
|
1019
1030
|
|
1020
1031
|
#### Shares Examples
|
@@ -1024,7 +1035,7 @@ only username/password and url are required (either on command line, or from con
|
|
1024
1035
|
Those can usually be provided on the command line:
|
1025
1036
|
|
1026
1037
|
```bash
|
1027
|
-
ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=
|
1038
|
+
ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=_pass_here_
|
1028
1039
|
```
|
1029
1040
|
|
1030
1041
|
This can also be provisioned in a config file:
|
@@ -1034,19 +1045,19 @@ This can also be provisioned in a config file:
|
|
1034
1045
|
```bash
|
1035
1046
|
ascli config preset set shares06 url https://10.25.0.6
|
1036
1047
|
ascli config preset set shares06 username john
|
1037
|
-
ascli config preset set shares06 password
|
1048
|
+
ascli config preset set shares06 password _pass_here_
|
1038
1049
|
```
|
1039
1050
|
|
1040
1051
|
This can also be done with one single command:
|
1041
1052
|
|
1042
1053
|
```javascript
|
1043
|
-
ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"
|
1054
|
+
ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"_pass_here_"}'
|
1044
1055
|
```
|
1045
1056
|
|
1046
1057
|
or
|
1047
1058
|
|
1048
1059
|
```bash
|
1049
|
-
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=
|
1060
|
+
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=_pass_here_
|
1050
1061
|
```
|
1051
1062
|
|
1052
1063
|
* Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
|
@@ -1128,6 +1139,54 @@ To test if a secret can be found use:
|
|
1128
1139
|
ascli conf vault get --username=access_key1
|
1129
1140
|
```
|
1130
1141
|
|
1142
|
+
### <a id="private_key"></a>Private Key
|
1143
|
+
|
1144
|
+
Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5...).
|
1145
|
+
It consists in generating a private key, or using a previouly generated key.
|
1146
|
+
The same key can be used for multiple applications.
|
1147
|
+
Technically, a private key contains the public key, which can be extracted.
|
1148
|
+
Currently, only private key not protected by a passphrase are supported.
|
1149
|
+
(TODO: add passphrase protection as option for aspera apps).
|
1150
|
+
|
1151
|
+
Several methods can be used to generate a key pair:
|
1152
|
+
|
1153
|
+
* `ascli`
|
1154
|
+
|
1155
|
+
The generated key is of type RSA 4096 bit. For convenience, the public key is also extracted.
|
1156
|
+
|
1157
|
+
```bash
|
1158
|
+
ascli config genkey ~/.aspera/ascli/my_private_key
|
1159
|
+
```
|
1160
|
+
|
1161
|
+
* `ssh-keygen`
|
1162
|
+
|
1163
|
+
```bash
|
1164
|
+
ssh-keygen -t rsa -f ~/.aspera/ascli/my_private_key -N ''
|
1165
|
+
```
|
1166
|
+
|
1167
|
+
* `openssl`
|
1168
|
+
|
1169
|
+
openssl is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS). To generate a privatekey pait without passphrase the following shall work on any system:
|
1170
|
+
|
1171
|
+
```bash
|
1172
|
+
APIKEY=~/.aspera/ascli/my_private_key
|
1173
|
+
openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
|
1174
|
+
openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
|
1175
|
+
openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
|
1176
|
+
rm -f ${APIKEY}.protected
|
1177
|
+
```
|
1178
|
+
|
1179
|
+
### <a id="certificates"></a>SSL CA certificate bundle
|
1180
|
+
|
1181
|
+
`ascli` uses ruby `openssl` gem, which uses the `openssl` library.
|
1182
|
+
Certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..).
|
1183
|
+
The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
|
1184
|
+
|
1185
|
+
`ascp` also needs to validate certificates when using WSS.
|
1186
|
+
By default, `ascp` uses primarily certificates from hard-coded path (e.g. on macOS: `/Library/Aspera/ssl`).
|
1187
|
+
`ascli` overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch.
|
1188
|
+
So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
|
1189
|
+
|
1131
1190
|
### Plugins
|
1132
1191
|
|
1133
1192
|
The CLI tool uses a plugin mechanism.
|
@@ -1142,13 +1201,13 @@ ascli conf plugin list
|
|
1142
1201
|
```
|
1143
1202
|
|
1144
1203
|
```output
|
1145
|
-
|
1146
|
-
| plugin | path
|
1147
|
-
|
1148
|
-
| shares |
|
1149
|
-
| node |
|
1204
|
+
+--------------+--------------------------------------------------------+
|
1205
|
+
| plugin | path |
|
1206
|
+
+--------------+--------------------------------------------------------+
|
1207
|
+
| shares | ..../aspera-cli/lib/aspera/cli/plugins/shares.rb |
|
1208
|
+
| node | ..../aspera-cli/lib/aspera/cli/plugins/node.rb |
|
1150
1209
|
...
|
1151
|
-
|
1210
|
+
+--------------+--------------------------------------------------------+
|
1152
1211
|
```
|
1153
1212
|
|
1154
1213
|
#### <a id="createownplugin"></a>Create your own plugin
|
@@ -1197,7 +1256,7 @@ To increase debug level, use parameter `log_level` (e.g. using command line `--l
|
|
1197
1256
|
It is also possible to activate traces before initialization using env var `AS_LOG_LEVEL`.
|
1198
1257
|
|
1199
1258
|
By default passwords and secrets are removed from logs.
|
1200
|
-
Use option `
|
1259
|
+
Use option `log_secrets` set to `yes` to reveal secrets in logs.
|
1201
1260
|
|
1202
1261
|
### Learning Aspera Product APIs (REST)
|
1203
1262
|
|
@@ -1219,7 +1278,7 @@ Ruby HTTP socket parameters can be adjusted.
|
|
1219
1278
|
| `open_timeout` | 60 |
|
1220
1279
|
| `keep_alive_timeout` | 2 |
|
1221
1280
|
|
1222
|
-
Values are in set
|
1281
|
+
Values are in set *seconds* and can be of type either integer or float.
|
1223
1282
|
Default values are the ones of Ruby.
|
1224
1283
|
For details refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1225
1284
|
|
@@ -1244,28 +1303,58 @@ It is also possible to force the graphical mode with option --ui :
|
|
1244
1303
|
* `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
1245
1304
|
* `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
1246
1305
|
|
1247
|
-
###
|
1306
|
+
### Proxy
|
1248
1307
|
|
1249
|
-
|
1308
|
+
There are several types of network connections, each of them use a different mechanism to define a *proxy*.
|
1250
1309
|
|
1251
|
-
|
1310
|
+
#### HTTP proxy for REST calls and transfers using HTTP gateway
|
1311
|
+
|
1312
|
+
To specify a HTTP proxy when ruby HTTP is used, set the `http_proxy` environment variable (lower case, preferred, or upper case).
|
1313
|
+
See [Ruby findproxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
1314
|
+
|
1315
|
+
```bash
|
1316
|
+
export http_proxy=http://myproxy.org.net:3128
|
1317
|
+
```
|
1252
1318
|
|
1253
|
-
|
1319
|
+
Note that ruby expects a URL and `myproxy.org.net:3128` alone is not accepted.
|
1254
1320
|
|
1255
|
-
|
1321
|
+
#### FASP proxy (forward) for transfers
|
1256
1322
|
|
1257
|
-
|
1323
|
+
To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `EX_fasp_proxy_url` (only supported with the `direct` agent).
|
1258
1324
|
|
1259
|
-
|
1325
|
+
#### HTTP proxy legacy Aspera HTTP fallback transfers
|
1260
1326
|
|
1261
|
-
|
1327
|
+
To specify a proxy for legacy HTTP fallback, set the [*transfer-spec*](#transferspec) parameter: `EX_http_proxy_url` (only supported with the `direct` agent).
|
1328
|
+
(It is also possible to use `EX_ascp_args` and native options in `direct`)
|
1329
|
+
|
1330
|
+
#### Proxy auto config
|
1331
|
+
|
1332
|
+
The `fpac` option allows use of a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) script defined as javascript value.
|
1333
|
+
To read the script from a URL (`http:`, `https:` and `file:`), use: `@uri:`.
|
1334
|
+
A minimal script can be specified, for example like this, to define the use of a local proxy:
|
1335
|
+
|
1336
|
+
```bash
|
1337
|
+
export ASCLI_FPAC='function FindProxyForURL(url, host){return "PROXY localhost:3128"}'
|
1338
|
+
```
|
1339
|
+
|
1340
|
+
The PAC file will be used for any HTTP/HTTPS/REST connection, but not other (e.g. FASP, HTTP fallback, HTTPGW)
|
1341
|
+
|
1342
|
+
The PAC file can be tested with command: `config proxy_check`. Example, using command line option:
|
1343
|
+
|
1344
|
+
```
|
1345
|
+
ascli conf proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:1234;DIRECT";}' http://example.com
|
1346
|
+
PROXY proxy.example.com:1234;DIRECT
|
1347
|
+
```
|
1262
1348
|
|
1263
1349
|
```bash
|
1264
|
-
ascli config proxy_check --fpac
|
1350
|
+
ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
|
1265
1351
|
PROXY proxy.example.com:8080
|
1266
1352
|
```
|
1267
1353
|
|
1268
|
-
|
1354
|
+
```bash
|
1355
|
+
ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.com
|
1356
|
+
PROXY proxy.example.com:8080
|
1357
|
+
```
|
1269
1358
|
|
1270
1359
|
### <a id="client"></a>FASP configuration
|
1271
1360
|
|
@@ -1411,14 +1500,14 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
|
1411
1500
|
|
1412
1501
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
1413
1502
|
|
1414
|
-
When a transfer needs to be started, a [
|
1415
|
-
This [
|
1503
|
+
When a transfer needs to be started, a [*transfer-spec*](#transferspec) has been internally prepared.
|
1504
|
+
This [*transfer-spec*](#transferspec) will be executed by a transfer client, here called "Transfer Agent".
|
1416
1505
|
|
1417
1506
|
There are currently 3 agents:
|
1418
1507
|
|
1419
1508
|
* [`direct`](#agt_direct) : a local execution of `ascp`
|
1420
1509
|
* [`connect`](#agt_connect) : use of a local Connect Client
|
1421
|
-
* [`node`](#agt_node) : use of an Aspera Transfer Node (potentially
|
1510
|
+
* [`node`](#agt_node) : use of an Aspera Transfer Node (potentially *remote*).
|
1422
1511
|
* [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
|
1423
1512
|
* [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
|
1424
1513
|
|
@@ -1426,7 +1515,7 @@ Note that all transfer operation are seen from the point of view of the agent.
|
|
1426
1515
|
For instance, a node agent making an "upload", or "package send" operation,
|
1427
1516
|
will effectively push files to the related server from the agent node.
|
1428
1517
|
|
1429
|
-
`ascli` standardizes on the use of a [
|
1518
|
+
`ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *raw* ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1430
1519
|
|
1431
1520
|
#### <a id="agt_direct"></a>Direct
|
1432
1521
|
|
@@ -1467,12 +1556,6 @@ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
|
|
1467
1556
|
ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
|
1468
1557
|
```
|
1469
1558
|
|
1470
|
-
To specify a FASP proxy (only supported with the `direct` agent), set the appropriate [_transfer-spec_](#transferspec) parameter:
|
1471
|
-
|
1472
|
-
* `EX_fasp_proxy_url`
|
1473
|
-
* `EX_http_proxy_url` (proxy for legacy http fallback)
|
1474
|
-
* `EX_ascp_args`
|
1475
|
-
|
1476
1559
|
#### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
|
1477
1560
|
|
1478
1561
|
By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
|
@@ -1483,7 +1566,6 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
|
|
1483
1566
|
Transfer Server using the Node API, either on a local or remote node.
|
1484
1567
|
Parameters provided in option `transfer_info` are:
|
1485
1568
|
|
1486
|
-
<table>
|
1487
1569
|
<tr><th>Name</th><th>Type</th><th>Description</th></tr>
|
1488
1570
|
<tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
|
1489
1571
|
<tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
|
@@ -1493,7 +1575,7 @@ Parameters provided in option `transfer_info` are:
|
|
1493
1575
|
|
1494
1576
|
Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
|
1495
1577
|
`--transfer-info=@preset:<psetname>` or be specified using the extended value syntax :
|
1496
|
-
`--transfer-info=@json:'{"url":"https://...","username":"theuser","password":"
|
1578
|
+
`--transfer-info=@json:'{"url":"https://...","username":"theuser","password":"_pass_here_"}'`
|
1497
1579
|
|
1498
1580
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
1499
1581
|
|
@@ -1520,7 +1602,7 @@ By default it will listen on local port `55002` on `127.0.0.1`.
|
|
1520
1602
|
### <a id="transferspec"></a>Transfer Specification
|
1521
1603
|
|
1522
1604
|
Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
|
1523
|
-
is described in a
|
1605
|
+
is described in a [*transfer-spec*](#transferspec) (Transfer Specification), such as:
|
1524
1606
|
|
1525
1607
|
* server address
|
1526
1608
|
* transfer user name
|
@@ -1528,23 +1610,23 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
|
|
1528
1610
|
* file list
|
1529
1611
|
* etc...
|
1530
1612
|
|
1531
|
-
`ascli` builds a default
|
1613
|
+
`ascli` builds a default [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
|
1532
1614
|
|
1533
|
-
If needed, it is possible to modify or add any of the supported
|
1615
|
+
If needed, it is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters. Multiple `ts` options on command line are cumulative.
|
1534
1616
|
|
1535
|
-
It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [
|
1617
|
+
It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
|
1536
1618
|
|
1537
|
-
The use of a
|
1619
|
+
The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has the advantage of:
|
1538
1620
|
|
1539
1621
|
* common to all [Transfer Agent](#agents)
|
1540
1622
|
* not dependent on command line limitations (special characters...)
|
1541
1623
|
|
1542
|
-
A [
|
1624
|
+
A [*transfer-spec*](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
|
1543
1625
|
|
1544
1626
|
### <a id="transferparams"></a>Transfer Parameters
|
1545
1627
|
|
1546
|
-
All standard
|
1547
|
-
[
|
1628
|
+
All standard [*transfer-spec*](#transferspec) parameters can be specified.
|
1629
|
+
[*transfer-spec*](#transferspec) can also be saved/overridden in the config file.
|
1548
1630
|
|
1549
1631
|
References:
|
1550
1632
|
|
@@ -1569,7 +1651,7 @@ Columns:
|
|
1569
1651
|
|
1570
1652
|
Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
|
1571
1653
|
|
1572
|
-
<table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>none, aes-128, aes-256<br/>Allowed values: aes128, aes192, aes256, aes128cfb, aes192cfb, aes256cfb, aes128gcm, aes192gcm, aes256gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side content protection. (encryption at rest)<br/>Allowed values: encrypt, decrypt</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password.</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer (older,string)<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td> </td><td>Y</td><td> </td><td>Remove SRC files after transfer success</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>http_fallback</td><td>string<br/>bool</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a fasp transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td> </td><td> </td><td>Specifies http port.<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port.</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td> </td><td>in bytes<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Required. Contains a path to the source (required) and a path to the destination.</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_access_key</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Node only?</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td> </td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies ssh (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td> </td><td> </td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIVATE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links by:<br/> following the link (follow),<br/> copying the link itself (copy),<br/> skipping (skip),<br/> or forcibly copying the link itself (copy+force).<br/> Default: follow<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td> </td><td> </td><td>Y</td><td>Returned by upload/download_setup node api.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td> </td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>specify version of protocol</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td> </td><td> </td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td> </td><td> </td><td>Preserve access control lists.<br/>Allowable values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td> </td><td> </td><td>TODO: check node, sdk<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Passphrase used for at rest encryption or decryption<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td> </td><td> </td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td> </td><td> </td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td> </td><td> </td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>rate_policy_allowed</td><td>string</td><td> </td><td> </td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/> Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>authentication</td><td>string</td><td> </td><td> </td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td> </td><td> </td><td>Add command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr></table>
|
1654
|
+
<table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt)</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source. The source and destination arguments must be directories that have matching names. Objects on the destination that have the same name but different type or size as objects on the source are not deleted.<br/>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td> </td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr><tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td> </td><td> </td><td>Specifies http port when no cipher is used<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t)</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td> </td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value. (0=no file is split)<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td> </td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>source_root_id</td><td>string</td><td> </td><td>Y</td><td> </td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td> </td><td> </td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td> </td><td> </td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td> </td><td> </td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td> </td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>specify version of protocol</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td> </td><td> </td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td> </td><td> </td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td> </td><td> </td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr><tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td> </td><td> </td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr><tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td> </td><td> </td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr><tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td> </td><td> </td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td> </td><td> </td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Passphrase used for at rest encryption or decryption. Prefer to use standard: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td> </td><td> </td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td> </td><td> </td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td> </td><td> </td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>rate_policy_allowed</td><td>string</td><td> </td><td> </td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>authentication</td><td>string</td><td> </td><td> </td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td> </td><td> </td><td>Add native command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr></table>
|
1573
1655
|
|
1574
1656
|
#### Destination folder for transfers
|
1575
1657
|
|
@@ -1578,7 +1660,7 @@ The destination folder is set by `ascli` by default to:
|
|
1578
1660
|
* `.` for downloads
|
1579
1661
|
* `/` for uploads
|
1580
1662
|
|
1581
|
-
It is specified by the [
|
1663
|
+
It is specified by the [*transfer-spec*](#transferspec) parameter `destination_root`.
|
1582
1664
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
1583
1665
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
1584
1666
|
`--to-folder=<path>` .
|
@@ -1598,10 +1680,10 @@ This is equivalent to:
|
|
1598
1680
|
ascli server upload --sources=@args ~/mysample.file secondfile
|
1599
1681
|
```
|
1600
1682
|
|
1601
|
-
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [
|
1683
|
+
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [*transfer-spec*](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
|
1602
1684
|
|
1603
1685
|
Note that this is different from the "ascp" command line. The paradigm used by `ascli` is:
|
1604
|
-
all transfer parameters are kept in [
|
1686
|
+
all transfer parameters are kept in [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
|
1605
1687
|
|
1606
1688
|
For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
|
1607
1689
|
|
@@ -1633,13 +1715,13 @@ providing a file list directly to ascp:
|
|
1633
1715
|
... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
|
1634
1716
|
```
|
1635
1717
|
|
1636
|
-
* Not recommended: It is possible to specify bare ascp arguments using the pseudo [
|
1718
|
+
* Not recommended: It is possible to specify bare ascp arguments using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`.
|
1637
1719
|
|
1638
1720
|
```javascript
|
1639
1721
|
--sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
|
1640
1722
|
```
|
1641
1723
|
|
1642
|
-
This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [
|
1724
|
+
This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [*transfer-spec*](#transferspec), which will be overridden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
|
1643
1725
|
|
1644
1726
|
In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
|
1645
1727
|
|
@@ -1690,6 +1772,29 @@ Multi-session spawn is done by `ascli`.
|
|
1690
1772
|
|
1691
1773
|
When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
|
1692
1774
|
|
1775
|
+
#### Content protection
|
1776
|
+
|
1777
|
+
Also known as Client-side encryption at reast (CSEAR), content protection allows a client to send files to a server
|
1778
|
+
which will store them encrypted (upload), and decrypt files as they are being downloaded from a server, both
|
1779
|
+
using a passphrase, only known by users sharing files. Files stay encrypted on server side.
|
1780
|
+
|
1781
|
+
activating CSEAR consists in using transfer spec parameters:
|
1782
|
+
|
1783
|
+
* `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
|
1784
|
+
* `content_protection_password` : the passphrase to be used.
|
1785
|
+
|
1786
|
+
Example: parameter to download a faspex package and decrypt on the fly
|
1787
|
+
|
1788
|
+
```json
|
1789
|
+
--ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
|
1790
|
+
```
|
1791
|
+
|
1792
|
+
Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
|
1793
|
+
|
1794
|
+
```json
|
1795
|
+
--ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
|
1796
|
+
```
|
1797
|
+
|
1693
1798
|
#### Transfer Spec Examples
|
1694
1799
|
|
1695
1800
|
* Change target rate
|
@@ -1824,15 +1929,14 @@ where:
|
|
1824
1929
|
* `dirname` is the folder name and can contain `/` to specify a subfolder.
|
1825
1930
|
* supported arguments are:
|
1826
1931
|
|
1827
|
-
|
1828
|
-
|
1829
|
-
|
1830
|
-
|
1831
|
-
|
1832
|
-
|
1833
|
-
|
1834
|
-
|
1835
|
-
</table>
|
1932
|
+
| Name | Type | Description |
|
1933
|
+
|--------|------|-------------|
|
1934
|
+
|count |int |mandatory|Number of files<br/>Mandatory|
|
1935
|
+
|file |string|Basename for files<br>Default: "file"|
|
1936
|
+
|size |int |Size of first file.<br>Default: 0|
|
1937
|
+
|inc |int |Increment applied to determine next file size<br>Default: 0|
|
1938
|
+
|seq |enum |Sequence in determining next file size<br/>Values: random, sequential<br/>Default: sequential|
|
1939
|
+
|buf_init|enum |How source data is initialized<br/>Option 'none' is not allowed for downloads.<br/>Values:none, zero, random<br/>Default:zero|
|
1836
1940
|
|
1837
1941
|
The sequence parameter is applied as follows:
|
1838
1942
|
|
@@ -1921,16 +2025,16 @@ ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_sec
|
|
1921
2025
|
ascli aoc admin resource node v3 name my_aoc_node1_name --secret=my_aoc_node1_secret access_key delete testsub1
|
1922
2026
|
ascli aoc admin resource workspace list
|
1923
2027
|
ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
1924
|
-
ascli aoc automation workflow
|
2028
|
+
ascli aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
|
1925
2029
|
ascli aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
|
1926
|
-
ascli aoc automation workflow delete
|
2030
|
+
ascli aoc automation workflow delete my_wf_id
|
1927
2031
|
ascli aoc automation workflow list
|
1928
|
-
ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
|
2032
|
+
ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
|
1929
2033
|
ascli aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
|
1930
2034
|
ascli aoc bearer_token --display=data --scope=user:all
|
1931
2035
|
ascli aoc faspex
|
1932
2036
|
ascli aoc files bearer /
|
1933
|
-
ascli aoc files bearer_token_node /
|
2037
|
+
ascli aoc files bearer_token_node / --cache-tokens=no
|
1934
2038
|
ascli aoc files browse /
|
1935
2039
|
ascli aoc files browse / -N --link=my_aoc_publink_folder
|
1936
2040
|
ascli aoc files delete /testsrc
|
@@ -1981,13 +2085,14 @@ ascli ats cluster list
|
|
1981
2085
|
ascli ats cluster show --cloud=aws --region=eu-west-1
|
1982
2086
|
ascli ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
1983
2087
|
ascli conf flush_tokens
|
1984
|
-
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath=
|
1985
|
-
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath=
|
2088
|
+
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes
|
2089
|
+
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
|
1986
2090
|
ascli config ascp connect info 'Aspera Connect for Windows'
|
1987
2091
|
ascli config ascp connect list
|
1988
2092
|
ascli config ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
|
1989
2093
|
ascli config ascp connect version 'Aspera Connect for Windows' list
|
1990
2094
|
ascli config ascp connect version 'Aspera Connect for Windows' open documentation
|
2095
|
+
ascli config ascp errors
|
1991
2096
|
ascli config ascp info
|
1992
2097
|
ascli config ascp install
|
1993
2098
|
ascli config ascp products list
|
@@ -2003,7 +2108,7 @@ ascli config export
|
|
2003
2108
|
ascli config genkey mykey
|
2004
2109
|
ascli config plugin create mycommand T
|
2005
2110
|
ascli config plugin list
|
2006
|
-
ascli config proxy_check --fpac
|
2111
|
+
ascli config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com
|
2007
2112
|
ascli console transfer current list
|
2008
2113
|
ascli console transfer smart list
|
2009
2114
|
ascli console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
|
@@ -2013,23 +2118,44 @@ ascli cos node access_key show self
|
|
2013
2118
|
ascli cos node download testfile.bin --to-folder=.
|
2014
2119
|
ascli cos node info
|
2015
2120
|
ascli cos node upload testfile.bin
|
2121
|
+
ascli faspex dropbox list --recipient="*my_faspex_dbx"
|
2122
|
+
ascli faspex dropbox list --recipient="*my_faspex_wkg"
|
2016
2123
|
ascli faspex health
|
2017
2124
|
ascli faspex package list
|
2018
|
-
ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
2019
|
-
ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
2125
|
+
ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
2126
|
+
ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
2127
|
+
ascli faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
2128
|
+
ascli faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
2020
2129
|
ascli faspex package recv "my_package_id" --to-folder=.
|
2021
2130
|
ascli faspex package recv "my_package_id" --to-folder=. --box=sent
|
2022
|
-
ascli faspex package recv --to-folder=. "my_package_id"
|
2023
2131
|
ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
|
2024
2132
|
ascli faspex package recv ALL --to-folder=. --once-only=yes
|
2133
|
+
ascli faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
|
2134
|
+
ascli faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
|
2135
|
+
ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
|
2136
|
+
ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
|
2025
2137
|
ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
|
2026
2138
|
ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
2027
2139
|
ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
2028
2140
|
ascli faspex source name "Server Files" node br /
|
2029
|
-
ascli
|
2141
|
+
ascli faspex v4 dmembership list
|
2142
|
+
ascli faspex v4 dropbox list
|
2143
|
+
ascli faspex v4 metadata_profile list
|
2144
|
+
ascli faspex v4 user list
|
2145
|
+
ascli faspex v4 wmembership list
|
2146
|
+
ascli faspex v4 workgroup list
|
2147
|
+
ascli faspex5 admin res accounts list
|
2148
|
+
ascli faspex5 admin res contacts list
|
2149
|
+
ascli faspex5 admin res jobs list
|
2150
|
+
ascli faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
|
2151
|
+
ascli faspex5 admin res oauth_clients list
|
2152
|
+
ascli faspex5 admin res registrations list
|
2153
|
+
ascli faspex5 admin res saml_configs list
|
2154
|
+
ascli faspex5 admin res shared_inboxes list
|
2155
|
+
ascli faspex5 admin res workgroups list
|
2030
2156
|
ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
|
2031
2157
|
ascli faspex5 package receive "my_package_id" --to-folder=.
|
2032
|
-
ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"
|
2158
|
+
ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin
|
2033
2159
|
ascli mycommand --plugin-folder=T
|
2034
2160
|
ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
|
2035
2161
|
ascli node -N -Ptst_node_preview access_key delete aoc_1
|
@@ -2042,17 +2168,19 @@ ascli node async show ALL
|
|
2042
2168
|
ascli node basic_token
|
2043
2169
|
ascli node browse / -r
|
2044
2170
|
ascli node delete folder_1/10MB.1
|
2045
|
-
ascli node delete
|
2046
|
-
ascli node download --to-folder=.
|
2171
|
+
ascli node delete testfile.bin
|
2172
|
+
ascli node download testfile.bin --to-folder=.
|
2173
|
+
ascli node download testfile.bin --to-folder=. --token-type=hybrid
|
2047
2174
|
ascli node health
|
2048
|
-
ascli node info
|
2175
|
+
ascli node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
2049
2176
|
ascli node search / --value=@json:'{"sort":"mtime"}'
|
2050
2177
|
ascli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
|
2051
2178
|
ascli node service delete service1
|
2052
2179
|
ascli node service list
|
2053
2180
|
ascli node transfer list --value=@json:'{"active_only":true}'
|
2054
2181
|
ascli node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
|
2055
|
-
ascli node upload --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
|
2182
|
+
ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
|
2183
|
+
ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
|
2056
2184
|
ascli orchestrator info
|
2057
2185
|
ascli orchestrator plugins
|
2058
2186
|
ascli orchestrator processes
|
@@ -2098,22 +2226,22 @@ ascli server mv folder_1/200KB.2 folder_1/to.delete
|
|
2098
2226
|
ascli server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
|
2099
2227
|
ascli server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
|
2100
2228
|
ascli server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
|
2229
|
+
ascli server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
|
2101
2230
|
ascli server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
|
2102
2231
|
ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
|
2103
2232
|
ascli shares admin share list
|
2104
2233
|
ascli shares repository browse /
|
2105
2234
|
ascli shares repository delete my_shares_upload/testfile.bin
|
2106
2235
|
ascli shares repository download --to-folder=. my_shares_upload/testfile.bin
|
2107
|
-
ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
2236
|
+
ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
|
2108
2237
|
ascli shares repository upload --to-folder=my_shares_upload testfile.bin
|
2109
|
-
ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
2238
|
+
ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
|
2110
2239
|
ascli shares2 appinfo
|
2111
2240
|
ascli shares2 organization list
|
2112
2241
|
ascli shares2 project list --organization=Sport
|
2113
2242
|
ascli shares2 repository browse /
|
2114
2243
|
ascli shares2 userinfo
|
2115
2244
|
ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
|
2116
|
-
|
2117
2245
|
...and more
|
2118
2246
|
```
|
2119
2247
|
|
@@ -2122,7 +2250,7 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
|
|
2122
2250
|
```bash
|
2123
2251
|
ascli -h
|
2124
2252
|
NAME
|
2125
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
2253
|
+
ascli -- a command line tool for Aspera Applications (v4.7.0)
|
2126
2254
|
|
2127
2255
|
SYNOPSIS
|
2128
2256
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -2131,12 +2259,12 @@ DESCRIPTION
|
|
2131
2259
|
Use Aspera application to perform operations on command line.
|
2132
2260
|
Documentation and examples: https://rubygems.org/gems/aspera-cli
|
2133
2261
|
execute: ascli conf doc
|
2134
|
-
or visit:
|
2262
|
+
or visit: https://www.rubydoc.info/gems/aspera-cli
|
2135
2263
|
source repo: https://github.com/IBM/aspera-cli
|
2136
2264
|
|
2137
2265
|
ENVIRONMENT VARIABLES
|
2138
|
-
ASCLI_HOME
|
2139
|
-
|
2266
|
+
ASCLI_HOME config folder, default: $HOME/.aspera/ascli
|
2267
|
+
Any option can be set as an environment variable, refer to the manual
|
2140
2268
|
|
2141
2269
|
COMMANDS
|
2142
2270
|
To list first level commands, execute: ascli
|
@@ -2145,56 +2273,58 @@ COMMANDS
|
|
2145
2273
|
OPTIONS
|
2146
2274
|
Options begin with a '-' (minus), and value is provided on command line.
|
2147
2275
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
2148
|
-
base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, stdin, preset
|
2276
|
+
base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, uri, stdin, preset
|
2149
2277
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
2150
2278
|
|
2151
2279
|
ARGS
|
2152
2280
|
Some commands require mandatory arguments, e.g. a path.
|
2153
2281
|
|
2154
2282
|
OPTIONS: global
|
2155
|
-
--interactive=ENUM use interactive input of missing params: yes, no
|
2156
|
-
--ask-options=ENUM ask even optional options: yes, no
|
2157
|
-
--format=ENUM output format: table, ruby, json, jsonpp, yaml, csv, nagios
|
2158
|
-
--display=ENUM output only some information: info, data, error
|
2283
|
+
--interactive=ENUM use interactive input of missing params: yes, [no]
|
2284
|
+
--ask-options=ENUM ask even optional options: yes, [no]
|
2285
|
+
--format=ENUM output format: [table], ruby, json, jsonpp, yaml, csv, nagios
|
2286
|
+
--display=ENUM output only some information: [info], data, error
|
2159
2287
|
--fields=VALUE comma separated list of fields, or ALL, or DEF
|
2160
2288
|
--select=VALUE select only some items in lists, extended value: hash (column, value)
|
2161
2289
|
--table-style=VALUE table display style
|
2162
|
-
--flat-hash=ENUM display hash values as additional keys: yes, no
|
2163
|
-
--transpose-single=ENUM single object fields output vertically: yes, no
|
2290
|
+
--flat-hash=ENUM display hash values as additional keys: [yes], no
|
2291
|
+
--transpose-single=ENUM single object fields output vertically: [yes], no
|
2292
|
+
--show-secrets=ENUM show secrets on command output: [yes], no
|
2164
2293
|
-h, --help Show this message.
|
2165
2294
|
--bash-comp generate bash completion for command
|
2166
2295
|
--show-config Display parameters used for the provided action.
|
2167
2296
|
-r, --rest-debug more debug for HTTP calls
|
2168
2297
|
-v, --version display version
|
2169
2298
|
-w, --warnings check for language warnings
|
2170
|
-
--ui=ENUM method to start browser: text, graphical
|
2171
|
-
--log-level=ENUM Log level: debug, info, warn, error, fatal, unknown
|
2172
|
-
--logger=ENUM log method: stderr, stdout, syslog
|
2299
|
+
--ui=ENUM method to start browser: text, [graphical]
|
2300
|
+
--log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
|
2301
|
+
--logger=ENUM log method: [stderr], stdout, syslog
|
2173
2302
|
--lock-port=VALUE prevent dual execution of a command, e.g. in cron
|
2174
2303
|
--query=VALUE additional filter for API calls (extended value) (some commands)
|
2175
2304
|
--http-options=VALUE options for http socket (extended value)
|
2176
|
-
--insecure=ENUM do not validate HTTPS certificate: yes, no
|
2177
|
-
--once-only=ENUM process only new items (some commands): yes, no
|
2178
|
-
--log-
|
2305
|
+
--insecure=ENUM do not validate HTTPS certificate: [yes], no
|
2306
|
+
--once-only=ENUM process only new items (some commands): yes, [no]
|
2307
|
+
--log-secrets=ENUM show passwords in logs: yes, [no]
|
2308
|
+
--cache-tokens=ENUM save and reuse Oauth tokens: [yes], no
|
2179
2309
|
|
2180
2310
|
COMMAND: config
|
2181
|
-
SUBCOMMANDS: list overview id preset open documentation genkey
|
2311
|
+
SUBCOMMANDS: list overview id preset open documentation genkey gem plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
|
2182
2312
|
OPTIONS:
|
2183
2313
|
--value=VALUE extended value for create, update, list filter
|
2184
2314
|
--property=VALUE name of property to set
|
2185
2315
|
--id=VALUE resource identifier (modify,delete,show)
|
2186
|
-
--config-file=VALUE read parameters from file in YAML format, current=/
|
2316
|
+
--config-file=VALUE read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
|
2187
2317
|
-N, --no-default do not load default configuration for plugin
|
2188
|
-
--override=ENUM Wizard: override existing value: yes, no
|
2189
|
-
--use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, no
|
2190
|
-
--default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, no
|
2191
|
-
--test-mode=ENUM Wizard: skip private key check step: yes, no
|
2318
|
+
--override=ENUM Wizard: override existing value: yes, [no]
|
2319
|
+
--use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, [no]
|
2320
|
+
--default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, [no]
|
2321
|
+
--test-mode=ENUM Wizard: skip private key check step: yes, [no]
|
2192
2322
|
-P, --presetVALUE load the named option preset from current config file
|
2193
2323
|
--pkeypath=VALUE Wizard: path to private key for JWT
|
2194
2324
|
--ascp-path=VALUE path to ascp
|
2195
2325
|
--use-product=VALUE use ascp from specified product
|
2196
2326
|
--smtp=VALUE smtp configuration (extended value: hash)
|
2197
|
-
--fpac=VALUE proxy auto configuration
|
2327
|
+
--fpac=VALUE proxy auto configuration script
|
2198
2328
|
--secret=VALUE default secret
|
2199
2329
|
--secrets=VALUE secret vault
|
2200
2330
|
--sdk-url=VALUE URL to get SDK
|
@@ -2241,7 +2371,7 @@ OPTIONS:
|
|
2241
2371
|
--password=VALUE user's password
|
2242
2372
|
--params=VALUE parameters hash table, use @json:{"param":"value"}
|
2243
2373
|
--result=VALUE specify result value as: 'work step:parameter'
|
2244
|
-
--synchronous=ENUM work step:parameter expected as result: yes, no
|
2374
|
+
--synchronous=ENUM work step:parameter expected as result: yes, [no]
|
2245
2375
|
--ret-style=ENUM how return type is requested in api: header, arg, ext
|
2246
2376
|
--auth-style=ENUM authentication type: arg_pass, head_basic, apikey
|
2247
2377
|
|
@@ -2275,7 +2405,7 @@ OPTIONS:
|
|
2275
2405
|
|
2276
2406
|
|
2277
2407
|
COMMAND: faspex5
|
2278
|
-
SUBCOMMANDS:
|
2408
|
+
SUBCOMMANDS: package admin
|
2279
2409
|
OPTIONS:
|
2280
2410
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2281
2411
|
--username=VALUE username to log in
|
@@ -2283,14 +2413,14 @@ OPTIONS:
|
|
2283
2413
|
--client-id=VALUE OAuth client identifier
|
2284
2414
|
--client-secret=VALUE OAuth client secret
|
2285
2415
|
--redirect-uri=VALUE OAuth redirect URI
|
2286
|
-
--auth=ENUM OAuth type of authentication:
|
2416
|
+
--auth=ENUM OAuth type of authentication: web, jwt, boot
|
2287
2417
|
--private-key=VALUE Oauth RSA private key PEM value for JWT (prefix file path with @val:@file:)
|
2288
2418
|
|
2289
2419
|
|
2290
2420
|
COMMAND: cos
|
2291
2421
|
SUBCOMMANDS: node
|
2292
2422
|
OPTIONS:
|
2293
|
-
--bucket=VALUE IBM Cloud Object
|
2423
|
+
--bucket=VALUE IBM Cloud Object Storage bucket name
|
2294
2424
|
--endpoint=VALUE storage endpoint url
|
2295
2425
|
--apikey=VALUE storage API key
|
2296
2426
|
--crn=VALUE ressource instance id
|
@@ -2312,17 +2442,6 @@ OPTIONS:
|
|
2312
2442
|
--box=ENUM package box: inbox, archive, sent
|
2313
2443
|
|
2314
2444
|
|
2315
|
-
COMMAND: shares2
|
2316
|
-
SUBCOMMANDS: repository organization project team share appinfo userinfo admin
|
2317
|
-
OPTIONS:
|
2318
|
-
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2319
|
-
--username=VALUE username to log in
|
2320
|
-
--password=VALUE user's password
|
2321
|
-
--organization=VALUE organization
|
2322
|
-
--project=VALUE project
|
2323
|
-
--share=VALUE share
|
2324
|
-
|
2325
|
-
|
2326
2445
|
COMMAND: preview
|
2327
2446
|
SUBCOMMANDS: scan events trevents check test
|
2328
2447
|
OPTIONS:
|
@@ -2330,7 +2449,7 @@ OPTIONS:
|
|
2330
2449
|
--username=VALUE username to log in
|
2331
2450
|
--password=VALUE user's password
|
2332
2451
|
--skip-format=ENUM skip this preview format (multiple possible): png, mp4
|
2333
|
-
--folder-reset-cache=ENUM force detection of generated preview by refresh cache: no, header, read
|
2452
|
+
--folder-reset-cache=ENUM force detection of generated preview by refresh cache: [no], header, read
|
2334
2453
|
--skip-types=VALUE skip types in comma separated list
|
2335
2454
|
--previews-folder=VALUE preview folder in storage root
|
2336
2455
|
--temp-folder=VALUE path to temp folder
|
@@ -2338,15 +2457,16 @@ OPTIONS:
|
|
2338
2457
|
--case=VALUE basename of output for for test
|
2339
2458
|
--scan-path=VALUE subpath in folder id to start scan in (default=/)
|
2340
2459
|
--scan-id=VALUE forder id in storage to start scan in, default is access key main folder id
|
2341
|
-
--mimemagic=ENUM use Mime type detection of gem mimemagic: yes, no
|
2342
|
-
--overwrite=ENUM when to overwrite result file: always, never, mtime
|
2343
|
-
--file-access=ENUM how to read and write files in repository: local, remote
|
2460
|
+
--mimemagic=ENUM use Mime type detection of gem mimemagic: yes, [no]
|
2461
|
+
--overwrite=ENUM when to overwrite result file: always, never, [mtime]
|
2462
|
+
--file-access=ENUM how to read and write files in repository: [local], remote
|
2344
2463
|
--max-size=VALUE maximum size (in bytes) of preview file
|
2345
2464
|
--thumb-vid-scale=VALUE png: video: size (ffmpeg scale argument)
|
2346
2465
|
--thumb-vid-fraction=VALUE png: video: position of snapshot
|
2347
2466
|
--thumb-img-size=VALUE png: non-video: height (and width)
|
2348
|
-
--
|
2349
|
-
--video-
|
2467
|
+
--thumb-text-font=VALUE png: plaintext: font to render text with image magick convert, list with: identify -list font
|
2468
|
+
--video-conversion=ENUM mp4: method for preview generation: [reencode], blend, clips
|
2469
|
+
--video-png-conv=ENUM mp4: method for thumbnail generation: [fixed], animated
|
2350
2470
|
--video-start-sec=VALUE mp4: start offset (seconds) of video preview
|
2351
2471
|
--video-scale=VALUE mp4: video scale (ffmpeg)
|
2352
2472
|
--blend-keyframes=VALUE mp4: blend: # key frames
|
@@ -2370,7 +2490,7 @@ OPTIONS:
|
|
2370
2490
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2371
2491
|
--username=VALUE username to log in
|
2372
2492
|
--password=VALUE user's password
|
2373
|
-
--auth=ENUM OAuth type of authentication:
|
2493
|
+
--auth=ENUM OAuth type of authentication: web, jwt
|
2374
2494
|
--operation=ENUM client operation for transfers: push, pull
|
2375
2495
|
--client-id=VALUE OAuth API client identifier in application
|
2376
2496
|
--client-secret=VALUE OAuth API client passcode
|
@@ -2383,8 +2503,8 @@ OPTIONS:
|
|
2383
2503
|
--new-user-option=VALUE new user creation option
|
2384
2504
|
--from-folder=VALUE share to share source folder
|
2385
2505
|
--scope=VALUE OAuth scope for AoC API calls
|
2386
|
-
--bulk=ENUM bulk operation: yes, no
|
2387
|
-
--default-ports=ENUM use standard FASP ports or get from node api: yes, no
|
2506
|
+
--bulk=ENUM bulk operation: yes, [no]
|
2507
|
+
--default-ports=ENUM use standard FASP ports or get from node api: yes, [no]
|
2388
2508
|
|
2389
2509
|
|
2390
2510
|
COMMAND: server
|
@@ -2442,11 +2562,17 @@ You can test with:
|
|
2442
2562
|
ascli aoc user profile show
|
2443
2563
|
```
|
2444
2564
|
|
2445
|
-
Optionally, it is possible to create a new organization-specific "integration".
|
2565
|
+
Optionally, it is possible to create a new organization-specific "integration", i.e. client application identification.
|
2446
2566
|
For this, specify the option: `--use-generic-client=no`.
|
2447
2567
|
|
2448
2568
|
This will guide you through the steps to create.
|
2449
2569
|
|
2570
|
+
If the wizard does not detect the application but you know the application, you can force it using option `value`:
|
2571
|
+
|
2572
|
+
```bash
|
2573
|
+
ascli config wizard --value=aoc
|
2574
|
+
```
|
2575
|
+
|
2450
2576
|
### <a id="aocmanual"></a>Configuration: using manual setup
|
2451
2577
|
|
2452
2578
|
If you used the wizard (recommended): skip this section.
|
@@ -2461,7 +2587,7 @@ Several types of OAuth authentication are supported:
|
|
2461
2587
|
|
2462
2588
|
The authentication method is controlled by option `auth`.
|
2463
2589
|
|
2464
|
-
For a
|
2590
|
+
For a *quick start*, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
|
2465
2591
|
|
2466
2592
|
For a more convenient, browser-less, experience follow the [JWT](#jwt) section (auth=jwt) in addition to Client Registration.
|
2467
2593
|
|
@@ -2500,7 +2626,7 @@ Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interacti
|
|
2500
2626
|
```bash
|
2501
2627
|
ascli config preset ask my_aoc_org url client_id client_secret
|
2502
2628
|
option: url> https://myorg.ibmaspera.com/
|
2503
|
-
option: client_id>
|
2629
|
+
option: client_id> my_BJbQiFw
|
2504
2630
|
option: client_secret> yFS1mu-crbKuQhGFtfhYuoRW...
|
2505
2631
|
updated: my_aoc_org
|
2506
2632
|
```
|
@@ -2519,37 +2645,8 @@ Note: Default `auth` method is `web` and default `redirect_uri` is `http://local
|
|
2519
2645
|
|
2520
2646
|
For a Browser-less, Private Key-based authentication, use the following steps.
|
2521
2647
|
|
2522
|
-
##### Key Pair Generation
|
2523
|
-
|
2524
2648
|
In order to use JWT for Aspera on Cloud API client authentication,
|
2525
|
-
a private/public key pair must be
|
2526
|
-
This can be done using any of the following method:
|
2527
|
-
|
2528
|
-
(TODO: add passphrase protection as option).
|
2529
|
-
|
2530
|
-
* using the CLI:
|
2531
|
-
|
2532
|
-
```bash
|
2533
|
-
ascli config genkey ~/.aspera/ascli/aocapikey
|
2534
|
-
```
|
2535
|
-
|
2536
|
-
* `ssh-keygen`:
|
2537
|
-
|
2538
|
-
```bash
|
2539
|
-
ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
|
2540
|
-
```
|
2541
|
-
|
2542
|
-
* `openssl`
|
2543
|
-
|
2544
|
-
(on some openssl implementation (mac) there is option: -nodes (no DES))
|
2545
|
-
|
2546
|
-
```bash
|
2547
|
-
APIKEY=~/.aspera/ascli/aocapikey
|
2548
|
-
openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
|
2549
|
-
openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
|
2550
|
-
openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
|
2551
|
-
rm -f ${APIKEY}.protected
|
2552
|
-
```
|
2649
|
+
a [private/public key pair](#private_key) must be used (without passphrase)
|
2553
2650
|
|
2554
2651
|
##### API Client JWT activation
|
2555
2652
|
|
@@ -2574,12 +2671,12 @@ ascli aoc admin res client list
|
|
2574
2671
|
:............:...............:
|
2575
2672
|
: id : name :
|
2576
2673
|
:............:...............:
|
2577
|
-
:
|
2674
|
+
: my_BJbQiFw : my-client-app :
|
2578
2675
|
:............:...............:
|
2579
2676
|
```
|
2580
2677
|
|
2581
2678
|
```javascript
|
2582
|
-
ascli aoc admin res client modify
|
2679
|
+
ascli aoc admin res client modify my_BJbQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
|
2583
2680
|
```
|
2584
2681
|
|
2585
2682
|
```output
|
@@ -2592,12 +2689,12 @@ The public key must be assigned to your user. This can be done in two manners:
|
|
2592
2689
|
|
2593
2690
|
##### Graphically
|
2594
2691
|
|
2595
|
-
|
2692
|
+
Open the previously generated public key located here: `$HOME/.aspera/ascli/my_private_key.pub`
|
2596
2693
|
|
2597
2694
|
* Open a web browser, log to your instance: https://myorg.ibmaspera.com/
|
2598
2695
|
* Click on the user's icon (top right)
|
2599
2696
|
* Select "Account Settings"
|
2600
|
-
* Paste the
|
2697
|
+
* Paste the *Public Key* in the "Public Key" section
|
2601
2698
|
* Click on "Submit"
|
2602
2699
|
|
2603
2700
|
##### Using command line
|
@@ -2616,7 +2713,7 @@ ascli aoc admin res user list
|
|
2616
2713
|
```
|
2617
2714
|
|
2618
2715
|
```ruby
|
2619
|
-
ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/
|
2716
|
+
ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/my_private_key.pub"))}'
|
2620
2717
|
```
|
2621
2718
|
|
2622
2719
|
```output
|
@@ -2636,7 +2733,7 @@ To activate default use of JWT authentication for `ascli` using the [option pres
|
|
2636
2733
|
Execute:
|
2637
2734
|
|
2638
2735
|
```bash
|
2639
|
-
ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/
|
2736
|
+
ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=laurent.martin.aspera@fr.ibm.com
|
2640
2737
|
```
|
2641
2738
|
|
2642
2739
|
Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.
|
@@ -2723,7 +2820,7 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
|
|
2723
2820
|
|
2724
2821
|
To execute an action on a specific resource, select it using one of those methods:
|
2725
2822
|
|
2726
|
-
*
|
2823
|
+
* *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
|
2727
2824
|
* give name on command line *after the action*: `aoc admin res node show name abc`
|
2728
2825
|
* provide option `id` : `aoc admin res node show --id=123`
|
2729
2826
|
* provide option `name` : `aoc admin res node show --name=abc`
|
@@ -3261,19 +3358,19 @@ f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
|
3261
3358
|
* Find files older than 1 year on a given node and store in file list
|
3262
3359
|
|
3263
3360
|
```bash
|
3264
|
-
ascli aoc admin res node --name='my node name' --secret='
|
3361
|
+
ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
|
3265
3362
|
```
|
3266
3363
|
|
3267
3364
|
* Delete the files, one by one
|
3268
3365
|
|
3269
3366
|
```bash
|
3270
|
-
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='
|
3367
|
+
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 delete "$path" ;done
|
3271
3368
|
```
|
3272
3369
|
|
3273
3370
|
* Delete the files in bulk
|
3274
3371
|
|
3275
3372
|
```bash
|
3276
|
-
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='
|
3373
|
+
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='_secret_here_' v3 delete @lines:@stdin:
|
3277
3374
|
```
|
3278
3375
|
|
3279
3376
|
## <a id="ats"></a>Plugin: Aspera Transfer Service
|
@@ -3359,13 +3456,13 @@ ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --a
|
|
3359
3456
|
Example: create access key on IBM Cloud (softlayer):
|
3360
3457
|
|
3361
3458
|
```javascript
|
3362
|
-
ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"
|
3459
|
+
ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"_secret_here_","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
|
3363
3460
|
```
|
3364
3461
|
|
3365
3462
|
Example: create access key on AWS:
|
3366
3463
|
|
3367
3464
|
```javascript
|
3368
|
-
ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"
|
3465
|
+
ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"_secret_here_"},"path":"/laurent"}}'
|
3369
3466
|
```
|
3370
3467
|
|
3371
3468
|
Example: create access key on Azure SAS:
|
@@ -3474,7 +3571,8 @@ It is possible to:
|
|
3474
3571
|
|
3475
3572
|
For transfers, it is possible to control how transfer is authorized using option: `token_type`:
|
3476
3573
|
|
3477
|
-
* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
|
3574
|
+
* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token, used as is.
|
3575
|
+
* `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
|
3478
3576
|
* `basic` : transfer spec is created like this:
|
3479
3577
|
|
3480
3578
|
```javascript
|
@@ -3483,11 +3581,11 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
3483
3581
|
"remote_user": "xfer",
|
3484
3582
|
"ssh_port": 33001,
|
3485
3583
|
"token": "Basic <base 64 encoded user/pass>",
|
3486
|
-
"direction": send/
|
3584
|
+
"direction": send/receive
|
3487
3585
|
}
|
3488
3586
|
```
|
3489
3587
|
|
3490
|
-
|
3588
|
+
Note that the port is assumed to be the default SSH port `33001` and transfer user is assumed to be `xfer`.
|
3491
3589
|
|
3492
3590
|
### Central
|
3493
3591
|
|
@@ -3505,10 +3603,10 @@ by providing the `validator` option, offline transfer validation can be done.
|
|
3505
3603
|
|
3506
3604
|
It is possible to start a FASPStream session using the node API:
|
3507
3605
|
|
3508
|
-
Use the "node stream create" command, then arguments are provided as a [
|
3606
|
+
Use the "node stream create" command, then arguments are provided as a [*transfer-spec*](#transferspec).
|
3509
3607
|
|
3510
3608
|
```javascript
|
3511
|
-
ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"
|
3609
|
+
ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"_pass_here_"}' --preset=stream
|
3512
3610
|
```
|
3513
3611
|
|
3514
3612
|
### Watchfolder
|
@@ -3573,15 +3671,63 @@ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"myst
|
|
3573
3671
|
|
3574
3672
|
## Plugin: IBM Aspera Faspex5
|
3575
3673
|
|
3674
|
+
This is currently in beta, limited operations are supported.
|
3675
|
+
|
3676
|
+
This was tested with version Beta 5.
|
3677
|
+
|
3678
|
+
The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "IBM Aspera Faspex API".
|
3679
|
+
|
3680
|
+
### Faspex 5 authentication
|
3681
|
+
|
3576
3682
|
3 authentication methods are supported:
|
3577
3683
|
|
3578
3684
|
* jwt
|
3579
3685
|
* web
|
3580
3686
|
* boot
|
3581
3687
|
|
3582
|
-
|
3688
|
+
#### Faspex 5 using JWT
|
3689
|
+
|
3690
|
+
This is the preferred method to use.
|
3691
|
+
|
3692
|
+
For JWT, create an API client in Faspex with JWT support:
|
3693
|
+
|
3694
|
+
* Identify a private key, if you don't have any refer to section [Private Key](#private_key)
|
3695
|
+
* Navigate to the web UI: Admin → Configurations → API Clients → Create
|
3696
|
+
* Activate JWT
|
3697
|
+
* Paste public key in the appropriate section
|
3698
|
+
* Click on Create Button
|
3699
|
+
* Take note of Client Id and Secret
|
3700
|
+
|
3701
|
+
Then use options:
|
3702
|
+
|
3703
|
+
```text
|
3704
|
+
--auth=jwt
|
3705
|
+
--client-id=xxx
|
3706
|
+
--client-secret=xxx
|
3707
|
+
--username=xxx
|
3708
|
+
--password=xxx
|
3709
|
+
--private-key=@file:../path/to/key.pem
|
3710
|
+
```
|
3711
|
+
|
3712
|
+
#### Faspex 5 using web browser
|
3713
|
+
|
3714
|
+
For web method, create an API client in Faspex without JWT:
|
3715
|
+
|
3716
|
+
* Navigate to the web UI: Admin → Configurations → API Clients → Create
|
3717
|
+
* Do not Activate JWT
|
3718
|
+
* enter https://127.0.0.1:8888 in the redirect URI
|
3719
|
+
* Click on Create Button
|
3720
|
+
* Take note of Client Id
|
3583
3721
|
|
3584
|
-
|
3722
|
+
Then use options:
|
3723
|
+
|
3724
|
+
```text
|
3725
|
+
--auth=web
|
3726
|
+
--client-id=xxx
|
3727
|
+
--redirect-uri=https://127.0.0.1:8888
|
3728
|
+
```
|
3729
|
+
|
3730
|
+
#### Faspex 5 using bootstrap
|
3585
3731
|
|
3586
3732
|
For boot method: (will be removed in future)
|
3587
3733
|
|
@@ -3596,8 +3742,6 @@ Use it as password and use `--auth=boot`.
|
|
3596
3742
|
ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
|
3597
3743
|
```
|
3598
3744
|
|
3599
|
-
Ready to use Faspex5 with CLI.
|
3600
|
-
|
3601
3745
|
## Plugin: IBM Aspera Faspex (4.x)
|
3602
3746
|
|
3603
3747
|
Notes:
|
@@ -3618,7 +3762,7 @@ By default it looks in box `inbox`, but the following boxes are also supported:
|
|
3618
3762
|
A user can receive a package because the recipient is:
|
3619
3763
|
|
3620
3764
|
* the user himself (default)
|
3621
|
-
* the user is
|
3765
|
+
* the user is member of a dropbox/workgroup: filter using option `recipient` set with value `*<name of dropbox/workgroup>`
|
3622
3766
|
|
3623
3767
|
#### Option `query`
|
3624
3768
|
|
@@ -3657,10 +3801,11 @@ ascli faspex package recv --id=12345
|
|
3657
3801
|
ascli faspex package recv --link=faspe://...
|
3658
3802
|
```
|
3659
3803
|
|
3660
|
-
If the package is in a specific dropbox
|
3804
|
+
If the package is in a specific **dropbox**/**workgroup**, add option `recipient` for both the `list` and `recv` commands.
|
3661
3805
|
|
3662
3806
|
```bash
|
3663
3807
|
ascli faspex package list --recipient='*thedropboxname'
|
3808
|
+
ascli faspex package recv 125 --recipient='*thedropboxname'
|
3664
3809
|
```
|
3665
3810
|
|
3666
3811
|
if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
|
@@ -3675,7 +3820,8 @@ Example:
|
|
3675
3820
|
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
3676
3821
|
```
|
3677
3822
|
|
3678
|
-
If the recipient is a dropbox
|
3823
|
+
If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
|
3824
|
+
`"recipients":["*MyDropboxName"]`
|
3679
3825
|
|
3680
3826
|
Additional optional parameters in `delivery_info`:
|
3681
3827
|
|
@@ -3716,7 +3862,7 @@ Example:
|
|
3716
3862
|
my_faspex_conf:
|
3717
3863
|
url: https://10.25.0.3/aspera/faspex
|
3718
3864
|
username: admin
|
3719
|
-
password:
|
3865
|
+
password: MyUserPassword
|
3720
3866
|
storage:
|
3721
3867
|
testlaurent:
|
3722
3868
|
node: "@preset:my_faspex_node"
|
@@ -3724,14 +3870,14 @@ my_faspex_conf:
|
|
3724
3870
|
my_faspex_node:
|
3725
3871
|
url: https://10.25.0.3:9092
|
3726
3872
|
username: node_faspex
|
3727
|
-
password:
|
3873
|
+
password: MyNodePassword
|
3728
3874
|
```
|
3729
3875
|
|
3730
3876
|
In this example, a faspex storage named "testlaurent" exists in Faspex, and is located
|
3731
3877
|
under the docroot in "/myfiles" (this must be the same as configured in Faspex).
|
3732
3878
|
The node configuration name is "my_faspex_node" here.
|
3733
3879
|
|
3734
|
-
Note: the v4 API
|
3880
|
+
Note: the v4 API provides an API for nodes and shares.
|
3735
3881
|
|
3736
3882
|
### Automated package download (cargo)
|
3737
3883
|
|
@@ -3746,12 +3892,6 @@ ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
|
|
3746
3892
|
|
3747
3893
|
Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
|
3748
3894
|
|
3749
|
-
In Shares2, users, groups listing are paged, to display sequential pages:
|
3750
|
-
|
3751
|
-
```bash
|
3752
|
-
for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}';done
|
3753
|
-
```
|
3754
|
-
|
3755
3895
|
## Plugin: IBM Cloud Object Storage
|
3756
3896
|
|
3757
3897
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
@@ -3799,14 +3939,14 @@ It consists in the following structure:
|
|
3799
3939
|
|
3800
3940
|
```javascript
|
3801
3941
|
{
|
3802
|
-
"apikey": "
|
3942
|
+
"apikey": "_api_key_here_",
|
3803
3943
|
"cos_hmac_keys": {
|
3804
|
-
"access_key_id": "
|
3805
|
-
"secret_access_key": "
|
3944
|
+
"access_key_id": "_access_key_here_",
|
3945
|
+
"secret_access_key": "_secret_here_"
|
3806
3946
|
},
|
3807
3947
|
"endpoints": "https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints",
|
3808
|
-
"iam_apikey_description": "my description ...",
|
3809
|
-
"iam_apikey_name": "my key name",
|
3948
|
+
"iam_apikey_description": "my description _here_ ...",
|
3949
|
+
"iam_apikey_name": "my key name _here_",
|
3810
3950
|
"iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",
|
3811
3951
|
"iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/xxxxxxx.....",
|
3812
3952
|
"resource_instance_id": "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxxxxx....."
|
@@ -3904,7 +4044,6 @@ The tool requires the following external tools available in the `PATH`:
|
|
3904
4044
|
* OptiPNG : `optipng`
|
3905
4045
|
* FFmpeg : `ffmpeg` `ffprobe`
|
3906
4046
|
* Libreoffice : `libreoffice`
|
3907
|
-
* ruby gem `mimemagic`
|
3908
4047
|
|
3909
4048
|
Here shown on Redhat/CentOS.
|
3910
4049
|
|
@@ -3916,32 +4055,16 @@ To check if all tools are found properly, execute:
|
|
3916
4055
|
ascli preview check
|
3917
4056
|
```
|
3918
4057
|
|
3919
|
-
#### mimemagic
|
3920
|
-
|
3921
|
-
To benefit from extra mime type detection install gem mimemagic:
|
3922
|
-
|
3923
|
-
```bash
|
3924
|
-
gem install mimemagic
|
3925
|
-
```
|
3926
|
-
|
3927
|
-
or to install an earlier version if any problem:
|
3928
|
-
|
3929
|
-
```bash
|
3930
|
-
gem install mimemagic -v '~> 0.3.0'
|
3931
|
-
```
|
3932
|
-
|
3933
|
-
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
|
3934
|
-
|
3935
|
-
If not used, Mime type used for conversion is the one provided by the node API.
|
3936
|
-
|
3937
|
-
If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
|
3938
|
-
|
3939
4058
|
#### Image: ImageMagick and optipng
|
3940
4059
|
|
3941
4060
|
```bash
|
3942
4061
|
yum install -y ImageMagick optipng
|
3943
4062
|
```
|
3944
4063
|
|
4064
|
+
You may also install `ghostscript` which adds fonts to ImageMagick.
|
4065
|
+
Available fonts, used to generate png for text, can be listed with `magick identify -list font`.
|
4066
|
+
Prefer ImageMagick version >=7.
|
4067
|
+
|
3945
4068
|
#### Video: FFmpeg
|
3946
4069
|
|
3947
4070
|
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
@@ -4114,7 +4237,13 @@ The mp4 video preview file is only for category `video`
|
|
4114
4237
|
|
4115
4238
|
File type is primarily based on file extension detected by the node API and translated info a mime type returned by the node API.
|
4116
4239
|
|
4117
|
-
The tool can also locally detect the mime type using
|
4240
|
+
The tool can also locally detect the mime type using option `mimemagic`.
|
4241
|
+
|
4242
|
+
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
|
4243
|
+
|
4244
|
+
If not used, Mime type used for conversion is the one provided by the node API.
|
4245
|
+
|
4246
|
+
If used, the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
|
4118
4247
|
|
4119
4248
|
### Access to original files and preview creation
|
4120
4249
|
|
@@ -4149,19 +4278,19 @@ The `smtp` option is a hash table (extended value) with the following fields:
|
|
4149
4278
|
```bash
|
4150
4279
|
ascli config preset set smtp_google server smtp.google.com
|
4151
4280
|
ascli config preset set smtp_google username john@gmail.com
|
4152
|
-
ascli config preset set smtp_google password
|
4281
|
+
ascli config preset set smtp_google password _pass_here_
|
4153
4282
|
```
|
4154
4283
|
|
4155
4284
|
or
|
4156
4285
|
|
4157
4286
|
```javascript
|
4158
|
-
ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"
|
4287
|
+
ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"_pass_here_"}'
|
4159
4288
|
```
|
4160
4289
|
|
4161
4290
|
or
|
4162
4291
|
|
4163
4292
|
```bash
|
4164
|
-
ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=
|
4293
|
+
ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=_pass_here_
|
4165
4294
|
```
|
4166
4295
|
|
4167
4296
|
Set this configuration as global default, for instance:
|
@@ -4224,7 +4353,7 @@ Transfer is: <%=global_transfer_status%>
|
|
4224
4353
|
This gem comes with a second executable tool providing a simplified standardized interface
|
4225
4354
|
to start a FASP session: `asession`.
|
4226
4355
|
|
4227
|
-
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [
|
4356
|
+
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transferspec) is:
|
4228
4357
|
|
4229
4358
|
* common to Aspera Node API (HTTP POST /ops/transfer)
|
4230
4359
|
* common to Aspera Connect API (browser javascript startTransfer)
|
@@ -4234,20 +4363,20 @@ Hopefully, IBM integrates this diectly in `ascp`, and this tool is made redundan
|
|
4234
4363
|
|
4235
4364
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
4236
4365
|
|
4237
|
-
The tool expect one single argument: a [
|
4366
|
+
The tool expect one single argument: a [*transfer-spec*](#transferspec).
|
4238
4367
|
|
4239
|
-
If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [
|
4368
|
+
If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
|
4240
4369
|
|
4241
4370
|
Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
|
4242
4371
|
|
4243
4372
|
During execution, it generates all low level events, one per line, in JSON format on stdout.
|
4244
4373
|
|
4245
|
-
Note that there are special "extended" [
|
4374
|
+
Note that there are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
|
4246
4375
|
|
4247
4376
|
* `EX_loglevel` to change log level of the tool
|
4248
4377
|
* `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
4249
4378
|
|
4250
|
-
Note that in addition, many "EX_" [
|
4379
|
+
Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
|
4251
4380
|
|
4252
4381
|
### Comparison of interfaces
|
4253
4382
|
|
@@ -4262,7 +4391,7 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
|
|
4262
4391
|
### Simple session
|
4263
4392
|
|
4264
4393
|
```javascript
|
4265
|
-
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"
|
4394
|
+
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
|
4266
4395
|
|
4267
4396
|
echo "${MY_TSPEC}"|asession
|
4268
4397
|
```
|
@@ -4271,11 +4400,11 @@ echo "${MY_TSPEC}"|asession
|
|
4271
4400
|
|
4272
4401
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -> `long_parameter`
|
4273
4402
|
|
4274
|
-
This is particularly useful for a persistent session ( with the [
|
4403
|
+
This is particularly useful for a persistent session ( with the [*transfer-spec*](#transferspec) parameter: `"keepalive":true` )
|
4275
4404
|
|
4276
4405
|
```javascript
|
4277
4406
|
asession
|
4278
|
-
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"
|
4407
|
+
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_pass_here_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
|
4279
4408
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
4280
4409
|
{"type":"DONE"}
|
4281
4410
|
```
|
@@ -4348,7 +4477,7 @@ Note that:
|
|
4348
4477
|
|
4349
4478
|
* `ascli` takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
|
4350
4479
|
* most, but not all native ascp arguments are available as standard transfer_spec parameters
|
4351
|
-
* native ascp arguments can be provided with the [
|
4480
|
+
* native ascp arguments can be provided with the [*transfer-spec*](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
|
4352
4481
|
|
4353
4482
|
#### server side and configuration
|
4354
4483
|
|
@@ -4419,11 +4548,11 @@ Main components:
|
|
4419
4548
|
A working example can be found in the gem, example:
|
4420
4549
|
|
4421
4550
|
```bash
|
4422
|
-
ascli config
|
4551
|
+
ascli config gem path
|
4423
4552
|
```
|
4424
4553
|
|
4425
4554
|
```bash
|
4426
|
-
cat $(ascli config
|
4555
|
+
cat $(ascli config gem path)/../examples/transfer.rb
|
4427
4556
|
```
|
4428
4557
|
|
4429
4558
|
This sample code shows some example of use of the API as well as REST API.
|
@@ -4454,11 +4583,27 @@ So, it evolved into `ascli`:
|
|
4454
4583
|
|
4455
4584
|
* portable: works on platforms supporting `ruby` (and `ascp`)
|
4456
4585
|
* easy to install with the `gem` utility
|
4457
|
-
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [
|
4586
|
+
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
|
4458
4587
|
* `ruby` is consistent with other Aspera products
|
4459
4588
|
|
4460
4589
|
## Changes (Release notes)
|
4461
4590
|
|
4591
|
+
* 4.7.0
|
4592
|
+
|
4593
|
+
* new: option to specify font used to generate image of text file in `preview`
|
4594
|
+
* new: #66 improvement for content protection (support standard transfer spec options for direct agent)
|
4595
|
+
* new: option `fpac` is now applicable to all ruby based HTTP connections, i.e. API calls
|
4596
|
+
* new: option `show_secrets` to reveal secrets in command output
|
4597
|
+
* new: added and updated commands for Faspex 5
|
4598
|
+
* new: option `cache_tokens`
|
4599
|
+
* new: Faspex4 dropbox packages can now be received by id
|
4600
|
+
* change: (break) command `conf gem path` replaces `conf gem_path`
|
4601
|
+
* change: (break) option `fpac` expects a value instead of URL
|
4602
|
+
* change: (break) option `cipher` in transfer spec must have hyphen
|
4603
|
+
* change: (break) renamed option `log_passwords` to `log_secrets`
|
4604
|
+
* change: (break) removed plugin `shares2` as products is now EOL
|
4605
|
+
* fix: After AoC version update, wizard did not detect AoC properly
|
4606
|
+
|
4462
4607
|
* 4.6.0
|
4463
4608
|
|
4464
4609
|
* new: command `conf plugin create`
|
@@ -4823,7 +4968,7 @@ So, it evolved into `ascli`:
|
|
4823
4968
|
|
4824
4969
|
* 0.9.7
|
4825
4970
|
|
4826
|
-
* homogeneous [
|
4971
|
+
* homogeneous [*transfer-spec*](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
|
4827
4972
|
* preview persistency goes to unique file by default
|
4828
4973
|
* catch mxf extension in preview as video
|
4829
4974
|
* Faspex: possibility to download all packages by specifying id=ALL
|
@@ -4898,24 +5043,21 @@ So, it evolved into `ascli`:
|
|
4898
5043
|
|
4899
5044
|
* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in [option preset](#lprt) "default".
|
4900
5045
|
|
4901
|
-
##
|
4902
|
-
|
4903
|
-
For issues or feature requests use the Github repository and issues.
|
5046
|
+
## Common problems
|
4904
5047
|
|
4905
|
-
|
5048
|
+
### Error "Remote host is not who we expected"
|
4906
5049
|
|
4907
|
-
|
5050
|
+
Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
4908
5051
|
|
4909
|
-
|
5052
|
+
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
|
4910
5053
|
|
4911
|
-
|
4912
|
-
|
4913
|
-
|
5054
|
+
```json
|
5055
|
+
--ts=@json:'{"sshfp":null}'
|
5056
|
+
```
|
4914
5057
|
|
4915
|
-
|
4916
|
-
As a solution, use the position specific notation for selection, i.e. provide the identified just after command and do not use option `id`.
|
5058
|
+
Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
|
4917
5059
|
|
4918
|
-
|
5060
|
+
References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
|
4919
5061
|
|
4920
5062
|
### ED255519 key not supported
|
4921
5063
|
|
@@ -4930,33 +5072,18 @@ OpenSSH keys only supported if ED25519 is available
|
|
4930
5072
|
Which meant that you do not have ruby support for ED25519 SSH keys.
|
4931
5073
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
4932
5074
|
|
4933
|
-
|
4934
|
-
|
4935
|
-
Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
4936
|
-
|
4937
|
-
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
|
4938
|
-
|
4939
|
-
```bash
|
4940
|
-
--ts=@json:'{"sshfp":null}'
|
4941
|
-
```
|
5075
|
+
## BUGS, FEATURES, CONTRIBUTION
|
4942
5076
|
|
4943
|
-
|
5077
|
+
For issues reports or feature requests use the Github repository and issues.
|
4944
5078
|
|
4945
|
-
|
5079
|
+
You can also contribute to this open source project.
|
4946
5080
|
|
4947
|
-
|
5081
|
+
One can also [create one's own plugin](#createownplugin).
|
4948
5082
|
|
4949
|
-
|
5083
|
+
## Miscellaneous
|
4950
5084
|
|
4951
|
-
|
5085
|
+
* replace rest and oauth classes with ruby standard gems:
|
4952
5086
|
* <https://github.com/rest-client/rest-client>
|
4953
|
-
|
4954
|
-
* use Thor or
|
4955
|
-
|
4956
|
-
* provide metadata in packages
|
4957
|
-
|
4958
|
-
* deliveries to dropboxes
|
4959
|
-
|
4960
|
-
* Going through proxy: use env var http_proxy and https_proxy, no_proxy
|
4961
|
-
|
4962
|
-
* easier use with <https://github.com/pmq20/ruby-packer>
|
5087
|
+
* <https://github.com/oauth-xx/oauth2>
|
5088
|
+
* use gem Thor <http://whatisthor.com/> (or other standard Ruby CLI manager)
|
5089
|
+
* Package with <https://github.com/pmq20/ruby-packer> (rubyc)
|