aspera-cli 4.6.0 → 4.7.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (96) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +427 -300
  3. data/bin/ascli +2 -1
  4. data/bin/asession +1 -0
  5. data/docs/test_env.conf +2 -0
  6. data/examples/aoc.rb +4 -3
  7. data/examples/faspex4.rb +21 -19
  8. data/examples/proxy.pac +1 -1
  9. data/examples/transfer.rb +15 -15
  10. data/lib/aspera/aoc.rb +135 -124
  11. data/lib/aspera/ascmd.rb +85 -75
  12. data/lib/aspera/ats_api.rb +11 -10
  13. data/lib/aspera/cli/basic_auth_plugin.rb +13 -14
  14. data/lib/aspera/cli/extended_value.rb +42 -33
  15. data/lib/aspera/cli/formater.rb +138 -111
  16. data/lib/aspera/cli/info.rb +17 -0
  17. data/lib/aspera/cli/listener/line_dump.rb +3 -2
  18. data/lib/aspera/cli/listener/logger.rb +2 -1
  19. data/lib/aspera/cli/listener/progress.rb +16 -18
  20. data/lib/aspera/cli/listener/progress_multi.rb +13 -16
  21. data/lib/aspera/cli/main.rb +122 -130
  22. data/lib/aspera/cli/manager.rb +146 -154
  23. data/lib/aspera/cli/plugin.rb +38 -34
  24. data/lib/aspera/cli/plugins/alee.rb +6 -6
  25. data/lib/aspera/cli/plugins/aoc.rb +273 -276
  26. data/lib/aspera/cli/plugins/ats.rb +82 -76
  27. data/lib/aspera/cli/plugins/bss.rb +14 -16
  28. data/lib/aspera/cli/plugins/config.rb +350 -306
  29. data/lib/aspera/cli/plugins/console.rb +23 -19
  30. data/lib/aspera/cli/plugins/cos.rb +18 -18
  31. data/lib/aspera/cli/plugins/faspex.rb +180 -159
  32. data/lib/aspera/cli/plugins/faspex5.rb +64 -54
  33. data/lib/aspera/cli/plugins/node.rb +147 -140
  34. data/lib/aspera/cli/plugins/orchestrator.rb +68 -66
  35. data/lib/aspera/cli/plugins/preview.rb +92 -96
  36. data/lib/aspera/cli/plugins/server.rb +79 -75
  37. data/lib/aspera/cli/plugins/shares.rb +23 -24
  38. data/lib/aspera/cli/plugins/sync.rb +20 -22
  39. data/lib/aspera/cli/transfer_agent.rb +40 -39
  40. data/lib/aspera/cli/version.rb +2 -1
  41. data/lib/aspera/colors.rb +35 -27
  42. data/lib/aspera/command_line_builder.rb +48 -34
  43. data/lib/aspera/cos_node.rb +29 -21
  44. data/lib/aspera/data_repository.rb +3 -2
  45. data/lib/aspera/environment.rb +50 -45
  46. data/lib/aspera/fasp/agent_base.rb +22 -20
  47. data/lib/aspera/fasp/agent_connect.rb +13 -11
  48. data/lib/aspera/fasp/agent_direct.rb +48 -59
  49. data/lib/aspera/fasp/agent_httpgw.rb +33 -39
  50. data/lib/aspera/fasp/agent_node.rb +15 -13
  51. data/lib/aspera/fasp/agent_trsdk.rb +12 -14
  52. data/lib/aspera/fasp/error.rb +2 -1
  53. data/lib/aspera/fasp/error_info.rb +68 -52
  54. data/lib/aspera/fasp/installation.rb +106 -94
  55. data/lib/aspera/fasp/listener.rb +1 -0
  56. data/lib/aspera/fasp/parameters.rb +83 -92
  57. data/lib/aspera/fasp/parameters.yaml +305 -249
  58. data/lib/aspera/fasp/resume_policy.rb +11 -14
  59. data/lib/aspera/fasp/transfer_spec.rb +26 -0
  60. data/lib/aspera/fasp/uri.rb +22 -21
  61. data/lib/aspera/faspex_gw.rb +55 -90
  62. data/lib/aspera/hash_ext.rb +4 -3
  63. data/lib/aspera/id_generator.rb +8 -7
  64. data/lib/aspera/keychain/encrypted_hash.rb +17 -16
  65. data/lib/aspera/keychain/macos_security.rb +6 -10
  66. data/lib/aspera/log.rb +25 -20
  67. data/lib/aspera/nagios.rb +13 -12
  68. data/lib/aspera/node.rb +30 -22
  69. data/lib/aspera/oauth.rb +175 -226
  70. data/lib/aspera/open_application.rb +4 -3
  71. data/lib/aspera/persistency_action_once.rb +6 -6
  72. data/lib/aspera/persistency_folder.rb +5 -9
  73. data/lib/aspera/preview/file_types.rb +6 -5
  74. data/lib/aspera/preview/generator.rb +25 -24
  75. data/lib/aspera/preview/options.rb +16 -14
  76. data/lib/aspera/preview/utils.rb +98 -98
  77. data/lib/aspera/{proxy_auto_config.erb.js → proxy_auto_config.js} +23 -31
  78. data/lib/aspera/proxy_auto_config.rb +111 -20
  79. data/lib/aspera/rest.rb +115 -113
  80. data/lib/aspera/rest_call_error.rb +2 -2
  81. data/lib/aspera/rest_error_analyzer.rb +23 -25
  82. data/lib/aspera/rest_errors_aspera.rb +15 -14
  83. data/lib/aspera/ssh.rb +12 -10
  84. data/lib/aspera/sync.rb +42 -41
  85. data/lib/aspera/temp_file_manager.rb +18 -14
  86. data/lib/aspera/timer_limiter.rb +2 -1
  87. data/lib/aspera/uri_reader.rb +7 -5
  88. data/lib/aspera/web_auth.rb +79 -76
  89. metadata +64 -21
  90. data/docs/Makefile +0 -65
  91. data/docs/README.erb.md +0 -4424
  92. data/docs/README.md +0 -13
  93. data/docs/diagrams.txt +0 -49
  94. data/docs/doc_tools.rb +0 -58
  95. data/lib/aspera/cli/plugins/shares2.rb +0 -114
  96. data/lib/aspera/fasp/default.rb +0 -17
data/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
4
4
 
5
- Version : 4.6.0
5
+ Version : 4.7.0
6
6
 
7
7
  Laurent/2016-2022
8
8
 
@@ -14,9 +14,10 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
14
14
 
15
15
  Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
16
16
 
17
- Required Ruby version: > 2.4
17
+ Minimum required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
18
18
 
19
- [Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
19
+ [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
20
+ [Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
20
21
 
21
22
  ## <a id="when_to_use"></a>When to use and when not to use
22
23
 
@@ -88,7 +89,7 @@ ascli --version
88
89
  ```
89
90
 
90
91
  ```bash
91
- 4.6.0
92
+ 4.7.0
92
93
  ```
93
94
 
94
95
  ### First use
@@ -126,7 +127,7 @@ If you want to use `ascli` with another server, and in order to make further cal
126
127
  * download a file
127
128
 
128
129
  ```bash
129
- ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
130
+ ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_
130
131
  ```
131
132
 
132
133
  ```output
@@ -185,7 +186,7 @@ It is possible to install *either* directly on the host operating system (Linux,
185
186
 
186
187
  The direct installation is recommended and consists in installing:
187
188
 
188
- * [Ruby](#ruby) version > 2.4
189
+ * [Ruby](#ruby) (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
189
190
  * [aspera-cli](#the_gem)
190
191
  * [Aspera SDK (ascp)](#fasp_prot)
191
192
 
@@ -238,7 +239,7 @@ Use this method to install on the native host.
238
239
 
239
240
  A ruby interpreter is required to run the tool or to use the gem and tool.
240
241
 
241
- Required Ruby version: > 2.4. Ruby version 3 is also supported.
242
+ Required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version.
242
243
 
243
244
  *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
244
245
 
@@ -246,7 +247,7 @@ Refer to the following sections for a proposed method for specific operating sys
246
247
 
247
248
  The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, macOS, Windows with cygwin, etc...).
248
249
  If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
249
- If you have a simpler better way to install Ruby version > 2.4 : use it !
250
+ If you have a simpler better way to install Ruby : use it ! (version: >= 2.4. Deprecation notice: the minimum will be 2.5 in a future version)
250
251
 
251
252
  #### Generic: RVM: single user installation (not root)
252
253
 
@@ -516,7 +517,7 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
516
517
  * Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
517
518
  * Supports Commands, Option values and Parameters shortcuts
518
519
  * FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
519
- * Transfer parameters can be altered by modification of _transfer-spec_, this includes requiring multi-session
520
+ * Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
520
521
  * Allows transfers from products to products, essentially at node level (using the node transfer agent)
521
522
  * Supports FaspStream creation (using Node API)
522
523
  * Supports Watchfolder creation (using Node API)
@@ -544,10 +545,10 @@ There are two types of command line arguments: Commands and Options. Example :
544
545
  ascli command subcommand --option-name=VAL1 VAL2
545
546
  ```
546
547
 
547
- * executes _command_: `command subcommand`
548
- * with one _option_: `option_name`
549
- * this option is given a _value_ of: `VAL1`
550
- * the command has one additional _argument_: `VAL2`
548
+ * executes *command*: `command subcommand`
549
+ * with one *option*: `option_name`
550
+ * this option is given a *value* of: `VAL1`
551
+ * the command has one additional *argument*: `VAL2`
551
552
 
552
553
  When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `ascli conf ov` is the same as `ascli config overview`.
553
554
 
@@ -580,7 +581,7 @@ Note that here, `--sample` is taken as an argument, and not as an option, due to
580
581
 
581
582
  Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
582
583
 
583
- The value for _any_ options can come from the following locations (in this order, last value evaluated overrides previous value):
584
+ The value for *any* options can come from the following locations (in this order, last value evaluated overrides previous value):
584
585
 
585
586
  * [Configuration file](#configfile).
586
587
  * Environment variable
@@ -677,6 +678,9 @@ The option `display` controls the level of output:
677
678
  * `data` display `data` and `error` messages
678
679
  * `error` display only error messages.
679
680
 
681
+ By default, secrets are shown on output.
682
+ To hide secrets from results, set option `show_secrets` to `no`.
683
+
680
684
  #### Selection of output object properties
681
685
 
682
686
  By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
@@ -702,11 +706,12 @@ The difference between reader and decoder is order and ordinality. Both act like
702
706
 
703
707
  The following "readers" are supported (returns value in []):
704
708
 
705
- * @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
706
- * @file:PATH : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
707
- * @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
708
- * @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
709
- * @stdin: : [String] read from stdin (no value on right)
709
+ * @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
710
+ * @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
711
+ * @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
712
+ * @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
713
+ * @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
714
+ * @stdin: : [String] read from stdin (no value on right)
710
715
  * @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
711
716
 
712
717
  In addition it is possible to decode a value, using one or multiple decoders :
@@ -775,7 +780,7 @@ Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["co
775
780
 
776
781
  Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
777
782
 
778
- For instance, a [_transfer-spec_](#transferspec) is expected to be a _Structured Value_.
783
+ For instance, a [*transfer-spec*](#transferspec) is expected to be a _Structured Value_.
779
784
 
780
785
  Structured values shall be described using the [Extended Value Syntax](#extended).
781
786
  A convenient way to specify a _Structured Value_ is to use the `@json:` decoder, and describe the value in JSON format. The `@ruby:` decoder can also be used. For an array of hash tables, the `@csvt:` decoder can be used.
@@ -784,12 +789,12 @@ It is also possible to provide a _Structured Value_ in a file using `@json:@file
784
789
 
785
790
  ### <a id="conffolder"></a>Configuration and Persistency Folder
786
791
 
787
- `ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/ascli`.
792
+ `ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored `[config folder]`: `[User's home folder]/.aspera/ascli`.
788
793
 
789
794
  Note: `[User's home folder]` is found using ruby's `Dir.home` (`rb_w32_home_dir`).
790
795
  It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
791
796
 
792
- The main folder can be displayed using :
797
+ The [config folder] can be displayed using :
793
798
 
794
799
  ```bash
795
800
  ascli config folder
@@ -811,6 +816,12 @@ ascli config folder
811
816
  C:\Users\Kenji\.aspera\ascli
812
817
  ```
813
818
 
819
+ When OAuth is used (AoC, Faspex4 apiv4, Faspex5) `ascli` keeps a cache of generated bearer tokens in `[config folder]/persist_store` by default.
820
+ Option `cache_tokens` (**yes**/no) allows to control if Oauth tokens are cached on file system, or generated for each request.
821
+ The command `config flush_tokens` deletes all existing tokens.
822
+ Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
823
+ Tokens that can be refreshed are refreshed. Else tokens are re-generated if expired.
824
+
814
825
  ### <a id="configfile"></a>Configuration file
815
826
 
816
827
  On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
@@ -839,7 +850,7 @@ ascli config preset set|delete|show|initialize|update <option preset>
839
850
  The command `update` allows the easy creation of [option preset](#lprt) by simply providing the options in their command line format, e.g. :
840
851
 
841
852
  ```bash
842
- ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
853
+ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_pass_here_ --ts=@json:'{"precalculate_job_size":true}'
843
854
  ```
844
855
 
845
856
  * This creates a [option preset](#lprt) `demo_server` with all provided options.
@@ -847,13 +858,13 @@ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --u
847
858
  The command `set` allows setting individual options in a [option preset](#lprt).
848
859
 
849
860
  ```bash
850
- ascli config preset set demo_server password _demo_pass_
861
+ ascli config preset set demo_server password _pass_here_
851
862
  ```
852
863
 
853
- The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
864
+ The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [*Structured Value*](#native).
854
865
 
855
866
  ```javascript
856
- ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
867
+ ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_pass_here_","ts":{"precalculate_job_size":true}}'
857
868
  ```
858
869
 
859
870
  A full terminal based overview of the configuration can be displayed using:
@@ -933,7 +944,7 @@ cli_default:
933
944
  demo_server:
934
945
  url: ssh://demo.asperasoft.com:33001
935
946
  username: asperaweb
936
- password: _demo_pass_
947
+ password: _pass_here_
937
948
  ```
938
949
 
939
950
  We can see here:
@@ -956,7 +967,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended).
956
967
  Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
957
968
 
958
969
  ```bash
959
- ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/aocapikey"
970
+ ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
960
971
  ```
961
972
 
962
973
  This creates the [option preset](#lprt):
@@ -964,7 +975,7 @@ This creates the [option preset](#lprt):
964
975
  ```yaml
965
976
  ...
966
977
  my_aoc_org:
967
- private_key: @file:"/Users/laurent/.aspera/ascli/aocapikey"
978
+ private_key: @file:"/Users/laurent/.aspera/ascli/my_private_key"
968
979
  ...
969
980
  ```
970
981
 
@@ -1008,13 +1019,13 @@ ascli config preset unset cli_default interactive
1008
1019
  Example: Define options using command line:
1009
1020
 
1010
1021
  ```bash
1011
- ascli -N --url=x --password=y --username=y node --show-config
1022
+ ascli -N --url=_url_here_ --password=_pass_here_ --username=_name_here_ node --show-config
1012
1023
  ```
1013
1024
 
1014
1025
  Example: Define options using a hash:
1015
1026
 
1016
1027
  ```javascript
1017
- ascli -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
1028
+ ascli -N --preset=@json:'{"url":"_url_here_","password":"_pass_here_","username":"_name_here_"}' node --show-config
1018
1029
  ```
1019
1030
 
1020
1031
  #### Shares Examples
@@ -1024,7 +1035,7 @@ only username/password and url are required (either on command line, or from con
1024
1035
  Those can usually be provided on the command line:
1025
1036
 
1026
1037
  ```bash
1027
- ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
1038
+ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=_pass_here_
1028
1039
  ```
1029
1040
 
1030
1041
  This can also be provisioned in a config file:
@@ -1034,19 +1045,19 @@ This can also be provisioned in a config file:
1034
1045
  ```bash
1035
1046
  ascli config preset set shares06 url https://10.25.0.6
1036
1047
  ascli config preset set shares06 username john
1037
- ascli config preset set shares06 password 4sp3ra
1048
+ ascli config preset set shares06 password _pass_here_
1038
1049
  ```
1039
1050
 
1040
1051
  This can also be done with one single command:
1041
1052
 
1042
1053
  ```javascript
1043
- ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
1054
+ ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"_pass_here_"}'
1044
1055
  ```
1045
1056
 
1046
1057
  or
1047
1058
 
1048
1059
  ```bash
1049
- ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=4sp3ra
1060
+ ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=_pass_here_
1050
1061
  ```
1051
1062
 
1052
1063
  * Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
@@ -1128,6 +1139,54 @@ To test if a secret can be found use:
1128
1139
  ascli conf vault get --username=access_key1
1129
1140
  ```
1130
1141
 
1142
+ ### <a id="private_key"></a>Private Key
1143
+
1144
+ Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5...).
1145
+ It consists in generating a private key, or using a previouly generated key.
1146
+ The same key can be used for multiple applications.
1147
+ Technically, a private key contains the public key, which can be extracted.
1148
+ Currently, only private key not protected by a passphrase are supported.
1149
+ (TODO: add passphrase protection as option for aspera apps).
1150
+
1151
+ Several methods can be used to generate a key pair:
1152
+
1153
+ * `ascli`
1154
+
1155
+ The generated key is of type RSA 4096 bit. For convenience, the public key is also extracted.
1156
+
1157
+ ```bash
1158
+ ascli config genkey ~/.aspera/ascli/my_private_key
1159
+ ```
1160
+
1161
+ * `ssh-keygen`
1162
+
1163
+ ```bash
1164
+ ssh-keygen -t rsa -f ~/.aspera/ascli/my_private_key -N ''
1165
+ ```
1166
+
1167
+ * `openssl`
1168
+
1169
+ openssl is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS). To generate a privatekey pait without passphrase the following shall work on any system:
1170
+
1171
+ ```bash
1172
+ APIKEY=~/.aspera/ascli/my_private_key
1173
+ openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
1174
+ openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
1175
+ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
1176
+ rm -f ${APIKEY}.protected
1177
+ ```
1178
+
1179
+ ### <a id="certificates"></a>SSL CA certificate bundle
1180
+
1181
+ `ascli` uses ruby `openssl` gem, which uses the `openssl` library.
1182
+ Certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..).
1183
+ The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
1184
+
1185
+ `ascp` also needs to validate certificates when using WSS.
1186
+ By default, `ascp` uses primarily certificates from hard-coded path (e.g. on macOS: `/Library/Aspera/ssl`).
1187
+ `ascli` overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch.
1188
+ So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
1189
+
1131
1190
  ### Plugins
1132
1191
 
1133
1192
  The CLI tool uses a plugin mechanism.
@@ -1142,13 +1201,13 @@ ascli conf plugin list
1142
1201
  ```
1143
1202
 
1144
1203
  ```output
1145
- +--------------+-----------------------------------------------------------------------------------+
1146
- | plugin | path |
1147
- +--------------+-----------------------------------------------------------------------------------+
1148
- | shares | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/shares.rb |
1149
- | node | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/node.rb |
1204
+ +--------------+--------------------------------------------------------+
1205
+ | plugin | path |
1206
+ +--------------+--------------------------------------------------------+
1207
+ | shares | ..../aspera-cli/lib/aspera/cli/plugins/shares.rb |
1208
+ | node | ..../aspera-cli/lib/aspera/cli/plugins/node.rb |
1150
1209
  ...
1151
- +--------------+-----------------------------------------------------------------------------------+
1210
+ +--------------+--------------------------------------------------------+
1152
1211
  ```
1153
1212
 
1154
1213
  #### <a id="createownplugin"></a>Create your own plugin
@@ -1197,7 +1256,7 @@ To increase debug level, use parameter `log_level` (e.g. using command line `--l
1197
1256
  It is also possible to activate traces before initialization using env var `AS_LOG_LEVEL`.
1198
1257
 
1199
1258
  By default passwords and secrets are removed from logs.
1200
- Use option `log_passwords` to change this behaviour.
1259
+ Use option `log_secrets` set to `yes` to reveal secrets in logs.
1201
1260
 
1202
1261
  ### Learning Aspera Product APIs (REST)
1203
1262
 
@@ -1219,7 +1278,7 @@ Ruby HTTP socket parameters can be adjusted.
1219
1278
  | `open_timeout` | 60 |
1220
1279
  | `keep_alive_timeout` | 2 |
1221
1280
 
1222
- Values are in set **seconds** and can be of type either integer or float.
1281
+ Values are in set *seconds* and can be of type either integer or float.
1223
1282
  Default values are the ones of Ruby.
1224
1283
  For details refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
1225
1284
 
@@ -1244,28 +1303,58 @@ It is also possible to force the graphical mode with option --ui :
1244
1303
  * `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
1245
1304
  * `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
1246
1305
 
1247
- ### HTTP proxy for REST
1306
+ ### Proxy
1248
1307
 
1249
- To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honored by Ruby when calling REST APIs.
1308
+ There are several types of network connections, each of them use a different mechanism to define a *proxy*.
1250
1309
 
1251
- ### <a id="certificates"></a>SSL CA certificate bundle
1310
+ #### HTTP proxy for REST calls and transfers using HTTP gateway
1311
+
1312
+ To specify a HTTP proxy when ruby HTTP is used, set the `http_proxy` environment variable (lower case, preferred, or upper case).
1313
+ See [Ruby findproxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
1314
+
1315
+ ```bash
1316
+ export http_proxy=http://myproxy.org.net:3128
1317
+ ```
1252
1318
 
1253
- `ascli` uses ruby `openssl` gem, which uses the `openssl` library, so certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..). The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
1319
+ Note that ruby expects a URL and `myproxy.org.net:3128` alone is not accepted.
1254
1320
 
1255
- `ascp` also needs to validate certificates when using WSS. By default, `ascp` uses primarily certificates from hard coded path (e.g. on macOS: `/Library/Aspera/ssl`). `ascli` overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch. So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
1321
+ #### FASP proxy (forward) for transfers
1256
1322
 
1257
- ### Proxy auto config
1323
+ To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `EX_fasp_proxy_url` (only supported with the `direct` agent).
1258
1324
 
1259
- The `fpac` option allows specification of a Proxy Auto Configuration (PAC) file, by its URL for local FASP agent. Supported schemes are : http:, https: and file:.
1325
+ #### HTTP proxy legacy Aspera HTTP fallback transfers
1260
1326
 
1261
- The PAC file can be tested with command: `config proxy_check` , example:
1327
+ To specify a proxy for legacy HTTP fallback, set the [*transfer-spec*](#transferspec) parameter: `EX_http_proxy_url` (only supported with the `direct` agent).
1328
+ (It is also possible to use `EX_ascp_args` and native options in `direct`)
1329
+
1330
+ #### Proxy auto config
1331
+
1332
+ The `fpac` option allows use of a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) script defined as javascript value.
1333
+ To read the script from a URL (`http:`, `https:` and `file:`), use: `@uri:`.
1334
+ A minimal script can be specified, for example like this, to define the use of a local proxy:
1335
+
1336
+ ```bash
1337
+ export ASCLI_FPAC='function FindProxyForURL(url, host){return "PROXY localhost:3128"}'
1338
+ ```
1339
+
1340
+ The PAC file will be used for any HTTP/HTTPS/REST connection, but not other (e.g. FASP, HTTP fallback, HTTPGW)
1341
+
1342
+ The PAC file can be tested with command: `config proxy_check`. Example, using command line option:
1343
+
1344
+ ```
1345
+ ascli conf proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:1234;DIRECT";}' http://example.com
1346
+ PROXY proxy.example.com:1234;DIRECT
1347
+ ```
1262
1348
 
1263
1349
  ```bash
1264
- ascli config proxy_check --fpac=file:///./proxy.pac http://www.example.com
1350
+ ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
1265
1351
  PROXY proxy.example.com:8080
1266
1352
  ```
1267
1353
 
1268
- This is not yet implemented to specify http proxy, so use `http_proxy` env vars.
1354
+ ```bash
1355
+ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.com
1356
+ PROXY proxy.example.com:8080
1357
+ ```
1269
1358
 
1270
1359
  ### <a id="client"></a>FASP configuration
1271
1360
 
@@ -1411,14 +1500,14 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
1411
1500
 
1412
1501
  Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
1413
1502
 
1414
- When a transfer needs to be started, a [_transfer-spec_](#transferspec) has been internally prepared.
1415
- This [_transfer-spec_](#transferspec) will be executed by a transfer client, here called "Transfer Agent".
1503
+ When a transfer needs to be started, a [*transfer-spec*](#transferspec) has been internally prepared.
1504
+ This [*transfer-spec*](#transferspec) will be executed by a transfer client, here called "Transfer Agent".
1416
1505
 
1417
1506
  There are currently 3 agents:
1418
1507
 
1419
1508
  * [`direct`](#agt_direct) : a local execution of `ascp`
1420
1509
  * [`connect`](#agt_connect) : use of a local Connect Client
1421
- * [`node`](#agt_node) : use of an Aspera Transfer Node (potentially _remote_).
1510
+ * [`node`](#agt_node) : use of an Aspera Transfer Node (potentially *remote*).
1422
1511
  * [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
1423
1512
  * [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
1424
1513
 
@@ -1426,7 +1515,7 @@ Note that all transfer operation are seen from the point of view of the agent.
1426
1515
  For instance, a node agent making an "upload", or "package send" operation,
1427
1516
  will effectively push files to the related server from the agent node.
1428
1517
 
1429
- `ascli` standardizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1518
+ `ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *raw* ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
1430
1519
 
1431
1520
  #### <a id="agt_direct"></a>Direct
1432
1521
 
@@ -1467,12 +1556,6 @@ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1467
1556
  ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1468
1557
  ```
1469
1558
 
1470
- To specify a FASP proxy (only supported with the `direct` agent), set the appropriate [_transfer-spec_](#transferspec) parameter:
1471
-
1472
- * `EX_fasp_proxy_url`
1473
- * `EX_http_proxy_url` (proxy for legacy http fallback)
1474
- * `EX_ascp_args`
1475
-
1476
1559
  #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1477
1560
 
1478
1561
  By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
@@ -1483,7 +1566,6 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
1483
1566
  Transfer Server using the Node API, either on a local or remote node.
1484
1567
  Parameters provided in option `transfer_info` are:
1485
1568
 
1486
- <table>
1487
1569
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1488
1570
  <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
1489
1571
  <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
@@ -1493,7 +1575,7 @@ Parameters provided in option `transfer_info` are:
1493
1575
 
1494
1576
  Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
1495
1577
  `--transfer-info=@preset:<psetname>` or be specified using the extended value syntax :
1496
- `--transfer-info=@json:'{"url":"https://...","username":"theuser","password":"thepass"}'`
1578
+ `--transfer-info=@json:'{"url":"https://...","username":"theuser","password":"_pass_here_"}'`
1497
1579
 
1498
1580
  If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
1499
1581
 
@@ -1520,7 +1602,7 @@ By default it will listen on local port `55002` on `127.0.0.1`.
1520
1602
  ### <a id="transferspec"></a>Transfer Specification
1521
1603
 
1522
1604
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
1523
- is described in a _transfer-spec_ (Transfer Specification), such as:
1605
+ is described in a [*transfer-spec*](#transferspec) (Transfer Specification), such as:
1524
1606
 
1525
1607
  * server address
1526
1608
  * transfer user name
@@ -1528,23 +1610,23 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
1528
1610
  * file list
1529
1611
  * etc...
1530
1612
 
1531
- `ascli` builds a default _transfer-spec_ internally, so it is not necessary to provide additional parameters on the command line for this transfer.
1613
+ `ascli` builds a default [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
1532
1614
 
1533
- If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cumulative.
1615
+ If needed, it is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters. Multiple `ts` options on command line are cumulative.
1534
1616
 
1535
- It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
1617
+ It is possible to specify ascp options when the `transfer` option is set to [`direct`](#agt_direct) using the special [*transfer-spec*](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is especially useful for ascp command line parameters not supported yet in the transfer spec.
1536
1618
 
1537
- The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
1619
+ The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has the advantage of:
1538
1620
 
1539
1621
  * common to all [Transfer Agent](#agents)
1540
1622
  * not dependent on command line limitations (special characters...)
1541
1623
 
1542
- A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
1624
+ A [*transfer-spec*](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
1543
1625
 
1544
1626
  ### <a id="transferparams"></a>Transfer Parameters
1545
1627
 
1546
- All standard _transfer-spec_ parameters can be specified.
1547
- [_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
1628
+ All standard [*transfer-spec*](#transferspec) parameters can be specified.
1629
+ [*transfer-spec*](#transferspec) can also be saved/overridden in the config file.
1548
1630
 
1549
1631
  References:
1550
1632
 
@@ -1569,7 +1651,7 @@ Columns:
1569
1651
 
1570
1652
  Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
1571
1653
 
1572
- <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>none, aes-128, aes-256<br/>Allowed values: aes128, aes192, aes256, aes128cfb, aes192cfb, aes256cfb, aes128gcm, aes192gcm, aes256gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side content protection. (encryption at rest)<br/>Allowed values: encrypt, decrypt</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password.</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer (older,string)<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>http_fallback</td><td>string<br/>bool</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a fasp transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port.<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port.</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>in bytes<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Required. Contains a path to the source (required) and a path to the destination.</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_access_key</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Node only?</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies ssh (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIVATE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links by:<br/> following the link (follow),<br/> copying the link itself (copy),<br/> skipping (skip),<br/> or forcibly copying the link itself (copy+force).<br/> Default: follow<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node api.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowable values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>TODO: check node, sdk<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Passphrase used for at rest encryption or decryption<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/> Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr></table>
1654
+ <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt)</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source. The source and destination arguments must be directories that have matching names. Objects on the destination that have the same name but different type or size as objects on the source are not deleted.<br/>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr><tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port when no cipher is used<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t)</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value. (0=no file is split)<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr><tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr><tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr><tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Passphrase used for at rest encryption or decryption. Prefer to use standard: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr></table>
1573
1655
 
1574
1656
  #### Destination folder for transfers
1575
1657
 
@@ -1578,7 +1660,7 @@ The destination folder is set by `ascli` by default to:
1578
1660
  * `.` for downloads
1579
1661
  * `/` for uploads
1580
1662
 
1581
- It is specified by the [_transfer-spec_](#transferspec) parameter `destination_root`.
1663
+ It is specified by the [*transfer-spec*](#transferspec) parameter `destination_root`.
1582
1664
  As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
1583
1665
  The option `to_folder` provides an equivalent and convenient way to change this parameter:
1584
1666
  `--to-folder=<path>` .
@@ -1598,10 +1680,10 @@ This is equivalent to:
1598
1680
  ascli server upload --sources=@args ~/mysample.file secondfile
1599
1681
  ```
1600
1682
 
1601
- More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
1683
+ More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [*transfer-spec*](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
1602
1684
 
1603
1685
  Note that this is different from the "ascp" command line. The paradigm used by `ascli` is:
1604
- all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
1686
+ all transfer parameters are kept in [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
1605
1687
 
1606
1688
  For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
1607
1689
 
@@ -1633,13 +1715,13 @@ providing a file list directly to ascp:
1633
1715
  ... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
1634
1716
  ```
1635
1717
 
1636
- * Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
1718
+ * Not recommended: It is possible to specify bare ascp arguments using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`.
1637
1719
 
1638
1720
  ```javascript
1639
1721
  --sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
1640
1722
  ```
1641
1723
 
1642
- This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overridden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
1724
+ This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [*transfer-spec*](#transferspec), which will be overridden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
1643
1725
 
1644
1726
  In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
1645
1727
 
@@ -1690,6 +1772,29 @@ Multi-session spawn is done by `ascli`.
1690
1772
 
1691
1773
  When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
1692
1774
 
1775
+ #### Content protection
1776
+
1777
+ Also known as Client-side encryption at reast (CSEAR), content protection allows a client to send files to a server
1778
+ which will store them encrypted (upload), and decrypt files as they are being downloaded from a server, both
1779
+ using a passphrase, only known by users sharing files. Files stay encrypted on server side.
1780
+
1781
+ activating CSEAR consists in using transfer spec parameters:
1782
+
1783
+ * `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
1784
+ * `content_protection_password` : the passphrase to be used.
1785
+
1786
+ Example: parameter to download a faspex package and decrypt on the fly
1787
+
1788
+ ```json
1789
+ --ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
1790
+ ```
1791
+
1792
+ Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
1793
+
1794
+ ```json
1795
+ --ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
1796
+ ```
1797
+
1693
1798
  #### Transfer Spec Examples
1694
1799
 
1695
1800
  * Change target rate
@@ -1824,15 +1929,14 @@ where:
1824
1929
  * `dirname` is the folder name and can contain `/` to specify a subfolder.
1825
1930
  * supported arguments are:
1826
1931
 
1827
- <table>
1828
- <tr><th>name</th><th>type</th><th>default</th><th>description</th></tr>
1829
- <tr><td>count</td><td>int</td><td>mandatory</td><td>Number of files</td></tr>
1830
- <tr><td>file</td><td>string</td><td>file</td><td>Basename for files</td></tr>
1831
- <tr><td>size</td><td>int</td><td>0</td><td>Size of first file.</td></tr>
1832
- <tr><td>inc</td><td>int</td><td>0</td><td>Increment applied to determine next file size</td></tr>
1833
- <tr><td>seq</td><td>sequential<br/>random</td><td>sequential</td><td>Sequence in determining next file size</td></tr>
1834
- <tr><td>buf_init</td><td>none<br/>zero<br/>random</td><td>zero</td><td>How source data is initialized<br/>Option 'none' is not allowed for downloads.</td></tr>
1835
- </table>
1932
+ | Name | Type | Description |
1933
+ |--------|------|-------------|
1934
+ |count |int |mandatory|Number of files<br/>Mandatory|
1935
+ |file |string|Basename for files<br>Default: "file"|
1936
+ |size |int |Size of first file.<br>Default: 0|
1937
+ |inc |int |Increment applied to determine next file size<br>Default: 0|
1938
+ |seq |enum |Sequence in determining next file size<br/>Values: random, sequential<br/>Default: sequential|
1939
+ |buf_init|enum |How source data is initialized<br/>Option 'none' is not allowed for downloads.<br/>Values:none, zero, random<br/>Default:zero|
1836
1940
 
1837
1941
  The sequence parameter is applied as follows:
1838
1942
 
@@ -1921,16 +2025,16 @@ ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_sec
1921
2025
  ascli aoc admin resource node v3 name my_aoc_node1_name --secret=my_aoc_node1_secret access_key delete testsub1
1922
2026
  ascli aoc admin resource workspace list
1923
2027
  ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
1924
- ascli aoc automation workflow "my_wf_id" action create --value=@json:'{"name":"toto"}' | tee action.info
2028
+ ascli aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
1925
2029
  ascli aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
1926
- ascli aoc automation workflow delete "my_wf_id"
2030
+ ascli aoc automation workflow delete my_wf_id
1927
2031
  ascli aoc automation workflow list
1928
- ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data > test
2032
+ ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
1929
2033
  ascli aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
1930
2034
  ascli aoc bearer_token --display=data --scope=user:all
1931
2035
  ascli aoc faspex
1932
2036
  ascli aoc files bearer /
1933
- ascli aoc files bearer_token_node /
2037
+ ascli aoc files bearer_token_node / --cache-tokens=no
1934
2038
  ascli aoc files browse /
1935
2039
  ascli aoc files browse / -N --link=my_aoc_publink_folder
1936
2040
  ascli aoc files delete /testsrc
@@ -1981,13 +2085,14 @@ ascli ats cluster list
1981
2085
  ascli ats cluster show --cloud=aws --region=eu-west-1
1982
2086
  ascli ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
1983
2087
  ascli conf flush_tokens
1984
- ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes
1985
- ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
2088
+ ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes
2089
+ ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
1986
2090
  ascli config ascp connect info 'Aspera Connect for Windows'
1987
2091
  ascli config ascp connect list
1988
2092
  ascli config ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
1989
2093
  ascli config ascp connect version 'Aspera Connect for Windows' list
1990
2094
  ascli config ascp connect version 'Aspera Connect for Windows' open documentation
2095
+ ascli config ascp errors
1991
2096
  ascli config ascp info
1992
2097
  ascli config ascp install
1993
2098
  ascli config ascp products list
@@ -2003,7 +2108,7 @@ ascli config export
2003
2108
  ascli config genkey mykey
2004
2109
  ascli config plugin create mycommand T
2005
2110
  ascli config plugin list
2006
- ascli config proxy_check --fpac=file:///examples/proxy.pac https://eudemo.asperademo.com
2111
+ ascli config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com
2007
2112
  ascli console transfer current list
2008
2113
  ascli console transfer smart list
2009
2114
  ascli console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
@@ -2013,23 +2118,44 @@ ascli cos node access_key show self
2013
2118
  ascli cos node download testfile.bin --to-folder=.
2014
2119
  ascli cos node info
2015
2120
  ascli cos node upload testfile.bin
2121
+ ascli faspex dropbox list --recipient="*my_faspex_dbx"
2122
+ ascli faspex dropbox list --recipient="*my_faspex_wkg"
2016
2123
  ascli faspex health
2017
2124
  ascli faspex package list
2018
- ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
2019
- ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
2125
+ ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
2126
+ ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
2127
+ ascli faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
2128
+ ascli faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
2020
2129
  ascli faspex package recv "my_package_id" --to-folder=.
2021
2130
  ascli faspex package recv "my_package_id" --to-folder=. --box=sent
2022
- ascli faspex package recv --to-folder=. "my_package_id"
2023
2131
  ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
2024
2132
  ascli faspex package recv ALL --to-folder=. --once-only=yes
2133
+ ascli faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
2134
+ ascli faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
2135
+ ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
2136
+ ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
2025
2137
  ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
2026
2138
  ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
2027
2139
  ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
2028
2140
  ascli faspex source name "Server Files" node br /
2029
- ascli faspex5 node list --value=@json:'{"type":"received","subtype":"mypackages"}'
2141
+ ascli faspex v4 dmembership list
2142
+ ascli faspex v4 dropbox list
2143
+ ascli faspex v4 metadata_profile list
2144
+ ascli faspex v4 user list
2145
+ ascli faspex v4 wmembership list
2146
+ ascli faspex v4 workgroup list
2147
+ ascli faspex5 admin res accounts list
2148
+ ascli faspex5 admin res contacts list
2149
+ ascli faspex5 admin res jobs list
2150
+ ascli faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
2151
+ ascli faspex5 admin res oauth_clients list
2152
+ ascli faspex5 admin res registrations list
2153
+ ascli faspex5 admin res saml_configs list
2154
+ ascli faspex5 admin res shared_inboxes list
2155
+ ascli faspex5 admin res workgroups list
2030
2156
  ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
2031
2157
  ascli faspex5 package receive "my_package_id" --to-folder=.
2032
- ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"${f5_user}"}]}' testfile.bin
2158
+ ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin
2033
2159
  ascli mycommand --plugin-folder=T
2034
2160
  ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
2035
2161
  ascli node -N -Ptst_node_preview access_key delete aoc_1
@@ -2042,17 +2168,19 @@ ascli node async show ALL
2042
2168
  ascli node basic_token
2043
2169
  ascli node browse / -r
2044
2170
  ascli node delete folder_1/10MB.1
2045
- ascli node delete folder_1/testfile.bin
2046
- ascli node download --to-folder=. folder_1/testfile.bin
2171
+ ascli node delete testfile.bin
2172
+ ascli node download testfile.bin --to-folder=.
2173
+ ascli node download testfile.bin --to-folder=. --token-type=hybrid
2047
2174
  ascli node health
2048
- ascli node info
2175
+ ascli node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
2049
2176
  ascli node search / --value=@json:'{"sort":"mtime"}'
2050
2177
  ascli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
2051
2178
  ascli node service delete service1
2052
2179
  ascli node service list
2053
2180
  ascli node transfer list --value=@json:'{"active_only":true}'
2054
2181
  ascli node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
2055
- ascli node upload --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' testfile.bin
2182
+ ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
2183
+ ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
2056
2184
  ascli orchestrator info
2057
2185
  ascli orchestrator plugins
2058
2186
  ascli orchestrator processes
@@ -2098,22 +2226,22 @@ ascli server mv folder_1/200KB.2 folder_1/to.delete
2098
2226
  ascli server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
2099
2227
  ascli server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
2100
2228
  ascli server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
2229
+ ascli server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
2101
2230
  ascli server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
2102
2231
  ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
2103
2232
  ascli shares admin share list
2104
2233
  ascli shares repository browse /
2105
2234
  ascli shares repository delete my_shares_upload/testfile.bin
2106
2235
  ascli shares repository download --to-folder=. my_shares_upload/testfile.bin
2107
- ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://"my_http_gw_fqdn"/aspera/http-gwy/v1"}'
2236
+ ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
2108
2237
  ascli shares repository upload --to-folder=my_shares_upload testfile.bin
2109
- ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://"my_http_gw_fqdn"/aspera/http-gwy/v1"}'
2238
+ ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
2110
2239
  ascli shares2 appinfo
2111
2240
  ascli shares2 organization list
2112
2241
  ascli shares2 project list --organization=Sport
2113
2242
  ascli shares2 repository browse /
2114
2243
  ascli shares2 userinfo
2115
2244
  ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
2116
-
2117
2245
  ...and more
2118
2246
  ```
2119
2247
 
@@ -2122,7 +2250,7 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
2122
2250
  ```bash
2123
2251
  ascli -h
2124
2252
  NAME
2125
- ascli -- a command line tool for Aspera Applications (v4.6.0)
2253
+ ascli -- a command line tool for Aspera Applications (v4.7.0)
2126
2254
 
2127
2255
  SYNOPSIS
2128
2256
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -2131,12 +2259,12 @@ DESCRIPTION
2131
2259
  Use Aspera application to perform operations on command line.
2132
2260
  Documentation and examples: https://rubygems.org/gems/aspera-cli
2133
2261
  execute: ascli conf doc
2134
- or visit: http://www.rubydoc.info/gems/aspera-cli
2262
+ or visit: https://www.rubydoc.info/gems/aspera-cli
2135
2263
  source repo: https://github.com/IBM/aspera-cli
2136
2264
 
2137
2265
  ENVIRONMENT VARIABLES
2138
- ASCLI_HOME config folder, default: $HOME/.aspera/ascli
2139
- any option can be set as an environment variable, refer to the manual
2266
+ ASCLI_HOME config folder, default: $HOME/.aspera/ascli
2267
+ Any option can be set as an environment variable, refer to the manual
2140
2268
 
2141
2269
  COMMANDS
2142
2270
  To list first level commands, execute: ascli
@@ -2145,56 +2273,58 @@ COMMANDS
2145
2273
  OPTIONS
2146
2274
  Options begin with a '-' (minus), and value is provided on command line.
2147
2275
  Special values are supported beginning with special prefix @pfx:, where pfx is one of:
2148
- base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, stdin, preset
2276
+ base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, uri, stdin, preset
2149
2277
  Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
2150
2278
 
2151
2279
  ARGS
2152
2280
  Some commands require mandatory arguments, e.g. a path.
2153
2281
 
2154
2282
  OPTIONS: global
2155
- --interactive=ENUM use interactive input of missing params: yes, no
2156
- --ask-options=ENUM ask even optional options: yes, no
2157
- --format=ENUM output format: table, ruby, json, jsonpp, yaml, csv, nagios
2158
- --display=ENUM output only some information: info, data, error
2283
+ --interactive=ENUM use interactive input of missing params: yes, [no]
2284
+ --ask-options=ENUM ask even optional options: yes, [no]
2285
+ --format=ENUM output format: [table], ruby, json, jsonpp, yaml, csv, nagios
2286
+ --display=ENUM output only some information: [info], data, error
2159
2287
  --fields=VALUE comma separated list of fields, or ALL, or DEF
2160
2288
  --select=VALUE select only some items in lists, extended value: hash (column, value)
2161
2289
  --table-style=VALUE table display style
2162
- --flat-hash=ENUM display hash values as additional keys: yes, no
2163
- --transpose-single=ENUM single object fields output vertically: yes, no
2290
+ --flat-hash=ENUM display hash values as additional keys: [yes], no
2291
+ --transpose-single=ENUM single object fields output vertically: [yes], no
2292
+ --show-secrets=ENUM show secrets on command output: [yes], no
2164
2293
  -h, --help Show this message.
2165
2294
  --bash-comp generate bash completion for command
2166
2295
  --show-config Display parameters used for the provided action.
2167
2296
  -r, --rest-debug more debug for HTTP calls
2168
2297
  -v, --version display version
2169
2298
  -w, --warnings check for language warnings
2170
- --ui=ENUM method to start browser: text, graphical
2171
- --log-level=ENUM Log level: debug, info, warn, error, fatal, unknown
2172
- --logger=ENUM log method: stderr, stdout, syslog
2299
+ --ui=ENUM method to start browser: text, [graphical]
2300
+ --log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
2301
+ --logger=ENUM log method: [stderr], stdout, syslog
2173
2302
  --lock-port=VALUE prevent dual execution of a command, e.g. in cron
2174
2303
  --query=VALUE additional filter for API calls (extended value) (some commands)
2175
2304
  --http-options=VALUE options for http socket (extended value)
2176
- --insecure=ENUM do not validate HTTPS certificate: yes, no
2177
- --once-only=ENUM process only new items (some commands): yes, no
2178
- --log-passwords=ENUM show passwords in logs: yes, no
2305
+ --insecure=ENUM do not validate HTTPS certificate: [yes], no
2306
+ --once-only=ENUM process only new items (some commands): yes, [no]
2307
+ --log-secrets=ENUM show passwords in logs: yes, [no]
2308
+ --cache-tokens=ENUM save and reuse Oauth tokens: [yes], no
2179
2309
 
2180
2310
  COMMAND: config
2181
- SUBCOMMANDS: list overview id preset open documentation genkey gem_path plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
2311
+ SUBCOMMANDS: list overview id preset open documentation genkey gem plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
2182
2312
  OPTIONS:
2183
2313
  --value=VALUE extended value for create, update, list filter
2184
2314
  --property=VALUE name of property to set
2185
2315
  --id=VALUE resource identifier (modify,delete,show)
2186
- --config-file=VALUE read parameters from file in YAML format, current=/Users/FooBar/.aspera/ascli/config.yaml
2316
+ --config-file=VALUE read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
2187
2317
  -N, --no-default do not load default configuration for plugin
2188
- --override=ENUM Wizard: override existing value: yes, no
2189
- --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, no
2190
- --default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, no
2191
- --test-mode=ENUM Wizard: skip private key check step: yes, no
2318
+ --override=ENUM Wizard: override existing value: yes, [no]
2319
+ --use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: yes, [no]
2320
+ --default=ENUM Wizard: set as default configuration for specified plugin (also: update): yes, [no]
2321
+ --test-mode=ENUM Wizard: skip private key check step: yes, [no]
2192
2322
  -P, --presetVALUE load the named option preset from current config file
2193
2323
  --pkeypath=VALUE Wizard: path to private key for JWT
2194
2324
  --ascp-path=VALUE path to ascp
2195
2325
  --use-product=VALUE use ascp from specified product
2196
2326
  --smtp=VALUE smtp configuration (extended value: hash)
2197
- --fpac=VALUE proxy auto configuration URL
2327
+ --fpac=VALUE proxy auto configuration script
2198
2328
  --secret=VALUE default secret
2199
2329
  --secrets=VALUE secret vault
2200
2330
  --sdk-url=VALUE URL to get SDK
@@ -2241,7 +2371,7 @@ OPTIONS:
2241
2371
  --password=VALUE user's password
2242
2372
  --params=VALUE parameters hash table, use @json:{"param":"value"}
2243
2373
  --result=VALUE specify result value as: 'work step:parameter'
2244
- --synchronous=ENUM work step:parameter expected as result: yes, no
2374
+ --synchronous=ENUM work step:parameter expected as result: yes, [no]
2245
2375
  --ret-style=ENUM how return type is requested in api: header, arg, ext
2246
2376
  --auth-style=ENUM authentication type: arg_pass, head_basic, apikey
2247
2377
 
@@ -2275,7 +2405,7 @@ OPTIONS:
2275
2405
 
2276
2406
 
2277
2407
  COMMAND: faspex5
2278
- SUBCOMMANDS: node package auth_client jobs
2408
+ SUBCOMMANDS: package admin
2279
2409
  OPTIONS:
2280
2410
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2281
2411
  --username=VALUE username to log in
@@ -2283,14 +2413,14 @@ OPTIONS:
2283
2413
  --client-id=VALUE OAuth client identifier
2284
2414
  --client-secret=VALUE OAuth client secret
2285
2415
  --redirect-uri=VALUE OAuth redirect URI
2286
- --auth=ENUM OAuth type of authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey, boot
2416
+ --auth=ENUM OAuth type of authentication: web, jwt, boot
2287
2417
  --private-key=VALUE Oauth RSA private key PEM value for JWT (prefix file path with @val:@file:)
2288
2418
 
2289
2419
 
2290
2420
  COMMAND: cos
2291
2421
  SUBCOMMANDS: node
2292
2422
  OPTIONS:
2293
- --bucket=VALUE IBM Cloud Object storage bucket
2423
+ --bucket=VALUE IBM Cloud Object Storage bucket name
2294
2424
  --endpoint=VALUE storage endpoint url
2295
2425
  --apikey=VALUE storage API key
2296
2426
  --crn=VALUE ressource instance id
@@ -2312,17 +2442,6 @@ OPTIONS:
2312
2442
  --box=ENUM package box: inbox, archive, sent
2313
2443
 
2314
2444
 
2315
- COMMAND: shares2
2316
- SUBCOMMANDS: repository organization project team share appinfo userinfo admin
2317
- OPTIONS:
2318
- --url=VALUE URL of application, e.g. https://org.asperafiles.com
2319
- --username=VALUE username to log in
2320
- --password=VALUE user's password
2321
- --organization=VALUE organization
2322
- --project=VALUE project
2323
- --share=VALUE share
2324
-
2325
-
2326
2445
  COMMAND: preview
2327
2446
  SUBCOMMANDS: scan events trevents check test
2328
2447
  OPTIONS:
@@ -2330,7 +2449,7 @@ OPTIONS:
2330
2449
  --username=VALUE username to log in
2331
2450
  --password=VALUE user's password
2332
2451
  --skip-format=ENUM skip this preview format (multiple possible): png, mp4
2333
- --folder-reset-cache=ENUM force detection of generated preview by refresh cache: no, header, read
2452
+ --folder-reset-cache=ENUM force detection of generated preview by refresh cache: [no], header, read
2334
2453
  --skip-types=VALUE skip types in comma separated list
2335
2454
  --previews-folder=VALUE preview folder in storage root
2336
2455
  --temp-folder=VALUE path to temp folder
@@ -2338,15 +2457,16 @@ OPTIONS:
2338
2457
  --case=VALUE basename of output for for test
2339
2458
  --scan-path=VALUE subpath in folder id to start scan in (default=/)
2340
2459
  --scan-id=VALUE forder id in storage to start scan in, default is access key main folder id
2341
- --mimemagic=ENUM use Mime type detection of gem mimemagic: yes, no
2342
- --overwrite=ENUM when to overwrite result file: always, never, mtime
2343
- --file-access=ENUM how to read and write files in repository: local, remote
2460
+ --mimemagic=ENUM use Mime type detection of gem mimemagic: yes, [no]
2461
+ --overwrite=ENUM when to overwrite result file: always, never, [mtime]
2462
+ --file-access=ENUM how to read and write files in repository: [local], remote
2344
2463
  --max-size=VALUE maximum size (in bytes) of preview file
2345
2464
  --thumb-vid-scale=VALUE png: video: size (ffmpeg scale argument)
2346
2465
  --thumb-vid-fraction=VALUE png: video: position of snapshot
2347
2466
  --thumb-img-size=VALUE png: non-video: height (and width)
2348
- --video-conversion=ENUM mp4: method for preview generation: reencode, blend, clips
2349
- --video-png-conv=ENUM mp4: method for thumbnail generation: fixed, animated
2467
+ --thumb-text-font=VALUE png: plaintext: font to render text with image magick convert, list with: identify -list font
2468
+ --video-conversion=ENUM mp4: method for preview generation: [reencode], blend, clips
2469
+ --video-png-conv=ENUM mp4: method for thumbnail generation: [fixed], animated
2350
2470
  --video-start-sec=VALUE mp4: start offset (seconds) of video preview
2351
2471
  --video-scale=VALUE mp4: video scale (ffmpeg)
2352
2472
  --blend-keyframes=VALUE mp4: blend: # key frames
@@ -2370,7 +2490,7 @@ OPTIONS:
2370
2490
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2371
2491
  --username=VALUE username to log in
2372
2492
  --password=VALUE user's password
2373
- --auth=ENUM OAuth type of authentication: body_userpass, header_userpass, web, jwt, url_token, ibm_apikey
2493
+ --auth=ENUM OAuth type of authentication: web, jwt
2374
2494
  --operation=ENUM client operation for transfers: push, pull
2375
2495
  --client-id=VALUE OAuth API client identifier in application
2376
2496
  --client-secret=VALUE OAuth API client passcode
@@ -2383,8 +2503,8 @@ OPTIONS:
2383
2503
  --new-user-option=VALUE new user creation option
2384
2504
  --from-folder=VALUE share to share source folder
2385
2505
  --scope=VALUE OAuth scope for AoC API calls
2386
- --bulk=ENUM bulk operation: yes, no
2387
- --default-ports=ENUM use standard FASP ports or get from node api: yes, no
2506
+ --bulk=ENUM bulk operation: yes, [no]
2507
+ --default-ports=ENUM use standard FASP ports or get from node api: yes, [no]
2388
2508
 
2389
2509
 
2390
2510
  COMMAND: server
@@ -2442,11 +2562,17 @@ You can test with:
2442
2562
  ascli aoc user profile show
2443
2563
  ```
2444
2564
 
2445
- Optionally, it is possible to create a new organization-specific "integration".
2565
+ Optionally, it is possible to create a new organization-specific "integration", i.e. client application identification.
2446
2566
  For this, specify the option: `--use-generic-client=no`.
2447
2567
 
2448
2568
  This will guide you through the steps to create.
2449
2569
 
2570
+ If the wizard does not detect the application but you know the application, you can force it using option `value`:
2571
+
2572
+ ```bash
2573
+ ascli config wizard --value=aoc
2574
+ ```
2575
+
2450
2576
  ### <a id="aocmanual"></a>Configuration: using manual setup
2451
2577
 
2452
2578
  If you used the wizard (recommended): skip this section.
@@ -2461,7 +2587,7 @@ Several types of OAuth authentication are supported:
2461
2587
 
2462
2588
  The authentication method is controlled by option `auth`.
2463
2589
 
2464
- For a _quick start_, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
2590
+ For a *quick start*, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
2465
2591
 
2466
2592
  For a more convenient, browser-less, experience follow the [JWT](#jwt) section (auth=jwt) in addition to Client Registration.
2467
2593
 
@@ -2500,7 +2626,7 @@ Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interacti
2500
2626
  ```bash
2501
2627
  ascli config preset ask my_aoc_org url client_id client_secret
2502
2628
  option: url> https://myorg.ibmaspera.com/
2503
- option: client_id> BJLPObQiFw
2629
+ option: client_id> my_BJbQiFw
2504
2630
  option: client_secret> yFS1mu-crbKuQhGFtfhYuoRW...
2505
2631
  updated: my_aoc_org
2506
2632
  ```
@@ -2519,37 +2645,8 @@ Note: Default `auth` method is `web` and default `redirect_uri` is `http://local
2519
2645
 
2520
2646
  For a Browser-less, Private Key-based authentication, use the following steps.
2521
2647
 
2522
- ##### Key Pair Generation
2523
-
2524
2648
  In order to use JWT for Aspera on Cloud API client authentication,
2525
- a private/public key pair must be generated (without passphrase)
2526
- This can be done using any of the following method:
2527
-
2528
- (TODO: add passphrase protection as option).
2529
-
2530
- * using the CLI:
2531
-
2532
- ```bash
2533
- ascli config genkey ~/.aspera/ascli/aocapikey
2534
- ```
2535
-
2536
- * `ssh-keygen`:
2537
-
2538
- ```bash
2539
- ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
2540
- ```
2541
-
2542
- * `openssl`
2543
-
2544
- (on some openssl implementation (mac) there is option: -nodes (no DES))
2545
-
2546
- ```bash
2547
- APIKEY=~/.aspera/ascli/aocapikey
2548
- openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
2549
- openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
2550
- openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
2551
- rm -f ${APIKEY}.protected
2552
- ```
2649
+ a [private/public key pair](#private_key) must be used (without passphrase)
2553
2650
 
2554
2651
  ##### API Client JWT activation
2555
2652
 
@@ -2574,12 +2671,12 @@ ascli aoc admin res client list
2574
2671
  :............:...............:
2575
2672
  : id : name :
2576
2673
  :............:...............:
2577
- : BJLPObQiFw : my-client-app :
2674
+ : my_BJbQiFw : my-client-app :
2578
2675
  :............:...............:
2579
2676
  ```
2580
2677
 
2581
2678
  ```javascript
2582
- ascli aoc admin res client modify BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2679
+ ascli aoc admin res client modify my_BJbQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
2583
2680
  ```
2584
2681
 
2585
2682
  ```output
@@ -2592,12 +2689,12 @@ The public key must be assigned to your user. This can be done in two manners:
2592
2689
 
2593
2690
  ##### Graphically
2594
2691
 
2595
- open the previously generated public key located here: `$HOME/.aspera/ascli/aocapikey.pub`
2692
+ Open the previously generated public key located here: `$HOME/.aspera/ascli/my_private_key.pub`
2596
2693
 
2597
2694
  * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
2598
2695
  * Click on the user's icon (top right)
2599
2696
  * Select "Account Settings"
2600
- * Paste the _Public Key_ in the "Public Key" section
2697
+ * Paste the *Public Key* in the "Public Key" section
2601
2698
  * Click on "Submit"
2602
2699
 
2603
2700
  ##### Using command line
@@ -2616,7 +2713,7 @@ ascli aoc admin res user list
2616
2713
  ```
2617
2714
 
2618
2715
  ```ruby
2619
- ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/aocapikey.pub"))}'
2716
+ ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/my_private_key.pub"))}'
2620
2717
  ```
2621
2718
 
2622
2719
  ```output
@@ -2636,7 +2733,7 @@ To activate default use of JWT authentication for `ascli` using the [option pres
2636
2733
  Execute:
2637
2734
 
2638
2735
  ```bash
2639
- ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/aocapikey --username=laurent.martin.aspera@fr.ibm.com
2736
+ ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=laurent.martin.aspera@fr.ibm.com
2640
2737
  ```
2641
2738
 
2642
2739
  Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.
@@ -2723,7 +2820,7 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
2723
2820
 
2724
2821
  To execute an action on a specific resource, select it using one of those methods:
2725
2822
 
2726
- * **recommended:** give id directly on command line *after the action*: `aoc admin res node show 123`
2823
+ * *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
2727
2824
  * give name on command line *after the action*: `aoc admin res node show name abc`
2728
2825
  * provide option `id` : `aoc admin res node show --id=123`
2729
2826
  * provide option `name` : `aoc admin res node show --name=abc`
@@ -3261,19 +3358,19 @@ f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
3261
3358
  * Find files older than 1 year on a given node and store in file list
3262
3359
 
3263
3360
  ```bash
3264
- ascli aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
3361
+ ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
3265
3362
  ```
3266
3363
 
3267
3364
  * Delete the files, one by one
3268
3365
 
3269
3366
  ```bash
3270
- cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
3367
+ cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='_secret_here_' v4 delete "$path" ;done
3271
3368
  ```
3272
3369
 
3273
3370
  * Delete the files in bulk
3274
3371
 
3275
3372
  ```bash
3276
- cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
3373
+ cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='_secret_here_' v3 delete @lines:@stdin:
3277
3374
  ```
3278
3375
 
3279
3376
  ## <a id="ats"></a>Plugin: Aspera Transfer Service
@@ -3359,13 +3456,13 @@ ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --a
3359
3456
  Example: create access key on IBM Cloud (softlayer):
3360
3457
 
3361
3458
  ```javascript
3362
- ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
3459
+ ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"_secret_here_","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
3363
3460
  ```
3364
3461
 
3365
3462
  Example: create access key on AWS:
3366
3463
 
3367
3464
  ```javascript
3368
- ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
3465
+ ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"_secret_here_"},"path":"/laurent"}}'
3369
3466
  ```
3370
3467
 
3371
3468
  Example: create access key on Azure SAS:
@@ -3474,7 +3571,8 @@ It is possible to:
3474
3571
 
3475
3572
  For transfers, it is possible to control how transfer is authorized using option: `token_type`:
3476
3573
 
3477
- * `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
3574
+ * `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token, used as is.
3575
+ * `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
3478
3576
  * `basic` : transfer spec is created like this:
3479
3577
 
3480
3578
  ```javascript
@@ -3483,11 +3581,11 @@ For transfers, it is possible to control how transfer is authorized using option
3483
3581
  "remote_user": "xfer",
3484
3582
  "ssh_port": 33001,
3485
3583
  "token": "Basic <base 64 encoded user/pass>",
3486
- "direction": send/recv
3584
+ "direction": send/receive
3487
3585
  }
3488
3586
  ```
3489
3587
 
3490
- * `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
3588
+ Note that the port is assumed to be the default SSH port `33001` and transfer user is assumed to be `xfer`.
3491
3589
 
3492
3590
  ### Central
3493
3591
 
@@ -3505,10 +3603,10 @@ by providing the `validator` option, offline transfer validation can be done.
3505
3603
 
3506
3604
  It is possible to start a FASPStream session using the node API:
3507
3605
 
3508
- Use the "node stream create" command, then arguments are provided as a [_transfer-spec_](#transferspec).
3606
+ Use the "node stream create" command, then arguments are provided as a [*transfer-spec*](#transferspec).
3509
3607
 
3510
3608
  ```javascript
3511
- ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
3609
+ ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"_pass_here_"}' --preset=stream
3512
3610
  ```
3513
3611
 
3514
3612
  ### Watchfolder
@@ -3573,15 +3671,63 @@ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"myst
3573
3671
 
3574
3672
  ## Plugin: IBM Aspera Faspex5
3575
3673
 
3674
+ This is currently in beta, limited operations are supported.
3675
+
3676
+ This was tested with version Beta 5.
3677
+
3678
+ The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "IBM Aspera Faspex API".
3679
+
3680
+ ### Faspex 5 authentication
3681
+
3576
3682
  3 authentication methods are supported:
3577
3683
 
3578
3684
  * jwt
3579
3685
  * web
3580
3686
  * boot
3581
3687
 
3582
- For JWT, create an API client in Faspex with jwt support, and use: `--auth=jwt`.
3688
+ #### Faspex 5 using JWT
3689
+
3690
+ This is the preferred method to use.
3691
+
3692
+ For JWT, create an API client in Faspex with JWT support:
3693
+
3694
+ * Identify a private key, if you don't have any refer to section [Private Key](#private_key)
3695
+ * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
3696
+ * Activate JWT
3697
+ * Paste public key in the appropriate section
3698
+ * Click on Create Button
3699
+ * Take note of Client Id and Secret
3700
+
3701
+ Then use options:
3702
+
3703
+ ```text
3704
+ --auth=jwt
3705
+ --client-id=xxx
3706
+ --client-secret=xxx
3707
+ --username=xxx
3708
+ --password=xxx
3709
+ --private-key=@file:../path/to/key.pem
3710
+ ```
3711
+
3712
+ #### Faspex 5 using web browser
3713
+
3714
+ For web method, create an API client in Faspex without JWT:
3715
+
3716
+ * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
3717
+ * Do not Activate JWT
3718
+ * enter https://127.0.0.1:8888 in the redirect URI
3719
+ * Click on Create Button
3720
+ * Take note of Client Id
3583
3721
 
3584
- For web method, create an API client in Faspex, and use: --auth=web
3722
+ Then use options:
3723
+
3724
+ ```text
3725
+ --auth=web
3726
+ --client-id=xxx
3727
+ --redirect-uri=https://127.0.0.1:8888
3728
+ ```
3729
+
3730
+ #### Faspex 5 using bootstrap
3585
3731
 
3586
3732
  For boot method: (will be removed in future)
3587
3733
 
@@ -3596,8 +3742,6 @@ Use it as password and use `--auth=boot`.
3596
3742
  ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
3597
3743
  ```
3598
3744
 
3599
- Ready to use Faspex5 with CLI.
3600
-
3601
3745
  ## Plugin: IBM Aspera Faspex (4.x)
3602
3746
 
3603
3747
  Notes:
@@ -3618,7 +3762,7 @@ By default it looks in box `inbox`, but the following boxes are also supported:
3618
3762
  A user can receive a package because the recipient is:
3619
3763
 
3620
3764
  * the user himself (default)
3621
- * the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
3765
+ * the user is member of a dropbox/workgroup: filter using option `recipient` set with value `*<name of dropbox/workgroup>`
3622
3766
 
3623
3767
  #### Option `query`
3624
3768
 
@@ -3657,10 +3801,11 @@ ascli faspex package recv --id=12345
3657
3801
  ascli faspex package recv --link=faspe://...
3658
3802
  ```
3659
3803
 
3660
- If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
3804
+ If the package is in a specific **dropbox**/**workgroup**, add option `recipient` for both the `list` and `recv` commands.
3661
3805
 
3662
3806
  ```bash
3663
3807
  ascli faspex package list --recipient='*thedropboxname'
3808
+ ascli faspex package recv 125 --recipient='*thedropboxname'
3664
3809
  ```
3665
3810
 
3666
3811
  if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
@@ -3675,7 +3820,8 @@ Example:
3675
3820
  ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
3676
3821
  ```
3677
3822
 
3678
- If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
3823
+ If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
3824
+ `"recipients":["*MyDropboxName"]`
3679
3825
 
3680
3826
  Additional optional parameters in `delivery_info`:
3681
3827
 
@@ -3716,7 +3862,7 @@ Example:
3716
3862
  my_faspex_conf:
3717
3863
  url: https://10.25.0.3/aspera/faspex
3718
3864
  username: admin
3719
- password: MyPassword
3865
+ password: MyUserPassword
3720
3866
  storage:
3721
3867
  testlaurent:
3722
3868
  node: "@preset:my_faspex_node"
@@ -3724,14 +3870,14 @@ my_faspex_conf:
3724
3870
  my_faspex_node:
3725
3871
  url: https://10.25.0.3:9092
3726
3872
  username: node_faspex
3727
- password: MyPassword
3873
+ password: MyNodePassword
3728
3874
  ```
3729
3875
 
3730
3876
  In this example, a faspex storage named "testlaurent" exists in Faspex, and is located
3731
3877
  under the docroot in "/myfiles" (this must be the same as configured in Faspex).
3732
3878
  The node configuration name is "my_faspex_node" here.
3733
3879
 
3734
- Note: the v4 API provide an API for nodes and shares.
3880
+ Note: the v4 API provides an API for nodes and shares.
3735
3881
 
3736
3882
  ### Automated package download (cargo)
3737
3883
 
@@ -3746,12 +3892,6 @@ ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
3746
3892
 
3747
3893
  Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
3748
3894
 
3749
- In Shares2, users, groups listing are paged, to display sequential pages:
3750
-
3751
- ```bash
3752
- for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}';done
3753
- ```
3754
-
3755
3895
  ## Plugin: IBM Cloud Object Storage
3756
3896
 
3757
3897
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
@@ -3799,14 +3939,14 @@ It consists in the following structure:
3799
3939
 
3800
3940
  ```javascript
3801
3941
  {
3802
- "apikey": "xxxxxxx.....",
3942
+ "apikey": "_api_key_here_",
3803
3943
  "cos_hmac_keys": {
3804
- "access_key_id": "xxxxxxx.....",
3805
- "secret_access_key": "xxxxxxx....."
3944
+ "access_key_id": "_access_key_here_",
3945
+ "secret_access_key": "_secret_here_"
3806
3946
  },
3807
3947
  "endpoints": "https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints",
3808
- "iam_apikey_description": "my description ...",
3809
- "iam_apikey_name": "my key name",
3948
+ "iam_apikey_description": "my description _here_ ...",
3949
+ "iam_apikey_name": "my key name _here_",
3810
3950
  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",
3811
3951
  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/xxxxxxx.....",
3812
3952
  "resource_instance_id": "crn:v1:bluemix:public:cloud-object-storage:global:a/xxxxxxx....."
@@ -3904,7 +4044,6 @@ The tool requires the following external tools available in the `PATH`:
3904
4044
  * OptiPNG : `optipng`
3905
4045
  * FFmpeg : `ffmpeg` `ffprobe`
3906
4046
  * Libreoffice : `libreoffice`
3907
- * ruby gem `mimemagic`
3908
4047
 
3909
4048
  Here shown on Redhat/CentOS.
3910
4049
 
@@ -3916,32 +4055,16 @@ To check if all tools are found properly, execute:
3916
4055
  ascli preview check
3917
4056
  ```
3918
4057
 
3919
- #### mimemagic
3920
-
3921
- To benefit from extra mime type detection install gem mimemagic:
3922
-
3923
- ```bash
3924
- gem install mimemagic
3925
- ```
3926
-
3927
- or to install an earlier version if any problem:
3928
-
3929
- ```bash
3930
- gem install mimemagic -v '~> 0.3.0'
3931
- ```
3932
-
3933
- To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
3934
-
3935
- If not used, Mime type used for conversion is the one provided by the node API.
3936
-
3937
- If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
3938
-
3939
4058
  #### Image: ImageMagick and optipng
3940
4059
 
3941
4060
  ```bash
3942
4061
  yum install -y ImageMagick optipng
3943
4062
  ```
3944
4063
 
4064
+ You may also install `ghostscript` which adds fonts to ImageMagick.
4065
+ Available fonts, used to generate png for text, can be listed with `magick identify -list font`.
4066
+ Prefer ImageMagick version >=7.
4067
+
3945
4068
  #### Video: FFmpeg
3946
4069
 
3947
4070
  The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
@@ -4114,7 +4237,13 @@ The mp4 video preview file is only for category `video`
4114
4237
 
4115
4238
  File type is primarily based on file extension detected by the node API and translated info a mime type returned by the node API.
4116
4239
 
4117
- The tool can also locally detect the mime type using gem `mimemagic`.
4240
+ The tool can also locally detect the mime type using option `mimemagic`.
4241
+
4242
+ To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
4243
+
4244
+ If not used, Mime type used for conversion is the one provided by the node API.
4245
+
4246
+ If used, the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
4118
4247
 
4119
4248
  ### Access to original files and preview creation
4120
4249
 
@@ -4149,19 +4278,19 @@ The `smtp` option is a hash table (extended value) with the following fields:
4149
4278
  ```bash
4150
4279
  ascli config preset set smtp_google server smtp.google.com
4151
4280
  ascli config preset set smtp_google username john@gmail.com
4152
- ascli config preset set smtp_google password P@ssw0rd
4281
+ ascli config preset set smtp_google password _pass_here_
4153
4282
  ```
4154
4283
 
4155
4284
  or
4156
4285
 
4157
4286
  ```javascript
4158
- ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
4287
+ ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"_pass_here_"}'
4159
4288
  ```
4160
4289
 
4161
4290
  or
4162
4291
 
4163
4292
  ```bash
4164
- ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
4293
+ ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=_pass_here_
4165
4294
  ```
4166
4295
 
4167
4296
  Set this configuration as global default, for instance:
@@ -4224,7 +4353,7 @@ Transfer is: <%=global_transfer_status%>
4224
4353
  This gem comes with a second executable tool providing a simplified standardized interface
4225
4354
  to start a FASP session: `asession`.
4226
4355
 
4227
- It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [_transfer-spec_](#transferspec) is:
4356
+ It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transferspec) is:
4228
4357
 
4229
4358
  * common to Aspera Node API (HTTP POST /ops/transfer)
4230
4359
  * common to Aspera Connect API (browser javascript startTransfer)
@@ -4234,20 +4363,20 @@ Hopefully, IBM integrates this diectly in `ascp`, and this tool is made redundan
4234
4363
 
4235
4364
  This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
4236
4365
 
4237
- The tool expect one single argument: a [_transfer-spec_](#transferspec).
4366
+ The tool expect one single argument: a [*transfer-spec*](#transferspec).
4238
4367
 
4239
- If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [_transfer-spec_](#transferspec) on stdin.
4368
+ If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
4240
4369
 
4241
4370
  Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
4242
4371
 
4243
4372
  During execution, it generates all low level events, one per line, in JSON format on stdout.
4244
4373
 
4245
- Note that there are special "extended" [_transfer-spec_](#transferspec) parameters supported by `asession`:
4374
+ Note that there are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
4246
4375
 
4247
4376
  * `EX_loglevel` to change log level of the tool
4248
4377
  * `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
4249
4378
 
4250
- Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
4379
+ Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
4251
4380
 
4252
4381
  ### Comparison of interfaces
4253
4382
 
@@ -4262,7 +4391,7 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
4262
4391
  ### Simple session
4263
4392
 
4264
4393
  ```javascript
4265
- MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
4394
+ MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
4266
4395
 
4267
4396
  echo "${MY_TSPEC}"|asession
4268
4397
  ```
@@ -4271,11 +4400,11 @@ echo "${MY_TSPEC}"|asession
4271
4400
 
4272
4401
  `asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -&gt; `long_parameter`
4273
4402
 
4274
- This is particularly useful for a persistent session ( with the [_transfer-spec_](#transferspec) parameter: `"keepalive":true` )
4403
+ This is particularly useful for a persistent session ( with the [*transfer-spec*](#transferspec) parameter: `"keepalive":true` )
4275
4404
 
4276
4405
  ```javascript
4277
4406
  asession
4278
- {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
4407
+ {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_pass_here_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
4279
4408
  {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
4280
4409
  {"type":"DONE"}
4281
4410
  ```
@@ -4348,7 +4477,7 @@ Note that:
4348
4477
 
4349
4478
  * `ascli` takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
4350
4479
  * most, but not all native ascp arguments are available as standard transfer_spec parameters
4351
- * native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
4480
+ * native ascp arguments can be provided with the [*transfer-spec*](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
4352
4481
 
4353
4482
  #### server side and configuration
4354
4483
 
@@ -4419,11 +4548,11 @@ Main components:
4419
4548
  A working example can be found in the gem, example:
4420
4549
 
4421
4550
  ```bash
4422
- ascli config gem_path
4551
+ ascli config gem path
4423
4552
  ```
4424
4553
 
4425
4554
  ```bash
4426
- cat $(ascli config gem_path)/../examples/transfer.rb
4555
+ cat $(ascli config gem path)/../examples/transfer.rb
4427
4556
  ```
4428
4557
 
4429
4558
  This sample code shows some example of use of the API as well as REST API.
@@ -4454,11 +4583,27 @@ So, it evolved into `ascli`:
4454
4583
 
4455
4584
  * portable: works on platforms supporting `ruby` (and `ascp`)
4456
4585
  * easy to install with the `gem` utility
4457
- * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
4586
+ * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
4458
4587
  * `ruby` is consistent with other Aspera products
4459
4588
 
4460
4589
  ## Changes (Release notes)
4461
4590
 
4591
+ * 4.7.0
4592
+
4593
+ * new: option to specify font used to generate image of text file in `preview`
4594
+ * new: #66 improvement for content protection (support standard transfer spec options for direct agent)
4595
+ * new: option `fpac` is now applicable to all ruby based HTTP connections, i.e. API calls
4596
+ * new: option `show_secrets` to reveal secrets in command output
4597
+ * new: added and updated commands for Faspex 5
4598
+ * new: option `cache_tokens`
4599
+ * new: Faspex4 dropbox packages can now be received by id
4600
+ * change: (break) command `conf gem path` replaces `conf gem_path`
4601
+ * change: (break) option `fpac` expects a value instead of URL
4602
+ * change: (break) option `cipher` in transfer spec must have hyphen
4603
+ * change: (break) renamed option `log_passwords` to `log_secrets`
4604
+ * change: (break) removed plugin `shares2` as products is now EOL
4605
+ * fix: After AoC version update, wizard did not detect AoC properly
4606
+
4462
4607
  * 4.6.0
4463
4608
 
4464
4609
  * new: command `conf plugin create`
@@ -4823,7 +4968,7 @@ So, it evolved into `ascli`:
4823
4968
 
4824
4969
  * 0.9.7
4825
4970
 
4826
- * homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4971
+ * homogeneous [*transfer-spec*](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4827
4972
  * preview persistency goes to unique file by default
4828
4973
  * catch mxf extension in preview as video
4829
4974
  * Faspex: possibility to download all packages by specifying id=ALL
@@ -4898,24 +5043,21 @@ So, it evolved into `ascli`:
4898
5043
 
4899
5044
  * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in [option preset](#lprt) "default".
4900
5045
 
4901
- ## BUGS, FEATURES, CONTRIBUTION
4902
-
4903
- For issues or feature requests use the Github repository and issues.
5046
+ ## Common problems
4904
5047
 
4905
- You can also contribute to this open source project.
5048
+ ### Error "Remote host is not who we expected"
4906
5049
 
4907
- One can also [create one's own plugin](#createownplugin).
5050
+ Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
4908
5051
 
4909
- ### Only one value for any option
5052
+ Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
4910
5053
 
4911
- Some commands and sub commands may ask for the same option name.
4912
- Currently, since option definition is position independent (last one wins), it is not possible
4913
- to give an option to a command and the same option with different value to a sub command.
5054
+ ```json
5055
+ --ts=@json:'{"sshfp":null}'
5056
+ ```
4914
5057
 
4915
- For instance, if an entity is identified by the option `id` but later on the command line another `id` option is required, then the later will override the earlier one, and both entity will use the same id.
4916
- As a solution, use the position specific notation for selection, i.e. provide the identified just after command and do not use option `id`.
5058
+ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
4917
5059
 
4918
- This happens typically for the `node` sub command, e.g. identify the node by name instead of id.
5060
+ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
4919
5061
 
4920
5062
  ### ED255519 key not supported
4921
5063
 
@@ -4930,33 +5072,18 @@ OpenSSH keys only supported if ED25519 is available
4930
5072
  Which meant that you do not have ruby support for ED25519 SSH keys.
4931
5073
  You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
4932
5074
 
4933
- ### Error "Remote host is not who we expected"
4934
-
4935
- Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
4936
-
4937
- Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
4938
-
4939
- ```bash
4940
- --ts=@json:'{"sshfp":null}'
4941
- ```
5075
+ ## BUGS, FEATURES, CONTRIBUTION
4942
5076
 
4943
- Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
5077
+ For issues reports or feature requests use the Github repository and issues.
4944
5078
 
4945
- References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
5079
+ You can also contribute to this open source project.
4946
5080
 
4947
- ### Miscellaneous
5081
+ One can also [create one's own plugin](#createownplugin).
4948
5082
 
4949
- * remove rest and oauth classes and use ruby standard gems:
5083
+ ## Miscellaneous
4950
5084
 
4951
- * oauth
5085
+ * replace rest and oauth classes with ruby standard gems:
4952
5086
  * <https://github.com/rest-client/rest-client>
4953
-
4954
- * use Thor or any standard Ruby CLI manager
4955
-
4956
- * provide metadata in packages
4957
-
4958
- * deliveries to dropboxes
4959
-
4960
- * Going through proxy: use env var http_proxy and https_proxy, no_proxy
4961
-
4962
- * easier use with <https://github.com/pmq20/ruby-packer>
5087
+ * <https://github.com/oauth-xx/oauth2>
5088
+ * use gem Thor <http://whatisthor.com/> (or other standard Ruby CLI manager)
5089
+ * Package with <https://github.com/pmq20/ruby-packer> (rubyc)