aspera-cli 4.16.0 → 4.17.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +50 -19
- data/CONTRIBUTING.md +3 -1
- data/README.md +965 -793
- data/bin/asession +29 -21
- data/lib/aspera/{fasp/agent_alpha.rb → agent/alpha.rb} +26 -25
- data/lib/aspera/{fasp/agent_base.rb → agent/base.rb} +15 -12
- data/lib/aspera/{fasp/agent_connect.rb → agent/connect.rb} +13 -11
- data/lib/aspera/{fasp/agent_direct.rb → agent/direct.rb} +49 -53
- data/lib/aspera/{fasp/agent_httpgw.rb → agent/httpgw.rb} +20 -19
- data/lib/aspera/{fasp/agent_node.rb → agent/node.rb} +20 -33
- data/lib/aspera/{fasp/agent_trsdk.rb → agent/trsdk.rb} +11 -11
- data/lib/aspera/api/aoc.rb +586 -0
- data/lib/aspera/api/ats.rb +46 -0
- data/lib/aspera/api/cos_node.rb +95 -0
- data/lib/aspera/api/node.rb +344 -0
- data/lib/aspera/ascmd.rb +46 -10
- data/lib/aspera/{fasp → ascp}/installation.rb +5 -5
- data/lib/aspera/{fasp → ascp}/management.rb +3 -8
- data/lib/aspera/{fasp → ascp}/products.rb +1 -1
- data/lib/aspera/assert.rb +30 -30
- data/lib/aspera/cli/basic_auth_plugin.rb +11 -10
- data/lib/aspera/cli/extended_value.rb +1 -1
- data/lib/aspera/cli/formatter.rb +13 -13
- data/lib/aspera/cli/hints.rb +5 -5
- data/lib/aspera/cli/main.rb +35 -28
- data/lib/aspera/cli/manager.rb +25 -24
- data/lib/aspera/cli/plugin.rb +22 -15
- data/lib/aspera/cli/plugin_factory.rb +61 -0
- data/lib/aspera/cli/plugins/alee.rb +7 -7
- data/lib/aspera/cli/plugins/aoc.rb +83 -77
- data/lib/aspera/cli/plugins/ats.rb +32 -33
- data/lib/aspera/cli/plugins/bss.rb +3 -4
- data/lib/aspera/cli/plugins/config.rb +169 -186
- data/lib/aspera/cli/plugins/console.rb +8 -6
- data/lib/aspera/cli/plugins/cos.rb +19 -18
- data/lib/aspera/cli/plugins/faspex.rb +61 -54
- data/lib/aspera/cli/plugins/faspex5.rb +150 -103
- data/lib/aspera/cli/plugins/node.rb +68 -73
- data/lib/aspera/cli/plugins/orchestrator.rb +34 -44
- data/lib/aspera/cli/plugins/preview.rb +31 -31
- data/lib/aspera/cli/plugins/server.rb +31 -33
- data/lib/aspera/cli/plugins/shares.rb +13 -11
- data/lib/aspera/cli/sync_actions.rb +8 -8
- data/lib/aspera/cli/transfer_agent.rb +32 -19
- data/lib/aspera/cli/transfer_progress.rb +1 -1
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/colors.rb +5 -0
- data/lib/aspera/command_line_builder.rb +14 -14
- data/lib/aspera/coverage.rb +1 -2
- data/lib/aspera/data_repository.rb +1 -1
- data/lib/aspera/environment.rb +2 -3
- data/lib/aspera/faspex_gw.rb +5 -6
- data/lib/aspera/faspex_postproc.rb +1 -1
- data/lib/aspera/id_generator.rb +2 -2
- data/lib/aspera/json_rpc.rb +5 -5
- data/lib/aspera/keychain/encrypted_hash.rb +6 -6
- data/lib/aspera/keychain/macos_security.rb +27 -22
- data/lib/aspera/log.rb +2 -2
- data/lib/aspera/nagios.rb +3 -3
- data/lib/aspera/node_simulator.rb +5 -6
- data/lib/aspera/oauth/base.rb +143 -0
- data/lib/aspera/oauth/factory.rb +124 -0
- data/lib/aspera/oauth/generic.rb +34 -0
- data/lib/aspera/oauth/jwt.rb +51 -0
- data/lib/aspera/oauth/url_json.rb +31 -0
- data/lib/aspera/oauth/web.rb +50 -0
- data/lib/aspera/oauth.rb +5 -331
- data/lib/aspera/open_application.rb +7 -7
- data/lib/aspera/persistency_action_once.rb +4 -4
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/generator.rb +5 -5
- data/lib/aspera/preview/terminal.rb +3 -2
- data/lib/aspera/preview/utils.rb +3 -3
- data/lib/aspera/proxy_auto_config.rb +4 -4
- data/lib/aspera/rest.rb +175 -144
- data/lib/aspera/rest_errors_aspera.rb +3 -3
- data/lib/aspera/resumer.rb +77 -0
- data/lib/aspera/ssh.rb +6 -1
- data/lib/aspera/{fasp → transfer}/error.rb +3 -3
- data/lib/aspera/{fasp → transfer}/error_info.rb +1 -1
- data/lib/aspera/{fasp → transfer}/faux_file.rb +1 -1
- data/lib/aspera/{fasp → transfer}/parameters.rb +58 -89
- data/lib/aspera/{fasp/transfer_spec.rb → transfer/spec.rb} +18 -16
- data/lib/aspera/{fasp/parameters.yaml → transfer/spec.yaml} +4 -99
- data/lib/aspera/{fasp → transfer}/sync.rb +32 -32
- data/lib/aspera/{fasp → transfer}/uri.rb +9 -8
- data/lib/aspera/web_server_simple.rb +11 -3
- data.tar.gz.sig +0 -0
- metadata +36 -63
- metadata.gz.sig +0 -0
- data/lib/aspera/aoc.rb +0 -601
- data/lib/aspera/ats_api.rb +0 -47
- data/lib/aspera/cos_node.rb +0 -94
- data/lib/aspera/fasp/resume_policy.rb +0 -79
- data/lib/aspera/node.rb +0 -339
data/README.md
CHANGED
@@ -10,7 +10,7 @@
|
|
10
10
|
|
11
11
|
## Introduction
|
12
12
|
|
13
|
-
Version : 4.
|
13
|
+
Version : 4.17.0
|
14
14
|
|
15
15
|
Laurent/2016-2024
|
16
16
|
|
@@ -37,7 +37,7 @@ A PDF version of this documentation is available here: [docs/Manual.pdf](docs/Ma
|
|
37
37
|
|
38
38
|
Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
39
39
|
|
40
|
-
###
|
40
|
+
### When to use and when not to use
|
41
41
|
|
42
42
|
`ascli` is designed to be used as a command line tool to:
|
43
43
|
|
@@ -52,7 +52,7 @@ It is designed for:
|
|
52
52
|
`ascli` can be seen as a command line tool integrating:
|
53
53
|
|
54
54
|
- A configuration file (`config.yaml`)
|
55
|
-
- Advanced command line options ([Extended Value](#extended))
|
55
|
+
- Advanced command line options ([Extended Value](#extended-value-syntax))
|
56
56
|
- `curl` (for REST calls)
|
57
57
|
- Aspera transfer (`ascp`)
|
58
58
|
|
@@ -75,10 +75,10 @@ Command line parameters in examples beginning with `my_`, e.g. `my_param_value`,
|
|
75
75
|
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
|
76
76
|
|
77
77
|
Some commands will start an Aspera-based transfer (e.g. `upload`).
|
78
|
-
The transfer is not directly implemented in `ascli`, rather `ascli` uses one of the external Aspera Transfer Clients called **[Transfer Agents](#agents)**.
|
78
|
+
The transfer is not directly implemented in `ascli`, rather `ascli` uses one of the external Aspera Transfer Clients called **[Transfer Agents](#transfer-clients-agents)**.
|
79
79
|
|
80
|
-
> **Note:** A **[Transfer Agents](#agents)** is a client for the remote Transfer Server (HSTS).
|
81
|
-
The **[Transfer Agents](#agents)** may be local or remote...
|
80
|
+
> **Note:** A **[Transfer Agents](#transfer-clients-agents)** is a client for the remote Transfer Server (HSTS).
|
81
|
+
The **[Transfer Agents](#transfer-clients-agents)** may be local or remote...
|
82
82
|
For example a remote Aspera Server may be used as a transfer agent (using node API).
|
83
83
|
i.e. using option `--transfer=node`
|
84
84
|
|
@@ -92,14 +92,14 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
92
92
|
|
93
93
|
```console
|
94
94
|
$ ascli --version
|
95
|
-
4.
|
95
|
+
4.17.0
|
96
96
|
```
|
97
97
|
|
98
98
|
### First use
|
99
99
|
|
100
100
|
Once installation is completed, you can proceed to the first use with a demo server:
|
101
101
|
|
102
|
-
If you want to test with Aspera on Cloud, jump to section: [Wizard](#
|
102
|
+
If you want to test with Aspera on Cloud, jump to section: [Wizard](#configuration-using-wizard).
|
103
103
|
|
104
104
|
To test with Aspera demo transfer server, setup the environment and then test:
|
105
105
|
|
@@ -122,10 +122,10 @@ ascli server browse /
|
|
122
122
|
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
123
123
|
```
|
124
124
|
|
125
|
-
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [
|
125
|
+
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset)' for the server's authentication options.
|
126
126
|
The following example will:
|
127
127
|
|
128
|
-
- Create a [
|
128
|
+
- Create a [Option Preset](#option-preset)'
|
129
129
|
- Define it as default for the `server` plugin
|
130
130
|
- List files in a folder
|
131
131
|
- Download a file
|
@@ -177,19 +177,19 @@ complete
|
|
177
177
|
|
178
178
|
### Going further
|
179
179
|
|
180
|
-
Get familiar with configuration, options, commands : [Command Line Interface](#
|
180
|
+
Get familiar with configuration, options, commands : [Command Line Interface](#command-line-interface).
|
181
181
|
|
182
182
|
Then, follow the section relative to the product you want to interact with ( Aspera on Cloud, Faspex, ...) : [Application Plugins](plugins)
|
183
183
|
|
184
|
-
##
|
184
|
+
## Installation
|
185
185
|
|
186
186
|
It is possible to install **either** directly on the host operating system (Linux, macOS, Windows) or as a [container](#container) (`docker`, `podman`, `singularity`).
|
187
187
|
|
188
188
|
The direct installation is recommended and consists in installing:
|
189
189
|
|
190
190
|
- [Ruby](#ruby)
|
191
|
-
- [aspera-cli](#
|
192
|
-
- [Aspera SDK (`ascp`)](#
|
191
|
+
- [aspera-cli](#ruby-gem)
|
192
|
+
- [Aspera SDK (`ascp`)](#fasp-protocol)
|
193
193
|
|
194
194
|
Ruby version: >= 2.6.
|
195
195
|
|
@@ -198,11 +198,11 @@ Ruby version: >= 2.6.
|
|
198
198
|
The following sections provide information on the various installation methods.
|
199
199
|
|
200
200
|
An internet connection is required for the installation.
|
201
|
-
If you don't have internet for the installation, refer to section [Installation without internet access](#
|
201
|
+
If you don't have internet for the installation, refer to section [Installation without internet access](#installation-in-air-gapped-environment).
|
202
202
|
|
203
203
|
A package with pre-installed Ruby, gem and ascp may also be provided.
|
204
204
|
|
205
|
-
###
|
205
|
+
### Ruby
|
206
206
|
|
207
207
|
Use this method to install on the native host (e.g. your Windows, macOS or Linux system).
|
208
208
|
|
@@ -253,7 +253,7 @@ Install the chosen pre-compiled Ruby version:
|
|
253
253
|
rvm install 3.2.2
|
254
254
|
```
|
255
255
|
|
256
|
-
Ruby is now installed for the user, go to [Gem installation](#
|
256
|
+
Ruby is now installed for the user, go to [Gem installation](#ruby-gem).
|
257
257
|
|
258
258
|
Alternatively RVM can be installed system-wide, for this execute as `root`.
|
259
259
|
It then installs by default in `/usr/local/rvm` for all users and creates `/etc/profile.d/rvm.sh`.
|
@@ -434,11 +434,34 @@ make
|
|
434
434
|
make install
|
435
435
|
```
|
436
436
|
|
437
|
-
|
437
|
+
#### JRuby
|
438
|
+
|
439
|
+
`ascli` can also run with the [JRuby](https://www.jruby.org/) interpreter.
|
440
|
+
All what is needed is a JVM (Java Virtual Machine) on your system (`java`).
|
441
|
+
The JRuby package comes pre-complied and does not require compilation of native extensions.
|
442
|
+
Use a version of JRuby compatible with Ruby version supported by `ascli`.
|
443
|
+
Refer to [the wikipedia page](https://en.wikipedia.org/wiki/JRuby) to match JRuby and Ruby versions.
|
444
|
+
Choose the latest version from:
|
438
445
|
|
439
446
|
<https://www.jruby.org/download>
|
440
447
|
|
441
|
-
> **Note:**
|
448
|
+
> **Note:** The startup time is slightly longer using `jruby` than the native Ruby, refer to the [JRuby wiki](https://github.com/jruby/jruby/wiki) for details. This can be reduced by using the `--dev` option.
|
449
|
+
> The transfer speed is not impacted (executed by `ascp` binary).
|
450
|
+
>
|
451
|
+
> **Note:** JRuby can be [installed](https://www.jruby.org/getting-started) using `rvm`.
|
452
|
+
|
453
|
+
Example: start `ascli` with JRuby and reduce startup time:
|
454
|
+
|
455
|
+
```bash
|
456
|
+
export JRUBY_OPTS=--dev
|
457
|
+
ascli -v
|
458
|
+
```
|
459
|
+
|
460
|
+
or
|
461
|
+
|
462
|
+
```bash
|
463
|
+
JRUBY_OPTS=--dev ascli -v
|
464
|
+
```
|
442
465
|
|
443
466
|
#### Optional gems
|
444
467
|
|
@@ -456,7 +479,7 @@ gem install rmagick grpc mimemagic
|
|
456
479
|
|
457
480
|
> **Note:** Those are not installed as part of dependencies because they involve compilation of native code.
|
458
481
|
|
459
|
-
###
|
482
|
+
### Ruby Gem
|
460
483
|
|
461
484
|
Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
|
462
485
|
|
@@ -479,7 +502,7 @@ To check if a new version is available (independently of `version_check_days`):
|
|
479
502
|
ascli config check_update
|
480
503
|
```
|
481
504
|
|
482
|
-
###
|
505
|
+
### FASP Protocol
|
483
506
|
|
484
507
|
Most file transfers will be executed using the **FASP** protocol, using `ascp`.
|
485
508
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
@@ -516,13 +539,13 @@ For instance, Aspera Connect Client can be installed by visiting the page:
|
|
516
539
|
[https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
|
517
540
|
|
518
541
|
`ascli` will detect most of Aspera transfer products in standard locations and use the first one found by default.
|
519
|
-
Refer to section [FASP](#
|
542
|
+
Refer to section [FASP](#fasp-configuration) for details on how to select a client or set path to the FASP protocol.
|
520
543
|
|
521
544
|
Several methods are provided to start a transfer.
|
522
|
-
Use of a local client ([`direct`](#
|
523
|
-
Refer to section: [Transfer Agents](#agents)
|
545
|
+
Use of a local client ([`direct`](#agent-direct) transfer agent) is one of them, but other methods are available.
|
546
|
+
Refer to section: [Transfer Agents](#transfer-clients-agents)
|
524
547
|
|
525
|
-
###
|
548
|
+
### Installation in air gapped environment
|
526
549
|
|
527
550
|
> **Note:** No pre-packaged version is provided yet.
|
528
551
|
|
@@ -636,7 +659,7 @@ ascli -v
|
|
636
659
|
```
|
637
660
|
|
638
661
|
```text
|
639
|
-
4.
|
662
|
+
4.17.0
|
640
663
|
```
|
641
664
|
|
642
665
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
@@ -779,7 +802,9 @@ Or get a shell with access to `ascli` like this:
|
|
779
802
|
singularity shell ascli.sif
|
780
803
|
```
|
781
804
|
|
782
|
-
##
|
805
|
+
## Command Line Interface
|
806
|
+
|
807
|
+
The command line tool is: ``ascli``
|
783
808
|
|
784
809
|
The `aspera-cli` gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
785
810
|
|
@@ -796,8 +821,8 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
|
|
796
821
|
- Supports commands to Aspera server products (on-premise and SaaS)
|
797
822
|
- Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
|
798
823
|
- Supports Commands, Option values and Parameters shortcuts
|
799
|
-
- FASP [Transfer Agents](#agents) can be: local `ascp`, or Connect Client, or any transfer node
|
800
|
-
- Transfer parameters can be altered by modification of [*transfer-spec*](#
|
824
|
+
- FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
|
825
|
+
- Transfer parameters can be altered by modification of [*transfer-spec*](#transfer-specification), this includes requiring multi-session
|
801
826
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
802
827
|
- Supports FaspStream creation (using Node API)
|
803
828
|
- Supports **Watchfolder** creation (using Node API)
|
@@ -828,16 +853,18 @@ Using `ascli` with plugin `server` for command line gives advantages over `ascp`
|
|
828
853
|
|
829
854
|
Moreover all `ascp` options are supported either through transfer spec parameters (listed with `conf ascp spec`) and with the possibility to provide `ascp` arguments directly when the `direct` agent is used (`ascp_args` in `transfer_info`).
|
830
855
|
|
831
|
-
###
|
856
|
+
### Command line parsing, Special Characters
|
832
857
|
|
833
858
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
834
859
|
`ascli` receives its arguments from this shell (through the Operating System).
|
835
860
|
|
836
861
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
837
862
|
|
838
|
-
Linux command line parsing is easy:
|
863
|
+
Linux command line parsing is easy:
|
864
|
+
It is fully documented in the shell's documentation.
|
839
865
|
|
840
866
|
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
867
|
+
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
841
868
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
842
869
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
843
870
|
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
@@ -846,7 +873,10 @@ Special character handling (quotes, spaces, env vars, ...) is handled by the she
|
|
846
873
|
|
847
874
|
#### Shell parsing for Windows
|
848
875
|
|
849
|
-
|
876
|
+
Command line parsing first depends on the shell used.
|
877
|
+
MS Windows command line parsing is not like Unix-like systems simply because Windows does not provide a list of arguments to the executable (Ruby): it provides the whole command line as a single string, but the shell may interpret some special characters.
|
878
|
+
|
879
|
+
So command line parsing is not handled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the executable.
|
850
880
|
Typically, Windows executables use the [microsoft library for this parsing](https://learn.microsoft.com/en-us/cpp/cpp/main-function-command-line-args).
|
851
881
|
|
852
882
|
As far as `ascli` is concerned: the executable is Ruby.
|
@@ -855,7 +885,36 @@ It has its own parsing algorithm, close to a Linux shell parsing.
|
|
855
885
|
Thankfully, `ascli` provides a command to check the value of an argument after parsing: `config echo`.
|
856
886
|
One can also run `ascli` with option `--log-level=debug` to display the command line after parsing.
|
857
887
|
|
858
|
-
|
888
|
+
It is also possible to display arguments received by Ruby using this command:
|
889
|
+
|
890
|
+
```console
|
891
|
+
C:> ruby -e 'puts ARGV' "Hello World" 1 2
|
892
|
+
Hello World
|
893
|
+
1
|
894
|
+
2
|
895
|
+
```
|
896
|
+
|
897
|
+
Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' `CreateProcess` with just the whole command line as a single string.
|
898
|
+
(Unlike Unix-like systems where the command line is split into arguments by the shell.)
|
899
|
+
|
900
|
+
It's up to the program to split arguments:
|
901
|
+
|
902
|
+
- [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
|
903
|
+
- [Understand Quoting and Escaping of Windows Command Line Arguments](https://web.archive.org/web/20190316094059/http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
904
|
+
|
905
|
+
is a Ruby program, so Ruby parses the command line into arguments and provides them to the program.
|
906
|
+
Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
|
907
|
+
(See `w32_cmdvector` in Ruby source [`win32.c`](https://github.com/ruby/ruby/blob/master/win32/win32.c#L1766)) : <!--cspell:disable-line-->
|
908
|
+
|
909
|
+
- Space characters: split arguments (space, tab, newline)
|
910
|
+
- Backslash: `\` escape single special character
|
911
|
+
- Globing characters: `*?[]{}` for file globing
|
912
|
+
- Double quotes: `"`
|
913
|
+
- Single quotes: `'`
|
914
|
+
|
915
|
+
#### Shell parsing for Windows: `cmd.exe`
|
916
|
+
|
917
|
+
The following examples give the same result on Windows using `cmd.exe`:
|
859
918
|
|
860
919
|
- Single quote protects the double quote
|
861
920
|
|
@@ -875,40 +934,49 @@ The following examples give the same result on Windows:
|
|
875
934
|
ascli config echo @json:"{\"url\":\"https://...\"}"
|
876
935
|
```
|
877
936
|
|
878
|
-
More details: on Windows, `cmd.exe` is typically used to start .
|
879
937
|
`cmd.exe` handles some special characters: `^"<>|%&`.
|
880
938
|
Basically it handles I/O redirection (`<>|`), shell variables (`%`), multiple commands (`&`) and handles those special characters from the command line.
|
881
939
|
Eventually, all those special characters are removed from the command line unless escaped with `^` or `"`.
|
882
940
|
`"` are kept and given to the program.
|
883
941
|
|
884
|
-
|
885
|
-
Unlike Unix-like systems where the command line is split into arguments by the shell.
|
942
|
+
#### Shell parsing for Windows: Powershell
|
886
943
|
|
887
|
-
|
944
|
+
For Powershell, it actually depends on the version of it.
|
888
945
|
|
889
|
-
|
890
|
-
- [Understand Quoting and Escaping of Windows Command Line Arguments](https://web.archive.org/web/20190316094059/http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
946
|
+
A difficulty is that Powershell parses the command line for its own use and manages special characters, but then it passes the command line to the program (Ruby) as a single string, possibly without the special characters.
|
891
947
|
|
892
|
-
|
893
|
-
Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
|
894
|
-
(See `w32_cmdvector` in Ruby source [`win32.c`](https://github.com/ruby/ruby/blob/master/win32/win32.c#L1766)) : <!--cspell:disable-line-->
|
948
|
+
Details can be found here:
|
895
949
|
|
896
|
-
-
|
897
|
-
|
898
|
-
-
|
899
|
-
|
900
|
-
|
950
|
+
- [Passing arguments with quotes](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_parsing?view=powershell-7.4#passing-arguments-that-contain-quote-characters)
|
951
|
+
|
952
|
+
- [quoting rules](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7.4)
|
953
|
+
|
954
|
+
The following examples give the same result on Windows using Powershell:
|
955
|
+
|
956
|
+
```console
|
957
|
+
PS C:\> echo $psversiontable.psversion
|
958
|
+
|
959
|
+
Major Minor Build Revision
|
960
|
+
----- ----- ----- --------
|
961
|
+
5 1 19041 4046
|
962
|
+
|
963
|
+
PS C:\> ascli conf echo --% @json:'{"k":"v","x":"y"}'
|
964
|
+
|
965
|
+
PS C:\> ascli conf echo @json:'{"""k""":"""v""","""x""":"""y"""}'
|
966
|
+
```
|
967
|
+
|
968
|
+
> **Note:** The special powershell argument `--%` places powershell in legacy parsing mode.
|
901
969
|
|
902
970
|
#### Extended Values (JSON, Ruby, ...)
|
903
971
|
|
904
|
-
Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
972
|
+
Some of the `ascli` parameters are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
905
973
|
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
906
974
|
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
907
975
|
|
908
976
|
#### Testing Extended Values
|
909
977
|
|
910
978
|
In case of doubt of argument values after parsing, one can test using command `config echo`.
|
911
|
-
`config echo` takes exactly **one** argument which can use the [Extended Value](#extended) syntax.
|
979
|
+
`config echo` takes exactly **one** argument which can use the [Extended Value](#extended-value-syntax) syntax.
|
912
980
|
Unprocessed command line arguments are shown in the error message.
|
913
981
|
|
914
982
|
Example:
|
@@ -927,7 +995,7 @@ ERROR: Argument: unprocessed values: ["2", "3"]
|
|
927
995
|
|
928
996
|
> **Note:** It gets its value after shell command line parsing and `ascli` extended value parsing.
|
929
997
|
|
930
|
-
In the following examples (using a POSIX shell, such as `bash`), several equivalent
|
998
|
+
In the following examples (using a POSIX shell, such as `bash`), several equivalent commands are provided.
|
931
999
|
For all example, most of special character handling is not specific to `ascli`:
|
932
1000
|
It depends on the underlying syntax: shell , JSON, etc...
|
933
1001
|
Depending on the case, a different `format` option is used to display the actual value.
|
@@ -978,7 +1046,8 @@ ascli config echo '"'
|
|
978
1046
|
"
|
979
1047
|
```
|
980
1048
|
|
981
|
-
Double quote in JSON is a little tricky because `"` is special both for the shell and JSON.
|
1049
|
+
Double quote in JSON is a little tricky because `"` is special both for the shell and JSON.
|
1050
|
+
Both shell and JSON syntax allow to protect `"`, but only the shell allows protection using single quote.
|
982
1051
|
|
983
1052
|
```bash
|
984
1053
|
ascli config echo @json:'"\""' --format=text
|
@@ -990,7 +1059,8 @@ ascli config echo @ruby:\'\"\' --format=text
|
|
990
1059
|
"
|
991
1060
|
```
|
992
1061
|
|
993
|
-
Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional `\` is added to protect the `"` for JSON.
|
1062
|
+
Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional `\` is added to protect the `"` for JSON.
|
1063
|
+
But as `\` is also shell special, then it is protected by another `\`.
|
994
1064
|
|
995
1065
|
#### Shell and JSON or Ruby special characters in extended value
|
996
1066
|
|
@@ -1087,7 +1157,7 @@ ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
1087
1157
|
### Commands, Options, Positional Arguments
|
1088
1158
|
|
1089
1159
|
Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
|
1090
|
-
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#parsing).
|
1160
|
+
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#command-line-parsing-special-characters).
|
1091
1161
|
|
1092
1162
|
`ascli` handles three types of command line arguments:
|
1093
1163
|
|
@@ -1108,7 +1178,7 @@ ascli command subcommand --option-name=VAL1 VAL2
|
|
1108
1178
|
If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
|
1109
1179
|
For example `ascli config pre ov` is the same as `ascli config preset overview`.
|
1110
1180
|
|
1111
|
-
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
|
1181
|
+
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended-value-syntax).
|
1112
1182
|
|
1113
1183
|
#### Commands
|
1114
1184
|
|
@@ -1142,9 +1212,9 @@ It could also be designed as an option, but since it is mandatory and typically
|
|
1142
1212
|
The advantages of using a positional argument instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
|
1143
1213
|
|
1144
1214
|
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
|
1145
|
-
Nevertheless, [Extended Values](#extended) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@preset:`).
|
1215
|
+
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@preset:`).
|
1146
1216
|
|
1147
|
-
If a Positional Arguments begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see below).
|
1217
|
+
If a Positional Arguments begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
1148
1218
|
|
1149
1219
|
A few positional arguments are optional, they are located at the end of the command line.
|
1150
1220
|
|
@@ -1184,7 +1254,7 @@ But some are mandatory, so they can be placed in a configuration file, for examp
|
|
1184
1254
|
|
1185
1255
|
The value for **any** options can come from the following locations (in this order, last value evaluated overrides previous value):
|
1186
1256
|
|
1187
|
-
- [Configuration file](#
|
1257
|
+
- [Configuration file](#configuration-file)
|
1188
1258
|
- Environment variable
|
1189
1259
|
- Command line
|
1190
1260
|
|
@@ -1293,9 +1363,9 @@ Examples:
|
|
1293
1363
|
- `DEF,-a,b` : default property list, remove `a` and add `b`
|
1294
1364
|
- `@ruby:'/^server/'` : Display all properties whose name begin with `server`
|
1295
1365
|
|
1296
|
-
####
|
1366
|
+
#### Option: `select`
|
1297
1367
|
|
1298
|
-
Table output can be filtered using option `select`.
|
1368
|
+
Table output (`object_list`) can be filtered using option `select`.
|
1299
1369
|
This parameter is either a `Hash` or `Proc`.
|
1300
1370
|
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
|
1301
1371
|
|
@@ -1336,7 +1406,7 @@ Syntax: `%<field>:<value>`
|
|
1336
1406
|
|
1337
1407
|
> **Note:** The legacy option `id` is deprecated: `--id=1234` (options have a single value and thus do not provide the possibility to identify sub-entities)
|
1338
1408
|
|
1339
|
-
###
|
1409
|
+
### Extended Value Syntax
|
1340
1410
|
|
1341
1411
|
Most options and arguments are specified by a simple string (e.g. username or url).
|
1342
1412
|
Sometime it is convenient to read a value from a file: for example read the PEM value of a private key, or a list of files.
|
@@ -1460,7 +1530,7 @@ EOF
|
|
1460
1530
|
{"key1":"value1","key2":["item1","item2"],"key3":{"key4":"value4","key5":"value5"}}
|
1461
1531
|
```
|
1462
1532
|
|
1463
|
-
###
|
1533
|
+
### Configuration and Persistency Folder
|
1464
1534
|
|
1465
1535
|
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored by default in `[User's home folder]/.aspera/ascli`.
|
1466
1536
|
|
@@ -1503,46 +1573,46 @@ Some temporary files may be needed during runtime.
|
|
1503
1573
|
The temporary folder may be specified with option: `temp_folder`.
|
1504
1574
|
Temporary files are deleted at the end of execution unless option: `clean_temp` is set to `no`.
|
1505
1575
|
|
1506
|
-
###
|
1576
|
+
### Configuration file
|
1507
1577
|
|
1508
1578
|
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
1509
1579
|
Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
|
1510
1580
|
|
1511
1581
|
Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
|
1512
1582
|
|
1513
|
-
All options for `ascli` can be set on command line, or by env vars, or using [
|
1583
|
+
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset)' in the configuration file.
|
1514
1584
|
|
1515
1585
|
A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
|
1516
1586
|
|
1517
1587
|
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or equivalent env var).
|
1518
1588
|
|
1519
|
-
The configuration file is simply a catalog of pre-defined lists of options, called: [
|
1589
|
+
The configuration file is simply a catalog of pre-defined lists of options, called: [Option Preset](#option-preset)'. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset)' (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
|
1520
1590
|
|
1521
|
-
####
|
1591
|
+
#### Option Preset
|
1522
1592
|
|
1523
|
-
A [
|
1593
|
+
A [Option Preset](#option-preset)' is simply a collection of parameters and their associated values in a named section in the configuration file.
|
1524
1594
|
|
1525
|
-
A named [
|
1595
|
+
A named [Option Preset](#option-preset)' can be modified directly using `ascli`, which will update the configuration file :
|
1526
1596
|
|
1527
1597
|
```bash
|
1528
1598
|
ascli config preset set|delete|show|initialize|update <option preset>
|
1529
1599
|
```
|
1530
1600
|
|
1531
|
-
The command `update` allows the easy creation of [
|
1601
|
+
The command `update` allows the easy creation of [Option Preset](#option-preset)' by simply providing the options in their command line format, e.g. :
|
1532
1602
|
|
1533
1603
|
```bash
|
1534
1604
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
|
1535
1605
|
```
|
1536
1606
|
|
1537
|
-
- This creates a [
|
1607
|
+
- This creates a [Option Preset](#option-preset)' `demo_server` with all provided options.
|
1538
1608
|
|
1539
|
-
The command `set` allows setting individual options in a [
|
1609
|
+
The command `set` allows setting individual options in a [Option Preset](#option-preset)'.
|
1540
1610
|
|
1541
1611
|
```bash
|
1542
1612
|
ascli config preset set demo_server password my_password_here
|
1543
1613
|
```
|
1544
1614
|
|
1545
|
-
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended).
|
1615
|
+
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
|
1546
1616
|
|
1547
1617
|
```bash
|
1548
1618
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
|
@@ -1554,7 +1624,7 @@ A full terminal based overview of the configuration can be displayed using:
|
|
1554
1624
|
ascli config preset over
|
1555
1625
|
```
|
1556
1626
|
|
1557
|
-
A list of [
|
1627
|
+
A list of [Option Preset](#option-preset)' can be displayed using:
|
1558
1628
|
|
1559
1629
|
```bash
|
1560
1630
|
ascli config preset list
|
@@ -1577,11 +1647,11 @@ ascli config preset over
|
|
1577
1647
|
ascli config preset list
|
1578
1648
|
```
|
1579
1649
|
|
1580
|
-
####
|
1650
|
+
#### Special Option Preset: `config`
|
1581
1651
|
|
1582
1652
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
1583
1653
|
|
1584
|
-
####
|
1654
|
+
#### Special Option Preset: `default`
|
1585
1655
|
|
1586
1656
|
This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
|
1587
1657
|
|
@@ -1603,24 +1673,24 @@ ascli config preset get default _plugin_name_
|
|
1603
1673
|
"_default_preset_for_plugin_"
|
1604
1674
|
```
|
1605
1675
|
|
1606
|
-
####
|
1676
|
+
#### Plugin: `config`: Configuration
|
1607
1677
|
|
1608
1678
|
Plugin `config` provides general commands for `ascli`:
|
1609
1679
|
|
1610
|
-
- Option
|
1680
|
+
- Option Preset, configuration file operations
|
1611
1681
|
- `wizard`
|
1612
1682
|
- `vault`
|
1613
1683
|
- `ascp`
|
1614
1684
|
|
1615
1685
|
The default preset for `config` is read for any plugin invocation, this allows setting global options, such as `--log-level` or `--interactive`.
|
1616
|
-
When `ascli` starts, it looks for the `default` Option
|
1686
|
+
When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
|
1617
1687
|
If set, it loads the options independently of the plugin used.
|
1618
1688
|
|
1619
1689
|
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global parameters (e.g. `config ascp use`)
|
1620
1690
|
>
|
1621
1691
|
> **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
|
1622
1692
|
|
1623
|
-
Show current default (global) Option
|
1693
|
+
Show current default (global) Option Preset (`config` plugin):
|
1624
1694
|
|
1625
1695
|
```console
|
1626
1696
|
$ ascli config preset get default config
|
@@ -1631,90 +1701,95 @@ global_common_defaults
|
|
1631
1701
|
ascli config preset set GLOBAL version_check_days 0
|
1632
1702
|
```
|
1633
1703
|
|
1634
|
-
If the default global Option
|
1704
|
+
If the default global Option Preset is not set, and you want to use a different name:
|
1635
1705
|
|
1636
1706
|
```bash
|
1637
1707
|
ascli config preset set GLOBAL version_check_days 0
|
1638
1708
|
ascli config preset set default config my_common_defaults
|
1639
1709
|
```
|
1640
1710
|
|
1641
|
-
|
1642
|
-
|
1643
|
-
|
1644
|
-
|
1645
|
-
|
1646
|
-
|
1647
|
-
|
1648
|
-
|
1649
|
-
|
1650
|
-
|
1651
|
-
|
1652
|
-
|
1653
|
-
|
1654
|
-
|
1655
|
-
|
1656
|
-
|
1657
|
-
|
1658
|
-
|
1659
|
-
|
1660
|
-
|
1661
|
-
|
1662
|
-
|
1663
|
-
|
1664
|
-
|
1665
|
-
|
1666
|
-
|
1667
|
-
|
1668
|
-
|
1669
|
-
|
1670
|
-
|
1671
|
-
|
1672
|
-
|
1673
|
-
|
1674
|
-
|
1675
|
-
|
1676
|
-
|
1677
|
-
|
1678
|
-
|
1679
|
-
|
1680
|
-
|
1681
|
-
|
1682
|
-
|
1683
|
-
|
1684
|
-
|
1685
|
-
|
1686
|
-
|
1687
|
-
|
1688
|
-
|
1689
|
-
|
1690
|
-
|
1691
|
-
|
1692
|
-
|
1693
|
-
|
1694
|
-
|
1695
|
-
|
1696
|
-
|
1697
|
-
|
1698
|
-
|
1699
|
-
|
1700
|
-
|
1701
|
-
|
1702
|
-
|
1703
|
-
|
1704
|
-
|
1705
|
-
|
1706
|
-
|
1707
|
-
|
1708
|
-
|
1709
|
-
|
1710
|
-
|
1711
|
-
|
1712
|
-
|
1713
|
-
|
1714
|
-
|
1715
|
-
|
1716
|
-
|
1717
|
-
|
1711
|
+
### Config sample commands
|
1712
|
+
|
1713
|
+
> **Note:** Add `ascli config` in front of the commands:
|
1714
|
+
|
1715
|
+
```bash
|
1716
|
+
ascp connect info 'Aspera Connect for Windows'
|
1717
|
+
ascp connect list
|
1718
|
+
ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
|
1719
|
+
ascp connect version 'Aspera Connect for Windows' list
|
1720
|
+
ascp connect version 'Aspera Connect for Windows' open documentation
|
1721
|
+
ascp errors
|
1722
|
+
ascp info --sdk-folder=sdk_test_dir
|
1723
|
+
ascp install
|
1724
|
+
ascp install --sdk-folder=sdk_test_dir
|
1725
|
+
ascp products list
|
1726
|
+
ascp products use 'IBM Aspera Connect'
|
1727
|
+
ascp show
|
1728
|
+
ascp spec
|
1729
|
+
ascp use /usr/bin/ascp
|
1730
|
+
check_update
|
1731
|
+
coffee
|
1732
|
+
coffee --ui=text
|
1733
|
+
coffee --ui=text --query=@json:'{"text":"true"}'
|
1734
|
+
detect https://faspex4.example.com/path
|
1735
|
+
detect https://faspex5.example.com/path
|
1736
|
+
detect https://node.example.com/path
|
1737
|
+
detect https://shares.example.com/path shares
|
1738
|
+
detect my_org aoc
|
1739
|
+
doc
|
1740
|
+
doc transfer-parameters
|
1741
|
+
echo -- --special-string
|
1742
|
+
echo @base64:SGVsbG8gV29ybGQK
|
1743
|
+
echo @csvt:@stdin:
|
1744
|
+
echo @env:USER
|
1745
|
+
echo @lines:@stdin:
|
1746
|
+
echo @list:,1,2,3
|
1747
|
+
echo @secret:
|
1748
|
+
echo @uri:/etc/hosts
|
1749
|
+
echo @uri:file:/etc/hosts
|
1750
|
+
echo @uri:http://ifconfig.me
|
1751
|
+
echo @uri:https://ifconfig.me
|
1752
|
+
echo @vault:my_preset.password
|
1753
|
+
echo @zlib:@stdin:
|
1754
|
+
echo hello
|
1755
|
+
email_test --notify-to=my_email_external
|
1756
|
+
flush_tokens
|
1757
|
+
folder
|
1758
|
+
gem name
|
1759
|
+
gem path
|
1760
|
+
gem version
|
1761
|
+
genkey my_key
|
1762
|
+
genkey my_key 4096
|
1763
|
+
initdemo
|
1764
|
+
open
|
1765
|
+
plugin create my_command
|
1766
|
+
plugin list
|
1767
|
+
preset delete conf_name
|
1768
|
+
preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
|
1769
|
+
preset list
|
1770
|
+
preset overview
|
1771
|
+
preset set conf_name param value
|
1772
|
+
preset set default shares conf_name
|
1773
|
+
preset show conf_name
|
1774
|
+
preset unset conf_name param
|
1775
|
+
preset update conf_name --p1=v1 --p2=v2
|
1776
|
+
proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
|
1777
|
+
pubkey @file:my_key
|
1778
|
+
remote_certificate chain https://node.example.com/path
|
1779
|
+
remote_certificate name https://node.example.com/path
|
1780
|
+
remote_certificate only https://node.example.com/path
|
1781
|
+
vault create my_label @json:'{"password":"my_password_here","description":"my secret"}'
|
1782
|
+
vault delete my_label
|
1783
|
+
vault list
|
1784
|
+
vault show my_label
|
1785
|
+
wizard https://console.example.com/path console
|
1786
|
+
wizard https://faspex4.example.com/path faspex --username=test --password=test
|
1787
|
+
wizard https://faspex5.example.com/path faspex5 --key-path=my_private_key
|
1788
|
+
wizard https://node.example.com/path node --username=test --password=test
|
1789
|
+
wizard https://orch.example.com/path orchestrator --username=test --password=test
|
1790
|
+
wizard https://shares.example.com/path shares --username=test --password=test
|
1791
|
+
wizard my_org aoc --key-path=my_private_key --username=my_user_email
|
1792
|
+
wizard my_org aoc --key-path=my_private_key --username=my_user_email --use-generic-client=yes
|
1718
1793
|
```
|
1719
1794
|
|
1720
1795
|
#### Format of file
|
@@ -1738,27 +1813,27 @@ demo_server:
|
|
1738
1813
|
We can see here:
|
1739
1814
|
|
1740
1815
|
- The configuration was created with `ascli` version 0.3.7
|
1741
|
-
- The default [
|
1742
|
-
- The [
|
1743
|
-
- The default [
|
1816
|
+
- The default [Option Preset](#option-preset)' to load for `server` plugin is : `demo_server`
|
1817
|
+
- The [Option Preset](#option-preset)' `demo_server` defines some parameters: the URL and credentials
|
1818
|
+
- The default [Option Preset](#option-preset)' to load in any case is : `cli_default`
|
1744
1819
|
|
1745
|
-
Two [
|
1820
|
+
Two [Option Preset](#option-preset)' are reserved:
|
1746
1821
|
|
1747
1822
|
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
1748
1823
|
It is used to check compatibility.
|
1749
|
-
- `default` is reserved to define the default [
|
1824
|
+
- `default` is reserved to define the default [Option Preset](#option-preset)' name used for known plugins.
|
1750
1825
|
|
1751
|
-
The user may create as many [
|
1826
|
+
The user may create as many [Option Preset](#option-preset)' as needed. For instance, a particular [Option Preset](#option-preset)' can be created for a particular application instance and contain URL and credentials.
|
1752
1827
|
|
1753
|
-
Values in the configuration also follow the [Extended Value Syntax](#extended).
|
1828
|
+
Values in the configuration also follow the [Extended Value Syntax](#extended-value-syntax).
|
1754
1829
|
|
1755
|
-
> **Note:** If the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
1830
|
+
> **Note:** If the user wants to use the [Extended Value Syntax](#extended-value-syntax) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
1756
1831
|
|
1757
1832
|
```bash
|
1758
1833
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
1759
1834
|
```
|
1760
1835
|
|
1761
|
-
This creates the [
|
1836
|
+
This creates the [Option Preset](#option-preset)':
|
1762
1837
|
|
1763
1838
|
```yaml
|
1764
1839
|
my_aoc_org:
|
@@ -1774,20 +1849,20 @@ Some options are global, some options are available only for some plugins. (the
|
|
1774
1849
|
Options are loaded using this algorithm:
|
1775
1850
|
|
1776
1851
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
1777
|
-
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [
|
1778
|
-
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [
|
1852
|
+
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset)' for it, and loads it.
|
1853
|
+
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset)' specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
|
1779
1854
|
- Environment variables are evaluated
|
1780
1855
|
- Command line options are evaluated
|
1781
1856
|
|
1782
1857
|
Parameters are evaluated in the order of command line.
|
1783
1858
|
|
1784
|
-
To avoid loading the default [
|
1859
|
+
To avoid loading the default [Option Preset](#option-preset)' for a plugin, use: `-N`
|
1785
1860
|
|
1786
1861
|
On command line, words in parameter names are separated by a dash (`-`).
|
1787
1862
|
In configuration file, separator is an underscore.
|
1788
1863
|
E.g. `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
|
1789
1864
|
|
1790
|
-
The main plugin name is `config`, so it is possible to define a default [
|
1865
|
+
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset)' for the main plugin with:
|
1791
1866
|
|
1792
1867
|
```bash
|
1793
1868
|
ascli config preset set cli_default interactive no
|
@@ -1797,7 +1872,7 @@ ascli config preset set cli_default interactive no
|
|
1797
1872
|
ascli config preset set default config cli_default
|
1798
1873
|
```
|
1799
1874
|
|
1800
|
-
A [
|
1875
|
+
A [Option Preset](#option-preset)' value can be removed with `unset`:
|
1801
1876
|
|
1802
1877
|
```bash
|
1803
1878
|
ascli config preset unset cli_default interactive
|
@@ -1817,7 +1892,7 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
|
|
1817
1892
|
|
1818
1893
|
#### Wizard
|
1819
1894
|
|
1820
|
-
The wizard is a command that asks the user for information and creates a [
|
1895
|
+
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset)' with the provided information.
|
1821
1896
|
|
1822
1897
|
It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
|
1823
1898
|
|
@@ -1839,7 +1914,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my
|
|
1839
1914
|
|
1840
1915
|
This can also be provisioned in a configuration file:
|
1841
1916
|
|
1842
|
-
- Build [
|
1917
|
+
- Build [Option Preset](#option-preset)'
|
1843
1918
|
|
1844
1919
|
```bash
|
1845
1920
|
ascli config preset set shares06 url https://10.25.0.6
|
@@ -1859,7 +1934,7 @@ or
|
|
1859
1934
|
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=my_password_here
|
1860
1935
|
```
|
1861
1936
|
|
1862
|
-
- Define this [
|
1937
|
+
- Define this [Option Preset](#option-preset)' as the default [Option Preset](#option-preset)' for the specified plugin (`shares`)
|
1863
1938
|
|
1864
1939
|
```bash
|
1865
1940
|
ascli config preset set default shares shares06
|
@@ -1877,7 +1952,7 @@ ascli config preset overview
|
|
1877
1952
|
ascli shares repo browse /
|
1878
1953
|
```
|
1879
1954
|
|
1880
|
-
###
|
1955
|
+
### Secret Vault
|
1881
1956
|
|
1882
1957
|
Secrets (e.g. passwords) are usually command options.
|
1883
1958
|
They can be provided on command line, env vars, files etc.
|
@@ -1956,7 +2031,7 @@ Then secrets can be manipulated using commands:
|
|
1956
2031
|
ascli config vault create mylabel @json:'{"password":"my_password_here","description":"for this account"}'
|
1957
2032
|
```
|
1958
2033
|
|
1959
|
-
####
|
2034
|
+
#### Configuration Finder
|
1960
2035
|
|
1961
2036
|
When a secret is needed by a sub command, the command can search for existing configurations in the configuration file.
|
1962
2037
|
|
@@ -1964,7 +2039,7 @@ The lookup is done by comparing the service URL and username (or access key).
|
|
1964
2039
|
|
1965
2040
|
#### Securing passwords and secrets
|
1966
2041
|
|
1967
|
-
A passwords can be saved in clear in a [
|
2042
|
+
A passwords can be saved in clear in a [Option Preset](#option-preset)' together with other account information (URL, username, etc...).
|
1968
2043
|
Example:
|
1969
2044
|
|
1970
2045
|
```bash
|
@@ -1983,7 +2058,7 @@ ascli config vault create myconf @json:'{"password":"my_password_here"}'
|
|
1983
2058
|
|
1984
2059
|
> **Note:** Use `@val:` in front of `@vault:` so that the extended value is not evaluated.
|
1985
2060
|
|
1986
|
-
###
|
2061
|
+
### Private Key
|
1987
2062
|
|
1988
2063
|
Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5, ...).
|
1989
2064
|
It consists in using a pair of keys: the private key and its associated public key.
|
@@ -2063,7 +2138,9 @@ openssl rsa -des3 -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.with_des
|
|
2063
2138
|
mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
|
2064
2139
|
```
|
2065
2140
|
|
2066
|
-
###
|
2141
|
+
### SSL CA certificate bundle
|
2142
|
+
|
2143
|
+
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders, by default it uses Ruby's default certificate store.
|
2067
2144
|
|
2068
2145
|
To display trusted certificate store locations:
|
2069
2146
|
|
@@ -2071,40 +2148,50 @@ To display trusted certificate store locations:
|
|
2071
2148
|
ascli --show-config --fields=cert_stores
|
2072
2149
|
```
|
2073
2150
|
|
2074
|
-
|
2075
|
-
If you use this option, then default locations are
|
2076
|
-
|
2151
|
+
Use option `cert_stores` to modify the locations of certificate stores (files or folders).
|
2152
|
+
If you use this option, then default locations are not used.
|
2153
|
+
Default locations can be added using special value `DEF`.
|
2154
|
+
The value can be either an `Array` or `String` (path).
|
2155
|
+
Successive options add paths incrementally.
|
2156
|
+
All files of a folders are added.
|
2157
|
+
|
2158
|
+
`ascli` uses the Ruby `openssl` gem.
|
2159
|
+
By default it uses the system's `openssl` library, but JRuby uses its own implementation.
|
2160
|
+
|
2161
|
+
For example, on Linux to use the system's certificate store:
|
2162
|
+
|
2163
|
+
```bash
|
2164
|
+
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
2165
|
+
```
|
2077
2166
|
|
2078
|
-
`ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
|
2079
2167
|
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
2080
2168
|
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
2081
2169
|
|
2082
|
-
|
2170
|
+
One can display those default values:
|
2083
2171
|
|
2084
2172
|
```bash
|
2085
2173
|
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
2086
2174
|
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
2087
2175
|
```
|
2088
2176
|
|
2089
|
-
`ascp` also needs to validate certificates when using **WSS
|
2177
|
+
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of SSH).
|
2090
2178
|
|
2091
|
-
|
2179
|
+
By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
|
2180
|
+
Original `ascp`'s hardcoded locations can be found using:
|
2092
2181
|
|
2093
2182
|
```bash
|
2094
2183
|
ascli config ascp info --fields=openssldir
|
2095
2184
|
```
|
2096
2185
|
|
2097
|
-
|
2098
|
-
|
2099
|
-
|
2100
|
-
strings $(ascli config ascp info --fields=ascp)|grep -w OPENSSLDIR
|
2101
|
-
```
|
2186
|
+
E.g. on macOS: `/Library/Aspera/ssl`.
|
2187
|
+
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
2188
|
+
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
2102
2189
|
|
2103
2190
|
To update trusted root certificates for `ascli`:
|
2104
2191
|
Display the trusted certificate store locations used by `ascli`.
|
2105
2192
|
Typically done by updating the system's root certificate store.
|
2106
2193
|
|
2107
|
-
An up-to-date version of the certificate bundle can be retrieved with:
|
2194
|
+
An up-to-date version of the certificate bundle can also be retrieved with:
|
2108
2195
|
|
2109
2196
|
```bash
|
2110
2197
|
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
@@ -2113,18 +2200,18 @@ ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
|
2113
2200
|
To download that certificate store:
|
2114
2201
|
|
2115
2202
|
```bash
|
2116
|
-
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
2203
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
|
2117
2204
|
```
|
2118
2205
|
|
2119
|
-
Then, use this store by setting the
|
2206
|
+
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
2120
2207
|
|
2121
|
-
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate to a file:
|
2208
|
+
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
2122
2209
|
|
2123
2210
|
```bash
|
2124
|
-
ascli config remote_certificate https://localhost:9092
|
2211
|
+
ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
|
2125
2212
|
```
|
2126
2213
|
|
2127
|
-
> **Note:**
|
2214
|
+
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
2128
2215
|
|
2129
2216
|
Then, use this file as certificate store (e.g. here, Node API):
|
2130
2217
|
|
@@ -2146,7 +2233,7 @@ The following options can be specified in the option `query`:
|
|
2146
2233
|
| double | Display double text resolution (half characters) (Bool) |
|
2147
2234
|
| font_ratio | Font height/width ratio in terminal (Float) |
|
2148
2235
|
|
2149
|
-
###
|
2236
|
+
### Graphical Interactions: Browser and Text Editor
|
2150
2237
|
|
2151
2238
|
Some actions may require the use of a graphical tool:
|
2152
2239
|
|
@@ -2174,7 +2261,7 @@ Available loggers: `stdout`, `stderr`, `syslog`.
|
|
2174
2261
|
|
2175
2262
|
Available levels: `debug`, `info`, `warn`, `error`.
|
2176
2263
|
|
2177
|
-
> **Note:** When using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` options and `ascp_args`, see [`direct`](#
|
2264
|
+
> **Note:** When using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` options and `ascp_args`, see [`direct`](#agent-direct).
|
2178
2265
|
|
2179
2266
|
Examples:
|
2180
2267
|
|
@@ -2201,7 +2288,7 @@ To get traces of execution, with dump of API calls, use argument : `--log-level=
|
|
2201
2288
|
To display HTTP/S traffic set option `log_level` to `trace2`: `--log-level=trace2`.
|
2202
2289
|
It will display the exact content of HTTP requests and responses.
|
2203
2290
|
|
2204
|
-
###
|
2291
|
+
### HTTP socket parameters
|
2205
2292
|
|
2206
2293
|
To ignore SSL certificate for **any** address/port, use option: `insecure`, i.e. `--insecure=yes`.
|
2207
2294
|
To ignore SSL certificate for a list of specific address/port, use option `ignore_certificate`, set to an `Array` of URL for which certificate will be ignored (only the address and port are matched), e.g. `--ignore-certificate=@list:,https://127.0.0.1:9092`
|
@@ -2306,13 +2393,25 @@ ascli --proxy-credentials=@list::__username_here__:__password_here__ ...
|
|
2306
2393
|
|
2307
2394
|
#### Proxy for Legacy Aspera HTTP/S Fallback
|
2308
2395
|
|
2309
|
-
Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallback, use `ascp` native option `-x` and `ascp_args`: `--transfer-info=@json:'{"ascp_args":["-x","url_here"]}'`.
|
2396
|
+
Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallback, use `ascp` native option `-x` and `ascp_args`: `--transfer-info=@json:'{"ascp_args":["-x","url_here"]}'`.
|
2310
2397
|
|
2311
2398
|
#### FASP proxy (forward) for transfers
|
2312
2399
|
|
2313
|
-
To specify a FASP proxy (forward), set the [*transfer-spec*](#
|
2400
|
+
To specify a FASP proxy (forward), set the [*transfer-spec*](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
|
2401
|
+
|
2402
|
+
For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
|
2403
|
+
|
2404
|
+
```bash
|
2405
|
+
--ts=@json:'{"proxy":"dnat://proxy.example.org:9091"}'
|
2406
|
+
```
|
2407
|
+
|
2408
|
+
Or, alternatively, (prefer transfer spec like above, generally):
|
2409
|
+
|
2410
|
+
```bash
|
2411
|
+
--transfer-info=@json:'{"ascp_args":["--proxy","dnat://proxy.example.org:9091"]}'
|
2412
|
+
```
|
2314
2413
|
|
2315
|
-
###
|
2414
|
+
### FASP configuration
|
2316
2415
|
|
2317
2416
|
The `config` plugin also allows specification for the use of a local FASP **client**.
|
2318
2417
|
It provides the following commands for `ascp` subcommand:
|
@@ -2344,7 +2443,7 @@ ascli config ascp info
|
|
2344
2443
|
...
|
2345
2444
|
```
|
2346
2445
|
|
2347
|
-
#### Selection of `ascp` location for [`direct`](#
|
2446
|
+
#### Selection of `ascp` location for [`direct`](#agent-direct) agent
|
2348
2447
|
|
2349
2448
|
By default, `ascli` uses any found local product with `ascp`, including Transfer SDK.
|
2350
2449
|
|
@@ -2376,7 +2475,7 @@ Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\a
|
|
2376
2475
|
Saved to default global preset global_common_defaults
|
2377
2476
|
```
|
2378
2477
|
|
2379
|
-
If the path has spaces, read section: [Shell and Command line parsing](#parsing).
|
2478
|
+
If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
|
2380
2479
|
|
2381
2480
|
#### List locally installed Aspera Transfer products
|
2382
2481
|
|
@@ -2397,7 +2496,7 @@ ascli config ascp products list
|
|
2397
2496
|
+---------------------------------------+----------------------------------------+
|
2398
2497
|
```
|
2399
2498
|
|
2400
|
-
#### Selection of local client for `ascp` for [`direct`](#
|
2499
|
+
#### Selection of local client for `ascp` for [`direct`](#agent-direct) agent
|
2401
2500
|
|
2402
2501
|
If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
|
2403
2502
|
|
@@ -2453,58 +2552,64 @@ Time: 00:00:02 ============================================= 100% 27766 KB/sec T
|
|
2453
2552
|
Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
2454
2553
|
```
|
2455
2554
|
|
2456
|
-
###
|
2555
|
+
### Transfer Clients: Agents
|
2457
2556
|
|
2458
2557
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
2459
2558
|
|
2460
|
-
When a transfer needs to be started, a [*transfer-spec*](#
|
2461
|
-
This [*transfer-spec*](#
|
2559
|
+
When a transfer needs to be started, a [*transfer-spec*](#transfer-specification) has been internally prepared.
|
2560
|
+
This [*transfer-spec*](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
|
2462
2561
|
|
2463
2562
|
There are currently 3 agents, set with option `transfer`:
|
2464
2563
|
|
2465
|
-
- [`direct`](#
|
2466
|
-
- [`
|
2467
|
-
- [`
|
2468
|
-
- [`
|
2469
|
-
- [`
|
2564
|
+
- [`direct`](#agent-direct) : execution of `ascp`
|
2565
|
+
- [`trsdk`](#agent-transfer-sdk) : use of Aspera Transfer SDK (local)
|
2566
|
+
- [`connect`](#agent-connect-client) : use Connect Client (local)
|
2567
|
+
- [`alpha`](#agent-desktop-client) : use the new Desktop Client (local)
|
2568
|
+
- [`node`](#agent-node-api) : use an Aspera Transfer Node (**remote**).
|
2569
|
+
- [`httpgw`](#agent-http-gateway) : use an Aspera HTTP Gateway (**remote**)
|
2470
2570
|
|
2471
2571
|
> **Note:** All transfer operations are seen from the point of view of the agent.
|
2472
2572
|
For example, a node agent executing an **upload**, or **package send** operation
|
2473
2573
|
will effectively push files to the related server from the agent node.
|
2474
2574
|
|
2475
|
-
`ascli` standardizes on the use of a [*transfer-spec*](#
|
2575
|
+
`ascli` standardizes on the use of a [*transfer-spec*](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
2476
2576
|
|
2477
2577
|
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2478
2578
|
|
2479
|
-
####
|
2579
|
+
#### Agent: Direct
|
2480
2580
|
|
2481
2581
|
The `direct` agent directly executes a local `ascp`.
|
2482
2582
|
This is the default agent for `ascli` (option `--transfer=direct`).
|
2483
2583
|
`ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
|
2484
|
-
Refer to section [FASP](#
|
2584
|
+
Refer to section [FASP](#fasp-configuration).
|
2485
2585
|
|
2486
2586
|
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
|
2487
2587
|
|
2488
2588
|
| Name | Type | Description |
|
2489
2589
|
|------------------------|-------|-------------|
|
2490
2590
|
| `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
|
2491
|
-
| `ascp_args` | Array | Array of strings with native `ascp` arguments
|
2591
|
+
| `ascp_args` | Array | Array of strings with native `ascp` arguments.<br/>Default: [] |
|
2492
2592
|
| `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
|
2493
2593
|
| `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
|
2494
2594
|
| `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
|
2495
|
-
| `
|
2496
|
-
| `resume
|
2497
|
-
| `resume.
|
2498
|
-
| `resume.
|
2499
|
-
| `resume.
|
2595
|
+
| `trusted_certs` | Array | List of repositories for trusted certificates. |
|
2596
|
+
| `resume` | Hash | Resume parameters. See below |
|
2597
|
+
| `resume.iter_max` | int | Max number of retry on error<br/>Default: 7 |
|
2598
|
+
| `resume.sleep_initial` | int | First Sleep before retry<br/>Default: 2 |
|
2599
|
+
| `resume.sleep_factor` | int | Multiplier of sleep period between attempts<br/>Default: 2 |
|
2600
|
+
| `resume.sleep_max` | int | Default: 60 |
|
2500
2601
|
|
2501
2602
|
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
|
2502
|
-
Sleep between iterations is:
|
2603
|
+
Sleep between iterations is given by the following formula where `iter_index` is the current iteration index, starting at 0:
|
2503
2604
|
|
2504
2605
|
```bash
|
2505
|
-
max( sleep_max , sleep_initial * sleep_factor ^
|
2606
|
+
max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
|
2506
2607
|
```
|
2507
2608
|
|
2609
|
+
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
2610
|
+
In order to use a custom certificate store, use the `trusted_certs` parameter.
|
2611
|
+
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
2612
|
+
|
2508
2613
|
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
2509
2614
|
The list of known protocol errors and retry level can be listed:
|
2510
2615
|
|
@@ -2519,9 +2624,6 @@ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":20}}'
|
|
2519
2624
|
ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
|
2520
2625
|
```
|
2521
2626
|
|
2522
|
-
> **Note:** The `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
|
2523
|
-
But it is preferred to use the option `transfer_info` with parameter `ascp_args`.
|
2524
|
-
|
2525
2627
|
This can be useful to activate logging using option `-L` of `ascp`.
|
2526
2628
|
For example, to activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`):
|
2527
2629
|
|
@@ -2533,15 +2635,9 @@ This is useful to debug if a transfer fails.
|
|
2533
2635
|
|
2534
2636
|
To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--transfer-info=@json:'{"ascp_args":["-L","/path/to/folder"]}'`.
|
2535
2637
|
|
2536
|
-
> **Note:** When transfer agent [`direct`](#
|
2638
|
+
> **Note:** When transfer agent [`direct`](#agent-direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided using `transfer_info` parameter `ascp_args`).
|
2537
2639
|
|
2538
|
-
In addition to standard methods described in section [File List](#
|
2539
|
-
|
2540
|
-
- Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_file_list`
|
2541
|
-
|
2542
|
-
```bash
|
2543
|
-
--sources=@ts --ts=@json:'{"EX_file_list":"file_list.txt"}'
|
2544
|
-
```
|
2640
|
+
In addition to standard methods described in section [File List](#list-of-files-for-transfers), it is possible to specify the list of file using those additional methods:
|
2545
2641
|
|
2546
2642
|
- Using option `transfer_info` parameter `ascp_args`
|
2547
2643
|
|
@@ -2553,7 +2649,7 @@ In addition to standard methods described in section [File List](#file_list), it
|
|
2553
2649
|
>
|
2554
2650
|
> **Note:** Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
|
2555
2651
|
>
|
2556
|
-
> **Note:** Those methods have limitations: they apply **only** to the [`direct`](#
|
2652
|
+
> **Note:** Those methods have limitations: they apply **only** to the [`direct`](#agent-direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
|
2557
2653
|
|
2558
2654
|
This agent supports a local configuration file: `aspera.conf` where Virtual links can be configured:
|
2559
2655
|
|
@@ -2616,13 +2712,20 @@ It is also possible to set a schedule with different time and days, for example
|
|
2616
2712
|
start=08 end=19 days=mon,tue,wed,thu capacity=900000;1000000
|
2617
2713
|
```
|
2618
2714
|
|
2619
|
-
####
|
2715
|
+
#### Agent: Connect Client
|
2716
|
+
|
2717
|
+
By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed **IBM Aspera Connect Client**.
|
2718
|
+
There are no option for `transfer_info`.
|
2719
|
+
|
2720
|
+
#### Agent: Desktop Client
|
2620
2721
|
|
2621
|
-
By specifying option: `--transfer=
|
2722
|
+
By specifying option: `--transfer=alpha`, `ascli` will start transfers using the locally installed **IBM Aspera Desktop Client**.
|
2723
|
+
There are no option for `transfer_info`.
|
2622
2724
|
|
2623
|
-
####
|
2725
|
+
#### Agent: Node API
|
2624
2726
|
|
2625
2727
|
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
2728
|
+
This is especially useful for direct node-to-node transfers.
|
2626
2729
|
Parameters provided in option `transfer_info` are:
|
2627
2730
|
|
2628
2731
|
| Name | Type | Description |
|
@@ -2632,7 +2735,7 @@ Parameters provided in option `transfer_info` are:
|
|
2632
2735
|
| password | string | Password, secret or bearer token</br>Mandatory |
|
2633
2736
|
| root_id | string | Root file id</br>Mandatory only for bearer token |
|
2634
2737
|
|
2635
|
-
Like any other option, `transfer_info` can get its value from a pre-configured [
|
2738
|
+
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset)' :
|
2636
2739
|
|
2637
2740
|
```bash
|
2638
2741
|
--transfer-info=@preset:_name_here_
|
@@ -2649,7 +2752,7 @@ If `transfer_info` is not specified and a default node has been configured (name
|
|
2649
2752
|
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node.
|
2650
2753
|
It can be either the access key's root file id, or any authorized file id underneath it.
|
2651
2754
|
|
2652
|
-
####
|
2755
|
+
#### Agent: HTTP Gateway
|
2653
2756
|
|
2654
2757
|
If it possible to send using a HTTP gateway, in case use of FASP is not allowed.
|
2655
2758
|
|
@@ -2670,7 +2773,7 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
|
|
2670
2773
|
|
2671
2774
|
> **Note:** The gateway only supports transfers authorized with a token.
|
2672
2775
|
|
2673
|
-
####
|
2776
|
+
#### Agent: Transfer SDK
|
2674
2777
|
|
2675
2778
|
Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
|
2676
2779
|
Set option `transfer` to `trsdk`.
|
@@ -2706,10 +2809,10 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
2706
2809
|
|
2707
2810
|
<!-- spellchecker: enable -->
|
2708
2811
|
|
2709
|
-
###
|
2812
|
+
### Transfer Specification
|
2710
2813
|
|
2711
2814
|
Some commands lead to file transfer (upload/download).
|
2712
|
-
All parameters necessary for this transfer are described in a [*transfer-spec*](#
|
2815
|
+
All parameters necessary for this transfer are described in a [*transfer-spec*](#transfer-specification) (Transfer Specification), such as:
|
2713
2816
|
|
2714
2817
|
- Server address
|
2715
2818
|
- Transfer user name
|
@@ -2717,29 +2820,29 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
|
|
2717
2820
|
- File list
|
2718
2821
|
- Etc...
|
2719
2822
|
|
2720
|
-
`ascli` builds the [*transfer-spec*](#
|
2823
|
+
`ascli` builds the [*transfer-spec*](#transfer-specification) internally as a `Hash`.
|
2721
2824
|
It is not necessary to provide additional parameters on the command line for this transfer.
|
2722
2825
|
|
2723
|
-
It is possible to modify or add any of the supported [*transfer-spec*](#
|
2724
|
-
The `ts` option accepts a [`Hash` Extended Value](#extended) containing one or several [*transfer-spec*](#
|
2826
|
+
It is possible to modify or add any of the supported [*transfer-spec*](#transfer-specification) parameter using the `ts` option.
|
2827
|
+
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [*transfer-spec*](#transfer-specification) parameters.
|
2725
2828
|
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
2726
2829
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
2727
2830
|
|
2728
2831
|
> **Note:** Default transfer spec values can be displayed with command: `config ascp info --flat-hash=no` under field `ts`.
|
2729
2832
|
|
2730
|
-
It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#
|
2833
|
+
It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#agent-direct) using `transfer_info` option parameter: `ascp_args`.
|
2731
2834
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
2732
2835
|
This is especially useful for `ascp` command line parameters not supported in the transfer spec.
|
2733
2836
|
|
2734
|
-
The use of a [*transfer-spec*](#
|
2837
|
+
The use of a [*transfer-spec*](#transfer-specification) instead of `ascp` parameters has the advantage of:
|
2735
2838
|
|
2736
|
-
- Common to all [Transfer Agent](#agents)
|
2839
|
+
- Common to all [Transfer Agent](#transfer-clients-agents)
|
2737
2840
|
- Not dependent on command line limitations (special characters...)
|
2738
2841
|
|
2739
|
-
###
|
2842
|
+
### Transfer Parameters
|
2740
2843
|
|
2741
|
-
All standard [*transfer-spec*](#
|
2742
|
-
[*transfer-spec*](#
|
2844
|
+
All standard [*transfer-spec*](#transfer-specification) parameters can be specified.
|
2845
|
+
[*transfer-spec*](#transfer-specification) can also be saved/overridden in the configuration file.
|
2743
2846
|
|
2744
2847
|
References:
|
2745
2848
|
|
@@ -2764,48 +2867,46 @@ Columns:
|
|
2764
2867
|
|
2765
2868
|
`ascp` argument or environment variable is provided in description.
|
2766
2869
|
|
2767
|
-
Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
|
2768
|
-
|
2769
2870
|
| Field | Type | D | N | C | T | H | Description |
|
2770
2871
|
| ----- | ---- | - | - | - | - | - | ----------- |
|
2771
2872
|
| apply_local_docroot | bool | Y | | | | | Apply local docroot to source paths.<br/>(--apply-local-docroot) |
|
2772
|
-
| authentication | string | | | Y | | | value=token for SSH bypass keys, else password asked if not provided.<br/>(
|
2873
|
+
| authentication | string | | | Y | | | value=token for SSH bypass keys, else password asked if not provided.<br/>(<ignored>) |
|
2773
2874
|
| cipher | string | Y | Y | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
2774
|
-
| cipher_allowed | string | Y | Y | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none".<br/>(
|
2775
|
-
| compression | int | | | | | | ascp4 only, 0 / 1?<br/>(
|
2875
|
+
| cipher_allowed | string | Y | Y | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none".<br/>(<ignored>) |
|
2876
|
+
| compression | int | | | | | | ascp4 only, 0 / 1?<br/>(<ignored>) |
|
2776
2877
|
| content_protection | string | Y | Y | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
|
2777
2878
|
| content_protection_password | string | Y | Y | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
|
2778
2879
|
| cookie | string | Y | Y | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
|
2779
2880
|
| create_dir | bool | Y | Y | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
|
2780
2881
|
| delete_before_transfer | bool | Y | Y | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
|
2781
2882
|
| delete_source | bool | Y | Y | | | | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
|
2782
|
-
| destination_root | string | Y | Y | Y | Y | Y | Destination root directory.<br/>(
|
2783
|
-
| destination_root_id | string | | | | | | The file ID of the destination root directory.<br/>Required when using Bearer token auth for the destination node.<br/>(
|
2883
|
+
| destination_root | string | Y | Y | Y | Y | Y | Destination root directory.<br/>(<special>) |
|
2884
|
+
| destination_root_id | string | | | | | | The file ID of the destination root directory.<br/>Required when using Bearer token auth for the destination node.<br/>(<ignored>) |
|
2784
2885
|
| dgram_size | int | Y | Y | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
|
2785
2886
|
| direction | string | Y | Y | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
|
2786
2887
|
| exclude_newer_than | int | Y | | | | | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
|
2787
2888
|
| exclude_older_than | int | Y | | | | | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
|
2788
2889
|
| fasp_port | int | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
|
2789
|
-
| fasp_url | string | | | | | | Only used in Faspex.<br/>(
|
2790
|
-
| file_checksum | string | Y | Y | | | | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(
|
2890
|
+
| fasp_url | string | | | | | | Only used in Faspex.<br/>(<ignored>) |
|
2891
|
+
| file_checksum | string | Y | Y | | | | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(<ignored>) |
|
2791
2892
|
| http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
|
2792
2893
|
| http_fallback_port | int | Y | | | | | Specifies http port when no cipher is used<br/>(-t {int}) |
|
2793
2894
|
| https_fallback_port | int | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
|
2794
2895
|
| keepalive | bool | Y | | | | | The session is running in persistent session mode.<br/>(--keepalive) |
|
2795
|
-
| lock_min_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(
|
2796
|
-
| lock_min_rate_kbps | bool | | | Y | | | If true, lock the minimum transfer rate to the value set for min_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(
|
2797
|
-
| lock_rate_policy | bool | | | Y | | | If true, lock the rate policy to the default value.<br/>(
|
2798
|
-
| lock_target_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(
|
2799
|
-
| lock_target_rate_kbps | bool | Y | Y | Y | Y | Y | If true, lock the target transfer rate to the default value set for target_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(
|
2800
|
-
| min_rate_cap_kbps | int | Y | Y | Y | Y | Y | The highest minimum rate that an incoming transfer can request, in kilobits per second.<br/>Client minimum rate requests that exceed the minimum rate cap are ignored.<br/>The default value of unlimited applies no cap to the minimum rate. (Default: 0)<br/>(
|
2896
|
+
| lock_min_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2897
|
+
| lock_min_rate_kbps | bool | | | Y | | | If true, lock the minimum transfer rate to the value set for min_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
|
2898
|
+
| lock_rate_policy | bool | | | Y | | | If true, lock the rate policy to the default value.<br/>(<ignored>) |
|
2899
|
+
| lock_target_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2900
|
+
| lock_target_rate_kbps | bool | Y | Y | Y | Y | Y | If true, lock the target transfer rate to the default value set for target_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
|
2901
|
+
| min_rate_cap_kbps | int | Y | Y | Y | Y | Y | The highest minimum rate that an incoming transfer can request, in kilobits per second.<br/>Client minimum rate requests that exceed the minimum rate cap are ignored.<br/>The default value of unlimited applies no cap to the minimum rate. (Default: 0)<br/>(<ignored>) |
|
2801
2902
|
| min_rate_kbps | int | Y | Y | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
|
2802
2903
|
| move_after_transfer | string | Y | Y | | | | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
|
2803
|
-
| multi_session | int | Y | Y | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/><br/>(
|
2904
|
+
| multi_session | int | Y | Y | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/><br/>(<special>) |
|
2804
2905
|
| multi_session_threshold | int | Y | Y | | | | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
|
2805
|
-
| obfuscate_file_names | bool | | | | | Y | HTTP Gateway obfuscates file names when set to true.<br/>(
|
2906
|
+
| obfuscate_file_names | bool | | | | | Y | HTTP Gateway obfuscates file names when set to true.<br/>(<ignored>) |
|
2806
2907
|
| overwrite | string | Y | Y | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
|
2807
|
-
| password | string | | Y | | | | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.<br/>(
|
2808
|
-
| paths | array | Y | Y | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional).<br/>(
|
2908
|
+
| password | string | | Y | | | | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.<br/>(<ignored>) |
|
2909
|
+
| paths | array | Y | Y | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional).<br/>(<special>) |
|
2809
2910
|
| precalculate_job_size | bool | Y | Y | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
|
2810
2911
|
| preserve_access_time | bool | Y | Y | Y | Y | Y | Preserve the source-file access timestamps at the destination.<br/>Because source access times are updated by the transfer operation,<br/>the timestamp that is preserved is the one just before to the transfer.<br/>(--preserve-access-time) |
|
2811
2912
|
| preserve_acls | string | Y | | | | | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
|
@@ -2819,9 +2920,9 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2819
2920
|
| preserve_times | bool | | Y | | | | Preserve file timestamps.<br/>(--preserve-times) |
|
2820
2921
|
| proxy | string | Y | | | | | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
|
2821
2922
|
| rate_policy | string | Y | Y | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
|
2822
|
-
| rate_policy_allowed | string | | | Y | | | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed<br/>(
|
2823
|
-
| read_threads | int | | | | | | ascp4 only<br/>(
|
2824
|
-
| remote_access_key | string | | | | | | The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node.<br/>(
|
2923
|
+
| rate_policy_allowed | string | | | Y | | | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed<br/>(<ignored>) |
|
2924
|
+
| read_threads | int | | | | | | ascp4 only<br/>(<ignored>) |
|
2925
|
+
| remote_access_key | string | | | | | | The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node.<br/>(<ignored>) |
|
2825
2926
|
| remote_host | string | Y | Y | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
|
2826
2927
|
| remote_password | string | Y | Y | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
|
2827
2928
|
| remote_user | string | Y | Y | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
|
@@ -2830,38 +2931,27 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2830
2931
|
| remove_empty_source_directory | bool | Y | | | | | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
|
2831
2932
|
| remove_skipped | bool | Y | Y | Y | | | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
|
2832
2933
|
| resume_policy | string | Y | Y | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
|
2833
|
-
| retry_duration | string<br/>int | | Y | Y | | | Specifies how long to wait before retrying transfer. (e.g. "5min")<br/>(
|
2934
|
+
| retry_duration | string<br/>int | | Y | Y | | | Specifies how long to wait before retrying transfer. (e.g. "5min")<br/>(<ignored>) |
|
2834
2935
|
| source_root | string | Y | Y | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
|
2835
|
-
| source_root_id | string | | Y | | | | The file ID of the source root directory. Required when using Bearer token auth for the source node.<br/>(
|
2936
|
+
| source_root_id | string | | Y | | | | The file ID of the source root directory. Required when using Bearer token auth for the source node.<br/>(<ignored>) |
|
2836
2937
|
| src_base | string | Y | Y | | | | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
|
2837
|
-
| ssh_args | string | | | | | | Array of arguments to pass to SSH. Use with caution.<br/>(
|
2938
|
+
| ssh_args | string | | | | | | Array of arguments to pass to SSH. Use with caution.<br/>(<ignored>) |
|
2838
2939
|
| ssh_port | int | Y | Y | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
|
2839
2940
|
| ssh_private_key | string | Y | | | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
|
2840
2941
|
| ssh_private_key_passphrase | string | Y | | | | | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
|
2841
2942
|
| sshfp | string | Y | Y | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
|
2842
2943
|
| symlink_policy | string | Y | Y | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
|
2843
2944
|
| tags | hash | Y | Y | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
|
2844
|
-
| target_rate_cap_kbps | int | | | Y | | | Returned by upload/download_setup node API.<br/>(
|
2945
|
+
| target_rate_cap_kbps | int | | | Y | | | Returned by upload/download_setup node API.<br/>(<ignored>) |
|
2845
2946
|
| target_rate_kbps | int | Y | Y | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
|
2846
|
-
| target_rate_percentage | string | Y | Y | Y | Y | Y | TODO: remove ?<br/>(
|
2847
|
-
| title | string | | Y | Y | | | Title of the transfer<br/>(
|
2947
|
+
| target_rate_percentage | string | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2948
|
+
| title | string | | Y | Y | | | Title of the transfer<br/>(<ignored>) |
|
2848
2949
|
| token | string | Y | Y | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
|
2849
|
-
| use_ascp4 | bool | Y | Y | | | | specify version of protocol<br/>(
|
2850
|
-
| use_system_ssh | string | | | | | | TODO, comment...<br/>(
|
2851
|
-
| write_threads | int | | | | | | ascp4 only<br/>(
|
2852
|
-
| wss_enabled | bool | Y | Y | Y | Y | Y | Server has Web Socket service enabled<br/>(
|
2853
|
-
| wss_port | int | Y | Y | Y | Y | Y | TCP port used for websocket service feed<br/>(
|
2854
|
-
| EX_ascp_args | array | Y | | | | | DEPRECATED: (4.13) Use option transfer_info.ascp_args<br/>Add native command line arguments to ascp<br/>(<special>) |
|
2855
|
-
| EX_at_rest_password | string | Y | | | | | DEPRECATED: (4.13) Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2856
|
-
| EX_file_list | string | Y | | | | | DEPRECATED: (4.14) Use command line file list, or option transfer_info.ascp_args<br/>source file list<br/>(<special>) |
|
2857
|
-
| EX_file_pair_list | string | Y | | | | | DEPRECATED: (4.14) Use command line file pair list, or option transfer_info.ascp_args<br/>source file pair list<br/>(<special>) |
|
2858
|
-
| EX_http_proxy_url | string | Y | | | | | DEPRECATED: (4.14) TODO, use proxy option ?<br/>Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2859
|
-
| EX_http_transfer_jpeg | int | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>HTTP transfers as JPEG file<br/>(-j {int}) |
|
2860
|
-
| EX_license_text | string | Y | | | | | DEPRECATED: (4.14) Use env var ASPERA_SCP_LICENSE<br/>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2861
|
-
| EX_no_read | bool | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no read source<br/>(--no-read) |
|
2862
|
-
| EX_no_write | bool | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no write on destination<br/>(--no-write) |
|
2863
|
-
| EX_proxy_password | string | Y | | | | | DEPRECATED: (4.14) Use env var ASPERA_PROXY_PASS<br/>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL provided in parameter: proxy.<br/>(env:ASPERA_PROXY_PASS) |
|
2864
|
-
| EX_ssh_key_paths | array | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2950
|
+
| use_ascp4 | bool | Y | Y | | | | specify version of protocol<br/>(<special>) |
|
2951
|
+
| use_system_ssh | string | | | | | | TODO, comment...<br/>(<ignored>) |
|
2952
|
+
| write_threads | int | | | | | | ascp4 only<br/>(<ignored>) |
|
2953
|
+
| wss_enabled | bool | Y | Y | Y | Y | Y | Server has Web Socket service enabled<br/>(<special>) |
|
2954
|
+
| wss_port | int | Y | Y | Y | Y | Y | TCP port used for websocket service feed<br/>(<special>) |
|
2865
2955
|
|
2866
2956
|
#### Destination folder for transfers
|
2867
2957
|
|
@@ -2870,12 +2960,12 @@ The destination folder is set by `ascli` by default to:
|
|
2870
2960
|
- `.` for downloads
|
2871
2961
|
- `/` for uploads
|
2872
2962
|
|
2873
|
-
It is specified by the [*transfer-spec*](#
|
2963
|
+
It is specified by the [*transfer-spec*](#transfer-specification) parameter `destination_root`.
|
2874
2964
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
2875
2965
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
2876
2966
|
`--to-folder=<path>` .
|
2877
2967
|
|
2878
|
-
####
|
2968
|
+
#### List of files for transfers
|
2879
2969
|
|
2880
2970
|
When uploading, downloading or sending files, the user must specify the list of files to transfer.
|
2881
2971
|
|
@@ -2884,8 +2974,8 @@ By default the list of files to transfer is simply provided on the command line.
|
|
2884
2974
|
The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
|
2885
2975
|
The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
|
2886
2976
|
|
2887
|
-
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [*transfer-spec*](#
|
2888
|
-
So, eventually, the list of files to transfer is provided to the transfer agent using the [*transfer-spec*](#
|
2977
|
+
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [*transfer-spec*](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
|
2978
|
+
So, eventually, the list of files to transfer is provided to the transfer agent using the [*transfer-spec*](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
|
2889
2979
|
The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
|
2890
2980
|
|
2891
2981
|
Possible values for option `sources` are:
|
@@ -2903,7 +2993,7 @@ So, by default, the list of files to transfer will be simply specified on the co
|
|
2903
2993
|
ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile
|
2904
2994
|
```
|
2905
2995
|
|
2906
|
-
- An [Extended Value](#extended) with type **Array of String**
|
2996
|
+
- An [Extended Value](#extended-value-syntax) with type **Array of String**
|
2907
2997
|
|
2908
2998
|
> **Note:** extended values can be tested with the command `config echo`
|
2909
2999
|
|
@@ -3026,7 +3116,7 @@ Advanced Example: Send files `./file1` and `./folder2/files2` to server (e.g. `/
|
|
3026
3116
|
- Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--src-base=tmp_base tmp_base/file1 tmp_base/folder2/files2`
|
3027
3117
|
- One can also similarly use `--sources=@ts` and specify the list of files in the `paths` field of transfer spec with both `source` and `destination` for each file.
|
3028
3118
|
|
3029
|
-
####
|
3119
|
+
#### Multi-session transfer
|
3030
3120
|
|
3031
3121
|
Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on `direct` and `node` agents, not yet on connect.
|
3032
3122
|
|
@@ -3100,7 +3190,7 @@ The same progress bar is used for any type of transfer, using `ascp`, server to
|
|
3100
3190
|
|
3101
3191
|
To display the native progress bar of `ascp`, use `--progress-bar=no --transfer-info=@json:'{"quiet":false}'`.
|
3102
3192
|
|
3103
|
-
###
|
3193
|
+
### Scheduler
|
3104
3194
|
|
3105
3195
|
It is useful to configure automated scheduled execution.
|
3106
3196
|
`ascli` does not provide an internal scheduler.
|
@@ -3147,7 +3237,7 @@ EOF
|
|
3147
3237
|
|
3148
3238
|
> **Note:** Logging options are kept here in the cron file instead of configuration file to allow execution on command line with output on command line.
|
3149
3239
|
|
3150
|
-
###
|
3240
|
+
### Locking for exclusive execution
|
3151
3241
|
|
3152
3242
|
In some cases one needs to ensure that `ascli` is not executed several times in parallel.
|
3153
3243
|
|
@@ -3219,7 +3309,7 @@ The simplified format is:
|
|
3219
3309
|
One of the adapters, used in this manual, for testing, is `faux`.
|
3220
3310
|
It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
|
3221
3311
|
|
3222
|
-
###
|
3312
|
+
### `faux:` for testing
|
3223
3313
|
|
3224
3314
|
This is an extract of the man page of `ascp`.
|
3225
3315
|
This feature is a feature of `ascp`, not `ascli`.
|
@@ -3301,12 +3391,12 @@ ascli server upload /tmp/sample --to-folder=faux://
|
|
3301
3391
|
ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
|
3302
3392
|
```
|
3303
3393
|
|
3304
|
-
###
|
3394
|
+
### Usage
|
3305
3395
|
|
3306
3396
|
```text
|
3307
3397
|
ascli -h
|
3308
3398
|
NAME
|
3309
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3399
|
+
ascli -- a command line tool for Aspera Applications (v4.17.0)
|
3310
3400
|
|
3311
3401
|
SYNOPSIS
|
3312
3402
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -3395,14 +3485,14 @@ OPTIONS:
|
|
3395
3485
|
--silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
|
3396
3486
|
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
3397
3487
|
--http-options=VALUE Options for HTTP/S socket (Hash)
|
3398
|
-
--cache-tokens=ENUM Save and reuse
|
3488
|
+
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
3399
3489
|
--fpac=VALUE Proxy auto configuration script
|
3400
3490
|
--proxy-credentials=VALUE HTTP proxy credentials (user and password) (Array)
|
3401
3491
|
--ts=VALUE Override transfer spec values (Hash)
|
3402
3492
|
--to-folder=VALUE Destination folder for transferred files
|
3403
3493
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
3404
3494
|
--src-type=ENUM Type of file list: [list], pair
|
3405
|
-
--transfer=ENUM Type of transfer agent:
|
3495
|
+
--transfer=ENUM Type of transfer agent: node, [direct], alpha, httpgw, connect, trsdk
|
3406
3496
|
--transfer-info=VALUE Parameters for transfer agent (Hash)
|
3407
3497
|
|
3408
3498
|
|
@@ -3410,7 +3500,7 @@ COMMAND: shares
|
|
3410
3500
|
SUBCOMMANDS: admin files health
|
3411
3501
|
OPTIONS:
|
3412
3502
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3413
|
-
--username=VALUE
|
3503
|
+
--username=VALUE User's name to log in
|
3414
3504
|
--password=VALUE User's password
|
3415
3505
|
|
3416
3506
|
|
@@ -3418,7 +3508,7 @@ COMMAND: node
|
|
3418
3508
|
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer upload watch_folder
|
3419
3509
|
OPTIONS:
|
3420
3510
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3421
|
-
--username=VALUE
|
3511
|
+
--username=VALUE User's name to log in
|
3422
3512
|
--password=VALUE User's password
|
3423
3513
|
--validator=VALUE Identifier of validator (optional for central)
|
3424
3514
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
@@ -3432,7 +3522,7 @@ COMMAND: orchestrator
|
|
3432
3522
|
SUBCOMMANDS: health info plugins processes workflow
|
3433
3523
|
OPTIONS:
|
3434
3524
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3435
|
-
--username=VALUE
|
3525
|
+
--username=VALUE User's name to log in
|
3436
3526
|
--password=VALUE User's password
|
3437
3527
|
--result=VALUE Specify result value as: 'work_step:parameter'
|
3438
3528
|
--synchronous=ENUM Wait for completion: [no], yes
|
@@ -3444,7 +3534,7 @@ COMMAND: bss
|
|
3444
3534
|
SUBCOMMANDS: subscription
|
3445
3535
|
OPTIONS:
|
3446
3536
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3447
|
-
--username=VALUE
|
3537
|
+
--username=VALUE User's name to log in
|
3448
3538
|
--password=VALUE User's password
|
3449
3539
|
|
3450
3540
|
|
@@ -3452,7 +3542,7 @@ COMMAND: alee
|
|
3452
3542
|
SUBCOMMANDS: entitlement
|
3453
3543
|
OPTIONS:
|
3454
3544
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3455
|
-
--username=VALUE
|
3545
|
+
--username=VALUE User's name to log in
|
3456
3546
|
--password=VALUE User's password
|
3457
3547
|
|
3458
3548
|
|
@@ -3469,15 +3559,15 @@ OPTIONS:
|
|
3469
3559
|
|
3470
3560
|
|
3471
3561
|
COMMAND: faspex5
|
3472
|
-
SUBCOMMANDS: admin bearer_token gateway health packages postprocessing shared_folders user version
|
3562
|
+
SUBCOMMANDS: admin bearer_token gateway health invitations packages postprocessing shared_folders user version
|
3473
3563
|
OPTIONS:
|
3474
3564
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3475
|
-
--username=VALUE
|
3565
|
+
--username=VALUE User's name to log in
|
3476
3566
|
--password=VALUE User's password
|
3477
3567
|
--client-id=VALUE OAuth client identifier
|
3478
3568
|
--client-secret=VALUE OAuth client secret
|
3479
3569
|
--redirect-uri=VALUE OAuth redirect URI for web authentication
|
3480
|
-
--auth=ENUM OAuth type of authentication:
|
3570
|
+
--auth=ENUM OAuth type of authentication: web, [jwt], boot
|
3481
3571
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3482
3572
|
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
3483
3573
|
--box=VALUE Package inbox, either shared inbox name or one of: inbox, inbox_history, inbox_all, inbox_all_history, outbox, outbox_history, pending, pending_history, all or ALL
|
@@ -3489,19 +3579,19 @@ COMMAND: cos
|
|
3489
3579
|
SUBCOMMANDS: node
|
3490
3580
|
OPTIONS:
|
3491
3581
|
--bucket=VALUE Bucket name
|
3492
|
-
--endpoint=VALUE Storage endpoint
|
3582
|
+
--endpoint=VALUE Storage endpoint (URL)
|
3493
3583
|
--apikey=VALUE Storage API key
|
3494
|
-
--crn=VALUE Resource instance id
|
3584
|
+
--crn=VALUE Resource instance id (CRN)
|
3495
3585
|
--service-credentials=VALUE IBM Cloud service credentials (Hash)
|
3496
3586
|
--region=VALUE Storage region
|
3497
|
-
--identity=VALUE Authentication
|
3587
|
+
--identity=VALUE Authentication URL (https://iam.cloud.ibm.com/identity)
|
3498
3588
|
|
3499
3589
|
|
3500
3590
|
COMMAND: faspex
|
3501
3591
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
3502
3592
|
OPTIONS:
|
3503
3593
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3504
|
-
--username=VALUE
|
3594
|
+
--username=VALUE User's name to log in
|
3505
3595
|
--password=VALUE User's password
|
3506
3596
|
--link=VALUE Public link for specific operation
|
3507
3597
|
--delivery-info=VALUE Package delivery information (Hash)
|
@@ -3515,7 +3605,7 @@ COMMAND: preview
|
|
3515
3605
|
SUBCOMMANDS: check events scan show test trevents
|
3516
3606
|
OPTIONS:
|
3517
3607
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3518
|
-
--username=VALUE
|
3608
|
+
--username=VALUE User's name to log in
|
3519
3609
|
--password=VALUE User's password
|
3520
3610
|
--skip-format=ENUM Skip this preview format (multiple possible): png, mp4
|
3521
3611
|
--folder-reset-cache=ENUM Force detection of generated preview by refresh cache: [no], header, read
|
@@ -3551,7 +3641,7 @@ COMMAND: aoc
|
|
3551
3641
|
SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
|
3552
3642
|
OPTIONS:
|
3553
3643
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3554
|
-
--username=VALUE
|
3644
|
+
--username=VALUE User's name to log in
|
3555
3645
|
--password=VALUE User's password
|
3556
3646
|
--auth=ENUM OAuth type of authentication: web, [jwt]
|
3557
3647
|
--client-id=VALUE OAuth API client identifier
|
@@ -3563,13 +3653,19 @@ OPTIONS:
|
|
3563
3653
|
--workspace=VALUE Name of workspace (String, NilClass)
|
3564
3654
|
--new-user-option=VALUE New user creation option for unknown package recipients
|
3565
3655
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
3656
|
+
--validator=VALUE Identifier of validator (optional for central)
|
3657
|
+
--asperabrowserurl=VALUE URL for simple aspera web ui
|
3658
|
+
--sync-name=VALUE Sync name
|
3659
|
+
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
3660
|
+
--root-id=VALUE File id of top folder if using bearer tokens
|
3661
|
+
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3566
3662
|
|
3567
3663
|
|
3568
3664
|
COMMAND: server
|
3569
3665
|
SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
|
3570
3666
|
OPTIONS:
|
3571
3667
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3572
|
-
--username=VALUE
|
3668
|
+
--username=VALUE User's name to log in
|
3573
3669
|
--password=VALUE User's password
|
3574
3670
|
--ssh-keys=VALUE SSH key path list (Array or single)
|
3575
3671
|
--passphrase=VALUE SSH private key passphrase
|
@@ -3581,7 +3677,7 @@ COMMAND: console
|
|
3581
3677
|
SUBCOMMANDS: health transfer
|
3582
3678
|
OPTIONS:
|
3583
3679
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3584
|
-
--username=VALUE
|
3680
|
+
--username=VALUE User's name to log in
|
3585
3681
|
--password=VALUE User's password
|
3586
3682
|
--filter-from=DATE Only after date
|
3587
3683
|
--filter-to=DATE Only before date
|
@@ -3594,7 +3690,7 @@ OPTIONS:
|
|
3594
3690
|
### Bulk creation and deletion of resources
|
3595
3691
|
|
3596
3692
|
Bulk creation and deletion of resources are possible using option `bulk` (`yes`,`no`(default)).
|
3597
|
-
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended).
|
3693
|
+
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended-value-syntax).
|
3598
3694
|
This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
|
3599
3695
|
|
3600
3696
|
### Plugins
|
@@ -3646,13 +3742,13 @@ Created ./foo.rb
|
|
3646
3742
|
ascli --plugin-folder=. foo
|
3647
3743
|
```
|
3648
3744
|
|
3649
|
-
##
|
3745
|
+
## Plugin: `aoc`: IBM Aspera on Cloud
|
3650
3746
|
|
3651
3747
|
Aspera on Cloud API requires the use of Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
|
3652
3748
|
|
3653
3749
|
It is recommended to use the wizard to set it up, although manual configuration is also possible.
|
3654
3750
|
|
3655
|
-
###
|
3751
|
+
### Configuration: Using Wizard
|
3656
3752
|
|
3657
3753
|
`ascli` provides a configuration wizard.
|
3658
3754
|
|
@@ -3695,7 +3791,7 @@ ascli config wizard _your_instance_ aoc
|
|
3695
3791
|
|
3696
3792
|
> **Note:** In above example, replace `_your_instance_` with the first part of your actual AoC URL: `https://_your_instance_.ibmaspera.com`.
|
3697
3793
|
|
3698
|
-
###
|
3794
|
+
### Configuration: Using manual setup
|
3699
3795
|
|
3700
3796
|
> **Note:** If you used the wizard (recommended): skip this section.
|
3701
3797
|
|
@@ -3709,15 +3805,17 @@ Several types of OAuth authentication are supported:
|
|
3709
3805
|
|
3710
3806
|
The authentication method is controlled by option `auth`.
|
3711
3807
|
|
3712
|
-
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#
|
3808
|
+
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset)' for Aspera on Cloud](#configuration-for-aspera-on-cloud).
|
3713
3809
|
|
3714
|
-
For a more convenient, browser-less, experience follow the [JWT](#
|
3810
|
+
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
|
3715
3811
|
|
3716
3812
|
In Oauth, a **Bearer** token is generated to authenticate REST calls.
|
3717
3813
|
Bearer tokens are valid for a period of time defined (by the AoC app, configurable by admin) at its creation.
|
3718
3814
|
`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
|
3719
3815
|
|
3720
|
-
####
|
3816
|
+
#### API Client Registration
|
3817
|
+
|
3818
|
+
> **Optional**
|
3721
3819
|
|
3722
3820
|
If you use the built-in client_id and client_secret, skip this and do not set them in next section.
|
3723
3821
|
|
@@ -3742,11 +3840,11 @@ Let's start by a registration with web based authentication (auth=web):
|
|
3742
3840
|
|
3743
3841
|
Once the client is registered, a **Client ID** and **Secret** are created, these values will be used in the next step.
|
3744
3842
|
|
3745
|
-
####
|
3843
|
+
#### Configuration for Aspera on Cloud
|
3746
3844
|
|
3747
|
-
If you did not use the wizard, you can also manually create a [
|
3845
|
+
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset)' for `ascli` in its configuration file.
|
3748
3846
|
|
3749
|
-
Lets create a [
|
3847
|
+
Lets create a [Option Preset](#option-preset)' called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
|
3750
3848
|
|
3751
3849
|
```bash
|
3752
3850
|
ascli config preset ask my_aoc_org url client_id client_secret
|
@@ -3760,7 +3858,7 @@ updated: my_aoc_org
|
|
3760
3858
|
|
3761
3859
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
3762
3860
|
|
3763
|
-
Define this [
|
3861
|
+
Define this [Option Preset](#option-preset)' as default configuration for the `aspera` plugin:
|
3764
3862
|
|
3765
3863
|
```bash
|
3766
3864
|
ascli config preset set default aoc my_aoc_org
|
@@ -3768,12 +3866,12 @@ ascli config preset set default aoc my_aoc_org
|
|
3768
3866
|
|
3769
3867
|
> **Note:** Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
3770
3868
|
|
3771
|
-
####
|
3869
|
+
#### Authentication with private key
|
3772
3870
|
|
3773
3871
|
For a Browser-less, Private Key-based authentication, use the following steps.
|
3774
3872
|
|
3775
|
-
In order to use JWT for Aspera on Cloud API client authentication,
|
3776
|
-
a [private/public key pair](#
|
3873
|
+
In order to use JSON Web Token (JWT) for Aspera on Cloud API client authentication,
|
3874
|
+
a [private/public key pair](#private-key) must be used.
|
3777
3875
|
|
3778
3876
|
##### API Client JWT activation
|
3779
3877
|
|
@@ -3851,9 +3949,9 @@ modified
|
|
3851
3949
|
|
3852
3950
|
> **Note:** The `aspera user info show` command can be used to verify modifications.
|
3853
3951
|
|
3854
|
-
#### [
|
3952
|
+
#### [Option Preset](#option-preset)' modification for JWT
|
3855
3953
|
|
3856
|
-
To activate default use of JWT authentication for `ascli` using the [
|
3954
|
+
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset)', do the following:
|
3857
3955
|
|
3858
3956
|
- Change auth method to JWT
|
3859
3957
|
- Provide location of private key
|
@@ -3885,9 +3983,9 @@ So, provide the same options as for regular authentication, and provide the priv
|
|
3885
3983
|
A user may not be part of any workspace, but still have access to shared folders (using private links).
|
3886
3984
|
In that case, it is possible to list those shared folder by using a value for option `workspace` equal to `@none:` or `@json:null` or `@ruby:nil`.
|
3887
3985
|
|
3888
|
-
####
|
3986
|
+
#### AoC: First Use
|
3889
3987
|
|
3890
|
-
Once client has been registered and [
|
3988
|
+
Once client has been registered and [Option Preset](#option-preset)' created: `ascli` can be used:
|
3891
3989
|
|
3892
3990
|
```bash
|
3893
3991
|
ascli aoc files br /
|
@@ -3930,7 +4028,7 @@ It allows actions (create, update, delete) on **resources**: users, group, nodes
|
|
3930
4028
|
The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
|
3931
4029
|
|
3932
4030
|
The option `query` can be optionally used.
|
3933
|
-
It expects a `Hash` using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`.
|
4031
|
+
It expects a `Hash` using [Extended Value Syntax](#extended-value-syntax), generally provided using: `--query=@json:{...}`.
|
3934
4032
|
Values are directly sent to the API call and used as a filter on server side.
|
3935
4033
|
|
3936
4034
|
The following parameters are supported:
|
@@ -3975,9 +4073,9 @@ ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"s
|
|
3975
4073
|
|
3976
4074
|
Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
|
3977
4075
|
|
3978
|
-
> **Note:** The option `select` can also be used to further refine selection, refer to [section earlier](#
|
4076
|
+
> **Note:** The option `select` can also be used to further refine selection, refer to [section earlier](#option-select).
|
3979
4077
|
|
3980
|
-
####
|
4078
|
+
#### Selecting a resource
|
3981
4079
|
|
3982
4080
|
Resources are identified by a unique `id`, as well as a unique `name` (case insensitive).
|
3983
4081
|
|
@@ -3988,7 +4086,7 @@ To execute an action on a specific resource, select it using one of those method
|
|
3988
4086
|
- Provide option `id` : `aoc admin res node show 123`
|
3989
4087
|
- Provide option `name` : `aoc admin res node show --name=abc`
|
3990
4088
|
|
3991
|
-
####
|
4089
|
+
#### Creating a resource
|
3992
4090
|
|
3993
4091
|
New resources (users, groups, workspaces, etc..) can be created using a command like:
|
3994
4092
|
|
@@ -4043,7 +4141,7 @@ For example in a command like:
|
|
4043
4141
|
ascli aoc admin res node 123 --secret="my_secret_here" v3 info
|
4044
4142
|
```
|
4045
4143
|
|
4046
|
-
It is also possible to store secrets in the [secret vault](#vault) and then automatically find the related secret using the [config finder](#
|
4144
|
+
It is also possible to store secrets in the [secret vault](#secret-vault) and then automatically find the related secret using the [config finder](#configuration-finder).
|
4047
4145
|
|
4048
4146
|
#### Activity
|
4049
4147
|
|
@@ -4094,7 +4192,7 @@ The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports
|
|
4094
4192
|
|
4095
4193
|
#### Using ATS
|
4096
4194
|
|
4097
|
-
Refer to section **Examples** of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
|
4195
|
+
Refer to section **Examples** of [ATS](#plugin-ats-ibm-aspera-transfer-service) and substitute command `ats` with `aoc admin ats`.
|
4098
4196
|
|
4099
4197
|
#### Files with type `link`
|
4100
4198
|
|
@@ -4169,7 +4267,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --di
|
|
4169
4267
|
+-------+---------+
|
4170
4268
|
```
|
4171
4269
|
|
4172
|
-
#### Example:
|
4270
|
+
#### Example: Find deactivated users since more than 2 years
|
4173
4271
|
|
4174
4272
|
```ruby
|
4175
4273
|
ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
|
@@ -4248,7 +4346,7 @@ Other query parameters:
|
|
4248
4346
|
{"workspace_membership_through":true,"include_indirect":true}
|
4249
4347
|
```
|
4250
4348
|
|
4251
|
-
#### Example:
|
4349
|
+
#### Example: Add all members of a workspace to another workspace
|
4252
4350
|
|
4253
4351
|
a- Get id of first workspace
|
4254
4352
|
|
@@ -4267,7 +4365,7 @@ WS2ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --se
|
|
4267
4365
|
c- Extract membership information
|
4268
4366
|
|
4269
4367
|
```bash
|
4270
|
-
ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp
|
4368
|
+
ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp --output=ws1_members.json
|
4271
4369
|
```
|
4272
4370
|
|
4273
4371
|
d- Convert to creation data for second workspace:
|
@@ -4432,7 +4530,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
4432
4530
|
|
4433
4531
|
- Create the access key on ATS
|
4434
4532
|
|
4435
|
-
The creation options are the ones of ATS API, refer to the [section on ATS](#
|
4533
|
+
The creation options are the ones of ATS API, refer to the [section on ATS](#ats-access-key-creation-parameters) for more details and examples.
|
4436
4534
|
|
4437
4535
|
```bash
|
4438
4536
|
ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
|
@@ -4461,7 +4559,7 @@ Creation of a node with a self-managed node is similar, but the command `aoc adm
|
|
4461
4559
|
### List of files to transfer
|
4462
4560
|
|
4463
4561
|
Source files are provided as a list with the `sources` option.
|
4464
|
-
Refer to section [File list](#
|
4562
|
+
Refer to section [File list](#list-of-files-for-transfers)
|
4465
4563
|
|
4466
4564
|
> **Note:** A special case is when the source files are located on **Aspera on Cloud** (i.e. using access keys and the `file id` API).
|
4467
4565
|
|
@@ -4490,17 +4588,26 @@ General syntax:
|
|
4490
4588
|
ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
|
4491
4589
|
```
|
4492
4590
|
|
4493
|
-
|
4591
|
+
Package creation parameter are sent as positional argument.
|
4592
|
+
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
4593
|
+
|
4594
|
+
List allowed shared inbox destinations with:
|
4595
|
+
|
4596
|
+
```bash
|
4597
|
+
ascli aoc packages shared_inboxes list
|
4598
|
+
```
|
4599
|
+
|
4600
|
+
Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: **user** or **shared inbox**:
|
4601
|
+
|
4602
|
+
- Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
4603
|
+
- or just names: `"recipients":[{"The Dest"}]`.
|
4604
|
+
|
4605
|
+
ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
|
4494
4606
|
|
4495
|
-
|
4496
|
-
|
4497
|
-
-
|
4498
|
-
-
|
4499
|
-
- Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
4500
|
-
- or just names: `"recipients":[{"The Dest"}]` . ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
|
4501
|
-
- If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
|
4502
|
-
- if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
|
4503
|
-
- if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
4607
|
+
If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and:
|
4608
|
+
|
4609
|
+
- if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
|
4610
|
+
- if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
4504
4611
|
|
4505
4612
|
#### Example: Send a package with one file to two users, using their email
|
4506
4613
|
|
@@ -4532,6 +4639,8 @@ To list packages in a shared inbox, the query has to be specified with the the s
|
|
4532
4639
|
Additional parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode).
|
4533
4640
|
The current workspace is added unless specified in the query.
|
4534
4641
|
|
4642
|
+
> **Note:** By default, `exclude_dropbox_packages` is set to true for user packages, and to false for shared inbox packages. This can be overridden in the query.
|
4643
|
+
|
4535
4644
|
Using shared inbox name:
|
4536
4645
|
|
4537
4646
|
```bash
|
@@ -4579,7 +4688,7 @@ Let's send a package with the file `10M.dat` from subfolder /src_folder in a pac
|
|
4579
4688
|
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
|
4580
4689
|
```
|
4581
4690
|
|
4582
|
-
####
|
4691
|
+
#### Receive new packages only (Cargo)
|
4583
4692
|
|
4584
4693
|
It is possible to automatically download new packages, like using Aspera Cargo:
|
4585
4694
|
|
@@ -4657,13 +4766,13 @@ ascli aoc files short_link public delete _id_
|
|
4657
4766
|
|
4658
4767
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
4659
4768
|
|
4660
|
-
Although optional, the creation of [
|
4769
|
+
Although optional, the creation of [Option Preset](#option-preset)' is recommended to avoid placing all parameters in the command line.
|
4661
4770
|
|
4662
4771
|
Procedure to send a file from org1 to org2:
|
4663
4772
|
|
4664
|
-
- Get access to Organization 1 and create a [
|
4773
|
+
- Get access to Organization 1 and create a [Option Preset](#option-preset)': e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
|
4665
4774
|
- Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
|
4666
|
-
- Get access to Organization 2 and create a [
|
4775
|
+
- Get access to Organization 2 and create a [Option Preset](#option-preset)': e.g. `org2`
|
4667
4776
|
- Check that access works and locate the destination folder `mydestfolder`
|
4668
4777
|
- Execute the following:
|
4669
4778
|
|
@@ -4680,7 +4789,7 @@ Explanation:
|
|
4680
4789
|
- `|` the standard output of the first command is fed into the second one
|
4681
4790
|
- `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
|
4682
4791
|
- `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
|
4683
|
-
- `--transfer=node` use transfer agent type `node` instead of default [`direct`](#
|
4792
|
+
- `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agent-direct)
|
4684
4793
|
- `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
|
4685
4794
|
|
4686
4795
|
#### Find Files
|
@@ -4695,120 +4804,123 @@ ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find
|
|
4695
4804
|
|
4696
4805
|
For instructions, refer to section `find` for plugin `node`.
|
4697
4806
|
|
4698
|
-
###
|
4699
|
-
|
4700
|
-
|
4701
|
-
|
4702
|
-
|
4703
|
-
|
4704
|
-
|
4705
|
-
|
4706
|
-
|
4707
|
-
|
4708
|
-
|
4709
|
-
|
4710
|
-
|
4711
|
-
|
4712
|
-
|
4713
|
-
|
4714
|
-
|
4715
|
-
|
4716
|
-
|
4717
|
-
|
4718
|
-
|
4719
|
-
|
4720
|
-
|
4721
|
-
|
4722
|
-
|
4723
|
-
|
4724
|
-
|
4725
|
-
|
4726
|
-
|
4727
|
-
|
4728
|
-
|
4729
|
-
|
4730
|
-
|
4731
|
-
|
4732
|
-
|
4733
|
-
|
4734
|
-
|
4735
|
-
|
4736
|
-
|
4737
|
-
|
4738
|
-
|
4739
|
-
|
4740
|
-
|
4741
|
-
|
4742
|
-
|
4743
|
-
|
4744
|
-
|
4745
|
-
|
4746
|
-
|
4747
|
-
|
4748
|
-
|
4749
|
-
|
4750
|
-
|
4751
|
-
|
4752
|
-
|
4753
|
-
|
4754
|
-
|
4755
|
-
|
4756
|
-
|
4757
|
-
|
4758
|
-
|
4759
|
-
|
4760
|
-
|
4761
|
-
|
4762
|
-
|
4763
|
-
|
4764
|
-
|
4765
|
-
|
4766
|
-
|
4767
|
-
|
4768
|
-
|
4769
|
-
|
4770
|
-
|
4771
|
-
|
4772
|
-
|
4773
|
-
|
4774
|
-
|
4775
|
-
|
4776
|
-
|
4777
|
-
|
4778
|
-
|
4779
|
-
|
4780
|
-
|
4781
|
-
|
4782
|
-
|
4783
|
-
|
4784
|
-
|
4785
|
-
|
4786
|
-
|
4787
|
-
|
4788
|
-
|
4789
|
-
|
4790
|
-
|
4791
|
-
|
4792
|
-
|
4793
|
-
|
4794
|
-
|
4795
|
-
|
4796
|
-
|
4797
|
-
|
4798
|
-
|
4799
|
-
|
4800
|
-
|
4801
|
-
|
4802
|
-
|
4803
|
-
|
4804
|
-
|
4805
|
-
|
4806
|
-
|
4807
|
-
|
4808
|
-
|
4809
|
-
|
4810
|
-
|
4811
|
-
|
4807
|
+
### Aoc sample commands
|
4808
|
+
|
4809
|
+
> **Note:** Add `ascli aoc` in front of the commands:
|
4810
|
+
|
4811
|
+
```bash
|
4812
|
+
admin analytics transfers nodes
|
4813
|
+
admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
4814
|
+
admin analytics transfers users --once_only=yes
|
4815
|
+
admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4816
|
+
admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4817
|
+
admin ats access_key delete ak1ibmcloud
|
4818
|
+
admin ats access_key list --fields=name,id
|
4819
|
+
admin ats access_key node ak1ibmcloud --secret=my_secret_here browse /
|
4820
|
+
admin ats cluster clouds
|
4821
|
+
admin ats cluster list
|
4822
|
+
admin ats cluster show --cloud=aws --region=eu-west-1
|
4823
|
+
admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
4824
|
+
admin auth_providers list
|
4825
|
+
admin res application list
|
4826
|
+
admin res client list
|
4827
|
+
admin res client_access_key list
|
4828
|
+
admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
|
4829
|
+
admin res client_registration_token delete client_reg_id
|
4830
|
+
admin res client_registration_token list
|
4831
|
+
admin res contact list
|
4832
|
+
admin res dropbox list
|
4833
|
+
admin res dropbox_membership list
|
4834
|
+
admin res group list
|
4835
|
+
admin res kms_profile list
|
4836
|
+
admin res node list
|
4837
|
+
admin res operation list
|
4838
|
+
admin res organization show
|
4839
|
+
admin res package list --http-options=@json:'{"read_timeout":120.0}'
|
4840
|
+
admin res saml_configuration list
|
4841
|
+
admin res self show
|
4842
|
+
admin res short_link list
|
4843
|
+
admin res user list
|
4844
|
+
admin res user modify %name:my_user_email @json:'{"deactivated":false}'
|
4845
|
+
admin res workspace_membership list
|
4846
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret browse /
|
4847
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret delete /folder1
|
4848
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4849
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4850
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4851
|
+
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4852
|
+
admin resource workspace list
|
4853
|
+
admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4854
|
+
admin subscription
|
4855
|
+
automation workflow action wf_id create @json:'{"name":"toto"}' \
|
4856
|
+
automation workflow create @json:'{"name":"test_workflow"}'
|
4857
|
+
automation workflow delete wf_id
|
4858
|
+
automation workflow list
|
4859
|
+
automation workflow list --query=@json:'{"show_org_workflows":"true"}' --scope=admin:all
|
4860
|
+
automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data --output=test
|
4861
|
+
bearer_token --display=data --scope=user:all
|
4862
|
+
files bearer /
|
4863
|
+
files bearer_token_node / --cache-tokens=no
|
4864
|
+
files browse /
|
4865
|
+
files browse / --url=my_private_link
|
4866
|
+
files browse / --url=my_public_link_folder_no_pass
|
4867
|
+
files browse / --url=my_public_link_folder_pass --password=my_public_link_password
|
4868
|
+
files delete /testsrc
|
4869
|
+
files download --transfer=alpha testdst/test_file.bin
|
4870
|
+
files download --transfer=connect testdst/test_file.bin
|
4871
|
+
files find / '\.partial$'
|
4872
|
+
files http_node_download --to-folder=. testdst/test_file.bin
|
4873
|
+
files mkdir /testsrc
|
4874
|
+
files modify my_test_folder
|
4875
|
+
files permission my_test_folder list
|
4876
|
+
files rename /some_folder testdst
|
4877
|
+
files short_link private create /testdst
|
4878
|
+
files short_link private list /testdst
|
4879
|
+
files short_link public create testdst
|
4880
|
+
files show %id:aoc_file_id
|
4881
|
+
files show /
|
4882
|
+
files show testdst/test_file.bin
|
4883
|
+
files sync admin status --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
4884
|
+
files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
4885
|
+
files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
4886
|
+
files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
4887
|
+
files thumbnail my_test_folder/video_file.mpg
|
4888
|
+
files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
|
4889
|
+
files transfer push /testsrc --to-folder=/testdst test_file.bin
|
4890
|
+
files upload --to-folder=/ test_file.bin --url=my_public_link_folder_no_pass
|
4891
|
+
files upload --to-folder=/testsrc test_file.bin
|
4892
|
+
files upload --workspace=my_other_workspace --to-folder=my_other_folder test_file.bin --transfer=node --transfer-info=@json:@stdin:
|
4893
|
+
files v3 info
|
4894
|
+
gateway --pid-file=pid_aocfxgw https://localhost:12345/aspera/faspex &
|
4895
|
+
org --url=my_public_link_recv_from_aoc_user
|
4896
|
+
organization
|
4897
|
+
packages browse package_id3 /contents
|
4898
|
+
packages list
|
4899
|
+
packages list --query=@json:'{"dropbox_name":"my_shared_inbox_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
|
4900
|
+
packages receive ALL --once-only=yes --to-folder=. --lock-port=12345
|
4901
|
+
packages receive ALL --once-only=yes --to-folder=. --lock-port=12345 --query=@json:'{"dropbox_name":"my_shared_inbox_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
4902
|
+
packages receive INIT --once-only=yes --query=@json:'{"dropbox_name":"my_shared_inbox_name"}'
|
4903
|
+
packages receive package_id3 --to-folder=.
|
4904
|
+
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' test_file.bin
|
4905
|
+
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
|
4906
|
+
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":{"Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
|
4907
|
+
packages send --workspace=my_workspace_shared_inbox @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_name"]}' test_file.bin
|
4908
|
+
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_external"]}' --new-user-option=@json:'{"package_contact":true}' test_file.bin
|
4909
|
+
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"],"note":"my note"}' test_file.bin
|
4910
|
+
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_aoc_user --password=my_public_link_send_use_pass
|
4911
|
+
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_shared_inbox
|
4912
|
+
packages shared_inboxes list
|
4913
|
+
remind --username=my_user_email
|
4914
|
+
servers
|
4915
|
+
user pref modify @json:'{"default_language":"en-us"}'
|
4916
|
+
user pref show
|
4917
|
+
user profile modify @json:'{"name":"dummy change"}'
|
4918
|
+
user profile show
|
4919
|
+
user workspaces current
|
4920
|
+
user workspaces list
|
4921
|
+
```
|
4922
|
+
|
4923
|
+
## Plugin: `ats`: IBM Aspera Transfer Service
|
4812
4924
|
|
4813
4925
|
ATS is usable either :
|
4814
4926
|
|
@@ -4889,7 +5001,7 @@ ascli ats api_key create
|
|
4889
5001
|
ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
4890
5002
|
```
|
4891
5003
|
|
4892
|
-
###
|
5004
|
+
### ATS Access key creation parameters
|
4893
5005
|
|
4894
5006
|
When creating an ATS access key, the option `params` must contain an extended value with the creation parameters.
|
4895
5007
|
Those are directly the parameters expected by the [ATS API](https://developer.ibm.com/apis/catalog?search=%22Aspera%20ATS%20API%22).
|
@@ -4930,28 +5042,30 @@ ascli ats access_key list --field=id --format=csv | ascli ats access_key delete
|
|
4930
5042
|
|
4931
5043
|
The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
|
4932
5044
|
|
4933
|
-
###
|
5045
|
+
### Ats sample commands
|
5046
|
+
|
5047
|
+
> **Note:** Add `ascli ats` in front of the commands:
|
4934
5048
|
|
4935
5049
|
```bash
|
4936
|
-
|
4937
|
-
|
4938
|
-
|
4939
|
-
|
4940
|
-
|
4941
|
-
|
4942
|
-
|
4943
|
-
|
4944
|
-
|
4945
|
-
|
4946
|
-
|
4947
|
-
|
4948
|
-
|
4949
|
-
|
4950
|
-
|
4951
|
-
|
5050
|
+
access_key cluster ak2ibmcloud --secret=my_secret_here
|
5051
|
+
access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
5052
|
+
access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak2ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
5053
|
+
access_key delete ak2ibmcloud
|
5054
|
+
access_key delete ak_aws
|
5055
|
+
access_key entitlement ak2ibmcloud
|
5056
|
+
access_key list --fields=name,id
|
5057
|
+
access_key node ak2ibmcloud browse / --secret=my_secret_here
|
5058
|
+
access_key show ak2ibmcloud
|
5059
|
+
api_key create
|
5060
|
+
api_key instances
|
5061
|
+
api_key list
|
5062
|
+
cluster clouds
|
5063
|
+
cluster list
|
5064
|
+
cluster show --cloud=aws --region=eu-west-1
|
5065
|
+
cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
4952
5066
|
```
|
4953
5067
|
|
4954
|
-
##
|
5068
|
+
## Plugin: `server`: IBM Aspera High Speed Transfer Server (SSH)
|
4955
5069
|
|
4956
5070
|
The `server` plugin is used for operations on Aspera HSTS using SSH authentication.
|
4957
5071
|
It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
|
@@ -4962,39 +5076,41 @@ then commands `ascp` (for transfers) and `ascmd` (for file operations) are execu
|
|
4962
5076
|
|
4963
5077
|
### Server sample commands
|
4964
5078
|
|
4965
|
-
|
4966
|
-
|
4967
|
-
|
4968
|
-
|
4969
|
-
|
4970
|
-
|
4971
|
-
|
4972
|
-
|
4973
|
-
|
4974
|
-
|
4975
|
-
|
4976
|
-
|
4977
|
-
|
4978
|
-
|
4979
|
-
|
4980
|
-
|
4981
|
-
|
4982
|
-
|
4983
|
-
|
4984
|
-
|
4985
|
-
|
4986
|
-
|
4987
|
-
|
4988
|
-
|
4989
|
-
|
4990
|
-
|
4991
|
-
|
4992
|
-
|
4993
|
-
|
4994
|
-
|
4995
|
-
|
4996
|
-
|
4997
|
-
|
5079
|
+
> **Note:** Add `ascli server` in front of the commands:
|
5080
|
+
|
5081
|
+
```bash
|
5082
|
+
browse /
|
5083
|
+
browse / --password=@none: --ssh-options=@json:'{"number_of_password_prompts":0}' --ssh-keys=$aspera_key_path
|
5084
|
+
browse my_inside_folder/test_file.bin
|
5085
|
+
browse my_upload_folder/target_hot
|
5086
|
+
cp my_inside_folder/test_file.bin my_upload_folder/200KB.2
|
5087
|
+
delete my_inside_folder
|
5088
|
+
delete my_upload_folder/to.delete
|
5089
|
+
df
|
5090
|
+
download my_inside_folder/test_file.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
|
5091
|
+
download my_inside_folder/test_file.bin --to-folder=my_upload_folder --transfer=node
|
5092
|
+
du /
|
5093
|
+
health transfer --to-folder=my_upload_folder --format=nagios
|
5094
|
+
info
|
5095
|
+
md5sum my_inside_folder/test_file.bin
|
5096
|
+
mkdir my_inside_folder --logger=stdout
|
5097
|
+
mkdir my_upload_folder/target_hot
|
5098
|
+
mv my_upload_folder/200KB.2 my_upload_folder/to.delete
|
5099
|
+
sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
|
5100
|
+
sync admin status --sync-info=@json:'{"name":"sync2"}'
|
5101
|
+
sync admin status my_sync --sync-info=@json:'{"sessions":[{"name":"my_sync","local_dir":"/data/local_sync"}]}'
|
5102
|
+
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
|
5103
|
+
sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
|
5104
|
+
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
|
5105
|
+
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
|
5106
|
+
upload 'test_file.bin' --to-folder=my_inside_folder --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":100000}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress-bar=yes
|
5107
|
+
upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-list","filelist.txt"]}' --to-folder=my_inside_folder
|
5108
|
+
upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-pair-list","file_pair_list.txt"]}'
|
5109
|
+
upload --sources=@ts --ts=@json:'{"paths":[{"source":"test_file.bin","destination":"my_inside_folder/other_name_4"}]}' --transfer=trsdk
|
5110
|
+
upload --src-type=pair 'test_file.bin' my_inside_folder/other_name_2 --notify-to=my_email_external --transfer-info=@json:'{"ascp_args":["-l","100m"]}'
|
5111
|
+
upload --src-type=pair --sources=@json:'["test_file.bin","my_inside_folder/other_name_3"]' --transfer-info=@json:'{"quiet":false}' --ts=@json:'{"use_ascp4":true}' --progress=no
|
5112
|
+
upload --src-type=pair test_file.bin my_upload_folder/other_name_5 --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"my_secret_here","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
|
5113
|
+
upload --to-folder=my_upload_folder/target_hot --lock-port=12345 --transfer-info=@json:'{"ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
|
4998
5114
|
```
|
4999
5115
|
|
5000
5116
|
### Authentication on Server with SSH session
|
@@ -5087,7 +5203,7 @@ ascli server browse /aspera-test-dir-large
|
|
5087
5203
|
ascli server download /aspera-test-dir-large/200MB
|
5088
5204
|
```
|
5089
5205
|
|
5090
|
-
`initdemo` creates a [
|
5206
|
+
`initdemo` creates a [Option Preset](#option-preset)' `demoserver` and set it as default for plugin `server`.
|
5091
5207
|
|
5092
5208
|
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
|
5093
5209
|
|
@@ -5095,7 +5211,7 @@ If an SSH private key is used for authentication with a passphrase, the passphra
|
|
5095
5211
|
ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ssh_keys=_private_key_path_here_ --passphrase=_passphrase_here_
|
5096
5212
|
```
|
5097
5213
|
|
5098
|
-
##
|
5214
|
+
## Plugin: `node`: IBM Aspera High Speed Transfer Server Node
|
5099
5215
|
|
5100
5216
|
This plugin gives access to capabilities provided by the HSTS node API.
|
5101
5217
|
|
@@ -5221,7 +5337,7 @@ There are three sync types of operations:
|
|
5221
5337
|
|
5222
5338
|
It is possible to start a FASPStream session using the node API:
|
5223
5339
|
|
5224
|
-
Use the command `ascli node stream create --ts=@json:<value>`, with [*transfer-spec*](#
|
5340
|
+
Use the command `ascli node stream create --ts=@json:<value>`, with [*transfer-spec*](#transfer-specification):
|
5225
5341
|
|
5226
5342
|
```json
|
5227
5343
|
{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
|
@@ -5390,7 +5506,7 @@ Let's assume that the access key was created, and a default configuration is set
|
|
5390
5506
|
```
|
5391
5507
|
|
5392
5508
|
> **Note:** This key is not used for authentication, it is used to sign bearer tokens.
|
5393
|
-
> Refer to section [private key](#
|
5509
|
+
> Refer to section [private key](#private-key) for more details on generation.
|
5394
5510
|
|
5395
5511
|
- The corresponding public key shall be placed as an attribute of the **access key**:
|
5396
5512
|
|
@@ -5435,7 +5551,7 @@ Let's assume that the access key was created, and a default configuration is set
|
|
5435
5551
|
- Create a Bearer token for the user:
|
5436
5552
|
|
5437
5553
|
```bash
|
5438
|
-
ascli node bearer_token @file:./myorgkey.pem @json:'{"user_id":"'$my_user_id'","_validity":3600}'
|
5554
|
+
ascli node bearer_token @file:./myorgkey.pem @json:'{"user_id":"'$my_user_id'","_validity":3600}' --output=bearer.txt
|
5439
5555
|
```
|
5440
5556
|
|
5441
5557
|
#### Bearer token: User side
|
@@ -5454,75 +5570,86 @@ ascli node -N --url=... --password="Bearer $(cat bearer.txt)" --root-id=$my_fold
|
|
5454
5570
|
|
5455
5571
|
### Node sample commands
|
5456
5572
|
|
5457
|
-
|
5458
|
-
|
5459
|
-
|
5460
|
-
|
5461
|
-
|
5462
|
-
|
5463
|
-
|
5464
|
-
|
5465
|
-
|
5466
|
-
|
5467
|
-
|
5468
|
-
|
5469
|
-
|
5470
|
-
|
5471
|
-
|
5472
|
-
|
5473
|
-
|
5474
|
-
|
5475
|
-
|
5476
|
-
|
5477
|
-
|
5478
|
-
|
5479
|
-
|
5480
|
-
|
5481
|
-
|
5482
|
-
|
5483
|
-
|
5484
|
-
|
5485
|
-
|
5486
|
-
|
5487
|
-
|
5488
|
-
|
5489
|
-
|
5490
|
-
|
5491
|
-
|
5492
|
-
|
5493
|
-
|
5494
|
-
|
5495
|
-
|
5496
|
-
|
5497
|
-
|
5498
|
-
|
5499
|
-
|
5500
|
-
|
5501
|
-
|
5502
|
-
|
5503
|
-
|
5504
|
-
|
5505
|
-
|
5506
|
-
|
5507
|
-
|
5508
|
-
|
5509
|
-
|
5510
|
-
|
5511
|
-
|
5512
|
-
|
5513
|
-
|
5514
|
-
|
5515
|
-
|
5516
|
-
|
5517
|
-
|
5518
|
-
|
5519
|
-
|
5520
|
-
|
5521
|
-
|
5522
|
-
|
5523
|
-
|
5524
|
-
|
5525
|
-
|
5573
|
+
> **Note:** Add `ascli node` in front of the commands:
|
5574
|
+
|
5575
|
+
```bash
|
5576
|
+
--url=https://tst.example.com/path --password="Bearer bearer_666" --root-id=root_id access_key do self br /
|
5577
|
+
access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
|
5578
|
+
access_key delete my_username
|
5579
|
+
access_key do my_ak_name browse /
|
5580
|
+
access_key do my_ak_name delete /folder2
|
5581
|
+
access_key do my_ak_name delete testfile1
|
5582
|
+
access_key do my_ak_name download testfile1 --to-folder=.
|
5583
|
+
access_key do my_ak_name find my_test_folder
|
5584
|
+
access_key do my_ak_name find my_test_folder @ruby:'->(f){f["name"].end_with?(".jpg")}'
|
5585
|
+
access_key do my_ak_name find my_test_folder exec:'f["name"].end_with?(".jpg")'
|
5586
|
+
access_key do my_ak_name mkdir /folder1
|
5587
|
+
access_key do my_ak_name node_info /
|
5588
|
+
access_key do my_ak_name rename /folder1 folder2
|
5589
|
+
access_key do my_ak_name show %id:1
|
5590
|
+
access_key do my_ak_name show /testfile1
|
5591
|
+
access_key do my_ak_name upload 'faux:///testfile1?1k' --default_ports=no
|
5592
|
+
access_key do self permission %id:root_id create @json:'{"access_type":"user","access_id":"666"}'
|
5593
|
+
access_key do self show / --fields=id --output=root_id
|
5594
|
+
access_key list
|
5595
|
+
access_key set_bearer_key self @file:my_private_key
|
5596
|
+
access_key show %id:self
|
5597
|
+
api_details
|
5598
|
+
asperabrowser
|
5599
|
+
async bandwidth 1
|
5600
|
+
async counters 1
|
5601
|
+
async files 1
|
5602
|
+
async list
|
5603
|
+
async show 1
|
5604
|
+
async show ALL
|
5605
|
+
basic_token
|
5606
|
+
bearer_token @file:my_private_key @json:'{"user_id":"666"}' --output=bearer_666
|
5607
|
+
browse / --log-level=trace2
|
5608
|
+
central file list
|
5609
|
+
central file modify --validator=1 --query=@json:'{"files":[]}'
|
5610
|
+
central session list
|
5611
|
+
delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
|
5612
|
+
delete my_upload_folder/test_file.bin
|
5613
|
+
download my_upload_folder/test_file.bin --to-folder=.
|
5614
|
+
health
|
5615
|
+
http_node_download my_upload_folder/test_file.bin --to-folder=.
|
5616
|
+
info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
5617
|
+
license
|
5618
|
+
mkdir my_upload_folder/a_folder
|
5619
|
+
mkfile my_upload_folder/a_file1 "hello world"
|
5620
|
+
mklink my_upload_folder/a_folder my_upload_folder/tdlink
|
5621
|
+
rename my_upload_folder a_file1 a_file
|
5622
|
+
search / --query=@json:'{"sort":"mtime"}'
|
5623
|
+
service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
|
5624
|
+
service delete service1
|
5625
|
+
service list
|
5626
|
+
slash
|
5627
|
+
space /
|
5628
|
+
ssync bandwidth %name:my_node_sync
|
5629
|
+
ssync counters %name:my_node_sync
|
5630
|
+
ssync create @json:'{"configuration":{"name":"my_node_sync","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password_here","path":"my_remote_path"}}}'
|
5631
|
+
ssync delete %name:my_node_sync
|
5632
|
+
ssync files %name:my_node_sync
|
5633
|
+
ssync list
|
5634
|
+
ssync show %name:my_node_sync
|
5635
|
+
ssync start %name:my_node_sync
|
5636
|
+
ssync state %name:my_node_sync
|
5637
|
+
ssync stop %name:my_node_sync
|
5638
|
+
ssync summary %name:my_node_sync
|
5639
|
+
stream list
|
5640
|
+
sync admin status --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5641
|
+
sync admin status --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5642
|
+
sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5643
|
+
sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5644
|
+
transfer list --query=@json:'{"active_only":true}'
|
5645
|
+
transfer sessions
|
5646
|
+
upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
5647
|
+
upload --username=my_ak_name --password=my_ak_secret test_file.bin
|
5648
|
+
upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
|
5649
|
+
watch_folder list
|
5650
|
+
```
|
5651
|
+
|
5652
|
+
## Plugin: `faspex5`: IBM Aspera Faspex v5
|
5526
5653
|
|
5527
5654
|
IBM Aspera's newer self-managed application.
|
5528
5655
|
|
@@ -5537,7 +5664,7 @@ IBM Aspera's newer self-managed application.
|
|
5537
5664
|
|
5538
5665
|
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
5539
5666
|
|
5540
|
-
For a quick start, one can use the wizard, which will help creating a [
|
5667
|
+
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset)':
|
5541
5668
|
|
5542
5669
|
```bash
|
5543
5670
|
ascli config wizard
|
@@ -5610,7 +5737,7 @@ Activation is in two steps:
|
|
5610
5737
|
- Select `Account Settings`
|
5611
5738
|
- on the bottom in the text field: `Public key in PEM format` paste the **public** key corresponding to the private key used by the user.
|
5612
5739
|
|
5613
|
-
**Note:** If you don't have any refer to section [Private Key](#
|
5740
|
+
**Note:** If you don't have any refer to section [Private Key](#private-key)
|
5614
5741
|
|
5615
5742
|
Then use these options:
|
5616
5743
|
|
@@ -5670,7 +5797,60 @@ Use this token as password and use `--auth=boot`.
|
|
5670
5797
|
ascli config preset update f5boot --url=https://localhost/aspera/faspex --auth=boot --password=_token_here_
|
5671
5798
|
```
|
5672
5799
|
|
5673
|
-
###
|
5800
|
+
### Faspex5 sample commands
|
5801
|
+
|
5802
|
+
> **Note:** Add `ascli faspex5` in front of the commands:
|
5803
|
+
|
5804
|
+
```bash
|
5805
|
+
admin res accounts list
|
5806
|
+
admin res contacts list
|
5807
|
+
admin res jobs list
|
5808
|
+
admin res metadata_profiles list
|
5809
|
+
admin res node list
|
5810
|
+
admin res oauth_clients list
|
5811
|
+
admin res registrations list
|
5812
|
+
admin res saml_configs list
|
5813
|
+
admin res shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
5814
|
+
admin res shared_inboxes list
|
5815
|
+
admin res shared_inboxes list --query=@json:'{"all":true}'
|
5816
|
+
admin res shared_inboxes members %name:my_shared_box_name create %name:john@example.com
|
5817
|
+
admin res shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
|
5818
|
+
admin res shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
|
5819
|
+
admin res shared_inboxes members %name:my_shared_box_name list
|
5820
|
+
admin res workgroups list
|
5821
|
+
admin smtp show
|
5822
|
+
admin smtp test my_email_external
|
5823
|
+
bearer_token
|
5824
|
+
gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
5825
|
+
health
|
5826
|
+
invitation list
|
5827
|
+
invitations create @json:'{"email_address":"aspera.user1+u@gmail.com"}'
|
5828
|
+
packages list --box=my_shared_box_name
|
5829
|
+
packages list --box=my_workgroup --group-type=workgroups
|
5830
|
+
packages list --query=@json:'{"mailbox":"inbox","status":"completed"}'
|
5831
|
+
packages receive --box=my_shared_box_name package_box_id1 --to-folder=.
|
5832
|
+
packages receive --box=my_workgroup --group-type=workgroups workgroup_package_id1 --to-folder=.
|
5833
|
+
packages receive ALL --once-only=yes --to-folder=.
|
5834
|
+
packages receive INIT --once-only=yes
|
5835
|
+
packages receive f5_p31 --to-folder=. --ts=@json:'{"content_protection_password":"my_secret_here"}'
|
5836
|
+
packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file
|
5837
|
+
packages send --url=my_public_link_send_f5_user @json:'{"title":"test title"}' test_file.bin
|
5838
|
+
packages send --url=my_public_link_send_shared_box @json:'{"title":"test title"}' test_file.bin
|
5839
|
+
packages send @json:'{"title":"test title","recipients":["my_shared_box_name"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' test_file.bin
|
5840
|
+
packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' test_file.bin
|
5841
|
+
packages send @json:'{"title":"test title","recipients":[{"name":"my_username"}]my_meta}' test_file.bin --ts=@json:'{"content_protection_password":"my_secret_here"}'
|
5842
|
+
packages show --box=my_shared_box_name package_box_id1
|
5843
|
+
packages show --box=my_workgroup --group-type=workgroups workgroup_package_id1
|
5844
|
+
packages show f5_p31
|
5845
|
+
packages status f5_p31
|
5846
|
+
postprocessing --pid-file=pid_f5_postproc @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":""}}' &
|
5847
|
+
shared browse %name:my_src
|
5848
|
+
shared list
|
5849
|
+
shared_folders browse %name:my_shared_folder_name
|
5850
|
+
shared_folders list
|
5851
|
+
user profile modify @json:'{"preference":{"connect_disabled":false}}'
|
5852
|
+
user profile show
|
5853
|
+
```
|
5674
5854
|
|
5675
5855
|
Most commands are directly REST API calls.
|
5676
5856
|
Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional argument for creation.
|
@@ -5678,53 +5858,6 @@ One can conveniently use the JSON format with prefix `@json:`.
|
|
5678
5858
|
|
5679
5859
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
5680
5860
|
|
5681
|
-
```bash
|
5682
|
-
faspex5 admin res accounts list
|
5683
|
-
faspex5 admin res contacts list
|
5684
|
-
faspex5 admin res jobs list
|
5685
|
-
faspex5 admin res metadata_profiles list
|
5686
|
-
faspex5 admin res node list
|
5687
|
-
faspex5 admin res oauth_clients list
|
5688
|
-
faspex5 admin res registrations list
|
5689
|
-
faspex5 admin res saml_configs list
|
5690
|
-
faspex5 admin res shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
5691
|
-
faspex5 admin res shared_inboxes list
|
5692
|
-
faspex5 admin res shared_inboxes list --query=@json:'{"all":true}'
|
5693
|
-
faspex5 admin res shared_inboxes members %name:my_shared_box_name create %name:john@example.com
|
5694
|
-
faspex5 admin res shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
|
5695
|
-
faspex5 admin res shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
|
5696
|
-
faspex5 admin res shared_inboxes members %name:my_shared_box_name list
|
5697
|
-
faspex5 admin res workgroups list
|
5698
|
-
faspex5 admin smtp show
|
5699
|
-
faspex5 admin smtp test my_email_external
|
5700
|
-
faspex5 bearer_token
|
5701
|
-
faspex5 gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
5702
|
-
faspex5 health
|
5703
|
-
faspex5 packages list --box=my_shared_box_name
|
5704
|
-
faspex5 packages list --box=my_workgroup --group-type=workgroups
|
5705
|
-
faspex5 packages list --query=@json:'{"mailbox":"inbox","state":["released"]}'
|
5706
|
-
faspex5 packages receive --box=my_shared_box_name package_box_id1 --to-folder=.
|
5707
|
-
faspex5 packages receive --box=my_workgroup --group-type=workgroups workgroup_package_id1 --to-folder=.
|
5708
|
-
faspex5 packages receive ALL --once-only=yes --to-folder=.
|
5709
|
-
faspex5 packages receive INIT --once-only=yes
|
5710
|
-
faspex5 packages receive f5_p31 --to-folder=. --ts=@json:'{"content_protection_password":"my_secret_here"}'
|
5711
|
-
faspex5 packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file
|
5712
|
-
faspex5 packages send @json:'{"title":"test title","recipients":["my_shared_box_name"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' test_file.bin
|
5713
|
-
faspex5 packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' test_file.bin
|
5714
|
-
faspex5 packages send @json:'{"title":"test title","recipients":[{"name":"my_username"}]my_meta}' test_file.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
|
5715
|
-
faspex5 packages show --box=my_shared_box_name package_box_id1
|
5716
|
-
faspex5 packages show --box=my_workgroup --group-type=workgroups workgroup_package_id1
|
5717
|
-
faspex5 packages show f5_p31
|
5718
|
-
faspex5 packages status f5_p31
|
5719
|
-
faspex5 postprocessing --pid-file=pid_f5_postproc @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":""}}' &
|
5720
|
-
faspex5 shared browse %name:my_src
|
5721
|
-
faspex5 shared list
|
5722
|
-
faspex5 shared_folders browse %name:my_shared_folder_name
|
5723
|
-
faspex5 shared_folders list
|
5724
|
-
faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
|
5725
|
-
faspex5 user profile show
|
5726
|
-
```
|
5727
|
-
|
5728
5861
|
### Faspex 5: Inbox selection
|
5729
5862
|
|
5730
5863
|
By default, package operations (send, receive, list) are done on the user's inbox.
|
@@ -5829,6 +5962,20 @@ Examples:
|
|
5829
5962
|
ascli faspex5 packages list @ruby:'->(p){p["state"].eql?("released")}'
|
5830
5963
|
```
|
5831
5964
|
|
5965
|
+
### Faspex 5: Content of a received Package
|
5966
|
+
|
5967
|
+
> **Note:** Listing content also applies to shares folder, nodes, and received packages (using `--box=outbox`).
|
5968
|
+
|
5969
|
+
To list the content of a package, use command `faspex5 packages browse /`.
|
5970
|
+
|
5971
|
+
Option `query` is available.
|
5972
|
+
|
5973
|
+
To list recursively add option `--query=@json:{"recursive":true}`.
|
5974
|
+
|
5975
|
+
> **Note:** Option `recirsive` makes recursive API calls, so it can take a long time on large packages.
|
5976
|
+
|
5977
|
+
To limit the number of items retrieved, use option `--query=@json:{"max":10}`.
|
5978
|
+
|
5832
5979
|
### Faspex 5: Receive a package
|
5833
5980
|
|
5834
5981
|
To receive one, or several packages at once, use command `faspex5 packages receive`.
|
@@ -5930,6 +6077,14 @@ To initialize, and skip all current package so that next time `ALL` is used, onl
|
|
5930
6077
|
ascli faspex5 packages receive INIT --once-only=yes
|
5931
6078
|
```
|
5932
6079
|
|
6080
|
+
### Faspex 5: Invitations
|
6081
|
+
|
6082
|
+
There are two types of invitations of package submission: public or private.
|
6083
|
+
|
6084
|
+
Public invitations are for external users, provide just the email address.
|
6085
|
+
|
6086
|
+
Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
|
6087
|
+
|
5933
6088
|
### Faspex 5: Faspex 4-style postprocessing
|
5934
6089
|
|
5935
6090
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
@@ -5945,8 +6100,7 @@ The following parameters are supported:
|
|
5945
6100
|
|
5946
6101
|
| parameter | type | default | description |
|
5947
6102
|
|----------------------------|---------|------------------------|-----------------------------------------------------|
|
5948
|
-
<!-- markdownlint-disable-
|
5949
|
-
| url | string | http://localhost:8080 | Base url on which requests are listened |
|
6103
|
+
| url | string | http://localhost:8080 | Base url on which requests are listened | <!-- markdownlint-disable-line -->
|
5950
6104
|
| certificate | hash | nil | Certificate information (if HTTPS) |
|
5951
6105
|
| certificate.key | string | nil | Path to private key file |
|
5952
6106
|
| certificate.cert | string | nil | Path to certificate |
|
@@ -5978,12 +6132,12 @@ Then, the postprocessing script executed will be `script1.sh`.
|
|
5978
6132
|
|
5979
6133
|
Environment variables at set to the values provided by the web hook which are the same as Faspex 4 postprocessing.
|
5980
6134
|
|
5981
|
-
##
|
6135
|
+
## Plugin: `faspex`: IBM Aspera Faspex v4
|
5982
6136
|
|
5983
|
-
|
6137
|
+
> **Note:** For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
|
5984
6138
|
|
5985
|
-
|
5986
|
-
|
6139
|
+
This plugin uses APIs versions 3 Faspex v4.
|
6140
|
+
The `v4` command requires the use of API v4, refer to the Faspex Admin manual on how to activate.
|
5987
6141
|
|
5988
6142
|
### Listing Packages
|
5989
6143
|
|
@@ -6130,49 +6284,51 @@ The node configuration name is `my_faspex_node` here.
|
|
6130
6284
|
### Automated package download (cargo)
|
6131
6285
|
|
6132
6286
|
It is possible to tell `ascli` to download newly received packages, much like the official
|
6133
|
-
cargo client, or drive. Refer to the [same section](#
|
6287
|
+
cargo client, or drive. Refer to the [same section](#receive-new-packages-only-cargo) in the Aspera on Cloud plugin:
|
6134
6288
|
|
6135
6289
|
```bash
|
6136
6290
|
ascli faspex packages recv ALL --once-only=yes --lock-port=12345
|
6137
6291
|
```
|
6138
6292
|
|
6139
|
-
### Faspex
|
6140
|
-
|
6141
|
-
|
6142
|
-
|
6143
|
-
|
6144
|
-
|
6145
|
-
|
6146
|
-
|
6147
|
-
|
6148
|
-
|
6149
|
-
|
6150
|
-
|
6151
|
-
|
6152
|
-
|
6153
|
-
|
6154
|
-
|
6155
|
-
|
6156
|
-
|
6157
|
-
|
6158
|
-
|
6159
|
-
|
6160
|
-
|
6161
|
-
|
6162
|
-
|
6163
|
-
|
6164
|
-
|
6165
|
-
|
6166
|
-
|
6167
|
-
|
6168
|
-
|
6169
|
-
|
6170
|
-
|
6171
|
-
|
6172
|
-
|
6173
|
-
|
6174
|
-
|
6175
|
-
|
6293
|
+
### Faspex sample commands
|
6294
|
+
|
6295
|
+
> **Note:** Add `ascli faspex` in front of the commands:
|
6296
|
+
|
6297
|
+
```bash
|
6298
|
+
address_book
|
6299
|
+
dropbox list --recipient="*my_dbx"
|
6300
|
+
health
|
6301
|
+
login_methods
|
6302
|
+
me
|
6303
|
+
package list --box=sent --query=@json:'{"max":1}' --fields=package_id --display=data --format=csv --output=f4_prs2
|
6304
|
+
package list --query=@json:'{"max":1}' --fields=package_id --display=data --format=csv --output=f4_prs1
|
6305
|
+
package list --query=@json:'{"max":5}'
|
6306
|
+
package list --recipient="*my_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}' --output=f4_db_id1
|
6307
|
+
package list --recipient="*my_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}' --output=f4_db_id2
|
6308
|
+
package receive --to-folder=. --link=https://app.example.com/recv_from_user_path
|
6309
|
+
package receive ALL --once-only=yes --to-folder=. --query=@json:'{"max":10}'
|
6310
|
+
package receive f4_db_id1 --recipient="*my_dbx" --to-folder=.
|
6311
|
+
package receive f4_db_id2 --recipient="*my_wkg" --to-folder=.
|
6312
|
+
package receive f4_pri1 --to-folder=.
|
6313
|
+
package receive f4_prs2 --to-folder=. --box=sent
|
6314
|
+
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_dbx"]}' test_file.bin
|
6315
|
+
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_wkg"]}' test_file.bin
|
6316
|
+
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal","my_username"]}' test_file.bin
|
6317
|
+
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"]}' --remote_source=%name:my_src sample_source.txt
|
6318
|
+
package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
|
6319
|
+
package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
|
6320
|
+
source info %name:my_src --storage=@preset:faspex4_storage
|
6321
|
+
source list
|
6322
|
+
source node %name:my_src br / --storage=@preset:faspex4_storage
|
6323
|
+
v4 dmembership list
|
6324
|
+
v4 dropbox list
|
6325
|
+
v4 metadata_profile list
|
6326
|
+
v4 user list
|
6327
|
+
v4 wmembership list
|
6328
|
+
v4 workgroup list
|
6329
|
+
```
|
6330
|
+
|
6331
|
+
## Plugin: `shares`: IBM Aspera Shares v1
|
6176
6332
|
|
6177
6333
|
Aspera Shares supports the **node API** for the file transfer part.
|
6178
6334
|
|
@@ -6192,69 +6348,77 @@ user_id=$(ascli shares admin user list --select=@json:'{"username":"entity1"}' -
|
|
6192
6348
|
ascli shares admin share user_permissions $share_id create @json:'{"user_id":'$user_id',"browse_permission":true, "download_permission":true, "mkdir_permission":true,"delete_permission":true,"rename_permission":true,"content_availability_permission":true,"manage_permission":true}'
|
6193
6349
|
```
|
6194
6350
|
|
6195
|
-
|
6351
|
+
To figure out the entities payload, for example for creation, refer to the API documentation above.
|
6352
|
+
|
6353
|
+
### Shares sample commands
|
6354
|
+
|
6355
|
+
> **Note:** Add `ascli shares` in front of the commands:
|
6196
6356
|
|
6197
6357
|
```bash
|
6198
|
-
|
6199
|
-
|
6200
|
-
|
6201
|
-
|
6202
|
-
|
6203
|
-
|
6204
|
-
|
6205
|
-
|
6206
|
-
|
6207
|
-
|
6208
|
-
|
6209
|
-
|
6210
|
-
|
6211
|
-
|
6212
|
-
|
6213
|
-
|
6214
|
-
|
6215
|
-
|
6216
|
-
|
6217
|
-
|
6358
|
+
admin group all list
|
6359
|
+
admin node list
|
6360
|
+
admin share list --fields=DEF,-status,status_message
|
6361
|
+
admin share user_permissions 1 list
|
6362
|
+
admin user all app_authorizations 1 modify @json:'{"app_login":true}'
|
6363
|
+
admin user all app_authorizations 1 show
|
6364
|
+
admin user all list
|
6365
|
+
admin user all share_permissions 1 list
|
6366
|
+
admin user all share_permissions 1 show 1
|
6367
|
+
admin user ldap add the_name
|
6368
|
+
admin user local list
|
6369
|
+
admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
|
6370
|
+
files browse /
|
6371
|
+
files delete my_share1/test_file.bin
|
6372
|
+
files download --to-folder=. my_share1/test_file.bin
|
6373
|
+
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
|
6374
|
+
files upload --to-folder=my_share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6375
|
+
files upload --to-folder=my_share1 test_file.bin
|
6376
|
+
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
|
6377
|
+
health
|
6218
6378
|
```
|
6219
6379
|
|
6220
|
-
##
|
6380
|
+
## Plugin: `console`: IBM Aspera Console
|
6221
6381
|
|
6222
6382
|
### Console sample commands
|
6223
6383
|
|
6384
|
+
> **Note:** Add `ascli console` in front of the commands:
|
6385
|
+
|
6224
6386
|
```bash
|
6225
|
-
|
6226
|
-
|
6227
|
-
|
6228
|
-
|
6387
|
+
health
|
6388
|
+
transfer current list
|
6389
|
+
transfer smart list
|
6390
|
+
transfer smart sub my_smart_id @json:'{"source":{"paths":["my_smart_file"]},"source_type":"user_selected"}'
|
6229
6391
|
```
|
6230
6392
|
|
6231
|
-
##
|
6393
|
+
## Plugin: `orchestrator`:IBM Aspera Orchestrator
|
6232
6394
|
|
6233
6395
|
### Orchestrator sample commands
|
6234
6396
|
|
6397
|
+
> **Note:** Add `ascli orchestrator` in front of the commands:
|
6398
|
+
|
6235
6399
|
```bash
|
6236
|
-
|
6237
|
-
|
6238
|
-
|
6239
|
-
|
6240
|
-
|
6241
|
-
|
6242
|
-
|
6243
|
-
|
6244
|
-
|
6245
|
-
|
6246
|
-
|
6247
|
-
|
6400
|
+
health
|
6401
|
+
info
|
6402
|
+
plugins
|
6403
|
+
processes
|
6404
|
+
workflow details my_workflow_id
|
6405
|
+
workflow export my_workflow_id
|
6406
|
+
workflow inputs my_workflow_id
|
6407
|
+
workflow list
|
6408
|
+
workflow start my_workflow_id @json:'{"Param":"world !"}'
|
6409
|
+
workflow start my_workflow_id @json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
|
6410
|
+
workflow status ALL
|
6411
|
+
workflow status my_workflow_id
|
6248
6412
|
```
|
6249
6413
|
|
6250
|
-
##
|
6414
|
+
## Plugin: `cos`: IBM Cloud Object Storage
|
6251
6415
|
|
6252
6416
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
6253
6417
|
It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
|
6254
6418
|
Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
|
6255
6419
|
|
6256
6420
|
There are two possibilities to provide credentials.
|
6257
|
-
If you already have the endpoint, API key and CRN, use the first method.
|
6421
|
+
If you already have the endpoint, API key and Resource Instance ID (CRN), use the first method.
|
6258
6422
|
If you don't have credentials but have access to the IBM Cloud console, then use the second method.
|
6259
6423
|
|
6260
6424
|
### Using endpoint, API key and Resource Instance ID (CRN)
|
@@ -6322,12 +6486,12 @@ The field `resource_instance_id` is for option `crn`
|
|
6322
6486
|
|
6323
6487
|
The field `apikey` is for option `apikey`
|
6324
6488
|
|
6325
|
-
|
6489
|
+
> **Note:** endpoints for regions can be found by querying the `endpoints` URL from file or from the IBM Cloud Console.
|
6326
6490
|
|
6327
6491
|
The required options for this method are:
|
6328
6492
|
|
6329
6493
|
- `bucket` bucket name
|
6330
|
-
- `region` bucket region, e.g. eu-de
|
6494
|
+
- `region` bucket region, e.g. `eu-de`
|
6331
6495
|
- `service_credentials` see below
|
6332
6496
|
|
6333
6497
|
For example, let us create a default configuration:
|
@@ -6339,7 +6503,7 @@ ascli config preset set default cos mycos
|
|
6339
6503
|
|
6340
6504
|
### Operations, transfers
|
6341
6505
|
|
6342
|
-
Let's assume you created a default configuration from
|
6506
|
+
Let's assume you created a default configuration from one of the two previous steps (else specify the access options on command lines).
|
6343
6507
|
|
6344
6508
|
A subset of `node` plugin operations are supported, basically node API:
|
6345
6509
|
|
@@ -6348,20 +6512,22 @@ ascli cos node info
|
|
6348
6512
|
ascli cos node upload 'faux:///sample1G?1g'
|
6349
6513
|
```
|
6350
6514
|
|
6351
|
-
> **Note:**
|
6515
|
+
> **Note:** A dummy file `sample1G` of size 2GB is generated using the `faux` PVCL (man `ascp` and section above), but you can, of course, send a real file by specifying a real file instead.
|
6352
6516
|
|
6353
|
-
###
|
6517
|
+
### Cos sample commands
|
6518
|
+
|
6519
|
+
> **Note:** Add `ascli cos` in front of the commands:
|
6354
6520
|
|
6355
6521
|
```bash
|
6356
|
-
|
6357
|
-
|
6358
|
-
|
6359
|
-
|
6360
|
-
|
6361
|
-
|
6522
|
+
node access_key show self
|
6523
|
+
node download test_file.bin --to-folder=.
|
6524
|
+
node info --bucket=my_bucket --endpoint=my_endpoint --apikey=my_api_key --crn=my_resource_instance_id
|
6525
|
+
node info --bucket=my_bucket --region=my_region --service-credentials=@json:@file:my_cos_svc_cred
|
6526
|
+
node info --log-level=trace2
|
6527
|
+
node upload test_file.bin
|
6362
6528
|
```
|
6363
6529
|
|
6364
|
-
##
|
6530
|
+
## Plugin: `preview`: Preview generator for AoC
|
6365
6531
|
|
6366
6532
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
6367
6533
|
It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
@@ -6409,7 +6575,7 @@ If you use a value different than 16777216, then specify it using option `max_si
|
|
6409
6575
|
|
6410
6576
|
> **Note:** The HSTS parameter (`max_request_file_create_size_kb`) is in **kiloBytes** while the generator parameter is in **Bytes** (factor of 1024).
|
6411
6577
|
|
6412
|
-
###
|
6578
|
+
### External tools: Linux
|
6413
6579
|
|
6414
6580
|
`ascli` requires the following external tools available in the `PATH`:
|
6415
6581
|
|
@@ -6476,7 +6642,7 @@ chmod a+x /usr/bin/unoconv
|
|
6476
6642
|
The preview generator should be executed as a non-user.
|
6477
6643
|
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
6478
6644
|
|
6479
|
-
Like any `ascli` commands, parameters can be passed on command line or using a configuration [
|
6645
|
+
Like any `ascli` commands, parameters can be passed on command line or using a configuration [Option Preset](#option-preset)'.
|
6480
6646
|
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
6481
6647
|
|
6482
6648
|
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
@@ -6654,23 +6820,26 @@ If the preview generator does not have access to files on the file system (it is
|
|
6654
6820
|
|
6655
6821
|
### Preview sample commands
|
6656
6822
|
|
6823
|
+
> **Note:** Add `ascli preview` in front of the commands:
|
6824
|
+
|
6657
6825
|
```bash
|
6658
|
-
|
6659
|
-
|
6660
|
-
|
6661
|
-
|
6662
|
-
|
6663
|
-
|
6664
|
-
|
6665
|
-
|
6666
|
-
|
6667
|
-
|
6668
|
-
|
6669
|
-
|
6670
|
-
|
6826
|
+
check --skip-types=office
|
6827
|
+
scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
6828
|
+
scan --skip-types=office --log-level=info
|
6829
|
+
show --base=test my_docx
|
6830
|
+
show --base=test my_mpg --video-png-conv=animated
|
6831
|
+
show --base=test my_mpg --video-png-conv=fixed
|
6832
|
+
show --base=test my_mpg mp4 --video-conversion=clips
|
6833
|
+
show --base=test my_mpg mp4 --video-conversion=reencode
|
6834
|
+
show --base=test my_pdf
|
6835
|
+
test --base=test my_dcm
|
6836
|
+
test --base=test my_mxf mp4 --video-conversion=blend --query=@json:'{"text":true,"double":true}'
|
6837
|
+
test --mimemagic=yes --base=test my_dcm
|
6838
|
+
test --mimemagic=yes --base=test my_jpg_unk
|
6839
|
+
trevents --once-only=yes --skip-types=office --log-level=info
|
6671
6840
|
```
|
6672
6841
|
|
6673
|
-
##
|
6842
|
+
## IBM Aspera Sync
|
6674
6843
|
|
6675
6844
|
An interface for the `async` utility is provided in the following plugins:
|
6676
6845
|
|
@@ -6733,11 +6902,11 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
6733
6902
|
- `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
|
6734
6903
|
- `--src-base` (`src_base`) if top level folder name shall not be created on destination
|
6735
6904
|
|
6736
|
-
> **Note:** `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#
|
6905
|
+
> **Note:** `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transfer-specification), with `ts` option.
|
6737
6906
|
>
|
6738
|
-
> **Note:** Most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#
|
6907
|
+
> **Note:** Most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transfer-specification) parameters.
|
6739
6908
|
>
|
6740
|
-
> **Note:** Only for the [`direct`](#
|
6909
|
+
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
6741
6910
|
|
6742
6911
|
#### Server side and configuration
|
6743
6912
|
|
@@ -6745,7 +6914,7 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
6745
6914
|
|
6746
6915
|
> **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
6747
6916
|
>
|
6748
|
-
> **Note:** Parameters may be saved in a [
|
6917
|
+
> **Note:** Parameters may be saved in a [Option Preset](#option-preset)' and used with `-P`.
|
6749
6918
|
|
6750
6919
|
#### Scheduling
|
6751
6920
|
|
@@ -6912,7 +7081,7 @@ Transfer is: <%=global_transfer_status%>
|
|
6912
7081
|
|
6913
7082
|
This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
|
6914
7083
|
|
6915
|
-
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#
|
7084
|
+
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transfer-specification) is:
|
6916
7085
|
|
6917
7086
|
- Common to Aspera Node API (HTTP POST /ops/transfer)
|
6918
7087
|
- Common to Aspera Connect API (browser javascript startTransfer)
|
@@ -6922,20 +7091,22 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
|
|
6922
7091
|
|
6923
7092
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
6924
7093
|
|
6925
|
-
`ascli` expect one single argument: a [*transfer-spec*](#
|
7094
|
+
`ascli` expect one single argument: a session specification that contains parameters and a [*transfer-spec*](#transfer-specification).
|
6926
7095
|
|
6927
|
-
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted
|
7096
|
+
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted on stdin.
|
6928
7097
|
|
6929
7098
|
> **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the `Hash` using JSON syntax.
|
6930
7099
|
|
6931
7100
|
During execution, it generates all low level events, one per line, in JSON format on stdout.
|
6932
7101
|
|
6933
|
-
|
6934
|
-
|
6935
|
-
- `EX_loglevel` to change log level of `ascli`
|
6936
|
-
- `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
7102
|
+
Top level parameters supported by `asession`:
|
6937
7103
|
|
6938
|
-
|
7104
|
+
| parameter | description |
|
7105
|
+
|-----------|-------------|
|
7106
|
+
| `spec` | the [*transfer-spec*](#transfer-specification) |
|
7107
|
+
| `agent` | same parameters as transfer-info for agent `direct` |
|
7108
|
+
| `loglevel` | log level of `asession` |
|
7109
|
+
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
|
6939
7110
|
|
6940
7111
|
### Comparison of interfaces
|
6941
7112
|
|
@@ -6965,7 +7136,7 @@ asession < session.json
|
|
6965
7136
|
|
6966
7137
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` -> `long_parameter`
|
6967
7138
|
|
6968
|
-
This is particularly useful for a persistent session ( with the [*transfer-spec*](#
|
7139
|
+
This is particularly useful for a persistent session ( with the [*transfer-spec*](#transfer-specification) parameter: `"keepalive":true` )
|
6969
7140
|
|
6970
7141
|
```json
|
6971
7142
|
asession
|
@@ -6987,23 +7158,24 @@ asession -h
|
|
6987
7158
|
USAGE
|
6988
7159
|
asession
|
6989
7160
|
asession -h|--help
|
6990
|
-
asession <
|
7161
|
+
asession <session spec extended value>
|
6991
7162
|
|
6992
7163
|
If no argument is provided, default will be used: @json:@stdin
|
6993
7164
|
-h, --help display this message
|
6994
|
-
<
|
7165
|
+
<session spec extended value> a dictionary value (Hash)
|
6995
7166
|
The value can be either:
|
6996
7167
|
the JSON description itself, e.g. @json:'{"xx":"yy",...}'
|
6997
7168
|
@json:@stdin, if the JSON is provided from stdin
|
6998
7169
|
@json:@file:<path>, if the JSON is provided from a file
|
7170
|
+
Parameter spec is mandatory, it contains the transfer spec
|
6999
7171
|
Asynchronous commands can be provided on STDIN, examples:
|
7000
7172
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
7001
7173
|
{"type":"START","source":"xx","destination":"yy"}
|
7002
7174
|
{"type":"DONE"}
|
7003
|
-
Note: debug information can be placed on STDERR, using the "
|
7175
|
+
Note: debug information can be placed on STDERR, using the "loglevel" parameter in session spec (debug=0)
|
7004
7176
|
EXAMPLES
|
7005
|
-
asession @json:'{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}'
|
7006
|
-
echo '{"remote_host":...}'|asession @json:@stdin
|
7177
|
+
asession @json:'{"spec":{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}}'
|
7178
|
+
echo '{"spec":{"remote_host":...}}'|asession @json:@stdin
|
7007
7179
|
|
7008
7180
|
```
|
7009
7181
|
|
@@ -7012,7 +7184,7 @@ EXAMPLES
|
|
7012
7184
|
Main components:
|
7013
7185
|
|
7014
7186
|
- `Aspera` generic classes for REST and OAuth
|
7015
|
-
- `Aspera::
|
7187
|
+
- `Aspera::Agent::Direct`: starting and monitoring transfers. It can be considered as a FASPManager class for Ruby.
|
7016
7188
|
- `Aspera::Cli`: `ascli`.
|
7017
7189
|
|
7018
7190
|
Working examples can be found in repo: <https://github.com/laurent-martin/aspera-api-examples> in Ruby examples.
|
@@ -7033,7 +7205,7 @@ So, it evolved into `ascli`:
|
|
7033
7205
|
|
7034
7206
|
- Portable: works on platforms supporting `ruby` (and `ascp`)
|
7035
7207
|
- Easy to install with the `gem` utility
|
7036
|
-
- Supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from `ascp` command line to [*transfer-spec*](#
|
7208
|
+
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [*transfer-spec*](#transfer-specification) (more reliable , more standard)
|
7037
7209
|
- `ruby` is consistent with other Aspera products
|
7038
7210
|
|
7039
7211
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|