aspera-cli 4.18.1 → 4.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +33 -0
- data/CONTRIBUTING.md +17 -12
- data/README.md +396 -185
- data/bin/asession +26 -19
- data/examples/build_exec +74 -0
- data/examples/{rubyc → build_exec_rubyc} +18 -2
- data/examples/get_proto_file.rb +7 -0
- data/lib/aspera/agent/alpha.rb +8 -8
- data/lib/aspera/agent/base.rb +4 -18
- data/lib/aspera/agent/connect.rb +14 -13
- data/lib/aspera/agent/direct.rb +123 -120
- data/lib/aspera/agent/httpgw.rb +2 -3
- data/lib/aspera/agent/node.rb +10 -10
- data/lib/aspera/agent/trsdk.rb +17 -20
- data/lib/aspera/api/alee.rb +15 -0
- data/lib/aspera/api/aoc.rb +128 -99
- data/lib/aspera/api/ats.rb +1 -1
- data/lib/aspera/api/cos_node.rb +1 -1
- data/lib/aspera/api/httpgw.rb +104 -64
- data/lib/aspera/api/node.rb +33 -12
- data/lib/aspera/ascmd.rb +56 -48
- data/lib/aspera/ascp/installation.rb +142 -70
- data/lib/aspera/ascp/management.rb +7 -3
- data/lib/aspera/ascp/products.rb +13 -7
- data/lib/aspera/assert.rb +10 -5
- data/lib/aspera/cli/formatter.rb +42 -26
- data/lib/aspera/cli/hints.rb +2 -1
- data/lib/aspera/cli/info.rb +12 -10
- data/lib/aspera/cli/main.rb +16 -13
- data/lib/aspera/cli/manager.rb +15 -10
- data/lib/aspera/cli/plugin.rb +17 -31
- data/lib/aspera/cli/plugin_factory.rb +10 -1
- data/lib/aspera/cli/plugins/alee.rb +3 -3
- data/lib/aspera/cli/plugins/aoc.rb +222 -194
- data/lib/aspera/cli/plugins/ats.rb +16 -14
- data/lib/aspera/cli/plugins/config.rb +66 -53
- data/lib/aspera/cli/plugins/console.rb +3 -3
- data/lib/aspera/cli/plugins/faspex.rb +11 -21
- data/lib/aspera/cli/plugins/faspex5.rb +44 -42
- data/lib/aspera/cli/plugins/faspio.rb +2 -2
- data/lib/aspera/cli/plugins/httpgw.rb +1 -1
- data/lib/aspera/cli/plugins/node.rb +155 -96
- data/lib/aspera/cli/plugins/orchestrator.rb +14 -13
- data/lib/aspera/cli/plugins/preview.rb +8 -9
- data/lib/aspera/cli/plugins/server.rb +6 -10
- data/lib/aspera/cli/plugins/shares.rb +13 -9
- data/lib/aspera/cli/sync_actions.rb +72 -31
- data/lib/aspera/cli/transfer_agent.rb +13 -14
- data/lib/aspera/cli/transfer_progress.rb +36 -18
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/command_line_builder.rb +3 -4
- data/lib/aspera/coverage.rb +13 -1
- data/lib/aspera/environment.rb +59 -10
- data/lib/aspera/faspex_gw.rb +3 -3
- data/lib/aspera/json_rpc.rb +1 -1
- data/lib/aspera/keychain/encrypted_hash.rb +2 -0
- data/lib/aspera/keychain/macos_security.rb +7 -12
- data/lib/aspera/log.rb +4 -4
- data/lib/aspera/node_simulator.rb +1 -1
- data/lib/aspera/oauth/base.rb +39 -45
- data/lib/aspera/oauth/factory.rb +11 -4
- data/lib/aspera/oauth/generic.rb +4 -8
- data/lib/aspera/oauth/jwt.rb +4 -4
- data/lib/aspera/oauth/url_json.rb +3 -2
- data/lib/aspera/oauth/web.rb +10 -6
- data/lib/aspera/persistency_action_once.rb +16 -8
- data/lib/aspera/preview/utils.rb +5 -16
- data/lib/aspera/rest.rb +100 -76
- data/lib/aspera/secret_hider.rb +3 -2
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/transfer/faux_file.rb +7 -5
- data/lib/aspera/transfer/parameters.rb +41 -35
- data/lib/aspera/transfer/spec.rb +16 -18
- data/lib/aspera/transfer/sync.rb +51 -50
- data/lib/aspera/transfer/uri.rb +1 -1
- data/lib/aspera/uri_reader.rb +1 -1
- data/lib/aspera/web_auth.rb +166 -18
- data/lib/aspera/web_server_simple.rb +27 -15
- data/lib/transfer_pb.rb +84 -0
- data/lib/transfer_services_pb.rb +82 -0
- data.tar.gz.sig +0 -0
- metadata +25 -6
- metadata.gz.sig +0 -0
data/README.md
CHANGED
@@ -1,8 +1,13 @@
|
|
1
1
|
# Command Line Interface for IBM Aspera products
|
2
|
-
<!--
|
3
|
-
|
2
|
+
<!--
|
3
|
+
Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md
|
4
4
|
|
5
|
-
|
5
|
+
markdownlint-disable MD033 MD003 MD053
|
6
|
+
cspell:ignore Serban Antipolis
|
7
|
+
PANDOC_META_BEGIN
|
8
|
+
subtitle: "ascli 4.20.0"
|
9
|
+
PANDOC_META_END
|
10
|
+
-->
|
6
11
|
|
7
12
|
[](https://badge.fury.io/rb/aspera-cli)
|
8
13
|
[](https://github.com/IBM/aspera-cli/actions)
|
@@ -10,9 +15,9 @@
|
|
10
15
|
|
11
16
|
## Introduction
|
12
17
|
|
13
|
-
Version : 4.
|
18
|
+
Version : 4.20.0
|
14
19
|
|
15
|
-
Laurent/2016-
|
20
|
+
Laurent/2016-2025
|
16
21
|
|
17
22
|
This gem provides the `ascli` CLI (Command Line Interface) to IBM Aspera software.
|
18
23
|
|
@@ -37,6 +42,11 @@ A PDF version of this documentation is available here: [docs/Manual.pdf](docs/Ma
|
|
37
42
|
|
38
43
|
Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
39
44
|
|
45
|
+
This documentation does not provide ALL the detailed description of all options and commands.
|
46
|
+
The reason is that most commands and payloads are directly Rest API calls on the various Aspera products.
|
47
|
+
So, to get the full description of all options and commands, refer to the official Aspera API documentation.
|
48
|
+
(To see which API is used, set option `--log-level=debug`)
|
49
|
+
|
40
50
|
### When to use and when not to use
|
41
51
|
|
42
52
|
`ascli` is designed to be used as a command line tool to:
|
@@ -59,13 +69,23 @@ It is designed for:
|
|
59
69
|
If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
|
60
70
|
|
61
71
|
- Product APIs (REST) : e.g. AoC, Faspex, node
|
62
|
-
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Ruby, etc...)
|
72
|
+
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Go, Ruby, Rust, etc...)
|
63
73
|
|
64
74
|
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
65
75
|
Code examples here: <https://github.com/laurent-martin/aspera-api-examples>
|
66
76
|
|
67
77
|
For scripting and ad'hoc command line operations, `ascli` is perfect.
|
68
78
|
|
79
|
+
So, which is Aspera's CLI ? `ascli` or `ascp`
|
80
|
+
|
81
|
+
`ascp` is the low level Aspera command line for **transfers**.
|
82
|
+
It is in fact the implementation of the FASP protocol.
|
83
|
+
So, ANY Aspera transfer leads to one ascp process running on client side and another on server side.
|
84
|
+
`ascp` can be used as a command line, but it is very low level, and practically it can be used on command line only if there is no Aspera web ap (AoC, Faspex, etc..) and ONLY to do a transfer (send/receive), not for any operation on Aspera apps (e.g. listing remote files).
|
85
|
+
`ascp` does not provide a configuration file to store credentials or options, it does not resume automatically on transfer error.
|
86
|
+
|
87
|
+
In fact, `ascli` can do everything that `ascp` does, and much more, and in an easier way.
|
88
|
+
|
69
89
|
### Notations, Shell, Examples
|
70
90
|
|
71
91
|
Command line operations examples are shown using a shell such as: `bash` or `zsh`.
|
@@ -92,7 +112,7 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
92
112
|
|
93
113
|
```console
|
94
114
|
$ ascli --version
|
95
|
-
4.
|
115
|
+
4.20.0
|
96
116
|
```
|
97
117
|
|
98
118
|
### First use
|
@@ -515,15 +535,27 @@ This can be installed either be installing an Aspera transfer software, or using
|
|
515
535
|
ascli config ascp install
|
516
536
|
```
|
517
537
|
|
538
|
+
This command will retrieve the list of current archives for all platforms from: <https://ibm.biz/sdk_location> and then select the latest version for the current platform.
|
539
|
+
In this case, the default value for option `sdk_url` is `DEF`.
|
540
|
+
|
518
541
|
If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
|
519
542
|
|
520
|
-
|
521
|
-
curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
|
522
|
-
```
|
543
|
+
1. Locate the appropriate SDK archive for your platform, by visiting either:
|
523
544
|
|
524
|
-
|
525
|
-
|
526
|
-
|
545
|
+
- [IBM API Hub](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20SDK%22)
|
546
|
+
- or looking at this file: <https://ibm.biz/sdk_location>
|
547
|
+
|
548
|
+
2. Download the SDK archive using a browser, or `curl` or `wget`, etc...
|
549
|
+
|
550
|
+
```bash
|
551
|
+
curl -Lso sdk.zip https://...
|
552
|
+
```
|
553
|
+
|
554
|
+
3. Install using the local archive
|
555
|
+
|
556
|
+
```bash
|
557
|
+
ascli config ascp install --sdk-url=file:///sdk.zip
|
558
|
+
```
|
527
559
|
|
528
560
|
The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
|
529
561
|
|
@@ -706,7 +738,7 @@ ascli -v
|
|
706
738
|
```
|
707
739
|
|
708
740
|
```text
|
709
|
-
4.
|
741
|
+
4.20.0
|
710
742
|
```
|
711
743
|
|
712
744
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
@@ -969,7 +1001,7 @@ ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
|
|
969
1001
|
|
970
1002
|
## Command Line Interface
|
971
1003
|
|
972
|
-
The command line tool is:
|
1004
|
+
The command line tool is: `ascli`
|
973
1005
|
|
974
1006
|
The `aspera-cli` gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
975
1007
|
|
@@ -1060,6 +1092,8 @@ Hello World
|
|
1060
1092
|
2
|
1061
1093
|
```
|
1062
1094
|
|
1095
|
+
> **Note:** Use `pp` instead of `puts` to display as ruby Array.
|
1096
|
+
|
1063
1097
|
Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' `CreateProcess` with just the whole command line as a single string.
|
1064
1098
|
(Unlike Unix-like systems where the command line is split into arguments by the shell.)
|
1065
1099
|
|
@@ -1328,7 +1362,7 @@ The tokenization of the command line is typically done by the shell, refer to th
|
|
1328
1362
|
`ascli` handles two types of command line arguments:
|
1329
1363
|
|
1330
1364
|
- **Positional Arguments** : position is significant
|
1331
|
-
- **Options** : only order is significant, but not position
|
1365
|
+
- **Options** : only order is significant, but not absolute position
|
1332
1366
|
|
1333
1367
|
For example:
|
1334
1368
|
|
@@ -1349,11 +1383,11 @@ The value of **Options** and **Positional Arguments** is evaluated with the [Ext
|
|
1349
1383
|
**Positional Arguments** are either:
|
1350
1384
|
|
1351
1385
|
- **Commands**, typically at the beginning
|
1352
|
-
- **Command Parameters
|
1386
|
+
- **Command Parameters**, mandatory arguments, e.g. creation data or entity identifier
|
1353
1387
|
|
1354
1388
|
When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
|
1355
1389
|
|
1356
|
-
**Commands** are typically entity types or verbs to act on those entities.
|
1390
|
+
**Commands** are typically entity types (e.g. `users`) or verbs (e.g. `create`) to act on those entities.
|
1357
1391
|
Its value is a `String` that must belong to a fixed list of values in a given context.
|
1358
1392
|
|
1359
1393
|
Example:
|
@@ -1372,19 +1406,22 @@ Order is significant.
|
|
1372
1406
|
The provided command must match one of the supported commands in the given context.
|
1373
1407
|
If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
1374
1408
|
|
1375
|
-
|
1409
|
+
Standard **Commands** are: `create`, `show`, `list`, `modify`, `delete`.
|
1410
|
+
Some entities also support additional commands.
|
1411
|
+
When those additional commands are related to an entity also reachable in another context, then those commands are located below command `do`.
|
1412
|
+
For example sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin node do 1234 browse /`: `browse` is a sub-command of `node`.
|
1376
1413
|
|
1377
1414
|
**Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
|
1378
1415
|
|
1379
1416
|
> **Note:** It could also have been designed as an option.
|
1380
|
-
But since it is mandatory and typically these data
|
1381
|
-
The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position)
|
1382
|
-
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable
|
1417
|
+
But since it is mandatory and typically these data do not need to be set in a configuration file, it is better designed as a Command Parameter, rather than as an additional specific option.
|
1418
|
+
The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position), the value is clearly mandatory and position clearly indicates its role.
|
1419
|
+
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable using an option value.
|
1383
1420
|
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
|
1384
1421
|
|
1385
1422
|
If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
1386
1423
|
|
1387
|
-
A few **Command Parameters** are optional, they are located at the end of the command line.
|
1424
|
+
A few **Command Parameters** are optional, they are always located at the end of the command line.
|
1388
1425
|
|
1389
1426
|
#### Options
|
1390
1427
|
|
@@ -1493,6 +1530,7 @@ When in flatten mode, it is possible to filter fields using the option `fields`
|
|
1493
1530
|
Object lists are displayed one per line, with attributes as columns.
|
1494
1531
|
Single objects (or tables with a single result) are transposed: one attribute per line.
|
1495
1532
|
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
1533
|
+
If option `multi_table` is `yes`, then elements of a table are displayed individually in a table.
|
1496
1534
|
|
1497
1535
|
The style of output can be set using the `format` option, supporting:
|
1498
1536
|
|
@@ -1554,7 +1592,7 @@ The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda e
|
|
1554
1592
|
Example:
|
1555
1593
|
|
1556
1594
|
```bash
|
1557
|
-
ascli aoc admin
|
1595
|
+
ascli aoc admin user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
1558
1596
|
```
|
1559
1597
|
|
1560
1598
|
```output
|
@@ -1579,10 +1617,10 @@ In above example, the same result is obtained with option:
|
|
1579
1617
|
The percent selector allows identification of an entity by another unique identifier other than the native identifier.
|
1580
1618
|
|
1581
1619
|
When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command:
|
1582
|
-
e.g. `ascli aoc admin
|
1620
|
+
e.g. `ascli aoc admin user show 1234` where `1234` is the user's identifier.
|
1583
1621
|
|
1584
1622
|
Some commands provide the following capability:
|
1585
|
-
If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin
|
1623
|
+
If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin user show %name:john` where `john` is the user name.
|
1586
1624
|
|
1587
1625
|
Syntax: `%<field>:<value>`
|
1588
1626
|
|
@@ -1924,8 +1962,8 @@ ascp use /usr/bin/ascp
|
|
1924
1962
|
check_update
|
1925
1963
|
coffee
|
1926
1964
|
coffee --ui=text
|
1927
|
-
coffee --ui=text --
|
1928
|
-
|
1965
|
+
coffee --ui=text --image=@json:'{"text":true,"double":false}'
|
1966
|
+
coffee --ui=text --image=@json:'{"text":true}'
|
1929
1967
|
detect https://faspex5.example.com/path
|
1930
1968
|
detect https://node.example.com/path
|
1931
1969
|
detect https://shares.example.com/path shares
|
@@ -1934,6 +1972,7 @@ detect https://tst.example.com/path httpgw
|
|
1934
1972
|
detect my_org aoc
|
1935
1973
|
doc
|
1936
1974
|
doc transfer-parameters
|
1975
|
+
echo '<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"><circle cx="50" cy="50" r="50" fill="#006699"/></svg>' --format=image
|
1937
1976
|
echo -- --special-string
|
1938
1977
|
echo @base64:SGVsbG8gV29ybGQK
|
1939
1978
|
echo @csvt:@stdin:
|
@@ -1949,6 +1988,8 @@ echo @vault:my_preset.password
|
|
1949
1988
|
echo @zlib:@stdin:
|
1950
1989
|
echo hello
|
1951
1990
|
email_test --notify-to=my_email_external
|
1991
|
+
file
|
1992
|
+
file --logger=syslog --log-level=debug
|
1952
1993
|
flush_tokens
|
1953
1994
|
folder
|
1954
1995
|
gem name
|
@@ -1975,8 +2016,10 @@ pubkey @file:my_key
|
|
1975
2016
|
remote_certificate chain https://node.example.com/path
|
1976
2017
|
remote_certificate name https://node.example.com/path
|
1977
2018
|
remote_certificate only https://node.example.com/path
|
2019
|
+
smtp_settings
|
1978
2020
|
vault create my_label @json:'{"password":"my_password_here","description":"my secret"}'
|
1979
2021
|
vault delete my_label
|
2022
|
+
vault info
|
1980
2023
|
vault list
|
1981
2024
|
vault show my_label
|
1982
2025
|
wizard https://console.example.com/path console
|
@@ -2437,14 +2480,18 @@ To disable the warning, use option `silent_insecure` set to `no`.
|
|
2437
2480
|
|
2438
2481
|
HTTP connection parameters (not `ascp` wss) can be adjusted using option `http_options`:
|
2439
2482
|
|
2440
|
-
| Parameter
|
2441
|
-
|
2442
|
-
| `read_timeout`
|
2443
|
-
| `write_timeout`
|
2444
|
-
| `open_timeout`
|
2445
|
-
| `keep_alive_timeout`
|
2446
|
-
|
2447
|
-
|
2483
|
+
| Parameter | Default | Where |
|
2484
|
+
|---------------------------|---------------|-----------|
|
2485
|
+
| `read_timeout` | 60 | ruby |
|
2486
|
+
| `write_timeout` | 60 | ruby |
|
2487
|
+
| `open_timeout` | 60 | ruby |
|
2488
|
+
| `keep_alive_timeout` | 2 | ruby |
|
2489
|
+
| `user_agent` | `ascli` | `ascli` |
|
2490
|
+
| `download_partial_suffix` | .http_partial | `ascli` |
|
2491
|
+
| `retry_on_error` | 0 | `ascli` |
|
2492
|
+
| `retry_sleep` | nil | `ascli` |
|
2493
|
+
|
2494
|
+
Time values are in set **seconds** and can be of type either integer or float.
|
2448
2495
|
Default values are the ones of Ruby:
|
2449
2496
|
For a full list, refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
2450
2497
|
|
@@ -2453,7 +2500,7 @@ Like any other option, those can be set either on command line, or in configurat
|
|
2453
2500
|
Example:
|
2454
2501
|
|
2455
2502
|
```bash
|
2456
|
-
ascli aoc admin
|
2503
|
+
ascli aoc admin package list --http-options=@json:'{"read_timeout":10.0}'
|
2457
2504
|
```
|
2458
2505
|
|
2459
2506
|
### Proxy
|
@@ -2745,19 +2792,21 @@ Refer to section [FASP](#fasp-configuration).
|
|
2745
2792
|
|
2746
2793
|
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
|
2747
2794
|
|
2748
|
-
| Name | Type
|
2749
|
-
|
2750
|
-
| `wss` | Bool
|
2751
|
-
| `
|
2752
|
-
| `
|
2753
|
-
| `
|
2754
|
-
| `
|
2755
|
-
| `
|
2756
|
-
| `
|
2757
|
-
| `
|
2758
|
-
| `resume
|
2759
|
-
| `resume.
|
2760
|
-
| `resume.
|
2795
|
+
| Name | Type | Description |
|
2796
|
+
|------------------------|---------|-------------|
|
2797
|
+
| `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
|
2798
|
+
| `quiet` | Bool | If `true`, then `ascp` progress bar is not shown.<br/>Default: `false` |
|
2799
|
+
| `trusted_certs` | Array | List of repositories for trusted certificates. |
|
2800
|
+
| `client_ssh_key` | String | SSH Keys to use for token-based transfers. One of: `dsa_rsa`, `rsa`, `per_client`. Default: `rsa` |
|
2801
|
+
| `ascp_args` | Array | Array of strings with native `ascp` arguments.<br/>Default: `[]` |
|
2802
|
+
| `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: `3` |
|
2803
|
+
| `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: `2` |
|
2804
|
+
| `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If `true`, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: `true` on Windows, else `false` |
|
2805
|
+
| `resume` | Hash | Resume parameters. See below |
|
2806
|
+
| `resume.iter_max` | Integer | Max number of retry on error<br/>Default: `7` |
|
2807
|
+
| `resume.sleep_initial` | Integer | First Sleep before retry<br/>Default: `2` |
|
2808
|
+
| `resume.sleep_factor` | Integer | Multiplier of sleep period between attempts<br/>Default: `2` |
|
2809
|
+
| `resume.sleep_max` | Integer | Default: `60` |
|
2761
2810
|
|
2762
2811
|
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
|
2763
2812
|
Sleep between iterations is given by the following formula where `iter_index` is the current iteration index, starting at 0:
|
@@ -2767,7 +2816,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
|
|
2767
2816
|
```
|
2768
2817
|
|
2769
2818
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
2770
|
-
In order to use a custom certificate store, use the `trusted_certs` option
|
2819
|
+
In order to use a custom certificate store, use the `trusted_certs` option of direct agent's option `transfer_info`.
|
2771
2820
|
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
2772
2821
|
|
2773
2822
|
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
@@ -3561,7 +3610,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
3561
3610
|
```text
|
3562
3611
|
ascli -h
|
3563
3612
|
NAME
|
3564
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3613
|
+
ascli -- a command line tool for Aspera Applications (v4.20.0)
|
3565
3614
|
|
3566
3615
|
SYNOPSIS
|
3567
3616
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -3599,8 +3648,9 @@ OPTIONS: global
|
|
3599
3648
|
--fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
|
3600
3649
|
--select=VALUE Select only some items in lists: column, value (Hash, Proc)
|
3601
3650
|
--table-style=VALUE Table display style (Hash)
|
3602
|
-
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
3603
|
-
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
3651
|
+
--flat-hash=ENUM (Table) Display deep values as additional keys: no, [yes]
|
3652
|
+
--transpose-single=ENUM (Table) Single object fields output vertically: no, [yes]
|
3653
|
+
--multi-table=ENUM (Table) Each element of a table are displayed as a table: [no], yes
|
3604
3654
|
--show-secrets=ENUM Show secrets on command output: [no], yes
|
3605
3655
|
--image=VALUE Options for image display (Hash)
|
3606
3656
|
-h, --help Show this message
|
@@ -3615,17 +3665,12 @@ OPTIONS: global
|
|
3615
3665
|
--log-secrets=ENUM Show passwords in logs: [no], yes
|
3616
3666
|
--clean-temp=ENUM Cleanup temporary files on exit: no, [yes]
|
3617
3667
|
--pid-file=VALUE Write process identifier to file, delete on exit (String)
|
3618
|
-
|
3619
|
-
COMMAND: config
|
3620
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
3621
|
-
OPTIONS:
|
3622
3668
|
--home=VALUE Home folder for tool (String)
|
3623
3669
|
--config-file=VALUE Path to YAML file with preset configuration
|
3624
3670
|
--secret=VALUE Secret for access keys
|
3625
3671
|
--vault=VALUE Vault for secrets (Hash)
|
3626
3672
|
--vault-password=VALUE Vault password
|
3627
|
-
--query=VALUE Additional filter for for some commands (list/delete) (Hash)
|
3628
|
-
--value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
|
3673
|
+
--query=VALUE Additional filter for for some commands (list/delete) (Hash, Array)
|
3629
3674
|
--property=VALUE Name of property to set (modify operation)
|
3630
3675
|
--bulk=ENUM Bulk operation (only some): [no], yes
|
3631
3676
|
--bfail=ENUM Bulk operation error handling: no, [yes]
|
@@ -3653,7 +3698,7 @@ OPTIONS:
|
|
3653
3698
|
--http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
|
3654
3699
|
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
3655
3700
|
--fpac=VALUE Proxy auto configuration script
|
3656
|
-
--proxy-credentials=VALUE HTTP proxy credentials for fpac
|
3701
|
+
--proxy-credentials=VALUE HTTP proxy credentials for fpac: user, password (Array)
|
3657
3702
|
--ts=VALUE Override transfer spec values (Hash)
|
3658
3703
|
--to-folder=VALUE Destination folder for transferred files
|
3659
3704
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
@@ -3661,6 +3706,9 @@ OPTIONS:
|
|
3661
3706
|
--transfer=ENUM Type of transfer agent: node, [direct], alpha, httpgw, connect, trsdk
|
3662
3707
|
--transfer-info=VALUE Parameters for transfer agent (Hash)
|
3663
3708
|
|
3709
|
+
COMMAND: config
|
3710
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open platform plugins preset proxy_check pubkey remote_certificate smtp_settings test vault wizard
|
3711
|
+
|
3664
3712
|
|
3665
3713
|
COMMAND: shares
|
3666
3714
|
SUBCOMMANDS: admin files health
|
@@ -3680,7 +3728,8 @@ OPTIONS:
|
|
3680
3728
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
3681
3729
|
--sync-name=VALUE Sync name
|
3682
3730
|
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
3683
|
-
--
|
3731
|
+
--node-cache=ENUM Set to no to force actual file system read (gen4): no, [yes]
|
3732
|
+
--root-id=VALUE File id of top folder when using access key (override AK root id)
|
3684
3733
|
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3685
3734
|
|
3686
3735
|
|
@@ -3829,12 +3878,6 @@ OPTIONS:
|
|
3829
3878
|
--workspace=VALUE Name of workspace (String, NilClass)
|
3830
3879
|
--new-user-option=VALUE New user creation option for unknown package recipients (Hash)
|
3831
3880
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
3832
|
-
--validator=VALUE Identifier of validator (optional for central)
|
3833
|
-
--asperabrowserurl=VALUE URL for simple aspera web ui
|
3834
|
-
--sync-name=VALUE Sync name
|
3835
|
-
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
3836
|
-
--root-id=VALUE File id of top folder if using bearer tokens
|
3837
|
-
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3838
3881
|
|
3839
3882
|
|
3840
3883
|
COMMAND: server
|
@@ -3869,6 +3912,22 @@ Bulk creation and deletion of resources are possible using option `bulk` (`yes`,
|
|
3869
3912
|
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended-value-syntax).
|
3870
3913
|
This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
|
3871
3914
|
|
3915
|
+
### Option: `query`
|
3916
|
+
|
3917
|
+
The `query` option can generally be used to add URL parameters to commands that list ressources.
|
3918
|
+
It takes either a `Hash` or an `Array`, corresponding to key/value pairs that appear in the query part of request.
|
3919
|
+
|
3920
|
+
For example: `--query=@json:'{"p1":"v1","p2":"v2"}'` leads to query: `?p1=v1&p2=v2`.
|
3921
|
+
|
3922
|
+
If the same parameter needs to be provided several times, then it's possible as well to provide an Array or 2-element Array: `--query=@json:'[["p1":,"v1"],["p2":"v2"]]'` leads to the same result as previously.
|
3923
|
+
|
3924
|
+
If PHP's style array is used, then one can use either:
|
3925
|
+
|
3926
|
+
- `--query=@json:'{"a":["[]","v1","v2"]}'`
|
3927
|
+
- `--query=@json:'[["a[]","v1"],["a[]","v2"]]'`
|
3928
|
+
|
3929
|
+
Both result in: `?a[]=v1&a[]=v2`.
|
3930
|
+
|
3872
3931
|
### Plugins
|
3873
3932
|
|
3874
3933
|
`ascli` uses a plugin mechanism.
|
@@ -4012,7 +4071,7 @@ Let's start by a registration with web based authentication (auth=web):
|
|
4012
4071
|
- leave the JWT part for now
|
4013
4072
|
- **Save**
|
4014
4073
|
|
4015
|
-
> **Note:** For web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For
|
4074
|
+
> **Note:** For web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For `ascli`, HTTP is required, and 12345 is the default port.
|
4016
4075
|
|
4017
4076
|
Once the client is registered, a **Client ID** and **Secret** are created, these values will be used in the next step.
|
4018
4077
|
|
@@ -4066,7 +4125,7 @@ If you are not using the built-in client_id and secret, JWT needs to be authoriz
|
|
4066
4125
|
- Using command line
|
4067
4126
|
|
4068
4127
|
```bash
|
4069
|
-
ascli aoc admin
|
4128
|
+
ascli aoc admin client list
|
4070
4129
|
```
|
4071
4130
|
|
4072
4131
|
```output
|
@@ -4078,7 +4137,7 @@ ascli aoc admin res client list
|
|
4078
4137
|
```
|
4079
4138
|
|
4080
4139
|
```bash
|
4081
|
-
ascli aoc admin
|
4140
|
+
ascli aoc admin client modify my_BJbQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
|
4082
4141
|
```
|
4083
4142
|
|
4084
4143
|
```output
|
@@ -4103,7 +4162,7 @@ Open the previously generated public key located here: `$HOME/.aspera/ascli/my_p
|
|
4103
4162
|
##### Using command line
|
4104
4163
|
|
4105
4164
|
```bash
|
4106
|
-
ascli aoc admin
|
4165
|
+
ascli aoc admin user list
|
4107
4166
|
```
|
4108
4167
|
|
4109
4168
|
```output
|
@@ -4190,7 +4249,7 @@ ascli aoc files bearer_token_node /
|
|
4190
4249
|
```
|
4191
4250
|
|
4192
4251
|
```bash
|
4193
|
-
ascli aoc admin
|
4252
|
+
ascli aoc admin node v4 1234 --secret=_ak_secret_here_ bearer_token_node /
|
4194
4253
|
```
|
4195
4254
|
|
4196
4255
|
### Administration
|
@@ -4201,7 +4260,8 @@ It allows actions (create, update, delete) on **resources**: users, group, nodes
|
|
4201
4260
|
|
4202
4261
|
#### Listing resources
|
4203
4262
|
|
4204
|
-
The command `aoc admin
|
4263
|
+
The command `aoc admin <type> list` lists all entities of given type.
|
4264
|
+
It uses paging and multiple requests if necessary.
|
4205
4265
|
|
4206
4266
|
The option `query` can be optionally used.
|
4207
4267
|
It expects a `Hash` using [Extended Value Syntax](#extended-value-syntax), generally provided using: `--query=@json:{...}`.
|
@@ -4232,19 +4292,19 @@ Examples:
|
|
4232
4292
|
- List users with `laurent` in name:
|
4233
4293
|
|
4234
4294
|
```bash
|
4235
|
-
ascli aoc admin
|
4295
|
+
ascli aoc admin user list --query=@json:'{"q":"laurent"}'
|
4236
4296
|
```
|
4237
4297
|
|
4238
4298
|
- List users who logged-in before a date:
|
4239
4299
|
|
4240
4300
|
```bash
|
4241
|
-
ascli aoc admin
|
4301
|
+
ascli aoc admin user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
4242
4302
|
```
|
4243
4303
|
|
4244
4304
|
- List external users and sort in reverse alphabetical order using name:
|
4245
4305
|
|
4246
4306
|
```bash
|
4247
|
-
ascli aoc admin
|
4307
|
+
ascli aoc admin user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
|
4248
4308
|
```
|
4249
4309
|
|
4250
4310
|
Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
|
@@ -4257,17 +4317,17 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
|
|
4257
4317
|
|
4258
4318
|
To execute an action on a specific resource, select it using one of those methods:
|
4259
4319
|
|
4260
|
-
- **recommended**: give id directly on command line **after the action**: `aoc admin
|
4261
|
-
- Give name on command line **after the action**: `aoc admin
|
4262
|
-
- Provide option `id` : `aoc admin
|
4263
|
-
- Provide option `name` : `aoc admin
|
4320
|
+
- **recommended**: give id directly on command line **after the action**: `aoc admin node show 123`
|
4321
|
+
- Give name on command line **after the action**: `aoc admin node show name abc`
|
4322
|
+
- Provide option `id` : `aoc admin node show 123`
|
4323
|
+
- Provide option `name` : `aoc admin node show --name=abc`
|
4264
4324
|
|
4265
4325
|
#### Creating a resource
|
4266
4326
|
|
4267
4327
|
New resources (users, groups, workspaces, etc..) can be created using a command like:
|
4268
4328
|
|
4269
4329
|
```bash
|
4270
|
-
ascli aoc admin
|
4330
|
+
ascli aoc admin create <resource type> @json:'{<...parameters...>}'
|
4271
4331
|
```
|
4272
4332
|
|
4273
4333
|
Some of the API endpoints are described [here](https://developer.ibm.com/apis/catalog?search=%22aspera%20on%20cloud%20api%22). Sadly, not all.
|
@@ -4275,7 +4335,7 @@ Some of the API endpoints are described [here](https://developer.ibm.com/apis/ca
|
|
4275
4335
|
Nevertheless, it is possible to guess the structure of the creation value by simply dumping an existing resource, and use the same parameters for the creation.
|
4276
4336
|
|
4277
4337
|
```bash
|
4278
|
-
ascli aoc admin
|
4338
|
+
ascli aoc admin group show 12345 --format=json
|
4279
4339
|
```
|
4280
4340
|
|
4281
4341
|
```json
|
@@ -4287,7 +4347,7 @@ Remove the parameters that are either obviously added by the system: `id`, `crea
|
|
4287
4347
|
And then craft your command:
|
4288
4348
|
|
4289
4349
|
```bash
|
4290
|
-
ascli aoc admin
|
4350
|
+
ascli aoc admin group create @json:'{"description":"test to delete","name":"test 1 to delete","saml_group":false}'
|
4291
4351
|
```
|
4292
4352
|
|
4293
4353
|
If the command returns an error, example:
|
@@ -4314,7 +4374,7 @@ The secret is provided using the `secret` option.
|
|
4314
4374
|
For example in a command like:
|
4315
4375
|
|
4316
4376
|
```bash
|
4317
|
-
ascli aoc admin
|
4377
|
+
ascli aoc admin node 123 --secret="my_secret_here" v3 info
|
4318
4378
|
```
|
4319
4379
|
|
4320
4380
|
It is also possible to store secrets in the [secret vault](#secret-vault) and then automatically find the related secret using the [config finder](#configuration-finder).
|
@@ -4403,7 +4463,7 @@ Current Workspace: Default (default)
|
|
4403
4463
|
#### Example: Bulk creation of users
|
4404
4464
|
|
4405
4465
|
```bash
|
4406
|
-
ascli aoc admin
|
4466
|
+
ascli aoc admin user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
|
4407
4467
|
```
|
4408
4468
|
|
4409
4469
|
```output
|
@@ -4418,7 +4478,7 @@ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.
|
|
4418
4478
|
#### Example: Find with filter and delete
|
4419
4479
|
|
4420
4480
|
```bash
|
4421
|
-
ascli aoc admin
|
4481
|
+
ascli aoc admin user list --query='@json:{"q":"dummyuser"}' --fields=id,email
|
4422
4482
|
```
|
4423
4483
|
|
4424
4484
|
```output
|
@@ -4431,7 +4491,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,emai
|
|
4431
4491
|
```
|
4432
4492
|
|
4433
4493
|
```bash
|
4434
|
-
ascli aoc admin
|
4494
|
+
ascli aoc admin user list --query='@json:{"q":"dummyuser"}' --fields=id --display=data --format=csv | ascli aoc admin user delete @lines:@stdin: --bulk=yes
|
4435
4495
|
```
|
4436
4496
|
|
4437
4497
|
```output
|
@@ -4446,7 +4506,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --di
|
|
4446
4506
|
#### Example: Find deactivated users since more than 2 years
|
4447
4507
|
|
4448
4508
|
```ruby
|
4449
|
-
ascli aoc admin
|
4509
|
+
ascli aoc admin user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
|
4450
4510
|
```
|
4451
4511
|
|
4452
4512
|
To delete them use the same method as before
|
@@ -4478,7 +4538,7 @@ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key
|
|
4478
4538
|
#### Example: Display transfer events (ops/transfer)
|
4479
4539
|
|
4480
4540
|
```bash
|
4481
|
-
ascli aoc admin
|
4541
|
+
ascli aoc admin node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
|
4482
4542
|
```
|
4483
4543
|
|
4484
4544
|
Examples of query:
|
@@ -4494,13 +4554,13 @@ Examples of query:
|
|
4494
4554
|
#### Example: Display node events (events)
|
4495
4555
|
|
4496
4556
|
```bash
|
4497
|
-
ascli aoc admin
|
4557
|
+
ascli aoc admin node --secret=_secret_ v3 events
|
4498
4558
|
```
|
4499
4559
|
|
4500
4560
|
#### Example: Display members of a workspace
|
4501
4561
|
|
4502
4562
|
```bash
|
4503
|
-
ascli aoc admin
|
4563
|
+
ascli aoc admin workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4504
4564
|
```
|
4505
4565
|
|
4506
4566
|
```output
|
@@ -4508,7 +4568,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
|
|
4508
4568
|
| member_type | manager | member.email |
|
4509
4569
|
+-------------+---------+----------------------------------+
|
4510
4570
|
| user | true | john.curtis@email.com |
|
4511
|
-
| user | false | someuser@example.com
|
4571
|
+
| user | false | someuser@example.com |
|
4512
4572
|
| user | false | jean.dupont@me.com |
|
4513
4573
|
| user | false | another.user@example.com |
|
4514
4574
|
| group | false | |
|
@@ -4528,20 +4588,20 @@ a- Get id of first workspace
|
|
4528
4588
|
|
4529
4589
|
```bash
|
4530
4590
|
WS1='First Workspace'
|
4531
|
-
WS1ID=$(ascli aoc admin
|
4591
|
+
WS1ID=$(ascli aoc admin workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
|
4532
4592
|
```
|
4533
4593
|
|
4534
4594
|
b- Get id of second workspace
|
4535
4595
|
|
4536
4596
|
```bash
|
4537
4597
|
WS2='Second Workspace'
|
4538
|
-
WS2ID=$(ascli aoc admin
|
4598
|
+
WS2ID=$(ascli aoc admin workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
|
4539
4599
|
```
|
4540
4600
|
|
4541
4601
|
c- Extract membership information
|
4542
4602
|
|
4543
4603
|
```bash
|
4544
|
-
ascli aoc admin
|
4604
|
+
ascli aoc admin workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp --output=ws1_members.json
|
4545
4605
|
```
|
4546
4606
|
|
4547
4607
|
d- Convert to creation data for second workspace:
|
@@ -4559,13 +4619,13 @@ jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"
|
|
4559
4619
|
e- Add members to second workspace
|
4560
4620
|
|
4561
4621
|
```bash
|
4562
|
-
ascli aoc admin
|
4622
|
+
ascli aoc admin workspace_membership create --bulk=yes @json:@file:ws2_members.json
|
4563
4623
|
```
|
4564
4624
|
|
4565
4625
|
#### Example: Get users who did not log since a date
|
4566
4626
|
|
4567
4627
|
```bash
|
4568
|
-
ascli aoc admin
|
4628
|
+
ascli aoc admin user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
4569
4629
|
```
|
4570
4630
|
|
4571
4631
|
```output
|
@@ -4580,7 +4640,7 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
|
|
4580
4640
|
#### Example: List **Limited** users
|
4581
4641
|
|
4582
4642
|
```bash
|
4583
|
-
ascli aoc admin
|
4643
|
+
ascli aoc admin user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
|
4584
4644
|
```
|
4585
4645
|
|
4586
4646
|
#### Example: Create a group, add to workspace and add user to group
|
@@ -4588,7 +4648,7 @@ ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_wor
|
|
4588
4648
|
- Create the group and take note of `id`
|
4589
4649
|
|
4590
4650
|
```bash
|
4591
|
-
ascli aoc admin
|
4651
|
+
ascli aoc admin group create @json:'{"name":"group 1","description":"my super group"}'
|
4592
4652
|
```
|
4593
4653
|
|
4594
4654
|
Group: `11111`
|
@@ -4596,7 +4656,7 @@ Group: `11111`
|
|
4596
4656
|
- Get the workspace id
|
4597
4657
|
|
4598
4658
|
```bash
|
4599
|
-
ascli aoc admin
|
4659
|
+
ascli aoc admin workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
|
4600
4660
|
```
|
4601
4661
|
|
4602
4662
|
Workspace: 22222
|
@@ -4604,13 +4664,13 @@ Workspace: 22222
|
|
4604
4664
|
- Add group to workspace
|
4605
4665
|
|
4606
4666
|
```bash
|
4607
|
-
ascli aoc admin
|
4667
|
+
ascli aoc admin workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
|
4608
4668
|
```
|
4609
4669
|
|
4610
4670
|
- Get a user's id
|
4611
4671
|
|
4612
4672
|
```bash
|
4613
|
-
ascli aoc admin
|
4673
|
+
ascli aoc admin user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
|
4614
4674
|
```
|
4615
4675
|
|
4616
4676
|
User: 33333
|
@@ -4618,7 +4678,7 @@ User: 33333
|
|
4618
4678
|
- Add user to group
|
4619
4679
|
|
4620
4680
|
```bash
|
4621
|
-
ascli aoc admin
|
4681
|
+
ascli aoc admin group_membership create @json:'{"group_id":11111,"member_type":"user","member_id":33333}'
|
4622
4682
|
```
|
4623
4683
|
|
4624
4684
|
#### Example: Perform a multi Gbps transfer between two remote shared folders
|
@@ -4666,7 +4726,7 @@ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AW
|
|
4666
4726
|
#### Example: Create registration key to register a node
|
4667
4727
|
|
4668
4728
|
```bash
|
4669
|
-
ascli aoc admin
|
4729
|
+
ascli aoc admin client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
4670
4730
|
```
|
4671
4731
|
|
4672
4732
|
```output
|
@@ -4676,7 +4736,7 @@ jfqslfdjlfdjfhdjklqfhdkl
|
|
4676
4736
|
#### Example: Delete all registration keys
|
4677
4737
|
|
4678
4738
|
```bash
|
4679
|
-
ascli aoc admin
|
4739
|
+
ascli aoc admin client list --fields=id --format=csv|ascli aoc admin client delete @lines:@stdin: --bulk=yes
|
4680
4740
|
```
|
4681
4741
|
|
4682
4742
|
```output
|
@@ -4727,7 +4787,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
4727
4787
|
Then use the returned address for the `url` key to actually create the AoC Node entity:
|
4728
4788
|
|
4729
4789
|
```bash
|
4730
|
-
ascli aoc admin
|
4790
|
+
ascli aoc admin node create @json:'{"name":"myname","access_key":"myaccesskeyid","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
|
4731
4791
|
```
|
4732
4792
|
|
4733
4793
|
Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
|
@@ -4794,13 +4854,13 @@ ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":
|
|
4794
4854
|
#### Example: Send a package to a shared inbox with metadata
|
4795
4855
|
|
4796
4856
|
```bash
|
4797
|
-
ascli aoc packages send --workspace=
|
4857
|
+
ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
|
4798
4858
|
```
|
4799
4859
|
|
4800
4860
|
It is also possible to use identifiers and API parameters:
|
4801
4861
|
|
4802
4862
|
```bash
|
4803
|
-
ascli aoc packages send --workspace=
|
4863
|
+
ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
4804
4864
|
```
|
4805
4865
|
|
4806
4866
|
#### Example: List packages in a given shared inbox
|
@@ -4903,13 +4963,47 @@ ascli aoc files download <single file path>
|
|
4903
4963
|
|
4904
4964
|
#### Shared folders
|
4905
4965
|
|
4906
|
-
Shared
|
4966
|
+
Like in AoC web UI, "Shared Folders" can be created and shared with either **Private** or **Public** links.
|
4967
|
+
**Private** links require the collaborator to log-in to access the shared folder.
|
4968
|
+
**Public** links include a passcode that enables the user to access the shared folder without login-in.
|
4969
|
+
|
4970
|
+
Shared folders can be created either:
|
4971
|
+
|
4972
|
+
- by users in a workspace: they can share personal folders with other users in the same workspace: `aoc files perm`
|
4973
|
+
- by administrators: they can share a folder with users in any workspace: `aoc admin node do [node id] perm`
|
4974
|
+
|
4975
|
+
Technically (API), shared folder are managed through [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction) on node and an event is sent to AoC to create a **link** in the user's home folder to the shared folder.
|
4976
|
+
In both cases, it is necessary to specify a workspace.
|
4977
|
+
|
4978
|
+
The basic payload (last argument at creation usually specified with `@json:`) is:
|
4979
|
+
|
4980
|
+
```json
|
4981
|
+
{
|
4982
|
+
"file_id": "50",
|
4983
|
+
"access_levels": ["list","read","write","delete","mkdir","rename","preview"],
|
4984
|
+
"access_type": "user",
|
4985
|
+
"access_id": "john@example.com",
|
4986
|
+
"tags": {...},
|
4987
|
+
}
|
4988
|
+
```
|
4989
|
+
|
4990
|
+
`ascli` expects the same payload for creation.
|
4991
|
+
`ascli` automatically populates this payload like this:
|
4992
|
+
|
4993
|
+
- `file_id` : the id of the folder to share whose path is specified in the command line
|
4994
|
+
- `access_levels` : are set by default to full access.
|
4995
|
+
- `tags` : are set with expected values for AoC: user name who creates, and workspace in which the shared folder is created.
|
4996
|
+
- `access_type` and `access_id` : need to be set by the user, or using special key as follows.
|
4997
|
+
|
4998
|
+
To change `access_levels`, just provide the list of levels in the `@json:` payload.
|
4999
|
+
|
5000
|
+
In order to declare/create the shared folder in the workspace, a special value for `access_id` is used: `ASPERA_ACCESS_KEY_ADMIN_WS_[workspace id]]`. This is conveniently set by `ascli` using an empty string for the pseudo key `with`. In order to share a folder with a different, special tags are set, but this is conveniently done by `ascli` using the `as` key.
|
4907
5001
|
|
4908
|
-
|
4909
|
-
`ascli` expects the same payload for creation, but it will automatically populate required tags if needed.
|
5002
|
+
The following optional additional helper keys are supported by `ascli`:
|
4910
5003
|
|
4911
|
-
|
4912
|
-
|
5004
|
+
- `with` : Recipient of shared folder. Can be a user name, a group name, or a workspace name. `ascli` will resolve the name to the proper type and id in fields `access_type` and `access_id`. If the value is the empty string, then it declares the shared folder in the workspace (first action to do, see below).
|
5005
|
+
- `link_name` : The name of the link file created in the user's home folder for private links.
|
5006
|
+
- `as` : The name of the link file created in the user's home folder for admin shared folders.
|
4913
5007
|
|
4914
5008
|
- List permissions on a shared folder as user
|
4915
5009
|
|
@@ -4932,12 +5026,24 @@ ascli aoc files perm /shared_folder_test1 delete 6161
|
|
4932
5026
|
Public and Private short links can be managed with command:
|
4933
5027
|
|
4934
5028
|
```bash
|
4935
|
-
ascli aoc files short_link private create
|
4936
|
-
ascli aoc files short_link private list
|
4937
|
-
ascli aoc files short_link public list
|
5029
|
+
ascli aoc files short_link _path_here_ private create
|
5030
|
+
ascli aoc files short_link _path_here_ private list
|
5031
|
+
ascli aoc files short_link _path_here_ public list
|
4938
5032
|
ascli aoc files short_link public delete _id_
|
4939
5033
|
```
|
4940
5034
|
|
5035
|
+
- Create an admin shared folder and shared with a user or group or workspace
|
5036
|
+
|
5037
|
+
```bash
|
5038
|
+
ascli aoc admin node do 1234 mkdir folder_on_node
|
5039
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"","as":"folder_for_users"}' --workspace="my ws"
|
5040
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"john@example.com","as":"folder_for_users"}' --workspace="my ws"
|
5041
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"group 1","as":"folder_for_users"}' --workspace="my ws"
|
5042
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"my ws","as":"folder_for_users"}' --workspace="my ws"
|
5043
|
+
```
|
5044
|
+
|
5045
|
+
> **Note:** The node can be conveniently identified using the **percent selector** instead of numerical id.
|
5046
|
+
|
4941
5047
|
#### Cross Organization transfers
|
4942
5048
|
|
4943
5049
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
@@ -4975,7 +5081,7 @@ The command `aoc files find` allows to search for files in a given workspace.
|
|
4975
5081
|
It works also on `node` resource using the `v4` command:
|
4976
5082
|
|
4977
5083
|
```bash
|
4978
|
-
ascli aoc admin
|
5084
|
+
ascli aoc admin node --name='my node name' --secret='my_secret_here' v4 find ...
|
4979
5085
|
```
|
4980
5086
|
|
4981
5087
|
For instructions, refer to section `find` for plugin `node`.
|
@@ -4987,7 +5093,7 @@ For instructions, refer to section `find` for plugin `node`.
|
|
4987
5093
|
```bash
|
4988
5094
|
admin analytics transfers nodes
|
4989
5095
|
admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
4990
|
-
admin analytics transfers users --
|
5096
|
+
admin analytics transfers users --once-only=yes
|
4991
5097
|
admin application list
|
4992
5098
|
admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
4993
5099
|
admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
@@ -5009,12 +5115,17 @@ admin dropbox list
|
|
5009
5115
|
admin dropbox_membership list
|
5010
5116
|
admin group list
|
5011
5117
|
admin kms_profile list
|
5012
|
-
admin node do %name:
|
5013
|
-
admin node do %name:
|
5014
|
-
admin node do %name:
|
5015
|
-
admin node do %name:
|
5016
|
-
admin node do %name:
|
5017
|
-
admin node do %name:
|
5118
|
+
admin node do %name:my_node_name --secret=my_ak_secret browse /
|
5119
|
+
admin node do %name:my_node_name --secret=my_ak_secret delete /folder1
|
5120
|
+
admin node do %name:my_node_name --secret=my_ak_secret mkdir /folder1
|
5121
|
+
admin node do %name:my_node_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
5122
|
+
admin node do %name:my_node_name --secret=my_ak_secret v3 access_key delete testsub1
|
5123
|
+
admin node do %name:my_node_name --secret=my_ak_secret v3 events
|
5124
|
+
admin node do %name:my_node_name delete test_shared_folder
|
5125
|
+
admin node do %name:my_node_name mkdir test_shared_folder
|
5126
|
+
admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
|
5127
|
+
admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"my_user_email","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
|
5128
|
+
admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"my_user_group","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
|
5018
5129
|
admin node list
|
5019
5130
|
admin operation list
|
5020
5131
|
admin organization show
|
@@ -5047,22 +5158,27 @@ files browse my_remote_folder/
|
|
5047
5158
|
files delete /testsrc
|
5048
5159
|
files download --transfer=alpha testdst/test_file.bin
|
5049
5160
|
files download --transfer=connect testdst/test_file.bin
|
5161
|
+
files find /
|
5050
5162
|
files find / '\.partial$'
|
5163
|
+
files find / @ruby:'->(f){f["type"].eql?("file")}'
|
5051
5164
|
files http_node_download --to-folder=. testdst/test_file.bin
|
5052
5165
|
files mkdir /testsrc
|
5053
5166
|
files modify my_test_folder
|
5054
5167
|
files permission my_test_folder list
|
5055
5168
|
files rename /some_folder testdst
|
5056
|
-
files short_link private create
|
5057
|
-
files short_link private list
|
5058
|
-
files short_link public create
|
5169
|
+
files short_link /testdst private create
|
5170
|
+
files short_link /testdst private list
|
5171
|
+
files short_link /testdst public create
|
5059
5172
|
files show %id:aoc_file_id
|
5060
5173
|
files show /
|
5061
5174
|
files show testdst/test_file.bin
|
5062
5175
|
files sync admin status --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
5063
5176
|
files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
5177
|
+
files sync start --sync-info=@json:'{"name":"my_aoc_sync1","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
5064
5178
|
files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
5065
5179
|
files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
5180
|
+
files sync start pull /testdst /data/local_sync --sync-info=@json:'{"reset":true,"quiet":false}'
|
5181
|
+
files sync start pull /testdst /data/local_sync --sync-info=@json:'{"sessions":[{"reset":true}]}'
|
5066
5182
|
files thumbnail my_test_folder/video_file.mpg
|
5067
5183
|
files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
|
5068
5184
|
files transfer push /testsrc --to-folder=/testdst test_file.bin
|
@@ -5089,8 +5205,10 @@ packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["
|
|
5089
5205
|
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_aoc_user --password=my_public_link_send_use_pass
|
5090
5206
|
packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_shared_inbox
|
5091
5207
|
packages shared_inboxes list
|
5208
|
+
packages shared_inboxes show %name:my_shared_inbox_name
|
5092
5209
|
remind --username=my_user_email
|
5093
5210
|
servers
|
5211
|
+
tier_restrictions
|
5094
5212
|
user pref modify @json:'{"default_language":"en-us"}'
|
5095
5213
|
user pref show
|
5096
5214
|
user profile modify @json:'{"name":"dummy change"}'
|
@@ -5247,11 +5365,18 @@ cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
|
5247
5365
|
## Plugin: `server`: IBM Aspera High Speed Transfer Server (SSH)
|
5248
5366
|
|
5249
5367
|
The `server` plugin is used for operations on Aspera HSTS using SSH authentication.
|
5250
|
-
It is the
|
5368
|
+
It is the original way of accessing an Aspera Server, often used for server to server transfers.
|
5251
5369
|
An SSH session is established, authenticated with either a password or an SSH private key,
|
5252
5370
|
then commands `ascp` (for transfers) and `ascmd` (for file operations) are executed.
|
5253
5371
|
|
5254
|
-
|
5372
|
+
The URL to be provided with option `url` shall be like `ssh://_server_address_:33001`, then option `username` is used to specify the transfer user, and finally either option `password` or `ssh_keys` (with one or several paths) for the authentication.
|
5373
|
+
|
5374
|
+
Typically:
|
5375
|
+
|
5376
|
+
```console
|
5377
|
+
`ascli` server --url=ssh://hsts.example.com:33001 --username=john --password=_something_here_ ...
|
5378
|
+
`ascli` server --url=ssh://hsts.example.com:33001 --username=john --ssh-keys=~/.ssh/id_rsa ...
|
5379
|
+
```
|
5255
5380
|
|
5256
5381
|
### Server sample commands
|
5257
5382
|
|
@@ -5301,7 +5426,8 @@ If SSH is the session protocol (by default i.e. not WSS), then following session
|
|
5301
5426
|
|
5302
5427
|
If `username` is not provided then the default transfer user `xfer` is used.
|
5303
5428
|
|
5304
|
-
If
|
5429
|
+
If neither SSH password nor key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass key(s) is used.
|
5430
|
+
|
5305
5431
|
Example:
|
5306
5432
|
|
5307
5433
|
```bash
|
@@ -5420,11 +5546,13 @@ Native API parameters can be placed in option `query`.
|
|
5420
5546
|
|
5421
5547
|
Special parameters can be placed in option `query` for "gen3" browse:
|
5422
5548
|
|
5423
|
-
| Parameter
|
5424
|
-
|
5549
|
+
| Parameter | Description |
|
5550
|
+
|-------------|-------------|
|
5425
5551
|
| `recursive` | Recursively list files |
|
5426
|
-
| `max`
|
5427
|
-
| `self`
|
5552
|
+
| `max` | Maximum number of files to list |
|
5553
|
+
| `self` | Offset in the list |
|
5554
|
+
|
5555
|
+
Option `node_cache` can be set to `no` to avoid use of folder cache (Redis) and force actual read of file system.
|
5428
5556
|
|
5429
5557
|
### Operation `find` on **gen4/access key**
|
5430
5558
|
|
@@ -5501,6 +5629,24 @@ ascli node access_keys do self find / @ruby:'->(f){f["type"].eql?("file") and (D
|
|
5501
5629
|
|
5502
5630
|
> **Note:** The pipe `|` character on the last line.
|
5503
5631
|
|
5632
|
+
### Listing transfer events
|
5633
|
+
|
5634
|
+
When a transfer is run, its information is stored (typicall, 1 day) in the HSTS database (Redis).
|
5635
|
+
This information can be retrieved with command: `transfer list`.
|
5636
|
+
|
5637
|
+
If the number of transfer is too large, then the list will be retrieved in several API calls.
|
5638
|
+
|
5639
|
+
In addition, it is possible to list "only new information" using option `once_only`.
|
5640
|
+
|
5641
|
+
```bash
|
5642
|
+
ascli node transfer list --once-only=yes
|
5643
|
+
```
|
5644
|
+
|
5645
|
+
The `iteratin_token` that keeps memory of latest event is stored in the persistance repository of `ascli`.
|
5646
|
+
To reset it, add option: `--query=@json:'{"reset": true}'`.
|
5647
|
+
To list only a number of events, use the `max` parameter in query.
|
5648
|
+
Other parameters are directly transmitted to the underlying API (`GET /ops/transfers`).
|
5649
|
+
|
5504
5650
|
### Central
|
5505
5651
|
|
5506
5652
|
The central subcommand uses the **reliable query** API (session and file).
|
@@ -5555,8 +5701,8 @@ Follow the Aspera Transfer Server configuration to activate this feature.
|
|
5555
5701
|
|
5556
5702
|
The following command lists one file that requires validation, and assign it to the unique validator identifier provided:
|
5557
5703
|
|
5558
|
-
```
|
5559
|
-
ascli node central file list --validator=ascli
|
5704
|
+
```json
|
5705
|
+
ascli node central file list --validator=ascli @json:'{"file_transfer_filter":{"max_result":1}}'
|
5560
5706
|
```
|
5561
5707
|
|
5562
5708
|
```output
|
@@ -5569,8 +5715,8 @@ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_fil
|
|
5569
5715
|
|
5570
5716
|
To update the status of the file, use the following command:
|
5571
5717
|
|
5572
|
-
```
|
5573
|
-
ascli node central file update --validator=ascli
|
5718
|
+
```json
|
5719
|
+
ascli node central file update --validator=ascli @json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
|
5574
5720
|
```
|
5575
5721
|
|
5576
5722
|
```output
|
@@ -5644,11 +5790,11 @@ ascli node info --field=@ruby:'/^access_key_configuration_capabilities.*/'
|
|
5644
5790
|
|
5645
5791
|
Bearer tokens are part of the **gen4/access key** API.
|
5646
5792
|
It follows the model of OAuth 2.
|
5647
|
-
For example they are used in Aspera on Cloud.
|
5793
|
+
For example, they are used in Aspera on Cloud.
|
5648
5794
|
This is also available for developers for any application integrating Aspera.
|
5649
5795
|
In this API, files, users and groups are identified by an id (a String, e.g. `"125"`, not necessarily numerical).
|
5650
5796
|
|
5651
|
-
Bearer tokens are typically generated by the authentication application
|
5797
|
+
Bearer tokens are typically generated by the authentication application and then recognized by the node API.
|
5652
5798
|
A bearer token is authorized on the node by creating `permissions` on a **folder**.
|
5653
5799
|
|
5654
5800
|
Bearer tokens can be generated using command `bearer_token`: it takes two arguments:
|
@@ -5674,6 +5820,11 @@ Bearer tokens can be generated using command `bearer_token`: it takes two argume
|
|
5674
5820
|
|
5675
5821
|
#### Bearer token: Environment
|
5676
5822
|
|
5823
|
+
An access key shall be created to grant access for transfers to its storage.
|
5824
|
+
The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
|
5825
|
+
|
5826
|
+
They way to create access keys depend slightly on the type of HSTS:
|
5827
|
+
|
5677
5828
|
- If a self-managed Aspera node is used, then a **node user admin** must be created:
|
5678
5829
|
It has no `docroot` but has at least one file restriction (for testing, one can use `*` to accept creation of an access key with any storage root path).
|
5679
5830
|
Refer to the Aspera HSTS documentation.
|
@@ -5682,12 +5833,10 @@ Bearer tokens can be generated using command `bearer_token`: it takes two argume
|
|
5682
5833
|
|
5683
5834
|
- If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
|
5684
5835
|
|
5685
|
-
- An access key shall be created to grant access for transfers to its storage.
|
5686
|
-
The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
|
5687
|
-
|
5688
5836
|
#### Bearer token: Preparation
|
5689
5837
|
|
5690
5838
|
Let's assume that the access key was created, and a default configuration is set to use this **access key**.
|
5839
|
+
Using `ascli`, an access key can be created using the `access_key create` on the node (using main node credentials) or ATS.
|
5691
5840
|
|
5692
5841
|
- Create a private key (organization key) that will be used to sign bearer tokens:
|
5693
5842
|
|
@@ -5707,6 +5856,7 @@ Let's assume that the access key was created, and a default configuration is set
|
|
5707
5856
|
|
5708
5857
|
> **Note:** Either the public or private key can be provided, and only the public key is used.
|
5709
5858
|
> This will enable to check the signature of the bearer token.
|
5859
|
+
> Above command is executed with access key credentials.
|
5710
5860
|
|
5711
5861
|
Alternatively, use the following equivalent command, as `ascli` kindly extracts the public key with extension `.pub`:
|
5712
5862
|
|
@@ -5768,18 +5918,18 @@ ascli node -N --url=... --password="Bearer $(cat bearer.txt)" --root-id=$my_fold
|
|
5768
5918
|
access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
|
5769
5919
|
access_key delete my_username
|
5770
5920
|
access_key do my_ak_name browse /
|
5771
|
-
access_key do my_ak_name delete /
|
5772
|
-
access_key do my_ak_name delete
|
5773
|
-
access_key do my_ak_name download
|
5921
|
+
access_key do my_ak_name delete /test_nd_ak2
|
5922
|
+
access_key do my_ak_name delete test_nd_ak3
|
5923
|
+
access_key do my_ak_name download test_nd_ak3 --to-folder=.
|
5774
5924
|
access_key do my_ak_name find my_test_folder
|
5775
5925
|
access_key do my_ak_name find my_test_folder @ruby:'->(f){f["name"].end_with?(".jpg")}'
|
5776
5926
|
access_key do my_ak_name find my_test_folder exec:'f["name"].end_with?(".jpg")'
|
5777
|
-
access_key do my_ak_name mkdir /
|
5927
|
+
access_key do my_ak_name mkdir /tst_nd_ak
|
5778
5928
|
access_key do my_ak_name node_info /
|
5779
|
-
access_key do my_ak_name rename /
|
5929
|
+
access_key do my_ak_name rename /tst_nd_ak test_nd_ak2
|
5780
5930
|
access_key do my_ak_name show %id:1
|
5781
|
-
access_key do my_ak_name show /
|
5782
|
-
access_key do my_ak_name upload 'faux:///
|
5931
|
+
access_key do my_ak_name show /test_nd_ak3
|
5932
|
+
access_key do my_ak_name upload 'faux:///test_nd_ak3?100k' --default-ports=no
|
5783
5933
|
access_key do self permission %id:root_id create @json:'{"access_type":"user","access_id":"666"}'
|
5784
5934
|
access_key do self show / --fields=id --output=root_id
|
5785
5935
|
access_key list
|
@@ -5797,7 +5947,7 @@ basic_token
|
|
5797
5947
|
bearer_token @file:my_private_key @json:'{"user_id":"666"}' --output=bearer_666
|
5798
5948
|
browse / --log-level=trace2
|
5799
5949
|
central file list
|
5800
|
-
central file modify --validator=1
|
5950
|
+
central file modify --validator=1 @json:'{"files":[]}'
|
5801
5951
|
central session list
|
5802
5952
|
delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
|
5803
5953
|
delete my_upload_folder/test_file.bin
|
@@ -5830,9 +5980,11 @@ ssync summary %name:my_node_sync
|
|
5830
5980
|
stream list
|
5831
5981
|
sync admin status --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5832
5982
|
sync admin status --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5983
|
+
sync start --sync-info=@json:'{"name":"my_node_sync1","reset":true,"quiet":false,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5833
5984
|
sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5834
|
-
|
5985
|
+
transfer list --once-only=yes
|
5835
5986
|
transfer list --query=@json:'{"active_only":true}'
|
5987
|
+
transfer list --query=@json:'{"reset":true}' --once-only=yes
|
5836
5988
|
transfer sessions
|
5837
5989
|
transport
|
5838
5990
|
upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
@@ -6005,7 +6157,7 @@ admin event web
|
|
6005
6157
|
admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
|
6006
6158
|
admin metadata_profiles list
|
6007
6159
|
admin node list
|
6008
|
-
admin oauth_clients list
|
6160
|
+
admin oauth_clients list --query=@json:'[["client_types[]","public"]]'
|
6009
6161
|
admin registrations list
|
6010
6162
|
admin saml_configs list
|
6011
6163
|
admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
@@ -6198,26 +6350,26 @@ To keep the content encrypted, use option: `--ts=@json:'{"content_protection":nu
|
|
6198
6350
|
If you are a regular user, to list work groups you belong to:
|
6199
6351
|
|
6200
6352
|
```bash
|
6201
|
-
ascli faspex5 admin
|
6353
|
+
ascli faspex5 admin workgroup list
|
6202
6354
|
```
|
6203
6355
|
|
6204
6356
|
If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
|
6205
6357
|
Example:
|
6206
6358
|
|
6207
6359
|
```bash
|
6208
|
-
ascli faspex5 admin
|
6360
|
+
ascli faspex5 admin shared list --query=@json:'{"all":true}' --fields=id,name
|
6209
6361
|
```
|
6210
6362
|
|
6211
6363
|
Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
|
6212
6364
|
|
6213
6365
|
```bash
|
6214
|
-
ascli faspex5 admin
|
6366
|
+
ascli faspex5 admin shared_inboxes invite '%name:the shared inbox' john@example.com
|
6215
6367
|
```
|
6216
6368
|
|
6217
6369
|
It is equivalent to:
|
6218
6370
|
|
6219
6371
|
```bash
|
6220
|
-
ascli faspex5 admin
|
6372
|
+
ascli faspex5 admin shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
6221
6373
|
```
|
6222
6374
|
|
6223
6375
|
Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
|
@@ -6229,13 +6381,13 @@ Other payload parameters are possible for `invite` in this last `Hash` **Command
|
|
6229
6381
|
### Faspex 5: Create Metadata profile
|
6230
6382
|
|
6231
6383
|
```bash
|
6232
|
-
ascli faspex5 admin
|
6384
|
+
ascli faspex5 admin metadata_profiles create @json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
|
6233
6385
|
```
|
6234
6386
|
|
6235
6387
|
### Faspex 5: Create a Shared inbox with specific metadata profile
|
6236
6388
|
|
6237
6389
|
```bash
|
6238
|
-
ascli faspex5 admin
|
6390
|
+
ascli faspex5 admin shared create @json:'{"name":"the shared inbox","metadata_profile_id":1}'
|
6239
6391
|
```
|
6240
6392
|
|
6241
6393
|
### Faspex 5: List content in Shared folder and send package from remote source
|
@@ -6282,6 +6434,10 @@ There are two types of invitations of package submission: public or private.
|
|
6282
6434
|
|
6283
6435
|
Public invitations are for external users, provide just the email address.
|
6284
6436
|
|
6437
|
+
```bash
|
6438
|
+
ascli faspex5 invitations create @json:'{"email_address":"john@example.com"}' --fields=access_url
|
6439
|
+
```
|
6440
|
+
|
6285
6441
|
Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
|
6286
6442
|
|
6287
6443
|
### Faspex 5: Cleanup packages
|
@@ -6362,7 +6518,30 @@ Then, the postprocessing script executed will be `script1.sh`.
|
|
6362
6518
|
|
6363
6519
|
Environment variables at set to the values provided by the web hook which are the same as Faspex 4 postprocessing.
|
6364
6520
|
|
6365
|
-
### Faspex 5:
|
6521
|
+
### Faspex 5: Faspex 4 Gateway
|
6522
|
+
|
6523
|
+
> **Note:** This is not a feature for production. It's provided for testing only.
|
6524
|
+
|
6525
|
+
For legacy faspex client applications that use the `send` API (only) of Faspex v4, the command `gateway` provides the capability to present an API compatible with Faspex 4, and it will call the Faspex 5 API.
|
6526
|
+
|
6527
|
+
It takes a single argument which is the url at which the gateway will be located (locally):
|
6528
|
+
|
6529
|
+
```bash
|
6530
|
+
ascli faspex5 gateway https://localhost:12345/aspera/faspex
|
6531
|
+
```
|
6532
|
+
|
6533
|
+
There are many limitations:
|
6534
|
+
|
6535
|
+
- It's only to emulate the Faspex 4 `send` API (send package)
|
6536
|
+
- No support for remote sources, only for an actual file transfer by the client
|
6537
|
+
- The client must use the transfer spec returned by the API (not faspe: URL)
|
6538
|
+
- tags returned in transfer spec must be used in transfer
|
6539
|
+
- only a single authentication is possible (per gateway) on Faspex5
|
6540
|
+
- no authentication of F4 side (ignored)
|
6541
|
+
|
6542
|
+
Behavior: The API client calls the Faspex 4 API on the gateway, then the gateway transforms this into a Faspex5 API call, which returns a transfer spec, which is returned to the calling client. The calling client uses this to start a transfer to HSTS which is actually managed by Faspex 5.
|
6543
|
+
|
6544
|
+
### Faspex 5: Get Bearer token to use API
|
6366
6545
|
|
6367
6546
|
If a command is missing, then it is still possible to execute command by calling directly the API on the command line using `curl`:
|
6368
6547
|
|
@@ -6552,7 +6731,7 @@ package receive f4_prs2 --to-folder=. --box=sent
|
|
6552
6731
|
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_dbx"]}' test_file.bin
|
6553
6732
|
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_wkg"]}' test_file.bin
|
6554
6733
|
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal","my_username"]}' test_file.bin
|
6555
|
-
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"]}' --
|
6734
|
+
package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"]}' --remote-source=%name:my_src sample_source.txt
|
6556
6735
|
package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
|
6557
6736
|
package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
|
6558
6737
|
source info %name:my_src --storage=@preset:faspex4_storage
|
@@ -6596,12 +6775,12 @@ To figure out the entities payload, for example for creation, refer to the API d
|
|
6596
6775
|
admin group all list
|
6597
6776
|
admin node list
|
6598
6777
|
admin share list --fields=DEF,-status,status_message
|
6599
|
-
admin share user_permissions
|
6778
|
+
admin share user_permissions 3 list
|
6600
6779
|
admin user all app_authorizations 1 modify @json:'{"app_login":true}'
|
6601
6780
|
admin user all app_authorizations 1 show
|
6602
6781
|
admin user all list
|
6603
6782
|
admin user all share_permissions 1 list
|
6604
|
-
admin user all share_permissions 1 show
|
6783
|
+
admin user all share_permissions 1 show 3
|
6605
6784
|
admin user ldap add the_name
|
6606
6785
|
admin user local list
|
6607
6786
|
admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
|
@@ -6609,9 +6788,10 @@ files browse /
|
|
6609
6788
|
files delete my_share1/new_folder
|
6610
6789
|
files delete my_share1/test_file.bin
|
6611
6790
|
files download --to-folder=. my_share1/test_file.bin
|
6612
|
-
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6791
|
+
files download --to-folder=. my_share1/test_file.bin my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6613
6792
|
files mkdir my_share1/new_folder
|
6614
6793
|
files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6794
|
+
files upload --to-folder=https://shares.share1 sendfolder --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6615
6795
|
files upload --to-folder=my_share1 test_file.bin
|
6616
6796
|
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
6617
6797
|
health
|
@@ -6677,7 +6857,7 @@ ascli config preset update mycos --bucket=mybucket --endpoint=https://s3.us-east
|
|
6677
6857
|
ascli config preset set default cos mycos
|
6678
6858
|
```
|
6679
6859
|
|
6680
|
-
Then, jump to the transfer example.
|
6860
|
+
Then, jump to the [transfer example](#operations-transfers).
|
6681
6861
|
|
6682
6862
|
### Using service credential file
|
6683
6863
|
|
@@ -6752,14 +6932,13 @@ ascli cos node info
|
|
6752
6932
|
ascli cos node upload 'faux:///sample1G?1g'
|
6753
6933
|
```
|
6754
6934
|
|
6755
|
-
> **Note:** A dummy file `sample1G` of size 2GB is generated using the `faux` PVCL (man `ascp` and section above), but you can, of course, send a real file by specifying a real file instead.
|
6935
|
+
> **Note:** A dummy file `sample1G` of size 2GB is generated using the `faux` PVCL (man `ascp` and section above), but you can, of course, send a real file by specifying a real file path instead.
|
6756
6936
|
|
6757
6937
|
### Cos sample commands
|
6758
6938
|
|
6759
6939
|
> **Note:** Add `ascli cos` in front of the commands:
|
6760
6940
|
|
6761
6941
|
```bash
|
6762
|
-
node access_key show self
|
6763
6942
|
node download test_file.bin --to-folder=.
|
6764
6943
|
node info --bucket=my_bucket --endpoint=my_endpoint --apikey=my_api_key --crn=my_resource_instance_id
|
6765
6944
|
node info --bucket=my_bucket --region=my_region --service-credentials=@json:@file:my_cos_svc_cred
|
@@ -6775,6 +6954,7 @@ node upload test_file.bin
|
|
6775
6954
|
|
6776
6955
|
```bash
|
6777
6956
|
health
|
6957
|
+
info
|
6778
6958
|
```
|
6779
6959
|
|
6780
6960
|
## Plugin: `faspio`: Faspio Gateway
|
@@ -6799,7 +6979,8 @@ Retrieve information on subscription.
|
|
6799
6979
|
> **Note:** Add `ascli alee` in front of the commands:
|
6800
6980
|
|
6801
6981
|
```bash
|
6802
|
-
|
6982
|
+
entitlement
|
6983
|
+
health -N
|
6803
6984
|
```
|
6804
6985
|
|
6805
6986
|
## Plugin: `preview`: Preview generator for AoC
|
@@ -7124,28 +7305,52 @@ An interface for the `async` utility is provided in the following plugins:
|
|
7124
7305
|
- `aoc files sync` (uses node)
|
7125
7306
|
- `shares files sync` (uses node)
|
7126
7307
|
|
7127
|
-
|
7308
|
+
One advantage over the `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
7309
|
+
Moreover `ascli` supports sync with application requiring token-based authorization.
|
7128
7310
|
|
7129
|
-
|
7311
|
+
Some of the `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
|
7130
7312
|
|
7131
|
-
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
|
7313
|
+
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`). The Transfer SDK 1.3 includes this.
|
7132
7314
|
|
7133
|
-
|
7315
|
+
`sync` supports 0 or 3 arguments:
|
7134
7316
|
|
7135
|
-
|
7317
|
+
- direction
|
7318
|
+
- local folder
|
7319
|
+
- remote folder
|
7320
|
+
|
7321
|
+
Additional options can be provided with option `sync_info`, for which two syntax are possible, as follows.
|
7322
|
+
|
7323
|
+
### `async` API and conf format
|
7136
7324
|
|
7137
7325
|
It is the same payload as specified on the option `--conf` of `async` or in node API `/asyncs`.
|
7138
7326
|
This is the preferred syntax and allows a single session definition.
|
7139
|
-
|
7327
|
+
|
7328
|
+
> **Note:** No progress, nor error messages is provided on terminal.
|
7140
7329
|
|
7141
7330
|
Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
|
7142
7331
|
|
7143
|
-
|
7332
|
+
If 3 arguments are provided they are mapped to:
|
7333
|
+
|
7334
|
+
- `direction`
|
7335
|
+
- `local.path`
|
7336
|
+
- `remote.path`
|
7337
|
+
|
7338
|
+
### `async` options mapping
|
7144
7339
|
|
7145
7340
|
`ascli` defines a JSON equivalent to regular `async`options.
|
7146
7341
|
It is based on a JSON representation of `async` command line options.
|
7147
7342
|
It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
|
7148
7343
|
|
7344
|
+
This is the mode selection if there is either keys `sessions` or `instance` in option `sync_info`.
|
7345
|
+
|
7346
|
+
> **Note:** Progress and error messages are provided on terminal like regular command line invocation of `async`.
|
7347
|
+
|
7348
|
+
If 3 arguments are provided, they are applied to the first (and only) session and mapped to:
|
7349
|
+
|
7350
|
+
- `direction`
|
7351
|
+
- `local_dir`
|
7352
|
+
- `remote_dir`
|
7353
|
+
|
7149
7354
|
## Hot folder
|
7150
7355
|
|
7151
7356
|
### Requirements
|
@@ -7338,10 +7543,12 @@ A default e-mail template is used, but it can be overridden with option `notify_
|
|
7338
7543
|
|
7339
7544
|
The environment provided contains the following additional variables:
|
7340
7545
|
|
7341
|
-
- `subject`
|
7342
|
-
- `
|
7343
|
-
- `
|
7344
|
-
- `
|
7546
|
+
- `subject` : a default subject including transfer status
|
7547
|
+
- `status` : global status of transfer
|
7548
|
+
- `ts` : the [**transfer-spec**](#transfer-specification) used for the transfer
|
7549
|
+
- `from_email` : email of sender (from `smtp` configuration)
|
7550
|
+
- `from_name` : name of sender (from `smtp` configuration)
|
7551
|
+
- `to` : recipient of the email (from `notify_to`)
|
7345
7552
|
|
7346
7553
|
Example of template:
|
7347
7554
|
|
@@ -7350,7 +7557,7 @@ From: <%=from_name%> <<%=from_email%>>
|
|
7350
7557
|
To: <<%=to%>>
|
7351
7558
|
Subject: <%=subject%>
|
7352
7559
|
|
7353
|
-
Transfer is: <%=
|
7560
|
+
Transfer is: <%=status%>
|
7354
7561
|
```
|
7355
7562
|
|
7356
7563
|
## Tool: `asession`
|
@@ -7434,21 +7641,25 @@ asession -h
|
|
7434
7641
|
USAGE
|
7435
7642
|
asession
|
7436
7643
|
asession -h|--help
|
7437
|
-
asession <session spec extended value>
|
7644
|
+
asession [<session spec extended value>]
|
7438
7645
|
|
7439
7646
|
If no argument is provided, default will be used: @json:@stdin
|
7440
7647
|
-h, --help display this message
|
7441
|
-
<session spec extended value> a dictionary
|
7648
|
+
<session spec extended value> a dictionary (Hash)
|
7442
7649
|
The value can be either:
|
7443
7650
|
the JSON description itself, e.g. @json:'{"xx":"yy",...}'
|
7444
7651
|
@json:@stdin, if the JSON is provided from stdin
|
7445
7652
|
@json:@file:<path>, if the JSON is provided from a file
|
7446
|
-
|
7653
|
+
The following keys are recognized in session spec:
|
7654
|
+
spec : mandatory, contains the transfer spec
|
7655
|
+
loglevel : modify log level (to stderr)
|
7656
|
+
agent : modify transfer agent parameters, e.g. ascp_args
|
7657
|
+
file_list_folder : location of temporary files
|
7658
|
+
sdk : location of SDK (ascp)
|
7447
7659
|
Asynchronous commands can be provided on STDIN, examples:
|
7448
7660
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
7449
7661
|
{"type":"START","source":"xx","destination":"yy"}
|
7450
7662
|
{"type":"DONE"}
|
7451
|
-
Note: debug information can be placed on STDERR, using the "loglevel" parameter in session spec (debug=0)
|
7452
7663
|
EXAMPLES
|
7453
7664
|
asession @json:'{"spec":{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}}'
|
7454
7665
|
echo '{"spec":{"remote_host":...}}'|asession @json:@stdin
|