aspera-cli 4.18.1 → 4.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/CHANGELOG.md +33 -0
  4. data/CONTRIBUTING.md +17 -12
  5. data/README.md +396 -185
  6. data/bin/asession +26 -19
  7. data/examples/build_exec +74 -0
  8. data/examples/{rubyc → build_exec_rubyc} +18 -2
  9. data/examples/get_proto_file.rb +7 -0
  10. data/lib/aspera/agent/alpha.rb +8 -8
  11. data/lib/aspera/agent/base.rb +4 -18
  12. data/lib/aspera/agent/connect.rb +14 -13
  13. data/lib/aspera/agent/direct.rb +123 -120
  14. data/lib/aspera/agent/httpgw.rb +2 -3
  15. data/lib/aspera/agent/node.rb +10 -10
  16. data/lib/aspera/agent/trsdk.rb +17 -20
  17. data/lib/aspera/api/alee.rb +15 -0
  18. data/lib/aspera/api/aoc.rb +128 -99
  19. data/lib/aspera/api/ats.rb +1 -1
  20. data/lib/aspera/api/cos_node.rb +1 -1
  21. data/lib/aspera/api/httpgw.rb +104 -64
  22. data/lib/aspera/api/node.rb +33 -12
  23. data/lib/aspera/ascmd.rb +56 -48
  24. data/lib/aspera/ascp/installation.rb +142 -70
  25. data/lib/aspera/ascp/management.rb +7 -3
  26. data/lib/aspera/ascp/products.rb +13 -7
  27. data/lib/aspera/assert.rb +10 -5
  28. data/lib/aspera/cli/formatter.rb +42 -26
  29. data/lib/aspera/cli/hints.rb +2 -1
  30. data/lib/aspera/cli/info.rb +12 -10
  31. data/lib/aspera/cli/main.rb +16 -13
  32. data/lib/aspera/cli/manager.rb +15 -10
  33. data/lib/aspera/cli/plugin.rb +17 -31
  34. data/lib/aspera/cli/plugin_factory.rb +10 -1
  35. data/lib/aspera/cli/plugins/alee.rb +3 -3
  36. data/lib/aspera/cli/plugins/aoc.rb +222 -194
  37. data/lib/aspera/cli/plugins/ats.rb +16 -14
  38. data/lib/aspera/cli/plugins/config.rb +66 -53
  39. data/lib/aspera/cli/plugins/console.rb +3 -3
  40. data/lib/aspera/cli/plugins/faspex.rb +11 -21
  41. data/lib/aspera/cli/plugins/faspex5.rb +44 -42
  42. data/lib/aspera/cli/plugins/faspio.rb +2 -2
  43. data/lib/aspera/cli/plugins/httpgw.rb +1 -1
  44. data/lib/aspera/cli/plugins/node.rb +155 -96
  45. data/lib/aspera/cli/plugins/orchestrator.rb +14 -13
  46. data/lib/aspera/cli/plugins/preview.rb +8 -9
  47. data/lib/aspera/cli/plugins/server.rb +6 -10
  48. data/lib/aspera/cli/plugins/shares.rb +13 -9
  49. data/lib/aspera/cli/sync_actions.rb +72 -31
  50. data/lib/aspera/cli/transfer_agent.rb +13 -14
  51. data/lib/aspera/cli/transfer_progress.rb +36 -18
  52. data/lib/aspera/cli/version.rb +1 -1
  53. data/lib/aspera/command_line_builder.rb +3 -4
  54. data/lib/aspera/coverage.rb +13 -1
  55. data/lib/aspera/environment.rb +59 -10
  56. data/lib/aspera/faspex_gw.rb +3 -3
  57. data/lib/aspera/json_rpc.rb +1 -1
  58. data/lib/aspera/keychain/encrypted_hash.rb +2 -0
  59. data/lib/aspera/keychain/macos_security.rb +7 -12
  60. data/lib/aspera/log.rb +4 -4
  61. data/lib/aspera/node_simulator.rb +1 -1
  62. data/lib/aspera/oauth/base.rb +39 -45
  63. data/lib/aspera/oauth/factory.rb +11 -4
  64. data/lib/aspera/oauth/generic.rb +4 -8
  65. data/lib/aspera/oauth/jwt.rb +4 -4
  66. data/lib/aspera/oauth/url_json.rb +3 -2
  67. data/lib/aspera/oauth/web.rb +10 -6
  68. data/lib/aspera/persistency_action_once.rb +16 -8
  69. data/lib/aspera/preview/utils.rb +5 -16
  70. data/lib/aspera/rest.rb +100 -76
  71. data/lib/aspera/secret_hider.rb +3 -2
  72. data/lib/aspera/ssh.rb +1 -1
  73. data/lib/aspera/transfer/faux_file.rb +7 -5
  74. data/lib/aspera/transfer/parameters.rb +41 -35
  75. data/lib/aspera/transfer/spec.rb +16 -18
  76. data/lib/aspera/transfer/sync.rb +51 -50
  77. data/lib/aspera/transfer/uri.rb +1 -1
  78. data/lib/aspera/uri_reader.rb +1 -1
  79. data/lib/aspera/web_auth.rb +166 -18
  80. data/lib/aspera/web_server_simple.rb +27 -15
  81. data/lib/transfer_pb.rb +84 -0
  82. data/lib/transfer_services_pb.rb +82 -0
  83. data.tar.gz.sig +0 -0
  84. metadata +25 -6
  85. metadata.gz.sig +0 -0
data/README.md CHANGED
@@ -1,8 +1,13 @@
1
1
  # Command Line Interface for IBM Aspera products
2
- <!-- markdownlint-disable MD033 MD003 MD053 -->
3
- <!-- cspell:ignore Serban Antipolis -->
2
+ <!--
3
+ Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md
4
4
 
5
- [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
5
+ markdownlint-disable MD033 MD003 MD053
6
+ cspell:ignore Serban Antipolis
7
+ PANDOC_META_BEGIN
8
+ subtitle: "ascli 4.20.0"
9
+ PANDOC_META_END
10
+ -->
6
11
 
7
12
  [![Gem Version](https://badge.fury.io/rb/aspera-cli.svg)](https://badge.fury.io/rb/aspera-cli)
8
13
  [![unit tests](https://github.com/IBM/aspera-cli/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/IBM/aspera-cli/actions)
@@ -10,9 +15,9 @@
10
15
 
11
16
  ## Introduction
12
17
 
13
- Version : 4.18.1
18
+ Version : 4.20.0
14
19
 
15
- Laurent/2016-2024
20
+ Laurent/2016-2025
16
21
 
17
22
  This gem provides the `ascli` CLI (Command Line Interface) to IBM Aspera software.
18
23
 
@@ -37,6 +42,11 @@ A PDF version of this documentation is available here: [docs/Manual.pdf](docs/Ma
37
42
 
38
43
  Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
39
44
 
45
+ This documentation does not provide ALL the detailed description of all options and commands.
46
+ The reason is that most commands and payloads are directly Rest API calls on the various Aspera products.
47
+ So, to get the full description of all options and commands, refer to the official Aspera API documentation.
48
+ (To see which API is used, set option `--log-level=debug`)
49
+
40
50
  ### When to use and when not to use
41
51
 
42
52
  `ascli` is designed to be used as a command line tool to:
@@ -59,13 +69,23 @@ It is designed for:
59
69
  If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
60
70
 
61
71
  - Product APIs (REST) : e.g. AoC, Faspex, node
62
- - Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Ruby, etc...)
72
+ - Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Go, Ruby, Rust, etc...)
63
73
 
64
74
  Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
65
75
  Code examples here: <https://github.com/laurent-martin/aspera-api-examples>
66
76
 
67
77
  For scripting and ad'hoc command line operations, `ascli` is perfect.
68
78
 
79
+ So, which is Aspera's CLI ? `ascli` or `ascp`
80
+
81
+ `ascp` is the low level Aspera command line for **transfers**.
82
+ It is in fact the implementation of the FASP protocol.
83
+ So, ANY Aspera transfer leads to one ascp process running on client side and another on server side.
84
+ `ascp` can be used as a command line, but it is very low level, and practically it can be used on command line only if there is no Aspera web ap (AoC, Faspex, etc..) and ONLY to do a transfer (send/receive), not for any operation on Aspera apps (e.g. listing remote files).
85
+ `ascp` does not provide a configuration file to store credentials or options, it does not resume automatically on transfer error.
86
+
87
+ In fact, `ascli` can do everything that `ascp` does, and much more, and in an easier way.
88
+
69
89
  ### Notations, Shell, Examples
70
90
 
71
91
  Command line operations examples are shown using a shell such as: `bash` or `zsh`.
@@ -92,7 +112,7 @@ Once the gem is installed, `ascli` shall be accessible:
92
112
 
93
113
  ```console
94
114
  $ ascli --version
95
- 4.18.1
115
+ 4.20.0
96
116
  ```
97
117
 
98
118
  ### First use
@@ -515,15 +535,27 @@ This can be installed either be installing an Aspera transfer software, or using
515
535
  ascli config ascp install
516
536
  ```
517
537
 
538
+ This command will retrieve the list of current archives for all platforms from: <https://ibm.biz/sdk_location> and then select the latest version for the current platform.
539
+ In this case, the default value for option `sdk_url` is `DEF`.
540
+
518
541
  If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
519
542
 
520
- ```bash
521
- curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
522
- ```
543
+ 1. Locate the appropriate SDK archive for your platform, by visiting either:
523
544
 
524
- ```bash
525
- ascli config ascp install --sdk-url=file:///sdk.zip
526
- ```
545
+ - [IBM API Hub](https://developer.ibm.com/apis/catalog?search=%22aspera%20transfer%20SDK%22)
546
+ - or looking at this file: <https://ibm.biz/sdk_location>
547
+
548
+ 2. Download the SDK archive using a browser, or `curl` or `wget`, etc...
549
+
550
+ ```bash
551
+ curl -Lso sdk.zip https://...
552
+ ```
553
+
554
+ 3. Install using the local archive
555
+
556
+ ```bash
557
+ ascli config ascp install --sdk-url=file:///sdk.zip
558
+ ```
527
559
 
528
560
  The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
529
561
 
@@ -706,7 +738,7 @@ ascli -v
706
738
  ```
707
739
 
708
740
  ```text
709
- 4.18.1
741
+ 4.20.0
710
742
  ```
711
743
 
712
744
  In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
@@ -969,7 +1001,7 @@ ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
969
1001
 
970
1002
  ## Command Line Interface
971
1003
 
972
- The command line tool is: ``ascli``
1004
+ The command line tool is: `ascli`
973
1005
 
974
1006
  The `aspera-cli` gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
975
1007
 
@@ -1060,6 +1092,8 @@ Hello World
1060
1092
  2
1061
1093
  ```
1062
1094
 
1095
+ > **Note:** Use `pp` instead of `puts` to display as ruby Array.
1096
+
1063
1097
  Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' `CreateProcess` with just the whole command line as a single string.
1064
1098
  (Unlike Unix-like systems where the command line is split into arguments by the shell.)
1065
1099
 
@@ -1328,7 +1362,7 @@ The tokenization of the command line is typically done by the shell, refer to th
1328
1362
  `ascli` handles two types of command line arguments:
1329
1363
 
1330
1364
  - **Positional Arguments** : position is significant
1331
- - **Options** : only order is significant, but not position
1365
+ - **Options** : only order is significant, but not absolute position
1332
1366
 
1333
1367
  For example:
1334
1368
 
@@ -1349,11 +1383,11 @@ The value of **Options** and **Positional Arguments** is evaluated with the [Ext
1349
1383
  **Positional Arguments** are either:
1350
1384
 
1351
1385
  - **Commands**, typically at the beginning
1352
- - **Command Parameters** , e.g. creation data or entity identifier
1386
+ - **Command Parameters**, mandatory arguments, e.g. creation data or entity identifier
1353
1387
 
1354
1388
  When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
1355
1389
 
1356
- **Commands** are typically entity types or verbs to act on those entities.
1390
+ **Commands** are typically entity types (e.g. `users`) or verbs (e.g. `create`) to act on those entities.
1357
1391
  Its value is a `String` that must belong to a fixed list of values in a given context.
1358
1392
 
1359
1393
  Example:
@@ -1372,19 +1406,22 @@ Order is significant.
1372
1406
  The provided command must match one of the supported commands in the given context.
1373
1407
  If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
1374
1408
 
1375
- Some sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
1409
+ Standard **Commands** are: `create`, `show`, `list`, `modify`, `delete`.
1410
+ Some entities also support additional commands.
1411
+ When those additional commands are related to an entity also reachable in another context, then those commands are located below command `do`.
1412
+ For example sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin node do 1234 browse /`: `browse` is a sub-command of `node`.
1376
1413
 
1377
1414
  **Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
1378
1415
 
1379
1416
  > **Note:** It could also have been designed as an option.
1380
- But since it is mandatory and typically these data need **not** be set in a configuration file, then it is better designed as a **Command Parameter**, and not designed an additional specific option.
1381
- The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position) and the value is clearly mandatory.
1382
- The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
1417
+ But since it is mandatory and typically these data do not need to be set in a configuration file, it is better designed as a Command Parameter, rather than as an additional specific option.
1418
+ The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position), the value is clearly mandatory and position clearly indicates its role.
1419
+ The disadvantage is that it is not possible to define a default value in a configuration file or environment variable using an option value.
1383
1420
  Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
1384
1421
 
1385
1422
  If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
1386
1423
 
1387
- A few **Command Parameters** are optional, they are located at the end of the command line.
1424
+ A few **Command Parameters** are optional, they are always located at the end of the command line.
1388
1425
 
1389
1426
  #### Options
1390
1427
 
@@ -1493,6 +1530,7 @@ When in flatten mode, it is possible to filter fields using the option `fields`
1493
1530
  Object lists are displayed one per line, with attributes as columns.
1494
1531
  Single objects (or tables with a single result) are transposed: one attribute per line.
1495
1532
  If transposition of single object is not desired, use option: `transpose_single` set to `no`.
1533
+ If option `multi_table` is `yes`, then elements of a table are displayed individually in a table.
1496
1534
 
1497
1535
  The style of output can be set using the `format` option, supporting:
1498
1536
 
@@ -1554,7 +1592,7 @@ The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda e
1554
1592
  Example:
1555
1593
 
1556
1594
  ```bash
1557
- ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
1595
+ ascli aoc admin user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
1558
1596
  ```
1559
1597
 
1560
1598
  ```output
@@ -1579,10 +1617,10 @@ In above example, the same result is obtained with option:
1579
1617
  The percent selector allows identification of an entity by another unique identifier other than the native identifier.
1580
1618
 
1581
1619
  When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command:
1582
- e.g. `ascli aoc admin res user show 1234` where `1234` is the user's identifier.
1620
+ e.g. `ascli aoc admin user show 1234` where `1234` is the user's identifier.
1583
1621
 
1584
1622
  Some commands provide the following capability:
1585
- If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin res user show %name:john` where `john` is the user name.
1623
+ If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin user show %name:john` where `john` is the user name.
1586
1624
 
1587
1625
  Syntax: `%<field>:<value>`
1588
1626
 
@@ -1924,8 +1962,8 @@ ascp use /usr/bin/ascp
1924
1962
  check_update
1925
1963
  coffee
1926
1964
  coffee --ui=text
1927
- coffee --ui=text --query=@json:'{"text":"true"}'
1928
- detect https://faspex4.example.com/path
1965
+ coffee --ui=text --image=@json:'{"text":true,"double":false}'
1966
+ coffee --ui=text --image=@json:'{"text":true}'
1929
1967
  detect https://faspex5.example.com/path
1930
1968
  detect https://node.example.com/path
1931
1969
  detect https://shares.example.com/path shares
@@ -1934,6 +1972,7 @@ detect https://tst.example.com/path httpgw
1934
1972
  detect my_org aoc
1935
1973
  doc
1936
1974
  doc transfer-parameters
1975
+ echo '<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"><circle cx="50" cy="50" r="50" fill="#006699"/></svg>' --format=image
1937
1976
  echo -- --special-string
1938
1977
  echo @base64:SGVsbG8gV29ybGQK
1939
1978
  echo @csvt:@stdin:
@@ -1949,6 +1988,8 @@ echo @vault:my_preset.password
1949
1988
  echo @zlib:@stdin:
1950
1989
  echo hello
1951
1990
  email_test --notify-to=my_email_external
1991
+ file
1992
+ file --logger=syslog --log-level=debug
1952
1993
  flush_tokens
1953
1994
  folder
1954
1995
  gem name
@@ -1975,8 +2016,10 @@ pubkey @file:my_key
1975
2016
  remote_certificate chain https://node.example.com/path
1976
2017
  remote_certificate name https://node.example.com/path
1977
2018
  remote_certificate only https://node.example.com/path
2019
+ smtp_settings
1978
2020
  vault create my_label @json:'{"password":"my_password_here","description":"my secret"}'
1979
2021
  vault delete my_label
2022
+ vault info
1980
2023
  vault list
1981
2024
  vault show my_label
1982
2025
  wizard https://console.example.com/path console
@@ -2437,14 +2480,18 @@ To disable the warning, use option `silent_insecure` set to `no`.
2437
2480
 
2438
2481
  HTTP connection parameters (not `ascp` wss) can be adjusted using option `http_options`:
2439
2482
 
2440
- | Parameter | Default |
2441
- |----------------------|---------|
2442
- | `read_timeout` | 60 |
2443
- | `write_timeout` | 60 |
2444
- | `open_timeout` | 60 |
2445
- | `keep_alive_timeout` | 2 |
2446
-
2447
- Values are in set **seconds** and can be of type either integer or float.
2483
+ | Parameter | Default | Where |
2484
+ |---------------------------|---------------|-----------|
2485
+ | `read_timeout` | 60 | ruby |
2486
+ | `write_timeout` | 60 | ruby |
2487
+ | `open_timeout` | 60 | ruby |
2488
+ | `keep_alive_timeout` | 2 | ruby |
2489
+ | `user_agent` | `ascli` | `ascli` |
2490
+ | `download_partial_suffix` | .http_partial | `ascli` |
2491
+ | `retry_on_error` | 0 | `ascli` |
2492
+ | `retry_sleep` | nil | `ascli` |
2493
+
2494
+ Time values are in set **seconds** and can be of type either integer or float.
2448
2495
  Default values are the ones of Ruby:
2449
2496
  For a full list, refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
2450
2497
 
@@ -2453,7 +2500,7 @@ Like any other option, those can be set either on command line, or in configurat
2453
2500
  Example:
2454
2501
 
2455
2502
  ```bash
2456
- ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
2503
+ ascli aoc admin package list --http-options=@json:'{"read_timeout":10.0}'
2457
2504
  ```
2458
2505
 
2459
2506
  ### Proxy
@@ -2745,19 +2792,21 @@ Refer to section [FASP](#fasp-configuration).
2745
2792
 
2746
2793
  The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
2747
2794
 
2748
- | Name | Type | Description |
2749
- |------------------------|-------|-------------|
2750
- | `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
2751
- | `ascp_args` | Array | Array of strings with native `ascp` arguments.<br/>Default: [] |
2752
- | `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
2753
- | `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
2754
- | `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
2755
- | `trusted_certs` | Array | List of repositories for trusted certificates. |
2756
- | `resume` | Hash | Resume parameters. See below |
2757
- | `resume.iter_max` | int | Max number of retry on error<br/>Default: 7 |
2758
- | `resume.sleep_initial` | int | First Sleep before retry<br/>Default: 2 |
2759
- | `resume.sleep_factor` | int | Multiplier of sleep period between attempts<br/>Default: 2 |
2760
- | `resume.sleep_max` | int | Default: 60 |
2795
+ | Name | Type | Description |
2796
+ |------------------------|---------|-------------|
2797
+ | `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
2798
+ | `quiet` | Bool | If `true`, then `ascp` progress bar is not shown.<br/>Default: `false` |
2799
+ | `trusted_certs` | Array | List of repositories for trusted certificates. |
2800
+ | `client_ssh_key` | String | SSH Keys to use for token-based transfers. One of: `dsa_rsa`, `rsa`, `per_client`. Default: `rsa` |
2801
+ | `ascp_args` | Array | Array of strings with native `ascp` arguments.<br/>Default: `[]` |
2802
+ | `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: `3` |
2803
+ | `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: `2` |
2804
+ | `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If `true`, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: `true` on Windows, else `false` |
2805
+ | `resume` | Hash | Resume parameters. See below |
2806
+ | `resume.iter_max` | Integer | Max number of retry on error<br/>Default: `7` |
2807
+ | `resume.sleep_initial` | Integer | First Sleep before retry<br/>Default: `2` |
2808
+ | `resume.sleep_factor` | Integer | Multiplier of sleep period between attempts<br/>Default: `2` |
2809
+ | `resume.sleep_max` | Integer | Default: `60` |
2761
2810
 
2762
2811
  In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
2763
2812
  Sleep between iterations is given by the following formula where `iter_index` is the current iteration index, starting at 0:
@@ -2767,7 +2816,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
2767
2816
  ```
2768
2817
 
2769
2818
  By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
2770
- In order to use a custom certificate store, use the `trusted_certs` option.
2819
+ In order to use a custom certificate store, use the `trusted_certs` option of direct agent's option `transfer_info`.
2771
2820
  To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
2772
2821
 
2773
2822
  Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
@@ -3561,7 +3610,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
3561
3610
  ```text
3562
3611
  ascli -h
3563
3612
  NAME
3564
- ascli -- a command line tool for Aspera Applications (v4.18.1)
3613
+ ascli -- a command line tool for Aspera Applications (v4.20.0)
3565
3614
 
3566
3615
  SYNOPSIS
3567
3616
  ascli COMMANDS [OPTIONS] [ARGS]
@@ -3599,8 +3648,9 @@ OPTIONS: global
3599
3648
  --fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
3600
3649
  --select=VALUE Select only some items in lists: column, value (Hash, Proc)
3601
3650
  --table-style=VALUE Table display style (Hash)
3602
- --flat-hash=ENUM Display deep values as additional keys: no, [yes]
3603
- --transpose-single=ENUM Single object fields output vertically: no, [yes]
3651
+ --flat-hash=ENUM (Table) Display deep values as additional keys: no, [yes]
3652
+ --transpose-single=ENUM (Table) Single object fields output vertically: no, [yes]
3653
+ --multi-table=ENUM (Table) Each element of a table are displayed as a table: [no], yes
3604
3654
  --show-secrets=ENUM Show secrets on command output: [no], yes
3605
3655
  --image=VALUE Options for image display (Hash)
3606
3656
  -h, --help Show this message
@@ -3615,17 +3665,12 @@ OPTIONS: global
3615
3665
  --log-secrets=ENUM Show passwords in logs: [no], yes
3616
3666
  --clean-temp=ENUM Cleanup temporary files on exit: no, [yes]
3617
3667
  --pid-file=VALUE Write process identifier to file, delete on exit (String)
3618
-
3619
- COMMAND: config
3620
- SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
3621
- OPTIONS:
3622
3668
  --home=VALUE Home folder for tool (String)
3623
3669
  --config-file=VALUE Path to YAML file with preset configuration
3624
3670
  --secret=VALUE Secret for access keys
3625
3671
  --vault=VALUE Vault for secrets (Hash)
3626
3672
  --vault-password=VALUE Vault password
3627
- --query=VALUE Additional filter for for some commands (list/delete) (Hash)
3628
- --value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
3673
+ --query=VALUE Additional filter for for some commands (list/delete) (Hash, Array)
3629
3674
  --property=VALUE Name of property to set (modify operation)
3630
3675
  --bulk=ENUM Bulk operation (only some): [no], yes
3631
3676
  --bfail=ENUM Bulk operation error handling: no, [yes]
@@ -3653,7 +3698,7 @@ OPTIONS:
3653
3698
  --http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
3654
3699
  --cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
3655
3700
  --fpac=VALUE Proxy auto configuration script
3656
- --proxy-credentials=VALUE HTTP proxy credentials for fpac. Array: user,password (Array)
3701
+ --proxy-credentials=VALUE HTTP proxy credentials for fpac: user, password (Array)
3657
3702
  --ts=VALUE Override transfer spec values (Hash)
3658
3703
  --to-folder=VALUE Destination folder for transferred files
3659
3704
  --sources=VALUE How list of transferred files is provided (@args,@ts,Array)
@@ -3661,6 +3706,9 @@ OPTIONS:
3661
3706
  --transfer=ENUM Type of transfer agent: node, [direct], alpha, httpgw, connect, trsdk
3662
3707
  --transfer-info=VALUE Parameters for transfer agent (Hash)
3663
3708
 
3709
+ COMMAND: config
3710
+ SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open platform plugins preset proxy_check pubkey remote_certificate smtp_settings test vault wizard
3711
+
3664
3712
 
3665
3713
  COMMAND: shares
3666
3714
  SUBCOMMANDS: admin files health
@@ -3680,7 +3728,8 @@ OPTIONS:
3680
3728
  --asperabrowserurl=VALUE URL for simple aspera web ui
3681
3729
  --sync-name=VALUE Sync name
3682
3730
  --default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
3683
- --root-id=VALUE File id of top folder if using bearer tokens
3731
+ --node-cache=ENUM Set to no to force actual file system read (gen4): no, [yes]
3732
+ --root-id=VALUE File id of top folder when using access key (override AK root id)
3684
3733
  --sync-info=VALUE Information for sync instance and sessions (Hash)
3685
3734
 
3686
3735
 
@@ -3829,12 +3878,6 @@ OPTIONS:
3829
3878
  --workspace=VALUE Name of workspace (String, NilClass)
3830
3879
  --new-user-option=VALUE New user creation option for unknown package recipients (Hash)
3831
3880
  --validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
3832
- --validator=VALUE Identifier of validator (optional for central)
3833
- --asperabrowserurl=VALUE URL for simple aspera web ui
3834
- --sync-name=VALUE Sync name
3835
- --default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
3836
- --root-id=VALUE File id of top folder if using bearer tokens
3837
- --sync-info=VALUE Information for sync instance and sessions (Hash)
3838
3881
 
3839
3882
 
3840
3883
  COMMAND: server
@@ -3869,6 +3912,22 @@ Bulk creation and deletion of resources are possible using option `bulk` (`yes`,
3869
3912
  In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended-value-syntax).
3870
3913
  This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
3871
3914
 
3915
+ ### Option: `query`
3916
+
3917
+ The `query` option can generally be used to add URL parameters to commands that list ressources.
3918
+ It takes either a `Hash` or an `Array`, corresponding to key/value pairs that appear in the query part of request.
3919
+
3920
+ For example: `--query=@json:'{"p1":"v1","p2":"v2"}'` leads to query: `?p1=v1&p2=v2`.
3921
+
3922
+ If the same parameter needs to be provided several times, then it's possible as well to provide an Array or 2-element Array: `--query=@json:'[["p1":,"v1"],["p2":"v2"]]'` leads to the same result as previously.
3923
+
3924
+ If PHP's style array is used, then one can use either:
3925
+
3926
+ - `--query=@json:'{"a":["[]","v1","v2"]}'`
3927
+ - `--query=@json:'[["a[]","v1"],["a[]","v2"]]'`
3928
+
3929
+ Both result in: `?a[]=v1&a[]=v2`.
3930
+
3872
3931
  ### Plugins
3873
3932
 
3874
3933
  `ascli` uses a plugin mechanism.
@@ -4012,7 +4071,7 @@ Let's start by a registration with web based authentication (auth=web):
4012
4071
  - leave the JWT part for now
4013
4072
  - **Save**
4014
4073
 
4015
- > **Note:** For web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
4074
+ > **Note:** For web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For `ascli`, HTTP is required, and 12345 is the default port.
4016
4075
 
4017
4076
  Once the client is registered, a **Client ID** and **Secret** are created, these values will be used in the next step.
4018
4077
 
@@ -4066,7 +4125,7 @@ If you are not using the built-in client_id and secret, JWT needs to be authoriz
4066
4125
  - Using command line
4067
4126
 
4068
4127
  ```bash
4069
- ascli aoc admin res client list
4128
+ ascli aoc admin client list
4070
4129
  ```
4071
4130
 
4072
4131
  ```output
@@ -4078,7 +4137,7 @@ ascli aoc admin res client list
4078
4137
  ```
4079
4138
 
4080
4139
  ```bash
4081
- ascli aoc admin res client modify my_BJbQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
4140
+ ascli aoc admin client modify my_BJbQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
4082
4141
  ```
4083
4142
 
4084
4143
  ```output
@@ -4103,7 +4162,7 @@ Open the previously generated public key located here: `$HOME/.aspera/ascli/my_p
4103
4162
  ##### Using command line
4104
4163
 
4105
4164
  ```bash
4106
- ascli aoc admin res user list
4165
+ ascli aoc admin user list
4107
4166
  ```
4108
4167
 
4109
4168
  ```output
@@ -4190,7 +4249,7 @@ ascli aoc files bearer_token_node /
4190
4249
  ```
4191
4250
 
4192
4251
  ```bash
4193
- ascli aoc admin res node v4 1234 --secret=_ak_secret_here_ bearer_token_node /
4252
+ ascli aoc admin node v4 1234 --secret=_ak_secret_here_ bearer_token_node /
4194
4253
  ```
4195
4254
 
4196
4255
  ### Administration
@@ -4201,7 +4260,8 @@ It allows actions (create, update, delete) on **resources**: users, group, nodes
4201
4260
 
4202
4261
  #### Listing resources
4203
4262
 
4204
- The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
4263
+ The command `aoc admin <type> list` lists all entities of given type.
4264
+ It uses paging and multiple requests if necessary.
4205
4265
 
4206
4266
  The option `query` can be optionally used.
4207
4267
  It expects a `Hash` using [Extended Value Syntax](#extended-value-syntax), generally provided using: `--query=@json:{...}`.
@@ -4232,19 +4292,19 @@ Examples:
4232
4292
  - List users with `laurent` in name:
4233
4293
 
4234
4294
  ```bash
4235
- ascli aoc admin res user list --query=@json:'{"q":"laurent"}'
4295
+ ascli aoc admin user list --query=@json:'{"q":"laurent"}'
4236
4296
  ```
4237
4297
 
4238
4298
  - List users who logged-in before a date:
4239
4299
 
4240
4300
  ```bash
4241
- ascli aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
4301
+ ascli aoc admin user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
4242
4302
  ```
4243
4303
 
4244
4304
  - List external users and sort in reverse alphabetical order using name:
4245
4305
 
4246
4306
  ```bash
4247
- ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
4307
+ ascli aoc admin user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
4248
4308
  ```
4249
4309
 
4250
4310
  Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
@@ -4257,17 +4317,17 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
4257
4317
 
4258
4318
  To execute an action on a specific resource, select it using one of those methods:
4259
4319
 
4260
- - **recommended**: give id directly on command line **after the action**: `aoc admin res node show 123`
4261
- - Give name on command line **after the action**: `aoc admin res node show name abc`
4262
- - Provide option `id` : `aoc admin res node show 123`
4263
- - Provide option `name` : `aoc admin res node show --name=abc`
4320
+ - **recommended**: give id directly on command line **after the action**: `aoc admin node show 123`
4321
+ - Give name on command line **after the action**: `aoc admin node show name abc`
4322
+ - Provide option `id` : `aoc admin node show 123`
4323
+ - Provide option `name` : `aoc admin node show --name=abc`
4264
4324
 
4265
4325
  #### Creating a resource
4266
4326
 
4267
4327
  New resources (users, groups, workspaces, etc..) can be created using a command like:
4268
4328
 
4269
4329
  ```bash
4270
- ascli aoc admin res create <resource type> @json:'{<...parameters...>}'
4330
+ ascli aoc admin create <resource type> @json:'{<...parameters...>}'
4271
4331
  ```
4272
4332
 
4273
4333
  Some of the API endpoints are described [here](https://developer.ibm.com/apis/catalog?search=%22aspera%20on%20cloud%20api%22). Sadly, not all.
@@ -4275,7 +4335,7 @@ Some of the API endpoints are described [here](https://developer.ibm.com/apis/ca
4275
4335
  Nevertheless, it is possible to guess the structure of the creation value by simply dumping an existing resource, and use the same parameters for the creation.
4276
4336
 
4277
4337
  ```bash
4278
- ascli aoc admin res group show 12345 --format=json
4338
+ ascli aoc admin group show 12345 --format=json
4279
4339
  ```
4280
4340
 
4281
4341
  ```json
@@ -4287,7 +4347,7 @@ Remove the parameters that are either obviously added by the system: `id`, `crea
4287
4347
  And then craft your command:
4288
4348
 
4289
4349
  ```bash
4290
- ascli aoc admin res group create @json:'{"description":"test to delete","name":"test 1 to delete","saml_group":false}'
4350
+ ascli aoc admin group create @json:'{"description":"test to delete","name":"test 1 to delete","saml_group":false}'
4291
4351
  ```
4292
4352
 
4293
4353
  If the command returns an error, example:
@@ -4314,7 +4374,7 @@ The secret is provided using the `secret` option.
4314
4374
  For example in a command like:
4315
4375
 
4316
4376
  ```bash
4317
- ascli aoc admin res node 123 --secret="my_secret_here" v3 info
4377
+ ascli aoc admin node 123 --secret="my_secret_here" v3 info
4318
4378
  ```
4319
4379
 
4320
4380
  It is also possible to store secrets in the [secret vault](#secret-vault) and then automatically find the related secret using the [config finder](#configuration-finder).
@@ -4403,7 +4463,7 @@ Current Workspace: Default (default)
4403
4463
  #### Example: Bulk creation of users
4404
4464
 
4405
4465
  ```bash
4406
- ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
4466
+ ascli aoc admin user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
4407
4467
  ```
4408
4468
 
4409
4469
  ```output
@@ -4418,7 +4478,7 @@ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.
4418
4478
  #### Example: Find with filter and delete
4419
4479
 
4420
4480
  ```bash
4421
- ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
4481
+ ascli aoc admin user list --query='@json:{"q":"dummyuser"}' --fields=id,email
4422
4482
  ```
4423
4483
 
4424
4484
  ```output
@@ -4431,7 +4491,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,emai
4431
4491
  ```
4432
4492
 
4433
4493
  ```bash
4434
- ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --display=data --format=csv | ascli aoc admin res user delete @lines:@stdin: --bulk=yes
4494
+ ascli aoc admin user list --query='@json:{"q":"dummyuser"}' --fields=id --display=data --format=csv | ascli aoc admin user delete @lines:@stdin: --bulk=yes
4435
4495
  ```
4436
4496
 
4437
4497
  ```output
@@ -4446,7 +4506,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --di
4446
4506
  #### Example: Find deactivated users since more than 2 years
4447
4507
 
4448
4508
  ```ruby
4449
- ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
4509
+ ascli aoc admin user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
4450
4510
  ```
4451
4511
 
4452
4512
  To delete them use the same method as before
@@ -4478,7 +4538,7 @@ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key
4478
4538
  #### Example: Display transfer events (ops/transfer)
4479
4539
 
4480
4540
  ```bash
4481
- ascli aoc admin res node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
4541
+ ascli aoc admin node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
4482
4542
  ```
4483
4543
 
4484
4544
  Examples of query:
@@ -4494,13 +4554,13 @@ Examples of query:
4494
4554
  #### Example: Display node events (events)
4495
4555
 
4496
4556
  ```bash
4497
- ascli aoc admin res node --secret=_secret_ v3 events
4557
+ ascli aoc admin node --secret=_secret_ v3 events
4498
4558
  ```
4499
4559
 
4500
4560
  #### Example: Display members of a workspace
4501
4561
 
4502
4562
  ```bash
4503
- ascli aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
4563
+ ascli aoc admin workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
4504
4564
  ```
4505
4565
 
4506
4566
  ```output
@@ -4508,7 +4568,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
4508
4568
  | member_type | manager | member.email |
4509
4569
  +-------------+---------+----------------------------------+
4510
4570
  | user | true | john.curtis@email.com |
4511
- | user | false | someuser@example.com |
4571
+ | user | false | someuser@example.com |
4512
4572
  | user | false | jean.dupont@me.com |
4513
4573
  | user | false | another.user@example.com |
4514
4574
  | group | false | |
@@ -4528,20 +4588,20 @@ a- Get id of first workspace
4528
4588
 
4529
4589
  ```bash
4530
4590
  WS1='First Workspace'
4531
- WS1ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
4591
+ WS1ID=$(ascli aoc admin workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
4532
4592
  ```
4533
4593
 
4534
4594
  b- Get id of second workspace
4535
4595
 
4536
4596
  ```bash
4537
4597
  WS2='Second Workspace'
4538
- WS2ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
4598
+ WS2ID=$(ascli aoc admin workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
4539
4599
  ```
4540
4600
 
4541
4601
  c- Extract membership information
4542
4602
 
4543
4603
  ```bash
4544
- ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp --output=ws1_members.json
4604
+ ascli aoc admin workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp --output=ws1_members.json
4545
4605
  ```
4546
4606
 
4547
4607
  d- Convert to creation data for second workspace:
@@ -4559,13 +4619,13 @@ jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"
4559
4619
  e- Add members to second workspace
4560
4620
 
4561
4621
  ```bash
4562
- ascli aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
4622
+ ascli aoc admin workspace_membership create --bulk=yes @json:@file:ws2_members.json
4563
4623
  ```
4564
4624
 
4565
4625
  #### Example: Get users who did not log since a date
4566
4626
 
4567
4627
  ```bash
4568
- ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
4628
+ ascli aoc admin user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
4569
4629
  ```
4570
4630
 
4571
4631
  ```output
@@ -4580,7 +4640,7 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
4580
4640
  #### Example: List **Limited** users
4581
4641
 
4582
4642
  ```bash
4583
- ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
4643
+ ascli aoc admin user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
4584
4644
  ```
4585
4645
 
4586
4646
  #### Example: Create a group, add to workspace and add user to group
@@ -4588,7 +4648,7 @@ ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_wor
4588
4648
  - Create the group and take note of `id`
4589
4649
 
4590
4650
  ```bash
4591
- ascli aoc admin res group create @json:'{"name":"group 1","description":"my super group"}'
4651
+ ascli aoc admin group create @json:'{"name":"group 1","description":"my super group"}'
4592
4652
  ```
4593
4653
 
4594
4654
  Group: `11111`
@@ -4596,7 +4656,7 @@ Group: `11111`
4596
4656
  - Get the workspace id
4597
4657
 
4598
4658
  ```bash
4599
- ascli aoc admin res workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
4659
+ ascli aoc admin workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
4600
4660
  ```
4601
4661
 
4602
4662
  Workspace: 22222
@@ -4604,13 +4664,13 @@ Workspace: 22222
4604
4664
  - Add group to workspace
4605
4665
 
4606
4666
  ```bash
4607
- ascli aoc admin res workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
4667
+ ascli aoc admin workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
4608
4668
  ```
4609
4669
 
4610
4670
  - Get a user's id
4611
4671
 
4612
4672
  ```bash
4613
- ascli aoc admin res user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
4673
+ ascli aoc admin user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
4614
4674
  ```
4615
4675
 
4616
4676
  User: 33333
@@ -4618,7 +4678,7 @@ User: 33333
4618
4678
  - Add user to group
4619
4679
 
4620
4680
  ```bash
4621
- ascli aoc admin res group_membership create @json:'{"group_id":11111,"member_type":"user","member_id":33333}'
4681
+ ascli aoc admin group_membership create @json:'{"group_id":11111,"member_type":"user","member_id":33333}'
4622
4682
  ```
4623
4683
 
4624
4684
  #### Example: Perform a multi Gbps transfer between two remote shared folders
@@ -4666,7 +4726,7 @@ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AW
4666
4726
  #### Example: Create registration key to register a node
4667
4727
 
4668
4728
  ```bash
4669
- ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
4729
+ ascli aoc admin client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
4670
4730
  ```
4671
4731
 
4672
4732
  ```output
@@ -4676,7 +4736,7 @@ jfqslfdjlfdjfhdjklqfhdkl
4676
4736
  #### Example: Delete all registration keys
4677
4737
 
4678
4738
  ```bash
4679
- ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete @lines:@stdin: --bulk=yes
4739
+ ascli aoc admin client list --fields=id --format=csv|ascli aoc admin client delete @lines:@stdin: --bulk=yes
4680
4740
  ```
4681
4741
 
4682
4742
  ```output
@@ -4727,7 +4787,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
4727
4787
  Then use the returned address for the `url` key to actually create the AoC Node entity:
4728
4788
 
4729
4789
  ```bash
4730
- ascli aoc admin res node create @json:'{"name":"myname","access_key":"myaccesskeyid","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
4790
+ ascli aoc admin node create @json:'{"name":"myname","access_key":"myaccesskeyid","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
4731
4791
  ```
4732
4792
 
4733
4793
  Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
@@ -4794,13 +4854,13 @@ ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":
4794
4854
  #### Example: Send a package to a shared inbox with metadata
4795
4855
 
4796
4856
  ```bash
4797
- ascli aoc packages send --workspace=eudemo @json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
4857
+ ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
4798
4858
  ```
4799
4859
 
4800
4860
  It is also possible to use identifiers and API parameters:
4801
4861
 
4802
4862
  ```bash
4803
- ascli aoc packages send --workspace=eudemo @json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
4863
+ ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
4804
4864
  ```
4805
4865
 
4806
4866
  #### Example: List packages in a given shared inbox
@@ -4903,13 +4963,47 @@ ascli aoc files download <single file path>
4903
4963
 
4904
4964
  #### Shared folders
4905
4965
 
4906
- Shared folder created by users are managed through **permissions**.
4966
+ Like in AoC web UI, "Shared Folders" can be created and shared with either **Private** or **Public** links.
4967
+ **Private** links require the collaborator to log-in to access the shared folder.
4968
+ **Public** links include a passcode that enables the user to access the shared folder without login-in.
4969
+
4970
+ Shared folders can be created either:
4971
+
4972
+ - by users in a workspace: they can share personal folders with other users in the same workspace: `aoc files perm`
4973
+ - by administrators: they can share a folder with users in any workspace: `aoc admin node do [node id] perm`
4974
+
4975
+ Technically (API), shared folder are managed through [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction) on node and an event is sent to AoC to create a **link** in the user's home folder to the shared folder.
4976
+ In both cases, it is necessary to specify a workspace.
4977
+
4978
+ The basic payload (last argument at creation usually specified with `@json:`) is:
4979
+
4980
+ ```json
4981
+ {
4982
+ "file_id": "50",
4983
+ "access_levels": ["list","read","write","delete","mkdir","rename","preview"],
4984
+ "access_type": "user",
4985
+ "access_id": "john@example.com",
4986
+ "tags": {...},
4987
+ }
4988
+ ```
4989
+
4990
+ `ascli` expects the same payload for creation.
4991
+ `ascli` automatically populates this payload like this:
4992
+
4993
+ - `file_id` : the id of the folder to share whose path is specified in the command line
4994
+ - `access_levels` : are set by default to full access.
4995
+ - `tags` : are set with expected values for AoC: user name who creates, and workspace in which the shared folder is created.
4996
+ - `access_type` and `access_id` : need to be set by the user, or using special key as follows.
4997
+
4998
+ To change `access_levels`, just provide the list of levels in the `@json:` payload.
4999
+
5000
+ In order to declare/create the shared folder in the workspace, a special value for `access_id` is used: `ASPERA_ACCESS_KEY_ADMIN_WS_[workspace id]]`. This is conveniently set by `ascli` using an empty string for the pseudo key `with`. In order to share a folder with a different, special tags are set, but this is conveniently done by `ascli` using the `as` key.
4907
5001
 
4908
- For creation, parameters are the same as for node API [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
4909
- `ascli` expects the same payload for creation, but it will automatically populate required tags if needed.
5002
+ The following optional additional helper keys are supported by `ascli`:
4910
5003
 
4911
- Also, the pseudo key `with` is available: it will lookup the name in the contacts and fill the proper type and id.
4912
- The pseudo parameter `link_name` allows changing default **shared as** name.
5004
+ - `with` : Recipient of shared folder. Can be a user name, a group name, or a workspace name. `ascli` will resolve the name to the proper type and id in fields `access_type` and `access_id`. If the value is the empty string, then it declares the shared folder in the workspace (first action to do, see below).
5005
+ - `link_name` : The name of the link file created in the user's home folder for private links.
5006
+ - `as` : The name of the link file created in the user's home folder for admin shared folders.
4913
5007
 
4914
5008
  - List permissions on a shared folder as user
4915
5009
 
@@ -4932,12 +5026,24 @@ ascli aoc files perm /shared_folder_test1 delete 6161
4932
5026
  Public and Private short links can be managed with command:
4933
5027
 
4934
5028
  ```bash
4935
- ascli aoc files short_link private create _path_here_
4936
- ascli aoc files short_link private list _path_here_
4937
- ascli aoc files short_link public list _path_here_
5029
+ ascli aoc files short_link _path_here_ private create
5030
+ ascli aoc files short_link _path_here_ private list
5031
+ ascli aoc files short_link _path_here_ public list
4938
5032
  ascli aoc files short_link public delete _id_
4939
5033
  ```
4940
5034
 
5035
+ - Create an admin shared folder and shared with a user or group or workspace
5036
+
5037
+ ```bash
5038
+ ascli aoc admin node do 1234 mkdir folder_on_node
5039
+ ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"","as":"folder_for_users"}' --workspace="my ws"
5040
+ ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"john@example.com","as":"folder_for_users"}' --workspace="my ws"
5041
+ ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"group 1","as":"folder_for_users"}' --workspace="my ws"
5042
+ ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"my ws","as":"folder_for_users"}' --workspace="my ws"
5043
+ ```
5044
+
5045
+ > **Note:** The node can be conveniently identified using the **percent selector** instead of numerical id.
5046
+
4941
5047
  #### Cross Organization transfers
4942
5048
 
4943
5049
  It is possible to transfer files directly between organizations without having to first download locally and then upload...
@@ -4975,7 +5081,7 @@ The command `aoc files find` allows to search for files in a given workspace.
4975
5081
  It works also on `node` resource using the `v4` command:
4976
5082
 
4977
5083
  ```bash
4978
- ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find ...
5084
+ ascli aoc admin node --name='my node name' --secret='my_secret_here' v4 find ...
4979
5085
  ```
4980
5086
 
4981
5087
  For instructions, refer to section `find` for plugin `node`.
@@ -4987,7 +5093,7 @@ For instructions, refer to section `find` for plugin `node`.
4987
5093
  ```bash
4988
5094
  admin analytics transfers nodes
4989
5095
  admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
4990
- admin analytics transfers users --once_only=yes
5096
+ admin analytics transfers users --once-only=yes
4991
5097
  admin application list
4992
5098
  admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
4993
5099
  admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
@@ -5009,12 +5115,17 @@ admin dropbox list
5009
5115
  admin dropbox_membership list
5010
5116
  admin group list
5011
5117
  admin kms_profile list
5012
- admin node do %name:my_ak_name --secret=my_ak_secret browse /
5013
- admin node do %name:my_ak_name --secret=my_ak_secret delete /folder1
5014
- admin node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
5015
- admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
5016
- admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
5017
- admin node do %name:my_ak_name --secret=my_ak_secret v3 events
5118
+ admin node do %name:my_node_name --secret=my_ak_secret browse /
5119
+ admin node do %name:my_node_name --secret=my_ak_secret delete /folder1
5120
+ admin node do %name:my_node_name --secret=my_ak_secret mkdir /folder1
5121
+ admin node do %name:my_node_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
5122
+ admin node do %name:my_node_name --secret=my_ak_secret v3 access_key delete testsub1
5123
+ admin node do %name:my_node_name --secret=my_ak_secret v3 events
5124
+ admin node do %name:my_node_name delete test_shared_folder
5125
+ admin node do %name:my_node_name mkdir test_shared_folder
5126
+ admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
5127
+ admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"my_user_email","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
5128
+ admin node do %name:my_node_name perm test_shared_folder create @json:'{"with":"my_user_group","as":"other_name_shared"}' --workspace=my_workspace_shared_inbox
5018
5129
  admin node list
5019
5130
  admin operation list
5020
5131
  admin organization show
@@ -5047,22 +5158,27 @@ files browse my_remote_folder/
5047
5158
  files delete /testsrc
5048
5159
  files download --transfer=alpha testdst/test_file.bin
5049
5160
  files download --transfer=connect testdst/test_file.bin
5161
+ files find /
5050
5162
  files find / '\.partial$'
5163
+ files find / @ruby:'->(f){f["type"].eql?("file")}'
5051
5164
  files http_node_download --to-folder=. testdst/test_file.bin
5052
5165
  files mkdir /testsrc
5053
5166
  files modify my_test_folder
5054
5167
  files permission my_test_folder list
5055
5168
  files rename /some_folder testdst
5056
- files short_link private create /testdst
5057
- files short_link private list /testdst
5058
- files short_link public create testdst
5169
+ files short_link /testdst private create
5170
+ files short_link /testdst private list
5171
+ files short_link /testdst public create
5059
5172
  files show %id:aoc_file_id
5060
5173
  files show /
5061
5174
  files show testdst/test_file.bin
5062
5175
  files sync admin status --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
5063
5176
  files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
5177
+ files sync start --sync-info=@json:'{"name":"my_aoc_sync1","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
5064
5178
  files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
5065
5179
  files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
5180
+ files sync start pull /testdst /data/local_sync --sync-info=@json:'{"reset":true,"quiet":false}'
5181
+ files sync start pull /testdst /data/local_sync --sync-info=@json:'{"sessions":[{"reset":true}]}'
5066
5182
  files thumbnail my_test_folder/video_file.mpg
5067
5183
  files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
5068
5184
  files transfer push /testsrc --to-folder=/testdst test_file.bin
@@ -5089,8 +5205,10 @@ packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["
5089
5205
  packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_aoc_user --password=my_public_link_send_use_pass
5090
5206
  packages send @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin --url=my_public_link_send_shared_inbox
5091
5207
  packages shared_inboxes list
5208
+ packages shared_inboxes show %name:my_shared_inbox_name
5092
5209
  remind --username=my_user_email
5093
5210
  servers
5211
+ tier_restrictions
5094
5212
  user pref modify @json:'{"default_language":"en-us"}'
5095
5213
  user pref show
5096
5214
  user profile modify @json:'{"name":"dummy change"}'
@@ -5247,11 +5365,18 @@ cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
5247
5365
  ## Plugin: `server`: IBM Aspera High Speed Transfer Server (SSH)
5248
5366
 
5249
5367
  The `server` plugin is used for operations on Aspera HSTS using SSH authentication.
5250
- It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
5368
+ It is the original way of accessing an Aspera Server, often used for server to server transfers.
5251
5369
  An SSH session is established, authenticated with either a password or an SSH private key,
5252
5370
  then commands `ascp` (for transfers) and `ascmd` (for file operations) are executed.
5253
5371
 
5254
- > **Note:** The URL to be provided is usually: `ssh://_server_address_:33001`
5372
+ The URL to be provided with option `url` shall be like `ssh://_server_address_:33001`, then option `username` is used to specify the transfer user, and finally either option `password` or `ssh_keys` (with one or several paths) for the authentication.
5373
+
5374
+ Typically:
5375
+
5376
+ ```console
5377
+ `ascli` server --url=ssh://hsts.example.com:33001 --username=john --password=_something_here_ ...
5378
+ `ascli` server --url=ssh://hsts.example.com:33001 --username=john --ssh-keys=~/.ssh/id_rsa ...
5379
+ ```
5255
5380
 
5256
5381
  ### Server sample commands
5257
5382
 
@@ -5301,7 +5426,8 @@ If SSH is the session protocol (by default i.e. not WSS), then following session
5301
5426
 
5302
5427
  If `username` is not provided then the default transfer user `xfer` is used.
5303
5428
 
5304
- If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used.
5429
+ If neither SSH password nor key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass key(s) is used.
5430
+
5305
5431
  Example:
5306
5432
 
5307
5433
  ```bash
@@ -5420,11 +5546,13 @@ Native API parameters can be placed in option `query`.
5420
5546
 
5421
5547
  Special parameters can be placed in option `query` for "gen3" browse:
5422
5548
 
5423
- | Parameter | Description |
5424
- |-----------|-------------|
5549
+ | Parameter | Description |
5550
+ |-------------|-------------|
5425
5551
  | `recursive` | Recursively list files |
5426
- | `max` | Maximum number of files to list |
5427
- | `self` | Offset in the list |
5552
+ | `max` | Maximum number of files to list |
5553
+ | `self` | Offset in the list |
5554
+
5555
+ Option `node_cache` can be set to `no` to avoid use of folder cache (Redis) and force actual read of file system.
5428
5556
 
5429
5557
  ### Operation `find` on **gen4/access key**
5430
5558
 
@@ -5501,6 +5629,24 @@ ascli node access_keys do self find / @ruby:'->(f){f["type"].eql?("file") and (D
5501
5629
 
5502
5630
  > **Note:** The pipe `|` character on the last line.
5503
5631
 
5632
+ ### Listing transfer events
5633
+
5634
+ When a transfer is run, its information is stored (typicall, 1 day) in the HSTS database (Redis).
5635
+ This information can be retrieved with command: `transfer list`.
5636
+
5637
+ If the number of transfer is too large, then the list will be retrieved in several API calls.
5638
+
5639
+ In addition, it is possible to list "only new information" using option `once_only`.
5640
+
5641
+ ```bash
5642
+ ascli node transfer list --once-only=yes
5643
+ ```
5644
+
5645
+ The `iteratin_token` that keeps memory of latest event is stored in the persistance repository of `ascli`.
5646
+ To reset it, add option: `--query=@json:'{"reset": true}'`.
5647
+ To list only a number of events, use the `max` parameter in query.
5648
+ Other parameters are directly transmitted to the underlying API (`GET /ops/transfers`).
5649
+
5504
5650
  ### Central
5505
5651
 
5506
5652
  The central subcommand uses the **reliable query** API (session and file).
@@ -5555,8 +5701,8 @@ Follow the Aspera Transfer Server configuration to activate this feature.
5555
5701
 
5556
5702
  The following command lists one file that requires validation, and assign it to the unique validator identifier provided:
5557
5703
 
5558
- ```bash
5559
- ascli node central file list --validator=ascli --data=@json:'{"file_transfer_filter":{"max_result":1}}'
5704
+ ```json
5705
+ ascli node central file list --validator=ascli @json:'{"file_transfer_filter":{"max_result":1}}'
5560
5706
  ```
5561
5707
 
5562
5708
  ```output
@@ -5569,8 +5715,8 @@ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_fil
5569
5715
 
5570
5716
  To update the status of the file, use the following command:
5571
5717
 
5572
- ```bash
5573
- ascli node central file update --validator=ascli --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
5718
+ ```json
5719
+ ascli node central file update --validator=ascli @json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
5574
5720
  ```
5575
5721
 
5576
5722
  ```output
@@ -5644,11 +5790,11 @@ ascli node info --field=@ruby:'/^access_key_configuration_capabilities.*/'
5644
5790
 
5645
5791
  Bearer tokens are part of the **gen4/access key** API.
5646
5792
  It follows the model of OAuth 2.
5647
- For example they are used in Aspera on Cloud.
5793
+ For example, they are used in Aspera on Cloud.
5648
5794
  This is also available for developers for any application integrating Aspera.
5649
5795
  In this API, files, users and groups are identified by an id (a String, e.g. `"125"`, not necessarily numerical).
5650
5796
 
5651
- Bearer tokens are typically generated by the authentication application, and then recognized by the node API.
5797
+ Bearer tokens are typically generated by the authentication application and then recognized by the node API.
5652
5798
  A bearer token is authorized on the node by creating `permissions` on a **folder**.
5653
5799
 
5654
5800
  Bearer tokens can be generated using command `bearer_token`: it takes two arguments:
@@ -5674,6 +5820,11 @@ Bearer tokens can be generated using command `bearer_token`: it takes two argume
5674
5820
 
5675
5821
  #### Bearer token: Environment
5676
5822
 
5823
+ An access key shall be created to grant access for transfers to its storage.
5824
+ The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
5825
+
5826
+ They way to create access keys depend slightly on the type of HSTS:
5827
+
5677
5828
  - If a self-managed Aspera node is used, then a **node user admin** must be created:
5678
5829
  It has no `docroot` but has at least one file restriction (for testing, one can use `*` to accept creation of an access key with any storage root path).
5679
5830
  Refer to the Aspera HSTS documentation.
@@ -5682,12 +5833,10 @@ Bearer tokens can be generated using command `bearer_token`: it takes two argume
5682
5833
 
5683
5834
  - If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
5684
5835
 
5685
- - An access key shall be created to grant access for transfers to its storage.
5686
- The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
5687
-
5688
5836
  #### Bearer token: Preparation
5689
5837
 
5690
5838
  Let's assume that the access key was created, and a default configuration is set to use this **access key**.
5839
+ Using `ascli`, an access key can be created using the `access_key create` on the node (using main node credentials) or ATS.
5691
5840
 
5692
5841
  - Create a private key (organization key) that will be used to sign bearer tokens:
5693
5842
 
@@ -5707,6 +5856,7 @@ Let's assume that the access key was created, and a default configuration is set
5707
5856
 
5708
5857
  > **Note:** Either the public or private key can be provided, and only the public key is used.
5709
5858
  > This will enable to check the signature of the bearer token.
5859
+ > Above command is executed with access key credentials.
5710
5860
 
5711
5861
  Alternatively, use the following equivalent command, as `ascli` kindly extracts the public key with extension `.pub`:
5712
5862
 
@@ -5768,18 +5918,18 @@ ascli node -N --url=... --password="Bearer $(cat bearer.txt)" --root-id=$my_fold
5768
5918
  access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
5769
5919
  access_key delete my_username
5770
5920
  access_key do my_ak_name browse /
5771
- access_key do my_ak_name delete /folder2
5772
- access_key do my_ak_name delete testfile1
5773
- access_key do my_ak_name download testfile1 --to-folder=.
5921
+ access_key do my_ak_name delete /test_nd_ak2
5922
+ access_key do my_ak_name delete test_nd_ak3
5923
+ access_key do my_ak_name download test_nd_ak3 --to-folder=.
5774
5924
  access_key do my_ak_name find my_test_folder
5775
5925
  access_key do my_ak_name find my_test_folder @ruby:'->(f){f["name"].end_with?(".jpg")}'
5776
5926
  access_key do my_ak_name find my_test_folder exec:'f["name"].end_with?(".jpg")'
5777
- access_key do my_ak_name mkdir /folder1
5927
+ access_key do my_ak_name mkdir /tst_nd_ak
5778
5928
  access_key do my_ak_name node_info /
5779
- access_key do my_ak_name rename /folder1 folder2
5929
+ access_key do my_ak_name rename /tst_nd_ak test_nd_ak2
5780
5930
  access_key do my_ak_name show %id:1
5781
- access_key do my_ak_name show /testfile1
5782
- access_key do my_ak_name upload 'faux:///testfile1?1k' --default_ports=no
5931
+ access_key do my_ak_name show /test_nd_ak3
5932
+ access_key do my_ak_name upload 'faux:///test_nd_ak3?100k' --default-ports=no
5783
5933
  access_key do self permission %id:root_id create @json:'{"access_type":"user","access_id":"666"}'
5784
5934
  access_key do self show / --fields=id --output=root_id
5785
5935
  access_key list
@@ -5797,7 +5947,7 @@ basic_token
5797
5947
  bearer_token @file:my_private_key @json:'{"user_id":"666"}' --output=bearer_666
5798
5948
  browse / --log-level=trace2
5799
5949
  central file list
5800
- central file modify --validator=1 --query=@json:'{"files":[]}'
5950
+ central file modify --validator=1 @json:'{"files":[]}'
5801
5951
  central session list
5802
5952
  delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
5803
5953
  delete my_upload_folder/test_file.bin
@@ -5830,9 +5980,11 @@ ssync summary %name:my_node_sync
5830
5980
  stream list
5831
5981
  sync admin status --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
5832
5982
  sync admin status --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
5983
+ sync start --sync-info=@json:'{"name":"my_node_sync1","reset":true,"quiet":false,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
5833
5984
  sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
5834
- sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
5985
+ transfer list --once-only=yes
5835
5986
  transfer list --query=@json:'{"active_only":true}'
5987
+ transfer list --query=@json:'{"reset":true}' --once-only=yes
5836
5988
  transfer sessions
5837
5989
  transport
5838
5990
  upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
@@ -6005,7 +6157,7 @@ admin event web
6005
6157
  admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
6006
6158
  admin metadata_profiles list
6007
6159
  admin node list
6008
- admin oauth_clients list
6160
+ admin oauth_clients list --query=@json:'[["client_types[]","public"]]'
6009
6161
  admin registrations list
6010
6162
  admin saml_configs list
6011
6163
  admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
@@ -6198,26 +6350,26 @@ To keep the content encrypted, use option: `--ts=@json:'{"content_protection":nu
6198
6350
  If you are a regular user, to list work groups you belong to:
6199
6351
 
6200
6352
  ```bash
6201
- ascli faspex5 admin res workgroup list
6353
+ ascli faspex5 admin workgroup list
6202
6354
  ```
6203
6355
 
6204
6356
  If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
6205
6357
  Example:
6206
6358
 
6207
6359
  ```bash
6208
- ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
6360
+ ascli faspex5 admin shared list --query=@json:'{"all":true}' --fields=id,name
6209
6361
  ```
6210
6362
 
6211
6363
  Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
6212
6364
 
6213
6365
  ```bash
6214
- ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' john@example.com
6366
+ ascli faspex5 admin shared_inboxes invite '%name:the shared inbox' john@example.com
6215
6367
  ```
6216
6368
 
6217
6369
  It is equivalent to:
6218
6370
 
6219
6371
  ```bash
6220
- ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
6372
+ ascli faspex5 admin shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
6221
6373
  ```
6222
6374
 
6223
6375
  Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
@@ -6229,13 +6381,13 @@ Other payload parameters are possible for `invite` in this last `Hash` **Command
6229
6381
  ### Faspex 5: Create Metadata profile
6230
6382
 
6231
6383
  ```bash
6232
- ascli faspex5 admin res metadata_profiles create @json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
6384
+ ascli faspex5 admin metadata_profiles create @json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
6233
6385
  ```
6234
6386
 
6235
6387
  ### Faspex 5: Create a Shared inbox with specific metadata profile
6236
6388
 
6237
6389
  ```bash
6238
- ascli faspex5 admin res shared create @json:'{"name":"the shared inbox","metadata_profile_id":1}'
6390
+ ascli faspex5 admin shared create @json:'{"name":"the shared inbox","metadata_profile_id":1}'
6239
6391
  ```
6240
6392
 
6241
6393
  ### Faspex 5: List content in Shared folder and send package from remote source
@@ -6282,6 +6434,10 @@ There are two types of invitations of package submission: public or private.
6282
6434
 
6283
6435
  Public invitations are for external users, provide just the email address.
6284
6436
 
6437
+ ```bash
6438
+ ascli faspex5 invitations create @json:'{"email_address":"john@example.com"}' --fields=access_url
6439
+ ```
6440
+
6285
6441
  Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
6286
6442
 
6287
6443
  ### Faspex 5: Cleanup packages
@@ -6362,7 +6518,30 @@ Then, the postprocessing script executed will be `script1.sh`.
6362
6518
 
6363
6519
  Environment variables at set to the values provided by the web hook which are the same as Faspex 4 postprocessing.
6364
6520
 
6365
- ### Faspex 5: Missing commands
6521
+ ### Faspex 5: Faspex 4 Gateway
6522
+
6523
+ > **Note:** This is not a feature for production. It's provided for testing only.
6524
+
6525
+ For legacy faspex client applications that use the `send` API (only) of Faspex v4, the command `gateway` provides the capability to present an API compatible with Faspex 4, and it will call the Faspex 5 API.
6526
+
6527
+ It takes a single argument which is the url at which the gateway will be located (locally):
6528
+
6529
+ ```bash
6530
+ ascli faspex5 gateway https://localhost:12345/aspera/faspex
6531
+ ```
6532
+
6533
+ There are many limitations:
6534
+
6535
+ - It's only to emulate the Faspex 4 `send` API (send package)
6536
+ - No support for remote sources, only for an actual file transfer by the client
6537
+ - The client must use the transfer spec returned by the API (not faspe: URL)
6538
+ - tags returned in transfer spec must be used in transfer
6539
+ - only a single authentication is possible (per gateway) on Faspex5
6540
+ - no authentication of F4 side (ignored)
6541
+
6542
+ Behavior: The API client calls the Faspex 4 API on the gateway, then the gateway transforms this into a Faspex5 API call, which returns a transfer spec, which is returned to the calling client. The calling client uses this to start a transfer to HSTS which is actually managed by Faspex 5.
6543
+
6544
+ ### Faspex 5: Get Bearer token to use API
6366
6545
 
6367
6546
  If a command is missing, then it is still possible to execute command by calling directly the API on the command line using `curl`:
6368
6547
 
@@ -6552,7 +6731,7 @@ package receive f4_prs2 --to-folder=. --box=sent
6552
6731
  package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_dbx"]}' test_file.bin
6553
6732
  package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["*my_wkg"]}' test_file.bin
6554
6733
  package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal","my_username"]}' test_file.bin
6555
- package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"]}' --remote_source=%name:my_src sample_source.txt
6734
+ package send --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_email_internal"]}' --remote-source=%name:my_src sample_source.txt
6556
6735
  package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
6557
6736
  package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"$(notdir test) PACKAGE_TITLE_BASE"}' test_file.bin
6558
6737
  source info %name:my_src --storage=@preset:faspex4_storage
@@ -6596,12 +6775,12 @@ To figure out the entities payload, for example for creation, refer to the API d
6596
6775
  admin group all list
6597
6776
  admin node list
6598
6777
  admin share list --fields=DEF,-status,status_message
6599
- admin share user_permissions 1 list
6778
+ admin share user_permissions 3 list
6600
6779
  admin user all app_authorizations 1 modify @json:'{"app_login":true}'
6601
6780
  admin user all app_authorizations 1 show
6602
6781
  admin user all list
6603
6782
  admin user all share_permissions 1 list
6604
- admin user all share_permissions 1 show 1
6783
+ admin user all share_permissions 1 show 3
6605
6784
  admin user ldap add the_name
6606
6785
  admin user local list
6607
6786
  admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
@@ -6609,9 +6788,10 @@ files browse /
6609
6788
  files delete my_share1/new_folder
6610
6789
  files delete my_share1/test_file.bin
6611
6790
  files download --to-folder=. my_share1/test_file.bin
6612
- files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
6791
+ files download --to-folder=. my_share1/test_file.bin my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
6613
6792
  files mkdir my_share1/new_folder
6614
6793
  files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
6794
+ files upload --to-folder=https://shares.share1 sendfolder --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
6615
6795
  files upload --to-folder=my_share1 test_file.bin
6616
6796
  files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
6617
6797
  health
@@ -6677,7 +6857,7 @@ ascli config preset update mycos --bucket=mybucket --endpoint=https://s3.us-east
6677
6857
  ascli config preset set default cos mycos
6678
6858
  ```
6679
6859
 
6680
- Then, jump to the transfer example.
6860
+ Then, jump to the [transfer example](#operations-transfers).
6681
6861
 
6682
6862
  ### Using service credential file
6683
6863
 
@@ -6752,14 +6932,13 @@ ascli cos node info
6752
6932
  ascli cos node upload 'faux:///sample1G?1g'
6753
6933
  ```
6754
6934
 
6755
- > **Note:** A dummy file `sample1G` of size 2GB is generated using the `faux` PVCL (man `ascp` and section above), but you can, of course, send a real file by specifying a real file instead.
6935
+ > **Note:** A dummy file `sample1G` of size 2GB is generated using the `faux` PVCL (man `ascp` and section above), but you can, of course, send a real file by specifying a real file path instead.
6756
6936
 
6757
6937
  ### Cos sample commands
6758
6938
 
6759
6939
  > **Note:** Add `ascli cos` in front of the commands:
6760
6940
 
6761
6941
  ```bash
6762
- node access_key show self
6763
6942
  node download test_file.bin --to-folder=.
6764
6943
  node info --bucket=my_bucket --endpoint=my_endpoint --apikey=my_api_key --crn=my_resource_instance_id
6765
6944
  node info --bucket=my_bucket --region=my_region --service-credentials=@json:@file:my_cos_svc_cred
@@ -6775,6 +6954,7 @@ node upload test_file.bin
6775
6954
 
6776
6955
  ```bash
6777
6956
  health
6957
+ info
6778
6958
  ```
6779
6959
 
6780
6960
  ## Plugin: `faspio`: Faspio Gateway
@@ -6799,7 +6979,8 @@ Retrieve information on subscription.
6799
6979
  > **Note:** Add `ascli alee` in front of the commands:
6800
6980
 
6801
6981
  ```bash
6802
- health
6982
+ entitlement
6983
+ health -N
6803
6984
  ```
6804
6985
 
6805
6986
  ## Plugin: `preview`: Preview generator for AoC
@@ -7124,28 +7305,52 @@ An interface for the `async` utility is provided in the following plugins:
7124
7305
  - `aoc files sync` (uses node)
7125
7306
  - `shares files sync` (uses node)
7126
7307
 
7127
- The main advantage over the `async` command line when using `server` is the possibility to use a configuration file, using standard options of `ascli`.
7308
+ One advantage over the `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
7309
+ Moreover `ascli` supports sync with application requiring token-based authorization.
7128
7310
 
7129
- In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
7311
+ Some of the `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
7130
7312
 
7131
- > **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
7313
+ > **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`). The Transfer SDK 1.3 includes this.
7132
7314
 
7133
- Two JSON syntax are supported for option `sync_info`.
7315
+ `sync` supports 0 or 3 arguments:
7134
7316
 
7135
- ### `async` JSON: API format
7317
+ - direction
7318
+ - local folder
7319
+ - remote folder
7320
+
7321
+ Additional options can be provided with option `sync_info`, for which two syntax are possible, as follows.
7322
+
7323
+ ### `async` API and conf format
7136
7324
 
7137
7325
  It is the same payload as specified on the option `--conf` of `async` or in node API `/asyncs`.
7138
7326
  This is the preferred syntax and allows a single session definition.
7139
- But there is no progress output nor error messages.
7327
+
7328
+ > **Note:** No progress, nor error messages is provided on terminal.
7140
7329
 
7141
7330
  Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
7142
7331
 
7143
- ### `async` JSON: Options mapping
7332
+ If 3 arguments are provided they are mapped to:
7333
+
7334
+ - `direction`
7335
+ - `local.path`
7336
+ - `remote.path`
7337
+
7338
+ ### `async` options mapping
7144
7339
 
7145
7340
  `ascli` defines a JSON equivalent to regular `async`options.
7146
7341
  It is based on a JSON representation of `async` command line options.
7147
7342
  It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
7148
7343
 
7344
+ This is the mode selection if there is either keys `sessions` or `instance` in option `sync_info`.
7345
+
7346
+ > **Note:** Progress and error messages are provided on terminal like regular command line invocation of `async`.
7347
+
7348
+ If 3 arguments are provided, they are applied to the first (and only) session and mapped to:
7349
+
7350
+ - `direction`
7351
+ - `local_dir`
7352
+ - `remote_dir`
7353
+
7149
7354
  ## Hot folder
7150
7355
 
7151
7356
  ### Requirements
@@ -7338,10 +7543,12 @@ A default e-mail template is used, but it can be overridden with option `notify_
7338
7543
 
7339
7544
  The environment provided contains the following additional variables:
7340
7545
 
7341
- - `subject`
7342
- - `body`
7343
- - `global_transfer_status`
7344
- - `ts`
7546
+ - `subject` : a default subject including transfer status
7547
+ - `status` : global status of transfer
7548
+ - `ts` : the [**transfer-spec**](#transfer-specification) used for the transfer
7549
+ - `from_email` : email of sender (from `smtp` configuration)
7550
+ - `from_name` : name of sender (from `smtp` configuration)
7551
+ - `to` : recipient of the email (from `notify_to`)
7345
7552
 
7346
7553
  Example of template:
7347
7554
 
@@ -7350,7 +7557,7 @@ From: <%=from_name%> <<%=from_email%>>
7350
7557
  To: <<%=to%>>
7351
7558
  Subject: <%=subject%>
7352
7559
 
7353
- Transfer is: <%=global_transfer_status%>
7560
+ Transfer is: <%=status%>
7354
7561
  ```
7355
7562
 
7356
7563
  ## Tool: `asession`
@@ -7434,21 +7641,25 @@ asession -h
7434
7641
  USAGE
7435
7642
  asession
7436
7643
  asession -h|--help
7437
- asession <session spec extended value>
7644
+ asession [<session spec extended value>]
7438
7645
 
7439
7646
  If no argument is provided, default will be used: @json:@stdin
7440
7647
  -h, --help display this message
7441
- <session spec extended value> a dictionary value (Hash)
7648
+ <session spec extended value> a dictionary (Hash)
7442
7649
  The value can be either:
7443
7650
  the JSON description itself, e.g. @json:'{"xx":"yy",...}'
7444
7651
  @json:@stdin, if the JSON is provided from stdin
7445
7652
  @json:@file:<path>, if the JSON is provided from a file
7446
- Parameter spec is mandatory, it contains the transfer spec
7653
+ The following keys are recognized in session spec:
7654
+ spec : mandatory, contains the transfer spec
7655
+ loglevel : modify log level (to stderr)
7656
+ agent : modify transfer agent parameters, e.g. ascp_args
7657
+ file_list_folder : location of temporary files
7658
+ sdk : location of SDK (ascp)
7447
7659
  Asynchronous commands can be provided on STDIN, examples:
7448
7660
  {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
7449
7661
  {"type":"START","source":"xx","destination":"yy"}
7450
7662
  {"type":"DONE"}
7451
- Note: debug information can be placed on STDERR, using the "loglevel" parameter in session spec (debug=0)
7452
7663
  EXAMPLES
7453
7664
  asession @json:'{"spec":{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}}'
7454
7665
  echo '{"spec":{"remote_host":...}}'|asession @json:@stdin