aspera-cli 4.21.2 → 4.22.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (97) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/BUGS.md +1 -1
  4. data/CHANGELOG.md +34 -16
  5. data/CONTRIBUTING.md +6 -10
  6. data/README.md +805 -574
  7. data/examples/get_proto_file.rb +1 -1
  8. data/lib/aspera/agent/base.rb +9 -5
  9. data/lib/aspera/agent/connect.rb +30 -28
  10. data/lib/aspera/agent/desktop.rb +29 -25
  11. data/lib/aspera/agent/direct.rb +137 -125
  12. data/lib/aspera/agent/httpgw.rb +22 -26
  13. data/lib/aspera/agent/node.rb +14 -11
  14. data/lib/aspera/agent/transferd.rb +6 -2
  15. data/lib/aspera/api/aoc.rb +6 -6
  16. data/lib/aspera/api/cos_node.rb +1 -1
  17. data/lib/aspera/api/httpgw.rb +7 -3
  18. data/lib/aspera/api/node.rb +6 -4
  19. data/lib/aspera/ascmd.rb +3 -3
  20. data/lib/aspera/ascp/installation.rb +15 -16
  21. data/lib/aspera/ascp/management.rb +1 -1
  22. data/lib/aspera/assert.rb +11 -2
  23. data/lib/aspera/cli/error.rb +2 -2
  24. data/lib/aspera/cli/extended_value.rb +38 -19
  25. data/lib/aspera/cli/formatter.rb +48 -48
  26. data/lib/aspera/cli/hints.rb +1 -1
  27. data/lib/aspera/cli/main.rb +190 -168
  28. data/lib/aspera/cli/manager.rb +15 -15
  29. data/lib/aspera/cli/plugin.rb +23 -20
  30. data/lib/aspera/cli/plugin_factory.rb +1 -1
  31. data/lib/aspera/cli/plugins/alee.rb +1 -1
  32. data/lib/aspera/cli/plugins/aoc.rb +144 -107
  33. data/lib/aspera/cli/plugins/ats.rb +19 -17
  34. data/lib/aspera/cli/plugins/config.rb +67 -83
  35. data/lib/aspera/cli/plugins/console.rb +5 -3
  36. data/lib/aspera/cli/plugins/faspex.rb +39 -35
  37. data/lib/aspera/cli/plugins/faspex5.rb +104 -80
  38. data/lib/aspera/cli/plugins/faspio.rb +13 -1
  39. data/lib/aspera/cli/plugins/httpgw.rb +13 -1
  40. data/lib/aspera/cli/plugins/node.rb +306 -179
  41. data/lib/aspera/cli/plugins/orchestrator.rb +34 -40
  42. data/lib/aspera/cli/plugins/preview.rb +3 -3
  43. data/lib/aspera/cli/plugins/server.rb +6 -6
  44. data/lib/aspera/cli/plugins/shares.rb +5 -5
  45. data/lib/aspera/cli/sync_actions.rb +19 -18
  46. data/lib/aspera/cli/transfer_agent.rb +5 -5
  47. data/lib/aspera/cli/transfer_progress.rb +2 -2
  48. data/lib/aspera/cli/version.rb +1 -1
  49. data/lib/aspera/command_line_builder.rb +116 -95
  50. data/lib/aspera/coverage.rb +4 -3
  51. data/lib/aspera/environment.rb +6 -6
  52. data/lib/aspera/faspex_gw.rb +14 -14
  53. data/lib/aspera/faspex_postproc.rb +7 -6
  54. data/lib/aspera/hash_ext.rb +2 -2
  55. data/lib/aspera/json_rpc.rb +1 -1
  56. data/lib/aspera/keychain/encrypted_hash.rb +47 -34
  57. data/lib/aspera/keychain/factory.rb +41 -0
  58. data/lib/aspera/keychain/hashicorp_vault.rb +71 -0
  59. data/lib/aspera/keychain/macos_security.rb +19 -11
  60. data/lib/aspera/log.rb +28 -34
  61. data/lib/aspera/nagios.rb +6 -6
  62. data/lib/aspera/node_simulator.rb +8 -8
  63. data/lib/aspera/oauth/base.rb +8 -6
  64. data/lib/aspera/oauth/factory.rb +5 -6
  65. data/lib/aspera/oauth/url_json.rb +6 -6
  66. data/lib/aspera/persistency_action_once.rb +6 -4
  67. data/lib/aspera/persistency_folder.rb +2 -2
  68. data/lib/aspera/preview/generator.rb +1 -1
  69. data/lib/aspera/preview/options.rb +16 -16
  70. data/lib/aspera/preview/terminal.rb +3 -3
  71. data/lib/aspera/preview/utils.rb +11 -13
  72. data/lib/aspera/products/connect.rb +1 -1
  73. data/lib/aspera/products/desktop.rb +1 -1
  74. data/lib/aspera/products/transferd.rb +1 -1
  75. data/lib/aspera/proxy_auto_config.rb +2 -2
  76. data/lib/aspera/rest.rb +52 -43
  77. data/lib/aspera/rest_errors_aspera.rb +1 -1
  78. data/lib/aspera/secret_hider.rb +5 -5
  79. data/lib/aspera/ssh.rb +4 -4
  80. data/lib/aspera/transfer/convert.rb +29 -0
  81. data/lib/aspera/transfer/error_info.rb +66 -66
  82. data/lib/aspera/transfer/parameters.rb +13 -68
  83. data/lib/aspera/transfer/spec.rb +5 -6
  84. data/lib/aspera/transfer/spec.schema.yaml +753 -0
  85. data/lib/aspera/transfer/spec_doc.rb +62 -0
  86. data/lib/aspera/transfer/sync.rb +23 -72
  87. data/lib/aspera/transfer/sync_instance.schema.yaml +13 -0
  88. data/lib/aspera/transfer/sync_session.schema.yaml +79 -0
  89. data/lib/aspera/transfer/uri.rb +6 -6
  90. data/lib/aspera/uri_reader.rb +1 -1
  91. data/lib/aspera/web_auth.rb +1 -1
  92. data/lib/aspera/web_server_simple.rb +53 -44
  93. data.tar.gz.sig +1 -2
  94. metadata +37 -4
  95. metadata.gz.sig +0 -0
  96. data/examples/build_package.sh +0 -28
  97. data/lib/aspera/transfer/spec.yaml +0 -718
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3d66238170234ebc23390a21ef7a492ec50c8707200c774ef2f3195fdb13261f
4
- data.tar.gz: 47623d1ce5961c1d1050989e36afeb1e008dc24c3b81d34d5b484f39d0826107
3
+ metadata.gz: 4142ef4954b98f748d8d0d19c63a53e0faace015b8286480b31b652b994fa15c
4
+ data.tar.gz: 2c77b1badfd80ddbc7c8d98acdb303039c87ee2e7fe64b6fbcecd7935101426c
5
5
  SHA512:
6
- metadata.gz: dfa7a18bfa476bfda979e88017bf643ef181ecd643f53a6872f7b0af833c6e2dc46700d5afc913126942cda6eb0804f7332b5555e8054ab1d9dba97873000e6e
7
- data.tar.gz: 2a598cd0d4aab0fb8ec6e2660f590d78fd41a62870f554f96f26069cf0499753dcedb30f807691d8156441340dbbed2910f560b29d1e6e5bbedfd7416f542825
6
+ metadata.gz: 524c91bde0ca7c7267a18be627ea00f89de4fd6376fc7d93eb75eb204c0507eb15f3800e5d51536f16996fddf3a26831e5dc3ce7a516701db9098b658d098116
7
+ data.tar.gz: 4c37acd0e5834e723367594a972dea4506416b3b49ce2c7fc4b0b9d6caa9719272f0fa0158d627a8d5bfc02913aba863a40bff346330ff07c10dc1a6754b9aa6
checksums.yaml.gz.sig CHANGED
Binary file
data/BUGS.md CHANGED
@@ -12,7 +12,7 @@ Please, make sure to include the following information:
12
12
 
13
13
  IMPORTANT: Remove any confidential information from the logs before posting them.
14
14
 
15
- [Use Github (Public Issue Reporting)](https://github.com/IBM/aspera-cli/issues)
15
+ [Use GitHub (Public Issue Reporting)](https://github.com/IBM/aspera-cli/issues)
16
16
 
17
17
  ## Scope of Support
18
18
 
data/CHANGELOG.md CHANGED
@@ -1,12 +1,30 @@
1
1
  # Changes (Release notes)
2
2
 
3
+ * 4.22.0
4
+
5
+ * New Features:
6
+ * `faspex5`: Support paging for Faspex5 browsing.
7
+ * `aoc`: #196 Command `packages list` now also supports option `once_only`.
8
+ * `vault`: Support for IBM HashiCorp Vault to store secrets.
9
+ * `wizard`: Preset name can now be specified as optional positional parameter.
10
+ * `config`: New command `ascp schema` displays JSON schema for transfer spec for all, or just one agent.
11
+ * `node`: #198 By default do not allow creation of folder if a link exists with the same name. Use option `query` with parameter `check` set to `false` to disable.
12
+ * `node`: In gen4 operations, also used in `aoc files`, new commands: `mklink`, `mkfile`.
13
+ * Issues Fixed:
14
+ * `aoc`: #195 `package receive ALL` for shared inbox without workspace now works.
15
+ * Breaking Changes:
16
+ * `faspex5`, `aoc`: `gateway` now takes argument as `Hash` with `url` instead of only `String`.
17
+ * `faspex5 postprocessing`: Now takes a flat `Hash`, instead of multi-level `Hash`.
18
+ * HTTP: More retry parameters.
19
+ * `node`: renamed command `http_node_download` to `cat`, and it directly displays the content of the file in the terminal unless option `--output` is specified.
20
+
3
21
  * 4.21.2
4
22
 
5
23
  * New Features:
6
24
  * **container**: Updated Ruby to 3.4.2
7
25
  * Issues Fixed:
8
26
  * **global**: #185 `@val:` shall stop processing extended values
9
- * **global**: #186 Removed dependency on openssl 3.3 gem to avoid msys dep on Windows.
27
+ * **global**: #186 Removed dependency on OpenSSL 3.3 gem to avoid MSYS2 dep on Windows.
10
28
  * `echo`: Display of list (Array) was showing only first element of it.
11
29
  * `transferd`: support for version 1.1.5+
12
30
  * Breaking Changes:
@@ -25,7 +43,7 @@
25
43
  * `config`: #175 `ascli config preset set GLOBAL version_check_days 0` causes a bad `config.yaml` to be written
26
44
  * `config`: #180 problem in `ascp install`
27
45
  * `config`: Soft links in transfer SDK archive are correctly extracted
28
- * `aoc`: #184 token cache shall be different per aoc org.
46
+ * `aoc`: #184 token cache shall be different per AoC org.
29
47
  * `aoc`: Fix `packages delete` not working.
30
48
  * `direct` agent: #174 Race condition fix with `ascp`: timeout waiting mgt port connect (select not readable)
31
49
  * `preview`: #177 fix bug that prevents preview generation to work.
@@ -33,8 +51,8 @@
33
51
  * `transferd`: Use of Aspera Transfer Daemon requires minimum version 1.1.4. agent `trsdk` renamed `transferd`.
34
52
  * `ascp`: Default SDK version is now 1.1.4. Removes support for ascp4.
35
53
  * `node`: Removed deprecated command prefix `exec:`, use `@ruby:` instead.
36
- * **global**: Now uses openssl 3.
37
- * **global**: Ruby minimum versions is now 3.1 (mainly due to switch to openssl 3). Future minimum is 3.2. Recommended is 3.4. (that removes macOS default ruby support. Newer Ruby version shall be installed on macOS with `brew`)
54
+ * **global**: Now uses OpenSSL 3.
55
+ * **global**: Ruby minimum versions is now 3.1 (mainly due to switch to OpenSSL 3). Future minimum is 3.2. Recommended is 3.4. (that removes macOS default ruby support. Newer Ruby version shall be installed on macOS with `brew`)
38
56
  * **global**: Options `transpose_single` and `multi_table` replaced with single option `multi_single` and values: `no`, `yes`, `single`.
39
57
  * **global**: Column name for single object is now `field` instead of `key`.
40
58
 
@@ -45,15 +63,15 @@
45
63
  * New Features:
46
64
  * `aoc`: Improved usability for creation of Admin shared folders.
47
65
  * `node`: New option `node_cache` (bool) for gen4 operations.
48
- * `node`: Option `root_id` now works always for node gen4, as well as `%id:` for file selection in addition to path.
66
+ * `node`: Option `root_id` now always works for node gen4, as well as `%id:` for file selection in addition to path.
49
67
  * `node`: `transfer list` now uses the `iteration_token` to retrieve all values. Option `once_only` is now supported.
50
68
  * **global**: option `http_options` now include retry options.
51
69
  * Issues Fixed:
52
70
  * `aoc`: Fixed `find` command not working. (undefined variable)
53
- * `aoc`: #165 aoc mkdir now follows the last link of containing folder
71
+ * `aoc`: #165 AoC mkdir now follows the last link of containing folder
54
72
  * Breaking Changes:
55
73
  * Internal: Basic REST calls now return data directly. (no more `data` key). For advanced calls, use `call`.
56
- * Internal: Transfer SDK download is now a 2-step procedure: First get the YAML file from github with URLS for the various platforms and versions, and then download the archive from the official IBM repository.
74
+ * Internal: Transfer SDK download is now a 2-step procedure: First get the YAML file from GitHub with URLs for the various platforms and versions, and then download the archive from the official IBM repository.
57
75
  **global**: option `format=multi` is replaced with option `multi_table=yes`
58
76
  * `faspex5`: removed deprecated option `value` replaced with positional parameter.
59
77
 
@@ -61,7 +79,7 @@
61
79
 
62
80
  * New Features:
63
81
  * `server`: add support for `async` (Aspera Sync) from Transfer SDK
64
- * **global**: #156 support sending folders with httpgw
82
+ * **global**: #156 support sending folders with `httpgw`
65
83
  * **global**: new value for option `format`: `multi`
66
84
  * Issues Fixed:
67
85
  * `aoc`: #157 fix problem with `files browse` on a link
@@ -172,7 +190,7 @@
172
190
  * **global**: option `pid_file` to write tool's PID during execution, deleted on exit
173
191
  * `config`: command `remote_certificate` to retrieve a remote certificate
174
192
  * `config`: added logger level `trace1` and `trace2`
175
- * `config`: `wizard` can detect multiple applications at the same address or url.
193
+ * `config`: `wizard` can detect multiple applications at the same address or URL.
176
194
  * `aoc`: wizard accepts public links
177
195
  * `aoc`: support private links, and possibility to list shared folder with workspace `@json:null`
178
196
  * `orchestrator`: error analysis for workflow start
@@ -180,7 +198,7 @@
180
198
  * `node`: added command `transfer sessions` to list all sessions of all transfers
181
199
  * `node`: generate bearer token from private key and user information
182
200
  * `node`: access node API with bearer token as credentials
183
- * **global**: agent `direct` allows ignoring certificate for wss using http options
201
+ * **global**: agent `direct` allows ignoring certificate for WSS using HTTP options
184
202
  * `preview`: command `show` generates a preview and displays it in terminal
185
203
  * Issues Fixed:
186
204
  * Ruby warning: `net/protocol.rb:68: warning: already initialized constant Net::ProtocRetryError` solved by removing dependency on `net-smtp` from gem spec (already in base ruby).
@@ -245,7 +263,7 @@
245
263
  * `faspex5`: list content of package, and allow partial download of package
246
264
  * `faspex5`: list packages support multiple pages and items limitations (`max` and `pmax`)
247
265
  * `aoc`: files operations with workspace-less user (#109)
248
- * `node`: async with gen3 token (#110)
266
+ * `node`: `async` with gen3 token (#110)
249
267
  * `node`: display of preview of file in terminal for access keys
250
268
  * Issues Fixed:
251
269
  * `cos`: do not use refresh token when not supported
@@ -261,10 +279,10 @@
261
279
  * **global**: `delete` operation supports option `value` for deletion parameters
262
280
  * `aoc`: command `aoc packages recv` accepts option `query` to specify a shared inbox
263
281
  * `faspex`: (v4) user delete accepts option `value` with value `{"destroy":true}` to delete users permanently
264
- * `faspex`: (v4) gateway to faspex 5 for package send
282
+ * `faspex`: (v4) gateway to Faspex 5 for package send
265
283
  * `faspex5`: possibility to change email templates
266
284
  * `faspex5`: shared folder list and browse
267
- * `faspex5`: emulate faspex 4 postprocessing, plugin: `faspex5` command: `postprocessing`
285
+ * `faspex5`: emulate Faspex 4 post-processing, plugin: `faspex5` command: `postprocessing`
268
286
  * `faspex5`: send package from remote source
269
287
  * `shares`: option `type` for command `shares admin user`
270
288
  * `shares`: full support for shares admin operations
@@ -280,14 +298,14 @@
280
298
  * New Features:
281
299
  * **global**: `vault`: secret finder, migration from config file
282
300
  * **global**: allow removal of transfer spec parameter by setting value to `null`
283
- * **global**: option `ascp_opts` allows to provide native `ascp` options on command line
301
+ * **global**: option `ascp_opts` allows providing native `ascp` options on command line
284
302
  * `node`, `server`: command `sync` added to `node` (gen4) and `server` plugins, also available in `aoc`
285
303
  * Issues Fixed:
286
304
  * **global**: security: no shell interpolation
287
305
  * **global**: agent `node`: when WSS is used: no localhost (certificate)
288
306
  * `aoc`: #99 `file download` for single shared folder
289
- * `faspex5`: change of API in faspex 5 for send package (paths is mandatory for any type of transfer now)
290
- * **global**: Oauth web authentication was broken, fixed now
307
+ * `faspex5`: change of API in Faspex 5 for send package (paths is mandatory for any type of transfer now)
308
+ * **global**: OAuth web authentication was broken, fixed now
291
309
  * Breaking Changes:
292
310
  * **container**: image has entry point
293
311
  * `aoc`: `admin res node` commands `v3` and `v4` replaced with `do` and command `v3` moved inside `do`
data/CONTRIBUTING.md CHANGED
@@ -29,7 +29,7 @@ If you want to contribute, please:
29
29
 
30
30
  - Fork the project.
31
31
  - Make your feature addition or bug fix.
32
- - Add tests for it. This is important so I don't break it in a future version unintentionally.
32
+ - Add tests for it. This is important, so it doesn't break in a future version unintentionally.
33
33
  - Run `rubocop` to comply for coding standards
34
34
  - Update `CHANGELOG.md`
35
35
  - Send a pull request on GitHub.
@@ -50,9 +50,9 @@ Transfer agents, in: `lib/aspera/agent`.
50
50
 
51
51
  ## Ruby version
52
52
 
53
- Install Ruby using your prefered method.
53
+ Install Ruby using your preferred method.
54
54
 
55
- To cleanup installed gems to start fresh:
55
+ To clean up installed gems to start fresh:
56
56
 
57
57
  ```bash
58
58
  make clean_gems
@@ -206,15 +206,12 @@ cd container
206
206
 
207
207
  ## Single executable build
208
208
 
209
- Initially, `rubyc` (gem [`ruby-packer`](https://github.com/pmq20/ruby-packer) and [here](https://github.com/you54f/ruby-packer)) was used to build a single executable.
209
+ See [Executable build](./binary/README.md).
210
210
 
211
- <https://www.tebako.org/>
212
-
213
- A modern version of this is now used: [`tebako`](https://github.com/tamatebako/tebako) for which a container is provided.
211
+ For operations, move to the folder:
214
212
 
215
213
  ```bash
216
214
  cd binary
217
- make GEM_VERSION=4.11.0
218
215
  ```
219
216
 
220
217
  ## Development check list for new release
@@ -225,9 +222,8 @@ When preparing for a new release do the following:
225
222
 
226
223
  ## Long Term Implementation and delivery improvements
227
224
 
228
- - replace rest and oauth classes with ruby standard gems:
225
+ - replace Rest and OAuth classes with ruby standard gems:
229
226
  - <https://github.com/rest-client/rest-client>
230
227
  - <https://github.com/oauth-xx/oauth2>
231
228
  - use gem Thor <http://whatisthor.com/> (or other standard Ruby CLI manager)
232
- - Package a single-file executable for various architectures with <https://github.com/pmq20/ruby-packer> (`rubyc`)
233
229
  - look at <https://github.com/phusion/traveling-ruby>