aspera-cli 4.13.0 → 4.15.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (99) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/CHANGELOG.md +81 -7
  4. data/CONTRIBUTING.md +22 -6
  5. data/README.md +2038 -1080
  6. data/bin/ascli +18 -9
  7. data/bin/asession +12 -14
  8. data/examples/dascli +1 -1
  9. data/examples/proxy.pac +1 -1
  10. data/examples/rubyc +24 -0
  11. data/lib/aspera/aoc.rb +219 -159
  12. data/lib/aspera/ascmd.rb +25 -14
  13. data/lib/aspera/cli/basic_auth_plugin.rb +12 -9
  14. data/lib/aspera/cli/error.rb +17 -0
  15. data/lib/aspera/cli/extended_value.rb +47 -12
  16. data/lib/aspera/cli/formatter.rb +260 -179
  17. data/lib/aspera/cli/hints.rb +80 -0
  18. data/lib/aspera/cli/main.rb +104 -156
  19. data/lib/aspera/cli/manager.rb +259 -209
  20. data/lib/aspera/cli/plugin.rb +123 -63
  21. data/lib/aspera/cli/plugins/alee.rb +2 -3
  22. data/lib/aspera/cli/plugins/aoc.rb +341 -261
  23. data/lib/aspera/cli/plugins/ats.rb +22 -21
  24. data/lib/aspera/cli/plugins/bss.rb +5 -5
  25. data/lib/aspera/cli/plugins/config.rb +578 -627
  26. data/lib/aspera/cli/plugins/console.rb +44 -6
  27. data/lib/aspera/cli/plugins/cos.rb +15 -17
  28. data/lib/aspera/cli/plugins/faspex.rb +114 -100
  29. data/lib/aspera/cli/plugins/faspex5.rb +411 -264
  30. data/lib/aspera/cli/plugins/node.rb +354 -259
  31. data/lib/aspera/cli/plugins/orchestrator.rb +61 -29
  32. data/lib/aspera/cli/plugins/preview.rb +82 -90
  33. data/lib/aspera/cli/plugins/server.rb +79 -32
  34. data/lib/aspera/cli/plugins/shares.rb +55 -42
  35. data/lib/aspera/cli/sync_actions.rb +68 -0
  36. data/lib/aspera/cli/transfer_agent.rb +66 -73
  37. data/lib/aspera/cli/transfer_progress.rb +74 -0
  38. data/lib/aspera/cli/version.rb +1 -1
  39. data/lib/aspera/colors.rb +12 -8
  40. data/lib/aspera/command_line_builder.rb +14 -11
  41. data/lib/aspera/cos_node.rb +3 -2
  42. data/lib/aspera/data/6 +0 -0
  43. data/lib/aspera/environment.rb +24 -9
  44. data/lib/aspera/fasp/agent_aspera.rb +126 -0
  45. data/lib/aspera/fasp/agent_base.rb +31 -77
  46. data/lib/aspera/fasp/agent_connect.rb +25 -21
  47. data/lib/aspera/fasp/agent_direct.rb +89 -103
  48. data/lib/aspera/fasp/agent_httpgw.rb +231 -149
  49. data/lib/aspera/fasp/agent_node.rb +41 -34
  50. data/lib/aspera/fasp/agent_trsdk.rb +75 -32
  51. data/lib/aspera/fasp/error_info.rb +4 -2
  52. data/lib/aspera/fasp/faux_file.rb +52 -0
  53. data/lib/aspera/fasp/installation.rb +53 -195
  54. data/lib/aspera/fasp/management.rb +244 -0
  55. data/lib/aspera/fasp/parameters.rb +71 -37
  56. data/lib/aspera/fasp/parameters.yaml +76 -8
  57. data/lib/aspera/fasp/products.rb +162 -0
  58. data/lib/aspera/fasp/resume_policy.rb +3 -3
  59. data/lib/aspera/fasp/transfer_spec.rb +7 -6
  60. data/lib/aspera/fasp/uri.rb +26 -24
  61. data/lib/aspera/faspex_gw.rb +2 -2
  62. data/lib/aspera/faspex_postproc.rb +2 -2
  63. data/lib/aspera/hash_ext.rb +14 -4
  64. data/lib/aspera/json_rpc.rb +49 -0
  65. data/lib/aspera/keychain/macos_security.rb +13 -13
  66. data/lib/aspera/line_logger.rb +23 -0
  67. data/lib/aspera/log.rb +58 -16
  68. data/lib/aspera/node.rb +157 -92
  69. data/lib/aspera/oauth.rb +37 -19
  70. data/lib/aspera/open_application.rb +4 -4
  71. data/lib/aspera/persistency_action_once.rb +1 -1
  72. data/lib/aspera/persistency_folder.rb +2 -2
  73. data/lib/aspera/preview/file_types.rb +4 -2
  74. data/lib/aspera/preview/generator.rb +22 -35
  75. data/lib/aspera/preview/options.rb +2 -0
  76. data/lib/aspera/preview/terminal.rb +73 -16
  77. data/lib/aspera/preview/utils.rb +21 -28
  78. data/lib/aspera/proxy_auto_config.js +2 -2
  79. data/lib/aspera/rest.rb +136 -68
  80. data/lib/aspera/rest_call_error.rb +1 -1
  81. data/lib/aspera/rest_error_analyzer.rb +15 -14
  82. data/lib/aspera/rest_errors_aspera.rb +37 -34
  83. data/lib/aspera/secret_hider.rb +18 -15
  84. data/lib/aspera/ssh.rb +5 -2
  85. data/lib/aspera/sync.rb +127 -119
  86. data/lib/aspera/temp_file_manager.rb +10 -3
  87. data/lib/aspera/web_auth.rb +10 -7
  88. data/lib/aspera/web_server_simple.rb +9 -4
  89. data.tar.gz.sig +0 -0
  90. metadata +34 -17
  91. metadata.gz.sig +0 -0
  92. data/docs/test_env.conf +0 -186
  93. data/lib/aspera/cli/listener/line_dump.rb +0 -19
  94. data/lib/aspera/cli/listener/logger.rb +0 -22
  95. data/lib/aspera/cli/listener/progress.rb +0 -50
  96. data/lib/aspera/cli/listener/progress_multi.rb +0 -84
  97. data/lib/aspera/cli/plugins/sync.rb +0 -44
  98. data/lib/aspera/data/7 +0 -0
  99. data/lib/aspera/fasp/listener.rb +0 -13
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 86109a9ac1ef2f30ea4fd192b5535367d72ca679bcf588f86452289cf451dc72
4
- data.tar.gz: 94d5d460d4ed2ab49637119711f03e81f2ab1793e75f8f6c5da098c8d93c26dc
3
+ metadata.gz: 007b7653b79d11d35c5741fe091d6118204b25521a3ad30082e2c1970124d91a
4
+ data.tar.gz: a7b763b7f993f414d0ec78b79eeb1607dcb1618697bb4360f05fb39ad159443c
5
5
  SHA512:
6
- metadata.gz: '05729a6bcf5db7f2831883e2da68c2fc3e3cc542ea3fb7969a5095b87d094b0cc98845ac73f300865f358ff70751cdbde4cab16dc10349076168e5266c62a98a'
7
- data.tar.gz: 7ebdeaaf9a8408137f7057665f0b8e12f6314f2328cd235be78cfd531df9ffa422f0574331d3fed2436fd3e97f3aff452c9ce778dc06112fc012f6e48b5adaf8
6
+ metadata.gz: 176ba1755a5e545b2886b311e7be53c33983c3a8eaf75846801d83dd9dfc498bfd5e44cf0f65c51267374309e1f4ffa2657a793fc659622a9a9ba41d94b71344
7
+ data.tar.gz: 1e0f022813ba1b6e7cdb90a632811193689ac69175d9eac65a0589128c5d12181f283beb1a973109d9d1768ab770535be77aa9c37220363c8d0090b9139c8196
checksums.yaml.gz.sig CHANGED
Binary file
data/CHANGELOG.md CHANGED
@@ -1,5 +1,79 @@
1
1
  # Changes (Release notes)
2
2
 
3
+ * 4.15.0
4
+
5
+ * General: removed many redundant options, more consistency between plugins, see below in "break".
6
+ * New Features:
7
+ * global: added resolution hints for well known issues.
8
+ * global: extended value expression `@extend:` finds and replace extended values in a string (e.g. for JSON)
9
+ * global: option `fields` now supports `RegExp`
10
+ * global: option `home` to set the main folder for configuration and cache
11
+ * global: option `ignore_certificate` to specify specific URLs instead of global option `insecure`
12
+ * global: option `cert_stores` to specify alternate certificate stores
13
+ * global: uniform progress bar for any type of transfer.
14
+ * global: add extended value types: `re` and `yaml`
15
+ * global: option `pid_file` to write tool's PID during execution, deleted on exit
16
+ * config: command `remote_certificate` to retrieve a remote certificate
17
+ * config: added logger level `trace1` and `trace2`
18
+ * wizard: can detect multiple applications at the same address or url.
19
+ * aoc: wizard accepts public links
20
+ * aoc: support private links, and possibility to list shared folder with workspace `@json:null`
21
+ * orchestrator: error analysis for workflow start
22
+ * httpgw: now supports pseudo file for testing: e.g. `faux:///testfile?1k`
23
+ * node: added command `transfer sessions` to list all sessions of all transfers
24
+ * node: generate bearer token from private key and user information
25
+ * node: access node API with bearer token as credentials
26
+ * agent: `direct` allows ignoring certificate for wss using http options
27
+ * preview: command `show` generates a preview and displays it in terminal
28
+ * Issues Fixed:
29
+ * Ruby warning: `net/protocol.rb:68: warning: already initialized constant Net::ProtocRetryError` solved by removing dependency on `net-smtp` from gem spec (already in base ruby).
30
+ * Breaking Changes:
31
+ * global: commands `detect` and `wizard` takes now a mandatory argument: address or url instead of option `url`.
32
+ * global: renamed option `pkeypath` to `key_path`
33
+ * global: renamed option `notif_to` to `notify_to` and `notif_template` to `notify_template`
34
+ * global: removed extended value handler `incps`, as it is never used (use `extend` instead).
35
+ * global: option `fields`: `+prop` is replaced with: `DEF,prop` and `-field` is replaced with: `DEF,-field`, and whole list is evaluated.
36
+ * global: replaced option `progress` with option `progressbar` (bool)
37
+ * global: removed option `rest_debug` and `-r`, replaced with `--log-level=trace2`
38
+ * global: the default file name for private key when using wizard is change from `aspera_aoc_key` to `my_private_key.pem`
39
+ * faspex5: removed option and `auth` type `link`: simply provide the public link as `url`
40
+ * faspex: remote source selection now uses percent selector instead of parameter `id` or `name`
41
+ * faspex: option `source_name` is now `remote_source`
42
+ * aoc: selection by name uses percent selector instead of option or parameter `name`
43
+ * aoc: removed option `link`: use `url` instead
44
+ * aoc: in command `short_link`, place type before command, e.g. `short_link private create /blah`
45
+ * aoc: replaced option `operation` with mandatory positional parameter for command `files transfer`
46
+ * aoc: replaced option `from_folder` with mandatory positional parameter for command `files transfer`
47
+ * orchestrator: workflow start takes arguments as optional positional extended value instead of option `param`
48
+ * node: `find` command now takes an optional `@ruby:` extended value instead of option `query` with prefix: `exec:`
49
+ * sync: plugin `sync` is removed: actions are available through `server` and `node` plugins.
50
+ * sync: replaced option `sync_session` with optional positional parameter.
51
+ * preview: command `scan`, `events` and `trevents` replaced option `query` with optional positional parameter for filter (like `find`).
52
+ * trsdk: parameters `host` and `port` in option `transfer_info` are replaced with parameter `url`, like `grpc://host:port`
53
+
54
+ * 4.14.0
55
+
56
+ * new: server: option `passphrase` for simpler command line (#114)
57
+ * new: percent selector for entities identifier
58
+ * new: faspex5: shared inbox and workgroup membership management
59
+ * new: faspex5: invite external user to shared inbox
60
+ * new: faspex5: package list and receive from workgroup and shared inbox
61
+ * new: config: Command `ascp info` shows default transfer spec.
62
+ * new: httpgw: synchronous and asynchronous upload modes
63
+ * new: node: command `bandwidth_average` to get average bandwidth of node, per periods
64
+ * fix: option `ts`: deep add and remove of keys. (#117)
65
+ * fix: faspex5: user lookup for `packages send` shall be exact match (#120)
66
+ * fix: direct: if transfer spec contains "paths" and elements with "destination", but first element has only "source", then destinations were ignored. Now "destination" all or none is enforced.
67
+ * change: using `aoc files` or node gen4 operations (`browse`, `delete`) on a link will follow the link only if path ends with /
68
+ * change: shares: command `repository` is changed to `files` for consistency with aoc and upcoming faspex5, but is still available as alias
69
+ * change: aoc: better handling of shared links
70
+ * change: global: option `value` is deprecated. Use positional parameter for creation data and option `query` for list/delete operations.
71
+ * change: config: remove deprecated command: `export_to_cli`
72
+ * change: config: removed all legacy preset command, newer command `preset` shall be used now.
73
+ * change: config: SDK is now installed in $HOME/.aspera/sdk instead of $HOME/.aspera/ascli/sdk
74
+ * change(break): aoc/node: Simplification: gen4 operations: show modify permission thumbnail are now directly under node gen 4 command. Command `file` is suppressed. Option `path` is suppressed. The default expected argument is a path. To provide a file id, use selector syntax: %id:_file_id_
75
+ * change(break): node: option `token_type` is removed, as starting with HSTS 4.3 basic token is only allowed with access keys, so use gen4 operations: `acc do self`
76
+
3
77
  * 4.13.0
4
78
 
5
79
  * new: preview: option `reencode_ffmpeg` allows overriding all re-encoding options
@@ -70,7 +144,7 @@
70
144
  * new: improved documentation on faspex and aoc package send
71
145
  * fix: `node do` command fixed
72
146
  * fix: improved secret hiding from logs
73
- * change(break): removed rarely commands nodeadmin, configuration, userdata, ctl from plugin `server`
147
+ * change(break): removed rarely commands `nodeadmin`, `configuration`, `userdata`, `ctl` from plugin `server`
74
148
  as well as option `cmd_prefix`
75
149
  * change: `ascli` runs as user `cliuser` instead of `root` in container
76
150
  * change: default access right for config folder is now user only, including private keys
@@ -189,9 +263,9 @@
189
263
  * new: agent `http_gw` now supports upload
190
264
  * new: added option `sdk_url` to install SDK from local file for offline install
191
265
  * new: check new gem version periodically
192
- * new: the --fields= option, support -_fieldname_ to remove a field from default fields
266
+ * new: the --fields= option, support -_field_name_ to remove a field from default fields
193
267
  * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
194
- * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
268
+ * new: `mimemagic` is now optional, needs manual install for `preview`, compatible with version 0.4.x
195
269
  * new: AoC a password can be provided for a public link
196
270
  * new: `conf doc` take an optional parameter to go to a section
197
271
  * new: initial support for Faspex 5 Beta 1
@@ -209,7 +283,7 @@
209
283
 
210
284
  * 0.11.8
211
285
 
212
- * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb)
286
+ * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using `Xvfb`)
213
287
 
214
288
  * 0.11.7
215
289
 
@@ -330,7 +404,7 @@
330
404
  * 0.10.4
331
405
 
332
406
  * new options for AoC : `secrets`
333
- * ACLI-533 temp file list folder to use file lists is set by default, and used by `asession`
407
+ * `ACLI-533` temp file list folder to use file lists is set by default, and used by `asession`
334
408
 
335
409
  * 0.10.3
336
410
 
@@ -370,13 +444,13 @@
370
444
 
371
445
  * new command to display basic token of node
372
446
  * new command to display bearer token of node in AoC
373
- * the --fields= option, support +_fieldname_ to add a field to default fields
447
+ * the --fields= option, support +_field_name_ to add a field to default fields
374
448
  * many small changes
375
449
 
376
450
  * 0.9.32
377
451
 
378
452
  * all Faspex public links are now supported
379
- * removed faspex operation recv_publink
453
+ * removed faspex operation `recv_publink`
380
454
  * replaced with option `link` (consistent with AoC)
381
455
 
382
456
  * 0.9.31
data/CONTRIBUTING.md CHANGED
@@ -1,5 +1,5 @@
1
1
  # Contributing
2
-
2
+ <!-- cspell:words passin -->
3
3
  ## Reporting Issues and Vulnerabilities
4
4
 
5
5
  You can report issues at <https://github.com/IBM/aspera-cli/issues>
@@ -32,7 +32,7 @@ If you want to contribute, please:
32
32
  - Add tests for it. This is important so I don't break it in a future version unintentionally.
33
33
  - **Bonus Points** go out to anyone who also updates `CHANGELOG.md` :)
34
34
  - Send a pull request on GitHub.
35
- - run Rubocop to comply for coding standards
35
+ - run `rubocop` to comply for coding standards
36
36
 
37
37
  ## Architecture
38
38
 
@@ -192,11 +192,11 @@ make dpush
192
192
 
193
193
  ### Specific version image build
194
194
 
195
- It is possible to build a specific version by setting envvar `GEMVERS` and `make` with option `-e`:
195
+ It is possible to build a specific version by setting envvar `GEM_VERSION` and `make` with option `-e`:
196
196
 
197
197
  ```bash
198
- GEMVERS=4.11.0 make -e docker
199
- GEMVERS=4.11.0 make -e dpush
198
+ GEM_VERSION=4.11.0 make -e docker
199
+ GEM_VERSION=4.11.0 make -e dpush
200
200
  ```
201
201
 
202
202
  ### Development version image build
@@ -209,10 +209,26 @@ make dockerbeta
209
209
  make dpushversion
210
210
  ```
211
211
 
212
+ ## Single executable build
213
+
214
+ It is possible to build `ascli` as a single native executable using `rubyc` (gem `ruby-packer`).
215
+
216
+ ```bash
217
+ make single
218
+ ```
219
+
220
+ This is based on tool `rubyc` (gem `ruby-packer`): either place the executable in folder `tmp`, or the architecture-specific version will be automatically downloaded.
221
+
222
+ > **Note:** Perl `Findbin` is needed, install on Linux with:
223
+
224
+ ```bash
225
+ sudo dnf install -y perl perl-FindBin perl-IPC-Cmd
226
+ ```
227
+
212
228
  ## Long Term Implementation and delivery improvements
213
229
 
214
230
  - replace rest and oauth classes with ruby standard gems:
215
231
  - <https://github.com/rest-client/rest-client>
216
232
  - <https://github.com/oauth-xx/oauth2>
217
233
  - use gem Thor <http://whatisthor.com/> (or other standard Ruby CLI manager)
218
- - Package a single-file executable for various architectures with <https://github.com/pmq20/ruby-packer> (rubyc)
234
+ - Package a single-file executable for various architectures with <https://github.com/pmq20/ruby-packer> (`rubyc`)