aspera-cli 4.24.2 → 4.25.0.pre

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (65) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/CHANGELOG.md +1064 -758
  4. data/CONTRIBUTING.md +43 -100
  5. data/README.md +671 -419
  6. data/lib/aspera/api/aoc.rb +71 -43
  7. data/lib/aspera/api/cos_node.rb +3 -2
  8. data/lib/aspera/api/faspex.rb +6 -5
  9. data/lib/aspera/api/node.rb +10 -12
  10. data/lib/aspera/ascmd.rb +1 -2
  11. data/lib/aspera/ascp/installation.rb +53 -39
  12. data/lib/aspera/assert.rb +25 -3
  13. data/lib/aspera/cli/error.rb +4 -2
  14. data/lib/aspera/cli/extended_value.rb +84 -60
  15. data/lib/aspera/cli/formatter.rb +55 -22
  16. data/lib/aspera/cli/main.rb +21 -14
  17. data/lib/aspera/cli/manager.rb +348 -247
  18. data/lib/aspera/cli/plugins/alee.rb +3 -3
  19. data/lib/aspera/cli/plugins/aoc.rb +70 -14
  20. data/lib/aspera/cli/plugins/base.rb +57 -49
  21. data/lib/aspera/cli/plugins/config.rb +69 -84
  22. data/lib/aspera/cli/plugins/console.rb +13 -8
  23. data/lib/aspera/cli/plugins/cos.rb +1 -1
  24. data/lib/aspera/cli/plugins/faspex.rb +32 -26
  25. data/lib/aspera/cli/plugins/faspex5.rb +45 -43
  26. data/lib/aspera/cli/plugins/faspio.rb +5 -5
  27. data/lib/aspera/cli/plugins/httpgw.rb +1 -1
  28. data/lib/aspera/cli/plugins/node.rb +131 -120
  29. data/lib/aspera/cli/plugins/oauth.rb +1 -1
  30. data/lib/aspera/cli/plugins/orchestrator.rb +114 -32
  31. data/lib/aspera/cli/plugins/preview.rb +26 -46
  32. data/lib/aspera/cli/plugins/server.rb +6 -8
  33. data/lib/aspera/cli/plugins/shares.rb +27 -32
  34. data/lib/aspera/cli/sync_actions.rb +49 -38
  35. data/lib/aspera/cli/transfer_agent.rb +16 -34
  36. data/lib/aspera/cli/version.rb +1 -1
  37. data/lib/aspera/cli/wizard.rb +8 -5
  38. data/lib/aspera/command_line_builder.rb +20 -17
  39. data/lib/aspera/coverage.rb +1 -1
  40. data/lib/aspera/environment.rb +41 -34
  41. data/lib/aspera/faspex_gw.rb +1 -1
  42. data/lib/aspera/keychain/factory.rb +1 -2
  43. data/lib/aspera/markdown.rb +31 -0
  44. data/lib/aspera/nagios.rb +6 -5
  45. data/lib/aspera/oauth/base.rb +17 -27
  46. data/lib/aspera/oauth/factory.rb +1 -1
  47. data/lib/aspera/oauth/url_json.rb +2 -1
  48. data/lib/aspera/preview/file_types.rb +23 -37
  49. data/lib/aspera/products/connect.rb +3 -3
  50. data/lib/aspera/rest.rb +51 -39
  51. data/lib/aspera/rest_error_analyzer.rb +4 -4
  52. data/lib/aspera/ssh.rb +5 -2
  53. data/lib/aspera/ssl.rb +41 -0
  54. data/lib/aspera/sync/conf.schema.yaml +182 -34
  55. data/lib/aspera/sync/database.rb +2 -1
  56. data/lib/aspera/sync/operations.rb +125 -69
  57. data/lib/aspera/transfer/parameters.rb +3 -4
  58. data/lib/aspera/transfer/spec.rb +2 -3
  59. data/lib/aspera/transfer/spec.schema.yaml +48 -18
  60. data/lib/aspera/transfer/spec_doc.rb +14 -14
  61. data/lib/aspera/uri_reader.rb +1 -1
  62. data/lib/transferd_pb.rb +2 -2
  63. data.tar.gz.sig +0 -0
  64. metadata +19 -6
  65. metadata.gz.sig +3 -2
data/CONTRIBUTING.md CHANGED
@@ -1,5 +1,5 @@
1
1
  # Contributing
2
- <!-- cspell:words passin -->
2
+
3
3
  ## Reporting Issues and Vulnerabilities
4
4
 
5
5
  If you encounter a problem or vulnerability, please report it at [GitHub Issues](https://github.com/IBM/aspera-cli/issues).
@@ -10,7 +10,7 @@ Before submitting a new issue:
10
10
 
11
11
  To help us assist you efficiently, include the following in your report:
12
12
 
13
- - The version of ascli you are using:
13
+ - The version of `ascli` you are using:
14
14
 
15
15
  ```bash
16
16
  ascli version
@@ -20,20 +20,22 @@ To help us assist you efficiently, include the following in your report:
20
20
  - The output of your Ruby environment:
21
21
 
22
22
  ```bash
23
- ruby -e "puts RUBY_DESCRIPTION"
23
+ ruby -v
24
24
  ```
25
25
 
26
26
  ## Making Contributions
27
27
 
28
- We welcome contributions to improve the aspera-cli project!
28
+ We welcome contributions to improve the `aspera-cli` project!
29
29
 
30
30
  ### Getting Started
31
31
 
32
- Clone the repository and navigate to the project folder:
32
+ Clone the repository and navigate to the project's main folder:
33
33
 
34
34
  ```bash
35
35
  git clone https://github.com/IBM/aspera-cli.git
36
36
  cd aspera-cli
37
+ bundle install
38
+ bundle exec rake -T
37
39
  ```
38
40
 
39
41
  For testing instructions, refer to [Running Tests](#running-tests).
@@ -61,7 +63,7 @@ To submit a contribution:
61
63
 
62
64
  ## Architecture
63
65
 
64
- The overall architecture of aspera-cli is modular and extensible.
66
+ The overall architecture of `aspera-cli` is modular and extensible.
65
67
 
66
68
  ![Architecture](docs/architecture.png)
67
69
 
@@ -84,12 +86,12 @@ A list of classes are provided in <docs/uml.png>
84
86
  ## Ruby version
85
87
 
86
88
  `aspera-cli` is built with Ruby.
87
- You can install Ruby using any method you prefer (e.g., rbenv, rvm, system package manager).
89
+ You can install Ruby using any method you prefer (e.g., `rbenv`, `rvm`, system package manager).
88
90
 
89
- To start with a clean slate and remove all installed gems:
91
+ To start with a clean state and remove all installed gems:
90
92
 
91
93
  ```bash
92
- make clean_gems
94
+ bundle exec rake tools:clean_gems
93
95
  ```
94
96
 
95
97
  > [!TIP]
@@ -99,30 +101,28 @@ make clean_gems
99
101
 
100
102
  TODO: document installation of tool chain.
101
103
 
102
- Build system uses GNU Make.
104
+ Build system uses Ruby's `rake`.
103
105
 
104
106
  ### Environment
105
107
 
106
108
  A few macros/env vars control some aspects:
107
109
 
108
- | `make` macro or Env var | Description |
109
- |-----------------------------|--------------------------------------|
110
- | `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests |
111
- | `ASPERA_CLI_TEST_MACOS` | Set to `true` if local HSTS running on macOS |
112
- | `ASPERA_CLI_TEST_PRIVATE` | Path to private folder |
113
- | `ASPERA_CLI_DOC_CHECK_LINKS`| Check links still exist during doc generation. |
114
- | `ASPERA_CLI_DOC_DEBUG` | Enable debug in doc generation. |
115
- | `ENABLE_COVERAGE` | Tests with coverage analysis if set. |
116
- | `SIGNING_KEY` | Path to signing key to build Gem. |
117
- | `GEM_VERSION` | Gem version to build container |
110
+ | Environment variable | Description |
111
+ |-----------------------------|----------------------------------------------------|
112
+ | `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests. |
113
+ | `ASPERA_CLI_DOC_CHECK_LINKS`| Check links still exist during doc generation. |
114
+ | `LOG_LEVEL` | Change log level in `rake` tasks. |
115
+ | `ENABLE_COVERAGE` | Tests with coverage analysis if set. |
116
+ | `SIGNING_KEY` | Path to signing key to build Gem. |
117
+ | `GEM_VERSION` | Override gem version for builds. |
118
118
 
119
- Those macros can be set either in an env var, or on the `make` command line.
119
+ Those macros can be set either in an env var, or on the `rake` command line.
120
120
 
121
121
  > [!NOTE]
122
- > Env vars `ASPERA_CLI_TEST_*` are typically set in user's shell profile for development.
123
- > Others are more for "one shot" use.
122
+ > Env vars `ASPERA_CLI_*` are typically set in user's shell profile for development.
123
+ > Others are more for "one shot" use (on command line).
124
124
 
125
- To use the CLI directly from the development environment, add this to your shell profile:
125
+ To use the CLI directly from the development environment, add this to your shell profile (adapt the real path):
126
126
 
127
127
  ```bash
128
128
  dev_ascli=$HOME/github/aspera-cli
@@ -153,100 +153,49 @@ To check URL during doc generation, set env var: `ASPERA_CLI_DOC_CHECK_LINKS=1`.
153
153
 
154
154
  To debug doc generation, set env var: `ASPERA_CLI_DOC_DEBUG=debug`.
155
155
 
156
- ## Running Tests
157
-
158
- First, a testing configuration file must be created.
159
- From project top folder, execute:
156
+ ## Test Environment
160
157
 
161
- ```bash
162
- mkdir ~/some_secure_folder
163
- cp docs/test_env.conf ~/some_secure_folder/.
164
- ```
158
+ Refer to <tests/README.md>.
165
159
 
166
- Fill `~/some_secure_folder/test_env.conf` with system URLs and credentials for tests.
160
+ ## Build
167
161
 
168
- Then, tell where this file is located:
162
+ The gem is built with:
169
163
 
170
164
  ```bash
171
- export ASPERA_CLI_TEST_CONF_FILE=~/some_secure_folder/test_env.conf
165
+ bundle install
166
+ bundle exec rake build
172
167
  ```
173
168
 
174
- This project uses a `Makefile` for tests, in main folder:
169
+ If you don't want to install optional gems:
175
170
 
176
171
  ```bash
177
- make test
172
+ bundle config set without optional
178
173
  ```
179
174
 
180
- When new commands are added to the CLI, new tests shall be added to the test suite in `tests/Makefile`.
181
-
182
- One can also go to the `tests` folder:
175
+ Build beta version:
183
176
 
184
177
  ```bash
185
- cd tests
186
- make
187
- make full
188
- make list
189
- make torc
190
- make skip_torc
191
- make skip T=orch_wizard
178
+ export GEM_VERSION=$(env -u GEM_VERSION rake tools:version).$(date +%Y%m%d%H%M)
192
179
  ```
193
180
 
194
- ### Special tests
195
-
196
- Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
197
- By default, tests that use those gems are skipped.
198
- To run them: `make optional`.
199
- Those tests also require the optional gems to be installed: `make install_optional_gems`.
200
-
201
- Some other tests require interactive input. To run them: `make interactive`
202
-
203
- To run every test: `make full`
204
-
205
- ### Pre-release tests
206
-
207
- For preparation of a release, do the following:
208
-
209
- 1. Select a ruby version to test with.
210
- 2. Remove all gems: `make clean_gems`
211
- 3. `cd tests && make full`
212
-
213
- To test additional Ruby version, repeat the procedure with other Ruby versions.
214
-
215
- ## Coverage
181
+ ### Signed gem
216
182
 
217
- A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
218
- Enable coverage monitoring using macro/envvar `ENABLE_COVERAGE`.
219
-
220
- ```bash
221
- cd tests
222
- make ENABLE_COVERAGE=1
223
- ```
224
-
225
- Once tests are completed, or during test, consult the page: [coverage/index.html](coverage/index.html)
226
-
227
- ## Build
228
-
229
- By default, the gem is built signed: `make`.
230
183
  A private key is required to generate a signed Gem.
231
- Its path must be set using macro/envvar `SIGNING_KEY`, see below.
232
- The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
184
+ Its path must be set using envvar `SIGNING_KEY`.
185
+ The gem is signed with the public certificate found in `certs` and the private key pointed by `SIGNING_KEY` (kept secret by maintainer).
233
186
 
234
187
  ```bash
235
- make SIGNING_KEY=/path/to/vault/gem-private_key.pem
188
+ bundle exec rake build SIGNING_KEY=/path/to/vault/gem-private_key.pem
236
189
  ```
237
190
 
238
- It is also possible to build an unsigned version for development purpose: `make unsigned_gem`.
239
-
240
- ### Gem Signature
241
-
242
- Refer to <certs/README.md>
191
+ Refer to <certs/README.md>.
243
192
 
244
193
  ### gRPC stubs for transfer SDK
245
194
 
246
195
  Update with:
247
196
 
248
197
  ```bash
249
- make grpc
198
+ bundle exec rake tools:grpc
250
199
  ```
251
200
 
252
201
  It downloads the latest proto file and then compiles it.
@@ -255,20 +204,14 @@ It downloads the latest proto file and then compiles it.
255
204
 
256
205
  See [Container build](./container/README.md).
257
206
 
258
- For operations, move to the folder:
259
-
260
- ```bash
261
- cd container
262
- ```
263
-
264
207
  ## Single executable build
265
208
 
266
- See [Executable build](./binary/README.md).
209
+ See [Executable build](build/binary/README.md).
267
210
 
268
- For operations, move to the folder:
211
+ To list operations:
269
212
 
270
213
  ```bash
271
- cd binary
214
+ bundle exec rake -T ^binary:
272
215
  ```
273
216
 
274
217
  ## Development check list for new release