aspera-cli 4.24.2 → 4.25.0.pre
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +1064 -758
- data/CONTRIBUTING.md +43 -100
- data/README.md +671 -419
- data/lib/aspera/api/aoc.rb +71 -43
- data/lib/aspera/api/cos_node.rb +3 -2
- data/lib/aspera/api/faspex.rb +6 -5
- data/lib/aspera/api/node.rb +10 -12
- data/lib/aspera/ascmd.rb +1 -2
- data/lib/aspera/ascp/installation.rb +53 -39
- data/lib/aspera/assert.rb +25 -3
- data/lib/aspera/cli/error.rb +4 -2
- data/lib/aspera/cli/extended_value.rb +84 -60
- data/lib/aspera/cli/formatter.rb +55 -22
- data/lib/aspera/cli/main.rb +21 -14
- data/lib/aspera/cli/manager.rb +348 -247
- data/lib/aspera/cli/plugins/alee.rb +3 -3
- data/lib/aspera/cli/plugins/aoc.rb +70 -14
- data/lib/aspera/cli/plugins/base.rb +57 -49
- data/lib/aspera/cli/plugins/config.rb +69 -84
- data/lib/aspera/cli/plugins/console.rb +13 -8
- data/lib/aspera/cli/plugins/cos.rb +1 -1
- data/lib/aspera/cli/plugins/faspex.rb +32 -26
- data/lib/aspera/cli/plugins/faspex5.rb +45 -43
- data/lib/aspera/cli/plugins/faspio.rb +5 -5
- data/lib/aspera/cli/plugins/httpgw.rb +1 -1
- data/lib/aspera/cli/plugins/node.rb +131 -120
- data/lib/aspera/cli/plugins/oauth.rb +1 -1
- data/lib/aspera/cli/plugins/orchestrator.rb +114 -32
- data/lib/aspera/cli/plugins/preview.rb +26 -46
- data/lib/aspera/cli/plugins/server.rb +6 -8
- data/lib/aspera/cli/plugins/shares.rb +27 -32
- data/lib/aspera/cli/sync_actions.rb +49 -38
- data/lib/aspera/cli/transfer_agent.rb +16 -34
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/cli/wizard.rb +8 -5
- data/lib/aspera/command_line_builder.rb +20 -17
- data/lib/aspera/coverage.rb +1 -1
- data/lib/aspera/environment.rb +41 -34
- data/lib/aspera/faspex_gw.rb +1 -1
- data/lib/aspera/keychain/factory.rb +1 -2
- data/lib/aspera/markdown.rb +31 -0
- data/lib/aspera/nagios.rb +6 -5
- data/lib/aspera/oauth/base.rb +17 -27
- data/lib/aspera/oauth/factory.rb +1 -1
- data/lib/aspera/oauth/url_json.rb +2 -1
- data/lib/aspera/preview/file_types.rb +23 -37
- data/lib/aspera/products/connect.rb +3 -3
- data/lib/aspera/rest.rb +51 -39
- data/lib/aspera/rest_error_analyzer.rb +4 -4
- data/lib/aspera/ssh.rb +5 -2
- data/lib/aspera/ssl.rb +41 -0
- data/lib/aspera/sync/conf.schema.yaml +182 -34
- data/lib/aspera/sync/database.rb +2 -1
- data/lib/aspera/sync/operations.rb +125 -69
- data/lib/aspera/transfer/parameters.rb +3 -4
- data/lib/aspera/transfer/spec.rb +2 -3
- data/lib/aspera/transfer/spec.schema.yaml +48 -18
- data/lib/aspera/transfer/spec_doc.rb +14 -14
- data/lib/aspera/uri_reader.rb +1 -1
- data/lib/transferd_pb.rb +2 -2
- data.tar.gz.sig +0 -0
- metadata +19 -6
- metadata.gz.sig +3 -2
data/CONTRIBUTING.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
# Contributing
|
|
2
|
-
|
|
2
|
+
|
|
3
3
|
## Reporting Issues and Vulnerabilities
|
|
4
4
|
|
|
5
5
|
If you encounter a problem or vulnerability, please report it at [GitHub Issues](https://github.com/IBM/aspera-cli/issues).
|
|
@@ -10,7 +10,7 @@ Before submitting a new issue:
|
|
|
10
10
|
|
|
11
11
|
To help us assist you efficiently, include the following in your report:
|
|
12
12
|
|
|
13
|
-
- The version of ascli you are using:
|
|
13
|
+
- The version of `ascli` you are using:
|
|
14
14
|
|
|
15
15
|
```bash
|
|
16
16
|
ascli version
|
|
@@ -20,20 +20,22 @@ To help us assist you efficiently, include the following in your report:
|
|
|
20
20
|
- The output of your Ruby environment:
|
|
21
21
|
|
|
22
22
|
```bash
|
|
23
|
-
ruby -
|
|
23
|
+
ruby -v
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
## Making Contributions
|
|
27
27
|
|
|
28
|
-
We welcome contributions to improve the aspera-cli project!
|
|
28
|
+
We welcome contributions to improve the `aspera-cli` project!
|
|
29
29
|
|
|
30
30
|
### Getting Started
|
|
31
31
|
|
|
32
|
-
Clone the repository and navigate to the project folder:
|
|
32
|
+
Clone the repository and navigate to the project's main folder:
|
|
33
33
|
|
|
34
34
|
```bash
|
|
35
35
|
git clone https://github.com/IBM/aspera-cli.git
|
|
36
36
|
cd aspera-cli
|
|
37
|
+
bundle install
|
|
38
|
+
bundle exec rake -T
|
|
37
39
|
```
|
|
38
40
|
|
|
39
41
|
For testing instructions, refer to [Running Tests](#running-tests).
|
|
@@ -61,7 +63,7 @@ To submit a contribution:
|
|
|
61
63
|
|
|
62
64
|
## Architecture
|
|
63
65
|
|
|
64
|
-
The overall architecture of aspera-cli is modular and extensible.
|
|
66
|
+
The overall architecture of `aspera-cli` is modular and extensible.
|
|
65
67
|
|
|
66
68
|

|
|
67
69
|
|
|
@@ -84,12 +86,12 @@ A list of classes are provided in <docs/uml.png>
|
|
|
84
86
|
## Ruby version
|
|
85
87
|
|
|
86
88
|
`aspera-cli` is built with Ruby.
|
|
87
|
-
You can install Ruby using any method you prefer (e.g., rbenv
|
|
89
|
+
You can install Ruby using any method you prefer (e.g., `rbenv`, `rvm`, system package manager).
|
|
88
90
|
|
|
89
|
-
To start with a clean
|
|
91
|
+
To start with a clean state and remove all installed gems:
|
|
90
92
|
|
|
91
93
|
```bash
|
|
92
|
-
|
|
94
|
+
bundle exec rake tools:clean_gems
|
|
93
95
|
```
|
|
94
96
|
|
|
95
97
|
> [!TIP]
|
|
@@ -99,30 +101,28 @@ make clean_gems
|
|
|
99
101
|
|
|
100
102
|
TODO: document installation of tool chain.
|
|
101
103
|
|
|
102
|
-
Build system uses
|
|
104
|
+
Build system uses Ruby's `rake`.
|
|
103
105
|
|
|
104
106
|
### Environment
|
|
105
107
|
|
|
106
108
|
A few macros/env vars control some aspects:
|
|
107
109
|
|
|
108
|
-
|
|
|
109
|
-
|
|
110
|
-
| `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests |
|
|
111
|
-
| `
|
|
112
|
-
| `
|
|
113
|
-
| `
|
|
114
|
-
| `
|
|
115
|
-
| `
|
|
116
|
-
| `SIGNING_KEY` | Path to signing key to build Gem. |
|
|
117
|
-
| `GEM_VERSION` | Gem version to build container |
|
|
110
|
+
| Environment variable | Description |
|
|
111
|
+
|-----------------------------|----------------------------------------------------|
|
|
112
|
+
| `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests. |
|
|
113
|
+
| `ASPERA_CLI_DOC_CHECK_LINKS`| Check links still exist during doc generation. |
|
|
114
|
+
| `LOG_LEVEL` | Change log level in `rake` tasks. |
|
|
115
|
+
| `ENABLE_COVERAGE` | Tests with coverage analysis if set. |
|
|
116
|
+
| `SIGNING_KEY` | Path to signing key to build Gem. |
|
|
117
|
+
| `GEM_VERSION` | Override gem version for builds. |
|
|
118
118
|
|
|
119
|
-
Those macros can be set either in an env var, or on the `
|
|
119
|
+
Those macros can be set either in an env var, or on the `rake` command line.
|
|
120
120
|
|
|
121
121
|
> [!NOTE]
|
|
122
|
-
> Env vars `
|
|
123
|
-
> Others are more for "one shot" use.
|
|
122
|
+
> Env vars `ASPERA_CLI_*` are typically set in user's shell profile for development.
|
|
123
|
+
> Others are more for "one shot" use (on command line).
|
|
124
124
|
|
|
125
|
-
To use the CLI directly from the development environment, add this to your shell profile:
|
|
125
|
+
To use the CLI directly from the development environment, add this to your shell profile (adapt the real path):
|
|
126
126
|
|
|
127
127
|
```bash
|
|
128
128
|
dev_ascli=$HOME/github/aspera-cli
|
|
@@ -153,100 +153,49 @@ To check URL during doc generation, set env var: `ASPERA_CLI_DOC_CHECK_LINKS=1`.
|
|
|
153
153
|
|
|
154
154
|
To debug doc generation, set env var: `ASPERA_CLI_DOC_DEBUG=debug`.
|
|
155
155
|
|
|
156
|
-
##
|
|
157
|
-
|
|
158
|
-
First, a testing configuration file must be created.
|
|
159
|
-
From project top folder, execute:
|
|
156
|
+
## Test Environment
|
|
160
157
|
|
|
161
|
-
|
|
162
|
-
mkdir ~/some_secure_folder
|
|
163
|
-
cp docs/test_env.conf ~/some_secure_folder/.
|
|
164
|
-
```
|
|
158
|
+
Refer to <tests/README.md>.
|
|
165
159
|
|
|
166
|
-
|
|
160
|
+
## Build
|
|
167
161
|
|
|
168
|
-
|
|
162
|
+
The gem is built with:
|
|
169
163
|
|
|
170
164
|
```bash
|
|
171
|
-
|
|
165
|
+
bundle install
|
|
166
|
+
bundle exec rake build
|
|
172
167
|
```
|
|
173
168
|
|
|
174
|
-
|
|
169
|
+
If you don't want to install optional gems:
|
|
175
170
|
|
|
176
171
|
```bash
|
|
177
|
-
|
|
172
|
+
bundle config set without optional
|
|
178
173
|
```
|
|
179
174
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
One can also go to the `tests` folder:
|
|
175
|
+
Build beta version:
|
|
183
176
|
|
|
184
177
|
```bash
|
|
185
|
-
|
|
186
|
-
make
|
|
187
|
-
make full
|
|
188
|
-
make list
|
|
189
|
-
make torc
|
|
190
|
-
make skip_torc
|
|
191
|
-
make skip T=orch_wizard
|
|
178
|
+
export GEM_VERSION=$(env -u GEM_VERSION rake tools:version).$(date +%Y%m%d%H%M)
|
|
192
179
|
```
|
|
193
180
|
|
|
194
|
-
###
|
|
195
|
-
|
|
196
|
-
Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
|
|
197
|
-
By default, tests that use those gems are skipped.
|
|
198
|
-
To run them: `make optional`.
|
|
199
|
-
Those tests also require the optional gems to be installed: `make install_optional_gems`.
|
|
200
|
-
|
|
201
|
-
Some other tests require interactive input. To run them: `make interactive`
|
|
202
|
-
|
|
203
|
-
To run every test: `make full`
|
|
204
|
-
|
|
205
|
-
### Pre-release tests
|
|
206
|
-
|
|
207
|
-
For preparation of a release, do the following:
|
|
208
|
-
|
|
209
|
-
1. Select a ruby version to test with.
|
|
210
|
-
2. Remove all gems: `make clean_gems`
|
|
211
|
-
3. `cd tests && make full`
|
|
212
|
-
|
|
213
|
-
To test additional Ruby version, repeat the procedure with other Ruby versions.
|
|
214
|
-
|
|
215
|
-
## Coverage
|
|
181
|
+
### Signed gem
|
|
216
182
|
|
|
217
|
-
A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
|
|
218
|
-
Enable coverage monitoring using macro/envvar `ENABLE_COVERAGE`.
|
|
219
|
-
|
|
220
|
-
```bash
|
|
221
|
-
cd tests
|
|
222
|
-
make ENABLE_COVERAGE=1
|
|
223
|
-
```
|
|
224
|
-
|
|
225
|
-
Once tests are completed, or during test, consult the page: [coverage/index.html](coverage/index.html)
|
|
226
|
-
|
|
227
|
-
## Build
|
|
228
|
-
|
|
229
|
-
By default, the gem is built signed: `make`.
|
|
230
183
|
A private key is required to generate a signed Gem.
|
|
231
|
-
Its path must be set using
|
|
232
|
-
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
|
184
|
+
Its path must be set using envvar `SIGNING_KEY`.
|
|
185
|
+
The gem is signed with the public certificate found in `certs` and the private key pointed by `SIGNING_KEY` (kept secret by maintainer).
|
|
233
186
|
|
|
234
187
|
```bash
|
|
235
|
-
|
|
188
|
+
bundle exec rake build SIGNING_KEY=/path/to/vault/gem-private_key.pem
|
|
236
189
|
```
|
|
237
190
|
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
### Gem Signature
|
|
241
|
-
|
|
242
|
-
Refer to <certs/README.md>
|
|
191
|
+
Refer to <certs/README.md>.
|
|
243
192
|
|
|
244
193
|
### gRPC stubs for transfer SDK
|
|
245
194
|
|
|
246
195
|
Update with:
|
|
247
196
|
|
|
248
197
|
```bash
|
|
249
|
-
|
|
198
|
+
bundle exec rake tools:grpc
|
|
250
199
|
```
|
|
251
200
|
|
|
252
201
|
It downloads the latest proto file and then compiles it.
|
|
@@ -255,20 +204,14 @@ It downloads the latest proto file and then compiles it.
|
|
|
255
204
|
|
|
256
205
|
See [Container build](./container/README.md).
|
|
257
206
|
|
|
258
|
-
For operations, move to the folder:
|
|
259
|
-
|
|
260
|
-
```bash
|
|
261
|
-
cd container
|
|
262
|
-
```
|
|
263
|
-
|
|
264
207
|
## Single executable build
|
|
265
208
|
|
|
266
|
-
See [Executable build](
|
|
209
|
+
See [Executable build](build/binary/README.md).
|
|
267
210
|
|
|
268
|
-
|
|
211
|
+
To list operations:
|
|
269
212
|
|
|
270
213
|
```bash
|
|
271
|
-
|
|
214
|
+
bundle exec rake -T ^binary:
|
|
272
215
|
```
|
|
273
216
|
|
|
274
217
|
## Development check list for new release
|