aspera-cli 4.24.2 → 4.25.0.pre2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +1067 -758
- data/CONTRIBUTING.md +93 -120
- data/README.md +817 -510
- data/lib/aspera/agent/direct.rb +14 -12
- data/lib/aspera/agent/transferd.rb +4 -4
- data/lib/aspera/api/aoc.rb +71 -43
- data/lib/aspera/api/cos_node.rb +3 -2
- data/lib/aspera/api/faspex.rb +6 -5
- data/lib/aspera/api/node.rb +10 -12
- data/lib/aspera/ascmd.rb +1 -2
- data/lib/aspera/ascp/installation.rb +55 -41
- data/lib/aspera/ascp/management.rb +9 -5
- data/lib/aspera/assert.rb +28 -6
- data/lib/aspera/cli/error.rb +4 -2
- data/lib/aspera/cli/extended_value.rb +94 -62
- data/lib/aspera/cli/formatter.rb +55 -22
- data/lib/aspera/cli/main.rb +21 -14
- data/lib/aspera/cli/manager.rb +349 -248
- data/lib/aspera/cli/plugins/alee.rb +3 -3
- data/lib/aspera/cli/plugins/aoc.rb +94 -51
- data/lib/aspera/cli/plugins/base.rb +62 -49
- data/lib/aspera/cli/plugins/config.rb +85 -96
- data/lib/aspera/cli/plugins/console.rb +15 -9
- data/lib/aspera/cli/plugins/cos.rb +1 -1
- data/lib/aspera/cli/plugins/faspex.rb +34 -27
- data/lib/aspera/cli/plugins/faspex5.rb +47 -44
- data/lib/aspera/cli/plugins/faspio.rb +7 -6
- data/lib/aspera/cli/plugins/httpgw.rb +3 -2
- data/lib/aspera/cli/plugins/node.rb +132 -120
- data/lib/aspera/cli/plugins/oauth.rb +1 -1
- data/lib/aspera/cli/plugins/orchestrator.rb +116 -33
- data/lib/aspera/cli/plugins/preview.rb +26 -46
- data/lib/aspera/cli/plugins/server.rb +9 -10
- data/lib/aspera/cli/plugins/shares.rb +77 -43
- data/lib/aspera/cli/sync_actions.rb +49 -38
- data/lib/aspera/cli/transfer_agent.rb +16 -34
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/cli/wizard.rb +8 -5
- data/lib/aspera/command_line_builder.rb +20 -17
- data/lib/aspera/coverage.rb +6 -2
- data/lib/aspera/environment.rb +71 -84
- data/lib/aspera/faspex_gw.rb +1 -1
- data/lib/aspera/faspex_postproc.rb +1 -1
- data/lib/aspera/keychain/factory.rb +1 -2
- data/lib/aspera/keychain/macos_security.rb +2 -2
- data/lib/aspera/log.rb +2 -1
- data/lib/aspera/markdown.rb +31 -0
- data/lib/aspera/nagios.rb +6 -5
- data/lib/aspera/oauth/base.rb +17 -27
- data/lib/aspera/oauth/factory.rb +1 -1
- data/lib/aspera/oauth/url_json.rb +2 -1
- data/lib/aspera/preview/file_types.rb +23 -37
- data/lib/aspera/preview/terminal.rb +95 -29
- data/lib/aspera/preview/utils.rb +6 -5
- data/lib/aspera/products/connect.rb +3 -3
- data/lib/aspera/rest.rb +51 -39
- data/lib/aspera/rest_error_analyzer.rb +4 -4
- data/lib/aspera/ssh.rb +5 -2
- data/lib/aspera/ssl.rb +41 -0
- data/lib/aspera/sync/conf.schema.yaml +182 -34
- data/lib/aspera/sync/database.rb +2 -1
- data/lib/aspera/sync/operations.rb +128 -72
- data/lib/aspera/transfer/parameters.rb +3 -4
- data/lib/aspera/transfer/spec.rb +2 -3
- data/lib/aspera/transfer/spec.schema.yaml +49 -19
- data/lib/aspera/transfer/spec_doc.rb +14 -14
- data/lib/aspera/uri_reader.rb +1 -1
- data/lib/transferd_pb.rb +2 -2
- data.tar.gz.sig +0 -0
- metadata +33 -6
- metadata.gz.sig +0 -0
data/CONTRIBUTING.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Contributing
|
|
2
|
-
|
|
2
|
+
|
|
3
3
|
## Reporting Issues and Vulnerabilities
|
|
4
4
|
|
|
5
|
-
If you encounter a problem or vulnerability, please report it
|
|
5
|
+
If you encounter a problem or a security vulnerability, please report it on [GitHub Issues](https://github.com/IBM/aspera-cli/issues).
|
|
6
6
|
|
|
7
7
|
Before submitting a new issue:
|
|
8
8
|
|
|
@@ -10,30 +10,32 @@ Before submitting a new issue:
|
|
|
10
10
|
|
|
11
11
|
To help us assist you efficiently, include the following in your report:
|
|
12
12
|
|
|
13
|
-
- The version of ascli you are using:
|
|
13
|
+
- The version of `ascli` you are using:
|
|
14
14
|
|
|
15
15
|
```bash
|
|
16
16
|
ascli version
|
|
17
17
|
```
|
|
18
18
|
|
|
19
|
-
- Confirmation that you
|
|
20
|
-
-
|
|
19
|
+
- Confirmation that you are using the latest version (update it if needed).
|
|
20
|
+
- Your Ruby version information:
|
|
21
21
|
|
|
22
22
|
```bash
|
|
23
|
-
ruby -
|
|
23
|
+
ruby -v
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
## Making Contributions
|
|
27
27
|
|
|
28
|
-
We welcome contributions to improve the aspera-cli project!
|
|
28
|
+
We welcome contributions to improve the `aspera-cli` project!
|
|
29
29
|
|
|
30
30
|
### Getting Started
|
|
31
31
|
|
|
32
|
-
Clone the repository and navigate to the project
|
|
32
|
+
Clone the repository and navigate to the project's root directory:
|
|
33
33
|
|
|
34
34
|
```bash
|
|
35
35
|
git clone https://github.com/IBM/aspera-cli.git
|
|
36
36
|
cd aspera-cli
|
|
37
|
+
bundle install
|
|
38
|
+
bundle exec rake -T
|
|
37
39
|
```
|
|
38
40
|
|
|
39
41
|
For testing instructions, refer to [Running Tests](#running-tests).
|
|
@@ -48,7 +50,7 @@ To submit a contribution:
|
|
|
48
50
|
|
|
49
51
|
1. **Implement** your feature or bug fix.
|
|
50
52
|
|
|
51
|
-
1. **Write tests** to ensure your changes are robust and
|
|
53
|
+
1. **Write tests** to ensure your changes are robust and prevent regressions.
|
|
52
54
|
|
|
53
55
|
1. Run `rubocop` to ensure your code follows the Ruby style guide.
|
|
54
56
|
|
|
@@ -61,68 +63,64 @@ To submit a contribution:
|
|
|
61
63
|
|
|
62
64
|
## Architecture
|
|
63
65
|
|
|
64
|
-
The overall architecture of aspera-cli is modular and extensible.
|
|
66
|
+
The overall architecture of `aspera-cli` is modular and extensible.
|
|
65
67
|
|
|
66
68
|

|
|
67
69
|
|
|
68
|
-
Structure Highlights
|
|
70
|
+
### Structure Highlights
|
|
69
71
|
|
|
70
72
|
- Entry Point:
|
|
71
73
|
|
|
72
|
-
`lib/aspera/cli/main.rb
|
|
74
|
+
`lib/aspera/cli/main.rb`, CLI startup logic.
|
|
73
75
|
|
|
74
76
|
- Plugins:
|
|
75
77
|
|
|
76
|
-
Located in `lib/aspera/cli/plugins
|
|
78
|
+
Located in `lib/aspera/cli/plugins`; plugins extend CLI functionality and encapsulate specific features.
|
|
77
79
|
|
|
78
80
|
- Transfer Agents:
|
|
79
81
|
|
|
80
|
-
Found in `lib/aspera/agent`, these handle data
|
|
82
|
+
Found in `lib/aspera/agent`, these handle data transfer operations.
|
|
81
83
|
|
|
82
|
-
|
|
84
|
+
Class diagrams are provided in <docs/uml.png>
|
|
83
85
|
|
|
84
|
-
## Ruby
|
|
86
|
+
## Ruby Environment
|
|
85
87
|
|
|
86
|
-
`aspera-cli` is
|
|
87
|
-
You can install Ruby using any method you prefer (e.g., rbenv
|
|
88
|
+
`aspera-cli` is written in Ruby.
|
|
89
|
+
You can install Ruby using any method you prefer (e.g., `rbenv`, `rvm`, system package manager).
|
|
88
90
|
|
|
89
|
-
To start with a clean
|
|
91
|
+
To start with a clean state and remove all installed gems:
|
|
90
92
|
|
|
91
93
|
```bash
|
|
92
|
-
|
|
94
|
+
bundle exec rake tools:clean_gems
|
|
93
95
|
```
|
|
94
96
|
|
|
95
97
|
> [!TIP]
|
|
96
98
|
> This is especially useful before testing across different Ruby versions or preparing for a release.
|
|
97
99
|
|
|
98
|
-
##
|
|
99
|
-
|
|
100
|
-
TODO: document installation of tool chain.
|
|
100
|
+
## Toolchain
|
|
101
101
|
|
|
102
|
-
|
|
102
|
+
The build system uses Ruby's `rake`.
|
|
103
103
|
|
|
104
104
|
### Environment
|
|
105
105
|
|
|
106
|
-
A few macros
|
|
106
|
+
A few macros and environment variables control certain aspects of the build:
|
|
107
107
|
|
|
108
|
-
|
|
|
109
|
-
|
|
110
|
-
| `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests
|
|
111
|
-
| `
|
|
112
|
-
| `
|
|
113
|
-
| `
|
|
114
|
-
| `
|
|
115
|
-
| `
|
|
116
|
-
| `SIGNING_KEY` | Path to signing key to build Gem. |
|
|
117
|
-
| `GEM_VERSION` | Gem version to build container |
|
|
108
|
+
| Environment variable | Description |
|
|
109
|
+
|-----------------------------|-----------------------------------------------------|
|
|
110
|
+
| `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests. |
|
|
111
|
+
| `ASPERA_CLI_DOC_CHECK_LINKS`| Check links still exist during doc generation. |
|
|
112
|
+
| `LOG_LEVEL` | Change log level in `rake` tasks. |
|
|
113
|
+
| `ENABLE_COVERAGE` | Enable test coverage analysis when set. |
|
|
114
|
+
| `SIGNING_KEY` | Path to the signing key used to build the gem file. |
|
|
115
|
+
| `GEM_VERSION` | Override gem version for builds. |
|
|
118
116
|
|
|
119
|
-
|
|
117
|
+
These can be set either as environment variables or directly on the `rake` command line.
|
|
120
118
|
|
|
121
119
|
> [!NOTE]
|
|
122
|
-
>
|
|
123
|
-
> Others are
|
|
120
|
+
> Environment variables `ASPERA_CLI_*` are typically set in the user’s shell profile for development.
|
|
121
|
+
> Others are intended for one‑time use on the command line.
|
|
124
122
|
|
|
125
|
-
To use the CLI directly from the development environment, add this to your shell profile:
|
|
123
|
+
To use the CLI directly from the development environment, add this to your shell profile (adapt the real path):
|
|
126
124
|
|
|
127
125
|
```bash
|
|
128
126
|
dev_ascli=$HOME/github/aspera-cli
|
|
@@ -134,9 +132,9 @@ export RUBYLIB=$dev_ascli/lib:$RUBYLIB
|
|
|
134
132
|
|
|
135
133
|
Documentation is generated with `pandoc` and `LaTeX`.
|
|
136
134
|
|
|
137
|
-
IBM
|
|
135
|
+
The IBM `Plex` font is used; for installation instructions, see [IBM Plex](https://www.ibm.com/plex/).
|
|
138
136
|
|
|
139
|
-
On macOS to install `lualatex` and all packages:
|
|
137
|
+
On macOS, to install `lualatex` and all packages:
|
|
140
138
|
|
|
141
139
|
```bash
|
|
142
140
|
brew install texlive
|
|
@@ -149,138 +147,113 @@ tlmgr update --self
|
|
|
149
147
|
tlmgr install fvextra selnolig lualatex-math
|
|
150
148
|
```
|
|
151
149
|
|
|
152
|
-
To check
|
|
150
|
+
To check URLs during documentation generation, set the environment variable: `ASPERA_CLI_DOC_CHECK_LINKS=1`.
|
|
151
|
+
|
|
152
|
+
To debug documentation generation, set the environment variable: `ASPERA_CLI_DOC_DEBUG=debug`.
|
|
153
|
+
|
|
154
|
+
## Test Environment
|
|
153
155
|
|
|
154
|
-
|
|
156
|
+
Refer to <tests/README.md>.
|
|
155
157
|
|
|
156
|
-
##
|
|
158
|
+
## Build
|
|
157
159
|
|
|
158
|
-
|
|
159
|
-
From project top folder, execute:
|
|
160
|
+
The gem is built with:
|
|
160
161
|
|
|
161
162
|
```bash
|
|
162
|
-
|
|
163
|
-
|
|
163
|
+
bundle install
|
|
164
|
+
bundle exec rake build
|
|
164
165
|
```
|
|
165
166
|
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
Then, tell where this file is located:
|
|
167
|
+
If you don't want to install optional gems:
|
|
169
168
|
|
|
170
169
|
```bash
|
|
171
|
-
|
|
170
|
+
bundle config set without optional
|
|
172
171
|
```
|
|
173
172
|
|
|
174
|
-
|
|
173
|
+
To build a beta version:
|
|
175
174
|
|
|
176
175
|
```bash
|
|
177
|
-
|
|
176
|
+
export GEM_VERSION=$(env -u GEM_VERSION rake tools:version).$(date +%Y%m%d%H%M)
|
|
178
177
|
```
|
|
179
178
|
|
|
180
|
-
|
|
179
|
+
### Signed gem
|
|
181
180
|
|
|
182
|
-
|
|
181
|
+
A private key is required to generate a signed gem.
|
|
182
|
+
Its path must be set using environment variable `SIGNING_KEY`.
|
|
183
|
+
The gem is signed with the public certificate found in `certs` and the private key specified by `SIGNING_KEY` (kept secret by the maintainer).
|
|
183
184
|
|
|
184
185
|
```bash
|
|
185
|
-
|
|
186
|
-
make
|
|
187
|
-
make full
|
|
188
|
-
make list
|
|
189
|
-
make torc
|
|
190
|
-
make skip_torc
|
|
191
|
-
make skip T=orch_wizard
|
|
186
|
+
bundle exec rake build SIGNING_KEY=/path/to/vault/gem-private_key.pem
|
|
192
187
|
```
|
|
193
188
|
|
|
194
|
-
|
|
189
|
+
Refer to <certs/README.md>.
|
|
195
190
|
|
|
196
|
-
|
|
197
|
-
By default, tests that use those gems are skipped.
|
|
198
|
-
To run them: `make optional`.
|
|
199
|
-
Those tests also require the optional gems to be installed: `make install_optional_gems`.
|
|
191
|
+
### gRPC stubs for transfer SDK
|
|
200
192
|
|
|
201
|
-
|
|
193
|
+
Update with:
|
|
202
194
|
|
|
203
|
-
|
|
195
|
+
```bash
|
|
196
|
+
bundle exec rake tools:grpc
|
|
197
|
+
```
|
|
204
198
|
|
|
205
|
-
|
|
199
|
+
It downloads the latest `proto` file and then compiles it.
|
|
206
200
|
|
|
207
|
-
|
|
201
|
+
## Container image build
|
|
208
202
|
|
|
209
|
-
|
|
210
|
-
2. Remove all gems: `make clean_gems`
|
|
211
|
-
3. `cd tests && make full`
|
|
203
|
+
See [Container build](./container/README.md).
|
|
212
204
|
|
|
213
|
-
|
|
205
|
+
## Single executable build
|
|
214
206
|
|
|
215
|
-
|
|
207
|
+
See [Executable build](build/binary/README.md).
|
|
216
208
|
|
|
217
|
-
|
|
218
|
-
Enable coverage monitoring using macro/envvar `ENABLE_COVERAGE`.
|
|
209
|
+
To list operations:
|
|
219
210
|
|
|
220
211
|
```bash
|
|
221
|
-
|
|
222
|
-
make ENABLE_COVERAGE=1
|
|
212
|
+
bundle exec rake -T ^binary:
|
|
223
213
|
```
|
|
224
214
|
|
|
225
|
-
|
|
215
|
+
## Release
|
|
226
216
|
|
|
227
|
-
|
|
217
|
+
### Checklist before a new release
|
|
218
|
+
|
|
219
|
+
When preparing for a new release, do the following:
|
|
228
220
|
|
|
229
|
-
|
|
230
|
-
A private key is required to generate a signed Gem.
|
|
231
|
-
Its path must be set using macro/envvar `SIGNING_KEY`, see below.
|
|
232
|
-
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
|
221
|
+
- Run the test suite:
|
|
233
222
|
|
|
234
223
|
```bash
|
|
235
|
-
|
|
224
|
+
bundle exec rake test:run
|
|
236
225
|
```
|
|
237
226
|
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
### Gem Signature
|
|
241
|
-
|
|
242
|
-
Refer to <certs/README.md>
|
|
243
|
-
|
|
244
|
-
### gRPC stubs for transfer SDK
|
|
245
|
-
|
|
246
|
-
Update with:
|
|
227
|
+
- Set beta version (to use the gem file built):
|
|
247
228
|
|
|
248
229
|
```bash
|
|
249
|
-
|
|
230
|
+
export GEM_VERSION=$(env -u GEM_VERSION rake tools:version).$(date +%Y%m%d%H%M)
|
|
250
231
|
```
|
|
251
232
|
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
## Container image build
|
|
255
|
-
|
|
256
|
-
See [Container build](./container/README.md).
|
|
257
|
-
|
|
258
|
-
For operations, move to the folder:
|
|
233
|
+
- Verify that the container builds successfully (using the beta version):
|
|
259
234
|
|
|
260
235
|
```bash
|
|
261
|
-
|
|
236
|
+
bundle exec rake container:build
|
|
237
|
+
bundle exec rake container:test
|
|
262
238
|
```
|
|
263
239
|
|
|
264
|
-
|
|
240
|
+
### Version Release Process
|
|
265
241
|
|
|
266
|
-
|
|
242
|
+
Once the development branch is ready for release:
|
|
267
243
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
```bash
|
|
271
|
-
cd binary
|
|
272
|
-
```
|
|
244
|
+
- Decide on the version number
|
|
273
245
|
|
|
274
|
-
|
|
246
|
+
- Update the version in `lib/aspera/cli/version.rb`
|
|
275
247
|
|
|
276
|
-
|
|
248
|
+
- Execute `bundle exec rake release` (see `.github/workflows/deploy.yml`)
|
|
277
249
|
|
|
278
|
-
-
|
|
250
|
+
- It builds the `.gem` file in the `pkg` folder.
|
|
251
|
+
- Then runs `gem push`.
|
|
279
252
|
|
|
280
|
-
## Long
|
|
253
|
+
## Long‑Term Implementation and Delivery Improvements
|
|
281
254
|
|
|
282
|
-
-
|
|
255
|
+
- Replace custom REST and OAuth classes with standard Ruby gems:
|
|
283
256
|
- <https://github.com/rest-client/rest-client>
|
|
284
257
|
- <https://github.com/oauth-xx/oauth2>
|
|
285
|
-
-
|
|
286
|
-
-
|
|
258
|
+
- Use the `thor` gem <http://whatisthor.com/> (or other standard Ruby CLI manager)
|
|
259
|
+
- Look at <https://github.com/phusion/traveling-ruby>
|