aspera-cli 4.21.1 → 4.21.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +4 -2
- data/CHANGELOG.md +22 -10
- data/CONTRIBUTING.md +68 -143
- data/README.md +175 -145
- data/bin/ascli +5 -14
- data/bin/asession +1 -3
- data/examples/get_proto_file.rb +4 -3
- data/examples/proxy.pac +20 -20
- data/lib/aspera/agent/base.rb +2 -0
- data/lib/aspera/agent/{alpha.rb → desktop.rb} +7 -7
- data/lib/aspera/agent/direct.rb +9 -1
- data/lib/aspera/agent/transferd.rb +24 -17
- data/lib/aspera/api/alee.rb +1 -1
- data/lib/aspera/api/cos_node.rb +1 -1
- data/lib/aspera/api/node.rb +4 -4
- data/lib/aspera/ascp/installation.rb +40 -58
- data/lib/aspera/cli/extended_value.rb +9 -3
- data/lib/aspera/cli/formatter.rb +8 -1
- data/lib/aspera/cli/info.rb +1 -0
- data/lib/aspera/cli/main.rb +2 -2
- data/lib/aspera/cli/manager.rb +3 -3
- data/lib/aspera/cli/plugins/aoc.rb +106 -55
- data/lib/aspera/cli/plugins/config.rb +10 -31
- data/lib/aspera/cli/plugins/faspex5.rb +8 -5
- data/lib/aspera/cli/plugins/node.rb +6 -3
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/coverage.rb +5 -3
- data/lib/aspera/environment.rb +20 -11
- data/lib/aspera/faspex_postproc.rb +3 -5
- data/lib/aspera/hash_ext.rb +2 -12
- data/lib/aspera/oauth/base.rb +6 -1
- data/lib/aspera/preview/generator.rb +12 -9
- data/lib/aspera/preview/options.rb +2 -2
- data/lib/aspera/preview/terminal.rb +1 -1
- data/lib/aspera/preview/utils.rb +4 -4
- data/lib/aspera/products/connect.rb +34 -0
- data/lib/aspera/products/{alpha.rb → desktop.rb} +2 -2
- data/lib/aspera/products/transferd.rb +8 -1
- data/lib/aspera/rest.rb +4 -4
- data/lib/aspera/secret_hider.rb +7 -0
- data/lib/aspera/temp_file_manager.rb +5 -4
- data/lib/aspera/uri_reader.rb +18 -1
- data.tar.gz.sig +0 -0
- metadata +4 -174
- metadata.gz.sig +0 -0
- data/examples/build_exec +0 -74
- data/examples/build_exec_rubyc +0 -40
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3d66238170234ebc23390a21ef7a492ec50c8707200c774ef2f3195fdb13261f
|
4
|
+
data.tar.gz: 47623d1ce5961c1d1050989e36afeb1e008dc24c3b81d34d5b484f39d0826107
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: dfa7a18bfa476bfda979e88017bf643ef181ecd643f53a6872f7b0af833c6e2dc46700d5afc913126942cda6eb0804f7332b5555e8054ab1d9dba97873000e6e
|
7
|
+
data.tar.gz: 2a598cd0d4aab0fb8ec6e2660f590d78fd41a62870f554f96f26069cf0499753dcedb30f807691d8156441340dbbed2910f560b29d1e6e5bbedfd7416f542825
|
checksums.yaml.gz.sig
CHANGED
@@ -1,2 +1,4 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
���?7�n�$��a�m�i�;Kl���Q���o���v~�^��>�ӴNb���@8�`���T�6� ܾ0
|
2
|
+
(��U�$�Y��б�Ҽ�g,���(�b�!#On!;;���C��
|
3
|
+
���M_C��Əy��&��b�g�y��io�f�*E��9����#���Py'�Y��B:L�0)�4H� t|�����s�N�
|
4
|
+
b���5#K��|k��/�f�j��uߢKԋ"��#cF������mc��gdvs��5>2,���R^Ԧ�iNUL����SD�U�j`��g����R�8�
|
data/CHANGELOG.md
CHANGED
@@ -1,30 +1,42 @@
|
|
1
1
|
# Changes (Release notes)
|
2
2
|
|
3
|
+
* 4.21.2
|
4
|
+
|
5
|
+
* New Features:
|
6
|
+
* **container**: Updated Ruby to 3.4.2
|
7
|
+
* Issues Fixed:
|
8
|
+
* **global**: #185 `@val:` shall stop processing extended values
|
9
|
+
* **global**: #186 Removed dependency on openssl 3.3 gem to avoid msys dep on Windows.
|
10
|
+
* `echo`: Display of list (Array) was showing only first element of it.
|
11
|
+
* `transferd`: support for version 1.1.5+
|
12
|
+
* Breaking Changes:
|
13
|
+
* `preview`: Updated Image Magick to v7+
|
14
|
+
* `aoc`: `admin subscription` split into `admin subscription account` and `admin subscription usage`
|
15
|
+
* **agent**: `alpha` renamed to `desktop`
|
16
|
+
|
3
17
|
* 4.21.1
|
4
18
|
|
5
19
|
* New Features:
|
6
20
|
* `config`: New command: `transferd` to list and install specific version of `asperatransferd` and `ascp`
|
7
|
-
* `config`: New command: `token` to list and flush cached tokens.
|
8
|
-
* `faspex5`: New command: `admin contact reset_password`
|
9
21
|
* `config`: New command: `tokens` with `list`, `show`, `flush` (replace `flush_tokens`)
|
22
|
+
* `faspex5`: New command: `admin contact reset_password`
|
10
23
|
* `aoc`: #178 packages can be browsed, and individual files can be downloaded now.
|
11
24
|
* Issues Fixed:
|
25
|
+
* `config`: #175 `ascli config preset set GLOBAL version_check_days 0` causes a bad `config.yaml` to be written
|
12
26
|
* `config`: #180 problem in `ascp install`
|
13
|
-
* `aoc`: #184 token cache shall be different per aoc org.
|
14
27
|
* `config`: Soft links in transfer SDK archive are correctly extracted
|
28
|
+
* `aoc`: #184 token cache shall be different per aoc org.
|
15
29
|
* `aoc`: Fix `packages delete` not working.
|
16
30
|
* `direct` agent: #174 Race condition fix with `ascp`: timeout waiting mgt port connect (select not readable)
|
17
|
-
* `config`: #175 `ascli config preset set GLOBAL version_check_days 0` causes a bad `config.yaml` to be written
|
18
31
|
* `preview`: #177 fix bug that prevents preview generation to work.
|
19
32
|
* Breaking Changes:
|
20
|
-
* **global** : Ruby minimum versions is now 3.1 (mainly due to switch to openssl 3). Future minimum is 3.2. Recommended is 3.4.
|
21
33
|
* `transferd`: Use of Aspera Transfer Daemon requires minimum version 1.1.4. agent `trsdk` renamed `transferd`.
|
22
34
|
* `ascp`: Default SDK version is now 1.1.4. Removes support for ascp4.
|
23
|
-
* **global** : Options `transpose_single` and `multi_table` replaced with single option `multi_single` and values: `no`, `yes`, `single`.
|
24
|
-
* **global** : Column name for single object is now `field` instead of `key`.
|
25
|
-
* **global** : Ruby minimum versions is now 2.7 (2.6 previously, that removes macOS default ruby support, but newer ruby version can be installed on macOS)
|
26
35
|
* `node`: Removed deprecated command prefix `exec:`, use `@ruby:` instead.
|
27
|
-
* **global
|
36
|
+
* **global**: Now uses openssl 3.
|
37
|
+
* **global**: Ruby minimum versions is now 3.1 (mainly due to switch to openssl 3). Future minimum is 3.2. Recommended is 3.4. (that removes macOS default ruby support. Newer Ruby version shall be installed on macOS with `brew`)
|
38
|
+
* **global**: Options `transpose_single` and `multi_table` replaced with single option `multi_single` and values: `no`, `yes`, `single`.
|
39
|
+
* **global**: Column name for single object is now `field` instead of `key`.
|
28
40
|
|
29
41
|
* 4.20.0
|
30
42
|
|
@@ -35,7 +47,7 @@
|
|
35
47
|
* `node`: New option `node_cache` (bool) for gen4 operations.
|
36
48
|
* `node`: Option `root_id` now works always for node gen4, as well as `%id:` for file selection in addition to path.
|
37
49
|
* `node`: `transfer list` now uses the `iteration_token` to retrieve all values. Option `once_only` is now supported.
|
38
|
-
* **global
|
50
|
+
* **global**: option `http_options` now include retry options.
|
39
51
|
* Issues Fixed:
|
40
52
|
* `aoc`: Fixed `find` command not working. (undefined variable)
|
41
53
|
* `aoc`: #165 aoc mkdir now follows the last link of containing folder
|
data/CONTRIBUTING.md
CHANGED
@@ -9,34 +9,34 @@ Before you go ahead please search existing issues for your problem.
|
|
9
9
|
To make sure that we can help you quickly please include and check the following information:
|
10
10
|
|
11
11
|
- Include the `ascli` version you are running in your report.
|
12
|
-
- If you are not running the latest version (please check)
|
12
|
+
- If you are not running the latest version (please check) then update.
|
13
13
|
- Include your `ruby -e "puts RUBY_DESCRIPTION"`.
|
14
14
|
|
15
15
|
Thanks!
|
16
16
|
|
17
17
|
## Making Contributions
|
18
18
|
|
19
|
-
To fetch
|
19
|
+
To fetch and test the gem for development, do:
|
20
20
|
|
21
21
|
```bash
|
22
22
|
git clone https://github.com/IBM/aspera-cli.git
|
23
23
|
cd aspera-cli
|
24
|
-
bundle install
|
25
|
-
make test
|
26
24
|
```
|
27
25
|
|
26
|
+
Then see [Running Tests](#running-tests).
|
27
|
+
|
28
28
|
If you want to contribute, please:
|
29
29
|
|
30
30
|
- Fork the project.
|
31
31
|
- Make your feature addition or bug fix.
|
32
32
|
- Add tests for it. This is important so I don't break it in a future version unintentionally.
|
33
|
-
-
|
33
|
+
- Run `rubocop` to comply for coding standards
|
34
|
+
- Update `CHANGELOG.md`
|
34
35
|
- Send a pull request on GitHub.
|
35
|
-
- run `rubocop` to comply for coding standards
|
36
36
|
|
37
37
|
## Architecture
|
38
38
|
|
39
|
-
A list of classes are provided in <docs/
|
39
|
+
A list of classes are provided in <docs/uml.png>
|
40
40
|
|
41
41
|
Architecture:
|
42
42
|
|
@@ -46,22 +46,49 @@ The entry point is: `lib/aspera/cli/main.rb`.
|
|
46
46
|
|
47
47
|
Plugins are located in: `lib/aspera/cli/plugins`.
|
48
48
|
|
49
|
-
Transfer agents, in: `lib/aspera/
|
49
|
+
Transfer agents, in: `lib/aspera/agent`.
|
50
50
|
|
51
51
|
## Ruby version
|
52
52
|
|
53
|
-
Install
|
53
|
+
Install Ruby using your prefered method.
|
54
54
|
|
55
|
-
|
55
|
+
To cleanup installed gems to start fresh:
|
56
56
|
|
57
57
|
```bash
|
58
|
-
make
|
58
|
+
make clean_gems
|
59
59
|
```
|
60
60
|
|
61
61
|
## Tool chain
|
62
62
|
|
63
63
|
TODO: document installation of tool chain.
|
64
64
|
|
65
|
+
Build system uses GNU Make.
|
66
|
+
|
67
|
+
### Environment
|
68
|
+
|
69
|
+
A few macros/envvars control some aspects:
|
70
|
+
|
71
|
+
| macro | description |
|
72
|
+
|-----------------------------|--------------------------------------|
|
73
|
+
| `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests |
|
74
|
+
| `ASPERA_CLI_TEST_MACOS` | Set to `true` if local HSTS running on macOS |
|
75
|
+
| `ASPERA_CLI_TEST_PRIVATE` | Path to private folder |
|
76
|
+
| `ENABLE_COVERAGE` | Tests with coverage analysis if set. |
|
77
|
+
| `SIGNING_KEY` | Path to signing key to build Gem. |
|
78
|
+
| `GEM_VERSION` | Gem version to build container |
|
79
|
+
|
80
|
+
Those macros can be set either in an env var, or on the `make` command line.
|
81
|
+
|
82
|
+
> **Note:** vars `ASPERA_CLI_TEST_` are typically set in user's shell profile.
|
83
|
+
|
84
|
+
To use the CLI directly from the development environment, add this to your shell profile:
|
85
|
+
|
86
|
+
```bash
|
87
|
+
dev_ascli=$HOME/github/aspera-cli
|
88
|
+
export PATH=$dev_ascli/bin:$PATH
|
89
|
+
export RUBYLIB=$dev_ascli/lib:$RUBYLIB
|
90
|
+
```
|
91
|
+
|
65
92
|
### Documentation
|
66
93
|
|
67
94
|
Documentation is generated with `pandoc` and `LaTeX`.
|
@@ -83,7 +110,8 @@ tlmgr install fvextra selnolig lualatex-math
|
|
83
110
|
|
84
111
|
## Running Tests
|
85
112
|
|
86
|
-
First, a testing configuration file must be created
|
113
|
+
First, a testing configuration file must be created.
|
114
|
+
From project top folder, execute:
|
87
115
|
|
88
116
|
```bash
|
89
117
|
mkdir ~/some_secure_folder
|
@@ -104,7 +132,7 @@ This project uses a `Makefile` for tests:
|
|
104
132
|
make test
|
105
133
|
```
|
106
134
|
|
107
|
-
When new commands are added to the CLI, new tests shall be added to the test suite
|
135
|
+
When new commands are added to the CLI, new tests shall be added to the test suite in `tests/Makefile`.
|
108
136
|
|
109
137
|
### Special tests
|
110
138
|
|
@@ -121,110 +149,40 @@ To run every test: `make full`
|
|
121
149
|
|
122
150
|
For preparation of a release, do the following:
|
123
151
|
|
124
|
-
1. Select a ruby version to test with
|
152
|
+
1. Select a ruby version to test with.
|
125
153
|
2. Remove all gems: `make clean_gems`
|
126
|
-
3.
|
127
|
-
4. Install optional gems: `make install_optional_gems`
|
128
|
-
5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
|
129
|
-
|
130
|
-
This ensures that optional gems do not prevent `ascli` to run.
|
154
|
+
3. `cd tests && make full`
|
131
155
|
|
132
156
|
To test additional Ruby version, repeat the procedure with other Ruby versions.
|
133
157
|
|
134
158
|
## Coverage
|
135
159
|
|
136
160
|
A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
|
137
|
-
Enable coverage monitoring using
|
161
|
+
Enable coverage monitoring using macro/envvar `ENABLE_COVERAGE`.
|
138
162
|
|
139
163
|
```bash
|
140
|
-
|
164
|
+
cd tests
|
165
|
+
make ENABLE_COVERAGE=1
|
141
166
|
```
|
142
167
|
|
143
168
|
Once tests are completed, or during test, consult the page: [coverage/index.html](coverage/index.html)
|
144
169
|
|
145
170
|
## Build
|
146
171
|
|
147
|
-
By default the gem is built signed: `make`.
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
### Gem Signature
|
152
|
-
|
153
|
-
Refer to:
|
154
|
-
|
155
|
-
- <https://guides.rubygems.org/security/>
|
156
|
-
- <https://ruby-doc.org/current/stdlibs/rubygems/Gem/Security.html>
|
157
|
-
- `gem cert --help`
|
158
|
-
|
159
|
-
Then procedure is as follows:
|
160
|
-
|
161
|
-
- The maintainer creates the initial certificate and a private key:
|
162
|
-
|
163
|
-
```bash
|
164
|
-
cd /path/to/vault
|
165
|
-
gem cert --build maintainer@example.com
|
166
|
-
```
|
167
|
-
|
168
|
-
> **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
|
169
|
-
|
170
|
-
This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
|
171
|
-
|
172
|
-
- `gem-private_key.pem` : This file shall be kept secret in a vault.
|
173
|
-
- `gem-public_cert.pem` : This file is copied to a public place, here in folder `certs`
|
174
|
-
|
175
|
-
> **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
|
176
|
-
|
177
|
-
- The maintainer builds the signed gem.
|
178
|
-
|
179
|
-
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
180
|
-
|
181
|
-
To build the signed gem:
|
182
|
-
|
183
|
-
```bash
|
184
|
-
SIGNING_KEY=/path/to/vault/gem-private_key.pem make
|
185
|
-
```
|
186
|
-
|
187
|
-
- The user can activate gem signature verification on installation:
|
188
|
-
|
189
|
-
Add the certificate to gem trusted certificates:
|
190
|
-
|
191
|
-
```bash
|
192
|
-
curl https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem -so aspera-cli-certificate.pem
|
193
|
-
gem cert --add aspera-cli-certificate.pem
|
194
|
-
rm aspera-cli-certificate.pem
|
195
|
-
```
|
196
|
-
|
197
|
-
- The user installs the gem with `HighSecurity` or `MediumSecurity`: this will succeed only of the gem is trusted.
|
172
|
+
By default, the gem is built signed: `make`.
|
173
|
+
A private key is required to generate a signed Gem.
|
174
|
+
Its path must be set using macro/envvar `SIGNING_KEY`, see below.
|
175
|
+
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
198
176
|
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
- The maintainer can renew the certificate when it is expired using the same private key:
|
204
|
-
|
205
|
-
```bash
|
206
|
-
SIGNING_KEY=/path/to/vault/gem-private_key.pem make update-cert
|
207
|
-
```
|
208
|
-
|
209
|
-
Alternatively, to generate a new certificate with the same key:
|
210
|
-
|
211
|
-
```bash
|
212
|
-
SIGNING_KEY=/path/to/vault/gem-private_key.pem make new-cert
|
213
|
-
```
|
214
|
-
|
215
|
-
- Show the current certificate contents
|
216
|
-
|
217
|
-
```bash
|
218
|
-
make show-cert
|
219
|
-
```
|
177
|
+
```bash
|
178
|
+
make SIGNING_KEY=/path/to/vault/gem-private_key.pem
|
179
|
+
```
|
220
180
|
|
221
|
-
|
181
|
+
It is also possible to build an unsigned version for development purpose: `make unsigned_gem`.
|
222
182
|
|
223
|
-
|
183
|
+
### Gem Signature
|
224
184
|
|
225
|
-
|
226
|
-
SIGNING_KEY=/path/to/vault/gem-private_key.pem make check-cert-key
|
227
|
-
```
|
185
|
+
Refer to <certs/README.md>
|
228
186
|
|
229
187
|
### gRPC stubs for transfer SDK
|
230
188
|
|
@@ -236,54 +194,14 @@ make grpc
|
|
236
194
|
|
237
195
|
It downloads the latest proto file and then compiles it.
|
238
196
|
|
239
|
-
##
|
240
|
-
|
241
|
-
The `Dockerfile.tmpl.erb` template allows customization of the optional copying of the `aspera-cli.gem` file, as opposed to installing from rubygems.org. It also supports customizing the retrieval of the SDK.
|
197
|
+
## Container image build
|
242
198
|
|
243
|
-
|
199
|
+
See [Container build](./container/README.md).
|
244
200
|
|
245
|
-
|
201
|
+
For operations, move to the folder:
|
246
202
|
|
247
203
|
```bash
|
248
|
-
|
249
|
-
```
|
250
|
-
|
251
|
-
This does the following:
|
252
|
-
|
253
|
-
- Install the official gem version directly from <rubygems.org>.
|
254
|
-
- Build the image for the version number in the current repository
|
255
|
-
- creates tags for both the version and `latest`
|
256
|
-
|
257
|
-
> **Note:** This target creates the `Dockerfile` from an `ERB` (embedded Ruby) template (Makefile target `dockerfile`).
|
258
|
-
A template is used as it allows some level of customization to tell where to take the gem from, as as for the SDK file.
|
259
|
-
|
260
|
-
Then, to push to the image registry (both tags: version and `latest`):
|
261
|
-
|
262
|
-
```bash
|
263
|
-
make docker_push
|
264
|
-
```
|
265
|
-
|
266
|
-
### Specific version image build
|
267
|
-
|
268
|
-
To build a specific version: override `make` macro `GEM_VERSION`:
|
269
|
-
|
270
|
-
```bash
|
271
|
-
make docker GEM_VERSION=4.11.0
|
272
|
-
make docker_push GEM_VERSION=4.11.0
|
273
|
-
```
|
274
|
-
|
275
|
-
> **Note:** This does not use the locally generated gem file.
|
276
|
-
Only the local docker file is used.
|
277
|
-
The gem is installed from rubygems.org. This also sets the `latest` tag.
|
278
|
-
|
279
|
-
### Development version image build
|
280
|
-
|
281
|
-
To build/push a beta/development container:
|
282
|
-
it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
|
283
|
-
|
284
|
-
```bash
|
285
|
-
make docker_beta
|
286
|
-
make docker_push_beta
|
204
|
+
cd container
|
287
205
|
```
|
288
206
|
|
289
207
|
## Single executable build
|
@@ -295,9 +213,16 @@ Initially, `rubyc` (gem [`ruby-packer`](https://github.com/pmq20/ruby-packer) an
|
|
295
213
|
A modern version of this is now used: [`tebako`](https://github.com/tamatebako/tebako) for which a container is provided.
|
296
214
|
|
297
215
|
```bash
|
298
|
-
|
216
|
+
cd binary
|
217
|
+
make GEM_VERSION=4.11.0
|
299
218
|
```
|
300
219
|
|
220
|
+
## Development check list for new release
|
221
|
+
|
222
|
+
When preparing for a new release do the following:
|
223
|
+
|
224
|
+
- TODO
|
225
|
+
|
301
226
|
## Long Term Implementation and delivery improvements
|
302
227
|
|
303
228
|
- replace rest and oauth classes with ruby standard gems:
|