aspera-cli 4.21.1 → 4.21.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +4 -2
  3. data/CHANGELOG.md +22 -10
  4. data/CONTRIBUTING.md +68 -143
  5. data/README.md +175 -145
  6. data/bin/ascli +5 -14
  7. data/bin/asession +1 -3
  8. data/examples/get_proto_file.rb +4 -3
  9. data/examples/proxy.pac +20 -20
  10. data/lib/aspera/agent/base.rb +2 -0
  11. data/lib/aspera/agent/{alpha.rb → desktop.rb} +7 -7
  12. data/lib/aspera/agent/direct.rb +9 -1
  13. data/lib/aspera/agent/transferd.rb +24 -17
  14. data/lib/aspera/api/alee.rb +1 -1
  15. data/lib/aspera/api/cos_node.rb +1 -1
  16. data/lib/aspera/api/node.rb +4 -4
  17. data/lib/aspera/ascp/installation.rb +40 -58
  18. data/lib/aspera/cli/extended_value.rb +9 -3
  19. data/lib/aspera/cli/formatter.rb +8 -1
  20. data/lib/aspera/cli/info.rb +1 -0
  21. data/lib/aspera/cli/main.rb +2 -2
  22. data/lib/aspera/cli/manager.rb +3 -3
  23. data/lib/aspera/cli/plugins/aoc.rb +106 -55
  24. data/lib/aspera/cli/plugins/config.rb +10 -31
  25. data/lib/aspera/cli/plugins/faspex5.rb +8 -5
  26. data/lib/aspera/cli/plugins/node.rb +6 -3
  27. data/lib/aspera/cli/version.rb +1 -1
  28. data/lib/aspera/coverage.rb +5 -3
  29. data/lib/aspera/environment.rb +20 -11
  30. data/lib/aspera/faspex_postproc.rb +3 -5
  31. data/lib/aspera/hash_ext.rb +2 -12
  32. data/lib/aspera/oauth/base.rb +6 -1
  33. data/lib/aspera/preview/generator.rb +12 -9
  34. data/lib/aspera/preview/options.rb +2 -2
  35. data/lib/aspera/preview/terminal.rb +1 -1
  36. data/lib/aspera/preview/utils.rb +4 -4
  37. data/lib/aspera/products/connect.rb +34 -0
  38. data/lib/aspera/products/{alpha.rb → desktop.rb} +2 -2
  39. data/lib/aspera/products/transferd.rb +8 -1
  40. data/lib/aspera/rest.rb +4 -4
  41. data/lib/aspera/secret_hider.rb +7 -0
  42. data/lib/aspera/temp_file_manager.rb +5 -4
  43. data/lib/aspera/uri_reader.rb +18 -1
  44. data.tar.gz.sig +0 -0
  45. metadata +4 -174
  46. metadata.gz.sig +0 -0
  47. data/examples/build_exec +0 -74
  48. data/examples/build_exec_rubyc +0 -40
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 94d8abaf1bf8e938fe9ef79a7f0ecc06870b799d44acb0783a8638e88be8585d
4
- data.tar.gz: b8aeca6b8c34446aac0f4b54fac0ee52db502a6e28149583e30536cc7ba3ab58
3
+ metadata.gz: 3d66238170234ebc23390a21ef7a492ec50c8707200c774ef2f3195fdb13261f
4
+ data.tar.gz: 47623d1ce5961c1d1050989e36afeb1e008dc24c3b81d34d5b484f39d0826107
5
5
  SHA512:
6
- metadata.gz: e90b2de02a88fc84ca2e97df962f054200e14c8c2020dd1a1f42f271b15e0f8ca1e67a4ffaefda7c10f6a69c28611a83d400e140c8ccc9e4edb6f56399db8fe8
7
- data.tar.gz: 6688107fb04c162b4c18bcc45c1450c73170ecae5d430af782ba596e5879fdaf0ecd01020e8ba48c2b39a543d3f0bb3b88f689242ee98714cfa6bd658b92b520
6
+ metadata.gz: dfa7a18bfa476bfda979e88017bf643ef181ecd643f53a6872f7b0af833c6e2dc46700d5afc913126942cda6eb0804f7332b5555e8054ab1d9dba97873000e6e
7
+ data.tar.gz: 2a598cd0d4aab0fb8ec6e2660f590d78fd41a62870f554f96f26069cf0499753dcedb30f807691d8156441340dbbed2910f560b29d1e6e5bbedfd7416f542825
checksums.yaml.gz.sig CHANGED
@@ -1,2 +1,4 @@
1
- f9�~1�8�*�#5���%�.1\���?L�-�Z ��(p%GW�,�eUV�-N���NLh���)���r����1#��[ Ҷ~��[�u0������g��M&�D�D�v�>�@���cU��:18�dl��t���'��ɂк����]�G�d��e2�Ҹ� *�lO'��z���<�0�]����`��RYWM�N�_����B+��y驆RC�Y[����Š�IC�Ju��8�
2
- ��f�B��V��Vw�������ܞ��*�|���afdm
1
+ ���?7n�$��am�i�;Kl���Q���o���v~�^��>�ӴNb���@8�`���T�6� ܾ0
2
+ (��U�$�Y��б�Ҽ�g,���(�b�!#On!;;���C��
3
+ ���M_C��Əy��&��b�g�y��io�f�*E��9� ���#���Py'�Y��B:L�0)�4H� t|�����s�N�
4
+ b���5#K­��|k��/�f�j��uߢKԋ"��#cF������mc��gdvs��5>2,���R^Ԧ�iNUL����SD�U�j`��g����R�8�
data/CHANGELOG.md CHANGED
@@ -1,30 +1,42 @@
1
1
  # Changes (Release notes)
2
2
 
3
+ * 4.21.2
4
+
5
+ * New Features:
6
+ * **container**: Updated Ruby to 3.4.2
7
+ * Issues Fixed:
8
+ * **global**: #185 `@val:` shall stop processing extended values
9
+ * **global**: #186 Removed dependency on openssl 3.3 gem to avoid msys dep on Windows.
10
+ * `echo`: Display of list (Array) was showing only first element of it.
11
+ * `transferd`: support for version 1.1.5+
12
+ * Breaking Changes:
13
+ * `preview`: Updated Image Magick to v7+
14
+ * `aoc`: `admin subscription` split into `admin subscription account` and `admin subscription usage`
15
+ * **agent**: `alpha` renamed to `desktop`
16
+
3
17
  * 4.21.1
4
18
 
5
19
  * New Features:
6
20
  * `config`: New command: `transferd` to list and install specific version of `asperatransferd` and `ascp`
7
- * `config`: New command: `token` to list and flush cached tokens.
8
- * `faspex5`: New command: `admin contact reset_password`
9
21
  * `config`: New command: `tokens` with `list`, `show`, `flush` (replace `flush_tokens`)
22
+ * `faspex5`: New command: `admin contact reset_password`
10
23
  * `aoc`: #178 packages can be browsed, and individual files can be downloaded now.
11
24
  * Issues Fixed:
25
+ * `config`: #175 `ascli config preset set GLOBAL version_check_days 0` causes a bad `config.yaml` to be written
12
26
  * `config`: #180 problem in `ascp install`
13
- * `aoc`: #184 token cache shall be different per aoc org.
14
27
  * `config`: Soft links in transfer SDK archive are correctly extracted
28
+ * `aoc`: #184 token cache shall be different per aoc org.
15
29
  * `aoc`: Fix `packages delete` not working.
16
30
  * `direct` agent: #174 Race condition fix with `ascp`: timeout waiting mgt port connect (select not readable)
17
- * `config`: #175 `ascli config preset set GLOBAL version_check_days 0` causes a bad `config.yaml` to be written
18
31
  * `preview`: #177 fix bug that prevents preview generation to work.
19
32
  * Breaking Changes:
20
- * **global** : Ruby minimum versions is now 3.1 (mainly due to switch to openssl 3). Future minimum is 3.2. Recommended is 3.4.
21
33
  * `transferd`: Use of Aspera Transfer Daemon requires minimum version 1.1.4. agent `trsdk` renamed `transferd`.
22
34
  * `ascp`: Default SDK version is now 1.1.4. Removes support for ascp4.
23
- * **global** : Options `transpose_single` and `multi_table` replaced with single option `multi_single` and values: `no`, `yes`, `single`.
24
- * **global** : Column name for single object is now `field` instead of `key`.
25
- * **global** : Ruby minimum versions is now 2.7 (2.6 previously, that removes macOS default ruby support, but newer ruby version can be installed on macOS)
26
35
  * `node`: Removed deprecated command prefix `exec:`, use `@ruby:` instead.
27
- * **global** : Now uses openssl 3.
36
+ * **global**: Now uses openssl 3.
37
+ * **global**: Ruby minimum versions is now 3.1 (mainly due to switch to openssl 3). Future minimum is 3.2. Recommended is 3.4. (that removes macOS default ruby support. Newer Ruby version shall be installed on macOS with `brew`)
38
+ * **global**: Options `transpose_single` and `multi_table` replaced with single option `multi_single` and values: `no`, `yes`, `single`.
39
+ * **global**: Column name for single object is now `field` instead of `key`.
28
40
 
29
41
  * 4.20.0
30
42
 
@@ -35,7 +47,7 @@
35
47
  * `node`: New option `node_cache` (bool) for gen4 operations.
36
48
  * `node`: Option `root_id` now works always for node gen4, as well as `%id:` for file selection in addition to path.
37
49
  * `node`: `transfer list` now uses the `iteration_token` to retrieve all values. Option `once_only` is now supported.
38
- * **global** : option `http_options` now include retry options.
50
+ * **global**: option `http_options` now include retry options.
39
51
  * Issues Fixed:
40
52
  * `aoc`: Fixed `find` command not working. (undefined variable)
41
53
  * `aoc`: #165 aoc mkdir now follows the last link of containing folder
data/CONTRIBUTING.md CHANGED
@@ -9,34 +9,34 @@ Before you go ahead please search existing issues for your problem.
9
9
  To make sure that we can help you quickly please include and check the following information:
10
10
 
11
11
  - Include the `ascli` version you are running in your report.
12
- - If you are not running the latest version (please check), update.
12
+ - If you are not running the latest version (please check) then update.
13
13
  - Include your `ruby -e "puts RUBY_DESCRIPTION"`.
14
14
 
15
15
  Thanks!
16
16
 
17
17
  ## Making Contributions
18
18
 
19
- To fetch & test the gem for development, do:
19
+ To fetch and test the gem for development, do:
20
20
 
21
21
  ```bash
22
22
  git clone https://github.com/IBM/aspera-cli.git
23
23
  cd aspera-cli
24
- bundle install
25
- make test
26
24
  ```
27
25
 
26
+ Then see [Running Tests](#running-tests).
27
+
28
28
  If you want to contribute, please:
29
29
 
30
30
  - Fork the project.
31
31
  - Make your feature addition or bug fix.
32
32
  - Add tests for it. This is important so I don't break it in a future version unintentionally.
33
- - **Bonus Points** go out to anyone who also updates `CHANGELOG.md` :)
33
+ - Run `rubocop` to comply for coding standards
34
+ - Update `CHANGELOG.md`
34
35
  - Send a pull request on GitHub.
35
- - run `rubocop` to comply for coding standards
36
36
 
37
37
  ## Architecture
38
38
 
39
- A list of classes are provided in <docs/ml.png>
39
+ A list of classes are provided in <docs/uml.png>
40
40
 
41
41
  Architecture:
42
42
 
@@ -46,22 +46,49 @@ The entry point is: `lib/aspera/cli/main.rb`.
46
46
 
47
47
  Plugins are located in: `lib/aspera/cli/plugins`.
48
48
 
49
- Transfer agents, in: `lib/aspera/fasp`.
49
+ Transfer agents, in: `lib/aspera/agent`.
50
50
 
51
51
  ## Ruby version
52
52
 
53
- Install ruby with [RVM](https://rvm.io).
53
+ Install Ruby using your prefered method.
54
54
 
55
- Cleanup installed gems in `rvm` to start fresh:
55
+ To cleanup installed gems to start fresh:
56
56
 
57
57
  ```bash
58
- make cleanup_gems
58
+ make clean_gems
59
59
  ```
60
60
 
61
61
  ## Tool chain
62
62
 
63
63
  TODO: document installation of tool chain.
64
64
 
65
+ Build system uses GNU Make.
66
+
67
+ ### Environment
68
+
69
+ A few macros/envvars control some aspects:
70
+
71
+ | macro | description |
72
+ |-----------------------------|--------------------------------------|
73
+ | `ASPERA_CLI_TEST_CONF_FILE` | Path to configuration file with secrets for tests |
74
+ | `ASPERA_CLI_TEST_MACOS` | Set to `true` if local HSTS running on macOS |
75
+ | `ASPERA_CLI_TEST_PRIVATE` | Path to private folder |
76
+ | `ENABLE_COVERAGE` | Tests with coverage analysis if set. |
77
+ | `SIGNING_KEY` | Path to signing key to build Gem. |
78
+ | `GEM_VERSION` | Gem version to build container |
79
+
80
+ Those macros can be set either in an env var, or on the `make` command line.
81
+
82
+ > **Note:** vars `ASPERA_CLI_TEST_` are typically set in user's shell profile.
83
+
84
+ To use the CLI directly from the development environment, add this to your shell profile:
85
+
86
+ ```bash
87
+ dev_ascli=$HOME/github/aspera-cli
88
+ export PATH=$dev_ascli/bin:$PATH
89
+ export RUBYLIB=$dev_ascli/lib:$RUBYLIB
90
+ ```
91
+
65
92
  ### Documentation
66
93
 
67
94
  Documentation is generated with `pandoc` and `LaTeX`.
@@ -83,7 +110,8 @@ tlmgr install fvextra selnolig lualatex-math
83
110
 
84
111
  ## Running Tests
85
112
 
86
- First, a testing configuration file must be created, from project top folder:
113
+ First, a testing configuration file must be created.
114
+ From project top folder, execute:
87
115
 
88
116
  ```bash
89
117
  mkdir ~/some_secure_folder
@@ -104,7 +132,7 @@ This project uses a `Makefile` for tests:
104
132
  make test
105
133
  ```
106
134
 
107
- When new commands are added to the CLI, new tests shall be added to the test suite.
135
+ When new commands are added to the CLI, new tests shall be added to the test suite in `tests/Makefile`.
108
136
 
109
137
  ### Special tests
110
138
 
@@ -121,110 +149,40 @@ To run every test: `make full`
121
149
 
122
150
  For preparation of a release, do the following:
123
151
 
124
- 1. Select a ruby version to test with: `rvm use 3.2.2`
152
+ 1. Select a ruby version to test with.
125
153
  2. Remove all gems: `make clean_gems`
126
- 3. Starts tests: `make test`
127
- 4. Install optional gems: `make install_optional_gems`
128
- 5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
129
-
130
- This ensures that optional gems do not prevent `ascli` to run.
154
+ 3. `cd tests && make full`
131
155
 
132
156
  To test additional Ruby version, repeat the procedure with other Ruby versions.
133
157
 
134
158
  ## Coverage
135
159
 
136
160
  A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
137
- Enable coverage monitoring using env var `ENABLE_COVERAGE`.
161
+ Enable coverage monitoring using macro/envvar `ENABLE_COVERAGE`.
138
162
 
139
163
  ```bash
140
- ENABLE_COVERAGE=1 make test
164
+ cd tests
165
+ make ENABLE_COVERAGE=1
141
166
  ```
142
167
 
143
168
  Once tests are completed, or during test, consult the page: [coverage/index.html](coverage/index.html)
144
169
 
145
170
  ## Build
146
171
 
147
- By default the gem is built signed: `make`.
148
- The appropriate signing key is required, and its path must be set to env var `SIGNING_KEY`.
149
- It is also possible to build a non-signed version for development purpose: `make unsigned_gem`.
150
-
151
- ### Gem Signature
152
-
153
- Refer to:
154
-
155
- - <https://guides.rubygems.org/security/>
156
- - <https://ruby-doc.org/current/stdlibs/rubygems/Gem/Security.html>
157
- - `gem cert --help`
158
-
159
- Then procedure is as follows:
160
-
161
- - The maintainer creates the initial certificate and a private key:
162
-
163
- ```bash
164
- cd /path/to/vault
165
- gem cert --build maintainer@example.com
166
- ```
167
-
168
- > **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
169
-
170
- This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
171
-
172
- - `gem-private_key.pem` : This file shall be kept secret in a vault.
173
- - `gem-public_cert.pem` : This file is copied to a public place, here in folder `certs`
174
-
175
- > **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
176
-
177
- - The maintainer builds the signed gem.
178
-
179
- The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
180
-
181
- To build the signed gem:
182
-
183
- ```bash
184
- SIGNING_KEY=/path/to/vault/gem-private_key.pem make
185
- ```
186
-
187
- - The user can activate gem signature verification on installation:
188
-
189
- Add the certificate to gem trusted certificates:
190
-
191
- ```bash
192
- curl https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem -so aspera-cli-certificate.pem
193
- gem cert --add aspera-cli-certificate.pem
194
- rm aspera-cli-certificate.pem
195
- ```
196
-
197
- - The user installs the gem with `HighSecurity` or `MediumSecurity`: this will succeed only of the gem is trusted.
172
+ By default, the gem is built signed: `make`.
173
+ A private key is required to generate a signed Gem.
174
+ Its path must be set using macro/envvar `SIGNING_KEY`, see below.
175
+ The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
198
176
 
199
- ```bash
200
- gem install -P HighSecurity aspera-cli
201
- ```
202
-
203
- - The maintainer can renew the certificate when it is expired using the same private key:
204
-
205
- ```bash
206
- SIGNING_KEY=/path/to/vault/gem-private_key.pem make update-cert
207
- ```
208
-
209
- Alternatively, to generate a new certificate with the same key:
210
-
211
- ```bash
212
- SIGNING_KEY=/path/to/vault/gem-private_key.pem make new-cert
213
- ```
214
-
215
- - Show the current certificate contents
216
-
217
- ```bash
218
- make show-cert
219
- ```
177
+ ```bash
178
+ make SIGNING_KEY=/path/to/vault/gem-private_key.pem
179
+ ```
220
180
 
221
- > Note: to provide a passphrase add argument: `-passin pass:_value_` to `openssl`
181
+ It is also possible to build an unsigned version for development purpose: `make unsigned_gem`.
222
182
 
223
- - Check that the signing key is the same as used to sign the certificate:
183
+ ### Gem Signature
224
184
 
225
- ```bash
226
- SIGNING_KEY=/path/to/vault/gem-private_key.pem make check-cert-key
227
- ```
185
+ Refer to <certs/README.md>
228
186
 
229
187
  ### gRPC stubs for transfer SDK
230
188
 
@@ -236,54 +194,14 @@ make grpc
236
194
 
237
195
  It downloads the latest proto file and then compiles it.
238
196
 
239
- ## Docker image build
240
-
241
- The `Dockerfile.tmpl.erb` template allows customization of the optional copying of the `aspera-cli.gem` file, as opposed to installing from rubygems.org. It also supports customizing the retrieval of the SDK.
197
+ ## Container image build
242
198
 
243
- ### Default image build
199
+ See [Container build](./container/README.md).
244
200
 
245
- Build the image:
201
+ For operations, move to the folder:
246
202
 
247
203
  ```bash
248
- make docker
249
- ```
250
-
251
- This does the following:
252
-
253
- - Install the official gem version directly from <rubygems.org>.
254
- - Build the image for the version number in the current repository
255
- - creates tags for both the version and `latest`
256
-
257
- > **Note:** This target creates the `Dockerfile` from an `ERB` (embedded Ruby) template (Makefile target `dockerfile`).
258
- A template is used as it allows some level of customization to tell where to take the gem from, as as for the SDK file.
259
-
260
- Then, to push to the image registry (both tags: version and `latest`):
261
-
262
- ```bash
263
- make docker_push
264
- ```
265
-
266
- ### Specific version image build
267
-
268
- To build a specific version: override `make` macro `GEM_VERSION`:
269
-
270
- ```bash
271
- make docker GEM_VERSION=4.11.0
272
- make docker_push GEM_VERSION=4.11.0
273
- ```
274
-
275
- > **Note:** This does not use the locally generated gem file.
276
- Only the local docker file is used.
277
- The gem is installed from rubygems.org. This also sets the `latest` tag.
278
-
279
- ### Development version image build
280
-
281
- To build/push a beta/development container:
282
- it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
283
-
284
- ```bash
285
- make docker_beta
286
- make docker_push_beta
204
+ cd container
287
205
  ```
288
206
 
289
207
  ## Single executable build
@@ -295,9 +213,16 @@ Initially, `rubyc` (gem [`ruby-packer`](https://github.com/pmq20/ruby-packer) an
295
213
  A modern version of this is now used: [`tebako`](https://github.com/tamatebako/tebako) for which a container is provided.
296
214
 
297
215
  ```bash
298
- make single GEM_VERSION=4.11.0
216
+ cd binary
217
+ make GEM_VERSION=4.11.0
299
218
  ```
300
219
 
220
+ ## Development check list for new release
221
+
222
+ When preparing for a new release do the following:
223
+
224
+ - TODO
225
+
301
226
  ## Long Term Implementation and delivery improvements
302
227
 
303
228
  - replace rest and oauth classes with ruby standard gems: