aspera-cli 4.12.0 → 4.14.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +45 -5
- data/CONTRIBUTING.md +113 -22
- data/README.md +1289 -754
- data/bin/ascli +3 -3
- data/examples/dascli +1 -1
- data/examples/rubyc +24 -0
- data/lib/aspera/aoc.rb +63 -74
- data/lib/aspera/ascmd.rb +5 -3
- data/lib/aspera/cli/basic_auth_plugin.rb +6 -6
- data/lib/aspera/cli/extended_value.rb +24 -37
- data/lib/aspera/cli/formatter.rb +23 -25
- data/lib/aspera/cli/info.rb +2 -4
- data/lib/aspera/cli/main.rb +27 -27
- data/lib/aspera/cli/manager.rb +143 -120
- data/lib/aspera/cli/plugin.rb +88 -43
- data/lib/aspera/cli/plugins/alee.rb +2 -2
- data/lib/aspera/cli/plugins/aoc.rb +235 -104
- data/lib/aspera/cli/plugins/ats.rb +16 -18
- data/lib/aspera/cli/plugins/bss.rb +3 -3
- data/lib/aspera/cli/plugins/config.rb +190 -373
- data/lib/aspera/cli/plugins/console.rb +4 -6
- data/lib/aspera/cli/plugins/cos.rb +12 -13
- data/lib/aspera/cli/plugins/faspex.rb +21 -21
- data/lib/aspera/cli/plugins/faspex5.rb +399 -150
- data/lib/aspera/cli/plugins/node.rb +260 -174
- data/lib/aspera/cli/plugins/orchestrator.rb +15 -18
- data/lib/aspera/cli/plugins/preview.rb +40 -62
- data/lib/aspera/cli/plugins/server.rb +33 -16
- data/lib/aspera/cli/plugins/shares.rb +24 -33
- data/lib/aspera/cli/plugins/sync.rb +6 -6
- data/lib/aspera/cli/transfer_agent.rb +47 -30
- data/lib/aspera/cli/version.rb +2 -1
- data/lib/aspera/colors.rb +9 -7
- data/lib/aspera/command_line_builder.rb +2 -1
- data/lib/aspera/cos_node.rb +1 -1
- data/lib/aspera/data/6 +0 -0
- data/lib/aspera/environment.rb +7 -3
- data/lib/aspera/fasp/agent_connect.rb +6 -1
- data/lib/aspera/fasp/agent_direct.rb +17 -17
- data/lib/aspera/fasp/agent_httpgw.rb +138 -60
- data/lib/aspera/fasp/agent_node.rb +14 -4
- data/lib/aspera/fasp/agent_trsdk.rb +2 -0
- data/lib/aspera/fasp/error_info.rb +2 -0
- data/lib/aspera/fasp/installation.rb +19 -19
- data/lib/aspera/fasp/parameters.rb +29 -20
- data/lib/aspera/fasp/parameters.yaml +5 -2
- data/lib/aspera/fasp/resume_policy.rb +3 -3
- data/lib/aspera/fasp/transfer_spec.rb +8 -5
- data/lib/aspera/fasp/uri.rb +23 -21
- data/lib/aspera/faspex_gw.rb +1 -0
- data/lib/aspera/faspex_postproc.rb +3 -3
- data/lib/aspera/hash_ext.rb +12 -2
- data/lib/aspera/keychain/macos_security.rb +13 -13
- data/lib/aspera/log.rb +1 -0
- data/lib/aspera/node.rb +73 -84
- data/lib/aspera/oauth.rb +4 -3
- data/lib/aspera/persistency_action_once.rb +1 -1
- data/lib/aspera/preview/file_types.rb +8 -6
- data/lib/aspera/preview/generator.rb +23 -11
- data/lib/aspera/preview/options.rb +3 -2
- data/lib/aspera/preview/terminal.rb +80 -0
- data/lib/aspera/preview/utils.rb +11 -11
- data/lib/aspera/proxy_auto_config.js +2 -2
- data/lib/aspera/rest.rb +42 -4
- data/lib/aspera/rest_call_error.rb +3 -1
- data/lib/aspera/secret_hider.rb +10 -5
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/sync.rb +41 -33
- data/lib/aspera/web_server_simple.rb +22 -18
- data.tar.gz.sig +0 -0
- metadata +40 -48
- metadata.gz.sig +0 -0
- data/docs/test_env.conf +0 -179
- data/examples/aoc.rb +0 -30
- data/examples/faspex4.rb +0 -94
- data/examples/node.rb +0 -96
- data/examples/server.rb +0 -93
- data/lib/aspera/data/7 +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 98bd3e1d6ade2f96ff4ddbf4ae825e1926f56990b6552ff22aab6007546fd32e
|
4
|
+
data.tar.gz: edab64f8c8a7108a4614f4f5b7494ddb73611587c54bf6a4c270385303d9d53b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 26c290fd4cd0d6d2a564b8e8dacaab4f584a0548c096999870b43dc755128faf23db58105ccbc16f424aac5384e79a51d07eada5e7a82e66b3d76abec69ad8e7
|
7
|
+
data.tar.gz: 1ee2fe285c913db6851ac97198d73c7159972c35c3c30e2b57342d226d0387e4940df19b7bd737769bfac3b2ab74fec3813d41a1f70c2d39e5202c20c35b8369
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,45 @@
|
|
1
1
|
# Changes (Release notes)
|
2
2
|
|
3
|
+
* 4.14.0
|
4
|
+
|
5
|
+
* new: server: option `passphrase` for simpler command line (#114)
|
6
|
+
* new: percent selector for entities identifier
|
7
|
+
* new: faspex5: shared inbox and workgroup membership management
|
8
|
+
* new: faspex5: invite external user to shared inbox
|
9
|
+
* new: faspex5: package list and receive from workgroups and shared inboxes
|
10
|
+
* new: config: Command `ascp info` shows default transfer spec.
|
11
|
+
* new: httpgw: synchronous and asynchronous upload modes
|
12
|
+
* new: node: command `bandwidth_average` to get average bandwidth of node, per periods
|
13
|
+
* fix: option `ts`: deep add and remove of keys. (#117)
|
14
|
+
* fix: faspex5: user lookup for `packages send` shall be exact match (#120)
|
15
|
+
* fix: direct: if transfer spec contains "paths" and elements with "destination", but first element has only "source", then destinations were ignored. Now "destination" all or none is enforced.
|
16
|
+
* change: using `aoc files` or node gen4 operations (`browse`, `delete`) on a link will follow the link only if path ends with /
|
17
|
+
* change: shares: command `repository` is changed to `files` for consistency with aoc and upcoming faspex5, but is still available as alias
|
18
|
+
* change: aoc: better handling of shared links
|
19
|
+
* change: global: option `value` is deprecated. Use positional parameter for creation data and option `query` for list/delete operations.
|
20
|
+
* change: config: remove deprecated command: `export_to_cli`
|
21
|
+
* change: config: removed all legacy preset command, newer command `preset` shall be used now.
|
22
|
+
* change: config: SDK is now installed in $HOME/.aspera/sdk instead of $HOME/.aspera/ascli/sdk
|
23
|
+
* change(break): aoc/node: Simplification: gen4 operations: show modify permission thumbnail are now directly under node gen 4 command. Command `file` is suppressed. Option `path` is suppressed. The default expected argument is a path. To provide a file id, use selector syntax: %id:_file_id_
|
24
|
+
* change(break): node: option `token_type` is removed, as starting with HSTS 4.3 basic token is only allowed with access keys, so use gen4 operations: `acc do self`
|
25
|
+
|
26
|
+
* 4.13.0
|
27
|
+
|
28
|
+
* new: preview: option `reencode_ffmpeg` allows overriding all re-encoding options
|
29
|
+
* new: faspex5: package delete (#107)
|
30
|
+
* new: faspex5: package recv for inboxes and regular users (#108)
|
31
|
+
* new: faspex5: smtp management
|
32
|
+
* new: faspex5: use public link for authorization of package download, using option `link`
|
33
|
+
* new: faspex5: list content of package, and allow partial download of package
|
34
|
+
* new: faspex5: list packages support multiple pages and items limitations (`max` and `pmax`)
|
35
|
+
* new: aoc: files operations with workspace-less user (#109)
|
36
|
+
* new: node: async with gen3 token (#110)
|
37
|
+
* new: node: display of preview of file in terminal for access keys
|
38
|
+
* change: option `transfer_info` is now cumulative, setting several times merge values
|
39
|
+
* change(deprecation): Removed support of Ruby 2.4 and 2.5 : too old, no security update since a long time. If you need older ruby version use older gem version.
|
40
|
+
* fix: cos: do not use refresh token when not supported
|
41
|
+
* fix: container: SDK installed in other folder than `ascli` (#106)
|
42
|
+
|
3
43
|
* 4.12.0
|
4
44
|
|
5
45
|
* new: docker: build image from official gem version, possibility to deploy beta as well
|
@@ -172,7 +212,7 @@
|
|
172
212
|
* new: agent `http_gw` now supports upload
|
173
213
|
* new: added option `sdk_url` to install SDK from local file for offline install
|
174
214
|
* new: check new gem version periodically
|
175
|
-
* new: the --fields= option, support -
|
215
|
+
* new: the --fields= option, support -_field_name_ to remove a field from default fields
|
176
216
|
* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
|
177
217
|
* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
|
178
218
|
* new: AoC a password can be provided for a public link
|
@@ -192,7 +232,7 @@
|
|
192
232
|
|
193
233
|
* 0.11.8
|
194
234
|
|
195
|
-
* Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb)
|
235
|
+
* Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using `Xvfb`)
|
196
236
|
|
197
237
|
* 0.11.7
|
198
238
|
|
@@ -313,7 +353,7 @@
|
|
313
353
|
* 0.10.4
|
314
354
|
|
315
355
|
* new options for AoC : `secrets`
|
316
|
-
* ACLI-533 temp file list folder to use file lists is set by default, and used by `asession`
|
356
|
+
* `ACLI-533` temp file list folder to use file lists is set by default, and used by `asession`
|
317
357
|
|
318
358
|
* 0.10.3
|
319
359
|
|
@@ -353,13 +393,13 @@
|
|
353
393
|
|
354
394
|
* new command to display basic token of node
|
355
395
|
* new command to display bearer token of node in AoC
|
356
|
-
* the --fields= option, support +
|
396
|
+
* the --fields= option, support +_field_name_ to add a field to default fields
|
357
397
|
* many small changes
|
358
398
|
|
359
399
|
* 0.9.32
|
360
400
|
|
361
401
|
* all Faspex public links are now supported
|
362
|
-
* removed faspex operation recv_publink
|
402
|
+
* removed faspex operation `recv_publink`
|
363
403
|
* replaced with option `link` (consistent with AoC)
|
364
404
|
|
365
405
|
* 0.9.31
|
data/CONTRIBUTING.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
# Contributing
|
2
|
-
|
2
|
+
<!-- cspell:words passin GEMVERS -->
|
3
3
|
## Reporting Issues and Vulnerabilities
|
4
4
|
|
5
5
|
You can report issues at <https://github.com/IBM/aspera-cli/issues>
|
@@ -34,6 +34,20 @@ If you want to contribute, please:
|
|
34
34
|
- Send a pull request on GitHub.
|
35
35
|
- run Rubocop to comply for coding standards
|
36
36
|
|
37
|
+
## Architecture
|
38
|
+
|
39
|
+
A list of classes are provided in <docs/ml.png>
|
40
|
+
|
41
|
+
Architecture:
|
42
|
+
|
43
|
+
![Architecture](docs/architecture.png)
|
44
|
+
|
45
|
+
The entry point is lib/aspera/cli/main.rb
|
46
|
+
|
47
|
+
Plugins are located in lib/aspera/cli/plugins
|
48
|
+
|
49
|
+
Transfer agents: lib/aspera/fasp
|
50
|
+
|
37
51
|
## Running Tests
|
38
52
|
|
39
53
|
First, a testing environment must be created:
|
@@ -73,51 +87,111 @@ It is also possible to build a non-signed version for development purpose: `make
|
|
73
87
|
|
74
88
|
### Gem Signature
|
75
89
|
|
76
|
-
Refer to:
|
90
|
+
Refer to:
|
77
91
|
|
78
|
-
|
92
|
+
- <https://guides.rubygems.org/security/>
|
93
|
+
- <https://ruby-doc.org/current/stdlibs/rubygems/Gem/Security.html>
|
94
|
+
- `gem cert --help`
|
79
95
|
|
80
|
-
|
96
|
+
Then procedure is as follows:
|
81
97
|
|
82
|
-
|
83
|
-
SIGNING_KEY=/path/to/signing_key.pem make
|
84
|
-
```
|
98
|
+
- The maintainer creates the initial certificate and a private key:
|
85
99
|
|
86
|
-
|
100
|
+
```bash
|
101
|
+
cd /path/to/vault
|
102
|
+
gem cert --build maintainer@example.com
|
103
|
+
```
|
87
104
|
|
88
|
-
|
105
|
+
> **Note:** the email must match the field `spec.email` in `aspera-cli.gemspec`
|
89
106
|
|
90
|
-
|
91
|
-
|
92
|
-
gem
|
93
|
-
|
94
|
-
```
|
107
|
+
This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
|
108
|
+
|
109
|
+
- `gem-private_key.pem` : This file shall be kept secret in a vault.
|
110
|
+
- `gem-public_cert.pem` : This file is copied to a public place, here in folder `certs`
|
95
111
|
|
96
|
-
|
112
|
+
> **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
|
97
113
|
|
98
|
-
|
99
|
-
|
100
|
-
|
114
|
+
- The maintainer build the signed gem.
|
115
|
+
|
116
|
+
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
117
|
+
|
118
|
+
To build the signed gem:
|
119
|
+
|
120
|
+
```bash
|
121
|
+
SIGNING_KEY=/path/to/vault/gem-private_key.pem make
|
122
|
+
```
|
123
|
+
|
124
|
+
- The user can activate gem signature verification on installation:
|
125
|
+
|
126
|
+
Add the certificate to gem trusted certificates:
|
127
|
+
|
128
|
+
```bash
|
129
|
+
curl https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem -so aspera-cli-certificate.pem
|
130
|
+
gem cert --add aspera-cli-certificate.pem
|
131
|
+
rm aspera-cli-certificate.pem
|
132
|
+
```
|
133
|
+
|
134
|
+
- The user installs the gem with `HighSecurity` or `MediumSecurity`: this will succeed only of the gem is trusted.
|
135
|
+
|
136
|
+
```bash
|
137
|
+
gem install -P HighSecurity aspera-cli
|
138
|
+
```
|
139
|
+
|
140
|
+
- The maintainer can renew the certificate when it is expired using the same private key:
|
141
|
+
|
142
|
+
```bash
|
143
|
+
SIGNING_KEY=/path/to/vault/gem-private_key.pem make update-cert
|
144
|
+
```
|
145
|
+
|
146
|
+
Alternatively, to generate a new certificate with the same key:
|
147
|
+
|
148
|
+
```bash
|
149
|
+
SIGNING_KEY=/path/to/vault/gem-private_key.pem make new-cert
|
150
|
+
```
|
151
|
+
|
152
|
+
- Show the current certificate contents
|
153
|
+
|
154
|
+
```bash
|
155
|
+
make show-cert
|
156
|
+
```
|
157
|
+
|
158
|
+
> Note: to provide a passphrase add argument: `-passin pass:_value_` to `openssl`
|
159
|
+
|
160
|
+
- Check that the signing key is the same as used to sign the certificate:
|
161
|
+
|
162
|
+
```bash
|
163
|
+
SIGNING_KEY=/path/to/vault/gem-private_key.pem make check-cert-key
|
164
|
+
```
|
101
165
|
|
102
166
|
## Docker image build
|
103
167
|
|
104
|
-
The
|
168
|
+
The Dockerfile template allows customizing the optional copy of gem file , versus install from rubygems,org as well as customizing the retrieval of the SDK.
|
169
|
+
|
170
|
+
### Default image build
|
105
171
|
|
106
|
-
|
172
|
+
Build the image:
|
107
173
|
|
108
174
|
```bash
|
109
175
|
make docker
|
110
176
|
```
|
111
177
|
|
178
|
+
This does the following:
|
179
|
+
|
180
|
+
- Install the official gem version directly from <rubygems.org>.
|
181
|
+
- Build the image for the version number in the current repository
|
182
|
+
- creates tags for both the version and `latest`
|
183
|
+
|
112
184
|
> **Note:** This target creates the `Dockerfile` from an `ERB` (embedded Ruby) template (Makefile target `dockerfile`).
|
113
185
|
A template is used as it allows some level of customization to tell where to take the gem from, as as for the SDK file.
|
114
186
|
|
115
|
-
Then, to push to the image registry:
|
187
|
+
Then, to push to the image registry (both tags: version and `latest`):
|
116
188
|
|
117
189
|
```bash
|
118
190
|
make dpush
|
119
191
|
```
|
120
192
|
|
193
|
+
### Specific version image build
|
194
|
+
|
121
195
|
It is possible to build a specific version by setting envvar `GEMVERS` and `make` with option `-e`:
|
122
196
|
|
123
197
|
```bash
|
@@ -125,14 +199,31 @@ GEMVERS=4.11.0 make -e docker
|
|
125
199
|
GEMVERS=4.11.0 make -e dpush
|
126
200
|
```
|
127
201
|
|
202
|
+
### Development version image build
|
203
|
+
|
128
204
|
To build/push a beta/development container:
|
205
|
+
it does not create the `latest` tag, it uses the gem file generated locally.
|
129
206
|
|
130
207
|
```bash
|
131
208
|
make dockerbeta
|
132
209
|
make dpushversion
|
133
210
|
```
|
134
211
|
|
135
|
-
|
212
|
+
## Single executable build
|
213
|
+
|
214
|
+
It is possible to build `ascli` as a single native executable using `rubyc` (gem `ruby-packer`).
|
215
|
+
|
216
|
+
```bash
|
217
|
+
make single
|
218
|
+
```
|
219
|
+
|
220
|
+
This is based on tool `rubyc` (gem `ruby-packer`): either place the executable in folder `tmp`, or the architecture-specific version will be automatically downloaded.
|
221
|
+
|
222
|
+
> **Note:** Perl Findbin is needed, install on Linux with:
|
223
|
+
|
224
|
+
```bash
|
225
|
+
sudo dnf install -y perl perl-FindBin perl-IPC-Cmd
|
226
|
+
```
|
136
227
|
|
137
228
|
## Long Term Implementation and delivery improvements
|
138
229
|
|