aspera-cli 4.15.0 → 4.17.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/BUGS.md +29 -3
- data/CHANGELOG.md +375 -280
- data/CONTRIBUTING.md +71 -18
- data/README.md +1978 -1656
- data/bin/ascli +13 -31
- data/bin/asession +32 -22
- data/examples/dascli +2 -2
- data/lib/aspera/agent/alpha.rb +117 -0
- data/lib/aspera/agent/base.rb +61 -0
- data/lib/aspera/{fasp/agent_connect.rb → agent/connect.rb} +13 -11
- data/lib/aspera/{fasp/agent_direct.rb → agent/direct.rb} +116 -116
- data/lib/aspera/{fasp/agent_httpgw.rb → agent/httpgw.rb} +21 -19
- data/lib/aspera/{fasp/agent_node.rb → agent/node.rb} +21 -33
- data/lib/aspera/agent/trsdk.rb +188 -0
- data/lib/aspera/api/aoc.rb +586 -0
- data/lib/aspera/api/ats.rb +46 -0
- data/lib/aspera/api/cos_node.rb +95 -0
- data/lib/aspera/api/node.rb +344 -0
- data/lib/aspera/ascmd.rb +47 -14
- data/lib/aspera/{fasp → ascp}/installation.rb +54 -15
- data/lib/aspera/{fasp → ascp}/management.rb +14 -14
- data/lib/aspera/{fasp → ascp}/products.rb +1 -1
- data/lib/aspera/assert.rb +45 -0
- data/lib/aspera/cli/basic_auth_plugin.rb +11 -10
- data/lib/aspera/cli/extended_value.rb +5 -5
- data/lib/aspera/cli/formatter.rb +27 -14
- data/lib/aspera/cli/hints.rb +7 -6
- data/lib/aspera/cli/main.rb +49 -29
- data/lib/aspera/cli/manager.rb +46 -36
- data/lib/aspera/cli/plugin.rb +34 -20
- data/lib/aspera/cli/plugin_factory.rb +61 -0
- data/lib/aspera/cli/plugins/alee.rb +7 -7
- data/lib/aspera/cli/plugins/aoc.rb +168 -132
- data/lib/aspera/cli/plugins/ats.rb +33 -33
- data/lib/aspera/cli/plugins/bss.rb +3 -4
- data/lib/aspera/cli/plugins/config.rb +250 -272
- data/lib/aspera/cli/plugins/console.rb +8 -6
- data/lib/aspera/cli/plugins/cos.rb +20 -19
- data/lib/aspera/cli/plugins/faspex.rb +71 -60
- data/lib/aspera/cli/plugins/faspex5.rb +212 -133
- data/lib/aspera/cli/plugins/node.rb +83 -75
- data/lib/aspera/cli/plugins/orchestrator.rb +36 -44
- data/lib/aspera/cli/plugins/preview.rb +33 -31
- data/lib/aspera/cli/plugins/server.rb +33 -32
- data/lib/aspera/cli/plugins/shares.rb +39 -33
- data/lib/aspera/cli/sync_actions.rb +9 -9
- data/lib/aspera/cli/transfer_agent.rb +45 -25
- data/lib/aspera/cli/transfer_progress.rb +2 -3
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/colors.rb +5 -0
- data/lib/aspera/command_line_builder.rb +16 -14
- data/lib/aspera/coverage.rb +21 -0
- data/lib/aspera/data_repository.rb +33 -2
- data/lib/aspera/environment.rb +5 -4
- data/lib/aspera/faspex_gw.rb +13 -11
- data/lib/aspera/faspex_postproc.rb +6 -5
- data/lib/aspera/id_generator.rb +4 -2
- data/lib/aspera/json_rpc.rb +10 -8
- data/lib/aspera/keychain/encrypted_hash.rb +46 -11
- data/lib/aspera/keychain/macos_security.rb +29 -22
- data/lib/aspera/log.rb +5 -4
- data/lib/aspera/nagios.rb +7 -2
- data/lib/aspera/node_simulator.rb +213 -0
- data/lib/aspera/oauth/base.rb +143 -0
- data/lib/aspera/oauth/factory.rb +124 -0
- data/lib/aspera/oauth/generic.rb +34 -0
- data/lib/aspera/oauth/jwt.rb +51 -0
- data/lib/aspera/oauth/url_json.rb +31 -0
- data/lib/aspera/oauth/web.rb +50 -0
- data/lib/aspera/oauth.rb +5 -328
- data/lib/aspera/open_application.rb +7 -7
- data/lib/aspera/persistency_action_once.rb +13 -14
- data/lib/aspera/persistency_folder.rb +3 -2
- data/lib/aspera/preview/file_types.rb +53 -267
- data/lib/aspera/preview/generator.rb +7 -5
- data/lib/aspera/preview/terminal.rb +17 -7
- data/lib/aspera/preview/utils.rb +8 -7
- data/lib/aspera/proxy_auto_config.rb +6 -3
- data/lib/aspera/rest.rb +187 -140
- data/lib/aspera/rest_error_analyzer.rb +1 -0
- data/lib/aspera/rest_errors_aspera.rb +5 -3
- data/lib/aspera/resumer.rb +77 -0
- data/lib/aspera/secret_hider.rb +5 -2
- data/lib/aspera/ssh.rb +15 -8
- data/lib/aspera/temp_file_manager.rb +1 -1
- data/lib/aspera/{fasp → transfer}/error.rb +3 -3
- data/lib/aspera/{fasp → transfer}/error_info.rb +1 -1
- data/lib/aspera/{fasp → transfer}/faux_file.rb +1 -1
- data/lib/aspera/{fasp → transfer}/parameters.rb +95 -120
- data/lib/aspera/{fasp/transfer_spec.rb → transfer/spec.rb} +23 -19
- data/lib/aspera/{fasp/parameters.yaml → transfer/spec.yaml} +4 -99
- data/lib/aspera/transfer/sync.rb +273 -0
- data/lib/aspera/{fasp → transfer}/uri.rb +10 -9
- data/lib/aspera/web_server_simple.rb +12 -3
- data.tar.gz.sig +0 -0
- metadata +92 -68
- metadata.gz.sig +0 -0
- data/lib/aspera/aoc.rb +0 -606
- data/lib/aspera/ats_api.rb +0 -47
- data/lib/aspera/cos_node.rb +0 -93
- data/lib/aspera/fasp/agent_aspera.rb +0 -126
- data/lib/aspera/fasp/agent_base.rb +0 -48
- data/lib/aspera/fasp/agent_trsdk.rb +0 -146
- data/lib/aspera/fasp/resume_policy.rb +0 -77
- data/lib/aspera/node.rb +0 -338
- data/lib/aspera/sync.rb +0 -219
data/CONTRIBUTING.md
CHANGED
@@ -42,32 +42,81 @@ Architecture:
|
|
42
42
|
|
43
43
|
![Architecture](docs/architecture.png)
|
44
44
|
|
45
|
-
The entry point is lib/aspera/cli/main.rb
|
45
|
+
The entry point is: `lib/aspera/cli/main.rb`.
|
46
46
|
|
47
|
-
Plugins are located in lib/aspera/cli/plugins
|
47
|
+
Plugins are located in: `lib/aspera/cli/plugins`.
|
48
48
|
|
49
|
-
Transfer agents: lib/aspera/fasp
|
49
|
+
Transfer agents, in: `lib/aspera/fasp`.
|
50
|
+
|
51
|
+
## Ruby version
|
52
|
+
|
53
|
+
Install ruby with [RVM](https://rvm.io).
|
54
|
+
|
55
|
+
Cleanup installed gems in `rvm` to start fresh:
|
56
|
+
|
57
|
+
```bash
|
58
|
+
make cleanup_gems
|
59
|
+
```
|
60
|
+
|
61
|
+
## Tool chain
|
62
|
+
|
63
|
+
TODO: document installation of tool chain.
|
64
|
+
|
65
|
+
### Documentation
|
66
|
+
|
67
|
+
Documentation is generated with `pandoc` and `LaTeX`.
|
68
|
+
IBM font `Plex` is used, see [IBM Plex](https://www.ibm.com/plex/).
|
50
69
|
|
51
70
|
## Running Tests
|
52
71
|
|
53
|
-
First, a testing
|
72
|
+
First, a testing configuration file must be created, from project top folder:
|
54
73
|
|
55
74
|
```bash
|
56
|
-
mkdir
|
57
|
-
cp docs/test_env.conf
|
75
|
+
mkdir ~/some_secure_folder
|
76
|
+
cp docs/test_env.conf ~/some_secure_folder/.
|
58
77
|
```
|
59
78
|
|
60
|
-
Fill
|
79
|
+
Fill `~/some_secure_folder/test_env.conf` with system URLs and credentials for tests.
|
61
80
|
|
62
|
-
|
81
|
+
Then tell where this file is located:
|
63
82
|
|
64
83
|
```bash
|
65
|
-
|
66
|
-
|
84
|
+
export ASPERA_CLI_TEST_CONF_FILE=~/some_secure_folder/test_env.conf
|
85
|
+
```
|
86
|
+
|
87
|
+
This project uses a `Makefile` for tests:
|
88
|
+
|
89
|
+
```bash
|
90
|
+
make test
|
67
91
|
```
|
68
92
|
|
69
93
|
When new commands are added to the CLI, new tests shall be added to the test suite.
|
70
94
|
|
95
|
+
### Special tests
|
96
|
+
|
97
|
+
Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
|
98
|
+
By default, tests that use those gems are skipped.
|
99
|
+
To run them: `make optional`.
|
100
|
+
Those tests also require the optional gems to be installed: `make install_optional_gems`.
|
101
|
+
|
102
|
+
Some other tests require interactive input. To run them: `make interactive`
|
103
|
+
|
104
|
+
To run every test: `make full`
|
105
|
+
|
106
|
+
### Pre-release tests
|
107
|
+
|
108
|
+
For preparation of a release, do the following:
|
109
|
+
|
110
|
+
1. Select a ruby version to test with: `rvm use 3.2.2`
|
111
|
+
2. Remove all gems: `make clean_gems`
|
112
|
+
3. Starts tests: `make test`
|
113
|
+
4. Install optional gems: `make install_optional_gems`
|
114
|
+
5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
|
115
|
+
|
116
|
+
This ensures that optional gems do not prevent `ascli` to run.
|
117
|
+
|
118
|
+
To test additional Ruby version, repeat the procedure with other Ruby versions.
|
119
|
+
|
71
120
|
## Coverage
|
72
121
|
|
73
122
|
A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
|
@@ -102,7 +151,7 @@ Then procedure is as follows:
|
|
102
151
|
gem cert --build maintainer@example.com
|
103
152
|
```
|
104
153
|
|
105
|
-
> **Note:**
|
154
|
+
> **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
|
106
155
|
|
107
156
|
This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
|
108
157
|
|
@@ -111,7 +160,7 @@ Then procedure is as follows:
|
|
111
160
|
|
112
161
|
> **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
|
113
162
|
|
114
|
-
- The maintainer
|
163
|
+
- The maintainer builds the signed gem.
|
115
164
|
|
116
165
|
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
117
166
|
|
@@ -187,26 +236,30 @@ A template is used as it allows some level of customization to tell where to tak
|
|
187
236
|
Then, to push to the image registry (both tags: version and `latest`):
|
188
237
|
|
189
238
|
```bash
|
190
|
-
make
|
239
|
+
make docker_push
|
191
240
|
```
|
192
241
|
|
193
242
|
### Specific version image build
|
194
243
|
|
195
|
-
|
244
|
+
To build a specific version: set envvar `GEM_VERSION` and `make` with option `-e`:
|
196
245
|
|
197
246
|
```bash
|
198
247
|
GEM_VERSION=4.11.0 make -e docker
|
199
|
-
GEM_VERSION=4.11.0 make -e
|
248
|
+
GEM_VERSION=4.11.0 make -e docker_push
|
200
249
|
```
|
201
250
|
|
251
|
+
> **Note:** This does not use the locally generated gem file.
|
252
|
+
Only the local docker file is used.
|
253
|
+
The gem is installed from rubygems.org. This also sets the `latest` tag.
|
254
|
+
|
202
255
|
### Development version image build
|
203
256
|
|
204
257
|
To build/push a beta/development container:
|
205
|
-
it does not create the `latest` tag, it uses the gem file generated locally.
|
258
|
+
it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
|
206
259
|
|
207
260
|
```bash
|
208
|
-
make
|
209
|
-
make
|
261
|
+
make docker_beta
|
262
|
+
make docker_push_beta
|
210
263
|
```
|
211
264
|
|
212
265
|
## Single executable build
|