aspera-cli 4.15.0 → 4.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/BUGS.md +29 -3
- data/CHANGELOG.md +292 -228
- data/CONTRIBUTING.md +69 -18
- data/README.md +1102 -952
- data/bin/ascli +13 -31
- data/bin/asession +3 -1
- data/examples/dascli +2 -2
- data/lib/aspera/aoc.rb +28 -33
- data/lib/aspera/ascmd.rb +3 -6
- data/lib/aspera/assert.rb +45 -0
- data/lib/aspera/cli/extended_value.rb +5 -5
- data/lib/aspera/cli/formatter.rb +26 -13
- data/lib/aspera/cli/hints.rb +4 -3
- data/lib/aspera/cli/main.rb +16 -3
- data/lib/aspera/cli/manager.rb +45 -36
- data/lib/aspera/cli/plugin.rb +20 -13
- data/lib/aspera/cli/plugins/aoc.rb +103 -73
- data/lib/aspera/cli/plugins/ats.rb +4 -3
- data/lib/aspera/cli/plugins/config.rb +114 -119
- data/lib/aspera/cli/plugins/cos.rb +2 -2
- data/lib/aspera/cli/plugins/faspex.rb +23 -19
- data/lib/aspera/cli/plugins/faspex5.rb +75 -43
- data/lib/aspera/cli/plugins/node.rb +28 -15
- data/lib/aspera/cli/plugins/orchestrator.rb +4 -2
- data/lib/aspera/cli/plugins/preview.rb +9 -7
- data/lib/aspera/cli/plugins/server.rb +6 -3
- data/lib/aspera/cli/plugins/shares.rb +30 -26
- data/lib/aspera/cli/sync_actions.rb +9 -9
- data/lib/aspera/cli/transfer_agent.rb +21 -14
- data/lib/aspera/cli/transfer_progress.rb +2 -3
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/command_line_builder.rb +13 -11
- data/lib/aspera/cos_node.rb +3 -2
- data/lib/aspera/coverage.rb +22 -0
- data/lib/aspera/data_repository.rb +33 -2
- data/lib/aspera/environment.rb +4 -2
- data/lib/aspera/fasp/{agent_aspera.rb → agent_alpha.rb} +29 -39
- data/lib/aspera/fasp/agent_base.rb +17 -7
- data/lib/aspera/fasp/agent_direct.rb +88 -84
- data/lib/aspera/fasp/agent_httpgw.rb +4 -3
- data/lib/aspera/fasp/agent_node.rb +3 -2
- data/lib/aspera/fasp/agent_trsdk.rb +79 -37
- data/lib/aspera/fasp/installation.rb +51 -12
- data/lib/aspera/fasp/management.rb +11 -6
- data/lib/aspera/fasp/parameters.rb +53 -47
- data/lib/aspera/fasp/resume_policy.rb +7 -5
- data/lib/aspera/fasp/sync.rb +273 -0
- data/lib/aspera/fasp/transfer_spec.rb +10 -8
- data/lib/aspera/fasp/uri.rb +2 -2
- data/lib/aspera/faspex_gw.rb +11 -8
- data/lib/aspera/faspex_postproc.rb +6 -5
- data/lib/aspera/id_generator.rb +3 -1
- data/lib/aspera/json_rpc.rb +10 -8
- data/lib/aspera/keychain/encrypted_hash.rb +46 -11
- data/lib/aspera/keychain/macos_security.rb +15 -13
- data/lib/aspera/log.rb +4 -3
- data/lib/aspera/nagios.rb +7 -2
- data/lib/aspera/node.rb +17 -16
- data/lib/aspera/node_simulator.rb +214 -0
- data/lib/aspera/oauth.rb +22 -19
- data/lib/aspera/persistency_action_once.rb +13 -14
- data/lib/aspera/persistency_folder.rb +3 -2
- data/lib/aspera/preview/file_types.rb +53 -267
- data/lib/aspera/preview/generator.rb +7 -5
- data/lib/aspera/preview/terminal.rb +14 -5
- data/lib/aspera/preview/utils.rb +8 -7
- data/lib/aspera/proxy_auto_config.rb +6 -3
- data/lib/aspera/rest.rb +29 -13
- data/lib/aspera/rest_error_analyzer.rb +1 -0
- data/lib/aspera/rest_errors_aspera.rb +2 -0
- data/lib/aspera/secret_hider.rb +5 -2
- data/lib/aspera/ssh.rb +10 -8
- data/lib/aspera/temp_file_manager.rb +1 -1
- data/lib/aspera/web_server_simple.rb +2 -1
- data.tar.gz.sig +0 -0
- metadata +96 -45
- metadata.gz.sig +0 -0
- data/lib/aspera/sync.rb +0 -219
data/CONTRIBUTING.md
CHANGED
@@ -42,32 +42,81 @@ Architecture:
|
|
42
42
|
|
43
43
|

|
44
44
|
|
45
|
-
The entry point is lib/aspera/cli/main.rb
|
45
|
+
The entry point is: `lib/aspera/cli/main.rb`.
|
46
46
|
|
47
|
-
Plugins are located in lib/aspera/cli/plugins
|
47
|
+
Plugins are located in: `lib/aspera/cli/plugins`.
|
48
48
|
|
49
|
-
Transfer agents: lib/aspera/fasp
|
49
|
+
Transfer agents, in: `lib/aspera/fasp`.
|
50
|
+
|
51
|
+
## Ruby version
|
52
|
+
|
53
|
+
Install ruby with [RVM](https://rvm.io).
|
54
|
+
|
55
|
+
Cleanup installed gems in `rvm` to start fresh:
|
56
|
+
|
57
|
+
```bash
|
58
|
+
make cleanup_gems
|
59
|
+
```
|
60
|
+
|
61
|
+
## Tool chain
|
62
|
+
|
63
|
+
TODO: document installation of tool chain.
|
64
|
+
|
65
|
+
### Documentation
|
66
|
+
|
67
|
+
Documentation is generated with `pandoc` and `LaTeX`.
|
68
|
+
IBM font `Plex` is used, see [IBM Plex](https://www.ibm.com/plex/).
|
50
69
|
|
51
70
|
## Running Tests
|
52
71
|
|
53
|
-
First, a testing
|
72
|
+
First, a testing configuration file must be created, from project top folder:
|
54
73
|
|
55
74
|
```bash
|
56
|
-
mkdir
|
57
|
-
cp docs/test_env.conf
|
75
|
+
mkdir ~/some_secure_folder
|
76
|
+
cp docs/test_env.conf ~/some_secure_folder/.
|
58
77
|
```
|
59
78
|
|
60
|
-
Fill
|
79
|
+
Fill `~/some_secure_folder/test_env.conf` with system URLs and credentials for tests.
|
61
80
|
|
62
|
-
|
81
|
+
Then tell where this file is located:
|
63
82
|
|
64
83
|
```bash
|
65
|
-
|
66
|
-
|
84
|
+
export ASPERA_CLI_TEST_CONF_FILE=~/some_secure_folder/test_env.conf
|
85
|
+
```
|
86
|
+
|
87
|
+
This project uses a `Makefile` for tests:
|
88
|
+
|
89
|
+
```bash
|
90
|
+
make test
|
67
91
|
```
|
68
92
|
|
69
93
|
When new commands are added to the CLI, new tests shall be added to the test suite.
|
70
94
|
|
95
|
+
### Special tests
|
96
|
+
|
97
|
+
Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
|
98
|
+
By default, tests that use those gems are skipped.
|
99
|
+
To run them: `make optional`.
|
100
|
+
Those tests also require the optional gems to be installed: `make install_optional_gems`.
|
101
|
+
|
102
|
+
Some other tests require interactive input. To run them: `make interactive`
|
103
|
+
|
104
|
+
To run every test: `make full`
|
105
|
+
|
106
|
+
### Pre-release tests
|
107
|
+
|
108
|
+
For preparation of a release, do the following:
|
109
|
+
|
110
|
+
1. Select a ruby version to test with: `rvm use 3.2.2`
|
111
|
+
2. Remove all gems: `make clean_gems`
|
112
|
+
3. Starts tests: `make test`
|
113
|
+
4. Install optional gems: `make install_optional_gems`
|
114
|
+
5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
|
115
|
+
|
116
|
+
This ensures that optional gems do not prevent `ascli` to run.
|
117
|
+
|
118
|
+
To test additional Ruby version, repeat the procedure with other Ruby versions.
|
119
|
+
|
71
120
|
## Coverage
|
72
121
|
|
73
122
|
A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
|
@@ -102,7 +151,7 @@ Then procedure is as follows:
|
|
102
151
|
gem cert --build maintainer@example.com
|
103
152
|
```
|
104
153
|
|
105
|
-
> **Note:**
|
154
|
+
> **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
|
106
155
|
|
107
156
|
This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
|
108
157
|
|
@@ -111,7 +160,7 @@ Then procedure is as follows:
|
|
111
160
|
|
112
161
|
> **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
|
113
162
|
|
114
|
-
- The maintainer
|
163
|
+
- The maintainer builds the signed gem.
|
115
164
|
|
116
165
|
The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
|
117
166
|
|
@@ -187,26 +236,28 @@ A template is used as it allows some level of customization to tell where to tak
|
|
187
236
|
Then, to push to the image registry (both tags: version and `latest`):
|
188
237
|
|
189
238
|
```bash
|
190
|
-
make
|
239
|
+
make docker_push
|
191
240
|
```
|
192
241
|
|
193
242
|
### Specific version image build
|
194
243
|
|
195
|
-
|
244
|
+
To build a specific version: set envvar `GEM_VERSION` and `make` with option `-e`:
|
196
245
|
|
197
246
|
```bash
|
198
247
|
GEM_VERSION=4.11.0 make -e docker
|
199
|
-
GEM_VERSION=4.11.0 make -e
|
248
|
+
GEM_VERSION=4.11.0 make -e docker_push
|
200
249
|
```
|
201
250
|
|
251
|
+
> **Note:** This does not use the gem locally, only the docker file is used locally, the gem is installed from rubygems.org.
|
252
|
+
|
202
253
|
### Development version image build
|
203
254
|
|
204
255
|
To build/push a beta/development container:
|
205
|
-
it does not create the `latest` tag, it uses the gem file generated locally.
|
256
|
+
it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
|
206
257
|
|
207
258
|
```bash
|
208
|
-
make
|
209
|
-
make
|
259
|
+
make docker_beta
|
260
|
+
make docker_push_beta
|
210
261
|
```
|
211
262
|
|
212
263
|
## Single executable build
|