aspera-cli 4.15.0 → 4.16.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/BUGS.md +29 -3
  4. data/CHANGELOG.md +292 -228
  5. data/CONTRIBUTING.md +69 -18
  6. data/README.md +1102 -952
  7. data/bin/ascli +13 -31
  8. data/bin/asession +3 -1
  9. data/examples/dascli +2 -2
  10. data/lib/aspera/aoc.rb +28 -33
  11. data/lib/aspera/ascmd.rb +3 -6
  12. data/lib/aspera/assert.rb +45 -0
  13. data/lib/aspera/cli/extended_value.rb +5 -5
  14. data/lib/aspera/cli/formatter.rb +26 -13
  15. data/lib/aspera/cli/hints.rb +4 -3
  16. data/lib/aspera/cli/main.rb +16 -3
  17. data/lib/aspera/cli/manager.rb +45 -36
  18. data/lib/aspera/cli/plugin.rb +20 -13
  19. data/lib/aspera/cli/plugins/aoc.rb +103 -73
  20. data/lib/aspera/cli/plugins/ats.rb +4 -3
  21. data/lib/aspera/cli/plugins/config.rb +114 -119
  22. data/lib/aspera/cli/plugins/cos.rb +2 -2
  23. data/lib/aspera/cli/plugins/faspex.rb +23 -19
  24. data/lib/aspera/cli/plugins/faspex5.rb +75 -43
  25. data/lib/aspera/cli/plugins/node.rb +28 -15
  26. data/lib/aspera/cli/plugins/orchestrator.rb +4 -2
  27. data/lib/aspera/cli/plugins/preview.rb +9 -7
  28. data/lib/aspera/cli/plugins/server.rb +6 -3
  29. data/lib/aspera/cli/plugins/shares.rb +30 -26
  30. data/lib/aspera/cli/sync_actions.rb +9 -9
  31. data/lib/aspera/cli/transfer_agent.rb +21 -14
  32. data/lib/aspera/cli/transfer_progress.rb +2 -3
  33. data/lib/aspera/cli/version.rb +1 -1
  34. data/lib/aspera/command_line_builder.rb +13 -11
  35. data/lib/aspera/cos_node.rb +3 -2
  36. data/lib/aspera/coverage.rb +22 -0
  37. data/lib/aspera/data_repository.rb +33 -2
  38. data/lib/aspera/environment.rb +4 -2
  39. data/lib/aspera/fasp/{agent_aspera.rb → agent_alpha.rb} +29 -39
  40. data/lib/aspera/fasp/agent_base.rb +17 -7
  41. data/lib/aspera/fasp/agent_direct.rb +88 -84
  42. data/lib/aspera/fasp/agent_httpgw.rb +4 -3
  43. data/lib/aspera/fasp/agent_node.rb +3 -2
  44. data/lib/aspera/fasp/agent_trsdk.rb +79 -37
  45. data/lib/aspera/fasp/installation.rb +51 -12
  46. data/lib/aspera/fasp/management.rb +11 -6
  47. data/lib/aspera/fasp/parameters.rb +53 -47
  48. data/lib/aspera/fasp/resume_policy.rb +7 -5
  49. data/lib/aspera/fasp/sync.rb +273 -0
  50. data/lib/aspera/fasp/transfer_spec.rb +10 -8
  51. data/lib/aspera/fasp/uri.rb +2 -2
  52. data/lib/aspera/faspex_gw.rb +11 -8
  53. data/lib/aspera/faspex_postproc.rb +6 -5
  54. data/lib/aspera/id_generator.rb +3 -1
  55. data/lib/aspera/json_rpc.rb +10 -8
  56. data/lib/aspera/keychain/encrypted_hash.rb +46 -11
  57. data/lib/aspera/keychain/macos_security.rb +15 -13
  58. data/lib/aspera/log.rb +4 -3
  59. data/lib/aspera/nagios.rb +7 -2
  60. data/lib/aspera/node.rb +17 -16
  61. data/lib/aspera/node_simulator.rb +214 -0
  62. data/lib/aspera/oauth.rb +22 -19
  63. data/lib/aspera/persistency_action_once.rb +13 -14
  64. data/lib/aspera/persistency_folder.rb +3 -2
  65. data/lib/aspera/preview/file_types.rb +53 -267
  66. data/lib/aspera/preview/generator.rb +7 -5
  67. data/lib/aspera/preview/terminal.rb +14 -5
  68. data/lib/aspera/preview/utils.rb +8 -7
  69. data/lib/aspera/proxy_auto_config.rb +6 -3
  70. data/lib/aspera/rest.rb +29 -13
  71. data/lib/aspera/rest_error_analyzer.rb +1 -0
  72. data/lib/aspera/rest_errors_aspera.rb +2 -0
  73. data/lib/aspera/secret_hider.rb +5 -2
  74. data/lib/aspera/ssh.rb +10 -8
  75. data/lib/aspera/temp_file_manager.rb +1 -1
  76. data/lib/aspera/web_server_simple.rb +2 -1
  77. data.tar.gz.sig +0 -0
  78. metadata +96 -45
  79. metadata.gz.sig +0 -0
  80. data/lib/aspera/sync.rb +0 -219
data/CONTRIBUTING.md CHANGED
@@ -42,32 +42,81 @@ Architecture:
42
42
 
43
43
  ![Architecture](docs/architecture.png)
44
44
 
45
- The entry point is lib/aspera/cli/main.rb
45
+ The entry point is: `lib/aspera/cli/main.rb`.
46
46
 
47
- Plugins are located in lib/aspera/cli/plugins
47
+ Plugins are located in: `lib/aspera/cli/plugins`.
48
48
 
49
- Transfer agents: lib/aspera/fasp
49
+ Transfer agents, in: `lib/aspera/fasp`.
50
+
51
+ ## Ruby version
52
+
53
+ Install ruby with [RVM](https://rvm.io).
54
+
55
+ Cleanup installed gems in `rvm` to start fresh:
56
+
57
+ ```bash
58
+ make cleanup_gems
59
+ ```
60
+
61
+ ## Tool chain
62
+
63
+ TODO: document installation of tool chain.
64
+
65
+ ### Documentation
66
+
67
+ Documentation is generated with `pandoc` and `LaTeX`.
68
+ IBM font `Plex` is used, see [IBM Plex](https://www.ibm.com/plex/).
50
69
 
51
70
  ## Running Tests
52
71
 
53
- First, a testing environment must be created:
72
+ First, a testing configuration file must be created, from project top folder:
54
73
 
55
74
  ```bash
56
- mkdir local
57
- cp docs/test_env.conf local/.
75
+ mkdir ~/some_secure_folder
76
+ cp docs/test_env.conf ~/some_secure_folder/.
58
77
  ```
59
78
 
60
- Fill `local/test_env.conf` with system URLs and credentials for tests.
79
+ Fill `~/some_secure_folder/test_env.conf` with system URLs and credentials for tests.
61
80
 
62
- This project uses a Makefile for tests:
81
+ Then tell where this file is located:
63
82
 
64
83
  ```bash
65
- cd tests
66
- make
84
+ export ASPERA_CLI_TEST_CONF_FILE=~/some_secure_folder/test_env.conf
85
+ ```
86
+
87
+ This project uses a `Makefile` for tests:
88
+
89
+ ```bash
90
+ make test
67
91
  ```
68
92
 
69
93
  When new commands are added to the CLI, new tests shall be added to the test suite.
70
94
 
95
+ ### Special tests
96
+
97
+ Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
98
+ By default, tests that use those gems are skipped.
99
+ To run them: `make optional`.
100
+ Those tests also require the optional gems to be installed: `make install_optional_gems`.
101
+
102
+ Some other tests require interactive input. To run them: `make interactive`
103
+
104
+ To run every test: `make full`
105
+
106
+ ### Pre-release tests
107
+
108
+ For preparation of a release, do the following:
109
+
110
+ 1. Select a ruby version to test with: `rvm use 3.2.2`
111
+ 2. Remove all gems: `make clean_gems`
112
+ 3. Starts tests: `make test`
113
+ 4. Install optional gems: `make install_optional_gems`
114
+ 5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
115
+
116
+ This ensures that optional gems do not prevent `ascli` to run.
117
+
118
+ To test additional Ruby version, repeat the procedure with other Ruby versions.
119
+
71
120
  ## Coverage
72
121
 
73
122
  A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
@@ -102,7 +151,7 @@ Then procedure is as follows:
102
151
  gem cert --build maintainer@example.com
103
152
  ```
104
153
 
105
- > **Note:** the email must match the field `spec.email` in `aspera-cli.gemspec`
154
+ > **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
106
155
 
107
156
  This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
108
157
 
@@ -111,7 +160,7 @@ Then procedure is as follows:
111
160
 
112
161
  > **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
113
162
 
114
- - The maintainer build the signed gem.
163
+ - The maintainer builds the signed gem.
115
164
 
116
165
  The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
117
166
 
@@ -187,26 +236,28 @@ A template is used as it allows some level of customization to tell where to tak
187
236
  Then, to push to the image registry (both tags: version and `latest`):
188
237
 
189
238
  ```bash
190
- make dpush
239
+ make docker_push
191
240
  ```
192
241
 
193
242
  ### Specific version image build
194
243
 
195
- It is possible to build a specific version by setting envvar `GEM_VERSION` and `make` with option `-e`:
244
+ To build a specific version: set envvar `GEM_VERSION` and `make` with option `-e`:
196
245
 
197
246
  ```bash
198
247
  GEM_VERSION=4.11.0 make -e docker
199
- GEM_VERSION=4.11.0 make -e dpush
248
+ GEM_VERSION=4.11.0 make -e docker_push
200
249
  ```
201
250
 
251
+ > **Note:** This does not use the gem locally, only the docker file is used locally, the gem is installed from rubygems.org.
252
+
202
253
  ### Development version image build
203
254
 
204
255
  To build/push a beta/development container:
205
- it does not create the `latest` tag, it uses the gem file generated locally.
256
+ it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
206
257
 
207
258
  ```bash
208
- make dockerbeta
209
- make dpushversion
259
+ make docker_beta
260
+ make docker_push_beta
210
261
  ```
211
262
 
212
263
  ## Single executable build