aspera-cli 4.15.0 → 4.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/BUGS.md +29 -3
  4. data/CHANGELOG.md +292 -228
  5. data/CONTRIBUTING.md +69 -18
  6. data/README.md +1102 -952
  7. data/bin/ascli +13 -31
  8. data/bin/asession +3 -1
  9. data/examples/dascli +2 -2
  10. data/lib/aspera/aoc.rb +28 -33
  11. data/lib/aspera/ascmd.rb +3 -6
  12. data/lib/aspera/assert.rb +45 -0
  13. data/lib/aspera/cli/extended_value.rb +5 -5
  14. data/lib/aspera/cli/formatter.rb +26 -13
  15. data/lib/aspera/cli/hints.rb +4 -3
  16. data/lib/aspera/cli/main.rb +16 -3
  17. data/lib/aspera/cli/manager.rb +45 -36
  18. data/lib/aspera/cli/plugin.rb +20 -13
  19. data/lib/aspera/cli/plugins/aoc.rb +103 -73
  20. data/lib/aspera/cli/plugins/ats.rb +4 -3
  21. data/lib/aspera/cli/plugins/config.rb +114 -119
  22. data/lib/aspera/cli/plugins/cos.rb +2 -2
  23. data/lib/aspera/cli/plugins/faspex.rb +23 -19
  24. data/lib/aspera/cli/plugins/faspex5.rb +75 -43
  25. data/lib/aspera/cli/plugins/node.rb +28 -15
  26. data/lib/aspera/cli/plugins/orchestrator.rb +4 -2
  27. data/lib/aspera/cli/plugins/preview.rb +9 -7
  28. data/lib/aspera/cli/plugins/server.rb +6 -3
  29. data/lib/aspera/cli/plugins/shares.rb +30 -26
  30. data/lib/aspera/cli/sync_actions.rb +9 -9
  31. data/lib/aspera/cli/transfer_agent.rb +21 -14
  32. data/lib/aspera/cli/transfer_progress.rb +2 -3
  33. data/lib/aspera/cli/version.rb +1 -1
  34. data/lib/aspera/command_line_builder.rb +13 -11
  35. data/lib/aspera/cos_node.rb +3 -2
  36. data/lib/aspera/coverage.rb +22 -0
  37. data/lib/aspera/data_repository.rb +33 -2
  38. data/lib/aspera/environment.rb +4 -2
  39. data/lib/aspera/fasp/{agent_aspera.rb → agent_alpha.rb} +29 -39
  40. data/lib/aspera/fasp/agent_base.rb +17 -7
  41. data/lib/aspera/fasp/agent_direct.rb +88 -84
  42. data/lib/aspera/fasp/agent_httpgw.rb +4 -3
  43. data/lib/aspera/fasp/agent_node.rb +3 -2
  44. data/lib/aspera/fasp/agent_trsdk.rb +79 -37
  45. data/lib/aspera/fasp/installation.rb +51 -12
  46. data/lib/aspera/fasp/management.rb +11 -6
  47. data/lib/aspera/fasp/parameters.rb +53 -47
  48. data/lib/aspera/fasp/resume_policy.rb +7 -5
  49. data/lib/aspera/fasp/sync.rb +273 -0
  50. data/lib/aspera/fasp/transfer_spec.rb +10 -8
  51. data/lib/aspera/fasp/uri.rb +2 -2
  52. data/lib/aspera/faspex_gw.rb +11 -8
  53. data/lib/aspera/faspex_postproc.rb +6 -5
  54. data/lib/aspera/id_generator.rb +3 -1
  55. data/lib/aspera/json_rpc.rb +10 -8
  56. data/lib/aspera/keychain/encrypted_hash.rb +46 -11
  57. data/lib/aspera/keychain/macos_security.rb +15 -13
  58. data/lib/aspera/log.rb +4 -3
  59. data/lib/aspera/nagios.rb +7 -2
  60. data/lib/aspera/node.rb +17 -16
  61. data/lib/aspera/node_simulator.rb +214 -0
  62. data/lib/aspera/oauth.rb +22 -19
  63. data/lib/aspera/persistency_action_once.rb +13 -14
  64. data/lib/aspera/persistency_folder.rb +3 -2
  65. data/lib/aspera/preview/file_types.rb +53 -267
  66. data/lib/aspera/preview/generator.rb +7 -5
  67. data/lib/aspera/preview/terminal.rb +14 -5
  68. data/lib/aspera/preview/utils.rb +8 -7
  69. data/lib/aspera/proxy_auto_config.rb +6 -3
  70. data/lib/aspera/rest.rb +29 -13
  71. data/lib/aspera/rest_error_analyzer.rb +1 -0
  72. data/lib/aspera/rest_errors_aspera.rb +2 -0
  73. data/lib/aspera/secret_hider.rb +5 -2
  74. data/lib/aspera/ssh.rb +10 -8
  75. data/lib/aspera/temp_file_manager.rb +1 -1
  76. data/lib/aspera/web_server_simple.rb +2 -1
  77. data.tar.gz.sig +0 -0
  78. metadata +96 -45
  79. metadata.gz.sig +0 -0
  80. data/lib/aspera/sync.rb +0 -219
data/CONTRIBUTING.md CHANGED
@@ -42,32 +42,81 @@ Architecture:
42
42
 
43
43
  ![Architecture](docs/architecture.png)
44
44
 
45
- The entry point is lib/aspera/cli/main.rb
45
+ The entry point is: `lib/aspera/cli/main.rb`.
46
46
 
47
- Plugins are located in lib/aspera/cli/plugins
47
+ Plugins are located in: `lib/aspera/cli/plugins`.
48
48
 
49
- Transfer agents: lib/aspera/fasp
49
+ Transfer agents, in: `lib/aspera/fasp`.
50
+
51
+ ## Ruby version
52
+
53
+ Install ruby with [RVM](https://rvm.io).
54
+
55
+ Cleanup installed gems in `rvm` to start fresh:
56
+
57
+ ```bash
58
+ make cleanup_gems
59
+ ```
60
+
61
+ ## Tool chain
62
+
63
+ TODO: document installation of tool chain.
64
+
65
+ ### Documentation
66
+
67
+ Documentation is generated with `pandoc` and `LaTeX`.
68
+ IBM font `Plex` is used, see [IBM Plex](https://www.ibm.com/plex/).
50
69
 
51
70
  ## Running Tests
52
71
 
53
- First, a testing environment must be created:
72
+ First, a testing configuration file must be created, from project top folder:
54
73
 
55
74
  ```bash
56
- mkdir local
57
- cp docs/test_env.conf local/.
75
+ mkdir ~/some_secure_folder
76
+ cp docs/test_env.conf ~/some_secure_folder/.
58
77
  ```
59
78
 
60
- Fill `local/test_env.conf` with system URLs and credentials for tests.
79
+ Fill `~/some_secure_folder/test_env.conf` with system URLs and credentials for tests.
61
80
 
62
- This project uses a Makefile for tests:
81
+ Then tell where this file is located:
63
82
 
64
83
  ```bash
65
- cd tests
66
- make
84
+ export ASPERA_CLI_TEST_CONF_FILE=~/some_secure_folder/test_env.conf
85
+ ```
86
+
87
+ This project uses a `Makefile` for tests:
88
+
89
+ ```bash
90
+ make test
67
91
  ```
68
92
 
69
93
  When new commands are added to the CLI, new tests shall be added to the test suite.
70
94
 
95
+ ### Special tests
96
+
97
+ Some gems are optional: `rmagick` and `grpc`, as they require compilation of native code which may cause problems.
98
+ By default, tests that use those gems are skipped.
99
+ To run them: `make optional`.
100
+ Those tests also require the optional gems to be installed: `make install_optional_gems`.
101
+
102
+ Some other tests require interactive input. To run them: `make interactive`
103
+
104
+ To run every test: `make full`
105
+
106
+ ### Pre-release tests
107
+
108
+ For preparation of a release, do the following:
109
+
110
+ 1. Select a ruby version to test with: `rvm use 3.2.2`
111
+ 2. Remove all gems: `make clean_gems`
112
+ 3. Starts tests: `make test`
113
+ 4. Install optional gems: `make install_optional_gems`
114
+ 5. Test remaining cases: `cd tests` and then `make optional` and `make interactive`
115
+
116
+ This ensures that optional gems do not prevent `ascli` to run.
117
+
118
+ To test additional Ruby version, repeat the procedure with other Ruby versions.
119
+
71
120
  ## Coverage
72
121
 
73
122
  A coverage report can be generated in folder `coverage` using gem `SimpleCov`.
@@ -102,7 +151,7 @@ Then procedure is as follows:
102
151
  gem cert --build maintainer@example.com
103
152
  ```
104
153
 
105
- > **Note:** the email must match the field `spec.email` in `aspera-cli.gemspec`
154
+ > **Note:** The email must match the field `spec.email` in `aspera-cli.gemspec`
106
155
 
107
156
  This creates two files in folder `/path/to/vault` (e.g. $HOME/.ssh):
108
157
 
@@ -111,7 +160,7 @@ Then procedure is as follows:
111
160
 
112
161
  > **Note:** Alternatively, use an existing key or generate one, and then `make new-cert`
113
162
 
114
- - The maintainer build the signed gem.
163
+ - The maintainer builds the signed gem.
115
164
 
116
165
  The gem is signed with the public certificate found in `certs` and the private key (kept secret by maintainer).
117
166
 
@@ -187,26 +236,28 @@ A template is used as it allows some level of customization to tell where to tak
187
236
  Then, to push to the image registry (both tags: version and `latest`):
188
237
 
189
238
  ```bash
190
- make dpush
239
+ make docker_push
191
240
  ```
192
241
 
193
242
  ### Specific version image build
194
243
 
195
- It is possible to build a specific version by setting envvar `GEM_VERSION` and `make` with option `-e`:
244
+ To build a specific version: set envvar `GEM_VERSION` and `make` with option `-e`:
196
245
 
197
246
  ```bash
198
247
  GEM_VERSION=4.11.0 make -e docker
199
- GEM_VERSION=4.11.0 make -e dpush
248
+ GEM_VERSION=4.11.0 make -e docker_push
200
249
  ```
201
250
 
251
+ > **Note:** This does not use the gem locally, only the docker file is used locally, the gem is installed from rubygems.org.
252
+
202
253
  ### Development version image build
203
254
 
204
255
  To build/push a beta/development container:
205
- it does not create the `latest` tag, it uses the gem file generated locally.
256
+ it does not create the `latest` tag, it uses the gem file generated locally with a special version number.
206
257
 
207
258
  ```bash
208
- make dockerbeta
209
- make dpushversion
259
+ make docker_beta
260
+ make docker_push_beta
210
261
  ```
211
262
 
212
263
  ## Single executable build