aspera-cli 4.25.3 → 4.25.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d01aecb4590218d1028c4d4653c255394294cca477508cd802cf53b72acee927
4
- data.tar.gz: 0ecb307d12fcc0daaa6d465928533d5fc95ac3f66218cc5b6c552d7e044aa822
3
+ metadata.gz: 34cb77d2badea1e3650f2aed72fb3a91c419e6457a08acd62808c9a6d9ab0e74
4
+ data.tar.gz: 31e1e1a28d99afebd6b6a3f7b9e848bf18256d47fdc7108368a6e964eef03b04
5
5
  SHA512:
6
- metadata.gz: 01ba11f7c24c0f9f869770c9035edd0e2d7d06b77fb71ca00a16cf59829e6abaf758fe5c9a0e131019e6a11e09a58b1c1e9fb4cc9a7818bcda63fdf3ef8e8969
7
- data.tar.gz: a70ffa0f67294841bb18a53720c656c2eea4d904cf0c14db259203cf75aa05090a2c64b040807d1d5bfd7eb7288969feb1756adc97415bc3b142cf3b55d2d48d
6
+ metadata.gz: e15bfb65919b8b8a6112120eca273dac962ff6d6b29d890df1e9293f4321e557bdd68697c394cb6ef738a3d5bcc3a75a2f7f177494b66542c2e970c4da99e9d0
7
+ data.tar.gz: 1d3570c590e7bfbda3ce5fc49396637471f48ff6fccce7235ae31adcc23c1e1ccc4a978d6bdbe9a4c1c06f06e02ed5336043267891c0b56a995b4d0199ab0d92
checksums.yaml.gz.sig CHANGED
Binary file
data/CHANGELOG.md CHANGED
@@ -2,6 +2,24 @@
2
2
 
3
3
  <!-- markdownlint-configure-file { "no-duplicate-heading": { "siblings_only": true } } -->
4
4
 
5
+ ## 4.25.4
6
+
7
+ Released: 2026-03-04
8
+
9
+ ### New Features
10
+
11
+ * **general**: If `@:` is used, then marker `END` optionally marks the end of collected arguments.
12
+ * `format`: `display` defaults to `info` only if `format` is set to `table`, else defaults to `data`.
13
+ * `node`: Parameter `accept_v4` of option `node_api` (boolean, defaults to `true`) allows using gen4 browsing with `Accept-Version: 4.0` for best performance when there are thousands of files.
14
+
15
+ ### Issues Fixed
16
+
17
+ * `faspex5`: Listing content, or receiving a package requires API parameter: `recipient_user_id` or `recipient_workgroup_id`, else error `Not authorized` is returned.
18
+
19
+ ### Breaking Changes
20
+
21
+ * `node`: Options `node_cache` and `default_ports` are replaced with option: `node_api` (`Hash`) with boolean parameters (keys): `cache` and `standard_ports` with default value `true`.
22
+
5
23
  ## 4.25.3
6
24
 
7
25
  Released: 2026-02-18
data/CONTRIBUTING.md CHANGED
@@ -2,22 +2,23 @@
2
2
 
3
3
  ## Reporting Issues and Vulnerabilities
4
4
 
5
- If you encounter a problem or a security vulnerability, please report it on [GitHub Issues](https://github.com/IBM/aspera-cli/issues).
5
+ If you encounter a bug or a security vulnerability, please report it via [GitHub Issues](https://github.com/IBM/aspera-cli/issues).
6
6
 
7
- Before submitting a new issue:
7
+ Before submitting a new report:
8
8
 
9
- - Search existing issues to see if your problem has already been reported or resolved.
9
+ - **Search existing issues** to determine if the problem has already been documented or resolved.
10
10
 
11
- To help us assist you efficiently, include the following in your report:
11
+ To help us diagnose and resolve the issue efficiently, please include the following in your report:
12
12
 
13
- - The version of `ascli` you are using:
13
+ - The `ascli` version you are using:
14
14
 
15
15
  ```bash
16
- ascli version
16
+ ascli -v
17
17
  ```
18
18
 
19
- - Confirmation that you are using the latest version (update it if needed).
20
- - Your Ruby version information:
19
+ - **Update confirmation**: Verify that you are running the latest version.
20
+
21
+ - **Your Ruby environment details**:
21
22
 
22
23
  ```bash
23
24
  ruby -v
@@ -29,7 +30,7 @@ We welcome contributions to improve the `aspera-cli` project!
29
30
 
30
31
  ### Getting Started
31
32
 
32
- Clone the repository and navigate to the project's root directory:
33
+ Clone the repository to initialize the development environment:
33
34
 
34
35
  ```bash
35
36
  git clone https://github.com/IBM/aspera-cli.git
@@ -38,55 +39,55 @@ bundle install
38
39
  bundle exec rake -T
39
40
  ```
40
41
 
41
- For testing instructions, refer to [Running Tests](#running-tests).
42
+ For detailed testing instructions, please refer to [Running Tests](#running-tests).
42
43
 
43
44
  ### How to Contribute
44
45
 
45
- To submit a contribution:
46
+ To submit a contribution, follow these steps:
46
47
 
47
48
  1. **Fork** the repository on GitHub.
48
49
 
49
- 1. **Create a feature branch** for your changes.
50
+ 1. **Create a feature branch** specifically for your changes.
50
51
 
51
52
  1. **Implement** your feature or bug fix.
52
53
 
53
54
  1. **Write tests** to ensure your changes are robust and prevent regressions.
54
55
 
55
- 1. Run `rubocop` to ensure your code follows the Ruby style guide.
56
+ 1. **Run** `rubocop` to ensure your code adheres to the Ruby style guide.
56
57
 
57
- 1. Update `CHANGELOG.md` with a summary of your changes.
58
+ 1. **Update** `CHANGELOG.md` with a concise summary of your changes.
58
59
 
59
- 1. Submit a **pull request** with a clear description of your contribution.
60
+ 1. **Submit a pull request** with a detailed description of your work.
60
61
 
61
62
  > [!TIP]
62
- > Make sure your pull request is focused and includes only relevant changes.
63
+ > Keep pull requests focused; include only changes relevant to the specific feature or fix.
63
64
 
64
65
  ## Architecture
65
66
 
66
- The overall architecture of `aspera-cli` is modular and extensible.
67
+ The `aspera-cli` architecture is designed to be modular and extensible.
67
68
 
68
69
  ![Architecture](docs/architecture.png)
69
70
 
70
71
  ### Structure Highlights
71
72
 
72
- - Entry Point:
73
+ - **Entry Point**:
73
74
 
74
- `lib/aspera/cli/main.rb`, CLI startup logic.
75
+ `lib/aspera/cli/main.rb` contains the core CLI startup logic.
75
76
 
76
- - Plugins:
77
+ - **Plugins**:
77
78
 
78
- Located in `lib/aspera/cli/plugins`; plugins extend CLI functionality and encapsulate specific features.
79
+ Located in `lib/aspera/cli/plugins`, these extend CLI functionality and encapsulate specific features.
79
80
 
80
- - Transfer Agents:
81
+ - **Transfer Agents**:
81
82
 
82
- Found in `lib/aspera/agent`, these handle data transfer operations.
83
+ Located in `lib/aspera/agent`, these components manage data transfer operations.
83
84
 
84
- Class diagrams are provided in <docs/uml.png>
85
+ Detailed class diagrams are available in <docs/uml.png>
85
86
 
86
87
  ## Ruby Environment
87
88
 
88
- `aspera-cli` is written in Ruby.
89
- You can install Ruby using any method you prefer (e.g., `rbenv`, `rvm`, system package manager).
89
+ `aspera-cli` is built with Ruby.
90
+ You can manage your Ruby installation using your preferred tool (e.g., `rbenv`, `rvm`, or a system package manager).
90
91
 
91
92
  To start with a clean state and remove all installed gems:
92
93
 
@@ -95,35 +96,34 @@ bundle exec rake tools:clean_gems
95
96
  ```
96
97
 
97
98
  > [!TIP]
98
- > This is especially useful before testing across different Ruby versions or preparing for a release.
99
+ > This is particularly useful when testing across different Ruby versions or preparing for a new release.
99
100
 
100
101
  ## Toolchain
101
102
 
102
- The build system uses Ruby's `rake`.
103
+ The build system is powered by Ruby's `rake`.
103
104
 
104
- ### Environment
105
+ ### Environment Configuration
105
106
 
106
- A few macros and environment variables control certain aspects of the build:
107
+ The following environment variables and macros control specific build behaviors:
107
108
 
108
- | Environment variable | Description |
109
- |-----------------------------|-----------------------------------------------------|
110
- | `ASPERA_CLI_TEST_CONF_URL` | URL for configuration file with secrets for tests. |
111
- | `ASPERA_CLI_DOC_CHECK_LINKS`| Check links still exist during doc generation. |
112
- | `LOG_LEVEL` | Change log level in `rake` tasks. |
113
- | `LOG_SECRETS` | Change log secrets in `rake` tasks. |
114
- | `ENABLE_COVERAGE` | Enable test coverage analysis when set. |
115
- | `SIGNING_KEY` | Path to the signing key used to build the gem file. |
116
- | `SIGNING_KEY_PEM` | PEM of signing key. |
109
+ | Environment variable | Contents | Description |
110
+ |-----------------------------|------------| -------------------------------------------------------------|
111
+ | `ASPERA_CLI_TEST_CONF_URL` | URL | URL for the configuration file containing secrets for tests. |
112
+ | `ASPERA_CLI_DOC_CHECK_LINKS`| yes/no | Validates that links exist during documentation generation. |
113
+ | `LOG_SECRETS` | yes/no | Toggles the logging of secrets in `rake` tasks. |
114
+ | `LOG_LEVEL` | debug, ... | Sets the logging verbosity for `rake` tasks. |
115
+ | `ENABLE_COVERAGE` | set/unset | Enables test coverage analysis when defined. |
116
+ | `SIGNING_KEY` | File path | Path to the signing key used for building the gem file. |
117
+ | `SIGNING_KEY_PEM` | PEM Value | The PEM content of the signing key. |
117
118
 
118
- These can be set either as environment variables or directly on the `rake` command line.
119
+ These values can be set as standard environment variables or passed directly to the `rake` command.
119
120
 
120
- Setting `SIGNING_KEY_PEM` creates file `$HOME/.gem/signing_key.pem` and sets `SIGNING_KEY` to that path.
121
+ Setting `SIGNING_KEY_PEM` automatically generates a file at `$HOME/.gem/signing_key.pem` and sets the `SIGNING_KEY` variable accordingly.
121
122
 
122
123
  > [!NOTE]
123
- > Environment variables `ASPERA_CLI_*` are typically set in the user’s shell profile for development.
124
- > Others are intended for use on the command line.
124
+ > `ASPERA_CLI_*` variables are typically defined in your shell profile for development, while others are intended for ad-hoc command-line use.
125
125
 
126
- To use the CLI directly from the development environment, add this to your shell profile (adapt the real path):
126
+ To run the CLI directly from your source directory, add the following to your shell profile (adjust the path as necessary):
127
127
 
128
128
  ```bash
129
129
  dev_ascli=$HOME/github/aspera-cli
@@ -135,26 +135,27 @@ export RUBYLIB=$dev_ascli/lib:$RUBYLIB
135
135
 
136
136
  Documentation is generated with `pandoc` and `LaTeX`.
137
137
 
138
- The IBM `Plex` font is used; for installation instructions, see [IBM Plex](https://www.ibm.com/plex/).
138
+ The project utilizes the **IBM Plex font**.
139
+ Installation instructions can be found at [IBM Plex](https://www.ibm.com/plex/).
139
140
 
140
- On macOS, to install `lualatex` and all packages:
141
+ On macOS, install `lualatex` and required packages via Homebrew:
141
142
 
142
143
  ```bash
143
144
  brew install texlive
144
145
  ```
145
146
 
146
- If `lualatex` is installed using another method, ensure that the following packages are installed:
147
+ If using an alternative installation method, ensure the following packages are present:
147
148
 
148
149
  ```bash
149
150
  tlmgr update --self
150
151
  tlmgr install fvextra selnolig lualatex-math
151
152
  ```
152
153
 
153
- To check URLs during documentation generation, set the environment variable: `ASPERA_CLI_DOC_CHECK_LINKS=1`.
154
+ - To validate URLs during generation: `ASPERA_CLI_DOC_CHECK_LINKS=1`.
154
155
 
155
- To debug documentation generation, set the environment variable: `ASPERA_CLI_DOC_DEBUG=debug`.
156
+ - To debug the generation process: `ASPERA_CLI_DOC_DEBUG=debug`.
156
157
 
157
- To generate documentation:
158
+ - To build the documentation:
158
159
 
159
160
  ```bash
160
161
  rake doc:build
@@ -162,18 +163,18 @@ rake doc:build
162
163
 
163
164
  ## Test Environment
164
165
 
165
- Refer to <tests/README.md>.
166
+ Detailed testing information can be found in <tests/README.md>.
166
167
 
167
168
  ## Build
168
169
 
169
- The unsigned gem is built with:
170
+ To build an unsigned gem:
170
171
 
171
172
  ```bash
172
173
  bundle install
173
174
  bundle exec rake unsigned
174
175
  ```
175
176
 
176
- If you don't want to install optional gems:
177
+ To exclude optional gems from the installation:
177
178
 
178
179
  ```bash
179
180
  bundle config set without optional
@@ -181,35 +182,34 @@ bundle config set without optional
181
182
 
182
183
  ### Signed gem
183
184
 
184
- A private key is required to generate a signed gem.
185
- Its path must be set using environment variable `SIGNING_KEY`.
186
- The gem is signed with the public certificate found in `certs` and the private key specified by `SIGNING_KEY` (kept secret by the maintainer).
185
+ Generating a signed gem requires a **private key**, specified via the `SIGNING_KEY` environment variable.
186
+ The gem is signed using the public certificate in `certs` and the **private key**.
187
187
 
188
188
  ```bash
189
189
  bundle exec rake SIGNING_KEY=/path/to/vault/gem-private_key.pem
190
190
  ```
191
191
 
192
- Refer to <certs/README.md>.
192
+ For more details, see <certs/README.md>.
193
193
 
194
- ### gRPC stubs for transfer SDK
194
+ ### gRPC stubs for Transfer SDK
195
195
 
196
- Update with:
196
+ To update the stubs:
197
197
 
198
198
  ```bash
199
199
  bundle exec rake tools:grpc
200
200
  ```
201
201
 
202
- It downloads the latest `proto` file and then compiles it into ruby sources included in the repo.
202
+ This task downloads the latest `.proto` files and compiles them into the Ruby source files included in the repository.
203
203
 
204
204
  ## Container image build
205
205
 
206
- See [Container build](./container/README.md).
206
+ Refer to the [Container build guide](./container/README.md).
207
207
 
208
208
  ## Single executable build
209
209
 
210
- See [Executable build](build/binary/README.md).
210
+ Refer to the [Executable build guide](build/binary/README.md).
211
211
 
212
- To list operations:
212
+ To list related `rake` tasks:
213
213
 
214
214
  ```bash
215
215
  bundle exec rake -T ^binary:
@@ -219,25 +219,26 @@ bundle exec rake -T ^binary:
219
219
 
220
220
  ### Branching Strategy
221
221
 
222
- This project uses a single `main` branch for development.
223
- During the development cycle, the version in `lib/aspera/cli/version.rb` uses a `.pre` suffix (e.g., `x.y.z.pre`) to indicate a pre-release state.
222
+ This project maintains a single `main` branch.
223
+ During development, the version in `lib/aspera/cli/version.rb` includes a `.pre` suffix (e.g., `x.y.z.pre`).
224
+
225
+ Contributions are handled as follows:
224
226
 
225
- Feature development and bug fixes can be done either:
227
+ - **Direct commits** to `main`: Permitted for minor changes.
226
228
 
227
- - Directly on `main` for small changes
228
- - Via feature branches with pull requests for larger changes
229
+ - **Feature branches**: Required for significant changes via pull requests.
229
230
 
230
- ### Checklist Before a New Release
231
+ ### Pre-Release Checklist
231
232
 
232
- When preparing for a new release, do the following:
233
+ Before a new release, ensure the following:
233
234
 
234
- - Run the test suite:
235
+ - **Pass all tests**:
235
236
 
236
237
  ```bash
237
238
  bundle exec rake test:run
238
239
  ```
239
240
 
240
- - Verify that the container builds successfully (using the local gem file):
241
+ - **Verify container builds** (using the local gem):
241
242
 
242
243
  ```bash
243
244
  bundle exec rake container:build'[local]'
@@ -246,60 +247,61 @@ bundle exec rake container:test
246
247
 
247
248
  ### Automated Release Process
248
249
 
249
- Releases are triggered via the GitHub Actions UI using the **Release** workflow (`.github/workflows/release.yml`).
250
+ Releases are managed through the GitHub Actions UI via the **New Release on GitHub** workflow (`.github/workflows/release.yml`).
251
+
252
+ 1. Navigate to **Actions** > **New Release on GitHub**
253
+ 2. Select **Run workflow**
254
+ 3. (Optionally) Specify:
255
+ - **Release version**: Defaults to the current `version.rb` value (minus the `.pre` suffix).
250
256
 
251
- To create a release:
257
+ e.g. current `a.b.c.pre` &rarr; `a.b.c`.
258
+ - **Next development version**: Defaults to an incremented minor version with the `.pre` suffix.
252
259
 
253
- 1. Navigate to **Actions** > **Release** in the GitHub repository
254
- 2. Click **Run workflow**
255
- 3. Optionally specify:
256
- - **Release version**: The version to release. If left empty, uses the current version from `version.rb` without the `.pre` suffix.
257
- - **Next development version**: The next version to prepare for. If left empty, auto-increments the minor version. The `.pre` suffix is added automatically.
260
+ e.g. release `a.b.c` &rarr; `a.(b+1).0.pre`.
258
261
  4. Click **Run workflow**
259
262
 
260
- The workflow automatically:
263
+ The automated workflow performs the following:
261
264
 
262
- 1. Updates `version.rb` with the release version
263
- 2. Rebuilds documentation (PDF manual, Markdown README)
265
+ 1. Updates `version.rb` to the release version
266
+ 2. Rebuilds all documentation (PDF and Markdown)
264
267
  3. Commits the changes
265
268
  4. Creates and pushes the release tag
266
269
  5. Triggers the `deploy` workflow to publish to [rubygems.org](https://rubygems.org/gems/aspera-cli)
267
- 6. Updates `version.rb` to the next development version with `.pre` suffix
268
- 7. Commits and pushes the version bump in main branch.
270
+ 6. Increments `version.rb` to the next development version.
271
+ 7. Commits and pushes the version bump to `main`.
269
272
 
270
273
  ### Manual Release Process (Alternative)
271
274
 
272
- If needed, releases can still be done manually.
273
- Basically, follow the same procedure as in the GitHub action:
275
+ If necessary, you can mirror the automated process manually:
274
276
 
275
277
  - Update the version in `lib/aspera/cli/version.rb` (remove `.pre` suffix)
276
278
 
277
- - Build the PDF manual in `pkg`:
279
+ - Build the PDF manual:
278
280
 
279
281
  ```shell
280
282
  bundle exec rake doc:build
281
283
  ```
282
284
 
283
- - Build the signed `.gem` in `pkg`:
285
+ - Build the signed gem:
284
286
 
285
287
  ```shell
286
288
  bundle exec rake SIGNING_KEY=/path/to/vault/gem-private_key.pem
287
289
  ```
288
290
 
289
- - Create the release version tag and push it to GitHub:
291
+ - Tag the release and push to GitHub:
290
292
 
291
293
  ```shell
292
294
  bundle exec rake release_tag
293
295
  ```
294
296
 
295
- This will trigger the action `.github/workflows/deploy.yml`, which builds the gem file and pushes it to RubyGems.
297
+ This triggers the `.github/workflows/deploy.yml` action to publish to RubyGems.
296
298
 
297
- - After release, update `version.rb` to the next development version with `.pre` suffix
299
+ - Update `version.rb` to the next `.pre` development version.
298
300
 
299
301
  ## Future Improvements
300
302
 
301
- - Replace custom REST and OAuth classes with standard Ruby gems ?
302
- - <https://github.com/rest-client/rest-client>
303
- - <https://github.com/oauth-xx/oauth2>
304
- - Use the `thor` gem <http://whatisthor.com/> (or other standard Ruby CLI manager)
305
- - Look at <https://github.com/phusion/traveling-ruby>
303
+ - Evaluate replacing custom REST and OAuth implementations with standard gems:
304
+ - [rest-client](https://github.com/rest-client/rest-client)
305
+ - [oauth2](https://github.com/oauth-xx/oauth2)
306
+ - Integrate `thor` <http://whatisthor.com/> or another standard Ruby CLI framework.
307
+ - Explore [Traveling Ruby](https://github.com/phusion/traveling-ruby) for distribution.
@@ -145,11 +145,10 @@ module Aspera
145
145
  # Call `block` with same query using paging and response information.
146
146
  # Block must return a 2 element `Array` with data and http response
147
147
  # @param query [Hash] Additionnal query parameters
148
- # @param progress [nil, Object] Uses methods: `long_operation_running` and `long_operation_terminated`
149
148
  # @return [Hash] Items and total number of items
150
149
  # @option return [Array<Hash>] :items The list of items
151
150
  # @option return [Integer] :total The total number of items
152
- def call_paging(query: {}, progress: nil)
151
+ def call_paging(query: {})
153
152
  Aspera.assert_type(query, Hash){'query'}
154
153
  Aspera.assert(block_given?)
155
154
  # set default large page if user does not specify own parameters. AoC Caps to 1000 anyway
@@ -177,9 +176,9 @@ module Aspera
177
176
  break if !max_items.nil? && item_list.count >= max_items
178
177
  break if !max_pages.nil? && page_count >= max_pages
179
178
  break if total_count&.<=(item_list.count)
180
- progress&.long_operation_running("#{item_list.count} / #{total_count}") unless total_count.eql?(item_list.count.to_s)
179
+ RestParameters.instance.spinner_cb.call("#{item_list.count} / #{total_count}") unless total_count.eql?(item_list.count.to_s)
181
180
  end
182
- progress&.long_operation_terminated
181
+ RestParameters.instance.spinner_cb.call(action: :success)
183
182
  item_list = item_list[0..max_items - 1] if !max_items.nil? && item_list.count > max_items
184
183
  return {items: item_list, total: total_count}
185
184
  end
@@ -232,8 +231,7 @@ module Aspera
232
231
  username: nil,
233
232
  password: nil,
234
233
  workspace: nil,
235
- secret_finder: nil,
236
- progress_disp: nil
234
+ secret_finder: nil
237
235
  )
238
236
  # Test here because link may set url
239
237
  Aspera.assert(url, 'Missing mandatory option: url', type: ParameterError)
@@ -244,7 +242,6 @@ module Aspera
244
242
  # key: access key
245
243
  # value: associated secret
246
244
  @secret_finder = secret_finder
247
- @progress_disp = progress_disp
248
245
  @workspace_name = workspace
249
246
  @cache_user_info = nil
250
247
  @cache_url_token_info = nil
@@ -309,7 +306,7 @@ module Aspera
309
306
  # @param query [nil, Hash] Additional query
310
307
  # @return [Hash] {items: , total: }
311
308
  def read_with_paging(subpath, query = nil)
312
- return self.class.call_paging(query: query, progress: @progress_disp) do |paged_query|
309
+ return self.class.call_paging(query: query) do |paged_query|
313
310
  read(subpath, query: paged_query, ret: :both)
314
311
  end
315
312
  end
@@ -10,6 +10,7 @@ module Aspera
10
10
  class CosNode < Node
11
11
  IBM_CLOUD_TOKEN_URL = 'https://iam.cloud.ibm.com/identity'
12
12
  TOKEN_FIELD = 'delegated_refresh_token'
13
+ FASP_INFO_KEYS = %w[ATSEndpoint AccessKey].freeze
13
14
  class << self
14
15
  def parameters_from_svc_credentials(service_credentials, bucket_region)
15
16
  # check necessary contents
@@ -60,6 +61,9 @@ module Aspera
60
61
  ).body
61
62
  ats_info = XmlSimple.xml_in(xml_result_text, {'ForceArray' => false})
62
63
  Log.dump(:ats_info, ats_info)
64
+ Aspera.assert_hash_all(ats_info, String, nil){'ats_info'}
65
+ Aspera.assert((FASP_INFO_KEYS - ats_info.keys).empty?){'ats_info missing required keys'}
66
+ Aspera.assert_hash_all(ats_info['AccessKey'], String, String){'ats_info'}
63
67
  @storage_credentials = {
64
68
  'type' => 'token',
65
69
  'token' => {TOKEN_FIELD => nil}
@@ -91,7 +91,7 @@ module Aspera
91
91
  JOB_RUNNING = %w[queued working].freeze
92
92
  PATH_STANDARD_ROOT = '/aspera/faspex'
93
93
  PATH_API_DETECT = "#{PATH_API_V5}/#{PATH_HEALTH}"
94
- HEADER_ITERATION_TOKEN = 'X-Aspera-Next-Iteration-Token'
94
+ HEADER_X_NEXT_ITER_TOKEN = 'X-Aspera-Next-Iteration-Token'
95
95
  HEADER_FASPEX_VERSION = 'X-IBM-Aspera'
96
96
  EMAIL_NOTIF_LIST = %w[
97
97
  welcome_email
@@ -136,6 +136,15 @@ module Aspera
136
136
  end
137
137
  attr_reader :pub_link_context
138
138
 
139
+ # @param url Faspex URL, can be a public link
140
+ # @param auth Authentication method: :boot (token in header), :web (open browser), :jwt (client_id + private key), :public_link (context in URL)
141
+ # @param password For :boot auth, the token copied directly from browser in developer mode
142
+ # @param client_id For :web and :jwt auth, the client_id of web UI application
143
+ # @param client_secret For :web auth, the client_secret of web UI application (not needed for :jwt)
144
+ # @param redirect_uri For :web auth, the redirect_uri of web UI application (must be the same as in application configuration)
145
+ # @param username For :jwt auth, the username of the user to impersonate
146
+ # @param private_key For :jwt auth, the private key to sign JWT token
147
+ # @param passphrase For :jwt auth, the passphrase of the private key
139
148
  def initialize(
140
149
  url:,
141
150
  auth:,
@@ -6,16 +6,16 @@ require 'aspera/oauth'
6
6
  require 'aspera/log'
7
7
  require 'aspera/assert'
8
8
  require 'aspera/environment'
9
- require 'zlib'
10
9
  require 'base64'
11
10
  require 'openssl'
12
11
  require 'pathname'
12
+ require 'zlib'
13
13
  require 'net/ssh/buffer'
14
14
 
15
15
  module Aspera
16
16
  module Api
17
17
  # Provides additional functions using node API with gen4 extensions (access keys)
18
- class Node < Aspera::Rest
18
+ class Node < Rest
19
19
  # Format of node scope : node.<access key>:<scope>
20
20
  module Scope
21
21
  # Node sub-scopes
@@ -42,9 +42,10 @@ module Aspera
42
42
  ACCESS_LEVELS = %w[delete list mkdir preview read rename write].freeze
43
43
  # Special HTTP Headers
44
44
  HEADER_X_ASPERA_ACCESS_KEY = 'X-Aspera-AccessKey'
45
- HEADER_X_TOTAL_COUNT = 'X-Total-Count'
46
45
  HEADER_X_CACHE_CONTROL = 'X-Aspera-Cache-Control'
46
+ HEADER_X_TOTAL_COUNT = 'X-Total-Count'
47
47
  HEADER_X_NEXT_ITER_TOKEN = 'X-Aspera-Next-Iteration-Token'
48
+ HEADER_ACCEPT_VERSION = 'Accept-Version'
48
49
  # / in cloud
49
50
  PATH_SEPARATOR = '/'
50
51
 
@@ -52,22 +53,36 @@ module Aspera
52
53
  OAuth::Factory.instance.register_decoder(lambda{ |token| Node.decode_bearer_token(token)})
53
54
 
54
55
  # Class instance variable, access with accessors on class
55
- @use_standard_ports = true
56
- @use_node_cache = true
57
-
58
- class << self
56
+ @api_options = {
59
57
  # Set to false to read transfer parameters from download_setup
60
- attr_accessor :use_standard_ports
58
+ standard_ports: true,
61
59
  # Set to false to bypass cache in redis
62
- attr_accessor :use_node_cache
60
+ cache: true,
61
+ accept_v4: true
62
+ }
63
+ OPTIONS = @api_options.keys.freeze
64
+
65
+ class << self
66
+ attr_reader :api_options
63
67
  attr_reader :use_dynamic_key
64
68
 
65
- # Adds cache control header, as globally specified to read request
66
- # Use like this: read(...,**cache_control)
67
- def cache_control
68
- headers = {'Accept' => Mime::JSON}
69
- headers[HEADER_X_CACHE_CONTROL] = 'no-cache' unless use_node_cache
70
- {headers: headers}
69
+ def api_options=(h)
70
+ Aspera.assert_type(h, Hash)
71
+ h.each do |k, v|
72
+ Aspera.assert(@api_options.key?(k.to_sym)){"unknown api option: #{k} (#{OPTIONS.join(', ')})"}
73
+ Aspera.assert_type(v, TrueClass, FalseClass){"api options value for #{k} should be boolean"}
74
+ @api_options[k.to_sym] = v
75
+ end
76
+ end
77
+
78
+ # Adds cache control header for node API /files/:id
79
+ # as globally specified to read request
80
+ # Use like this: read(..., headers: add_cache_control)
81
+ # @param headers [Hash] optional initial headers to add to
82
+ # @return [Hash] headers with cache control header added if needed
83
+ def add_cache_control(headers = {})
84
+ headers[HEADER_X_CACHE_CONTROL] = 'no-cache' unless api_options[:cache]
85
+ headers
71
86
  end
72
87
 
73
88
  # Set private key to be used
@@ -180,8 +195,8 @@ module Aspera
180
195
  Aspera.assert(!access_key.nil?)
181
196
  end
182
197
  return {
183
- Node::HEADER_X_ASPERA_ACCESS_KEY => access_key,
184
- 'Authorization' => bearer_auth
198
+ HEADER_X_ASPERA_ACCESS_KEY => access_key,
199
+ 'Authorization' => bearer_auth
185
200
  }
186
201
  end
187
202
  end
@@ -248,6 +263,50 @@ module Aspera
248
263
  return false
249
264
  end
250
265
 
266
+ # Read folder content, with pagination management for gen4, not recursive
267
+ # if `Accept-Version: 4.0` is not specified:
268
+ # if `page` and `per_page` are not specified, then all entries are returned.
269
+ # if either `page` or `per_page` is specified, then both are required, else 400
270
+ # if `Accept-Version: 4.0` is specified:
271
+ # those queries are not available: page (not mentioned), sort, min_size, max_size, min_modified_time, max_modified_time, target_id, target_node_id, files_prefetch_count, page, name_iglob : either ignored or result in API error 400.
272
+ # query include is accepted, but seems to do nothing as access_levels and recursive_counts are already included in results.
273
+ # query `iteration_token` is accepted and allows to get paginated results, with `X-Aspera-Next-Iteration-Token` header in response to get next page token. `X-Aspera-Total-Count` header gives total count of entries.
274
+ def read_folder_content(file_id, query = nil, exception: true, path: nil)
275
+ folder_items = []
276
+ begin
277
+ query ||= {}
278
+ headers = self.class.add_cache_control
279
+ use_v4 = self.class.api_options[:accept_v4]
280
+ return read("files/#{file_id}/files", query, headers: headers) unless use_v4 || query.key?('page') || query.key?('per_page')
281
+ if use_v4
282
+ headers[HEADER_ACCEPT_VERSION] = '4.0'
283
+ query['per_page'] = 1000 unless query.key?('per_page')
284
+ elsif query.key?('per_page') && !query.key?('page')
285
+ query['page'] = 0
286
+ end
287
+ loop do
288
+ RestParameters.instance.spinner_cb.call(folder_items.count)
289
+ data, http = read("files/#{file_id}/files", query, headers: headers, ret: :both)
290
+ folder_items.concat(data)
291
+ if use_v4
292
+ iteration_token = http[HEADER_X_NEXT_ITER_TOKEN]
293
+ break if iteration_token.nil? || iteration_token.empty?
294
+ query['iteration_token'] = iteration_token
295
+ else
296
+ break if data['item_count'].eql?(0)
297
+ query['offset'] += data['item_count']
298
+ end
299
+ end
300
+ rescue StandardError => e
301
+ raise e if exception
302
+ Log.log.warn{"#{path}: #{e.class} #{e.message}"}
303
+ Log.log.debug{(['Backtrace:'] + e.backtrace).join("\n")}
304
+ ensure
305
+ RestParameters.instance.spinner_cb.call(folder_items.count, action: :success)
306
+ end
307
+ folder_items
308
+ end
309
+
251
310
  # Recursively browse in a folder (with non-recursive method)
252
311
  # Entries of folders are processed if the processing method returns true
253
312
  # Links are processed on the respective node
@@ -267,14 +326,7 @@ module Aspera
267
326
  current_item = folders_to_explore.shift
268
327
  Log.log.debug{"Exploring #{current_item[:path]}".bg_green}
269
328
  # Get folder content
270
- folder_contents =
271
- begin
272
- # TODO: use header
273
- read("files/#{current_item[:id]}/files", query, **self.class.cache_control)
274
- rescue StandardError => e
275
- Log.log.warn{"#{current_item[:path]}: #{e.class} #{e.message}"}
276
- []
277
- end
329
+ folder_contents = read_folder_content(current_item[:id], query, exception: false, path: current_item[:path])
278
330
  Log.dump(:folder_contents, folder_contents)
279
331
  folder_contents.each do |entry|
280
332
  if entry.key?('error')
@@ -309,7 +361,7 @@ module Aspera
309
361
  # @param path [String] file or folder path (end with "/" is like setting process_last_link)
310
362
  # @param process_last_link [Boolean] if true, follow the last link
311
363
  # @return [Hash] Result data
312
- # @option return [Aspera::Rest] :api REST client instance
364
+ # @option return [Rest] :api REST client instance
313
365
  # @option return [String] :file_id File identifier
314
366
  def resolve_api_fid(top_file_id, path, process_last_link = false)
315
367
  Aspera.assert_type(top_file_id, String)
@@ -445,7 +497,7 @@ module Aspera
445
497
  # Add application specific tags (AoC)
446
498
  @app_info[:api].add_ts_tags(transfer_spec: transfer_spec, app_info: @app_info) unless @app_info.nil?
447
499
  # Add remote host info
448
- if self.class.use_standard_ports
500
+ if self.class.api_options[:standard_ports]
449
501
  # Get default TCP/UDP ports and transfer user
450
502
  transfer_spec.merge!(Transfer::Spec::AK_TSPEC_BASE)
451
503
  # By default: same address as node API
@@ -128,20 +128,34 @@ module Aspera
128
128
  @spinner = nil
129
129
  end
130
130
 
131
- # call this after REST calls if several api calls are expected
132
- def long_operation_running(title = '')
131
+ def long_operation(title = nil, action: :spin)
133
132
  return unless Environment.terminal?
134
- if @spinner.nil?
133
+ return if %i[error data].include?(@options[:display])
134
+
135
+ # Handle the "delayed start" state
136
+ return @spinner = :starting if action == :spin && @spinner.nil?
137
+
138
+ # Cleanup if we try to stop a spinner that never actually started
139
+ @spinner = nil if action != :spin && @spinner == :starting
140
+ return if @spinner.nil?
141
+
142
+ # Initialize the real TTY object if it's currently just the :starting symbol
143
+ if @spinner == :starting
135
144
  @spinner = TTY::Spinner.new('[:spinner] :title', format: :classic)
145
+ @spinner.update(title: '')
136
146
  @spinner.start
137
147
  end
138
- @spinner.update(title: title)
139
- @spinner.spin
140
- end
141
148
 
142
- def long_operation_terminated
143
- @spinner&.stop
144
- @spinner = nil
149
+ @spinner.update(title: title) if title
150
+
151
+ case action
152
+ when :spin
153
+ @spinner.spin
154
+ when :success, :fail
155
+ action == :success ? @spinner.success : @spinner.error
156
+ @spinner.stop
157
+ @spinner = nil
158
+ end
145
159
  end
146
160
 
147
161
  def declare_options(options)
@@ -150,9 +164,9 @@ module Aspera
150
164
  else
151
165
  {}
152
166
  end
167
+ options.declare(:display, 'Output only some information', allowed: DISPLAY_LEVELS, handler: {o: self, m: :option_handler}, default: :data)
153
168
  options.declare(:format, 'Output format', allowed: DISPLAY_FORMATS, handler: {o: self, m: :option_handler}, default: :table)
154
169
  options.declare(:output, 'Destination for results', handler: {o: self, m: :option_handler})
155
- options.declare(:display, 'Output only some information', allowed: DISPLAY_LEVELS, handler: {o: self, m: :option_handler}, default: :info)
156
170
  options.declare(
157
171
  :fields, "Comma separated list of: fields, or #{SpecialValues::ALL}, or #{SpecialValues::DEF}", handler: {o: self, m: :option_handler},
158
172
  allowed: [String, Array, Regexp, Proc],
@@ -178,6 +192,8 @@ module Aspera
178
192
  @options[option_symbol] = value
179
193
  # special handling of some options
180
194
  case option_symbol
195
+ when :format
196
+ @options[:display] = value.eql?(:table) ? :info : :data
181
197
  when :output
182
198
  $stdout = if value.eql?('-')
183
199
  STDOUT # rubocop:disable Style/GlobalStdStream
@@ -197,7 +213,7 @@ module Aspera
197
213
  nil
198
214
  end
199
215
 
200
- # main output method
216
+ # Main output method
201
217
  # data: for requested data, not displayed if level==error
202
218
  # info: additional info, displayed if level==info
203
219
  # error: always displayed on stderr
@@ -330,7 +330,7 @@ module Aspera
330
330
 
331
331
  # @param descr [String] description for help
332
332
  # @param mandatory [Boolean] if true, raise error if option not set
333
- # @param multiple [Boolean] if true, return remaining arguments (Array)
333
+ # @param multiple [Boolean] if true, return remaining arguments (Array) unil END
334
334
  # @param accept_list [Array, NilClass] list of allowed values (Symbol)
335
335
  # @param validation [Class, Array, NilClass] Accepted value type(s) or list of Symbols
336
336
  # @param aliases [Hash] map of aliases: key = alias, value = real value
@@ -345,10 +345,19 @@ module Aspera
345
345
  descr = "#{descr} (#{validation.join(', ')})" unless validation.nil? || validation.eql?(Allowed::TYPES_STRING)
346
346
  result =
347
347
  if !@unprocessed_cmd_line_arguments.empty?
348
- how_many = multiple ? @unprocessed_cmd_line_arguments.length : 1
349
- values = @unprocessed_cmd_line_arguments.shift(how_many)
348
+ if multiple
349
+ index = @unprocessed_cmd_line_arguments.index(SpecialValues::EOA)
350
+ if index.nil?
351
+ values = @unprocessed_cmd_line_arguments.shift(@unprocessed_cmd_line_arguments.length)
352
+ else
353
+ values = @unprocessed_cmd_line_arguments.shift(index)
354
+ @unprocessed_cmd_line_arguments.shift # remove EOA
355
+ end
356
+ else
357
+ values = [@unprocessed_cmd_line_arguments.shift]
358
+ end
350
359
  values = values.map{ |v| ExtendedValue.instance.evaluate(v, context: "argument: #{descr}", allowed: validation)}
351
- # if expecting list and only one arg of type array : it is the list
360
+ # If expecting list and only one arg of type array : it is the list
352
361
  values = values.first if multiple && values.length.eql?(1) && values.first.is_a?(Array)
353
362
  if accept_list
354
363
  allowed_values = [].concat(accept_list)
@@ -233,8 +233,7 @@ module Aspera
233
233
  defaults: {workspace: nil},
234
234
  scope: @scope,
235
235
  subpath: aoc_base_path,
236
- secret_finder: config,
237
- progress_disp: formatter
236
+ secret_finder: config
238
237
  ))
239
238
  end
240
239
 
@@ -256,7 +255,10 @@ module Aspera
256
255
  # @param hash [Hash,nil] Optional base `Hash` (modified)
257
256
  # @param string [Boolean] `true` to set key as `String`, else as `Symbol`
258
257
  # @param name [Boolean] Include name
259
- # @return [Hash] with key `workspace_[id,name]` (symbol or string) only if defined
258
+ # @return [Hash{Symbol, String => String}] the modified hash containing:
259
+ # * `workspace_id` [String] the unique identifier.
260
+ # * `workspace_name` [String] (optional) the name, included if +name+ is true.
261
+ # @note The key type (String or Symbol) depends on the +string+ parameter.
260
262
  def workspace_id_hash(hash = nil, string: false, name: false)
261
263
  info = aoc_api.workspace
262
264
  hash = {} if hash.nil?
@@ -300,9 +300,9 @@ module Aspera
300
300
  remain_pages -= 1 unless remain_pages.nil?
301
301
  break if remain_pages == 0
302
302
  offset += page_result[items_key].length
303
- formatter.long_operation_running
303
+ RestParameters.instance.spinner_cb.call("#{result.length} / #{total_count || '?'}")
304
304
  end
305
- formatter.long_operation_terminated
305
+ RestParameters.instance.spinner_cb.call(action: :success)
306
306
  return result, total_count
307
307
  end
308
308
 
@@ -191,7 +191,8 @@ module Aspera
191
191
  def setup_rest_and_transfer_runtime
192
192
  RestParameters.instance.user_agent = Info::CMD_NAME
193
193
  RestParameters.instance.progress_bar = @progress_bar
194
- RestParameters.instance.session_cb = lambda{ |http_session| update_http_session(http_session)}
194
+ RestParameters.instance.session_cb = ->(http_session){update_http_session(http_session)}
195
+ RestParameters.instance.spinner_cb = ->(title = nil, action: :spin){formatter.long_operation(title, action: action)}
195
196
  # Check http options that are global
196
197
  keys_to_delete = []
197
198
  @option_http_options.each do |k, v|
@@ -164,10 +164,10 @@ module Aspera
164
164
  loop do
165
165
  result = @api_v5.read("jobs/#{job_id}", {type: :formatted})
166
166
  break unless Api::Faspex::JOB_RUNNING.include?(result['status'])
167
- formatter.long_operation_running(result['status'])
167
+ RestParameters.instance.spinner_cb.call(result['status'])
168
168
  sleep(0.5)
169
169
  end
170
- formatter.long_operation_terminated
170
+ RestParameters.instance.spinner_cb.call(action: :success)
171
171
  return result
172
172
  end
173
173
 
@@ -192,6 +192,16 @@ module Aspera
192
192
  return list.select(&filter), total
193
193
  end
194
194
 
195
+ # Build query to get package recipients based on package info in case of shared inbox or workgroup recipient
196
+ # @param package_id [String] the package id to get info from
197
+ def recipient_query(package_id)
198
+ package_info = @api_v5.read("packages/#{package_id}")
199
+ base_query = {}
200
+ base_query['recipient_workgroup_id'] = package_info['recipients'].first['id'] if WORKGROUP_TYPES.include?(package_info['recipients'].first['recipient_type'])
201
+ base_query['recipient_user_id'] = package_info['recipients'].first['id'] if package_info['recipients'].first['recipient_type'].eql?('user')
202
+ base_query
203
+ end
204
+
195
205
  def package_receive(package_ids)
196
206
  # prepare persistency if needed
197
207
  skip_ids_persistency = nil
@@ -241,7 +251,7 @@ module Aspera
241
251
  type: Api::Faspex.box_type(box),
242
252
  transfer_type: Api::Faspex::TRANSFER_CONNECT
243
253
  }
244
- download_params[:recipient_workgroup_id] = lookup_entity_by_field(api: @api_v5, entity: options.get_option(:group_type), value: box)['id'] if !Api::Faspex::API_LIST_MAILBOX_TYPES.include?(box) && box != SpecialValues::ALL
254
+ # download_params[:recipient_workgroup_id] = lookup_entity_by_field(api: @api_v5, entity: options.get_option(:group_type), value: box)['id'] if !Api::Faspex::API_LIST_MAILBOX_TYPES.include?(box) && box != SpecialValues::ALL
245
255
  packages.each do |package|
246
256
  pkg_id = package['id']
247
257
  formatter.display_status("Receiving package #{pkg_id}")
@@ -249,7 +259,7 @@ module Aspera
249
259
  transfer_spec = @api_v5.call(
250
260
  operation: 'POST',
251
261
  subpath: "packages/#{pkg_id}/transfer_spec/download",
252
- query: download_params,
262
+ query: download_params.merge(recipient_query(pkg_id)),
253
263
  content_type: Mime::JSON,
254
264
  body: param_file_list,
255
265
  headers: {'Accept' => Mime::JSON}
@@ -319,9 +329,9 @@ module Aspera
319
329
 
320
330
  # Browse a folder
321
331
  # @param browse_endpoint [String] the endpoint to browse
322
- def browse_folder(browse_endpoint)
332
+ def browse_folder(browse_endpoint, base_query = {})
323
333
  folders_to_process = [options.get_next_argument('folder path', default: '/')]
324
- query = query_read_delete(default: {})
334
+ query = base_query.merge(query_read_delete(default: {}))
325
335
  filters = query.delete('filters'){{}}
326
336
  Aspera.assert_type(filters, Hash)
327
337
  filters['basenames'] ||= []
@@ -357,7 +367,7 @@ module Aspera
357
367
  end
358
368
  folders_to_process.concat(data['items'].select{ |i| i['type'].eql?('directory')}.map{ |i| i['path']}) if recursive
359
369
  if use_paging
360
- iteration_token = http[Api::Faspex::HEADER_ITERATION_TOKEN]
370
+ iteration_token = http[Api::Faspex::HEADER_X_NEXT_ITER_TOKEN]
361
371
  break if iteration_token.nil? || iteration_token.empty?
362
372
  query['iteration_token'] = iteration_token
363
373
  else
@@ -365,12 +375,11 @@ module Aspera
365
375
  break if data['item_count'].eql?(0)
366
376
  query['offset'] += data['item_count']
367
377
  end
368
- formatter.long_operation_running(all_items.count)
378
+ RestParameters.instance.spinner_cb.call(all_items.count)
369
379
  end
370
380
  query.delete('iteration_token')
371
381
  end
372
- formatter.long_operation_terminated
373
-
382
+ RestParameters.instance.spinner_cb.call(action: :success)
374
383
  return Main.result_object_list(all_items, total: total_count)
375
384
  end
376
385
 
@@ -384,7 +393,7 @@ module Aspera
384
393
  when :show
385
394
  return Main.result_single_object(@api_v5.read("packages/#{package_id}"))
386
395
  when :browse
387
- return browse_folder("packages/#{package_id}/files/#{Api::Faspex.box_type(options.get_option(:box))}")
396
+ return browse_folder("packages/#{package_id}/files/#{Api::Faspex.box_type(options.get_option(:box))}", recipient_query(package_id))
388
397
  when :status
389
398
  status_list = options.get_next_argument('list of states, or nothing', mandatory: false, validation: Array)
390
399
  status = wait_package_status(package_id, status_list: status_list)
@@ -447,6 +456,8 @@ module Aspera
447
456
  res_id_query = {'all': true}
448
457
  when :nodes
449
458
  available_commands += %i[shared_folders browse]
459
+ when :jobs
460
+ exec_args[:display_fields] = %w[id job_name job_type status]
450
461
  end
451
462
  res_command = options.get_next_command(available_commands)
452
463
  return Main.result_value_list(Api::Faspex::EMAIL_NOTIF_LIST, name: 'email_id') if res_command.eql?(:list) && res_sym.eql?(:email_notifications)
@@ -719,7 +730,8 @@ module Aspera
719
730
  end
720
731
  SHARED_INBOX_MEMBER_LEVELS = %i[submit_only standard shared_inbox_admin].freeze
721
732
  ACCOUNT_TYPES = %w{local_user saml_user self_registered_user external_user}.freeze
722
- CONTACT_TYPES = %w{workgroup shared_inbox distribution_list user external_user}.freeze
733
+ WORKGROUP_TYPES = %w{workgroup shared_inbox}.freeze
734
+ CONTACT_TYPES = (WORKGROUP_TYPES + %w{distribution_list user external_user}).freeze
723
735
  private_constant :SHARED_INBOX_MEMBER_LEVELS, :ACCOUNT_TYPES, :CONTACT_TYPES
724
736
  end
725
737
  end
@@ -93,12 +93,9 @@ module Aspera
93
93
  options.declare(:validator, 'Identifier of validator (optional for central)')
94
94
  options.declare(:asperabrowserurl, 'URL for simple aspera web ui', default: 'https://asperabrowser.mybluemix.net')
95
95
  options.declare(
96
- :default_ports, 'Gen4: Use standard FASP ports (true) or get from node API (false)', allowed: Allowed::TYPES_BOOLEAN, default: true,
97
- handler: {o: Api::Node, m: :use_standard_ports}
98
- )
99
- options.declare(
100
- :node_cache, 'Gen4: Set to no to force actual file system read', allowed: Allowed::TYPES_BOOLEAN,
101
- handler: {o: Api::Node, m: :use_node_cache}
96
+ :node_api, 'Gen4: standard_ports: Use standard FASP ports (true) or get from node API (false). cache: Set to false to force actual file system read',
97
+ allowed: Hash,
98
+ handler: {o: Api::Node, m: :api_options}
102
99
  )
103
100
  options.declare(:root_id, 'Gen4: File id of top folder when using access key (override AK root id)')
104
101
  options.declare(:dynamic_key, 'Private key PEM to use for dynamic key auth', handler: {o: Api::Node, m: :use_dynamic_key})
@@ -242,14 +239,14 @@ module Aspera
242
239
  break if all_items.count >= total_count
243
240
  offset += items.count
244
241
  query['skip'] = offset
245
- formatter.long_operation_running(all_items.count)
242
+ RestParameters.instance.spinner_cb.call(all_items.count)
246
243
  end
247
244
  query.delete('skip')
248
245
  end
249
246
  @prefixer&.remove_in_object_list!(all_items)
250
247
  return Main.result_object_list(all_items)
251
248
  ensure
252
- formatter.long_operation_terminated
249
+ RestParameters.instance.spinner_cb.call(action: :success)
253
250
  end
254
251
 
255
252
  # Create async transfer spec request from direction and folders
@@ -388,7 +385,7 @@ module Aspera
388
385
  when :transport
389
386
  return Main.result_single_object(@api_node.transport_params)
390
387
  when :spec
391
- return Main.result_single_object(@api_node.base_spec)
388
+ return Main.result_single_object(@api_node.base_spec, fields: Formatter.all_but(Transfer::Spec::SPECIFIC))
392
389
  end
393
390
  Aspera.error_unreachable_line
394
391
  end
@@ -523,7 +520,7 @@ module Aspera
523
520
  return Main.result_text(result[:password])
524
521
  when :browse
525
522
  apifid = apifid_from_next_arg(top_file_id)
526
- file_info = apifid[:api].read("files/#{apifid[:file_id]}", **Api::Node.cache_control)
523
+ file_info = apifid[:api].read("files/#{apifid[:file_id]}", headers: Api::Node.add_cache_control)
527
524
  unless file_info['type'].eql?('folder')
528
525
  # a single file
529
526
  return Main.result_object_list([file_info], fields: GEN4_LS_FIELDS)
@@ -8,6 +8,7 @@ module Aspera
8
8
  INIT = 'INIT'
9
9
  ALL = 'ALL'
10
10
  DEF = 'DEF'
11
+ EOA = 'END'
11
12
  end
12
13
  end
13
14
  end
@@ -4,6 +4,6 @@ module Aspera
4
4
  module Cli
5
5
  # For beta add extension : .beta1
6
6
  # For dev version add extension : .pre
7
- VERSION = '4.25.3'
7
+ VERSION = '4.25.4'
8
8
  end
9
9
  end
@@ -62,7 +62,7 @@ module Aspera
62
62
  detection_info = nil
63
63
  begin
64
64
  Log.log.debug{"detecting #{plugin_name_sym} at #{app_url}"}
65
- formatter.long_operation_running("#{plugin_name_sym}\r")
65
+ RestParameters.instance.spinner_cb.call(plugin_name_sym.to_s)
66
66
  detection_info = plugin_klass.detect(app_url)
67
67
  rescue OpenSSL::SSL::SSLError => e
68
68
  Log.log.warn(e.message)
@@ -78,6 +78,7 @@ module Aspera
78
78
  # If there is a redirect, then the detector can override the url.
79
79
  found_apps.push({product: plugin_name_sym, name: app_name, url: app_url, version: 'unknown'}.merge(detection_info))
80
80
  end
81
+ RestParameters.instance.spinner_cb.call(action: :success)
81
82
  raise "No known application found at #{app_url}" if found_apps.empty?
82
83
  Aspera.assert(found_apps.all?{ |a| a.keys.all?(Symbol)})
83
84
  return found_apps
data/lib/aspera/rest.rb CHANGED
@@ -33,7 +33,7 @@ module Aspera
33
33
  class RestParameters
34
34
  include Singleton
35
35
 
36
- attr_accessor :user_agent, :download_partial_suffix, :retry_on_error, :retry_on_timeout, :retry_on_unavailable, :retry_max, :retry_sleep, :session_cb, :progress_bar
36
+ attr_accessor :user_agent, :download_partial_suffix, :retry_on_error, :retry_on_timeout, :retry_on_unavailable, :retry_max, :retry_sleep, :session_cb, :progress_bar, :spinner_cb
37
37
 
38
38
  private
39
39
 
@@ -47,6 +47,7 @@ module Aspera
47
47
  @retry_sleep = 4
48
48
  @session_cb = nil
49
49
  @progress_bar = nil
50
+ @spinner_cb = nil
50
51
  end
51
52
  end
52
53
 
@@ -88,8 +89,9 @@ module Aspera
88
89
  query
89
90
  end
90
91
 
91
- # Build URI from URL and parameters and check it is http or https
92
- # Check iof php style is specified
92
+ # Build URI from URL and parameters and check it is `http` or `https`.
93
+ # Check if php style is specified.
94
+ # `nil` values in query result in key without value, e.g. `?a`, while empty string values result in `?a=`.
93
95
  # @param url [String] The URL without query.
94
96
  # @param query [Hash,Array,String] The query.
95
97
  def build_uri(url, query)
@@ -105,18 +107,19 @@ module Aspera
105
107
  URI.encode_www_form(h_to_query_array(query))
106
108
  when Array
107
109
  Aspera.assert(query.all?{ |i| i.is_a?(Array) && i.length.eql?(2)}){'Query must be array of arrays of 2 elements'}
108
- URI.encode_www_form(query)
110
+ URI.encode_www_form(query) # remove nil values
109
111
  else Aspera.error_unexpected_value(query.class){'query type'}
110
112
  end.gsub('%5B%5D=', '[]=')
111
113
  # [] is allowed in url parameters
112
114
  uri
113
115
  end
114
116
 
117
+ # Support array for query parameter, there is no standard.
118
+ # Either p=1&p=2 (default)
119
+ # or p[]=1&p[]=2 (if `:x_array_php_style` is set to true in query)
115
120
  # @param query [Hash] HTTP query as hash
116
121
  def h_to_query_array(query)
117
122
  Aspera.assert_type(query, Hash)
118
- # Support array for query parameter, there is no standard.
119
- # Either p[]=1&p[]=2, or p=1&p=2
120
123
  suffix = query.delete(:x_array_php_style) ? '[]' : nil
121
124
  query.each_with_object([]) do |(k, v), query_array|
122
125
  case v
@@ -412,7 +415,7 @@ module Aspera
412
415
  http_session.request(req) do |response|
413
416
  result_http = response
414
417
  result_mime = self.class.parse_header(result_http['Content-Type'] || Mime::TEXT)[:type]
415
- Log.log.debug{"response: code=#{result_http.code}, mime=#{result_mime}, mime2= #{response['Content-Type']}"}
418
+ Log.log.debug{"response: code=#{result_http.code}, mime=#{result_mime}, content-type=#{response['Content-Type']}"}
416
419
  # JSON data needs to be parsed, in case it contains an error code
417
420
  if !save_to_file.nil? &&
418
421
  result_http.code.to_s.start_with?('2') &&
@@ -453,12 +456,10 @@ module Aspera
453
456
  # TODO : related to mime type encoding ?
454
457
  # result_http.body.force_encoding('UTF-8') if result_http.body.is_a?(String)
455
458
  # Log.log.debug{"result: body=#{result_http.body}"}
456
- if Mime.json?(result_mime)
457
- result_data = JSON.parse(result_http.body) rescue result_http.body
458
- Log.dump(:result_data, result_data)
459
- else # Mime::TEXT
460
- result_data = result_http.body
461
- end
459
+ result_data = result_http.body
460
+ Log.dump(:result_data_raw, result_data, level: :trace1)
461
+ result_data = JSON.parse(result_data) if Mime.json?(result_mime) && !result_data.nil? && !result_data.empty?
462
+ Log.dump(:result_data, result_data)
462
463
  RestErrorAnalyzer.instance.raise_on_error(req, result_data, result_http)
463
464
  unless file_saved || save_to_file.nil?
464
465
  FileUtils.mkdir_p(File.dirname(save_to_file))
@@ -536,27 +537,39 @@ module Aspera
536
537
 
537
538
  # Create: `POST`
538
539
  def create(subpath, params, **kwargs)
539
- return call(operation: 'POST', subpath: subpath, headers: {'Accept' => Mime::JSON}, body: params, content_type: Mime::JSON, **kwargs)
540
+ kwargs[:headers] ||= {}
541
+ kwargs[:headers]['Accept'] = Mime::JSON unless kwargs[:headers].key?('Accept')
542
+ kwargs[:content_type] = Mime::JSON unless kwargs.key?(:content_type)
543
+ return call(operation: 'POST', subpath: subpath, body: params, **kwargs)
540
544
  end
541
545
 
542
546
  # Read: `GET`
543
547
  def read(subpath, query = nil, **kwargs)
544
- return call(operation: 'GET', subpath: subpath, headers: {'Accept' => Mime::JSON}, query: query, **kwargs)
548
+ kwargs[:headers] ||= {}
549
+ kwargs[:headers]['Accept'] = Mime::JSON unless kwargs[:headers].key?('Accept')
550
+ return call(operation: 'GET', subpath: subpath, query: query, **kwargs)
545
551
  end
546
552
 
547
553
  # Update: `PUT`
548
554
  def update(subpath, params, **kwargs)
549
- return call(operation: 'PUT', subpath: subpath, headers: {'Accept' => Mime::JSON}, body: params, content_type: Mime::JSON, **kwargs)
555
+ kwargs[:headers] ||= {}
556
+ kwargs[:headers]['Accept'] = Mime::JSON unless kwargs[:headers].key?('Accept')
557
+ kwargs[:content_type] = Mime::JSON unless kwargs.key?(:content_type)
558
+ return call(operation: 'PUT', subpath: subpath, body: params, **kwargs)
550
559
  end
551
560
 
552
561
  # Delete: `DELETE`
553
562
  def delete(subpath, params = nil, **kwargs)
554
- return call(operation: 'DELETE', subpath: subpath, headers: {'Accept' => Mime::JSON}, query: params, **kwargs)
563
+ kwargs[:headers] ||= {}
564
+ kwargs[:headers]['Accept'] = Mime::JSON unless kwargs[:headers].key?('Accept')
565
+ return call(operation: 'DELETE', subpath: subpath, query: params, **kwargs)
555
566
  end
556
567
 
557
568
  # Cancel: `CANCEL`
558
569
  def cancel(subpath, **kwargs)
559
- return call(operation: 'CANCEL', subpath: subpath, headers: {'Accept' => Mime::JSON}, **kwargs)
570
+ kwargs[:headers] ||= {}
571
+ kwargs[:headers]['Accept'] = Mime::JSON unless kwargs[:headers].key?('Accept')
572
+ return call(operation: 'CANCEL', subpath: subpath, **kwargs)
560
573
  end
561
574
 
562
575
  # Query entity by general search (read with parameter `q`)
@@ -25,6 +25,7 @@ module Aspera
25
25
  TRANSPORT_FIELDS = (%w[remote_host] + AK_TSPEC_BASE.keys + WSS_FIELDS).freeze
26
26
  # reserved tag for Aspera
27
27
  TAG_RESERVED = 'aspera'
28
+ SPECIFIC = %w{token paths direction source_root destination_root}.freeze
28
29
  class << self
29
30
  # wrong def in transferd
30
31
  POLICY_FIX = {
data.tar.gz.sig CHANGED
Binary file
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: aspera-cli
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.25.3
4
+ version: 4.25.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Laurent Martin
metadata.gz.sig CHANGED
Binary file