aspera-cli 4.27.2 → 4.27.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +133 -0
- data/CONTRIBUTING.md +5 -2
- data/bin/ascli +3 -1
- data/docs/README.md +1006 -766
- data/lib/aspera/agent/base.rb +7 -2
- data/lib/aspera/agent/connect.rb +6 -8
- data/lib/aspera/agent/desktop.rb +2 -6
- data/lib/aspera/agent/direct.rb +52 -22
- data/lib/aspera/agent/node.rb +9 -8
- data/lib/aspera/agent/transferd.rb +2 -2
- data/lib/aspera/api/alee.rb +1 -1
- data/lib/aspera/api/aoc.rb +14 -12
- data/lib/aspera/api/ats.rb +1 -1
- data/lib/aspera/api/cos_node.rb +2 -2
- data/lib/aspera/api/faspex.rb +11 -7
- data/lib/aspera/api/httpgw.rb +38 -35
- data/lib/aspera/api/node.rb +39 -34
- data/lib/aspera/ascmd.rb +3 -1
- data/lib/aspera/ascp/installation.rb +63 -28
- data/lib/aspera/ascp/management.rb +1 -0
- data/lib/aspera/assert.rb +4 -0
- data/lib/aspera/cli/ascp_actions.rb +20 -41
- data/lib/aspera/cli/async_transfer_store.rb +12 -11
- data/lib/aspera/cli/bootstrapper.rb +14 -16
- data/lib/aspera/cli/command_line.rb +252 -0
- data/lib/aspera/cli/command_registry.rb +215 -37
- data/lib/aspera/cli/command_spec.rb +104 -15
- data/lib/aspera/cli/completion/ascli.bash +12 -0
- data/lib/aspera/cli/completion/ascli.fish +16 -0
- data/lib/aspera/cli/completion/ascli.zsh +19 -0
- data/lib/aspera/cli/context.rb +3 -0
- data/lib/aspera/cli/deprecation.rb +37 -0
- data/lib/aspera/cli/extended_value.rb +6 -3
- data/lib/aspera/cli/formatter.rb +94 -80
- data/lib/aspera/cli/gem_checker.rb +1 -1
- data/lib/aspera/cli/hints.rb +7 -6
- data/lib/aspera/cli/http.rb +22 -34
- data/lib/aspera/cli/info.rb +3 -0
- data/lib/aspera/cli/mcp_tool.rb +47 -83
- data/lib/aspera/cli/option_declarator.rb +33 -42
- data/lib/aspera/cli/option_registry.rb +69 -0
- data/lib/aspera/cli/option_types.rb +105 -0
- data/lib/aspera/cli/option_value.rb +278 -0
- data/lib/aspera/cli/options.schema.yaml +124 -15
- data/lib/aspera/cli/parser.rb +333 -862
- data/lib/aspera/cli/plugins/alee.rb +7 -4
- data/lib/aspera/cli/plugins/aoc.rb +545 -518
- data/lib/aspera/cli/plugins/ats.rb +59 -80
- data/lib/aspera/cli/plugins/base.rb +221 -265
- data/lib/aspera/cli/plugins/basic_auth.rb +2 -10
- data/lib/aspera/cli/plugins/config.rb +263 -184
- data/lib/aspera/cli/plugins/console.rb +103 -39
- data/lib/aspera/cli/plugins/cos.rb +6 -23
- data/lib/aspera/cli/plugins/factory.rb +3 -0
- data/lib/aspera/cli/plugins/faspex5.rb +204 -182
- data/lib/aspera/cli/plugins/faspio.rb +6 -11
- data/lib/aspera/cli/plugins/httpgw.rb +8 -11
- data/lib/aspera/cli/plugins/mcp.rb +20 -55
- data/lib/aspera/cli/plugins/node.rb +300 -327
- data/lib/aspera/cli/plugins/orchestrator.rb +152 -110
- data/lib/aspera/cli/plugins/preview.rb +96 -105
- data/lib/aspera/cli/plugins/server.rb +78 -53
- data/lib/aspera/cli/plugins/shares.rb +80 -131
- data/lib/aspera/cli/preset_actions.rb +44 -27
- data/lib/aspera/cli/preset_manager.rb +44 -19
- data/lib/aspera/cli/prompt.rb +36 -0
- data/lib/aspera/cli/result.rb +42 -36
- data/lib/aspera/cli/runner.rb +32 -59
- data/lib/aspera/cli/special_values.rb +5 -0
- data/lib/aspera/cli/sync_actions.rb +51 -46
- data/lib/aspera/cli/terminal_formatter.rb +9 -3
- data/lib/aspera/cli/transfer_actions.rb +14 -9
- data/lib/aspera/cli/transfer_agent.rb +34 -38
- data/lib/aspera/cli/transfer_progress.rb +290 -55
- data/lib/aspera/cli/vault_manager.rb +0 -17
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/cli/wizard.rb +5 -3
- data/lib/aspera/coverage.rb +1 -1
- data/lib/aspera/environment.rb +35 -5
- data/lib/aspera/faspex_gw.rb +2 -1
- data/lib/aspera/faspex_postproc.rb +1 -0
- data/lib/aspera/graphql.rb +5 -5
- data/lib/aspera/json_rpc/client.rb +5 -5
- data/lib/aspera/keychain/encrypted_hash.rb +2 -2
- data/lib/aspera/keychain/factory.rb +2 -1
- data/lib/aspera/keychain/one_password_api.rb +1 -1
- data/lib/aspera/link_header.rb +2 -2
- data/lib/aspera/log.rb +47 -27
- data/lib/aspera/markdown.rb +2 -0
- data/lib/aspera/mime.rb +25 -0
- data/lib/aspera/node_emulator.rb +759 -0
- data/lib/aspera/oauth/base.rb +37 -26
- data/lib/aspera/oauth/factory.rb +7 -3
- data/lib/aspera/oauth/generic.rb +1 -1
- data/lib/aspera/oauth/json_credentials.rb +34 -0
- data/lib/aspera/oauth/jwt.rb +4 -5
- data/lib/aspera/oauth/web.rb +9 -8
- data/lib/aspera/oauth.rb +1 -0
- data/lib/aspera/persistency_folder.rb +1 -3
- data/lib/aspera/preview/file_types.rb +4 -4
- data/lib/aspera/preview/generator.rb +11 -1
- data/lib/aspera/preview/options.schema.yaml +119 -0
- data/lib/aspera/preview/terminal.rb +4 -3
- data/lib/aspera/preview/utils.rb +9 -6
- data/lib/aspera/products/connect.rb +1 -1
- data/lib/aspera/rainbow.rb +7 -0
- data/lib/aspera/rest/aspera_errors.rb +72 -0
- data/lib/aspera/rest/call_error.rb +27 -0
- data/lib/aspera/rest/client.rb +523 -0
- data/lib/aspera/rest/error_analyzer.rb +113 -0
- data/lib/aspera/rest/list.rb +149 -0
- data/lib/aspera/rest/parameters.rb +55 -0
- data/lib/aspera/rest/util.rb +176 -0
- data/lib/aspera/rest.rb +7 -621
- data/lib/aspera/schema/IBM Aspera Console-enhanced.yaml +1125 -0
- data/lib/aspera/schema/IBM Aspera Faspex API-5.0-enhanced.yaml +0 -20
- data/lib/aspera/schema/IBM Aspera Orchestrator API-v1.yaml +1784 -0
- data/lib/aspera/schema/IBM Aspera on Cloud API-0.2.6-enhanced.yaml +1230 -137
- data/lib/aspera/schema/IBM Aspera on Cloud Automation API-1.0.5-enhanced.yaml +2395 -0
- data/lib/aspera/schema/IBM_Aspera_Shares.yaml +14 -13
- data/lib/aspera/schema/documentation.rb +13 -3
- data/lib/aspera/schema/reader.rb +12 -18
- data/lib/aspera/schema/registry.rb +23 -1
- data/lib/aspera/schema/validator.rb +92 -0
- data/lib/aspera/secret_hider.rb +36 -25
- data/lib/aspera/string_ext.rb +15 -0
- data/lib/aspera/temp_file_manager.rb +6 -5
- data/lib/aspera/transfer/parameters.rb +2 -0
- data/lib/aspera/transfer/spec.rb +1 -0
- data/lib/aspera/uri_reader.rb +11 -11
- data/lib/aspera/web_auth/index.html +147 -0
- data/lib/aspera/web_auth/server.rb +81 -0
- data.tar.gz.sig +0 -0
- metadata +43 -9
- metadata.gz.sig +0 -0
- data/lib/aspera/colors.rb +0 -79
- data/lib/aspera/node_simulator.rb +0 -344
- data/lib/aspera/preview/options.rb +0 -45
- data/lib/aspera/rest_call_error.rb +0 -25
- data/lib/aspera/rest_error_analyzer.rb +0 -111
- data/lib/aspera/rest_errors_aspera.rb +0 -58
- data/lib/aspera/rest_list.rb +0 -136
- data/lib/aspera/web_auth.rb +0 -211
data/docs/README.md
CHANGED
|
@@ -11,7 +11,7 @@ EDITING GUIDELINES (developers and AI):
|
|
|
11
11
|
DO NOT EDIT: THIS FILE IS GENERATED, edit docs/README.erb.md.
|
|
12
12
|
PANDOC_DEFAULTS_BEGIN
|
|
13
13
|
metadata:
|
|
14
|
-
subtitle: "ascli 4.27.
|
|
14
|
+
subtitle: "ascli 4.27.4"
|
|
15
15
|
author: "Laurent Martin"
|
|
16
16
|
PANDOC_DEFAULTS_END
|
|
17
17
|
-->
|
|
@@ -62,10 +62,11 @@ This manual is organized into the following sections:
|
|
|
62
62
|
|
|
63
63
|
1. **Quick Start** - Getting started with basic operations
|
|
64
64
|
1. **Installation** - Setup procedures for various platforms
|
|
65
|
-
1. **Command Line Interface** - Syntax, options, and usage patterns
|
|
66
|
-
1. **
|
|
67
|
-
1. **
|
|
68
|
-
1. **
|
|
65
|
+
1. **Command Line Interface** - Syntax, options, configuration, transfers, and usage patterns
|
|
66
|
+
1. **Plugin sections** (one per product: AoC, ATS, HSTS, Faspex 5, and so on) - Product-specific operations and examples
|
|
67
|
+
1. **Operational Utilities** - Sync, hot folders, health checks, email notifications
|
|
68
|
+
1. **Common problems** - Common issues and solutions
|
|
69
|
+
1. **About** - History and references
|
|
69
70
|
|
|
70
71
|
### When to use and when not to use
|
|
71
72
|
|
|
@@ -81,7 +82,7 @@ Internally, `ascli` integrates several components:
|
|
|
81
82
|
- A configuration file (`config.yaml`) for persistent settings
|
|
82
83
|
- Advanced command-line options (see [Extended Value](#extended-value-syntax))
|
|
83
84
|
- REST API calls, including OAuth (like `curl`)
|
|
84
|
-
- Aspera
|
|
85
|
+
- Aspera's `ascp` for high-speed file transfers
|
|
85
86
|
|
|
86
87
|
For programmatic integration with languages such as C/C++, Go, Python, NodeJS, and others, it is recommended to use the [Aspera APIs](https://ibm.biz/aspera_api) directly.
|
|
87
88
|
These include:
|
|
@@ -120,9 +121,9 @@ Using [Windows PowerShell or cmd](#shell-parsing-for-windows) is also possible.
|
|
|
120
121
|
|
|
121
122
|
Command line examples listed in sections titled **Tested commands for `_plugin_name_`** are verified during version validation.
|
|
122
123
|
|
|
123
|
-
Command line arguments formatted as `<NAME>` in examples represent user-provided values, not
|
|
124
|
+
Command line arguments formatted as `<NAME>` in examples represent user-provided values, not literal values.
|
|
124
125
|
|
|
125
|
-
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, and so on)
|
|
126
|
+
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, and so on).
|
|
126
127
|
|
|
127
128
|
Some commands will start an Aspera transfer (for example, `upload`).
|
|
128
129
|
The transfer is not implemented directly in `ascli`; rather, `ascli` uses one of the external Aspera Transfer Clients called **[Transfer Agents](#transfer-clients-agents)**.
|
|
@@ -130,8 +131,7 @@ The transfer is not implemented directly in `ascli`; rather, `ascli` uses one of
|
|
|
130
131
|
> [!NOTE]
|
|
131
132
|
> A **[Transfer Agent](#transfer-clients-agents)** is a client for the remote Transfer Server (HSTS/HSTE).
|
|
132
133
|
> It can be local, or remote.
|
|
133
|
-
> For example a remote Aspera Transfer Server may be used as a transfer agent
|
|
134
|
-
> that is, using the option `--transfer=node`
|
|
134
|
+
> For example, a remote Aspera Transfer Server may be used as a transfer agent through its Node API, using option `--transfer=node`.
|
|
135
135
|
|
|
136
136
|
## Quick Start
|
|
137
137
|
|
|
@@ -145,8 +145,7 @@ This section walks you through your first interaction with `ascli` on Linux.
|
|
|
145
145
|
|
|
146
146
|
```shell
|
|
147
147
|
mkdir -p $HOME/bin
|
|
148
|
-
tar zxvf ascli
|
|
149
|
-
mv ascli.4.27.2.linux-x86_64 $HOME/bin/ascli
|
|
148
|
+
tar -C $HOME/bin -zxvf ascli-4.27.4-linux-x86_64.tgz
|
|
150
149
|
export PATH=$PATH:$HOME/bin
|
|
151
150
|
```
|
|
152
151
|
|
|
@@ -160,7 +159,7 @@ ascli -v
|
|
|
160
159
|
```
|
|
161
160
|
|
|
162
161
|
```text
|
|
163
|
-
4.27.
|
|
162
|
+
4.27.4
|
|
164
163
|
```
|
|
165
164
|
|
|
166
165
|
- Install the Aspera transfer runtime (tested version), as it is not included in the `ascli` package:
|
|
@@ -211,7 +210,7 @@ Time: 00:00:02 ====================================== 100% 100 Mbps Time: 00:00:
|
|
|
211
210
|
complete
|
|
212
211
|
```
|
|
213
212
|
|
|
214
|
-
### Option B -
|
|
213
|
+
### Option B - Test with Your Own HSTS
|
|
215
214
|
|
|
216
215
|
To use `ascli` with a server of your own, it is recommended to save its connection details as an [Option Preset](#option-preset).
|
|
217
216
|
This avoids repeating credentials on every command.
|
|
@@ -220,12 +219,12 @@ The steps below create a preset, set it as the default for the server plugin, br
|
|
|
220
219
|
- Create a preset with your server's connection details:
|
|
221
220
|
|
|
222
221
|
```shell
|
|
223
|
-
ascli config preset update <SERVER_PRESET_NAME> --url=ssh://
|
|
222
|
+
ascli config preset update <SERVER_PRESET_NAME> --url=ssh://hsts.example.com:33001 --username=<USERNAME> --password=<PASSWORD>
|
|
224
223
|
```
|
|
225
224
|
|
|
226
225
|
```text
|
|
226
|
+
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
|
|
227
227
|
Updated: <SERVER_PRESET_NAME>
|
|
228
|
-
Saving config file.
|
|
229
228
|
```
|
|
230
229
|
|
|
231
230
|
- Set the preset as the default for the server plugin:
|
|
@@ -235,8 +234,8 @@ ascli config preset set default server <SERVER_PRESET_NAME>
|
|
|
235
234
|
```
|
|
236
235
|
|
|
237
236
|
```text
|
|
238
|
-
Updated: default: server <- <SERVER_PRESET_NAME>
|
|
239
|
-
Saving config file.
|
|
237
|
+
INFO Updated: default: server <- <SERVER_PRESET_NAME>
|
|
238
|
+
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
|
|
240
239
|
```
|
|
241
240
|
|
|
242
241
|
- Once your preset is set, follow the same browse and download steps as in [Option A](#option-a---test-with-the-aspera-demo-server).
|
|
@@ -266,7 +265,7 @@ Recommended Workflow:
|
|
|
266
265
|
Example Prompt:
|
|
267
266
|
|
|
268
267
|
```text
|
|
269
|
-
|
|
268
|
+
Use only the attached ascli manual (it matches the version I use).
|
|
270
269
|
Generate a command to send a package via Aspera on Cloud using the Bash shell.
|
|
271
270
|
Set a custom title and note.
|
|
272
271
|
Define specific recipients.
|
|
@@ -279,7 +278,7 @@ By providing the documentation as a direct reference, you reduce "hallucinations
|
|
|
279
278
|
|
|
280
279
|
- Learn the CLI: Read [Command Line Interface](#command-line-interface) to understand configuration, options, and commands.
|
|
281
280
|
|
|
282
|
-
- Explore plugins: Jump to the section for the relevant product
|
|
281
|
+
- Explore plugins: Jump to the section for the relevant product, for example: [Aspera on Cloud](#plugin-aoc-ibm-aspera-on-cloud), [HSTS](#plugin-server-ibm-aspera-high-speed-transfer-server-ssh), [Faspex 5](#plugin-faspex5-ibm-aspera-faspex-v5).
|
|
283
282
|
|
|
284
283
|
## Installation
|
|
285
284
|
|
|
@@ -295,6 +294,13 @@ There are several ways to install `ascli`:
|
|
|
295
294
|
- As a [single file executable](#single-file-executable)
|
|
296
295
|
|
|
297
296
|
This method is simple, but only a limited number of platforms are supported.
|
|
297
|
+
- On Linux, as a [portable package](#linux-portable-package)
|
|
298
|
+
|
|
299
|
+
This method is simple on Linux: extract and run, without root access. It includes Ruby, gems and `ascp`.
|
|
300
|
+
- On Windows, as a [portable package](#windows-portable-package)
|
|
301
|
+
|
|
302
|
+
This method is the simplest on Windows: extract and run. It includes Ruby, gems and `ascp`.
|
|
303
|
+
- On Windows, with the [Chocolatey package](#windows-chocolatey-package) (installs Ruby and the gem).
|
|
298
304
|
- As a [container](#container) (`docker`, `podman`, `singularity`).
|
|
299
305
|
|
|
300
306
|
The following sections describe the various installation methods.
|
|
@@ -313,21 +319,20 @@ This executable includes the Ruby runtime and gems, but not the transfer SDK.
|
|
|
313
319
|
#### Installing the single file executable
|
|
314
320
|
|
|
315
321
|
> [!NOTE]
|
|
316
|
-
> Replace
|
|
322
|
+
> Replace `<VERSION>` and `<PLATFORM>` with the values of the downloaded archive, for example: `linux-x86_64-glibc2.28-ocran`.
|
|
323
|
+
> The archive contains the executable `ascli` and its `README.ascli.md`.
|
|
317
324
|
> Installation of `ascp` is still required separately.
|
|
318
325
|
> See [Install `ascp`](#installing-ascp-through-transferd).
|
|
319
326
|
|
|
320
327
|
```shell
|
|
321
|
-
tar zxvf
|
|
322
|
-
mv ascli-<VERSION>-<PLATFORM> ascli
|
|
323
|
-
chmod a+x ascli
|
|
328
|
+
tar zxvf aspera-cli-<VERSION>-<PLATFORM>.tgz
|
|
324
329
|
./ascli config transferd install
|
|
325
330
|
```
|
|
326
331
|
|
|
327
332
|
#### Linux: Checking the GLIBC version
|
|
328
333
|
|
|
329
334
|
> [!WARNING]
|
|
330
|
-
> On Linux, the executable requires a minimum GLIBC version, specified in the
|
|
335
|
+
> On Linux, the executable requires a minimum GLIBC version, specified in the archive name on the download site (for example: `glibc2.28`).
|
|
331
336
|
> If the minimum version is not met, then executables (`ascp`, `transferd`) will exit with error.
|
|
332
337
|
|
|
333
338
|
On Linux, you can check your system's GLIBC version on this site: [repology.org](https://repology.org/project/glibc/versions), or check your GLIBC version with `ldd`:
|
|
@@ -353,11 +358,73 @@ objdump -p /bin/bash | sed -n 's/^.*GLIBC_//p' | sort -V | tail -n1
|
|
|
353
358
|
> [!NOTE]
|
|
354
359
|
> If `objdump` is not available, then use `strings` or `grep -z 'GLIBC_'|tr \\0 \\n`
|
|
355
360
|
|
|
356
|
-
The required GLIBC version for `ascp` can be found in the [Release Notes of HSTS](https://www.ibm.com/docs/en/ahts) or [
|
|
361
|
+
The required GLIBC version for `ascp` can be found in the [Release Notes of HSTS](https://www.ibm.com/docs/en/ahts) or [on this page](https://eudemo.asperademo.com/download/sdk.html).
|
|
362
|
+
|
|
363
|
+
### Linux: Portable package
|
|
364
|
+
|
|
365
|
+
A ready-to-use archive for Linux (x86_64) is available in the [Releases](https://github.com/IBM/aspera-cli/releases): `aspera-cli-<VERSION>-linux-x86_64-glibc2.28-portable.tgz`.
|
|
366
|
+
|
|
367
|
+
It contains the Ruby runtime with the shared libraries it needs, the aspera-cli gem with its dependencies, and the Aspera Transfer SDK (`ascp`).
|
|
368
|
+
No installation step, no root access, and no internet access are required.
|
|
369
|
+
The GLIBC of the system must be at least the version in the archive name, see [Checking the GLIBC version](#linux-checking-the-glibc-version).
|
|
370
|
+
|
|
371
|
+
1. Extract the archive, for example in `~/.local/share`, and check that `ascli` runs:
|
|
372
|
+
|
|
373
|
+
```shell
|
|
374
|
+
tar -xzf aspera-cli-<VERSION>-linux-x86_64-glibc2.28-portable.tgz -C ~/.local/share
|
|
375
|
+
~/.local/share/aspera-cli-<VERSION>-linux-x86_64-glibc2.28-portable/ascli -v
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
2. Optionally, place a symbolic link to the launcher in a folder of the `PATH`.
|
|
379
|
+
Then, `ascli` can be used from any folder:
|
|
380
|
+
|
|
381
|
+
```shell
|
|
382
|
+
ln -s ~/.local/share/aspera-cli-<VERSION>-linux-x86_64-glibc2.28-portable/ascli ~/.local/bin/ascli
|
|
383
|
+
ascli -v
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
> [!NOTE]
|
|
387
|
+
> The launcher `ascli` uses the `ascp` located in folder `sdk` of the package, unless environment variable `ASCLI_SDK_FOLDER` is set.
|
|
388
|
+
> So, `ascli config transferd install` is not needed.
|
|
389
|
+
|
|
390
|
+
The configuration is stored in the [main folder](#main-configuration-and-persistency-folder), like for other installation methods.
|
|
391
|
+
To upgrade, extract the new version, and update the symbolic link.
|
|
392
|
+
To uninstall, delete the folder, and the symbolic link if it was created.
|
|
393
|
+
|
|
394
|
+
### Windows: Portable package
|
|
395
|
+
|
|
396
|
+
A ready-to-use ZIP archive for Windows (x64) is available in the [Releases](https://github.com/IBM/aspera-cli/releases): `aspera-cli-<VERSION>-windows-x86_64-portable.zip`.
|
|
397
|
+
|
|
398
|
+
It contains the Ruby runtime, the aspera-cli gem with its dependencies, and the Aspera Transfer SDK (`ascp`).
|
|
399
|
+
No installation step, no administrator rights, and no internet access are required.
|
|
400
|
+
|
|
401
|
+
1. Download the ZIP archive, then right-click on it and select **Extract All...**.
|
|
402
|
+
Preferably, extract in a folder writable by the user, for example: `%LOCALAPPDATA%\Programs`.
|
|
403
|
+
|
|
404
|
+
2. In a terminal, in the extracted folder, check that `ascli` runs:
|
|
405
|
+
|
|
406
|
+
```batchfile
|
|
407
|
+
.\ascli.cmd -v
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
3. Optionally, double-click on `add_to_path.cmd` to add the folder to the user's `PATH`.
|
|
411
|
+
Then, in a new terminal, `ascli` can be used from any folder:
|
|
412
|
+
|
|
413
|
+
```batchfile
|
|
414
|
+
ascli -v
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
> [!NOTE]
|
|
418
|
+
> The launcher `ascli.cmd` uses the `ascp` located in folder `sdk` of the package, unless environment variable `ASCLI_SDK_FOLDER` is set.
|
|
419
|
+
> So, `ascli config transferd install` is not needed.
|
|
420
|
+
|
|
421
|
+
The configuration is stored in the [main folder](#main-configuration-and-persistency-folder), like for other installation methods.
|
|
422
|
+
To upgrade, extract the new version, and update the `PATH` if the folder name changed.
|
|
423
|
+
To uninstall, delete the folder, and remove it from the `PATH` if it was added.
|
|
357
424
|
|
|
358
|
-
|
|
425
|
+
### Windows: Chocolatey package
|
|
359
426
|
|
|
360
|
-
`ascli` can be
|
|
427
|
+
If you use [Chocolatey](https://chocolatey.org/), `ascli` can be installed with the `aspera-cli` package: it installs Ruby (if not already present) and the `aspera-cli` gem.
|
|
361
428
|
|
|
362
429
|
In a PowerShell as Administrator:
|
|
363
430
|
|
|
@@ -365,6 +432,8 @@ In a PowerShell as Administrator:
|
|
|
365
432
|
choco install aspera-cli -y
|
|
366
433
|
```
|
|
367
434
|
|
|
435
|
+
Then, install the Aspera Transfer Daemon, see [Installing `ascp` through `transferd`](#installing-ascp-through-transferd).
|
|
436
|
+
|
|
368
437
|
### Ruby
|
|
369
438
|
|
|
370
439
|
A Ruby interpreter is required to run `ascli`.
|
|
@@ -376,7 +445,7 @@ Required Ruby version is version: >= 3.1.
|
|
|
376
445
|
|
|
377
446
|
**Ruby can be installed using any of the following methods**: `rpm`, `yum`, `dnf`, `rvm`, `rbenv`, `brew`, Windows installer, ...
|
|
378
447
|
|
|
379
|
-
**
|
|
448
|
+
**First**, refer to the official Ruby documentation:
|
|
380
449
|
|
|
381
450
|
- [Official Ruby Installation Guide](https://www.ruby-lang.org/en/documentation/installation/)
|
|
382
451
|
- [Official Ruby Download](https://www.ruby-lang.org/en/downloads/)
|
|
@@ -398,7 +467,7 @@ Manual installation:
|
|
|
398
467
|
|
|
399
468
|
Automated installation (with internet access):
|
|
400
469
|
|
|
401
|
-
The Ruby installer supports silent installation
|
|
470
|
+
The Ruby installer supports silent installation. To see the options, execute it with `/help`, or refer to the [Ruby Installer FAQ](https://github.com/oneclick/rubyinstaller2/wiki/FAQ).
|
|
402
471
|
|
|
403
472
|
Download the Ruby installer executable from <https://rubyinstaller.org/downloads/> and then install:
|
|
404
473
|
|
|
@@ -437,14 +506,14 @@ This installs a recent Ruby version suitable for `ascli`.
|
|
|
437
506
|
After installation, Homebrew's Ruby is not on the `PATH` by default (this is shown at the end of the `brew install ruby` output and by `brew info ruby`).
|
|
438
507
|
Add it to your shell configuration file:
|
|
439
508
|
|
|
440
|
-
- **zsh** (default shell on macOS
|
|
509
|
+
- **zsh** (default shell on macOS; run this once in a terminal):
|
|
441
510
|
|
|
442
511
|
```shell
|
|
443
512
|
echo 'PATH="$(brew --prefix ruby)/bin:$($(brew --prefix ruby)/bin/gem env gemdir)/bin:$PATH"' >> ~/.zprofile
|
|
444
513
|
source ~/.zprofile
|
|
445
514
|
```
|
|
446
515
|
|
|
447
|
-
- **bash
|
|
516
|
+
- **bash**: replace `~/.zprofile` with `~/.bash_profile` in the commands above.
|
|
448
517
|
|
|
449
518
|
#### Linux: Package
|
|
450
519
|
|
|
@@ -595,7 +664,7 @@ For example for AIX, one can look at:
|
|
|
595
664
|
|
|
596
665
|
If your Unix does not provide a pre-built Ruby, you can get it using one of those [methods](https://www.ruby-lang.org/en/documentation/installation/).
|
|
597
666
|
|
|
598
|
-
For instance to build from source and install in `/opt/ruby
|
|
667
|
+
For instance, to build from source and install in `/opt/ruby`:
|
|
599
668
|
|
|
600
669
|
```shell
|
|
601
670
|
wget https://cache.ruby-lang.org/pub/ruby/x.y/ruby-x.y.z.tar.gz
|
|
@@ -620,7 +689,7 @@ make install
|
|
|
620
689
|
`ascli` can also run with the [JRuby](https://www.jruby.org/) interpreter.
|
|
621
690
|
All that is needed is a JVM (Java Virtual Machine) on your system (`java`).
|
|
622
691
|
The JRuby package comes pre-compiled and does not require compilation of native extensions.
|
|
623
|
-
Use a version of JRuby compatible with Ruby version supported by `ascli`.
|
|
692
|
+
Use a version of JRuby compatible with a Ruby version supported by `ascli`.
|
|
624
693
|
See [the Wikipedia page](https://en.wikipedia.org/wiki/JRuby) to match JRuby and Ruby versions.
|
|
625
694
|
Choose the latest version from:
|
|
626
695
|
|
|
@@ -651,7 +720,7 @@ JRUBY_OPTS=--dev ascli -v
|
|
|
651
720
|
#### Installing optional gems
|
|
652
721
|
|
|
653
722
|
Some additional gems are required for either development or specific runtime features.
|
|
654
|
-
For JRuby, some
|
|
723
|
+
For JRuby, some of them have a replacement gem, and others are not available.
|
|
655
724
|
Those are not installed as part of dependencies because they involve compilation of native code but concern less-used features.
|
|
656
725
|
|
|
657
726
|
See [Gemfile](../Gemfile):
|
|
@@ -752,7 +821,7 @@ gem install -P MediumSecurity aspera-cli
|
|
|
752
821
|
|
|
753
822
|
#### Installing a beta release of the gem
|
|
754
823
|
|
|
755
|
-
|
|
824
|
+
A beta version of the gem can be found here: <https://ibm.biz/aspera-cli-beta>
|
|
756
825
|
|
|
757
826
|
On Linux/macOS, install in a terminal:
|
|
758
827
|
|
|
@@ -761,7 +830,7 @@ curl -sLo aspera-cli-beta.gem https://ibm.biz/aspera-cli-beta
|
|
|
761
830
|
gem install aspera-cli-beta.gem
|
|
762
831
|
```
|
|
763
832
|
|
|
764
|
-
On Windows, download the
|
|
833
|
+
On Windows, download the file `aspera-cli-beta.gem` from the link, then install it with `gem install aspera-cli-beta.gem`.
|
|
765
834
|
|
|
766
835
|
### FASP Protocol: `ascp`
|
|
767
836
|
|
|
@@ -802,7 +871,7 @@ The installation of the transfer binaries follows those steps:
|
|
|
802
871
|
| `locations_url` | `https://ibm.biz/sdk_location` | URL to get download URLs of Aspera Transfer Daemon from IBM official repository. |
|
|
803
872
|
| `sdk_folder` | `$HOME/.aspera/sdk` | Folder where the SDK archive is extracted. |
|
|
804
873
|
|
|
805
|
-
|
|
874
|
+
Transfer Daemon versions available from `locations_url` can be listed with: `ascli config transferd list`
|
|
806
875
|
|
|
807
876
|
To install a specific version, for example, 1.1.3:
|
|
808
877
|
|
|
@@ -822,7 +891,7 @@ To download it, pipe to `config download`:
|
|
|
822
891
|
ascli config transferd list --select.platform=osx-arm64 --select.version=1.1.3 --fields=url | ascli config download @stdin:
|
|
823
892
|
```
|
|
824
893
|
|
|
825
|
-
|
|
894
|
+
To install from a local file (air-gapped installation) instead of fetching from the internet, specify the location of the SDK file with option `sdk_url`:
|
|
826
895
|
|
|
827
896
|
```shell
|
|
828
897
|
ascli config transferd install --sdk-url=file:///macos-arm64-1.1.3-c6c7a2a.zip
|
|
@@ -850,12 +919,12 @@ If the embedded method is not used, the following packages are also suitable:
|
|
|
850
919
|
For instance, Aspera Connect Client can be installed by visiting the page:
|
|
851
920
|
[https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
|
|
852
921
|
|
|
853
|
-
`ascli`
|
|
922
|
+
`ascli` detects most Aspera transfer products in standard locations and use the first one found by default.
|
|
854
923
|
See [FASP](#fasp-configuration) for details on how to select a client or set path to the FASP protocol.
|
|
855
924
|
|
|
856
925
|
Several methods are provided to start a transfer.
|
|
857
926
|
Use of a local client ([`direct`](#agent-direct) transfer agent) is one of them, but other methods are available.
|
|
858
|
-
See [Transfer Agents](#transfer-clients-agents)
|
|
927
|
+
See [Transfer Agents](#transfer-clients-agents).
|
|
859
928
|
|
|
860
929
|
### Installing in an air-gapped environment
|
|
861
930
|
|
|
@@ -898,11 +967,11 @@ Alternatively, the necessary gems can be packaged into a `tar.gz` archive as fol
|
|
|
898
967
|
|
|
899
968
|
```shell
|
|
900
969
|
mkdir temp_folder
|
|
901
|
-
gem install aspera-cli:4.27.
|
|
970
|
+
gem install aspera-cli:4.27.4 --no-document --install-dir temp_folder
|
|
902
971
|
find temp_folder
|
|
903
|
-
mv temp_folder/cache aspera-cli-4.27.
|
|
972
|
+
mv temp_folder/cache aspera-cli-4.27.4-gems
|
|
904
973
|
rm -fr temp_folder
|
|
905
|
-
tar zcvf aspera-cli-4.27.
|
|
974
|
+
tar zcvf aspera-cli-4.27.4-gems.tgz aspera-cli-4.27.4-gems
|
|
906
975
|
```
|
|
907
976
|
|
|
908
977
|
#### Unix-like: Alternative installation using `rvm`
|
|
@@ -938,6 +1007,9 @@ The following procedure applies when using RVM for the Ruby installation:
|
|
|
938
1007
|
|
|
939
1008
|
#### Windows: Installing in an air-gapped environment
|
|
940
1009
|
|
|
1010
|
+
> [!TIP]
|
|
1011
|
+
> The simplest method is to use the [portable package](#windows-portable-package), which requires no internet access on the target system.
|
|
1012
|
+
|
|
941
1013
|
The procedure is similar to the internet-connected Windows installation. Copy the required files from a system with internet access, then install them on the target system.
|
|
942
1014
|
|
|
943
1015
|
1. Download the Ruby installer from <https://rubyinstaller.org/downloads/>:
|
|
@@ -999,7 +1071,7 @@ podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/as
|
|
|
999
1071
|
Then, execute individual `ascli` commands such as:
|
|
1000
1072
|
|
|
1001
1073
|
```shell
|
|
1002
|
-
ascli config
|
|
1074
|
+
ascli config initdemo
|
|
1003
1075
|
ascli config preset overview
|
|
1004
1076
|
ascli config ascp info
|
|
1005
1077
|
ascli server ls /
|
|
@@ -1007,10 +1079,9 @@ ascli server ls /
|
|
|
1007
1079
|
|
|
1008
1080
|
That is simple, but there are limitations:
|
|
1009
1081
|
|
|
1010
|
-
- Everything happens in the container
|
|
1011
|
-
- Any generated
|
|
1012
|
-
|
|
1013
|
-
- No possibility to upload files located on the host system
|
|
1082
|
+
- Everything happens in the container.
|
|
1083
|
+
- Any file generated in the container, including configuration files and downloaded files, is lost when the container (shell) exits.
|
|
1084
|
+
- Files located on the host system cannot be uploaded.
|
|
1014
1085
|
|
|
1015
1086
|
#### Container: Details
|
|
1016
1087
|
|
|
@@ -1036,10 +1107,10 @@ ascli -v
|
|
|
1036
1107
|
```
|
|
1037
1108
|
|
|
1038
1109
|
```text
|
|
1039
|
-
4.27.
|
|
1110
|
+
4.27.4
|
|
1040
1111
|
```
|
|
1041
1112
|
|
|
1042
|
-
To
|
|
1113
|
+
To persist the configuration on the host, specify your user's configuration folder as a volume for the container.
|
|
1043
1114
|
To enable write access, a possibility is to run as `root` in the container (and set the default configuration folder to `/home/cliuser/.aspera/ascli`).
|
|
1044
1115
|
Add options:
|
|
1045
1116
|
|
|
@@ -1061,9 +1132,9 @@ As shown in the quick start, if you prefer to keep a running container with a sh
|
|
|
1061
1132
|
> [!WARNING]
|
|
1062
1133
|
> `ascli` is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
|
1063
1134
|
|
|
1064
|
-
You may also
|
|
1065
|
-
For example, files transferred with `ascli` through folder `/xferfiles` (right
|
|
1066
|
-
In this case you also need to specify the shared transfer folder as a volume:
|
|
1135
|
+
You may also want files downloaded in the container to be available on the host.
|
|
1136
|
+
For example, files transferred with `ascli` through folder `/xferfiles` (right-hand side) would be available on the host in `$HOME/xferdir`.
|
|
1137
|
+
In this case, you also need to specify the shared transfer folder as a volume:
|
|
1067
1138
|
|
|
1068
1139
|
```shell
|
|
1069
1140
|
--volume $HOME/xferdir:/xferfiles
|
|
@@ -1085,12 +1156,7 @@ asclish
|
|
|
1085
1156
|
|
|
1086
1157
|
#### Container: Sample start script
|
|
1087
1158
|
|
|
1088
|
-
A convenience sample script is also provided: download the script [`dascli`](../container/dascli) from [the
|
|
1089
|
-
|
|
1090
|
-
> [!NOTE]
|
|
1091
|
-
> If you have installed `ascli`, the script `dascli` can also be found like this:
|
|
1092
|
-
>
|
|
1093
|
-
> `cp $(ascli config gem path)/../container/dascli ascli`
|
|
1159
|
+
A convenience sample script is also provided: download the script [`dascli`](../build/container/dascli) from [the GitHub repository](https://raw.githubusercontent.com/IBM/aspera-cli/main/build/container/dascli).
|
|
1094
1160
|
|
|
1095
1161
|
Some environment variables can be set for this script to adapt its behavior:
|
|
1096
1162
|
|
|
@@ -1101,34 +1167,33 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
|
1101
1167
|
| `image` | Container image name | `docker.io/martinlaurent/ascli` | n/a |
|
|
1102
1168
|
| `version` | Container image version | Latest | `4.8.0.pre` |
|
|
1103
1169
|
|
|
1104
|
-
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
|
1105
|
-
|
|
1106
|
-
This allows having persistent configuration on the host.
|
|
1170
|
+
The wrapping script maps the folder `$ASCLI_HOME` on the host to `/home/cliuser/.aspera/ascli` in the container (the configuration folder expected in the container).
|
|
1171
|
+
This keeps the configuration persistent on the host.
|
|
1107
1172
|
|
|
1108
1173
|
To add local storage as a volume, you can use the env var `docker_args`:
|
|
1109
1174
|
|
|
1110
1175
|
Example of use:
|
|
1111
1176
|
|
|
1112
1177
|
```shell
|
|
1113
|
-
curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/container/dascli
|
|
1178
|
+
curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/build/container/dascli
|
|
1114
1179
|
chmod a+x ascli
|
|
1115
1180
|
export xferdir=$HOME/xferdir
|
|
1116
1181
|
mkdir -p $xferdir
|
|
1117
1182
|
chmod -R 777 $xferdir
|
|
1118
1183
|
export docker_args="--volume $xferdir:/xferfiles"
|
|
1119
1184
|
|
|
1120
|
-
./ascli config
|
|
1185
|
+
./ascli config initdemo
|
|
1121
1186
|
|
|
1122
1187
|
echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
1123
1188
|
./ascli server upload /xferfiles/samplefile.txt --to-folder=/Upload
|
|
1124
1189
|
```
|
|
1125
1190
|
|
|
1126
1191
|
> [!NOTE]
|
|
1127
|
-
> The local file
|
|
1192
|
+
> The local file path is the one seen from the container (`/xferfiles/samplefile.txt`), where `/xferfiles` is mapped to the host folder `$HOME/xferdir`.
|
|
1128
1193
|
|
|
1129
1194
|
> [!WARNING]
|
|
1130
1195
|
> Do not use too many volumes, as the legacy `aufs` driver limits their number.
|
|
1131
|
-
>
|
|
1196
|
+
> Prefer the `overlay2` driver.
|
|
1132
1197
|
|
|
1133
1198
|
#### Container: Installing in an air-gapped environment
|
|
1134
1199
|
|
|
@@ -1139,7 +1204,7 @@ podman pull docker.io/martinlaurent/ascli
|
|
|
1139
1204
|
podman save docker.io/martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
|
1140
1205
|
```
|
|
1141
1206
|
|
|
1142
|
-
- Then, on air-gapped system:
|
|
1207
|
+
- Then, on the air-gapped system:
|
|
1143
1208
|
|
|
1144
1209
|
```shell
|
|
1145
1210
|
podman load -i ascli_image_latest.tar.gz
|
|
@@ -1148,8 +1213,8 @@ podman load -i ascli_image_latest.tar.gz
|
|
|
1148
1213
|
#### Container: `aspera.conf`
|
|
1149
1214
|
|
|
1150
1215
|
`ascp`'s configuration file `aspera.conf` is located in the container at: `/ibm_aspera/aspera.conf` (see Dockerfile).
|
|
1151
|
-
As the container is immutable,
|
|
1152
|
-
|
|
1216
|
+
As the container is immutable, modifying this file is not recommended.
|
|
1217
|
+
To change the configuration, tell `ascp` to use another file using `ascp` option `-f`, for example, by locating it on the host folder `$HOME/.aspera/ascli` mapped to the container folder `/home/cliuser/.aspera/ascli`:
|
|
1153
1218
|
|
|
1154
1219
|
```shell
|
|
1155
1220
|
echo '<CONF/>' > $HOME/.aspera/ascli/aspera.conf
|
|
@@ -1163,7 +1228,7 @@ Then, tell `ascp` to use that other configuration file:
|
|
|
1163
1228
|
|
|
1164
1229
|
#### Container: Singularity
|
|
1165
1230
|
|
|
1166
|
-
Singularity is another
|
|
1231
|
+
Singularity is another container runtime.
|
|
1167
1232
|
|
|
1168
1233
|
On Linux install:
|
|
1169
1234
|
|
|
@@ -1199,7 +1264,7 @@ To display the version of **OpenSSL** used in `ascli`:
|
|
|
1199
1264
|
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
1200
1265
|
```
|
|
1201
1266
|
|
|
1202
|
-
|
|
1267
|
+
To use another SSL library or version, execute:
|
|
1203
1268
|
|
|
1204
1269
|
```shell
|
|
1205
1270
|
gem install openssl -- --with-openssl-dir=[openssl library folder]
|
|
@@ -1228,26 +1293,20 @@ gem install openssl -- --with-openssl-dir=$(openssl version -e|sed -n 's|ENGINES
|
|
|
1228
1293
|
SSL certificates are validated using a certificate store.
|
|
1229
1294
|
By default, it is the one of the system's `openssl` library.
|
|
1230
1295
|
|
|
1231
|
-
To display trusted certificate store locations:
|
|
1296
|
+
To display the default trusted certificate store locations:
|
|
1232
1297
|
|
|
1233
1298
|
```shell
|
|
1234
|
-
ascli
|
|
1299
|
+
ascli config echo '@ruby:[OpenSSL::X509::DEFAULT_CERT_DIR,OpenSSL::X509::DEFAULT_CERT_FILE]'
|
|
1235
1300
|
```
|
|
1236
1301
|
|
|
1237
1302
|
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, and so on).
|
|
1238
1303
|
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
|
1239
|
-
|
|
1240
|
-
One can display those default values:
|
|
1241
|
-
|
|
1242
|
-
```shell
|
|
1243
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
|
1244
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
|
1245
|
-
```
|
|
1304
|
+
If neither these env vars are set nor the default locations exist (for example, a single executable built on another Linux distribution), `ascli` sets `SSL_CERT_FILE` to the first system CA bundle found: `/etc/ssl/certs/ca-certificates.crt`, `/etc/pki/tls/certs/ca-bundle.crt`, `/etc/ssl/ca-bundle.pem`, `/etc/ssl/cert.pem`.
|
|
1246
1305
|
|
|
1247
1306
|
To get certificate validation, the CA certificate bundle must be up-to-date.
|
|
1248
1307
|
Check this repository on how to update the system's CA certificate bundle: [https://github.com/millermatt/osca](https://github.com/millermatt/osca).
|
|
1249
1308
|
|
|
1250
|
-
For example on RHEL/Rocky Linux:
|
|
1309
|
+
For example, on RHEL/Rocky Linux:
|
|
1251
1310
|
|
|
1252
1311
|
```shell
|
|
1253
1312
|
dnf install -y ca-certificates
|
|
@@ -1255,7 +1314,7 @@ update-ca-trust extract
|
|
|
1255
1314
|
```
|
|
1256
1315
|
|
|
1257
1316
|
The SSL CA certificate bundle can be specified using the `cert_stores` option, which accepts a list of files or directories.
|
|
1258
|
-
By default, Ruby
|
|
1317
|
+
By default, Ruby's system certificate store is used.
|
|
1259
1318
|
|
|
1260
1319
|
When `cert_stores` is provided:
|
|
1261
1320
|
|
|
@@ -1267,7 +1326,7 @@ When `cert_stores` is provided:
|
|
|
1267
1326
|
> [!NOTE]
|
|
1268
1327
|
> JRuby uses its own implementation and CA bundles.
|
|
1269
1328
|
|
|
1270
|
-
For example, on Linux to force the use the system's certificate store:
|
|
1329
|
+
For example, on Linux, to force the use of the system's certificate store:
|
|
1271
1330
|
|
|
1272
1331
|
```shell
|
|
1273
1332
|
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
|
@@ -1275,7 +1334,7 @@ For example, on Linux to force the use the system's certificate store:
|
|
|
1275
1334
|
|
|
1276
1335
|
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
|
|
1277
1336
|
|
|
1278
|
-
By default
|
|
1337
|
+
By default, `ascp` uses a hard-coded root location `OPENSSLDIR`.
|
|
1279
1338
|
Original `ascp`'s hard-coded locations can be found using:
|
|
1280
1339
|
|
|
1281
1340
|
```shell
|
|
@@ -1284,22 +1343,20 @@ ascli config ascp info --fields=openssldir
|
|
|
1284
1343
|
|
|
1285
1344
|
For example, on macOS: `/Library/Aspera/ssl`.
|
|
1286
1345
|
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
|
1287
|
-
`ascli` overrides the default hard
|
|
1346
|
+
`ascli` overrides the default hard-coded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
|
1288
1347
|
|
|
1289
|
-
To update trusted root certificates for `ascli
|
|
1290
|
-
Display the trusted certificate store locations used by `ascli`.
|
|
1291
|
-
Typically done by updating the system's root certificate store.
|
|
1348
|
+
To update trusted root certificates for `ascli`, update the system's root certificate store (see the locations displayed above).
|
|
1292
1349
|
|
|
1293
1350
|
An up-to-date version of the certificate bundle can also be retrieved with:
|
|
1294
1351
|
|
|
1295
1352
|
```shell
|
|
1296
|
-
ascli config echo @uri:https://curl.
|
|
1353
|
+
ascli config echo @uri:https://curl.se/ca/cacert.pem --format=text
|
|
1297
1354
|
```
|
|
1298
1355
|
|
|
1299
1356
|
To download that certificate store:
|
|
1300
1357
|
|
|
1301
1358
|
```shell
|
|
1302
|
-
ascli config echo @uri:https://curl.
|
|
1359
|
+
ascli config echo @uri:https://curl.se/ca/cacert.pem --format=text --out.file=/tmp/cacert.pem
|
|
1303
1360
|
```
|
|
1304
1361
|
|
|
1305
1362
|
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
|
@@ -1356,7 +1413,7 @@ ascli -h
|
|
|
1356
1413
|
See [Usage](#usage).
|
|
1357
1414
|
|
|
1358
1415
|
> [!NOTE]
|
|
1359
|
-
> `ascli`
|
|
1416
|
+
> Not all features of `ascli` are documented here: explore commands on the command line, using `-h`.
|
|
1360
1417
|
|
|
1361
1418
|
### Command Line Arguments
|
|
1362
1419
|
|
|
@@ -1364,7 +1421,7 @@ Command line arguments are the units of command line typically separated by spac
|
|
|
1364
1421
|
|
|
1365
1422
|
`ascli` handles the following types of command line arguments:
|
|
1366
1423
|
|
|
1367
|
-
- [**Options**](#options): absolute position is not important, but order is
|
|
1424
|
+
- [**Options**](#options): absolute position is not important, but relative order is, as a given option may be provided several times
|
|
1368
1425
|
- [**Plugins**](#plugins) for example, `config`, on first position
|
|
1369
1426
|
- [**Resource Types**](#resource-types) for example, `users`
|
|
1370
1427
|
- [**Verbs**](#verbs) for example, `create`, to act on those resources or plugins.
|
|
@@ -1375,6 +1432,7 @@ Command line arguments that are not options are referred to as **Positional Argu
|
|
|
1375
1432
|
|
|
1376
1433
|
For example:
|
|
1377
1434
|
|
|
1435
|
+
|
|
1378
1436
|
```shell
|
|
1379
1437
|
ascli plugin command verb --option-name=VAL1 VAL2
|
|
1380
1438
|
```
|
|
@@ -1431,9 +1489,9 @@ A resource type can also be a grouping of other resource types, for example `adm
|
|
|
1431
1489
|
Standard resource **Verbs** are: `create`, `show`, `list`, `modify`, `delete`.
|
|
1432
1490
|
Some entities also support additional verbs.
|
|
1433
1491
|
When such additional commands relate to a resource also accessible in another context, they are placed under the `do` command.
|
|
1434
|
-
|
|
1492
|
+
Such subcommands appear after the resource identifier, for example, in `ascli aoc admin node do <NODE_ID> browse /`, `browse` is a subcommand of `node`.
|
|
1435
1493
|
|
|
1436
|
-
Typically, the `create` verb takes
|
|
1494
|
+
Typically, the `create` verb takes resource creation data as a parameter.
|
|
1437
1495
|
`show`, `modify` and `delete` take an identifier, unless manipulating a singleton.
|
|
1438
1496
|
`list` typically uses the `query`, `select` options.
|
|
1439
1497
|
`list` and `show` typically use the `fields` option.
|
|
@@ -1442,7 +1500,7 @@ Typically, the `create` verb takes a resource creation data as a parameter.
|
|
|
1442
1500
|
|
|
1443
1501
|
Identifiers uniquely identify a resource.
|
|
1444
1502
|
They are typically located immediately after a verb like `show`, `modify` or `delete`, itself placed after the resource type, for example: `user show foobar`.
|
|
1445
|
-
Some resources
|
|
1503
|
+
Some resources can also be selected by a unique field other than the native identifier (typically: `id`), using the [**percent selector**](#percent-selector).
|
|
1446
1504
|
|
|
1447
1505
|
##### Percent selector
|
|
1448
1506
|
|
|
@@ -1474,7 +1532,7 @@ ascli aoc admin user show %name:john
|
|
|
1474
1532
|
|
|
1475
1533
|
If a **Command Parameter** begins with `-`, then either use the `@val:` syntax (see [Extended Value](#extended-value-syntax)), or use the `--` separator (see below).
|
|
1476
1534
|
|
|
1477
|
-
A few **Command Parameters** are optional
|
|
1535
|
+
A few **Command Parameters** are optional: they are always located at the end of the command line.
|
|
1478
1536
|
|
|
1479
1537
|
#### Enumerations
|
|
1480
1538
|
|
|
@@ -1490,9 +1548,10 @@ The following are enumerations:
|
|
|
1490
1548
|
|
|
1491
1549
|
Examples:
|
|
1492
1550
|
|
|
1493
|
-
|
|
1494
|
-
-
|
|
1495
|
-
- Option
|
|
1551
|
+
|
|
1552
|
+
- Positional: `ascli config pre ov --for=c` is the same as `ascli config preset overview --format=csv`
|
|
1553
|
+
- Option name: `--log-l=debug` is the same as `--log-level=debug`
|
|
1554
|
+
- Option value: `--format=c` is the same as `--format=csv`
|
|
1496
1555
|
|
|
1497
1556
|
> [!NOTE]
|
|
1498
1557
|
> While prefix matching works for option names, using full names is recommended for clarity.
|
|
@@ -1518,15 +1577,18 @@ A [dot-path](#dot-path-notation) is a `String` where segments are separated by `
|
|
|
1518
1577
|
- A **string segment** designates a key in a `Hash` (associative array).
|
|
1519
1578
|
- An **integer segment** designates an index in an `Array`.
|
|
1520
1579
|
|
|
1521
|
-
For example, the path `a.b.0` means: key `a
|
|
1580
|
+
For example, the path `a.b.0` means: key `a`, then key `b`, then the first element of an array.
|
|
1522
1581
|
|
|
1523
1582
|
When a **value** is assigned to the path (**write** with `=`), it is automatically converted to the simplest matching type: `Boolean`, `Integer`, `Float`, or `String`.
|
|
1583
|
+
Values `true` and `yes` are converted to `Boolean` `true`, and values `false` and `no` to `false`.
|
|
1584
|
+
For an option expecting a list of values that includes `yes` or `no` (e.g. `--out.table.pivot=no`), the `Boolean` is converted back to that value.
|
|
1524
1585
|
|
|
1525
1586
|
> [!NOTE]
|
|
1526
1587
|
> A value of `1` will be automatically converted to an `Integer`.
|
|
1527
1588
|
> When a specific type is required for the value, the [Extended Value](#extended-value-syntax) syntax modifiers `@json:` or `@ruby:` can be used.
|
|
1528
1589
|
> For example: `--opt.x=1` generates `{"x": 1}`.
|
|
1529
1590
|
> To get a `String`: `--opt.x=@json:\"1\"` or `--opt.x=@ruby:%q{1}`.
|
|
1591
|
+
> Likewise, to get the `String` `yes`: `--opt.x=@json:\"yes\"`.
|
|
1530
1592
|
|
|
1531
1593
|
Example: [dot-path](#dot-path-notation) to JSON output
|
|
1532
1594
|
|
|
@@ -1596,7 +1658,7 @@ ascli aoc packages send @: name="<TITLE>" recipients.0=user@example.com END file
|
|
|
1596
1658
|
> [!NOTE]
|
|
1597
1659
|
> `@:` can also be used as an option value (for example, `--query=@: a=b`).
|
|
1598
1660
|
> In that case, only positional arguments **after the option's position** in the original command line are consumed, so sub-commands before the option are not affected.
|
|
1599
|
-
> Use `END` as usual to stop collection when further positional arguments must follow: `ascli aoc
|
|
1661
|
+
> Use `END` as usual to stop collection when further positional arguments must follow: `ascli aoc tier_restrictions --query=@: a=b END other_arg`.
|
|
1600
1662
|
|
|
1601
1663
|
#### Options
|
|
1602
1664
|
|
|
@@ -1606,7 +1668,7 @@ Command-line options, such as `--log-level=debug`, follow these conventions:
|
|
|
1606
1668
|
All options begin with `--`.
|
|
1607
1669
|
- **Naming**:
|
|
1608
1670
|
Option names on command line use lowercase letters and hyphens (`-`) as word separators.
|
|
1609
|
-
Option
|
|
1671
|
+
Option names in the configuration file use underscores (`_`) as word separators.
|
|
1610
1672
|
Example: `--log-level=debug` is `log_level` in config file.
|
|
1611
1673
|
- **Values**:
|
|
1612
1674
|
An option's value is assigned using `=` (for example, `--log-level=debug`).
|
|
@@ -1647,8 +1709,8 @@ Example:
|
|
|
1647
1709
|
ascli config echo -- --sample
|
|
1648
1710
|
```
|
|
1649
1711
|
|
|
1650
|
-
```
|
|
1651
|
-
|
|
1712
|
+
```text
|
|
1713
|
+
--sample
|
|
1652
1714
|
```
|
|
1653
1715
|
|
|
1654
1716
|
> [!NOTE]
|
|
@@ -1668,7 +1730,7 @@ The value for **any** options can come from the following locations (in this ord
|
|
|
1668
1730
|
- Environment variable
|
|
1669
1731
|
- Command line
|
|
1670
1732
|
|
|
1671
|
-
Environment
|
|
1733
|
+
Environment variables starting with prefix ASCLI_ are taken as option values, for example, `ASCLI_OPTION_NAME` is for `--option-name`.
|
|
1672
1734
|
|
|
1673
1735
|
Option `show_config` dry runs the configuration, and then returns currently set values for options.
|
|
1674
1736
|
|
|
@@ -1682,7 +1744,7 @@ A command line argument is typically designed as option if:
|
|
|
1682
1744
|
|
|
1683
1745
|
### Interactive Input
|
|
1684
1746
|
|
|
1685
|
-
Some options and **Command Parameters** are mandatory and
|
|
1747
|
+
Some options and **Command Parameters** are mandatory and others are optional.
|
|
1686
1748
|
By default, `ascli` prompts for missing mandatory options or **Command Parameters** during interactive execution.
|
|
1687
1749
|
|
|
1688
1750
|
The behavior can be controlled with:
|
|
@@ -1698,7 +1760,7 @@ The behavior can be controlled with:
|
|
|
1698
1760
|
Command execution will result in output (terminal, stdout/stderr).
|
|
1699
1761
|
The information displayed depends on the action.
|
|
1700
1762
|
|
|
1701
|
-
To redirect results to a file, use option
|
|
1763
|
+
To redirect results to a file, use option `--out.file`.
|
|
1702
1764
|
|
|
1703
1765
|
#### Types of output data
|
|
1704
1766
|
|
|
@@ -1706,13 +1768,15 @@ Depending on action, the output will contain:
|
|
|
1706
1768
|
|
|
1707
1769
|
| Result Type | Description |
|
|
1708
1770
|
|-----------------|-----------------------------------------------------------------------------------|
|
|
1709
|
-
| `single_object` | Displayed as a
|
|
1710
|
-
| `object_list` | Displayed as a
|
|
1771
|
+
| `single_object` | Displayed as a two-dimensional table: one line per field, first column is field name, and second is field value. Nested hashes are collapsed. |
|
|
1772
|
+
| `object_list` | Displayed as a two-dimensional table: one line per item, one column per field. |
|
|
1711
1773
|
| `value_list` | A table with one column. |
|
|
1712
1774
|
| `empty` | nothing |
|
|
1713
1775
|
| `status` | A message. |
|
|
1714
1776
|
| `other_struct` | A complex structure that cannot be displayed as an array. |
|
|
1715
1777
|
|
|
1778
|
+
With a structured format (`json`, `jsonpp`, `yaml`, `ruby`), the output is always valid in this format: a status is a string, an empty list is `[]`, and no result is `null`.
|
|
1779
|
+
|
|
1716
1780
|
#### Enhanced display of special values
|
|
1717
1781
|
|
|
1718
1782
|
Special values are highlighted as follows in `format=table`:
|
|
@@ -1757,7 +1821,7 @@ The style of output can be set using the `format` option:
|
|
|
1757
1821
|
| `image` | URL or data for a [picture/video](#image-and-video-thumbnails) |
|
|
1758
1822
|
| `nagios` | Suitable for Nagios |
|
|
1759
1823
|
|
|
1760
|
-
By default,
|
|
1824
|
+
By default, results of type `single_object` and `object_list` are displayed using format `table`.
|
|
1761
1825
|
|
|
1762
1826
|
#### Option: `--out.table`
|
|
1763
1827
|
|
|
@@ -1768,16 +1832,16 @@ For `format=table`, options are the ones described in gem [`terminal-table`](htt
|
|
|
1768
1832
|
For example, to display a table with thick Unicode borders:
|
|
1769
1833
|
|
|
1770
1834
|
```shell
|
|
1771
|
-
ascli config preset
|
|
1835
|
+
ascli config preset overview --out.table=@ruby:'{border: :unicode_thick_edge}'
|
|
1772
1836
|
```
|
|
1773
1837
|
|
|
1774
1838
|
> [!NOTE]
|
|
1775
1839
|
> Other border styles exist, not limited to: `:unicode`, `:unicode_round`.
|
|
1776
1840
|
|
|
1777
|
-
By default, if the terminal
|
|
1841
|
+
By default, if the terminal supports Unicode (see [`--out.utf8`](#terminal-rendering-colors-and-utf-8)), then `border=unicode_round` is used.
|
|
1778
1842
|
|
|
1779
1843
|
A special parameter is defined: `str_lst_sep` (`String`), default is `\n`.
|
|
1780
|
-
It defines how
|
|
1844
|
+
It defines how lists of strings are displayed.
|
|
1781
1845
|
Alternatively, set to `,`.
|
|
1782
1846
|
|
|
1783
1847
|
For `format=csv`, options are described in gem [`csv`](https://ruby.github.io/csv/CSV.html#class-CSV-label-Options+for+Generating).
|
|
@@ -1802,7 +1866,7 @@ If value is `yes` (default), then objects are "flattened" using [dot-path](#dot-
|
|
|
1802
1866
|
- `Array` of `Hash` with only `name` keys are displayed as comma separated list of values
|
|
1803
1867
|
- `Array` of `Hash` with only `name` and `value` keys are displayed like a `Hash` with value of `name` as key.
|
|
1804
1868
|
|
|
1805
|
-
Example:
|
|
1869
|
+
Example: The result of the command is a single object:
|
|
1806
1870
|
|
|
1807
1871
|
```shell
|
|
1808
1872
|
ascli config echo @json:'{"A":"a","B":[{"name":"B1","value":"b1"},{"name":"B2","value":"b2"}],"C":[{"C1":"c1"},{"C2":"c2"}],"D":{"D1":"d1","D2":"d2"}}'
|
|
@@ -1838,7 +1902,8 @@ For the same command, adding option `--out.flat=no`:
|
|
|
1838
1902
|
#### Option: `--out.table.pivot`
|
|
1839
1903
|
|
|
1840
1904
|
This option controls how result fields are displayed as columns or lines, when option `format` is set to `table`.
|
|
1841
|
-
|
|
1905
|
+
Values are `false` (default), `true` or `single`.
|
|
1906
|
+
On command line, `no` and `yes` can be used as well (converted to `Boolean`, see [dot-path](#dot-path-notation)), but a structured value (e.g. `--out=@json:'{"table":{"pivot":true}}'`) expects a `Boolean`.
|
|
1842
1907
|
There are two types of results that are affected by this option:
|
|
1843
1908
|
|
|
1844
1909
|
| Result | Description |
|
|
@@ -1855,7 +1920,7 @@ An item (object) is displayed in one of those 2 ways:
|
|
|
1855
1920
|
|
|
1856
1921
|
The display of result is as follows:
|
|
1857
1922
|
|
|
1858
|
-
| Result | `
|
|
1923
|
+
| Result | `false` | `true` | `single` |
|
|
1859
1924
|
|-----------------|------------|-------------|-------------------------------------|
|
|
1860
1925
|
| `single_object` | Simple | Simple | Simple |
|
|
1861
1926
|
| `object_list` | Transposed | Simple<br/>(Multiple objects) | Simple if 1 object.<br/>transposed if 2+ objects. |
|
|
@@ -1938,7 +2003,7 @@ Display with `yes` (multiple Simple):
|
|
|
1938
2003
|
|
|
1939
2004
|
#### Option: `--out.level`: Verbosity of output
|
|
1940
2005
|
|
|
1941
|
-
Output messages are categorized
|
|
2006
|
+
Output messages are categorized into three types:
|
|
1942
2007
|
|
|
1943
2008
|
- `info` output contains additional information, such as the number of elements in a table
|
|
1944
2009
|
- `data` output contains the actual output of the command (object, or list of objects)
|
|
@@ -1947,15 +2012,33 @@ Output messages are categorized in 3 types:
|
|
|
1947
2012
|
The option `--out.level` controls the level of output:
|
|
1948
2013
|
|
|
1949
2014
|
- `info` displays all messages: `info`, `data`, and `error`
|
|
1950
|
-
- `data`
|
|
1951
|
-
- `error`
|
|
2015
|
+
- `data` displays `data` and `error` messages
|
|
2016
|
+
- `error` displays only error messages
|
|
1952
2017
|
|
|
1953
2018
|
#### Option: `--out.secrets`: Hide or show secrets in results
|
|
1954
2019
|
|
|
1955
2020
|
- If value is `no` (default), then secrets are redacted from command results.
|
|
1956
|
-
- If value is `yes`, then secrets shown in clear in results.
|
|
2021
|
+
- If value is `yes`, then secrets are shown in clear in results.
|
|
1957
2022
|
- If `--out.level` is `data`, secrets are included to allow piping results.
|
|
1958
2023
|
|
|
2024
|
+
#### Terminal rendering: Colors and UTF-8
|
|
2025
|
+
|
|
2026
|
+
By default, `ascli` detects the capabilities of the terminal:
|
|
2027
|
+
|
|
2028
|
+
| Option | Effect when `yes` | Auto-detection (default) |
|
|
2029
|
+
|----------------|---------------------------------------------------------------|--------------------------------------------------------------------------------|
|
|
2030
|
+
| `--out.colors` | ANSI colors and styles in results, messages and logs | `yes` if both `stdout` and `stderr` are terminals and `TERM` is not `dumb`, or if `CLICOLOR_FORCE=1` |
|
|
2031
|
+
| `--out.utf8` | Unicode characters: table borders, check marks | `yes` if `stdout` is a terminal and the locale is UTF-8 |
|
|
2032
|
+
|
|
2033
|
+
Set either option to `yes` or `no` to override detection, for example to keep colors when piping to `less -R`, or to get plain ASCII output in a terminal:
|
|
2034
|
+
|
|
2035
|
+
```shell
|
|
2036
|
+
ascli config preset overview --out.colors=no --out.utf8=no
|
|
2037
|
+
```
|
|
2038
|
+
|
|
2039
|
+
> [!NOTE]
|
|
2040
|
+
> Logs issued before the option is processed (e.g. at startup) use the detected value.
|
|
2041
|
+
|
|
1959
2042
|
#### Option: `fields`: Selection of output object fields
|
|
1960
2043
|
|
|
1961
2044
|
Depending on the command, results may include by default all fields, or only some selected fields.
|
|
@@ -2050,13 +2133,13 @@ The following decoders are supported:
|
|
|
2050
2133
|
|----------|----------|----------|------------------------------------------------------------------------------------------|
|
|
2051
2134
|
| `base64` | `String` | `String` | Decode a base64 encoded string. |
|
|
2052
2135
|
| `csvt` | `String` | `Array` | Decode a titled CSV value. |
|
|
2053
|
-
| `env` | `String` | `String` | Read from
|
|
2054
|
-
| `file` | `String` | `String` | Read value from specified file (prefix `~/` is replaced with the user's home folder).
|
|
2136
|
+
| `env` | `String` | `String` | Read from the named environment variable. For example, `--password=@env:MYPASSVAR` |
|
|
2137
|
+
| `file` | `String` | `String` | Read value from specified file (prefix `~/` is replaced with the user's home folder). For example, `--key=@file:~/.ssh/mykey` |
|
|
2055
2138
|
| `json` | `String` | Any | Decode JSON values. Convenient to provide complex structures. |
|
|
2056
2139
|
| `lines` | `String` | `Array` | Split a string in multiple lines and return an `Array`. |
|
|
2057
2140
|
| `list` | `String` | `Array` | Split a string in multiple items taking first character as separator and return an `Array`. |
|
|
2058
2141
|
| `none` | None | Nil | A `null` value. |
|
|
2059
|
-
| `path` | `String` | `String` | Performs path expansion on specified path (prefix `~/` is replaced with the user's home folder).
|
|
2142
|
+
| `path` | `String` | `String` | Performs path expansion on specified path (prefix `~/` is replaced with the user's home folder). For example, `--config-file=@path:~/sample_config.yml` |
|
|
2060
2143
|
| `preset` | `String` | `Hash` | Get value from configuration file using [dot-path](#dot-path-notation) notation. |
|
|
2061
2144
|
| `extend` | `String` | `String` | Evaluates embedded [Extended Value](#extended-value-syntax) syntax in string. |
|
|
2062
2145
|
| `re` | `String` | `Regexp` | Ruby Regular Expression (short for `@ruby:/.../`) |
|
|
@@ -2064,8 +2147,8 @@ The following decoders are supported:
|
|
|
2064
2147
|
| `s` | Any | `String` | Converts argument to `String`. |
|
|
2065
2148
|
| `secret` | `String` | `String` | Ask password interactively (hides input). Argument is the prompt. |
|
|
2066
2149
|
| `stdin` | `String` | `String` | Read from stdin in text mode. Argument: `<empty>`, `bin` or `chomp`. |
|
|
2067
|
-
| `uri` | `String` | `String` | Read value from specified URL. Supported schemes: `http:`, `https:`, `data:`, `file:`.
|
|
2068
|
-
| `val` | `String` | `String` | Prevent decoders on the right
|
|
2150
|
+
| `uri` | `String` | `String` | Read value from specified URL. Supported schemes: `http:`, `https:`, `data:`, `file:`. For example, `--fpac=@uri:http://serv/f.pac` or `--key=@uri:file:/path/to/key.pem` |
|
|
2151
|
+
| `val` | `String` | `String` | Prevent decoding by the decoders on the right. For example, `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
|
2069
2152
|
| `yaml` | `String` | Any | Decode YAML. |
|
|
2070
2153
|
| `zlib` | `String` | `String` | Decompress data using zlib. |
|
|
2071
2154
|
| `<empty>`| None | Any | The **dot-path** modifier: argument `@:` collects the following positional arguments as `key.subkey=value` assignments and builds a `Hash` or `Array` using [dot-path](#dot-path-notation) notation. Shell-friendly alternative to `@json:` for structured values.<br/>See [Positional Arguments with Dot-path](#positional-arguments-with-dot-path) for full syntax. Use `END` to stop collection when further positional arguments must follow. |
|
|
@@ -2184,7 +2267,7 @@ Example: read a CSV file and create an `Array` of `Hash` for bulk provisioning:
|
|
|
2184
2267
|
cat test.csv
|
|
2185
2268
|
```
|
|
2186
2269
|
|
|
2187
|
-
```
|
|
2270
|
+
```text
|
|
2188
2271
|
name,email
|
|
2189
2272
|
lolo,laurent@example.com
|
|
2190
2273
|
toto,titi@tutu.tata
|
|
@@ -2248,8 +2331,8 @@ ascli aoc packages send
|
|
|
2248
2331
|
```
|
|
2249
2332
|
|
|
2250
2333
|
```text
|
|
2251
|
-
ERRR Missing argument:
|
|
2252
|
-
HINT
|
|
2334
|
+
ERRR Missing: Missing argument: package (Hash)
|
|
2335
|
+
HINT:Give `help` as argument to retrieve the schema of the missing argument.
|
|
2253
2336
|
```
|
|
2254
2337
|
|
|
2255
2338
|
Following the hint and passing `help` as the argument displays the schema:
|
|
@@ -2259,22 +2342,22 @@ ascli aoc packages send help
|
|
|
2259
2342
|
```
|
|
2260
2343
|
|
|
2261
2344
|
```text
|
|
2262
|
-
INFO Schema: argument:
|
|
2263
|
-
|
|
2264
|
-
|
|
2265
|
-
|
|
2266
|
-
|
|
2267
|
-
|
|
2268
|
-
|
|
2269
|
-
|
|
2270
|
-
|
|
2271
|
-
|
|
2272
|
-
|
|
2273
|
-
|
|
2274
|
-
|
|
2275
|
-
|
|
2345
|
+
INFO Schema: argument: package (Hash)
|
|
2346
|
+
╭───────────────────────┬────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
|
|
2347
|
+
│ name │ type │ required │ description │
|
|
2348
|
+
╞═══════════════════════╪════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════╡
|
|
2349
|
+
│ bcc_recipients │ array │ false │ <empty string> │
|
|
2350
|
+
│ bcc_recipients[].id │ string │ true │ The ID of the recipient. │
|
|
2351
|
+
│ bcc_recipients[].type │ enum │ false │ The entity type of the recipient. │
|
|
2352
|
+
│ │ │ │ Allowed: user, group │
|
|
2353
|
+
│ name │ string │ true │ Package name. Required for POST. Optional for PUT. │
|
|
2354
|
+
│ note │ string │ false │ The sender's message to recipients to include with the package. Maximum characters: 65535. │
|
|
2355
|
+
│ recipients │ array │ false │ <empty string> │
|
|
2356
|
+
│ recipients[].id │ string │ true │ The ID of the recipient. │
|
|
2357
|
+
│ recipients[].type │ enum │ false │ The entity type of the recipient. │
|
|
2358
|
+
│ │ │ │ Allowed: user, group │
|
|
2276
2359
|
...
|
|
2277
|
-
|
|
2360
|
+
╰───────────────────────┴────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
2278
2361
|
```
|
|
2279
2362
|
|
|
2280
2363
|
The same applies to options: display the schema of the transfer-spec option `ts`:
|
|
@@ -2285,23 +2368,36 @@ ascli --ts=help
|
|
|
2285
2368
|
|
|
2286
2369
|
```text
|
|
2287
2370
|
INFO Schema: option: ts
|
|
2288
|
-
|
|
2289
|
-
│ name
|
|
2290
|
-
|
|
2291
|
-
│ apply_local_docroot
|
|
2292
|
-
│
|
|
2293
|
-
│
|
|
2294
|
-
│
|
|
2295
|
-
│ cipher │ string │ In transit encryption algorithms. │
|
|
2296
|
-
│ │ │ Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, │
|
|
2297
|
-
│ │ │ aes-256-gcm. │
|
|
2298
|
-
│ │ │ Default: none. │
|
|
2371
|
+
╭─────────────────────┬─────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
|
|
2372
|
+
│ name │ type │ required │ description │
|
|
2373
|
+
╞═════════════════════╪═════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
|
|
2374
|
+
│ apply_local_docroot │ boolean │ false │ Apply local docroot to source paths. │
|
|
2375
|
+
│ cipher │ enum │ false │ In transit encryption algorithms. │
|
|
2376
|
+
│ │ │ │ Allowed: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm │
|
|
2377
|
+
│ authentication │ string │ false │ Set to `token` for SSH bypass keys, else password asked if not provided. │
|
|
2299
2378
|
...
|
|
2300
|
-
|
|
2379
|
+
╰─────────────────────┴─────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
|
2301
2380
|
```
|
|
2302
2381
|
|
|
2303
2382
|
This works for any `Hash` option or positional parameter that has a defined schema.
|
|
2304
2383
|
|
|
2384
|
+
#### Schema Validation
|
|
2385
|
+
|
|
2386
|
+
The value of an option or **Command Parameter** with a schema defined by `ascli` (e.g. options `ts`, `transfer`, `out`, `http_options`, argument of `orchestrator workflows start`) is validated against it before the action.
|
|
2387
|
+
An invalid value is rejected with the path of the invalid element and the reason:
|
|
2388
|
+
|
|
2389
|
+
```shell
|
|
2390
|
+
ascli config echo 1 --ts=@json:'{"direction":"sideways"}'
|
|
2391
|
+
```
|
|
2392
|
+
|
|
2393
|
+
```text
|
|
2394
|
+
ERRR Argument: Option ts: value at `/direction` is not one of: ["send", "receive"] (use --ts=help for schema)
|
|
2395
|
+
Use option -h to get help.
|
|
2396
|
+
```
|
|
2397
|
+
|
|
2398
|
+
Option values are merged from several sources (presets, command line), so mandatory fields are not checked for options.
|
|
2399
|
+
Request bodies of product APIs (e.g. `create` and `modify` commands) are not validated by `ascli`: the API validates them.
|
|
2400
|
+
|
|
2305
2401
|
#### Testing Extended Value
|
|
2306
2402
|
|
|
2307
2403
|
Two complementary commands help verify that a value is parsed as expected:
|
|
@@ -2334,9 +2430,9 @@ Example: the shell parses three arguments (`1`, `2`, `3`), but `config echo` onl
|
|
|
2334
2430
|
ascli config echo 1 2 3
|
|
2335
2431
|
```
|
|
2336
2432
|
|
|
2337
|
-
```
|
|
2338
|
-
|
|
2339
|
-
|
|
2433
|
+
```text
|
|
2434
|
+
1
|
|
2435
|
+
ERRR Argument: unprocessed values: ["2", "3"]
|
|
2340
2436
|
```
|
|
2341
2437
|
|
|
2342
2438
|
**Checking an option value with `--show-config`**:
|
|
@@ -2344,18 +2440,19 @@ ERROR: Argument: unprocessed values: ["2", "3"]
|
|
|
2344
2440
|
Adding `--show-config` to any command line performs a dry run and displays the resolved value of all options that would be used, without executing the command.
|
|
2345
2441
|
|
|
2346
2442
|
To display a specific option, add `--fields=<option_name>`.
|
|
2347
|
-
Add `--flat=no` when the option holds a structured value (`Hash`, `Array`) to display it as-is rather than flattened into dot-path keys:
|
|
2443
|
+
Add `--out.flat=no` when the option holds a structured value (`Hash`, `Array`) to display it as-is rather than flattened into dot-path keys:
|
|
2444
|
+
|
|
2348
2445
|
|
|
2349
2446
|
```shell
|
|
2350
|
-
ascli --opt=@json:'{"a":1,"b":"two"}' some_plugin --show-config --fields=opt --flat=no
|
|
2351
|
-
ascli --opt.a=1 --opt.b=two some_plugin --show-config --fields=opt --flat=no
|
|
2447
|
+
ascli --opt=@json:'{"a":1,"b":"two"}' some_plugin --show-config --fields=opt --out.flat=no
|
|
2448
|
+
ascli --opt.a=1 --opt.b=two some_plugin --show-config --fields=opt --out.flat=no
|
|
2352
2449
|
```
|
|
2353
2450
|
|
|
2354
2451
|
Both lines above display the same resolved value for option `opt`.
|
|
2355
2452
|
|
|
2356
2453
|
In the following examples (using a POSIX shell, such as `bash`), several equivalent commands are provided.
|
|
2357
2454
|
For all examples, most special character handling is not specific to `ascli`:
|
|
2358
|
-
It depends on the underlying syntax: shell, JSON, and so on
|
|
2455
|
+
It depends on the underlying syntax: shell, JSON, and so on.
|
|
2359
2456
|
Depending on the case, a different `format` option is used to display the actual value.
|
|
2360
2457
|
|
|
2361
2458
|
For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
|
|
@@ -2379,13 +2476,13 @@ Hello World
|
|
|
2379
2476
|
The default value is `[User's home folder]/.aspera/ascli`.
|
|
2380
2477
|
|
|
2381
2478
|
> [!NOTE]
|
|
2382
|
-
> The `[User's home folder]` is determined using Ruby
|
|
2479
|
+
> The `[User's home folder]` is determined using Ruby's `Dir.home` method.
|
|
2383
2480
|
> Primary source: The HOME environment variable.
|
|
2384
2481
|
> On Windows: Ruby also checks `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%` (via `rb_w32_home_dir`).
|
|
2385
2482
|
> Additionally, `ascli` sets the `%HOME%` environment variable to the value of `%USERPROFILE%` if it exists and is valid.
|
|
2386
2483
|
> Therefore, on Windows, `%USERPROFILE%` is preferred because it is generally more reliable than `%HOMEDRIVE%%HOMEPATH%`.
|
|
2387
2484
|
|
|
2388
|
-
The configuration folder can be displayed using
|
|
2485
|
+
The configuration folder can be displayed using:
|
|
2389
2486
|
|
|
2390
2487
|
```shell
|
|
2391
2488
|
ascli config folder
|
|
@@ -2396,7 +2493,7 @@ ascli config folder
|
|
|
2396
2493
|
```
|
|
2397
2494
|
|
|
2398
2495
|
> [!NOTE]
|
|
2399
|
-
> This is equivalent to
|
|
2496
|
+
> This is equivalent to displaying the value of the `home` option.
|
|
2400
2497
|
|
|
2401
2498
|
```shell
|
|
2402
2499
|
ascli --show-config --fields=home
|
|
@@ -2412,7 +2509,7 @@ ascli config folder
|
|
|
2412
2509
|
C:\Users\Kenji\.aspera\ascli
|
|
2413
2510
|
```
|
|
2414
2511
|
|
|
2415
|
-
When OAuth is used (AoC,
|
|
2512
|
+
When OAuth is used (AoC, Faspex 5), `ascli` keeps a cache of generated bearer tokens in folder `persist_store` located in the configuration folder by default.
|
|
2416
2513
|
Option `cache_tokens` (**yes**/no) allows controlling if OAuth tokens are cached on file system, or generated for each request.
|
|
2417
2514
|
The command `config tokens flush` clears that cache.
|
|
2418
2515
|
Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
|
|
@@ -2422,7 +2519,7 @@ When a token has expired, then a new token is generated, either using a `refresh
|
|
|
2422
2519
|
|
|
2423
2520
|
On the first execution of `ascli`, an empty configuration file is created in the configuration folder (`ascli config folder`).
|
|
2424
2521
|
There is no mandatory information required in this file.
|
|
2425
|
-
|
|
2522
|
+
Its use is optional, as any option can be provided on the command line.
|
|
2426
2523
|
|
|
2427
2524
|
Although the file is a standard `YAML` file, `ascli` provides commands to read and modify it using the `config` command.
|
|
2428
2525
|
|
|
@@ -2468,7 +2565,7 @@ Two [Option Presets](#option-preset) are reserved:
|
|
|
2468
2565
|
It is used to check compatibility.
|
|
2469
2566
|
- `default` is reserved to define the default [Option Preset](#option-preset) name used for known plugins.
|
|
2470
2567
|
|
|
2471
|
-
The user may create as many [Option
|
|
2568
|
+
The user may create as many [Option Presets](#option-preset) as needed.
|
|
2472
2569
|
For instance, a particular [Option Preset](#option-preset) can be created for a particular application instance and contain URL and credentials.
|
|
2473
2570
|
|
|
2474
2571
|
Values in the configuration also follow the [Extended Value](#extended-value-syntax) syntax.
|
|
@@ -2489,7 +2586,7 @@ This creates the [Option Preset](#option-preset):
|
|
|
2489
2586
|
private_key: "@file:/Users/laurent/.aspera/ascli/<PKEY_NAME>"
|
|
2490
2587
|
```
|
|
2491
2588
|
|
|
2492
|
-
So, the key file
|
|
2589
|
+
So, the key file is read only at execution time and is not embedded in the configuration file.
|
|
2493
2590
|
|
|
2494
2591
|
> [!NOTE]
|
|
2495
2592
|
> The main use of the configuration file is to store collections of options.
|
|
@@ -2527,7 +2624,7 @@ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --u
|
|
|
2527
2624
|
This creates an [Option Preset](#option-preset) `demo_server` with all provided options.
|
|
2528
2625
|
|
|
2529
2626
|
> [!NOTE]
|
|
2530
|
-
> `update` takes **ALL** options provided
|
|
2627
|
+
> `update` takes **ALL** options provided on the command line (starting with `--` with a value).
|
|
2531
2628
|
|
|
2532
2629
|
The command `set` allows setting individual options in an [Option Preset](#option-preset):
|
|
2533
2630
|
|
|
@@ -2549,7 +2646,7 @@ ascli config preset set GLOBAL out.table.pivot single
|
|
|
2549
2646
|
ascli config preset set GLOBAL out.level data
|
|
2550
2647
|
```
|
|
2551
2648
|
|
|
2552
|
-
The parameter value is **automatically coerced** to its natural type: integers, floats and booleans (`true`/`false`) are stored as native YAML types rather than strings.
|
|
2649
|
+
The parameter value is **automatically coerced** to its natural type: integers, floats and booleans (`true`/`yes`/`false`/`no`) are stored as native YAML types rather than strings.
|
|
2553
2650
|
|
|
2554
2651
|
To **delete** a key from a preset, pass `@none:` as the value (evaluates to `nil`):
|
|
2555
2652
|
|
|
@@ -2560,10 +2657,10 @@ ascli config preset set GLOBAL out.table.pivot @none:
|
|
|
2560
2657
|
A full terminal based overview of the configuration can be displayed using:
|
|
2561
2658
|
|
|
2562
2659
|
```shell
|
|
2563
|
-
ascli config preset
|
|
2660
|
+
ascli config preset overview
|
|
2564
2661
|
```
|
|
2565
2662
|
|
|
2566
|
-
|
|
2663
|
+
The list of [Option Presets](#option-preset) can be displayed using:
|
|
2567
2664
|
|
|
2568
2665
|
```shell
|
|
2569
2666
|
ascli config preset list
|
|
@@ -2593,14 +2690,6 @@ ascli config open
|
|
|
2593
2690
|
> [!NOTE]
|
|
2594
2691
|
> This starts the editor specified by env var `EDITOR` if defined.
|
|
2595
2692
|
|
|
2596
|
-
The former format for commands is still supported:
|
|
2597
|
-
|
|
2598
|
-
```shell
|
|
2599
|
-
ascli config preset set|delete|show|initialize|update <NAME>
|
|
2600
|
-
ascli config preset over
|
|
2601
|
-
ascli config preset list
|
|
2602
|
-
```
|
|
2603
|
-
|
|
2604
2693
|
It is possible to load an [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
|
|
2605
2694
|
For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
|
|
2606
2695
|
|
|
@@ -2617,7 +2706,7 @@ This is the version of `ascli` which created the file.
|
|
|
2617
2706
|
|
|
2618
2707
|
#### Special Option Preset: `default`
|
|
2619
2708
|
|
|
2620
|
-
This preset name is reserved and contains
|
|
2709
|
+
This preset name is reserved and contains a `Hash`, where the key is the name of a plugin, and the value is the name of another preset.
|
|
2621
2710
|
Usually, [Option presets](#option-preset) are used to contain pre-defined options and values, but this preset contains names of presets to be used by default for plugins.
|
|
2622
2711
|
|
|
2623
2712
|
When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option `--no-default` (or `-N`) is used.
|
|
@@ -2662,8 +2751,8 @@ The default value is `_<>:"/\|?*`, corresponding to replacement character `_` an
|
|
|
2662
2751
|
Some temporary files may be needed during runtime.
|
|
2663
2752
|
The temporary folder may be specified with option: `temp_folder`.
|
|
2664
2753
|
Temporary files are deleted at the end of execution unless option: `clean_temp` is set to `no`.
|
|
2665
|
-
By default
|
|
2666
|
-
A special value of `@env`
|
|
2754
|
+
By default (`@sys`), the temporary folder is the system's temporary folder for the current user (Ruby `Etc.systmpdir`).
|
|
2755
|
+
A special value of `@env` sets the folder to Ruby `Dir.tmpdir`, which uses the usual environment variables (for example, `TMPDIR`).
|
|
2667
2756
|
|
|
2668
2757
|
### Plugin: `config`: Configuration
|
|
2669
2758
|
|
|
@@ -2675,7 +2764,7 @@ Plugin `config` provides general commands for `ascli`:
|
|
|
2675
2764
|
- `ascp`
|
|
2676
2765
|
- `transferd`
|
|
2677
2766
|
|
|
2678
|
-
The default preset for `config` is read for any plugin invocation
|
|
2767
|
+
The default preset for `config` is read for any plugin invocation: this allows setting global options, such as `--log-level` or `--interactive`.
|
|
2679
2768
|
When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
|
|
2680
2769
|
If set, it loads the options independently of the plugin used.
|
|
2681
2770
|
|
|
@@ -2737,6 +2826,9 @@ coffee --ui=text
|
|
|
2737
2826
|
coffee --ui=text --out.img.text=true
|
|
2738
2827
|
coffee --ui=text --out.img=@json:'{"text":true,"double":false}'
|
|
2739
2828
|
commands
|
|
2829
|
+
commands aoc --expand-mounts=yes
|
|
2830
|
+
commands aoc files
|
|
2831
|
+
commands server
|
|
2740
2832
|
detect app.example.com
|
|
2741
2833
|
detect https://f5.example.com/path
|
|
2742
2834
|
detect https://f5.example.com/path faspex5
|
|
@@ -2757,6 +2849,7 @@ echo @csvt:@stdin:
|
|
|
2757
2849
|
echo @env:USER
|
|
2758
2850
|
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=single
|
|
2759
2851
|
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=yes
|
|
2852
|
+
echo @json:'{"empty":"","bool":true}' --out.colors=yes --out.utf8=yes
|
|
2760
2853
|
echo @lines:@stdin:
|
|
2761
2854
|
echo @list:,1,2,3
|
|
2762
2855
|
echo @secret:
|
|
@@ -2786,6 +2879,7 @@ initdemo
|
|
|
2786
2879
|
open
|
|
2787
2880
|
options aoc
|
|
2788
2881
|
options server
|
|
2882
|
+
options server --select=@json:'{"option":"--display"}' --fields=replacement
|
|
2789
2883
|
plugins create my_command .
|
|
2790
2884
|
plugins list
|
|
2791
2885
|
preset delete conf_name
|
|
@@ -2836,15 +2930,14 @@ wizard my_org aoc mypreset --key-path=my_private_key --username=my_user_email
|
|
|
2836
2930
|
|
|
2837
2931
|
#### Evaluation order of options
|
|
2838
2932
|
|
|
2839
|
-
Some options are global,
|
|
2840
|
-
(the plugin is the first level command).
|
|
2933
|
+
Some options are global, others are available only for some plugins (the plugin is the first-level command).
|
|
2841
2934
|
|
|
2842
2935
|
Options are loaded using this algorithm:
|
|
2843
2936
|
|
|
2844
2937
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
|
2845
2938
|
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset) for it, and loads it.
|
|
2846
2939
|
- If option `--preset=<NAME>` is specified (or `-P<NAME>`), this reads the [Option Preset](#option-preset) specified from the configuration file by name.
|
|
2847
|
-
- If option `--preset=<EXTENDED_VALUE_HASH
|
|
2940
|
+
- If option `--preset=<EXTENDED_VALUE_HASH>` is specified, it is used as option values (`Hash` of option/value pairs).
|
|
2848
2941
|
- Environment variables are evaluated.
|
|
2849
2942
|
- Command line options are evaluated.
|
|
2850
2943
|
|
|
@@ -2853,7 +2946,7 @@ Options are evaluated in the order of command line.
|
|
|
2853
2946
|
To avoid loading the default [Option Preset](#option-preset) for a plugin, use: `-N`
|
|
2854
2947
|
|
|
2855
2948
|
On command line, words in option names are separated by a dash (`-`).
|
|
2856
|
-
In configuration file, separator is an underscore.
|
|
2949
|
+
In the configuration file, the separator is an underscore.
|
|
2857
2950
|
For example, `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
|
|
2858
2951
|
|
|
2859
2952
|
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset) for the main plugin with:
|
|
@@ -2887,66 +2980,50 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"<PASSWORD>","username":
|
|
|
2887
2980
|
#### Shell Completion
|
|
2888
2981
|
|
|
2889
2982
|
`ascli` supports shell tab-completion for **Bash**, **Zsh**, and **Fish**.
|
|
2890
|
-
|
|
2891
|
-
|
|
2892
|
-
All scripts call `ascli config completion bash [words...]` internally to query available sub-commands at any depth.
|
|
2983
|
+
The command `ascli config completion <shell>` displays the completion script for the given shell.
|
|
2893
2984
|
|
|
2894
|
-
|
|
2895
|
-
|
|
2896
|
-
To enable it, source the script in your shell profile (e.g. `~/.bashrc` or `~/.bash_profile`):
|
|
2985
|
+
To activate completion, add the line for your shell to its startup file:
|
|
2897
2986
|
|
|
2898
2987
|
```bash
|
|
2899
|
-
|
|
2988
|
+
# Bash, in ~/.bashrc
|
|
2989
|
+
eval "$(ascli config completion bash)"
|
|
2900
2990
|
```
|
|
2901
2991
|
|
|
2902
|
-
Or copy it to the system completion directory:
|
|
2903
|
-
|
|
2904
|
-
```bash
|
|
2905
|
-
cp $(gem contents aspera-cli | grep bash_autocomplete) /etc/bash_completion.d/ascli
|
|
2906
|
-
```
|
|
2907
|
-
|
|
2908
|
-
##### Zsh
|
|
2909
|
-
|
|
2910
|
-
Place the script somewhere on your `$fpath` and rebuild the completion cache:
|
|
2911
|
-
|
|
2912
2992
|
```zsh
|
|
2913
|
-
|
|
2914
|
-
|
|
2915
|
-
# fpath=(~/.zsh/completions $fpath)
|
|
2916
|
-
# autoload -Uz compinit && compinit
|
|
2917
|
-
exec zsh
|
|
2993
|
+
# Zsh, in ~/.zshrc, after compinit
|
|
2994
|
+
eval "$(ascli config completion zsh)"
|
|
2918
2995
|
```
|
|
2919
2996
|
|
|
2920
|
-
|
|
2921
|
-
|
|
2922
|
-
|
|
2923
|
-
ascli <Tab> # lists all plugins: aoc, server, node, ...
|
|
2924
|
-
ascli server <Tab> # lists server sub-commands: upload, download, ls, ...
|
|
2925
|
-
ascli aoc admin <Tab> # lists aoc admin sub-commands: user, node, ...
|
|
2997
|
+
```fish
|
|
2998
|
+
# Fish, in ~/.config/fish/config.fish
|
|
2999
|
+
ascli config completion fish | source
|
|
2926
3000
|
```
|
|
2927
3001
|
|
|
2928
|
-
|
|
2929
|
-
|
|
2930
|
-
Copy the completion script to Fish's completions directory:
|
|
3002
|
+
The startup file then executes `ascli` each time a shell starts.
|
|
3003
|
+
Alternatively, save the script once in the completion folder of the shell (and save it again after an upgrade of `ascli`):
|
|
2931
3004
|
|
|
2932
|
-
```
|
|
2933
|
-
|
|
3005
|
+
```bash
|
|
3006
|
+
# Bash, with package bash-completion
|
|
3007
|
+
ascli config completion bash > ~/.local/share/bash-completion/completions/ascli
|
|
3008
|
+
# Zsh (the folder must be in $fpath before compinit)
|
|
3009
|
+
ascli config completion zsh > ~/.zsh/completions/_ascli
|
|
3010
|
+
# Fish
|
|
3011
|
+
ascli config completion fish > ~/.config/fish/completions/ascli.fish
|
|
2934
3012
|
```
|
|
2935
3013
|
|
|
2936
|
-
No further configuration is needed - Fish loads files from `~/.config/fish/completions/` automatically.
|
|
2937
|
-
|
|
2938
3014
|
Once active, press `Tab` to complete commands at any depth:
|
|
2939
3015
|
|
|
2940
|
-
```
|
|
3016
|
+
```bash
|
|
2941
3017
|
ascli <Tab> # lists all plugins: aoc, server, node, ...
|
|
2942
3018
|
ascli server <Tab> # lists server sub-commands: upload, download, ls, ...
|
|
2943
3019
|
ascli aoc admin <Tab> # lists aoc admin sub-commands: user, node, ...
|
|
2944
3020
|
```
|
|
2945
3021
|
|
|
2946
|
-
|
|
3022
|
+
The scripts call `ascli config completion words [<word>...]`, which lists the words that can follow the given ones.
|
|
3023
|
+
It can also be used directly to inspect available commands:
|
|
2947
3024
|
|
|
2948
3025
|
```bash
|
|
2949
|
-
ascli config completion
|
|
3026
|
+
ascli config completion words aoc admin
|
|
2950
3027
|
```
|
|
2951
3028
|
|
|
2952
3029
|
#### Wizard
|
|
@@ -2969,8 +3046,8 @@ Options are also available for the wizard:
|
|
|
2969
3046
|
| `override` | yes/[no] | Override existing default preset name for the plugin, if it exists. |
|
|
2970
3047
|
| `key_path` | path | Path to private key for JWT. |
|
|
2971
3048
|
|
|
2972
|
-
Other plugin-specific options can be provided to the wizard, such as `--username
|
|
2973
|
-
They
|
|
3049
|
+
Other plugin-specific options can be provided to the wizard, such as `--username`.
|
|
3050
|
+
They are added to the [Option Preset](#option-preset) created by the wizard.
|
|
2974
3051
|
|
|
2975
3052
|
The simplest invocation is:
|
|
2976
3053
|
|
|
@@ -2978,17 +3055,17 @@ The simplest invocation is:
|
|
|
2978
3055
|
ascli config wizard
|
|
2979
3056
|
```
|
|
2980
3057
|
|
|
2981
|
-
If the application requires a private key, the user can
|
|
3058
|
+
If the application requires a private key, the user can provide the path to it with option `key_path`, or let the wizard generate one.
|
|
2982
3059
|
The user is told where to place the associated public key PEM in the application.
|
|
2983
3060
|
|
|
2984
3061
|
#### Example of configuration for a plugin
|
|
2985
3062
|
|
|
2986
3063
|
For Faspex 5, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
|
2987
3064
|
only username/password and URL are required (either on command line, or from configuration file).
|
|
2988
|
-
Those can be
|
|
3065
|
+
Those can be provided on the command line:
|
|
2989
3066
|
|
|
2990
3067
|
```shell
|
|
2991
|
-
ascli shares
|
|
3068
|
+
ascli shares files browse / --url=https://10.25.0.6 --username=john --password=<PASSWORD>
|
|
2992
3069
|
```
|
|
2993
3070
|
|
|
2994
3071
|
This can also be provisioned in a configuration file:
|
|
@@ -3004,7 +3081,7 @@ ascli config preset set shares06 password <PASSWORD>
|
|
|
3004
3081
|
This can also be done with one single command:
|
|
3005
3082
|
|
|
3006
3083
|
```shell
|
|
3007
|
-
ascli config preset
|
|
3084
|
+
ascli config preset initialize shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"<PASSWORD>"}'
|
|
3008
3085
|
```
|
|
3009
3086
|
|
|
3010
3087
|
Or:
|
|
@@ -3019,22 +3096,22 @@ ascli config preset update shares06 --url=https://10.25.0.6 --username=john --pa
|
|
|
3019
3096
|
ascli config preset set default shares shares06
|
|
3020
3097
|
```
|
|
3021
3098
|
|
|
3022
|
-
- Display the content of configuration file in table format
|
|
3099
|
+
- Display the content of the configuration file in table format
|
|
3023
3100
|
|
|
3024
3101
|
```shell
|
|
3025
3102
|
ascli config preset overview
|
|
3026
3103
|
```
|
|
3027
3104
|
|
|
3028
|
-
- Execute a command on the **Shares
|
|
3105
|
+
- Execute a command on the **Shares** application using default options
|
|
3029
3106
|
|
|
3030
3107
|
```shell
|
|
3031
|
-
ascli shares
|
|
3108
|
+
ascli shares files browse /
|
|
3032
3109
|
```
|
|
3033
3110
|
|
|
3034
3111
|
### Secret Vault
|
|
3035
3112
|
|
|
3036
3113
|
Secrets, for example, passwords, keys, are needed when connecting to applications.
|
|
3037
|
-
Those secrets are usually provided as command options
|
|
3114
|
+
Those secrets are usually provided as command options: on the command line, in env vars, in files, and so on.
|
|
3038
3115
|
|
|
3039
3116
|
For security reasons, those secrets shall not be exposed in clear, either:
|
|
3040
3117
|
|
|
@@ -3105,7 +3182,7 @@ vault server -dev -dev-root-token-id=dev-only-token
|
|
|
3105
3182
|
| `vault` | `String` | Name or ID of the 1Password vault to use with the CLI (used when `source` is `cli`). Uses the default vault if omitted. |
|
|
3106
3183
|
|
|
3107
3184
|
```shell
|
|
3108
|
-
--vault=@json:'{"type":"vault","url":"http://127.0.0.1:8200"}' --
|
|
3185
|
+
--vault=@json:'{"type":"vault","url":"http://127.0.0.1:8200"}' --vault-password=dev-only-token
|
|
3109
3186
|
```
|
|
3110
3187
|
|
|
3111
3188
|
#### Vault: System keychain
|
|
@@ -3159,7 +3236,7 @@ docker run -d --name op-connect \
|
|
|
3159
3236
|
|
|
3160
3237
|
```shell
|
|
3161
3238
|
--vault=@json:'{"type":"1password","source":"api","url":"http://localhost:8080","vault_id":"<VAULT_ID>"}' \
|
|
3162
|
-
--
|
|
3239
|
+
--vault-password=<CONNECT_TOKEN>
|
|
3163
3240
|
```
|
|
3164
3241
|
|
|
3165
3242
|
> [!TIP]
|
|
@@ -3175,7 +3252,7 @@ docker run -d --name op-connect \
|
|
|
3175
3252
|
<https://developer.1password.com/docs/cli/>
|
|
3176
3253
|
|
|
3177
3254
|
Requires the `op` CLI to be installed and signed in.
|
|
3178
|
-
No server to deploy
|
|
3255
|
+
No server to deploy: authentication is handled by the 1Password desktop app (biometric unlock) or by `op signin`.
|
|
3179
3256
|
|
|
3180
3257
|
```shell
|
|
3181
3258
|
--vault=@json:'{"type":"1password","source":"cli"}'
|
|
@@ -3191,11 +3268,14 @@ No server to deploy — authentication is handled by the 1Password desktop app (
|
|
|
3191
3268
|
|
|
3192
3269
|
Secrets can be manipulated using the `config vault` command:
|
|
3193
3270
|
|
|
3194
|
-
- `
|
|
3195
|
-
- `
|
|
3196
|
-
- `list`
|
|
3197
|
-
- `
|
|
3198
|
-
- `
|
|
3271
|
+
- `info` : Show vault information
|
|
3272
|
+
- `ids` : List secret labels in the vault
|
|
3273
|
+
- `list` : List all secrets with full details
|
|
3274
|
+
- `show` : Show a secret by label (or id)
|
|
3275
|
+
- `create` : Add a new secret to the vault
|
|
3276
|
+
- `delete` : Delete a secret by label (or id)
|
|
3277
|
+
- `password` : Change the vault password
|
|
3278
|
+
- `import` : Import secrets from a JSON array (supports `--bulk`)
|
|
3199
3279
|
|
|
3200
3280
|
To add a new password entry in the vault for label `<NAME>`:
|
|
3201
3281
|
|
|
@@ -3205,22 +3285,22 @@ ascli config vault create @: label=<NAME> password=@secret:password description=
|
|
|
3205
3285
|
|
|
3206
3286
|
#### Vault: Migration between vaults
|
|
3207
3287
|
|
|
3208
|
-
To migrate all secrets from one vault backend to another (for example, from the encrypted file vault to 1Password), use `vault
|
|
3288
|
+
To migrate all secrets from one vault backend to another (for example, from the encrypted file vault to 1Password), use `vault list` piped into `vault import`.
|
|
3209
3289
|
|
|
3210
3290
|
> [!NOTE]
|
|
3211
|
-
> Use `
|
|
3291
|
+
> Use `list` (not `ids`) as the source: `ids` returns only labels, while `list` returns the full secret details needed for import.
|
|
3212
3292
|
|
|
3213
3293
|
```shell
|
|
3214
|
-
ascli config vault
|
|
3294
|
+
ascli config vault list --format=json --out.level=data \
|
|
3215
3295
|
--vault=@json:'{"type":"file","name":"<SOURCE_VAULT_FILE>"}' \
|
|
3216
|
-
--
|
|
3217
|
-
ascli config vault import @json:@stdin: --bulk \
|
|
3296
|
+
--vault-password=<SOURCE_PASSWORD> | \
|
|
3297
|
+
ascli config vault import @json:@stdin: --bulk=yes \
|
|
3218
3298
|
--vault=@json:'{"type":"1password","url":"<CONNECT_URL>","vault_id":"<VAULT_ID>"}' \
|
|
3219
|
-
--
|
|
3299
|
+
--vault-password=<CONNECT_TOKEN>
|
|
3220
3300
|
```
|
|
3221
3301
|
|
|
3222
3302
|
> [!TIP]
|
|
3223
|
-
> Use `--out.level=data` on the `
|
|
3303
|
+
> Use `--out.level=data` on the `list` command so that only the raw JSON array is written to stdout, with no table headers or status lines.
|
|
3224
3304
|
|
|
3225
3305
|
The `import` command accepts a JSON array where each element is a vault secret object (same schema as `create`).
|
|
3226
3306
|
`--bulk` makes each entry reported individually in the result table; omit it to get a single-line summary.
|
|
@@ -3272,12 +3352,12 @@ To disable this behavior for a single command, pass `--vault=@none:`.
|
|
|
3272
3352
|
Some Aspera applications allow the user to be authenticated using [Public Key Cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography):
|
|
3273
3353
|
|
|
3274
3354
|
- For SSH: Server
|
|
3275
|
-
- For OAuth JWT: AoC,
|
|
3355
|
+
- For OAuth JWT: AoC, Faspex 5, faspio Gateway
|
|
3276
3356
|
|
|
3277
|
-
It consists
|
|
3357
|
+
It consists of using a pair of associated keys: a private key and a public key.
|
|
3278
3358
|
The same pair can be used for multiple applications.
|
|
3279
3359
|
The file containing the private key (key pair) can optionally be protected by a passphrase.
|
|
3280
|
-
If the key is protected by a passphrase, then
|
|
3360
|
+
If the key is protected by a passphrase, then the passphrase is prompted when the key is used.
|
|
3281
3361
|
Some plugins support option `passphrase`.
|
|
3282
3362
|
|
|
3283
3363
|
By default, `ascli` does not support `ed25519` type, nor OpenSSH encoded keys.
|
|
@@ -3331,7 +3411,7 @@ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
|
|
|
3331
3411
|
|
|
3332
3412
|
#### `openssl`
|
|
3333
3413
|
|
|
3334
|
-
To generate a key pair with a passphrase the following can be used on any system:
|
|
3414
|
+
To generate a key pair with a passphrase, the following can be used on any system:
|
|
3335
3415
|
|
|
3336
3416
|
```shell
|
|
3337
3417
|
openssl genrsa -passout pass:_passphrase_here_ -out ${KEY_PAIR_PATH} 4096
|
|
@@ -3348,7 +3428,7 @@ openssl rsa -passin pass:_passphrase_here_ -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_
|
|
|
3348
3428
|
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
|
|
3349
3429
|
```
|
|
3350
3430
|
|
|
3351
|
-
To change (or add) the passphrase for a key
|
|
3431
|
+
To change (or add) the passphrase for a key:
|
|
3352
3432
|
|
|
3353
3433
|
```shell
|
|
3354
3434
|
openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
|
|
@@ -3366,13 +3446,13 @@ For example: <https://cryptotools.net/rsagen>
|
|
|
3366
3446
|
### Web service
|
|
3367
3447
|
|
|
3368
3448
|
Some plugins start a local web server.
|
|
3369
|
-
This server can serve HTTP or HTTPS (with certificate)
|
|
3449
|
+
This server can serve HTTP or HTTPS (with certificate).
|
|
3370
3450
|
|
|
3371
3451
|
The following parameters are supported:
|
|
3372
3452
|
|
|
3373
3453
|
| Parameter | Type | Default | Description |
|
|
3374
3454
|
|-------------------|----------|-------------------------|------------------------------------------------------------------|
|
|
3375
|
-
| `url` | `String` | `http://localhost:8080` | Base URL on which requests are
|
|
3455
|
+
| `url` | `String` | `http://localhost:8080` | Base URL on which requests are received; a path can be included. | <!-- markdownlint-disable-line -->
|
|
3376
3456
|
| `cert` | `String` | - | (HTTPS) Path to certificate file (with ext. `.pfx` or `.p12` for `PKCS12`). |
|
|
3377
3457
|
| `key` | `String` | - | (HTTPS) Path to private key file (PEM), or passphrase for `PKCS12`. |
|
|
3378
3458
|
| `chain` | `String` | - | (HTTPS) Path to certificate chain (PEM only). |
|
|
@@ -3381,7 +3461,7 @@ Parameter `url` (base URL) defines:
|
|
|
3381
3461
|
|
|
3382
3462
|
- If `http` or `https` is used
|
|
3383
3463
|
- The local port number (default 443 for HTTPS, 80 for HTTP)
|
|
3384
|
-
- The **base path**, that is, the path under which requests are received
|
|
3464
|
+
- The **base path**, that is, the path under which requests are received (useful for routing when a reverse proxy is used).
|
|
3385
3465
|
|
|
3386
3466
|
### Image and video thumbnails
|
|
3387
3467
|
|
|
@@ -3394,7 +3474,7 @@ This feature can be used:
|
|
|
3394
3474
|
- `coffee` and `image` commands of `config` plugin.
|
|
3395
3475
|
- Any displayed value which is a URL to image can be displayed with option `format` set to `image`
|
|
3396
3476
|
|
|
3397
|
-
The following options can be specified in
|
|
3477
|
+
The following options can be specified in option `--out.img`:
|
|
3398
3478
|
|
|
3399
3479
|
| Field | Type | Description |
|
|
3400
3480
|
|------------|---------|----------------------------------------------------------------------------------|
|
|
@@ -3428,12 +3508,12 @@ ascli config image @stdin:bin < A-team.jpg
|
|
|
3428
3508
|
Some actions may require the use of a graphical tool:
|
|
3429
3509
|
|
|
3430
3510
|
- A browser for Aspera on Cloud authentication (web auth method)
|
|
3431
|
-
- A text editor for configuration file
|
|
3511
|
+
- A text editor for editing the configuration file
|
|
3432
3512
|
|
|
3433
|
-
By default, `ascli` assumes that a graphical environment is available on Windows
|
|
3434
|
-
It is also possible to force the graphical mode with option `ui
|
|
3513
|
+
By default, `ascli` assumes that a graphical environment is available on Windows; on other systems, it relies on the presence of the `DISPLAY` environment variable.
|
|
3514
|
+
It is also possible to force the graphical mode with option `ui`:
|
|
3435
3515
|
|
|
3436
|
-
- `--ui=graphical` forces a graphical environment
|
|
3516
|
+
- `--ui=graphical` forces a graphical environment: a browser is opened for URLs, or a text editor for files.
|
|
3437
3517
|
- `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
|
3438
3518
|
|
|
3439
3519
|
### Logging, Debugging
|
|
@@ -3476,19 +3556,19 @@ The default formatter is:
|
|
|
3476
3556
|
- Display debugging log on `stdout`:
|
|
3477
3557
|
|
|
3478
3558
|
```shell
|
|
3479
|
-
ascli config
|
|
3559
|
+
ascli config preset overview --log-level=debug --logger=stdout
|
|
3480
3560
|
```
|
|
3481
3561
|
|
|
3482
3562
|
Or equivalently using dot-path notation:
|
|
3483
3563
|
|
|
3484
3564
|
```shell
|
|
3485
|
-
ascli config
|
|
3565
|
+
ascli config preset overview --log.level=debug --log.type=stdout
|
|
3486
3566
|
```
|
|
3487
3567
|
|
|
3488
3568
|
- Log errors to `syslog`:
|
|
3489
3569
|
|
|
3490
3570
|
```shell
|
|
3491
|
-
ascli config
|
|
3571
|
+
ascli config preset overview --log-level=error --logger=syslog
|
|
3492
3572
|
```
|
|
3493
3573
|
|
|
3494
3574
|
Or using the composite option in a preset:
|
|
@@ -3514,7 +3594,7 @@ It will display the exact content of HTTP requests and responses.
|
|
|
3514
3594
|
### HTTP socket parameters
|
|
3515
3595
|
|
|
3516
3596
|
To ignore SSL certificate for **any** address/port, use option: `insecure`, that is, `--insecure=yes`.
|
|
3517
|
-
To ignore SSL certificate for a list of specific address/port, use option `ignore_certificate`, set to an `Array` of
|
|
3597
|
+
To ignore SSL certificate for a list of specific address/port, use option `ignore_certificate`, set to an `Array` of URLs for which the certificate is ignored (only the address and port are matched), for example, `--ignore-certificate=@list:,https://127.0.0.1:9092`
|
|
3518
3598
|
|
|
3519
3599
|
> [!NOTE]
|
|
3520
3600
|
> Ignoring certificate also applies to `ascp` WSS.
|
|
@@ -3571,7 +3651,7 @@ Example:
|
|
|
3571
3651
|
|
|
3572
3652
|
### Proxy
|
|
3573
3653
|
|
|
3574
|
-
There are several types of network connections, each of them
|
|
3654
|
+
There are several types of network connections, each of them uses a different mechanism to define a (forward) **proxy**:
|
|
3575
3655
|
|
|
3576
3656
|
- REST calls (APIs) and HTTP Gateway
|
|
3577
3657
|
- `ascp` WSS and Legacy Aspera HTTP/S Fallback
|
|
@@ -3625,7 +3705,7 @@ ascli config proxy_check --fpac='function FindProxyForURL(url, host) {return "PR
|
|
|
3625
3705
|
```
|
|
3626
3706
|
|
|
3627
3707
|
```text
|
|
3628
|
-
|
|
3708
|
+
proxy://proxy.example.com:3128
|
|
3629
3709
|
```
|
|
3630
3710
|
|
|
3631
3711
|
```shell
|
|
@@ -3633,7 +3713,7 @@ ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
|
|
|
3633
3713
|
```
|
|
3634
3714
|
|
|
3635
3715
|
```text
|
|
3636
|
-
|
|
3716
|
+
proxy://proxy.example.com:8080
|
|
3637
3717
|
```
|
|
3638
3718
|
|
|
3639
3719
|
```shell
|
|
@@ -3641,7 +3721,7 @@ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.
|
|
|
3641
3721
|
```
|
|
3642
3722
|
|
|
3643
3723
|
```text
|
|
3644
|
-
|
|
3724
|
+
proxy://proxy.example.com:8080
|
|
3645
3725
|
```
|
|
3646
3726
|
|
|
3647
3727
|
If the proxy found with the PAC requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
|
@@ -3687,18 +3767,20 @@ By default, `ascli` uses the `ascp` binary found in **well known locations**, th
|
|
|
3687
3767
|
The `config` plugin allows finding and specifying the location of `ascp`.
|
|
3688
3768
|
It provides the following commands for `ascp` sub-command:
|
|
3689
3769
|
|
|
3690
|
-
- `show` :
|
|
3691
|
-
- `
|
|
3692
|
-
- `
|
|
3693
|
-
- `
|
|
3770
|
+
- `show` : Show the path of `ascp` used
|
|
3771
|
+
- `info` : Show information on `ascp` and its environment
|
|
3772
|
+
- `install` : Install the Transfer SDK (same as `config transferd install`)
|
|
3773
|
+
- `spec` : List transfer spec parameters supported by `ascp`
|
|
3774
|
+
- `schema` : Show the JSON schema of the transfer spec
|
|
3775
|
+
- `errors` : List known `ascp` errors and whether they are retry-able
|
|
3776
|
+
- `products list` : List Aspera transfer products available locally
|
|
3694
3777
|
|
|
3695
3778
|
#### Selection of `ascp` location for [`direct`](#agent-direct) agent
|
|
3696
3779
|
|
|
3697
3780
|
Option: `sdk_folder` is used to specify the location of `ascp`.
|
|
3698
|
-
|
|
3699
|
-
By default, `ascli` uses any found local product with `ascp`, including Transfer Daemon (SDK).
|
|
3781
|
+
By default, `ascli` uses `ascp` from the Transfer SDK installed in its configuration folder (see `config ascp install`).
|
|
3700
3782
|
|
|
3701
|
-
To override and use an alternate `ascp` path use option `sdk_folder` (`--sdk-folder=`)
|
|
3783
|
+
To override and use an alternate `ascp` path, use option `sdk_folder` (`--sdk-folder=`).
|
|
3702
3784
|
|
|
3703
3785
|
For a permanent change, set a global default.
|
|
3704
3786
|
For example, `<INSTALL_DIR>` could be `~/my_install_dir` on Linux, or `C:\Users\admin\.aspera\ascli\sdk` on Windows.
|
|
@@ -3710,9 +3792,8 @@ ascli config preset set GLOBAL sdk_folder <INSTALL_DIR>
|
|
|
3710
3792
|
```
|
|
3711
3793
|
|
|
3712
3794
|
```text
|
|
3713
|
-
|
|
3714
|
-
|
|
3715
|
-
Saved to default global preset global_common_defaults
|
|
3795
|
+
INFO Updated: global_common_defaults: sdk_folder <- <INSTALL_DIR>
|
|
3796
|
+
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
|
|
3716
3797
|
```
|
|
3717
3798
|
|
|
3718
3799
|
If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
|
|
@@ -3720,6 +3801,7 @@ If the path has spaces, read section: [Shell and Command line parsing](#command-
|
|
|
3720
3801
|
A special value `product:<PRODUCT_NAME>` can be used for option `sdk_folder`.
|
|
3721
3802
|
It specifies to use `ascp` from the given product name.
|
|
3722
3803
|
A special value for product name is `FIRST`, which means: use the first product found in the internal list.
|
|
3804
|
+
In that case, other files (SSH keys, `aspera.conf`, `transferd`) are still taken from the default SDK folder.
|
|
3723
3805
|
|
|
3724
3806
|
Locally installed Aspera products can be listed with:
|
|
3725
3807
|
|
|
@@ -3742,9 +3824,12 @@ To permanently use the `ascp` of a product:
|
|
|
3742
3824
|
|
|
3743
3825
|
```shell
|
|
3744
3826
|
ascli config preset set GLOBAL sdk_folder 'product:IBM Aspera Connect'
|
|
3745
|
-
|
|
3746
|
-
|
|
3747
|
-
|
|
3827
|
+
```
|
|
3828
|
+
|
|
3829
|
+
```text
|
|
3830
|
+
INFO Updated: default: config <- global_common_defaults
|
|
3831
|
+
INFO Updated: global_common_defaults: sdk_folder <- product:IBM Aspera Connect
|
|
3832
|
+
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
|
|
3748
3833
|
```
|
|
3749
3834
|
|
|
3750
3835
|
To show the path of currently used `ascp`:
|
|
@@ -3853,13 +3938,15 @@ All transfer agents support asynchronous mode:
|
|
|
3853
3938
|
| `node` | REST `ops/transfers/{id}` | Yes |
|
|
3854
3939
|
| `connect` | REST `transfers/info/{id}` (auto-discovered URL) | Yes |
|
|
3855
3940
|
| `transferd` | gRPC `monitor_transfers` | Yes |
|
|
3856
|
-
| `direct` | In-process thread state (re-queryable while process lives, e.g. MCP mode) | No
|
|
3857
|
-
| `httpgw` | In-process thread state (re-queryable while process lives, e.g. MCP mode) | No
|
|
3941
|
+
| `direct` | In-process thread state (re-queryable while process lives, e.g. MCP mode) | No: returns `unknown` after restart |
|
|
3942
|
+
| `httpgw` | In-process thread state (re-queryable while process lives, e.g. MCP mode) | No: returns `unknown` after restart |
|
|
3858
3943
|
|
|
3859
3944
|
For `direct` and `httpgw`, the transfer runs as a Ruby thread inside the `ascli` process.
|
|
3860
3945
|
The `job_id` is persisted on disk but the live thread state is only available as long as the same process is running.
|
|
3861
3946
|
If the process is restarted, `config transfer status` returns `unknown` for those jobs.
|
|
3862
3947
|
|
|
3948
|
+
Parameters of the agent are persisted, except secrets (e.g. `password` of agent `node`): `config transfer status` takes them from the current `transfer` option, if it is for the same agent (e.g. same option on command line, or in a preset).
|
|
3949
|
+
|
|
3863
3950
|
> [!NOTE]
|
|
3864
3951
|
> **`asynchronous` and MCP** - When `ascli` is used as an MCP server, an AI assistant calling
|
|
3865
3952
|
> a transfer command may time out or cancel the request and retry, causing duplicate transfers.
|
|
@@ -3886,7 +3973,7 @@ The `transfer` option accepts the following optional parameters to control multi
|
|
|
3886
3973
|
| `file_list` | `Bool` | If `true`, source paths are written to a temp file passed to `ascp` via `--file-list` or `--file-pair-list`.<br/>If `false`, source paths are placed directly on the `ascp` command line.<br/>Default: `true`. |
|
|
3887
3974
|
| `monitor` | `Bool` | Enable use of the `ascp` management port for transfer monitoring.<br/>Default: `true`. |
|
|
3888
3975
|
| `multi_incr_udp` | `Bool` | Multi session - Increment UDP port for each session.<br/>If `true`, each session uses a different UDP port starting at `fasp_port` (default: 33001).<br/>If `false`, all sessions use the same `fasp_port` (or `ascp` default).<br/>Default: `true` on Windows, `false` on other platforms. |
|
|
3889
|
-
| `quiet` | `Bool` | Suppress the `ascp` progress bar display.<br/>Default: `true`. |
|
|
3976
|
+
| `quiet` | `Bool` | Suppress the `ascp` progress bar display.<br/>If `false`, the progress bar of option `progress_bar` is not displayed, unless that option is set.<br/>Default: `true`. |
|
|
3890
3977
|
| `resume.iter_max` | `Integer` | Maximum number of retry attempts on error.<br/>Default: `7`. |
|
|
3891
3978
|
| `resume.sleep_factor` | `Integer` | Multiplier applied to sleep duration between consecutive retry attempts.<br/>Default: `2`. |
|
|
3892
3979
|
| `resume.sleep_initial` | `Integer` | Initial sleep duration (in seconds) before first retry.<br/>Default: `2`. |
|
|
@@ -3897,25 +3984,25 @@ The `transfer` option accepts the following optional parameters to control multi
|
|
|
3897
3984
|
| `trusted_certs` | `Array[String]` | List of trusted certificate repositories. |
|
|
3898
3985
|
| `wss` | `Bool` | Enable Web Socket Session when available.<br/>Default: `true`. |
|
|
3899
3986
|
|
|
3900
|
-
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max`
|
|
3987
|
+
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` times.
|
|
3901
3988
|
Sleep between iterations is given by the following formula where `iter_index` is the current iteration index, starting at 0:
|
|
3902
3989
|
|
|
3903
|
-
```
|
|
3904
|
-
|
|
3990
|
+
```text
|
|
3991
|
+
min( sleep_max, sleep_initial * sleep_factor ^ iter_index )
|
|
3905
3992
|
```
|
|
3906
3993
|
|
|
3907
|
-
|
|
3994
|
+
By default, the progress bar of `ascli` is displayed (see [Transfer progress bar](#transfer-progress-bar)): it aggregates all sessions of a multi-session transfer.
|
|
3995
|
+
|
|
3996
|
+
To display the native progress bar of `ascp` instead, set parameter `quiet` to `false`:
|
|
3908
3997
|
|
|
3909
3998
|
```shell
|
|
3910
|
-
--
|
|
3999
|
+
--transfer.quiet=false
|
|
3911
4000
|
```
|
|
3912
4001
|
|
|
3913
|
-
|
|
3914
|
-
|
|
4002
|
+
In that case, the progress bar of `ascli` is not displayed, unless option `progress_bar` is set.
|
|
4003
|
+
For multi-session transfers, each `ascp` process displays its own progress bar.
|
|
3915
4004
|
|
|
3916
|
-
|
|
3917
|
-
--transfer.monitor=false --transfer.quiet=false
|
|
3918
|
-
```
|
|
4005
|
+
To skip usage of management port (which disables the progress bar of `ascli`), set option `monitor` to `false`.
|
|
3919
4006
|
|
|
3920
4007
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (for example, WSS).
|
|
3921
4008
|
To use a custom certificate store, use the `trusted_certs` option of direct agent's option `transfer`.
|
|
@@ -3925,7 +4012,7 @@ To use `ascp`'s default, use option:
|
|
|
3925
4012
|
--transfer.trusted_certs=@none:
|
|
3926
4013
|
```
|
|
3927
4014
|
|
|
3928
|
-
Some transfer errors are considered **retry-able** (for example, timeout) and
|
|
4015
|
+
Some transfer errors are considered **retry-able** (for example, timeout) and others are not (for example, wrong password).
|
|
3929
4016
|
The list of known protocol errors and retry level can be listed:
|
|
3930
4017
|
|
|
3931
4018
|
```shell
|
|
@@ -3939,7 +4026,7 @@ ascli ... --transfer.wss=true --transfer.resume.iter_max=20
|
|
|
3939
4026
|
ascli ... --transfer.spawn_delay_sec=2.5 --transfer.multi_incr_udp=false
|
|
3940
4027
|
```
|
|
3941
4028
|
|
|
3942
|
-
|
|
4029
|
+
Parameter `ascp_args` can also be used to activate logging using option `-L` of `ascp`.
|
|
3943
4030
|
For example, to activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`):
|
|
3944
4031
|
|
|
3945
4032
|
```shell
|
|
@@ -3954,26 +4041,24 @@ To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--trans
|
|
|
3954
4041
|
> When transfer agent [`direct`](#agent-direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a temp file list, unless `--file-list` or `--file-pair-list` is already provided via `transfer` parameter `ascp_args`.
|
|
3955
4042
|
> To place source paths directly on the `ascp` command line instead of using a temp file, set `file_list` to `false` in `transfer`: `--transfer.file_list=false`.
|
|
3956
4043
|
|
|
3957
|
-
In addition to standard methods described in section [File List](#list-of-files-for-transfers), it is possible to specify the list of
|
|
3958
|
-
|
|
3959
|
-
- Using option `transfer` parameter `ascp_args`
|
|
4044
|
+
In addition to standard methods described in section [File List](#list-of-files-for-transfers), it is also possible to specify the list of files using `transfer` parameter `ascp_args`:
|
|
3960
4045
|
|
|
3961
4046
|
```shell
|
|
3962
4047
|
--sources=@ts --transfer=@json:'{"ascp_args":["--file-list","myfilelist"]}'
|
|
3963
4048
|
```
|
|
3964
4049
|
|
|
3965
4050
|
> [!NOTE]
|
|
3966
|
-
> File lists
|
|
4051
|
+
> File lists are shown here; similar options exist for file pair lists.
|
|
3967
4052
|
|
|
3968
4053
|
> [!NOTE]
|
|
3969
|
-
>
|
|
4054
|
+
> This method avoids the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file list is read and parsed, and a new file list is created in a temporary folder.
|
|
3970
4055
|
|
|
3971
4056
|
> [!NOTE]
|
|
3972
|
-
>
|
|
4057
|
+
> This method has limitations: it applies **only** to the [`direct`](#agent-direct) transfer agent (that is, local `ascp`) and not to Aspera on Cloud.
|
|
3973
4058
|
|
|
3974
4059
|
##### Agent: Direct: Management messages
|
|
3975
4060
|
|
|
3976
|
-
By default, `ascli` gets
|
|
4061
|
+
By default, `ascli` gets notifications from `ascp` on its management port.
|
|
3977
4062
|
This can be disabled with parameter: `monitor=false` of `transfer`.
|
|
3978
4063
|
|
|
3979
4064
|
It is also possible to send messages to `ascp` using this management port.
|
|
@@ -3996,19 +4081,19 @@ ps -axo pid,command|grep ascli|grep -v grep|cut -f1 -d' '
|
|
|
3996
4081
|
Example to change the target rate:
|
|
3997
4082
|
|
|
3998
4083
|
```shell
|
|
3999
|
-
echo '{"type":"RATE","
|
|
4084
|
+
echo '{"type":"RATE","rate":300000}' > ~/.aspera/ascli/send_67470
|
|
4000
4085
|
```
|
|
4001
4086
|
|
|
4002
4087
|
When `ascli` detects this file, it uses it during a transfer and then deletes it.
|
|
4003
4088
|
|
|
4004
4089
|
> [!NOTE]
|
|
4005
4090
|
> The JSON's keys use **snake case**, that is, lower case with `_` as word separator.
|
|
4006
|
-
> The list of message `type` can be found in `aspera/ascp/management.rb
|
|
4007
|
-
> The list of parameters (capitalized
|
|
4091
|
+
> The list of message `type` values can be found in `aspera/ascp/management.rb`: `OPERATIONS`.
|
|
4092
|
+
> The list of parameters is `PARAMETERS` (native names are capitalized, keys in the JSON file are in snake case).
|
|
4008
4093
|
|
|
4009
4094
|
##### Agent: Direct: `aspera.conf`: Virtual Links
|
|
4010
4095
|
|
|
4011
|
-
This agent supports a local configuration file
|
|
4096
|
+
This agent supports a local configuration file, `aspera.conf`, where Virtual links can be configured.
|
|
4012
4097
|
|
|
4013
4098
|
On a server (HSTS), the following commands can be used to set a global virtual link:
|
|
4014
4099
|
|
|
@@ -4019,7 +4104,7 @@ asconfigurator -x 'set_node_data;transfer_in_bandwidth_aggregate_trunk_id,1'
|
|
|
4019
4104
|
asconfigurator -x 'set_node_data;transfer_out_bandwidth_aggregate_trunk_id,2'
|
|
4020
4105
|
```
|
|
4021
4106
|
|
|
4022
|
-
|
|
4107
|
+
This command is not available on clients: instead, edit the file `aspera.conf` (its location is shown by `ascli config ascp info --fields=aspera_conf`) and modify the sections `default` and `trunks` like this for a global 100 Mbps virtual link (capacity is in kbps):
|
|
4023
4108
|
|
|
4024
4109
|
```xml
|
|
4025
4110
|
<?xml version='1.0' encoding='UTF-8'?>
|
|
@@ -4048,14 +4133,14 @@ But this command is not available on clients, so edit the file `aspera.conf`, yo
|
|
|
4048
4133
|
<name>in</name>
|
|
4049
4134
|
<on>true</on>
|
|
4050
4135
|
<capacity>
|
|
4051
|
-
<schedule format="ranges">
|
|
4136
|
+
<schedule format="ranges">100000</schedule>
|
|
4052
4137
|
</capacity>
|
|
4053
4138
|
</trunk>
|
|
4054
4139
|
<trunk>
|
|
4055
4140
|
<id>2</id>
|
|
4056
4141
|
<name>out</name>
|
|
4057
4142
|
<capacity>
|
|
4058
|
-
<schedule format="ranges">
|
|
4143
|
+
<schedule format="ranges">100000</schedule>
|
|
4059
4144
|
</capacity>
|
|
4060
4145
|
<on>true</on>
|
|
4061
4146
|
</trunk>
|
|
@@ -4084,7 +4169,7 @@ For example, to replace illegal character `|` with an underscore `_`:
|
|
|
4084
4169
|
ascli config ascp info --fields=aspera_conf
|
|
4085
4170
|
```
|
|
4086
4171
|
|
|
4087
|
-
Typically, it is located at `$HOME/sdk/aspera.conf`
|
|
4172
|
+
Typically, it is located at `$HOME/.aspera/sdk/aspera.conf`
|
|
4088
4173
|
|
|
4089
4174
|
1. Edit this file, and add the following line inside the XML section `CONF.default.file_system`:
|
|
4090
4175
|
|
|
@@ -4107,9 +4192,7 @@ The result should look like this:
|
|
|
4107
4192
|
</CONF>
|
|
4108
4193
|
```
|
|
4109
4194
|
|
|
4110
|
-
|
|
4111
|
-
|
|
4112
|
-
The parameter works as follows:
|
|
4195
|
+
According to the [documentation](https://www.ibm.com/docs/en/ahts/4.4.x?topic=reference-user-group-default-configurations), the parameter works as follows:
|
|
4113
4196
|
|
|
4114
4197
|
- The first character in the value is the replacement character.
|
|
4115
4198
|
- All other characters listed after it are the **illegal** ones to be replaced.
|
|
@@ -4130,8 +4213,8 @@ In this example:
|
|
|
4130
4213
|
|
|
4131
4214
|
So, for example:
|
|
4132
4215
|
|
|
4133
|
-
- `report|final?.txt`
|
|
4134
|
-
- `data*backup"2025".csv`
|
|
4216
|
+
- `report|final?.txt` becomes `report_final_.txt`
|
|
4217
|
+
- `data*backup"2025".csv` becomes `data_backup_2025_.csv`
|
|
4135
4218
|
|
|
4136
4219
|
#### Agent: Connect Client
|
|
4137
4220
|
|
|
@@ -4160,13 +4243,13 @@ Parameters provided in option `transfer` are:
|
|
|
4160
4243
|
Like any other option, `transfer` can get its value from a pre-configured [Option Preset](#option-preset):
|
|
4161
4244
|
|
|
4162
4245
|
```shell
|
|
4163
|
-
--transfer=@preset:_name_here_
|
|
4246
|
+
--transfer=@preset:_name_here_ --transfer.agent=node
|
|
4164
4247
|
```
|
|
4165
4248
|
|
|
4166
4249
|
It can also directly use the [Extended Value](#extended-value-syntax) syntax:
|
|
4167
4250
|
|
|
4168
4251
|
```shell
|
|
4169
|
-
--transfer=@json:'{"url":"https://...","username":"_user_here_","password":"<PASSWORD>"}'
|
|
4252
|
+
--transfer=@json:'{"agent":"node","url":"https://...","username":"_user_here_","password":"<PASSWORD>"}'
|
|
4170
4253
|
```
|
|
4171
4254
|
|
|
4172
4255
|
If `transfer` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
|
@@ -4192,7 +4275,7 @@ Parameters provided in option `transfer` are:
|
|
|
4192
4275
|
Example:
|
|
4193
4276
|
|
|
4194
4277
|
```shell
|
|
4195
|
-
ascli faspex5 packages
|
|
4278
|
+
ascli faspex5 packages receive 323 --transfer=httpgw --transfer.url=https://asperagw.example.com:9443/aspera/http-gwy
|
|
4196
4279
|
```
|
|
4197
4280
|
|
|
4198
4281
|
> [!NOTE]
|
|
@@ -4220,7 +4303,7 @@ Options for `transfer` are:
|
|
|
4220
4303
|
For example, to use an external, already running `transferd`, use option:
|
|
4221
4304
|
|
|
4222
4305
|
```shell
|
|
4223
|
-
--transfer=@json:'{"url":":55002","start":false,"stop":false}'
|
|
4306
|
+
--transfer=@json:'{"agent":"transferd","url":":55002","start":false,"stop":false}'
|
|
4224
4307
|
```
|
|
4225
4308
|
|
|
4226
4309
|
The gem `grpc` is not part of default dependencies, as it requires compilation of a native part.
|
|
@@ -4266,7 +4349,7 @@ All parameters necessary for this transfer are described in a [**transfer-spec**
|
|
|
4266
4349
|
`ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
|
|
4267
4350
|
It is not necessary to provide additional parameters on the command line for a transfer.
|
|
4268
4351
|
|
|
4269
|
-
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification)
|
|
4352
|
+
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameters using the `ts` option.
|
|
4270
4353
|
The `ts` option accepts a `Hash` [Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
|
|
4271
4354
|
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
|
4272
4355
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
|
@@ -4293,10 +4376,10 @@ Or an equivalent (using dotted expression):
|
|
|
4293
4376
|
|
|
4294
4377
|
This is especially useful for `ascp` command line parameters not supported in the transfer spec.
|
|
4295
4378
|
|
|
4296
|
-
The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the
|
|
4379
|
+
The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the following advantages:
|
|
4297
4380
|
|
|
4298
|
-
-
|
|
4299
|
-
-
|
|
4381
|
+
- It is common to all [Transfer Agents](#transfer-clients-agents)
|
|
4382
|
+
- It does not depend on command line limitations (special characters, and so on)
|
|
4300
4383
|
|
|
4301
4384
|
#### Transfer Parameters
|
|
4302
4385
|
|
|
@@ -4328,7 +4411,7 @@ An optional parameter can be specified to display the schema for a specific tran
|
|
|
4328
4411
|
ascli config ascp schema transferd --format=jsonpp
|
|
4329
4412
|
```
|
|
4330
4413
|
|
|
4331
|
-
`ascp` argument or environment variable
|
|
4414
|
+
The description gives the corresponding `ascp` argument or environment variable.
|
|
4332
4415
|
|
|
4333
4416
|
#### Transfer Specification Reference
|
|
4334
4417
|
|
|
@@ -4463,7 +4546,6 @@ The `sources` and `src_type` options provide convenient ways to populate the tra
|
|
|
4463
4546
|
Possible values for option `sources` are:
|
|
4464
4547
|
|
|
4465
4548
|
- `@args` : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with `-`) are considered as source files.
|
|
4466
|
-
By default, the list of files to transfer is specified on the command line.
|
|
4467
4549
|
|
|
4468
4550
|
> [!IMPORTANT]
|
|
4469
4551
|
> When using `@:` to build a command parameter and `--sources=@args` (default),
|
|
@@ -4473,7 +4555,7 @@ By default, the list of files to transfer is specified on the command line.
|
|
|
4473
4555
|
**Example**:
|
|
4474
4556
|
|
|
4475
4557
|
```shell
|
|
4476
|
-
ascli server upload ~/
|
|
4558
|
+
ascli server upload ~/mysample.file secondfile
|
|
4477
4559
|
```
|
|
4478
4560
|
|
|
4479
4561
|
This is the same as (with default values):
|
|
@@ -4500,7 +4582,7 @@ By default, the list of files to transfer is specified on the command line.
|
|
|
4500
4582
|
|
|
4501
4583
|
Use the file list: one path per line:
|
|
4502
4584
|
|
|
4503
|
-
```
|
|
4585
|
+
```shell
|
|
4504
4586
|
--sources=@lines:@file:myfilelist.txt
|
|
4505
4587
|
```
|
|
4506
4588
|
|
|
@@ -4549,7 +4631,7 @@ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
|
|
4549
4631
|
|
|
4550
4632
|
#### Source directory structure on destination
|
|
4551
4633
|
|
|
4552
|
-
This section is not specific to `ascli
|
|
4634
|
+
This section is not specific to `ascli`: it describes `ascp` behavior.
|
|
4553
4635
|
|
|
4554
4636
|
The transfer destination is normally expected to designate a destination folder.
|
|
4555
4637
|
|
|
@@ -4560,11 +4642,11 @@ But there is one exception: The destination specifies the new item name when the
|
|
|
4560
4642
|
- Destination is not an existing folder
|
|
4561
4643
|
- The `dirname` of destination is an existing folder
|
|
4562
4644
|
|
|
4563
|
-
For this reason it is recommended to set `create_dir` to `true` for consistent behavior between single and multiple
|
|
4645
|
+
For this reason, it is recommended to set `create_dir` to `true` for consistent behavior between single and multiple item transfers; this is the default in `ascli`.
|
|
4564
4646
|
|
|
4565
4647
|
If a simple source file list is provided (no `destination` in `paths`, that is, no `file_pair_list` provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.
|
|
4566
4648
|
|
|
4567
|
-
The inner structure of source items that are
|
|
4649
|
+
The inner structure of source items that are folders is preserved on destination.
|
|
4568
4650
|
|
|
4569
4651
|
A leading `/` on destination is ignored (relative to docroot) unless docroot is not set (relative to home).
|
|
4570
4652
|
|
|
@@ -4598,11 +4680,11 @@ Advanced Example: Send files `./file1` and `./folder2/files2` to server (for exa
|
|
|
4598
4680
|
then destination will be: `/Upload/file1 /Upload/files2`
|
|
4599
4681
|
|
|
4600
4682
|
- One possibility is to specify a file pair list: `--src-type=pair file1 file1 folder2/files2 folder2/files2`
|
|
4601
|
-
- Another possibility is to specify a source base: `--
|
|
4683
|
+
- Another possibility is to specify a source base (transfer spec parameter `src_base`): `--ts.src_base=$PWD $PWD/file1 $PWD/folder2/files2`
|
|
4602
4684
|
|
|
4603
4685
|
The `.` path cannot be used as a source base.
|
|
4604
4686
|
|
|
4605
|
-
- Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--
|
|
4687
|
+
- Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--ts.src_base=tmp_base tmp_base/file1 tmp_base/folder2/files2`
|
|
4606
4688
|
- One can also similarly use `--sources=@ts` and specify the list of files in the `paths` field of transfer spec with both `source` and `destination` for each file.
|
|
4607
4689
|
|
|
4608
4690
|
#### Multi-session transfer
|
|
@@ -4632,7 +4714,7 @@ When multi-session is used, one separate UDP port is used per session (refer to
|
|
|
4632
4714
|
|
|
4633
4715
|
#### Content protection
|
|
4634
4716
|
|
|
4635
|
-
Content protection (Client-Side Encryption at
|
|
4717
|
+
Content protection (Client-Side Encryption at Rest, CSEAR) ensures that files remain encrypted while stored on the server.
|
|
4636
4718
|
With CSEAR, the client encrypts files during upload and decrypts files during download, using a passphrase known only to the users sharing the files.
|
|
4637
4719
|
|
|
4638
4720
|
- Upload: Files are encrypted on the client side before being sent to the server.
|
|
@@ -4640,7 +4722,7 @@ With CSEAR, the client encrypts files during upload and decrypts files during do
|
|
|
4640
4722
|
|
|
4641
4723
|
At all times, files remain encrypted on the server; encryption and decryption occur exclusively on the client side.
|
|
4642
4724
|
|
|
4643
|
-
Activating CSEAR consists
|
|
4725
|
+
Activating CSEAR consists of setting transfer spec parameters:
|
|
4644
4726
|
|
|
4645
4727
|
- `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
|
|
4646
4728
|
- `content_protection_password` : the passphrase to be used.
|
|
@@ -4684,9 +4766,19 @@ Example: parameter to download a Faspex package and decrypt on the fly
|
|
|
4684
4766
|
|
|
4685
4767
|
### Transfer progress bar
|
|
4686
4768
|
|
|
4687
|
-
File transfer operations are monitored, and a progress bar is displayed on the
|
|
4769
|
+
File transfer operations are monitored, and a progress bar is displayed on the standard error if option `progress_bar` (`Bool`) is set to `yes` (default if the standard error is a terminal).
|
|
4770
|
+
So, the progress bar is displayed even if the output of the command is redirected to a file.
|
|
4771
|
+
|
|
4772
|
+
The same progress bar is used for any type of transfer: using `ascp`, server to server, using HTTPS, and so on.
|
|
4773
|
+
|
|
4774
|
+
It shows the elapsed time, the percentage, the rate in megabits per second (`Mbps`, 1,000,000 bits per second, averaged over the last 5 seconds) and the estimated remaining time.
|
|
4775
|
+
As long as the total size is not known (for example, when the job size is not pre-calculated), the transferred size is shown instead of the percentage.
|
|
4776
|
+
For multi-session transfers, sessions are aggregated, and the number of running sessions is shown in brackets.
|
|
4777
|
+
Files already at destination (resumed transfer) count in the progress, but not in the rate.
|
|
4778
|
+
If the transfer fails, the progress bar stops at the reached progress, and shows `failed`.
|
|
4779
|
+
Log lines are displayed above the progress bar.
|
|
4688
4780
|
|
|
4689
|
-
|
|
4781
|
+
Agent `direct` can display the native progress bar of `ascp` instead (see [`direct`](#agent-direct)).
|
|
4690
4782
|
|
|
4691
4783
|
### Scheduler
|
|
4692
4784
|
|
|
@@ -4695,9 +4787,9 @@ Automated execution should therefore rely on operating system facilities.
|
|
|
4695
4787
|
|
|
4696
4788
|
Two common execution modes are supported:
|
|
4697
4789
|
|
|
4698
|
-
- Scheduled execution
|
|
4790
|
+
- Scheduled execution: run `ascli` commands periodically.
|
|
4699
4791
|
|
|
4700
|
-
- Daemon/service mode
|
|
4792
|
+
- Daemon/service mode: run `ascli` continuously as a server.
|
|
4701
4793
|
|
|
4702
4794
|
#### Creating a wrapping script
|
|
4703
4795
|
|
|
@@ -4734,9 +4826,9 @@ Windows provides the [Task Scheduler](https://docs.microsoft.com/en-us/windows/w
|
|
|
4734
4826
|
|
|
4735
4827
|
Tasks can be configured using:
|
|
4736
4828
|
|
|
4737
|
-
- [`schtasks.exe`](https://learn.microsoft.com/
|
|
4829
|
+
- [`schtasks.exe`](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/schtasks-create)
|
|
4738
4830
|
|
|
4739
|
-
- PowerShell
|
|
4831
|
+
- PowerShell module [`ScheduledTasks`](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
|
4740
4832
|
|
|
4741
4833
|
- `taskschd.msc` (UI)
|
|
4742
4834
|
|
|
@@ -4745,7 +4837,7 @@ By default, Windows Task Scheduler prevents overlapping executions.
|
|
|
4745
4837
|
#### Linux: `systemd` Timer
|
|
4746
4838
|
|
|
4747
4839
|
Most modern Linux distributions use `systemd` which provides scheduling via [`timer`](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) units.
|
|
4748
|
-
Define a name for the
|
|
4840
|
+
Define a name for the job, for example: `ascli_job` as `<NAME>` below.
|
|
4749
4841
|
|
|
4750
4842
|
1. Create the service
|
|
4751
4843
|
|
|
@@ -4811,7 +4903,7 @@ Example of `crontab` for user `xfer`.
|
|
|
4811
4903
|
```shell
|
|
4812
4904
|
crontab<<EOF
|
|
4813
4905
|
0 * * * * /home/xfer/bin/ascli_tool preview scan --logger=syslog --out.level=error
|
|
4814
|
-
2-59 * * * * /home/xfer/bin/ascli_tool preview
|
|
4906
|
+
2-59 * * * * /home/xfer/bin/ascli_tool preview trevents --logger=syslog --out.level=error
|
|
4815
4907
|
EOF
|
|
4816
4908
|
```
|
|
4817
4909
|
|
|
@@ -4823,7 +4915,7 @@ Linux also provides `anacron` for daily or hourly jobs that must run even if the
|
|
|
4823
4915
|
#### Running as system service (Daemon mode)
|
|
4824
4916
|
|
|
4825
4917
|
Some commands run continuously (for example, listening on a network port).
|
|
4826
|
-
In this case it is recommended to run `ascli` as a system service.
|
|
4918
|
+
In this case, it is recommended to run `ascli` as a system service.
|
|
4827
4919
|
|
|
4828
4920
|
On Linux this is typically done using [`systemd`](https://systemd.io/).
|
|
4829
4921
|
|
|
@@ -4900,7 +4992,7 @@ ascli config echo @ruby:'sleep 30' --lock-port=12345
|
|
|
4900
4992
|
|
|
4901
4993
|
- The second instance will exit immediately with:
|
|
4902
4994
|
|
|
4903
|
-
```
|
|
4995
|
+
```text
|
|
4904
4996
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
|
4905
4997
|
```
|
|
4906
4998
|
|
|
@@ -4975,7 +5067,7 @@ To activate a PVCL library, place the corresponding shared library in the same f
|
|
|
4975
5067
|
Example:
|
|
4976
5068
|
|
|
4977
5069
|
```shell
|
|
4978
|
-
cp /opt/aspera/lib/pvcl/libpvcl_cloud.so $(ascli
|
|
5070
|
+
cp /opt/aspera/lib/pvcl/libpvcl_cloud.so $(ascli config ascp info --fields=root)
|
|
4979
5071
|
```
|
|
4980
5072
|
|
|
4981
5073
|
Then check available modules as shown previously (`ascp info`).
|
|
@@ -5003,7 +5095,7 @@ Where:
|
|
|
5003
5095
|
> Characters `?` and `&` are shell special characters (wildcard and background), so `faux` file specification on command line should be protected (using quotes or `\`).
|
|
5004
5096
|
> If not, the shell may give error: `no matches found` or equivalent.
|
|
5005
5097
|
|
|
5006
|
-
For all sizes, a suffix can be added (case-insensitive) to the size: k, m, g, t, p, e (values are
|
|
5098
|
+
For all sizes, a suffix can be added (case-insensitive) to the size: k, m, g, t, p, e (values are powers of 2, for example, 1M is 2<sup>20</sup>, that is, 1 mebibyte, not megabyte).
|
|
5007
5099
|
The maximum allowed value is 8\*2<sup>60</sup>.
|
|
5008
5100
|
Extremely large `faux` file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is `faux://`.
|
|
5009
5101
|
|
|
@@ -5048,7 +5140,7 @@ Filenames generated are of the form: `<FILE>_<00000 ... count>_<FILESIZE>`
|
|
|
5048
5140
|
|
|
5049
5141
|
Examples:
|
|
5050
5142
|
|
|
5051
|
-
- Upload 20
|
|
5143
|
+
- Upload 20 gibibytes of generated data to file `myfile` in directory `/Upload`
|
|
5052
5144
|
|
|
5053
5145
|
```shell
|
|
5054
5146
|
ascli server upload faux:///myfile\?20g --to-folder=/Upload
|
|
@@ -5091,11 +5183,11 @@ See [HSTS `ascp` command reference](https://www.ibm.com/docs/en/ahts/4.4.x?topic
|
|
|
5091
5183
|
|
|
5092
5184
|
Key query parameters:
|
|
5093
5185
|
|
|
5094
|
-
| Parameter | Description
|
|
5095
|
-
|
|
5096
|
-
| `grow` | **(Required)** Wait time in seconds after last file change before the transfer is declared complete
|
|
5097
|
-
| `wait_start` | How the wait time is measured
|
|
5098
|
-
| `confirm_stop` | Set to `true` to let an external program signal completion by setting
|
|
5186
|
+
| Parameter | Description |
|
|
5187
|
+
|----------------|----------------------------------------------------------------------|
|
|
5188
|
+
| `grow` | **(Required)** Wait time in seconds after last file change before the transfer is declared complete.<br/>Default wait time is 10 s if set to a non-numeric string. |
|
|
5189
|
+
| `wait_start` | How the wait time is measured:<br/>- `mtime` (default) file modification time<br/>- `null_read` first zero-byte read. |
|
|
5190
|
+
| `confirm_stop` | Set to `true` to let an external program signal completion by setting:<br/>`mtime < current_time - wait_time`.<br/>Ignored when `wait_start=null_read`. |
|
|
5099
5191
|
|
|
5100
5192
|
> [!NOTE]
|
|
5101
5193
|
> `ascp` requires that all sources in a single transfer session share the same PVCL URI scheme.
|
|
@@ -5107,13 +5199,13 @@ Key query parameters:
|
|
|
5107
5199
|
Place only the bare filename(s) in the file list, and pass the `file:` URI as the source prefix so that the query parameters apply uniformly to every entry:
|
|
5108
5200
|
|
|
5109
5201
|
```shell
|
|
5110
|
-
ascli server upload growing --to-folder=/Upload --ts.source_root='file:///?grow=120' --
|
|
5202
|
+
ascli server upload growing --to-folder=/Upload --ts.source_root='file:///?grow=120' --transfer.quiet=false
|
|
5111
5203
|
```
|
|
5112
5204
|
|
|
5113
5205
|
- **URI directly on the command line with `file_list=false`**
|
|
5114
5206
|
|
|
5115
5207
|
```shell
|
|
5116
|
-
ascli server upload 'file:///./growing?grow=120' --to-folder=/Upload --transfer.file_list=false --transfer.quiet=false
|
|
5208
|
+
ascli server upload 'file:///./growing?grow=120' --to-folder=/Upload --transfer.file_list=false --transfer.quiet=false
|
|
5117
5209
|
```
|
|
5118
5210
|
|
|
5119
5211
|
### Usage
|
|
@@ -5121,7 +5213,7 @@ Key query parameters:
|
|
|
5121
5213
|
```text
|
|
5122
5214
|
ascli -h
|
|
5123
5215
|
NAME
|
|
5124
|
-
ascli -- a command line tool for Aspera Applications (v4.27.
|
|
5216
|
+
ascli -- a command line tool for Aspera Applications (v4.27.4)
|
|
5125
5217
|
|
|
5126
5218
|
SYNOPSIS
|
|
5127
5219
|
ascli [GLOBAL_OPTIONS] <command> [OPTIONS] [ARGS]
|
|
@@ -5152,17 +5244,17 @@ ARGS
|
|
|
5152
5244
|
OPTIONS: global
|
|
5153
5245
|
--interactive=yes|no Use interactive input of missing params
|
|
5154
5246
|
--ask-options=yes|no Ask even optional options
|
|
5155
|
-
--out=HASH Output rendering options (dot-notation: format, level, file, fields, select, table[.pivot], flat, secrets, img)
|
|
5156
|
-
--display=info|data|error Output only some information (deprecated: use --out.level)
|
|
5247
|
+
--out=HASH Output rendering options (dot-notation: format, level, file, fields, select, table[.pivot], flat, secrets, colors, utf8, img)
|
|
5248
|
+
--display=info|data|error Output only some information (deprecated after 4.27.0: use --out.level)
|
|
5157
5249
|
--format=ENUM Output format (also: --out.format)
|
|
5158
|
-
--output=VALUE Destination for results (deprecated: use --out.file)
|
|
5250
|
+
--output=VALUE Destination for results (deprecated after 4.27.0: use --out.file)
|
|
5159
5251
|
--fields=LIST Comma separated list of: fields, or ALL, or DEF (also: --out.fields)
|
|
5160
5252
|
--select=HASH Select only some items in lists: column, value (also: --out.select)
|
|
5161
|
-
--table-style=HASH (Table) Display style (deprecated: use --out.table)
|
|
5162
|
-
--flat-hash=yes|no (Table) Display deep values as additional keys (deprecated: use --out.flat)
|
|
5163
|
-
--multi-single=no|yes|single (Table) Control how object list is displayed as single table, or multiple objects (deprecated: use --out.table.pivot)
|
|
5164
|
-
--show-secrets=yes|no Show secrets on command output (deprecated: use --out.secrets)
|
|
5165
|
-
--image=HASH Options for displaying images and thumbnails in the terminal (deprecated: use --out.img)
|
|
5253
|
+
--table-style=HASH (Table) Display style (deprecated after 4.27.0: use --out.table)
|
|
5254
|
+
--flat-hash=yes|no (Table) Display deep values as additional keys (deprecated after 4.27.0: use --out.flat)
|
|
5255
|
+
--multi-single=no|yes|single (Table) Control how object list is displayed as single table, or multiple objects (deprecated after 4.27.0: use --out.table.pivot)
|
|
5256
|
+
--show-secrets=yes|no Show secrets on command output (deprecated after 4.27.0: use --out.secrets)
|
|
5257
|
+
--image=HASH Options for displaying images and thumbnails in the terminal (deprecated after 4.27.0: use --out.img)
|
|
5166
5258
|
-h, --help Show this message
|
|
5167
5259
|
--show-config Display parameters used for the provided action
|
|
5168
5260
|
-v, --version Display version
|
|
@@ -5198,9 +5290,7 @@ OPTIONS: global
|
|
|
5198
5290
|
--notify-to=VALUE Email: Recipient for notification of transfers
|
|
5199
5291
|
--notify-template=VALUE Email: ERB template for notification of transfers
|
|
5200
5292
|
--cache-tokens=yes|no Save and reuse OAuth tokens
|
|
5201
|
-
--
|
|
5202
|
-
--bulk=yes|no Bulk operation (only some)
|
|
5203
|
-
--bfail=yes|no Bulk operation error handling
|
|
5293
|
+
--expand-mounts=yes|no Commands: list commands of sub-trees provided by another plugin
|
|
5204
5294
|
-N, --no-default Do not load default configuration for plugin
|
|
5205
5295
|
--override=yes|no Wizard: override existing value
|
|
5206
5296
|
--default=yes|no Wizard: set as default configuration for specified plugin (also: update)
|
|
@@ -5211,12 +5301,15 @@ OPTIONS: global
|
|
|
5211
5301
|
--cert-stores=LIST HTTP/S: List of folder with trusted certificates
|
|
5212
5302
|
--http-options=HASH HTTP/S connection parameters for REST calls (not `ascp` WSS)
|
|
5213
5303
|
--http-proxy=VALUE HTTP/S: URL for proxy with optional credentials
|
|
5304
|
+
--query=HASH Additional filter for for some commands (list/delete)
|
|
5305
|
+
--bulk=yes|no Bulk operation (only some)
|
|
5306
|
+
--bfail=yes|no Bulk operation error handling
|
|
5214
5307
|
--ts=HASH Override transfer spec values
|
|
5215
5308
|
--to-folder=VALUE Destination folder for transferred files
|
|
5216
5309
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
|
5217
5310
|
--src-type=list|pair Type of file list
|
|
5218
5311
|
--transfer=HASH Transfer agent type, or agent parameters with optional agent key
|
|
5219
|
-
--transfer-info=HASH Parameters for transfer agent (deprecated: use --transfer instead)
|
|
5312
|
+
--transfer-info=HASH Parameters for transfer agent (deprecated after 4.26.2: use --transfer instead)
|
|
5220
5313
|
|
|
5221
5314
|
PLUGINS
|
|
5222
5315
|
alee Aspera License Entitlement Engine
|
|
@@ -5241,12 +5334,12 @@ PLUGINS
|
|
|
5241
5334
|
|
|
5242
5335
|
Bulk creation and deletion of resources are possible using option `bulk` (`yes`,`no`(default)).
|
|
5243
5336
|
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value](#extended-value-syntax) syntax.
|
|
5244
|
-
This option is available only for some resources: if you need it
|
|
5337
|
+
This option is available only for some resources: if you need it, try and see if the entities you create or delete support this option.
|
|
5245
5338
|
|
|
5246
5339
|
### Option: `query`
|
|
5247
5340
|
|
|
5248
5341
|
The `query` option can generally be used to add URL parameters to commands that list resources.
|
|
5249
|
-
It takes
|
|
5342
|
+
It takes a `Hash`, corresponding to key/value pairs that appear in the query part of the request.
|
|
5250
5343
|
|
|
5251
5344
|
For example: `--query=@json:'{"p1":"v1","p2":"v2"}'` leads to query: `?p1=v1&p2=v2`.
|
|
5252
5345
|
|
|
@@ -5270,17 +5363,17 @@ Each plugin usually represents commands sent to a specific application.
|
|
|
5270
5363
|
Available plugins can be found using command:
|
|
5271
5364
|
|
|
5272
5365
|
```shell
|
|
5273
|
-
ascli config
|
|
5366
|
+
ascli config plugins list
|
|
5274
5367
|
```
|
|
5275
5368
|
|
|
5276
5369
|
```text
|
|
5277
|
-
|
|
5278
|
-
|
|
5279
|
-
|
|
5280
|
-
|
|
5281
|
-
|
|
5370
|
+
╭────────┬────────┬────────┬─────────────────────────────────────────────────╮
|
|
5371
|
+
│ plugin │ detect │ wizard │ path │
|
|
5372
|
+
╞════════╪════════╪════════╪═════════════════════════════════════════════════╡
|
|
5373
|
+
│ shares │ ✓ │ ✓ │ .../aspera-cli/lib/aspera/cli/plugins/shares.rb │
|
|
5374
|
+
│ node │ ✓ │ ✓ │ .../aspera-cli/lib/aspera/cli/plugins/node.rb │
|
|
5282
5375
|
...
|
|
5283
|
-
|
|
5376
|
+
╰────────┴────────┴────────┴─────────────────────────────────────────────────╯
|
|
5284
5377
|
```
|
|
5285
5378
|
|
|
5286
5379
|
Most plugins will take the URL option: `url` to identify their location.
|
|
@@ -5289,7 +5382,7 @@ REST APIs of Aspera legacy applications (Aspera Node, Shares, Console, Orchestra
|
|
|
5289
5382
|
|
|
5290
5383
|
Aspera on Cloud and Faspex 5 rely on OAuth.
|
|
5291
5384
|
|
|
5292
|
-
By default, plugins are looked
|
|
5385
|
+
By default, plugins are looked up in folders specified by (multi-value) option `plugin_folder`:
|
|
5293
5386
|
|
|
5294
5387
|
```shell
|
|
5295
5388
|
ascli --show-config --fields=plugin_folder
|
|
@@ -5298,13 +5391,14 @@ ascli --show-config --fields=plugin_folder
|
|
|
5298
5391
|
You can create the skeleton of a new plugin like this:
|
|
5299
5392
|
|
|
5300
5393
|
```shell
|
|
5301
|
-
ascli config
|
|
5394
|
+
ascli config plugins create foo .
|
|
5302
5395
|
```
|
|
5303
5396
|
|
|
5304
5397
|
```text
|
|
5305
5398
|
Created ./foo.rb
|
|
5306
5399
|
```
|
|
5307
5400
|
|
|
5401
|
+
|
|
5308
5402
|
```shell
|
|
5309
5403
|
ascli --plugin-folder=. foo
|
|
5310
5404
|
```
|
|
@@ -5323,7 +5417,7 @@ A Transfer Agent is used by setting the option `transfer` (for example, `--trans
|
|
|
5323
5417
|
|
|
5324
5418
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
|
5325
5419
|
`ascli` receives its arguments on the command line.
|
|
5326
|
-
The way arguments are parsed and provided to `ascli`
|
|
5420
|
+
The way arguments are parsed and provided to `ascli` depends on the operating system and shell.
|
|
5327
5421
|
|
|
5328
5422
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
|
5329
5423
|
|
|
@@ -5332,10 +5426,10 @@ It is fully documented in the shell's documentation.
|
|
|
5332
5426
|
|
|
5333
5427
|
On Unix-like environments, this is typically a POSIX-like shell (`bash`, `zsh`, `ksh`, `sh`).
|
|
5334
5428
|
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
|
5335
|
-
In this environment the shell parses the command line, possibly replacing variables, and so on
|
|
5429
|
+
In this environment, the shell parses the command line, possibly replacing variables, and so on.
|
|
5336
5430
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
|
5337
5431
|
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
|
5338
|
-
Ruby receives
|
|
5432
|
+
Ruby receives the list of command line arguments from the shell and gives it to `ascli`.
|
|
5339
5433
|
Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
|
|
5340
5434
|
|
|
5341
5435
|
#### Shell parsing for Windows
|
|
@@ -5377,10 +5471,10 @@ It's up to the program to split arguments:
|
|
|
5377
5471
|
|
|
5378
5472
|
`ascli` is a Ruby program, so Ruby parses the command line (received with `GetCommandLineW`) into arguments and provides them to the Ruby code (`$0` and `ARGV`).
|
|
5379
5473
|
Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
|
|
5380
|
-
(See `w32_cmdvector` in Ruby source [`win32.c`](https://github.com/ruby/ruby/blob/master/win32/win32.c#L1766))
|
|
5474
|
+
(See `w32_cmdvector` in Ruby source [`win32.c`](https://github.com/ruby/ruby/blob/master/win32/win32.c#L1766)): <!--cspell:disable-line-->
|
|
5381
5475
|
|
|
5382
5476
|
- Space characters: split arguments (space, tab, newline)
|
|
5383
|
-
- Backslash: `\`
|
|
5477
|
+
- Backslash: `\` escapes a single special character
|
|
5384
5478
|
- Globbing characters: `*?[]{}` for file globbing
|
|
5385
5479
|
- Double quotes: `"`
|
|
5386
5480
|
- Single quotes: `'`
|
|
@@ -5408,7 +5502,7 @@ The following examples give the same result on Windows using `cmd.exe`:
|
|
|
5408
5502
|
```
|
|
5409
5503
|
|
|
5410
5504
|
`cmd.exe` handles some special characters: `^"<>|%&`.
|
|
5411
|
-
It handles I/O redirection (`<>|`), shell variables (`%`), multiple commands (`&`)
|
|
5505
|
+
It handles I/O redirection (`<>|`), shell variables (`%`), and multiple commands (`&`).
|
|
5412
5506
|
Eventually, all those special characters are removed from the command line unless escaped with `^` or `"`.
|
|
5413
5507
|
`"` are kept and given to the program.
|
|
5414
5508
|
|
|
@@ -5417,17 +5511,17 @@ Eventually, all those special characters are removed from the command line unles
|
|
|
5417
5511
|
For PowerShell, the behavior depends on the version (5.1, 7.3+).
|
|
5418
5512
|
|
|
5419
5513
|
A difficulty is that PowerShell parses the command line for its own use and manages special characters, but then it passes the command line to the program (Ruby) as a single string, possibly without the special characters.
|
|
5420
|
-
If not using PowerShell features (for example,
|
|
5514
|
+
If not using PowerShell features (for example, variables), one can use the "stop-parsing" token `--%`.
|
|
5421
5515
|
|
|
5422
5516
|
Details can be found here:
|
|
5423
5517
|
|
|
5424
5518
|
- [Passing arguments with quotes](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_parsing#passing-arguments-that-contain-quote-characters)
|
|
5425
5519
|
|
|
5426
|
-
- [
|
|
5520
|
+
- [Quoting rules](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules)
|
|
5427
5521
|
|
|
5428
5522
|
##### PowerShell 5
|
|
5429
5523
|
|
|
5430
|
-
- Check your
|
|
5524
|
+
- Check your PowerShell version:
|
|
5431
5525
|
|
|
5432
5526
|
```powershell
|
|
5433
5527
|
$psversiontable.psversion.Major
|
|
@@ -5509,19 +5603,19 @@ ascli config echo "@json:$(@{ k = $var; x = $true } | ConvertTo-Json -Compress)"
|
|
|
5509
5603
|
|
|
5510
5604
|
#### Extended Value (JSON, Ruby, ...)
|
|
5511
5605
|
|
|
5512
|
-
Some values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended
|
|
5606
|
+
Some values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), that is, not a simple `String`, but a composite structure (`Hash`, `Array`).
|
|
5513
5607
|
|
|
5514
5608
|
For small structures with a few keys, the [dot-path](#dot-path-notation) `key.subkey=value` notation (using `@:` to collect positional arguments, or `--opt.key=value` directly for options) is often the most convenient: it requires no quoting and reads naturally on the command line.
|
|
5515
5609
|
For larger or more complex structures (deep nesting, arrays of hashes, values copied from API documentation), the `@json:` modifier is typically a better fit.
|
|
5516
|
-
`@json:` expects a [JSON](https://www.json.org/) value; note that `"` is used to enclose a `String` in JSON, which may be difficult to specify in shells
|
|
5610
|
+
`@json:` expects a [JSON](https://www.json.org/) value; note that `"` is used to enclose a `String` in JSON, which may be difficult to specify in shells: enclose the whole argument in single quotes to avoid this.
|
|
5517
5611
|
|
|
5518
5612
|
Any option or **Command Parameter** expecting a `Hash` value accepts the special value `help` to display its schema.
|
|
5519
5613
|
See [Schema Discovery with `help`](#schema-discovery-with-help).
|
|
5520
5614
|
|
|
5521
5615
|
#### Using a shell variable, parsed by shell, in an Extended Value
|
|
5522
5616
|
|
|
5523
|
-
To be evaluated by shell, the shell variable must not be in single quotes.
|
|
5524
|
-
|
|
5617
|
+
To be evaluated by the shell, the shell variable must not be in single quotes.
|
|
5618
|
+
Enclose the variable in double quotes so that it results in a single argument for `ascli` even if its value contains spaces: `bash` splits the result of an unquoted variable expansion into several words (`zsh` does not, by default).
|
|
5525
5619
|
|
|
5526
5620
|
> [!NOTE]
|
|
5527
5621
|
> We use a simple shell variable in this example.
|
|
@@ -5529,8 +5623,8 @@ Even if the variable contains spaces it results only in one argument for `ascli`
|
|
|
5529
5623
|
|
|
5530
5624
|
```shell
|
|
5531
5625
|
MYVAR="Hello World"
|
|
5532
|
-
ascli config echo @json:'{"title":"'$MYVAR'"}' --format=json
|
|
5533
|
-
ascli config echo @json:{\"title\":\"$MYVAR\"} --format=json
|
|
5626
|
+
ascli config echo @json:'{"title":"'"$MYVAR"'"}' --format=json
|
|
5627
|
+
ascli config echo "@json:{\"title\":\"$MYVAR\"}" --format=json
|
|
5534
5628
|
```
|
|
5535
5629
|
|
|
5536
5630
|
```json
|
|
@@ -5612,7 +5706,7 @@ ascli config echo @ruby:"{'title'=>gets.chomp}" --format=json
|
|
|
5612
5706
|
|
|
5613
5707
|
#### Command line arguments from a file
|
|
5614
5708
|
|
|
5615
|
-
If you need to provide a list of command line
|
|
5709
|
+
If you need to provide a list of command line arguments from lines that are in a file, on Linux you can use the `xargs` command:
|
|
5616
5710
|
|
|
5617
5711
|
```shell
|
|
5618
5712
|
xargs -a lines.txt -d \\n ascli config echo
|
|
@@ -5626,7 +5720,7 @@ ascli config echo [line1] [line2] [line3] ...
|
|
|
5626
5720
|
|
|
5627
5721
|
If there are spaces in the lines, those are not taken as separator, as we provide option `-d \\n` to `xargs`.
|
|
5628
5722
|
|
|
5629
|
-
#### Extended value using special characters read from
|
|
5723
|
+
#### Extended value using special characters read from environment variables or files
|
|
5630
5724
|
|
|
5631
5725
|
Using a text editor or shell: create a file `title.txt` (and env var) that contains exactly the text required: `Test " ' & \` :
|
|
5632
5726
|
|
|
@@ -5792,7 +5886,7 @@ The first step is to declare `ascli` in Aspera on Cloud using the admin interfac
|
|
|
5792
5886
|
|
|
5793
5887
|
To register with web-based authentication (auth=web):
|
|
5794
5888
|
|
|
5795
|
-
- Open a web browser, log to your instance: for example, `https://<ORG_NAME>.ibmaspera.com/`
|
|
5889
|
+
- Open a web browser, log in to your instance: for example, `https://<ORG_NAME>.ibmaspera.com/`
|
|
5796
5890
|
(use your actual AoC instance URL)
|
|
5797
5891
|
- Go to (Apps) → Admin → Organization → Integrations
|
|
5798
5892
|
- Click **Create New**
|
|
@@ -5807,7 +5901,7 @@ To register with web-based authentication (auth=web):
|
|
|
5807
5901
|
> For web based authentication, `ascli` listens on a local port (for example, specified by the `redirect_uri` parameter, in this example: `12345`), and the browser will provide the OAuth code there.
|
|
5808
5902
|
> For `ascli`, HTTP is required, and `12345` is the default port.
|
|
5809
5903
|
|
|
5810
|
-
Once the client is registered, a **Client ID** and **Secret** are created
|
|
5904
|
+
Once the client is registered, a **Client ID** and **Secret** are created; these values are used in the next step.
|
|
5811
5905
|
|
|
5812
5906
|
#### Configuration for Aspera on Cloud
|
|
5813
5907
|
|
|
@@ -5828,7 +5922,7 @@ updated: <AOC_ORG>
|
|
|
5828
5922
|
|
|
5829
5923
|
(This can also be done in one line using the command `config preset update <AOC_ORG> --url=...`)
|
|
5830
5924
|
|
|
5831
|
-
Define this [Option Preset](#option-preset) as default configuration for the `
|
|
5925
|
+
Define this [Option Preset](#option-preset) as default configuration for the `aoc` plugin:
|
|
5832
5926
|
|
|
5833
5927
|
```shell
|
|
5834
5928
|
ascli config preset set default aoc <AOC_ORG>
|
|
@@ -5840,7 +5934,7 @@ ascli config preset set default aoc <AOC_ORG>
|
|
|
5840
5934
|
|
|
5841
5935
|
#### Authentication with private key
|
|
5842
5936
|
|
|
5843
|
-
For
|
|
5937
|
+
For browser-less, private key-based authentication, use the following steps.
|
|
5844
5938
|
|
|
5845
5939
|
To use JSON Web Token (JWT) for Aspera on Cloud API client authentication,
|
|
5846
5940
|
a [private/public key pair](#private-key) must be used.
|
|
@@ -5848,11 +5942,11 @@ a [private/public key pair](#private-key) must be used.
|
|
|
5848
5942
|
##### API Client JWT activation
|
|
5849
5943
|
|
|
5850
5944
|
If you are not using the built-in client_id and secret, JWT needs to be authorized in Aspera on Cloud.
|
|
5851
|
-
This can be done in two
|
|
5945
|
+
This can be done in two ways:
|
|
5852
5946
|
|
|
5853
5947
|
- Graphically
|
|
5854
5948
|
|
|
5855
|
-
- Open a web browser, log to your instance: `https://<ORG_NAME>.ibmaspera.com/`
|
|
5949
|
+
- Open a web browser, log in to your instance: `https://<ORG_NAME>.ibmaspera.com/`
|
|
5856
5950
|
(Use your actual AoC instance URL)
|
|
5857
5951
|
- Go to Apps → Admin → Organization → Integrations
|
|
5858
5952
|
- Click the previously created application
|
|
@@ -5889,13 +5983,13 @@ modified
|
|
|
5889
5983
|
#### User key registration
|
|
5890
5984
|
|
|
5891
5985
|
The public key must be assigned to your user.
|
|
5892
|
-
This can be done in two
|
|
5986
|
+
This can be done in two ways as follows.
|
|
5893
5987
|
|
|
5894
5988
|
##### Graphically
|
|
5895
5989
|
|
|
5896
5990
|
Open the previously generated public key located here: `$HOME/.aspera/ascli/<PKEY_NAME>.pub`
|
|
5897
5991
|
|
|
5898
|
-
- Open a web browser, log to your instance: `https://<ORG_NAME>.ibmaspera.com/`
|
|
5992
|
+
- Open a web browser, log in to your instance: `https://<ORG_NAME>.ibmaspera.com/`
|
|
5899
5993
|
(Use your actual AoC instance URL)
|
|
5900
5994
|
- Click the user icon (top right)
|
|
5901
5995
|
- Select **Account Settings**
|
|
@@ -5927,7 +6021,7 @@ modified
|
|
|
5927
6021
|
```
|
|
5928
6022
|
|
|
5929
6023
|
> [!TIP]
|
|
5930
|
-
> The `
|
|
6024
|
+
> The `ascli aoc user profile show` command can be used to verify modifications.
|
|
5931
6025
|
|
|
5932
6026
|
#### [Option Preset](#option-preset) modification for JWT
|
|
5933
6027
|
|
|
@@ -5968,7 +6062,7 @@ Alternatively:
|
|
|
5968
6062
|
For a simpler use, configure a preset with the `url` option, and optionally `username`.
|
|
5969
6063
|
(If the username is not provided, then the subject from the token is used, else both must match.)
|
|
5970
6064
|
|
|
5971
|
-
Use the cookie string
|
|
6065
|
+
Use the cookie string as the value of option `password`; an environment variable is convenient, as the value is temporary anyway:
|
|
5972
6066
|
|
|
5973
6067
|
```shell
|
|
5974
6068
|
export ASCLI_PASSWORD="...; aoc.token=...; aoc.refresh=...; ..."
|
|
@@ -5980,7 +6074,7 @@ ascli aoc user profile show --auth=boot
|
|
|
5980
6074
|
> The cookie string contains `aoc.token` (bearer JWT, mandatory) and `aoc.refresh` (refresh token, optional).
|
|
5981
6075
|
> Only those two are used.
|
|
5982
6076
|
> On first use, the tokens are cached locally.
|
|
5983
|
-
> Subsequent calls reuse the cache and refresh automatically:
|
|
6077
|
+
> Subsequent calls reuse the cache and refresh automatically: the `password` option is used only to get the `username` unless the option `username` is already provided.
|
|
5984
6078
|
|
|
5985
6079
|
#### Public and private links
|
|
5986
6080
|
|
|
@@ -5993,14 +6087,14 @@ Private links require the user to authenticate.
|
|
|
5993
6087
|
So, provide the same options as for regular authentication, and provide the private link using option `url`.
|
|
5994
6088
|
|
|
5995
6089
|
A user may not be part of any workspace, but still have access to shared folders (using private links).
|
|
5996
|
-
In that case, it is possible to list those shared
|
|
6090
|
+
In that case, it is possible to list those shared folders by using a value for option `workspace` equal to `@none:` or `@json:null` or `@ruby:nil`.
|
|
5997
6091
|
|
|
5998
6092
|
#### AoC: First Use
|
|
5999
6093
|
|
|
6000
6094
|
Once client has been registered and [Option Preset](#option-preset) created: `ascli` can be used:
|
|
6001
6095
|
|
|
6002
6096
|
```shell
|
|
6003
|
-
ascli aoc files
|
|
6097
|
+
ascli aoc files browse /
|
|
6004
6098
|
```
|
|
6005
6099
|
|
|
6006
6100
|
```text
|
|
@@ -6010,7 +6104,7 @@ empty
|
|
|
6010
6104
|
|
|
6011
6105
|
### Calling AoC APIs from command line
|
|
6012
6106
|
|
|
6013
|
-
The command `ascli aoc
|
|
6107
|
+
The command `ascli aoc bearer_token` can be used to generate an OAuth token suitable to call any AoC API.
|
|
6014
6108
|
This can be useful when a command is not yet available.
|
|
6015
6109
|
|
|
6016
6110
|
Example:
|
|
@@ -6026,14 +6120,14 @@ ascli aoc files bearer_token_node /
|
|
|
6026
6120
|
```
|
|
6027
6121
|
|
|
6028
6122
|
```shell
|
|
6029
|
-
ascli aoc admin node bearer_token <NODE_ID>
|
|
6123
|
+
ascli aoc admin node bearer_token <NODE_ID>
|
|
6030
6124
|
```
|
|
6031
6125
|
|
|
6032
6126
|
### Administration
|
|
6033
6127
|
|
|
6034
|
-
The `admin` command allows several administrative tasks (and
|
|
6128
|
+
The `admin` command allows several administrative tasks (and requires admin privilege).
|
|
6035
6129
|
|
|
6036
|
-
It allows actions (create, update, delete) on **resources**: users, groups, nodes,
|
|
6130
|
+
It allows actions (create, update, delete) on **resources**: users, groups, nodes, workspaces, and so on, with the `admin <RESOURCE_TYPE>` commands.
|
|
6037
6131
|
|
|
6038
6132
|
#### Listing resources
|
|
6039
6133
|
|
|
@@ -6061,8 +6155,8 @@ The following parameters are supported:
|
|
|
6061
6155
|
> [!NOTE]
|
|
6062
6156
|
> Both `max` and `pmax` are processed internally in `ascli`, not included in actual API call and limit the number of successive pages requested to API.
|
|
6063
6157
|
> `ascli` will return all values using paging if not provided.
|
|
6064
|
-
> `page` and `per_page` are normally added by `ascli` to build successive API calls to get all values if there are more than 1000
|
|
6065
|
-
(AoC allows a maximum page size of 1000).
|
|
6158
|
+
> `page` and `per_page` are normally added by `ascli` to build successive API calls to get all values if there are more than 1000
|
|
6159
|
+
> (AoC allows a maximum page size of 1000).
|
|
6066
6160
|
> Other parameters depend on the type of resource (refer to AoC API) and are directly sent as parameters to the `GET` request on API.
|
|
6067
6161
|
|
|
6068
6162
|
> [!TIP]
|
|
@@ -6080,7 +6174,7 @@ Examples:
|
|
|
6080
6174
|
ascli aoc admin user list --query.q=laurent
|
|
6081
6175
|
```
|
|
6082
6176
|
|
|
6083
|
-
- List users who logged
|
|
6177
|
+
- List users who logged in before a date:
|
|
6084
6178
|
|
|
6085
6179
|
```shell
|
|
6086
6180
|
ascli aoc admin user list --query.q='last_login_at:<2018-05-28'
|
|
@@ -6098,17 +6192,15 @@ Resources are identified by a unique `id` and a unique `name` (case-insensitive)
|
|
|
6098
6192
|
|
|
6099
6193
|
To execute an action on a specific resource, select it using one of those methods:
|
|
6100
6194
|
|
|
6101
|
-
- **recommended**: give ID directly on command line **after the action**: `aoc admin node show 123`
|
|
6102
|
-
- Give name
|
|
6103
|
-
- Provide option `id` : `aoc admin node show 123`
|
|
6104
|
-
- Provide option `name` : `aoc admin node show %name:abc`
|
|
6195
|
+
- **recommended**: give the ID directly on the command line **after the action**: `aoc admin node show 123`
|
|
6196
|
+
- Give another unique field, such as the name, using the [percent selector](#percent-selector) **after the action**: `aoc admin node show %name:abc`
|
|
6105
6197
|
|
|
6106
6198
|
#### Creating a resource
|
|
6107
6199
|
|
|
6108
6200
|
New resources (users, groups, workspaces, and so on) can be created using a command like:
|
|
6109
6201
|
|
|
6110
6202
|
```shell
|
|
6111
|
-
ascli aoc admin
|
|
6203
|
+
ascli aoc admin <RESOURCE_TYPE> create @json:'{<...parameters...>}'
|
|
6112
6204
|
```
|
|
6113
6205
|
|
|
6114
6206
|
Some API endpoints are described in [IBM API Hub](https://developer.ibm.com/apis/catalog?search=%22aspera%20on%20cloud%20api%22).
|
|
@@ -6124,7 +6216,7 @@ ascli aoc admin group show 12345 --format=json
|
|
|
6124
6216
|
{"created_at":"2018-07-24T21:46:39.000Z","description":null,"id":"12345","manager":false,"name":"A8Demo WS1","owner":false,"queued_operation_count":0,"running_operation_count":0,"stopped_operation_count":0,"updated_at":"2018-07-24T21:46:39.000Z","saml_group":false,"saml_group_dn":null,"system_group":true,"system_group_type":"workspace_members"}
|
|
6125
6217
|
```
|
|
6126
6218
|
|
|
6127
|
-
Remove the parameters that are
|
|
6219
|
+
Remove the parameters that are set by the system (`id`, `created_at`, `updated_at`), and optional ones.
|
|
6128
6220
|
|
|
6129
6221
|
And then craft your command:
|
|
6130
6222
|
|
|
@@ -6132,16 +6224,16 @@ And then craft your command:
|
|
|
6132
6224
|
ascli aoc admin group create @json:'{"wrong":"param"}'
|
|
6133
6225
|
```
|
|
6134
6226
|
|
|
6135
|
-
If the command returns an error, example:
|
|
6227
|
+
If the command returns an error, for example:
|
|
6136
6228
|
|
|
6137
6229
|
```text
|
|
6138
|
-
|
|
6230
|
+
ERRR Rest: found unpermitted parameter: :wrong
|
|
6139
6231
|
code: unpermitted_parameters
|
|
6140
6232
|
request_id: 2a487dbc-bc5c-41ab-86c8-3b9972dfd4c4
|
|
6141
6233
|
api.ibmaspera.com 422 Unprocessable Entity
|
|
6142
6234
|
```
|
|
6143
6235
|
|
|
6144
|
-
|
|
6236
|
+
Remove the offending parameters and try again.
|
|
6145
6237
|
|
|
6146
6238
|
> [!NOTE]
|
|
6147
6239
|
> Some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
|
|
@@ -6150,10 +6242,10 @@ Well, remove the offending parameters and try again.
|
|
|
6150
6242
|
|
|
6151
6243
|
To access some administrative actions on **nodes** (in fact, access keys), the associated secret may be required.
|
|
6152
6244
|
The secret is provided using the `secret` option.
|
|
6153
|
-
For example in a command like:
|
|
6245
|
+
For example, in a command like:
|
|
6154
6246
|
|
|
6155
6247
|
```shell
|
|
6156
|
-
ascli aoc admin node <NODE_ID> v3 info
|
|
6248
|
+
ascli aoc admin node do <NODE_ID> v3 info
|
|
6157
6249
|
```
|
|
6158
6250
|
|
|
6159
6251
|
It is also possible to store secrets in the [secret vault](#secret-vault) and then automatically find the related secret using the [config finder](#configuration-finder).
|
|
@@ -6166,8 +6258,8 @@ The activity app can be queried with:
|
|
|
6166
6258
|
ascli aoc admin analytics transfers
|
|
6167
6259
|
```
|
|
6168
6260
|
|
|
6169
|
-
It
|
|
6170
|
-
A template is defined using option `notify_template
|
|
6261
|
+
It supports filters and can send notifications using option `notify_to`.
|
|
6262
|
+
A template is defined using option `notify_template`:
|
|
6171
6263
|
|
|
6172
6264
|
`mytemplate.erb`:
|
|
6173
6265
|
|
|
@@ -6195,13 +6287,13 @@ ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 --query=@j
|
|
|
6195
6287
|
|
|
6196
6288
|
Options:
|
|
6197
6289
|
|
|
6198
|
-
- `once_only
|
|
6199
|
-
- `query
|
|
6200
|
-
- `
|
|
6290
|
+
- `once_only`: keep track of the last date it was called, so that the next call gets only new events
|
|
6291
|
+
- `query`: filter (on API call)
|
|
6292
|
+
- `notify_to`, `notify_template`: send an email as specified by the template; the template can be placed in a file with the `@file:` modifier.
|
|
6201
6293
|
|
|
6202
6294
|
> [!NOTE]
|
|
6203
6295
|
> This must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months.
|
|
6204
|
-
The period is `[date of previous execution]..[now]`.
|
|
6296
|
+
> The period is `[date of previous execution]..[now]`.
|
|
6205
6297
|
|
|
6206
6298
|
#### Using ATS
|
|
6207
6299
|
|
|
@@ -6212,35 +6304,35 @@ See the section **Examples** of [ATS](#plugin-ats-ibm-aspera-transfer-service) a
|
|
|
6212
6304
|
Aspera on Cloud Shared folders are implemented through a special type of file: `link`.
|
|
6213
6305
|
A `link` is the equivalent of a symbolic link on a file system: it points to another folder (not file).
|
|
6214
6306
|
|
|
6215
|
-
Listing a link (
|
|
6307
|
+
Listing a link (as the last element of the path) shows information on the link itself, not the content of the folder it points to.
|
|
6216
6308
|
To list the target folder content, add a `/` at the end of the path.
|
|
6217
6309
|
|
|
6218
6310
|
Example:
|
|
6219
6311
|
|
|
6220
6312
|
```shell
|
|
6221
|
-
ascli aoc files
|
|
6313
|
+
ascli aoc files browse the_link
|
|
6222
6314
|
```
|
|
6223
6315
|
|
|
6224
6316
|
```text
|
|
6225
6317
|
Current Workspace: Default (default)
|
|
6226
|
-
|
|
6227
|
-
|
|
6228
|
-
|
|
6229
|
-
|
|
6230
|
-
|
|
6318
|
+
╭──────────┬──────┬────────────────┬──────┬──────────────────────┬──────────────╮
|
|
6319
|
+
│ name │ type │ recursive_size │ size │ modified_time │ access_level │
|
|
6320
|
+
╞══════════╪══════╪════════════════╪══════╪══════════════════════╪══════════════╡
|
|
6321
|
+
│ the_link │ link │ │ │ 2021-04-28T09:17:14Z │ edit │
|
|
6322
|
+
╰──────────┴──────┴────────────────┴──────┴──────────────────────┴──────────────╯
|
|
6231
6323
|
```
|
|
6232
6324
|
|
|
6233
6325
|
```shell
|
|
6234
|
-
ascli aoc files
|
|
6326
|
+
ascli aoc files browse the_link/
|
|
6235
6327
|
```
|
|
6236
6328
|
|
|
6237
6329
|
```text
|
|
6238
6330
|
Current Workspace: Default (default)
|
|
6239
|
-
|
|
6240
|
-
|
|
6241
|
-
|
|
6242
|
-
|
|
6243
|
-
|
|
6331
|
+
╭─────────────┬──────┬────────────────┬──────┬──────────────────────┬──────────────╮
|
|
6332
|
+
│ name │ type │ recursive_size │ size │ modified_time │ access_level │
|
|
6333
|
+
╞═════════════╪══════╪════════════════╪══════╪══════════════════════╪══════════════╡
|
|
6334
|
+
│ file_inside │ file │ │ │ 2021-04-26T09:00:00Z │ edit │
|
|
6335
|
+
╰─────────────┴──────┴────────────────┴──────┴──────────────────────┴──────────────╯
|
|
6244
6336
|
```
|
|
6245
6337
|
|
|
6246
6338
|
#### Example: Bulk creation of users
|
|
@@ -6250,12 +6342,12 @@ ascli aoc admin user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"
|
|
|
6250
6342
|
```
|
|
6251
6343
|
|
|
6252
6344
|
```text
|
|
6253
|
-
|
|
6254
|
-
|
|
6255
|
-
|
|
6256
|
-
|
|
6257
|
-
|
|
6258
|
-
|
|
6345
|
+
╭───────┬─────────╮
|
|
6346
|
+
│ id │ status │
|
|
6347
|
+
╞═══════╪═════════╡
|
|
6348
|
+
│ 98398 │ created │
|
|
6349
|
+
│ 98399 │ created │
|
|
6350
|
+
╰───────┴─────────╯
|
|
6259
6351
|
```
|
|
6260
6352
|
|
|
6261
6353
|
#### Example: Find with filter and delete
|
|
@@ -6265,12 +6357,12 @@ ascli aoc admin user list --query.q=dummyuser --fields=id,email
|
|
|
6265
6357
|
```
|
|
6266
6358
|
|
|
6267
6359
|
```text
|
|
6268
|
-
|
|
6269
|
-
|
|
6270
|
-
|
|
6271
|
-
|
|
6272
|
-
|
|
6273
|
-
|
|
6360
|
+
╭───────┬────────────────────────╮
|
|
6361
|
+
│ id │ email │
|
|
6362
|
+
╞═══════╪════════════════════════╡
|
|
6363
|
+
│ 98398 │ dummyuser1@example.com │
|
|
6364
|
+
│ 98399 │ dummyuser2@example.com │
|
|
6365
|
+
╰───────┴────────────────────────╯
|
|
6274
6366
|
```
|
|
6275
6367
|
|
|
6276
6368
|
```shell
|
|
@@ -6278,12 +6370,12 @@ ascli aoc admin user list --query.q=dummyuser --fields=id --out.level=data --for
|
|
|
6278
6370
|
```
|
|
6279
6371
|
|
|
6280
6372
|
```text
|
|
6281
|
-
|
|
6282
|
-
|
|
6283
|
-
|
|
6284
|
-
|
|
6285
|
-
|
|
6286
|
-
|
|
6373
|
+
╭───────┬─────────╮
|
|
6374
|
+
│ id │ status │
|
|
6375
|
+
╞═══════╪═════════╡
|
|
6376
|
+
│ 98398 │ deleted │
|
|
6377
|
+
│ 98399 │ deleted │
|
|
6378
|
+
╰───────┴─────────╯
|
|
6287
6379
|
```
|
|
6288
6380
|
|
|
6289
6381
|
#### Example: Find deactivated users for more than 2 years
|
|
@@ -6317,8 +6409,8 @@ The `aoc user settings` sub-command manages persistent client-side settings stor
|
|
|
6317
6409
|
|
|
6318
6410
|
```shell
|
|
6319
6411
|
ascli aoc user settings list
|
|
6320
|
-
ascli aoc user settings show <
|
|
6321
|
-
ascli aoc user settings modify <
|
|
6412
|
+
ascli aoc user settings show <ID>
|
|
6413
|
+
ascli aoc user settings modify <ID> @json:'{"value":"..."}'
|
|
6322
6414
|
```
|
|
6323
6415
|
|
|
6324
6416
|
> [!NOTE]
|
|
@@ -6332,10 +6424,10 @@ ascli aoc user settings modify <id> @json:'{"value":"..."}'
|
|
|
6332
6424
|
|
|
6333
6425
|
#### Example: Create a sub access key in a `node`
|
|
6334
6426
|
|
|
6335
|
-
Creation of a sub-access key is like creation of access key with the following
|
|
6427
|
+
Creation of a sub-access key is like creation of an access key, with the following differences: authentication to the Node API is made with an access key (the master access key), and only the path parameter is provided, relative to the storage root of the master key (`id` and `secret` are optional).
|
|
6336
6428
|
|
|
6337
6429
|
```shell
|
|
6338
|
-
ascli aoc admin
|
|
6430
|
+
ascli aoc admin node do %name:'<NODE_NAME>' v3 access_keys create @: storage.path=/folder1
|
|
6339
6431
|
```
|
|
6340
6432
|
|
|
6341
6433
|
#### Example: Display transfer events (ops/transfer)
|
|
@@ -6357,7 +6449,7 @@ Examples of query:
|
|
|
6357
6449
|
#### Example: Display node events (events)
|
|
6358
6450
|
|
|
6359
6451
|
```shell
|
|
6360
|
-
ascli aoc admin node v3 events
|
|
6452
|
+
ascli aoc admin node do <NODE_ID> v3 events
|
|
6361
6453
|
```
|
|
6362
6454
|
|
|
6363
6455
|
#### Example: Display members of a workspace
|
|
@@ -6367,16 +6459,16 @@ ascli aoc admin workspace_membership list --fields=member_type,manager,member.em
|
|
|
6367
6459
|
```
|
|
6368
6460
|
|
|
6369
6461
|
```text
|
|
6370
|
-
|
|
6371
|
-
|
|
6372
|
-
|
|
6373
|
-
|
|
6374
|
-
|
|
6375
|
-
|
|
6376
|
-
|
|
6377
|
-
|
|
6378
|
-
|
|
6379
|
-
|
|
6462
|
+
╭─────────────┬─────────┬──────────────────────────╮
|
|
6463
|
+
│ member_type │ manager │ member.email │
|
|
6464
|
+
╞═════════════╪═════════╪══════════════════════════╡
|
|
6465
|
+
│ user │ true │ john.curtis@email.com │
|
|
6466
|
+
│ user │ false │ someuser@example.com │
|
|
6467
|
+
│ user │ false │ jean.dupont@me.com │
|
|
6468
|
+
│ user │ false │ another.user@example.com │
|
|
6469
|
+
│ group │ false │ │
|
|
6470
|
+
│ user │ false │ aspera.user@gmail.com │
|
|
6471
|
+
╰─────────────┴─────────┴──────────────────────────╯
|
|
6380
6472
|
```
|
|
6381
6473
|
|
|
6382
6474
|
Other query parameters:
|
|
@@ -6425,19 +6517,19 @@ e- Add members to second workspace
|
|
|
6425
6517
|
ascli aoc admin workspace_membership create --bulk=yes @json:@file:ws2_members.json
|
|
6426
6518
|
```
|
|
6427
6519
|
|
|
6428
|
-
#### Example: Get users who did not log since a date
|
|
6520
|
+
#### Example: Get users who did not log in since a date
|
|
6429
6521
|
|
|
6430
6522
|
```shell
|
|
6431
6523
|
ascli aoc admin user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
|
6432
6524
|
```
|
|
6433
6525
|
|
|
6434
6526
|
```text
|
|
6435
|
-
|
|
6436
|
-
|
|
6437
|
-
|
|
6438
|
-
|
|
6439
|
-
|
|
6440
|
-
|
|
6527
|
+
╭──────────────────────────╮
|
|
6528
|
+
│ email │
|
|
6529
|
+
╞══════════════════════════╡
|
|
6530
|
+
│ John.curtis@acme.com │
|
|
6531
|
+
│ Jean.Dupont@tropfort.com │
|
|
6532
|
+
╰──────────────────────────╯
|
|
6441
6533
|
```
|
|
6442
6534
|
|
|
6443
6535
|
#### Example: List **Limited** users
|
|
@@ -6473,7 +6565,7 @@ Workspace: <WORKSPACE_ID>
|
|
|
6473
6565
|
- Add group to workspace
|
|
6474
6566
|
|
|
6475
6567
|
```shell
|
|
6476
|
-
ascli aoc admin workspace_membership create @json:'{"workspace_id":<WORKSPACE_ID>,"member_type":"
|
|
6568
|
+
ascli aoc admin workspace_membership create @json:'{"workspace_id":<WORKSPACE_ID>,"member_type":"group","member_id":<GROUP_ID>}'
|
|
6477
6569
|
```
|
|
6478
6570
|
|
|
6479
6571
|
- Get a user's ID
|
|
@@ -6494,42 +6586,20 @@ ascli aoc admin group_membership create @json:'{"group_id":<GROUP_ID>,"member_ty
|
|
|
6494
6586
|
|
|
6495
6587
|
In this example, a user has access to a workspace where two shared folders are located on different sites, for example, different cloud regions.
|
|
6496
6588
|
|
|
6497
|
-
First, set up the environment (skip if already done)
|
|
6589
|
+
First, set up the environment (skip if already done), see [AoC configuration: Using Wizard](#aoc-configuration-using-wizard):
|
|
6498
6590
|
|
|
6499
6591
|
```shell
|
|
6500
|
-
ascli config wizard
|
|
6592
|
+
ascli config wizard https://sedemo.ibmaspera.com aoc aoc_sedemo --username=someuser@example.com
|
|
6501
6593
|
```
|
|
6502
6594
|
|
|
6503
|
-
|
|
6504
|
-
Detected: Aspera on Cloud
|
|
6505
|
-
Preparing preset: aoc_sedemo
|
|
6506
|
-
Using existing key:
|
|
6507
|
-
/Users/laurent/.aspera/ascli/aspera_aoc_key
|
|
6508
|
-
Using global client_id.
|
|
6509
|
-
Please Login to your Aspera on Cloud instance.
|
|
6510
|
-
Navigate to your "Account Settings"
|
|
6511
|
-
Check or update the value of "Public Key" to be:
|
|
6512
|
-
-----BEGIN PUBLIC KEY-----
|
|
6513
|
-
SOME PUBLIC KEY PEM DATA HERE
|
|
6514
|
-
-----END PUBLIC KEY-----
|
|
6515
|
-
Once updated or validated, press enter.
|
|
6516
|
-
|
|
6517
|
-
creating new config preset: aoc_sedemo
|
|
6518
|
-
Setting config preset as default for aspera
|
|
6519
|
-
saving configuration file
|
|
6520
|
-
Done.
|
|
6521
|
-
You can test with:
|
|
6522
|
-
ascli aoc user profile show
|
|
6523
|
-
```
|
|
6524
|
-
|
|
6525
|
-
This creates the option preset `aoc_[org name]` to allow seamless command line access and sets it as default for Aspera on Cloud.
|
|
6595
|
+
This creates the option preset `aoc_sedemo` to allow seamless command line access and sets it as default for Aspera on Cloud.
|
|
6526
6596
|
|
|
6527
6597
|
Then, create two shared folders located in two regions, in your files home, in a workspace.
|
|
6528
6598
|
|
|
6529
6599
|
Then, transfer between those:
|
|
6530
6600
|
|
|
6531
6601
|
```shell
|
|
6532
|
-
ascli -
|
|
6602
|
+
ascli -Paoc_sedemo aoc files transfer push 'IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":1000000,"multi_session":10,"multi_session_threshold":1}'
|
|
6533
6603
|
```
|
|
6534
6604
|
|
|
6535
6605
|
#### Example: Delete all registration keys
|
|
@@ -6539,14 +6609,14 @@ ascli aoc admin client_registration_token list --fields=id --format=csv|ascli ao
|
|
|
6539
6609
|
```
|
|
6540
6610
|
|
|
6541
6611
|
```text
|
|
6542
|
-
|
|
6543
|
-
|
|
6544
|
-
|
|
6545
|
-
|
|
6546
|
-
|
|
6547
|
-
|
|
6548
|
-
|
|
6549
|
-
|
|
6612
|
+
╭─────┬─────────╮
|
|
6613
|
+
│ id │ status │
|
|
6614
|
+
╞═════╪═════════╡
|
|
6615
|
+
│ 99 │ deleted │
|
|
6616
|
+
│ 100 │ deleted │
|
|
6617
|
+
│ 101 │ deleted │
|
|
6618
|
+
│ 102 │ deleted │
|
|
6619
|
+
╰─────┴─────────╯
|
|
6550
6620
|
```
|
|
6551
6621
|
|
|
6552
6622
|
#### Example: Create a tethered Node
|
|
@@ -6555,7 +6625,7 @@ Follow these steps to configure a new HSTS and link it to your existing Aspera o
|
|
|
6555
6625
|
|
|
6556
6626
|
- Retrieve the Organization Public Key
|
|
6557
6627
|
|
|
6558
|
-
First, obtain the public key
|
|
6628
|
+
First, obtain the organization's public key.
|
|
6559
6629
|
This key is used to verify bearer tokens generated by your organization.
|
|
6560
6630
|
This key remains constant for the lifetime of your Organization.
|
|
6561
6631
|
|
|
@@ -6580,7 +6650,7 @@ ascli aoc admin node do %name:'<NODE_NAME>' v3 access_keys show self --fields=to
|
|
|
6580
6650
|
> Record the generated secret immediately; it cannot be retrieved later, only reset.
|
|
6581
6651
|
|
|
6582
6652
|
```shell
|
|
6583
|
-
ascli node
|
|
6653
|
+
ascli node access_keys create @: id=<ACCESS_KEY_ID> secret=<SECRET> storage.type=local storage.path=/data/aoc token_verification_key=@file:mypubkey.pem
|
|
6584
6654
|
```
|
|
6585
6655
|
|
|
6586
6656
|
- Register the Node in AoC
|
|
@@ -6601,14 +6671,14 @@ ascli aoc admin node create @: url=https://aspera.example.com access_key=<ACCESS
|
|
|
6601
6671
|
> If the node is configured for admin user, then add options: `--username=<ACCESS_KEY_ID> --password=<SECRET>`.
|
|
6602
6672
|
|
|
6603
6673
|
```shell
|
|
6604
|
-
ascli node
|
|
6674
|
+
ascli node access_keys do self permission / create @: access_type=user access_id='F4 System'
|
|
6605
6675
|
```
|
|
6606
6676
|
|
|
6607
6677
|
```shell
|
|
6608
|
-
ascli node
|
|
6678
|
+
ascli node access_keys do self permission / create @: access_type=user access_id=NODE_OWNER
|
|
6609
6679
|
```
|
|
6610
6680
|
|
|
6611
|
-
- Optional next
|
|
6681
|
+
- Optional next steps
|
|
6612
6682
|
|
|
6613
6683
|
To register an Aspera Event Journal (AEJ) as described in the HSTS manual, refer to:
|
|
6614
6684
|
|
|
@@ -6644,13 +6714,13 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
|
6644
6714
|
The creation options are the ones of ATS API, refer to the [section on ATS](#ats-access-key-creation-parameters) for more details and examples.
|
|
6645
6715
|
|
|
6646
6716
|
```shell
|
|
6647
|
-
ascli aoc admin ats access_key create
|
|
6717
|
+
ascli aoc admin ats access_key create @json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}' --cloud=softlayer --region=eu-de
|
|
6648
6718
|
```
|
|
6649
6719
|
|
|
6650
|
-
Once executed, the access key `id` and `secret`, randomly generated by the Node API,
|
|
6720
|
+
Once executed, the access key `id` and `secret`, randomly generated by the Node API, are displayed.
|
|
6651
6721
|
|
|
6652
6722
|
> [!NOTE]
|
|
6653
|
-
> Once returned by the API, the secret will not be available anymore, so store
|
|
6723
|
+
> Once returned by the API, the secret will not be available anymore, so store it securely.
|
|
6654
6724
|
> ATS secrets can only be reset by asking IBM support.
|
|
6655
6725
|
|
|
6656
6726
|
- Create the AoC node resource
|
|
@@ -6667,11 +6737,11 @@ Then use the returned address for the `url` key to create the AoC Node resource:
|
|
|
6667
6737
|
ascli aoc admin node create @json:'{"name":"myname","access_key":"myaccesskeyid","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
|
|
6668
6738
|
```
|
|
6669
6739
|
|
|
6670
|
-
Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node
|
|
6740
|
+
Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_keys create` on the private node itself.
|
|
6671
6741
|
|
|
6672
6742
|
#### Example: Deactivate an application in a workspace
|
|
6673
6743
|
|
|
6674
|
-
This is a two-
|
|
6744
|
+
This is a two-step procedure:
|
|
6675
6745
|
|
|
6676
6746
|
1. Find the application ID in the workspace:
|
|
6677
6747
|
|
|
@@ -6690,13 +6760,13 @@ This is a two-steps procedure:
|
|
|
6690
6760
|
2. Deactivate the application:
|
|
6691
6761
|
|
|
6692
6762
|
```shell
|
|
6693
|
-
ascli aoc admin application instance modify
|
|
6763
|
+
ascli aoc admin application instance packages modify <APP_ID> @: enabled=false inherit_organization_app_settings=false
|
|
6694
6764
|
```
|
|
6695
6765
|
|
|
6696
6766
|
### List of files to transfer
|
|
6697
6767
|
|
|
6698
6768
|
Source files are provided as a list with the `sources` option.
|
|
6699
|
-
By default, the list of files on the command line.
|
|
6769
|
+
By default, the list of files is provided on the command line.
|
|
6700
6770
|
See [File list](#list-of-files-for-transfers).
|
|
6701
6771
|
|
|
6702
6772
|
### Packages app
|
|
@@ -6766,20 +6836,20 @@ ascli aoc files browse /src_folder
|
|
|
6766
6836
|
```
|
|
6767
6837
|
|
|
6768
6838
|
```text
|
|
6769
|
-
|
|
6770
|
-
|
|
6771
|
-
|
|
6772
|
-
|
|
6773
|
-
|
|
6774
|
-
|
|
6775
|
-
|
|
6776
|
-
|
|
6839
|
+
╭──────────────┬──────┬────────────────┬──────────────┬──────────────────────┬──────────────╮
|
|
6840
|
+
│ name │ type │ recursive_size │ size │ modified_time │ access_level │
|
|
6841
|
+
╞══════════════╪══════╪════════════════╪══════════════╪══════════════════════╪══════════════╡
|
|
6842
|
+
│ sample_video │ link │ │ │ 2020-11-29T22:49:09Z │ edit │
|
|
6843
|
+
│ 100G │ file │ │ 107374182400 │ 2021-04-21T18:19:25Z │ edit │
|
|
6844
|
+
│ 10M.dat │ file │ │ 10485760 │ 2021-05-18T08:22:39Z │ edit │
|
|
6845
|
+
│ Test.pdf │ file │ │ 1265103 │ 2022-06-16T12:49:55Z │ edit │
|
|
6846
|
+
╰──────────────┴──────┴────────────────┴──────────────┴──────────────────────┴──────────────╯
|
|
6777
6847
|
```
|
|
6778
6848
|
|
|
6779
6849
|
To send a package with the file `10M.dat` from subfolder /src_folder:
|
|
6780
6850
|
|
|
6781
6851
|
```shell
|
|
6782
|
-
ascli aoc files node_info /src_folder --format=json --out.level=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=@json:@stdin:
|
|
6852
|
+
ascli aoc files node_info /src_folder --format=json --out.level=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=@json:@stdin: --transfer.agent=node
|
|
6783
6853
|
```
|
|
6784
6854
|
|
|
6785
6855
|
#### Receive packages
|
|
@@ -6837,7 +6907,7 @@ The `package_folder` option (`Hash`) controls how downloaded packages are organi
|
|
|
6837
6907
|
ascli aoc packages recv ALL --workspace=_workspace_ --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"_shared_inbox_name_","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
|
6838
6908
|
```
|
|
6839
6909
|
|
|
6840
|
-
To list packages that would be downloaded, without downloading them, replace `recv ALL` with `list` (keep options `once_only` and `query`)
|
|
6910
|
+
To list packages that would be downloaded, without downloading them, replace `recv ALL` with `list` (keep options `once_only` and `query`).
|
|
6841
6911
|
|
|
6842
6912
|
##### Receive new packages only (Cargo)
|
|
6843
6913
|
|
|
@@ -6861,7 +6931,7 @@ To list the content of a package, use command `packages browse <PACKAGE_ID> <FOL
|
|
|
6861
6931
|
Example:
|
|
6862
6932
|
|
|
6863
6933
|
```shell
|
|
6864
|
-
ascli aoc
|
|
6934
|
+
ascli aoc packages browse xx5CnbeWng /
|
|
6865
6935
|
```
|
|
6866
6936
|
|
|
6867
6937
|
Use command `find` to list recursively.
|
|
@@ -6869,7 +6939,7 @@ Use command `find` to list recursively.
|
|
|
6869
6939
|
For advanced users, it is also possible to pipe node information for the package and use node operations:
|
|
6870
6940
|
|
|
6871
6941
|
```shell
|
|
6872
|
-
ascli aoc
|
|
6942
|
+
ascli aoc packages node_info <PACKAGE_ID> / --format=json --out.secrets=yes --out.level=data | ascli node -N --preset=@json:@stdin: access_keys do self browse /
|
|
6873
6943
|
```
|
|
6874
6944
|
|
|
6875
6945
|
#### List packages
|
|
@@ -6914,7 +6984,7 @@ ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archiv
|
|
|
6914
6984
|
Using shared inbox identifier: first retrieve the ID of the shared inbox, and then list packages with the appropriate filter.
|
|
6915
6985
|
|
|
6916
6986
|
```shell
|
|
6917
|
-
shared_box_id=$(ascli aoc packages shared_inboxes show
|
|
6987
|
+
shared_box_id=$(ascli aoc packages shared_inboxes show %name:'My Shared Inbox' --format=csv --out.level=data --fields=id)
|
|
6918
6988
|
```
|
|
6919
6989
|
|
|
6920
6990
|
```shell
|
|
@@ -6978,7 +7048,7 @@ When creating a Shared Folder, `ascli` expects a `Hash` payload (typically passe
|
|
|
6978
7048
|
"access_levels": ["list","read","write","delete","mkdir","rename","preview"],
|
|
6979
7049
|
"access_type": "user",
|
|
6980
7050
|
"access_id": "john@example.com",
|
|
6981
|
-
"tags": {...}
|
|
7051
|
+
"tags": {...}
|
|
6982
7052
|
}
|
|
6983
7053
|
```
|
|
6984
7054
|
|
|
@@ -6997,13 +7067,13 @@ When creating a Shared Folder, `ascli` expects a `Hash` payload (typically passe
|
|
|
6997
7067
|
| `link_name` | `ascli` | Name of the link file created in the user's home folder for private links. |
|
|
6998
7068
|
| `as` | `ascli` | Name of the link file created in the user's home folder for admin shared folders. |
|
|
6999
7069
|
|
|
7000
|
-
To declare or create the shared folder in the workspace, a special value for `access_id` is used: `ASPERA_ACCESS_KEY_ADMIN_WS_[workspace ID]`, with
|
|
7070
|
+
To declare or create the shared folder in the workspace, a special value for `access_id` is used: `ASPERA_ACCESS_KEY_ADMIN_WS_[workspace ID]`, with an `access_type` of `user`.
|
|
7001
7071
|
This is conveniently set by `ascli` using an **empty string** for field `with`.
|
|
7002
7072
|
To share a folder with a different user, special tags are set, but this is conveniently done by `ascli` using the `as` field.
|
|
7003
7073
|
|
|
7004
7074
|
##### User Shared Folders
|
|
7005
7075
|
|
|
7006
|
-
Personal shared folders, created by users in a workspace follow the syntax:
|
|
7076
|
+
Personal shared folders, created by users in a workspace, follow the syntax:
|
|
7007
7077
|
|
|
7008
7078
|
```shell
|
|
7009
7079
|
ascli aoc files permission --workspace=<WORKSPACE_NAME> <PATH_TO_FOLDER> ...
|
|
@@ -7024,7 +7094,7 @@ ascli aoc admin node do <NODE_ID> permission --workspace=<WORKSPACE_NAME> <PATH_
|
|
|
7024
7094
|
> [!TIP]
|
|
7025
7095
|
> The node is identified by identifier.
|
|
7026
7096
|
> To use a name instead, one can use the [percent selector](#percent-selector), like `%name:"<NODE_NAME>"`.
|
|
7027
|
-
> The
|
|
7097
|
+
> The folder is identified by its path; a file ID can be specified instead, with `%id:123`.
|
|
7028
7098
|
> If the ID is left blank: `%id:`, then it means `*`, that is, "all".
|
|
7029
7099
|
|
|
7030
7100
|
##### Example: List permissions on a user shared folder
|
|
@@ -7054,8 +7124,8 @@ ascli aoc files short_link <PATH_TO_FOLDER> private create
|
|
|
7054
7124
|
ascli aoc files short_link <PATH_TO_FOLDER> private list
|
|
7055
7125
|
ascli aoc files short_link <PATH_TO_FOLDER> public create @json:'{...}'
|
|
7056
7126
|
ascli aoc files short_link <PATH_TO_FOLDER> public list
|
|
7057
|
-
ascli aoc files short_link public delete <ID>
|
|
7058
|
-
ascli aoc files short_link public modify <ID> @json:'{...}'
|
|
7127
|
+
ascli aoc files short_link <PATH_TO_FOLDER> public delete <ID>
|
|
7128
|
+
ascli aoc files short_link <PATH_TO_FOLDER> public modify <ID> @json:'{...}'
|
|
7059
7129
|
```
|
|
7060
7130
|
|
|
7061
7131
|
Only `public` short links can be modified.
|
|
@@ -7176,7 +7246,7 @@ ascli aoc admin node do <NODE_ID> permission <FOLDER_PATH> create @json:'{"with"
|
|
|
7176
7246
|
> [!NOTE]
|
|
7177
7247
|
> In the previous commands, field `as` is optional.
|
|
7178
7248
|
|
|
7179
|
-
##### Example: List all workspace admin shared
|
|
7249
|
+
##### Example: List all workspace admin shared folders in a workspace
|
|
7180
7250
|
|
|
7181
7251
|
```shell
|
|
7182
7252
|
ascli aoc admin workspace shared_folder %name:'<WORKSPACE_NAME>' list
|
|
@@ -7210,10 +7280,10 @@ ascli aoc admin workspace shared_folder %name:'<WORKSPACE_NAME>' member 198 list
|
|
|
7210
7280
|
If you have the node ID of the shared folder, then it is equivalent to:
|
|
7211
7281
|
|
|
7212
7282
|
```shell
|
|
7213
|
-
ascli aoc admin node do
|
|
7283
|
+
ascli aoc admin node do 8666 permission /project1 list --query=@json:'{"tag":"aspera.files.workspace.id=<WORKSPACE_ID>"}'
|
|
7214
7284
|
```
|
|
7215
7285
|
|
|
7216
|
-
##### Example: List all workspace admin shared
|
|
7286
|
+
##### Example: List all workspace admin shared folders on a node
|
|
7217
7287
|
|
|
7218
7288
|
First get the workspace identifier:
|
|
7219
7289
|
|
|
@@ -7245,14 +7315,14 @@ Although optional, the creation of [Option Preset](#option-preset) is recommende
|
|
|
7245
7315
|
|
|
7246
7316
|
Procedure to send a file from org1 to org2:
|
|
7247
7317
|
|
|
7248
|
-
- Get access to Organization 1 and create an [Option Preset](#option-preset)
|
|
7249
|
-
- Check that access works and locate the source
|
|
7250
|
-
- Get access to Organization 2 and create an [Option Preset](#option-preset)
|
|
7318
|
+
- Get access to Organization 1 and create an [Option Preset](#option-preset), for example `org1` (for instance, using the [Wizard](#wizard))
|
|
7319
|
+
- Check that access works and locate the source folder `<SOURCE_FOLDER>` and the file `<SOURCE_FILE>` in it, for example, using command `files browse`
|
|
7320
|
+
- Get access to Organization 2 and create an [Option Preset](#option-preset), for example `org2`
|
|
7251
7321
|
- Check that access works and locate the destination folder `<DEST_FOLDER>`
|
|
7252
7322
|
- Execute the following:
|
|
7253
7323
|
|
|
7254
7324
|
```shell
|
|
7255
|
-
ascli -Porg1 aoc files node_info <
|
|
7325
|
+
ascli -Porg1 aoc files node_info <SOURCE_FOLDER> --format=json --out.level=data | ascli -Porg2 aoc files upload <SOURCE_FILE> --to-folder=<DEST_FOLDER> --transfer=@json:@stdin: --transfer.agent=node
|
|
7256
7326
|
```
|
|
7257
7327
|
|
|
7258
7328
|
Explanation:
|
|
@@ -7260,13 +7330,14 @@ Explanation:
|
|
|
7260
7330
|
- `ascli` is the command executed by the shell
|
|
7261
7331
|
- `-Porg1` loads options for preset `org1` (URL and credentials)
|
|
7262
7332
|
- `aoc` uses the Aspera on Cloud plugin
|
|
7263
|
-
- `files node_info
|
|
7333
|
+
- `files node_info <SOURCE_FOLDER>` generates transfer information for the source folder: Node API URL, credentials and root file ID, suitable for the next command
|
|
7264
7334
|
- `--format=json` formats the output as JSON (instead of the default text table)
|
|
7265
7335
|
- `--out.level=data` displays only the result, removing other information such as workspace name
|
|
7266
7336
|
- `|` pipes the standard output of the first command into the second one
|
|
7267
7337
|
- `-Porg2 aoc` uses the Aspera on Cloud plugin and loads credentials for `org2`
|
|
7268
|
-
- `files upload <SOURCE_FILE>` uploads the file
|
|
7269
|
-
- `--transfer=@json:@stdin:`
|
|
7338
|
+
- `files upload <SOURCE_FILE> --to-folder=<DEST_FOLDER>` uploads the file `<SOURCE_FILE>` (located in `<SOURCE_FOLDER>` of `org1`) to `<DEST_FOLDER>` in `org2`
|
|
7339
|
+
- `--transfer=@json:@stdin:` reads the Node API information from standard input (JSON) and uses it as parameters of the transfer agent
|
|
7340
|
+
- `--transfer.agent=node` selects the `node` transfer agent: the source node of `org1` pushes the file to `org2`
|
|
7270
7341
|
|
|
7271
7342
|
#### Find Files
|
|
7272
7343
|
|
|
@@ -7348,6 +7419,8 @@ admin subscription usage
|
|
|
7348
7419
|
admin subscription usage MONTH
|
|
7349
7420
|
admin user list
|
|
7350
7421
|
admin user modify %name:my_user_email @: deactivated=false
|
|
7422
|
+
admin user notifications %name:my_user_email show
|
|
7423
|
+
admin user preferences %name:my_user_email show
|
|
7351
7424
|
admin workspace dropbox %name:my_other_workspace list
|
|
7352
7425
|
admin workspace list
|
|
7353
7426
|
admin workspace shared_folder %name:my_other_workspace list
|
|
@@ -7364,9 +7437,9 @@ bearer_token --out.level=data
|
|
|
7364
7437
|
files bearer /
|
|
7365
7438
|
files bearer_token_node / --cache-tokens=no
|
|
7366
7439
|
files browse /
|
|
7367
|
-
files browse / --url=
|
|
7440
|
+
files browse / --url=my_private_link_shared_folder
|
|
7368
7441
|
files browse / --url=my_public_link_folder_no_pass
|
|
7369
|
-
files browse / --url=
|
|
7442
|
+
files browse / --url=my_public_link_folder_with_pass --password=my_public_link_folder_password
|
|
7370
7443
|
files browse my_remote_file
|
|
7371
7444
|
files browse my_remote_folder
|
|
7372
7445
|
files browse my_remote_folder/
|
|
@@ -7418,7 +7491,13 @@ packages send @: 'name=package title' END test_file.bin --url=my_public_link_sen
|
|
|
7418
7491
|
packages send @: 'name=package title' recipients.0=my_username 'note=some notes' END test_file.bin
|
|
7419
7492
|
packages send @json:'{"name":"package title","recipients":["my_email_external"]}' --new-user-option.package_contact=true test_file.bin
|
|
7420
7493
|
packages shared_inboxes list
|
|
7494
|
+
packages shared_inboxes short_link public %name:my_shared_inbox_name create --fields=id
|
|
7495
|
+
packages shared_inboxes short_link public %name:my_shared_inbox_name delete <aoc_shared_inbox_short_link_create>
|
|
7496
|
+
packages shared_inboxes short_link public %name:my_shared_inbox_name list
|
|
7497
|
+
packages shared_inboxes short_link public %name:my_shared_inbox_name modify <aoc_shared_inbox_short_link_create> @: password=my_public_link_folder_password
|
|
7498
|
+
packages shared_inboxes short_link public %name:my_shared_inbox_name show <aoc_shared_inbox_short_link_create>
|
|
7421
7499
|
packages shared_inboxes show %name:my_shared_inbox_name
|
|
7500
|
+
packages show '%name:package title'
|
|
7422
7501
|
remind --username=my_user_email --url=https://aoc.example.com/path
|
|
7423
7502
|
servers --url=https://aoc.example.com/path
|
|
7424
7503
|
tier_restrictions
|
|
@@ -7433,11 +7512,11 @@ user workspaces list
|
|
|
7433
7512
|
|
|
7434
7513
|
## Plugin: `ats`: IBM Aspera Transfer Service
|
|
7435
7514
|
|
|
7436
|
-
ATS is usable either
|
|
7515
|
+
ATS is usable either:
|
|
7437
7516
|
|
|
7438
|
-
- From an AoC subscription
|
|
7517
|
+
- From an AoC subscription: `ascli aoc admin ats`: use AoC authentication
|
|
7439
7518
|
|
|
7440
|
-
- Or from an IBM Cloud subscription
|
|
7519
|
+
- Or from an IBM Cloud subscription: `ascli ats`: use IBM Cloud API key authentication
|
|
7441
7520
|
|
|
7442
7521
|
### IBM Cloud ATS: Creation of API key
|
|
7443
7522
|
|
|
@@ -7445,7 +7524,7 @@ ATS is usable either :
|
|
|
7445
7524
|
> If you are using ATS as part of AoC, then authentication is through AoC, not IBM Cloud.
|
|
7446
7525
|
> See the AoC section instead.
|
|
7447
7526
|
|
|
7448
|
-
This section is about using ATS with an IBM
|
|
7527
|
+
This section is about using ATS with an IBM Cloud subscription.
|
|
7449
7528
|
|
|
7450
7529
|
First get your IBM Cloud API key.
|
|
7451
7530
|
For instance, it can be created using the IBM Cloud web interface, or using command line:
|
|
@@ -7492,11 +7571,7 @@ ascli ats api_key instances
|
|
|
7492
7571
|
```
|
|
7493
7572
|
|
|
7494
7573
|
```text
|
|
7495
|
-
|
|
7496
|
-
| instance |
|
|
7497
|
-
+--------------------------------------+
|
|
7498
|
-
| aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee |
|
|
7499
|
-
+--------------------------------------+
|
|
7574
|
+
aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
|
|
7500
7575
|
```
|
|
7501
7576
|
|
|
7502
7577
|
```shell
|
|
@@ -7504,22 +7579,26 @@ ascli config preset update <PRESET_NAME> --instance=aaaaaaaa-bbbb-cccc-dddd-eeee
|
|
|
7504
7579
|
```
|
|
7505
7580
|
|
|
7506
7581
|
```shell
|
|
7507
|
-
ascli ats api_key create
|
|
7582
|
+
ascli ats api_key create --out.secrets=yes
|
|
7508
7583
|
```
|
|
7509
7584
|
|
|
7510
7585
|
```text
|
|
7511
|
-
|
|
7512
|
-
|
|
7513
|
-
|
|
7514
|
-
|
|
7515
|
-
|
|
7516
|
-
|
|
7586
|
+
╭────────┬──────────────────────────────────────────────╮
|
|
7587
|
+
│ field │ value │
|
|
7588
|
+
╞════════╪══════════════════════════════════════════════╡
|
|
7589
|
+
│ id │ ats_XXXXXXXXXXXXXXXXXXXXXXXX │
|
|
7590
|
+
│ secret │ YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY │
|
|
7591
|
+
╰────────┴──────────────────────────────────────────────╯
|
|
7592
|
+
```
|
|
7593
|
+
|
|
7594
|
+
```shell
|
|
7517
7595
|
ascli config preset update <PRESET_NAME> --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
|
7518
7596
|
```
|
|
7519
7597
|
|
|
7520
7598
|
### ATS Access key creation parameters
|
|
7521
7599
|
|
|
7522
|
-
When creating an ATS access key, the
|
|
7600
|
+
When creating an ATS access key, the creation parameters are provided as a positional argument of `access_key create`, as a `Hash` [Extended Value](#extended-value-syntax).
|
|
7601
|
+
If key `transfer_server_id` is not provided, the transfer server is selected with options `cloud` and `region`.
|
|
7523
7602
|
Those are directly the parameters expected by the [ATS API](https://developer.ibm.com/apis/catalog?search=%22Aspera%20ATS%20API%22).
|
|
7524
7603
|
|
|
7525
7604
|
### Misc. Examples
|
|
@@ -7527,34 +7606,34 @@ Those are directly the parameters expected by the [ATS API](https://developer.ib
|
|
|
7527
7606
|
Example: create access key on IBM Cloud (Softlayer):
|
|
7528
7607
|
|
|
7529
7608
|
```shell
|
|
7530
|
-
ascli ats access_key create
|
|
7609
|
+
ascli ats access_key create @json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"<SECRET>","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}' --cloud=softlayer --region=ams
|
|
7531
7610
|
```
|
|
7532
7611
|
|
|
7533
7612
|
Example: create access key on AWS:
|
|
7534
7613
|
|
|
7535
7614
|
```shell
|
|
7536
|
-
ascli ats access_key create
|
|
7615
|
+
ascli ats access_key create @json:'{"id":"<ACCESS_KEY>","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"_access_key_id_here_","secret_access_key":"<SECRET>"},"path":"/laurent"}}' --cloud=aws --region=eu-west-1
|
|
7537
7616
|
```
|
|
7538
7617
|
|
|
7539
7618
|
Example: create access key on Azure SAS:
|
|
7540
7619
|
|
|
7541
7620
|
```shell
|
|
7542
|
-
ascli ats access_key create
|
|
7621
|
+
ascli ats access_key create @json:'{"id":"<ACCESS_KEY>","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}' --cloud=azure --region=eastus
|
|
7543
7622
|
```
|
|
7544
7623
|
|
|
7545
7624
|
> [!NOTE]
|
|
7546
|
-
> The blob name is mandatory after server address and before parameters, and
|
|
7625
|
+
> The blob name is mandatory after the server address and before the parameters, and parameter `sr=c` is mandatory.
|
|
7547
7626
|
|
|
7548
7627
|
Example: create access key on Azure:
|
|
7549
7628
|
|
|
7550
7629
|
```shell
|
|
7551
|
-
ascli ats access_key create
|
|
7630
|
+
ascli ats access_key create @json:'{"id":"<ACCESS_KEY>","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"<ACCESS_KEY>","storage_endpoint":"myblob"},"path":"/"}}' --cloud=azure --region=eastus
|
|
7552
7631
|
```
|
|
7553
7632
|
|
|
7554
7633
|
Delete all access keys:
|
|
7555
7634
|
|
|
7556
7635
|
```shell
|
|
7557
|
-
ascli ats access_key list --
|
|
7636
|
+
ascli ats access_key list --fields=id --format=csv | ascli ats access_key delete @lines:@stdin: --bulk=yes
|
|
7558
7637
|
```
|
|
7559
7638
|
|
|
7560
7639
|
The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
|
|
@@ -7642,23 +7721,24 @@ upload 'faux:///test.bin?1k' --to-folder=my_upload_folder
|
|
|
7642
7721
|
upload --sources=@ts --transfer.ascp_args=@list:,--file-list,file_list.txt --to-folder=my_inside_folder
|
|
7643
7722
|
upload --sources=@ts --transfer.ascp_args=@list:,--file-pair-list,file_pair_list.txt
|
|
7644
7723
|
upload --sources=@ts --ts=@json:'{"paths":[{"source":"test_file.bin","destination":"my_inside_folder/other_name_4"}]}' --transfer.agent=transferd
|
|
7645
|
-
upload --src-type=pair --sources=@json:'["test_file.bin","my_inside_folder/other_name_3"]' --transfer.quiet=false
|
|
7724
|
+
upload --src-type=pair --sources=@json:'["test_file.bin","my_inside_folder/other_name_3"]' --transfer.quiet=false
|
|
7646
7725
|
upload --src-type=pair test_file.bin my_inside_folder/other_name_2 --notify-to=my_email_external '--transfer.ascp_args=@list: -l 100m'
|
|
7647
7726
|
upload --src-type=pair test_file.bin my_upload_folder/other_name_5 --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"my_secret_here","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":"-1","exclude_older_than":"-10000","fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum"}'
|
|
7648
7727
|
upload --to-folder=my_upload_folder/target_hot --lock-port=50101 --transfer.ascp_args=@list:,--remove-after-transfer,--remove-empty-directories,--exclude-newer-than=-8,--src-base,hot_folder hot_folder
|
|
7728
|
+
upload /test_file.bin --to-folder=my_upload_folder --transfer.agent=node --transfer.url=http://localhost:12348 --transfer.username=sim --transfer.password=sim
|
|
7649
7729
|
upload test_file.bin --to-folder=my_inside_folder --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":100000}' --transfer=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress-bar=yes
|
|
7650
7730
|
```
|
|
7651
7731
|
|
|
7652
7732
|
### Authentication on Server with SSH session
|
|
7653
7733
|
|
|
7654
|
-
If SSH is the session protocol (by default, that is, not WSS), then following session authentication methods are supported:
|
|
7734
|
+
If SSH is the session protocol (by default, that is, not WSS), then the following session authentication methods are supported:
|
|
7655
7735
|
|
|
7656
7736
|
- `password`: SSH password
|
|
7657
7737
|
- `ssh_keys`: SSH keys (Multiple SSH key paths can be provided.)
|
|
7658
7738
|
|
|
7659
7739
|
If `username` is not provided then the default transfer user `xfer` is used.
|
|
7660
7740
|
|
|
7661
|
-
If neither SSH password nor key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass
|
|
7741
|
+
If neither SSH password nor key is provided and a transfer token is provided in transfer spec (option `ts`), then the standard SSH bypass keys are used.
|
|
7662
7742
|
|
|
7663
7743
|
Example:
|
|
7664
7744
|
|
|
@@ -7681,7 +7761,7 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
|
|
|
7681
7761
|
ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
|
|
7682
7762
|
```
|
|
7683
7763
|
|
|
7684
|
-
For file operation
|
|
7764
|
+
For file operation commands (browse, delete), the Ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options` (additive option like `ts`).
|
|
7685
7765
|
|
|
7686
7766
|
For a list of SSH client options, refer to the Ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html#method-c-start).
|
|
7687
7767
|
|
|
@@ -7695,19 +7775,19 @@ By default, the SSH library will check if a local `ssh-agent` is running.
|
|
|
7695
7775
|
|
|
7696
7776
|
On Linux, if you get an error message such as:
|
|
7697
7777
|
|
|
7698
|
-
```
|
|
7778
|
+
```text
|
|
7699
7779
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: Agent not configured
|
|
7700
7780
|
```
|
|
7701
7781
|
|
|
7702
7782
|
Or on Windows:
|
|
7703
7783
|
|
|
7704
|
-
```
|
|
7784
|
+
```text
|
|
7705
7785
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
|
7706
7786
|
```
|
|
7707
7787
|
|
|
7708
|
-
This means that your environment suggests using an agent, but
|
|
7788
|
+
This means that your environment suggests using an agent, but no SSH agent is running. In that case:
|
|
7709
7789
|
|
|
7710
|
-
- Check env var: `
|
|
7790
|
+
- Check env var: `SSH_AUTH_SOCK`
|
|
7711
7791
|
- Check your file: `$HOME/.ssh/config`
|
|
7712
7792
|
- Check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
|
|
7713
7793
|
- [Check the Ruby SSH options in start method](https://github.com/net-ssh/net-ssh/blob/master/lib/net/ssh.rb)
|
|
@@ -7725,15 +7805,15 @@ It is equivalent to setting both options `ssh_options.passphrase` and `ts.ssh_pr
|
|
|
7725
7805
|
|
|
7726
7806
|
### Other session channels for `server`
|
|
7727
7807
|
|
|
7728
|
-
URL schemes `local` and `https` are also supported (mainly for testing
|
|
7808
|
+
URL schemes `local` and `https` are also supported (mainly for testing purposes).
|
|
7729
7809
|
(`--url=local:`, `--url=https://...`)
|
|
7730
7810
|
|
|
7731
7811
|
- `local` will execute `ascmd` locally, instead of using an SSH connection.
|
|
7732
7812
|
- `https` will use Web Socket Session:
|
|
7733
7813
|
This requires the use of a transfer token.
|
|
7734
|
-
For example a `Basic` token can be used.
|
|
7814
|
+
For example, a `Basic` token can be used.
|
|
7735
7815
|
|
|
7736
|
-
As
|
|
7816
|
+
As SSH is used most of the time, if an `http` scheme is provided without a token, the plugin falls back to SSH and port 33001.
|
|
7737
7817
|
|
|
7738
7818
|
### Examples: `server`
|
|
7739
7819
|
|
|
@@ -7750,14 +7830,14 @@ ascli server download /aspera-test-dir-large/200MB
|
|
|
7750
7830
|
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options` (for browsing) and `ts` (for transfers):
|
|
7751
7831
|
|
|
7752
7832
|
```shell
|
|
7753
|
-
ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --
|
|
7833
|
+
ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ssh-keys=_private_key_path_here_ --passphrase=_passphrase_here_
|
|
7754
7834
|
```
|
|
7755
7835
|
|
|
7756
7836
|
## Plugin: `node`: IBM Aspera High Speed Transfer Server Node
|
|
7757
7837
|
|
|
7758
7838
|
This plugin gives access to capabilities provided by the HSTS Node API.
|
|
7759
7839
|
|
|
7760
|
-
|
|
7840
|
+
Authentication uses either a Node API username and password, or an access key and secret, provided with options `username` and `password`.
|
|
7761
7841
|
|
|
7762
7842
|
> [!NOTE]
|
|
7763
7843
|
> Capabilities of this plugin are used in other plugins that access the Node API, such as `aoc`, `ats`, `shares`.
|
|
@@ -7780,7 +7860,7 @@ When using an access key, the so-called **gen4/access key** API is also supporte
|
|
|
7780
7860
|
Example:
|
|
7781
7861
|
|
|
7782
7862
|
- `ascli node browse /` : list files with **gen3/node user** API
|
|
7783
|
-
- `ascli node
|
|
7863
|
+
- `ascli node access_keys do self browse /` : list files with **gen4/access key** API
|
|
7784
7864
|
|
|
7785
7865
|
#### Browse
|
|
7786
7866
|
|
|
@@ -7801,7 +7881,7 @@ Special parameters can be placed in option `query` for "gen3" browse:
|
|
|
7801
7881
|
|
|
7802
7882
|
##### Gen4
|
|
7803
7883
|
|
|
7804
|
-
This is when executing `browse` in `aoc files` or in `
|
|
7884
|
+
This is when executing `browse` in `aoc files` or in `node access_keys do`.
|
|
7805
7885
|
|
|
7806
7886
|
Option `node_api` (`Hash`) controls some options of API used, with the following parameters:
|
|
7807
7887
|
|
|
@@ -7832,7 +7912,7 @@ Examples of expressions:
|
|
|
7832
7912
|
- Find all files and folders under `/`
|
|
7833
7913
|
|
|
7834
7914
|
```shell
|
|
7835
|
-
ascli node access_keys do self find
|
|
7915
|
+
ascli node access_keys do self find /
|
|
7836
7916
|
```
|
|
7837
7917
|
|
|
7838
7918
|
- Find all text files in `/Documents`
|
|
@@ -7913,7 +7993,7 @@ Other query parameters are passed through to the underlying API (`GET /ops/trans
|
|
|
7913
7993
|
The `central` sub-command uses the **reliable query** API (session and file).
|
|
7914
7994
|
Use it to list transfer sessions and transferred files.
|
|
7915
7995
|
|
|
7916
|
-
To
|
|
7996
|
+
To list transferred files:
|
|
7917
7997
|
|
|
7918
7998
|
```shell
|
|
7919
7999
|
ascli node central file list
|
|
@@ -7943,7 +8023,7 @@ For the `async` subcommands `show` and `delete`, you can use the special identif
|
|
|
7943
8023
|
You can start a FASP Stream session from the Node API.
|
|
7944
8024
|
|
|
7945
8025
|
Run the following command:
|
|
7946
|
-
`ascli node stream create
|
|
8026
|
+
`ascli node stream create @json:<VALUE>`
|
|
7947
8027
|
with the following [**transfer-spec**](#transfer-specification):
|
|
7948
8028
|
|
|
7949
8029
|
```json
|
|
@@ -7976,17 +8056,17 @@ ascli node central file list --validator=ascli @json:'{"file_transfer_filter":{"
|
|
|
7976
8056
|
```
|
|
7977
8057
|
|
|
7978
8058
|
```text
|
|
7979
|
-
|
|
7980
|
-
|
|
7981
|
-
|
|
7982
|
-
|
|
7983
|
-
|
|
8059
|
+
╭──────────────┬──────────────┬────────────┬─────────────────────────────────────╮
|
|
8060
|
+
│ session_uuid │ file_id │ status │ path │
|
|
8061
|
+
╞══════════════╪══════════════╪════════════╪═════════════════════════════════════╡
|
|
8062
|
+
│ 1a74444c-... │ 084fb181-... │ validating │ /home/xfer.../PKG - <TITLE>/200KB.1 │
|
|
8063
|
+
╰──────────────┴──────────────┴────────────┴─────────────────────────────────────╯
|
|
7984
8064
|
```
|
|
7985
8065
|
|
|
7986
8066
|
To update the status of the file, use the following command:
|
|
7987
8067
|
|
|
7988
8068
|
```shell
|
|
7989
|
-
ascli node central file
|
|
8069
|
+
ascli node central file modify --validator=ascli @json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
|
|
7990
8070
|
```
|
|
7991
8071
|
|
|
7992
8072
|
```text
|
|
@@ -7998,12 +8078,12 @@ updated
|
|
|
7998
8078
|
Scenario: Access to a **Shares on Demand** (SHOD) server on AWS is provided by a partner.
|
|
7999
8079
|
We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
|
|
8000
8080
|
Create an **Aspera Transfer Service** instance, which provides access to the Node API.
|
|
8001
|
-
Then create
|
|
8002
|
-
Create another
|
|
8081
|
+
Then create an [Option Preset](#option-preset) with the Node API URL and credentials of the **SHOD** instance, named `aws_shod`.
|
|
8082
|
+
Create another [Option Preset](#option-preset) for the Azure ATS instance, named `azure_ats`.
|
|
8003
8083
|
Then execute the following command:
|
|
8004
8084
|
|
|
8005
8085
|
```shell
|
|
8006
|
-
ascli node download /share/sourcefile --to-folder=/destination_folder --preset=aws_shod --transfer=@preset:azure_ats
|
|
8086
|
+
ascli node download /share/sourcefile --to-folder=/destination_folder --preset=aws_shod --transfer=@preset:azure_ats --transfer.agent=node
|
|
8007
8087
|
```
|
|
8008
8088
|
|
|
8009
8089
|
This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
|
|
@@ -8023,10 +8103,10 @@ gem install rmagick rainbow
|
|
|
8023
8103
|
For example, it is possible to display the preview of a file, if it exists, using an access key on node:
|
|
8024
8104
|
|
|
8025
8105
|
```shell
|
|
8026
|
-
ascli node
|
|
8106
|
+
ascli node access_keys do self thumbnail /preview_samples/Aspera.mpg
|
|
8027
8107
|
```
|
|
8028
8108
|
|
|
8029
|
-
Previews are mainly used in AoC
|
|
8109
|
+
Previews are mainly used in AoC; this also works with AoC:
|
|
8030
8110
|
|
|
8031
8111
|
```shell
|
|
8032
8112
|
ascli aoc files thumbnail /preview_samples/Aspera.mpg
|
|
@@ -8041,12 +8121,12 @@ ascli aoc files thumbnail /preview_samples/Aspera.mpg
|
|
|
8041
8121
|
### Creating an access key
|
|
8042
8122
|
|
|
8043
8123
|
```shell
|
|
8044
|
-
ascli node
|
|
8124
|
+
ascli node access_keys create @json:'{"id":"<ACCESS_KEY>","secret":"<SECRET>","storage":{"type":"local","path":"/data/mydir"}}'
|
|
8045
8125
|
```
|
|
8046
8126
|
|
|
8047
8127
|
> [!TIP]
|
|
8048
8128
|
> The `id` and `secret` fields are optional.
|
|
8049
|
-
> If not provided, they will be generated and returned
|
|
8129
|
+
> If not provided, they will be generated and returned in the result.
|
|
8050
8130
|
> In that case, provide option `--out.secrets=yes` to get the generated secret.
|
|
8051
8131
|
|
|
8052
8132
|
Access keys support extra overriding parameters using parameter: `configuration` and sub keys `transfer` and `server`.
|
|
@@ -8059,7 +8139,7 @@ For example, an access key can be modified or created with the following options
|
|
|
8059
8139
|
The list of supported options can be displayed using command:
|
|
8060
8140
|
|
|
8061
8141
|
```shell
|
|
8062
|
-
ascli node info --
|
|
8142
|
+
ascli node info --fields=@ruby:'/^access_key_configuration_capabilities.*/'
|
|
8063
8143
|
```
|
|
8064
8144
|
|
|
8065
8145
|
### Generating and using a bearer token
|
|
@@ -8106,7 +8186,7 @@ The way to create access keys depends slightly on the type of HSTS:
|
|
|
8106
8186
|
It has no `docroot` but has at least one file restriction (for testing, one can use `*` to accept creation of an access key with any storage root path).
|
|
8107
8187
|
See the Aspera HSTS documentation.
|
|
8108
8188
|
|
|
8109
|
-
- If Cloud Pak for
|
|
8189
|
+
- If Cloud Pak for Integration is used, then the node admin is created automatically.
|
|
8110
8190
|
|
|
8111
8191
|
- If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
|
|
8112
8192
|
|
|
@@ -8118,7 +8198,7 @@ The following sections assume that an access key has been created and that `ascl
|
|
|
8118
8198
|
#### Bearer token: Preparation
|
|
8119
8199
|
|
|
8120
8200
|
Assume that the access key was created, and a default configuration is set to use this **access key**.
|
|
8121
|
-
Using `ascli`, an access key can be created using the `
|
|
8201
|
+
Using `ascli`, an access key can be created using the `access_keys create` command on the node (using main node credentials) or on ATS.
|
|
8122
8202
|
|
|
8123
8203
|
Create a private key (organization key) that will be used to sign bearer tokens:
|
|
8124
8204
|
|
|
@@ -8135,18 +8215,18 @@ ascli config genkey $my_private_pem
|
|
|
8135
8215
|
The corresponding public key shall be placed as an attribute of the **access key** (done with `PUT /access_keys/<ID>`):
|
|
8136
8216
|
|
|
8137
8217
|
```shell
|
|
8138
|
-
ascli node
|
|
8218
|
+
ascli node access_keys set_bearer_key self @file:$my_private_pem
|
|
8139
8219
|
```
|
|
8140
8220
|
|
|
8141
8221
|
> [!NOTE]
|
|
8142
8222
|
> Either the public or private key can be provided, and only the public key is used.
|
|
8143
|
-
> This
|
|
8223
|
+
> This enables checking the signature of bearer tokens.
|
|
8144
8224
|
> Above command is executed with access key credentials.
|
|
8145
8225
|
|
|
8146
|
-
Alternatively, use the following equivalent command, as `
|
|
8226
|
+
Alternatively, use the following equivalent command, as `config genkey` also saves the public key with extension `.pub`:
|
|
8147
8227
|
|
|
8148
8228
|
```shell
|
|
8149
|
-
ascli node
|
|
8229
|
+
ascli node access_keys modify %id:self @ruby:'{token_verification_key: File.read("'$my_private_pem'.pub")}'
|
|
8150
8230
|
```
|
|
8151
8231
|
|
|
8152
8232
|
#### Bearer token: Configuration for user
|
|
@@ -8154,7 +8234,7 @@ ascli node access_key modify %id:self @ruby:'{token_verification_key: File.read(
|
|
|
8154
8234
|
- Select a folder for which to grant access to a user, and get its identifier:
|
|
8155
8235
|
|
|
8156
8236
|
```shell
|
|
8157
|
-
my_folder_id=$(ascli node
|
|
8237
|
+
my_folder_id=$(ascli node access_keys do self show / --fields=id)
|
|
8158
8238
|
```
|
|
8159
8239
|
|
|
8160
8240
|
> [!NOTE]
|
|
@@ -8173,7 +8253,7 @@ ascli node access_key modify %id:self @ruby:'{token_verification_key: File.read(
|
|
|
8173
8253
|
- Grant this user access to the selected folder:
|
|
8174
8254
|
|
|
8175
8255
|
```shell
|
|
8176
|
-
ascli node
|
|
8256
|
+
ascli node access_keys do self permission %id:$my_folder_id create @json:'{"access_type":"user","access_id":"'$my_user_id'"}'
|
|
8177
8257
|
```
|
|
8178
8258
|
|
|
8179
8259
|
- Create a Bearer token for the user:
|
|
@@ -8198,7 +8278,7 @@ Assume the role of the user, with the following information:
|
|
|
8198
8278
|
To use this information:
|
|
8199
8279
|
|
|
8200
8280
|
```shell
|
|
8201
|
-
ascli node -N --url=https://... --password="Bearer $(cat bearer.txt)" --root-id=$my_folder_id
|
|
8281
|
+
ascli node -N --url=https://... --password="Bearer $(cat bearer.txt)" --root-id=$my_folder_id access_keys do self browse /
|
|
8202
8282
|
```
|
|
8203
8283
|
|
|
8204
8284
|
### Tested commands for `node`
|
|
@@ -8207,6 +8287,8 @@ ascli node -N --url=https://... --password="Bearer $(cat bearer.txt)" --root-id=
|
|
|
8207
8287
|
> Add `ascli node` in front of the following commands:
|
|
8208
8288
|
|
|
8209
8289
|
```shell
|
|
8290
|
+
--url=http://localhost:12348 --username=sim --password=sim browse / --fields=path --format=csv --select=@json:'{"basename":"test_file.bin"}'
|
|
8291
|
+
--url=http://localhost:12348 --username=sim --password=sim transfer list --fields=status
|
|
8210
8292
|
--url=https://tst.example.com/path --password='Bearer <nd_bearer_token>' --root-id=<id> access_key do self browse /
|
|
8211
8293
|
access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
|
|
8212
8294
|
access_key delete my_username
|
|
@@ -8251,6 +8333,7 @@ central session list
|
|
|
8251
8333
|
delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
|
|
8252
8334
|
delete my_upload_folder/test_file.bin
|
|
8253
8335
|
download my_upload_folder/test_file.bin --to-folder=.
|
|
8336
|
+
emulator @json:'{"url":"http://localhost:12348","username":"sim","password":"sim","docroot":"/data"}'
|
|
8254
8337
|
health
|
|
8255
8338
|
info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
|
8256
8339
|
license
|
|
@@ -8318,7 +8401,7 @@ Identify the region and the endpoint URL will be `https://otlp-[region]-saas.ins
|
|
|
8318
8401
|
For convenience, those parameters can be provided in a preset, for example, named `otel_default`.
|
|
8319
8402
|
|
|
8320
8403
|
```shell
|
|
8321
|
-
ascli config preset
|
|
8404
|
+
ascli config preset initialize otel_default @json:'{"url":"https://otlp-orange-saas.instana.io:4318","key":"*********","interval":1.1}'
|
|
8322
8405
|
```
|
|
8323
8406
|
|
|
8324
8407
|
Then it is invoked like this (assuming a default node is configured):
|
|
@@ -8333,11 +8416,84 @@ In Instana, create a custom Dashboard to visualize the OTel data:
|
|
|
8333
8416
|
- Data Source: Infrastructure and Platforms
|
|
8334
8417
|
- Metric: search `transfer`
|
|
8335
8418
|
|
|
8419
|
+
### Node emulator
|
|
8420
|
+
|
|
8421
|
+
> [!NOTE]
|
|
8422
|
+
> This is not a feature for production.
|
|
8423
|
+
> It's provided for testing only.
|
|
8424
|
+
|
|
8425
|
+
The command `emulator` starts a local web server that answers a subset of the Node API, and executes transfers with the Transfer Daemon (`transferd`).
|
|
8426
|
+
It allows testing, without HSTS, the [Node API agent](#agent-node-api) (`--transfer.agent=node`) and the `node` commands `info`, `browse`, `transfer list|show|modify|cancel`.
|
|
8427
|
+
|
|
8428
|
+
The Transfer Daemon and the gem `grpc` must be installed (see [Agent: Transfer Daemon](#agent-transfer-daemon)).
|
|
8429
|
+
The emulator starts its own `transferd`, and stops it on exit.
|
|
8430
|
+
|
|
8431
|
+
It takes an optional `Hash` argument with the following parameters:
|
|
8432
|
+
|
|
8433
|
+
| Field | Type | Description |
|
|
8434
|
+
|---------------|---------|----------------------------------------------------------------------------------|
|
|
8435
|
+
| `cert` | `String` | Path to the TLS certificate file. Accepted formats: PEM (`.pem`) or PKCS12 (`.p12` / `.pfx`).<br/>Example: `/path/to/cert.pem`. |
|
|
8436
|
+
| `chain` | `String` | Path to the PEM certificate chain file (appended as extra chain certificates).<br/>Example: `/path/to/chain.pem`. |
|
|
8437
|
+
| `docroot` | `String` | Local folder of the node files. Paths of `/files/browse` and local paths of transfers (sources of `send`, destination of `receive`) are relative to it, and confined in it. Defaults to the current working directory.<br/>Example: `/data/aspera`. |
|
|
8438
|
+
| `key` | `String` | Path to the PEM private key file, or the PKCS12 passphrase when `cert` is a `.p12`/`.pfx` file.<br/>Example: `/path/to/key.pem`. |
|
|
8439
|
+
| `password` | `String` | Password expected from clients in HTTP Basic authentication, for the above `username`.<br/>Example: `my_password`. |
|
|
8440
|
+
| `retention_sec` | `Integer` | Time in seconds a transfer stays in the list of transfers after it ended (completed, failed or canceled, and no more retried).<br/>Default: `86400`. |
|
|
8441
|
+
| `url` | `String` | Address and port the emulator listens on. Use `https://` with `cert`/`key` for TLS.<br/>Default: `http://localhost:8080`. |
|
|
8442
|
+
| `username` | `String` | Username expected from clients in HTTP Basic authentication. Set together with `password`. When not set, requests are accepted without authentication.<br/>Example: `node_user`. |
|
|
8443
|
+
|
|
8444
|
+
For details on `url` and HTTPS, see [Web service](#web-service).
|
|
8445
|
+
|
|
8446
|
+
Like on a real node, the files of the emulator are in its `docroot`: paths of `browse`, and local paths of transfers, are relative to it.
|
|
8447
|
+
Local paths are the sources of an upload (`send`), and the destination of a download (`receive`).
|
|
8448
|
+
Paths leading out of the `docroot` are rejected.
|
|
8449
|
+
|
|
8450
|
+
Start the emulator, it runs until interrupted:
|
|
8451
|
+
|
|
8452
|
+
```shell
|
|
8453
|
+
ascli node emulator @json:'{"url":"http://localhost:12348","username":"sim","password":"sim","docroot":"/data"}'
|
|
8454
|
+
```
|
|
8455
|
+
|
|
8456
|
+
Then, in another terminal, use it as a node, for example, to list the files in `/data`:
|
|
8457
|
+
|
|
8458
|
+
```shell
|
|
8459
|
+
ascli node --url=http://localhost:12348 --username=sim --password=sim browse /
|
|
8460
|
+
```
|
|
8461
|
+
|
|
8462
|
+
Or as the transfer agent, for example, to upload the file `/data/my_file.dat` to a transfer server:
|
|
8463
|
+
|
|
8464
|
+
```shell
|
|
8465
|
+
ascli server upload /my_file.dat --transfer.agent=node --transfer.url=http://localhost:12348 --transfer.username=sim --transfer.password=sim
|
|
8466
|
+
```
|
|
8467
|
+
|
|
8468
|
+
The transfer is executed by `transferd`, and its status is available on the emulator:
|
|
8469
|
+
|
|
8470
|
+
```shell
|
|
8471
|
+
ascli node --url=http://localhost:12348 --username=sim --password=sim transfer list
|
|
8472
|
+
```
|
|
8473
|
+
|
|
8474
|
+
Supported endpoints:
|
|
8475
|
+
|
|
8476
|
+
| Verb | Path | Action |
|
|
8477
|
+
|----------|-----------------------|---------------------------------------------------------------------------------------------------|
|
|
8478
|
+
| `GET` | `/info` | Node information, version and license from `transferd`. |
|
|
8479
|
+
| `GET` | `/ops/transfers` | List transfers. Query parameters: `active_only`, `direction`, `count`. |
|
|
8480
|
+
| `POST` | `/ops/transfers` | Start a transfer with `transferd`. |
|
|
8481
|
+
| `GET` | `/ops/transfers/{id}` | Transfer information, with sessions and files. |
|
|
8482
|
+
| `PUT` | `/ops/transfers/{id}` | Modify `target_rate_kbps`, `min_rate_kbps` or `rate_policy`, or cancel with `status`: `canceled`. |
|
|
8483
|
+
| `CANCEL` | `/ops/transfers/{id}` | Cancel a transfer. |
|
|
8484
|
+
| `POST` | `/files/browse` | List a folder of the `docroot`. |
|
|
8485
|
+
|
|
8486
|
+
Limitations:
|
|
8487
|
+
|
|
8488
|
+
- Only transfers started through the emulator are known. They are kept in memory, and lost when the emulator stops. Ended transfers are removed from the list after `retention_sec` (default: one day).
|
|
8489
|
+
- Only Basic authentication is supported: when `username` and `password` are set, bearer tokens and access keys are rejected. When they are not set, all requests are accepted.
|
|
8490
|
+
- Not supported: `files/upload_setup` and `files/download_setup` (so, `node upload|download` on the emulator), `ops/transfers/bandwidth`, pause and resume of transfers, query parameters `iteration_token` and `tag`.
|
|
8491
|
+
|
|
8336
8492
|
## Plugin: `faspex5`: IBM Aspera Faspex v5
|
|
8337
8493
|
|
|
8338
|
-
IBM Aspera's newer self-managed application.
|
|
8494
|
+
Faspex 5 is IBM Aspera's newer self-managed application.
|
|
8339
8495
|
|
|
8340
|
-
|
|
8496
|
+
The following authentication methods are supported (option `auth`):
|
|
8341
8497
|
|
|
8342
8498
|
| Method | Description |
|
|
8343
8499
|
|---------------|---------------------------------------------------------------------|
|
|
@@ -8363,20 +8519,20 @@ Then, answer questions interactively:
|
|
|
8363
8519
|
argument: url> faspex5.example.com
|
|
8364
8520
|
```
|
|
8365
8521
|
|
|
8366
|
-
|
|
8522
|
+
If multiple applications are detected, the wizard asks which one to use (this step is skipped if only Faspex is detected):
|
|
8367
8523
|
|
|
8368
8524
|
```text
|
|
8369
8525
|
Multiple applications detected:
|
|
8370
|
-
|
|
8371
|
-
|
|
8372
|
-
|
|
8373
|
-
|
|
8374
|
-
|
|
8375
|
-
|
|
8526
|
+
╭─────────┬───────────────────────────────────────────┬─────────────╮
|
|
8527
|
+
│ product │ url │ version │
|
|
8528
|
+
╞═════════╪═══════════════════════════════════════════╪═════════════╡
|
|
8529
|
+
│ faspex5 │ https://faspex5.example.com/aspera/faspex │ F5.0.6 │
|
|
8530
|
+
│ server │ ssh://faspex5.example.com:22 │ OpenSSH_8.3 │
|
|
8531
|
+
╰─────────┴───────────────────────────────────────────┴─────────────╯
|
|
8376
8532
|
product> faspex5
|
|
8377
8533
|
```
|
|
8378
8534
|
|
|
8379
|
-
When Faspex is detected,
|
|
8535
|
+
When Faspex is detected, the wizard asks for the path to a private key.
|
|
8380
8536
|
If you do not have a private key, leave that field blank, and one will be generated or a previously generated key will be used.
|
|
8381
8537
|
|
|
8382
8538
|
```text
|
|
@@ -8403,7 +8559,7 @@ Create an API client with:
|
|
|
8403
8559
|
- name: ascli
|
|
8404
8560
|
- JWT: enabled
|
|
8405
8561
|
Then, logged in as someuser@example.com go to your profile:
|
|
8406
|
-
() → Account Settings → Preferences
|
|
8562
|
+
(User) → Account Settings → Preferences → Public Key in PEM:
|
|
8407
8563
|
-----BEGIN PUBLIC KEY-----
|
|
8408
8564
|
redacted
|
|
8409
8565
|
-----END PUBLIC KEY-----
|
|
@@ -8456,7 +8612,7 @@ Activation is in two steps:
|
|
|
8456
8612
|
- At the bottom, in the `Public key in PEM format` field, paste the **public key** that corresponds to the private key assigned to your account.
|
|
8457
8613
|
|
|
8458
8614
|
> [!TIP]
|
|
8459
|
-
> If you don
|
|
8615
|
+
> If you don't have a private key, see [Private Key](#private-key) to generate one.
|
|
8460
8616
|
|
|
8461
8617
|
Then use these options:
|
|
8462
8618
|
|
|
@@ -8472,7 +8628,7 @@ Then use these options:
|
|
|
8472
8628
|
> Use the `private_key` option to provide the PEM content (not the file path).
|
|
8473
8629
|
> To load from a file, prefix the path with `@file:`, for example, `@file:/path/to/key.pem`.
|
|
8474
8630
|
|
|
8475
|
-
Typically, users create a preset so they don
|
|
8631
|
+
Typically, users create a preset so they don't have to enter these options each time.
|
|
8476
8632
|
|
|
8477
8633
|
Example:
|
|
8478
8634
|
|
|
@@ -8486,7 +8642,7 @@ ascli faspex5 user profile show
|
|
|
8486
8642
|
|
|
8487
8643
|
### Faspex 5 web authentication
|
|
8488
8644
|
|
|
8489
|
-
For web-based authentication, the administrator must create an **API client** in Faspex
|
|
8645
|
+
For web-based authentication, the administrator must create an **API client** in Faspex to support an external web app:
|
|
8490
8646
|
|
|
8491
8647
|
- As Admin, Navigate to the web UI: Admin → Configurations → API Clients → Create
|
|
8492
8648
|
- Do not Activate JWT
|
|
@@ -8575,6 +8731,7 @@ gateway @: url=https://localhost:12346/aspera/faspex
|
|
|
8575
8731
|
health --url=https://f5.example.com/path
|
|
8576
8732
|
invitation list
|
|
8577
8733
|
invitations create @: email_address=aspera.user1+u@gmail.com
|
|
8734
|
+
packages browse --url=my_public_link_recv_fr_user /
|
|
8578
8735
|
packages browse <id> --query.recursive=true
|
|
8579
8736
|
packages delete <id>
|
|
8580
8737
|
packages list --box=ALL
|
|
@@ -8584,11 +8741,14 @@ packages list --box=outbox --fields=DEF,sender.email,recipients.0.recipient_type
|
|
|
8584
8741
|
packages list --query=@json:'{"mailbox":"inbox","status":"completed"}'
|
|
8585
8742
|
packages receive --box=my_shared_box_name <id> --to-folder=.
|
|
8586
8743
|
packages receive --box=my_workgroup --group-type=workgroups <id> --to-folder=.
|
|
8744
|
+
packages receive --url=my_public_link_recv_fr_user --to-folder=.
|
|
8745
|
+
packages receive --url=my_public_link_recv_fr_user ALL --to-folder=.
|
|
8587
8746
|
packages receive <id> --to-folder=. --ts.content_protection_password=my_secret_here
|
|
8747
|
+
packages receive <id> <f5_pack_first_file> --to-folder=. --ts.content_protection_password=my_secret_here
|
|
8588
8748
|
packages receive ALL --once-only=yes --to-folder=. --query.max=5
|
|
8589
8749
|
packages receive INIT --once-only=yes
|
|
8590
|
-
packages send --url=my_public_link_send_f5_user @json:'{"title":"test title"}' test_file.bin
|
|
8591
8750
|
packages send --url=my_public_link_send_shared_box @json:'{"title":"test title"}' test_file.bin
|
|
8751
|
+
packages send --url=my_public_link_send_to_user @json:'{"title":"test title"}' test_file.bin
|
|
8592
8752
|
packages send @: 'title=for shared inbox' recipients.0=my_shared_box_name metadata.Options=Opt1 'metadata.TextInput=example text' END test_file.bin
|
|
8593
8753
|
packages send @: 'title=test title' recipients.0.name=my_username END test_file.bin --ts.content_protection_password=my_secret_here
|
|
8594
8754
|
packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' test_file.bin
|
|
@@ -8620,7 +8780,7 @@ For multiple parameters or when copying directly from API documentation, `@json:
|
|
|
8620
8780
|
|
|
8621
8781
|
### Faspex 5: Inbox selection
|
|
8622
8782
|
|
|
8623
|
-
By default, package operations
|
|
8783
|
+
By default, package operations `receive` and `list` are performed on all the user's inboxes, not archived (`inbox_all`).
|
|
8624
8784
|
|
|
8625
8785
|
To select another inbox, use option `box` with one of the following values:
|
|
8626
8786
|
|
|
@@ -8636,7 +8796,7 @@ To select another inbox, use option `box` with one of the following values:
|
|
|
8636
8796
|
| `pending_history` | Archived pending packages. |
|
|
8637
8797
|
| `all` | All boxes accessible by current user. |
|
|
8638
8798
|
| `ALL` | All boxes of all users. **admin only**. |
|
|
8639
|
-
| `<NAME>` | Name of shared
|
|
8799
|
+
| `<NAME>` | Name of shared inbox or workgroup.<br/>If option `group_type` is `shared_inboxes`: name of a shared inbox (default).<br/>If `group_type` is `workgroups`: name of workgroup. |
|
|
8640
8800
|
|
|
8641
8801
|
> [!NOTE]
|
|
8642
8802
|
> In case the name of the `box` is an open value, use option `group_type` set to either `shared_inboxes` or `workgroups`.
|
|
@@ -8651,7 +8811,7 @@ ascli faspex5 packages send <PACKAGE_DATA> <FILE_LIST> ...
|
|
|
8651
8811
|
```
|
|
8652
8812
|
|
|
8653
8813
|
The `Hash` passed as a command parameter corresponds to the Faspex 5 API endpoint [`POST /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#createPackageRecord).
|
|
8654
|
-
See the API reference for a full list of supported fields, or inspect such request
|
|
8814
|
+
See the API reference for a full list of supported fields, or inspect such a request in the browser developer tools.
|
|
8655
8815
|
|
|
8656
8816
|
The following fields are required:
|
|
8657
8817
|
|
|
@@ -8729,7 +8889,7 @@ To limit automatic contact lookup to one or more specific types, include the `re
|
|
|
8729
8889
|
To enable content protection (CSEAR), set parameter `ear_enabled` to `true` in the package creation payload.
|
|
8730
8890
|
See the Faspex package creation API for full details.
|
|
8731
8891
|
|
|
8732
|
-
The following error is returned by Faspex
|
|
8892
|
+
The following error is returned by Faspex if CSEAR was not specified in the package creation and if it is configured as mandatory on the server:
|
|
8733
8893
|
|
|
8734
8894
|
```text
|
|
8735
8895
|
the provided encryption value (no) does not match the expected server side encryption value (yes)
|
|
@@ -8761,7 +8921,7 @@ Option `query` can be used to filter the list of packages, based on native API p
|
|
|
8761
8921
|
| `pmax` | Special | Maximum number of **pages** to request.<br/>Stop pages when the maximum is passed. |
|
|
8762
8922
|
|
|
8763
8923
|
A **Command Parameter** in last position, of type `Proc`, can be used to filter the list of packages.
|
|
8764
|
-
|
|
8924
|
+
The advantage of this method is that the expression can be any test, even complex, as it is Ruby code.
|
|
8765
8925
|
But the disadvantage is that the filtering is done in `ascli` and not in Faspex 5, so it is less efficient.
|
|
8766
8926
|
|
|
8767
8927
|
Examples:
|
|
@@ -8784,7 +8944,7 @@ Several entities support folder browsing: Packages, Nodes, Shared Folders.
|
|
|
8784
8944
|
All support two modes: paging and legacy API.
|
|
8785
8945
|
By default, paging is used.
|
|
8786
8946
|
|
|
8787
|
-
Option `query` is available with parameters supported by the API and `ascli
|
|
8947
|
+
Option `query` is available with parameters supported by the API and `ascli`:
|
|
8788
8948
|
|
|
8789
8949
|
| Parameter | Evaluation | Default | Description |
|
|
8790
8950
|
|-----------|--------------|-------------------| ----------------------------------------|
|
|
@@ -8821,7 +8981,8 @@ In this case, typically, only `completed` packages should be downloaded, so use
|
|
|
8821
8981
|
If a package is password protected, then the content protection password is asked interactively.
|
|
8822
8982
|
To keep the content encrypted, use option: `--ts=@json:'{"content_protection":null}'`, or provide the password instead of `null`.
|
|
8823
8983
|
|
|
8824
|
-
>
|
|
8984
|
+
> [!TIP]
|
|
8985
|
+
> If you use option `query` and/or positional `filter`, you can use the `list` command for a dry run.
|
|
8825
8986
|
|
|
8826
8987
|
### Faspex 5: List all shared inboxes and work groups
|
|
8827
8988
|
|
|
@@ -8832,29 +8993,29 @@ To keep the content encrypted, use option: `--ts=@json:'{"content_protection":nu
|
|
|
8832
8993
|
If you are a regular user, to list work groups you belong to:
|
|
8833
8994
|
|
|
8834
8995
|
```shell
|
|
8835
|
-
ascli faspex5 admin
|
|
8996
|
+
ascli faspex5 admin workgroups list
|
|
8836
8997
|
```
|
|
8837
8998
|
|
|
8838
|
-
If you are admin or manager, add option
|
|
8999
|
+
If you are admin or manager, add option `--query=@json:'{"all":true}'`: this lists items you manage, even if you do not belong to them.
|
|
8839
9000
|
Example:
|
|
8840
9001
|
|
|
8841
9002
|
```shell
|
|
8842
|
-
ascli faspex5 admin
|
|
9003
|
+
ascli faspex5 admin shared_inboxes list --query=@json:'{"all":true}' --fields=id,name
|
|
8843
9004
|
```
|
|
8844
9005
|
|
|
8845
9006
|
Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
|
|
8846
9007
|
|
|
8847
9008
|
```shell
|
|
8848
|
-
ascli faspex5 admin shared_inboxes
|
|
9009
|
+
ascli faspex5 admin shared_inboxes invite_external_collaborator '%name:the shared inbox' @: email_address=john@example.com
|
|
8849
9010
|
```
|
|
8850
9011
|
|
|
8851
9012
|
It is equivalent to:
|
|
8852
9013
|
|
|
8853
9014
|
```shell
|
|
8854
|
-
ascli faspex5 admin shared_inboxes
|
|
9015
|
+
ascli faspex5 admin shared_inboxes invite_external_collaborator '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
|
8855
9016
|
```
|
|
8856
9017
|
|
|
8857
|
-
Other payload parameters are possible for `
|
|
9018
|
+
Other payload parameters are possible for `invite_external_collaborator` in this last `Hash` **Command Parameter**:
|
|
8858
9019
|
|
|
8859
9020
|
```json
|
|
8860
9021
|
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3}
|
|
@@ -8869,7 +9030,7 @@ ascli faspex5 admin metadata_profiles create @json:'{"name":"the profile","defau
|
|
|
8869
9030
|
### Faspex 5: Create a Shared inbox with specific metadata profile
|
|
8870
9031
|
|
|
8871
9032
|
```shell
|
|
8872
|
-
ascli faspex5 admin
|
|
9033
|
+
ascli faspex5 admin shared_inboxes create @json:'{"name":"the shared inbox","metadata_profile_id":1}'
|
|
8873
9034
|
```
|
|
8874
9035
|
|
|
8875
9036
|
### Faspex 5: List content in Shared folder and send package from remote source
|
|
@@ -8891,7 +9052,7 @@ ascli faspex5 shared_folders list --fields=id,name
|
|
|
8891
9052
|
```
|
|
8892
9053
|
|
|
8893
9054
|
```shell
|
|
8894
|
-
ascli faspex5 shared_folders
|
|
9055
|
+
ascli faspex5 shared_folders browse %name:'Server Files' /folder
|
|
8895
9056
|
```
|
|
8896
9057
|
|
|
8897
9058
|
```shell
|
|
@@ -8899,7 +9060,7 @@ ascli faspex5 packages send @json:'{"title":"hello","recipients":[{"name":"_reci
|
|
|
8899
9060
|
```
|
|
8900
9061
|
|
|
8901
9062
|
> [!TIP]
|
|
8902
|
-
> The shared folder can be identified by its numerical `id` or by name using [percent selector](#percent-selector): `%<FIELD>:<VALUE>`.
|
|
9063
|
+
> The shared folder can be identified by its numerical `id` or by name using the [percent selector](#percent-selector): `%<FIELD>:<VALUE>`. For example: `--shared-folder=3` or `--shared-folder=%name:partages`.
|
|
8903
9064
|
|
|
8904
9065
|
### Faspex 5: Receive all packages (cargo)
|
|
8905
9066
|
|
|
@@ -8909,7 +9070,7 @@ To receive all packages, only once, through persistency of already received pack
|
|
|
8909
9070
|
ascli faspex5 packages receive ALL --once-only=yes --query=@json:'{"status":"completed"}'
|
|
8910
9071
|
```
|
|
8911
9072
|
|
|
8912
|
-
To initialize,
|
|
9073
|
+
To initialize, that is, skip all current packages, so that next time `ALL` is used, only newer packages are downloaded:
|
|
8913
9074
|
|
|
8914
9075
|
```shell
|
|
8915
9076
|
ascli faspex5 packages receive INIT --once-only=yes
|
|
@@ -8917,7 +9078,7 @@ ascli faspex5 packages receive INIT --once-only=yes
|
|
|
8917
9078
|
|
|
8918
9079
|
### Faspex 5: Invitations
|
|
8919
9080
|
|
|
8920
|
-
There are two types of invitations
|
|
9081
|
+
There are two types of invitations for package submission: public or private.
|
|
8921
9082
|
|
|
8922
9083
|
Public invitations are for external users; provide only the email address.
|
|
8923
9084
|
|
|
@@ -8925,7 +9086,7 @@ Public invitations are for external users; provide only the email address.
|
|
|
8925
9086
|
ascli faspex5 invitations create @json:'{"email_address":"john@example.com"}' --fields=access_url
|
|
8926
9087
|
```
|
|
8927
9088
|
|
|
8928
|
-
Private invitations are for internal users
|
|
9089
|
+
Private invitations are for internal users: provide the user or shared inbox identifier through field `recipient_name`.
|
|
8929
9090
|
|
|
8930
9091
|
### Faspex 5: Cleanup packages
|
|
8931
9092
|
|
|
@@ -8979,16 +9140,16 @@ ascli faspex5 admin accounts modify %name:some.user@example.com @json:'{"account
|
|
|
8979
9140
|
> [!TIP]
|
|
8980
9141
|
> This example uses the [percent selector](#percent-selector), but the numerical ID can be used as well.
|
|
8981
9142
|
|
|
8982
|
-
To send a password reset link to a user, use command `
|
|
9143
|
+
To send a password reset link to a user, use command `faspex5 admin accounts reset_password <ACCOUNT_ID>`.
|
|
8983
9144
|
|
|
8984
9145
|
### Faspex 5: Faspex 4-style post-processing
|
|
8985
9146
|
|
|
8986
9147
|
The command `ascli faspex5 postprocessing` emulates Faspex 4 post-processing script execution in Faspex 5.
|
|
8987
9148
|
It implements a web hook for Faspex 5 and calls a script with the same environment variables as set by Faspex 4.
|
|
8988
|
-
Environment variables
|
|
9149
|
+
Environment variables are set to the values provided by the web hook, which are the same as Faspex 4 post-processing.
|
|
8989
9150
|
|
|
8990
9151
|
It allows migrating workflows from Faspex 4 to Faspex 5 while preserving scripts.
|
|
8991
|
-
Nevertheless,
|
|
9152
|
+
Nevertheless, in the long term, a native approach shall be considered, such as using Aspera Orchestrator or other workflow engine, using Faspex 5 native web hooks or File Processing.
|
|
8992
9153
|
|
|
8993
9154
|
It is invoked like this:
|
|
8994
9155
|
|
|
@@ -9020,12 +9181,12 @@ ascli faspex5 postprocessing @json:'{"url":"http://localhost:8080/processing","s
|
|
|
9020
9181
|
```
|
|
9021
9182
|
|
|
9022
9183
|
In Faspex 5, the URL of the webhook endpoint shall be reachable from within Faspex containers.
|
|
9023
|
-
For example, if `ascli`
|
|
9184
|
+
For example, if `ascli` is running on the host, the URL hostname shall not be `localhost`, as this refers to the local address inside the Faspex container.
|
|
9024
9185
|
Instead, one can specify the **IP address of the host** or `host.containers.internal` (Check `podman` manual).
|
|
9025
9186
|
|
|
9026
9187
|
Define the web hook as follows:
|
|
9027
9188
|
|
|
9028
|
-
**Webhook endpoint URI
|
|
9189
|
+
**Webhook endpoint URI**: `http://host.containers.internal:8080/processing/script1.sh`
|
|
9029
9190
|
|
|
9030
9191
|
Then the post-processing script executed will be `/opt/scripts/script1.sh`.
|
|
9031
9192
|
|
|
@@ -9049,32 +9210,32 @@ There are many limitations:
|
|
|
9049
9210
|
- No support for remote sources, only for an actual file transfer by the client.
|
|
9050
9211
|
- The client must use the transfer spec returned by the API (not `faspe:` URL).
|
|
9051
9212
|
- Tags returned in transfer spec must be used in transfer.
|
|
9052
|
-
- Only a single authentication is possible (per gateway) on
|
|
9053
|
-
- No authentication
|
|
9213
|
+
- Only a single authentication is possible (per gateway) on Faspex 5.
|
|
9214
|
+
- No authentication on the Faspex 4 side (ignored).
|
|
9054
9215
|
|
|
9055
9216
|
Behavior:
|
|
9056
|
-
The API client calls the Faspex 4 API on the gateway, then the gateway transforms this into a
|
|
9217
|
+
The API client calls the Faspex 4 API on the gateway, then the gateway transforms this into a Faspex 5 API call, which returns a transfer spec, which is returned to the calling client.
|
|
9057
9218
|
The calling client uses this to start a transfer to HSTS, which is managed by Faspex 5.
|
|
9058
9219
|
|
|
9059
9220
|
For other parameters, see [Web service](#web-service).
|
|
9060
9221
|
|
|
9061
9222
|
### Faspex 5: Get Bearer token to use API
|
|
9062
9223
|
|
|
9063
|
-
If a command is missing, then it is still possible to
|
|
9224
|
+
If a command is missing, then it is still possible to call the API directly on the command line using `curl`:
|
|
9064
9225
|
|
|
9065
9226
|
```shell
|
|
9066
|
-
curl -H "Authorization: $(ascli
|
|
9227
|
+
curl -H "Authorization: $(ascli faspex5 bearer_token)" https://faspex5.example.com/aspera/faspex/api/v5/api_endpoint_here
|
|
9067
9228
|
```
|
|
9068
9229
|
|
|
9069
9230
|
## Plugin: `shares`: IBM Aspera Shares v1
|
|
9070
9231
|
|
|
9071
9232
|
Aspera Shares supports the **Node API** for the file transfer part.
|
|
9072
9233
|
|
|
9073
|
-
Supported commands are listed in
|
|
9234
|
+
Supported commands are listed in the Shares API documentation:
|
|
9074
9235
|
|
|
9075
9236
|
<https://developer.ibm.com/apis/catalog/aspera--aspera-shares-api/Introduction>
|
|
9076
9237
|
|
|
9077
|
-
The payload for creation is the same as for the API
|
|
9238
|
+
The payload for creation is the same as for the API: parameters are provided as a positional `Hash`.
|
|
9078
9239
|
|
|
9079
9240
|
Example: Create a Node: Attributes are like API:
|
|
9080
9241
|
|
|
@@ -9090,18 +9251,25 @@ Example: Create a Node: Attributes are like API:
|
|
|
9090
9251
|
| `timeout` | | `30s` |
|
|
9091
9252
|
| `open_timeout` | | `10s` |
|
|
9092
9253
|
|
|
9093
|
-
Example: Create a share
|
|
9254
|
+
Example: Create a share, grant access to a user and list user permissions on it.
|
|
9094
9255
|
|
|
9095
9256
|
```shell
|
|
9096
9257
|
ascli shares admin share create @json:'{"node_id":1,"name":"test1","directory":"test1","create_directory":true}'
|
|
9097
9258
|
|
|
9098
|
-
|
|
9259
|
+
user_id=$(ascli shares admin user all show %username:john@example.com --fields=id --out.level=data)
|
|
9099
9260
|
|
|
9100
|
-
|
|
9261
|
+
ascli shares admin share user_permissions %name:test1 create @json:'{"user_id":'$user_id',"browse_permission":true,"download_permission":true,"upload_permission":true}'
|
|
9101
9262
|
|
|
9102
|
-
ascli shares admin share user_permissions
|
|
9263
|
+
ascli shares admin share user_permissions %name:test1 list
|
|
9103
9264
|
```
|
|
9104
9265
|
|
|
9266
|
+
Share permissions (`user_permissions`, `group_permissions`) are identified by the user or group identifier: e.g. `show %username:john@example.com`.
|
|
9267
|
+
Permissions not specified on creation take their default value.
|
|
9268
|
+
Available permissions: `browse_permission`, `download_permission`, `upload_permission`, `mkdir_permission`, `delete_permission`, `rename_permission`, `content_availability_permission`, `manage_permission`.
|
|
9269
|
+
|
|
9270
|
+
> [!NOTE]
|
|
9271
|
+
> Permissions of a user or group on all shares (`user all share_permissions`, `group all share_permissions`) are read-only: only `list` and `show` are available.
|
|
9272
|
+
|
|
9105
9273
|
### Tested commands for `shares`
|
|
9106
9274
|
|
|
9107
9275
|
> [!NOTE]
|
|
@@ -9112,6 +9280,7 @@ admin group all list
|
|
|
9112
9280
|
admin node list
|
|
9113
9281
|
admin share list --fields=DEF,-status,status_message
|
|
9114
9282
|
admin share user_permissions %name:my_share list
|
|
9283
|
+
admin share user_permissions %name:my_share modify %username:my_username @json:'{"browse_permission":true}'
|
|
9115
9284
|
admin transfer_settings modify @: min_connect_version=3.6.1
|
|
9116
9285
|
admin transfer_settings show --format=json
|
|
9117
9286
|
admin user all app_authorizations %username:my_username modify @: app_login=true
|
|
@@ -9144,7 +9313,7 @@ info
|
|
|
9144
9313
|
|
|
9145
9314
|
Listing transfers supports the API syntax.
|
|
9146
9315
|
|
|
9147
|
-
In addition, it is possible to place a single `query` parameter in the request to filter the results
|
|
9316
|
+
In addition, it is possible to place a single `query` parameter in the request to filter the results: `filter`, following the syntax:
|
|
9148
9317
|
|
|
9149
9318
|
```text
|
|
9150
9319
|
(field operator value)and(field operator value)...
|
|
@@ -9156,7 +9325,9 @@ In addition, it is possible to place a single `query` parameter in the request t
|
|
|
9156
9325
|
> Add `ascli console` in front of the following commands:
|
|
9157
9326
|
|
|
9158
9327
|
```shell
|
|
9328
|
+
endpoint list
|
|
9159
9329
|
health
|
|
9330
|
+
ssh_key list
|
|
9160
9331
|
transfer current files <id>
|
|
9161
9332
|
transfer current list --query.filter='(transfer_name contain aoc)'
|
|
9162
9333
|
transfer current list --query=@json:'{"filter1":"transfer_name","comp1":"contain","val1":"aoc"}'
|
|
@@ -9167,24 +9338,69 @@ transfer smart sub my_smart_id @: source.paths.0=my_smart_file source_type=user_
|
|
|
9167
9338
|
|
|
9168
9339
|
## Plugin: `orchestrator`: IBM Aspera Orchestrator
|
|
9169
9340
|
|
|
9341
|
+
### Authentication
|
|
9342
|
+
|
|
9343
|
+
The Orchestrator plugin supports different credentials and authentication styles configured via `--auth_style`:
|
|
9344
|
+
|
|
9345
|
+
- **Username / Password** (`--username` and `--password`):
|
|
9346
|
+
- `--auth_style=token` (default): Exchanges credentials for a JWT Bearer token via `/api/login`.
|
|
9347
|
+
- `--auth_style=basic`: Standard HTTP Basic Authentication.
|
|
9348
|
+
- `--auth_style=query`: Passes credentials in URL query parameters (`?login=...&password=...`).
|
|
9349
|
+
- **API Key** (`--apikey`):
|
|
9350
|
+
- `--auth_style=token` (default): Exchanges the API key for a JWT Bearer token via `/api/login`.
|
|
9351
|
+
- `--auth_style=query`: Passes the API key in URL query parameter (`?apikey=...`).
|
|
9352
|
+
|
|
9353
|
+
### Start a workflow
|
|
9354
|
+
|
|
9355
|
+
Command `workflows start` creates a work order:
|
|
9356
|
+
|
|
9357
|
+
```shell
|
|
9358
|
+
ascli orchestrator workflows start <WORKFLOW_ID> [<PARAMETERS>] [<EXECUTION>]
|
|
9359
|
+
```
|
|
9360
|
+
|
|
9361
|
+
- `parameters`: `Hash` of external parameters of the workflow (optional).
|
|
9362
|
+
- `execution`: `Hash` controlling the execution of the work order (optional):
|
|
9363
|
+
|
|
9364
|
+
| Key | Type | Description |
|
|
9365
|
+
|---------------|-----------|-------------|
|
|
9366
|
+
| `synchronous` | `Boolean` | Wait for completion of the work order (default: `false`). |
|
|
9367
|
+
| `step` | `String` | Name of the work step providing the result. |
|
|
9368
|
+
| `variable` | `String` | Name of the output variable of `step` returned as result. |
|
|
9369
|
+
|
|
9370
|
+
`step` and `variable` must be provided together, and imply `synchronous`.
|
|
9371
|
+
|
|
9372
|
+
By default, the call is asynchronous and returns the work order information.
|
|
9373
|
+
|
|
9374
|
+
Example: Start workflow `1234` with parameter `Param`, wait for completion and display the value of output `Complete_status_message` of step `ResultStep`:
|
|
9375
|
+
|
|
9376
|
+
```shell
|
|
9377
|
+
ascli orchestrator workflows start 1234 @json:'{"Param":"world !"}' @json:'{"step":"ResultStep","variable":"Complete_status_message"}'
|
|
9378
|
+
```
|
|
9379
|
+
|
|
9380
|
+
> [!NOTE]
|
|
9381
|
+
> Options `synchronous` and `result` (`--result=<WORK_STEP>:<VARIABLE>`) are deprecated: use `execution` instead.
|
|
9382
|
+
|
|
9170
9383
|
### Tested commands for `orchestrator`
|
|
9171
9384
|
|
|
9172
9385
|
> [!NOTE]
|
|
9173
9386
|
> Add `ascli orchestrator` in front of the following commands:
|
|
9174
9387
|
|
|
9175
9388
|
```shell
|
|
9389
|
+
--auth_style=query workflow list
|
|
9390
|
+
--auth_style=token workflow list
|
|
9176
9391
|
health
|
|
9177
9392
|
info
|
|
9178
9393
|
monitors
|
|
9179
|
-
plugins
|
|
9394
|
+
plugins list
|
|
9180
9395
|
processes
|
|
9181
9396
|
workflow details my_workflow_id
|
|
9182
9397
|
workflow export my_workflow_id
|
|
9183
9398
|
workflow inputs my_workflow_id
|
|
9184
9399
|
workflow list
|
|
9185
9400
|
workflow outputs my_workflow_id
|
|
9401
|
+
workflow start my_sleep_workflow_id --fields=work_order.id
|
|
9186
9402
|
workflow start my_workflow_id @: 'Param=world !'
|
|
9187
|
-
workflow start my_workflow_id @: 'Param=world !'
|
|
9403
|
+
workflow start my_workflow_id @: 'Param=world !' END @: step=ResultStep variable=Complete_status_message
|
|
9188
9404
|
workflow status ALL
|
|
9189
9405
|
workflow status my_workflow_id
|
|
9190
9406
|
workflow workorders my_workflow_id
|
|
@@ -9193,8 +9409,9 @@ workorder cancel <id>
|
|
|
9193
9409
|
workorder output <id>
|
|
9194
9410
|
workorder reset <id>
|
|
9195
9411
|
workorder status <id>
|
|
9196
|
-
|
|
9197
|
-
workstep
|
|
9412
|
+
workorder status <orch_wf_start_sleep> --fields=status_details
|
|
9413
|
+
workstep cancel <id>
|
|
9414
|
+
workstep status <id>
|
|
9198
9415
|
```
|
|
9199
9416
|
|
|
9200
9417
|
## Plugin: `cos`: IBM Cloud Object Storage
|
|
@@ -9310,7 +9527,7 @@ ascli cos node upload 'faux:///sample1G?1g'
|
|
|
9310
9527
|
```
|
|
9311
9528
|
|
|
9312
9529
|
> [!NOTE]
|
|
9313
|
-
> The file `sample1G` is a dummy file of size
|
|
9530
|
+
> The file `sample1G` is a dummy file of size 1 GiB, generated using the `faux` PVCL scheme (see previous section and `man ascp`).
|
|
9314
9531
|
> To upload a real file, replace the `faux:///...` URI with the actual file path.
|
|
9315
9532
|
|
|
9316
9533
|
### Tested commands for `cos`
|
|
@@ -9399,7 +9616,7 @@ Using `ascli` is an alternative to <https://github.com/IBM/aspera-on-cloud-file-
|
|
|
9399
9616
|
|
|
9400
9617
|
### Aspera Server configuration
|
|
9401
9618
|
|
|
9402
|
-
Specify the
|
|
9619
|
+
Specify the previews folder as shown in:
|
|
9403
9620
|
|
|
9404
9621
|
<https://ibmaspera.com/help/admin/organization/installing_the_preview_maker>
|
|
9405
9622
|
|
|
@@ -9414,7 +9631,7 @@ asnodeadmin --reload
|
|
|
9414
9631
|
```
|
|
9415
9632
|
|
|
9416
9633
|
> [!NOTE]
|
|
9417
|
-
> The configuration `preview_dir` is **relative** to the storage root
|
|
9634
|
+
> The configuration `preview_dir` is **relative** to the storage root: no leading or trailing `/` is needed.
|
|
9418
9635
|
> Set the value to `previews`.
|
|
9419
9636
|
|
|
9420
9637
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
|
@@ -9451,7 +9668,7 @@ If you use a value different from `16777216`, then specify it using option `max_
|
|
|
9451
9668
|
- **FFmpeg** : `ffmpeg` `ffprobe`
|
|
9452
9669
|
- **LibreOffice** : `unoconv`
|
|
9453
9670
|
|
|
9454
|
-
|
|
9671
|
+
Installation is shown here for Red Hat/Rocky Linux.
|
|
9455
9672
|
|
|
9456
9673
|
Other OSes should work as well, but are not tested.
|
|
9457
9674
|
|
|
@@ -9520,7 +9737,7 @@ rm -rf /opt/ffmpeg* /usr/bin/{ffmpeg,ffprobe}
|
|
|
9520
9737
|
|
|
9521
9738
|
To skip office document preview generation, use option: `--skip-types=office`
|
|
9522
9739
|
|
|
9523
|
-
The generation of
|
|
9740
|
+
The generation of previews is based on LibreOffice's `unoconv`.
|
|
9524
9741
|
|
|
9525
9742
|
- RHEL 8/Rocky Linux 8+
|
|
9526
9743
|
|
|
@@ -9541,12 +9758,12 @@ chmod a+x /usr/bin/unoconv
|
|
|
9541
9758
|
|
|
9542
9759
|
### Configuration
|
|
9543
9760
|
|
|
9544
|
-
The preview generator should be executed as a non-user.
|
|
9761
|
+
The preview generator should be executed as a non-root user.
|
|
9545
9762
|
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights.
|
|
9546
9763
|
The following procedure uses `xfer` as the running user.
|
|
9547
9764
|
|
|
9548
9765
|
Like any `ascli` commands, options can be passed on command line or using a configuration [Option Preset](#option-preset).
|
|
9549
|
-
The configuration file must be created
|
|
9766
|
+
The configuration file must be created by the same user that runs the generator, so that it is used at runtime.
|
|
9550
9767
|
|
|
9551
9768
|
The `xfer` user has a special protected shell: `aspshell`, so to update the configuration and when changing identity, specify an alternate shell.
|
|
9552
9769
|
For example:
|
|
@@ -9565,16 +9782,39 @@ This example assumes that Office file generation is disabled. Remove `--skip-typ
|
|
|
9565
9782
|
One can check if the access key is well configured using:
|
|
9566
9783
|
|
|
9567
9784
|
```shell
|
|
9568
|
-
ascli -
|
|
9785
|
+
ascli -P<PREVIEW_PRESET_NAME> node browse /
|
|
9569
9786
|
```
|
|
9570
9787
|
|
|
9571
9788
|
This shall list the contents of the storage root of the access key.
|
|
9572
9789
|
|
|
9573
9790
|
### Options for generated files
|
|
9574
9791
|
|
|
9575
|
-
When generating preview files,
|
|
9576
|
-
|
|
9577
|
-
|
|
9792
|
+
When generating preview files, the following options can be modified on command line (or in a preset):
|
|
9793
|
+
|
|
9794
|
+
| Field | Type | Description |
|
|
9795
|
+
|---------------------|---------|----------------------------------------------------------------------------------|
|
|
9796
|
+
| `blend_fps` | `Integer` | MP4 video preview (`blend`): frames per second.<br/>Default: `15`. |
|
|
9797
|
+
| `blend_keyframes` | `Integer` | MP4 video preview (`blend`): number of key frames.<br/>Default: `30`. |
|
|
9798
|
+
| `blend_pauseframes` | `Integer` | MP4 video preview (`blend`): number of pause frames (repetitions of each key frame).<br/>Default: `3`. |
|
|
9799
|
+
| `blend_transframes` | `Integer` | MP4 video preview (`blend`): number of transition frames between key frames.<br/>Default: `5`. |
|
|
9800
|
+
| `clips_count` | `Integer` | MP4 video preview (`clips`): number of clips.<br/>Default: `5`. |
|
|
9801
|
+
| `clips_length` | `Integer` | Video: length (in seconds) of each clip of MP4 video preview (`clips`), and of `animated` PNG thumbnail.<br/>Default: `5`. |
|
|
9802
|
+
| `max_size` | `Integer` | Maximum size (in bytes) of a preview file: a warning is logged if exceeded.<br/>Default: `16777216`. |
|
|
9803
|
+
| `office_conversion` | `String` | PNG thumbnail of office document: conversion tool.<br/>Allowed values: `soffice`, `unoconv`.<br/>Default: `soffice`. |
|
|
9804
|
+
| `reencode_ffmpeg.in` | `Array` | Input options. |
|
|
9805
|
+
| `reencode_ffmpeg.out` | `Array` | Output options. |
|
|
9806
|
+
| `reencode_ffmpeg` | `Hash` | MP4 video preview (`reencode`): `ffmpeg` options replacing the default ones.<br/>Default: `{}`. |
|
|
9807
|
+
| `thumb_img_size` | `Integer` | PNG thumbnail of image, PDF, office document or text: size (in pixels).<br/>Default: `800`. |
|
|
9808
|
+
| `thumb_text_font` | `String` | PNG thumbnail of text: font name, as listed by `magick identify -list font`.<br/>Default: `Courier`. |
|
|
9809
|
+
| `thumb_vid_fraction` | `Number` | PNG thumbnail of video (`fixed`): position of the snapshot, as a fraction of the video duration.<br/>Default: `0.1`. |
|
|
9810
|
+
| `thumb_vid_scale` | `String` | PNG thumbnail of video: frame size, as `ffmpeg` scale filter argument.<br/>Default: `-1:min(ih,100)`. |
|
|
9811
|
+
| `video_codec` | `String` | MP4 video preview (`reencode`): `ffmpeg` video codec, e.g. `libx264`, `h264_videotoolbox`, `h264_nvenc`. Default: first available H.264 encoder. |
|
|
9812
|
+
| `video_conversion` | `String` | MP4 video preview: generation method. `reencode`: re-encode the beginning of the video, `blend`: key frames with transitions, `clips`: concatenation of short clips.<br/>Allowed values: `reencode`, `blend`, `clips`.<br/>Default: `reencode`. |
|
|
9813
|
+
| `video_png_conv` | `String` | PNG thumbnail of video: generation method. `fixed`: single frame, `animated`: animated PNG.<br/>Allowed values: `fixed`, `animated`.<br/>Default: `fixed`. |
|
|
9814
|
+
| `video_scale` | `String` | MP4 video preview: frame size, as `ffmpeg` scale filter argument.<br/>Default: `min(iw,360):-2`. |
|
|
9815
|
+
| `video_start_sec` | `Integer` | Video: start offset (in seconds) of MP4 video preview and of `animated` PNG thumbnail.<br/>Default: `10`. |
|
|
9816
|
+
|
|
9817
|
+
For video preview with method `reencode`, the whole set of `ffmpeg` options can be overridden with option `reencode_ffmpeg`: it is a `Hash` with two keys: `in` and `out`, each is an `Array` with the native options to `ffmpeg`.
|
|
9578
9818
|
|
|
9579
9819
|
### Execution
|
|
9580
9820
|
|
|
@@ -9603,8 +9843,8 @@ Then:
|
|
|
9603
9843
|
ascli preview scan --overwrite=always
|
|
9604
9844
|
```
|
|
9605
9845
|
|
|
9606
|
-
When the preview generator is first executed it
|
|
9607
|
-
On subsequent
|
|
9846
|
+
When the preview generator is first executed, it creates a file `.aspera_access_key` in the previews folder, which contains the access key used.
|
|
9847
|
+
On subsequent runs, it reads this file and checks that previews are generated for the same access key, else it fails.
|
|
9608
9848
|
This is to prevent clash of different access keys using the same root.
|
|
9609
9849
|
|
|
9610
9850
|
### Configuration for Execution in scheduler
|
|
@@ -9613,7 +9853,7 @@ Details are provided in section [Scheduler](#scheduler).
|
|
|
9613
9853
|
|
|
9614
9854
|
Shorter commands can be specified if a configuration preset was created as shown previously.
|
|
9615
9855
|
|
|
9616
|
-
For example the timeout value can be differentiated depending on the
|
|
9856
|
+
For example, the timeout value can be differentiated depending on the command: event versus scan:
|
|
9617
9857
|
|
|
9618
9858
|
```shell
|
|
9619
9859
|
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
|
@@ -9636,7 +9876,7 @@ ascli preview scan %id:<file_id>
|
|
|
9636
9876
|
ascli preview scan /videos --filter='@ruby:->(f){f["name"].end_with?(".mp4")}'
|
|
9637
9877
|
```
|
|
9638
9878
|
|
|
9639
|
-
Once
|
|
9879
|
+
Once candidates are selected, a preview is always generated if it does not already exist; if a preview already exists, generation depends on the value of option `overwrite`:
|
|
9640
9880
|
|
|
9641
9881
|
- `always` : preview is always generated, even if it already exists and is newer than original
|
|
9642
9882
|
- `never` : preview is generated only if it does not exist already
|
|
@@ -9644,8 +9884,8 @@ Once candidate are selected, a preview is always generated if it does not exist
|
|
|
9644
9884
|
|
|
9645
9885
|
Deletion of preview for deleted source files: not implemented yet (TODO).
|
|
9646
9886
|
|
|
9647
|
-
If the `scan` or `events` detection method is used, then
|
|
9648
|
-
It expects a list of
|
|
9887
|
+
If the `scan` or `events` detection method is used, then option `skip_folders` can be used to skip some folders.
|
|
9888
|
+
It expects a list of paths relative to the storage root (docroot), starting with a slash, for example:
|
|
9649
9889
|
|
|
9650
9890
|
```shell
|
|
9651
9891
|
ascli preview scan --skip-folders=@json:'["/not_here"]'
|
|
@@ -9682,8 +9922,8 @@ The mp4 video preview file is only for category `video`.
|
|
|
9682
9922
|
|
|
9683
9923
|
By default, the Mime type used for conversion is the one returned by the Node API, based on file name extension.
|
|
9684
9924
|
|
|
9685
|
-
It is also possible to detect the MIME type using option `
|
|
9686
|
-
To use it, set option `
|
|
9925
|
+
It is also possible to detect the MIME type using option `detect_mime`.
|
|
9926
|
+
To use it, set option `detect_mime` to `yes`: `--detect-mime=yes`.
|
|
9687
9927
|
|
|
9688
9928
|
In this case the `preview` command will first analyze the file content using gem `marcel`, and if no match, will try by extension.
|
|
9689
9929
|
|
|
@@ -9701,7 +9941,7 @@ Nevertheless, `ascli` may or may not have direct file system access to the acces
|
|
|
9701
9941
|
| `root_url` | Description |
|
|
9702
9942
|
|---------------|-------------------------------------------------------------------------------|
|
|
9703
9943
|
| `<empty>` | (Default) If the access key storage type is `local`, then the storage root is used as the main folder.<br/>This assumes that `ascli` runs on the same system as HSTS, or has access through a common "mount".<br/>Else, remote access is assumed. |
|
|
9704
|
-
| `aspera:` | Source files are **downloaded** to a temporary directory, and preview files are **uploaded** to the storage.<br/>Two transfers are
|
|
9944
|
+
| `aspera:` | Source files are **downloaded** to a temporary directory, and preview files are **uploaded** to the storage.<br/>Two transfers are performed using Aspera: one download transfer for source files, one upload transfer for preview files. |
|
|
9705
9945
|
| `file:///<path>` | Files are accessed from the specified path locally. |
|
|
9706
9946
|
|
|
9707
9947
|
### Tested commands for `preview`
|
|
@@ -9722,8 +9962,8 @@ show my_pdf --base=test
|
|
|
9722
9962
|
show my_small_mp4 --base=test --video-png-conv=animated
|
|
9723
9963
|
show my_small_mp4 --base=test --video-png-conv=fixed
|
|
9724
9964
|
test my_dcm --base=test
|
|
9725
|
-
test my_dcm --base=test --
|
|
9726
|
-
test my_jpg_unk --base=test --
|
|
9965
|
+
test my_dcm --base=test --detect-mime=yes
|
|
9966
|
+
test my_jpg_unk --base=test --detect-mime=yes
|
|
9727
9967
|
test my_mpg mp4 --base=test --video-conversion=clips
|
|
9728
9968
|
test my_mpg mp4 --base=test --video-conversion=reencode
|
|
9729
9969
|
test my_mxf mp4 --base=test --video-conversion=blend --query.text=true --query.double=true
|
|
@@ -9741,7 +9981,7 @@ The server registers a single tool, `execute_ascli_command`, which executes any
|
|
|
9741
9981
|
|
|
9742
9982
|
> [!IMPORTANT]
|
|
9743
9983
|
> The `mcp` and `rack` gems are required.
|
|
9744
|
-
> Check section [Installing Optional Gems](#installing-optional-gems)
|
|
9984
|
+
> Check section [Installing Optional Gems](#installing-optional-gems).
|
|
9745
9985
|
> Install them with the following command:
|
|
9746
9986
|
|
|
9747
9987
|
```shell
|
|
@@ -9779,7 +10019,7 @@ The `server` command accepts an optional [Hash](#extended-value-syntax) argument
|
|
|
9779
10019
|
|
|
9780
10020
|
> [!NOTE]
|
|
9781
10021
|
> **`stdio` transport and server description** - The MCP protocol does not carry a `description` field in the `initialize` handshake.
|
|
9782
|
-
> For `stdio` servers, AI clients (Claude Desktop, VS Code, Bob,
|
|
10022
|
+
> For `stdio` servers, AI clients (Claude Desktop, VS Code, Bob, and so on) cannot retrieve the description automatically.
|
|
9783
10023
|
> Add a `"description"` field directly in the client's `mcpServers` configuration to display it in the UI.
|
|
9784
10024
|
|
|
9785
10025
|
#### `http` transport (Streamable HTTP)
|
|
@@ -9843,7 +10083,7 @@ For [**IBM Bob**](https://bob.ibm.com/docs/ide/configuration/mcp/mcp-in-bob), ad
|
|
|
9843
10083
|
}
|
|
9844
10084
|
```
|
|
9845
10085
|
|
|
9846
|
-
For other clients (Claude Desktop, VS Code,
|
|
10086
|
+
For other clients (Claude Desktop, VS Code, and so on):
|
|
9847
10087
|
|
|
9848
10088
|
```json
|
|
9849
10089
|
{
|
|
@@ -9882,13 +10122,13 @@ For [**IBM Bob**](https://bob.ibm.com/docs/ide/configuration/mcp/mcp-in-bob), ad
|
|
|
9882
10122
|
}
|
|
9883
10123
|
```
|
|
9884
10124
|
|
|
9885
|
-
For other clients (Claude Desktop, VS Code,
|
|
10125
|
+
For other clients (Claude Desktop, VS Code, and so on), the configuration is identical.
|
|
9886
10126
|
|
|
9887
10127
|
##### Claude Desktop `stdio`
|
|
9888
10128
|
|
|
9889
10129
|
Find the configuration file as specified in [Claude Desktop Documentation](https://modelcontextprotocol.io/docs/2026-07-28/develop/connect-local-servers).
|
|
9890
10130
|
|
|
9891
|
-
|
|
10131
|
+
Place this section in `mcpServers`:
|
|
9892
10132
|
|
|
9893
10133
|
```json
|
|
9894
10134
|
{
|
|
@@ -9951,23 +10191,27 @@ credential safety.
|
|
|
9951
10191
|
|
|
9952
10192
|
Always start a session with two discovery calls before doing anything else:
|
|
9953
10193
|
|
|
9954
|
-
```
|
|
10194
|
+
```json
|
|
9955
10195
|
["config", "preset", "list"]
|
|
9956
10196
|
["config", "preset", "show", "default"]
|
|
9957
10197
|
```
|
|
9958
10198
|
|
|
9959
|
-
The second call returns the `plugin
|
|
10199
|
+
The second call returns the mapping from `plugin` to `preset_name` so you know which credentials
|
|
9960
10200
|
are active for each plugin.
|
|
9961
10201
|
|
|
9962
10202
|
#### Command discovery
|
|
9963
10203
|
|
|
9964
|
-
Always use `["config", "commands"]` to enumerate
|
|
9965
|
-
|
|
10204
|
+
Always use `["config", "commands", "<plugin>"]` to enumerate the commands of a plugin and their syntax.
|
|
10205
|
+
Add command words to list only the commands under that path, e.g. `["config", "commands", "aoc", "files"]`.
|
|
10206
|
+
A line ending with `<command...>` provides the commands of another plugin, given by `(see: ...)`.
|
|
10207
|
+
Use option `--expand-mounts=yes` to list them in place.
|
|
10208
|
+
Omit `<plugin>` to list the commands of all plugins (much larger result).
|
|
10209
|
+
Never guess command names from training data: names like `shared_folders` vs
|
|
9966
10210
|
`shared_inboxes` are easily confused.
|
|
9967
10211
|
|
|
9968
10212
|
#### Schema introspection for Hash arguments
|
|
9969
10213
|
|
|
9970
|
-
Whenever a command syntax shows a `<
|
|
10214
|
+
Whenever a command syntax shows a Hash argument (e.g. `<account:Hash>`), call `help` in its place **before** the real call:
|
|
9971
10215
|
|
|
9972
10216
|
```json
|
|
9973
10217
|
["<plugin>", "<cmd>", ..., "help"]
|
|
@@ -9979,7 +10223,7 @@ server error messages.
|
|
|
9979
10223
|
#### Async transfers and cross-call status tracking
|
|
9980
10224
|
|
|
9981
10225
|
The `direct` agent keeps transfer state in-memory. A job started in one MCP call **cannot**
|
|
9982
|
-
be monitored in a subsequent call
|
|
10226
|
+
be monitored in a subsequent call: the in-memory agent is gone between calls.
|
|
9983
10227
|
|
|
9984
10228
|
Use the `transferd` agent when you need to check transfer status in a later call:
|
|
9985
10229
|
|
|
@@ -9994,7 +10238,7 @@ The `desktop` agent is also unaffected because it runs in an external process.
|
|
|
9994
10238
|
`aoc files` and `aoc packages` commands require a workspace context. If no default
|
|
9995
10239
|
workspace is configured in the preset, always add `--workspace=NAME`:
|
|
9996
10240
|
|
|
9997
|
-
```
|
|
10241
|
+
```json
|
|
9998
10242
|
["aoc", "files", "ls", "/", "--workspace=MyWorkspace"]
|
|
9999
10243
|
```
|
|
10000
10244
|
|
|
@@ -10005,14 +10249,14 @@ List available workspaces with `["aoc", "user", "workspaces", "list"]`.
|
|
|
10005
10249
|
Before calling any `admin` sub-command, verify that the active preset has admin rights.
|
|
10006
10250
|
`access_denied` typically means the wrong preset is active, not a syntax error. Check with:
|
|
10007
10251
|
|
|
10008
|
-
```
|
|
10252
|
+
```json
|
|
10009
10253
|
["config", "preset", "show", "<preset_name>"]
|
|
10010
10254
|
```
|
|
10011
10255
|
|
|
10012
10256
|
## Operational Utilities
|
|
10013
10257
|
|
|
10014
10258
|
This section covers the specialized modules and utilities used to integrate `ascli` into your broader operational infrastructure.
|
|
10015
|
-
While the core plugins handle data movement, these tools provide the "integration layer" for enterprise environments: Aspera Sync and Hot Folder enable automated, folder-based synchronization; Nagios and SMTP modules provide health monitoring and automated email alerting for transfer status; and `asession` and module
|
|
10259
|
+
While the core plugins handle data movement, these tools provide the "integration layer" for enterprise environments: Aspera Sync and Hot Folder enable automated, folder-based synchronization; Nagios and SMTP modules provide health monitoring and automated email alerting for transfer status; and the `asession` tool and the Ruby module `Aspera` allow integration of transfers into other programs.
|
|
10016
10260
|
Together, these features transform the CLI from a manual tool into a fully integrated component of an automated, monitored data workflow.
|
|
10017
10261
|
|
|
10018
10262
|
### IBM Aspera Sync
|
|
@@ -10027,11 +10271,11 @@ An interface for the `async` utility is provided in the following plugins:
|
|
|
10027
10271
|
The `sync` command, available in above plugins, performs the following actions:
|
|
10028
10272
|
|
|
10029
10273
|
- Start a local Sync session by executing the `async` command with the appropriate parameters.
|
|
10030
|
-
- Get local Sync session information accessing
|
|
10274
|
+
- Get local Sync session information by accessing the Async snap database directly.
|
|
10031
10275
|
- Get local Sync session information using the `asyncadmin` command, if available.
|
|
10032
10276
|
|
|
10033
10277
|
One advantage of using `ascli` over the `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
|
10034
|
-
Moreover, `ascli` supports sync with
|
|
10278
|
+
Moreover, `ascli` supports sync with applications requiring token-based authorization.
|
|
10035
10279
|
|
|
10036
10280
|
Some `sync` parameters are filled by the related plugin using transfer spec parameters (for example, including token).
|
|
10037
10281
|
|
|
@@ -10129,7 +10373,7 @@ ascli config sync spec
|
|
|
10129
10373
|
|
|
10130
10374
|
> [!NOTE]
|
|
10131
10375
|
> `ascli` accepts the following fields within the `sync_info` Hash.
|
|
10132
|
-
> The
|
|
10376
|
+
> The options listed in the **Description** column correspond to the equivalent parameters used by the low-level `async` command.
|
|
10133
10377
|
|
|
10134
10378
|
| Field | Type | Description |
|
|
10135
10379
|
|------------------------------------------|---------------|----------------------------------------------------------------------------------|
|
|
@@ -10273,7 +10517,7 @@ This is the **legacy** syntax.
|
|
|
10273
10517
|
It is based on a JSON representation of `async` command line options.
|
|
10274
10518
|
Technically, it allows definition of multiple sync sessions in a single command, but `ascli` only accepts a single session for consistency with the previous syntax.
|
|
10275
10519
|
|
|
10276
|
-
This is
|
|
10520
|
+
This format is selected if the `sync_info` `Hash` has either key `sessions` or `instance`.
|
|
10277
10521
|
|
|
10278
10522
|
The following parameters are automatically filled from mandatory arguments, and are not allowed:
|
|
10279
10523
|
|
|
@@ -10461,12 +10705,12 @@ Instead, you create a Hot Folder by combining the upload or download commands wi
|
|
|
10461
10705
|
|
|
10462
10706
|
#### Requirements
|
|
10463
10707
|
|
|
10464
|
-
`ascli`
|
|
10465
|
-
A hot folder
|
|
10708
|
+
`ascli` may be used as a simple hot folder engine.
|
|
10709
|
+
A hot folder is defined as a tool that:
|
|
10466
10710
|
|
|
10467
10711
|
- Locally (or remotely) detects new files in a top folder
|
|
10468
|
-
-
|
|
10469
|
-
- Only sends new files,
|
|
10712
|
+
- Sends detected files to a remote (respectively, local) repository
|
|
10713
|
+
- Only sends new files, and does not re-send already sent files
|
|
10470
10714
|
- Optionally: sends only files that are not still **growing**
|
|
10471
10715
|
- Optionally: after transfer of files, deletes or moves to an archive
|
|
10472
10716
|
|
|
@@ -10474,15 +10718,15 @@ In addition: the detection should be made **continuously** or on specific time/d
|
|
|
10474
10718
|
|
|
10475
10719
|
#### Setting up a hot folder
|
|
10476
10720
|
|
|
10477
|
-
The general idea is to rely on
|
|
10721
|
+
The general idea is to rely on:
|
|
10478
10722
|
|
|
10479
10723
|
- Existing `ascp` features for detection and transfer
|
|
10480
|
-
-
|
|
10724
|
+
- `ascli` configuration capabilities and server-side knowledge
|
|
10481
10725
|
- The OS scheduler for reliability and continuous operation
|
|
10482
10726
|
|
|
10483
10727
|
##### `ascp` features
|
|
10484
10728
|
|
|
10485
|
-
|
|
10729
|
+
Useful `ascp` features are available as arguments (see the `ascp` manual):
|
|
10486
10730
|
|
|
10487
10731
|
- Sending only **new** files
|
|
10488
10732
|
- Option `-k 1,2,3` (`resume_policy`)
|
|
@@ -10517,9 +10761,8 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
|
10517
10761
|
|
|
10518
10762
|
##### Scheduling
|
|
10519
10763
|
|
|
10520
|
-
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, for example, every minute
|
|
10521
|
-
See [Scheduler](#scheduler).
|
|
10522
|
-
(on use of option `lock_port`)
|
|
10764
|
+
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, for example, every minute or every 5 minutes.
|
|
10765
|
+
See [Scheduler](#scheduler) (and option `lock_port`).
|
|
10523
10766
|
|
|
10524
10767
|
#### Example: Upload hot folder
|
|
10525
10768
|
|
|
@@ -10550,11 +10793,11 @@ ascli aoc files download . --to-folder=. --lock-port=12345 --progress-bar=no --o
|
|
|
10550
10793
|
> Option `delete_before_transfer` will delete files locally, if they are not present on remote side.
|
|
10551
10794
|
|
|
10552
10795
|
> [!NOTE]
|
|
10553
|
-
> Options `
|
|
10796
|
+
> Options `progress_bar` and `--out.level` limit output for headless operation (for example, a cron job).
|
|
10554
10797
|
|
|
10555
10798
|
### Health check and Nagios
|
|
10556
10799
|
|
|
10557
|
-
Most
|
|
10800
|
+
Most plugins provide a `health` command that checks the health status of the application.
|
|
10558
10801
|
Example:
|
|
10559
10802
|
|
|
10560
10803
|
```shell
|
|
@@ -10569,14 +10812,10 @@ ascli console health
|
|
|
10569
10812
|
╰────────┴─────────────┴────────────╯
|
|
10570
10813
|
```
|
|
10571
10814
|
|
|
10572
|
-
Typically, the health check uses the REST API of the application with the following exception: the `server` plugin
|
|
10573
|
-
|
|
10574
|
-
- Issuing a transfer to the server
|
|
10575
|
-
- Checking web app status with `asctl all:status`
|
|
10576
|
-
- Checking daemons process status
|
|
10815
|
+
Typically, the health check uses the REST API of the application, with the following exception: the `server` plugin checks health by issuing a transfer to the server (`server health transfer`).
|
|
10577
10816
|
|
|
10578
10817
|
`ascli` can be called by Nagios to check the health status of an Aspera server.
|
|
10579
|
-
The output can be made compatible
|
|
10818
|
+
The output can be made compatible with Nagios with option `--format=nagios`:
|
|
10580
10819
|
|
|
10581
10820
|
```shell
|
|
10582
10821
|
ascli server health transfer --to-folder=/Upload --format=nagios --progress-bar=no
|
|
@@ -10588,8 +10827,8 @@ OK - [transfer:ok]
|
|
|
10588
10827
|
|
|
10589
10828
|
### SMTP for email notifications
|
|
10590
10829
|
|
|
10591
|
-
`ascli` can send
|
|
10592
|
-
|
|
10830
|
+
`ascli` can send emails.
|
|
10831
|
+
To do so, set up the SMTP configuration with option `smtp`.
|
|
10593
10832
|
|
|
10594
10833
|
The `smtp` option is a `Hash` ([Extended Value](#extended-value-syntax)) with the following fields:
|
|
10595
10834
|
|
|
@@ -10616,11 +10855,12 @@ ascli config preset set smtp_google password <PASSWORD>
|
|
|
10616
10855
|
or
|
|
10617
10856
|
|
|
10618
10857
|
```shell
|
|
10619
|
-
ascli config preset
|
|
10858
|
+
ascli config preset initialize smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"<PASSWORD>"}'
|
|
10620
10859
|
```
|
|
10621
10860
|
|
|
10622
10861
|
or
|
|
10623
10862
|
|
|
10863
|
+
|
|
10624
10864
|
```shell
|
|
10625
10865
|
ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=<PASSWORD>
|
|
10626
10866
|
```
|
|
@@ -10652,8 +10892,8 @@ Check settings with `smtp_settings` command.
|
|
|
10652
10892
|
Send test email with `email_test`.
|
|
10653
10893
|
|
|
10654
10894
|
```shell
|
|
10655
|
-
ascli config --smtp=@preset:smtp_google
|
|
10656
|
-
ascli config --smtp=@preset:smtp_google
|
|
10895
|
+
ascli config smtp_settings --smtp=@preset:smtp_google
|
|
10896
|
+
ascli config email_test --smtp=@preset:smtp_google --notify-to=sample.dest@example.com
|
|
10657
10897
|
```
|
|
10658
10898
|
|
|
10659
10899
|
#### Notifications for transfer status
|
|
@@ -10697,12 +10937,12 @@ Ideally, IBM will integrate this directly into `ascp`, making this tool redundan
|
|
|
10697
10937
|
|
|
10698
10938
|
Integration with any language is possible, provided that the language can spawn a subprocess, write to its STDIN, read from STDOUT, and generate and parse JSON.
|
|
10699
10939
|
|
|
10700
|
-
`
|
|
10940
|
+
`asession` expects a single argument: a session specification that contains parameters and a [**transfer-spec**](#transfer-specification).
|
|
10701
10941
|
|
|
10702
|
-
If no argument is provided, it assumes a value of: `@json:@stdin:`, that is,
|
|
10942
|
+
If no argument is provided, it assumes a value of: `@json:@stdin:`, that is, JSON on stdin.
|
|
10703
10943
|
|
|
10704
10944
|
> [!NOTE]
|
|
10705
|
-
> If JSON is the format, specify `@json:` to tell `
|
|
10945
|
+
> If JSON is the format, specify `@json:` to tell `asession` to decode the `Hash` using JSON syntax.
|
|
10706
10946
|
|
|
10707
10947
|
During execution, it generates all low level events, one per line, in JSON format on stdout.
|
|
10708
10948
|
|
|
@@ -10747,7 +10987,7 @@ Instead of the traditional text protocol as described in `ascp` manual, the form
|
|
|
10747
10987
|
|
|
10748
10988
|
This is particularly useful for a persistent session (with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true`)
|
|
10749
10989
|
|
|
10750
|
-
```
|
|
10990
|
+
```text
|
|
10751
10991
|
asession
|
|
10752
10992
|
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"<PASSWORD>","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
|
|
10753
10993
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
|
@@ -10806,7 +11046,7 @@ Working examples can be found in repo: <https://github.com/laurent-martin/aspera
|
|
|
10806
11046
|
### Error: "Remote host is not who we expected"
|
|
10807
11047
|
|
|
10808
11048
|
Cause: `ascp` >= 4.x checks fingerprint of the highest server host key, including ECDSA.
|
|
10809
|
-
`ascp` < 4.0 (3.9.6 and earlier)
|
|
11049
|
+
`ascp` < 4.0 (3.9.6 and earlier) supports only RSA (and ignores ECDSA presented by the server).
|
|
10810
11050
|
`aspera.conf` supports a single fingerprint.
|
|
10811
11051
|
|
|
10812
11052
|
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the configuration file):
|
|
@@ -10817,13 +11057,13 @@ Workaround on client side: To ignore the certificate (SSH fingerprint) add optio
|
|
|
10817
11057
|
|
|
10818
11058
|
Workaround on server side: Either remove the fingerprint from `aspera.conf`, or keep only RSA host keys in `sshd_config`.
|
|
10819
11059
|
|
|
10820
|
-
References: ES-1944 in release notes of 4.1 and
|
|
11060
|
+
References: ES-1944 in the release notes of 4.1, and the [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
|
|
10821
11061
|
|
|
10822
11062
|
### Error: "can't find header files for ruby"
|
|
10823
11063
|
|
|
10824
11064
|
Some Ruby gems dependencies require compilation of native parts (C).
|
|
10825
11065
|
This also requires Ruby header files.
|
|
10826
|
-
If Ruby was installed as a Linux
|
|
11066
|
+
If Ruby was installed as a Linux package, then also install the Ruby development package:
|
|
10827
11067
|
`ruby-dev` or `ruby-devel`, depending on distribution.
|
|
10828
11068
|
|
|
10829
11069
|
### Private key type: `ed25519` not supported by default
|
|
@@ -10831,7 +11071,7 @@ If Ruby was installed as a Linux Packages, then also install Ruby development pa
|
|
|
10831
11071
|
There are a few aspects concerning ED25519 keys.
|
|
10832
11072
|
|
|
10833
11073
|
By default, the `aspera-cli` gem does not depend on the `ed25519` gem because it requires compilation of native code which can cause problems and prevent the installation of `ascli`, especially when using JRuby.
|
|
10834
|
-
See [
|
|
11074
|
+
See [net-ssh issue 565](https://github.com/net-ssh/net-ssh/issues/565).
|
|
10835
11075
|
If you want to use `ed25519` keys, then install the required gems:
|
|
10836
11076
|
|
|
10837
11077
|
```shell
|
|
@@ -10869,7 +11109,7 @@ For example:
|
|
|
10869
11109
|
|
|
10870
11110
|
### Error: "SSL_read: unexpected eof while reading"
|
|
10871
11111
|
|
|
10872
|
-
Newer OpenSSL
|
|
11112
|
+
Newer OpenSSL libraries expect a clean SSL close.
|
|
10873
11113
|
To deactivate this error, enable option `IGNORE_UNEXPECTED_EOF` for `ssl_options` in option `http_options`.
|
|
10874
11114
|
|
|
10875
11115
|
```shell
|
|
@@ -10886,7 +11126,7 @@ Workaround: Install an older version of `transferd`:
|
|
|
10886
11126
|
ascli config transferd install 1.1.2
|
|
10887
11127
|
```
|
|
10888
11128
|
|
|
10889
|
-
See [
|
|
11129
|
+
See [Single file executable](#single-file-executable).
|
|
10890
11130
|
|
|
10891
11131
|
### Error: Cannot rename partial file
|
|
10892
11132
|
|
|
@@ -10899,7 +11139,7 @@ This often happens when two transfers start in parallel for the same file:
|
|
|
10899
11139
|
- Session 1 finishes, and renames file1.partial to file1.
|
|
10900
11140
|
- Session 2 finishes, and tries to rename file1.partial to file1, but it fails as it does not exist anymore...
|
|
10901
11141
|
|
|
10902
|
-
By default, `ascli` creates a
|
|
11142
|
+
By default, `ascli` creates a configuration file `~/.aspera/sdk/aspera.conf` like this:
|
|
10903
11143
|
|
|
10904
11144
|
```xml
|
|
10905
11145
|
<?xml version='1.0' encoding='UTF-8'?>
|
|
@@ -10932,11 +11172,11 @@ Another possibility is to add this option: `--transfer=@json:'{"ascp_args":["--p
|
|
|
10932
11172
|
|
|
10933
11173
|
Hootput lives in the terminal, watching over every command with wide, unblinking eyes.
|
|
10934
11174
|
Known for concise output and sharp insight, this owl thrives where others get lost in the dark.
|
|
10935
|
-
It doesn
|
|
11175
|
+
It doesn't chatter; it hoots: clear, precise, and always on time.
|
|
10936
11176
|
|
|
10937
11177
|
Like `ascli`, Hootput is built for action: launching transfers, parsing options, and navigating APIs without hesitation.
|
|
10938
11178
|
Light on feathers but heavy on wisdom, it turns complexity into simple one-liners.
|
|
10939
|
-
When you hear Hootput
|
|
11179
|
+
When you hear Hootput's call, you know your data is already in flight.
|
|
10940
11180
|
|
|
10941
11181
|
### History
|
|
10942
11182
|
|