aspera-cli 4.21.2 → 4.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/BUGS.md +1 -1
- data/CHANGELOG.md +402 -374
- data/CONTRIBUTING.md +6 -10
- data/README.md +1018 -687
- data/lib/aspera/agent/base.rb +9 -5
- data/lib/aspera/agent/connect.rb +30 -28
- data/lib/aspera/agent/desktop.rb +29 -25
- data/lib/aspera/agent/direct.rb +137 -125
- data/lib/aspera/agent/httpgw.rb +22 -26
- data/lib/aspera/agent/node.rb +14 -11
- data/lib/aspera/agent/transferd.rb +6 -2
- data/lib/aspera/api/aoc.rb +15 -18
- data/lib/aspera/api/cos_node.rb +1 -1
- data/lib/aspera/api/httpgw.rb +15 -7
- data/lib/aspera/api/node.rb +6 -4
- data/lib/aspera/ascmd.rb +17 -9
- data/lib/aspera/ascp/installation.rb +21 -19
- data/lib/aspera/ascp/management.rb +1 -1
- data/lib/aspera/assert.rb +14 -5
- data/lib/aspera/cli/error.rb +2 -2
- data/lib/aspera/cli/extended_value.rb +38 -19
- data/lib/aspera/cli/formatter.rb +48 -48
- data/lib/aspera/cli/hints.rb +10 -2
- data/lib/aspera/cli/main.rb +190 -168
- data/lib/aspera/cli/manager.rb +16 -16
- data/lib/aspera/cli/plugin.rb +24 -21
- data/lib/aspera/cli/plugin_factory.rb +1 -1
- data/lib/aspera/cli/plugins/alee.rb +1 -1
- data/lib/aspera/cli/plugins/aoc.rb +173 -126
- data/lib/aspera/cli/plugins/ats.rb +19 -17
- data/lib/aspera/cli/plugins/config.rb +87 -98
- data/lib/aspera/cli/plugins/console.rb +5 -3
- data/lib/aspera/cli/plugins/faspex.rb +39 -35
- data/lib/aspera/cli/plugins/faspex5.rb +104 -80
- data/lib/aspera/cli/plugins/faspio.rb +13 -1
- data/lib/aspera/cli/plugins/httpgw.rb +13 -1
- data/lib/aspera/cli/plugins/node.rb +336 -205
- data/lib/aspera/cli/plugins/orchestrator.rb +34 -40
- data/lib/aspera/cli/plugins/preview.rb +3 -3
- data/lib/aspera/cli/plugins/server.rb +7 -6
- data/lib/aspera/cli/plugins/shares.rb +5 -5
- data/lib/aspera/cli/sync_actions.rb +19 -18
- data/lib/aspera/cli/transfer_agent.rb +11 -15
- data/lib/aspera/cli/transfer_progress.rb +2 -2
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/command_line_builder.rb +116 -95
- data/lib/aspera/coverage.rb +4 -3
- data/lib/aspera/data_repository.rb +1 -0
- data/lib/aspera/environment.rb +7 -6
- data/lib/aspera/faspex_gw.rb +14 -14
- data/lib/aspera/faspex_postproc.rb +7 -6
- data/lib/aspera/hash_ext.rb +2 -2
- data/lib/aspera/json_rpc.rb +1 -1
- data/lib/aspera/keychain/encrypted_hash.rb +47 -34
- data/lib/aspera/keychain/factory.rb +41 -0
- data/lib/aspera/keychain/hashicorp_vault.rb +71 -0
- data/lib/aspera/keychain/macos_security.rb +19 -11
- data/lib/aspera/log.rb +29 -34
- data/lib/aspera/nagios.rb +6 -6
- data/lib/aspera/node_simulator.rb +8 -8
- data/lib/aspera/oauth/base.rb +10 -6
- data/lib/aspera/oauth/factory.rb +6 -6
- data/lib/aspera/oauth/url_json.rb +6 -6
- data/lib/aspera/persistency_action_once.rb +6 -4
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/file_types.rb +40 -33
- data/lib/aspera/preview/generator.rb +1 -1
- data/lib/aspera/preview/options.rb +16 -16
- data/lib/aspera/preview/terminal.rb +3 -3
- data/lib/aspera/preview/utils.rb +11 -13
- data/lib/aspera/products/connect.rb +2 -1
- data/lib/aspera/products/desktop.rb +1 -1
- data/lib/aspera/products/transferd.rb +1 -1
- data/lib/aspera/proxy_auto_config.rb +2 -2
- data/lib/aspera/rest.rb +70 -50
- data/lib/aspera/rest_error_analyzer.rb +1 -0
- data/lib/aspera/rest_errors_aspera.rb +1 -1
- data/lib/aspera/secret_hider.rb +5 -5
- data/lib/aspera/ssh.rb +5 -5
- data/lib/aspera/temp_file_manager.rb +1 -0
- data/lib/aspera/timer_limiter.rb +7 -5
- data/lib/aspera/transfer/async_conf.schema.yaml +716 -0
- data/lib/aspera/transfer/convert.rb +29 -0
- data/lib/aspera/transfer/error_info.rb +66 -66
- data/lib/aspera/transfer/parameters.rb +13 -68
- data/lib/aspera/transfer/spec.rb +5 -6
- data/lib/aspera/transfer/spec.schema.yaml +753 -0
- data/lib/aspera/transfer/spec_doc.rb +62 -0
- data/lib/aspera/transfer/sync.rb +37 -76
- data/lib/aspera/transfer/sync_instance.schema.yaml +20 -0
- data/lib/aspera/transfer/sync_session.schema.yaml +86 -0
- data/lib/aspera/transfer/uri.rb +6 -6
- data/lib/aspera/uri_reader.rb +1 -1
- data/lib/aspera/web_auth.rb +1 -1
- data/lib/aspera/web_server_simple.rb +53 -44
- data.tar.gz.sig +0 -0
- metadata +38 -7
- metadata.gz.sig +0 -0
- data/examples/build_package.sh +0 -28
- data/examples/dascli +0 -30
- data/examples/get_proto_file.rb +0 -8
- data/examples/proxy.pac +0 -60
- data/lib/aspera/transfer/spec.yaml +0 -718
data/README.md
CHANGED
@@ -1,27 +1,27 @@
|
|
1
1
|
# Command Line Interface for IBM Aspera products
|
2
2
|
<!--
|
3
|
-
|
4
|
-
|
5
|
-
markdownlint-disable MD033 MD003 MD053
|
6
|
-
cspell:ignore Serban Antipolis
|
3
|
+
DO NOT EDIT: THIS FILE IS GENERATED, edit docs/README.erb.md, for details, read docs/README.md
|
7
4
|
PANDOC_META_BEGIN
|
8
|
-
subtitle: "ascli 4.
|
5
|
+
subtitle: "ascli 4.23.0"
|
6
|
+
author: "Laurent MARTIN"
|
9
7
|
PANDOC_META_END
|
10
8
|
-->
|
11
9
|
|
10
|
+
<!-- xmarkdownlint-disable MD033 MD003 MD053 -->
|
11
|
+
|
12
12
|
[](https://badge.fury.io/rb/aspera-cli)
|
13
13
|
[](https://github.com/IBM/aspera-cli/actions)
|
14
14
|
[](https://bestpractices.coreinfrastructure.org/projects/5861)
|
15
15
|
|
16
16
|
## Introduction
|
17
17
|
|
18
|
-
Version : 4.
|
18
|
+
Version : 4.23.0
|
19
19
|
|
20
20
|
Laurent/2016-2025
|
21
21
|
|
22
22
|
This gem provides the `ascli` CLI (Command Line Interface) to IBM Aspera software.
|
23
23
|
|
24
|
-
`ascli` is
|
24
|
+
`ascli` is also great tool to learn Aspera APIs.
|
25
25
|
|
26
26
|
Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/aspera-cli)
|
27
27
|
|
@@ -66,25 +66,28 @@ It is designed for:
|
|
66
66
|
- `curl` (for REST calls)
|
67
67
|
- Aspera transfer (`ascp`)
|
68
68
|
|
69
|
-
If the need is to perform operations programmatically in languages such as: C
|
69
|
+
If the need is to perform operations programmatically in languages such as: C/C++, Go, Python, NodeJS, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
|
70
70
|
|
71
71
|
- Product APIs (REST) : e.g. AoC, Faspex, node
|
72
|
-
- Transfer SDK : with gRPC interface and language stubs (C
|
72
|
+
- Transfer SDK : with gRPC interface and language stubs (C/C++, Python, .NET/C#, java, Go, Ruby, Rust, etc...)
|
73
73
|
|
74
74
|
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
75
75
|
Code examples here: <https://github.com/laurent-martin/aspera-api-examples>
|
76
76
|
|
77
77
|
For scripting and ad'hoc command line operations, `ascli` is perfect.
|
78
78
|
|
79
|
+
`ascli` is developer-friendly, designed for quickly testing APIs and learning how to work with Aspera APIs.
|
80
|
+
Refer to section: [Logging, Debugging](#logging-debugging).
|
81
|
+
|
79
82
|
So, which is Aspera's CLI ? `ascli` or `ascp`
|
80
83
|
|
81
84
|
`ascp` is the low level Aspera command line for **transfers**.
|
82
85
|
It is in fact the implementation of the FASP protocol.
|
83
|
-
So, ANY Aspera transfer leads to one ascp process running on client side and another on server side.
|
84
|
-
`ascp` can be used as a command line, but it is very low level, and practically it can be used on command line only if there is no Aspera web
|
86
|
+
So, ANY Aspera transfer leads to one `ascp` process running on client side and another on server side.
|
87
|
+
`ascp` can be used as a command line, but it is very low level, and practically it can be used on command line only if there is no Aspera web app (AoC, Faspex, etc...) and ONLY to do a transfer (send/receive), not for any operation on Aspera apps (e.g. listing remote files).
|
85
88
|
`ascp` does not provide a configuration file to store credentials or options, it does not resume automatically on transfer error.
|
86
89
|
|
87
|
-
In fact, `ascli` can do everything that `ascp` does, and much more,
|
90
|
+
In fact, `ascli` can do everything that `ascp` does, and much more, in an easier way.
|
88
91
|
|
89
92
|
### Notations, Shell, Examples
|
90
93
|
|
@@ -94,17 +97,17 @@ Command line arguments beginning with `my_` in examples, e.g. `my_param_value`,
|
|
94
97
|
|
95
98
|
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
|
96
99
|
|
97
|
-
Some commands will start an Aspera
|
100
|
+
Some commands will start an Aspera transfer (e.g. `upload`).
|
98
101
|
The transfer is not directly implemented in `ascli`, rather `ascli` uses one of the external Aspera Transfer Clients called **[Transfer Agents](#transfer-clients-agents)**.
|
99
102
|
|
100
|
-
> **Note:** A **[Transfer
|
101
|
-
|
102
|
-
For example a remote Aspera Server may be used as a transfer agent (using
|
103
|
+
> **Note:** A **[Transfer Agent](#transfer-clients-agents)** is a client for the remote Transfer Server (HSTS/HSTE).
|
104
|
+
A **[Transfer Agent](#transfer-clients-agents)** can be local or remote...
|
105
|
+
For example a remote Aspera Transfer Server may be used as a transfer agent (using Node API).
|
103
106
|
i.e. using option `--transfer=node`
|
104
107
|
|
105
108
|
## Quick Start
|
106
109
|
|
107
|
-
This section guides you from installation
|
110
|
+
This section guides you from installation to first use and advanced use.
|
108
111
|
|
109
112
|
First, follow section: [Installation](#installation) (Ruby, Gem, FASP) to start using `ascli`.
|
110
113
|
|
@@ -112,7 +115,7 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
112
115
|
|
113
116
|
```console
|
114
117
|
$ ascli --version
|
115
|
-
4.
|
118
|
+
4.23.0
|
116
119
|
```
|
117
120
|
|
118
121
|
### First use
|
@@ -121,7 +124,7 @@ Once installation is completed, you can proceed to the first use with a demo ser
|
|
121
124
|
|
122
125
|
If you want to test with Aspera on Cloud, jump to section: [Wizard](#configuration-using-wizard).
|
123
126
|
|
124
|
-
To test with Aspera demo transfer server,
|
127
|
+
To test with Aspera demo transfer server, set up the environment and then test:
|
125
128
|
|
126
129
|
```bash
|
127
130
|
ascli config initdemo
|
@@ -142,10 +145,10 @@ ascli server browse /
|
|
142
145
|
╰────────────┴──────┴───────────┴───────┴───────────────────────────┴───────────────────────╯
|
143
146
|
```
|
144
147
|
|
145
|
-
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define
|
148
|
+
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define an [Option Preset](#option-preset) for the server's authentication options.
|
146
149
|
The following example will:
|
147
150
|
|
148
|
-
- Create
|
151
|
+
- Create an [Option Preset](#option-preset)
|
149
152
|
- Define it as default for the `server` plugin
|
150
153
|
- List files in a folder
|
151
154
|
- Download a file
|
@@ -198,7 +201,7 @@ complete
|
|
198
201
|
|
199
202
|
Get familiar with configuration, options, commands : [Command Line Interface](#command-line-interface).
|
200
203
|
|
201
|
-
Then, follow the section relative to the product you want to interact with (
|
204
|
+
Then, follow the section relative to the product you want to interact with (Aspera on Cloud, Faspex, ...) : [Application Plugins](#plugins)
|
202
205
|
|
203
206
|
## Installation
|
204
207
|
|
@@ -224,27 +227,36 @@ A package with pre-installed Ruby, gem and `ascp` may also be provided.
|
|
224
227
|
### `ascli` executable
|
225
228
|
|
226
229
|
It is planned to provide `ascli` as a single platform-dependent executable.
|
227
|
-
[
|
230
|
+
[Beta releases can be found here](https://ibm.biz/aspera-cli-exe).
|
228
231
|
|
229
|
-
**Note:** This is
|
232
|
+
**Note:** This is a Beta feature.
|
233
|
+
On Linux, the executable requires a minimum GLIBC version.
|
234
|
+
Installation of `ascp` is still required separately.
|
235
|
+
Refer to [Install `ascp`](#installation-of-ascp-through-transferd).
|
230
236
|
|
231
|
-
On Linux, check your GLIBC version with `ldd`:
|
237
|
+
On Linux, check the minimum required GLIBC on this site: [repology.org](https://repology.org/project/glibc/versions), or check your GLIBC version with `ldd`:
|
238
|
+
|
239
|
+
```bash
|
240
|
+
ldd --version | head -n1
|
241
|
+
```
|
232
242
|
|
233
243
|
```console
|
234
|
-
$ ldd --version | head -n1
|
235
244
|
ldd (GNU libc) 2.34
|
236
245
|
```
|
237
246
|
|
238
|
-
Check an executable's
|
247
|
+
Check an executable's (e.g. `/bin/bash`, `ascli`, `ascp`) minimum required GLIBC version:
|
248
|
+
|
249
|
+
```bash
|
250
|
+
objdump -p /bin/bash | sed -n 's/^.*GLIBC_//p' | sort -V | tail -n1
|
251
|
+
```
|
239
252
|
|
240
253
|
```console
|
241
|
-
$ objdump -p /bin/bash | sed -n 's/^.*GLIBC_//p' | sort -V | tail -n1
|
242
254
|
2.34
|
243
255
|
```
|
244
256
|
|
245
|
-
> **Note:**
|
257
|
+
> **Note:** If `objdump` is not available, then use `strings` or `grep -z 'GLIBC_'|tr \\0 \\n`
|
246
258
|
|
247
|
-
The `
|
259
|
+
The required GLIBC version for `ascp` can be found in the [Release Notes of HSTS](https://www.ibm.com/docs/en/ahts) or [in this page](https://eudemo.asperademo.com/download/sdk.html).
|
248
260
|
|
249
261
|
### Ruby
|
250
262
|
|
@@ -254,7 +266,7 @@ Required Ruby version: >= 3.1.
|
|
254
266
|
|
255
267
|
> **Deprecation notice**: the minimum Ruby version will be 3.2 in a future version.
|
256
268
|
|
257
|
-
**Ruby can be installed using any method** : rpm
|
269
|
+
**Ruby can be installed using any method** : `rpm`, `yum`, `dnf`, `rvm`, `brew`, Windows installer, ...
|
258
270
|
|
259
271
|
**In priority**, refer to the official Ruby documentation:
|
260
272
|
|
@@ -263,7 +275,7 @@ Required Ruby version: >= 3.1.
|
|
263
275
|
|
264
276
|
For convenience, you may refer to the following sections for a proposed method for specific operating systems.
|
265
277
|
|
266
|
-
Latest version of `ascli` requires a
|
278
|
+
Latest version of `ascli` requires a Ruby version [at least under maintenance support](https://www.ruby-lang.org/en/downloads/branches/).
|
267
279
|
If an older Ruby version is needed, then use an older version of `ascli` that supports it.
|
268
280
|
|
269
281
|
#### Windows: Installer
|
@@ -272,9 +284,9 @@ Manual installation:
|
|
272
284
|
|
273
285
|
- Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) → **Downloads**.
|
274
286
|
- Download the latest Ruby installer **"with devkit"**. (`Msys2` is needed to install some native extensions, such as `grpc`)
|
275
|
-
- Execute the installer which installs by default in: `C:\RubyVV-x64` (VV is the version number)
|
276
|
-
- At the end of the installation procedure, the `Msys2` installer is automatically executed, select option 3 (`Msys2` and mingw)
|
277
|
-
- Then install the aspera-cli gem and Aspera SDK (see next sections)
|
287
|
+
- Execute the installer which installs by default in: `C:\RubyVV-x64` (`VV` is the version number)
|
288
|
+
- At the end of the installation procedure, the `Msys2` installer is automatically executed, select option 3 (`Msys2` and `mingw`)
|
289
|
+
- Then install the `aspera-cli` gem and Aspera SDK (see next sections)
|
278
290
|
|
279
291
|
Automated installation, with internet access:
|
280
292
|
|
@@ -289,18 +301,41 @@ rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\asper
|
|
289
301
|
#### macOS: `brew`
|
290
302
|
|
291
303
|
**macOS** comes with Ruby 2.6.
|
292
|
-
It is an old unsupported version and [Apple has deprecated it](https://developer.apple.com/documentation/macos-release-notes/macos-catalina-10_15-release-notes)
|
304
|
+
It is an old unsupported version and [Apple has deprecated it](https://developer.apple.com/documentation/macos-release-notes/macos-catalina-10_15-release-notes).
|
305
|
+
It will be removed from macOS in the future.
|
293
306
|
Do not use it.
|
294
307
|
|
295
|
-
The recommended way is to
|
308
|
+
The recommended way is to use [Homebrew](https://brew.sh/).
|
296
309
|
|
297
310
|
```bash
|
298
311
|
brew install ruby
|
299
312
|
```
|
300
313
|
|
301
|
-
This installs a recent
|
314
|
+
This installs a recent Ruby suitable for `ascli`.
|
302
315
|
|
303
|
-
|
316
|
+
To add PATH to Ruby on Apple Silicon, add this in your shell configuration file (e.g. `~/.bash_profile` or `~/.zshrc`):
|
317
|
+
|
318
|
+
```bash
|
319
|
+
use_ruby(){
|
320
|
+
version=$1
|
321
|
+
local prefix=$(brew --prefix ruby${version:+@}$version)
|
322
|
+
if ! test -d $prefix;then
|
323
|
+
echo "No such ruby version: $version"
|
324
|
+
brew list|grep ruby
|
325
|
+
return 1
|
326
|
+
fi
|
327
|
+
export PATH="$prefix/bin:$PATH"
|
328
|
+
export PATH="$(gem env gemdir)/bin:$PATH"
|
329
|
+
export LDFLAGS="-L$prefix/lib"
|
330
|
+
export CPPFLAGS="-I$prefix/include"
|
331
|
+
export PKG_CONFIG_PATH="$prefix/lib/pkgconfig"
|
332
|
+
echo "Using: $prefix"
|
333
|
+
ruby -v
|
334
|
+
}
|
335
|
+
use_ruby
|
336
|
+
```
|
337
|
+
|
338
|
+
If using [RVM](https://rvm.io/), one way to force use of OpenSSL 3.0 is:
|
304
339
|
|
305
340
|
```bash
|
306
341
|
RUBY_CONFIGURE_OPTS="--with-openssl-dir=$(brew --prefix openssl@3.0)" rvm install 3.4.0
|
@@ -365,7 +400,7 @@ Install `rvm`.
|
|
365
400
|
Follow [https://rvm.io/](https://rvm.io/).
|
366
401
|
|
367
402
|
Execute the shell/curl command.
|
368
|
-
As regular user, it installs in the user's home: `~/.rvm
|
403
|
+
As regular user, it installs in the user's home: `~/.rvm`.
|
369
404
|
|
370
405
|
```bash
|
371
406
|
\curl -sSL https://get.rvm.io | bash -s stable
|
@@ -420,7 +455,7 @@ rvm version
|
|
420
455
|
|
421
456
|
#### Linux as non-root
|
422
457
|
|
423
|
-
If you don't have root access, you can install Ruby in your home directory using `rbenv` see [rbenv-installer](https://github.com/rbenv/rbenv-installer#rbenv-installer):
|
458
|
+
If you don't have root access, you can install Ruby in your home directory using `rbenv` see [`rbenv-installer`](https://github.com/rbenv/rbenv-installer#rbenv-installer):
|
424
459
|
|
425
460
|
```bash
|
426
461
|
curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-installer | bash
|
@@ -473,7 +508,7 @@ make install
|
|
473
508
|
All what is needed is a JVM (Java Virtual Machine) on your system (`java`).
|
474
509
|
The JRuby package comes pre-complied and does not require compilation of native extensions.
|
475
510
|
Use a version of JRuby compatible with Ruby version supported by `ascli`.
|
476
|
-
Refer to [the
|
511
|
+
Refer to [the Wikipedia page](https://en.wikipedia.org/wiki/JRuby) to match JRuby and Ruby versions.
|
477
512
|
Choose the latest version from:
|
478
513
|
|
479
514
|
<https://www.jruby.org/download>
|
@@ -502,18 +537,18 @@ Some additional gems are required for some specific features, see [Gemfile](Gemf
|
|
502
537
|
|
503
538
|
| name | version | comment |
|
504
539
|
| ---- | ------- | ------- |
|
505
|
-
| grpc | ~> 1.
|
540
|
+
| grpc | ~> 1.71 | for Aspera Transfer Daemon |
|
506
541
|
| mimemagic | ~> 0.4 | for preview |
|
507
|
-
| rmagick | ~>
|
508
|
-
| symmetric-encryption | ~> 4.6 | for file
|
542
|
+
| rmagick | ~> 6.1 | for terminal view |
|
543
|
+
| symmetric-encryption | ~> 4.6 | for encrypted hash file secrets |
|
509
544
|
| bigdecimal | ~> 3.1.9 | if RUBY_VERSION >= '3.4' for symmetric-encryption ? |
|
510
545
|
|
511
546
|
Install like this:
|
512
547
|
|
513
548
|
```bash
|
514
|
-
gem install grpc -v '~> 1.
|
549
|
+
gem install grpc -v '~> 1.71'
|
515
550
|
gem install mimemagic -v '~> 0.4'
|
516
|
-
gem install rmagick -v '~>
|
551
|
+
gem install rmagick -v '~> 6.1'
|
517
552
|
gem install symmetric-encryption -v '~> 4.6'
|
518
553
|
gem install bigdecimal -v '~> 3.1.9'
|
519
554
|
```
|
@@ -525,7 +560,7 @@ gem install bigdecimal -v '~> 3.1.9'
|
|
525
560
|
Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
|
526
561
|
|
527
562
|
```bash
|
528
|
-
gem install aspera-cli
|
563
|
+
gem install aspera-cli
|
529
564
|
```
|
530
565
|
|
531
566
|
To upgrade to the latest version:
|
@@ -535,7 +570,7 @@ gem update aspera-cli
|
|
535
570
|
```
|
536
571
|
|
537
572
|
During its execution, `ascli` checks every week if a new version is available and notifies the user in a WARN log.
|
538
|
-
To
|
573
|
+
To deactivate this feature, globally set the option `version_check_days` to `0`, or specify a different period in days.
|
539
574
|
|
540
575
|
To check if a new version is available (independently of `version_check_days`):
|
541
576
|
|
@@ -545,7 +580,7 @@ ascli config check_update
|
|
545
580
|
|
546
581
|
#### Gem installation with signature verification
|
547
582
|
|
548
|
-
The gem is signed with a private key, and the public certificate is available in the
|
583
|
+
The gem is signed with a private key, and the public certificate is available in the GitHub repository (`certs/aspera-cli-public-cert.pem`).
|
549
584
|
When installing the gem, the signature can be optionally verified.
|
550
585
|
|
551
586
|
For [secure installation](https://guides.rubygems.org/security/#using-gems), one can install the gem with the public key:
|
@@ -568,16 +603,13 @@ Most file transfers will be executed using the **FASP** protocol, using `ascp`.
|
|
568
603
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
569
604
|
|
570
605
|
- `ascp`
|
571
|
-
- `aspera-license` (in same folder, or
|
606
|
+
- `aspera-license` (in same folder, or `../etc`)
|
572
607
|
|
573
608
|
This can be installed either be installing an Aspera transfer software or using an `ascli` command.
|
574
609
|
|
575
610
|
#### Installation of `ascp` through `transferd`
|
576
611
|
|
577
|
-
The easiest option to install `ascp` is through the use of the IBM Aspera Transfer Daemon.
|
578
|
-
|
579
|
-
Supported platforms are listed in the [Release Notes](https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Release+notes) and archives can be downloaded from [Downloads](https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/downloads/downloads.json).
|
580
|
-
|
612
|
+
The easiest option to install `ascp` is through the use of the IBM Aspera Transfer Daemon (`transferd`).
|
581
613
|
Install using `ascli` for the current platform with:
|
582
614
|
|
583
615
|
```bash
|
@@ -590,14 +622,18 @@ or
|
|
590
622
|
ascli config transferd install
|
591
623
|
```
|
592
624
|
|
593
|
-
The installation of the transfer
|
625
|
+
The installation of the transfer binaries follows those steps:
|
594
626
|
|
595
|
-
- Check the value of option `sdk_url`: if the value is the default value `DEF`, then the procedure follows, else it
|
596
|
-
-
|
597
|
-
-
|
627
|
+
- Check the value of option `sdk_url`: if the value is the default value `DEF`, then the procedure follows, else it specifies directly the URL where to take the archive from.
|
628
|
+
- Download the YAML file from the URL specified by option `locations_url` whose default value is <https://ibm.biz/sdk_location>. This file provides the list of supported OS, CPU and versions of the Aspera Transfer Daemon.
|
629
|
+
- Select the archive for the current system architecture (CPU and OS) is selected and downloaded. An alternate version can be specified as position argument, e.g. `1.1.3`.
|
630
|
+
- By default, the archive is extracted to `$HOME/.aspera/sdk`, this can be changed by setting the `sdk_folder` option.
|
598
631
|
|
599
|
-
|
600
|
-
|
632
|
+
| Option | Default | Description |
|
633
|
+
|-----------------|---------|-------------|
|
634
|
+
| `sdk_url` | `DEF` | URL to download the Aspera Transfer SDK archive. `DEF` means: select from available archives. |
|
635
|
+
| `locations_url` | `https://ibm.biz/sdk_location` | URL to get download URLs of Aspera Transfer Daemon from IBM official repository. |
|
636
|
+
| `sdk_folder` | `$HOME/.aspera/sdk` | Folder where the SDK archive is extracted. |
|
601
637
|
|
602
638
|
Available Transfer Daemon versions available from `locations_url` can be listed with: `ascli config transferd list`
|
603
639
|
|
@@ -619,7 +655,7 @@ To download it, pipe to `config download`:
|
|
619
655
|
ascli config transferd list --select=@json:'{"platform":"osx-arm64","version":"1.1.3"}' --fields=url | ascli config download @stdin:
|
620
656
|
```
|
621
657
|
|
622
|
-
If installation from a local file preferred instead of fetching from internet: one can specify the location of the SDK file with option `sdk_url`:
|
658
|
+
If installation from a local file preferred (airgap installation) instead of fetching from internet: one can specify the location of the SDK file with option `sdk_url`:
|
623
659
|
|
624
660
|
```bash
|
625
661
|
ascli config ascp install --sdk-url=file:///macos-arm64-1.1.3-c6c7a2a.zip
|
@@ -627,6 +663,8 @@ ascli config ascp install --sdk-url=file:///macos-arm64-1.1.3-c6c7a2a.zip
|
|
627
663
|
|
628
664
|
The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
|
629
665
|
|
666
|
+
Supported platforms are listed in the [Release Notes](https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Release+notes) and archives can be downloaded from [Downloads](https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/downloads/downloads.json).
|
667
|
+
|
630
668
|
#### Installation of `ascp` through other component
|
631
669
|
|
632
670
|
If the embedded method is not used, the following packages are also suitable:
|
@@ -634,7 +672,7 @@ If the embedded method is not used, the following packages are also suitable:
|
|
634
672
|
- IBM Aspera Connect Client (Free)
|
635
673
|
- IBM Aspera Desktop Client (Free)
|
636
674
|
- IBM Aspera High Speed Transfer Server (Licensed)
|
637
|
-
- IBM Aspera High Speed Transfer
|
675
|
+
- IBM Aspera High Speed Transfer Endpoint (Licensed)
|
638
676
|
|
639
677
|
For instance, Aspera Connect Client can be installed by visiting the page:
|
640
678
|
[https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
|
@@ -652,7 +690,7 @@ Refer to section: [Transfer Agents](#transfer-clients-agents)
|
|
652
690
|
|
653
691
|
#### Gem files and dependencies
|
654
692
|
|
655
|
-
The sample script: [
|
693
|
+
The sample script: [windows/build_package.sh](windows/build_package.sh) can be used to download all necessary gems and dependencies in a `tar.gz`.
|
656
694
|
|
657
695
|
```console
|
658
696
|
$ ./build_package.sh aspera-cli 4.18.0
|
@@ -690,7 +728,7 @@ source ~/.rvm/scripts/rvm
|
|
690
728
|
ascli config ascp install --sdk-url=file:///[SDK archive file path]
|
691
729
|
```
|
692
730
|
|
693
|
-
- Add those lines to shell
|
731
|
+
- Add those lines to shell environment (`.profile`)
|
694
732
|
|
695
733
|
```bash
|
696
734
|
source ~/.rvm/scripts/rvm
|
@@ -711,7 +749,7 @@ It is essentially the same procedure as installation for Windows with internet,
|
|
711
749
|
|
712
750
|
- Create an archive with necessary gems like in previous section
|
713
751
|
|
714
|
-
- Download the SDK following [Install `ascp`](#installation-of-ascp-through-transferd)
|
752
|
+
- Download the SDK following: [Install `ascp`](#installation-of-ascp-through-transferd)
|
715
753
|
|
716
754
|
- Create a Zip with all those files and transfer to the target system.
|
717
755
|
|
@@ -736,7 +774,7 @@ gem install --force --local *.gem
|
|
736
774
|
ascli config ascp install --sdk-url=file:///sdk.zip
|
737
775
|
```
|
738
776
|
|
739
|
-
> **Note:** An example of installation script is provided: [
|
777
|
+
> **Note:** An example of installation script is provided: [windows/install.bat](windows/install.bat)
|
740
778
|
|
741
779
|
### Container
|
742
780
|
|
@@ -751,13 +789,14 @@ podman --version
|
|
751
789
|
#### Container: Quick start
|
752
790
|
|
753
791
|
**Wanna start quickly ?** With an interactive shell ?
|
792
|
+
|
754
793
|
Execute this:
|
755
794
|
|
756
795
|
```bash
|
757
796
|
podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/ascli:latest
|
758
797
|
```
|
759
798
|
|
760
|
-
> **Note:** This command changes the
|
799
|
+
> **Note:** This command changes the entry point to an interactive shell instead of direct execution of `ascli`.
|
761
800
|
|
762
801
|
Then, execute individual `ascli` commands such as:
|
763
802
|
|
@@ -776,7 +815,7 @@ That is simple, but there are limitations:
|
|
776
815
|
|
777
816
|
#### Container: Details
|
778
817
|
|
779
|
-
The container image is built from this [Dockerfile](Dockerfile.tmpl.erb).
|
818
|
+
The container image is built from this [Dockerfile](container/Dockerfile.tmpl.erb).
|
780
819
|
The entry point is `ascli` and the default command is `help`.
|
781
820
|
|
782
821
|
The container can be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
@@ -798,7 +837,7 @@ ascli -v
|
|
798
837
|
```
|
799
838
|
|
800
839
|
```text
|
801
|
-
4.
|
840
|
+
4.23.0
|
802
841
|
```
|
803
842
|
|
804
843
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
@@ -809,7 +848,7 @@ Add options:
|
|
809
848
|
--user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli
|
810
849
|
```
|
811
850
|
|
812
|
-
> **Note:** If you are using a `podman machine`, e.g. on macOS
|
851
|
+
> **Note:** If you are using a `podman machine`, e.g. on macOS, make sure that the folder is also shared between the VM and the host, so that sharing is: container → VM → Host: `podman machine init ... --volume="/Users:/Users"`
|
813
852
|
|
814
853
|
As shown in the quick start, if you prefer to keep a running container with a shell and `ascli` available,
|
815
854
|
you can change the entry point, add option:
|
@@ -843,9 +882,9 @@ asclish
|
|
843
882
|
|
844
883
|
#### Container: Sample start script
|
845
884
|
|
846
|
-
A convenience sample script is also provided: download the script [`dascli`](../
|
885
|
+
A convenience sample script is also provided: download the script [`dascli`](../container/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/container/dascli) :
|
847
886
|
|
848
|
-
> **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli config gem path)/../
|
887
|
+
> **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli config gem path)/../container/dascli ascli`
|
849
888
|
|
850
889
|
Some environment variables can be set for this script to adapt its behavior:
|
851
890
|
|
@@ -853,7 +892,7 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
853
892
|
|----------------|------------------------------------|--------------------------|--------------------------|
|
854
893
|
| `ASCLI_HOME` | Configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
855
894
|
| `docker_args` | Additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
856
|
-
| `image` | Container image name | `docker.io/martinlaurent/ascli`
|
895
|
+
| `image` | Container image name | `docker.io/martinlaurent/ascli` | n/a |
|
857
896
|
| `version` | Container image version | Latest | `4.8.0.pre` |
|
858
897
|
|
859
898
|
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
@@ -865,7 +904,7 @@ To add local storage as a volume, you can use the env var `docker_args`:
|
|
865
904
|
Example of use:
|
866
905
|
|
867
906
|
```bash
|
868
|
-
curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/
|
907
|
+
curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/container/dascli
|
869
908
|
chmod a+x ascli
|
870
909
|
export xferdir=$HOME/xferdir
|
871
910
|
mkdir -p $xferdir
|
@@ -900,7 +939,7 @@ podman load -i ascli_image_latest.tar.gz
|
|
900
939
|
#### Container: `aspera.conf`
|
901
940
|
|
902
941
|
`ascp`'s configuration file `aspera.conf` is located in the container at: `/ibm_aspera/aspera.conf` (see Dockerfile).
|
903
|
-
As the container is immutable, it is not recommended
|
942
|
+
As the container is immutable, it is not recommended modifying this file.
|
904
943
|
If one wants to change the content, it is possible to tell `ascp` to use another file using `ascp` option `-f`, e.g. by locating it on the host folder `$HOME/.aspera/ascli` mapped to the container folder `/home/cliuser/.aspera/ascli`:
|
905
944
|
|
906
945
|
```bash
|
@@ -945,7 +984,7 @@ singularity shell ascli.sif
|
|
945
984
|
|
946
985
|
`ascli` uses the Ruby `openssl` gem which uses by default the system's `openssl` library and its CA certificate bundle.
|
947
986
|
|
948
|
-
To display the version of **
|
987
|
+
To display the version of **OpenSSL** used in `ascli`:
|
949
988
|
|
950
989
|
```bash
|
951
990
|
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
@@ -977,7 +1016,7 @@ To display trusted certificate store locations:
|
|
977
1016
|
ascli --show-config --fields=cert_stores
|
978
1017
|
```
|
979
1018
|
|
980
|
-
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc
|
1019
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc...).
|
981
1020
|
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
982
1021
|
|
983
1022
|
One can display those default values:
|
@@ -998,13 +1037,13 @@ update-ca-trust extract
|
|
998
1037
|
```
|
999
1038
|
|
1000
1039
|
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders.
|
1001
|
-
By default it uses Ruby's default certificate store.
|
1040
|
+
By default, it uses Ruby's default certificate store.
|
1002
1041
|
|
1003
1042
|
If you use this option, then default locations are not used.
|
1004
1043
|
Default locations can be added using special value `DEF`.
|
1005
1044
|
The value can be either an `Array` or `String` (path).
|
1006
1045
|
Successive options add paths incrementally.
|
1007
|
-
All files of a
|
1046
|
+
All files of a folder are added.
|
1008
1047
|
|
1009
1048
|
JRuby uses its own implementation and CA bundles.
|
1010
1049
|
|
@@ -1016,8 +1055,8 @@ For example, on Linux to force the use the system's certificate store:
|
|
1016
1055
|
|
1017
1056
|
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
|
1018
1057
|
|
1019
|
-
By default,`ascp` uses
|
1020
|
-
Original `ascp`'s
|
1058
|
+
By default,`ascp` uses a hard coded root location `OPENSSLDIR`.
|
1059
|
+
Original `ascp`'s hard coded locations can be found using:
|
1021
1060
|
|
1022
1061
|
```bash
|
1023
1062
|
ascli config ascp info --fields=openssldir
|
@@ -1025,7 +1064,7 @@ ascli config ascp info --fields=openssldir
|
|
1025
1064
|
|
1026
1065
|
E.g. on macOS: `/Library/Aspera/ssl`.
|
1027
1066
|
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
1028
|
-
`ascli` overrides the default
|
1067
|
+
`ascli` overrides the default hard coded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
1029
1068
|
|
1030
1069
|
To update trusted root certificates for `ascli`:
|
1031
1070
|
Display the trusted certificate store locations used by `ascli`.
|
@@ -1075,17 +1114,17 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
|
|
1075
1114
|
|
1076
1115
|
`ascli` provides the following features:
|
1077
1116
|
|
1078
|
-
-
|
1117
|
+
- Commands to Aspera server products (on-premise and SaaS)
|
1079
1118
|
- Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
|
1080
|
-
-
|
1119
|
+
- Commands, Options, and Option values shortcuts
|
1081
1120
|
- FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
|
1082
1121
|
- Transfer parameters can be altered by modification of [**transfer-spec**](#transfer-specification), this includes requiring multi-session
|
1083
1122
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
1084
|
-
-
|
1085
|
-
-
|
1123
|
+
- faspstream creation (using Node API)
|
1124
|
+
- **Watchfolder** creation (using Node API)
|
1086
1125
|
- Additional command plugins can be written by the user
|
1087
|
-
-
|
1088
|
-
-
|
1126
|
+
- Download of Faspex and Aspera on Cloud "external" links
|
1127
|
+
- **Legacy** SSH based FASP transfers and remote commands (`ascmd`)
|
1089
1128
|
|
1090
1129
|
Basic usage is displayed by executing:
|
1091
1130
|
|
@@ -1108,7 +1147,7 @@ Using `ascli` with plugin `server` for command line gives advantages over `ascp`
|
|
1108
1147
|
- Choice of transfer agents
|
1109
1148
|
- Integrated support of multi-session
|
1110
1149
|
|
1111
|
-
|
1150
|
+
All `ascp` options are supported either through transfer spec parameters (listed with `conf ascp spec`) and with the possibility to provide `ascp` arguments directly when the `direct` agent is used (`ascp_args` in `transfer_info`).
|
1112
1151
|
|
1113
1152
|
### Command line parsing, Special Characters
|
1114
1153
|
|
@@ -1118,10 +1157,10 @@ The way arguments are parsed and provided to `ascli` depend on the Operating Sys
|
|
1118
1157
|
|
1119
1158
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
1120
1159
|
|
1121
|
-
Linux command line parsing is well
|
1160
|
+
Linux command line parsing is well-defined:
|
1122
1161
|
It is fully documented in the shell's documentation.
|
1123
1162
|
|
1124
|
-
On Unix-like environments, this is typically a POSIX-like shell (bash
|
1163
|
+
On Unix-like environments, this is typically a POSIX-like shell (`bash`, `zsh`, `ksh`, `sh`).
|
1125
1164
|
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
1126
1165
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
1127
1166
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
@@ -1135,7 +1174,7 @@ Command line parsing first depends on the shell used.
|
|
1135
1174
|
MS Windows command line parsing is not like Unix-like systems simply because Windows does not provide a list of arguments to the executable (Ruby): it provides the whole command line as a single string, but the shell may interpret some special characters.
|
1136
1175
|
|
1137
1176
|
So command line parsing is not handled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the executable.
|
1138
|
-
Typically, Windows executables use the [
|
1177
|
+
Typically, Windows executables use the [Microsoft library for this parsing](https://learn.microsoft.com/en-us/cpp/cpp/main-function-command-line-args).
|
1139
1178
|
|
1140
1179
|
As far as `ascli` is concerned: the executable is Ruby.
|
1141
1180
|
It has its own parsing algorithm, close to a Linux shell parsing.
|
@@ -1152,7 +1191,7 @@ Hello World
|
|
1152
1191
|
2
|
1153
1192
|
```
|
1154
1193
|
|
1155
|
-
> **Note:** Use `pp` instead of `puts` to display as
|
1194
|
+
> **Note:** Use `pp` instead of `puts` to display as Ruby `Array`.
|
1156
1195
|
|
1157
1196
|
Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' `CreateProcess` with just the whole command line as a single string.
|
1158
1197
|
(Unlike Unix-like systems where the command line is split into arguments by the shell.)
|
@@ -1225,11 +1264,11 @@ PS C:\> ascli config echo --% @json:'{"k":"v","x":"y"}'
|
|
1225
1264
|
PS C:\> ascli config echo @json:'{"""k""":"""v""","""x""":"""y"""}'
|
1226
1265
|
```
|
1227
1266
|
|
1228
|
-
> **Note:** The special
|
1267
|
+
> **Note:** The special Powershell argument `--%` places Powershell in legacy parsing mode.
|
1229
1268
|
|
1230
1269
|
#### Extended Values (JSON, Ruby, ...)
|
1231
1270
|
|
1232
|
-
Some
|
1271
|
+
Some values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
1233
1272
|
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
1234
1273
|
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
1235
1274
|
|
@@ -1256,8 +1295,8 @@ ERROR: Argument: unprocessed values: ["2", "3"]
|
|
1256
1295
|
> **Note:** It gets its value after shell command line parsing and `ascli` extended value parsing.
|
1257
1296
|
|
1258
1297
|
In the following examples (using a POSIX shell, such as `bash`), several equivalent commands are provided.
|
1259
|
-
For all example, most
|
1260
|
-
It depends on the underlying syntax: shell
|
1298
|
+
For all example, most special character handling is not specific to `ascli`:
|
1299
|
+
It depends on the underlying syntax: shell, JSON, etc...
|
1261
1300
|
Depending on the case, a different `format` option is used to display the actual value.
|
1262
1301
|
|
1263
1302
|
For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
|
@@ -1307,7 +1346,7 @@ ascli config echo '"'
|
|
1307
1346
|
```
|
1308
1347
|
|
1309
1348
|
Double quote in JSON is a little tricky because `"` is special both for the shell and JSON.
|
1310
|
-
Both shell and JSON syntax allow
|
1349
|
+
Both shell and JSON syntax allow protecting `"`, but only the shell allows protection using single quote.
|
1311
1350
|
|
1312
1351
|
```bash
|
1313
1352
|
ascli config echo @json:'"\""' --format=text
|
@@ -1479,7 +1518,7 @@ The advantages of using a **Command Parameter** instead of an option for the sam
|
|
1479
1518
|
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable using an option value.
|
1480
1519
|
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
|
1481
1520
|
|
1482
|
-
If a **Command
|
1521
|
+
If a **Command Parameter** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
1483
1522
|
|
1484
1523
|
A few **Command Parameters** are optional, they are always located at the end of the command line.
|
1485
1524
|
|
@@ -1488,7 +1527,7 @@ A few **Command Parameters** are optional, they are always located at the end of
|
|
1488
1527
|
All options, e.g. `--log-level=debug`, are command line arguments that:
|
1489
1528
|
|
1490
1529
|
- Start with `--`
|
1491
|
-
- Have a name, in lowercase, using `-` as word separator in name
|
1530
|
+
- Have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
|
1492
1531
|
- Have a value, separated from name with a `=`
|
1493
1532
|
- Can be used by prefix (not recommended), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
|
1494
1533
|
- Is optional on command line (it has a default value or no value)
|
@@ -1512,7 +1551,7 @@ ascli config echo -- --sample
|
|
1512
1551
|
|
1513
1552
|
> **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
|
1514
1553
|
|
1515
|
-
Options may have a (
|
1554
|
+
Options may have a (hard coded) default value.
|
1516
1555
|
|
1517
1556
|
Options can be placed anywhere on command line and are evaluated in order.
|
1518
1557
|
Usually the last value evaluated overrides previous values, but some options are cumulative, e.g. `ts`.
|
@@ -1591,7 +1630,7 @@ By default, result of type `single_object` and `object_list` are displayed using
|
|
1591
1630
|
|
1592
1631
|
The table style can be customized with option: `table_style` which expects a `Hash`, options are the ones described in gem [`terminal-table`](https://github.com/tj/terminal-table).
|
1593
1632
|
|
1594
|
-
For example, to display a table with thick
|
1633
|
+
For example, to display a table with thick Unicode borders:
|
1595
1634
|
|
1596
1635
|
```bash
|
1597
1636
|
ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
|
@@ -1601,13 +1640,13 @@ ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
|
|
1601
1640
|
|
1602
1641
|
#### Option: `flat_hash`: `.`-join keys
|
1603
1642
|
|
1604
|
-
This
|
1643
|
+
This option controls how object fields are displayed for complex objects.
|
1605
1644
|
|
1606
1645
|
Effective only when `format` is `table` to display `single_object` or `object_list`.
|
1607
1646
|
|
1608
1647
|
If value is `no`, then object's `field` names are only the first level keys of the `Hash` result and values that are `Hash` are displayed as such in Ruby syntax.
|
1609
1648
|
|
1610
|
-
If value is `yes` (default), then object are flattened, i.e. deep `Hash` are transformed into 1-level Hash
|
1649
|
+
If value is `yes` (default), then object are flattened, i.e. deep `Hash` are transformed into 1-level `Hash`, where keys are `.`-junction of deep keys.
|
1611
1650
|
In this case, it is possible to filter fields using the option `fields` using the compound field name using `.` (dot) as separator.
|
1612
1651
|
|
1613
1652
|
Example: Result of command is a list of objects with a single object:
|
@@ -1635,7 +1674,7 @@ This option controls if object fields are displayed as columns or lines.
|
|
1635
1674
|
If value is `no` (default), `object_list` are displayed with one object per line, with fields as columns (see above).
|
1636
1675
|
`single_object` are displayed with one field per line (and columns are: `field`, `value`).
|
1637
1676
|
|
1638
|
-
If
|
1677
|
+
If a `object_list` has a single element, it is possible to have `ascli` display the object as a single object (one field per line instead of columns) with option: `multi_single` set to `single`.
|
1639
1678
|
|
1640
1679
|
This parameter can be set as a global default with:
|
1641
1680
|
|
@@ -1672,11 +1711,9 @@ The option `display` controls the level of output:
|
|
1672
1711
|
|
1673
1712
|
#### Option: `show_secrets`: Hide or show secrets in results
|
1674
1713
|
|
1675
|
-
If value is `no` (default), then secrets are redacted from command results.
|
1676
|
-
|
1677
|
-
If
|
1678
|
-
|
1679
|
-
If `display` is `data`, secrets are included to allows piping results.
|
1714
|
+
- If value is `no` (default), then secrets are redacted from command results.
|
1715
|
+
- If value is `yes`, then secrets shown in clear in results.
|
1716
|
+
- If `display` is `data`, secrets are included to allow piping results.
|
1680
1717
|
|
1681
1718
|
#### Option: `fields`: Selection of output object fields
|
1682
1719
|
|
@@ -1704,7 +1741,7 @@ Examples:
|
|
1704
1741
|
|
1705
1742
|
Table output (`object_list`) can be filtered using option `select`.
|
1706
1743
|
This option is either a `Hash` or `Proc`.
|
1707
|
-
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that shall
|
1744
|
+
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that shall return `true` to select or `false` to remove an entry.
|
1708
1745
|
|
1709
1746
|
Example:
|
1710
1747
|
|
@@ -1753,7 +1790,7 @@ ascli aoc admin user show %name:john
|
|
1753
1790
|
### Extended Value Syntax
|
1754
1791
|
|
1755
1792
|
Most options and arguments are specified by a simple string (e.g. `username` or `url`).
|
1756
|
-
|
1793
|
+
Sometimes it is convenient to read a value from a file: for example read the PEM value of a private key, or a list of files.
|
1757
1794
|
Some options expect a more complex value such as `Hash` or `Array`.
|
1758
1795
|
|
1759
1796
|
The **Extended Value** Syntax allows to specify such values and even read values from other sources than the command line itself.
|
@@ -1764,7 +1801,7 @@ The syntax is:
|
|
1764
1801
|
<0 or more decoders><some text value or nothing>
|
1765
1802
|
```
|
1766
1803
|
|
1767
|
-
Decoders act like a function with its parameter on right
|
1804
|
+
Decoders act like a function with its parameter on right-hand side and are recognized by the prefix: `@` and suffix `:`
|
1768
1805
|
|
1769
1806
|
The following decoders are supported:
|
1770
1807
|
|
@@ -1779,7 +1816,7 @@ The following decoders are supported:
|
|
1779
1816
|
| `list` | `String` | `Array` | Split a string in multiple items taking first character as separator and return an array |
|
1780
1817
|
| `none` | None | Nil | A null value |
|
1781
1818
|
| `path` | `String` | `String` | Performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml` |
|
1782
|
-
| `preset` | `String` | `Hash` | Get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]` |
|
1819
|
+
| `preset` | `String` | `Hash` | Get whole option preset value by name. Sub-values can also be used, using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]` |
|
1783
1820
|
| `extend` | `String` | `String` | Evaluates embedded extended value syntax in string |
|
1784
1821
|
| `re` | `String` | `Regexp` | Ruby Regular Expression (short for `@ruby:/.../`) |
|
1785
1822
|
| `ruby` | `String` | Any | Execute specified Ruby code |
|
@@ -1789,7 +1826,7 @@ The following decoders are supported:
|
|
1789
1826
|
| `uri` | `String` | `String` | Read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
1790
1827
|
| `val` | `String` | `String` | Prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
1791
1828
|
| `yaml` | `String` | Any | Decode YAML |
|
1792
|
-
| `zlib` | `String` | `String` |
|
1829
|
+
| `zlib` | `String` | `String` | Decompress data using zlib |
|
1793
1830
|
|
1794
1831
|
> **Note:** A few commands support a value of type `Proc` (lambda expression).
|
1795
1832
|
For example, the **Extended Value** `@ruby:'->(i){i["attr"]}'` is a lambda expression that returns the value for key `attr` of the `Hash` `i`.
|
@@ -1817,7 +1854,7 @@ Example: Create a `Hash` value with one key and the value is read from a file:
|
|
1817
1854
|
ascli config echo @ruby:'{"token_verification_key"=>File.read("mykey.txt")}'
|
1818
1855
|
```
|
1819
1856
|
|
1820
|
-
Example: read a
|
1857
|
+
Example: read a CSV file and create an `Array` of `Hash` for bulk provisioning:
|
1821
1858
|
|
1822
1859
|
```bash
|
1823
1860
|
cat test.csv
|
@@ -1857,7 +1894,7 @@ ascli config echo @json:@extend:'{"hello":true,"version":"@preset:config.version
|
|
1857
1894
|
╰─────────┴────────╯
|
1858
1895
|
```
|
1859
1896
|
|
1860
|
-
Example: Create a `Hash` from YAML provided as shell **
|
1897
|
+
Example: Create a `Hash` from YAML provided as shell **Here document**:
|
1861
1898
|
|
1862
1899
|
```bash
|
1863
1900
|
ascli config echo @yaml:@stdin: --format=json<<EOF
|
@@ -1907,7 +1944,7 @@ C:\Users\Kenji\.aspera\ascli
|
|
1907
1944
|
```
|
1908
1945
|
|
1909
1946
|
When OAuth is used (AoC, Faspex5) `ascli` keeps a cache of generated bearer tokens in folder `persist_store` in configuration folder by default.
|
1910
|
-
Option `cache_tokens` (**yes**/no) allows
|
1947
|
+
Option `cache_tokens` (**yes**/no) allows controlling if OAuth tokens are cached on file system, or generated for each request.
|
1911
1948
|
The command `config tokens flush` clears that cache.
|
1912
1949
|
Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
|
1913
1950
|
When a token has expired, then a new token is generated, either using a refresh_token if it is available, or by the default method.
|
@@ -1933,11 +1970,11 @@ A configuration file provides a way to define default values, especially for aut
|
|
1933
1970
|
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or its env var).
|
1934
1971
|
|
1935
1972
|
The configuration file is a catalog of named lists of options, called: [Option Preset](#option-preset).
|
1936
|
-
Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of
|
1973
|
+
Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of an [Option Preset](#option-preset) (e.g. `mypreset`) using the option `preset`: `--preset=mypreset` or its shortcut: `-Pmypreset`.
|
1937
1974
|
|
1938
1975
|
#### Option Preset
|
1939
1976
|
|
1940
|
-
|
1977
|
+
An [Option Preset](#option-preset) is a collection of options and their associated values in a named section in the configuration file.
|
1941
1978
|
|
1942
1979
|
A named [Option Preset](#option-preset) can be modified directly using `ascli`, which will update the configuration file :
|
1943
1980
|
|
@@ -1951,9 +1988,9 @@ The command `update` allows the easy creation of [Option Preset](#option-preset)
|
|
1951
1988
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
|
1952
1989
|
```
|
1953
1990
|
|
1954
|
-
- This creates
|
1991
|
+
- This creates an [Option Preset](#option-preset) `demo_server` with all provided options.
|
1955
1992
|
|
1956
|
-
The command `set` allows setting individual options in
|
1993
|
+
The command `set` allows setting individual options in an [Option Preset](#option-preset).
|
1957
1994
|
|
1958
1995
|
```bash
|
1959
1996
|
ascli config preset set demo_server password my_password_here
|
@@ -1978,7 +2015,7 @@ ascli config preset list
|
|
1978
2015
|
```
|
1979
2016
|
|
1980
2017
|
A good practice is to not manually edit the configuration file and use modification commands instead.
|
1981
|
-
If necessary, the configuration file can opened in a text editor with:
|
2018
|
+
If necessary, the configuration file can be opened in a text editor with:
|
1982
2019
|
|
1983
2020
|
```bash
|
1984
2021
|
ascli config open
|
@@ -1986,7 +2023,7 @@ ascli config open
|
|
1986
2023
|
|
1987
2024
|
> **Note:** This starts the editor specified by env var `EDITOR` if defined.
|
1988
2025
|
|
1989
|
-
|
2026
|
+
The former format for commands is still supported:
|
1990
2027
|
|
1991
2028
|
```bash
|
1992
2029
|
ascli config preset set|delete|show|initialize|update <name>
|
@@ -1994,14 +2031,14 @@ ascli config preset over
|
|
1994
2031
|
ascli config preset list
|
1995
2032
|
```
|
1996
2033
|
|
1997
|
-
It is possible to load
|
2034
|
+
It is possible to load an [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
|
1998
2035
|
For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
|
1999
2036
|
|
2000
2037
|
```bash
|
2001
2038
|
ascli config preset set pspecific preset pcommon
|
2002
2039
|
```
|
2003
2040
|
|
2004
|
-
When `pspecific` is loaded, then cumulative option `preset` will be set and it will also load `pcommon`.
|
2041
|
+
When `pspecific` is loaded, then cumulative option `preset` will be set, and it will also load `pcommon`.
|
2005
2042
|
|
2006
2043
|
#### Special Option Preset: `config`
|
2007
2044
|
|
@@ -2009,7 +2046,7 @@ This preset name is reserved and contains a single key: `version`. This is the v
|
|
2009
2046
|
|
2010
2047
|
#### Special Option Preset: `default`
|
2011
2048
|
|
2012
|
-
This preset name is reserved and contains an array of key-value
|
2049
|
+
This preset name is reserved and contains an array of key-value, where the key is the name of a plugin, and the value is the name of another preset.
|
2013
2050
|
|
2014
2051
|
When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option --no-default (or -N) is used.
|
2015
2052
|
|
@@ -2081,6 +2118,7 @@ ascp install --sdk-folder=sdk_test_dir
|
|
2081
2118
|
ascp install 1.1.3
|
2082
2119
|
ascp products list
|
2083
2120
|
ascp products use 'IBM Aspera Connect'
|
2121
|
+
ascp schema --format=jsonpp
|
2084
2122
|
ascp show
|
2085
2123
|
ascp spec
|
2086
2124
|
check_update
|
@@ -2137,7 +2175,7 @@ preset set default shares conf_name
|
|
2137
2175
|
preset show conf_name
|
2138
2176
|
preset unset conf_name param
|
2139
2177
|
preset update conf_name --p1=v1 --p2=v2
|
2140
|
-
proxy_check --fpac=@file:
|
2178
|
+
proxy_check --fpac=@file:proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
|
2141
2179
|
pubkey @file:my_key
|
2142
2180
|
remote_certificate chain https://node.example.com/path
|
2143
2181
|
remote_certificate name https://node.example.com/path
|
@@ -2186,7 +2224,7 @@ We can see here:
|
|
2186
2224
|
- The [Option Preset](#option-preset) `demo_server` defines some options: the URL and credentials
|
2187
2225
|
- The default [Option Preset](#option-preset) to load in any case is : `cli_default`
|
2188
2226
|
|
2189
|
-
Two [Option
|
2227
|
+
Two [Option Presets](#option-preset) are reserved:
|
2190
2228
|
|
2191
2229
|
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
2192
2230
|
It is used to check compatibility.
|
@@ -2241,7 +2279,7 @@ ascli config preset set cli_default interactive no
|
|
2241
2279
|
ascli config preset set default config cli_default
|
2242
2280
|
```
|
2243
2281
|
|
2244
|
-
|
2282
|
+
An [Option Preset](#option-preset) value can be removed with `unset`:
|
2245
2283
|
|
2246
2284
|
```bash
|
2247
2285
|
ascli config preset unset cli_default interactive
|
@@ -2261,9 +2299,25 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
|
|
2261
2299
|
|
2262
2300
|
#### Wizard
|
2263
2301
|
|
2264
|
-
The wizard is a command that asks the user for information and creates
|
2302
|
+
The wizard is a command that asks the user for information and creates an [Option Preset](#option-preset) with the provided information for a given application.
|
2303
|
+
|
2304
|
+
It takes three optional arguments:
|
2265
2305
|
|
2266
|
-
|
2306
|
+
- The URL of the application, else it will ask for it;
|
2307
|
+
- The plugin name: it limits detection to a given plugin, else it will try to detect known plugins from the URL
|
2308
|
+
- The preset name: it will create a new [Option Preset](#option-preset) with this name, else it will use specific information to generate a unique preset name.
|
2309
|
+
|
2310
|
+
Special options are also available to the wizard:
|
2311
|
+
|
2312
|
+
| Option | Value | Description |
|
2313
|
+
|-------------|----------|-------------|
|
2314
|
+
| `default` | [yes]/no | Set as default configuration for specified plugin. |
|
2315
|
+
| `override` | yes/[no] | Override existing default preset name for the plugin, if it exists. |
|
2316
|
+
| `key-path` | path | Path to private key for JWT. |
|
2317
|
+
| `test-mode` | yes/[no] | Skip private key check step. |
|
2318
|
+
|
2319
|
+
Other options can be provided to the wizard, such as `--username`, etc...
|
2320
|
+
They will be added to the [Option Preset](#option-preset) created by the wizard.
|
2267
2321
|
|
2268
2322
|
The simplest invocation is:
|
2269
2323
|
|
@@ -2275,7 +2329,7 @@ ascli config wizard
|
|
2275
2329
|
|
2276
2330
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
2277
2331
|
only username/password and URL are required (either on command line, or from configuration file).
|
2278
|
-
Those can usually
|
2332
|
+
Those can be usually provided on the command line:
|
2279
2333
|
|
2280
2334
|
```bash
|
2281
2335
|
ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my_password_here
|
@@ -2315,7 +2369,7 @@ ascli config preset set default shares shares06
|
|
2315
2369
|
ascli config preset overview
|
2316
2370
|
```
|
2317
2371
|
|
2318
|
-
- Execute a command on the
|
2372
|
+
- Execute a command on the **Shares** application using default options
|
2319
2373
|
|
2320
2374
|
```bash
|
2321
2375
|
ascli shares repo browse /
|
@@ -2323,8 +2377,8 @@ ascli shares repo browse /
|
|
2323
2377
|
|
2324
2378
|
### Secret Vault
|
2325
2379
|
|
2326
|
-
Secrets
|
2327
|
-
|
2380
|
+
Secrets, e.g. passwords, keys, are needed when connecting to applications.
|
2381
|
+
Those secrets are usually provided as command options, on command line, env vars, files etc.
|
2328
2382
|
|
2329
2383
|
For security reasons, those secrets shall not be exposed in clear, either:
|
2330
2384
|
|
@@ -2332,13 +2386,13 @@ For security reasons, those secrets shall not be exposed in clear, either:
|
|
2332
2386
|
- In logs
|
2333
2387
|
- In command output
|
2334
2388
|
|
2335
|
-
Instead, they shall be hidden or encrypted.
|
2389
|
+
Instead, they shall be hidden (logs) or encrypted (configuration).
|
2336
2390
|
|
2337
2391
|
Terminal output secret removal is controlled by option `show_secrets` (default: `no`).
|
2338
2392
|
Log secret removal is controlled by option `log_secrets` (default: `no`).
|
2339
2393
|
Mandatory command line options can be requested interactively (e.g. password) using option `interactive`.
|
2340
2394
|
Or it is possible to use extended value `@secret:[name]` to ask for a secret interactively.
|
2341
|
-
It is also possible to enter an option as an environment variable, e.g. `ASCLI_PASSWORD` for option `password` and read the env var
|
2395
|
+
It is also possible to enter an option as an environment variable, e.g. `ASCLI_PASSWORD` for option `password` and read the env var like this:
|
2342
2396
|
|
2343
2397
|
```bash
|
2344
2398
|
read -s ASCLI_PASSWORD
|
@@ -2358,18 +2412,31 @@ The vault is used with options `vault` and `vault_password`.
|
|
2358
2412
|
`vault_password` specifies the password for the vault.
|
2359
2413
|
|
2360
2414
|
Although it can be specified on command line, for security reason you should avoid exposing the secret.
|
2361
|
-
For example it can be securely specified on command line like this:
|
2415
|
+
For example, it can be securely specified on command line like this:
|
2362
2416
|
|
2363
2417
|
```bash
|
2364
2418
|
read -s ASCLI_VAULT_PASSWORD
|
2365
2419
|
export ASCLI_VAULT_PASSWORD
|
2366
2420
|
```
|
2367
2421
|
|
2368
|
-
#### Vault:
|
2422
|
+
#### Vault: IBM HashiCorp Vault
|
2423
|
+
|
2424
|
+
<https://developer.hashicorp.com/vault>
|
2425
|
+
|
2426
|
+
Quick start macOS:
|
2427
|
+
|
2428
|
+
```bash
|
2429
|
+
gem install vault
|
2430
|
+
brew tap hashicorp/tap
|
2431
|
+
brew install hashicorp/tap/vault
|
2432
|
+
vault server -dev -dev-root-token-id=dev-only-token
|
2433
|
+
```
|
2434
|
+
|
2435
|
+
#### Vault: System keychain
|
2369
2436
|
|
2370
2437
|
> **Note:** **macOS only**
|
2371
2438
|
|
2372
|
-
It is possible to manage secrets in macOS
|
2439
|
+
It is possible to manage secrets in macOS keychain (only read supported currently).
|
2373
2440
|
|
2374
2441
|
```bash
|
2375
2442
|
--vault=@json:'{"type":"system","name":"ascli"}'
|
@@ -2408,7 +2475,7 @@ The lookup is done by comparing the service URL and username (or access key).
|
|
2408
2475
|
|
2409
2476
|
#### Securing passwords and secrets
|
2410
2477
|
|
2411
|
-
A
|
2478
|
+
A password can be saved in clear in an [Option Preset](#option-preset) together with other account information (URL, username, etc...).
|
2412
2479
|
Example:
|
2413
2480
|
|
2414
2481
|
```bash
|
@@ -2512,15 +2579,35 @@ For example: <https://cryptotools.net/rsagen>
|
|
2512
2579
|
|
2513
2580
|
> **Note:** Be careful that private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
|
2514
2581
|
|
2582
|
+
### Web service
|
2583
|
+
|
2584
|
+
Some plugins start a local web server.
|
2585
|
+
This server can server HTTP or HTTPS (with certificate):
|
2586
|
+
|
2587
|
+
The following parameters are supported:
|
2588
|
+
|
2589
|
+
| Parameter | Type | Default | Description |
|
2590
|
+
|-------------------|----------|---------|-----------------------------------------------------|
|
2591
|
+
| `url` | `String` | `http://localhost:8080` | Base URL on which requests are listened, a path can be provided. | <!-- markdownlint-disable-line -->
|
2592
|
+
| `cert` | `String` | - | (HTTPS) Path to certificate file (with ext. `.pfx` or `.p12` for PKCS12) |
|
2593
|
+
| `key` | `String` | - | (HTTPS) Path to private key file (PEM), or passphrase for PKCS12 |
|
2594
|
+
| `chain` | `String` | - | (HTTPS) Path to certificate chain (PEM only) |
|
2595
|
+
|
2596
|
+
Parameter `url` (base URL) defines:
|
2597
|
+
|
2598
|
+
- If `http` or `https` is used
|
2599
|
+
- The local port number (default 443 for HTTPS, 80 for HTTP)
|
2600
|
+
- The **base path**, i.e. the path under which requests are received, if a reverse proxy is used this can be used to route.
|
2601
|
+
|
2515
2602
|
### Image and video thumbnails
|
2516
2603
|
|
2517
2604
|
`ascli` can display thumbnails for images and videos in the terminal.
|
2518
2605
|
This is available:
|
2519
2606
|
|
2520
|
-
-
|
2521
|
-
-
|
2607
|
+
- In the `thumbnail` command of `node` when using **gen4/access key** API.
|
2608
|
+
- When using the `show` command of `preview` plugin.
|
2522
2609
|
- `coffee` and `image` commands of `config` plugin.
|
2523
|
-
-
|
2610
|
+
- Any displayed value which is a URL to image can be displayed with option `format` set to `image`
|
2524
2611
|
|
2525
2612
|
The following options can be specified in the option `image`:
|
2526
2613
|
|
@@ -2550,8 +2637,8 @@ Some actions may require the use of a graphical tool:
|
|
2550
2637
|
- A browser for Aspera on Cloud authentication (web auth method)
|
2551
2638
|
- A text editor for configuration file edition
|
2552
2639
|
|
2553
|
-
By default `ascli` assumes that a graphical environment is available on Windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
2554
|
-
It is also possible to force the graphical mode with option
|
2640
|
+
By default, `ascli` assumes that a graphical environment is available on Windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
2641
|
+
It is also possible to force the graphical mode with option `--ui` :
|
2555
2642
|
|
2556
2643
|
- `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
2557
2644
|
- `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
@@ -2559,10 +2646,10 @@ It is also possible to force the graphical mode with option --ui :
|
|
2559
2646
|
### Logging, Debugging
|
2560
2647
|
|
2561
2648
|
The gem is equipped with traces, mainly for debugging and learning APIs.
|
2562
|
-
By default logging level is `warn` and the output channel is `stderr`.
|
2649
|
+
By default, logging level is `warn` and the output channel is `stderr`.
|
2563
2650
|
To increase debug level, use option `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or an [Option Preset](#option-preset)).
|
2564
2651
|
|
2565
|
-
By default passwords and secrets are redacted from logs.
|
2652
|
+
By default, passwords and secrets are redacted from logs.
|
2566
2653
|
Set option `log_secrets` to `yes` to include secrets in logs.
|
2567
2654
|
|
2568
2655
|
Option `logger`: `stdout`, `stderr`, `syslog`.
|
@@ -2609,16 +2696,19 @@ To disable the warning, use option `silent_insecure` set to `no`.
|
|
2609
2696
|
|
2610
2697
|
HTTP connection parameters (not `ascp` WSS) can be adjusted using option `http_options`:
|
2611
2698
|
|
2612
|
-
| Parameter | Default
|
2613
|
-
|
2614
|
-
| `read_timeout` | 60 |
|
2615
|
-
| `write_timeout` | 60 |
|
2616
|
-
| `open_timeout` | 60 |
|
2617
|
-
| `keep_alive_timeout` | 2 |
|
2618
|
-
| `user_agent` | `ascli`
|
2619
|
-
| `download_partial_suffix` |
|
2620
|
-
| `retry_on_error` |
|
2621
|
-
| `
|
2699
|
+
| Parameter | Type | Default | Handler |
|
2700
|
+
|---------------------------|------|-----------------|-----------|
|
2701
|
+
| `read_timeout` | int | `60` | Ruby |
|
2702
|
+
| `write_timeout` | int | `60` | Ruby |
|
2703
|
+
| `open_timeout` | int | `60` | Ruby |
|
2704
|
+
| `keep_alive_timeout` | int | `2` | Ruby |
|
2705
|
+
| `user_agent` | int | `ascli` | `ascli` |
|
2706
|
+
| `download_partial_suffix` | int | `.http_partial` | `ascli` |
|
2707
|
+
| `retry_on_error` | bool | `false` | `ascli` |
|
2708
|
+
| `retry_on_timeout` | bool | `true` | `ascli` |
|
2709
|
+
| `retry_on_unavailable` | bool | `true` | `ascli` |
|
2710
|
+
| `retry_max` | int | `1` | `ascli` |
|
2711
|
+
| `retry_sleep` | int | `4` | `ascli` |
|
2622
2712
|
|
2623
2713
|
Time values are in set **seconds** and can be of type either `Integer` or `Float`.
|
2624
2714
|
Default values are the ones of Ruby:
|
@@ -2647,7 +2737,7 @@ Refer to the following sections.
|
|
2647
2737
|
REST API calls and transfers based on HTTP Gateway both use Ruby's `Net::HTTP` class.
|
2648
2738
|
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
2649
2739
|
|
2650
|
-
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used
|
2740
|
+
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used.
|
2651
2741
|
|
2652
2742
|
The `http_proxy` environment variable (**lower case**) can be set to the **URL** of the proxy (with optional credentials).
|
2653
2743
|
Syntax is: `(http|https)://[user:password@]host:port`.
|
@@ -2667,7 +2757,7 @@ Option `http_proxy` does the same (set env var) but on command line:
|
|
2667
2757
|
ascli --http-proxy=http://username:password@host:port ...
|
2668
2758
|
```
|
2669
2759
|
|
2670
|
-
Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config)
|
2760
|
+
Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) JavaScript value.
|
2671
2761
|
|
2672
2762
|
Note that proxy credentials are not supported in PAC files.
|
2673
2763
|
|
@@ -2739,7 +2829,7 @@ Or, alternatively, (prefer transfer spec like above, generally):
|
|
2739
2829
|
|
2740
2830
|
`ascli` uses one of the transfer agents to execute transfers.
|
2741
2831
|
|
2742
|
-
By default it uses the `direct` agent, which is basically a local `ascp`.
|
2832
|
+
By default, it uses the `direct` agent, which is basically a local `ascp`.
|
2743
2833
|
Nevertheless, `ascli` does not come with `ascp` installed.
|
2744
2834
|
This is the reason why it is advised to install the Aspera Transfer SDK during installation (`ascli config ascp install`).
|
2745
2835
|
|
@@ -2750,11 +2840,11 @@ The way to specify the location of `ascp` is to use either options:
|
|
2750
2840
|
- `ascp_path`
|
2751
2841
|
- `use_product`
|
2752
2842
|
|
2753
|
-
The `config` plugin allows
|
2754
|
-
It provides the following commands for `ascp`
|
2843
|
+
The `config` plugin allows finding and specifying the location of `ascp`.
|
2844
|
+
It provides the following commands for `ascp` sub-command:
|
2755
2845
|
|
2756
2846
|
- `show` : shows the path of `ascp` used
|
2757
|
-
- `use` : specify the ascp path to use
|
2847
|
+
- `use` : specify the `ascp` path to use
|
2758
2848
|
- `products` : list Aspera transfer products available locally
|
2759
2849
|
- `connect` : list and download connect client versions available on internet
|
2760
2850
|
|
@@ -2891,7 +2981,7 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
|
2891
2981
|
|
2892
2982
|
### Transfer Clients: Agents
|
2893
2983
|
|
2894
|
-
Some
|
2984
|
+
Some actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
2895
2985
|
Transfers will be executed by a transfer client, here called **Transfer Agent**.
|
2896
2986
|
|
2897
2987
|
The following agents are supported and selected with option `transfer`:
|
@@ -2908,7 +2998,7 @@ The following agents are supported and selected with option `transfer`:
|
|
2908
2998
|
> **Note:** All transfer operations are seen from the point of view of the agent.
|
2909
2999
|
For example, an agent executing an **upload**, or **package send** operation will effectively push files to the related server from the system where the agent runs.
|
2910
3000
|
|
2911
|
-
All
|
3001
|
+
All above agents (including `direct`) receive transfer parameters as a [**transfer-spec**](#transfer-specification).
|
2912
3002
|
Parameters in transfer-spec can be modified with option `ts`.
|
2913
3003
|
|
2914
3004
|
**Specific** options for agents are provided with option `transfer_info`.
|
@@ -2930,7 +3020,7 @@ The `transfer_info` option accepts the following optional parameters to control
|
|
2930
3020
|
| `quiet` | `Bool` | If `true`, then `ascp` progress bar is not shown.<br/>Default: `false` |
|
2931
3021
|
| `trusted_certs` | `Array` | List of repositories for trusted certificates. |
|
2932
3022
|
| `client_ssh_key` | `String` | SSH Keys to use for token-based transfers. One of: `dsa_rsa`, `rsa`, `per_client`. Default: `rsa` |
|
2933
|
-
| `ascp_args` | `Array` | Array of strings with native `ascp` arguments.<br/>Default: `[]` |
|
3023
|
+
| `ascp_args` | `Array` | `Array` of strings with native `ascp` arguments.<br/>Default: `[]` |
|
2934
3024
|
| `spawn_timeout_sec` | `Float` | Multi session<br/>Verification time that `ascp` is running<br/>Default: `3` |
|
2935
3025
|
| `spawn_delay_sec` | `Float` | Multi session<br/>Delay between startup of sessions<br/>Default: `2` |
|
2936
3026
|
| `multi_incr_udp` | `Bool` | Multi Session<br/>Increment UDP port on multi-session<br/>If `true`, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: `true` on Windows, else `false` |
|
@@ -2939,14 +3029,20 @@ The `transfer_info` option accepts the following optional parameters to control
|
|
2939
3029
|
| `resume.sleep_initial` | `Integer` | First Sleep before retry<br/>Default: `2` |
|
2940
3030
|
| `resume.sleep_factor` | `Integer` | Multiplier of sleep period between attempts<br/>Default: `2` |
|
2941
3031
|
| `resume.sleep_max` | `Integer` | Default: `60` |
|
3032
|
+
| `monitor` | `Bool` | Use management port.<br/>Default: `true` |
|
2942
3033
|
|
2943
3034
|
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
|
2944
3035
|
Sleep between iterations is given by the following formula where `iter_index` is the current iteration index, starting at 0:
|
2945
3036
|
|
2946
3037
|
```bash
|
2947
|
-
max( sleep_max
|
3038
|
+
max( sleep_max, sleep_initial * sleep_factor ^ iter_index )
|
2948
3039
|
```
|
2949
3040
|
|
3041
|
+
To display the native progress bar of `ascp`, use `--progress-bar=no --transfer-info=@json:'{"quiet":false}'`.
|
3042
|
+
|
3043
|
+
To skip usage of management port (which disables custom progress bar), set option `monitor` to `false`.
|
3044
|
+
In that, use the native progress bar: `--transfer-info=@json:'{"monitor":false,"quiet":false}'`
|
3045
|
+
|
2950
3046
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
2951
3047
|
In order to use a custom certificate store, use the `trusted_certs` option of direct agent's option `transfer_info`.
|
2952
3048
|
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
@@ -3070,12 +3166,12 @@ This is especially useful for direct node-to-node transfers.
|
|
3070
3166
|
|
3071
3167
|
Parameters provided in option `transfer_info` are:
|
3072
3168
|
|
3073
|
-
|
|
3074
|
-
|
3075
|
-
| `url` | `String` | URL of the
|
3076
|
-
| `username` | `String` | Node
|
3169
|
+
| Parameter | Type | Description |
|
3170
|
+
|------------|----------|----------------------------------------------------|
|
3171
|
+
| `url` | `String` | URL of the Node API<br/>Mandatory |
|
3172
|
+
| `username` | `String` | Node API user or access key<br/>Mandatory |
|
3077
3173
|
| `password` | `String` | Password, secret or bearer token<br/>Mandatory |
|
3078
|
-
| `root_id` | `String` | Root file
|
3174
|
+
| `root_id` | `String` | Root file ID<br/>Mandatory only for bearer token |
|
3079
3175
|
|
3080
3176
|
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset) :
|
3081
3177
|
|
@@ -3091,8 +3187,8 @@ or be specified using the extended value syntax :
|
|
3091
3187
|
|
3092
3188
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
3093
3189
|
|
3094
|
-
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file
|
3095
|
-
It can be either the access key's root file
|
3190
|
+
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file ID of the top folder to use on the node using this access key.
|
3191
|
+
It can be either the access key's root file ID, or any authorized file ID underneath it.
|
3096
3192
|
|
3097
3193
|
#### Agent: HTTP Gateway
|
3098
3194
|
|
@@ -3147,11 +3243,11 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
3147
3243
|
|
3148
3244
|
- `cannot find -lx64-ucrt-ruby310`
|
3149
3245
|
|
3150
|
-
→ copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the dll extension)
|
3246
|
+
→ copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the `dll` extension)
|
3151
3247
|
|
3152
3248
|
- `conflicting types for 'gettimeofday'`
|
3153
3249
|
|
3154
|
-
→ edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)
|
3250
|
+
→ edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)`, i.e. change `struct timezone` to `void`
|
3155
3251
|
|
3156
3252
|
<!-- spellchecker: enable -->
|
3157
3253
|
|
@@ -3161,13 +3257,13 @@ Some commands lead to file transfer (upload/download).
|
|
3161
3257
|
All parameters necessary for this transfer are described in a [**transfer-spec**](#transfer-specification) (Transfer Specification), such as:
|
3162
3258
|
|
3163
3259
|
- Server address
|
3164
|
-
- Transfer
|
3260
|
+
- Transfer username
|
3165
3261
|
- Credentials
|
3166
3262
|
- File list
|
3167
3263
|
- Etc...
|
3168
3264
|
|
3169
3265
|
`ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
|
3170
|
-
It is not necessary to provide additional parameters on the command line for
|
3266
|
+
It is not necessary to provide additional parameters on the command line for a transfer.
|
3171
3267
|
|
3172
3268
|
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameter using the `ts` option.
|
3173
3269
|
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
|
@@ -3188,7 +3284,7 @@ The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` comm
|
|
3188
3284
|
### Transfer Parameters
|
3189
3285
|
|
3190
3286
|
All standard [**transfer-spec**](#transfer-specification) parameters can be specified.
|
3191
|
-
[**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
|
3287
|
+
A [**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
|
3192
3288
|
|
3193
3289
|
References:
|
3194
3290
|
|
@@ -3203,102 +3299,126 @@ ascli config ascp spec
|
|
3203
3299
|
ascli config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
|
3204
3300
|
```
|
3205
3301
|
|
3206
|
-
|
3302
|
+
A JSON Schema can be generated with command:
|
3303
|
+
|
3304
|
+
```bash
|
3305
|
+
ascli config ascp schema --format=jsonpp
|
3306
|
+
```
|
3307
|
+
|
3308
|
+
An optional parameter can be specified to display the schema for a specific transfer agent:
|
3207
3309
|
|
3208
|
-
|
3209
|
-
|
3210
|
-
|
3211
|
-
- T=Transfer SDK
|
3212
|
-
- H=HTTP Gateway
|
3310
|
+
```bash
|
3311
|
+
ascli config ascp schema transferd --format=jsonpp
|
3312
|
+
```
|
3213
3313
|
|
3214
3314
|
`ascp` argument or environment variable is provided in description.
|
3215
3315
|
|
3216
|
-
|
|
3316
|
+
| ID | Name |
|
3317
|
+
| -- | ---- |
|
3318
|
+
| A | Direct |
|
3319
|
+
| C | Connect |
|
3320
|
+
| D | Desktop |
|
3321
|
+
| H | Httpgw |
|
3322
|
+
| N | Node |
|
3323
|
+
| T | Transferd |
|
3324
|
+
|
3325
|
+
| Field | Type | A | C | D | H | N | T | Description |
|
3217
3326
|
| ----- | ---- | - | - | - | - | - | - | ----------- |
|
3218
|
-
| apply_local_docroot |
|
3219
|
-
| authentication | string | | | | | |
|
3220
|
-
| cipher | string | Y | Y | Y | Y | Y | Y | In transit encryption
|
3221
|
-
| cipher_allowed | string |
|
3222
|
-
| compression |
|
3223
|
-
| content_protection | string | Y | Y | Y | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt
|
3327
|
+
| apply_local_docroot | boolean | Y | | | | | Y | Apply local docroot to source paths.<br/>(--apply-local-docroot) |
|
3328
|
+
| authentication | string | | Y | | | | | value=token for SSH bypass keys, else password asked if not provided. |
|
3329
|
+
| cipher | string | Y | Y | Y | Y | Y | Y | In transit encryption algorithms.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
3330
|
+
| cipher_allowed | string | | Y | | | | | returned by node API. Valid literals include "aes-128" and "none". |
|
3331
|
+
| compression | integer | | | | | | | ascp4 only, 0 / 1? |
|
3332
|
+
| content_protection | string | Y | Y | Y | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt={enum}) |
|
3224
3333
|
| content_protection_password | string | Y | Y | Y | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
|
3225
3334
|
| cookie | string | Y | Y | Y | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
|
3226
|
-
| create_dir |
|
3227
|
-
| delete_before_transfer |
|
3228
|
-
| delete_source |
|
3335
|
+
| create_dir | boolean | Y | Y | Y | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
|
3336
|
+
| delete_before_transfer | boolean | Y | Y | Y | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects on the source are not deleted.<br/>(--delete-before-transfer) |
|
3337
|
+
| delete_source | boolean | Y | | | | Y | Y | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
|
3229
3338
|
| destination_root | string | Y | Y | Y | Y | Y | Y | Destination root directory.<br/>(<special>) |
|
3230
|
-
| destination_root_id | string | | | | | |
|
3231
|
-
| dgram_size |
|
3232
|
-
| direction | string | Y | Y | Y | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode
|
3233
|
-
| exclude_newer_than | string | Y | Y | Y | Y | Y | Y | Exclude files, but not directories, from the transfer if they are newer than the specified number of seconds added to the source computer's epoch
|
3234
|
-
| exclude_older_than | string | Y | Y | Y | Y | Y | Y | Exclude files, but not directories, from the transfer if they are older than the specified number of seconds added to the source computer's epoch
|
3235
|
-
|
|
3236
|
-
|
|
3237
|
-
|
|
3238
|
-
|
|
3239
|
-
|
|
3240
|
-
|
|
3241
|
-
|
|
3242
|
-
|
|
3243
|
-
|
|
3244
|
-
|
|
3245
|
-
|
|
3246
|
-
|
|
3247
|
-
|
|
3248
|
-
|
|
3249
|
-
|
|
3250
|
-
|
|
3251
|
-
|
|
3252
|
-
|
|
3253
|
-
|
|
3254
|
-
|
|
3339
|
+
| destination_root_id | string | | | | | | Y | The file ID of the destination root directory.<br/>Required when using Bearer token auth for the destination node. |
|
3340
|
+
| dgram_size | integer | Y | Y | Y | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {integer}) |
|
3341
|
+
| direction | string | Y | Y | Y | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode=(conversion){enum}) |
|
3342
|
+
| exclude_newer_than | string | Y | Y | Y | Y | Y | Y | Exclude files, but not directories, from the transfer if they are newer than the specified number of seconds added to the source computer's epoch.<br/>e.g. "-86400" for newer than a day back.<br/>(--exclude-newer-than={string}) |
|
3343
|
+
| exclude_older_than | string | Y | Y | Y | Y | Y | Y | Exclude files, but not directories, from the transfer if they are older than the specified number of seconds added to the source computer's epoch.<br/>e.g. "-86400" for older than a day back.<br/>(--exclude-older-than={string}) |
|
3344
|
+
| fail_bad_filepass | boolean | Y | | | | | Y | Fail on bad file decryption passphrase.<br/>(--fail-bad-filepass) |
|
3345
|
+
| fasp_port | integer | Y | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {integer}) |
|
3346
|
+
| fasp_proxy | object | | | | | | Y | Proxy for communications between the remote server and the (local) client. |
|
3347
|
+
| fasp_url | string | | | | | | | Only used in Faspex. |
|
3348
|
+
| file_checksum | string | Y | | | | Y | | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(--file-checksum={enum}) |
|
3349
|
+
| http_fallback | boolean<br/>string | Y | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){boolean\|string}) |
|
3350
|
+
| http_fallback_port | integer | Y | Y | Y | Y | Y | Y | Specifies http port when no cipher is used<br/>(-t {integer}) |
|
3351
|
+
| https_fallback_port | integer | Y | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {integer}) |
|
3352
|
+
| icos | object | | | | | | Y | Configuration parameters for IBM Cloud Object Storage (ICOS). |
|
3353
|
+
| keepalive | boolean | Y | | | | | Y | The session is running in persistent session mode.<br/>(--keepalive) |
|
3354
|
+
| lock_min_rate | boolean | | Y | | | | | TODO: remove ? |
|
3355
|
+
| lock_min_rate_kbps | boolean | | Y | | | | Y | If true, lock the minimum transfer rate to the value set for min_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps. |
|
3356
|
+
| lock_rate_policy | boolean | | Y | | | | Y | If true, lock the rate policy to the default value. |
|
3357
|
+
| lock_target_rate | boolean | | Y | | | | | TODO: remove ? |
|
3358
|
+
| lock_target_rate_kbps | boolean | | Y | | | | Y | If true, lock the target transfer rate to the default value set for target_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps. |
|
3359
|
+
| min_rate_cap_kbps | integer | | Y | | | | Y | The highest minimum rate that an incoming transfer can request, in kilobits per second.<br/>Client minimum rate requests that exceed the minimum rate cap are ignored.<br/>The default value of unlimited applies no cap to the minimum rate. (Default: 0) |
|
3360
|
+
| min_rate_kbps | integer | Y | Y | Y | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {integer}) |
|
3361
|
+
| move_after_transfer | string | Y | | | | Y | Y | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer={string}) |
|
3362
|
+
| multi_session | integer | Y | Y | Y | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/>(<special>) |
|
3363
|
+
| multi_session_threshold | integer | Y | | | | Y | Y | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold={integer}) |
|
3364
|
+
| obfuscate_file_names | boolean | | | | Y | | | HTTP Gateway obfuscates file names when set to true. |
|
3365
|
+
| overwrite | string | Y | Y | Y | Y | Y | Y | Overwrite files at the destination with source files of the same name based on the policy:<br/>- always – Always overwrite the file.<br/>- never – Never overwrite the file. If the destination contains partial files that are older or the same as the source files and resume is enabled, the partial files resume transfer. Partial files with checksums or sizes that differ from the source files are not overwritten.<br/>- diff – Overwrite the file if it is different from the source, depending on the compare method (default is size). If the destination is object storage, diff has the same effect as always. If resume is not enabled, partial files are overwritten if they are different from the source, otherwise they are skipped. If resume is enabled, only partial files with different sizes or checksums from the source are overwritten; otherwise, files resume. <br/>- diff+older – Overwrite the file if it is older and different from the source, depending on the compare method (default is size). If resume is not enabled, partial files are overwritten if they are older and different from the source, otherwise they are skipped. If resume is enabled, only partial files that are different and older than the source are overwritten, otherwise they are resumed. <br/>- older – Overwrite the file if its timestamp is older than the source timestamp. <br/>If you set an overwrite policy of diff or diff+older, difference is determined by the value set for resume_policy:<br/>"none" - the source and destination files are always considered different and the destination file is always overwritten<br/>"attributes" - the source and destination files are compared based on file attributes <br/>"sparse_checksum" - the source and destination files are compared based on sparse checksums, (currently file size)<br/>"full_checksum" - the source and destination files are compared based on full checksums <br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite={enum}) |
|
3366
|
+
| password | string | | | | | Y | | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only. |
|
3255
3367
|
| paths | array | Y | Y | Y | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional).<br/>(<special>) |
|
3256
|
-
| precalculate_job_size |
|
3257
|
-
| preserve_access_time |
|
3258
|
-
| preserve_acls | string | Y | |
|
3259
|
-
| preserve_creation_time |
|
3260
|
-
| preserve_extended_attrs | string |
|
3261
|
-
| preserve_file_owner_gid |
|
3262
|
-
| preserve_file_owner_uid |
|
3263
|
-
| preserve_modification_time |
|
3264
|
-
| preserve_remote_acls | string | Y | |
|
3265
|
-
|
|
3266
|
-
|
|
3267
|
-
|
|
3268
|
-
|
|
3269
|
-
|
|
3270
|
-
|
|
3271
|
-
|
|
3272
|
-
|
|
3368
|
+
| precalculate_job_size | boolean | Y | Y | Y | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
|
3369
|
+
| preserve_access_time | boolean | Y | Y | Y | Y | Y | Y | Preserve the source-file access timestamps at the destination.<br/>Because source access times are updated by the transfer operation, the timestamp that is preserved is the one just before to the transfer.<br/>(--preserve-access-time) |
|
3370
|
+
| preserve_acls | string | Y | | | | | Y | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls={enum}) |
|
3371
|
+
| preserve_creation_time | boolean | Y | Y | Y | Y | Y | Y | (Windows only) Preserve source-file creation timestamps at the destination.<br/>Only Windows systems retain information about creation time.<br/>If the destination is not a Windows computer, this option is ignored.<br/>(--preserve-creation-time) |
|
3372
|
+
| preserve_extended_attrs | string | Y | | | | | Y | Preserve the extended attributes.<br/>Allowed values: none, native, metafile<br/>(--preserve-xattrs={enum}) |
|
3373
|
+
| preserve_file_owner_gid | boolean | Y | | | | | Y | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
|
3374
|
+
| preserve_file_owner_uid | boolean | Y | | | | | Y | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
|
3375
|
+
| preserve_modification_time | boolean | Y | Y | Y | Y | Y | Y | Set the modification time, the last time a file or directory was modified (written), of a transferred file to the modification of the source file or directory.<br/>Preserve source-file modification timestamps at the destination.<br/>(--preserve-modification-time) |
|
3376
|
+
| preserve_remote_acls | string | Y | | | | | Y | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls={enum}) |
|
3377
|
+
| preserve_remote_extended_attrs | string | Y | | | | | Y | Preserve remote extended attributes.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-xattrs={enum}) |
|
3378
|
+
| preserve_source_access_time | boolean | Y | | | | | Y | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
|
3379
|
+
| preserve_times | boolean | Y | | | | Y | Y | Preserve file timestamps.<br/>(-p {boolean}) |
|
3380
|
+
| proxy | string | Y | | | | | | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy={string}) |
|
3381
|
+
| rate_policy | string | Y | Y | Y | Y | Y | Y | The transfer rate policy to use when sharing bandwidth. Allowable values:<br/>- high : When sharing bandwidth, transfer at twice the rate of a transfer using a fair policy.<br/>- fair : (Default) Share bandwidth equally with other traffic.<br/>- low : Use only unused bandwidth.<br/>- fixed : Transfer at the target rate, regardless of the actual network capacity. Do not share bandwidth. Aspera recommends that you do not use this setting except under special circumstances, otherwise the destination storage can be damaged. <br/>Allowed values: low, fair, high, fixed<br/>(--policy={enum}) |
|
3382
|
+
| rate_policy_allowed | string | | Y | | | | | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed |
|
3383
|
+
| read_threads | integer | | | | | | | ascp4 only |
|
3384
|
+
| remote_access_key | string | | | | | | Y | The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node. |
|
3385
|
+
| remote_host | string | Y | Y | Y | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host={string}) |
|
3273
3386
|
| remote_password | string | Y | Y | Y | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
|
3274
|
-
| remote_user | string | Y | Y | Y | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user
|
3275
|
-
| remove_after_transfer |
|
3276
|
-
| remove_empty_directories |
|
3277
|
-
|
|
3278
|
-
|
|
3279
|
-
|
|
3280
|
-
|
|
3281
|
-
|
|
3282
|
-
|
|
3283
|
-
|
|
3284
|
-
|
|
3285
|
-
|
|
3286
|
-
|
|
3287
|
-
|
|
3288
|
-
|
|
3289
|
-
|
|
3290
|
-
|
|
3291
|
-
|
|
3292
|
-
|
|
3293
|
-
|
|
3294
|
-
|
|
3387
|
+
| remote_user | string | Y | Y | Y | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user={string}) |
|
3388
|
+
| remove_after_transfer | boolean | Y | | | | Y | Y | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
|
3389
|
+
| remove_empty_directories | boolean | Y | | | | Y | Y | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
|
3390
|
+
| remove_empty_source_dir | boolean | | | | | | Y | Remove empty source subdirectories and remove the source directory itself, if empty. |
|
3391
|
+
| remove_empty_source_directory | boolean | Y | | | | | | Remove empty source subdirectories and remove the source directory itself, if empty.<br/>(--remove-empty-source-directory) |
|
3392
|
+
| remove_skipped | boolean | Y | Y | | | Y | | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
|
3393
|
+
| resume_policy | string | Y | Y | Y | Y | Y | Y | If a transfer is interrupted or fails to finish, this policy directs the transfer to resume without retransferring the files. Allowable values:<br/>- none : Always re-transfer the entire file<br/>- attrs : Compare file attributes and resume if they match, and re-transfer if they do not<br/>- sparse_csum : Compare file attributes and the sparse file checksums; resume if they match, and re-transfer if they do not<br/>- full_csum : Compare file attributes and the full file checksums; resume if they match, and re-transfer if they do not. <br/>Note: transferd uses values: attributes, sparse_checksum, full_checksum.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
|
3394
|
+
| retry_duration | integer<br/>string | | | | | | Y | Specifies how long to wait before retrying transfer. (e.g. "5min") |
|
3395
|
+
| save_before_overwrite | boolean | Y | | | | Y | Y | If a transfer would result in an existing file <filename>.<ext> being overwritten, move that file to <filename>.yyyy.mm.dd.hh.mm.ss.index.<ext> (where index is set to 1 at the beginning of each new second and incremented for each file saved in this manner during the same second) in the same directory before writing the new file.<br/>File attributes are maintained in the renamed file.<br/>(--save-before-overwrite) |
|
3396
|
+
| skip_duplicate_check | boolean | Y | | | | | Y | Don't check for duplicate files at the destination.<br/>(--skip-dir-traversal-dupes) |
|
3397
|
+
| skip_special_files | boolean | Y | | | | | Y | All assets other than files, directories and symbolic links are considered special. A transfer will fail if the user attempts to transfer special assets. If true, ascp skips special assets and proceeds with the transfer of all other assets. <br/>(--skip-special-files) |
|
3398
|
+
| source_root | string | Y | Y | Y | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64=(conversion){string}) |
|
3399
|
+
| source_root_id | string | | | | | Y | Y | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
|
3400
|
+
| src_base | string | Y | | | | Y | Y | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64=(conversion){string}) |
|
3401
|
+
| src_base64 | string | Y | | | | | Y | The folder name below which the directory structure is preserved (base64 encoded).<br/>(--src-base64={string}) |
|
3402
|
+
| ssh_args | array | Y | | | | | Y | Array of arguments to pass to SSH. Use with caution.<br/>(-i {array}) |
|
3403
|
+
| ssh_port | integer | Y | Y | Y | Y | Y | Y | Specifies SSH (TCP) port.<br/>(-P {integer}) |
|
3404
|
+
| ssh_private_key | string | Y | | | | | Y | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
|
3405
|
+
| ssh_private_key_passphrase | string | Y | | | | | Y | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
|
3406
|
+
| ssh_private_key_path | string | Y | | | | | Y | Path to private key for SSH.<br/>(-i {string}) |
|
3407
|
+
| sshfp | string | Y | Y | Y | Y | Y | Y | Check it against server SSH host key fingerprint.<br/>(--check-sshfp={string}) |
|
3408
|
+
| symlink_policy | string | Y | Y | Y | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links={enum}) |
|
3409
|
+
| tags | object | Y | Y | Y | Y | Y | Y | Metadata for transfer as JSON. Key `aspera` is reserved. Key `aspera.xfer_retry` specifies a retry timeout for node api initiated transfers.<br/>(--tags64=(conversion){object}) |
|
3410
|
+
| tags64 | string | Y | | | | | Y | Metadata for transfer as JSON. Key `aspera` is reserved. Key `aspera.xfer_retry` specifies a retry timeout for node api initiated transfers.<br/>(--tags64={string}) |
|
3411
|
+
| target_rate_cap_kbps | integer | | Y | | | | Y | Maximum target rate for incoming transfers, in kilobits per second. Returned by upload/download_setup node API. |
|
3412
|
+
| target_rate_kbps | integer | Y | Y | Y | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {integer}) |
|
3413
|
+
| target_rate_percentage | string | | | | | | | TODO: remove ? |
|
3414
|
+
| title | string | | Y | | | Y | Y | Title of the transfer. |
|
3295
3415
|
| token | string | Y | Y | Y | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
|
3296
|
-
| use_ascp4 |
|
3297
|
-
| use_system_ssh | string |
|
3298
|
-
| write_threads |
|
3299
|
-
| wss_enabled |
|
3300
|
-
| wss_port |
|
3301
|
-
| xfer_max_retries |
|
3416
|
+
| use_ascp4 | boolean | Y | | | | Y | Y | Specify version of protocol. Do not use ascp4.<br/>(<special>) |
|
3417
|
+
| use_system_ssh | string | Y | | | | | Y | TODO, comment...<br/>(-SSH {string}) |
|
3418
|
+
| write_threads | integer | | | | | | | ascp4 only |
|
3419
|
+
| wss_enabled | boolean | Y | Y | Y | Y | Y | Y | Server has Web Socket service enabled<br/>(<special>) |
|
3420
|
+
| wss_port | integer | Y | Y | Y | Y | Y | Y | TCP port used for websocket service feed<br/>(<special>) |
|
3421
|
+
| xfer_max_retries | integer | | | | | Y | | Maximum number of retries, for node API initiated transfers. Shall not exceed aspera.conf `transfer_manager_max_retries` (default 5). |
|
3302
3422
|
|
3303
3423
|
#### Destination folder for transfers
|
3304
3424
|
|
@@ -3310,18 +3430,18 @@ The destination folder is set by `ascli` by default to:
|
|
3310
3430
|
It is specified by the [**transfer-spec**](#transfer-specification) parameter `destination_root`.
|
3311
3431
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
3312
3432
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
3313
|
-
`--to-folder=<path
|
3433
|
+
`--to-folder=<path>`.
|
3314
3434
|
|
3315
3435
|
#### List of files for transfers
|
3316
3436
|
|
3317
3437
|
When uploading, downloading or sending files, the user must specify the list of files to transfer.
|
3318
3438
|
|
3319
|
-
By default the list of files to transfer is simply provided on the command line.
|
3439
|
+
By default, the list of files to transfer is simply provided on the command line.
|
3320
3440
|
|
3321
3441
|
The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
|
3322
3442
|
The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
|
3323
3443
|
|
3324
|
-
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer
|
3444
|
+
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer daemon...).
|
3325
3445
|
So, eventually, the list of files to transfer is provided to the transfer agent using the [**transfer-spec**](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
|
3326
3446
|
The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
|
3327
3447
|
|
@@ -3430,16 +3550,16 @@ In the following table source folder `d3` contains 2 files: `f1` and `d4/f2`.
|
|
3430
3550
|
|
3431
3551
|
| Source files | Destination | Folders on Dest. |`create_dir`| Destination Files |
|
3432
3552
|
|--------------|-------------|-------------------|------------|-----------------------------|
|
3433
|
-
| f1
|
3434
|
-
| f1
|
3435
|
-
| f1
|
3436
|
-
| f1
|
3437
|
-
| f1 f2
|
3438
|
-
| d3
|
3439
|
-
| f1
|
3440
|
-
| f1 f2
|
3441
|
-
| d1/f1 d2/f2
|
3442
|
-
| d3
|
3553
|
+
| `f1` | `d/f` | - | false | Error: `d` does not exist. |
|
3554
|
+
| `f1` | `d/f` | `d` | false | `d/f` (renamed) |
|
3555
|
+
| `f1` | `d/f/.` | `d` | false | `d/f` (renamed) |
|
3556
|
+
| `f1` | `d/f` | `d/f` | false | `d/f/f1` |
|
3557
|
+
| `f1` `f2` | `d` | `d` | false | `d/f1` `d/f2` |
|
3558
|
+
| `d3` | `d` | - | false | `d/f1` `d/f2` (renamed) |
|
3559
|
+
| `f1` | `d` | - | true | `d/f1` |
|
3560
|
+
| `f1` `f2` | `d` | - | true | `d/f1` `d/f2` |
|
3561
|
+
| `d1/f1` `d2/f2` | `d` | - | true | `d/f1` `d/f2` |
|
3562
|
+
| `d3` | `d` | - | true | `d/d3/f1` `d/d3/d4/f2` |
|
3443
3563
|
|
3444
3564
|
If a file par list is provided then it is possible to rename or specify a different destination folder for each source (relative to the destination).
|
3445
3565
|
|
@@ -3447,7 +3567,7 @@ If transfer spec has a `src_base`, it has the side effect that the simple source
|
|
3447
3567
|
|
3448
3568
|
| Source files | Destination |`src_base`| Destination Files |
|
3449
3569
|
|-------------------|-------------|----------|-----------------------------|
|
3450
|
-
| d1/d2/f2 d1/d3/f3 | d
|
3570
|
+
| `d1/d2/f2` `d1/d3/f3` | `d` | `d1` | `d/d2/f2` `d/d3/f3` |
|
3451
3571
|
|
3452
3572
|
Advanced Example: Send files `./file1` and `./folder2/files2` to server (e.g. `/Upload`) and keep the original file names and folders, i.e. send `file1` to `/Upload/file1` and `files2` to `/Upload/folder2/files2`.
|
3453
3573
|
|
@@ -3497,7 +3617,7 @@ Activating CSEAR consists in using transfer spec parameters:
|
|
3497
3617
|
- `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
|
3498
3618
|
- `content_protection_password` : the passphrase to be used.
|
3499
3619
|
|
3500
|
-
Example: parameter to download a
|
3620
|
+
Example: parameter to download a Faspex package and decrypt on the fly
|
3501
3621
|
|
3502
3622
|
```bash
|
3503
3623
|
--ts=@json:'{"content_protection":"decrypt","content_protection_password":"my_password_here"}'
|
@@ -3517,7 +3637,7 @@ Example: parameter to download a faspex package and decrypt on the fly
|
|
3517
3637
|
--ts=@json:'{"ssh_port":33002}'
|
3518
3638
|
```
|
3519
3639
|
|
3520
|
-
- Force
|
3640
|
+
- Force HTTP fallback mode:
|
3521
3641
|
|
3522
3642
|
```bash
|
3523
3643
|
--ts=@json:'{"http_fallback":"force"}'
|
@@ -3531,12 +3651,10 @@ Example: parameter to download a faspex package and decrypt on the fly
|
|
3531
3651
|
|
3532
3652
|
### Transfer progress bar
|
3533
3653
|
|
3534
|
-
File transfer operations are monitored and a progress bar is displayed on the terminal if option `progress_bar` (`Bool`) is set to `yes` (default if the output is a terminal).
|
3654
|
+
File transfer operations are monitored, and a progress bar is displayed on the terminal if option `progress_bar` (`Bool`) is set to `yes` (default if the output is a terminal).
|
3535
3655
|
|
3536
3656
|
The same progress bar is used for any type of transfer, using `ascp`, server to server, using HTTPS, etc...
|
3537
3657
|
|
3538
|
-
To display the native progress bar of `ascp`, use `--progress-bar=no --transfer-info=@json:'{"quiet":false}'`.
|
3539
|
-
|
3540
3658
|
### Scheduler
|
3541
3659
|
|
3542
3660
|
It is useful to configure automated scheduled execution.
|
@@ -3550,17 +3668,17 @@ It can be configured:
|
|
3550
3668
|
|
3551
3669
|
- Using utility [`schtasks.exe`](https://learn.microsoft.com/fr-fr/windows-server/administration/windows-commands/schtasks-create)
|
3552
3670
|
|
3553
|
-
- Using
|
3671
|
+
- Using Powershell function [`scheduletasks`](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
3554
3672
|
|
3555
3673
|
- Using `taskschd.msc` (UI)
|
3556
3674
|
|
3557
3675
|
#### Unix-like Scheduler
|
3558
3676
|
|
3559
|
-
Unix-like systems (Linux, ...) provide cron
|
3677
|
+
Unix-like systems (Linux, ...) provide `cron`, configured using a [`crontab`](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
3560
3678
|
|
3561
3679
|
Linux also provides `anacron`, if tasks are hourly or daily.
|
3562
3680
|
|
3563
|
-
For example, on Linux it is convenient to create a wrapping script, e.g. `cron_ascli` that will
|
3681
|
+
For example, on Linux it is convenient to create a wrapping script, e.g. `cron_ascli` that will set up the environment (e.g. Ruby) to properly start `ascli`:
|
3564
3682
|
|
3565
3683
|
```bash
|
3566
3684
|
#!/bin/bash
|
@@ -3582,13 +3700,70 @@ crontab<<EOF
|
|
3582
3700
|
EOF
|
3583
3701
|
```
|
3584
3702
|
|
3585
|
-
> **Note:** Logging options are kept here in the cron file instead of configuration file to allow execution on command line with output on command line.
|
3703
|
+
> **Note:** Logging options are kept here in the `cron` file instead of configuration file to allow execution on command line with output on command line.
|
3704
|
+
|
3705
|
+
### Running as service
|
3706
|
+
|
3707
|
+
Some commands result in `ascli` running as a server, listening on a port.
|
3708
|
+
In this case, it is usually desirable to run `ascli` as a service.
|
3709
|
+
On Linux, typically, [SystemD](https://systemd.io/) is used.
|
3710
|
+
|
3711
|
+
A convenient way is to write a startup script, and run this script as a service.
|
3712
|
+
|
3713
|
+
Let's give a base name for our service: `my_ascli_svc`
|
3714
|
+
|
3715
|
+
The startup script can be simply the `ascli` command line, for example: `/usr/local/bin/start_my_ascli_svc.sh`:
|
3716
|
+
|
3717
|
+
```bash
|
3718
|
+
#!/bin/bash
|
3719
|
+
set -e
|
3720
|
+
echo "Starting my_ascli_svc at $(date)"
|
3721
|
+
# set PATH to find ascli, and other environment setup
|
3722
|
+
exec ascli .....
|
3723
|
+
```
|
3724
|
+
|
3725
|
+
And make this script executable:
|
3726
|
+
|
3727
|
+
```bash
|
3728
|
+
chmod a+x /usr/local/bin/start_my_ascli_svc.sh
|
3729
|
+
```
|
3730
|
+
|
3731
|
+
Create a startup file: `/etc/systemd/system/my_ascli_svc.service` :
|
3732
|
+
|
3733
|
+
```ini
|
3734
|
+
[Unit]
|
3735
|
+
Description=My ascli daemon
|
3736
|
+
After=network.target
|
3737
|
+
|
3738
|
+
[Service]
|
3739
|
+
ExecStart=/usr/local/bin/start_my_ascli_svc.sh
|
3740
|
+
Restart=on-failure
|
3741
|
+
RestartSec=15
|
3742
|
+
User=xfer
|
3743
|
+
# Optional, specify a working directory
|
3744
|
+
# WorkingDirectory=/path/to/working/dir
|
3745
|
+
# Optional, redirect output
|
3746
|
+
StandardOutput=journal
|
3747
|
+
StandardError=journal
|
3748
|
+
|
3749
|
+
[Install]
|
3750
|
+
WantedBy=multi-user.target
|
3751
|
+
```
|
3752
|
+
|
3753
|
+
Then enable and start with:
|
3754
|
+
|
3755
|
+
```bash
|
3756
|
+
sudo systemctl daemon-reload
|
3757
|
+
sudo systemctl enable --now my_ascli_svc.service
|
3758
|
+
systemctl status my_ascli_svc.service
|
3759
|
+
journalctl -u my_ascli_svc.service
|
3760
|
+
```
|
3586
3761
|
|
3587
3762
|
### Locking for exclusive execution
|
3588
3763
|
|
3589
3764
|
In some cases one needs to ensure that `ascli` is not executed several times in parallel.
|
3590
3765
|
|
3591
|
-
When `ascli` is executed automatically on a schedule basis, one generally desires that a new execution is not started if a previous execution is still running because an
|
3766
|
+
When `ascli` is executed automatically on a schedule basis, one generally desires that a new execution is not started if a previous execution is still running because an ongoing operation may last longer than the scheduling period:
|
3592
3767
|
|
3593
3768
|
- Executing instances may pile-up and kill the system
|
3594
3769
|
- The same file may be transferred by multiple instances at the same time.
|
@@ -3597,7 +3772,7 @@ When `ascli` is executed automatically on a schedule basis, one generally desire
|
|
3597
3772
|
Usually the OS native scheduler already provides some sort of protection against parallel execution:
|
3598
3773
|
|
3599
3774
|
- The Windows scheduler does this by default
|
3600
|
-
- Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
|
3775
|
+
- Linux `cron` can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
|
3601
3776
|
|
3602
3777
|
```bash
|
3603
3778
|
/usr/bin/flock -w 0 /var/cron.lock ascli ...
|
@@ -3658,10 +3833,9 @@ It is a pseudo file system allowing generation of file data without actual stora
|
|
3658
3833
|
|
3659
3834
|
### `faux:` for testing
|
3660
3835
|
|
3661
|
-
This is an extract of the man page of `ascp`.
|
3662
|
-
This feature is a feature of `ascp`, not `ascli`.
|
3663
|
-
|
3664
3836
|
This adapter can be used to simulate a file or a directory.
|
3837
|
+
This is a feature of `ascp`, not `ascli`.
|
3838
|
+
The following is an extract of the man page of `ascp`.
|
3665
3839
|
|
3666
3840
|
To discard data at the destination, the destination argument is set to `faux://`.
|
3667
3841
|
|
@@ -3678,7 +3852,7 @@ where:
|
|
3678
3852
|
|
3679
3853
|
> **Note:** Characters `?` and `&` are shell special characters (wildcard and background), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
|
3680
3854
|
|
3681
|
-
For all sizes, a suffix can be added (case
|
3855
|
+
For all sizes, a suffix can be added (case-insensitive) to the size: k, m, g, t, p, e (values are power of 2, e.g. 1M is 2<sup>20</sup>, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8\*2<sup>60</sup>. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is `faux://`.
|
3682
3856
|
|
3683
3857
|
To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
|
3684
3858
|
|
@@ -3704,17 +3878,18 @@ The sequence parameter is applied as follows:
|
|
3704
3878
|
|
3705
3879
|
- If `seq` is `random` then each file size is:
|
3706
3880
|
|
3707
|
-
- size +/- (inc * rand())
|
3881
|
+
- `size +/- (inc * rand())`
|
3708
3882
|
- Where rand is a random number between 0 and 1
|
3709
3883
|
- Note that file size must not be negative, inc will be set to size if it is greater than size
|
3710
|
-
- Similarly, overall file size must be less than 8
|
3884
|
+
- Similarly, overall file size must be less than 8\*2<sup>60</sup>.
|
3885
|
+
If size + inc is greater, inc will be reduced to limit size + inc to 7\*2<sup>60</sup>.
|
3711
3886
|
|
3712
3887
|
- If `seq` is `sequential` then each file size is:
|
3713
3888
|
|
3714
3889
|
- `size + ((file_index - 1) * inc)`
|
3715
3890
|
- Where first file is index 1
|
3716
3891
|
- So file1 is `size` bytes, file2 is `size + inc` bytes, file3 is `size + inc * 2` bytes, etc.
|
3717
|
-
- As with `random`, `inc` will be adjusted if `size + (count * inc)` is not less
|
3892
|
+
- As with `random`, `inc` will be adjusted if `size + (count * inc)` is not less than 8\*2<sup>60</sup>.
|
3718
3893
|
|
3719
3894
|
Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
|
3720
3895
|
|
@@ -3732,18 +3907,24 @@ ascli server upload faux:///myfile\?20g --to-folder=/Upload
|
|
3732
3907
|
ascli server upload /tmp/sample --to-folder=faux://
|
3733
3908
|
```
|
3734
3909
|
|
3735
|
-
- Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the
|
3910
|
+
- Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the base name of each file being `testfile` to /Upload
|
3736
3911
|
|
3737
3912
|
```bash
|
3738
3913
|
ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
|
3739
3914
|
```
|
3740
3915
|
|
3916
|
+
- Upload a faux directory `mydir` containing 1000 files, of size 1 byte, with the base name of each file being `testfile` to /Upload
|
3917
|
+
|
3918
|
+
```bash
|
3919
|
+
ascli server upload "faux:///mydir?file=testfile&count=1000&size=1" --to-folder=/Upload
|
3920
|
+
```
|
3921
|
+
|
3741
3922
|
### Usage
|
3742
3923
|
|
3743
3924
|
```text
|
3744
3925
|
ascli -h
|
3745
3926
|
NAME
|
3746
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3927
|
+
ascli -- a command line tool for Aspera Applications (v4.23.0)
|
3747
3928
|
|
3748
3929
|
SYNOPSIS
|
3749
3930
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -3814,21 +3995,21 @@ OPTIONS: global
|
|
3814
3995
|
--default=ENUM Wizard: set as default configuration for specified plugin (also: update): no, [yes]
|
3815
3996
|
--test-mode=ENUM Wizard: skip private key check step: [no], yes
|
3816
3997
|
--key-path=VALUE Wizard: path to private key for JWT
|
3817
|
-
--ascp-path=VALUE Path to ascp
|
3818
|
-
--use-product=VALUE Use ascp from specified product
|
3819
|
-
--sdk-url=VALUE URL to get Aspera Transfer
|
3820
|
-
--locations-url=VALUE URL to get locations of Aspera Transfer Daemon
|
3821
|
-
--sdk-folder=VALUE SDK folder path
|
3998
|
+
--ascp-path=VALUE Ascp: Path to ascp
|
3999
|
+
--use-product=VALUE Ascp: Use ascp from specified product
|
4000
|
+
--sdk-url=VALUE Ascp: URL to get Aspera Transfer Executables
|
4001
|
+
--locations-url=VALUE Ascp: URL to get locations of Aspera Transfer Daemon
|
4002
|
+
--sdk-folder=VALUE Ascp: SDK folder path
|
3822
4003
|
--progress-bar=ENUM Display progress bar: [no], yes
|
3823
|
-
--smtp=VALUE SMTP configuration (Hash)
|
3824
|
-
--notify-to=VALUE Email
|
3825
|
-
--notify-template=VALUE Email ERB template for notification of transfers
|
3826
|
-
--insecure=ENUM Do not validate any
|
3827
|
-
--ignore-certificate=VALUE Do not validate
|
3828
|
-
--silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
|
3829
|
-
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
3830
|
-
--http-options=VALUE Options for HTTP/S socket (Hash)
|
3831
|
-
--http-proxy=VALUE URL for
|
4004
|
+
--smtp=VALUE Email: SMTP configuration (Hash)
|
4005
|
+
--notify-to=VALUE Email: Recipient for notification of transfers
|
4006
|
+
--notify-template=VALUE Email: ERB template for notification of transfers
|
4007
|
+
--insecure=ENUM HTTP/S: Do not validate any certificate: [no], yes
|
4008
|
+
--ignore-certificate=VALUE HTTP/S: Do not validate certificate for these URLs (Array)
|
4009
|
+
--silent-insecure=ENUM HTTP/S: Issue a warning if certificate is ignored: no, [yes]
|
4010
|
+
--cert-stores=VALUE HTTP/S: List of folder with trusted certificates (Array, String)
|
4011
|
+
--http-options=VALUE HTTP/S: Options for HTTP/S socket (Hash)
|
4012
|
+
--http-proxy=VALUE HTTP/S: URL for proxy with optional credentials (String)
|
3832
4013
|
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
3833
4014
|
--fpac=VALUE Proxy auto configuration script
|
3834
4015
|
--proxy-credentials=VALUE HTTP proxy credentials for fpac: user, password (Array)
|
@@ -3852,7 +4033,7 @@ OPTIONS:
|
|
3852
4033
|
|
3853
4034
|
|
3854
4035
|
COMMAND: node
|
3855
|
-
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health
|
4036
|
+
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse cat central delete download events health info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync telemetry transfer transport upload watch_folder
|
3856
4037
|
OPTIONS:
|
3857
4038
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3858
4039
|
--username=VALUE User's name to log in
|
@@ -4011,6 +4192,7 @@ OPTIONS:
|
|
4011
4192
|
--workspace=VALUE Name of workspace (String, NilClass)
|
4012
4193
|
--new-user-option=VALUE New user creation option for unknown package recipients (Hash)
|
4013
4194
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
4195
|
+
--package-folder=VALUE Field of package to use as folder name, or @none: (String, NilClass)
|
4014
4196
|
|
4015
4197
|
|
4016
4198
|
COMMAND: server
|
@@ -4043,16 +4225,16 @@ OPTIONS:
|
|
4043
4225
|
|
4044
4226
|
Bulk creation and deletion of resources are possible using option `bulk` (`yes`,`no`(default)).
|
4045
4227
|
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended-value-syntax).
|
4046
|
-
This option is available only for some
|
4228
|
+
This option is available only for some resources: if you need it: try and see if the entities you try to create or delete support this option.
|
4047
4229
|
|
4048
4230
|
### Option: `query`
|
4049
4231
|
|
4050
|
-
The `query` option can generally be used to add URL parameters to commands that list
|
4232
|
+
The `query` option can generally be used to add URL parameters to commands that list resources.
|
4051
4233
|
It takes either a `Hash` or an `Array`, corresponding to key/value pairs that appear in the query part of request.
|
4052
4234
|
|
4053
4235
|
For example: `--query=@json:'{"p1":"v1","p2":"v2"}'` leads to query: `?p1=v1&p2=v2`.
|
4054
4236
|
|
4055
|
-
If the same parameter needs to be provided several times, then it's possible as well to provide an Array or 2-element Array
|
4237
|
+
If the same parameter needs to be provided several times, then it's possible as well to provide an `Array` or 2-element `Array`: `--query=@json:'[["p1":,"v1"],["p2":"v2"]]'` leads to the same result as previously.
|
4056
4238
|
|
4057
4239
|
If PHP's style array is used, then one can use either:
|
4058
4240
|
|
@@ -4088,9 +4270,9 @@ Most plugins will take the URL option: `url` to identify their location.
|
|
4088
4270
|
|
4089
4271
|
REST APIs of Aspera legacy applications (Aspera Node, Faspex 4, Shares, Console, Orchestrator) use simple username/password authentication: HTTP Basic Authentication using options: `username` and `password`.
|
4090
4272
|
|
4091
|
-
Aspera on Cloud and Faspex 5 rely on
|
4273
|
+
Aspera on Cloud and Faspex 5 rely on OAuth.
|
4092
4274
|
|
4093
|
-
By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
4275
|
+
By default, plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
4094
4276
|
|
4095
4277
|
```bash
|
4096
4278
|
ascli --show-config --fields=plugin_folder
|
@@ -4112,17 +4294,18 @@ ascli --plugin-folder=. foo
|
|
4112
4294
|
|
4113
4295
|
## Plugin: `aoc`: IBM Aspera on Cloud
|
4114
4296
|
|
4115
|
-
Aspera on Cloud API requires the use of
|
4297
|
+
Aspera on Cloud API requires the use of OAuth v2 mechanism for authentication (HTTP Basic authentication is not supported).
|
4116
4298
|
|
4117
4299
|
It is recommended to use the wizard to set it up, although manual configuration is also possible.
|
4118
4300
|
|
4119
4301
|
### Configuration: Using Wizard
|
4120
4302
|
|
4121
|
-
`ascli` provides a configuration wizard.
|
4303
|
+
`ascli` provides a configuration [wizard](#wizard).
|
4122
4304
|
|
4123
4305
|
The wizard guides you through the steps to create a new configuration preset for Aspera on Cloud.
|
4124
4306
|
|
4125
|
-
The first
|
4307
|
+
The first optional argument is the URL of your Aspera on Cloud instance, e.g. `https://_your_instance_.ibmaspera.com` or simply the organization name, and a second optional argument can also be provided to specify the plugin name, e.g. `aoc` for Aspera on Cloud.
|
4308
|
+
If optional arguments are not provided, the wizard will ask interactively and try to detect the application.
|
4126
4309
|
|
4127
4310
|
Here is a sample invocation :
|
4128
4311
|
|
@@ -4159,6 +4342,13 @@ ascli config wizard _your_instance_ aoc
|
|
4159
4342
|
|
4160
4343
|
> **Note:** In above example, replace `_your_instance_` with the first part of your actual AoC URL: `https://_your_instance_.ibmaspera.com`.
|
4161
4344
|
|
4345
|
+
After successful completion of the wizard, a new configuration preset is created, and set as default for the `aoc` plugin.
|
4346
|
+
This can be verified with command:
|
4347
|
+
|
4348
|
+
```bash
|
4349
|
+
ascli config preset over
|
4350
|
+
```
|
4351
|
+
|
4162
4352
|
### Configuration: Using manual setup
|
4163
4353
|
|
4164
4354
|
> **Note:** If you used the wizard (recommended): skip this section.
|
@@ -4167,17 +4357,17 @@ ascli config wizard _your_instance_ aoc
|
|
4167
4357
|
|
4168
4358
|
Several types of OAuth authentication are supported:
|
4169
4359
|
|
4170
|
-
- JSON Web Token (JWT) : authentication is secured by a private key (recommended
|
4360
|
+
- JSON Web Token (JWT) : authentication is secured by a private key (recommended)
|
4171
4361
|
- Web based authentication : authentication is made by user using a browser
|
4172
|
-
- URL Token : external
|
4362
|
+
- URL Token : external user's authentication with URL tokens (public links)
|
4173
4363
|
|
4174
4364
|
The authentication method is controlled by option `auth`.
|
4175
4365
|
|
4176
4366
|
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset) for Aspera on Cloud](#configuration-for-aspera-on-cloud).
|
4177
4367
|
|
4178
|
-
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
|
4368
|
+
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (`auth=jwt`) in addition to Client Registration.
|
4179
4369
|
|
4180
|
-
In
|
4370
|
+
In OAuth, a **Bearer** token is generated to authenticate REST calls.
|
4181
4371
|
Bearer tokens are valid for a period of time defined (by the AoC app, configurable by admin) at its creation.
|
4182
4372
|
`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
|
4183
4373
|
|
@@ -4189,7 +4379,7 @@ If you use the built-in client_id and client_secret, skip this and do not set th
|
|
4189
4379
|
|
4190
4380
|
Else you can use a specific OAuth API client_id, the first step is to declare `ascli` in Aspera on Cloud using the admin interface.
|
4191
4381
|
|
4192
|
-
([AoC documentation: Registering an API Client](https://ibmaspera.com/help/admin/organization/registering_an_api_client)
|
4382
|
+
([AoC documentation: Registering an API Client](https://ibmaspera.com/help/admin/organization/registering_an_api_client)).
|
4193
4383
|
|
4194
4384
|
Let's start by a registration with web based authentication (auth=web):
|
4195
4385
|
|
@@ -4210,9 +4400,9 @@ Once the client is registered, a **Client ID** and **Secret** are created, these
|
|
4210
4400
|
|
4211
4401
|
#### Configuration for Aspera on Cloud
|
4212
4402
|
|
4213
|
-
If you did not use the wizard, you can also manually create
|
4403
|
+
If you did not use the wizard, you can also manually create an [Option Preset](#option-preset) for `ascli` in its configuration file.
|
4214
4404
|
|
4215
|
-
|
4405
|
+
Let's create an [Option Preset](#option-preset) called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
|
4216
4406
|
|
4217
4407
|
```bash
|
4218
4408
|
ascli config preset ask my_aoc_org url client_id client_secret
|
@@ -4262,11 +4452,15 @@ ascli aoc admin client list
|
|
4262
4452
|
```
|
4263
4453
|
|
4264
4454
|
```output
|
4265
|
-
|
4266
|
-
|
4267
|
-
|
4268
|
-
|
4269
|
-
|
4455
|
+
Items: 4/4
|
4456
|
+
╭────────────┬─────────────────────╮
|
4457
|
+
│ id │ name │
|
4458
|
+
╞════════════╪═════════════════════╡
|
4459
|
+
│ oXPUyJ7JpQ │ PRI Sydney │
|
4460
|
+
│ TaoAmAG8Rg │ ascli_test_web │
|
4461
|
+
│ TDNl2bLZqw │ ascli_web │
|
4462
|
+
│ VTh92i5OfQ │ shannon │
|
4463
|
+
╰────────────┴─────────────────────╯
|
4270
4464
|
```
|
4271
4465
|
|
4272
4466
|
```bash
|
@@ -4333,7 +4527,7 @@ ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.asp
|
|
4333
4527
|
|
4334
4528
|
> **Note:** The private key argument represents the actual PEM string.
|
4335
4529
|
In order to read the content from a file, use the `@file:` prefix.
|
4336
|
-
But if the
|
4530
|
+
But if the `@file:` argument is used as is, it will read the file and set in the configuration file.
|
4337
4531
|
So, to keep the `@file:` tag in the configuration file, the `@val:` prefix is added.
|
4338
4532
|
|
4339
4533
|
After this last step, commands do not require web login anymore.
|
@@ -4402,12 +4596,12 @@ Values are directly sent to the API call and used as a filter on server side.
|
|
4402
4596
|
|
4403
4597
|
The following parameters are supported:
|
4404
4598
|
|
4405
|
-
- `q` : a filter on name of resource (case
|
4599
|
+
- `q` : a filter on name of resource (case-insensitive, matches if value is contained in name)
|
4406
4600
|
- `sort`: name of fields to sort results, prefix with `-` for reverse order.
|
4407
4601
|
- `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
4408
4602
|
- `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
4409
|
-
- `page` : native
|
4410
|
-
- `per_page` : native
|
4603
|
+
- `page` : native API parameter, in general do not use (added by `ascli`)
|
4604
|
+
- `per_page` : native API parameter, number of items par API call, in general do not use
|
4411
4605
|
- Other specific parameters depending on resource type.
|
4412
4606
|
|
4413
4607
|
Both `max` and `pmax` are processed internally in `ascli`, not included in actual API call and limit the number of successive pages requested to API. `ascli` will return all values using paging if not provided.
|
@@ -4446,24 +4640,25 @@ Refer to the AoC API for full list of query parameters, or use the browser in de
|
|
4446
4640
|
|
4447
4641
|
#### Selecting a resource
|
4448
4642
|
|
4449
|
-
Resources are identified by a unique `id`, as well as a unique `name` (case
|
4643
|
+
Resources are identified by a unique `id`, as well as a unique `name` (case-insensitive).
|
4450
4644
|
|
4451
4645
|
To execute an action on a specific resource, select it using one of those methods:
|
4452
4646
|
|
4453
|
-
- **recommended**: give
|
4647
|
+
- **recommended**: give ID directly on command line **after the action**: `aoc admin node show 123`
|
4454
4648
|
- Give name on command line **after the action**: `aoc admin node show name abc`
|
4455
4649
|
- Provide option `id` : `aoc admin node show 123`
|
4456
4650
|
- Provide option `name` : `aoc admin node show --name=abc`
|
4457
4651
|
|
4458
4652
|
#### Creating a resource
|
4459
4653
|
|
4460
|
-
New resources (users, groups, workspaces, etc
|
4654
|
+
New resources (users, groups, workspaces, etc...) can be created using a command like:
|
4461
4655
|
|
4462
4656
|
```bash
|
4463
4657
|
ascli aoc admin create <resource type> @json:'{<...parameters...>}'
|
4464
4658
|
```
|
4465
4659
|
|
4466
|
-
Some
|
4660
|
+
Some API endpoints are described in [IBM API Hub](https://developer.ibm.com/apis/catalog?search=%22aspera%20on%20cloud%20api%22).
|
4661
|
+
Sadly, not all.
|
4467
4662
|
|
4468
4663
|
Nevertheless, it is possible to guess the structure of the creation value by simply dumping an existing resource, and use the same parameters for the creation.
|
4469
4664
|
|
@@ -4520,7 +4715,8 @@ The activity app can be queried with:
|
|
4520
4715
|
ascli aoc admin analytics transfers
|
4521
4716
|
```
|
4522
4717
|
|
4523
|
-
It can also support filters and send notification using option `notify_to`.
|
4718
|
+
It can also support filters and send notification using option `notify_to`.
|
4719
|
+
A template is defined using option `notify_template` :
|
4524
4720
|
|
4525
4721
|
`mytemplate.erb`:
|
4526
4722
|
|
@@ -4552,11 +4748,11 @@ Options:
|
|
4552
4748
|
- `query` filter (on API call)
|
4553
4749
|
- `notify` send an email as specified by template, this could be places in a file with the `@file` modifier.
|
4554
4750
|
|
4555
|
-
> **Note:** This must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now]
|
4751
|
+
> **Note:** This must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is `[date of previous execution]..[now]`.
|
4556
4752
|
|
4557
4753
|
#### Transfer: Using specific transfer ports
|
4558
4754
|
|
4559
|
-
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
4755
|
+
By default, transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
4560
4756
|
The option `default_ports` ([yes]/no) allows `ascli` to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
4561
4757
|
|
4562
4758
|
#### Using ATS
|
@@ -4569,7 +4765,7 @@ Aspera on Cloud Shared folders are implemented through a special type of file: `
|
|
4569
4765
|
A `link` is the equivalent of a symbolic link on a file system: it points to another folder (not file).
|
4570
4766
|
|
4571
4767
|
Listing a link (in terminal position of path) will information on the link itself, not the content of the folder it points to.
|
4572
|
-
To list the target folder content, add a `/`
|
4768
|
+
To list the target folder content, add a `/` at the end of the path.
|
4573
4769
|
|
4574
4770
|
Example:
|
4575
4771
|
|
@@ -4662,7 +4858,7 @@ ascli aoc user workspaces list
|
|
4662
4858
|
|
4663
4859
|
#### Example: Create a sub access key in a `node`
|
4664
4860
|
|
4665
|
-
Creation of a sub-access key is like creation of access key with the following difference: authentication to
|
4861
|
+
Creation of a sub-access key is like creation of access key with the following difference: authentication to Node API is made with access key (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
|
4666
4862
|
|
4667
4863
|
```bash
|
4668
4864
|
ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create @json:'{"storage":{"path":"/folder1"}}'
|
@@ -4717,14 +4913,14 @@ Other query parameters:
|
|
4717
4913
|
|
4718
4914
|
#### Example: Add all members of a workspace to another workspace
|
4719
4915
|
|
4720
|
-
a- Get
|
4916
|
+
a- Get ID of first workspace
|
4721
4917
|
|
4722
4918
|
```bash
|
4723
4919
|
WS1='First Workspace'
|
4724
4920
|
WS1ID=$(ascli aoc admin workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
|
4725
4921
|
```
|
4726
4922
|
|
4727
|
-
b- Get
|
4923
|
+
b- Get ID of second workspace
|
4728
4924
|
|
4729
4925
|
```bash
|
4730
4926
|
WS2='Second Workspace'
|
@@ -4743,7 +4939,7 @@ d- Convert to creation data for second workspace:
|
|
4743
4939
|
grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
|
4744
4940
|
```
|
4745
4941
|
|
4746
|
-
or, using jq
|
4942
|
+
or, using `jq`:
|
4747
4943
|
|
4748
4944
|
```bash
|
4749
4945
|
jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
|
@@ -4786,7 +4982,7 @@ ascli aoc admin group create @json:'{"name":"group 1","description":"my super gr
|
|
4786
4982
|
|
4787
4983
|
Group: `11111`
|
4788
4984
|
|
4789
|
-
- Get the workspace
|
4985
|
+
- Get the workspace ID
|
4790
4986
|
|
4791
4987
|
```bash
|
4792
4988
|
ascli aoc admin workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
|
@@ -4800,7 +4996,7 @@ Workspace: 22222
|
|
4800
4996
|
ascli aoc admin workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
|
4801
4997
|
```
|
4802
4998
|
|
4803
|
-
- Get a user's
|
4999
|
+
- Get a user's ID
|
4804
5000
|
|
4805
5001
|
```bash
|
4806
5002
|
ascli aoc admin user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
|
@@ -4818,7 +5014,7 @@ ascli aoc admin group_membership create @json:'{"group_id":11111,"member_type":"
|
|
4818
5014
|
|
4819
5015
|
In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
|
4820
5016
|
|
4821
|
-
First,
|
5017
|
+
First, set up the environment (skip if already done)
|
4822
5018
|
|
4823
5019
|
```bash
|
4824
5020
|
ascli config wizard --url=https://sedemo.ibmaspera.com --username=someuser@example.com
|
@@ -4846,7 +5042,7 @@ You can test with:
|
|
4846
5042
|
ascli aoc user profile show
|
4847
5043
|
```
|
4848
5044
|
|
4849
|
-
This creates the option preset `aoc_[org name]` to allow seamless command line access and sets it as default for
|
5045
|
+
This creates the option preset `aoc_[org name]` to allow seamless command line access and sets it as default for Aspera on Cloud.
|
4850
5046
|
|
4851
5047
|
Then, create two shared folders located in two regions, in your files home, in a workspace.
|
4852
5048
|
|
@@ -4856,20 +5052,18 @@ Then, transfer between those:
|
|
4856
5052
|
ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
4857
5053
|
```
|
4858
5054
|
|
4859
|
-
#### Example: Create registration key to register a node
|
5055
|
+
#### Example: Create registration key to register a tethered node
|
4860
5056
|
|
4861
|
-
|
4862
|
-
ascli aoc admin client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
4863
|
-
```
|
5057
|
+
The following command will create and display a secret token to register a self-managed Aspera Transfer Server:
|
4864
5058
|
|
4865
|
-
```
|
4866
|
-
|
5059
|
+
```bash
|
5060
|
+
ascli aoc admin client_registration_token create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --show-secrets=yes
|
4867
5061
|
```
|
4868
5062
|
|
4869
5063
|
#### Example: Delete all registration keys
|
4870
5064
|
|
4871
5065
|
```bash
|
4872
|
-
ascli aoc admin
|
5066
|
+
ascli aoc admin client_registration_token list --fields=id --format=csv|ascli aoc admin client_registration_token delete @lines:@stdin: --bulk=yes
|
4873
5067
|
```
|
4874
5068
|
|
4875
5069
|
```output
|
@@ -4905,9 +5099,9 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
4905
5099
|
ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
|
4906
5100
|
```
|
4907
5101
|
|
4908
|
-
Once executed, the access key `id` and `secret`, randomly generated by the
|
5102
|
+
Once executed, the access key `id` and `secret`, randomly generated by the Node API, is displayed.
|
4909
5103
|
|
4910
|
-
> **Note:** Once returned by the API, the secret will not be available anymore, so store this preciously. ATS secrets can only be reset by asking
|
5104
|
+
> **Note:** Once returned by the API, the secret will not be available anymore, so store this preciously. ATS secrets can only be reset by asking IBM support.
|
4911
5105
|
|
4912
5106
|
- Create the AoC node entity
|
4913
5107
|
|
@@ -4930,12 +5124,12 @@ Creation of a node with a self-managed node is similar, but the command `aoc adm
|
|
4930
5124
|
Source files are provided as a list with the `sources` option.
|
4931
5125
|
Refer to section [File list](#list-of-files-for-transfers)
|
4932
5126
|
|
4933
|
-
> **Note:** A special case is when the source files are located on **Aspera on Cloud** (i.e. using access keys and the `file
|
5127
|
+
> **Note:** A special case is when the source files are located on **Aspera on Cloud** (i.e. using access keys and the `file ID` API).
|
4934
5128
|
|
4935
5129
|
Source files are located on **Aspera on cloud**, when :
|
4936
5130
|
|
4937
|
-
- The server is Aspera on Cloud, and executing a download or recv
|
4938
|
-
- The agent is Aspera on Cloud, and executing an upload or send
|
5131
|
+
- The server is Aspera on Cloud, and executing a `download` or `recv`
|
5132
|
+
- The agent is Aspera on Cloud, and executing an `upload` or `send`
|
4939
5133
|
|
4940
5134
|
In this case:
|
4941
5135
|
|
@@ -4945,7 +5139,7 @@ In this case:
|
|
4945
5139
|
- Specify the source folder as first item in the list
|
4946
5140
|
- followed by the list of file names.
|
4947
5141
|
|
4948
|
-
### Packages
|
5142
|
+
### Packages app
|
4949
5143
|
|
4950
5144
|
The web-mail-like application.
|
4951
5145
|
|
@@ -4954,7 +5148,7 @@ The web-mail-like application.
|
|
4954
5148
|
General syntax:
|
4955
5149
|
|
4956
5150
|
```bash
|
4957
|
-
ascli aoc packages send [package extended value] [other parameters such as
|
5151
|
+
ascli aoc packages send [package extended value] [other parameters such as options and file list]
|
4958
5152
|
```
|
4959
5153
|
|
4960
5154
|
Package creation parameter are sent as **Command Parameter**.
|
@@ -4968,37 +5162,23 @@ ascli aoc packages shared_inboxes list
|
|
4968
5162
|
|
4969
5163
|
Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: **user** or **shared inbox**:
|
4970
5164
|
|
4971
|
-
- Provide either
|
5165
|
+
- Provide either IDs as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
4972
5166
|
- or just names: `"recipients":[{"The Dest"}]`.
|
4973
5167
|
|
4974
|
-
ascli will resolve the list of email addresses and dropbox names to the expected type/
|
5168
|
+
ascli will resolve the list of email addresses and dropbox names to the expected type/ID list, based on case-insensitive partial match.
|
4975
5169
|
|
4976
5170
|
If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and:
|
4977
5171
|
|
4978
5172
|
- if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
|
4979
5173
|
- if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
4980
5174
|
|
4981
|
-
|
4982
|
-
|
4983
|
-
By default, when using `aoc packages list`, the following `query` is performed:
|
4984
|
-
|
4985
|
-
| query parameter | value |
|
4986
|
-
|----------------------------|-------|
|
4987
|
-
| `archived` | `false` |
|
4988
|
-
| `has_content` | `true` |
|
4989
|
-
| `received` | `true` |
|
4990
|
-
| `completed` | `true` |
|
4991
|
-
| `workspace_id` | based on current workspace |
|
4992
|
-
| `exclude_dropbox_packages` | `true` or `false` depending if watching a dropbox|
|
4993
|
-
| `dropbox_id` | set accoring to `dropbox_name` |
|
4994
|
-
|
4995
|
-
#### Example: Send a package with one file to two users, using their email
|
5175
|
+
##### Example: Send a package with one file to two users, using their email
|
4996
5176
|
|
4997
5177
|
```bash
|
4998
5178
|
ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["someuser@example.com","other@example.com"]}' my_file.dat
|
4999
5179
|
```
|
5000
5180
|
|
5001
|
-
|
5181
|
+
##### Example: Send a package to a shared inbox with metadata
|
5002
5182
|
|
5003
5183
|
```bash
|
5004
5184
|
ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
|
@@ -5010,43 +5190,7 @@ It is also possible to use identifiers and API parameters:
|
|
5010
5190
|
ascli aoc packages send --workspace="my ws" @json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
5011
5191
|
```
|
5012
5192
|
|
5013
|
-
|
5014
|
-
|
5015
|
-
When user packages are listed, the following query is used:
|
5016
|
-
|
5017
|
-
```json
|
5018
|
-
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
5019
|
-
```
|
5020
|
-
|
5021
|
-
To list packages in a shared inbox, the query has to be specified with the the shared inbox by name or its identifier.
|
5022
|
-
Additional parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode).
|
5023
|
-
The current workspace is added unless specified in the query.
|
5024
|
-
|
5025
|
-
> **Note:** By default, `exclude_dropbox_packages` is set to `true` for user packages, and to false for shared inbox packages. This can be overridden in the query.
|
5026
|
-
|
5027
|
-
Using shared inbox name:
|
5028
|
-
|
5029
|
-
```bash
|
5030
|
-
ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
5031
|
-
```
|
5032
|
-
|
5033
|
-
Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
5034
|
-
|
5035
|
-
```bash
|
5036
|
-
shared_box_id=$(ascli aoc packages shared_inboxes show --name='My Shared Inbox' --format=csv --display=data --fields=id)
|
5037
|
-
```
|
5038
|
-
|
5039
|
-
```bash
|
5040
|
-
ascli aoc packages list --query=@json:'{"dropbox_id":"'$shared_box_id'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
5041
|
-
```
|
5042
|
-
|
5043
|
-
#### Example: Receive all packages from a given shared inbox
|
5044
|
-
|
5045
|
-
```bash
|
5046
|
-
ascli aoc packages recv ALL --workspace=_workspace_ --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"_shared_inbox_name_","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
5047
|
-
```
|
5048
|
-
|
5049
|
-
#### Example: Send a package with files from the Files app
|
5193
|
+
##### Example: Send a package with files from the Files app
|
5050
5194
|
|
5051
5195
|
Find files in Files app:
|
5052
5196
|
|
@@ -5071,7 +5215,35 @@ Let's send a package with the file `10M.dat` from subfolder /src_folder in a pac
|
|
5071
5215
|
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
|
5072
5216
|
```
|
5073
5217
|
|
5074
|
-
#### Receive
|
5218
|
+
#### Receive packages
|
5219
|
+
|
5220
|
+
The command to receive one or multiple packages is:
|
5221
|
+
|
5222
|
+
```bash
|
5223
|
+
ascli aoc packages recv <package id> [<file> ...]
|
5224
|
+
```
|
5225
|
+
|
5226
|
+
Where `<package id>` is the identifier of the package to receive or `ALL` to receive all packages matching the query.
|
5227
|
+
Option `once_only` is supported, see below.
|
5228
|
+
|
5229
|
+
To download only some files from the package, just add the path of the files on the command line: `[<file> ...]`, see option `sources`.
|
5230
|
+
By default, all files in the package are downloaded, i.e. `.` is used as the file list.
|
5231
|
+
|
5232
|
+
Option `package_folder` defines the attribute of folder used as destination sub folder in the `to_folder` path (see description earlier).
|
5233
|
+
The default value is `@none:` : package files will be downloaded directly inside the folder specified by option `to_folder`.
|
5234
|
+
The option `package_folder` can be set to the name of **any** attributes of the package.
|
5235
|
+
Notably, `id` or `name` can be used.
|
5236
|
+
Using option `--package-folder=id` ensures that every downloaded package is placed in a subfolder named after its unique ID.
|
5237
|
+
|
5238
|
+
##### Example: Receive all packages from a given shared inbox
|
5239
|
+
|
5240
|
+
```bash
|
5241
|
+
ascli aoc packages recv ALL --workspace=_workspace_ --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"_shared_inbox_name_","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
5242
|
+
```
|
5243
|
+
|
5244
|
+
To list packages that would be downloaded, without actually downloading them, replace `recv ALL` with `list` (keep options `once_only` and `query`)
|
5245
|
+
|
5246
|
+
##### Receive new packages only (Cargo)
|
5075
5247
|
|
5076
5248
|
It is possible to automatically download new packages, like using Aspera Cargo:
|
5077
5249
|
|
@@ -5079,13 +5251,13 @@ It is possible to automatically download new packages, like using Aspera Cargo:
|
|
5079
5251
|
ascli aoc packages recv ALL --once-only=yes --lock-port=12345
|
5080
5252
|
```
|
5081
5253
|
|
5082
|
-
- `ALL` (case
|
5254
|
+
- `ALL` (case-sensitive) will download all packages
|
5083
5255
|
- `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
|
5084
5256
|
- `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
|
5085
5257
|
|
5086
5258
|
Typically, one would execute this command on a regular basis, using the method of your choice: see [Scheduler](#scheduler).
|
5087
5259
|
|
5088
|
-
|
5260
|
+
##### Example: Content of a received Package
|
5089
5261
|
|
5090
5262
|
Some `node` operations are available for a package, such as `browse` and `find`.
|
5091
5263
|
|
@@ -5095,17 +5267,62 @@ To list the content of a package, use command `packages browse <package id> <fol
|
|
5095
5267
|
ascli aoc package browse my5CnbeWng /
|
5096
5268
|
```
|
5097
5269
|
|
5098
|
-
|
5099
|
-
|
5100
|
-
To download only some of the files listed in the package, just add the path of the files on the command line.
|
5270
|
+
Use command `find` to list recursively.
|
5101
5271
|
|
5102
5272
|
For advanced users, it's also possible to pipe node information for the package and use node operations:
|
5103
5273
|
|
5104
5274
|
```bash
|
5105
|
-
ascli aoc package node_info <package
|
5275
|
+
ascli aoc package node_info <package ID here> / --format=json --show-secrets=yes --display=data | ascli node -N --preset=@json:@stdin: access_key do self browse /
|
5276
|
+
```
|
5277
|
+
|
5278
|
+
#### List packages
|
5279
|
+
|
5280
|
+
By default, when using `aoc packages list` or `aoc packages receive ALL`, the following `query` is performed:
|
5281
|
+
|
5282
|
+
| Query parameter | Value |
|
5283
|
+
|----------------------------|---------|
|
5284
|
+
| `archived` | `false` |
|
5285
|
+
| `has_content` | `true` |
|
5286
|
+
| `received` | `true` |
|
5287
|
+
| `completed` | `true` |
|
5288
|
+
| `workspace_id` | Set based on current workspace. |
|
5289
|
+
| `dropbox_id` | Set according to `dropbox_name`, if provided. |
|
5290
|
+
| `exclude_dropbox_packages` | `true` unless `dropbox_id` is provided. |
|
5291
|
+
|
5292
|
+
Parameters provided using option `query` override this query.
|
5293
|
+
To remove a parameter, set it to `null`.
|
5294
|
+
|
5295
|
+
##### Example: List packages in a given shared inbox
|
5296
|
+
|
5297
|
+
When user packages are listed, the following query is used:
|
5298
|
+
|
5299
|
+
```json
|
5300
|
+
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
5301
|
+
```
|
5302
|
+
|
5303
|
+
To list packages in a shared inbox, the query has to be specified with the shared inbox by name or its identifier.
|
5304
|
+
Additional parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode).
|
5305
|
+
The current workspace is added unless specified in the query.
|
5306
|
+
|
5307
|
+
> **Note:** By default, `exclude_dropbox_packages` is set to `true` for user packages, and to false for shared inbox packages. This can be overridden in the query.
|
5308
|
+
|
5309
|
+
Using shared inbox name:
|
5310
|
+
|
5311
|
+
```bash
|
5312
|
+
ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
5106
5313
|
```
|
5107
5314
|
|
5108
|
-
|
5315
|
+
Using shared inbox identifier: first retrieve the ID of the shared inbox, and then list packages with the appropriate filter.
|
5316
|
+
|
5317
|
+
```bash
|
5318
|
+
shared_box_id=$(ascli aoc packages shared_inboxes show --name='My Shared Inbox' --format=csv --display=data --fields=id)
|
5319
|
+
```
|
5320
|
+
|
5321
|
+
```bash
|
5322
|
+
ascli aoc packages list --query=@json:'{"dropbox_id":"'$shared_box_id'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
5323
|
+
```
|
5324
|
+
|
5325
|
+
### Files app
|
5109
5326
|
|
5110
5327
|
The Files application presents a **Home** folder to users in a given workspace.
|
5111
5328
|
Files located here are either user's files, or shared folders.
|
@@ -5131,13 +5348,13 @@ ascli aoc files download <single file path>
|
|
5131
5348
|
#### Shared folders
|
5132
5349
|
|
5133
5350
|
Like in AoC web UI, "Shared Folders" can be created and shared with either **Private** or **Public** links.
|
5134
|
-
**Private** links require the collaborator to log
|
5351
|
+
**Private** links require the collaborator to log in to access the shared folder.
|
5135
5352
|
**Public** links include a passcode that enables the user to access the shared folder without login-in.
|
5136
5353
|
|
5137
5354
|
Shared folders can be created either:
|
5138
5355
|
|
5139
5356
|
- by users in a workspace: they can share personal folders with other users in the same workspace: `aoc files perm`
|
5140
|
-
- by administrators: they can share a folder with users in any workspace: `aoc admin node do [node
|
5357
|
+
- by administrators: they can share a folder with users in any workspace: `aoc admin node do [node ID] perm`
|
5141
5358
|
|
5142
5359
|
Technically (API), shared folder are managed through [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction) on node and an event is sent to AoC to create a **link** in the user's home folder to the shared folder.
|
5143
5360
|
In both cases, it is necessary to specify a workspace.
|
@@ -5155,42 +5372,46 @@ The basic payload (last argument at creation usually specified with `@json:`) is
|
|
5155
5372
|
```
|
5156
5373
|
|
5157
5374
|
`ascli` expects the same payload for creation.
|
5158
|
-
`ascli` automatically populates
|
5375
|
+
`ascli` automatically populates some payload fields and provides convenient additional fields that generate native fields:
|
5159
5376
|
|
5160
|
-
-
|
5161
|
-
|
5162
|
-
|
5163
|
-
|
5377
|
+
<!-- markdownlint-disable MD033 -->
|
5378
|
+
| Field | Type | Description |
|
5379
|
+
|-----------------|----------|-------------|
|
5380
|
+
| `file_id` | Native<br/>Auto | ID of the folder to share, as specified in the command line by path. |
|
5381
|
+
| `access_levels` | Native<br/>Optional | List of access levels to set for the shared folder. Defaults to full access. |
|
5382
|
+
| `tags` | Native<br/>Auto | Set with expected values for AoC: username who creates, and workspace in which the shared folder is created. |
|
5383
|
+
| `access_type` | Native<br/>Required | Type of access, such as `user`, `group`, or `workspace`. Can be set with parameter `with`. |
|
5384
|
+
| `access_id` | Native<br/>Required | ID of the user, group, or workspace (see `with`) |
|
5385
|
+
| `with` | `ascli` | Recipient of shared folder. Can be a username, a group name, or a workspace name. `ascli` will resolve the name to the proper type and ID in fields `access_type` and `access_id`. If the value is the empty string, then it declares the shared folder in the workspace (first action to do, see below). |
|
5386
|
+
| `link_name` | `ascli` | Name of the link file created in the user's home folder for private links. |
|
5387
|
+
| `as` | `ascli` | Name of the link file created in the user's home folder for admin shared folders. |
|
5388
|
+
<!-- markdownlint-enable MD033 -->
|
5164
5389
|
|
5165
|
-
|
5390
|
+
In order to declare/create the shared folder in the workspace, a special value for `access_id` is used: `ASPERA_ACCESS_KEY_ADMIN_WS_[workspace ID]]`.
|
5391
|
+
This is conveniently set by `ascli` using an empty string for the pseudo key `with`.
|
5392
|
+
In order to share a folder with a different, special tags are set, but this is conveniently done by `ascli` using the `as` key.
|
5166
5393
|
|
5167
|
-
|
5168
|
-
|
5169
|
-
The following optional additional helper keys are supported by `ascli`:
|
5170
|
-
|
5171
|
-
- `with` : Recipient of shared folder. Can be a user name, a group name, or a workspace name. `ascli` will resolve the name to the proper type and id in fields `access_type` and `access_id`. If the value is the empty string, then it declares the shared folder in the workspace (first action to do, see below).
|
5172
|
-
- `link_name` : The name of the link file created in the user's home folder for private links.
|
5173
|
-
- `as` : The name of the link file created in the user's home folder for admin shared folders.
|
5174
|
-
|
5175
|
-
- List permissions on a shared folder as user
|
5394
|
+
##### Example: List permissions on a shared folder
|
5176
5395
|
|
5177
5396
|
```bash
|
5178
5397
|
ascli aoc files perm /shared_folder_test1 list
|
5179
5398
|
```
|
5180
5399
|
|
5181
|
-
|
5400
|
+
##### Example: Share a personal folder with other users
|
5182
5401
|
|
5183
5402
|
```bash
|
5184
5403
|
ascli aoc files perm /shared_folder_test1 create @json:'{"with":"laurent"}'
|
5185
5404
|
```
|
5186
5405
|
|
5187
|
-
|
5406
|
+
##### Example: Revoke shared access
|
5188
5407
|
|
5189
5408
|
```bash
|
5190
5409
|
ascli aoc files perm /shared_folder_test1 delete 6161
|
5191
5410
|
```
|
5192
5411
|
|
5193
|
-
Public and Private short links
|
5412
|
+
##### Example: Public and Private short links
|
5413
|
+
|
5414
|
+
They can be managed with commands:
|
5194
5415
|
|
5195
5416
|
```bash
|
5196
5417
|
ascli aoc files short_link _path_here_ private create
|
@@ -5199,17 +5420,63 @@ ascli aoc files short_link _path_here_ public list
|
|
5199
5420
|
ascli aoc files short_link public delete _id_
|
5200
5421
|
```
|
5201
5422
|
|
5202
|
-
|
5423
|
+
##### Example: Create a workspace shared folder
|
5424
|
+
|
5425
|
+
First, identify the node ID where the shared folder will be created.
|
5426
|
+
|
5427
|
+
To use the default node for workspace `my ws`, use the command:
|
5428
|
+
|
5429
|
+
```bash
|
5430
|
+
ascli aoc admin workspace show %name:'my ws' --fields=node_id
|
5431
|
+
```
|
5432
|
+
|
5433
|
+
Alternatively (longer):
|
5434
|
+
|
5435
|
+
```bash
|
5436
|
+
ascli aoc admin workspace list --select=@json:'{"name":"my ws"}' --fields=node_id
|
5437
|
+
```
|
5438
|
+
|
5439
|
+
Or select manually from the list of nodes:
|
5440
|
+
|
5441
|
+
```bash
|
5442
|
+
ascli aoc admin node list --fields=id,name
|
5443
|
+
```
|
5444
|
+
|
5445
|
+
In the following commands, replace:
|
5446
|
+
|
5447
|
+
- `1234` with the node ID
|
5448
|
+
- `my ws` with the workspace name
|
5449
|
+
- `folder_on_node` with the name of the folder on the node
|
5450
|
+
|
5451
|
+
The node can also be conveniently identified using the **percent selector** instead of numerical ID: `%name:"my node"`.
|
5452
|
+
|
5453
|
+
If the shared folder does not exist, then create it:
|
5203
5454
|
|
5204
5455
|
```bash
|
5205
5456
|
ascli aoc admin node do 1234 mkdir folder_on_node
|
5457
|
+
```
|
5458
|
+
|
5459
|
+
Create the shared folder in workspace `my ws` (set `with` to empty string, or do not specify it).
|
5460
|
+
Optionally use `as` to set the name of the shared folder if different from the folder name on the node.
|
5461
|
+
For other options, refer to the previous section on shared folders.
|
5462
|
+
|
5463
|
+
```bash
|
5206
5464
|
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"","as":"folder_for_users"}' --workspace="my ws"
|
5207
|
-
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"john@example.com","as":"folder_for_users"}' --workspace="my ws"
|
5208
|
-
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"group 1","as":"folder_for_users"}' --workspace="my ws"
|
5209
|
-
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"my ws","as":"folder_for_users"}' --workspace="my ws"
|
5210
5465
|
```
|
5211
5466
|
|
5212
|
-
|
5467
|
+
To share with a user, group, or workspace, use the `with` parameter:
|
5468
|
+
|
5469
|
+
```bash
|
5470
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"john@example.com","as":"folder_for_one_user"}' --workspace="my ws"
|
5471
|
+
```
|
5472
|
+
|
5473
|
+
```bash
|
5474
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"group 1","as":"folder_for_a_group"}' --workspace="my ws"
|
5475
|
+
```
|
5476
|
+
|
5477
|
+
```bash
|
5478
|
+
ascli aoc admin node do 1234 perm folder_on_node create @json:'{"with":"my ws","as":"folder_for_all_workspace"}' --workspace="my ws"
|
5479
|
+
```
|
5213
5480
|
|
5214
5481
|
#### Cross Organization transfers
|
5215
5482
|
|
@@ -5219,9 +5486,9 @@ Although optional, the creation of [Option Preset](#option-preset) is recommende
|
|
5219
5486
|
|
5220
5487
|
Procedure to send a file from org1 to org2:
|
5221
5488
|
|
5222
|
-
- Get access to Organization 1 and create
|
5489
|
+
- Get access to Organization 1 and create an [Option Preset](#option-preset): e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
|
5223
5490
|
- Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
|
5224
|
-
- Get access to Organization 2 and create
|
5491
|
+
- Get access to Organization 2 and create an [Option Preset](#option-preset): e.g. `org2`
|
5225
5492
|
- Check that access works and locate the destination folder `mydestfolder`
|
5226
5493
|
- Execute the following:
|
5227
5494
|
|
@@ -5232,18 +5499,18 @@ ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | as
|
|
5232
5499
|
Explanation:
|
5233
5500
|
|
5234
5501
|
- `-Porg1 aoc` use Aspera on Cloud plugin and load credentials for `org1`
|
5235
|
-
- `files node_info /mydestfolder` generate transfer information including
|
5502
|
+
- `files node_info /mydestfolder` generate transfer information including Node API credential and root ID, suitable for the next command
|
5236
5503
|
- `--format=json` format the output in JSON (instead of default text table)
|
5237
5504
|
- `--display=data` display only the result, and remove other information, such as workspace name
|
5238
5505
|
- `|` the standard output of the first command is fed into the second one
|
5239
5506
|
- `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
|
5240
5507
|
- `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
|
5241
5508
|
- `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agent-direct)
|
5242
|
-
- `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e.
|
5509
|
+
- `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. Node API credentials, those are expected in JSON format and read from standard input
|
5243
5510
|
|
5244
5511
|
#### Find Files
|
5245
5512
|
|
5246
|
-
The command `aoc files find` allows
|
5513
|
+
The command `aoc files find` allows searching for files in a given workspace.
|
5247
5514
|
|
5248
5515
|
It works also on `node` resource using the `v4` command:
|
5249
5516
|
|
@@ -5283,6 +5550,7 @@ admin dropbox_membership list
|
|
5283
5550
|
admin group list
|
5284
5551
|
admin kms_profile list
|
5285
5552
|
admin node do %name:my_node_name --secret=my_ak_secret browse /
|
5553
|
+
admin node do %name:my_node_name --secret=my_ak_secret browse /folder_sub --node-cache=no
|
5286
5554
|
admin node do %name:my_node_name --secret=my_ak_secret delete /folder1
|
5287
5555
|
admin node do %name:my_node_name --secret=my_ak_secret delete /folder_sub
|
5288
5556
|
admin node do %name:my_node_name --secret=my_ak_secret mkdir /folder1
|
@@ -5326,13 +5594,13 @@ files browse / --url=my_public_link_folder_pass --password=my_public_link_passwo
|
|
5326
5594
|
files browse my_remote_file
|
5327
5595
|
files browse my_remote_folder
|
5328
5596
|
files browse my_remote_folder/
|
5597
|
+
files cat --to-folder=. testdst/test_file.bin
|
5329
5598
|
files delete /testsrc
|
5330
5599
|
files download --transfer=connect testdst/test_file.bin
|
5331
5600
|
files download --transfer=desktop testdst/test_file.bin
|
5332
5601
|
files find /
|
5333
5602
|
files find / '\.partial$'
|
5334
5603
|
files find / @ruby:'->(f){f["type"].eql?("file")}'
|
5335
|
-
files http_node_download --to-folder=. testdst/test_file.bin
|
5336
5604
|
files mkdir /testsrc
|
5337
5605
|
files modify my_test_folder
|
5338
5606
|
files permission my_test_folder list
|
@@ -5348,8 +5616,8 @@ files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","
|
|
5348
5616
|
files sync start --sync-info=@json:'{"name":"my_aoc_sync1","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
5349
5617
|
files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
5350
5618
|
files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
5351
|
-
files sync start pull /
|
5352
|
-
files sync start pull /
|
5619
|
+
files sync start pull /data/local_sync /testdst --sync-info=@json:'{"reset":true,"quiet":false}'
|
5620
|
+
files sync start pull /data/local_sync /testdst --sync-info=@json:'{"sessions":[{"reset":true}]}'
|
5353
5621
|
files thumbnail my_test_folder/video_file.mpg
|
5354
5622
|
files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
|
5355
5623
|
files transfer push /testsrc --to-folder=/testdst test_file.bin
|
@@ -5357,7 +5625,7 @@ files upload --to-folder=/ test_file.bin --url=my_public_link_folder_no_pass
|
|
5357
5625
|
files upload --to-folder=/testsrc test_file.bin
|
5358
5626
|
files upload --workspace=my_other_workspace --to-folder=my_other_folder test_file.bin --transfer=node --transfer-info=@json:@stdin:
|
5359
5627
|
files v3 info
|
5360
|
-
gateway --pid-file=pid_aocfxgw https://localhost:12345/aspera/faspex &
|
5628
|
+
gateway --pid-file=pid_aocfxgw @json:'{"url":"https://localhost:12345/aspera/faspex"}' &
|
5361
5629
|
org --url=my_public_link_recv_from_aoc_user
|
5362
5630
|
organization
|
5363
5631
|
packages browse package_id3 /
|
@@ -5367,7 +5635,7 @@ packages receive ALL --once-only=yes --to-folder=. --lock-port=12345
|
|
5367
5635
|
packages receive ALL --once-only=yes --to-folder=. --lock-port=12345 --query=@json:'{"dropbox_name":"my_shared_inbox_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
5368
5636
|
packages receive INIT --once-only=yes --query=@json:'{"dropbox_name":"my_shared_inbox_name"}'
|
5369
5637
|
packages receive package_id3 --to-folder=.
|
5370
|
-
packages receive package_id3 --to-folder=. /
|
5638
|
+
packages receive package_id3 --to-folder=. / --package-folder=name
|
5371
5639
|
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' test_file.bin
|
5372
5640
|
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
|
5373
5641
|
packages send --workspace=my_workspace_shared_inbox --validate-metadata=yes @json:'{"name":"$(notdir test) PACKAGE_TITLE_BASE","recipients":["my_shared_inbox_meta"],"metadata":{"Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
|
@@ -5381,6 +5649,7 @@ packages shared_inboxes show %name:my_shared_inbox_name
|
|
5381
5649
|
remind --username=my_user_email
|
5382
5650
|
servers
|
5383
5651
|
tier_restrictions
|
5652
|
+
user contacts list
|
5384
5653
|
user pref modify @json:'{"default_language":"en-us"}'
|
5385
5654
|
user pref show
|
5386
5655
|
user profile modify @json:'{"name":"dummy change"}'
|
@@ -5397,12 +5666,12 @@ ATS is usable either :
|
|
5397
5666
|
|
5398
5667
|
- Or from an IBM Cloud subscription : `ascli ats` : use IBM Cloud API key authentication
|
5399
5668
|
|
5400
|
-
### IBM Cloud ATS : Creation of
|
5669
|
+
### IBM Cloud ATS : Creation of API key
|
5401
5670
|
|
5402
5671
|
This section is about using ATS with an IBM cloud subscription.
|
5403
5672
|
If you are using ATS as part of AoC, then authentication is through AoC, not IBM Cloud.
|
5404
5673
|
|
5405
|
-
First get your IBM Cloud
|
5674
|
+
First get your IBM Cloud API key. For instance, it can be created using the IBM Cloud web interface, or using command line:
|
5406
5675
|
|
5407
5676
|
```bash
|
5408
5677
|
ibmcloud iam api-key-create mykeyname -d 'my sample key'
|
@@ -5430,7 +5699,7 @@ References:
|
|
5430
5699
|
|
5431
5700
|
<!-- spellchecker: enable -->
|
5432
5701
|
|
5433
|
-
Then, to register the key by default for the ats plugin, create a preset. Execute:
|
5702
|
+
Then, to register the key by default for the `ats` plugin, create a preset. Execute:
|
5434
5703
|
|
5435
5704
|
```bash
|
5436
5705
|
ascli config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here
|
@@ -5477,7 +5746,7 @@ Those are directly the parameters expected by the [ATS API](https://developer.ib
|
|
5477
5746
|
|
5478
5747
|
### Misc. Examples
|
5479
5748
|
|
5480
|
-
Example: create access key on IBM Cloud (
|
5749
|
+
Example: create access key on IBM Cloud (Softlayer):
|
5481
5750
|
|
5482
5751
|
```bash
|
5483
5752
|
ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"my_secret_here","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
|
@@ -5495,7 +5764,7 @@ Example: create access key on Azure SAS:
|
|
5495
5764
|
ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"myaccesskey","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
|
5496
5765
|
```
|
5497
5766
|
|
5498
|
-
|
5767
|
+
> **Note:** The blob name is mandatory after server address and before parameters, and that parameter `sr=c` is mandatory.
|
5499
5768
|
|
5500
5769
|
Example: create access key on Azure:
|
5501
5770
|
|
@@ -5623,13 +5892,13 @@ For file operation command (browse, delete), the Ruby SSH client library `Net::S
|
|
5623
5892
|
|
5624
5893
|
For a list of SSH client options, refer to the Ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html#method-c-start).
|
5625
5894
|
|
5626
|
-
|
5895
|
+
Among the 50 available SSH options:
|
5627
5896
|
|
5628
5897
|
- `verbose`
|
5629
5898
|
- `use_agent`
|
5630
5899
|
- `passphrase`
|
5631
5900
|
|
5632
|
-
By default the SSH library will check if a local `ssh-agent` is running.
|
5901
|
+
By default, the SSH library will check if a local `ssh-agent` is running.
|
5633
5902
|
|
5634
5903
|
On Linux, if you get an error message such as:
|
5635
5904
|
|
@@ -5643,7 +5912,7 @@ or on Windows:
|
|
5643
5912
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
5644
5913
|
```
|
5645
5914
|
|
5646
|
-
This means that your environment suggests
|
5915
|
+
This means that your environment suggests using an agent, but you don't have such an SSH agent running, then:
|
5647
5916
|
|
5648
5917
|
- Check env var: `SSH_AGENT_SOCK`
|
5649
5918
|
- Check your file: `$HOME/.ssh/config`
|
@@ -5663,12 +5932,12 @@ It is equivalent to setting both options `ssh_options.passphrase` and `ts.ssh_pr
|
|
5663
5932
|
### Other session channels for `server`
|
5664
5933
|
|
5665
5934
|
URL schemes `local` and `https` are also supported (mainly for testing purpose).
|
5666
|
-
(`--url=local
|
5935
|
+
(`--url=local:`, `--url=https://...`)
|
5667
5936
|
|
5668
5937
|
- `local` will execute `ascmd` locally, instead of using an SSH connection.
|
5669
5938
|
- `https` will use Web Socket Session: This requires the use of a transfer token. For example a `Basic` token can be used.
|
5670
5939
|
|
5671
|
-
As, most of the time, SSH is used, if
|
5940
|
+
As, most of the time, SSH is used, if a `http` scheme is provided without token, the plugin will fallback to SSH and port 33001.
|
5672
5941
|
|
5673
5942
|
### Examples: `server`
|
5674
5943
|
|
@@ -5680,9 +5949,9 @@ ascli server browse /aspera-test-dir-large
|
|
5680
5949
|
ascli server download /aspera-test-dir-large/200MB
|
5681
5950
|
```
|
5682
5951
|
|
5683
|
-
`initdemo` creates
|
5952
|
+
`initdemo` creates an [Option Preset](#option-preset) `demoserver` and set it as default for plugin `server`.
|
5684
5953
|
|
5685
|
-
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options
|
5954
|
+
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options` (for browsing) and `ts` (for transfers):
|
5686
5955
|
|
5687
5956
|
```bash
|
5688
5957
|
ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ssh_keys=_private_key_path_here_ --passphrase=_passphrase_here_
|
@@ -5690,13 +5959,13 @@ ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ss
|
|
5690
5959
|
|
5691
5960
|
## Plugin: `node`: IBM Aspera High Speed Transfer Server Node
|
5692
5961
|
|
5693
|
-
This plugin gives access to capabilities provided by the HSTS
|
5962
|
+
This plugin gives access to capabilities provided by the HSTS Node API.
|
5694
5963
|
|
5695
5964
|
The authentication is `username` and `password` or `access_key` and `secret` through options: `username` and `password`.
|
5696
5965
|
|
5697
|
-
> **Note:** Capabilities of this plugin are used in other plugins which access to the
|
5966
|
+
> **Note:** Capabilities of this plugin are used in other plugins which access to the Node API, such as `aoc`, `ats`, `shares`.
|
5698
5967
|
>
|
5699
|
-
> **Note:** This plugin can be used with any type of **Aspera Node**, either on
|
5968
|
+
> **Note:** This plugin can be used with any type of **Aspera Node**, either on premise or ATS, provided that you have Node API credentials. Those credentials can be either Node API user or Access Key (e.g. on ATS).
|
5700
5969
|
|
5701
5970
|
### File Operations
|
5702
5971
|
|
@@ -5735,7 +6004,7 @@ The command `find <folder> [filter_expr]` is available for **gen4/access key**,
|
|
5735
6004
|
The argument `<folder>` is mandatory and is the root from which search is performed.
|
5736
6005
|
The argument `[filter_expr]` is optional and represent the matching criteria.
|
5737
6006
|
|
5738
|
-
It recursively scans storage to find files/folders matching
|
6007
|
+
It recursively scans storage to find files/folders matching criteria and then returns a list of matching entries.
|
5739
6008
|
|
5740
6009
|
`[filter_expr]` is either:
|
5741
6010
|
|
@@ -5777,7 +6046,7 @@ The following are examples of Ruby lambda code to be provided in the following t
|
|
5777
6046
|
->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))>365}
|
5778
6047
|
```
|
5779
6048
|
|
5780
|
-
- Find files larger than
|
6049
|
+
- Find files larger than 1 MB
|
5781
6050
|
|
5782
6051
|
```ruby
|
5783
6052
|
->(f){f["type"].eql?("file") and f["size"].to_i>1000000}
|
@@ -5805,7 +6074,7 @@ ascli node access_keys do self find / @ruby:'->(f){f["type"].eql?("file") and (D
|
|
5805
6074
|
|
5806
6075
|
### Listing transfer events
|
5807
6076
|
|
5808
|
-
When a transfer is run, its information is stored (
|
6077
|
+
When a transfer is run, its information is stored (typically, 1 day) in the HSTS database (Redis).
|
5809
6078
|
This information can be retrieved with command: `transfer list`.
|
5810
6079
|
|
5811
6080
|
If the number of transfers is too large, then the list will be retrieved using several API calls.
|
@@ -5816,14 +6085,14 @@ In addition, it is possible to list "only new information" using option `once_on
|
|
5816
6085
|
ascli node transfer list --once-only=yes
|
5817
6086
|
```
|
5818
6087
|
|
5819
|
-
The `iteration_token` that keeps memory of latest event is stored in the
|
6088
|
+
The `iteration_token` that keeps memory of the latest event is stored in the persistence repository of `ascli`.
|
5820
6089
|
To reset it, add option: `--query=@json:'{"reset": true}'`.
|
5821
6090
|
To list only a number of events, use the `max` parameter in query.
|
5822
6091
|
Other parameters are directly transmitted to the underlying API (`GET /ops/transfers`).
|
5823
6092
|
|
5824
6093
|
### Central
|
5825
6094
|
|
5826
|
-
The central
|
6095
|
+
The central sub-command uses the **reliable query** API (session and file).
|
5827
6096
|
It allows listing transfer sessions and transferred files.
|
5828
6097
|
|
5829
6098
|
Filtering can be applied:
|
@@ -5841,12 +6110,12 @@ By providing the `validator` option, offline transfer validation can be done.
|
|
5841
6110
|
There are three sync types of operations:
|
5842
6111
|
|
5843
6112
|
- `sync`: perform a local sync, by executing `async` locally
|
5844
|
-
- `async`: calls legacy async API on node : `/async`
|
5845
|
-
- `ssync` : calls newer async API on node : `/asyncs`
|
6113
|
+
- `async`: calls legacy `async` API on node : `/async`
|
6114
|
+
- `ssync` : calls newer `async` API on node : `/asyncs`
|
5846
6115
|
|
5847
6116
|
### FASP Stream
|
5848
6117
|
|
5849
|
-
It is possible to start a
|
6118
|
+
It is possible to start a faspstream session using the Node API:
|
5850
6119
|
|
5851
6120
|
Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-spec**](#transfer-specification):
|
5852
6121
|
|
@@ -5858,10 +6127,10 @@ Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-
|
|
5858
6127
|
|
5859
6128
|
Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
|
5860
6129
|
|
5861
|
-
`ascli` supports remote operations through the
|
6130
|
+
`ascli` supports remote operations through the Node API. Operations are:
|
5862
6131
|
|
5863
6132
|
- Start `watchd` and `watchfolderd` services running as a system user having access to files
|
5864
|
-
- Configure a **
|
6133
|
+
- Configure a **Watchfolder** to define automated transfers
|
5865
6134
|
|
5866
6135
|
```bash
|
5867
6136
|
ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
@@ -5901,7 +6170,7 @@ updated
|
|
5901
6170
|
|
5902
6171
|
Scenario: Access to a **Shares on Demand** (SHOD) server on AWS is provided by a partner.
|
5903
6172
|
We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
|
5904
|
-
Simply create an **Aspera Transfer Service** instance, which provides access to the
|
6173
|
+
Simply create an **Aspera Transfer Service** instance, which provides access to the Node API.
|
5905
6174
|
Then create a configuration for the **SHOD** instance in the configuration file: in section **shares**, a configuration named: `aws_shod`.
|
5906
6175
|
Create another configuration for the Azure ATS instance: in section **node**, named `azure_ats`.
|
5907
6176
|
Then execute the following command:
|
@@ -5914,7 +6183,7 @@ This will get transfer information from the SHOD instance and tell the Azure ATS
|
|
5914
6183
|
|
5915
6184
|
### Node file information
|
5916
6185
|
|
5917
|
-
When
|
6186
|
+
When Node API is used with an **Access key**, extra information can be retrieved, such as preview.
|
5918
6187
|
|
5919
6188
|
> **Note:** Display of preview on terminal requires installation of extra gem: `rmagick`
|
5920
6189
|
|
@@ -5935,7 +6204,7 @@ Previews are mainly used in AoC, this also works with AoC:
|
|
5935
6204
|
ascli aoc files thumbnail /preview_samples/Aspera.mpg
|
5936
6205
|
```
|
5937
6206
|
|
5938
|
-
> **Note:** To specify the file by its file
|
6207
|
+
> **Note:** To specify the file by its file ID, use the selector syntax: `%id:_file_id_here_`
|
5939
6208
|
>
|
5940
6209
|
> **Note:** To force textual display of the preview on **iTerm**, prefix command with: `env -u TERM_PROGRAM -u LC_TERMINAL` or use option: ``
|
5941
6210
|
|
@@ -5946,7 +6215,7 @@ ascli node access_key create @json:'{"id":"myaccesskey","secret":"my_secret_here
|
|
5946
6215
|
```
|
5947
6216
|
|
5948
6217
|
> **Note:** The `id` and `secret` are optional.
|
5949
|
-
> If not provided, they will be generated and returned
|
6218
|
+
> If not provided, they will be generated and returned into the result.
|
5950
6219
|
|
5951
6220
|
Access keys support extra overriding parameters using parameter: `configuration` and sub keys `transfer` and `server`. For example, an access key can be modified or created with the following options:
|
5952
6221
|
|
@@ -5966,27 +6235,27 @@ Bearer tokens are part of the **gen4/access key** API.
|
|
5966
6235
|
It follows the model of OAuth 2.
|
5967
6236
|
For example, they are used in Aspera on Cloud.
|
5968
6237
|
This is also available for developers for any application integrating Aspera.
|
5969
|
-
In this API, files, users and groups are identified by an
|
6238
|
+
In this API, files, users and groups are identified by an ID (a `String`, e.g. `"125"`, not necessarily numerical).
|
5970
6239
|
|
5971
|
-
Bearer tokens are typically generated by the
|
6240
|
+
Bearer tokens are typically generated by the authenticating application and then recognized by the Node API.
|
5972
6241
|
A bearer token is authorized on the node by creating `permissions` on a **folder**.
|
5973
6242
|
|
5974
|
-
Bearer tokens can be generated using command `bearer_token`: it takes two arguments:
|
6243
|
+
Bearer tokens can be generated using `ascli` command `bearer_token`: it takes two arguments:
|
5975
6244
|
|
5976
|
-
- The private key used to sign the token
|
6245
|
+
- The private key used to sign the token.
|
5977
6246
|
- The token information, which is a `Hash` containing the following elements:
|
5978
6247
|
|
5979
|
-
|
|
6248
|
+
| Parameter | Default | Type | Description |
|
5980
6249
|
| ------------------------ |-----------------------------|-----------|----------------------------------------------------------|
|
5981
|
-
| _scope
|
5982
|
-
| _validity
|
5983
|
-
| user_id
|
5984
|
-
| scope
|
5985
|
-
| expires_at
|
5986
|
-
| auth_type
|
5987
|
-
| group_ids
|
5988
|
-
| organization_id
|
5989
|
-
| watermarking_json_base64
|
6250
|
+
| `_scope` | `user:all` | Special | Either `user:all` or `admin:all` |
|
6251
|
+
| `_validity` | 86400 | Special | Validity in seconds from now. |
|
6252
|
+
| `user_id` | - | Mandatory | Identifier of user |
|
6253
|
+
| `scope` | `node.<access_key>:<_scope>`| Mandatory | API scope, e.g. `node.<access_key>:<node scope>` |
|
6254
|
+
| `expires_at` | `now+<_validity>` | Mandatory | Format: `%Y-%m-%dT%H:%M:%SZ` e.g. `2021-12-31T23:59:59Z` |
|
6255
|
+
| `auth_type` | `access_key` | Optional | `access_key`, `node_user` |
|
6256
|
+
| `group_ids` | - | Optional | List of group IDs |
|
6257
|
+
| `organization_id` | - | Optional | Organization ID |
|
6258
|
+
| `watermarking_json_base64` | - | Optional | Watermarking information (not used) |
|
5990
6259
|
|
5991
6260
|
> **Note:** For convenience, `ascli` provides additional parameters `_scope` and `_validity`.
|
5992
6261
|
> They are not part of the API and are removed from the final payload.
|
@@ -5995,7 +6264,7 @@ Bearer tokens can be generated using command `bearer_token`: it takes two argume
|
|
5995
6264
|
#### Bearer token: Environment
|
5996
6265
|
|
5997
6266
|
An access key shall be created to grant access for transfers to its storage.
|
5998
|
-
The
|
6267
|
+
The access key and its secret represent administrative access to the storage as it has access rights to the whole storage of the access key.
|
5999
6268
|
|
6000
6269
|
They way to create access keys depend slightly on the type of HSTS:
|
6001
6270
|
|
@@ -6007,36 +6276,40 @@ They way to create access keys depend slightly on the type of HSTS:
|
|
6007
6276
|
|
6008
6277
|
- If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
|
6009
6278
|
|
6279
|
+
> **Note:** Refer to [HSTS manual](https://www.ibm.com/docs/en/ahts): `Access key authentication` section for more details on access key creation.
|
6280
|
+
|
6281
|
+
In the next sections, we will assume that an access key has been created and that `ascli` is configured to use this access key by default using `node`.
|
6282
|
+
|
6010
6283
|
#### Bearer token: Preparation
|
6011
6284
|
|
6012
6285
|
Let's assume that the access key was created, and a default configuration is set to use this **access key**.
|
6013
6286
|
Using `ascli`, an access key can be created using the `access_key create` on the node (using main node credentials) or ATS.
|
6014
6287
|
|
6015
|
-
|
6288
|
+
Create a private key (organization key) that will be used to sign bearer tokens:
|
6016
6289
|
|
6017
|
-
|
6018
|
-
|
6019
|
-
|
6020
|
-
|
6290
|
+
```bash
|
6291
|
+
my_private_pem=./myorgkey.pem
|
6292
|
+
ascli config genkey $my_private_pem
|
6293
|
+
```
|
6021
6294
|
|
6022
|
-
|
6023
|
-
|
6295
|
+
> **Note:** This key is not used for authentication, it is used to sign bearer tokens.
|
6296
|
+
> Refer to section [private key](#private-key) for more details on generation.
|
6024
6297
|
|
6025
|
-
|
6298
|
+
The corresponding public key shall be placed as an attribute of the **access key** (done with `PUT /access_keys/<id>`):
|
6026
6299
|
|
6027
|
-
|
6028
|
-
|
6029
|
-
|
6300
|
+
```bash
|
6301
|
+
ascli node access_key set_bearer_key self @file:$my_private_pem
|
6302
|
+
```
|
6030
6303
|
|
6031
|
-
|
6032
|
-
|
6033
|
-
|
6304
|
+
> **Note:** Either the public or private key can be provided, and only the public key is used.
|
6305
|
+
> This will enable to check the signature of the bearer token.
|
6306
|
+
> Above command is executed with access key credentials.
|
6034
6307
|
|
6035
|
-
|
6308
|
+
Alternatively, use the following equivalent command, as `ascli` kindly extracts the public key with extension `.pub`:
|
6036
6309
|
|
6037
|
-
|
6038
|
-
|
6039
|
-
|
6310
|
+
```bash
|
6311
|
+
ascli node access_key modify %id:self @ruby:'{token_verification_key: File.read("'$my_private_pem'.pub")}'
|
6312
|
+
```
|
6040
6313
|
|
6041
6314
|
#### Bearer token: Configuration for user
|
6042
6315
|
|
@@ -6048,14 +6321,14 @@ Using `ascli`, an access key can be created using the `access_key create` on the
|
|
6048
6321
|
|
6049
6322
|
> **Note:** Here we simply select `/`, but any folder can be selected in the access key storage.
|
6050
6323
|
|
6051
|
-
- Let's designate a user by its
|
6324
|
+
- Let's designate a user by its ID:
|
6052
6325
|
|
6053
6326
|
```bash
|
6054
6327
|
my_user_id=777
|
6055
6328
|
```
|
6056
6329
|
|
6057
6330
|
> **Note:** This is an arbitrary identifier, typically managed by the web application.
|
6058
|
-
> Not related to Linux user
|
6331
|
+
> Not related to Linux user IDs or anything else.
|
6059
6332
|
|
6060
6333
|
- Grant this user access to the selected folder:
|
6061
6334
|
|
@@ -6069,18 +6342,20 @@ Using `ascli`, an access key can be created using the `access_key create` on the
|
|
6069
6342
|
ascli node bearer_token @file:./myorgkey.pem @json:'{"user_id":"'$my_user_id'","_validity":3600}' --output=bearer.txt
|
6070
6343
|
```
|
6071
6344
|
|
6345
|
+
> **Note:** The Bearer token can also be created using command `asnodeadmin` on HSTS. Refer to the [HSTS manual](https://www.ibm.com/docs/en/ahts): `Bearer tokens` section. Code for token generation is provided in [`lib/aspera/api/node.rb`](lib/aspera/api/node.rb)
|
6346
|
+
|
6072
6347
|
#### Bearer token: User side
|
6073
6348
|
|
6074
6349
|
Now, let's assume we are the user, the only information received are:
|
6075
6350
|
|
6076
|
-
- The URL of the
|
6351
|
+
- The URL of the Node API
|
6077
6352
|
- A Bearer token
|
6078
|
-
- A file
|
6353
|
+
- A file ID for which we have access
|
6079
6354
|
|
6080
6355
|
Let's use it:
|
6081
6356
|
|
6082
6357
|
```bash
|
6083
|
-
ascli node -N --url
|
6358
|
+
ascli node -N --url=https://... --password="Bearer $(cat bearer.txt)" --root-id=$my_folder_id access_key do self br /
|
6084
6359
|
```
|
6085
6360
|
|
6086
6361
|
### Node sample commands
|
@@ -6120,6 +6395,7 @@ async show ALL
|
|
6120
6395
|
basic_token
|
6121
6396
|
bearer_token @file:my_private_key @json:'{"user_id":"666"}' --output=bearer_666
|
6122
6397
|
browse / --log-level=trace2
|
6398
|
+
cat my_upload_folder/test_file.bin --to-folder=.
|
6123
6399
|
central file list
|
6124
6400
|
central file modify --validator=1 @json:'{"files":[]}'
|
6125
6401
|
central session list
|
@@ -6127,7 +6403,6 @@ delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder
|
|
6127
6403
|
delete my_upload_folder/test_file.bin
|
6128
6404
|
download my_upload_folder/test_file.bin --to-folder=.
|
6129
6405
|
health
|
6130
|
-
http_node_download my_upload_folder/test_file.bin --to-folder=.
|
6131
6406
|
info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
6132
6407
|
license
|
6133
6408
|
mkdir my_upload_folder/a_folder
|
@@ -6167,6 +6442,43 @@ upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_
|
|
6167
6442
|
watch_folder list
|
6168
6443
|
```
|
6169
6444
|
|
6445
|
+
### Open Telemetry
|
6446
|
+
|
6447
|
+
The `node` plugin supports Open Telemetry (OTel) for monitoring and tracing.
|
6448
|
+
|
6449
|
+
> **Note:** This is an experimental feature and currently only available for the `node` plugin and Instana backend.
|
6450
|
+
|
6451
|
+
`ascli` polls the Node API for transfer events and sends them to an OTel collector.
|
6452
|
+
|
6453
|
+
The command expects the following parameters provided as a `Hash` positional parameter:
|
6454
|
+
|
6455
|
+
| Parameter | Type | Default | Description |
|
6456
|
+
|-------------|----------|---------|---------------------------------------------------|
|
6457
|
+
| `url` | `String` | - | URL of the Instana HTTPS backend for OTel. |
|
6458
|
+
| `key` | `String` | - | Agent key for the backend. |
|
6459
|
+
| `interval` | `Float` | 10 | Polling interval in seconds. `0` for single shot. |
|
6460
|
+
|
6461
|
+
To retrieve OTel backend information: Go to the Instana web interface, **More** → **Agents** → **Docker** and identify the agent endpoint and key, e.g. `endpoint=ingress-blue-saas.instana.io`.
|
6462
|
+
Identify the region and the endpoint URL will be `https://otlp-[region]-saas.instana.io`, i.e. replace `ingress` with `otlp`.
|
6463
|
+
|
6464
|
+
For convenience, those parameters can be provided in a preset, e.g. named `otel_default`.
|
6465
|
+
|
6466
|
+
```bash
|
6467
|
+
ascli config preset init otel_default @json:'{"url":"https://otlp-orange-saas.instana.io:4318","key":"*********","interval":1.1}'
|
6468
|
+
```
|
6469
|
+
|
6470
|
+
Then it is invoked like this (assuming a default node is configured):
|
6471
|
+
|
6472
|
+
```bash
|
6473
|
+
ascli node telemetry @preset:otel_default
|
6474
|
+
```
|
6475
|
+
|
6476
|
+
In Instana, create a custom Dashboard to visualize the OTel data:
|
6477
|
+
|
6478
|
+
- Add Widget: Histogram
|
6479
|
+
- Data Source: Infrastructure and Platforms
|
6480
|
+
- Metric: search `transfer`
|
6481
|
+
|
6170
6482
|
## Plugin: `faspex5`: IBM Aspera Faspex v5
|
6171
6483
|
|
6172
6484
|
IBM Aspera's newer self-managed application.
|
@@ -6182,7 +6494,7 @@ IBM Aspera's newer self-managed application.
|
|
6182
6494
|
|
6183
6495
|
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
6184
6496
|
|
6185
|
-
For a quick start, one can use the wizard, which will help
|
6497
|
+
For a quick start, one can use the wizard, which will help to create an [Option Preset](#option-preset):
|
6186
6498
|
|
6187
6499
|
```bash
|
6188
6500
|
ascli config wizard
|
@@ -6204,7 +6516,7 @@ option: key_path>
|
|
6204
6516
|
Using existing key:
|
6205
6517
|
/Users/someuser/.aspera/ascli/my_key
|
6206
6518
|
option: username> someuser@example.com
|
6207
|
-
Ask the ascli client
|
6519
|
+
Ask the ascli client ID and secret to your Administrator.
|
6208
6520
|
Admin should login to: https://faspex5.example.com/aspera/faspex
|
6209
6521
|
Navigate to: :: → Admin → Configurations → API clients
|
6210
6522
|
Create an API client with:
|
@@ -6243,11 +6555,11 @@ Activation is in two steps:
|
|
6243
6555
|
- As Admin, Navigate to the web UI: Admin → Configurations → API Clients → Create
|
6244
6556
|
- Give a name, like `ascli`
|
6245
6557
|
- Activate JWT
|
6246
|
-
- There is an option to set a
|
6558
|
+
- There is an option to set a global public key allowing the owner of the private key to impersonate any user. Unless you want to do this, leave this field empty.
|
6247
6559
|
- Click on `Create` Button
|
6248
|
-
- Take note of Client
|
6560
|
+
- Take note of Client ID (and Client Secret, but not used in current version)
|
6249
6561
|
|
6250
|
-
- The user will authenticate with a private key and set the public key in his
|
6562
|
+
- The user will authenticate with a private key and set the public key in his Faspex 5 profile.
|
6251
6563
|
|
6252
6564
|
**Note:** If you don't have a private key refer to section [Private Key](#private-key) to generate one.
|
6253
6565
|
|
@@ -6289,7 +6601,7 @@ The administrator must create an API client in Faspex for an external web app su
|
|
6289
6601
|
- Do not Activate JWT
|
6290
6602
|
- Set **Redirect URI** to `https://127.0.0.1:8888`
|
6291
6603
|
- Click on `Create` Button
|
6292
|
-
- Take note of Client Id (and Client Secret, but not used in current version)
|
6604
|
+
- Take note of the Client Id (and Client Secret, but not used in current version)
|
6293
6605
|
|
6294
6606
|
The user will use the following options:
|
6295
6607
|
|
@@ -6351,7 +6663,7 @@ admin smtp show
|
|
6351
6663
|
admin smtp test my_email_external
|
6352
6664
|
admin workgroups list
|
6353
6665
|
bearer_token
|
6354
|
-
gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
6666
|
+
gateway --pid-file=pid_f5_fxgw @json:'{"url":"https://localhost:12346/aspera/faspex"}' &
|
6355
6667
|
health
|
6356
6668
|
invitation list
|
6357
6669
|
invitations create @json:'{"email_address":"aspera.user1+u@gmail.com"}'
|
@@ -6365,7 +6677,7 @@ packages receive --box=my_workgroup --group-type=workgroups workgroup_package_id
|
|
6365
6677
|
packages receive ALL --once-only=yes --to-folder=.
|
6366
6678
|
packages receive INIT --once-only=yes
|
6367
6679
|
packages receive f5_pack_id --to-folder=. --ts=@json:'{"content_protection_password":"my_secret_here"}'
|
6368
|
-
packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file --fields=id --display=data --output=
|
6680
|
+
packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file --fields=id --display=data --output=f5_pack_id
|
6369
6681
|
packages send --url=my_public_link_send_f5_user @json:'{"title":"test title"}' test_file.bin
|
6370
6682
|
packages send --url=my_public_link_send_shared_box @json:'{"title":"test title"}' test_file.bin
|
6371
6683
|
packages send @json:'{"title":"test title","recipients":["my_shared_box_name"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' test_file.bin
|
@@ -6375,7 +6687,7 @@ packages show --box=my_shared_box_name package_box_id1
|
|
6375
6687
|
packages show --box=my_workgroup --group-type=workgroups workgroup_package_id1
|
6376
6688
|
packages show f5_pack_id
|
6377
6689
|
packages status f5_pack_id
|
6378
|
-
postprocessing --pid-file=pid_f5_postproc @json:'{"url":"
|
6690
|
+
postprocessing --pid-file=pid_f5_postproc @json:'{"url":"https://localhost:8553/asclihook","script_folder":"","cert":"localhost.p12","key":"changeit"}' &
|
6379
6691
|
shared browse %name:my_src
|
6380
6692
|
shared list
|
6381
6693
|
shared_folders browse %name:my_shared_folder_name
|
@@ -6428,7 +6740,7 @@ Example (assuming a default preset is created for the connection information):
|
|
6428
6740
|
ascli faspex5 packages send @json:'{"title":"some title","recipients":["user@example.com"]}' mybygfile1
|
6429
6741
|
```
|
6430
6742
|
|
6431
|
-
Longer example for the
|
6743
|
+
Longer example for the payload of `@json:`:
|
6432
6744
|
|
6433
6745
|
```json
|
6434
6746
|
{"title":"some title","recipients":[{"recipient_type":"user","name":"user@example.com"}]}
|
@@ -6445,10 +6757,10 @@ Longer example for the ayload of `@json:`:
|
|
6445
6757
|
`ascli` adds some convenience:
|
6446
6758
|
The API expects the field `recipients` to be an `Array` of `Hash`, each with field `name` and optionally `recipient_type`.
|
6447
6759
|
`ascli` also accepts an `Array` of `String`, with simply a recipient name.
|
6448
|
-
Then, `ascli` will
|
6760
|
+
Then, `ascli` will look up existing contacts among all possible types, use it if a single match is found, and set the `name` and `recipient_type` accordingly.
|
6449
6761
|
Else an exception is sent.
|
6450
6762
|
|
6451
|
-
> **Note:** The lookup is case
|
6763
|
+
> **Note:** The lookup is case-insensitive and on partial matches.
|
6452
6764
|
|
6453
6765
|
```json
|
6454
6766
|
{"title":"some title","recipients":["user@example.com"]}
|
@@ -6477,11 +6789,11 @@ Option `box` can be used to list packages from a specific box (see [Inbox Select
|
|
6477
6789
|
|
6478
6790
|
Option `query` can be used to filter the list of packages, based on native API parameters, directly sent to [Faspex 5 API `GET /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#getAllPackages).
|
6479
6791
|
|
6480
|
-
|
|
6792
|
+
| Parameter | Type | Description |
|
6481
6793
|
|-----------|---------|-------------|
|
6482
6794
|
| `offset` | Native | Managed by `ascli`: Offset of first package. Default: 0 |
|
6483
6795
|
| `limit` | Native | Managed by `ascli`: # of packages per API call. Default: 100 |
|
6484
|
-
| `q` | Native | General search string (case
|
6796
|
+
| `q` | Native | General search string (case-insensitive, matches if value is contained in several fields) |
|
6485
6797
|
| ... | Native | Other native parameters are supported (Refer to API documentation) |
|
6486
6798
|
| `max` | Special | Maximum number of items to retrieve (stop pages when the maximum is passed) |
|
6487
6799
|
| `pmax` | Special | Maximum number of pages to request (stop pages when the maximum is passed) |
|
@@ -6504,32 +6816,43 @@ Examples:
|
|
6504
6816
|
ascli faspex5 packages list @ruby:'->(p){p["state"].eql?("released")}'
|
6505
6817
|
```
|
6506
6818
|
|
6507
|
-
### Faspex 5:
|
6819
|
+
### Faspex 5: Browsing folder content
|
6508
6820
|
|
6509
|
-
|
6821
|
+
Several entities support folder browsing: Packages, Nodes, Shared Folders.
|
6822
|
+
All support two modes: paging and legacy API.
|
6823
|
+
By default, paging is used.
|
6510
6824
|
|
6511
|
-
|
6825
|
+
Option `query` is available with parameters supported by the API and `ascli` :
|
6512
6826
|
|
6513
|
-
|
6827
|
+
| Parameter | Evaluation | Description |
|
6828
|
+
|--------------|--------------|-------------|
|
6829
|
+
| `paging` | `ascli` | Use paging API. Default: `true` |
|
6830
|
+
| `recursive` | `ascli` | List inside folders. Default: `false` |
|
6831
|
+
| `max` | `ascli` | Maximum number of items. |
|
6832
|
+
| `filter` | API | Refer to API doc. Default: `{"basenames":[]}` |
|
6833
|
+
| `offset` | API (legacy) | Index of first item. Default: `0` |
|
6834
|
+
| `limit` | API (legacy) | Number of items in one API call result. Default: `500` |
|
6835
|
+
| `per_page` | API (paging) | Number of items in one API call result. Default: `500` |
|
6514
6836
|
|
6515
|
-
|
6837
|
+
### Faspex 5: Content of a received Package
|
6516
6838
|
|
6517
|
-
> **Note:**
|
6839
|
+
> **Note:** Listing content also applies to sent packages using `--box=outbox`.
|
6518
6840
|
|
6519
|
-
To
|
6841
|
+
To list the content of a received package, use command `faspex5 packages browse <package id>`.
|
6842
|
+
Optionally, provide a folder path.
|
6520
6843
|
|
6521
6844
|
### Faspex 5: Receive a package
|
6522
6845
|
|
6523
6846
|
To receive one, or several packages at once, use command `faspex5 packages receive`.
|
6524
|
-
Provide either a single package
|
6847
|
+
Provide either a single package ID, or an extended value `Array` of package IDs, e.g. `@list:,1,2,3` as argument.
|
6525
6848
|
|
6526
6849
|
The same options as for `faspex5 packages list` can be used to select the box and filter the packages to download.
|
6527
6850
|
I.e. options `box` and `query`, as well as last **Command Parameter** `Proc` (filter).
|
6528
6851
|
|
6529
6852
|
Option `--once-only=yes` can be used, for "cargo-like" behavior.
|
6530
|
-
Special package
|
6853
|
+
Special package ID `INIT` initializes the persistency of already received packages when option `--once-only=yes` is used.
|
6531
6854
|
|
6532
|
-
Special package
|
6855
|
+
Special package ID `ALL` selects all packages (of the selected box).
|
6533
6856
|
In this case, typically, only `completed` packages should be downloaded, so use option `--query=@json:'{"status":"completed"}'`.
|
6534
6857
|
|
6535
6858
|
If a package is password protected, then the content protection password is asked interactively.
|
@@ -6654,7 +6977,7 @@ To start package purge, i.e. permanently remove packages marked for deletion old
|
|
6654
6977
|
ascli faspex5 admin clean_deleted
|
6655
6978
|
```
|
6656
6979
|
|
6657
|
-
> **Note:** The expiration
|
6980
|
+
> **Note:** The expiration period taken by default is the one from `admin configuration show`. To use a different period than the default, specify it on command line with: `@json:'{"days_before_deleting_package_records":15}'`
|
6658
6981
|
|
6659
6982
|
To delete all packages, one can use the following command:
|
6660
6983
|
|
@@ -6667,7 +6990,7 @@ ascli faspex5 packages list --box=ALL --format=yaml --fields=id | ascli faspex5
|
|
6667
6990
|
|
6668
6991
|
### Faspex 5: Admin: Unlock user
|
6669
6992
|
|
6670
|
-
To unlock a user, you can
|
6993
|
+
To unlock a user, you can deactivate and then re-activate the user:
|
6671
6994
|
|
6672
6995
|
```bash
|
6673
6996
|
ascli faspex5 admin accounts modify %name:some.user@example.com @json:'{"account_activated":false}'
|
@@ -6677,89 +7000,83 @@ ascli faspex5 admin accounts modify %name:some.user@example.com @json:'{"account
|
|
6677
7000
|
ascli faspex5 admin accounts modify %name:some.user@example.com @json:'{"account_activated":true}'
|
6678
7001
|
```
|
6679
7002
|
|
6680
|
-
> **Note:**
|
7003
|
+
> **Note:** Here we use the convenient percent selector, but the numerical ID can be used as well.
|
6681
7004
|
|
6682
7005
|
To send a password reset link to a user, use command `reset_password` on the `account`.
|
6683
7006
|
|
6684
|
-
### Faspex 5: Faspex 4-style
|
7007
|
+
### Faspex 5: Faspex 4-style post-processing
|
6685
7008
|
|
6686
|
-
The command
|
6687
|
-
It implements Faspex 5
|
6688
|
-
Environment variables at set to the values provided by the web hook which are the same as Faspex 4
|
7009
|
+
The command `ascli faspex5 postprocessing` emulates Faspex 4 post-processing script execution in Faspex 5.
|
7010
|
+
It implements a web hook for Faspex 5 and calls a script with the same environment variables as set by Faspex 4.
|
7011
|
+
Environment variables at set to the values provided by the web hook which are the same as Faspex 4 post-processing.
|
6689
7012
|
|
6690
|
-
It allows to quickly migrate workflows to Faspex 5 while preserving scripts.
|
6691
|
-
Nevertheless, on long term, a native approach shall be considered, such as using Aspera Orchestrator or other workflow engine.
|
7013
|
+
It allows to quickly migrate workflows from Faspex 4 to Faspex 5 while preserving scripts.
|
7014
|
+
Nevertheless, on long term, a native approach shall be considered, such as using Aspera Orchestrator or other workflow engine, using Faspex 5 native web hooks or File Processing.
|
6692
7015
|
|
6693
7016
|
It is invoked like this:
|
6694
7017
|
|
6695
7018
|
```bash
|
6696
|
-
ascli faspex5 postprocessing
|
7019
|
+
ascli faspex5 postprocessing
|
6697
7020
|
```
|
6698
7021
|
|
6699
|
-
|
7022
|
+
An optional positional parameter can be provided as extended value `Hash`:
|
6700
7023
|
|
6701
|
-
|
|
6702
|
-
|
6703
|
-
|
|
6704
|
-
|
|
6705
|
-
|
|
6706
|
-
|
|
6707
|
-
| certificate.chain | `String` | nil | Path to certificate chain |
|
6708
|
-
| processing | `Hash` | nil | Behavior of post processing |
|
6709
|
-
| processing.script_folder | `String` | . | Prefix added to script path |
|
6710
|
-
| processing.fail_on_error | `Bool` | false | Fail if true and process exits with non zero code |
|
6711
|
-
| processing.timeout_seconds | `Integer`| 60 | Max. execution time before script is killed |
|
7024
|
+
| Parameter | Type | Default | Description |
|
7025
|
+
|-------------------|----------|-------|-----------------------------------------------------|
|
7026
|
+
| **server info** | - | - | See [Web service](#web-service). |
|
7027
|
+
| `script_folder` | `String` | `.` | Prefix added to script path (Default: CWD) |
|
7028
|
+
| `fail_on_error` | `Bool` | false | Fail if true and process exits with non-zero code |
|
7029
|
+
| `timeout_seconds` | `Integer`| 60 | Time out before script is killed |
|
6712
7030
|
|
6713
|
-
|
6714
|
-
|
6715
|
-
- If `http` or `https` is used
|
6716
|
-
- The local port number (default 443 for HTTPS, 80 for HTTP)
|
6717
|
-
- The **base path**, i.e. the path under which requests are received, if a reverse proxy is used this can be used to route.
|
6718
|
-
|
6719
|
-
When a request is received the following happens:
|
7031
|
+
When a request on `ascli` is received the following happens:
|
6720
7032
|
|
6721
7033
|
- `ascli` gets the path of the URL called
|
6722
7034
|
- It removes the **base path** of base URL.
|
6723
7035
|
- It prepends it with the value of `script_folder`
|
6724
|
-
- It executes the script
|
7036
|
+
- It executes the script at that path
|
6725
7037
|
- Upon success, a success code is returned
|
6726
7038
|
|
6727
7039
|
For example:
|
6728
7040
|
|
6729
|
-
|
6730
|
-
|
7041
|
+
```bash
|
7042
|
+
ascli faspex5 postprocessing @json:'{"url":"http://localhost:8080/processing","script_folder":"/opt/scripts"}'
|
7043
|
+
```
|
6731
7044
|
|
6732
7045
|
In Faspex 5, the URL of the webhook endpoint shall be reachable from within Faspex containers.
|
6733
7046
|
For example, if `ascli` in running in the base host, the URL hostname shall not be localhost, as this refers to the local address inside Faspex container.
|
6734
|
-
Instead, one can specify the IP address of the host or `host.containers.internal
|
7047
|
+
Instead, one can specify the **IP address of the host** or `host.containers.internal` (Check `podman` manual).
|
7048
|
+
|
7049
|
+
Let's define the web hook:
|
6735
7050
|
|
6736
7051
|
**Webhook endpoint URI** : `http://host.containers.internal:8080/processing/script1.sh`
|
6737
7052
|
|
6738
|
-
Then the
|
7053
|
+
Then the post-processing script executed will be `/opt/scripts/script1.sh`.
|
6739
7054
|
|
6740
7055
|
### Faspex 5: Faspex 4 Gateway
|
6741
7056
|
|
6742
7057
|
> **Note:** This is not a feature for production. It's provided for testing only.
|
6743
7058
|
|
6744
|
-
For legacy
|
7059
|
+
For legacy Faspex client applications that use the `send` API (only) of Faspex v4, the command `gateway` provides the capability to present an API compatible with Faspex 4, and it will call the Faspex 5 API.
|
6745
7060
|
|
6746
7061
|
It takes a single argument which is the URL at which the gateway will be located (locally):
|
6747
7062
|
|
6748
7063
|
```bash
|
6749
|
-
ascli faspex5 gateway https://localhost:12345/aspera/faspex
|
7064
|
+
ascli faspex5 gateway @json:'{"url":"https://localhost:12345/aspera/faspex"}'
|
6750
7065
|
```
|
6751
7066
|
|
6752
7067
|
There are many limitations:
|
6753
7068
|
|
6754
|
-
- It's only to emulate the Faspex 4 `send` API (send package)
|
6755
|
-
- No support for remote sources, only for an actual file transfer by the client
|
6756
|
-
- The client must use the transfer spec returned by the API (not faspe
|
6757
|
-
-
|
6758
|
-
-
|
6759
|
-
-
|
7069
|
+
- It's only to emulate the Faspex 4 `send` API (send package).
|
7070
|
+
- No support for remote sources, only for an actual file transfer by the client.
|
7071
|
+
- The client must use the transfer spec returned by the API (not `faspe:` URL).
|
7072
|
+
- Tags returned in transfer spec must be used in transfer.
|
7073
|
+
- Only a single authentication is possible (per gateway) on Faspex5.
|
7074
|
+
- No authentication of F4 side (ignored).
|
6760
7075
|
|
6761
7076
|
Behavior: The API client calls the Faspex 4 API on the gateway, then the gateway transforms this into a Faspex5 API call, which returns a transfer spec, which is returned to the calling client. The calling client uses this to start a transfer to HSTS which is actually managed by Faspex 5.
|
6762
7077
|
|
7078
|
+
For other parameters, see [Web service](#web-service).
|
7079
|
+
|
6763
7080
|
### Faspex 5: Get Bearer token to use API
|
6764
7081
|
|
6765
7082
|
If a command is missing, then it is still possible to execute command by calling directly the API on the command line using `curl`:
|
@@ -6770,7 +7087,7 @@ curl -H "Authorization: $(ascli ascli bearer)" https://faspex5.example.com/asper
|
|
6770
7087
|
|
6771
7088
|
## Plugin: `faspex`: IBM Aspera Faspex v4
|
6772
7089
|
|
6773
|
-
> **Note:** Faspex v4 is end of support since Sept. 30th, 2024. So this plugin for
|
7090
|
+
> **Note:** Faspex v4 is end of support since Sept. 30th, 2024. So this plugin for Faspex v4 is deprecated. If you still need to use Faspex4, then use `ascli` version 4.19.0 or earlier.
|
6774
7091
|
>
|
6775
7092
|
> **Note:** For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
|
6776
7093
|
|
@@ -6783,7 +7100,7 @@ Command: `faspex package list`
|
|
6783
7100
|
|
6784
7101
|
#### Option `box`
|
6785
7102
|
|
6786
|
-
By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
|
7103
|
+
By default, it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
|
6787
7104
|
|
6788
7105
|
#### Option `recipient`
|
6789
7106
|
|
@@ -6796,18 +7113,20 @@ A user can receive a package because the recipient is:
|
|
6796
7113
|
|
6797
7114
|
As inboxes may be large, it is possible to use the following query parameters:
|
6798
7115
|
|
6799
|
-
|
6800
|
-
|
6801
|
-
|
6802
|
-
|
6803
|
-
|
7116
|
+
| Parameter | Evaluation | Description |
|
7117
|
+
|--------------|------------|-------------|
|
7118
|
+
| `count` | API | Number of items in one API call result (default=0, equivalent to 10) |
|
7119
|
+
| `page` | API | ID of page in call (default=0) |
|
7120
|
+
| `startIndex` | API | Index of item to start (default=0) |
|
7121
|
+
| `max` | `ascli` | Maximum number of items |
|
7122
|
+
| `pmax` | `ascli` | Maximum number of pages |
|
6804
7123
|
|
6805
7124
|
(SQL query is `LIMIT <startIndex>, <count>`)
|
6806
7125
|
|
6807
7126
|
The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under **Services (API v.3)**.
|
6808
7127
|
|
6809
7128
|
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
6810
|
-
By default `count` is `0` (`10`), it can be increased to issue less HTTP calls.
|
7129
|
+
By default, `count` is `0` (`10`), it can be increased to issue less HTTP calls.
|
6811
7130
|
|
6812
7131
|
#### Example: List packages in dropbox
|
6813
7132
|
|
@@ -6821,7 +7140,7 @@ List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in recei
|
|
6821
7140
|
|
6822
7141
|
The command is `package recv`, possible methods are:
|
6823
7142
|
|
6824
|
-
- Provide a package
|
7143
|
+
- Provide a package ID with option `id`
|
6825
7144
|
- Provide a public link with option `link`
|
6826
7145
|
- Provide a `faspe:` URI with option `link`
|
6827
7146
|
|
@@ -6837,13 +7156,13 @@ ascli faspex package list --recipient='*dropbox_name'
|
|
6837
7156
|
ascli faspex package recv 125 --recipient='*dropbox_name'
|
6838
7157
|
```
|
6839
7158
|
|
6840
|
-
|
7159
|
+
If `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
|
6841
7160
|
|
6842
7161
|
### Sending a Package
|
6843
7162
|
|
6844
7163
|
The command is `faspex package send`.
|
6845
7164
|
Package information (title, note, metadata, options) is provided in option `delivery_info`.
|
6846
|
-
The
|
7165
|
+
The content of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages).
|
6847
7166
|
|
6848
7167
|
Example:
|
6849
7168
|
|
@@ -6859,7 +7178,7 @@ Additional optional parameters in mandatory option `delivery_info`:
|
|
6859
7178
|
- Package Note: : `"note":"note this and that"`
|
6860
7179
|
- Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
6861
7180
|
|
6862
|
-
It is possible to send from a remote source using option `remote_source`, providing either the numerical
|
7181
|
+
It is possible to send from a remote source using option `remote_source`, providing either the numerical ID, or the name of the remote source using percent selector: `%name:<name>`.
|
6863
7182
|
|
6864
7183
|
Remote source can be browsed if option `storage` is provided.
|
6865
7184
|
`storage` is a `Hash` extended value.
|
@@ -6867,11 +7186,11 @@ The key is the storage name, as listed in `source list` command.
|
|
6867
7186
|
The value is a `Hash` with the following keys:
|
6868
7187
|
|
6869
7188
|
- `node` is a `Hash` with keys: `url`, `username`, `password`
|
6870
|
-
- `path` is the
|
7189
|
+
- `path` is the sub-path inside the node, as configured in Faspex
|
6871
7190
|
|
6872
7191
|
### Email notification on transfer
|
6873
7192
|
|
6874
|
-
Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template
|
7193
|
+
Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template`.
|
6875
7194
|
|
6876
7195
|
Example:
|
6877
7196
|
|
@@ -6881,7 +7200,7 @@ ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipien
|
|
6881
7200
|
|
6882
7201
|
In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
|
6883
7202
|
|
6884
|
-
###
|
7203
|
+
### Operations on dropbox
|
6885
7204
|
|
6886
7205
|
Example:
|
6887
7206
|
|
@@ -6894,7 +7213,7 @@ ascli faspex v4 dropbox delete 36
|
|
6894
7213
|
### Remote sources
|
6895
7214
|
|
6896
7215
|
Faspex lacks an API to list the contents of a remote source (available in web UI).
|
6897
|
-
To
|
7216
|
+
To work around this, the Node API is used, for this it is required to set option: `storage` that links a storage name to a node configuration and sub path.
|
6898
7217
|
|
6899
7218
|
Example:
|
6900
7219
|
|
@@ -6913,7 +7232,7 @@ my_faspex_node:
|
|
6913
7232
|
password: MyNodePassword
|
6914
7233
|
```
|
6915
7234
|
|
6916
|
-
In this example, a
|
7235
|
+
In this example, a Faspex storage named `my_storage` exists in Faspex, and is located
|
6917
7236
|
under the docroot in `/mydir` (this must be the same as configured in Faspex).
|
6918
7237
|
The node configuration name is `my_faspex_node` here.
|
6919
7238
|
|
@@ -6968,7 +7287,7 @@ v4 workgroup list
|
|
6968
7287
|
|
6969
7288
|
## Plugin: `shares`: IBM Aspera Shares v1
|
6970
7289
|
|
6971
|
-
Aspera Shares supports the **
|
7290
|
+
Aspera Shares supports the **Node API** for the file transfer part.
|
6972
7291
|
|
6973
7292
|
Supported commands are listed in Share's API documentation:
|
6974
7293
|
|
@@ -6986,7 +7305,7 @@ user_id=$(ascli shares admin user list --select=@json:'{"username":"entity1"}' -
|
|
6986
7305
|
ascli shares admin share user_permissions $share_id create @json:'{"user_id":'$user_id',"browse_permission":true, "download_permission":true, "mkdir_permission":true,"delete_permission":true,"rename_permission":true,"content_availability_permission":true,"manage_permission":true}'
|
6987
7306
|
```
|
6988
7307
|
|
6989
|
-
To figure out
|
7308
|
+
To figure out an entity's payload, for example for creation, refer to the API documentation above.
|
6990
7309
|
|
6991
7310
|
### Shares sample commands
|
6992
7311
|
|
@@ -7011,6 +7330,8 @@ files delete my_share1/test_file.bin
|
|
7011
7330
|
files download --to-folder=. my_share1/test_file.bin
|
7012
7331
|
files download --to-folder=. my_share1/test_file.bin my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
7013
7332
|
files mkdir my_share1/new_folder
|
7333
|
+
files sync start push /data/local_syncsynctst /london-sh1/synctst --sync-info=@json:'{"quiet":false}'
|
7334
|
+
files sync start push /data/local_syncsynctst /london-sh1/synctst --sync-info=@json:'{"reset":true}'
|
7014
7335
|
files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
7015
7336
|
files upload --to-folder=https://shares.share1 sendfolder --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
7016
7337
|
files upload --to-folder=my_share1 test_file.bin
|
@@ -7069,7 +7390,7 @@ If you have those parameters already, then following options shall be provided:
|
|
7069
7390
|
- `bucket` bucket name
|
7070
7391
|
- `endpoint` storage endpoint URL, e.g. `https://s3.hkg02.cloud-object-storage.appdomain.cloud`
|
7071
7392
|
- `apikey` API Key
|
7072
|
-
- `crn` resource instance
|
7393
|
+
- `crn` resource instance ID
|
7073
7394
|
|
7074
7395
|
For example, let us create a default configuration:
|
7075
7396
|
|
@@ -7146,14 +7467,14 @@ ascli config preset set default cos mycos
|
|
7146
7467
|
|
7147
7468
|
Let's assume you created a default configuration from one of the two previous steps (else specify the access options on command lines).
|
7148
7469
|
|
7149
|
-
A subset of `node` plugin operations are supported, basically
|
7470
|
+
A subset of `node` plugin operations are supported, basically Node API:
|
7150
7471
|
|
7151
7472
|
```bash
|
7152
7473
|
ascli cos node info
|
7153
7474
|
ascli cos node upload 'faux:///sample1G?1g'
|
7154
7475
|
```
|
7155
7476
|
|
7156
|
-
> **Note:** A dummy file `sample1G` of size
|
7477
|
+
> **Note:** A dummy file `sample1G` of size 2 GB is generated using the `faux` PVCL scheme (see previous section and `man ascp`), but you can, of course, send a real file by specifying a real file path instead.
|
7157
7478
|
|
7158
7479
|
### Cos sample commands
|
7159
7480
|
|
@@ -7207,7 +7528,7 @@ health -N
|
|
7207
7528
|
## Plugin: `preview`: Preview generator for AoC
|
7208
7529
|
|
7209
7530
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
7210
|
-
It uses the **
|
7531
|
+
It uses the **Node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
7211
7532
|
Several options can be used to tune several aspects:
|
7212
7533
|
|
7213
7534
|
- Methods for detection of new files needing generation
|
@@ -7218,7 +7539,7 @@ See also <https://github.com/IBM/aspera-on-cloud-file-previews>
|
|
7218
7539
|
|
7219
7540
|
### Aspera Server configuration
|
7220
7541
|
|
7221
|
-
Specify the
|
7542
|
+
Specify the preview's folder as shown in:
|
7222
7543
|
|
7223
7544
|
<https://ibmaspera.com/help/admin/organization/installing_the_preview_maker>
|
7224
7545
|
|
@@ -7235,8 +7556,8 @@ asnodeadmin --reload
|
|
7235
7556
|
|
7236
7557
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
7237
7558
|
|
7238
|
-
The HSTS
|
7239
|
-
This size is internally capped to `1<<24` Bytes (16777216)
|
7559
|
+
The HSTS Node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 Bytes).
|
7560
|
+
This size is internally capped to `1<<24` Bytes (16777216), i.e. 16384 KB, i.e. 16 MB.
|
7240
7561
|
|
7241
7562
|
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
7242
7563
|
To display the value, use `asuserdata`:
|
@@ -7253,22 +7574,22 @@ asuserdata -a | grep max_request_file_create_size_kb
|
|
7253
7574
|
asconfigurator -x "server; max_request_file_create_size_kb,16384"
|
7254
7575
|
```
|
7255
7576
|
|
7256
|
-
If you use a value different
|
7577
|
+
If you use a value different from `16777216`, then specify it using option `max_size`.
|
7257
7578
|
|
7258
|
-
> **Note:** The HSTS parameter (`max_request_file_create_size_kb`) is in **
|
7579
|
+
> **Note:** The HSTS parameter (`max_request_file_create_size_kb`) is in **kilo Bytes** while the generator parameter is in **Bytes** (factor of 1024).
|
7259
7580
|
|
7260
7581
|
### External tools: Linux
|
7261
7582
|
|
7262
7583
|
`ascli` requires the following external tools available in the `PATH`:
|
7263
7584
|
|
7264
|
-
- **ImageMagick** v7+: `magick` `composite`
|
7585
|
+
- **ImageMagick** v7+: `magick` (for tools: `convert` and `composite`)
|
7265
7586
|
- **OptiPNG** : `optipng`
|
7266
7587
|
- **FFmpeg** : `ffmpeg` `ffprobe`
|
7267
|
-
- **
|
7588
|
+
- **LibreOffice** : `unoconv`
|
7268
7589
|
|
7269
|
-
Here shown on
|
7590
|
+
Here shown on Red Hat/Rocky Linux.
|
7270
7591
|
|
7271
|
-
Other OSes should work as well, but are
|
7592
|
+
Other OSes should work as well, but are not tested.
|
7272
7593
|
|
7273
7594
|
To check if all tools are found properly, execute:
|
7274
7595
|
|
@@ -7283,24 +7604,37 @@ dnf install -y ImageMagick optipng
|
|
7283
7604
|
```
|
7284
7605
|
|
7285
7606
|
You may also install `ghostscript` which adds fonts to ImageMagick.
|
7286
|
-
Available fonts, used to generate
|
7607
|
+
Available fonts, used to generate PNG for text, can be listed with `magick identify -list font`.
|
7287
7608
|
Prefer ImageMagick version >=7.
|
7288
7609
|
|
7289
7610
|
More info on ImageMagick at <https://imagemagick.org/>
|
7290
7611
|
|
7612
|
+
If your OS has only ImageMagick v6, then you can create a script called `magick` and add it to your `PATH`:
|
7613
|
+
|
7614
|
+
```bash
|
7615
|
+
#!/bin/bash
|
7616
|
+
exec "$@"
|
7617
|
+
```
|
7618
|
+
|
7619
|
+
make it executable:
|
7620
|
+
|
7621
|
+
```bash
|
7622
|
+
chmod a+x /usr/local/bin/magick
|
7623
|
+
```
|
7624
|
+
|
7291
7625
|
#### Video: FFmpeg
|
7292
7626
|
|
7293
|
-
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
7627
|
+
The easiest method is to download and install the latest released version of `ffmpeg` with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
7294
7628
|
|
7295
7629
|
```bash
|
7296
7630
|
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
|
7297
7631
|
```
|
7298
7632
|
|
7299
|
-
#### Office: `unoconv` and
|
7633
|
+
#### Office: `unoconv` and LibreOffice
|
7300
7634
|
|
7301
7635
|
If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
|
7302
7636
|
|
7303
|
-
The generation of preview in based on the use of
|
7637
|
+
The generation of preview in based on the use of LibreOffice's `unoconv`.
|
7304
7638
|
|
7305
7639
|
- RHEL 8/Rocky Linux 8+
|
7306
7640
|
|
@@ -7357,13 +7691,14 @@ For video preview, the whole set of options can be overridden with option `reenc
|
|
7357
7691
|
|
7358
7692
|
### Execution
|
7359
7693
|
|
7360
|
-
`ascli` intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST
|
7694
|
+
`ascli` intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST API too quickly during the scan or event method).
|
7361
7695
|
It needs to be run on a regular basis to create or update preview files.
|
7362
7696
|
For that use your best reliable scheduler, see [Scheduler](#scheduler).
|
7363
7697
|
|
7364
|
-
Typically, for **Access key** access, the system/transfer is `xfer`.
|
7698
|
+
Typically, for **Access key** access, the system/transfer is `xfer`.
|
7699
|
+
So, in order to be consistent, and generate the appropriate access rights, the generation process should be run as user `xfer`.
|
7365
7700
|
|
7366
|
-
|
7701
|
+
Let's do a one shot test, using the configuration previously created:
|
7367
7702
|
|
7368
7703
|
```bash
|
7369
7704
|
su -s /bin/bash - xfer
|
@@ -7372,7 +7707,7 @@ ascli preview scan --overwrite=always
|
|
7372
7707
|
```
|
7373
7708
|
|
7374
7709
|
When the preview generator is first executed it will create a file: `.aspera_access_key`
|
7375
|
-
in the
|
7710
|
+
in the preview's folder which contains the access key used.
|
7376
7711
|
On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
|
7377
7712
|
|
7378
7713
|
### Configuration for Execution in scheduler
|
@@ -7440,15 +7775,15 @@ The preview generator supports rendering of those file categories:
|
|
7440
7775
|
|
7441
7776
|
To avoid generation for some categories, specify a list using option `skip_types`.
|
7442
7777
|
|
7443
|
-
Each category has a specific rendering method to produce the
|
7778
|
+
Each category has a specific rendering method to produce the PNG thumbnail.
|
7444
7779
|
|
7445
7780
|
The mp4 video preview file is only for category `video`
|
7446
7781
|
|
7447
|
-
File type is primarily based on file extension detected by the
|
7782
|
+
File type is primarily based on file extension detected by the Node API and translated info a mime type returned by the Node API.
|
7448
7783
|
|
7449
7784
|
### `mimemagic`
|
7450
7785
|
|
7451
|
-
By default, the Mime type used for conversion is the one returned by the
|
7786
|
+
By default, the Mime type used for conversion is the one returned by the Node API, based on file name extension.
|
7452
7787
|
|
7453
7788
|
It is also possible to detect the mime type using option `mimemagic`.
|
7454
7789
|
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`.
|
@@ -7509,6 +7844,7 @@ check --skip-types=office
|
|
7509
7844
|
events --once-only=yes --skip-types=office --log-level=info
|
7510
7845
|
scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
7511
7846
|
scan --skip-types=office --log-level=info
|
7847
|
+
show --base=test /etc/hosts
|
7512
7848
|
show --base=test my_docx
|
7513
7849
|
show --base=test my_mpg --video-png-conv=animated
|
7514
7850
|
show --base=test my_mpg --video-png-conv=fixed
|
@@ -7531,29 +7867,30 @@ An interface for the `async` utility is provided in the following plugins:
|
|
7531
7867
|
- `aoc files sync` (uses node)
|
7532
7868
|
- `shares files sync` (uses node)
|
7533
7869
|
|
7534
|
-
One advantage over the `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
7535
|
-
Moreover `ascli` supports sync with application requiring token-based authorization.
|
7870
|
+
One advantage of using `ascli` over the `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
7871
|
+
Moreover, `ascli` supports sync with application requiring token-based authorization.
|
7536
7872
|
|
7537
|
-
Some
|
7873
|
+
Some `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
|
7538
7874
|
|
7539
|
-
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`). The Transfer SDK 1.3 includes this.
|
7875
|
+
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`). The Aspera Transfer SDK 1.3+ includes this.
|
7540
7876
|
|
7541
|
-
`sync` supports 0 or 3 arguments
|
7877
|
+
`sync` supports 0 or 3 arguments.
|
7878
|
+
If 3 arguments are provided, they are applied to the first (and only) session and mapped, in that order, to:
|
7542
7879
|
|
7543
|
-
- direction
|
7544
|
-
- local
|
7545
|
-
- remote
|
7880
|
+
- `direction`
|
7881
|
+
- `local.path`
|
7882
|
+
- `remote.path`
|
7546
7883
|
|
7547
7884
|
Additional options can be provided with option `sync_info`, for which two syntax are possible, as follows.
|
7548
7885
|
|
7549
|
-
### `async` API and conf format
|
7886
|
+
### `async` API and `conf` format
|
7550
7887
|
|
7551
|
-
It is the same payload as specified on the option `--conf` of `async` or in
|
7552
|
-
This is the preferred syntax and allows a single session definition.
|
7888
|
+
It is the same payload as specified on the option `--conf` of `async` or in Node API `/asyncs`.
|
7889
|
+
This is the **preferred** syntax and allows a single session definition.
|
7553
7890
|
|
7554
|
-
> **Note:**
|
7891
|
+
> **Note:** By default, no progress, nor error messages is provided on terminal. To activate messages, set option `sync_info` parameter `quiet` to `false`.
|
7555
7892
|
|
7556
|
-
Documentation on Async
|
7893
|
+
Documentation on Async Node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
|
7557
7894
|
|
7558
7895
|
If 3 arguments are provided they are mapped to:
|
7559
7896
|
|
@@ -7567,16 +7904,10 @@ If 3 arguments are provided they are mapped to:
|
|
7567
7904
|
It is based on a JSON representation of `async` command line options.
|
7568
7905
|
It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
|
7569
7906
|
|
7570
|
-
This is the mode selection if there
|
7907
|
+
This is the mode selection if there are either keys `sessions` or `instance` in option `sync_info`.
|
7571
7908
|
|
7572
7909
|
> **Note:** Progress and error messages are provided on terminal like regular command line invocation of `async`.
|
7573
7910
|
|
7574
|
-
If 3 arguments are provided, they are applied to the first (and only) session and mapped to:
|
7575
|
-
|
7576
|
-
- `direction`
|
7577
|
-
- `local_dir`
|
7578
|
-
- `remote_dir`
|
7579
|
-
|
7580
7911
|
## Hot folder
|
7581
7912
|
|
7582
7913
|
### Requirements
|
@@ -7613,7 +7944,7 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
7613
7944
|
>
|
7614
7945
|
> **Note:** Usual native `ascp` arguments are available as standard [**transfer-spec**](#transfer-specification) parameters, but not special or advanced options.
|
7615
7946
|
>
|
7616
|
-
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info
|
7947
|
+
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info`.
|
7617
7948
|
|
7618
7949
|
#### Server side and configuration
|
7619
7950
|
|
@@ -7621,11 +7952,11 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
7621
7952
|
|
7622
7953
|
> **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
7623
7954
|
>
|
7624
|
-
> **Note:** Options may be saved in
|
7955
|
+
> **Note:** Options may be saved in an [Option Preset](#option-preset) and used with `-P`.
|
7625
7956
|
|
7626
7957
|
#### Scheduling
|
7627
7958
|
|
7628
|
-
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every
|
7959
|
+
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every minute, or 5 minutes, etc...
|
7629
7960
|
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
7630
7961
|
|
7631
7962
|
### Example: Upload hot folder
|
@@ -7634,7 +7965,7 @@ Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
|
7634
7965
|
ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"remove_after_transfer":true,"remove_empty_directories":true,"exclude_newer_than:-8,"src_base":"source_hot"}'
|
7635
7966
|
```
|
7636
7967
|
|
7637
|
-
The local folder (here, relative path: `source_hot`) is sent (upload) to an
|
7968
|
+
The local folder (here, relative path: `source_hot`) is sent (upload) to an Aspera server.
|
7638
7969
|
Source files are deleted after transfer.
|
7639
7970
|
Growing files will be sent only once they don't grow anymore (based on an 8-second cool-off period).
|
7640
7971
|
If a transfer takes more than the execution period, then the subsequent execution is skipped (`lock_port`) preventing multiple concurrent runs.
|
@@ -7696,13 +8027,13 @@ OK - [transfer:ok]
|
|
7696
8027
|
The `smtp` option is a `Hash` (extended value) with the following fields:
|
7697
8028
|
|
7698
8029
|
<!-- markdownlint-disable MD034 -->
|
7699
|
-
| field |
|
8030
|
+
| field | Default | example | Description |
|
7700
8031
|
|--------------|---------------------|----------------------------|----------------------------------|
|
7701
|
-
| `server` | - | smtp.gmail.com
|
7702
|
-
| `tls` | true
|
7703
|
-
| `ssl` | false
|
7704
|
-
| `port` | 587 or 465 or 25
|
7705
|
-
| `domain` | domain of server
|
8032
|
+
| `server` | - | `smtp.gmail.com` | SMTP server address |
|
8033
|
+
| `tls` | `true` | `true` | Enable `STARTTLS` (port 587) |
|
8034
|
+
| `ssl` | `false` | `false` | Enable `TLS` (port 465) |
|
8035
|
+
| `port` | `587` or `465` or `25` | `587` | Port for service |
|
8036
|
+
| `domain` | *domain of server* | gmail.com | Email domain of user |
|
7706
8037
|
| `username` | - | john@example.com | User to authenticate on SMTP server, leave empty for open auth. |
|
7707
8038
|
| `password` | - | my_password_here | Password for above username |
|
7708
8039
|
| `from_email` | username if defined | johnny@example.com | Address used if receiver replies |
|
@@ -7738,7 +8069,7 @@ ascli config preset set default config cli_default
|
|
7738
8069
|
|
7739
8070
|
### Email templates
|
7740
8071
|
|
7741
|
-
Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
|
8072
|
+
Sent emails are built using a template that uses the [`ERB`](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
|
7742
8073
|
|
7743
8074
|
The template is the full SMTP message, including headers.
|
7744
8075
|
|
@@ -7790,10 +8121,10 @@ Transfer is: <%=status%>
|
|
7790
8121
|
|
7791
8122
|
This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
|
7792
8123
|
|
7793
|
-
It aims at simplifying the startup of a FASP session from a programmatic
|
8124
|
+
It aims at simplifying the startup of a FASP session from a programmatic standpoint as formatting a [**transfer-spec**](#transfer-specification) is:
|
7794
8125
|
|
7795
8126
|
- Common to Aspera Node API (HTTP POST /ops/transfer)
|
7796
|
-
- Common to Aspera Connect API (browser
|
8127
|
+
- Common to Aspera Connect API (browser JavaScript `startTransfer`)
|
7797
8128
|
- Easy to generate by using any third party language specific JSON library
|
7798
8129
|
|
7799
8130
|
Hopefully, IBM integrates this directly in `ascp`, and this tool is made redundant.
|
@@ -7810,16 +8141,16 @@ During execution, it generates all low level events, one per line, in JSON forma
|
|
7810
8141
|
|
7811
8142
|
Top level parameters supported by `asession`:
|
7812
8143
|
|
7813
|
-
|
|
8144
|
+
| Parameter | Description |
|
7814
8145
|
|-----------|-------------|
|
7815
8146
|
| `spec` | the [**transfer-spec**](#transfer-specification) |
|
7816
8147
|
| `agent` | same parameters as transfer-info for agent `direct` |
|
7817
8148
|
| `loglevel` | log level of `asession` |
|
7818
|
-
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
|
8149
|
+
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default, it is `[system tmp folder]/[username]_asession_filelists` |
|
7819
8150
|
|
7820
8151
|
### Comparison of interfaces
|
7821
8152
|
|
7822
|
-
| feature/tool | Transfer SDK |
|
8153
|
+
| feature/tool | Transfer SDK | FASPManager | `ascp` | `asession` |
|
7823
8154
|
|-----------------------|--------------|---------------------------------|--------|------------|
|
7824
8155
|
| language integration | Many | C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/> | Any | Any |
|
7825
8156
|
| required additional components to `ascp` | Daemon | Library<br/>(+headers) | - | Ruby<br/>Aspera gem |
|
@@ -7843,9 +8174,9 @@ asession < session.json
|
|
7843
8174
|
|
7844
8175
|
### Asynchronous commands and Persistent session
|
7845
8176
|
|
7846
|
-
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter`
|
8177
|
+
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` → `long_parameter`
|
7847
8178
|
|
7848
|
-
This is particularly useful for a persistent session (
|
8179
|
+
This is particularly useful for a persistent session (with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true`)
|
7849
8180
|
|
7850
8181
|
```json
|
7851
8182
|
asession
|
@@ -7858,7 +8189,7 @@ asession
|
|
7858
8189
|
|
7859
8190
|
### Example of language wrapper
|
7860
8191
|
|
7861
|
-
|
8192
|
+
NodeJS: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/aspera)
|
7862
8193
|
|
7863
8194
|
### Help
|
7864
8195
|
|
@@ -7918,7 +8249,7 @@ So, it evolved into `ascli`:
|
|
7918
8249
|
|
7919
8250
|
- Portable: works on platforms supporting `ruby` (and `ascp`)
|
7920
8251
|
- Easy to install with the `gem` utility
|
7921
|
-
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable
|
8252
|
+
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable, more standard)
|
7922
8253
|
- `ruby` is consistent with other Aspera products
|
7923
8254
|
|
7924
8255
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
@@ -7938,7 +8269,7 @@ ascli config coffee
|
|
7938
8269
|
|
7939
8270
|
### Error: "Remote host is not who we expected"
|
7940
8271
|
|
7941
|
-
Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
8272
|
+
Cause: `ascp` >= 4.x checks fingerprint of the highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
7942
8273
|
|
7943
8274
|
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the configuration file):
|
7944
8275
|
|
@@ -7973,5 +8304,5 @@ OpenSSH keys only supported if ED25519 is available
|
|
7973
8304
|
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
7974
8305
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
7975
8306
|
|
7976
|
-
In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also
|
8307
|
+
In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also deactivated by default.
|
7977
8308
|
To re-activate, set env var `ASCLI_ENABLE_ECDSHA2` to `true`.
|