aspera-cli 4.12.0 → 4.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +45 -5
- data/CONTRIBUTING.md +113 -22
- data/README.md +1289 -754
- data/bin/ascli +3 -3
- data/examples/dascli +1 -1
- data/examples/rubyc +24 -0
- data/lib/aspera/aoc.rb +63 -74
- data/lib/aspera/ascmd.rb +5 -3
- data/lib/aspera/cli/basic_auth_plugin.rb +6 -6
- data/lib/aspera/cli/extended_value.rb +24 -37
- data/lib/aspera/cli/formatter.rb +23 -25
- data/lib/aspera/cli/info.rb +2 -4
- data/lib/aspera/cli/main.rb +27 -27
- data/lib/aspera/cli/manager.rb +143 -120
- data/lib/aspera/cli/plugin.rb +88 -43
- data/lib/aspera/cli/plugins/alee.rb +2 -2
- data/lib/aspera/cli/plugins/aoc.rb +235 -104
- data/lib/aspera/cli/plugins/ats.rb +16 -18
- data/lib/aspera/cli/plugins/bss.rb +3 -3
- data/lib/aspera/cli/plugins/config.rb +190 -373
- data/lib/aspera/cli/plugins/console.rb +4 -6
- data/lib/aspera/cli/plugins/cos.rb +12 -13
- data/lib/aspera/cli/plugins/faspex.rb +21 -21
- data/lib/aspera/cli/plugins/faspex5.rb +399 -150
- data/lib/aspera/cli/plugins/node.rb +260 -174
- data/lib/aspera/cli/plugins/orchestrator.rb +15 -18
- data/lib/aspera/cli/plugins/preview.rb +40 -62
- data/lib/aspera/cli/plugins/server.rb +33 -16
- data/lib/aspera/cli/plugins/shares.rb +24 -33
- data/lib/aspera/cli/plugins/sync.rb +6 -6
- data/lib/aspera/cli/transfer_agent.rb +47 -30
- data/lib/aspera/cli/version.rb +2 -1
- data/lib/aspera/colors.rb +9 -7
- data/lib/aspera/command_line_builder.rb +2 -1
- data/lib/aspera/cos_node.rb +1 -1
- data/lib/aspera/data/6 +0 -0
- data/lib/aspera/environment.rb +7 -3
- data/lib/aspera/fasp/agent_connect.rb +6 -1
- data/lib/aspera/fasp/agent_direct.rb +17 -17
- data/lib/aspera/fasp/agent_httpgw.rb +138 -60
- data/lib/aspera/fasp/agent_node.rb +14 -4
- data/lib/aspera/fasp/agent_trsdk.rb +2 -0
- data/lib/aspera/fasp/error_info.rb +2 -0
- data/lib/aspera/fasp/installation.rb +19 -19
- data/lib/aspera/fasp/parameters.rb +29 -20
- data/lib/aspera/fasp/parameters.yaml +5 -2
- data/lib/aspera/fasp/resume_policy.rb +3 -3
- data/lib/aspera/fasp/transfer_spec.rb +8 -5
- data/lib/aspera/fasp/uri.rb +23 -21
- data/lib/aspera/faspex_gw.rb +1 -0
- data/lib/aspera/faspex_postproc.rb +3 -3
- data/lib/aspera/hash_ext.rb +12 -2
- data/lib/aspera/keychain/macos_security.rb +13 -13
- data/lib/aspera/log.rb +1 -0
- data/lib/aspera/node.rb +73 -84
- data/lib/aspera/oauth.rb +4 -3
- data/lib/aspera/persistency_action_once.rb +1 -1
- data/lib/aspera/preview/file_types.rb +8 -6
- data/lib/aspera/preview/generator.rb +23 -11
- data/lib/aspera/preview/options.rb +3 -2
- data/lib/aspera/preview/terminal.rb +80 -0
- data/lib/aspera/preview/utils.rb +11 -11
- data/lib/aspera/proxy_auto_config.js +2 -2
- data/lib/aspera/rest.rb +42 -4
- data/lib/aspera/rest_call_error.rb +3 -1
- data/lib/aspera/secret_hider.rb +10 -5
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/sync.rb +41 -33
- data/lib/aspera/web_server_simple.rb +22 -18
- data.tar.gz.sig +0 -0
- metadata +40 -48
- metadata.gz.sig +0 -0
- data/docs/test_env.conf +0 -179
- data/examples/aoc.rb +0 -30
- data/examples/faspex4.rb +0 -94
- data/examples/node.rb +0 -96
- data/examples/server.rb +0 -93
- data/lib/aspera/data/7 +0 -0
data/README.md
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
# Command Line Interface for IBM Aspera products
|
2
2
|
<!-- markdownlint-disable MD033 MD003 MD053 -->
|
3
|
-
<!--
|
4
|
-
|
5
|
-
<!-- cSpell:ignoreRegExp /my[a-z]+/g -->
|
3
|
+
<!-- cspell:ignore devkit zcvf zxvf noded secondfile filesize sedemo eudemo webmail csum eascp loglevel cronfile magick keepalive inotify eastus bluemix trev sshfp struct genkey passout ibmaspera unpermitted schtasks taskschd dascli -->
|
4
|
+
|
6
5
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
7
6
|
|
8
7
|
##
|
9
8
|
|
10
|
-
Version : 4.
|
9
|
+
Version : 4.14.0
|
11
10
|
|
12
11
|
Laurent/2016-2023
|
13
12
|
|
@@ -19,9 +18,9 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
|
|
19
18
|
|
20
19
|
Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
|
21
20
|
|
22
|
-
Minimum required Ruby version: >= 2.
|
21
|
+
Minimum required Ruby version: >= 2.6.
|
23
22
|
|
24
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
23
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
25
24
|
|
26
25
|
[Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs&sort=title_asc)
|
27
26
|
[Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
|
@@ -58,7 +57,7 @@ So it is designed for:
|
|
58
57
|
If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
|
59
58
|
|
60
59
|
- Product APIs (REST) : e.g. AoC, Faspex, node
|
61
|
-
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java,
|
60
|
+
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Ruby, etc...)
|
62
61
|
|
63
62
|
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
64
63
|
|
@@ -93,7 +92,7 @@ ascli --version
|
|
93
92
|
```
|
94
93
|
|
95
94
|
```bash
|
96
|
-
4.
|
95
|
+
4.14.0
|
97
96
|
```
|
98
97
|
|
99
98
|
### First use
|
@@ -104,8 +103,6 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
|
|
104
103
|
|
105
104
|
To test with Aspera demo transfer server, setup the environment and then test:
|
106
105
|
|
107
|
-
<!-- spellchecker: disable -->
|
108
|
-
|
109
106
|
```bash
|
110
107
|
ascli config initdemo
|
111
108
|
```
|
@@ -115,18 +112,16 @@ ascli server browse /
|
|
115
112
|
```
|
116
113
|
|
117
114
|
```output
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
115
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
116
|
+
| zmode | zuid | zgid | size | mtime | name |
|
117
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
118
|
+
| drwxr-xr-x | asperaweb | asperaweb | 90112 | 2023-04-05 15:31:21 +0200 | Upload |
|
119
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:16 +0200 | aspera-test-dir-large |
|
120
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-small |
|
121
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-tiny |
|
122
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
126
123
|
```
|
127
124
|
|
128
|
-
<!-- spellchecker: enable -->
|
129
|
-
|
130
125
|
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [option preset](#lprt) for the server's authentication options. The following example will:
|
131
126
|
|
132
127
|
- create a [option preset](#lprt)
|
@@ -155,17 +150,17 @@ ascli server browse /aspera-test-dir-large
|
|
155
150
|
```
|
156
151
|
|
157
152
|
```output
|
158
|
-
|
159
|
-
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
164
|
-
|
165
|
-
|
166
|
-
|
167
|
-
|
168
|
-
|
153
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
154
|
+
| zmode | zuid | zgid | size | mtime | name |
|
155
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
156
|
+
| -r-xr-x--- | asperaweb | asperaweb | 104857600 | 2022-10-27 16:06:38 +0200 | 100MB |
|
157
|
+
| -r-xr-x--- | asperaweb | asperaweb | 10737418240 | 2022-10-27 16:08:12 +0200 | 10GB |
|
158
|
+
| -r-xr-x--- | asperaweb | asperaweb | 500000000000 | 2022-10-27 16:06:26 +0200 | 500GB |
|
159
|
+
| -r-xr-x--- | asperaweb | asperaweb | 524288000 | 2022-10-27 14:53:00 +0200 | 500MB |
|
160
|
+
| -r-xr-x--- | asperaweb | asperaweb | 1048576000 | 2022-10-27 16:06:37 +0200 | 1GB |
|
161
|
+
| -r-xr-x--- | asperaweb | asperaweb | 5368709120 | 2022-10-27 14:53:47 +0200 | 5GB |
|
162
|
+
| -r-xr-x--- | asperaweb | asperaweb | 209715200 | 2022-10-27 14:52:56 +0200 | 200MB |
|
163
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
169
164
|
```
|
170
165
|
|
171
166
|
```bash
|
@@ -185,7 +180,7 @@ Then, follow the section relative to the product you want to interact with ( Asp
|
|
185
180
|
|
186
181
|
## <a id="installation"></a>Installation
|
187
182
|
|
188
|
-
It is possible to install
|
183
|
+
It is possible to install **either** directly on the host operating system (Linux, macOS, Windows) or as a container (`docker`).
|
189
184
|
|
190
185
|
The direct installation is recommended and consists in installing:
|
191
186
|
|
@@ -193,28 +188,30 @@ The direct installation is recommended and consists in installing:
|
|
193
188
|
- [aspera-cli](#the_gem)
|
194
189
|
- [Aspera SDK (`ascp`)](#fasp_prot)
|
195
190
|
|
196
|
-
Ruby version: >= 2.
|
191
|
+
Ruby version: >= 2.6.
|
197
192
|
|
198
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
193
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
199
194
|
|
200
195
|
The following sections provide information on the various installation methods.
|
201
196
|
|
202
197
|
An internet connection is required for the installation. If you don't have internet for the installation, refer to section [Installation without internet access](#offline_install).
|
203
198
|
|
204
|
-
###
|
199
|
+
### Container
|
205
200
|
|
206
|
-
The image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
|
201
|
+
The container image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
|
207
202
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
208
|
-
To use the container, ensure that you have `
|
203
|
+
To use the container, ensure that you have `podman` (or `docker`) installed.
|
209
204
|
|
210
205
|
```bash
|
211
|
-
|
206
|
+
podman --version
|
212
207
|
```
|
213
208
|
|
209
|
+
#### Container: quick start
|
210
|
+
|
214
211
|
**Wanna start quickly ?** With an interactive shell ? Execute this:
|
215
212
|
|
216
213
|
```bash
|
217
|
-
|
214
|
+
podman run --tty --interactive --entrypoint bash martinlaurent/ascli:latest
|
218
215
|
```
|
219
216
|
|
220
217
|
Then, execute individual `ascli` commands such as:
|
@@ -232,18 +229,22 @@ That is simple, but there are limitations:
|
|
232
229
|
- Any generated file in the container will be lost on container (shell) exit. Including configuration files and downloaded files.
|
233
230
|
- No possibility to upload files located on the host system
|
234
231
|
|
235
|
-
|
232
|
+
#### Container: Details
|
233
|
+
|
234
|
+
The container image is built from this [Dockerfile](Dockerfile.tmpl.erb): the entry point is `ascli` and the default command is `help`.
|
235
|
+
|
236
|
+
If you want to run the image with a shell, execute with option: `--entrypoint bash`, and give argument `-l` (`bash` login option to override the `help` default argument)
|
236
237
|
|
237
|
-
The container can also be
|
238
|
+
The container can also be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
238
239
|
|
239
240
|
```bash
|
240
|
-
|
241
|
+
podman run --rm --tty --interactive martinlaurent/ascli:latest
|
241
242
|
```
|
242
243
|
|
243
244
|
For more convenience, you may define a shell alias:
|
244
245
|
|
245
246
|
```bash
|
246
|
-
alias ascli='
|
247
|
+
alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
|
247
248
|
```
|
248
249
|
|
249
250
|
Then, you can execute the container like a local command:
|
@@ -253,11 +254,11 @@ ascli -v
|
|
253
254
|
```
|
254
255
|
|
255
256
|
```text
|
256
|
-
4.
|
257
|
+
4.14.0
|
257
258
|
```
|
258
259
|
|
259
260
|
In order to keep persistency of configuration on the host,
|
260
|
-
you should
|
261
|
+
you should specify your user's config folder as a volume for the container.
|
261
262
|
To enable write access, a possibility is to run as `root` in the container (and set the default configuration folder to `/home/cliuser/.aspera/ascli`).
|
262
263
|
Add options:
|
263
264
|
|
@@ -265,7 +266,7 @@ Add options:
|
|
265
266
|
--user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli
|
266
267
|
```
|
267
268
|
|
268
|
-
> **Note:** if you are using a `podman machine`, e.g. on
|
269
|
+
> **Note:** if you are using a `podman machine`, e.g. on macOS , make sure that the folder is also shared between the VM and the host, so that sharing is: container → VM → Host: `podman machine init ... --volume="/Users:/Users"`
|
269
270
|
|
270
271
|
As shown in the quick start, if you prefer to keep a running container with a shell and `ascli` available,
|
271
272
|
you can change the entry point, add option:
|
@@ -274,8 +275,8 @@ you can change the entry point, add option:
|
|
274
275
|
--entrypoint bash
|
275
276
|
```
|
276
277
|
|
277
|
-
You may also probably want that files downloaded in the container are
|
278
|
-
In this case you need also to
|
278
|
+
You may also probably want that files downloaded in the container are directed to the host.
|
279
|
+
In this case you need also to specify the shared transfer folder as a volume:
|
279
280
|
|
280
281
|
```bash
|
281
282
|
--volume $HOME/xferdir:/xferfiles
|
@@ -286,7 +287,7 @@ In this case you need also to mount the shared transfer folder:
|
|
286
287
|
And if you want all the above, simply use all the options:
|
287
288
|
|
288
289
|
```bash
|
289
|
-
alias asclish="
|
290
|
+
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
|
290
291
|
```
|
291
292
|
|
292
293
|
```bash
|
@@ -297,6 +298,8 @@ mkdir -p $HOME/.aspera/ascli
|
|
297
298
|
asclish
|
298
299
|
```
|
299
300
|
|
301
|
+
#### Container: Sample start script
|
302
|
+
|
300
303
|
A convenience sample script is also provided: download the script [`dascli`](../examples/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
|
301
304
|
|
302
305
|
> **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli conf gem path)/../examples/dascli ascli`
|
@@ -305,8 +308,8 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
305
308
|
|
306
309
|
| env var | description | default | example |
|
307
310
|
|--------------|------------------------------------|--------------------------|--------------------------|
|
308
|
-
| ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.
|
309
|
-
| docker_args | additional options to `
|
311
|
+
| ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
312
|
+
| docker_args | additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
310
313
|
| image | container image name | martinlaurent/ascli | |
|
311
314
|
| version | container image version | latest | `4.8.0.pre` |
|
312
315
|
|
@@ -333,22 +336,88 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
333
336
|
```
|
334
337
|
|
335
338
|
> **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
|
339
|
+
>
|
340
|
+
> **Note:** Do not use too many volumes, as the AUFS limits the number.
|
341
|
+
|
342
|
+
#### Container: Offline installation
|
343
|
+
|
344
|
+
- First create the image archive:
|
345
|
+
|
346
|
+
```bash
|
347
|
+
podman pull martinlaurent/ascli
|
348
|
+
podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
349
|
+
```
|
350
|
+
|
351
|
+
- Then, on air-gapped system:
|
352
|
+
|
353
|
+
```bash
|
354
|
+
podman load -i ascli_image_latest.tar.gz
|
355
|
+
```
|
356
|
+
|
357
|
+
#### Container: `aspera.conf`
|
358
|
+
|
359
|
+
`ascp`'s configuration file `aspera.conf` is located in the container at: `/aspera_sdk/aspera.conf` (see Dockerfile).
|
360
|
+
As the container is immutable, it is not recommended to modify this file.
|
361
|
+
If one wants to change the content, it is possible to tell `ascp` to use another file using `ascp` option `-f`, e.g. by locating it on the host folder `$HOME/.aspera/ascli` mapped to the container folder `/home/cliuser/.aspera/ascli`:
|
362
|
+
|
363
|
+
```bash
|
364
|
+
echo '<CONF/>' > $HOME/.aspera/ascli/aspera.conf
|
365
|
+
```
|
366
|
+
|
367
|
+
Then, tell `ascp` to use that other conf file:
|
368
|
+
|
369
|
+
```bash
|
370
|
+
--transfer-info=@json:'{"ascp_args":["-f","/home/cliuser/.aspera/ascli/aspera.conf"]}'
|
371
|
+
```
|
372
|
+
|
373
|
+
#### Container: Singularity
|
374
|
+
|
375
|
+
Singularity is another type of use of container.
|
376
|
+
|
377
|
+
On Linux install:
|
378
|
+
|
379
|
+
```console
|
380
|
+
dnf install singularity-ce
|
381
|
+
```
|
382
|
+
|
383
|
+
Build an image like this:
|
384
|
+
|
385
|
+
```bash
|
386
|
+
singularity build ascli.sif docker://martinlaurent/ascli
|
387
|
+
```
|
388
|
+
|
389
|
+
The use like this:
|
390
|
+
|
391
|
+
```bash
|
392
|
+
singularity run ascli.sif
|
393
|
+
```
|
394
|
+
|
395
|
+
Or get a shell with access to the tool like this:
|
396
|
+
|
397
|
+
```bash
|
398
|
+
singularity shell ascli.sif
|
399
|
+
```
|
336
400
|
|
337
401
|
### <a id="ruby"></a>Ruby
|
338
402
|
|
339
403
|
Use this method to install on the native host.
|
340
404
|
|
341
|
-
A
|
405
|
+
A Ruby interpreter is required to run the tool or to use the gem and tool.
|
342
406
|
|
343
|
-
Required Ruby version: >= 2.
|
407
|
+
Required Ruby version: >= 2.6.
|
344
408
|
|
345
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
409
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
346
410
|
|
347
411
|
*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
348
412
|
|
349
|
-
|
413
|
+
In priority, refer to the official Ruby documentation:
|
414
|
+
|
415
|
+
- [Download Ruby](https://www.ruby-lang.org/en/downloads/)
|
416
|
+
- [Installation Guide](https://www.ruby-lang.org/en/documentation/installation/)
|
417
|
+
|
418
|
+
Else, refer to the following sections for a proposed method for specific operating systems.
|
350
419
|
|
351
|
-
The recommended installation method is `rvm` for
|
420
|
+
The recommended installation method is `rvm` for Unix-like systems (Linux, AIX, macOS, Windows with cygwin, etc...).
|
352
421
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
353
422
|
If you have a simpler better way to install Ruby : use it !
|
354
423
|
|
@@ -356,7 +425,7 @@ If you have a simpler better way to install Ruby : use it !
|
|
356
425
|
|
357
426
|
Use this method which provides more flexibility.
|
358
427
|
|
359
|
-
Install
|
428
|
+
Install `rvm`: follow [https://rvm.io/](https://rvm.io/) :
|
360
429
|
|
361
430
|
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
362
431
|
|
@@ -372,7 +441,7 @@ If you keep the same terminal (not needed if re-login):
|
|
372
441
|
source ~/.rvm/scripts/rvm
|
373
442
|
```
|
374
443
|
|
375
|
-
It is advised to get one of the pre-compiled
|
444
|
+
It is advised to get one of the pre-compiled Ruby version, you can list with:
|
376
445
|
|
377
446
|
```bash
|
378
447
|
rvm list --remote
|
@@ -381,10 +450,10 @@ rvm list --remote
|
|
381
450
|
Install the chosen pre-compiled Ruby version:
|
382
451
|
|
383
452
|
```bash
|
384
|
-
rvm install 2.
|
453
|
+
rvm install 3.2.2
|
385
454
|
```
|
386
455
|
|
387
|
-
Ruby is now installed for the user, go
|
456
|
+
Ruby is now installed for the user, go to [Gem installation](#the_gem).
|
388
457
|
|
389
458
|
#### Generic: RVM: global installation (as root)
|
390
459
|
|
@@ -399,7 +468,7 @@ curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
|
399
468
|
|
400
469
|
As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
|
401
470
|
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
402
|
-
To activate
|
471
|
+
To activate Ruby (and ascli) later, source it:
|
403
472
|
|
404
473
|
```bash
|
405
474
|
source /etc/profile.d/rvm.sh.ok
|
@@ -434,25 +503,36 @@ brew install ruby
|
|
434
503
|
|
435
504
|
#### Linux: package
|
436
505
|
|
437
|
-
If your Linux distribution provides a standard
|
506
|
+
If your Linux distribution provides a standard Ruby package, you can use it provided that the version supported.
|
438
507
|
|
439
|
-
Example
|
508
|
+
**Example:** RHEL 8+, Rocky Linux 8+, Centos 8 Stream: with extensions to compile native gems
|
440
509
|
|
441
|
-
|
442
|
-
yum module install ruby:3.1
|
443
|
-
```
|
510
|
+
- Check available ruby versions:
|
444
511
|
|
445
|
-
|
512
|
+
```bash
|
513
|
+
dnf module list ruby
|
514
|
+
```
|
446
515
|
|
447
|
-
|
448
|
-
yum install make automake gcc gcc-c++ kernel-devel
|
449
|
-
yum install redhat-rpm-config
|
450
|
-
dnf module reset ruby
|
451
|
-
dnf module enable ruby:3.1
|
452
|
-
dnf module -y install ruby:3.1/common
|
453
|
-
```
|
516
|
+
- If ruby was already installed with an older version, remove it:
|
454
517
|
|
455
|
-
|
518
|
+
```bash
|
519
|
+
dnf module -y reset ruby
|
520
|
+
```
|
521
|
+
|
522
|
+
- Install packages needed to build native gems:
|
523
|
+
|
524
|
+
```bash
|
525
|
+
dnf install -y make automake gcc gcc-c++ kernel-devel
|
526
|
+
```
|
527
|
+
|
528
|
+
- Enable the Ruby version you want:
|
529
|
+
|
530
|
+
```bash
|
531
|
+
dnf module -y enable ruby:3.1
|
532
|
+
dnf install -y ruby-devel
|
533
|
+
```
|
534
|
+
|
535
|
+
**Other examples:**
|
456
536
|
|
457
537
|
```bash
|
458
538
|
yum install -y ruby ruby-devel rubygems ruby-json
|
@@ -462,7 +542,7 @@ yum install -y ruby ruby-devel rubygems ruby-json
|
|
462
542
|
apt install -y ruby ruby-dev rubygems ruby-json
|
463
543
|
```
|
464
544
|
|
465
|
-
One can cleanup the whole yum-installed
|
545
|
+
One can cleanup the whole yum-installed Ruby environment like this to uninstall:
|
466
546
|
|
467
547
|
```bash
|
468
548
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
@@ -475,7 +555,7 @@ For example for AIX, one can look at:
|
|
475
555
|
|
476
556
|
<https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha#R>
|
477
557
|
|
478
|
-
If your Unix does not provide a pre-built
|
558
|
+
If your Unix does not provide a pre-built Ruby, you can get it using one of those
|
479
559
|
[methods](https://www.ruby-lang.org/en/documentation/installation/).
|
480
560
|
|
481
561
|
For instance to build from source, and install in `/opt/ruby` :
|
@@ -502,7 +582,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
|
|
502
582
|
|
503
583
|
<https://www.jruby.org/download>
|
504
584
|
|
505
|
-
> **Note:** Using jruby the startup time is longer than the native
|
585
|
+
> **Note:** Using jruby the startup time is longer than the native Ruby, but the transfer speed is not impacted (executed by `ascp` binary).
|
506
586
|
|
507
587
|
### <a id="the_gem"></a>`aspera-cli` gem
|
508
588
|
|
@@ -518,9 +598,10 @@ To upgrade to the latest version:
|
|
518
598
|
gem update aspera-cli
|
519
599
|
```
|
520
600
|
|
521
|
-
`ascli` checks every week if a new version is available and notify the user in a WARN log.
|
601
|
+
`ascli` checks every week if a new version is available and notify the user in a WARN log.
|
602
|
+
To de-activate this feature, globally set the option `version_check_days` to `0`, or specify a different period in days.
|
522
603
|
|
523
|
-
To check
|
604
|
+
To check if a new version is available (independently of `version_check_days`):
|
524
605
|
|
525
606
|
```bash
|
526
607
|
ascli conf check_update
|
@@ -556,7 +637,6 @@ If the embedded method is not used, the following packages are also suitable:
|
|
556
637
|
|
557
638
|
- IBM Aspera Connect Client (Free)
|
558
639
|
- IBM Aspera Desktop Client (Free)
|
559
|
-
- IBM Aspera CLI (Free)
|
560
640
|
- IBM Aspera High Speed Transfer Server (Licensed)
|
561
641
|
- IBM Aspera High Speed Transfer EndPoint (Licensed)
|
562
642
|
|
@@ -670,27 +750,72 @@ Moreover all `ascp` options are supported either through transfer spec parameter
|
|
670
750
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
671
751
|
`ascli` receives its arguments from this shell (through Operating System).
|
672
752
|
|
673
|
-
#### Shell parsing for Linux,
|
753
|
+
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
674
754
|
|
675
|
-
|
755
|
+
Linux command line parsing is easy: It is fully documented in the shell's documentation.
|
756
|
+
|
757
|
+
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
676
758
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
677
|
-
|
678
|
-
|
759
|
+
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
760
|
+
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
679
761
|
Ruby receives a list parameters from shell and gives it to `ascli`.
|
680
|
-
So special character handling (quotes, spaces, env vars, ...) is
|
762
|
+
So special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
|
681
763
|
|
682
764
|
#### Shell parsing for Windows
|
683
765
|
|
684
|
-
|
685
|
-
|
686
|
-
|
766
|
+
MS Windows command line parsing is not easy: It is not hasndled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the application (here Ruby).
|
767
|
+
|
768
|
+
As far as `ascli` is concerned: it is close to a Linux shell parsing.
|
769
|
+
|
770
|
+
Thanksfully, `ascli` provides a command to check the value of an argument after parsing: `config echo`.
|
771
|
+
One can also run `ascli` with option `--log-level=debug` to display the command line after parsing.
|
772
|
+
|
773
|
+
The following examples give the same result on Windows:
|
774
|
+
|
775
|
+
- single quote protects the double quote
|
776
|
+
|
777
|
+
```cmd
|
778
|
+
conf echo @json:'{"url":"https://..."}'
|
779
|
+
```
|
780
|
+
|
781
|
+
- triple double quotes are replaced with a single double quote
|
782
|
+
|
783
|
+
```cmd
|
784
|
+
conf echo @json:{"""url""":"""https://..."""}
|
785
|
+
```
|
786
|
+
|
787
|
+
- double quote is escaped with backslash within double quotes
|
788
|
+
|
789
|
+
```cmd
|
790
|
+
conf echo @json:"{\"url\":\"https://...\"}"
|
791
|
+
```
|
792
|
+
|
793
|
+
On Windows, `cmd.exe` is typically used to start .
|
794
|
+
`cmd.exe` handles some special characters: `^"<>|%&`.
|
795
|
+
Basically it handles I/O redirections (`<>|`), shell variables (`%`), multiple commands (`&`) and handles those special characters from the command line.
|
796
|
+
Eventually, all those special characters are removed from the command line unless escaped with `^` or `"`.
|
797
|
+
`"` are kept and given to the program.
|
798
|
+
|
799
|
+
Then, Windows `CreateProcess` is called with just the whole command line as a single string, unlike Unix-like systems where the command line is split into arguments by the shell.
|
800
|
+
|
801
|
+
It's up to the program to split arguments:
|
687
802
|
|
688
803
|
- [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
|
689
|
-
- [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
804
|
+
- [Understand Quoting and Escaping of Windows Command Line Arguments](https://web.archive.org/web/20190316094059/http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
805
|
+
|
806
|
+
is a Ruby program, so Ruby parses the command line into arguments and provides them to the program.
|
807
|
+
Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
|
808
|
+
(See `w32_cmdvector` in Ruby source [`win32.c`](https://github.com/ruby/ruby/blob/master/win32/win32.c#L1766)) : <!--cspell:disable-line-->
|
809
|
+
|
810
|
+
- space characters: split arguments (space, tab, newline)
|
811
|
+
- backslash: `\` escape single special character
|
812
|
+
- globbing characters: `*?[]{}` for file globbing
|
813
|
+
- double quotes: `"`
|
814
|
+
- single quotes: `'`
|
690
815
|
|
691
816
|
#### Extended Values (JSON, Ruby, ...)
|
692
817
|
|
693
|
-
Some of the
|
818
|
+
Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple strings, but a complex structure (Hash, Array).
|
694
819
|
Typically, the `@json:` modifier is used, it expects a JSON string. JSON itself has some special syntax: for example `"` is used to denote strings.
|
695
820
|
|
696
821
|
#### Testing Extended Values
|
@@ -703,12 +828,12 @@ Example: The shell parses three arguments (as strings: `1`, `2` and `3`), so the
|
|
703
828
|
ascli conf echo 1 2 3
|
704
829
|
```
|
705
830
|
|
706
|
-
```
|
831
|
+
```ruby
|
707
832
|
"1"
|
708
833
|
ERROR: Argument: unprocessed values: ["2", "3"]
|
709
834
|
```
|
710
835
|
|
711
|
-
`config echo` displays the value of the first argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
|
836
|
+
`config echo` displays the value of the **first** argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
|
712
837
|
|
713
838
|
> **Note:** It gets its value after shell command line parsing and `ascli` extended value parsing.
|
714
839
|
|
@@ -719,7 +844,7 @@ Depending on the case, a different `format` is used to display the actual value.
|
|
719
844
|
For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
|
720
845
|
|
721
846
|
Double quotes are processed by the shell to create a single string argument.
|
722
|
-
For POSIX shells
|
847
|
+
For **POSIX shells**, single quotes can also be used in this case, or protect the special character ` ` (space) with a backslash. <!-- markdownlint-disable-line -->
|
723
848
|
|
724
849
|
```bash
|
725
850
|
ascli conf echo "Hello World" --format=text
|
@@ -736,7 +861,7 @@ Hello World
|
|
736
861
|
To be evaluated by shell, the shell variable must not be in single quotes.
|
737
862
|
Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
|
738
863
|
|
739
|
-
> **Note:** we use a
|
864
|
+
> **Note:** we use a shell variable here: the variable is not necessarily an environment variable (`export`).
|
740
865
|
|
741
866
|
```bash
|
742
867
|
MYVAR="Hello World"
|
@@ -819,6 +944,22 @@ ascli conf echo @ruby:"{'title'=>gets.chomp}" --format=json
|
|
819
944
|
|
820
945
|
`gets` is Ruby's method of terminal input (terminated by `\n`), and `chomp` removes the trailing `\n`.
|
821
946
|
|
947
|
+
#### Command line arguments from a file
|
948
|
+
|
949
|
+
If you need to provide a list of command line argument from lines that are in a file, on Linux you can use the `xargs` command:
|
950
|
+
|
951
|
+
```bash
|
952
|
+
xargs -a lines.txt -d \\n ascli conf echo
|
953
|
+
```
|
954
|
+
|
955
|
+
This is equivalent to execution of:
|
956
|
+
|
957
|
+
```bash
|
958
|
+
ascli conf echo [line1] [line2] [line3] ...
|
959
|
+
```
|
960
|
+
|
961
|
+
If there are spaces in the lines, those are not taken as separator, as we provide option `-d \\n` to `xargs`.
|
962
|
+
|
822
963
|
#### Extended value using special characters read from environmental variables or files
|
823
964
|
|
824
965
|
Using a text editor or shell: create a file `title.txt` (and env var) that contains exactly the text required: `Test " ' & \` :
|
@@ -841,7 +982,7 @@ ascli conf echo @env:MYTITLE --format=text
|
|
841
982
|
Test " ' & \
|
842
983
|
```
|
843
984
|
|
844
|
-
If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any
|
985
|
+
If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any Ruby code in expression, including reading from file or env var. In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.
|
845
986
|
|
846
987
|
```bash
|
847
988
|
ascli conf echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
|
@@ -852,25 +993,50 @@ ascli conf echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
852
993
|
{"title":"Test \" ' & \\"}
|
853
994
|
```
|
854
995
|
|
855
|
-
###
|
996
|
+
### Commands, Options, Positional Values
|
997
|
+
|
998
|
+
Command line arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
|
856
999
|
|
857
|
-
|
1000
|
+
`ascli` considers three types of command line arguments:
|
858
1001
|
|
859
|
-
|
1002
|
+
- Commands
|
1003
|
+
- Options
|
1004
|
+
- Positional Values
|
860
1005
|
|
861
1006
|
```bash
|
862
1007
|
ascli command subcommand --option-name=VAL1 VAL2
|
863
1008
|
```
|
864
1009
|
|
865
1010
|
- executes *command*: `command subcommand`
|
866
|
-
- with one *option*: `option_name`
|
867
|
-
-
|
868
|
-
- the command has one additional *argument*: `VAL2`
|
1011
|
+
- with one *option*: `option_name`and its *value*: `VAL1`
|
1012
|
+
- the command has one additional mandatory *argument*: `VAL2`
|
869
1013
|
|
870
|
-
When the value of a command, option or argument is constrained by a fixed list of values
|
1014
|
+
When the value of a command, option or argument is constrained by a fixed list of values.
|
1015
|
+
It is possible to use the first letters of the value only, provided that it uniquely identifies a value.
|
1016
|
+
For example `ascli conf ov` is the same as `ascli config overview`.
|
871
1017
|
|
872
1018
|
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
|
873
1019
|
|
1020
|
+
#### Commands
|
1021
|
+
|
1022
|
+
Commands are typically entity types or verbs to act on those entities.
|
1023
|
+
|
1024
|
+
Example:
|
1025
|
+
|
1026
|
+
```bash
|
1027
|
+
ascli conf ascp info
|
1028
|
+
```
|
1029
|
+
|
1030
|
+
- `ascli` is the executable executed by the shell
|
1031
|
+
- `conf` is the first level command, and is also the name f the plugin to be used
|
1032
|
+
- `ascp` is the second level command, and is also the name of the component (singleton)
|
1033
|
+
- `info` is the third level command, and is also the action to be performed
|
1034
|
+
|
1035
|
+
Typically, commands are located at the beginning of the command line.
|
1036
|
+
Order is significant.
|
1037
|
+
The provided command must match one of the supported commands in the given context.
|
1038
|
+
If a wrong , or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
1039
|
+
|
874
1040
|
#### Options
|
875
1041
|
|
876
1042
|
All options, e.g. `--log-level=debug`, are command line arguments that:
|
@@ -878,7 +1044,7 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
|
|
878
1044
|
- start with `--`
|
879
1045
|
- have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
|
880
1046
|
- have a value, separated from name with a `=`
|
881
|
-
- can be used by prefix, provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
|
1047
|
+
- can be used by prefix, provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug` (avoid)
|
882
1048
|
|
883
1049
|
Exceptions:
|
884
1050
|
|
@@ -886,17 +1052,24 @@ Exceptions:
|
|
886
1052
|
- some options (flags) don't take a value, e.g. `-r`
|
887
1053
|
- the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
|
888
1054
|
|
889
|
-
```bash
|
890
|
-
ascli config echo -- --sample
|
891
|
-
```
|
1055
|
+
```bash
|
1056
|
+
ascli config echo -- --sample
|
1057
|
+
```
|
892
1058
|
|
893
|
-
```bash
|
894
|
-
"--sample"
|
895
|
-
```
|
1059
|
+
```bash
|
1060
|
+
"--sample"
|
1061
|
+
```
|
896
1062
|
|
897
1063
|
> **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
|
898
1064
|
|
899
|
-
Options
|
1065
|
+
Options may have an (hardcoded) default value.
|
1066
|
+
|
1067
|
+
Options can be placed anywhere on command line and evaluated in order.
|
1068
|
+
|
1069
|
+
Options are typically either:
|
1070
|
+
|
1071
|
+
- optional : typically to change the default behavior
|
1072
|
+
- mandatory : typically, connection information are options that are mandatory (so they can be placed in a config file)
|
900
1073
|
|
901
1074
|
The value for *any* options can come from the following locations (in this order, last value evaluated overrides previous value):
|
902
1075
|
|
@@ -908,9 +1081,16 @@ Environment variable starting with prefix: ASCLI_ are taken as option values, e.
|
|
908
1081
|
|
909
1082
|
Options values can be displayed for a given command by providing the `--show-config` option: `ascli node --show-config`
|
910
1083
|
|
911
|
-
####
|
1084
|
+
#### Positional Values
|
1085
|
+
|
1086
|
+
Positional Values are typically mandatory values for a command, such as entity creation data.
|
1087
|
+
|
1088
|
+
If a Positional Values begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
|
912
1089
|
|
913
|
-
|
1090
|
+
The advantages of using a positional value instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
|
1091
|
+
|
1092
|
+
The disadvantage is that it is not possible to define a default value in a config file or environment variable like for options.
|
1093
|
+
Nevertheless, [Extended Values](#extended) syntax is supported, so it is possible to retrieve a value from the config file or environment variable.
|
914
1094
|
|
915
1095
|
### Interactive Input
|
916
1096
|
|
@@ -972,16 +1152,26 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
|
|
972
1152
|
```
|
973
1153
|
|
974
1154
|
```output
|
975
|
-
|
976
|
-
|
977
|
-
|
978
|
-
|
979
|
-
|
980
|
-
|
1155
|
+
+-------------------------------+----------------------------------+-----------+
|
1156
|
+
| name | email | ats_admin |
|
1157
|
+
+-------------------------------+----------------------------------+-----------+
|
1158
|
+
| John Curtis | john@example.com | true |
|
1159
|
+
| Laurent Martin | laurent@example.com | true |
|
1160
|
+
+-------------------------------+----------------------------------+-----------+
|
981
1161
|
```
|
982
1162
|
|
983
1163
|
> **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
|
984
1164
|
|
1165
|
+
#### entity identifier
|
1166
|
+
|
1167
|
+
When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command: e.g. `ascli aoc admin res user show 1234` where `1234` is the user identifier.
|
1168
|
+
|
1169
|
+
> **Note:** The legacy option `id` is deprecated: `--id=1234` as it does not provide the possibility to have sub-entities.
|
1170
|
+
|
1171
|
+
Only some commands provide the following capability: If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin res user show %name:john` where `john` is the user name.
|
1172
|
+
|
1173
|
+
Syntax: `%<field>:<value>`
|
1174
|
+
|
985
1175
|
#### Verbosity of output
|
986
1176
|
|
987
1177
|
Output messages are categorized in 3 types:
|
@@ -1012,38 +1202,38 @@ By default, a table output will display one line per entry, and columns for each
|
|
1012
1202
|
|
1013
1203
|
### <a id="extended"></a>Extended Value Syntax
|
1014
1204
|
|
1015
|
-
|
1205
|
+
Some options and arguments are specified by a simple string.
|
1206
|
+
But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
|
1016
1207
|
|
1017
1208
|
The extended value syntax is:
|
1018
1209
|
|
1019
1210
|
```bash
|
1020
|
-
<0 or more decoders><
|
1211
|
+
<0 or more decoders><nothing or some text value>
|
1021
1212
|
```
|
1022
1213
|
|
1023
|
-
|
1024
|
-
|
1025
|
-
Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
|
1026
|
-
|
1027
|
-
The following "readers" are supported (returns value in []):
|
1214
|
+
Decoders act like a function of value on right hand side.
|
1215
|
+
Decoders are recognized by the prefix: `@` and suffix `:`
|
1028
1216
|
|
1029
|
-
|
1030
|
-
- @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
|
1031
|
-
- @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
1032
|
-
- @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
1033
|
-
- @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
1034
|
-
- @stdin: : [String] read from stdin (no value on right)
|
1035
|
-
- @preset:NAME : [Hash] get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
|
1217
|
+
The following decoders are supported:
|
1036
1218
|
|
1037
|
-
|
1038
|
-
|
1039
|
-
|
1040
|
-
|
1041
|
-
|
1042
|
-
|
1043
|
-
|
1044
|
-
|
1045
|
-
|
1046
|
-
|
1219
|
+
| decoder | parameter | returns | description |
|
1220
|
+
|---------|-----------|---------|-------------|
|
1221
|
+
| base64 | String | String | decode a base64 encoded string
|
1222
|
+
| csvt | String | Array | decode a titled CSV value
|
1223
|
+
| env | String | String | read from a named env var name, e.g.--password=@env:MYPASSVAR
|
1224
|
+
| file | String | String | read value from specified file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey` |
|
1225
|
+
| incps | Hash | Hash | include values of presets specified by key `incps` in input hash
|
1226
|
+
| json | String | any | decode JSON values (convenient to provide complex structures)
|
1227
|
+
| lines | String | Array | split a string in multiple lines and return an array
|
1228
|
+
| list | String | Array | split a string in multiple items taking first character as separator and return an array
|
1229
|
+
| path | String | String | performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml` |
|
1230
|
+
| preset | String | Hash | get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
|
1231
|
+
| ruby | String | any | execute specified Ruby code
|
1232
|
+
| secret | None | String | Ask password interactively (hides input)
|
1233
|
+
| stdin | None | String | read from stdin (no value on right)
|
1234
|
+
| uri | String | String | read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
1235
|
+
| val | String | String | prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
1236
|
+
| zlib | String | String | un-compress data
|
1047
1237
|
|
1048
1238
|
To display the result of an extended value, use the `config echo` command.
|
1049
1239
|
|
@@ -1076,12 +1266,12 @@ ascli config echo @csvt:@file:test.csv
|
|
1076
1266
|
```
|
1077
1267
|
|
1078
1268
|
```output
|
1079
|
-
|
1080
|
-
|
1081
|
-
|
1082
|
-
|
1083
|
-
|
1084
|
-
|
1269
|
+
+------+---------------------+
|
1270
|
+
| name | email |
|
1271
|
+
+------+---------------------+
|
1272
|
+
| lolo | laurent@example.com |
|
1273
|
+
| toto | titi@tutu.tata |
|
1274
|
+
+------+---------------------+
|
1085
1275
|
```
|
1086
1276
|
|
1087
1277
|
Example: create a hash and include values from preset named "config" of config file in this hash
|
@@ -1111,7 +1301,7 @@ It is also possible to provide a [Extended Value](#extended) in a file using `@j
|
|
1111
1301
|
|
1112
1302
|
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored `[config folder]`: `[User's home folder]/.aspera/ascli`.
|
1113
1303
|
|
1114
|
-
Note: `[User's home folder]` is found using
|
1304
|
+
Note: `[User's home folder]` is found using Ruby's `Dir.home` (`rb_w32_home_dir`).
|
1115
1305
|
It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
|
1116
1306
|
|
1117
1307
|
The [config folder] can be displayed using :
|
@@ -1211,9 +1401,9 @@ ascli config open
|
|
1211
1401
|
Older format for commands are still supported:
|
1212
1402
|
|
1213
1403
|
```bash
|
1214
|
-
ascli config
|
1215
|
-
ascli config over
|
1216
|
-
ascli config list
|
1404
|
+
ascli config preset set|delete|show|initialize|update <name>
|
1405
|
+
ascli config preset over
|
1406
|
+
ascli config preset list
|
1217
1407
|
```
|
1218
1408
|
|
1219
1409
|
#### <a id="lprtconf"></a>Special Option preset: config
|
@@ -1242,15 +1432,39 @@ ascli config preset get default _plugin_name_
|
|
1242
1432
|
"_default_preset_for_plugin_"
|
1243
1433
|
```
|
1244
1434
|
|
1245
|
-
#### <a id="config"></a>Plugin: `config`:
|
1435
|
+
#### <a id="config"></a>Plugin: `config`: Configuration
|
1436
|
+
|
1437
|
+
Plugin `config` provides general commands for `ascli`:
|
1438
|
+
|
1439
|
+
- Option preset, config file operations
|
1440
|
+
- wizard
|
1441
|
+
- vault
|
1442
|
+
- ascp
|
1443
|
+
|
1444
|
+
The default configuration for `config` is read for any plugin invocation, this allows setting global options, such as `--log-level` or `--interactive`.
|
1445
|
+
When `ascli` starts, it looks for the `default` Option preset and checks the value for `config`.
|
1446
|
+
If set, it loads the option values for any plugin used.
|
1447
|
+
|
1448
|
+
> **Note:** If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
|
1246
1449
|
|
1247
|
-
|
1450
|
+
Show current default (global) Option preset (`config` plugin):
|
1248
1451
|
|
1249
|
-
|
1452
|
+
```console
|
1453
|
+
$ ascli conf preset get default config
|
1454
|
+
global_common_defaults
|
1455
|
+
```
|
1456
|
+
|
1457
|
+
```bash
|
1458
|
+
ascli conf preset set global_common_defaults version_check_days 0
|
1459
|
+
```
|
1250
1460
|
|
1251
|
-
If
|
1461
|
+
If the default global Option preset is not set:
|
1462
|
+
|
1463
|
+
```bash
|
1464
|
+
ascli conf preset set default config global_common_defaults
|
1465
|
+
```
|
1252
1466
|
|
1253
|
-
|
1467
|
+
#### Config sample commands
|
1254
1468
|
|
1255
1469
|
```bash
|
1256
1470
|
config ascp connect info 'Aspera Connect for Windows'
|
@@ -1265,9 +1479,11 @@ config ascp products list
|
|
1265
1479
|
config ascp show
|
1266
1480
|
config ascp spec
|
1267
1481
|
config check_update
|
1482
|
+
config coffee
|
1483
|
+
config coffee --ui=text
|
1484
|
+
config detect --url=https://faspex4.example.com/path
|
1268
1485
|
config detect --url=https://my_aoc_org.ibmaspera.com
|
1269
|
-
config detect --url=
|
1270
|
-
config detect --url=my_node_url
|
1486
|
+
config detect --url=https://node_simple.example.com/path
|
1271
1487
|
config doc
|
1272
1488
|
config doc transfer-parameters
|
1273
1489
|
config echo 'hello'
|
@@ -1280,7 +1496,7 @@ config echo @uri:/etc/hosts
|
|
1280
1496
|
config echo @uri:file:/etc/hosts
|
1281
1497
|
config echo @uri:http://www.ibm.com
|
1282
1498
|
config echo @uri:https://www.ibm.com
|
1283
|
-
config echo @val:@file:
|
1499
|
+
config echo @val:@file:no_such_file
|
1284
1500
|
config echo @zlib:@stdin:
|
1285
1501
|
config email_test --notif-to=my_recipient_email
|
1286
1502
|
config export
|
@@ -1313,15 +1529,15 @@ demo_server:
|
|
1313
1529
|
|
1314
1530
|
We can see here:
|
1315
1531
|
|
1316
|
-
- The configuration was created with
|
1532
|
+
- The configuration was created with `ascli` version 0.3.7
|
1317
1533
|
- the default [option preset](#lprt) to load for `server` plugin is : `demo_server`
|
1318
1534
|
- the [option preset](#lprt) `demo_server` defines some parameters: the URL and credentials
|
1319
1535
|
- the default [option preset](#lprt) to load in any case is : `cli_default`
|
1320
1536
|
|
1321
1537
|
Two [option presets](#lprt) are reserved:
|
1322
1538
|
|
1323
|
-
- `config` contains a single value: `version` showing the
|
1324
|
-
|
1539
|
+
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
1540
|
+
It is used to check compatibility.
|
1325
1541
|
- `default` is reserved to define the default [option preset](#lprt) name used for known plugins.
|
1326
1542
|
|
1327
1543
|
The user may create as many [option presets](#lprt) as needed. For instance, a particular [option preset](#lprt) can be created for a particular application instance and contain URL and credentials.
|
@@ -1351,7 +1567,7 @@ Some options are global, some options are available only for some plugins. (the
|
|
1351
1567
|
|
1352
1568
|
Options are loaded using this algorithm:
|
1353
1569
|
|
1354
|
-
- If option `--no-default` (or `-N`) is specified, then no default value is loaded
|
1570
|
+
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
1355
1571
|
- else it looks for the name of the plugin as key in section `default`, the value is the name of the default [option preset](#lprt) for it, and loads it.
|
1356
1572
|
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or of the value is a Hash, it uses it as options values.
|
1357
1573
|
- Environment variables are evaluated
|
@@ -1433,7 +1649,7 @@ ascli config preset set default shares shares06
|
|
1433
1649
|
- Display the content of configuration file in table format
|
1434
1650
|
|
1435
1651
|
```bash
|
1436
|
-
ascli config overview
|
1652
|
+
ascli config preset overview
|
1437
1653
|
```
|
1438
1654
|
|
1439
1655
|
- Execute a command on the shares application using default parameters
|
@@ -1530,13 +1746,13 @@ Some applications allow the user to be authenticated using a private key (Server
|
|
1530
1746
|
It consists in using a pair of keys: the private key and its associated public key.
|
1531
1747
|
The same key can be used for multiple applications.
|
1532
1748
|
Technically, a private key contains the public key, which can be extracted from it.
|
1533
|
-
The private key can be protected by a passphrase
|
1534
|
-
If the key is protected by a passphrase, then it will be prompted.
|
1749
|
+
The file containing the private key can optionally be protected by a passphrase.
|
1750
|
+
If the key is protected by a passphrase, then it will be prompted when used.
|
1535
1751
|
(some plugins support option `passphrase`)
|
1536
1752
|
|
1537
1753
|
The following commands use the shell variable `PRIVKEYFILE`.
|
1538
1754
|
Set it to the desired safe location of the private key.
|
1539
|
-
Typically, in `$HOME/.ssh` or `$HOME/.aspera/ascli`:
|
1755
|
+
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`:
|
1540
1756
|
|
1541
1757
|
```bash
|
1542
1758
|
PRIVKEYFILE=~/.aspera/ascli/my_private_key
|
@@ -1544,6 +1760,8 @@ PRIVKEYFILE=~/.aspera/ascli/my_private_key
|
|
1544
1760
|
|
1545
1761
|
Several methods can be used to generate a key pair.
|
1546
1762
|
|
1763
|
+
The format expected for private keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
|
1764
|
+
|
1547
1765
|
#### `ascli` for key generation
|
1548
1766
|
|
1549
1767
|
The generated key is of type RSA, by default: 4096 bit.
|
@@ -1565,45 +1783,51 @@ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
|
|
1565
1783
|
#### `openssl`
|
1566
1784
|
|
1567
1785
|
To generate a private key pair with a passphrase the following can be used on any system:
|
1568
|
-
|
1786
|
+
|
1569
1787
|
```bash
|
1570
|
-
openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE}
|
1788
|
+
openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE} 4096
|
1571
1789
|
openssl rsa -pubout -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.pub
|
1572
1790
|
```
|
1573
|
-
<!-- spellchecker: enable -->
|
1574
1791
|
|
1575
1792
|
`openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
|
1576
1793
|
In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` to generate a key without passphrase.
|
1577
1794
|
|
1578
1795
|
If option `-nodes` is not available, the passphrase can be removed using this method:
|
1579
1796
|
|
1580
|
-
<!-- spellchecker: disable -->
|
1581
1797
|
```bash
|
1582
|
-
openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE}
|
1583
|
-
|
1798
|
+
openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.no_des
|
1799
|
+
mv ${PRIVKEYFILE}.no_des ${PRIVKEYFILE}
|
1584
1800
|
```
|
1585
|
-
<!-- spellchecker: enable -->
|
1586
1801
|
|
1587
1802
|
To change (or add) the passphrase for a key do:
|
1588
1803
|
|
1589
1804
|
```bash
|
1590
|
-
openssl rsa -des3 -in
|
1805
|
+
openssl rsa -des3 -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.with_des
|
1806
|
+
mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
|
1591
1807
|
```
|
1592
1808
|
|
1593
1809
|
### <a id="certificates"></a>SSL CA certificate bundle
|
1594
1810
|
|
1595
|
-
`ascli` uses
|
1596
|
-
Certificates are checked against the
|
1597
|
-
|
1811
|
+
`ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
|
1812
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
1813
|
+
|
1814
|
+
To display the current root certificate store locations:
|
1598
1815
|
|
1599
|
-
|
1600
|
-
|
1601
|
-
|
1602
|
-
|
1816
|
+
```bash
|
1817
|
+
ascli conf echo @ruby:'[OpenSSL::X509::DEFAULT_CERT_FILE,OpenSSL::X509::DEFAULT_CERT_DIR]'
|
1818
|
+
```
|
1819
|
+
|
1820
|
+
Ruby's default values can be overridden by env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
1821
|
+
|
1822
|
+
`ascp` also needs to validate certificates when using **WSS**.
|
1823
|
+
By default, `ascp` uses primarily certificates from hard-coded path (e.g. on macOS: `/Library/Aspera/ssl`) for WSS.
|
1824
|
+
`ascli` overrides and sets the default Ruby certificate path as well for `ascp` using `-i` switch.
|
1825
|
+
|
1826
|
+
To update `ascli` trusted root certificates, just update your system's root certificates or use env vars specified here above.
|
1603
1827
|
|
1604
1828
|
### Plugins
|
1605
1829
|
|
1606
|
-
|
1830
|
+
`ascli` uses a plugin mechanism.
|
1607
1831
|
The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
|
1608
1832
|
Each plugin usually represents commands sent to a specific application.
|
1609
1833
|
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
@@ -1684,20 +1908,20 @@ Examples:
|
|
1684
1908
|
- display debugging log on `stdout`:
|
1685
1909
|
|
1686
1910
|
```bash
|
1687
|
-
ascli conf over --log-level=debug --logger=stdout
|
1911
|
+
ascli conf pre over --log-level=debug --logger=stdout
|
1688
1912
|
```
|
1689
1913
|
|
1690
1914
|
- log errors to `syslog`:
|
1691
1915
|
|
1692
1916
|
```bash
|
1693
|
-
ascli conf over --log-level=error --logger=syslog
|
1917
|
+
ascli conf pre over --log-level=error --logger=syslog
|
1694
1918
|
```
|
1695
1919
|
|
1696
1920
|
When `ascli` is used interactively in a shell, the shell itself will usually log executed commands in the history file.
|
1697
1921
|
|
1698
1922
|
### Learning Aspera Product APIs (REST)
|
1699
1923
|
|
1700
|
-
|
1924
|
+
`ascli` uses mainly Aspera applications REST APIs.
|
1701
1925
|
To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
|
1702
1926
|
|
1703
1927
|
In order to get traces of execution, use argument : `--log-level=debug`
|
@@ -1706,7 +1930,7 @@ In order to get traces of execution, use argument : `--log-level=debug`
|
|
1706
1930
|
|
1707
1931
|
If the server does not provide a valid certificate, use option: `--insecure=yes`.
|
1708
1932
|
|
1709
|
-
|
1933
|
+
HTTP socket parameters can be adjusted using option `http_options`:
|
1710
1934
|
|
1711
1935
|
| parameter | default |
|
1712
1936
|
|----------------------|---------|
|
@@ -1716,8 +1940,8 @@ Ruby HTTP socket parameters can be adjusted.
|
|
1716
1940
|
| `keep_alive_timeout` | 2 |
|
1717
1941
|
|
1718
1942
|
Values are in set *seconds* and can be of type either integer or float.
|
1719
|
-
Default values are the ones of Ruby
|
1720
|
-
|
1943
|
+
Default values are the ones of Ruby:
|
1944
|
+
refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1721
1945
|
|
1722
1946
|
Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
|
1723
1947
|
|
@@ -1734,7 +1958,7 @@ Some actions may require the use of a graphical tool:
|
|
1734
1958
|
- a browser for Aspera on Cloud authentication (web auth method)
|
1735
1959
|
- a text editor for configuration file edition
|
1736
1960
|
|
1737
|
-
By default
|
1961
|
+
By default `ascli` assumes that a graphical environment is available on windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
1738
1962
|
It is also possible to force the graphical mode with option --ui :
|
1739
1963
|
|
1740
1964
|
- `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
@@ -1814,7 +2038,7 @@ Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallb
|
|
1814
2038
|
|
1815
2039
|
### FASP proxy (forward) for transfers
|
1816
2040
|
|
1817
|
-
To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `
|
2041
|
+
To specify a FASP proxy (forward), set the [*transfer-spec*](#transferspec) parameter: `proxy` (only supported with the `direct` agent).
|
1818
2042
|
|
1819
2043
|
### <a id="client"></a>FASP configuration
|
1820
2044
|
|
@@ -1870,7 +2094,7 @@ Saved to default global preset global_common_defaults
|
|
1870
2094
|
|
1871
2095
|
Windows:
|
1872
2096
|
|
1873
|
-
```
|
2097
|
+
```text
|
1874
2098
|
ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1875
2099
|
```
|
1876
2100
|
|
@@ -1891,14 +2115,14 @@ ascli config ascp products list
|
|
1891
2115
|
```
|
1892
2116
|
|
1893
2117
|
```output
|
1894
|
-
|
1895
|
-
|
1896
|
-
|
1897
|
-
|
1898
|
-
|
1899
|
-
|
1900
|
-
|
1901
|
-
|
2118
|
+
+---------------------------------------+----------------------------------------+
|
2119
|
+
| name | app_root |
|
2120
|
+
+---------------------------------------+----------------------------------------+
|
2121
|
+
| IBM Aspera SDK | /Users/laurent/.aspera/ascli/sdk |
|
2122
|
+
| Aspera Connect | /Applications/Aspera Connect.app |
|
2123
|
+
| IBM Aspera CLI | /Users/laurent/Applications/Aspera CLI |
|
2124
|
+
| IBM Aspera High-Speed Transfer Server | /Library/Aspera |
|
2125
|
+
+---------------------------------------+----------------------------------------+
|
1902
2126
|
```
|
1903
2127
|
|
1904
2128
|
#### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
|
@@ -1964,7 +2188,7 @@ Some of the actions on Aspera Applications lead to file transfers (upload and do
|
|
1964
2188
|
When a transfer needs to be started, a [*transfer-spec*](#transferspec) has been internally prepared.
|
1965
2189
|
This [*transfer-spec*](#transferspec) will be executed by a transfer client, here called **Transfer Agent**.
|
1966
2190
|
|
1967
|
-
There are currently 3 agents
|
2191
|
+
There are currently 3 agents, set with option `transfer`:
|
1968
2192
|
|
1969
2193
|
- [`direct`](#agt_direct) : a local execution of `ascp`
|
1970
2194
|
- [`connect`](#agt_connect) : use of a local Connect Client
|
@@ -1978,6 +2202,8 @@ will effectively push files to the related server from the agent node.
|
|
1978
2202
|
|
1979
2203
|
`ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *native* `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1980
2204
|
|
2205
|
+
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2206
|
+
|
1981
2207
|
#### <a id="agt_direct"></a>Direct
|
1982
2208
|
|
1983
2209
|
The `direct` agent directly executes a local `ascp`.
|
@@ -2052,22 +2278,82 @@ In addition to standard methods described in section [File List](#file_list), it
|
|
2052
2278
|
>
|
2053
2279
|
> **Note:** Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
|
2054
2280
|
|
2281
|
+
This agent supports a local configuration file: `aspera.conf` where Virtual links can be configured:
|
2282
|
+
|
2283
|
+
On a server (HSTS), the following commands can be used to set a global virtual link:
|
2284
|
+
|
2285
|
+
```bash
|
2286
|
+
asconfigurator -x 'set_trunk_data;id,1;trunk_name,in;trunk_capacity,45000;trunk_on,true'
|
2287
|
+
asconfigurator -x 'set_trunk_data;id,2;trunk_name,out;trunk_capacity,45000;trunk_on,true'
|
2288
|
+
asconfigurator -x 'set_node_data;transfer_in_bandwidth_aggregate_trunk_id,1'
|
2289
|
+
asconfigurator -x 'set_node_data;transfer_out_bandwidth_aggregate_trunk_id,2'
|
2290
|
+
```
|
2291
|
+
|
2292
|
+
But this command is not available on clients, so edit the file `aspera.conf`, you can find the location with: `ascli conf ascp info --fields=aspera_conf` and modify the sections `default` and `trunks` like this for a global 100 Mbps virtual link:
|
2293
|
+
|
2294
|
+
```xml
|
2295
|
+
<?xml version='1.0' encoding='UTF-8'?>
|
2296
|
+
<CONF version="2">
|
2297
|
+
<default>
|
2298
|
+
<transfer>
|
2299
|
+
<in>
|
2300
|
+
<bandwidth>
|
2301
|
+
<aggregate>
|
2302
|
+
<trunk_id>1</trunk_id>
|
2303
|
+
</aggregate>
|
2304
|
+
</bandwidth>
|
2305
|
+
</in>
|
2306
|
+
<out>
|
2307
|
+
<bandwidth>
|
2308
|
+
<aggregate>
|
2309
|
+
<trunk_id>2</trunk_id>
|
2310
|
+
</aggregate>
|
2311
|
+
</bandwidth>
|
2312
|
+
</out>
|
2313
|
+
</transfer>
|
2314
|
+
</default>
|
2315
|
+
<trunks>
|
2316
|
+
<trunk>
|
2317
|
+
<id>1</id>
|
2318
|
+
<name>in</name>
|
2319
|
+
<on>true</on>
|
2320
|
+
<capacity>
|
2321
|
+
<schedule format="ranges">1000000</schedule>
|
2322
|
+
</capacity>
|
2323
|
+
</trunk>
|
2324
|
+
<trunk>
|
2325
|
+
<id>2</id>
|
2326
|
+
<name>out</name>
|
2327
|
+
<capacity>
|
2328
|
+
<schedule format="ranges">1000000</schedule>
|
2329
|
+
</capacity>
|
2330
|
+
<on>true</on>
|
2331
|
+
</trunk>
|
2332
|
+
</trunks>
|
2333
|
+
</CONF>
|
2334
|
+
```
|
2335
|
+
|
2336
|
+
It is also possible to set a schedule with different time and days, for example for the value of `schedule`:
|
2337
|
+
|
2338
|
+
```text
|
2339
|
+
start=08 end=19 days=mon,tue,wed,thu capacity=900000;1000000
|
2340
|
+
```
|
2341
|
+
|
2055
2342
|
#### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
|
2056
2343
|
|
2057
2344
|
By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
|
2058
2345
|
|
2059
2346
|
#### <a id="agt_node"></a>Aspera Node API : Node to node transfers
|
2060
2347
|
|
2061
|
-
By specifying option: `--transfer=node`,
|
2062
|
-
Transfer Server using the Node API, either on a local or remote node.
|
2348
|
+
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
2063
2349
|
Parameters provided in option `transfer_info` are:
|
2064
2350
|
|
2065
|
-
| Name
|
2066
|
-
|
2067
|
-
| url
|
2351
|
+
| Name | Type | Description |
|
2352
|
+
|----------|--------|-------------|
|
2353
|
+
| url | string | URL of the node API</br>Mandatory |
|
2068
2354
|
| username | string | node api user or access key</br>Mandatory |
|
2069
2355
|
| password | string | password, secret or bearer token</br>Mandatory |
|
2070
|
-
| root_id
|
2356
|
+
| root_id | string | password or secret</br>Mandatory only for bearer token |
|
2071
2357
|
|
2072
2358
|
Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
|
2073
2359
|
`--transfer-info=@preset:_name_here_` or be specified using the extended value syntax :
|
@@ -2088,11 +2374,13 @@ Parameters provided in option `transfer_info` are:
|
|
2088
2374
|
| url | string | URL of the HTTP GW</br>Mandatory |
|
2089
2375
|
| upload_bar_refresh_sec | float | Refresh rate for upload progress bar |
|
2090
2376
|
| upload_chunk_size | int | Size in bytes of chunks for upload |
|
2377
|
+
| api_version | string | v1 or v2, for force use of version |
|
2378
|
+
| synchronous | bool | wait for each message acknowledgment |
|
2091
2379
|
|
2092
2380
|
Example:
|
2093
2381
|
|
2094
2382
|
```bash
|
2095
|
-
ascli faspex package recv
|
2383
|
+
ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy"}'
|
2096
2384
|
```
|
2097
2385
|
|
2098
2386
|
> **Note:** The gateway only supports transfers authorized with a token.
|
@@ -2120,8 +2408,8 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
2120
2408
|
|
2121
2409
|
### <a id="transferspec"></a>Transfer Specification
|
2122
2410
|
|
2123
|
-
Some commands lead to file transfer (upload/download)
|
2124
|
-
|
2411
|
+
Some commands lead to file transfer (upload/download).
|
2412
|
+
All parameters necessary for this transfer are described in a [*transfer-spec*](#transferspec) (Transfer Specification), such as:
|
2125
2413
|
|
2126
2414
|
- server address
|
2127
2415
|
- transfer user name
|
@@ -2129,9 +2417,16 @@ is described in a [*transfer-spec*](#transferspec) (Transfer Specification), suc
|
|
2129
2417
|
- file list
|
2130
2418
|
- etc...
|
2131
2419
|
|
2132
|
-
`ascli` builds
|
2420
|
+
`ascli` builds the [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
|
2421
|
+
|
2422
|
+
The [*transfer-spec*](#transferspec) is a Hash (dictionary), so it is described on the command line with the [Extended Value Syntax](#extended).
|
2423
|
+
|
2424
|
+
It is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option.
|
2425
|
+
The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters in a `Hash`.
|
2426
|
+
Multiple `ts` options on command line are cumulative, and Hash is deeply merged.
|
2427
|
+
To remove a (deep) key from transfer spec, set the value to `null`.
|
2133
2428
|
|
2134
|
-
|
2429
|
+
> **Note:** Default transfer spec values can be displayed with command: `config ascp info --flat-hash=no` under field `ts`.
|
2135
2430
|
|
2136
2431
|
It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#agt_direct) using `transfer_info` option parameter: `ascp_args`.
|
2137
2432
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
@@ -2142,8 +2437,6 @@ The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has t
|
|
2142
2437
|
- common to all [Transfer Agent](#agents)
|
2143
2438
|
- not dependent on command line limitations (special characters...)
|
2144
2439
|
|
2145
|
-
A [*transfer-spec*](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
|
2146
|
-
|
2147
2440
|
### <a id="transferparams"></a>Transfer Parameters
|
2148
2441
|
|
2149
2442
|
All standard [*transfer-spec*](#transferspec) parameters can be specified.
|
@@ -2173,18 +2466,7 @@ Columns:
|
|
2173
2466
|
Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
|
2174
2467
|
|
2175
2468
|
| Field | Type | D | N | C | Description |
|
2176
|
-
|
2177
|
-
| EX_ascp_args | array | Y | | | Add native command line arguments to ascp |
|
2178
|
-
| EX_at_rest_password | string | Y | | | DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2179
|
-
| EX_file_list | string | Y | | | source file list |
|
2180
|
-
| EX_file_pair_list | string | Y | | | source file pair list |
|
2181
|
-
| EX_http_proxy_url | string | Y | | | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2182
|
-
| EX_http_transfer_jpeg | int | Y | | | HTTP transfers as JPEG file<br/>(-j {int}) |
|
2183
|
-
| EX_license_text | string | Y | | | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2184
|
-
| EX_no_read | bool | Y | | | no read source<br/>(--no-read) |
|
2185
|
-
| EX_no_write | bool | Y | | | no write on destination<br/>(--no-write) |
|
2186
|
-
| EX_proxy_password | string | Y | | | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS) |
|
2187
|
-
| EX_ssh_key_paths | array | Y | | | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2469
|
+
| ----- | ---- | - | - | - | ----------- |
|
2188
2470
|
| apply_local_docroot | bool | Y | | | (--apply-local-docroot) |
|
2189
2471
|
| authentication | string | | | Y | value=token for SSH bypass keys, else password asked if not provided. |
|
2190
2472
|
| cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
@@ -2244,7 +2526,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2244
2526
|
| source_root_id | string | | Y | | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
|
2245
2527
|
| src_base | string | Y | Y | | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
|
2246
2528
|
| ssh_port | int | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
|
2247
|
-
| ssh_private_key | string | Y | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY
|
2529
|
+
| ssh_private_key | string | Y | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
|
2248
2530
|
| ssh_private_key_passphrase | string | Y | | | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
|
2249
2531
|
| sshfp | string | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
|
2250
2532
|
| symlink_policy | string | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
|
@@ -2257,6 +2539,17 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2257
2539
|
| use_ascp4 | bool | Y | Y | | specify version of protocol |
|
2258
2540
|
| wss_enabled | bool | Y | Y | Y | Server has Web Socket service enabled |
|
2259
2541
|
| wss_port | int | Y | Y | Y | TCP port used for websocket service feed |
|
2542
|
+
| EX_ascp_args | array | Y | | | DEPRECATED: Use parameter ascp_args in option transfer_info<br/>Add native command line arguments to ascp |
|
2543
|
+
| EX_at_rest_password | string | Y | | | DEPRECATED: Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2544
|
+
| EX_file_list | string | Y | | | source file list |
|
2545
|
+
| EX_file_pair_list | string | Y | | | source file pair list |
|
2546
|
+
| EX_http_proxy_url | string | Y | | | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2547
|
+
| EX_http_transfer_jpeg | int | Y | | | HTTP transfers as JPEG file<br/>(-j {int}) |
|
2548
|
+
| EX_license_text | string | Y | | | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2549
|
+
| EX_no_read | bool | Y | | | no read source<br/>(--no-read) |
|
2550
|
+
| EX_no_write | bool | Y | | | no write on destination<br/>(--no-write) |
|
2551
|
+
| EX_proxy_password | string | Y | | | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL provided in parameter: proxy.<br/>(env:ASPERA_PROXY_PASS) |
|
2552
|
+
| EX_ssh_key_paths | array | Y | | | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2260
2553
|
|
2261
2554
|
#### Destination folder for transfers
|
2262
2555
|
|
@@ -2331,7 +2624,7 @@ So, by default, the list of files to transfer will be simply specified on the co
|
|
2331
2624
|
--sources=@lines:@stdin:
|
2332
2625
|
```
|
2333
2626
|
|
2334
|
-
- Using
|
2627
|
+
- Using Ruby code (one path per line in file)
|
2335
2628
|
|
2336
2629
|
```ruby
|
2337
2630
|
--sources=@ruby:'File.read("myfilelist.txt").split("\n")'
|
@@ -2363,11 +2656,63 @@ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
|
2363
2656
|
|
2364
2657
|
> **Note:** There are some specific rules to specify a file list when using **Aspera on Cloud**, refer to the AoC plugin section.
|
2365
2658
|
|
2659
|
+
#### Source directory structure on destination
|
2660
|
+
|
2661
|
+
This section is not specific to `ascli`, it is `ascp` behaviour.
|
2662
|
+
|
2663
|
+
The transfer destination is normally expected to designate a destination folder.
|
2664
|
+
|
2665
|
+
But there is one exception: The destination specifies the new item name when the following are met:
|
2666
|
+
|
2667
|
+
- there is a single source item (file or folder)
|
2668
|
+
- transfer spec `create_dir` is not set to `true` (`ascp` option `-d` not provided)
|
2669
|
+
- destination is not an existing folder
|
2670
|
+
- the `dirname` of destination is an existing folder
|
2671
|
+
|
2672
|
+
For this reason it is recommended to set `create_dir` to `true` for consistent behaviour between single and multiple items transfer, this is the default in `ascli`.
|
2673
|
+
|
2674
|
+
If a simple source file list is provided (no `destination` in `paths`, i.e. no `file_pair_list` provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.
|
2675
|
+
|
2676
|
+
The inner structure of source items that are folder is preserved on destination.
|
2677
|
+
|
2678
|
+
A leading `/` on destination is ignored (relative to docroot) unless docroot is not set (relative to home).
|
2679
|
+
|
2680
|
+
In the following table source folder `d3` contains 2 files: `f1` and `d4/f2`.
|
2681
|
+
|
2682
|
+
| Source files | Destination | Folders on Dest. |`create_dir`| Destination Files |
|
2683
|
+
|--------------|-------------|-------------------|------------|-----------------------------|
|
2684
|
+
| f1 | d/f | - | false | Error: `d` does not exist. |
|
2685
|
+
| f1 | d/f | d | false | d/f (renamed) |
|
2686
|
+
| f1 | d/f/. | d | false | d/f (renamed) |
|
2687
|
+
| f1 | d/f | d/f | false | d/f/f1 |
|
2688
|
+
| f1 f2 | d | d | false | d/f1 d/f2 |
|
2689
|
+
| d3 | d | - | false | d/f1 d/f2 (renamed) |
|
2690
|
+
| f1 | d | - | true | d/f1 |
|
2691
|
+
| f1 f2 | d | - | true | d/f1 d/f2 |
|
2692
|
+
| d1/f1 d2/f2 | d | - | true | d/f1 d/f2 |
|
2693
|
+
| d3 | d | - | true | d/d3/f1 d/d3/d4/f2 |
|
2694
|
+
|
2695
|
+
If a file par list is provided then it is possible to rename or specify a different destination folder for each source (relative to the destination).
|
2696
|
+
|
2697
|
+
If transfer spec has a `src_base`, it has the side effect that the simple source file list is considered as a file pair list, and so the lower structure of source folders is preserved on destination.
|
2698
|
+
|
2699
|
+
| Source files | Destination |`src_base`| Destination Files |
|
2700
|
+
|-------------------|-------------|----------|-----------------------------|
|
2701
|
+
| d1/d2/f2 d1/d3/f3 | d | d1 | d/d2/f2 d/d3/f3 |
|
2702
|
+
|
2703
|
+
Advanced Example: Send files `./file1` and `./folder2/files2` to server (e.g. `/Upload`) and keep the original file names and folders, i.e. send `file1` to `/Upload/file1` and `files2` to `/Upload/folder2/files2`.
|
2704
|
+
|
2705
|
+
- If files are specified as `./file1 ./folder2/files2`, then destination will be: `/Upload/file1 /Upload/files2`
|
2706
|
+
- One possibility is to specify a file pair list: `--src-type=pair file1 file1 folder2/files2 folder2/files2`
|
2707
|
+
- Another possibility is to specify a source base: `--src-base=$PWD $PWD/file1 $PWD/folder2/files2` (note that `.` cannot be used as source base)
|
2708
|
+
- Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--src-base=tmp_base tmp_base/file1 tmp_base/folder2/files2`
|
2709
|
+
- One can also similarly use `--sources=@ts` and specify the list of files in the `paths` field of transfer spec with both `source` and `destination` for each file.
|
2710
|
+
|
2366
2711
|
#### <a id="multisession"></a>Support of multi-session
|
2367
2712
|
|
2368
|
-
Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on
|
2713
|
+
Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on `direct` and `node` agents, not yet on connect.
|
2369
2714
|
|
2370
|
-
-
|
2715
|
+
- `--transfer=node`
|
2371
2716
|
|
2372
2717
|
```bash
|
2373
2718
|
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
|
@@ -2375,7 +2720,7 @@ Multi session, i.e. starting a transfer of a file set using multiple sessions (o
|
|
2375
2720
|
|
2376
2721
|
Multi-session is directly supported by the node daemon.
|
2377
2722
|
|
2378
|
-
-
|
2723
|
+
- `--transfer=direct`
|
2379
2724
|
|
2380
2725
|
```bash
|
2381
2726
|
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
|
@@ -2389,11 +2734,10 @@ When multi-session is used, one separate UDP port is used per session (refer to
|
|
2389
2734
|
|
2390
2735
|
#### Content protection
|
2391
2736
|
|
2392
|
-
Also known as Client-side encryption at rest (CSEAR), content protection allows a client to send files to a server
|
2393
|
-
|
2394
|
-
using a passphrase, only known by users sharing files. Files stay encrypted on server side.
|
2737
|
+
Also known as Client-side encryption at rest (CSEAR), content protection allows a client to send files to a server which will store them encrypted (upload), and decrypt files as they are being downloaded from a server, both using a passphrase, only known by users sharing files.
|
2738
|
+
Files stay encrypted on server side.
|
2395
2739
|
|
2396
|
-
|
2740
|
+
Activating CSEAR consists in using transfer spec parameters:
|
2397
2741
|
|
2398
2742
|
- `content_protection` : activate encryption (`encrypt` for upload) or decryption (`decrypt` for download)
|
2399
2743
|
- `content_protection_password` : the passphrase to be used.
|
@@ -2430,15 +2774,58 @@ Example: parameter to download a faspex package and decrypt on the fly
|
|
2430
2774
|
--ts=@json:'{"precalculate_job_size":true}'
|
2431
2775
|
```
|
2432
2776
|
|
2433
|
-
### <a id="
|
2777
|
+
### <a id="scheduler"></a>Scheduler
|
2434
2778
|
|
2435
2779
|
It is useful to configure automated scheduled execution.
|
2780
|
+
`ascli` does not provide an internal scheduler.
|
2781
|
+
Instead, use the service provided by the Operating system:
|
2436
2782
|
|
2437
|
-
####
|
2783
|
+
#### Windows Scheduler
|
2438
2784
|
|
2439
|
-
|
2785
|
+
Windows provides the [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page).
|
2786
|
+
It can be configured:
|
2440
2787
|
|
2441
|
-
|
2788
|
+
- Using utility [`schtasks.exe`](https://learn.microsoft.com/fr-fr/windows-server/administration/windows-commands/schtasks-create)
|
2789
|
+
|
2790
|
+
- Using powershell function [scheduletasks](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
2791
|
+
|
2792
|
+
- Using `taskschd.msc` (UI)
|
2793
|
+
|
2794
|
+
#### Unix-like Scheduler
|
2795
|
+
|
2796
|
+
Unix-like systems (Linux, ...) provide cron, configured using a [crontab](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
2797
|
+
|
2798
|
+
Linux also provides `anacron`, if tasks are hourly or daily.
|
2799
|
+
|
2800
|
+
For example, on Linux it is convenient to create a wrapping script, e.g. `cron_ascli` that will setup the environment (e.g. Ruby) to properly start `ascli`:
|
2801
|
+
|
2802
|
+
```bash
|
2803
|
+
#!/bin/bash
|
2804
|
+
# load the ruby environment
|
2805
|
+
. /etc/profile.d/rvm.sh
|
2806
|
+
rvm use 2.6 --quiet
|
2807
|
+
# set a timeout protection, just in case ascli is frozen
|
2808
|
+
tmout=30m
|
2809
|
+
# forward arguments to ascli
|
2810
|
+
exec timeout ${tmout} ascli "${@}"
|
2811
|
+
```
|
2812
|
+
|
2813
|
+
Example of cronjob created for user `xfer`.
|
2814
|
+
|
2815
|
+
```bash
|
2816
|
+
crontab<<EOF
|
2817
|
+
0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
|
2818
|
+
2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
|
2819
|
+
EOF
|
2820
|
+
```
|
2821
|
+
|
2822
|
+
> **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
|
2823
|
+
|
2824
|
+
### <a id="locking"></a>Locking for exclusive execution
|
2825
|
+
|
2826
|
+
In some cases one needs to ensure that `ascli` is not executed several times in parallel.
|
2827
|
+
|
2828
|
+
When `ascli` is executed automatically on a schedule basis, one generally desires that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
|
2442
2829
|
|
2443
2830
|
- Executing instances may pile-up and kill the system
|
2444
2831
|
- The same file may be transferred by multiple instances at the same time.
|
@@ -2456,7 +2843,7 @@ Usually the OS native scheduler already provides some sort of protection against
|
|
2456
2843
|
`ascli` natively supports a locking mechanism with option `lock_port`.
|
2457
2844
|
(Technically, this opens a local TCP server port, and fails if this port is already used, providing a local lock. Lock is released when process exits).
|
2458
2845
|
|
2459
|
-
|
2846
|
+
Testing `ascli` locking:
|
2460
2847
|
|
2461
2848
|
Run this same command in two separate terminals within less than 30 seconds:
|
2462
2849
|
|
@@ -2470,18 +2857,6 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
|
|
2470
2857
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
2471
2858
|
```
|
2472
2859
|
|
2473
|
-
#### <a id="scheduler"></a>Scheduler
|
2474
|
-
|
2475
|
-
`ascli` does not provide an internal scheduler.
|
2476
|
-
|
2477
|
-
Instead, use the service provided by the Operating system:
|
2478
|
-
|
2479
|
-
- Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
|
2480
|
-
- Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
2481
|
-
- etc...
|
2482
|
-
|
2483
|
-
Linux also provides `anacron`, if tasks are hourly or daily.
|
2484
|
-
|
2485
2860
|
### "Provençale"
|
2486
2861
|
|
2487
2862
|
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
|
@@ -2524,10 +2899,13 @@ One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo
|
|
2524
2899
|
|
2525
2900
|
### <a id="faux_testing"></a>`faux:` for testing
|
2526
2901
|
|
2527
|
-
This is an extract of the man page of `ascp`.
|
2902
|
+
This is an extract of the man page of `ascp`.
|
2903
|
+
This feature is a feature of `ascp`, not `ascli`.
|
2528
2904
|
|
2529
2905
|
This adapter can be used to simulate a file or a directory.
|
2530
2906
|
|
2907
|
+
To discard data at the destination, the destination argument is set to `faux://`.
|
2908
|
+
|
2531
2909
|
To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
|
2532
2910
|
|
2533
2911
|
```bash
|
@@ -2581,8 +2959,6 @@ The sequence parameter is applied as follows:
|
|
2581
2959
|
|
2582
2960
|
Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
|
2583
2961
|
|
2584
|
-
To discard data at the destination, the destination argument is set to `faux://` .
|
2585
|
-
|
2586
2962
|
Examples:
|
2587
2963
|
|
2588
2964
|
- Upload 20 gibibytes of random data to file myfile to directory /Upload
|
@@ -2608,7 +2984,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
2608
2984
|
```text
|
2609
2985
|
ascli -h
|
2610
2986
|
NAME
|
2611
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
2987
|
+
ascli -- a command line tool for Aspera Applications (v4.14.0)
|
2612
2988
|
|
2613
2989
|
SYNOPSIS
|
2614
2990
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -2631,59 +3007,59 @@ COMMANDS
|
|
2631
3007
|
OPTIONS
|
2632
3008
|
Options begin with a '-' (minus), and value is provided on command line.
|
2633
3009
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
2634
|
-
base64,
|
3010
|
+
base64, csvt, env, file, json, lines, list, path, ruby, secret, stdin, uri, val, zlib, preset, incps, vault
|
2635
3011
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
2636
3012
|
|
2637
3013
|
ARGS
|
2638
3014
|
Some commands require mandatory arguments, e.g. a path.
|
2639
3015
|
|
2640
3016
|
OPTIONS: global
|
2641
|
-
--interactive=ENUM
|
2642
|
-
--ask-options=ENUM
|
2643
|
-
--format=ENUM
|
2644
|
-
--display=ENUM
|
2645
|
-
--fields=VALUE
|
2646
|
-
--select=VALUE
|
2647
|
-
--table-style=VALUE
|
2648
|
-
--flat-hash=ENUM
|
2649
|
-
--transpose-single=ENUM
|
2650
|
-
--show-secrets=ENUM
|
2651
|
-
-h, --help Show this message
|
2652
|
-
--bash-comp
|
2653
|
-
--show-config Display parameters used for the provided action
|
2654
|
-
-r, --rest-debug
|
2655
|
-
-v, --version
|
2656
|
-
-w, --warnings
|
2657
|
-
--ui=ENUM
|
3017
|
+
--interactive=ENUM Use interactive input of missing params: [no], yes
|
3018
|
+
--ask-options=ENUM Ask even optional options: [no], yes
|
3019
|
+
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
|
3020
|
+
--display=ENUM Output only some information: [info], data, error
|
3021
|
+
--fields=VALUE Comma separated list of fields, or ALL, or DEF
|
3022
|
+
--select=VALUE Select only some items in lists: column, value (Hash)
|
3023
|
+
--table-style=VALUE Table display style
|
3024
|
+
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
3025
|
+
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
3026
|
+
--show-secrets=ENUM Show secrets on command output: [no], yes
|
3027
|
+
-h, --help Show this message
|
3028
|
+
--bash-comp Generate bash completion for command
|
3029
|
+
--show-config Display parameters used for the provided action
|
3030
|
+
-r, --rest-debug More debug for HTTP calls (REST)
|
3031
|
+
-v, --version Display version
|
3032
|
+
-w, --warnings Check for language warnings
|
3033
|
+
--ui=ENUM Method to start browser: text, [graphical]
|
2658
3034
|
--log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
|
2659
|
-
--logger=ENUM
|
2660
|
-
--lock-port=VALUE
|
2661
|
-
--http-options=VALUE
|
2662
|
-
--insecure=ENUM
|
2663
|
-
--once-only=ENUM
|
2664
|
-
--log-secrets=ENUM
|
2665
|
-
--cache-tokens=ENUM
|
3035
|
+
--logger=ENUM Logging method: [stderr], stdout, syslog
|
3036
|
+
--lock-port=VALUE Prevent dual execution of a command, e.g. in cron
|
3037
|
+
--http-options=VALUE Options for http socket (Hash)
|
3038
|
+
--insecure=ENUM Do not validate HTTPS certificate: [no], yes
|
3039
|
+
--once-only=ENUM Process only new items (some commands): [no], yes
|
3040
|
+
--log-secrets=ENUM Show passwords in logs: [no], yes
|
3041
|
+
--cache-tokens=ENUM Save and reuse Oauth tokens: no, [yes]
|
2666
3042
|
|
2667
3043
|
COMMAND: config
|
2668
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test
|
3044
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugin preset proxy_check smtp_settings vault wizard
|
2669
3045
|
OPTIONS:
|
2670
|
-
--query=VALUE
|
2671
|
-
--value=VALUE
|
2672
|
-
--property=VALUE
|
2673
|
-
--id=VALUE
|
3046
|
+
--query=VALUE Additional filter for for some commands (list/delete) (Hash)
|
3047
|
+
--value=VALUE Value for create, update, list filter (Hash) (deprecated: Use positional value for create/modify or option: query for list/delete)
|
3048
|
+
--property=VALUE Name of property to set (modify operation)
|
3049
|
+
--id=VALUE Resource identifier (deprecated: Use identifier after verb (modify,delete,show))
|
2674
3050
|
--bulk=ENUM Bulk operation (only some): [no], yes
|
2675
|
-
--bfail=ENUM Bulk operation error handling:
|
2676
|
-
--config-file=VALUE
|
2677
|
-
-N, --no-default
|
3051
|
+
--bfail=ENUM Bulk operation error handling: no, [yes]
|
3052
|
+
--config-file=VALUE Read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
|
3053
|
+
-N, --no-default Do not load default configuration for plugin
|
2678
3054
|
--override=ENUM Wizard: override existing value: [no], yes
|
2679
|
-
--use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id:
|
2680
|
-
--default=ENUM Wizard: set as default configuration for specified plugin (also: update):
|
3055
|
+
--use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: no, [yes]
|
3056
|
+
--default=ENUM Wizard: set as default configuration for specified plugin (also: update): no, [yes]
|
2681
3057
|
--test-mode=ENUM Wizard: skip private key check step: [no], yes
|
2682
|
-
-P, --presetVALUE
|
3058
|
+
-P, --presetVALUE Load the named option preset from current config file
|
2683
3059
|
--pkeypath=VALUE Wizard: path to private key for JWT
|
2684
3060
|
--ascp-path=VALUE Path to ascp
|
2685
3061
|
--use-product=VALUE Use ascp from specified product
|
2686
|
-
--smtp=VALUE SMTP configuration (
|
3062
|
+
--smtp=VALUE SMTP configuration (Hash)
|
2687
3063
|
--fpac=VALUE Proxy auto configuration script
|
2688
3064
|
--proxy-credentials=VALUE HTTP proxy credentials (Array with user and password)
|
2689
3065
|
--secret=VALUE Secret for access keys
|
@@ -2695,65 +3071,63 @@ OPTIONS:
|
|
2695
3071
|
--notif-template=VALUE Email ERB template for notification of transfers
|
2696
3072
|
--version-check-days=VALUE Period in days to check new version (zero to disable)
|
2697
3073
|
--plugin-folder=VALUE Folder where to find additional plugins
|
2698
|
-
--ts=VALUE Override transfer spec values (Hash
|
3074
|
+
--ts=VALUE Override transfer spec values (Hash)
|
2699
3075
|
--to-folder=VALUE Destination folder for transferred files
|
2700
3076
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
2701
|
-
--src-type=ENUM Type of file list: list, pair
|
2702
|
-
--transfer=ENUM Type of transfer agent: direct, node, connect, httpgw, trsdk
|
2703
|
-
--transfer-info=VALUE Parameters for transfer agent
|
2704
|
-
--progress=ENUM Type of progress bar: none, native, multi
|
3077
|
+
--src-type=ENUM Type of file list: [list], pair
|
3078
|
+
--transfer=ENUM Type of transfer agent: [direct], node, connect, httpgw, trsdk
|
3079
|
+
--transfer-info=VALUE Parameters for transfer agent (Hash)
|
3080
|
+
--progress=ENUM Type of progress bar: none, [native], multi
|
2705
3081
|
|
2706
3082
|
|
2707
3083
|
COMMAND: shares
|
2708
|
-
SUBCOMMANDS: admin health
|
3084
|
+
SUBCOMMANDS: admin files health
|
2709
3085
|
OPTIONS:
|
2710
3086
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2711
|
-
--username=VALUE
|
2712
|
-
--password=VALUE
|
2713
|
-
--type=ENUM Type of user/group for operations: any, local, ldap, saml
|
3087
|
+
--username=VALUE Username to log in
|
3088
|
+
--password=VALUE User's password
|
3089
|
+
--type=ENUM Type of user/group for operations: [any], local, ldap, saml
|
2714
3090
|
|
2715
3091
|
|
2716
3092
|
COMMAND: node
|
2717
|
-
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
|
3093
|
+
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
|
2718
3094
|
OPTIONS:
|
2719
3095
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2720
|
-
--username=VALUE
|
2721
|
-
--password=VALUE
|
2722
|
-
--validator=VALUE
|
3096
|
+
--username=VALUE Username to log in
|
3097
|
+
--password=VALUE User's password
|
3098
|
+
--validator=VALUE Identifier of validator (optional for central)
|
2723
3099
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
2724
|
-
--sync-name=VALUE
|
2725
|
-
--
|
2726
|
-
--token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
|
2727
|
-
--default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
|
3100
|
+
--sync-name=VALUE Sync name
|
3101
|
+
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
2728
3102
|
|
2729
3103
|
|
2730
3104
|
COMMAND: orchestrator
|
2731
3105
|
SUBCOMMANDS: health info plugins processes workflow
|
2732
3106
|
OPTIONS:
|
2733
3107
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2734
|
-
--username=VALUE
|
2735
|
-
--password=VALUE
|
2736
|
-
--params=VALUE parameters
|
2737
|
-
--result=VALUE
|
2738
|
-
--synchronous=ENUM
|
2739
|
-
--ret-style=ENUM
|
2740
|
-
--auth-style=ENUM
|
3108
|
+
--username=VALUE Username to log in
|
3109
|
+
--password=VALUE User's password
|
3110
|
+
--params=VALUE Start parameters (Hash)
|
3111
|
+
--result=VALUE Specify result value as: 'work step:parameter'
|
3112
|
+
--synchronous=ENUM Work step:parameter expected as result: [no], yes
|
3113
|
+
--ret-style=ENUM How return type is requested in api: header, [arg], ext
|
3114
|
+
--auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
|
2741
3115
|
|
2742
3116
|
|
2743
3117
|
COMMAND: bss
|
2744
3118
|
SUBCOMMANDS: subscription
|
2745
3119
|
OPTIONS:
|
2746
3120
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2747
|
-
--username=VALUE
|
2748
|
-
--password=VALUE
|
3121
|
+
--username=VALUE Username to log in
|
3122
|
+
--password=VALUE User's password
|
2749
3123
|
|
2750
3124
|
|
2751
3125
|
COMMAND: alee
|
2752
3126
|
SUBCOMMANDS: entitlement
|
2753
3127
|
OPTIONS:
|
2754
3128
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2755
|
-
--username=VALUE
|
2756
|
-
--password=VALUE
|
3129
|
+
--username=VALUE Username to log in
|
3130
|
+
--password=VALUE User's password
|
2757
3131
|
|
2758
3132
|
|
2759
3133
|
COMMAND: ats
|
@@ -2769,18 +3143,21 @@ OPTIONS:
|
|
2769
3143
|
|
2770
3144
|
|
2771
3145
|
COMMAND: faspex5
|
2772
|
-
SUBCOMMANDS: admin bearer_token gateway health
|
3146
|
+
SUBCOMMANDS: admin bearer_token gateway health packages postprocessing shared_folders user version
|
2773
3147
|
OPTIONS:
|
2774
3148
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2775
|
-
--username=VALUE
|
2776
|
-
--password=VALUE
|
3149
|
+
--username=VALUE Username to log in
|
3150
|
+
--password=VALUE User's password
|
2777
3151
|
--client-id=VALUE OAuth client identifier
|
2778
3152
|
--client-secret=VALUE OAuth client secret
|
2779
3153
|
--redirect-uri=VALUE OAuth redirect URI for web authentication
|
2780
|
-
--auth=ENUM OAuth type of authentication: boot, web, jwt
|
3154
|
+
--auth=ENUM OAuth type of authentication: boot, link, web, [jwt]
|
2781
3155
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
2782
|
-
--passphrase=VALUE RSA private key passphrase
|
2783
|
-
--
|
3156
|
+
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
3157
|
+
--link=VALUE Public link authorization (specific operations)
|
3158
|
+
--box=VALUE Package inbox, either shared inbox name or one of ["inbox", "inbox_history", "inbox_all", "inbox_all_history", "outbox", "outbox_history", "pending", "pending_history", "all"] or ALL
|
3159
|
+
--shared-folder=VALUE Send package with files from shared folder
|
3160
|
+
--group-type=ENUM Shared inbox or workgroup: [shared_inboxes], workgroups
|
2784
3161
|
|
2785
3162
|
|
2786
3163
|
COMMAND: cos
|
@@ -2799,49 +3176,50 @@ COMMAND: faspex
|
|
2799
3176
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
2800
3177
|
OPTIONS:
|
2801
3178
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2802
|
-
--username=VALUE
|
2803
|
-
--password=VALUE
|
2804
|
-
--link=VALUE
|
2805
|
-
--delivery-info=VALUE
|
2806
|
-
--source-name=VALUE
|
3179
|
+
--username=VALUE Username to log in
|
3180
|
+
--password=VALUE User's password
|
3181
|
+
--link=VALUE Public link for specific operation
|
3182
|
+
--delivery-info=VALUE Package delivery information (Hash)
|
3183
|
+
--source-name=VALUE Create package from remote source (by name)
|
2807
3184
|
--storage=VALUE Faspex local storage definition
|
2808
|
-
--recipient=VALUE
|
2809
|
-
--box=ENUM
|
3185
|
+
--recipient=VALUE Use if recipient is a dropbox (with *)
|
3186
|
+
--box=ENUM Package box: [inbox], archive, sent
|
2810
3187
|
|
2811
3188
|
|
2812
3189
|
COMMAND: preview
|
2813
3190
|
SUBCOMMANDS: check events scan test trevents
|
2814
3191
|
OPTIONS:
|
2815
3192
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2816
|
-
--username=VALUE
|
2817
|
-
--password=VALUE
|
2818
|
-
--skip-format=ENUM
|
2819
|
-
--folder-reset-cache=ENUM
|
2820
|
-
--skip-types=VALUE
|
2821
|
-
--previews-folder=VALUE
|
2822
|
-
--temp-folder=VALUE
|
2823
|
-
--skip-folders=VALUE
|
2824
|
-
--case=VALUE
|
2825
|
-
--scan-path=VALUE
|
2826
|
-
--scan-id=VALUE
|
2827
|
-
--mimemagic=ENUM
|
2828
|
-
--overwrite=ENUM
|
2829
|
-
--file-access=ENUM
|
2830
|
-
--max-size=VALUE
|
2831
|
-
--thumb-vid-scale=VALUE
|
2832
|
-
--thumb-vid-fraction=VALUE
|
2833
|
-
--thumb-img-size=VALUE
|
2834
|
-
--thumb-text-font=VALUE
|
2835
|
-
--video-conversion=ENUM
|
2836
|
-
--video-png-conv=ENUM
|
2837
|
-
--video-
|
2838
|
-
--video-
|
2839
|
-
--
|
2840
|
-
--blend-
|
2841
|
-
--blend-
|
2842
|
-
--blend-
|
2843
|
-
--
|
2844
|
-
--clips-
|
3193
|
+
--username=VALUE Username to log in
|
3194
|
+
--password=VALUE User's password
|
3195
|
+
--skip-format=ENUM Skip this preview format (multiple possible): png, mp4
|
3196
|
+
--folder-reset-cache=ENUM Force detection of generated preview by refresh cache: [no], header, read
|
3197
|
+
--skip-types=VALUE Skip types in comma separated list
|
3198
|
+
--previews-folder=VALUE Preview folder in storage root
|
3199
|
+
--temp-folder=VALUE Path to temp folder
|
3200
|
+
--skip-folders=VALUE List of folder to skip
|
3201
|
+
--case=VALUE Basename of output for for test
|
3202
|
+
--scan-path=VALUE Subpath in folder id to start scan in (default=/)
|
3203
|
+
--scan-id=VALUE Folder id in storage to start scan in, default is access key main folder id
|
3204
|
+
--mimemagic=ENUM Use Mime type detection of gem mimemagic: [no], yes
|
3205
|
+
--overwrite=ENUM When to overwrite result file: always, never, [mtime]
|
3206
|
+
--file-access=ENUM How to read and write files in repository: [local], remote
|
3207
|
+
--max-size=VALUE Maximum size (in bytes) of preview file
|
3208
|
+
--thumb-vid-scale=VALUE Png: video: size (ffmpeg scale argument)
|
3209
|
+
--thumb-vid-fraction=VALUE Png: video: time percent position of snapshot
|
3210
|
+
--thumb-img-size=VALUE Png: non-video: height (and width)
|
3211
|
+
--thumb-text-font=VALUE Png: plaintext: font to render text with imagemagick convert (identify -list font)
|
3212
|
+
--video-conversion=ENUM Mp4: method for preview generation: [reencode], blend, clips
|
3213
|
+
--video-png-conv=ENUM Mp4: method for thumbnail generation: [fixed], animated
|
3214
|
+
--video-scale=VALUE Mp4: all: video scale (ffmpeg)
|
3215
|
+
--video-start-sec=VALUE Mp4: all: start offset (seconds) of video preview
|
3216
|
+
--reencode-ffmpeg=VALUE Mp4: reencode: options to ffmpeg
|
3217
|
+
--blend-keyframes=VALUE Mp4: blend: # key frames
|
3218
|
+
--blend-pauseframes=VALUE Mp4: blend: # pause frames
|
3219
|
+
--blend-transframes=VALUE Mp4: blend: # transition blend frames
|
3220
|
+
--blend-fps=VALUE Mp4: blend: frame per second
|
3221
|
+
--clips-count=VALUE Mp4: clips: number of clips
|
3222
|
+
--clips-length=VALUE Mp4: clips: length in seconds of each clips
|
2845
3223
|
|
2846
3224
|
|
2847
3225
|
COMMAND: sync
|
@@ -2855,10 +3233,10 @@ COMMAND: aoc
|
|
2855
3233
|
SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
|
2856
3234
|
OPTIONS:
|
2857
3235
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2858
|
-
--username=VALUE
|
2859
|
-
--password=VALUE
|
2860
|
-
--auth=ENUM OAuth type of authentication: web, jwt
|
2861
|
-
--operation=ENUM
|
3236
|
+
--username=VALUE Username to log in
|
3237
|
+
--password=VALUE User's password
|
3238
|
+
--auth=ENUM OAuth type of authentication: web, [jwt]
|
3239
|
+
--operation=ENUM Client operation for transfers: [push], pull
|
2862
3240
|
--client-id=VALUE OAuth API client identifier
|
2863
3241
|
--client-secret=VALUE OAuth API client secret
|
2864
3242
|
--redirect-uri=VALUE OAuth API client redirect URI
|
@@ -2870,26 +3248,25 @@ OPTIONS:
|
|
2870
3248
|
--link=VALUE Public link to shared resource
|
2871
3249
|
--new-user-option=VALUE New user creation option for unknown package recipients
|
2872
3250
|
--from-folder=VALUE Source folder for Folder-to-Folder transfer
|
2873
|
-
--validate-metadata=ENUM Validate shared inbox metadata:
|
3251
|
+
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
2874
3252
|
|
2875
3253
|
COMMAND: node
|
2876
|
-
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
|
3254
|
+
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
|
2877
3255
|
OPTIONS:
|
2878
|
-
--validator=VALUE
|
3256
|
+
--validator=VALUE Identifier of validator (optional for central)
|
2879
3257
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
2880
|
-
--sync-name=VALUE
|
2881
|
-
--
|
2882
|
-
--token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
|
2883
|
-
--default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
|
3258
|
+
--sync-name=VALUE Sync name
|
3259
|
+
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
2884
3260
|
|
2885
3261
|
|
2886
3262
|
COMMAND: server
|
2887
3263
|
SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
|
2888
3264
|
OPTIONS:
|
2889
3265
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2890
|
-
--username=VALUE
|
2891
|
-
--password=VALUE
|
3266
|
+
--username=VALUE Username to log in
|
3267
|
+
--password=VALUE User's password
|
2892
3268
|
--ssh-keys=VALUE SSH key path list (Array or single)
|
3269
|
+
--passphrase=VALUE SSH private key passphrase
|
2893
3270
|
--ssh-options=VALUE SSH options (Hash)
|
2894
3271
|
|
2895
3272
|
|
@@ -2897,10 +3274,10 @@ COMMAND: console
|
|
2897
3274
|
SUBCOMMANDS: health transfer
|
2898
3275
|
OPTIONS:
|
2899
3276
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2900
|
-
--username=VALUE
|
2901
|
-
--password=VALUE
|
2902
|
-
--filter-from=DATE
|
2903
|
-
--filter-to=DATE
|
3277
|
+
--username=VALUE Username to log in
|
3278
|
+
--password=VALUE User's password
|
3279
|
+
--filter-from=DATE Only after date
|
3280
|
+
--filter-to=DATE Only before date
|
2904
3281
|
|
2905
3282
|
|
2906
3283
|
```
|
@@ -2948,10 +3325,10 @@ For this, specify the option: `--use-generic-client=no`.
|
|
2948
3325
|
|
2949
3326
|
This will guide you through the steps to create.
|
2950
3327
|
|
2951
|
-
If the wizard does not detect the application but you know the application, you can force it using option `
|
3328
|
+
If the wizard does not detect the application but you know the application, you can force it using option `query`:
|
2952
3329
|
|
2953
3330
|
```bash
|
2954
|
-
ascli config wizard --
|
3331
|
+
ascli config wizard --query=aoc
|
2955
3332
|
```
|
2956
3333
|
|
2957
3334
|
### <a id="aocmanual"></a>Configuration: using manual setup
|
@@ -2962,7 +3339,7 @@ ascli config wizard --value=aoc
|
|
2962
3339
|
|
2963
3340
|
Several types of OAuth authentication are supported:
|
2964
3341
|
|
2965
|
-
- JSON Web Token (JWT) : authentication is secured by a private key (recommended for
|
3342
|
+
- JSON Web Token (JWT) : authentication is secured by a private key (recommended for `ascli`)
|
2966
3343
|
- Web based authentication : authentication is made by user using a browser
|
2967
3344
|
- URL Token : external users authentication with url tokens (public links)
|
2968
3345
|
|
@@ -3004,7 +3381,6 @@ If you did not use the wizard, you can also manually create a [option preset](#l
|
|
3004
3381
|
|
3005
3382
|
Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interactive input (client info from previous step):
|
3006
3383
|
|
3007
|
-
<!-- spellchecker: disable -->
|
3008
3384
|
```bash
|
3009
3385
|
ascli config preset ask my_aoc_org url client_id client_secret
|
3010
3386
|
option: url> https://myorg.ibmaspera.com/
|
@@ -3012,7 +3388,6 @@ option: client_id> my_client_id_here
|
|
3012
3388
|
option: client_secret> my_client_secret_here
|
3013
3389
|
updated: my_aoc_org
|
3014
3390
|
```
|
3015
|
-
<!-- spellchecker: enable -->
|
3016
3391
|
|
3017
3392
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
3018
3393
|
|
@@ -3051,11 +3426,11 @@ ascli aoc admin res client list
|
|
3051
3426
|
```
|
3052
3427
|
|
3053
3428
|
```output
|
3054
|
-
|
3055
|
-
|
3056
|
-
|
3057
|
-
|
3058
|
-
|
3429
|
+
+------------+---------------+
|
3430
|
+
| id | name |
|
3431
|
+
+------------+---------------+
|
3432
|
+
| my_BJbQiFw | my-client-app |
|
3433
|
+
+------------+---------------+
|
3059
3434
|
```
|
3060
3435
|
|
3061
3436
|
```bash
|
@@ -3087,12 +3462,12 @@ ascli aoc admin res user list
|
|
3087
3462
|
```
|
3088
3463
|
|
3089
3464
|
```output
|
3090
|
-
|
3091
|
-
|
3092
|
-
|
3093
|
-
|
3094
|
-
|
3095
|
-
|
3465
|
+
+--------+----------------+
|
3466
|
+
| id | name |
|
3467
|
+
+--------+----------------+
|
3468
|
+
| 109952 | Tech Support |
|
3469
|
+
| 109951 | LAURENT MARTIN |
|
3470
|
+
+--------+----------------+
|
3096
3471
|
```
|
3097
3472
|
|
3098
3473
|
```ruby
|
@@ -3221,7 +3596,7 @@ To execute an action on a specific resource, select it using one of those method
|
|
3221
3596
|
|
3222
3597
|
- *recommended*: give id directly on command line *after the action*: `aoc admin res node show 123`
|
3223
3598
|
- give name on command line *after the action*: `aoc admin res node show name abc`
|
3224
|
-
- provide option `id` : `aoc admin res node show
|
3599
|
+
- provide option `id` : `aoc admin res node show 123`
|
3225
3600
|
- provide option `name` : `aoc admin res node show --name=abc`
|
3226
3601
|
|
3227
3602
|
#### <a id="res_create"></a>Creating a resource
|
@@ -3276,7 +3651,7 @@ The secret is provided using the `secret` option.
|
|
3276
3651
|
For example in a command like:
|
3277
3652
|
|
3278
3653
|
```bash
|
3279
|
-
ascli aoc admin res node
|
3654
|
+
ascli aoc admin res node 123 --secret="my_secret_here" v3 info
|
3280
3655
|
```
|
3281
3656
|
|
3282
3657
|
It is also possible to store secrets in the [secret vault](#vault) and then automatically find the related secret using the [config finder](#config_finder).
|
@@ -3334,6 +3709,36 @@ The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports
|
|
3334
3709
|
|
3335
3710
|
Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
|
3336
3711
|
|
3712
|
+
#### Files with type `link`
|
3713
|
+
|
3714
|
+
Aspera on Cloud Shared folders are implemented through a special type of file: `link`.
|
3715
|
+
A `link` is the equivalent of a symbolic link on a file system: it points to another folder (not file).
|
3716
|
+
|
3717
|
+
Listing a link (in terminal position of path) will information on the link itself, not the content of the folder it points to.
|
3718
|
+
To list the target folder content, add a `/` a the end of the path.
|
3719
|
+
|
3720
|
+
Example:
|
3721
|
+
|
3722
|
+
```console
|
3723
|
+
$ ascli aoc files br the_link
|
3724
|
+
Current Workspace: Default (default)
|
3725
|
+
+------------+------+----------------+------+----------------------+--------------+
|
3726
|
+
| name | type | recursive_size | size | modified_time | access_level |
|
3727
|
+
+------------+------+----------------+------+----------------------+--------------+
|
3728
|
+
| the_link | link | | | 2021-04-28T09:17:14Z | edit |
|
3729
|
+
+------------+------+----------------+------+----------------------+--------------+
|
3730
|
+
```
|
3731
|
+
|
3732
|
+
```console
|
3733
|
+
$ ascli aoc files br the_link/
|
3734
|
+
Current Workspace: Default (default)
|
3735
|
+
+-------------+------+----------------+------+----------------------+--------------+
|
3736
|
+
| name | type | recursive_size | size | modified_time | access_level |
|
3737
|
+
+-------------+------+----------------+------+----------------------+--------------+
|
3738
|
+
| file_inside | file | | | 2021-04-26T09:00:00Z | edit |
|
3739
|
+
+-------------+------+----------------+------+----------------------+--------------+
|
3740
|
+
```
|
3741
|
+
|
3337
3742
|
#### Example: Bulk creation of users
|
3338
3743
|
|
3339
3744
|
```bash
|
@@ -3341,29 +3746,27 @@ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.
|
|
3341
3746
|
```
|
3342
3747
|
|
3343
3748
|
```output
|
3344
|
-
|
3345
|
-
|
3346
|
-
|
3347
|
-
|
3348
|
-
|
3349
|
-
|
3749
|
+
+-------+---------+
|
3750
|
+
| id | status |
|
3751
|
+
+-------+---------+
|
3752
|
+
| 98398 | created |
|
3753
|
+
| 98399 | created |
|
3754
|
+
+-------+---------+
|
3350
3755
|
```
|
3351
3756
|
|
3352
3757
|
#### Example: Find with filter and delete
|
3353
3758
|
|
3354
|
-
<!-- spellchecker: disable -->
|
3355
|
-
|
3356
3759
|
```bash
|
3357
3760
|
ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
|
3358
3761
|
```
|
3359
3762
|
|
3360
3763
|
```output
|
3361
|
-
|
3362
|
-
|
3363
|
-
|
3364
|
-
|
3365
|
-
|
3366
|
-
|
3764
|
+
+-------+------------------------+
|
3765
|
+
| id | email |
|
3766
|
+
+-------+------------------------+
|
3767
|
+
| 98398 | dummyuser1@example.com |
|
3768
|
+
| 98399 | dummyuser2@example.com |
|
3769
|
+
+-------+------------------------+
|
3367
3770
|
```
|
3368
3771
|
|
3369
3772
|
```bash
|
@@ -3379,20 +3782,18 @@ echo $thelist
|
|
3379
3782
|
```
|
3380
3783
|
|
3381
3784
|
```bash
|
3382
|
-
ascli aoc admin res user
|
3785
|
+
ascli aoc admin res user delete @json:"$thelist" --bulk=yes
|
3383
3786
|
```
|
3384
3787
|
|
3385
3788
|
```output
|
3386
|
-
|
3387
|
-
|
3388
|
-
|
3389
|
-
|
3390
|
-
|
3391
|
-
|
3789
|
+
+-------+---------+
|
3790
|
+
| id | status |
|
3791
|
+
+-------+---------+
|
3792
|
+
| 98398 | deleted |
|
3793
|
+
| 98399 | deleted |
|
3794
|
+
+-------+---------+
|
3392
3795
|
```
|
3393
3796
|
|
3394
|
-
<!-- spellchecker: enable -->
|
3395
|
-
|
3396
3797
|
#### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
|
3397
3798
|
|
3398
3799
|
```ruby
|
@@ -3408,13 +3809,13 @@ ascli aoc user workspaces list
|
|
3408
3809
|
```
|
3409
3810
|
|
3410
3811
|
```output
|
3411
|
-
|
3412
|
-
|
3413
|
-
|
3414
|
-
|
3415
|
-
|
3416
|
-
|
3417
|
-
|
3812
|
+
+------+----------------------------+
|
3813
|
+
| id | name |
|
3814
|
+
+------+----------------------------+
|
3815
|
+
| 16 | Engineering |
|
3816
|
+
| 17 | Marketing |
|
3817
|
+
| 18 | Sales |
|
3818
|
+
+------+----------------------------+
|
3418
3819
|
```
|
3419
3820
|
|
3420
3821
|
#### Example: Create a sub access key in a "node"
|
@@ -3422,13 +3823,13 @@ ascli aoc user workspaces list
|
|
3422
3823
|
Creation of a sub-access key is like creation of access key with the following difference: authentication to node API is made with accesskey (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
|
3423
3824
|
|
3424
3825
|
```bash
|
3425
|
-
ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create
|
3826
|
+
ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create @json:'{"storage":{"path":"/folder1"}}'
|
3426
3827
|
```
|
3427
3828
|
|
3428
3829
|
#### Example: Display transfer events (ops/transfer)
|
3429
3830
|
|
3430
3831
|
```bash
|
3431
|
-
ascli aoc admin res node --secret=_secret_ v3 transfer list --
|
3832
|
+
ascli aoc admin res node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
|
3432
3833
|
```
|
3433
3834
|
|
3434
3835
|
Examples of query (TODO: cleanup):
|
@@ -3454,16 +3855,16 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
|
|
3454
3855
|
```
|
3455
3856
|
|
3456
3857
|
```output
|
3457
|
-
|
3458
|
-
|
3459
|
-
|
3460
|
-
|
3461
|
-
|
3462
|
-
|
3463
|
-
|
3464
|
-
|
3465
|
-
|
3466
|
-
|
3858
|
+
+-------------+---------+----------------------------------+
|
3859
|
+
| member_type | manager | member.email |
|
3860
|
+
+-------------+---------+----------------------------------+
|
3861
|
+
| user | true | john.curtis@email.com |
|
3862
|
+
| user | false | laurent.martin.aspera@fr.ibm.com |
|
3863
|
+
| user | false | jean.dupont@me.com |
|
3864
|
+
| user | false | another.user@example.com |
|
3865
|
+
| group | false | |
|
3866
|
+
| user | false | aspera.user@gmail.com |
|
3867
|
+
+-------------+---------+----------------------------------+
|
3467
3868
|
```
|
3468
3869
|
|
3469
3870
|
Other query parameters:
|
@@ -3519,12 +3920,12 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
|
|
3519
3920
|
```
|
3520
3921
|
|
3521
3922
|
```output
|
3522
|
-
|
3523
|
-
|
3524
|
-
|
3525
|
-
|
3526
|
-
|
3527
|
-
|
3923
|
+
+-------------------------------+
|
3924
|
+
| email |
|
3925
|
+
+-------------------------------+
|
3926
|
+
| John.curtis@acme.com |
|
3927
|
+
| Jean.Dupont@tropfort.com |
|
3928
|
+
+-------------------------------+
|
3528
3929
|
```
|
3529
3930
|
|
3530
3931
|
#### Example: List "Limited" users
|
@@ -3615,8 +4016,6 @@ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AW
|
|
3615
4016
|
|
3616
4017
|
#### Example: create registration key to register a node
|
3617
4018
|
|
3618
|
-
<!-- spellchecker: disable -->
|
3619
|
-
|
3620
4019
|
```bash
|
3621
4020
|
ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
3622
4021
|
```
|
@@ -3628,7 +4027,7 @@ jfqslfdjlfdjfhdjklqfhdkl
|
|
3628
4027
|
#### Example: delete all registration keys
|
3629
4028
|
|
3630
4029
|
```bash
|
3631
|
-
ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete --bulk=yes
|
4030
|
+
ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete @lines:@stdin: --bulk=yes
|
3632
4031
|
```
|
3633
4032
|
|
3634
4033
|
```output
|
@@ -3642,8 +4041,6 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
|
|
3642
4041
|
+-----+---------+
|
3643
4042
|
```
|
3644
4043
|
|
3645
|
-
<!-- spellchecker: enable -->
|
3646
|
-
|
3647
4044
|
#### Example: Create a Node
|
3648
4045
|
|
3649
4046
|
AoC nodes as actually composed with two related entities:
|
@@ -3715,12 +4112,13 @@ The webmail-like application.
|
|
3715
4112
|
General syntax:
|
3716
4113
|
|
3717
4114
|
```bash
|
3718
|
-
ascli aoc packages send
|
4115
|
+
ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
|
3719
4116
|
```
|
3720
4117
|
|
3721
4118
|
Notes:
|
3722
4119
|
|
3723
|
-
-
|
4120
|
+
- Package creation parameter are sent as positional mandatory parameter.
|
4121
|
+
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
3724
4122
|
- List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
|
3725
4123
|
- Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
3726
4124
|
- Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
@@ -3732,19 +4130,19 @@ Notes:
|
|
3732
4130
|
#### Example: Send a package with one file to two users, using their email
|
3733
4131
|
|
3734
4132
|
```bash
|
3735
|
-
ascli aoc packages send
|
4133
|
+
ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
|
3736
4134
|
```
|
3737
4135
|
|
3738
4136
|
#### Example: Send a package to a shared inbox with metadata
|
3739
4137
|
|
3740
4138
|
```bash
|
3741
|
-
ascli aoc packages send --workspace=eudemo
|
4139
|
+
ascli aoc packages send --workspace=eudemo @json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
|
3742
4140
|
```
|
3743
4141
|
|
3744
4142
|
It is also possible to use identifiers and API parameters:
|
3745
4143
|
|
3746
4144
|
```bash
|
3747
|
-
ascli aoc packages send --workspace=eudemo
|
4145
|
+
ascli aoc packages send --workspace=eudemo @json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
3748
4146
|
```
|
3749
4147
|
|
3750
4148
|
#### Example: List packages in a given shared inbox
|
@@ -3755,7 +4153,7 @@ When user packages are listed, the following query is used:
|
|
3755
4153
|
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
3756
4154
|
```
|
3757
4155
|
|
3758
|
-
To list packages in a shared inbox, the query has to be specified with
|
4156
|
+
To list packages in a shared inbox, the query has to be specified with the the shared inbox by name or its identifier.
|
3759
4157
|
Additional parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode).
|
3760
4158
|
The current workspace is added unless specified in the query.
|
3761
4159
|
|
@@ -3803,7 +4201,7 @@ ascli aoc files browse /src_folder
|
|
3803
4201
|
Let's send a package with the file `10M.dat` from subfolder /src_folder in a package:
|
3804
4202
|
|
3805
4203
|
```bash
|
3806
|
-
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send
|
4204
|
+
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["laurent.martin.aspera@fr.ibm.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
|
3807
4205
|
```
|
3808
4206
|
|
3809
4207
|
#### <a id="aoccargo"></a>Receive new packages only (Cargo)
|
@@ -3811,10 +4209,10 @@ ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc p
|
|
3811
4209
|
It is possible to automatically download new packages, like using Aspera Cargo:
|
3812
4210
|
|
3813
4211
|
```bash
|
3814
|
-
ascli aoc packages recv
|
4212
|
+
ascli aoc packages recv ALL --once-only=yes --lock-port=12345
|
3815
4213
|
```
|
3816
4214
|
|
3817
|
-
-
|
4215
|
+
- `ALL` (case sensitive) will download all packages
|
3818
4216
|
- `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder
|
3819
4217
|
- `--lock-port=12345` ensures that only one instance is started at the same time, to avoid running two downloads in parallel
|
3820
4218
|
|
@@ -3843,46 +4241,28 @@ ascli aoc files download <single file path>
|
|
3843
4241
|
|
3844
4242
|
#### Shared folders
|
3845
4243
|
|
3846
|
-
Shared folder by users are managed through **permissions**.
|
4244
|
+
Shared folder created by users are managed through **permissions**.
|
3847
4245
|
For creation, parameters are the same as for node api [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
|
3848
|
-
`ascli` expects the same payload for creation, but it will automatically
|
3849
|
-
Also, the pseudo key `with` is
|
4246
|
+
`ascli` expects the same payload for creation, but it will automatically populate required tags if needed.
|
4247
|
+
Also, the pseudo key `with` is available: it will lookup the name in the contacts and fill the proper type and id.
|
3850
4248
|
The pseudo parameter `link_name` allows changing default "shared as" name.
|
3851
4249
|
|
3852
4250
|
- List permissions on a shared folder as user
|
3853
4251
|
|
3854
4252
|
```bash
|
3855
|
-
ascli aoc files
|
4253
|
+
ascli aoc files perm /shared_folder_test1 list
|
3856
4254
|
```
|
3857
4255
|
|
3858
4256
|
- Share a personal folder with other users
|
3859
4257
|
|
3860
4258
|
```bash
|
3861
|
-
ascli aoc files
|
4259
|
+
ascli aoc files perm /shared_folder_test1 create @json:'{"with":"laurent"}'
|
3862
4260
|
```
|
3863
4261
|
|
3864
4262
|
- Revoke shared access
|
3865
4263
|
|
3866
4264
|
```bash
|
3867
|
-
ascli aoc files
|
3868
|
-
```
|
3869
|
-
|
3870
|
-
- List shared folders in node
|
3871
|
-
|
3872
|
-
```bash
|
3873
|
-
ascli aoc admin res node --id=8669 shared_folders
|
3874
|
-
```
|
3875
|
-
|
3876
|
-
- List shared folders in workspace
|
3877
|
-
|
3878
|
-
```bash
|
3879
|
-
ascli aoc admin res workspace --id=10818 shared_folders
|
3880
|
-
```
|
3881
|
-
|
3882
|
-
- List members of shared folder
|
3883
|
-
|
3884
|
-
```bash
|
3885
|
-
ascli aoc admin res node --id=8669 v4 perm 82 show
|
4265
|
+
ascli aoc files perm /shared_folder_test1 delete 6161
|
3886
4266
|
```
|
3887
4267
|
|
3888
4268
|
#### Cross Organization transfers
|
@@ -3917,42 +4297,52 @@ Explanation:
|
|
3917
4297
|
|
3918
4298
|
#### Find Files
|
3919
4299
|
|
3920
|
-
The command `aoc files find [--
|
4300
|
+
The command `aoc files find [--query=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
|
3921
4301
|
|
3922
4302
|
The expression can be of 3 formats:
|
3923
4303
|
|
3924
4304
|
- empty (default) : all files, equivalent to value: `exec:true`
|
3925
|
-
- not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax
|
4305
|
+
- not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax, equivalent to value: `exec:f['name'].match(/expression/)`
|
3926
4306
|
|
3927
|
-
For instance, to find files with a special extension, use `--
|
4307
|
+
For instance, to find files with a special extension, use `--query='\.myext$'`
|
3928
4308
|
|
3929
|
-
- starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true
|
4309
|
+
- starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true.
|
3930
4310
|
|
3931
|
-
Examples of expressions: (using like this: `--
|
4311
|
+
Examples of expressions: (using like this: `--query=exec:'<expression>'`)
|
3932
4312
|
|
3933
4313
|
- Find files more recent than 100 days
|
3934
4314
|
|
3935
|
-
```
|
3936
|
-
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
3937
|
-
```
|
4315
|
+
```ruby
|
4316
|
+
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
4317
|
+
```
|
3938
4318
|
|
3939
4319
|
- Find files older than 1 year on a given node and store in file list
|
3940
4320
|
|
3941
|
-
```
|
3942
|
-
|
3943
|
-
```
|
4321
|
+
```ruby
|
4322
|
+
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
4323
|
+
```
|
4324
|
+
|
4325
|
+
```bash
|
4326
|
+
ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find / --fields=path --query='exec:<above expression here>' --format=csv > my_file_list.txt
|
4327
|
+
```
|
4328
|
+
|
4329
|
+
- Find files larger than 1MB
|
4330
|
+
|
4331
|
+
```ruby
|
4332
|
+
f["type"].eql?("file") and f["size"].to_i>1000000
|
4333
|
+
```
|
3944
4334
|
|
3945
4335
|
- Delete the files, one by one
|
3946
4336
|
|
3947
|
-
```bash
|
3948
|
-
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 delete "$path" ;done
|
3949
|
-
```
|
4337
|
+
```bash
|
4338
|
+
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 delete "$path" ;done
|
4339
|
+
```
|
3950
4340
|
|
3951
4341
|
- Delete the files in bulk
|
3952
4342
|
|
3953
|
-
```bash
|
3954
|
-
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my_secret_here' v3 delete @lines:@stdin:
|
3955
|
-
```
|
4343
|
+
```bash
|
4344
|
+
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my_secret_here' v3 delete @lines:@stdin:
|
4345
|
+
```
|
3956
4346
|
|
3957
4347
|
### AoC sample commands
|
3958
4348
|
|
@@ -3991,18 +4381,18 @@ aoc admin res workspace_membership list
|
|
3991
4381
|
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do browse /
|
3992
4382
|
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do delete /folder1
|
3993
4383
|
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do mkdir /folder1
|
3994
|
-
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 access_key create
|
4384
|
+
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
3995
4385
|
aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret do v3 events
|
3996
4386
|
aoc admin resource node do name my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key delete testsub1
|
3997
4387
|
aoc admin resource workspace list
|
3998
4388
|
aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
3999
4389
|
aoc admin subscription
|
4000
|
-
aoc automation workflow action my_wf_id create
|
4001
|
-
aoc automation workflow create
|
4390
|
+
aoc automation workflow action my_wf_id create @json:'{"name":"toto"}' \
|
4391
|
+
aoc automation workflow create @json:'{"name":"test_workflow"}'
|
4002
4392
|
aoc automation workflow delete my_wf_id
|
4003
4393
|
aoc automation workflow list
|
4394
|
+
aoc automation workflow list --query=@json:'{"show_org_workflows":"true"}' --scope=admin:all
|
4004
4395
|
aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
|
4005
|
-
aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
|
4006
4396
|
aoc bearer_token --display=data --scope=user:all
|
4007
4397
|
aoc faspex
|
4008
4398
|
aoc files bearer /
|
@@ -4011,25 +4401,28 @@ aoc files browse /
|
|
4011
4401
|
aoc files browse / --link=my_aoc_publink_folder
|
4012
4402
|
aoc files delete /testsrc
|
4013
4403
|
aoc files download --transfer=connect /200KB.1
|
4014
|
-
aoc files
|
4015
|
-
aoc files file permission --path=my_aoc_test_folder list
|
4016
|
-
aoc files file show --path=/200KB.1
|
4017
|
-
aoc files file show my_file_id
|
4018
|
-
aoc files find / --value='\.partial$'
|
4404
|
+
aoc files find / --query='\.partial$'
|
4019
4405
|
aoc files http_node_download --to-folder=. /200KB.1
|
4020
4406
|
aoc files mkdir /testsrc
|
4407
|
+
aoc files modify my_aoc_test_folder
|
4408
|
+
aoc files permission my_aoc_test_folder list
|
4021
4409
|
aoc files rename /somefolder testdst
|
4022
|
-
aoc files short_link create
|
4023
|
-
aoc files short_link create
|
4024
|
-
aoc files short_link list
|
4025
|
-
aoc files
|
4026
|
-
aoc files
|
4410
|
+
aoc files short_link create /testdst private
|
4411
|
+
aoc files short_link create testdst public
|
4412
|
+
aoc files short_link list /testdst private
|
4413
|
+
aoc files show %id:my_file_id
|
4414
|
+
aoc files show /200KB.1
|
4415
|
+
aoc files sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
|
4416
|
+
aoc files sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
|
4417
|
+
aoc files sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
|
4418
|
+
aoc files sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
|
4419
|
+
aoc files thumbnail my_aoc_media_file
|
4027
4420
|
aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
|
4028
4421
|
aoc files upload --to-folder=/ testfile.bin --link=my_aoc_publink_folder
|
4029
4422
|
aoc files upload --to-folder=/testsrc testfile.bin
|
4030
4423
|
aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
|
4031
4424
|
aoc files v3 info
|
4032
|
-
aoc gateway
|
4425
|
+
aoc gateway https://localhost:12345/aspera/faspex
|
4033
4426
|
aoc org --link=my_aoc_publink_recv_from_aocuser
|
4034
4427
|
aoc organization
|
4035
4428
|
aoc packages browse "my_package_id" /contents
|
@@ -4038,13 +4431,13 @@ aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-rec
|
|
4038
4431
|
aoc packages recv "my_package_id" --to-folder=.
|
4039
4432
|
aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
|
4040
4433
|
aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"my_aoc_shbx_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
4041
|
-
aoc packages send --
|
4042
|
-
aoc packages send --
|
4043
|
-
aoc packages send --
|
4044
|
-
aoc packages send
|
4045
|
-
aoc packages send
|
4046
|
-
aoc packages send
|
4047
|
-
aoc packages send
|
4434
|
+
aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
|
4435
|
+
aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
|
4436
|
+
aoc packages send --workspace="my_aoc_shbx_ws" @json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
|
4437
|
+
aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
|
4438
|
+
aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
|
4439
|
+
aoc packages send @json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
|
4440
|
+
aoc packages send @json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
|
4048
4441
|
aoc packages shared_inboxes list
|
4049
4442
|
aoc remind --username=my_aoc_user_email
|
4050
4443
|
aoc servers
|
@@ -4069,8 +4462,6 @@ If you are using ATS as part of AoC, then authentication is through AoC, not IBM
|
|
4069
4462
|
|
4070
4463
|
First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
|
4071
4464
|
|
4072
|
-
<!-- spellchecker: disable -->
|
4073
|
-
|
4074
4465
|
```bash
|
4075
4466
|
ibmcloud iam api-key-create mykeyname -d 'my sample key'
|
4076
4467
|
```
|
@@ -4090,6 +4481,7 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
|
|
4090
4481
|
```
|
4091
4482
|
|
4092
4483
|
References:
|
4484
|
+
<!-- spellchecker: disable -->
|
4093
4485
|
|
4094
4486
|
- [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
|
4095
4487
|
- [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
|
@@ -4098,8 +4490,6 @@ References:
|
|
4098
4490
|
|
4099
4491
|
Then, to register the key by default for the ats plugin, create a preset. Execute:
|
4100
4492
|
|
4101
|
-
<!-- spellchecker: disable -->
|
4102
|
-
|
4103
4493
|
```bash
|
4104
4494
|
ascli config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here
|
4105
4495
|
```
|
@@ -4138,8 +4528,6 @@ ascli ats api_key create
|
|
4138
4528
|
ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
4139
4529
|
```
|
4140
4530
|
|
4141
|
-
<!-- spellchecker: enable -->
|
4142
|
-
|
4143
4531
|
### <a id="ats_params"></a>ATS Access key creation parameters
|
4144
4532
|
|
4145
4533
|
When creating an ATS access key, the option `params` must contain an extended value with the creation parameters.
|
@@ -4176,7 +4564,7 @@ ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":
|
|
4176
4564
|
delete all my access keys:
|
4177
4565
|
|
4178
4566
|
```bash
|
4179
|
-
|
4567
|
+
ascli ats access_key list --field=id --format=csv | ascli ats access_key delete @lines:@stdin: --bulk=yes
|
4180
4568
|
```
|
4181
4569
|
|
4182
4570
|
The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
|
@@ -4214,7 +4602,6 @@ then commands `ascp` (for transfers) and `ascmd` (for file operations) are execu
|
|
4214
4602
|
### Server sample commands
|
4215
4603
|
|
4216
4604
|
```bash
|
4217
|
-
server br /
|
4218
4605
|
server browse /
|
4219
4606
|
server browse NEW_SERVER_FOLDER/testfile.bin
|
4220
4607
|
server browse folder_1/target_hot
|
@@ -4232,10 +4619,10 @@ server md5sum NEW_SERVER_FOLDER/testfile.bin
|
|
4232
4619
|
server mkdir NEW_SERVER_FOLDER --logger=stdout
|
4233
4620
|
server mkdir folder_1/target_hot
|
4234
4621
|
server mv folder_1/200KB.2 folder_1/to.delete
|
4235
|
-
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"
|
4236
|
-
server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"
|
4237
|
-
server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"
|
4238
|
-
server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"
|
4622
|
+
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"}}'
|
4623
|
+
server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"my_local_sync_dir"}]}'
|
4624
|
+
server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"},"remote":{"path":"'"NEW_SERVER_FOLDER"'"},"reset":true,"quiet":false}'
|
4625
|
+
server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"my_local_sync_dir","reset":true}]}'
|
4239
4626
|
server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
|
4240
4627
|
server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
|
4241
4628
|
server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
|
@@ -4266,7 +4653,7 @@ ascli server --url=ssh://_server_address_:33001 ... --ts=@json:'{"token":"Basic
|
|
4266
4653
|
|
4267
4654
|
> **Note:** If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
|
4268
4655
|
|
4269
|
-
The value of the `ssh_keys` option can be a single value or an
|
4656
|
+
The value of the `ssh_keys` option can be a single value or an `Array`.
|
4270
4657
|
Each value is a **path** to a private key and is expanded (`~` is replaced with the user's home folder).
|
4271
4658
|
|
4272
4659
|
Examples:
|
@@ -4277,9 +4664,9 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
|
|
4277
4664
|
ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
|
4278
4665
|
```
|
4279
4666
|
|
4280
|
-
For file operation command (browse, delete), the
|
4667
|
+
For file operation command (browse, delete), the Ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
|
4281
4668
|
|
4282
|
-
For a list of SSH client options, refer to the
|
4669
|
+
For a list of SSH client options, refer to the Ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html#method-c-start).
|
4283
4670
|
|
4284
4671
|
Some of the 50 available SSH options:
|
4285
4672
|
|
@@ -4301,7 +4688,7 @@ or on Windows:
|
|
4301
4688
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
4302
4689
|
```
|
4303
4690
|
|
4304
|
-
This means that your environment
|
4691
|
+
This means that your environment suggests to use an agent but you don't have such an SSH agent running, then:
|
4305
4692
|
|
4306
4693
|
- Check env var: `SSH_AGENT_SOCK`
|
4307
4694
|
- Check your file: `$HOME/.ssh/config`
|
@@ -4315,6 +4702,9 @@ ascli server --ssh-options=@json:'{"use_agent": false}' ...
|
|
4315
4702
|
|
4316
4703
|
> **Note:** This can also be set using a preset.
|
4317
4704
|
|
4705
|
+
If one of the SSH private keys is passphrase-protected, then option `passphrase` can be used.
|
4706
|
+
It is equivalent to setting both options `ssh_options.passphrase` and `ts.ssh_private_key_passphrase`.
|
4707
|
+
|
4318
4708
|
### Other session channels for `server`
|
4319
4709
|
|
4320
4710
|
URL schemes `local` and `https` are also supported (mainly for testing purpose).
|
@@ -4337,10 +4727,18 @@ ascli server download /aspera-test-dir-large/200MB
|
|
4337
4727
|
|
4338
4728
|
`initdemo` creates a [option preset](#lprt) `demoserver` and set it as default for plugin `server`.
|
4339
4729
|
|
4730
|
+
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
|
4731
|
+
|
4732
|
+
```bash
|
4733
|
+
ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ssh_keys=_private_key_path_here_ --passphrase=_passphrase_here_
|
4734
|
+
```
|
4735
|
+
|
4340
4736
|
## <a id="node"></a>Plugin: `node`: IBM Aspera High Speed Transfer Server Node
|
4341
4737
|
|
4342
4738
|
This plugin gives access to capabilities provided by HSTS node API.
|
4343
4739
|
|
4740
|
+
> **Note:** capabilities of this plugin are used in other plugins which access to the node API, such as `aoc`.
|
4741
|
+
|
4344
4742
|
### File Operations
|
4345
4743
|
|
4346
4744
|
It is possible to:
|
@@ -4349,27 +4747,10 @@ It is possible to:
|
|
4349
4747
|
- transfer (upload / download)
|
4350
4748
|
- ...
|
4351
4749
|
|
4352
|
-
For transfers, it is possible to control how transfer is authorized using option: `token_type`:
|
4353
|
-
|
4354
|
-
- `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token, used as is.
|
4355
|
-
- `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
|
4356
|
-
- `basic` : transfer spec is created like this:
|
4357
|
-
|
4358
|
-
```json
|
4359
|
-
{
|
4360
|
-
"remote_host": "<address of node url>",
|
4361
|
-
"remote_user": "xfer",
|
4362
|
-
"ssh_port": 33001,
|
4363
|
-
"token": "Basic <base 64 encoded user/pass>",
|
4364
|
-
"direction": "[send|receive]"
|
4365
|
-
}
|
4366
|
-
```
|
4367
|
-
|
4368
|
-
> **Note:** the port is assumed to be the default Aspera SSH port `33001` and transfer user is assumed to be `xfer`.
|
4369
|
-
|
4370
4750
|
### Central
|
4371
4751
|
|
4372
|
-
The central subcommand uses the "reliable query" API (session and file).
|
4752
|
+
The central subcommand uses the "reliable query" API (session and file).
|
4753
|
+
It allows listing transfer sessions and transferred files.
|
4373
4754
|
|
4374
4755
|
Filtering can be applied:
|
4375
4756
|
|
@@ -4398,16 +4779,12 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
|
|
4398
4779
|
- Start watchd and watchfolderd services running as a system user having access to files
|
4399
4780
|
- configure a watchfolder to define automated transfers
|
4400
4781
|
|
4401
|
-
<!-- spellchecker: disable -->
|
4402
|
-
|
4403
4782
|
```bash
|
4404
4783
|
ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
4405
4784
|
ascli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
|
4406
4785
|
ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
|
4407
4786
|
```
|
4408
4787
|
|
4409
|
-
<!-- spellchecker: enable -->
|
4410
|
-
|
4411
4788
|
### Out of Transfer File Validation
|
4412
4789
|
|
4413
4790
|
Follow the Aspera Transfer Server configuration to activate this feature.
|
@@ -4417,11 +4794,11 @@ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_fil
|
|
4417
4794
|
```
|
4418
4795
|
|
4419
4796
|
```output
|
4420
|
-
|
4421
|
-
|
4422
|
-
|
4423
|
-
|
4424
|
-
|
4797
|
+
+--------------+--------------+------------+--------------------------------------+
|
4798
|
+
| session_uuid | file_id | status | path |
|
4799
|
+
+--------------+--------------+------------+--------------------------------------+
|
4800
|
+
| 1a74444c-... | 084fb181-... | validating | /home/xfer.../PKG - my title/200KB.1 |
|
4801
|
+
+--------------+--------------+------------+--------------------------------------+
|
4425
4802
|
```
|
4426
4803
|
|
4427
4804
|
```bash
|
@@ -4447,27 +4824,55 @@ ascli node download /share/sourcefile --to-folder=/destination_folder --preset=a
|
|
4447
4824
|
|
4448
4825
|
This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
|
4449
4826
|
|
4827
|
+
### node file information
|
4828
|
+
|
4829
|
+
When node api is used with an **Access key**, extra information can be retrieved, such as preview.
|
4830
|
+
|
4831
|
+
> **Note:** Display of preview on terminal requires installation of extra gem: `rmagick`
|
4832
|
+
|
4833
|
+
```bash
|
4834
|
+
dnf install -y ImageMagick-devel
|
4835
|
+
gem install rmagick rainbow
|
4836
|
+
```
|
4837
|
+
|
4838
|
+
For example it is possible to display the preview of a file, if it exists, using:
|
4839
|
+
|
4840
|
+
```bash
|
4841
|
+
ascli aoc files thumbnail /preview_samples/Aspera.mpg
|
4842
|
+
```
|
4843
|
+
|
4844
|
+
Using direct node access and an access key , one can do:
|
4845
|
+
|
4846
|
+
```bash
|
4847
|
+
ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
|
4848
|
+
```
|
4849
|
+
|
4850
|
+
> **Note:** To specify the file by its file id, use the selector syntax: `%id:_file_id_here_`
|
4851
|
+
>
|
4852
|
+
> **Note:** To force textual display of the preview on iTerm, prefix command with: `env -u TERM_PROGRAM -u LC_TERMINAL`
|
4853
|
+
|
4450
4854
|
### Create access key
|
4451
4855
|
|
4452
4856
|
```bash
|
4453
|
-
ascli node access_key create
|
4857
|
+
ascli node access_key create @json:'{"id":"myaccesskey","secret":"my_secret_here","storage":{"type":"local","path":"/data/mydir"}}'
|
4454
4858
|
```
|
4455
4859
|
|
4456
4860
|
### Node sample commands
|
4457
4861
|
|
4458
4862
|
```bash
|
4459
|
-
node access_key create
|
4460
|
-
node access_key delete
|
4863
|
+
node access_key create @json:'{"id":"testingAK1","storage":{"type":"local","path":"/"}}'
|
4864
|
+
node access_key delete testingAK1
|
4461
4865
|
node access_key do my_aoc_ak_name browse /
|
4462
4866
|
node access_key do my_aoc_ak_name delete /folder2
|
4463
4867
|
node access_key do my_aoc_ak_name delete testfile1
|
4464
4868
|
node access_key do my_aoc_ak_name download testfile1 --to-folder=.
|
4465
|
-
node access_key do my_aoc_ak_name file show --path=/testfile1
|
4466
|
-
node access_key do my_aoc_ak_name file show 1
|
4467
4869
|
node access_key do my_aoc_ak_name find /
|
4468
4870
|
node access_key do my_aoc_ak_name mkdir /folder1
|
4469
4871
|
node access_key do my_aoc_ak_name node_info /
|
4470
4872
|
node access_key do my_aoc_ak_name rename /folder1 folder2
|
4873
|
+
node access_key do my_aoc_ak_name show %id:1
|
4874
|
+
node access_key do my_aoc_ak_name show /testfile1
|
4875
|
+
node access_key do my_aoc_ak_name thumbnail /testfile1
|
4471
4876
|
node access_key do my_aoc_ak_name upload 'faux:///testfile1?1k' --default_ports=no
|
4472
4877
|
node access_key list
|
4473
4878
|
node api_details
|
@@ -4481,10 +4886,9 @@ node basic_token
|
|
4481
4886
|
node browse / -r
|
4482
4887
|
node delete /todelete
|
4483
4888
|
node delete @list:,folder_1/todelete,folder_1/tdlink,folder_1/delfile
|
4484
|
-
node delete folder_1/10MB.
|
4889
|
+
node delete folder_1/10MB.2
|
4485
4890
|
node delete testfile.bin
|
4486
4891
|
node download testfile.bin --to-folder=.
|
4487
|
-
node download testfile.bin --to-folder=. --token-type=hybrid
|
4488
4892
|
node health
|
4489
4893
|
node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
4490
4894
|
node license
|
@@ -4492,27 +4896,30 @@ node mkdir folder_1/todelete
|
|
4492
4896
|
node mkfile folder_1/delfile1 "hello world"
|
4493
4897
|
node mklink folder_1/todelete folder_1/tdlink
|
4494
4898
|
node rename folder_1 delfile1 delfile
|
4495
|
-
node search / --
|
4899
|
+
node search / --query=@json:'{"sort":"mtime"}'
|
4496
4900
|
node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
|
4497
4901
|
node service delete service1
|
4498
4902
|
node service list
|
4499
4903
|
node space /
|
4500
|
-
node
|
4501
|
-
node
|
4502
|
-
node
|
4503
|
-
node
|
4504
|
-
node
|
4505
|
-
node
|
4506
|
-
node
|
4507
|
-
node
|
4508
|
-
node
|
4509
|
-
node
|
4510
|
-
node
|
4511
|
-
node
|
4512
|
-
node
|
4513
|
-
node
|
4904
|
+
node ssync bandwidth my_syncid
|
4905
|
+
node ssync counters my_syncid
|
4906
|
+
node ssync create @json:'{"configuration":{"name":"sync1","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password","path":"my_remote_path"}}}'
|
4907
|
+
node ssync delete my_syncid
|
4908
|
+
node ssync files my_syncid
|
4909
|
+
node ssync list
|
4910
|
+
node ssync show my_syncid
|
4911
|
+
node ssync start my_syncid
|
4912
|
+
node ssync state my_syncid
|
4913
|
+
node ssync stop my_syncid
|
4914
|
+
node ssync summary my_syncid
|
4915
|
+
node sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
4916
|
+
node sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
4917
|
+
node sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
4918
|
+
node sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
4919
|
+
node transfer list --query=@json:'{"active_only":true}'
|
4920
|
+
node upload --to-folder=folder_1 --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass_here"}'
|
4921
|
+
node upload --username=my_aoc_ak_name --password=my_aoc_ak_secret testfile.bin
|
4514
4922
|
node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
|
4515
|
-
node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
|
4516
4923
|
```
|
4517
4924
|
|
4518
4925
|
## <a id="faspex5"></a>Plugin: `faspex5`: IBM Aspera Faspex v5
|
@@ -4521,22 +4928,37 @@ IBM Aspera's newer self-managed application.
|
|
4521
4928
|
|
4522
4929
|
3 authentication methods are supported:
|
4523
4930
|
|
4524
|
-
- jwt
|
4525
|
-
-
|
4526
|
-
-
|
4931
|
+
- jwt : general purpose, private-key based authentication
|
4932
|
+
- link : public link authentication
|
4933
|
+
- web : requires authentication with web browser
|
4934
|
+
- boot : use authentication token copied from browser (experimental)
|
4527
4935
|
|
4528
4936
|
### Faspex 5 JWT authentication
|
4529
4937
|
|
4530
|
-
This is the **recommended** method to use.
|
4938
|
+
This is the general purpose and **recommended** method to use.
|
4531
4939
|
|
4532
|
-
|
4940
|
+
Activation is in two steps:
|
4533
4941
|
|
4534
|
-
-
|
4535
|
-
|
4536
|
-
|
4537
|
-
|
4538
|
-
-
|
4539
|
-
-
|
4942
|
+
- The admninistrator must create an API client in Faspex with JWT support
|
4943
|
+
|
4944
|
+
This operation is generally done only once:
|
4945
|
+
|
4946
|
+
- As Admin, Navigate to the web UI: Admin → Configurations → API Clients → Create
|
4947
|
+
- Give a name, like `ascli`
|
4948
|
+
- Activate JWT
|
4949
|
+
- There is an option to set a general public key allowing the owner of the private key to impersonate any user. Unless you want to do this, leave this field empty.
|
4950
|
+
- Click on `Create` Button
|
4951
|
+
- Take note of Client Id (and Client Secret, but not used in current version)
|
4952
|
+
|
4953
|
+
- The user uses a private key and sets the public key in his faspex 5 profile
|
4954
|
+
|
4955
|
+
This operation is done by each user using the CLI.
|
4956
|
+
|
4957
|
+
- As user, click on the user logo, left to the app switcher on top right.
|
4958
|
+
- Select `Account Settings`
|
4959
|
+
- on the bottom in the text field: `Public key in PEM format` paste the **public** key corresponding to the private key used by the user.
|
4960
|
+
|
4961
|
+
**Note:** If you don't have any refer to section [Private Key](#private_key)
|
4540
4962
|
|
4541
4963
|
Then use these options:
|
4542
4964
|
|
@@ -4548,19 +4970,31 @@ Then use these options:
|
|
4548
4970
|
--private-key=@file:.../path/to/key.pem
|
4549
4971
|
```
|
4550
4972
|
|
4551
|
-
> **Note:** The `private_key` option must contain the PEM value of the private key which can be read from a file using the modifier: `@file:`, e.g. `@file:/path/to/key.pem`.
|
4973
|
+
> **Note:** The `private_key` option must contain the PEM **value** (not file path) of the private key which can be read from a file using the modifier: `@file:`, e.g. `@file:/path/to/key.pem`.
|
4974
|
+
|
4975
|
+
As usual, typically a user will create preset to avoid having to type these options each time.
|
4976
|
+
|
4977
|
+
Example:
|
4978
|
+
|
4979
|
+
```bash
|
4980
|
+
ascli conf preset update myf5 --auth=jwt --client-id=_client_id_here_ --client-secret=my_secret_here --username=_username_here_ --private-key=@file:.../path/to/key.pem
|
4981
|
+
|
4982
|
+
ascli conf preset set default faspx5 myf5
|
4983
|
+
|
4984
|
+
ascli faspex5 user profile show
|
4985
|
+
```
|
4552
4986
|
|
4553
4987
|
### Faspex 5 web authentication
|
4554
4988
|
|
4555
|
-
|
4989
|
+
The admninistrator must create an API client in Faspex for an external web app support:
|
4556
4990
|
|
4557
|
-
- Navigate to the web UI: Admin → Configurations → API Clients → Create
|
4991
|
+
- As Admin, Navigate to the web UI: Admin → Configurations → API Clients → Create
|
4558
4992
|
- Do not Activate JWT
|
4559
4993
|
- Set **Redirect URI** to `https://127.0.0.1:8888`
|
4560
|
-
- Click on Create Button
|
4994
|
+
- Click on `Create` Button
|
4561
4995
|
- Take note of Client Id (and Client Secret, but not used in current version)
|
4562
4996
|
|
4563
|
-
|
4997
|
+
The user will use the following options:
|
4564
4998
|
|
4565
4999
|
```text
|
4566
5000
|
--auth=web
|
@@ -4573,7 +5007,7 @@ Then use options:
|
|
4573
5007
|
|
4574
5008
|
For `boot` method: (will be removed in future)
|
4575
5009
|
|
4576
|
-
- Open a Web Browser
|
5010
|
+
- As user: Open a Web Browser
|
4577
5011
|
- Start developer mode
|
4578
5012
|
- Login to Faspex 5
|
4579
5013
|
- Find the first API call with `Authorization` header, and copy the value of the token (series of base64 values with dots)
|
@@ -4581,23 +5015,14 @@ For `boot` method: (will be removed in future)
|
|
4581
5015
|
Use this token as password and use `--auth=boot`.
|
4582
5016
|
|
4583
5017
|
```bash
|
4584
|
-
ascli conf
|
5018
|
+
ascli conf preset update f5boot --url=https://localhost/aspera/faspex --auth=boot --password=_token_here_
|
4585
5019
|
```
|
4586
5020
|
|
4587
|
-
### Faspex 5 packages
|
4588
|
-
|
4589
|
-
The `value` option provided to command `faspex5 package send` is the same as for the Faspex 5 API: `POST /packages`.
|
4590
|
-
|
4591
|
-
In addition, `ascli` adds some convenience: the field `recipients` is normally an Array of Hash, each with field `name` and optionally `recipient_type`, but it is also possible to provide an Array of String, with simply a recipient name.
|
4592
|
-
Then `ascli` will lookup existing contacts, and if a single match is found will use it, and set the `name` and `recipient_type` accordingly.
|
4593
|
-
|
4594
|
-
> **Note:** The lookup is case insensitive and on partial matches.
|
4595
|
-
|
4596
5021
|
### Faspex 5 sample commands
|
4597
5022
|
|
4598
5023
|
Most commands are directly REST API calls.
|
4599
|
-
Parameters to commands are carried through option `
|
4600
|
-
|
5024
|
+
Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional parameter for creation.
|
5025
|
+
One can conveniently use the JSON format with prefix `@json:`.
|
4601
5026
|
|
4602
5027
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
4603
5028
|
|
@@ -4606,70 +5031,200 @@ faspex5 admin res accounts list
|
|
4606
5031
|
faspex5 admin res contacts list
|
4607
5032
|
faspex5 admin res jobs list
|
4608
5033
|
faspex5 admin res metadata_profiles list
|
4609
|
-
faspex5 admin res node list
|
5034
|
+
faspex5 admin res node list
|
4610
5035
|
faspex5 admin res oauth_clients list
|
4611
5036
|
faspex5 admin res registrations list
|
4612
5037
|
faspex5 admin res saml_configs list
|
5038
|
+
faspex5 admin res shared_inboxes invite %name:'ascli shinbox' johnny@example.com
|
4613
5039
|
faspex5 admin res shared_inboxes list
|
5040
|
+
faspex5 admin res shared_inboxes members %name:'ascli shinbox' create %name:john@example.com
|
5041
|
+
faspex5 admin res shared_inboxes members %name:'ascli shinbox' delete %name:john@example.com
|
5042
|
+
faspex5 admin res shared_inboxes members %name:'ascli shinbox' delete %name:johnny@example.com
|
5043
|
+
faspex5 admin res shared_inboxes members %name:'ascli shinbox' list
|
4614
5044
|
faspex5 admin res workgroups list
|
5045
|
+
faspex5 admin smtp show
|
5046
|
+
faspex5 admin smtp test my_email_external
|
4615
5047
|
faspex5 bearer_token
|
4616
|
-
faspex5 gateway
|
5048
|
+
faspex5 gateway https://localhost:12345/aspera/faspex
|
4617
5049
|
faspex5 health
|
4618
|
-
faspex5
|
4619
|
-
faspex5
|
4620
|
-
faspex5
|
4621
|
-
faspex5
|
4622
|
-
faspex5
|
4623
|
-
faspex5
|
5050
|
+
faspex5 packages list --box=my_faspex5_shinbox
|
5051
|
+
faspex5 packages list --box=my_faspex5_workgroup --group-type=workgroups
|
5052
|
+
faspex5 packages list --query=@json:'{"mailbox":"inbox","state":["released"]}'
|
5053
|
+
faspex5 packages receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
|
5054
|
+
faspex5 packages receive --box=my_faspex5_shinbox "my_package_id" --to-folder=.
|
5055
|
+
faspex5 packages receive --box=my_faspex5_workgroup --group-type=workgroups "my_package_id" --to-folder=.
|
5056
|
+
faspex5 packages receive ALL --once-only=yes --to-folder=.
|
5057
|
+
faspex5 packages receive INIT --once-only=yes
|
5058
|
+
faspex5 packages send @json:'{"title":"test title","recipients":["my_shinbox"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' testfile.bin
|
5059
|
+
faspex5 packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' testfile.bin
|
5060
|
+
faspex5 packages send @json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
|
5061
|
+
faspex5 packages show "my_package_id"
|
5062
|
+
faspex5 packages show --box=my_faspex5_shinbox "my_package_id"
|
5063
|
+
faspex5 packages show --box=my_faspex5_workgroup --group-type=workgroups "my_package_id"
|
5064
|
+
faspex5 postprocessing @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":"tests"},"certificate":{"key":"../local/k","cert":"../local/c","chain":"../local/ch"}}'
|
4624
5065
|
faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
|
4625
5066
|
faspex5 user profile show
|
4626
5067
|
```
|
4627
5068
|
|
4628
|
-
|
5069
|
+
### Faspex 5: inbox selection
|
5070
|
+
|
5071
|
+
By default, package operations (send, receive, list) are done on the user's inbox.
|
5072
|
+
|
5073
|
+
To select another inbox, use option `box` with one of the following values:
|
5074
|
+
|
5075
|
+
- inbox
|
5076
|
+
- inbox_history
|
5077
|
+
- inbox_all
|
5078
|
+
- inbox_all_history
|
5079
|
+
- outbox
|
5080
|
+
- outbox_history
|
5081
|
+
- pending
|
5082
|
+
- pending_history
|
5083
|
+
- all
|
5084
|
+
- ALL (only admin)
|
5085
|
+
- name of a shared inbox or workgroup
|
5086
|
+
|
5087
|
+
> **Note:** specify if the box is a shared inbox or a workgroup using option `group_type` with either `shared_inboxes` or `workgroups`
|
5088
|
+
|
5089
|
+
### Faspex 5: Send a package
|
4629
5090
|
|
4630
|
-
|
5091
|
+
The `Hash` creation parameter provided to command `faspex5 packages send` corresponds to the Faspex 5 API: `POST /packages`.
|
5092
|
+
|
5093
|
+
Required fields are `title` and `recipients`.
|
5094
|
+
Example using `@json:` format:
|
5095
|
+
|
5096
|
+
```json
|
5097
|
+
{"title":"some title","recipients":[{"recipient_type":"user","name":"user@example.com"}]}
|
5098
|
+
```
|
5099
|
+
|
5100
|
+
`recipient_type` is one of (Refer to API):
|
5101
|
+
|
5102
|
+
- user
|
5103
|
+
- workgroup
|
5104
|
+
- external_user
|
5105
|
+
- distribution_list
|
5106
|
+
- shared_inbox
|
5107
|
+
|
5108
|
+
`ascli` adds some convenience: The API expects the field `recipients` to be an `Array` of `Hash`, each with field `name` and optionally `recipient_type`.
|
5109
|
+
It is also possible to provide an `Array` of `String`, with simply a recipient name.
|
5110
|
+
Then `ascli` will lookup existing contacts among all possible types, use it if a single match is found, and set the `name` and `recipient_type` accordingly.
|
5111
|
+
Else an exception is sent.
|
5112
|
+
|
5113
|
+
> **Note:** The lookup is case insensitive and on partial matches.
|
5114
|
+
|
5115
|
+
```json
|
5116
|
+
{"title":"some title","recipients":["user@example.com"]}
|
5117
|
+
```
|
5118
|
+
|
5119
|
+
If the lookup needs to be only on certain types, you can specify the field: `recipient_types` with either a single value or an Array of values (from the list above). e.g. :
|
5120
|
+
|
5121
|
+
```json
|
5122
|
+
{"title":"test title","recipient_types":"user","recipients":["user1@example.com","user2@example.com"]}
|
5123
|
+
```
|
5124
|
+
|
5125
|
+
### Faspex 5: Send a package with metadata
|
4631
5126
|
|
4632
5127
|
The interface is the one of the API (Refer to API documentation, or look at request in browser):
|
4633
5128
|
|
4634
5129
|
```bash
|
4635
|
-
ascli faspex5
|
5130
|
+
ascli faspex5 packages send @json:'{"title":"test title","recipients":["my shared inbox"],"metadata":{"Confidential":"Yes","Drop menu":"Option 1"}}' 'faux:///test1?k1'
|
4636
5131
|
```
|
4637
5132
|
|
4638
5133
|
Basically, add the field `metadata`, with one key per metadata and the value is directly the metadata value.
|
4639
5134
|
|
4640
|
-
|
5135
|
+
### Faspex 5: Receive a package
|
5136
|
+
|
5137
|
+
The (numeric) identifier of the package t receive is given as argument to command `faspex5 packages receive`.
|
5138
|
+
|
5139
|
+
> **Note:** option `box` applies.
|
5140
|
+
|
5141
|
+
### Faspex 5: List packages
|
5142
|
+
|
5143
|
+
The following parameters in option `query` are supported:
|
5144
|
+
|
5145
|
+
- `q` : a filter on name (case insensitive, matches if value is contained in name)
|
5146
|
+
- `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
5147
|
+
- `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
5148
|
+
- `offset` : native api parameter, in general do not use (added by `ascli`)
|
5149
|
+
- `limit` : native api parameter, number of items par api call, in general do not use (added by `ascli`)
|
5150
|
+
|
5151
|
+
Admin only: If the value `ALL` is provided to option `box`, then all packages are selected.
|
5152
|
+
|
5153
|
+
### Faspex 5: List all shared inboxes
|
5154
|
+
|
5155
|
+
```bash
|
5156
|
+
ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
|
5157
|
+
```
|
5158
|
+
|
5159
|
+
Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
|
4641
5160
|
|
4642
5161
|
```bash
|
4643
|
-
ascli faspex5 admin res
|
5162
|
+
ascli faspex5 admin res shared_inboxes invite '%name:ascli shinbox' john@example.com
|
5163
|
+
```
|
5164
|
+
|
5165
|
+
It is equivalent to:
|
5166
|
+
|
5167
|
+
```bash
|
5168
|
+
ascli faspex5 admin res shared_inboxes invite '%name:ascli shinbox' @json:'{"email_address":"john@example.com"}'
|
5169
|
+
```
|
5170
|
+
|
5171
|
+
Other payload parameters are possible in Hash format:
|
5172
|
+
|
5173
|
+
```json
|
5174
|
+
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
|
4644
5175
|
```
|
4645
5176
|
|
4646
|
-
|
5177
|
+
### Faspex 5: Create Metadata profile
|
4647
5178
|
|
4648
5179
|
```bash
|
4649
|
-
ascli faspex5 admin res metadata_profiles create
|
5180
|
+
ascli faspex5 admin res metadata_profiles create @json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
|
4650
5181
|
```
|
4651
5182
|
|
4652
|
-
|
5183
|
+
### Faspex 5: Create a Shared inbox with specific metadata profile
|
4653
5184
|
|
4654
5185
|
```bash
|
4655
|
-
ascli faspex5 admin res shared create
|
5186
|
+
ascli faspex5 admin res shared create @json:'{"name":"the shared inbox","metadata_profile_id":1}'
|
4656
5187
|
```
|
4657
5188
|
|
4658
|
-
|
5189
|
+
### Faspex 5: List content in Shared folder and send package from remote source
|
4659
5190
|
|
4660
5191
|
```bash
|
4661
5192
|
ascli faspex5 shared_folders list
|
4662
5193
|
```
|
4663
5194
|
|
5195
|
+
```markdown
|
5196
|
+
+----+----------+---------+-----+
|
5197
|
+
| id | name | node_id | ... |
|
5198
|
+
+----+----------+---------+-----+
|
5199
|
+
| 3 | partages | 2 | ... |
|
5200
|
+
+----+----------+---------+-----+
|
5201
|
+
```
|
5202
|
+
|
4664
5203
|
```bash
|
4665
|
-
ascli faspex5 shared_folders br
|
5204
|
+
ascli faspex5 shared_folders br %name:partages /folder
|
4666
5205
|
```
|
4667
5206
|
|
4668
5207
|
```bash
|
4669
|
-
ascli faspex5
|
5208
|
+
ascli faspex5 packages send @json:'{"title":"hello","recipients":[{"name":"_recipient_here_"}]}' --shared-folder=%name:partages /folder/file
|
4670
5209
|
```
|
4671
5210
|
|
4672
|
-
|
5211
|
+
> **Note:** The shared folder can be identified by its numerical `id` or by name using percent selector: `%<field>:<value>`. e.g. `--shared-folder=3`
|
5212
|
+
|
5213
|
+
### Faspex 5: receive all packages (cargo)
|
5214
|
+
|
5215
|
+
To receive all packages, only once, through persistency of already received packages:
|
5216
|
+
|
5217
|
+
```bash
|
5218
|
+
ascli faspex5 packages receive ALL --once-only=yes
|
5219
|
+
```
|
5220
|
+
|
5221
|
+
To initialize, and skip all current package so that next time `ALL` is used, only newer packages are downloaded:
|
5222
|
+
|
5223
|
+
```bash
|
5224
|
+
ascli faspex5 packages receive INIT --once-only=yes
|
5225
|
+
```
|
5226
|
+
|
5227
|
+
### Faspex 5: Faspex 4-style postprocessing
|
4673
5228
|
|
4674
5229
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
4675
5230
|
It implements Faspex 5 web hooks, and calls a local script with the same environment as Faspex 4.
|
@@ -4677,19 +5232,19 @@ It implements Faspex 5 web hooks, and calls a local script with the same environ
|
|
4677
5232
|
It is invoked like this:
|
4678
5233
|
|
4679
5234
|
```bash
|
4680
|
-
ascli faspex5 postprocessing
|
5235
|
+
ascli faspex5 postprocessing @json:'{"url":"http://localhost:8080/processing"}'
|
4681
5236
|
```
|
4682
5237
|
|
4683
5238
|
The following parameters are supported:
|
4684
5239
|
|
4685
5240
|
| parameter | type | default | description |
|
4686
5241
|
|----------------------------|---------|------------------------|-----------------------------------------------------|
|
4687
|
-
| url | string | http://localhost:8080 | Defines the base url on which requests are listened |
|
5242
|
+
| url | string | <http://localhost:8080> | Defines the base url on which requests are listened |
|
4688
5243
|
| certificate | hash | nil | used to define certificate if https is used |
|
4689
5244
|
| certificate.key | string | nil | path to private key file |
|
4690
5245
|
| certificate.cert | string | nil | path to certificate |
|
4691
5246
|
| certificate.chain | string | nil | path to intermediary certificates |
|
4692
|
-
| processing | hash | nil |
|
5247
|
+
| processing | hash | nil | behavior of post processing |
|
4693
5248
|
| processing.script_folder | string | . | prefix added to script path |
|
4694
5249
|
| processing.fail_on_error | bool | false | if true and process exit with non zero, then fail |
|
4695
5250
|
| processing.timeout_seconds | integer | 60 | processing script is killed if takes more time |
|
@@ -4704,7 +5259,7 @@ When a request is received the following happens:
|
|
4704
5259
|
|
4705
5260
|
- the processor get the path of the url called
|
4706
5261
|
- it removes the "domain
|
4707
|
-
- it
|
5262
|
+
- it prepends it with the value of `script_folder`
|
4708
5263
|
- it executes the script
|
4709
5264
|
- upon success, a success code is returned
|
4710
5265
|
|
@@ -4752,7 +5307,8 @@ As inboxes may be large, it is possible to use the following query parameters:
|
|
4752
5307
|
|
4753
5308
|
The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "Services (API v.3)".
|
4754
5309
|
|
4755
|
-
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
5310
|
+
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
5311
|
+
By default `count` is `0` (`10`), it can be increased to issue less HTTP calls.
|
4756
5312
|
|
4757
5313
|
#### Example: list packages in dropbox
|
4758
5314
|
|
@@ -4771,7 +5327,7 @@ The command is `package recv`, possible methods are:
|
|
4771
5327
|
- provide a `faspe:` URI with option `link`
|
4772
5328
|
|
4773
5329
|
```bash
|
4774
|
-
ascli faspex package recv
|
5330
|
+
ascli faspex package recv 12345
|
4775
5331
|
ascli faspex package recv --link=faspe://...
|
4776
5332
|
```
|
4777
5333
|
|
@@ -4820,9 +5376,9 @@ In this example the notification template is directly provided on command line.
|
|
4820
5376
|
Example:
|
4821
5377
|
|
4822
5378
|
```bash
|
4823
|
-
ascli faspex v4 dropbox create
|
5379
|
+
ascli faspex v4 dropbox create @json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
|
4824
5380
|
ascli faspex v4 dropbox list
|
4825
|
-
ascli faspex v4 dropbox delete
|
5381
|
+
ascli faspex v4 dropbox delete 36
|
4826
5382
|
```
|
4827
5383
|
|
4828
5384
|
### Remote sources
|
@@ -4860,7 +5416,7 @@ It is possible to tell `ascli` to download newly received packages, much like th
|
|
4860
5416
|
cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on Cloud plugin:
|
4861
5417
|
|
4862
5418
|
```bash
|
4863
|
-
ascli faspex packages recv
|
5419
|
+
ascli faspex packages recv ALL --once-only=yes --lock-port=12345
|
4864
5420
|
```
|
4865
5421
|
|
4866
5422
|
### Faspex 4 sample commands
|
@@ -4879,18 +5435,18 @@ faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_i
|
|
4879
5435
|
faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
4880
5436
|
faspex package recv "my_package_id" --to-folder=.
|
4881
5437
|
faspex package recv "my_package_id" --to-folder=. --box=sent
|
4882
|
-
faspex package recv --to-folder=. --link=
|
5438
|
+
faspex package recv --to-folder=. --link=https://app.example.com/recv_from_user_path
|
4883
5439
|
faspex package recv ALL --to-folder=. --once-only=yes
|
4884
5440
|
faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
|
4885
5441
|
faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
|
4886
5442
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
|
4887
5443
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
|
4888
5444
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
|
4889
|
-
faspex package send --link=
|
4890
|
-
faspex package send --link=
|
5445
|
+
faspex package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
5446
|
+
faspex package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
4891
5447
|
faspex source list
|
4892
|
-
faspex source name
|
4893
|
-
faspex source name
|
5448
|
+
faspex source name my_faspex_src info
|
5449
|
+
faspex source name my_faspex_src node br /
|
4894
5450
|
faspex v4 dmembership list
|
4895
5451
|
faspex v4 dropbox list
|
4896
5452
|
faspex v4 metadata_profile list
|
@@ -4901,7 +5457,7 @@ faspex v4 workgroup list
|
|
4901
5457
|
|
4902
5458
|
## <a id="shares"></a>Plugin: `shares`: IBM Aspera Shares v1
|
4903
5459
|
|
4904
|
-
Aspera Shares supports the "node API" for the file transfer part.
|
5460
|
+
Aspera Shares supports the "node API" for the file transfer part.
|
4905
5461
|
|
4906
5462
|
### Shares 1 sample commands
|
4907
5463
|
|
@@ -4910,20 +5466,20 @@ shares admin group list
|
|
4910
5466
|
shares admin node list
|
4911
5467
|
shares admin share list --fields=-status,status_message
|
4912
5468
|
shares admin share user_permissions 1 list
|
4913
|
-
shares admin user add --type=ldap
|
4914
|
-
shares admin user app_authorizations 1 modify
|
5469
|
+
shares admin user add --type=ldap the_name
|
5470
|
+
shares admin user app_authorizations 1 modify @json:'{"app_login":true}'
|
4915
5471
|
shares admin user app_authorizations 1 show
|
4916
|
-
shares admin user import --type=saml
|
5472
|
+
shares admin user import --type=saml @json:'{"id":"the_id","name_id":"the_name"}'
|
4917
5473
|
shares admin user list
|
4918
5474
|
shares admin user share_permissions 1 list
|
4919
5475
|
shares admin user share_permissions 1 show 1
|
5476
|
+
shares files browse /
|
5477
|
+
shares files delete my_shares_upload/testfile.bin
|
5478
|
+
shares files download --to-folder=. my_shares_upload/testfile.bin
|
5479
|
+
shares files download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
|
5480
|
+
shares files upload --to-folder=my_shares_upload testfile.bin
|
5481
|
+
shares files upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
|
4920
5482
|
shares health
|
4921
|
-
shares repository browse /
|
4922
|
-
shares repository delete my_shares_upload/testfile.bin
|
4923
|
-
shares repository download --to-folder=. my_shares_upload/testfile.bin
|
4924
|
-
shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
|
4925
|
-
shares repository upload --to-folder=my_shares_upload testfile.bin
|
4926
|
-
shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy"}'
|
4927
5483
|
```
|
4928
5484
|
|
4929
5485
|
## <a id="console"></a>Plugin: `console`: IBM Aspera Console
|
@@ -4978,8 +5534,8 @@ If you have those parameters already, then following options shall be provided:
|
|
4978
5534
|
For example, let us create a default configuration:
|
4979
5535
|
|
4980
5536
|
```bash
|
4981
|
-
ascli conf
|
4982
|
-
ascli conf
|
5537
|
+
ascli conf preset update mycos --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
|
5538
|
+
ascli conf preset set default cos mycos
|
4983
5539
|
```
|
4984
5540
|
|
4985
5541
|
Then, jump to the transfer example.
|
@@ -5011,8 +5567,6 @@ ibmcloud resource service-key _service_key_name_here_ --output JSON|jq '.[0].cre
|
|
5011
5567
|
|
5012
5568
|
It consists in the following structure:
|
5013
5569
|
|
5014
|
-
<!-- spellchecker: disable -->
|
5015
|
-
|
5016
5570
|
```json
|
5017
5571
|
{
|
5018
5572
|
"apikey": "my_api_key_here",
|
@@ -5029,8 +5583,6 @@ It consists in the following structure:
|
|
5029
5583
|
}
|
5030
5584
|
```
|
5031
5585
|
|
5032
|
-
<!-- spellchecker: enable -->
|
5033
|
-
|
5034
5586
|
The field `resource_instance_id` is for option `crn`
|
5035
5587
|
|
5036
5588
|
The field `apikey` is for option `apikey`
|
@@ -5046,8 +5598,8 @@ The required options for this method are:
|
|
5046
5598
|
For example, let us create a default configuration:
|
5047
5599
|
|
5048
5600
|
```bash
|
5049
|
-
ascli conf
|
5050
|
-
ascli conf
|
5601
|
+
ascli conf preset update mycos --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
|
5602
|
+
ascli conf preset set default cos mycos
|
5051
5603
|
```
|
5052
5604
|
|
5053
5605
|
### Operations, transfers
|
@@ -5076,20 +5628,30 @@ cos node upload testfile.bin
|
|
5076
5628
|
|
5077
5629
|
## <a id="async"></a>Plugin: `async`: IBM Aspera Sync
|
5078
5630
|
|
5079
|
-
A basic plugin to start an
|
5080
|
-
The main advantage over bare `async` command line is the possibility to use a configuration file, using
|
5631
|
+
A basic plugin to start an `async` using `ascli`.
|
5632
|
+
The main advantage over bare `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
5081
5633
|
|
5082
|
-
|
5083
|
-
In this case, some of the `sync` parameters are
|
5634
|
+
The `sync` command is also made available through the `server sync`, `aoc files sync` and `node sync` commands.
|
5635
|
+
In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (including token).
|
5084
5636
|
|
5085
5637
|
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
|
5086
|
-
|
5087
|
-
|
5088
|
-
|
5089
|
-
|
5638
|
+
|
5639
|
+
Two JSON syntax are supported for option `sync_info`.
|
5640
|
+
|
5641
|
+
### async native JSON
|
5642
|
+
|
5643
|
+
It is the same payload as specified on the `async` option `--conf` or in the latest node API.
|
5644
|
+
This is the preferred syntax and allows a single session definition.
|
5645
|
+
But there is no progress output nor error messages.
|
5090
5646
|
|
5091
5647
|
Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
|
5092
5648
|
|
5649
|
+
### async options as JSON
|
5650
|
+
|
5651
|
+
This is specific to `ascli`.
|
5652
|
+
It is based on a JSON representation of `async` command line options.
|
5653
|
+
It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
|
5654
|
+
|
5093
5655
|
### Sync sample commands
|
5094
5656
|
|
5095
5657
|
```bash
|
@@ -5165,7 +5727,7 @@ ascli preview check
|
|
5165
5727
|
#### Image: ImageMagick and optipng
|
5166
5728
|
|
5167
5729
|
```bash
|
5168
|
-
|
5730
|
+
dnf install -y ImageMagick optipng
|
5169
5731
|
```
|
5170
5732
|
|
5171
5733
|
You may also install `ghostscript` which adds fonts to ImageMagick.
|
@@ -5194,7 +5756,6 @@ dnf install unoconv
|
|
5194
5756
|
|
5195
5757
|
- Amazon Linux
|
5196
5758
|
|
5197
|
-
<!-- spellchecker: disable -->
|
5198
5759
|
```bash
|
5199
5760
|
amazon-linux-extras enable libreoffice
|
5200
5761
|
yum clean metadata
|
@@ -5203,15 +5764,17 @@ wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
|
|
5203
5764
|
mv unoconv /usr/bin
|
5204
5765
|
chmod a+x /usr/bin/unoconv
|
5205
5766
|
```
|
5206
|
-
<!-- spellchecker: enable -->
|
5207
5767
|
|
5208
5768
|
### Configuration
|
5209
5769
|
|
5210
|
-
The preview generator
|
5770
|
+
The preview generator should be executed as a non-user.
|
5771
|
+
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
5211
5772
|
|
5212
|
-
Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt).
|
5773
|
+
Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt).
|
5774
|
+
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
5213
5775
|
|
5214
|
-
The `xfer` user has a special protected shell: `aspshell`, so changing identity
|
5776
|
+
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
5777
|
+
E.g.:
|
5215
5778
|
|
5216
5779
|
```bash
|
5217
5780
|
su -s /bin/bash - xfer
|
@@ -5232,11 +5795,17 @@ ascli -Ppreviewconf node browse /
|
|
5232
5795
|
|
5233
5796
|
This shall list the contents of the storage root of the access key.
|
5234
5797
|
|
5798
|
+
### Options for generated files
|
5799
|
+
|
5800
|
+
When generating preview files, some options are provided by default.
|
5801
|
+
Some values for the options can be modified on command line.
|
5802
|
+
For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a Hash with two keys: `in` and `out`, each is an array of strings with the native options to `ffmpeg`.
|
5803
|
+
|
5235
5804
|
### Execution
|
5236
5805
|
|
5237
5806
|
The tool intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
5238
5807
|
It needs to be run on a regular basis to create or update preview files.
|
5239
|
-
For that use your best reliable scheduler, see [
|
5808
|
+
For that use your best reliable scheduler, see [Scheduler](#scheduler).
|
5240
5809
|
|
5241
5810
|
Typically, for **Access key** access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
|
5242
5811
|
|
@@ -5254,37 +5823,16 @@ On subsequent run it reads this file and check that previews are generated for t
|
|
5254
5823
|
|
5255
5824
|
### Configuration for Execution in scheduler
|
5256
5825
|
|
5257
|
-
|
5258
|
-
Adapt the scripts to your own needs.
|
5259
|
-
|
5260
|
-
We assume here that a configuration preset was created as shown previously.
|
5826
|
+
Details are provided in section [Scheduler](#scheduler).
|
5261
5827
|
|
5262
|
-
|
5828
|
+
Shorter commands can be specified if a configuration preset was created as shown previously.
|
5263
5829
|
|
5264
|
-
|
5830
|
+
For example the timeout value can be differentiated depending on the option: event versus scan:
|
5265
5831
|
|
5266
|
-
<!-- spellchecker: disable -->
|
5267
5832
|
```bash
|
5268
|
-
#!/bin/bash
|
5269
|
-
# set a timeout protection, just in case
|
5270
5833
|
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
5271
|
-
. /etc/profile.d/rvm.sh
|
5272
|
-
rvm use 2.6 --quiet
|
5273
|
-
exec timeout ${tmout} ascli "${@}"
|
5274
|
-
```
|
5275
|
-
<!-- spellchecker: enable -->
|
5276
|
-
|
5277
|
-
Here the cronjob is created for user `xfer`.
|
5278
|
-
|
5279
|
-
```bash
|
5280
|
-
crontab<<EOF
|
5281
|
-
0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
|
5282
|
-
2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
|
5283
|
-
EOF
|
5284
5834
|
```
|
5285
5835
|
|
5286
|
-
> **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
|
5287
|
-
|
5288
5836
|
### Candidate detection for creation or update (or deletion)
|
5289
5837
|
|
5290
5838
|
The tool generates preview files using those commands:
|
@@ -5313,20 +5861,20 @@ ascli preview scan --skip-folders=@json:'["/not_here"]'
|
|
5313
5861
|
|
5314
5862
|
The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
|
5315
5863
|
|
5316
|
-
When scanning the option `
|
5864
|
+
When scanning the option `query` has the same behavior as for the `node find` command.
|
5317
5865
|
|
5318
5866
|
For instance to filter out files beginning with `._` do:
|
5319
5867
|
|
5320
5868
|
```bash
|
5321
|
-
|
5869
|
+
--query='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
|
5322
5870
|
```
|
5323
5871
|
|
5324
5872
|
### Preview File types
|
5325
5873
|
|
5326
5874
|
Two types of preview can be generated:
|
5327
5875
|
|
5328
|
-
- png
|
5329
|
-
- mp4
|
5876
|
+
- `png`: thumbnail
|
5877
|
+
- `mp4`: video preview (only for video)
|
5330
5878
|
|
5331
5879
|
Use option `skip_format` to skip generation of a format.
|
5332
5880
|
|
@@ -5379,10 +5927,10 @@ If the `mimemagic` gem complains about missing mime info file:
|
|
5379
5927
|
|
5380
5928
|
- Close the `cmd` and restart a new one if needed to get refreshed env vars
|
5381
5929
|
|
5382
|
-
- Linux
|
5930
|
+
- Linux RHEL 8+:
|
5383
5931
|
|
5384
5932
|
```bash
|
5385
|
-
|
5933
|
+
dnf install shared-mime-info
|
5386
5934
|
```
|
5387
5935
|
|
5388
5936
|
- macOS:
|
@@ -5391,7 +5939,7 @@ yum install shared-mime-info
|
|
5391
5939
|
brew install shared-mime-info
|
5392
5940
|
```
|
5393
5941
|
|
5394
|
-
###
|
5942
|
+
### Generation: Read source files and write preview
|
5395
5943
|
|
5396
5944
|
Standard open source tools are used to create thumbnails and video previews.
|
5397
5945
|
Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
|
@@ -5406,7 +5954,7 @@ If the preview generator does not have access to files on the file system (it is
|
|
5406
5954
|
|
5407
5955
|
```bash
|
5408
5956
|
preview check --skip-types=office
|
5409
|
-
preview
|
5957
|
+
preview scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
5410
5958
|
preview scan --skip-types=office --log-level=info
|
5411
5959
|
preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
|
5412
5960
|
preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
|
@@ -5421,20 +5969,23 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
|
|
5421
5969
|
|
5422
5970
|
## SMTP for email notifications
|
5423
5971
|
|
5424
|
-
|
5972
|
+
`ascli` can send email, for that setup SMTP configuration. This is done with option `smtp`.
|
5425
5973
|
|
5426
5974
|
The `smtp` option is a hash table (extended value) with the following fields:
|
5427
5975
|
|
5976
|
+
<!-- markdownlint-disable MD034 -->
|
5428
5977
|
| field | default | example | description |
|
5429
5978
|
|--------------|---------------------|----------------------------|----------------------------------|
|
5430
5979
|
| `server` | - | smtp.gmail.com | SMTP server address |
|
5431
|
-
| `tls` | true |
|
5432
|
-
| `
|
5980
|
+
| `tls` | true | true | enable STARTTLS (port 587) |
|
5981
|
+
| `ssl` | false | false | enable TLS (port 465) |
|
5982
|
+
| `port` | 587 or 465 or 25 | 587 | port for service |
|
5433
5983
|
| `domain` | domain of server | gmail.com | email domain of user |
|
5434
5984
|
| `username` | - | john@example.com | user to authenticate on SMTP server, leave empty for open auth. |
|
5435
5985
|
| `password` | - | my_password_here | password for above username |
|
5436
5986
|
| `from_email` | username if defined | johnny@example.com | address used if receiver replies |
|
5437
5987
|
| `from_name` | same as email | John Wayne | display name of sender |
|
5988
|
+
<!-- markdownlint-enable MD034 -->
|
5438
5989
|
|
5439
5990
|
### Example of configuration
|
5440
5991
|
|
@@ -5548,7 +6099,7 @@ There are special "extended" [*transfer-spec*](#transferspec) parameters support
|
|
5548
6099
|
| required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
|
5549
6100
|
| startup | JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn) | command line arguments | API | daemon |
|
5550
6101
|
| events | JSON on stdout | none by default<br/>or need to open management port<br/>and proprietary text syntax | callback | callback |
|
5551
|
-
| platforms | any with
|
6102
|
+
| platforms | any with Ruby and `ascp` | any with `ascp` (and SDK if compiled) | any with `ascp` | any with `ascp` and transfer daemon |
|
5552
6103
|
|
5553
6104
|
### Simple session
|
5554
6105
|
|
@@ -5642,11 +6193,11 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
5642
6193
|
- `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
|
5643
6194
|
- `--src-base` (`src_base`) if top level folder name shall not be created on destination
|
5644
6195
|
|
5645
|
-
> **Note:** `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transferspec), with
|
6196
|
+
> **Note:** `ascli` takes transfer parameters exclusively as a [*transfer-spec*](#transferspec), with `ts` option.
|
5646
6197
|
>
|
5647
6198
|
> **Note:** Most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transferspec) parameters.
|
5648
6199
|
>
|
5649
|
-
> **Note:** Only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided option `transfer_info`
|
6200
|
+
> **Note:** Only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
5650
6201
|
|
5651
6202
|
#### server side and configuration
|
5652
6203
|
|
@@ -5659,7 +6210,7 @@ Virtually any transfer on a "repository" on a regular basis might emulate a hot
|
|
5659
6210
|
#### Scheduling
|
5660
6211
|
|
5661
6212
|
Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
5662
|
-
Refer to section [
|
6213
|
+
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
5663
6214
|
|
5664
6215
|
### Example: upload hot folder
|
5665
6216
|
|
@@ -5739,30 +6290,7 @@ Main components:
|
|
5739
6290
|
- `Aspera::Fasp`: starting and monitoring transfers. It can be considered as a FASPManager class for Ruby.
|
5740
6291
|
- `Aspera::Cli`: `ascli`.
|
5741
6292
|
|
5742
|
-
|
5743
|
-
|
5744
|
-
```bash
|
5745
|
-
ascli config gem path
|
5746
|
-
```
|
5747
|
-
|
5748
|
-
```bash
|
5749
|
-
cat $(ascli config gem path)/../examples/transfer.rb
|
5750
|
-
```
|
5751
|
-
|
5752
|
-
This sample code shows some example of use of the API as well as REST API.
|
5753
|
-
Note: although nice, it's probably a good idea to use RestClient for REST.
|
5754
|
-
|
5755
|
-
Example of use of the API of Aspera on Cloud:
|
5756
|
-
|
5757
|
-
```ruby
|
5758
|
-
require 'aspera/aoc'
|
5759
|
-
|
5760
|
-
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/ascli/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
5761
|
-
|
5762
|
-
aoc.read('self')
|
5763
|
-
```
|
5764
|
-
|
5765
|
-
<https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
|
6293
|
+
Working examples can be found in repo: <https://github.com/laurent-martin/aspera-api-examples> in Ruby examples.
|
5766
6294
|
|
5767
6295
|
## Changes (Release notes)
|
5768
6296
|
|
@@ -5790,6 +6318,13 @@ So, it evolved into `ascli`:
|
|
5790
6318
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
5791
6319
|
It had the advantage of being relatively easy to installed, as a single executable (well, still using `ascp`), but it was too limited IMHO, and lacked a lot of the features of this CLI.
|
5792
6320
|
|
6321
|
+
Enjoy a coffee on me:
|
6322
|
+
|
6323
|
+
```bash
|
6324
|
+
ascli conf coffee
|
6325
|
+
ascli conf coffee --ui=text
|
6326
|
+
```
|
6327
|
+
|
5793
6328
|
## Common problems
|
5794
6329
|
|
5795
6330
|
### Error: "Remote host is not who we expected"
|
@@ -5810,7 +6345,7 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
5810
6345
|
|
5811
6346
|
Some Ruby gems dependencies require compilation of native parts (C).
|
5812
6347
|
This also requires Ruby header files.
|
5813
|
-
If Ruby was installed as a Linux Packages, then also install
|
6348
|
+
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
5814
6349
|
`ruby-dev` ir `ruby-devel`, depending on distribution.
|
5815
6350
|
|
5816
6351
|
### ED255519 key not supported
|
@@ -5823,5 +6358,5 @@ Without this deactivation, if such key was present the following error was gener
|
|
5823
6358
|
OpenSSH keys only supported if ED25519 is available
|
5824
6359
|
```
|
5825
6360
|
|
5826
|
-
Which meant that you do not have
|
6361
|
+
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
5827
6362
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|