aspera-cli 4.14.0 → 4.15.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +54 -3
- data/CONTRIBUTING.md +7 -7
- data/README.md +1457 -880
- data/bin/ascli +18 -9
- data/bin/asession +12 -14
- data/examples/proxy.pac +1 -1
- data/lib/aspera/aoc.rb +198 -127
- data/lib/aspera/ascmd.rb +24 -14
- data/lib/aspera/cli/basic_auth_plugin.rb +9 -6
- data/lib/aspera/cli/error.rb +17 -0
- data/lib/aspera/cli/extended_value.rb +47 -12
- data/lib/aspera/cli/formatter.rb +260 -171
- data/lib/aspera/cli/hints.rb +80 -0
- data/lib/aspera/cli/main.rb +101 -147
- data/lib/aspera/cli/manager.rb +160 -124
- data/lib/aspera/cli/plugin.rb +70 -59
- data/lib/aspera/cli/plugins/alee.rb +0 -1
- data/lib/aspera/cli/plugins/aoc.rb +239 -273
- data/lib/aspera/cli/plugins/ats.rb +8 -5
- data/lib/aspera/cli/plugins/bss.rb +2 -2
- data/lib/aspera/cli/plugins/config.rb +516 -375
- data/lib/aspera/cli/plugins/console.rb +40 -0
- data/lib/aspera/cli/plugins/cos.rb +4 -5
- data/lib/aspera/cli/plugins/faspex.rb +99 -84
- data/lib/aspera/cli/plugins/faspex5.rb +179 -148
- data/lib/aspera/cli/plugins/node.rb +219 -153
- data/lib/aspera/cli/plugins/orchestrator.rb +52 -17
- data/lib/aspera/cli/plugins/preview.rb +46 -32
- data/lib/aspera/cli/plugins/server.rb +57 -17
- data/lib/aspera/cli/plugins/shares.rb +34 -12
- data/lib/aspera/cli/sync_actions.rb +68 -0
- data/lib/aspera/cli/transfer_agent.rb +45 -55
- data/lib/aspera/cli/transfer_progress.rb +74 -0
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/colors.rb +3 -1
- data/lib/aspera/command_line_builder.rb +14 -11
- data/lib/aspera/cos_node.rb +3 -2
- data/lib/aspera/environment.rb +17 -6
- data/lib/aspera/fasp/agent_aspera.rb +126 -0
- data/lib/aspera/fasp/agent_base.rb +31 -77
- data/lib/aspera/fasp/agent_connect.rb +21 -22
- data/lib/aspera/fasp/agent_direct.rb +88 -102
- data/lib/aspera/fasp/agent_httpgw.rb +196 -192
- data/lib/aspera/fasp/agent_node.rb +41 -34
- data/lib/aspera/fasp/agent_trsdk.rb +75 -34
- data/lib/aspera/fasp/error_info.rb +2 -2
- data/lib/aspera/fasp/faux_file.rb +52 -0
- data/lib/aspera/fasp/installation.rb +43 -184
- data/lib/aspera/fasp/management.rb +244 -0
- data/lib/aspera/fasp/parameters.rb +59 -26
- data/lib/aspera/fasp/parameters.yaml +75 -8
- data/lib/aspera/fasp/products.rb +162 -0
- data/lib/aspera/fasp/transfer_spec.rb +1 -1
- data/lib/aspera/fasp/uri.rb +4 -4
- data/lib/aspera/faspex_gw.rb +2 -2
- data/lib/aspera/faspex_postproc.rb +2 -2
- data/lib/aspera/hash_ext.rb +2 -2
- data/lib/aspera/json_rpc.rb +49 -0
- data/lib/aspera/line_logger.rb +23 -0
- data/lib/aspera/log.rb +57 -16
- data/lib/aspera/node.rb +97 -14
- data/lib/aspera/oauth.rb +36 -18
- data/lib/aspera/open_application.rb +4 -4
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/file_types.rb +4 -2
- data/lib/aspera/preview/generator.rb +22 -35
- data/lib/aspera/preview/options.rb +2 -0
- data/lib/aspera/preview/terminal.rb +24 -13
- data/lib/aspera/preview/utils.rb +19 -26
- data/lib/aspera/rest.rb +103 -72
- data/lib/aspera/rest_call_error.rb +1 -1
- data/lib/aspera/rest_error_analyzer.rb +15 -14
- data/lib/aspera/rest_errors_aspera.rb +37 -34
- data/lib/aspera/secret_hider.rb +14 -16
- data/lib/aspera/ssh.rb +4 -1
- data/lib/aspera/sync.rb +128 -122
- data/lib/aspera/temp_file_manager.rb +10 -3
- data/lib/aspera/web_auth.rb +10 -7
- data/lib/aspera/web_server_simple.rb +9 -4
- data.tar.gz.sig +0 -0
- metadata +33 -15
- metadata.gz.sig +0 -0
- data/lib/aspera/cli/listener/line_dump.rb +0 -19
- data/lib/aspera/cli/listener/logger.rb +0 -22
- data/lib/aspera/cli/listener/progress.rb +0 -50
- data/lib/aspera/cli/listener/progress_multi.rb +0 -84
- data/lib/aspera/cli/plugins/sync.rb +0 -44
- data/lib/aspera/fasp/listener.rb +0 -13
data/README.md
CHANGED
@@ -1,16 +1,20 @@
|
|
1
1
|
# Command Line Interface for IBM Aspera products
|
2
2
|
<!-- markdownlint-disable MD033 MD003 MD053 -->
|
3
|
-
<!-- cspell:ignore
|
3
|
+
<!-- cspell:ignore Serban Antipolis -->
|
4
4
|
|
5
5
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
6
6
|
|
7
|
-
|
7
|
+
[![Gem Version](https://badge.fury.io/rb/aspera-cli.svg)](https://badge.fury.io/rb/aspera-cli)
|
8
|
+
[![unit tests](https://github.com/IBM/aspera-cli/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/IBM/aspera-cli/actions)
|
9
|
+
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5861/badge)](https://bestpractices.coreinfrastructure.org/projects/5861)
|
10
|
+
|
11
|
+
## Introduction
|
8
12
|
|
9
|
-
Version : 4.
|
13
|
+
Version : 4.15.0
|
10
14
|
|
11
15
|
Laurent/2016-2023
|
12
16
|
|
13
|
-
This gem provides the `ascli` Command Line Interface to IBM Aspera software.
|
17
|
+
This gem provides the `ascli` CLI (Command Line Interface) to IBM Aspera software.
|
14
18
|
|
15
19
|
`ascli` is a also great tool to learn Aspera APIs.
|
16
20
|
|
@@ -27,15 +31,13 @@ Minimum required Ruby version: >= 2.6.
|
|
27
31
|
|
28
32
|
Release notes: see [CHANGELOG.md](CHANGELOG.md)
|
29
33
|
|
30
|
-
|
34
|
+
A PDF version of this documentation is available here: [docs/Manual.pdf](docs/Manual.pdf).
|
31
35
|
|
32
|
-
|
36
|
+
### BUGS, FEATURES, CONTRIBUTION
|
33
37
|
|
34
38
|
Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
35
39
|
|
36
|
-
|
37
|
-
|
38
|
-
## <a id="when_to_use"></a>When to use and when not to use
|
40
|
+
### <a id="when_to_use"></a>When to use and when not to use
|
39
41
|
|
40
42
|
`ascli` is designed to be used as a command line tool to:
|
41
43
|
|
@@ -63,7 +65,7 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
|
|
63
65
|
|
64
66
|
For scripting and ad'hoc command line operations, `ascli` is perfect.
|
65
67
|
|
66
|
-
|
68
|
+
### Notations, Shell, Examples
|
67
69
|
|
68
70
|
Command line operations examples are shown using a shell such: `bash` or `zsh`.
|
69
71
|
|
@@ -92,7 +94,7 @@ ascli --version
|
|
92
94
|
```
|
93
95
|
|
94
96
|
```bash
|
95
|
-
4.
|
97
|
+
4.15.0
|
96
98
|
```
|
97
99
|
|
98
100
|
### First use
|
@@ -112,14 +114,14 @@ ascli server browse /
|
|
112
114
|
```
|
113
115
|
|
114
116
|
```output
|
115
|
-
|
116
|
-
| zmode | zuid
|
117
|
-
|
118
|
-
| drwxr-xr-x |
|
119
|
-
| dr-xr-xr-x |
|
120
|
-
| dr-xr-xr-x |
|
121
|
-
| dr-xr-xr-x |
|
122
|
-
|
117
|
+
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
118
|
+
| zmode | zuid | zgid | size | mtime | name |
|
119
|
+
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
120
|
+
| drwxr-xr-x | aspera | asperaweb | 90112 | 2023-04-05 15:31:21 +0200 | Upload |
|
121
|
+
| dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:16 +0200 | aspera-test-dir-large |
|
122
|
+
| dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-small |
|
123
|
+
| dr-xr-xr-x | aspera | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-tiny |
|
124
|
+
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
123
125
|
```
|
124
126
|
|
125
127
|
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [option preset](#lprt) for the server's authentication options. The following example will:
|
@@ -168,7 +170,7 @@ ascli server download /aspera-test-dir-large/200MB
|
|
168
170
|
```
|
169
171
|
|
170
172
|
```output
|
171
|
-
Time: 00:00:02
|
173
|
+
Time: 00:00:02 ============================================= 100% 100 Mbps Time: 00:00:00
|
172
174
|
complete
|
173
175
|
```
|
174
176
|
|
@@ -198,7 +200,7 @@ An internet connection is required for the installation. If you don't have inter
|
|
198
200
|
|
199
201
|
### Container
|
200
202
|
|
201
|
-
The container image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
|
203
|
+
The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
|
202
204
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
203
205
|
To use the container, ensure that you have `podman` (or `docker`) installed.
|
204
206
|
|
@@ -206,7 +208,7 @@ To use the container, ensure that you have `podman` (or `docker`) installed.
|
|
206
208
|
podman --version
|
207
209
|
```
|
208
210
|
|
209
|
-
#### Container:
|
211
|
+
#### Container: Quick start
|
210
212
|
|
211
213
|
**Wanna start quickly ?** With an interactive shell ? Execute this:
|
212
214
|
|
@@ -254,7 +256,7 @@ ascli -v
|
|
254
256
|
```
|
255
257
|
|
256
258
|
```text
|
257
|
-
4.
|
259
|
+
4.15.0
|
258
260
|
```
|
259
261
|
|
260
262
|
In order to keep persistency of configuration on the host,
|
@@ -306,12 +308,12 @@ A convenience sample script is also provided: download the script [`dascli`](../
|
|
306
308
|
|
307
309
|
Some environment variables can be set for this script to adapt its behavior:
|
308
310
|
|
309
|
-
| env var
|
310
|
-
|
311
|
-
| ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
312
|
-
| docker_args | additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
313
|
-
| image | container image name | martinlaurent/ascli
|
314
|
-
| version | container image version | latest | `4.8.0.pre` |
|
311
|
+
| env var | description | default | example |
|
312
|
+
|----------------|------------------------------------|--------------------------|--------------------------|
|
313
|
+
| `ASCLI_HOME` | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
314
|
+
| `docker_args` | additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
315
|
+
| `image` | container image name | `martinlaurent/ascli` | |
|
316
|
+
| `version` | container image version | latest | `4.8.0.pre` |
|
315
317
|
|
316
318
|
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
317
319
|
(value expected in the container).
|
@@ -337,7 +339,7 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
337
339
|
|
338
340
|
> **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
|
339
341
|
>
|
340
|
-
> **Note:** Do not use too many volumes, as the
|
342
|
+
> **Note:** Do not use too many volumes, as the legacy `aufs` limits the number. (anyway, prefer to use `overlay2`)
|
341
343
|
|
342
344
|
#### Container: Offline installation
|
343
345
|
|
@@ -364,7 +366,7 @@ If one wants to change the content, it is possible to tell `ascp` to use another
|
|
364
366
|
echo '<CONF/>' > $HOME/.aspera/ascli/aspera.conf
|
365
367
|
```
|
366
368
|
|
367
|
-
Then, tell `ascp` to use that other
|
369
|
+
Then, tell `ascp` to use that other configuration file:
|
368
370
|
|
369
371
|
```bash
|
370
372
|
--transfer-info=@json:'{"ascp_args":["-f","/home/cliuser/.aspera/ascli/aspera.conf"]}'
|
@@ -376,7 +378,7 @@ Singularity is another type of use of container.
|
|
376
378
|
|
377
379
|
On Linux install:
|
378
380
|
|
379
|
-
```
|
381
|
+
```bash
|
380
382
|
dnf install singularity-ce
|
381
383
|
```
|
382
384
|
|
@@ -386,13 +388,13 @@ Build an image like this:
|
|
386
388
|
singularity build ascli.sif docker://martinlaurent/ascli
|
387
389
|
```
|
388
390
|
|
389
|
-
|
391
|
+
Then, start `ascli` like this:
|
390
392
|
|
391
393
|
```bash
|
392
394
|
singularity run ascli.sif
|
393
395
|
```
|
394
396
|
|
395
|
-
Or get a shell with access to
|
397
|
+
Or get a shell with access to `ascli` like this:
|
396
398
|
|
397
399
|
```bash
|
398
400
|
singularity shell ascli.sif
|
@@ -402,13 +404,13 @@ singularity shell ascli.sif
|
|
402
404
|
|
403
405
|
Use this method to install on the native host.
|
404
406
|
|
405
|
-
A Ruby interpreter is required to run
|
407
|
+
A Ruby interpreter is required to run `ascli` or to use the gem and tool.
|
406
408
|
|
407
409
|
Required Ruby version: >= 2.6.
|
408
410
|
|
409
411
|
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
410
412
|
|
411
|
-
|
413
|
+
**Ruby can be installed using any method** : rpm, yum, dnf, rvm, brew, Windows installer, ... .
|
412
414
|
|
413
415
|
In priority, refer to the official Ruby documentation:
|
414
416
|
|
@@ -421,7 +423,7 @@ The recommended installation method is `rvm` for Unix-like systems (Linux, AIX,
|
|
421
423
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
422
424
|
If you have a simpler better way to install Ruby : use it !
|
423
425
|
|
424
|
-
#### Generic: RVM:
|
426
|
+
#### Generic: RVM: Single user installation (not root)
|
425
427
|
|
426
428
|
Use this method which provides more flexibility.
|
427
429
|
|
@@ -455,7 +457,7 @@ rvm install 3.2.2
|
|
455
457
|
|
456
458
|
Ruby is now installed for the user, go to [Gem installation](#the_gem).
|
457
459
|
|
458
|
-
#### Generic: RVM:
|
460
|
+
#### Generic: RVM: Global installation (as root)
|
459
461
|
|
460
462
|
Follow the same method as single user install, but execute as "root".
|
461
463
|
|
@@ -467,7 +469,12 @@ curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
|
467
469
|
```
|
468
470
|
|
469
471
|
As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
|
470
|
-
If so, one can rename the login script:
|
472
|
+
If so, one can rename the login script:
|
473
|
+
|
474
|
+
```bash
|
475
|
+
mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok
|
476
|
+
```
|
477
|
+
|
471
478
|
To activate Ruby (and ascli) later, source it:
|
472
479
|
|
473
480
|
```bash
|
@@ -480,16 +487,69 @@ rvm version
|
|
480
487
|
|
481
488
|
#### Windows: Installer
|
482
489
|
|
483
|
-
|
490
|
+
Manual installation:
|
484
491
|
|
485
492
|
- Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) → **Downloads**.
|
486
|
-
- Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
|
493
|
+
- Download the latest Ruby installer **"with devkit"**. (`Msys2` is needed to install some native extensions, such as `grpc`)
|
487
494
|
- Execute the installer which installs by default in: `C:\RubyVV-x64` (VV is the version number)
|
488
|
-
- At the end of the installation procedure, the Msys2 installer is automatically executed, select option 3 (
|
495
|
+
- At the end of the installation procedure, the `Msys2` installer is automatically executed, select option 3 (`Msys2` and mingw)
|
496
|
+
- then install the aspera-cli gem and Aspera SDK (see next sections)
|
497
|
+
|
498
|
+
Automated installation, with internet access:
|
499
|
+
|
500
|
+
The ruby installer supports silent installation, to see the options, execute it with `/help`, or refer to the [Ruby Installer FAQ](https://github.com/oneclick/rubyinstaller2/wiki/FAQ)
|
501
|
+
|
502
|
+
Download the ruby installer executable from <https://rubyinstaller.org/downloads/> and then install:
|
503
|
+
|
504
|
+
```bat
|
505
|
+
rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
|
506
|
+
```
|
507
|
+
|
508
|
+
Installation without network:
|
509
|
+
|
510
|
+
It is essentially the same procedure, but instead of retrieving files from internet, one copies the files from a machine with internet access, and then install from those archives:
|
511
|
+
|
512
|
+
- Download the `exe` ruby installer from <https://rubyinstaller.org/downloads/>
|
513
|
+
- Create an archive with necessary gems: <https://help.rubygems.org/kb/rubygems/installing-gems-with-no-network>
|
514
|
+
|
515
|
+
```bat
|
516
|
+
gem install aspera-cli -N -i my_gems
|
517
|
+
```
|
518
|
+
|
519
|
+
Zip the files `*.gem` from folder `repo/my_gems`
|
520
|
+
|
521
|
+
- Download the SDK from: <https://ibm.biz/aspera_sdk>
|
522
|
+
|
523
|
+
Create a Zip with all those files, and transfer to the target system.
|
524
|
+
|
525
|
+
Then, on the target system:
|
526
|
+
|
527
|
+
- Unzip the archive
|
528
|
+
- Execute the installer:
|
529
|
+
|
530
|
+
```bat
|
531
|
+
rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
|
532
|
+
```
|
533
|
+
|
534
|
+
- Install the gems:
|
535
|
+
|
536
|
+
```bat
|
537
|
+
gem install --force --local *.gem
|
538
|
+
```
|
539
|
+
|
540
|
+
- install the SDK
|
541
|
+
|
542
|
+
```bash
|
543
|
+
ascli conf ascp install --sdk-url=file:///sdk.zip
|
544
|
+
```
|
489
545
|
|
490
|
-
|
546
|
+
> **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
|
491
547
|
|
492
|
-
macOS
|
548
|
+
#### macOS: Pre-installed or `brew`
|
549
|
+
|
550
|
+
macOS 10.13+ (High Sierra) comes with a recent Ruby.
|
551
|
+
So you can use it directly.
|
552
|
+
You will need to install aspera-cli using `sudo` :
|
493
553
|
|
494
554
|
```bash
|
495
555
|
sudo gem install aspera-cli
|
@@ -501,7 +561,7 @@ Alternatively, if you use [Homebrew](https://brew.sh/) already you can install R
|
|
501
561
|
brew install ruby
|
502
562
|
```
|
503
563
|
|
504
|
-
#### Linux:
|
564
|
+
#### Linux: Package
|
505
565
|
|
506
566
|
If your Linux distribution provides a standard Ruby package, you can use it provided that the version supported.
|
507
567
|
|
@@ -521,9 +581,9 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
|
|
521
581
|
|
522
582
|
- Install packages needed to build native gems:
|
523
583
|
|
524
|
-
|
525
|
-
|
526
|
-
|
584
|
+
```bash
|
585
|
+
dnf install -y make automake gcc gcc-c++ kernel-devel
|
586
|
+
```
|
527
587
|
|
528
588
|
- Enable the Ruby version you want:
|
529
589
|
|
@@ -545,7 +605,7 @@ apt install -y ruby ruby-dev rubygems ruby-json
|
|
545
605
|
One can cleanup the whole yum-installed Ruby environment like this to uninstall:
|
546
606
|
|
547
607
|
```bash
|
548
|
-
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
608
|
+
gem uninstall -axI $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
549
609
|
```
|
550
610
|
|
551
611
|
#### Other Unixes (AIX)
|
@@ -582,7 +642,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
|
|
582
642
|
|
583
643
|
<https://www.jruby.org/download>
|
584
644
|
|
585
|
-
> **Note:** Using jruby the startup time is longer than the native Ruby, but
|
645
|
+
> **Note:** Using `jruby`, the startup time is longer than the native Ruby, but transfer speed is not impacted (executed by `ascp` binary).
|
586
646
|
|
587
647
|
### <a id="the_gem"></a>`aspera-cli` gem
|
588
648
|
|
@@ -624,11 +684,11 @@ ascli conf ascp install
|
|
624
684
|
If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
|
625
685
|
|
626
686
|
```bash
|
627
|
-
curl -Lso
|
687
|
+
curl -Lso sdk.zip https://ibm.biz/aspera_sdk
|
628
688
|
```
|
629
689
|
|
630
690
|
```bash
|
631
|
-
ascli conf ascp install --sdk-url=file:///
|
691
|
+
ascli conf ascp install --sdk-url=file:///sdk.zip
|
632
692
|
```
|
633
693
|
|
634
694
|
The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
|
@@ -673,7 +733,7 @@ ascli conf --show-config --fields=sdk_url
|
|
673
733
|
- Download the SDK archive from that URL.
|
674
734
|
|
675
735
|
```bash
|
676
|
-
curl -Lso
|
736
|
+
curl -Lso sdk.zip https://ibm.biz/aspera_sdk
|
677
737
|
```
|
678
738
|
|
679
739
|
- Transfer those 2 files to the target system
|
@@ -687,7 +747,7 @@ tar zxvf rvm-ascli.tgz
|
|
687
747
|
|
688
748
|
source ~/.rvm/scripts/rvm
|
689
749
|
|
690
|
-
ascli conf ascp install --sdk-url=file:///
|
750
|
+
ascli conf ascp install --sdk-url=file:///sdk.zip
|
691
751
|
```
|
692
752
|
|
693
753
|
- Add those lines to shell init (`.profile`)
|
@@ -717,10 +777,10 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
|
|
717
777
|
- Transfer parameters can be altered by modification of [*transfer-spec*](#transferspec), this includes requiring multi-session
|
718
778
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
719
779
|
- Supports FaspStream creation (using Node API)
|
720
|
-
- Supports Watchfolder creation (using Node API)
|
780
|
+
- Supports "Watchfolder" creation (using Node API)
|
721
781
|
- Additional command plugins can be written by the user
|
722
782
|
- Supports download of faspex and Aspera on Cloud "external" links
|
723
|
-
- Supports "legacy" ssh based FASP transfers and remote commands (ascmd)
|
783
|
+
- Supports "legacy" ssh based FASP transfers and remote commands (`ascmd`)
|
724
784
|
|
725
785
|
Basic usage is displayed by executing:
|
726
786
|
|
@@ -763,11 +823,13 @@ So special character handling (quotes, spaces, env vars, ...) is handled by the
|
|
763
823
|
|
764
824
|
#### Shell parsing for Windows
|
765
825
|
|
766
|
-
MS Windows command line parsing is not
|
826
|
+
MS Windows command line parsing is not handled by the shell (`cmd.exe`), not handled by the operating system, but it is handled by the executable.
|
827
|
+
Typically, Windows executables use the [microsoft library for this parsing](https://learn.microsoft.com/en-us/cpp/cpp/main-function-command-line-args).
|
767
828
|
|
768
|
-
As far as `ascli` is concerned:
|
829
|
+
As far as `ascli` is concerned: the executable is Ruby.
|
830
|
+
It has its own parsing algorithm, close to a Linux shell parsing.
|
769
831
|
|
770
|
-
|
832
|
+
Thankfully, `ascli` provides a command to check the value of an argument after parsing: `config echo`.
|
771
833
|
One can also run `ascli` with option `--log-level=debug` to display the command line after parsing.
|
772
834
|
|
773
835
|
The following examples give the same result on Windows:
|
@@ -790,9 +852,9 @@ The following examples give the same result on Windows:
|
|
790
852
|
conf echo @json:"{\"url\":\"https://...\"}"
|
791
853
|
```
|
792
854
|
|
793
|
-
|
855
|
+
More details: on Windows, `cmd.exe` is typically used to start .
|
794
856
|
`cmd.exe` handles some special characters: `^"<>|%&`.
|
795
|
-
Basically it handles I/O
|
857
|
+
Basically it handles I/O redirection (`<>|`), shell variables (`%`), multiple commands (`&`) and handles those special characters from the command line.
|
796
858
|
Eventually, all those special characters are removed from the command line unless escaped with `^` or `"`.
|
797
859
|
`"` are kept and given to the program.
|
798
860
|
|
@@ -809,20 +871,21 @@ Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
|
|
809
871
|
|
810
872
|
- space characters: split arguments (space, tab, newline)
|
811
873
|
- backslash: `\` escape single special character
|
812
|
-
-
|
874
|
+
- globing characters: `*?[]{}` for file globing
|
813
875
|
- double quotes: `"`
|
814
876
|
- single quotes: `'`
|
815
877
|
|
816
878
|
#### Extended Values (JSON, Ruby, ...)
|
817
879
|
|
818
|
-
Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple
|
819
|
-
Typically, the `@json:` modifier is used, it expects a JSON string.
|
880
|
+
Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple `String`, but a complex structure (`Hash`, `Array`).
|
881
|
+
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
882
|
+
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
820
883
|
|
821
884
|
#### Testing Extended Values
|
822
885
|
|
823
886
|
In case of doubt of argument values after parsing, one can test using command `config echo`. `config echo` takes exactly **one** argument which can use the [Extended Value](#extended) syntax. Unprocessed command line arguments are shown in the error message.
|
824
887
|
|
825
|
-
Example: The shell parses three arguments (as
|
888
|
+
Example: The shell parses three arguments (as `String`: `1`, `2` and `3`), so the additional two arguments are not processed by the `echo` command.
|
826
889
|
|
827
890
|
```bash
|
828
891
|
ascli conf echo 1 2 3
|
@@ -993,27 +1056,29 @@ ascli conf echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
993
1056
|
{"title":"Test \" ' & \\"}
|
994
1057
|
```
|
995
1058
|
|
996
|
-
### Commands, Options, Positional
|
1059
|
+
### Commands, Options, Positional Arguments
|
997
1060
|
|
998
|
-
Command line arguments are the units of command line
|
1061
|
+
Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
|
1062
|
+
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#parsing).
|
999
1063
|
|
1000
1064
|
`ascli` considers three types of command line arguments:
|
1001
1065
|
|
1002
1066
|
- Commands
|
1003
1067
|
- Options
|
1004
|
-
- Positional
|
1068
|
+
- Positional Arguments
|
1069
|
+
|
1070
|
+
For example:
|
1005
1071
|
|
1006
1072
|
```bash
|
1007
1073
|
ascli command subcommand --option-name=VAL1 VAL2
|
1008
1074
|
```
|
1009
1075
|
|
1010
|
-
- executes
|
1011
|
-
- with one
|
1012
|
-
- the command has one additional
|
1076
|
+
- executes **command**: `command subcommand`
|
1077
|
+
- with one **option**: `option_name` and its **value**: `VAL1`
|
1078
|
+
- the command has one additional **positional argument**: `VAL2`
|
1013
1079
|
|
1014
|
-
|
1015
|
-
|
1016
|
-
For example `ascli conf ov` is the same as `ascli config overview`.
|
1080
|
+
If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
|
1081
|
+
For example `ascli conf pre ov` is the same as `ascli config preset overview`.
|
1017
1082
|
|
1018
1083
|
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
|
1019
1084
|
|
@@ -1032,10 +1097,12 @@ ascli conf ascp info
|
|
1032
1097
|
- `ascp` is the second level command, and is also the name of the component (singleton)
|
1033
1098
|
- `info` is the third level command, and is also the action to be performed
|
1034
1099
|
|
1035
|
-
Typically, commands are located at the beginning of the command line.
|
1100
|
+
Typically, commands are located at the **beginning** of the command line.
|
1036
1101
|
Order is significant.
|
1037
1102
|
The provided command must match one of the supported commands in the given context.
|
1038
|
-
If
|
1103
|
+
If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
1104
|
+
|
1105
|
+
Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
1039
1106
|
|
1040
1107
|
#### Options
|
1041
1108
|
|
@@ -1049,16 +1116,16 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
|
|
1049
1116
|
Exceptions:
|
1050
1117
|
|
1051
1118
|
- some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
|
1052
|
-
- some options (flags) don't take a value, e.g. `-
|
1119
|
+
- some options (flags) don't take a value, e.g. `-N`
|
1053
1120
|
- the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
|
1054
1121
|
|
1055
|
-
|
1056
|
-
|
1057
|
-
|
1122
|
+
```bash
|
1123
|
+
ascli config echo -- --sample
|
1124
|
+
```
|
1058
1125
|
|
1059
|
-
|
1060
|
-
|
1061
|
-
|
1126
|
+
```bash
|
1127
|
+
"--sample"
|
1128
|
+
```
|
1062
1129
|
|
1063
1130
|
> **Note:** Here, `--sample` is taken as an argument, and not as an option, due to `--`.
|
1064
1131
|
|
@@ -1066,14 +1133,14 @@ Options may have an (hardcoded) default value.
|
|
1066
1133
|
|
1067
1134
|
Options can be placed anywhere on command line and evaluated in order.
|
1068
1135
|
|
1069
|
-
Options are typically
|
1136
|
+
Options are typically:
|
1070
1137
|
|
1071
|
-
- optional :
|
1072
|
-
- mandatory :
|
1138
|
+
- optional : to change the default behavior
|
1139
|
+
- mandatory : so they can be placed in a config file, for example: connection information
|
1073
1140
|
|
1074
|
-
The value for
|
1141
|
+
The value for **any** options can come from the following locations (in this order, last value evaluated overrides previous value):
|
1075
1142
|
|
1076
|
-
- [Configuration file](#configfile)
|
1143
|
+
- [Configuration file](#configfile)
|
1077
1144
|
- Environment variable
|
1078
1145
|
- Command line
|
1079
1146
|
|
@@ -1081,27 +1148,37 @@ Environment variable starting with prefix: ASCLI_ are taken as option values, e.
|
|
1081
1148
|
|
1082
1149
|
Options values can be displayed for a given command by providing the `--show-config` option: `ascli node --show-config`
|
1083
1150
|
|
1084
|
-
|
1151
|
+
Parameters are typically designed as options if:
|
1152
|
+
|
1153
|
+
- it is a mandatory parameters that would benefit from being set in a config file or environment variable
|
1154
|
+
- it is optional
|
1085
1155
|
|
1086
|
-
Positional
|
1156
|
+
#### Positional Arguments
|
1087
1157
|
|
1088
|
-
|
1158
|
+
Positional Arguments are typically mandatory values for a command, such as entity creation data.
|
1089
1159
|
|
1090
|
-
|
1160
|
+
It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
|
1161
|
+
|
1162
|
+
The advantages of using a positional argument instead of an option for the same are that the command line is shorter(no option name, just the position) and the value is clearly mandatory.
|
1091
1163
|
|
1092
1164
|
The disadvantage is that it is not possible to define a default value in a config file or environment variable like for options.
|
1093
1165
|
Nevertheless, [Extended Values](#extended) syntax is supported, so it is possible to retrieve a value from the config file or environment variable.
|
1094
1166
|
|
1167
|
+
If a Positional Arguments begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
|
1168
|
+
|
1169
|
+
Very few positional arguments are optional, they are located at the end of the command line.
|
1170
|
+
|
1095
1171
|
### Interactive Input
|
1096
1172
|
|
1097
|
-
Some options and parameters are mandatory and other optional.
|
1173
|
+
Some options and parameters are mandatory and other optional.
|
1174
|
+
By default, `ascli` will ask for missing mandatory options or parameters for interactive execution.
|
1098
1175
|
|
1099
1176
|
The behavior can be controlled with:
|
1100
1177
|
|
1101
|
-
-
|
1178
|
+
- `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
|
1102
1179
|
- yes : missing mandatory parameters/options are asked to the user
|
1103
1180
|
- no : missing mandatory parameters/options raise an error message
|
1104
|
-
-
|
1181
|
+
- `--ask-options=<yes|no>` (default=no)
|
1105
1182
|
- optional parameters/options are asked to user
|
1106
1183
|
|
1107
1184
|
### Output
|
@@ -1126,8 +1203,8 @@ By default, result of type single_object and object_list are displayed using for
|
|
1126
1203
|
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
1127
1204
|
|
1128
1205
|
In a table format, when displaying "objects" (single, or list), by default, sub object are
|
1129
|
-
flattened (option `flat_hash`). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name
|
1130
|
-
Setting `flat_hash` to `false` will only display one field:
|
1206
|
+
flattened (option `flat_hash`). So, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
|
1207
|
+
Setting `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
|
1131
1208
|
When in flatten mode, it is possible to filter fields by "dotted" field name.
|
1132
1209
|
|
1133
1210
|
Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
|
@@ -1143,32 +1220,14 @@ The style of output can be set using the `format` parameter, supporting:
|
|
1143
1220
|
- `yaml` : YAML
|
1144
1221
|
- `csv` : Comma Separated Values
|
1145
1222
|
|
1146
|
-
####
|
1147
|
-
|
1148
|
-
Table output can be filtered using the `select` parameter. Example:
|
1149
|
-
|
1150
|
-
```bash
|
1151
|
-
ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
1152
|
-
```
|
1153
|
-
|
1154
|
-
```output
|
1155
|
-
+-------------------------------+----------------------------------+-----------+
|
1156
|
-
| name | email | ats_admin |
|
1157
|
-
+-------------------------------+----------------------------------+-----------+
|
1158
|
-
| John Curtis | john@example.com | true |
|
1159
|
-
| Laurent Martin | laurent@example.com | true |
|
1160
|
-
+-------------------------------+----------------------------------+-----------+
|
1161
|
-
```
|
1162
|
-
|
1163
|
-
> **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
|
1164
|
-
|
1165
|
-
#### entity identifier
|
1223
|
+
#### Entity identifier
|
1166
1224
|
|
1167
|
-
When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command: e.g. `ascli aoc admin res user show 1234` where `1234` is the user identifier.
|
1225
|
+
When a command is executed on a single entity, the entity is identified by a unique identifier that follows the command: e.g. `ascli aoc admin res user show 1234` where `1234` is the user's identifier.
|
1168
1226
|
|
1169
1227
|
> **Note:** The legacy option `id` is deprecated: `--id=1234` as it does not provide the possibility to have sub-entities.
|
1170
1228
|
|
1171
|
-
Only some commands provide the following capability:
|
1229
|
+
Only some commands provide the following capability:
|
1230
|
+
If the entity can also be uniquely identified by a name, then the name can be used instead of the identifier, using the **percent selector**: `ascli aoc admin res user show %name:john` where `john` is the user name.
|
1172
1231
|
|
1173
1232
|
Syntax: `%<field>:<value>`
|
1174
1233
|
|
@@ -1189,67 +1248,122 @@ The option `display` controls the level of output:
|
|
1189
1248
|
By default, secrets are removed from output: option `show_secrets` defaults to `no`, unless `display` is `data`, to allows piping results.
|
1190
1249
|
To hide secrets from output, set option `show_secrets` to `no`.
|
1191
1250
|
|
1192
|
-
#### Selection of output object properties
|
1251
|
+
#### Option: `fields`: Selection of output object properties
|
1252
|
+
|
1253
|
+
By default, a table output will display one line per entry, and columns for each properties.
|
1254
|
+
Depending on the command, columns may include by default all properties, or only some selected properties.
|
1255
|
+
It is possible to define specific columns to be displayed, by setting the `fields` option.
|
1256
|
+
|
1257
|
+
The `fields` option can be either a comma separated list, or an extended value array.
|
1258
|
+
|
1259
|
+
Elements of the list can be:
|
1260
|
+
|
1261
|
+
- `DEF` : default display of columns (that's the default, when not set)
|
1262
|
+
- `ALL` : all columns available
|
1263
|
+
- -property : remove property from the current list
|
1264
|
+
- property : add property to the current list
|
1265
|
+
- a ruby `RegEx` : using `@ruby:'/.../'`
|
1266
|
+
|
1267
|
+
Examples:
|
1268
|
+
|
1269
|
+
- `a,b,c` : the list of attributes specified as a comma separated list
|
1270
|
+
- `@json:'["a","b","c"]'` : Array extended value: same as above
|
1271
|
+
- `DEF,-a,b` : default property list, remove `a` and add `b`
|
1272
|
+
- `@ruby:'/^server/'` : Display all properties whose name begin with `server`
|
1273
|
+
|
1274
|
+
#### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
|
1275
|
+
|
1276
|
+
Table output can be filtered using option `select`.
|
1277
|
+
This parameter is either a `Hash` or `Proc`.
|
1278
|
+
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
|
1279
|
+
|
1280
|
+
Example:
|
1281
|
+
|
1282
|
+
```bash
|
1283
|
+
ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
1284
|
+
```
|
1285
|
+
|
1286
|
+
```output
|
1287
|
+
+-------------------------------+----------------------------------+-----------+
|
1288
|
+
| name | email | ats_admin |
|
1289
|
+
+-------------------------------+----------------------------------+-----------+
|
1290
|
+
| John Curtis | john@example.com | true |
|
1291
|
+
| Laurent Martin | laurent@example.com | true |
|
1292
|
+
+-------------------------------+----------------------------------+-----------+
|
1293
|
+
```
|
1294
|
+
|
1295
|
+
> **Note:** `select` filters elements from the result of command, while the `query` parameters gives filtering parameters to the API when listing elements.
|
1193
1296
|
|
1194
|
-
|
1297
|
+
In above example, the same result is obtained with option:
|
1195
1298
|
|
1196
|
-
|
1197
|
-
|
1198
|
-
|
1199
|
-
- Array extended value: for instance, @json:'["a","b","c"]' same as above
|
1200
|
-
- +a,b,c : add selected properties to the default selection.
|
1201
|
-
- -a,b,c : remove selected properties from the default selection.
|
1299
|
+
```bash
|
1300
|
+
--select=@ruby:'->(i){i["ats_admin"]}'
|
1301
|
+
```
|
1202
1302
|
|
1203
1303
|
### <a id="extended"></a>Extended Value Syntax
|
1204
1304
|
|
1205
|
-
|
1206
|
-
|
1305
|
+
Most options and arguments are specified by a simple string (e.g. username or url).
|
1306
|
+
Sometime it is convenient to read a value from a file: for example read the PEM value of a private key, or a list of files.
|
1307
|
+
Some options expect a more complex value such as `Hash` or `Array`.
|
1207
1308
|
|
1208
|
-
The
|
1309
|
+
The **Extended Value** Syntax allows to specify such values and even read values from other sources than the command line itself.
|
1310
|
+
|
1311
|
+
The syntax is:
|
1209
1312
|
|
1210
1313
|
```bash
|
1211
|
-
<0 or more decoders><
|
1314
|
+
<0 or more decoders><some text value or nothing>
|
1212
1315
|
```
|
1213
1316
|
|
1214
|
-
Decoders act like a function
|
1215
|
-
Decoders are recognized by the prefix: `@` and suffix `:`
|
1317
|
+
Decoders act like a function with its parameter on right hand side and are recognized by the prefix: `@` and suffix `:`
|
1216
1318
|
|
1217
1319
|
The following decoders are supported:
|
1218
1320
|
|
1219
1321
|
| decoder | parameter | returns | description |
|
1220
1322
|
|---------|-----------|---------|-------------|
|
1221
1323
|
| base64 | String | String | decode a base64 encoded string
|
1222
|
-
| csvt
|
1223
|
-
| env
|
1224
|
-
| file
|
1225
|
-
|
|
1226
|
-
|
|
1227
|
-
|
|
1228
|
-
|
|
1229
|
-
| path
|
1324
|
+
| `csvt` | String | Array | decode a titled CSV value
|
1325
|
+
| `env` | String | String | read from a named env var name, e.g. `--password=@env:MYPASSVAR`
|
1326
|
+
| `file` | String | String | read value from specified file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
1327
|
+
| `json` | String | Any | decode JSON values (convenient to provide complex structures)
|
1328
|
+
| `lines` | String | Array | split a string in multiple lines and return an array
|
1329
|
+
| `list` | String | Array | split a string in multiple items taking first character as separator and return an array
|
1330
|
+
| `none` | None | Nil | A null value
|
1331
|
+
| `path` | String | String | performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
1230
1332
|
| preset | String | Hash | get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
|
1231
|
-
|
|
1333
|
+
| extend | String | String | evaluates embedded extended value syntax in string
|
1334
|
+
| `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`)
|
1335
|
+
| `ruby` | String | Any | execute specified Ruby code
|
1232
1336
|
| secret | None | String | Ask password interactively (hides input)
|
1233
|
-
| stdin
|
1234
|
-
| uri
|
1235
|
-
| val
|
1236
|
-
|
|
1337
|
+
| `stdin` | None | String | read from stdin (no value on right)
|
1338
|
+
| `uri` | String | String | read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac`
|
1339
|
+
| `val` | String | String | prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`.
|
1340
|
+
| `yaml` | String | Any | decode YAML
|
1341
|
+
| `zlib` | String | String | un-compress data
|
1237
1342
|
|
1238
1343
|
To display the result of an extended value, use the `config echo` command.
|
1239
1344
|
|
1345
|
+
The `extend` decoder is useful to evaluate embedded extended value syntax in a string.
|
1346
|
+
It expects a `@` to close the embedded extended value syntax.
|
1347
|
+
|
1348
|
+
Example: Create a `Hash` value with the convenient `@json:` decoder:
|
1349
|
+
|
1350
|
+
```bash
|
1351
|
+
ascli config echo @json:'{"key1":"value1","key2":"value2"}'
|
1352
|
+
```
|
1353
|
+
|
1240
1354
|
Example: read the content of the specified file, then, base64 decode, then unzip:
|
1241
1355
|
|
1242
1356
|
```bash
|
1243
1357
|
ascli config echo @zlib:@base64:@file:myfile.dat
|
1244
1358
|
```
|
1245
1359
|
|
1246
|
-
Example: Create a value
|
1360
|
+
Example: Create a `Hash` value with one key and the value is read from a file:
|
1247
1361
|
|
1248
1362
|
```bash
|
1249
1363
|
ascli config echo @ruby:'{"token_verification_key"=>File.read("mykey.txt")}'
|
1250
1364
|
```
|
1251
1365
|
|
1252
|
-
Example: read a csv file and create
|
1366
|
+
Example: read a csv file and create an `Array` of `Hash` for bulk provisioning:
|
1253
1367
|
|
1254
1368
|
```bash
|
1255
1369
|
cat test.csv
|
@@ -1274,37 +1388,49 @@ ascli config echo @csvt:@file:test.csv
|
|
1274
1388
|
+------+---------------------+
|
1275
1389
|
```
|
1276
1390
|
|
1277
|
-
Example: create a
|
1391
|
+
Example: create a `Hash` with values coming from a preset named `config`
|
1278
1392
|
|
1279
1393
|
```bash
|
1280
|
-
ascli config echo @
|
1394
|
+
ascli config echo @json:@extend:'{"hello":true,"version":"@preset:config.version@"}'
|
1281
1395
|
```
|
1282
1396
|
|
1283
|
-
```
|
1284
|
-
|
1397
|
+
```output
|
1398
|
+
+---------+-----------+
|
1399
|
+
| key | value |
|
1400
|
+
+---------+-----------+
|
1401
|
+
| hello | true |
|
1402
|
+
| version | 4.14.0 |
|
1403
|
+
+---------+-----------+
|
1285
1404
|
```
|
1286
1405
|
|
1287
|
-
|
1288
|
-
|
1289
|
-
### <a id="native"></a>Structured Value
|
1290
|
-
|
1291
|
-
Some options and parameters expect a [Extended Value](#extended), i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
|
1406
|
+
Example: Create a `Hash` from YAML provided as **heredoc**:
|
1292
1407
|
|
1293
|
-
|
1294
|
-
|
1295
|
-
|
1296
|
-
|
1408
|
+
```bash
|
1409
|
+
ascli conf echo @yaml:@stdin: --format=json<<EOF
|
1410
|
+
key1: value1
|
1411
|
+
key2:
|
1412
|
+
- item1
|
1413
|
+
- item2
|
1414
|
+
key3:
|
1415
|
+
key4: value4
|
1416
|
+
key5: value5
|
1417
|
+
EOF
|
1418
|
+
```
|
1297
1419
|
|
1298
|
-
|
1420
|
+
```json
|
1421
|
+
{"key1":"value1","key2":["item1","item2"],"key3":{"key4":"value4","key5":"value5"}}
|
1422
|
+
```
|
1299
1423
|
|
1300
1424
|
### <a id="conffolder"></a>Configuration and Persistency Folder
|
1301
1425
|
|
1302
|
-
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored
|
1426
|
+
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored by default in `[User's home folder]/.aspera/ascli`.
|
1303
1427
|
|
1304
|
-
Note
|
1305
|
-
It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`.
|
1428
|
+
> **Note:** `[User's home folder]` is found using Ruby's `Dir.home` (`rb_w32_home_dir`).
|
1429
|
+
It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`.
|
1430
|
+
`ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists.
|
1431
|
+
So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
|
1306
1432
|
|
1307
|
-
The
|
1433
|
+
The configuration folder can be displayed using :
|
1308
1434
|
|
1309
1435
|
```bash
|
1310
1436
|
ascli config folder
|
@@ -1314,7 +1440,7 @@ ascli config folder
|
|
1314
1440
|
/Users/kenji/.aspera/ascli
|
1315
1441
|
```
|
1316
1442
|
|
1317
|
-
It can be overridden using
|
1443
|
+
It can be overridden using option `home`.
|
1318
1444
|
|
1319
1445
|
Example (Windows):
|
1320
1446
|
|
@@ -1326,11 +1452,17 @@ ascli config folder
|
|
1326
1452
|
C:\Users\Kenji\.aspera\ascli
|
1327
1453
|
```
|
1328
1454
|
|
1329
|
-
When OAuth is used (AoC, Faspex4 api v4, Faspex5) `ascli` keeps a cache of generated bearer tokens in `
|
1455
|
+
When OAuth is used (AoC, Faspex4 api v4, Faspex5) `ascli` keeps a cache of generated bearer tokens in folder `persist_store` in configuration folder by default.
|
1330
1456
|
Option `cache_tokens` (**yes**/no) allows to control if Oauth tokens are cached on file system, or generated for each request.
|
1331
|
-
The command `config flush_tokens`
|
1457
|
+
The command `config flush_tokens` clears that cache.
|
1332
1458
|
Tokens are kept on disk for a maximum of 30 minutes (`TOKEN_CACHE_EXPIRY_SEC`) and garbage collected after that.
|
1333
|
-
|
1459
|
+
When a token has expired, then a new token is generated, either using a refresh_token if it is available, or by the default method.
|
1460
|
+
|
1461
|
+
### Temporary files
|
1462
|
+
|
1463
|
+
Some temporary files may be needed during runtime.
|
1464
|
+
The temporary folder may be specified with option: `temp_folder`.
|
1465
|
+
Temporary files are deleted at the end of execution unless option: `clean_temp` is set to `no`.
|
1334
1466
|
|
1335
1467
|
### <a id="configfile"></a>Configuration file
|
1336
1468
|
|
@@ -1371,7 +1503,7 @@ The command `set` allows setting individual options in a [option preset](#lprt).
|
|
1371
1503
|
ascli config preset set demo_server password my_password_here
|
1372
1504
|
```
|
1373
1505
|
|
1374
|
-
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [
|
1506
|
+
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended).
|
1375
1507
|
|
1376
1508
|
```bash
|
1377
1509
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
|
@@ -1406,11 +1538,11 @@ ascli config preset over
|
|
1406
1538
|
ascli config preset list
|
1407
1539
|
```
|
1408
1540
|
|
1409
|
-
#### <a id="lprtconf"></a>Special Option preset: config
|
1541
|
+
#### <a id="lprtconf"></a>Special Option preset: `config`
|
1410
1542
|
|
1411
1543
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
1412
1544
|
|
1413
|
-
#### <a id="lprtdef"></a>Special Option preset: default
|
1545
|
+
#### <a id="lprtdef"></a>Special Option preset: `default`
|
1414
1546
|
|
1415
1547
|
This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
|
1416
1548
|
|
@@ -1439,13 +1571,15 @@ Plugin `config` provides general commands for `ascli`:
|
|
1439
1571
|
- Option preset, config file operations
|
1440
1572
|
- wizard
|
1441
1573
|
- vault
|
1442
|
-
- ascp
|
1574
|
+
- `ascp`
|
1443
1575
|
|
1444
|
-
The default
|
1576
|
+
The default preset for `config` is read for any plugin invocation, this allows setting global options, such as `--log-level` or `--interactive`.
|
1445
1577
|
When `ascli` starts, it looks for the `default` Option preset and checks the value for `config`.
|
1446
|
-
If set, it loads the
|
1578
|
+
If set, it loads the options independently of the plugin used.
|
1447
1579
|
|
1448
|
-
> **Note:** If no global default is set by the user,
|
1580
|
+
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
|
1581
|
+
>
|
1582
|
+
> **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
|
1449
1583
|
|
1450
1584
|
Show current default (global) Option preset (`config` plugin):
|
1451
1585
|
|
@@ -1455,13 +1589,14 @@ global_common_defaults
|
|
1455
1589
|
```
|
1456
1590
|
|
1457
1591
|
```bash
|
1458
|
-
ascli conf preset set
|
1592
|
+
ascli conf preset set GLOBAL version_check_days 0
|
1459
1593
|
```
|
1460
1594
|
|
1461
|
-
If the default global Option preset is not set:
|
1595
|
+
If the default global Option preset is not set, and you want to use a different name:
|
1462
1596
|
|
1463
1597
|
```bash
|
1464
|
-
ascli conf preset set
|
1598
|
+
ascli conf preset set my_common_defaults version_check_days 0
|
1599
|
+
ascli conf preset set default config my_common_defaults
|
1465
1600
|
```
|
1466
1601
|
|
1467
1602
|
#### Config sample commands
|
@@ -1473,45 +1608,76 @@ config ascp connect version 'Aspera Connect for Windows' download 'Windows Insta
|
|
1473
1608
|
config ascp connect version 'Aspera Connect for Windows' list
|
1474
1609
|
config ascp connect version 'Aspera Connect for Windows' open documentation
|
1475
1610
|
config ascp errors
|
1476
|
-
config ascp info --sdk-folder=
|
1477
|
-
config ascp install
|
1611
|
+
config ascp info --sdk-folder=sdk_test_dir
|
1612
|
+
config ascp install
|
1613
|
+
config ascp install --sdk-folder=sdk_test_dir
|
1478
1614
|
config ascp products list
|
1615
|
+
config ascp products use 'IBM Aspera Connect'
|
1479
1616
|
config ascp show
|
1480
1617
|
config ascp spec
|
1618
|
+
config ascp use /usr/bin/ascp
|
1481
1619
|
config check_update
|
1482
1620
|
config coffee
|
1483
1621
|
config coffee --ui=text
|
1484
|
-
config detect
|
1485
|
-
config detect
|
1486
|
-
config detect
|
1622
|
+
config detect https://faspex4.example.com/path
|
1623
|
+
config detect https://faspex5.example.com/path
|
1624
|
+
config detect https://node.example.com/path
|
1625
|
+
config detect https://shares.example.com/path shares
|
1626
|
+
config detect my_org aoc
|
1487
1627
|
config doc
|
1488
1628
|
config doc transfer-parameters
|
1489
|
-
config echo 'hello'
|
1490
1629
|
config echo @base64:SGVsbG8gV29ybGQK
|
1491
1630
|
config echo @csvt:@stdin:
|
1492
1631
|
config echo @env:USER
|
1493
1632
|
config echo @lines:@stdin:
|
1494
1633
|
config echo @list:,1,2,3
|
1634
|
+
config echo @secret:
|
1495
1635
|
config echo @uri:/etc/hosts
|
1496
1636
|
config echo @uri:file:/etc/hosts
|
1497
|
-
config echo @uri:http://
|
1498
|
-
config echo @uri:https://
|
1499
|
-
config echo @
|
1637
|
+
config echo @uri:http://ifconfig.me
|
1638
|
+
config echo @uri:https://ifconfig.me
|
1639
|
+
config echo @vault:my_preset.password
|
1500
1640
|
config echo @zlib:@stdin:
|
1501
|
-
config
|
1502
|
-
config
|
1641
|
+
config echo hello
|
1642
|
+
config email_test --notify-to=my_email_external
|
1503
1643
|
config flush_tokens
|
1504
|
-
config
|
1505
|
-
config
|
1644
|
+
config folder
|
1645
|
+
config gem name
|
1646
|
+
config gem path
|
1647
|
+
config gem version
|
1648
|
+
config genkey my_key
|
1649
|
+
config genkey my_key 4096
|
1650
|
+
config initdemo
|
1651
|
+
config open
|
1652
|
+
config plugin create my_command
|
1506
1653
|
config plugin list
|
1507
|
-
config
|
1508
|
-
config
|
1509
|
-
config
|
1654
|
+
config preset delete conf_name
|
1655
|
+
config preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
|
1656
|
+
config preset list
|
1657
|
+
config preset overview
|
1658
|
+
config preset set conf_name param value
|
1659
|
+
config preset set default shares conf_name
|
1660
|
+
config preset show conf_name
|
1661
|
+
config preset unset conf_name param
|
1662
|
+
config preset update conf_name --p1=v1 --p2=v2
|
1663
|
+
config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
|
1664
|
+
config vault create my_label @json:'{"password":"my_password_here","description":"my secret"}'
|
1665
|
+
config vault delete my_label
|
1666
|
+
config vault list
|
1667
|
+
config vault show my_label
|
1668
|
+
config wizard https://console.example.com/path console
|
1669
|
+
config wizard https://faspex4.example.com/path faspex --username=test --password=test
|
1670
|
+
config wizard https://faspex5.example.com/path faspex5 --key-path=my_private_key
|
1671
|
+
config wizard https://node.example.com/path node --username=test --password=test
|
1672
|
+
config wizard https://orch.example.com/path orchestrator --username=test --password=test
|
1673
|
+
config wizard https://shares.example.com/path shares --username=test --password=test
|
1674
|
+
config wizard my_org aoc --key-path= --username=my_user_email
|
1675
|
+
config wizard my_org aoc --key-path= --username=my_user_email --use-generic-client=yes
|
1510
1676
|
```
|
1511
1677
|
|
1512
1678
|
#### Format of file
|
1513
1679
|
|
1514
|
-
The configuration file is a
|
1680
|
+
The configuration file is a `Hash` in a YAML file. Example:
|
1515
1681
|
|
1516
1682
|
```yaml
|
1517
1683
|
config:
|
@@ -1544,7 +1710,7 @@ The user may create as many [option presets](#lprt) as needed. For instance, a p
|
|
1544
1710
|
|
1545
1711
|
Values in the configuration also follow the [Extended Value Syntax](#extended).
|
1546
1712
|
|
1547
|
-
Note
|
1713
|
+
> **Note:** if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
1548
1714
|
|
1549
1715
|
```bash
|
1550
1716
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
@@ -1561,7 +1727,7 @@ my_aoc_org:
|
|
1561
1727
|
|
1562
1728
|
So, the key file will be read only at execution time, but not be embedded in the configuration file.
|
1563
1729
|
|
1564
|
-
####
|
1730
|
+
#### Evaluation order of options
|
1565
1731
|
|
1566
1732
|
Some options are global, some options are available only for some plugins. (the plugin is the first level command).
|
1567
1733
|
|
@@ -1569,7 +1735,7 @@ Options are loaded using this algorithm:
|
|
1569
1735
|
|
1570
1736
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
1571
1737
|
- else it looks for the name of the plugin as key in section `default`, the value is the name of the default [option preset](#lprt) for it, and loads it.
|
1572
|
-
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or
|
1738
|
+
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [option preset](#lprt) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
|
1573
1739
|
- Environment variables are evaluated
|
1574
1740
|
- Command line options are evaluated
|
1575
1741
|
|
@@ -1602,13 +1768,25 @@ Example: Define options using command line:
|
|
1602
1768
|
ascli -N --url=_url_here_ --password=my_password_here --username=_name_here_ node --show-config
|
1603
1769
|
```
|
1604
1770
|
|
1605
|
-
Example: Define options using a
|
1771
|
+
Example: Define options using a `Hash`:
|
1606
1772
|
|
1607
1773
|
```bash
|
1608
1774
|
ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","username":"_name_here_"}' node --show-config
|
1609
1775
|
```
|
1610
1776
|
|
1611
|
-
####
|
1777
|
+
#### Wizard
|
1778
|
+
|
1779
|
+
The wizard is a command that asks the user for information and creates a [option preset](#lprt) with the provided information.
|
1780
|
+
|
1781
|
+
It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
|
1782
|
+
|
1783
|
+
The simplest invocation is:
|
1784
|
+
|
1785
|
+
```bash
|
1786
|
+
ascli config wizard
|
1787
|
+
```
|
1788
|
+
|
1789
|
+
#### Example of configuration for a plugin
|
1612
1790
|
|
1613
1791
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
1614
1792
|
only username/password and url are required (either on command line, or from config file).
|
@@ -1666,7 +1844,7 @@ A more secure option is to retrieve values from a secret vault.
|
|
1666
1844
|
|
1667
1845
|
The vault is used with options `vault` and `vault_password`.
|
1668
1846
|
|
1669
|
-
`vault` defines the vault to be used and shall be a Hash
|
1847
|
+
`vault` defines the vault to be used and shall be a `Hash`, example:
|
1670
1848
|
|
1671
1849
|
```json
|
1672
1850
|
{"type":"system","name":"ascli"}
|
@@ -1728,14 +1906,17 @@ A passwords can be saved in clear in a [option preset](#lprt) together with othe
|
|
1728
1906
|
Example:
|
1729
1907
|
|
1730
1908
|
```bash
|
1731
|
-
|
1909
|
+
ascli conf preset update myconf --url=... --username=... --password=...
|
1732
1910
|
```
|
1733
1911
|
|
1734
1912
|
For a more secure storage one can do:
|
1735
1913
|
|
1736
1914
|
```bash
|
1737
|
-
|
1738
|
-
|
1915
|
+
ascli conf preset update myconf --url=... --username=... --password=@val:@vault:myconf.password
|
1916
|
+
```
|
1917
|
+
|
1918
|
+
```bash
|
1919
|
+
ascli conf vault create myconf @json:'{"password":"my_password_here"}'
|
1739
1920
|
```
|
1740
1921
|
|
1741
1922
|
> **Note:** use `@val:` in front of `@vault:` so that the extended value is not evaluated.
|
@@ -1764,7 +1945,7 @@ The format expected for private keys is [PEM](https://en.wikipedia.org/wiki/Priv
|
|
1764
1945
|
|
1765
1946
|
#### `ascli` for key generation
|
1766
1947
|
|
1767
|
-
The generated key is of type RSA
|
1948
|
+
The generated key is of type `RSA`, by default: **4096** bit.
|
1768
1949
|
For convenience, the public key is also extracted with extension `.pub`.
|
1769
1950
|
The key is not passphrase protected.
|
1770
1951
|
|
@@ -1772,6 +1953,14 @@ The key is not passphrase protected.
|
|
1772
1953
|
ascli config genkey ${PRIVKEYFILE} 4096
|
1773
1954
|
```
|
1774
1955
|
|
1956
|
+
> **Note:** `ascli` uses the `openssl` library.
|
1957
|
+
|
1958
|
+
To display the version of **openssl** used in `ascli`:
|
1959
|
+
|
1960
|
+
```bash
|
1961
|
+
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION
|
1962
|
+
```
|
1963
|
+
|
1775
1964
|
#### `ssh-keygen`
|
1776
1965
|
|
1777
1966
|
Both private and public keys are generated, option `-N` is for passphrase.
|
@@ -1782,7 +1971,7 @@ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
|
|
1782
1971
|
|
1783
1972
|
#### `openssl`
|
1784
1973
|
|
1785
|
-
To generate a private key
|
1974
|
+
To generate a private key with a passphrase the following can be used on any system:
|
1786
1975
|
|
1787
1976
|
```bash
|
1788
1977
|
openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE} 4096
|
@@ -1808,83 +1997,99 @@ mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
|
|
1808
1997
|
|
1809
1998
|
### <a id="certificates"></a>SSL CA certificate bundle
|
1810
1999
|
|
1811
|
-
|
1812
|
-
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
1813
|
-
|
1814
|
-
To display the current root certificate store locations:
|
2000
|
+
To display trusted certificate store locations:
|
1815
2001
|
|
1816
2002
|
```bash
|
1817
|
-
ascli
|
2003
|
+
ascli --show-config --fields=cert_stores
|
1818
2004
|
```
|
1819
2005
|
|
1820
|
-
|
2006
|
+
To modify the locations of certificate store, use option `cert_stores`.
|
2007
|
+
If you use this option, then default locations are removed, but they can be added using special value `DEF`.
|
2008
|
+
The value can be either an `Array`, or successive options.
|
1821
2009
|
|
1822
|
-
`
|
1823
|
-
|
1824
|
-
|
2010
|
+
`ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
|
2011
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
2012
|
+
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
1825
2013
|
|
1826
|
-
|
2014
|
+
> **Note:** One can display those values like this:
|
1827
2015
|
|
1828
|
-
|
2016
|
+
```bash
|
2017
|
+
ascli conf echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
2018
|
+
ascli conf echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
2019
|
+
```
|
1829
2020
|
|
1830
|
-
`
|
1831
|
-
The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
|
1832
|
-
Each plugin usually represents commands sent to a specific application.
|
1833
|
-
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
2021
|
+
`ascp` also needs to validate certificates when using **WSS**.
|
1834
2022
|
|
1835
|
-
|
2023
|
+
> **Note:** `ascli` overrides the default hardcoded location used by `ascp` for WSS (e.g. on macOS: `/Library/Aspera/ssl`) and uses the same locations as specified in `cert_stores` (using `-i` switch of `ascp`). Hardcoded locations can be found using:
|
1836
2024
|
|
1837
2025
|
```bash
|
1838
|
-
ascli conf
|
2026
|
+
strings $(ascli conf ascp info --fields=ascp)|grep -w OPENSSLDIR
|
1839
2027
|
```
|
1840
2028
|
|
1841
|
-
|
1842
|
-
|
1843
|
-
|
1844
|
-
|
1845
|
-
| shares | ..../aspera-cli/lib/aspera/cli/plugins/shares.rb |
|
1846
|
-
| node | ..../aspera-cli/lib/aspera/cli/plugins/node.rb |
|
1847
|
-
...
|
1848
|
-
+--------------+--------------------------------------------------------+
|
2029
|
+
or
|
2030
|
+
|
2031
|
+
```bash
|
2032
|
+
ascli conf ascp info --fields=openssldir
|
1849
2033
|
```
|
1850
2034
|
|
1851
|
-
|
2035
|
+
To update trusted root certificates for `ascli`:
|
2036
|
+
Display the trusted certificate store locations used by `ascli`.
|
2037
|
+
Typically done by updating the system's root certificate store.
|
1852
2038
|
|
1853
|
-
|
2039
|
+
An up-to-date version of the certificate bundle can be retrieved with:
|
1854
2040
|
|
1855
2041
|
```bash
|
1856
|
-
ascli
|
2042
|
+
ascli conf echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
1857
2043
|
```
|
1858
2044
|
|
1859
|
-
|
2045
|
+
To download that certificate store:
|
1860
2046
|
|
1861
2047
|
```bash
|
1862
|
-
ascli conf
|
2048
|
+
ascli conf echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text > /tmp/cacert.pem
|
1863
2049
|
```
|
1864
2050
|
|
1865
|
-
|
1866
|
-
|
2051
|
+
Then, use this store by setting the option `` or env var export SSL_CERT_FILE
|
2052
|
+
|
2053
|
+
To trust a certificate (e.g. self-signed), provided that the `CN` is correct, save the certificate to a file:
|
2054
|
+
|
2055
|
+
```bash
|
2056
|
+
ascli conf remote_certificate https://localhost:9092 > myserver.pem
|
1867
2057
|
```
|
1868
2058
|
|
2059
|
+
> **Note:** the saved certificate shows the CN as first line.
|
2060
|
+
|
2061
|
+
Then, use this file as certificate store (e.g. here, Node API):
|
2062
|
+
|
1869
2063
|
```bash
|
1870
|
-
ascli --
|
2064
|
+
ascli conf echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
|
1871
2065
|
```
|
1872
2066
|
|
1873
|
-
|
2067
|
+
### Image and video thumbnails
|
2068
|
+
|
2069
|
+
`ascli` can display thumbnails for images and videos in the terminal.
|
2070
|
+
This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
|
2071
|
+
It's also available when using the `show` command of `preview` plugin.
|
1874
2072
|
|
1875
|
-
|
2073
|
+
The following options can be specified in the option `query`:
|
1876
2074
|
|
1877
|
-
|
2075
|
+
| option | description |
|
2076
|
+
|------------|-------------|
|
2077
|
+
| text | display text instead of image (Bool) |
|
2078
|
+
| double | display double text resolution (half characters) (Bool) |
|
2079
|
+
| font_ratio | Font height/width ratio in terminal (Float) |
|
1878
2080
|
|
1879
|
-
|
2081
|
+
### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
|
2082
|
+
|
2083
|
+
Some actions may require the use of a graphical tool:
|
1880
2084
|
|
1881
|
-
-
|
1882
|
-
-
|
1883
|
-
- password
|
2085
|
+
- a browser for Aspera on Cloud authentication (web auth method)
|
2086
|
+
- a text editor for configuration file edition
|
1884
2087
|
|
1885
|
-
|
2088
|
+
By default `ascli` assumes that a graphical environment is available on Windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
2089
|
+
It is also possible to force the graphical mode with option --ui :
|
1886
2090
|
|
1887
|
-
|
2091
|
+
- `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
2092
|
+
- `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
1888
2093
|
|
1889
2094
|
### Logging, Debugging
|
1890
2095
|
|
@@ -1921,16 +2126,21 @@ When `ascli` is used interactively in a shell, the shell itself will usually log
|
|
1921
2126
|
|
1922
2127
|
### Learning Aspera Product APIs (REST)
|
1923
2128
|
|
1924
|
-
`ascli` uses mainly Aspera applications
|
1925
|
-
To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
|
2129
|
+
`ascli` uses mainly REST APIs to interact with Aspera applications.
|
1926
2130
|
|
1927
|
-
|
2131
|
+
To get traces of execution, with dump of API calls, use argument : `--log-level=debug`.
|
2132
|
+
|
2133
|
+
To display HTTP/S traffic set option `log_level` to `trace2`: `--log-level=trace2`.
|
2134
|
+
It will display the exact content of HTTP requests and responses.
|
1928
2135
|
|
1929
2136
|
### <a id="http_options"></a>HTTP socket parameters
|
1930
2137
|
|
1931
|
-
|
2138
|
+
To ignore SSL certificate for any address/port, use option: `insecure`, i.e. `--insecure=yes`.
|
2139
|
+
To ignore SSL certificate for specific address/port, use option `ignore_certificate`, set to an `Array` of URL for which certificate will be ignored (only the address and port are matched), e.g. `--ignore-certificate=@list:,https://127.0.0.1:9092`
|
2140
|
+
|
2141
|
+
> **Note:** Ignoring certificate also applies to `ascp`'s wss.
|
1932
2142
|
|
1933
|
-
HTTP
|
2143
|
+
HTTP connection parameters (not `ascp` wss) can be adjusted using option `http_options`:
|
1934
2144
|
|
1935
2145
|
| parameter | default |
|
1936
2146
|
|----------------------|---------|
|
@@ -1939,9 +2149,9 @@ HTTP socket parameters can be adjusted using option `http_options`:
|
|
1939
2149
|
| `open_timeout` | 60 |
|
1940
2150
|
| `keep_alive_timeout` | 2 |
|
1941
2151
|
|
1942
|
-
Values are in set
|
2152
|
+
Values are in set **seconds** and can be of type either integer or float.
|
1943
2153
|
Default values are the ones of Ruby:
|
1944
|
-
refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
2154
|
+
For a full list, refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1945
2155
|
|
1946
2156
|
Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
|
1947
2157
|
|
@@ -1951,34 +2161,22 @@ Example:
|
|
1951
2161
|
ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
|
1952
2162
|
```
|
1953
2163
|
|
1954
|
-
### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
|
1955
|
-
|
1956
|
-
Some actions may require the use of a graphical tool:
|
1957
|
-
|
1958
|
-
- a browser for Aspera on Cloud authentication (web auth method)
|
1959
|
-
- a text editor for configuration file edition
|
1960
|
-
|
1961
|
-
By default `ascli` assumes that a graphical environment is available on windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
1962
|
-
It is also possible to force the graphical mode with option --ui :
|
1963
|
-
|
1964
|
-
- `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
1965
|
-
- `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
1966
|
-
|
1967
2164
|
### Proxy
|
1968
2165
|
|
1969
2166
|
There are several types of network connections, each of them use a different mechanism to define a (forward) **proxy**:
|
1970
2167
|
|
1971
|
-
- Ruby HTTP: REST and
|
1972
|
-
- Legacy Aspera HTTP/S Fallback
|
2168
|
+
- Ruby HTTP: REST and HTTP Gateway client
|
2169
|
+
- Legacy Aspera HTTP/S Fallback and `ascp` wss
|
1973
2170
|
- Aspera FASP
|
1974
2171
|
|
1975
2172
|
Refer to the following sections.
|
1976
2173
|
|
1977
|
-
### Proxy for REST and
|
2174
|
+
### Proxy for REST and HTTP Gateway
|
1978
2175
|
|
1979
2176
|
There are two possibilities to define an HTTP proxy to be used when Ruby HTTP is used.
|
1980
2177
|
|
1981
|
-
The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy
|
2178
|
+
The `http_proxy` environment variable (**lower case**, preferred) can be set to the URL of the proxy.
|
2179
|
+
E.g. `http://myproxy.org.net:3128`.
|
1982
2180
|
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
1983
2181
|
|
1984
2182
|
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** accepted.
|
@@ -2177,7 +2375,7 @@ ascli config ascp connect version 'Aspera Connect for Mac Intel' download enclos
|
|
2177
2375
|
```
|
2178
2376
|
|
2179
2377
|
```output
|
2180
|
-
Time: 00:00:02
|
2378
|
+
Time: 00:00:02 ============================================= 100% 27766 KB/sec Time: 00:00:02
|
2181
2379
|
Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
2182
2380
|
```
|
2183
2381
|
|
@@ -2192,7 +2390,7 @@ There are currently 3 agents, set with option `transfer`:
|
|
2192
2390
|
|
2193
2391
|
- [`direct`](#agt_direct) : a local execution of `ascp`
|
2194
2392
|
- [`connect`](#agt_connect) : use of a local Connect Client
|
2195
|
-
- [`node`](#agt_node) : use of an Aspera Transfer Node (potentially
|
2393
|
+
- [`node`](#agt_node) : use of an Aspera Transfer Node (potentially **remote**).
|
2196
2394
|
- [`httpgw`](#agt_httpgw) : use of an Aspera HTTP Gateway
|
2197
2395
|
- [`trsdk`](#agt_trsdk) : use of Aspera Transfer SDK
|
2198
2396
|
|
@@ -2200,7 +2398,7 @@ There are currently 3 agents, set with option `transfer`:
|
|
2200
2398
|
For example, a node agent executing an "upload", or "package send" operation
|
2201
2399
|
will effectively push files to the related server from the agent node.
|
2202
2400
|
|
2203
|
-
`ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of
|
2401
|
+
`ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
2204
2402
|
|
2205
2403
|
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2206
2404
|
|
@@ -2209,23 +2407,23 @@ Specific options for agents are provided with option `transfer_info`, cumulative
|
|
2209
2407
|
The `direct` agent directly executes a local `ascp`.
|
2210
2408
|
This is the default agent for `ascli`.
|
2211
2409
|
This is equivalent to option `--transfer=direct`.
|
2212
|
-
`ascli` will
|
2410
|
+
`ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
|
2213
2411
|
Refer to section [FASP](#client).
|
2214
2412
|
|
2215
2413
|
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
|
2216
2414
|
|
2217
2415
|
| Name | Type | Description |
|
2218
2416
|
|----------------------|-------|-------------|
|
2219
|
-
| wss
|
2220
|
-
| ascp_args
|
2221
|
-
| spawn_timeout_sec
|
2222
|
-
| spawn_delay_sec
|
2223
|
-
| multi_incr_udp
|
2224
|
-
| resume
|
2225
|
-
| resume.iter_max
|
2226
|
-
| resume.sleep_initial | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
|
2227
|
-
| resume.sleep_factor | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
|
2228
|
-
| resume.sleep_max
|
2417
|
+
| `wss` | Bool | Web Socket Session<br/>Enable use of web socket session in case it is available<br/>Default: true |
|
2418
|
+
| `ascp_args` | Array | Array of strings with native `ascp` arguments<br/>Use this instead of deprecated `EX_ascp_args`.<br/>Default: [] |
|
2419
|
+
| `spawn_timeout_sec` | Float | Multi session<br/>Verification time that `ascp` is running<br/>Default: 3 |
|
2420
|
+
| `spawn_delay_sec` | Float | Multi session<br/>Delay between startup of sessions<br/>Default: 2 |
|
2421
|
+
| `multi_incr_udp` | Bool | Multi Session<br/>Increment UDP port on multi-session<br/>If true, each session will have a different UDP port starting at `fasp_port` (or default 33001)<br/>Else, each session will use `fasp_port` (or `ascp` default)<br/>Default: true |
|
2422
|
+
| `resume` | Hash | Resume<br/>parameters<br/>See below |
|
2423
|
+
| `resume.iter_max` | int | Resume<br/>Max number of retry on error<br/>Default: 7 |
|
2424
|
+
| `resume.sleep_initial` | int | Resume<br/>First Sleep before retry<br/>Default: 2 |
|
2425
|
+
| `resume.sleep_factor` | int | Resume<br/>Multiplier of sleep period between attempts<br/>Default: 2 |
|
2426
|
+
| `resume.sleep_max` | int | Resume<br/>Default: 60 |
|
2229
2427
|
|
2230
2428
|
In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
|
2231
2429
|
Sleep between iterations is:
|
@@ -2234,7 +2432,7 @@ Sleep between iterations is:
|
|
2234
2432
|
max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
|
2235
2433
|
```
|
2236
2434
|
|
2237
|
-
Some transfer errors are considered
|
2435
|
+
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
2238
2436
|
The list of known protocol errors and retry level can be listed:
|
2239
2437
|
|
2240
2438
|
```bash
|
@@ -2252,8 +2450,14 @@ ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
|
|
2252
2450
|
But it is preferred to use the option `transfer_info` with parameter `ascp_args`.
|
2253
2451
|
|
2254
2452
|
This can be useful to activate logging using option `-L` of `ascp`.
|
2255
|
-
For example
|
2256
|
-
|
2453
|
+
For example, to activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`):
|
2454
|
+
|
2455
|
+
```bash
|
2456
|
+
--transfer-info=@json:'{"ascp_args":["-DDL-"]}'
|
2457
|
+
```
|
2458
|
+
|
2459
|
+
This is useful to debug if a transfer fails.
|
2460
|
+
|
2257
2461
|
To store `ascp` logs in file `aspera-scp-transfer.log` in a folder, use `--transfer-info=@json:'{"ascp_args":["-L","/path/to/folder"]}'`.
|
2258
2462
|
|
2259
2463
|
> **Note:** When transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided using `transfer_info` parameter `ascp_args`).
|
@@ -2356,26 +2560,34 @@ Parameters provided in option `transfer_info` are:
|
|
2356
2560
|
| root_id | string | password or secret</br>Mandatory only for bearer token |
|
2357
2561
|
|
2358
2562
|
Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
|
2359
|
-
|
2360
|
-
|
2563
|
+
|
2564
|
+
```bash
|
2565
|
+
--transfer-info=@preset:_name_here_
|
2566
|
+
```
|
2567
|
+
|
2568
|
+
or be specified using the extended value syntax :
|
2569
|
+
|
2570
|
+
```bash
|
2571
|
+
--transfer-info=@json:'{"url":"https://...","username":"_user_here_","password":"my_password_here"}'
|
2572
|
+
```
|
2361
2573
|
|
2362
2574
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
2363
2575
|
|
2364
|
-
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node.
|
2576
|
+
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node.
|
2577
|
+
It can be either the access key's root file id, or any authorized file id underneath it.
|
2365
2578
|
|
2366
2579
|
#### <a id="agt_httpgw"></a>HTTP Gateway
|
2367
2580
|
|
2368
|
-
If it possible to send using a HTTP gateway, in case FASP is not allowed.
|
2581
|
+
If it possible to send using a HTTP gateway, in case use of FASP is not allowed.
|
2369
2582
|
|
2370
2583
|
Parameters provided in option `transfer_info` are:
|
2371
2584
|
|
2372
2585
|
| Name | Type | Description |
|
2373
2586
|
|------------------------|--------|---------------------------------------|
|
2374
2587
|
| url | string | URL of the HTTP GW</br>Mandatory |
|
2375
|
-
|
|
2376
|
-
|
|
2377
|
-
|
|
2378
|
-
| synchronous | bool | wait for each message acknowledgment |
|
2588
|
+
| upload_chunk_size | int | Size in bytes of chunks for upload<br/>Default: 64000 |
|
2589
|
+
| api_version | string | v1 or v2, for force use of version<br/>Default: v2 |
|
2590
|
+
| synchronous | bool | wait for each message acknowledgment<br/>Default: false |
|
2379
2591
|
|
2380
2592
|
Example:
|
2381
2593
|
|
@@ -2388,8 +2600,17 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
|
|
2388
2600
|
#### <a id="agt_trsdk"></a>Transfer SDK
|
2389
2601
|
|
2390
2602
|
Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
|
2603
|
+
Set option `transfer` to `trsdk`.
|
2604
|
+
|
2605
|
+
Options for `transfer_info` are:
|
2606
|
+
|
2607
|
+
| Name | Type | Description |
|
2608
|
+
|----------|--------|-------------|
|
2609
|
+
| url | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: grpc://127.0.0.1:0 |
|
2610
|
+
| external | bool | Use external daemon, do not start<br/>Default: false |
|
2611
|
+
| keep | bool | Keep the daemon running after exiting `ascli`<br/>Default: false |
|
2391
2612
|
|
2392
|
-
|
2613
|
+
> **Note:** if port zero is specified in the URL, then the daemon will listen on a random available port.
|
2393
2614
|
|
2394
2615
|
The gem `grpc` was removed from dependencies, as it requires compilation of a native part.
|
2395
2616
|
So, to use the Transfer SDK you should install this gem:
|
@@ -2399,11 +2620,17 @@ gem install grpc
|
|
2399
2620
|
```
|
2400
2621
|
|
2401
2622
|
On Windows the compilation may fail for various reasons (3.1.1):
|
2623
|
+
|
2402
2624
|
<!-- spellchecker: disable -->
|
2625
|
+
|
2403
2626
|
- `cannot find -lx64-ucrt-ruby310`
|
2627
|
+
|
2404
2628
|
→ copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the dll extension)
|
2629
|
+
|
2405
2630
|
- `conflicting types for 'gettimeofday'`
|
2631
|
+
|
2406
2632
|
→ edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)` ,i.e. change `struct timezone` to `void`
|
2633
|
+
|
2407
2634
|
<!-- spellchecker: enable -->
|
2408
2635
|
|
2409
2636
|
### <a id="transferspec"></a>Transfer Specification
|
@@ -2417,13 +2644,12 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
|
|
2417
2644
|
- file list
|
2418
2645
|
- etc...
|
2419
2646
|
|
2420
|
-
`ascli` builds the [*transfer-spec*](#transferspec) internally
|
2421
|
-
|
2422
|
-
The [*transfer-spec*](#transferspec) is a Hash (dictionary), so it is described on the command line with the [Extended Value Syntax](#extended).
|
2647
|
+
`ascli` builds the [*transfer-spec*](#transferspec) internally as a `Hash`.
|
2648
|
+
It is not necessary to provide additional parameters on the command line for this transfer.
|
2423
2649
|
|
2424
2650
|
It is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option.
|
2425
|
-
The `ts` option accepts a [
|
2426
|
-
Multiple `ts` options on command line are cumulative, and Hash is deeply merged.
|
2651
|
+
The `ts` option accepts a [`Hash` Extended Value](#extended) containing one or several [*transfer-spec*](#transferspec) parameters.
|
2652
|
+
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
2427
2653
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
2428
2654
|
|
2429
2655
|
> **Note:** Default transfer spec values can be displayed with command: `config ascp info --flat-hash=no` under field `ts`.
|
@@ -2460,96 +2686,109 @@ Columns:
|
|
2460
2686
|
- D=Direct (local `ascp` execution)
|
2461
2687
|
- N=Node API
|
2462
2688
|
- C=Connect Client
|
2689
|
+
- T=Transfer SDK
|
2690
|
+
- H=HTTP Gateway
|
2463
2691
|
|
2464
2692
|
`ascp` argument or environment variable is provided in description.
|
2465
2693
|
|
2466
2694
|
Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
|
2467
2695
|
|
2468
|
-
| Field | Type | D | N | C | Description |
|
2469
|
-
| ----- | ---- | - | - | - | ----------- |
|
2470
|
-
| apply_local_docroot | bool | Y | | | (--apply-local-docroot) |
|
2471
|
-
| authentication | string | | | Y | value=token for SSH bypass keys, else password asked if not provided
|
2472
|
-
| cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
2473
|
-
| cipher_allowed | string | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none"
|
2474
|
-
|
|
2475
|
-
|
|
2476
|
-
|
|
2477
|
-
|
|
2478
|
-
|
|
2479
|
-
|
|
2480
|
-
|
|
2481
|
-
|
|
2482
|
-
|
|
2483
|
-
|
|
2484
|
-
|
|
2485
|
-
|
|
2486
|
-
|
|
2487
|
-
|
|
2488
|
-
|
|
2489
|
-
|
|
2490
|
-
|
|
2491
|
-
|
|
2492
|
-
|
|
2493
|
-
|
|
2494
|
-
|
|
2495
|
-
|
|
2496
|
-
|
|
2497
|
-
|
|
2498
|
-
|
|
2499
|
-
|
|
2500
|
-
|
|
2501
|
-
|
|
2502
|
-
|
|
2503
|
-
|
|
2504
|
-
|
|
2505
|
-
|
|
2506
|
-
|
|
2507
|
-
|
|
2508
|
-
|
|
2509
|
-
|
|
2510
|
-
|
|
2511
|
-
|
|
2512
|
-
|
|
2513
|
-
|
|
2514
|
-
|
|
2515
|
-
|
|
2516
|
-
|
|
2517
|
-
|
|
2518
|
-
|
|
2519
|
-
|
|
2520
|
-
|
|
2521
|
-
|
|
2522
|
-
|
|
2523
|
-
|
|
2524
|
-
|
|
2525
|
-
|
|
2526
|
-
|
|
2527
|
-
|
|
2528
|
-
|
|
2529
|
-
|
|
2530
|
-
|
|
2531
|
-
|
|
2532
|
-
|
|
2533
|
-
|
|
2534
|
-
|
|
2535
|
-
|
|
2536
|
-
|
|
2537
|
-
|
|
2538
|
-
|
|
2539
|
-
|
|
2540
|
-
|
|
2541
|
-
|
|
2542
|
-
|
|
2543
|
-
|
|
2544
|
-
|
|
2545
|
-
|
|
2546
|
-
|
|
2547
|
-
|
|
2548
|
-
|
|
2549
|
-
|
|
2550
|
-
|
|
2551
|
-
|
|
2552
|
-
|
|
2696
|
+
| Field | Type | D | N | C | T | H | Description |
|
2697
|
+
| ----- | ---- | - | - | - | - | - | ----------- |
|
2698
|
+
| apply_local_docroot | bool | Y | | | | | Apply local docroot to source paths.<br/>(--apply-local-docroot) |
|
2699
|
+
| authentication | string | | | Y | | | value=token for SSH bypass keys, else password asked if not provided.<br/>(<ignored>) |
|
2700
|
+
| cipher | string | Y | Y | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
2701
|
+
| cipher_allowed | string | Y | Y | Y | Y | Y | returned by node API. Valid literals include "aes-128" and "none".<br/>(<ignored>) |
|
2702
|
+
| compression | int | | | | | | ascp4 only, 0 / 1?<br/>(<ignored>) |
|
2703
|
+
| content_protection | string | Y | Y | Y | Y | Y | Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt {enum}) |
|
2704
|
+
| content_protection_password | string | Y | Y | Y | Y | Y | Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS) |
|
2705
|
+
| cookie | string | Y | Y | Y | Y | Y | Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE) |
|
2706
|
+
| create_dir | bool | Y | Y | Y | Y | Y | Specifies whether to create new directories.<br/>(-d) |
|
2707
|
+
| delete_before_transfer | bool | Y | Y | Y | Y | Y | Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer) |
|
2708
|
+
| delete_source | bool | Y | Y | | | | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
|
2709
|
+
| destination_root | string | Y | Y | Y | Y | Y | Destination root directory.<br/>(<special>) |
|
2710
|
+
| destination_root_id | string | | | | | | The file ID of the destination root directory.<br/>Required when using Bearer token auth for the destination node.<br/>(<ignored>) |
|
2711
|
+
| dgram_size | int | Y | Y | Y | Y | Y | UDP datagram size in bytes<br/>(-Z {int}) |
|
2712
|
+
| direction | string | Y | Y | Y | Y | Y | Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode (conversion){enum}) |
|
2713
|
+
| exclude_newer_than | int | Y | | | | | skip src files with mtime > arg<br/>(--exclude-newer-than {int}) |
|
2714
|
+
| exclude_older_than | int | Y | | | | | skip src files with mtime < arg<br/>(--exclude-older-than {int}) |
|
2715
|
+
| fasp_port | int | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
|
2716
|
+
| fasp_url | string | | | | | | Only used in Faspex.<br/>(<ignored>) |
|
2717
|
+
| file_checksum | string | Y | Y | | | | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(<ignored>) |
|
2718
|
+
| http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}|{string}) |
|
2719
|
+
| http_fallback_port | int | Y | | | | | Specifies http port when no cipher is used<br/>(-t {int}) |
|
2720
|
+
| https_fallback_port | int | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
|
2721
|
+
| keepalive | bool | Y | | | | | The session is running in persistent session mode.<br/>(--keepalive) |
|
2722
|
+
| lock_min_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2723
|
+
| lock_min_rate_kbps | bool | | | Y | | | If true, lock the minimum transfer rate to the value set for min_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
|
2724
|
+
| lock_rate_policy | bool | | | Y | | | If true, lock the rate policy to the default value.<br/>(<ignored>) |
|
2725
|
+
| lock_target_rate | bool | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2726
|
+
| lock_target_rate_kbps | bool | Y | Y | Y | Y | Y | If true, lock the target transfer rate to the default value set for target_rate_kbps.<br/>If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.<br/>(<ignored>) |
|
2727
|
+
| min_rate_cap_kbps | int | Y | Y | Y | Y | Y | The highest minimum rate that an incoming transfer can request, in kilobits per second.<br/>Client minimum rate requests that exceed the minimum rate cap are ignored.<br/>The default value of unlimited applies no cap to the minimum rate. (Default: 0)<br/>(<ignored>) |
|
2728
|
+
| min_rate_kbps | int | Y | Y | Y | Y | Y | Set the minimum transfer rate in kilobits per second.<br/>(-m {int}) |
|
2729
|
+
| move_after_transfer | string | Y | Y | | | | The relative path to which the files will be moved after the transfer at the source side. Available as of 3.8.0.<br/>(--move-after-transfer {string}) |
|
2730
|
+
| multi_session | int | Y | Y | Y | Y | Y | Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/><br/>(<special>) |
|
2731
|
+
| multi_session_threshold | int | Y | Y | | | | Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold {int}) |
|
2732
|
+
| obfuscate_file_names | bool | | | | | Y | HTTP Gateway obfuscates file names when set to true.<br/>(<ignored>) |
|
2733
|
+
| overwrite | string | Y | Y | Y | Y | Y | Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite {enum}) |
|
2734
|
+
| password | string | | Y | | | | Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.<br/>(<ignored>) |
|
2735
|
+
| paths | array | Y | Y | Y | Y | Y | Array of path to the source (required) and a path to the destination (optional).<br/>(<special>) |
|
2736
|
+
| precalculate_job_size | bool | Y | Y | Y | Y | Y | Specifies whether to precalculate the job size.<br/>(--precalculate-job-size) |
|
2737
|
+
| preserve_access_time | bool | Y | Y | Y | Y | Y | Preserve the source-file access timestamps at the destination.<br/>Because source access times are updated by the transfer operation,<br/>the timestamp that is preserved is the one just before to the transfer.<br/>(--preserve-access-time) |
|
2738
|
+
| preserve_acls | string | Y | | | | | Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls {enum}) |
|
2739
|
+
| preserve_creation_time | bool | Y | Y | Y | Y | Y | (Windows only) Preserve source-file creation timestamps at the destination.<br/>Only Windows systems retain information about creation time.<br/>If the destination is not a Windows computer, this option is ignored.<br/>(--preserve-creation-time) |
|
2740
|
+
| preserve_extended_attrs | string | | | | | | Preserve the extended attributes.<br/>Allowed values: none, native, metafile<br/>(--preserve-xattrs {enum}) |
|
2741
|
+
| preserve_file_owner_gid | bool | Y | | | | | Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid) |
|
2742
|
+
| preserve_file_owner_uid | bool | Y | | | | | Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid) |
|
2743
|
+
| preserve_modification_time | bool | Y | Y | Y | Y | Y | Set the modification time, the last time a file or directory was modified (written), of a transferred file<br/>to the modification of the source file or directory.<br/>Preserve source-file modification timestamps at the destination.<br/>(--preserve-modification-time) |
|
2744
|
+
| preserve_remote_acls | string | Y | | | | | Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls {enum}) |
|
2745
|
+
| preserve_source_access_time | bool | Y | | | | | Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time) |
|
2746
|
+
| preserve_times | bool | | Y | | | | Preserve file timestamps.<br/>(--preserve-times) |
|
2747
|
+
| proxy | string | Y | | | | | Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy {string}) |
|
2748
|
+
| rate_policy | string | Y | Y | Y | Y | Y | The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy {enum}) |
|
2749
|
+
| rate_policy_allowed | string | | | Y | | | Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed<br/>(<ignored>) |
|
2750
|
+
| read_threads | int | | | | | | ascp4 only<br/>(<ignored>) |
|
2751
|
+
| remote_access_key | string | | | | | | The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node.<br/>(<ignored>) |
|
2752
|
+
| remote_host | string | Y | Y | Y | Y | Y | IP or fully qualified domain name of the remote server<br/>(--host {string}) |
|
2753
|
+
| remote_password | string | Y | Y | Y | Y | Y | SSH session password<br/>(env:ASPERA_SCP_PASS) |
|
2754
|
+
| remote_user | string | Y | Y | Y | Y | Y | Remote user. Default value is "xfer" on node or connect.<br/>(--user {string}) |
|
2755
|
+
| remove_after_transfer | bool | Y | Y | | | | Remove SRC files after transfer success<br/>(--remove-after-transfer) |
|
2756
|
+
| remove_empty_directories | bool | Y | Y | | | | Specifies whether to remove empty directories.<br/>(--remove-empty-directories) |
|
2757
|
+
| remove_empty_source_directory | bool | Y | | | | | Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory) |
|
2758
|
+
| remove_skipped | bool | Y | Y | Y | | | Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped) |
|
2759
|
+
| resume_policy | string | Y | Y | Y | Y | Y | If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k (conversion){enum}) |
|
2760
|
+
| retry_duration | string<br/>int | | Y | Y | | | Specifies how long to wait before retrying transfer. (e.g. "5min")<br/>(<ignored>) |
|
2761
|
+
| source_root | string | Y | Y | Y | Y | Y | Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64 (conversion){string}) |
|
2762
|
+
| source_root_id | string | | Y | | | | The file ID of the source root directory. Required when using Bearer token auth for the source node.<br/>(<ignored>) |
|
2763
|
+
| src_base | string | Y | Y | | | | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
|
2764
|
+
| ssh_args | string | | | | | | Array of arguments to pass to SSH. Use with caution.<br/>(<ignored>) |
|
2765
|
+
| ssh_port | int | Y | Y | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
|
2766
|
+
| ssh_private_key | string | Y | | | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
|
2767
|
+
| ssh_private_key_passphrase | string | Y | | | | | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
|
2768
|
+
| sshfp | string | Y | Y | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
|
2769
|
+
| symlink_policy | string | Y | Y | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
|
2770
|
+
| tags | hash | Y | Y | Y | Y | Y | Metadata for transfer as JSON<br/>(--tags64 (conversion){hash}) |
|
2771
|
+
| target_rate_cap_kbps | int | | | Y | | | Returned by upload/download_setup node API.<br/>(<ignored>) |
|
2772
|
+
| target_rate_kbps | int | Y | Y | Y | Y | Y | Specifies desired speed for the transfer.<br/>(-l {int}) |
|
2773
|
+
| target_rate_percentage | string | Y | Y | Y | Y | Y | TODO: remove ?<br/>(<ignored>) |
|
2774
|
+
| title | string | | Y | Y | | | Title of the transfer<br/>(<ignored>) |
|
2775
|
+
| token | string | Y | Y | Y | Y | Y | Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN) |
|
2776
|
+
| use_ascp4 | bool | Y | Y | | | | specify version of protocol<br/>(<special>) |
|
2777
|
+
| use_system_ssh | string | | | | | | TODO, comment...<br/>(<ignored>) |
|
2778
|
+
| write_threads | int | | | | | | ascp4 only<br/>(<ignored>) |
|
2779
|
+
| wss_enabled | bool | Y | Y | Y | Y | Y | Server has Web Socket service enabled<br/>(<special>) |
|
2780
|
+
| wss_port | int | Y | Y | Y | Y | Y | TCP port used for websocket service feed<br/>(<special>) |
|
2781
|
+
| EX_ascp_args | array | Y | | | | | DEPRECATED: (4.13) Use option transfer_info.ascp_args<br/>Add native command line arguments to ascp<br/>(<special>) |
|
2782
|
+
| EX_at_rest_password | string | Y | | | | | DEPRECATED: (4.13) Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2783
|
+
| EX_file_list | string | Y | | | | | DEPRECATED: (4.14) Use command line file list, or option transfer_info.ascp_args<br/>source file list<br/>(<special>) |
|
2784
|
+
| EX_file_pair_list | string | Y | | | | | DEPRECATED: (4.14) Use command line file pair list, or option transfer_info.ascp_args<br/>source file pair list<br/>(<special>) |
|
2785
|
+
| EX_http_proxy_url | string | Y | | | | | DEPRECATED: (4.14) TODO, use proxy option ?<br/>Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2786
|
+
| EX_http_transfer_jpeg | int | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>HTTP transfers as JPEG file<br/>(-j {int}) |
|
2787
|
+
| EX_license_text | string | Y | | | | | DEPRECATED: (4.14) Use env var ASPERA_SCP_LICENSE<br/>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2788
|
+
| EX_no_read | bool | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no read source<br/>(--no-read) |
|
2789
|
+
| EX_no_write | bool | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>no write on destination<br/>(--no-write) |
|
2790
|
+
| EX_proxy_password | string | Y | | | | | DEPRECATED: (4.14) Use env var ASPERA_PROXY_PASS<br/>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL provided in parameter: proxy.<br/>(env:ASPERA_PROXY_PASS) |
|
2791
|
+
| EX_ssh_key_paths | array | Y | | | | | DEPRECATED: (4.14) Use option transfer_info.ascp_args<br/>Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2553
2792
|
|
2554
2793
|
#### Destination folder for transfers
|
2555
2794
|
|
@@ -2658,7 +2897,7 @@ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
|
2658
2897
|
|
2659
2898
|
#### Source directory structure on destination
|
2660
2899
|
|
2661
|
-
This section is not specific to `ascli
|
2900
|
+
This section is not specific to `ascli` it is `ascp` behavior.
|
2662
2901
|
|
2663
2902
|
The transfer destination is normally expected to designate a destination folder.
|
2664
2903
|
|
@@ -2669,7 +2908,7 @@ But there is one exception: The destination specifies the new item name when the
|
|
2669
2908
|
- destination is not an existing folder
|
2670
2909
|
- the `dirname` of destination is an existing folder
|
2671
2910
|
|
2672
|
-
For this reason it is recommended to set `create_dir` to `true` for consistent
|
2911
|
+
For this reason it is recommended to set `create_dir` to `true` for consistent behavior between single and multiple items transfer, this is the default in `ascli`.
|
2673
2912
|
|
2674
2913
|
If a simple source file list is provided (no `destination` in `paths`, i.e. no `file_pair_list` provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.
|
2675
2914
|
|
@@ -2702,9 +2941,15 @@ If transfer spec has a `src_base`, it has the side effect that the simple source
|
|
2702
2941
|
|
2703
2942
|
Advanced Example: Send files `./file1` and `./folder2/files2` to server (e.g. `/Upload`) and keep the original file names and folders, i.e. send `file1` to `/Upload/file1` and `files2` to `/Upload/folder2/files2`.
|
2704
2943
|
|
2705
|
-
- If files are specified as `./file1 ./folder2/files2`,
|
2944
|
+
- If files are specified as `./file1 ./folder2/files2`,
|
2945
|
+
|
2946
|
+
then destination will be: `/Upload/file1 /Upload/files2`
|
2947
|
+
|
2706
2948
|
- One possibility is to specify a file pair list: `--src-type=pair file1 file1 folder2/files2 folder2/files2`
|
2707
|
-
- Another possibility is to specify a source base: `--src-base=$PWD $PWD/file1 $PWD/folder2/files2`
|
2949
|
+
- Another possibility is to specify a source base: `--src-base=$PWD $PWD/file1 $PWD/folder2/files2`
|
2950
|
+
|
2951
|
+
(note that `.` cannot be used as source base)
|
2952
|
+
|
2708
2953
|
- Similarly, create a temporary soft link (Linux): `ln -s . tmp_base` and use `--src-base=tmp_base tmp_base/file1 tmp_base/folder2/files2`
|
2709
2954
|
- One can also similarly use `--sources=@ts` and specify the list of files in the `paths` field of transfer spec with both `source` and `destination` for each file.
|
2710
2955
|
|
@@ -2726,7 +2971,7 @@ Multi-session is directly supported by the node daemon.
|
|
2726
2971
|
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
|
2727
2972
|
```
|
2728
2973
|
|
2729
|
-
Note
|
2974
|
+
> **Note:** `resume_policy` set to `attr` may cause problems: `none` or `sparse_csum` shall be preferred.
|
2730
2975
|
|
2731
2976
|
`ascli` starts multiple `ascp` for Multi-session using `direct` agent.
|
2732
2977
|
|
@@ -2774,6 +3019,14 @@ Example: parameter to download a faspex package and decrypt on the fly
|
|
2774
3019
|
--ts=@json:'{"precalculate_job_size":true}'
|
2775
3020
|
```
|
2776
3021
|
|
3022
|
+
### Transfer progress bar
|
3023
|
+
|
3024
|
+
File transfer operations are monitored and a progress bar is displayed on the terminal if option `progress_bar` (`Bool`) is set to `yes` (default if the output is a terminal).
|
3025
|
+
|
3026
|
+
The same progress bar is used for any type of transfer, using `ascp`, server to server, using HTTPS, etc...
|
3027
|
+
|
3028
|
+
To display the native progress bar of `ascp`, use `--progress-bar=no --transfer-info=@json:'{"quiet":false}'`.
|
3029
|
+
|
2777
3030
|
### <a id="scheduler"></a>Scheduler
|
2778
3031
|
|
2779
3032
|
It is useful to configure automated scheduled execution.
|
@@ -2787,7 +3040,7 @@ It can be configured:
|
|
2787
3040
|
|
2788
3041
|
- Using utility [`schtasks.exe`](https://learn.microsoft.com/fr-fr/windows-server/administration/windows-commands/schtasks-create)
|
2789
3042
|
|
2790
|
-
- Using powershell function [scheduletasks](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
3043
|
+
- Using powershell function [`scheduletasks`](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
2791
3044
|
|
2792
3045
|
- Using `taskschd.msc` (UI)
|
2793
3046
|
|
@@ -2819,7 +3072,7 @@ crontab<<EOF
|
|
2819
3072
|
EOF
|
2820
3073
|
```
|
2821
3074
|
|
2822
|
-
> **Note:** The logging options are kept here in the
|
3075
|
+
> **Note:** The logging options are kept here in the cron file instead of conf file to allow execution on command line with output on command line.
|
2823
3076
|
|
2824
3077
|
### <a id="locking"></a>Locking for exclusive execution
|
2825
3078
|
|
@@ -2857,37 +3110,32 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
|
|
2857
3110
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
2858
3111
|
```
|
2859
3112
|
|
2860
|
-
### "Provenç
|
3113
|
+
### "Provençal"
|
2861
3114
|
|
2862
|
-
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others.
|
2863
|
-
|
3115
|
+
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others.
|
3116
|
+
This mechanism is called **PVCL** (from **Provençal**, a restaurant located in Sophia Antipolis).
|
3117
|
+
Several **PVCL** adapters are available, one is embedded in `ascp`, the others are provided in shared libraries and must be activated.
|
2864
3118
|
|
2865
|
-
The list of supported
|
3119
|
+
The list of supported **PVCL** adapters can be retrieved with command:
|
2866
3120
|
|
2867
3121
|
```bash
|
2868
|
-
ascli conf ascp info
|
3122
|
+
ascli conf ascp info --fields=@re:'^pvcl'
|
2869
3123
|
```
|
2870
3124
|
|
2871
3125
|
```output
|
2872
|
-
|
2873
|
-
|
2874
|
-
|
2875
|
-
|
2876
|
-
|
2877
|
-
|
2878
|
-
|
2879
|
-
| shares | pvcl |
|
2880
|
-
| noded | pvcl |
|
2881
|
-
| faux | pvcl |
|
2882
|
-
| file | pvcl |
|
2883
|
-
| stdio | pvcl |
|
2884
|
-
| stdio-tar | pvcl |
|
2885
|
-
+--------------------+-----------------------------------------------------------+
|
3126
|
+
process v1
|
3127
|
+
shares v1
|
3128
|
+
noded v1
|
3129
|
+
faux v1
|
3130
|
+
file v1
|
3131
|
+
stdio v1
|
3132
|
+
stdio-tar v1
|
2886
3133
|
```
|
2887
3134
|
|
2888
3135
|
Here we can see the adapters: `process`, `shares`, `noded`, `faux`, `file`, `stdio`, `stdio-tar`.
|
2889
3136
|
|
2890
|
-
Those adapters can be used wherever a file path is used in `ascp` including configuration.
|
3137
|
+
Those adapters can be used wherever a file path is used in `ascp` including configuration.
|
3138
|
+
They act as a pseudo "drive".
|
2891
3139
|
|
2892
3140
|
The simplified format is:
|
2893
3141
|
|
@@ -2895,7 +3143,8 @@ The simplified format is:
|
|
2895
3143
|
<adapter>:///<sub file path>?<arg1>=<val1>&...
|
2896
3144
|
```
|
2897
3145
|
|
2898
|
-
One of the adapters, used in this manual, for testing, is `faux`.
|
3146
|
+
One of the adapters, used in this manual, for testing, is `faux`.
|
3147
|
+
It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
|
2899
3148
|
|
2900
3149
|
### <a id="faux_testing"></a>`faux:` for testing
|
2901
3150
|
|
@@ -2917,7 +3166,7 @@ where:
|
|
2917
3166
|
- `filename` is the name that will be assigned to the file on the destination
|
2918
3167
|
- `filesize` is the number of bytes that will be sent (in decimal).
|
2919
3168
|
|
2920
|
-
Note
|
3169
|
+
> **Note:** characters `?` and `&` are shell special characters (wildcard and background), so `faux` file specification on command line should be protected (using quotes or `\`). If not, the shell may give error: `no matches found` or equivalent.
|
2921
3170
|
|
2922
3171
|
For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2<sup>20</sup>, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2<sup>60</sup>. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is `faux://`.
|
2923
3172
|
|
@@ -2961,7 +3210,7 @@ Filenames generated are of the form: `<file>_<00000 ... count>_<filesize>`
|
|
2961
3210
|
|
2962
3211
|
Examples:
|
2963
3212
|
|
2964
|
-
- Upload 20
|
3213
|
+
- Upload 20 gibibyte of random data to file `myfile` to directory /Upload
|
2965
3214
|
|
2966
3215
|
```bash
|
2967
3216
|
ascli server upload faux:///myfile\?20g --to-folder=/Upload
|
@@ -2984,7 +3233,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
2984
3233
|
```text
|
2985
3234
|
ascli -h
|
2986
3235
|
NAME
|
2987
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
3236
|
+
ascli -- a command line tool for Aspera Applications (v4.15.0)
|
2988
3237
|
|
2989
3238
|
SYNOPSIS
|
2990
3239
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -2997,7 +3246,6 @@ DESCRIPTION
|
|
2997
3246
|
source repo: https://github.com/IBM/aspera-cli
|
2998
3247
|
|
2999
3248
|
ENVIRONMENT VARIABLES
|
3000
|
-
ASCLI_HOME config folder, default: $HOME/.aspera/ascli
|
3001
3249
|
Any option can be set as an environment variable, refer to the manual
|
3002
3250
|
|
3003
3251
|
COMMANDS
|
@@ -3007,7 +3255,7 @@ COMMANDS
|
|
3007
3255
|
OPTIONS
|
3008
3256
|
Options begin with a '-' (minus), and value is provided on command line.
|
3009
3257
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
3010
|
-
base64, csvt, env, file, json, lines, list, path, ruby, secret, stdin,
|
3258
|
+
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
|
3011
3259
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
3012
3260
|
|
3013
3261
|
ARGS
|
@@ -3018,8 +3266,8 @@ OPTIONS: global
|
|
3018
3266
|
--ask-options=ENUM Ask even optional options: [no], yes
|
3019
3267
|
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
|
3020
3268
|
--display=ENUM Output only some information: [info], data, error
|
3021
|
-
--fields=VALUE Comma separated list of fields, or ALL, or DEF
|
3022
|
-
--select=VALUE Select only some items in lists: column, value (Hash)
|
3269
|
+
--fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
|
3270
|
+
--select=VALUE Select only some items in lists: column, value (Hash, Proc)
|
3023
3271
|
--table-style=VALUE Table display style
|
3024
3272
|
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
3025
3273
|
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
@@ -3027,89 +3275,93 @@ OPTIONS: global
|
|
3027
3275
|
-h, --help Show this message
|
3028
3276
|
--bash-comp Generate bash completion for command
|
3029
3277
|
--show-config Display parameters used for the provided action
|
3030
|
-
-r, --rest-debug More debug for HTTP calls (REST)
|
3031
3278
|
-v, --version Display version
|
3032
3279
|
-w, --warnings Check for language warnings
|
3033
3280
|
--ui=ENUM Method to start browser: text, [graphical]
|
3034
|
-
--log-level=ENUM Log level: debug, info, [warn], error, fatal, unknown
|
3281
|
+
--log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
|
3035
3282
|
--logger=ENUM Logging method: [stderr], stdout, syslog
|
3036
|
-
--lock-port=VALUE Prevent dual execution of a command, e.g. in cron
|
3037
|
-
--http-options=VALUE Options for http socket (Hash)
|
3038
|
-
--insecure=ENUM Do not validate HTTPS certificate: [no], yes
|
3283
|
+
--lock-port=VALUE Prevent dual execution of a command, e.g. in cron (Integer)
|
3039
3284
|
--once-only=ENUM Process only new items (some commands): [no], yes
|
3040
3285
|
--log-secrets=ENUM Show passwords in logs: [no], yes
|
3041
|
-
--
|
3286
|
+
--clean-temp=ENUM Cleanup temporary files on exit: no, [yes]
|
3287
|
+
--pid-file=VALUE Write process identifier to file, delete on exit (String)
|
3042
3288
|
|
3043
3289
|
COMMAND: config
|
3044
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open
|
3290
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check remote_certificate smtp_settings throw vault wizard
|
3045
3291
|
OPTIONS:
|
3292
|
+
--home=VALUE Home folder for tool (String)
|
3293
|
+
--config-file=VALUE Path to YAML file with preset configuration
|
3294
|
+
--secret=VALUE Secret for access keys
|
3295
|
+
--vault=VALUE Vault for secrets (Hash)
|
3296
|
+
--vault-password=VALUE Vault password
|
3046
3297
|
--query=VALUE Additional filter for for some commands (list/delete) (Hash)
|
3047
|
-
--value=VALUE Value for create, update, list filter (Hash) (deprecated: Use positional value for create/modify or option: query for list/delete)
|
3298
|
+
--value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
|
3048
3299
|
--property=VALUE Name of property to set (modify operation)
|
3049
|
-
--id=VALUE Resource identifier (deprecated: Use identifier after verb (modify,delete,show))
|
3300
|
+
--id=VALUE Resource identifier (deprecated: (4.14) Use positional identifier after verb (modify,delete,show))
|
3050
3301
|
--bulk=ENUM Bulk operation (only some): [no], yes
|
3051
3302
|
--bfail=ENUM Bulk operation error handling: no, [yes]
|
3052
|
-
--config-file=VALUE Read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
|
3053
3303
|
-N, --no-default Do not load default configuration for plugin
|
3304
|
+
-P, --presetVALUE Load the named option preset from current config file
|
3305
|
+
--version-check-days=VALUE Period in days to check new version (zero to disable)
|
3306
|
+
--plugin-folder=VALUE Folder where to find additional plugins
|
3054
3307
|
--override=ENUM Wizard: override existing value: [no], yes
|
3055
|
-
--use-generic-client=ENUM Wizard: AoC: use global or org specific jwt client id: no, [yes]
|
3056
3308
|
--default=ENUM Wizard: set as default configuration for specified plugin (also: update): no, [yes]
|
3057
3309
|
--test-mode=ENUM Wizard: skip private key check step: [no], yes
|
3058
|
-
|
3059
|
-
--pkeypath=VALUE Wizard: path to private key for JWT
|
3310
|
+
--key-path=VALUE Wizard: path to private key for JWT
|
3060
3311
|
--ascp-path=VALUE Path to ascp
|
3061
3312
|
--use-product=VALUE Use ascp from specified product
|
3062
|
-
--smtp=VALUE SMTP configuration (Hash)
|
3063
|
-
--fpac=VALUE Proxy auto configuration script
|
3064
|
-
--proxy-credentials=VALUE HTTP proxy credentials (Array with user and password)
|
3065
|
-
--secret=VALUE Secret for access keys
|
3066
|
-
--vault=VALUE Vault for secrets
|
3067
|
-
--vault-password=VALUE Vault password
|
3068
3313
|
--sdk-url=VALUE URL to get SDK
|
3069
3314
|
--sdk-folder=VALUE SDK folder path
|
3070
|
-
--
|
3071
|
-
--
|
3072
|
-
--
|
3073
|
-
--
|
3315
|
+
--progress-bar=ENUM Display progress bar: [no], yes
|
3316
|
+
--smtp=VALUE SMTP configuration (Hash)
|
3317
|
+
--notify-to=VALUE Email recipient for notification of transfers
|
3318
|
+
--notify-template=VALUE Email ERB template for notification of transfers
|
3319
|
+
--insecure=ENUM Do not validate any HTTPS certificate: [no], yes
|
3320
|
+
--ignore-certificate=VALUE List of HTTPS url where to no validate certificate (Array)
|
3321
|
+
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
3322
|
+
--http-options=VALUE Options for HTTP/S socket (Hash)
|
3323
|
+
--cache-tokens=ENUM Save and reuse Oauth tokens: no, [yes]
|
3324
|
+
--fpac=VALUE Proxy auto configuration script
|
3325
|
+
--proxy-credentials=VALUE HTTP proxy credentials (user and password) (Array)
|
3074
3326
|
--ts=VALUE Override transfer spec values (Hash)
|
3075
3327
|
--to-folder=VALUE Destination folder for transferred files
|
3076
3328
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
3077
3329
|
--src-type=ENUM Type of file list: [list], pair
|
3078
3330
|
--transfer=ENUM Type of transfer agent: [direct], node, connect, httpgw, trsdk
|
3079
3331
|
--transfer-info=VALUE Parameters for transfer agent (Hash)
|
3080
|
-
--progress=ENUM Type of progress bar: none, [native], multi
|
3081
3332
|
|
3082
3333
|
|
3083
3334
|
COMMAND: shares
|
3084
3335
|
SUBCOMMANDS: admin files health
|
3085
3336
|
OPTIONS:
|
3086
|
-
--url=VALUE URL of application, e.g. https://
|
3337
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3087
3338
|
--username=VALUE Username to log in
|
3088
3339
|
--password=VALUE User's password
|
3089
3340
|
--type=ENUM Type of user/group for operations: [any], local, ldap, saml
|
3090
3341
|
|
3091
3342
|
|
3092
3343
|
COMMAND: node
|
3093
|
-
SUBCOMMANDS:
|
3344
|
+
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service slash space ssync stream sync transfer upload watch_folder
|
3094
3345
|
OPTIONS:
|
3095
|
-
--url=VALUE URL of application, e.g. https://
|
3346
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3096
3347
|
--username=VALUE Username to log in
|
3097
3348
|
--password=VALUE User's password
|
3098
3349
|
--validator=VALUE Identifier of validator (optional for central)
|
3099
3350
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
3100
3351
|
--sync-name=VALUE Sync name
|
3101
3352
|
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
3353
|
+
--root-id=VALUE File id of top folder if using bearer tokens
|
3354
|
+
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3102
3355
|
|
3103
3356
|
|
3104
3357
|
COMMAND: orchestrator
|
3105
3358
|
SUBCOMMANDS: health info plugins processes workflow
|
3106
3359
|
OPTIONS:
|
3107
|
-
--url=VALUE URL of application, e.g. https://
|
3360
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3108
3361
|
--username=VALUE Username to log in
|
3109
3362
|
--password=VALUE User's password
|
3110
|
-
--
|
3111
|
-
--
|
3112
|
-
--synchronous=ENUM Work step:parameter expected as result: [no], yes
|
3363
|
+
--result=VALUE Specify result value as: 'work_step:parameter'
|
3364
|
+
--synchronous=ENUM Wait for completion: [no], yes
|
3113
3365
|
--ret-style=ENUM How return type is requested in api: header, [arg], ext
|
3114
3366
|
--auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
|
3115
3367
|
|
@@ -3117,7 +3369,7 @@ OPTIONS:
|
|
3117
3369
|
COMMAND: bss
|
3118
3370
|
SUBCOMMANDS: subscription
|
3119
3371
|
OPTIONS:
|
3120
|
-
--url=VALUE URL of application, e.g. https://
|
3372
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3121
3373
|
--username=VALUE Username to log in
|
3122
3374
|
--password=VALUE User's password
|
3123
3375
|
|
@@ -3125,7 +3377,7 @@ OPTIONS:
|
|
3125
3377
|
COMMAND: alee
|
3126
3378
|
SUBCOMMANDS: entitlement
|
3127
3379
|
OPTIONS:
|
3128
|
-
--url=VALUE URL of application, e.g. https://
|
3380
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3129
3381
|
--username=VALUE Username to log in
|
3130
3382
|
--password=VALUE User's password
|
3131
3383
|
|
@@ -3145,19 +3397,18 @@ OPTIONS:
|
|
3145
3397
|
COMMAND: faspex5
|
3146
3398
|
SUBCOMMANDS: admin bearer_token gateway health packages postprocessing shared_folders user version
|
3147
3399
|
OPTIONS:
|
3148
|
-
--url=VALUE URL of application, e.g. https://
|
3400
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3149
3401
|
--username=VALUE Username to log in
|
3150
3402
|
--password=VALUE User's password
|
3151
3403
|
--client-id=VALUE OAuth client identifier
|
3152
3404
|
--client-secret=VALUE OAuth client secret
|
3153
3405
|
--redirect-uri=VALUE OAuth redirect URI for web authentication
|
3154
|
-
--auth=ENUM OAuth type of authentication: boot,
|
3406
|
+
--auth=ENUM OAuth type of authentication: boot, web, [jwt]
|
3155
3407
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3156
3408
|
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
3157
|
-
--link=VALUE Public link authorization (specific operations)
|
3158
3409
|
--box=VALUE Package inbox, either shared inbox name or one of ["inbox", "inbox_history", "inbox_all", "inbox_all_history", "outbox", "outbox_history", "pending", "pending_history", "all"] or ALL
|
3159
3410
|
--shared-folder=VALUE Send package with files from shared folder
|
3160
|
-
--group-type=ENUM
|
3411
|
+
--group-type=ENUM Type of shared box: [shared_inboxes], workgroups
|
3161
3412
|
|
3162
3413
|
|
3163
3414
|
COMMAND: cos
|
@@ -3175,21 +3426,21 @@ OPTIONS:
|
|
3175
3426
|
COMMAND: faspex
|
3176
3427
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
3177
3428
|
OPTIONS:
|
3178
|
-
--url=VALUE URL of application, e.g. https://
|
3429
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3179
3430
|
--username=VALUE Username to log in
|
3180
3431
|
--password=VALUE User's password
|
3181
3432
|
--link=VALUE Public link for specific operation
|
3182
3433
|
--delivery-info=VALUE Package delivery information (Hash)
|
3183
|
-
--source
|
3184
|
-
--storage=VALUE Faspex local storage definition
|
3434
|
+
--remote-source=VALUE Remote source for package send (id or %name:)
|
3435
|
+
--storage=VALUE Faspex local storage definition (for browsing source)
|
3185
3436
|
--recipient=VALUE Use if recipient is a dropbox (with *)
|
3186
3437
|
--box=ENUM Package box: [inbox], archive, sent
|
3187
3438
|
|
3188
3439
|
|
3189
3440
|
COMMAND: preview
|
3190
|
-
SUBCOMMANDS: check events scan test trevents
|
3441
|
+
SUBCOMMANDS: check events scan show test trevents
|
3191
3442
|
OPTIONS:
|
3192
|
-
--url=VALUE URL of application, e.g. https://
|
3443
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3193
3444
|
--username=VALUE Username to log in
|
3194
3445
|
--password=VALUE User's password
|
3195
3446
|
--skip-format=ENUM Skip this preview format (multiple possible): png, mp4
|
@@ -3198,7 +3449,7 @@ OPTIONS:
|
|
3198
3449
|
--previews-folder=VALUE Preview folder in storage root
|
3199
3450
|
--temp-folder=VALUE Path to temp folder
|
3200
3451
|
--skip-folders=VALUE List of folder to skip
|
3201
|
-
--
|
3452
|
+
--base=VALUE Basename of output for for test
|
3202
3453
|
--scan-path=VALUE Subpath in folder id to start scan in (default=/)
|
3203
3454
|
--scan-id=VALUE Folder id in storage to start scan in, default is access key main folder id
|
3204
3455
|
--mimemagic=ENUM Use Mime type detection of gem mimemagic: [no], yes
|
@@ -3222,58 +3473,40 @@ OPTIONS:
|
|
3222
3473
|
--clips-length=VALUE Mp4: clips: length in seconds of each clips
|
3223
3474
|
|
3224
3475
|
|
3225
|
-
COMMAND: sync
|
3226
|
-
SUBCOMMANDS: admin start
|
3227
|
-
OPTIONS:
|
3228
|
-
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3229
|
-
--sync-session=VALUE Name of session to use for admin commands. default: first in parameters
|
3230
|
-
|
3231
|
-
|
3232
3476
|
COMMAND: aoc
|
3233
3477
|
SUBCOMMANDS: admin automation bearer_token files gateway organization packages reminder servers tier_restrictions user
|
3234
3478
|
OPTIONS:
|
3235
|
-
--url=VALUE URL of application, e.g. https://
|
3479
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3236
3480
|
--username=VALUE Username to log in
|
3237
3481
|
--password=VALUE User's password
|
3238
3482
|
--auth=ENUM OAuth type of authentication: web, [jwt]
|
3239
|
-
--operation=ENUM Client operation for transfers: [push], pull
|
3240
3483
|
--client-id=VALUE OAuth API client identifier
|
3241
3484
|
--client-secret=VALUE OAuth API client secret
|
3485
|
+
--scope=VALUE OAuth scope for AoC API calls
|
3242
3486
|
--redirect-uri=VALUE OAuth API client redirect URI
|
3243
3487
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
3244
|
-
--scope=VALUE OAuth scope for AoC API calls
|
3245
3488
|
--passphrase=VALUE RSA private key passphrase
|
3246
|
-
--workspace=VALUE Name of workspace
|
3247
|
-
--name=VALUE Resource name (prefer to use keyword name)
|
3248
|
-
--link=VALUE Public link to shared resource
|
3489
|
+
--workspace=VALUE Name of workspace (String, NilClass)
|
3249
3490
|
--new-user-option=VALUE New user creation option for unknown package recipients
|
3250
|
-
--from-folder=VALUE Source folder for Folder-to-Folder transfer
|
3251
3491
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
3252
3492
|
|
3253
|
-
COMMAND: node
|
3254
|
-
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
|
3255
|
-
OPTIONS:
|
3256
|
-
--validator=VALUE Identifier of validator (optional for central)
|
3257
|
-
--asperabrowserurl=VALUE URL for simple aspera web ui
|
3258
|
-
--sync-name=VALUE Sync name
|
3259
|
-
--default-ports=ENUM Use standard FASP ports or get from node api (gen4): no, [yes]
|
3260
|
-
|
3261
3493
|
|
3262
3494
|
COMMAND: server
|
3263
3495
|
SUBCOMMANDS: browse cp delete df download du health info ls md5sum mkdir mv rename rm sync upload
|
3264
3496
|
OPTIONS:
|
3265
|
-
--url=VALUE URL of application, e.g. https://
|
3497
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3266
3498
|
--username=VALUE Username to log in
|
3267
3499
|
--password=VALUE User's password
|
3268
3500
|
--ssh-keys=VALUE SSH key path list (Array or single)
|
3269
3501
|
--passphrase=VALUE SSH private key passphrase
|
3270
3502
|
--ssh-options=VALUE SSH options (Hash)
|
3503
|
+
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
3271
3504
|
|
3272
3505
|
|
3273
3506
|
COMMAND: console
|
3274
3507
|
SUBCOMMANDS: health transfer
|
3275
3508
|
OPTIONS:
|
3276
|
-
--url=VALUE URL of application, e.g. https://
|
3509
|
+
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
3277
3510
|
--username=VALUE Username to log in
|
3278
3511
|
--password=VALUE User's password
|
3279
3512
|
--filter-from=DATE Only after date
|
@@ -3286,19 +3519,74 @@ OPTIONS:
|
|
3286
3519
|
|
3287
3520
|
### Bulk creation and deletion of resources
|
3288
3521
|
|
3289
|
-
Bulk creation and deletion of resources are possible using option `bulk` (yes
|
3290
|
-
In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
|
3522
|
+
Bulk creation and deletion of resources are possible using option `bulk` (`yes`,`no`(default)).
|
3523
|
+
In that case, the operation expects an `Array` of `Hash` instead of a simple `Hash` using the [Extended Value Syntax](#extended).
|
3291
3524
|
This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
|
3292
3525
|
|
3526
|
+
### Plugins
|
3527
|
+
|
3528
|
+
`ascli` uses a plugin mechanism.
|
3529
|
+
The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
|
3530
|
+
Each plugin usually represents commands sent to a specific application.
|
3531
|
+
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
3532
|
+
|
3533
|
+
Available plugins can be found using command:
|
3534
|
+
|
3535
|
+
```bash
|
3536
|
+
ascli conf plugin list
|
3537
|
+
```
|
3538
|
+
|
3539
|
+
```output
|
3540
|
+
+--------------+--------+--------+-------------------------------------------------------+
|
3541
|
+
| plugin | detect | wizard | path |
|
3542
|
+
+--------------+--------+--------+-------------------------------------------------------+
|
3543
|
+
| shares | Y | Y | .../aspera-cli/lib/aspera/cli/plugins/shares.rb |
|
3544
|
+
| node | Y | Y | .../aspera-cli/lib/aspera/cli/plugins/node.rb |
|
3545
|
+
...
|
3546
|
+
+--------------+--------+--------+-------------------------------------------------------+
|
3547
|
+
```
|
3548
|
+
|
3549
|
+
Most plugins will take the URL option: `url` to identify their location.
|
3550
|
+
|
3551
|
+
REST APIs of Aspera legacy applications (Aspera Node, Faspex 4, Shares, Console, Orchestrator) use simple username/password authentication: HTTP Basic Authentication using options: `username` and `password`.
|
3552
|
+
|
3553
|
+
Aspera on Cloud and Faspex 5 rely on Oauth.
|
3554
|
+
|
3555
|
+
By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
3556
|
+
|
3557
|
+
```bash
|
3558
|
+
ascli --show-config --select=@json:'{"key":"plugin_folder"}'
|
3559
|
+
```
|
3560
|
+
|
3561
|
+
You can create the skeleton of a new plugin like this:
|
3562
|
+
|
3563
|
+
```bash
|
3564
|
+
ascli conf plugin create foo .
|
3565
|
+
```
|
3566
|
+
|
3567
|
+
```output
|
3568
|
+
Created ./foo.rb
|
3569
|
+
```
|
3570
|
+
|
3571
|
+
```bash
|
3572
|
+
ascli --plugin-folder=. foo
|
3573
|
+
```
|
3574
|
+
|
3293
3575
|
## <a id="aoc"></a>Plugin: `aoc`: IBM Aspera on Cloud
|
3294
3576
|
|
3295
|
-
Aspera on Cloud
|
3577
|
+
Aspera on Cloud API requires the use of Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
|
3578
|
+
|
3579
|
+
It is recommended to use the wizard to set it up, although manual configuration is also possible.
|
3580
|
+
|
3581
|
+
### <a id="aocwizard"></a>Configuration: Using Wizard
|
3582
|
+
|
3583
|
+
`ascli` provides a configuration wizard.
|
3296
3584
|
|
3297
|
-
|
3585
|
+
The wizard guides you through the steps to create a new configuration preset for Aspera on Cloud.
|
3298
3586
|
|
3299
|
-
|
3587
|
+
The first
|
3300
3588
|
|
3301
|
-
|
3589
|
+
Here is a sample invocation :
|
3302
3590
|
|
3303
3591
|
```text
|
3304
3592
|
ascli config wizard
|
@@ -3306,7 +3594,7 @@ option: url> https://myorg.ibmaspera.com
|
|
3306
3594
|
Detected: Aspera on Cloud
|
3307
3595
|
Preparing preset: aoc_myorg
|
3308
3596
|
Please provide path to your private RSA key, or empty to generate one:
|
3309
|
-
option:
|
3597
|
+
option: key_path>
|
3310
3598
|
using existing key:
|
3311
3599
|
/Users/myself/.aspera/ascli/aspera_aoc_key
|
3312
3600
|
Using global client_id.
|
@@ -3323,15 +3611,13 @@ ascli aoc user profile show
|
|
3323
3611
|
Optionally, it is possible to create a new organization-specific "integration", i.e. client application identification.
|
3324
3612
|
For this, specify the option: `--use-generic-client=no`.
|
3325
3613
|
|
3326
|
-
|
3327
|
-
|
3328
|
-
If the wizard does not detect the application but you know the application, you can force it using option `query`:
|
3614
|
+
If you already know the application, and want to limit the detection to it, provide url and plugin name:
|
3329
3615
|
|
3330
3616
|
```bash
|
3331
|
-
ascli config wizard
|
3617
|
+
ascli config wizard myorg aoc
|
3332
3618
|
```
|
3333
3619
|
|
3334
|
-
### <a id="aocmanual"></a>Configuration:
|
3620
|
+
### <a id="aocmanual"></a>Configuration: Using manual setup
|
3335
3621
|
|
3336
3622
|
> **Note:** If you used the wizard (recommended): skip this section.
|
3337
3623
|
|
@@ -3345,11 +3631,13 @@ Several types of OAuth authentication are supported:
|
|
3345
3631
|
|
3346
3632
|
The authentication method is controlled by option `auth`.
|
3347
3633
|
|
3348
|
-
For a
|
3634
|
+
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#clientreg) (auth=web) as well as [[option preset](#lprt) for Aspera on Cloud](#aocpreset).
|
3349
3635
|
|
3350
3636
|
For a more convenient, browser-less, experience follow the [JWT](#jwt) section (auth=jwt) in addition to Client Registration.
|
3351
3637
|
|
3352
|
-
In Oauth, a "Bearer" token
|
3638
|
+
In Oauth, a "Bearer" token is generated to authenticate REST calls.
|
3639
|
+
Bearer tokens are valid for a period of time defined (by the AoC app, configurable by admin) at its creation.
|
3640
|
+
`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
|
3353
3641
|
|
3354
3642
|
#### <a id="clientreg"></a>Optional: API Client Registration
|
3355
3643
|
|
@@ -3371,7 +3659,7 @@ Let's start by a registration with web based authentication (auth=web):
|
|
3371
3659
|
- leave the JWT part for now
|
3372
3660
|
- Save
|
3373
3661
|
|
3374
|
-
Note
|
3662
|
+
> **Note:** for web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
|
3375
3663
|
|
3376
3664
|
Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
|
3377
3665
|
|
@@ -3397,7 +3685,7 @@ Define this [option preset](#lprt) as default configuration for the `aspera` plu
|
|
3397
3685
|
ascli config preset set default aoc my_aoc_org
|
3398
3686
|
```
|
3399
3687
|
|
3400
|
-
Note
|
3688
|
+
> **Note:** Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
3401
3689
|
|
3402
3690
|
#### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
|
3403
3691
|
|
@@ -3452,7 +3740,7 @@ Open the previously generated public key located here: `$HOME/.aspera/ascli/my_p
|
|
3452
3740
|
- Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
|
3453
3741
|
- Click on the user's icon (top right)
|
3454
3742
|
- Select "Account Settings"
|
3455
|
-
- Paste the
|
3743
|
+
- Paste the **Public Key** in the "Public Key" section
|
3456
3744
|
- Click on "Submit"
|
3457
3745
|
|
3458
3746
|
##### Using command line
|
@@ -3478,7 +3766,7 @@ ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("
|
|
3478
3766
|
modified
|
3479
3767
|
```
|
3480
3768
|
|
3481
|
-
Note
|
3769
|
+
> **Note:** the `aspera user info show` command can be used to verify modifications.
|
3482
3770
|
|
3483
3771
|
#### [option preset](#lprt) modification for JWT
|
3484
3772
|
|
@@ -3491,13 +3779,26 @@ To activate default use of JWT authentication for `ascli` using the [option pres
|
|
3491
3779
|
Execute:
|
3492
3780
|
|
3493
3781
|
```bash
|
3494
|
-
ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=
|
3782
|
+
ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=someuser@example.com
|
3495
3783
|
```
|
3496
3784
|
|
3497
|
-
Note
|
3785
|
+
> **Note:** the private key argument represents the actual PEM string. In order to read the content from a file, use the `@file:` prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the `@val:` prefix is added.
|
3498
3786
|
|
3499
3787
|
After this last step, commands do not require web login anymore.
|
3500
3788
|
|
3789
|
+
#### Public and private links
|
3790
|
+
|
3791
|
+
AoC gives the possibility to generate public links for both the `Files` and `Packages` modules.
|
3792
|
+
Public links embed the authorization of access.
|
3793
|
+
Provide the public link using option `url` alone.
|
3794
|
+
|
3795
|
+
In addition, the `Files` application supports private links.
|
3796
|
+
Private links require the user to authenticate.
|
3797
|
+
So, provide the same options as for regular authentication, and provide the private link using option `url`.
|
3798
|
+
|
3799
|
+
A user may not be part of any workspace, but still have access to shared folders (using private links).
|
3800
|
+
In that case, it is possible to list those shared folder by using a value for option `workspace` equal to `@none:` or `@json:null` or `@ruby:nil`.
|
3801
|
+
|
3501
3802
|
#### <a id="aocfirst"></a>First Use
|
3502
3803
|
|
3503
3804
|
Once client has been registered and [option preset](#lprt) created: `ascli` can be used:
|
@@ -3542,7 +3843,9 @@ It allows actions (create, update, delete) on "resources": users, group, nodes,
|
|
3542
3843
|
|
3543
3844
|
The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
|
3544
3845
|
|
3545
|
-
The option `query` can be optionally used.
|
3846
|
+
The option `query` can be optionally used.
|
3847
|
+
It expects a `Hash` using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`.
|
3848
|
+
Values are directly sent to the API call and used as a filter on server side.
|
3546
3849
|
|
3547
3850
|
The following parameters are supported:
|
3548
3851
|
|
@@ -3569,7 +3872,7 @@ Examples:
|
|
3569
3872
|
- List users with `laurent` in name:
|
3570
3873
|
|
3571
3874
|
```bash
|
3572
|
-
ascli aoc admin res user list --query
|
3875
|
+
ascli aoc admin res user list --query=@json:'{"q":"laurent"}'
|
3573
3876
|
```
|
3574
3877
|
|
3575
3878
|
- List users who logged-in before a date:
|
@@ -3594,8 +3897,8 @@ Resources are identified by a unique `id`, as well as a unique `name` (case inse
|
|
3594
3897
|
|
3595
3898
|
To execute an action on a specific resource, select it using one of those methods:
|
3596
3899
|
|
3597
|
-
-
|
3598
|
-
- give name on command line
|
3900
|
+
- **recommended**: give id directly on command line **after the action**: `aoc admin res node show 123`
|
3901
|
+
- give name on command line **after the action**: `aoc admin res node show name abc`
|
3599
3902
|
- provide option `id` : `aoc admin res node show 123`
|
3600
3903
|
- provide option `name` : `aoc admin res node show --name=abc`
|
3601
3904
|
|
@@ -3664,7 +3967,7 @@ The activity app can be queried with:
|
|
3664
3967
|
ascli aoc admin analytics transfers
|
3665
3968
|
```
|
3666
3969
|
|
3667
|
-
It can also support filters and send notification using option `
|
3970
|
+
It can also support filters and send notification using option `notify_to`. a template is defined using option `notify_template` :
|
3668
3971
|
|
3669
3972
|
`mytemplate.erb`:
|
3670
3973
|
|
@@ -3687,9 +3990,7 @@ The environment provided contains the following additional variable:
|
|
3687
3990
|
Example:
|
3688
3991
|
|
3689
3992
|
```bash
|
3690
|
-
ascli aoc admin analytics transfers --once-only=yes --lock-port=12345
|
3691
|
-
--query=@json:'{"status":"completed","direction":"receive"}' \
|
3692
|
-
--notif-to=active --notif-template=@file:mytemplate.erb
|
3993
|
+
ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=active --notify-template=@file:mytemplate.erb
|
3693
3994
|
```
|
3694
3995
|
|
3695
3996
|
Options:
|
@@ -3770,19 +4071,7 @@ ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,emai
|
|
3770
4071
|
```
|
3771
4072
|
|
3772
4073
|
```bash
|
3773
|
-
|
3774
|
-
```
|
3775
|
-
|
3776
|
-
```bash
|
3777
|
-
echo $thelist
|
3778
|
-
```
|
3779
|
-
|
3780
|
-
```json
|
3781
|
-
["113501","354061"]
|
3782
|
-
```
|
3783
|
-
|
3784
|
-
```bash
|
3785
|
-
ascli aoc admin res user delete @json:"$thelist" --bulk=yes
|
4074
|
+
ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --display=data --format=csv | ascli aoc admin res user delete @lines:@stdin: --bulk=yes
|
3786
4075
|
```
|
3787
4076
|
|
3788
4077
|
```output
|
@@ -3832,7 +4121,7 @@ ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key
|
|
3832
4121
|
ascli aoc admin res node --secret=_secret_ v3 transfer list --query=@json:'[["q","*"],["count",5]]'
|
3833
4122
|
```
|
3834
4123
|
|
3835
|
-
Examples of query
|
4124
|
+
Examples of query:
|
3836
4125
|
|
3837
4126
|
```json
|
3838
4127
|
{"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
|
@@ -3859,7 +4148,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
|
|
3859
4148
|
| member_type | manager | member.email |
|
3860
4149
|
+-------------+---------+----------------------------------+
|
3861
4150
|
| user | true | john.curtis@email.com |
|
3862
|
-
| user | false |
|
4151
|
+
| user | false | someuser@example.com |
|
3863
4152
|
| user | false | jean.dupont@me.com |
|
3864
4153
|
| user | false | another.user@example.com |
|
3865
4154
|
| group | false | |
|
@@ -3934,7 +4223,7 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
|
|
3934
4223
|
ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
|
3935
4224
|
```
|
3936
4225
|
|
3937
|
-
#### Example:
|
4226
|
+
#### Example: Create a group, add to workspace and add user to group
|
3938
4227
|
|
3939
4228
|
- Create the group and take note of `id`
|
3940
4229
|
|
@@ -3979,7 +4268,7 @@ In this example, a user has access to a workspace where two shared folders are l
|
|
3979
4268
|
First, setup the environment (skip if already done)
|
3980
4269
|
|
3981
4270
|
```bash
|
3982
|
-
ascli conf wizard --url=https://sedemo.ibmaspera.com --username=
|
4271
|
+
ascli conf wizard --url=https://sedemo.ibmaspera.com --username=someuser@example.com
|
3983
4272
|
```
|
3984
4273
|
|
3985
4274
|
```output
|
@@ -4014,7 +4303,7 @@ Then, transfer between those:
|
|
4014
4303
|
ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
4015
4304
|
```
|
4016
4305
|
|
4017
|
-
#### Example:
|
4306
|
+
#### Example: Create registration key to register a node
|
4018
4307
|
|
4019
4308
|
```bash
|
4020
4309
|
ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
@@ -4024,7 +4313,7 @@ ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_s
|
|
4024
4313
|
jfqslfdjlfdjfhdjklqfhdkl
|
4025
4314
|
```
|
4026
4315
|
|
4027
|
-
#### Example:
|
4316
|
+
#### Example: Delete all registration keys
|
4028
4317
|
|
4029
4318
|
```bash
|
4030
4319
|
ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete @lines:@stdin: --bulk=yes
|
@@ -4105,7 +4394,7 @@ In this case:
|
|
4105
4394
|
|
4106
4395
|
### Packages
|
4107
4396
|
|
4108
|
-
The
|
4397
|
+
The web-mail-like application.
|
4109
4398
|
|
4110
4399
|
#### Send a Package
|
4111
4400
|
|
@@ -4117,7 +4406,7 @@ ascli aoc packages send [package extended value] [other parameters such as file
|
|
4117
4406
|
|
4118
4407
|
Notes:
|
4119
4408
|
|
4120
|
-
- Package creation parameter are sent as positional
|
4409
|
+
- Package creation parameter are sent as positional argument.
|
4121
4410
|
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
4122
4411
|
- List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
|
4123
4412
|
- Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
@@ -4130,7 +4419,7 @@ Notes:
|
|
4130
4419
|
#### Example: Send a package with one file to two users, using their email
|
4131
4420
|
|
4132
4421
|
```bash
|
4133
|
-
ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["
|
4422
|
+
ascli aoc packages send @json:'{"name":"my title","note":"my note","recipients":["someuser@example.com","other@example.com"]}' my_file.dat
|
4134
4423
|
```
|
4135
4424
|
|
4136
4425
|
#### Example: Send a package to a shared inbox with metadata
|
@@ -4166,7 +4455,7 @@ ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archiv
|
|
4166
4455
|
Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
4167
4456
|
|
4168
4457
|
```bash
|
4169
|
-
shared_box_id=$(ascli aoc packages shared_inboxes show name
|
4458
|
+
shared_box_id=$(ascli aoc packages shared_inboxes show --name='My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
|
4170
4459
|
```
|
4171
4460
|
|
4172
4461
|
```bash
|
@@ -4187,21 +4476,21 @@ Find files in Files app:
|
|
4187
4476
|
ascli aoc files browse /src_folder
|
4188
4477
|
```
|
4189
4478
|
|
4190
|
-
```
|
4191
|
-
|
4192
|
-
| name
|
4193
|
-
|
4194
|
-
| sample_video
|
4195
|
-
| 100G
|
4196
|
-
| 10M.dat
|
4197
|
-
| Test.pdf
|
4198
|
-
|
4479
|
+
```text
|
4480
|
+
+---------------+--------+----------------+--------------+----------------------+--------------+
|
4481
|
+
| name | type | recursive_size | size | modified_time | access_level |
|
4482
|
+
+---------------+--------+----------------+--------------+----------------------+--------------+
|
4483
|
+
| sample_video | link | | | 2020-11-29T22:49:09Z | edit |
|
4484
|
+
| 100G | file | | 107374182400 | 2021-04-21T18:19:25Z | edit |
|
4485
|
+
| 10M.dat | file | | 10485760 | 2021-05-18T08:22:39Z | edit |
|
4486
|
+
| Test.pdf | file | | 1265103 | 2022-06-16T12:49:55Z | edit |
|
4487
|
+
+---------------+--------+----------------+--------------+----------------------+--------------+
|
4199
4488
|
```
|
4200
4489
|
|
4201
4490
|
Let's send a package with the file `10M.dat` from subfolder /src_folder in a package:
|
4202
4491
|
|
4203
4492
|
```bash
|
4204
|
-
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["
|
4493
|
+
ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc packages send @json:'{"name":"test","recipients":["someuser@example.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
|
4205
4494
|
```
|
4206
4495
|
|
4207
4496
|
#### <a id="aoccargo"></a>Receive new packages only (Cargo)
|
@@ -4223,6 +4512,8 @@ Typically, one would execute this command on a regular basis, using the method o
|
|
4223
4512
|
The Files application presents a **Home** folder to users in a given workspace.
|
4224
4513
|
Files located here are either user's files, or shared folders.
|
4225
4514
|
|
4515
|
+
> **Note:** All commands under `files` are the same as under `access_keys do self` for plugin `node`, i.e. **gen4/access key** operations.
|
4516
|
+
|
4226
4517
|
#### Download Files
|
4227
4518
|
|
4228
4519
|
The general download command is:
|
@@ -4242,8 +4533,10 @@ ascli aoc files download <single file path>
|
|
4242
4533
|
#### Shared folders
|
4243
4534
|
|
4244
4535
|
Shared folder created by users are managed through **permissions**.
|
4245
|
-
|
4536
|
+
|
4537
|
+
For creation, parameters are the same as for node API [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
|
4246
4538
|
`ascli` expects the same payload for creation, but it will automatically populate required tags if needed.
|
4539
|
+
|
4247
4540
|
Also, the pseudo key `with` is available: it will lookup the name in the contacts and fill the proper type and id.
|
4248
4541
|
The pseudo parameter `link_name` allows changing default "shared as" name.
|
4249
4542
|
|
@@ -4265,6 +4558,15 @@ ascli aoc files perm /shared_folder_test1 create @json:'{"with":"laurent"}'
|
|
4265
4558
|
ascli aoc files perm /shared_folder_test1 delete 6161
|
4266
4559
|
```
|
4267
4560
|
|
4561
|
+
Public and Private short links can be managed with command:
|
4562
|
+
|
4563
|
+
```bash
|
4564
|
+
ascli aoc files short_link private create _path_here_
|
4565
|
+
ascli aoc files short_link private list _path_here_
|
4566
|
+
ascli aoc files short_link public list _path_here_
|
4567
|
+
ascli aoc files short_link public delete _id_
|
4568
|
+
```
|
4569
|
+
|
4268
4570
|
#### Cross Organization transfers
|
4269
4571
|
|
4270
4572
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
@@ -4297,59 +4599,22 @@ Explanation:
|
|
4297
4599
|
|
4298
4600
|
#### Find Files
|
4299
4601
|
|
4300
|
-
The command `aoc files find
|
4301
|
-
|
4302
|
-
The expression can be of 3 formats:
|
4303
|
-
|
4304
|
-
- empty (default) : all files, equivalent to value: `exec:true`
|
4305
|
-
- not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax, equivalent to value: `exec:f['name'].match(/expression/)`
|
4306
|
-
|
4307
|
-
For instance, to find files with a special extension, use `--query='\.myext$'`
|
4308
|
-
|
4309
|
-
- starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true.
|
4310
|
-
|
4311
|
-
Examples of expressions: (using like this: `--query=exec:'<expression>'`)
|
4312
|
-
|
4313
|
-
- Find files more recent than 100 days
|
4314
|
-
|
4315
|
-
```ruby
|
4316
|
-
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
4317
|
-
```
|
4318
|
-
|
4319
|
-
- Find files older than 1 year on a given node and store in file list
|
4320
|
-
|
4321
|
-
```ruby
|
4322
|
-
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
4323
|
-
```
|
4324
|
-
|
4325
|
-
```bash
|
4326
|
-
ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find / --fields=path --query='exec:<above expression here>' --format=csv > my_file_list.txt
|
4327
|
-
```
|
4328
|
-
|
4329
|
-
- Find files larger than 1MB
|
4330
|
-
|
4331
|
-
```ruby
|
4332
|
-
f["type"].eql?("file") and f["size"].to_i>1000000
|
4333
|
-
```
|
4334
|
-
|
4335
|
-
- Delete the files, one by one
|
4602
|
+
The command `aoc files find` allows to search for files in a given workspace.
|
4336
4603
|
|
4337
|
-
|
4338
|
-
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 delete "$path" ;done
|
4339
|
-
```
|
4604
|
+
It works also on `node` resource using the `v4` command:
|
4340
4605
|
|
4341
|
-
|
4606
|
+
```bash
|
4607
|
+
ascli aoc admin res node --name='my node name' --secret='my_secret_here' v4 find ...
|
4608
|
+
```
|
4342
4609
|
|
4343
|
-
|
4344
|
-
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my_secret_here' v3 delete @lines:@stdin:
|
4345
|
-
```
|
4610
|
+
For instructions, refer to section `find` for plugin `node`.
|
4346
4611
|
|
4347
4612
|
### AoC sample commands
|
4348
4613
|
|
4349
4614
|
```bash
|
4350
|
-
aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --
|
4351
|
-
aoc admin ats access_key create --cloud=aws --region=
|
4352
|
-
aoc admin ats access_key create --cloud=softlayer --region=
|
4615
|
+
aoc admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
4616
|
+
aoc admin ats access_key create --cloud=aws --region=my_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
|
4617
|
+
aoc admin ats access_key create --cloud=softlayer --region=my_bucket_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
|
4353
4618
|
aoc admin ats access_key delete ak1ibmcloud
|
4354
4619
|
aoc admin ats access_key list --fields=name,id
|
4355
4620
|
aoc admin ats access_key node ak1ibmcloud --secret=my_secret_here browse /
|
@@ -4362,7 +4627,7 @@ aoc admin res application list
|
|
4362
4627
|
aoc admin res client list
|
4363
4628
|
aoc admin res client_access_key list
|
4364
4629
|
aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
|
4365
|
-
aoc admin res client_registration_token delete
|
4630
|
+
aoc admin res client_registration_token delete client_reg_id
|
4366
4631
|
aoc admin res client_registration_token list
|
4367
4632
|
aoc admin res contact list
|
4368
4633
|
aoc admin res dropbox list
|
@@ -4377,69 +4642,73 @@ aoc admin res saml_configuration list
|
|
4377
4642
|
aoc admin res self show
|
4378
4643
|
aoc admin res short_link list
|
4379
4644
|
aoc admin res user list
|
4645
|
+
aoc admin res user modify %name:my_user_email @json:'{"deactivated":false}'
|
4380
4646
|
aoc admin res workspace_membership list
|
4381
|
-
aoc admin resource node
|
4382
|
-
aoc admin resource node
|
4383
|
-
aoc admin resource node
|
4384
|
-
aoc admin resource node
|
4385
|
-
aoc admin resource node
|
4386
|
-
aoc admin resource node do name
|
4647
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret browse /
|
4648
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret delete /folder1
|
4649
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
4650
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
4651
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
4652
|
+
aoc admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
|
4387
4653
|
aoc admin resource workspace list
|
4388
4654
|
aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
4389
4655
|
aoc admin subscription
|
4390
|
-
aoc automation workflow action
|
4656
|
+
aoc automation workflow action wf_id create @json:'{"name":"toto"}' \
|
4391
4657
|
aoc automation workflow create @json:'{"name":"test_workflow"}'
|
4392
|
-
aoc automation workflow delete
|
4658
|
+
aoc automation workflow delete wf_id
|
4393
4659
|
aoc automation workflow list
|
4394
4660
|
aoc automation workflow list --query=@json:'{"show_org_workflows":"true"}' --scope=admin:all
|
4395
4661
|
aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
|
4396
4662
|
aoc bearer_token --display=data --scope=user:all
|
4397
|
-
aoc faspex
|
4398
4663
|
aoc files bearer /
|
4399
4664
|
aoc files bearer_token_node / --cache-tokens=no
|
4400
4665
|
aoc files browse /
|
4401
|
-
aoc files browse / --
|
4666
|
+
aoc files browse / --url=my_private_link
|
4667
|
+
aoc files browse / --url=my_public_link_folder_no_pass
|
4668
|
+
aoc files browse / --url=my_public_link_folder_pass --password=my_public_link_password
|
4402
4669
|
aoc files delete /testsrc
|
4403
|
-
aoc files download --transfer=connect /
|
4404
|
-
aoc files find /
|
4405
|
-
aoc files http_node_download --to-folder=. /
|
4670
|
+
aoc files download --transfer=connect testdst/test_file.bin
|
4671
|
+
aoc files find / '\.partial$'
|
4672
|
+
aoc files http_node_download --to-folder=. testdst/test_file.bin
|
4406
4673
|
aoc files mkdir /testsrc
|
4407
|
-
aoc files modify
|
4408
|
-
aoc files permission
|
4409
|
-
aoc files rename /
|
4410
|
-
aoc files short_link create /testdst
|
4411
|
-
aoc files short_link
|
4412
|
-
aoc files short_link
|
4413
|
-
aoc files show %id:
|
4414
|
-
aoc files show /
|
4415
|
-
aoc files
|
4416
|
-
aoc files sync
|
4417
|
-
aoc files sync
|
4418
|
-
aoc files sync start --sync-info=@json:'{"
|
4419
|
-
aoc files
|
4420
|
-
aoc files
|
4421
|
-
aoc files
|
4422
|
-
aoc files
|
4423
|
-
aoc files upload
|
4674
|
+
aoc files modify my_test_folder
|
4675
|
+
aoc files permission my_test_folder list
|
4676
|
+
aoc files rename /some_folder testdst
|
4677
|
+
aoc files short_link private create /testdst
|
4678
|
+
aoc files short_link private list /testdst
|
4679
|
+
aoc files short_link public create testdst
|
4680
|
+
aoc files show %id:aoc_file_id
|
4681
|
+
aoc files show /
|
4682
|
+
aoc files show testdst/test_file.bin
|
4683
|
+
aoc files sync admin status --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
4684
|
+
aoc files sync admin status --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
4685
|
+
aoc files sync start --sync-info=@json:'{"name":"my_aoc_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/testdst"}}'
|
4686
|
+
aoc files sync start --sync-info=@json:'{"sessions":[{"name":"my_aoc_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/testdst","reset":true}]}'
|
4687
|
+
aoc files thumbnail my_test_folder/video_file.mpg
|
4688
|
+
aoc files thumbnail my_test_folder/video_file.mpg --query=@json:'{"text":true,"double":true}'
|
4689
|
+
aoc files transfer push /testsrc --to-folder=/testdst test_file.bin
|
4690
|
+
aoc files upload --to-folder=/ test_file.bin --url=my_public_link_folder_no_pass
|
4691
|
+
aoc files upload --to-folder=/testsrc test_file.bin
|
4692
|
+
aoc files upload --workspace=my_other_workspace --to-folder=my_other_folder test_file.bin --transfer=node --transfer-info=@json:@stdin:
|
4424
4693
|
aoc files v3 info
|
4425
|
-
aoc gateway https://localhost:12345/aspera/faspex
|
4426
|
-
aoc org --
|
4694
|
+
aoc gateway --pid-file=server_pid https://localhost:12345/aspera/faspex
|
4695
|
+
aoc org --url=my_public_link_recv_from_aoc_user
|
4427
4696
|
aoc organization
|
4428
|
-
aoc packages browse
|
4697
|
+
aoc packages browse package_id3 /contents
|
4429
4698
|
aoc packages list
|
4430
|
-
aoc packages list --query=@json:'{"dropbox_name":"
|
4431
|
-
aoc packages recv "my_package_id" --to-folder=.
|
4699
|
+
aoc packages list --query=@json:'{"dropbox_name":"my_shared_inbox_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
|
4432
4700
|
aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
|
4433
|
-
aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"
|
4434
|
-
aoc packages
|
4435
|
-
aoc packages send --workspace=
|
4436
|
-
aoc packages send --workspace=
|
4437
|
-
aoc packages send @json:'{"name":"Important files delivery","recipients":["
|
4438
|
-
aoc packages send @json:'{"name":"Important files delivery","recipients":["
|
4439
|
-
aoc packages send @json:'{"name":"Important files delivery"}'
|
4440
|
-
aoc packages send @json:'{"name":"Important files delivery"}'
|
4701
|
+
aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345 --query=@json:'{"dropbox_name":"my_shared_inbox_name","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false}' --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000}'
|
4702
|
+
aoc packages recv package_id3 --to-folder=.
|
4703
|
+
aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' test_file.bin
|
4704
|
+
aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' test_file.bin
|
4705
|
+
aoc packages send --workspace=my_workspace_shared_inbox @json:'{"name":"Important files delivery","recipients":["my_shared_inbox_name"]}' test_file.bin
|
4706
|
+
aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_external"]}' --new-user-option=@json:'{"package_contact":true}' test_file.bin
|
4707
|
+
aoc packages send @json:'{"name":"Important files delivery","recipients":["my_email_internal"],"note":"my note"}' test_file.bin
|
4708
|
+
aoc packages send @json:'{"name":"Important files delivery"}' test_file.bin --url=my_public_link_send_aoc_user --password=my_public_link_send_use_pass
|
4709
|
+
aoc packages send @json:'{"name":"Important files delivery"}' test_file.bin --url=my_public_link_send_shared_inbox
|
4441
4710
|
aoc packages shared_inboxes list
|
4442
|
-
aoc remind --username=
|
4711
|
+
aoc remind --username=my_user_email
|
4443
4712
|
aoc servers
|
4444
4713
|
aoc user profile modify @json:'{"name":"dummy change"}'
|
4445
4714
|
aoc user profile show
|
@@ -4455,7 +4724,7 @@ ATS is usable either :
|
|
4455
4724
|
|
4456
4725
|
- or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
|
4457
4726
|
|
4458
|
-
### IBM Cloud ATS :
|
4727
|
+
### IBM Cloud ATS : Creation of api key
|
4459
4728
|
|
4460
4729
|
This section is about using ATS with an IBM cloud subscription.
|
4461
4730
|
If you are using ATS as part of AoC, then authentication is through AoC, not IBM Cloud.
|
@@ -4573,8 +4842,8 @@ The parameters provided to ATS for access key creation are the ones of [ATS API]
|
|
4573
4842
|
|
4574
4843
|
```bash
|
4575
4844
|
ats access_key cluster ak2ibmcloud --secret=my_secret_here
|
4576
|
-
ats access_key create --cloud=aws --region=
|
4577
|
-
ats access_key create --cloud=softlayer --region=
|
4845
|
+
ats access_key create --cloud=aws --region=my_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
|
4846
|
+
ats access_key create --cloud=softlayer --region=my_bucket_region --params=@json:'{"id":"ak2ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket_name","credentials":{"access_key_id":"my_bucket_key","secret_access_key":"my_bucket_secret"},"path":"/"}}'
|
4578
4847
|
ats access_key delete ak2ibmcloud
|
4579
4848
|
ats access_key delete ak_aws
|
4580
4849
|
ats access_key entitlement ak2ibmcloud
|
@@ -4603,36 +4872,36 @@ then commands `ascp` (for transfers) and `ascmd` (for file operations) are execu
|
|
4603
4872
|
|
4604
4873
|
```bash
|
4605
4874
|
server browse /
|
4606
|
-
server browse
|
4607
|
-
server browse
|
4608
|
-
server cp
|
4609
|
-
server delete
|
4610
|
-
server delete
|
4611
|
-
server delete folder_1/to.delete
|
4875
|
+
server browse my_inside_folder/test_file.bin
|
4876
|
+
server browse my_upload_folder/target_hot
|
4877
|
+
server cp my_inside_folder/test_file.bin my_upload_folder/200KB.2
|
4878
|
+
server delete my_inside_folder
|
4879
|
+
server delete my_upload_folder/to.delete
|
4612
4880
|
server df
|
4613
|
-
server download
|
4614
|
-
server download
|
4881
|
+
server download my_inside_folder/test_file.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
|
4882
|
+
server download my_inside_folder/test_file.bin --to-folder=my_upload_folder --transfer=node
|
4615
4883
|
server du /
|
4616
|
-
server health transfer --to-folder=
|
4884
|
+
server health transfer --to-folder=my_upload_folder --format=nagios
|
4617
4885
|
server info
|
4618
|
-
server md5sum
|
4619
|
-
server mkdir
|
4620
|
-
server mkdir
|
4621
|
-
server mv
|
4622
|
-
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"
|
4623
|
-
server sync admin status --sync-
|
4624
|
-
server sync
|
4625
|
-
server sync start --sync-info=@json:'{"sessions":[{"name":"
|
4626
|
-
server
|
4627
|
-
server upload --
|
4628
|
-
server upload --
|
4629
|
-
server upload --
|
4630
|
-
server upload --sources=@ts --
|
4631
|
-
server upload --
|
4632
|
-
server upload --
|
4633
|
-
server upload --src-type=pair
|
4634
|
-
server upload --
|
4635
|
-
server upload
|
4886
|
+
server md5sum my_inside_folder/test_file.bin
|
4887
|
+
server mkdir my_inside_folder --logger=stdout
|
4888
|
+
server mkdir my_upload_folder/target_hot
|
4889
|
+
server mv my_upload_folder/200KB.2 my_upload_folder/to.delete
|
4890
|
+
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
|
4891
|
+
server sync admin status --sync-info=@json:'{"name":"sync2"}'
|
4892
|
+
server sync admin status my_sync --sync-info=@json:'{"sessions":[{"name":"my_sync","local_dir":"/data/local_sync"}]}'
|
4893
|
+
server sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
|
4894
|
+
server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
|
4895
|
+
server upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
|
4896
|
+
server upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
|
4897
|
+
server upload 'test_file.bin' --to-folder=my_inside_folder --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":100000}' --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}' --progress-bar=yes
|
4898
|
+
server upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-list","filelist.txt"]}' --to-folder=my_inside_folder
|
4899
|
+
server upload --sources=@ts --transfer-info=@json:'{"ascp_args":["--file-pair-list","file_pair_list.txt"]}'
|
4900
|
+
server upload --sources=@ts --ts=@json:'{"paths":[{"source":"test_file.bin","destination":"my_inside_folder/other_name_4"}]}' --transfer=trsdk
|
4901
|
+
server upload --src-type=pair 'test_file.bin' my_inside_folder/other_name_2 --notify-to=my_email_external --transfer-info=@json:'{"ascp_args":["-l","100m"]}'
|
4902
|
+
server upload --src-type=pair --sources=@json:'["test_file.bin","my_inside_folder/other_name_3"]' --transfer-info=@json:'{"quiet":false}' --progress=no
|
4903
|
+
server upload --src-type=pair test_file.bin my_upload_folder/other_name_5 --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"my_secret_here","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
|
4904
|
+
server upload --to-folder=my_upload_folder/target_hot --lock-port=12345 --transfer-info=@json:'{"ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
|
4636
4905
|
```
|
4637
4906
|
|
4638
4907
|
### Authentication on Server with SSH session
|
@@ -4735,21 +5004,106 @@ ascli server --url=ssh://_server_address_here_:33001 --username=_user_here_ --ss
|
|
4735
5004
|
|
4736
5005
|
## <a id="node"></a>Plugin: `node`: IBM Aspera High Speed Transfer Server Node
|
4737
5006
|
|
4738
|
-
This plugin gives access to capabilities provided by HSTS node API.
|
5007
|
+
This plugin gives access to capabilities provided by the HSTS node API.
|
4739
5008
|
|
4740
|
-
|
5009
|
+
The authentication is `username` and `password` or `access_key` and `secret` through options: `username` and `password`.
|
5010
|
+
|
5011
|
+
> **Note:** Capabilities of this plugin are used in other plugins which access to the node API, such as `aoc`, `ats`, `shares`.
|
4741
5012
|
|
4742
5013
|
### File Operations
|
4743
5014
|
|
4744
|
-
It is possible to:
|
5015
|
+
It is possible to do **gen3/node user** operations:
|
4745
5016
|
|
4746
5017
|
- browse
|
4747
|
-
- transfer (upload / download)
|
5018
|
+
- transfer (upload / download / sync)
|
5019
|
+
- delete
|
4748
5020
|
- ...
|
4749
5021
|
|
5022
|
+
When using an access key, so called **gen4/access key** API is also supported through sub commands using `access_keys do self`.
|
5023
|
+
|
5024
|
+
Example:
|
5025
|
+
|
5026
|
+
- `ascli node browse /` : list files with **gen3/node user** API
|
5027
|
+
- `ascli node access_key do self browse /` : list files with **gen4/access key** API
|
5028
|
+
|
5029
|
+
### Operation `find` on **gen4/access key**
|
5030
|
+
|
5031
|
+
The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
|
5032
|
+
|
5033
|
+
The argument `<folder>` is mandatory and is the root from which search is performed.
|
5034
|
+
The argument `[filter_expr]` is optional and represent the matching criteria.
|
5035
|
+
|
5036
|
+
It recursively scans storage to find files/folders matching a criteria and then returns a list of matching entries.
|
5037
|
+
|
5038
|
+
`[filter_expr]` is either:
|
5039
|
+
|
5040
|
+
- Optional (default) : all files and folder are selected
|
5041
|
+
- type `String` : the expression is similar to shell globing, refer to **Ruby** function: [`File.fnmatch`](https://ruby-doc.org/3.2.2/File.html#method-c-fnmatch)
|
5042
|
+
- Type `Proc` : the expression is a Ruby lambda that takes one argument: a `Hash` that contains the current folder entry to test. Refer to the following examples.
|
5043
|
+
|
5044
|
+
Examples of expressions:
|
5045
|
+
|
5046
|
+
- Find all files and folders under `/`
|
5047
|
+
|
5048
|
+
```bash
|
5049
|
+
ascli node access_keys do self find
|
5050
|
+
```
|
5051
|
+
|
5052
|
+
- Find all text files `/Documents`
|
5053
|
+
|
5054
|
+
```bash
|
5055
|
+
ascli node access_keys do self find /Documents '*.txt'
|
5056
|
+
```
|
5057
|
+
|
5058
|
+
The following are examples of `ruby_lambda` to be provided in the following template command:
|
5059
|
+
|
5060
|
+
```bash
|
5061
|
+
ascli node access_keys do self find / @ruby:'ruby_lambda'
|
5062
|
+
```
|
5063
|
+
|
5064
|
+
> **Note:** Single quotes are used here above to protect the whole **Ruby** expression from the shell. Then double quotes are used for strings in the **Ruby** expression to not mix with the shell.
|
5065
|
+
|
5066
|
+
- Find files more recent than 100 days
|
5067
|
+
|
5068
|
+
```ruby
|
5069
|
+
->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100}
|
5070
|
+
```
|
5071
|
+
|
5072
|
+
- Find files older than 1 year
|
5073
|
+
|
5074
|
+
```ruby
|
5075
|
+
->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))>365}
|
5076
|
+
```
|
5077
|
+
|
5078
|
+
- Find files larger than 1MB
|
5079
|
+
|
5080
|
+
```ruby
|
5081
|
+
->(f){f["type"].eql?("file") and f["size"].to_i>1000000}
|
5082
|
+
```
|
5083
|
+
|
5084
|
+
- Filter out files beginning with `._` or named `.DS_Store`:
|
5085
|
+
|
5086
|
+
```ruby
|
5087
|
+
->(f){!(f["name"].start_with?("._") or f["name"].eql?(".DS_Store"))}
|
5088
|
+
```
|
5089
|
+
|
5090
|
+
- Match files using a [Ruby Regex](https://ruby-doc.org/core/Regexp.html) : `\.gif$`
|
5091
|
+
|
5092
|
+
```ruby
|
5093
|
+
->(f){f["name"].match?(/\.gif$/)}
|
5094
|
+
```
|
5095
|
+
|
5096
|
+
`ascli` commands can be piped in order to combine operations, such as "find and delete":
|
5097
|
+
|
5098
|
+
```bash
|
5099
|
+
ascli node access_keys do self find / @ruby:'->(f){f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))>365}' --fields=path --format=csv | ascli node --bulk=yes delete @lines:@stdin:
|
5100
|
+
```
|
5101
|
+
|
5102
|
+
> **Note:** the pipe `|` character on the last line.
|
5103
|
+
|
4750
5104
|
### Central
|
4751
5105
|
|
4752
|
-
The central subcommand uses the
|
5106
|
+
The central subcommand uses the **reliable query** API (session and file).
|
4753
5107
|
It allows listing transfer sessions and transferred files.
|
4754
5108
|
|
4755
5109
|
Filtering can be applied:
|
@@ -4758,26 +5112,36 @@ Filtering can be applied:
|
|
4758
5112
|
ascli node central file list
|
4759
5113
|
```
|
4760
5114
|
|
4761
|
-
|
5115
|
+
By providing the `validator` option, offline transfer validation can be done.
|
5116
|
+
|
5117
|
+
> **Note:** This API will probably be deprecated in the future.
|
5118
|
+
|
5119
|
+
### Sync
|
5120
|
+
|
5121
|
+
There are three sync types of operations:
|
5122
|
+
|
5123
|
+
- `sync`: perform a local sync, by executing `async` locally
|
5124
|
+
- `async`: calls legacy async API on node : `/async`
|
5125
|
+
- `ssync` : calls newer async API on node : `/asyncs`
|
4762
5126
|
|
4763
5127
|
### FASP Stream
|
4764
5128
|
|
4765
5129
|
It is possible to start a FASPStream session using the node API:
|
4766
5130
|
|
4767
|
-
Use the
|
5131
|
+
Use the command `ascli node stream create --ts=@json:<value>`, with [*transfer-spec*](#transferspec):
|
4768
5132
|
|
4769
|
-
```
|
4770
|
-
|
5133
|
+
```json
|
5134
|
+
{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
|
4771
5135
|
```
|
4772
5136
|
|
4773
|
-
### Watchfolder
|
5137
|
+
### "Watchfolder"
|
4774
5138
|
|
4775
5139
|
Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
|
4776
5140
|
|
4777
5141
|
`ascli` supports remote operations through the node API. Operations are:
|
4778
5142
|
|
4779
|
-
- Start watchd and watchfolderd services running as a system user having access to files
|
4780
|
-
- configure a watchfolder to define automated transfers
|
5143
|
+
- Start `watchd` and `watchfolderd` services running as a system user having access to files
|
5144
|
+
- configure a "watchfolder" to define automated transfers
|
4781
5145
|
|
4782
5146
|
```bash
|
4783
5147
|
ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
@@ -4835,21 +5199,21 @@ dnf install -y ImageMagick-devel
|
|
4835
5199
|
gem install rmagick rainbow
|
4836
5200
|
```
|
4837
5201
|
|
4838
|
-
For example it is possible to display the preview of a file, if it exists, using:
|
5202
|
+
For example, it is possible to display the preview of a file, if it exists, using an access key on node:
|
4839
5203
|
|
4840
5204
|
```bash
|
4841
|
-
ascli
|
5205
|
+
ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
|
4842
5206
|
```
|
4843
5207
|
|
4844
|
-
|
5208
|
+
Previews are mainly used in AoC, this also works with AoC:
|
4845
5209
|
|
4846
5210
|
```bash
|
4847
|
-
ascli
|
5211
|
+
ascli aoc files thumbnail /preview_samples/Aspera.mpg
|
4848
5212
|
```
|
4849
5213
|
|
4850
5214
|
> **Note:** To specify the file by its file id, use the selector syntax: `%id:_file_id_here_`
|
4851
5215
|
>
|
4852
|
-
> **Note:** To force textual display of the preview on iTerm
|
5216
|
+
> **Note:** To force textual display of the preview on **iTerm**, prefix command with: `env -u TERM_PROGRAM -u LC_TERMINAL`
|
4853
5217
|
|
4854
5218
|
### Create access key
|
4855
5219
|
|
@@ -4857,24 +5221,163 @@ ascli node access_key do self thumbnail /preview_samples/Aspera.mpg
|
|
4857
5221
|
ascli node access_key create @json:'{"id":"myaccesskey","secret":"my_secret_here","storage":{"type":"local","path":"/data/mydir"}}'
|
4858
5222
|
```
|
4859
5223
|
|
5224
|
+
> **Note:** The `id` and `secret` are optional.
|
5225
|
+
> If not provided, they will be generated and returned in the result.
|
5226
|
+
|
5227
|
+
Access keys support extra overriding parameters using parameter: `configuration` and sub keys `transfer` and `server`. For example, an access key can be modified or created with the following options:
|
5228
|
+
|
5229
|
+
```json
|
5230
|
+
{"configuration":{"transfer":{"target_rate_cap_kbps":500000}}}
|
5231
|
+
```
|
5232
|
+
|
5233
|
+
The list of supported options can be displayed using command:
|
5234
|
+
|
5235
|
+
```bash
|
5236
|
+
ascli node info --field=@ruby:'/^access_key_configuration_capabilities.*/'
|
5237
|
+
```
|
5238
|
+
|
5239
|
+
### Generate and use bearer token
|
5240
|
+
|
5241
|
+
Bearer tokens are part of the **gen4/access key** API.
|
5242
|
+
It follows the model of OAuth 2.
|
5243
|
+
For example they are used in Aspera on Cloud.
|
5244
|
+
This is also available for developers for any application integrating Aspera.
|
5245
|
+
In this API, files, users and groups are identified by an id (a String, e.g. `"125"`, not necessarily numerical).
|
5246
|
+
|
5247
|
+
Bearer tokens are typically generated by the authentication application, and then recognized by the node API.
|
5248
|
+
A bearer token is authorized on the node by creating `permissions` on a **folder**.
|
5249
|
+
|
5250
|
+
Bearer tokens can be generated using command `bearer_token`: it takes two arguments:
|
5251
|
+
|
5252
|
+
- the private key used to sign the token
|
5253
|
+
- the token information, which is a `Hash` containing the following elements:
|
5254
|
+
|
5255
|
+
| parameter | Default | type | description |
|
5256
|
+
| ------------------------ |-----------------------------|-----------|----------------------------------------------------------|
|
5257
|
+
| user_id | - | Mandatory | Identifier of user |
|
5258
|
+
| scope | `node.<access_key>:<_scope>`| Mandatory | API scope, e.g. `node.<access_key>:<node scope>` |
|
5259
|
+
| expires_at | `now+<_validity>` | Mandatory | Format: `%Y-%m-%dT%H:%M:%SZ` e.g. `2021-12-31T23:59:59Z` |
|
5260
|
+
| auth_type | `access_key` | Optional | `access_key`, `node_user` |
|
5261
|
+
| group_ids | - | Optional | List of group ids |
|
5262
|
+
| organization_id | - | Optional | Organization id |
|
5263
|
+
| watermarking_json_base64 | - | Optional | Watermarking information (not used) |
|
5264
|
+
| _scope | `user:all` | Special | Either `user:all` or `admin:all` |
|
5265
|
+
| _validity | 86400 | Special | Validity in seconds from now. |
|
5266
|
+
|
5267
|
+
> **Note:** For convenience, `ascli` provides additional parameters `_scope` and `_validity`.
|
5268
|
+
> They are not part of the API and are removed from the final payload.
|
5269
|
+
> They are used respectively to build the default value of `scope` and `expires_at`.
|
5270
|
+
|
5271
|
+
#### Bearer token: Environment
|
5272
|
+
|
5273
|
+
- If a self-managed Aspera node is used, then a "node user admin" must be created:
|
5274
|
+
It has no `docroot` but has at least one file restriction (for testing, one can use `*` to accept creation of an access key with any storage root path).
|
5275
|
+
Refer to the Aspera HSTS documentation.
|
5276
|
+
|
5277
|
+
- If Cloud Pak for integration is used, then the node admin is created automatically.
|
5278
|
+
|
5279
|
+
- If Aspera on Cloud or ATS is used, then the SaaS API for access key creation is used.
|
5280
|
+
|
5281
|
+
- An access key shall be created to grant access for transfers to its storage.
|
5282
|
+
The access_key and its secrets represent an administrative access to the storage as it has access rights to the whole storage of the access key.
|
5283
|
+
|
5284
|
+
#### Bearer token: Preparation
|
5285
|
+
|
5286
|
+
Let's assume that the access key was created, and a default configuration is set to use this **access key**.
|
5287
|
+
|
5288
|
+
- Create a private key (organization key) that will be used to sign bearer tokens:
|
5289
|
+
|
5290
|
+
```bash
|
5291
|
+
my_private_pem=./myorgkey.pem
|
5292
|
+
ascli conf genkey $my_private_pem
|
5293
|
+
```
|
5294
|
+
|
5295
|
+
> **Note:** This key is not used for authentication, it is used to sign bearer tokens.
|
5296
|
+
> Refer to section [private key](#private_key) for more details on generation.
|
5297
|
+
|
5298
|
+
- The corresponding public key shall be placed as an attribute of the **access key**:
|
5299
|
+
|
5300
|
+
```bash
|
5301
|
+
ascli node access_key set_bearer_key self @file:$my_private_pem
|
5302
|
+
```
|
5303
|
+
|
5304
|
+
> **Note:** Either the public or private key can be provided, and only the public key is used.
|
5305
|
+
> This will enable to check the signature of the bearer token.
|
5306
|
+
|
5307
|
+
Alternatively, use the following equivalent command, as `ascli` kindly extracts the public key with extension `.pub`:
|
5308
|
+
|
5309
|
+
```bash
|
5310
|
+
ascli node access_key modify %id:self @ruby:'{token_verification_key: File.read("'$my_private_pem'.pub")}'
|
5311
|
+
```
|
5312
|
+
|
5313
|
+
#### Bearer token: Configuration for user
|
5314
|
+
|
5315
|
+
- Select a folder for which we want to grant access to a user, and get its identifier:
|
5316
|
+
|
5317
|
+
```bash
|
5318
|
+
my_folder_id=$(ascli node access_key do self show / --fields=id)
|
5319
|
+
```
|
5320
|
+
|
5321
|
+
> **Note:** Here we simply select `/`, but any folder can be selected in the access key storage.
|
5322
|
+
|
5323
|
+
- Let's designate a user by its id:
|
5324
|
+
|
5325
|
+
```bash
|
5326
|
+
my_user_id=777
|
5327
|
+
```
|
5328
|
+
|
5329
|
+
> **Note:** This is an arbitrary identifier, typically managed by the web application.
|
5330
|
+
> Not related to Linux user ids or anything else.
|
5331
|
+
|
5332
|
+
- Grant this user access to the selected folder:
|
5333
|
+
|
5334
|
+
```bash
|
5335
|
+
ascli node access_key do self permission %id:$my_folder_id create @json:'{"access_type":"user","access_id":"'$my_user_id'"}'
|
5336
|
+
```
|
5337
|
+
|
5338
|
+
- Create a Bearer token for the user:
|
5339
|
+
|
5340
|
+
```bash
|
5341
|
+
ascli node bearer_token @file:./myorgkey.pem @json:'{"user_id":"'$my_user_id'","_validity":3600}' > bearer.txt
|
5342
|
+
```
|
5343
|
+
|
5344
|
+
#### Bearer token: User side
|
5345
|
+
|
5346
|
+
Now, let's assume we are the user, the only information received are:
|
5347
|
+
|
5348
|
+
- the url of the node API
|
5349
|
+
- a Bearer token
|
5350
|
+
- a file `id` for which we have access
|
5351
|
+
|
5352
|
+
Let's use it:
|
5353
|
+
|
5354
|
+
```bash
|
5355
|
+
ascli node -N --url=... --password="Bearer $(cat bearer.txt)" --root-id=$my_folder_id access_key do self br /
|
5356
|
+
```
|
5357
|
+
|
4860
5358
|
### Node sample commands
|
4861
5359
|
|
4862
5360
|
```bash
|
4863
|
-
node
|
4864
|
-
node access_key
|
4865
|
-
node access_key
|
4866
|
-
node access_key do
|
4867
|
-
node access_key do
|
4868
|
-
node access_key do
|
4869
|
-
node access_key do
|
4870
|
-
node access_key do
|
4871
|
-
node access_key do
|
4872
|
-
node access_key do
|
4873
|
-
node access_key do
|
4874
|
-
node access_key do
|
4875
|
-
node access_key do
|
4876
|
-
node access_key do
|
5361
|
+
node --url=https://tst.example.com/path --password="Bearer bearer_666" --root-id=root_id access_key do self br /
|
5362
|
+
node access_key create @json:'{"id":"my_username","secret":"my_password_here","storage":{"type":"local","path":"/"}}'
|
5363
|
+
node access_key delete my_username
|
5364
|
+
node access_key do my_ak_name browse /
|
5365
|
+
node access_key do my_ak_name delete /folder2
|
5366
|
+
node access_key do my_ak_name delete testfile1
|
5367
|
+
node access_key do my_ak_name download testfile1 --to-folder=.
|
5368
|
+
node access_key do my_ak_name find my_test_folder
|
5369
|
+
node access_key do my_ak_name find my_test_folder @ruby:'->(f){f["name"].end_with?(".jpg")}'
|
5370
|
+
node access_key do my_ak_name find my_test_folder exec:'f["name"].end_with?(".jpg")'
|
5371
|
+
node access_key do my_ak_name mkdir /folder1
|
5372
|
+
node access_key do my_ak_name node_info /
|
5373
|
+
node access_key do my_ak_name rename /folder1 folder2
|
5374
|
+
node access_key do my_ak_name show %id:1
|
5375
|
+
node access_key do my_ak_name show /testfile1
|
5376
|
+
node access_key do my_ak_name upload 'faux:///testfile1?1k' --default_ports=no
|
5377
|
+
node access_key do self permission %id:root_id create @json:'{"access_type":"user","access_id":"666"}'
|
5378
|
+
node access_key do self show / --fields=id
|
4877
5379
|
node access_key list
|
5380
|
+
node access_key set_bearer_key self @file:my_private_key
|
4878
5381
|
node api_details
|
4879
5382
|
node async bandwidth 1
|
4880
5383
|
node async counters 1
|
@@ -4883,55 +5386,107 @@ node async list
|
|
4883
5386
|
node async show 1
|
4884
5387
|
node async show ALL
|
4885
5388
|
node basic_token
|
4886
|
-
node
|
4887
|
-
node
|
4888
|
-
node delete @list:,
|
4889
|
-
node delete
|
4890
|
-
node
|
4891
|
-
node download testfile.bin --to-folder=.
|
5389
|
+
node bearer_token @file:my_private_key @json:'{"user_id":"666"}'
|
5390
|
+
node browse / --log-level=trace2
|
5391
|
+
node delete @list:,my_upload_folder/a_folder,my_upload_folder/tdlink,my_upload_folder/a_file
|
5392
|
+
node delete my_upload_folder/test_file.bin
|
5393
|
+
node download my_upload_folder/test_file.bin --to-folder=.
|
4892
5394
|
node health
|
4893
5395
|
node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
|
4894
5396
|
node license
|
4895
|
-
node mkdir
|
4896
|
-
node mkfile
|
4897
|
-
node mklink
|
4898
|
-
node rename
|
5397
|
+
node mkdir my_upload_folder/a_folder
|
5398
|
+
node mkfile my_upload_folder/a_file1 "hello world"
|
5399
|
+
node mklink my_upload_folder/a_folder my_upload_folder/tdlink
|
5400
|
+
node rename my_upload_folder a_file1 a_file
|
4899
5401
|
node search / --query=@json:'{"sort":"mtime"}'
|
4900
5402
|
node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
|
4901
5403
|
node service delete service1
|
4902
5404
|
node service list
|
4903
5405
|
node space /
|
4904
|
-
node ssync bandwidth
|
4905
|
-
node ssync counters
|
4906
|
-
node ssync create @json:'{"configuration":{"name":"
|
4907
|
-
node ssync delete
|
4908
|
-
node ssync files
|
5406
|
+
node ssync bandwidth %name:my_node_sync
|
5407
|
+
node ssync counters %name:my_node_sync
|
5408
|
+
node ssync create @json:'{"configuration":{"name":"my_node_sync","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password_here","path":"my_remote_path"}}}'
|
5409
|
+
node ssync delete %name:my_node_sync
|
5410
|
+
node ssync files %name:my_node_sync
|
4909
5411
|
node ssync list
|
4910
|
-
node ssync show
|
4911
|
-
node ssync start
|
4912
|
-
node ssync state
|
4913
|
-
node ssync stop
|
4914
|
-
node ssync summary
|
4915
|
-
node sync
|
4916
|
-
node sync
|
4917
|
-
node sync start --sync-info=@json:'{"name":"
|
4918
|
-
node sync start --sync-info=@json:'{"sessions":[{"name":"
|
5412
|
+
node ssync show %name:my_node_sync
|
5413
|
+
node ssync start %name:my_node_sync
|
5414
|
+
node ssync state %name:my_node_sync
|
5415
|
+
node ssync stop %name:my_node_sync
|
5416
|
+
node ssync summary %name:my_node_sync
|
5417
|
+
node sync admin status --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5418
|
+
node sync admin status --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
5419
|
+
node sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"pull","local":{"path":"/data/local_sync"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
5420
|
+
node sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
4919
5421
|
node transfer list --query=@json:'{"active_only":true}'
|
4920
|
-
node
|
4921
|
-
node upload --
|
4922
|
-
node upload
|
5422
|
+
node transfer sessions
|
5423
|
+
node upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
5424
|
+
node upload --username=my_ak_name --password=my_ak_secret test_file.bin
|
5425
|
+
node upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
|
4923
5426
|
```
|
4924
5427
|
|
4925
5428
|
## <a id="faspex5"></a>Plugin: `faspex5`: IBM Aspera Faspex v5
|
4926
5429
|
|
4927
5430
|
IBM Aspera's newer self-managed application.
|
4928
5431
|
|
4929
|
-
3 authentication methods are supported:
|
5432
|
+
3 authentication methods are supported (option `auth`):
|
5433
|
+
|
5434
|
+
| Method | Description |
|
5435
|
+
| ------------ | ------------------------------------------------------------------- |
|
5436
|
+
| `jwt` | General purpose, private-key based authentication |
|
5437
|
+
| `web` | Requires authentication with web browser |
|
5438
|
+
| `boot` | Use authentication token copied from browser (experimental) |
|
5439
|
+
| `public_link`| Public link authentication (set when option `url` is a public link) |
|
5440
|
+
|
5441
|
+
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
5442
|
+
|
5443
|
+
For a quick start, one can use the wizard, which will help creating a [option preset](#lprt):
|
4930
5444
|
|
4931
|
-
|
4932
|
-
|
4933
|
-
|
4934
|
-
|
5445
|
+
```bash
|
5446
|
+
ascli config wizard
|
5447
|
+
```
|
5448
|
+
|
5449
|
+
```text
|
5450
|
+
argument: url> faspex5.example.com
|
5451
|
+
Multiple applications detected:
|
5452
|
+
+---------+-------------------------------------------+-------------+
|
5453
|
+
| product | url | version |
|
5454
|
+
+---------+-------------------------------------------+-------------+
|
5455
|
+
| faspex5 | https://faspex5.example.com/aspera/faspex | F5.0.6 |
|
5456
|
+
| server | ssh://faspex5.example.com:22 | OpenSSH_8.3 |
|
5457
|
+
+---------+-------------------------------------------+-------------+
|
5458
|
+
product> faspex5
|
5459
|
+
Using: Faspex at https://faspex5.example.com/aspera/faspex
|
5460
|
+
Please provide the path to your private RSA key, or nothing to generate one:
|
5461
|
+
option: key_path>
|
5462
|
+
Using existing key:
|
5463
|
+
/Users/someuser/.aspera/ascli/my_key
|
5464
|
+
option: username> someuser@example.com
|
5465
|
+
Ask the ascli client id and secret to your Administrator.
|
5466
|
+
Admin should login to: https://faspex5.example.com/aspera/faspex
|
5467
|
+
Navigate to: :: → Admin → Configurations → API clients
|
5468
|
+
Create an API client with:
|
5469
|
+
- name: ascli
|
5470
|
+
- JWT: enabled
|
5471
|
+
Then, logged in as someuser@example.com go to your profile:
|
5472
|
+
() → Account Settings → Preferences -> Public Key in PEM:
|
5473
|
+
-----BEGIN PUBLIC KEY-----
|
5474
|
+
redacted
|
5475
|
+
-----END PUBLIC KEY-----
|
5476
|
+
Once set, fill in the parameters:
|
5477
|
+
option: client_id> _my_key_here_
|
5478
|
+
option: client_secret> ****
|
5479
|
+
Preparing preset: faspex5_example_com_user
|
5480
|
+
Setting config preset as default for faspex5
|
5481
|
+
Done.
|
5482
|
+
You can test with:
|
5483
|
+
ascli faspex5 user profile show
|
5484
|
+
Saving config file.
|
5485
|
+
```
|
5486
|
+
|
5487
|
+
> **Note:** Include the public key `BEGIN` and `END` lines when pasting in the user profile.
|
5488
|
+
|
5489
|
+
For details on the JWT method, see the following section.
|
4935
5490
|
|
4936
5491
|
### Faspex 5 JWT authentication
|
4937
5492
|
|
@@ -4939,7 +5494,7 @@ This is the general purpose and **recommended** method to use.
|
|
4939
5494
|
|
4940
5495
|
Activation is in two steps:
|
4941
5496
|
|
4942
|
-
- The
|
5497
|
+
- The administrator must create an API client in Faspex with JWT support
|
4943
5498
|
|
4944
5499
|
This operation is generally done only once:
|
4945
5500
|
|
@@ -4986,7 +5541,7 @@ ascli faspex5 user profile show
|
|
4986
5541
|
|
4987
5542
|
### Faspex 5 web authentication
|
4988
5543
|
|
4989
|
-
The
|
5544
|
+
The administrator must create an API client in Faspex for an external web app support:
|
4990
5545
|
|
4991
5546
|
- As Admin, Navigate to the web UI: Admin → Configurations → API Clients → Create
|
4992
5547
|
- Do not Activate JWT
|
@@ -5021,7 +5576,7 @@ ascli conf preset update f5boot --url=https://localhost/aspera/faspex --auth=boo
|
|
5021
5576
|
### Faspex 5 sample commands
|
5022
5577
|
|
5023
5578
|
Most commands are directly REST API calls.
|
5024
|
-
Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional
|
5579
|
+
Parameters to commands are carried through option `query`, as extended value, for `list`, or through positional argument for creation.
|
5025
5580
|
One can conveniently use the JSON format with prefix `@json:`.
|
5026
5581
|
|
5027
5582
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
@@ -5031,42 +5586,48 @@ faspex5 admin res accounts list
|
|
5031
5586
|
faspex5 admin res contacts list
|
5032
5587
|
faspex5 admin res jobs list
|
5033
5588
|
faspex5 admin res metadata_profiles list
|
5034
|
-
faspex5 admin res node list
|
5589
|
+
faspex5 admin res node list
|
5035
5590
|
faspex5 admin res oauth_clients list
|
5036
5591
|
faspex5 admin res registrations list
|
5037
5592
|
faspex5 admin res saml_configs list
|
5038
|
-
faspex5 admin res shared_inboxes invite %name:
|
5593
|
+
faspex5 admin res shared_inboxes invite %name:my_shared_inbox_name johnny@example.com
|
5039
5594
|
faspex5 admin res shared_inboxes list
|
5040
|
-
faspex5 admin res shared_inboxes members %name:
|
5041
|
-
faspex5 admin res shared_inboxes members %name:
|
5042
|
-
faspex5 admin res shared_inboxes members %name:
|
5043
|
-
faspex5 admin res shared_inboxes members %name:
|
5595
|
+
faspex5 admin res shared_inboxes members %name:my_shared_inbox_name create %name:john@example.com
|
5596
|
+
faspex5 admin res shared_inboxes members %name:my_shared_inbox_name delete %name:john@example.com
|
5597
|
+
faspex5 admin res shared_inboxes members %name:my_shared_inbox_name delete %name:johnny@example.com
|
5598
|
+
faspex5 admin res shared_inboxes members %name:my_shared_inbox_name list
|
5044
5599
|
faspex5 admin res workgroups list
|
5045
5600
|
faspex5 admin smtp show
|
5046
5601
|
faspex5 admin smtp test my_email_external
|
5047
5602
|
faspex5 bearer_token
|
5048
|
-
faspex5 gateway https://localhost:12345/aspera/faspex
|
5603
|
+
faspex5 gateway --pid-file=server_pid https://localhost:12345/aspera/faspex
|
5049
5604
|
faspex5 health
|
5050
|
-
faspex5 packages list --box=
|
5051
|
-
faspex5 packages list --box=
|
5605
|
+
faspex5 packages list --box=my_shared_inbox_name
|
5606
|
+
faspex5 packages list --box=my_workgroup --group-type=workgroups
|
5052
5607
|
faspex5 packages list --query=@json:'{"mailbox":"inbox","state":["released"]}'
|
5053
|
-
faspex5 packages receive
|
5054
|
-
faspex5 packages receive --box=
|
5055
|
-
faspex5 packages receive --box=my_faspex5_workgroup --group-type=workgroups "my_package_id" --to-folder=.
|
5608
|
+
faspex5 packages receive --box=my_shared_inbox_name package_box_id1 --to-folder=.
|
5609
|
+
faspex5 packages receive --box=my_workgroup --group-type=workgroups workgroup_package_id1 --to-folder=.
|
5056
5610
|
faspex5 packages receive ALL --once-only=yes --to-folder=.
|
5057
5611
|
faspex5 packages receive INIT --once-only=yes
|
5058
|
-
faspex5 packages
|
5059
|
-
faspex5 packages send @json:'{"title":"test title","recipients":["
|
5060
|
-
faspex5 packages send @json:'{"title":"test title","recipients":[
|
5061
|
-
faspex5 packages
|
5062
|
-
faspex5 packages
|
5063
|
-
faspex5 packages show --box=
|
5064
|
-
faspex5
|
5612
|
+
faspex5 packages receive f5_p31 --to-folder=. --ts=@json:'{"content_protection_password":"my_secret_here"}'
|
5613
|
+
faspex5 packages send --shared-folder=%name:my_shared_folder_name @json:'{"title":"test title","recipients":["my_email_internal"]}' my_shared_folder_file
|
5614
|
+
faspex5 packages send @json:'{"title":"test title","recipients":["my_shared_inbox_name"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' test_file.bin
|
5615
|
+
faspex5 packages send @json:'{"title":"test title","recipients":["my_workgroup"]}' test_file.bin
|
5616
|
+
faspex5 packages send @json:'{"title":"test title","recipients":[{"name":"my_username"}]my_meta}' test_file.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
|
5617
|
+
faspex5 packages show --box=my_shared_inbox_name package_box_id1
|
5618
|
+
faspex5 packages show --box=my_workgroup --group-type=workgroups workgroup_package_id1
|
5619
|
+
faspex5 packages show f5_p31
|
5620
|
+
faspex5 packages status f5_p31
|
5621
|
+
faspex5 postprocessing --pid-file=server_pid @json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":"tests"}}'
|
5622
|
+
faspex5 shared browse %name:my_src
|
5623
|
+
faspex5 shared list
|
5624
|
+
faspex5 shared_folders browse %name:my_shared_folder_name
|
5625
|
+
faspex5 shared_folders list
|
5065
5626
|
faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
|
5066
5627
|
faspex5 user profile show
|
5067
5628
|
```
|
5068
5629
|
|
5069
|
-
### Faspex 5:
|
5630
|
+
### Faspex 5: Inbox selection
|
5070
5631
|
|
5071
5632
|
By default, package operations (send, receive, list) are done on the user's inbox.
|
5072
5633
|
|
@@ -5150,7 +5711,15 @@ The following parameters in option `query` are supported:
|
|
5150
5711
|
|
5151
5712
|
Admin only: If the value `ALL` is provided to option `box`, then all packages are selected.
|
5152
5713
|
|
5153
|
-
### Faspex 5: List all shared inboxes
|
5714
|
+
### Faspex 5: List all shared inboxes and work groups
|
5715
|
+
|
5716
|
+
If you are a regular user, to list work groups you belong to:
|
5717
|
+
|
5718
|
+
```bash
|
5719
|
+
ascli faspex5 admin res workgroup list
|
5720
|
+
```
|
5721
|
+
|
5722
|
+
If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
|
5154
5723
|
|
5155
5724
|
```bash
|
5156
5725
|
ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
|
@@ -5159,16 +5728,16 @@ ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,nam
|
|
5159
5728
|
Shared inbox members can also be listed, added, removed, and external users can be invited to a shared inbox.
|
5160
5729
|
|
5161
5730
|
```bash
|
5162
|
-
ascli faspex5 admin res shared_inboxes invite '%name:
|
5731
|
+
ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' john@example.com
|
5163
5732
|
```
|
5164
5733
|
|
5165
5734
|
It is equivalent to:
|
5166
5735
|
|
5167
5736
|
```bash
|
5168
|
-
ascli faspex5 admin res shared_inboxes invite '%name:
|
5737
|
+
ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
5169
5738
|
```
|
5170
5739
|
|
5171
|
-
Other payload parameters are possible
|
5740
|
+
Other payload parameters are possible the `Hash` value:
|
5172
5741
|
|
5173
5742
|
```json
|
5174
5743
|
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
|
@@ -5210,7 +5779,7 @@ ascli faspex5 packages send @json:'{"title":"hello","recipients":[{"name":"_reci
|
|
5210
5779
|
|
5211
5780
|
> **Note:** The shared folder can be identified by its numerical `id` or by name using percent selector: `%<field>:<value>`. e.g. `--shared-folder=3`
|
5212
5781
|
|
5213
|
-
### Faspex 5:
|
5782
|
+
### Faspex 5: Receive all packages (cargo)
|
5214
5783
|
|
5215
5784
|
To receive all packages, only once, through persistency of already received packages:
|
5216
5785
|
|
@@ -5310,7 +5879,7 @@ The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/cat
|
|
5310
5879
|
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
5311
5880
|
By default `count` is `0` (`10`), it can be increased to issue less HTTP calls.
|
5312
5881
|
|
5313
|
-
#### Example:
|
5882
|
+
#### Example: List packages in dropbox
|
5314
5883
|
|
5315
5884
|
```bash
|
5316
5885
|
ascli faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
|
@@ -5342,13 +5911,14 @@ if `id` is set to `ALL`, then all packages are downloaded, and if option `once_o
|
|
5342
5911
|
|
5343
5912
|
### Sending a Package
|
5344
5913
|
|
5345
|
-
The command is `faspex package send`.
|
5346
|
-
|
5914
|
+
The command is `faspex package send`.
|
5915
|
+
Package information (title, note, metadata, options) is provided in option `delivery_info`.
|
5916
|
+
The contents of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages).
|
5347
5917
|
|
5348
5918
|
Example:
|
5349
5919
|
|
5350
5920
|
```bash
|
5351
|
-
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["
|
5921
|
+
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
5352
5922
|
```
|
5353
5923
|
|
5354
5924
|
If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
|
@@ -5359,19 +5929,29 @@ Additional optional parameters in `delivery_info`:
|
|
5359
5929
|
- Package Note: : `"note":"note this and that"`
|
5360
5930
|
- Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
5361
5931
|
|
5932
|
+
It is possible to send from a remote source using option `remote_source`, providing either the numerical id, or the name of the remote source using percent selector: `%name:<name>`.
|
5933
|
+
|
5934
|
+
Remote source can be browsed if option `storage` is provided.
|
5935
|
+
`storage` is a `Hash` extended value.
|
5936
|
+
The key is the storage name, as listed in `source list` command.
|
5937
|
+
The value is a `Hash` with the following keys:
|
5938
|
+
|
5939
|
+
- `node` is a `Hash` with keys: `url`, `username`, `password`
|
5940
|
+
- `path` is the subpath inside the node, as configured in Faspex
|
5941
|
+
|
5362
5942
|
### Email notification on transfer
|
5363
5943
|
|
5364
|
-
Like for any transfer, a notification can be sent by email using parameters: `
|
5944
|
+
Like for any transfer, a notification can be sent by email using parameters: `notify_to` and `notify_template` .
|
5365
5945
|
|
5366
5946
|
Example:
|
5367
5947
|
|
5368
5948
|
```bash
|
5369
|
-
ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --
|
5949
|
+
ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notify-to=aspera.user1@gmail.com --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
|
5370
5950
|
```
|
5371
5951
|
|
5372
5952
|
In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
|
5373
5953
|
|
5374
|
-
### Operation on dropboxes
|
5954
|
+
### Operation on "dropboxes"
|
5375
5955
|
|
5376
5956
|
Example:
|
5377
5957
|
|
@@ -5406,9 +5986,9 @@ my_faspex_node:
|
|
5406
5986
|
|
5407
5987
|
In this example, a faspex storage named `my_storage` exists in Faspex, and is located
|
5408
5988
|
under the docroot in `/mydir` (this must be the same as configured in Faspex).
|
5409
|
-
The node configuration name is
|
5989
|
+
The node configuration name is `my_faspex_node` here.
|
5410
5990
|
|
5411
|
-
Note
|
5991
|
+
> **Note:** the v4 API provides an API for nodes and shares.
|
5412
5992
|
|
5413
5993
|
### Automated package download (cargo)
|
5414
5994
|
|
@@ -5423,30 +6003,30 @@ ascli faspex packages recv ALL --once-only=yes --lock-port=12345
|
|
5423
6003
|
|
5424
6004
|
```bash
|
5425
6005
|
faspex address_book
|
5426
|
-
faspex dropbox list --recipient="*
|
5427
|
-
faspex dropbox list --recipient="*my_faspex_wkg"
|
6006
|
+
faspex dropbox list --recipient="*my_dbx"
|
5428
6007
|
faspex health
|
5429
6008
|
faspex login_methods
|
5430
6009
|
faspex me
|
5431
|
-
faspex package list
|
5432
6010
|
faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
5433
6011
|
faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
|
5434
|
-
faspex package list --
|
5435
|
-
faspex package list --recipient="*
|
5436
|
-
faspex package
|
5437
|
-
faspex package recv "my_package_id" --to-folder=. --box=sent
|
6012
|
+
faspex package list --query=@json:'{"max":5}'
|
6013
|
+
faspex package list --recipient="*my_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
6014
|
+
faspex package list --recipient="*my_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
5438
6015
|
faspex package recv --to-folder=. --link=https://app.example.com/recv_from_user_path
|
5439
|
-
faspex package recv ALL --to-folder=. --once-only=yes
|
5440
|
-
faspex package recv
|
5441
|
-
faspex package recv
|
5442
|
-
faspex package
|
5443
|
-
faspex package
|
5444
|
-
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["
|
5445
|
-
faspex package send --
|
5446
|
-
faspex package send --
|
6016
|
+
faspex package recv ALL --to-folder=. --once-only=yes --query=@json:'{"max":10}'
|
6017
|
+
faspex package recv f4_db_id1 --recipient="*my_dbx" --to-folder=.
|
6018
|
+
faspex package recv f4_db_id2 --recipient="*my_wkg" --to-folder=.
|
6019
|
+
faspex package recv f4_pri1 --to-folder=.
|
6020
|
+
faspex package recv f4_prs2 --to-folder=. --box=sent
|
6021
|
+
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_dbx"]}' test_file.bin
|
6022
|
+
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_wkg"]}' test_file.bin
|
6023
|
+
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal","my_username"]}' test_file.bin
|
6024
|
+
faspex package send --delivery-info=@json:'{"title":"TIMESTAMP package remote","recipients":["my_email_internal"]}' --remote_source=%name:my_src sample_source.txt
|
6025
|
+
faspex package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"Important files delivery"}' test_file.bin
|
6026
|
+
faspex package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"Important files delivery"}' test_file.bin
|
6027
|
+
faspex source info %name:my_src --storage=@preset:faspex4_storage
|
5447
6028
|
faspex source list
|
5448
|
-
faspex source name
|
5449
|
-
faspex source name my_faspex_src node br /
|
6029
|
+
faspex source node %name:my_src br / --storage=@preset:faspex4_storage
|
5450
6030
|
faspex v4 dmembership list
|
5451
6031
|
faspex v4 dropbox list
|
5452
6032
|
faspex v4 metadata_profile list
|
@@ -5464,21 +6044,23 @@ Aspera Shares supports the "node API" for the file transfer part.
|
|
5464
6044
|
```bash
|
5465
6045
|
shares admin group list
|
5466
6046
|
shares admin node list
|
5467
|
-
shares admin share list --fields
|
6047
|
+
shares admin share list --fields=DEF,-status,status_message
|
5468
6048
|
shares admin share user_permissions 1 list
|
5469
6049
|
shares admin user add --type=ldap the_name
|
5470
6050
|
shares admin user app_authorizations 1 modify @json:'{"app_login":true}'
|
5471
6051
|
shares admin user app_authorizations 1 show
|
5472
6052
|
shares admin user import --type=saml @json:'{"id":"the_id","name_id":"the_name"}'
|
5473
6053
|
shares admin user list
|
6054
|
+
shares admin user list --type=local
|
5474
6055
|
shares admin user share_permissions 1 list
|
5475
6056
|
shares admin user share_permissions 1 show 1
|
5476
6057
|
shares files browse /
|
5477
|
-
shares files delete
|
5478
|
-
shares files download --to-folder=.
|
5479
|
-
shares files download --to-folder=.
|
5480
|
-
shares files upload --to-folder=
|
5481
|
-
shares files upload --to-folder=
|
6058
|
+
shares files delete my_share1/test_file.bin
|
6059
|
+
shares files download --to-folder=. my_share1/test_file.bin
|
6060
|
+
shares files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
|
6061
|
+
shares files upload --to-folder=my_share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
6062
|
+
shares files upload --to-folder=my_share1 test_file.bin
|
6063
|
+
shares files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn_port/aspera/http-gwy"}'
|
5482
6064
|
shares health
|
5483
6065
|
```
|
5484
6066
|
|
@@ -5490,7 +6072,7 @@ shares health
|
|
5490
6072
|
console health
|
5491
6073
|
console transfer current list
|
5492
6074
|
console transfer smart list
|
5493
|
-
console transfer smart sub
|
6075
|
+
console transfer smart sub my_smart_id @json:'{"source":{"paths":["my_smart_file"]},"source_type":"user_selected"}'
|
5494
6076
|
```
|
5495
6077
|
|
5496
6078
|
## <a id="orchestrator"></a>Plugin: `orchestrator`:IBM Aspera Orchestrator
|
@@ -5502,14 +6084,14 @@ orchestrator health
|
|
5502
6084
|
orchestrator info
|
5503
6085
|
orchestrator plugins
|
5504
6086
|
orchestrator processes
|
5505
|
-
orchestrator workflow details
|
5506
|
-
orchestrator workflow export
|
5507
|
-
orchestrator workflow inputs
|
6087
|
+
orchestrator workflow details my_workflow_id
|
6088
|
+
orchestrator workflow export my_workflow_id
|
6089
|
+
orchestrator workflow inputs my_workflow_id
|
5508
6090
|
orchestrator workflow list
|
5509
|
-
orchestrator workflow start
|
5510
|
-
orchestrator workflow start
|
6091
|
+
orchestrator workflow start my_workflow_id @json:'{"Param":"world !"}'
|
6092
|
+
orchestrator workflow start my_workflow_id @json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
|
5511
6093
|
orchestrator workflow status ALL
|
5512
|
-
orchestrator workflow status
|
6094
|
+
orchestrator workflow status my_workflow_id
|
5513
6095
|
```
|
5514
6096
|
|
5515
6097
|
## <a id="cos"></a>Plugin: `cos`: IBM Cloud Object Storage
|
@@ -5519,10 +6101,10 @@ It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Ser
|
|
5519
6101
|
Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
|
5520
6102
|
|
5521
6103
|
There are two possibilities to provide credentials.
|
5522
|
-
If you already have the endpoint,
|
6104
|
+
If you already have the endpoint, API key and CRN, use the first method.
|
5523
6105
|
If you don't have credentials but have access to the IBM Cloud console, then use the second method.
|
5524
6106
|
|
5525
|
-
### Using endpoint,
|
6107
|
+
### Using endpoint, API key and Resource Instance ID (CRN)
|
5526
6108
|
|
5527
6109
|
If you have those parameters already, then following options shall be provided:
|
5528
6110
|
|
@@ -5613,56 +6195,54 @@ ascli cos node info
|
|
5613
6195
|
ascli cos node upload 'faux:///sample1G?1g'
|
5614
6196
|
```
|
5615
6197
|
|
5616
|
-
Note
|
6198
|
+
> **Note:** we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man `ascp` and section above), but you can of course send a real file by specifying a real file instead.
|
5617
6199
|
|
5618
6200
|
### COS sample commands
|
5619
6201
|
|
5620
6202
|
```bash
|
5621
|
-
cos --bucket=
|
5622
|
-
cos --bucket=
|
6203
|
+
cos --bucket=my_bucket_name --endpoint=my_bucket_endpoint --apikey=my_bucket_apikey --crn=my_resource_instance_id node info
|
6204
|
+
cos --bucket=my_bucket_name --region=my_bucket_region --service-credentials=@json:@file:service_creds.json node info
|
5623
6205
|
cos node access_key show self
|
5624
|
-
cos node download
|
5625
|
-
cos node info
|
5626
|
-
cos node upload
|
6206
|
+
cos node download test_file.bin --to-folder=.
|
6207
|
+
cos node info --log-level=trace2
|
6208
|
+
cos node upload test_file.bin
|
5627
6209
|
```
|
5628
6210
|
|
5629
|
-
## <a id="async"></a>
|
6211
|
+
## <a id="async"></a>IBM Aspera Sync
|
6212
|
+
|
6213
|
+
An interface for the `async` utility is provided in the following plugins:
|
6214
|
+
|
6215
|
+
- server sync
|
6216
|
+
- node sync
|
6217
|
+
- aoc files sync (uses node)
|
6218
|
+
- shares files sync (uses node)
|
5630
6219
|
|
5631
|
-
|
5632
|
-
The main advantage over bare `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
6220
|
+
The main advantage over the `async` command line when using `server` is the possibility to use a configuration file, using standard options of `ascli`.
|
5633
6221
|
|
5634
|
-
|
5635
|
-
In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (including token).
|
6222
|
+
In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (e.g. including token).
|
5636
6223
|
|
5637
6224
|
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
|
5638
6225
|
|
5639
6226
|
Two JSON syntax are supported for option `sync_info`.
|
5640
6227
|
|
5641
|
-
### async
|
6228
|
+
### async JSON: API format
|
5642
6229
|
|
5643
|
-
It is the same payload as specified on the
|
6230
|
+
It is the same payload as specified on the option `--conf` of `async` or in node API `/asyncs`.
|
5644
6231
|
This is the preferred syntax and allows a single session definition.
|
5645
6232
|
But there is no progress output nor error messages.
|
5646
6233
|
|
5647
6234
|
Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
|
5648
6235
|
|
5649
|
-
### async
|
6236
|
+
### async JSON: Options mapping
|
5650
6237
|
|
5651
|
-
|
6238
|
+
`ascli` defines a JSON equivalent to regular `async`options.
|
5652
6239
|
It is based on a JSON representation of `async` command line options.
|
5653
6240
|
It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
|
5654
6241
|
|
5655
|
-
### Sync sample commands
|
5656
|
-
|
5657
|
-
```bash
|
5658
|
-
sync admin status --sync-info=@json:'{"sessions":[{"name":"test","local_dir":"contents"}]}'
|
5659
|
-
sync start --sync-info=@json:'{"instance":{"quiet":true},"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_paths":["my_local_user_key"]}]}'
|
5660
|
-
```
|
5661
|
-
|
5662
6242
|
## <a id="preview"></a>Plugin: `preview`: Preview generator for AoC
|
5663
6243
|
|
5664
6244
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
5665
|
-
It uses the **node API** of Aspera HSTS and requires use of Access Keys and
|
6245
|
+
It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
5666
6246
|
Several parameters can be used to tune several aspects:
|
5667
6247
|
|
5668
6248
|
- Methods for detection of new files needing generation
|
@@ -5684,7 +6264,7 @@ asconfigurator -x "server;preview_dir,previews"
|
|
5684
6264
|
asnodeadmin --reload
|
5685
6265
|
```
|
5686
6266
|
|
5687
|
-
Note
|
6267
|
+
> **Note:** the configuration `preview_dir` is **relative** to the storage root, no need leading or trailing `/`. In general just set the value to `previews`
|
5688
6268
|
|
5689
6269
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
5690
6270
|
|
@@ -5703,16 +6283,16 @@ asconfigurator -x "server; max_request_file_create_size_kb,16384"
|
|
5703
6283
|
|
5704
6284
|
If you use a value different than 16777216, then specify it using option `max_size`.
|
5705
6285
|
|
5706
|
-
Note
|
6286
|
+
> **Note:** the HSTS parameter (`max_request_file_create_size_kb`) is in **kiloBytes** while the generator parameter is in **Bytes** (factor of 1024).
|
5707
6287
|
|
5708
6288
|
### <a id="prev_ext"></a>External tools: Linux
|
5709
6289
|
|
5710
|
-
|
6290
|
+
`ascli` requires the following external tools available in the `PATH`:
|
5711
6291
|
|
5712
6292
|
- ImageMagick : `convert` `composite`
|
5713
|
-
- OptiPNG : `optipng`
|
5714
|
-
- FFmpeg : `ffmpeg` `ffprobe`
|
5715
|
-
- Libreoffice : `libreoffice`
|
6293
|
+
- "OptiPNG" : `optipng`
|
6294
|
+
- "FFmpeg" : `ffmpeg` `ffprobe`
|
6295
|
+
- "Libreoffice" : `libreoffice`
|
5716
6296
|
|
5717
6297
|
Here shown on Redhat/CentOS.
|
5718
6298
|
|
@@ -5724,7 +6304,7 @@ To check if all tools are found properly, execute:
|
|
5724
6304
|
ascli preview check
|
5725
6305
|
```
|
5726
6306
|
|
5727
|
-
#### Image: ImageMagick and optipng
|
6307
|
+
#### Image: ImageMagick and `optipng`
|
5728
6308
|
|
5729
6309
|
```bash
|
5730
6310
|
dnf install -y ImageMagick optipng
|
@@ -5734,6 +6314,8 @@ You may also install `ghostscript` which adds fonts to ImageMagick.
|
|
5734
6314
|
Available fonts, used to generate png for text, can be listed with `magick identify -list font`.
|
5735
6315
|
Prefer ImageMagick version >=7.
|
5736
6316
|
|
6317
|
+
More info on ImageMagick at <https://imagemagick.org/>
|
6318
|
+
|
5737
6319
|
#### Video: FFmpeg
|
5738
6320
|
|
5739
6321
|
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
@@ -5742,7 +6324,7 @@ The easiest method is to download and install the latest released version of ffm
|
|
5742
6324
|
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
|
5743
6325
|
```
|
5744
6326
|
|
5745
|
-
#### Office:
|
6327
|
+
#### Office: `unoconv` and Libreoffice
|
5746
6328
|
|
5747
6329
|
If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
|
5748
6330
|
|
@@ -5799,11 +6381,11 @@ This shall list the contents of the storage root of the access key.
|
|
5799
6381
|
|
5800
6382
|
When generating preview files, some options are provided by default.
|
5801
6383
|
Some values for the options can be modified on command line.
|
5802
|
-
For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a Hash with two keys: `in` and `out`, each is an
|
6384
|
+
For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a `Hash` with two keys: `in` and `out`, each is an `Array` of strings with the native options to `ffmpeg`.
|
5803
6385
|
|
5804
6386
|
### Execution
|
5805
6387
|
|
5806
|
-
|
6388
|
+
`ascli` intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
5807
6389
|
It needs to be run on a regular basis to create or update preview files.
|
5808
6390
|
For that use your best reliable scheduler, see [Scheduler](#scheduler).
|
5809
6391
|
|
@@ -5835,7 +6417,7 @@ case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
|
5835
6417
|
|
5836
6418
|
### Candidate detection for creation or update (or deletion)
|
5837
6419
|
|
5838
|
-
|
6420
|
+
`ascli` generates preview files using those commands:
|
5839
6421
|
|
5840
6422
|
- `trevents` : only recently uploaded files will be tested (transfer events)
|
5841
6423
|
- `events` : only recently uploaded files will be tested (file events: not working)
|
@@ -5861,13 +6443,9 @@ ascli preview scan --skip-folders=@json:'["/not_here"]'
|
|
5861
6443
|
|
5862
6444
|
The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
|
5863
6445
|
|
5864
|
-
When scanning the option `query` has the same behavior as for the `node find` command.
|
5865
|
-
|
5866
|
-
For instance to filter out files beginning with `._` do:
|
6446
|
+
When scanning the option `query` has the same behavior as for the `node access_keys do self find` command.
|
5867
6447
|
|
5868
|
-
|
5869
|
-
--query='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
|
5870
|
-
```
|
6448
|
+
Refer to that section for details.
|
5871
6449
|
|
5872
6450
|
### Preview File types
|
5873
6451
|
|
@@ -5896,28 +6474,28 @@ The mp4 video preview file is only for category `video`
|
|
5896
6474
|
|
5897
6475
|
File type is primarily based on file extension detected by the node API and translated info a mime type returned by the node API.
|
5898
6476
|
|
5899
|
-
### mimemagic
|
6477
|
+
### `mimemagic`
|
5900
6478
|
|
5901
6479
|
By default, the Mime type used for conversion is the one returned by the node API, based on file name extension.
|
5902
6480
|
|
5903
6481
|
It is also possible to detect the mime type using option `mimemagic`.
|
5904
6482
|
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`.
|
5905
6483
|
|
5906
|
-
This requires to manually install the mimemagic gem: `gem install mimemagic`.
|
6484
|
+
This requires to manually install the `mimemagic` gem: `gem install mimemagic`.
|
5907
6485
|
|
5908
|
-
In this case the `preview` command will first analyze the file content using mimemagic
|
6486
|
+
In this case the `preview` command will first analyze the file content using `mimemagic`, and if no match, will try by extension.
|
5909
6487
|
|
5910
6488
|
If the `mimemagic` gem complains about missing mime info file:
|
5911
6489
|
|
5912
6490
|
- any OS:
|
5913
6491
|
|
5914
6492
|
- Examine the error message
|
5915
|
-
- Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
|
5916
|
-
- move and rename this file to one of the locations expected by mimemagic as specified in the error message
|
6493
|
+
- Download the file: [`freedesktop.org.xml.in`](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
|
6494
|
+
- move and rename this file to one of the locations expected by `mimemagic` as specified in the error message
|
5917
6495
|
|
5918
6496
|
- Windows:
|
5919
6497
|
|
5920
|
-
- Download the file: [freedesktop.org.xml.in](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
|
6498
|
+
- Download the file: [`freedesktop.org.xml.in`](https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in)
|
5921
6499
|
- Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
|
5922
6500
|
- Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
|
5923
6501
|
|
@@ -5943,7 +6521,7 @@ brew install shared-mime-info
|
|
5943
6521
|
|
5944
6522
|
Standard open source tools are used to create thumbnails and video previews.
|
5945
6523
|
Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
|
5946
|
-
|
6524
|
+
`ascli` provides 2 ways to read and write files with the option: `file_access`
|
5947
6525
|
|
5948
6526
|
If the preview generator is run on a system that has direct access to the file system, then the value `local` can be used. In this case, no transfer happen, source files are directly read from the storage, and preview files
|
5949
6527
|
are directly written to the storage.
|
@@ -5956,14 +6534,14 @@ If the preview generator does not have access to files on the file system (it is
|
|
5956
6534
|
preview check --skip-types=office
|
5957
6535
|
preview scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
5958
6536
|
preview scan --skip-types=office --log-level=info
|
5959
|
-
preview
|
5960
|
-
preview
|
5961
|
-
preview
|
5962
|
-
preview
|
5963
|
-
preview
|
5964
|
-
preview
|
5965
|
-
preview test --
|
5966
|
-
preview test --
|
6537
|
+
preview show --base=test my_docx
|
6538
|
+
preview show --base=test my_mpg --video-png-conv=animated
|
6539
|
+
preview show --base=test my_mpg --video-png-conv=fixed
|
6540
|
+
preview show --base=test my_mpg mp4 --video-conversion=clips
|
6541
|
+
preview show --base=test my_mpg mp4 --video-conversion=reencode
|
6542
|
+
preview show --base=test my_pdf
|
6543
|
+
preview test --base=test my_dcm
|
6544
|
+
preview test --base=test my_mxf mp4 --video-conversion=blend --query=@json:'{"text":true,"double":true}'
|
5967
6545
|
preview trevents --once-only=yes --skip-types=office --log-level=info
|
5968
6546
|
```
|
5969
6547
|
|
@@ -5971,7 +6549,7 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
|
|
5971
6549
|
|
5972
6550
|
`ascli` can send email, for that setup SMTP configuration. This is done with option `smtp`.
|
5973
6551
|
|
5974
|
-
The `smtp` option is a
|
6552
|
+
The `smtp` option is a `Hash` (extended value) with the following fields:
|
5975
6553
|
|
5976
6554
|
<!-- markdownlint-disable MD034 -->
|
5977
6555
|
| field | default | example | description |
|
@@ -6034,16 +6612,16 @@ Check settings with `smtp_settings` command. Send test email with `email_test`.
|
|
6034
6612
|
|
6035
6613
|
```bash
|
6036
6614
|
ascli config --smtp=@preset:smtp_google smtp
|
6037
|
-
ascli config --smtp=@preset:smtp_google email --
|
6615
|
+
ascli config --smtp=@preset:smtp_google email --notify-to=sample.dest@example.com
|
6038
6616
|
```
|
6039
6617
|
|
6040
6618
|
### Notifications for transfer status
|
6041
6619
|
|
6042
6620
|
An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
|
6043
6621
|
|
6044
|
-
To activate, use option `
|
6622
|
+
To activate, use option `notify_to`.
|
6045
6623
|
|
6046
|
-
A default e-mail template is used, but it can be overridden with option `
|
6624
|
+
A default e-mail template is used, but it can be overridden with option `notify_template`.
|
6047
6625
|
|
6048
6626
|
The environment provided contains the following additional variables:
|
6049
6627
|
|
@@ -6076,24 +6654,24 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
|
|
6076
6654
|
|
6077
6655
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
6078
6656
|
|
6079
|
-
|
6657
|
+
`ascli` expect one single argument: a [*transfer-spec*](#transferspec).
|
6080
6658
|
|
6081
6659
|
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
|
6082
6660
|
|
6083
|
-
> **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the
|
6661
|
+
> **Note:** If JSON is the format, specify `@json:` to tell `ascli` to decode the `Hash` using JSON syntax.
|
6084
6662
|
|
6085
6663
|
During execution, it generates all low level events, one per line, in JSON format on stdout.
|
6086
6664
|
|
6087
6665
|
There are special "extended" [*transfer-spec*](#transferspec) parameters supported by `asession`:
|
6088
6666
|
|
6089
|
-
- `EX_loglevel` to change log level of
|
6667
|
+
- `EX_loglevel` to change log level of `ascli`
|
6090
6668
|
- `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
6091
6669
|
|
6092
6670
|
> **Note:** In addition, many "EX_" [*transfer-spec*](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [*transfer-spec*](#transferspec).
|
6093
6671
|
|
6094
6672
|
### Comparison of interfaces
|
6095
6673
|
|
6096
|
-
| feature/tool | asession | `ascp` | FaspManager | Transfer SDK |
|
6674
|
+
| feature/tool | `asession` | `ascp` | FaspManager | Transfer SDK |
|
6097
6675
|
|--------------|----------|--------|-------------|--------------|
|
6098
6676
|
| language integration | any | any | C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/> | many |
|
6099
6677
|
| required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
|
@@ -6203,7 +6781,7 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
6203
6781
|
|
6204
6782
|
Virtually any transfer on a "repository" on a regular basis might emulate a hot folder.
|
6205
6783
|
|
6206
|
-
> **Note:** file detection is not based on events (inotify
|
6784
|
+
> **Note:** file detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
6207
6785
|
>
|
6208
6786
|
> **Note:** parameters may be saved in a [option preset](#lprt) and used with `-P`.
|
6209
6787
|
|
@@ -6212,7 +6790,7 @@ Virtually any transfer on a "repository" on a regular basis might emulate a hot
|
|
6212
6790
|
Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
6213
6791
|
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
6214
6792
|
|
6215
|
-
### Example:
|
6793
|
+
### Example: Upload hot folder
|
6216
6794
|
|
6217
6795
|
```bash
|
6218
6796
|
ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"remove_after_transfer":true,"remove_empty_directories":true,"exclude_newer_than:-8,"src_base":"source_hot"}'
|
@@ -6223,7 +6801,7 @@ Source files are deleted after transfer.
|
|
6223
6801
|
Growing files will be sent only once they don't grow anymore (based on an 8-second cool-off period).
|
6224
6802
|
If a transfer takes more than the execution period, then the subsequent execution is skipped (`lock_port`) preventing multiple concurrent runs.
|
6225
6803
|
|
6226
|
-
### Example:
|
6804
|
+
### Example: Unidirectional synchronization (upload) to server
|
6227
6805
|
|
6228
6806
|
```bash
|
6229
6807
|
ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=12345 --ts=@json:'{"resume_policy":"sparse_csum","exclude_newer_than":-8,"src_base":"source_sync"}'
|
@@ -6231,11 +6809,10 @@ ascli server upload source_sync --to-folder=/Upload/target_sync --lock-port=1234
|
|
6231
6809
|
|
6232
6810
|
This can also be used with other folder-based applications: Aspera on Cloud, Shares, Node:
|
6233
6811
|
|
6234
|
-
### Example:
|
6812
|
+
### Example: Unidirectional synchronization (download) from Aspera on Cloud Files
|
6235
6813
|
|
6236
6814
|
```bash
|
6237
|
-
ascli aoc files download . --to-folder=. --lock-port=12345 --progress=
|
6238
|
-
--ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
|
6815
|
+
ascli aoc files download . --to-folder=. --lock-port=12345 --progress-bar=no --display=data --ts=@json:'{"resume_policy":"sparse_csum","target_rate_kbps":50000,"exclude_newer_than":-8,"delete_before_transfer":true}'
|
6239
6816
|
```
|
6240
6817
|
|
6241
6818
|
> **Note:** option `delete_before_transfer` will delete files locally, if they are not present on remote side.
|
@@ -6267,7 +6844,7 @@ Typically, the health check uses the REST API of the application with the follow
|
|
6267
6844
|
`ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
|
6268
6845
|
|
6269
6846
|
```bash
|
6270
|
-
ascli server health transfer --to-folder=/Upload --format=nagios --progress=
|
6847
|
+
ascli server health transfer --to-folder=/Upload --format=nagios --progress-bar=no
|
6271
6848
|
```
|
6272
6849
|
|
6273
6850
|
```output
|
@@ -6305,8 +6882,8 @@ Also, because there was already the `ascp` tool, I thought of an extended tool :
|
|
6305
6882
|
|
6306
6883
|
There were a few pitfalls:
|
6307
6884
|
|
6308
|
-
-
|
6309
|
-
-
|
6885
|
+
- `ascli` was written in the aging `perl` language while most Aspera web application products (but the Transfer Server) are written in `ruby`.
|
6886
|
+
- `ascli` was only for transfers, but not able to call other products APIs
|
6310
6887
|
|
6311
6888
|
So, it evolved into `ascli`:
|
6312
6889
|
|
@@ -6346,7 +6923,7 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
6346
6923
|
Some Ruby gems dependencies require compilation of native parts (C).
|
6347
6924
|
This also requires Ruby header files.
|
6348
6925
|
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
6349
|
-
`ruby-dev`
|
6926
|
+
`ruby-dev` or `ruby-devel`, depending on distribution.
|
6350
6927
|
|
6351
6928
|
### ED255519 key not supported
|
6352
6929
|
|