aspera-cli 4.17.0 → 4.18.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +3 -4
- data/CHANGELOG.md +33 -0
- data/CONTRIBUTING.md +15 -1
- data/README.md +711 -432
- data/bin/ascli +5 -0
- data/bin/asession +2 -2
- data/examples/build_package.sh +28 -0
- data/lib/aspera/agent/alpha.rb +10 -8
- data/lib/aspera/agent/base.rb +9 -6
- data/lib/aspera/agent/connect.rb +7 -8
- data/lib/aspera/agent/direct.rb +56 -37
- data/lib/aspera/agent/httpgw.rb +23 -324
- data/lib/aspera/agent/node.rb +19 -20
- data/lib/aspera/agent/trsdk.rb +19 -20
- data/lib/aspera/api/aoc.rb +17 -14
- data/lib/aspera/api/cos_node.rb +4 -4
- data/lib/aspera/api/httpgw.rb +342 -0
- data/lib/aspera/api/node.rb +135 -89
- data/lib/aspera/ascmd.rb +4 -3
- data/lib/aspera/ascp/installation.rb +15 -7
- data/lib/aspera/ascp/management.rb +2 -2
- data/lib/aspera/ascp/products.rb +1 -1
- data/lib/aspera/cli/basic_auth_plugin.rb +5 -9
- data/lib/aspera/cli/extended_value.rb +35 -16
- data/lib/aspera/cli/formatter.rb +161 -70
- data/lib/aspera/cli/hints.rb +18 -0
- data/lib/aspera/cli/main.rb +32 -39
- data/lib/aspera/cli/manager.rb +151 -119
- data/lib/aspera/cli/plugin.rb +27 -21
- data/lib/aspera/cli/plugin_factory.rb +31 -20
- data/lib/aspera/cli/plugins/alee.rb +14 -2
- data/lib/aspera/cli/plugins/aoc.rb +152 -141
- data/lib/aspera/cli/plugins/ats.rb +1 -1
- data/lib/aspera/cli/plugins/config.rb +72 -65
- data/lib/aspera/cli/plugins/console.rb +8 -5
- data/lib/aspera/cli/plugins/faspex.rb +32 -23
- data/lib/aspera/cli/plugins/faspex5.rb +232 -156
- data/lib/aspera/cli/plugins/faspio.rb +85 -0
- data/lib/aspera/cli/plugins/httpgw.rb +55 -0
- data/lib/aspera/cli/plugins/node.rb +129 -64
- data/lib/aspera/cli/plugins/orchestrator.rb +33 -30
- data/lib/aspera/cli/plugins/preview.rb +7 -3
- data/lib/aspera/cli/plugins/server.rb +6 -6
- data/lib/aspera/cli/plugins/shares.rb +16 -14
- data/lib/aspera/cli/special_values.rb +13 -0
- data/lib/aspera/cli/sync_actions.rb +10 -10
- data/lib/aspera/cli/transfer_agent.rb +7 -6
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/environment.rb +70 -9
- data/lib/aspera/faspex_gw.rb +5 -4
- data/lib/aspera/faspex_postproc.rb +2 -2
- data/lib/aspera/log.rb +6 -3
- data/lib/aspera/node_simulator.rb +2 -2
- data/lib/aspera/oauth/base.rb +31 -19
- data/lib/aspera/oauth/factory.rb +12 -13
- data/lib/aspera/oauth/generic.rb +1 -0
- data/lib/aspera/oauth/jwt.rb +18 -15
- data/lib/aspera/oauth/url_json.rb +8 -6
- data/lib/aspera/oauth/web.rb +2 -2
- data/lib/aspera/persistency_folder.rb +2 -2
- data/lib/aspera/preview/generator.rb +3 -3
- data/lib/aspera/preview/options.rb +3 -3
- data/lib/aspera/preview/terminal.rb +4 -4
- data/lib/aspera/preview/utils.rb +3 -3
- data/lib/aspera/proxy_auto_config.rb +5 -1
- data/lib/aspera/rest.rb +105 -88
- data/lib/aspera/rest_call_error.rb +1 -1
- data/lib/aspera/rest_error_analyzer.rb +2 -2
- data/lib/aspera/rest_errors_aspera.rb +1 -1
- data/lib/aspera/resumer.rb +1 -1
- data/lib/aspera/secret_hider.rb +2 -4
- data/lib/aspera/ssh.rb +1 -1
- data/lib/aspera/transfer/parameters.rb +39 -36
- data/lib/aspera/transfer/spec.rb +2 -0
- data/lib/aspera/transfer/sync.rb +2 -1
- data/lib/aspera/transfer/uri.rb +1 -1
- data/lib/aspera/uri_reader.rb +5 -4
- data/lib/aspera/web_auth.rb +1 -1
- data/lib/aspera/web_server_simple.rb +4 -3
- data.tar.gz.sig +0 -0
- metadata +7 -4
- metadata.gz.sig +0 -0
- data/lib/aspera/cli/plugins/bss.rb +0 -71
- data/lib/aspera/open_application.rb +0 -71
data/README.md
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
## Introduction
|
|
12
12
|
|
|
13
|
-
Version : 4.
|
|
13
|
+
Version : 4.18.1
|
|
14
14
|
|
|
15
15
|
Laurent/2016-2024
|
|
16
16
|
|
|
@@ -70,7 +70,7 @@ For scripting and ad'hoc command line operations, `ascli` is perfect.
|
|
|
70
70
|
|
|
71
71
|
Command line operations examples are shown using a shell such as: `bash` or `zsh`.
|
|
72
72
|
|
|
73
|
-
Command line
|
|
73
|
+
Command line arguments beginning with `my_` in examples, e.g. `my_param_value`, are user-provided value and not fixed value commands.
|
|
74
74
|
|
|
75
75
|
`ascli` is an API **Client** toward the remote Aspera application **Server** (Faspex, HSTS, etc...)
|
|
76
76
|
|
|
@@ -92,7 +92,7 @@ Once the gem is installed, `ascli` shall be accessible:
|
|
|
92
92
|
|
|
93
93
|
```console
|
|
94
94
|
$ ascli --version
|
|
95
|
-
4.
|
|
95
|
+
4.18.1
|
|
96
96
|
```
|
|
97
97
|
|
|
98
98
|
### First use
|
|
@@ -122,10 +122,10 @@ ascli server browse /
|
|
|
122
122
|
+------------+--------+-----------+-------+---------------------------+-----------------------+
|
|
123
123
|
```
|
|
124
124
|
|
|
125
|
-
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset)
|
|
125
|
+
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [Option Preset](#option-preset) for the server's authentication options.
|
|
126
126
|
The following example will:
|
|
127
127
|
|
|
128
|
-
- Create a [Option Preset](#option-preset)
|
|
128
|
+
- Create a [Option Preset](#option-preset)
|
|
129
129
|
- Define it as default for the `server` plugin
|
|
130
130
|
- List files in a folder
|
|
131
131
|
- Download a file
|
|
@@ -188,8 +188,8 @@ It is possible to install **either** directly on the host operating system (Linu
|
|
|
188
188
|
The direct installation is recommended and consists in installing:
|
|
189
189
|
|
|
190
190
|
- [Ruby](#ruby)
|
|
191
|
-
- [aspera-cli](#ruby-gem)
|
|
192
|
-
- [Aspera SDK (`ascp`)](#fasp-protocol)
|
|
191
|
+
- [aspera-cli](#ruby-gem-aspera-cli) <!-- markdownlint-disable-line -->
|
|
192
|
+
- [Aspera SDK (`ascp`)](#fasp-protocol-ascp)
|
|
193
193
|
|
|
194
194
|
Ruby version: >= 2.6.
|
|
195
195
|
|
|
@@ -253,7 +253,7 @@ Install the chosen pre-compiled Ruby version:
|
|
|
253
253
|
rvm install 3.2.2
|
|
254
254
|
```
|
|
255
255
|
|
|
256
|
-
Ruby is now installed for the user, go to [Gem installation](#ruby-gem).
|
|
256
|
+
Ruby is now installed for the user, go to [Gem installation](#ruby-gem-aspera-cli). <!-- markdownlint-disable-line -->
|
|
257
257
|
|
|
258
258
|
Alternatively RVM can be installed system-wide, for this execute as `root`.
|
|
259
259
|
It then installs by default in `/usr/local/rvm` for all users and creates `/etc/profile.d/rvm.sh`.
|
|
@@ -300,52 +300,6 @@ Download the Ruby installer executable from <https://rubyinstaller.org/downloads
|
|
|
300
300
|
rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
|
|
301
301
|
```
|
|
302
302
|
|
|
303
|
-
Installation without network:
|
|
304
|
-
|
|
305
|
-
It is essentially the same procedure, but instead of retrieving files from internet, copy the files from a machine with internet access, and then install from those archives:
|
|
306
|
-
|
|
307
|
-
- Download the `exe` Ruby installer from <https://rubyinstaller.org/downloads/>
|
|
308
|
-
|
|
309
|
-
```bash
|
|
310
|
-
v=$(curl -s https://rubyinstaller.org/downloads/|sed -nEe 's|.*(https://.*/releases/download/.*exe).*|\1|p'|head -n 1)
|
|
311
|
-
curl -o ${v##*/} $v
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
- Create an archive with necessary gems: <https://help.rubygems.org/kb/rubygems/installing-gems-with-no-network>
|
|
315
|
-
|
|
316
|
-
```bat
|
|
317
|
-
gem install aspera-cli -N -i my_gems
|
|
318
|
-
```
|
|
319
|
-
|
|
320
|
-
Zip the files `*.gem` from folder `repo/my_gems`
|
|
321
|
-
|
|
322
|
-
- Download the SDK from: <https://ibm.biz/aspera_sdk>
|
|
323
|
-
|
|
324
|
-
Create a Zip with all those files, and transfer to the target system.
|
|
325
|
-
|
|
326
|
-
Then, on the target system:
|
|
327
|
-
|
|
328
|
-
- Unzip the archive
|
|
329
|
-
- Execute the installer:
|
|
330
|
-
|
|
331
|
-
```bat
|
|
332
|
-
rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
|
|
333
|
-
```
|
|
334
|
-
|
|
335
|
-
- Install the gems:
|
|
336
|
-
|
|
337
|
-
```bat
|
|
338
|
-
gem install --force --local *.gem
|
|
339
|
-
```
|
|
340
|
-
|
|
341
|
-
- Install the SDK
|
|
342
|
-
|
|
343
|
-
```bash
|
|
344
|
-
ascli config ascp install --sdk-url=file:///sdk.zip
|
|
345
|
-
```
|
|
346
|
-
|
|
347
|
-
> **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
|
|
348
|
-
|
|
349
303
|
#### macOS: `brew`
|
|
350
304
|
|
|
351
305
|
**macOS** come with Ruby.
|
|
@@ -388,6 +342,12 @@ If your Linux distribution provides a standard Ruby package, you can use it prov
|
|
|
388
342
|
dnf install -y ruby-devel
|
|
389
343
|
```
|
|
390
344
|
|
|
345
|
+
**Example:** Ubuntu
|
|
346
|
+
|
|
347
|
+
```bash
|
|
348
|
+
apt-get install -y ruby-full
|
|
349
|
+
```
|
|
350
|
+
|
|
391
351
|
**Other examples:**
|
|
392
352
|
|
|
393
353
|
```bash
|
|
@@ -404,6 +364,26 @@ One can remove all installed gems, for example to start fresh:
|
|
|
404
364
|
gem uninstall -axI $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
|
405
365
|
```
|
|
406
366
|
|
|
367
|
+
#### Linux as simple user
|
|
368
|
+
|
|
369
|
+
If you don't have root access, you can install Ruby in your home directory using `rbenv` see [rbenv-installer](https://github.com/rbenv/rbenv-installer#rbenv-installer):
|
|
370
|
+
|
|
371
|
+
```bash
|
|
372
|
+
curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-installer | bash
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
Then open a new terminal, or "source" the shell initialization script:
|
|
376
|
+
|
|
377
|
+
```bash
|
|
378
|
+
source ~/.bashrc
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
Then install Ruby:
|
|
382
|
+
|
|
383
|
+
```bash
|
|
384
|
+
rbenv install 3.2.2
|
|
385
|
+
```
|
|
386
|
+
|
|
407
387
|
#### Other Unixes (AIX)
|
|
408
388
|
|
|
409
389
|
Ruby is sometimes made available as an installable package through third party providers.
|
|
@@ -479,7 +459,7 @@ gem install rmagick grpc mimemagic
|
|
|
479
459
|
|
|
480
460
|
> **Note:** Those are not installed as part of dependencies because they involve compilation of native code.
|
|
481
461
|
|
|
482
|
-
### Ruby Gem
|
|
462
|
+
### Ruby Gem: `aspera-cli`
|
|
483
463
|
|
|
484
464
|
Once you have Ruby and rights to install gems, install the `aspera-cli` gem and its dependencies:
|
|
485
465
|
|
|
@@ -502,7 +482,26 @@ To check if a new version is available (independently of `version_check_days`):
|
|
|
502
482
|
ascli config check_update
|
|
503
483
|
```
|
|
504
484
|
|
|
505
|
-
|
|
485
|
+
#### Gem installation with signature verification
|
|
486
|
+
|
|
487
|
+
The gem is signed with a private key, and the public certificate is available in the github repository (`certs/aspera-cli-public-cert.pem`).
|
|
488
|
+
When installing the gem, the signature can be optionally verified.
|
|
489
|
+
|
|
490
|
+
For [secure installation](https://guides.rubygems.org/security/#using-gems), one can install the gem with the public key:
|
|
491
|
+
|
|
492
|
+
- import the verification certificate:
|
|
493
|
+
|
|
494
|
+
```bash
|
|
495
|
+
gem cert --add <(curl -Ls https://raw.githubusercontent.com/IBM/aspera-cli/main/certs/aspera-cli-public-cert.pem)
|
|
496
|
+
```
|
|
497
|
+
|
|
498
|
+
- Then, install the gem:
|
|
499
|
+
|
|
500
|
+
```bash
|
|
501
|
+
gem install -P MediumSecurity aspera-cli
|
|
502
|
+
```
|
|
503
|
+
|
|
504
|
+
### FASP Protocol: `ascp`
|
|
506
505
|
|
|
507
506
|
Most file transfers will be executed using the **FASP** protocol, using `ascp`.
|
|
508
507
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
|
@@ -519,7 +518,7 @@ ascli config ascp install
|
|
|
519
518
|
If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
|
|
520
519
|
|
|
521
520
|
```bash
|
|
522
|
-
curl -Lso sdk.zip https://ibm.biz/
|
|
521
|
+
curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
|
|
523
522
|
```
|
|
524
523
|
|
|
525
524
|
```bash
|
|
@@ -549,6 +548,18 @@ Refer to section: [Transfer Agents](#transfer-clients-agents)
|
|
|
549
548
|
|
|
550
549
|
> **Note:** No pre-packaged version is provided yet.
|
|
551
550
|
|
|
551
|
+
#### Gem files and dependencies
|
|
552
|
+
|
|
553
|
+
The sample script: [examples/build_package.sh](examples/build_package.sh) can be used to download all necessary gems and dependencies in a tar gz.
|
|
554
|
+
|
|
555
|
+
```console
|
|
556
|
+
$ ./build_package.sh aspera-cli 4.18.0
|
|
557
|
+
|
|
558
|
+
Archive: aspera-cli-4.18.0-gems.tgz
|
|
559
|
+
```
|
|
560
|
+
|
|
561
|
+
#### Unix-like
|
|
562
|
+
|
|
552
563
|
A method to build one is provided here:
|
|
553
564
|
|
|
554
565
|
The procedure:
|
|
@@ -570,7 +581,7 @@ ascli --show-config --fields=sdk_url
|
|
|
570
581
|
- Download the SDK archive from that URL
|
|
571
582
|
|
|
572
583
|
```bash
|
|
573
|
-
curl -Lso sdk.zip https://ibm.biz/
|
|
584
|
+
curl -Lso sdk.zip https://ibm.biz/aspera_transfer_sdk
|
|
574
585
|
```
|
|
575
586
|
|
|
576
587
|
- Transfer those 2 files to the target system
|
|
@@ -593,15 +604,51 @@ ascli config ascp install --sdk-url=file:///sdk.zip
|
|
|
593
604
|
source ~/.rvm/scripts/rvm
|
|
594
605
|
```
|
|
595
606
|
|
|
596
|
-
|
|
607
|
+
#### Windows
|
|
608
|
+
|
|
609
|
+
Installation without network:
|
|
610
|
+
|
|
611
|
+
It is essentially the same procedure as installation for Windows with internet, but instead of retrieving files from internet, copy the files from a machine with internet access, and then install from those archives:
|
|
612
|
+
|
|
613
|
+
- Download the `exe` Ruby installer from <https://rubyinstaller.org/downloads/>
|
|
614
|
+
|
|
615
|
+
```bash
|
|
616
|
+
v=$(curl -s https://rubyinstaller.org/downloads/|sed -nEe 's|.*(https://.*/releases/download/.*exe).*|\1|p'|head -n 1)
|
|
617
|
+
curl -o ${v##*/} $v
|
|
618
|
+
```
|
|
619
|
+
|
|
620
|
+
- Create an archive with necessary gems like in previous section
|
|
621
|
+
|
|
622
|
+
- Download the SDK from: <https://ibm.biz/aspera_transfer_sdk>
|
|
623
|
+
|
|
624
|
+
- Create a Zip with all those files, and transfer to the target system.
|
|
625
|
+
|
|
626
|
+
Then, on the target system:
|
|
627
|
+
|
|
628
|
+
- Unzip the archive
|
|
629
|
+
- Execute the installer:
|
|
630
|
+
|
|
631
|
+
```bat
|
|
632
|
+
rubyinstaller-devkit-3.2.2-1-x64.exe /silent /currentuser /noicons /dir=C:\aspera-cli
|
|
633
|
+
```
|
|
634
|
+
|
|
635
|
+
- Install the gems: Extract the gem archive, and then:
|
|
636
|
+
|
|
637
|
+
```bat
|
|
638
|
+
gem install --force --local *.gem
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
- Install the SDK
|
|
597
642
|
|
|
598
643
|
```bash
|
|
599
|
-
|
|
644
|
+
ascli config ascp install --sdk-url=file:///sdk.zip
|
|
600
645
|
```
|
|
601
646
|
|
|
647
|
+
> **Note:** An example of installation script is provided: [docs/install.bat](docs/install.bat)
|
|
648
|
+
|
|
602
649
|
### Container
|
|
603
650
|
|
|
604
|
-
The container image is: [`martinlaurent/ascli`](https://hub.docker.com/r/martinlaurent/ascli).
|
|
651
|
+
The container image is: [`docker.io/martinlaurent/ascli`](https://hub.docker.com/r/docker.io/martinlaurent/ascli).
|
|
605
652
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
|
606
653
|
To use the container, ensure that you have `podman` (or `docker`) installed.
|
|
607
654
|
|
|
@@ -615,7 +662,7 @@ podman --version
|
|
|
615
662
|
Execute this:
|
|
616
663
|
|
|
617
664
|
```bash
|
|
618
|
-
podman run --rm --tty --interactive --entrypoint bash martinlaurent/ascli:latest
|
|
665
|
+
podman run --rm --tty --interactive --entrypoint bash docker.io/martinlaurent/ascli:latest
|
|
619
666
|
```
|
|
620
667
|
|
|
621
668
|
> **Note:** This command changes the entrypoint to an interactive shell instead of direct execution of `ascli`.
|
|
@@ -643,13 +690,13 @@ The entry point is `ascli` and the default command is `help`.
|
|
|
643
690
|
The container can be executed for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
|
644
691
|
|
|
645
692
|
```bash
|
|
646
|
-
podman run --rm --tty --interactive martinlaurent/ascli:latest
|
|
693
|
+
podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest
|
|
647
694
|
```
|
|
648
695
|
|
|
649
696
|
For more convenience, you may define a shell alias:
|
|
650
697
|
|
|
651
698
|
```bash
|
|
652
|
-
alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
|
|
699
|
+
alias ascli='podman run --rm --tty --interactive docker.io/martinlaurent/ascli:latest'
|
|
653
700
|
```
|
|
654
701
|
|
|
655
702
|
Then, you can execute the container like a local command:
|
|
@@ -659,7 +706,7 @@ ascli -v
|
|
|
659
706
|
```
|
|
660
707
|
|
|
661
708
|
```text
|
|
662
|
-
4.
|
|
709
|
+
4.18.1
|
|
663
710
|
```
|
|
664
711
|
|
|
665
712
|
In order to keep persistency of configuration on the host, you should specify your user's configuration folder as a volume for the container.
|
|
@@ -686,12 +733,12 @@ In this case you need also to specify the shared transfer folder as a volume:
|
|
|
686
733
|
--volume $HOME/xferdir:/xferfiles
|
|
687
734
|
```
|
|
688
735
|
|
|
689
|
-
> **Note:** ascli is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
|
736
|
+
> **Note:** `ascli` is run inside the container, so transfers are also executed inside the container and do not have access to host storage by default.
|
|
690
737
|
|
|
691
738
|
And if you want all the above, simply use all the options:
|
|
692
739
|
|
|
693
740
|
```bash
|
|
694
|
-
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
|
|
741
|
+
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash docker.io/martinlaurent/ascli:latest"
|
|
695
742
|
```
|
|
696
743
|
|
|
697
744
|
```bash
|
|
@@ -714,7 +761,7 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
|
714
761
|
|----------------|------------------------------------|--------------------------|--------------------------|
|
|
715
762
|
| `ASCLI_HOME` | Configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascli_config` |
|
|
716
763
|
| `docker_args` | Additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
|
717
|
-
| `image` | Container image name | `martinlaurent/ascli` | |
|
|
764
|
+
| `image` | Container image name | `docker.io/martinlaurent/ascli` | |
|
|
718
765
|
| `version` | Container image version | Latest | `4.8.0.pre` |
|
|
719
766
|
|
|
720
767
|
The wrapping script maps the folder `$ASCLI_HOME` on host to `/home/cliuser/.aspera/ascli` in the container.
|
|
@@ -748,8 +795,8 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
|
748
795
|
- First create the image archive:
|
|
749
796
|
|
|
750
797
|
```bash
|
|
751
|
-
podman pull martinlaurent/ascli
|
|
752
|
-
podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
|
798
|
+
podman pull docker.io/martinlaurent/ascli
|
|
799
|
+
podman save docker.io/martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
|
753
800
|
```
|
|
754
801
|
|
|
755
802
|
- Then, on air-gapped system:
|
|
@@ -787,7 +834,7 @@ dnf install singularity-ce
|
|
|
787
834
|
Build an image like this:
|
|
788
835
|
|
|
789
836
|
```bash
|
|
790
|
-
singularity build ascli.sif docker://martinlaurent/ascli
|
|
837
|
+
singularity build ascli.sif docker://docker.io/martinlaurent/ascli
|
|
791
838
|
```
|
|
792
839
|
|
|
793
840
|
Then, start `ascli` like this:
|
|
@@ -802,6 +849,124 @@ Or get a shell with access to `ascli` like this:
|
|
|
802
849
|
singularity shell ascli.sif
|
|
803
850
|
```
|
|
804
851
|
|
|
852
|
+
### SSL library
|
|
853
|
+
|
|
854
|
+
`ascli` uses the Ruby `openssl` gem which uses by default the system's `openssl` library and its CA certificate bundle.
|
|
855
|
+
|
|
856
|
+
To display the version of **openssl** used in `ascli`:
|
|
857
|
+
|
|
858
|
+
```bash
|
|
859
|
+
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
860
|
+
```
|
|
861
|
+
|
|
862
|
+
It is possible to specify to use another SSL library or version by executing:
|
|
863
|
+
|
|
864
|
+
```bash
|
|
865
|
+
gem install openssl -- --with-openssl-dir=[openssl library folder]
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
Where `[openssl library folder]` is the path to the folder containing the `lib` and `include` folders of the `openssl` library.
|
|
869
|
+
|
|
870
|
+
For example, on macOS, to use the `openssl@3` library installed with `brew`:
|
|
871
|
+
|
|
872
|
+
```bash
|
|
873
|
+
$ openssl version -e|sed -n 's|ENGINESDIR: "\(.*\)/lib[^/]*/.*|\1|p'
|
|
874
|
+
/opt/homebrew/Cellar/openssl@3/3.3.0
|
|
875
|
+
$ gem install openssl -- --with-openssl-dir=/opt/homebrew/Cellar/openssl@3/3.3.0
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
### SSL CA certificate bundle
|
|
879
|
+
|
|
880
|
+
SSL certificates are validated using a certificate store, by default it is the one of the system's `openssl` library.
|
|
881
|
+
|
|
882
|
+
To display trusted certificate store locations:
|
|
883
|
+
|
|
884
|
+
```bash
|
|
885
|
+
ascli --show-config --fields=cert_stores
|
|
886
|
+
```
|
|
887
|
+
|
|
888
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
|
889
|
+
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
|
890
|
+
|
|
891
|
+
One can display those default values:
|
|
892
|
+
|
|
893
|
+
```bash
|
|
894
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
|
895
|
+
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
In order to get certificate validation, the CA certificate bundle must be up-to-date.
|
|
899
|
+
Check this repository on how to update the system's CA certificate bundle: [https://github.com/millermatt/osca](https://github.com/millermatt/osca).
|
|
900
|
+
|
|
901
|
+
For example on RHEL/Rocky Linux:
|
|
902
|
+
|
|
903
|
+
```bash
|
|
904
|
+
dnf install -y ca-certificates
|
|
905
|
+
update-ca-trust extract
|
|
906
|
+
```
|
|
907
|
+
|
|
908
|
+
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders.
|
|
909
|
+
By default it uses Ruby's default certificate store.
|
|
910
|
+
|
|
911
|
+
If you use this option, then default locations are not used.
|
|
912
|
+
Default locations can be added using special value `DEF`.
|
|
913
|
+
The value can be either an `Array` or `String` (path).
|
|
914
|
+
Successive options add paths incrementally.
|
|
915
|
+
All files of a folders are added.
|
|
916
|
+
|
|
917
|
+
JRuby uses its own implementation and CA bundles.
|
|
918
|
+
|
|
919
|
+
For example, on Linux to force the use the system's certificate store:
|
|
920
|
+
|
|
921
|
+
```bash
|
|
922
|
+
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
|
923
|
+
```
|
|
924
|
+
|
|
925
|
+
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of **SSH**).
|
|
926
|
+
|
|
927
|
+
By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
|
|
928
|
+
Original `ascp`'s hardcoded locations can be found using:
|
|
929
|
+
|
|
930
|
+
```bash
|
|
931
|
+
ascli config ascp info --fields=openssldir
|
|
932
|
+
```
|
|
933
|
+
|
|
934
|
+
E.g. on macOS: `/Library/Aspera/ssl`.
|
|
935
|
+
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
|
936
|
+
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
|
937
|
+
|
|
938
|
+
To update trusted root certificates for `ascli`:
|
|
939
|
+
Display the trusted certificate store locations used by `ascli`.
|
|
940
|
+
Typically done by updating the system's root certificate store.
|
|
941
|
+
|
|
942
|
+
An up-to-date version of the certificate bundle can also be retrieved with:
|
|
943
|
+
|
|
944
|
+
```bash
|
|
945
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
|
946
|
+
```
|
|
947
|
+
|
|
948
|
+
To download that certificate store:
|
|
949
|
+
|
|
950
|
+
```bash
|
|
951
|
+
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text --output=/tmp/cacert.pem
|
|
952
|
+
```
|
|
953
|
+
|
|
954
|
+
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
|
955
|
+
|
|
956
|
+
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
|
957
|
+
|
|
958
|
+
```bash
|
|
959
|
+
ascli config remote_certificate chain https://localhost:9092 --insecure=yes --output=myserver.pem
|
|
960
|
+
```
|
|
961
|
+
|
|
962
|
+
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
|
963
|
+
|
|
964
|
+
Then, use this file as certificate store (e.g. here, Node API):
|
|
965
|
+
|
|
966
|
+
```bash
|
|
967
|
+
ascli config echo @uri:https://localhost:9092/ping --cert-stores=myserver.pem
|
|
968
|
+
```
|
|
969
|
+
|
|
805
970
|
## Command Line Interface
|
|
806
971
|
|
|
807
972
|
The command line tool is: ``ascli``
|
|
@@ -820,9 +985,9 @@ The `aspera-cli` gem provides a command line interface (CLI) which interacts wit
|
|
|
820
985
|
|
|
821
986
|
- Supports commands to Aspera server products (on-premise and SaaS)
|
|
822
987
|
- Any command line **options** (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
|
|
823
|
-
- Supports Commands, Option values
|
|
988
|
+
- Supports Commands, Options, and Option values shortcuts
|
|
824
989
|
- FASP [Transfer Agents](#transfer-clients-agents) can be: local `ascp`, or Connect Client, or any transfer node
|
|
825
|
-
- Transfer parameters can be altered by modification of [
|
|
990
|
+
- Transfer parameters can be altered by modification of [**transfer-spec**](#transfer-specification), this includes requiring multi-session
|
|
826
991
|
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
|
827
992
|
- Supports FaspStream creation (using Node API)
|
|
828
993
|
- Supports **Watchfolder** creation (using Node API)
|
|
@@ -856,19 +1021,20 @@ Moreover all `ascp` options are supported either through transfer spec parameter
|
|
|
856
1021
|
### Command line parsing, Special Characters
|
|
857
1022
|
|
|
858
1023
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
|
859
|
-
`ascli` receives its arguments
|
|
1024
|
+
`ascli` receives its arguments on the command line.
|
|
1025
|
+
The way arguments are parsed and provided to `ascli` depend on the Operating System and shell.
|
|
860
1026
|
|
|
861
1027
|
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
|
862
1028
|
|
|
863
|
-
Linux command line parsing is
|
|
1029
|
+
Linux command line parsing is well defined:
|
|
864
1030
|
It is fully documented in the shell's documentation.
|
|
865
1031
|
|
|
866
|
-
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
|
1032
|
+
On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh).
|
|
867
1033
|
A c-shell (`csh`, `tcsh`) or other shell can also be used.
|
|
868
1034
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
|
869
1035
|
See [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
|
870
1036
|
The shell builds the list of arguments and then `fork`/`exec` Ruby with that list.
|
|
871
|
-
Ruby receives a list
|
|
1037
|
+
Ruby receives a list command line arguments from shell and gives it to `ascli`.
|
|
872
1038
|
Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
|
|
873
1039
|
|
|
874
1040
|
#### Shell parsing for Windows
|
|
@@ -960,16 +1126,16 @@ Major Minor Build Revision
|
|
|
960
1126
|
----- ----- ----- --------
|
|
961
1127
|
5 1 19041 4046
|
|
962
1128
|
|
|
963
|
-
PS C:\> ascli
|
|
1129
|
+
PS C:\> ascli config echo --% @json:'{"k":"v","x":"y"}'
|
|
964
1130
|
|
|
965
|
-
PS C:\> ascli
|
|
1131
|
+
PS C:\> ascli config echo @json:'{"""k""":"""v""","""x""":"""y"""}'
|
|
966
1132
|
```
|
|
967
1133
|
|
|
968
1134
|
> **Note:** The special powershell argument `--%` places powershell in legacy parsing mode.
|
|
969
1135
|
|
|
970
1136
|
#### Extended Values (JSON, Ruby, ...)
|
|
971
1137
|
|
|
972
|
-
Some of the `ascli`
|
|
1138
|
+
Some of the values provided to `ascli` (options, **Command Parameters**) are expected to be [Extended Values](#extended-value-syntax), i.e. not a simple `String`, but a composite structure (`Hash`, `Array`).
|
|
973
1139
|
Typically, the `@json:` modifier is used, it expects a [JSON](https://www.json.org/) string.
|
|
974
1140
|
JSON itself has some special syntax: for example `"` is used to enclose a `String`.
|
|
975
1141
|
|
|
@@ -1154,16 +1320,15 @@ ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
|
|
|
1154
1320
|
{"title":"Test \" ' & \\"}
|
|
1155
1321
|
```
|
|
1156
1322
|
|
|
1157
|
-
###
|
|
1323
|
+
### Positional Arguments and Options
|
|
1158
1324
|
|
|
1159
1325
|
Command line arguments are the units of command line typically separated by spaces (the `argv` of C).
|
|
1160
1326
|
The tokenization of the command line is typically done by the shell, refer to the previous section [Command Line Parsing](#command-line-parsing-special-characters).
|
|
1161
1327
|
|
|
1162
|
-
`ascli` handles
|
|
1328
|
+
`ascli` handles two types of command line arguments:
|
|
1163
1329
|
|
|
1164
|
-
-
|
|
1165
|
-
-
|
|
1166
|
-
- Options
|
|
1330
|
+
- **Positional Arguments** : position is significant
|
|
1331
|
+
- **Options** : only order is significant, but not position
|
|
1167
1332
|
|
|
1168
1333
|
For example:
|
|
1169
1334
|
|
|
@@ -1171,18 +1336,24 @@ For example:
|
|
|
1171
1336
|
ascli command subcommand --option-name=VAL1 VAL2
|
|
1172
1337
|
```
|
|
1173
1338
|
|
|
1174
|
-
- Executes **
|
|
1175
|
-
- With one **
|
|
1176
|
-
- The command has one additional **positional argument**: `VAL2`
|
|
1339
|
+
- Executes **Command** and its **Command Parameters** (**Positional Arguments**): `command subcommand VAL2`
|
|
1340
|
+
- With one **Option**: `option_name` and its **value**: `VAL1`
|
|
1177
1341
|
|
|
1178
1342
|
If the value of a command, option or argument is constrained by a fixed list of values, then it is possible to use a few of the first letters of the value, provided that it uniquely identifies the value.
|
|
1179
1343
|
For example `ascli config pre ov` is the same as `ascli config preset overview`.
|
|
1180
1344
|
|
|
1181
|
-
The value of
|
|
1345
|
+
The value of **Options** and **Positional Arguments** is evaluated with the [Extended Value Syntax](#extended-value-syntax).
|
|
1346
|
+
|
|
1347
|
+
#### Positional Arguments
|
|
1348
|
+
|
|
1349
|
+
**Positional Arguments** are either:
|
|
1182
1350
|
|
|
1183
|
-
|
|
1351
|
+
- **Commands**, typically at the beginning
|
|
1352
|
+
- **Command Parameters** , e.g. creation data or entity identifier
|
|
1184
1353
|
|
|
1185
|
-
|
|
1354
|
+
When options are removed from the command line, the remaining arguments are typically **Positional Arguments** with a pre-defined order.
|
|
1355
|
+
|
|
1356
|
+
**Commands** are typically entity types or verbs to act on those entities.
|
|
1186
1357
|
Its value is a `String` that must belong to a fixed list of values in a given context.
|
|
1187
1358
|
|
|
1188
1359
|
Example:
|
|
@@ -1196,27 +1367,24 @@ ascli config ascp info
|
|
|
1196
1367
|
- `ascp` is the second level command: name of the component (singleton)
|
|
1197
1368
|
- `info` is the third level command: action to be performed
|
|
1198
1369
|
|
|
1199
|
-
Typically,
|
|
1370
|
+
Typically, **Commands** are located at the **beginning** of the command line.
|
|
1200
1371
|
Order is significant.
|
|
1201
1372
|
The provided command must match one of the supported commands in the given context.
|
|
1202
1373
|
If wrong, or no command is provided when expected, an error message is displayed and the list of supported commands is displayed.
|
|
1203
1374
|
|
|
1204
|
-
Some sub-commands appear after entity selection, e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
|
1205
|
-
|
|
1206
|
-
#### Positional Arguments
|
|
1207
|
-
|
|
1208
|
-
Positional Arguments are typically mandatory values for a command, such as entity creation data.
|
|
1209
|
-
|
|
1210
|
-
It could also be designed as an option, but since it is mandatory and typically creation parameters need not be set in a configuration file, then it is better to use a positional argument, and not define specific options.
|
|
1375
|
+
Some sub-commands appear after entity selection (identifier), e.g. `ascli aoc admin res node do 8669 browse /`: `browse` is a sub-command of `node`.
|
|
1211
1376
|
|
|
1212
|
-
|
|
1377
|
+
**Command Parameters** are typically mandatory values for a command, such as entity creation data or entity identifier.
|
|
1213
1378
|
|
|
1379
|
+
> **Note:** It could also have been designed as an option.
|
|
1380
|
+
But since it is mandatory and typically these data need **not** be set in a configuration file, then it is better designed as a **Command Parameter**, and not designed an additional specific option.
|
|
1381
|
+
The advantages of using a **Command Parameter** instead of an option for the same are that the command line is shorter (no option name, just the position) and the value is clearly mandatory.
|
|
1214
1382
|
The disadvantage is that it is not possible to define a default value in a configuration file or environment variable like for options.
|
|
1215
|
-
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file or environment variable (using `@
|
|
1383
|
+
Nevertheless, [Extended Values](#extended-value-syntax) syntax is supported, so it is possible to retrieve a value from the configuration file (using `@preset:`) or environment variable (using `@env:`).
|
|
1216
1384
|
|
|
1217
|
-
If a
|
|
1385
|
+
If a **Command Parameters** begins with `-`, then either use the `@val:` syntax (see [Extended Values](#extended-value-syntax)), or use the `--` separator (see below).
|
|
1218
1386
|
|
|
1219
|
-
A few
|
|
1387
|
+
A few **Command Parameters** are optional, they are located at the end of the command line.
|
|
1220
1388
|
|
|
1221
1389
|
#### Options
|
|
1222
1390
|
|
|
@@ -1225,13 +1393,15 @@ All options, e.g. `--log-level=debug`, are command line arguments that:
|
|
|
1225
1393
|
- Start with `--`
|
|
1226
1394
|
- Have a name, in lowercase, using `-` as word separator in name (e.g. `--log-level=debug`)
|
|
1227
1395
|
- Have a value, separated from name with a `=`
|
|
1228
|
-
- Can be used by prefix (
|
|
1396
|
+
- Can be used by prefix (not recommended), provided that it is unique. E.g. `--log-l=debug` is the same as `--log-level=debug`
|
|
1397
|
+
- Is optional on command line (it has a default value or no value)
|
|
1398
|
+
- whose position is not significant, but order is significant
|
|
1229
1399
|
|
|
1230
1400
|
Exceptions:
|
|
1231
1401
|
|
|
1232
1402
|
- Some options accept a short form, e.g. `-Ptoto` is equivalent to `--preset=toto`, refer to the manual or `-h`.
|
|
1233
1403
|
- Some options (flags) don't take a value, e.g. `-N`
|
|
1234
|
-
- The special option `--` stops option processing and is ignored, following command line arguments are taken as
|
|
1404
|
+
- The special option `--` stops option processing and is ignored, following command line arguments are taken as **Positional Arguments**, including the ones starting with a `-`.
|
|
1235
1405
|
|
|
1236
1406
|
Example:
|
|
1237
1407
|
|
|
@@ -1248,6 +1418,7 @@ ascli config echo -- --sample
|
|
|
1248
1418
|
Options may have a (hardcoded) default value.
|
|
1249
1419
|
|
|
1250
1420
|
Options can be placed anywhere on command line and are evaluated in order.
|
|
1421
|
+
Usually the last value evaluated overrides previous values, but some options are cumulative, e.g. `ts`.
|
|
1251
1422
|
|
|
1252
1423
|
Options are typically optional: to change the default behavior.
|
|
1253
1424
|
But some are mandatory, so they can be placed in a configuration file, for example: connection information.
|
|
@@ -1264,23 +1435,23 @@ Option `show_config` dry runs the configuration, and then returns currently set
|
|
|
1264
1435
|
`ascli --show-config` outputs global options only, and `ascli [plugin] --show-config` outputs global and plugin default options.
|
|
1265
1436
|
In addition, option `--show-config` can be added at the end of any full command line, this displays the options that would be used for the command.
|
|
1266
1437
|
|
|
1267
|
-
A
|
|
1438
|
+
A command line argument is typically designed as option if:
|
|
1268
1439
|
|
|
1269
1440
|
- It is optional, or
|
|
1270
|
-
- It is a mandatory parameter that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
|
1441
|
+
- It is a mandatory parameter with a default value that would benefit from being set persistently (i.e. in a configuration file or environment variable, e.g. URL and credentials).
|
|
1271
1442
|
|
|
1272
1443
|
### Interactive Input
|
|
1273
1444
|
|
|
1274
|
-
Some options and
|
|
1275
|
-
By default, `ascli` will ask for missing mandatory options or
|
|
1445
|
+
Some options and **Command Parameters** are mandatory and other optional.
|
|
1446
|
+
By default, `ascli` will ask for missing mandatory options or **Command Parameters** for interactive execution.
|
|
1276
1447
|
|
|
1277
1448
|
The behavior can be controlled with:
|
|
1278
1449
|
|
|
1279
1450
|
- `--interactive=<yes|no>` (default=yes if STDIN is a terminal, else no)
|
|
1280
|
-
- yes : missing mandatory parameters/
|
|
1281
|
-
- no : missing mandatory parameters/
|
|
1451
|
+
- yes : missing mandatory parameters/arguments are asked to the user
|
|
1452
|
+
- no : missing mandatory parameters/arguments raise an error message
|
|
1282
1453
|
- `--ask-options=<yes|no>` (default=no)
|
|
1283
|
-
- optional parameters/
|
|
1454
|
+
- optional parameters/arguments are asked to user
|
|
1284
1455
|
|
|
1285
1456
|
### Output
|
|
1286
1457
|
|
|
@@ -1303,25 +1474,36 @@ Depending on action, the output will contain:
|
|
|
1303
1474
|
#### Format of output
|
|
1304
1475
|
|
|
1305
1476
|
By default, result of type single_object and object_list are displayed using format `table`.
|
|
1306
|
-
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
|
1307
1477
|
|
|
1308
|
-
|
|
1309
|
-
|
|
1310
|
-
|
|
1311
|
-
When in flatten mode, it is possible to filter fields by compound field name using dot as separator.
|
|
1478
|
+
The table style can be customized with option: `table_style` which expects a `Hash`, options are the ones described in gem [`terminal-table`](https://github.com/tj/terminal-table).
|
|
1479
|
+
|
|
1480
|
+
For example, to display a table with thick unicode borders:
|
|
1312
1481
|
|
|
1313
|
-
|
|
1482
|
+
```bash
|
|
1483
|
+
ascli config preset over --table-style=@ruby:'{border: :unicode_thick_edge}'
|
|
1484
|
+
```
|
|
1485
|
+
|
|
1486
|
+
> **Note:** Other border styles exist, not limited to: `:unicode`, `:unicode_round`.
|
|
1487
|
+
|
|
1488
|
+
In a table format, when displaying **Objects** (single, or list), by default, sub object are flattened (option `flat_hash`).
|
|
1489
|
+
For example, object `{"user":{"id":1,"name":"toto"}}` will have attributes: `user.id` and `user.name`.
|
|
1490
|
+
Setting option `flat_hash` to `false` will only display one field: `user` and value is the sub `Hash`.
|
|
1491
|
+
When in flatten mode, it is possible to filter fields using the option `fields` using the compound field name using `.` (dot) as separator.
|
|
1492
|
+
|
|
1493
|
+
Object lists are displayed one per line, with attributes as columns.
|
|
1494
|
+
Single objects (or tables with a single result) are transposed: one attribute per line.
|
|
1314
1495
|
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
|
1315
1496
|
|
|
1316
|
-
The style of output can be set using the `format`
|
|
1497
|
+
The style of output can be set using the `format` option, supporting:
|
|
1317
1498
|
|
|
1499
|
+
- `table` : Text table (default)
|
|
1318
1500
|
- `text` : Value as String
|
|
1319
|
-
- `table` : Text table
|
|
1320
1501
|
- `ruby` : Ruby code
|
|
1321
1502
|
- `json` : JSON code
|
|
1322
1503
|
- `jsonpp` : JSON pretty printed
|
|
1323
1504
|
- `yaml` : YAML
|
|
1324
1505
|
- `csv` : Comma Separated Values
|
|
1506
|
+
- `image` : Image URL or Image data
|
|
1325
1507
|
|
|
1326
1508
|
#### Verbosity of output
|
|
1327
1509
|
|
|
@@ -1342,31 +1524,31 @@ To hide secrets from output, set option `show_secrets` to `no`.
|
|
|
1342
1524
|
|
|
1343
1525
|
#### Option: `fields`: Selection of output object properties
|
|
1344
1526
|
|
|
1345
|
-
By default, a table output will display one line per entry, and columns for each properties.
|
|
1346
|
-
Depending on the command, columns may include by default all
|
|
1527
|
+
By default, a `table` output format will display one line per entry, and columns for each fields (attributes, properties).
|
|
1528
|
+
Depending on the command, columns may include by default all fields, or only some selected fields.
|
|
1347
1529
|
It is possible to define specific columns to be displayed, by setting the `fields` option.
|
|
1348
1530
|
|
|
1349
|
-
The `fields` option can be either a comma separated list
|
|
1531
|
+
The `fields` option is a list that can be either a comma separated list or an extended value `Array`.
|
|
1350
1532
|
|
|
1351
|
-
|
|
1533
|
+
Individual elements of the list can be:
|
|
1352
1534
|
|
|
1353
|
-
- `DEF` : default display of columns (that's the default, when not set)
|
|
1354
|
-
- `ALL` : all columns available
|
|
1355
|
-
- `-`**property** : remove property from the current list
|
|
1356
1535
|
- **property** : add property to the current list
|
|
1357
|
-
-
|
|
1536
|
+
- `-`**property** : remove property from the current list
|
|
1537
|
+
- `DEF` : default list of properties (that's the default, when not set)
|
|
1538
|
+
- `ALL` : all properties
|
|
1539
|
+
- A Ruby `RegEx` : using `@ruby:'/.../'`, or `@re:...` add those matching to the list
|
|
1358
1540
|
|
|
1359
1541
|
Examples:
|
|
1360
1542
|
|
|
1361
|
-
- `a,b,c` : the list of attributes specified as a comma separated list
|
|
1362
|
-
- `@json:'["a","b","c"]'` : Array extended value: same as above
|
|
1363
|
-
- `DEF,-a
|
|
1543
|
+
- `a,b,c` : the list of attributes specified as a comma separated list (overrides the all default)
|
|
1544
|
+
- `@json:'["a","b","c"]'` : `Array` extended value: same as above
|
|
1545
|
+
- `b,DEF,-a` : default property list, remove `a` and add `b` in first position
|
|
1364
1546
|
- `@ruby:'/^server/'` : Display all properties whose name begin with `server`
|
|
1365
1547
|
|
|
1366
1548
|
#### Option: `select`
|
|
1367
1549
|
|
|
1368
1550
|
Table output (`object_list`) can be filtered using option `select`.
|
|
1369
|
-
This
|
|
1551
|
+
This option is either a `Hash` or `Proc`.
|
|
1370
1552
|
The `Proc` takes as argument a line (`Hash`) in the table and is a Ruby lambda expression that returns `true` or `false`.
|
|
1371
1553
|
|
|
1372
1554
|
Example:
|
|
@@ -1384,7 +1566,7 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
|
|
|
1384
1566
|
+-------------------------------+----------------------------------+-----------+
|
|
1385
1567
|
```
|
|
1386
1568
|
|
|
1387
|
-
> **Note:** `select` filters elements from the result of command, while the `query`
|
|
1569
|
+
> **Note:** Option `select` filters elements from the result of command, while the `query` option gives filtering parameters to the API when listing elements.
|
|
1388
1570
|
|
|
1389
1571
|
In above example, the same result is obtained with option:
|
|
1390
1572
|
|
|
@@ -1440,7 +1622,8 @@ The following decoders are supported:
|
|
|
1440
1622
|
| `re` | String | Regexp | Ruby Regular Expression (short for `@ruby:/.../`) |
|
|
1441
1623
|
| `ruby` | String | Any | Execute specified Ruby code |
|
|
1442
1624
|
| `secret` | None | String | Ask password interactively (hides input) |
|
|
1443
|
-
| `stdin` | None | String | Read from stdin (no value on right) |
|
|
1625
|
+
| `stdin` | None | String | Read from stdin in text mode (no value on right) |
|
|
1626
|
+
| `stdbin` | None | String | Read from stdin in binary mode (no value on right) |
|
|
1444
1627
|
| `uri` | String | String | Read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
|
1445
1628
|
| `val` | String | String | Prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
|
1446
1629
|
| `yaml` | String | Any | Decode YAML |
|
|
@@ -1575,44 +1758,46 @@ Temporary files are deleted at the end of execution unless option: `clean_temp`
|
|
|
1575
1758
|
|
|
1576
1759
|
### Configuration file
|
|
1577
1760
|
|
|
1578
|
-
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
|
1579
|
-
|
|
1761
|
+
On the first execution of `ascli`, an empty configuration file is created in the configuration folder (`ascli config folder`).
|
|
1762
|
+
There is no mandatory information required in this file.
|
|
1763
|
+
The use of it is optional as any option can be provided on the command line.
|
|
1580
1764
|
|
|
1581
|
-
Although the file is a standard YAML file, `ascli` provides commands to read and modify it using the `config` command.
|
|
1765
|
+
Although the file is a standard `YAML` file, `ascli` provides commands to read and modify it using the `config` command.
|
|
1582
1766
|
|
|
1583
|
-
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset)
|
|
1767
|
+
All options for `ascli` can be set on command line, or by env vars, or using [Option Preset](#option-preset) in the configuration file.
|
|
1584
1768
|
|
|
1585
|
-
A configuration file provides a way to define default values, especially for authentication
|
|
1769
|
+
A configuration file provides a way to define default values, especially for authentication options, thus avoiding to always having to specify those options on the command line.
|
|
1586
1770
|
|
|
1587
|
-
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or
|
|
1771
|
+
The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can be overridden with option `--config-file=path` or its env var).
|
|
1588
1772
|
|
|
1589
|
-
The configuration file is
|
|
1773
|
+
The configuration file is a catalog of named lists of options, called: [Option Preset](#option-preset).
|
|
1774
|
+
Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [Option Preset](#option-preset) (e.g. `mypreset`) using the option `preset`: `--preset=mypreset` or its shortcut: `-Pmypreset`.
|
|
1590
1775
|
|
|
1591
1776
|
#### Option Preset
|
|
1592
1777
|
|
|
1593
|
-
A [Option Preset](#option-preset)
|
|
1778
|
+
A [Option Preset](#option-preset) is a collection of options and their associated values in a named section in the configuration file.
|
|
1594
1779
|
|
|
1595
|
-
A named [Option Preset](#option-preset)
|
|
1780
|
+
A named [Option Preset](#option-preset) can be modified directly using `ascli`, which will update the configuration file :
|
|
1596
1781
|
|
|
1597
1782
|
```bash
|
|
1598
1783
|
ascli config preset set|delete|show|initialize|update <option preset>
|
|
1599
1784
|
```
|
|
1600
1785
|
|
|
1601
|
-
The command `update` allows the easy creation of [Option Preset](#option-preset)
|
|
1786
|
+
The command `update` allows the easy creation of [Option Preset](#option-preset) by simply providing the options in their command line format, e.g. :
|
|
1602
1787
|
|
|
1603
1788
|
```bash
|
|
1604
1789
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=my_password_here --ts=@json:'{"precalculate_job_size":true}'
|
|
1605
1790
|
```
|
|
1606
1791
|
|
|
1607
|
-
- This creates a [Option Preset](#option-preset)
|
|
1792
|
+
- This creates a [Option Preset](#option-preset) `demo_server` with all provided options.
|
|
1608
1793
|
|
|
1609
|
-
The command `set` allows setting individual options in a [Option Preset](#option-preset)
|
|
1794
|
+
The command `set` allows setting individual options in a [Option Preset](#option-preset).
|
|
1610
1795
|
|
|
1611
1796
|
```bash
|
|
1612
1797
|
ascli config preset set demo_server password my_password_here
|
|
1613
1798
|
```
|
|
1614
1799
|
|
|
1615
|
-
The command `initialize`, like `update` allows to set several
|
|
1800
|
+
The command `initialize`, like `update` allows to set several options at once, but it deletes an existing configuration instead of updating it, and expects a [`Hash` Extended Value](#extended-value-syntax).
|
|
1616
1801
|
|
|
1617
1802
|
```bash
|
|
1618
1803
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"my_pass_here","ts":{"precalculate_job_size":true}}'
|
|
@@ -1624,7 +1809,7 @@ A full terminal based overview of the configuration can be displayed using:
|
|
|
1624
1809
|
ascli config preset over
|
|
1625
1810
|
```
|
|
1626
1811
|
|
|
1627
|
-
A list of [Option Preset](#option-preset)
|
|
1812
|
+
A list of [Option Preset](#option-preset) can be displayed using:
|
|
1628
1813
|
|
|
1629
1814
|
```bash
|
|
1630
1815
|
ascli config preset list
|
|
@@ -1647,6 +1832,15 @@ ascli config preset over
|
|
|
1647
1832
|
ascli config preset list
|
|
1648
1833
|
```
|
|
1649
1834
|
|
|
1835
|
+
It is possible to load a [Option Preset](#option-preset) from within another [Option Preset](#option-preset) using the `preset` option.
|
|
1836
|
+
For example if `pcommon` is a preset with common options, and `pspecific` is a preset with specific options, then `pspecific` can load `pcommon` using:
|
|
1837
|
+
|
|
1838
|
+
```bash
|
|
1839
|
+
ascli config preset set pspecific preset pcommon
|
|
1840
|
+
```
|
|
1841
|
+
|
|
1842
|
+
When `pspecific` is loaded, then cumulative option `preset` will be set and it will also load `pcommon`.
|
|
1843
|
+
|
|
1650
1844
|
#### Special Option Preset: `config`
|
|
1651
1845
|
|
|
1652
1846
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
|
@@ -1686,7 +1880,7 @@ The default preset for `config` is read for any plugin invocation, this allows s
|
|
|
1686
1880
|
When `ascli` starts, it looks for the `default` Option Preset and checks the value for `config`.
|
|
1687
1881
|
If set, it loads the options independently of the plugin used.
|
|
1688
1882
|
|
|
1689
|
-
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global
|
|
1883
|
+
> **Note:** If no global default is set by the user, `ascli` will use `global_common_defaults` when setting global options (e.g. `config ascp use`)
|
|
1690
1884
|
>
|
|
1691
1885
|
> **Note:** If you don't know the name of the global preset, you can use `GLOBAL` to refer to it.
|
|
1692
1886
|
|
|
@@ -1735,6 +1929,8 @@ detect https://faspex4.example.com/path
|
|
|
1735
1929
|
detect https://faspex5.example.com/path
|
|
1736
1930
|
detect https://node.example.com/path
|
|
1737
1931
|
detect https://shares.example.com/path shares
|
|
1932
|
+
detect https://tst.example.com/path faspio
|
|
1933
|
+
detect https://tst.example.com/path httpgw
|
|
1738
1934
|
detect my_org aoc
|
|
1739
1935
|
doc
|
|
1740
1936
|
doc transfer-parameters
|
|
@@ -1760,10 +1956,11 @@ gem path
|
|
|
1760
1956
|
gem version
|
|
1761
1957
|
genkey my_key
|
|
1762
1958
|
genkey my_key 4096
|
|
1959
|
+
image https://eudemo.asperademo.com/wallpaper.jpg
|
|
1763
1960
|
initdemo
|
|
1764
1961
|
open
|
|
1765
|
-
|
|
1766
|
-
|
|
1962
|
+
plugins create my_command
|
|
1963
|
+
plugins list
|
|
1767
1964
|
preset delete conf_name
|
|
1768
1965
|
preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
|
|
1769
1966
|
preset list
|
|
@@ -1813,17 +2010,17 @@ demo_server:
|
|
|
1813
2010
|
We can see here:
|
|
1814
2011
|
|
|
1815
2012
|
- The configuration was created with `ascli` version 0.3.7
|
|
1816
|
-
- The default [Option Preset](#option-preset)
|
|
1817
|
-
- The [Option Preset](#option-preset)
|
|
1818
|
-
- The default [Option Preset](#option-preset)
|
|
2013
|
+
- The default [Option Preset](#option-preset) to load for `server` plugin is : `demo_server`
|
|
2014
|
+
- The [Option Preset](#option-preset) `demo_server` defines some options: the URL and credentials
|
|
2015
|
+
- The default [Option Preset](#option-preset) to load in any case is : `cli_default`
|
|
1819
2016
|
|
|
1820
|
-
Two [Option Preset](#option-preset)
|
|
2017
|
+
Two [Option Preset](#option-preset) are reserved:
|
|
1821
2018
|
|
|
1822
2019
|
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
|
1823
2020
|
It is used to check compatibility.
|
|
1824
|
-
- `default` is reserved to define the default [Option Preset](#option-preset)
|
|
2021
|
+
- `default` is reserved to define the default [Option Preset](#option-preset) name used for known plugins.
|
|
1825
2022
|
|
|
1826
|
-
The user may create as many [Option Preset](#option-preset)
|
|
2023
|
+
The user may create as many [Option Preset](#option-preset) as needed. For instance, a particular [Option Preset](#option-preset) can be created for a particular application instance and contain URL and credentials.
|
|
1827
2024
|
|
|
1828
2025
|
Values in the configuration also follow the [Extended Value Syntax](#extended-value-syntax).
|
|
1829
2026
|
|
|
@@ -1833,7 +2030,7 @@ Values in the configuration also follow the [Extended Value Syntax](#extended-va
|
|
|
1833
2030
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/my_private_key"
|
|
1834
2031
|
```
|
|
1835
2032
|
|
|
1836
|
-
This creates the [Option Preset](#option-preset)
|
|
2033
|
+
This creates the [Option Preset](#option-preset):
|
|
1837
2034
|
|
|
1838
2035
|
```yaml
|
|
1839
2036
|
my_aoc_org:
|
|
@@ -1849,20 +2046,20 @@ Some options are global, some options are available only for some plugins. (the
|
|
|
1849
2046
|
Options are loaded using this algorithm:
|
|
1850
2047
|
|
|
1851
2048
|
- If option `--no-default` (or `-N`) is specified, then no default value is loaded for the plugin
|
|
1852
|
-
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset)
|
|
1853
|
-
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset)
|
|
2049
|
+
- Else it looks for the name of the plugin as key in section `default`, the value is the name of the default [Option Preset](#option-preset) for it, and loads it.
|
|
2050
|
+
- If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the [Option Preset](#option-preset) specified from the configuration file, or if the value is a `Hash`, it uses it as options values.
|
|
1854
2051
|
- Environment variables are evaluated
|
|
1855
2052
|
- Command line options are evaluated
|
|
1856
2053
|
|
|
1857
|
-
|
|
2054
|
+
Options are evaluated in the order of command line.
|
|
1858
2055
|
|
|
1859
|
-
To avoid loading the default [Option Preset](#option-preset)
|
|
2056
|
+
To avoid loading the default [Option Preset](#option-preset) for a plugin, use: `-N`
|
|
1860
2057
|
|
|
1861
|
-
On command line, words in
|
|
2058
|
+
On command line, words in option names are separated by a dash (`-`).
|
|
1862
2059
|
In configuration file, separator is an underscore.
|
|
1863
2060
|
E.g. `--xxx-yyy` on command line gives `xxx_yyy` in configuration file.
|
|
1864
2061
|
|
|
1865
|
-
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset)
|
|
2062
|
+
The main plugin name is `config`, so it is possible to define a default [Option Preset](#option-preset) for the main plugin with:
|
|
1866
2063
|
|
|
1867
2064
|
```bash
|
|
1868
2065
|
ascli config preset set cli_default interactive no
|
|
@@ -1872,7 +2069,7 @@ ascli config preset set cli_default interactive no
|
|
|
1872
2069
|
ascli config preset set default config cli_default
|
|
1873
2070
|
```
|
|
1874
2071
|
|
|
1875
|
-
A [Option Preset](#option-preset)
|
|
2072
|
+
A [Option Preset](#option-preset) value can be removed with `unset`:
|
|
1876
2073
|
|
|
1877
2074
|
```bash
|
|
1878
2075
|
ascli config preset unset cli_default interactive
|
|
@@ -1892,7 +2089,7 @@ ascli -N --preset=@json:'{"url":"_url_here_","password":"my_password_here","user
|
|
|
1892
2089
|
|
|
1893
2090
|
#### Wizard
|
|
1894
2091
|
|
|
1895
|
-
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset)
|
|
2092
|
+
The wizard is a command that asks the user for information and creates a [Option Preset](#option-preset) with the provided information.
|
|
1896
2093
|
|
|
1897
2094
|
It takes an optional argument: the URL of the application, and an **option**: `query` which allows limiting the detection to a given plugin.
|
|
1898
2095
|
|
|
@@ -1914,7 +2111,7 @@ ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=my
|
|
|
1914
2111
|
|
|
1915
2112
|
This can also be provisioned in a configuration file:
|
|
1916
2113
|
|
|
1917
|
-
- Build [Option Preset](#option-preset)
|
|
2114
|
+
- Build [Option Preset](#option-preset)
|
|
1918
2115
|
|
|
1919
2116
|
```bash
|
|
1920
2117
|
ascli config preset set shares06 url https://10.25.0.6
|
|
@@ -1934,7 +2131,7 @@ or
|
|
|
1934
2131
|
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=my_password_here
|
|
1935
2132
|
```
|
|
1936
2133
|
|
|
1937
|
-
- Define this [Option Preset](#option-preset)
|
|
2134
|
+
- Define this [Option Preset](#option-preset) as the default [Option Preset](#option-preset) for the specified plugin (`shares`)
|
|
1938
2135
|
|
|
1939
2136
|
```bash
|
|
1940
2137
|
ascli config preset set default shares shares06
|
|
@@ -1946,7 +2143,7 @@ ascli config preset set default shares shares06
|
|
|
1946
2143
|
ascli config preset overview
|
|
1947
2144
|
```
|
|
1948
2145
|
|
|
1949
|
-
- Execute a command on the shares application using default
|
|
2146
|
+
- Execute a command on the shares application using default options
|
|
1950
2147
|
|
|
1951
2148
|
```bash
|
|
1952
2149
|
ascli shares repo browse /
|
|
@@ -2039,7 +2236,7 @@ The lookup is done by comparing the service URL and username (or access key).
|
|
|
2039
2236
|
|
|
2040
2237
|
#### Securing passwords and secrets
|
|
2041
2238
|
|
|
2042
|
-
A passwords can be saved in clear in a [Option Preset](#option-preset)
|
|
2239
|
+
A passwords can be saved in clear in a [Option Preset](#option-preset) together with other account information (URL, username, etc...).
|
|
2043
2240
|
Example:
|
|
2044
2241
|
|
|
2045
2242
|
```bash
|
|
@@ -2060,25 +2257,31 @@ ascli config vault create myconf @json:'{"password":"my_password_here"}'
|
|
|
2060
2257
|
|
|
2061
2258
|
### Private Key
|
|
2062
2259
|
|
|
2063
|
-
Some applications allow the user to be authenticated using
|
|
2064
|
-
|
|
2065
|
-
|
|
2066
|
-
|
|
2067
|
-
|
|
2260
|
+
Some Aspera applications allow the user to be authenticated using [Public Key Cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography):
|
|
2261
|
+
|
|
2262
|
+
- for SSH: Server
|
|
2263
|
+
- for OAuth JWT: AoC, Faspex5, Faspex, Shares
|
|
2264
|
+
|
|
2265
|
+
It consists in using a pair of associated keys: a private key and a public key.
|
|
2266
|
+
The same pair can be used for multiple applications.
|
|
2267
|
+
The file containing the private key (key pair) can optionally be protected by a passphrase.
|
|
2068
2268
|
If the key is protected by a passphrase, then it will be prompted when used.
|
|
2069
2269
|
(some plugins support option `passphrase`)
|
|
2070
2270
|
|
|
2071
|
-
The following commands use the shell variable `
|
|
2271
|
+
The following sample commands use the shell variable `KEY_PAIR_PATH`.
|
|
2072
2272
|
Set it to the desired safe location of the private key.
|
|
2073
|
-
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli
|
|
2273
|
+
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`.
|
|
2274
|
+
For example:
|
|
2074
2275
|
|
|
2075
2276
|
```bash
|
|
2076
|
-
|
|
2277
|
+
KEY_PAIR_PATH=~/.aspera/ascli/my_private_key
|
|
2077
2278
|
```
|
|
2078
2279
|
|
|
2079
2280
|
Several methods can be used to generate a key pair.
|
|
2080
2281
|
|
|
2081
|
-
The format expected for
|
|
2282
|
+
The format expected for keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
|
|
2283
|
+
|
|
2284
|
+
If another format is used, such as `DER`, it can be converted to `PEM`, e.g. using `openssl`.
|
|
2082
2285
|
|
|
2083
2286
|
#### `ascli` for key generation
|
|
2084
2287
|
|
|
@@ -2087,21 +2290,13 @@ For convenience, the public key is also extracted with extension `.pub`.
|
|
|
2087
2290
|
The key is not passphrase protected.
|
|
2088
2291
|
|
|
2089
2292
|
```bash
|
|
2090
|
-
ascli config genkey ${
|
|
2293
|
+
ascli config genkey ${KEY_PAIR_PATH} 4096
|
|
2091
2294
|
```
|
|
2092
2295
|
|
|
2093
|
-
> **Note:** `ascli` uses the `openssl` library.
|
|
2094
|
-
|
|
2095
2296
|
To display the public key of a private key:
|
|
2096
2297
|
|
|
2097
2298
|
```bash
|
|
2098
|
-
ascli config pubkey @file:${
|
|
2099
|
-
```
|
|
2100
|
-
|
|
2101
|
-
To display the version of **openssl** used in `ascli`:
|
|
2102
|
-
|
|
2103
|
-
```bash
|
|
2104
|
-
ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
2299
|
+
ascli config pubkey @file:${KEY_PAIR_PATH}
|
|
2105
2300
|
```
|
|
2106
2301
|
|
|
2107
2302
|
#### `ssh-keygen`
|
|
@@ -2109,16 +2304,16 @@ ascli config echo @ruby:OpenSSL::OPENSSL_VERSION --format=text
|
|
|
2109
2304
|
Both private and public keys are generated, option `-N` is for passphrase.
|
|
2110
2305
|
|
|
2111
2306
|
```bash
|
|
2112
|
-
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${
|
|
2307
|
+
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
|
|
2113
2308
|
```
|
|
2114
2309
|
|
|
2115
2310
|
#### `openssl`
|
|
2116
2311
|
|
|
2117
|
-
To generate a
|
|
2312
|
+
To generate a key pair with a passphrase the following can be used on any system:
|
|
2118
2313
|
|
|
2119
2314
|
```bash
|
|
2120
|
-
openssl genrsa -passout pass:_passphrase_here_ -out ${
|
|
2121
|
-
openssl rsa -pubout -in ${
|
|
2315
|
+
openssl genrsa -passout pass:_passphrase_here_ -out ${KEY_PAIR_PATH} 4096
|
|
2316
|
+
openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub
|
|
2122
2317
|
```
|
|
2123
2318
|
|
|
2124
2319
|
`openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
|
|
@@ -2127,112 +2322,55 @@ In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` t
|
|
|
2127
2322
|
If option `-nodes` is not available, the passphrase can be removed using this method:
|
|
2128
2323
|
|
|
2129
2324
|
```bash
|
|
2130
|
-
openssl rsa -passin pass:_passphrase_here_ -in ${
|
|
2131
|
-
mv ${
|
|
2325
|
+
openssl rsa -passin pass:_passphrase_here_ -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
|
|
2326
|
+
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
|
|
2132
2327
|
```
|
|
2133
2328
|
|
|
2134
2329
|
To change (or add) the passphrase for a key do:
|
|
2135
2330
|
|
|
2136
2331
|
```bash
|
|
2137
|
-
openssl rsa -des3 -in ${
|
|
2138
|
-
mv ${
|
|
2332
|
+
openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
|
|
2333
|
+
mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}
|
|
2139
2334
|
```
|
|
2140
2335
|
|
|
2141
|
-
|
|
2142
|
-
|
|
2143
|
-
The SSL CA certificate bundle can be specified using option `cert_stores`, which is a list of files or folders, by default it uses Ruby's default certificate store.
|
|
2144
|
-
|
|
2145
|
-
To display trusted certificate store locations:
|
|
2146
|
-
|
|
2147
|
-
```bash
|
|
2148
|
-
ascli --show-config --fields=cert_stores
|
|
2149
|
-
```
|
|
2150
|
-
|
|
2151
|
-
Use option `cert_stores` to modify the locations of certificate stores (files or folders).
|
|
2152
|
-
If you use this option, then default locations are not used.
|
|
2153
|
-
Default locations can be added using special value `DEF`.
|
|
2154
|
-
The value can be either an `Array` or `String` (path).
|
|
2155
|
-
Successive options add paths incrementally.
|
|
2156
|
-
All files of a folders are added.
|
|
2336
|
+
#### Using an application to generate a key pair
|
|
2157
2337
|
|
|
2158
|
-
|
|
2159
|
-
|
|
2338
|
+
Many applications are available, including on internet, to generate key pairs.
|
|
2339
|
+
For example: <https://cryptotools.net/rsagen>
|
|
2160
2340
|
|
|
2161
|
-
|
|
2341
|
+
> **Note:** Be careful that private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
|
|
2162
2342
|
|
|
2163
|
-
|
|
2164
|
-
--cert-stores=$(openssl version -d|cut -f2 -d'"')/cert.pem
|
|
2165
|
-
```
|
|
2166
|
-
|
|
2167
|
-
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
|
2168
|
-
Ruby's default values can be overridden using env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
|
2169
|
-
|
|
2170
|
-
One can display those default values:
|
|
2171
|
-
|
|
2172
|
-
```bash
|
|
2173
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_DIR --format=text
|
|
2174
|
-
ascli config echo @ruby:OpenSSL::X509::DEFAULT_CERT_FILE --format=text
|
|
2175
|
-
```
|
|
2176
|
-
|
|
2177
|
-
`ascp` also needs to validate certificates when using **WSS** for transfer TCP part (instead of SSH).
|
|
2178
|
-
|
|
2179
|
-
By default,`ascp` uses an hardcoded root location `OPENSSLDIR`.
|
|
2180
|
-
Original `ascp`'s hardcoded locations can be found using:
|
|
2181
|
-
|
|
2182
|
-
```bash
|
|
2183
|
-
ascli config ascp info --fields=openssldir
|
|
2184
|
-
```
|
|
2185
|
-
|
|
2186
|
-
E.g. on macOS: `/Library/Aspera/ssl`.
|
|
2187
|
-
Then trusted certificates are taken from `[OPENSSLDIR]/cert.pem` and files in `[OPENSSLDIR]/certs`.
|
|
2188
|
-
`ascli` overrides the default hardcoded location used by `ascp` for WSS and uses the same locations as specified in `cert_stores` (using the `-i` option of `ascp`).
|
|
2343
|
+
### Image and video thumbnails
|
|
2189
2344
|
|
|
2190
|
-
|
|
2191
|
-
|
|
2192
|
-
Typically done by updating the system's root certificate store.
|
|
2345
|
+
`ascli` can display thumbnails for images and videos in the terminal.
|
|
2346
|
+
This is available:
|
|
2193
2347
|
|
|
2194
|
-
|
|
2348
|
+
- in the `thumbnail` command of `node` when using **gen4/access key** API.
|
|
2349
|
+
- when using the `show` command of `preview` plugin.
|
|
2350
|
+
- `coffee` and `image` commands of `config` plugin.
|
|
2351
|
+
- any displayed value which is an url to image can be displayed with option `format` set to `image`
|
|
2195
2352
|
|
|
2196
|
-
|
|
2197
|
-
ascli config echo @uri:https://curl.haxx.se/ca/cacert.pem --format=text
|
|
2198
|
-
```
|
|
2353
|
+
The following options can be specified in the option `image`:
|
|
2199
2354
|
|
|
2200
|
-
|
|
2355
|
+
| Option | Type | Description |
|
|
2356
|
+
|------------|---------|-------------|
|
|
2357
|
+
| reserve | Integer | Lines reserved to display a status |
|
|
2358
|
+
| text | Bool | Display text instead of image|
|
|
2359
|
+
| double | Bool | Display double text resolution (half characters) |
|
|
2360
|
+
| font_ratio | Float | Font height/width ratio in terminal |
|
|
2201
2361
|
|
|
2202
2362
|
```bash
|
|
2203
|
-
ascli config
|
|
2363
|
+
ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --image=@json:'{"text":true}'
|
|
2204
2364
|
```
|
|
2205
2365
|
|
|
2206
|
-
Then, use this store by setting the option `cert_stores` (or env var `SSL_CERT_FILE`).
|
|
2207
|
-
|
|
2208
|
-
To trust a specific certificate (e.g. self-signed), **provided that the `CN` is correct**, save the certificate chain to a file:
|
|
2209
|
-
|
|
2210
2366
|
```bash
|
|
2211
|
-
|
|
2367
|
+
curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdbin:
|
|
2212
2368
|
```
|
|
2213
2369
|
|
|
2214
|
-
> **Note:** Use command `name` to display the remote common name of the remote certificate.
|
|
2215
|
-
|
|
2216
|
-
Then, use this file as certificate store (e.g. here, Node API):
|
|
2217
|
-
|
|
2218
2370
|
```bash
|
|
2219
|
-
ascli config
|
|
2371
|
+
echo -n https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @uri:@stdin:
|
|
2220
2372
|
```
|
|
2221
2373
|
|
|
2222
|
-
### Image and video thumbnails
|
|
2223
|
-
|
|
2224
|
-
`ascli` can display thumbnails for images and videos in the terminal.
|
|
2225
|
-
This is available with the `thumbnail` command of `node` when using **gen4/access key** API.
|
|
2226
|
-
It's also available when using the `show` command of `preview` plugin.
|
|
2227
|
-
|
|
2228
|
-
The following options can be specified in the option `query`:
|
|
2229
|
-
|
|
2230
|
-
| Option | Description |
|
|
2231
|
-
|------------|-------------|
|
|
2232
|
-
| text | Display text instead of image (Bool) |
|
|
2233
|
-
| double | Display double text resolution (half characters) (Bool) |
|
|
2234
|
-
| font_ratio | Font height/width ratio in terminal (Float) |
|
|
2235
|
-
|
|
2236
2374
|
### Graphical Interactions: Browser and Text Editor
|
|
2237
2375
|
|
|
2238
2376
|
Some actions may require the use of a graphical tool:
|
|
@@ -2250,9 +2388,7 @@ It is also possible to force the graphical mode with option --ui :
|
|
|
2250
2388
|
|
|
2251
2389
|
The gem is equipped with traces, mainly for debugging and learning APIs.
|
|
2252
2390
|
By default logging level is `warn` and the output channel is `stderr`.
|
|
2253
|
-
To increase debug level, use
|
|
2254
|
-
|
|
2255
|
-
It is also possible to activate traces before log facility initialization using env var `ASCLI_LOG_LEVEL`.
|
|
2391
|
+
To increase debug level, use option `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or an [Option Preset](#option-preset)).
|
|
2256
2392
|
|
|
2257
2393
|
By default passwords and secrets are removed from logs.
|
|
2258
2394
|
Use option `log_secrets` set to `yes` to reveal secrets in logs.
|
|
@@ -2332,21 +2468,33 @@ Refer to the following sections.
|
|
|
2332
2468
|
|
|
2333
2469
|
#### Proxy for REST and HTTP Gateway
|
|
2334
2470
|
|
|
2335
|
-
REST API calls and transfers based on HTTP Gateway both use Ruby `Net::HTTP`
|
|
2471
|
+
REST API calls and transfers based on HTTP Gateway both use Ruby's `Net::HTTP` class.
|
|
2472
|
+
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
|
2336
2473
|
|
|
2337
|
-
|
|
2474
|
+
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used .
|
|
2338
2475
|
|
|
2339
|
-
The `http_proxy` environment variable (**lower case
|
|
2476
|
+
The `http_proxy` environment variable (**lower case**) can be set to the **URL** of the proxy (with optional credentials).
|
|
2477
|
+
Syntax is: `(http|https)://[user:password@]host:port`.
|
|
2340
2478
|
E.g. `http://myproxy.org.net:3128`.
|
|
2341
|
-
Refer to [Ruby find proxy](https://rubyapi.org/3.0/o/uri/generic#method-i-find_proxy).
|
|
2342
2479
|
|
|
2343
|
-
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not**
|
|
2480
|
+
> **Note:** Ruby expects a URL and `myproxy.org.net:3128` alone is **not** valid.
|
|
2481
|
+
|
|
2482
|
+
Credentials for proxy are optional but can also be specified:
|
|
2344
2483
|
|
|
2345
2484
|
```bash
|
|
2346
|
-
export http_proxy=http://proxy.example.com:3128
|
|
2485
|
+
export http_proxy=http://user:password@proxy.example.com:3128
|
|
2486
|
+
```
|
|
2487
|
+
|
|
2488
|
+
Option `http_proxy` does the same (set env var) but on command line:
|
|
2489
|
+
|
|
2490
|
+
```bash
|
|
2491
|
+
ascli --http-proxy=http://username:password@host:port ...
|
|
2347
2492
|
```
|
|
2348
2493
|
|
|
2349
2494
|
Alternatively, the `fpac` option (function for proxy auto config) can be set to a [Proxy Auto Configuration (PAC)](https://en.wikipedia.org/wiki/Proxy_auto-config) javascript value.
|
|
2495
|
+
|
|
2496
|
+
Note that proxy credentials are not supported in PAC files.
|
|
2497
|
+
|
|
2350
2498
|
To read the script from a URL (`http:`, `https:` and `file:`), use prefix: `@uri:`.
|
|
2351
2499
|
A minimal script can be specified to define the use of a local proxy:
|
|
2352
2500
|
|
|
@@ -2381,7 +2529,7 @@ ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.
|
|
|
2381
2529
|
PROXY proxy.example.com:8080
|
|
2382
2530
|
```
|
|
2383
2531
|
|
|
2384
|
-
If the proxy requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
|
2532
|
+
If the proxy found with the PAC requires credentials, then use option `proxy_credentials` with username and password provided as an `Array`:
|
|
2385
2533
|
|
|
2386
2534
|
```bash
|
|
2387
2535
|
ascli --proxy-credentials=@json:'["__username_here__","__password_here__"]' ...
|
|
@@ -2397,7 +2545,7 @@ Only supported with the `direct` agent: To specify a proxy for legacy HTTP fallb
|
|
|
2397
2545
|
|
|
2398
2546
|
#### FASP proxy (forward) for transfers
|
|
2399
2547
|
|
|
2400
|
-
To specify a FASP proxy (forward), set the [
|
|
2548
|
+
To specify a FASP proxy (forward), set the [**transfer-spec**](#transfer-specification) parameter: `proxy` (only supported with the `direct` agent).
|
|
2401
2549
|
|
|
2402
2550
|
For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
|
|
2403
2551
|
|
|
@@ -2413,7 +2561,20 @@ Or, alternatively, (prefer transfer spec like above, generally):
|
|
|
2413
2561
|
|
|
2414
2562
|
### FASP configuration
|
|
2415
2563
|
|
|
2416
|
-
|
|
2564
|
+
`ascli` uses one of the transfer agents to execute transfers.
|
|
2565
|
+
|
|
2566
|
+
By default it uses the `direct` agent, which is basically a local `ascp`.
|
|
2567
|
+
Nevertheless, `ascli` does not come with `ascp` installed.
|
|
2568
|
+
This is the reason why it is advised to install the Aspera Transfer SDK during installation (`ascli config ascp install`).
|
|
2569
|
+
|
|
2570
|
+
By default, `ascli` uses the `ascp` binary found in **well known locations**, i.e. typical Aspera product installation paths.
|
|
2571
|
+
|
|
2572
|
+
The way to specify the location of `ascp` is to use either options:
|
|
2573
|
+
|
|
2574
|
+
- `ascp_path`
|
|
2575
|
+
- `use_product`
|
|
2576
|
+
|
|
2577
|
+
The `config` plugin allows to find and specify the location of `ascp`.
|
|
2417
2578
|
It provides the following commands for `ascp` subcommand:
|
|
2418
2579
|
|
|
2419
2580
|
- `show` : shows the path of `ascp` used
|
|
@@ -2447,9 +2608,9 @@ ascli config ascp info
|
|
|
2447
2608
|
|
|
2448
2609
|
By default, `ascli` uses any found local product with `ascp`, including Transfer SDK.
|
|
2449
2610
|
|
|
2450
|
-
To
|
|
2611
|
+
To override and use an alternate `ascp` path use option `ascp_path` (`--ascp-path=`)
|
|
2451
2612
|
|
|
2452
|
-
For a permanent change, the command `config ascp use` sets the same
|
|
2613
|
+
For a permanent change, the command `config ascp use` sets the same option for the global default.
|
|
2453
2614
|
|
|
2454
2615
|
Using a POSIX shell:
|
|
2455
2616
|
|
|
@@ -2477,7 +2638,10 @@ Saved to default global preset global_common_defaults
|
|
|
2477
2638
|
|
|
2478
2639
|
If the path has spaces, read section: [Shell and Command line parsing](#command-line-parsing-special-characters).
|
|
2479
2640
|
|
|
2480
|
-
|
|
2641
|
+
If option `ascp_path` is not set, then the product identified with option `use_product` is used.
|
|
2642
|
+
|
|
2643
|
+
If `use_product` is not set, then the first product found is used,
|
|
2644
|
+
this is equivalent to using option: `--use-product=FIRST`.
|
|
2481
2645
|
|
|
2482
2646
|
Locally installed Aspera products can be listed with:
|
|
2483
2647
|
|
|
@@ -2496,10 +2660,6 @@ ascli config ascp products list
|
|
|
2496
2660
|
+---------------------------------------+----------------------------------------+
|
|
2497
2661
|
```
|
|
2498
2662
|
|
|
2499
|
-
#### Selection of local client for `ascp` for [`direct`](#agent-direct) agent
|
|
2500
|
-
|
|
2501
|
-
If no `ascp` is selected, this is equivalent to using option: `--use-product=FIRST`.
|
|
2502
|
-
|
|
2503
2663
|
Using the option `use_product` finds the `ascp` binary of the selected product.
|
|
2504
2664
|
|
|
2505
2665
|
To permanently use the `ascp` of a product:
|
|
@@ -2556,12 +2716,12 @@ Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
|
|
2556
2716
|
|
|
2557
2717
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
|
2558
2718
|
|
|
2559
|
-
When a transfer needs to be started, a [
|
|
2560
|
-
This [
|
|
2719
|
+
When a transfer needs to be started, a [**transfer-spec**](#transfer-specification) has been internally prepared.
|
|
2720
|
+
This [**transfer-spec**](#transfer-specification) will be executed by a transfer client, here called **Transfer Agent**.
|
|
2561
2721
|
|
|
2562
|
-
|
|
2722
|
+
The following agents are supported and selected with option `transfer`:
|
|
2563
2723
|
|
|
2564
|
-
- [`direct`](#agent-direct) : execution of `ascp`
|
|
2724
|
+
- [`direct`](#agent-direct) : direct execution of `ascp` (local)
|
|
2565
2725
|
- [`trsdk`](#agent-transfer-sdk) : use of Aspera Transfer SDK (local)
|
|
2566
2726
|
- [`connect`](#agent-connect-client) : use Connect Client (local)
|
|
2567
2727
|
- [`alpha`](#agent-desktop-client) : use the new Desktop Client (local)
|
|
@@ -2572,7 +2732,7 @@ There are currently 3 agents, set with option `transfer`:
|
|
|
2572
2732
|
For example, a node agent executing an **upload**, or **package send** operation
|
|
2573
2733
|
will effectively push files to the related server from the agent node.
|
|
2574
2734
|
|
|
2575
|
-
`ascli` standardizes on the use of a [
|
|
2735
|
+
`ascli` standardizes on the use of a [**transfer-spec**](#transfer-specification) instead of **native** `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
|
2576
2736
|
|
|
2577
2737
|
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
|
2578
2738
|
|
|
@@ -2583,7 +2743,7 @@ This is the default agent for `ascli` (option `--transfer=direct`).
|
|
|
2583
2743
|
`ascli` will search locally installed Aspera products, including SDK, and use `ascp` from that component.
|
|
2584
2744
|
Refer to section [FASP](#fasp-configuration).
|
|
2585
2745
|
|
|
2586
|
-
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and
|
|
2746
|
+
The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to `ascp`:
|
|
2587
2747
|
|
|
2588
2748
|
| Name | Type | Description |
|
|
2589
2749
|
|------------------------|-------|-------------|
|
|
@@ -2607,7 +2767,7 @@ max( sleep_max , sleep_initial * sleep_factor ^ iter_index )
|
|
|
2607
2767
|
```
|
|
2608
2768
|
|
|
2609
2769
|
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by `ascp` (e.g. WSS).
|
|
2610
|
-
In order to use a custom certificate store, use the `trusted_certs`
|
|
2770
|
+
In order to use a custom certificate store, use the `trusted_certs` option.
|
|
2611
2771
|
To use `ascp`'s default, use option: `--transfer-info=@json:'{"trusted_certs":null}'`.
|
|
2612
2772
|
|
|
2613
2773
|
Some transfer errors are considered **retry-able** (e.g. timeout) and some other not (e.g. wrong password).
|
|
@@ -2726,6 +2886,7 @@ There are no option for `transfer_info`.
|
|
|
2726
2886
|
|
|
2727
2887
|
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
|
2728
2888
|
This is especially useful for direct node-to-node transfers.
|
|
2889
|
+
|
|
2729
2890
|
Parameters provided in option `transfer_info` are:
|
|
2730
2891
|
|
|
2731
2892
|
| Name | Type | Description |
|
|
@@ -2735,7 +2896,7 @@ Parameters provided in option `transfer_info` are:
|
|
|
2735
2896
|
| password | string | Password, secret or bearer token</br>Mandatory |
|
|
2736
2897
|
| root_id | string | Root file id</br>Mandatory only for bearer token |
|
|
2737
2898
|
|
|
2738
|
-
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset)
|
|
2899
|
+
Like any other option, `transfer_info` can get its value from a pre-configured [Option Preset](#option-preset) :
|
|
2739
2900
|
|
|
2740
2901
|
```bash
|
|
2741
2902
|
--transfer-info=@preset:_name_here_
|
|
@@ -2749,12 +2910,14 @@ or be specified using the extended value syntax :
|
|
|
2749
2910
|
|
|
2750
2911
|
If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
|
|
2751
2912
|
|
|
2752
|
-
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the
|
|
2913
|
+
If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the file id of the top folder to use on the node using this access key.
|
|
2753
2914
|
It can be either the access key's root file id, or any authorized file id underneath it.
|
|
2754
2915
|
|
|
2755
2916
|
#### Agent: HTTP Gateway
|
|
2756
2917
|
|
|
2757
|
-
|
|
2918
|
+
The Aspera HTTP Gateway is a service that allows to send and receive files using HTTPS.
|
|
2919
|
+
|
|
2920
|
+
By specifying option: `--transfer=httpgw`, `ascli` will start transfers using the Aspera HTTP Gateway.
|
|
2758
2921
|
|
|
2759
2922
|
Parameters provided in option `transfer_info` are:
|
|
2760
2923
|
|
|
@@ -2773,6 +2936,8 @@ ascli faspex package recv 323 --transfer=httpgw --transfer-info=@json:'{"url":"h
|
|
|
2773
2936
|
|
|
2774
2937
|
> **Note:** The gateway only supports transfers authorized with a token.
|
|
2775
2938
|
|
|
2939
|
+
If the application, e.g. AoC or Faspex 5, is configured to use the HTTP Gateway, then `ascli` will automatically use the gateway URL if `--transfer=httpgw` is specified, so `transfer_info` becomes optional.
|
|
2940
|
+
|
|
2776
2941
|
#### Agent: Transfer SDK
|
|
2777
2942
|
|
|
2778
2943
|
Another possibility is to use the Transfer SDK daemon (`asperatransferd`).
|
|
@@ -2783,7 +2948,7 @@ Options for `transfer_info` are:
|
|
|
2783
2948
|
| Name | Type | Description |
|
|
2784
2949
|
|----------|--------|-------------|
|
|
2785
2950
|
| `url` | string | IP address and port listened by the daemon</br>Mandatory<br/>Default: `:0` |
|
|
2786
|
-
| `external` | bool | Use external daemon, do not start
|
|
2951
|
+
| `external` | bool | Use external daemon, do not start one.<br/>Default: `false` |
|
|
2787
2952
|
| `keep` | bool | Keep the daemon running after exiting `ascli`<br/>Default: `false` |
|
|
2788
2953
|
|
|
2789
2954
|
> **Note:** If port zero is specified in the URL, then the daemon will listen on a random available port. If no address is specified, then `127.0.0.1` is used.
|
|
@@ -2812,7 +2977,7 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
|
2812
2977
|
### Transfer Specification
|
|
2813
2978
|
|
|
2814
2979
|
Some commands lead to file transfer (upload/download).
|
|
2815
|
-
All parameters necessary for this transfer are described in a [
|
|
2980
|
+
All parameters necessary for this transfer are described in a [**transfer-spec**](#transfer-specification) (Transfer Specification), such as:
|
|
2816
2981
|
|
|
2817
2982
|
- Server address
|
|
2818
2983
|
- Transfer user name
|
|
@@ -2820,11 +2985,11 @@ All parameters necessary for this transfer are described in a [*transfer-spec*](
|
|
|
2820
2985
|
- File list
|
|
2821
2986
|
- Etc...
|
|
2822
2987
|
|
|
2823
|
-
`ascli` builds the [
|
|
2988
|
+
`ascli` builds the [**transfer-spec**](#transfer-specification) internally as a `Hash`.
|
|
2824
2989
|
It is not necessary to provide additional parameters on the command line for this transfer.
|
|
2825
2990
|
|
|
2826
|
-
It is possible to modify or add any of the supported [
|
|
2827
|
-
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [
|
|
2991
|
+
It is possible to modify or add any of the supported [**transfer-spec**](#transfer-specification) parameter using the `ts` option.
|
|
2992
|
+
The `ts` option accepts a [`Hash` Extended Value](#extended-value-syntax) containing one or several [**transfer-spec**](#transfer-specification) parameters.
|
|
2828
2993
|
Multiple `ts` options on command line are cumulative, and the `Hash` value is deeply merged.
|
|
2829
2994
|
To remove a (deep) key from transfer spec, set the value to `null`.
|
|
2830
2995
|
|
|
@@ -2834,15 +2999,15 @@ It is possible to specify `ascp` options when the `transfer` option is set to [`
|
|
|
2834
2999
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
|
2835
3000
|
This is especially useful for `ascp` command line parameters not supported in the transfer spec.
|
|
2836
3001
|
|
|
2837
|
-
The use of a [
|
|
3002
|
+
The use of a [**transfer-spec**](#transfer-specification) instead of `ascp` command line arguments has the advantage of:
|
|
2838
3003
|
|
|
2839
3004
|
- Common to all [Transfer Agent](#transfer-clients-agents)
|
|
2840
3005
|
- Not dependent on command line limitations (special characters...)
|
|
2841
3006
|
|
|
2842
3007
|
### Transfer Parameters
|
|
2843
3008
|
|
|
2844
|
-
All standard [
|
|
2845
|
-
[
|
|
3009
|
+
All standard [**transfer-spec**](#transfer-specification) parameters can be specified.
|
|
3010
|
+
[**transfer-spec**](#transfer-specification) can also be saved/overridden in the configuration file.
|
|
2846
3011
|
|
|
2847
3012
|
References:
|
|
2848
3013
|
|
|
@@ -2889,7 +3054,7 @@ Columns:
|
|
|
2889
3054
|
| fasp_port | int | Y | Y | Y | Y | Y | Specifies fasp (UDP) port.<br/>(-O {int}) |
|
|
2890
3055
|
| fasp_url | string | | | | | | Only used in Faspex.<br/>(<ignored>) |
|
|
2891
3056
|
| file_checksum | string | Y | Y | | | | Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none<br/>(<ignored>) |
|
|
2892
|
-
| http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}
|
|
3057
|
+
| http_fallback | bool<br/>string | Y | Y | Y | Y | Y | When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y (conversion){bool}\|{string}) |
|
|
2893
3058
|
| http_fallback_port | int | Y | | | | | Specifies http port when no cipher is used<br/>(-t {int}) |
|
|
2894
3059
|
| https_fallback_port | int | Y | Y | Y | Y | Y | Specifies https port when cipher is used<br/>(-t {int}) |
|
|
2895
3060
|
| keepalive | bool | Y | | | | | The session is running in persistent session mode.<br/>(--keepalive) |
|
|
@@ -2960,7 +3125,7 @@ The destination folder is set by `ascli` by default to:
|
|
|
2960
3125
|
- `.` for downloads
|
|
2961
3126
|
- `/` for uploads
|
|
2962
3127
|
|
|
2963
|
-
It is specified by the [
|
|
3128
|
+
It is specified by the [**transfer-spec**](#transfer-specification) parameter `destination_root`.
|
|
2964
3129
|
As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path>"}'`.
|
|
2965
3130
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
|
2966
3131
|
`--to-folder=<path>` .
|
|
@@ -2974,8 +3139,8 @@ By default the list of files to transfer is simply provided on the command line.
|
|
|
2974
3139
|
The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
|
|
2975
3140
|
The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
|
|
2976
3141
|
|
|
2977
|
-
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [
|
|
2978
|
-
So, eventually, the list of files to transfer is provided to the transfer agent using the [
|
|
3142
|
+
In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [**transfer-spec**](#transfer-specification) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
|
|
3143
|
+
So, eventually, the list of files to transfer is provided to the transfer agent using the [**transfer-spec**](#transfer-specification) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
|
|
2979
3144
|
The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
|
|
2980
3145
|
|
|
2981
3146
|
Possible values for option `sources` are:
|
|
@@ -3396,7 +3561,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
|
3396
3561
|
```text
|
|
3397
3562
|
ascli -h
|
|
3398
3563
|
NAME
|
|
3399
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
|
3564
|
+
ascli -- a command line tool for Aspera Applications (v4.18.1)
|
|
3400
3565
|
|
|
3401
3566
|
SYNOPSIS
|
|
3402
3567
|
ascli COMMANDS [OPTIONS] [ARGS]
|
|
@@ -3418,7 +3583,7 @@ COMMANDS
|
|
|
3418
3583
|
OPTIONS
|
|
3419
3584
|
Options begin with a '-' (minus), and value is provided on command line.
|
|
3420
3585
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
|
3421
|
-
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, yaml, zlib, extend, preset, vault
|
|
3586
|
+
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, secret, stdin, stdbin, yaml, zlib, extend, preset, vault
|
|
3422
3587
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
|
3423
3588
|
|
|
3424
3589
|
ARGS
|
|
@@ -3427,20 +3592,21 @@ ARGS
|
|
|
3427
3592
|
OPTIONS: global
|
|
3428
3593
|
--interactive=ENUM Use interactive input of missing params: [no], yes
|
|
3429
3594
|
--ask-options=ENUM Ask even optional options: [no], yes
|
|
3430
|
-
--
|
|
3595
|
+
--struct-parser=ENUM Default parser when expected value is a struct: json, ruby
|
|
3596
|
+
--format=ENUM Output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv, image
|
|
3431
3597
|
--output=VALUE Destination for results (String)
|
|
3432
3598
|
--display=ENUM Output only some information: [info], data, error
|
|
3433
3599
|
--fields=VALUE Comma separated list of: fields, or ALL, or DEF (String, Array, Regexp, Proc)
|
|
3434
3600
|
--select=VALUE Select only some items in lists: column, value (Hash, Proc)
|
|
3435
|
-
--table-style=VALUE Table display style
|
|
3601
|
+
--table-style=VALUE Table display style (Hash)
|
|
3436
3602
|
--flat-hash=ENUM Display deep values as additional keys: no, [yes]
|
|
3437
3603
|
--transpose-single=ENUM Single object fields output vertically: no, [yes]
|
|
3438
3604
|
--show-secrets=ENUM Show secrets on command output: [no], yes
|
|
3605
|
+
--image=VALUE Options for image display (Hash)
|
|
3439
3606
|
-h, --help Show this message
|
|
3440
3607
|
--bash-comp Generate bash completion for command
|
|
3441
3608
|
--show-config Display parameters used for the provided action
|
|
3442
3609
|
-v, --version Display version
|
|
3443
|
-
-w, --warnings Check for language warnings
|
|
3444
3610
|
--ui=ENUM Method to start browser: text, [graphical]
|
|
3445
3611
|
--log-level=ENUM Log level: trace2, trace1, debug, info, [warn], error, fatal, unknown
|
|
3446
3612
|
--logger=ENUM Logging method: [stderr], stdout, syslog
|
|
@@ -3451,7 +3617,7 @@ OPTIONS: global
|
|
|
3451
3617
|
--pid-file=VALUE Write process identifier to file, delete on exit (String)
|
|
3452
3618
|
|
|
3453
3619
|
COMMAND: config
|
|
3454
|
-
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
|
3620
|
+
SUBCOMMANDS: ascp check_update coffee detect documentation echo email_test file flush_tokens folder gem genkey image initdemo open plugins preset proxy_check pubkey remote_certificate smtp_settings throw vault wizard
|
|
3455
3621
|
OPTIONS:
|
|
3456
3622
|
--home=VALUE Home folder for tool (String)
|
|
3457
3623
|
--config-file=VALUE Path to YAML file with preset configuration
|
|
@@ -3461,7 +3627,6 @@ OPTIONS:
|
|
|
3461
3627
|
--query=VALUE Additional filter for for some commands (list/delete) (Hash)
|
|
3462
3628
|
--value=VALUE Value for create, update, list filter (Hash) (deprecated: (4.14) Use positional value for create/modify or option: query for list/delete)
|
|
3463
3629
|
--property=VALUE Name of property to set (modify operation)
|
|
3464
|
-
--id=VALUE Resource identifier (deprecated: (4.14) Use positional identifier after verb (modify,delete,show))
|
|
3465
3630
|
--bulk=ENUM Bulk operation (only some): [no], yes
|
|
3466
3631
|
--bfail=ENUM Bulk operation error handling: no, [yes]
|
|
3467
3632
|
-N, --no-default Do not load default configuration for plugin
|
|
@@ -3485,9 +3650,10 @@ OPTIONS:
|
|
|
3485
3650
|
--silent-insecure=ENUM Issue a warning if certificate is ignored: no, [yes]
|
|
3486
3651
|
--cert-stores=VALUE List of folder with trusted certificates (Array, String)
|
|
3487
3652
|
--http-options=VALUE Options for HTTP/S socket (Hash)
|
|
3653
|
+
--http-proxy=VALUE URL for HTTP proxy with optional credentials (String)
|
|
3488
3654
|
--cache-tokens=ENUM Save and reuse OAuth tokens: no, [yes]
|
|
3489
3655
|
--fpac=VALUE Proxy auto configuration script
|
|
3490
|
-
--proxy-credentials=VALUE HTTP proxy credentials
|
|
3656
|
+
--proxy-credentials=VALUE HTTP proxy credentials for fpac. Array: user,password (Array)
|
|
3491
3657
|
--ts=VALUE Override transfer spec values (Hash)
|
|
3492
3658
|
--to-folder=VALUE Destination folder for transferred files
|
|
3493
3659
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
|
@@ -3505,7 +3671,7 @@ OPTIONS:
|
|
|
3505
3671
|
|
|
3506
3672
|
|
|
3507
3673
|
COMMAND: node
|
|
3508
|
-
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer upload watch_folder
|
|
3674
|
+
SUBCOMMANDS: access_keys api_details asperabrowser async basic_token bearer_token browse central delete download events health http_node_download info license mkdir mkfile mklink rename search service simulator slash space ssync stream sync transfer transport upload watch_folder
|
|
3509
3675
|
OPTIONS:
|
|
3510
3676
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
|
3511
3677
|
--username=VALUE User's name to log in
|
|
@@ -3518,28 +3684,32 @@ OPTIONS:
|
|
|
3518
3684
|
--sync-info=VALUE Information for sync instance and sessions (Hash)
|
|
3519
3685
|
|
|
3520
3686
|
|
|
3521
|
-
COMMAND:
|
|
3522
|
-
SUBCOMMANDS: health
|
|
3687
|
+
COMMAND: faspio
|
|
3688
|
+
SUBCOMMANDS: bridges health
|
|
3523
3689
|
OPTIONS:
|
|
3524
3690
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
|
3525
3691
|
--username=VALUE User's name to log in
|
|
3526
3692
|
--password=VALUE User's password
|
|
3527
|
-
--
|
|
3528
|
-
--
|
|
3529
|
-
--
|
|
3530
|
-
--
|
|
3693
|
+
--auth=ENUM OAuth type of authentication: jwt, basic
|
|
3694
|
+
--client-id=VALUE OAuth client identifier
|
|
3695
|
+
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
|
3696
|
+
--passphrase=VALUE OAuth JWT RSA private key passphrase
|
|
3531
3697
|
|
|
3532
3698
|
|
|
3533
|
-
COMMAND:
|
|
3534
|
-
SUBCOMMANDS:
|
|
3699
|
+
COMMAND: orchestrator
|
|
3700
|
+
SUBCOMMANDS: health info plugins processes workflow
|
|
3535
3701
|
OPTIONS:
|
|
3536
3702
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
|
3537
3703
|
--username=VALUE User's name to log in
|
|
3538
3704
|
--password=VALUE User's password
|
|
3705
|
+
--result=VALUE Specify result value as: 'work_step:parameter'
|
|
3706
|
+
--synchronous=ENUM Wait for completion: [no], yes
|
|
3707
|
+
--ret-style=ENUM How return type is requested in api: header, [arg], ext
|
|
3708
|
+
--auth-style=ENUM Authentication type: arg_pass, [head_basic], apikey
|
|
3539
3709
|
|
|
3540
3710
|
|
|
3541
3711
|
COMMAND: alee
|
|
3542
|
-
SUBCOMMANDS: entitlement
|
|
3712
|
+
SUBCOMMANDS: entitlement health
|
|
3543
3713
|
OPTIONS:
|
|
3544
3714
|
--url=VALUE URL of application, e.g. https://faspex.example.com/aspera/faspex
|
|
3545
3715
|
--username=VALUE User's name to log in
|
|
@@ -3587,6 +3757,12 @@ OPTIONS:
|
|
|
3587
3757
|
--identity=VALUE Authentication URL (https://iam.cloud.ibm.com/identity)
|
|
3588
3758
|
|
|
3589
3759
|
|
|
3760
|
+
COMMAND: httpgw
|
|
3761
|
+
SUBCOMMANDS: health info
|
|
3762
|
+
OPTIONS:
|
|
3763
|
+
--url=VALUE URL of application, e.g. https://app.example.com/aspera/app
|
|
3764
|
+
|
|
3765
|
+
|
|
3590
3766
|
COMMAND: faspex
|
|
3591
3767
|
SUBCOMMANDS: address_book dropbox health login_methods me package source v4
|
|
3592
3768
|
OPTIONS:
|
|
@@ -3649,9 +3825,9 @@ OPTIONS:
|
|
|
3649
3825
|
--scope=VALUE OAuth scope for AoC API calls
|
|
3650
3826
|
--redirect-uri=VALUE OAuth API client redirect URI
|
|
3651
3827
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
|
3652
|
-
--passphrase=VALUE RSA private key passphrase
|
|
3828
|
+
--passphrase=VALUE RSA private key passphrase (String)
|
|
3653
3829
|
--workspace=VALUE Name of workspace (String, NilClass)
|
|
3654
|
-
--new-user-option=VALUE New user creation option for unknown package recipients
|
|
3830
|
+
--new-user-option=VALUE New user creation option for unknown package recipients (Hash)
|
|
3655
3831
|
--validate-metadata=ENUM Validate shared inbox metadata: no, [yes]
|
|
3656
3832
|
--validator=VALUE Identifier of validator (optional for central)
|
|
3657
3833
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
|
@@ -3725,7 +3901,7 @@ Aspera on Cloud and Faspex 5 rely on Oauth.
|
|
|
3725
3901
|
By default plugins are looked-up in folders specified by (multi-value) option `plugin_folder`:
|
|
3726
3902
|
|
|
3727
3903
|
```bash
|
|
3728
|
-
ascli --show-config --
|
|
3904
|
+
ascli --show-config --fields=plugin_folder
|
|
3729
3905
|
```
|
|
3730
3906
|
|
|
3731
3907
|
You can create the skeleton of a new plugin like this:
|
|
@@ -3805,7 +3981,7 @@ Several types of OAuth authentication are supported:
|
|
|
3805
3981
|
|
|
3806
3982
|
The authentication method is controlled by option `auth`.
|
|
3807
3983
|
|
|
3808
|
-
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset)
|
|
3984
|
+
For a **quick start**, follow the mandatory and sufficient section: [API Client Registration](#api-client-registration) (auth=web) as well as [[Option Preset](#option-preset) for Aspera on Cloud](#configuration-for-aspera-on-cloud).
|
|
3809
3985
|
|
|
3810
3986
|
For a more convenient, browser-less, experience follow the [JWT](#authentication-with-private-key) section (auth=jwt) in addition to Client Registration.
|
|
3811
3987
|
|
|
@@ -3842,9 +4018,9 @@ Once the client is registered, a **Client ID** and **Secret** are created, these
|
|
|
3842
4018
|
|
|
3843
4019
|
#### Configuration for Aspera on Cloud
|
|
3844
4020
|
|
|
3845
|
-
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset)
|
|
4021
|
+
If you did not use the wizard, you can also manually create a [Option Preset](#option-preset) for `ascli` in its configuration file.
|
|
3846
4022
|
|
|
3847
|
-
Lets create a [Option Preset](#option-preset)
|
|
4023
|
+
Lets create a [Option Preset](#option-preset) called: `my_aoc_org` using `ask` for interactive input (client info from previous step):
|
|
3848
4024
|
|
|
3849
4025
|
```bash
|
|
3850
4026
|
ascli config preset ask my_aoc_org url client_id client_secret
|
|
@@ -3858,7 +4034,7 @@ updated: my_aoc_org
|
|
|
3858
4034
|
|
|
3859
4035
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
|
3860
4036
|
|
|
3861
|
-
Define this [Option Preset](#option-preset)
|
|
4037
|
+
Define this [Option Preset](#option-preset) as default configuration for the `aspera` plugin:
|
|
3862
4038
|
|
|
3863
4039
|
```bash
|
|
3864
4040
|
ascli config preset set default aoc my_aoc_org
|
|
@@ -3949,9 +4125,9 @@ modified
|
|
|
3949
4125
|
|
|
3950
4126
|
> **Note:** The `aspera user info show` command can be used to verify modifications.
|
|
3951
4127
|
|
|
3952
|
-
#### [Option Preset](#option-preset)
|
|
4128
|
+
#### [Option Preset](#option-preset) modification for JWT
|
|
3953
4129
|
|
|
3954
|
-
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset)
|
|
4130
|
+
To activate default use of JWT authentication for `ascli` using the [Option Preset](#option-preset), do the following:
|
|
3955
4131
|
|
|
3956
4132
|
- Change auth method to JWT
|
|
3957
4133
|
- Provide location of private key
|
|
@@ -3985,7 +4161,7 @@ In that case, it is possible to list those shared folder by using a value for op
|
|
|
3985
4161
|
|
|
3986
4162
|
#### AoC: First Use
|
|
3987
4163
|
|
|
3988
|
-
Once client has been registered and [Option Preset](#option-preset)
|
|
4164
|
+
Once client has been registered and [Option Preset](#option-preset) created: `ascli` can be used:
|
|
3989
4165
|
|
|
3990
4166
|
```bash
|
|
3991
4167
|
ascli aoc files br /
|
|
@@ -4188,7 +4364,7 @@ Options:
|
|
|
4188
4364
|
#### Transfer: Using specific transfer ports
|
|
4189
4365
|
|
|
4190
4366
|
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
|
4191
|
-
The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
|
4367
|
+
The option `default_ports` ([yes]/no) allows `ascli` to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
|
4192
4368
|
|
|
4193
4369
|
#### Using ATS
|
|
4194
4370
|
|
|
@@ -4588,7 +4764,7 @@ General syntax:
|
|
|
4588
4764
|
ascli aoc packages send [package extended value] [other parameters such as file list and transfer parameters]
|
|
4589
4765
|
```
|
|
4590
4766
|
|
|
4591
|
-
Package creation parameter are sent as
|
|
4767
|
+
Package creation parameter are sent as **Command Parameter**.
|
|
4592
4768
|
Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
|
4593
4769
|
|
|
4594
4770
|
List allowed shared inbox destinations with:
|
|
@@ -4766,13 +4942,13 @@ ascli aoc files short_link public delete _id_
|
|
|
4766
4942
|
|
|
4767
4943
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
|
4768
4944
|
|
|
4769
|
-
Although optional, the creation of [Option Preset](#option-preset)
|
|
4945
|
+
Although optional, the creation of [Option Preset](#option-preset) is recommended to avoid placing all parameters in the command line.
|
|
4770
4946
|
|
|
4771
4947
|
Procedure to send a file from org1 to org2:
|
|
4772
4948
|
|
|
4773
|
-
- Get access to Organization 1 and create a [Option Preset](#option-preset)
|
|
4949
|
+
- Get access to Organization 1 and create a [Option Preset](#option-preset): e.g. `org1`, for instance, use the [Wizard](#configuration-using-wizard)
|
|
4774
4950
|
- Check that access works and locate the source file e.g. `mysourcefile`, e.g. using command `files browse`
|
|
4775
|
-
- Get access to Organization 2 and create a [Option Preset](#option-preset)
|
|
4951
|
+
- Get access to Organization 2 and create a [Option Preset](#option-preset): e.g. `org2`
|
|
4776
4952
|
- Check that access works and locate the destination folder `mydestfolder`
|
|
4777
4953
|
- Execute the following:
|
|
4778
4954
|
|
|
@@ -4812,6 +4988,7 @@ For instructions, refer to section `find` for plugin `node`.
|
|
|
4812
4988
|
admin analytics transfers nodes
|
|
4813
4989
|
admin analytics transfers organization --query=@json:'{"status":"completed","direction":"receive"}' --notify-to=my_email_external --notify-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
|
|
4814
4990
|
admin analytics transfers users --once_only=yes
|
|
4991
|
+
admin application list
|
|
4815
4992
|
admin ats access_key create --cloud=aws --region=my_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
|
4816
4993
|
admin ats access_key create --cloud=softlayer --region=my_region --params=@json:'{"id":"ak1ibmcloud","secret":"my_secret_here","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_bucket","credentials":{"access_key_id":"my_access_key","secret_access_key":"my_secret_key"},"path":"/"}}'
|
|
4817
4994
|
admin ats access_key delete ak1ibmcloud
|
|
@@ -4822,36 +4999,35 @@ admin ats cluster list
|
|
|
4822
4999
|
admin ats cluster show --cloud=aws --region=eu-west-1
|
|
4823
5000
|
admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
|
4824
5001
|
admin auth_providers list
|
|
4825
|
-
admin
|
|
4826
|
-
admin
|
|
4827
|
-
admin
|
|
4828
|
-
admin
|
|
4829
|
-
admin
|
|
4830
|
-
admin
|
|
4831
|
-
admin
|
|
4832
|
-
admin
|
|
4833
|
-
admin
|
|
4834
|
-
admin
|
|
4835
|
-
admin
|
|
4836
|
-
admin
|
|
4837
|
-
admin
|
|
4838
|
-
admin
|
|
4839
|
-
admin
|
|
4840
|
-
admin
|
|
4841
|
-
admin
|
|
4842
|
-
admin
|
|
4843
|
-
admin
|
|
4844
|
-
admin
|
|
4845
|
-
admin
|
|
4846
|
-
admin
|
|
4847
|
-
admin
|
|
4848
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
|
4849
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
|
4850
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
|
4851
|
-
admin resource node do %name:my_ak_name --secret=my_ak_secret v3 events
|
|
4852
|
-
admin resource workspace list
|
|
4853
|
-
admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
|
5002
|
+
admin client list
|
|
5003
|
+
admin client_access_key list
|
|
5004
|
+
admin client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
|
|
5005
|
+
admin client_registration_token delete client_reg_id
|
|
5006
|
+
admin client_registration_token list
|
|
5007
|
+
admin contact list
|
|
5008
|
+
admin dropbox list
|
|
5009
|
+
admin dropbox_membership list
|
|
5010
|
+
admin group list
|
|
5011
|
+
admin kms_profile list
|
|
5012
|
+
admin node do %name:my_ak_name --secret=my_ak_secret browse /
|
|
5013
|
+
admin node do %name:my_ak_name --secret=my_ak_secret delete /folder1
|
|
5014
|
+
admin node do %name:my_ak_name --secret=my_ak_secret mkdir /folder1
|
|
5015
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key create @json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
|
5016
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 access_key delete testsub1
|
|
5017
|
+
admin node do %name:my_ak_name --secret=my_ak_secret v3 events
|
|
5018
|
+
admin node list
|
|
5019
|
+
admin operation list
|
|
5020
|
+
admin organization show
|
|
5021
|
+
admin package list --http-options=@json:'{"read_timeout":120.0}'
|
|
5022
|
+
admin saml_configuration list
|
|
5023
|
+
admin self show
|
|
5024
|
+
admin short_link list
|
|
4854
5025
|
admin subscription
|
|
5026
|
+
admin user list
|
|
5027
|
+
admin user modify %name:my_user_email @json:'{"deactivated":false}'
|
|
5028
|
+
admin workspace list
|
|
5029
|
+
admin workspace_membership list
|
|
5030
|
+
admin workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
|
4855
5031
|
automation workflow action wf_id create @json:'{"name":"toto"}' \
|
|
4856
5032
|
automation workflow create @json:'{"name":"test_workflow"}'
|
|
4857
5033
|
automation workflow delete wf_id
|
|
@@ -4865,6 +5041,9 @@ files browse /
|
|
|
4865
5041
|
files browse / --url=my_private_link
|
|
4866
5042
|
files browse / --url=my_public_link_folder_no_pass
|
|
4867
5043
|
files browse / --url=my_public_link_folder_pass --password=my_public_link_password
|
|
5044
|
+
files browse my_remote_file
|
|
5045
|
+
files browse my_remote_folder
|
|
5046
|
+
files browse my_remote_folder/
|
|
4868
5047
|
files delete /testsrc
|
|
4869
5048
|
files download --transfer=alpha testdst/test_file.bin
|
|
4870
5049
|
files download --transfer=connect testdst/test_file.bin
|
|
@@ -4924,9 +5103,9 @@ user workspaces list
|
|
|
4924
5103
|
|
|
4925
5104
|
ATS is usable either :
|
|
4926
5105
|
|
|
4927
|
-
- From an AoC subscription : ascli aoc admin ats : use AoC authentication
|
|
5106
|
+
- From an AoC subscription : `ascli aoc admin ats` : use AoC authentication
|
|
4928
5107
|
|
|
4929
|
-
- Or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
|
|
5108
|
+
- Or from an IBM Cloud subscription : `ascli ats` : use IBM Cloud API key authentication
|
|
4930
5109
|
|
|
4931
5110
|
### IBM Cloud ATS : Creation of api key
|
|
4932
5111
|
|
|
@@ -5098,8 +5277,8 @@ mkdir my_upload_folder/target_hot
|
|
|
5098
5277
|
mv my_upload_folder/200KB.2 my_upload_folder/to.delete
|
|
5099
5278
|
sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"}}'
|
|
5100
5279
|
sync admin status --sync-info=@json:'{"name":"sync2"}'
|
|
5101
|
-
sync admin status
|
|
5102
|
-
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"
|
|
5280
|
+
sync admin status my_srv_sync --sync-info=@json:'{"sessions":[{"name":"my_srv_sync","local_dir":"/data/local_sync"}]}'
|
|
5281
|
+
sync start --sync-info=@json:'{"instance":{"quiet":false},"sessions":[{"name":"my_srv_sync","direction":"pull","remote_dir":"my_inside_folder","local_dir":"/data/local_sync","reset":true}]}'
|
|
5103
5282
|
sync start --sync-info=@json:'{"name":"sync2","local":{"path":"/data/local_sync"},"remote":{"path":"my_inside_folder"},"reset":true,"quiet":false}'
|
|
5104
5283
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}'
|
|
5105
5284
|
upload 'faux:///test1?100m' 'faux:///test2?100m' --to-folder=/Upload --ts=@json:'{"target_rate_kbps":1000000,"resume_policy":"none","precalculate_job_size":true}' --transfer-info=@json:'{"quiet":false}' --progress=no
|
|
@@ -5203,7 +5382,7 @@ ascli server browse /aspera-test-dir-large
|
|
|
5203
5382
|
ascli server download /aspera-test-dir-large/200MB
|
|
5204
5383
|
```
|
|
5205
5384
|
|
|
5206
|
-
`initdemo` creates a [Option Preset](#option-preset)
|
|
5385
|
+
`initdemo` creates a [Option Preset](#option-preset) `demoserver` and set it as default for plugin `server`.
|
|
5207
5386
|
|
|
5208
5387
|
If an SSH private key is used for authentication with a passphrase, the passphrase needs to be provided to both options: `ssh_options`, for browsing, and `ts` for transfers:
|
|
5209
5388
|
|
|
@@ -5235,6 +5414,18 @@ Example:
|
|
|
5235
5414
|
- `ascli node browse /` : list files with **gen3/node user** API
|
|
5236
5415
|
- `ascli node access_key do self browse /` : list files with **gen4/access key** API
|
|
5237
5416
|
|
|
5417
|
+
#### Browse
|
|
5418
|
+
|
|
5419
|
+
Native API parameters can be placed in option `query`.
|
|
5420
|
+
|
|
5421
|
+
Special parameters can be placed in option `query` for "gen3" browse:
|
|
5422
|
+
|
|
5423
|
+
| Parameter | Description |
|
|
5424
|
+
|-----------|-------------|
|
|
5425
|
+
| `recursive` | Recursively list files |
|
|
5426
|
+
| `max` | Maximum number of files to list |
|
|
5427
|
+
| `self` | Offset in the list |
|
|
5428
|
+
|
|
5238
5429
|
### Operation `find` on **gen4/access key**
|
|
5239
5430
|
|
|
5240
5431
|
The command `find <folder> [filter_expr]` is available for **gen4/access key**, under `access_keys do self`.
|
|
@@ -5337,7 +5528,7 @@ There are three sync types of operations:
|
|
|
5337
5528
|
|
|
5338
5529
|
It is possible to start a FASPStream session using the node API:
|
|
5339
5530
|
|
|
5340
|
-
Use the command `ascli node stream create --ts=@json:<value>`, with [
|
|
5531
|
+
Use the command `ascli node stream create --ts=@json:<value>`, with [**transfer-spec**](#transfer-specification):
|
|
5341
5532
|
|
|
5342
5533
|
```json
|
|
5343
5534
|
{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"my_pass_here"}
|
|
@@ -5643,6 +5834,7 @@ sync start --sync-info=@json:'{"name":"my_node_sync2","reset":true,"direction":"
|
|
|
5643
5834
|
sync start --sync-info=@json:'{"sessions":[{"name":"my_node_sync1","direction":"pull","local_dir":"/data/local_sync","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
|
5644
5835
|
transfer list --query=@json:'{"active_only":true}'
|
|
5645
5836
|
transfer sessions
|
|
5837
|
+
transport
|
|
5646
5838
|
upload --to-folder=my_upload_folder --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"https://node.example.com/path@","username":"my_username","password":"my_password_here"}'
|
|
5647
5839
|
upload --username=my_ak_name --password=my_ak_secret test_file.bin
|
|
5648
5840
|
upload test_file.bin --to-folder=my_upload_folder --ts=@json:'{"target_rate_cap_kbps":10000}'
|
|
@@ -5664,7 +5856,7 @@ IBM Aspera's newer self-managed application.
|
|
|
5664
5856
|
|
|
5665
5857
|
> **Note:** If you have a Faspex 5 public link, provide it, unchanged, through the option `url`
|
|
5666
5858
|
|
|
5667
|
-
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset)
|
|
5859
|
+
For a quick start, one can use the wizard, which will help creating a [Option Preset](#option-preset):
|
|
5668
5860
|
|
|
5669
5861
|
```bash
|
|
5670
5862
|
ascli config wizard
|
|
@@ -5802,24 +5994,30 @@ ascli config preset update f5boot --url=https://localhost/aspera/faspex --auth=b
|
|
|
5802
5994
|
> **Note:** Add `ascli faspex5` in front of the commands:
|
|
5803
5995
|
|
|
5804
5996
|
```bash
|
|
5805
|
-
admin
|
|
5806
|
-
admin
|
|
5807
|
-
admin
|
|
5808
|
-
admin
|
|
5809
|
-
admin
|
|
5810
|
-
admin
|
|
5811
|
-
admin
|
|
5812
|
-
admin
|
|
5813
|
-
admin
|
|
5814
|
-
admin
|
|
5815
|
-
admin
|
|
5816
|
-
admin
|
|
5817
|
-
admin
|
|
5818
|
-
admin
|
|
5819
|
-
admin
|
|
5820
|
-
admin
|
|
5997
|
+
admin accounts list
|
|
5998
|
+
admin clean_deleted
|
|
5999
|
+
admin contacts list
|
|
6000
|
+
admin distribution_lists create @json:'{"name":"test4","contacts":[{"name":"john@example.com"}]}'
|
|
6001
|
+
admin distribution_lists delete %name:test4
|
|
6002
|
+
admin distribution_lists list --query=@json:'{"type":"global"}'
|
|
6003
|
+
admin event app
|
|
6004
|
+
admin event web
|
|
6005
|
+
admin jobs list --query=@json:'{"job_type":"email","status":"failed"}' --fields=id,error_desc
|
|
6006
|
+
admin metadata_profiles list
|
|
6007
|
+
admin node list
|
|
6008
|
+
admin oauth_clients list
|
|
6009
|
+
admin registrations list
|
|
6010
|
+
admin saml_configs list
|
|
6011
|
+
admin shared_inboxes invite %name:my_shared_box_name johnny@example.com
|
|
6012
|
+
admin shared_inboxes list
|
|
6013
|
+
admin shared_inboxes list --query=@json:'{"all":true}'
|
|
6014
|
+
admin shared_inboxes members %name:my_shared_box_name create %name:john@example.com
|
|
6015
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:john@example.com
|
|
6016
|
+
admin shared_inboxes members %name:my_shared_box_name delete %name:johnny@example.com
|
|
6017
|
+
admin shared_inboxes members %name:my_shared_box_name list
|
|
5821
6018
|
admin smtp show
|
|
5822
6019
|
admin smtp test my_email_external
|
|
6020
|
+
admin workgroups list
|
|
5823
6021
|
bearer_token
|
|
5824
6022
|
gateway --pid-file=pid_f5_fxgw https://localhost:12346/aspera/faspex &
|
|
5825
6023
|
health
|
|
@@ -5853,7 +6051,7 @@ user profile show
|
|
|
5853
6051
|
```
|
|
5854
6052
|
|
|
5855
6053
|
Most commands are directly REST API calls.
|
|
5856
|
-
Parameters to commands are carried through option `query`, as extended value, for `list`, or through
|
|
6054
|
+
Parameters to commands are carried through option `query`, as extended value, for `list`, or through **Command Parameter** for creation.
|
|
5857
6055
|
One can conveniently use the JSON format with prefix `@json:`.
|
|
5858
6056
|
|
|
5859
6057
|
> **Note:** The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/catalog?search="faspex+5") under **IBM Aspera Faspex API**.
|
|
@@ -5880,7 +6078,7 @@ To select another inbox, use option `box` with one of the following values:
|
|
|
5880
6078
|
|
|
5881
6079
|
### Faspex 5: Send a package
|
|
5882
6080
|
|
|
5883
|
-
The `Hash` creation
|
|
6081
|
+
The `Hash` creation **Command Parameter** provided to command `faspex5 packages send [extended value: Hash with package info ] [files...]` corresponds to the Faspex 5 API: `POST /packages`.
|
|
5884
6082
|
|
|
5885
6083
|
The interface is the one of the API (Refer to Faspex5 API documentation, or look at request in browser).
|
|
5886
6084
|
|
|
@@ -5931,7 +6129,7 @@ Basically, add the field `metadata`, with one key per metadata and the value is
|
|
|
5931
6129
|
|
|
5932
6130
|
### Faspex 5: List packages
|
|
5933
6131
|
|
|
5934
|
-
Option `box` can be used to list packages from a specific box (see
|
|
6132
|
+
Option `box` can be used to list packages from a specific box (see [Inbox Selection](#faspex-5-inbox-selection) above).
|
|
5935
6133
|
|
|
5936
6134
|
Option `query` can be used to filter the list of packages, based on native API parameters, directly sent to [Faspex 5 API `GET /packages`](https://developer.ibm.com/apis/catalog/aspera--ibm-aspera-faspex-5-0-api/api/API--aspera--ibm-aspera-faspex-api#getAllPackages).
|
|
5937
6135
|
|
|
@@ -5944,7 +6142,7 @@ Option `query` can be used to filter the list of packages, based on native API p
|
|
|
5944
6142
|
| `max` | Special | Maximum number of items to retrieve (stop pages when the maximum is passed) |
|
|
5945
6143
|
| `pmax` | Special | Maximum number of pages to request (stop pages when the maximum is passed) |
|
|
5946
6144
|
|
|
5947
|
-
A
|
|
6145
|
+
A **Command Parameter** in last position, of type `Proc`, can be used to filter the list of packages.
|
|
5948
6146
|
This advantage of this method is that the expression can be any test, even complex, as it is Ruby code.
|
|
5949
6147
|
But the disadvantage is that the filtering is done in `ascli` and not in Faspex 5, so it is less efficient.
|
|
5950
6148
|
|
|
@@ -5972,7 +6170,7 @@ Option `query` is available.
|
|
|
5972
6170
|
|
|
5973
6171
|
To list recursively add option `--query=@json:{"recursive":true}`.
|
|
5974
6172
|
|
|
5975
|
-
> **Note:** Option `
|
|
6173
|
+
> **Note:** Option `recursive` makes recursive API calls, so it can take a long time on large packages.
|
|
5976
6174
|
|
|
5977
6175
|
To limit the number of items retrieved, use option `--query=@json:{"max":10}`.
|
|
5978
6176
|
|
|
@@ -5982,7 +6180,7 @@ To receive one, or several packages at once, use command `faspex5 packages recei
|
|
|
5982
6180
|
Provide either a single package id, or an extended value `Array` of package ids, e.g. `@list:,1,2,3` as argument.
|
|
5983
6181
|
|
|
5984
6182
|
The same options as for `faspex5 packages list` can be used to select the box and filter the packages to download.
|
|
5985
|
-
I.e. options `box` and `query`, as well as last
|
|
6183
|
+
I.e. options `box` and `query`, as well as last **Command Parameter** `Proc` (filter).
|
|
5986
6184
|
|
|
5987
6185
|
Option `--once-only=yes` can be used, for "cargo-like" behavior.
|
|
5988
6186
|
Special package id `INIT` initializes the persistency of already received packages when option `--once-only=yes` is used.
|
|
@@ -6004,6 +6202,7 @@ ascli faspex5 admin res workgroup list
|
|
|
6004
6202
|
```
|
|
6005
6203
|
|
|
6006
6204
|
If you are admin or manager, add option: `--query=@json:'{"all":true}'`, this will list items you manage, even if you do not belong to them.
|
|
6205
|
+
Example:
|
|
6007
6206
|
|
|
6008
6207
|
```bash
|
|
6009
6208
|
ascli faspex5 admin res shared list --query=@json:'{"all":true}' --fields=id,name
|
|
@@ -6021,10 +6220,10 @@ It is equivalent to:
|
|
|
6021
6220
|
ascli faspex5 admin res shared_inboxes invite '%name:the shared inbox' @json:'{"email_address":"john@example.com"}'
|
|
6022
6221
|
```
|
|
6023
6222
|
|
|
6024
|
-
Other payload parameters are possible
|
|
6223
|
+
Other payload parameters are possible for `invite` in this last `Hash` **Command Parameter**:
|
|
6025
6224
|
|
|
6026
6225
|
```json
|
|
6027
|
-
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3
|
|
6226
|
+
{"description":"blah","prevent_http_upload":true,"custom_link_expiration_policy":false,"invitation_expires_after_upload":false,"set_invitation_link_expiration":false,"invitation_expiration_days":3}
|
|
6028
6227
|
```
|
|
6029
6228
|
|
|
6030
6229
|
### Faspex 5: Create Metadata profile
|
|
@@ -6085,6 +6284,37 @@ Public invitations are for external users, provide just the email address.
|
|
|
6085
6284
|
|
|
6086
6285
|
Private invitations are for internal users, provide the user or shared inbox identifier through field `recipient_name`.
|
|
6087
6286
|
|
|
6287
|
+
### Faspex 5: Cleanup packages
|
|
6288
|
+
|
|
6289
|
+
> **Note:** Operation requires admin level.
|
|
6290
|
+
|
|
6291
|
+
Automated cleanup period can be displayed with:
|
|
6292
|
+
|
|
6293
|
+
```bash
|
|
6294
|
+
ascli faspex5 admin configuration show --fields=days_before_deleting_package_records
|
|
6295
|
+
```
|
|
6296
|
+
|
|
6297
|
+
This parameter can also be modified, for example:
|
|
6298
|
+
|
|
6299
|
+
```bash
|
|
6300
|
+
ascli faspex5 admin configuration modify @json:'{"days_before_deleting_package_records":30}'
|
|
6301
|
+
```
|
|
6302
|
+
|
|
6303
|
+
To start package purge, i.e. permanently remove packages marked for deletion older than `days_before_deleting_package_records`, use command:
|
|
6304
|
+
|
|
6305
|
+
```bash
|
|
6306
|
+
ascli faspex5 admin clean_deleted
|
|
6307
|
+
```
|
|
6308
|
+
|
|
6309
|
+
To delete all packages, one can use the following command:
|
|
6310
|
+
|
|
6311
|
+
```bash
|
|
6312
|
+
ascli faspex5 packages list --box=ALL --format=yaml --fields=id | ascli faspex5 packages delete @yaml:@stdin:
|
|
6313
|
+
```
|
|
6314
|
+
|
|
6315
|
+
> **Note:** Above command will mark all packages for deletion, and will be permanently removed after the configured period (`clean_deleted` command).
|
|
6316
|
+
> It is possible to add a filter to the list command to only delete packages matching some criteria, e.g. using `--select=@ruby:`.
|
|
6317
|
+
|
|
6088
6318
|
### Faspex 5: Faspex 4-style postprocessing
|
|
6089
6319
|
|
|
6090
6320
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
|
@@ -6132,6 +6362,14 @@ Then, the postprocessing script executed will be `script1.sh`.
|
|
|
6132
6362
|
|
|
6133
6363
|
Environment variables at set to the values provided by the web hook which are the same as Faspex 4 postprocessing.
|
|
6134
6364
|
|
|
6365
|
+
### Faspex 5: Missing commands
|
|
6366
|
+
|
|
6367
|
+
If a command is missing, then it is still possible to execute command by calling directly the API on the command line using `curl`:
|
|
6368
|
+
|
|
6369
|
+
```bash
|
|
6370
|
+
curl -H "Authorization: $(ascli ascli bearer)" https://faspex5.example.com/aspera/faspex/api/v5/api_endpoint_here
|
|
6371
|
+
```
|
|
6372
|
+
|
|
6135
6373
|
## Plugin: `faspex`: IBM Aspera Faspex v4
|
|
6136
6374
|
|
|
6137
6375
|
> **Note:** For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
|
|
@@ -6210,13 +6448,13 @@ The contents of `delivery_info` is directly the contents of the `send` v3 [API o
|
|
|
6210
6448
|
Example:
|
|
6211
6449
|
|
|
6212
6450
|
```bash
|
|
6213
|
-
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}'
|
|
6451
|
+
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["someuser@example.com"]}' /tmp/file1 /home/bar/file2
|
|
6214
6452
|
```
|
|
6215
6453
|
|
|
6216
6454
|
If the recipient is a dropbox or workgroup: provide the name of the dropbox or workgroup preceded with `*` in the `recipients` field of the `delivery_info` option:
|
|
6217
6455
|
`"recipients":["*MyDropboxName"]`
|
|
6218
6456
|
|
|
6219
|
-
Additional optional parameters in `delivery_info`:
|
|
6457
|
+
Additional optional parameters in mandatory option `delivery_info`:
|
|
6220
6458
|
|
|
6221
6459
|
- Package Note: : `"note":"note this and that"`
|
|
6222
6460
|
- Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
|
@@ -6233,7 +6471,7 @@ The value is a `Hash` with the following keys:
|
|
|
6233
6471
|
|
|
6234
6472
|
### Email notification on transfer
|
|
6235
6473
|
|
|
6236
|
-
Like for any transfer, a notification can be sent by email using
|
|
6474
|
+
Like for any transfer, a notification can be sent by email using options: `notify_to` and `notify_template` .
|
|
6237
6475
|
|
|
6238
6476
|
Example:
|
|
6239
6477
|
|
|
@@ -6368,12 +6606,14 @@ admin user ldap add the_name
|
|
|
6368
6606
|
admin user local list
|
|
6369
6607
|
admin user saml import @json:'{"id":"the_id","name_id":"the_name"}'
|
|
6370
6608
|
files browse /
|
|
6609
|
+
files delete my_share1/new_folder
|
|
6371
6610
|
files delete my_share1/test_file.bin
|
|
6372
6611
|
files download --to-folder=. my_share1/test_file.bin
|
|
6373
|
-
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
|
6374
|
-
files
|
|
6612
|
+
files download --to-folder=. my_share1/test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
|
6613
|
+
files mkdir my_share1/new_folder
|
|
6614
|
+
files upload --to-folder=https://shares.share1 'faux:///testfile?1m' --transfer=httpgw --transfer-info=@json:'{"url":"my_example.com/path@","synchronous":true,"api_version":"v1","upload_chunk_size":100000}'
|
|
6375
6615
|
files upload --to-folder=my_share1 test_file.bin
|
|
6376
|
-
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
|
6616
|
+
files upload --to-folder=my_share1 test_file.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://tst.example.com/path@"}'
|
|
6377
6617
|
health
|
|
6378
6618
|
```
|
|
6379
6619
|
|
|
@@ -6527,11 +6767,46 @@ node info --log-level=trace2
|
|
|
6527
6767
|
node upload test_file.bin
|
|
6528
6768
|
```
|
|
6529
6769
|
|
|
6770
|
+
## Plugin: `httpgw`: HTTP Gateway
|
|
6771
|
+
|
|
6772
|
+
### Httpgw sample commands
|
|
6773
|
+
|
|
6774
|
+
> **Note:** Add `ascli httpgw` in front of the commands:
|
|
6775
|
+
|
|
6776
|
+
```bash
|
|
6777
|
+
health
|
|
6778
|
+
```
|
|
6779
|
+
|
|
6780
|
+
## Plugin: `faspio`: Faspio Gateway
|
|
6781
|
+
|
|
6782
|
+
### Faspio sample commands
|
|
6783
|
+
|
|
6784
|
+
> **Note:** Add `ascli faspio` in front of the commands:
|
|
6785
|
+
|
|
6786
|
+
```bash
|
|
6787
|
+
bridges create @json:'{"name":"test1","local":{"protocol":"tcp","tls_enabled":false,"port":"3000","bind_address":"127.0.0.1"},"forward":{"protocol":"fasp","tls_enabled":false,"port":"3994","bind_address":"127.0.0.1","host":["10.0.0.1"]}}'
|
|
6788
|
+
bridges delete --bulk=yes @json:@stdin:
|
|
6789
|
+
bridges list
|
|
6790
|
+
health
|
|
6791
|
+
```
|
|
6792
|
+
|
|
6793
|
+
## Plugin: `alee`: Aspera License Entitlement Engine
|
|
6794
|
+
|
|
6795
|
+
Retrieve information on subscription.
|
|
6796
|
+
|
|
6797
|
+
### Alee sample commands
|
|
6798
|
+
|
|
6799
|
+
> **Note:** Add `ascli alee` in front of the commands:
|
|
6800
|
+
|
|
6801
|
+
```bash
|
|
6802
|
+
health
|
|
6803
|
+
```
|
|
6804
|
+
|
|
6530
6805
|
## Plugin: `preview`: Preview generator for AoC
|
|
6531
6806
|
|
|
6532
6807
|
The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
|
6533
6808
|
It uses the **node API** of Aspera HSTS and requires use of Access Keys and its **storage root**.
|
|
6534
|
-
Several
|
|
6809
|
+
Several options can be used to tune several aspects:
|
|
6535
6810
|
|
|
6536
6811
|
- Methods for detection of new files needing generation
|
|
6537
6812
|
- Methods for generation of video preview
|
|
@@ -6558,10 +6833,11 @@ asnodeadmin --reload
|
|
|
6558
6833
|
|
|
6559
6834
|
If another folder is configured on the HSTS, then specify it to `ascli` using the option `previews_folder`.
|
|
6560
6835
|
|
|
6561
|
-
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024
|
|
6562
|
-
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384
|
|
6836
|
+
The HSTS node API limits any preview file to a parameter: `max_request_file_create_size_kb` (1 KB is 1024 Bytes).
|
|
6837
|
+
This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KB, i.e. 16MB.
|
|
6563
6838
|
|
|
6564
|
-
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
|
6839
|
+
To change this parameter in `aspera.conf`, use `asconfigurator`.
|
|
6840
|
+
To display the value, use `asuserdata`:
|
|
6565
6841
|
|
|
6566
6842
|
```bash
|
|
6567
6843
|
asuserdata -a | grep max_request_file_create_size_kb
|
|
@@ -6642,7 +6918,7 @@ chmod a+x /usr/bin/unoconv
|
|
|
6642
6918
|
The preview generator should be executed as a non-user.
|
|
6643
6919
|
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
|
6644
6920
|
|
|
6645
|
-
Like any `ascli` commands,
|
|
6921
|
+
Like any `ascli` commands, options can be passed on command line or using a configuration [Option Preset](#option-preset).
|
|
6646
6922
|
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
|
6647
6923
|
|
|
6648
6924
|
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
|
@@ -6902,9 +7178,9 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
|
6902
7178
|
- `ascp` has an option to send only files not modified since the last X seconds: `--exclude-newer-than`, `--exclude-older-than` (`exclude_newer_than`,`exclude_older_than`)
|
|
6903
7179
|
- `--src-base` (`src_base`) if top level folder name shall not be created on destination
|
|
6904
7180
|
|
|
6905
|
-
> **Note:** `ascli` takes transfer parameters exclusively as a [
|
|
7181
|
+
> **Note:** `ascli` takes transfer parameters exclusively as a [**transfer-spec**](#transfer-specification), with `ts` option.
|
|
6906
7182
|
>
|
|
6907
|
-
> **Note:** Most, but not all, native `ascp` arguments are available as standard [
|
|
7183
|
+
> **Note:** Most, but not all, native `ascp` arguments are available as standard [**transfer-spec**](#transfer-specification) parameters.
|
|
6908
7184
|
>
|
|
6909
7185
|
> **Note:** Only for the [`direct`](#agent-direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
|
6910
7186
|
|
|
@@ -6914,11 +7190,11 @@ Virtually any transfer on a **repository** on a regular basis might emulate a ho
|
|
|
6914
7190
|
|
|
6915
7191
|
> **Note:** File detection is not based on events (`inotify`, etc...), but on a simple folder scan on source side.
|
|
6916
7192
|
>
|
|
6917
|
-
> **Note:**
|
|
7193
|
+
> **Note:** Options may be saved in a [Option Preset](#option-preset) and used with `-P`.
|
|
6918
7194
|
|
|
6919
7195
|
#### Scheduling
|
|
6920
7196
|
|
|
6921
|
-
Once `ascli`
|
|
7197
|
+
Once `ascli` command line arguments are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
|
6922
7198
|
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
|
6923
7199
|
|
|
6924
7200
|
### Example: Upload hot folder
|
|
@@ -7081,7 +7357,7 @@ Transfer is: <%=global_transfer_status%>
|
|
|
7081
7357
|
|
|
7082
7358
|
This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
|
|
7083
7359
|
|
|
7084
|
-
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [
|
|
7360
|
+
It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [**transfer-spec**](#transfer-specification) is:
|
|
7085
7361
|
|
|
7086
7362
|
- Common to Aspera Node API (HTTP POST /ops/transfer)
|
|
7087
7363
|
- Common to Aspera Connect API (browser javascript startTransfer)
|
|
@@ -7091,7 +7367,7 @@ Hopefully, IBM integrates this directly in `ascp`, and this tool is made redunda
|
|
|
7091
7367
|
|
|
7092
7368
|
This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.
|
|
7093
7369
|
|
|
7094
|
-
`ascli` expect one single argument: a session specification that contains parameters and a [
|
|
7370
|
+
`ascli` expect one single argument: a session specification that contains parameters and a [**transfer-spec**](#transfer-specification).
|
|
7095
7371
|
|
|
7096
7372
|
If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted on stdin.
|
|
7097
7373
|
|
|
@@ -7103,7 +7379,7 @@ Top level parameters supported by `asession`:
|
|
|
7103
7379
|
|
|
7104
7380
|
| parameter | description |
|
|
7105
7381
|
|-----------|-------------|
|
|
7106
|
-
| `spec` | the [
|
|
7382
|
+
| `spec` | the [**transfer-spec**](#transfer-specification) |
|
|
7107
7383
|
| `agent` | same parameters as transfer-info for agent `direct` |
|
|
7108
7384
|
| `loglevel` | log level of `asession` |
|
|
7109
7385
|
| `file_list_folder` | the folder used to store (for garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists` |
|
|
@@ -7136,7 +7412,7 @@ asession < session.json
|
|
|
7136
7412
|
|
|
7137
7413
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in `ascp` manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be **snake** style, for example: `LongParameter` -> `long_parameter`
|
|
7138
7414
|
|
|
7139
|
-
This is particularly useful for a persistent session ( with the [
|
|
7415
|
+
This is particularly useful for a persistent session ( with the [**transfer-spec**](#transfer-specification) parameter: `"keepalive":true` )
|
|
7140
7416
|
|
|
7141
7417
|
```json
|
|
7142
7418
|
asession
|
|
@@ -7205,7 +7481,7 @@ So, it evolved into `ascli`:
|
|
|
7205
7481
|
|
|
7206
7482
|
- Portable: works on platforms supporting `ruby` (and `ascp`)
|
|
7207
7483
|
- Easy to install with the `gem` utility
|
|
7208
|
-
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [
|
|
7484
|
+
- Supports transfers with multiple [Transfer Agents](#transfer-clients-agents), that's why transfer parameters moved from `ascp` command line to [**transfer-spec**](#transfer-specification) (more reliable , more standard)
|
|
7209
7485
|
- `ruby` is consistent with other Aspera products
|
|
7210
7486
|
|
|
7211
7487
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
|
@@ -7215,7 +7491,7 @@ Enjoy a coffee on me:
|
|
|
7215
7491
|
|
|
7216
7492
|
```bash
|
|
7217
7493
|
ascli config coffee --ui=text
|
|
7218
|
-
ascli config coffee --ui=text --
|
|
7494
|
+
ascli config coffee --ui=text --image=@json:'{"text":true}'
|
|
7219
7495
|
ascli config coffee
|
|
7220
7496
|
```
|
|
7221
7497
|
|
|
@@ -7244,11 +7520,13 @@ This also requires Ruby header files.
|
|
|
7244
7520
|
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
|
7245
7521
|
`ruby-dev` or `ruby-devel`, depending on distribution.
|
|
7246
7522
|
|
|
7247
|
-
###
|
|
7523
|
+
### ED25519 key not supported
|
|
7248
7524
|
|
|
7249
7525
|
ED25519 keys are deactivated since `ascli` version 0.9.24 as it requires additional gems that require native compilation and thus caused problems.
|
|
7250
7526
|
This type of key will just be ignored.
|
|
7251
7527
|
|
|
7528
|
+
To re-activate, set env var `ASCLI_ENABLE_ED25519` to `true`.
|
|
7529
|
+
|
|
7252
7530
|
Without this deactivation, if such key was present in user's `.ssh` folder then the following error was generated:
|
|
7253
7531
|
|
|
7254
7532
|
```output
|
|
@@ -7258,4 +7536,5 @@ OpenSSH keys only supported if ED25519 is available
|
|
|
7258
7536
|
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
|
7259
7537
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
|
7260
7538
|
|
|
7261
|
-
|
|
7539
|
+
In addition, host keys of type: `ecdsa-sha2` and `ecdh-sha2` are also de-activated by default.
|
|
7540
|
+
To re-activate, set env var `ASCLI_ENABLE_ECDSHA2` to `true`.
|