aspera-cli 4.12.0 → 4.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +17 -0
- data/CONTRIBUTING.md +97 -22
- data/README.md +548 -394
- data/bin/ascli +3 -3
- data/docs/test_env.conf +12 -5
- data/lib/aspera/aoc.rb +42 -42
- data/lib/aspera/ascmd.rb +4 -3
- data/lib/aspera/cli/extended_value.rb +24 -37
- data/lib/aspera/cli/formatter.rb +6 -0
- data/lib/aspera/cli/info.rb +2 -4
- data/lib/aspera/cli/main.rb +6 -0
- data/lib/aspera/cli/manager.rb +15 -6
- data/lib/aspera/cli/plugin.rb +1 -5
- data/lib/aspera/cli/plugins/aoc.rb +23 -6
- data/lib/aspera/cli/plugins/config.rb +13 -6
- data/lib/aspera/cli/plugins/faspex.rb +4 -3
- data/lib/aspera/cli/plugins/faspex5.rb +175 -42
- data/lib/aspera/cli/plugins/node.rb +107 -50
- data/lib/aspera/cli/plugins/preview.rb +3 -3
- data/lib/aspera/cli/plugins/server.rb +11 -1
- data/lib/aspera/cli/plugins/sync.rb +3 -3
- data/lib/aspera/cli/transfer_agent.rb +24 -10
- data/lib/aspera/cli/version.rb +2 -1
- data/lib/aspera/command_line_builder.rb +2 -1
- data/lib/aspera/cos_node.rb +1 -1
- data/lib/aspera/fasp/agent_connect.rb +1 -1
- data/lib/aspera/fasp/agent_direct.rb +12 -12
- data/lib/aspera/fasp/agent_node.rb +14 -4
- data/lib/aspera/fasp/installation.rb +1 -0
- data/lib/aspera/fasp/parameters.rb +11 -3
- data/lib/aspera/fasp/parameters.yaml +3 -1
- data/lib/aspera/fasp/transfer_spec.rb +3 -1
- data/lib/aspera/faspex_gw.rb +1 -0
- data/lib/aspera/faspex_postproc.rb +2 -2
- data/lib/aspera/node.rb +11 -4
- data/lib/aspera/oauth.rb +3 -2
- data/lib/aspera/preview/file_types.rb +8 -6
- data/lib/aspera/preview/generator.rb +23 -11
- data/lib/aspera/preview/options.rb +3 -2
- data/lib/aspera/preview/terminal.rb +34 -0
- data/lib/aspera/preview/utils.rb +8 -8
- data/lib/aspera/rest.rb +5 -4
- data/lib/aspera/rest_call_error.rb +3 -1
- data/lib/aspera/secret_hider.rb +4 -4
- data/lib/aspera/sync.rb +39 -33
- data/lib/aspera/web_server_simple.rb +22 -18
- data.tar.gz.sig +0 -0
- metadata +39 -46
- metadata.gz.sig +0 -0
- data/examples/aoc.rb +0 -30
- data/examples/faspex4.rb +0 -94
- data/examples/node.rb +0 -96
- data/examples/server.rb +0 -93
data/README.md
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
# Command Line Interface for IBM Aspera products
|
2
2
|
<!-- markdownlint-disable MD033 MD003 MD053 -->
|
3
|
-
<!-- cSpell:ignore devkit zcvf zxvf noded secondfile filesize sedemo eudemo webmail csum eascp loglevel cronfile magick keepalive inotify eastus bluemix trev sshfp struct genkey passout ibmaspera unpermitted -->
|
4
|
-
|
5
|
-
<!-- cSpell:ignoreRegExp /my[a-z]+/g -->
|
3
|
+
<!-- cSpell:ignore devkit zcvf zxvf noded secondfile filesize sedemo eudemo webmail csum eascp loglevel cronfile magick keepalive inotify eastus bluemix trev sshfp struct genkey passout ibmaspera unpermitted schtasks taskschd -->
|
4
|
+
|
6
5
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
7
6
|
|
8
7
|
##
|
9
8
|
|
10
|
-
Version : 4.
|
9
|
+
Version : 4.13.0
|
11
10
|
|
12
11
|
Laurent/2016-2023
|
13
12
|
|
@@ -19,9 +18,9 @@ Ruby Gem: [https://rubygems.org/gems/aspera-cli](https://rubygems.org/gems/asper
|
|
19
18
|
|
20
19
|
Ruby Doc: [https://www.rubydoc.info/gems/aspera-cli](https://www.rubydoc.info/gems/aspera-cli)
|
21
20
|
|
22
|
-
Minimum required Ruby version: >= 2.
|
21
|
+
Minimum required Ruby version: >= 2.6.
|
23
22
|
|
24
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
23
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
25
24
|
|
26
25
|
[Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs&sort=title_asc)
|
27
26
|
[Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
|
@@ -58,7 +57,7 @@ So it is designed for:
|
|
58
57
|
If the need is to perform operations programmatically in languages such as: C, Go, Python, nodejs, ... then it is better to directly use [Aspera APIs](https://ibm.biz/aspera_api)
|
59
58
|
|
60
59
|
- Product APIs (REST) : e.g. AoC, Faspex, node
|
61
|
-
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java,
|
60
|
+
- Transfer SDK : with gRPC interface and language stubs (C, C++, Python, .NET/C#, java, Ruby, etc...)
|
62
61
|
|
63
62
|
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
64
63
|
|
@@ -93,7 +92,7 @@ ascli --version
|
|
93
92
|
```
|
94
93
|
|
95
94
|
```bash
|
96
|
-
4.
|
95
|
+
4.13.0
|
97
96
|
```
|
98
97
|
|
99
98
|
### First use
|
@@ -104,8 +103,6 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
|
|
104
103
|
|
105
104
|
To test with Aspera demo transfer server, setup the environment and then test:
|
106
105
|
|
107
|
-
<!-- spellchecker: disable -->
|
108
|
-
|
109
106
|
```bash
|
110
107
|
ascli config initdemo
|
111
108
|
```
|
@@ -115,18 +112,16 @@ ascli server browse /
|
|
115
112
|
```
|
116
113
|
|
117
114
|
```output
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
115
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
116
|
+
| zmode | zuid | zgid | size | mtime | name |
|
117
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
118
|
+
| drwxr-xr-x | asperaweb | asperaweb | 90112 | 2023-04-05 15:31:21 +0200 | Upload |
|
119
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:16 +0200 | aspera-test-dir-large |
|
120
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-small |
|
121
|
+
| dr-xr-xr-x | asperaweb | asperaweb | 4096 | 2022-10-27 16:08:17 +0200 | aspera-test-dir-tiny |
|
122
|
+
+------------+-----------+-----------+-------+---------------------------+-----------------------+
|
126
123
|
```
|
127
124
|
|
128
|
-
<!-- spellchecker: enable -->
|
129
|
-
|
130
125
|
If you want to use `ascli` with another server, and in order to make further calls more convenient, it is advised to define a [option preset](#lprt) for the server's authentication options. The following example will:
|
131
126
|
|
132
127
|
- create a [option preset](#lprt)
|
@@ -155,17 +150,17 @@ ascli server browse /aspera-test-dir-large
|
|
155
150
|
```
|
156
151
|
|
157
152
|
```output
|
158
|
-
|
159
|
-
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
164
|
-
|
165
|
-
|
166
|
-
|
167
|
-
|
168
|
-
|
153
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
154
|
+
| zmode | zuid | zgid | size | mtime | name |
|
155
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
156
|
+
| -r-xr-x--- | asperaweb | asperaweb | 104857600 | 2022-10-27 16:06:38 +0200 | 100MB |
|
157
|
+
| -r-xr-x--- | asperaweb | asperaweb | 10737418240 | 2022-10-27 16:08:12 +0200 | 10GB |
|
158
|
+
| -r-xr-x--- | asperaweb | asperaweb | 500000000000 | 2022-10-27 16:06:26 +0200 | 500GB |
|
159
|
+
| -r-xr-x--- | asperaweb | asperaweb | 524288000 | 2022-10-27 14:53:00 +0200 | 500MB |
|
160
|
+
| -r-xr-x--- | asperaweb | asperaweb | 1048576000 | 2022-10-27 16:06:37 +0200 | 1GB |
|
161
|
+
| -r-xr-x--- | asperaweb | asperaweb | 5368709120 | 2022-10-27 14:53:47 +0200 | 5GB |
|
162
|
+
| -r-xr-x--- | asperaweb | asperaweb | 209715200 | 2022-10-27 14:52:56 +0200 | 200MB |
|
163
|
+
+------------+-----------+-----------+--------------+---------------------------+-------+
|
169
164
|
```
|
170
165
|
|
171
166
|
```bash
|
@@ -185,7 +180,7 @@ Then, follow the section relative to the product you want to interact with ( Asp
|
|
185
180
|
|
186
181
|
## <a id="installation"></a>Installation
|
187
182
|
|
188
|
-
It is possible to install
|
183
|
+
It is possible to install **either** directly on the host operating system (Linux, macOS, Windows) or as a container (`docker`).
|
189
184
|
|
190
185
|
The direct installation is recommended and consists in installing:
|
191
186
|
|
@@ -193,9 +188,9 @@ The direct installation is recommended and consists in installing:
|
|
193
188
|
- [aspera-cli](#the_gem)
|
194
189
|
- [Aspera SDK (`ascp`)](#fasp_prot)
|
195
190
|
|
196
|
-
Ruby version: >= 2.
|
191
|
+
Ruby version: >= 2.6.
|
197
192
|
|
198
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
193
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
199
194
|
|
200
195
|
The following sections provide information on the various installation methods.
|
201
196
|
|
@@ -205,16 +200,18 @@ An internet connection is required for the installation. If you don't have inter
|
|
205
200
|
|
206
201
|
The image is: [martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli).
|
207
202
|
The container contains: Ruby, `ascli` and the Aspera Transfer SDK.
|
208
|
-
To use the container, ensure that you have `
|
203
|
+
To use the container, ensure that you have `podman` (or `docker`) installed.
|
209
204
|
|
210
205
|
```bash
|
211
|
-
|
206
|
+
podman --version
|
212
207
|
```
|
213
208
|
|
209
|
+
#### Container quick start
|
210
|
+
|
214
211
|
**Wanna start quickly ?** With an interactive shell ? Execute this:
|
215
212
|
|
216
213
|
```bash
|
217
|
-
|
214
|
+
podman run --tty --interactive --entrypoint bash martinlaurent/ascli:latest
|
218
215
|
```
|
219
216
|
|
220
217
|
Then, execute individual `ascli` commands such as:
|
@@ -232,18 +229,22 @@ That is simple, but there are limitations:
|
|
232
229
|
- Any generated file in the container will be lost on container (shell) exit. Including configuration files and downloaded files.
|
233
230
|
- No possibility to upload files located on the host system
|
234
231
|
|
232
|
+
#### Details on the container
|
233
|
+
|
235
234
|
The container image is built from this [Dockerfile](Dockerfile): the entry point is `ascli` and the default command is `help`.
|
236
235
|
|
236
|
+
If you want to run the image with a shell, execute with option: `--entrypoint bash`, and give argument `-l` (bash login to override the `help` default argument)
|
237
|
+
|
237
238
|
The container can also be execute for individual commands like this: (add `ascli` commands and options at the end of the command line, e.g. `-v` to display the version)
|
238
239
|
|
239
240
|
```bash
|
240
|
-
|
241
|
+
podman run --rm --tty --interactive martinlaurent/ascli:latest
|
241
242
|
```
|
242
243
|
|
243
244
|
For more convenience, you may define a shell alias:
|
244
245
|
|
245
246
|
```bash
|
246
|
-
alias ascli='
|
247
|
+
alias ascli='podman run --rm --tty --interactive martinlaurent/ascli:latest'
|
247
248
|
```
|
248
249
|
|
249
250
|
Then, you can execute the container like a local command:
|
@@ -253,7 +254,7 @@ ascli -v
|
|
253
254
|
```
|
254
255
|
|
255
256
|
```text
|
256
|
-
4.
|
257
|
+
4.13.0
|
257
258
|
```
|
258
259
|
|
259
260
|
In order to keep persistency of configuration on the host,
|
@@ -265,7 +266,7 @@ Add options:
|
|
265
266
|
--user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli
|
266
267
|
```
|
267
268
|
|
268
|
-
> **Note:** if you are using a `podman machine`, e.g. on
|
269
|
+
> **Note:** if you are using a `podman machine`, e.g. on macOS , make sure that the folder is also shared between the VM and the host, so that sharing is: container → VM → Host: `podman machine init ... --volume="/Users:/Users"`
|
269
270
|
|
270
271
|
As shown in the quick start, if you prefer to keep a running container with a shell and `ascli` available,
|
271
272
|
you can change the entry point, add option:
|
@@ -286,7 +287,7 @@ In this case you need also to mount the shared transfer folder:
|
|
286
287
|
And if you want all the above, simply use all the options:
|
287
288
|
|
288
289
|
```bash
|
289
|
-
alias asclish="
|
290
|
+
alias asclish="podman run --rm --tty --interactive --user root --env ASCLI_HOME=/home/cliuser/.aspera/ascli --volume $HOME/.aspera/ascli:/home/cliuser/.aspera/ascli --volume $HOME/xferdir:/xferfiles --entrypoint bash martinlaurent/ascli:latest"
|
290
291
|
```
|
291
292
|
|
292
293
|
```bash
|
@@ -297,6 +298,8 @@ mkdir -p $HOME/.aspera/ascli
|
|
297
298
|
asclish
|
298
299
|
```
|
299
300
|
|
301
|
+
#### Sample container script
|
302
|
+
|
300
303
|
A convenience sample script is also provided: download the script [`dascli`](../examples/dascli) from [the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
|
301
304
|
|
302
305
|
> **Note:** If you have installed `ascli`, the script `dascli` can also be found: `cp $(ascli conf gem path)/../examples/dascli ascli`
|
@@ -306,7 +309,7 @@ Some environment variables can be set for this script to adapt its behavior:
|
|
306
309
|
| env var | description | default | example |
|
307
310
|
|--------------|------------------------------------|--------------------------|--------------------------|
|
308
311
|
| ASCLI_HOME | configuration folder (persistency) | `$HOME/.aspera/ascli` | `$HOME/.ascliconfig` |
|
309
|
-
| docker_args | additional options to `
|
312
|
+
| docker_args | additional options to `podman` | <empty> | `--volume /Users:/Users` |
|
310
313
|
| image | container image name | martinlaurent/ascli | |
|
311
314
|
| version | container image version | latest | `4.8.0.pre` |
|
312
315
|
|
@@ -333,22 +336,44 @@ echo 'Local file to transfer' > $xferdir/samplefile.txt
|
|
333
336
|
```
|
334
337
|
|
335
338
|
> **Note:** The local file (`samplefile.txt`) is specified relative to storage view from container (`/xferfiles`) mapped to the host folder `$HOME/xferdir`
|
339
|
+
>
|
340
|
+
> **Note:** Do not use too many volumes, as the AUFS limits the number.
|
341
|
+
|
342
|
+
#### Offline installation of the container
|
343
|
+
|
344
|
+
- First create the image archive:
|
345
|
+
|
346
|
+
```bash
|
347
|
+
podman pull martinlaurent/ascli
|
348
|
+
podman save martinlaurent/ascli|gzip>ascli_image_latest.tar.gz
|
349
|
+
```
|
350
|
+
|
351
|
+
- Then, on air-gapped system:
|
352
|
+
|
353
|
+
```bash
|
354
|
+
podman load -i ascli_image_latest.tar.gz
|
355
|
+
```
|
336
356
|
|
337
357
|
### <a id="ruby"></a>Ruby
|
338
358
|
|
339
359
|
Use this method to install on the native host.
|
340
360
|
|
341
|
-
A
|
361
|
+
A Ruby interpreter is required to run the tool or to use the gem and tool.
|
342
362
|
|
343
|
-
Required Ruby version: >= 2.
|
363
|
+
Required Ruby version: >= 2.6.
|
344
364
|
|
345
|
-
> **Deprecation notice**: the minimum Ruby version will be
|
365
|
+
> **Deprecation notice**: the minimum Ruby version will be 3.0 in a future version.
|
346
366
|
|
347
367
|
*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
348
368
|
|
349
|
-
|
369
|
+
In priority, refer to the official Ruby documentation:
|
350
370
|
|
351
|
-
|
371
|
+
- [Download Ruby](https://www.ruby-lang.org/en/downloads/)
|
372
|
+
- [Installation Guide](https://www.ruby-lang.org/en/documentation/installation/)
|
373
|
+
|
374
|
+
Else, refer to the following sections for a proposed method for specific operating systems.
|
375
|
+
|
376
|
+
The recommended installation method is `rvm` for Unix-like systems (Linux, AIX, macOS, Windows with cygwin, etc...).
|
352
377
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
353
378
|
If you have a simpler better way to install Ruby : use it !
|
354
379
|
|
@@ -356,7 +381,7 @@ If you have a simpler better way to install Ruby : use it !
|
|
356
381
|
|
357
382
|
Use this method which provides more flexibility.
|
358
383
|
|
359
|
-
Install
|
384
|
+
Install `rvm`: follow [https://rvm.io/](https://rvm.io/) :
|
360
385
|
|
361
386
|
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
362
387
|
|
@@ -372,7 +397,7 @@ If you keep the same terminal (not needed if re-login):
|
|
372
397
|
source ~/.rvm/scripts/rvm
|
373
398
|
```
|
374
399
|
|
375
|
-
It is advised to get one of the pre-compiled
|
400
|
+
It is advised to get one of the pre-compiled Ruby version, you can list with:
|
376
401
|
|
377
402
|
```bash
|
378
403
|
rvm list --remote
|
@@ -381,10 +406,10 @@ rvm list --remote
|
|
381
406
|
Install the chosen pre-compiled Ruby version:
|
382
407
|
|
383
408
|
```bash
|
384
|
-
rvm install 2.
|
409
|
+
rvm install 3.2.2
|
385
410
|
```
|
386
411
|
|
387
|
-
Ruby is now installed for the user, go
|
412
|
+
Ruby is now installed for the user, go to [Gem installation](#the_gem).
|
388
413
|
|
389
414
|
#### Generic: RVM: global installation (as root)
|
390
415
|
|
@@ -399,7 +424,7 @@ curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
|
399
424
|
|
400
425
|
As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
|
401
426
|
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
402
|
-
To activate
|
427
|
+
To activate Ruby (and ascli) later, source it:
|
403
428
|
|
404
429
|
```bash
|
405
430
|
source /etc/profile.d/rvm.sh.ok
|
@@ -434,7 +459,7 @@ brew install ruby
|
|
434
459
|
|
435
460
|
#### Linux: package
|
436
461
|
|
437
|
-
If your Linux distribution provides a standard
|
462
|
+
If your Linux distribution provides a standard Ruby package, you can use it provided that the version is compatible (check at beginning of section).
|
438
463
|
|
439
464
|
Example: RHEL 8 and 9: basic installation
|
440
465
|
|
@@ -462,7 +487,7 @@ yum install -y ruby ruby-devel rubygems ruby-json
|
|
462
487
|
apt install -y ruby ruby-dev rubygems ruby-json
|
463
488
|
```
|
464
489
|
|
465
|
-
One can cleanup the whole yum-installed
|
490
|
+
One can cleanup the whole yum-installed Ruby environment like this to uninstall:
|
466
491
|
|
467
492
|
```bash
|
468
493
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
@@ -475,7 +500,7 @@ For example for AIX, one can look at:
|
|
475
500
|
|
476
501
|
<https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha#R>
|
477
502
|
|
478
|
-
If your Unix does not provide a pre-built
|
503
|
+
If your Unix does not provide a pre-built Ruby, you can get it using one of those
|
479
504
|
[methods](https://www.ruby-lang.org/en/documentation/installation/).
|
480
505
|
|
481
506
|
For instance to build from source, and install in `/opt/ruby` :
|
@@ -502,7 +527,7 @@ If you already have a Java JVM on your system (`java`), it is possible to use `j
|
|
502
527
|
|
503
528
|
<https://www.jruby.org/download>
|
504
529
|
|
505
|
-
> **Note:** Using jruby the startup time is longer than the native
|
530
|
+
> **Note:** Using jruby the startup time is longer than the native Ruby, but the transfer speed is not impacted (executed by `ascp` binary).
|
506
531
|
|
507
532
|
### <a id="the_gem"></a>`aspera-cli` gem
|
508
533
|
|
@@ -556,7 +581,6 @@ If the embedded method is not used, the following packages are also suitable:
|
|
556
581
|
|
557
582
|
- IBM Aspera Connect Client (Free)
|
558
583
|
- IBM Aspera Desktop Client (Free)
|
559
|
-
- IBM Aspera CLI (Free)
|
560
584
|
- IBM Aspera High Speed Transfer Server (Licensed)
|
561
585
|
- IBM Aspera High Speed Transfer EndPoint (Licensed)
|
562
586
|
|
@@ -670,9 +694,9 @@ Moreover all `ascp` options are supported either through transfer spec parameter
|
|
670
694
|
`ascli` is typically executed in a shell, either interactively or in a script.
|
671
695
|
`ascli` receives its arguments from this shell (through Operating System).
|
672
696
|
|
673
|
-
#### Shell parsing for Linux,
|
697
|
+
#### Shell parsing for Unix-like systems: Linux, macOS, AIX
|
674
698
|
|
675
|
-
On
|
699
|
+
On Unix-like environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
|
676
700
|
In this environment the shell parses the command line, possibly replacing variables, etc...
|
677
701
|
see [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
|
678
702
|
Then it builds a list of arguments and then `ascli` (Ruby) is executed.
|
@@ -690,7 +714,7 @@ It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ para
|
|
690
714
|
|
691
715
|
#### Extended Values (JSON, Ruby, ...)
|
692
716
|
|
693
|
-
Some of the
|
717
|
+
Some of the `ascli` parameters are expected to be [Extended Values](#extended), i.e. not a simple strings, but a complex structure (Hash, Array).
|
694
718
|
Typically, the `@json:` modifier is used, it expects a JSON string. JSON itself has some special syntax: for example `"` is used to denote strings.
|
695
719
|
|
696
720
|
#### Testing Extended Values
|
@@ -841,7 +865,7 @@ ascli conf echo @env:MYTITLE --format=text
|
|
841
865
|
Test " ' & \
|
842
866
|
```
|
843
867
|
|
844
|
-
If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any
|
868
|
+
If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any Ruby code in expression, including reading from file or env var. In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.
|
845
869
|
|
846
870
|
```bash
|
847
871
|
ascli conf echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
|
@@ -972,12 +996,12 @@ ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sor
|
|
972
996
|
```
|
973
997
|
|
974
998
|
```output
|
975
|
-
|
976
|
-
|
977
|
-
|
978
|
-
|
979
|
-
|
980
|
-
|
999
|
+
+-------------------------------+----------------------------------+-----------+
|
1000
|
+
| name | email | ats_admin |
|
1001
|
+
+-------------------------------+----------------------------------+-----------+
|
1002
|
+
| John Curtis | john@example.com | true |
|
1003
|
+
| Laurent Martin | laurent@example.com | true |
|
1004
|
+
+-------------------------------+----------------------------------+-----------+
|
981
1005
|
```
|
982
1006
|
|
983
1007
|
> **Note:** `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
|
@@ -1012,38 +1036,38 @@ By default, a table output will display one line per entry, and columns for each
|
|
1012
1036
|
|
1013
1037
|
### <a id="extended"></a>Extended Value Syntax
|
1014
1038
|
|
1015
|
-
|
1039
|
+
Some options and arguments are specified by a simple string.
|
1040
|
+
But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
|
1016
1041
|
|
1017
1042
|
The extended value syntax is:
|
1018
1043
|
|
1019
1044
|
```bash
|
1020
|
-
<0 or more decoders><
|
1045
|
+
<0 or more decoders><nothing or some text value>
|
1021
1046
|
```
|
1022
1047
|
|
1023
|
-
|
1024
|
-
|
1025
|
-
Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
|
1026
|
-
|
1027
|
-
The following "readers" are supported (returns value in []):
|
1048
|
+
Decoders act like a function of value on right hand side.
|
1049
|
+
Decoders are recognized by the prefix: `@` and suffix `:`
|
1028
1050
|
|
1029
|
-
|
1030
|
-
- @file:PATH : [String] read value from a URL, e.g. `--fpac=@uri:http://serv/f.pac`
|
1031
|
-
- @uri:URL : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
1032
|
-
- @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
1033
|
-
- @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
1034
|
-
- @stdin: : [String] read from stdin (no value on right)
|
1035
|
-
- @preset:NAME : [Hash] get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
|
1051
|
+
The following decoders are supported:
|
1036
1052
|
|
1037
|
-
|
1038
|
-
|
1039
|
-
|
1040
|
-
|
1041
|
-
|
1042
|
-
|
1043
|
-
|
1044
|
-
|
1045
|
-
|
1046
|
-
|
1053
|
+
| decoder | parameter | returns | description |
|
1054
|
+
|---------|-----------|---------|-------------|
|
1055
|
+
| base64 | String | String | decode a base64 encoded string
|
1056
|
+
| csvt | String | Array | decode a titled CSV value
|
1057
|
+
| env | String | String | read from a named env var name, e.g.--password=@env:MYPASSVAR
|
1058
|
+
| file | String | String | read value from specified file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey` |
|
1059
|
+
| incps | Hash | Hash | include values of presets specified by key `incps` in input hash
|
1060
|
+
| json | String | any | decode JSON values (convenient to provide complex structures)
|
1061
|
+
| lines | String | Array | split a string in multiple lines and return an array
|
1062
|
+
| list | String | Array | split a string in multiple items taking first character as separator and return an array
|
1063
|
+
| path | String | String | performs path expansion on specified path (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml` |
|
1064
|
+
| preset | String | Hash | get whole option preset value by name. Sub-values can also be used using `.` as separator. e.g. `foo.bar` is `conf[foo][bar]`
|
1065
|
+
| ruby | String | any | execute specified Ruby code
|
1066
|
+
| secret | None | String | Ask password interactively (hides input)
|
1067
|
+
| stdin | None | String | read from stdin (no value on right)
|
1068
|
+
| uri | String | String | read value from specified URL, e.g. `--fpac=@uri:http://serv/f.pac` |
|
1069
|
+
| val | String | String | prevent decoders on the right to be decoded. e.g. `--key=@val:@file:foo` sets the option `key` to value `@file:foo`. |
|
1070
|
+
| zlib | String | String | un-compress data
|
1047
1071
|
|
1048
1072
|
To display the result of an extended value, use the `config echo` command.
|
1049
1073
|
|
@@ -1076,12 +1100,12 @@ ascli config echo @csvt:@file:test.csv
|
|
1076
1100
|
```
|
1077
1101
|
|
1078
1102
|
```output
|
1079
|
-
|
1080
|
-
|
1081
|
-
|
1082
|
-
|
1083
|
-
|
1084
|
-
|
1103
|
+
+------+---------------------+
|
1104
|
+
| name | email |
|
1105
|
+
+------+---------------------+
|
1106
|
+
| lolo | laurent@example.com |
|
1107
|
+
| toto | titi@tutu.tata |
|
1108
|
+
+------+---------------------+
|
1085
1109
|
```
|
1086
1110
|
|
1087
1111
|
Example: create a hash and include values from preset named "config" of config file in this hash
|
@@ -1111,7 +1135,7 @@ It is also possible to provide a [Extended Value](#extended) in a file using `@j
|
|
1111
1135
|
|
1112
1136
|
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored `[config folder]`: `[User's home folder]/.aspera/ascli`.
|
1113
1137
|
|
1114
|
-
Note: `[User's home folder]` is found using
|
1138
|
+
Note: `[User's home folder]` is found using Ruby's `Dir.home` (`rb_w32_home_dir`).
|
1115
1139
|
It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMEDRIVE%%HOMEPATH%` and `%USERPROFILE%`. `ascli` sets the env var `%HOME%` to the value of `%USERPROFILE%` if set and exists. So, on Windows `%USERPROFILE%` is used as it is more reliable than `%HOMEDRIVE%%HOMEPATH%`.
|
1116
1140
|
|
1117
1141
|
The [config folder] can be displayed using :
|
@@ -1242,7 +1266,7 @@ ascli config preset get default _plugin_name_
|
|
1242
1266
|
"_default_preset_for_plugin_"
|
1243
1267
|
```
|
1244
1268
|
|
1245
|
-
#### <a id="config"></a>Plugin: `config`:
|
1269
|
+
#### <a id="config"></a>Plugin: `config`: Configuration
|
1246
1270
|
|
1247
1271
|
Plugin `config` is used to configure `ascli` and also contains global options.
|
1248
1272
|
|
@@ -1265,9 +1289,11 @@ config ascp products list
|
|
1265
1289
|
config ascp show
|
1266
1290
|
config ascp spec
|
1267
1291
|
config check_update
|
1292
|
+
config coffee
|
1293
|
+
config coffee --ui=text
|
1294
|
+
config detect --url=https://faspex4.example.com/path
|
1268
1295
|
config detect --url=https://my_aoc_org.ibmaspera.com
|
1269
|
-
config detect --url=
|
1270
|
-
config detect --url=my_node_url
|
1296
|
+
config detect --url=https://node_simple.example.com/path
|
1271
1297
|
config doc
|
1272
1298
|
config doc transfer-parameters
|
1273
1299
|
config echo 'hello'
|
@@ -1280,7 +1306,7 @@ config echo @uri:/etc/hosts
|
|
1280
1306
|
config echo @uri:file:/etc/hosts
|
1281
1307
|
config echo @uri:http://www.ibm.com
|
1282
1308
|
config echo @uri:https://www.ibm.com
|
1283
|
-
config echo @val:@file:
|
1309
|
+
config echo @val:@file:no_such_file
|
1284
1310
|
config echo @zlib:@stdin:
|
1285
1311
|
config email_test --notif-to=my_recipient_email
|
1286
1312
|
config export
|
@@ -1313,15 +1339,15 @@ demo_server:
|
|
1313
1339
|
|
1314
1340
|
We can see here:
|
1315
1341
|
|
1316
|
-
- The configuration was created with
|
1342
|
+
- The configuration was created with `ascli` version 0.3.7
|
1317
1343
|
- the default [option preset](#lprt) to load for `server` plugin is : `demo_server`
|
1318
1344
|
- the [option preset](#lprt) `demo_server` defines some parameters: the URL and credentials
|
1319
1345
|
- the default [option preset](#lprt) to load in any case is : `cli_default`
|
1320
1346
|
|
1321
1347
|
Two [option presets](#lprt) are reserved:
|
1322
1348
|
|
1323
|
-
- `config` contains a single value: `version` showing the
|
1324
|
-
|
1349
|
+
- `config` contains a single value: `version` showing the version used to create the configuration file.
|
1350
|
+
It is used to check compatibility.
|
1325
1351
|
- `default` is reserved to define the default [option preset](#lprt) name used for known plugins.
|
1326
1352
|
|
1327
1353
|
The user may create as many [option presets](#lprt) as needed. For instance, a particular [option preset](#lprt) can be created for a particular application instance and contain URL and credentials.
|
@@ -1530,13 +1556,13 @@ Some applications allow the user to be authenticated using a private key (Server
|
|
1530
1556
|
It consists in using a pair of keys: the private key and its associated public key.
|
1531
1557
|
The same key can be used for multiple applications.
|
1532
1558
|
Technically, a private key contains the public key, which can be extracted from it.
|
1533
|
-
The private key can be protected by a passphrase
|
1534
|
-
If the key is protected by a passphrase, then it will be prompted.
|
1559
|
+
The file containing the private key can optionally be protected by a passphrase.
|
1560
|
+
If the key is protected by a passphrase, then it will be prompted when used.
|
1535
1561
|
(some plugins support option `passphrase`)
|
1536
1562
|
|
1537
1563
|
The following commands use the shell variable `PRIVKEYFILE`.
|
1538
1564
|
Set it to the desired safe location of the private key.
|
1539
|
-
Typically, in `$HOME/.ssh` or `$HOME/.aspera/ascli`:
|
1565
|
+
Typically, located in folder `$HOME/.ssh` or `$HOME/.aspera/ascli`:
|
1540
1566
|
|
1541
1567
|
```bash
|
1542
1568
|
PRIVKEYFILE=~/.aspera/ascli/my_private_key
|
@@ -1544,6 +1570,8 @@ PRIVKEYFILE=~/.aspera/ascli/my_private_key
|
|
1544
1570
|
|
1545
1571
|
Several methods can be used to generate a key pair.
|
1546
1572
|
|
1573
|
+
The format expected for private keys is [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
|
1574
|
+
|
1547
1575
|
#### `ascli` for key generation
|
1548
1576
|
|
1549
1577
|
The generated key is of type RSA, by default: 4096 bit.
|
@@ -1565,45 +1593,51 @@ ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${PRIVKEYFILE}
|
|
1565
1593
|
#### `openssl`
|
1566
1594
|
|
1567
1595
|
To generate a private key pair with a passphrase the following can be used on any system:
|
1568
|
-
|
1596
|
+
|
1569
1597
|
```bash
|
1570
|
-
openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE}
|
1598
|
+
openssl genrsa -passout pass:_passphrase_here_ -out ${PRIVKEYFILE} 4096
|
1571
1599
|
openssl rsa -pubout -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.pub
|
1572
1600
|
```
|
1573
|
-
<!-- spellchecker: enable -->
|
1574
1601
|
|
1575
1602
|
`openssl` is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
|
1576
1603
|
In that case, add option `-nodes` instead of `-passout pass:_passphrase_here_` to generate a key without passphrase.
|
1577
1604
|
|
1578
1605
|
If option `-nodes` is not available, the passphrase can be removed using this method:
|
1579
1606
|
|
1580
|
-
<!-- spellchecker: disable -->
|
1581
1607
|
```bash
|
1582
|
-
openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE}
|
1583
|
-
|
1608
|
+
openssl rsa -passin pass:_passphrase_here_ -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.no_des
|
1609
|
+
mv ${PRIVKEYFILE}.no_des ${PRIVKEYFILE}
|
1584
1610
|
```
|
1585
|
-
<!-- spellchecker: enable -->
|
1586
1611
|
|
1587
1612
|
To change (or add) the passphrase for a key do:
|
1588
1613
|
|
1589
1614
|
```bash
|
1590
|
-
openssl rsa -des3 -in
|
1615
|
+
openssl rsa -des3 -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.with_des
|
1616
|
+
mv ${PRIVKEYFILE}.with_des ${PRIVKEYFILE}
|
1591
1617
|
```
|
1592
1618
|
|
1593
1619
|
### <a id="certificates"></a>SSL CA certificate bundle
|
1594
1620
|
|
1595
|
-
`ascli` uses
|
1596
|
-
Certificates are checked against the
|
1597
|
-
The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
|
1621
|
+
`ascli` uses the Ruby `openssl` gem, which uses the `openssl` library.
|
1622
|
+
Certificates are checked against the [Ruby default certificate store](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html) `OpenSSL::X509::DEFAULT_CERT_FILE` and `OpenSSL::X509::DEFAULT_CERT_DIR`, which are typically the ones of `openssl` on Unix-like systems (Linux, macOS, etc..).
|
1598
1623
|
|
1599
|
-
|
1600
|
-
|
1601
|
-
|
1602
|
-
|
1624
|
+
To display the current root certificate store locations:
|
1625
|
+
|
1626
|
+
```bash
|
1627
|
+
ascli conf echo @ruby:'[OpenSSL::X509::DEFAULT_CERT_FILE,OpenSSL::X509::DEFAULT_CERT_DIR]'
|
1628
|
+
```
|
1629
|
+
|
1630
|
+
Ruby's default values can be overridden by env vars: `SSL_CERT_FILE` and `SSL_CERT_DIR`.
|
1631
|
+
|
1632
|
+
`ascp` also needs to validate certificates when using **WSS**.
|
1633
|
+
By default, `ascp` uses primarily certificates from hard-coded path (e.g. on macOS: `/Library/Aspera/ssl`) for WSS.
|
1634
|
+
`ascli` overrides and sets the default Ruby certificate path as well for `ascp` using `-i` switch.
|
1635
|
+
|
1636
|
+
To update `ascli` trusted root certificates, just update your system's root certificates or use env vars specified here above.
|
1603
1637
|
|
1604
1638
|
### Plugins
|
1605
1639
|
|
1606
|
-
|
1640
|
+
`ascli` uses a plugin mechanism.
|
1607
1641
|
The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
|
1608
1642
|
Each plugin usually represents commands sent to a specific application.
|
1609
1643
|
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
@@ -1697,7 +1731,7 @@ When `ascli` is used interactively in a shell, the shell itself will usually log
|
|
1697
1731
|
|
1698
1732
|
### Learning Aspera Product APIs (REST)
|
1699
1733
|
|
1700
|
-
|
1734
|
+
`ascli` uses mainly Aspera applications REST APIs.
|
1701
1735
|
To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
|
1702
1736
|
|
1703
1737
|
In order to get traces of execution, use argument : `--log-level=debug`
|
@@ -1706,7 +1740,7 @@ In order to get traces of execution, use argument : `--log-level=debug`
|
|
1706
1740
|
|
1707
1741
|
If the server does not provide a valid certificate, use option: `--insecure=yes`.
|
1708
1742
|
|
1709
|
-
|
1743
|
+
HTTP socket parameters can be adjusted using option `http_options`:
|
1710
1744
|
|
1711
1745
|
| parameter | default |
|
1712
1746
|
|----------------------|---------|
|
@@ -1716,8 +1750,8 @@ Ruby HTTP socket parameters can be adjusted.
|
|
1716
1750
|
| `keep_alive_timeout` | 2 |
|
1717
1751
|
|
1718
1752
|
Values are in set *seconds* and can be of type either integer or float.
|
1719
|
-
Default values are the ones of Ruby
|
1720
|
-
|
1753
|
+
Default values are the ones of Ruby:
|
1754
|
+
refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1721
1755
|
|
1722
1756
|
Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
|
1723
1757
|
|
@@ -1734,7 +1768,7 @@ Some actions may require the use of a graphical tool:
|
|
1734
1768
|
- a browser for Aspera on Cloud authentication (web auth method)
|
1735
1769
|
- a text editor for configuration file edition
|
1736
1770
|
|
1737
|
-
By default
|
1771
|
+
By default `ascli` assumes that a graphical environment is available on windows, and on other systems, rely on the presence of the `DISPLAY` environment variable.
|
1738
1772
|
It is also possible to force the graphical mode with option --ui :
|
1739
1773
|
|
1740
1774
|
- `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
@@ -1870,7 +1904,7 @@ Saved to default global preset global_common_defaults
|
|
1870
1904
|
|
1871
1905
|
Windows:
|
1872
1906
|
|
1873
|
-
```
|
1907
|
+
```text
|
1874
1908
|
ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1875
1909
|
```
|
1876
1910
|
|
@@ -1891,14 +1925,14 @@ ascli config ascp products list
|
|
1891
1925
|
```
|
1892
1926
|
|
1893
1927
|
```output
|
1894
|
-
|
1895
|
-
|
1896
|
-
|
1897
|
-
|
1898
|
-
|
1899
|
-
|
1900
|
-
|
1901
|
-
|
1928
|
+
+---------------------------------------+----------------------------------------+
|
1929
|
+
| name | app_root |
|
1930
|
+
+---------------------------------------+----------------------------------------+
|
1931
|
+
| IBM Aspera SDK | /Users/laurent/.aspera/ascli/sdk |
|
1932
|
+
| Aspera Connect | /Applications/Aspera Connect.app |
|
1933
|
+
| IBM Aspera CLI | /Users/laurent/Applications/Aspera CLI |
|
1934
|
+
| IBM Aspera High-Speed Transfer Server | /Library/Aspera |
|
1935
|
+
+---------------------------------------+----------------------------------------+
|
1902
1936
|
```
|
1903
1937
|
|
1904
1938
|
#### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
|
@@ -1964,7 +1998,7 @@ Some of the actions on Aspera Applications lead to file transfers (upload and do
|
|
1964
1998
|
When a transfer needs to be started, a [*transfer-spec*](#transferspec) has been internally prepared.
|
1965
1999
|
This [*transfer-spec*](#transferspec) will be executed by a transfer client, here called **Transfer Agent**.
|
1966
2000
|
|
1967
|
-
There are currently 3 agents
|
2001
|
+
There are currently 3 agents, set with option `transfer`:
|
1968
2002
|
|
1969
2003
|
- [`direct`](#agt_direct) : a local execution of `ascp`
|
1970
2004
|
- [`connect`](#agt_connect) : use of a local Connect Client
|
@@ -1978,6 +2012,8 @@ will effectively push files to the related server from the agent node.
|
|
1978
2012
|
|
1979
2013
|
`ascli` standardizes on the use of a [*transfer-spec*](#transferspec) instead of *native* `ascp` options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1980
2014
|
|
2015
|
+
Specific options for agents are provided with option `transfer_info`, cumulatively.
|
2016
|
+
|
1981
2017
|
#### <a id="agt_direct"></a>Direct
|
1982
2018
|
|
1983
2019
|
The `direct` agent directly executes a local `ascp`.
|
@@ -2052,22 +2088,82 @@ In addition to standard methods described in section [File List](#file_list), it
|
|
2052
2088
|
>
|
2053
2089
|
> **Note:** Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
|
2054
2090
|
|
2091
|
+
This agent supports a local configuration file: `aspera.conf` where Virtual links can be configured:
|
2092
|
+
|
2093
|
+
On a server (HSTS), the following commands can be used to set a global virtual link:
|
2094
|
+
|
2095
|
+
```bash
|
2096
|
+
asconfigurator -x 'set_trunk_data;id,1;trunk_name,in;trunk_capacity,45000;trunk_on,true'
|
2097
|
+
asconfigurator -x 'set_trunk_data;id,2;trunk_name,out;trunk_capacity,45000;trunk_on,true'
|
2098
|
+
asconfigurator -x 'set_node_data;transfer_in_bandwidth_aggregate_trunk_id,1'
|
2099
|
+
asconfigurator -x 'set_node_data;transfer_out_bandwidth_aggregate_trunk_id,2'
|
2100
|
+
```
|
2101
|
+
|
2102
|
+
But this command is not available on clients, so edit the file `aspera.conf`, you can find the location with: `ascli conf ascp info --fields=aspera_conf` and modify the sections `default` and `trunks` like this for a global 100 Mbps virtual link:
|
2103
|
+
|
2104
|
+
```xml
|
2105
|
+
<?xml version='1.0' encoding='UTF-8'?>
|
2106
|
+
<CONF version="2">
|
2107
|
+
<default>
|
2108
|
+
<transfer>
|
2109
|
+
<in>
|
2110
|
+
<bandwidth>
|
2111
|
+
<aggregate>
|
2112
|
+
<trunk_id>1</trunk_id>
|
2113
|
+
</aggregate>
|
2114
|
+
</bandwidth>
|
2115
|
+
</in>
|
2116
|
+
<out>
|
2117
|
+
<bandwidth>
|
2118
|
+
<aggregate>
|
2119
|
+
<trunk_id>2</trunk_id>
|
2120
|
+
</aggregate>
|
2121
|
+
</bandwidth>
|
2122
|
+
</out>
|
2123
|
+
</transfer>
|
2124
|
+
</default>
|
2125
|
+
<trunks>
|
2126
|
+
<trunk>
|
2127
|
+
<id>1</id>
|
2128
|
+
<name>in</name>
|
2129
|
+
<on>true</on>
|
2130
|
+
<capacity>
|
2131
|
+
<schedule format="ranges">1000000</schedule>
|
2132
|
+
</capacity>
|
2133
|
+
</trunk>
|
2134
|
+
<trunk>
|
2135
|
+
<id>2</id>
|
2136
|
+
<name>out</name>
|
2137
|
+
<capacity>
|
2138
|
+
<schedule format="ranges">1000000</schedule>
|
2139
|
+
</capacity>
|
2140
|
+
<on>true</on>
|
2141
|
+
</trunk>
|
2142
|
+
</trunks>
|
2143
|
+
</CONF>
|
2144
|
+
```
|
2145
|
+
|
2146
|
+
It is also possible to set a schedule with different time and days, for example for the value of `schedule`:
|
2147
|
+
|
2148
|
+
```text
|
2149
|
+
start=08 end=19 days=mon,tue,wed,thu capacity=900000;1000000
|
2150
|
+
```
|
2151
|
+
|
2055
2152
|
#### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
|
2056
2153
|
|
2057
2154
|
By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
|
2058
2155
|
|
2059
2156
|
#### <a id="agt_node"></a>Aspera Node API : Node to node transfers
|
2060
2157
|
|
2061
|
-
By specifying option: `--transfer=node`,
|
2062
|
-
Transfer Server using the Node API, either on a local or remote node.
|
2158
|
+
By specifying option: `--transfer=node`, `ascli` starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
|
2063
2159
|
Parameters provided in option `transfer_info` are:
|
2064
2160
|
|
2065
|
-
| Name
|
2066
|
-
|
2067
|
-
| url
|
2161
|
+
| Name | Type | Description |
|
2162
|
+
|----------|--------|-------------|
|
2163
|
+
| url | string | URL of the node API</br>Mandatory |
|
2068
2164
|
| username | string | node api user or access key</br>Mandatory |
|
2069
2165
|
| password | string | password, secret or bearer token</br>Mandatory |
|
2070
|
-
| root_id
|
2166
|
+
| root_id | string | password or secret</br>Mandatory only for bearer token |
|
2071
2167
|
|
2072
2168
|
Like any other option, `transfer_info` can get its value from a pre-configured [option preset](#lprt) :
|
2073
2169
|
`--transfer-info=@preset:_name_here_` or be specified using the extended value syntax :
|
@@ -2120,8 +2216,8 @@ On Windows the compilation may fail for various reasons (3.1.1):
|
|
2120
2216
|
|
2121
2217
|
### <a id="transferspec"></a>Transfer Specification
|
2122
2218
|
|
2123
|
-
Some commands lead to file transfer (upload/download)
|
2124
|
-
|
2219
|
+
Some commands lead to file transfer (upload/download).
|
2220
|
+
All parameters necessary for this transfer are described in a [*transfer-spec*](#transferspec) (Transfer Specification), such as:
|
2125
2221
|
|
2126
2222
|
- server address
|
2127
2223
|
- transfer user name
|
@@ -2129,9 +2225,13 @@ is described in a [*transfer-spec*](#transferspec) (Transfer Specification), suc
|
|
2129
2225
|
- file list
|
2130
2226
|
- etc...
|
2131
2227
|
|
2132
|
-
`ascli` builds
|
2228
|
+
`ascli` builds the [*transfer-spec*](#transferspec) internally, so it is not necessary to provide additional parameters on the command line for this transfer.
|
2229
|
+
|
2230
|
+
The [*transfer-spec*](#transferspec) is a Hash (dictionary), so it is described on the command line with the [Extended Value Syntax](#extended).
|
2133
2231
|
|
2134
|
-
|
2232
|
+
It is possible to modify or add any of the supported [*transfer-spec*](#transferspec) parameter using the `ts` option.
|
2233
|
+
The `ts` option accepts a [Structured Value](#native) containing one or several [*transfer-spec*](#transferspec) parameters in a `Hash`.
|
2234
|
+
Multiple `ts` options on command line are cumulative.
|
2135
2235
|
|
2136
2236
|
It is possible to specify `ascp` options when the `transfer` option is set to [`direct`](#agt_direct) using `transfer_info` option parameter: `ascp_args`.
|
2137
2237
|
Example: `--transfer-info=@json:'{"ascp_args":["-l","100m"]}'`.
|
@@ -2142,8 +2242,6 @@ The use of a [*transfer-spec*](#transferspec) instead of `ascp` parameters has t
|
|
2142
2242
|
- common to all [Transfer Agent](#agents)
|
2143
2243
|
- not dependent on command line limitations (special characters...)
|
2144
2244
|
|
2145
|
-
A [*transfer-spec*](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
|
2146
|
-
|
2147
2245
|
### <a id="transferparams"></a>Transfer Parameters
|
2148
2246
|
|
2149
2247
|
All standard [*transfer-spec*](#transferspec) parameters can be specified.
|
@@ -2173,18 +2271,7 @@ Columns:
|
|
2173
2271
|
Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
|
2174
2272
|
|
2175
2273
|
| Field | Type | D | N | C | Description |
|
2176
|
-
|
2177
|
-
| EX_ascp_args | array | Y | | | Add native command line arguments to ascp |
|
2178
|
-
| EX_at_rest_password | string | Y | | | DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2179
|
-
| EX_file_list | string | Y | | | source file list |
|
2180
|
-
| EX_file_pair_list | string | Y | | | source file pair list |
|
2181
|
-
| EX_http_proxy_url | string | Y | | | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2182
|
-
| EX_http_transfer_jpeg | int | Y | | | HTTP transfers as JPEG file<br/>(-j {int}) |
|
2183
|
-
| EX_license_text | string | Y | | | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2184
|
-
| EX_no_read | bool | Y | | | no read source<br/>(--no-read) |
|
2185
|
-
| EX_no_write | bool | Y | | | no write on destination<br/>(--no-write) |
|
2186
|
-
| EX_proxy_password | string | Y | | | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS) |
|
2187
|
-
| EX_ssh_key_paths | array | Y | | | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2274
|
+
| ----- | ---- | - | - | - | ----------- |
|
2188
2275
|
| apply_local_docroot | bool | Y | | | (--apply-local-docroot) |
|
2189
2276
|
| authentication | string | | | Y | value=token for SSH bypass keys, else password asked if not provided. |
|
2190
2277
|
| cipher | string | Y | Y | Y | In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c (conversion){enum}) |
|
@@ -2244,7 +2331,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2244
2331
|
| source_root_id | string | | Y | | The file ID of the source root directory. Required when using Bearer token auth for the source node. |
|
2245
2332
|
| src_base | string | Y | Y | | Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64 (conversion){string}) |
|
2246
2333
|
| ssh_port | int | Y | Y | Y | Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P {int}) |
|
2247
|
-
| ssh_private_key | string | Y | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY
|
2334
|
+
| ssh_private_key | string | Y | | | Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY) |
|
2248
2335
|
| ssh_private_key_passphrase | string | Y | | | The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS) |
|
2249
2336
|
| sshfp | string | Y | Y | Y | Check it against server SSH host key fingerprint<br/>(--check-sshfp {string}) |
|
2250
2337
|
| symlink_policy | string | Y | Y | Y | Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links {enum}) |
|
@@ -2257,6 +2344,17 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
2257
2344
|
| use_ascp4 | bool | Y | Y | | specify version of protocol |
|
2258
2345
|
| wss_enabled | bool | Y | Y | Y | Server has Web Socket service enabled |
|
2259
2346
|
| wss_port | int | Y | Y | Y | TCP port used for websocket service feed |
|
2347
|
+
| EX_ascp_args | array | Y | | | DEPRECATED: Use parameter ascp_args in option transfer_info<br/>Add native command line arguments to ascp |
|
2348
|
+
| EX_at_rest_password | string | Y | | | DEPRECATED: Use standard spec parameter: content_protection_password<br/>Content protection password<br/>(env:ASPERA_SCP_FILEPASS) |
|
2349
|
+
| EX_file_list | string | Y | | | source file list |
|
2350
|
+
| EX_file_pair_list | string | Y | | | source file pair list |
|
2351
|
+
| EX_http_proxy_url | string | Y | | | Specify the proxy server address used by HTTP Fallback<br/>(-x {string}) |
|
2352
|
+
| EX_http_transfer_jpeg | int | Y | | | HTTP transfers as JPEG file<br/>(-j {int}) |
|
2353
|
+
| EX_license_text | string | Y | | | License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE) |
|
2354
|
+
| EX_no_read | bool | Y | | | no read source<br/>(--no-read) |
|
2355
|
+
| EX_no_write | bool | Y | | | no write on destination<br/>(--no-write) |
|
2356
|
+
| EX_proxy_password | string | Y | | | Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS) |
|
2357
|
+
| EX_ssh_key_paths | array | Y | | | Use public key authentication for SSH and specify the private key file paths<br/>(-i {array}) |
|
2260
2358
|
|
2261
2359
|
#### Destination folder for transfers
|
2262
2360
|
|
@@ -2331,7 +2429,7 @@ So, by default, the list of files to transfer will be simply specified on the co
|
|
2331
2429
|
--sources=@lines:@stdin:
|
2332
2430
|
```
|
2333
2431
|
|
2334
|
-
- Using
|
2432
|
+
- Using Ruby code (one path per line in file)
|
2335
2433
|
|
2336
2434
|
```ruby
|
2337
2435
|
--sources=@ruby:'File.read("myfilelist.txt").split("\n")'
|
@@ -2365,9 +2463,9 @@ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
|
2365
2463
|
|
2366
2464
|
#### <a id="multisession"></a>Support of multi-session
|
2367
2465
|
|
2368
|
-
Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on
|
2466
|
+
Multi session, i.e. starting a transfer of a file set using multiple sessions (one `ascp` process per session) is supported on `direct` and `node` agents, not yet on connect.
|
2369
2467
|
|
2370
|
-
-
|
2468
|
+
- `--transfer=node`
|
2371
2469
|
|
2372
2470
|
```bash
|
2373
2471
|
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
|
@@ -2375,7 +2473,7 @@ Multi session, i.e. starting a transfer of a file set using multiple sessions (o
|
|
2375
2473
|
|
2376
2474
|
Multi-session is directly supported by the node daemon.
|
2377
2475
|
|
2378
|
-
-
|
2476
|
+
- `--transfer=direct`
|
2379
2477
|
|
2380
2478
|
```bash
|
2381
2479
|
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
|
@@ -2430,15 +2528,58 @@ Example: parameter to download a faspex package and decrypt on the fly
|
|
2430
2528
|
--ts=@json:'{"precalculate_job_size":true}'
|
2431
2529
|
```
|
2432
2530
|
|
2433
|
-
### <a id="
|
2531
|
+
### <a id="scheduler"></a>Scheduler
|
2434
2532
|
|
2435
2533
|
It is useful to configure automated scheduled execution.
|
2534
|
+
`ascli` does not provide an internal scheduler.
|
2535
|
+
Instead, use the service provided by the Operating system:
|
2536
|
+
|
2537
|
+
#### Windows Scheduler
|
2538
|
+
|
2539
|
+
Windows provides the [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page).
|
2540
|
+
It can be configured:
|
2541
|
+
|
2542
|
+
- Using utility [`schtasks.exe`](https://learn.microsoft.com/fr-fr/windows-server/administration/windows-commands/schtasks-create)
|
2543
|
+
|
2544
|
+
- Using powershell function [scheduletasks](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks)
|
2545
|
+
|
2546
|
+
- Using `taskschd.msc` (UI)
|
2436
2547
|
|
2437
|
-
####
|
2548
|
+
#### Unix-like Scheduler
|
2438
2549
|
|
2439
|
-
|
2550
|
+
Unix-like systems (Linux, ...) provide cron, configured using a [crontab](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
2440
2551
|
|
2441
|
-
|
2552
|
+
Linux also provides `anacron`, if tasks are hourly or daily.
|
2553
|
+
|
2554
|
+
For example, on Linux it is convenient to create a wrapping script, e.g. `cron_ascli` that will setup the environment (e.g. Ruby) to properly start `ascli`:
|
2555
|
+
|
2556
|
+
```bash
|
2557
|
+
#!/bin/bash
|
2558
|
+
# load the ruby environment
|
2559
|
+
. /etc/profile.d/rvm.sh
|
2560
|
+
rvm use 2.6 --quiet
|
2561
|
+
# set a timeout protection, just in case ascli is frozen
|
2562
|
+
tmout=30m
|
2563
|
+
# forward arguments to ascli
|
2564
|
+
exec timeout ${tmout} ascli "${@}"
|
2565
|
+
```
|
2566
|
+
|
2567
|
+
Example of cronjob created for user `xfer`.
|
2568
|
+
|
2569
|
+
```bash
|
2570
|
+
crontab<<EOF
|
2571
|
+
0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
|
2572
|
+
2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
|
2573
|
+
EOF
|
2574
|
+
```
|
2575
|
+
|
2576
|
+
> **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
|
2577
|
+
|
2578
|
+
### <a id="locking"></a>Locking for exclusive execution
|
2579
|
+
|
2580
|
+
In some cases one needs to ensure that `ascli` is not executed several times in parallel.
|
2581
|
+
|
2582
|
+
When `ascli` is executed automatically on a schedule basis, one generally desires that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
|
2442
2583
|
|
2443
2584
|
- Executing instances may pile-up and kill the system
|
2444
2585
|
- The same file may be transferred by multiple instances at the same time.
|
@@ -2456,7 +2597,7 @@ Usually the OS native scheduler already provides some sort of protection against
|
|
2456
2597
|
`ascli` natively supports a locking mechanism with option `lock_port`.
|
2457
2598
|
(Technically, this opens a local TCP server port, and fails if this port is already used, providing a local lock. Lock is released when process exits).
|
2458
2599
|
|
2459
|
-
|
2600
|
+
Testing `ascli` locking:
|
2460
2601
|
|
2461
2602
|
Run this same command in two separate terminals within less than 30 seconds:
|
2462
2603
|
|
@@ -2470,18 +2611,6 @@ The first instance will sleep 30 seconds, the second one will immediately exit l
|
|
2470
2611
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
2471
2612
|
```
|
2472
2613
|
|
2473
|
-
#### <a id="scheduler"></a>Scheduler
|
2474
|
-
|
2475
|
-
`ascli` does not provide an internal scheduler.
|
2476
|
-
|
2477
|
-
Instead, use the service provided by the Operating system:
|
2478
|
-
|
2479
|
-
- Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
|
2480
|
-
- Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
2481
|
-
- etc...
|
2482
|
-
|
2483
|
-
Linux also provides `anacron`, if tasks are hourly or daily.
|
2484
|
-
|
2485
2614
|
### "Provençale"
|
2486
2615
|
|
2487
2616
|
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
|
@@ -2608,7 +2737,7 @@ ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=seque
|
|
2608
2737
|
```text
|
2609
2738
|
ascli -h
|
2610
2739
|
NAME
|
2611
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
2740
|
+
ascli -- a command line tool for Aspera Applications (v4.13.0)
|
2612
2741
|
|
2613
2742
|
SYNOPSIS
|
2614
2743
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -2631,7 +2760,7 @@ COMMANDS
|
|
2631
2760
|
OPTIONS
|
2632
2761
|
Options begin with a '-' (minus), and value is provided on command line.
|
2633
2762
|
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
|
2634
|
-
base64,
|
2763
|
+
base64, csvt, env, file, json, lines, list, path, ruby, secret, stdin, uri, val, zlib, preset, incps, vault
|
2635
2764
|
Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
|
2636
2765
|
|
2637
2766
|
ARGS
|
@@ -2700,7 +2829,7 @@ OPTIONS:
|
|
2700
2829
|
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
|
2701
2830
|
--src-type=ENUM Type of file list: list, pair
|
2702
2831
|
--transfer=ENUM Type of transfer agent: direct, node, connect, httpgw, trsdk
|
2703
|
-
--transfer-info=VALUE Parameters for transfer agent
|
2832
|
+
--transfer-info=VALUE Parameters for transfer agent (Hash)
|
2704
2833
|
--progress=ENUM Type of progress bar: none, native, multi
|
2705
2834
|
|
2706
2835
|
|
@@ -2714,7 +2843,7 @@ OPTIONS:
|
|
2714
2843
|
|
2715
2844
|
|
2716
2845
|
COMMAND: node
|
2717
|
-
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
|
2846
|
+
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
|
2718
2847
|
OPTIONS:
|
2719
2848
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2720
2849
|
--username=VALUE username to log in
|
@@ -2723,7 +2852,7 @@ OPTIONS:
|
|
2723
2852
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
2724
2853
|
--sync-name=VALUE sync name
|
2725
2854
|
--path=VALUE file or folder path for gen4 operation "file"
|
2726
|
-
--token-type=ENUM
|
2855
|
+
--token-type=ENUM type of token used for transfers: aspera, basic, hybrid
|
2727
2856
|
--default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
|
2728
2857
|
|
2729
2858
|
|
@@ -2769,7 +2898,7 @@ OPTIONS:
|
|
2769
2898
|
|
2770
2899
|
|
2771
2900
|
COMMAND: faspex5
|
2772
|
-
SUBCOMMANDS: admin bearer_token gateway health
|
2901
|
+
SUBCOMMANDS: admin bearer_token gateway health packages postprocessing shared_folders user version
|
2773
2902
|
OPTIONS:
|
2774
2903
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2775
2904
|
--username=VALUE username to log in
|
@@ -2777,10 +2906,12 @@ OPTIONS:
|
|
2777
2906
|
--client-id=VALUE OAuth client identifier
|
2778
2907
|
--client-secret=VALUE OAuth client secret
|
2779
2908
|
--redirect-uri=VALUE OAuth redirect URI for web authentication
|
2780
|
-
--auth=ENUM OAuth type of authentication: boot, web, jwt
|
2909
|
+
--auth=ENUM OAuth type of authentication: boot, link, web, jwt
|
2910
|
+
--box=VALUE Package inbox, either shared inbox name or one of ["inbox", "inbox_history", "inbox_all", "inbox_all_history", "outbox", "outbox_history", "pending", "pending_history", "all"]
|
2781
2911
|
--private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
|
2782
2912
|
--passphrase=VALUE RSA private key passphrase
|
2783
2913
|
--shared-folder=VALUE Shared folder source for package files
|
2914
|
+
--link=VALUE public link for specific operation
|
2784
2915
|
|
2785
2916
|
|
2786
2917
|
COMMAND: cos
|
@@ -2834,8 +2965,9 @@ OPTIONS:
|
|
2834
2965
|
--thumb-text-font=VALUE png: plaintext: font to render text with imagemagick convert (identify -list font)
|
2835
2966
|
--video-conversion=ENUM mp4: method for preview generation: [reencode], blend, clips
|
2836
2967
|
--video-png-conv=ENUM mp4: method for thumbnail generation: [fixed], animated
|
2837
|
-
--video-
|
2838
|
-
--video-
|
2968
|
+
--video-scale=VALUE mp4: all: video scale (ffmpeg)
|
2969
|
+
--video-start-sec=VALUE mp4: all: start offset (seconds) of video preview
|
2970
|
+
--reencode-ffmpeg=VALUE mp4: reencode: options to ffmpeg
|
2839
2971
|
--blend-keyframes=VALUE mp4: blend: # key frames
|
2840
2972
|
--blend-pauseframes=VALUE mp4: blend: # pause frames
|
2841
2973
|
--blend-transframes=VALUE mp4: blend: # transition blend frames
|
@@ -2873,13 +3005,13 @@ OPTIONS:
|
|
2873
3005
|
--validate-metadata=ENUM Validate shared inbox metadata: [no], yes
|
2874
3006
|
|
2875
3007
|
COMMAND: node
|
2876
|
-
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space stream sync transfer upload watch_folder
|
3008
|
+
SUBCOMMANDS: access_key api_details asperabrowser async basic_token browse central delete download events health info license mkdir mkfile mklink rename search service space ssync stream sync transfer upload watch_folder
|
2877
3009
|
OPTIONS:
|
2878
3010
|
--validator=VALUE identifier of validator (optional for central)
|
2879
3011
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
2880
3012
|
--sync-name=VALUE sync name
|
2881
3013
|
--path=VALUE file or folder path for gen4 operation "file"
|
2882
|
-
--token-type=ENUM
|
3014
|
+
--token-type=ENUM type of token used for transfers: aspera, basic, hybrid
|
2883
3015
|
--default-ports=ENUM use standard FASP ports or get from node api (gen4): [no], yes
|
2884
3016
|
|
2885
3017
|
|
@@ -2962,7 +3094,7 @@ ascli config wizard --value=aoc
|
|
2962
3094
|
|
2963
3095
|
Several types of OAuth authentication are supported:
|
2964
3096
|
|
2965
|
-
- JSON Web Token (JWT) : authentication is secured by a private key (recommended for
|
3097
|
+
- JSON Web Token (JWT) : authentication is secured by a private key (recommended for `ascli`)
|
2966
3098
|
- Web based authentication : authentication is made by user using a browser
|
2967
3099
|
- URL Token : external users authentication with url tokens (public links)
|
2968
3100
|
|
@@ -3004,7 +3136,6 @@ If you did not use the wizard, you can also manually create a [option preset](#l
|
|
3004
3136
|
|
3005
3137
|
Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interactive input (client info from previous step):
|
3006
3138
|
|
3007
|
-
<!-- spellchecker: disable -->
|
3008
3139
|
```bash
|
3009
3140
|
ascli config preset ask my_aoc_org url client_id client_secret
|
3010
3141
|
option: url> https://myorg.ibmaspera.com/
|
@@ -3012,7 +3143,6 @@ option: client_id> my_client_id_here
|
|
3012
3143
|
option: client_secret> my_client_secret_here
|
3013
3144
|
updated: my_aoc_org
|
3014
3145
|
```
|
3015
|
-
<!-- spellchecker: enable -->
|
3016
3146
|
|
3017
3147
|
(This can also be done in one line using the command `config preset update my_aoc_org --url=...`)
|
3018
3148
|
|
@@ -3051,11 +3181,11 @@ ascli aoc admin res client list
|
|
3051
3181
|
```
|
3052
3182
|
|
3053
3183
|
```output
|
3054
|
-
|
3055
|
-
|
3056
|
-
|
3057
|
-
|
3058
|
-
|
3184
|
+
+------------+---------------+
|
3185
|
+
| id | name |
|
3186
|
+
+------------+---------------+
|
3187
|
+
| my_BJbQiFw | my-client-app |
|
3188
|
+
+------------+---------------+
|
3059
3189
|
```
|
3060
3190
|
|
3061
3191
|
```bash
|
@@ -3087,12 +3217,12 @@ ascli aoc admin res user list
|
|
3087
3217
|
```
|
3088
3218
|
|
3089
3219
|
```output
|
3090
|
-
|
3091
|
-
|
3092
|
-
|
3093
|
-
|
3094
|
-
|
3095
|
-
|
3220
|
+
+--------+----------------+
|
3221
|
+
| id | name |
|
3222
|
+
+--------+----------------+
|
3223
|
+
| 109952 | Tech Support |
|
3224
|
+
| 109951 | LAURENT MARTIN |
|
3225
|
+
+--------+----------------+
|
3096
3226
|
```
|
3097
3227
|
|
3098
3228
|
```ruby
|
@@ -3341,29 +3471,27 @@ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.
|
|
3341
3471
|
```
|
3342
3472
|
|
3343
3473
|
```output
|
3344
|
-
|
3345
|
-
|
3346
|
-
|
3347
|
-
|
3348
|
-
|
3349
|
-
|
3474
|
+
+-------+---------+
|
3475
|
+
| id | status |
|
3476
|
+
+-------+---------+
|
3477
|
+
| 98398 | created |
|
3478
|
+
| 98399 | created |
|
3479
|
+
+-------+---------+
|
3350
3480
|
```
|
3351
3481
|
|
3352
3482
|
#### Example: Find with filter and delete
|
3353
3483
|
|
3354
|
-
<!-- spellchecker: disable -->
|
3355
|
-
|
3356
3484
|
```bash
|
3357
3485
|
ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
|
3358
3486
|
```
|
3359
3487
|
|
3360
3488
|
```output
|
3361
|
-
|
3362
|
-
|
3363
|
-
|
3364
|
-
|
3365
|
-
|
3366
|
-
|
3489
|
+
+-------+------------------------+
|
3490
|
+
| id | email |
|
3491
|
+
+-------+------------------------+
|
3492
|
+
| 98398 | dummyuser1@example.com |
|
3493
|
+
| 98399 | dummyuser2@example.com |
|
3494
|
+
+-------+------------------------+
|
3367
3495
|
```
|
3368
3496
|
|
3369
3497
|
```bash
|
@@ -3383,16 +3511,14 @@ ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
|
|
3383
3511
|
```
|
3384
3512
|
|
3385
3513
|
```output
|
3386
|
-
|
3387
|
-
|
3388
|
-
|
3389
|
-
|
3390
|
-
|
3391
|
-
|
3514
|
+
+-------+---------+
|
3515
|
+
| id | status |
|
3516
|
+
+-------+---------+
|
3517
|
+
| 98398 | deleted |
|
3518
|
+
| 98399 | deleted |
|
3519
|
+
+-------+---------+
|
3392
3520
|
```
|
3393
3521
|
|
3394
|
-
<!-- spellchecker: enable -->
|
3395
|
-
|
3396
3522
|
#### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
|
3397
3523
|
|
3398
3524
|
```ruby
|
@@ -3408,13 +3534,13 @@ ascli aoc user workspaces list
|
|
3408
3534
|
```
|
3409
3535
|
|
3410
3536
|
```output
|
3411
|
-
|
3412
|
-
|
3413
|
-
|
3414
|
-
|
3415
|
-
|
3416
|
-
|
3417
|
-
|
3537
|
+
+------+----------------------------+
|
3538
|
+
| id | name |
|
3539
|
+
+------+----------------------------+
|
3540
|
+
| 16 | Engineering |
|
3541
|
+
| 17 | Marketing |
|
3542
|
+
| 18 | Sales |
|
3543
|
+
+------+----------------------------+
|
3418
3544
|
```
|
3419
3545
|
|
3420
3546
|
#### Example: Create a sub access key in a "node"
|
@@ -3454,16 +3580,16 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
|
|
3454
3580
|
```
|
3455
3581
|
|
3456
3582
|
```output
|
3457
|
-
|
3458
|
-
|
3459
|
-
|
3460
|
-
|
3461
|
-
|
3462
|
-
|
3463
|
-
|
3464
|
-
|
3465
|
-
|
3466
|
-
|
3583
|
+
+-------------+---------+----------------------------------+
|
3584
|
+
| member_type | manager | member.email |
|
3585
|
+
+-------------+---------+----------------------------------+
|
3586
|
+
| user | true | john.curtis@email.com |
|
3587
|
+
| user | false | laurent.martin.aspera@fr.ibm.com |
|
3588
|
+
| user | false | jean.dupont@me.com |
|
3589
|
+
| user | false | another.user@example.com |
|
3590
|
+
| group | false | |
|
3591
|
+
| user | false | aspera.user@gmail.com |
|
3592
|
+
+-------------+---------+----------------------------------+
|
3467
3593
|
```
|
3468
3594
|
|
3469
3595
|
Other query parameters:
|
@@ -3519,12 +3645,12 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
|
|
3519
3645
|
```
|
3520
3646
|
|
3521
3647
|
```output
|
3522
|
-
|
3523
|
-
|
3524
|
-
|
3525
|
-
|
3526
|
-
|
3527
|
-
|
3648
|
+
+-------------------------------+
|
3649
|
+
| email |
|
3650
|
+
+-------------------------------+
|
3651
|
+
| John.curtis@acme.com |
|
3652
|
+
| Jean.Dupont@tropfort.com |
|
3653
|
+
+-------------------------------+
|
3528
3654
|
```
|
3529
3655
|
|
3530
3656
|
#### Example: List "Limited" users
|
@@ -3615,8 +3741,6 @@ ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AW
|
|
3615
3741
|
|
3616
3742
|
#### Example: create registration key to register a node
|
3617
3743
|
|
3618
|
-
<!-- spellchecker: disable -->
|
3619
|
-
|
3620
3744
|
```bash
|
3621
3745
|
ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
3622
3746
|
```
|
@@ -3642,8 +3766,6 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
|
|
3642
3766
|
+-----+---------+
|
3643
3767
|
```
|
3644
3768
|
|
3645
|
-
<!-- spellchecker: enable -->
|
3646
|
-
|
3647
3769
|
#### Example: Create a Node
|
3648
3770
|
|
3649
3771
|
AoC nodes as actually composed with two related entities:
|
@@ -3755,7 +3877,7 @@ When user packages are listed, the following query is used:
|
|
3755
3877
|
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
3756
3878
|
```
|
3757
3879
|
|
3758
|
-
To list packages in a shared inbox, the query has to be specified with
|
3880
|
+
To list packages in a shared inbox, the query has to be specified with the the shared inbox by name or its identifier.
|
3759
3881
|
Additional parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode).
|
3760
3882
|
The current workspace is added unless specified in the query.
|
3761
3883
|
|
@@ -4022,8 +4144,10 @@ aoc files rename /somefolder testdst
|
|
4022
4144
|
aoc files short_link create --to-folder=/testdst --value=private
|
4023
4145
|
aoc files short_link create --to-folder=/testdst --value=public
|
4024
4146
|
aoc files short_link list --value=@json:'{"purpose":"shared_folder_auth_link"}'
|
4025
|
-
aoc files sync
|
4026
|
-
aoc files sync
|
4147
|
+
aoc files sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
|
4148
|
+
aoc files sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
|
4149
|
+
aoc files sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/testdst"}}'
|
4150
|
+
aoc files sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/testdst","reset":true}]}'
|
4027
4151
|
aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
|
4028
4152
|
aoc files upload --to-folder=/ testfile.bin --link=my_aoc_publink_folder
|
4029
4153
|
aoc files upload --to-folder=/testsrc testfile.bin
|
@@ -4069,8 +4193,6 @@ If you are using ATS as part of AoC, then authentication is through AoC, not IBM
|
|
4069
4193
|
|
4070
4194
|
First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
|
4071
4195
|
|
4072
|
-
<!-- spellchecker: disable -->
|
4073
|
-
|
4074
4196
|
```bash
|
4075
4197
|
ibmcloud iam api-key-create mykeyname -d 'my sample key'
|
4076
4198
|
```
|
@@ -4090,6 +4212,7 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
|
|
4090
4212
|
```
|
4091
4213
|
|
4092
4214
|
References:
|
4215
|
+
<!-- spellchecker: disable -->
|
4093
4216
|
|
4094
4217
|
- [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
|
4095
4218
|
- [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
|
@@ -4098,8 +4221,6 @@ References:
|
|
4098
4221
|
|
4099
4222
|
Then, to register the key by default for the ats plugin, create a preset. Execute:
|
4100
4223
|
|
4101
|
-
<!-- spellchecker: disable -->
|
4102
|
-
|
4103
4224
|
```bash
|
4104
4225
|
ascli config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here
|
4105
4226
|
```
|
@@ -4138,8 +4259,6 @@ ascli ats api_key create
|
|
4138
4259
|
ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
4139
4260
|
```
|
4140
4261
|
|
4141
|
-
<!-- spellchecker: enable -->
|
4142
|
-
|
4143
4262
|
### <a id="ats_params"></a>ATS Access key creation parameters
|
4144
4263
|
|
4145
4264
|
When creating an ATS access key, the option `params` must contain an extended value with the creation parameters.
|
@@ -4214,7 +4333,6 @@ then commands `ascp` (for transfers) and `ascmd` (for file operations) are execu
|
|
4214
4333
|
### Server sample commands
|
4215
4334
|
|
4216
4335
|
```bash
|
4217
|
-
server br /
|
4218
4336
|
server browse /
|
4219
4337
|
server browse NEW_SERVER_FOLDER/testfile.bin
|
4220
4338
|
server browse folder_1/target_hot
|
@@ -4232,10 +4350,10 @@ server md5sum NEW_SERVER_FOLDER/testfile.bin
|
|
4232
4350
|
server mkdir NEW_SERVER_FOLDER --logger=stdout
|
4233
4351
|
server mkdir folder_1/target_hot
|
4234
4352
|
server mv folder_1/200KB.2 folder_1/to.delete
|
4235
|
-
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"
|
4236
|
-
server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"
|
4237
|
-
server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"
|
4238
|
-
server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"
|
4353
|
+
server sync admin status --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"}}'
|
4354
|
+
server sync admin status --sync-session=mysync --sync-info=@json:'{"sessions":[{"name":"mysync","local_dir":"my_local_sync_dir"}]}'
|
4355
|
+
server sync start --sync-info=@json:'{"name":"sync2","local":{"path":"my_local_sync_dir"},"remote":{"path":"'"NEW_SERVER_FOLDER"'"},"reset":true,"quiet":false}'
|
4356
|
+
server sync start --sync-info=@json:'{"sessions":[{"name":"mysync","direction":"pull","remote_dir":"'"NEW_SERVER_FOLDER"'","local_dir":"my_local_sync_dir","reset":true}]}'
|
4239
4357
|
server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
|
4240
4358
|
server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
|
4241
4359
|
server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
|
@@ -4277,9 +4395,9 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
|
|
4277
4395
|
ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
|
4278
4396
|
```
|
4279
4397
|
|
4280
|
-
For file operation command (browse, delete), the
|
4398
|
+
For file operation command (browse, delete), the Ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
|
4281
4399
|
|
4282
|
-
For a list of SSH client options, refer to the
|
4400
|
+
For a list of SSH client options, refer to the Ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html#method-c-start).
|
4283
4401
|
|
4284
4402
|
Some of the 50 available SSH options:
|
4285
4403
|
|
@@ -4301,7 +4419,7 @@ or on Windows:
|
|
4301
4419
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
4302
4420
|
```
|
4303
4421
|
|
4304
|
-
This means that your environment
|
4422
|
+
This means that your environment suggests to use an agent but you don't have such an SSH agent running, then:
|
4305
4423
|
|
4306
4424
|
- Check env var: `SSH_AGENT_SOCK`
|
4307
4425
|
- Check your file: `$HOME/.ssh/config`
|
@@ -4341,6 +4459,8 @@ ascli server download /aspera-test-dir-large/200MB
|
|
4341
4459
|
|
4342
4460
|
This plugin gives access to capabilities provided by HSTS node API.
|
4343
4461
|
|
4462
|
+
> **Note:** capabilities of this plugin are used in other plugins which access to the node API, such as `aoc`.
|
4463
|
+
|
4344
4464
|
### File Operations
|
4345
4465
|
|
4346
4466
|
It is possible to:
|
@@ -4369,7 +4489,8 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
4369
4489
|
|
4370
4490
|
### Central
|
4371
4491
|
|
4372
|
-
The central subcommand uses the "reliable query" API (session and file).
|
4492
|
+
The central subcommand uses the "reliable query" API (session and file).
|
4493
|
+
It allows listing transfer sessions and transferred files.
|
4373
4494
|
|
4374
4495
|
Filtering can be applied:
|
4375
4496
|
|
@@ -4398,16 +4519,12 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
|
|
4398
4519
|
- Start watchd and watchfolderd services running as a system user having access to files
|
4399
4520
|
- configure a watchfolder to define automated transfers
|
4400
4521
|
|
4401
|
-
<!-- spellchecker: disable -->
|
4402
|
-
|
4403
4522
|
```bash
|
4404
4523
|
ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
4405
4524
|
ascli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
|
4406
4525
|
ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
|
4407
4526
|
```
|
4408
4527
|
|
4409
|
-
<!-- spellchecker: enable -->
|
4410
|
-
|
4411
4528
|
### Out of Transfer File Validation
|
4412
4529
|
|
4413
4530
|
Follow the Aspera Transfer Server configuration to activate this feature.
|
@@ -4417,11 +4534,11 @@ ascli node central file list --validator=ascli --data=@json:'{"file_transfer_fil
|
|
4417
4534
|
```
|
4418
4535
|
|
4419
4536
|
```output
|
4420
|
-
|
4421
|
-
|
4422
|
-
|
4423
|
-
|
4424
|
-
|
4537
|
+
+--------------+--------------+------------+--------------------------------------+
|
4538
|
+
| session_uuid | file_id | status | path |
|
4539
|
+
+--------------+--------------+------------+--------------------------------------+
|
4540
|
+
| 1a74444c-... | 084fb181-... | validating | /home/xfer.../PKG - my title/200KB.1 |
|
4541
|
+
+--------------+--------------+------------+--------------------------------------+
|
4425
4542
|
```
|
4426
4543
|
|
4427
4544
|
```bash
|
@@ -4447,6 +4564,28 @@ ascli node download /share/sourcefile --to-folder=/destination_folder --preset=a
|
|
4447
4564
|
|
4448
4565
|
This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
|
4449
4566
|
|
4567
|
+
### node file information
|
4568
|
+
|
4569
|
+
When node api is used with an **Access key**, extra information can be retrieved, such as preview.
|
4570
|
+
|
4571
|
+
> **Note:** Display of preview on terminal requires installation of extra gem: `rmagick`
|
4572
|
+
|
4573
|
+
```bash
|
4574
|
+
gem install rmagick
|
4575
|
+
```
|
4576
|
+
|
4577
|
+
For example it is possible to display the preview of a file, if it exists, using:
|
4578
|
+
|
4579
|
+
```bash
|
4580
|
+
ascli aoc files file thumbnail --path=/preview_samples/Aspera.mpg
|
4581
|
+
```
|
4582
|
+
|
4583
|
+
Using direct node access and an access key , one can do:
|
4584
|
+
|
4585
|
+
```bash
|
4586
|
+
ascli node access_key do self file thumbnail --path=/preview_samples/Aspera.mpg
|
4587
|
+
```
|
4588
|
+
|
4450
4589
|
### Create access key
|
4451
4590
|
|
4452
4591
|
```bash
|
@@ -4456,14 +4595,15 @@ ascli node access_key create --value=@json:'{"id":"myaccesskey","secret":"my_sec
|
|
4456
4595
|
### Node sample commands
|
4457
4596
|
|
4458
4597
|
```bash
|
4459
|
-
node access_key create --value=@json:'{"id":"
|
4460
|
-
node access_key delete
|
4598
|
+
node access_key create --value=@json:'{"id":"testingAK1","storage":{"type":"local","path":"/"}}'
|
4599
|
+
node access_key delete testingAK1
|
4461
4600
|
node access_key do my_aoc_ak_name browse /
|
4462
4601
|
node access_key do my_aoc_ak_name delete /folder2
|
4463
4602
|
node access_key do my_aoc_ak_name delete testfile1
|
4464
4603
|
node access_key do my_aoc_ak_name download testfile1 --to-folder=.
|
4465
4604
|
node access_key do my_aoc_ak_name file show --path=/testfile1
|
4466
4605
|
node access_key do my_aoc_ak_name file show 1
|
4606
|
+
node access_key do my_aoc_ak_name file thumbnail --path=/testfile1
|
4467
4607
|
node access_key do my_aoc_ak_name find /
|
4468
4608
|
node access_key do my_aoc_ak_name mkdir /folder1
|
4469
4609
|
node access_key do my_aoc_ak_name node_info /
|
@@ -4481,7 +4621,7 @@ node basic_token
|
|
4481
4621
|
node browse / -r
|
4482
4622
|
node delete /todelete
|
4483
4623
|
node delete @list:,folder_1/todelete,folder_1/tdlink,folder_1/delfile
|
4484
|
-
node delete folder_1/10MB.
|
4624
|
+
node delete folder_1/10MB.2
|
4485
4625
|
node delete testfile.bin
|
4486
4626
|
node download testfile.bin --to-folder=.
|
4487
4627
|
node download testfile.bin --to-folder=. --token-type=hybrid
|
@@ -4497,19 +4637,23 @@ node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"us
|
|
4497
4637
|
node service delete service1
|
4498
4638
|
node service list
|
4499
4639
|
node space /
|
4500
|
-
node
|
4501
|
-
node
|
4502
|
-
node
|
4503
|
-
node
|
4504
|
-
node
|
4505
|
-
node
|
4506
|
-
node
|
4507
|
-
node
|
4508
|
-
node
|
4509
|
-
node
|
4510
|
-
node
|
4640
|
+
node ssync bandwidth my_syncid
|
4641
|
+
node ssync counters my_syncid
|
4642
|
+
node ssync create --value=@json:'{"configuration":{"name":"sync1","local":{"path":"my_local_path"},"remote":{"host":"my_host","port":my_port,"user":"my_username","pass":"my_password","path":"my_remote_path"}}}'
|
4643
|
+
node ssync delete my_syncid
|
4644
|
+
node ssync files my_syncid
|
4645
|
+
node ssync list
|
4646
|
+
node ssync show my_syncid
|
4647
|
+
node ssync start my_syncid
|
4648
|
+
node ssync state my_syncid
|
4649
|
+
node ssync stop my_syncid
|
4650
|
+
node ssync summary my_syncid
|
4651
|
+
node sync ad st --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
4652
|
+
node sync ad st --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
4653
|
+
node sync start --sync-info=@json:'{"name":"syncv2","reset":true,"direction":"pull","local":{"path":"my_local_sync_dir"},"remote":{"path":"/aspera-test-dir-tiny"}}'
|
4654
|
+
node sync start --sync-info=@json:'{"sessions":[{"name":"syncv1","direction":"pull","local_dir":"my_local_sync_dir","remote_dir":"/aspera-test-dir-tiny","reset":true}]}'
|
4511
4655
|
node transfer list --value=@json:'{"active_only":true}'
|
4512
|
-
node upload --to-folder=
|
4656
|
+
node upload --to-folder=folder_1 --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.2"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass_here"}'
|
4513
4657
|
node upload --username=my_aoc_ak_name --password=my_aoc_ak_secret testfile.bin --token-type=basic
|
4514
4658
|
node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
|
4515
4659
|
node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
|
@@ -4593,6 +4737,9 @@ Then `ascli` will lookup existing contacts, and if a single match is found will
|
|
4593
4737
|
|
4594
4738
|
> **Note:** The lookup is case insensitive and on partial matches.
|
4595
4739
|
|
4740
|
+
On reception, option `box` (default to `inbox`) can be set to the same values as API accepts, or to the name of a shared inbox.
|
4741
|
+
If the value `ALL` is provided to option `box`, then all packages are selected.
|
4742
|
+
|
4596
4743
|
### Faspex 5 sample commands
|
4597
4744
|
|
4598
4745
|
Most commands are directly REST API calls.
|
@@ -4606,20 +4753,24 @@ faspex5 admin res accounts list
|
|
4606
4753
|
faspex5 admin res contacts list
|
4607
4754
|
faspex5 admin res jobs list
|
4608
4755
|
faspex5 admin res metadata_profiles list
|
4609
|
-
faspex5 admin res node list
|
4756
|
+
faspex5 admin res node list
|
4610
4757
|
faspex5 admin res oauth_clients list
|
4611
4758
|
faspex5 admin res registrations list
|
4612
4759
|
faspex5 admin res saml_configs list
|
4613
4760
|
faspex5 admin res shared_inboxes list
|
4614
4761
|
faspex5 admin res workgroups list
|
4762
|
+
faspex5 admin smtp show
|
4763
|
+
faspex5 admin smtp test my_email_external
|
4615
4764
|
faspex5 bearer_token
|
4616
4765
|
faspex5 gateway --value=https://localhost:12345/aspera/faspex &\
|
4617
4766
|
faspex5 health
|
4618
|
-
faspex5
|
4619
|
-
faspex5
|
4620
|
-
faspex5
|
4621
|
-
faspex5
|
4622
|
-
faspex5
|
4767
|
+
faspex5 packages list --value=@json:'{"mailbox":"inbox","state":["released"]}'
|
4768
|
+
faspex5 packages receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
|
4769
|
+
faspex5 packages receive ALL --once-only=yes --to-folder=.
|
4770
|
+
faspex5 packages receive INIT --once-only=yes
|
4771
|
+
faspex5 packages send --value=@json:'{"title":"test title","recipients":["my_shinbox"],"metadata":{"Options":"Opt1","TextInput":"example text"}}' testfile.bin
|
4772
|
+
faspex5 packages send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"my_passphrase_here"}'
|
4773
|
+
faspex5 packages show "my_package_id"
|
4623
4774
|
faspex5 postprocessing --value=@json:'{"url":"https://localhost:8443/domain","processing":{"script_folder":"tests"},"certificate":{"key":"../local/k","cert":"../local/c","chain":"../local/ch"}}' &\
|
4624
4775
|
faspex5 user profile modify @json:'{"preference":{"connect_disabled":false}}'
|
4625
4776
|
faspex5 user profile show
|
@@ -4627,6 +4778,16 @@ faspex5 user profile show
|
|
4627
4778
|
|
4628
4779
|
Other examples:
|
4629
4780
|
|
4781
|
+
- list packages
|
4782
|
+
|
4783
|
+
The following parameters in option `value` are supported:
|
4784
|
+
|
4785
|
+
- `q` : a filter on name (case insensitive, matches if value is contained in name)
|
4786
|
+
- `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
4787
|
+
- `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
4788
|
+
- `offset` : native api parameter, in general do not use (added by `ascli`)
|
4789
|
+
- `limit` : native api parameter, number of items par api call, in general do not use (added by `ascli`)
|
4790
|
+
|
4630
4791
|
- Send a package with metadata
|
4631
4792
|
|
4632
4793
|
The interface is the one of the API (Refer to API documentation, or look at request in browser):
|
@@ -4669,6 +4830,20 @@ ascli faspex5 shared_folders br 3 /folder
|
|
4669
4830
|
ascli faspex5 package send --value=@json:'{"title":"hello","recipients":[{"name":"_recipient_here_"}]}' --shared-folder=3 /folder/file
|
4670
4831
|
```
|
4671
4832
|
|
4833
|
+
- receive all packages (cargo)
|
4834
|
+
|
4835
|
+
To receive all packages, only once, through persistency of already received packages:
|
4836
|
+
|
4837
|
+
```bash
|
4838
|
+
ascli faspex5 packages receive ALL --once-only=yes
|
4839
|
+
```
|
4840
|
+
|
4841
|
+
To initialize, and skip all current package so that next time `ALL` is used, only newer packages are downloaded:
|
4842
|
+
|
4843
|
+
```bash
|
4844
|
+
ascli faspex5 packages receive INIT --once-only=yes
|
4845
|
+
```
|
4846
|
+
|
4672
4847
|
### Faspex 4-style postprocessing script with Faspex 5
|
4673
4848
|
|
4674
4849
|
`ascli` provides command `postprocessing` in plugin `faspex5` to emulate Faspex 4 postprocessing.
|
@@ -4684,12 +4859,12 @@ The following parameters are supported:
|
|
4684
4859
|
|
4685
4860
|
| parameter | type | default | description |
|
4686
4861
|
|----------------------------|---------|------------------------|-----------------------------------------------------|
|
4687
|
-
| url | string | http://localhost:8080 | Defines the base url on which requests are listened |
|
4862
|
+
| url | string | <http://localhost:8080> | Defines the base url on which requests are listened |
|
4688
4863
|
| certificate | hash | nil | used to define certificate if https is used |
|
4689
4864
|
| certificate.key | string | nil | path to private key file |
|
4690
4865
|
| certificate.cert | string | nil | path to certificate |
|
4691
4866
|
| certificate.chain | string | nil | path to intermediary certificates |
|
4692
|
-
| processing | hash | nil |
|
4867
|
+
| processing | hash | nil | behavior of post processing |
|
4693
4868
|
| processing.script_folder | string | . | prefix added to script path |
|
4694
4869
|
| processing.fail_on_error | bool | false | if true and process exit with non zero, then fail |
|
4695
4870
|
| processing.timeout_seconds | integer | 60 | processing script is killed if takes more time |
|
@@ -4704,7 +4879,7 @@ When a request is received the following happens:
|
|
4704
4879
|
|
4705
4880
|
- the processor get the path of the url called
|
4706
4881
|
- it removes the "domain
|
4707
|
-
- it
|
4882
|
+
- it prepends it with the value of `script_folder`
|
4708
4883
|
- it executes the script
|
4709
4884
|
- upon success, a success code is returned
|
4710
4885
|
|
@@ -4752,7 +4927,8 @@ As inboxes may be large, it is possible to use the following query parameters:
|
|
4752
4927
|
|
4753
4928
|
The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/catalog/?search=faspex) under "Services (API v.3)".
|
4754
4929
|
|
4755
|
-
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
4930
|
+
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`).
|
4931
|
+
By default `count` is `0` (`10`), it can be increased to issue less HTTP calls.
|
4756
4932
|
|
4757
4933
|
#### Example: list packages in dropbox
|
4758
4934
|
|
@@ -4879,18 +5055,18 @@ faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_i
|
|
4879
5055
|
faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
|
4880
5056
|
faspex package recv "my_package_id" --to-folder=.
|
4881
5057
|
faspex package recv "my_package_id" --to-folder=. --box=sent
|
4882
|
-
faspex package recv --to-folder=. --link=
|
5058
|
+
faspex package recv --to-folder=. --link=https://app.example.com/recv_from_user_path
|
4883
5059
|
faspex package recv ALL --to-folder=. --once-only=yes
|
4884
5060
|
faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
|
4885
5061
|
faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
|
4886
5062
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
|
4887
5063
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
|
4888
5064
|
faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
|
4889
|
-
faspex package send --link=
|
4890
|
-
faspex package send --link=
|
5065
|
+
faspex package send --link=https://app.example.com/send_to_dropbox_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
5066
|
+
faspex package send --link=https://app.example.com/send_to_user_path --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
4891
5067
|
faspex source list
|
4892
|
-
faspex source name
|
4893
|
-
faspex source name
|
5068
|
+
faspex source name my_faspex_src info
|
5069
|
+
faspex source name my_faspex_src node br /
|
4894
5070
|
faspex v4 dmembership list
|
4895
5071
|
faspex v4 dropbox list
|
4896
5072
|
faspex v4 metadata_profile list
|
@@ -4901,7 +5077,7 @@ faspex v4 workgroup list
|
|
4901
5077
|
|
4902
5078
|
## <a id="shares"></a>Plugin: `shares`: IBM Aspera Shares v1
|
4903
5079
|
|
4904
|
-
Aspera Shares supports the "node API" for the file transfer part.
|
5080
|
+
Aspera Shares supports the "node API" for the file transfer part.
|
4905
5081
|
|
4906
5082
|
### Shares 1 sample commands
|
4907
5083
|
|
@@ -5011,8 +5187,6 @@ ibmcloud resource service-key _service_key_name_here_ --output JSON|jq '.[0].cre
|
|
5011
5187
|
|
5012
5188
|
It consists in the following structure:
|
5013
5189
|
|
5014
|
-
<!-- spellchecker: disable -->
|
5015
|
-
|
5016
5190
|
```json
|
5017
5191
|
{
|
5018
5192
|
"apikey": "my_api_key_here",
|
@@ -5029,8 +5203,6 @@ It consists in the following structure:
|
|
5029
5203
|
}
|
5030
5204
|
```
|
5031
5205
|
|
5032
|
-
<!-- spellchecker: enable -->
|
5033
|
-
|
5034
5206
|
The field `resource_instance_id` is for option `crn`
|
5035
5207
|
|
5036
5208
|
The field `apikey` is for option `apikey`
|
@@ -5076,20 +5248,30 @@ cos node upload testfile.bin
|
|
5076
5248
|
|
5077
5249
|
## <a id="async"></a>Plugin: `async`: IBM Aspera Sync
|
5078
5250
|
|
5079
|
-
A basic plugin to start an
|
5080
|
-
The main advantage over bare `async` command line is the possibility to use a configuration file, using
|
5251
|
+
A basic plugin to start an `async` using `ascli`.
|
5252
|
+
The main advantage over bare `async` command line is the possibility to use a configuration file, using standard options of `ascli`.
|
5081
5253
|
|
5082
|
-
|
5083
|
-
In this case, some of the `sync` parameters are
|
5254
|
+
The `sync` command is also made available through the `server sync`, `aoc files sync` and `node sync` commands.
|
5255
|
+
In this case, some of the `sync` parameters are filled by the related plugin using transfer spec parameters (including token).
|
5084
5256
|
|
5085
5257
|
> **Note:** All `sync` commands require an `async` enabled license and availability of the `async` executable (and `asyncadmin`).
|
5086
|
-
|
5087
|
-
|
5088
|
-
|
5089
|
-
|
5258
|
+
|
5259
|
+
Two JSON syntax are supported for option `sync_info`.
|
5260
|
+
|
5261
|
+
### async native JSON
|
5262
|
+
|
5263
|
+
It is the same payload as specified on the `async` option `--conf` or in the latest node API.
|
5264
|
+
This is the preferred syntax and allows a single session definition.
|
5265
|
+
But there is no progress output nor error messages.
|
5090
5266
|
|
5091
5267
|
Documentation on Async node API can be found on [IBM Developer Portal](https://developer.ibm.com/apis/catalog?search=%22aspera%20sync%20api%22).
|
5092
5268
|
|
5269
|
+
### async options as JSON
|
5270
|
+
|
5271
|
+
This is specific to `ascli`.
|
5272
|
+
It is based on a JSON representation of `async` command line options.
|
5273
|
+
It allows definition of multiple sync sessions in a single command, although usually only one sync session is defined.
|
5274
|
+
|
5093
5275
|
### Sync sample commands
|
5094
5276
|
|
5095
5277
|
```bash
|
@@ -5194,7 +5376,6 @@ dnf install unoconv
|
|
5194
5376
|
|
5195
5377
|
- Amazon Linux
|
5196
5378
|
|
5197
|
-
<!-- spellchecker: disable -->
|
5198
5379
|
```bash
|
5199
5380
|
amazon-linux-extras enable libreoffice
|
5200
5381
|
yum clean metadata
|
@@ -5203,15 +5384,17 @@ wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
|
|
5203
5384
|
mv unoconv /usr/bin
|
5204
5385
|
chmod a+x /usr/bin/unoconv
|
5205
5386
|
```
|
5206
|
-
<!-- spellchecker: enable -->
|
5207
5387
|
|
5208
5388
|
### Configuration
|
5209
5389
|
|
5210
|
-
The preview generator
|
5390
|
+
The preview generator should be executed as a non-user.
|
5391
|
+
When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
5211
5392
|
|
5212
|
-
Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt).
|
5393
|
+
Like any `ascli` commands, parameters can be passed on command line or using a configuration [option preset](#lprt).
|
5394
|
+
The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
5213
5395
|
|
5214
|
-
The `xfer` user has a special protected shell: `aspshell`, so changing identity
|
5396
|
+
The `xfer` user has a special protected shell: `aspshell`, so in order to update the configuration, and when changing identity, specify an alternate shell.
|
5397
|
+
E.g.:
|
5215
5398
|
|
5216
5399
|
```bash
|
5217
5400
|
su -s /bin/bash - xfer
|
@@ -5232,11 +5415,17 @@ ascli -Ppreviewconf node browse /
|
|
5232
5415
|
|
5233
5416
|
This shall list the contents of the storage root of the access key.
|
5234
5417
|
|
5418
|
+
### Options for generated files
|
5419
|
+
|
5420
|
+
When generating preview files, some options are provided by default.
|
5421
|
+
Some values for the options can be modified on command line.
|
5422
|
+
For video preview, the whole set of options can be overridden with option `reencode_ffmpeg`: it is a Hash with two keys: `in` and `out`, each is an array of strings with the native options to `ffmpeg`.
|
5423
|
+
|
5235
5424
|
### Execution
|
5236
5425
|
|
5237
5426
|
The tool intentionally supports only a **one shot** mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
5238
5427
|
It needs to be run on a regular basis to create or update preview files.
|
5239
|
-
For that use your best reliable scheduler, see [
|
5428
|
+
For that use your best reliable scheduler, see [Scheduler](#scheduler).
|
5240
5429
|
|
5241
5430
|
Typically, for **Access key** access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
|
5242
5431
|
|
@@ -5254,36 +5443,15 @@ On subsequent run it reads this file and check that previews are generated for t
|
|
5254
5443
|
|
5255
5444
|
### Configuration for Execution in scheduler
|
5256
5445
|
|
5257
|
-
|
5258
|
-
Adapt the scripts to your own needs.
|
5259
|
-
|
5260
|
-
We assume here that a configuration preset was created as shown previously.
|
5446
|
+
Details are provided in section [Scheduler](#scheduler).
|
5261
5447
|
|
5262
|
-
|
5448
|
+
Shorter commands can be specified if a configuration preset was created as shown previously.
|
5263
5449
|
|
5264
|
-
|
5450
|
+
For example the timeout value can be differentiated depending on the option: event versus scan:
|
5265
5451
|
|
5266
|
-
<!-- spellchecker: disable -->
|
5267
5452
|
```bash
|
5268
|
-
#!/bin/bash
|
5269
|
-
# set a timeout protection, just in case
|
5270
5453
|
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
5271
|
-
. /etc/profile.d/rvm.sh
|
5272
|
-
rvm use 2.6 --quiet
|
5273
|
-
exec timeout ${tmout} ascli "${@}"
|
5274
5454
|
```
|
5275
|
-
<!-- spellchecker: enable -->
|
5276
|
-
|
5277
|
-
Here the cronjob is created for user `xfer`.
|
5278
|
-
|
5279
|
-
```bash
|
5280
|
-
crontab<<EOF
|
5281
|
-
0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
|
5282
|
-
2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
|
5283
|
-
EOF
|
5284
|
-
```
|
5285
|
-
|
5286
|
-
> **Note:** The logging options are kept here in the cronfile instead of conf file to allow execution on command line with output on command line.
|
5287
5455
|
|
5288
5456
|
### Candidate detection for creation or update (or deletion)
|
5289
5457
|
|
@@ -5325,8 +5493,8 @@ For instance to filter out files beginning with `._` do:
|
|
5325
5493
|
|
5326
5494
|
Two types of preview can be generated:
|
5327
5495
|
|
5328
|
-
- png
|
5329
|
-
- mp4
|
5496
|
+
- `png`: thumbnail
|
5497
|
+
- `mp4`: video preview (only for video)
|
5330
5498
|
|
5331
5499
|
Use option `skip_format` to skip generation of a format.
|
5332
5500
|
|
@@ -5391,7 +5559,7 @@ yum install shared-mime-info
|
|
5391
5559
|
brew install shared-mime-info
|
5392
5560
|
```
|
5393
5561
|
|
5394
|
-
###
|
5562
|
+
### Generation: Read source files and write preview
|
5395
5563
|
|
5396
5564
|
Standard open source tools are used to create thumbnails and video previews.
|
5397
5565
|
Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
|
@@ -5406,7 +5574,7 @@ If the preview generator does not have access to files on the file system (it is
|
|
5406
5574
|
|
5407
5575
|
```bash
|
5408
5576
|
preview check --skip-types=office
|
5409
|
-
preview
|
5577
|
+
preview scan --scan-id=1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
5410
5578
|
preview scan --skip-types=office --log-level=info
|
5411
5579
|
preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
|
5412
5580
|
preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
|
@@ -5421,10 +5589,11 @@ preview trevents --once-only=yes --skip-types=office --log-level=info
|
|
5421
5589
|
|
5422
5590
|
## SMTP for email notifications
|
5423
5591
|
|
5424
|
-
|
5592
|
+
`ascli` can send email, for that setup SMTP configuration. This is done with option `smtp`.
|
5425
5593
|
|
5426
5594
|
The `smtp` option is a hash table (extended value) with the following fields:
|
5427
5595
|
|
5596
|
+
<!-- markdownlint-disable MD034 -->
|
5428
5597
|
| field | default | example | description |
|
5429
5598
|
|--------------|---------------------|----------------------------|----------------------------------|
|
5430
5599
|
| `server` | - | smtp.gmail.com | SMTP server address |
|
@@ -5435,6 +5604,7 @@ The `smtp` option is a hash table (extended value) with the following fields:
|
|
5435
5604
|
| `password` | - | my_password_here | password for above username |
|
5436
5605
|
| `from_email` | username if defined | johnny@example.com | address used if receiver replies |
|
5437
5606
|
| `from_name` | same as email | John Wayne | display name of sender |
|
5607
|
+
<!-- markdownlint-enable MD034 -->
|
5438
5608
|
|
5439
5609
|
### Example of configuration
|
5440
5610
|
|
@@ -5548,7 +5718,7 @@ There are special "extended" [*transfer-spec*](#transferspec) parameters support
|
|
5548
5718
|
| required additional components to `ascp` | Ruby<br/>Aspera | - | library<br/>(headers) | daemon |
|
5549
5719
|
| startup | JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn) | command line arguments | API | daemon |
|
5550
5720
|
| events | JSON on stdout | none by default<br/>or need to open management port<br/>and proprietary text syntax | callback | callback |
|
5551
|
-
| platforms | any with
|
5721
|
+
| platforms | any with Ruby and `ascp` | any with `ascp` (and SDK if compiled) | any with `ascp` | any with `ascp` and transfer daemon |
|
5552
5722
|
|
5553
5723
|
### Simple session
|
5554
5724
|
|
@@ -5646,7 +5816,7 @@ Interesting `ascp` features are found in its arguments: (see `ascp` manual):
|
|
5646
5816
|
>
|
5647
5817
|
> **Note:** Most, but not all, native `ascp` arguments are available as standard [*transfer-spec*](#transferspec) parameters.
|
5648
5818
|
>
|
5649
|
-
> **Note:** Only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided option `transfer_info`
|
5819
|
+
> **Note:** Only for the [`direct`](#agt_direct) transfer agent (not others, like connect or node), native `ascp` arguments can be provided with parameter `ascp_args` of option `transfer_info` .
|
5650
5820
|
|
5651
5821
|
#### server side and configuration
|
5652
5822
|
|
@@ -5659,7 +5829,7 @@ Virtually any transfer on a "repository" on a regular basis might emulate a hot
|
|
5659
5829
|
#### Scheduling
|
5660
5830
|
|
5661
5831
|
Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc...
|
5662
|
-
Refer to section [
|
5832
|
+
Refer to section [Scheduler](#scheduler). (on use of option `lock_port`)
|
5663
5833
|
|
5664
5834
|
### Example: upload hot folder
|
5665
5835
|
|
@@ -5739,30 +5909,7 @@ Main components:
|
|
5739
5909
|
- `Aspera::Fasp`: starting and monitoring transfers. It can be considered as a FASPManager class for Ruby.
|
5740
5910
|
- `Aspera::Cli`: `ascli`.
|
5741
5911
|
|
5742
|
-
|
5743
|
-
|
5744
|
-
```bash
|
5745
|
-
ascli config gem path
|
5746
|
-
```
|
5747
|
-
|
5748
|
-
```bash
|
5749
|
-
cat $(ascli config gem path)/../examples/transfer.rb
|
5750
|
-
```
|
5751
|
-
|
5752
|
-
This sample code shows some example of use of the API as well as REST API.
|
5753
|
-
Note: although nice, it's probably a good idea to use RestClient for REST.
|
5754
|
-
|
5755
|
-
Example of use of the API of Aspera on Cloud:
|
5756
|
-
|
5757
|
-
```ruby
|
5758
|
-
require 'aspera/aoc'
|
5759
|
-
|
5760
|
-
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/ascli/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
5761
|
-
|
5762
|
-
aoc.read('self')
|
5763
|
-
```
|
5764
|
-
|
5765
|
-
<https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
|
5912
|
+
Working examples can be found in repo: <https://github.com/laurent-martin/aspera-api-examples> in Ruby examples.
|
5766
5913
|
|
5767
5914
|
## Changes (Release notes)
|
5768
5915
|
|
@@ -5790,6 +5937,13 @@ So, it evolved into `ascli`:
|
|
5790
5937
|
Over the time, a supported command line tool `aspera` was developed in C++, it was later on deprecated.
|
5791
5938
|
It had the advantage of being relatively easy to installed, as a single executable (well, still using `ascp`), but it was too limited IMHO, and lacked a lot of the features of this CLI.
|
5792
5939
|
|
5940
|
+
Enjoy a coffee on me:
|
5941
|
+
|
5942
|
+
```bash
|
5943
|
+
ascli conf coffee
|
5944
|
+
ascli conf coffee --ui=text
|
5945
|
+
```
|
5946
|
+
|
5793
5947
|
## Common problems
|
5794
5948
|
|
5795
5949
|
### Error: "Remote host is not who we expected"
|
@@ -5810,7 +5964,7 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
5810
5964
|
|
5811
5965
|
Some Ruby gems dependencies require compilation of native parts (C).
|
5812
5966
|
This also requires Ruby header files.
|
5813
|
-
If Ruby was installed as a Linux Packages, then also install
|
5967
|
+
If Ruby was installed as a Linux Packages, then also install Ruby development package:
|
5814
5968
|
`ruby-dev` ir `ruby-devel`, depending on distribution.
|
5815
5969
|
|
5816
5970
|
### ED255519 key not supported
|
@@ -5823,5 +5977,5 @@ Without this deactivation, if such key was present the following error was gener
|
|
5823
5977
|
OpenSSH keys only supported if ED25519 is available
|
5824
5978
|
```
|
5825
5979
|
|
5826
|
-
Which meant that you do not have
|
5980
|
+
Which meant that you do not have Ruby support for ED25519 SSH keys.
|
5827
5981
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|