aspera-cli 4.5.0 → 4.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +1063 -853
- data/docs/Makefile +3 -1
- data/docs/README.erb.md +1011 -808
- data/docs/doc_tools.rb +1 -1
- data/docs/test_env.conf +1 -0
- data/lib/aspera/aoc.rb +21 -0
- data/lib/aspera/ascmd.rb +14 -14
- data/lib/aspera/cli/formater.rb +8 -1
- data/lib/aspera/cli/main.rb +23 -21
- data/lib/aspera/cli/manager.rb +5 -1
- data/lib/aspera/cli/plugins/aoc.rb +94 -69
- data/lib/aspera/cli/plugins/config.rb +99 -49
- data/lib/aspera/cli/plugins/node.rb +55 -57
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/environment.rb +1 -1
- data/lib/aspera/faspex_gw.rb +2 -1
- data/lib/aspera/log.rb +1 -1
- data/lib/aspera/rest.rb +1 -0
- metadata +6 -3
data/README.md
CHANGED
@@ -1,9 +1,10 @@
|
|
1
|
+
# Command Line Interface for IBM Aspera products
|
2
|
+
|
1
3
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
2
|
-
<font size="+12"><center>`ascli` : Command Line Interface for IBM Aspera products</center></font>
|
3
4
|
|
4
|
-
Version : 4.
|
5
|
+
Version : 4.6.0
|
5
6
|
|
6
|
-
|
7
|
+
Laurent/2016-2022
|
7
8
|
|
8
9
|
This gem provides the `ascli` Command Line Interface to IBM Aspera software.
|
9
10
|
|
@@ -17,7 +18,7 @@ Required Ruby version: > 2.4
|
|
17
18
|
|
18
19
|
[Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
|
19
20
|
|
20
|
-
|
21
|
+
## <a id="when_to_use"></a>When to use and when not to use
|
21
22
|
|
22
23
|
`ascli` is designed to be used as a command line tool to:
|
23
24
|
|
@@ -46,7 +47,7 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
|
|
46
47
|
|
47
48
|
For scripting and ad'hoc command line operations, `ascli` is perfect.
|
48
49
|
|
49
|
-
|
50
|
+
## <a id="parsing"></a>Notations, Shell and Command line parsing
|
50
51
|
|
51
52
|
In examples, command line operations are shown using a shell such: `bash` or `zsh`.
|
52
53
|
|
@@ -63,18 +64,18 @@ On Windows, `cmd.exe` is typically used. Windows process creation does not recei
|
|
63
64
|
|
64
65
|
In case of doubt of argument values after parsing test like this:
|
65
66
|
|
66
|
-
```
|
67
|
+
```bash
|
67
68
|
ascli conf echo "Hello World" arg2 3
|
68
69
|
```
|
69
70
|
|
70
|
-
```
|
71
|
+
```bash
|
71
72
|
"Hello World"
|
72
73
|
ERROR: Argument: unprocessed values: ["arg2", "3"]
|
73
74
|
```
|
74
75
|
|
75
76
|
`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (`ascli`), next command line arguments are shown in the error message.
|
76
77
|
|
77
|
-
|
78
|
+
## Quick Start
|
78
79
|
|
79
80
|
This section guides you from installation, first use and advanced use.
|
80
81
|
|
@@ -82,15 +83,15 @@ First, follow the section: [Installation](#installation) (Ruby, Gem, FASP) to st
|
|
82
83
|
|
83
84
|
Once the gem is installed, `ascli` shall be accessible:
|
84
85
|
|
85
|
-
```
|
86
|
+
```bash
|
86
87
|
ascli --version
|
87
88
|
```
|
88
89
|
|
89
|
-
```
|
90
|
-
4.
|
90
|
+
```bash
|
91
|
+
4.6.0
|
91
92
|
```
|
92
93
|
|
93
|
-
|
94
|
+
### First use
|
94
95
|
|
95
96
|
Once installation is completed, you can proceed to the first use with a demo server:
|
96
97
|
|
@@ -98,15 +99,15 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
|
|
98
99
|
|
99
100
|
To test with Aspera demo transfer server, setup the environment and then test:
|
100
101
|
|
101
|
-
```
|
102
|
+
```bash
|
102
103
|
ascli config initdemo
|
103
104
|
```
|
104
105
|
|
105
|
-
```
|
106
|
+
```bash
|
106
107
|
ascli server browse /
|
107
108
|
```
|
108
109
|
|
109
|
-
```
|
110
|
+
```output
|
110
111
|
:............:...........:......:........:...........................:.......................:
|
111
112
|
: zmode : zuid : zgid : size : mtime : name :
|
112
113
|
:............:...........:......:........:...........................:.......................:
|
@@ -124,27 +125,27 @@ If you want to use `ascli` with another server, and in order to make further cal
|
|
124
125
|
* list files in a folder
|
125
126
|
* download a file
|
126
127
|
|
127
|
-
```
|
128
|
+
```bash
|
128
129
|
ascli config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
|
129
130
|
```
|
130
131
|
|
131
|
-
```
|
132
|
+
```output
|
132
133
|
updated: myserver
|
133
134
|
```
|
134
135
|
|
135
|
-
```
|
136
|
+
```bash
|
136
137
|
ascli config preset set default server myserver
|
137
138
|
```
|
138
139
|
|
139
|
-
```
|
140
|
+
```output
|
140
141
|
updated: default→server to myserver
|
141
142
|
```
|
142
143
|
|
143
|
-
```
|
144
|
+
```bash
|
144
145
|
ascli server browse /aspera-test-dir-large
|
145
146
|
```
|
146
147
|
|
147
|
-
```
|
148
|
+
```output
|
148
149
|
:............:...........:......:..............:...........................:............................:
|
149
150
|
: zmode : zuid : zgid : size : mtime : name :
|
150
151
|
:............:...........:......:..............:...........................:............................:
|
@@ -163,22 +164,22 @@ ascli server browse /aspera-test-dir-large
|
|
163
164
|
:............:...........:......:..............:...........................:............................:
|
164
165
|
```
|
165
166
|
|
166
|
-
```
|
167
|
+
```bash
|
167
168
|
ascli server download /aspera-test-dir-large/200MB
|
168
169
|
```
|
169
170
|
|
170
|
-
```
|
171
|
+
```output
|
171
172
|
Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
|
172
173
|
complete
|
173
174
|
```
|
174
175
|
|
175
|
-
|
176
|
+
### Going further
|
176
177
|
|
177
178
|
Get familiar with configuration, options, commands : [Command Line Interface](#cli).
|
178
179
|
|
179
180
|
Then, follow the section relative to the product you want to interact with ( Aspera on Cloud, Faspex, ...) : [Application Plugins](plugins)
|
180
181
|
|
181
|
-
|
182
|
+
## <a id="installation"></a>Installation
|
182
183
|
|
183
184
|
It is possible to install *either* directly on the host operating system (Linux, Windows, macOS) or as a docker container.
|
184
185
|
|
@@ -192,7 +193,7 @@ The following sections provide information on the various installation methods.
|
|
192
193
|
|
193
194
|
An internet connection is required for the installation. If you don't have internet for the installation, refer to section [Installation without internet access](#offline_install).
|
194
195
|
|
195
|
-
|
196
|
+
### Docker container
|
196
197
|
|
197
198
|
Use this method only if you know what you do, else use the standard recommended method as described here above.
|
198
199
|
|
@@ -202,23 +203,23 @@ The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.
|
|
202
203
|
|
203
204
|
Ensure that you have Docker installed.
|
204
205
|
|
205
|
-
```
|
206
|
+
```bash
|
206
207
|
docker --version
|
207
208
|
```
|
208
209
|
|
209
210
|
Download the wrapping script:
|
210
211
|
|
211
|
-
```
|
212
|
+
```bash
|
212
213
|
curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
|
213
214
|
```
|
214
215
|
|
215
|
-
```
|
216
|
+
```bash
|
216
217
|
chmod a+x ascli
|
217
218
|
```
|
218
219
|
|
219
220
|
Install the container image:
|
220
221
|
|
221
|
-
```
|
222
|
+
```bash
|
222
223
|
./ascli install
|
223
224
|
```
|
224
225
|
|
@@ -231,7 +232,7 @@ The wrapping script maps the container folder `/usr/src/app/config` to configura
|
|
231
232
|
To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
|
232
233
|
To add local storage as a volume edit the script: `ascli` and add a `--volume` stanza.
|
233
234
|
|
234
|
-
|
235
|
+
### <a id="ruby"></a>Ruby
|
235
236
|
|
236
237
|
Use this method to install on the native host.
|
237
238
|
|
@@ -247,7 +248,7 @@ The recommended installation method is `rvm` for systems with "bash-like" shell
|
|
247
248
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
248
249
|
If you have a simpler better way to install Ruby version > 2.4 : use it !
|
249
250
|
|
250
|
-
|
251
|
+
#### Generic: RVM: single user installation (not root)
|
251
252
|
|
252
253
|
Use this method which provides more flexibility.
|
253
254
|
|
@@ -255,44 +256,44 @@ Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
|
|
255
256
|
|
256
257
|
Install the 2 keys
|
257
258
|
|
258
|
-
```
|
259
|
+
```bash
|
259
260
|
gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
|
260
261
|
```
|
261
262
|
|
262
263
|
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
263
264
|
|
264
|
-
```
|
265
|
+
```bash
|
265
266
|
\curl -sSL https://get.rvm.io | bash -s stable
|
266
267
|
```
|
267
268
|
|
268
269
|
If you keep the same terminal (not needed if re-login):
|
269
270
|
|
270
|
-
```
|
271
|
+
```bash
|
271
272
|
source ~/.rvm/scripts/rvm
|
272
273
|
```
|
273
274
|
|
274
275
|
It is advised to get one of the pre-compiled ruby version, you can list with:
|
275
276
|
|
276
|
-
```
|
277
|
+
```bash
|
277
278
|
rvm list --remote
|
278
279
|
```
|
279
280
|
|
280
281
|
Install the chosen pre-compiled Ruby version:
|
281
282
|
|
282
|
-
```
|
283
|
+
```bash
|
283
284
|
rvm install 2.7.2 --binary
|
284
285
|
```
|
285
286
|
|
286
287
|
Ruby is now installed for the user, go on to Gem installation.
|
287
288
|
|
288
|
-
|
289
|
+
#### Generic: RVM: global installation (as root)
|
289
290
|
|
290
291
|
Follow the same method as single user install, but execute as "root".
|
291
292
|
|
292
293
|
As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
|
293
294
|
One can install in another location with :
|
294
295
|
|
295
|
-
```
|
296
|
+
```bash
|
296
297
|
curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
297
298
|
```
|
298
299
|
|
@@ -300,15 +301,15 @@ As root, make sure this will not collide with other application using Ruby (e.g.
|
|
300
301
|
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
301
302
|
To activate ruby (and ascli) later, source it:
|
302
303
|
|
303
|
-
```
|
304
|
+
```bash
|
304
305
|
source /etc/profile.d/rvm.sh.ok
|
305
306
|
```
|
306
307
|
|
307
|
-
```
|
308
|
+
```bash
|
308
309
|
rvm version
|
309
310
|
```
|
310
311
|
|
311
|
-
|
312
|
+
#### Windows: Installer
|
312
313
|
|
313
314
|
Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
|
314
315
|
|
@@ -316,48 +317,53 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
|
|
316
317
|
* Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
|
317
318
|
* At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
|
318
319
|
|
319
|
-
|
320
|
+
#### macOS: pre-installed or `brew`
|
320
321
|
|
321
322
|
macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install aspera-cli using `sudo` :
|
322
323
|
|
323
|
-
```
|
324
|
+
```bash
|
324
325
|
sudo gem install aspera-cli
|
325
326
|
```
|
326
327
|
|
327
328
|
Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
|
328
329
|
|
329
|
-
```
|
330
|
+
```bash
|
330
331
|
brew install ruby
|
331
332
|
```
|
332
333
|
|
333
|
-
|
334
|
+
#### Linux: package
|
334
335
|
|
335
336
|
If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
|
336
337
|
|
337
338
|
Example:
|
338
339
|
|
339
|
-
```
|
340
|
+
```bash
|
340
341
|
yum install -y ruby rubygems ruby-json
|
341
342
|
```
|
342
343
|
|
343
344
|
One can cleanup the whole yum-installed ruby environment like this to uninstall:
|
344
345
|
|
345
|
-
```
|
346
|
+
```bash
|
346
347
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
347
348
|
```
|
348
349
|
|
349
|
-
```
|
350
|
+
```bash
|
350
351
|
yum remove -y ruby ruby-libs
|
351
352
|
```
|
352
353
|
|
353
|
-
|
354
|
+
#### Other Unixes (AIX)
|
355
|
+
|
356
|
+
Ruby is sometimes made available as installable package through third party providers.
|
357
|
+
For example for AIX, one can look at:
|
358
|
+
|
359
|
+
<https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha#R>
|
354
360
|
|
355
361
|
If your Unix does not provide a pre-built ruby, you can get it using one of those
|
356
|
-
[methods](https://www.ruby-lang.org/en/documentation/installation/)
|
362
|
+
[methods](https://www.ruby-lang.org/en/documentation/installation/).
|
357
363
|
|
358
364
|
For instance to build from source, and install in `/opt/ruby` :
|
359
365
|
|
360
|
-
```
|
366
|
+
```bash
|
361
367
|
wget https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.gz
|
362
368
|
|
363
369
|
gzip -d ruby-2.7.2.tar.gz
|
@@ -375,17 +381,23 @@ make
|
|
375
381
|
make install
|
376
382
|
```
|
377
383
|
|
378
|
-
|
384
|
+
If you already have a Java JVM on your system (`java`), it is possible to use `jruby`:
|
385
|
+
|
386
|
+
<https://www.jruby.org/download>
|
387
|
+
|
388
|
+
Note that using jruby the startup time is longer than the native ruby, but transfer speed is not impacted (executed by `ascp` binary).
|
389
|
+
|
390
|
+
### <a id="the_gem"></a>`aspera-cli` gem
|
379
391
|
|
380
392
|
Once you have Ruby and rights to install gems: Install the gem and its dependencies:
|
381
393
|
|
382
|
-
```
|
394
|
+
```bash
|
383
395
|
gem install aspera-cli
|
384
396
|
```
|
385
397
|
|
386
398
|
To upgrade to the latest version:
|
387
399
|
|
388
|
-
```
|
400
|
+
```bash
|
389
401
|
gem update aspera-cli
|
390
402
|
```
|
391
403
|
|
@@ -393,11 +405,11 @@ gem update aspera-cli
|
|
393
405
|
|
394
406
|
To check manually:
|
395
407
|
|
396
|
-
```
|
408
|
+
```bash
|
397
409
|
ascli conf check_update
|
398
410
|
```
|
399
411
|
|
400
|
-
|
412
|
+
### <a id="fasp_prot"></a>FASP Protocol
|
401
413
|
|
402
414
|
Most file transfers will be done using the FASP protocol, using `ascp`.
|
403
415
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
@@ -407,17 +419,17 @@ Only two additional files are required to perform an Aspera Transfer, which are
|
|
407
419
|
|
408
420
|
This can be installed either be installing an Aspera transfer software, or using an embedded command:
|
409
421
|
|
410
|
-
```
|
422
|
+
```bash
|
411
423
|
ascli conf ascp install
|
412
424
|
```
|
413
425
|
|
414
426
|
If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
|
415
427
|
|
416
|
-
```
|
428
|
+
```bash
|
417
429
|
curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
418
430
|
```
|
419
431
|
|
420
|
-
```
|
432
|
+
```bash
|
421
433
|
ascli conf ascp install --sdk-url=file:///SDK.zip
|
422
434
|
```
|
423
435
|
|
@@ -440,7 +452,7 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
|
|
440
452
|
Several methods are provided to start a transfer.
|
441
453
|
Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
|
442
454
|
|
443
|
-
|
455
|
+
### <a id="offline_install"></a>Installation in air gapped environment
|
444
456
|
|
445
457
|
Note that currently no pre-packaged version exist yet.
|
446
458
|
A method to build one is provided here:
|
@@ -450,19 +462,19 @@ The procedure:
|
|
450
462
|
* Follow the non-root installation procedure with RVM, including gem
|
451
463
|
* Archive (zip, tar) the main RVM folder (includes ascli):
|
452
464
|
|
453
|
-
```
|
465
|
+
```bash
|
454
466
|
cd $HOME && tar zcvf rvm-ascli.tgz .rvm
|
455
467
|
```
|
456
468
|
|
457
469
|
* Get the Aspera SDK.
|
458
470
|
|
459
|
-
```
|
471
|
+
```bash
|
460
472
|
ascli conf --show-config --fields=sdk_url
|
461
473
|
```
|
462
474
|
|
463
475
|
* Download the SDK archive from that URL.
|
464
476
|
|
465
|
-
```
|
477
|
+
```bash
|
466
478
|
curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
467
479
|
```
|
468
480
|
|
@@ -470,7 +482,7 @@ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
|
470
482
|
|
471
483
|
* On target system
|
472
484
|
|
473
|
-
```
|
485
|
+
```bash
|
474
486
|
cd $HOME
|
475
487
|
|
476
488
|
tar zxvf rvm-ascli.tgz
|
@@ -482,11 +494,11 @@ ascli conf ascp install --sdk-url=file:///SDK.zip
|
|
482
494
|
|
483
495
|
* Add those lines to shell init (`.profile`)
|
484
496
|
|
485
|
-
```
|
497
|
+
```bash
|
486
498
|
source ~/.rvm/scripts/rvm
|
487
499
|
```
|
488
500
|
|
489
|
-
|
501
|
+
## <a id="cli"></a>Command Line Interface: `ascli`
|
490
502
|
|
491
503
|
The `aspera-cli` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
492
504
|
|
@@ -514,7 +526,7 @@ The `aspera-cli` Gem provides a command line interface (CLI) which interacts wit
|
|
514
526
|
|
515
527
|
Basic usage is displayed by executing:
|
516
528
|
|
517
|
-
```
|
529
|
+
```bash
|
518
530
|
ascli -h
|
519
531
|
```
|
520
532
|
|
@@ -522,26 +534,26 @@ Refer to sections: [Usage](#usage) and [Sample Commands](#commands).
|
|
522
534
|
|
523
535
|
Not all `ascli` features are fully documented here, the user may explore commands on the command line.
|
524
536
|
|
525
|
-
|
537
|
+
### Arguments : Commands and options
|
526
538
|
|
527
539
|
Arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
|
528
540
|
|
529
|
-
There are two types of arguments: Commands and Options. Example :
|
541
|
+
There are two types of command line arguments: Commands and Options. Example :
|
530
542
|
|
531
|
-
```
|
532
|
-
ascli command --option-name=VAL1 VAL2
|
543
|
+
```bash
|
544
|
+
ascli command subcommand --option-name=VAL1 VAL2
|
533
545
|
```
|
534
546
|
|
535
|
-
* executes _command_: `command`
|
547
|
+
* executes _command_: `command subcommand`
|
536
548
|
* with one _option_: `option_name`
|
537
|
-
* this option
|
549
|
+
* this option is given a _value_ of: `VAL1`
|
538
550
|
* the command has one additional _argument_: `VAL2`
|
539
551
|
|
540
552
|
When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `ascli conf ov` is the same as `ascli config overview`.
|
541
553
|
|
542
554
|
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
|
543
555
|
|
544
|
-
|
556
|
+
#### Options
|
545
557
|
|
546
558
|
All options, e.g. `--log-level=debug`, are command line arguments that:
|
547
559
|
|
@@ -556,15 +568,15 @@ Exceptions:
|
|
556
568
|
* some options (flags) don't take a value, e.g. `-r`
|
557
569
|
* the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
|
558
570
|
|
559
|
-
```
|
571
|
+
```bash
|
560
572
|
ascli config echo -- --sample
|
561
573
|
```
|
562
574
|
|
563
|
-
```
|
575
|
+
```bash
|
564
576
|
"--sample"
|
565
577
|
```
|
566
578
|
|
567
|
-
Note that `--sample` is taken as an argument, and not option
|
579
|
+
Note that here, `--sample` is taken as an argument, and not as an option, due to `--`.
|
568
580
|
|
569
581
|
Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
|
570
582
|
|
@@ -574,33 +586,32 @@ The value for _any_ options can come from the following locations (in this order
|
|
574
586
|
* Environment variable
|
575
587
|
* Command line
|
576
588
|
|
577
|
-
Environment variable starting with prefix: ASCLI_ are taken as option values,
|
578
|
-
e.g. `ASCLI_OPTION_NAME` is for `--option-name`.
|
589
|
+
Environment variable starting with prefix: ASCLI_ are taken as option values, e.g. `ASCLI_OPTION_NAME` is for `--option-name`.
|
579
590
|
|
580
591
|
Options values can be displayed for a given command by providing the `--show-config` option: `ascli node --show-config`
|
581
592
|
|
582
|
-
|
593
|
+
#### Commands and Arguments
|
583
594
|
|
584
595
|
Command line arguments that are not options are either commands or arguments. If an argument must begin with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
|
585
596
|
|
586
|
-
|
597
|
+
### Interactive Input
|
587
598
|
|
588
599
|
Some options and parameters are mandatory and other optional. By default, the tool will ask for missing mandatory options or parameters for interactive execution.
|
589
600
|
|
590
601
|
The behavior can be controlled with:
|
591
602
|
|
592
603
|
* --interactive=<yes|no> (default=yes if STDIN is a terminal, else no)
|
593
|
-
|
594
|
-
|
604
|
+
* yes : missing mandatory parameters/options are asked to the user
|
605
|
+
* no : missing mandatory parameters/options raise an error message
|
595
606
|
* --ask-options=<yes|no> (default=no)
|
596
|
-
|
607
|
+
* optional parameters/options are asked to user
|
597
608
|
|
598
|
-
|
609
|
+
### Output
|
599
610
|
|
600
611
|
Command execution will result in output (terminal, stdout/stderr).
|
601
612
|
The information displayed depends on the action.
|
602
613
|
|
603
|
-
|
614
|
+
#### Types of output data
|
604
615
|
|
605
616
|
Depending on action, the output will contain:
|
606
617
|
|
@@ -611,15 +622,18 @@ Depending on action, the output will contain:
|
|
611
622
|
* `status` : a message
|
612
623
|
* `other_struct` : a complex structure that cannot be displayed as an array
|
613
624
|
|
614
|
-
|
625
|
+
#### Format of output
|
615
626
|
|
616
627
|
By default, result of type single_object and object_list are displayed using format `table`.
|
617
628
|
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
618
629
|
|
619
630
|
In a table format, when displaying "objects" (single, or list), by default, sub object are
|
620
|
-
|
621
|
-
field: "user" and value is the sub hash table.
|
622
|
-
filter fields by "dotted" field name.
|
631
|
+
flattened (option `flat_hash`). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name.
|
632
|
+
Setting `flat_hash` to `false` will only display one field: "user" and value is the sub hash table.
|
633
|
+
When in flatten mode, it is possible to filter fields by "dotted" field name.
|
634
|
+
|
635
|
+
Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
|
636
|
+
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
623
637
|
|
624
638
|
The style of output can be set using the `format` parameter, supporting:
|
625
639
|
|
@@ -630,26 +644,26 @@ The style of output can be set using the `format` parameter, supporting:
|
|
630
644
|
* `yaml` : YAML
|
631
645
|
* `csv` : Comma Separated Values
|
632
646
|
|
633
|
-
|
647
|
+
#### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
|
634
648
|
|
635
649
|
Table output can be filtered using the `select` parameter. Example:
|
636
650
|
|
637
|
-
```
|
651
|
+
```javascript
|
638
652
|
ascli aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
639
653
|
```
|
640
654
|
|
641
|
-
```
|
655
|
+
```output
|
642
656
|
:...............................:..................................:...........:
|
643
657
|
: name : email : ats_admin :
|
644
658
|
:...............................:..................................:...........:
|
645
|
-
: John
|
659
|
+
: John Curtis : john@example.com : true :
|
646
660
|
: Laurent Martin : laurent@example.com : true :
|
647
661
|
:...............................:..................................:...........:
|
648
662
|
```
|
649
663
|
|
650
664
|
Note that `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
|
651
665
|
|
652
|
-
|
666
|
+
#### Verbosity of output
|
653
667
|
|
654
668
|
Output messages are categorized in 3 types:
|
655
669
|
|
@@ -663,7 +677,7 @@ The option `display` controls the level of output:
|
|
663
677
|
* `data` display `data` and `error` messages
|
664
678
|
* `error` display only error messages.
|
665
679
|
|
666
|
-
|
680
|
+
#### Selection of output object properties
|
667
681
|
|
668
682
|
By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
|
669
683
|
|
@@ -674,13 +688,13 @@ By default, a table output will display one line per entry, and columns for each
|
|
674
688
|
* +a,b,c : add selected properties to the default selection.
|
675
689
|
* -a,b,c : remove selected properties from the default selection.
|
676
690
|
|
677
|
-
|
691
|
+
### <a id="extended"></a>Extended Value Syntax
|
678
692
|
|
679
693
|
Usually, values of options and arguments are specified by a simple string. But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
|
680
694
|
|
681
695
|
The extended value syntax is:
|
682
696
|
|
683
|
-
```
|
697
|
+
```bash
|
684
698
|
<0 or more decoders><0 or 1 reader><nothing or some text value>
|
685
699
|
```
|
686
700
|
|
@@ -689,11 +703,11 @@ The difference between reader and decoder is order and ordinality. Both act like
|
|
689
703
|
The following "readers" are supported (returns value in []):
|
690
704
|
|
691
705
|
* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
|
692
|
-
* @file:PATH : [String] read value from a file (prefix
|
693
|
-
* @path:PATH : [String] performs path expansion (prefix
|
706
|
+
* @file:PATH : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
707
|
+
* @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
694
708
|
* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
695
709
|
* @stdin: : [String] read from stdin (no value on right)
|
696
|
-
* @preset:NAME : [Hash] get whole option preset value by name
|
710
|
+
* @preset:NAME : [Hash] get whole option preset value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
|
697
711
|
|
698
712
|
In addition it is possible to decode a value, using one or multiple decoders :
|
699
713
|
|
@@ -710,33 +724,33 @@ To display the result of an extended value, use the `config echo` command.
|
|
710
724
|
|
711
725
|
Example: read the content of the specified file, then, base64 decode, then unzip:
|
712
726
|
|
713
|
-
```
|
727
|
+
```bash
|
714
728
|
ascli config echo @zlib:@base64:@file:myfile.dat
|
715
729
|
```
|
716
730
|
|
717
731
|
Example: create a value as a hash, with one key and the value is read from a file:
|
718
732
|
|
719
|
-
```
|
733
|
+
```bash
|
720
734
|
ascli config echo @ruby:'{"token_verification_key"=>File.read("pubkey.txt")}'
|
721
735
|
```
|
722
736
|
|
723
737
|
Example: read a csv file and create a list of hash for bulk provisioning:
|
724
738
|
|
725
|
-
```
|
739
|
+
```bash
|
726
740
|
cat test.csv
|
727
741
|
```
|
728
742
|
|
729
|
-
```
|
743
|
+
```bash
|
730
744
|
name,email
|
731
745
|
lolo,laurent@example.com
|
732
746
|
toto,titi@tutu.tata
|
733
747
|
```
|
734
748
|
|
735
|
-
```
|
749
|
+
```bash
|
736
750
|
ascli config echo @csvt:@file:test.csv
|
737
751
|
```
|
738
752
|
|
739
|
-
```
|
753
|
+
```output
|
740
754
|
:......:.....................:
|
741
755
|
: name : email :
|
742
756
|
:......:.....................:
|
@@ -747,17 +761,17 @@ ascli config echo @csvt:@file:test.csv
|
|
747
761
|
|
748
762
|
Example: create a hash and include values from preset named "config" of config file in this hash
|
749
763
|
|
750
|
-
```
|
764
|
+
```javascript
|
751
765
|
ascli config echo @incps:@json:'{"hello":true,"incps":["config"]}'
|
752
766
|
```
|
753
767
|
|
754
|
-
```
|
768
|
+
```bash
|
755
769
|
{"version"=>"0.9", "hello"=>true}
|
756
770
|
```
|
757
771
|
|
758
772
|
Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
|
759
773
|
|
760
|
-
|
774
|
+
### <a id="native"></a>Structured Value
|
761
775
|
|
762
776
|
Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
|
763
777
|
|
@@ -768,7 +782,7 @@ A convenient way to specify a _Structured Value_ is to use the `@json:` decoder,
|
|
768
782
|
|
769
783
|
It is also possible to provide a _Structured Value_ in a file using `@json:@file:<path>`
|
770
784
|
|
771
|
-
|
785
|
+
### <a id="conffolder"></a>Configuration and Persistency Folder
|
772
786
|
|
773
787
|
`ascli` configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/ascli`.
|
774
788
|
|
@@ -777,11 +791,11 @@ It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMED
|
|
777
791
|
|
778
792
|
The main folder can be displayed using :
|
779
793
|
|
780
|
-
```
|
794
|
+
```bash
|
781
795
|
ascli config folder
|
782
796
|
```
|
783
797
|
|
784
|
-
```
|
798
|
+
```bash
|
785
799
|
/Users/kenji/.aspera/ascli
|
786
800
|
```
|
787
801
|
|
@@ -789,13 +803,15 @@ It can be overridden using the environment variable `ASCLI_HOME`.
|
|
789
803
|
|
790
804
|
Example (Windows):
|
791
805
|
|
792
|
-
```
|
806
|
+
```output
|
793
807
|
set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli
|
808
|
+
|
794
809
|
ascli config folder
|
810
|
+
|
795
811
|
C:\Users\Kenji\.aspera\ascli
|
796
812
|
```
|
797
813
|
|
798
|
-
|
814
|
+
### <a id="configfile"></a>Configuration file
|
799
815
|
|
800
816
|
On the first execution of `ascli`, an empty configuration file is created in the configuration folder.
|
801
817
|
Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
|
@@ -810,19 +826,19 @@ The default configuration file is: `$HOME/.aspera/ascli/config.yaml` (this can b
|
|
810
826
|
|
811
827
|
The configuration file is simply a catalog of pre-defined lists of options, called: [option presets](#lprt). Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a [option preset](#lprt) (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
|
812
828
|
|
813
|
-
|
829
|
+
#### <a id="lprt"></a>Option preset
|
814
830
|
|
815
831
|
A [option preset](#lprt) is simply a collection of parameters and their associated values in a named section in the configuration file.
|
816
832
|
|
817
833
|
A named [option preset](#lprt) can be modified directly using `ascli`, which will update the configuration file :
|
818
834
|
|
819
|
-
```
|
835
|
+
```bash
|
820
836
|
ascli config preset set|delete|show|initialize|update <option preset>
|
821
837
|
```
|
822
838
|
|
823
839
|
The command `update` allows the easy creation of [option preset](#lprt) by simply providing the options in their command line format, e.g. :
|
824
840
|
|
825
|
-
```
|
841
|
+
```bash
|
826
842
|
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
|
827
843
|
```
|
828
844
|
|
@@ -830,49 +846,49 @@ ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --u
|
|
830
846
|
|
831
847
|
The command `set` allows setting individual options in a [option preset](#lprt).
|
832
848
|
|
833
|
-
```
|
849
|
+
```bash
|
834
850
|
ascli config preset set demo_server password _demo_pass_
|
835
851
|
```
|
836
852
|
|
837
853
|
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
|
838
854
|
|
839
|
-
```
|
855
|
+
```javascript
|
840
856
|
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
|
841
857
|
```
|
842
858
|
|
843
859
|
A full terminal based overview of the configuration can be displayed using:
|
844
860
|
|
845
|
-
```
|
861
|
+
```bash
|
846
862
|
ascli config preset over
|
847
863
|
```
|
848
864
|
|
849
865
|
A list of [option preset](#lprt) can be displayed using:
|
850
866
|
|
851
|
-
```
|
867
|
+
```bash
|
852
868
|
ascli config preset list
|
853
869
|
```
|
854
870
|
|
855
871
|
A good practice is to not manually edit the configuration file and use modification commands instead.
|
856
872
|
If necessary, the configuration file can opened in a text editor with:
|
857
873
|
|
858
|
-
```
|
874
|
+
```bash
|
859
875
|
ascli config open
|
860
876
|
```
|
861
877
|
|
862
878
|
Older format for commands are still supported:
|
863
879
|
|
864
|
-
```
|
880
|
+
```bash
|
865
881
|
ascli config id <name> set|delete|show|initialize|update
|
866
882
|
ascli config over
|
867
883
|
ascli config list
|
868
884
|
```
|
869
885
|
|
870
886
|
|
871
|
-
|
887
|
+
#### <a id="lprtconf"></a>Special Option preset: config
|
872
888
|
|
873
889
|
This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
|
874
890
|
|
875
|
-
|
891
|
+
#### <a id="lprtdef"></a>Special Option preset: default
|
876
892
|
|
877
893
|
This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
|
878
894
|
|
@@ -882,13 +898,19 @@ Note that special plugin name: `config` can be associated with a preset that is
|
|
882
898
|
|
883
899
|
Operations on this preset are done using regular `config` operations:
|
884
900
|
|
885
|
-
```
|
901
|
+
```bash
|
886
902
|
ascli config preset set default _plugin_name_ _default_preset_for_plugin_
|
903
|
+
```
|
904
|
+
|
905
|
+
```bash
|
887
906
|
ascli config preset get default _plugin_name_
|
907
|
+
```
|
908
|
+
|
909
|
+
```javascript
|
888
910
|
"_default_preset_for_plugin_"
|
889
911
|
```
|
890
912
|
|
891
|
-
|
913
|
+
#### <a id="lplugconf"></a>Special Plugin: config
|
892
914
|
|
893
915
|
Plugin `config` (not to be confused with Option preset config) is used to configure `ascli` but it also contains global options.
|
894
916
|
|
@@ -896,7 +918,7 @@ When `ascli` starts, it looks for the `default` Option preset and if there is a
|
|
896
918
|
|
897
919
|
If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
|
898
920
|
|
899
|
-
|
921
|
+
#### Format of file
|
900
922
|
|
901
923
|
The configuration file is a hash in a YAML file. Example:
|
902
924
|
|
@@ -933,13 +955,13 @@ Values in the configuration also follow the [Extended Value Syntax](#extended).
|
|
933
955
|
|
934
956
|
Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
935
957
|
|
936
|
-
```
|
958
|
+
```bash
|
937
959
|
ascli config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/ascli/aocapikey"
|
938
960
|
```
|
939
961
|
|
940
962
|
This creates the [option preset](#lprt):
|
941
963
|
|
942
|
-
```
|
964
|
+
```yaml
|
943
965
|
...
|
944
966
|
my_aoc_org:
|
945
967
|
private_key: @file:"/Users/laurent/.aspera/ascli/aocapikey"
|
@@ -948,7 +970,7 @@ my_aoc_org:
|
|
948
970
|
|
949
971
|
So, the key file will be read only at execution time, but not be embedded in the configuration file.
|
950
972
|
|
951
|
-
|
973
|
+
#### Options evaluation order
|
952
974
|
|
953
975
|
Some options are global, some options are available only for some plugins. (the plugin is the first level command).
|
954
976
|
|
@@ -969,39 +991,39 @@ is an underscore. E.g. --xxx-yyy on command line gives xxx_yyy in configuration
|
|
969
991
|
|
970
992
|
The main plugin name is `config`, so it is possible to define a default [option preset](#lprt) for the main plugin with:
|
971
993
|
|
972
|
-
```
|
994
|
+
```bash
|
973
995
|
ascli config preset set cli_default interactive no
|
974
996
|
```
|
975
997
|
|
976
|
-
```
|
998
|
+
```bash
|
977
999
|
ascli config preset set default config cli_default
|
978
1000
|
```
|
979
1001
|
|
980
1002
|
A [option preset](#lprt) value can be removed with `unset`:
|
981
1003
|
|
982
|
-
```
|
1004
|
+
```bash
|
983
1005
|
ascli config preset unset cli_default interactive
|
984
1006
|
```
|
985
1007
|
|
986
1008
|
Example: Define options using command line:
|
987
1009
|
|
988
|
-
```
|
1010
|
+
```bash
|
989
1011
|
ascli -N --url=x --password=y --username=y node --show-config
|
990
1012
|
```
|
991
1013
|
|
992
1014
|
Example: Define options using a hash:
|
993
1015
|
|
994
|
-
```
|
1016
|
+
```javascript
|
995
1017
|
ascli -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
|
996
1018
|
```
|
997
1019
|
|
998
|
-
|
1020
|
+
#### Shares Examples
|
999
1021
|
|
1000
1022
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
1001
1023
|
only username/password and url are required (either on command line, or from config file).
|
1002
1024
|
Those can usually be provided on the command line:
|
1003
1025
|
|
1004
|
-
```
|
1026
|
+
```bash
|
1005
1027
|
ascli shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
|
1006
1028
|
```
|
1007
1029
|
|
@@ -1009,49 +1031,49 @@ This can also be provisioned in a config file:
|
|
1009
1031
|
|
1010
1032
|
* Build [option preset](#lprt)
|
1011
1033
|
|
1012
|
-
```
|
1034
|
+
```bash
|
1013
1035
|
ascli config preset set shares06 url https://10.25.0.6
|
1014
1036
|
ascli config preset set shares06 username john
|
1015
1037
|
ascli config preset set shares06 password 4sp3ra
|
1016
1038
|
```
|
1017
1039
|
|
1018
|
-
|
1040
|
+
This can also be done with one single command:
|
1019
1041
|
|
1020
|
-
```
|
1042
|
+
```javascript
|
1021
1043
|
ascli config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
|
1022
1044
|
```
|
1023
1045
|
|
1024
1046
|
or
|
1025
1047
|
|
1026
|
-
```
|
1048
|
+
```bash
|
1027
1049
|
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=4sp3ra
|
1028
1050
|
```
|
1029
1051
|
|
1030
1052
|
* Define this [option preset](#lprt) as the default [option preset](#lprt) for the specified plugin (`shares`)
|
1031
1053
|
|
1032
|
-
```
|
1054
|
+
```bash
|
1033
1055
|
ascli config preset set default shares shares06
|
1034
1056
|
```
|
1035
1057
|
|
1036
1058
|
* Display the content of configuration file in table format
|
1037
1059
|
|
1038
|
-
```
|
1060
|
+
```bash
|
1039
1061
|
ascli config overview
|
1040
1062
|
```
|
1041
1063
|
|
1042
1064
|
* Execute a command on the shares application using default parameters
|
1043
1065
|
|
1044
|
-
```
|
1066
|
+
```bash
|
1045
1067
|
ascli shares repo browse /
|
1046
1068
|
```
|
1047
1069
|
|
1048
|
-
|
1070
|
+
### <a id="vault"></a>Secret Vault
|
1049
1071
|
|
1050
1072
|
When a secret or password is needed, it is possible to store in the secret vault.
|
1051
1073
|
|
1052
|
-
By default the vault is defined using option `secrets
|
1074
|
+
By default the vault is defined using option `secrets`, which can be stored in the configuration file.
|
1053
1075
|
|
1054
|
-
|
1076
|
+
#### Using system keychain
|
1055
1077
|
|
1056
1078
|
Only on macOS.
|
1057
1079
|
|
@@ -1059,7 +1081,7 @@ It is possible to store secrets in macOS keychain (only read supported currently
|
|
1059
1081
|
|
1060
1082
|
Set option `secrets` to value `system` to use the default keychain or use value `system:[name]` to use a custom keychain.
|
1061
1083
|
|
1062
|
-
|
1084
|
+
#### Modern config file format: encrypted in config file
|
1063
1085
|
|
1064
1086
|
It is possible to store and use secrets encrypted.
|
1065
1087
|
For this use the `config vault` command.
|
@@ -1075,13 +1097,13 @@ Then secrets can be manipulated using commands:
|
|
1075
1097
|
|
1076
1098
|
Secrets must be uniquely identified by `url` and `username`. An optional description can be provided using option `value`.
|
1077
1099
|
|
1078
|
-
|
1100
|
+
#### Legacy config file format
|
1079
1101
|
|
1080
1102
|
The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
|
1081
1103
|
|
1082
1104
|
For example, choose a repository name, for example `my_secrets`, and populate it like this:
|
1083
1105
|
|
1084
|
-
```
|
1106
|
+
```bash
|
1085
1107
|
ascli conf id my_secrets set 'access_key1' 'secret1'
|
1086
1108
|
|
1087
1109
|
ascli conf id my_secrets set 'access_key2' 'secret2'
|
@@ -1094,7 +1116,7 @@ cli_default
|
|
1094
1116
|
Here above, one has already set a `config` global preset to preset `cli_default` (refer to earlier in documentation).
|
1095
1117
|
So the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
|
1096
1118
|
|
1097
|
-
```
|
1119
|
+
```bash
|
1098
1120
|
ascli conf id cli_default set secrets @val:@preset:my_secrets
|
1099
1121
|
```
|
1100
1122
|
|
@@ -1102,35 +1124,56 @@ A secret repository can always be selected at runtime using `--secrets=@preset:x
|
|
1102
1124
|
|
1103
1125
|
To test if a secret can be found use:
|
1104
1126
|
|
1105
|
-
```
|
1127
|
+
```bash
|
1106
1128
|
ascli conf vault get --username=access_key1
|
1107
1129
|
```
|
1108
1130
|
|
1109
|
-
|
1131
|
+
### Plugins
|
1110
1132
|
|
1111
|
-
The CLI tool uses a plugin mechanism.
|
1112
|
-
|
1133
|
+
The CLI tool uses a plugin mechanism.
|
1134
|
+
The first level command (just after `ascli` on the command line) is the name of the concerned plugin which will execute the command.
|
1135
|
+
Each plugin usually represents commands sent to a specific application.
|
1136
|
+
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
1113
1137
|
|
1114
|
-
|
1138
|
+
Available plugins can be found using command:
|
1115
1139
|
|
1140
|
+
```bash
|
1141
|
+
ascli conf plugin list
|
1116
1142
|
```
|
1117
|
-
|
1118
|
-
|
1119
|
-
|
1120
|
-
|
1121
|
-
|
1122
|
-
|
1123
|
-
|
1124
|
-
|
1125
|
-
|
1126
|
-
|
1127
|
-
|
1128
|
-
|
1129
|
-
|
1130
|
-
|
1143
|
+
|
1144
|
+
```output
|
1145
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1146
|
+
| plugin | path |
|
1147
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1148
|
+
| shares | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/shares.rb |
|
1149
|
+
| node | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/node.rb |
|
1150
|
+
...
|
1151
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1152
|
+
```
|
1153
|
+
|
1154
|
+
#### <a id="createownplugin"></a>Create your own plugin
|
1155
|
+
|
1156
|
+
By default plugins are looked-up in folders specifed by (multi-value) option `plugin_folder`:
|
1157
|
+
|
1158
|
+
```javascript
|
1159
|
+
ascli --show-config --select=@json:'{"key":"plugin_folder"}'
|
1160
|
+
```
|
1161
|
+
|
1162
|
+
You can create the skeleton of a new plugin like this:
|
1163
|
+
|
1164
|
+
```bash
|
1165
|
+
ascli conf plugin create foo .
|
1166
|
+
```
|
1167
|
+
|
1168
|
+
```output
|
1169
|
+
Created ./foo.rb
|
1170
|
+
```
|
1171
|
+
|
1172
|
+
```bash
|
1173
|
+
ascli --plugin-folder=. foo
|
1131
1174
|
```
|
1132
1175
|
|
1133
|
-
|
1176
|
+
#### <a id="plugins"></a>Plugins: Application URL and Authentication
|
1134
1177
|
|
1135
1178
|
`ascli` comes with several Aspera application plugins.
|
1136
1179
|
|
@@ -1146,7 +1189,7 @@ Those can be provided using command line, parameter set, env var, see section ab
|
|
1146
1189
|
|
1147
1190
|
Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
|
1148
1191
|
|
1149
|
-
|
1192
|
+
### Logging, Debugging
|
1150
1193
|
|
1151
1194
|
The gem is equipped with traces. By default logging level is `warn`.
|
1152
1195
|
To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or parameter in con file).
|
@@ -1156,31 +1199,39 @@ It is also possible to activate traces before initialization using env var `AS_L
|
|
1156
1199
|
By default passwords and secrets are removed from logs.
|
1157
1200
|
Use option `log_passwords` to change this behaviour.
|
1158
1201
|
|
1159
|
-
|
1202
|
+
### Learning Aspera Product APIs (REST)
|
1160
1203
|
|
1161
1204
|
This CLI uses REST APIs.
|
1162
1205
|
To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
|
1163
1206
|
|
1164
1207
|
In order to get traces of execution, use argument : `--log-level=debug`
|
1165
1208
|
|
1166
|
-
|
1209
|
+
### <a id="http_options"></a>HTTP socket parameters
|
1167
1210
|
|
1168
|
-
If the server does not provide a valid certificate, use
|
1211
|
+
If the server does not provide a valid certificate, use option: `--insecure=yes`.
|
1169
1212
|
|
1170
|
-
|
1213
|
+
Ruby HTTP socket parameters can be adjusted.
|
1171
1214
|
|
1172
|
-
|
1173
|
-
|
1215
|
+
| parameter | default |
|
1216
|
+
|----------------------|---------|
|
1217
|
+
| `read_timeout` | 60 |
|
1218
|
+
| `write_timeout` | 60 |
|
1219
|
+
| `open_timeout` | 60 |
|
1220
|
+
| `keep_alive_timeout` | 2 |
|
1174
1221
|
|
1222
|
+
Values are in set **seconds** and can be of type either integer or float.
|
1175
1223
|
Default values are the ones of Ruby.
|
1224
|
+
For details refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1225
|
+
|
1226
|
+
Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
|
1176
1227
|
|
1177
1228
|
Example:
|
1178
1229
|
|
1179
|
-
```
|
1230
|
+
```javascript
|
1180
1231
|
ascli aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
|
1181
1232
|
```
|
1182
1233
|
|
1183
|
-
|
1234
|
+
### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
|
1184
1235
|
|
1185
1236
|
Some actions may require the use of a graphical tool:
|
1186
1237
|
|
@@ -1193,30 +1244,30 @@ It is also possible to force the graphical mode with option --ui :
|
|
1193
1244
|
* `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
1194
1245
|
* `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
1195
1246
|
|
1196
|
-
|
1247
|
+
### HTTP proxy for REST
|
1197
1248
|
|
1198
1249
|
To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honored by Ruby when calling REST APIs.
|
1199
1250
|
|
1200
|
-
|
1251
|
+
### <a id="certificates"></a>SSL CA certificate bundle
|
1201
1252
|
|
1202
1253
|
`ascli` uses ruby `openssl` gem, which uses the `openssl` library, so certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..). The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
|
1203
1254
|
|
1204
1255
|
`ascp` also needs to validate certificates when using WSS. By default, `ascp` uses primarily certificates from hard coded path (e.g. on macOS: `/Library/Aspera/ssl`). `ascli` overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch. So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
|
1205
1256
|
|
1206
|
-
|
1257
|
+
### Proxy auto config
|
1207
1258
|
|
1208
1259
|
The `fpac` option allows specification of a Proxy Auto Configuration (PAC) file, by its URL for local FASP agent. Supported schemes are : http:, https: and file:.
|
1209
1260
|
|
1210
1261
|
The PAC file can be tested with command: `config proxy_check` , example:
|
1211
1262
|
|
1212
|
-
```
|
1263
|
+
```bash
|
1213
1264
|
ascli config proxy_check --fpac=file:///./proxy.pac http://www.example.com
|
1214
1265
|
PROXY proxy.example.com:8080
|
1215
1266
|
```
|
1216
1267
|
|
1217
1268
|
This is not yet implemented to specify http proxy, so use `http_proxy` env vars.
|
1218
1269
|
|
1219
|
-
|
1270
|
+
### <a id="client"></a>FASP configuration
|
1220
1271
|
|
1221
1272
|
The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
|
1222
1273
|
|
@@ -1225,12 +1276,21 @@ The `config` plugin also allows specification for the use of a local FASP client
|
|
1225
1276
|
* `products` : list Aspera transfer products available locally
|
1226
1277
|
* `connect` : list,download connect client versions available on internet
|
1227
1278
|
|
1228
|
-
|
1279
|
+
#### Show path of currently used `ascp`
|
1229
1280
|
|
1230
|
-
```
|
1281
|
+
```bash
|
1231
1282
|
ascli config ascp show
|
1283
|
+
```
|
1284
|
+
|
1285
|
+
```output
|
1232
1286
|
/Users/laurent/.aspera/ascli/sdk/ascp
|
1287
|
+
```
|
1288
|
+
|
1289
|
+
```bash
|
1233
1290
|
ascli config ascp info
|
1291
|
+
```
|
1292
|
+
|
1293
|
+
```output
|
1234
1294
|
+--------------------+-----------------------------------------------------------+
|
1235
1295
|
| key | value |
|
1236
1296
|
+--------------------+-----------------------------------------------------------+
|
@@ -1238,7 +1298,7 @@ ascli config ascp info
|
|
1238
1298
|
...
|
1239
1299
|
```
|
1240
1300
|
|
1241
|
-
|
1301
|
+
#### Selection of `ascp` location for [`direct`](#agt_direct) agent
|
1242
1302
|
|
1243
1303
|
By default, `ascli` uses any found local product with ascp, including SDK.
|
1244
1304
|
|
@@ -1248,8 +1308,11 @@ For a permanent change, the command `config ascp use` sets the same parameter fo
|
|
1248
1308
|
|
1249
1309
|
Using a POSIX shell:
|
1250
1310
|
|
1311
|
+
```bash
|
1312
|
+
ascli config ascp use @path:'~/Applications/Aspera CLI/bin/ascp'
|
1251
1313
|
```
|
1252
|
-
|
1314
|
+
|
1315
|
+
```output
|
1253
1316
|
ascp version: 4.0.0.182279
|
1254
1317
|
Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
|
1255
1318
|
Saved to default global preset global_common_defaults
|
@@ -1257,8 +1320,11 @@ Saved to default global preset global_common_defaults
|
|
1257
1320
|
|
1258
1321
|
Windows:
|
1259
1322
|
|
1260
|
-
```
|
1323
|
+
```bash
|
1261
1324
|
ascli config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1325
|
+
```
|
1326
|
+
|
1327
|
+
```output
|
1262
1328
|
ascp version: 4.0.0.182279
|
1263
1329
|
Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1264
1330
|
Saved to default global preset global_common_defaults
|
@@ -1266,12 +1332,15 @@ Saved to default global preset global_common_defaults
|
|
1266
1332
|
|
1267
1333
|
If the path has spaces, read section: [Shell and Command line parsing](#parsing).
|
1268
1334
|
|
1269
|
-
|
1335
|
+
#### List locally installed Aspera Transfer products
|
1270
1336
|
|
1271
1337
|
Locally installed Aspera products can be listed with:
|
1272
1338
|
|
1273
|
-
```
|
1339
|
+
```bash
|
1274
1340
|
ascli config ascp products list
|
1341
|
+
```
|
1342
|
+
|
1343
|
+
```output
|
1275
1344
|
:.........................................:................................................:
|
1276
1345
|
: name : app_root :
|
1277
1346
|
:.........................................:................................................:
|
@@ -1282,7 +1351,7 @@ ascli config ascp products list
|
|
1282
1351
|
:.........................................:................................................:
|
1283
1352
|
```
|
1284
1353
|
|
1285
|
-
|
1354
|
+
#### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
|
1286
1355
|
|
1287
1356
|
If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
|
1288
1357
|
|
@@ -1290,45 +1359,55 @@ Using the option use_product finds the ascp binary of the selected product.
|
|
1290
1359
|
|
1291
1360
|
To permanently use the ascp of a product:
|
1292
1361
|
|
1293
|
-
```
|
1362
|
+
```bash
|
1294
1363
|
ascli config ascp products use 'Aspera Connect'
|
1295
1364
|
saved to default global preset /Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
|
1296
1365
|
```
|
1297
1366
|
|
1298
|
-
|
1367
|
+
#### Installation of Connect Client on command line
|
1299
1368
|
|
1300
|
-
```
|
1369
|
+
```bash
|
1301
1370
|
ascli config ascp connect list
|
1302
|
-
|
1303
|
-
|
1304
|
-
|
1305
|
-
|
1306
|
-
|
1307
|
-
|
1308
|
-
|
1309
|
-
|
1310
|
-
|
1311
|
-
|
1312
|
-
|
1313
|
-
|
1314
|
-
|
1315
|
-
|
1316
|
-
|
1317
|
-
|
1318
|
-
|
1319
|
-
|
1320
|
-
|
1321
|
-
|
1322
|
-
|
1323
|
-
|
1324
|
-
|
1325
|
-
|
1326
|
-
|
1327
|
-
|
1328
|
-
|
1329
|
-
|
1330
|
-
|
1331
|
-
|
1371
|
+
```
|
1372
|
+
|
1373
|
+
```output
|
1374
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1375
|
+
| id | title | version |
|
1376
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1377
|
+
| urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows | 3.11.2.63 |
|
1378
|
+
| urn:uuid:A3820D20-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows 64-bit | 3.11.2.63 |
|
1379
|
+
| urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows XP | 3.11.2.63 |
|
1380
|
+
| urn:uuid:55425020-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows XP 64-bit | 3.11.2.63 |
|
1381
|
+
| urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF | Aspera Connect for Mac Intel | 3.11.2.63 |
|
1382
|
+
| urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 | Aspera Connect for Linux 64 | 3.11.2.63 |
|
1383
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1384
|
+
```
|
1385
|
+
|
1386
|
+
```bash
|
1387
|
+
ascli config ascp connect version 'Aspera Connect for Mac Intel' list
|
1388
|
+
```
|
1389
|
+
|
1390
|
+
```output
|
1391
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1392
|
+
| title | type | href | hreflang | rel |
|
1393
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1394
|
+
| Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstaller-3.11.2.63.dmg | en | enclosure |
|
1395
|
+
| Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstallerOneClick-3.11.2.63.dmg | en | enclosure-one-click |
|
1396
|
+
| Aspera Connect for Mac HTML Documentation | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=aspera-connect-user-guide-macos | en | documentation |
|
1397
|
+
| Aspera Connect for Mac Release Notes | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=notes-release-aspera-connect-3112 | en | release-notes |
|
1398
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1399
|
+
```
|
1400
|
+
|
1401
|
+
```bash
|
1402
|
+
ascli config ascp connect version 'Aspera Connect for Mac Intel' download enclosure --to-folder=.
|
1403
|
+
```
|
1404
|
+
|
1405
|
+
```output
|
1406
|
+
Time: 00:00:02 ======================================================================= 100% 27766 KB/sec Time: 00:00:02
|
1407
|
+
Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
1408
|
+
```
|
1409
|
+
|
1410
|
+
### <a id="agents"></a>Transfer Agents
|
1332
1411
|
|
1333
1412
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
1334
1413
|
|
@@ -1349,8 +1428,7 @@ will effectively push files to the related server from the agent node.
|
|
1349
1428
|
|
1350
1429
|
`ascli` standardizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1351
1430
|
|
1352
|
-
|
1353
|
-
### <a id="agt_direct"></a>Direct
|
1431
|
+
#### <a id="agt_direct"></a>Direct
|
1354
1432
|
|
1355
1433
|
The `direct` agent directly executes a local ascp.
|
1356
1434
|
This is the default for `ascli`.
|
@@ -1376,7 +1454,7 @@ The `transfer-info` accepts the following optional parameters to control multi-s
|
|
1376
1454
|
Resume: In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
|
1377
1455
|
Sleep between iterations is:
|
1378
1456
|
|
1379
|
-
```
|
1457
|
+
```bash
|
1380
1458
|
max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
|
1381
1459
|
```
|
1382
1460
|
|
@@ -1384,7 +1462,7 @@ Some transfer errors are considered "retryable" (e.g. timeout) and some other no
|
|
1384
1462
|
|
1385
1463
|
Examples:
|
1386
1464
|
|
1387
|
-
```
|
1465
|
+
```javascript
|
1388
1466
|
ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
|
1389
1467
|
ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
|
1390
1468
|
```
|
@@ -1395,11 +1473,11 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
|
|
1395
1473
|
* `EX_http_proxy_url` (proxy for legacy http fallback)
|
1396
1474
|
* `EX_ascp_args`
|
1397
1475
|
|
1398
|
-
|
1476
|
+
#### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
|
1399
1477
|
|
1400
1478
|
By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
|
1401
1479
|
|
1402
|
-
|
1480
|
+
#### <a id="agt_node"></a>Aspera Node API : Node to node transfers
|
1403
1481
|
|
1404
1482
|
By specifying option: `--transfer=node`, the CLI will start transfers in an Aspera
|
1405
1483
|
Transfer Server using the Node API, either on a local or remote node.
|
@@ -1421,25 +1499,25 @@ If `transfer_info` is not specified and a default node has been configured (name
|
|
1421
1499
|
|
1422
1500
|
If the `password` value begins with `Bearer ` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
|
1423
1501
|
|
1424
|
-
|
1502
|
+
#### <a id="agt_httpgw"></a>HTTP Gateway
|
1425
1503
|
|
1426
1504
|
If it possible to send using a HTTP gateway, in case FASP is not allowed. `transfer_info` shall have a single mandatory parameter: `url`.
|
1427
1505
|
|
1428
1506
|
Example:
|
1429
1507
|
|
1430
|
-
```
|
1508
|
+
```javascript
|
1431
1509
|
ascli faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
|
1432
1510
|
```
|
1433
1511
|
|
1434
1512
|
Note that the gateway only supports transfers authorized with a token.
|
1435
1513
|
|
1436
|
-
|
1514
|
+
#### <a id="agt_trsdk"></a>Transfer SDK
|
1437
1515
|
|
1438
1516
|
Another possibility is to use the Transfer SDK daemon (asperatransferd).
|
1439
1517
|
|
1440
1518
|
By default it will listen on local port `55002` on `127.0.0.1`.
|
1441
1519
|
|
1442
|
-
|
1520
|
+
### <a id="transferspec"></a>Transfer Specification
|
1443
1521
|
|
1444
1522
|
Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
|
1445
1523
|
is described in a _transfer-spec_ (Transfer Specification), such as:
|
@@ -1463,7 +1541,7 @@ The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
|
|
1463
1541
|
|
1464
1542
|
A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
|
1465
1543
|
|
1466
|
-
|
1544
|
+
### <a id="transferparams"></a>Transfer Parameters
|
1467
1545
|
|
1468
1546
|
All standard _transfer-spec_ parameters can be specified.
|
1469
1547
|
[_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
|
@@ -1476,7 +1554,7 @@ References:
|
|
1476
1554
|
|
1477
1555
|
Parameters can be displayed with commands:
|
1478
1556
|
|
1479
|
-
```
|
1557
|
+
```javascript
|
1480
1558
|
ascli config ascp spec
|
1481
1559
|
ascli config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
|
1482
1560
|
```
|
@@ -1493,7 +1571,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
1493
1571
|
|
1494
1572
|
<table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>none, aes-128, aes-256<br/>Allowed values: aes128, aes192, aes256, aes128cfb, aes192cfb, aes256cfb, aes128gcm, aes192gcm, aes256gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side content protection. (encryption at rest)<br/>Allowed values: encrypt, decrypt</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password.</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer (older,string)<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td> </td><td>Y</td><td> </td><td>Remove SRC files after transfer success</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td> </td><td> </td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>http_fallback</td><td>string<br/>bool</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a fasp transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td> </td><td> </td><td>Specifies http port.<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port.</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td> </td><td>in bytes<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Required. Contains a path to the source (required) and a path to the destination.</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_access_key</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Node only?</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td> </td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies ssh (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td> </td><td> </td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIVATE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links by:<br/> following the link (follow),<br/> copying the link itself (copy),<br/> skipping (skip),<br/> or forcibly copying the link itself (copy+force).<br/> Default: follow<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td> </td><td> </td><td>Y</td><td>Returned by upload/download_setup node api.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td> </td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td> </td><td>specify version of protocol</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td> </td><td> </td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td> </td><td> </td><td>Preserve access control lists.<br/>Allowable values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td> </td><td> </td><td>TODO: check node, sdk<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Passphrase used for at rest encryption or decryption<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td> </td><td> </td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td> </td><td> </td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td> </td><td> </td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td> </td><td> </td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td> </td><td> </td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td> </td><td> </td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>rate_policy_allowed</td><td>string</td><td> </td><td> </td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/> Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>authentication</td><td>string</td><td> </td><td> </td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td> </td><td> </td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td> </td><td> </td><td>Add command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td> </td></tr></table>
|
1495
1573
|
|
1496
|
-
|
1574
|
+
#### Destination folder for transfers
|
1497
1575
|
|
1498
1576
|
The destination folder is set by `ascli` by default to:
|
1499
1577
|
|
@@ -1505,18 +1583,18 @@ As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path
|
|
1505
1583
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
1506
1584
|
`--to-folder=<path>` .
|
1507
1585
|
|
1508
|
-
|
1586
|
+
#### List of files for transfers
|
1509
1587
|
|
1510
1588
|
When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
|
1511
1589
|
So, by default, the list of files to transfer will be simply specified on the command line:
|
1512
1590
|
|
1513
|
-
```
|
1591
|
+
```bash
|
1514
1592
|
ascli server upload ~/mysample.file secondfile
|
1515
1593
|
```
|
1516
1594
|
|
1517
1595
|
This is equivalent to:
|
1518
1596
|
|
1519
|
-
```
|
1597
|
+
```bash
|
1520
1598
|
ascli server upload --sources=@args ~/mysample.file secondfile
|
1521
1599
|
```
|
1522
1600
|
|
@@ -1531,27 +1609,33 @@ For ease of use and flexibility, the list of files to transfer is specified by t
|
|
1531
1609
|
|
1532
1610
|
* an [Extended Value](#extended) holding an *Array of String*. Examples:
|
1533
1611
|
|
1534
|
-
```
|
1612
|
+
```javascript
|
1535
1613
|
--sources=@json:'["file1","file2"]'
|
1614
|
+
```
|
1615
|
+
|
1616
|
+
```bash
|
1536
1617
|
--sources=@lines:@stdin:
|
1618
|
+
```
|
1619
|
+
|
1620
|
+
```ruby
|
1537
1621
|
--sources=@ruby:'File.read("myfilelist").split("\n")'
|
1538
1622
|
```
|
1539
1623
|
|
1540
1624
|
* `@ts` : the user provides the list of files directly in the `ts` option, in its `paths` field. Example:
|
1541
1625
|
|
1542
|
-
```
|
1626
|
+
```javascript
|
1543
1627
|
--sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
|
1544
1628
|
```
|
1545
1629
|
|
1546
1630
|
providing a file list directly to ascp:
|
1547
1631
|
|
1548
|
-
```
|
1632
|
+
```javascript
|
1549
1633
|
... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
|
1550
1634
|
```
|
1551
1635
|
|
1552
1636
|
* Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
|
1553
1637
|
|
1554
|
-
```
|
1638
|
+
```javascript
|
1555
1639
|
--sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
|
1556
1640
|
```
|
1557
1641
|
|
@@ -1564,7 +1648,7 @@ In case the file list is provided on the command line i.e. using `--sources=@arg
|
|
1564
1648
|
|
1565
1649
|
Example:
|
1566
1650
|
|
1567
|
-
```
|
1651
|
+
```bash
|
1568
1652
|
ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
1569
1653
|
```
|
1570
1654
|
|
@@ -1581,13 +1665,13 @@ Source files are located on "Aspera on cloud", when :
|
|
1581
1665
|
* the server is Aspera on Cloud, and making a download / recv
|
1582
1666
|
* the agent is Aspera on Cloud, and making an upload / send
|
1583
1667
|
|
1584
|
-
|
1668
|
+
#### <a id="multisession"></a>Support of multi-session
|
1585
1669
|
|
1586
1670
|
Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
|
1587
1671
|
|
1588
1672
|
* when agent=node :
|
1589
1673
|
|
1590
|
-
```
|
1674
|
+
```javascript
|
1591
1675
|
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
|
1592
1676
|
```
|
1593
1677
|
|
@@ -1595,7 +1679,7 @@ Multi-session is directly supported by the node daemon.
|
|
1595
1679
|
|
1596
1680
|
* when agent=direct :
|
1597
1681
|
|
1598
|
-
```
|
1682
|
+
```javascript
|
1599
1683
|
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
|
1600
1684
|
```
|
1601
1685
|
|
@@ -1606,33 +1690,33 @@ Multi-session spawn is done by `ascli`.
|
|
1606
1690
|
|
1607
1691
|
When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
|
1608
1692
|
|
1609
|
-
|
1693
|
+
#### Transfer Spec Examples
|
1610
1694
|
|
1611
1695
|
* Change target rate
|
1612
1696
|
|
1613
|
-
```
|
1697
|
+
```javascript
|
1614
1698
|
--ts=@json:'{"target_rate_kbps":500000}'
|
1615
1699
|
```
|
1616
1700
|
|
1617
1701
|
* Override the FASP SSH port to a specific TCP port:
|
1618
1702
|
|
1619
|
-
```
|
1703
|
+
```javascript
|
1620
1704
|
--ts=@json:'{"ssh_port":33002}'
|
1621
1705
|
```
|
1622
1706
|
|
1623
1707
|
* Force http fallback mode:
|
1624
1708
|
|
1625
|
-
```
|
1709
|
+
```javascript
|
1626
1710
|
--ts=@json:'{"http_fallback":"force"}'
|
1627
1711
|
```
|
1628
1712
|
|
1629
1713
|
* Activate progress when not activated by default on server
|
1630
1714
|
|
1631
|
-
```
|
1715
|
+
```javascript
|
1632
1716
|
--ts=@json:'{"precalculate_job_size":true}'
|
1633
1717
|
```
|
1634
1718
|
|
1635
|
-
|
1719
|
+
### <a id="scheduling"></a>Lock for exclusive execution
|
1636
1720
|
|
1637
1721
|
In some conditions, it may be desirable to ensure that `ascli` is not executed several times in parallel.
|
1638
1722
|
|
@@ -1647,7 +1731,7 @@ Usually the OS native scheduler already provides some sort of protection against
|
|
1647
1731
|
* The Windows scheduler does this by default
|
1648
1732
|
* Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
|
1649
1733
|
|
1650
|
-
```
|
1734
|
+
```bash
|
1651
1735
|
/usr/bin/flock -w 0 /var/cron.lock ascli ...
|
1652
1736
|
```
|
1653
1737
|
|
@@ -1658,25 +1742,28 @@ Example:
|
|
1658
1742
|
|
1659
1743
|
Run this same command in two separate terminals within less than 30 seconds:
|
1660
1744
|
|
1661
|
-
```
|
1745
|
+
```bash
|
1662
1746
|
ascli config echo @ruby:'sleep(30)' --lock-port=12345
|
1663
1747
|
```
|
1664
1748
|
|
1665
1749
|
The first instance will sleep 30 seconds, the second one will immediately exit like this:
|
1666
1750
|
|
1667
|
-
```
|
1751
|
+
```bash
|
1668
1752
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
1669
1753
|
```
|
1670
1754
|
|
1671
|
-
|
1755
|
+
### "Provençale"
|
1672
1756
|
|
1673
1757
|
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
|
1674
1758
|
, some are provided om shared libraries and must be activated. (e.g. using `trapd`)
|
1675
1759
|
|
1676
1760
|
The list of supported *PVCL* adapters can be retrieved with command:
|
1677
1761
|
|
1678
|
-
```
|
1762
|
+
```bash
|
1679
1763
|
ascli conf ascp info
|
1764
|
+
```
|
1765
|
+
|
1766
|
+
```output
|
1680
1767
|
+--------------------+-----------------------------------------------------------+
|
1681
1768
|
| key | value |
|
1682
1769
|
+--------------------+-----------------------------------------------------------+
|
@@ -1699,13 +1786,13 @@ Those adapters can be used wherever a file path is used in `ascp` including conf
|
|
1699
1786
|
|
1700
1787
|
The simplified format is:
|
1701
1788
|
|
1702
|
-
```
|
1789
|
+
```bash
|
1703
1790
|
<adapter>:///<sub file path>?<arg1>=<val1>&...
|
1704
1791
|
```
|
1705
1792
|
|
1706
1793
|
One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
|
1707
1794
|
|
1708
|
-
|
1795
|
+
### <a id="faux_testing"></a>`faux:` for testing
|
1709
1796
|
|
1710
1797
|
This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not `ascli`.
|
1711
1798
|
|
@@ -1713,7 +1800,7 @@ This adapter can be used to simulate a file or a directory.
|
|
1713
1800
|
|
1714
1801
|
To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
|
1715
1802
|
|
1716
|
-
```
|
1803
|
+
```bash
|
1717
1804
|
faux:///filename?filesize
|
1718
1805
|
```
|
1719
1806
|
|
@@ -1728,7 +1815,7 @@ For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e
|
|
1728
1815
|
|
1729
1816
|
To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
|
1730
1817
|
|
1731
|
-
```
|
1818
|
+
```bash
|
1732
1819
|
faux:///dirname?<arg1>=<val1>&...
|
1733
1820
|
```
|
1734
1821
|
|
@@ -1771,27 +1858,27 @@ Examples:
|
|
1771
1858
|
|
1772
1859
|
* Upload 20 gibibytes of random data to file myfile to directory /Upload
|
1773
1860
|
|
1774
|
-
```
|
1861
|
+
```bash
|
1775
1862
|
ascli server upload faux:///myfile\?20g --to-folder=/Upload
|
1776
1863
|
```
|
1777
1864
|
|
1778
1865
|
* Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
|
1779
1866
|
|
1780
|
-
```
|
1867
|
+
```bash
|
1781
1868
|
ascli server upload /tmp/sample --to-folder=faux://
|
1782
1869
|
```
|
1783
1870
|
|
1784
1871
|
* Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
|
1785
1872
|
|
1786
|
-
```
|
1873
|
+
```bash
|
1787
1874
|
ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
|
1788
1875
|
```
|
1789
1876
|
|
1790
|
-
|
1877
|
+
### <a id="commands"></a>Sample Commands
|
1791
1878
|
|
1792
1879
|
A non complete list of commands used in unit tests:
|
1793
1880
|
|
1794
|
-
```
|
1881
|
+
```bash
|
1795
1882
|
ascli
|
1796
1883
|
ascli -h
|
1797
1884
|
ascli aoc -N remind --username=my_aoc_user_email
|
@@ -1826,12 +1913,12 @@ ascli aoc admin res self show
|
|
1826
1913
|
ascli aoc admin res short_link list
|
1827
1914
|
ascli aoc admin res user list
|
1828
1915
|
ascli aoc admin res workspace_membership list
|
1829
|
-
ascli aoc admin resource node --name=
|
1830
|
-
ascli aoc admin resource node --name=
|
1831
|
-
ascli aoc admin resource node --name=
|
1832
|
-
ascli aoc admin resource node --name=
|
1833
|
-
ascli aoc admin resource node --name=
|
1834
|
-
ascli aoc admin resource node v3 name
|
1916
|
+
ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
|
1917
|
+
ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v3 events
|
1918
|
+
ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 browse /
|
1919
|
+
ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 delete /folder1
|
1920
|
+
ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 mkdir /folder1
|
1921
|
+
ascli aoc admin resource node v3 name my_aoc_node1_name --secret=my_aoc_node1_secret access_key delete testsub1
|
1835
1922
|
ascli aoc admin resource workspace list
|
1836
1923
|
ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
1837
1924
|
ascli aoc automation workflow "my_wf_id" action create --value=@json:'{"name":"toto"}' | tee action.info
|
@@ -1864,19 +1951,21 @@ ascli aoc files v3 info
|
|
1864
1951
|
ascli aoc org -N --link=my_aoc_publink_recv_from_aocuser
|
1865
1952
|
ascli aoc organization
|
1866
1953
|
ascli aoc packages list
|
1867
|
-
ascli aoc packages list --query=@json:'{"
|
1954
|
+
ascli aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
|
1868
1955
|
ascli aoc packages recv "my_package_id" --to-folder=.
|
1869
1956
|
ascli aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
|
1870
|
-
ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["
|
1871
|
-
ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["
|
1957
|
+
ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
|
1958
|
+
ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
|
1959
|
+
ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
|
1960
|
+
ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
|
1872
1961
|
ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
|
1873
1962
|
ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
|
1874
1963
|
ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
|
1875
|
-
ascli aoc
|
1876
|
-
ascli aoc user
|
1877
|
-
ascli aoc user
|
1878
|
-
ascli aoc user workspaces
|
1879
|
-
ascli aoc
|
1964
|
+
ascli aoc packages shared_inboxes list
|
1965
|
+
ascli aoc user profile modify @json:'{"name":"dummy change"}'
|
1966
|
+
ascli aoc user profile show
|
1967
|
+
ascli aoc user workspaces current
|
1968
|
+
ascli aoc user workspaces list
|
1880
1969
|
ascli ats access_key cluster akibmcloud --secret=somesecret
|
1881
1970
|
ascli ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
|
1882
1971
|
ascli ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
|
@@ -1894,24 +1983,26 @@ ascli ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
|
|
1894
1983
|
ascli conf flush_tokens
|
1895
1984
|
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes
|
1896
1985
|
ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath='' --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
|
1897
|
-
ascli config ascp connect
|
1898
|
-
ascli config ascp connect id 'Aspera Connect for Windows' links id 'Windows Installer' download --to-folder=.
|
1899
|
-
ascli config ascp connect id 'Aspera Connect for Windows' links list
|
1986
|
+
ascli config ascp connect info 'Aspera Connect for Windows'
|
1900
1987
|
ascli config ascp connect list
|
1988
|
+
ascli config ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
|
1989
|
+
ascli config ascp connect version 'Aspera Connect for Windows' list
|
1990
|
+
ascli config ascp connect version 'Aspera Connect for Windows' open documentation
|
1901
1991
|
ascli config ascp info
|
1902
1992
|
ascli config ascp install
|
1903
1993
|
ascli config ascp products list
|
1904
1994
|
ascli config ascp show
|
1905
1995
|
ascli config ascp spec
|
1906
1996
|
ascli config check_update
|
1907
|
-
ascli config detect --url=https://my_aoc_org.ibmaspera.com
|
1997
|
+
ascli config detect --url=https://my_aoc_org.ibmaspera.com
|
1908
1998
|
ascli config detect --url=my_faspex_url
|
1909
1999
|
ascli config doc
|
1910
2000
|
ascli config doc transfer-parameters
|
1911
2001
|
ascli config email_test --notif-to=my_recipient_email
|
1912
2002
|
ascli config export
|
1913
2003
|
ascli config genkey mykey
|
1914
|
-
ascli config
|
2004
|
+
ascli config plugin create mycommand T
|
2005
|
+
ascli config plugin list
|
1915
2006
|
ascli config proxy_check --fpac=file:///examples/proxy.pac https://eudemo.asperademo.com
|
1916
2007
|
ascli console transfer current list
|
1917
2008
|
ascli console transfer smart list
|
@@ -1924,14 +2015,14 @@ ascli cos node info
|
|
1924
2015
|
ascli cos node upload testfile.bin
|
1925
2016
|
ascli faspex health
|
1926
2017
|
ascli faspex package list
|
1927
|
-
ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
|
1928
|
-
ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
|
2018
|
+
ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
|
2019
|
+
ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}')
|
1929
2020
|
ascli faspex package recv "my_package_id" --to-folder=.
|
1930
2021
|
ascli faspex package recv "my_package_id" --to-folder=. --box=sent
|
1931
2022
|
ascli faspex package recv --to-folder=. "my_package_id"
|
1932
2023
|
ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
|
1933
2024
|
ascli faspex package recv ALL --to-folder=. --once-only=yes
|
1934
|
-
ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["
|
2025
|
+
ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
|
1935
2026
|
ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
1936
2027
|
ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
|
1937
2028
|
ascli faspex source name "Server Files" node br /
|
@@ -1939,6 +2030,7 @@ ascli faspex5 node list --value=@json:'{"type":"received","subtype":"mypackages"
|
|
1939
2030
|
ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
|
1940
2031
|
ascli faspex5 package receive "my_package_id" --to-folder=.
|
1941
2032
|
ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"${f5_user}"}]}' testfile.bin
|
2033
|
+
ascli mycommand --plugin-folder=T
|
1942
2034
|
ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
|
1943
2035
|
ascli node -N -Ptst_node_preview access_key delete aoc_1
|
1944
2036
|
ascli node async bandwidth 1
|
@@ -1964,23 +2056,23 @@ ascli node upload --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000
|
|
1964
2056
|
ascli orchestrator info
|
1965
2057
|
ascli orchestrator plugins
|
1966
2058
|
ascli orchestrator processes
|
1967
|
-
ascli orchestrator workflow inputs
|
2059
|
+
ascli orchestrator workflow inputs my_orch_workflow_id
|
1968
2060
|
ascli orchestrator workflow list
|
1969
|
-
ascli orchestrator workflow start
|
1970
|
-
ascli orchestrator workflow start
|
2061
|
+
ascli orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}'
|
2062
|
+
ascli orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
|
1971
2063
|
ascli orchestrator workflow status ALL
|
1972
|
-
ascli orchestrator workflow status
|
2064
|
+
ascli orchestrator workflow status my_orch_workflow_id
|
1973
2065
|
ascli preview check --skip-types=office
|
1974
2066
|
ascli preview folder 1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
|
1975
2067
|
ascli preview scan --skip-types=office --log-level=info
|
1976
|
-
ascli preview test --case=test mp4
|
1977
|
-
ascli preview test --case=test mp4
|
1978
|
-
ascli preview test --case=test mp4
|
1979
|
-
ascli preview test --case=test png
|
1980
|
-
ascli preview test --case=test png
|
1981
|
-
ascli preview test --case=test png
|
1982
|
-
ascli preview test --case=test png
|
1983
|
-
ascli preview test --case=test png
|
2068
|
+
ascli preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
|
2069
|
+
ascli preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
|
2070
|
+
ascli preview test --case=test mp4 my_file_mxf --video-conversion=reencode --log-level=debug
|
2071
|
+
ascli preview test --case=test png my_file_dcm --log-level=debug
|
2072
|
+
ascli preview test --case=test png my_file_docx --log-level=debug
|
2073
|
+
ascli preview test --case=test png my_file_mxf --video-png-conv=animated --log-level=debug
|
2074
|
+
ascli preview test --case=test png my_file_mxf --video-png-conv=fixed --log-level=debug
|
2075
|
+
ascli preview test --case=test png my_file_pdf --log-level=debug
|
1984
2076
|
ascli preview trevents --once-only=yes --skip-types=office --log-level=info
|
1985
2077
|
ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user ctl all:status
|
1986
2078
|
ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user health app_services --format=nagios
|
@@ -2010,11 +2102,11 @@ ascli server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json
|
|
2010
2102
|
ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
|
2011
2103
|
ascli shares admin share list
|
2012
2104
|
ascli shares repository browse /
|
2013
|
-
ascli shares repository delete /
|
2014
|
-
ascli shares repository download --to-folder=. /
|
2015
|
-
ascli shares repository download --to-folder=. /
|
2016
|
-
ascli shares repository upload --to-folder
|
2017
|
-
ascli shares repository upload --to-folder
|
2105
|
+
ascli shares repository delete my_shares_upload/testfile.bin
|
2106
|
+
ascli shares repository download --to-folder=. my_shares_upload/testfile.bin
|
2107
|
+
ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://"my_http_gw_fqdn"/aspera/http-gwy/v1"}'
|
2108
|
+
ascli shares repository upload --to-folder=my_shares_upload testfile.bin
|
2109
|
+
ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://"my_http_gw_fqdn"/aspera/http-gwy/v1"}'
|
2018
2110
|
ascli shares2 appinfo
|
2019
2111
|
ascli shares2 organization list
|
2020
2112
|
ascli shares2 project list --organization=Sport
|
@@ -2025,12 +2117,12 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
|
|
2025
2117
|
...and more
|
2026
2118
|
```
|
2027
2119
|
|
2028
|
-
|
2120
|
+
### <a id="usage"></a>Usage
|
2029
2121
|
|
2030
|
-
```
|
2122
|
+
```bash
|
2031
2123
|
ascli -h
|
2032
2124
|
NAME
|
2033
|
-
ascli -- a command line tool for Aspera Applications (v4.
|
2125
|
+
ascli -- a command line tool for Aspera Applications (v4.6.0)
|
2034
2126
|
|
2035
2127
|
SYNOPSIS
|
2036
2128
|
ascli COMMANDS [OPTIONS] [ARGS]
|
@@ -2068,6 +2160,7 @@ OPTIONS: global
|
|
2068
2160
|
--select=VALUE select only some items in lists, extended value: hash (column, value)
|
2069
2161
|
--table-style=VALUE table display style
|
2070
2162
|
--flat-hash=ENUM display hash values as additional keys: yes, no
|
2163
|
+
--transpose-single=ENUM single object fields output vertically: yes, no
|
2071
2164
|
-h, --help Show this message.
|
2072
2165
|
--bash-comp generate bash completion for command
|
2073
2166
|
--show-config Display parameters used for the provided action.
|
@@ -2085,7 +2178,7 @@ OPTIONS: global
|
|
2085
2178
|
--log-passwords=ENUM show passwords in logs: yes, no
|
2086
2179
|
|
2087
2180
|
COMMAND: config
|
2088
|
-
SUBCOMMANDS: list overview id preset open documentation genkey gem_path
|
2181
|
+
SUBCOMMANDS: list overview id preset open documentation genkey gem_path plugin flush_tokens echo wizard export_to_cli detect coffee ascp email_test smtp_settings proxy_check folder file check_update initdemo vault
|
2089
2182
|
OPTIONS:
|
2090
2183
|
--value=VALUE extended value for create, update, list filter
|
2091
2184
|
--property=VALUE name of property to set
|
@@ -2109,6 +2202,7 @@ OPTIONS:
|
|
2109
2202
|
--notif-to=VALUE email recipient for notification of transfers
|
2110
2203
|
--notif-template=VALUE email ERB template for notification of transfers
|
2111
2204
|
--version-check-days=VALUE period in days to check new version (zero to disable)
|
2205
|
+
--plugin-folder=VALUE folder where to find additional plugins
|
2112
2206
|
--ts=VALUE override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
|
2113
2207
|
--local-resume=VALUE set resume policy (Hash, use @json: prefix), current=
|
2114
2208
|
--to-folder=VALUE destination folder for downloaded files
|
@@ -2135,7 +2229,7 @@ OPTIONS:
|
|
2135
2229
|
--password=VALUE user's password
|
2136
2230
|
--validator=VALUE identifier of validator (optional for central)
|
2137
2231
|
--asperabrowserurl=VALUE URL for simple aspera web ui
|
2138
|
-
--name=VALUE
|
2232
|
+
--sync-name=VALUE sync name
|
2139
2233
|
--token-type=ENUM Type of token used for transfers: aspera, basic, hybrid
|
2140
2234
|
|
2141
2235
|
|
@@ -2271,7 +2365,7 @@ OPTIONS:
|
|
2271
2365
|
|
2272
2366
|
|
2273
2367
|
COMMAND: aoc
|
2274
|
-
SUBCOMMANDS: reminder bearer_token organization tier_restrictions user
|
2368
|
+
SUBCOMMANDS: reminder servers bearer_token organization tier_restrictions user packages files admin automation gateway
|
2275
2369
|
OPTIONS:
|
2276
2370
|
--url=VALUE URL of application, e.g. https://org.asperafiles.com
|
2277
2371
|
--username=VALUE username to log in
|
@@ -2318,17 +2412,17 @@ OPTIONS:
|
|
2318
2412
|
|
2319
2413
|
Note that actions and parameter values can be written in short form.
|
2320
2414
|
|
2321
|
-
|
2415
|
+
## <a id="aoc"></a>Plugin: Aspera on Cloud
|
2322
2416
|
|
2323
2417
|
Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
|
2324
2418
|
|
2325
2419
|
It is recommended to use the wizard to set it up, but manual configuration is also possible.
|
2326
2420
|
|
2327
|
-
|
2421
|
+
### <a id="aocwizard"></a>Configuration: using Wizard
|
2328
2422
|
|
2329
2423
|
`ascli` provides a configuration wizard. Here is a sample invocation :
|
2330
2424
|
|
2331
|
-
```
|
2425
|
+
```bash
|
2332
2426
|
ascli config wizard
|
2333
2427
|
option: url> https://myorg.ibmaspera.com
|
2334
2428
|
Detected: Aspera on Cloud
|
@@ -2345,7 +2439,7 @@ Setting config preset as default for aspera
|
|
2345
2439
|
saving config file
|
2346
2440
|
Done.
|
2347
2441
|
You can test with:
|
2348
|
-
ascli aoc user
|
2442
|
+
ascli aoc user profile show
|
2349
2443
|
```
|
2350
2444
|
|
2351
2445
|
Optionally, it is possible to create a new organization-specific "integration".
|
@@ -2353,11 +2447,11 @@ For this, specify the option: `--use-generic-client=no`.
|
|
2353
2447
|
|
2354
2448
|
This will guide you through the steps to create.
|
2355
2449
|
|
2356
|
-
|
2450
|
+
### <a id="aocmanual"></a>Configuration: using manual setup
|
2357
2451
|
|
2358
2452
|
If you used the wizard (recommended): skip this section.
|
2359
2453
|
|
2360
|
-
|
2454
|
+
#### Configuration details
|
2361
2455
|
|
2362
2456
|
Several types of OAuth authentication are supported:
|
2363
2457
|
|
@@ -2373,7 +2467,7 @@ For a more convenient, browser-less, experience follow the [JWT](#jwt) section (
|
|
2373
2467
|
|
2374
2468
|
In Oauth, a "Bearer" token are generated to authenticate REST calls. Bearer tokens are valid for a period of time.`ascli` saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
|
2375
2469
|
|
2376
|
-
|
2470
|
+
#### <a id="clientreg"></a>Optional: API Client Registration
|
2377
2471
|
|
2378
2472
|
If you use the built-in client_id and client_secret, skip this and do not set them in next section.
|
2379
2473
|
|
@@ -2386,24 +2480,24 @@ Let's start by a registration with web based authentication (auth=web):
|
|
2386
2480
|
* Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
|
2387
2481
|
* Go to Apps→Admin→Organization→Integrations
|
2388
2482
|
* Click "Create New"
|
2389
|
-
|
2390
|
-
|
2391
|
-
|
2392
|
-
|
2393
|
-
|
2483
|
+
* Client Name: `ascli`
|
2484
|
+
* Redirect URIs: `http://localhost:12345`
|
2485
|
+
* Origins: `localhost`
|
2486
|
+
* uncheck "Prompt users to allow client to access"
|
2487
|
+
* leave the JWT part for now
|
2394
2488
|
* Save
|
2395
2489
|
|
2396
2490
|
Note: for web based authentication, `ascli` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``ascli`, HTTP is required, and 12345 is the default port.
|
2397
2491
|
|
2398
2492
|
Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
|
2399
2493
|
|
2400
|
-
|
2494
|
+
#### <a id="aocpreset"></a>[option preset](#lprt) for Aspera on Cloud
|
2401
2495
|
|
2402
2496
|
If you did not use the wizard, you can also manually create a [option preset](#lprt) for `ascli` in its configuration file.
|
2403
2497
|
|
2404
2498
|
Lets create an [option preset](#lprt) called: `my_aoc_org` using `ask` interactive input (client info from previous step):
|
2405
2499
|
|
2406
|
-
```
|
2500
|
+
```bash
|
2407
2501
|
ascli config preset ask my_aoc_org url client_id client_secret
|
2408
2502
|
option: url> https://myorg.ibmaspera.com/
|
2409
2503
|
option: client_id> BJLPObQiFw
|
@@ -2415,17 +2509,17 @@ updated: my_aoc_org
|
|
2415
2509
|
|
2416
2510
|
Define this [option preset](#lprt) as default configuration for the `aspera` plugin:
|
2417
2511
|
|
2418
|
-
```
|
2512
|
+
```bash
|
2419
2513
|
ascli config preset set default aoc my_aoc_org
|
2420
2514
|
```
|
2421
2515
|
|
2422
2516
|
Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
2423
2517
|
|
2424
|
-
|
2518
|
+
#### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
|
2425
2519
|
|
2426
2520
|
For a Browser-less, Private Key-based authentication, use the following steps.
|
2427
2521
|
|
2428
|
-
|
2522
|
+
##### Key Pair Generation
|
2429
2523
|
|
2430
2524
|
In order to use JWT for Aspera on Cloud API client authentication,
|
2431
2525
|
a private/public key pair must be generated (without passphrase)
|
@@ -2435,13 +2529,13 @@ This can be done using any of the following method:
|
|
2435
2529
|
|
2436
2530
|
* using the CLI:
|
2437
2531
|
|
2438
|
-
```
|
2532
|
+
```bash
|
2439
2533
|
ascli config genkey ~/.aspera/ascli/aocapikey
|
2440
2534
|
```
|
2441
2535
|
|
2442
2536
|
* `ssh-keygen`:
|
2443
2537
|
|
2444
|
-
```
|
2538
|
+
```bash
|
2445
2539
|
ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
|
2446
2540
|
```
|
2447
2541
|
|
@@ -2449,7 +2543,7 @@ ssh-keygen -t rsa -f ~/.aspera/ascli/aocapikey -N ''
|
|
2449
2543
|
|
2450
2544
|
(on some openssl implementation (mac) there is option: -nodes (no DES))
|
2451
2545
|
|
2452
|
-
```
|
2546
|
+
```bash
|
2453
2547
|
APIKEY=~/.aspera/ascli/aocapikey
|
2454
2548
|
openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
|
2455
2549
|
openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
|
@@ -2457,63 +2551,81 @@ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
|
|
2457
2551
|
rm -f ${APIKEY}.protected
|
2458
2552
|
```
|
2459
2553
|
|
2460
|
-
|
2554
|
+
##### API Client JWT activation
|
2461
2555
|
|
2462
2556
|
If you are not using the built-in client_id and secret, JWT needs to be authorized in Aspera on Cloud. This can be done in two manners:
|
2463
2557
|
|
2464
2558
|
* Graphically
|
2465
2559
|
|
2466
|
-
|
2467
|
-
|
2468
|
-
|
2469
|
-
|
2470
|
-
|
2471
|
-
|
2560
|
+
* Open a web browser, log to your instance: https://myorg.ibmaspera.com/
|
2561
|
+
* Go to Apps→Admin→Organization→Integrations
|
2562
|
+
* Click on the previously created application
|
2563
|
+
* select tab : "JSON Web Token Auth"
|
2564
|
+
* Modify options if necessary, for instance: activate both options in section "Settings"
|
2565
|
+
* Click "Save"
|
2472
2566
|
|
2473
2567
|
* Using command line
|
2474
2568
|
|
2475
|
-
```
|
2569
|
+
```bash
|
2476
2570
|
ascli aoc admin res client list
|
2477
|
-
|
2478
|
-
|
2479
|
-
|
2480
|
-
|
2481
|
-
|
2482
|
-
|
2571
|
+
```
|
2572
|
+
|
2573
|
+
```output
|
2574
|
+
:............:...............:
|
2575
|
+
: id : name :
|
2576
|
+
:............:...............:
|
2577
|
+
: BJLPObQiFw : my-client-app :
|
2578
|
+
:............:...............:
|
2579
|
+
```
|
2580
|
+
|
2581
|
+
```javascript
|
2582
|
+
ascli aoc admin res client modify BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
|
2583
|
+
```
|
2584
|
+
|
2585
|
+
```output
|
2483
2586
|
modified
|
2484
2587
|
```
|
2485
2588
|
|
2486
|
-
|
2589
|
+
#### User key registration
|
2487
2590
|
|
2488
2591
|
The public key must be assigned to your user. This can be done in two manners:
|
2489
2592
|
|
2490
|
-
|
2593
|
+
##### Graphically
|
2491
2594
|
|
2492
2595
|
open the previously generated public key located here: `$HOME/.aspera/ascli/aocapikey.pub`
|
2493
2596
|
|
2494
|
-
|
2495
|
-
|
2496
|
-
|
2497
|
-
|
2498
|
-
|
2597
|
+
* Open a web browser, log to your instance: https://myorg.ibmaspera.com/
|
2598
|
+
* Click on the user's icon (top right)
|
2599
|
+
* Select "Account Settings"
|
2600
|
+
* Paste the _Public Key_ in the "Public Key" section
|
2601
|
+
* Click on "Submit"
|
2499
2602
|
|
2500
|
-
|
2603
|
+
##### Using command line
|
2501
2604
|
|
2502
|
-
```
|
2605
|
+
```bash
|
2503
2606
|
ascli aoc admin res user list
|
2607
|
+
```
|
2608
|
+
|
2609
|
+
```output
|
2504
2610
|
:........:................:
|
2505
2611
|
: id : name :
|
2506
2612
|
:........:................:
|
2507
2613
|
: 109952 : Tech Support :
|
2508
2614
|
: 109951 : LAURENT MARTIN :
|
2509
2615
|
:........:................:
|
2510
|
-
|
2616
|
+
```
|
2617
|
+
|
2618
|
+
```ruby
|
2619
|
+
ascli aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/ascli/aocapikey.pub"))}'
|
2620
|
+
```
|
2621
|
+
|
2622
|
+
```output
|
2511
2623
|
modified
|
2512
2624
|
```
|
2513
2625
|
|
2514
2626
|
Note: the `aspera user info show` command can be used to verify modifications.
|
2515
2627
|
|
2516
|
-
|
2628
|
+
#### [option preset](#lprt) modification for JWT
|
2517
2629
|
|
2518
2630
|
To activate default use of JWT authentication for `ascli` using the [option preset](#lprt), do the following:
|
2519
2631
|
|
@@ -2523,7 +2635,7 @@ To activate default use of JWT authentication for `ascli` using the [option pres
|
|
2523
2635
|
|
2524
2636
|
Execute:
|
2525
2637
|
|
2526
|
-
```
|
2638
|
+
```bash
|
2527
2639
|
ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/aocapikey --username=laurent.martin.aspera@fr.ibm.com
|
2528
2640
|
```
|
2529
2641
|
|
@@ -2531,30 +2643,31 @@ Note: the private key argument represents the actual PEM string. In order to rea
|
|
2531
2643
|
|
2532
2644
|
After this last step, commands do not require web login anymore.
|
2533
2645
|
|
2534
|
-
|
2535
|
-
### <a id="aocfirst"></a>First Use
|
2646
|
+
#### <a id="aocfirst"></a>First Use
|
2536
2647
|
|
2537
2648
|
Once client has been registered and [option preset](#lprt) created: `ascli` can be used:
|
2538
2649
|
|
2539
|
-
```
|
2650
|
+
```bash
|
2540
2651
|
ascli aoc files br /
|
2652
|
+
```
|
2653
|
+
|
2654
|
+
```output
|
2541
2655
|
Current Workspace: Default Workspace (default)
|
2542
2656
|
empty
|
2543
2657
|
```
|
2544
2658
|
|
2545
|
-
|
2546
|
-
## Administration
|
2659
|
+
### Administration
|
2547
2660
|
|
2548
2661
|
The `admin` command allows several administrative tasks (and require admin privilege).
|
2549
2662
|
|
2550
2663
|
It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
|
2551
2664
|
|
2552
|
-
|
2665
|
+
#### Bulk creation and deletion of resource
|
2553
2666
|
|
2554
2667
|
Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
|
2555
2668
|
In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
|
2556
2669
|
|
2557
|
-
|
2670
|
+
#### Listing resources
|
2558
2671
|
|
2559
2672
|
The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
|
2560
2673
|
|
@@ -2566,7 +2679,7 @@ The following parameters are supported:
|
|
2566
2679
|
* `sort`: name of fields to sort results, prefix with `-` for reverse order.
|
2567
2680
|
* `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
2568
2681
|
* `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
2569
|
-
* `page` : native api parameter, in general do not use (added by
|
2682
|
+
* `page` : native api parameter, in general do not use (added by
|
2570
2683
|
* `per_page` : native api parameter, number of items par api call, in general do not use
|
2571
2684
|
* Other specific parameters depending on resource type.
|
2572
2685
|
|
@@ -2584,19 +2697,19 @@ Examples:
|
|
2584
2697
|
|
2585
2698
|
* List users with `laurent` in name:
|
2586
2699
|
|
2587
|
-
```
|
2700
|
+
```javascript
|
2588
2701
|
ascli aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
|
2589
2702
|
```
|
2590
2703
|
|
2591
2704
|
* List users who logged-in before a date:
|
2592
2705
|
|
2593
|
-
```
|
2706
|
+
```javascript
|
2594
2707
|
ascli aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
2595
2708
|
```
|
2596
2709
|
|
2597
2710
|
* List external users and sort in reverse alphabetical order using name:
|
2598
2711
|
|
2599
|
-
```
|
2712
|
+
```javascript
|
2600
2713
|
ascli aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
|
2601
2714
|
```
|
2602
2715
|
|
@@ -2604,34 +2717,34 @@ Refer to the AoC API for full list of query parameters, or use the browser in de
|
|
2604
2717
|
|
2605
2718
|
Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
|
2606
2719
|
|
2607
|
-
|
2720
|
+
#### <a id="res_select"></a>Selecting a resource
|
2608
2721
|
|
2609
2722
|
Resources are identified by a unique `id`, as well as a unique `name` (case insensitive).
|
2610
2723
|
|
2611
2724
|
To execute an action on a specific resource, select it using one of those methods:
|
2612
2725
|
|
2613
|
-
*
|
2726
|
+
* **recommended:** give id directly on command line *after the action*: `aoc admin res node show 123`
|
2614
2727
|
* give name on command line *after the action*: `aoc admin res node show name abc`
|
2615
2728
|
* provide option `id` : `aoc admin res node show --id=123`
|
2616
2729
|
* provide option `name` : `aoc admin res node show --name=abc`
|
2617
2730
|
|
2618
|
-
|
2731
|
+
#### Access Key secrets
|
2619
2732
|
|
2620
2733
|
In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
|
2621
2734
|
It is usually provided using the `secret` option.
|
2622
2735
|
For example in a command like:
|
2623
2736
|
|
2624
|
-
```
|
2737
|
+
```bash
|
2625
2738
|
ascli aoc admin res node --id=123 --secret="secret1" v3 info
|
2626
2739
|
```
|
2627
2740
|
|
2628
2741
|
It is also possible to provide a set of secrets used on a regular basis using the [secret vault](#vault).
|
2629
2742
|
|
2630
|
-
|
2743
|
+
#### Activity
|
2631
2744
|
|
2632
2745
|
The activity app can be queried with:
|
2633
2746
|
|
2634
|
-
```
|
2747
|
+
```bash
|
2635
2748
|
ascli aoc admin analytics transfers
|
2636
2749
|
```
|
2637
2750
|
|
@@ -2639,7 +2752,7 @@ It can also support filters and send notification using option `notif_to`. a tem
|
|
2639
2752
|
|
2640
2753
|
`mytemplate.erb`:
|
2641
2754
|
|
2642
|
-
```
|
2755
|
+
```bash
|
2643
2756
|
From: <%=from_name%> <<%=from_email%>>
|
2644
2757
|
To: <<%=ev['user_email']%>>
|
2645
2758
|
Subject: <%=ev['files_completed']%> files received
|
@@ -2650,13 +2763,14 @@ We received <%=ev['files_completed']%> files for a total of <%=ev['transferred_b
|
|
2650
2763
|
|
2651
2764
|
Thank you.
|
2652
2765
|
```
|
2766
|
+
|
2653
2767
|
The environment provided contains the following additional variable:
|
2654
2768
|
|
2655
2769
|
* ev : all details on the transfer event
|
2656
2770
|
|
2657
2771
|
Example:
|
2658
2772
|
|
2659
|
-
```
|
2773
|
+
```javascript
|
2660
2774
|
ascli aoc admin analytics transfers --once-only=yes --lock-port=12345 \
|
2661
2775
|
--query=@json:'{"status":"completed","direction":"receive"}' \
|
2662
2776
|
--notif-to=active --notif-template=@file:mytemplate.erb
|
@@ -2670,19 +2784,22 @@ Options:
|
|
2670
2784
|
|
2671
2785
|
Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
|
2672
2786
|
|
2673
|
-
|
2787
|
+
#### Transfer: Using specific transfer ports
|
2674
2788
|
|
2675
2789
|
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
2676
2790
|
The option `default_ports` ([yes]/no) allows ascli to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
2677
2791
|
|
2678
|
-
|
2792
|
+
#### Using ATS
|
2679
2793
|
|
2680
2794
|
Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
|
2681
2795
|
|
2682
|
-
|
2796
|
+
#### Example: Bulk creation of users
|
2683
2797
|
|
2684
|
-
```
|
2798
|
+
```javascript
|
2685
2799
|
ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
|
2800
|
+
```
|
2801
|
+
|
2802
|
+
```output
|
2686
2803
|
:.......:.........:
|
2687
2804
|
: id : status :
|
2688
2805
|
:.......:.........:
|
@@ -2691,20 +2808,38 @@ ascli aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.
|
|
2691
2808
|
:.......:.........:
|
2692
2809
|
```
|
2693
2810
|
|
2694
|
-
|
2811
|
+
#### Example: Find with filter and delete
|
2695
2812
|
|
2696
|
-
```
|
2813
|
+
```javascript
|
2697
2814
|
ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
|
2815
|
+
```
|
2816
|
+
|
2817
|
+
```output
|
2698
2818
|
:.......:........................:
|
2699
2819
|
: id : email :
|
2700
2820
|
:.......:........................:
|
2701
2821
|
: 98398 : dummyuser1@example.com :
|
2702
2822
|
: 98399 : dummyuser2@example.com :
|
2703
2823
|
:.......:........................:
|
2824
|
+
```
|
2825
|
+
|
2826
|
+
```bash
|
2704
2827
|
thelist=$(ascli aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
|
2828
|
+
```
|
2829
|
+
|
2830
|
+
```bash
|
2705
2831
|
echo $thelist
|
2832
|
+
```
|
2833
|
+
|
2834
|
+
```javascript
|
2706
2835
|
["113501","354061"]
|
2836
|
+
```
|
2837
|
+
|
2838
|
+
```bash
|
2707
2839
|
ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
|
2840
|
+
```
|
2841
|
+
|
2842
|
+
```output
|
2708
2843
|
:.......:.........:
|
2709
2844
|
: id : status :
|
2710
2845
|
:.......:.........:
|
@@ -2713,18 +2848,21 @@ ascli aoc admin res user --bulk=yes --id=@json:"$thelist" delete
|
|
2713
2848
|
:.......:.........:
|
2714
2849
|
```
|
2715
2850
|
|
2716
|
-
|
2851
|
+
#### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
|
2717
2852
|
|
2718
|
-
```
|
2853
|
+
```ruby
|
2719
2854
|
ascli aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
|
2720
2855
|
```
|
2721
2856
|
|
2722
2857
|
To delete them use the same method as before
|
2723
2858
|
|
2724
|
-
|
2859
|
+
#### Example: Display current user's workspaces
|
2725
2860
|
|
2861
|
+
```bash
|
2862
|
+
ascli aoc user workspaces list
|
2726
2863
|
```
|
2727
|
-
|
2864
|
+
|
2865
|
+
```output
|
2728
2866
|
:......:............................:
|
2729
2867
|
: id : name :
|
2730
2868
|
:......:............................:
|
@@ -2734,45 +2872,43 @@ ascli aoc user workspaces
|
|
2734
2872
|
:......:............................:
|
2735
2873
|
```
|
2736
2874
|
|
2737
|
-
|
2875
|
+
#### Example: Create a sub access key in a "node"
|
2738
2876
|
|
2739
2877
|
Creation of a sub-access key is like creation of access key with the following difference: authentication to node API is made with accesskey (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
|
2740
2878
|
|
2741
|
-
```
|
2879
|
+
```bash
|
2742
2880
|
ascli aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create --value=@json:'{"storage":{"path":"/folder1"}}'
|
2743
2881
|
```
|
2744
2882
|
|
2745
|
-
|
2883
|
+
#### Example: Display transfer events (ops/transfer)
|
2746
2884
|
|
2747
|
-
```
|
2885
|
+
```bash
|
2748
2886
|
ascli aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
|
2749
2887
|
```
|
2750
2888
|
|
2751
2889
|
Examples of query (TODO: cleanup):
|
2752
2890
|
|
2753
|
-
```
|
2891
|
+
```javascript
|
2754
2892
|
{"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
|
2893
|
+
```
|
2755
2894
|
|
2895
|
+
```javascript
|
2756
2896
|
{"tag":"aspera.files.package_id=LA8OU3p8w"}
|
2757
|
-
|
2758
|
-
# filter= 'id', 'short_summary', or 'summary'
|
2759
|
-
# count=nnn
|
2760
|
-
# tag=x.y.z%3Dvalue
|
2761
|
-
# iteration_token=nnn
|
2762
|
-
# after_time=2016-05-01T23:53:09Z
|
2763
|
-
# active_only=true|false
|
2764
2897
|
```
|
2765
2898
|
|
2766
|
-
|
2899
|
+
#### Example: Display node events (events)
|
2767
2900
|
|
2768
|
-
```
|
2901
|
+
```bash
|
2769
2902
|
ascli aoc admin res node --secret=_secret_ v3 events
|
2770
2903
|
```
|
2771
2904
|
|
2772
|
-
|
2905
|
+
#### Example: Display members of a workspace
|
2773
2906
|
|
2774
|
-
```
|
2907
|
+
```javascript
|
2775
2908
|
ascli aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
2909
|
+
```
|
2910
|
+
|
2911
|
+
```output
|
2776
2912
|
:.............:.........:..................................:
|
2777
2913
|
: member_type : manager : member.email :
|
2778
2914
|
:.............:.........:..................................:
|
@@ -2787,54 +2923,57 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
|
|
2787
2923
|
|
2788
2924
|
other query parameters:
|
2789
2925
|
|
2790
|
-
```
|
2926
|
+
```javascript
|
2791
2927
|
{"workspace_membership_through":true,"include_indirect":true}
|
2792
2928
|
```
|
2793
2929
|
|
2794
|
-
|
2930
|
+
#### Example: <a id="aoc_sample_member"></a>add all members of a workspace to another workspace
|
2795
2931
|
|
2796
2932
|
a- Get id of first workspace
|
2797
2933
|
|
2798
|
-
```
|
2934
|
+
```bash
|
2799
2935
|
WS1='First Workspace'
|
2800
2936
|
WS1ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
|
2801
2937
|
```
|
2802
2938
|
|
2803
2939
|
b- Get id of second workspace
|
2804
2940
|
|
2805
|
-
```
|
2941
|
+
```bash
|
2806
2942
|
WS2='Second Workspace'
|
2807
2943
|
WS2ID=$(ascli aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
|
2808
2944
|
```
|
2809
2945
|
|
2810
2946
|
c- Extract membership information
|
2811
2947
|
|
2812
|
-
```
|
2948
|
+
```bash
|
2813
2949
|
ascli aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
|
2814
2950
|
```
|
2815
2951
|
|
2816
2952
|
d- Convert to creation data for second workspace:
|
2817
2953
|
|
2818
|
-
```
|
2954
|
+
```bash
|
2819
2955
|
grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
|
2820
2956
|
```
|
2821
2957
|
|
2822
2958
|
or, using jq:
|
2823
2959
|
|
2824
|
-
```
|
2960
|
+
```bash
|
2825
2961
|
jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
|
2826
2962
|
```
|
2827
2963
|
|
2828
2964
|
e- Add members to second workspace
|
2829
2965
|
|
2830
|
-
```
|
2966
|
+
```bash
|
2831
2967
|
ascli aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
|
2832
2968
|
```
|
2833
2969
|
|
2834
|
-
|
2970
|
+
#### Example: Get users who did not log since a date
|
2835
2971
|
|
2836
|
-
```
|
2972
|
+
```javascript
|
2837
2973
|
ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
2974
|
+
```
|
2975
|
+
|
2976
|
+
```output
|
2838
2977
|
:...............................:
|
2839
2978
|
: email :
|
2840
2979
|
:...............................:
|
@@ -2843,20 +2982,23 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
|
|
2843
2982
|
:...............................:
|
2844
2983
|
```
|
2845
2984
|
|
2846
|
-
|
2985
|
+
#### Example: List "Limited" users
|
2847
2986
|
|
2848
|
-
```
|
2987
|
+
```javascript
|
2849
2988
|
ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
|
2850
2989
|
```
|
2851
2990
|
|
2852
|
-
|
2991
|
+
#### Example: Perform a multi Gbps transfer between two remote shared folders
|
2853
2992
|
|
2854
2993
|
In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
|
2855
2994
|
|
2856
2995
|
First, setup the environment (skip if already done)
|
2857
2996
|
|
2858
|
-
```
|
2997
|
+
```bash
|
2859
2998
|
ascli conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
|
2999
|
+
```
|
3000
|
+
|
3001
|
+
```output
|
2860
3002
|
Detected: Aspera on Cloud
|
2861
3003
|
Preparing preset: aoc_sedemo
|
2862
3004
|
Using existing key:
|
@@ -2875,7 +3017,7 @@ Setting config preset as default for aspera
|
|
2875
3017
|
saving config file
|
2876
3018
|
Done.
|
2877
3019
|
You can test with:
|
2878
|
-
ascli aoc user
|
3020
|
+
ascli aoc user profile show
|
2879
3021
|
```
|
2880
3022
|
|
2881
3023
|
This creates the option preset "aoc_<org name>" to allow seamless command line access and sets it as default for aspera on cloud.
|
@@ -2884,21 +3026,27 @@ Then, create two shared folders located in two regions, in your files home, in a
|
|
2884
3026
|
|
2885
3027
|
Then, transfer between those:
|
2886
3028
|
|
2887
|
-
```
|
3029
|
+
```javascript
|
2888
3030
|
ascli -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
2889
3031
|
```
|
2890
3032
|
|
2891
|
-
|
3033
|
+
#### Example: create registration key to register a node
|
2892
3034
|
|
2893
|
-
```
|
3035
|
+
```javascript
|
2894
3036
|
ascli aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
3037
|
+
```
|
3038
|
+
|
3039
|
+
```output
|
2895
3040
|
jfqslfdjlfdjfhdjklqfhdkl
|
2896
3041
|
```
|
2897
3042
|
|
2898
|
-
|
3043
|
+
#### Example: delete all registration keys
|
2899
3044
|
|
2900
|
-
```
|
3045
|
+
```bash
|
2901
3046
|
ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res client delete --bulk=yes --id=@lines:@stdin:
|
3047
|
+
```
|
3048
|
+
|
3049
|
+
```output
|
2902
3050
|
+-----+---------+
|
2903
3051
|
| id | status |
|
2904
3052
|
+-----+---------+
|
@@ -2909,7 +3057,7 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
|
|
2909
3057
|
+-----+---------+
|
2910
3058
|
```
|
2911
3059
|
|
2912
|
-
|
3060
|
+
#### Example: Create a node
|
2913
3061
|
|
2914
3062
|
AoC nodes as actually composed with two related entities:
|
2915
3063
|
|
@@ -2923,7 +3071,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
2923
3071
|
|
2924
3072
|
* create the access key on ATS
|
2925
3073
|
|
2926
|
-
```
|
3074
|
+
```javascript
|
2927
3075
|
ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
|
2928
3076
|
```
|
2929
3077
|
|
@@ -2931,79 +3079,90 @@ Take a note of the randomly generated `id` and `secret`.
|
|
2931
3079
|
|
2932
3080
|
* Retrieve the ATS node address
|
2933
3081
|
|
2934
|
-
```
|
2935
|
-
ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv
|
3082
|
+
```bash
|
3083
|
+
ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
|
2936
3084
|
```
|
2937
3085
|
|
2938
3086
|
* Create the node entity
|
2939
3087
|
|
2940
|
-
```
|
3088
|
+
```javascript
|
2941
3089
|
ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
|
2942
3090
|
```
|
2943
3091
|
|
2944
3092
|
Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
|
2945
3093
|
|
2946
|
-
###
|
2947
|
-
|
2948
|
-
First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
2949
|
-
(To find out available filters, consult the API definition, or use the web interface in developer mode).
|
2950
|
-
|
2951
|
-
Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
|
2952
|
-
|
2953
|
-
```
|
2954
|
-
shbxid=$(ascli aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
|
2955
|
-
|
2956
|
-
ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2957
|
-
```
|
2958
|
-
|
2959
|
-
## Packages
|
3094
|
+
### Packages
|
2960
3095
|
|
2961
3096
|
The webmail-like application.
|
2962
3097
|
|
2963
|
-
|
3098
|
+
#### Send a Package
|
2964
3099
|
|
2965
|
-
|
3100
|
+
General syntax:
|
2966
3101
|
|
2967
|
-
```
|
3102
|
+
```bash
|
2968
3103
|
ascli aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
|
2969
3104
|
```
|
2970
3105
|
|
2971
3106
|
Notes:
|
2972
3107
|
|
2973
3108
|
* The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
2974
|
-
* List allowed shared inbox destinations with: `ascli aoc
|
2975
|
-
* Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
3109
|
+
* List allowed shared inbox destinations with: `ascli aoc packages shared_inboxes list`
|
3110
|
+
* Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
2976
3111
|
* Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
2977
3112
|
* or just names: `"recipients":[{"The Dest"}]` . ascli will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
|
2978
3113
|
* If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
|
2979
3114
|
* if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
|
2980
3115
|
* if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
2981
3116
|
|
2982
|
-
|
3117
|
+
#### Example: Send a package with one file to two users, using their email
|
3118
|
+
|
3119
|
+
```javascript
|
3120
|
+
ascli aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
|
3121
|
+
```
|
2983
3122
|
|
2984
|
-
|
3123
|
+
#### Example: Send a package to a shared inbox with metadata
|
2985
3124
|
|
3125
|
+
```javascript
|
3126
|
+
ascli aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
|
2986
3127
|
```
|
2987
|
-
|
3128
|
+
|
3129
|
+
It is also possible to use identifiers and API parameters:
|
3130
|
+
|
3131
|
+
```javascript
|
3132
|
+
ascli aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
2988
3133
|
```
|
2989
3134
|
|
2990
|
-
|
3135
|
+
#### Example: List packages in a given shared inbox
|
3136
|
+
|
3137
|
+
When user packages are listed, the following query is used:
|
2991
3138
|
|
3139
|
+
```javascript
|
3140
|
+
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
2992
3141
|
```
|
2993
|
-
|
3142
|
+
|
3143
|
+
To list packages in a shared inbox, the query has to be specified with withe the shared inbox by name or its identifier. Additionnal parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode). The current workspace is added unless specified in the query.
|
3144
|
+
|
3145
|
+
Using shared inbox name:
|
3146
|
+
|
3147
|
+
```javascript
|
3148
|
+
ascli aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2994
3149
|
```
|
2995
3150
|
|
2996
|
-
|
3151
|
+
Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
2997
3152
|
|
3153
|
+
```bash
|
3154
|
+
shbxid=$(ascli aoc packages shared_inboxes show name 'My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
|
2998
3155
|
```
|
2999
|
-
|
3156
|
+
|
3157
|
+
```javascript
|
3158
|
+
ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
3000
3159
|
```
|
3001
3160
|
|
3002
|
-
|
3161
|
+
#### <a id="aoccargo"></a>Receive new packages only (Cargo)
|
3003
3162
|
|
3004
3163
|
It is possible to automatically download new packages, like using Aspera Cargo:
|
3005
3164
|
|
3006
|
-
```
|
3165
|
+
```bash
|
3007
3166
|
ascli aoc packages recv --id=ALL --once-only=yes --lock-port=12345
|
3008
3167
|
```
|
3009
3168
|
|
@@ -3017,38 +3176,38 @@ Typically, one would execute this command on a regular basis, using the method o
|
|
3017
3176
|
* Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
3018
3177
|
* etc...
|
3019
3178
|
|
3020
|
-
|
3179
|
+
### Files
|
3021
3180
|
|
3022
3181
|
Folder sharing app.
|
3023
3182
|
|
3024
|
-
|
3183
|
+
#### Download Files
|
3025
3184
|
|
3026
3185
|
Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
|
3027
3186
|
|
3028
3187
|
* if only one source is provided, it is downloaded
|
3029
3188
|
* if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
|
3030
3189
|
|
3031
|
-
|
3190
|
+
#### Shared folders
|
3032
3191
|
|
3033
3192
|
* list shared folders in node
|
3034
3193
|
|
3035
|
-
```
|
3194
|
+
```bash
|
3036
3195
|
ascli aoc admin res node --id=8669 shared_folders
|
3037
3196
|
```
|
3038
3197
|
|
3039
3198
|
* list shared folders in workspace
|
3040
3199
|
|
3041
|
-
```
|
3200
|
+
```bash
|
3042
3201
|
ascli aoc admin res workspace --id=10818 shared_folders
|
3043
3202
|
```
|
3044
3203
|
|
3045
3204
|
* list members of shared folder
|
3046
3205
|
|
3047
|
-
```
|
3206
|
+
```bash
|
3048
3207
|
ascli aoc admin res node --id=8669 v4 perm 82 show
|
3049
3208
|
```
|
3050
3209
|
|
3051
|
-
|
3210
|
+
#### Cross Organization transfers
|
3052
3211
|
|
3053
3212
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
3054
3213
|
|
@@ -3062,7 +3221,7 @@ Procedure to send a file from org1 to org2:
|
|
3062
3221
|
* Check that access works and locate the destination folder `mydestfolder`
|
3063
3222
|
* execute the following:
|
3064
3223
|
|
3065
|
-
```
|
3224
|
+
```bash
|
3066
3225
|
ascli -Porg1 aoc files node_info /mydestfolder --format=json --display=data | ascli -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
|
3067
3226
|
```
|
3068
3227
|
|
@@ -3078,7 +3237,7 @@ Explanation:
|
|
3078
3237
|
* `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
|
3079
3238
|
* `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
|
3080
3239
|
|
3081
|
-
|
3240
|
+
#### Find Files
|
3082
3241
|
|
3083
3242
|
The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
|
3084
3243
|
|
@@ -3095,29 +3254,29 @@ Examples of expressions: (using like this: `--value=exec:'<expression>'`)
|
|
3095
3254
|
|
3096
3255
|
* Find files more recent than 100 days
|
3097
3256
|
|
3098
|
-
```
|
3257
|
+
```bash
|
3099
3258
|
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
3100
3259
|
```
|
3101
3260
|
|
3102
3261
|
* Find files older than 1 year on a given node and store in file list
|
3103
3262
|
|
3104
|
-
```
|
3263
|
+
```bash
|
3105
3264
|
ascli aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
|
3106
3265
|
```
|
3107
3266
|
|
3108
3267
|
* Delete the files, one by one
|
3109
3268
|
|
3110
|
-
```
|
3269
|
+
```bash
|
3111
3270
|
cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
|
3112
3271
|
```
|
3113
3272
|
|
3114
3273
|
* Delete the files in bulk
|
3115
3274
|
|
3116
|
-
```
|
3275
|
+
```bash
|
3117
3276
|
cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
|
3118
3277
|
```
|
3119
3278
|
|
3120
|
-
|
3279
|
+
## <a id="ats"></a>Plugin: Aspera Transfer Service
|
3121
3280
|
|
3122
3281
|
ATS is usable either :
|
3123
3282
|
|
@@ -3125,15 +3284,18 @@ ATS is usable either :
|
|
3125
3284
|
|
3126
3285
|
* or from an IBM Cloud subscription : ascli ats : use IBM Cloud API key authentication
|
3127
3286
|
|
3128
|
-
|
3287
|
+
### IBM Cloud ATS : creation of api key
|
3129
3288
|
|
3130
3289
|
This section is about using ATS with an IBM cloud subscription.
|
3131
3290
|
If you are using ATS as part of AoC, then authentication is thropugh AoC, not IBM Cloud.
|
3132
3291
|
|
3133
3292
|
First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
|
3134
3293
|
|
3135
|
-
```
|
3294
|
+
```bash
|
3136
3295
|
ibmcloud iam api-key-create mykeyname -d 'my sample key'
|
3296
|
+
```
|
3297
|
+
|
3298
|
+
```output
|
3137
3299
|
OK
|
3138
3300
|
API key mykeyname was created
|
3139
3301
|
|
@@ -3149,22 +3311,40 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
|
|
3149
3311
|
|
3150
3312
|
References:
|
3151
3313
|
|
3152
|
-
|
3153
|
-
|
3314
|
+
* [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
|
3315
|
+
* [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
|
3154
3316
|
|
3155
3317
|
Then, to register the key by default for the ats plugin, create a preset. Execute:
|
3156
3318
|
|
3157
|
-
```
|
3319
|
+
```bash
|
3158
3320
|
ascli config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here_8f8d9fdakjhfsashjk678
|
3321
|
+
```
|
3322
|
+
|
3323
|
+
```bash
|
3159
3324
|
ascli config preset set default ats my_ibm_ats
|
3325
|
+
```
|
3326
|
+
|
3327
|
+
```bash
|
3160
3328
|
ascli ats api_key instances
|
3329
|
+
```
|
3330
|
+
|
3331
|
+
```output
|
3161
3332
|
+--------------------------------------+
|
3162
3333
|
| instance |
|
3163
3334
|
+--------------------------------------+
|
3164
3335
|
| aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee |
|
3165
3336
|
+--------------------------------------+
|
3337
|
+
```
|
3338
|
+
|
3339
|
+
```bash
|
3166
3340
|
ascli config preset update my_ibm_ats --instance=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
|
3341
|
+
```
|
3342
|
+
|
3343
|
+
```bash
|
3167
3344
|
ascli ats api_key create
|
3345
|
+
```
|
3346
|
+
|
3347
|
+
```output
|
3168
3348
|
+--------+----------------------------------------------+
|
3169
3349
|
| key | value |
|
3170
3350
|
+--------+----------------------------------------------+
|
@@ -3174,52 +3354,49 @@ ascli ats api_key create
|
|
3174
3354
|
ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
3175
3355
|
```
|
3176
3356
|
|
3177
|
-
|
3357
|
+
### Misc. Examples
|
3178
3358
|
|
3179
3359
|
Example: create access key on IBM Cloud (softlayer):
|
3180
3360
|
|
3181
|
-
```
|
3361
|
+
```javascript
|
3182
3362
|
ascli ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
|
3183
3363
|
```
|
3184
3364
|
|
3185
3365
|
Example: create access key on AWS:
|
3186
3366
|
|
3187
|
-
```
|
3367
|
+
```javascript
|
3188
3368
|
ascli ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
|
3189
|
-
|
3190
3369
|
```
|
3191
3370
|
|
3192
3371
|
Example: create access key on Azure SAS:
|
3193
3372
|
|
3194
|
-
```
|
3373
|
+
```javascript
|
3195
3374
|
ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
|
3196
|
-
|
3197
3375
|
```
|
3198
3376
|
|
3199
3377
|
(Note that the blob name is mandatory after server address and before parameters. and that parameter sr=c is mandatory.)
|
3200
3378
|
|
3201
3379
|
Example: create access key on Azure:
|
3202
3380
|
|
3203
|
-
```
|
3381
|
+
```javascript
|
3204
3382
|
ascli ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'
|
3205
|
-
|
3206
3383
|
```
|
3207
3384
|
|
3208
3385
|
delete all my access keys:
|
3209
3386
|
|
3210
|
-
```
|
3387
|
+
```bash
|
3211
3388
|
for k in $(ascli ats access_key list --field=id --format=csv);do ascli ats access_key id $k delete;done
|
3212
3389
|
```
|
3213
3390
|
|
3214
3391
|
The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
|
3215
3392
|
|
3216
|
-
|
3393
|
+
## Plugin: IBM Aspera High Speed Transfer Server (transfer)
|
3217
3394
|
|
3218
3395
|
This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
|
3219
3396
|
It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
|
3220
3397
|
Modern mode is to use the node API and transfer tokens.
|
3221
3398
|
|
3222
|
-
|
3399
|
+
### Authentication
|
3223
3400
|
|
3224
3401
|
Both password and SSH keys auth are supported.
|
3225
3402
|
|
@@ -3227,7 +3404,7 @@ If username is not provided, the default transfer user `xfer` is used.
|
|
3227
3404
|
|
3228
3405
|
If no SSH password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used:
|
3229
3406
|
|
3230
|
-
```
|
3407
|
+
```javascript
|
3231
3408
|
ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
|
3232
3409
|
```
|
3233
3410
|
|
@@ -3237,7 +3414,7 @@ Each value is a path to a private key and is expanded (`~` is replaced with the
|
|
3237
3414
|
|
3238
3415
|
Examples:
|
3239
3416
|
|
3240
|
-
```
|
3417
|
+
```bash
|
3241
3418
|
ascli server --ssh-keys=~/.ssh/id_rsa
|
3242
3419
|
ascli server --ssh-keys=@list:,~/.ssh/id_rsa
|
3243
3420
|
ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
|
@@ -3248,13 +3425,13 @@ By default the ssh library expect that an ssh-agent is running.
|
|
3248
3425
|
|
3249
3426
|
On Linux, if you get an error message such as:
|
3250
3427
|
|
3251
|
-
```
|
3428
|
+
```bash
|
3252
3429
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: Agent not configured
|
3253
3430
|
```
|
3254
3431
|
|
3255
3432
|
or on Windows:
|
3256
3433
|
|
3257
|
-
```
|
3434
|
+
```bash
|
3258
3435
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
3259
3436
|
```
|
3260
3437
|
|
@@ -3265,17 +3442,17 @@ This means that you don't have such an ssh agent running, then:
|
|
3265
3442
|
* [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
|
3266
3443
|
* To disable use of `ssh-agent`, use the option `ssh_option` like this:
|
3267
3444
|
|
3268
|
-
```
|
3445
|
+
```bash
|
3269
3446
|
ascli server --ssh-options=@ruby:'{use_agent: false}' ...
|
3270
3447
|
```
|
3271
3448
|
|
3272
3449
|
This can also be set as default using a global preset.
|
3273
3450
|
|
3274
|
-
|
3451
|
+
### Example
|
3275
3452
|
|
3276
3453
|
One can test the `server` application using the well known demo server:
|
3277
3454
|
|
3278
|
-
```
|
3455
|
+
```bash
|
3279
3456
|
ascli config initdemo
|
3280
3457
|
ascli server browse /aspera-test-dir-large
|
3281
3458
|
ascli server download /aspera-test-dir-large/200MB
|
@@ -3283,13 +3460,14 @@ ascli server download /aspera-test-dir-large/200MB
|
|
3283
3460
|
|
3284
3461
|
`initdemo` creates a [option preset](#lprt) `demoserver` and set it as default for plugin `server`.
|
3285
3462
|
|
3286
|
-
|
3463
|
+
## Plugin: IBM Aspera High Speed Transfer Server (node)
|
3287
3464
|
|
3288
3465
|
This plugin gives access to capabilities provided by HSTS node API.
|
3289
3466
|
|
3290
|
-
|
3467
|
+
### File Operations
|
3291
3468
|
|
3292
3469
|
It is possible to:
|
3470
|
+
|
3293
3471
|
* browse
|
3294
3472
|
* transfer (upload / download)
|
3295
3473
|
* ...
|
@@ -3299,7 +3477,7 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
3299
3477
|
* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
|
3300
3478
|
* `basic` : transfer spec is created like this:
|
3301
3479
|
|
3302
|
-
```
|
3480
|
+
```javascript
|
3303
3481
|
{
|
3304
3482
|
"remote_host": address of node url,
|
3305
3483
|
"remote_user": "xfer",
|
@@ -3311,29 +3489,29 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
3311
3489
|
|
3312
3490
|
* `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
|
3313
3491
|
|
3314
|
-
|
3492
|
+
### Central
|
3315
3493
|
|
3316
3494
|
The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transferred files.
|
3317
3495
|
|
3318
3496
|
Filtering can be applied:
|
3319
3497
|
|
3320
|
-
```
|
3498
|
+
```bash
|
3321
3499
|
ascli node central file list
|
3322
3500
|
```
|
3323
3501
|
|
3324
3502
|
by providing the `validator` option, offline transfer validation can be done.
|
3325
3503
|
|
3326
|
-
|
3504
|
+
### FASP Stream
|
3327
3505
|
|
3328
3506
|
It is possible to start a FASPStream session using the node API:
|
3329
3507
|
|
3330
3508
|
Use the "node stream create" command, then arguments are provided as a [_transfer-spec_](#transferspec).
|
3331
3509
|
|
3332
|
-
```
|
3510
|
+
```javascript
|
3333
3511
|
ascli node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
|
3334
3512
|
```
|
3335
3513
|
|
3336
|
-
|
3514
|
+
### Watchfolder
|
3337
3515
|
|
3338
3516
|
Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
|
3339
3517
|
|
@@ -3342,52 +3520,58 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
|
|
3342
3520
|
* Start watchd and watchfolderd services running as a system user having access to files
|
3343
3521
|
* configure a watchfolder to define automated transfers
|
3344
3522
|
|
3345
|
-
```
|
3523
|
+
```javascript
|
3346
3524
|
ascli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
3347
3525
|
ascli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
|
3348
3526
|
ascli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
|
3349
3527
|
```
|
3350
3528
|
|
3351
|
-
|
3529
|
+
### Out of Transfer File Validation
|
3352
3530
|
|
3353
3531
|
Follow the Aspera Transfer Server configuration to activate this feature.
|
3354
3532
|
|
3355
|
-
```
|
3533
|
+
```javascript
|
3356
3534
|
ascli node central file list --validator=ascli --data=@json:'{"file_transfer_filter":{"max_result":1}}'
|
3535
|
+
```
|
3536
|
+
|
3537
|
+
```output
|
3357
3538
|
:..............:..............:............:......................................:
|
3358
3539
|
: session_uuid : file_id : status : path :
|
3359
3540
|
:..............:..............:............:......................................:
|
3360
3541
|
: 1a74444c-... : 084fb181-... : validating : /home/xfer.../PKG - my title/200KB.1 :
|
3361
3542
|
:..............:..............:............:......................................:
|
3543
|
+
```
|
3544
|
+
|
3545
|
+
```javascript
|
3362
3546
|
ascli node central file update --validator=ascli --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
|
3547
|
+
```
|
3548
|
+
|
3549
|
+
```output
|
3363
3550
|
updated
|
3364
3551
|
```
|
3365
3552
|
|
3366
|
-
|
3553
|
+
### Example: SHOD to ATS
|
3367
3554
|
|
3368
|
-
Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
|
3369
|
-
transfer files from this third party SHOD instance into our Azure BLOB storage.
|
3370
|
-
Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which
|
3371
|
-
|
3372
|
-
Then create a configuration for the "SHOD" instance in the configuration file: in section
|
3373
|
-
"shares", a configuration named: awsshod.
|
3555
|
+
Scenario: Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
|
3556
|
+
We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
|
3557
|
+
Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which provides access to the node API.
|
3558
|
+
Then create a configuration for the "SHOD" instance in the configuration file: in section "shares", a configuration named: awsshod.
|
3374
3559
|
Create another configuration for the Azure ATS instance: in section "node", named azureats.
|
3375
3560
|
Then execute the following command:
|
3376
3561
|
|
3377
|
-
```
|
3562
|
+
```bash
|
3378
3563
|
ascli node download /share/sourcefile --to-folder=/destinationfolder --preset=awsshod --transfer=node --transfer-info=@preset:azureats
|
3379
3564
|
```
|
3380
3565
|
|
3381
|
-
This will get transfer information from the SHOD instance and tell the Azure ATS instance
|
3382
|
-
to download files.
|
3566
|
+
This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
|
3383
3567
|
|
3384
|
-
|
3568
|
+
### Create access key
|
3385
3569
|
|
3386
|
-
```
|
3570
|
+
```javascript
|
3387
3571
|
ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
|
3388
3572
|
```
|
3389
3573
|
|
3390
|
-
|
3574
|
+
## Plugin: IBM Aspera Faspex5
|
3391
3575
|
|
3392
3576
|
3 authentication methods are supported:
|
3393
3577
|
|
@@ -3408,35 +3592,35 @@ For boot method: (will be removed in future)
|
|
3408
3592
|
|
3409
3593
|
Use it as password and use `--auth=boot`.
|
3410
3594
|
|
3411
|
-
```
|
3595
|
+
```bash
|
3412
3596
|
ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
|
3413
3597
|
```
|
3414
3598
|
|
3415
3599
|
Ready to use Faspex5 with CLI.
|
3416
3600
|
|
3417
|
-
|
3601
|
+
## Plugin: IBM Aspera Faspex (4.x)
|
3418
3602
|
|
3419
3603
|
Notes:
|
3420
3604
|
|
3421
3605
|
* The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
|
3422
3606
|
* For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
|
3423
3607
|
|
3424
|
-
|
3608
|
+
### Listing Packages
|
3425
3609
|
|
3426
3610
|
Command: `faspex package list`
|
3427
3611
|
|
3428
|
-
|
3612
|
+
#### Option `box`
|
3429
3613
|
|
3430
3614
|
By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
|
3431
3615
|
|
3432
|
-
|
3616
|
+
#### Option `recipient`
|
3433
3617
|
|
3434
3618
|
A user can receive a package because the recipient is:
|
3435
3619
|
|
3436
3620
|
* the user himself (default)
|
3437
3621
|
* the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
|
3438
3622
|
|
3439
|
-
|
3623
|
+
#### Option `query`
|
3440
3624
|
|
3441
3625
|
As inboxes may be large, it is possible to use the following query parameters:
|
3442
3626
|
|
@@ -3452,15 +3636,15 @@ The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/cat
|
|
3452
3636
|
|
3453
3637
|
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`). By default page is `0` (`10`), it can be increased to have less calls.
|
3454
3638
|
|
3455
|
-
|
3639
|
+
#### Example: list packages in dropbox
|
3456
3640
|
|
3457
|
-
```
|
3641
|
+
```javascript
|
3458
3642
|
ascli faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
|
3459
3643
|
```
|
3460
3644
|
|
3461
3645
|
List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
|
3462
3646
|
|
3463
|
-
|
3647
|
+
### Receiving a Package
|
3464
3648
|
|
3465
3649
|
The command is `package recv`, possible methods are:
|
3466
3650
|
|
@@ -3468,26 +3652,26 @@ The command is `package recv`, possible methods are:
|
|
3468
3652
|
* provide a public link with option `link`
|
3469
3653
|
* provide a `faspe:` URI with option `link`
|
3470
3654
|
|
3471
|
-
```
|
3655
|
+
```bash
|
3472
3656
|
ascli faspex package recv --id=12345
|
3473
3657
|
ascli faspex package recv --link=faspe://...
|
3474
3658
|
```
|
3475
3659
|
|
3476
3660
|
If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
|
3477
3661
|
|
3478
|
-
```
|
3662
|
+
```bash
|
3479
3663
|
ascli faspex package list --recipient='*thedropboxname'
|
3480
3664
|
```
|
3481
3665
|
|
3482
3666
|
if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
|
3483
3667
|
|
3484
|
-
|
3668
|
+
### Sending a Package
|
3485
3669
|
|
3486
3670
|
The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
|
3487
3671
|
|
3488
3672
|
Example:
|
3489
3673
|
|
3490
|
-
```
|
3674
|
+
```javascript
|
3491
3675
|
ascli faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
3492
3676
|
```
|
3493
3677
|
|
@@ -3498,29 +3682,29 @@ Additional optional parameters in `delivery_info`:
|
|
3498
3682
|
* Package Note: : `"note":"note this and that"`
|
3499
3683
|
* Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
3500
3684
|
|
3501
|
-
|
3685
|
+
### Email notification on transfer
|
3502
3686
|
|
3503
3687
|
Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
|
3504
3688
|
|
3505
3689
|
Example:
|
3506
3690
|
|
3507
|
-
```
|
3691
|
+
```javascript
|
3508
3692
|
ascli faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: Package sent: <%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
|
3509
3693
|
```
|
3510
3694
|
|
3511
3695
|
In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
|
3512
3696
|
|
3513
|
-
|
3697
|
+
### Operation on dropboxes
|
3514
3698
|
|
3515
3699
|
Example:
|
3516
3700
|
|
3517
|
-
```
|
3701
|
+
```javascript
|
3518
3702
|
ascli faspex v4 dropbox create --value=@json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
|
3519
3703
|
ascli faspex v4 dropbox list
|
3520
3704
|
ascli faspex v4 dropbox delete --id=36
|
3521
3705
|
```
|
3522
3706
|
|
3523
|
-
|
3707
|
+
### Remote sources
|
3524
3708
|
|
3525
3709
|
Faspex lacks an API to list the contents of a remote source (available in web UI). To workaround this,
|
3526
3710
|
the node API is used, for this it is required to add a section ":storage" that links
|
@@ -3549,26 +3733,26 @@ The node configuration name is "my_faspex_node" here.
|
|
3549
3733
|
|
3550
3734
|
Note: the v4 API provide an API for nodes and shares.
|
3551
3735
|
|
3552
|
-
|
3736
|
+
### Automated package download (cargo)
|
3553
3737
|
|
3554
3738
|
It is possible to tell `ascli` to download newly received packages, much like the official
|
3555
3739
|
cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on Cloud plugin:
|
3556
3740
|
|
3557
|
-
```
|
3741
|
+
```bash
|
3558
3742
|
ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
|
3559
3743
|
```
|
3560
3744
|
|
3561
|
-
|
3745
|
+
## Plugin: IBM Aspera Shares
|
3562
3746
|
|
3563
3747
|
Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
|
3564
3748
|
|
3565
3749
|
In Shares2, users, groups listing are paged, to display sequential pages:
|
3566
3750
|
|
3567
|
-
```
|
3751
|
+
```bash
|
3568
3752
|
for p in 1 2 3;do ascli shares2 admin users list --value=@json:'{"page":'$p'}';done
|
3569
3753
|
```
|
3570
3754
|
|
3571
|
-
|
3755
|
+
## Plugin: IBM Cloud Object Storage
|
3572
3756
|
|
3573
3757
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
3574
3758
|
It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
|
@@ -3576,7 +3760,7 @@ Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
|
|
3576
3760
|
|
3577
3761
|
There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
|
3578
3762
|
|
3579
|
-
|
3763
|
+
### Using endpoint, apikey and Resource Instance ID (CRN)
|
3580
3764
|
|
3581
3765
|
If you have those parameters already, then following options shall be provided:
|
3582
3766
|
|
@@ -3587,14 +3771,14 @@ If you have those parameters already, then following options shall be provided:
|
|
3587
3771
|
|
3588
3772
|
For example, let us create a default configuration:
|
3589
3773
|
|
3590
|
-
```
|
3774
|
+
```bash
|
3591
3775
|
ascli conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
|
3592
3776
|
ascli conf id default set cos mycos
|
3593
3777
|
```
|
3594
3778
|
|
3595
3779
|
Then, jump to the transfer example.
|
3596
3780
|
|
3597
|
-
|
3781
|
+
### Using service credential file
|
3598
3782
|
|
3599
3783
|
If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
|
3600
3784
|
|
@@ -3604,7 +3788,7 @@ Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
|
|
3604
3788
|
|
3605
3789
|
or using the IBM Cloud CLI:
|
3606
3790
|
|
3607
|
-
```
|
3791
|
+
```bash
|
3608
3792
|
ibmcloud resource service-keys
|
3609
3793
|
ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HOME/service_creds.json
|
3610
3794
|
```
|
@@ -3613,7 +3797,7 @@ ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HO
|
|
3613
3797
|
|
3614
3798
|
It consists in the following structure:
|
3615
3799
|
|
3616
|
-
```
|
3800
|
+
```javascript
|
3617
3801
|
{
|
3618
3802
|
"apikey": "xxxxxxx.....",
|
3619
3803
|
"cos_hmac_keys": {
|
@@ -3643,40 +3827,40 @@ The required options for this method are:
|
|
3643
3827
|
|
3644
3828
|
For example, let us create a default configuration:
|
3645
3829
|
|
3646
|
-
```
|
3830
|
+
```bash
|
3647
3831
|
ascli conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
|
3648
3832
|
ascli conf id default set cos mycos
|
3649
3833
|
```
|
3650
3834
|
|
3651
|
-
|
3835
|
+
### Operations, transfers
|
3652
3836
|
|
3653
3837
|
Let's assume you created a default configuration from once of the two previous steps (else specify the access options on command lines).
|
3654
3838
|
|
3655
3839
|
A subset of `node` plugin operations are supported, basically node API:
|
3656
3840
|
|
3657
|
-
```
|
3841
|
+
```bash
|
3658
3842
|
ascli cos node info
|
3659
3843
|
ascli cos node upload 'faux:///sample1G?1g'
|
3660
3844
|
```
|
3661
3845
|
|
3662
3846
|
Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
|
3663
3847
|
|
3664
|
-
|
3848
|
+
## Plugin: IBM Aspera Sync
|
3665
3849
|
|
3666
3850
|
A basic plugin to start an "async" using `ascli`.
|
3667
3851
|
The main advantage is the possibility to start from ma configuration file, using `ascli` standard options.
|
3668
3852
|
|
3669
|
-
|
3853
|
+
## Plugin: Preview
|
3670
3854
|
|
3671
3855
|
The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
3672
3856
|
This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
|
3673
3857
|
Several parameters can be used to tune several aspects:
|
3674
3858
|
|
3675
|
-
|
3676
|
-
|
3677
|
-
|
3859
|
+
* methods for detection of new files needing generation
|
3860
|
+
* methods for generation of video preview
|
3861
|
+
* parameters for video handling
|
3678
3862
|
|
3679
|
-
|
3863
|
+
### Aspera Server configuration
|
3680
3864
|
|
3681
3865
|
Specify the previews folder as shown in:
|
3682
3866
|
|
@@ -3684,7 +3868,7 @@ Specify the previews folder as shown in:
|
|
3684
3868
|
|
3685
3869
|
By default, the `preview` plugin expects previews to be generated in a folder named `previews` located in the storage root. On the transfer server execute:
|
3686
3870
|
|
3687
|
-
```
|
3871
|
+
```bash
|
3688
3872
|
PATH=/opt/aspera/bin:$PATH
|
3689
3873
|
|
3690
3874
|
asconfigurator -x "server;preview_dir,previews"
|
@@ -3700,7 +3884,7 @@ This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KBytes.
|
|
3700
3884
|
|
3701
3885
|
To change this parameter in `aspera.conf`, use `asconfigurator`. To display the value, use `asuserdata`:
|
3702
3886
|
|
3703
|
-
```
|
3887
|
+
```bash
|
3704
3888
|
asuserdata -a | grep max_request_file_create_size_kb
|
3705
3889
|
|
3706
3890
|
max_request_file_create_size_kb: "1024"
|
@@ -3712,7 +3896,7 @@ If you use a value different than 16777216, then specify it using option `max_si
|
|
3712
3896
|
|
3713
3897
|
Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* while the generator parameter is in *Bytes* (factor of 1024).
|
3714
3898
|
|
3715
|
-
|
3899
|
+
### <a id="prev_ext"></a>External tools: Linux
|
3716
3900
|
|
3717
3901
|
The tool requires the following external tools available in the `PATH`:
|
3718
3902
|
|
@@ -3728,21 +3912,21 @@ Other OSes should work as well, but are note tested.
|
|
3728
3912
|
|
3729
3913
|
To check if all tools are found properly, execute:
|
3730
3914
|
|
3731
|
-
```
|
3915
|
+
```bash
|
3732
3916
|
ascli preview check
|
3733
3917
|
```
|
3734
3918
|
|
3735
|
-
|
3919
|
+
#### mimemagic
|
3736
3920
|
|
3737
3921
|
To benefit from extra mime type detection install gem mimemagic:
|
3738
3922
|
|
3739
|
-
```
|
3923
|
+
```bash
|
3740
3924
|
gem install mimemagic
|
3741
3925
|
```
|
3742
3926
|
|
3743
3927
|
or to install an earlier version if any problem:
|
3744
3928
|
|
3745
|
-
```
|
3929
|
+
```bash
|
3746
3930
|
gem install mimemagic -v '~> 0.3.0'
|
3747
3931
|
```
|
3748
3932
|
|
@@ -3752,21 +3936,21 @@ If not used, Mime type used for conversion is the one provided by the node API.
|
|
3752
3936
|
|
3753
3937
|
If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
|
3754
3938
|
|
3755
|
-
|
3939
|
+
#### Image: ImageMagick and optipng
|
3756
3940
|
|
3757
|
-
```
|
3941
|
+
```bash
|
3758
3942
|
yum install -y ImageMagick optipng
|
3759
3943
|
```
|
3760
3944
|
|
3761
|
-
|
3945
|
+
#### Video: FFmpeg
|
3762
3946
|
|
3763
3947
|
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
3764
3948
|
|
3765
|
-
```
|
3949
|
+
```bash
|
3766
3950
|
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
|
3767
3951
|
```
|
3768
3952
|
|
3769
|
-
|
3953
|
+
#### Office: Unoconv and Libreoffice
|
3770
3954
|
|
3771
3955
|
If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
|
3772
3956
|
|
@@ -3774,13 +3958,13 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
|
|
3774
3958
|
|
3775
3959
|
* CentOS 8
|
3776
3960
|
|
3777
|
-
```
|
3961
|
+
```bash
|
3778
3962
|
dnf install unoconv
|
3779
3963
|
```
|
3780
3964
|
|
3781
3965
|
* Amazon Linux
|
3782
3966
|
|
3783
|
-
```
|
3967
|
+
```bash
|
3784
3968
|
amazon-linux-extras enable libreoffice
|
3785
3969
|
yum clean metadata
|
3786
3970
|
yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
|
@@ -3789,7 +3973,7 @@ mv unoconv /usr/bin
|
|
3789
3973
|
chmod a+x /usr/bin/unoconv
|
3790
3974
|
```
|
3791
3975
|
|
3792
|
-
|
3976
|
+
### Configuration
|
3793
3977
|
|
3794
3978
|
The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
3795
3979
|
|
@@ -3797,7 +3981,7 @@ Like any `ascli` commands, parameters can be passed on command line or using a c
|
|
3797
3981
|
|
3798
3982
|
Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
|
3799
3983
|
|
3800
|
-
```
|
3984
|
+
```bash
|
3801
3985
|
su -s /bin/bash - xfer
|
3802
3986
|
|
3803
3987
|
ascli config preset update previewconf --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
|
@@ -3810,13 +3994,13 @@ Here we assume that Office file generation is disabled, else remove this option.
|
|
3810
3994
|
|
3811
3995
|
One can check if the access key is well configured using:
|
3812
3996
|
|
3813
|
-
```
|
3997
|
+
```bash
|
3814
3998
|
ascli -Ppreviewconf node browse /
|
3815
3999
|
```
|
3816
4000
|
|
3817
4001
|
This shall list the contents of the storage root of the access key.
|
3818
4002
|
|
3819
|
-
|
4003
|
+
### Execution
|
3820
4004
|
|
3821
4005
|
The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
3822
4006
|
It needs to be run on a regular basis to create or update preview files. For that use your best
|
@@ -3826,7 +4010,7 @@ Typically, for "Access key" access, the system/transfer is `xfer`. So, in order
|
|
3826
4010
|
|
3827
4011
|
Lets do a one shot test, using the configuration previously created:
|
3828
4012
|
|
3829
|
-
```
|
4013
|
+
```bash
|
3830
4014
|
su -s /bin/bash - xfer
|
3831
4015
|
|
3832
4016
|
ascli preview scan --overwrite=always
|
@@ -3836,7 +4020,7 @@ When the preview generator is first executed it will create a file: `.aspera_acc
|
|
3836
4020
|
in the previews folder which contains the access key used.
|
3837
4021
|
On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
|
3838
4022
|
|
3839
|
-
|
4023
|
+
### Configuration for Execution in scheduler
|
3840
4024
|
|
3841
4025
|
Here is an example of configuration for use with cron on Linux.
|
3842
4026
|
Adapt the scripts to your own preference.
|
@@ -3847,7 +4031,7 @@ Lets first setup a script that will be used in the scheduler and sets up the env
|
|
3847
4031
|
|
3848
4032
|
Example of startup script `cron_ascli`, which sets the Ruby environment and adds some timeout protection:
|
3849
4033
|
|
3850
|
-
```
|
4034
|
+
```bash
|
3851
4035
|
#!/bin/bash
|
3852
4036
|
# set a timeout protection, just in case
|
3853
4037
|
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
@@ -3858,7 +4042,7 @@ exec timeout ${tmout} ascli "${@}"
|
|
3858
4042
|
|
3859
4043
|
Here the cronjob is created for user `xfer`.
|
3860
4044
|
|
3861
|
-
```
|
4045
|
+
```bash
|
3862
4046
|
crontab<<EOF
|
3863
4047
|
0 * * * * /home/xfer/cron_ascli preview scan --logger=syslog --display=error
|
3864
4048
|
2-59 * * * * /home/xfer/cron_ascli preview trev --logger=syslog --display=error
|
@@ -3867,7 +4051,7 @@ EOF
|
|
3867
4051
|
|
3868
4052
|
Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
|
3869
4053
|
|
3870
|
-
|
4054
|
+
### Candidate detection for creation or update (or deletion)
|
3871
4055
|
|
3872
4056
|
The tool generates preview files using those commands:
|
3873
4057
|
|
@@ -3889,7 +4073,7 @@ Deletion of preview for deleted source files: not implemented yet (TODO).
|
|
3889
4073
|
|
3890
4074
|
If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
|
3891
4075
|
|
3892
|
-
```
|
4076
|
+
```bash
|
3893
4077
|
ascli preview scan --skip-folders=@json:'["/not_here"]'
|
3894
4078
|
```
|
3895
4079
|
|
@@ -3899,20 +4083,20 @@ When scanning the option `value` has the same behavior as for the `node find` co
|
|
3899
4083
|
|
3900
4084
|
For instance to filter out files beginning with `._` do:
|
3901
4085
|
|
3902
|
-
```
|
4086
|
+
```bash
|
3903
4087
|
... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
|
3904
4088
|
```
|
3905
4089
|
|
3906
|
-
|
4090
|
+
### Preview File types
|
3907
4091
|
|
3908
4092
|
Two types of preview can be generated:
|
3909
4093
|
|
3910
|
-
|
3911
|
-
|
4094
|
+
* png: thumbnail
|
4095
|
+
* mp4: video preview (only for video)
|
3912
4096
|
|
3913
4097
|
Use option `skip_format` to skip generation of a format.
|
3914
4098
|
|
3915
|
-
|
4099
|
+
### Supported input Files types
|
3916
4100
|
|
3917
4101
|
The preview generator supports rendering of those file categories:
|
3918
4102
|
|
@@ -3932,7 +4116,7 @@ File type is primarily based on file extension detected by the node API and tran
|
|
3932
4116
|
|
3933
4117
|
The tool can also locally detect the mime type using gem `mimemagic`.
|
3934
4118
|
|
3935
|
-
|
4119
|
+
### Access to original files and preview creation
|
3936
4120
|
|
3937
4121
|
Standard open source tools are used to create thumbnails and video previews.
|
3938
4122
|
Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
|
@@ -3943,8 +4127,7 @@ are directly written to the storage.
|
|
3943
4127
|
|
3944
4128
|
If the preview generator does not have access to files on the file system (it is remote, no mount, or is an object storage), then the original file is first downloaded, then the result is uploaded, use method `remote`.
|
3945
4129
|
|
3946
|
-
|
3947
|
-
# SMTP for email notifications
|
4130
|
+
## SMTP for email notifications
|
3948
4131
|
|
3949
4132
|
Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
|
3950
4133
|
|
@@ -3961,9 +4144,9 @@ The `smtp` option is a hash table (extended value) with the following fields:
|
|
3961
4144
|
<tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
|
3962
4145
|
</table>
|
3963
4146
|
|
3964
|
-
|
4147
|
+
### Example of configuration:
|
3965
4148
|
|
3966
|
-
```
|
4149
|
+
```bash
|
3967
4150
|
ascli config preset set smtp_google server smtp.google.com
|
3968
4151
|
ascli config preset set smtp_google username john@gmail.com
|
3969
4152
|
ascli config preset set smtp_google password P@ssw0rd
|
@@ -3971,24 +4154,24 @@ ascli config preset set smtp_google password P@ssw0rd
|
|
3971
4154
|
|
3972
4155
|
or
|
3973
4156
|
|
3974
|
-
```
|
4157
|
+
```javascript
|
3975
4158
|
ascli config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
|
3976
4159
|
```
|
3977
4160
|
|
3978
4161
|
or
|
3979
4162
|
|
3980
|
-
```
|
4163
|
+
```bash
|
3981
4164
|
ascli config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
|
3982
4165
|
```
|
3983
4166
|
|
3984
4167
|
Set this configuration as global default, for instance:
|
3985
4168
|
|
3986
|
-
```
|
4169
|
+
```bash
|
3987
4170
|
ascli config preset set cli_default smtp @val:@preset:smtp_google
|
3988
4171
|
ascli config preset set default config cli_default
|
3989
4172
|
```
|
3990
4173
|
|
3991
|
-
|
4174
|
+
### Email templates
|
3992
4175
|
|
3993
4176
|
Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
|
3994
4177
|
|
@@ -4002,16 +4185,16 @@ The following variables are defined by default:
|
|
4002
4185
|
|
4003
4186
|
Other variables are defined depending on context.
|
4004
4187
|
|
4005
|
-
|
4188
|
+
### Test
|
4006
4189
|
|
4007
4190
|
Check settings with `smtp_settings` command. Send test email with `email_test`.
|
4008
4191
|
|
4009
|
-
```
|
4192
|
+
```bash
|
4010
4193
|
ascli config --smtp=@preset:smtp_google smtp
|
4011
4194
|
ascli config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
|
4012
4195
|
```
|
4013
4196
|
|
4014
|
-
|
4197
|
+
### Notifications for transfer status
|
4015
4198
|
|
4016
4199
|
An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
|
4017
4200
|
|
@@ -4028,7 +4211,7 @@ The environment provided contains the following additional variables:
|
|
4028
4211
|
|
4029
4212
|
Example of template:
|
4030
4213
|
|
4031
|
-
```
|
4214
|
+
```text
|
4032
4215
|
From: <%=from_name%> <<%=from_email%>>
|
4033
4216
|
To: <<%=to%>>
|
4034
4217
|
Subject: <%=subject%>
|
@@ -4036,7 +4219,7 @@ Subject: <%=subject%>
|
|
4036
4219
|
Transfer is: <%=global_transfer_status%>
|
4037
4220
|
```
|
4038
4221
|
|
4039
|
-
|
4222
|
+
## Tool: `asession`
|
4040
4223
|
|
4041
4224
|
This gem comes with a second executable tool providing a simplified standardized interface
|
4042
4225
|
to start a FASP session: `asession`.
|
@@ -4061,12 +4244,12 @@ During execution, it generates all low level events, one per line, in JSON forma
|
|
4061
4244
|
|
4062
4245
|
Note that there are special "extended" [_transfer-spec_](#transferspec) parameters supported by `asession`:
|
4063
4246
|
|
4064
|
-
|
4065
|
-
|
4247
|
+
* `EX_loglevel` to change log level of the tool
|
4248
|
+
* `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
4066
4249
|
|
4067
4250
|
Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
|
4068
4251
|
|
4069
|
-
|
4252
|
+
### Comparison of interfaces
|
4070
4253
|
|
4071
4254
|
<table>
|
4072
4255
|
<tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
|
@@ -4076,21 +4259,21 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
|
|
4076
4259
|
<tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
|
4077
4260
|
<tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp</td><td>any with ascp</td><td>any with ascp and transferdaemon</td></tr></table>
|
4078
4261
|
|
4079
|
-
|
4262
|
+
### Simple session
|
4080
4263
|
|
4081
|
-
```
|
4264
|
+
```javascript
|
4082
4265
|
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
|
4083
4266
|
|
4084
4267
|
echo "${MY_TSPEC}"|asession
|
4085
4268
|
```
|
4086
4269
|
|
4087
|
-
|
4270
|
+
### Asynchronous commands and Persistent session
|
4088
4271
|
|
4089
4272
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -> `long_parameter`
|
4090
4273
|
|
4091
4274
|
This is particularly useful for a persistent session ( with the [_transfer-spec_](#transferspec) parameter: `"keepalive":true` )
|
4092
4275
|
|
4093
|
-
```
|
4276
|
+
```javascript
|
4094
4277
|
asession
|
4095
4278
|
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
|
4096
4279
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
@@ -4099,13 +4282,13 @@ asession
|
|
4099
4282
|
|
4100
4283
|
(events from FASP are not shown in above example. They would appear after each command)
|
4101
4284
|
|
4102
|
-
|
4285
|
+
### Example of language wrapper
|
4103
4286
|
|
4104
4287
|
Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/aspera)
|
4105
4288
|
|
4106
|
-
|
4289
|
+
### Help
|
4107
4290
|
|
4108
|
-
```
|
4291
|
+
```bash
|
4109
4292
|
asession -h
|
4110
4293
|
USAGE
|
4111
4294
|
asession
|
@@ -4130,9 +4313,9 @@ EXAMPLES
|
|
4130
4313
|
|
4131
4314
|
```
|
4132
4315
|
|
4133
|
-
|
4316
|
+
## Hot folder
|
4134
4317
|
|
4135
|
-
|
4318
|
+
### Requirements
|
4136
4319
|
|
4137
4320
|
`ascli` maybe used as a simple hot folder engine. A hot folder being defined as a tool that:
|
4138
4321
|
|
@@ -4144,7 +4327,7 @@ EXAMPLES
|
|
4144
4327
|
|
4145
4328
|
In addition: the detection should be made "continuously" or on specific time/date.
|
4146
4329
|
|
4147
|
-
|
4330
|
+
### Setup procedure
|
4148
4331
|
|
4149
4332
|
The general idea is to rely on :
|
4150
4333
|
|
@@ -4152,7 +4335,7 @@ The general idea is to rely on :
|
|
4152
4335
|
* take advantage of `ascli` configuration capabilities and server side knowledge
|
4153
4336
|
* the OS scheduler for reliability and continuous operation
|
4154
4337
|
|
4155
|
-
|
4338
|
+
#### ascp features
|
4156
4339
|
|
4157
4340
|
Interesting ascp features are found in its arguments: (see ascp manual):
|
4158
4341
|
|
@@ -4167,31 +4350,33 @@ Note that:
|
|
4167
4350
|
* most, but not all native ascp arguments are available as standard transfer_spec parameters
|
4168
4351
|
* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
|
4169
4352
|
|
4170
|
-
|
4353
|
+
#### server side and configuration
|
4171
4354
|
|
4172
4355
|
Virtually any transfer on a "repository" on a regular basis might emulate a hot folder. Note that file detection is not based on events (inotify, etc...), but on a stateless scan on source side.
|
4173
4356
|
|
4174
4357
|
Note: parameters may be saved in a [option preset](#lprt) and used with `-P`.
|
4175
4358
|
|
4176
|
-
|
4359
|
+
#### Scheduling
|
4177
4360
|
|
4178
4361
|
Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [_Scheduling_](#_scheduling_).
|
4179
4362
|
|
4180
|
-
|
4363
|
+
### Example: upload folder
|
4181
4364
|
|
4182
|
-
```
|
4365
|
+
```javascript
|
4183
4366
|
ascli server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
|
4184
|
-
|
4185
4367
|
```
|
4186
4368
|
|
4187
4369
|
The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
4188
4370
|
|
4189
|
-
|
4371
|
+
## Health check and Nagios
|
4190
4372
|
|
4191
4373
|
Most plugin provide a `health` command that will check the health status of the application. Example:
|
4192
4374
|
|
4193
|
-
```
|
4375
|
+
```bash
|
4194
4376
|
ascli console health
|
4377
|
+
```
|
4378
|
+
|
4379
|
+
```output
|
4195
4380
|
+--------+-------------+------------+
|
4196
4381
|
| status | component | message |
|
4197
4382
|
+--------+-------------+------------+
|
@@ -4207,14 +4392,23 @@ Typically, the health check uses the REST API of the application with the follow
|
|
4207
4392
|
|
4208
4393
|
`ascli` can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
|
4209
4394
|
|
4210
|
-
```
|
4395
|
+
```bash
|
4211
4396
|
ascli server health transfer --to-folder=/Upload --format=nagios --progress=none
|
4397
|
+
```
|
4398
|
+
|
4399
|
+
```output
|
4212
4400
|
OK - [transfer:ok]
|
4401
|
+
```
|
4402
|
+
|
4403
|
+
```bash
|
4213
4404
|
ascli server health asctlstatus --cmd_prefix='sudo ' --format=nagios
|
4405
|
+
```
|
4406
|
+
|
4407
|
+
```output
|
4214
4408
|
OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
|
4215
4409
|
```
|
4216
4410
|
|
4217
|
-
|
4411
|
+
## Ruby Module: `Aspera`
|
4218
4412
|
|
4219
4413
|
Main components:
|
4220
4414
|
|
@@ -4224,18 +4418,20 @@ Main components:
|
|
4224
4418
|
|
4225
4419
|
A working example can be found in the gem, example:
|
4226
4420
|
|
4227
|
-
```
|
4421
|
+
```bash
|
4228
4422
|
ascli config gem_path
|
4423
|
+
```
|
4424
|
+
|
4425
|
+
```bash
|
4229
4426
|
cat $(ascli config gem_path)/../examples/transfer.rb
|
4230
4427
|
```
|
4231
4428
|
|
4232
|
-
This sample code shows some example of use of the API as well as
|
4233
|
-
REST API.
|
4429
|
+
This sample code shows some example of use of the API as well as REST API.
|
4234
4430
|
Note: although nice, it's probably a good idea to use RestClient for REST.
|
4235
4431
|
|
4236
4432
|
Example of use of the API of Aspera on Cloud:
|
4237
4433
|
|
4238
|
-
```
|
4434
|
+
```ruby
|
4239
4435
|
require 'aspera/aoc'
|
4240
4436
|
|
4241
4437
|
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/ascli/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
@@ -4243,7 +4439,9 @@ aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user
|
|
4243
4439
|
aoc.read('self')
|
4244
4440
|
```
|
4245
4441
|
|
4246
|
-
|
4442
|
+
<https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
|
4443
|
+
|
4444
|
+
## History
|
4247
4445
|
|
4248
4446
|
When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
|
4249
4447
|
|
@@ -4259,213 +4457,228 @@ So, it evolved into `ascli`:
|
|
4259
4457
|
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
|
4260
4458
|
* `ruby` is consistent with other Aspera products
|
4261
4459
|
|
4262
|
-
|
4460
|
+
## Changes (Release notes)
|
4461
|
+
|
4462
|
+
* 4.6.0
|
4463
|
+
|
4464
|
+
* new: command `conf plugin create`
|
4465
|
+
* new: global option `plugin_folder`
|
4466
|
+
* new: global option `transpose_single`
|
4467
|
+
* new: simplified metadata passing for shared inbox package creation in AoC
|
4468
|
+
* change: (break) command `aoc packages shared_inboxes list` replaces `aoc user shared_inboxes`
|
4469
|
+
* change: (break) command `aoc user profile` replaces `aoc user info`
|
4470
|
+
* change: (break) command `aoc user workspaces list` replaces `aoc user workspaces`
|
4471
|
+
* change: (break) command `aoc user workspaces current` replaces `aoc workspace`
|
4472
|
+
* change: (break) command `conf plugin list` replaces `conf plugins`
|
4473
|
+
* change: (break) command `conf connect` simplified
|
4474
|
+
* fix: #60 ascli executable was not installed by default in 4.5.0
|
4475
|
+
* fix: add password hiding case in logs
|
4263
4476
|
|
4264
4477
|
* 4.5.0
|
4265
4478
|
|
4266
|
-
|
4267
|
-
|
4268
|
-
|
4269
|
-
|
4270
|
-
|
4271
|
-
|
4272
|
-
|
4273
|
-
|
4274
|
-
|
4275
|
-
|
4276
|
-
|
4277
|
-
|
4479
|
+
* new: support transfer agent: [Transfer SDK](#agt_trsdk)
|
4480
|
+
* new: support [http socket options](#http_options)
|
4481
|
+
* new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
|
4482
|
+
* new: `config vault` supports encrypted passwords, also macos keychain
|
4483
|
+
* new: `config preset` command for consistency with id
|
4484
|
+
* new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
|
4485
|
+
* change: when using wss, use [ruby's CA certs](#certificates)
|
4486
|
+
* change: unexpected parameter makes exit code not zero
|
4487
|
+
* change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
|
4488
|
+
* change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
|
4489
|
+
* change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
|
4490
|
+
* fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
|
4278
4491
|
|
4279
4492
|
* 4.4.0
|
4280
4493
|
|
4281
|
-
|
4282
|
-
|
4283
|
-
|
4284
|
-
|
4285
|
-
|
4286
|
-
|
4287
|
-
|
4494
|
+
* new: `aoc packages list` add possibility to add filter with option `query`
|
4495
|
+
* new: `aoc admin res xxx list` now get all items by default #50
|
4496
|
+
* new: `preset` option can specify name or hash value
|
4497
|
+
* new: `node` plugin accepts bearer token and access key as credential
|
4498
|
+
* new: `node` option `token_type` allows using basic token in addition to aspera type.
|
4499
|
+
* change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
|
4500
|
+
* change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
|
4288
4501
|
|
4289
4502
|
* 4.3.0
|
4290
4503
|
|
4291
|
-
|
4292
|
-
|
4504
|
+
* new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
|
4505
|
+
* new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
|
4293
4506
|
|
4294
4507
|
* 4.2.2
|
4295
4508
|
|
4296
|
-
|
4297
|
-
|
4298
|
-
|
4299
|
-
|
4300
|
-
|
4301
|
-
|
4302
|
-
|
4303
|
-
|
4304
|
-
|
4509
|
+
* new: `faspex package list` retrieves the whole list, not just first page
|
4510
|
+
* new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
|
4511
|
+
* new: the error "Remote host is not who we expected" displays a special remediation message
|
4512
|
+
* new: `conf ascp spec` displays supported transfer spec
|
4513
|
+
* new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
|
4514
|
+
* fix: space character in `faspe:` url are precent encoded if needed
|
4515
|
+
* fix: `preview scan`: if file_id is unknown, ignore and continue scan
|
4516
|
+
* change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then `ascli` exits with code 1 (instead of zero=success)
|
4517
|
+
* change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
|
4305
4518
|
|
4306
4519
|
* 4.2.1
|
4307
4520
|
|
4308
|
-
|
4309
|
-
|
4521
|
+
* new: command `faspex package recv` supports link of type: `faspe:`
|
4522
|
+
* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
|
4310
4523
|
|
4311
4524
|
* 4.2.0
|
4312
4525
|
|
4313
|
-
|
4314
|
-
|
4315
|
-
|
4316
|
-
|
4317
|
-
|
4318
|
-
|
4319
|
-
|
4320
|
-
|
4321
|
-
|
4322
|
-
|
4526
|
+
* new: command `aoc remind` to receive organization membership by email
|
4527
|
+
* new: in `preview` option `value` to filter out on file name
|
4528
|
+
* new: `initdemo` to initialize for demo server
|
4529
|
+
* new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
|
4530
|
+
* fix: on Windows `conf ascp use` expects ascp.exe
|
4531
|
+
* fix: (break) multi_session_threshold is Integer, not String
|
4532
|
+
* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
|
4533
|
+
* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
|
4534
|
+
* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
|
4535
|
+
* change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
|
4323
4536
|
|
4324
4537
|
* 4.1.0
|
4325
4538
|
|
4326
|
-
|
4327
|
-
|
4328
|
-
|
4329
|
-
|
4330
|
-
|
4331
|
-
|
4332
|
-
|
4333
|
-
|
4334
|
-
|
4335
|
-
|
4336
|
-
|
4337
|
-
|
4539
|
+
* fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
|
4540
|
+
* new: update documentation with regard to offline and docker installation
|
4541
|
+
* new: renamed command `nagios_check` to `health`
|
4542
|
+
* new: agent `http_gw` now supports upload
|
4543
|
+
* new: added option `sdk_url` to install SDK from local file for offline install
|
4544
|
+
* new: check new gem version periodically
|
4545
|
+
* new: the --fields= option, support -_fieldname_ to remove a field from default fields
|
4546
|
+
* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
|
4547
|
+
* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
|
4548
|
+
* new: AoC a password can be provided for a public link
|
4549
|
+
* new: `conf doc` take an optional parameter to go to a section
|
4550
|
+
* new: initial support for Faspex 5 Beta 1
|
4338
4551
|
|
4339
4552
|
* 4.0.0
|
4340
4553
|
|
4341
|
-
|
4342
|
-
|
4343
|
-
|
4344
|
-
|
4345
|
-
|
4346
|
-
|
4347
|
-
|
4348
|
-
|
4554
|
+
* now available as open source at [https://github.com/IBM/aspera-cli](https://github.com/IBM/aspera-cli) with general cleanup
|
4555
|
+
* changed default tool name from `mlia` to `ascli`
|
4556
|
+
* changed `aspera` command to `aoc`
|
4557
|
+
* changed gem name from `asperalm` to `aspera-cli`
|
4558
|
+
* changed module name from `Asperalm` to `Aspera`
|
4559
|
+
* removed command `folder` in `preview`, merged to `scan`
|
4560
|
+
* persistency files go to sub folder instead of main folder
|
4561
|
+
* added possibility to install SDK: `config ascp install`
|
4349
4562
|
|
4350
4563
|
* 0.11.8
|
4351
4564
|
|
4352
|
-
|
4565
|
+
* Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
|
4353
4566
|
|
4354
4567
|
* 0.11.7
|
4355
4568
|
|
4356
|
-
|
4357
|
-
|
4358
|
-
|
4359
|
-
|
4360
|
-
|
4361
|
-
|
4569
|
+
* rework on rest call error handling
|
4570
|
+
* use option `display` with value `data` to remove out of extraneous information
|
4571
|
+
* fixed option `lock_port` not working
|
4572
|
+
* generate special icon if preview failed
|
4573
|
+
* possibility to choose transfer progress bar type with option `progress`
|
4574
|
+
* AoC package creation now output package id
|
4362
4575
|
|
4363
4576
|
* 0.11.6
|
4364
4577
|
|
4365
|
-
|
4366
|
-
|
4367
|
-
|
4368
|
-
|
4369
|
-
|
4370
|
-
|
4371
|
-
|
4578
|
+
* orchestrator : added more choice in auth type
|
4579
|
+
* preview: cleanup in generator (removed and renamed parameters)
|
4580
|
+
* preview: better documentation
|
4581
|
+
* preview: animated thumbnails for video (option: `video_png_conv=animated`)
|
4582
|
+
* preview: new event trigger: `trevents` (`events` seems broken)
|
4583
|
+
* preview: unique tmp folder to avoid clash of multiple instances
|
4584
|
+
* repo: added template for secrets used for testing
|
4372
4585
|
|
4373
4586
|
* 0.11.5
|
4374
4587
|
|
4375
|
-
|
4376
|
-
|
4377
|
-
|
4378
|
-
|
4379
|
-
|
4380
|
-
|
4588
|
+
* added option `default_ports` for AoC (see manual)
|
4589
|
+
* allow bulk delete in `aspera files` with option `bulk=yes`
|
4590
|
+
* fix getting connect versions
|
4591
|
+
* added section for Aix
|
4592
|
+
* support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
|
4593
|
+
* added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
|
4381
4594
|
|
4382
4595
|
* 0.11.4
|
4383
4596
|
|
4384
|
-
|
4597
|
+
* possibility to give shared inbox name when sending a package (else use id and type)
|
4385
4598
|
|
4386
4599
|
* 0.11.3
|
4387
4600
|
|
4388
|
-
|
4601
|
+
* minor fixes on multi-session: avoid exception on progress bar
|
4389
4602
|
|
4390
4603
|
* 0.11.2
|
4391
4604
|
|
4392
|
-
|
4605
|
+
* fixes on multi-session: progress bat and transfer spec param for "direct"
|
4393
4606
|
|
4394
4607
|
* 0.11.1
|
4395
4608
|
|
4396
|
-
|
4609
|
+
* enhanced short_link creation commands (see examples)
|
4397
4610
|
|
4398
4611
|
* 0.11
|
4399
4612
|
|
4400
|
-
|
4613
|
+
* add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
|
4401
4614
|
|
4402
4615
|
* 0.10.18
|
4403
4616
|
|
4404
|
-
|
4617
|
+
* new option in. `server` : `ssh_options`
|
4405
4618
|
|
4406
4619
|
* 0.10.17
|
4407
4620
|
|
4408
|
-
|
4409
|
-
|
4621
|
+
* fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
|
4622
|
+
* new modifier: `@list:<separator>val1<separator>...`
|
4410
4623
|
|
4411
4624
|
* 0.10.16
|
4412
4625
|
|
4413
|
-
|
4626
|
+
* added list of shared inboxes in workspace (or global), use `--query=@json:'{}'`
|
4414
4627
|
|
4415
4628
|
* 0.10.15
|
4416
4629
|
|
4417
|
-
|
4418
|
-
|
4630
|
+
* in case of command line error, display the error cause first, and non-parsed argument second
|
4631
|
+
* AoC : Activity / Analytics
|
4419
4632
|
|
4420
4633
|
* 0.10.14
|
4421
4634
|
|
4422
|
-
|
4635
|
+
* added missing bss plugin
|
4423
4636
|
|
4424
4637
|
* 0.10.13
|
4425
4638
|
|
4426
|
-
|
4639
|
+
* added Faspex5 (use option `value` to give API arguments)
|
4427
4640
|
|
4428
4641
|
* 0.10.12
|
4429
4642
|
|
4430
|
-
|
4431
|
-
|
4432
|
-
|
4433
|
-
|
4434
|
-
|
4435
|
-
|
4643
|
+
* added support for AoC node registration keys
|
4644
|
+
* replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
|
4645
|
+
* Transfer agent is no more a Singleton instance, but only one is used in CLI
|
4646
|
+
* `@incps` : new extended value modifier
|
4647
|
+
* ATS: no more provides access keys secrets: now user must provide it
|
4648
|
+
* begin work on "aoc" transfer agent
|
4436
4649
|
|
4437
4650
|
* 0.10.11
|
4438
4651
|
|
4439
|
-
|
4652
|
+
* minor refactor and fixes
|
4440
4653
|
|
4441
4654
|
* 0.10.10
|
4442
4655
|
|
4443
|
-
|
4656
|
+
* fix on documentation
|
4444
4657
|
|
4445
4658
|
* 0.10.9.1
|
4446
4659
|
|
4447
|
-
|
4448
|
-
|
4449
|
-
|
4660
|
+
* add total number of items for AoC resource list
|
4661
|
+
* better gem version dependency (and fixes to support Ruby 2.0.0)
|
4662
|
+
* removed aoc search_nodes
|
4450
4663
|
|
4451
4664
|
* 0.10.8
|
4452
4665
|
|
4453
|
-
|
4454
|
-
|
4455
|
-
|
4666
|
+
* removed option: `fasp_proxy`, use pseudo transfer spec parameter: `EX_fasp_proxy_url`
|
4667
|
+
* removed option: `http_proxy`, use pseudo transfer spec parameter: `EX_http_proxy_url`
|
4668
|
+
* several other changes..
|
4456
4669
|
|
4457
4670
|
* 0.10.7
|
4458
4671
|
|
4459
|
-
|
4672
|
+
* fix: ascli fails when username cannot be computed on Linux.
|
4460
4673
|
|
4461
4674
|
* 0.10.6
|
4462
4675
|
|
4463
|
-
|
4464
|
-
|
4676
|
+
* FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
|
4677
|
+
* gem version requirements made more open
|
4465
4678
|
|
4466
4679
|
* 0.10.5
|
4467
4680
|
|
4468
|
-
|
4681
|
+
* fix faspex package receive command not working
|
4469
4682
|
|
4470
4683
|
* 0.10.4
|
4471
4684
|
|
@@ -4474,220 +4687,218 @@ So, it evolved into `ascli`:
|
|
4474
4687
|
|
4475
4688
|
* 0.10.3
|
4476
4689
|
|
4477
|
-
|
4690
|
+
* included user name in oauth bearer token cache for AoC when JWT is used.
|
4478
4691
|
|
4479
4692
|
* 0.10.2
|
4480
4693
|
|
4481
|
-
|
4482
|
-
|
4483
|
-
|
4694
|
+
* updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
|
4695
|
+
* added doc for "cargo" like actions
|
4696
|
+
* added doc for multi-session
|
4484
4697
|
|
4485
4698
|
* 0.10.1
|
4486
4699
|
|
4487
|
-
|
4488
|
-
|
4700
|
+
* AoC and node v4 "browse" works now on non-folder items: file, link
|
4701
|
+
* initial support for AoC automation (do not use yet)
|
4489
4702
|
|
4490
4703
|
* 0.10
|
4491
4704
|
|
4492
|
-
|
4493
|
-
|
4705
|
+
* support for transfer using IBM Cloud Object Storage
|
4706
|
+
* improved `find` action using arbitrary expressions
|
4494
4707
|
|
4495
4708
|
* 0.9.36
|
4496
4709
|
|
4497
|
-
|
4710
|
+
* added option to specify file pair lists
|
4498
4711
|
|
4499
4712
|
* 0.9.35
|
4500
4713
|
|
4501
|
-
|
4502
|
-
|
4714
|
+
* updated plugin `preview` , changed parameter names, added documentation
|
4715
|
+
* fix in `ats` plugin : instance id needed in request header
|
4503
4716
|
|
4504
4717
|
* 0.9.34
|
4505
4718
|
|
4506
|
-
|
4507
|
-
|
4719
|
+
* parser "@preset" can be used again in option "transfer_info"
|
4720
|
+
* some documentation re-organizing
|
4508
4721
|
|
4509
4722
|
* 0.9.33
|
4510
4723
|
|
4511
|
-
|
4512
|
-
|
4513
|
-
|
4514
|
-
|
4724
|
+
* new command to display basic token of node
|
4725
|
+
* new command to display bearer token of node in AoC
|
4726
|
+
* the --fields= option, support +_fieldname_ to add a field to default fields
|
4727
|
+
* many small changes
|
4515
4728
|
|
4516
4729
|
* 0.9.32
|
4517
4730
|
|
4518
|
-
|
4519
|
-
|
4520
|
-
|
4731
|
+
* all Faspex public links are now supported
|
4732
|
+
* removed faspex operation recv_publink
|
4733
|
+
* replaced with option `link` (consistent with AoC)
|
4521
4734
|
|
4522
4735
|
* 0.9.31
|
4523
4736
|
|
4524
|
-
|
4525
|
-
|
4526
|
-
|
4737
|
+
* added more support for public link: receive and send package, to user or dropbox and files view.
|
4738
|
+
* delete expired file lists
|
4739
|
+
* changed text table gem from text-table to terminal-table because it supports multiline values
|
4527
4740
|
|
4528
4741
|
* 0.9.27
|
4529
4742
|
|
4530
|
-
|
4531
|
-
|
4743
|
+
* basic email support with SMTP
|
4744
|
+
* basic proxy auto config support
|
4532
4745
|
|
4533
4746
|
* 0.9.26
|
4534
4747
|
|
4535
|
-
|
4536
|
-
|
4748
|
+
* table display with --fields=ALL now includes all column names from all lines, not only first one
|
4749
|
+
* unprocessed argument shows error even if there is an error beforehand
|
4537
4750
|
|
4538
4751
|
* 0.9.25
|
4539
4752
|
|
4540
|
-
|
4541
|
-
|
4542
|
-
|
4753
|
+
* the option `value` of command `find`, to filter on name, is not optional
|
4754
|
+
* `find` now also reports all types (file, folder, link)
|
4755
|
+
* `find` now is able to report all fields (type, size, etc...)
|
4543
4756
|
|
4544
4757
|
* 0.9.24
|
4545
4758
|
|
4546
|
-
|
4547
|
-
|
4759
|
+
* fix bug where AoC node to node transfer did not work
|
4760
|
+
* fix bug on error if ED25519 private key is defined in .ssh
|
4548
4761
|
|
4549
4762
|
* 0.9.23
|
4550
4763
|
|
4551
|
-
|
4552
|
-
|
4764
|
+
* defined REST error handlers, more error conditions detected
|
4765
|
+
* commands to select specific ascp location
|
4553
4766
|
|
4554
4767
|
* 0.9.21
|
4555
4768
|
|
4556
|
-
|
4557
|
-
|
4769
|
+
* supports simplified wizard using global client
|
4770
|
+
* only ascp binary is required, other SDK (keys) files are now generated
|
4558
4771
|
|
4559
4772
|
* 0.9.20
|
4560
4773
|
|
4561
|
-
|
4562
|
-
|
4563
|
-
|
4774
|
+
* improved wizard (prepare for AoC global client id)
|
4775
|
+
* preview generator: addedoption : --skip-format=<png,mp4>
|
4776
|
+
* removed outdated pictures from this doc
|
4564
4777
|
|
4565
4778
|
* 0.9.19
|
4566
4779
|
|
4567
|
-
|
4780
|
+
* added command aspera bearer --scope=xx
|
4568
4781
|
|
4569
4782
|
* 0.9.18
|
4570
4783
|
|
4571
|
-
|
4784
|
+
* enhanced aspera admin events to support query
|
4572
4785
|
|
4573
4786
|
* 0.9.16
|
4574
4787
|
|
4575
|
-
|
4576
|
-
|
4788
|
+
* AoC transfers are now reported in activity app
|
4789
|
+
* new interface for Rest class authentication (keep backward compatibility)
|
4577
4790
|
|
4578
4791
|
* 0.9.15
|
4579
4792
|
|
4580
|
-
|
4581
|
-
|
4793
|
+
* new feature: "find" command in aspera files
|
4794
|
+
* sample code for transfer API
|
4582
4795
|
|
4583
4796
|
* 0.9.12
|
4584
4797
|
|
4585
|
-
|
4586
|
-
|
4798
|
+
* add nagios commands
|
4799
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
4587
4800
|
|
4588
4801
|
* 0.9.11
|
4589
4802
|
|
4590
|
-
|
4591
|
-
|
4803
|
+
* Breaking change: @stdin is now @stdin:
|
4804
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
4592
4805
|
|
4593
4806
|
|
4594
4807
|
* 0.9.10
|
4595
4808
|
|
4596
|
-
|
4597
|
-
|
4598
|
-
|
4599
|
-
|
4809
|
+
* Breaking change: parameter transfer-node becomes more generic: transfer-info
|
4810
|
+
* Display SaaS storage usage with command: aspera admin res node --id=nn info
|
4811
|
+
* cleaner way of specifying source file list for transfers
|
4812
|
+
* Breaking change: replaced download_mode option with http_download action
|
4600
4813
|
|
4601
4814
|
* 0.9.9
|
4602
4815
|
|
4603
|
-
|
4604
|
-
|
4816
|
+
* Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
|
4817
|
+
* Now supports "cargo" for Aspera on Cloud (automatic package download)
|
4605
4818
|
|
4606
4819
|
* 0.9.8
|
4607
4820
|
|
4608
|
-
|
4609
|
-
|
4821
|
+
* Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
|
4822
|
+
* AoC: share to share transfer with command "transfer"
|
4610
4823
|
|
4611
4824
|
* 0.9.7
|
4612
4825
|
|
4613
|
-
|
4614
|
-
|
4615
|
-
|
4616
|
-
|
4617
|
-
|
4826
|
+
* homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
|
4827
|
+
* preview persistency goes to unique file by default
|
4828
|
+
* catch mxf extension in preview as video
|
4829
|
+
* Faspex: possibility to download all packages by specifying id=ALL
|
4830
|
+
* Faspex: to come: cargo-like function to download only new packages with id=NEW
|
4618
4831
|
|
4619
4832
|
* 0.9.6
|
4620
4833
|
|
4621
|
-
|
4622
|
-
|
4834
|
+
* Breaking change: `@param:`is now `@preset:` and is generic
|
4835
|
+
* AoC: added command to display current workspace information
|
4623
4836
|
|
4624
4837
|
* 0.9.5
|
4625
4838
|
|
4626
|
-
|
4627
|
-
|
4839
|
+
* new parameter: new_user_option used to choose between public_link and invite of external users.
|
4840
|
+
* fixed bug in wizard, and wizard uses now product detection
|
4628
4841
|
|
4629
4842
|
* 0.9.4
|
4630
4843
|
|
4631
|
-
|
4632
|
-
|
4633
|
-
|
4844
|
+
* Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
|
4845
|
+
* AoC Package send supports external users
|
4846
|
+
* new command to export AoC config to Aspera CLI config
|
4634
4847
|
|
4635
4848
|
* 0.9.3
|
4636
4849
|
|
4637
|
-
|
4638
|
-
|
4639
|
-
|
4640
|
-
|
4641
|
-
|
4642
|
-
|
4850
|
+
* REST error message show host and code
|
4851
|
+
* option for quiet display
|
4852
|
+
* modified transfer interface and allow token re-generation on error
|
4853
|
+
* async add admin command
|
4854
|
+
* async add db parameters
|
4855
|
+
* Breaking change: new option "sources" to specify files to transfer
|
4643
4856
|
|
4644
4857
|
* 0.9.2
|
4645
4858
|
|
4646
|
-
|
4859
|
+
* Breaking change: changed AoC package creation to match API, see AoC section
|
4647
4860
|
|
4648
4861
|
* 0.9.1
|
4649
4862
|
|
4650
|
-
|
4863
|
+
* Breaking change: changed faspex package creation to match API, see Faspex section
|
4651
4864
|
|
4652
4865
|
* 0.9
|
4653
4866
|
|
4654
|
-
|
4655
|
-
|
4867
|
+
* Renamed the CLI from aslmcli to `ascli`
|
4868
|
+
* Automatic rename and conversion of former config folder from aslmcli to `ascli`
|
4656
4869
|
|
4657
4870
|
* 0.7.6
|
4658
4871
|
|
4659
|
-
|
4872
|
+
* add "sync" plugin
|
4660
4873
|
|
4661
4874
|
* 0.7
|
4662
4875
|
|
4663
|
-
|
4664
|
-
|
4665
|
-
|
4666
|
-
|
4667
|
-
|
4876
|
+
* Breaking change: AoC package recv take option if for package instead of argument.
|
4877
|
+
* Breaking change: Rest class and Oauth class changed init parameters
|
4878
|
+
* AoC: receive package from public link
|
4879
|
+
* select by col value on output
|
4880
|
+
* added rename (AoC, node)
|
4668
4881
|
|
4669
4882
|
* 0.6.19
|
4670
4883
|
|
4671
|
-
|
4672
|
-
|
4673
|
-
|
4674
|
-
|
4675
|
-
|
4676
|
-
|
4677
|
-
|
4678
|
-
|
4679
|
-
* ats api_key list → ats credential list
|
4680
|
-
* ats access_key id xxx → ats access_key --id=xxx
|
4884
|
+
* change: (break) ats server list provisioned → ats cluster list
|
4885
|
+
* change: (break) ats server list clouds → ats cluster clouds
|
4886
|
+
* change: (break) ats server list instance --cloud=x --region=y → ats cluster show --cloud=x --region=y
|
4887
|
+
* change: (break) ats server id xxx → ats cluster show --id=xxx
|
4888
|
+
* change: (break) ats subscriptions → ats credential subscriptions
|
4889
|
+
* change: (break) ats api_key repository list → ats credential cache list
|
4890
|
+
* change: (break) ats api_key list → ats credential list
|
4891
|
+
* change: (break) ats access_key id xxx → ats access_key --id=xxx
|
4681
4892
|
|
4682
4893
|
* 0.6.18
|
4683
4894
|
|
4684
|
-
|
4895
|
+
* some commands take now --id option instead of id command.
|
4685
4896
|
|
4686
4897
|
* 0.6.15
|
4687
4898
|
|
4688
|
-
|
4899
|
+
* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in [option preset](#lprt) "default".
|
4689
4900
|
|
4690
|
-
|
4901
|
+
## BUGS, FEATURES, CONTRIBUTION
|
4691
4902
|
|
4692
4903
|
For issues or feature requests use the Github repository and issues.
|
4693
4904
|
|
@@ -4695,7 +4906,7 @@ You can also contribute to this open source project.
|
|
4695
4906
|
|
4696
4907
|
One can also [create one's own plugin](#createownplugin).
|
4697
4908
|
|
4698
|
-
|
4909
|
+
### Only one value for any option
|
4699
4910
|
|
4700
4911
|
Some commands and sub commands may ask for the same option name.
|
4701
4912
|
Currently, since option definition is position independent (last one wins), it is not possible
|
@@ -4706,27 +4917,26 @@ As a solution, use the position specific notation for selection, i.e. provide th
|
|
4706
4917
|
|
4707
4918
|
This happens typically for the `node` sub command, e.g. identify the node by name instead of id.
|
4708
4919
|
|
4709
|
-
|
4710
|
-
## ED255519 key not supported
|
4920
|
+
### ED255519 key not supported
|
4711
4921
|
|
4712
4922
|
ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
|
4713
4923
|
|
4714
4924
|
Without this deactivation, if such key was present the following error was generated:
|
4715
4925
|
|
4716
|
-
```
|
4926
|
+
```output
|
4717
4927
|
OpenSSH keys only supported if ED25519 is available
|
4718
4928
|
```
|
4719
4929
|
|
4720
4930
|
Which meant that you do not have ruby support for ED25519 SSH keys.
|
4721
4931
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
4722
4932
|
|
4723
|
-
|
4933
|
+
### Error "Remote host is not who we expected"
|
4724
4934
|
|
4725
4935
|
Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
4726
4936
|
|
4727
4937
|
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
|
4728
4938
|
|
4729
|
-
```
|
4939
|
+
```bash
|
4730
4940
|
--ts=@json:'{"sshfp":null}'
|
4731
4941
|
```
|
4732
4942
|
|
@@ -4734,12 +4944,12 @@ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or
|
|
4734
4944
|
|
4735
4945
|
References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
|
4736
4946
|
|
4737
|
-
|
4947
|
+
### Miscellaneous
|
4738
4948
|
|
4739
4949
|
* remove rest and oauth classes and use ruby standard gems:
|
4740
4950
|
|
4741
|
-
|
4742
|
-
|
4951
|
+
* oauth
|
4952
|
+
* <https://github.com/rest-client/rest-client>
|
4743
4953
|
|
4744
4954
|
* use Thor or any standard Ruby CLI manager
|
4745
4955
|
|
@@ -4749,4 +4959,4 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
4749
4959
|
|
4750
4960
|
* Going through proxy: use env var http_proxy and https_proxy, no_proxy
|
4751
4961
|
|
4752
|
-
* easier use with https://github.com/pmq20/ruby-packer
|
4962
|
+
* easier use with <https://github.com/pmq20/ruby-packer>
|