aspera-cli 4.5.0 → 4.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +1063 -853
- data/docs/Makefile +3 -1
- data/docs/README.erb.md +1011 -808
- data/docs/doc_tools.rb +1 -1
- data/docs/test_env.conf +1 -0
- data/lib/aspera/aoc.rb +21 -0
- data/lib/aspera/ascmd.rb +14 -14
- data/lib/aspera/cli/formater.rb +8 -1
- data/lib/aspera/cli/main.rb +23 -21
- data/lib/aspera/cli/manager.rb +5 -1
- data/lib/aspera/cli/plugins/aoc.rb +94 -69
- data/lib/aspera/cli/plugins/config.rb +99 -49
- data/lib/aspera/cli/plugins/node.rb +55 -57
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/environment.rb +1 -1
- data/lib/aspera/faspex_gw.rb +2 -1
- data/lib/aspera/log.rb +1 -1
- data/lib/aspera/rest.rb +1 -0
- metadata +6 -3
data/docs/README.erb.md
CHANGED
@@ -1,10 +1,11 @@
|
|
1
|
+
# Command Line Interface for IBM Aspera products
|
2
|
+
|
1
3
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
2
4
|
<%load File.join(File.dirname(__FILE__),'doc_tools.rb')-%>
|
3
|
-
<font size="+12"><center><%=tool%> : Command Line Interface for IBM Aspera products</center></font>
|
4
5
|
|
5
6
|
Version : <%=gemspec.version.to_s%>
|
6
7
|
|
7
|
-
|
8
|
+
Laurent/2016-<%=Time.new.year%>
|
8
9
|
|
9
10
|
This gem provides the <%=tool%> Command Line Interface to IBM Aspera software.
|
10
11
|
|
@@ -18,7 +19,7 @@ Required Ruby version: <%=gemspec.required_ruby_version%>
|
|
18
19
|
|
19
20
|
[Aspera APIs](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
|
20
21
|
|
21
|
-
|
22
|
+
## <a id="when_to_use"></a>When to use and when not to use
|
22
23
|
|
23
24
|
<%=tool%> is designed to be used as a command line tool to:
|
24
25
|
|
@@ -47,7 +48,7 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
|
|
47
48
|
|
48
49
|
For scripting and ad'hoc command line operations, <%=tool%> is perfect.
|
49
50
|
|
50
|
-
|
51
|
+
## <a id="parsing"></a>Notations, Shell and Command line parsing
|
51
52
|
|
52
53
|
In examples, command line operations are shown using a shell such: `bash` or `zsh`.
|
53
54
|
|
@@ -64,18 +65,18 @@ On Windows, `cmd.exe` is typically used. Windows process creation does not recei
|
|
64
65
|
|
65
66
|
In case of doubt of argument values after parsing test like this:
|
66
67
|
|
67
|
-
```
|
68
|
+
```bash
|
68
69
|
<%=cmd%> conf echo "Hello World" arg2 3
|
69
70
|
```
|
70
71
|
|
71
|
-
```
|
72
|
+
```bash
|
72
73
|
"Hello World"
|
73
74
|
ERROR: Argument: unprocessed values: ["arg2", "3"]
|
74
75
|
```
|
75
76
|
|
76
77
|
`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (<%=tool%>), next command line arguments are shown in the error message.
|
77
78
|
|
78
|
-
|
79
|
+
## Quick Start
|
79
80
|
|
80
81
|
This section guides you from installation, first use and advanced use.
|
81
82
|
|
@@ -83,15 +84,15 @@ First, follow the section: [Installation](#installation) (Ruby, Gem, FASP) to st
|
|
83
84
|
|
84
85
|
Once the gem is installed, <%=tool%> shall be accessible:
|
85
86
|
|
86
|
-
```
|
87
|
+
```bash
|
87
88
|
<%=cmd%> --version
|
88
89
|
```
|
89
90
|
|
90
|
-
```
|
91
|
+
```bash
|
91
92
|
<%=gemspec.version.to_s%>
|
92
93
|
```
|
93
94
|
|
94
|
-
|
95
|
+
### First use
|
95
96
|
|
96
97
|
Once installation is completed, you can proceed to the first use with a demo server:
|
97
98
|
|
@@ -99,15 +100,15 @@ If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
|
|
99
100
|
|
100
101
|
To test with Aspera demo transfer server, setup the environment and then test:
|
101
102
|
|
102
|
-
```
|
103
|
+
```bash
|
103
104
|
<%=cmd%> config initdemo
|
104
105
|
```
|
105
106
|
|
106
|
-
```
|
107
|
+
```bash
|
107
108
|
<%=cmd%> server browse /
|
108
109
|
```
|
109
110
|
|
110
|
-
```
|
111
|
+
```output
|
111
112
|
:............:...........:......:........:...........................:.......................:
|
112
113
|
: zmode : zuid : zgid : size : mtime : name :
|
113
114
|
:............:...........:......:........:...........................:.......................:
|
@@ -125,27 +126,27 @@ If you want to use <%=tool%> with another server, and in order to make further c
|
|
125
126
|
* list files in a folder
|
126
127
|
* download a file
|
127
128
|
|
128
|
-
```
|
129
|
+
```bash
|
129
130
|
<%=cmd%> config preset update myserver --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
|
130
131
|
```
|
131
132
|
|
132
|
-
```
|
133
|
+
```output
|
133
134
|
updated: myserver
|
134
135
|
```
|
135
136
|
|
136
|
-
```
|
137
|
+
```bash
|
137
138
|
<%=cmd%> config preset set default server myserver
|
138
139
|
```
|
139
140
|
|
140
|
-
```
|
141
|
+
```output
|
141
142
|
updated: default→server to myserver
|
142
143
|
```
|
143
144
|
|
144
|
-
```
|
145
|
+
```bash
|
145
146
|
<%=cmd%> server browse /aspera-test-dir-large
|
146
147
|
```
|
147
148
|
|
148
|
-
```
|
149
|
+
```output
|
149
150
|
:............:...........:......:..............:...........................:............................:
|
150
151
|
: zmode : zuid : zgid : size : mtime : name :
|
151
152
|
:............:...........:......:..............:...........................:............................:
|
@@ -164,22 +165,22 @@ updated: default→server to myserver
|
|
164
165
|
:............:...........:......:..............:...........................:............................:
|
165
166
|
```
|
166
167
|
|
167
|
-
```
|
168
|
+
```bash
|
168
169
|
<%=cmd%> server download /aspera-test-dir-large/200MB
|
169
170
|
```
|
170
171
|
|
171
|
-
```
|
172
|
+
```output
|
172
173
|
Time: 00:00:02 ========================================================================================================== 100% 100 Mbps Time: 00:00:00
|
173
174
|
complete
|
174
175
|
```
|
175
176
|
|
176
|
-
|
177
|
+
### Going further
|
177
178
|
|
178
179
|
Get familiar with configuration, options, commands : [Command Line Interface](#cli).
|
179
180
|
|
180
181
|
Then, follow the section relative to the product you want to interact with ( Aspera on Cloud, Faspex, ...) : [Application Plugins](plugins)
|
181
182
|
|
182
|
-
|
183
|
+
## <a id="installation"></a>Installation
|
183
184
|
|
184
185
|
It is possible to install *either* directly on the host operating system (Linux, Windows, macOS) or as a docker container.
|
185
186
|
|
@@ -193,7 +194,7 @@ The following sections provide information on the various installation methods.
|
|
193
194
|
|
194
195
|
An internet connection is required for the installation. If you don't have internet for the installation, refer to section [Installation without internet access](#offline_install).
|
195
196
|
|
196
|
-
|
197
|
+
### Docker container
|
197
198
|
|
198
199
|
Use this method only if you know what you do, else use the standard recommended method as described here above.
|
199
200
|
|
@@ -203,23 +204,23 @@ The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.
|
|
203
204
|
|
204
205
|
Ensure that you have Docker installed.
|
205
206
|
|
206
|
-
```
|
207
|
+
```bash
|
207
208
|
docker --version
|
208
209
|
```
|
209
210
|
|
210
211
|
Download the wrapping script:
|
211
212
|
|
212
|
-
```
|
213
|
+
```bash
|
213
214
|
curl -o <%=cmd%> https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
|
214
215
|
```
|
215
216
|
|
216
|
-
```
|
217
|
+
```bash
|
217
218
|
chmod a+x <%=cmd%>
|
218
219
|
```
|
219
220
|
|
220
221
|
Install the container image:
|
221
222
|
|
222
|
-
```
|
223
|
+
```bash
|
223
224
|
./<%=cmd%> install
|
224
225
|
```
|
225
226
|
|
@@ -232,7 +233,7 @@ The wrapping script maps the container folder `/usr/src/app/config` to configura
|
|
232
233
|
To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
|
233
234
|
To add local storage as a volume edit the script: <%=tool%> and add a `--volume` stanza.
|
234
235
|
|
235
|
-
|
236
|
+
### <a id="ruby"></a>Ruby
|
236
237
|
|
237
238
|
Use this method to install on the native host.
|
238
239
|
|
@@ -248,7 +249,7 @@ The recommended installation method is `rvm` for systems with "bash-like" shell
|
|
248
249
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
249
250
|
If you have a simpler better way to install Ruby version <%=gemspec.required_ruby_version%> : use it !
|
250
251
|
|
251
|
-
|
252
|
+
#### Generic: RVM: single user installation (not root)
|
252
253
|
|
253
254
|
Use this method which provides more flexibility.
|
254
255
|
|
@@ -256,44 +257,44 @@ Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
|
|
256
257
|
|
257
258
|
Install the 2 keys
|
258
259
|
|
259
|
-
```
|
260
|
+
```bash
|
260
261
|
gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
|
261
262
|
```
|
262
263
|
|
263
264
|
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
264
265
|
|
265
|
-
```
|
266
|
+
```bash
|
266
267
|
\curl -sSL https://get.rvm.io | bash -s stable
|
267
268
|
```
|
268
269
|
|
269
270
|
If you keep the same terminal (not needed if re-login):
|
270
271
|
|
271
|
-
```
|
272
|
+
```bash
|
272
273
|
source ~/.rvm/scripts/rvm
|
273
274
|
```
|
274
275
|
|
275
276
|
It is advised to get one of the pre-compiled ruby version, you can list with:
|
276
277
|
|
277
|
-
```
|
278
|
+
```bash
|
278
279
|
rvm list --remote
|
279
280
|
```
|
280
281
|
|
281
282
|
Install the chosen pre-compiled Ruby version:
|
282
283
|
|
283
|
-
```
|
284
|
+
```bash
|
284
285
|
rvm install 2.7.2 --binary
|
285
286
|
```
|
286
287
|
|
287
288
|
Ruby is now installed for the user, go on to Gem installation.
|
288
289
|
|
289
|
-
|
290
|
+
#### Generic: RVM: global installation (as root)
|
290
291
|
|
291
292
|
Follow the same method as single user install, but execute as "root".
|
292
293
|
|
293
294
|
As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
|
294
295
|
One can install in another location with :
|
295
296
|
|
296
|
-
```
|
297
|
+
```bash
|
297
298
|
curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
298
299
|
```
|
299
300
|
|
@@ -301,15 +302,15 @@ As root, make sure this will not collide with other application using Ruby (e.g.
|
|
301
302
|
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
302
303
|
To activate ruby (and <%=cmd%>) later, source it:
|
303
304
|
|
304
|
-
```
|
305
|
+
```bash
|
305
306
|
source /etc/profile.d/rvm.sh.ok
|
306
307
|
```
|
307
308
|
|
308
|
-
```
|
309
|
+
```bash
|
309
310
|
rvm version
|
310
311
|
```
|
311
312
|
|
312
|
-
|
313
|
+
#### Windows: Installer
|
313
314
|
|
314
315
|
Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
|
315
316
|
|
@@ -317,48 +318,53 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
|
|
317
318
|
* Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
|
318
319
|
* At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
|
319
320
|
|
320
|
-
|
321
|
+
#### macOS: pre-installed or `brew`
|
321
322
|
|
322
323
|
macOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%=gemspec.name%> using `sudo` :
|
323
324
|
|
324
|
-
```
|
325
|
+
```bash
|
325
326
|
sudo gem install <%=gemspec.name%><%=geminstadd%>
|
326
327
|
```
|
327
328
|
|
328
329
|
Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
|
329
330
|
|
330
|
-
```
|
331
|
+
```bash
|
331
332
|
brew install ruby
|
332
333
|
```
|
333
334
|
|
334
|
-
|
335
|
+
#### Linux: package
|
335
336
|
|
336
337
|
If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
|
337
338
|
|
338
339
|
Example:
|
339
340
|
|
340
|
-
```
|
341
|
+
```bash
|
341
342
|
yum install -y ruby rubygems ruby-json
|
342
343
|
```
|
343
344
|
|
344
345
|
One can cleanup the whole yum-installed ruby environment like this to uninstall:
|
345
346
|
|
346
|
-
```
|
347
|
+
```bash
|
347
348
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
348
349
|
```
|
349
350
|
|
350
|
-
```
|
351
|
+
```bash
|
351
352
|
yum remove -y ruby ruby-libs
|
352
353
|
```
|
353
354
|
|
354
|
-
|
355
|
+
#### Other Unixes (AIX)
|
356
|
+
|
357
|
+
Ruby is sometimes made available as installable package through third party providers.
|
358
|
+
For example for AIX, one can look at:
|
359
|
+
|
360
|
+
<https://www.ibm.com/support/pages/aix-toolbox-open-source-software-downloads-alpha#R>
|
355
361
|
|
356
362
|
If your Unix does not provide a pre-built ruby, you can get it using one of those
|
357
|
-
[methods](https://www.ruby-lang.org/en/documentation/installation/)
|
363
|
+
[methods](https://www.ruby-lang.org/en/documentation/installation/).
|
358
364
|
|
359
365
|
For instance to build from source, and install in `/opt/ruby` :
|
360
366
|
|
361
|
-
```
|
367
|
+
```bash
|
362
368
|
wget https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.gz
|
363
369
|
|
364
370
|
gzip -d ruby-2.7.2.tar.gz
|
@@ -376,17 +382,23 @@ make
|
|
376
382
|
make install
|
377
383
|
```
|
378
384
|
|
379
|
-
|
385
|
+
If you already have a Java JVM on your system (`java`), it is possible to use `jruby`:
|
386
|
+
|
387
|
+
<https://www.jruby.org/download>
|
388
|
+
|
389
|
+
Note that using jruby the startup time is longer than the native ruby, but transfer speed is not impacted (executed by `ascp` binary).
|
390
|
+
|
391
|
+
### <a id="the_gem"></a>`<%=gemspec.name%>` gem
|
380
392
|
|
381
393
|
Once you have Ruby and rights to install gems: Install the gem and its dependencies:
|
382
394
|
|
383
|
-
```
|
395
|
+
```bash
|
384
396
|
gem install <%=gemspec.name%><%=geminstadd%>
|
385
397
|
```
|
386
398
|
|
387
399
|
To upgrade to the latest version:
|
388
400
|
|
389
|
-
```
|
401
|
+
```bash
|
390
402
|
gem update <%=gemspec.name%>
|
391
403
|
```
|
392
404
|
|
@@ -394,11 +406,11 @@ gem update <%=gemspec.name%>
|
|
394
406
|
|
395
407
|
To check manually:
|
396
408
|
|
397
|
-
```
|
409
|
+
```bash
|
398
410
|
<%=cmd%> conf check_update
|
399
411
|
```
|
400
412
|
|
401
|
-
|
413
|
+
### <a id="fasp_prot"></a>FASP Protocol
|
402
414
|
|
403
415
|
Most file transfers will be done using the FASP protocol, using `ascp`.
|
404
416
|
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
@@ -408,17 +420,17 @@ Only two additional files are required to perform an Aspera Transfer, which are
|
|
408
420
|
|
409
421
|
This can be installed either be installing an Aspera transfer software, or using an embedded command:
|
410
422
|
|
411
|
-
```
|
423
|
+
```bash
|
412
424
|
<%=cmd%> conf ascp install
|
413
425
|
```
|
414
426
|
|
415
427
|
If a local SDK installation is preferred instead of fetching from internet: one can specify the location of the SDK file:
|
416
428
|
|
417
|
-
```
|
429
|
+
```bash
|
418
430
|
curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
419
431
|
```
|
420
432
|
|
421
|
-
```
|
433
|
+
```bash
|
422
434
|
<%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
423
435
|
```
|
424
436
|
|
@@ -441,7 +453,7 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
|
|
441
453
|
Several methods are provided to start a transfer.
|
442
454
|
Use of a local client ([`direct`](#agt_direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
|
443
455
|
|
444
|
-
|
456
|
+
### <a id="offline_install"></a>Installation in air gapped environment
|
445
457
|
|
446
458
|
Note that currently no pre-packaged version exist yet.
|
447
459
|
A method to build one is provided here:
|
@@ -451,19 +463,19 @@ The procedure:
|
|
451
463
|
* Follow the non-root installation procedure with RVM, including gem
|
452
464
|
* Archive (zip, tar) the main RVM folder (includes <%=cmd%>):
|
453
465
|
|
454
|
-
```
|
466
|
+
```bash
|
455
467
|
cd $HOME && tar zcvf rvm-<%=cmd%>.tgz .rvm
|
456
468
|
```
|
457
469
|
|
458
470
|
* Get the Aspera SDK.
|
459
471
|
|
460
|
-
```
|
472
|
+
```bash
|
461
473
|
<%=cmd%> conf --show-config --fields=sdk_url
|
462
474
|
```
|
463
475
|
|
464
476
|
* Download the SDK archive from that URL.
|
465
477
|
|
466
|
-
```
|
478
|
+
```bash
|
467
479
|
curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
468
480
|
```
|
469
481
|
|
@@ -471,7 +483,7 @@ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
|
471
483
|
|
472
484
|
* On target system
|
473
485
|
|
474
|
-
```
|
486
|
+
```bash
|
475
487
|
cd $HOME
|
476
488
|
|
477
489
|
tar zxvf rvm-<%=cmd%>.tgz
|
@@ -483,11 +495,11 @@ source ~/.rvm/scripts/rvm
|
|
483
495
|
|
484
496
|
* Add those lines to shell init (`.profile`)
|
485
497
|
|
486
|
-
```
|
498
|
+
```bash
|
487
499
|
source ~/.rvm/scripts/rvm
|
488
500
|
```
|
489
501
|
|
490
|
-
|
502
|
+
## <a id="cli"></a>Command Line Interface: <%=tool%>
|
491
503
|
|
492
504
|
The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
493
505
|
|
@@ -515,7 +527,7 @@ The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which intera
|
|
515
527
|
|
516
528
|
Basic usage is displayed by executing:
|
517
529
|
|
518
|
-
```
|
530
|
+
```bash
|
519
531
|
<%=cmd%> -h
|
520
532
|
```
|
521
533
|
|
@@ -523,26 +535,26 @@ Refer to sections: [Usage](#usage) and [Sample Commands](#commands).
|
|
523
535
|
|
524
536
|
Not all <%=tool%> features are fully documented here, the user may explore commands on the command line.
|
525
537
|
|
526
|
-
|
538
|
+
### Arguments : Commands and options
|
527
539
|
|
528
540
|
Arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
|
529
541
|
|
530
|
-
There are two types of arguments: Commands and Options. Example :
|
542
|
+
There are two types of command line arguments: Commands and Options. Example :
|
531
543
|
|
532
|
-
```
|
533
|
-
<%=cmd%> command --option-name=VAL1 VAL2
|
544
|
+
```bash
|
545
|
+
<%=cmd%> command subcommand --option-name=VAL1 VAL2
|
534
546
|
```
|
535
547
|
|
536
|
-
* executes _command_: `command`
|
548
|
+
* executes _command_: `command subcommand`
|
537
549
|
* with one _option_: `option_name`
|
538
|
-
* this option
|
550
|
+
* this option is given a _value_ of: `VAL1`
|
539
551
|
* the command has one additional _argument_: `VAL2`
|
540
552
|
|
541
553
|
When the value of a command, option or argument is constrained by a fixed list of values, it is possible to use the first letters of the value only, provided that it uniquely identifies a value. For example `<%=cmd%> conf ov` is the same as `<%=cmd%> config overview`.
|
542
554
|
|
543
555
|
The value of options and arguments is evaluated with the [Extended Value Syntax](#extended).
|
544
556
|
|
545
|
-
|
557
|
+
#### Options
|
546
558
|
|
547
559
|
All options, e.g. `--log-level=debug`, are command line arguments that:
|
548
560
|
|
@@ -557,15 +569,15 @@ Exceptions:
|
|
557
569
|
* some options (flags) don't take a value, e.g. `-r`
|
558
570
|
* the special option `--` stops option processing and is ignored, following command line arguments are taken as arguments, including the ones starting with a `-`. Example:
|
559
571
|
|
560
|
-
```
|
572
|
+
```bash
|
561
573
|
<%=cmd%> config echo -- --sample
|
562
574
|
```
|
563
575
|
|
564
|
-
```
|
576
|
+
```bash
|
565
577
|
"--sample"
|
566
578
|
```
|
567
579
|
|
568
|
-
Note that `--sample` is taken as an argument, and not option
|
580
|
+
Note that here, `--sample` is taken as an argument, and not as an option, due to `--`.
|
569
581
|
|
570
582
|
Options can be optional or mandatory, with or without (hardcoded) default value. Options can be placed anywhere on command line and evaluated in order.
|
571
583
|
|
@@ -575,33 +587,32 @@ The value for _any_ options can come from the following locations (in this order
|
|
575
587
|
* Environment variable
|
576
588
|
* Command line
|
577
589
|
|
578
|
-
Environment variable starting with prefix: <%=evp%> are taken as option values,
|
579
|
-
e.g. `<%=evp%>OPTION_NAME` is for `--option-name`.
|
590
|
+
Environment variable starting with prefix: <%=evp%> are taken as option values, e.g. `<%=evp%>OPTION_NAME` is for `--option-name`.
|
580
591
|
|
581
592
|
Options values can be displayed for a given command by providing the `--show-config` option: `<%=cmd%> node --show-config`
|
582
593
|
|
583
|
-
|
594
|
+
#### Commands and Arguments
|
584
595
|
|
585
596
|
Command line arguments that are not options are either commands or arguments. If an argument must begin with `-`, then either use the `@val:` syntax (see [Extended Values](#extended)), or use the `--` separator (see above).
|
586
597
|
|
587
|
-
|
598
|
+
### Interactive Input
|
588
599
|
|
589
600
|
Some options and parameters are mandatory and other optional. By default, the tool will ask for missing mandatory options or parameters for interactive execution.
|
590
601
|
|
591
602
|
The behavior can be controlled with:
|
592
603
|
|
593
604
|
* --interactive=<yes|no> (default=yes if STDIN is a terminal, else no)
|
594
|
-
|
595
|
-
|
605
|
+
* yes : missing mandatory parameters/options are asked to the user
|
606
|
+
* no : missing mandatory parameters/options raise an error message
|
596
607
|
* --ask-options=<yes|no> (default=no)
|
597
|
-
|
608
|
+
* optional parameters/options are asked to user
|
598
609
|
|
599
|
-
|
610
|
+
### Output
|
600
611
|
|
601
612
|
Command execution will result in output (terminal, stdout/stderr).
|
602
613
|
The information displayed depends on the action.
|
603
614
|
|
604
|
-
|
615
|
+
#### Types of output data
|
605
616
|
|
606
617
|
Depending on action, the output will contain:
|
607
618
|
|
@@ -612,15 +623,18 @@ Depending on action, the output will contain:
|
|
612
623
|
* `status` : a message
|
613
624
|
* `other_struct` : a complex structure that cannot be displayed as an array
|
614
625
|
|
615
|
-
|
626
|
+
#### Format of output
|
616
627
|
|
617
628
|
By default, result of type single_object and object_list are displayed using format `table`.
|
618
629
|
The table style can be customized with parameter: `table_style` (horizontal, vertical and intersection characters) and is `:.:` by default.
|
619
630
|
|
620
631
|
In a table format, when displaying "objects" (single, or list), by default, sub object are
|
621
|
-
|
622
|
-
field: "user" and value is the sub hash table.
|
623
|
-
filter fields by "dotted" field name.
|
632
|
+
flattened (option `flat_hash`). So, object {"user":{"id":1,"name":"toto"}} will have attributes: user.id and user.name.
|
633
|
+
Setting `flat_hash` to `false` will only display one field: "user" and value is the sub hash table.
|
634
|
+
When in flatten mode, it is possible to filter fields by "dotted" field name.
|
635
|
+
|
636
|
+
Object lists are displayed one per line, with attributes as columns. Single objects are transposed: one attribute per line.
|
637
|
+
If transposition of single object is not desired, use option: `transpose_single` set to `no`.
|
624
638
|
|
625
639
|
The style of output can be set using the `format` parameter, supporting:
|
626
640
|
|
@@ -631,26 +645,26 @@ The style of output can be set using the `format` parameter, supporting:
|
|
631
645
|
* `yaml` : YAML
|
632
646
|
* `csv` : Comma Separated Values
|
633
647
|
|
634
|
-
|
648
|
+
#### <a id="option_select"></a>Option: `select`: Filter on columns values for `object_list`
|
635
649
|
|
636
650
|
Table output can be filtered using the `select` parameter. Example:
|
637
651
|
|
638
|
-
```
|
652
|
+
```javascript
|
639
653
|
<%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
640
654
|
```
|
641
655
|
|
642
|
-
```
|
656
|
+
```output
|
643
657
|
:...............................:..................................:...........:
|
644
658
|
: name : email : ats_admin :
|
645
659
|
:...............................:..................................:...........:
|
646
|
-
: John
|
660
|
+
: John Curtis : john@example.com : true :
|
647
661
|
: Laurent Martin : laurent@example.com : true :
|
648
662
|
:...............................:..................................:...........:
|
649
663
|
```
|
650
664
|
|
651
665
|
Note that `select` filters selected elements from the result of API calls, while the `query` parameters gives filtering parameters to the API when listing elements.
|
652
666
|
|
653
|
-
|
667
|
+
#### Verbosity of output
|
654
668
|
|
655
669
|
Output messages are categorized in 3 types:
|
656
670
|
|
@@ -664,7 +678,7 @@ The option `display` controls the level of output:
|
|
664
678
|
* `data` display `data` and `error` messages
|
665
679
|
* `error` display only error messages.
|
666
680
|
|
667
|
-
|
681
|
+
#### Selection of output object properties
|
668
682
|
|
669
683
|
By default, a table output will display one line per entry, and columns for each entries. Depending on the command, columns may include by default all properties, or only some selected properties. It is possible to define specific columns to be displayed, by setting the `fields` option to one of the following value:
|
670
684
|
|
@@ -675,13 +689,13 @@ By default, a table output will display one line per entry, and columns for each
|
|
675
689
|
* +a,b,c : add selected properties to the default selection.
|
676
690
|
* -a,b,c : remove selected properties from the default selection.
|
677
691
|
|
678
|
-
|
692
|
+
### <a id="extended"></a>Extended Value Syntax
|
679
693
|
|
680
694
|
Usually, values of options and arguments are specified by a simple string. But sometime it is convenient to read a value from a file, or decode it, or have a value more complex than a string (e.g. Hash table).
|
681
695
|
|
682
696
|
The extended value syntax is:
|
683
697
|
|
684
|
-
```
|
698
|
+
```bash
|
685
699
|
<0 or more decoders><0 or 1 reader><nothing or some text value>
|
686
700
|
```
|
687
701
|
|
@@ -690,11 +704,11 @@ The difference between reader and decoder is order and ordinality. Both act like
|
|
690
704
|
The following "readers" are supported (returns value in []):
|
691
705
|
|
692
706
|
* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
|
693
|
-
* @file:PATH : [String] read value from a file (prefix
|
694
|
-
* @path:PATH : [String] performs path expansion (prefix
|
707
|
+
* @file:PATH : [String] read value from a file (prefix `~/` is replaced with the users home folder), e.g. `--key=@file:~/.ssh/mykey`
|
708
|
+
* @path:PATH : [String] performs path expansion (prefix `~/` is replaced with the users home folder), e.g. `--config-file=@path:~/sample_config.yml`
|
695
709
|
* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
696
710
|
* @stdin: : [String] read from stdin (no value on right)
|
697
|
-
* @preset:NAME : [Hash] get whole <%=opprst%> value by name
|
711
|
+
* @preset:NAME : [Hash] get whole <%=opprst%> value by name. Subvalues can also be used using `.` as separator. e.g. foo.bar is conf[foo][bar]
|
698
712
|
|
699
713
|
In addition it is possible to decode a value, using one or multiple decoders :
|
700
714
|
|
@@ -711,33 +725,33 @@ To display the result of an extended value, use the `config echo` command.
|
|
711
725
|
|
712
726
|
Example: read the content of the specified file, then, base64 decode, then unzip:
|
713
727
|
|
714
|
-
```
|
728
|
+
```bash
|
715
729
|
<%=cmd%> config echo @zlib:@base64:@file:myfile.dat
|
716
730
|
```
|
717
731
|
|
718
732
|
Example: create a value as a hash, with one key and the value is read from a file:
|
719
733
|
|
720
|
-
```
|
734
|
+
```bash
|
721
735
|
<%=cmd%> config echo @ruby:'{"token_verification_key"=>File.read("pubkey.txt")}'
|
722
736
|
```
|
723
737
|
|
724
738
|
Example: read a csv file and create a list of hash for bulk provisioning:
|
725
739
|
|
726
|
-
```
|
740
|
+
```bash
|
727
741
|
cat test.csv
|
728
742
|
```
|
729
743
|
|
730
|
-
```
|
744
|
+
```bash
|
731
745
|
name,email
|
732
746
|
lolo,laurent@example.com
|
733
747
|
toto,titi@tutu.tata
|
734
748
|
```
|
735
749
|
|
736
|
-
```
|
750
|
+
```bash
|
737
751
|
<%=cmd%> config echo @csvt:@file:test.csv
|
738
752
|
```
|
739
753
|
|
740
|
-
```
|
754
|
+
```output
|
741
755
|
:......:.....................:
|
742
756
|
: name : email :
|
743
757
|
:......:.....................:
|
@@ -748,17 +762,17 @@ toto,titi@tutu.tata
|
|
748
762
|
|
749
763
|
Example: create a hash and include values from preset named "config" of config file in this hash
|
750
764
|
|
751
|
-
```
|
765
|
+
```javascript
|
752
766
|
<%=cmd%> config echo @incps:@json:'{"hello":true,"incps":["config"]}'
|
753
767
|
```
|
754
768
|
|
755
|
-
```
|
769
|
+
```bash
|
756
770
|
{"version"=>"0.9", "hello"=>true}
|
757
771
|
```
|
758
772
|
|
759
773
|
Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["config"]}'` is equivalent to: `@preset:config`
|
760
774
|
|
761
|
-
|
775
|
+
### <a id="native"></a>Structured Value
|
762
776
|
|
763
777
|
Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
|
764
778
|
|
@@ -769,7 +783,7 @@ A convenient way to specify a _Structured Value_ is to use the `@json:` decoder,
|
|
769
783
|
|
770
784
|
It is also possible to provide a _Structured Value_ in a file using `@json:@file:<path>`
|
771
785
|
|
772
|
-
|
786
|
+
### <a id="conffolder"></a>Configuration and Persistency Folder
|
773
787
|
|
774
788
|
<%=tool%> configuration and other runtime files (token cache, file lists, persistency files, SDK) are stored in folder `[User's home folder]/.aspera/<%=cmd%>`.
|
775
789
|
|
@@ -778,11 +792,11 @@ It uses the `HOME` env var primarily, and on MS Windows it also looks at `%HOMED
|
|
778
792
|
|
779
793
|
The main folder can be displayed using :
|
780
794
|
|
781
|
-
```
|
795
|
+
```bash
|
782
796
|
<%=cmd%> config folder
|
783
797
|
```
|
784
798
|
|
785
|
-
```
|
799
|
+
```bash
|
786
800
|
/Users/kenji/.aspera/<%=cmd%>
|
787
801
|
```
|
788
802
|
|
@@ -790,13 +804,15 @@ It can be overridden using the environment variable `<%=evp%>HOME`.
|
|
790
804
|
|
791
805
|
Example (Windows):
|
792
806
|
|
793
|
-
```
|
807
|
+
```output
|
794
808
|
set <%=evp%>HOME=C:\Users\Kenji\.aspera\<%=cmd%>
|
809
|
+
|
795
810
|
<%=cmd%> config folder
|
811
|
+
|
796
812
|
C:\Users\Kenji\.aspera\<%=cmd%>
|
797
813
|
```
|
798
814
|
|
799
|
-
|
815
|
+
### <a id="configfile"></a>Configuration file
|
800
816
|
|
801
817
|
On the first execution of <%=tool%>, an empty configuration file is created in the configuration folder.
|
802
818
|
Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
|
@@ -811,19 +827,19 @@ The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml` (this ca
|
|
811
827
|
|
812
828
|
The configuration file is simply a catalog of pre-defined lists of options, called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
|
813
829
|
|
814
|
-
|
830
|
+
#### <a id="lprt"></a><%=prstt%>
|
815
831
|
|
816
832
|
A <%=prst%> is simply a collection of parameters and their associated values in a named section in the configuration file.
|
817
833
|
|
818
834
|
A named <%=prst%> can be modified directly using <%=tool%>, which will update the configuration file :
|
819
835
|
|
820
|
-
```
|
836
|
+
```bash
|
821
837
|
<%=cmd%> config preset set|delete|show|initialize|update <<%=opprst%>>
|
822
838
|
```
|
823
839
|
|
824
840
|
The command `update` allows the easy creation of <%=prst%> by simply providing the options in their command line format, e.g. :
|
825
841
|
|
826
|
-
```
|
842
|
+
```bash
|
827
843
|
<%=cmd%> config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
|
828
844
|
```
|
829
845
|
|
@@ -831,49 +847,49 @@ The command `update` allows the easy creation of <%=prst%> by simply providing t
|
|
831
847
|
|
832
848
|
The command `set` allows setting individual options in a <%=prst%>.
|
833
849
|
|
834
|
-
```
|
850
|
+
```bash
|
835
851
|
<%=cmd%> config preset set demo_server password _demo_pass_
|
836
852
|
```
|
837
853
|
|
838
854
|
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
|
839
855
|
|
840
|
-
```
|
856
|
+
```javascript
|
841
857
|
<%=cmd%> config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
|
842
858
|
```
|
843
859
|
|
844
860
|
A full terminal based overview of the configuration can be displayed using:
|
845
861
|
|
846
|
-
```
|
862
|
+
```bash
|
847
863
|
<%=cmd%> config preset over
|
848
864
|
```
|
849
865
|
|
850
866
|
A list of <%=prst%> can be displayed using:
|
851
867
|
|
852
|
-
```
|
868
|
+
```bash
|
853
869
|
<%=cmd%> config preset list
|
854
870
|
```
|
855
871
|
|
856
872
|
A good practice is to not manually edit the configuration file and use modification commands instead.
|
857
873
|
If necessary, the configuration file can opened in a text editor with:
|
858
874
|
|
859
|
-
```
|
875
|
+
```bash
|
860
876
|
<%=cmd%> config open
|
861
877
|
```
|
862
878
|
|
863
879
|
Older format for commands are still supported:
|
864
880
|
|
865
|
-
```
|
881
|
+
```bash
|
866
882
|
<%=cmd%> config id <name> set|delete|show|initialize|update
|
867
883
|
<%=cmd%> config over
|
868
884
|
<%=cmd%> config list
|
869
885
|
```
|
870
886
|
|
871
887
|
|
872
|
-
|
888
|
+
#### <a id="lprtconf"></a>Special <%=prstt%>: config
|
873
889
|
|
874
890
|
This preset name is reserved and contains a single key: `version`. This is the version of <%=tool%> which created the file.
|
875
891
|
|
876
|
-
|
892
|
+
#### <a id="lprtdef"></a>Special <%=prstt%>: default
|
877
893
|
|
878
894
|
This preset name is reserved and contains an array of key-value , where the key is the name of a plugin, and the value is the name of another preset.
|
879
895
|
|
@@ -883,13 +899,19 @@ Note that special plugin name: `config` can be associated with a preset that is
|
|
883
899
|
|
884
900
|
Operations on this preset are done using regular `config` operations:
|
885
901
|
|
886
|
-
```
|
902
|
+
```bash
|
887
903
|
<%=cmd%> config preset set default _plugin_name_ _default_preset_for_plugin_
|
904
|
+
```
|
905
|
+
|
906
|
+
```bash
|
888
907
|
<%=cmd%> config preset get default _plugin_name_
|
908
|
+
```
|
909
|
+
|
910
|
+
```javascript
|
889
911
|
"_default_preset_for_plugin_"
|
890
912
|
```
|
891
913
|
|
892
|
-
|
914
|
+
#### <a id="lplugconf"></a>Special Plugin: config
|
893
915
|
|
894
916
|
Plugin `config` (not to be confused with <%=prstt%> config) is used to configure <%=tool%> but it also contains global options.
|
895
917
|
|
@@ -897,7 +919,7 @@ When <%=tool%> starts, it looks for the `default` <%=prstt%> and if there is a v
|
|
897
919
|
|
898
920
|
If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
|
899
921
|
|
900
|
-
|
922
|
+
#### Format of file
|
901
923
|
|
902
924
|
The configuration file is a hash in a YAML file. Example:
|
903
925
|
|
@@ -934,13 +956,13 @@ Values in the configuration also follow the [Extended Value Syntax](#extended).
|
|
934
956
|
|
935
957
|
Note: if the user wants to use the [Extended Value Syntax](#extended) inside the configuration file, using the `config preset update` command, the user shall use the `@val:` prefix. Example:
|
936
958
|
|
937
|
-
```
|
959
|
+
```bash
|
938
960
|
<%=cmd%> config preset set my_aoc_org private_key @val:@file:"$HOME/.aspera/<%=cmd%>/aocapikey"
|
939
961
|
```
|
940
962
|
|
941
963
|
This creates the <%=prst%>:
|
942
964
|
|
943
|
-
```
|
965
|
+
```yaml
|
944
966
|
...
|
945
967
|
my_aoc_org:
|
946
968
|
private_key: @file:"/Users/laurent/.aspera/<%=cmd%>/aocapikey"
|
@@ -949,7 +971,7 @@ my_aoc_org:
|
|
949
971
|
|
950
972
|
So, the key file will be read only at execution time, but not be embedded in the configuration file.
|
951
973
|
|
952
|
-
|
974
|
+
#### Options evaluation order
|
953
975
|
|
954
976
|
Some options are global, some options are available only for some plugins. (the plugin is the first level command).
|
955
977
|
|
@@ -970,39 +992,39 @@ is an underscore. E.g. --xxx-yyy on command line gives xxx_yyy in configuration
|
|
970
992
|
|
971
993
|
The main plugin name is `config`, so it is possible to define a default <%=prst%> for the main plugin with:
|
972
994
|
|
973
|
-
```
|
995
|
+
```bash
|
974
996
|
<%=cmd%> config preset set cli_default interactive no
|
975
997
|
```
|
976
998
|
|
977
|
-
```
|
999
|
+
```bash
|
978
1000
|
<%=cmd%> config preset set default config cli_default
|
979
1001
|
```
|
980
1002
|
|
981
1003
|
A <%=prst%> value can be removed with `unset`:
|
982
1004
|
|
983
|
-
```
|
1005
|
+
```bash
|
984
1006
|
<%=cmd%> config preset unset cli_default interactive
|
985
1007
|
```
|
986
1008
|
|
987
1009
|
Example: Define options using command line:
|
988
1010
|
|
989
|
-
```
|
1011
|
+
```bash
|
990
1012
|
<%=cmd%> -N --url=x --password=y --username=y node --show-config
|
991
1013
|
```
|
992
1014
|
|
993
1015
|
Example: Define options using a hash:
|
994
1016
|
|
995
|
-
```
|
1017
|
+
```javascript
|
996
1018
|
<%=cmd%> -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
|
997
1019
|
```
|
998
1020
|
|
999
|
-
|
1021
|
+
#### Shares Examples
|
1000
1022
|
|
1001
1023
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
1002
1024
|
only username/password and url are required (either on command line, or from config file).
|
1003
1025
|
Those can usually be provided on the command line:
|
1004
1026
|
|
1005
|
-
```
|
1027
|
+
```bash
|
1006
1028
|
<%=cmd%> shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra
|
1007
1029
|
```
|
1008
1030
|
|
@@ -1010,49 +1032,49 @@ This can also be provisioned in a config file:
|
|
1010
1032
|
|
1011
1033
|
* Build <%=prst%>
|
1012
1034
|
|
1013
|
-
```
|
1035
|
+
```bash
|
1014
1036
|
<%=cmd%> config preset set shares06 url https://10.25.0.6
|
1015
1037
|
<%=cmd%> config preset set shares06 username john
|
1016
1038
|
<%=cmd%> config preset set shares06 password 4sp3ra
|
1017
1039
|
```
|
1018
1040
|
|
1019
|
-
|
1041
|
+
This can also be done with one single command:
|
1020
1042
|
|
1021
|
-
```
|
1043
|
+
```javascript
|
1022
1044
|
<%=cmd%> config preset init shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'
|
1023
1045
|
```
|
1024
1046
|
|
1025
1047
|
or
|
1026
1048
|
|
1027
|
-
```
|
1049
|
+
```bash
|
1028
1050
|
<%=cmd%> config preset update shares06 --url=https://10.25.0.6 --username=john --password=4sp3ra
|
1029
1051
|
```
|
1030
1052
|
|
1031
1053
|
* Define this <%=prst%> as the default <%=prst%> for the specified plugin (`shares`)
|
1032
1054
|
|
1033
|
-
```
|
1055
|
+
```bash
|
1034
1056
|
<%=cmd%> config preset set default shares shares06
|
1035
1057
|
```
|
1036
1058
|
|
1037
1059
|
* Display the content of configuration file in table format
|
1038
1060
|
|
1039
|
-
```
|
1061
|
+
```bash
|
1040
1062
|
<%=cmd%> config overview
|
1041
1063
|
```
|
1042
1064
|
|
1043
1065
|
* Execute a command on the shares application using default parameters
|
1044
1066
|
|
1045
|
-
```
|
1067
|
+
```bash
|
1046
1068
|
<%=cmd%> shares repo browse /
|
1047
1069
|
```
|
1048
1070
|
|
1049
|
-
|
1071
|
+
### <a id="vault"></a>Secret Vault
|
1050
1072
|
|
1051
1073
|
When a secret or password is needed, it is possible to store in the secret vault.
|
1052
1074
|
|
1053
|
-
By default the vault is defined using option `secrets
|
1075
|
+
By default the vault is defined using option `secrets`, which can be stored in the configuration file.
|
1054
1076
|
|
1055
|
-
|
1077
|
+
#### Using system keychain
|
1056
1078
|
|
1057
1079
|
Only on macOS.
|
1058
1080
|
|
@@ -1060,7 +1082,7 @@ It is possible to store secrets in macOS keychain (only read supported currently
|
|
1060
1082
|
|
1061
1083
|
Set option `secrets` to value `system` to use the default keychain or use value `system:[name]` to use a custom keychain.
|
1062
1084
|
|
1063
|
-
|
1085
|
+
#### Modern config file format: encrypted in config file
|
1064
1086
|
|
1065
1087
|
It is possible to store and use secrets encrypted.
|
1066
1088
|
For this use the `config vault` command.
|
@@ -1076,13 +1098,13 @@ Then secrets can be manipulated using commands:
|
|
1076
1098
|
|
1077
1099
|
Secrets must be uniquely identified by `url` and `username`. An optional description can be provided using option `value`.
|
1078
1100
|
|
1079
|
-
|
1101
|
+
#### Legacy config file format
|
1080
1102
|
|
1081
1103
|
The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
|
1082
1104
|
|
1083
1105
|
For example, choose a repository name, for example `my_secrets`, and populate it like this:
|
1084
1106
|
|
1085
|
-
```
|
1107
|
+
```bash
|
1086
1108
|
<%=cmd%> conf id my_secrets set 'access_key1' 'secret1'
|
1087
1109
|
|
1088
1110
|
<%=cmd%> conf id my_secrets set 'access_key2' 'secret2'
|
@@ -1095,7 +1117,7 @@ cli_default
|
|
1095
1117
|
Here above, one has already set a `config` global preset to preset `cli_default` (refer to earlier in documentation).
|
1096
1118
|
So the repository can be read by default like this (note the prefix `@val:` to avoid the evaluation of prefix `@preset:`):
|
1097
1119
|
|
1098
|
-
```
|
1120
|
+
```bash
|
1099
1121
|
<%=cmd%> conf id cli_default set secrets @val:@preset:my_secrets
|
1100
1122
|
```
|
1101
1123
|
|
@@ -1103,35 +1125,56 @@ A secret repository can always be selected at runtime using `--secrets=@preset:x
|
|
1103
1125
|
|
1104
1126
|
To test if a secret can be found use:
|
1105
1127
|
|
1106
|
-
```
|
1128
|
+
```bash
|
1107
1129
|
<%=cmd%> conf vault get --username=access_key1
|
1108
1130
|
```
|
1109
1131
|
|
1110
|
-
|
1132
|
+
### Plugins
|
1111
1133
|
|
1112
|
-
The CLI tool uses a plugin mechanism.
|
1113
|
-
|
1134
|
+
The CLI tool uses a plugin mechanism.
|
1135
|
+
The first level command (just after <%=tool%> on the command line) is the name of the concerned plugin which will execute the command.
|
1136
|
+
Each plugin usually represents commands sent to a specific application.
|
1137
|
+
For instance, the plugin `faspex` allows operations on the application "Aspera Faspex".
|
1114
1138
|
|
1115
|
-
|
1139
|
+
Available plugins can be found using command:
|
1116
1140
|
|
1141
|
+
```bash
|
1142
|
+
<%=cmd%> conf plugin list
|
1117
1143
|
```
|
1118
|
-
|
1119
|
-
|
1120
|
-
|
1121
|
-
|
1122
|
-
|
1123
|
-
|
1124
|
-
|
1125
|
-
|
1126
|
-
|
1127
|
-
|
1128
|
-
|
1129
|
-
|
1130
|
-
|
1131
|
-
|
1144
|
+
|
1145
|
+
```output
|
1146
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1147
|
+
| plugin | path |
|
1148
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1149
|
+
| shares | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/shares.rb |
|
1150
|
+
| node | /Users/laurent/workspace/aspera/aspera-cli/lib/aspera/cli/plugins/node.rb |
|
1151
|
+
...
|
1152
|
+
+--------------+-----------------------------------------------------------------------------------+
|
1153
|
+
```
|
1154
|
+
|
1155
|
+
#### <a id="createownplugin"></a>Create your own plugin
|
1156
|
+
|
1157
|
+
By default plugins are looked-up in folders specifed by (multi-value) option `plugin_folder`:
|
1158
|
+
|
1159
|
+
```javascript
|
1160
|
+
ascli --show-config --select=@json:'{"key":"plugin_folder"}'
|
1161
|
+
```
|
1162
|
+
|
1163
|
+
You can create the skeleton of a new plugin like this:
|
1164
|
+
|
1165
|
+
```bash
|
1166
|
+
<%=cmd%> conf plugin create foo .
|
1167
|
+
```
|
1168
|
+
|
1169
|
+
```output
|
1170
|
+
Created ./foo.rb
|
1171
|
+
```
|
1172
|
+
|
1173
|
+
```bash
|
1174
|
+
<%=cmd%> --plugin-folder=. foo
|
1132
1175
|
```
|
1133
1176
|
|
1134
|
-
|
1177
|
+
#### <a id="plugins"></a>Plugins: Application URL and Authentication
|
1135
1178
|
|
1136
1179
|
<%=tool%> comes with several Aspera application plugins.
|
1137
1180
|
|
@@ -1147,7 +1190,7 @@ Those can be provided using command line, parameter set, env var, see section ab
|
|
1147
1190
|
|
1148
1191
|
Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
|
1149
1192
|
|
1150
|
-
|
1193
|
+
### Logging, Debugging
|
1151
1194
|
|
1152
1195
|
The gem is equipped with traces. By default logging level is `warn`.
|
1153
1196
|
To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `<%=evp%>LOG_LEVEL`, or parameter in con file).
|
@@ -1157,31 +1200,39 @@ It is also possible to activate traces before initialization using env var `AS_L
|
|
1157
1200
|
By default passwords and secrets are removed from logs.
|
1158
1201
|
Use option `log_passwords` to change this behaviour.
|
1159
1202
|
|
1160
|
-
|
1203
|
+
### Learning Aspera Product APIs (REST)
|
1161
1204
|
|
1162
1205
|
This CLI uses REST APIs.
|
1163
1206
|
To display HTTP calls, use argument `-r` or `--rest-debug`, this is useful to display exact content of HTTP requests and responses.
|
1164
1207
|
|
1165
1208
|
In order to get traces of execution, use argument : `--log-level=debug`
|
1166
1209
|
|
1167
|
-
|
1210
|
+
### <a id="http_options"></a>HTTP socket parameters
|
1168
1211
|
|
1169
|
-
If the server does not provide a valid certificate, use
|
1212
|
+
If the server does not provide a valid certificate, use option: `--insecure=yes`.
|
1170
1213
|
|
1171
|
-
|
1214
|
+
Ruby HTTP socket parameters can be adjusted.
|
1172
1215
|
|
1173
|
-
|
1174
|
-
|
1216
|
+
| parameter | default |
|
1217
|
+
|----------------------|---------|
|
1218
|
+
| `read_timeout` | 60 |
|
1219
|
+
| `write_timeout` | 60 |
|
1220
|
+
| `open_timeout` | 60 |
|
1221
|
+
| `keep_alive_timeout` | 2 |
|
1175
1222
|
|
1223
|
+
Values are in set **seconds** and can be of type either integer or float.
|
1176
1224
|
Default values are the ones of Ruby.
|
1225
|
+
For details refer to the Ruby library: [`Net::HTTP`](https://ruby-doc.org/stdlib/libdoc/net/http/rdoc/Net/HTTP.html).
|
1226
|
+
|
1227
|
+
Like any other option, those can be set either on command line, or in config file, either in a global preset or server-specific one.
|
1177
1228
|
|
1178
1229
|
Example:
|
1179
1230
|
|
1180
|
-
```
|
1231
|
+
```javascript
|
1181
1232
|
<%=cmd%> aoc admin res package list --http-options=@json:'{"read_timeout":10.0}'
|
1182
1233
|
```
|
1183
1234
|
|
1184
|
-
|
1235
|
+
### <a id="graphical"></a>Graphical Interactions: Browser and Text Editor
|
1185
1236
|
|
1186
1237
|
Some actions may require the use of a graphical tool:
|
1187
1238
|
|
@@ -1194,30 +1245,30 @@ It is also possible to force the graphical mode with option --ui :
|
|
1194
1245
|
* `--ui=graphical` forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
|
1195
1246
|
* `--ui=text` forces a text environment, the URL or file path to open is displayed on terminal.
|
1196
1247
|
|
1197
|
-
|
1248
|
+
### HTTP proxy for REST
|
1198
1249
|
|
1199
1250
|
To specify a HTTP proxy, set the HTTP_PROXY environment variable (or HTTPS_PROXY), those are honored by Ruby when calling REST APIs.
|
1200
1251
|
|
1201
|
-
|
1252
|
+
### <a id="certificates"></a>SSL CA certificate bundle
|
1202
1253
|
|
1203
1254
|
<%=tool%> uses ruby `openssl` gem, which uses the `openssl` library, so certificates are checked against the ruby default certificates [OpenSSL::X509::DEFAULT_CERT_FILE](https://ruby-doc.org/stdlib-3.0.3/libdoc/openssl/rdoc/OpenSSL/X509/Store.html), which are typically the ones of `openssl` on Unix systems (Linux, macOS, etc..). The environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR` are used if defined.
|
1204
1255
|
|
1205
1256
|
`ascp` also needs to validate certificates when using WSS. By default, `ascp` uses primarily certificates from hard coded path (e.g. on macOS: `/Library/Aspera/ssl`). <%=tool%> overrides and sets the default ruby certificate path as well for `ascp` using `-i` switch. So to update certificates, update ruby's `openssl` gem, or use env vars `SSL_CERT_*`.
|
1206
1257
|
|
1207
|
-
|
1258
|
+
### Proxy auto config
|
1208
1259
|
|
1209
1260
|
The `fpac` option allows specification of a Proxy Auto Configuration (PAC) file, by its URL for local FASP agent. Supported schemes are : http:, https: and file:.
|
1210
1261
|
|
1211
1262
|
The PAC file can be tested with command: `config proxy_check` , example:
|
1212
1263
|
|
1213
|
-
```
|
1264
|
+
```bash
|
1214
1265
|
<%=cmd%> config proxy_check --fpac=file:///./proxy.pac http://www.example.com
|
1215
1266
|
PROXY proxy.example.com:8080
|
1216
1267
|
```
|
1217
1268
|
|
1218
1269
|
This is not yet implemented to specify http proxy, so use `http_proxy` env vars.
|
1219
1270
|
|
1220
|
-
|
1271
|
+
### <a id="client"></a>FASP configuration
|
1221
1272
|
|
1222
1273
|
The `config` plugin also allows specification for the use of a local FASP client. It provides the following commands for `ascp` subcommand:
|
1223
1274
|
|
@@ -1226,12 +1277,21 @@ The `config` plugin also allows specification for the use of a local FASP client
|
|
1226
1277
|
* `products` : list Aspera transfer products available locally
|
1227
1278
|
* `connect` : list,download connect client versions available on internet
|
1228
1279
|
|
1229
|
-
|
1280
|
+
#### Show path of currently used `ascp`
|
1230
1281
|
|
1231
|
-
```
|
1282
|
+
```bash
|
1232
1283
|
<%=cmd%> config ascp show
|
1284
|
+
```
|
1285
|
+
|
1286
|
+
```output
|
1233
1287
|
/Users/laurent/.aspera/ascli/sdk/ascp
|
1288
|
+
```
|
1289
|
+
|
1290
|
+
```bash
|
1234
1291
|
<%=cmd%> config ascp info
|
1292
|
+
```
|
1293
|
+
|
1294
|
+
```output
|
1235
1295
|
+--------------------+-----------------------------------------------------------+
|
1236
1296
|
| key | value |
|
1237
1297
|
+--------------------+-----------------------------------------------------------+
|
@@ -1239,7 +1299,7 @@ The `config` plugin also allows specification for the use of a local FASP client
|
|
1239
1299
|
...
|
1240
1300
|
```
|
1241
1301
|
|
1242
|
-
|
1302
|
+
#### Selection of `ascp` location for [`direct`](#agt_direct) agent
|
1243
1303
|
|
1244
1304
|
By default, <%=tool%> uses any found local product with ascp, including SDK.
|
1245
1305
|
|
@@ -1249,8 +1309,11 @@ For a permanent change, the command `config ascp use` sets the same parameter fo
|
|
1249
1309
|
|
1250
1310
|
Using a POSIX shell:
|
1251
1311
|
|
1312
|
+
```bash
|
1313
|
+
<%=cmd%> config ascp use @path:'~/Applications/Aspera CLI/bin/ascp'
|
1252
1314
|
```
|
1253
|
-
|
1315
|
+
|
1316
|
+
```output
|
1254
1317
|
ascp version: 4.0.0.182279
|
1255
1318
|
Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
|
1256
1319
|
Saved to default global preset global_common_defaults
|
@@ -1258,8 +1321,11 @@ Saved to default global preset global_common_defaults
|
|
1258
1321
|
|
1259
1322
|
Windows:
|
1260
1323
|
|
1261
|
-
```
|
1324
|
+
```bash
|
1262
1325
|
<%=cmd%> config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1326
|
+
```
|
1327
|
+
|
1328
|
+
```output
|
1263
1329
|
ascp version: 4.0.0.182279
|
1264
1330
|
Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1265
1331
|
Saved to default global preset global_common_defaults
|
@@ -1267,12 +1333,15 @@ Saved to default global preset global_common_defaults
|
|
1267
1333
|
|
1268
1334
|
If the path has spaces, read section: [Shell and Command line parsing](#parsing).
|
1269
1335
|
|
1270
|
-
|
1336
|
+
#### List locally installed Aspera Transfer products
|
1271
1337
|
|
1272
1338
|
Locally installed Aspera products can be listed with:
|
1273
1339
|
|
1274
|
-
```
|
1340
|
+
```bash
|
1275
1341
|
<%=cmd%> config ascp products list
|
1342
|
+
```
|
1343
|
+
|
1344
|
+
```output
|
1276
1345
|
:.........................................:................................................:
|
1277
1346
|
: name : app_root :
|
1278
1347
|
:.........................................:................................................:
|
@@ -1283,7 +1352,7 @@ Locally installed Aspera products can be listed with:
|
|
1283
1352
|
:.........................................:................................................:
|
1284
1353
|
```
|
1285
1354
|
|
1286
|
-
|
1355
|
+
#### Selection of local client for `ascp` for [`direct`](#agt_direct) agent
|
1287
1356
|
|
1288
1357
|
If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
|
1289
1358
|
|
@@ -1291,45 +1360,55 @@ Using the option use_product finds the ascp binary of the selected product.
|
|
1291
1360
|
|
1292
1361
|
To permanently use the ascp of a product:
|
1293
1362
|
|
1294
|
-
```
|
1363
|
+
```bash
|
1295
1364
|
<%=cmd%> config ascp products use 'Aspera Connect'
|
1296
1365
|
saved to default global preset /Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
|
1297
1366
|
```
|
1298
1367
|
|
1299
|
-
|
1368
|
+
#### Installation of Connect Client on command line
|
1300
1369
|
|
1301
|
-
```
|
1370
|
+
```bash
|
1302
1371
|
<%=cmd%> config ascp connect list
|
1303
|
-
|
1304
|
-
|
1305
|
-
|
1306
|
-
|
1307
|
-
|
1308
|
-
|
1309
|
-
|
1310
|
-
|
1311
|
-
|
1312
|
-
|
1313
|
-
|
1314
|
-
|
1315
|
-
|
1316
|
-
|
1317
|
-
|
1318
|
-
|
1319
|
-
|
1320
|
-
|
1321
|
-
|
1322
|
-
|
1323
|
-
|
1324
|
-
|
1325
|
-
|
1326
|
-
|
1327
|
-
|
1328
|
-
|
1329
|
-
|
1330
|
-
|
1331
|
-
|
1332
|
-
|
1372
|
+
```
|
1373
|
+
|
1374
|
+
```output
|
1375
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1376
|
+
| id | title | version |
|
1377
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1378
|
+
| urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows | 3.11.2.63 |
|
1379
|
+
| urn:uuid:A3820D20-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows 64-bit | 3.11.2.63 |
|
1380
|
+
| urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E | Aspera Connect for Windows XP | 3.11.2.63 |
|
1381
|
+
| urn:uuid:55425020-083E-11E2-892E-0800200C9A66 | Aspera Connect for Windows XP 64-bit | 3.11.2.63 |
|
1382
|
+
| urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF | Aspera Connect for Mac Intel | 3.11.2.63 |
|
1383
|
+
| urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 | Aspera Connect for Linux 64 | 3.11.2.63 |
|
1384
|
+
+-----------------------------------------------+--------------------------------------+-----------+
|
1385
|
+
```
|
1386
|
+
|
1387
|
+
```bash
|
1388
|
+
<%=cmd%> config ascp connect version 'Aspera Connect for Mac Intel' list
|
1389
|
+
```
|
1390
|
+
|
1391
|
+
```output
|
1392
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1393
|
+
| title | type | href | hreflang | rel |
|
1394
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1395
|
+
| Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstaller-3.11.2.63.dmg | en | enclosure |
|
1396
|
+
| Mac Intel Installer | application/octet-stream | bin/IBMAsperaConnectInstallerOneClick-3.11.2.63.dmg | en | enclosure-one-click |
|
1397
|
+
| Aspera Connect for Mac HTML Documentation | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=aspera-connect-user-guide-macos | en | documentation |
|
1398
|
+
| Aspera Connect for Mac Release Notes | text/html | https://www.ibm.com/docs/en/aspera-connect/3.11?topic=notes-release-aspera-connect-3112 | en | release-notes |
|
1399
|
+
+-------------------------------------------+--------------------------+-----------------------------------------------------------------------------------------+----------+---------------------+
|
1400
|
+
```
|
1401
|
+
|
1402
|
+
```bash
|
1403
|
+
<%=cmd%> config ascp connect version 'Aspera Connect for Mac Intel' download enclosure --to-folder=.
|
1404
|
+
```
|
1405
|
+
|
1406
|
+
```output
|
1407
|
+
Time: 00:00:02 ======================================================================= 100% 27766 KB/sec Time: 00:00:02
|
1408
|
+
Downloaded: IBMAsperaConnectInstaller-3.11.2.63.dmg
|
1409
|
+
```
|
1410
|
+
|
1411
|
+
### <a id="agents"></a>Transfer Agents
|
1333
1412
|
|
1334
1413
|
Some of the actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (`ascp`).
|
1335
1414
|
|
@@ -1350,8 +1429,7 @@ will effectively push files to the related server from the agent node.
|
|
1350
1429
|
|
1351
1430
|
<%=tool%> standardizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1352
1431
|
|
1353
|
-
|
1354
|
-
### <a id="agt_direct"></a>Direct
|
1432
|
+
#### <a id="agt_direct"></a>Direct
|
1355
1433
|
|
1356
1434
|
The `direct` agent directly executes a local ascp.
|
1357
1435
|
This is the default for <%=tool%>.
|
@@ -1377,7 +1455,7 @@ The `transfer-info` accepts the following optional parameters to control multi-s
|
|
1377
1455
|
Resume: In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
|
1378
1456
|
Sleep between iterations is:
|
1379
1457
|
|
1380
|
-
```
|
1458
|
+
```bash
|
1381
1459
|
max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
|
1382
1460
|
```
|
1383
1461
|
|
@@ -1385,7 +1463,7 @@ Some transfer errors are considered "retryable" (e.g. timeout) and some other no
|
|
1385
1463
|
|
1386
1464
|
Examples:
|
1387
1465
|
|
1388
|
-
```
|
1466
|
+
```javascript
|
1389
1467
|
<%=cmd%> ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
|
1390
1468
|
<%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
|
1391
1469
|
```
|
@@ -1396,11 +1474,11 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
|
|
1396
1474
|
* `EX_http_proxy_url` (proxy for legacy http fallback)
|
1397
1475
|
* `EX_ascp_args`
|
1398
1476
|
|
1399
|
-
|
1477
|
+
#### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
|
1400
1478
|
|
1401
1479
|
By specifying option: `--transfer=connect`, <%=tool%> will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
|
1402
1480
|
|
1403
|
-
|
1481
|
+
#### <a id="agt_node"></a>Aspera Node API : Node to node transfers
|
1404
1482
|
|
1405
1483
|
By specifying option: `--transfer=node`, the CLI will start transfers in an Aspera
|
1406
1484
|
Transfer Server using the Node API, either on a local or remote node.
|
@@ -1422,25 +1500,25 @@ If `transfer_info` is not specified and a default node has been configured (name
|
|
1422
1500
|
|
1423
1501
|
If the `password` value begins with `Bearer ` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
|
1424
1502
|
|
1425
|
-
|
1503
|
+
#### <a id="agt_httpgw"></a>HTTP Gateway
|
1426
1504
|
|
1427
1505
|
If it possible to send using a HTTP gateway, in case FASP is not allowed. `transfer_info` shall have a single mandatory parameter: `url`.
|
1428
1506
|
|
1429
1507
|
Example:
|
1430
1508
|
|
1431
|
-
```
|
1509
|
+
```javascript
|
1432
1510
|
<%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
|
1433
1511
|
```
|
1434
1512
|
|
1435
1513
|
Note that the gateway only supports transfers authorized with a token.
|
1436
1514
|
|
1437
|
-
|
1515
|
+
#### <a id="agt_trsdk"></a>Transfer SDK
|
1438
1516
|
|
1439
1517
|
Another possibility is to use the Transfer SDK daemon (asperatransferd).
|
1440
1518
|
|
1441
1519
|
By default it will listen on local port `55002` on `127.0.0.1`.
|
1442
1520
|
|
1443
|
-
|
1521
|
+
### <a id="transferspec"></a>Transfer Specification
|
1444
1522
|
|
1445
1523
|
Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
|
1446
1524
|
is described in a _transfer-spec_ (Transfer Specification), such as:
|
@@ -1464,7 +1542,7 @@ The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
|
|
1464
1542
|
|
1465
1543
|
A [_transfer-spec_](#transferspec) is a Hash table, so it is described on the command line with the [Extended Value Syntax](#extended).
|
1466
1544
|
|
1467
|
-
|
1545
|
+
### <a id="transferparams"></a>Transfer Parameters
|
1468
1546
|
|
1469
1547
|
All standard _transfer-spec_ parameters can be specified.
|
1470
1548
|
[_transfer-spec_](#transferspec) can also be saved/overridden in the config file.
|
@@ -1477,7 +1555,7 @@ References:
|
|
1477
1555
|
|
1478
1556
|
Parameters can be displayed with commands:
|
1479
1557
|
|
1480
|
-
```
|
1558
|
+
```javascript
|
1481
1559
|
<%=cmd%> config ascp spec
|
1482
1560
|
<%=cmd%> config ascp spec --select=@json:'{"d":"Y"}' --fields=-d,n,c
|
1483
1561
|
```
|
@@ -1494,7 +1572,7 @@ Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct).
|
|
1494
1572
|
|
1495
1573
|
<%=spec_table%>
|
1496
1574
|
|
1497
|
-
|
1575
|
+
#### Destination folder for transfers
|
1498
1576
|
|
1499
1577
|
The destination folder is set by <%=tool%> by default to:
|
1500
1578
|
|
@@ -1506,18 +1584,18 @@ As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path
|
|
1506
1584
|
The option `to_folder` provides an equivalent and convenient way to change this parameter:
|
1507
1585
|
`--to-folder=<path>` .
|
1508
1586
|
|
1509
|
-
|
1587
|
+
#### List of files for transfers
|
1510
1588
|
|
1511
1589
|
When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
|
1512
1590
|
So, by default, the list of files to transfer will be simply specified on the command line:
|
1513
1591
|
|
1514
|
-
```
|
1592
|
+
```bash
|
1515
1593
|
<%=cmd%> server upload ~/mysample.file secondfile
|
1516
1594
|
```
|
1517
1595
|
|
1518
1596
|
This is equivalent to:
|
1519
1597
|
|
1520
|
-
```
|
1598
|
+
```bash
|
1521
1599
|
<%=cmd%> server upload --sources=@args ~/mysample.file secondfile
|
1522
1600
|
```
|
1523
1601
|
|
@@ -1532,27 +1610,33 @@ For ease of use and flexibility, the list of files to transfer is specified by t
|
|
1532
1610
|
|
1533
1611
|
* an [Extended Value](#extended) holding an *Array of String*. Examples:
|
1534
1612
|
|
1535
|
-
```
|
1613
|
+
```javascript
|
1536
1614
|
--sources=@json:'["file1","file2"]'
|
1615
|
+
```
|
1616
|
+
|
1617
|
+
```bash
|
1537
1618
|
--sources=@lines:@stdin:
|
1619
|
+
```
|
1620
|
+
|
1621
|
+
```ruby
|
1538
1622
|
--sources=@ruby:'File.read("myfilelist").split("\n")'
|
1539
1623
|
```
|
1540
1624
|
|
1541
1625
|
* `@ts` : the user provides the list of files directly in the `ts` option, in its `paths` field. Example:
|
1542
1626
|
|
1543
|
-
```
|
1627
|
+
```javascript
|
1544
1628
|
--sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
|
1545
1629
|
```
|
1546
1630
|
|
1547
1631
|
providing a file list directly to ascp:
|
1548
1632
|
|
1549
|
-
```
|
1633
|
+
```javascript
|
1550
1634
|
... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
|
1551
1635
|
```
|
1552
1636
|
|
1553
1637
|
* Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
|
1554
1638
|
|
1555
|
-
```
|
1639
|
+
```javascript
|
1556
1640
|
--sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
|
1557
1641
|
```
|
1558
1642
|
|
@@ -1565,7 +1649,7 @@ In case the file list is provided on the command line i.e. using `--sources=@arg
|
|
1565
1649
|
|
1566
1650
|
Example:
|
1567
1651
|
|
1568
|
-
```
|
1652
|
+
```bash
|
1569
1653
|
<%=cmd%> server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
1570
1654
|
```
|
1571
1655
|
|
@@ -1582,13 +1666,13 @@ Source files are located on "Aspera on cloud", when :
|
|
1582
1666
|
* the server is Aspera on Cloud, and making a download / recv
|
1583
1667
|
* the agent is Aspera on Cloud, and making an upload / send
|
1584
1668
|
|
1585
|
-
|
1669
|
+
#### <a id="multisession"></a>Support of multi-session
|
1586
1670
|
|
1587
1671
|
Multi session, i.e. starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on "direct" and "node" agents, not yet on connect.
|
1588
1672
|
|
1589
1673
|
* when agent=node :
|
1590
1674
|
|
1591
|
-
```
|
1675
|
+
```javascript
|
1592
1676
|
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
|
1593
1677
|
```
|
1594
1678
|
|
@@ -1596,7 +1680,7 @@ Multi-session is directly supported by the node daemon.
|
|
1596
1680
|
|
1597
1681
|
* when agent=direct :
|
1598
1682
|
|
1599
|
-
```
|
1683
|
+
```javascript
|
1600
1684
|
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
|
1601
1685
|
```
|
1602
1686
|
|
@@ -1607,33 +1691,33 @@ Multi-session spawn is done by <%=tool%>.
|
|
1607
1691
|
|
1608
1692
|
When multi-session is used, one separate UDP port is used per session (refer to `ascp` manual page).
|
1609
1693
|
|
1610
|
-
|
1694
|
+
#### Transfer Spec Examples
|
1611
1695
|
|
1612
1696
|
* Change target rate
|
1613
1697
|
|
1614
|
-
```
|
1698
|
+
```javascript
|
1615
1699
|
--ts=@json:'{"target_rate_kbps":500000}'
|
1616
1700
|
```
|
1617
1701
|
|
1618
1702
|
* Override the FASP SSH port to a specific TCP port:
|
1619
1703
|
|
1620
|
-
```
|
1704
|
+
```javascript
|
1621
1705
|
--ts=@json:'{"ssh_port":33002}'
|
1622
1706
|
```
|
1623
1707
|
|
1624
1708
|
* Force http fallback mode:
|
1625
1709
|
|
1626
|
-
```
|
1710
|
+
```javascript
|
1627
1711
|
--ts=@json:'{"http_fallback":"force"}'
|
1628
1712
|
```
|
1629
1713
|
|
1630
1714
|
* Activate progress when not activated by default on server
|
1631
1715
|
|
1632
|
-
```
|
1716
|
+
```javascript
|
1633
1717
|
--ts=@json:'{"precalculate_job_size":true}'
|
1634
1718
|
```
|
1635
1719
|
|
1636
|
-
|
1720
|
+
### <a id="scheduling"></a>Lock for exclusive execution
|
1637
1721
|
|
1638
1722
|
In some conditions, it may be desirable to ensure that <%=tool%> is not executed several times in parallel.
|
1639
1723
|
|
@@ -1648,7 +1732,7 @@ Usually the OS native scheduler already provides some sort of protection against
|
|
1648
1732
|
* The Windows scheduler does this by default
|
1649
1733
|
* Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
|
1650
1734
|
|
1651
|
-
```
|
1735
|
+
```bash
|
1652
1736
|
/usr/bin/flock -w 0 /var/cron.lock ascli ...
|
1653
1737
|
```
|
1654
1738
|
|
@@ -1659,25 +1743,28 @@ Example:
|
|
1659
1743
|
|
1660
1744
|
Run this same command in two separate terminals within less than 30 seconds:
|
1661
1745
|
|
1662
|
-
```
|
1746
|
+
```bash
|
1663
1747
|
<%=cmd%> config echo @ruby:'sleep(30)' --lock-port=12345
|
1664
1748
|
```
|
1665
1749
|
|
1666
1750
|
The first instance will sleep 30 seconds, the second one will immediately exit like this:
|
1667
1751
|
|
1668
|
-
```
|
1752
|
+
```bash
|
1669
1753
|
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
1670
1754
|
```
|
1671
1755
|
|
1672
|
-
|
1756
|
+
### "Provençale"
|
1673
1757
|
|
1674
1758
|
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
|
1675
1759
|
, some are provided om shared libraries and must be activated. (e.g. using `trapd`)
|
1676
1760
|
|
1677
1761
|
The list of supported *PVCL* adapters can be retrieved with command:
|
1678
1762
|
|
1679
|
-
```
|
1763
|
+
```bash
|
1680
1764
|
<%=cmd%> conf ascp info
|
1765
|
+
```
|
1766
|
+
|
1767
|
+
```output
|
1681
1768
|
+--------------------+-----------------------------------------------------------+
|
1682
1769
|
| key | value |
|
1683
1770
|
+--------------------+-----------------------------------------------------------+
|
@@ -1700,13 +1787,13 @@ Those adapters can be used wherever a file path is used in `ascp` including conf
|
|
1700
1787
|
|
1701
1788
|
The simplified format is:
|
1702
1789
|
|
1703
|
-
```
|
1790
|
+
```bash
|
1704
1791
|
<adapter>:///<sub file path>?<arg1>=<val1>&...
|
1705
1792
|
```
|
1706
1793
|
|
1707
1794
|
One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
|
1708
1795
|
|
1709
|
-
|
1796
|
+
### <a id="faux_testing"></a>`faux:` for testing
|
1710
1797
|
|
1711
1798
|
This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not <%=tool%>.
|
1712
1799
|
|
@@ -1714,7 +1801,7 @@ This adapter can be used to simulate a file or a directory.
|
|
1714
1801
|
|
1715
1802
|
To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
|
1716
1803
|
|
1717
|
-
```
|
1804
|
+
```bash
|
1718
1805
|
faux:///filename?filesize
|
1719
1806
|
```
|
1720
1807
|
|
@@ -1729,7 +1816,7 @@ For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e
|
|
1729
1816
|
|
1730
1817
|
To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
|
1731
1818
|
|
1732
|
-
```
|
1819
|
+
```bash
|
1733
1820
|
faux:///dirname?<arg1>=<val1>&...
|
1734
1821
|
```
|
1735
1822
|
|
@@ -1772,34 +1859,34 @@ Examples:
|
|
1772
1859
|
|
1773
1860
|
* Upload 20 gibibytes of random data to file myfile to directory /Upload
|
1774
1861
|
|
1775
|
-
```
|
1862
|
+
```bash
|
1776
1863
|
<%=cmd%> server upload faux:///myfile\?20g --to-folder=/Upload
|
1777
1864
|
```
|
1778
1865
|
|
1779
1866
|
* Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
|
1780
1867
|
|
1781
|
-
```
|
1868
|
+
```bash
|
1782
1869
|
<%=cmd%> server upload /tmp/sample --to-folder=faux://
|
1783
1870
|
```
|
1784
1871
|
|
1785
1872
|
* Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the basename of each file being `testfile` to /Upload
|
1786
1873
|
|
1787
|
-
```
|
1874
|
+
```bash
|
1788
1875
|
<%=cmd%> server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
|
1789
1876
|
```
|
1790
1877
|
|
1791
|
-
|
1878
|
+
### <a id="commands"></a>Sample Commands
|
1792
1879
|
|
1793
1880
|
A non complete list of commands used in unit tests:
|
1794
1881
|
|
1795
|
-
```
|
1882
|
+
```bash
|
1796
1883
|
<%=File.read(ENV["INCL_COMMANDS"])%>
|
1797
1884
|
...and more
|
1798
1885
|
```
|
1799
1886
|
|
1800
|
-
|
1887
|
+
### <a id="usage"></a>Usage
|
1801
1888
|
|
1802
|
-
```
|
1889
|
+
```bash
|
1803
1890
|
<%=cmd%> -h
|
1804
1891
|
<%=File.read(ENV["INCL_USAGE"])%>
|
1805
1892
|
|
@@ -1807,17 +1894,17 @@ A non complete list of commands used in unit tests:
|
|
1807
1894
|
|
1808
1895
|
Note that actions and parameter values can be written in short form.
|
1809
1896
|
|
1810
|
-
|
1897
|
+
## <a id="aoc"></a>Plugin: Aspera on Cloud
|
1811
1898
|
|
1812
1899
|
Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
|
1813
1900
|
|
1814
1901
|
It is recommended to use the wizard to set it up, but manual configuration is also possible.
|
1815
1902
|
|
1816
|
-
|
1903
|
+
### <a id="aocwizard"></a>Configuration: using Wizard
|
1817
1904
|
|
1818
1905
|
<%=tool%> provides a configuration wizard. Here is a sample invocation :
|
1819
1906
|
|
1820
|
-
```
|
1907
|
+
```bash
|
1821
1908
|
<%=cmd%> config wizard
|
1822
1909
|
option: url> https://myorg.ibmaspera.com
|
1823
1910
|
Detected: Aspera on Cloud
|
@@ -1834,7 +1921,7 @@ Setting config preset as default for aspera
|
|
1834
1921
|
saving config file
|
1835
1922
|
Done.
|
1836
1923
|
You can test with:
|
1837
|
-
<%=cmd%> aoc user
|
1924
|
+
<%=cmd%> aoc user profile show
|
1838
1925
|
```
|
1839
1926
|
|
1840
1927
|
Optionally, it is possible to create a new organization-specific "integration".
|
@@ -1842,11 +1929,11 @@ For this, specify the option: `--use-generic-client=no`.
|
|
1842
1929
|
|
1843
1930
|
This will guide you through the steps to create.
|
1844
1931
|
|
1845
|
-
|
1932
|
+
### <a id="aocmanual"></a>Configuration: using manual setup
|
1846
1933
|
|
1847
1934
|
If you used the wizard (recommended): skip this section.
|
1848
1935
|
|
1849
|
-
|
1936
|
+
#### Configuration details
|
1850
1937
|
|
1851
1938
|
Several types of OAuth authentication are supported:
|
1852
1939
|
|
@@ -1862,7 +1949,7 @@ For a more convenient, browser-less, experience follow the [JWT](#jwt) section (
|
|
1862
1949
|
|
1863
1950
|
In Oauth, a "Bearer" token are generated to authenticate REST calls. Bearer tokens are valid for a period of time.<%=tool%> saves generated tokens in its configuration folder, tries to re-use them or regenerates them when they have expired.
|
1864
1951
|
|
1865
|
-
|
1952
|
+
#### <a id="clientreg"></a>Optional: API Client Registration
|
1866
1953
|
|
1867
1954
|
If you use the built-in client_id and client_secret, skip this and do not set them in next section.
|
1868
1955
|
|
@@ -1875,24 +1962,24 @@ Let's start by a registration with web based authentication (auth=web):
|
|
1875
1962
|
* Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
|
1876
1963
|
* Go to Apps→Admin→Organization→Integrations
|
1877
1964
|
* Click "Create New"
|
1878
|
-
|
1879
|
-
|
1880
|
-
|
1881
|
-
|
1882
|
-
|
1965
|
+
* Client Name: <%=tool%>
|
1966
|
+
* Redirect URIs: `http://localhost:12345`
|
1967
|
+
* Origins: `localhost`
|
1968
|
+
* uncheck "Prompt users to allow client to access"
|
1969
|
+
* leave the JWT part for now
|
1883
1970
|
* Save
|
1884
1971
|
|
1885
1972
|
Note: for web based authentication, <%=tool%> listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For `<%=tool%>, HTTP is required, and 12345 is the default port.
|
1886
1973
|
|
1887
1974
|
Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
|
1888
1975
|
|
1889
|
-
|
1976
|
+
#### <a id="aocpreset"></a><%=prst%> for Aspera on Cloud
|
1890
1977
|
|
1891
1978
|
If you did not use the wizard, you can also manually create a <%=prst%> for <%=tool%> in its configuration file.
|
1892
1979
|
|
1893
1980
|
Lets create an <%=prst%> called: `my_aoc_org` using `ask` interactive input (client info from previous step):
|
1894
1981
|
|
1895
|
-
```
|
1982
|
+
```bash
|
1896
1983
|
<%=cmd%> config preset ask my_aoc_org url client_id client_secret
|
1897
1984
|
option: url> https://myorg.ibmaspera.com/
|
1898
1985
|
option: client_id> BJLPObQiFw
|
@@ -1904,17 +1991,17 @@ updated: my_aoc_org
|
|
1904
1991
|
|
1905
1992
|
Define this <%=prst%> as default configuration for the `aspera` plugin:
|
1906
1993
|
|
1907
|
-
```
|
1994
|
+
```bash
|
1908
1995
|
<%=cmd%> config preset set default aoc my_aoc_org
|
1909
1996
|
```
|
1910
1997
|
|
1911
1998
|
Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
1912
1999
|
|
1913
|
-
|
2000
|
+
#### <a id="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
|
1914
2001
|
|
1915
2002
|
For a Browser-less, Private Key-based authentication, use the following steps.
|
1916
2003
|
|
1917
|
-
|
2004
|
+
##### Key Pair Generation
|
1918
2005
|
|
1919
2006
|
In order to use JWT for Aspera on Cloud API client authentication,
|
1920
2007
|
a private/public key pair must be generated (without passphrase)
|
@@ -1924,13 +2011,13 @@ This can be done using any of the following method:
|
|
1924
2011
|
|
1925
2012
|
* using the CLI:
|
1926
2013
|
|
1927
|
-
```
|
2014
|
+
```bash
|
1928
2015
|
<%=cmd%> config genkey ~/.aspera/<%=cmd%>/aocapikey
|
1929
2016
|
```
|
1930
2017
|
|
1931
2018
|
* `ssh-keygen`:
|
1932
2019
|
|
1933
|
-
```
|
2020
|
+
```bash
|
1934
2021
|
ssh-keygen -t rsa -f ~/.aspera/<%=cmd%>/aocapikey -N ''
|
1935
2022
|
```
|
1936
2023
|
|
@@ -1938,7 +2025,7 @@ ssh-keygen -t rsa -f ~/.aspera/<%=cmd%>/aocapikey -N ''
|
|
1938
2025
|
|
1939
2026
|
(on some openssl implementation (mac) there is option: -nodes (no DES))
|
1940
2027
|
|
1941
|
-
```
|
2028
|
+
```bash
|
1942
2029
|
APIKEY=~/.aspera/<%=cmd%>/aocapikey
|
1943
2030
|
openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
|
1944
2031
|
openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
|
@@ -1946,63 +2033,81 @@ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
|
|
1946
2033
|
rm -f ${APIKEY}.protected
|
1947
2034
|
```
|
1948
2035
|
|
1949
|
-
|
2036
|
+
##### API Client JWT activation
|
1950
2037
|
|
1951
2038
|
If you are not using the built-in client_id and secret, JWT needs to be authorized in Aspera on Cloud. This can be done in two manners:
|
1952
2039
|
|
1953
2040
|
* Graphically
|
1954
2041
|
|
1955
|
-
|
1956
|
-
|
1957
|
-
|
1958
|
-
|
1959
|
-
|
1960
|
-
|
2042
|
+
* Open a web browser, log to your instance: https://myorg.ibmaspera.com/
|
2043
|
+
* Go to Apps→Admin→Organization→Integrations
|
2044
|
+
* Click on the previously created application
|
2045
|
+
* select tab : "JSON Web Token Auth"
|
2046
|
+
* Modify options if necessary, for instance: activate both options in section "Settings"
|
2047
|
+
* Click "Save"
|
1961
2048
|
|
1962
2049
|
* Using command line
|
1963
2050
|
|
1964
|
-
```
|
2051
|
+
```bash
|
1965
2052
|
<%=cmd%> aoc admin res client list
|
1966
|
-
|
1967
|
-
|
1968
|
-
|
1969
|
-
|
1970
|
-
|
1971
|
-
|
2053
|
+
```
|
2054
|
+
|
2055
|
+
```output
|
2056
|
+
:............:...............:
|
2057
|
+
: id : name :
|
2058
|
+
:............:...............:
|
2059
|
+
: BJLPObQiFw : my-client-app :
|
2060
|
+
:............:...............:
|
2061
|
+
```
|
2062
|
+
|
2063
|
+
```javascript
|
2064
|
+
<%=cmd%> aoc admin res client modify BJLPObQiFw @json:'{"jwt_grant_enabled":true,"explicit_authorization_required":false}'
|
2065
|
+
```
|
2066
|
+
|
2067
|
+
```output
|
1972
2068
|
modified
|
1973
2069
|
```
|
1974
2070
|
|
1975
|
-
|
2071
|
+
#### User key registration
|
1976
2072
|
|
1977
2073
|
The public key must be assigned to your user. This can be done in two manners:
|
1978
2074
|
|
1979
|
-
|
2075
|
+
##### Graphically
|
1980
2076
|
|
1981
2077
|
open the previously generated public key located here: `$HOME/.aspera/<%=cmd%>/aocapikey.pub`
|
1982
2078
|
|
1983
|
-
|
1984
|
-
|
1985
|
-
|
1986
|
-
|
1987
|
-
|
2079
|
+
* Open a web browser, log to your instance: https://myorg.ibmaspera.com/
|
2080
|
+
* Click on the user's icon (top right)
|
2081
|
+
* Select "Account Settings"
|
2082
|
+
* Paste the _Public Key_ in the "Public Key" section
|
2083
|
+
* Click on "Submit"
|
1988
2084
|
|
1989
|
-
|
2085
|
+
##### Using command line
|
1990
2086
|
|
1991
|
-
```
|
2087
|
+
```bash
|
1992
2088
|
<%=cmd%> aoc admin res user list
|
2089
|
+
```
|
2090
|
+
|
2091
|
+
```output
|
1993
2092
|
:........:................:
|
1994
2093
|
: id : name :
|
1995
2094
|
:........:................:
|
1996
2095
|
: 109952 : Tech Support :
|
1997
2096
|
: 109951 : LAURENT MARTIN :
|
1998
2097
|
:........:................:
|
1999
|
-
|
2098
|
+
```
|
2099
|
+
|
2100
|
+
```ruby
|
2101
|
+
<%=cmd%> aoc user profile modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/<%=cmd%>/aocapikey.pub"))}'
|
2102
|
+
```
|
2103
|
+
|
2104
|
+
```output
|
2000
2105
|
modified
|
2001
2106
|
```
|
2002
2107
|
|
2003
2108
|
Note: the `aspera user info show` command can be used to verify modifications.
|
2004
2109
|
|
2005
|
-
|
2110
|
+
#### <%=prst%> modification for JWT
|
2006
2111
|
|
2007
2112
|
To activate default use of JWT authentication for <%=tool%> using the <%=prst%>, do the following:
|
2008
2113
|
|
@@ -2012,7 +2117,7 @@ To activate default use of JWT authentication for <%=tool%> using the <%=prst%>,
|
|
2012
2117
|
|
2013
2118
|
Execute:
|
2014
2119
|
|
2015
|
-
```
|
2120
|
+
```bash
|
2016
2121
|
<%=cmd%> config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/<%=cmd%>/aocapikey --username=laurent.martin.aspera@fr.ibm.com
|
2017
2122
|
```
|
2018
2123
|
|
@@ -2020,30 +2125,31 @@ Note: the private key argument represents the actual PEM string. In order to rea
|
|
2020
2125
|
|
2021
2126
|
After this last step, commands do not require web login anymore.
|
2022
2127
|
|
2023
|
-
|
2024
|
-
### <a id="aocfirst"></a>First Use
|
2128
|
+
#### <a id="aocfirst"></a>First Use
|
2025
2129
|
|
2026
2130
|
Once client has been registered and <%=prst%> created: <%=tool%> can be used:
|
2027
2131
|
|
2028
|
-
```
|
2132
|
+
```bash
|
2029
2133
|
<%=cmd%> aoc files br /
|
2134
|
+
```
|
2135
|
+
|
2136
|
+
```output
|
2030
2137
|
Current Workspace: Default Workspace (default)
|
2031
2138
|
empty
|
2032
2139
|
```
|
2033
2140
|
|
2034
|
-
|
2035
|
-
## Administration
|
2141
|
+
### Administration
|
2036
2142
|
|
2037
2143
|
The `admin` command allows several administrative tasks (and require admin privilege).
|
2038
2144
|
|
2039
2145
|
It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
|
2040
2146
|
|
2041
|
-
|
2147
|
+
#### Bulk creation and deletion of resource
|
2042
2148
|
|
2043
2149
|
Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
|
2044
2150
|
In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
|
2045
2151
|
|
2046
|
-
|
2152
|
+
#### Listing resources
|
2047
2153
|
|
2048
2154
|
The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
|
2049
2155
|
|
@@ -2055,7 +2161,7 @@ The following parameters are supported:
|
|
2055
2161
|
* `sort`: name of fields to sort results, prefix with `-` for reverse order.
|
2056
2162
|
* `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
2057
2163
|
* `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
2058
|
-
* `page` : native api parameter, in general do not use (added by
|
2164
|
+
* `page` : native api parameter, in general do not use (added by
|
2059
2165
|
* `per_page` : native api parameter, number of items par api call, in general do not use
|
2060
2166
|
* Other specific parameters depending on resource type.
|
2061
2167
|
|
@@ -2073,19 +2179,19 @@ Examples:
|
|
2073
2179
|
|
2074
2180
|
* List users with `laurent` in name:
|
2075
2181
|
|
2076
|
-
```
|
2182
|
+
```javascript
|
2077
2183
|
<%=cmd%> aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
|
2078
2184
|
```
|
2079
2185
|
|
2080
2186
|
* List users who logged-in before a date:
|
2081
2187
|
|
2082
|
-
```
|
2188
|
+
```javascript
|
2083
2189
|
<%=cmd%> aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
2084
2190
|
```
|
2085
2191
|
|
2086
2192
|
* List external users and sort in reverse alphabetical order using name:
|
2087
2193
|
|
2088
|
-
```
|
2194
|
+
```javascript
|
2089
2195
|
<%=cmd%> aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
|
2090
2196
|
```
|
2091
2197
|
|
@@ -2093,34 +2199,34 @@ Refer to the AoC API for full list of query parameters, or use the browser in de
|
|
2093
2199
|
|
2094
2200
|
Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
|
2095
2201
|
|
2096
|
-
|
2202
|
+
#### <a id="res_select"></a>Selecting a resource
|
2097
2203
|
|
2098
2204
|
Resources are identified by a unique `id`, as well as a unique `name` (case insensitive).
|
2099
2205
|
|
2100
2206
|
To execute an action on a specific resource, select it using one of those methods:
|
2101
2207
|
|
2102
|
-
*
|
2208
|
+
* **recommended:** give id directly on command line *after the action*: `aoc admin res node show 123`
|
2103
2209
|
* give name on command line *after the action*: `aoc admin res node show name abc`
|
2104
2210
|
* provide option `id` : `aoc admin res node show --id=123`
|
2105
2211
|
* provide option `name` : `aoc admin res node show --name=abc`
|
2106
2212
|
|
2107
|
-
|
2213
|
+
#### Access Key secrets
|
2108
2214
|
|
2109
2215
|
In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
|
2110
2216
|
It is usually provided using the `secret` option.
|
2111
2217
|
For example in a command like:
|
2112
2218
|
|
2113
|
-
```
|
2219
|
+
```bash
|
2114
2220
|
<%=cmd%> aoc admin res node --id=123 --secret="secret1" v3 info
|
2115
2221
|
```
|
2116
2222
|
|
2117
2223
|
It is also possible to provide a set of secrets used on a regular basis using the [secret vault](#vault).
|
2118
2224
|
|
2119
|
-
|
2225
|
+
#### Activity
|
2120
2226
|
|
2121
2227
|
The activity app can be queried with:
|
2122
2228
|
|
2123
|
-
```
|
2229
|
+
```bash
|
2124
2230
|
<%=cmd%> aoc admin analytics transfers
|
2125
2231
|
```
|
2126
2232
|
|
@@ -2128,7 +2234,7 @@ It can also support filters and send notification using option `notif_to`. a tem
|
|
2128
2234
|
|
2129
2235
|
`mytemplate.erb`:
|
2130
2236
|
|
2131
|
-
```
|
2237
|
+
```bash
|
2132
2238
|
From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
|
2133
2239
|
To: <<%='<'%>%=ev['user_email']%>>
|
2134
2240
|
Subject: <%='<'%>%=ev['files_completed']%> files received
|
@@ -2139,13 +2245,14 @@ We received <%='<'%>%=ev['files_completed']%> files for a total of <%='<'%>%=ev[
|
|
2139
2245
|
|
2140
2246
|
Thank you.
|
2141
2247
|
```
|
2248
|
+
|
2142
2249
|
The environment provided contains the following additional variable:
|
2143
2250
|
|
2144
2251
|
* ev : all details on the transfer event
|
2145
2252
|
|
2146
2253
|
Example:
|
2147
2254
|
|
2148
|
-
```
|
2255
|
+
```javascript
|
2149
2256
|
<%=cmd%> aoc admin analytics transfers --once-only=yes --lock-port=12345 \
|
2150
2257
|
--query=@json:'{"status":"completed","direction":"receive"}' \
|
2151
2258
|
--notif-to=active --notif-template=@file:mytemplate.erb
|
@@ -2159,19 +2266,22 @@ Options:
|
|
2159
2266
|
|
2160
2267
|
Note this must not be executed in less than 5 minutes because the analytics interface accepts only a period of time between 5 minutes and 6 months. The period is [date of previous execution]..[now].
|
2161
2268
|
|
2162
|
-
|
2269
|
+
#### Transfer: Using specific transfer ports
|
2163
2270
|
|
2164
2271
|
By default transfer nodes are expected to use ports TCP/UDP 33001. The web UI enforces that.
|
2165
2272
|
The option `default_ports` ([yes]/no) allows <%=cmd%> to retrieve the server ports from an API call (download_setup) which reads the information from `aspera.conf` on the server.
|
2166
2273
|
|
2167
|
-
|
2274
|
+
#### Using ATS
|
2168
2275
|
|
2169
2276
|
Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `aoc admin ats`.
|
2170
2277
|
|
2171
|
-
|
2278
|
+
#### Example: Bulk creation of users
|
2172
2279
|
|
2173
|
-
```
|
2280
|
+
```javascript
|
2174
2281
|
<%=cmd%> aoc admin res user create --bulk=yes @json:'[{"email":"dummyuser1@example.com"},{"email":"dummyuser2@example.com"}]'
|
2282
|
+
```
|
2283
|
+
|
2284
|
+
```output
|
2175
2285
|
:.......:.........:
|
2176
2286
|
: id : status :
|
2177
2287
|
:.......:.........:
|
@@ -2180,20 +2290,38 @@ Refer to section "Examples" of [ATS](#ats) and substitute command `ats` with `ao
|
|
2180
2290
|
:.......:.........:
|
2181
2291
|
```
|
2182
2292
|
|
2183
|
-
|
2293
|
+
#### Example: Find with filter and delete
|
2184
2294
|
|
2185
|
-
```
|
2295
|
+
```javascript
|
2186
2296
|
<%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id,email
|
2297
|
+
```
|
2298
|
+
|
2299
|
+
```output
|
2187
2300
|
:.......:........................:
|
2188
2301
|
: id : email :
|
2189
2302
|
:.......:........................:
|
2190
2303
|
: 98398 : dummyuser1@example.com :
|
2191
2304
|
: 98399 : dummyuser2@example.com :
|
2192
2305
|
:.......:........................:
|
2306
|
+
```
|
2307
|
+
|
2308
|
+
```bash
|
2193
2309
|
thelist=$(<%=cmd%> aoc admin res user list --query='@json:{"q":"dummyuser"}' --fields=id --format=json --display=data|jq -cr 'map(.id)')
|
2310
|
+
```
|
2311
|
+
|
2312
|
+
```bash
|
2194
2313
|
echo $thelist
|
2314
|
+
```
|
2315
|
+
|
2316
|
+
```javascript
|
2195
2317
|
["113501","354061"]
|
2318
|
+
```
|
2319
|
+
|
2320
|
+
```bash
|
2196
2321
|
<%=cmd%> aoc admin res user --bulk=yes --id=@json:"$thelist" delete
|
2322
|
+
```
|
2323
|
+
|
2324
|
+
```output
|
2197
2325
|
:.......:.........:
|
2198
2326
|
: id : status :
|
2199
2327
|
:.......:.........:
|
@@ -2202,18 +2330,21 @@ echo $thelist
|
|
2202
2330
|
:.......:.........:
|
2203
2331
|
```
|
2204
2332
|
|
2205
|
-
|
2333
|
+
#### Example: <a id="deactuser"></a>Find deactivated users since more than 2 years
|
2206
2334
|
|
2207
|
-
```
|
2335
|
+
```ruby
|
2208
2336
|
<%=cmd%> aoc admin res user list --query=@ruby:'{"deactivated"=>true,"q"=>"last_login_at:<#{(DateTime.now.to_time.utc-2*365*86400).iso8601}"}'
|
2209
2337
|
```
|
2210
2338
|
|
2211
2339
|
To delete them use the same method as before
|
2212
2340
|
|
2213
|
-
|
2341
|
+
#### Example: Display current user's workspaces
|
2214
2342
|
|
2343
|
+
```bash
|
2344
|
+
<%=cmd%> aoc user workspaces list
|
2215
2345
|
```
|
2216
|
-
|
2346
|
+
|
2347
|
+
```output
|
2217
2348
|
:......:............................:
|
2218
2349
|
: id : name :
|
2219
2350
|
:......:............................:
|
@@ -2223,45 +2354,43 @@ To delete them use the same method as before
|
|
2223
2354
|
:......:............................:
|
2224
2355
|
```
|
2225
2356
|
|
2226
|
-
|
2357
|
+
#### Example: Create a sub access key in a "node"
|
2227
2358
|
|
2228
2359
|
Creation of a sub-access key is like creation of access key with the following difference: authentication to node API is made with accesskey (master access key) and only the path parameter is provided: it is relative to the storage root of the master key. (id and secret are optional)
|
2229
2360
|
|
2230
|
-
```
|
2361
|
+
```bash
|
2231
2362
|
<%=cmd%> aoc admin resource node --name=_node_name_ --secret=_secret_ v4 access_key create --value=@json:'{"storage":{"path":"/folder1"}}'
|
2232
2363
|
```
|
2233
2364
|
|
2234
|
-
|
2365
|
+
#### Example: Display transfer events (ops/transfer)
|
2235
2366
|
|
2236
|
-
```
|
2367
|
+
```bash
|
2237
2368
|
<%=cmd%> aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
|
2238
2369
|
```
|
2239
2370
|
|
2240
2371
|
Examples of query (TODO: cleanup):
|
2241
2372
|
|
2242
|
-
```
|
2373
|
+
```javascript
|
2243
2374
|
{"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
|
2375
|
+
```
|
2244
2376
|
|
2377
|
+
```javascript
|
2245
2378
|
{"tag":"aspera.files.package_id=LA8OU3p8w"}
|
2246
|
-
|
2247
|
-
# filter= 'id', 'short_summary', or 'summary'
|
2248
|
-
# count=nnn
|
2249
|
-
# tag=x.y.z%3Dvalue
|
2250
|
-
# iteration_token=nnn
|
2251
|
-
# after_time=2016-05-01T23:53:09Z
|
2252
|
-
# active_only=true|false
|
2253
2379
|
```
|
2254
2380
|
|
2255
|
-
|
2381
|
+
#### Example: Display node events (events)
|
2256
2382
|
|
2257
|
-
```
|
2383
|
+
```bash
|
2258
2384
|
<%=cmd%> aoc admin res node --secret=_secret_ v3 events
|
2259
2385
|
```
|
2260
2386
|
|
2261
|
-
|
2387
|
+
#### Example: Display members of a workspace
|
2262
2388
|
|
2263
|
-
```
|
2389
|
+
```javascript
|
2264
2390
|
<%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
2391
|
+
```
|
2392
|
+
|
2393
|
+
```output
|
2265
2394
|
:.............:.........:..................................:
|
2266
2395
|
: member_type : manager : member.email :
|
2267
2396
|
:.............:.........:..................................:
|
@@ -2276,54 +2405,57 @@ Examples of query (TODO: cleanup):
|
|
2276
2405
|
|
2277
2406
|
other query parameters:
|
2278
2407
|
|
2279
|
-
```
|
2408
|
+
```javascript
|
2280
2409
|
{"workspace_membership_through":true,"include_indirect":true}
|
2281
2410
|
```
|
2282
2411
|
|
2283
|
-
|
2412
|
+
#### Example: <a id="aoc_sample_member"></a>add all members of a workspace to another workspace
|
2284
2413
|
|
2285
2414
|
a- Get id of first workspace
|
2286
2415
|
|
2287
|
-
```
|
2416
|
+
```bash
|
2288
2417
|
WS1='First Workspace'
|
2289
2418
|
WS1ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
|
2290
2419
|
```
|
2291
2420
|
|
2292
2421
|
b- Get id of second workspace
|
2293
2422
|
|
2294
|
-
```
|
2423
|
+
```bash
|
2295
2424
|
WS2='Second Workspace'
|
2296
2425
|
WS2ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
|
2297
2426
|
```
|
2298
2427
|
|
2299
2428
|
c- Extract membership information
|
2300
2429
|
|
2301
|
-
```
|
2430
|
+
```bash
|
2302
2431
|
<%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
|
2303
2432
|
```
|
2304
2433
|
|
2305
2434
|
d- Convert to creation data for second workspace:
|
2306
2435
|
|
2307
|
-
```
|
2436
|
+
```bash
|
2308
2437
|
grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
|
2309
2438
|
```
|
2310
2439
|
|
2311
2440
|
or, using jq:
|
2312
2441
|
|
2313
|
-
```
|
2442
|
+
```bash
|
2314
2443
|
jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
|
2315
2444
|
```
|
2316
2445
|
|
2317
2446
|
e- Add members to second workspace
|
2318
2447
|
|
2319
|
-
```
|
2448
|
+
```bash
|
2320
2449
|
<%=cmd%> aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
|
2321
2450
|
```
|
2322
2451
|
|
2323
|
-
|
2452
|
+
#### Example: Get users who did not log since a date
|
2324
2453
|
|
2325
|
-
```
|
2454
|
+
```javascript
|
2326
2455
|
<%=cmd%> aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
2456
|
+
```
|
2457
|
+
|
2458
|
+
```output
|
2327
2459
|
:...............................:
|
2328
2460
|
: email :
|
2329
2461
|
:...............................:
|
@@ -2332,20 +2464,23 @@ e- Add members to second workspace
|
|
2332
2464
|
:...............................:
|
2333
2465
|
```
|
2334
2466
|
|
2335
|
-
|
2467
|
+
#### Example: List "Limited" users
|
2336
2468
|
|
2337
|
-
```
|
2469
|
+
```javascript
|
2338
2470
|
<%=cmd%> aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
|
2339
2471
|
```
|
2340
2472
|
|
2341
|
-
|
2473
|
+
#### Example: Perform a multi Gbps transfer between two remote shared folders
|
2342
2474
|
|
2343
2475
|
In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
|
2344
2476
|
|
2345
2477
|
First, setup the environment (skip if already done)
|
2346
2478
|
|
2347
|
-
```
|
2479
|
+
```bash
|
2348
2480
|
<%=cmd%> conf wizard --url=https://sedemo.ibmaspera.com --username=laurent.martin.aspera@fr.ibm.com
|
2481
|
+
```
|
2482
|
+
|
2483
|
+
```output
|
2349
2484
|
Detected: Aspera on Cloud
|
2350
2485
|
Preparing preset: aoc_sedemo
|
2351
2486
|
Using existing key:
|
@@ -2364,7 +2499,7 @@ Setting config preset as default for aspera
|
|
2364
2499
|
saving config file
|
2365
2500
|
Done.
|
2366
2501
|
You can test with:
|
2367
|
-
<%=cmd%> aoc user
|
2502
|
+
<%=cmd%> aoc user profile show
|
2368
2503
|
```
|
2369
2504
|
|
2370
2505
|
This creates the option preset "aoc_<org name>" to allow seamless command line access and sets it as default for aspera on cloud.
|
@@ -2373,21 +2508,27 @@ Then, create two shared folders located in two regions, in your files home, in a
|
|
2373
2508
|
|
2374
2509
|
Then, transfer between those:
|
2375
2510
|
|
2376
|
-
```
|
2511
|
+
```javascript
|
2377
2512
|
<%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
2378
2513
|
```
|
2379
2514
|
|
2380
|
-
|
2515
|
+
#### Example: create registration key to register a node
|
2381
2516
|
|
2382
|
-
```
|
2517
|
+
```javascript
|
2383
2518
|
<%=cmd%> aoc admin res client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
2519
|
+
```
|
2520
|
+
|
2521
|
+
```output
|
2384
2522
|
jfqslfdjlfdjfhdjklqfhdkl
|
2385
2523
|
```
|
2386
2524
|
|
2387
|
-
|
2525
|
+
#### Example: delete all registration keys
|
2388
2526
|
|
2389
|
-
```
|
2527
|
+
```bash
|
2390
2528
|
<%=cmd%> aoc admin res client list --fields=id --format=csv|<%=cmd%> aoc admin res client delete --bulk=yes --id=@lines:@stdin:
|
2529
|
+
```
|
2530
|
+
|
2531
|
+
```output
|
2391
2532
|
+-----+---------+
|
2392
2533
|
| id | status |
|
2393
2534
|
+-----+---------+
|
@@ -2398,7 +2539,7 @@ jfqslfdjlfdjfhdjklqfhdkl
|
|
2398
2539
|
+-----+---------+
|
2399
2540
|
```
|
2400
2541
|
|
2401
|
-
|
2542
|
+
#### Example: Create a node
|
2402
2543
|
|
2403
2544
|
AoC nodes as actually composed with two related entities:
|
2404
2545
|
|
@@ -2412,7 +2553,7 @@ So, for example, the creation of a node using ATS in IBM Cloud looks like (see o
|
|
2412
2553
|
|
2413
2554
|
* create the access key on ATS
|
2414
2555
|
|
2415
|
-
```
|
2556
|
+
```javascript
|
2416
2557
|
<%=cmd%> aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
|
2417
2558
|
```
|
2418
2559
|
|
@@ -2420,79 +2561,90 @@ Take a note of the randomly generated `id` and `secret`.
|
|
2420
2561
|
|
2421
2562
|
* Retrieve the ATS node address
|
2422
2563
|
|
2423
|
-
```
|
2424
|
-
<%=cmd%> aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv
|
2564
|
+
```bash
|
2565
|
+
<%=cmd%> aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
|
2425
2566
|
```
|
2426
2567
|
|
2427
2568
|
* Create the node entity
|
2428
2569
|
|
2429
|
-
```
|
2570
|
+
```javascript
|
2430
2571
|
<%=cmd%> aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
|
2431
2572
|
```
|
2432
2573
|
|
2433
2574
|
Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
|
2434
2575
|
|
2435
|
-
###
|
2436
|
-
|
2437
|
-
First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
2438
|
-
(To find out available filters, consult the API definition, or use the web interface in developer mode).
|
2439
|
-
|
2440
|
-
Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
|
2441
|
-
|
2442
|
-
```
|
2443
|
-
shbxid=$(<%=cmd%> aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
|
2444
|
-
|
2445
|
-
<%=cmd%> aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2446
|
-
```
|
2447
|
-
|
2448
|
-
## Packages
|
2576
|
+
### Packages
|
2449
2577
|
|
2450
2578
|
The webmail-like application.
|
2451
2579
|
|
2452
|
-
|
2580
|
+
#### Send a Package
|
2453
2581
|
|
2454
|
-
|
2582
|
+
General syntax:
|
2455
2583
|
|
2456
|
-
```
|
2584
|
+
```bash
|
2457
2585
|
<%=cmd%> aoc packages send --value=[package extended value] [other parameters such as file list and transfer parameters]
|
2458
2586
|
```
|
2459
2587
|
|
2460
2588
|
Notes:
|
2461
2589
|
|
2462
2590
|
* The `value` option can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package in JSON to list attributes.
|
2463
|
-
* List allowed shared inbox destinations with: `<%=cmd%> aoc
|
2464
|
-
* Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
2591
|
+
* List allowed shared inbox destinations with: `<%=cmd%> aoc packages shared_inboxes list`
|
2592
|
+
* Use fields: `recipients` and/or `bcc_recipients` to provide the list of recipients: user or shared inbox.
|
2465
2593
|
* Provide either ids as expected by API: `"recipients":[{"type":"dropbox","id":"1234"}]`
|
2466
2594
|
* or just names: `"recipients":[{"The Dest"}]` . <%=cmd%> will resolve the list of email addresses and dropbox names to the expected type/id list, based on case insensitive partial match.
|
2467
2595
|
* If a user recipient (email) is not already registered and the workspace allows external users, then the package is sent to an external user, and
|
2468
2596
|
* if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account
|
2469
2597
|
* if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
2470
2598
|
|
2471
|
-
|
2599
|
+
#### Example: Send a package with one file to two users, using their email
|
2472
2600
|
|
2473
|
-
|
2601
|
+
```javascript
|
2602
|
+
<%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' my_file.dat
|
2603
|
+
```
|
2604
|
+
|
2605
|
+
#### Example: Send a package to a shared inbox with metadata
|
2474
2606
|
|
2607
|
+
```javascript
|
2608
|
+
<%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox With Meta"],"metadata":{"Project Id":"123","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' ~/Documents/Samples/200KB.1
|
2475
2609
|
```
|
2476
|
-
|
2610
|
+
|
2611
|
+
It is also possible to use identifiers and API parameters:
|
2612
|
+
|
2613
|
+
```javascript
|
2614
|
+
<%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":[{"type":"dropbox","id":"12345"}],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
2477
2615
|
```
|
2478
2616
|
|
2479
|
-
|
2617
|
+
#### Example: List packages in a given shared inbox
|
2618
|
+
|
2619
|
+
When user packages are listed, the following query is used:
|
2480
2620
|
|
2621
|
+
```javascript
|
2622
|
+
{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}
|
2481
2623
|
```
|
2482
|
-
|
2624
|
+
|
2625
|
+
To list packages in a shared inbox, the query has to be specified with withe the shared inbox by name or its identifier. Additionnal parameters can be specified, as supported by the API (to find out available filters, consult the API definition, or use the web interface in developer mode). The current workspace is added unless specified in the query.
|
2626
|
+
|
2627
|
+
Using shared inbox name:
|
2628
|
+
|
2629
|
+
```javascript
|
2630
|
+
<%=cmd%> aoc packages list --query=@json:'{"dropbox_name":"My Shared Inbox","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2483
2631
|
```
|
2484
2632
|
|
2485
|
-
|
2633
|
+
Using shared inbox identifier: first retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
2486
2634
|
|
2635
|
+
```bash
|
2636
|
+
shbxid=$(<%=cmd%> aoc packages shared_inboxes show name 'My Shared Inbox' --format=csv --display=data --fields=id --transpose-single=no)
|
2487
2637
|
```
|
2488
|
-
|
2638
|
+
|
2639
|
+
```javascript
|
2640
|
+
<%=cmd%> aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2489
2641
|
```
|
2490
2642
|
|
2491
|
-
|
2643
|
+
#### <a id="aoccargo"></a>Receive new packages only (Cargo)
|
2492
2644
|
|
2493
2645
|
It is possible to automatically download new packages, like using Aspera Cargo:
|
2494
2646
|
|
2495
|
-
```
|
2647
|
+
```bash
|
2496
2648
|
<%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
|
2497
2649
|
```
|
2498
2650
|
|
@@ -2506,38 +2658,38 @@ Typically, one would execute this command on a regular basis, using the method o
|
|
2506
2658
|
* Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
2507
2659
|
* etc...
|
2508
2660
|
|
2509
|
-
|
2661
|
+
### Files
|
2510
2662
|
|
2511
2663
|
Folder sharing app.
|
2512
2664
|
|
2513
|
-
|
2665
|
+
#### Download Files
|
2514
2666
|
|
2515
2667
|
Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
|
2516
2668
|
|
2517
2669
|
* if only one source is provided, it is downloaded
|
2518
2670
|
* if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
|
2519
2671
|
|
2520
|
-
|
2672
|
+
#### Shared folders
|
2521
2673
|
|
2522
2674
|
* list shared folders in node
|
2523
2675
|
|
2524
|
-
```
|
2676
|
+
```bash
|
2525
2677
|
<%=cmd%> aoc admin res node --id=8669 shared_folders
|
2526
2678
|
```
|
2527
2679
|
|
2528
2680
|
* list shared folders in workspace
|
2529
2681
|
|
2530
|
-
```
|
2682
|
+
```bash
|
2531
2683
|
<%=cmd%> aoc admin res workspace --id=10818 shared_folders
|
2532
2684
|
```
|
2533
2685
|
|
2534
2686
|
* list members of shared folder
|
2535
2687
|
|
2536
|
-
```
|
2688
|
+
```bash
|
2537
2689
|
<%=cmd%> aoc admin res node --id=8669 v4 perm 82 show
|
2538
2690
|
```
|
2539
2691
|
|
2540
|
-
|
2692
|
+
#### Cross Organization transfers
|
2541
2693
|
|
2542
2694
|
It is possible to transfer files directly between organizations without having to first download locally and then upload...
|
2543
2695
|
|
@@ -2551,7 +2703,7 @@ Procedure to send a file from org1 to org2:
|
|
2551
2703
|
* Check that access works and locate the destination folder `mydestfolder`
|
2552
2704
|
* execute the following:
|
2553
2705
|
|
2554
|
-
```
|
2706
|
+
```bash
|
2555
2707
|
<%=cmd%> -Porg1 aoc files node_info /mydestfolder --format=json --display=data | <%=cmd%> -Porg2 aoc files upload mysourcefile --transfer=node --transfer-info=@json:@stdin:
|
2556
2708
|
```
|
2557
2709
|
|
@@ -2567,7 +2719,7 @@ Explanation:
|
|
2567
2719
|
* `--transfer=node` use transfer agent type `node` instead of default [`direct`](#agt_direct)
|
2568
2720
|
* `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
|
2569
2721
|
|
2570
|
-
|
2722
|
+
#### Find Files
|
2571
2723
|
|
2572
2724
|
The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
|
2573
2725
|
|
@@ -2584,29 +2736,29 @@ Examples of expressions: (using like this: `--value=exec:'<expression>'`)
|
|
2584
2736
|
|
2585
2737
|
* Find files more recent than 100 days
|
2586
2738
|
|
2587
|
-
```
|
2739
|
+
```bash
|
2588
2740
|
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
2589
2741
|
```
|
2590
2742
|
|
2591
2743
|
* Find files older than 1 year on a given node and store in file list
|
2592
2744
|
|
2593
|
-
```
|
2745
|
+
```bash
|
2594
2746
|
<%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
|
2595
2747
|
```
|
2596
2748
|
|
2597
2749
|
* Delete the files, one by one
|
2598
2750
|
|
2599
|
-
```
|
2751
|
+
```bash
|
2600
2752
|
cat my_file_list.txt|while read path;do echo <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
|
2601
2753
|
```
|
2602
2754
|
|
2603
2755
|
* Delete the files in bulk
|
2604
2756
|
|
2605
|
-
```
|
2757
|
+
```bash
|
2606
2758
|
cat my_file_list.txt | <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
|
2607
2759
|
```
|
2608
2760
|
|
2609
|
-
|
2761
|
+
## <a id="ats"></a>Plugin: Aspera Transfer Service
|
2610
2762
|
|
2611
2763
|
ATS is usable either :
|
2612
2764
|
|
@@ -2614,15 +2766,18 @@ ATS is usable either :
|
|
2614
2766
|
|
2615
2767
|
* or from an IBM Cloud subscription : <%=cmd%> ats : use IBM Cloud API key authentication
|
2616
2768
|
|
2617
|
-
|
2769
|
+
### IBM Cloud ATS : creation of api key
|
2618
2770
|
|
2619
2771
|
This section is about using ATS with an IBM cloud subscription.
|
2620
2772
|
If you are using ATS as part of AoC, then authentication is thropugh AoC, not IBM Cloud.
|
2621
2773
|
|
2622
2774
|
First get your IBM Cloud APIkey. For instance, it can be created using the IBM Cloud web interface, or using command line:
|
2623
2775
|
|
2624
|
-
```
|
2776
|
+
```bash
|
2625
2777
|
ibmcloud iam api-key-create mykeyname -d 'my sample key'
|
2778
|
+
```
|
2779
|
+
|
2780
|
+
```output
|
2626
2781
|
OK
|
2627
2782
|
API key mykeyname was created
|
2628
2783
|
|
@@ -2638,22 +2793,40 @@ UUID ApiKey-05b8fadf-e7fe-4bc4-93a9-6fd348c5ab1f
|
|
2638
2793
|
|
2639
2794
|
References:
|
2640
2795
|
|
2641
|
-
|
2642
|
-
|
2796
|
+
* [https://console.bluemix.net/docs/iam/userid_keys.html#userapikey](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey)
|
2797
|
+
* [https://ibm.ibmaspera.com/helpcenter/transfer-service](https://ibm.ibmaspera.com/helpcenter/transfer-service)
|
2643
2798
|
|
2644
2799
|
Then, to register the key by default for the ats plugin, create a preset. Execute:
|
2645
2800
|
|
2646
|
-
```
|
2801
|
+
```bash
|
2647
2802
|
<%=cmd%> config preset update my_ibm_ats --ibm-api-key=my_secret_api_key_here_8f8d9fdakjhfsashjk678
|
2803
|
+
```
|
2804
|
+
|
2805
|
+
```bash
|
2648
2806
|
<%=cmd%> config preset set default ats my_ibm_ats
|
2807
|
+
```
|
2808
|
+
|
2809
|
+
```bash
|
2649
2810
|
<%=cmd%> ats api_key instances
|
2811
|
+
```
|
2812
|
+
|
2813
|
+
```output
|
2650
2814
|
+--------------------------------------+
|
2651
2815
|
| instance |
|
2652
2816
|
+--------------------------------------+
|
2653
2817
|
| aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee |
|
2654
2818
|
+--------------------------------------+
|
2819
|
+
```
|
2820
|
+
|
2821
|
+
```bash
|
2655
2822
|
<%=cmd%> config preset update my_ibm_ats --instance=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
|
2823
|
+
```
|
2824
|
+
|
2825
|
+
```bash
|
2656
2826
|
<%=cmd%> ats api_key create
|
2827
|
+
```
|
2828
|
+
|
2829
|
+
```output
|
2657
2830
|
+--------+----------------------------------------------+
|
2658
2831
|
| key | value |
|
2659
2832
|
+--------+----------------------------------------------+
|
@@ -2663,52 +2836,49 @@ Then, to register the key by default for the ats plugin, create a preset. Execut
|
|
2663
2836
|
<%=cmd%> config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
|
2664
2837
|
```
|
2665
2838
|
|
2666
|
-
|
2839
|
+
### Misc. Examples
|
2667
2840
|
|
2668
2841
|
Example: create access key on IBM Cloud (softlayer):
|
2669
2842
|
|
2670
|
-
```
|
2843
|
+
```javascript
|
2671
2844
|
<%=cmd%> ats access_key create --cloud=softlayer --region=ams --params=@json:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'
|
2672
2845
|
```
|
2673
2846
|
|
2674
2847
|
Example: create access key on AWS:
|
2675
2848
|
|
2676
|
-
```
|
2849
|
+
```javascript
|
2677
2850
|
<%=cmd%> ats access_key create --cloud=aws --region=eu-west-1 --params=@json:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'
|
2678
|
-
|
2679
2851
|
```
|
2680
2852
|
|
2681
2853
|
Example: create access key on Azure SAS:
|
2682
2854
|
|
2683
|
-
```
|
2855
|
+
```javascript
|
2684
2856
|
<%=cmd%> ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://containername.blob.core.windows.net/blobname?sr=c&..."},"path":"/"}}'
|
2685
|
-
|
2686
2857
|
```
|
2687
2858
|
|
2688
2859
|
(Note that the blob name is mandatory after server address and before parameters. and that parameter sr=c is mandatory.)
|
2689
2860
|
|
2690
2861
|
Example: create access key on Azure:
|
2691
2862
|
|
2692
|
-
```
|
2863
|
+
```javascript
|
2693
2864
|
<%=cmd%> ats access_key create --cloud=azure --region=eastus --params=@json:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'
|
2694
|
-
|
2695
2865
|
```
|
2696
2866
|
|
2697
2867
|
delete all my access keys:
|
2698
2868
|
|
2699
|
-
```
|
2869
|
+
```bash
|
2700
2870
|
for k in $(<%=cmd%> ats access_key list --field=id --format=csv);do <%=cmd%> ats access_key id $k delete;done
|
2701
2871
|
```
|
2702
2872
|
|
2703
2873
|
The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
|
2704
2874
|
|
2705
|
-
|
2875
|
+
## Plugin: IBM Aspera High Speed Transfer Server (transfer)
|
2706
2876
|
|
2707
2877
|
This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
|
2708
2878
|
It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
|
2709
2879
|
Modern mode is to use the node API and transfer tokens.
|
2710
2880
|
|
2711
|
-
|
2881
|
+
### Authentication
|
2712
2882
|
|
2713
2883
|
Both password and SSH keys auth are supported.
|
2714
2884
|
|
@@ -2716,7 +2886,7 @@ If username is not provided, the default transfer user `xfer` is used.
|
|
2716
2886
|
|
2717
2887
|
If no SSH password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used:
|
2718
2888
|
|
2719
|
-
```
|
2889
|
+
```javascript
|
2720
2890
|
<%=cmd%> server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
|
2721
2891
|
```
|
2722
2892
|
|
@@ -2726,7 +2896,7 @@ Each value is a path to a private key and is expanded (`~` is replaced with the
|
|
2726
2896
|
|
2727
2897
|
Examples:
|
2728
2898
|
|
2729
|
-
```
|
2899
|
+
```bash
|
2730
2900
|
<%=cmd%> server --ssh-keys=~/.ssh/id_rsa
|
2731
2901
|
<%=cmd%> server --ssh-keys=@list:,~/.ssh/id_rsa
|
2732
2902
|
<%=cmd%> server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
|
@@ -2737,13 +2907,13 @@ By default the ssh library expect that an ssh-agent is running.
|
|
2737
2907
|
|
2738
2908
|
On Linux, if you get an error message such as:
|
2739
2909
|
|
2740
|
-
```
|
2910
|
+
```bash
|
2741
2911
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: Agent not configured
|
2742
2912
|
```
|
2743
2913
|
|
2744
2914
|
or on Windows:
|
2745
2915
|
|
2746
|
-
```
|
2916
|
+
```bash
|
2747
2917
|
ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
|
2748
2918
|
```
|
2749
2919
|
|
@@ -2754,17 +2924,17 @@ This means that you don't have such an ssh agent running, then:
|
|
2754
2924
|
* [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
|
2755
2925
|
* To disable use of `ssh-agent`, use the option `ssh_option` like this:
|
2756
2926
|
|
2757
|
-
```
|
2927
|
+
```bash
|
2758
2928
|
<%=cmd%> server --ssh-options=@ruby:'{use_agent: false}' ...
|
2759
2929
|
```
|
2760
2930
|
|
2761
2931
|
This can also be set as default using a global preset.
|
2762
2932
|
|
2763
|
-
|
2933
|
+
### Example
|
2764
2934
|
|
2765
2935
|
One can test the `server` application using the well known demo server:
|
2766
2936
|
|
2767
|
-
```
|
2937
|
+
```bash
|
2768
2938
|
<%=cmd%> config initdemo
|
2769
2939
|
<%=cmd%> server browse /aspera-test-dir-large
|
2770
2940
|
<%=cmd%> server download /aspera-test-dir-large/200MB
|
@@ -2772,13 +2942,14 @@ One can test the `server` application using the well known demo server:
|
|
2772
2942
|
|
2773
2943
|
`initdemo` creates a <%=prst%> `demoserver` and set it as default for plugin `server`.
|
2774
2944
|
|
2775
|
-
|
2945
|
+
## Plugin: IBM Aspera High Speed Transfer Server (node)
|
2776
2946
|
|
2777
2947
|
This plugin gives access to capabilities provided by HSTS node API.
|
2778
2948
|
|
2779
|
-
|
2949
|
+
### File Operations
|
2780
2950
|
|
2781
2951
|
It is possible to:
|
2952
|
+
|
2782
2953
|
* browse
|
2783
2954
|
* transfer (upload / download)
|
2784
2955
|
* ...
|
@@ -2788,7 +2959,7 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
2788
2959
|
* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
|
2789
2960
|
* `basic` : transfer spec is created like this:
|
2790
2961
|
|
2791
|
-
```
|
2962
|
+
```javascript
|
2792
2963
|
{
|
2793
2964
|
"remote_host": address of node url,
|
2794
2965
|
"remote_user": "xfer",
|
@@ -2800,29 +2971,29 @@ For transfers, it is possible to control how transfer is authorized using option
|
|
2800
2971
|
|
2801
2972
|
* `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
|
2802
2973
|
|
2803
|
-
|
2974
|
+
### Central
|
2804
2975
|
|
2805
2976
|
The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transferred files.
|
2806
2977
|
|
2807
2978
|
Filtering can be applied:
|
2808
2979
|
|
2809
|
-
```
|
2980
|
+
```bash
|
2810
2981
|
<%=cmd%> node central file list
|
2811
2982
|
```
|
2812
2983
|
|
2813
2984
|
by providing the `validator` option, offline transfer validation can be done.
|
2814
2985
|
|
2815
|
-
|
2986
|
+
### FASP Stream
|
2816
2987
|
|
2817
2988
|
It is possible to start a FASPStream session using the node API:
|
2818
2989
|
|
2819
2990
|
Use the "node stream create" command, then arguments are provided as a [_transfer-spec_](#transferspec).
|
2820
2991
|
|
2821
|
-
```
|
2992
|
+
```javascript
|
2822
2993
|
<%=cmd%> node stream create --ts=@json:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --preset=stream
|
2823
2994
|
```
|
2824
2995
|
|
2825
|
-
|
2996
|
+
### Watchfolder
|
2826
2997
|
|
2827
2998
|
Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/entsrv/3.7.4/es_admin_linux/webhelp/index.html#watchfolder_external/dita/json_conf.html) for watch folder creation.
|
2828
2999
|
|
@@ -2831,52 +3002,58 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
|
|
2831
3002
|
* Start watchd and watchfolderd services running as a system user having access to files
|
2832
3003
|
* configure a watchfolder to define automated transfers
|
2833
3004
|
|
2834
|
-
```
|
3005
|
+
```javascript
|
2835
3006
|
<%=cmd%> node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
2836
3007
|
<%=cmd%> node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
|
2837
3008
|
<%=cmd%> node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"mypassword"}}'
|
2838
3009
|
```
|
2839
3010
|
|
2840
|
-
|
3011
|
+
### Out of Transfer File Validation
|
2841
3012
|
|
2842
3013
|
Follow the Aspera Transfer Server configuration to activate this feature.
|
2843
3014
|
|
2844
|
-
```
|
3015
|
+
```javascript
|
2845
3016
|
<%=cmd%> node central file list --validator=<%=cmd%> --data=@json:'{"file_transfer_filter":{"max_result":1}}'
|
3017
|
+
```
|
3018
|
+
|
3019
|
+
```output
|
2846
3020
|
:..............:..............:............:......................................:
|
2847
3021
|
: session_uuid : file_id : status : path :
|
2848
3022
|
:..............:..............:............:......................................:
|
2849
3023
|
: 1a74444c-... : 084fb181-... : validating : /home/xfer.../PKG - my title/200KB.1 :
|
2850
3024
|
:..............:..............:............:......................................:
|
3025
|
+
```
|
3026
|
+
|
3027
|
+
```javascript
|
2851
3028
|
<%=cmd%> node central file update --validator=<%=cmd%> --data=@json:'{"files":[{"session_uuid": "1a74444c-...","file_id": "084fb181-...","status": "completed"}]}'
|
3029
|
+
```
|
3030
|
+
|
3031
|
+
```output
|
2852
3032
|
updated
|
2853
3033
|
```
|
2854
3034
|
|
2855
|
-
|
3035
|
+
### Example: SHOD to ATS
|
2856
3036
|
|
2857
|
-
Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
|
2858
|
-
transfer files from this third party SHOD instance into our Azure BLOB storage.
|
2859
|
-
Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which
|
2860
|
-
|
2861
|
-
Then create a configuration for the "SHOD" instance in the configuration file: in section
|
2862
|
-
"shares", a configuration named: awsshod.
|
3037
|
+
Scenario: Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
|
3038
|
+
We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
|
3039
|
+
Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which provides access to the node API.
|
3040
|
+
Then create a configuration for the "SHOD" instance in the configuration file: in section "shares", a configuration named: awsshod.
|
2863
3041
|
Create another configuration for the Azure ATS instance: in section "node", named azureats.
|
2864
3042
|
Then execute the following command:
|
2865
3043
|
|
2866
|
-
```
|
3044
|
+
```bash
|
2867
3045
|
<%=cmd%> node download /share/sourcefile --to-folder=/destinationfolder --preset=awsshod --transfer=node --transfer-info=@preset:azureats
|
2868
3046
|
```
|
2869
3047
|
|
2870
|
-
This will get transfer information from the SHOD instance and tell the Azure ATS instance
|
2871
|
-
to download files.
|
3048
|
+
This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.
|
2872
3049
|
|
2873
|
-
|
3050
|
+
### Create access key
|
2874
3051
|
|
2875
|
-
```
|
3052
|
+
```javascript
|
2876
3053
|
<%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
|
2877
3054
|
```
|
2878
3055
|
|
2879
|
-
|
3056
|
+
## Plugin: IBM Aspera Faspex5
|
2880
3057
|
|
2881
3058
|
3 authentication methods are supported:
|
2882
3059
|
|
@@ -2897,35 +3074,35 @@ For boot method: (will be removed in future)
|
|
2897
3074
|
|
2898
3075
|
Use it as password and use `--auth=boot`.
|
2899
3076
|
|
2900
|
-
```
|
3077
|
+
```bash
|
2901
3078
|
<%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
|
2902
3079
|
```
|
2903
3080
|
|
2904
3081
|
Ready to use Faspex5 with CLI.
|
2905
3082
|
|
2906
|
-
|
3083
|
+
## Plugin: IBM Aspera Faspex (4.x)
|
2907
3084
|
|
2908
3085
|
Notes:
|
2909
3086
|
|
2910
3087
|
* The command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
|
2911
3088
|
* For full details on Faspex API, refer to: [Reference on Developer Site](https://developer.ibm.com/apis/catalog/?search=faspex)
|
2912
3089
|
|
2913
|
-
|
3090
|
+
### Listing Packages
|
2914
3091
|
|
2915
3092
|
Command: `faspex package list`
|
2916
3093
|
|
2917
|
-
|
3094
|
+
#### Option `box`
|
2918
3095
|
|
2919
3096
|
By default it looks in box `inbox`, but the following boxes are also supported: `archive` and `sent`, selected with option `box`.
|
2920
3097
|
|
2921
|
-
|
3098
|
+
#### Option `recipient`
|
2922
3099
|
|
2923
3100
|
A user can receive a package because the recipient is:
|
2924
3101
|
|
2925
3102
|
* the user himself (default)
|
2926
3103
|
* the user is part of a dropbox or a workgroup (select with option `recipient` with value `*<name of WG or DB>`
|
2927
3104
|
|
2928
|
-
|
3105
|
+
#### Option `query`
|
2929
3106
|
|
2930
3107
|
As inboxes may be large, it is possible to use the following query parameters:
|
2931
3108
|
|
@@ -2941,15 +3118,15 @@ The API is listed in [Faspex 4 API Reference](https://developer.ibm.com/apis/cat
|
|
2941
3118
|
|
2942
3119
|
If no parameter `max` or `pmax` is provided, then all packages will be listed in the inbox, which result in paged API calls (using parameter: `count` and `page`). By default page is `0` (`10`), it can be increased to have less calls.
|
2943
3120
|
|
2944
|
-
|
3121
|
+
#### Example: list packages in dropbox
|
2945
3122
|
|
2946
|
-
```
|
3123
|
+
```javascript
|
2947
3124
|
<%=cmd%> faspex package list --box=inbox --recipient='*my_dropbox' --query=@json:'{"max":20,"pmax":2,"count":20}'
|
2948
3125
|
```
|
2949
3126
|
|
2950
3127
|
List a maximum of 20 items grouped by pages of 20, with maximum 2 pages in received box (inbox) when received in dropbox `*my_dropbox`.
|
2951
3128
|
|
2952
|
-
|
3129
|
+
### Receiving a Package
|
2953
3130
|
|
2954
3131
|
The command is `package recv`, possible methods are:
|
2955
3132
|
|
@@ -2957,26 +3134,26 @@ The command is `package recv`, possible methods are:
|
|
2957
3134
|
* provide a public link with option `link`
|
2958
3135
|
* provide a `faspe:` URI with option `link`
|
2959
3136
|
|
2960
|
-
```
|
3137
|
+
```bash
|
2961
3138
|
<%=cmd%> faspex package recv --id=12345
|
2962
3139
|
<%=cmd%> faspex package recv --link=faspe://...
|
2963
3140
|
```
|
2964
3141
|
|
2965
3142
|
If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
|
2966
3143
|
|
2967
|
-
```
|
3144
|
+
```bash
|
2968
3145
|
<%=cmd%> faspex package list --recipient='*thedropboxname'
|
2969
3146
|
```
|
2970
3147
|
|
2971
3148
|
if `id` is set to `ALL`, then all packages are downloaded, and if option `once_only`is used, then a persistency file is created to keep track of already downloaded packages.
|
2972
3149
|
|
2973
|
-
|
3150
|
+
### Sending a Package
|
2974
3151
|
|
2975
3152
|
The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
|
2976
3153
|
|
2977
3154
|
Example:
|
2978
3155
|
|
2979
|
-
```
|
3156
|
+
```javascript
|
2980
3157
|
<%=cmd%> faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
2981
3158
|
```
|
2982
3159
|
|
@@ -2987,29 +3164,29 @@ Additional optional parameters in `delivery_info`:
|
|
2987
3164
|
* Package Note: : `"note":"note this and that"`
|
2988
3165
|
* Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
2989
3166
|
|
2990
|
-
|
3167
|
+
### Email notification on transfer
|
2991
3168
|
|
2992
3169
|
Like for any transfer, a notification can be sent by email using parameters: `notif_to` and `notif_template` .
|
2993
3170
|
|
2994
3171
|
Example:
|
2995
3172
|
|
2996
|
-
```
|
3173
|
+
```javascript
|
2997
3174
|
<%=cmd%> faspex package send --delivery-info=@json:'{"title":"test pkg 1","recipients":["aspera.user1@gmail.com"]}' ~/Documents/Samples/200KB.1 --notif-to=aspera.user1@gmail.com --notif-template=@ruby:'%Q{From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>\nTo: <<%='<'%>%=to%>>\nSubject: Package sent: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["metadata"]["_pkg_name"]%> files received\n\nTo user: <%='<'%>%=ts["tags"]["aspera"]["faspex"]["recipients"].first["email"]%>}'
|
2998
3175
|
```
|
2999
3176
|
|
3000
3177
|
In this example the notification template is directly provided on command line. Package information placed in the message are directly taken from the tags in transfer spec. The template can be placed in a file using modifier: `@file:`
|
3001
3178
|
|
3002
|
-
|
3179
|
+
### Operation on dropboxes
|
3003
3180
|
|
3004
3181
|
Example:
|
3005
3182
|
|
3006
|
-
```
|
3183
|
+
```javascript
|
3007
3184
|
<%=cmd%> faspex v4 dropbox create --value=@json:'{"dropbox":{"e_wg_name":"test1","e_wg_desc":"test1"}}'
|
3008
3185
|
<%=cmd%> faspex v4 dropbox list
|
3009
3186
|
<%=cmd%> faspex v4 dropbox delete --id=36
|
3010
3187
|
```
|
3011
3188
|
|
3012
|
-
|
3189
|
+
### Remote sources
|
3013
3190
|
|
3014
3191
|
Faspex lacks an API to list the contents of a remote source (available in web UI). To workaround this,
|
3015
3192
|
the node API is used, for this it is required to add a section ":storage" that links
|
@@ -3038,26 +3215,26 @@ The node configuration name is "my_faspex_node" here.
|
|
3038
3215
|
|
3039
3216
|
Note: the v4 API provide an API for nodes and shares.
|
3040
3217
|
|
3041
|
-
|
3218
|
+
### Automated package download (cargo)
|
3042
3219
|
|
3043
3220
|
It is possible to tell <%=tool%> to download newly received packages, much like the official
|
3044
3221
|
cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on Cloud plugin:
|
3045
3222
|
|
3046
|
-
```
|
3223
|
+
```bash
|
3047
3224
|
<%=cmd%> faspex packages recv --id=ALL --once-only=yes --lock-port=12345
|
3048
3225
|
```
|
3049
3226
|
|
3050
|
-
|
3227
|
+
## Plugin: IBM Aspera Shares
|
3051
3228
|
|
3052
3229
|
Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
|
3053
3230
|
|
3054
3231
|
In Shares2, users, groups listing are paged, to display sequential pages:
|
3055
3232
|
|
3056
|
-
```
|
3233
|
+
```bash
|
3057
3234
|
for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p'}';done
|
3058
3235
|
```
|
3059
3236
|
|
3060
|
-
|
3237
|
+
## Plugin: IBM Cloud Object Storage
|
3061
3238
|
|
3062
3239
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
3063
3240
|
It uses the same transfer service as Aspera on Cloud, called Aspera Transfer Service (ATS).
|
@@ -3065,7 +3242,7 @@ Available ATS regions: [https://status.aspera.io](https://status.aspera.io)
|
|
3065
3242
|
|
3066
3243
|
There are two possibilities to provide credentials. If you already have the endpoint, apikey and CRN, use the first method. If you don't have credentials but have access to the IBM Cloud console, then use the second method.
|
3067
3244
|
|
3068
|
-
|
3245
|
+
### Using endpoint, apikey and Resource Instance ID (CRN)
|
3069
3246
|
|
3070
3247
|
If you have those parameters already, then following options shall be provided:
|
3071
3248
|
|
@@ -3076,14 +3253,14 @@ If you have those parameters already, then following options shall be provided:
|
|
3076
3253
|
|
3077
3254
|
For example, let us create a default configuration:
|
3078
3255
|
|
3079
|
-
```
|
3256
|
+
```bash
|
3080
3257
|
<%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
|
3081
3258
|
<%=cmd%> conf id default set cos mycos
|
3082
3259
|
```
|
3083
3260
|
|
3084
3261
|
Then, jump to the transfer example.
|
3085
3262
|
|
3086
|
-
|
3263
|
+
### Using service credential file
|
3087
3264
|
|
3088
3265
|
If you are the COS administrator and don't have yet the credential: Service credentials are directly created using the IBM cloud web ui. Navigate to:
|
3089
3266
|
|
@@ -3093,7 +3270,7 @@ Then save the copied value to a file, e.g. : `$HOME/cos_service_creds.json`
|
|
3093
3270
|
|
3094
3271
|
or using the IBM Cloud CLI:
|
3095
3272
|
|
3096
|
-
```
|
3273
|
+
```bash
|
3097
3274
|
ibmcloud resource service-keys
|
3098
3275
|
ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HOME/service_creds.json
|
3099
3276
|
```
|
@@ -3102,7 +3279,7 @@ ibmcloud resource service-key aoclaurent --output JSON|jq '.[0].credentials'>$HO
|
|
3102
3279
|
|
3103
3280
|
It consists in the following structure:
|
3104
3281
|
|
3105
|
-
```
|
3282
|
+
```javascript
|
3106
3283
|
{
|
3107
3284
|
"apikey": "xxxxxxx.....",
|
3108
3285
|
"cos_hmac_keys": {
|
@@ -3132,40 +3309,40 @@ The required options for this method are:
|
|
3132
3309
|
|
3133
3310
|
For example, let us create a default configuration:
|
3134
3311
|
|
3135
|
-
```
|
3312
|
+
```bash
|
3136
3313
|
<%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
|
3137
3314
|
<%=cmd%> conf id default set cos mycos
|
3138
3315
|
```
|
3139
3316
|
|
3140
|
-
|
3317
|
+
### Operations, transfers
|
3141
3318
|
|
3142
3319
|
Let's assume you created a default configuration from once of the two previous steps (else specify the access options on command lines).
|
3143
3320
|
|
3144
3321
|
A subset of `node` plugin operations are supported, basically node API:
|
3145
3322
|
|
3146
|
-
```
|
3323
|
+
```bash
|
3147
3324
|
<%=cmd%> cos node info
|
3148
3325
|
<%=cmd%> cos node upload 'faux:///sample1G?1g'
|
3149
3326
|
```
|
3150
3327
|
|
3151
3328
|
Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
|
3152
3329
|
|
3153
|
-
|
3330
|
+
## Plugin: IBM Aspera Sync
|
3154
3331
|
|
3155
3332
|
A basic plugin to start an "async" using <%=tool%>.
|
3156
3333
|
The main advantage is the possibility to start from ma configuration file, using <%=tool%> standard options.
|
3157
3334
|
|
3158
|
-
|
3335
|
+
## Plugin: Preview
|
3159
3336
|
|
3160
3337
|
The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
3161
3338
|
This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
|
3162
3339
|
Several parameters can be used to tune several aspects:
|
3163
3340
|
|
3164
|
-
|
3165
|
-
|
3166
|
-
|
3341
|
+
* methods for detection of new files needing generation
|
3342
|
+
* methods for generation of video preview
|
3343
|
+
* parameters for video handling
|
3167
3344
|
|
3168
|
-
|
3345
|
+
### Aspera Server configuration
|
3169
3346
|
|
3170
3347
|
Specify the previews folder as shown in:
|
3171
3348
|
|
@@ -3173,7 +3350,7 @@ Specify the previews folder as shown in:
|
|
3173
3350
|
|
3174
3351
|
By default, the `preview` plugin expects previews to be generated in a folder named `previews` located in the storage root. On the transfer server execute:
|
3175
3352
|
|
3176
|
-
```
|
3353
|
+
```bash
|
3177
3354
|
PATH=/opt/aspera/bin:$PATH
|
3178
3355
|
|
3179
3356
|
asconfigurator -x "server;preview_dir,previews"
|
@@ -3189,7 +3366,7 @@ This size is internally capped to `1<<24` Bytes (16777216) , i.e. 16384 KBytes.
|
|
3189
3366
|
|
3190
3367
|
To change this parameter in `aspera.conf`, use `asconfigurator`. To display the value, use `asuserdata`:
|
3191
3368
|
|
3192
|
-
```
|
3369
|
+
```bash
|
3193
3370
|
asuserdata -a | grep max_request_file_create_size_kb
|
3194
3371
|
|
3195
3372
|
max_request_file_create_size_kb: "1024"
|
@@ -3201,7 +3378,7 @@ If you use a value different than 16777216, then specify it using option `max_si
|
|
3201
3378
|
|
3202
3379
|
Note: the HSTS parameter (max_request_file_create_size_kb) is in *kiloBytes* while the generator parameter is in *Bytes* (factor of 1024).
|
3203
3380
|
|
3204
|
-
|
3381
|
+
### <a id="prev_ext"></a>External tools: Linux
|
3205
3382
|
|
3206
3383
|
The tool requires the following external tools available in the `PATH`:
|
3207
3384
|
|
@@ -3217,21 +3394,21 @@ Other OSes should work as well, but are note tested.
|
|
3217
3394
|
|
3218
3395
|
To check if all tools are found properly, execute:
|
3219
3396
|
|
3220
|
-
```
|
3397
|
+
```bash
|
3221
3398
|
<%=cmd%> preview check
|
3222
3399
|
```
|
3223
3400
|
|
3224
|
-
|
3401
|
+
#### mimemagic
|
3225
3402
|
|
3226
3403
|
To benefit from extra mime type detection install gem mimemagic:
|
3227
3404
|
|
3228
|
-
```
|
3405
|
+
```bash
|
3229
3406
|
gem install mimemagic
|
3230
3407
|
```
|
3231
3408
|
|
3232
3409
|
or to install an earlier version if any problem:
|
3233
3410
|
|
3234
|
-
```
|
3411
|
+
```bash
|
3235
3412
|
gem install mimemagic -v '~> 0.3.0'
|
3236
3413
|
```
|
3237
3414
|
|
@@ -3241,21 +3418,21 @@ If not used, Mime type used for conversion is the one provided by the node API.
|
|
3241
3418
|
|
3242
3419
|
If used, it the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
|
3243
3420
|
|
3244
|
-
|
3421
|
+
#### Image: ImageMagick and optipng
|
3245
3422
|
|
3246
|
-
```
|
3423
|
+
```bash
|
3247
3424
|
yum install -y ImageMagick optipng
|
3248
3425
|
```
|
3249
3426
|
|
3250
|
-
|
3427
|
+
#### Video: FFmpeg
|
3251
3428
|
|
3252
3429
|
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
3253
3430
|
|
3254
|
-
```
|
3431
|
+
```bash
|
3255
3432
|
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
|
3256
3433
|
```
|
3257
3434
|
|
3258
|
-
|
3435
|
+
#### Office: Unoconv and Libreoffice
|
3259
3436
|
|
3260
3437
|
If you don't want to have preview for office documents or if it is too complex you can skip office document preview generation by using option: `--skip-types=office`
|
3261
3438
|
|
@@ -3263,13 +3440,13 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
|
|
3263
3440
|
|
3264
3441
|
* CentOS 8
|
3265
3442
|
|
3266
|
-
```
|
3443
|
+
```bash
|
3267
3444
|
dnf install unoconv
|
3268
3445
|
```
|
3269
3446
|
|
3270
3447
|
* Amazon Linux
|
3271
3448
|
|
3272
|
-
```
|
3449
|
+
```bash
|
3273
3450
|
amazon-linux-extras enable libreoffice
|
3274
3451
|
yum clean metadata
|
3275
3452
|
yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
|
@@ -3278,7 +3455,7 @@ mv unoconv /usr/bin
|
|
3278
3455
|
chmod a+x /usr/bin/unoconv
|
3279
3456
|
```
|
3280
3457
|
|
3281
|
-
|
3458
|
+
### Configuration
|
3282
3459
|
|
3283
3460
|
The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
3284
3461
|
|
@@ -3286,7 +3463,7 @@ Like any <%=tool%> commands, parameters can be passed on command line or using a
|
|
3286
3463
|
|
3287
3464
|
Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
|
3288
3465
|
|
3289
|
-
```
|
3466
|
+
```bash
|
3290
3467
|
su -s /bin/bash - xfer
|
3291
3468
|
|
3292
3469
|
<%=cmd%> config preset update previewconf --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
|
@@ -3299,13 +3476,13 @@ Here we assume that Office file generation is disabled, else remove this option.
|
|
3299
3476
|
|
3300
3477
|
One can check if the access key is well configured using:
|
3301
3478
|
|
3302
|
-
```
|
3479
|
+
```bash
|
3303
3480
|
<%=cmd%> -Ppreviewconf node browse /
|
3304
3481
|
```
|
3305
3482
|
|
3306
3483
|
This shall list the contents of the storage root of the access key.
|
3307
3484
|
|
3308
|
-
|
3485
|
+
### Execution
|
3309
3486
|
|
3310
3487
|
The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
3311
3488
|
It needs to be run on a regular basis to create or update preview files. For that use your best
|
@@ -3315,7 +3492,7 @@ Typically, for "Access key" access, the system/transfer is `xfer`. So, in order
|
|
3315
3492
|
|
3316
3493
|
Lets do a one shot test, using the configuration previously created:
|
3317
3494
|
|
3318
|
-
```
|
3495
|
+
```bash
|
3319
3496
|
su -s /bin/bash - xfer
|
3320
3497
|
|
3321
3498
|
<%=cmd%> preview scan --overwrite=always
|
@@ -3325,7 +3502,7 @@ When the preview generator is first executed it will create a file: `.aspera_acc
|
|
3325
3502
|
in the previews folder which contains the access key used.
|
3326
3503
|
On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
|
3327
3504
|
|
3328
|
-
|
3505
|
+
### Configuration for Execution in scheduler
|
3329
3506
|
|
3330
3507
|
Here is an example of configuration for use with cron on Linux.
|
3331
3508
|
Adapt the scripts to your own preference.
|
@@ -3336,7 +3513,7 @@ Lets first setup a script that will be used in the scheduler and sets up the env
|
|
3336
3513
|
|
3337
3514
|
Example of startup script `cron_<%=cmd%>`, which sets the Ruby environment and adds some timeout protection:
|
3338
3515
|
|
3339
|
-
```
|
3516
|
+
```bash
|
3340
3517
|
#!/bin/bash
|
3341
3518
|
# set a timeout protection, just in case
|
3342
3519
|
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
@@ -3347,7 +3524,7 @@ exec timeout ${tmout} <%=cmd%> "${@}"
|
|
3347
3524
|
|
3348
3525
|
Here the cronjob is created for user `xfer`.
|
3349
3526
|
|
3350
|
-
```
|
3527
|
+
```bash
|
3351
3528
|
crontab<<EOF
|
3352
3529
|
0 * * * * /home/xfer/cron_<%=cmd%> preview scan --logger=syslog --display=error
|
3353
3530
|
2-59 * * * * /home/xfer/cron_<%=cmd%> preview trev --logger=syslog --display=error
|
@@ -3356,7 +3533,7 @@ EOF
|
|
3356
3533
|
|
3357
3534
|
Note that the logging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
|
3358
3535
|
|
3359
|
-
|
3536
|
+
### Candidate detection for creation or update (or deletion)
|
3360
3537
|
|
3361
3538
|
The tool generates preview files using those commands:
|
3362
3539
|
|
@@ -3378,7 +3555,7 @@ Deletion of preview for deleted source files: not implemented yet (TODO).
|
|
3378
3555
|
|
3379
3556
|
If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
|
3380
3557
|
|
3381
|
-
```
|
3558
|
+
```bash
|
3382
3559
|
<%=cmd%> preview scan --skip-folders=@json:'["/not_here"]'
|
3383
3560
|
```
|
3384
3561
|
|
@@ -3388,20 +3565,20 @@ When scanning the option `value` has the same behavior as for the `node find` co
|
|
3388
3565
|
|
3389
3566
|
For instance to filter out files beginning with `._` do:
|
3390
3567
|
|
3391
|
-
```
|
3568
|
+
```bash
|
3392
3569
|
... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
|
3393
3570
|
```
|
3394
3571
|
|
3395
|
-
|
3572
|
+
### Preview File types
|
3396
3573
|
|
3397
3574
|
Two types of preview can be generated:
|
3398
3575
|
|
3399
|
-
|
3400
|
-
|
3576
|
+
* png: thumbnail
|
3577
|
+
* mp4: video preview (only for video)
|
3401
3578
|
|
3402
3579
|
Use option `skip_format` to skip generation of a format.
|
3403
3580
|
|
3404
|
-
|
3581
|
+
### Supported input Files types
|
3405
3582
|
|
3406
3583
|
The preview generator supports rendering of those file categories:
|
3407
3584
|
|
@@ -3421,7 +3598,7 @@ File type is primarily based on file extension detected by the node API and tran
|
|
3421
3598
|
|
3422
3599
|
The tool can also locally detect the mime type using gem `mimemagic`.
|
3423
3600
|
|
3424
|
-
|
3601
|
+
### Access to original files and preview creation
|
3425
3602
|
|
3426
3603
|
Standard open source tools are used to create thumbnails and video previews.
|
3427
3604
|
Those tools require that original files are accessible in the local file system and also write generated files on the local file system.
|
@@ -3432,8 +3609,7 @@ are directly written to the storage.
|
|
3432
3609
|
|
3433
3610
|
If the preview generator does not have access to files on the file system (it is remote, no mount, or is an object storage), then the original file is first downloaded, then the result is uploaded, use method `remote`.
|
3434
3611
|
|
3435
|
-
|
3436
|
-
# SMTP for email notifications
|
3612
|
+
## SMTP for email notifications
|
3437
3613
|
|
3438
3614
|
Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
|
3439
3615
|
|
@@ -3450,9 +3626,9 @@ The `smtp` option is a hash table (extended value) with the following fields:
|
|
3450
3626
|
<tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
|
3451
3627
|
</table>
|
3452
3628
|
|
3453
|
-
|
3629
|
+
### Example of configuration:
|
3454
3630
|
|
3455
|
-
```
|
3631
|
+
```bash
|
3456
3632
|
<%=cmd%> config preset set smtp_google server smtp.google.com
|
3457
3633
|
<%=cmd%> config preset set smtp_google username john@gmail.com
|
3458
3634
|
<%=cmd%> config preset set smtp_google password P@ssw0rd
|
@@ -3460,24 +3636,24 @@ The `smtp` option is a hash table (extended value) with the following fields:
|
|
3460
3636
|
|
3461
3637
|
or
|
3462
3638
|
|
3463
|
-
```
|
3639
|
+
```javascript
|
3464
3640
|
<%=cmd%> config preset init smtp_google @json:'{"server":"smtp.google.com","username":"john@gmail.com","password":"P@ssw0rd"}'
|
3465
3641
|
```
|
3466
3642
|
|
3467
3643
|
or
|
3468
3644
|
|
3469
|
-
```
|
3645
|
+
```bash
|
3470
3646
|
<%=cmd%> config preset update smtp_google --server=smtp.google.com --username=john@gmail.com --password=P@ssw0rd
|
3471
3647
|
```
|
3472
3648
|
|
3473
3649
|
Set this configuration as global default, for instance:
|
3474
3650
|
|
3475
|
-
```
|
3651
|
+
```bash
|
3476
3652
|
<%=cmd%> config preset set cli_default smtp @val:@preset:smtp_google
|
3477
3653
|
<%=cmd%> config preset set default config cli_default
|
3478
3654
|
```
|
3479
3655
|
|
3480
|
-
|
3656
|
+
### Email templates
|
3481
3657
|
|
3482
3658
|
Sent emails are built using a template that uses the [ERB](https://www.tutorialspoint.com/ruby/eruby.htm) syntax.
|
3483
3659
|
|
@@ -3491,16 +3667,16 @@ The following variables are defined by default:
|
|
3491
3667
|
|
3492
3668
|
Other variables are defined depending on context.
|
3493
3669
|
|
3494
|
-
|
3670
|
+
### Test
|
3495
3671
|
|
3496
3672
|
Check settings with `smtp_settings` command. Send test email with `email_test`.
|
3497
3673
|
|
3498
|
-
```
|
3674
|
+
```bash
|
3499
3675
|
<%=cmd%> config --smtp=@preset:smtp_google smtp
|
3500
3676
|
<%=cmd%> config --smtp=@preset:smtp_google email --notif-to=sample.dest@example.com
|
3501
3677
|
```
|
3502
3678
|
|
3503
|
-
|
3679
|
+
### Notifications for transfer status
|
3504
3680
|
|
3505
3681
|
An e-mail notification can be sent upon transfer success and failure (one email per transfer job, one job being possibly multi session, and possibly after retry).
|
3506
3682
|
|
@@ -3517,7 +3693,7 @@ The environment provided contains the following additional variables:
|
|
3517
3693
|
|
3518
3694
|
Example of template:
|
3519
3695
|
|
3520
|
-
```
|
3696
|
+
```text
|
3521
3697
|
From: <%='<'%>%=from_name%> <<%='<'%>%=from_email%>>
|
3522
3698
|
To: <<%='<'%>%=to%>>
|
3523
3699
|
Subject: <%='<'%>%=subject%>
|
@@ -3525,7 +3701,7 @@ Subject: <%='<'%>%=subject%>
|
|
3525
3701
|
Transfer is: <%='<'%>%=global_transfer_status%>
|
3526
3702
|
```
|
3527
3703
|
|
3528
|
-
|
3704
|
+
## Tool: `asession`
|
3529
3705
|
|
3530
3706
|
This gem comes with a second executable tool providing a simplified standardized interface
|
3531
3707
|
to start a FASP session: `asession`.
|
@@ -3550,12 +3726,12 @@ During execution, it generates all low level events, one per line, in JSON forma
|
|
3550
3726
|
|
3551
3727
|
Note that there are special "extended" [_transfer-spec_](#transferspec) parameters supported by `asession`:
|
3552
3728
|
|
3553
|
-
|
3554
|
-
|
3729
|
+
* `EX_loglevel` to change log level of the tool
|
3730
|
+
* `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
3555
3731
|
|
3556
3732
|
Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#agt_direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
|
3557
3733
|
|
3558
|
-
|
3734
|
+
### Comparison of interfaces
|
3559
3735
|
|
3560
3736
|
<table>
|
3561
3737
|
<tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
|
@@ -3565,21 +3741,21 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
|
|
3565
3741
|
<tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
|
3566
3742
|
<tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp</td><td>any with ascp</td><td>any with ascp and transferdaemon</td></tr></table>
|
3567
3743
|
|
3568
|
-
|
3744
|
+
### Simple session
|
3569
3745
|
|
3570
|
-
```
|
3746
|
+
```javascript
|
3571
3747
|
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
|
3572
3748
|
|
3573
3749
|
echo "${MY_TSPEC}"|asession
|
3574
3750
|
```
|
3575
3751
|
|
3576
|
-
|
3752
|
+
### Asynchronous commands and Persistent session
|
3577
3753
|
|
3578
3754
|
`asession` also supports asynchronous commands (on the management port). Instead of the traditional text protocol as described in ascp manual, the format for commands is: one single line per command, formatted in JSON, where parameters shall be "snake" style, for example: `LongParameter` -> `long_parameter`
|
3579
3755
|
|
3580
3756
|
This is particularly useful for a persistent session ( with the [_transfer-spec_](#transferspec) parameter: `"keepalive":true` )
|
3581
3757
|
|
3582
|
-
```
|
3758
|
+
```javascript
|
3583
3759
|
asession
|
3584
3760
|
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
|
3585
3761
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
@@ -3588,20 +3764,20 @@ asession
|
|
3588
3764
|
|
3589
3765
|
(events from FASP are not shown in above example. They would appear after each command)
|
3590
3766
|
|
3591
|
-
|
3767
|
+
### Example of language wrapper
|
3592
3768
|
|
3593
3769
|
Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/aspera)
|
3594
3770
|
|
3595
|
-
|
3771
|
+
### Help
|
3596
3772
|
|
3597
|
-
```
|
3773
|
+
```bash
|
3598
3774
|
asession -h
|
3599
3775
|
<%=File.read(ENV["INCL_ASESSION"])%>
|
3600
3776
|
```
|
3601
3777
|
|
3602
|
-
|
3778
|
+
## Hot folder
|
3603
3779
|
|
3604
|
-
|
3780
|
+
### Requirements
|
3605
3781
|
|
3606
3782
|
<%=tool%> maybe used as a simple hot folder engine. A hot folder being defined as a tool that:
|
3607
3783
|
|
@@ -3613,7 +3789,7 @@ asession -h
|
|
3613
3789
|
|
3614
3790
|
In addition: the detection should be made "continuously" or on specific time/date.
|
3615
3791
|
|
3616
|
-
|
3792
|
+
### Setup procedure
|
3617
3793
|
|
3618
3794
|
The general idea is to rely on :
|
3619
3795
|
|
@@ -3621,7 +3797,7 @@ The general idea is to rely on :
|
|
3621
3797
|
* take advantage of <%=tool%> configuration capabilities and server side knowledge
|
3622
3798
|
* the OS scheduler for reliability and continuous operation
|
3623
3799
|
|
3624
|
-
|
3800
|
+
#### ascp features
|
3625
3801
|
|
3626
3802
|
Interesting ascp features are found in its arguments: (see ascp manual):
|
3627
3803
|
|
@@ -3636,31 +3812,33 @@ Note that:
|
|
3636
3812
|
* most, but not all native ascp arguments are available as standard transfer_spec parameters
|
3637
3813
|
* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#agt_direct) transfer agent (not connect or node)
|
3638
3814
|
|
3639
|
-
|
3815
|
+
#### server side and configuration
|
3640
3816
|
|
3641
3817
|
Virtually any transfer on a "repository" on a regular basis might emulate a hot folder. Note that file detection is not based on events (inotify, etc...), but on a stateless scan on source side.
|
3642
3818
|
|
3643
3819
|
Note: parameters may be saved in a <%=prst%> and used with `-P`.
|
3644
3820
|
|
3645
|
-
|
3821
|
+
#### Scheduling
|
3646
3822
|
|
3647
3823
|
Once <%=tool%> parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [_Scheduling_](#_scheduling_).
|
3648
3824
|
|
3649
|
-
|
3825
|
+
### Example: upload folder
|
3650
3826
|
|
3651
|
-
```
|
3827
|
+
```javascript
|
3652
3828
|
<%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}'
|
3653
|
-
|
3654
3829
|
```
|
3655
3830
|
|
3656
3831
|
The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they don't grow anymore (based on an 8-second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
3657
3832
|
|
3658
|
-
|
3833
|
+
## Health check and Nagios
|
3659
3834
|
|
3660
3835
|
Most plugin provide a `health` command that will check the health status of the application. Example:
|
3661
3836
|
|
3662
|
-
```
|
3837
|
+
```bash
|
3663
3838
|
<%=cmd%> console health
|
3839
|
+
```
|
3840
|
+
|
3841
|
+
```output
|
3664
3842
|
+--------+-------------+------------+
|
3665
3843
|
| status | component | message |
|
3666
3844
|
+--------+-------------+------------+
|
@@ -3676,14 +3854,23 @@ Typically, the health check uses the REST API of the application with the follow
|
|
3676
3854
|
|
3677
3855
|
<%=tool%> can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
|
3678
3856
|
|
3679
|
-
```
|
3857
|
+
```bash
|
3680
3858
|
<%=cmd%> server health transfer --to-folder=/Upload --format=nagios --progress=none
|
3859
|
+
```
|
3860
|
+
|
3861
|
+
```output
|
3681
3862
|
OK - [transfer:ok]
|
3863
|
+
```
|
3864
|
+
|
3865
|
+
```bash
|
3682
3866
|
<%=cmd%> server health asctlstatus --cmd_prefix='sudo ' --format=nagios
|
3867
|
+
```
|
3868
|
+
|
3869
|
+
```output
|
3683
3870
|
OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
|
3684
3871
|
```
|
3685
3872
|
|
3686
|
-
|
3873
|
+
## Ruby Module: `Aspera`
|
3687
3874
|
|
3688
3875
|
Main components:
|
3689
3876
|
|
@@ -3693,18 +3880,20 @@ Main components:
|
|
3693
3880
|
|
3694
3881
|
A working example can be found in the gem, example:
|
3695
3882
|
|
3696
|
-
```
|
3883
|
+
```bash
|
3697
3884
|
<%=cmd%> config gem_path
|
3885
|
+
```
|
3886
|
+
|
3887
|
+
```bash
|
3698
3888
|
cat $(<%=cmd%> config gem_path)/../examples/transfer.rb
|
3699
3889
|
```
|
3700
3890
|
|
3701
|
-
This sample code shows some example of use of the API as well as
|
3702
|
-
REST API.
|
3891
|
+
This sample code shows some example of use of the API as well as REST API.
|
3703
3892
|
Note: although nice, it's probably a good idea to use RestClient for REST.
|
3704
3893
|
|
3705
3894
|
Example of use of the API of Aspera on Cloud:
|
3706
3895
|
|
3707
|
-
```
|
3896
|
+
```ruby
|
3708
3897
|
require 'aspera/aoc'
|
3709
3898
|
|
3710
3899
|
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/<%=cmd%>/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
@@ -3712,7 +3901,9 @@ aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user
|
|
3712
3901
|
aoc.read('self')
|
3713
3902
|
```
|
3714
3903
|
|
3715
|
-
|
3904
|
+
<https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
|
3905
|
+
|
3906
|
+
## History
|
3716
3907
|
|
3717
3908
|
When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
|
3718
3909
|
|
@@ -3728,213 +3919,228 @@ So, it evolved into <%=tool%>:
|
|
3728
3919
|
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
|
3729
3920
|
* `ruby` is consistent with other Aspera products
|
3730
3921
|
|
3731
|
-
|
3922
|
+
## Changes (Release notes)
|
3732
3923
|
|
3733
3924
|
* <%=gemspec.version.to_s%>
|
3734
3925
|
|
3735
|
-
|
3736
|
-
|
3737
|
-
|
3738
|
-
|
3739
|
-
|
3740
|
-
|
3741
|
-
|
3742
|
-
|
3743
|
-
|
3744
|
-
|
3745
|
-
|
3746
|
-
|
3926
|
+
* new: command `conf plugin create`
|
3927
|
+
* new: global option `plugin_folder`
|
3928
|
+
* new: global option `transpose_single`
|
3929
|
+
* new: simplified metadata passing for shared inbox package creation in AoC
|
3930
|
+
* change: (break) command `aoc packages shared_inboxes list` replaces `aoc user shared_inboxes`
|
3931
|
+
* change: (break) command `aoc user profile` replaces `aoc user info`
|
3932
|
+
* change: (break) command `aoc user workspaces list` replaces `aoc user workspaces`
|
3933
|
+
* change: (break) command `aoc user workspaces current` replaces `aoc workspace`
|
3934
|
+
* change: (break) command `conf plugin list` replaces `conf plugins`
|
3935
|
+
* change: (break) command `conf connect` simplified
|
3936
|
+
* fix: #60 ascli executable was not installed by default in 4.5.0
|
3937
|
+
* fix: add password hiding case in logs
|
3938
|
+
|
3939
|
+
* 4.5.0
|
3940
|
+
|
3941
|
+
* new: support transfer agent: [Transfer SDK](#agt_trsdk)
|
3942
|
+
* new: support [http socket options](#http_options)
|
3943
|
+
* new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
|
3944
|
+
* new: `config vault` supports encrypted passwords, also macos keychain
|
3945
|
+
* new: `config preset` command for consistency with id
|
3946
|
+
* new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
|
3947
|
+
* change: when using wss, use [ruby's CA certs](#certificates)
|
3948
|
+
* change: unexpected parameter makes exit code not zero
|
3949
|
+
* change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
|
3950
|
+
* change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
|
3951
|
+
* change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
|
3952
|
+
* fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
|
3747
3953
|
|
3748
3954
|
* 4.4.0
|
3749
3955
|
|
3750
|
-
|
3751
|
-
|
3752
|
-
|
3753
|
-
|
3754
|
-
|
3755
|
-
|
3756
|
-
|
3956
|
+
* new: `aoc packages list` add possibility to add filter with option `query`
|
3957
|
+
* new: `aoc admin res xxx list` now get all items by default #50
|
3958
|
+
* new: `preset` option can specify name or hash value
|
3959
|
+
* new: `node` plugin accepts bearer token and access key as credential
|
3960
|
+
* new: `node` option `token_type` allows using basic token in addition to aspera type.
|
3961
|
+
* change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
|
3962
|
+
* change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
|
3757
3963
|
|
3758
3964
|
* 4.3.0
|
3759
3965
|
|
3760
|
-
|
3761
|
-
|
3966
|
+
* new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
|
3967
|
+
* new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
|
3762
3968
|
|
3763
3969
|
* 4.2.2
|
3764
3970
|
|
3765
|
-
|
3766
|
-
|
3767
|
-
|
3768
|
-
|
3769
|
-
|
3770
|
-
|
3771
|
-
|
3772
|
-
|
3773
|
-
|
3971
|
+
* new: `faspex package list` retrieves the whole list, not just first page
|
3972
|
+
* new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
|
3973
|
+
* new: the error "Remote host is not who we expected" displays a special remediation message
|
3974
|
+
* new: `conf ascp spec` displays supported transfer spec
|
3975
|
+
* new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
|
3976
|
+
* fix: space character in `faspe:` url are precent encoded if needed
|
3977
|
+
* fix: `preview scan`: if file_id is unknown, ignore and continue scan
|
3978
|
+
* change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then <%=tool%> exits with code 1 (instead of zero=success)
|
3979
|
+
* change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
|
3774
3980
|
|
3775
3981
|
* 4.2.1
|
3776
3982
|
|
3777
|
-
|
3778
|
-
|
3983
|
+
* new: command `faspex package recv` supports link of type: `faspe:`
|
3984
|
+
* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
|
3779
3985
|
|
3780
3986
|
* 4.2.0
|
3781
3987
|
|
3782
|
-
|
3783
|
-
|
3784
|
-
|
3785
|
-
|
3786
|
-
|
3787
|
-
|
3788
|
-
|
3789
|
-
|
3790
|
-
|
3791
|
-
|
3988
|
+
* new: command `aoc remind` to receive organization membership by email
|
3989
|
+
* new: in `preview` option `value` to filter out on file name
|
3990
|
+
* new: `initdemo` to initialize for demo server
|
3991
|
+
* new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
|
3992
|
+
* fix: on Windows `conf ascp use` expects ascp.exe
|
3993
|
+
* fix: (break) multi_session_threshold is Integer, not String
|
3994
|
+
* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
|
3995
|
+
* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
|
3996
|
+
* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
|
3997
|
+
* change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
|
3792
3998
|
|
3793
3999
|
* 4.1.0
|
3794
4000
|
|
3795
|
-
|
3796
|
-
|
3797
|
-
|
3798
|
-
|
3799
|
-
|
3800
|
-
|
3801
|
-
|
3802
|
-
|
3803
|
-
|
3804
|
-
|
3805
|
-
|
3806
|
-
|
4001
|
+
* fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
|
4002
|
+
* new: update documentation with regard to offline and docker installation
|
4003
|
+
* new: renamed command `nagios_check` to `health`
|
4004
|
+
* new: agent `http_gw` now supports upload
|
4005
|
+
* new: added option `sdk_url` to install SDK from local file for offline install
|
4006
|
+
* new: check new gem version periodically
|
4007
|
+
* new: the --fields= option, support -_fieldname_ to remove a field from default fields
|
4008
|
+
* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
|
4009
|
+
* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
|
4010
|
+
* new: AoC a password can be provided for a public link
|
4011
|
+
* new: `conf doc` take an optional parameter to go to a section
|
4012
|
+
* new: initial support for Faspex 5 Beta 1
|
3807
4013
|
|
3808
4014
|
* 4.0.0
|
3809
4015
|
|
3810
|
-
|
3811
|
-
|
3812
|
-
|
3813
|
-
|
3814
|
-
|
3815
|
-
|
3816
|
-
|
3817
|
-
|
4016
|
+
* now available as open source at [<%=gemspec.homepage%>](<%=gemspec.homepage%>) with general cleanup
|
4017
|
+
* changed default tool name from `mlia` to `ascli`
|
4018
|
+
* changed `aspera` command to `aoc`
|
4019
|
+
* changed gem name from `asperalm` to `aspera-cli`
|
4020
|
+
* changed module name from `Asperalm` to `Aspera`
|
4021
|
+
* removed command `folder` in `preview`, merged to `scan`
|
4022
|
+
* persistency files go to sub folder instead of main folder
|
4023
|
+
* added possibility to install SDK: `config ascp install`
|
3818
4024
|
|
3819
4025
|
* 0.11.8
|
3820
4026
|
|
3821
|
-
|
4027
|
+
* Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
|
3822
4028
|
|
3823
4029
|
* 0.11.7
|
3824
4030
|
|
3825
|
-
|
3826
|
-
|
3827
|
-
|
3828
|
-
|
3829
|
-
|
3830
|
-
|
4031
|
+
* rework on rest call error handling
|
4032
|
+
* use option `display` with value `data` to remove out of extraneous information
|
4033
|
+
* fixed option `lock_port` not working
|
4034
|
+
* generate special icon if preview failed
|
4035
|
+
* possibility to choose transfer progress bar type with option `progress`
|
4036
|
+
* AoC package creation now output package id
|
3831
4037
|
|
3832
4038
|
* 0.11.6
|
3833
4039
|
|
3834
|
-
|
3835
|
-
|
3836
|
-
|
3837
|
-
|
3838
|
-
|
3839
|
-
|
3840
|
-
|
4040
|
+
* orchestrator : added more choice in auth type
|
4041
|
+
* preview: cleanup in generator (removed and renamed parameters)
|
4042
|
+
* preview: better documentation
|
4043
|
+
* preview: animated thumbnails for video (option: `video_png_conv=animated`)
|
4044
|
+
* preview: new event trigger: `trevents` (`events` seems broken)
|
4045
|
+
* preview: unique tmp folder to avoid clash of multiple instances
|
4046
|
+
* repo: added template for secrets used for testing
|
3841
4047
|
|
3842
4048
|
* 0.11.5
|
3843
4049
|
|
3844
|
-
|
3845
|
-
|
3846
|
-
|
3847
|
-
|
3848
|
-
|
3849
|
-
|
4050
|
+
* added option `default_ports` for AoC (see manual)
|
4051
|
+
* allow bulk delete in `aspera files` with option `bulk=yes`
|
4052
|
+
* fix getting connect versions
|
4053
|
+
* added section for Aix
|
4054
|
+
* support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
|
4055
|
+
* added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
|
3850
4056
|
|
3851
4057
|
* 0.11.4
|
3852
4058
|
|
3853
|
-
|
4059
|
+
* possibility to give shared inbox name when sending a package (else use id and type)
|
3854
4060
|
|
3855
4061
|
* 0.11.3
|
3856
4062
|
|
3857
|
-
|
4063
|
+
* minor fixes on multi-session: avoid exception on progress bar
|
3858
4064
|
|
3859
4065
|
* 0.11.2
|
3860
4066
|
|
3861
|
-
|
4067
|
+
* fixes on multi-session: progress bat and transfer spec param for "direct"
|
3862
4068
|
|
3863
4069
|
* 0.11.1
|
3864
4070
|
|
3865
|
-
|
4071
|
+
* enhanced short_link creation commands (see examples)
|
3866
4072
|
|
3867
4073
|
* 0.11
|
3868
4074
|
|
3869
|
-
|
4075
|
+
* add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
|
3870
4076
|
|
3871
4077
|
* 0.10.18
|
3872
4078
|
|
3873
|
-
|
4079
|
+
* new option in. `server` : `ssh_options`
|
3874
4080
|
|
3875
4081
|
* 0.10.17
|
3876
4082
|
|
3877
|
-
|
3878
|
-
|
4083
|
+
* fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
|
4084
|
+
* new modifier: `@list:<separator>val1<separator>...`
|
3879
4085
|
|
3880
4086
|
* 0.10.16
|
3881
4087
|
|
3882
|
-
|
4088
|
+
* added list of shared inboxes in workspace (or global), use `--query=@json:'{}'`
|
3883
4089
|
|
3884
4090
|
* 0.10.15
|
3885
4091
|
|
3886
|
-
|
3887
|
-
|
4092
|
+
* in case of command line error, display the error cause first, and non-parsed argument second
|
4093
|
+
* AoC : Activity / Analytics
|
3888
4094
|
|
3889
4095
|
* 0.10.14
|
3890
4096
|
|
3891
|
-
|
4097
|
+
* added missing bss plugin
|
3892
4098
|
|
3893
4099
|
* 0.10.13
|
3894
4100
|
|
3895
|
-
|
4101
|
+
* added Faspex5 (use option `value` to give API arguments)
|
3896
4102
|
|
3897
4103
|
* 0.10.12
|
3898
4104
|
|
3899
|
-
|
3900
|
-
|
3901
|
-
|
3902
|
-
|
3903
|
-
|
3904
|
-
|
4105
|
+
* added support for AoC node registration keys
|
4106
|
+
* replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
|
4107
|
+
* Transfer agent is no more a Singleton instance, but only one is used in CLI
|
4108
|
+
* `@incps` : new extended value modifier
|
4109
|
+
* ATS: no more provides access keys secrets: now user must provide it
|
4110
|
+
* begin work on "aoc" transfer agent
|
3905
4111
|
|
3906
4112
|
* 0.10.11
|
3907
4113
|
|
3908
|
-
|
4114
|
+
* minor refactor and fixes
|
3909
4115
|
|
3910
4116
|
* 0.10.10
|
3911
4117
|
|
3912
|
-
|
4118
|
+
* fix on documentation
|
3913
4119
|
|
3914
4120
|
* 0.10.9.1
|
3915
4121
|
|
3916
|
-
|
3917
|
-
|
3918
|
-
|
4122
|
+
* add total number of items for AoC resource list
|
4123
|
+
* better gem version dependency (and fixes to support Ruby 2.0.0)
|
4124
|
+
* removed aoc search_nodes
|
3919
4125
|
|
3920
4126
|
* 0.10.8
|
3921
4127
|
|
3922
|
-
|
3923
|
-
|
3924
|
-
|
4128
|
+
* removed option: `fasp_proxy`, use pseudo transfer spec parameter: `EX_fasp_proxy_url`
|
4129
|
+
* removed option: `http_proxy`, use pseudo transfer spec parameter: `EX_http_proxy_url`
|
4130
|
+
* several other changes..
|
3925
4131
|
|
3926
4132
|
* 0.10.7
|
3927
4133
|
|
3928
|
-
|
4134
|
+
* fix: <%=cmd%> fails when username cannot be computed on Linux.
|
3929
4135
|
|
3930
4136
|
* 0.10.6
|
3931
4137
|
|
3932
|
-
|
3933
|
-
|
4138
|
+
* FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
|
4139
|
+
* gem version requirements made more open
|
3934
4140
|
|
3935
4141
|
* 0.10.5
|
3936
4142
|
|
3937
|
-
|
4143
|
+
* fix faspex package receive command not working
|
3938
4144
|
|
3939
4145
|
* 0.10.4
|
3940
4146
|
|
@@ -3943,220 +4149,218 @@ So, it evolved into <%=tool%>:
|
|
3943
4149
|
|
3944
4150
|
* 0.10.3
|
3945
4151
|
|
3946
|
-
|
4152
|
+
* included user name in oauth bearer token cache for AoC when JWT is used.
|
3947
4153
|
|
3948
4154
|
* 0.10.2
|
3949
4155
|
|
3950
|
-
|
3951
|
-
|
3952
|
-
|
4156
|
+
* updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
|
4157
|
+
* added doc for "cargo" like actions
|
4158
|
+
* added doc for multi-session
|
3953
4159
|
|
3954
4160
|
* 0.10.1
|
3955
4161
|
|
3956
|
-
|
3957
|
-
|
4162
|
+
* AoC and node v4 "browse" works now on non-folder items: file, link
|
4163
|
+
* initial support for AoC automation (do not use yet)
|
3958
4164
|
|
3959
4165
|
* 0.10
|
3960
4166
|
|
3961
|
-
|
3962
|
-
|
4167
|
+
* support for transfer using IBM Cloud Object Storage
|
4168
|
+
* improved `find` action using arbitrary expressions
|
3963
4169
|
|
3964
4170
|
* 0.9.36
|
3965
4171
|
|
3966
|
-
|
4172
|
+
* added option to specify file pair lists
|
3967
4173
|
|
3968
4174
|
* 0.9.35
|
3969
4175
|
|
3970
|
-
|
3971
|
-
|
4176
|
+
* updated plugin `preview` , changed parameter names, added documentation
|
4177
|
+
* fix in `ats` plugin : instance id needed in request header
|
3972
4178
|
|
3973
4179
|
* 0.9.34
|
3974
4180
|
|
3975
|
-
|
3976
|
-
|
4181
|
+
* parser "@preset" can be used again in option "transfer_info"
|
4182
|
+
* some documentation re-organizing
|
3977
4183
|
|
3978
4184
|
* 0.9.33
|
3979
4185
|
|
3980
|
-
|
3981
|
-
|
3982
|
-
|
3983
|
-
|
4186
|
+
* new command to display basic token of node
|
4187
|
+
* new command to display bearer token of node in AoC
|
4188
|
+
* the --fields= option, support +_fieldname_ to add a field to default fields
|
4189
|
+
* many small changes
|
3984
4190
|
|
3985
4191
|
* 0.9.32
|
3986
4192
|
|
3987
|
-
|
3988
|
-
|
3989
|
-
|
4193
|
+
* all Faspex public links are now supported
|
4194
|
+
* removed faspex operation recv_publink
|
4195
|
+
* replaced with option `link` (consistent with AoC)
|
3990
4196
|
|
3991
4197
|
* 0.9.31
|
3992
4198
|
|
3993
|
-
|
3994
|
-
|
3995
|
-
|
4199
|
+
* added more support for public link: receive and send package, to user or dropbox and files view.
|
4200
|
+
* delete expired file lists
|
4201
|
+
* changed text table gem from text-table to terminal-table because it supports multiline values
|
3996
4202
|
|
3997
4203
|
* 0.9.27
|
3998
4204
|
|
3999
|
-
|
4000
|
-
|
4205
|
+
* basic email support with SMTP
|
4206
|
+
* basic proxy auto config support
|
4001
4207
|
|
4002
4208
|
* 0.9.26
|
4003
4209
|
|
4004
|
-
|
4005
|
-
|
4210
|
+
* table display with --fields=ALL now includes all column names from all lines, not only first one
|
4211
|
+
* unprocessed argument shows error even if there is an error beforehand
|
4006
4212
|
|
4007
4213
|
* 0.9.25
|
4008
4214
|
|
4009
|
-
|
4010
|
-
|
4011
|
-
|
4215
|
+
* the option `value` of command `find`, to filter on name, is not optional
|
4216
|
+
* `find` now also reports all types (file, folder, link)
|
4217
|
+
* `find` now is able to report all fields (type, size, etc...)
|
4012
4218
|
|
4013
4219
|
* 0.9.24
|
4014
4220
|
|
4015
|
-
|
4016
|
-
|
4221
|
+
* fix bug where AoC node to node transfer did not work
|
4222
|
+
* fix bug on error if ED25519 private key is defined in .ssh
|
4017
4223
|
|
4018
4224
|
* 0.9.23
|
4019
4225
|
|
4020
|
-
|
4021
|
-
|
4226
|
+
* defined REST error handlers, more error conditions detected
|
4227
|
+
* commands to select specific ascp location
|
4022
4228
|
|
4023
4229
|
* 0.9.21
|
4024
4230
|
|
4025
|
-
|
4026
|
-
|
4231
|
+
* supports simplified wizard using global client
|
4232
|
+
* only ascp binary is required, other SDK (keys) files are now generated
|
4027
4233
|
|
4028
4234
|
* 0.9.20
|
4029
4235
|
|
4030
|
-
|
4031
|
-
|
4032
|
-
|
4236
|
+
* improved wizard (prepare for AoC global client id)
|
4237
|
+
* preview generator: addedoption : --skip-format=<png,mp4>
|
4238
|
+
* removed outdated pictures from this doc
|
4033
4239
|
|
4034
4240
|
* 0.9.19
|
4035
4241
|
|
4036
|
-
|
4242
|
+
* added command aspera bearer --scope=xx
|
4037
4243
|
|
4038
4244
|
* 0.9.18
|
4039
4245
|
|
4040
|
-
|
4246
|
+
* enhanced aspera admin events to support query
|
4041
4247
|
|
4042
4248
|
* 0.9.16
|
4043
4249
|
|
4044
|
-
|
4045
|
-
|
4250
|
+
* AoC transfers are now reported in activity app
|
4251
|
+
* new interface for Rest class authentication (keep backward compatibility)
|
4046
4252
|
|
4047
4253
|
* 0.9.15
|
4048
4254
|
|
4049
|
-
|
4050
|
-
|
4255
|
+
* new feature: "find" command in aspera files
|
4256
|
+
* sample code for transfer API
|
4051
4257
|
|
4052
4258
|
* 0.9.12
|
4053
4259
|
|
4054
|
-
|
4055
|
-
|
4260
|
+
* add nagios commands
|
4261
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
4056
4262
|
|
4057
4263
|
* 0.9.11
|
4058
4264
|
|
4059
|
-
|
4060
|
-
|
4265
|
+
* Breaking change: @stdin is now @stdin:
|
4266
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
4061
4267
|
|
4062
4268
|
|
4063
4269
|
* 0.9.10
|
4064
4270
|
|
4065
|
-
|
4066
|
-
|
4067
|
-
|
4068
|
-
|
4271
|
+
* Breaking change: parameter transfer-node becomes more generic: transfer-info
|
4272
|
+
* Display SaaS storage usage with command: aspera admin res node --id=nn info
|
4273
|
+
* cleaner way of specifying source file list for transfers
|
4274
|
+
* Breaking change: replaced download_mode option with http_download action
|
4069
4275
|
|
4070
4276
|
* 0.9.9
|
4071
4277
|
|
4072
|
-
|
4073
|
-
|
4278
|
+
* Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
|
4279
|
+
* Now supports "cargo" for Aspera on Cloud (automatic package download)
|
4074
4280
|
|
4075
4281
|
* 0.9.8
|
4076
4282
|
|
4077
|
-
|
4078
|
-
|
4283
|
+
* Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
|
4284
|
+
* AoC: share to share transfer with command "transfer"
|
4079
4285
|
|
4080
4286
|
* 0.9.7
|
4081
4287
|
|
4082
|
-
|
4083
|
-
|
4084
|
-
|
4085
|
-
|
4086
|
-
|
4288
|
+
* homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
|
4289
|
+
* preview persistency goes to unique file by default
|
4290
|
+
* catch mxf extension in preview as video
|
4291
|
+
* Faspex: possibility to download all packages by specifying id=ALL
|
4292
|
+
* Faspex: to come: cargo-like function to download only new packages with id=NEW
|
4087
4293
|
|
4088
4294
|
* 0.9.6
|
4089
4295
|
|
4090
|
-
|
4091
|
-
|
4296
|
+
* Breaking change: `@param:`is now `@preset:` and is generic
|
4297
|
+
* AoC: added command to display current workspace information
|
4092
4298
|
|
4093
4299
|
* 0.9.5
|
4094
4300
|
|
4095
|
-
|
4096
|
-
|
4301
|
+
* new parameter: new_user_option used to choose between public_link and invite of external users.
|
4302
|
+
* fixed bug in wizard, and wizard uses now product detection
|
4097
4303
|
|
4098
4304
|
* 0.9.4
|
4099
4305
|
|
4100
|
-
|
4101
|
-
|
4102
|
-
|
4306
|
+
* Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
|
4307
|
+
* AoC Package send supports external users
|
4308
|
+
* new command to export AoC config to Aspera CLI config
|
4103
4309
|
|
4104
4310
|
* 0.9.3
|
4105
4311
|
|
4106
|
-
|
4107
|
-
|
4108
|
-
|
4109
|
-
|
4110
|
-
|
4111
|
-
|
4312
|
+
* REST error message show host and code
|
4313
|
+
* option for quiet display
|
4314
|
+
* modified transfer interface and allow token re-generation on error
|
4315
|
+
* async add admin command
|
4316
|
+
* async add db parameters
|
4317
|
+
* Breaking change: new option "sources" to specify files to transfer
|
4112
4318
|
|
4113
4319
|
* 0.9.2
|
4114
4320
|
|
4115
|
-
|
4321
|
+
* Breaking change: changed AoC package creation to match API, see AoC section
|
4116
4322
|
|
4117
4323
|
* 0.9.1
|
4118
4324
|
|
4119
|
-
|
4325
|
+
* Breaking change: changed faspex package creation to match API, see Faspex section
|
4120
4326
|
|
4121
4327
|
* 0.9
|
4122
4328
|
|
4123
|
-
|
4124
|
-
|
4329
|
+
* Renamed the CLI from aslmcli to <%=tool%>
|
4330
|
+
* Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
|
4125
4331
|
|
4126
4332
|
* 0.7.6
|
4127
4333
|
|
4128
|
-
|
4334
|
+
* add "sync" plugin
|
4129
4335
|
|
4130
4336
|
* 0.7
|
4131
4337
|
|
4132
|
-
|
4133
|
-
|
4134
|
-
|
4135
|
-
|
4136
|
-
|
4338
|
+
* Breaking change: AoC package recv take option if for package instead of argument.
|
4339
|
+
* Breaking change: Rest class and Oauth class changed init parameters
|
4340
|
+
* AoC: receive package from public link
|
4341
|
+
* select by col value on output
|
4342
|
+
* added rename (AoC, node)
|
4137
4343
|
|
4138
4344
|
* 0.6.19
|
4139
4345
|
|
4140
|
-
|
4141
|
-
|
4142
|
-
|
4143
|
-
|
4144
|
-
|
4145
|
-
|
4146
|
-
|
4147
|
-
|
4148
|
-
* ats api_key list → ats credential list
|
4149
|
-
* ats access_key id xxx → ats access_key --id=xxx
|
4346
|
+
* change: (break) ats server list provisioned → ats cluster list
|
4347
|
+
* change: (break) ats server list clouds → ats cluster clouds
|
4348
|
+
* change: (break) ats server list instance --cloud=x --region=y → ats cluster show --cloud=x --region=y
|
4349
|
+
* change: (break) ats server id xxx → ats cluster show --id=xxx
|
4350
|
+
* change: (break) ats subscriptions → ats credential subscriptions
|
4351
|
+
* change: (break) ats api_key repository list → ats credential cache list
|
4352
|
+
* change: (break) ats api_key list → ats credential list
|
4353
|
+
* change: (break) ats access_key id xxx → ats access_key --id=xxx
|
4150
4354
|
|
4151
4355
|
* 0.6.18
|
4152
4356
|
|
4153
|
-
|
4357
|
+
* some commands take now --id option instead of id command.
|
4154
4358
|
|
4155
4359
|
* 0.6.15
|
4156
4360
|
|
4157
|
-
|
4361
|
+
* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
|
4158
4362
|
|
4159
|
-
|
4363
|
+
## BUGS, FEATURES, CONTRIBUTION
|
4160
4364
|
|
4161
4365
|
For issues or feature requests use the Github repository and issues.
|
4162
4366
|
|
@@ -4164,7 +4368,7 @@ You can also contribute to this open source project.
|
|
4164
4368
|
|
4165
4369
|
One can also [create one's own plugin](#createownplugin).
|
4166
4370
|
|
4167
|
-
|
4371
|
+
### Only one value for any option
|
4168
4372
|
|
4169
4373
|
Some commands and sub commands may ask for the same option name.
|
4170
4374
|
Currently, since option definition is position independent (last one wins), it is not possible
|
@@ -4175,27 +4379,26 @@ As a solution, use the position specific notation for selection, i.e. provide th
|
|
4175
4379
|
|
4176
4380
|
This happens typically for the `node` sub command, e.g. identify the node by name instead of id.
|
4177
4381
|
|
4178
|
-
|
4179
|
-
## ED255519 key not supported
|
4382
|
+
### ED255519 key not supported
|
4180
4383
|
|
4181
4384
|
ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
|
4182
4385
|
|
4183
4386
|
Without this deactivation, if such key was present the following error was generated:
|
4184
4387
|
|
4185
|
-
```
|
4388
|
+
```output
|
4186
4389
|
OpenSSH keys only supported if ED25519 is available
|
4187
4390
|
```
|
4188
4391
|
|
4189
4392
|
Which meant that you do not have ruby support for ED25519 SSH keys.
|
4190
4393
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
4191
4394
|
|
4192
|
-
|
4395
|
+
### Error "Remote host is not who we expected"
|
4193
4396
|
|
4194
4397
|
Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
|
4195
4398
|
|
4196
4399
|
Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
|
4197
4400
|
|
4198
|
-
```
|
4401
|
+
```bash
|
4199
4402
|
--ts=@json:'{"sshfp":null}'
|
4200
4403
|
```
|
4201
4404
|
|
@@ -4203,12 +4406,12 @@ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or
|
|
4203
4406
|
|
4204
4407
|
References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
|
4205
4408
|
|
4206
|
-
|
4409
|
+
### Miscellaneous
|
4207
4410
|
|
4208
4411
|
* remove rest and oauth classes and use ruby standard gems:
|
4209
4412
|
|
4210
|
-
|
4211
|
-
|
4413
|
+
* oauth
|
4414
|
+
* <https://github.com/rest-client/rest-client>
|
4212
4415
|
|
4213
4416
|
* use Thor or any standard Ruby CLI manager
|
4214
4417
|
|
@@ -4218,4 +4421,4 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
4218
4421
|
|
4219
4422
|
* Going through proxy: use env var http_proxy and https_proxy, no_proxy
|
4220
4423
|
|
4221
|
-
* easier use with https://github.com/pmq20/ruby-packer
|
4424
|
+
* easier use with <https://github.com/pmq20/ruby-packer>
|