aspera-cli 4.0.0 → 4.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +518 -137
- data/bin/dascli +13 -0
- data/docs/README.erb.md +473 -105
- data/docs/test_env.conf +4 -1
- data/docs/transfer_spec.html +1 -1
- data/lib/aspera/aoc.rb +68 -86
- data/lib/aspera/cli/formater.rb +2 -0
- data/lib/aspera/cli/main.rb +27 -19
- data/lib/aspera/cli/plugin.rb +9 -4
- data/lib/aspera/cli/plugins/alee.rb +1 -1
- data/lib/aspera/cli/plugins/aoc.rb +173 -136
- data/lib/aspera/cli/plugins/config.rb +80 -27
- data/lib/aspera/cli/plugins/console.rb +2 -2
- data/lib/aspera/cli/plugins/faspex.rb +13 -6
- data/lib/aspera/cli/plugins/faspex5.rb +93 -37
- data/lib/aspera/cli/plugins/node.rb +3 -3
- data/lib/aspera/cli/plugins/preview.rb +25 -24
- data/lib/aspera/cli/plugins/server.rb +23 -8
- data/lib/aspera/cli/transfer_agent.rb +1 -1
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/fasp/connect.rb +28 -21
- data/lib/aspera/fasp/http_gw.rb +140 -28
- data/lib/aspera/fasp/installation.rb +28 -4
- data/lib/aspera/fasp/local.rb +24 -16
- data/lib/aspera/fasp/manager.rb +12 -0
- data/lib/aspera/fasp/node.rb +4 -4
- data/lib/aspera/fasp/parameters.rb +3 -16
- data/lib/aspera/log.rb +1 -1
- data/lib/aspera/node.rb +48 -1
- data/lib/aspera/oauth.rb +24 -11
- data/lib/aspera/persistency_folder.rb +9 -4
- data/lib/aspera/preview/file_types.rb +53 -21
- data/lib/aspera/preview/generator.rb +3 -3
- data/lib/aspera/rest.rb +38 -18
- data/lib/aspera/temp_file_manager.rb +19 -0
- metadata +37 -20
data/bin/dascli
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
#!/usr/bin/env bash
|
2
|
+
tag=martinlaurent/ascli:latest
|
3
|
+
case "$1" in install)
|
4
|
+
docker pull $tag
|
5
|
+
exit 0
|
6
|
+
esac
|
7
|
+
conf_host=${ASCLI_HOME:-$HOME/.aspera/ascli}
|
8
|
+
if test ! -d $conf_host;then
|
9
|
+
echo creating foder: $conf_host
|
10
|
+
mkdir -p $conf_host
|
11
|
+
fi
|
12
|
+
conf_img=/usr/src/app/config
|
13
|
+
exec docker run --tty --interactive --rm --env ASCLI_HOME="$conf_img" --volume "$conf_host:$conf_img" $tag ascli "$@"
|
data/docs/README.erb.md
CHANGED
@@ -15,27 +15,78 @@ prstt=opprst.capitalize # in title
|
|
15
15
|
gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
|
16
16
|
geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
|
17
17
|
-%>
|
18
|
-
# <%=tool%> :
|
18
|
+
# <%=tool%> : Command Line Interface for IBM Aspera products
|
19
19
|
|
20
20
|
Version : <%= gemspec.version.to_s %>
|
21
21
|
|
22
22
|
_Laurent/2016-<%=Time.new.year%>_
|
23
23
|
|
24
|
-
This gem provides a command line interface to Aspera Applications.
|
24
|
+
This gem provides <%=tool%>: a command line interface to Aspera Applications.
|
25
25
|
|
26
|
-
|
27
|
-
[<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
|
26
|
+
<%=tool%> is a also great tool to learn Aspera APIs.
|
28
27
|
|
29
|
-
|
28
|
+
Ruby Gem: [<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
|
30
29
|
|
31
|
-
|
30
|
+
Ruby Doc: [<%= gemspec.metadata['documentation_uri'] %>](<%= gemspec.metadata['documentation_uri'] %>)
|
32
31
|
|
33
|
-
|
32
|
+
Ruby version must be >= <%= gemspec.required_ruby_version %>
|
33
|
+
|
34
|
+
# <a name="when_to_use"></a>When to use and when not to use
|
35
|
+
|
36
|
+
<%=tool%> is designed to be used as a command line tool to:
|
37
|
+
|
38
|
+
* execute commands on Aspera products
|
39
|
+
* transfer to/from Aspera products
|
40
|
+
|
41
|
+
So it is designed for:
|
42
|
+
|
43
|
+
* Interactive operations on a text terminal (typically, VT100 compatible)
|
44
|
+
* Batch operations in (shell) scripts (e.g. cron job)
|
45
|
+
|
46
|
+
<%=tool%> can be seen as a command line tool integrating:
|
47
|
+
|
48
|
+
* a configuration file (config.yaml) and advanced command line options
|
49
|
+
* cURL (for REST calls)
|
50
|
+
* Aspera transfer (ascp)
|
51
|
+
|
52
|
+
One might be tempted to use it as an integration element, e.g. by building a command line programmatically, and then executing it. It is generally not a good idea.
|
53
|
+
For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
|
54
|
+
|
55
|
+
* Product APIs (REST) : e.g. AoC, Faspex, node
|
56
|
+
* Transfer SDK : with gRPC interface and laguage stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
|
57
|
+
|
58
|
+
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
59
|
+
|
60
|
+
For scripting and ad'hoc command line operations, <%=tool%> is perfect.
|
61
|
+
|
62
|
+
# Notations
|
34
63
|
|
35
64
|
In examples, command line operations (starting with `$`) are shown using a standard shell: `bash` or `zsh`.
|
65
|
+
Prompt `# ` refers to user `root`, prompt `xfer$ ` refer to user `xfer`.
|
36
66
|
|
37
67
|
Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
|
38
68
|
|
69
|
+
# <a name="parsing"></a>Shell and Command line parsing
|
70
|
+
|
71
|
+
<%=tool%> is typically executed in a shell, either interactively or in a script. <%=tool%> receives its arguments from this shell.
|
72
|
+
|
73
|
+
On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before <%=tool%> (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to <%=tool%>. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
|
74
|
+
|
75
|
+
On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
|
76
|
+
|
77
|
+
* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
|
78
|
+
* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
79
|
+
|
80
|
+
In case of doubt of argument values after parsing test like this:
|
81
|
+
|
82
|
+
```
|
83
|
+
$ <%=cmd%> conf echo "Hello World" arg2 3
|
84
|
+
"Hello World"
|
85
|
+
ERROR: Argument: unprocessed values: ["arg2", "3"]
|
86
|
+
```
|
87
|
+
|
88
|
+
`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (ascli), next command line arguments are shown in the error message.
|
89
|
+
|
39
90
|
# Quick Start
|
40
91
|
|
41
92
|
This section guides you from installation, first use and advanced use.
|
@@ -111,77 +162,154 @@ Then, follow the section relative to the product you want to interact with ( Asp
|
|
111
162
|
|
112
163
|
# <a name="installation"></a>Installation
|
113
164
|
|
114
|
-
|
165
|
+
It is possible to install *either* as a docker container or step by step on the native host:
|
115
166
|
|
116
|
-
* [Ruby](#ruby)
|
167
|
+
* [Ruby](#ruby) version >= <%= gemspec.required_ruby_version %>
|
117
168
|
* [<%= gemspec.name %>](#the_gem)
|
118
|
-
* [
|
169
|
+
* [Aspera SDK (ascp)](#fasp_prot)
|
119
170
|
|
120
171
|
The following sections provide information on the installation.
|
121
172
|
|
173
|
+
An internet connection is required for the installation. If you dont have internet for the installation, refer to section [Installation without internet access](#offline_install).
|
174
|
+
|
175
|
+
## Docker container
|
176
|
+
|
177
|
+
This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
|
178
|
+
|
179
|
+
Ensure that you have Docker installed.
|
180
|
+
|
181
|
+
```
|
182
|
+
$ docker --version
|
183
|
+
```
|
184
|
+
|
185
|
+
Download the wrapping script:
|
186
|
+
|
187
|
+
```
|
188
|
+
$ curl -o <%=cmd%> https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
|
189
|
+
$ chmod a+x <%=cmd%>
|
190
|
+
```
|
191
|
+
|
192
|
+
Install the container image:
|
193
|
+
|
194
|
+
```
|
195
|
+
$ ./<%=cmd%> install
|
196
|
+
```
|
197
|
+
|
198
|
+
Start using it !
|
199
|
+
|
200
|
+
Note that the tool is run in the container.
|
201
|
+
|
202
|
+
The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/<%=cmd%>`.
|
203
|
+
|
204
|
+
To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
|
205
|
+
|
122
206
|
## <a name="ruby"></a>Ruby
|
123
207
|
|
124
|
-
|
208
|
+
Use this method to install on the native host.
|
125
209
|
|
126
|
-
|
210
|
+
A ruby interpreter is required to run the tool or to use the gem and tool.
|
127
211
|
|
128
|
-
Ruby 3 is also supported.
|
212
|
+
Ruby minimum version: <%= gemspec.required_ruby_version %>. Ruby version 3 is also supported.
|
129
213
|
|
130
|
-
Any type of Ruby installation can be used
|
214
|
+
*Any type of Ruby installation can be used* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
131
215
|
|
132
216
|
Refer to the following sections for a proposed method for specific operating systems.
|
133
217
|
|
134
|
-
|
218
|
+
The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
|
219
|
+
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
220
|
+
If you have a simpler better way to install Ruby version >= <%= gemspec.required_ruby_version %> : use it !
|
135
221
|
|
136
|
-
|
222
|
+
### Generic: RVM: single user installation (not root)
|
223
|
+
|
224
|
+
Use this method which provides more flexibility.
|
225
|
+
|
226
|
+
Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
|
227
|
+
|
228
|
+
Install the 2 keys
|
137
229
|
|
138
230
|
```
|
139
|
-
$
|
231
|
+
$ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
|
140
232
|
```
|
141
233
|
|
142
|
-
|
234
|
+
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
143
235
|
|
144
236
|
```
|
145
|
-
$
|
237
|
+
$ \curl -sSL https://get.rvm.io | bash -s stable
|
146
238
|
```
|
147
239
|
|
148
|
-
|
240
|
+
If you keep the same terminal (ont needed if re-login):
|
149
241
|
|
150
|
-
|
242
|
+
```
|
243
|
+
$ source ~/.rvm/scripts/rvm
|
244
|
+
```
|
151
245
|
|
152
|
-
|
246
|
+
It is advised to get one of the pre-compiled ruby version, you can list with:
|
153
247
|
|
154
|
-
|
248
|
+
```
|
249
|
+
$ rvm list --remote
|
250
|
+
```
|
155
251
|
|
156
|
-
|
252
|
+
Install the chosen pre-compiled Ruby version:
|
253
|
+
|
254
|
+
```
|
255
|
+
$ rvm install 2.7.2 --binary
|
256
|
+
```
|
157
257
|
|
158
|
-
|
258
|
+
Ruby is now installed for the user, go on to Gem installation.
|
159
259
|
|
160
|
-
###
|
260
|
+
### Generic: RVM: global installation (as root)
|
161
261
|
|
162
|
-
|
163
|
-
|
262
|
+
Follow the same method as single user install, but execute as "root".
|
263
|
+
|
264
|
+
As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
|
265
|
+
One can install in another location with :
|
164
266
|
|
165
267
|
```
|
166
|
-
curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
268
|
+
# curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
167
269
|
```
|
168
270
|
|
169
|
-
|
271
|
+
As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
|
272
|
+
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
273
|
+
To activate ruby (and <%=cmd%>) later, source it:
|
170
274
|
|
171
275
|
```
|
172
|
-
# rvm
|
276
|
+
# source /etc/profile.d/rvm.sh.ok
|
277
|
+
# rvm version
|
173
278
|
```
|
174
279
|
|
175
|
-
|
176
|
-
|
280
|
+
### Windows: Installer
|
281
|
+
|
282
|
+
Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
|
283
|
+
|
284
|
+
* Go to "Downloads".
|
285
|
+
* Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
|
286
|
+
* At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
|
177
287
|
|
178
|
-
|
288
|
+
### macOS: pre-installed or `brew`
|
289
|
+
|
290
|
+
MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%= gemspec.name %> using `sudo` :
|
291
|
+
|
292
|
+
```
|
293
|
+
$ sudo gem install <%= gemspec.name %><%=geminstadd%>
|
294
|
+
```
|
295
|
+
|
296
|
+
Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
|
297
|
+
|
298
|
+
```
|
299
|
+
$ brew install ruby
|
300
|
+
```
|
301
|
+
|
302
|
+
### Linux: package
|
303
|
+
|
304
|
+
If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
|
305
|
+
|
306
|
+
Example:
|
179
307
|
|
180
308
|
```
|
181
309
|
# yum install -y ruby rubygems ruby-json
|
182
310
|
```
|
183
311
|
|
184
|
-
One can cleanup
|
312
|
+
One can cleanup the whole yum-installed ruby environment like this to uninstall:
|
185
313
|
|
186
314
|
```
|
187
315
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
@@ -206,6 +334,50 @@ For instance to build from source, and install in `/opt/ruby` :
|
|
206
334
|
# make install
|
207
335
|
```
|
208
336
|
|
337
|
+
### <a name="offline_install"></a>Installation without internet access
|
338
|
+
|
339
|
+
Note that currently no pre-packaged version exist yet.
|
340
|
+
A method to build one provided here:
|
341
|
+
|
342
|
+
On a server with the same OS version and with internet access follow the "Generic single user installation" method.
|
343
|
+
|
344
|
+
Then create an archive:
|
345
|
+
|
346
|
+
```
|
347
|
+
$ cd
|
348
|
+
$ tar zcvf rvm-<%=cmd%>.tgz .rvm
|
349
|
+
```
|
350
|
+
|
351
|
+
Get the Aspera SDK. Execute:
|
352
|
+
|
353
|
+
```
|
354
|
+
$ <%=cmd%> conf --show-config|grep sdk_url
|
355
|
+
```
|
356
|
+
|
357
|
+
Then download the SDK archive from that URL.
|
358
|
+
|
359
|
+
Another method for the SDK is to install the SDK (`<%=cmd%> conf ascp install`) on the first system, and archive `$HOME/.aspera`.
|
360
|
+
|
361
|
+
Transfer those 2 archives to the target system without internet access.
|
362
|
+
|
363
|
+
On the target system:
|
364
|
+
|
365
|
+
* Extract the RVM archive either in a global location, or in a user's home folder : `path_to_rvm_root`
|
366
|
+
* in the user's `.profile` add this line: (replace `path_to_rvm_root` with the actual location)
|
367
|
+
|
368
|
+
```
|
369
|
+
source path_to_rvm_root/scripts/rvm
|
370
|
+
rvm use 2.7.2
|
371
|
+
```
|
372
|
+
|
373
|
+
For the SDK, either install from archive:
|
374
|
+
|
375
|
+
```
|
376
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
377
|
+
```
|
378
|
+
|
379
|
+
or restore the `$HOME/.aspera` folder for the user.
|
380
|
+
|
209
381
|
## <a name="the_gem"></a>`<%= gemspec.name %>` gem
|
210
382
|
|
211
383
|
Once you have Ruby and rights to install gems: Install the gem and its dependencies:
|
@@ -220,21 +392,40 @@ To upgrade to the latest version:
|
|
220
392
|
# gem update <%= gemspec.name %>
|
221
393
|
```
|
222
394
|
|
395
|
+
<%=tool%> checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
|
396
|
+
|
397
|
+
To check manually:
|
398
|
+
|
399
|
+
```
|
400
|
+
# <%=cmd%> conf check_update
|
401
|
+
```
|
402
|
+
|
403
|
+
|
404
|
+
|
223
405
|
## <a name="fasp_prot"></a>FASP Protocol
|
224
406
|
|
225
|
-
Most file transfers will be done using the FASP protocol
|
226
|
-
an Aspera Transfer:
|
407
|
+
Most file transfers will be done using the FASP protocol, using `ascp`.
|
408
|
+
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
227
409
|
|
228
410
|
* ascp
|
229
411
|
* aspera-license (in same folder, or ../etc)
|
230
412
|
|
231
|
-
This can be installed
|
413
|
+
This can be installed either be installing an Aspera transfer sofware, or using an embedded command:
|
232
414
|
|
233
415
|
```
|
234
416
|
$ <%=cmd%> conf ascp install
|
235
417
|
```
|
236
418
|
|
237
|
-
|
419
|
+
If a local SDK installation is prefered instead of fetching from internet: one can specify the location of the SDK file:
|
420
|
+
|
421
|
+
```
|
422
|
+
$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
423
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
424
|
+
```
|
425
|
+
|
426
|
+
The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
|
427
|
+
|
428
|
+
If the embedded method is not used, the following packages are also suitable:
|
238
429
|
|
239
430
|
* IBM Aspera Connect Client (Free)
|
240
431
|
* IBM Aspera Desktop Client (Free)
|
@@ -243,7 +434,7 @@ Those can be found in one of IBM Aspera transfer server or client with its licen
|
|
243
434
|
* IBM Aspera High Speed Transfer EndPoint (Licensed)
|
244
435
|
|
245
436
|
For instance, Aspera Connect Client can be installed
|
246
|
-
by visiting the page: [
|
437
|
+
by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
|
247
438
|
|
248
439
|
<%=tool%> will detect most of Aspera transfer products in standard locations and use the first one found.
|
249
440
|
Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
|
@@ -251,6 +442,33 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
|
|
251
442
|
Several methods are provided on how to start a transfer. Use of a local client is one of them, but
|
252
443
|
other methods are available. Refer to section: [Transfer Agents](#agents)
|
253
444
|
|
445
|
+
## <a name="offline_install"></a>Offline Installation (without internet)
|
446
|
+
|
447
|
+
The procedure consists in:
|
448
|
+
|
449
|
+
* Follow the non-root installation procedure with RVM, including gem
|
450
|
+
* archive (zip, tar) the main RVM folder (includes <%=cmd%>):
|
451
|
+
|
452
|
+
```
|
453
|
+
$ cd ~
|
454
|
+
$ tar zcvf rvm_<%=cmd%>.tgz .rvm
|
455
|
+
```
|
456
|
+
|
457
|
+
* retrieve the SDK:
|
458
|
+
|
459
|
+
```
|
460
|
+
$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
461
|
+
```
|
462
|
+
|
463
|
+
* on the system without internet access:
|
464
|
+
|
465
|
+
```
|
466
|
+
$ cd ~
|
467
|
+
$ tar zxvf rvm_<%=cmd%>.tgz
|
468
|
+
$ source ~/.rvm/scripts/rvm
|
469
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
470
|
+
```
|
471
|
+
|
254
472
|
# <a name="cli"></a>Command Line Interface: <%=tool%>
|
255
473
|
|
256
474
|
The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
@@ -431,6 +649,7 @@ By default, a table output will display one line per entry, and columns for each
|
|
431
649
|
* a,b,c : the list of attributes specified by the comma separated list
|
432
650
|
* Array extended value: for instance, @json:'["a","b","c"]' same as above
|
433
651
|
* +a,b,c : add selected properties to the default selection.
|
652
|
+
* -a,b,c : remove selected properties from the default selection.
|
434
653
|
|
435
654
|
## <a name="extended"></a>Extended Value Syntax
|
436
655
|
|
@@ -444,25 +663,25 @@ The extended value syntax is:
|
|
444
663
|
|
445
664
|
The difference between reader and decoder is order and ordinality. Both act like a function of value on right hand side. Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
|
446
665
|
|
447
|
-
The following "readers" are supported:
|
666
|
+
The following "readers" are supported (returns value in []):
|
448
667
|
|
449
|
-
* @val:VALUE
|
450
|
-
* @file:PATH
|
451
|
-
* @path:PATH
|
452
|
-
* @env:ENVVAR
|
453
|
-
* @stdin:
|
454
|
-
* @preset:NAME
|
668
|
+
* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
|
669
|
+
* @file:PATH : [String] read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
|
670
|
+
* @path:PATH : [String] performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
|
671
|
+
* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
672
|
+
* @stdin: : [String] read from stdin (no value on right)
|
673
|
+
* @preset:NAME : [Hash] get whole <%=opprst%> value by name
|
455
674
|
|
456
675
|
In addition it is possible to decode a value, using one or multiple decoders :
|
457
676
|
|
458
|
-
* @base64: decode a base64 encoded string
|
459
|
-
* @json: decode JSON values (convenient to provide complex structures)
|
460
|
-
* @zlib: uncompress data
|
461
|
-
* @ruby: execute ruby code
|
462
|
-
* @csvt: decode a titled CSV value
|
463
|
-
* @lines: split a string in multiple lines and return an array
|
464
|
-
* @list: split a string in multiple items taking first character as separator and return an array
|
465
|
-
* @incps: include values of presets specified by key
|
677
|
+
* @base64: [String] decode a base64 encoded string
|
678
|
+
* @json: [any] decode JSON values (convenient to provide complex structures)
|
679
|
+
* @zlib: [String] uncompress data
|
680
|
+
* @ruby: [any] execute ruby code
|
681
|
+
* @csvt: [Array] decode a titled CSV value
|
682
|
+
* @lines: [Array] split a string in multiple lines and return an array
|
683
|
+
* @list: [Array] split a string in multiple items taking first character as separator and return an array
|
684
|
+
* @incps: [Hash] include values of presets specified by key `incps` in input hash
|
466
685
|
|
467
686
|
To display the result of an extended value, use the `config echo` command.
|
468
687
|
|
@@ -494,7 +713,7 @@ $ <%=cmd%> config echo @csvt:@file:test.csv
|
|
494
713
|
:......:.....................:
|
495
714
|
```
|
496
715
|
|
497
|
-
Example: create a hash and include values from preset named "config" of config file
|
716
|
+
Example: create a hash and include values from preset named "config" of config file in this hash
|
498
717
|
|
499
718
|
```
|
500
719
|
$ <%=cmd%> config echo @incps:@json:'{"hello":true,"incps":["config"]}'
|
@@ -533,9 +752,9 @@ It can be overriden using the envinonment variable `<%=evp%>HOME`.
|
|
533
752
|
Example (Windows):
|
534
753
|
|
535
754
|
```
|
536
|
-
$ set <%=evp%>HOME=C:\Users\Kenji\.aspera
|
755
|
+
$ set <%=evp%>HOME=C:\Users\Kenji\.aspera\<%=cmd%>
|
537
756
|
$ <%=cmd%> config folder
|
538
|
-
C:\Users\Kenji\.aspera
|
757
|
+
C:\Users\Kenji\.aspera\<%=cmd%>
|
539
758
|
```
|
540
759
|
|
541
760
|
## <a name="configfile"></a>Configuration file
|
@@ -983,9 +1202,11 @@ If it possible to send using a HTTP gateway, in case FASP is not allowed.
|
|
983
1202
|
Example:
|
984
1203
|
|
985
1204
|
```
|
986
|
-
$ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
1205
|
+
$ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
|
987
1206
|
```
|
988
1207
|
|
1208
|
+
Note that the gateway only supports transfers authorized with a token.
|
1209
|
+
|
989
1210
|
## <a name="transferspec"></a>Transfer Specification
|
990
1211
|
|
991
1212
|
Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
|
@@ -1032,8 +1253,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
|
|
1032
1253
|
|
1033
1254
|
### List of files for transfers
|
1034
1255
|
|
1035
|
-
When uploading, downloading or sending files, the user must specify
|
1036
|
-
|
1256
|
+
When uploading, downloading or sending files, the user must specify the list of files to transfer.
|
1257
|
+
Most of the time, the list of files to transfer will be simply specified on the command line:
|
1037
1258
|
|
1038
1259
|
```
|
1039
1260
|
$ <%=cmd%> server upload ~/mysample.file secondfile
|
@@ -1047,7 +1268,8 @@ $ <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
|
|
1047
1268
|
|
1048
1269
|
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
|
1049
1270
|
|
1050
|
-
Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
|
1271
|
+
Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
|
1272
|
+
all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
|
1051
1273
|
|
1052
1274
|
For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
|
1053
1275
|
|
@@ -1290,7 +1512,7 @@ updated: my_aoc_org
|
|
1290
1512
|
Define this <%=prst%> as default configuration for the `aspera` plugin:
|
1291
1513
|
|
1292
1514
|
```
|
1293
|
-
$ <%=cmd%> config id default set
|
1515
|
+
$ <%=cmd%> config id default set aoc my_aoc_org
|
1294
1516
|
```
|
1295
1517
|
|
1296
1518
|
Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
@@ -1664,7 +1886,7 @@ Then, create two shared folders located in two regions, in your files home, in a
|
|
1664
1886
|
Then, transfer between those:
|
1665
1887
|
|
1666
1888
|
```
|
1667
|
-
$ <%=cmd%> -Paoc_show
|
1889
|
+
$ <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
1668
1890
|
```
|
1669
1891
|
|
1670
1892
|
* create registration key to register a node
|
@@ -2065,23 +2287,67 @@ $ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":
|
|
2065
2287
|
|
2066
2288
|
# Plugin: IBM Aspera Faspex
|
2067
2289
|
|
2068
|
-
|
2290
|
+
Notes:
|
2291
|
+
|
2292
|
+
* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
|
2293
|
+
* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
|
2069
2294
|
|
2070
|
-
##
|
2295
|
+
## Faspex 5 Beta1
|
2296
|
+
|
2297
|
+
As the web UI does not yet allow adding API client yet, the way to use CLI is:
|
2298
|
+
|
2299
|
+
* open a browser
|
2300
|
+
* start developer mode
|
2301
|
+
* login to faspex 5
|
2302
|
+
* find the first API call with `Authorization` token, and copy it (kind of base64 long string)
|
2303
|
+
|
2304
|
+
Use it as password and use `--auth=boot`.
|
2305
|
+
|
2306
|
+
```
|
2307
|
+
$ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
|
2308
|
+
```
|
2309
|
+
|
2310
|
+
An JWT client can then be created with a private key:
|
2071
2311
|
|
2072
|
-
|
2312
|
+
```
|
2313
|
+
$ jsonk=$(openssl rsa -in ~/.aspera/ascli/aspera_on_cloud_key -pubout 2> /dev/null | sed -e :a -e N -e '$!ba' -e 's/\n/\\n/g')
|
2314
|
+
$ <%=cmd%> faspex5 -Pf5boot auth_client create --value=@json:'{"name":"hello","client_type":"public","redirect_uris":["https://localhost:12345"],"allow_jwt_grant":true,"public_key":"'$jsonk'"}'
|
2315
|
+
```
|
2316
|
+
|
2317
|
+
or deleted by name:
|
2073
2318
|
|
2074
2319
|
```
|
2075
|
-
--
|
2320
|
+
$ id=$(<%=cmd%> faspex5 auth_client list --select=@json:'{"name":"hello"}' --fields=client_id --format=csv)
|
2321
|
+
$ <%=cmd%> faspex5 auth_client delete --id=$id
|
2322
|
+
```
|
2323
|
+
|
2324
|
+
Once the API client is created with a client_id and secret (result of create command), create a configuration:
|
2325
|
+
|
2076
2326
|
```
|
2327
|
+
$ <%=cmd%> conf id f5 update --url=https://localhost/aspera/faspex --auth=jwt --client-id=abcd --client-secret=def --username=pierre@example.com --private-key=@val:@file:~/.aspera/ascli/aspera_on_cloud_key
|
2328
|
+
$ <%=cmd%> conf id default set faspex5 f5
|
2329
|
+
```
|
2330
|
+
|
2331
|
+
Ready to use Faspex5 with CLI.
|
2332
|
+
|
2333
|
+
Once the graphical registration form exist, ther bootstrap method can be removed.
|
2334
|
+
|
2335
|
+
## Sending a Package
|
2336
|
+
|
2337
|
+
The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
|
2338
|
+
|
2339
|
+
Example:
|
2077
2340
|
|
2078
|
-
|
2341
|
+
```
|
2342
|
+
$ <%=cmd%> faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
2343
|
+
```
|
2079
2344
|
|
2080
|
-
|
2345
|
+
If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
|
2081
2346
|
|
2347
|
+
Additional optional parameters in `delivery_info`:
|
2082
2348
|
|
2083
|
-
Note
|
2084
|
-
|
2349
|
+
* Package Note: : `"note":"note this and that"`
|
2350
|
+
* Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
2085
2351
|
|
2086
2352
|
## operation on dropboxes
|
2087
2353
|
|
@@ -2144,6 +2410,8 @@ $ for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p
|
|
2144
2410
|
# Plugin: IBM Cloud Object Storage
|
2145
2411
|
|
2146
2412
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
2413
|
+
It uses the same transfer service as Aspera on Cloud.
|
2414
|
+
see [https://status.aspera.io](https://status.aspera.io)
|
2147
2415
|
|
2148
2416
|
Required options are either:
|
2149
2417
|
|
@@ -2200,7 +2468,14 @@ Endpoints for regions can be found by querying the `endpoints` URL.
|
|
2200
2468
|
For convenience, let us create a default configuration, for example:
|
2201
2469
|
|
2202
2470
|
```
|
2203
|
-
$ <%=cmd%> conf id mycos update --service-credentials=@val:@json:@file
|
2471
|
+
$ <%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
|
2472
|
+
$ <%=cmd%> conf id default set cos mycos
|
2473
|
+
```
|
2474
|
+
|
2475
|
+
or using direct parameters:
|
2476
|
+
|
2477
|
+
```
|
2478
|
+
$ <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
|
2204
2479
|
$ <%=cmd%> conf id default set cos mycos
|
2205
2480
|
```
|
2206
2481
|
|
@@ -2266,6 +2541,7 @@ The tool requires the following external tools available in the `PATH`:
|
|
2266
2541
|
* OptiPNG : `optipng`
|
2267
2542
|
* FFmpeg : `ffmpeg` `ffprobe`
|
2268
2543
|
* Libreoffice : `libreoffice`
|
2544
|
+
* ruby gem `mimemagic`
|
2269
2545
|
|
2270
2546
|
Here shown on Redhat/CentOS.
|
2271
2547
|
|
@@ -2277,6 +2553,26 @@ To check if all tools are found properly, execute:
|
|
2277
2553
|
$ <%=cmd%> preview check
|
2278
2554
|
```
|
2279
2555
|
|
2556
|
+
### mimemagic
|
2557
|
+
|
2558
|
+
To benefit from extra mime type detection install gem mimemagic:
|
2559
|
+
|
2560
|
+
```
|
2561
|
+
# gem install mimemagic
|
2562
|
+
```
|
2563
|
+
|
2564
|
+
or to install an earlier version if any problem:
|
2565
|
+
|
2566
|
+
```
|
2567
|
+
# gem install mimemagic -v '~> 0.3.0'
|
2568
|
+
```
|
2569
|
+
|
2570
|
+
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
|
2571
|
+
|
2572
|
+
If not used, Mime type used for conversion is the one provided by the node API.
|
2573
|
+
|
2574
|
+
If used, it the `preview` command will first analyse the file content using mimemagic, and if no match, will try by extension.
|
2575
|
+
|
2280
2576
|
### Image: Imagemagick and optipng
|
2281
2577
|
|
2282
2578
|
```
|
@@ -2301,90 +2597,110 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
|
|
2301
2597
|
# dnf install unoconv
|
2302
2598
|
```
|
2303
2599
|
|
2600
|
+
* Amazon Linux
|
2601
|
+
|
2602
|
+
```
|
2603
|
+
# amazon-linux-extras enable libreoffice
|
2604
|
+
# yum clean metadata
|
2605
|
+
# yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
|
2606
|
+
# wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
|
2607
|
+
# mv unoconv /usr/bin
|
2608
|
+
# chmod a+x /usr/bin/unoconv
|
2609
|
+
```
|
2304
2610
|
|
2305
2611
|
## Configuration
|
2306
2612
|
|
2307
|
-
|
2613
|
+
The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
2614
|
+
|
2615
|
+
Like any <%=tool%> commands, parameters can be passed on command line or using a configuration <%=prst%>. The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
2616
|
+
|
2617
|
+
Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
|
2308
2618
|
|
2309
2619
|
```
|
2310
2620
|
# su -s /bin/bash - xfer
|
2311
|
-
$ <%=cmd%> config id
|
2312
|
-
$ <%=cmd%> config id default set preview
|
2621
|
+
$ <%=cmd%> config id previewconf update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
|
2622
|
+
$ <%=cmd%> config id default set preview previewconf
|
2313
2623
|
```
|
2314
2624
|
|
2315
|
-
Here we assume that Office file generation is disabled, else remove the option.
|
2625
|
+
Here we assume that Office file generation is disabled, else remove the option. `lock_port` prevents concurrent execution of generation when using a scheduler.
|
2316
2626
|
|
2317
2627
|
Once can check if the access key is well configured using:
|
2318
2628
|
|
2319
2629
|
```
|
2320
|
-
$ <%=cmd%> -
|
2630
|
+
$ <%=cmd%> -Ppreviewconf node browse /
|
2321
2631
|
```
|
2322
2632
|
|
2323
2633
|
This shall list the contents of the storage root of the access key.
|
2324
2634
|
|
2325
2635
|
## Execution
|
2326
2636
|
|
2327
|
-
The tool intentionally supports only a "one shot" mode in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
2328
|
-
It needs to be run
|
2637
|
+
The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
2638
|
+
It needs to be run on a regular basis to create or update preview files. For that use your best
|
2329
2639
|
reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
|
2330
2640
|
|
2331
|
-
Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be
|
2332
|
-
should be run as user `xfer`.
|
2641
|
+
Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
|
2333
2642
|
|
2334
2643
|
Lets do a one shot test, using the configuration previously created:
|
2335
2644
|
|
2336
2645
|
```
|
2337
2646
|
# su -s /bin/bash - xfer
|
2338
|
-
$ <%=cmd%> preview scan --overwrite=always
|
2647
|
+
xfer$ <%=cmd%> preview scan --overwrite=always
|
2339
2648
|
```
|
2340
2649
|
|
2341
2650
|
When the preview generator is first executed it will create a file: `.aspera_access_key`
|
2342
|
-
which contains the access key used.
|
2651
|
+
in the previews folder which contains the access key used.
|
2343
2652
|
On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
|
2344
2653
|
|
2345
2654
|
## Configuration for Execution in scheduler
|
2346
2655
|
|
2347
|
-
Here is an example of configuration for use with cron on Linux.
|
2656
|
+
Here is an example of configuration for use with cron on Linux.
|
2657
|
+
Adapt the scripts to your own preference.
|
2348
2658
|
|
2349
2659
|
We assume here that a configuration preset was created as shown previously.
|
2350
2660
|
|
2351
|
-
|
2661
|
+
Lets first setup a script that will be used in the sceduler and sets up the environment.
|
2662
|
+
|
2663
|
+
Example of startup script `cron_<%=cmd%>`, which sets the Ruby environment and adds some timeout protection:
|
2664
|
+
|
2665
|
+
```
|
2666
|
+
#!/bin/bash
|
2667
|
+
# set a timeout protection, just in case
|
2668
|
+
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
2669
|
+
. /etc/profile.d/rvm.sh
|
2670
|
+
rvm use 2.6 --quiet
|
2671
|
+
exec timeout ${tmout} <%=cmd%> "${@}"
|
2672
|
+
```
|
2673
|
+
|
2674
|
+
Here the cronjob is created for user `xfer`.
|
2352
2675
|
|
2353
2676
|
```
|
2354
|
-
|
2355
|
-
|
2356
|
-
|
2677
|
+
xfer$ crontab<<EOF
|
2678
|
+
0 * * * * /home/xfer/cron_<%=cmd%> preview scan --logger=syslog --display=error
|
2679
|
+
2-59 * * * * /home/xfer/cron_<%=cmd%> preview trev --logger=syslog --display=error
|
2357
2680
|
EOF
|
2358
2681
|
```
|
2359
2682
|
|
2360
|
-
|
2683
|
+
Note that the loging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
|
2361
2684
|
|
2362
2685
|
## Candidate detection for creation or update (or deletion)
|
2363
2686
|
|
2364
|
-
The tool
|
2687
|
+
The tool generates preview files using those commands:
|
2365
2688
|
|
2366
2689
|
* `trevents` : only recently uploaded files will be tested (transfer events)
|
2367
2690
|
* `events` : only recently uploaded files will be tested (file events: not working)
|
2368
|
-
* `scan` :
|
2369
|
-
* `
|
2370
|
-
* `file` : for an individual file generation
|
2371
|
-
|
2372
|
-
Note that for the `event`, the option `iteration_file` should be specified so that
|
2373
|
-
successive calls only process new events. This file will hold an identifier
|
2374
|
-
telling from where to get new events.
|
2375
|
-
|
2376
|
-
It is also possible to test a local file, using the `test` command.
|
2691
|
+
* `scan` : recursively scan all files under the access key's "storage root"
|
2692
|
+
* `test` : test using a local file
|
2377
2693
|
|
2378
2694
|
Once candidate are selected, once candidates are selected,
|
2379
2695
|
a preview is always generated if it does not exist already,
|
2380
2696
|
else if a preview already exist, it will be generated
|
2381
|
-
using one of three overwrite
|
2697
|
+
using one of three values for the `overwrite` option:
|
2382
2698
|
|
2383
2699
|
* `always` : preview is always generated, even if it already exists and is newer than original
|
2384
2700
|
* `never` : preview is generated only if it does not exist already
|
2385
2701
|
* `mtime` : preview is generated only if the original file is newer than the existing
|
2386
2702
|
|
2387
|
-
Deletion of preview for deleted source files: not implemented yet.
|
2703
|
+
Deletion of preview for deleted source files: not implemented yet (TODO).
|
2388
2704
|
|
2389
2705
|
If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
|
2390
2706
|
|
@@ -2618,6 +2934,34 @@ $ <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=1
|
|
2618
2934
|
|
2619
2935
|
The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
2620
2936
|
|
2937
|
+
# Aspera Health check and Nagios
|
2938
|
+
|
2939
|
+
Each plugin provide a `health` command that will check the health status of the application. Example:
|
2940
|
+
|
2941
|
+
```
|
2942
|
+
$ <%=cmd%> console health
|
2943
|
+
+--------+-------------+------------+
|
2944
|
+
| status | component | message |
|
2945
|
+
+--------+-------------+------------+
|
2946
|
+
| ok | console api | accessible |
|
2947
|
+
+--------+-------------+------------+
|
2948
|
+
```
|
2949
|
+
|
2950
|
+
Typically, the health check uses the REST API of the application with the following exception: the `server` plugin allows checking health by:
|
2951
|
+
|
2952
|
+
* issuing a transfer to the server
|
2953
|
+
* checking web app status with `asctl all:status`
|
2954
|
+
* checking daemons process status
|
2955
|
+
|
2956
|
+
<%=tool%> can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
|
2957
|
+
|
2958
|
+
```
|
2959
|
+
$ <%=cmd%> server health transfer --to-folder=/Upload --format=nagios --progress=none
|
2960
|
+
OK - [transfer:ok]
|
2961
|
+
$ <%=cmd%> server health asctlstatus --cmd_prefix='sudo ' --format=nagios
|
2962
|
+
OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
|
2963
|
+
```
|
2964
|
+
|
2621
2965
|
# Module: `Aspera`
|
2622
2966
|
|
2623
2967
|
Main components:
|
@@ -2637,6 +2981,16 @@ This sample code shows some example of use of the API as well as
|
|
2637
2981
|
REST API.
|
2638
2982
|
Note: although nice, it's probably a good idea to use RestClient for REST.
|
2639
2983
|
|
2984
|
+
Example of use of the API of Aspera on Cloud:
|
2985
|
+
|
2986
|
+
```
|
2987
|
+
require 'aspera/aoc'
|
2988
|
+
|
2989
|
+
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/<%=cmd%>/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
2990
|
+
|
2991
|
+
aoc.read('self')
|
2992
|
+
```
|
2993
|
+
|
2640
2994
|
# History
|
2641
2995
|
|
2642
2996
|
When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
|
@@ -2653,11 +3007,25 @@ So, it evolved into <%=tool%>:
|
|
2653
3007
|
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
|
2654
3008
|
* `ruby` is consistent with other Aspera products
|
2655
3009
|
|
3010
|
+
# Changes (Release notes)
|
2656
3011
|
|
3012
|
+
* 4.x
|
2657
3013
|
|
2658
|
-
|
3014
|
+
* fix: remove keys from transfer spec and command line when not needed
|
3015
|
+
* fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
|
3016
|
+
* new: update documentation with regard to offline and docker installation
|
3017
|
+
* new: renamed command `nagios_check` to `health`
|
3018
|
+
* new: agent `http_gw` now supports upload
|
3019
|
+
* new: added option `sdk_url` to install SDK from local file for offline install
|
3020
|
+
* new: check new gem version periodically
|
3021
|
+
* new: the --fields= option, support -_fieldname_ to remove a field from default fields
|
3022
|
+
* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
|
3023
|
+
* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
|
3024
|
+
* new: AoC a password can be provided for a public link
|
3025
|
+
* new: `conf doc` take an optional parameter to go to a section
|
3026
|
+
* new: initial support for Faspex 5 Beta 1
|
2659
3027
|
|
2660
|
-
* 4.0.0
|
3028
|
+
* 4.0.0
|
2661
3029
|
|
2662
3030
|
* now available as open source at [<%= gemspec.homepage %>](<%= gemspec.homepage %>) with general cleanup
|
2663
3031
|
* changed default tool name from `mlia` to `ascli`
|
@@ -2785,7 +3153,7 @@ So, it evolved into <%=tool%>:
|
|
2785
3153
|
|
2786
3154
|
* 0.10.6
|
2787
3155
|
|
2788
|
-
* FaspManager: transfer spec `authentication` no more needed for local tranfer to use
|
3156
|
+
* FaspManager: transfer spec `authentication` no more needed for local tranfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
|
2789
3157
|
* gem version requirements made more open
|
2790
3158
|
|
2791
3159
|
* 0.10.5
|