aspera-cli 4.0.0.pre3 → 4.2.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +695 -205
- data/bin/dascli +13 -0
- data/docs/README.erb.md +615 -157
- data/docs/test_env.conf +23 -5
- data/docs/transfer_spec.html +1 -1
- data/examples/aoc.rb +14 -3
- data/examples/faspex4.rb +78 -0
- data/lib/aspera/aoc.rb +87 -108
- data/lib/aspera/cli/formater.rb +2 -0
- data/lib/aspera/cli/main.rb +46 -34
- data/lib/aspera/cli/plugin.rb +9 -4
- data/lib/aspera/cli/plugins/alee.rb +1 -1
- data/lib/aspera/cli/plugins/aoc.rb +207 -182
- data/lib/aspera/cli/plugins/ats.rb +2 -2
- data/lib/aspera/cli/plugins/config.rb +173 -117
- data/lib/aspera/cli/plugins/console.rb +2 -2
- data/lib/aspera/cli/plugins/faspex.rb +51 -36
- data/lib/aspera/cli/plugins/faspex5.rb +82 -41
- data/lib/aspera/cli/plugins/node.rb +3 -3
- data/lib/aspera/cli/plugins/preview.rb +35 -25
- data/lib/aspera/cli/plugins/server.rb +23 -8
- data/lib/aspera/cli/transfer_agent.rb +7 -6
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/cos_node.rb +33 -28
- data/lib/aspera/environment.rb +2 -2
- data/lib/aspera/fasp/connect.rb +28 -21
- data/lib/aspera/fasp/http_gw.rb +140 -28
- data/lib/aspera/fasp/installation.rb +101 -53
- data/lib/aspera/fasp/local.rb +88 -45
- data/lib/aspera/fasp/manager.rb +15 -0
- data/lib/aspera/fasp/node.rb +4 -4
- data/lib/aspera/fasp/parameters.rb +6 -18
- data/lib/aspera/fasp/resume_policy.rb +13 -12
- data/lib/aspera/log.rb +1 -1
- data/lib/aspera/node.rb +61 -1
- data/lib/aspera/oauth.rb +49 -46
- data/lib/aspera/persistency_folder.rb +9 -4
- data/lib/aspera/preview/file_types.rb +53 -21
- data/lib/aspera/preview/generator.rb +3 -3
- data/lib/aspera/rest.rb +29 -18
- data/lib/aspera/secrets.rb +20 -0
- data/lib/aspera/temp_file_manager.rb +19 -0
- data/lib/aspera/web_auth.rb +105 -0
- metadata +42 -22
data/bin/dascli
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
#!/usr/bin/env bash
|
2
|
+
tag=martinlaurent/ascli:latest
|
3
|
+
case "$1" in install)
|
4
|
+
docker pull $tag
|
5
|
+
exit 0
|
6
|
+
esac
|
7
|
+
conf_host=${ASCLI_HOME:-$HOME/.aspera/ascli}
|
8
|
+
if test ! -d $conf_host;then
|
9
|
+
echo creating foder: $conf_host
|
10
|
+
mkdir -p $conf_host
|
11
|
+
fi
|
12
|
+
conf_img=/usr/src/app/config
|
13
|
+
exec docker run --tty --interactive --rm --env ASCLI_HOME="$conf_img" --volume "$conf_host:$conf_img" $tag ascli "$@"
|
data/docs/README.erb.md
CHANGED
@@ -15,32 +15,77 @@ prstt=opprst.capitalize # in title
|
|
15
15
|
gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
|
16
16
|
geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
|
17
17
|
-%>
|
18
|
-
# <%=tool%> :
|
18
|
+
# <%=tool%> : Command Line Interface for IBM Aspera products
|
19
19
|
|
20
20
|
Version : <%= gemspec.version.to_s %>
|
21
21
|
|
22
22
|
_Laurent/2016-<%=Time.new.year%>_
|
23
23
|
|
24
|
-
This gem provides a command line interface to Aspera Applications.
|
24
|
+
This gem provides <%=tool%>: a command line interface to Aspera Applications.
|
25
25
|
|
26
|
-
|
27
|
-
[<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
|
26
|
+
<%=tool%> is a also great tool to learn Aspera APIs.
|
28
27
|
|
29
|
-
|
28
|
+
Ruby Gem: [<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
|
30
29
|
|
31
|
-
|
30
|
+
Ruby Doc: [<%= gemspec.metadata['documentation_uri'] %>](<%= gemspec.metadata['documentation_uri'] %>)
|
32
31
|
|
33
|
-
|
32
|
+
Ruby version must be >= <%= gemspec.required_ruby_version %>
|
34
33
|
|
35
|
-
|
34
|
+
# <a name="when_to_use"></a>When to use and when not to use
|
36
35
|
|
37
|
-
|
38
|
-
|
39
|
-
*
|
36
|
+
<%=tool%> is designed to be used as a command line tool to:
|
37
|
+
|
38
|
+
* execute commands on Aspera products
|
39
|
+
* transfer to/from Aspera products
|
40
|
+
|
41
|
+
So it is designed for:
|
42
|
+
|
43
|
+
* Interactive operations on a text terminal (typically, VT100 compatible)
|
44
|
+
* Batch operations in (shell) scripts (e.g. cron job)
|
45
|
+
|
46
|
+
<%=tool%> can be seen as a command line tool integrating:
|
47
|
+
|
48
|
+
* a configuration file (config.yaml) and advanced command line options
|
49
|
+
* cURL (for REST calls)
|
50
|
+
* Aspera transfer (ascp)
|
51
|
+
|
52
|
+
One might be tempted to use it as an integration element, e.g. by building a command line programmatically, and then executing it. It is generally not a good idea.
|
53
|
+
For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
|
54
|
+
|
55
|
+
* Product APIs (REST) : e.g. AoC, Faspex, node
|
56
|
+
* Transfer SDK : with gRPC interface and laguage stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
|
57
|
+
|
58
|
+
Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
|
59
|
+
|
60
|
+
For scripting and ad'hoc command line operations, <%=tool%> is perfect.
|
61
|
+
|
62
|
+
# Notations
|
40
63
|
|
41
64
|
In examples, command line operations (starting with `$`) are shown using a standard shell: `bash` or `zsh`.
|
65
|
+
Prompt `# ` refers to user `root`, prompt `xfer$ ` refer to user `xfer`.
|
66
|
+
|
67
|
+
Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
|
68
|
+
|
69
|
+
# <a name="parsing"></a>Shell and Command line parsing
|
70
|
+
|
71
|
+
<%=tool%> is typically executed in a shell, either interactively or in a script. <%=tool%> receives its arguments from this shell.
|
72
|
+
|
73
|
+
On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before <%=tool%> (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to <%=tool%>. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
|
74
|
+
|
75
|
+
On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
|
76
|
+
|
77
|
+
* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
|
78
|
+
* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
42
79
|
|
43
|
-
|
80
|
+
In case of doubt of argument values after parsing test like this:
|
81
|
+
|
82
|
+
```
|
83
|
+
$ <%=cmd%> conf echo "Hello World" arg2 3
|
84
|
+
"Hello World"
|
85
|
+
ERROR: Argument: unprocessed values: ["arg2", "3"]
|
86
|
+
```
|
87
|
+
|
88
|
+
`echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (ascli), next command line arguments are shown in the error message.
|
44
89
|
|
45
90
|
# Quick Start
|
46
91
|
|
@@ -61,9 +106,10 @@ Once installation is completed, you can proceed to the first use with a demo ser
|
|
61
106
|
|
62
107
|
If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
|
63
108
|
|
64
|
-
|
109
|
+
To test with Aspera demo transfer server, setup the environment and then test:
|
65
110
|
|
66
111
|
```
|
112
|
+
$ <%=cmd%> config initdemo
|
67
113
|
$ <%=cmd%> server browse /
|
68
114
|
:............:...........:......:........:...........................:.......................:
|
69
115
|
: zmode : zuid : zgid : size : mtime : name :
|
@@ -83,7 +129,7 @@ If you want to use <%=tool%> with another server, and in order to make further c
|
|
83
129
|
* download a file
|
84
130
|
|
85
131
|
```
|
86
|
-
$ <%=cmd%> config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=
|
132
|
+
$ <%=cmd%> config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
|
87
133
|
updated: myserver
|
88
134
|
$ <%=cmd%> config id default set server myserver
|
89
135
|
updated: default→server to myserver
|
@@ -117,79 +163,159 @@ Then, follow the section relative to the product you want to interact with ( Asp
|
|
117
163
|
|
118
164
|
# <a name="installation"></a>Installation
|
119
165
|
|
120
|
-
|
166
|
+
It is possible to install *either* directly on the host operating system (Linux, Windows, Macos) or as a docker container.
|
121
167
|
|
122
|
-
|
168
|
+
The direct installation is recommended and consists in installing:
|
169
|
+
|
170
|
+
* [Ruby](#ruby) version >= <%= gemspec.required_ruby_version %>
|
123
171
|
* [<%= gemspec.name %>](#the_gem)
|
124
|
-
* [
|
172
|
+
* [Aspera SDK (ascp)](#fasp_prot)
|
173
|
+
|
174
|
+
The following sections provide information on the various installation methods.
|
175
|
+
|
176
|
+
An internet connection is required for the installation. If you dont have internet for the installation, refer to section [Installation without internet access](#offline_install).
|
177
|
+
|
178
|
+
## Docker container
|
125
179
|
|
126
|
-
|
180
|
+
Use this method only if you know what you do, else use the standard recommended method as described here above.
|
181
|
+
|
182
|
+
This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
|
183
|
+
|
184
|
+
Ensure that you have Docker installed.
|
185
|
+
|
186
|
+
```
|
187
|
+
$ docker --version
|
188
|
+
```
|
189
|
+
|
190
|
+
Download the wrapping script:
|
191
|
+
|
192
|
+
```
|
193
|
+
$ curl -o <%=cmd%> https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
|
194
|
+
$ chmod a+x <%=cmd%>
|
195
|
+
```
|
196
|
+
|
197
|
+
Install the container image:
|
198
|
+
|
199
|
+
```
|
200
|
+
$ ./<%=cmd%> install
|
201
|
+
```
|
202
|
+
|
203
|
+
Start using it !
|
204
|
+
|
205
|
+
Note that the tool is run in the container, so transfers are also executed in the container, not calling host.
|
206
|
+
|
207
|
+
The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/<%=cmd%>` on host.
|
208
|
+
|
209
|
+
To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
|
210
|
+
To add local storage as a volume edit the script: ascli and add a `--volume` stanza.
|
127
211
|
|
128
212
|
## <a name="ruby"></a>Ruby
|
129
213
|
|
214
|
+
Use this method to install on the native host.
|
215
|
+
|
130
216
|
A ruby interpreter is required to run the tool or to use the gem and tool.
|
131
|
-
|
132
|
-
|
133
|
-
|
217
|
+
|
218
|
+
Ruby minimum version: <%= gemspec.required_ruby_version %>. Ruby version 3 is also supported.
|
219
|
+
|
220
|
+
*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
134
221
|
|
135
222
|
Refer to the following sections for a proposed method for specific operating systems.
|
136
223
|
|
137
|
-
|
224
|
+
The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
|
225
|
+
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
226
|
+
If you have a simpler better way to install Ruby version >= <%= gemspec.required_ruby_version %> : use it !
|
227
|
+
|
228
|
+
### Generic: RVM: single user installation (not root)
|
229
|
+
|
230
|
+
Use this method which provides more flexibility.
|
138
231
|
|
232
|
+
Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
|
139
233
|
|
140
|
-
|
234
|
+
Install the 2 keys
|
141
235
|
|
142
236
|
```
|
143
|
-
$
|
237
|
+
$ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
|
144
238
|
```
|
145
239
|
|
146
|
-
|
240
|
+
Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
|
147
241
|
|
148
242
|
```
|
149
|
-
$
|
243
|
+
$ \curl -sSL https://get.rvm.io | bash -s stable
|
150
244
|
```
|
151
245
|
|
152
|
-
|
246
|
+
If you keep the same terminal (ont needed if re-login):
|
153
247
|
|
154
|
-
|
248
|
+
```
|
249
|
+
$ source ~/.rvm/scripts/rvm
|
250
|
+
```
|
155
251
|
|
156
|
-
|
252
|
+
It is advised to get one of the pre-compiled ruby version, you can list with:
|
157
253
|
|
158
|
-
|
254
|
+
```
|
255
|
+
$ rvm list --remote
|
256
|
+
```
|
159
257
|
|
160
|
-
|
258
|
+
Install the chosen pre-compiled Ruby version:
|
161
259
|
|
162
|
-
|
260
|
+
```
|
261
|
+
$ rvm install 2.7.2 --binary
|
262
|
+
```
|
263
|
+
|
264
|
+
Ruby is now installed for the user, go on to Gem installation.
|
265
|
+
|
266
|
+
### Generic: RVM: global installation (as root)
|
267
|
+
|
268
|
+
Follow the same method as single user install, but execute as "root".
|
163
269
|
|
164
|
-
|
165
|
-
|
270
|
+
As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
|
271
|
+
One can install in another location with :
|
166
272
|
|
167
273
|
```
|
168
|
-
curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
274
|
+
# curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
|
169
275
|
```
|
170
276
|
|
171
|
-
|
277
|
+
As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
|
278
|
+
If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
|
279
|
+
To activate ruby (and <%=cmd%>) later, source it:
|
172
280
|
|
173
281
|
```
|
174
|
-
# rvm
|
282
|
+
# source /etc/profile.d/rvm.sh.ok
|
283
|
+
# rvm version
|
175
284
|
```
|
176
285
|
|
177
|
-
|
178
|
-
|
286
|
+
### Windows: Installer
|
287
|
+
|
288
|
+
Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
|
289
|
+
|
290
|
+
* Go to "Downloads".
|
291
|
+
* Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
|
292
|
+
* At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
|
293
|
+
|
294
|
+
### macOS: pre-installed or `brew`
|
179
295
|
|
180
|
-
|
296
|
+
MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%= gemspec.name %> using `sudo` :
|
181
297
|
|
182
298
|
```
|
183
|
-
|
299
|
+
$ sudo gem install <%= gemspec.name %><%=geminstadd%>
|
184
300
|
```
|
185
301
|
|
186
|
-
|
302
|
+
Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
|
187
303
|
|
188
304
|
```
|
189
|
-
|
305
|
+
$ brew install ruby
|
190
306
|
```
|
191
307
|
|
192
|
-
|
308
|
+
### Linux: package
|
309
|
+
|
310
|
+
If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
|
311
|
+
|
312
|
+
Example:
|
313
|
+
|
314
|
+
```
|
315
|
+
# yum install -y ruby rubygems ruby-json
|
316
|
+
```
|
317
|
+
|
318
|
+
One can cleanup the whole yum-installed ruby environment like this to uninstall:
|
193
319
|
|
194
320
|
```
|
195
321
|
gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
|
@@ -198,7 +324,7 @@ yum remove -y ruby ruby-libs
|
|
198
324
|
|
199
325
|
### Other Unixes: Aix, etc...
|
200
326
|
|
201
|
-
If your unix do not provide a pre-built ruby, you can get using one of those
|
327
|
+
If your unix do not provide a pre-built ruby, you can get it using one of those
|
202
328
|
[methods](https://www.ruby-lang.org/en/documentation/installation/)
|
203
329
|
|
204
330
|
For instance to build from source, and install in `/opt/ruby` :
|
@@ -214,6 +340,50 @@ For instance to build from source, and install in `/opt/ruby` :
|
|
214
340
|
# make install
|
215
341
|
```
|
216
342
|
|
343
|
+
### <a name="offline_install"></a>Installation without internet access
|
344
|
+
|
345
|
+
Note that currently no pre-packaged version exist yet.
|
346
|
+
A method to build one provided here:
|
347
|
+
|
348
|
+
On a server with the same OS version and with internet access follow the "Generic single user installation" method.
|
349
|
+
|
350
|
+
Then create an archive:
|
351
|
+
|
352
|
+
```
|
353
|
+
$ cd
|
354
|
+
$ tar zcvf rvm-<%=cmd%>.tgz .rvm
|
355
|
+
```
|
356
|
+
|
357
|
+
Get the Aspera SDK. Execute:
|
358
|
+
|
359
|
+
```
|
360
|
+
$ <%=cmd%> conf --show-config|grep sdk_url
|
361
|
+
```
|
362
|
+
|
363
|
+
Then download the SDK archive from that URL.
|
364
|
+
|
365
|
+
Another method for the SDK is to install the SDK (`<%=cmd%> conf ascp install`) on the first system, and archive `$HOME/.aspera`.
|
366
|
+
|
367
|
+
Transfer those 2 archives to the target system without internet access.
|
368
|
+
|
369
|
+
On the target system:
|
370
|
+
|
371
|
+
* Extract the RVM archive either in a global location, or in a user's home folder : `path_to_rvm_root`
|
372
|
+
* in the user's `.profile` add this line: (replace `path_to_rvm_root` with the actual location)
|
373
|
+
|
374
|
+
```
|
375
|
+
source path_to_rvm_root/scripts/rvm
|
376
|
+
rvm use 2.7.2
|
377
|
+
```
|
378
|
+
|
379
|
+
For the SDK, either install from archive:
|
380
|
+
|
381
|
+
```
|
382
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
383
|
+
```
|
384
|
+
|
385
|
+
or restore the `$HOME/.aspera` folder for the user.
|
386
|
+
|
217
387
|
## <a name="the_gem"></a>`<%= gemspec.name %>` gem
|
218
388
|
|
219
389
|
Once you have Ruby and rights to install gems: Install the gem and its dependencies:
|
@@ -228,21 +398,40 @@ To upgrade to the latest version:
|
|
228
398
|
# gem update <%= gemspec.name %>
|
229
399
|
```
|
230
400
|
|
401
|
+
<%=tool%> checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
|
402
|
+
|
403
|
+
To check manually:
|
404
|
+
|
405
|
+
```
|
406
|
+
# <%=cmd%> conf check_update
|
407
|
+
```
|
408
|
+
|
409
|
+
|
410
|
+
|
231
411
|
## <a name="fasp_prot"></a>FASP Protocol
|
232
412
|
|
233
|
-
Most file transfers will be done using the FASP protocol
|
234
|
-
an Aspera Transfer:
|
413
|
+
Most file transfers will be done using the FASP protocol, using `ascp`.
|
414
|
+
Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
|
235
415
|
|
236
416
|
* ascp
|
237
417
|
* aspera-license (in same folder, or ../etc)
|
238
418
|
|
239
|
-
This can be installed
|
419
|
+
This can be installed either be installing an Aspera transfer sofware, or using an embedded command:
|
240
420
|
|
241
421
|
```
|
242
422
|
$ <%=cmd%> conf ascp install
|
243
423
|
```
|
244
424
|
|
245
|
-
|
425
|
+
If a local SDK installation is prefered instead of fetching from internet: one can specify the location of the SDK file:
|
426
|
+
|
427
|
+
```
|
428
|
+
$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
429
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
430
|
+
```
|
431
|
+
|
432
|
+
The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
|
433
|
+
|
434
|
+
If the embedded method is not used, the following packages are also suitable:
|
246
435
|
|
247
436
|
* IBM Aspera Connect Client (Free)
|
248
437
|
* IBM Aspera Desktop Client (Free)
|
@@ -251,7 +440,7 @@ Those can be found in one of IBM Aspera transfer server or client with its licen
|
|
251
440
|
* IBM Aspera High Speed Transfer EndPoint (Licensed)
|
252
441
|
|
253
442
|
For instance, Aspera Connect Client can be installed
|
254
|
-
by visiting the page: [
|
443
|
+
by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
|
255
444
|
|
256
445
|
<%=tool%> will detect most of Aspera transfer products in standard locations and use the first one found.
|
257
446
|
Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
|
@@ -259,6 +448,33 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
|
|
259
448
|
Several methods are provided on how to start a transfer. Use of a local client is one of them, but
|
260
449
|
other methods are available. Refer to section: [Transfer Agents](#agents)
|
261
450
|
|
451
|
+
## <a name="offline_install"></a>Offline Installation (without internet)
|
452
|
+
|
453
|
+
The procedure consists in:
|
454
|
+
|
455
|
+
* Follow the non-root installation procedure with RVM, including gem
|
456
|
+
* archive (zip, tar) the main RVM folder (includes <%=cmd%>):
|
457
|
+
|
458
|
+
```
|
459
|
+
$ cd ~
|
460
|
+
$ tar zcvf rvm_<%=cmd%>.tgz .rvm
|
461
|
+
```
|
462
|
+
|
463
|
+
* retrieve the SDK:
|
464
|
+
|
465
|
+
```
|
466
|
+
$ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
|
467
|
+
```
|
468
|
+
|
469
|
+
* on the system without internet access:
|
470
|
+
|
471
|
+
```
|
472
|
+
$ cd ~
|
473
|
+
$ tar zxvf rvm_<%=cmd%>.tgz
|
474
|
+
$ source ~/.rvm/scripts/rvm
|
475
|
+
$ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
476
|
+
```
|
477
|
+
|
262
478
|
# <a name="cli"></a>Command Line Interface: <%=tool%>
|
263
479
|
|
264
480
|
The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
@@ -439,6 +655,7 @@ By default, a table output will display one line per entry, and columns for each
|
|
439
655
|
* a,b,c : the list of attributes specified by the comma separated list
|
440
656
|
* Array extended value: for instance, @json:'["a","b","c"]' same as above
|
441
657
|
* +a,b,c : add selected properties to the default selection.
|
658
|
+
* -a,b,c : remove selected properties from the default selection.
|
442
659
|
|
443
660
|
## <a name="extended"></a>Extended Value Syntax
|
444
661
|
|
@@ -452,25 +669,25 @@ The extended value syntax is:
|
|
452
669
|
|
453
670
|
The difference between reader and decoder is order and ordinality. Both act like a function of value on right hand side. Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
|
454
671
|
|
455
|
-
The following "readers" are supported:
|
672
|
+
The following "readers" are supported (returns value in []):
|
456
673
|
|
457
|
-
* @val:VALUE
|
458
|
-
* @file:PATH
|
459
|
-
* @path:PATH
|
460
|
-
* @env:ENVVAR
|
461
|
-
* @stdin:
|
462
|
-
* @preset:NAME
|
674
|
+
* @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
|
675
|
+
* @file:PATH : [String] read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
|
676
|
+
* @path:PATH : [String] performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
|
677
|
+
* @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
|
678
|
+
* @stdin: : [String] read from stdin (no value on right)
|
679
|
+
* @preset:NAME : [Hash] get whole <%=opprst%> value by name
|
463
680
|
|
464
681
|
In addition it is possible to decode a value, using one or multiple decoders :
|
465
682
|
|
466
|
-
* @base64: decode a base64 encoded string
|
467
|
-
* @json: decode JSON values (convenient to provide complex structures)
|
468
|
-
* @zlib: uncompress data
|
469
|
-
* @ruby: execute ruby code
|
470
|
-
* @csvt: decode a titled CSV value
|
471
|
-
* @lines: split a string in multiple lines and return an array
|
472
|
-
* @list: split a string in multiple items taking first character as separator and return an array
|
473
|
-
* @incps: include values of presets specified by key
|
683
|
+
* @base64: [String] decode a base64 encoded string
|
684
|
+
* @json: [any] decode JSON values (convenient to provide complex structures)
|
685
|
+
* @zlib: [String] uncompress data
|
686
|
+
* @ruby: [any] execute ruby code
|
687
|
+
* @csvt: [Array] decode a titled CSV value
|
688
|
+
* @lines: [Array] split a string in multiple lines and return an array
|
689
|
+
* @list: [Array] split a string in multiple items taking first character as separator and return an array
|
690
|
+
* @incps: [Hash] include values of presets specified by key `incps` in input hash
|
474
691
|
|
475
692
|
To display the result of an extended value, use the `config echo` command.
|
476
693
|
|
@@ -502,7 +719,7 @@ $ <%=cmd%> config echo @csvt:@file:test.csv
|
|
502
719
|
:......:.....................:
|
503
720
|
```
|
504
721
|
|
505
|
-
Example: create a hash and include values from preset named "config" of config file
|
722
|
+
Example: create a hash and include values from preset named "config" of config file in this hash
|
506
723
|
|
507
724
|
```
|
508
725
|
$ <%=cmd%> config echo @incps:@json:'{"hello":true,"incps":["config"]}'
|
@@ -541,9 +758,9 @@ It can be overriden using the envinonment variable `<%=evp%>HOME`.
|
|
541
758
|
Example (Windows):
|
542
759
|
|
543
760
|
```
|
544
|
-
$ set <%=evp%>HOME=C:\Users\Kenji\.aspera
|
761
|
+
$ set <%=evp%>HOME=C:\Users\Kenji\.aspera\<%=cmd%>
|
545
762
|
$ <%=cmd%> config folder
|
546
|
-
C:\Users\Kenji\.aspera
|
763
|
+
C:\Users\Kenji\.aspera\<%=cmd%>
|
547
764
|
```
|
548
765
|
|
549
766
|
## <a name="configfile"></a>Configuration file
|
@@ -578,7 +795,7 @@ $ <%=cmd%> config id <<%=opprst%>> set|delete|show|initialize|update
|
|
578
795
|
The command `update` allows the easy creation of <%=prst%> by simply providing the options in their command line format, e.g. :
|
579
796
|
|
580
797
|
```
|
581
|
-
$ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=
|
798
|
+
$ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
|
582
799
|
```
|
583
800
|
|
584
801
|
* This creates a <%=prst%> `demo_server` with all provided options.
|
@@ -586,13 +803,13 @@ $ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --
|
|
586
803
|
The command `set` allows setting individual options in a <%=prst%>.
|
587
804
|
|
588
805
|
```
|
589
|
-
$ <%=cmd%> config id demo_server set password
|
806
|
+
$ <%=cmd%> config id demo_server set password _demo_pass_
|
590
807
|
```
|
591
808
|
|
592
809
|
The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
|
593
810
|
|
594
811
|
```
|
595
|
-
$ <%=cmd%> config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"
|
812
|
+
$ <%=cmd%> config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
|
596
813
|
```
|
597
814
|
|
598
815
|
A good practice is to not manually edit the configuration file and use modification commands instead.
|
@@ -629,11 +846,19 @@ Note that special plugin name: `config` can be associated with a preset that is
|
|
629
846
|
Operations on this preset are done using regular `config` operations:
|
630
847
|
|
631
848
|
```
|
632
|
-
$ <%=cmd%> config id default set _plugin_name_
|
849
|
+
$ <%=cmd%> config id default set _plugin_name_ _default_preset_for_plugin_
|
633
850
|
$ <%=cmd%> config id default get _plugin_name_
|
634
|
-
"
|
851
|
+
"_default_preset_for_plugin_"
|
635
852
|
```
|
636
853
|
|
854
|
+
### <a name="lprtdef"></a>Special Plugin: config
|
855
|
+
|
856
|
+
Plugin `config` (not to be confused with <%=prstt%> config) is used to configure <%=tool%> but it also contains global options.
|
857
|
+
|
858
|
+
When <%=tool%> starts, it lookjs for the `default` <%=prstt%> and if there is a value for `config`, if so, it loads the option values for any plugin used.
|
859
|
+
|
860
|
+
If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
|
861
|
+
|
637
862
|
### Format of file
|
638
863
|
|
639
864
|
The configuration file is a hash in a YAML file. Example:
|
@@ -649,7 +874,7 @@ cli_default:
|
|
649
874
|
demo_server:
|
650
875
|
url: ssh://demo.asperasoft.com:33001
|
651
876
|
username: asperaweb
|
652
|
-
password:
|
877
|
+
password: _demo_pass_
|
653
878
|
```
|
654
879
|
|
655
880
|
We can see here:
|
@@ -718,7 +943,6 @@ A <%=prst%> value can be removed with `unset`:
|
|
718
943
|
$ <%=cmd%> config id cli_default unset interactive
|
719
944
|
```
|
720
945
|
|
721
|
-
|
722
946
|
### Examples
|
723
947
|
|
724
948
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
@@ -838,21 +1062,42 @@ The `config` plugin also allows specification for the use of a local FASP client
|
|
838
1062
|
|
839
1063
|
```
|
840
1064
|
$ <%=cmd%> config ascp show
|
841
|
-
/Users/laurent/
|
1065
|
+
/Users/laurent/.aspera/ascli/sdk/ascp
|
1066
|
+
$ <%=cmd%> config ascp info
|
1067
|
+
+--------------------+-----------------------------------------------------------+
|
1068
|
+
| key | value |
|
1069
|
+
+--------------------+-----------------------------------------------------------+
|
1070
|
+
| ascp | /Users/laurent/.aspera/ascli/sdk/ascp |
|
1071
|
+
...
|
842
1072
|
```
|
843
1073
|
|
844
1074
|
### Selection of local `ascp`
|
845
1075
|
|
1076
|
+
By default, <%=tool%> uses any found local product with ascp, including SDK.
|
1077
|
+
|
846
1078
|
To temporarily use an alternate ascp path use option `ascp_path` (`--ascp-path=`)
|
847
1079
|
|
848
|
-
|
1080
|
+
For a permanent change, the command `config ascp use` sets the same parameter for the global default.
|
1081
|
+
|
1082
|
+
Using a POSIX shell:
|
849
1083
|
|
850
1084
|
```
|
851
1085
|
$ <%=cmd%> config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
|
852
|
-
|
1086
|
+
ascp version: 4.0.0.182279
|
1087
|
+
Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
|
1088
|
+
Saved to default global preset global_common_defaults
|
1089
|
+
```
|
1090
|
+
|
1091
|
+
Windows:
|
1092
|
+
|
1093
|
+
```
|
1094
|
+
$ <%=cmd%> config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1095
|
+
ascp version: 4.0.0.182279
|
1096
|
+
Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
|
1097
|
+
Saved to default global preset global_common_defaults
|
853
1098
|
```
|
854
1099
|
|
855
|
-
|
1100
|
+
If the path has spaces, read section: [Shell and Command line parsing](#parsing).
|
856
1101
|
|
857
1102
|
### List locally installed Aspera Transfer products
|
858
1103
|
|
@@ -939,7 +1184,7 @@ will effectively push files to the related server from the agent node.
|
|
939
1184
|
|
940
1185
|
### <a name="direct"></a>Direct (local ascp using FASPManager API)
|
941
1186
|
|
942
|
-
By default
|
1187
|
+
By default <%=tool%> uses a local ascp, equivalent to specifying `--transfer=direct`.
|
943
1188
|
<%=tool%> will detect locally installed Aspera products.
|
944
1189
|
Refer to section [FASP](#client).
|
945
1190
|
|
@@ -949,17 +1194,33 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
|
|
949
1194
|
* `EX_http_proxy_url` (proxy for legacy http fallback)
|
950
1195
|
* `EX_ascp_args`
|
951
1196
|
|
952
|
-
The `transfer-info`
|
1197
|
+
The `transfer-info` accepts the following optional parameters:
|
953
1198
|
|
954
1199
|
<table>
|
955
|
-
<tr><th>Name</th><th>Default</th><th>Feature</th><th>Description</th></tr>
|
956
|
-
<tr><td>
|
957
|
-
<tr><td>
|
958
|
-
<tr><td>
|
959
|
-
<tr><td>
|
960
|
-
<tr><td>wss</td> <td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
|
1200
|
+
<tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
|
1201
|
+
<tr><td>spawn_timeout_sec</td><td>Float</td><td>3</td><td>Multi session</td><td>Verification time that ascp is running</td></tr>
|
1202
|
+
<tr><td>spawn_delay_sec</td><td>Float</td><td>2</td><td>Multi session</td><td>Delay between startup of sessions</td></tr>
|
1203
|
+
<tr><td>wss</td><td>Bool</td><td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
|
1204
|
+
<tr><td>resume</td><td>Hash</td><td>nil</td><td>Resumer parameters</td><td>See below</td></tr>
|
961
1205
|
</table>
|
962
1206
|
|
1207
|
+
Resume parameters:
|
1208
|
+
|
1209
|
+
<table>
|
1210
|
+
<tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
|
1211
|
+
<tr><td>iter_max</td><td>int</td><td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
|
1212
|
+
<tr><td>sleep_initial</td><td>int</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
|
1213
|
+
<tr><td>sleep_factor</td><td>int</td><td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
|
1214
|
+
<tr><td>sleep_max</td><td>int</td><td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
|
1215
|
+
</table>
|
1216
|
+
|
1217
|
+
Examples:
|
1218
|
+
|
1219
|
+
```
|
1220
|
+
$ <%=cmd%> ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
|
1221
|
+
$ <%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5}'
|
1222
|
+
```
|
1223
|
+
|
963
1224
|
### IBM Aspera Connect Client GUI
|
964
1225
|
|
965
1226
|
By specifying option: `--transfer=connect`, <%=tool%> will start transfers
|
@@ -991,9 +1252,11 @@ If it possible to send using a HTTP gateway, in case FASP is not allowed.
|
|
991
1252
|
Example:
|
992
1253
|
|
993
1254
|
```
|
994
|
-
$ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://
|
1255
|
+
$ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
|
995
1256
|
```
|
996
1257
|
|
1258
|
+
Note that the gateway only supports transfers authorized with a token.
|
1259
|
+
|
997
1260
|
## <a name="transferspec"></a>Transfer Specification
|
998
1261
|
|
999
1262
|
Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
|
@@ -1007,7 +1270,7 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
|
|
1007
1270
|
|
1008
1271
|
<%=tool%> builds a default _transfer-spec_ internally, so it is not necessary to provide additional parameters on the command line for this transfer.
|
1009
1272
|
|
1010
|
-
If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters.
|
1273
|
+
If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
|
1011
1274
|
|
1012
1275
|
It is possible to specify ascp options when the `transfer` option is set to `direct` using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
|
1013
1276
|
|
@@ -1040,8 +1303,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
|
|
1040
1303
|
|
1041
1304
|
### List of files for transfers
|
1042
1305
|
|
1043
|
-
When uploading, downloading or sending files, the user must specify
|
1044
|
-
|
1306
|
+
When uploading, downloading or sending files, the user must specify the list of files to transfer.
|
1307
|
+
Most of the time, the list of files to transfer will be simply specified on the command line:
|
1045
1308
|
|
1046
1309
|
```
|
1047
1310
|
$ <%=cmd%> server upload ~/mysample.file secondfile
|
@@ -1055,7 +1318,8 @@ $ <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
|
|
1055
1318
|
|
1056
1319
|
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
|
1057
1320
|
|
1058
|
-
Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
|
1321
|
+
Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
|
1322
|
+
all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
|
1059
1323
|
|
1060
1324
|
For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
|
1061
1325
|
|
@@ -1298,7 +1562,7 @@ updated: my_aoc_org
|
|
1298
1562
|
Define this <%=prst%> as default configuration for the `aspera` plugin:
|
1299
1563
|
|
1300
1564
|
```
|
1301
|
-
$ <%=cmd%> config id default set
|
1565
|
+
$ <%=cmd%> config id default set aoc my_aoc_org
|
1302
1566
|
```
|
1303
1567
|
|
1304
1568
|
Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
|
@@ -1672,7 +1936,7 @@ Then, create two shared folders located in two regions, in your files home, in a
|
|
1672
1936
|
Then, transfer between those:
|
1673
1937
|
|
1674
1938
|
```
|
1675
|
-
$ <%=cmd%> -Paoc_show
|
1939
|
+
$ <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
|
1676
1940
|
```
|
1677
1941
|
|
1678
1942
|
* create registration key to register a node
|
@@ -1735,8 +1999,9 @@ Notes:
|
|
1735
1999
|
Examples:
|
1736
2000
|
|
1737
2001
|
```
|
1738
|
-
$ <%=cmd%> aoc
|
1739
|
-
$ <%=cmd%> aoc
|
2002
|
+
$ <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
|
2003
|
+
$ <%=cmd%> aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
|
2004
|
+
$ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
1740
2005
|
```
|
1741
2006
|
|
1742
2007
|
## <a name="aoccargo"></a>Receive new packages only
|
@@ -1751,10 +2016,10 @@ $ <%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
|
|
1751
2016
|
* `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
|
1752
2017
|
* `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
|
1753
2018
|
|
1754
|
-
Typically, one would regularly execute this command on a regular basis, using the method
|
2019
|
+
Typically, one would regularly execute this command on a regular basis, using the method of your choice:
|
1755
2020
|
|
1756
|
-
* Windows scheduler
|
1757
|
-
* cron
|
2021
|
+
* Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
|
2022
|
+
* Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
1758
2023
|
* etc...
|
1759
2024
|
|
1760
2025
|
## Download Files
|
@@ -1974,7 +2239,7 @@ This can also be set as default using a preset
|
|
1974
2239
|
One can test the "server" application using the well known demo server:
|
1975
2240
|
|
1976
2241
|
```
|
1977
|
-
$ <%=cmd%> config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=
|
2242
|
+
$ <%=cmd%> config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
|
1978
2243
|
$ <%=cmd%> config id default set server aspera_demo_server
|
1979
2244
|
$ <%=cmd%> server browse /aspera-test-dir-large
|
1980
2245
|
$ <%=cmd%> server download /aspera-test-dir-large/200MB
|
@@ -2070,25 +2335,80 @@ to download files.
|
|
2070
2335
|
$ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
|
2071
2336
|
```
|
2072
2337
|
|
2073
|
-
# Plugin: IBM Aspera
|
2338
|
+
# Plugin: IBM Aspera Faspex5
|
2074
2339
|
|
2075
|
-
|
2340
|
+
3 authentication methods are supported:
|
2076
2341
|
|
2077
|
-
|
2342
|
+
* boot
|
2343
|
+
* web
|
2344
|
+
* jwt
|
2345
|
+
|
2346
|
+
For boot method:
|
2078
2347
|
|
2079
|
-
|
2348
|
+
* open a browser
|
2349
|
+
* start developer mode
|
2350
|
+
* login to faspex 5
|
2351
|
+
* find the first API call with `Authorization` token, and copy it (kind of base64 long string)
|
2352
|
+
|
2353
|
+
Use it as password and use `--auth=boot`.
|
2080
2354
|
|
2081
2355
|
```
|
2082
|
-
--
|
2356
|
+
$ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
|
2083
2357
|
```
|
2084
2358
|
|
2085
|
-
|
2359
|
+
For web method, create an API client in Faspex, and use: --auth=web
|
2360
|
+
|
2361
|
+
For JWT, create an API client in Faspex with jwt supporot, and use: --auth=jwt
|
2362
|
+
as of beta£3 this does not allow regular users.
|
2086
2363
|
|
2087
|
-
|
2364
|
+
Ready to use Faspex5 with CLI.
|
2088
2365
|
|
2366
|
+
Once the graphical registration form exist, ther bootstrap method can be removed.
|
2367
|
+
|
2368
|
+
# Plugin: IBM Aspera Faspex (4.x)
|
2369
|
+
|
2370
|
+
Notes:
|
2089
2371
|
|
2090
|
-
|
2091
|
-
[Reference on Developer Site](https://
|
2372
|
+
* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
|
2373
|
+
* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
|
2374
|
+
|
2375
|
+
## Receiving a Package
|
2376
|
+
|
2377
|
+
The command is `package recv`, possible methosd are:
|
2378
|
+
|
2379
|
+
* provide a package id with option `id`
|
2380
|
+
* provide a public link with option `link`
|
2381
|
+
* provide a `faspe:` URI with option `link`
|
2382
|
+
|
2383
|
+
```
|
2384
|
+
$ <%=cmd%> faspex package recv --id=12345
|
2385
|
+
$ <%=cmd%> faspex package recv --link=faspe://...
|
2386
|
+
```
|
2387
|
+
|
2388
|
+
If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
|
2389
|
+
|
2390
|
+
```
|
2391
|
+
$ <%=cmd%> faspex package list --recipient='*thedropboxname'
|
2392
|
+
```
|
2393
|
+
|
2394
|
+
|
2395
|
+
|
2396
|
+
## Sending a Package
|
2397
|
+
|
2398
|
+
The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
|
2399
|
+
|
2400
|
+
Example:
|
2401
|
+
|
2402
|
+
```
|
2403
|
+
$ <%=cmd%> faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
|
2404
|
+
```
|
2405
|
+
|
2406
|
+
If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
|
2407
|
+
|
2408
|
+
Additional optional parameters in `delivery_info`:
|
2409
|
+
|
2410
|
+
* Package Note: : `"note":"note this and that"`
|
2411
|
+
* Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
|
2092
2412
|
|
2093
2413
|
## operation on dropboxes
|
2094
2414
|
|
@@ -2151,6 +2471,8 @@ $ for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p
|
|
2151
2471
|
# Plugin: IBM Cloud Object Storage
|
2152
2472
|
|
2153
2473
|
The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
|
2474
|
+
It uses the same transfer service as Aspera on Cloud.
|
2475
|
+
see [https://status.aspera.io](https://status.aspera.io)
|
2154
2476
|
|
2155
2477
|
Required options are either:
|
2156
2478
|
|
@@ -2207,7 +2529,14 @@ Endpoints for regions can be found by querying the `endpoints` URL.
|
|
2207
2529
|
For convenience, let us create a default configuration, for example:
|
2208
2530
|
|
2209
2531
|
```
|
2210
|
-
$ <%=cmd%> conf id mycos update --service-credentials=@val:@json:@file
|
2532
|
+
$ <%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
|
2533
|
+
$ <%=cmd%> conf id default set cos mycos
|
2534
|
+
```
|
2535
|
+
|
2536
|
+
or using direct parameters:
|
2537
|
+
|
2538
|
+
```
|
2539
|
+
$ <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
|
2211
2540
|
$ <%=cmd%> conf id default set cos mycos
|
2212
2541
|
```
|
2213
2542
|
|
@@ -2225,7 +2554,7 @@ to start from ma configuration file, using <%=tool%> standard options.
|
|
2225
2554
|
|
2226
2555
|
# Plugin: Preview
|
2227
2556
|
|
2228
|
-
The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on
|
2557
|
+
The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
|
2229
2558
|
This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
|
2230
2559
|
Several parameters can be used to tune several aspects:
|
2231
2560
|
|
@@ -2273,6 +2602,7 @@ The tool requires the following external tools available in the `PATH`:
|
|
2273
2602
|
* OptiPNG : `optipng`
|
2274
2603
|
* FFmpeg : `ffmpeg` `ffprobe`
|
2275
2604
|
* Libreoffice : `libreoffice`
|
2605
|
+
* ruby gem `mimemagic`
|
2276
2606
|
|
2277
2607
|
Here shown on Redhat/CentOS.
|
2278
2608
|
|
@@ -2284,6 +2614,26 @@ To check if all tools are found properly, execute:
|
|
2284
2614
|
$ <%=cmd%> preview check
|
2285
2615
|
```
|
2286
2616
|
|
2617
|
+
### mimemagic
|
2618
|
+
|
2619
|
+
To benefit from extra mime type detection install gem mimemagic:
|
2620
|
+
|
2621
|
+
```
|
2622
|
+
# gem install mimemagic
|
2623
|
+
```
|
2624
|
+
|
2625
|
+
or to install an earlier version if any problem:
|
2626
|
+
|
2627
|
+
```
|
2628
|
+
# gem install mimemagic -v '~> 0.3.0'
|
2629
|
+
```
|
2630
|
+
|
2631
|
+
To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
|
2632
|
+
|
2633
|
+
If not used, Mime type used for conversion is the one provided by the node API.
|
2634
|
+
|
2635
|
+
If used, it the `preview` command will first analyse the file content using mimemagic, and if no match, will try by extension.
|
2636
|
+
|
2287
2637
|
### Image: Imagemagick and optipng
|
2288
2638
|
|
2289
2639
|
```
|
@@ -2308,90 +2658,111 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
|
|
2308
2658
|
# dnf install unoconv
|
2309
2659
|
```
|
2310
2660
|
|
2661
|
+
* Amazon Linux
|
2662
|
+
|
2663
|
+
```
|
2664
|
+
# amazon-linux-extras enable libreoffice
|
2665
|
+
# yum clean metadata
|
2666
|
+
# yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
|
2667
|
+
# wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
|
2668
|
+
# mv unoconv /usr/bin
|
2669
|
+
# chmod a+x /usr/bin/unoconv
|
2670
|
+
```
|
2311
2671
|
|
2312
2672
|
## Configuration
|
2313
2673
|
|
2314
|
-
|
2674
|
+
The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
|
2675
|
+
|
2676
|
+
Like any <%=tool%> commands, parameters can be passed on command line or using a configuration <%=prst%>. The configuration file must be created with the same user used to run so that it is properly used on runtime.
|
2677
|
+
|
2678
|
+
Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
|
2315
2679
|
|
2316
2680
|
```
|
2317
2681
|
# su -s /bin/bash - xfer
|
2318
|
-
$ <%=cmd%> config id
|
2319
|
-
$ <%=cmd%> config id default set preview
|
2682
|
+
$ <%=cmd%> config id previewconf update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
|
2683
|
+
$ <%=cmd%> config id default set preview previewconf
|
2320
2684
|
```
|
2321
2685
|
|
2322
|
-
Here we assume that Office file generation is disabled, else remove
|
2686
|
+
Here we assume that Office file generation is disabled, else remove this option.
|
2687
|
+
`lock_port` prevents concurrent execution of generation when using a scheduler.
|
2323
2688
|
|
2324
|
-
|
2689
|
+
One can check if the access key is well configured using:
|
2325
2690
|
|
2326
2691
|
```
|
2327
|
-
$ <%=cmd%> -
|
2692
|
+
$ <%=cmd%> -Ppreviewconf node browse /
|
2328
2693
|
```
|
2329
2694
|
|
2330
2695
|
This shall list the contents of the storage root of the access key.
|
2331
2696
|
|
2332
2697
|
## Execution
|
2333
2698
|
|
2334
|
-
The tool intentionally supports only a "one shot" mode in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
2335
|
-
It needs to be run
|
2699
|
+
The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
|
2700
|
+
It needs to be run on a regular basis to create or update preview files. For that use your best
|
2336
2701
|
reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
|
2337
2702
|
|
2338
|
-
Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be
|
2339
|
-
should be run as user `xfer`.
|
2703
|
+
Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
|
2340
2704
|
|
2341
2705
|
Lets do a one shot test, using the configuration previously created:
|
2342
2706
|
|
2343
2707
|
```
|
2344
2708
|
# su -s /bin/bash - xfer
|
2345
|
-
$ <%=cmd%> preview scan --overwrite=always
|
2709
|
+
xfer$ <%=cmd%> preview scan --overwrite=always
|
2346
2710
|
```
|
2347
2711
|
|
2348
2712
|
When the preview generator is first executed it will create a file: `.aspera_access_key`
|
2349
|
-
which contains the access key used.
|
2713
|
+
in the previews folder which contains the access key used.
|
2350
2714
|
On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
|
2351
2715
|
|
2352
2716
|
## Configuration for Execution in scheduler
|
2353
2717
|
|
2354
|
-
Here is an example of configuration for use with cron on Linux.
|
2718
|
+
Here is an example of configuration for use with cron on Linux.
|
2719
|
+
Adapt the scripts to your own preference.
|
2355
2720
|
|
2356
2721
|
We assume here that a configuration preset was created as shown previously.
|
2357
2722
|
|
2358
|
-
|
2723
|
+
Lets first setup a script that will be used in the sceduler and sets up the environment.
|
2724
|
+
|
2725
|
+
Example of startup script `cron_<%=cmd%>`, which sets the Ruby environment and adds some timeout protection:
|
2726
|
+
|
2727
|
+
```
|
2728
|
+
#!/bin/bash
|
2729
|
+
# set a timeout protection, just in case
|
2730
|
+
case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
|
2731
|
+
. /etc/profile.d/rvm.sh
|
2732
|
+
rvm use 2.6 --quiet
|
2733
|
+
exec timeout ${tmout} <%=cmd%> "${@}"
|
2734
|
+
```
|
2735
|
+
|
2736
|
+
Here the cronjob is created for user `xfer`.
|
2359
2737
|
|
2360
2738
|
```
|
2361
|
-
|
2362
|
-
|
2363
|
-
|
2739
|
+
xfer$ crontab<<EOF
|
2740
|
+
0 * * * * /home/xfer/cron_<%=cmd%> preview scan --logger=syslog --display=error
|
2741
|
+
2-59 * * * * /home/xfer/cron_<%=cmd%> preview trev --logger=syslog --display=error
|
2364
2742
|
EOF
|
2365
2743
|
```
|
2366
2744
|
|
2367
|
-
|
2745
|
+
Note that the loging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
|
2368
2746
|
|
2369
2747
|
## Candidate detection for creation or update (or deletion)
|
2370
2748
|
|
2371
|
-
The tool
|
2749
|
+
The tool generates preview files using those commands:
|
2372
2750
|
|
2373
2751
|
* `trevents` : only recently uploaded files will be tested (transfer events)
|
2374
2752
|
* `events` : only recently uploaded files will be tested (file events: not working)
|
2375
|
-
* `scan` :
|
2376
|
-
* `
|
2377
|
-
* `file` : for an individual file generation
|
2378
|
-
|
2379
|
-
Note that for the `event`, the option `iteration_file` should be specified so that
|
2380
|
-
successive calls only process new events. This file will hold an identifier
|
2381
|
-
telling from where to get new events.
|
2382
|
-
|
2383
|
-
It is also possible to test a local file, using the `test` command.
|
2753
|
+
* `scan` : recursively scan all files under the access key's "storage root"
|
2754
|
+
* `test` : test using a local file
|
2384
2755
|
|
2385
2756
|
Once candidate are selected, once candidates are selected,
|
2386
2757
|
a preview is always generated if it does not exist already,
|
2387
2758
|
else if a preview already exist, it will be generated
|
2388
|
-
using one of three overwrite
|
2759
|
+
using one of three values for the `overwrite` option:
|
2389
2760
|
|
2390
2761
|
* `always` : preview is always generated, even if it already exists and is newer than original
|
2391
2762
|
* `never` : preview is generated only if it does not exist already
|
2392
2763
|
* `mtime` : preview is generated only if the original file is newer than the existing
|
2393
2764
|
|
2394
|
-
Deletion of preview for deleted source files: not implemented yet.
|
2765
|
+
Deletion of preview for deleted source files: not implemented yet (TODO).
|
2395
2766
|
|
2396
2767
|
If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
|
2397
2768
|
|
@@ -2401,6 +2772,14 @@ $ <%=cmd%> preview scan --skip-folders=@json:'["/not_here"]'
|
|
2401
2772
|
|
2402
2773
|
The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
|
2403
2774
|
|
2775
|
+
When scanning the option `value` has the same behaviour as for the `node find` command.
|
2776
|
+
|
2777
|
+
For instance to filter out files beginning with `._` do:
|
2778
|
+
|
2779
|
+
```
|
2780
|
+
... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
|
2781
|
+
```
|
2782
|
+
|
2404
2783
|
## Preview File types
|
2405
2784
|
|
2406
2785
|
Two types of preview can be generated:
|
@@ -2538,7 +2917,7 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
|
|
2538
2917
|
## Simple session
|
2539
2918
|
|
2540
2919
|
```
|
2541
|
-
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"
|
2920
|
+
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
|
2542
2921
|
|
2543
2922
|
echo "${MY_TSPEC}"|asession
|
2544
2923
|
```
|
@@ -2551,7 +2930,7 @@ This is particularly useful for a persistent session ( with the [_transfer-spec_
|
|
2551
2930
|
|
2552
2931
|
```
|
2553
2932
|
$ asession
|
2554
|
-
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"
|
2933
|
+
{"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
|
2555
2934
|
{"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
|
2556
2935
|
{"type":"DONE"}
|
2557
2936
|
```
|
@@ -2625,6 +3004,34 @@ $ <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=1
|
|
2625
3004
|
|
2626
3005
|
The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
2627
3006
|
|
3007
|
+
# Aspera Health check and Nagios
|
3008
|
+
|
3009
|
+
Each plugin provide a `health` command that will check the health status of the application. Example:
|
3010
|
+
|
3011
|
+
```
|
3012
|
+
$ <%=cmd%> console health
|
3013
|
+
+--------+-------------+------------+
|
3014
|
+
| status | component | message |
|
3015
|
+
+--------+-------------+------------+
|
3016
|
+
| ok | console api | accessible |
|
3017
|
+
+--------+-------------+------------+
|
3018
|
+
```
|
3019
|
+
|
3020
|
+
Typically, the health check uses the REST API of the application with the following exception: the `server` plugin allows checking health by:
|
3021
|
+
|
3022
|
+
* issuing a transfer to the server
|
3023
|
+
* checking web app status with `asctl all:status`
|
3024
|
+
* checking daemons process status
|
3025
|
+
|
3026
|
+
<%=tool%> can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
|
3027
|
+
|
3028
|
+
```
|
3029
|
+
$ <%=cmd%> server health transfer --to-folder=/Upload --format=nagios --progress=none
|
3030
|
+
OK - [transfer:ok]
|
3031
|
+
$ <%=cmd%> server health asctlstatus --cmd_prefix='sudo ' --format=nagios
|
3032
|
+
OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
|
3033
|
+
```
|
3034
|
+
|
2628
3035
|
# Module: `Aspera`
|
2629
3036
|
|
2630
3037
|
Main components:
|
@@ -2644,6 +3051,16 @@ This sample code shows some example of use of the API as well as
|
|
2644
3051
|
REST API.
|
2645
3052
|
Note: although nice, it's probably a good idea to use RestClient for REST.
|
2646
3053
|
|
3054
|
+
Example of use of the API of Aspera on Cloud:
|
3055
|
+
|
3056
|
+
```
|
3057
|
+
require 'aspera/aoc'
|
3058
|
+
|
3059
|
+
aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/<%=cmd%>/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
|
3060
|
+
|
3061
|
+
aoc.read('self')
|
3062
|
+
```
|
3063
|
+
|
2647
3064
|
# History
|
2648
3065
|
|
2649
3066
|
When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
|
@@ -2660,11 +3077,43 @@ So, it evolved into <%=tool%>:
|
|
2660
3077
|
* supports transfers with multiple [Transfer Agents](#agents), that's why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
|
2661
3078
|
* `ruby` is consistent with other Aspera products
|
2662
3079
|
|
3080
|
+
# Changes (Release notes)
|
3081
|
+
|
3082
|
+
* <%= gemspec.version.to_s %>
|
2663
3083
|
|
3084
|
+
* new: command `faspex package recv` supports link of type: `faspe:`
|
3085
|
+
* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
|
2664
3086
|
|
2665
|
-
|
3087
|
+
* 4.2.0
|
2666
3088
|
|
2667
|
-
*
|
3089
|
+
* new: command `aoc remind` to receive organization membership by email
|
3090
|
+
* new: in `preview` option `value` to filter out on file name
|
3091
|
+
* new: `initdemo` to initialize for demo server
|
3092
|
+
* new: `direct` transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
|
3093
|
+
* fix: on Windows `conf ascp use` expects ascp.exe
|
3094
|
+
* fix: (break) multi_session_threshold is Integer, not String
|
3095
|
+
* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
|
3096
|
+
* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
|
3097
|
+
* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
|
3098
|
+
* change: (break) parameters for resume in `transfer-info` for `direct` are now in sub-key `"resume"`
|
3099
|
+
|
3100
|
+
* 4.1.0
|
3101
|
+
|
3102
|
+
* fix: remove keys from transfer spec and command line when not needed
|
3103
|
+
* fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
|
3104
|
+
* new: update documentation with regard to offline and docker installation
|
3105
|
+
* new: renamed command `nagios_check` to `health`
|
3106
|
+
* new: agent `http_gw` now supports upload
|
3107
|
+
* new: added option `sdk_url` to install SDK from local file for offline install
|
3108
|
+
* new: check new gem version periodically
|
3109
|
+
* new: the --fields= option, support -_fieldname_ to remove a field from default fields
|
3110
|
+
* new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
|
3111
|
+
* new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
|
3112
|
+
* new: AoC a password can be provided for a public link
|
3113
|
+
* new: `conf doc` take an optional parameter to go to a section
|
3114
|
+
* new: initial support for Faspex 5 Beta 1
|
3115
|
+
|
3116
|
+
* 4.0.0
|
2668
3117
|
|
2669
3118
|
* now available as open source at [<%= gemspec.homepage %>](<%= gemspec.homepage %>) with general cleanup
|
2670
3119
|
* changed default tool name from `mlia` to `ascli`
|
@@ -2792,7 +3241,7 @@ So, it evolved into <%=tool%>:
|
|
2792
3241
|
|
2793
3242
|
* 0.10.6
|
2794
3243
|
|
2795
|
-
* FaspManager: transfer spec `authentication` no more needed for local tranfer to use
|
3244
|
+
* FaspManager: transfer spec `authentication` no more needed for local tranfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
|
2796
3245
|
* gem version requirements made more open
|
2797
3246
|
|
2798
3247
|
* 0.10.5
|
@@ -3019,11 +3468,15 @@ Breaking change:
|
|
3019
3468
|
|
3020
3469
|
* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
|
3021
3470
|
|
3022
|
-
# BUGS
|
3471
|
+
# BUGS, FEATURES, CONTRIBUTION
|
3472
|
+
|
3473
|
+
For issues or feature requests use the Github repository and issues.
|
3023
3474
|
|
3024
|
-
|
3475
|
+
You can also contribute to this open source project.
|
3025
3476
|
|
3026
|
-
|
3477
|
+
One can also create one's own command nplugin.
|
3478
|
+
|
3479
|
+
## Only one value for any option
|
3027
3480
|
|
3028
3481
|
Some commands and sub commands may ask for the same option name.
|
3029
3482
|
Currently, since option definition is position independant (last one wins), it is not possible
|
@@ -3037,7 +3490,8 @@ This happens typically for the `node` sub command, e.g. identify the node by nam
|
|
3037
3490
|
|
3038
3491
|
## ED255519 key not supported
|
3039
3492
|
|
3040
|
-
|
3493
|
+
ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
|
3494
|
+
|
3041
3495
|
Without this deactivation, if such key was present the following error was generated:
|
3042
3496
|
|
3043
3497
|
```
|
@@ -3047,7 +3501,17 @@ OpenSSH keys only supported if ED25519 is available
|
|
3047
3501
|
Which meant that you do not have ruby support for ED25519 SSH keys.
|
3048
3502
|
You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
|
3049
3503
|
|
3050
|
-
|
3504
|
+
## Error "Remote host is not who we expected"
|
3505
|
+
|
3506
|
+
`ascp` version 4.x changed the algorithm used to check the SSH server certificate. To ignore the certificate (SSH fingerprint) add option on client side:
|
3507
|
+
|
3508
|
+
```
|
3509
|
+
--ts=@json:'{"sshfp":null}'
|
3510
|
+
```
|
3511
|
+
|
3512
|
+
Refer to ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server): if you have access to server side, basically disable other SSH host keys than RSA.
|
3513
|
+
|
3514
|
+
## Miscelaneous
|
3051
3515
|
|
3052
3516
|
* remove rest and oauth classes and use ruby standard gems:
|
3053
3517
|
|
@@ -3063,9 +3527,3 @@ You may either install the suggested Gems, or remove your ed25519 key from your
|
|
3063
3527
|
* Going through proxy: use env var http_proxy and https_proxy, no_proxy
|
3064
3528
|
|
3065
3529
|
* easier use with https://github.com/pmq20/ruby-packer
|
3066
|
-
|
3067
|
-
# Contribution
|
3068
|
-
|
3069
|
-
Send comments !
|
3070
|
-
|
3071
|
-
Create your own plugin !
|