aspera-cli 4.0.0.pre3 → 4.2.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +695 -205
  3. data/bin/dascli +13 -0
  4. data/docs/README.erb.md +615 -157
  5. data/docs/test_env.conf +23 -5
  6. data/docs/transfer_spec.html +1 -1
  7. data/examples/aoc.rb +14 -3
  8. data/examples/faspex4.rb +78 -0
  9. data/lib/aspera/aoc.rb +87 -108
  10. data/lib/aspera/cli/formater.rb +2 -0
  11. data/lib/aspera/cli/main.rb +46 -34
  12. data/lib/aspera/cli/plugin.rb +9 -4
  13. data/lib/aspera/cli/plugins/alee.rb +1 -1
  14. data/lib/aspera/cli/plugins/aoc.rb +207 -182
  15. data/lib/aspera/cli/plugins/ats.rb +2 -2
  16. data/lib/aspera/cli/plugins/config.rb +173 -117
  17. data/lib/aspera/cli/plugins/console.rb +2 -2
  18. data/lib/aspera/cli/plugins/faspex.rb +51 -36
  19. data/lib/aspera/cli/plugins/faspex5.rb +82 -41
  20. data/lib/aspera/cli/plugins/node.rb +3 -3
  21. data/lib/aspera/cli/plugins/preview.rb +35 -25
  22. data/lib/aspera/cli/plugins/server.rb +23 -8
  23. data/lib/aspera/cli/transfer_agent.rb +7 -6
  24. data/lib/aspera/cli/version.rb +1 -1
  25. data/lib/aspera/cos_node.rb +33 -28
  26. data/lib/aspera/environment.rb +2 -2
  27. data/lib/aspera/fasp/connect.rb +28 -21
  28. data/lib/aspera/fasp/http_gw.rb +140 -28
  29. data/lib/aspera/fasp/installation.rb +101 -53
  30. data/lib/aspera/fasp/local.rb +88 -45
  31. data/lib/aspera/fasp/manager.rb +15 -0
  32. data/lib/aspera/fasp/node.rb +4 -4
  33. data/lib/aspera/fasp/parameters.rb +6 -18
  34. data/lib/aspera/fasp/resume_policy.rb +13 -12
  35. data/lib/aspera/log.rb +1 -1
  36. data/lib/aspera/node.rb +61 -1
  37. data/lib/aspera/oauth.rb +49 -46
  38. data/lib/aspera/persistency_folder.rb +9 -4
  39. data/lib/aspera/preview/file_types.rb +53 -21
  40. data/lib/aspera/preview/generator.rb +3 -3
  41. data/lib/aspera/rest.rb +29 -18
  42. data/lib/aspera/secrets.rb +20 -0
  43. data/lib/aspera/temp_file_manager.rb +19 -0
  44. data/lib/aspera/web_auth.rb +105 -0
  45. metadata +42 -22
data/bin/dascli ADDED
@@ -0,0 +1,13 @@
1
+ #!/usr/bin/env bash
2
+ tag=martinlaurent/ascli:latest
3
+ case "$1" in install)
4
+ docker pull $tag
5
+ exit 0
6
+ esac
7
+ conf_host=${ASCLI_HOME:-$HOME/.aspera/ascli}
8
+ if test ! -d $conf_host;then
9
+ echo creating foder: $conf_host
10
+ mkdir -p $conf_host
11
+ fi
12
+ conf_img=/usr/src/app/config
13
+ exec docker run --tty --interactive --rm --env ASCLI_HOME="$conf_img" --volume "$conf_host:$conf_img" $tag ascli "$@"
data/docs/README.erb.md CHANGED
@@ -15,32 +15,77 @@ prstt=opprst.capitalize # in title
15
15
  gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
16
16
  geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
17
17
  -%>
18
- # <%=tool%> : a Command Line for IBM Aspera products
18
+ # <%=tool%> : Command Line Interface for IBM Aspera products
19
19
 
20
20
  Version : <%= gemspec.version.to_s %>
21
21
 
22
22
  _Laurent/2016-<%=Time.new.year%>_
23
23
 
24
- This gem provides a command line interface to Aspera Applications.
24
+ This gem provides <%=tool%>: a command line interface to Aspera Applications.
25
25
 
26
- Location (once released):
27
- [<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
26
+ <%=tool%> is a also great tool to learn Aspera APIs.
28
27
 
29
- Disclaimers:
28
+ Ruby Gem: [<%= gemspec.metadata['rubygems_uri'] %>](<%= gemspec.metadata['rubygems_uri'] %>)
30
29
 
31
- * This has not yet been officially released so things may change
30
+ Ruby Doc: [<%= gemspec.metadata['documentation_uri'] %>](<%= gemspec.metadata['documentation_uri'] %>)
32
31
 
33
- That being said, <%=tool%> is very powerful and gets things done, it&apos;s also a great tool to learn Aspera APIs.
32
+ Ruby version must be >= <%= gemspec.required_ruby_version %>
34
33
 
35
- This manual addresses three parts:
34
+ # <a name="when_to_use"></a>When to use and when not to use
36
35
 
37
- * <%=tool%> : The command line tool
38
- * `asession` : starting a FASP Session with JSON parameters
39
- * `Aspera` : includes a Ruby "FASPManager"
36
+ <%=tool%> is designed to be used as a command line tool to:
37
+
38
+ * execute commands on Aspera products
39
+ * transfer to/from Aspera products
40
+
41
+ So it is designed for:
42
+
43
+ * Interactive operations on a text terminal (typically, VT100 compatible)
44
+ * Batch operations in (shell) scripts (e.g. cron job)
45
+
46
+ <%=tool%> can be seen as a command line tool integrating:
47
+
48
+ * a configuration file (config.yaml) and advanced command line options
49
+ * cURL (for REST calls)
50
+ * Aspera transfer (ascp)
51
+
52
+ One might be tempted to use it as an integration element, e.g. by building a command line programmatically, and then executing it. It is generally not a good idea.
53
+ For such integration cases, e.g. performing operations and transfer to aspera products, it is preferred to use [Aspera APIs](https://ibm.biz/aspera_api):
54
+
55
+ * Product APIs (REST) : e.g. AoC, Faspex, node
56
+ * Transfer SDK : with gRPC interface and laguage stubs (C, C++, Python, .NET/C#, java, ruby, etc...)
57
+
58
+ Using APIs (application REST API and transfer SDK) will prove to be easier to develop and maintain.
59
+
60
+ For scripting and ad'hoc command line operations, <%=tool%> is perfect.
61
+
62
+ # Notations
40
63
 
41
64
  In examples, command line operations (starting with `$`) are shown using a standard shell: `bash` or `zsh`.
65
+ Prompt `# ` refers to user `root`, prompt `xfer$ ` refer to user `xfer`.
66
+
67
+ Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
68
+
69
+ # <a name="parsing"></a>Shell and Command line parsing
70
+
71
+ <%=tool%> is typically executed in a shell, either interactively or in a script. <%=tool%> receives its arguments from this shell.
72
+
73
+ On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before <%=tool%> (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to <%=tool%>. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
74
+
75
+ On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
76
+
77
+ * [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
78
+ * [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
42
79
 
43
- Command line parameters in example beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
80
+ In case of doubt of argument values after parsing test like this:
81
+
82
+ ```
83
+ $ <%=cmd%> conf echo "Hello World" arg2 3
84
+ "Hello World"
85
+ ERROR: Argument: unprocessed values: ["arg2", "3"]
86
+ ```
87
+
88
+ `echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (ascli), next command line arguments are shown in the error message.
44
89
 
45
90
  # Quick Start
46
91
 
@@ -61,9 +106,10 @@ Once installation is completed, you can proceed to the first use with a demo ser
61
106
 
62
107
  If you want to test with Aspera on Cloud, jump to section: [Wizard](#aocwizard)
63
108
 
64
- If you want to test with Aspera demo transfer server, a default configuration is created on first use:
109
+ To test with Aspera demo transfer server, setup the environment and then test:
65
110
 
66
111
  ```
112
+ $ <%=cmd%> config initdemo
67
113
  $ <%=cmd%> server browse /
68
114
  :............:...........:......:........:...........................:.......................:
69
115
  : zmode : zuid : zgid : size : mtime : name :
@@ -83,7 +129,7 @@ If you want to use <%=tool%> with another server, and in order to make further c
83
129
  * download a file
84
130
 
85
131
  ```
86
- $ <%=cmd%> config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera
132
+ $ <%=cmd%> config id myserver update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
87
133
  updated: myserver
88
134
  $ <%=cmd%> config id default set server myserver
89
135
  updated: default&rarr;server to myserver
@@ -117,79 +163,159 @@ Then, follow the section relative to the product you want to interact with ( Asp
117
163
 
118
164
  # <a name="installation"></a>Installation
119
165
 
120
- In order to use the tool or the gem, it is necessary to install those components:
166
+ It is possible to install *either* directly on the host operating system (Linux, Windows, Macos) or as a docker container.
121
167
 
122
- * [Ruby](#ruby)
168
+ The direct installation is recommended and consists in installing:
169
+
170
+ * [Ruby](#ruby) version >= <%= gemspec.required_ruby_version %>
123
171
  * [<%= gemspec.name %>](#the_gem)
124
- * [FASP](#fasp_prot)
172
+ * [Aspera SDK (ascp)](#fasp_prot)
173
+
174
+ The following sections provide information on the various installation methods.
175
+
176
+ An internet connection is required for the installation. If you dont have internet for the installation, refer to section [Installation without internet access](#offline_install).
177
+
178
+ ## Docker container
125
179
 
126
- The following sections provide information on the installation.
180
+ Use this method only if you know what you do, else use the standard recommended method as described here above.
181
+
182
+ This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
183
+
184
+ Ensure that you have Docker installed.
185
+
186
+ ```
187
+ $ docker --version
188
+ ```
189
+
190
+ Download the wrapping script:
191
+
192
+ ```
193
+ $ curl -o <%=cmd%> https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
194
+ $ chmod a+x <%=cmd%>
195
+ ```
196
+
197
+ Install the container image:
198
+
199
+ ```
200
+ $ ./<%=cmd%> install
201
+ ```
202
+
203
+ Start using it !
204
+
205
+ Note that the tool is run in the container, so transfers are also executed in the container, not calling host.
206
+
207
+ The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/<%=cmd%>` on host.
208
+
209
+ To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
210
+ To add local storage as a volume edit the script: ascli and add a `--volume` stanza.
127
211
 
128
212
  ## <a name="ruby"></a>Ruby
129
213
 
214
+ Use this method to install on the native host.
215
+
130
216
  A ruby interpreter is required to run the tool or to use the gem and tool.
131
- The Ruby version shall be at least <%= gemspec.required_ruby_version %>.
132
- Any type of Ruby installation can be used.
133
- Ruby 3 is not yet tested.
217
+
218
+ Ruby minimum version: <%= gemspec.required_ruby_version %>. Ruby version 3 is also supported.
219
+
220
+ *Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
134
221
 
135
222
  Refer to the following sections for a proposed method for specific operating systems.
136
223
 
137
- ### macOS
224
+ The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
225
+ If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
226
+ If you have a simpler better way to install Ruby version >= <%= gemspec.required_ruby_version %> : use it !
227
+
228
+ ### Generic: RVM: single user installation (not root)
229
+
230
+ Use this method which provides more flexibility.
138
231
 
232
+ Install "rvm": follow [https://rvm.io/](https://rvm.io/) :
139
233
 
140
- MacOS 10.13+ (High Sierra) comes with a recent Ruby, so you can use it directly, you will need to install <%= gemspec.name %> using `sudo` :
234
+ Install the 2 keys
141
235
 
142
236
  ```
143
- $ sudo gem install <%= gemspec.name %><%=geminstadd%>
237
+ $ gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
144
238
  ```
145
239
 
146
- Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
240
+ Execute the shell/curl command. As regular user, it install in the user's home: `~/.rvm` .
147
241
 
148
242
  ```
149
- $ brew install ruby
243
+ $ \curl -sSL https://get.rvm.io | bash -s stable
150
244
  ```
151
245
 
152
- ### Windows
246
+ If you keep the same terminal (ont needed if re-login):
153
247
 
154
- Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/).
248
+ ```
249
+ $ source ~/.rvm/scripts/rvm
250
+ ```
155
251
 
156
- Go to "Downloads".
252
+ It is advised to get one of the pre-compiled ruby version, you can list with:
157
253
 
158
- Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
254
+ ```
255
+ $ rvm list --remote
256
+ ```
159
257
 
160
- At the end of the installer uncheck the box to skip the installation of "MSys2".
258
+ Install the chosen pre-compiled Ruby version:
161
259
 
162
- ### Linux
260
+ ```
261
+ $ rvm install 2.7.2 --binary
262
+ ```
263
+
264
+ Ruby is now installed for the user, go on to Gem installation.
265
+
266
+ ### Generic: RVM: global installation (as root)
267
+
268
+ Follow the same method as single user install, but execute as "root".
163
269
 
164
- Install Latest Ruby 2 using "rvm" [https://rvm.io/](https://rvm.io/) .
165
- It installs by default in /usr/local/rvm , but you can install in another location:
270
+ As root, it installs by default in /usr/local/rvm for all users and creates `/etc/profile.d/rvm.sh`.
271
+ One can install in another location with :
166
272
 
167
273
  ```
168
- curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
274
+ # curl -sSL https://get.rvm.io | bash -s -- --path /usr/local
169
275
  ```
170
276
 
171
- Once installed, you can install latest ruby:
277
+ As root, make sure this will not collide with other application using Ruby (e.g. Faspex).
278
+ If so, one can rename the login script: `mv /etc/profile.d/rvm.sh /etc/profile.d/rvm.sh.ok`.
279
+ To activate ruby (and <%=cmd%>) later, source it:
172
280
 
173
281
  ```
174
- # rvm install ruby
282
+ # source /etc/profile.d/rvm.sh.ok
283
+ # rvm version
175
284
  ```
176
285
 
177
- If you dont want all users to have ruby by default,
178
- rename the file: `/etc/profile.d/rvm.sh` with another extension, and source it to get rvm.
286
+ ### Windows: Installer
287
+
288
+ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
289
+
290
+ * Go to "Downloads".
291
+ * Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
292
+ * At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
293
+
294
+ ### macOS: pre-installed or `brew`
179
295
 
180
- Alternatively, only if you know what you do, on RPM based systems (CentOs, Redhat), install the ruby provided by yum which may be 2.0.
296
+ MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%= gemspec.name %> using `sudo` :
181
297
 
182
298
  ```
183
- # yum install -y ruby rubygems ruby-json
299
+ $ sudo gem install <%= gemspec.name %><%=geminstadd%>
184
300
  ```
185
301
 
186
- If necessary, pre-install jwt for older Ruby < 2.1.
302
+ Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
187
303
 
188
304
  ```
189
- # gem install jwt -v 1.5.6
305
+ $ brew install ruby
190
306
  ```
191
307
 
192
- You can cleanup your whole yum-installed ruby environment like this to uninstall:
308
+ ### Linux: package
309
+
310
+ If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
311
+
312
+ Example:
313
+
314
+ ```
315
+ # yum install -y ruby rubygems ruby-json
316
+ ```
317
+
318
+ One can cleanup the whole yum-installed ruby environment like this to uninstall:
193
319
 
194
320
  ```
195
321
  gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
@@ -198,7 +324,7 @@ yum remove -y ruby ruby-libs
198
324
 
199
325
  ### Other Unixes: Aix, etc...
200
326
 
201
- If your unix do not provide a pre-built ruby, you can get using one of those
327
+ If your unix do not provide a pre-built ruby, you can get it using one of those
202
328
  [methods](https://www.ruby-lang.org/en/documentation/installation/)
203
329
 
204
330
  For instance to build from source, and install in `/opt/ruby` :
@@ -214,6 +340,50 @@ For instance to build from source, and install in `/opt/ruby` :
214
340
  # make install
215
341
  ```
216
342
 
343
+ ### <a name="offline_install"></a>Installation without internet access
344
+
345
+ Note that currently no pre-packaged version exist yet.
346
+ A method to build one provided here:
347
+
348
+ On a server with the same OS version and with internet access follow the "Generic single user installation" method.
349
+
350
+ Then create an archive:
351
+
352
+ ```
353
+ $ cd
354
+ $ tar zcvf rvm-<%=cmd%>.tgz .rvm
355
+ ```
356
+
357
+ Get the Aspera SDK. Execute:
358
+
359
+ ```
360
+ $ <%=cmd%> conf --show-config|grep sdk_url
361
+ ```
362
+
363
+ Then download the SDK archive from that URL.
364
+
365
+ Another method for the SDK is to install the SDK (`<%=cmd%> conf ascp install`) on the first system, and archive `$HOME/.aspera`.
366
+
367
+ Transfer those 2 archives to the target system without internet access.
368
+
369
+ On the target system:
370
+
371
+ * Extract the RVM archive either in a global location, or in a user's home folder : `path_to_rvm_root`
372
+ * in the user's `.profile` add this line: (replace `path_to_rvm_root` with the actual location)
373
+
374
+ ```
375
+ source path_to_rvm_root/scripts/rvm
376
+ rvm use 2.7.2
377
+ ```
378
+
379
+ For the SDK, either install from archive:
380
+
381
+ ```
382
+ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
383
+ ```
384
+
385
+ or restore the `$HOME/.aspera` folder for the user.
386
+
217
387
  ## <a name="the_gem"></a>`<%= gemspec.name %>` gem
218
388
 
219
389
  Once you have Ruby and rights to install gems: Install the gem and its dependencies:
@@ -228,21 +398,40 @@ To upgrade to the latest version:
228
398
  # gem update <%= gemspec.name %>
229
399
  ```
230
400
 
401
+ <%=tool%> checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
402
+
403
+ To check manually:
404
+
405
+ ```
406
+ # <%=cmd%> conf check_update
407
+ ```
408
+
409
+
410
+
231
411
  ## <a name="fasp_prot"></a>FASP Protocol
232
412
 
233
- Most file transfers will be done using the FASP protocol. Only two additional files are required to perform
234
- an Aspera Transfer:
413
+ Most file transfers will be done using the FASP protocol, using `ascp`.
414
+ Only two additional files are required to perform an Aspera Transfer, which are part of Aspera SDK:
235
415
 
236
416
  * ascp
237
417
  * aspera-license (in same folder, or ../etc)
238
418
 
239
- This can be installed directly with
419
+ This can be installed either be installing an Aspera transfer sofware, or using an embedded command:
240
420
 
241
421
  ```
242
422
  $ <%=cmd%> conf ascp install
243
423
  ```
244
424
 
245
- Those can be found in one of IBM Aspera transfer server or client with its license file (some are free):
425
+ If a local SDK installation is prefered instead of fetching from internet: one can specify the location of the SDK file:
426
+
427
+ ```
428
+ $ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
429
+ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
430
+ ```
431
+
432
+ The format is: `file:///<path>`, where `<path>` can be either a relative path (not starting with `/`), or an absolute path.
433
+
434
+ If the embedded method is not used, the following packages are also suitable:
246
435
 
247
436
  * IBM Aspera Connect Client (Free)
248
437
  * IBM Aspera Desktop Client (Free)
@@ -251,7 +440,7 @@ Those can be found in one of IBM Aspera transfer server or client with its licen
251
440
  * IBM Aspera High Speed Transfer EndPoint (Licensed)
252
441
 
253
442
  For instance, Aspera Connect Client can be installed
254
- by visiting the page: [http://downloads.asperasoft.com/connect2/](http://downloads.asperasoft.com/connect2/).
443
+ by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/aspera/connect/).
255
444
 
256
445
  <%=tool%> will detect most of Aspera transfer products in standard locations and use the first one found.
257
446
  Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
@@ -259,6 +448,33 @@ Refer to section [FASP](#client) for details on how to select a client or set pa
259
448
  Several methods are provided on how to start a transfer. Use of a local client is one of them, but
260
449
  other methods are available. Refer to section: [Transfer Agents](#agents)
261
450
 
451
+ ## <a name="offline_install"></a>Offline Installation (without internet)
452
+
453
+ The procedure consists in:
454
+
455
+ * Follow the non-root installation procedure with RVM, including gem
456
+ * archive (zip, tar) the main RVM folder (includes <%=cmd%>):
457
+
458
+ ```
459
+ $ cd ~
460
+ $ tar zcvf rvm_<%=cmd%>.tgz .rvm
461
+ ```
462
+
463
+ * retrieve the SDK:
464
+
465
+ ```
466
+ $ curl -Lso SDK.zip https://ibm.biz/aspera_sdk
467
+ ```
468
+
469
+ * on the system without internet access:
470
+
471
+ ```
472
+ $ cd ~
473
+ $ tar zxvf rvm_<%=cmd%>.tgz
474
+ $ source ~/.rvm/scripts/rvm
475
+ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
476
+ ```
477
+
262
478
  # <a name="cli"></a>Command Line Interface: <%=tool%>
263
479
 
264
480
  The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
@@ -439,6 +655,7 @@ By default, a table output will display one line per entry, and columns for each
439
655
  * a,b,c : the list of attributes specified by the comma separated list
440
656
  * Array extended value: for instance, @json:'["a","b","c"]' same as above
441
657
  * +a,b,c : add selected properties to the default selection.
658
+ * -a,b,c : remove selected properties from the default selection.
442
659
 
443
660
  ## <a name="extended"></a>Extended Value Syntax
444
661
 
@@ -452,25 +669,25 @@ The extended value syntax is:
452
669
 
453
670
  The difference between reader and decoder is order and ordinality. Both act like a function of value on right hand side. Decoders are at the beginning of the value, followed by a single optional reader, followed by the optional value.
454
671
 
455
- The following "readers" are supported:
672
+ The following "readers" are supported (returns value in []):
456
673
 
457
- * @val:VALUE , prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
458
- * @file:PATH , read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
459
- * @path:PATH , performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
460
- * @env:ENVVAR , read from a named env var, e.g.--password=@env:MYPASSVAR
461
- * @stdin: , read from stdin (no value on right)
462
- * @preset:NAME , get whole <%=opprst%> value by name
674
+ * @val:VALUE : [String] prevent further special prefix processing, e.g. `--username=@val:laurent` sets the option `username` to value `laurent`.
675
+ * @file:PATH : [String] read value from a file (prefix "~/" is replaced with the users home folder), e.g. --key=@file:~/.ssh/mykey
676
+ * @path:PATH : [String] performs path expansion (prefix "~/" is replaced with the users home folder), e.g. --config-file=@path:~/sample_config.yml
677
+ * @env:ENVVAR : [String] read from a named env var, e.g.--password=@env:MYPASSVAR
678
+ * @stdin: : [String] read from stdin (no value on right)
679
+ * @preset:NAME : [Hash] get whole <%=opprst%> value by name
463
680
 
464
681
  In addition it is possible to decode a value, using one or multiple decoders :
465
682
 
466
- * @base64: decode a base64 encoded string
467
- * @json: decode JSON values (convenient to provide complex structures)
468
- * @zlib: uncompress data
469
- * @ruby: execute ruby code
470
- * @csvt: decode a titled CSV value
471
- * @lines: split a string in multiple lines and return an array
472
- * @list: split a string in multiple items taking first character as separator and return an array
473
- * @incps: include values of presets specified by key include_presets in hash
683
+ * @base64: [String] decode a base64 encoded string
684
+ * @json: [any] decode JSON values (convenient to provide complex structures)
685
+ * @zlib: [String] uncompress data
686
+ * @ruby: [any] execute ruby code
687
+ * @csvt: [Array] decode a titled CSV value
688
+ * @lines: [Array] split a string in multiple lines and return an array
689
+ * @list: [Array] split a string in multiple items taking first character as separator and return an array
690
+ * @incps: [Hash] include values of presets specified by key `incps` in input hash
474
691
 
475
692
  To display the result of an extended value, use the `config echo` command.
476
693
 
@@ -502,7 +719,7 @@ $ <%=cmd%> config echo @csvt:@file:test.csv
502
719
  :......:.....................:
503
720
  ```
504
721
 
505
- Example: create a hash and include values from preset named "config" of config file
722
+ Example: create a hash and include values from preset named "config" of config file in this hash
506
723
 
507
724
  ```
508
725
  $ <%=cmd%> config echo @incps:@json:'{"hello":true,"incps":["config"]}'
@@ -541,9 +758,9 @@ It can be overriden using the envinonment variable `<%=evp%>HOME`.
541
758
  Example (Windows):
542
759
 
543
760
  ```
544
- $ set <%=evp%>HOME=C:\Users\Kenji\.aspera\ascli
761
+ $ set <%=evp%>HOME=C:\Users\Kenji\.aspera\<%=cmd%>
545
762
  $ <%=cmd%> config folder
546
- C:\Users\Kenji\.aspera\ascli
763
+ C:\Users\Kenji\.aspera\<%=cmd%>
547
764
  ```
548
765
 
549
766
  ## <a name="configfile"></a>Configuration file
@@ -578,7 +795,7 @@ $ <%=cmd%> config id <<%=opprst%>> set|delete|show|initialize|update
578
795
  The command `update` allows the easy creation of <%=prst%> by simply providing the options in their command line format, e.g. :
579
796
 
580
797
  ```
581
- $ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera --ts=@json:'{"precalculate_job_size":true}'
798
+ $ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_ --ts=@json:'{"precalculate_job_size":true}'
582
799
  ```
583
800
 
584
801
  * This creates a <%=prst%> `demo_server` with all provided options.
@@ -586,13 +803,13 @@ $ <%=cmd%> config id demo_server update --url=ssh://demo.asperasoft.com:33001 --
586
803
  The command `set` allows setting individual options in a <%=prst%>.
587
804
 
588
805
  ```
589
- $ <%=cmd%> config id demo_server set password demoaspera
806
+ $ <%=cmd%> config id demo_server set password _demo_pass_
590
807
  ```
591
808
 
592
809
  The command `initialize`, like `update` allows to set several parameters at once, but it deletes an existing configuration instead of updating it, and expects a _[Structured Value](#native)_.
593
810
 
594
811
  ```
595
- $ <%=cmd%> config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"demoaspera","ts":{"precalculate_job_size":true}}'
812
+ $ <%=cmd%> config id demo_server initialize @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"_demo_pass_","ts":{"precalculate_job_size":true}}'
596
813
  ```
597
814
 
598
815
  A good practice is to not manually edit the configuration file and use modification commands instead.
@@ -629,11 +846,19 @@ Note that special plugin name: `config` can be associated with a preset that is
629
846
  Operations on this preset are done using regular `config` operations:
630
847
 
631
848
  ```
632
- $ <%=cmd%> config id default set _plugin_name_ _defauklt_preset_for_plugin_
849
+ $ <%=cmd%> config id default set _plugin_name_ _default_preset_for_plugin_
633
850
  $ <%=cmd%> config id default get _plugin_name_
634
- "_defauklt_preset_for_plugin_"
851
+ "_default_preset_for_plugin_"
635
852
  ```
636
853
 
854
+ ### <a name="lprtdef"></a>Special Plugin: config
855
+
856
+ Plugin `config` (not to be confused with <%=prstt%> config) is used to configure <%=tool%> but it also contains global options.
857
+
858
+ When <%=tool%> starts, it lookjs for the `default` <%=prstt%> and if there is a value for `config`, if so, it loads the option values for any plugin used.
859
+
860
+ If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
861
+
637
862
  ### Format of file
638
863
 
639
864
  The configuration file is a hash in a YAML file. Example:
@@ -649,7 +874,7 @@ cli_default:
649
874
  demo_server:
650
875
  url: ssh://demo.asperasoft.com:33001
651
876
  username: asperaweb
652
- password: demoaspera
877
+ password: _demo_pass_
653
878
  ```
654
879
 
655
880
  We can see here:
@@ -718,7 +943,6 @@ A <%=prst%> value can be removed with `unset`:
718
943
  $ <%=cmd%> config id cli_default unset interactive
719
944
  ```
720
945
 
721
-
722
946
  ### Examples
723
947
 
724
948
  For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
@@ -838,21 +1062,42 @@ The `config` plugin also allows specification for the use of a local FASP client
838
1062
 
839
1063
  ```
840
1064
  $ <%=cmd%> config ascp show
841
- /Users/laurent/Applications/Aspera Connect.app/Contents/Resources/ascp
1065
+ /Users/laurent/.aspera/ascli/sdk/ascp
1066
+ $ <%=cmd%> config ascp info
1067
+ +--------------------+-----------------------------------------------------------+
1068
+ | key | value |
1069
+ +--------------------+-----------------------------------------------------------+
1070
+ | ascp | /Users/laurent/.aspera/ascli/sdk/ascp |
1071
+ ...
842
1072
  ```
843
1073
 
844
1074
  ### Selection of local `ascp`
845
1075
 
1076
+ By default, <%=tool%> uses any found local product with ascp, including SDK.
1077
+
846
1078
  To temporarily use an alternate ascp path use option `ascp_path` (`--ascp-path=`)
847
1079
 
848
- To permanently use another ascp:
1080
+ For a permanent change, the command `config ascp use` sets the same parameter for the global default.
1081
+
1082
+ Using a POSIX shell:
849
1083
 
850
1084
  ```
851
1085
  $ <%=cmd%> config ascp use '/Users/laurent/Applications/Aspera CLI/bin/ascp'
852
- saved to default global preset /Users/laurent/Applications/Aspera CLI/bin/ascp
1086
+ ascp version: 4.0.0.182279
1087
+ Updated: global_common_defaults: ascp_path <- /Users/laurent/Applications/Aspera CLI/bin/ascp
1088
+ Saved to default global preset global_common_defaults
1089
+ ```
1090
+
1091
+ Windows:
1092
+
1093
+ ```
1094
+ $ <%=cmd%> config ascp use C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1095
+ ascp version: 4.0.0.182279
1096
+ Updated: global_common_defaults: ascp_path <- C:\Users\admin\.aspera\ascli\sdk\ascp.exe
1097
+ Saved to default global preset global_common_defaults
853
1098
  ```
854
1099
 
855
- This sets up a global default.
1100
+ If the path has spaces, read section: [Shell and Command line parsing](#parsing).
856
1101
 
857
1102
  ### List locally installed Aspera Transfer products
858
1103
 
@@ -939,7 +1184,7 @@ will effectively push files to the related server from the agent node.
939
1184
 
940
1185
  ### <a name="direct"></a>Direct (local ascp using FASPManager API)
941
1186
 
942
- By default the CLI will use a local FASP protocol, equivalent to specifying `--transfer=direct`.
1187
+ By default <%=tool%> uses a local ascp, equivalent to specifying `--transfer=direct`.
943
1188
  <%=tool%> will detect locally installed Aspera products.
944
1189
  Refer to section [FASP](#client).
945
1190
 
@@ -949,17 +1194,33 @@ To specify a FASP proxy (only supported with the `direct` agent), set the approp
949
1194
  * `EX_http_proxy_url` (proxy for legacy http fallback)
950
1195
  * `EX_ascp_args`
951
1196
 
952
- The `transfer-info` optionally provides the following auto resume parameters:
1197
+ The `transfer-info` accepts the following optional parameters:
953
1198
 
954
1199
  <table>
955
- <tr><th>Name</th><th>Default</th><th>Feature</th><th>Description</th></tr>
956
- <tr><td>iter_max</td>. <td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
957
- <tr><td>sleep_initial</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
958
- <tr><td>sleep_factor</td> <td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
959
- <tr><td>sleep_max</td>. <td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
960
- <tr><td>wss</td> <td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
1200
+ <tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
1201
+ <tr><td>spawn_timeout_sec</td><td>Float</td><td>3</td><td>Multi session</td><td>Verification time that ascp is running</td></tr>
1202
+ <tr><td>spawn_delay_sec</td><td>Float</td><td>2</td><td>Multi session</td><td>Delay between startup of sessions</td></tr>
1203
+ <tr><td>wss</td><td>Bool</td><td>false</td><td>Web Socket Session</td><td>Enable use of web socket session in case it is available</td></tr>
1204
+ <tr><td>resume</td><td>Hash</td><td>nil</td><td>Resumer parameters</td><td>See below</td></tr>
961
1205
  </table>
962
1206
 
1207
+ Resume parameters:
1208
+
1209
+ <table>
1210
+ <tr><th>Name</th><th>Type</th><th>Default</th><th>Feature</th><th>Description</th></tr>
1211
+ <tr><td>iter_max</td><td>int</td><td>7</td><td>Resume</td><td>Max number of retry on error</td></tr>
1212
+ <tr><td>sleep_initial</td><td>int</td><td>2</td><td>Resume</td><td>First Sleep before retry</td></tr>
1213
+ <tr><td>sleep_factor</td><td>int</td><td>2</td><td>Resume</td><td>Multiplier of Sleep</td></tr>
1214
+ <tr><td>sleep_max</td><td>int</td><td>60</td><td>Resume</td><td>Maximum sleep</td></tr>
1215
+ </table>
1216
+
1217
+ Examples:
1218
+
1219
+ ```
1220
+ $ <%=cmd%> ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1221
+ $ <%=cmd%> ... --transfer-info=@json:'{"spawn_delay_sec":2.5}'
1222
+ ```
1223
+
963
1224
  ### IBM Aspera Connect Client GUI
964
1225
 
965
1226
  By specifying option: `--transfer=connect`, <%=tool%> will start transfers
@@ -991,9 +1252,11 @@ If it possible to send using a HTTP gateway, in case FASP is not allowed.
991
1252
  Example:
992
1253
 
993
1254
  ```
994
- $ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://eudemo.asperademo.com:9443/aspera/http-gwy/v1"}'
1255
+ $ <%=cmd%> faspex package recv --id=323 --transfer=httpgw --transfer-info=@json:'{"url":"https://asperagw.example.com:9443/aspera/http-gwy/v1"}'
995
1256
  ```
996
1257
 
1258
+ Note that the gateway only supports transfers authorized with a token.
1259
+
997
1260
  ## <a name="transferspec"></a>Transfer Specification
998
1261
 
999
1262
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
@@ -1007,7 +1270,7 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
1007
1270
 
1008
1271
  <%=tool%> builds a default _transfer-spec_ internally, so it is not necessary to provide additional parameters on the command line for this transfer.
1009
1272
 
1010
- If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters.
1273
+ If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
1011
1274
 
1012
1275
  It is possible to specify ascp options when the `transfer` option is set to `direct` using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
1013
1276
 
@@ -1040,8 +1303,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
1040
1303
 
1041
1304
  ### List of files for transfers
1042
1305
 
1043
- When uploading, downloading or sending files, the user must specify
1044
- the list of files to transfer. Most of the time, the list of files to transfer will be simply specified on the command line:
1306
+ When uploading, downloading or sending files, the user must specify the list of files to transfer.
1307
+ Most of the time, the list of files to transfer will be simply specified on the command line:
1045
1308
 
1046
1309
  ```
1047
1310
  $ <%=cmd%> server upload ~/mysample.file secondfile
@@ -1055,7 +1318,8 @@ $ <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
1055
1318
 
1056
1319
  More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
1057
1320
 
1058
- Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is: all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
1321
+ Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
1322
+ all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
1059
1323
 
1060
1324
  For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
1061
1325
 
@@ -1298,7 +1562,7 @@ updated: my_aoc_org
1298
1562
  Define this <%=prst%> as default configuration for the `aspera` plugin:
1299
1563
 
1300
1564
  ```
1301
- $ <%=cmd%> config id default set aspera my_aoc_org
1565
+ $ <%=cmd%> config id default set aoc my_aoc_org
1302
1566
  ```
1303
1567
 
1304
1568
  Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
@@ -1672,7 +1936,7 @@ Then, create two shared folders located in two regions, in your files home, in a
1672
1936
  Then, transfer between those:
1673
1937
 
1674
1938
  ```
1675
- $ <%=cmd%> -Paoc_show aspera files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
1939
+ $ <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folder='AWS Singapore' 100GB.file --ts=@json:'{"target_rate_kbps":"1000000","multi_session":10,"multi_session_threshold":1}'
1676
1940
  ```
1677
1941
 
1678
1942
  * create registration key to register a node
@@ -1735,8 +1999,9 @@ Notes:
1735
1999
  Examples:
1736
2000
 
1737
2001
  ```
1738
- $ <%=cmd%> aoc packages send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
1739
- $ <%=cmd%> aoc packages send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
2002
+ $ <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
2003
+ $ <%=cmd%> aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
2004
+ $ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
1740
2005
  ```
1741
2006
 
1742
2007
  ## <a name="aoccargo"></a>Receive new packages only
@@ -1751,10 +2016,10 @@ $ <%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
1751
2016
  * `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
1752
2017
  * `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
1753
2018
 
1754
- Typically, one would regularly execute this command on a regular basis, using the method oif your choice:
2019
+ Typically, one would regularly execute this command on a regular basis, using the method of your choice:
1755
2020
 
1756
- * Windows scheduler
1757
- * cron
2021
+ * Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
2022
+ * Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
1758
2023
  * etc...
1759
2024
 
1760
2025
  ## Download Files
@@ -1974,7 +2239,7 @@ This can also be set as default using a preset
1974
2239
  One can test the "server" application using the well known demo server:
1975
2240
 
1976
2241
  ```
1977
- $ <%=cmd%> config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera
2242
+ $ <%=cmd%> config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=_demo_pass_
1978
2243
  $ <%=cmd%> config id default set server aspera_demo_server
1979
2244
  $ <%=cmd%> server browse /aspera-test-dir-large
1980
2245
  $ <%=cmd%> server download /aspera-test-dir-large/200MB
@@ -2070,25 +2335,80 @@ to download files.
2070
2335
  $ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
2071
2336
  ```
2072
2337
 
2073
- # Plugin: IBM Aspera Faspex
2338
+ # Plugin: IBM Aspera Faspex5
2074
2339
 
2075
- Note that the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
2340
+ 3 authentication methods are supported:
2076
2341
 
2077
- ## Sending a Package
2342
+ * boot
2343
+ * web
2344
+ * jwt
2345
+
2346
+ For boot method:
2078
2347
 
2079
- Provide delivery info in JSON, example:
2348
+ * open a browser
2349
+ * start developer mode
2350
+ * login to faspex 5
2351
+ * find the first API call with `Authorization` token, and copy it (kind of base64 long string)
2352
+
2353
+ Use it as password and use `--auth=boot`.
2080
2354
 
2081
2355
  ```
2082
- --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}'
2356
+ $ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
2083
2357
  ```
2084
2358
 
2085
- a note can be added: `"note":"Please ..."`
2359
+ For web method, create an API client in Faspex, and use: --auth=web
2360
+
2361
+ For JWT, create an API client in Faspex with jwt supporot, and use: --auth=jwt
2362
+ as of beta£3 this does not allow regular users.
2086
2363
 
2087
- metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
2364
+ Ready to use Faspex5 with CLI.
2088
2365
 
2366
+ Once the graphical registration form exist, ther bootstrap method can be removed.
2367
+
2368
+ # Plugin: IBM Aspera Faspex (4.x)
2369
+
2370
+ Notes:
2089
2371
 
2090
- Note for full details, refer to:
2091
- [Reference on Developer Site](https://developer.asperasoft.com/web/faspex/sending)
2372
+ * the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
2373
+ * for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
2374
+
2375
+ ## Receiving a Package
2376
+
2377
+ The command is `package recv`, possible methosd are:
2378
+
2379
+ * provide a package id with option `id`
2380
+ * provide a public link with option `link`
2381
+ * provide a `faspe:` URI with option `link`
2382
+
2383
+ ```
2384
+ $ <%=cmd%> faspex package recv --id=12345
2385
+ $ <%=cmd%> faspex package recv --link=faspe://...
2386
+ ```
2387
+
2388
+ If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
2389
+
2390
+ ```
2391
+ $ <%=cmd%> faspex package list --recipient='*thedropboxname'
2392
+ ```
2393
+
2394
+
2395
+
2396
+ ## Sending a Package
2397
+
2398
+ The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
2399
+
2400
+ Example:
2401
+
2402
+ ```
2403
+ $ <%=cmd%> faspex package send --delivery-info=@json:'{"title":"my title","recipients":["laurent.martin.aspera@fr.ibm.com"]}' --url=https://faspex.corp.com/aspera/faspex --username=foo --password=bar /tmp/file1 /home/bar/file2
2404
+ ```
2405
+
2406
+ If the recipient is a dropbox, just provide the name of the dropbox in `recipients`: `"recipients":["My Dropbox Name"]`
2407
+
2408
+ Additional optional parameters in `delivery_info`:
2409
+
2410
+ * Package Note: : `"note":"note this and that"`
2411
+ * Package Metadata: `"metadata":{"Meta1":"Val1","Meta2":"Val2"}`
2092
2412
 
2093
2413
  ## operation on dropboxes
2094
2414
 
@@ -2151,6 +2471,8 @@ $ for p in 1 2 3;do <%=cmd%> shares2 admin users list --value=@json:'{"page":'$p
2151
2471
  # Plugin: IBM Cloud Object Storage
2152
2472
 
2153
2473
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
2474
+ It uses the same transfer service as Aspera on Cloud.
2475
+ see [https://status.aspera.io](https://status.aspera.io)
2154
2476
 
2155
2477
  Required options are either:
2156
2478
 
@@ -2207,7 +2529,14 @@ Endpoints for regions can be found by querying the `endpoints` URL.
2207
2529
  For convenience, let us create a default configuration, for example:
2208
2530
 
2209
2531
  ```
2210
- $ <%=cmd%> conf id mycos update --service-credentials=@val:@json:@file:$HOME/service_creds.json --region=us-south --bucket=laurent
2532
+ $ <%=cmd%> conf id mycos update --bucket=laurent --service-credentials=@val:@json:@file:~/service_creds.json --region=us-south
2533
+ $ <%=cmd%> conf id default set cos mycos
2534
+ ```
2535
+
2536
+ or using direct parameters:
2537
+
2538
+ ```
2539
+ $ <%=cmd%> conf id mycos update --bucket=mybucket --endpoint=https://s3.us-east.cloud-object-storage.appdomain.cloud --apikey=abcdefgh --crn=crn:v1:bluemix:public:iam-identity::a/xxxxxxx
2211
2540
  $ <%=cmd%> conf id default set cos mycos
2212
2541
  ```
2213
2542
 
@@ -2225,7 +2554,7 @@ to start from ma configuration file, using <%=tool%> standard options.
2225
2554
 
2226
2555
  # Plugin: Preview
2227
2556
 
2228
- The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on an Aspera HSTS for use primarily in the Aspera on Cloud application.
2557
+ The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
2229
2558
  This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
2230
2559
  Several parameters can be used to tune several aspects:
2231
2560
 
@@ -2273,6 +2602,7 @@ The tool requires the following external tools available in the `PATH`:
2273
2602
  * OptiPNG : `optipng`
2274
2603
  * FFmpeg : `ffmpeg` `ffprobe`
2275
2604
  * Libreoffice : `libreoffice`
2605
+ * ruby gem `mimemagic`
2276
2606
 
2277
2607
  Here shown on Redhat/CentOS.
2278
2608
 
@@ -2284,6 +2614,26 @@ To check if all tools are found properly, execute:
2284
2614
  $ <%=cmd%> preview check
2285
2615
  ```
2286
2616
 
2617
+ ### mimemagic
2618
+
2619
+ To benefit from extra mime type detection install gem mimemagic:
2620
+
2621
+ ```
2622
+ # gem install mimemagic
2623
+ ```
2624
+
2625
+ or to install an earlier version if any problem:
2626
+
2627
+ ```
2628
+ # gem install mimemagic -v '~> 0.3.0'
2629
+ ```
2630
+
2631
+ To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
2632
+
2633
+ If not used, Mime type used for conversion is the one provided by the node API.
2634
+
2635
+ If used, it the `preview` command will first analyse the file content using mimemagic, and if no match, will try by extension.
2636
+
2287
2637
  ### Image: Imagemagick and optipng
2288
2638
 
2289
2639
  ```
@@ -2308,90 +2658,111 @@ The generation of preview in based on the use of `unoconv` and `libreoffice`
2308
2658
  # dnf install unoconv
2309
2659
  ```
2310
2660
 
2661
+ * Amazon Linux
2662
+
2663
+ ```
2664
+ # amazon-linux-extras enable libreoffice
2665
+ # yum clean metadata
2666
+ # yum install libreoffice-core libreoffice-calc libreoffice-opensymbol-fonts libreoffice-ure libreoffice-writer libreoffice-pyuno libreoffice-impress
2667
+ # wget https://raw.githubusercontent.com/unoconv/unoconv/master/unoconv
2668
+ # mv unoconv /usr/bin
2669
+ # chmod a+x /usr/bin/unoconv
2670
+ ```
2311
2671
 
2312
2672
  ## Configuration
2313
2673
 
2314
- Like any <%=tool%> commands, parameters can be passed on command line or using a configuration <%=prst%>. Note that if you use the <%=tool%> run as `xfer` user, like here, the configuration file must be created as the same user. Example using a <%=prst%> named `my_preset_name` (choose any name relevant to you, e.g. the AoC node name, and replace in the following lines):
2674
+ The preview generator is run as a user, preferably a regular user (not root). When using object storage, any user can be used, but when using local storage it is usually better to use the user `xfer`, as uploaded files are under this identity: this ensures proper access rights. (we will assume this)
2675
+
2676
+ Like any <%=tool%> commands, parameters can be passed on command line or using a configuration <%=prst%>. The configuration file must be created with the same user used to run so that it is properly used on runtime.
2677
+
2678
+ Note that the `xfer` user has a special protected shell: `aspshell`, so changing identity requires specification of alternate shell:
2315
2679
 
2316
2680
  ```
2317
2681
  # su -s /bin/bash - xfer
2318
- $ <%=cmd%> config id my_preset_name update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
2319
- $ <%=cmd%> config id default set preview my_preset_name
2682
+ $ <%=cmd%> config id previewconf update --url=https://localhost:9092 --username=my_access_key --password=my_secret --skip-types=office --lock-port=12346
2683
+ $ <%=cmd%> config id default set preview previewconf
2320
2684
  ```
2321
2685
 
2322
- Here we assume that Office file generation is disabled, else remove the option. For the `lock_port` option refer to a previous section in thsi manual.
2686
+ Here we assume that Office file generation is disabled, else remove this option.
2687
+ `lock_port` prevents concurrent execution of generation when using a scheduler.
2323
2688
 
2324
- Once can check if the access key is well configured using:
2689
+ One can check if the access key is well configured using:
2325
2690
 
2326
2691
  ```
2327
- $ <%=cmd%> -Pmy_preset_name node browse /
2692
+ $ <%=cmd%> -Ppreviewconf node browse /
2328
2693
  ```
2329
2694
 
2330
2695
  This shall list the contents of the storage root of the access key.
2331
2696
 
2332
2697
  ## Execution
2333
2698
 
2334
- The tool intentionally supports only a "one shot" mode in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
2335
- It needs to be run regularly to create or update preview files. For that use your best
2699
+ The tool intentionally supports only a "one shot" mode (no infinite loop) in order to avoid having a hanging process or using too many resources (calling REST api too quickly during the scan or event method).
2700
+ It needs to be run on a regular basis to create or update preview files. For that use your best
2336
2701
  reliable scheduler. For instance use "CRON" on Linux or Task Scheduler on Windows.
2337
2702
 
2338
- Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consiustent have generate the appropriate access rights, the generation process
2339
- should be run as user `xfer`.
2703
+ Typically, for "Access key" access, the system/transfer is `xfer`. So, in order to be consistent have generate the appropriate access rights, the generation process should be run as user `xfer`.
2340
2704
 
2341
2705
  Lets do a one shot test, using the configuration previously created:
2342
2706
 
2343
2707
  ```
2344
2708
  # su -s /bin/bash - xfer
2345
- $ <%=cmd%> preview scan --overwrite=always
2709
+ xfer$ <%=cmd%> preview scan --overwrite=always
2346
2710
  ```
2347
2711
 
2348
2712
  When the preview generator is first executed it will create a file: `.aspera_access_key`
2349
- which contains the access key used.
2713
+ in the previews folder which contains the access key used.
2350
2714
  On subsequent run it reads this file and check that previews are generated for the same access key, else it fails. This is to prevent clash of different access keys using the same root.
2351
2715
 
2352
2716
  ## Configuration for Execution in scheduler
2353
2717
 
2354
- Here is an example of configuration for use with cron on Linux. Adapt the scripts to your own preference.
2718
+ Here is an example of configuration for use with cron on Linux.
2719
+ Adapt the scripts to your own preference.
2355
2720
 
2356
2721
  We assume here that a configuration preset was created as shown previously.
2357
2722
 
2358
- Here the cronjob is created for `root`, and changes the user to `xfer`, also overriding the shell which should be `aspshell`. (adapt the command below, as it would override existing crontab). It is also up to you to use directly the `xfer` user's crontab. This is an example only.
2723
+ Lets first setup a script that will be used in the sceduler and sets up the environment.
2724
+
2725
+ Example of startup script `cron_<%=cmd%>`, which sets the Ruby environment and adds some timeout protection:
2726
+
2727
+ ```
2728
+ #!/bin/bash
2729
+ # set a timeout protection, just in case
2730
+ case "$*" in *trev*) tmout=10m ;; *) tmout=30m ;; esac
2731
+ . /etc/profile.d/rvm.sh
2732
+ rvm use 2.6 --quiet
2733
+ exec timeout ${tmout} <%=cmd%> "${@}"
2734
+ ```
2735
+
2736
+ Here the cronjob is created for user `xfer`.
2359
2737
 
2360
2738
  ```
2361
- # crontab<<EOF
2362
- 2-59 * * * * su -s /bin/bash - xfer -c 'nice +10 timeout 10m <%=cmd%> preview event --log-level=info --logger=syslog --iteration-file=/tmp/preview_restart.txt'
2363
- 0 * * * * su -s /bin/bash - xfer -c 'nice +10 timeout 30m <%=cmd%> preview scan --log-level=info --logger=syslog'
2739
+ xfer$ crontab<<EOF
2740
+ 0 * * * * /home/xfer/cron_<%=cmd%> preview scan --logger=syslog --display=error
2741
+ 2-59 * * * * /home/xfer/cron_<%=cmd%> preview trev --logger=syslog --display=error
2364
2742
  EOF
2365
2743
  ```
2366
2744
 
2367
- Nopte that the options here may be located in the config preset, but it was left on the command line to keep stdout for command line execution of preview.
2745
+ Note that the loging options are kept in the cronfile instead of conf file to allow execution on command line with output on command line.
2368
2746
 
2369
2747
  ## Candidate detection for creation or update (or deletion)
2370
2748
 
2371
- The tool will find candidates for preview generation using three commands:
2749
+ The tool generates preview files using those commands:
2372
2750
 
2373
2751
  * `trevents` : only recently uploaded files will be tested (transfer events)
2374
2752
  * `events` : only recently uploaded files will be tested (file events: not working)
2375
- * `scan` : deeply scan all files under the access key&apos;s "storage root"
2376
- * `folder` : same as `scan`, but only on the specified folder&apos;s "file identifier"
2377
- * `file` : for an individual file generation
2378
-
2379
- Note that for the `event`, the option `iteration_file` should be specified so that
2380
- successive calls only process new events. This file will hold an identifier
2381
- telling from where to get new events.
2382
-
2383
- It is also possible to test a local file, using the `test` command.
2753
+ * `scan` : recursively scan all files under the access key&apos;s "storage root"
2754
+ * `test` : test using a local file
2384
2755
 
2385
2756
  Once candidate are selected, once candidates are selected,
2386
2757
  a preview is always generated if it does not exist already,
2387
2758
  else if a preview already exist, it will be generated
2388
- using one of three overwrite method:
2759
+ using one of three values for the `overwrite` option:
2389
2760
 
2390
2761
  * `always` : preview is always generated, even if it already exists and is newer than original
2391
2762
  * `never` : preview is generated only if it does not exist already
2392
2763
  * `mtime` : preview is generated only if the original file is newer than the existing
2393
2764
 
2394
- Deletion of preview for deleted source files: not implemented yet.
2765
+ Deletion of preview for deleted source files: not implemented yet (TODO).
2395
2766
 
2396
2767
  If the `scan` or `events` detection method is used, then the option : `skip_folders` can be used to skip some folders. It expects a list of path relative to the storage root (docroot) starting with slash, use the `@json:` notation, example:
2397
2768
 
@@ -2401,6 +2772,14 @@ $ <%=cmd%> preview scan --skip-folders=@json:'["/not_here"]'
2401
2772
 
2402
2773
  The option `folder_reset_cache` forces the node service to refresh folder contents using various methods.
2403
2774
 
2775
+ When scanning the option `value` has the same behaviour as for the `node find` command.
2776
+
2777
+ For instance to filter out files beginning with `._` do:
2778
+
2779
+ ```
2780
+ ... --value='exec:!f["name"].start_with?("._") or f["name"].eql?(".DS_Store")'
2781
+ ```
2782
+
2404
2783
  ## Preview File types
2405
2784
 
2406
2785
  Two types of preview can be generated:
@@ -2538,7 +2917,7 @@ Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters ar
2538
2917
  ## Simple session
2539
2918
 
2540
2919
  ```
2541
- MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
2920
+ MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_demo_pass_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
2542
2921
 
2543
2922
  echo "${MY_TSPEC}"|asession
2544
2923
  ```
@@ -2551,7 +2930,7 @@ This is particularly useful for a persistent session ( with the [_transfer-spec_
2551
2930
 
2552
2931
  ```
2553
2932
  $ asession
2554
- {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"demoaspera","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
2933
+ {"remote_host":"demo.asperasoft.com","ssh_port":33001,"remote_user":"asperaweb","remote_password":"_demo_pass_","direction":"receive","destination_root":".","keepalive":true,"resume_level":"none"}
2555
2934
  {"type":"START","source":"/aspera-test-dir-tiny/200KB.2"}
2556
2935
  {"type":"DONE"}
2557
2936
  ```
@@ -2625,6 +3004,34 @@ $ <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=1
2625
3004
 
2626
3005
  The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
2627
3006
 
3007
+ # Aspera Health check and Nagios
3008
+
3009
+ Each plugin provide a `health` command that will check the health status of the application. Example:
3010
+
3011
+ ```
3012
+ $ <%=cmd%> console health
3013
+ +--------+-------------+------------+
3014
+ | status | component | message |
3015
+ +--------+-------------+------------+
3016
+ | ok | console api | accessible |
3017
+ +--------+-------------+------------+
3018
+ ```
3019
+
3020
+ Typically, the health check uses the REST API of the application with the following exception: the `server` plugin allows checking health by:
3021
+
3022
+ * issuing a transfer to the server
3023
+ * checking web app status with `asctl all:status`
3024
+ * checking daemons process status
3025
+
3026
+ <%=tool%> can be called by Nagios to check the health status of an Aspera server. The output can be made compatible to Nagios with option `--format=nagios` :
3027
+
3028
+ ```
3029
+ $ <%=cmd%> server health transfer --to-folder=/Upload --format=nagios --progress=none
3030
+ OK - [transfer:ok]
3031
+ $ <%=cmd%> server health asctlstatus --cmd_prefix='sudo ' --format=nagios
3032
+ OK - [NP:running, MySQL:running, Mongrels:running, Background:running, DS:running, DB:running, Email:running, Apache:running]
3033
+ ```
3034
+
2628
3035
  # Module: `Aspera`
2629
3036
 
2630
3037
  Main components:
@@ -2644,6 +3051,16 @@ This sample code shows some example of use of the API as well as
2644
3051
  REST API.
2645
3052
  Note: although nice, it's probably a good idea to use RestClient for REST.
2646
3053
 
3054
+ Example of use of the API of Aspera on Cloud:
3055
+
3056
+ ```
3057
+ require 'aspera/aoc'
3058
+
3059
+ aoc=Aspera::AoC.new(url: 'https://sedemo.ibmaspera.com',auth: :jwt, scope: 'user:all', private_key: File.read(File.expand_path('~/.aspera/<%=cmd%>/aspera_on_cloud_key')),username: 'laurent.martin.aspera@fr.ibm.com',subpath: 'api/v1')
3060
+
3061
+ aoc.read('self')
3062
+ ```
3063
+
2647
3064
  # History
2648
3065
 
2649
3066
  When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
@@ -2660,11 +3077,43 @@ So, it evolved into <%=tool%>:
2660
3077
  * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [_transfer-spec_](#transferspec) (more reliable , more standard)
2661
3078
  * `ruby` is consistent with other Aspera products
2662
3079
 
3080
+ # Changes (Release notes)
3081
+
3082
+ * <%= gemspec.version.to_s %>
2663
3083
 
3084
+ * new: command `faspex package recv` supports link of type: `faspe:`
3085
+ * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
2664
3086
 
2665
- # Release Notes
3087
+ * 4.2.0
2666
3088
 
2667
- * 4.0.0.pre2
3089
+ * new: command `aoc remind` to receive organization membership by email
3090
+ * new: in `preview` option `value` to filter out on file name
3091
+ * new: `initdemo` to initialize for demo server
3092
+ * new: `direct` transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
3093
+ * fix: on Windows `conf ascp use` expects ascp.exe
3094
+ * fix: (break) multi_session_threshold is Integer, not String
3095
+ * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
3096
+ * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
3097
+ * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
3098
+ * change: (break) parameters for resume in `transfer-info` for `direct` are now in sub-key `"resume"`
3099
+
3100
+ * 4.1.0
3101
+
3102
+ * fix: remove keys from transfer spec and command line when not needed
3103
+ * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
3104
+ * new: update documentation with regard to offline and docker installation
3105
+ * new: renamed command `nagios_check` to `health`
3106
+ * new: agent `http_gw` now supports upload
3107
+ * new: added option `sdk_url` to install SDK from local file for offline install
3108
+ * new: check new gem version periodically
3109
+ * new: the --fields= option, support -_fieldname_ to remove a field from default fields
3110
+ * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
3111
+ * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
3112
+ * new: AoC a password can be provided for a public link
3113
+ * new: `conf doc` take an optional parameter to go to a section
3114
+ * new: initial support for Faspex 5 Beta 1
3115
+
3116
+ * 4.0.0
2668
3117
 
2669
3118
  * now available as open source at [<%= gemspec.homepage %>](<%= gemspec.homepage %>) with general cleanup
2670
3119
  * changed default tool name from `mlia` to `ascli`
@@ -2792,7 +3241,7 @@ So, it evolved into <%=tool%>:
2792
3241
 
2793
3242
  * 0.10.6
2794
3243
 
2795
- * FaspManager: transfer spec `authentication` no more needed for local tranfer to use aspera public keys. public keys will be used if there is a token and no key or password is provided.
3244
+ * FaspManager: transfer spec `authentication` no more needed for local tranfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
2796
3245
  * gem version requirements made more open
2797
3246
 
2798
3247
  * 0.10.5
@@ -3019,11 +3468,15 @@ Breaking change:
3019
3468
 
3020
3469
  * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
3021
3470
 
3022
- # BUGS
3471
+ # BUGS, FEATURES, CONTRIBUTION
3472
+
3473
+ For issues or feature requests use the Github repository and issues.
3023
3474
 
3024
- * This is best effort code without official support, dont expect full capabilities. This code is not supported by IBM/Aspera. You can contact the author for bugs or features.
3475
+ You can also contribute to this open source project.
3025
3476
 
3026
- ## only one value for any option
3477
+ One can also create one's own command nplugin.
3478
+
3479
+ ## Only one value for any option
3027
3480
 
3028
3481
  Some commands and sub commands may ask for the same option name.
3029
3482
  Currently, since option definition is position independant (last one wins), it is not possible
@@ -3037,7 +3490,8 @@ This happens typically for the `node` sub command, e.g. identify the node by nam
3037
3490
 
3038
3491
  ## ED255519 key not supported
3039
3492
 
3040
- ED255519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
3493
+ ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
3494
+
3041
3495
  Without this deactivation, if such key was present the following error was generated:
3042
3496
 
3043
3497
  ```
@@ -3047,7 +3501,17 @@ OpenSSH keys only supported if ED25519 is available
3047
3501
  Which meant that you do not have ruby support for ED25519 SSH keys.
3048
3502
  You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
3049
3503
 
3050
- # TODO
3504
+ ## Error "Remote host is not who we expected"
3505
+
3506
+ `ascp` version 4.x changed the algorithm used to check the SSH server certificate. To ignore the certificate (SSH fingerprint) add option on client side:
3507
+
3508
+ ```
3509
+ --ts=@json:'{"sshfp":null}'
3510
+ ```
3511
+
3512
+ Refer to ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server): if you have access to server side, basically disable other SSH host keys than RSA.
3513
+
3514
+ ## Miscelaneous
3051
3515
 
3052
3516
  * remove rest and oauth classes and use ruby standard gems:
3053
3517
 
@@ -3063,9 +3527,3 @@ You may either install the suggested Gems, or remove your ed25519 key from your
3063
3527
  * Going through proxy: use env var http_proxy and https_proxy, no_proxy
3064
3528
 
3065
3529
  * easier use with https://github.com/pmq20/ruby-packer
3066
-
3067
- # Contribution
3068
-
3069
- Send comments !
3070
-
3071
- Create your own plugin !