aspera-cli 4.3.0 → 4.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +445 -194
- data/docs/README.erb.md +449 -239
- data/docs/doc_tools.rb +58 -0
- data/lib/aspera/aoc.rb +4 -3
- data/lib/aspera/cli/plugin.rb +13 -6
- data/lib/aspera/cli/plugins/aoc.rb +87 -51
- data/lib/aspera/cli/plugins/config.rb +79 -50
- data/lib/aspera/cli/plugins/faspex.rb +8 -6
- data/lib/aspera/cli/plugins/node.rb +51 -23
- data/lib/aspera/cli/plugins/preview.rb +9 -7
- data/lib/aspera/cli/plugins/server.rb +16 -5
- data/lib/aspera/cli/transfer_agent.rb +5 -20
- data/lib/aspera/cli/version.rb +1 -1
- data/lib/aspera/fasp/http_gw.rb +47 -26
- data/lib/aspera/fasp/local.rb +0 -2
- data/lib/aspera/fasp/parameters.rb +16 -3
- data/lib/aspera/faspex_gw.rb +10 -9
- data/lib/aspera/node.rb +10 -0
- metadata +3 -3
- data/lib/aspera/fasp/aoc.rb +0 -24
data/docs/README.erb.md
CHANGED
@@ -1,58 +1,20 @@
|
|
1
1
|
[comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
|
2
|
-
<%
|
3
|
-
# check that required env vars exist, and files
|
4
|
-
%w{EXENAME GEMSPEC INCL_USAGE INCL_COMMANDS INCL_ASESSION INCL_DIR_GEM}.each do |e|
|
5
|
-
raise "missing env var #{e}" unless ENV.has_key?(e)
|
6
|
-
raise "missing file #{ENV[e]}" unless File.exist?(ENV[e]) or !e.start_with?('INCL_') #_
|
7
|
-
end
|
8
|
-
cmd=ENV['EXENAME'] # just command name
|
9
|
-
tool='`'+cmd+'`' # used in text with formatting of command
|
10
|
-
evp=cmd.upcase+'_' # prefix for env vars
|
11
|
-
opprst='option preset' # just the name for "option preset"
|
12
|
-
prst='['+opprst+'](#lprt)' # name with link
|
13
|
-
prsts='['+opprst+'s](#lprt)' # name with link (plural)
|
14
|
-
prstt=opprst.capitalize # in title
|
15
|
-
gemspec=Gem::Specification::load(ENV["GEMSPEC"]) or raise "error loading #{ENV["GEMSPEC"]}"
|
16
|
-
geminstadd=gemspec.version.to_s.match(/\.[^0-9]/)?' --pre':''
|
17
|
-
$LOAD_PATH.unshift(ENV["INCL_DIR_GEM"])
|
18
|
-
require 'aspera/fasp/parameters'
|
19
|
-
def spec_table
|
20
|
-
r='<table><tr><th>Field</th><th>Type</th>'
|
21
|
-
Aspera::Fasp::Parameters::SUPPORTED_AGENTS_SHORT.each do |c|
|
22
|
-
r << '<th>'<<c.to_s.upcase<<'</th>'
|
23
|
-
end
|
24
|
-
r << '<th>Description</th></tr>'
|
25
|
-
Aspera::Fasp::Parameters.man_table.each do |p|
|
26
|
-
p[:description] << (p[:description].empty? ? '' : "\n") << "(" << p[:cli] << ")" unless p[:cli].to_s.empty?
|
27
|
-
p.delete(:cli)
|
28
|
-
p.keys.each{|c|p[c]=' ' if p[c].to_s.empty?}
|
29
|
-
p[:description].gsub!("\n",'<br/>')
|
30
|
-
p[:type].gsub!(',','<br/>')
|
31
|
-
r << '<tr><td>'<<p[:name]<<'</td><td>'<<p[:type]<<'</td>'
|
32
|
-
Aspera::Fasp::Parameters::SUPPORTED_AGENTS_SHORT.each do |c|
|
33
|
-
r << '<td>'<<p[c]<<'</td>'
|
34
|
-
end
|
35
|
-
r << '<td>'<<p[:description]<<'</td></tr>'
|
36
|
-
end
|
37
|
-
r << '</table>'
|
38
|
-
return r
|
39
|
-
end #```
|
40
|
-
-%>
|
2
|
+
<%load File.join(File.dirname(__FILE__),'doc_tools.rb')-%>
|
41
3
|
<font size="+12"><center><%=tool%> : Command Line Interface for IBM Aspera products</center></font>
|
42
4
|
|
43
|
-
Version : <%=
|
5
|
+
Version : <%=gemspec.version.to_s%>
|
44
6
|
|
45
7
|
_Laurent/2016-<%=Time.new.year%>_
|
46
8
|
|
47
|
-
This gem provides <%=tool
|
9
|
+
This gem provides the <%=tool%> Command Line Interface to IBM Aspera software.
|
48
10
|
|
49
11
|
<%=tool%> is a also great tool to learn Aspera APIs.
|
50
12
|
|
51
|
-
Ruby Gem: [<%=
|
13
|
+
Ruby Gem: [<%=gemspec.metadata['rubygems_uri']%>](<%=gemspec.metadata['rubygems_uri']%>)
|
52
14
|
|
53
|
-
Ruby Doc: [<%=
|
15
|
+
Ruby Doc: [<%=gemspec.metadata['documentation_uri']%>](<%=gemspec.metadata['documentation_uri']%>)
|
54
16
|
|
55
|
-
Required Ruby version: <%=
|
17
|
+
Required Ruby version: <%=gemspec.required_ruby_version%>
|
56
18
|
|
57
19
|
# <a name="when_to_use"></a>When to use and when not to use
|
58
20
|
|
@@ -96,7 +58,7 @@ Command line parameters in examples beginning with `my_`, like `my_param_value`
|
|
96
58
|
|
97
59
|
On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before <%=tool%> (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to <%=tool%>. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
|
98
60
|
|
99
|
-
On Windows, `cmd` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
|
61
|
+
On Windows, `cmd.exe` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
|
100
62
|
|
101
63
|
* [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
|
102
64
|
* [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
|
@@ -121,7 +83,7 @@ Once the gem is installed, <%=tool%> shall be accessible:
|
|
121
83
|
|
122
84
|
```
|
123
85
|
$ <%=cmd%> --version
|
124
|
-
<%=
|
86
|
+
<%=gemspec.version.to_s%>
|
125
87
|
```
|
126
88
|
|
127
89
|
## First use
|
@@ -148,7 +110,7 @@ $ <%=cmd%> server browse /
|
|
148
110
|
If you want to use <%=tool%> with another server, and in order to make further calls more convenient, it is advised to define a <%=prst%> for the server's authentication options. The following example will:
|
149
111
|
|
150
112
|
* create a <%=prst%>
|
151
|
-
* define it as default for
|
113
|
+
* define it as default for `server` plugin
|
152
114
|
* list files in a folder
|
153
115
|
* download a file
|
154
116
|
|
@@ -191,8 +153,8 @@ It is possible to install *either* directly on the host operating system (Linux,
|
|
191
153
|
|
192
154
|
The direct installation is recommended and consists in installing:
|
193
155
|
|
194
|
-
* [Ruby](#ruby) version <%=
|
195
|
-
* [<%=
|
156
|
+
* [Ruby](#ruby) version <%=gemspec.required_ruby_version%>
|
157
|
+
* [<%=gemspec.name%>](#the_gem)
|
196
158
|
* [Aspera SDK (ascp)](#fasp_prot)
|
197
159
|
|
198
160
|
The following sections provide information on the various installation methods.
|
@@ -205,6 +167,8 @@ Use this method only if you know what you do, else use the standard recommended
|
|
205
167
|
|
206
168
|
This method installs a docker image that contains: Ruby, ascli and the FASP sdk.
|
207
169
|
|
170
|
+
The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli)
|
171
|
+
|
208
172
|
Ensure that you have Docker installed.
|
209
173
|
|
210
174
|
```
|
@@ -239,7 +203,7 @@ Use this method to install on the native host.
|
|
239
203
|
|
240
204
|
A ruby interpreter is required to run the tool or to use the gem and tool.
|
241
205
|
|
242
|
-
Required Ruby version: <%=
|
206
|
+
Required Ruby version: <%=gemspec.required_ruby_version%>. Ruby version 3 is also supported.
|
243
207
|
|
244
208
|
*Ruby can be installed using any method* : rpm, yum, dnf, rvm, brew, windows installer, ... .
|
245
209
|
|
@@ -247,7 +211,7 @@ Refer to the following sections for a proposed method for specific operating sys
|
|
247
211
|
|
248
212
|
The recommended installation method is `rvm` for systems with "bash-like" shell (Linux, Macos, Windows with cygwin, etc...).
|
249
213
|
If the generic install is not suitable (e.g. Windows, no cygwin), you can use one of OS-specific install method.
|
250
|
-
If you have a simpler better way to install Ruby version <%=
|
214
|
+
If you have a simpler better way to install Ruby version <%=gemspec.required_ruby_version%> : use it !
|
251
215
|
|
252
216
|
### Generic: RVM: single user installation (not root)
|
253
217
|
|
@@ -273,7 +237,7 @@ If you keep the same terminal (ont needed if re-login):
|
|
273
237
|
$ source ~/.rvm/scripts/rvm
|
274
238
|
```
|
275
239
|
|
276
|
-
It is advised to get one of the pre-compiled ruby version, you can list with:
|
240
|
+
It is advised to get one of the pre-compiled ruby version, you can list with:
|
277
241
|
|
278
242
|
```
|
279
243
|
$ rvm list --remote
|
@@ -317,10 +281,10 @@ Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstal
|
|
317
281
|
|
318
282
|
### macOS: pre-installed or `brew`
|
319
283
|
|
320
|
-
MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%=
|
284
|
+
MacOS 10.13+ (High Sierra) comes with a recent Ruby. So you can use it directly. You will need to install <%=gemspec.name%> using `sudo` :
|
321
285
|
|
322
286
|
```
|
323
|
-
$ sudo gem install <%=
|
287
|
+
$ sudo gem install <%=gemspec.name%><%=geminstadd%>
|
324
288
|
```
|
325
289
|
|
326
290
|
Alternatively, if you use [Homebrew](https://brew.sh/) already you can install Ruby with it:
|
@@ -408,18 +372,18 @@ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
|
408
372
|
|
409
373
|
or restore the `$HOME/.aspera` folder for the user.
|
410
374
|
|
411
|
-
## <a name="the_gem"></a>`<%=
|
375
|
+
## <a name="the_gem"></a>`<%=gemspec.name%>` gem
|
412
376
|
|
413
377
|
Once you have Ruby and rights to install gems: Install the gem and its dependencies:
|
414
378
|
|
415
379
|
```
|
416
|
-
# gem install <%=
|
380
|
+
# gem install <%=gemspec.name%><%=geminstadd%>
|
417
381
|
```
|
418
382
|
|
419
383
|
To upgrade to the latest version:
|
420
384
|
|
421
385
|
```
|
422
|
-
# gem update <%=
|
386
|
+
# gem update <%=gemspec.name%>
|
423
387
|
```
|
424
388
|
|
425
389
|
<%=tool%> checks every week if a new version is available and notify the user in a WARN log. To de-activate this feature set the option `version_check_days` to `0`, or specify a different period in days.
|
@@ -469,8 +433,8 @@ by visiting the page: [https://www.ibm.com/aspera/connect/](https://www.ibm.com/
|
|
469
433
|
<%=tool%> will detect most of Aspera transfer products in standard locations and use the first one found.
|
470
434
|
Refer to section [FASP](#client) for details on how to select a client or set path to the FASP protocol.
|
471
435
|
|
472
|
-
Several methods are provided
|
473
|
-
other methods are available. Refer to section: [Transfer Agents](#agents)
|
436
|
+
Several methods are provided to start a transfer.
|
437
|
+
Use of a local client ([`direct`](#direct) transfer agent) is one of them, but other methods are available. Refer to section: [Transfer Agents](#agents)
|
474
438
|
|
475
439
|
## <a name="offline_install"></a>Offline Installation (without internet)
|
476
440
|
|
@@ -501,7 +465,7 @@ $ <%=cmd%> conf ascp install --sdk-url=file:///SDK.zip
|
|
501
465
|
|
502
466
|
# <a name="cli"></a>Command Line Interface: <%=tool%>
|
503
467
|
|
504
|
-
The `<%=
|
468
|
+
The `<%=gemspec.name%>` Gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
|
505
469
|
|
506
470
|
* IBM Aspera High Speed Transfer Server (FASP and Node)
|
507
471
|
* IBM Aspera on Cloud (including ATS)
|
@@ -516,7 +480,7 @@ The `<%= gemspec.name %>` Gem provides a command line interface (CLI) which inte
|
|
516
480
|
* Supports most Aspera server products (on-premise and SaaS)
|
517
481
|
* Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files
|
518
482
|
* Supports Commands, Option values and Parameters shortcuts
|
519
|
-
* FASP [Transfer Agents](#agents) can be:
|
483
|
+
* FASP [Transfer Agents](#agents) can be: local ascp, or Connect Client, or any transfer node
|
520
484
|
* Transfer parameters can be altered by modification of _transfer-spec_, this includes requiring multi-session
|
521
485
|
* Allows transfers from products to products, essentially at node level (using the node transfer agent)
|
522
486
|
* Supports FaspStream creation (using Node API)
|
@@ -640,12 +604,12 @@ The style of output can be set using the `format` parameter, supporting:
|
|
640
604
|
* `yaml` : YAML
|
641
605
|
* `csv` : Comma Separated Values
|
642
606
|
|
643
|
-
###
|
607
|
+
### <a name="option_select"></a>Option: `select`: Filter on columns values for `object_list`
|
644
608
|
|
645
609
|
Table output can be filtered using the `select` parameter. Example:
|
646
610
|
|
647
611
|
```
|
648
|
-
$ <%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"
|
612
|
+
$ <%=cmd%> aoc admin res user list --fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --select=@json:'{"ats_admin":true}'
|
649
613
|
:...............................:..................................:...........:
|
650
614
|
: name : email : ats_admin :
|
651
615
|
:...............................:..................................:...........:
|
@@ -792,19 +756,15 @@ C:\Users\Kenji\.aspera\<%=cmd%>
|
|
792
756
|
On the first execution of <%=tool%>, an empty configuration file is created in the configuration folder.
|
793
757
|
Nevertheless, there is no mandatory information required in this file, the use of it is optional as any option can be provided on the command line.
|
794
758
|
|
795
|
-
Although the file is a standard YAML file, <%=tool%> provides commands to read and modify it
|
796
|
-
using the `config` command.
|
759
|
+
Although the file is a standard YAML file, <%=tool%> provides commands to read and modify it using the `config` command.
|
797
760
|
|
798
|
-
All options for <%=tool%>
|
761
|
+
All options for <%=tool%> can be set on command line, or by env vars, or using <%=prsts%> in the configuratin file.
|
799
762
|
|
800
|
-
A configuration file provides a way to define default values, especially
|
801
|
-
for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
|
763
|
+
A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always having to specify those parameters on the command line.
|
802
764
|
|
803
|
-
The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml`
|
804
|
-
(this can be overriden with option `--config-file=path` or equivalent env var).
|
765
|
+
The default configuration file is: `$HOME/.aspera/<%=cmd%>/config.yaml` (this can be overriden with option `--config-file=path` or equivalent env var).
|
805
766
|
|
806
|
-
|
807
|
-
called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
|
767
|
+
The configuration file is simply a catalog of pre-defined lists of options, called: <%=prsts%>. Then, instead of specifying some common options on the command line (e.g. address, credentials), it is possible to invoke the ones of a <%=prst%> (e.g. `mypreset`) using the option: `-Pmypreset` or `--preset=mypreset`.
|
808
768
|
|
809
769
|
### <a name="lprt"></a><%=prstt%>
|
810
770
|
|
@@ -904,7 +864,7 @@ demo_server:
|
|
904
864
|
We can see here:
|
905
865
|
|
906
866
|
* The configuration was created with CLI version 0.3.7
|
907
|
-
* the default <%=prst%> to load for plugin
|
867
|
+
* the default <%=prst%> to load for `server` plugin is : `demo_server`
|
908
868
|
* the <%=prst%> `demo_server` defines some parameters: the URL and credentials
|
909
869
|
* the default <%=prst%> to load in any case is : `cli_default`
|
910
870
|
|
@@ -935,26 +895,26 @@ my_aoc_org:
|
|
935
895
|
|
936
896
|
So, the key file will be read only at execution time, but not be embedded in the configuration file.
|
937
897
|
|
898
|
+
### Options evaluation order
|
899
|
+
|
900
|
+
Some options are global, some options are available only for some plugins. (the plugin is the first level command).
|
901
|
+
|
938
902
|
Options are loaded using this algorithm:
|
939
903
|
|
940
|
-
*
|
941
|
-
|
942
|
-
|
943
|
-
*
|
944
|
-
*
|
904
|
+
* If option `--no-default` (or `-N`) is specified, then no default value is loaded is loaded for the plugin
|
905
|
+
* else it looks for the name of the plugin as key in section `default`, the value is the name of the default <%=prst%> for it, and loads it.
|
906
|
+
* If option `--preset=<name or extended value hash>` is specified (or `-Pxxxx`), this reads the <%=prst%> specified from the configuration file, or of the value is a Hash, it uses it as options values.
|
907
|
+
* Environment variables are evaluated
|
908
|
+
* Command line options are evaluated
|
945
909
|
|
946
910
|
Parameters are evaluated in the order of command line.
|
947
911
|
|
948
|
-
To avoid loading the default <%=prst%> for a plugin,
|
912
|
+
To avoid loading the default <%=prst%> for a plugin, use: `-N`
|
949
913
|
|
950
914
|
On command line, words in parameter names are separated by a dash, in configuration file, separator
|
951
915
|
is an underscore. E.g. --xxx-yyy on command line gives xxx_yyy in configuration file.
|
952
916
|
|
953
|
-
|
954
|
-
convert olver versions, remove the leading ":" in front of keys.
|
955
|
-
|
956
|
-
The main plugin name is *config*, so it is possible to define a default <%=prst%> for
|
957
|
-
the main plugin with:
|
917
|
+
The main plugin name is `config`, so it is possible to define a default <%=prst%> for the main plugin with:
|
958
918
|
|
959
919
|
```
|
960
920
|
$ <%=cmd%> config id cli_default set interactive no
|
@@ -967,6 +927,18 @@ A <%=prst%> value can be removed with `unset`:
|
|
967
927
|
$ <%=cmd%> config id cli_default unset interactive
|
968
928
|
```
|
969
929
|
|
930
|
+
Example: Define options using command line:
|
931
|
+
|
932
|
+
```
|
933
|
+
$ <%=cmd%> -N --url=x --password=y --username=y node --show-config
|
934
|
+
```
|
935
|
+
|
936
|
+
Example: Define options using a hash:
|
937
|
+
|
938
|
+
```
|
939
|
+
$ <%=cmd%> -N --preset=@json:'{"url":"x","password":"y","username":"y"}' node --show-config
|
940
|
+
```
|
941
|
+
|
970
942
|
### Examples
|
971
943
|
|
972
944
|
For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console,
|
@@ -1095,7 +1067,7 @@ $ <%=cmd%> config ascp info
|
|
1095
1067
|
...
|
1096
1068
|
```
|
1097
1069
|
|
1098
|
-
### Selection of
|
1070
|
+
### Selection of `ascp` location for [`direct`](#direct) agent
|
1099
1071
|
|
1100
1072
|
By default, <%=tool%> uses any found local product with ascp, including SDK.
|
1101
1073
|
|
@@ -1139,7 +1111,7 @@ $ <%=cmd%> config ascp products list
|
|
1139
1111
|
:.........................................:................................................:
|
1140
1112
|
```
|
1141
1113
|
|
1142
|
-
### Selection of local client
|
1114
|
+
### Selection of local client for `ascp` for [`direct`](#direct) agent
|
1143
1115
|
|
1144
1116
|
If no ascp is selected, this is equivalent to using option: `--use-product=FIRST`.
|
1145
1117
|
|
@@ -1194,7 +1166,7 @@ This [_transfer-spec_](#transferspec) will be executed by a transfer client, her
|
|
1194
1166
|
|
1195
1167
|
There are currently 3 agents:
|
1196
1168
|
|
1197
|
-
* `direct` : a local execution of `ascp`
|
1169
|
+
* [`direct`](#direct) : a local execution of `ascp`
|
1198
1170
|
* `connect` : use of a local Connect Client
|
1199
1171
|
* `node` : use of an Aspera Transfer Node (potentially _remote_).
|
1200
1172
|
* `httpgw` : use of an Aspera HTTP Gateway
|
@@ -1206,7 +1178,7 @@ will effectively push files to the related server from the agent node.
|
|
1206
1178
|
<%=tool%> standadizes on the use of a [_transfer-spec_](#transferspec) instead of _raw_ ascp options to provide parameters for a transfer session, as a common method for those three Transfer Agents.
|
1207
1179
|
|
1208
1180
|
|
1209
|
-
### <a name="direct"></a>Direct (local ascp
|
1181
|
+
### <a name="direct"></a>Direct (local ascp execution)
|
1210
1182
|
|
1211
1183
|
By default <%=tool%> uses a local ascp, equivalent to specifying `--transfer=direct`.
|
1212
1184
|
<%=tool%> will detect locally installed Aspera products.
|
@@ -1304,7 +1276,7 @@ is described in a _transfer-spec_ (Transfer Specification), such as:
|
|
1304
1276
|
|
1305
1277
|
If needed, it is possible to modify or add any of the supported _transfer-spec_ parameter using the `ts` option. The `ts` option accepts a [Structured Value](#native) containing one or several _transfer-spec_ parameters. Multiple `ts` options on command line are cummulative.
|
1306
1278
|
|
1307
|
-
It is possible to specify ascp options when the `transfer` option is set to `direct` using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
|
1279
|
+
It is possible to specify ascp options when the `transfer` option is set to [`direct`](#direct) using the special [_transfer-spec_](#transferspec) parameter: `EX_ascp_args`. Example: `--ts=@json:'{"EX_ascp_args":["-l","100m"]}'`. This is espacially useful for ascp command line parameters not supported yet in the transfer spec.
|
1308
1280
|
|
1309
1281
|
The use of a _transfer-spec_ instead of `ascp` parameters has the advantage of:
|
1310
1282
|
|
@@ -1337,9 +1309,9 @@ Columns:
|
|
1337
1309
|
* C=Connect Client
|
1338
1310
|
* arg=`ascp` argument or environment variable
|
1339
1311
|
|
1340
|
-
Fields with EX_ prefix are extensions to transfer agent `direct
|
1312
|
+
Fields with EX_ prefix are extensions to transfer agent [`direct`](#direct). (only in <%=tool%>).
|
1341
1313
|
|
1342
|
-
<%=
|
1314
|
+
<%=spec_table%>
|
1343
1315
|
|
1344
1316
|
### Destination folder for transfers
|
1345
1317
|
|
@@ -1355,8 +1327,8 @@ The option `to_folder` provides an equivalent and convenient way to change this
|
|
1355
1327
|
|
1356
1328
|
### List of files for transfers
|
1357
1329
|
|
1358
|
-
When uploading, downloading or sending files, the user must specify the list of files to transfer.
|
1359
|
-
|
1330
|
+
When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files (Extensed value) is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
|
1331
|
+
So, by default, the list of files to transfer will be simply specified on the command line:
|
1360
1332
|
|
1361
1333
|
```
|
1362
1334
|
$ <%=cmd%> server upload ~/mysample.file secondfile
|
@@ -1368,7 +1340,7 @@ This is equivalent to:
|
|
1368
1340
|
$ <%=cmd%> server upload --sources=@args ~/mysample.file secondfile
|
1369
1341
|
```
|
1370
1342
|
|
1371
|
-
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
|
1343
|
+
More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [_transfer-spec_](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
|
1372
1344
|
|
1373
1345
|
Note that this is different from the "ascp" command line. The paradigm used by <%=tool%> is:
|
1374
1346
|
all transfer parameters are kept in [_transfer-spec_](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
|
@@ -1391,13 +1363,15 @@ For ease of use and flexibility, the list of files to transfer is specified by t
|
|
1391
1363
|
--sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
|
1392
1364
|
```
|
1393
1365
|
|
1394
|
-
*
|
1366
|
+
* Not recommended: It is possible to specify bare ascp arguments using the pseudo [_transfer-spec_](#transferspec) parameter `EX_ascp_args`.
|
1395
1367
|
|
1396
1368
|
```
|
1397
1369
|
--sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
|
1398
1370
|
```
|
1399
1371
|
|
1400
|
-
|
1372
|
+
This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [_transfer-spec_](#transferspec), which will be overriden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
|
1373
|
+
|
1374
|
+
In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
|
1401
1375
|
|
1402
1376
|
* `list` : (default) the path of destination is the same as source
|
1403
1377
|
* `pair` : in that case, the first element is the first source, the second element is the first destination, and so on.
|
@@ -1408,6 +1382,8 @@ Example:
|
|
1408
1382
|
$ <%=cmd%> server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
|
1409
1383
|
```
|
1410
1384
|
|
1385
|
+
Internally, when transfer agent [`direct`](#direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pait-list` is provided in `ts` in `EX_ascp_args`.
|
1386
|
+
|
1411
1387
|
Note the special case when the source files are located on "Aspera on Cloud", i.e. using access keys and the `file id` API:
|
1412
1388
|
|
1413
1389
|
* All files must be in the same source folder.
|
@@ -1472,22 +1448,161 @@ When multi-session is used, one separate UDP port is used per session (refer to
|
|
1472
1448
|
|
1473
1449
|
|
1474
1450
|
|
1475
|
-
## <a name="scheduling"></a>
|
1451
|
+
## <a name="scheduling"></a>Lock for exclusive execution
|
1452
|
+
|
1453
|
+
In some conditions, it may be desirable to ensure that <%=tool%> is not executed several times in parallel.
|
1454
|
+
|
1455
|
+
For instance when <%=tool%> is executed automatically on a schedule basis, one generally desire that a new execution is not started if a previous execution is still running because an on-going operation may last longer than the scheduling period:
|
1456
|
+
|
1457
|
+
* Executing instances may pile-up and kill the system
|
1458
|
+
* The same file may be transfered by multiple instances at the same time.
|
1459
|
+
* `preview` may generate the same files in multiple instances.
|
1460
|
+
|
1461
|
+
Usually the OS native scheduler already provides some sort of protection against parallel execution:
|
1462
|
+
|
1463
|
+
* The Windows scheduler does this by default
|
1464
|
+
* Linux cron can leverage the utility [`flock`](https://linux.die.net/man/1/flock) to do the same:
|
1465
|
+
|
1466
|
+
```
|
1467
|
+
/usr/bin/flock -w 0 /var/cron.lock ascli ...
|
1468
|
+
```
|
1469
|
+
|
1470
|
+
<%=tool%> natively supports a locking mechanism with option `lock_port`.
|
1471
|
+
(Technically, this opens a local TCP server port, and fails if this port is already used, providing a local lock. Lock is released when process exits).
|
1472
|
+
|
1473
|
+
Example:
|
1474
|
+
|
1475
|
+
Run this same command in two separate terminals within less than 30 seconds:
|
1476
|
+
|
1477
|
+
```
|
1478
|
+
ascli config echo @ruby:'sleep(30)' --lock-port=12345
|
1479
|
+
```
|
1480
|
+
|
1481
|
+
The first instance will sleep 30 seconds, the second one will immediately exit like this:
|
1482
|
+
|
1483
|
+
```
|
1484
|
+
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
|
1485
|
+
```
|
1486
|
+
|
1487
|
+
## "Provençale"
|
1488
|
+
|
1489
|
+
`ascp`, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's `open`,`read`,`write`,`close` primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is call *PVCL*. Several *PVCL* adapters are available, some are embedded in `ascp`
|
1490
|
+
, some are provided om shared libraries and must be activated. (e.g. using `trapd`)
|
1491
|
+
|
1492
|
+
The list of supported *PVCL* adapters can be retried with command:
|
1493
|
+
|
1494
|
+
```
|
1495
|
+
$ <%=cmd%> conf ascp info
|
1496
|
+
+--------------------+-----------------------------------------------------------+
|
1497
|
+
| key | value |
|
1498
|
+
+--------------------+-----------------------------------------------------------+
|
1499
|
+
-----8<----snip---------
|
1500
|
+
| product_name | IBM Aspera SDK |
|
1501
|
+
| product_version | 4.0.1.182389 |
|
1502
|
+
| process | pvcl |
|
1503
|
+
| shares | pvcl |
|
1504
|
+
| noded | pvcl |
|
1505
|
+
| faux | pvcl |
|
1506
|
+
| file | pvcl |
|
1507
|
+
| stdio | pvcl |
|
1508
|
+
| stdio-tar | pvcl |
|
1509
|
+
+--------------------+-----------------------------------------------------------+
|
1510
|
+
```
|
1511
|
+
|
1512
|
+
Here we can see the adapters: `process`, `shares`, `noded`, `faux`, `file`, `stdio`, `stdio-tar`.
|
1513
|
+
|
1514
|
+
Those adapters can be used wherever a file path is used in `ascp` including configuration. They act as a pseudo "drive".
|
1515
|
+
|
1516
|
+
The simplified format is:
|
1517
|
+
|
1518
|
+
```
|
1519
|
+
<adapter>:///<sub file path>?<arg1>=<val1>&...
|
1520
|
+
```
|
1521
|
+
|
1522
|
+
One of the adapters, used in this manual, for testing, is `faux`. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
|
1523
|
+
|
1524
|
+
## <a name="faux_testing"></a>`faux:` for testing
|
1525
|
+
|
1526
|
+
This is an extract of the man page of `ascp`. This feature is a feature of `ascp`, not <%=tool%>
|
1527
|
+
|
1528
|
+
This adapter can be used to simulate a file or a directory.
|
1529
|
+
|
1530
|
+
To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form `faux:///fname?fsize` where:
|
1531
|
+
|
1532
|
+
* `fname` is the name that will be assigned to the file on the destination
|
1533
|
+
* `fsize` is the number of bytes that will be sent (in decimal).
|
1534
|
+
|
1535
|
+
Note that the character `?` is a special shell character (wildcard), so `faux` file specification on command line shall be protected (using `\?` and `\&` or using quotes). If not, the shell may give error: `no matches found` or equivalent.
|
1536
|
+
|
1537
|
+
For all sizes, a suffix can be added (case insensitive) to the size: k,m,g,t,p,e (values are power of 2, e.g. 1M is 2^20, i.e. 1 mebibyte, not megabyte). The maximum allowed value is 8*2^60. Very large `faux` file sizes (petabyte range and above) will likely fail due to lack of system memory unless `faux://`.
|
1538
|
+
|
1539
|
+
To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
|
1540
|
+
|
1541
|
+
```
|
1542
|
+
faux:///dirname?<arg1>=<val1>&...
|
1543
|
+
```
|
1544
|
+
|
1545
|
+
`dirname` is the folder name and can contain `/` to specify a subfolder.
|
1546
|
+
|
1547
|
+
Supported arguments are:
|
1548
|
+
|
1549
|
+
<table>
|
1550
|
+
<tr><th>name</th><th>type</th><th>default</th><th>description</th></tr>
|
1551
|
+
<tr><td>count</td><td>int</td><td>mandatory</td><td>number of files</td></tr>
|
1552
|
+
<tr><td>file</td><td>string</td><td>file</td><td>basename for files</td></tr>
|
1553
|
+
<tr><td>size</td><td>int</td><td>0</td><td>size of first file.</td></tr>
|
1554
|
+
<tr><td>inc</td><td>int</td><td>0</td><td>increment applied to determine next file size</td></tr>
|
1555
|
+
<tr><td>seq</td><td>sequential<br/>random</td><td>sequential</td><td>sequence in determining next file size</td></tr>
|
1556
|
+
<tr><td>buf_init</td><td>none<br/>zero<br/>random</td><td>zero</td><td>how source data initialized.<br/>Option 'none' is not allowed for downloads.</td></tr>
|
1557
|
+
</table>
|
1558
|
+
|
1559
|
+
|
1560
|
+
The sequence parameter is applied as follows:
|
1561
|
+
|
1562
|
+
* If `seq` is `random` then each file size is:
|
1563
|
+
|
1564
|
+
* size +/- (inc * rand())
|
1565
|
+
* Where rand is a random number between 0 and 1
|
1566
|
+
* Note that file size must not be negative, inc will be set to size if it is greater than size
|
1567
|
+
* Similarly, overall file size must be less than 8 * 2^60. If size + inc is greater, inc will be reduced to limit size + inc to 7 * 2^60.
|
1476
1568
|
|
1477
|
-
|
1569
|
+
* If `seq` is `sequential` then each file size is:
|
1478
1570
|
|
1479
|
-
|
1571
|
+
* size + ((fileindex - 1) * inc)
|
1572
|
+
* Where first file is index 1
|
1573
|
+
* So file1 is size bytes, file2 is size + inc bytes, file3 is size + inc * 2 bytes, etc.
|
1574
|
+
* As with random, inc will be adjusted if size + (count * inc) is not less then 8 ^ 2^60.
|
1480
1575
|
|
1481
|
-
|
1576
|
+
Filenames generated are of the form: `<file>_<00000 . . . count>_<filesize>`
|
1482
1577
|
|
1483
|
-
|
1578
|
+
To discard data at the destination, the destination argument is set to `faux://` .
|
1579
|
+
|
1580
|
+
Examples:
|
1581
|
+
|
1582
|
+
* Upload 20 gigabytes of random data to file myfile to directory /Upload
|
1583
|
+
|
1584
|
+
```
|
1585
|
+
$ <%=cmd%> server upload faux:///myfile\?20g --to-folder=/Upload
|
1586
|
+
```
|
1587
|
+
|
1588
|
+
* Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
|
1589
|
+
|
1590
|
+
```
|
1591
|
+
$ <%=cmd%> server upload /tmp/sample --to-folder=faux://
|
1592
|
+
```
|
1593
|
+
|
1594
|
+
* Upload a faux directory `mydir` containing 1 million files, sequentially with sizes ranging from 0 to 2 M - 2 bytes, with the basename of each file being `testfile` to /Upload
|
1595
|
+
|
1596
|
+
```
|
1597
|
+
$ <%=cmd%> server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
|
1598
|
+
```
|
1484
1599
|
|
1485
1600
|
## <a name="commands"></a>Sample Commands
|
1486
1601
|
|
1487
1602
|
A non complete list of commands used in unit tests:
|
1488
1603
|
|
1489
1604
|
```
|
1490
|
-
<%=
|
1605
|
+
<%=File.read(ENV["INCL_COMMANDS"])%>
|
1491
1606
|
...and more
|
1492
1607
|
```
|
1493
1608
|
|
@@ -1495,7 +1610,7 @@ A non complete list of commands used in unit tests:
|
|
1495
1610
|
|
1496
1611
|
```
|
1497
1612
|
$ <%=cmd%> -h
|
1498
|
-
<%=
|
1613
|
+
<%=File.read(ENV["INCL_USAGE"])%>
|
1499
1614
|
|
1500
1615
|
```
|
1501
1616
|
|
@@ -1750,19 +1865,55 @@ It allows actions (create, update, delete) on "resources": users, group, nodes,
|
|
1750
1865
|
|
1751
1866
|
Bulk operations are possible using option `bulk` (yes,no(default)): currently: create only. In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
|
1752
1867
|
|
1753
|
-
|
1868
|
+
#### Listing resources
|
1869
|
+
|
1870
|
+
The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
|
1871
|
+
|
1872
|
+
The option `query` can be optionally used. It expects a Hash using [Extended Value Syntax](#extended), generally provided using: `--query=@json:{...}`. Values are directly sent to the API call and used as a filter on server side.
|
1873
|
+
|
1874
|
+
The following parameters are supported:
|
1875
|
+
|
1876
|
+
* `q` : a filter on name of resource (case insensitive, matches if value is contained in name)
|
1877
|
+
* `sort`: name of fields to sort results, prefix with `-` for reverse order.
|
1878
|
+
* `max` : maximum number of items to retrieve (stop pages when the maximum is passed)
|
1879
|
+
* `pmax` : maximum number of pages to request (stop pages when the maximum is passed)
|
1880
|
+
* `page` : native api parameter, in general do not use (added by
|
1881
|
+
* `per_page` : native api parameter, number of items par api call, in general do not use
|
1882
|
+
* Other specific parameters depending on resource type.
|
1883
|
+
|
1884
|
+
Both `max` and `pmax` are processed internally in <%=tool%>, not included in actual API call and limit the number of successive pages requested to API. <%=tool%> will return all values using paging if not provided.
|
1885
|
+
|
1886
|
+
Other parameters are directly sent as parameters to the GET request on API.
|
1887
|
+
|
1888
|
+
`page` and `per_page` are normally added by <%=tool%> to build successive API calls to get all values if there are more than 1000. (AoC allows a maximum page size of 1000).
|
1889
|
+
|
1890
|
+
`q` and `sort` are available on most resrouce types.
|
1891
|
+
|
1892
|
+
Other parameters depend on the type of entity (refer to AoC API).
|
1893
|
+
|
1894
|
+
Examples:
|
1895
|
+
|
1896
|
+
* List users with `laurent` in name:
|
1754
1897
|
|
1755
1898
|
```
|
1756
|
-
--query=@json:'{"
|
1899
|
+
<%=cmd%> aoc admin res user list --query=--query=@json:'{"q":"laurent"}'
|
1757
1900
|
```
|
1758
1901
|
|
1759
|
-
|
1902
|
+
* List users who logded-in before a date:
|
1903
|
+
|
1760
1904
|
```
|
1761
|
-
--query=@json:'{"
|
1762
|
-
--query=@json:'{"q":"laurent"}'
|
1905
|
+
<%=cmd%> aoc admin res user list --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
1763
1906
|
```
|
1764
1907
|
|
1765
|
-
|
1908
|
+
* List external users and sort in reverse alphabetical order using name:
|
1909
|
+
|
1910
|
+
```
|
1911
|
+
<%=cmd%> aoc admin res user list --query=@json:'{"member_of_any_workspace":false,"sort":"-name"}'
|
1912
|
+
```
|
1913
|
+
|
1914
|
+
Refer to the AoC API for full list of query parameters, or use the browser in developer mode with the web UI.
|
1915
|
+
|
1916
|
+
Note the option `select` can also be used to further refine selection, refer to [section earlier](#option_select).
|
1766
1917
|
|
1767
1918
|
#### Access Key secrets
|
1768
1919
|
|
@@ -1863,17 +2014,20 @@ $ <%=cmd%> aoc admin resource node --name=_node_name_ --secret=_secret_ v4 acces
|
|
1863
2014
|
$ <%=cmd%> aoc admin res node --secret=_secret_ v3 transfer list --value=@json:'[["q","*"],["count",5]]'
|
1864
2015
|
```
|
1865
2016
|
|
1866
|
-
|
1867
|
-
|
1868
|
-
|
1869
|
-
|
2017
|
+
Examples of query (TODO: cleanup):
|
2018
|
+
|
2019
|
+
```
|
2020
|
+
{"q":"type(file_upload OR file_delete OR file_download OR file_rename OR folder_create OR folder_delete OR folder_share OR folder_share_via_public_link)","sort":"-date"}
|
2021
|
+
|
2022
|
+
{"tag":"aspera.files.package_id=LA8OU3p8w"}
|
2023
|
+
|
1870
2024
|
# filter= 'id', 'short_summary', or 'summary'
|
1871
2025
|
# count=nnn
|
1872
2026
|
# tag=x.y.z%3Dvalue
|
1873
2027
|
# iteration_token=nnn
|
1874
2028
|
# after_time=2016-05-01T23:53:09Z
|
1875
2029
|
# active_only=true|false
|
1876
|
-
|
2030
|
+
```
|
1877
2031
|
|
1878
2032
|
* Display node events (events)
|
1879
2033
|
|
@@ -1884,7 +2038,7 @@ $ <%=cmd%> aoc admin res node --secret=_secret_ v3 events
|
|
1884
2038
|
* display members of a workspace
|
1885
2039
|
|
1886
2040
|
```
|
1887
|
-
$ <%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"
|
2041
|
+
$ <%=cmd%> aoc admin res workspace_membership list --fields=member_type,manager,member.email --query=@json:'{"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
|
1888
2042
|
:.............:.........:..................................:
|
1889
2043
|
: member_type : manager : member.email :
|
1890
2044
|
:.............:.........:..................................:
|
@@ -1905,27 +2059,27 @@ other query parameters:
|
|
1905
2059
|
|
1906
2060
|
* <a name="aoc_sample_member"></a>add all members of a workspace to another workspace
|
1907
2061
|
|
1908
|
-
a-
|
2062
|
+
a- Get id of first workspace
|
1909
2063
|
|
1910
2064
|
```
|
1911
2065
|
WS1='First Workspace'
|
1912
2066
|
WS1ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS1"'"}' --select=@json:'{"name":"'"$WS1"'"}' --fields=id --format=csv)
|
1913
2067
|
```
|
1914
2068
|
|
1915
|
-
b-
|
2069
|
+
b- Get id of second workspace
|
1916
2070
|
|
1917
2071
|
```
|
1918
2072
|
WS2='Second Workspace'
|
1919
2073
|
WS2ID=$(<%=cmd%> aoc admin res workspace list --query=@json:'{"q":"'"$WS2"'"}' --select=@json:'{"name":"'"$WS2"'"}' --fields=id --format=csv)
|
1920
2074
|
```
|
1921
2075
|
|
1922
|
-
c-
|
2076
|
+
c- Extract membership information
|
1923
2077
|
|
1924
2078
|
```
|
1925
|
-
$ <%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"
|
2079
|
+
$ <%=cmd%> aoc admin res workspace_membership list --fields=manager,member_id,member_type,workspace_id --query=@json:'{"workspace_id":'"$WS1ID"'}' --format=jsonpp > ws1_members.json
|
1926
2080
|
```
|
1927
2081
|
|
1928
|
-
d-
|
2082
|
+
d- Convert to creation data for second workspace:
|
1929
2083
|
|
1930
2084
|
```
|
1931
2085
|
grep -Eve '(direct|effective_manager|_count|storage|"id")' ws1_members.json|sed '/workspace_id/ s/"'"$WS1ID"'"/"'"$WS2ID"'"/g' > ws2_members.json
|
@@ -1937,16 +2091,16 @@ or, using jq:
|
|
1937
2091
|
jq '[.[] | {member_type,member_id,workspace_id,manager,workspace_id:"'"$WS2ID"'"}]' ws1_members.json > ws2_members.json
|
1938
2092
|
```
|
1939
2093
|
|
1940
|
-
e-
|
2094
|
+
e- Add members to second workspace
|
1941
2095
|
|
1942
2096
|
```
|
1943
2097
|
$ <%=cmd%> aoc admin res workspace_membership create --bulk=yes @json:@file:ws2_members.json
|
1944
2098
|
```
|
1945
2099
|
|
1946
|
-
*
|
2100
|
+
* Get users who did not log since a date
|
1947
2101
|
|
1948
2102
|
```
|
1949
|
-
$ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"
|
2103
|
+
$ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:<2018-05-28"}'
|
1950
2104
|
:...............................:
|
1951
2105
|
: email :
|
1952
2106
|
:...............................:
|
@@ -1955,10 +2109,10 @@ $ <%=cmd%> aoc admin res user list --fields=email --query=@json:'{"per_page":100
|
|
1955
2109
|
:...............................:
|
1956
2110
|
```
|
1957
2111
|
|
1958
|
-
*
|
2112
|
+
* List "Limited" users
|
1959
2113
|
|
1960
2114
|
```
|
1961
|
-
$ <%=cmd%> aoc admin res user list --fields=email --
|
2115
|
+
$ <%=cmd%> aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
|
1962
2116
|
```
|
1963
2117
|
|
1964
2118
|
* Perform a multi Gbps transfer between two remote shared folders
|
@@ -2001,6 +2155,7 @@ $ <%=cmd%> -Paoc_show aoc files transfer --from-folder='IBM Cloud SJ' --to-folde
|
|
2001
2155
|
```
|
2002
2156
|
|
2003
2157
|
* create registration key to register a node
|
2158
|
+
|
2004
2159
|
```
|
2005
2160
|
$ <%=cmd%> aoc admin res admin/client create @json:'{"data":{"name":"laurentnode","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}' --fields=token --format=csv
|
2006
2161
|
jfqslfdjlfdjfhdjklqfhdkl
|
@@ -2020,6 +2175,19 @@ $ <%=cmd%> aoc admin res admin/client list --fields=id --format=csv|<%=cmd%> aoc
|
|
2020
2175
|
+-----+---------+
|
2021
2176
|
```
|
2022
2177
|
|
2178
|
+
* List packages in a given shared inbox
|
2179
|
+
|
2180
|
+
First retrieve the id of the shared inbox, and then list packages with the appropriate filter.
|
2181
|
+
(To find out available filters, consult the API definition, or use the web interface in developer mode).
|
2182
|
+
|
2183
|
+
Note that when no query is provided, the query used by default is: `{"archived":false,"exclude_dropbox_packages":true,"has_content":true,"received":true}`. The workspace id is added if not already present in the query.
|
2184
|
+
|
2185
|
+
```
|
2186
|
+
shbxid=$(ascli aoc user shared_inboxes --select=@json:'{"dropbox.name":"My Shared Inbox"}' --format=csv --fields=dropbox_id --display=data)
|
2187
|
+
|
2188
|
+
ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
|
2189
|
+
```
|
2190
|
+
|
2023
2191
|
## Shared folders
|
2024
2192
|
|
2025
2193
|
* list shared folders in node
|
@@ -2052,7 +2220,7 @@ Notes:
|
|
2052
2220
|
|
2053
2221
|
* the `value` parameter can contain any supported package creation parameter. Refer to the AoC package creation API, or display an existing package to find attributes.
|
2054
2222
|
* to provide the list of recipients, use fields: "recipients" and/or "bcc_recipients". <%=cmd%> will resolve the list of email addresses to expected user ids.
|
2055
|
-
* a
|
2223
|
+
* a recipient can be a shared inbox, in this case just use the name of the shared inbox as recipient.
|
2056
2224
|
* If a recipient is not already registered and the workspace allows external users, then the package is sent to an external user, and
|
2057
2225
|
* if the option `new_user_option` is `@json:{"package_contact":true}` (default), then a public link is sent and the external user does not need to create an account.
|
2058
2226
|
* if the option `new_user_option` is `@json:{}`, then external users are invited to join the workspace
|
@@ -2061,7 +2229,13 @@ Examples:
|
|
2061
2229
|
|
2062
2230
|
```
|
2063
2231
|
$ <%=cmd%> aoc package send --value=@json:'{"name":"my title","note":"my note","recipients":["laurent.martin.aspera@fr.ibm.com","other@example.com"]}' --sources=@args my_file.dat
|
2232
|
+
```
|
2233
|
+
|
2234
|
+
```
|
2064
2235
|
$ <%=cmd%> aoc package send --value=@json:'{"name":"my file in shared inbox","recipients":["The Shared Inbox"]}' my_file.dat --ts=@json:'{"target_rate_kbps":100000}'
|
2236
|
+
```
|
2237
|
+
|
2238
|
+
```
|
2065
2239
|
$ <%=cmd%> aoc package send --workspace=eudemo --value=@json:'{"name":"my pack title","recipients":["Shared Inbox Name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' ~/Documents/Samples/200KB.1
|
2066
2240
|
```
|
2067
2241
|
|
@@ -2077,7 +2251,7 @@ $ <%=cmd%> aoc packages recv --id=ALL --once-only=yes --lock-port=12345
|
|
2077
2251
|
* `--once-only=yes` keeps memory of any downloaded package in persistency files located in the configuration folder.
|
2078
2252
|
* `--lock-port=12345` ensures that only one instance is started at the same time, to avoid collisions
|
2079
2253
|
|
2080
|
-
Typically, one would
|
2254
|
+
Typically, one would execute this command on a regular basis, using the method of your choice:
|
2081
2255
|
|
2082
2256
|
* Windows: [Task Scheduler](https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page)
|
2083
2257
|
* Linux/Unix: [cron](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
|
@@ -2085,45 +2259,45 @@ Typically, one would regularly execute this command on a regular basis, using th
|
|
2085
2259
|
|
2086
2260
|
## Download Files
|
2087
2261
|
|
2088
|
-
Download of files is straightforward with a specific syntax for the `
|
2262
|
+
Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
|
2089
2263
|
|
2090
2264
|
* if only one source is provided, it is downloaded
|
2091
2265
|
* if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
|
2092
2266
|
|
2093
2267
|
## Find Files
|
2094
2268
|
|
2095
|
-
The command `
|
2269
|
+
The command `aoc files find [--value=expression]` will recursively scan storage to find files matching the expression criteria. It works also on node resource using the v4 command. (see examples)
|
2096
2270
|
|
2097
2271
|
The expression can be of 3 formats:
|
2098
2272
|
|
2099
|
-
* empty (default) : all files, equivalent to: `exec:true`
|
2100
|
-
* not starting with `exec:` : the expression is a regular expression, using ruby
|
2273
|
+
* empty (default) : all files, equivalent to value: `exec:true`
|
2274
|
+
* not starting with `exec:` : the expression is a regular expression, using [Ruby Regex](https://ruby-doc.org/core/Regexp.html) syntax. equivalent to value: `exec:f['name'].match(/expression/)`
|
2101
2275
|
|
2102
2276
|
For instance, to find files with a special extension, use `--value='\.myext$'`
|
2103
2277
|
|
2104
|
-
* starting with `exec:` : the
|
2278
|
+
* starting with `exec:` : the Ruby code after the prefix is executed for each entry found. The entry variable name is `f`. The file is displayed if the result of the expression is true;
|
2105
2279
|
|
2106
|
-
Examples of expressions: (
|
2280
|
+
Examples of expressions: (using like this: `--value=exec:'<expression>'`)
|
2107
2281
|
|
2108
|
-
*
|
2282
|
+
* Find files more recent than 100 days
|
2109
2283
|
|
2110
2284
|
```
|
2111
2285
|
f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100
|
2112
2286
|
```
|
2113
2287
|
|
2114
|
-
*
|
2288
|
+
* Find files older than 1 year on a given node and store in file list
|
2115
2289
|
|
2116
2290
|
```
|
2117
2291
|
$ <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 find / --fields=path --value='exec:f["type"].eql?("file") and (DateTime.now-DateTime.parse(f["modified_time"]))<100' --format=csv > my_file_list.txt
|
2118
2292
|
```
|
2119
2293
|
|
2120
|
-
*
|
2294
|
+
* Delete the files, one by one
|
2121
2295
|
|
2122
2296
|
```
|
2123
2297
|
$ cat my_file_list.txt|while read path;do echo <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v4 delete "$path" ;done
|
2124
2298
|
```
|
2125
2299
|
|
2126
|
-
*
|
2300
|
+
* Delete the files in bulk
|
2127
2301
|
|
2128
2302
|
```
|
2129
2303
|
cat my_file_list.txt | <%=cmd%> aoc admin res node --name='my node name' --secret='my secret' v3 delete @lines:@stdin:
|
@@ -2258,7 +2432,7 @@ Explanation:
|
|
2258
2432
|
* `|` the standard output of the first command is fed into the second one
|
2259
2433
|
* `-Porg2 aoc` use Aspera on Cloud plugin and load credentials for `org2`
|
2260
2434
|
* `files upload mysourcefile` upload the file named `mysourcefile` (located in `org1`)
|
2261
|
-
* `--transfer=node` use transfer agent type `node` instead of default `direct`
|
2435
|
+
* `--transfer=node` use transfer agent type `node` instead of default [`direct`](#direct)
|
2262
2436
|
* `--transfer-info=@json:@stdin:` provide `node` transfer agent information, i.e. node API credentials, those are expected in JSON format and read from standard input
|
2263
2437
|
|
2264
2438
|
Note that when using a POSIX shell, another possibility to write `cmd1 | cmd2 --transfer-info=@json:stdin:` is `cmd2 --transfer-info=@json:$(cmd1)` instead of ``
|
@@ -2307,6 +2481,14 @@ This plugin works at FASP level (SSH/ascp/ascmd) and does not use the node API.
|
|
2307
2481
|
|
2308
2482
|
Both password and SSH keys auth are supported.
|
2309
2483
|
|
2484
|
+
If not username is provided, the default transfer user `xfer` is used.
|
2485
|
+
|
2486
|
+
If no ssh password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used.
|
2487
|
+
|
2488
|
+
```
|
2489
|
+
$ <%=cmd%> server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
|
2490
|
+
```
|
2491
|
+
|
2310
2492
|
Multiple SSH key paths can be provided. The value of the parameter `ssh_keys` can be a single value or an array. Each value is a path to a private key and is expanded ("~" is replaced with the user's home folder).
|
2311
2493
|
|
2312
2494
|
Examples:
|
@@ -2344,7 +2526,7 @@ This means that you dont have such an ssh agent running:
|
|
2344
2526
|
$ <%=cmd%> server --ssh-options=@ruby:'{use_agent: false}' ...
|
2345
2527
|
```
|
2346
2528
|
|
2347
|
-
This can also be set as default using a preset
|
2529
|
+
This can also be set as default using a preset.
|
2348
2530
|
|
2349
2531
|
## Example
|
2350
2532
|
|
@@ -2364,18 +2546,36 @@ This creates a <%=prst%> "aspera_demo_server" and set it as default for applicat
|
|
2364
2546
|
|
2365
2547
|
This plugin gives access to capabilities provided by HSTS node API.
|
2366
2548
|
|
2367
|
-
##
|
2549
|
+
## File Operations
|
2368
2550
|
|
2369
2551
|
It is possible to:
|
2370
2552
|
* browse
|
2371
2553
|
* transfer (upload / download)
|
2372
2554
|
* ...
|
2373
2555
|
|
2556
|
+
For transfers, it is possible to control how transfer is authorized using option: `token_type`:
|
2557
|
+
|
2558
|
+
* `aspera` : api `<upload|download>_setup` is called to create the transfer spec including the Aspera token
|
2559
|
+
* `basic` : transfer spec is created like this:
|
2560
|
+
|
2561
|
+
```
|
2562
|
+
{
|
2563
|
+
"remote_host": address of node url,
|
2564
|
+
"remote_user": "xfer",
|
2565
|
+
"ssh_port": 33001,
|
2566
|
+
"token": "Basic <base 64 encoded user/pass>",
|
2567
|
+
"direction": send/recv
|
2568
|
+
}
|
2569
|
+
```
|
2570
|
+
|
2571
|
+
* `hybrid` : same as `aspera`, but token is replaced with basic token like `basic`
|
2572
|
+
|
2374
2573
|
## Central
|
2375
2574
|
|
2376
2575
|
The central subcommand uses the "reliable query" API (session and file). It allows listing transfer sessions and transfered files.
|
2377
2576
|
|
2378
2577
|
Filtering can be applied:
|
2578
|
+
|
2379
2579
|
```
|
2380
2580
|
$ <%=cmd%> node central file list
|
2381
2581
|
```
|
@@ -2401,7 +2601,6 @@ Refer to [Aspera documentation](https://download.asperasoft.com/download/docs/en
|
|
2401
2601
|
* Start watchd and watchfolderd services running as a system user having access to files
|
2402
2602
|
* configure a watchfolder to define automated transfers
|
2403
2603
|
|
2404
|
-
|
2405
2604
|
```
|
2406
2605
|
$ <%=cmd%> node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
|
2407
2606
|
$ <%=cmd%> node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
|
@@ -2719,7 +2918,7 @@ $ <%=cmd%> cos node info
|
|
2719
2918
|
$ <%=cmd%> cos node upload 'faux:///sample1G?1g'
|
2720
2919
|
```
|
2721
2920
|
|
2722
|
-
Note: we generate a dummy file `sample1G`
|
2921
|
+
Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
|
2723
2922
|
|
2724
2923
|
# Plugin: IBM Aspera Sync
|
2725
2924
|
|
@@ -2816,8 +3015,10 @@ yum install -y ImageMagick optipng
|
|
2816
3015
|
|
2817
3016
|
### Video: FFmpeg
|
2818
3017
|
|
3018
|
+
The easiest method is to download and install the latest released version of ffmpeg with static libraries from [https://johnvansickle.com/ffmpeg/](https://johnvansickle.com/ffmpeg/)
|
3019
|
+
|
2819
3020
|
```
|
2820
|
-
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt &&
|
3021
|
+
curl -s https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz|(mkdir -p /opt && cd /opt && rm -f ffmpeg /usr/bin/{ffmpeg,ffprobe} && rm -fr ffmpeg-*-amd64-static && tar xJvf - && ln -s ffmpeg-* ffmpeg && ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin)
|
2821
3022
|
```
|
2822
3023
|
|
2823
3024
|
### Office: Unoconv and Libreoffice
|
@@ -3115,7 +3316,7 @@ Note that there are special "extended" [_transfer-spec_](#transferspec) paramete
|
|
3115
3316
|
* `EX_loglevel` to change log level of the tool
|
3116
3317
|
* `EX_file_list_folder` to set the folder used to store (exclusively, because of garbage collection) generated file lists. By default it is `[system tmp folder]/[username]_asession_filelists`
|
3117
3318
|
|
3118
|
-
Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the
|
3319
|
+
Note that in addition, many "EX_" [_transfer-spec_](#transferspec) parameters are supported for the [`direct`](#direct) transfer agent (used by `asession`), refer to section [_transfer-spec_](#transferspec).
|
3119
3320
|
|
3120
3321
|
## Comparison of interfaces
|
3121
3322
|
|
@@ -3158,7 +3359,7 @@ Nodejs: [https://www.npmjs.com/package/aspera](https://www.npmjs.com/package/asp
|
|
3158
3359
|
|
3159
3360
|
```
|
3160
3361
|
$ asession -h
|
3161
|
-
<%=
|
3362
|
+
<%=File.read(ENV["INCL_ASESSION"])%>
|
3162
3363
|
```
|
3163
3364
|
|
3164
3365
|
# Hot folder
|
@@ -3196,7 +3397,7 @@ Note that:
|
|
3196
3397
|
|
3197
3398
|
* <%=tool%> takes transfer parameters exclusively as a transfer_spec, with `--ts` parameter.
|
3198
3399
|
* most, but not all native ascp arguments are available as standard transfer_spec parameters
|
3199
|
-
* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the
|
3400
|
+
* native ascp arguments can be provided with the [_transfer-spec_](#transferspec) parameter: EX_ascp_args (array), only for the [`direct`](#direct) transfer agent (not connect or node)
|
3200
3401
|
|
3201
3402
|
### server side and configuration
|
3202
3403
|
|
@@ -3215,7 +3416,7 @@ $ <%=cmd%> server upload source_hot --to-folder=/Upload/target_hot --lock-port=1
|
|
3215
3416
|
|
3216
3417
|
```
|
3217
3418
|
|
3218
|
-
The local (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
3419
|
+
The local folder (here, relative path: source_hot) is sent (upload) to basic fasp server, source files are deleted after transfer. growing files will be sent only once they dont grow anymore (based ona 8 second cooloff period). If a transfer takes more than the execution period, then the subsequent execution is skipped (lock-port).
|
3219
3420
|
|
3220
3421
|
# Aspera Health check and Nagios
|
3221
3422
|
|
@@ -3292,9 +3493,19 @@ So, it evolved into <%=tool%>:
|
|
3292
3493
|
|
3293
3494
|
# Changes (Release notes)
|
3294
3495
|
|
3295
|
-
* <%=
|
3496
|
+
* <%=gemspec.version.to_s%>
|
3497
|
+
|
3498
|
+
* new: `aoc packages list` add possibility to add filter with option `query`
|
3499
|
+
* new: `aoc admin res xxx list` now get all items by default #50
|
3500
|
+
* new: `preset` option can specify name or hash value
|
3501
|
+
* new: `node` plugin accepts bearer token and access key as credential
|
3502
|
+
* new: `node` option `token_type` allows using basic token in addition to aspera type.
|
3503
|
+
* change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
|
3504
|
+
* change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
|
3505
|
+
|
3506
|
+
* 4.3.0
|
3296
3507
|
|
3297
|
-
* new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on `direct` transfer agent.
|
3508
|
+
* new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#direct) transfer agent.
|
3298
3509
|
* new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
|
3299
3510
|
|
3300
3511
|
* 4.2.2
|
@@ -3319,20 +3530,19 @@ So, it evolved into <%=tool%>:
|
|
3319
3530
|
* new: command `aoc remind` to receive organization membership by email
|
3320
3531
|
* new: in `preview` option `value` to filter out on file name
|
3321
3532
|
* new: `initdemo` to initialize for demo server
|
3322
|
-
* new: `direct` transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
|
3533
|
+
* new: [`direct`](#direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
|
3323
3534
|
* fix: on Windows `conf ascp use` expects ascp.exe
|
3324
3535
|
* fix: (break) multi_session_threshold is Integer, not String
|
3325
3536
|
* fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
|
3326
3537
|
* fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
|
3327
3538
|
* change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
|
3328
|
-
* change: (break) parameters for resume in `transfer-info` for `direct` are now in sub-key `"resume"`
|
3539
|
+
* change: (break) parameters for resume in `transfer-info` for [`direct`](#direct) are now in sub-key `"resume"`
|
3329
3540
|
|
3330
3541
|
* 4.1.0
|
3331
3542
|
|
3332
|
-
|
3333
|
-
|
3334
|
-
|
3335
|
-
* new: renamed command `nagios_check` to `health`
|
3543
|
+
* fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
|
3544
|
+
* new: update documentation with regard to offline and docker installation
|
3545
|
+
* new: renamed command `nagios_check` to `health`
|
3336
3546
|
* new: agent `http_gw` now supports upload
|
3337
3547
|
* new: added option `sdk_url` to install SDK from local file for offline install
|
3338
3548
|
* new: check new gem version periodically
|
@@ -3345,7 +3555,7 @@ So, it evolved into <%=tool%>:
|
|
3345
3555
|
|
3346
3556
|
* 4.0.0
|
3347
3557
|
|
3348
|
-
* now available as open source at [<%=
|
3558
|
+
* now available as open source at [<%=gemspec.homepage%>](<%=gemspec.homepage%>) with general cleanup
|
3349
3559
|
* changed default tool name from `mlia` to `ascli`
|
3350
3560
|
* changed `aspera` command to `aoc`
|
3351
3561
|
* changed gem name from `asperalm` to `aspera-cli`
|
@@ -3383,8 +3593,8 @@ So, it evolved into <%=tool%>:
|
|
3383
3593
|
* allow bulk delete in `aspera files` with option `bulk=yes`
|
3384
3594
|
* fix getting connect versions
|
3385
3595
|
* added section for Aix
|
3386
|
-
* support all ciphers for `
|
3387
|
-
* added transfer spec param `apply_local_docroot` for `
|
3596
|
+
* support all ciphers for [`direct`](#direct) agent (including gcm, etc..)
|
3597
|
+
* added transfer spec param `apply_local_docroot` for [`direct`](#direct)
|
3388
3598
|
|
3389
3599
|
* 0.11.4
|
3390
3600
|
|
@@ -3439,7 +3649,7 @@ So, it evolved into <%=tool%>:
|
|
3439
3649
|
* 0.10.12
|
3440
3650
|
|
3441
3651
|
* added support for AoC node registration keys
|
3442
|
-
* replaced option : `local_resume` with `transfer_info` for agent `direct`
|
3652
|
+
* replaced option : `local_resume` with `transfer_info` for agent [`direct`](#direct)
|
3443
3653
|
* Transfer agent is no more a Singleton instance, but only one is used in CLI
|
3444
3654
|
* `@incps` : new extended value modifier
|
3445
3655
|
* ATS: no more provides access keys secrets: now user must provide it
|
@@ -3489,9 +3699,9 @@ So, it evolved into <%=tool%>:
|
|
3489
3699
|
|
3490
3700
|
* 0.10.2
|
3491
3701
|
|
3492
|
-
|
3493
|
-
|
3494
|
-
|
3702
|
+
* updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
|
3703
|
+
* added doc for "cargo" like actions
|
3704
|
+
* added doc for multi-session
|
3495
3705
|
|
3496
3706
|
* 0.10.1
|
3497
3707
|
|
@@ -3519,22 +3729,22 @@ So, it evolved into <%=tool%>:
|
|
3519
3729
|
|
3520
3730
|
* 0.9.33
|
3521
3731
|
|
3522
|
-
|
3523
|
-
|
3524
|
-
|
3525
|
-
|
3732
|
+
* new command to display basic token of node
|
3733
|
+
* new command to display bearer token of node in AoC
|
3734
|
+
* the --fields= option, support +_fieldname_ to add a field to default fields
|
3735
|
+
* many small changes
|
3526
3736
|
|
3527
3737
|
* 0.9.32
|
3528
3738
|
|
3529
|
-
|
3530
|
-
|
3531
|
-
|
3739
|
+
* all Faspex public links are now supported
|
3740
|
+
* removed faspex operation recv_publink
|
3741
|
+
* replaced with option `link` (consistent with AoC)
|
3532
3742
|
|
3533
3743
|
* 0.9.31
|
3534
3744
|
|
3535
|
-
|
3536
|
-
|
3537
|
-
|
3745
|
+
* added more support for public link: receive and send package, to user or dropbox and files view.
|
3746
|
+
* delete expired file lists
|
3747
|
+
* changed text table gem from text-table to terminal-table because it supports multiline values
|
3538
3748
|
|
3539
3749
|
* 0.9.27
|
3540
3750
|
|
@@ -3554,149 +3764,149 @@ So, it evolved into <%=tool%>:
|
|
3554
3764
|
|
3555
3765
|
* 0.9.24
|
3556
3766
|
|
3557
|
-
|
3558
|
-
|
3767
|
+
* fix bug where AoC node to node transfer did not work
|
3768
|
+
* fix bug on error if ED25519 private key is defined in .ssh
|
3559
3769
|
|
3560
3770
|
* 0.9.23
|
3561
3771
|
|
3562
|
-
|
3563
|
-
|
3772
|
+
* defined REST error handlers, more error conditions detected
|
3773
|
+
* commands to select specific ascp location
|
3564
3774
|
|
3565
3775
|
* 0.9.21
|
3566
3776
|
|
3567
|
-
|
3568
|
-
|
3777
|
+
* supports simplified wizard using global client
|
3778
|
+
* only ascp binary is required, other SDK (keys) files are now generated
|
3569
3779
|
|
3570
3780
|
* 0.9.20
|
3571
3781
|
|
3572
|
-
|
3573
|
-
|
3574
|
-
|
3782
|
+
* improved wizard (prepare for AoC global client id)
|
3783
|
+
* preview generator: addedoption : --skip-format=<png,mp4>
|
3784
|
+
* removed outdated pictures from this doc
|
3575
3785
|
|
3576
3786
|
* 0.9.19
|
3577
3787
|
|
3578
|
-
|
3788
|
+
* added command aspera bearer --scope=xx
|
3579
3789
|
|
3580
3790
|
* 0.9.18
|
3581
3791
|
|
3582
|
-
|
3792
|
+
* enhanced aspera admin events to support query
|
3583
3793
|
|
3584
3794
|
* 0.9.16
|
3585
3795
|
|
3586
|
-
|
3587
|
-
|
3796
|
+
* AoC transfers are now reported in activity app
|
3797
|
+
* new interface for Rest class authentication (keep backward compatibility)
|
3588
3798
|
|
3589
3799
|
* 0.9.15
|
3590
3800
|
|
3591
|
-
|
3592
|
-
|
3801
|
+
* new feature: "find" command in aspera files
|
3802
|
+
* sample code for transfer API
|
3593
3803
|
|
3594
3804
|
* 0.9.12
|
3595
3805
|
|
3596
|
-
|
3597
|
-
|
3806
|
+
* add nagios commands
|
3807
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
3598
3808
|
|
3599
3809
|
* 0.9.11
|
3600
3810
|
|
3601
|
-
|
3602
|
-
|
3811
|
+
* Breaking change: @stdin is now @stdin:
|
3812
|
+
* support of ATS for IBM Cloud, removed old version based on aspera id
|
3603
3813
|
|
3604
3814
|
|
3605
3815
|
* 0.9.10
|
3606
3816
|
|
3607
|
-
|
3608
|
-
|
3609
|
-
|
3610
|
-
|
3817
|
+
* Breaking change: parameter transfer-node becomes more generic: transfer-info
|
3818
|
+
* Display SaaS storage usage with command: aspera admin res node --id=nn info
|
3819
|
+
* cleaner way of specifying source file list for transfers
|
3820
|
+
* Breaking change: replaced download_mode option with http_download action
|
3611
3821
|
|
3612
3822
|
* 0.9.9
|
3613
3823
|
|
3614
|
-
|
3615
|
-
|
3824
|
+
* Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
|
3825
|
+
* Now supports "cargo" for Aspera on Cloud (automatic package download)
|
3616
3826
|
|
3617
3827
|
* 0.9.8
|
3618
3828
|
|
3619
|
-
|
3620
|
-
|
3829
|
+
* Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
|
3830
|
+
* AoC: share to share transfer with command "transfer"
|
3621
3831
|
|
3622
3832
|
* 0.9.7
|
3623
3833
|
|
3624
|
-
|
3625
|
-
|
3626
|
-
|
3627
|
-
|
3628
|
-
|
3834
|
+
* homogeneous [_transfer-spec_](#transferspec) for `node` and [`direct`](#direct) transfer agents
|
3835
|
+
* preview persistency goes to unique file by default
|
3836
|
+
* catch mxf extension in preview as video
|
3837
|
+
* Faspex: possibility to download all paclages by specifying id=ALL
|
3838
|
+
* Faspex: to come: cargo-like function to download only new packages with id=NEW
|
3629
3839
|
|
3630
3840
|
* 0.9.6
|
3631
3841
|
|
3632
|
-
|
3633
|
-
|
3842
|
+
* Breaking change: `@param:`is now `@preset:` and is generic
|
3843
|
+
* AoC: added command to display current workspace information
|
3634
3844
|
|
3635
3845
|
* 0.9.5
|
3636
3846
|
|
3637
|
-
|
3638
|
-
|
3847
|
+
* new parameter: new_user_option used to choose between public_link and invite of external users.
|
3848
|
+
* fixed bug in wizard, and wizard uses now product detection
|
3639
3849
|
|
3640
3850
|
* 0.9.4
|
3641
3851
|
|
3642
|
-
|
3643
|
-
|
3644
|
-
|
3852
|
+
* Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
|
3853
|
+
* AoC Package send supports external users
|
3854
|
+
* new command to export AoC config to Aspera CLI config
|
3645
3855
|
|
3646
3856
|
* 0.9.3
|
3647
3857
|
|
3648
|
-
|
3649
|
-
|
3650
|
-
|
3651
|
-
|
3652
|
-
|
3653
|
-
|
3858
|
+
* REST error message show host and code
|
3859
|
+
* option for quiet display
|
3860
|
+
* modified transfer interface and allow token re-generation on error
|
3861
|
+
* async add admin command
|
3862
|
+
* async add db parameters
|
3863
|
+
* Breaking change: new option "sources" to specify files to transfer
|
3654
3864
|
|
3655
3865
|
* 0.9.2
|
3656
3866
|
|
3657
|
-
|
3867
|
+
* Breaking change: changed AoC package creation to match API, see AoC section
|
3658
3868
|
|
3659
3869
|
* 0.9.1
|
3660
3870
|
|
3661
|
-
|
3871
|
+
* Breaking change: changed faspex package creation to match API, see Faspex section
|
3662
3872
|
|
3663
3873
|
* 0.9
|
3664
3874
|
|
3665
|
-
|
3666
|
-
|
3875
|
+
* Renamed the CLI from aslmcli to <%=tool%>
|
3876
|
+
* Automatic rename and conversion of former config folder from aslmcli to <%=tool%>
|
3667
3877
|
|
3668
3878
|
* 0.7.6
|
3669
3879
|
|
3670
|
-
|
3880
|
+
* add "sync" plugin
|
3671
3881
|
|
3672
3882
|
* 0.7
|
3673
3883
|
|
3674
|
-
|
3675
|
-
|
3676
|
-
|
3677
|
-
|
3678
|
-
|
3884
|
+
* Breaking change: AoC package recv take option if for package instead of argument.
|
3885
|
+
* Breaking change: Rest class and Oauth class changed init parameters
|
3886
|
+
* AoC: receive package from public link
|
3887
|
+
* select by col value on output
|
3888
|
+
* added rename (AoC, node)
|
3679
3889
|
|
3680
3890
|
* 0.6.19
|
3681
3891
|
|
3682
3892
|
Breaking change:
|
3683
3893
|
|
3684
|
-
|
3685
|
-
|
3686
|
-
|
3687
|
-
|
3688
|
-
|
3689
|
-
|
3690
|
-
|
3691
|
-
|
3894
|
+
* ats server list provisioned → ats cluster list
|
3895
|
+
* ats server list clouds → ats cluster clouds
|
3896
|
+
* ats server list instance --cloud=x --region=y → ats cluster show --cloud=x --region=y
|
3897
|
+
* ats server id xxx → ats cluster show --id=xxx
|
3898
|
+
* ats subscriptions → ats credential subscriptions
|
3899
|
+
* ats api_key repository list → ats credential cache list
|
3900
|
+
* ats api_key list → ats credential list
|
3901
|
+
* ats access_key id xxx → ats access_key --id=xxx
|
3692
3902
|
|
3693
3903
|
* 0.6.18
|
3694
3904
|
|
3695
|
-
|
3905
|
+
* some commands take now --id option instead of id command.
|
3696
3906
|
|
3697
3907
|
* 0.6.15
|
3698
3908
|
|
3699
|
-
|
3909
|
+
* Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in <%=prst%> "default".
|
3700
3910
|
|
3701
3911
|
# BUGS, FEATURES, CONTRIBUTION
|
3702
3912
|
|
@@ -3749,8 +3959,8 @@ References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "C
|
|
3749
3959
|
|
3750
3960
|
* remove rest and oauth classes and use ruby standard gems:
|
3751
3961
|
|
3752
|
-
|
3753
|
-
|
3962
|
+
* oauth
|
3963
|
+
* https://github.com/rest-client/rest-client
|
3754
3964
|
|
3755
3965
|
* use Thor or any standard Ruby CLI manager
|
3756
3966
|
|