aspera-cli 4.7.0 → 4.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (96) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/README.md +1267 -999
  4. data/bin/ascli +20 -1
  5. data/bin/asession +37 -34
  6. data/docs/test_env.conf +7 -3
  7. data/examples/aoc.rb +13 -12
  8. data/examples/dascli +23 -0
  9. data/examples/faspex4.rb +34 -29
  10. data/examples/{transfer.rb → node.rb} +31 -59
  11. data/examples/server.rb +93 -0
  12. data/lib/aspera/aoc.rb +153 -143
  13. data/lib/aspera/ascmd.rb +56 -45
  14. data/lib/aspera/ats_api.rb +9 -6
  15. data/lib/aspera/cli/basic_auth_plugin.rb +18 -16
  16. data/lib/aspera/cli/extended_value.rb +33 -30
  17. data/lib/aspera/cli/formater.rb +105 -111
  18. data/lib/aspera/cli/info.rb +3 -2
  19. data/lib/aspera/cli/listener/line_dump.rb +1 -0
  20. data/lib/aspera/cli/listener/logger.rb +1 -0
  21. data/lib/aspera/cli/listener/progress.rb +13 -12
  22. data/lib/aspera/cli/listener/progress_multi.rb +21 -20
  23. data/lib/aspera/cli/main.rb +110 -90
  24. data/lib/aspera/cli/manager.rb +99 -88
  25. data/lib/aspera/cli/plugin.rb +98 -39
  26. data/lib/aspera/cli/plugins/alee.rb +6 -5
  27. data/lib/aspera/cli/plugins/aoc.rb +581 -450
  28. data/lib/aspera/cli/plugins/ats.rb +84 -83
  29. data/lib/aspera/cli/plugins/bss.rb +30 -27
  30. data/lib/aspera/cli/plugins/config.rb +488 -397
  31. data/lib/aspera/cli/plugins/console.rb +17 -15
  32. data/lib/aspera/cli/plugins/cos.rb +26 -35
  33. data/lib/aspera/cli/plugins/faspex.rb +206 -172
  34. data/lib/aspera/cli/plugins/faspex5.rb +109 -74
  35. data/lib/aspera/cli/plugins/node.rb +379 -189
  36. data/lib/aspera/cli/plugins/orchestrator.rb +71 -65
  37. data/lib/aspera/cli/plugins/preview.rb +131 -122
  38. data/lib/aspera/cli/plugins/server.rb +50 -150
  39. data/lib/aspera/cli/plugins/shares.rb +61 -27
  40. data/lib/aspera/cli/plugins/sync.rb +15 -14
  41. data/lib/aspera/cli/transfer_agent.rb +75 -64
  42. data/lib/aspera/cli/version.rb +2 -1
  43. data/lib/aspera/colors.rb +29 -28
  44. data/lib/aspera/command_line_builder.rb +50 -43
  45. data/lib/aspera/cos_node.rb +64 -38
  46. data/lib/aspera/data_repository.rb +1 -0
  47. data/lib/aspera/environment.rb +33 -10
  48. data/lib/aspera/fasp/agent_base.rb +35 -30
  49. data/lib/aspera/fasp/agent_connect.rb +35 -30
  50. data/lib/aspera/fasp/agent_direct.rb +68 -60
  51. data/lib/aspera/fasp/agent_httpgw.rb +71 -64
  52. data/lib/aspera/fasp/agent_node.rb +24 -23
  53. data/lib/aspera/fasp/agent_trsdk.rb +19 -20
  54. data/lib/aspera/fasp/error.rb +2 -1
  55. data/lib/aspera/fasp/error_info.rb +79 -68
  56. data/lib/aspera/fasp/installation.rb +130 -126
  57. data/lib/aspera/fasp/listener.rb +1 -0
  58. data/lib/aspera/fasp/parameters.rb +71 -60
  59. data/lib/aspera/fasp/parameters.yaml +69 -17
  60. data/lib/aspera/fasp/resume_policy.rb +14 -11
  61. data/lib/aspera/fasp/transfer_spec.rb +6 -5
  62. data/lib/aspera/fasp/uri.rb +25 -24
  63. data/lib/aspera/faspex_gw.rb +83 -72
  64. data/lib/aspera/hash_ext.rb +23 -13
  65. data/lib/aspera/id_generator.rb +16 -13
  66. data/lib/aspera/keychain/encrypted_hash.rb +61 -46
  67. data/lib/aspera/keychain/macos_security.rb +26 -24
  68. data/lib/aspera/log.rb +35 -39
  69. data/lib/aspera/nagios.rb +36 -28
  70. data/lib/aspera/node.rb +19 -19
  71. data/lib/aspera/oauth.rb +120 -100
  72. data/lib/aspera/open_application.rb +25 -22
  73. data/lib/aspera/persistency_action_once.rb +9 -8
  74. data/lib/aspera/persistency_folder.rb +13 -9
  75. data/lib/aspera/preview/file_types.rb +261 -266
  76. data/lib/aspera/preview/generator.rb +74 -73
  77. data/lib/aspera/preview/image_error.png +0 -0
  78. data/lib/aspera/preview/options.rb +7 -6
  79. data/lib/aspera/preview/utils.rb +30 -33
  80. data/lib/aspera/preview/video_error.png +0 -0
  81. data/lib/aspera/proxy_auto_config.rb +27 -23
  82. data/lib/aspera/rest.rb +73 -74
  83. data/lib/aspera/rest_call_error.rb +1 -0
  84. data/lib/aspera/rest_error_analyzer.rb +23 -19
  85. data/lib/aspera/rest_errors_aspera.rb +43 -40
  86. data/lib/aspera/secret_hider.rb +74 -0
  87. data/lib/aspera/ssh.rb +13 -10
  88. data/lib/aspera/sync.rb +49 -47
  89. data/lib/aspera/temp_file_manager.rb +7 -5
  90. data/lib/aspera/timer_limiter.rb +9 -8
  91. data/lib/aspera/uri_reader.rb +17 -18
  92. data/lib/aspera/web_auth.rb +17 -15
  93. data.tar.gz.sig +5 -0
  94. metadata +119 -35
  95. metadata.gz.sig +0 -0
  96. data/bin/dascli +0 -13
data/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
4
4
 
5
- Version : 4.7.0
5
+ Version : 4.9.0
6
6
 
7
7
  Laurent/2016-2022
8
8
 
@@ -19,6 +19,16 @@ Minimum required Ruby version: >= 2.4. Deprecation notice: the minimum will be 2
19
19
  [Aspera APIs on IBM developer](https://developer.ibm.com/?size=30&q=aspera&DWContentType[0]=APIs)
20
20
  [Link 2](https://developer.ibm.com/apis/catalog/?search=aspera)
21
21
 
22
+ Release notes: see [CHANGELOG.md](CHANGELOG.md)
23
+
24
+ [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5861/badge)](https://bestpractices.coreinfrastructure.org/projects/5861)
25
+
26
+ ## BUGS, FEATURES, CONTRIBUTION
27
+
28
+ Refer to [BUGS.md](BUGS.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
29
+
30
+ One can also [create one's own plugin](#createownplugin).
31
+
22
32
  ## <a id="when_to_use"></a>When to use and when not to use
23
33
 
24
34
  `ascli` is designed to be used as a command line tool to:
@@ -48,33 +58,12 @@ Using APIs (application REST API and transfer SDK) will prove to be easier to de
48
58
 
49
59
  For scripting and ad'hoc command line operations, `ascli` is perfect.
50
60
 
51
- ## <a id="parsing"></a>Notations, Shell and Command line parsing
61
+ ## Notations, Shell, Examples
52
62
 
53
63
  In examples, command line operations are shown using a shell such: `bash` or `zsh`.
54
64
 
55
65
  Command line parameters in examples beginning with `my_`, like `my_param_value` are user-provided value and not fixed value commands.
56
66
 
57
- `ascli` is typically executed in a shell, either interactively or in a script. `ascli` receives its arguments from this shell.
58
-
59
- On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh). In this environment shell command line parsing applies before `ascli` (Ruby) is executed, e.g. [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation). Ruby receives a list parameters and gives it to `ascli`. So special character handling (quotes, spaces, env vars, ...) is done in the shell.
60
-
61
- On Windows, `cmd.exe` is typically used. Windows process creation does not receive the list of arguments but just the whole line. It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
62
-
63
- * [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
64
- * [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
65
-
66
- In case of doubt of argument values after parsing test like this:
67
-
68
- ```bash
69
- ascli conf echo "Hello World" arg2 3
70
- ```
71
-
72
- ```bash
73
- "Hello World"
74
- ERROR: Argument: unprocessed values: ["arg2", "3"]
75
- ```
76
-
77
- `echo` displays the value of the first argument using ruby syntax (strings get double quotes) after command line parsing (shell) and extended value parsing (`ascli`), next command line arguments are shown in the error message.
78
67
 
79
68
  ## Quick Start
80
69
 
@@ -89,7 +78,7 @@ ascli --version
89
78
  ```
90
79
 
91
80
  ```bash
92
- 4.7.0
81
+ 4.9.0
93
82
  ```
94
83
 
95
84
  ### First use
@@ -196,11 +185,9 @@ An internet connection is required for the installation. If you don't have inter
196
185
 
197
186
  ### Docker container
198
187
 
199
- Use this method only if you know what you do, else use the standard recommended method as described here above.
188
+ This method installs a docker image that contains: Ruby, `ascli` and the FASP SDK.
200
189
 
201
- This method installs a docker image that contains: Ruby, `ascli` and the FASP sdk.
202
-
203
- The image is: [https://hub.docker.com/r/martinlaurent/ascli](https://hub.docker.com/r/martinlaurent/ascli)
190
+ The image is: <https://hub.docker.com/r/martinlaurent/ascli>. It is built from this [Dockerfile](Dockerfile).
204
191
 
205
192
  Ensure that you have Docker installed.
206
193
 
@@ -208,10 +195,16 @@ Ensure that you have Docker installed.
208
195
  docker --version
209
196
  ```
210
197
 
211
- Download the wrapping script:
198
+ An example of wrapping script is provided: `dascli`. If you have installed `ascli`, the script `dascli` can be found:
199
+
200
+ ```bash
201
+ cp $(ascli conf gem path)/../examples/dascli ascli
202
+ ```
203
+
204
+ Alternatively [download from the GIT repo](https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli) :
212
205
 
213
206
  ```bash
214
- curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/develop/bin/dascli
207
+ curl -o ascli https://raw.githubusercontent.com/IBM/aspera-cli/main/examples/dascli
215
208
  ```
216
209
 
217
210
  ```bash
@@ -224,14 +217,13 @@ Install the container image:
224
217
  ./ascli install
225
218
  ```
226
219
 
227
- Start using it !
228
-
229
- Note that the tool is run in the container, so transfers are also executed in the container, not calling host.
220
+ Note that ascli is run in the container, so transfers are also executed in the container (not calling host, like for the regular ascli).
230
221
 
231
- The wrapping script maps the container folder `/usr/src/app/config` to configuration folder `$HOME/.aspera/ascli` on host.
222
+ The wrapping script maps the folder `/usr/src/app/config` in the container to configuration folder `$HOME/.aspera/ascli` on host.
223
+ This allows having persistent configuration.
232
224
 
233
225
  To transfer to/from the native host, you will need to map a volume in docker or use the config folder (already mapped).
234
- To add local storage as a volume edit the script: `ascli` and add a `--volume` stanza.
226
+ To add local storage as a volume edit the script: `ascli` and add a `--volume` stanza near the existing one.
235
227
 
236
228
  ### <a id="ruby"></a>Ruby
237
229
 
@@ -312,11 +304,13 @@ rvm version
312
304
 
313
305
  #### Windows: Installer
314
306
 
315
- Install Latest stable Ruby using [https://rubyinstaller.org/](https://rubyinstaller.org/) :
307
+ Install Latest stable Ruby:
316
308
 
317
- * Go to "Downloads".
318
- * Select the Ruby 2 version "without devkit", x64 corresponding to the one recommended "with devkit". Devkit is not needed.
319
- * At the end of the installer uncheck the box to skip the installation of "MSys2": not needed.
309
+ * Navigate to [https://rubyinstaller.org/](https://rubyinstaller.org/) &rarr; **Downloads**.
310
+ * Download the latest Ruby installer **with devkit**. (Msys2 is needed to install some native extensions, such as `grpc`)
311
+ * Execute the installer which installs by default in: `RubyVV-x64` (VV is the version number)
312
+ * At the end of the installation procedure execute the Msys2 installer (ridk install) and select option 3 (msys and mingw)
313
+ * Install the "mime info" file as specified in [this section](#mimeinfo).
320
314
 
321
315
  #### macOS: pre-installed or `brew`
322
316
 
@@ -336,20 +330,31 @@ brew install ruby
336
330
 
337
331
  If your Linux distribution provides a standard ruby package, you can use it provided that the version is compatible (check at beginning of section).
338
332
 
339
- Example:
333
+ Example: Centos 8 Stream
340
334
 
341
335
  ```bash
342
- yum install -y ruby rubygems ruby-json
336
+ yum install make automake gcc gcc-c++ kernel-devel
337
+ yum install redhat-rpm-config
338
+ dnf module reset ruby
339
+ dnf module enable ruby:3.0
340
+ dnf module -y install ruby:3.0/common
341
+ gem install aspera-cli
343
342
  ```
344
343
 
345
- One can cleanup the whole yum-installed ruby environment like this to uninstall:
344
+ Other examples:
346
345
 
347
346
  ```bash
348
- gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
347
+ yum install -y ruby ruby-devel rubygems ruby-json
348
+ ```
349
+
350
+ ```bash
351
+ apt install -y ruby ruby-dev rubygems ruby-json
349
352
  ```
350
353
 
354
+ One can cleanup the whole yum-installed ruby environment like this to uninstall:
355
+
351
356
  ```bash
352
- yum remove -y ruby ruby-libs
357
+ gem uninstall $(ls $(gem env gemdir)/gems/|sed -e 's/-[^-]*$//'|sort -u)
353
358
  ```
354
359
 
355
360
  #### Other Unixes (AIX)
@@ -531,10 +536,208 @@ Basic usage is displayed by executing:
531
536
  ascli -h
532
537
  ```
533
538
 
534
- Refer to sections: [Usage](#usage) and [Sample Commands](#commands).
539
+ Refer to sections: [Usage](#usage).
535
540
 
536
541
  Not all `ascli` features are fully documented here, the user may explore commands on the command line.
537
542
 
543
+ ### `ascp` command line
544
+
545
+ If you want to use `ascp` directly as a command line, refer to IBM Aspera documentation of either [Desktop Client](https://www.ibm.com/docs/en/asdc), [Endpoint](https://www.ibm.com/docs/en/ahte) or [Transfer Server](https://www.ibm.com/docs/en/ahts) where [a section on `ascp` can be found](https://www.ibm.com/docs/en/ahts/4.4?topic=linux-ascp-transferring-from-command-line).
546
+
547
+ Using `ascli` with plugin `server` for command line gives advantages over ascp:
548
+
549
+ * automatic resume on error
550
+ * configuration file
551
+ * choice of transfer agents
552
+ * integrated support of multi-session
553
+
554
+ Moreover all `ascp` options are supported either through transfer spec parameters and with the possibility to provide `ascp` arguments directly when the `direct` agent is used (`EX_ascp_args`).
555
+
556
+ ### <a id="parsing"></a>Command line parsing, Special Characters
557
+
558
+ `ascli` is typically executed in a shell, either interactively or in a script.
559
+ `ascli` receives its arguments from this shell (through Operating System).
560
+
561
+ #### Shell parsing for Linux, Unix, Macos
562
+
563
+ On Linux and Unix environments, this is typically a POSIX shell (bash, zsh, ksh, sh).
564
+ In this environment the shell parses the command line, possibly replacing variables, etc...
565
+ see [bash shell operation](https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation).
566
+ Then it builds a list of arguments and then `ascli` (Ruby) is executed.
567
+ Ruby receives a list parameters from shell and gives it to `ascli`.
568
+ So special character handling (quotes, spaces, env vars, ...) is first done in the shell.
569
+
570
+ #### Shell parsing for Windows
571
+
572
+ On Windows, `cmd.exe` is typically used.
573
+ Windows process creation does not receive the list of arguments but just the whole line.
574
+ It's up to the program to parse arguments. Ruby follows the Microsoft C/C++ parameter parsing rules.
575
+
576
+ * [Windows: How Command Line Parameters Are Parsed](https://daviddeley.com/autohotkey/parameters/parameters.htm#RUBY)
577
+ * [Understand Quoting and Escaping of Windows Command Line Arguments](http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/)
578
+
579
+ #### Extended Values (JSON, Ruby, ...)
580
+
581
+ Some of the CLI parameters are expected to be [Extended Values](#extended), i.e. not a simple strings, but a complex structure (Hash, Array).
582
+ Typically, the `@json:` modifier is used, it expects a JSON string. JSON itself has some special syntax: for example `"` is used to denote strings.
583
+
584
+ #### Testing Extended Values
585
+
586
+ In case of doubt of argument values after parsing, one can test using command `config echo`. `config echo` takes exactly **one** argument which can use the [Extended Value](#extended) syntax. Unprocessed command line arguments are shown in the error message.
587
+
588
+ Example: The shell parses three arguments (as strings: `1`, `2` and `3`), so the additional two arguments are not processed by the `echo` command.
589
+
590
+ ```bash
591
+ ascli conf echo 1 2 3
592
+ ```
593
+
594
+ ```bash
595
+ "1"
596
+ ERROR: Argument: unprocessed values: ["2", "3"]
597
+ ```
598
+
599
+ `config echo` displays the value of the first argument using Ruby syntax: it surrounds a string with `"` and add `\` before special characters.
600
+ Note that it gets its value after shell command line parsing and `ascli` extended value parsing.
601
+
602
+ In the following examples (using a POSIX shell, such as `bash`), several sample commands are provided when equivalent.
603
+ For all example, most of special character handling is not specific to `ascli`: It depoends on the underlying syntax: shell , JSON, etc...
604
+ Depending on the case, a different `format` is used to display the actual value.
605
+
606
+ For example, in the simple string `Hello World`, the space character is special for the shell, so it must be escaped so that a single value is represented.
607
+
608
+ Double quotes are processed by the shell to create a single string argument.
609
+ For POSIX shells, single quotes can also be used in this case, or protext the special character ` ` (space) with a backslash.
610
+
611
+ ```bash
612
+ ascli conf echo "Hello World" --format=text
613
+ ascli conf echo 'Hello World' --format=text
614
+ ascli conf echo Hello\ World --format=text
615
+ ```
616
+
617
+ ```output
618
+ Hello World
619
+ ```
620
+
621
+ #### Using a shell variable, parsed by shell, in an extended value
622
+
623
+ To be evaluated by shell, the shell variable must not be in single quotes.
624
+ Note we use a simple variable here: the variable is not necessarily an environment variable.
625
+ Even if the variable contains spaces it makes only one argument to `ascli` because word parsing is made before variable expansion by shell.
626
+
627
+ ```bash
628
+ MYVAR="Hello World"
629
+ ascli conf echo @json:'{"title":"'$MYVAR'"}' --format=json
630
+ ascli conf echo @json:{\"title\":\"$MYVAR\"} --format=json
631
+ ```
632
+
633
+ ```json
634
+ {"title":"Hello World"}
635
+ ```
636
+
637
+ #### Double quote in strings in command line
638
+
639
+ Double quote is a shell special character.
640
+ Like any shell special character, it can be protected either by preceding with a backslash or by enclosing in a single quote.
641
+
642
+ ```bash
643
+ ascli conf echo \"
644
+ ascli conf echo '"'
645
+ ```
646
+
647
+ ```output
648
+ "
649
+ ```
650
+
651
+ Double quote in JSON is a little tricky because `"` is special both for the shell and JSON. Both shell and JSON syntax allow to protect `"`, but only the shell allows protection using single quote.
652
+
653
+ ```bash
654
+ ascli conf echo @json:'"\""' --format=text
655
+ ascli conf echo @json:\"\\\"\" --format=text
656
+ ascli conf echo @ruby:\'\"\' --format=text
657
+ ```
658
+
659
+ ```output
660
+ "
661
+ ```
662
+
663
+ Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional `\` is added to protect the `"` for JSON. But as `\` is also shell special, then it is protected by another `\`.
664
+
665
+ #### Shell and JSON or Ruby special characters in extended value
666
+
667
+ Construction of values with special characters is done like this:
668
+
669
+ * First select a syntax to represent the extended value, e.g. JSON or Ruby
670
+
671
+ * Write the expression using this syntax, for example, using JSON:
672
+
673
+ ```json
674
+ {"title":"Test \" ' & \\"}
675
+ ```
676
+
677
+ or using Ruby:
678
+
679
+ ```ruby
680
+ {"title"=>"Test \" ' & \\"}
681
+ {'title'=>%q{Test " ' & \\}}
682
+ ```
683
+
684
+ Both `"` and `\` are special characters for JSON and Ruby and can be protected with `\` (unless Ruby's extended single quote notation `%q` is used).
685
+
686
+ * Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding `\` for each character to protect, or by enclosing in single quote:
687
+
688
+ ```bash
689
+ ascli conf echo @json:{\"title\":\"Test\ \\\"\ \'\ \&\ \\\\\"} --format=json
690
+ ascli conf echo @json:'{"title":"Test \" '\'' & \\"}' --format=json
691
+ ascli conf echo @ruby:"{'title'=>%q{Test \" ' & \\\\}}" --format=json
692
+ ```
693
+
694
+ ```json
695
+ {"title":"Test \" ' & \\"}
696
+ ```
697
+
698
+ #### Reading special characters interractively
699
+
700
+ If `ascli` is used interractively (a user typing on terminal), it is easy to require the user to type values:
701
+
702
+ ```bash
703
+ ascli conf echo @ruby:"{'title'=>gets.chomp}" --format=json
704
+ ```
705
+
706
+ `gets` is Ruby's method of terminal input (terminated by `\n`), and `chomp` removes the trailing `\n`.
707
+
708
+ #### Extended value using special characters read from environmental variables or files
709
+
710
+ Using a text editor or shell: create a file `title.txt` (and env var) that contains exactly the text required: `Test " ' & \` :
711
+
712
+ ```bash
713
+ export MYTITLE='Test " '\'' & \'
714
+ echo -n $MYTITLE > title.txt
715
+ ```
716
+
717
+ Using those values will not require any escaping of characters since values do not go through shell or JSON parsing.
718
+
719
+ If the value is to be assigned directly to an option of ascli, then you can directly use the content of the file or env var using the `@file:` or `@env:` readers:
720
+
721
+ ```bash
722
+ ascli conf echo @file:title.txt --format=text
723
+ ascli conf echo @env:MYTITLE --format=text
724
+ ```
725
+
726
+ ```output
727
+ Test " ' & \
728
+ ```
729
+
730
+ If the value to be used is in a more complex structure, then the `@ruby:` modifier can be used: it allows any ruby code in expression, including reading from file or env var. In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.
731
+
732
+ ```bash
733
+ ascli conf echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
734
+ ascli conf echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
735
+ ```
736
+
737
+ ```json
738
+ {"title":"Test \" ' & \\"}
739
+ ```
740
+
538
741
  ### Arguments : Commands and options
539
742
 
540
743
  Arguments are the units of command line, as parsed by the shell, typically separated by spaces (and called "argv").
@@ -638,6 +841,7 @@ If transposition of single object is not desired, use option: `transpose_single`
638
841
 
639
842
  The style of output can be set using the `format` parameter, supporting:
640
843
 
844
+ * `text` : Value as String
641
845
  * `table` : Text table
642
846
  * `ruby` : Ruby code
643
847
  * `json` : JSON code
@@ -674,12 +878,12 @@ Output messages are categorized in 3 types:
674
878
 
675
879
  The option `display` controls the level of output:
676
880
 
677
- * `info` displays all messages
881
+ * `info` displays all messages: `info`, `data`, and `error`
678
882
  * `data` display `data` and `error` messages
679
883
  * `error` display only error messages.
680
884
 
681
- By default, secrets are shown on output.
682
- To hide secrets from results, set option `show_secrets` to `no`.
885
+ By default, secrets are removed from output: option `show_secrets` defaults to `no`, unless `display` is `data`, to allows piping results.
886
+ To hide secrets from output, set option `show_secrets` to `no`.
683
887
 
684
888
  #### Selection of output object properties
685
889
 
@@ -778,14 +982,14 @@ Note that `@incps:@json:'{"incps":["config"]}'` or `@incps:@ruby:'{"incps"=>["co
778
982
 
779
983
  ### <a id="native"></a>Structured Value
780
984
 
781
- Some options and parameters expect a _Structured Value_, i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
985
+ Some options and parameters expect a [Extended Value](#extended), i.e. a value more complex than a simple string. This is usually a Hash table or an Array, which could also contain sub structures.
782
986
 
783
- For instance, a [*transfer-spec*](#transferspec) is expected to be a _Structured Value_.
987
+ For instance, a [*transfer-spec*](#transferspec) is expected to be a [Extended Value](#extended).
784
988
 
785
989
  Structured values shall be described using the [Extended Value Syntax](#extended).
786
- A convenient way to specify a _Structured Value_ is to use the `@json:` decoder, and describe the value in JSON format. The `@ruby:` decoder can also be used. For an array of hash tables, the `@csvt:` decoder can be used.
990
+ A convenient way to specify a [Extended Value](#extended) is to use the `@json:` decoder, and describe the value in JSON format. The `@ruby:` decoder can also be used. For an array of hash tables, the `@csvt:` decoder can be used.
787
991
 
788
- It is also possible to provide a _Structured Value_ in a file using `@json:@file:<path>`
992
+ It is also possible to provide a [Extended Value](#extended) in a file using `@json:@file:<path>`
789
993
 
790
994
  ### <a id="conffolder"></a>Configuration and Persistency Folder
791
995
 
@@ -894,7 +1098,6 @@ ascli config over
894
1098
  ascli config list
895
1099
  ```
896
1100
 
897
-
898
1101
  #### <a id="lprtconf"></a>Special Option preset: config
899
1102
 
900
1103
  This preset name is reserved and contains a single key: `version`. This is the version of `ascli` which created the file.
@@ -929,6 +1132,49 @@ When `ascli` starts, it looks for the `default` Option preset and if there is a
929
1132
 
930
1133
  If no global default is set by the user, the tool will use `global_common_defaults` when setting global parameters (e.g. `conf ascp use`)
931
1134
 
1135
+ Sample commands
1136
+
1137
+ ```bash
1138
+ config ascp connect info 'Aspera Connect for Windows'
1139
+ config ascp connect list
1140
+ config ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
1141
+ config ascp connect version 'Aspera Connect for Windows' list
1142
+ config ascp connect version 'Aspera Connect for Windows' open documentation
1143
+ config ascp errors
1144
+ config ascp info --sdk-folder=Tsdk_test_dir
1145
+ config ascp install --sdk-folder=Tsdk_test_dir
1146
+ config ascp products list
1147
+ config ascp show
1148
+ config ascp spec
1149
+ config check_update
1150
+ config detect --url=https://my_aoc_org.ibmaspera.com
1151
+ config detect --url=my_faspex_url
1152
+ config detect --url=my_node_url
1153
+ config doc
1154
+ config doc transfer-parameters
1155
+ config echo 'hello'
1156
+ config echo @base64:SGVsbG8gV29ybGQK
1157
+ config echo @csvt:@stdin:
1158
+ config echo @env:USER
1159
+ config echo @lines:@stdin:
1160
+ config echo @list:,1,2,3
1161
+ config echo @uri:/etc/hosts
1162
+ config echo @uri:file:/etc/hosts
1163
+ config echo @uri:http://www.ibm.com
1164
+ config echo @uri:https://www.ibm.com
1165
+ config echo @val:@file:yo
1166
+ config echo @zlib:@stdin:
1167
+ config email_test --notif-to=my_recipient_email
1168
+ config export
1169
+ config flush_tokens
1170
+ config genkey mykey
1171
+ config plugin create mycommand T
1172
+ config plugin list
1173
+ config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com
1174
+ config wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes
1175
+ config wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
1176
+ ```
1177
+
932
1178
  #### Format of file
933
1179
 
934
1180
  The configuration file is a hash in a YAML file. Example:
@@ -1110,6 +1356,8 @@ Secrets must be uniquely identified by `url` and `username`. An optional descrip
1110
1356
 
1111
1357
  #### Legacy config file format
1112
1358
 
1359
+ THIS FORMAT WILL BE DEPRECATED
1360
+
1113
1361
  The value provided can be a Hash, where keys are usernames (or access key id), and values are the associated password or secrets in clear.
1114
1362
 
1115
1363
  For example, choose a repository name, for example `my_secrets`, and populate it like this:
@@ -1141,39 +1389,56 @@ ascli conf vault get --username=access_key1
1141
1389
 
1142
1390
  ### <a id="private_key"></a>Private Key
1143
1391
 
1144
- Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5...).
1145
- It consists in generating a private key, or using a previouly generated key.
1392
+ Some applications allow the user to be authenticated using a private key (Server, AoC, Faspex5, ...).
1393
+ It consists in using a pair of keys: the private key and its associated public key.
1146
1394
  The same key can be used for multiple applications.
1147
- Technically, a private key contains the public key, which can be extracted.
1148
- Currently, only private key not protected by a passphrase are supported.
1149
- (TODO: add passphrase protection as option for aspera apps).
1395
+ Technically, a private key contains the public key, which can be extracted from it.
1396
+ The private key can be protected by a passphrase or not.
1397
+ If the key is protected by a passphrase, then it will be prompted.
1398
+ (some plugins support option `passphrase`)
1150
1399
 
1151
1400
  Several methods can be used to generate a key pair:
1152
1401
 
1402
+ The following commands use this variable:
1403
+
1404
+ ```bash
1405
+ PRIVKEYFILE=~/.aspera/ascli/my_private_key
1406
+ ```
1407
+
1153
1408
  * `ascli`
1154
1409
 
1155
- The generated key is of type RSA 4096 bit. For convenience, the public key is also extracted.
1410
+ The generated key is of type RSA, by default: 4096 bit.
1411
+ For convenience, the public key is also extracted with extension `.pub`.
1412
+ The key is not passphrase protected.
1156
1413
 
1157
1414
  ```bash
1158
- ascli config genkey ~/.aspera/ascli/my_private_key
1415
+ ascli config genkey ${PRIVKEYFILE} 4096
1159
1416
  ```
1160
1417
 
1161
1418
  * `ssh-keygen`
1162
1419
 
1420
+ Both private and public keys are generated, option `-N` is for passphrase.
1421
+
1163
1422
  ```bash
1164
- ssh-keygen -t rsa -f ~/.aspera/ascli/my_private_key -N ''
1423
+ ssh-keygen -t rsa -b 4096 -N '' -f ${PRIVKEYFILE}
1165
1424
  ```
1166
1425
 
1167
1426
  * `openssl`
1168
1427
 
1169
- openssl is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS). To generate a privatekey pait without passphrase the following shall work on any system:
1428
+ openssl is sometimes compiled to support option `-nodes` (no DES, i.e. no passphrase, e.g. on macOS).
1429
+ To generate a private key pair without passphrase the following can be used on any system:
1430
+
1431
+ ```bash
1432
+ openssl genrsa -passout pass:dummypassword -out ${PRIVKEYFILE}.protected 4096
1433
+ openssl rsa -passin pass:dummypassword -in ${PRIVKEYFILE}.protected -out ${PRIVKEYFILE}
1434
+ openssl rsa -pubout -in ${PRIVKEYFILE} -out ${PRIVKEYFILE}.pub
1435
+ rm -f ${PRIVKEYFILE}.protected
1436
+ ```
1437
+
1438
+ To change (or add) the passphrase for a key do:
1170
1439
 
1171
1440
  ```bash
1172
- APIKEY=~/.aspera/ascli/my_private_key
1173
- openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
1174
- openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
1175
- openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
1176
- rm -f ${APIKEY}.protected
1441
+ openssl rsa -des3 -in old_file -out new_file
1177
1442
  ```
1178
1443
 
1179
1444
  ### <a id="certificates"></a>SSL CA certificate bundle
@@ -1250,14 +1515,37 @@ Aspera on Cloud relies on Oauth, refer to the [Aspera on Cloud](#aoc) section.
1250
1515
 
1251
1516
  ### Logging, Debugging
1252
1517
 
1253
- The gem is equipped with traces. By default logging level is `warn`.
1254
- To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or parameter in con file).
1518
+ The gem is equipped with traces, mainly for debugging and learning APIs.
1519
+ By default logging level is `warn` and the output channel is `stderr`.
1520
+ To increase debug level, use parameter `log_level` (e.g. using command line `--log-level=xx`, env var `ASCLI_LOG_LEVEL`, or a parameter in the configuration file).
1255
1521
 
1256
- It is also possible to activate traces before initialization using env var `AS_LOG_LEVEL`.
1522
+ It is also possible to activate traces before log facility initialization using env var `ASCLI_LOG_LEVEL`.
1257
1523
 
1258
1524
  By default passwords and secrets are removed from logs.
1259
1525
  Use option `log_secrets` set to `yes` to reveal secrets in logs.
1260
1526
 
1527
+ Available loggers: `stdout`, `stderr`, `syslog`.
1528
+
1529
+ Available levels: `debug`, `info`, `warn`, `error`.
1530
+
1531
+ Note that when using the `direct` agent (`ascp`), additional transfer logs can be activated using `ascp` option `EX_ascp_args`, see [`direct`](#agt_direct).
1532
+
1533
+ Examples:
1534
+
1535
+ * display debugging log on `stdout`:
1536
+
1537
+ ```bash
1538
+ ascli conf over --log-level=debug --logger=stdout
1539
+ ```
1540
+
1541
+ * log errors to `syslog`:
1542
+
1543
+ ```bash
1544
+ ascli conf over --log-level=error --logger=syslog
1545
+ ```
1546
+
1547
+ When `ascli` is used interactively in a shell, the shell itself will usually log executed commands in the history file.
1548
+
1261
1549
  ### Learning Aspera Product APIs (REST)
1262
1550
 
1263
1551
  This CLI uses REST APIs.
@@ -1341,7 +1629,7 @@ The PAC file will be used for any HTTP/HTTPS/REST connection, but not other (e.g
1341
1629
 
1342
1630
  The PAC file can be tested with command: `config proxy_check`. Example, using command line option:
1343
1631
 
1344
- ```
1632
+ ```bash
1345
1633
  ascli conf proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:1234;DIRECT";}' http://example.com
1346
1634
  PROXY proxy.example.com:1234;DIRECT
1347
1635
  ```
@@ -1520,12 +1808,12 @@ will effectively push files to the related server from the agent node.
1520
1808
  #### <a id="agt_direct"></a>Direct
1521
1809
 
1522
1810
  The `direct` agent directly executes a local ascp.
1523
- This is the default for `ascli`.
1524
- This is equivalent to specifying `--transfer=direct`.
1525
- `ascli` will detect locally installed Aspera products, including SDK.
1811
+ This is the default agent for `ascli`.
1812
+ This is equivalent to option `--transfer=direct`.
1813
+ `ascli` will detect locally installed Aspera products, including SDK, and use `ascp` from that component.
1526
1814
  Refer to section [FASP](#client).
1527
1815
 
1528
- The `transfer-info` accepts the following optional parameters to control multi-session, WSS
1816
+ The `transfer_info` option accepts the following optional parameters to control multi-session, Web Socket Session and Resume policy:
1529
1817
 
1530
1818
  <table>
1531
1819
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
@@ -1540,7 +1828,7 @@ The `transfer-info` accepts the following optional parameters to control multi-s
1540
1828
  <tr><td>resume.sleep_max</td><td>int</td><td>Resume<br/>Default: 60</td></tr>
1541
1829
  </table>
1542
1830
 
1543
- Resume: In case of transfer interruption, the agent will resume a transfer up to `iter_max` time.
1831
+ In case of transfer interruption, the agent will **resume** a transfer up to `iter_max` time.
1544
1832
  Sleep between iterations is:
1545
1833
 
1546
1834
  ```bash
@@ -1548,14 +1836,49 @@ max( sleep_max , sleep_initial * sleep_factor ^ (iter_index-1) )
1548
1836
  ```
1549
1837
 
1550
1838
  Some transfer errors are considered "retryable" (e.g. timeout) and some other not (e.g. wrong password).
1839
+ The list of known protocol errors and retry level can be listed:
1840
+
1841
+ ```bash
1842
+ ascli config ascp errors
1843
+ ```
1551
1844
 
1552
1845
  Examples:
1553
1846
 
1554
1847
  ```javascript
1555
- ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":10}}'
1848
+ ascli ... --transfer-info=@json:'{"wss":true,"resume":{"iter_max":20}}'
1556
1849
  ascli ... --transfer-info=@json:'{"spawn_delay_sec":2.5,"multi_incr_udp":false}'
1557
1850
  ```
1558
1851
 
1852
+ Note that the `direct` agent supports additional `transfer_spec` parameters starting with `EX_` (extended).
1853
+ In particular the field, `EX_ascp_args` which is a list of additional command line options to `ascp`.
1854
+
1855
+ This can be useful to activate logging using option `-L` of ascp.
1856
+ For example the option `--ts=@json:'{"EX_ascp_args":["-DDL-"]}'` will activate debug level 2 for `ascp` (`DD`), and display those logs on the terminal (`-`).
1857
+ This is useful if the transfer fails.
1858
+ To store ascp logs in file `aspera-scp-transfer.log` in a folder, use `--ts=@json:'{"EX_ascp_args":["-L","/path/to/folder"]}'`.
1859
+
1860
+ > Implementation note: when transfer agent [`direct`](#agt_direct) is used, the list of files to transfer is provided to `ascp` using either `--file-list` or `--file-pair-list` and a file list (or pair) file generated in a temporary folder. (unless `--file-list` or `--file-pair-list` is provided in option `ts` in `EX_ascp_args`).
1861
+
1862
+ In addition to standard methods described in section [File List](#file_list), it is possible to specify the list of file using those additional methods:
1863
+
1864
+ * Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_file_list`
1865
+
1866
+ ```javascript
1867
+ --sources=@ts --ts=@json:'{"EX_file_list":"filelist.txt"}'
1868
+ ```
1869
+
1870
+ * Using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`
1871
+
1872
+ ```javascript
1873
+ --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","myfilelist"]}'
1874
+ ```
1875
+
1876
+ > File lists is shown here, there are also similar options for file pair lists.
1877
+
1878
+ > Those 2 additional methods avoid the creation of a copy of the file list: if the standard options `--sources=@lines:@file:... --src-type=...` are used, then the file is list read and parsed, and a new file list is created in a temporary folder.
1879
+
1880
+ > Those methods have limitations: they apply **only** to the [`direct`](#agt_direct) transfer agent (i.e. local `ascp`) and not for Aspera on Cloud.
1881
+
1559
1882
  #### <a id="agt_connect"></a>IBM Aspera Connect Client GUI
1560
1883
 
1561
1884
  By specifying option: `--transfer=connect`, `ascli` will start transfers using the locally installed Aspera Connect Client. There are no option for `transfer_info`.
@@ -1566,6 +1889,7 @@ By specifying option: `--transfer=node`, the CLI will start transfers in an Aspe
1566
1889
  Transfer Server using the Node API, either on a local or remote node.
1567
1890
  Parameters provided in option `transfer_info` are:
1568
1891
 
1892
+ <table>
1569
1893
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
1570
1894
  <tr><td>url</td><td>string</td><td>URL of the node API</br>Mandatory</td></tr>
1571
1895
  <tr><td>username</td><td>string</td><td>node api user or access key</br>Mandatory</td></tr>
@@ -1579,7 +1903,7 @@ Like any other option, `transfer_info` can get its value from a pre-configured [
1579
1903
 
1580
1904
  If `transfer_info` is not specified and a default node has been configured (name in `node` for section `default`) then this node is used by default.
1581
1905
 
1582
- If the `password` value begins with `Bearer ` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
1906
+ If the `password` value begins with `Bearer` then the `username` is expected to be an access key and the parameter `root_id` is mandatory and specifies the root file id on the node. It can be either the access key's root file id, or any authorized file id underneath it.
1583
1907
 
1584
1908
  #### <a id="agt_httpgw"></a>HTTP Gateway
1585
1909
 
@@ -1599,6 +1923,19 @@ Another possibility is to use the Transfer SDK daemon (asperatransferd).
1599
1923
 
1600
1924
  By default it will listen on local port `55002` on `127.0.0.1`.
1601
1925
 
1926
+ The gem `grpc` was removed from dependencies, as it requires compilation of a native part. So, to use the Transfer SDK you should install this gem:
1927
+
1928
+ ```bash
1929
+ gem install grpc
1930
+ ```
1931
+
1932
+ On Windows the compilation may fail for various reasons (3.1.1):
1933
+
1934
+ * `cannot find -lx64-ucrt-ruby310`
1935
+ &rarr; copy the file `[Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a` to `[Ruby main dir]\lib\libx64-ucrt-ruby310.a` (remove the dll extension)
1936
+ * `conflicting types for 'gettimeofday'`
1937
+ &rarr; edit the file `[Ruby main dir]/include/ruby-[version]/ruby/win32.h` and change the signature of `gettimeofday` to `gettimeofday(struct timeval *, void *)` ,i.e. change `struct timezone` to `void`
1938
+
1602
1939
  ### <a id="transferspec"></a>Transfer Specification
1603
1940
 
1604
1941
  Some commands lead to file transfer (upload/download), all parameters necessary for this transfer
@@ -1651,7 +1988,91 @@ Columns:
1651
1988
 
1652
1989
  Fields with EX_ prefix are extensions to transfer agent [`direct`](#agt_direct). (only in `ascli`).
1653
1990
 
1654
- <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr><tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c)</td></tr><tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt)</td></tr><tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr><tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr><tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source. The source and destination arguments must be directories that have matching names. Objects on the destination that have the same name but different type or size as objects on the source are not deleted.<br/>(--delete-before-transfer)</td></tr><tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr><tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr><tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr><tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr><tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr><tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr><tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y)</td></tr><tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port when no cipher is used<br/>(-t)</td></tr><tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t)</td></tr><tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>(--move-after-transfer)</td></tr><tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/> Each participant on one host needs an independent UDP (-O) port.<br/> Large files are split between sessions only when transferring with resume_policy=none.</td></tr><tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value. (0=no file is split)<br/>(--multi-session-threshold)</td></tr><tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr><tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr><tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr><tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr><tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr><tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr><tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr><tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr><tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr><tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr><tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr><tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr><tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr><tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/> dnat(s)://[user[:password]@]server:port<br/> Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/> Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr><tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr><tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr><tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr><tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr><tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/> Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/> Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr><tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr><tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr><tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64)</td></tr><tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr><tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr><tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr><tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr><tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr><tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/> This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr><tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr><tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls)</td></tr><tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr><tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr><tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr><tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr><tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr><tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Passphrase used for at rest encryption or decryption. Prefer to use standard: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr><tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/> May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr><tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr><tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr><tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr><tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr><tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr><tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr><tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr><tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr><tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr><tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr><tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr><tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr><tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr><tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr><tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr><tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr><tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr></table>
1991
+ <table><tr><th>Field</th><th>Type</th><th>D</th><th>N</th><th>C</th><th>Description</th></tr>
1992
+ <tr><td>EX_ascp_args</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Add native command line arguments to ascp</td></tr>
1993
+ <tr><td>EX_at_rest_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>DEPRECATED: Prefer to use standard parameter: content_protection_password<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
1994
+ <tr><td>EX_file_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file list</td></tr>
1995
+ <tr><td>EX_file_pair_list</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>source file pair list</td></tr>
1996
+ <tr><td>EX_http_proxy_url</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the proxy server address used by HTTP Fallback<br/>(-x)</td></tr>
1997
+ <tr><td>EX_http_transfer_jpeg</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>HTTP transfers as JPEG file<br/>(-j)</td></tr>
1998
+ <tr><td>EX_license_text</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>License file text override.<br/>By default ascp looks for license file near executable.<br/>(env:ASPERA_SCP_LICENSE)</td></tr>
1999
+ <tr><td>EX_no_read</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no read source<br/>(--no-read)</td></tr>
2000
+ <tr><td>EX_no_write</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>no write on destination<br/>(--no-write)</td></tr>
2001
+ <tr><td>EX_proxy_password</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Password used for Aspera proxy server authentication.<br/>May be overridden by password in URL EX_fasp_proxy_url.<br/>(env:ASPERA_PROXY_PASS)</td></tr>
2002
+ <tr><td>EX_ssh_key_paths</td><td>array</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Use public key authentication for SSH and specify the private key file paths<br/>(-i)</td></tr>
2003
+ <tr><td>apply_local_docroot</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>(--apply-local-docroot)</td></tr>
2004
+ <tr><td>authentication</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>value=token for SSH bypass keys, else password asked if not provided.</td></tr>
2005
+ <tr><td>cipher</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>In transit encryption type.<br/>Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm<br/>(-c)</td></tr>
2006
+ <tr><td>cipher_allowed</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>returned by node API. Valid literals include "aes-128" and "none".</td></tr>
2007
+ <tr><td>content_protection</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Enable client-side encryption at rest. (CSEAR, content protection)<br/>Allowed values: encrypt, decrypt<br/>(--file-crypt)</td></tr>
2008
+ <tr><td>content_protection_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies CSEAR password. (content protection)<br/>(env:ASPERA_SCP_FILEPASS)</td></tr>
2009
+ <tr><td>cookie</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer specified by application<br/>(env:ASPERA_SCP_COOKIE)</td></tr>
2010
+ <tr><td>create_dir</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to create new directories.<br/>(-d)</td></tr>
2011
+ <tr><td>delete_before_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Before transfer, delete files that exist at the destination but not at the source.<br/>The source and destination arguments must be directories that have matching names.<br/>Objects on the destination that have the same name but different type or size as objects<br/>on the source are not deleted.<br/>(--delete-before-transfer)</td></tr>
2012
+ <tr><td>delete_source</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2013
+ <tr><td>destination_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Destination root directory.</td></tr>
2014
+ <tr><td>dgram_size</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>UDP datagram size in bytes<br/>(-Z)</td></tr>
2015
+ <tr><td>direction</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Direction of transfer (on client side)<br/>Allowed values: send, receive<br/>(--mode)</td></tr>
2016
+ <tr><td>exclude_newer_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime > arg<br/>(--exclude-newer-than)</td></tr>
2017
+ <tr><td>exclude_older_than</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>skip src files with mtime < arg<br/>(--exclude-older-than)</td></tr>
2018
+ <tr><td>fasp_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies fasp (UDP) port.<br/>(-O)</td></tr>
2019
+ <tr><td>file_checksum</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Enable checksum reporting for transferred files by specifying the hash to use.<br/>Allowed values: sha-512, sha-384, sha-256, sha1, md5, none</td></tr>
2020
+ <tr><td>http_fallback</td><td>bool<br/>string</td><td>Y</td><td>Y</td><td>Y</td><td>When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.<br/>(-y)</td></tr>
2021
+ <tr><td>http_fallback_port</td><td>int</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specifies http port when no cipher is used<br/>(-t)</td></tr>
2022
+ <tr><td>https_fallback_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies https port when cipher is used<br/>(-t)</td></tr>
2023
+ <tr><td>lock_min_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2024
+ <tr><td>lock_min_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2025
+ <tr><td>lock_rate_policy</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2026
+ <tr><td>lock_target_rate</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2027
+ <tr><td>lock_target_rate_kbps</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2028
+ <tr><td>min_rate_cap_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2029
+ <tr><td>min_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Set the minimum transfer rate in kilobits per second.<br/>(-m)</td></tr>
2030
+ <tr><td>move_after_transfer</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The relative path to which the files will be moved after the transfer at the source side.<br/>(--move-after-transfer)</td></tr>
2031
+ <tr><td>multi_session</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Use multi-session transfer. max 128.<br/>Each participant on one host needs an independent UDP (-O) port.<br/>Large files are split between sessions only when transferring with resume_policy=none.<br/></td></tr>
2032
+ <tr><td>multi_session_threshold</td><td>int</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.<br/>(0=no file is split)<br/>(--multi-session-threshold)</td></tr>
2033
+ <tr><td>overwrite</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Overwrite destination files with the source files of the same name.<br/>Allowed values: never, always, diff, older, diff+older<br/>(--overwrite)</td></tr>
2034
+ <tr><td>password</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>Password for local Windows user when transfer user associated with node api user is not the same as the one running asperanoded.<br/>Allows impersonating the transfer user and have access to resources (e.g. network shares).<br/>Windows only, node api only.</td></tr>
2035
+ <tr><td>paths</td><td>array</td><td>Y</td><td>Y</td><td>Y</td><td>Array of path to the source (required) and a path to the destination (optional).</td></tr>
2036
+ <tr><td>precalculate_job_size</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies whether to precalculate the job size.<br/>(--precalculate-job-size)</td></tr>
2037
+ <tr><td>preserve_access_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-access-time)</td></tr>
2038
+ <tr><td>preserve_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve access control lists.<br/>Allowed values: none, native, metafile<br/>(--preserve-acls)</td></tr>
2039
+ <tr><td>preserve_creation_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-creation-time)</td></tr>
2040
+ <tr><td>preserve_file_owner_gid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the group ID for a file owner<br/>(--preserve-file-owner-gid)</td></tr>
2041
+ <tr><td>preserve_file_owner_uid</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the user ID for a file owner<br/>(--preserve-file-owner-uid)</td></tr>
2042
+ <tr><td>preserve_modification_time</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-modification-time)</td></tr>
2043
+ <tr><td>preserve_remote_acls</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve remote access control lists.<br/>Allowed values: none, native, metafile<br/>(--remote-preserve-acls)</td></tr>
2044
+ <tr><td>preserve_source_access_time</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Preserve the time logged for when the source file was accessed<br/>(--preserve-source-access-time)</td></tr>
2045
+ <tr><td>preserve_times</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>(--preserve-times)</td></tr>
2046
+ <tr><td>proxy</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Specify the address of the Aspera high-speed proxy server.<br/>dnat(s)://[user[:password]@]server:port<br/>Default ports for DNAT and DNATS protocols are 9091 and 9092.<br/>Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.<br/>(--proxy)</td></tr>
2047
+ <tr><td>rate_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>The transfer rate policy to use when sharing bandwidth.<br/>Allowed values: low, fair, high, fixed<br/>(--policy)</td></tr>
2048
+ <tr><td>rate_policy_allowed</td><td>string</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Specifies most aggressive rate policy that is allowed.<br/>Returned by node API.<br/>Allowed values: low, fair, high, fixed</td></tr>
2049
+ <tr><td>remote_host</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>IP or fully qualified domain name of the remote server<br/>(--host)</td></tr>
2050
+ <tr><td>remote_password</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>SSH session password<br/>(env:ASPERA_SCP_PASS)</td></tr>
2051
+ <tr><td>remote_user</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Remote user. Default value is "xfer" on node or connect.<br/>(--user)</td></tr>
2052
+ <tr><td>remove_after_transfer</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Remove SRC files after transfer success<br/>(--remove-after-transfer)</td></tr>
2053
+ <tr><td>remove_empty_directories</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specifies whether to remove empty directories.<br/>(--remove-empty-directories)</td></tr>
2054
+ <tr><td>remove_empty_source_directory</td><td>bool</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Remove empty source subdirectories and remove the source directory itself, if empty<br/>(--remove-empty-source-directory)</td></tr>
2055
+ <tr><td>remove_skipped</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>Must also have remove_after_transfer set to true, Defaults to false, if true, skipped files will be removed as well.<br/>(--remove-skipped)</td></tr>
2056
+ <tr><td>resume_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>If a transfer is interrupted or fails to finish, resume without re-transferring the whole files.<br/>Allowed values: none, attrs, sparse_csum, full_csum<br/>(-k)</td></tr>
2057
+ <tr><td>retry_duration</td><td>string<br/>int</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Specifies how long to wait before retrying transfer. (e.g. "5min")</td></tr>
2058
+ <tr><td>source_root</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Path to be prepended to each source path.<br/>This is either a conventional path or it can be a URI but only if there is no root defined.<br/>(--source-prefix64)</td></tr>
2059
+ <tr><td>source_root_id</td><td>string</td><td>&nbsp;</td><td>Y</td><td>&nbsp;</td><td>The file ID of the source root directory. Required when using Bearer token auth for the source node.</td></tr>
2060
+ <tr><td>src_base</td><td>string</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>Specify the prefix to be stripped off from each source object.<br/>The remaining portion of the source path is kept intact at the destination.<br/>Special care must be taken when used with cloud storage.<br/>(--src-base64)</td></tr>
2061
+ <tr><td>ssh_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies SSH (TCP) port. Default: local:22, other:33001<br/>(-P)</td></tr>
2062
+ <tr><td>ssh_private_key</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>Private key used for SSH authentication.<br/>Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...<br/>Note the JSON encoding: \n for newlines.<br/>(env:ASPERA_SCP_KEY)</td></tr>
2063
+ <tr><td>ssh_private_key_passphrase</td><td>string</td><td>Y</td><td>&nbsp;</td><td>&nbsp;</td><td>The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.<br/>(env:ASPERA_SCP_PASS)</td></tr>
2064
+ <tr><td>sshfp</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Check it against server SSH host key fingerprint<br/>(--check-sshfp)</td></tr>
2065
+ <tr><td>symlink_policy</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Handle source side symbolic links<br/>Allowed values: follow, copy, copy+force, skip<br/>(--symbolic-links)</td></tr>
2066
+ <tr><td>tags</td><td>hash</td><td>Y</td><td>Y</td><td>Y</td><td>Metadata for transfer as JSON<br/>(--tags64)</td></tr>
2067
+ <tr><td>target_rate_cap_kbps</td><td>int</td><td>&nbsp;</td><td>&nbsp;</td><td>Y</td><td>Returned by upload/download_setup node API.</td></tr>
2068
+ <tr><td>target_rate_kbps</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>Specifies desired speed for the transfer.<br/>(-l)</td></tr>
2069
+ <tr><td>target_rate_percentage</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2070
+ <tr><td>title</td><td>string</td><td>&nbsp;</td><td>Y</td><td>Y</td><td>Title of the transfer</td></tr>
2071
+ <tr><td>token</td><td>string</td><td>Y</td><td>Y</td><td>Y</td><td>Authorization token: Bearer, Basic or ATM (Also arg -W)<br/>(env:ASPERA_SCP_TOKEN)</td></tr>
2072
+ <tr><td>use_ascp4</td><td>bool</td><td>Y</td><td>Y</td><td>&nbsp;</td><td>specify version of protocol</td></tr>
2073
+ <tr><td>wss_enabled</td><td>bool</td><td>Y</td><td>Y</td><td>Y</td><td>&nbsp;</td></tr>
2074
+ <tr><td>wss_port</td><td>int</td><td>Y</td><td>Y</td><td>Y</td><td>TCP port used for websocket service feed.</td></tr>
2075
+ </table>
1655
2076
 
1656
2077
  #### Destination folder for transfers
1657
2078
 
@@ -1665,87 +2086,98 @@ As such, it can be modified with option: `--ts=@json:'{"destination_root":"<path
1665
2086
  The option `to_folder` provides an equivalent and convenient way to change this parameter:
1666
2087
  `--to-folder=<path>` .
1667
2088
 
1668
- #### List of files for transfers
2089
+ #### <a id="file_list"></a>List of files for transfers
1669
2090
 
1670
- When uploading, downloading or sending files, the user must specify the list of files to transfer. The option to specify the list of files is `sources`, the default value is `@args`, which means: take remain non used arguments (not starting with `-` as list of files.
1671
- So, by default, the list of files to transfer will be simply specified on the command line:
2091
+ When uploading, downloading or sending files, the user must specify the list of files to transfer.
1672
2092
 
1673
- ```bash
1674
- ascli server upload ~/mysample.file secondfile
1675
- ```
2093
+ By default the list of files to transfer is simply provided on the command line.
1676
2094
 
1677
- This is equivalent to:
2095
+ The list of (source) files to transfer is specified by (extended value) option `sources` (default: `@args`).
2096
+ The list is either simply the list of source files, or a combined source/destination list (see below) depending on value of option `src_type` (default: `list`).
1678
2097
 
1679
- ```bash
1680
- ascli server upload --sources=@args ~/mysample.file secondfile
1681
- ```
2098
+ In `ascli`, all transfer parameters, including file list, are provided to the transfer agent in a [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer sdk...).
2099
+ So, eventually, the list of files to transfer is provided to the transfer agent using the [*transfer-spec*](#transferspec) field: `"paths"` which is a list (array) of pairs of `"source"` (mandatory) and `"destination"` (optional).
2100
+ The `sources` and `src_type` options provide convenient ways to populate the transfer spec with the source file list.
1682
2101
 
1683
- More advanced options are provided to adapt to various cases. In fact, list of files to transfer are normally conveyed using the [*transfer-spec*](#transferspec) using the field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
2102
+ Possible values for option `sources` are:
1684
2103
 
1685
- Note that this is different from the "ascp" command line. The paradigm used by `ascli` is:
1686
- all transfer parameters are kept in [*transfer-spec*](#transferspec) so that execution of a transfer is independent of the transfer agent. Note that other IBM Aspera interfaces use this: connect, node, transfer sdk.
2104
+ * `@args` : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with `-`) are considered as source files.
2105
+ So, by default, the list of files to transfer will be simply specified on the command line. Example:
1687
2106
 
1688
- For ease of use and flexibility, the list of files to transfer is specified by the option `sources`. Accepted values are:
2107
+ ```bash
2108
+ ascli server upload ~/first.file secondfile
2109
+ ```
1689
2110
 
1690
- * `@args` : (default value) the list of files is directly provided at the end of the command line (see at the beginning of this section).
2111
+ This is the same as (with default values):
1691
2112
 
1692
- * an [Extended Value](#extended) holding an *Array of String*. Examples:
2113
+ ```bash
2114
+ ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile
2115
+ ```
1693
2116
 
1694
- ```javascript
1695
- --sources=@json:'["file1","file2"]'
1696
- ```
2117
+ * an [Extended Value](#extended) with type **Array of String**
1697
2118
 
1698
- ```bash
1699
- --sources=@lines:@stdin:
1700
- ```
2119
+ > Note: extended values can be tested with the command `conf echo`
1701
2120
 
1702
- ```ruby
1703
- --sources=@ruby:'File.read("myfilelist").split("\n")'
1704
- ```
2121
+ Examples:
1705
2122
 
1706
- * `@ts` : the user provides the list of files directly in the `ts` option, in its `paths` field. Example:
2123
+ * Using extended value
1707
2124
 
1708
- ```javascript
1709
- --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
1710
- ```
2125
+ Create the file list:
1711
2126
 
1712
- providing a file list directly to ascp:
2127
+ ```bash
2128
+ echo ~/mysample.file > myfilelist.txt
2129
+ echo secondfile >> myfilelist.txt
2130
+ ```
1713
2131
 
1714
- ```javascript
1715
- ... --sources=@ts --ts=@json:'{"paths":[],"EX_file_list":"filelist.txt"}'
1716
- ```
2132
+ Use the file list: one path per line:
1717
2133
 
1718
- * Not recommended: It is possible to specify bare ascp arguments using the pseudo [*transfer-spec*](#transferspec) parameter `EX_ascp_args`.
2134
+ ```ruby
2135
+ --sources=@lines:@file:myfilelist.txt
2136
+ ```
1719
2137
 
1720
- ```javascript
1721
- --sources=@ts --ts=@json:'{"paths":[{"source":"dummy"}],"EX_ascp_args":["--file-list","myfilelist"]}'
1722
- ```
2138
+ * Using JSON array
1723
2139
 
1724
- This method avoids creating a copy of the file list, but has drawbacks: it applies *only* to the [`direct`](#agt_direct) transfer agent (i.e. bare ascp) and not for Aspera on Cloud. One must specify a dummy list in the [*transfer-spec*](#transferspec), which will be overridden by the bare ascp command line provided. (TODO) In next version, dummy source paths can be removed.
2140
+ ```javascript
2141
+ --sources=@json:'["file1","file2"]'
2142
+ ```
1725
2143
 
1726
- In case the file list is provided on the command line i.e. using `--sources=@args` or `--sources=<Array>` (but not `--sources=@ts`), then the list of files will be used either as a simple file list or a file pair list depending on the value of the option: `src_type`:
2144
+ * Using STDIN, one path per line
1727
2145
 
1728
- * `list` : (default) the path of destination is the same as source
1729
- * `pair` : in that case, the first element is the first source, the second element is the first destination, and so on.
2146
+ ```bash
2147
+ --sources=@lines:@stdin:
2148
+ ```
1730
2149
 
1731
- Example:
2150
+ * Using ruby code (one path per line in file)
1732
2151
 
1733
- ```bash
1734
- ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
1735
- ```
2152
+ ```ruby
2153
+ --sources=@ruby:'File.read("myfilelist.txt").split("\n")'
2154
+ ```
1736
2155
 
1737
- Internally, when transfer agent [`direct`](#agt_direct) is used, a temporary file list (or pair) file is generated and provided to ascp, unless `--file-list` or `--file-pair-list` is provided in `ts` in `EX_ascp_args`.
2156
+ * `@ts` : the user provides the list of files directly in the `paths` field of transfer spec (option `ts`).
2157
+ Examples:
1738
2158
 
1739
- Note the special case when the source files are located on "Aspera on Cloud", i.e. using access keys and the `file id` API:
2159
+ * Using transfer spec
1740
2160
 
1741
- * All files must be in the same source folder.
1742
- * If there is a single file : specify the full path
1743
- * For multiple files, specify the source folder as first item in the list followed by the list of file names.
2161
+ ```javascript
2162
+ --sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
2163
+ ```
1744
2164
 
1745
- Source files are located on "Aspera on cloud", when :
2165
+ The option `src_type` allows specifying if the list specified in option `sources` is a simple file list or if it is a file pair list.
2166
+
2167
+ > Note: Option `src_type` is not used if option `sources` is set to `@ts`
2168
+
2169
+ Supported values for `src_type` are:
2170
+
2171
+ * `list` : (default) the path of destination is the same as source and each entry is a source file path
2172
+ * `pair` : the first element is the first source, the second element is the first destination, and so on.
2173
+
2174
+ Example: Source file `200KB.1` is renamed `sample1` on destination:
2175
+
2176
+ ```bash
2177
+ ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
2178
+ ```
1746
2179
 
1747
- * the server is Aspera on Cloud, and making a download / recv
1748
- * the agent is Aspera on Cloud, and making an upload / send
2180
+ > Note there are some specific rules to specify file list when using "Aspera on Cloud", refer to the AoC plugin section.
1749
2181
 
1750
2182
  #### <a id="multisession"></a>Support of multi-session
1751
2183
 
@@ -1785,13 +2217,13 @@ activating CSEAR consists in using transfer spec parameters:
1785
2217
 
1786
2218
  Example: parameter to download a faspex package and decrypt on the fly
1787
2219
 
1788
- ```json
2220
+ ```javascript
1789
2221
  --ts=@json:'{"content_protection":"decrypt","content_protection_password":"_pass_here_"}'
1790
2222
  ```
1791
2223
 
1792
2224
  Note that up to version 4.6.0, the following parameters should be used for agent `direct`:
1793
2225
 
1794
- ```json
2226
+ ```javascript
1795
2227
  --ts=@json:'{"EX_ascp_args":["--file-crypt=decrypt"],"EX_at_rest_password":"_secret_here_"}'
1796
2228
  ```
1797
2229
 
@@ -1978,318 +2410,51 @@ ascli server upload /tmp/sample --to-folder=faux://
1978
2410
  ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
1979
2411
  ```
1980
2412
 
1981
- ### <a id="commands"></a>Sample Commands
1982
-
1983
- A non complete list of commands used in unit tests:
1984
-
1985
- ```bash
1986
- ascli
1987
- ascli -h
1988
- ascli aoc -N remind --username=my_aoc_user_email
1989
- ascli aoc -N servers
1990
- ascli aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
1991
- ascli aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
1992
- ascli aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
1993
- ascli aoc admin ats access_key delete akibmcloud
1994
- ascli aoc admin ats access_key list --fields=name,id
1995
- ascli aoc admin ats access_key node akibmcloud --secret=somesecret browse /
1996
- ascli aoc admin ats cluster clouds
1997
- ascli aoc admin ats cluster list
1998
- ascli aoc admin ats cluster show --cloud=aws --region=eu-west-1
1999
- ascli aoc admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
2000
- ascli aoc admin res application list
2001
- ascli aoc admin res client list
2002
- ascli aoc admin res client_access_key list
2003
- ascli aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
2004
- ascli aoc admin res client_registration_token delete my_clt_reg_id
2005
- ascli aoc admin res client_registration_token list
2006
- ascli aoc admin res contact list
2007
- ascli aoc admin res dropbox list
2008
- ascli aoc admin res dropbox_membership list
2009
- ascli aoc admin res group list
2010
- ascli aoc admin res kms_profile list
2011
- ascli aoc admin res node list
2012
- ascli aoc admin res operation list
2013
- ascli aoc admin res organization show
2014
- ascli aoc admin res package list --http-options=@json:'{"read_timeout":120.0}'
2015
- ascli aoc admin res saml_configuration list
2016
- ascli aoc admin res self show
2017
- ascli aoc admin res short_link list
2018
- ascli aoc admin res user list
2019
- ascli aoc admin res workspace_membership list
2020
- ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
2021
- ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v3 events
2022
- ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 browse /
2023
- ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 delete /folder1
2024
- ascli aoc admin resource node --name=my_aoc_node1_name --secret=my_aoc_node1_secret v4 mkdir /folder1
2025
- ascli aoc admin resource node v3 name my_aoc_node1_name --secret=my_aoc_node1_secret access_key delete testsub1
2026
- ascli aoc admin resource workspace list
2027
- ascli aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
2028
- ascli aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
2029
- ascli aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
2030
- ascli aoc automation workflow delete my_wf_id
2031
- ascli aoc automation workflow list
2032
- ascli aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
2033
- ascli aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
2034
- ascli aoc bearer_token --display=data --scope=user:all
2035
- ascli aoc faspex
2036
- ascli aoc files bearer /
2037
- ascli aoc files bearer_token_node / --cache-tokens=no
2038
- ascli aoc files browse /
2039
- ascli aoc files browse / -N --link=my_aoc_publink_folder
2040
- ascli aoc files delete /testsrc
2041
- ascli aoc files download --transfer=connect /200KB.1
2042
- ascli aoc files file show my_file_id
2043
- ascli aoc files find / --value='\.partial$'
2044
- ascli aoc files http_node_download --to-folder=. /200KB.1
2045
- ascli aoc files mkdir /testsrc
2046
- ascli aoc files rename /somefolder testdst
2047
- ascli aoc files short_link create --to-folder=/testdst --value=private
2048
- ascli aoc files short_link create --to-folder=/testdst --value=public
2049
- ascli aoc files short_link list --value=@json:'{"purpose":"shared_folder_auth_link"}'
2050
- ascli aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
2051
- ascli aoc files upload --to-folder=/testsrc testfile.bin
2052
- ascli aoc files upload -N --to-folder=/ testfile.bin --link=my_aoc_publink_folder
2053
- ascli aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
2054
- ascli aoc files v3 info
2055
- ascli aoc org -N --link=my_aoc_publink_recv_from_aocuser
2056
- ascli aoc organization
2057
- ascli aoc packages list
2058
- ascli aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
2059
- ascli aoc packages recv "my_package_id" --to-folder=.
2060
- ascli aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
2061
- ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
2062
- ascli aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
2063
- ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
2064
- ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
2065
- ascli aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
2066
- ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
2067
- ascli aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
2068
- ascli aoc packages shared_inboxes list
2069
- ascli aoc user profile modify @json:'{"name":"dummy change"}'
2070
- ascli aoc user profile show
2071
- ascli aoc user workspaces current
2072
- ascli aoc user workspaces list
2073
- ascli ats access_key cluster akibmcloud --secret=somesecret
2074
- ascli ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
2075
- ascli ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
2076
- ascli ats access_key delete ak_aws
2077
- ascli ats access_key delete akibmcloud
2078
- ascli ats access_key list --fields=name,id
2079
- ascli ats access_key node akibmcloud browse / --secret=somesecret
2080
- ascli ats api_key create
2081
- ascli ats api_key instances
2082
- ascli ats api_key list
2083
- ascli ats cluster clouds
2084
- ascli ats cluster list
2085
- ascli ats cluster show --cloud=aws --region=eu-west-1
2086
- ascli ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
2087
- ascli conf flush_tokens
2088
- ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes
2089
- ascli conf wiz --url=https://my_aoc_org.ibmaspera.com --config-file=SAMPLE_CONFIG_FILE --pkeypath= --username=my_aoc_user_email --test-mode=yes --use-generic-client=yes
2090
- ascli config ascp connect info 'Aspera Connect for Windows'
2091
- ascli config ascp connect list
2092
- ascli config ascp connect version 'Aspera Connect for Windows' download 'Windows Installer' --to-folder=.
2093
- ascli config ascp connect version 'Aspera Connect for Windows' list
2094
- ascli config ascp connect version 'Aspera Connect for Windows' open documentation
2095
- ascli config ascp errors
2096
- ascli config ascp info
2097
- ascli config ascp install
2098
- ascli config ascp products list
2099
- ascli config ascp show
2100
- ascli config ascp spec
2101
- ascli config check_update
2102
- ascli config detect --url=https://my_aoc_org.ibmaspera.com
2103
- ascli config detect --url=my_faspex_url
2104
- ascli config doc
2105
- ascli config doc transfer-parameters
2106
- ascli config email_test --notif-to=my_recipient_email
2107
- ascli config export
2108
- ascli config genkey mykey
2109
- ascli config plugin create mycommand T
2110
- ascli config plugin list
2111
- ascli config proxy_check --fpac=@file:examples/proxy.pac https://eudemo.asperademo.com
2112
- ascli console transfer current list
2113
- ascli console transfer smart list
2114
- ascli console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
2115
- ascli cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
2116
- ascli cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
2117
- ascli cos node access_key show self
2118
- ascli cos node download testfile.bin --to-folder=.
2119
- ascli cos node info
2120
- ascli cos node upload testfile.bin
2121
- ascli faspex dropbox list --recipient="*my_faspex_dbx"
2122
- ascli faspex dropbox list --recipient="*my_faspex_wkg"
2123
- ascli faspex health
2124
- ascli faspex package list
2125
- ascli faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
2126
- ascli faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
2127
- ascli faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
2128
- ascli faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
2129
- ascli faspex package recv "my_package_id" --to-folder=.
2130
- ascli faspex package recv "my_package_id" --to-folder=. --box=sent
2131
- ascli faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
2132
- ascli faspex package recv ALL --to-folder=. --once-only=yes
2133
- ascli faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
2134
- ascli faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
2135
- ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
2136
- ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
2137
- ascli faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
2138
- ascli faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
2139
- ascli faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
2140
- ascli faspex source name "Server Files" node br /
2141
- ascli faspex v4 dmembership list
2142
- ascli faspex v4 dropbox list
2143
- ascli faspex v4 metadata_profile list
2144
- ascli faspex v4 user list
2145
- ascli faspex v4 wmembership list
2146
- ascli faspex v4 workgroup list
2147
- ascli faspex5 admin res accounts list
2148
- ascli faspex5 admin res contacts list
2149
- ascli faspex5 admin res jobs list
2150
- ascli faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
2151
- ascli faspex5 admin res oauth_clients list
2152
- ascli faspex5 admin res registrations list
2153
- ascli faspex5 admin res saml_configs list
2154
- ascli faspex5 admin res shared_inboxes list
2155
- ascli faspex5 admin res workgroups list
2156
- ascli faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
2157
- ascli faspex5 package receive "my_package_id" --to-folder=.
2158
- ascli faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin
2159
- ascli mycommand --plugin-folder=T
2160
- ascli node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
2161
- ascli node -N -Ptst_node_preview access_key delete aoc_1
2162
- ascli node async bandwidth 1
2163
- ascli node async counters 1
2164
- ascli node async files 1
2165
- ascli node async list
2166
- ascli node async show 1
2167
- ascli node async show ALL
2168
- ascli node basic_token
2169
- ascli node browse / -r
2170
- ascli node delete folder_1/10MB.1
2171
- ascli node delete testfile.bin
2172
- ascli node download testfile.bin --to-folder=.
2173
- ascli node download testfile.bin --to-folder=. --token-type=hybrid
2174
- ascli node health
2175
- ascli node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
2176
- ascli node search / --value=@json:'{"sort":"mtime"}'
2177
- ascli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
2178
- ascli node service delete service1
2179
- ascli node service list
2180
- ascli node transfer list --value=@json:'{"active_only":true}'
2181
- ascli node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
2182
- ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
2183
- ascli node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
2184
- ascli orchestrator info
2185
- ascli orchestrator plugins
2186
- ascli orchestrator processes
2187
- ascli orchestrator workflow inputs my_orch_workflow_id
2188
- ascli orchestrator workflow list
2189
- ascli orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}'
2190
- ascli orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
2191
- ascli orchestrator workflow status ALL
2192
- ascli orchestrator workflow status my_orch_workflow_id
2193
- ascli preview check --skip-types=office
2194
- ascli preview folder 1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
2195
- ascli preview scan --skip-types=office --log-level=info
2196
- ascli preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
2197
- ascli preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
2198
- ascli preview test --case=test mp4 my_file_mxf --video-conversion=reencode --log-level=debug
2199
- ascli preview test --case=test png my_file_dcm --log-level=debug
2200
- ascli preview test --case=test png my_file_docx --log-level=debug
2201
- ascli preview test --case=test png my_file_mxf --video-png-conv=animated --log-level=debug
2202
- ascli preview test --case=test png my_file_mxf --video-png-conv=fixed --log-level=debug
2203
- ascli preview test --case=test png my_file_pdf --log-level=debug
2204
- ascli preview trevents --once-only=yes --skip-types=office --log-level=info
2205
- ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user ctl all:status
2206
- ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user health app_services --format=nagios
2207
- ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user health asctlstatus --format=nagios --cmd-prefix='sudo '
2208
- ascli server -N -Ptst_hstsfaspex_ssh -Plocal_user nodeadmin -- -l
2209
- ascli server -N -Ptst_server_bykey -Plocal_user br /
2210
- ascli server browse /
2211
- ascli server browse folder_1/target_hot
2212
- ascli server cp NEW_SERVER_FOLDER/testfile.bin folder_1/200KB.2
2213
- ascli server delete NEW_SERVER_FOLDER
2214
- ascli server delete folder_1/target_hot
2215
- ascli server delete folder_1/to.delete
2216
- ascli server df
2217
- ascli server download NEW_SERVER_FOLDER/testfile.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
2218
- ascli server download NEW_SERVER_FOLDER/testfile.bin --to-folder=folder_1 --transfer=node
2219
- ascli server du /
2220
- ascli server health transfer --to-folder=folder_1 --format=nagios
2221
- ascli server info
2222
- ascli server md5sum NEW_SERVER_FOLDER/testfile.bin
2223
- ascli server mkdir NEW_SERVER_FOLDER --logger=stdout
2224
- ascli server mkdir folder_1/target_hot
2225
- ascli server mv folder_1/200KB.2 folder_1/to.delete
2226
- ascli server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
2227
- ascli server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
2228
- ascli server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
2229
- ascli server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
2230
- ascli server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
2231
- ascli server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
2232
- ascli shares admin share list
2233
- ascli shares repository browse /
2234
- ascli shares repository delete my_shares_upload/testfile.bin
2235
- ascli shares repository download --to-folder=. my_shares_upload/testfile.bin
2236
- ascli shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
2237
- ascli shares repository upload --to-folder=my_shares_upload testfile.bin
2238
- ascli shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
2239
- ascli shares2 appinfo
2240
- ascli shares2 organization list
2241
- ascli shares2 project list --organization=Sport
2242
- ascli shares2 repository browse /
2243
- ascli shares2 userinfo
2244
- ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
2245
- ...and more
2246
- ```
2247
-
2248
2413
  ### <a id="usage"></a>Usage
2249
2414
 
2250
- ```bash
2415
+ ```text
2251
2416
  ascli -h
2252
2417
  NAME
2253
- ascli -- a command line tool for Aspera Applications (v4.7.0)
2418
+ ascli -- a command line tool for Aspera Applications (v4.9.0)
2254
2419
 
2255
2420
  SYNOPSIS
2256
- ascli COMMANDS [OPTIONS] [ARGS]
2421
+ ascli COMMANDS [OPTIONS] [ARGS]
2257
2422
 
2258
2423
  DESCRIPTION
2259
- Use Aspera application to perform operations on command line.
2260
- Documentation and examples: https://rubygems.org/gems/aspera-cli
2261
- execute: ascli conf doc
2262
- or visit: https://www.rubydoc.info/gems/aspera-cli
2263
- source repo: https://github.com/IBM/aspera-cli
2424
+ Use Aspera application to perform operations on command line.
2425
+ Documentation and examples: https://rubygems.org/gems/aspera-cli
2426
+ execute: ascli conf doc
2427
+ or visit: https://www.rubydoc.info/gems/aspera-cli
2428
+ source repo: https://github.com/IBM/aspera-cli
2264
2429
 
2265
2430
  ENVIRONMENT VARIABLES
2266
- ASCLI_HOME config folder, default: $HOME/.aspera/ascli
2267
- Any option can be set as an environment variable, refer to the manual
2431
+ ASCLI_HOME config folder, default: $HOME/.aspera/ascli
2432
+ Any option can be set as an environment variable, refer to the manual
2268
2433
 
2269
2434
  COMMANDS
2270
- To list first level commands, execute: ascli
2271
- Note that commands can be written shortened (provided it is unique).
2435
+ To list first level commands, execute: ascli
2436
+ Note that commands can be written shortened (provided it is unique).
2272
2437
 
2273
2438
  OPTIONS
2274
- Options begin with a '-' (minus), and value is provided on command line.
2275
- Special values are supported beginning with special prefix @pfx:, where pfx is one of:
2276
- base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, uri, stdin, preset
2277
- Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
2439
+ Options begin with a '-' (minus), and value is provided on command line.
2440
+ Special values are supported beginning with special prefix @pfx:, where pfx is one of:
2441
+ base64, json, zlib, ruby, csvt, lines, list, incps, val, file, path, env, uri, stdin, preset
2442
+ Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'
2278
2443
 
2279
2444
  ARGS
2280
- Some commands require mandatory arguments, e.g. a path.
2445
+ Some commands require mandatory arguments, e.g. a path.
2281
2446
 
2282
2447
  OPTIONS: global
2283
2448
  --interactive=ENUM use interactive input of missing params: yes, [no]
2284
2449
  --ask-options=ENUM ask even optional options: yes, [no]
2285
- --format=ENUM output format: [table], ruby, json, jsonpp, yaml, csv, nagios
2450
+ --format=ENUM output format: text, nagios, ruby, json, jsonpp, yaml, [table], csv
2286
2451
  --display=ENUM output only some information: [info], data, error
2287
2452
  --fields=VALUE comma separated list of fields, or ALL, or DEF
2288
2453
  --select=VALUE select only some items in lists, extended value: hash (column, value)
2289
2454
  --table-style=VALUE table display style
2290
2455
  --flat-hash=ENUM display hash values as additional keys: [yes], no
2291
2456
  --transpose-single=ENUM single object fields output vertically: [yes], no
2292
- --show-secrets=ENUM show secrets on command output: [yes], no
2457
+ --show-secrets=ENUM show secrets on command output: yes, [no]
2293
2458
  -h, --help Show this message.
2294
2459
  --bash-comp generate bash completion for command
2295
2460
  --show-config Display parameters used for the provided action.
@@ -2313,6 +2478,7 @@ OPTIONS:
2313
2478
  --value=VALUE extended value for create, update, list filter
2314
2479
  --property=VALUE name of property to set
2315
2480
  --id=VALUE resource identifier (modify,delete,show)
2481
+ --bulk=ENUM Bulk operation (only some): yes, [no]
2316
2482
  --config-file=VALUE read parameters from file in YAML format, current=/usershome/.aspera/ascli/config.yaml
2317
2483
  -N, --no-default do not load default configuration for plugin
2318
2484
  --override=ENUM Wizard: override existing value: yes, [no]
@@ -2335,16 +2501,16 @@ OPTIONS:
2335
2501
  --plugin-folder=VALUE folder where to find additional plugins
2336
2502
  --ts=VALUE override transfer spec values (Hash, use @json: prefix), current={"create_dir"=>true}
2337
2503
  --local-resume=VALUE set resume policy (Hash, use @json: prefix), current=
2338
- --to-folder=VALUE destination folder for downloaded files
2339
- --sources=VALUE list of source files (see doc)
2340
- --transfer-info=VALUE parameters for transfer agent
2504
+ --to-folder=VALUE destination folder for transfered files
2505
+ --sources=VALUE how list of transfered files is provided (@args,@ts,Array)
2341
2506
  --src-type=ENUM type of file list: list, pair
2342
2507
  --transfer=ENUM type of transfer agent: direct, node, connect, httpgw, trsdk
2508
+ --transfer-info=VALUE parameters for transfer agent
2343
2509
  --progress=ENUM type of progress bar: none, native, multi
2344
2510
 
2345
2511
 
2346
2512
  COMMAND: shares
2347
- SUBCOMMANDS: repository admin
2513
+ SUBCOMMANDS: health repository admin
2348
2514
  OPTIONS:
2349
2515
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2350
2516
  --username=VALUE username to log in
@@ -2364,7 +2530,7 @@ OPTIONS:
2364
2530
 
2365
2531
 
2366
2532
  COMMAND: orchestrator
2367
- SUBCOMMANDS: info workflow plugins processes
2533
+ SUBCOMMANDS: health info workflow plugins processes
2368
2534
  OPTIONS:
2369
2535
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2370
2536
  --username=VALUE username to log in
@@ -2405,27 +2571,29 @@ OPTIONS:
2405
2571
 
2406
2572
 
2407
2573
  COMMAND: faspex5
2408
- SUBCOMMANDS: package admin
2574
+ SUBCOMMANDS: health package admin user
2409
2575
  OPTIONS:
2410
2576
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2411
2577
  --username=VALUE username to log in
2412
2578
  --password=VALUE user's password
2413
2579
  --client-id=VALUE OAuth client identifier
2414
2580
  --client-secret=VALUE OAuth client secret
2415
- --redirect-uri=VALUE OAuth redirect URI
2581
+ --redirect-uri=VALUE OAuth redirect URI for web authentication
2416
2582
  --auth=ENUM OAuth type of authentication: web, jwt, boot
2417
- --private-key=VALUE Oauth RSA private key PEM value for JWT (prefix file path with @val:@file:)
2583
+ --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2584
+ --passphrase=VALUE RSA private key passphrase
2418
2585
 
2419
2586
 
2420
2587
  COMMAND: cos
2421
2588
  SUBCOMMANDS: node
2422
2589
  OPTIONS:
2423
- --bucket=VALUE IBM Cloud Object Storage bucket name
2424
- --endpoint=VALUE storage endpoint url
2425
- --apikey=VALUE storage API key
2426
- --crn=VALUE ressource instance id
2590
+ --bucket=VALUE Bucket name
2591
+ --endpoint=VALUE Storage endpoint url
2592
+ --apikey=VALUE Storage API key
2593
+ --crn=VALUE Ressource instance id
2427
2594
  --service-credentials=VALUE IBM Cloud service credentials (Hash)
2428
- --region=VALUE IBM Cloud Object storage region
2595
+ --region=VALUE Storage region
2596
+ --identity=VALUE Authentication url (https://iam.cloud.ibm.com/identity)
2429
2597
 
2430
2598
 
2431
2599
  COMMAND: faspex
@@ -2462,9 +2630,9 @@ OPTIONS:
2462
2630
  --file-access=ENUM how to read and write files in repository: [local], remote
2463
2631
  --max-size=VALUE maximum size (in bytes) of preview file
2464
2632
  --thumb-vid-scale=VALUE png: video: size (ffmpeg scale argument)
2465
- --thumb-vid-fraction=VALUE png: video: position of snapshot
2633
+ --thumb-vid-fraction=VALUE png: video: time percent position of snapshot
2466
2634
  --thumb-img-size=VALUE png: non-video: height (and width)
2467
- --thumb-text-font=VALUE png: plaintext: font to render text with image magick convert, list with: identify -list font
2635
+ --thumb-text-font=VALUE png: plaintext: font to render text with imagemagick convert (identify -list font)
2468
2636
  --video-conversion=ENUM mp4: method for preview generation: [reencode], blend, clips
2469
2637
  --video-png-conv=ENUM mp4: method for thumbnail generation: [fixed], animated
2470
2638
  --video-start-sec=VALUE mp4: start offset (seconds) of video preview
@@ -2495,27 +2663,27 @@ OPTIONS:
2495
2663
  --client-id=VALUE OAuth API client identifier in application
2496
2664
  --client-secret=VALUE OAuth API client passcode
2497
2665
  --redirect-uri=VALUE OAuth API client redirect URI
2498
- --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @val:@file:)
2666
+ --private-key=VALUE OAuth JWT RSA private key PEM value (prefix file path with @file:)
2667
+ --passphrase=VALUE RSA private key passphrase
2499
2668
  --workspace=VALUE name of workspace
2500
2669
  --name=VALUE resource name
2501
2670
  --path=VALUE file or folder path
2502
2671
  --link=VALUE public link to shared resource
2503
- --new-user-option=VALUE new user creation option
2672
+ --new-user-option=VALUE new user creation option for unknown package recipients
2504
2673
  --from-folder=VALUE share to share source folder
2505
2674
  --scope=VALUE OAuth scope for AoC API calls
2506
- --bulk=ENUM bulk operation: yes, [no]
2507
2675
  --default-ports=ENUM use standard FASP ports or get from node api: yes, [no]
2676
+ --validate-metadata=ENUM validate shared inbox metadata: yes, [no]
2508
2677
 
2509
2678
 
2510
2679
  COMMAND: server
2511
- SUBCOMMANDS: health nodeadmin userdata configurator ctl download upload browse delete rename ls rm mv du info mkdir cp df md5sum
2680
+ SUBCOMMANDS: health download upload browse delete rename ls rm mv du info mkdir cp df md5sum
2512
2681
  OPTIONS:
2513
2682
  --url=VALUE URL of application, e.g. https://org.asperafiles.com
2514
2683
  --username=VALUE username to log in
2515
2684
  --password=VALUE user's password
2516
- --ssh-keys=VALUE ssh key path list (Array or single)
2517
- --ssh-options=VALUE ssh options (Hash)
2518
- --cmd-prefix=VALUE prefix to add for as cmd execution, e.g. sudo or /opt/aspera/bin
2685
+ --ssh-keys=VALUE SSH key path list (Array or single)
2686
+ --ssh-options=VALUE SSH options (Hash)
2519
2687
 
2520
2688
 
2521
2689
  COMMAND: console
@@ -2532,9 +2700,15 @@ OPTIONS:
2532
2700
 
2533
2701
  Note that actions and parameter values can be written in short form.
2534
2702
 
2535
- ## <a id="aoc"></a>Plugin: Aspera on Cloud
2703
+ ### Bulk creation and deletion of resources
2536
2704
 
2537
- Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
2705
+ Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
2706
+ In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2707
+ This option is available only for some of the resources: if you need it: try and see if the entities you try to create or delete support this option.
2708
+
2709
+ ## <a id="aoc"></a>Plugin: Aspera on Cloud
2710
+
2711
+ Aspera on Cloud uses the more advanced Oauth v2 mechanism for authentication (HTTP Basic authentication is not supported).
2538
2712
 
2539
2713
  It is recommended to use the wizard to set it up, but manual configuration is also possible.
2540
2714
 
@@ -2542,7 +2716,7 @@ It is recommended to use the wizard to set it up, but manual configuration is al
2542
2716
 
2543
2717
  `ascli` provides a configuration wizard. Here is a sample invocation :
2544
2718
 
2545
- ```bash
2719
+ ```text
2546
2720
  ascli config wizard
2547
2721
  option: url> https://myorg.ibmaspera.com
2548
2722
  Detected: Aspera on Cloud
@@ -2575,7 +2749,7 @@ ascli config wizard --value=aoc
2575
2749
 
2576
2750
  ### <a id="aocmanual"></a>Configuration: using manual setup
2577
2751
 
2578
- If you used the wizard (recommended): skip this section.
2752
+ > If you used the wizard (recommended): skip this section.
2579
2753
 
2580
2754
  #### Configuration details
2581
2755
 
@@ -2646,7 +2820,7 @@ Note: Default `auth` method is `web` and default `redirect_uri` is `http://local
2646
2820
  For a Browser-less, Private Key-based authentication, use the following steps.
2647
2821
 
2648
2822
  In order to use JWT for Aspera on Cloud API client authentication,
2649
- a [private/public key pair](#private_key) must be used (without passphrase)
2823
+ a [private/public key pair](#private_key) must be used.
2650
2824
 
2651
2825
  ##### API Client JWT activation
2652
2826
 
@@ -2654,7 +2828,7 @@ If you are not using the built-in client_id and secret, JWT needs to be authoriz
2654
2828
 
2655
2829
  * Graphically
2656
2830
 
2657
- * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
2831
+ * Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
2658
2832
  * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
2659
2833
  * Click on the previously created application
2660
2834
  * select tab : "JSON Web Token Auth"
@@ -2691,7 +2865,7 @@ The public key must be assigned to your user. This can be done in two manners:
2691
2865
 
2692
2866
  Open the previously generated public key located here: `$HOME/.aspera/ascli/my_private_key.pub`
2693
2867
 
2694
- * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
2868
+ * Open a web browser, log to your instance: `https://myorg.ibmaspera.com/`
2695
2869
  * Click on the user's icon (top right)
2696
2870
  * Select "Account Settings"
2697
2871
  * Paste the *Public Key* in the "Public Key" section
@@ -2736,7 +2910,7 @@ Execute:
2736
2910
  ascli config preset update my_aoc_org --auth=jwt --private-key=@val:@file:~/.aspera/ascli/my_private_key --username=laurent.martin.aspera@fr.ibm.com
2737
2911
  ```
2738
2912
 
2739
- Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.
2913
+ Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the `@file:` prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the `@val:` prefix is added.
2740
2914
 
2741
2915
  After this last step, commands do not require web login anymore.
2742
2916
 
@@ -2753,17 +2927,33 @@ Current Workspace: Default Workspace (default)
2753
2927
  empty
2754
2928
  ```
2755
2929
 
2930
+ ### Calling AoC APIs from command line
2931
+
2932
+ The command `ascli aoc bearer` can be used to generate an OAuth token suitable to call any AoC API (use the `scope` option to change the scope, default is `user:all`).
2933
+ This can be useful when a command is not yet available.
2934
+
2935
+ Example:
2936
+
2937
+ ```bash
2938
+ curl -s -H "Authorization: $(ascli aoc bearer_token)" 'https://api.ibmaspera.com/api/v1/group_memberships?embed[]=dropbox&embed[]=workspace'|jq -r '.[]|(.workspace.name + " -> " + .dropbox.name)'
2939
+ ```
2940
+
2941
+ It is also possible to get the bearer token for node, as user or as admin using:
2942
+
2943
+ ```bash
2944
+ ascli aoc files bearer_token_node /
2945
+ ```
2946
+
2947
+ ```bash
2948
+ ascli aoc admin res node v4 1234 --secret=_ak_secret_here_ bearer_token_node /
2949
+ ```
2950
+
2756
2951
  ### Administration
2757
2952
 
2758
2953
  The `admin` command allows several administrative tasks (and require admin privilege).
2759
2954
 
2760
2955
  It allows actions (create, update, delete) on "resources": users, group, nodes, workspace, etc... with the `admin resource` command.
2761
2956
 
2762
- #### Bulk creation and deletion of resource
2763
-
2764
- Bulk creation and deletion of resources are possible using option `bulk` (yes,no(default)).
2765
- In that case, the operation expects an Array of Hash instead of a simple Hash using the [Extended Value Syntax](#extended).
2766
-
2767
2957
  #### Listing resources
2768
2958
 
2769
2959
  The command `aoc admin res <type> list` lists all entities of given type. It uses paging and multiple requests if necessary.
@@ -2825,6 +3015,50 @@ To execute an action on a specific resource, select it using one of those method
2825
3015
  * provide option `id` : `aoc admin res node show --id=123`
2826
3016
  * provide option `name` : `aoc admin res node show --name=abc`
2827
3017
 
3018
+ #### <a id="res_create"></a>Creating a resource
3019
+
3020
+ New resources (users, groups, workspaces, etc..) can be created using a command like:
3021
+
3022
+ ```bash
3023
+ ascli aoc admin res create <resource type> @json:'{<...parameters...>}'
3024
+ ```
3025
+
3026
+ Some of the API endpoints are described [here](https://developer.ibm.com/apis/catalog?search=%22aspera%20on%20cloud%20api%22). Sadly, not all.
3027
+
3028
+ Nevertheless, it is possible to guess the structure of the creation value by simply dumping an existing resource, and use the same parameters for the creation.
3029
+
3030
+ ```bash
3031
+ ascli aoc admin res group show 12345 --format=json
3032
+ ```
3033
+
3034
+ ```json
3035
+ {"created_at":"2018-07-24T21:46:39.000Z","description":null,"id":"12345","manager":false,"name":"A8Demo WS1","owner":false,"queued_operation_count":0,"running_operation_count":0,"stopped_operation_count":0,"updated_at":"2018-07-24T21:46:39.000Z","saml_group":false,"saml_group_dn":null,"system_group":true,"system_group_type":"workspace_members"}
3036
+ ```
3037
+
3038
+ Remove the parameters that are either obviously added by the system: `id`, `created_at`, `updated_at` or optional.
3039
+
3040
+ And then craft your command:
3041
+
3042
+ ```bash
3043
+ ascli aoc admin res group create @json:'{"description":"test to delete","name":"test 1 to delete","saml_group":false}'
3044
+ ```
3045
+
3046
+ If the command returns an error, example:
3047
+
3048
+ ```output
3049
+ +----+-----------------------------------------------------------------------------------+
3050
+ | id | status |
3051
+ +----+-----------------------------------------------------------------------------------+
3052
+ | | found unpermitted parameters: :manager, :owner, :system_group, :system_group_type |
3053
+ | | code: unpermitted_parameters |
3054
+ | | request_id: b0f45d5b-c00a-4711-acef-72b633f8a6ea |
3055
+ | | api.ibmaspera.com 422 Unprocessable Entity |
3056
+ +----+-----------------------------------------------------------------------------------+```
3057
+
3058
+ Well, remove the offending parameters and try again.
3059
+
3060
+ Note that some properties that are shown in the web UI, such as membership, are not listed directly in the resource, but instead another resource is created to link a user and its group: `group_membership`
3061
+
2828
3062
  #### Access Key secrets
2829
3063
 
2830
3064
  In order to access some administrative actions on "nodes" (in fact, access keys), the associated secret is required.
@@ -3018,7 +3252,7 @@ ascli aoc admin res workspace_membership list --fields=member_type,manager,membe
3018
3252
  :.............:.........:..................................:
3019
3253
  ```
3020
3254
 
3021
- other query parameters:
3255
+ Other query parameters:
3022
3256
 
3023
3257
  ```javascript
3024
3258
  {"workspace_membership_through":true,"include_indirect":true}
@@ -3085,6 +3319,44 @@ ascli aoc admin res user list --fields=email --query=@json:'{"q":"last_login_at:
3085
3319
  ascli aoc admin res user list --fields=email --select=@json:'{"member_of_any_workspace":false}'
3086
3320
  ```
3087
3321
 
3322
+ #### Example: create a group, add to workspace and add user to group
3323
+
3324
+ * Create the group and take note of `id`
3325
+
3326
+ ```bash
3327
+ ascli aoc admin res group create @json:'{"name":"group 1","description":"my super group"}'
3328
+ ```
3329
+
3330
+ Group: `11111`
3331
+
3332
+ * Get the workspace id
3333
+
3334
+ ```bash
3335
+ ascli aoc admin res workspace list --query=@json:'{"q":"myworkspace"}' --fields=id --format=csv --display=data
3336
+ ```
3337
+
3338
+ Workspace: 22222
3339
+
3340
+ * Add group to workspace
3341
+
3342
+ ```bash
3343
+ ascli aoc admin res workspace_membership create @json:'{"workspace_id":22222,"member_type":"user","member_id":11111}'
3344
+ ```
3345
+
3346
+ * Get a user's id
3347
+
3348
+ ```bash
3349
+ ascli aoc admin res user list --query=@json:'{"q":"manu.macron@example.com"}' --fields=id --format=csv --display=data
3350
+ ```
3351
+
3352
+ User: 33333
3353
+
3354
+ * Add user to group
3355
+
3356
+ ```bash
3357
+ ascli aoc admin res group_membership create @json:'{"group_id":11111,"member_type":"user","member_id":33333}'
3358
+ ```
3359
+
3088
3360
  #### Example: Perform a multi Gbps transfer between two remote shared folders
3089
3361
 
3090
3362
  In this example, a user has access to a workspace where two shared folders are located on different sites, e.g. different cloud regions.
@@ -3154,40 +3426,64 @@ ascli aoc admin res client list --fields=id --format=csv|ascli aoc admin res cli
3154
3426
  +-----+---------+
3155
3427
  ```
3156
3428
 
3157
- #### Example: Create a node
3429
+ #### Example: Create a Node
3158
3430
 
3159
3431
  AoC nodes as actually composed with two related entities:
3160
3432
 
3161
3433
  * An access key created on the Transfer Server (HSTS/ATS)
3162
3434
  * a `node` resource in the AoC application.
3163
3435
 
3164
- The web UI allows creation of both entities in one shot but not the CLI for more flexibility.
3165
- Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation.
3436
+ The web UI allows creation of both entities in one shot.
3437
+ For more flexibility, `ascli` allows this in two separate steps.
3438
+ Note that when selecting "Use existing access key" in the web UI, this actually skips access key creation (first step).
3166
3439
 
3167
3440
  So, for example, the creation of a node using ATS in IBM Cloud looks like (see other example in this manual):
3168
3441
 
3169
- * create the access key on ATS
3442
+ * Create the access key on ATS
3170
3443
 
3171
- ```javascript
3172
- ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
3173
- ```
3444
+ The creation options are the ones of ATS API, refer to the [section on ATS](#ats_params) for more details and examples.
3174
3445
 
3175
- Take a note of the randomly generated `id` and `secret`.
3446
+ ```javascript
3447
+ ascli aoc admin ats access_key create --cloud=softlayer --region=eu-de --params=@json:'{"storage":{"type":"ibm-s3","bucket":"mybucket","credentials":{"access_key_id":"mykey","secret_access_key":"mysecret"},"path":"/"}}'
3448
+ ```
3176
3449
 
3177
- * Retrieve the ATS node address
3450
+ Once executed, the access key `id` and `secret`, randomly generated by the node api, is displayed: take note of it as the secrets will not be disclosed again.
3178
3451
 
3179
- ```bash
3180
- ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
3181
- ```
3452
+ * Create the AoC node entity
3182
3453
 
3183
- * Create the node entity
3454
+ First, Retrieve the ATS node address
3184
3455
 
3185
- ```javascript
3186
- ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
3187
- ```
3456
+ ```bash
3457
+ ascli aoc admin ats cluster show --cloud=softlayer --region=eu-de --fields=transfer_setup_url --format=csv --transpose-single=no
3458
+ ```
3459
+
3460
+ Then use the returned address for the `url` key to actually create the AoC Node entity:
3461
+
3462
+ ```javascript
3463
+ ascli aoc admin res node create @json:'{"name":"myname","access_key":"*accesskeyid*","ats_access_key":true,"ats_storage_type":"ibm-s3","url":"https://ats-sl-fra-all.aspera.io"}'
3464
+ ```
3188
3465
 
3189
3466
  Creation of a node with a self-managed node is similar, but the command `aoc admin ats access_key create` is replaced with `node access_key create` on the private node itself.
3190
3467
 
3468
+ ### List of files to transfer
3469
+
3470
+ Source files are provided as a list with the `sources` option. Refer to section [File list](#file_list)
3471
+
3472
+ Note the special case when the source files are located on "Aspera on Cloud" (i.e. using access keys and the `file id` API).
3473
+
3474
+ Source files are located on "Aspera on cloud", when :
3475
+
3476
+ * the server is Aspera on Cloud, and executing a download or recv
3477
+ * the agent is Aspera on Cloud, and executing an upload or send
3478
+
3479
+ In this case:
3480
+
3481
+ * If there is a single file : specify the full path
3482
+ * Else, if there are multiple files:
3483
+ * All source files must be in the same source folder
3484
+ * Specify the source folder as first item in the list
3485
+ * followed by the list of file names.
3486
+
3191
3487
  ### Packages
3192
3488
 
3193
3489
  The webmail-like application.
@@ -3255,6 +3551,31 @@ shbxid=$(ascli aoc packages shared_inboxes show name 'My Shared Inbox' --format=
3255
3551
  ascli aoc packages list --query=@json:'{"dropbox_id":"'$shbxid'","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false,"include_draft":false,"sort":"-received_at"}'
3256
3552
  ```
3257
3553
 
3554
+ #### Example: Send a package with files from the Files app
3555
+
3556
+ Find files in Files app:
3557
+
3558
+ ```bash
3559
+ ascli aoc files browse /src_folder
3560
+ ```
3561
+
3562
+ ```bash
3563
+ +------------------------------+--------+----------------+--------------+----------------------+--------------+
3564
+ | name | type | recursive_size | size | modified_time | access_level |
3565
+ +------------------------------+--------+----------------+--------------+----------------------+--------------+
3566
+ | sample_video | link | | | 2020-11-29T22:49:09Z | edit |
3567
+ | 100G | file | | 107374182400 | 2021-04-21T18:19:25Z | edit |
3568
+ | 10M.dat | file | | 10485760 | 2021-05-18T08:22:39Z | edit |
3569
+ | Test.pdf | file | | 1265103 | 2022-06-16T12:49:55Z | edit |
3570
+ +------------------------------+--------+----------------+--------------+----------------------+--------------+
3571
+ ```
3572
+
3573
+ Let's send a package with the file `10M.dat` from subfolder /src_folder in a package:
3574
+
3575
+ ```bash
3576
+ ascli aoc files node_info /src_folder --format=json --display=data | ascli aoc package send --value=@json:'{"name":"test","recipients":["laurent.martin.aspera@fr.ibm.com"]}' 10M.dat --transfer=node --transfer-info=@json:@stdin:
3577
+ ```
3578
+
3258
3579
  #### <a id="aoccargo"></a>Receive new packages only (Cargo)
3259
3580
 
3260
3581
  It is possible to automatically download new packages, like using Aspera Cargo:
@@ -3279,26 +3600,47 @@ Folder sharing app.
3279
3600
 
3280
3601
  #### Download Files
3281
3602
 
3282
- Download of files is straightforward with a specific syntax for the `aoc files download` action: Like other commands the source file list is provided as a list with the `sources` option. Nevertheless, consider this:
3283
-
3284
- * if only one source is provided, it is downloaded
3285
- * if multiple sources must be downloaded, then the first in list is the path of the source folder, and the remaining items are the file names in this folder (without path).
3603
+ Download of files is straightforward using command: `aoc files download`
3286
3604
 
3287
3605
  #### Shared folders
3288
3606
 
3289
- * list shared folders in node
3607
+ Shared folder by users are managed through **permissions**.
3608
+ For creation, parameters are the same as for node api [permissions](https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--node-api#post960739960).
3609
+ `ascli` expects the same payload for creation, but it will automatically populated required tags if needed.
3610
+ Also, the pseudo key `with` is added: it will lookup the name in the contacts and fill the proper type and id.
3611
+ The pseudo parameter `link_name` allows changing default "shared as" name.
3612
+
3613
+ * List permissions on a shared folder as user
3614
+
3615
+ ```bash
3616
+ ascli aoc files file --path=/shared_folder_test1 perm list
3617
+ ```
3618
+
3619
+ * Share a personal folder with other users
3620
+
3621
+ ```bash
3622
+ ascli aoc files file --path=/shared_folder_test1 perm create @json:'{"with":"laurent"}'
3623
+ ```
3624
+
3625
+ * Revoke shared access
3626
+
3627
+ ```bash
3628
+ ascli aoc files file --path=/shared_folder_test1 perm delete 6161
3629
+ ```
3630
+
3631
+ * List shared folders in node
3290
3632
 
3291
3633
  ```bash
3292
3634
  ascli aoc admin res node --id=8669 shared_folders
3293
3635
  ```
3294
3636
 
3295
- * list shared folders in workspace
3637
+ * List shared folders in workspace
3296
3638
 
3297
3639
  ```bash
3298
3640
  ascli aoc admin res workspace --id=10818 shared_folders
3299
3641
  ```
3300
3642
 
3301
- * list members of shared folder
3643
+ * List members of shared folder
3302
3644
 
3303
3645
  ```bash
3304
3646
  ascli aoc admin res node --id=8669 v4 perm 82 show
@@ -3373,6 +3715,99 @@ cat my_file_list.txt|while read path;do echo ascli aoc admin res node --name='my
3373
3715
  cat my_file_list.txt | ascli aoc admin res node --name='my node name' --secret='_secret_here_' v3 delete @lines:@stdin:
3374
3716
  ```
3375
3717
 
3718
+ ### AoC sample commands
3719
+
3720
+ ```bash
3721
+ aoc -N remind --username=my_aoc_user_email
3722
+ aoc -N servers
3723
+ aoc admin analytics transfers --query=@json:'{"status":"completed","direction":"receive"}' --notif-to=my_recipient_email --notif-template=@ruby:'%Q{From: <%=from_name%> <<%=from_email%>>\nTo: <<%=to%>>\nSubject: <%=ev["files_completed"]%> files received\n\n<%=ev.to_yaml%>}'
3724
+ aoc admin ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3725
+ aoc admin ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3726
+ aoc admin ats access_key delete akibmcloud
3727
+ aoc admin ats access_key list --fields=name,id
3728
+ aoc admin ats access_key node akibmcloud --secret=somesecret browse /
3729
+ aoc admin ats cluster clouds
3730
+ aoc admin ats cluster list
3731
+ aoc admin ats cluster show --cloud=aws --region=eu-west-1
3732
+ aoc admin ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
3733
+ aoc admin res application list
3734
+ aoc admin res client list
3735
+ aoc admin res client_access_key list
3736
+ aoc admin res client_registration_token create @json:'{"data":{"name":"test_client_reg1","client_subject_scopes":["alee","aejd"],"client_subject_enabled":true}}'
3737
+ aoc admin res client_registration_token delete my_clt_reg_id
3738
+ aoc admin res client_registration_token list
3739
+ aoc admin res contact list
3740
+ aoc admin res dropbox list
3741
+ aoc admin res dropbox_membership list
3742
+ aoc admin res group list
3743
+ aoc admin res kms_profile list
3744
+ aoc admin res node list
3745
+ aoc admin res operation list
3746
+ aoc admin res organization show
3747
+ aoc admin res package list --http-options=@json:'{"read_timeout":120.0}'
3748
+ aoc admin res saml_configuration list
3749
+ aoc admin res self show
3750
+ aoc admin res short_link list
3751
+ aoc admin res user list
3752
+ aoc admin res workspace_membership list
3753
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 access_key create --value=@json:'{"id":"testsub1","storage":{"path":"/folder1"}}'
3754
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v3 events
3755
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 browse /
3756
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 delete /folder1
3757
+ aoc admin resource node --name=my_aoc_ak_name --secret=my_aoc_ak_secret v4 mkdir /folder1
3758
+ aoc admin resource node v3 name my_aoc_ak_name --secret=my_aoc_ak_secret access_key delete testsub1
3759
+ aoc admin resource workspace list
3760
+ aoc admin resource workspace_membership list --fields=ALL --query=@json:'{"page":1,"per_page":50,"embed":"member","inherited":false,"workspace_id":11363,"sort":"name"}'
3761
+ aoc automation workflow action my_wf_id create --value=@json:'{"name":"toto"}'
3762
+ aoc automation workflow create --value=@json:'{"name":"test_workflow"}'
3763
+ aoc automation workflow delete my_wf_id
3764
+ aoc automation workflow list
3765
+ aoc automation workflow list --select=@json:'{"name":"test_workflow"}' --fields=id --format=csv --display=data
3766
+ aoc automation workflow list --value=@json:'{"show_org_workflows":"true"}' --scope=admin:all
3767
+ aoc bearer_token --display=data --scope=user:all
3768
+ aoc faspex
3769
+ aoc files bearer /
3770
+ aoc files bearer_token_node / --cache-tokens=no
3771
+ aoc files browse /
3772
+ aoc files browse / -N --link=my_aoc_publink_folder
3773
+ aoc files delete /testsrc
3774
+ aoc files download --transfer=connect /200KB.1
3775
+ aoc files file permission --path=/ascli_test list
3776
+ aoc files file show --path=/200KB.1
3777
+ aoc files file show my_file_id
3778
+ aoc files find / --value='\.partial$'
3779
+ aoc files http_node_download --to-folder=. /200KB.1
3780
+ aoc files mkdir /testsrc
3781
+ aoc files rename /somefolder testdst
3782
+ aoc files short_link create --to-folder=/testdst --value=private
3783
+ aoc files short_link create --to-folder=/testdst --value=public
3784
+ aoc files short_link list --value=@json:'{"purpose":"shared_folder_auth_link"}'
3785
+ aoc files transfer --from-folder=/testsrc --to-folder=/testdst testfile.bin
3786
+ aoc files upload --to-folder=/testsrc testfile.bin
3787
+ aoc files upload -N --to-folder=/ testfile.bin --link=my_aoc_publink_folder
3788
+ aoc files upload Test.pdf --transfer=node --transfer-info=@json:@stdin:
3789
+ aoc files v3 info
3790
+ aoc org -N --link=my_aoc_publink_recv_from_aocuser
3791
+ aoc organization
3792
+ aoc packages browse "my_package_id" /contents
3793
+ aoc packages list
3794
+ aoc packages list --query=@json:'{"dropbox_name":"my_aoc_shbx_name","sort":"-received_at","archived":false,"received":true,"has_content":true,"exclude_dropbox_packages":false}'
3795
+ aoc packages recv "my_package_id" --to-folder=.
3796
+ aoc packages recv ALL --to-folder=. --once-only=yes --lock-port=12345
3797
+ aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_external_user"]}' --new-user-option=@json:'{"package_contact":true}' testfile.bin
3798
+ aoc packages send --value=@json:'{"name":"Important files delivery","recipients":["my_email_internal_user"],"note":"my note"}' testfile.bin
3799
+ aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":[{"input_type":"single-text","name":"Project Id","values":["123"]},{"input_type":"single-dropdown","name":"Type","values":["Opt2"]},{"input_type":"multiple-checkbox","name":"CheckThose","values":["Check1","Check2"]},{"input_type":"date","name":"Optional Date","values":["2021-01-13T15:02:00.000Z"]}]}' testfile.bin
3800
+ aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"],"metadata":{"Project Id":"456","Type":"Opt2","CheckThose":["Check1","Check2"],"Optional Date":"2021-01-13T15:02:00.000Z"}}' testfile.bin
3801
+ aoc packages send --workspace="my_aoc_shbx_ws" --value=@json:'{"name":"Important files delivery","recipients":["my_aoc_shbx_name"]}' testfile.bin
3802
+ aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_aoc_user --password=my_aoc_publink_send_use_pass
3803
+ aoc packages send -N --value=@json:'{"name":"Important files delivery"}' testfile.bin --link=my_aoc_publink_send_shd_inbox
3804
+ aoc packages shared_inboxes list
3805
+ aoc user profile modify @json:'{"name":"dummy change"}'
3806
+ aoc user profile show
3807
+ aoc user workspaces current
3808
+ aoc user workspaces list
3809
+ ```
3810
+
3376
3811
  ## <a id="ats"></a>Plugin: Aspera Transfer Service
3377
3812
 
3378
3813
  ATS is usable either :
@@ -3451,6 +3886,10 @@ ascli ats api_key create
3451
3886
  ascli config preset update my_ibm_ats --ats-key=ats_XXXXXXXXXXXXXXXXXXXXXXXX --ats-secret=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
3452
3887
  ```
3453
3888
 
3889
+ ### <a id="ats_params"></a>ATS Access key creation parameters
3890
+
3891
+ When creating an ATS access key, the option `params` must contain an extended value with the creation parameters. Those asre directly the parameters expected by the [ATS API](https://developer.ibm.com/apis/catalog?search=%22Aspera%20ATS%20API%22).
3892
+
3454
3893
  ### Misc. Examples
3455
3894
 
3456
3895
  Example: create access key on IBM Cloud (softlayer):
@@ -3487,27 +3926,82 @@ for k in $(ascli ats access_key list --field=id --format=csv);do ascli ats acces
3487
3926
 
3488
3927
  The parameters provided to ATS for access key creation are the ones of [ATS API](https://developer.ibm.com/apis/catalog?search=%22aspera%20ats%22) for the `POST /access_keys` endpoint.
3489
3928
 
3929
+ ### ATS sample commands
3930
+
3931
+ ```bash
3932
+ ats access_key cluster akibmcloud --secret=somesecret
3933
+ ats access_key create --cloud=aws --region=my_aws_bucket_region --params=@json:'{"id":"ak_aws","name":"my test key AWS","storage":{"type":"aws_s3","bucket":"my_aws_bucket_name","credentials":{"access_key_id":"my_aws_bucket_key","secret_access_key":"my_aws_bucket_secret"},"path":"/"}}'
3934
+ ats access_key create --cloud=softlayer --region=my_icos_bucket_region --params=@json:'{"id":"akibmcloud","secret":"somesecret","name":"my test key","storage":{"type":"ibm-s3","bucket":"my_icos_bucket_name","credentials":{"access_key_id":"my_icos_bucket_key","secret_access_key":"my_icos_bucket_secret"},"path":"/"}}'
3935
+ ats access_key delete ak_aws
3936
+ ats access_key delete akibmcloud
3937
+ ats access_key list --fields=name,id
3938
+ ats access_key node akibmcloud browse / --secret=somesecret
3939
+ ats api_key create
3940
+ ats api_key instances
3941
+ ats api_key list
3942
+ ats cluster clouds
3943
+ ats cluster list
3944
+ ats cluster show --cloud=aws --region=eu-west-1
3945
+ ats cluster show 1f412ae7-869a-445c-9c05-02ad16813be2
3946
+ ```
3947
+
3490
3948
  ## Plugin: IBM Aspera High Speed Transfer Server (transfer)
3491
3949
 
3492
3950
  This plugin uses SSH as a session protocol (using commands `ascp` and `ascmd`) and does not use the node API.
3493
3951
  It is the legacy way of accessing an Aspera Server, often used for server to server transfers.
3494
3952
  Modern mode is to use the node API and transfer tokens.
3495
3953
 
3496
- ### Authentication
3497
-
3498
- Both password and SSH keys auth are supported.
3954
+ ### Server sample commands
3955
+
3956
+ ```bash
3957
+ server -N -Ptst_server_bykey -Plocal_user br /
3958
+ server browse /
3959
+ server browse folder_1/target_hot
3960
+ server cp NEW_SERVER_FOLDER/testfile.bin folder_1/200KB.2
3961
+ server delete NEW_SERVER_FOLDER
3962
+ server delete folder_1/target_hot
3963
+ server delete folder_1/to.delete
3964
+ server df
3965
+ server download NEW_SERVER_FOLDER/testfile.bin --to-folder=. --transfer-info=@json:'{"wss":false,"resume":{"iter_max":1}}'
3966
+ server download NEW_SERVER_FOLDER/testfile.bin --to-folder=folder_1 --transfer=node
3967
+ server du /
3968
+ server health transfer --to-folder=folder_1 --format=nagios
3969
+ server info
3970
+ server md5sum NEW_SERVER_FOLDER/testfile.bin
3971
+ server mkdir NEW_SERVER_FOLDER --logger=stdout
3972
+ server mkdir folder_1/target_hot
3973
+ server mv folder_1/200KB.2 folder_1/to.delete
3974
+ server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-list","'"filelist.txt"'"]}' --to-folder=NEW_SERVER_FOLDER
3975
+ server upload --sources=@ts --ts=@json:'{"EX_ascp_args":["--file-pair-list","'"filepairlist.txt"'"]}'
3976
+ server upload --sources=@ts --ts=@json:'{"EX_file_list":"'"filelist.txt"'"}' --to-folder=NEW_SERVER_FOLDER
3977
+ server upload --sources=@ts --ts=@json:'{"EX_file_pair_list":"'"filepairlist.txt"'"}'
3978
+ server upload --sources=@ts --ts=@json:'{"paths":[{"source":"testfile.bin","destination":"NEW_SERVER_FOLDER/othername"}]}'
3979
+ server upload --src-type=pair --sources=@json:'["testfile.bin","NEW_SERVER_FOLDER/othername"]'
3980
+ server upload --src-type=pair testfile.bin NEW_SERVER_FOLDER/othername --notif-to=my_recipient_email
3981
+ server upload --src-type=pair testfile.bin folder_1/with_options --ts=@json:'{"cipher":"aes-192-gcm","content_protection":"encrypt","content_protection_password":"_secret_here_","cookie":"biscuit","create_dir":true,"delete_before_transfer":false,"delete_source":false,"exclude_newer_than":1,"exclude_older_than":10000,"fasp_port":33001,"http_fallback":false,"multi_session":0,"overwrite":"diff+older","precalculate_job_size":true,"preserve_access_time":true,"preserve_creation_time":true,"rate_policy":"fair","resume_policy":"sparse_csum","symlink_policy":"follow"}'
3982
+ server upload --to-folder=folder_1/target_hot --lock-port=12345 --ts=@json:'{"EX_ascp_args":["--remove-after-transfer","--remove-empty-directories","--exclude-newer-than=-8","--src-base","source_hot"]}' source_hot
3983
+ server upload testfile.bin --to-folder=NEW_SERVER_FOLDER --ts=@json:'{"multi_session":3,"multi_session_threshold":1,"resume_policy":"none","target_rate_kbps":1500}' --transfer-info=@json:'{"spawn_delay_sec":2.5}' --progress=multi
3984
+ ```
3985
+
3986
+ ### Authentication on Server with SSH session
3987
+
3988
+ If SSH is the session control protocol (i.e. not WSS), then following session authentication methods are supported:
3989
+
3990
+ * SSH password
3991
+ * SSH keys (Multiple SSH key paths can be provided.)
3499
3992
 
3500
3993
  If username is not provided, the default transfer user `xfer` is used.
3501
3994
 
3502
- If no SSH password or key is provided, and a token is provided in transfer spec, then standard bypass keys are used:
3995
+ If no SSH password or key is provided and a transfer token is provided in transfer spec (option `ts`), then standard SSH bypass keys are used. Example:
3503
3996
 
3504
3997
  ```javascript
3505
3998
  ascli server --url=ssh://... --ts=@json:'{"token":"Basic abc123"}'
3506
3999
  ```
3507
4000
 
3508
- Multiple SSH key paths can be provided.
3509
- The value of the parameter `ssh_keys` can be a single value or an array.
3510
- Each value is a path to a private key and is expanded (`~` is replaced with the user's home folder).
4001
+ > Note: If you need to use the Aspera public keys, then specify an empty token: `--ts=@json:'{"token":""}'` : Aspera public SSH keys will be used, but the protocol will ignore the empty token.
4002
+
4003
+ The value of the `ssh_keys` option can be a single value or an array.
4004
+ Each value is a **path** to a private key and is expanded (`~` is replaced with the user's home folder).
3511
4005
 
3512
4006
  Examples:
3513
4007
 
@@ -3517,8 +4011,10 @@ ascli server --ssh-keys=@list:,~/.ssh/id_rsa
3517
4011
  ascli server --ssh-keys=@json:'["~/.ssh/id_rsa"]'
3518
4012
  ```
3519
4013
 
3520
- The underlying ssh library `net::ssh` provides several options that may be used depending on environment.
3521
- By default the ssh library expect that an ssh-agent is running.
4014
+ For non-transfer related command (browse, delete), the ruby SSH client library `Net::SSH` is used and provides several options settable using option `ssh_options`.
4015
+ For a list of SSH client options, refer to the ruby documentation of [Net::SSH](http://net-ssh.github.io/net-ssh/Net/SSH.html).
4016
+
4017
+ By default the SSH library expect that a local ssh-agent is running.
3522
4018
 
3523
4019
  On Linux, if you get an error message such as:
3524
4020
 
@@ -3532,12 +4028,12 @@ or on Windows:
3532
4028
  ERROR -- net.ssh.authentication.agent: could not connect to ssh-agent: pageant process not running
3533
4029
  ```
3534
4030
 
3535
- This means that you don't have such an ssh agent running, then:
4031
+ This means that you don't have such an SSH agent running, then:
3536
4032
 
3537
4033
  * check env var: `SSH_AGENT_SOCK`
3538
- * check if the ssh key is protected with a passphrase
4034
+ * check if the SSH key is protected with a passphrase (then, use the `passphrase` SSH option)
3539
4035
  * [check the manual](https://net-ssh.github.io/ssh/v1/chapter-2.html#s2)
3540
- * To disable use of `ssh-agent`, use the option `ssh_option` like this:
4036
+ * To disable the use of `ssh-agent`, use the option `ssh_options` like this:
3541
4037
 
3542
4038
  ```bash
3543
4039
  ascli server --ssh-options=@ruby:'{use_agent: false}' ...
@@ -3652,7 +4148,7 @@ updated
3652
4148
 
3653
4149
  Scenario: Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner.
3654
4150
  We need to transfer files from this third party SHOD instance into our Azure BLOB storage.
3655
- Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which provides access to the node API.
4151
+ Simply create an "Aspera Transfer Service" instance, which provides access to the node API.
3656
4152
  Then create a configuration for the "SHOD" instance in the configuration file: in section "shares", a configuration named: awsshod.
3657
4153
  Create another configuration for the Azure ATS instance: in section "node", named azureats.
3658
4154
  Then execute the following command:
@@ -3669,6 +4165,47 @@ This will get transfer information from the SHOD instance and tell the Azure ATS
3669
4165
  ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
3670
4166
  ```
3671
4167
 
4168
+ ### Node sample commands
4169
+
4170
+ ```bash
4171
+ node -N -Ptst_node_preview access_key create --value=@json:'{"id":"aoc_1","storage":{"type":"local","path":"/"}}'
4172
+ node -N -Ptst_node_preview access_key delete aoc_1
4173
+ node access_key do my_aoc_ak_name br /
4174
+ node access_key list
4175
+ node api_details
4176
+ node async bandwidth 1
4177
+ node async counters 1
4178
+ node async files 1
4179
+ node async list
4180
+ node async show 1
4181
+ node async show ALL
4182
+ node basic_token
4183
+ node browse / -r
4184
+ node delete /todelete
4185
+ node delete @list:,/todelete,/tdlink,/delfile
4186
+ node delete folder_1/10MB.1
4187
+ node delete testfile.bin
4188
+ node download testfile.bin --to-folder=.
4189
+ node download testfile.bin --to-folder=. --token-type=hybrid
4190
+ node health
4191
+ node info --fpac='function FindProxyForURL(url,host){return "DIRECT"}'
4192
+ node license
4193
+ node mkdir /todelete
4194
+ node mkfile /delfile1 "hello world"
4195
+ node mklink /todelete /tdlink
4196
+ node rename / delfile1 delfile
4197
+ node search / --value=@json:'{"sort":"mtime"}'
4198
+ node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
4199
+ node service delete service1
4200
+ node service list
4201
+ node space /
4202
+ node transfer list --value=@json:'{"active_only":true}'
4203
+ node upload --to-folder="folder_1" --sources=@ts --ts=@json:'{"paths":[{"source":"/aspera-test-dir-small/10MB.1"}],"precalculate_job_size":true}' --transfer=node --transfer-info=@json:'{"url":"my_node_url","username":"my_node_user","password":"my_node_pass"}'
4204
+ node upload --username=my_aoc_ak_name --password=my_aoc_ak_secret testfile.bin --token-type=basic
4205
+ node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}'
4206
+ node upload testfile.bin --to-folder=folder_1 --ts=@json:'{"target_rate_cap_kbps":10000}' --token-type=hybrid
4207
+ ```
4208
+
3672
4209
  ## Plugin: IBM Aspera Faspex5
3673
4210
 
3674
4211
  This is currently in beta, limited operations are supported.
@@ -3689,47 +4226,47 @@ The API is listed in [Faspex 5 API Reference](https://developer.ibm.com/apis/cat
3689
4226
 
3690
4227
  This is the preferred method to use.
3691
4228
 
3692
- For JWT, create an API client in Faspex with JWT support:
4229
+ For `jwt`, create an API client in Faspex with JWT support:
3693
4230
 
3694
- * Identify a private key, if you don't have any refer to section [Private Key](#private_key)
4231
+ * Select a private key file: if you don't have any refer to section [Private Key](#private_key)
3695
4232
  * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
3696
4233
  * Activate JWT
3697
4234
  * Paste public key in the appropriate section
3698
4235
  * Click on Create Button
3699
- * Take note of Client Id and Secret
4236
+ * Take note of Client Id (and Client Secret, but not used in current version)
3700
4237
 
3701
- Then use options:
4238
+ Then use these options:
3702
4239
 
3703
4240
  ```text
3704
4241
  --auth=jwt
3705
4242
  --client-id=xxx
3706
4243
  --client-secret=xxx
3707
4244
  --username=xxx
3708
- --password=xxx
3709
- --private-key=@file:../path/to/key.pem
4245
+ --private-key=@file:.../path/to/key.pem
3710
4246
  ```
3711
4247
 
3712
4248
  #### Faspex 5 using web browser
3713
4249
 
3714
- For web method, create an API client in Faspex without JWT:
4250
+ For `web` method, create an API client in Faspex without JWT:
3715
4251
 
3716
4252
  * Navigate to the web UI: Admin &rarr; Configurations &rarr; API Clients &rarr; Create
3717
4253
  * Do not Activate JWT
3718
- * enter https://127.0.0.1:8888 in the redirect URI
4254
+ * enter `https://127.0.0.1:8888` in the redirect URI
3719
4255
  * Click on Create Button
3720
- * Take note of Client Id
4256
+ * Take note of Client Id (and Client Secret, but not used in current version)
3721
4257
 
3722
4258
  Then use options:
3723
4259
 
3724
4260
  ```text
3725
4261
  --auth=web
3726
4262
  --client-id=xxx
4263
+ --client-secret=xxx
3727
4264
  --redirect-uri=https://127.0.0.1:8888
3728
4265
  ```
3729
4266
 
3730
4267
  #### Faspex 5 using bootstrap
3731
4268
 
3732
- For boot method: (will be removed in future)
4269
+ For `boot` method: (will be removed in future)
3733
4270
 
3734
4271
  * open a browser
3735
4272
  * start developer mode
@@ -3742,6 +4279,44 @@ Use it as password and use `--auth=boot`.
3742
4279
  ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
3743
4280
  ```
3744
4281
 
4282
+ ### Faspex 5 sample commands
4283
+
4284
+ ```bash
4285
+ faspex5 admin res accounts list
4286
+ faspex5 admin res contacts list
4287
+ faspex5 admin res jobs list
4288
+ faspex5 admin res node list --value=@json:'{"type":"received","subtype":"mypackages"}'
4289
+ faspex5 admin res oauth_clients list
4290
+ faspex5 admin res registrations list
4291
+ faspex5 admin res saml_configs list
4292
+ faspex5 admin res shared_inboxes list
4293
+ faspex5 admin res workgroups list
4294
+ faspex5 health
4295
+ faspex5 package list --value=@json:'{"mailbox":"inbox","state":["released"]}'
4296
+ faspex5 package receive "my_package_id" --to-folder=. --ts=@json:'{"content_protection_password":"abc123_yo"}'
4297
+ faspex5 package send --value=@json:'{"title":"test title","recipients":[{"name":"my_f5_user"}]}' testfile.bin --ts=@json:'{"content_protection_password":"_content_prot_here_"}'
4298
+ ```
4299
+
4300
+ ### Faspex 5 other examples
4301
+
4302
+ * List all shared inboxes
4303
+
4304
+ ```javascript
4305
+ ascli faspex5 admin res shared list --value=@json:'{"all":true}' --fields=id,name
4306
+ ```
4307
+
4308
+ * Create Metadata profile
4309
+
4310
+ ```javascript
4311
+ ascli faspex5 admin res metadata_profiles create --value=@json:'{"name":"the profile","default":false,"title":{"max_length":200,"illegal_chars":[]},"note":{"max_length":400,"illegal_chars":[],"enabled":false},"fields":[{"ordering":0,"name":"field1","type":"text_area","require":true,"illegal_chars":[],"max_length":100},{"ordering":1,"name":"fff2","type":"option_list","require":false,"choices":["opt1","opt2"]}]}'
4312
+ ```
4313
+
4314
+ * Create a Shared inbox with specific metadata profile
4315
+
4316
+ ```javascript
4317
+ ascli faspex5 admin res shared create --value=@json:'{"name":"the shared inbox","metadata_profile_id":1}'
4318
+ ```
4319
+
3745
4320
  ## Plugin: IBM Aspera Faspex (4.x)
3746
4321
 
3747
4322
  Notes:
@@ -3812,7 +4387,8 @@ if `id` is set to `ALL`, then all packages are downloaded, and if option `once_o
3812
4387
 
3813
4388
  ### Sending a Package
3814
4389
 
3815
- The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`. (Refer to Faspex API).
4390
+ The command is `faspex package send`. Package information (title, note, metadata, options) is provided in option `delivery_info`.
4391
+ The contents of `delivery_info` is directly the contents of the `send` v3 [API of Faspex 4](https://developer.ibm.com/apis/catalog/aspera--aspera-faspex-client-sdk/API%20v.3:%20Send%20Packages), consult it for extended supported parameters.
3816
4392
 
3817
4393
  Example:
3818
4394
 
@@ -3888,10 +4464,90 @@ cargo client, or drive. Refer to the [same section](#aoccargo) in the Aspera on
3888
4464
  ascli faspex packages recv --id=ALL --once-only=yes --lock-port=12345
3889
4465
  ```
3890
4466
 
4467
+ ### Faspex 4 sample commands
4468
+
4469
+ ```bash
4470
+ faspex address_book
4471
+ faspex dropbox list --recipient="*my_faspex_dbx"
4472
+ faspex dropbox list --recipient="*my_faspex_wkg"
4473
+ faspex health
4474
+ faspex login_methods
4475
+ faspex me
4476
+ faspex package list
4477
+ faspex package list --box=sent --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
4478
+ faspex package list --fields=package_id --format=csv --display=data --query=@json:'{"max":1}'
4479
+ faspex package list --recipient="*my_faspex_dbx" --format=csv --fields=package_id --query=@json:'{"max":1}'
4480
+ faspex package list --recipient="*my_faspex_wkg" --format=csv --fields=package_id --query=@json:'{"max":1}'
4481
+ faspex package recv "my_package_id" --to-folder=.
4482
+ faspex package recv "my_package_id" --to-folder=. --box=sent
4483
+ faspex package recv --to-folder=. --link="my_faspex_publink_recv_from_fxuser"
4484
+ faspex package recv ALL --to-folder=. --once-only=yes
4485
+ faspex package recv my_pkgid --recipient="*my_faspex_dbx" --to-folder=.
4486
+ faspex package recv my_pkgid --recipient="*my_faspex_wkg" --to-folder=.
4487
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_dbx"]}' testfile.bin
4488
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["*my_faspex_wkg"]}' testfile.bin
4489
+ faspex package send --delivery-info=@json:'{"title":"Important files delivery","recipients":["my_email_internal_user","my_faspex_username"]}' testfile.bin
4490
+ faspex package send --link="my_faspex_publink_send_to_dropbox" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
4491
+ faspex package send --link="my_faspex_publink_send_to_fxuser" --delivery-info=@json:'{"title":"Important files delivery"}' testfile.bin
4492
+ faspex source list
4493
+ faspex source name "my_source_name" info
4494
+ faspex source name "my_source_name" node br /
4495
+ faspex v4 dmembership list
4496
+ faspex v4 dropbox list
4497
+ faspex v4 metadata_profile list
4498
+ faspex v4 user list
4499
+ faspex v4 wmembership list
4500
+ faspex v4 workgroup list
4501
+ ```
4502
+
3891
4503
  ## Plugin: IBM Aspera Shares
3892
4504
 
3893
4505
  Aspera Shares supports the "node API" for the file transfer part. (Shares 1 and 2)
3894
4506
 
4507
+ ### Shares 1 sample commands
4508
+
4509
+ ```bash
4510
+ shares admin share list
4511
+ shares admin share user_permissions 9
4512
+ shares admin user app_authorizations 1
4513
+ shares admin user list
4514
+ shares admin user share_permissions 1
4515
+ shares repository browse /
4516
+ shares repository delete my_shares_upload/testfile.bin
4517
+ shares repository download --to-folder=. my_shares_upload/testfile.bin
4518
+ shares repository download --to-folder=. my_shares_upload/testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
4519
+ shares repository upload --to-folder=my_shares_upload testfile.bin
4520
+ shares repository upload --to-folder=my_shares_upload testfile.bin --transfer=httpgw --transfer-info=@json:'{"url":"https://my_http_gw_fqdn/aspera/http-gwy/v1"}'
4521
+ ```
4522
+
4523
+ ## Plugin: Console
4524
+
4525
+ ### Console sample commands
4526
+
4527
+ ```bash
4528
+ console transfer current list
4529
+ console transfer smart list
4530
+ console transfer smart sub my_job_id @json:'{"source":{"paths":["my_file_name"]},"source_type":"user_selected"}'
4531
+ ```
4532
+
4533
+ ## Plugin: Orchestrator
4534
+
4535
+ ### Orchestrator sample commands
4536
+
4537
+ ```bash
4538
+ orchestrator info
4539
+ orchestrator plugins
4540
+ orchestrator processes
4541
+ orchestrator workflow details my_orch_workflow_id
4542
+ orchestrator workflow export my_orch_workflow_id
4543
+ orchestrator workflow inputs my_orch_workflow_id
4544
+ orchestrator workflow list
4545
+ orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}'
4546
+ orchestrator workflow start my_orch_workflow_id --params=@json:'{"Param":"world !"}' --result=ResultStep:Complete_status_message
4547
+ orchestrator workflow status ALL
4548
+ orchestrator workflow status my_orch_workflow_id
4549
+ ```
4550
+
3895
4551
  ## Plugin: IBM Cloud Object Storage
3896
4552
 
3897
4553
  The IBM Cloud Object Storage provides the possibility to execute transfers using FASP.
@@ -3905,7 +4561,7 @@ There are two possibilities to provide credentials. If you already have the endp
3905
4561
  If you have those parameters already, then following options shall be provided:
3906
4562
 
3907
4563
  * `bucket` bucket name
3908
- * `endpoint` storage endpoint url, e.g. https://s3.hkg02.cloud-object-storage.appdomain.cloud
4564
+ * `endpoint` storage endpoint url, e.g. `https://s3.hkg02.cloud-object-storage.appdomain.cloud`
3909
4565
  * `apikey` API Key
3910
4566
  * `crn` resource instance id
3911
4567
 
@@ -3985,20 +4641,37 @@ ascli cos node upload 'faux:///sample1G?1g'
3985
4641
 
3986
4642
  Note: we generate a dummy file `sample1G` of size 2GB using the `faux` PVCL (man ascp and section above), but you can of course send a real file by specifying a real file instead.
3987
4643
 
4644
+ ### COS sample commands
4645
+
4646
+ ```bash
4647
+ cos -N --bucket=my_icos_bucket_name --endpoint=my_icos_bucket_endpoint --apikey=my_icos_bucket_apikey --crn=my_icos_resource_instance_id node info
4648
+ cos -N --bucket=my_icos_bucket_name --region=my_icos_bucket_region --service-credentials=@json:@file:service_creds.json node info
4649
+ cos node access_key show self
4650
+ cos node download testfile.bin --to-folder=.
4651
+ cos node info
4652
+ cos node upload testfile.bin
4653
+ ```
4654
+
3988
4655
  ## Plugin: IBM Aspera Sync
3989
4656
 
3990
4657
  A basic plugin to start an "async" using `ascli`.
3991
4658
  The main advantage is the possibility to start from ma configuration file, using `ascli` standard options.
3992
4659
 
4660
+ ### Sync sample commands
4661
+
4662
+ ```bash
4663
+ sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"remote_dir":"/sync_test","local_dir":"contents","host":"my_remote_host","tcp_port":33001,"user":"my_remote_user","private_key_path":"my_local_user_key"}]}'
4664
+ ```
4665
+
3993
4666
  ## Plugin: Preview
3994
4667
 
3995
- The `preview` generates "previews" of graphical files, i.e. thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
3996
- This is based on the "node API" of Aspera HSTS when using Access Keys only inside it's "storage root".
4668
+ The `preview` generates thumbnails (office, images, video) and video previews on storage for use primarily in the Aspera on Cloud application.
4669
+ It uses the **node API** of Aspera HSTS and requires use of Access Keys and it's **storage root**.
3997
4670
  Several parameters can be used to tune several aspects:
3998
4671
 
3999
- * methods for detection of new files needing generation
4000
- * methods for generation of video preview
4001
- * parameters for video handling
4672
+ * Methods for detection of new files needing generation
4673
+ * Methods for generation of video preview
4674
+ * Parameters for video handling
4002
4675
 
4003
4676
  ### Aspera Server configuration
4004
4677
 
@@ -4237,13 +4910,48 @@ The mp4 video preview file is only for category `video`
4237
4910
 
4238
4911
  File type is primarily based on file extension detected by the node API and translated info a mime type returned by the node API.
4239
4912
 
4240
- The tool can also locally detect the mime type using option `mimemagic`.
4913
+ ### mimemagic
4914
+
4915
+ By default, the Mime type used for conversion is the one returned by the node API, based on file name extension.
4241
4916
 
4242
- To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`
4917
+ It is also possible to detect the mime type using option `mimemagic`.
4918
+ To use it, set option `mimemagic` to `yes`: `--mimemagic=yes`.
4243
4919
 
4244
- If not used, Mime type used for conversion is the one provided by the node API.
4920
+ This requires to manually install the mimemagic gem: `gem install mimemagic`.
4245
4921
 
4246
- If used, the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
4922
+ In this case the `preview` command will first analyze the file content using mimemagic, and if no match, will try by extension.
4923
+
4924
+ If the `mimemagic` gem complains about missing mime info file:
4925
+
4926
+ * any OS:
4927
+
4928
+ * Examine the error message
4929
+ * Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
4930
+ * move and rename this file to one of the locations expected by mimemagic as specified in the error message
4931
+
4932
+ * Windows:
4933
+
4934
+ * Download the file: <https://gitlab.freedesktop.org/xdg/shared-mime-info/-/raw/master/data/freedesktop.org.xml.in>
4935
+ * Place this file in the root of Ruby (or elsewhere): `C:\RubyVV-x64\freedesktop.org.xml.in`
4936
+ * Set a global variable using `SystemPropertiesAdvanced.exe` or using `cmd` (replace `VV` with version) to the exact path of this file:
4937
+
4938
+ ```cmd
4939
+ SETX FREEDESKTOP_MIME_TYPES_PATH C:\RubyVV-x64\freedesktop.org.xml.in
4940
+ ```
4941
+
4942
+ * Close the `cmd` and restart a new one if needed to get refreshed env vars
4943
+
4944
+ * Linux:
4945
+
4946
+ ```bash
4947
+ yum install shared-mime-info
4948
+ ```
4949
+
4950
+ * macOS:
4951
+
4952
+ ```bash
4953
+ brew install shared-mime-info
4954
+ ```
4247
4955
 
4248
4956
  ### Access to original files and preview creation
4249
4957
 
@@ -4256,6 +4964,23 @@ are directly written to the storage.
4256
4964
 
4257
4965
  If the preview generator does not have access to files on the file system (it is remote, no mount, or is an object storage), then the original file is first downloaded, then the result is uploaded, use method `remote`.
4258
4966
 
4967
+ ### Preview sample commands
4968
+
4969
+ ```bash
4970
+ preview check --skip-types=office
4971
+ preview folder 1 --skip-types=office --log-level=info --file-access=remote --ts=@json:'{"target_rate_kbps":1000000}'
4972
+ preview scan --skip-types=office --log-level=info
4973
+ preview test --case=test mp4 my_file_mxf --video-conversion=blend --log-level=debug
4974
+ preview test --case=test mp4 my_file_mxf --video-conversion=clips --log-level=debug
4975
+ preview test --case=test mp4 my_file_mxf --video-conversion=reencode --log-level=debug
4976
+ preview test --case=test png my_file_dcm --log-level=debug
4977
+ preview test --case=test png my_file_docx --log-level=debug
4978
+ preview test --case=test png my_file_mxf --video-png-conv=animated --log-level=debug
4979
+ preview test --case=test png my_file_mxf --video-png-conv=fixed --log-level=debug
4980
+ preview test --case=test png my_file_pdf --log-level=debug
4981
+ preview trevents --once-only=yes --skip-types=office --log-level=info
4982
+ ```
4983
+
4259
4984
  ## SMTP for email notifications
4260
4985
 
4261
4986
  Aspera CLI can send email, for that setup SMTP configuration. This is done with option `smtp`.
@@ -4273,7 +4998,7 @@ The `smtp` option is a hash table (extended value) with the following fields:
4273
4998
  <tr><td>`from_name`</td><td>same as email</td><td>John Wayne</td><td>display name of sender</td></tr>
4274
4999
  </table>
4275
5000
 
4276
- ### Example of configuration:
5001
+ ### Example of configuration
4277
5002
 
4278
5003
  ```bash
4279
5004
  ascli config preset set smtp_google server smtp.google.com
@@ -4350,8 +5075,7 @@ Transfer is: <%=global_transfer_status%>
4350
5075
 
4351
5076
  ## Tool: `asession`
4352
5077
 
4353
- This gem comes with a second executable tool providing a simplified standardized interface
4354
- to start a FASP session: `asession`.
5078
+ This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: `asession`.
4355
5079
 
4356
5080
  It aims at simplifying the startup of a FASP session from a programmatic stand point as formatting a [*transfer-spec*](#transferspec) is:
4357
5081
 
@@ -4365,7 +5089,7 @@ This makes it easy to integrate with any language provided that one can spawn a
4365
5089
 
4366
5090
  The tool expect one single argument: a [*transfer-spec*](#transferspec).
4367
5091
 
4368
- If not argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
5092
+ If no argument is provided, it assumes a value of: `@json:@stdin:`, i.e. a JSON formatted [*transfer-spec*](#transferspec) on stdin.
4369
5093
 
4370
5094
  Note that if JSON is the format, one has to specify `@json:` to tell the tool to decode the hash using JSON.
4371
5095
 
@@ -4382,18 +5106,24 @@ Note that in addition, many "EX_" [*transfer-spec*](#transferspec) parameters ar
4382
5106
 
4383
5107
  <table>
4384
5108
  <tr><th>feature/tool</th><th>asession</th><th>ascp</th><th>FaspManager</th><th>Transfer SDK</th></tr>
4385
- <tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>any</td></tr>
4386
- <tr><td>additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
5109
+ <tr><td>language integration</td><td>any</td><td>any</td><td>C/C++<br/>C#/.net<br/>Go<br/>Python<br/>java<br/></td><td>many</td></tr>
5110
+ <tr><td>required additional components to ascp</td><td>Ruby<br/>Aspera</td><td>-</td><td>library<br/>(headers)</td><td>daemon</td></tr>
4387
5111
  <tr><td>startup</td><td>JSON on stdin<br/>(standard APIs:<br/>JSON.generate<br/>Process.spawn)</td><td>command line arguments</td><td>API</td><td>daemon</td></tr>
4388
5112
  <tr><td>events</td><td>JSON on stdout</td><td>none by default<br/>or need to open management port<br/>and proprietary text syntax</td><td>callback</td><td>callback</td></tr>
4389
- <tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp</td><td>any with ascp</td><td>any with ascp and transferdaemon</td></tr></table>
5113
+ <tr><td>platforms</td><td>any with ruby and ascp</td><td>any with ascp (and SDK if compiled)</td><td>any with ascp</td><td>any with ascp and transfer daemon</td></tr></table>
4390
5114
 
4391
5115
  ### Simple session
4392
5116
 
4393
- ```javascript
4394
- MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}'
5117
+ Create a file `session.json` with:
5118
+
5119
+ ```json
5120
+ {"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"_pass_here_","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}],"resume_level":"none"}
5121
+ ````
4395
5122
 
4396
- echo "${MY_TSPEC}"|asession
5123
+ Then start the session:
5124
+
5125
+ ```
5126
+ asession < session.json
4397
5127
  ```
4398
5128
 
4399
5129
  ### Asynchronous commands and Persistent session
@@ -4487,7 +5217,7 @@ Note: parameters may be saved in a [option preset](#lprt) and used with `-P`.
4487
5217
 
4488
5218
  #### Scheduling
4489
5219
 
4490
- Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [_Scheduling_](#_scheduling_).
5220
+ Once `ascli` parameters are defined, run the command using the OS native scheduler, e.g. every minutes, or 5 minutes, etc... Refer to section [Scheduling](#scheduling).
4491
5221
 
4492
5222
  ### Example: upload folder
4493
5223
 
@@ -4570,6 +5300,10 @@ aoc.read('self')
4570
5300
 
4571
5301
  <https://github.com/IBM/aspera-cli/blob/main/examples/aoc.rb>
4572
5302
 
5303
+ ## Changes (Release notes)
5304
+
5305
+ See <CHANGELOG.md>
5306
+
4573
5307
  ## History
4574
5308
 
4575
5309
  When I joined Aspera, there was only one CLI: `ascp`, which is the implementation of the FASP protocol, but there was no CLI to access the various existing products (Server, Faspex, Shares). Once, Serban (founder) provided a shell script able to create a Faspex Package using Faspex REST API. Since all products relate to file transfers using FASP (ascp), I thought it would be interesting to have a unified CLI for transfers using FASP. Also, because there was already the `ascp` tool, I thought of an extended tool : `eascp.pl` which was accepting all `ascp` options for transfer but was also able to transfer to Faspex and Shares (destination was a kind of URI for the applications).
@@ -4586,472 +5320,15 @@ So, it evolved into `ascli`:
4586
5320
  * supports transfers with multiple [Transfer Agents](#agents), that&apos;s why transfer parameters moved from ascp command line to [*transfer-spec*](#transferspec) (more reliable , more standard)
4587
5321
  * `ruby` is consistent with other Aspera products
4588
5322
 
4589
- ## Changes (Release notes)
4590
-
4591
- * 4.7.0
4592
-
4593
- * new: option to specify font used to generate image of text file in `preview`
4594
- * new: #66 improvement for content protection (support standard transfer spec options for direct agent)
4595
- * new: option `fpac` is now applicable to all ruby based HTTP connections, i.e. API calls
4596
- * new: option `show_secrets` to reveal secrets in command output
4597
- * new: added and updated commands for Faspex 5
4598
- * new: option `cache_tokens`
4599
- * new: Faspex4 dropbox packages can now be received by id
4600
- * change: (break) command `conf gem path` replaces `conf gem_path`
4601
- * change: (break) option `fpac` expects a value instead of URL
4602
- * change: (break) option `cipher` in transfer spec must have hyphen
4603
- * change: (break) renamed option `log_passwords` to `log_secrets`
4604
- * change: (break) removed plugin `shares2` as products is now EOL
4605
- * fix: After AoC version update, wizard did not detect AoC properly
4606
-
4607
- * 4.6.0
4608
-
4609
- * new: command `conf plugin create`
4610
- * new: global option `plugin_folder`
4611
- * new: global option `transpose_single`
4612
- * new: simplified metadata passing for shared inbox package creation in AoC
4613
- * change: (break) command `aoc packages shared_inboxes list` replaces `aoc user shared_inboxes`
4614
- * change: (break) command `aoc user profile` replaces `aoc user info`
4615
- * change: (break) command `aoc user workspaces list` replaces `aoc user workspaces`
4616
- * change: (break) command `aoc user workspaces current` replaces `aoc workspace`
4617
- * change: (break) command `conf plugin list` replaces `conf plugins`
4618
- * change: (break) command `conf connect` simplified
4619
- * fix: #60 ascli executable was not installed by default in 4.5.0
4620
- * fix: add password hiding case in logs
4621
-
4622
- * 4.5.0
4623
-
4624
- * new: support transfer agent: [Transfer SDK](#agt_trsdk)
4625
- * new: support [http socket options](#http_options)
4626
- * new: logs hide passwords and secrets, option `log_passwords` to enable logging secrets
4627
- * new: `config vault` supports encrypted passwords, also macos keychain
4628
- * new: `config preset` command for consistency with id
4629
- * new: identifier can be provided using either option `id` or directly after the command, e.g. `delete 123` is the same as `delete --id=123`
4630
- * change: when using wss, use [ruby's CA certs](#certificates)
4631
- * change: unexpected parameter makes exit code not zero
4632
- * change: (break) options `id` and `name` cannot be specified at the same time anymore, use [positional identifer or name selection](#res_select)
4633
- * change: (break) `aoc admin res node` does not take workspace main node as default node if no `id` specified.
4634
- * change: (break): `orchestrator workflow status` requires id, and supports special id `ALL`
4635
- * fix: various smaller fixes and renaming of some internal classes (transfer agents and few other)
4636
-
4637
- * 4.4.0
4638
-
4639
- * new: `aoc packages list` add possibility to add filter with option `query`
4640
- * new: `aoc admin res xxx list` now get all items by default #50
4641
- * new: `preset` option can specify name or hash value
4642
- * new: `node` plugin accepts bearer token and access key as credential
4643
- * new: `node` option `token_type` allows using basic token in addition to aspera type.
4644
- * change: `server`: option `username` not mandatory anymore: xfer user is by default. If transfer spec token is provided, password or keys are optional, and bypass keys are used by default.
4645
- * change: (break) resource `apps_new` of `aoc` replaced with `application` (more clear)
4646
-
4647
- * 4.3.0
4648
-
4649
- * new: parameter `multi_incr_udp` for option `transfer_info`: control if UDP port is incremented when multi-session is used on [`direct`](#agt_direct) transfer agent.
4650
- * new: command `aoc files node_info` to get node information for a given folder in the Files application of AoC. Allows cross-org or cross-workspace transfers.
4651
-
4652
- * 4.2.2
4653
-
4654
- * new: `faspex package list` retrieves the whole list, not just first page
4655
- * new: support web based auth to aoc and faspex 5 using HTTPS, new dependency on gem `webrick`
4656
- * new: the error "Remote host is not who we expected" displays a special remediation message
4657
- * new: `conf ascp spec` displays supported transfer spec
4658
- * new: options `notif_to` and `notif_template` to send email notifications on transfer (and other events)
4659
- * fix: space character in `faspe:` url are precent encoded if needed
4660
- * fix: `preview scan`: if file_id is unknown, ignore and continue scan
4661
- * change: for commands that potentially execute several transfers (`package recv --id=ALL`), if one transfer fails then `ascli` exits with code 1 (instead of zero=success)
4662
- * change: (break) option `notify` or `aoc` replaced with `notif_to` and `notif_template`
4663
-
4664
- * 4.2.1
4665
-
4666
- * new: command `faspex package recv` supports link of type: `faspe:`
4667
- * new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
4668
-
4669
- * 4.2.0
4670
-
4671
- * new: command `aoc remind` to receive organization membership by email
4672
- * new: in `preview` option `value` to filter out on file name
4673
- * new: `initdemo` to initialize for demo server
4674
- * new: [`direct`](#agt_direct) transfer agent options: `spawn_timeout_sec` and `spawn_delay_sec`
4675
- * fix: on Windows `conf ascp use` expects ascp.exe
4676
- * fix: (break) multi_session_threshold is Integer, not String
4677
- * fix: `conf ascp install` renames sdk folder if it already exists (leftover shared lib may make fail)
4678
- * fix: removed replace_illegal_chars from default aspera.conf causing "Error creating illegal char conversion table"
4679
- * change: (break) `aoc apiinfo` is removed, use `aoc servers` to provide the list of cloud systems
4680
- * change: (break) parameters for resume in `transfer-info` for [`direct`](#agt_direct) are now in sub-key `"resume"`
4681
-
4682
- * 4.1.0
4683
-
4684
- * fix: remove keys from transfer spec and command line when not needed * fix: default to create_dir:true so that sending single file to a folder does not rename file if folder does not exist
4685
- * new: update documentation with regard to offline and docker installation
4686
- * new: renamed command `nagios_check` to `health`
4687
- * new: agent `http_gw` now supports upload
4688
- * new: added option `sdk_url` to install SDK from local file for offline install
4689
- * new: check new gem version periodically
4690
- * new: the --fields= option, support -_fieldname_ to remove a field from default fields
4691
- * new: Oauth tokens are discarded automatically after 30 minutes (useful for COS delegated refresh tokens)
4692
- * new: mimemagic is now optional, needs manual install for `preview`, compatible with version 0.4.x
4693
- * new: AoC a password can be provided for a public link
4694
- * new: `conf doc` take an optional parameter to go to a section
4695
- * new: initial support for Faspex 5 Beta 1
4696
-
4697
- * 4.0.0
4698
-
4699
- * now available as open source at [https://github.com/IBM/aspera-cli](https://github.com/IBM/aspera-cli) with general cleanup
4700
- * changed default tool name from `mlia` to `ascli`
4701
- * changed `aspera` command to `aoc`
4702
- * changed gem name from `asperalm` to `aspera-cli`
4703
- * changed module name from `Asperalm` to `Aspera`
4704
- * removed command `folder` in `preview`, merged to `scan`
4705
- * persistency files go to sub folder instead of main folder
4706
- * added possibility to install SDK: `config ascp install`
4707
-
4708
- * 0.11.8
4709
-
4710
- * Simplified to use `unoconv` instead of bare `libreoffice` for office conversion, as `unoconv` does not require a X server (previously using Xvfb
4711
-
4712
- * 0.11.7
4713
-
4714
- * rework on rest call error handling
4715
- * use option `display` with value `data` to remove out of extraneous information
4716
- * fixed option `lock_port` not working
4717
- * generate special icon if preview failed
4718
- * possibility to choose transfer progress bar type with option `progress`
4719
- * AoC package creation now output package id
4720
-
4721
- * 0.11.6
4722
-
4723
- * orchestrator : added more choice in auth type
4724
- * preview: cleanup in generator (removed and renamed parameters)
4725
- * preview: better documentation
4726
- * preview: animated thumbnails for video (option: `video_png_conv=animated`)
4727
- * preview: new event trigger: `trevents` (`events` seems broken)
4728
- * preview: unique tmp folder to avoid clash of multiple instances
4729
- * repo: added template for secrets used for testing
4730
-
4731
- * 0.11.5
4732
-
4733
- * added option `default_ports` for AoC (see manual)
4734
- * allow bulk delete in `aspera files` with option `bulk=yes`
4735
- * fix getting connect versions
4736
- * added section for Aix
4737
- * support all ciphers for [`direct`](#agt_direct) agent (including gcm, etc..)
4738
- * added transfer spec param `apply_local_docroot` for [`direct`](#agt_direct)
4739
-
4740
- * 0.11.4
4741
-
4742
- * possibility to give shared inbox name when sending a package (else use id and type)
4743
-
4744
- * 0.11.3
4745
-
4746
- * minor fixes on multi-session: avoid exception on progress bar
4747
-
4748
- * 0.11.2
4749
-
4750
- * fixes on multi-session: progress bat and transfer spec param for "direct"
4751
-
4752
- * 0.11.1
4753
-
4754
- * enhanced short_link creation commands (see examples)
4755
-
4756
- * 0.11
4757
-
4758
- * add transfer spec option (agent `direct` only) to provide file list directly to ascp: `EX_file_list`.
4759
-
4760
- * 0.10.18
4761
-
4762
- * new option in. `server` : `ssh_options`
4763
-
4764
- * 0.10.17
4765
-
4766
- * fixed problem on `server` for option `ssh_keys`, now accepts both single value and list.
4767
- * new modifier: `@list:<separator>val1<separator>...`
4768
-
4769
- * 0.10.16
4770
-
4771
- * added list of shared inboxes in workspace (or global), use `--query=@json:'{}'`
4772
-
4773
- * 0.10.15
4774
-
4775
- * in case of command line error, display the error cause first, and non-parsed argument second
4776
- * AoC : Activity / Analytics
4777
-
4778
- * 0.10.14
4779
-
4780
- * added missing bss plugin
4781
-
4782
- * 0.10.13
4783
-
4784
- * added Faspex5 (use option `value` to give API arguments)
4785
-
4786
- * 0.10.12
4787
-
4788
- * added support for AoC node registration keys
4789
- * replaced option : `local_resume` with `transfer_info` for agent [`direct`](#agt_direct)
4790
- * Transfer agent is no more a Singleton instance, but only one is used in CLI
4791
- * `@incps` : new extended value modifier
4792
- * ATS: no more provides access keys secrets: now user must provide it
4793
- * begin work on "aoc" transfer agent
4794
-
4795
- * 0.10.11
4796
-
4797
- * minor refactor and fixes
4798
-
4799
- * 0.10.10
4800
-
4801
- * fix on documentation
4802
-
4803
- * 0.10.9.1
4804
-
4805
- * add total number of items for AoC resource list
4806
- * better gem version dependency (and fixes to support Ruby 2.0.0)
4807
- * removed aoc search_nodes
4808
-
4809
- * 0.10.8
4810
-
4811
- * removed option: `fasp_proxy`, use pseudo transfer spec parameter: `EX_fasp_proxy_url`
4812
- * removed option: `http_proxy`, use pseudo transfer spec parameter: `EX_http_proxy_url`
4813
- * several other changes..
4814
-
4815
- * 0.10.7
4816
-
4817
- * fix: ascli fails when username cannot be computed on Linux.
4818
-
4819
- * 0.10.6
4820
-
4821
- * FaspManager: transfer spec `authentication` no more needed for local transfer to use Aspera public keys. public keys will be used if there is a token and no key or password is provided.
4822
- * gem version requirements made more open
4823
-
4824
- * 0.10.5
4825
-
4826
- * fix faspex package receive command not working
4827
-
4828
- * 0.10.4
4829
-
4830
- * new options for AoC : `secrets`
4831
- * ACLI-533 temp file list folder to use file lists is set by default, and used by asession
4832
-
4833
- * 0.10.3
4834
-
4835
- * included user name in oauth bearer token cache for AoC when JWT is used.
4836
-
4837
- * 0.10.2
4838
-
4839
- * updated `search_nodes` to be more generic, so it can search not only on access key, but also other queries.
4840
- * added doc for "cargo" like actions
4841
- * added doc for multi-session
4842
-
4843
- * 0.10.1
4844
-
4845
- * AoC and node v4 "browse" works now on non-folder items: file, link
4846
- * initial support for AoC automation (do not use yet)
4847
-
4848
- * 0.10
4849
-
4850
- * support for transfer using IBM Cloud Object Storage
4851
- * improved `find` action using arbitrary expressions
4852
-
4853
- * 0.9.36
4854
-
4855
- * added option to specify file pair lists
4856
-
4857
- * 0.9.35
4858
-
4859
- * updated plugin `preview` , changed parameter names, added documentation
4860
- * fix in `ats` plugin : instance id needed in request header
4861
-
4862
- * 0.9.34
4863
-
4864
- * parser "@preset" can be used again in option "transfer_info"
4865
- * some documentation re-organizing
4866
-
4867
- * 0.9.33
4868
-
4869
- * new command to display basic token of node
4870
- * new command to display bearer token of node in AoC
4871
- * the --fields= option, support +_fieldname_ to add a field to default fields
4872
- * many small changes
4873
-
4874
- * 0.9.32
4875
-
4876
- * all Faspex public links are now supported
4877
- * removed faspex operation recv_publink
4878
- * replaced with option `link` (consistent with AoC)
4879
-
4880
- * 0.9.31
4881
-
4882
- * added more support for public link: receive and send package, to user or dropbox and files view.
4883
- * delete expired file lists
4884
- * changed text table gem from text-table to terminal-table because it supports multiline values
4885
-
4886
- * 0.9.27
4887
-
4888
- * basic email support with SMTP
4889
- * basic proxy auto config support
4890
-
4891
- * 0.9.26
4892
-
4893
- * table display with --fields=ALL now includes all column names from all lines, not only first one
4894
- * unprocessed argument shows error even if there is an error beforehand
4895
-
4896
- * 0.9.25
4897
-
4898
- * the option `value` of command `find`, to filter on name, is not optional
4899
- * `find` now also reports all types (file, folder, link)
4900
- * `find` now is able to report all fields (type, size, etc...)
4901
-
4902
- * 0.9.24
4903
-
4904
- * fix bug where AoC node to node transfer did not work
4905
- * fix bug on error if ED25519 private key is defined in .ssh
4906
-
4907
- * 0.9.23
4908
-
4909
- * defined REST error handlers, more error conditions detected
4910
- * commands to select specific ascp location
4911
-
4912
- * 0.9.21
4913
-
4914
- * supports simplified wizard using global client
4915
- * only ascp binary is required, other SDK (keys) files are now generated
4916
-
4917
- * 0.9.20
4918
-
4919
- * improved wizard (prepare for AoC global client id)
4920
- * preview generator: addedoption : --skip-format=&lt;png,mp4&gt;
4921
- * removed outdated pictures from this doc
4922
-
4923
- * 0.9.19
4924
-
4925
- * added command aspera bearer --scope=xx
4926
-
4927
- * 0.9.18
4928
-
4929
- * enhanced aspera admin events to support query
4930
-
4931
- * 0.9.16
4932
-
4933
- * AoC transfers are now reported in activity app
4934
- * new interface for Rest class authentication (keep backward compatibility)
4935
-
4936
- * 0.9.15
4937
-
4938
- * new feature: "find" command in aspera files
4939
- * sample code for transfer API
4940
-
4941
- * 0.9.12
4942
-
4943
- * add nagios commands
4944
- * support of ATS for IBM Cloud, removed old version based on aspera id
4945
-
4946
- * 0.9.11
4947
-
4948
- * Breaking change: @stdin is now @stdin:
4949
- * support of ATS for IBM Cloud, removed old version based on aspera id
4950
-
4951
-
4952
- * 0.9.10
4953
-
4954
- * Breaking change: parameter transfer-node becomes more generic: transfer-info
4955
- * Display SaaS storage usage with command: aspera admin res node --id=nn info
4956
- * cleaner way of specifying source file list for transfers
4957
- * Breaking change: replaced download_mode option with http_download action
4958
-
4959
- * 0.9.9
4960
-
4961
- * Breaking change: "aspera package send" parameter deprecated, use the --value option instead with "recipients" value. See example.
4962
- * Now supports "cargo" for Aspera on Cloud (automatic package download)
4963
-
4964
- * 0.9.8
4965
-
4966
- * Faspex: use option once_only set to yes to enable cargo like function. id=NEW deprecated.
4967
- * AoC: share to share transfer with command "transfer"
4968
-
4969
- * 0.9.7
4970
-
4971
- * homogeneous [*transfer-spec*](#transferspec) for `node` and [`direct`](#agt_direct) transfer agents
4972
- * preview persistency goes to unique file by default
4973
- * catch mxf extension in preview as video
4974
- * Faspex: possibility to download all packages by specifying id=ALL
4975
- * Faspex: to come: cargo-like function to download only new packages with id=NEW
4976
-
4977
- * 0.9.6
4978
-
4979
- * Breaking change: `@param:`is now `@preset:` and is generic
4980
- * AoC: added command to display current workspace information
4981
-
4982
- * 0.9.5
4983
-
4984
- * new parameter: new_user_option used to choose between public_link and invite of external users.
4985
- * fixed bug in wizard, and wizard uses now product detection
4986
-
4987
- * 0.9.4
4988
-
4989
- * Breaking change: onCloud file list follow --source convention as well (plus specific case for download when first path is source folder, and other are source file names).
4990
- * AoC Package send supports external users
4991
- * new command to export AoC config to Aspera CLI config
4992
-
4993
- * 0.9.3
4994
-
4995
- * REST error message show host and code
4996
- * option for quiet display
4997
- * modified transfer interface and allow token re-generation on error
4998
- * async add admin command
4999
- * async add db parameters
5000
- * Breaking change: new option "sources" to specify files to transfer
5001
-
5002
- * 0.9.2
5003
-
5004
- * Breaking change: changed AoC package creation to match API, see AoC section
5005
-
5006
- * 0.9.1
5007
-
5008
- * Breaking change: changed faspex package creation to match API, see Faspex section
5009
-
5010
- * 0.9
5011
-
5012
- * Renamed the CLI from aslmcli to `ascli`
5013
- * Automatic rename and conversion of former config folder from aslmcli to `ascli`
5014
-
5015
- * 0.7.6
5016
-
5017
- * add "sync" plugin
5018
-
5019
- * 0.7
5020
-
5021
- * Breaking change: AoC package recv take option if for package instead of argument.
5022
- * Breaking change: Rest class and Oauth class changed init parameters
5023
- * AoC: receive package from public link
5024
- * select by col value on output
5025
- * added rename (AoC, node)
5026
-
5027
- * 0.6.19
5028
-
5029
- * change: (break) ats server list provisioned &rarr; ats cluster list
5030
- * change: (break) ats server list clouds &rarr; ats cluster clouds
5031
- * change: (break) ats server list instance --cloud=x --region=y &rarr; ats cluster show --cloud=x --region=y
5032
- * change: (break) ats server id xxx &rarr; ats cluster show --id=xxx
5033
- * change: (break) ats subscriptions &rarr; ats credential subscriptions
5034
- * change: (break) ats api_key repository list &rarr; ats credential cache list
5035
- * change: (break) ats api_key list &rarr; ats credential list
5036
- * change: (break) ats access_key id xxx &rarr; ats access_key --id=xxx
5037
-
5038
- * 0.6.18
5039
-
5040
- * some commands take now --id option instead of id command.
5041
-
5042
- * 0.6.15
5043
-
5044
- * Breaking change: "files" application renamed to "aspera" (for "Aspera on Cloud"). "repository" renamed to "files". Default is automatically reset, e.g. in config files and change key "files" to "aspera" in [option preset](#lprt) "default".
5045
-
5046
5323
  ## Common problems
5047
5324
 
5048
- ### Error "Remote host is not who we expected"
5325
+ ### Error: "Remote host is not who we expected"
5049
5326
 
5050
5327
  Cause: `ascp` >= 4.x checks fingerprint of highest server host key, including ECDSA. `ascp` < 4.0 (3.9.6 and earlier) support only to RSA level (and ignore ECDSA presented by server). `aspera.conf` supports a single fingerprint.
5051
5328
 
5052
5329
  Workaround on client side: To ignore the certificate (SSH fingerprint) add option on client side (this option can also be added permanently to the config file):
5053
5330
 
5054
- ```json
5331
+ ```javascript
5055
5332
  --ts=@json:'{"sshfp":null}'
5056
5333
  ```
5057
5334
 
@@ -5059,6 +5336,13 @@ Workaround on server side: Either remove the fingerprint from `aspera.conf`, or
5059
5336
 
5060
5337
  References: ES-1944 in release notes of 4.1 and to [HSTS admin manual section "Configuring Transfer Server Authentication With a Host-Key Fingerprint"](https://www.ibm.com/docs/en/ahts/4.2?topic=upgrades-configuring-ssh-server).
5061
5338
 
5339
+ ### Error "can't find header files for ruby"
5340
+
5341
+ Some Ruby gems dependencies require compilation of native parts (C).
5342
+ This also requires Ruby header files.
5343
+ If Ruby was installed as a Linux Packages, then also install ruby dev elopment package:
5344
+ `ruby-dev` ir `ruby-devel`, depending on distribution.
5345
+
5062
5346
  ### ED255519 key not supported
5063
5347
 
5064
5348
  ED25519 keys are deactivated since version 0.9.24 so this type of key will just be ignored.
@@ -5071,19 +5355,3 @@ OpenSSH keys only supported if ED25519 is available
5071
5355
 
5072
5356
  Which meant that you do not have ruby support for ED25519 SSH keys.
5073
5357
  You may either install the suggested Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
5074
-
5075
- ## BUGS, FEATURES, CONTRIBUTION
5076
-
5077
- For issues reports or feature requests use the Github repository and issues.
5078
-
5079
- You can also contribute to this open source project.
5080
-
5081
- One can also [create one's own plugin](#createownplugin).
5082
-
5083
- ## Miscellaneous
5084
-
5085
- * replace rest and oauth classes with ruby standard gems:
5086
- * <https://github.com/rest-client/rest-client>
5087
- * <https://github.com/oauth-xx/oauth2>
5088
- * use gem Thor <http://whatisthor.com/> (or other standard Ruby CLI manager)
5089
- * Package with <https://github.com/pmq20/ruby-packer> (rubyc)