aspera-cli 4.2.0 → 4.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1cc7508185a5c551f30a30e54a0cfa2d0ac7931a1bbb565bc7b15e98cca31479
-  data.tar.gz: ca9aae212a8411ddcfa4d1db4aa842fb273bf56fe247490b31037dcb0d628c94
+  metadata.gz: 89eb5bdf4ecda69281c147ad7e2da3027e98a55316f51774c3504c680c4411e4
+  data.tar.gz: 5d8f34bf429f575495890b656df0598f94a4ea67aa8d373d756ea7449c729562
 SHA512:
-  metadata.gz: 929bcc3dee9158eb60c810796ed8dc88cfd52d187abb47cb6733adb09971e67849ce1b0bf5cff7daf3dc1f7af327490b574ab8351c94435c6ac83f0674423cd8
-  data.tar.gz: 5c3cba44333dfca675a3cc3b0b77bb6d16654f322b580c936c911551dc85bcc55b16b01649faac2a5ae1f0805b3a2a83315c648ea6c1fb7987c569dc431887b4
+  metadata.gz: b5a4c5f67e5f9685a1e11e7c1dc8fce6474157471ca5d5d7ad3583b4318a268a04f2fc3b3dcd84c0866f5ccb41a73596e28921d607311347c9537ec1cacec194
+  data.tar.gz: 40901b7baa26597164855e3f66b347666e37737315c075027a8dad81054fde424dd78c3e3432cd40093d7bb06af060981f6643b91a766e08ffe9c86b9d673f93

data/README.md

@@ -1,7 +1,7 @@
 [comment1]: # (Do not edit this README.md, edit docs/README.erb.md, for details, read docs/README.md)
 # `ascli` : Command Line Interface for IBM Aspera products
 
-Version : 4.2.0
+Version : 4.2.1
 
 _Laurent/2016-2021_
 
@@ -81,7 +81,7 @@ Once the gem is installed, `ascli` shall be accessible:
 
 ```
 $ ascli --version
-4.2.0
+4.2.1
 ```
 
 ## First use
@@ -1734,7 +1734,7 @@ ascli sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"r
 ```
 $ ascli -h
 NAME
-	ascli -- a command line tool for Aspera Applications (v4.2.0)
+	ascli -- a command line tool for Aspera Applications (v4.2.1)
 
 SYNOPSIS
 	ascli COMMANDS [OPTIONS] [ARGS]
@@ -1879,7 +1879,7 @@ OPTIONS:
 
 
 COMMAND: faspex5
-SUBCOMMANDS: node package auth_client
+SUBCOMMANDS: node package auth_client jobs
 OPTIONS:
         --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
         --username=VALUE             username to log in
@@ -1912,6 +1912,7 @@ OPTIONS:
         --delivery-info=VALUE        package delivery information (extended value)
         --source-name=VALUE          create package from remote source (by name)
         --storage=VALUE              Faspex local storage definition
+        --recipient=VALUE            use if recipient is a dropbox (with *)
         --box=ENUM                   package box: inbox, sent, archive
 
 
@@ -2903,16 +2904,15 @@ to download files.
 $ ascli node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
 ```
 
-# Plugin: IBM Aspera Faspex
+# Plugin: IBM Aspera Faspex5
 
-Notes:
-
-* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
-* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+3 authentication methods are supported:
 
-## Faspex 5 Beta1
+* boot
+* web
+* jwt
 
-As the web UI does not yet allow adding API client yet, the way to use CLI is:
+For boot method:
 
 * open a browser
 * start developer mode
@@ -2925,30 +2925,42 @@ Use it as password and use `--auth=boot`.
 $ ascli conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
 ```
 
-An JWT client can then be created with a private key:
+For web method, create an API client in Faspex, and use: --auth=web
 
-```
-$ jsonk=$(openssl rsa -in  ~/.aspera/ascli/aspera_on_cloud_key -pubout 2> /dev/null | sed -e :a -e N -e '$!ba' -e 's/\n/\\n/g')
-$ ascli faspex5 -Pf5boot auth_client create --value=@json:'{"name":"hello","client_type":"public","redirect_uris":["https://localhost:12345"],"allow_jwt_grant":true,"public_key":"'$jsonk'"}'
-```
+For JWT, create an API client in Faspex with jwt supporot, and use: --auth=jwt
+as of beta£3 this does not allow regular users.
 
-or deleted by name:
+Ready to use Faspex5 with CLI.
+
+Once the graphical registration form exist, ther bootstrap method can be removed.
+
+# Plugin: IBM Aspera Faspex (4.x)
+
+Notes:
+
+* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+
+## Receiving a Package
+
+The command is `package recv`, possible methosd are:
+
+* provide a package id with option `id`
+* provide a public link with option `link`
+* provide a `faspe:` URI with option `link`
 
 ```
-$ id=$(ascli faspex5 auth_client list --select=@json:'{"name":"hello"}' --fields=client_id --format=csv)
-$ ascli faspex5 auth_client delete --id=$id
+$ ascli faspex package recv --id=12345
+$ ascli faspex package recv --link=faspe://...
 ```
 
-Once the API client is created with a client_id and secret (result of create command), create a configuration:
+If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
 
 ```
-$ ascli conf id f5 update --url=https://localhost/aspera/faspex --auth=jwt --client-id=abcd --client-secret=def --username=pierre@example.com --private-key=@val:@file:~/.aspera/ascli/aspera_on_cloud_key
-$ ascli conf id default set faspex5 f5
-``` 
+$ ascli faspex package list --recipient='*thedropboxname'
+```
 
-Ready to use Faspex5 with CLI.
 
-Once the graphical registration form exist, ther bootstrap method can be removed.
 
 ## Sending a Package
 
@@ -3656,6 +3668,11 @@ So, it evolved into `ascli`:
 
 # Changes (Release notes)
 
+* 4.2.1
+
+	* new: command `faspex package recv` supports link of type: `faspe:`
+	* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
+
 * 4.2.0
 
 	* new: command `aoc remind` to receive organization membership by email

data/docs/README.erb.md

@@ -2335,16 +2335,15 @@ to download files.
 $ <%=cmd%> node access_key create --value=@json:'{"id":"eudemo-sedemo","secret":"mystrongsecret","storage":{"type":"local","path":"/data/asperafiles"}}'
 ```
 
-# Plugin: IBM Aspera Faspex
+# Plugin: IBM Aspera Faspex5
 
-Notes:
-
-* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
-* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+3 authentication methods are supported:
 
-## Faspex 5 Beta1
+* boot
+* web
+* jwt
 
-As the web UI does not yet allow adding API client yet, the way to use CLI is:
+For boot method:
 
 * open a browser
 * start developer mode
@@ -2357,30 +2356,42 @@ Use it as password and use `--auth=boot`.
 $ <%=cmd%> conf id f5boot update --url=https://localhost/aspera/faspex --auth=boot --password=ABC.DEF.GHI...
 ```
 
-An JWT client can then be created with a private key:
+For web method, create an API client in Faspex, and use: --auth=web
 
-```
-$ jsonk=$(openssl rsa -in  ~/.aspera/ascli/aspera_on_cloud_key -pubout 2> /dev/null | sed -e :a -e N -e '$!ba' -e 's/\n/\\n/g')
-$ <%=cmd%> faspex5 -Pf5boot auth_client create --value=@json:'{"name":"hello","client_type":"public","redirect_uris":["https://localhost:12345"],"allow_jwt_grant":true,"public_key":"'$jsonk'"}'
-```
+For JWT, create an API client in Faspex with jwt supporot, and use: --auth=jwt
+as of beta£3 this does not allow regular users.
 
-or deleted by name:
+Ready to use Faspex5 with CLI.
+
+Once the graphical registration form exist, ther bootstrap method can be removed.
+
+# Plugin: IBM Aspera Faspex (4.x)
+
+Notes:
+
+* the command "v4" requires the use of APIv4, refer to the Faspex Admin manual on how to activate.
+* for full details on Faspex API, refer to: [Reference on Developer Site](https://www.ibm.com/products/aspera/developer)
+
+## Receiving a Package
+
+The command is `package recv`, possible methosd are:
+
+* provide a package id with option `id`
+* provide a public link with option `link`
+* provide a `faspe:` URI with option `link`
 
 ```
-$ id=$(<%=cmd%> faspex5 auth_client list --select=@json:'{"name":"hello"}' --fields=client_id --format=csv)
-$ <%=cmd%> faspex5 auth_client delete --id=$id
+$ <%=cmd%> faspex package recv --id=12345
+$ <%=cmd%> faspex package recv --link=faspe://...
 ```
 
-Once the API client is created with a client_id and secret (result of create command), create a configuration:
+If the package is in a specific dropbox, add option `recipient` for both the `list` and `recv` commands.
 
 ```
-$ <%=cmd%> conf id f5 update --url=https://localhost/aspera/faspex --auth=jwt --client-id=abcd --client-secret=def --username=pierre@example.com --private-key=@val:@file:~/.aspera/ascli/aspera_on_cloud_key
-$ <%=cmd%> conf id default set faspex5 f5
-``` 
+$ <%=cmd%> faspex package list --recipient='*thedropboxname'
+```
 
-Ready to use Faspex5 with CLI.
 
-Once the graphical registration form exist, ther bootstrap method can be removed.
 
 ## Sending a Package
 
@@ -3070,6 +3081,11 @@ So, it evolved into <%=tool%>:
 
 * <%= gemspec.version.to_s %>
 
+	* new: command `faspex package recv` supports link of type: `faspe:`
+	* new: command `faspex package recv` supports option `recipient` to specify dropbox with leading `*`
+
+* 4.2.0
+
 	* new: command `aoc remind` to receive organization membership by email
 	* new: in `preview` option `value` to filter out on file name
 	* new: `initdemo` to initialize for demo server

data/docs/test_env.conf

@@ -5,7 +5,7 @@ default:
   config: cli_default
   aoc: tst_aoc1
   faspex: tst_faspex
-  faspex5: tst_faspex5_boot
+  faspex5: tst_faspex5_web
   shares: tst_shares_1
   shares2: tst_shares2
   node: tst_node
@@ -52,6 +52,12 @@ tst_faspex5_boot:
   auth: your value here
   username: your value here
   password: your value here
+tst_faspex5_web:
+  url: your value here
+  auth: your value here
+  redirect_uri: your value here
+  client_id: your value here
+  client_secret: your value here
 tst_faspex5:
   url: your value here
   auth: your value here

data/examples/aoc.rb

@@ -4,11 +4,22 @@ require 'aspera/log'
 
 Aspera::Log.instance.level=:debug
 
+if ! ARGV.length.eql?(3)
+  Aspera::Log.log.error("wrong number of args: #{ARGV.length}")
+  Aspera::Log.log.error("Usage: #{$0} <aoc URL> <aoc username> <aoc private key content>")
+  Aspera::Log.log.error("Example: #{$0} https://myorg.ibmaspera.com john@example.com $(cat /home/john/my_key.pem)")
+  Process.exit(1)
+end
+
+aoc_url=ARGV[0]
+aoc_user=ARGV[1]
+aoc_key_value=ARGV[2]
+
 aocapi=Aspera::AoC.new(
-url: 'https://myorg.ibmaspera.com',
+url: aoc_url,
 auth: :jwt,
-private_key: File.read('path/to_your_private_key.pem'),
-username: 'my.email@example.com',
+private_key: aoc_key_value,
+username: aoc_user,
 scope: 'user:all',
 subpath: 'api/v1')
 

data/examples/faspex4.rb

@@ -0,0 +1,78 @@
+#!/usr/bin/env ruby
+require 'aspera/rest'
+require 'aspera/log'
+require 'aspera/fasp/local'
+require 'aspera/cli/listener/line_dump'
+require 'aspera/cli/extended_value'
+
+# set high log level for the example
+Aspera::Log.instance.level=:debug
+
+# set folder where SDK is installed
+# (if ascp is not there, the lib will try to find in usual locations)
+Aspera::Fasp::Installation.instance.folder='.'
+
+if ! ARGV.length.eql?(3)
+  Aspera::Log.log.error("wrong number of args: #{ARGV.length}")
+  Aspera::Log.log.error("Usage: #{$0} <faspex URL> <faspex username> <faspex password>")
+  Aspera::Log.log.error("Example: #{$0} https://faspex.com/aspera/faspex john p@sSw0rd")
+  Process.exit(1)
+end
+
+faspex_url=ARGV[0]
+faspex_user=ARGV[1]
+faspex_pass=ARGV[2]
+
+# 1: demo API v3
+#---------------
+
+# create REST API object
+api_v3=Aspera::Rest.new({
+  :base_url => faspex_url,
+  :auth     => {
+  :type     => :basic,
+  :username => faspex_user,
+  :password => faspex_pass
+  }})
+
+api_v3.read('me')
+
+# 2: send a package
+#---------------
+
+# create a sample file to send
+file_to_send='./myfile.bin'
+File.open(file_to_send, "w") {|f| f.write("sample data") }
+# package creation parameters
+package_create_params={'delivery'=>{'title'=>'test package','recipients'=>['aspera.user1@gmail.com'],'sources'=>[{'paths'=>[file_to_send]}]}}
+pkg_created=api_v3.create('send',package_create_params)[:data]
+# get transfer specification
+transfer_spec=pkg_created['xfer_sessions'].first
+# set paths of files to send
+transfer_spec['paths']=['source'=>file_to_send]
+# get the local agent (i.e. ascp)
+client=Aspera::Fasp::Local.new
+# disable ascp output on stdout to not mix with JSON events
+client.quiet=true
+# start transfer (asynchronous)
+job_id=client.start_transfer(transfer_spec)
+result=client.wait_for_transfers_completion
+#  notify of any transfer error
+result.select{|i|!i.eql?(:success)}.each do |e|
+  Aspera::Log.log.error("A transfer error occured: #{e.message}")
+end
+
+# 1: demo API v4
+#---------------
+api_v4=Aspera::Rest.new({
+  :base_url  => faspex_url+'/api',
+  :auth      => {
+  :type      => :oauth2,
+  :base_url  => faspex_url+'/auth/oauth2',
+  :grant     => :header_userpass,
+  :user_name => faspex_user,
+  :user_pass => faspex_pass,
+  :scope     => 'admin'
+  }})
+
+Aspera::Log.dump('users',api_v4.read('users')[:data])

data/lib/aspera/cli/main.rb

@@ -256,7 +256,7 @@ module Aspera
             @plugin_env[:formater].display_results({:type=>:single_object,:data=>@opt_mgr.declared_options(false)})
             Process.exit(0)
           end
-          # locking for singkle execution (only after "per plugin" option, in case lock port is there)
+          # locking for single execution (only after "per plugin" option, in case lock port is there)
           lock_port=@opt_mgr.get_option(:lock_port,:optional)
           if !lock_port.nil?
             begin

data/lib/aspera/cli/plugins/faspex.rb

@@ -26,6 +26,7 @@ module Aspera
           self.options.add_opt_simple(:delivery_info,'package delivery information (extended value)')
           self.options.add_opt_simple(:source_name,'create package from remote source (by name)')
           self.options.add_opt_simple(:storage,'Faspex local storage definition')
+          self.options.add_opt_simple(:recipient,'use if recipient is a dropbox (with *)')
           self.options.add_opt_list(:box,[:inbox,:sent,:archive],'package box')
           self.options.set_option(:box,:inbox)
           self.options.parse_options!
@@ -107,7 +108,7 @@ module Aspera
         PACKAGE_MATCH_FIELD='package_id'
 
         def mailbox_all_entries
-          my_user_name=self.options.get_option(:username,:mandatory)
+          recipient_name=self.options.get_option(:recipient,:optional) || self.options.get_option(:username,:mandatory)
           mailbox=self.options.get_option(:box,:mandatory)
           all_inbox_xml=api_v3.call({:operation=>'GET',:subpath=>"#{mailbox}.atom",:headers=>{'Accept'=>'application/xml'}})[:http].body
           all_inbox_data=XmlSimple.xml_in(all_inbox_xml, {'ForceArray' => true})
@@ -116,7 +117,7 @@ module Aspera
           result.each do |e|
             case mailbox
             when :inbox,:archive
-              recipient=e['to'].select{|i|i['name'].first.eql?(my_user_name)}.first
+              recipient=e['to'].select{|i|i['name'].first.eql?(recipient_name)}.first
               e[PACKAGE_MATCH_FIELD]=recipient.nil? ? 'n/a' : recipient['recipient_delivery_id'].first
             when :sent
               e[PACKAGE_MATCH_FIELD]=e['delivery_id'].first
@@ -208,9 +209,18 @@ module Aspera
               #Log.dump('transfer_spec',transfer_spec)
               return Main.result_transfer(self.transfer.start(transfer_spec,{:src=>:node_gen3}))
             when :recv
-              public_link_url=self.options.get_option(:link,:optional)
-              if !public_link_url.nil?
-                link_data=self.class.get_link_data(public_link_url)
+              link_url=self.options.get_option(:link,:optional)
+              # list of faspex ID/URI to download
+              pkg_id_uri=nil
+              skip_ids_data=[]
+              skip_ids_persistency=nil
+              case link_url
+              when nil
+                # usual case: no link
+              when /^faspe:/
+                pkg_id_uri=[{:id=>'package',:uri=>link_url}]
+              else
+                link_data=self.class.get_link_data(link_url)
                 if !link_data[:subpath].match(%r{external_deliveries/})
                   raise CliBadArgument,"pub link is #{link_data[:subpath]}, expecting external_deliveries/"
                 end
@@ -218,7 +228,7 @@ module Aspera
                 api_public_link=Rest.new({:base_url=>link_data[:base_url]})
                 pkgdatares=api_public_link.call({:operation=>'GET',:subpath=>link_data[:subpath],:url_params=>{:passcode=>link_data[:query]['passcode']},:headers=>{'Accept'=>'application/xml'}})
                 if !pkgdatares[:http].body.start_with?('<?xml ')
-                  OpenApplication.instance.uri(public_link_url)
+                  OpenApplication.instance.uri(link_url)
                   raise CliError, 'no such package'
                 end
                 package_entry=XmlSimple.xml_in(pkgdatares[:http].body, {'ForceArray' => false})
@@ -227,32 +237,30 @@ module Aspera
                 transfer_spec['direction']='receive'
                 return Main.result_transfer(self.transfer.start(transfer_spec,{:src=>:node_gen3}))
               end # public link
-              # get command line parameters
-              delivid=self.options.get_option(:id,:mandatory)
-              # list of faspex ID/URI to download
-              pkg_id_uri=nil
-              skip_ids_data=[]
-              skip_ids_persistency=nil
-              if self.options.get_option(:once_only,:mandatory)
-                skip_ids_persistency=PersistencyActionOnce.new(
-                manager: @agents[:persistency],
-                data: skip_ids_data,
-                ids:  ['faspex_recv',self.options.get_option(:url,:mandatory),self.options.get_option(:username,:mandatory),self.options.get_option(:box,:mandatory).to_s])
-              end
-              if delivid.eql?(VAL_ALL)
-                pkg_id_uri=mailbox_all_entries.map{|i|{:id=>i[PACKAGE_MATCH_FIELD],:uri=>self.class.get_fasp_uri_from_entry(i)}}
-                # TODO : remove ids from skip not present in inbox
-                # skip_ids_data.select!{|id|pkg_id_uri.select{|p|p[:id].eql?(id)}}
-                pkg_id_uri.select!{|i|!skip_ids_data.include?(i[:id])}
-              else
-                # TODO: delivery id is the right one if package was receive by group
-                endpoint=case self.options.get_option(:box,:mandatory)
-                when :inbox,:archive;'received'
-                when :sent; 'sent'
+              if pkg_id_uri.nil?
+                # get command line parameters
+                delivid=self.options.get_option(:id,:mandatory)
+                if self.options.get_option(:once_only,:mandatory)
+                  skip_ids_persistency=PersistencyActionOnce.new(
+                  manager: @agents[:persistency],
+                  data: skip_ids_data,
+                  ids:  ['faspex_recv',self.options.get_option(:url,:mandatory),self.options.get_option(:username,:mandatory),self.options.get_option(:box,:mandatory).to_s])
+                end
+                if delivid.eql?(VAL_ALL)
+                  pkg_id_uri=mailbox_all_entries.map{|i|{:id=>i[PACKAGE_MATCH_FIELD],:uri=>self.class.get_fasp_uri_from_entry(i)}}
+                  # TODO : remove ids from skip not present in inbox
+                  # skip_ids_data.select!{|id|pkg_id_uri.select{|p|p[:id].eql?(id)}}
+                  pkg_id_uri.select!{|i|!skip_ids_data.include?(i[:id])}
+                else
+                  # TODO: delivery id is the right one if package was receive by group
+                  endpoint=case self.options.get_option(:box,:mandatory)
+                  when :inbox,:archive;'received'
+                  when :sent; 'sent'
+                  end
+                  entry_xml=api_v3.call({:operation=>'GET',:subpath=>"#{endpoint}/#{delivid}",:headers=>{'Accept'=>'application/xml'}})[:http].body
+                  package_entry=XmlSimple.xml_in(entry_xml, {'ForceArray' => true})
+                  pkg_id_uri=[{:id=>delivid,:uri=>self.class.get_fasp_uri_from_entry(package_entry)}]
                 end
-                entry_xml=api_v3.call({:operation=>'GET',:subpath=>"#{endpoint}/#{delivid}",:headers=>{'Accept'=>'application/xml'}})[:http].body
-                package_entry=XmlSimple.xml_in(entry_xml, {'ForceArray' => true})
-                pkg_id_uri=[{:id=>delivid,:uri=>self.class.get_fasp_uri_from_entry(package_entry)}]
               end
               Log.dump(:pkg_id_uri,pkg_id_uri)
               return Main.result_status('no package') if pkg_id_uri.empty?

data/lib/aspera/cli/plugins/faspex5.rb

@@ -17,14 +17,20 @@ module Aspera
           options.set_option(:auth,:jwt)
           options.parse_options!
         end
-        ACTIONS=[ :node, :package, :auth_client ]
 
         def set_api
           faxpex5_api_base_url=options.get_option(:url,:mandatory)
           faxpex5_api_v5_url="#{faxpex5_api_base_url}/api/v5"
           faxpex5_api_auth_url="#{faxpex5_api_base_url}/auth"
           case options.get_option(:auth,:mandatory)
+          when :boot
+            # the password here is the token copied directly from browser in developer mode
+            @api_v5=Rest.new({
+              :base_url => faxpex5_api_v5_url,
+              :headers => {'Authorization'=>options.get_option(:password,:mandatory)},
+            })
           when :web
+            # opens a browser and ask user to auth using web
             @api_v5=Rest.new({
               :base_url => faxpex5_api_v5_url,
               :auth     => {
@@ -34,18 +40,9 @@ module Aspera
               :state          => SecureRandom.uuid,
               :client_id      => options.get_option(:client_id,:mandatory),
               :redirect_uri   => options.get_option(:redirect_uri,:mandatory),
-              #:token_field    =>'auth_token',
-              #:path_token     => 'token',
-              #:path_authorize => 'authorize',
-              #:userpass_body  => {name: faxpex5_username,password: faxpex5_password}
               }})
-          when :boot
-            @api_v5=Rest.new({
-              :base_url => faxpex5_api_v5_url,
-              :headers => {'Authorization'=>options.get_option(:password,:mandatory)},
-            })
           when :jwt
-            #raise "JWT to be implemented"
+            # currently Faspex 5 beta 3 only supports non-user based apis (e.g. jobs)
             app_client_id=options.get_option(:client_id,:mandatory)
             @api_v5=Rest.new({
               :base_url => faxpex5_api_v5_url,
@@ -55,16 +52,17 @@ module Aspera
               :grant               => :jwt,
               :client_id           => app_client_id,
               :client_secret       => options.get_option(:client_secret,:mandatory),
-              #:redirect_uri   => options.get_option(:redirect_uri,:mandatory),
               :jwt_subject         => "client:#{app_client_id}", # TODO Mmmm
               :jwt_audience        => app_client_id, # TODO Mmmm
               :jwt_private_key_obj => OpenSSL::PKey::RSA.new(options.get_option(:private_key,:mandatory)),
-              :jwt_is_f5           => true,
+              :jwt_is_f5           => true, # TODO: remove when clarified
               :jwt_headers         => {typ: 'JWT'}
               }})
           end
         end
 
+        ACTIONS=[ :node, :package, :auth_client, :jobs ]
+
         #
         def execute_action
           set_api
@@ -75,6 +73,9 @@ module Aspera
             return self.entity_action(api_auth,'oauth_clients',nil,:id,nil,true)
           when :node
             return self.entity_action(@api_v5,'nodes',nil,:id,nil,true)
+          when :jobs
+            # to test JWT
+            return self.entity_action(@api_v5,'jobs',nil,:id,nil,true)
           when :package
             command=options.get_next_command([:list,:show,:send,:receive])
             case command
@@ -86,7 +87,7 @@ module Aspera
               return {:type => :single_object, :data=>@api_v5.read("packages/#{id}")[:data]}
             when :send
               parameters=options.get_option(:value,:mandatory)
-              raise CliBadArgument,'package value must be hash, refer to API' unless parameters.is_a?(Hash)
+              raise CliBadArgument,'value must be hash, refer to API' unless parameters.is_a?(Hash)
               package=@api_v5.create('packages',parameters)[:data]
               transfer_spec=@api_v5.create("packages/#{package['id']}/transfer_spec/upload",{transfer_type: 'Connect'})[:data]
               transfer_spec.delete('authentication')
@@ -98,16 +99,17 @@ module Aspera
               skip_ids_data=[]
               skip_ids_persistency=nil
               if options.get_option(:once_only,:mandatory)
+                # read ids from persistency
                 skip_ids_persistency=PersistencyActionOnce.new(
                 manager: @agents[:persistency],
                 data: skip_ids_data,
                 ids:  ['faspex_recv',options.get_option(:url,:mandatory),options.get_option(:username,:mandatory),pkg_type])
               end
               if pack_id.eql?(VAL_ALL)
-                # todo: if packages have same name, they will overwrite
+                # TODO: if packages have same name, they will overwrite
                 parameters=options.get_option(:value,:optional)
-                parameters||={"type"=>"received","subtype"=>"mypackages","limit"=>1000}
-                raise CliBadArgument,'value filter must be hash (API GET)' unless parameters.is_a?(Hash)
+                parameters||={'type'=>'received','subtype'=>'mypackages','limit'=>1000}
+                raise CliBadArgument,'value filter must be Hash (API GET)' unless parameters.is_a?(Hash)
                 package_ids=@api_v5.read('packages',parameters)[:data]['packages'].map{|p|p['id']}
                 package_ids.select!{|i|!skip_ids_data.include?(i)}
               end

data/lib/aspera/cli/version.rb

@@ -1,5 +1,5 @@
 module Aspera
   module Cli
-    VERSION = "4.2.0"
+    VERSION = "4.2.1"
   end
 end

data/lib/aspera/fasp/manager.rb

@@ -72,6 +72,9 @@ module Aspera
       # start_transfer(transfer_spec,options) : start and wait for completion
       # wait_for_transfers_completion : wait for termination of all transfers, @return list of : :success or error message
       # optional: shutdown
+      
+      # This checks the validity of the value returned by wait_for_transfers_completion
+      # it must be a list of :success or exception
       def self.validate_status_list(statuses)
         raise "internal error: bad statuses type: #{statuses.class}" unless statuses.is_a?(Array)
         raise "internal error: bad statuses content: #{statuses}" unless statuses.select{|i|!i.eql?(:success) and !i.is_a?(StandardError)}.empty?

data/lib/aspera/oauth.rb

@@ -1,25 +1,26 @@
 require 'aspera/open_application'
+require 'aspera/web_auth'
 require 'base64'
 require 'date'
 require 'socket'
 require 'securerandom'
 
 module Aspera
-  # implement OAuth 2 for the REST client and generate a bearer token
+  # Implement OAuth 2 for the REST client and generate a bearer token
   # call get_authorization() to get a token.
   # bearer tokens are kept in memory and also in a file cache for later re-use
   # if a token is expired (api returns 4xx), call again get_authorization({:refresh=>true})
   class Oauth
     private
     # remove 5 minutes to account for time offset (TODO: configurable?)
-    JWT_NOTBEFORE_OFFSET=300
+    JWT_NOTBEFORE_OFFSET_SEC=300
     # one hour validity (TODO: configurable?)
-    JWT_EXPIRY_OFFSET=3600
+    JWT_EXPIRY_OFFSET_SEC=3600
+    # tokens older than 30 minutes will be discarded from cache
+    TOKEN_CACHE_EXPIRY_SEC=1800
+    # a prefix for persistency of tokens (garbage collect)
     PERSIST_CATEGORY_TOKEN='token'
-    # tokens older than 30 minutes will be discarded
-    TOKEN_EXPIRY_SECONDS=1800
-    THANK_YOU_HTML = "<html><head><title>Ok</title></head><body><h1>Thank you !</h1><p>You can close this window.</p></body></html>"
-    private_constant :JWT_NOTBEFORE_OFFSET,:JWT_EXPIRY_OFFSET,:PERSIST_CATEGORY_TOKEN,:TOKEN_EXPIRY_SECONDS,:THANK_YOU_HTML
+    private_constant :JWT_NOTBEFORE_OFFSET_SEC,:JWT_EXPIRY_OFFSET_SEC,:PERSIST_CATEGORY_TOKEN,:TOKEN_CACHE_EXPIRY_SEC
     class << self
       # OAuth methods supported
       def auth_types
@@ -89,12 +90,18 @@ module Aspera
         # we could check that host is localhost or local address
       end
       # cleanup expired tokens
-      self.class.persist_mgr.garbage_collect(PERSIST_CATEGORY_TOKEN,TOKEN_EXPIRY_SECONDS)
+      self.class.persist_mgr.garbage_collect(PERSIST_CATEGORY_TOKEN,TOKEN_CACHE_EXPIRY_SEC)
     end
 
     # open the login page, wait for code and check_code, then return code
     def goto_page_and_get_code(login_page_url,check_code)
-      request_params=self.class.goto_page_and_get_request(@params[:redirect_uri],login_page_url)
+      Log.log.info("login_page_url=#{login_page_url}".bg_red.gray)
+      # start a web server to receive request code
+      webserver=WebAuth.new(@params[:redirect_uri])
+      # start browser on login page
+      OpenApplication.instance.uri(login_page_url)
+      # wait for code in request
+      request_params=webserver.get_request
       Log.log.error("state does not match") if !check_code.eql?(request_params['state'])
       code=request_params['code']
       return code
@@ -222,8 +229,8 @@ module Aspera
             :iss => @params[:client_id],    # issuer
             :sub => @params[:jwt_subject],  # subject
             :aud => @params[:jwt_audience], # audience
-            :nbf => seconds_since_epoch-JWT_NOTBEFORE_OFFSET, # not before
-            :exp => seconds_since_epoch+JWT_EXPIRY_OFFSET # expiration
+            :nbf => seconds_since_epoch-JWT_NOTBEFORE_OFFSET_SEC, # not before
+            :exp => seconds_since_epoch+JWT_EXPIRY_OFFSET_SEC # expiration
           }
           # Hum.. compliant ? TODO: remove when Faspex5 API is clarified
           if @params[:jwt_is_f5]
@@ -311,32 +318,5 @@ module Aspera
       return 'Bearer '+token_data[@params[:token_field]]
     end
 
-    # open the login page, wait for code and return parameters
-    def self.goto_page_and_get_request(redirect_uri,login_page_url,html_page=THANK_YOU_HTML)
-      Log.log.info "login_page_url=#{login_page_url}".bg_red().gray()
-      # browser start is not blocking, we hope here that starting is slower than opening port
-      OpenApplication.instance.uri(login_page_url)
-      port=URI.parse(redirect_uri).port
-      Log.log.info "listening on port #{port}"
-      request_params=nil
-      TCPServer.open('127.0.0.1', port) { |webserver|
-        Log.log.info "server=#{webserver}"
-        websession = webserver.accept
-        sleep 1 # TODO: sometimes: returns nil ? use webrick ?
-        line = websession.gets.chomp
-        Log.log.info "line=#{line}"
-        if ! line.start_with?('GET /?') then
-          raise "unexpected request"
-        end
-        request = line.partition('?').last.partition(' ').first
-        data=URI.decode_www_form(request)
-        request_params=data.to_h
-        Log.log.debug "request_params=#{request_params}"
-        websession.print "HTTP/1.1 200/OK\r\nContent-type:text/html\r\n\r\n#{html_page}"
-        websession.close
-      }
-      return request_params
-    end
-
   end # OAuth
 end # Aspera

data/lib/aspera/web_auth.rb

@@ -0,0 +1,105 @@
+require 'webrick'
+require 'webrick/https'
+require 'thread'
+
+module Aspera
+  class WebAuth
+    # server for auth page
+    class FxGwServlet < WEBrick::HTTPServlet::AbstractServlet
+      def initialize(server,info) # additional args get here
+        @shared=info
+      end
+
+      def do_GET (request, response)
+        if ! request.path.eql?(@shared[:expected_path])
+          response.status=400
+          return
+        end
+        @shared[:mutex].synchronize do
+          @shared[:query]=request.query
+          @shared[:cond].signal
+        end
+        response.status=200
+        response.content_type = 'text/html'
+        response.body='<html><head><title>Ok</title></head><body><h1>Thank you !</h1><p>You can close this window.</p></body></html>'
+      end
+    end # FxGwServlet
+
+    # generates and adds self signed cert to provided webrick options
+    def fill_self_signed_cert(options)
+      key = OpenSSL::PKey::RSA.new(4096)
+      cert = OpenSSL::X509::Certificate.new
+      cert.subject = cert.issuer = OpenSSL::X509::Name.parse('/C=FR/O=Test/OU=Test/CN=Test')
+      cert.not_before = Time.now
+      cert.not_after = Time.now + 365 * 24 * 60 * 60
+      cert.public_key = key.public_key
+      cert.serial = 0x0
+      cert.version = 2
+      ef = OpenSSL::X509::ExtensionFactory.new
+      ef.issuer_certificate = cert
+      ef.subject_certificate = cert
+      cert.extensions = [
+        ef.create_extension('basicConstraints','CA:TRUE', true),
+        ef.create_extension('subjectKeyIdentifier', 'hash'),
+        # ef.create_extension('keyUsage', 'cRLSign,keyCertSign', true),
+      ]
+      cert.add_extension(ef.create_extension('authorityKeyIdentifier','keyid:always,issuer:always'))
+      cert.sign(key, OpenSSL::Digest::SHA256.new)
+      options[:SSLPrivateKey]  = key
+      options[:SSLCertificate] = cert
+    end
+
+    def initialize(endpoint_url)
+      uri=URI.parse(endpoint_url)
+      webrick_options = {
+        :app                => WebAuth,
+        :Port               => uri.port,
+        :Logger             => Log.log
+      }
+      uri_path=uri.path.empty? ? '/' : uri.path
+      case uri.scheme
+      when 'http'
+        Log.log.debug('HTTP mode')
+      when 'https'
+        webrick_options[:SSLEnable]=true
+        webrick_options[:SSLVerifyClient]=OpenSSL::SSL::VERIFY_NONE
+        case 0
+        when 0
+          # generate self signed cert
+          fill_self_signed_cert(webrick_options)
+        when 1
+          # short
+          webrick_options[:SSLCertName]    = [ [ 'CN',WEBrick::Utils::getservername ] ]
+          Log.log.error(">>>#{webrick_options[:SSLCertName]}")
+        when 2
+          # good cert
+          webrick_options[:SSLPrivateKey] =OpenSSL::PKey::RSA.new(File.read('/Users/laurent/workspace/Tools/certificate/myserver.key'))
+          webrick_options[:SSLCertificate] = OpenSSL::X509::Certificate.new(File.read('/Users/laurent/workspace/Tools/certificate/myserver.crt'))
+        end
+      end
+      # parameters for servlet
+      @shared_info={
+        expected_path: uri_path,
+        mutex: Mutex.new,
+        cond: ConditionVariable.new
+      }
+      @server = WEBrick::HTTPServer.new(webrick_options)
+      @server.mount(uri_path, FxGwServlet, @shared_info) # additional args provided to constructor
+      Thread.new { @server.start }
+    end
+
+    # wait for request on web server
+    # @return Hash the query
+    def get_request
+      Log.log.debug('get_request')
+      # called only once
+      raise "error, called twice ?" if @server.nil?
+      @shared_info[:mutex].synchronize do
+        @shared_info[:cond].wait(@shared_info[:mutex])
+      end
+      @server.shutdown
+      @server=nil
+      return @shared_info[:query]
+    end
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aspera-cli
 version: !ruby/object:Gem::Version
-  version: 4.2.0
+  version: 4.2.1
 platform: ruby
 authors:
 - Laurent Martin
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-08-24 00:00:00.000000000 Z
+date: 2021-09-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: xml-simple
@@ -213,6 +213,7 @@ files:
 - docs/test_env.conf
 - docs/transfer_spec.html
 - examples/aoc.rb
+- examples/faspex4.rb
 - examples/proxy.pac
 - examples/transfer.rb
 - lib/aspera/aoc.rb
@@ -298,6 +299,7 @@ files:
 - lib/aspera/sync.rb
 - lib/aspera/temp_file_manager.rb
 - lib/aspera/uri_reader.rb
+- lib/aspera/web_auth.rb
 homepage: https://github.com/IBM/aspera-cli
 licenses:
 - Apache-2.0