asperalm 0.9.19 → 0.9.20

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 400a613b56870ff842cfd56bc31eb466a7788198
4
- data.tar.gz: 7e33576d782dd876364aa76daa078693bcde467d
3
+ metadata.gz: 80b0249ed3560d572d6941da25db1ad8e3bc7f73
4
+ data.tar.gz: cc8a8e186b70aecdad35b8be07a5b470b2e03f79
5
5
  SHA512:
6
- metadata.gz: 30446bdeee6b2ad909d87fe7511c02a8777d2cb00fcf0579c2ef3a47675795071ec4f4724e9f24288dfcf493086253b17b260aee3cd1fe7fd74c4385f1d2a347
7
- data.tar.gz: a26e8c57ba063ff6706321bac53da9135bfd0a2b616683971beebd861dd5de7c7e90c4ee1e7a80722d8a4108651ab4ebdd5779d4c3d582a288a5a863ed205c48
6
+ metadata.gz: 802f9522472ba5d3da2646f191d69e7a9ea14d00a1b7bd088fcc8ef5db59292a6ef4c5086cbed16f3429a6f09e95a17681caaf8274f8d7b743d0051029adb58d
7
+ data.tar.gz: 592268864cf047b4b410e7961d0525cebbae20fcea4b5048251f903d997b33135caec1a053cf1f6820d5a6b2aecc6822ae4c854cc1382a0b1b2c24a64d00183d
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Asperalm - A Ruby library for Aspera transfers and "Amelia", the _Multi Layer IBM Aspera_ Command Line Tool
2
2
 
3
- Version : 0.9.19
3
+ Version : 0.9.20
4
4
 
5
5
 
6
6
  _Laurent/2016-2018_
@@ -37,7 +37,7 @@ Once the gem is installed, `mlia` shall be accessible:
37
37
 
38
38
  ```bash
39
39
  $ mlia --version
40
- 0.9.19
40
+ 0.9.20
41
41
  ```
42
42
 
43
43
  ## First use
@@ -987,7 +987,7 @@ mlia client connect id 'Aspera Connect for Windows' links id 'Windows Installer'
987
987
  mlia client connect id 'Aspera Connect for Windows' links list
988
988
  mlia client connect list
989
989
  mlia client current
990
- mlia conf wiz --url=my_url_here --config-file=$(SAMPLE_CONFIG_FILE) --client-id=$(HIDE_CLIENT_ID) --client-secret=$(HIDE_CLIENT_SECRET)
990
+ mlia conf wiz --url=my_url_here --config-file=$(SAMPLE_CONFIG_FILE) --client-id=$(HIDE_CLIENT_ID) --client-secret=$(HIDE_CLIENT_SECRET) --pkeypath=''
991
991
  mlia config export
992
992
  mlia config genkey sample_dest_folder/mykey
993
993
  mlia config id conf_name delete
@@ -1070,15 +1070,19 @@ mlia sync start --parameters=@json:'{"sessions":[{"name":"test","reset":true,"re
1070
1070
  ```bash
1071
1071
  $ mlia -h
1072
1072
  NAME
1073
- mlia -- a command line tool for Aspera Applications (v0.9.19)
1073
+ mlia -- a command line tool for Aspera Applications (v0.9.20)
1074
1074
 
1075
1075
  COMMAND: config
1076
1076
  SUBCOMMANDS: gem_path, genkey, plugins, flush_tokens, list, overview, open, echo, id, documentation, wizard, export_to_cli, detect, coffee
1077
1077
  OPTIONS:
1078
1078
  --value=VALUE extended value for create, update, list filter
1079
1079
  --id=VALUE resource identifier (modify,delete,show)
1080
+ --override=ENUM override existing value: yes, no
1080
1081
  --config-file=VALUE read parameters from file in YAML format, current=/Users/laurent/.aspera/mlia/config.yaml
1081
1082
  -N, --no-default do not load default configuration for plugin
1083
+ --global-client-id=ENUM wizard: AoC: use global or org specific jwt client id: yes, no
1084
+ --web-bootstrap=ENUM wizard: AoC: first login with web: yes, no
1085
+ --pkeypath=VALUE path to private key
1082
1086
  SYNOPSIS
1083
1087
  mlia COMMANDS [OPTIONS] [ARGS]
1084
1088
 
@@ -1123,7 +1127,6 @@ OPTIONS: global
1123
1127
  --query=VALUE additional filter for API calls (extended value) (some commands)
1124
1128
  --insecure=ENUM do not validate HTTPS certificate: yes, no
1125
1129
  --flat-hash=ENUM display hash values as additional keys: yes, no
1126
- --override=ENUM override existing value: yes, no
1127
1130
  --once-only=ENUM process only new items (some commands): yes, no
1128
1131
  --ts=VALUE override transfer spec values (Hash, use @json: prefix), current={}
1129
1132
  --to-folder=VALUE destination folder for downloaded files
@@ -1244,6 +1247,7 @@ OPTIONS:
1244
1247
  --password=VALUE user's password
1245
1248
  --file-access=ENUM how to read and write files in repository: local, remote
1246
1249
  --skip-types=VALUE skip types in comma separated list
1250
+ --skip-format=ENUM skip this preview format (multiple possible): png, mp4
1247
1251
  --overwrite=ENUM when to generate preview file: always, never, mtime
1248
1252
  --previews-folder=VALUE preview folder in files
1249
1253
  --iteration-file=VALUE path to iteration memory file
@@ -1328,7 +1332,7 @@ This requires additional setup.
1328
1332
  `mlia` provides a configuration wizard, to invoke it do:
1329
1333
 
1330
1334
  ```
1331
- $ mlia config wizard aspera --url=https://sedemo.ibmaspera.com
1335
+ $ mlia config wizard --url=https://sedemo.ibmaspera.com
1332
1336
  ```
1333
1337
 
1334
1338
  If the `url` parameter is not provided it will be asked on command line.
@@ -1359,21 +1363,17 @@ The first step is to declare `mlia` in Aspera on Cloud using the admin interface
1359
1363
 
1360
1364
  Let's start by a registration with web based authentication (auth=web):
1361
1365
 
1362
- <img src="docs/Auth-simple-web.png" alt="Screenshot: Web based auth API client registration form"/>
1363
-
1364
- * Open a web browser, log to your instance: e.g. `https://laurent.ibmaspera.com/`
1365
- * Go to Admin View-Organization-API Clients-Create
1366
- * Fill the API client creation form:
1366
+ * Open a web browser, log to your instance: e.g. `https://myorg.ibmaspera.com/`
1367
+ * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
1368
+ * Click "Create New"
1367
1369
  * Client Name: `mlia`
1368
1370
  * Redirect URIs: `http://localhost:12345`
1369
1371
  * Origins: `localhost`
1370
1372
  * uncheck "Prompt users to allow client to access"
1371
- * leave JWT unchecked for now
1372
- * Submit
1373
-
1374
- Note: for web based authentication, `mlia` listens on a local port (e.g. by default 12345), and the browser will provide the OAuth code there. For ``mlia` http is required, and 12345 is the default port.
1373
+ * leave the JWT part for now
1374
+ * Save
1375
1375
 
1376
- <img src="docs/Auth-registered-client.png" alt="Screenshot:Registered API Client"/>
1376
+ Note: for web based authentication, `mlia` listens on a local port (e.g. specified by the redirect_uri, in this example: 12345), and the browser will provide the OAuth code there. For ``mlia`, HTTP is required, and 12345 is the default port.
1377
1377
 
1378
1378
  Once the client is registered, a "Client ID" and "Secret" are created, these values will be used in the next step.
1379
1379
 
@@ -1383,7 +1383,7 @@ It is convenient to save several of those parameters in an [option preset](#lprt
1383
1383
 
1384
1384
  ```
1385
1385
  $ mlia config id my_aoc_org ask url client_id client_secret
1386
- option: url> https://laurent.ibmaspera.com/
1386
+ option: url> https://myorg.ibmaspera.com/
1387
1387
  option: client_id> BJLPObQiFw
1388
1388
  option: client_secret> yFS1mu-crbKuQhGFtfhYuoRW...
1389
1389
  updated: my_aoc_org
@@ -1397,7 +1397,7 @@ Define this [option preset](#lprt) as default configuration for the `aspera` plu
1397
1397
  $ mlia config id default set aspera my_aoc_org
1398
1398
  ```
1399
1399
 
1400
- Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`.
1400
+ Note: Default `auth` method is `web` and default `redirect_uri` is `http://localhost:12345`. Leave those default values.
1401
1401
 
1402
1402
  ### <a name="aocfirst"></a>First Use
1403
1403
 
@@ -1415,7 +1415,7 @@ For direct browser-less authentication, follow the [JWT](#jwt) section.
1415
1415
 
1416
1416
  ### <a name="jwt"></a>Activation of JSON Web Token (JWT) for direct authentication
1417
1417
 
1418
- In addition to basic API Client registration, the following steps are required for a Browser-less, Key-based authentication.
1418
+ In addition to basic API Client registration, the following steps are required for a Browser-less, Private Key-based authentication.
1419
1419
 
1420
1420
  #### Key Pair Generation
1421
1421
 
@@ -1455,19 +1455,12 @@ JWT needs to be authorized in Aspera on Cloud. This can be done in two manners:
1455
1455
 
1456
1456
  ##### Graphically
1457
1457
 
1458
- <img src="docs/AuthJWT.png" alt="Files-admin-organization-apiclient-create"/>
1459
-
1460
- * Open a web browser, log to your instance: https://laurent.ibmaspera.com/
1461
- * Go to Apps-Admin-Organization-Integrations
1458
+ * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
1459
+ * Go to Apps&rarr;Admin&rarr;Organization&rarr;Integrations
1462
1460
  * Click on the previously created application
1463
- * select tab : "Authentication Options"
1464
- * Modify options:
1465
- * activate "Enable JWT Grant Type"
1466
- * Client can retrieve tokens for: All Users
1467
- * Allowed keys: keep "User-specific keys"
1468
- * Enable admin tokens
1469
-
1470
- Note: It is also possible to allow a "super key" to impersonate any user by registering a "super key" at this step. (Select USer and Global, and then set public key).
1461
+ * select tab : "JSON Web Token Auth"
1462
+ * Modify options if necessary, for instance: activate both options in section "Settings"
1463
+ * Click "Save"
1471
1464
 
1472
1465
  ##### Using command line
1473
1466
 
@@ -1488,13 +1481,13 @@ The public key must be assigned to your user. This can be done in two manners:
1488
1481
 
1489
1482
  ##### Graphically
1490
1483
 
1491
- <img src="docs/AuthUsersKey.png" alt="Screenshot: Users key registration"/>
1484
+ open the previously generated public key located here: `$HOME/.aspera/mlia/aocapikey.pub`
1492
1485
 
1493
- * Open a web browser, log to your instance: https://laurent.ibmaspera.com/
1494
- * In User Admin View, click n the user's icon (top right)
1486
+ * Open a web browser, log to your instance: https://myorg.ibmaspera.com/
1487
+ * Click on the user's icon (top right)
1495
1488
  * Select "Account Settings"
1496
1489
  * Paste the _Public Key_ in the "Public Key" section
1497
- * Click on "Update"
1490
+ * Click on "Submit"
1498
1491
 
1499
1492
  ##### Using command line
1500
1493
 
@@ -1506,15 +1499,15 @@ $ mlia aspera admin res user list
1506
1499
  : 109952 : Tech Support :
1507
1500
  : 109951 : LAURENT MARTIN :
1508
1501
  :........:................:
1509
- $ mlia aspera admin res user --id=109951 modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/mlia/aocapikey.pub"))}'
1502
+ $ mlia aspera user info modify @ruby:'{"public_key"=>File.read(File.expand_path("~/.aspera/mlia/aocapikey.pub"))}'
1510
1503
  modified
1511
1504
  ```
1512
1505
 
1513
- Note: the `show` command can be used to verify modifications.
1506
+ Note: the `aspera user info show` command can be used to verify modifications.
1514
1507
 
1515
1508
  #### [option preset](#lprt) modification for JWT
1516
1509
 
1517
- To activate JWT authentication for `mlia` using the [option preset](#lprt), do the folowing:
1510
+ To activate default use of JWT authentication for `mlia` using the [option preset](#lprt), do the folowing:
1518
1511
 
1519
1512
  * change auth method to JWT
1520
1513
  * provide location of private key
@@ -2389,6 +2382,12 @@ Gems, or remove your ed25519 key from your `.ssh` folder to solve the issue.
2389
2382
 
2390
2383
  # Release Notes
2391
2384
 
2385
+ * version 0.9.20
2386
+
2387
+ * improved wizard (prepare for AoC global client id)
2388
+ * preview generator: addedoption : --skip-format=<png,mp4>
2389
+ * removed outdated pictures from this doc
2390
+
2392
2391
  * version 0.9.19
2393
2392
 
2394
2393
  * added command aspera bearer --scope=xx
@@ -1 +1 @@
1
- 0.9.19
1
+ 0.9.20
@@ -0,0 +1,57 @@
1
+ require 'asperalm/log'
2
+ require 'asperalm/rest'
3
+
4
+ module Asperalm
5
+ class ApiDetector
6
+ def self.discover_product(url)
7
+ uri=URI.parse(url)
8
+ api=Rest.new({:base_url=>url})
9
+ # Node
10
+ begin
11
+ result=api.call({:operation=>'GET',:subpath=>'ping'})
12
+ if result[:http].body.eql?('')
13
+ return {:product=>:node,:version=>'unknown'}
14
+ end
15
+ rescue SocketError => e
16
+ raise e
17
+ rescue => e
18
+ Log.log.debug("not node (#{e.class}: #{e})")
19
+ end
20
+ # AoC
21
+ begin
22
+ result=api.call({:operation=>'GET',:subpath=>'',:headers=>{'Accept'=>'text/html'}})
23
+ if result[:http].body.include?('content="AoC"')
24
+ return {:product=>:aoc,:version=>'unknown'}
25
+ end
26
+ rescue SocketError => e
27
+ raise e
28
+ rescue => e
29
+ Log.log.debug("not aoc (#{e.class}: #{e})")
30
+ end
31
+ # Faspex
32
+ begin
33
+ result=api.call({:operation=>'POST',:subpath=>'aspera/faspex',:headers=>{'Accept'=>'application/xrds+xml'},:text_body_params=>''})
34
+ if result[:http].body.start_with?('<?xml')
35
+ res_s=XmlSimple.xml_in(result[:http].body, {"ForceArray" => false})
36
+ version=res_s['XRD']['application']['version']
37
+ #return JSON.pretty_generate(res_s)
38
+ end
39
+ return {:product=>:faspex,:version=>version}
40
+ rescue
41
+ Log.log.debug("not faspex")
42
+ end
43
+ # Shares
44
+ begin
45
+ result=api.read('node_api/app')
46
+ Log.log.warn("not supposed to work")
47
+ rescue RestCallError => e
48
+ if e.response.code.to_s.eql?('401') and e.response.body.eql?('{"error":{"user_message":"API user authentication failed"}}')
49
+ return {:product=>:shares,:version=>'unknown'}
50
+ end
51
+ Log.log.warn("not shares: #{e.response.code} #{e.response.body}")
52
+ rescue
53
+ end
54
+ return {:product=>:unknown,:version=>'unknown'}
55
+ end
56
+ end
57
+ end
@@ -150,7 +150,6 @@ module Asperalm
150
150
  @opt_mgr.add_opt_simple(:query,"additional filter for API calls (extended value) (some commands)")
151
151
  @opt_mgr.add_opt_boolean(:insecure,"do not validate HTTPS certificate")
152
152
  @opt_mgr.add_opt_boolean(:flat_hash,"display hash values as additional keys")
153
- @opt_mgr.add_opt_boolean(:override,"override existing value")
154
153
  @opt_mgr.add_opt_boolean(:once_only,"process only new items (some commands)")
155
154
 
156
155
  @opt_mgr.set_option(:ui,OpenApplication.default_gui_mode)
@@ -181,8 +181,9 @@ module Asperalm
181
181
  self.options.set_option(:url,'https://'+uri.host)
182
182
  self.options.set_option(:public_token,url_token_value)
183
183
  self.options.set_option(:auth,:url_token)
184
- self.options.set_option(:client_id,FilesApi.random.first)
185
- self.options.set_option(:client_secret,FilesApi.random.last)
184
+ client_data=FilesApi.random_drive
185
+ self.options.set_option(:client_id,client_data.first)
186
+ self.options.set_option(:client_secret,client_data.last)
186
187
  end
187
188
  # Connection paramaters (url and auth) to Aspera on Cloud
188
189
  # pre populate rest parameters based on URL
@@ -194,6 +195,12 @@ module Asperalm
194
195
  :client_secret => self.options.get_option(:client_secret,:mandatory)
195
196
  })
196
197
 
198
+ # add jwt payload for global ids
199
+ if FilesApi.is_global_client_id?(aoc_rest_auth[:client_id])
200
+ org=aoc_rest_auth[:base_url].gsub(/.*\//,'')
201
+ aoc_rest_auth.merge!({:jwt_add=>{org: org}})
202
+ end
203
+
197
204
  # fill other auth parameters based on Oauth method
198
205
  case aoc_rest_auth[:grant]
199
206
  when :web
@@ -1,5 +1,6 @@
1
1
  require 'asperalm/cli/basic_auth_plugin'
2
2
  require 'asperalm/fasp/installation'
3
+ require 'asperalm/api_detector'
3
4
  require 'xmlsimple'
4
5
  require 'base64'
5
6
 
@@ -30,8 +31,14 @@ module Asperalm
30
31
  add_plugin_lookup_folder(File.join(Main.gem_root,@@GEM_PLUGINS_FOLDER))
31
32
  self.options.set_obj_attr(:override,self,:option_override,:no)
32
33
  self.options.set_obj_attr(:config_file,self,:option_config_file)
34
+ self.options.add_opt_boolean(:override,"override existing value")
33
35
  self.options.add_opt_simple(:config_file,"read parameters from file in YAML format, current=#{@option_config_file}")
34
36
  self.options.add_opt_switch(:no_default,"-N","do not load default configuration for plugin") { @use_plugin_defaults=false }
37
+ self.options.add_opt_boolean(:global_client_id,'wizard: AoC: use global or org specific jwt client id')
38
+ self.options.add_opt_boolean(:web_bootstrap,'wizard: AoC: first login with web')
39
+ self.options.add_opt_simple(:pkeypath,"path to private key")
40
+ self.options.set_option(:global_client_id,false)
41
+ self.options.set_option(:web_bootstrap,true)
35
42
  self.options.parse_options!
36
43
  end
37
44
 
@@ -49,11 +56,13 @@ module Asperalm
49
56
 
50
57
  # folder in $HOME for application files (config, cache)
51
58
  @@ASPERA_HOME_FOLDER_NAME='.aspera'
52
- # main config file
59
+ # default config file
53
60
  @@DEFAULT_CONFIG_FILENAME = 'config.yaml'
61
+ # reserved preset names
54
62
  @@CONF_PRESET_CONFIG='config'
55
63
  @@CONF_PRESET_VERSION='version'
56
64
  @@CONF_PRESET_DEFAULT='default'
65
+ # old tool name
57
66
  @@OLD_PROGRAM_NAME = 'aslmcli'
58
67
  # default redirect for AoC web auth
59
68
  @@DEFAULT_REDIRECT='http://localhost:12345'
@@ -62,13 +71,14 @@ module Asperalm
62
71
  # folder containing plugins in the gem's main folder
63
72
  @@GEM_PLUGINS_FOLDER='asperalm/cli/plugins'
64
73
  @@RUBY_FILE_EXT='.rb'
65
- @@RANDOM_CLIENT='YXNwZXJhLmdsb2JhbC1jbGktY2xpZW50OmZycG1zUnNHNG1qWjBQbHhDZ2RKbHZPTnFCZzRWbHB6X0lYN2dYbUJNQWZzZ01MeTJGTzZDWExvZEtmS0F1aHFuQ3FTcHRMYmVfd2Rtbm05SlJ1RVBPLVBwRnFwcV9LYgo='
74
+ @@OLD_AOC_COMMAND='files'
75
+ @@NEW_AOC_COMMAND='aspera'
66
76
 
67
- def generate_new_key(key_filepath)
77
+ def generate_new_key(private_key_path)
68
78
  require 'openssl'
69
79
  priv_key = OpenSSL::PKey::RSA.new(2048)
70
- File.write(key_filepath,priv_key.to_s)
71
- File.write(key_filepath+".pub",priv_key.public_key.to_s)
80
+ File.write(private_key_path,priv_key.to_s)
81
+ File.write(private_key_path+".pub",priv_key.public_key.to_s)
72
82
  nil
73
83
  end
74
84
 
@@ -129,12 +139,10 @@ module Asperalm
129
139
  save_required=false
130
140
  config_tested_version='0.6.14'
131
141
  if Gem::Version.new(version) <= Gem::Version.new(config_tested_version)
132
- old_plugin_name='files'
133
- new_plugin_name=Plugins::Aspera.name_sym.to_s
134
- if @config_presets[@@CONF_PRESET_DEFAULT].is_a?(Hash) and @config_presets[@@CONF_PRESET_DEFAULT].has_key?(old_plugin_name)
135
- @config_presets[@@CONF_PRESET_DEFAULT][new_plugin_name]=@config_presets[@@CONF_PRESET_DEFAULT][old_plugin_name]
136
- @config_presets[@@CONF_PRESET_DEFAULT].delete(old_plugin_name)
137
- Log.log.warn("Converted plugin default: #{old_plugin_name} -> #{new_plugin_name}")
142
+ if @config_presets[@@CONF_PRESET_DEFAULT].is_a?(Hash) and @config_presets[@@CONF_PRESET_DEFAULT].has_key?(@@OLD_AOC_COMMAND)
143
+ @config_presets[@@CONF_PRESET_DEFAULT][@@NEW_AOC_COMMAND]=@config_presets[@@CONF_PRESET_DEFAULT][@@OLD_AOC_COMMAND]
144
+ @config_presets[@@CONF_PRESET_DEFAULT].delete(@@OLD_AOC_COMMAND)
145
+ Log.log.warn("Converted plugin default: #{@@OLD_AOC_COMMAND} -> #{@@NEW_AOC_COMMAND}")
138
146
  save_required=true
139
147
  end
140
148
  end
@@ -268,9 +276,9 @@ module Asperalm
268
276
  OpenApplication.instance.uri("#{@option_config_file}") #file://
269
277
  return Main.result_nothing
270
278
  when :genkey # generate new rsa key
271
- key_filepath=self.options.get_next_argument('private key file path')
272
- generate_new_key(key_filepath)
273
- return Main.result_status('generated key: '+key_filepath)
279
+ private_key_path=self.options.get_next_argument('private key file path')
280
+ generate_new_key(private_key_path)
281
+ return Main.result_status('generated key: '+private_key_path)
274
282
  when :echo # display the content of a value given on command line
275
283
  result={:type=>:other_struct, :data=>self.options.get_next_argument("value")}
276
284
  # special for csv
@@ -287,72 +295,100 @@ module Asperalm
287
295
  return {:type=>:object_list,:data=>self.class.flatten_all_config(@config_presets)}
288
296
  when :wizard
289
297
  self.options.ask_missing_mandatory=true
298
+ #self.options.set_option(:interactive,:yes)
290
299
  # register url option
291
300
  BasicAuthPlugin.new(@agents.merge(skip_option_header: true))
292
301
  instance_url=self.options.get_option(:url,:mandatory)
293
- appli=discover_product(instance_url)
302
+ appli=ApiDetector.discover_product(instance_url)
294
303
  case appli[:product]
295
304
  when :aoc
296
305
  self.format.display_status("Detected: Aspera on Cloud")
297
- require 'asperalm/cli/plugins/aspera'
298
- files_plugin=Plugins::Aspera.new(@agents.merge({skip_basic_auth_options: true}))
299
- #self.options.set_option(:interactive,:yes)
300
- #self.options.set_option(:url,instance_url)
301
- self.options.set_option(:auth,:web)
302
- self.options.set_option(:redirect_uri,@@DEFAULT_REDIRECT)
303
306
  organization,instance_domain=FilesApi.parse_url(instance_url)
304
307
  aspera_preset_name='aoc_'+organization
305
- self.format.display_status("Creating preset: #{aspera_preset_name}")
306
- key_filepath=File.join(@main_folder,'aspera_on_cloud_key')
307
- if File.exist?(key_filepath)
308
- puts "key file already exists: #{key_filepath}, keeping it"
308
+ self.format.display_status("Preparing preset: #{aspera_preset_name}")
309
+ # init defaults if necessary
310
+ @config_presets[@@CONF_PRESET_DEFAULT]||=Hash.new
311
+ if !option_override
312
+ raise CliError,"a default configuration already exists for plugin '#{@@NEW_AOC_COMMAND}' (use --override=yes)" if @config_presets[@@CONF_PRESET_DEFAULT].has_key?(@@NEW_AOC_COMMAND)
313
+ raise CliError,"preset already exists: #{aspera_preset_name} (use --override=yes)" if @config_presets.has_key?(aspera_preset_name)
314
+ end
315
+ # lets see if path to priv key is provided
316
+ private_key_path=self.options.get_option(:pkeypath,:optional)
317
+ # give a chance to provide
318
+ if private_key_path.nil?
319
+ self.format.display_status("Please provide path to your private RSA key, or empty to generate one:")
320
+ private_key_path=self.options.get_option(:pkeypath,:mandatory).to_s
321
+ end
322
+ # else generate path
323
+ if private_key_path.empty?
324
+ private_key_path=File.join(@main_folder,'aspera_on_cloud_key')
325
+ end
326
+ if File.exist?(private_key_path)
327
+ self.format.display_status("using existing key:")
309
328
  else
310
- puts "generating: #{key_filepath}"
311
- generate_new_key(key_filepath)
329
+ self.format.display_status("generating:")
330
+ generate_new_key(private_key_path)
312
331
  end
313
- # if no api client info on command line, ask to get it
314
- if self.options.get_option(:client_id,:optional).nil? or self.options.get_option(:client_secret,:optional).nil?
315
- #client_data=Base64.decode64(@@RANDOM_CLIENT).split(':')
316
- #self.options.set_option(:client_id,client_data.first)
317
- #self.options.set_option(:client_secret,client_data.last);
318
- puts "Please login to your Aspera on Cloud instance as Administrator."
319
- puts "Go to: Admin->Organization->Integrations"
320
- puts "Create a new integration:"
321
- puts "- name: #{@tool_name}"
322
- puts "- redirect uri: #{@@DEFAULT_REDIRECT}"
323
- puts "- origin: localhost"
324
- puts "Once created please enter the following any required parameter:"
332
+ self.format.display_status("#{private_key_path}")
333
+
334
+ # define options
335
+ require 'asperalm/cli/plugins/aspera'
336
+ files_plugin=Plugins::Aspera.new(@agents.merge({skip_basic_auth_options: true}))
337
+
338
+ if self.options.get_option(:global_client_id)
339
+ self.format.display_status("Using global client_id.")
340
+ client_data=FilesApi.random_cli
341
+ self.options.set_option(:auth,:jwt)
342
+ self.options.set_option(:client_id,client_data.first)
343
+ self.options.set_option(:client_secret,client_data.last)
344
+ #self.options.set_option(:redirect_uri,'https://asperafiles.com/token')
345
+ # not supported
346
+ if self.options.get_option(:web_bootstrap)
347
+ self.format.display_status("web bootstrap not supported in global id")
348
+ self.options.set_option(:web_bootstrap, false)
349
+ end
350
+ else
351
+ self.format.display_status("Using organization specific client_id.")
352
+ self.options.set_option(:auth,:web)
353
+ self.options.set_option(:redirect_uri,@@DEFAULT_REDIRECT)
354
+ self.format.display_status("Please login to your Aspera on Cloud instance as Administrator.".red)
355
+ self.format.display_status("Go to: Admin->Organization->Integrations")
356
+ self.format.display_status("Check if there is an integration named:")
357
+ self.format.display_status("- name: #{@tool_name}")
358
+ self.format.display_status("If not, create a new integration:")
359
+ self.format.display_status("- name: #{@tool_name}")
360
+ self.format.display_status("- redirect uri: #{@@DEFAULT_REDIRECT}")
361
+ self.format.display_status("- origin: localhost")
362
+ self.format.display_status("Once created or identified,")
363
+ self.format.display_status("Please enter the following any required parameter:".red)
325
364
  OpenApplication.instance.uri(instance_url+"/admin/org/integrations")
326
365
  self.options.get_option(:client_id,:mandatory)
327
366
  self.options.get_option(:client_secret,:mandatory)
328
367
  end
329
- # init defaults if necessary
330
- @config_presets[@@CONF_PRESET_DEFAULT]||=Hash.new
331
- if !option_override
332
- raise CliError,"a default configuration already exists (use --override=yes)" if @config_presets[@@CONF_PRESET_DEFAULT].has_key?(Plugins::Aspera.name_sym.to_s)
333
- raise CliError,"preset already exists: #{aspera_preset_name} (use --override=yes)" if @config_presets.has_key?(aspera_preset_name)
334
- end
368
+
369
+ self.options.set_option(:private_key,'@file:'+private_key_path)
335
370
  # todo: check if key is identical
336
371
  api_aoc=files_plugin.get_aoc_api(true)
372
+
337
373
  myself=api_aoc.read('self')[:data]
338
374
  raise CliError,"public key is already set in profile (use --override=yes)" unless myself['public_key'].empty? or option_override
339
- puts "updating profile with new key"
340
- api_aoc.update("users/#{myself['id']}",{'public_key'=>File.read(key_filepath+'.pub')})
341
- puts "Enabling JWT for client"
375
+ self.format.display_status("updating profile with new key")
376
+ api_aoc.update("users/#{myself['id']}",{'public_key'=>File.read(private_key_path+'.pub')})
377
+ self.format.display_status("Enabling JWT for client")
342
378
  api_aoc.update("clients/#{self.options.get_option(:client_id)}",{'jwt_grant_enabled'=>true,'explicit_authorization_required'=>false})
343
- puts "creating new config preset: #{aspera_preset_name}"
379
+ self.format.display_status("creating new config preset: #{aspera_preset_name}")
344
380
  @config_presets[aspera_preset_name]={
345
381
  :url.to_s =>self.options.get_option(:url),
346
382
  :redirect_uri.to_s =>self.options.get_option(:redirect_uri),
347
383
  :client_id.to_s =>self.options.get_option(:client_id),
348
384
  :client_secret.to_s =>self.options.get_option(:client_secret),
349
385
  :auth.to_s =>:jwt.to_s,
350
- :private_key.to_s =>'@file:'+key_filepath,
386
+ :private_key.to_s =>'@file:'+private_key_path,
351
387
  :username.to_s =>myself['email'],
352
388
  }
353
- puts "setting config preset as default for #{Plugins::Aspera.name_sym.to_s}"
389
+ self.format.display_status("setting config preset as default for #{Plugins::Aspera.name_sym.to_s}")
354
390
  @config_presets[@@CONF_PRESET_DEFAULT][Plugins::Aspera.name_sym.to_s]=aspera_preset_name
355
- puts "saving config file"
391
+ self.format.display_status("saving config file")
356
392
  save_presets_to_config_file
357
393
  return Main.result_status("Done. You can test with:\n#{@tool_name} aspera user info show")
358
394
  else
@@ -392,7 +428,7 @@ module Asperalm
392
428
  when :detect
393
429
  # need url / username
394
430
  BasicAuthPlugin.new(@agents)
395
- return Main.result_status("found: #{discover_product(self.options.get_option(:url,:mandatory))}")
431
+ return Main.result_status("found: #{ApiDetector.discover_product(self.options.get_option(:url,:mandatory))}")
396
432
  when :coffee
397
433
  OpenApplication.instance.uri('https://enjoyjava.com/wp-content/uploads/2018/01/How-to-make-strong-coffee.jpg')
398
434
  return Main.result_nothing
@@ -429,40 +465,6 @@ module Asperalm
429
465
  return nil
430
466
  end
431
467
 
432
- def discover_product(url)
433
- uri=URI.parse(url)
434
- api=Rest.new({:base_url=>url})
435
- begin
436
- result=api.call({:operation=>'GET',:subpath=>'',:headers=>{'Accept'=>'text/html'}})
437
- if result[:http].body.include?('content="AoC"')
438
- return {:product=>:aoc,:version=>'unknown'}
439
- end
440
- rescue => e
441
- Log.log.debug("not aoc (#{e})")
442
- end
443
- begin
444
- result=api.call({:operation=>'POST',:subpath=>'aspera/faspex',:headers=>{'Accept'=>'application/xrds+xml'},:text_body_params=>''})
445
- if result[:http].body.start_with?('<?xml')
446
- res_s=XmlSimple.xml_in(result[:http].body, {"ForceArray" => false})
447
- version=res_s['XRD']['application']['version']
448
- #return JSON.pretty_generate(res_s)
449
- end
450
- return {:product=>:faspex,:version=>version}
451
- rescue
452
- Log.log.debug("not faspex")
453
- end
454
- begin
455
- result=api.read('node_api/app')
456
- Log.log.warn("not supposed to work")
457
- rescue RestCallError => e
458
- if e.response.code.to_s.eql?('401') and e.response.body.eql?('{"error":{"user_message":"API user authentication failed"}}')
459
- return {:product=>:shares,:version=>'unknown'}
460
- end
461
- Log.log.warn("not shares: #{e.response.code} #{e.response.body}")
462
- rescue
463
- end
464
- return {:product=>:unknown,:version=>'unknown'}
465
- end
466
468
 
467
469
  end
468
470
  end
@@ -10,10 +10,20 @@ module Asperalm
10
10
  module Cli
11
11
  module Plugins
12
12
  class Preview < BasicAuthPlugin
13
+ attr_accessor :option_overwrite
14
+ attr_accessor :option_previews_folder
15
+ attr_accessor :option_folder_reset_cache
16
+ attr_accessor :option_temp_folder
17
+ attr_accessor :option_skip_folders
13
18
  def initialize(env)
14
19
  super(env)
20
+ @skip_types=[]
21
+ @default_transfer_spec=nil
22
+ # by default generate all supported formats
23
+ @preview_formats=Asperalm::Preview::Generator.preview_formats
15
24
  # link CLI options to gen_info attributes
16
25
  self.options.set_obj_attr(:skip_types,self,:option_skip_types)
26
+ self.options.set_obj_attr(:skip_format,self,:option_skip_format,[])
17
27
  self.options.set_obj_attr(:overwrite,self,:option_overwrite,:mtime)
18
28
  self.options.set_obj_attr(:previews_folder,self,:option_previews_folder,'previews')
19
29
  self.options.set_obj_attr(:folder_reset_cache,self,:option_folder_reset_cache,:no)
@@ -38,6 +48,7 @@ module Asperalm
38
48
  self.options.set_obj_attr(:check_extension,Asperalm::Preview::Options.instance,:check_extension,:yes)
39
49
  self.options.add_opt_list(:file_access,[:local,:remote],"how to read and write files in repository")
40
50
  self.options.add_opt_simple(:skip_types,"skip types in comma separated list")
51
+ self.options.add_opt_list(:skip_format,Asperalm::Preview::Generator.preview_formats,"skip this preview format (multiple possible)")
41
52
  self.options.add_opt_list(:overwrite,Preview.overwrite_policies,"when to generate preview file")
42
53
  self.options.add_opt_simple(:previews_folder,"preview folder in files")
43
54
  self.options.add_opt_simple(:iteration_file,"path to iteration memory file")
@@ -63,16 +74,8 @@ module Asperalm
63
74
  self.options.add_opt_boolean(:check_extension,"check extra file extensions")
64
75
  self.options.set_option(:file_access,:local)
65
76
  self.options.parse_options!
66
- @skip_types=[]
67
- @default_transfer_spec=nil
68
77
  end
69
78
 
70
- attr_accessor :option_overwrite
71
- attr_accessor :option_previews_folder
72
- attr_accessor :option_folder_reset_cache
73
- attr_accessor :option_skip_folders
74
- attr_accessor :option_temp_folder
75
-
76
79
  # special tag to identify transfers related to generator
77
80
  PREV_GEN_TAG='preview_generator'
78
81
  # defined by node API
@@ -95,6 +98,14 @@ module Asperalm
95
98
  return @skip_types.map{|i|i.to_s}.join(',')
96
99
  end
97
100
 
101
+ def option_skip_format=(value)
102
+ @preview_formats.delete(value)
103
+ end
104
+
105
+ def option_skip_format
106
+ return @preview_formats.map{|i|i.to_s}.join(',')
107
+ end
108
+
98
109
  def action_list; [:scan,:events,:folder,:check,:test];end
99
110
 
100
111
  # /files/id/files is normally cached in redis, but we can discard the cache
@@ -226,7 +237,7 @@ module Asperalm
226
237
  # where previews will be generated for this particular entry
227
238
  local_entry_preview_dir=String.new
228
239
  # prepare generic information
229
- gen_infos=Asperalm::Preview::Generator.preview_formats.map do |preview_format|
240
+ gen_infos=@preview_formats.map do |preview_format|
230
241
  {
231
242
  :preview_format => preview_format,
232
243
  :base_dest => preview_filename(preview_format)
@@ -13,9 +13,13 @@ module Asperalm
13
13
  @@ARGS_PARAM='@args'
14
14
  @@TS_PARAM='@ts'
15
15
  def initialize(env)
16
+ # external objects: option manager, config file manager
16
17
  @env=env
18
+ # custom transfer spec provided on command line
17
19
  @transfer_spec_cmdline={}
20
+ # the actual selected transfer agent
18
21
  @agent=nil
22
+ # source/destination pair, like "paths" of transfer spec
19
23
  @transfer_paths=nil
20
24
  end
21
25
  public
@@ -56,32 +60,30 @@ module Asperalm
56
60
  when :node
57
61
  @agent=Fasp::Node.instance
58
62
  # way for code to setup alternate node api in avance
59
- if @agent.node_api.nil?
60
- # support: @param:<name>
61
- # support extended values
62
- node_config=@env[:options].get_option(:transfer_info,:optional)
63
- # of not specified, use default node
64
- if node_config.nil?
65
- param_set_name=@env[:config].get_plugin_default_config_name(:node)
66
- raise CliBadArgument,"No default node configured, Please specify --#{:transfer_info.to_s.gsub('_','-')}" if param_set_name.nil?
67
- node_config=@env[:config].preset_by_name(param_set_name)
68
- end
69
- Log.log.debug("node=#{node_config}")
70
- raise CliBadArgument,"the node configuration shall be a hash, use either @json:<json> or @preset:<parameter set name>" if !node_config.is_a?(Hash)
71
- # now check there are required parameters
72
- sym_config={}
73
- [:url,:username,:password].each do |param|
74
- raise CliBadArgument,"missing parameter [#{param}] in node specification: #{node_config}" if !node_config.has_key?(param.to_s)
75
- sym_config[param]=node_config[param.to_s]
76
- end
77
- @agent.node_api=Rest.new({
78
- :base_url => sym_config[:url],
79
- :auth => {
80
- :type =>:basic,
81
- :username => sym_config[:username],
82
- :password => sym_config[:password]
83
- }})
63
+ # support: @param:<name>
64
+ # support extended values
65
+ node_config=@env[:options].get_option(:transfer_info,:optional)
66
+ # if not specified: use default node
67
+ if node_config.nil?
68
+ param_set_name=@env[:config].get_plugin_default_config_name(:node)
69
+ raise CliBadArgument,"No default node configured, Please specify --#{:transfer_info.to_s.gsub('_','-')}" if param_set_name.nil?
70
+ node_config=@env[:config].preset_by_name(param_set_name)
71
+ end
72
+ Log.log.debug("node=#{node_config}")
73
+ raise CliBadArgument,"the node configuration shall be a hash, use either @json:<json> or @preset:<parameter set name>" if !node_config.is_a?(Hash)
74
+ # now check there are required parameters
75
+ sym_config=[:url,:username,:password].inject({}) do |h,param|
76
+ raise CliBadArgument,"missing parameter [#{param}] in node specification: #{node_config}" if !node_config.has_key?(param.to_s)
77
+ h[param]=node_config[param.to_s]
78
+ h
84
79
  end
80
+ @agent.node_api=Rest.new({
81
+ :base_url => sym_config[:url],
82
+ :auth => {
83
+ :type =>:basic,
84
+ :username => sym_config[:username],
85
+ :password => sym_config[:password]
86
+ }})
85
87
  when :aoc
86
88
  @agent=Fasp::Aoc.instance
87
89
  node_config=@env[:options].get_option(:transfer_info,:optional)
@@ -143,7 +145,8 @@ module Asperalm
143
145
  return @transfer_paths
144
146
  end
145
147
 
146
- # plugins shall use this method to start a transfer
148
+ # start a transfer
149
+ # plugins shall use this method
147
150
  # @param: options[:src] specifies how destination_root is set (how transfer spec was generated)
148
151
  # and not the default one
149
152
  def start(transfer_spec,options)
@@ -184,6 +187,7 @@ module Asperalm
184
187
  return @agent.wait_for_transfers_completion
185
188
  end
186
189
 
190
+ # shut down if agent requires it
187
191
  def shutdown
188
192
  @agent.shutdown if @agent.respond_to?(:shutdown)
189
193
  end
@@ -24,9 +24,14 @@ module Asperalm
24
24
 
25
25
  # some cool random string
26
26
  # strings /Applications/Aspera\ Drive.app/Contents/MacOS/AsperaDrive|grep -E '.{100}==$'|rev
27
- RANDOM_SEED='==QMGdXZsdkYMlDezZ3MNhDStYFWQNXNrZTOPRmYh10ZmJkN2EnZCFHbFxkRQtmNylTQOpHdtdUTPNFTxFGWFdFWFB1dr1iNK5WTadTLSFGWBlFTkVDdoxkYjx0MRp3ZlVlOlZXayRmLhJXZwNXY';
28
- def self.random
29
- Base64.strict_decode64(RANDOM_SEED.reverse).split(':')
27
+ RANDOM_DRIVE='==QMGdXZsdkYMlDezZ3MNhDStYFWQNXNrZTOPRmYh10ZmJkN2EnZCFHbFxkRQtmNylTQOpHdtdUTPNFTxFGWFdFWFB1dr1iNK5WTadTLSFGWBlFTkVDdoxkYjx0MRp3ZlVlOlZXayRmLhJXZwNXY';
28
+ @@RANDOM_CLIENT='YXNwZXJhLmdsb2JhbC1jbGktY2xpZW50OmZycG1zUnNHNG1qWjBQbHhDZ2RKbHZPTnFCZzRWbHB6X0lYN2dYbUJNQWZzZ01MeTJGTzZDWExvZEtmS0F1aHFuQ3FTcHRMYmVfd2Rtbm05SlJ1RVBPLVBwRnFwcV9LYgo='
29
+ def self.random_drive
30
+ Base64.strict_decode64(RANDOM_DRIVE.reverse).split(':')
31
+ end
32
+
33
+ def self.random_cli
34
+ Base64.strict_decode64(@@RANDOM_CLIENT).split(':')
30
35
  end
31
36
 
32
37
  def self.parse_url(aoc_org_url)
@@ -60,6 +65,10 @@ module Asperalm
60
65
  return 'node.'+access_key+':'+scope
61
66
  end
62
67
 
68
+ def self.is_global_client_id?(client_id)
69
+ client_id.start_with?('aspera.global')
70
+ end
71
+
63
72
  def initialize(rest_params)
64
73
  super(rest_params)
65
74
  @secrets={}
@@ -229,23 +229,28 @@ module Asperalm
229
229
  :state => UNUSED_STATE
230
230
  }))
231
231
  when :jwt
232
+ # https://tools.ietf.org/html/rfc7519
232
233
  require 'jwt'
233
234
  # remove 5 minutes to account for time offset
234
235
  seconds_since_epoch=Time.new.to_i-@@OFFSET_ALLOWANCE_SEC
235
236
  Log.log.info("seconds=#{seconds_since_epoch}")
236
237
 
237
238
  payload = {
238
- :iss => @params[:client_id],
239
- :sub => @params[:jwt_subject],
240
- :aud => @params[:jwt_audience],
239
+ :iss => @params[:client_id], # issuer
240
+ :sub => @params[:jwt_subject], # subject
241
+ :aud => @params[:jwt_audience], # audience
241
242
  :nbf => seconds_since_epoch,
242
243
  :exp => seconds_since_epoch+@@ASSERTION_VALIDITY_SEC # TODO: configurable ?
243
244
  }
244
245
 
246
+ # non standard, only for global ids
247
+ payload.merge!(@params[:jwt_add]) if @params.has_key?(:jwt_add)
248
+
245
249
  rsa_private=@params[:jwt_private_key_obj] # type: OpenSSL::PKey::RSA
246
250
 
247
251
  Log.log.debug("private=[#{rsa_private}]")
248
252
 
253
+ Log.log.debug("JWT assertion=[#{payload}]")
249
254
  assertion = JWT.encode(payload, rsa_private, 'RS256')
250
255
 
251
256
  Log.log.debug("assertion=[#{assertion}]")
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: asperalm
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.9.19
4
+ version: 0.9.20
5
5
  platform: ruby
6
6
  authors:
7
7
  - Laurent Martin
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-01-15 00:00:00.000000000 Z
11
+ date: 2019-01-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: xml-simple
@@ -188,17 +188,11 @@ files:
188
188
  - bin/asession
189
189
  - bin/aslmcli
190
190
  - bin/mlia
191
- - docs/Auth-registered-client.png
192
- - docs/Auth-simple-web.png
193
- - docs/Auth1.png
194
- - docs/Auth2.png
195
- - docs/Auth3.png
196
- - docs/AuthJWT.png
197
- - docs/AuthUsersKey.png
198
191
  - docs/diagrams.txt
199
192
  - docs/transfer_spec.html
200
193
  - examples/transfer.rb
201
194
  - lib/asperalm/VERSION
195
+ - lib/asperalm/api_detector.rb
202
196
  - lib/asperalm/ascmd.rb
203
197
  - lib/asperalm/ats_api.rb
204
198
  - lib/asperalm/cli/basic_auth_plugin.rb
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file