fog-aliyun 0.3.12 → 0.3.18

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c47b8a9b6a56d553d1b0161be29b2d2444671f41f083e8af34707bd1eadc1718
-  data.tar.gz: 17e83f9d11b7ae7b1802941ddb13427346bbd971b563d7ac04436f7d036d7ef7
+  metadata.gz: cff686947955adaead48c9bb70ec405a2ca807240ba53d7e86fe1da27b63ab4f
+  data.tar.gz: bec5610976802d1cefccbe9f8788943747cfe31059cf0d6b04d0ae589caace46
 SHA512:
-  metadata.gz: 52060f2e1a3881ccb72ccc6e5fbeddd988bd2820416c419a7409e714c6d4f08fb7b37f5f98ff1e8fc66b47c0f2aa5656349da06e939edfb0c78c4adba05bf1ef
-  data.tar.gz: 30bdcf81e3e78e2983545d391ef2062ebc49975bee90c9d93226f30a631b3eaa7627916b179d15c684b38211bbd2e9e78e84210a2471b8185c5c3d21fbfe3001
+  metadata.gz: 7bb3975226a2af2d130be85b64f057d61cab8564bd6ef9fc349080efa0008d79dff5149a3124142436bcf30ef893314e27d195b8706c9bc6c634d88c5452a33b
+  data.tar.gz: e21803702da802d2a3051c63428af92b0fb66c9f8a95f62c2c9c76a31ea80fa6ae0ee74081c9ee72d5c0570dae0f60c2764f8094d9692c39f8d93e1ade320ea0

data/CHANGELOG.md

@@ -0,0 +1,103 @@
+## 0.3.19 (Unreleased)
+## 0.3.18 (August 03, 2020)
+
+IMPROVEMENTS:
+
+- reconstruct perform test [GH-148](https://github.com/fog/fog-aliyun/pull/148)
+- Reconstruct fog-aliyun by using oss [GH-147](https://github.com/fog/fog-aliyun/pull/147)
+- reconstruct cover case test [GH-146](https://github.com/fog/fog-aliyun/pull/146)
+- reconstruct case test [GH-144](https://github.com/fog/fog-aliyun/pull/144)
+- reconstruct parts two of file [GH-143](https://github.com/fog/fog-aliyun/pull/143)
+- implement blobstore for cloud_controller_ng [GH-142](https://github.com/fog/fog-aliyun/pull/142)
+- reconstruct parts of file [GH-141](https://github.com/fog/fog-aliyun/pull/141)
+- reconstruct the files [GH-140](https://github.com/fog/fog-aliyun/pull/140)
+- reconstruct the directory [GH-139](https://github.com/fog/fog-aliyun/pull/139)
+- reconstruct the directories [GH-138](https://github.com/fog/fog-aliyun/pull/138)
+- improve files.get code [GH-137](https://github.com/fog/fog-aliyun/pull/137)
+- add testcase for testing head notexistfile [GH-136](https://github.com/fog/fog-aliyun/pull/136)
+- improve head_object using oss sdk [GH-135](https://github.com/fog/fog-aliyun/pull/135)
+
+BUG FIXES:
+
+- fix files all options problem [GH-149](https://github.com/fog/fog-aliyun/pull/149)
+
+## 0.3.17 (July 06, 2020)
+
+IMPROVEMENTS:
+- adater oss_sdk_log_path [GH-125](https://github.com/fog/fog-aliyun/pull/125)
+- update ruby sdk to 0.7.3 [GH-124](https://github.com/fog/fog-aliyun/pull/124)
+- adapter maxkeys conversion problem [GH-123](https://github.com/fog/fog-aliyun/pull/123)
+- [Enhance tests][Auth & Connectivity scenarios] Test that API cannot be accessed using incorrect credentials [GH-117](https://github.com/fog/fog-aliyun/pull/117)
+- [Enhance tests][Auth & Connectivity scenarios] Test that API can be accessed using valid credentials [GH-116](https://github.com/fog/fog-aliyun/pull/116)
+- adapter custom log environment variable [GH-114](https://github.com/fog/fog-aliyun/pull/114)
+- [Enhance tests][Buckets scenarios] (NEGATIVE TEST) Test that error is thrown when trying to access non-existing bucket [GH-110](https://github.com/fog/fog-aliyun/pull/110)
+- [Enhance tests][Buckets scenarios] (NEGATIVE TEST) Test that error is thrown when trying to create already existing bucket [GH-109](https://github.com/fog/fog-aliyun/pull/109)
+- [Enhance tests][Buckets scenarios] Test that it is possible to destroy a bucket [GH-108](https://github.com/fog/fog-aliyun/pull/108)
+- [Enhance tests][Buckets scenarios] Test that it is possible to create a new bucket [GH-107](https://github.com/fog/fog-aliyun/pull/107)
+- [Enhance tests][Buckets scenarios] Test that it is possible to list all buckets [GH-105](https://github.com/fog/fog-aliyun/pull/105)
+- [Enhance tests][Files & Directory scenarios] Test getting bucket when directory exists named with the same name as a bucket [GH-101](https://github.com/fog/fog-aliyun/pull/101)
+- [Enhance tests][Files & Directory scenarios] Test file copy operations [GH-100](https://github.com/fog/fog-aliyun/pull/100)
+- reset the last PR [GH-133](https://github.com/fog/fog-aliyun/pull/133)
+- improve put_object_with_body and head_object using sdk do_request [GH-131](https://github.com/fog/fog-aliyun/pull/131)
+
+BUG FIXES:
+- fix max key again [GH-128](https://github.com/fog/fog-aliyun/pull/128)
+- fix downloading object when pushing app twice [GH-127](https://github.com/fog/fog-aliyun/pull/127)
+- fix max key [GH-126](https://github.com/fog/fog-aliyun/pull/126)
+- fix max-keys conversion problem [GH-121](https://github.com/fog/fog-aliyun/pull/121)
+- fix @aliyun_oss_sdk_log_path is nil  [GH-132](https://github.com/fog/fog-aliyun/pull/132)
+
+## 0.3.16 (June 18, 2020)
+
+IMPROVEMENTS:
+- [Enhance tests][Files & Directory scenarios] Test get nested directories and files in nested directory [GH-98](https://github.com/fog/fog-aliyun/pull/98)
+- remove get_bucket_location and use ruby sdk to improve performance when uploading object [GH-97](https://github.com/fog/fog-aliyun/pull/97)
+- using bucket_exist to checking bucket [GH-95](https://github.com/fog/fog-aliyun/pull/95)
+- add change log [GH-94](https://github.com/fog/fog-aliyun/pull/94)
+
+BUG FIXES:
+- fix delete all of files bug when specifying a prefix [GH-102](https://github.com/fog/fog-aliyun/pull/102)
+
+## 0.3.15 (June 05, 2020)
+
+BUG FIXES:
+- change dependence ruby sdk to gems [GH-92](https://github.com/fog/fog-aliyun/pull/92)
+
+## 0.3.13 (June 02, 2020)
+
+IMPROVEMENTS:
+- using ruby sdk to delete object [GH-90](https://github.com/fog/fog-aliyun/pull/90)
+
+## 0.3.12 (May 28, 2020 )
+
+BUG FIXES:
+- add missing dependence [GH-88](https://github.com/fog/fog-aliyun/pull/88)
+
+## 0.3.11 (May 25, 2020)
+
+IMPROVEMENTS:
+- using oss ruby sdk to improve downloading object performance [GH-86](https://github.com/fog/fog-aliyun/pull/86)
+- Add performance tests [GH-85](https://github.com/fog/fog-aliyun/pull/85)
+- [Enhance tests][Entity operations]Add tests for each type of entity that validates the CURD operations [GH-84](https://github.com/fog/fog-aliyun/pull/84)
+- [Enhance tests][Auth & Connectivity scenarios] Test region is selected according to provider configuration [GH-83](https://github.com/fog/fog-aliyun/pull/83)
+- [Enhance tests][Files & Directory scenarios] test file listing using parameters such as prefix, marker, delimeter and maxKeys [GH-82](https://github.com/fog/fog-aliyun/pull/82)
+- [Enhance tests][Files & Directory scenarios]test directory listing using parameters such as prefix, marker, delimeter and maxKeys [GH-81](https://github.com/fog/fog-aliyun/pull/81)
+- [Enhance tests][Files & Directory scenarios]Test that it is possible to upload (write) large file (multi part upload) [GH-79](https://github.com/fog/fog-aliyun/pull/79)
+- upgrade deprecated code [GH-78](https://github.com/fog/fog-aliyun/pull/78)
+- improve fog/integration_spec [GH-77](https://github.com/fog/fog-aliyun/pull/77)
+- [Enhance tests][Files & Directory scenarios]Test that it is possible to upload (write) a file [GH-76](https://github.com/fog/fog-aliyun/pull/76)
+- upgrade deprecated code [GH-74](https://github.com/fog/fog-aliyun/pull/74)
+- support https scheme [GH-71](https://github.com/fog/fog-aliyun/pull/71)
+- [Enhance tests][Files & Directory scenarios]Test that it is possible to destroy a file/directory [GH-69](https://github.com/fog/fog-aliyun/pull/69)
+- improve fog/integration_spec [GH-68](https://github.com/fog/fog-aliyun/pull/68)
+- Implement basic integration tests [GH-66](https://github.com/fog/fog-aliyun/pull/66)
+
+## 0.3.10 (May 07, 2020)
+
+IMPROVEMENTS:
+- Set max limitation to 1000 when get objects [GH-64](https://github.com/fog/fog-aliyun/pull/64)
+
+## 0.3.9 (May 07, 2020)
+
+BUG FIXES:
+- diectories.get supports options to filter the specified objects [GH-62](https://github.com/fog/fog-aliyun/pull/62)

data/fog-aliyun.gemspec

@@ -28,11 +28,11 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'rubocop'
   spec.add_development_dependency 'simplecov'
   spec.add_development_dependency 'memory_profiler'
-  spec.add_development_dependency 'aliyun-sdk'
+  spec.add_development_dependency 'aliyun-sdk', '~> 0.7.3'
 
+  spec.add_dependency 'aliyun-sdk', '~> 0.7.3'
   spec.add_dependency 'fog-core'
   spec.add_dependency 'fog-json'
   spec.add_dependency 'ipaddress', '~> 0.8'
   spec.add_dependency 'xml-simple', '~> 1.1'
-  spec.add_dependency 'aliyun-sdk'
 end

data/lib/fog/aliyun/models/storage/file.rb

@@ -120,7 +120,8 @@ module Fog
                      directory_key + '/' + key
                    end
           if body.is_a?(::File)
-            data = service.put_object(object, body, options.merge(bucket: bucket_name)).data
+            service.put_object(object, body, options.merge(bucket: bucket_name))
+            data = service.head_object(object, bucket: bucket_name)
           elsif body.is_a?(String)
             data = service.put_object_with_body(object, body, options.merge(bucket: bucket_name)).data
           else

data/lib/fog/aliyun/models/storage/files.rb

@@ -36,14 +36,11 @@ module Fog
             # The bucket name can not contain '/', so if directory_key, return directory.
             if directory_key.include? '/'
               directory_key
+            elsif service.bucket_exists?(directory_key)
+              bucket_name = directory_key
+              directory_key = ''
             else
-              data = service.get_bucket(directory_key)
-              if data.class == Hash && data.key?('Code') && !data['Code'].nil? && !data['Code'].empty?
-                directory_key
-              else
-                bucket_name = directory_key
-                directory_key = ''
-              end
+              directory_key
             end
           end
           return bucket_name, directory_key
@@ -51,37 +48,24 @@ module Fog
 
         def all(options = {})
           requires :directory
-          bucket_name, directory_key = check_directory_key(directory.key)
-          remap_attributes(options, {
-              :delimiter  => 'delimiter',
-              :marker     => 'marker',
-              :max_keys   => 'max-keys',
-              :prefix     => 'prefix'
-          })
-          prefix_value = options['prefix']
-          prefix_value = directory_key + '/' + prefix if directory_key != '' && directory_key != '.' && !directory_key.nil?
-          options['prefix'] = prefix_value
-          options['bucket'] = bucket_name
-          files = service.list_objects(options)['Contents']
-          return if files.nil?
-          data = []
-          i = 0
-          files.each do |file|
-            next unless file['Key'][0][-1] != '/'
-            content_length = file['Size'][0].to_i
-            key = file['Key'][0]
-            lastModified = file['LastModified'][0]
-            last_modified = (Time.parse(lastModified).localtime if !lastModified.nil? && lastModified != '')
-            type = file['Type'][0]
-            data[i] = { content_length: content_length,
-                        key: key,
-                        last_modified: last_modified,
-                        etag: file['ETag'][0],
-                        object_type: type }
-            i += 1
+          options = {
+              'delimiter'   => delimiter,
+              'marker'      => marker,
+              'max-keys'    => max_keys.to_i,
+              'prefix'      => prefix
+          }.merge!(options)
+          options = options.reject {|key,value| value.nil? || value.to_s.empty?}
+          merge_attributes(options)
+          parent = directory.collection.get(
+              directory.key,
+              options
+          )
+          if parent
+            merge_attributes(parent.files.attributes)
+            load(parent.files.map {|file| file.attributes})
+          else
+            nil
           end
-
-          load(data)
         end
 
         alias each_file_this_page each
@@ -101,7 +85,7 @@ module Fog
           end
         end
 
-        def get(key)
+        def get(key, options = {}, &block)
           requires :directory
           bucket_name, directory_key = check_directory_key(directory.key)
           object = if directory_key == ''
@@ -110,24 +94,25 @@ module Fog
                      directory_key + '/' + key
                    end
           begin
-            data = service.get_object(object, nil, bucket: bucket_name)
-            lastModified = data['headers'][:last_modified]
+            data = service.get_object(object, options.merge({bucket: bucket_name}), &block)
+            headers = data.headers
+            lastModified = headers[:last_modified]
             last_modified = (Time.parse(lastModified).localtime if !lastModified.nil? && lastModified != '')
 
-            date = data['headers'][:date]
+            date = headers[:date]
             date = (Time.parse(date).localtime if !date.nil? && date != '')
             file_data = {
-                body: data[:body],
-                content_length: data['headers'][:content_length].to_i,
+                body: data.body,
+                content_length: headers[:content_length].to_i,
                 key: key,
                 last_modified: last_modified,
-                content_type: data['headers'][:content_type],
-                etag: data['headers'][:etag],
+                content_type: headers[:content_type],
+                etag: headers[:etag],
                 date: date,
-                connection: data['headers'][:connection],
-                accept_ranges: data['headers'][:accept_ranges],
-                server: data['headers'][:server],
-                object_type: data['headers'][:x_oss_object_type]
+                connection: headers[:connection],
+                accept_ranges: headers[:accept_ranges],
+                server: headers[:server],
+                object_type: headers[:x_oss_object_type]
             }
 
             new(file_data)

data/lib/fog/aliyun/requests/storage/copy_object.rb

@@ -25,8 +25,7 @@ module Fog
                   method: 'PUT',
                   path: target_object,
                   bucket: target_bucket,
-                  resource: resource,
-                  location: get_bucket_location(bucket))
+                  resource: resource)
         end
       end
     end

data/lib/fog/aliyun/requests/storage/delete_bucket.rb

@@ -15,8 +15,7 @@ module Fog
             expects: 204,
             method: 'DELETE',
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
         end
       end

data/lib/fog/aliyun/requests/storage/delete_container.rb

@@ -21,8 +21,7 @@ module Fog
             method: 'DELETE',
             path: object,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
         end
       end

data/lib/fog/aliyun/requests/storage/delete_object.rb

@@ -10,32 +10,15 @@ module Fog
         # * object<~String> - Name of object to delete
         #
         def delete_object(object, options = {})
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
-          resource = bucket + '/' + object
-          request(
-            expects: 204,
-            method: 'DELETE',
-            path: object,
-            bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
-          )
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
+          bucket = @oss_client.get_bucket(bucket_name)
+          bucket.delete_object(object)
         end
 
-        def abort_multipart_upload(bucket, object, endpoint, uploadid)
-          path = object + '?uploadId=' + uploadid
-          resource = bucket + '/' + path
-
-          ret = request(
-            expects: 204,
-            method: 'DELETE',
-            path: path,
-            bucket: bucket,
-            resource: resource,
-            endpoint: endpoint,
-            location: get_bucket_location(bucket)
-          )
+        def abort_multipart_upload(bucket_name, object, upload_id)
+          bucket = @oss_client.get_bucket(bucket_name)
+          bucket.abort_upload(upload_id, object)
         end
       end
     end

data/lib/fog/aliyun/requests/storage/get_bucket.rb

@@ -4,27 +4,32 @@ module Fog
   module Aliyun
     class Storage
       class Real
+        def bucket_exists?(bucket_name)
+          @oss_client.bucket_exists?(bucket_name)
+        end
+
         def get_bucket(bucket, options = {})
           prefix = options['prefix']
           marker = options['marker']
           # Set the GetBucket max limitation to 1000
           maxKeys = options['max-keys'] || 1000
-          maxKeys = [maxKeys, 1000].min
+          maxKeys = maxKeys.to_i
+          maxKeys = [maxKeys, 1000].min.to_s
           delimiter = options['delimiter']
           path = ''
           if prefix
             path += '/?prefix=' + prefix
             path += '&marker=' + marker if marker
-            path += '&max-keys=' + maxKeys.to_s if maxKeys
+            path += '&max-keys=' + maxKeys if maxKeys
             path += '&delimiter=' + delimiter if delimiter
 
           elsif marker
             path += '/?marker=' + marker
-            path += '&max-keys=' + maxKeys.to_s if maxKeys
+            path += '&max-keys=' + maxKeys if maxKeys
             path += '&delimiter=' + delimiter if delimiter
 
           elsif maxKeys
-            path += '/?max-keys=' + maxKeys.to_s
+            path += '/?max-keys=' + maxKeys
             path += '&delimiter=' + delimiter if delimiter
           elsif delimiter
             path += '/?delimiter=' + delimiter
@@ -36,33 +41,12 @@ module Fog
             method: 'GET',
             bucket: bucket,
             resource: resource,
-            location: get_bucket_location(bucket),
             path: path
           )
           xml = ret.data[:body]
           XmlSimple.xml_in(xml)
         end
 
-        def get_bucket_location(bucket)
-          attribute = '?location'
-          resource = bucket + '/' + attribute
-          ret = request(
-            expects: [200, 203, 403, 404],
-            method: 'GET',
-            path: attribute,
-            bucket: bucket,
-            resource: resource
-          )
-          # If there is an error, it will return a Hash with error code, host id and others
-          # If can not get a valid location, will return one using region
-          location = XmlSimple.xml_in(ret.data[:body])
-          if location.class == Hash && location.key?('HostId')
-            value = location['HostId']
-            location = value[0].split('.')[1]
-          end
-          location ||= 'oss-' + @aliyun_region_id
-        end
-
         def get_bucket_acl(bucket)
           attribute = '?acl'
           resource = bucket + '/' + attribute
@@ -71,8 +55,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['AccessControlList'][0]['Grant'][0]
         end
@@ -85,8 +68,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['CORSRule'][0] if ret.data[:status] != 404
         end
@@ -99,8 +81,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['Rule'][0] if ret.data[:status] != 404
         end
@@ -113,8 +94,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['LoggingEnabled'][0]['TargetPrefix']
         end
@@ -127,8 +107,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])
         end
@@ -141,8 +120,7 @@ module Fog
             method: 'GET',
             path: attribute,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body]) if ret.data[:status] != 404
         end

data/lib/fog/aliyun/requests/storage/get_container.rb

@@ -11,7 +11,9 @@ module Fog
           bucket ||= @aliyun_oss_bucket
 
           marker = options[:marker]
-          maxKeys = options[:maxKeys]
+          maxKeys = options[:maxKeys]||1000
+          maxKeys = maxKeys.to_i
+          maxKeys = [maxKeys,1000].min.to_s
           delimiter = '/'
 
           path = ''
@@ -38,7 +40,6 @@ module Fog
             path += '?delimiter=' + delimiter
           end
 
-          location = get_bucket_location(bucket)
           resource = bucket + '/'
           ret = request(
             expects: [200, 203, 400],

data/lib/fog/aliyun/requests/storage/get_containers.rb

@@ -19,7 +19,9 @@ module Fog
           bucket ||= @aliyun_oss_bucket
           prefix = options[:prefix]
           marker = options[:marker]
-          maxKeys = options[:maxKeys]
+          maxKeys = options[:maxKeys]||1000
+          maxKeys = maxKeys.to_i
+          maxKeys = [maxKeys,1000].min.to_s
           delimiter = '/'
 
           path = ''
@@ -42,18 +44,12 @@ module Fog
             path += '?delimiter=' + delimiter
           end
 
-          endpoint = options[:endpoint]
-          if endpoint.nil?
-            location = get_bucket_location(bucket)
-          end
           resource = bucket + '/'
           ret = request(
             expects: [200, 203, 400],
             method: 'GET',
             path: path,
             resource: resource,
-            endpoint: endpoint,
-            location: location,
             bucket: bucket
           )
           xml = ret.data[:body]

data/lib/fog/aliyun/requests/storage/get_object.rb

@@ -9,19 +9,29 @@ module Fog
         # ==== Parameters
         # * object<~String> - Name of object to look for
         #
-        def get_object(object, range = nil, options = {})
+        def get_object(object, options = {}, &block)
           options = options.reject { |_key, value| value.nil? }
           bucket_name = options[:bucket]
           bucket_name ||= @aliyun_oss_bucket
+          options.delete(:bucket)
           # Using OSS ruby SDK to fix performance issue
-          bucket = @oss_client.get_bucket(bucket_name)
-          body = Array.new
-          obj = bucket.get_object(object) do |chunk|
-            body << chunk
+          params = { :headers => {} }
+          params[:query] = options.delete('query') || {}
+
+          params[:headers].merge!(options)
+          if options['If-Modified-Since']
+            params[:headers]['If-Modified-Since'] = Fog::Time.at(options['If-Modified-Since'].to_i).to_date_header
+          end
+          if options['If-Unmodified-Since']
+            params[:headers]['If-Unmodified-Since'] = Fog::Time.at(options['If-Unmodified-Since'].to_i).to_date_header
+          end
+
+          if block_given?
+            params[:response_block] = Proc.new
           end
-          response = {}
-          obj.instance_variables.each {|var| response[var.to_s.delete("@")] = obj.instance_variable_get(var) }
-          response.merge({:body => body.join('')})
+
+          @oss_http.get({:bucket => bucket_name, :object => object}, params, &block)
+
         end
       end
     end

data/lib/fog/aliyun/requests/storage/get_object_http_url.rb

@@ -16,20 +16,20 @@ module Fog
         #   * body<~String> - url for object
         def get_object_http_url_public(object, expires, options = {})
           options = options.reject { |_key, value| value.nil? }
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
-          acl = get_bucket_acl(bucket)
-          location = get_bucket_location(bucket)
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
+          bucket = @oss_client.get_bucket(bucket_name)
+          acl = bucket.acl()
 
           if acl == 'private'
             expires_time = (Time.now.to_i + (expires.nil? ? 0 : expires.to_i)).to_s
-            resource = bucket + '/' + object
+            resource = bucket_name + '/' + object
             signature = sign('GET', expires_time, nil, resource)
-            'http://' + bucket + '.' + location + '.aliyuncs.com/' + object +
+            'http://' + bucket_name + '.' + @host + '/' + object +
               '?OSSAccessKeyId=' + @aliyun_accesskey_id + '&Expires=' + expires_time +
               '&Signature=' + URI.encode(signature, '/[^!*\'()\;?:@#&%=+$,{}[]<>`" ')
           elsif acl == 'public-read' || acl == 'public-read-write'
-            'http://' + bucket + '.' + location + '.aliyuncs.com/' + object
+            'http://' + bucket_name + '.' + @host + '/' + object
           else
             'acl is wrong with value:' + acl
           end

data/lib/fog/aliyun/requests/storage/get_object_https_url.rb

@@ -16,20 +16,20 @@ module Fog
         #   * body<~String> - url for object
         def get_object_https_url_public(object, expires, options = {})
           options = options.reject { |_key, value| value.nil? }
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
-          acl = get_bucket_acl(bucket)
-          location = get_bucket_location(bucket)
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
+          bucket = @oss_client.get_bucket(bucket_name)
+          acl = bucket.acl()
 
           if acl == 'private'
             expires_time = (Time.now.to_i + (expires.nil? ? 0 : expires.to_i)).to_s
-            resource = bucket + '/' + object
+            resource = bucket_name + '/' + object
             signature = sign('GET', expires_time, nil, resource)
-            'https://' + bucket + '.' + location + '.aliyuncs.com/' + object +
+            'https://' + bucket_name + '.' + @host + '/' + object +
               '?OSSAccessKeyId=' + @aliyun_accesskey_id + '&Expires=' + expires_time +
               '&Signature=' + URI.encode(signature, '/[^!*\'()\;?:@#&%=+$,{}[]<>`" ')
           elsif acl == 'public-read' || acl == 'public-read-write'
-            'https://' + bucket + '.' + location + '.aliyuncs.com/' + object
+            'https://' + bucket_name + '.' + @host + '/' + object
           else
             'acl is wrong with value:' + acl
           end

data/lib/fog/aliyun/requests/storage/head_object.rb

@@ -10,16 +10,15 @@ module Fog
         # * object<~String> - Name of object to look for
         #
         def head_object(object, options = {})
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
-          resource = bucket + '/' + object
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
+          resource = bucket_name + '/' + object
           ret = request(
             expects: [200, 404],
             method: 'HEAD',
             path: object,
-            bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            bucket: bucket_name,
+            resource: resource
           )
           ret
         end

data/lib/fog/aliyun/requests/storage/list_buckets.rb

@@ -7,7 +7,9 @@ module Fog
         def list_buckets(options = {})
           prefix = options[:prefix]
           marker = options[:marker]
-          maxKeys = options[:maxKeys]
+          maxKeys = options[:maxKeys] || 1000
+          maxKeys = maxKeys.to_i
+          maxKeys =[maxKeys,1000].min.to_s
 
           path = ''
           if prefix

data/lib/fog/aliyun/requests/storage/list_objects.rb

@@ -10,24 +10,25 @@ module Fog
           prefix = options['prefix']
           marker = options['marker']
           # Set the ListObjects max limitation to 1000
-          maxKeys = options['max-keys'] || 1000
-          maxKeys = [maxKeys, 1000].min
+          maxKeys = options[:maxKeys]||1000
+          maxKeys = maxKeys.to_i
+          maxKeys = [maxKeys,1000].min.to_s
           delimiter = options['delimiter']
 
           path = ''
           if prefix
             path += '/?prefix=' + prefix
             path += '&marker=' + marker if marker
-            path += '&max-keys=' + maxKeys.to_s if maxKeys
+            path += '&max-keys=' + maxKeys if maxKeys
             path += '&delimiter=' + delimiter if delimiter
 
           elsif marker
             path += '/?marker=' + marker
-            path += '&max-keys=' + maxKeys.to_s if maxKeys
+            path += '&max-keys=' + maxKeys if maxKeys
             path += '&delimiter=' + delimiter if delimiter
 
           elsif maxKeys
-            path += '/?max-keys=' + maxKeys.to_s
+            path += '/?max-keys=' + maxKeys
             path += '&delimiter=' + delimiter if delimiter
           elsif delimiter
             path += '/?delimiter=' + delimiter
@@ -45,9 +46,7 @@ module Fog
           XmlSimple.xml_in(xml)
         end
 
-        def list_multipart_uploads(bucket, location, _options = {})
-          location ||= get_bucket_location(bucket)
-
+        def list_multipart_uploads(bucket, _options = {})
           path = '?uploads'
           resource = bucket + '/' + path
 
@@ -56,15 +55,12 @@ module Fog
             method: 'GET',
             path: path,
             bucket: bucket,
-            resource: resource,
-            location: location
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['Upload']
         end
 
-        def list_parts(bucket, object, location, uploadid, _options = {})
-          location ||= get_bucket_location(bucket)
-
+        def list_parts(bucket, object, uploadid, _options = {})
           path = object + '?uploadId=' + uploadid
           resource = bucket + '/' + path
 
@@ -73,8 +69,7 @@ module Fog
             method: 'GET',
             path: path,
             bucket: bucket,
-            resource: resource,
-            location: location
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['Part']
         end

data/lib/fog/aliyun/requests/storage/put_container.rb

@@ -20,8 +20,7 @@ module Fog
             method: 'PUT',
             path: path,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
         end
       end

data/lib/fog/aliyun/requests/storage/put_object.rb

@@ -10,40 +10,29 @@ module Fog
         # * object<~String> - Name of object to look for
         #
         def put_object(object, file = nil, options = {})
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
-          return put_folder(bucket, object) if file.nil?
-
-          # put multiparts if object's size is over 100m
-          return put_multipart_object(bucket, object, file) if file.size > 104_857_600
-
-          body = file.read
-
-          resource = bucket + '/' + object
-          request(
-            expects: [200, 203],
-            method: 'PUT',
-            path: object,
-            bucket: bucket,
-            resource: resource,
-            body: body,
-            location: get_bucket_location(bucket)
-          )
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
+          bucket = @oss_client.get_bucket(bucket_name)
+          return bucket.put_object(object) if file.nil?
+          # With a single PUT operation you can upload objects up to 5 GB in size.
+          if file.size > 5_368_709_120
+            bucket.resumable_upload(object, file.path)
+          end
+          bucket.put_object(object, :file => file.path)
         end
 
         def put_object_with_body(object, body, options = {})
-          bucket = options[:bucket]
-          bucket ||= @aliyun_oss_bucket
+          bucket_name = options[:bucket]
+          bucket_name ||= @aliyun_oss_bucket
 
-          resource = bucket + '/' + object
+          resource = bucket_name + '/' + object
           request(
             expects: [200, 203],
             method: 'PUT',
             path: object,
-            bucket: bucket,
+            bucket: bucket_name,
             resource: resource,
-            body: body,
-            location: get_bucket_location(bucket)
+            body: body
           )
         end
 
@@ -55,27 +44,24 @@ module Fog
             method: 'PUT',
             path: path,
             bucket: bucket,
-            resource: resource,
-            location: get_bucket_location(bucket)
+            resource: resource
           )
         end
 
         def put_multipart_object(bucket, object, file)
-          location = get_bucket_location(bucket)
-
           # find the right uploadid
-          uploads = list_multipart_uploads(bucket, location)
+          uploads = list_multipart_uploads(bucket)
           upload = (uploads&.find { |tmpupload| tmpupload['Key'][0] == object })
 
           uploadedSize = 0
           start_partNumber = 1
           if !upload.nil?
             uploadId = upload['UploadId'][0]
-            parts = list_parts(bucket, object, location, uploadId)
+            parts = list_parts(bucket, object, uploadId)
             if !parts.nil? && !parts.empty?
               if parts[-1]['Size'][0].to_i != 5_242_880
                 # the part is the last one, if its size is over 5m, then finish this upload
-                complete_multipart_upload(bucket, object, location, uploadId)
+                complete_multipart_upload(bucket, object, uploadId)
                 return
               end
               uploadedSize = (parts[0]['Size'][0].to_i * (parts.size - 1)) + parts[-1]['Size'][0].to_i
@@ -83,11 +69,11 @@ module Fog
             end
           else
             # create upload ID
-            uploadId = initiate_multipart_upload(bucket, object, location)
+            uploadId = initiate_multipart_upload(bucket, object)
           end
 
           if file.size <= uploadedSize
-            complete_multipart_upload(bucket, object, location, uploadId)
+            complete_multipart_upload(bucket, object, uploadId)
             return
           end
 
@@ -96,14 +82,13 @@ module Fog
 
           for i in start_partNumber..end_partNumber
             body = file.read(5_242_880)
-            upload_part(bucket, object, location, i.to_s, uploadId, body)
+            upload_part(bucket, object, i.to_s, uploadId, body)
           end
 
-          complete_multipart_upload(bucket, object, location, uploadId)
+          complete_multipart_upload(bucket, object, uploadId)
         end
 
-        def initiate_multipart_upload(bucket, object, location)
-          location ||= get_bucket_location(bucket)
+        def initiate_multipart_upload(bucket, object)
           path = object + '?uploads'
           resource = bucket + '/' + path
           ret = request(
@@ -111,14 +96,12 @@ module Fog
             method: 'POST',
             path: path,
             bucket: bucket,
-            resource: resource,
-            location: location
+            resource: resource
           )
           XmlSimple.xml_in(ret.data[:body])['UploadId'][0]
         end
 
-        def upload_part(bucket, object, location, partNumber, uploadId, body)
-          location ||= get_bucket_location(bucket)
+        def upload_part(bucket, object, partNumber, uploadId, body)
           path = object + '?partNumber=' + partNumber + '&uploadId=' + uploadId
           resource = bucket + '/' + path
           request(
@@ -127,14 +110,12 @@ module Fog
             path: path,
             bucket: bucket,
             resource: resource,
-            body: body,
-            location: location
+            body: body
           )
         end
 
-        def complete_multipart_upload(bucket, object, location, uploadId)
-          location ||= get_bucket_location(bucket)
-          parts = list_parts(bucket, object, location, uploadId, options = {})
+        def complete_multipart_upload(bucket, object, uploadId)
+          parts = list_parts(bucket, object, uploadId, options = {})
           request_part = []
           return if parts.empty?
           for i in 0..(parts.size - 1)
@@ -151,7 +132,6 @@ module Fog
             path: path,
             bucket: bucket,
             resource: resource,
-            location: location,
             body: body
           )
         end

data/lib/fog/aliyun/storage.rb

@@ -19,7 +19,9 @@ module Fog
 
       recognizes :aliyun_oss_endpoint,
                  :aliyun_oss_location,
-                 :aliyun_region_id
+                 :aliyun_region_id,
+                 :aliyun_oss_sdk_log_path
+
       requires :aliyun_accesskey_id,
                :aliyun_accesskey_secret,
                :aliyun_oss_bucket
@@ -72,6 +74,7 @@ module Fog
         attr_reader :aliyun_oss_endpoint
         attr_reader :aliyun_region_id
         attr_reader :aliyun_oss_bucket
+        attr_reader :aliyun_oss_sdk_log_path
 
         def initialize(options = {})
           # initialize the parameters
@@ -80,7 +83,11 @@ module Fog
           @aliyun_accesskey_id = options[:aliyun_accesskey_id]
           @aliyun_accesskey_secret = options[:aliyun_accesskey_secret]
           @aliyun_oss_bucket = options[:aliyun_oss_bucket]
-
+          @aliyun_oss_sdk_log_path=options[:aliyun_oss_sdk_log_path]
+          if @aliyun_oss_sdk_log_path && !::File.exist?(@aliyun_oss_sdk_log_path)
+            `touch #{@aliyun_oss_sdk_log_path}`
+          end
+          ENV["ALIYUN_OSS_SDK_LOG_PATH"] = @aliyun_oss_sdk_log_path
           # check for the parameters
           missing_credentials = []
           missing_credentials << :aliyun_oss_bucket unless @aliyun_oss_bucket
@@ -106,6 +113,14 @@ module Fog
                                      :access_key_id => @aliyun_accesskey_id,
                                      :access_key_secret => @aliyun_accesskey_secret
           )
+
+          # initiate a aliyun oss ruby sdk config and using sdk http to invoke the OSS openapi
+          @oss_config = AliyunOssSdk::Config.new(
+              :endpoint => @aliyun_oss_endpoint,
+              :access_key_id => @aliyun_accesskey_id,
+              :access_key_secret => @aliyun_accesskey_secret
+          )
+          @oss_http = AliyunOssSdk::HTTP.new(@oss_config)
         end
 
         def reload
@@ -126,31 +141,14 @@ module Fog
           time = Time.new.utc
           date = time.strftime('%a, %d %b %Y %H:%M:%S GMT')
 
-          endpoint = params[:endpoint]
-          location = params[:location]
-          if endpoint
-            uri = URI.parse(endpoint)
-            host = uri.host
-            path = uri.path
-            port = uri.port
-            scheme = uri.scheme
-          elsif location
-            host = location + '.aliyuncs.com'
-          end
-
-          host ||= @host
-          path ||= @path
-          port ||= @port
-          scheme ||= @scheme
-
           bucket = params[:bucket]
           tmpHost = if bucket
-                      bucket + '.' + host
+                      bucket + '.' + @host
                     else
-                      host
+                      @host
                     end
 
-          @connection = Fog::Core::Connection.new("#{scheme}://#{tmpHost}", @persistent, @connection_options)
+          @connection = Fog::Core::Connection.new("#{@scheme}://#{tmpHost}", @persistent, @connection_options)
           contentType = params[:contentType]
 
           begin
@@ -166,7 +164,7 @@ module Fog
               'Authorization' => 'OSS ' + @aliyun_accesskey_id + ':' + signature,
               'Date' => date
             }.merge!(params[:headers] || {}),
-                                                        path: "#{path}/#{params[:path]}",
+                                                        path: "#{@path}/#{params[:path]}",
                                                         query: params[:query]))
           rescue Excon::Errors::HTTPStatusError => error
             raise case error

data/lib/fog/aliyun/version.rb

@@ -2,6 +2,6 @@
 
 module Fog
   module Aliyun
-    VERSION = '0.3.12'
+    VERSION = '0.3.18'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fog-aliyun
 version: !ruby/object:Gem::Version
-  version: 0.3.12
+  version: 0.3.18
 platform: ruby
 authors:
 - Qinsi Deng, Jianxun Li, Jane Han, Guimin He
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-05-28 00:00:00.000000000 Z
+date: 2020-08-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -132,16 +132,30 @@ dependencies:
   name: aliyun-sdk
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 0.7.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 0.7.3
+- !ruby/object:Gem::Dependency
+  name: aliyun-sdk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.7.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.7.3
 - !ruby/object:Gem::Dependency
   name: fog-core
   requirement: !ruby/object:Gem::Requirement
@@ -198,20 +212,6 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.1'
-- !ruby/object:Gem::Dependency
-  name: aliyun-sdk
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 description: As a FOG provider, fog-aliyun support aliyun OSS/ECS. It will support
   more aliyun services later.
 email:
@@ -228,6 +228,7 @@ files:
 - ".ruby-gemset"
 - ".ruby-version"
 - ".travis.yml"
+- CHANGELOG.md
 - CODE_OF_CONDUCT.md
 - Gemfile
 - LICENSE.txt