activestorage-openstack 1.1.1 → 1.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d33d6e1819fdb6cf66668b55885acb22ec5bfbd412de59c5227eaeb843a7b384
4
- data.tar.gz: 0215637536252f5aa653c1c8106f10f7fb446b976f1074fcf66411c29921dd19
3
+ metadata.gz: dbc6dd562ea11df43aac0d26b9f0d4c0aca0b8b9eef91cbcd7c570db7b9e9a35
4
+ data.tar.gz: f8320f16bdfc5bcaeb9d1b1f9683c32836fd53d9c336c5f88c25a441b4050a98
5
5
  SHA512:
6
- metadata.gz: a972f1b3f665be9ba244bbd7cf132e8dd4608a79f69fe4dd55f79893617aa7de6993c3898c92082f089e73346781472642047503841894b6a05d07b3a2ef09f9
7
- data.tar.gz: d922004c8f2a62b7ee089f8b20b9a5f62d9eba3f01dbfbca32c0973eba2f3504a6034f4711e87e1e32ee70d748c3e8684ba541614e72f402a6596c69cba85f35
6
+ metadata.gz: 8e45d6474289076b867a5bc64d98f7a0b7e802330884fc3da5154c9a973736130d7a926b01e2ed4387fd6e02bb1d0c2cac1ab00a65ba9381abc84938fe501b92
7
+ data.tar.gz: cfb46a535ec45c8f137c59e8c6f1c18b68a184156c68fb79d1abe86839b03ceaf8b40f8fc9a8c17d07db12f7edfc234823bb81c128651cbb8e5a74f14d257c58
data/README.md CHANGED
@@ -5,8 +5,6 @@ This rails plugin wraps the OpenStack Swift provider as an Active Storage servic
5
5
  [![Build Status](https://travis-ci.com/chaadow/activestorage-openstack.svg?branch=master)](https://travis-ci.com/chaadow/activestorage-openstack)
6
6
  [![Maintainability](https://api.codeclimate.com/v1/badges/4c070c101f86a579516f/maintainability)](https://codeclimate.com/github/chaadow/activestorage-openstack/maintainability)
7
7
  [![Test Coverage](https://api.codeclimate.com/v1/badges/4c070c101f86a579516f/test_coverage)](https://codeclimate.com/github/chaadow/activestorage-openstack/test_coverage)
8
-
9
-
10
8
  Starting from version `0.4`, this gem enforces version `>= 0.2.2` of `fog-openstack` which introduces breaking changes to the configuration keys in `config/storage.yml`. Please read the [MIGRATING from `0.1.x` to `0.2.x`](#migrating-from-fog-openstack-01x-to-02x) section
11
9
 
12
10
  **This gem currently supports `fog-openstack` version `~ 1.0`**
@@ -29,11 +27,15 @@ $ gem install activestorage-openstack
29
27
  ```
30
28
 
31
29
  ## Usage
32
- in `config/storage.yml`, in your Rails app, create an entry with the following keys:
30
+ in `config/storage.yml`, in your Rails app, you can create as many entries as
31
+ you wish. Here is an example with rails 6.1 new support for public containers
32
+
33
33
  ```yaml
34
- dev_openstack:
34
+
35
+ # Here you can have the common authentication credentials and config
36
+ # by defining a YAML anchor
37
+ default_config: &default_config
35
38
  service: OpenStack
36
- container: <container name> # Container name for your OpenStack provider
37
39
  credentials:
38
40
  openstack_auth_url: <auth url>
39
41
  openstack_username: <username>
@@ -42,19 +44,32 @@ dev_openstack:
42
44
  openstack_temp_url_key: <temp url key> # Mandatory, instructions below
43
45
  connection_options: # optional
44
46
  chunk_size: 2097152 # 2MBs - 1MB is the default
47
+
48
+ # starting from rails 6.1, you can have a public container generating public
49
+ # URLs
50
+ public_openstack:
51
+ <<: *default_config # we include the anchor defined above
52
+ public: true # important ; to tell rails that this is a public container
53
+ container: <container name> # Container name for your public OpenStack provider
54
+
55
+ # this config will generate signed/expired URLs (aka. private URLs)
56
+ private_openstack:
57
+ <<: *default_config # we include the anchor defined above
58
+ public: false # Optional in this case, because false is the default value
59
+ container: <container name> # Container name for your private OpenStack provider
45
60
  ```
46
61
 
47
- You can create as many entries as you would like for your different environments. For instance: `dev_openstack` for development, `test_openstack` for test environment, and `prod_openstack` for production. This way you can choose the appropriate container for each scenario.
62
+ You can create as many entries as you would like for your different environments. For instance: `public_openstack` for development, `test_openstack` for test environment, and `prod_openstack` for production. This way you can choose the appropriate container for each scenario.
48
63
 
49
64
  Then register the provider in your `config/{environment}.rb` (`config/development.rb`/`config/test.rb`/`config/production.rb`)
50
65
 
51
- For example, for the `dev_openstack` entry above, change the `config` variable in `config/development.rb` like the following:
66
+ For example, for the `public_openstack` entry above, change the `config` variable in `config/development.rb` like the following:
52
67
  ```ruby
53
68
  # Store uploaded files on the local file system (see config/storage.yml for options)
54
- config.active_storage.service = :dev_openstack
69
+ config.active_storage.service = :public_openstack
55
70
  ```
56
71
 
57
- ## Migrating from fog-openstack `0.1.x` to `0.2.x`
72
+ ## Migrating from fog-openstack `~> 0.1.x` to `>= 0.2.x`
58
73
 
59
74
  1- From your configuration file (`config/storage.yml`) change the `openstack_auth_uri` from :
60
75
  ```yaml
@@ -89,9 +104,16 @@ The next version of this plugin, will add a rails generator, or expose a method
89
104
 
90
105
  ## `ActiveStorage::Openstack`'s Content-Type handling
91
106
 
92
- OpenStack Swift handles the Content-Type of an object differently from other object storage services. You cannot overwrite the Content-Type via a temp URL. This gem will try very hard to set the right Content-Type for an object at object creation (either via server upload or direct upload) but this is wrong in some edge cases (e.g. you use direct upload and the browser provides a wrong mime type).
93
-
94
- For these edge cases `ActiveStorage::Blob::Identifiable` downloads the first 4K of a file, identifies the content type and saves the result in the database. For `ActiveStorage::Openstack` we also need to update the Content-Type of the object. This is done automatically with a little monkey patch.
107
+ OpenStack Swift handles the Content-Type of an object differently from other
108
+ object storage services.
109
+ You cannot overwrite the Content-Type via a temp URL. This gem will try very
110
+ hard to set the right Content-Type for an object at
111
+ object creation (either via server upload or direct upload) but this can be
112
+ wrong in some edge cases (e.g. you use direct upload and the browser provides
113
+ a wrong mime type).
114
+ Thankfully, by implementing the rails hook `#update_metadata`
115
+ this will update the object in your container by setting the new content type
116
+ after it's been uploaded.
95
117
 
96
118
  ## Testing
97
119
  First, run `bundle` to install the gem dependencies (both development and production)
@@ -1,18 +1,6 @@
1
1
  module ActiveStorage
2
2
  module Openstack
3
3
  class Railtie < ::Rails::Railtie
4
- initializer "active_storage_openstack.blob" do
5
- ActiveSupport.on_load(:active_storage_blob) do |klass|
6
- klass.after_commit do |blob|
7
- # overwrite content type if identification run and
8
- # the service responds to change_content_type
9
- if blob.identified? && !blob.content_type.blank? &&
10
- blob.service.respond_to?(:change_content_type)
11
- blob.service.change_content_type(blob.key, blob.content_type)
12
- end
13
- end
14
- end
15
- end
16
4
  end
17
5
  end
18
6
  end
@@ -1,5 +1,5 @@
1
1
  module ActiveStorage
2
2
  module Openstack
3
- VERSION = '1.1.1'
3
+ VERSION = '1.5.0'
4
4
  end
5
5
  end
@@ -1,159 +1,188 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require 'fog/openstack'
2
4
 
3
5
  module ActiveStorage
4
- class Service::OpenStackService < Service
5
- attr_reader :client, :container
6
-
7
- def initialize(container:, credentials:, connection_options: {})
8
- settings = credentials.reverse_merge(connection_options: connection_options)
9
-
10
- @client = Fog::OpenStack::Storage.new(settings)
11
- @container = Fog::OpenStack.escape(container)
12
- end
13
-
14
- def upload(key, io, checksum: nil, **)
15
- instrument :upload, key: key, checksum: checksum do
16
- params = { 'Content-Type' => guess_content_type(io) }
17
- params['ETag'] = convert_base64digest_to_hexdigest(checksum) if checksum
6
+ class Service
7
+ # ActiveStorage provider for OpenStack
8
+ class OpenStackService < Service
9
+ attr_reader :client, :container
10
+
11
+ def initialize(container:, credentials:, public: false, connection_options: {})
12
+ super()
13
+ settings = credentials.reverse_merge(connection_options: connection_options)
14
+
15
+ @public = public
16
+ @client = Fog::OpenStack::Storage.new(settings)
17
+ @container = Fog::OpenStack.escape(container)
18
+ end
18
19
 
19
- begin
20
- client.put_object(container, key, io, params)
21
- rescue Excon::Error::UnprocessableEntity
22
- raise ActiveStorage::IntegrityError
20
+ def upload(key, io, checksum: nil, disposition: nil, content_type: nil, filename: nil, **)
21
+ instrument :upload, key: key, checksum: checksum do
22
+ params = { 'Content-Type' => content_type || guess_content_type(io) }
23
+ params['ETag'] = convert_base64digest_to_hexdigest(checksum) if checksum
24
+ if disposition && filename
25
+ params['Content-Disposition'] =
26
+ content_disposition_with(type: disposition, filename: filename)
27
+ end
28
+
29
+ begin
30
+ client.put_object(container, key, io, params)
31
+ rescue Excon::Error::UnprocessableEntity
32
+ raise ActiveStorage::IntegrityError
33
+ end
23
34
  end
24
35
  end
25
- end
26
36
 
27
- def download(key, &block)
28
- if block_given?
29
- instrument :streaming_download, key: key do
30
- object_for(key, &block)
31
- end
32
- else
33
- instrument :download, key: key do
34
- object_for(key).body
37
+ def download(key, &block)
38
+ if block_given?
39
+ instrument :streaming_download, key: key do
40
+ object_for(key, &block)
41
+ end
42
+ else
43
+ instrument :download, key: key do
44
+ object_for(key).body
45
+ end
35
46
  end
47
+ rescue Fog::OpenStack::Storage::NotFound
48
+ raise ActiveStorage::FileNotFoundError if defined?(ActiveStorage::FileNotFoundError)
36
49
  end
37
- rescue Fog::OpenStack::Storage::NotFound
38
- raise ActiveStorage::FileNotFoundError
39
- end
40
50
 
41
- def download_chunk(key, range)
42
- instrument :download_chunk, key: key, range: range do
43
- chunk_buffer = []
51
+ def download_chunk(key, range)
52
+ instrument :download_chunk, key: key, range: range do
53
+ chunk_buffer = []
44
54
 
45
- object_for(key) do |chunk|
46
- chunk_buffer << chunk
47
- end
55
+ object_for(key) do |chunk|
56
+ chunk_buffer << chunk
57
+ end
48
58
 
49
- chunk_buffer.join[range]
50
- rescue Fog::OpenStack::Storage::NotFound
51
- raise ActiveStorage::FileNotFoundError
59
+ chunk_buffer.join[range]
60
+ rescue Fog::OpenStack::Storage::NotFound
61
+ raise ActiveStorage::FileNotFoundError if defined?(ActiveStorage::FileNotFoundError)
62
+ end
52
63
  end
53
- end
54
- def delete(key)
55
- instrument :delete, key: key do
56
- begin
64
+
65
+ def delete(key)
66
+ instrument :delete, key: key do
57
67
  client.delete_object(container, key)
58
68
  rescue Fog::OpenStack::Storage::NotFound
59
69
  false
60
70
  end
61
71
  end
62
- end
63
72
 
64
- def delete_prefixed(prefix)
65
- instrument :delete, prefix: prefix do
66
- directory = client.directories.get(container)
67
- filtered_files = client.files(directory: directory, prefix: prefix)
68
- filtered_files = filtered_files.map(&:key)
73
+ def delete_prefixed(prefix)
74
+ instrument :delete, prefix: prefix do
75
+ directory = client.directories.get(container)
76
+ filtered_files = client.files(directory: directory, prefix: prefix)
77
+ filtered_files = filtered_files.map(&:key)
69
78
 
70
- client.delete_multiple_objects(container, filtered_files)
79
+ client.delete_multiple_objects(container, filtered_files)
80
+ end
71
81
  end
72
- end
73
82
 
74
- def exist?(key)
75
- instrument :exist, key: key do |payload|
76
- begin
77
- answer = object_for(key)
83
+ def exist?(key)
84
+ instrument :exist, key: key do |payload|
85
+ answer = head_for(key)
78
86
  payload[:exist] = answer.present?
79
87
  rescue Fog::OpenStack::Storage::NotFound
80
88
  payload[:exist] = false
81
89
  end
82
90
  end
83
- end
84
91
 
85
- def url(key, expires_in:, disposition:, filename:, **)
86
- instrument :url, key: key do |payload|
87
- expire_at = unix_timestamp_expires_at(expires_in)
88
- generated_url =
89
- client.get_object_https_url(
92
+ def url(key, **options)
93
+ if ActiveStorage.version < Gem::Version.new('6.1-alpha')
94
+ instrument :url, key: key do |payload|
95
+ private_url(key, **options).tap do |generated_url|
96
+ payload[:url] = generated_url
97
+ end
98
+ end
99
+ else
100
+ super
101
+ end
102
+ end
103
+
104
+ def url_for_direct_upload(key, expires_in:, filename:, **)
105
+ instrument :url, key: key do |payload|
106
+ expire_at = unix_timestamp_expires_at(expires_in)
107
+ generated_url = client.create_temp_url(
90
108
  container,
91
109
  key,
92
110
  expire_at,
93
- filename: filename
94
- )
95
- generated_url += '&inline' if disposition.to_s != 'attachment'
96
- # unfortunately OpenStack Swift cannot overwrite the content type of an object via a temp url
97
- # so we just ignore the content_type argument here
98
- payload[:url] = generated_url
111
+ 'PUT',
112
+ port: 443,
113
+ filename: ActiveStorage::Filename.wrap(filename).sanitized.to_s,
114
+ scheme: 'https'
115
+ )
99
116
 
100
- generated_url
117
+ payload[:url] = generated_url
118
+
119
+ generated_url
120
+ end
101
121
  end
102
- end
103
122
 
104
- def url_for_direct_upload(key, expires_in:, **)
105
- instrument :url, key: key do |payload|
106
- expire_at = unix_timestamp_expires_at(expires_in)
107
- generated_url = client.create_temp_url(container,
108
- key,
109
- expire_at,
110
- 'PUT',
111
- port: 443,
112
- scheme: 'https')
123
+ def headers_for_direct_upload(_key, content_type:, checksum:, **)
124
+ {
125
+ 'Content-Type' => content_type,
126
+ 'ETag' => convert_base64digest_to_hexdigest(checksum)
127
+ }
128
+ end
129
+
130
+ def update_metadata(key, content_type:, disposition: nil, filename: nil, **)
131
+ instrument :update_metadata, key: key, content_type: content_type, disposition: disposition do
132
+ params = { 'Content-Type' => content_type }
133
+ if disposition && filename
134
+ params['Content-Disposition'] =
135
+ content_disposition_with(
136
+ type: disposition,
137
+ filename: ActiveStorage::Filename.wrap(filename)
138
+ )
139
+ end
140
+ client.post_object(container, key, params)
141
+ true
142
+ rescue Fog::OpenStack::Storage::NotFound
143
+ raise ActiveStorage::FileNotFoundError if defined?(ActiveStorage::FileNotFoundError)
144
+ end
145
+ end
113
146
 
114
- payload[:url] = generated_url
147
+ private
115
148
 
116
- generated_url
149
+ def head_for(key)
150
+ client.head_object(container, key)
117
151
  end
118
- end
119
152
 
120
- def headers_for_direct_upload(_key, content_type:, checksum:, **)
121
- {
122
- 'Content-Type' => content_type,
123
- 'ETag' => convert_base64digest_to_hexdigest(checksum)
124
- }
125
- end
153
+ def object_for(key, &block)
154
+ client.get_object(container, key, &block)
155
+ end
126
156
 
127
- # Non-standard method to change the content type of an existing object
128
- def change_content_type(key, content_type)
129
- client.post_object(container,
130
- key,
131
- 'Content-Type' => content_type)
132
- true
133
- rescue Fog::OpenStack::Storage::NotFound
134
- raise ActiveStorage::FileNotFoundError
135
- end
157
+ # ActiveStorage sends a `Digest::MD5.base64digest` checksum
158
+ # OpenStack expects a `Digest::MD5.hexdigest` ETag
159
+ def convert_base64digest_to_hexdigest(base64digest)
160
+ base64digest.unpack1('m0').unpack1('H*')
161
+ end
136
162
 
137
- private
163
+ def unix_timestamp_expires_at(seconds_from_now)
164
+ Time.current.advance(seconds: seconds_from_now).to_i
165
+ end
138
166
 
139
- def object_for(key, &block)
140
- client.get_object(container, key, &block)
141
- end
167
+ def guess_content_type(io)
168
+ Marcel::MimeType.for io,
169
+ name: io.try(:original_filename),
170
+ declared_type: io.try(:content_type)
171
+ end
142
172
 
143
- # ActiveStorage sends a `Digest::MD5.base64digest` checksum
144
- # OpenStack expects a `Digest::MD5.hexdigest` ETag
145
- def convert_base64digest_to_hexdigest(base64digest)
146
- base64digest.unpack('m0').first.unpack('H*').first
147
- end
173
+ def private_url(key, expires_in:, filename:, disposition:, **)
174
+ expire_at = unix_timestamp_expires_at(expires_in)
175
+ generated_url = client.get_object_https_url(container, key, expire_at, filename: filename.sanitized)
176
+ generated_url += '&inline' if disposition.to_s != 'attachment'
177
+ # unfortunately OpenStack Swift cannot overwrite the content type of an object via a temp url
178
+ # so we just ignore the content_type argument here
148
179
 
149
- def unix_timestamp_expires_at(seconds_from_now)
150
- Time.current.advance(seconds: seconds_from_now).to_i
151
- end
180
+ generated_url
181
+ end
152
182
 
153
- def guess_content_type(io)
154
- Marcel::MimeType.for io,
155
- name: io.try(:original_filename),
156
- declared_type: io.try(:content_type)
183
+ def public_url(key, **)
184
+ client.public_url(container, key)
185
+ end
157
186
  end
158
187
  end
159
188
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: activestorage-openstack
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.1
4
+ version: 1.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Chedli Bourguiba
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-10-25 00:00:00.000000000 Z
11
+ date: 2020-12-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fog-openstack
@@ -39,33 +39,33 @@ dependencies:
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
41
  - !ruby/object:Gem::Dependency
42
- name: mime-types
42
+ name: rails
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
45
  - - ">="
46
46
  - !ruby/object:Gem::Version
47
- version: '0'
47
+ version: 5.2.2
48
48
  type: :runtime
49
49
  prerelease: false
50
50
  version_requirements: !ruby/object:Gem::Requirement
51
51
  requirements:
52
52
  - - ">="
53
53
  - !ruby/object:Gem::Version
54
- version: '0'
54
+ version: 5.2.2
55
55
  - !ruby/object:Gem::Dependency
56
- name: rails
56
+ name: appraisal
57
57
  requirement: !ruby/object:Gem::Requirement
58
58
  requirements:
59
59
  - - ">="
60
60
  - !ruby/object:Gem::Version
61
- version: 5.2.2
62
- type: :runtime
61
+ version: '0'
62
+ type: :development
63
63
  prerelease: false
64
64
  version_requirements: !ruby/object:Gem::Requirement
65
65
  requirements:
66
66
  - - ">="
67
67
  - !ruby/object:Gem::Version
68
- version: 5.2.2
68
+ version: '0'
69
69
  - !ruby/object:Gem::Dependency
70
70
  name: sqlite3
71
71
  requirement: !ruby/object:Gem::Requirement
@@ -142,7 +142,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
142
142
  - !ruby/object:Gem::Version
143
143
  version: 1.8.11
144
144
  requirements: []
145
- rubygems_version: 3.0.3
145
+ rubygems_version: 3.1.3
146
146
  signing_key:
147
147
  specification_version: 4
148
148
  summary: ActiveStorage wrapper for OpenStack Storage