shrine 3.1.0 → 3.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +82 -0
- data/README.md +11 -4
- data/doc/advantages.md +4 -4
- data/doc/attacher.md +2 -2
- data/doc/carrierwave.md +24 -12
- data/doc/changing_derivatives.md +1 -1
- data/doc/changing_location.md +6 -5
- data/doc/design.md +134 -85
- data/doc/direct_s3.md +26 -0
- data/doc/external/articles.md +57 -45
- data/doc/external/extensions.md +41 -35
- data/doc/external/misc.md +23 -8
- data/doc/getting_started.md +156 -85
- data/doc/metadata.md +80 -44
- data/doc/multiple_files.md +1 -1
- data/doc/paperclip.md +28 -9
- data/doc/plugins/add_metadata.md +112 -35
- data/doc/plugins/atomic_helpers.md +41 -3
- data/doc/plugins/backgrounding.md +12 -2
- data/doc/plugins/column.md +36 -7
- data/doc/plugins/default_url.md +6 -3
- data/doc/plugins/derivatives.md +83 -44
- data/doc/plugins/download_endpoint.md +5 -5
- data/doc/plugins/dynamic_storage.md +1 -1
- data/doc/plugins/entity.md +12 -4
- data/doc/plugins/form_assign.md +5 -5
- data/doc/plugins/included.md +25 -5
- data/doc/plugins/infer_extension.md +9 -0
- data/doc/plugins/instrumentation.md +1 -1
- data/doc/plugins/metadata_attributes.md +1 -0
- data/doc/plugins/mirroring.md +1 -1
- data/doc/plugins/model.md +8 -3
- data/doc/plugins/persistence.md +10 -1
- data/doc/plugins/remote_url.md +6 -1
- data/doc/plugins/remove_invalid.md +9 -1
- data/doc/plugins/sequel.md +1 -1
- data/doc/plugins/store_dimensions.md +10 -0
- data/doc/plugins/type_predicates.md +96 -0
- data/doc/plugins/upload_endpoint.md +1 -1
- data/doc/plugins/upload_options.md +1 -1
- data/doc/plugins/url_options.md +4 -4
- data/doc/plugins/validation.md +14 -4
- data/doc/plugins/versions.md +7 -7
- data/doc/processing.md +287 -123
- data/doc/refile.md +9 -9
- data/doc/release_notes/2.8.0.md +1 -1
- data/doc/release_notes/3.0.0.md +1 -1
- data/doc/release_notes/3.2.0.md +96 -0
- data/doc/release_notes/3.2.1.md +31 -0
- data/doc/release_notes/3.2.2.md +14 -0
- data/doc/release_notes/3.3.0.md +105 -0
- data/doc/release_notes/3.4.0.md +35 -0
- data/doc/securing_uploads.md +2 -2
- data/doc/storage/memory.md +19 -0
- data/doc/storage/s3.md +104 -77
- data/doc/testing.md +12 -2
- data/doc/upgrading_to_3.md +99 -53
- data/lib/shrine.rb +9 -8
- data/lib/shrine/attacher.rb +20 -10
- data/lib/shrine/attachment.rb +2 -2
- data/lib/shrine/plugins.rb +22 -0
- data/lib/shrine/plugins/activerecord.rb +3 -3
- data/lib/shrine/plugins/add_metadata.rb +20 -5
- data/lib/shrine/plugins/backgrounding.rb +2 -2
- data/lib/shrine/plugins/default_url.rb +1 -1
- data/lib/shrine/plugins/derivation_endpoint.rb +13 -8
- data/lib/shrine/plugins/derivatives.rb +59 -30
- data/lib/shrine/plugins/determine_mime_type.rb +5 -3
- data/lib/shrine/plugins/entity.rb +12 -11
- data/lib/shrine/plugins/instrumentation.rb +12 -18
- data/lib/shrine/plugins/mirroring.rb +8 -8
- data/lib/shrine/plugins/model.rb +3 -3
- data/lib/shrine/plugins/presign_endpoint.rb +16 -4
- data/lib/shrine/plugins/pretty_location.rb +1 -1
- data/lib/shrine/plugins/processing.rb +1 -1
- data/lib/shrine/plugins/refresh_metadata.rb +2 -2
- data/lib/shrine/plugins/remote_url.rb +3 -3
- data/lib/shrine/plugins/remove_attachment.rb +5 -0
- data/lib/shrine/plugins/remove_invalid.rb +10 -5
- data/lib/shrine/plugins/sequel.rb +1 -1
- data/lib/shrine/plugins/store_dimensions.rb +4 -2
- data/lib/shrine/plugins/type_predicates.rb +113 -0
- data/lib/shrine/plugins/upload_endpoint.rb +10 -5
- data/lib/shrine/plugins/upload_options.rb +2 -2
- data/lib/shrine/plugins/url_options.rb +2 -2
- data/lib/shrine/plugins/validation.rb +9 -7
- data/lib/shrine/storage/linter.rb +4 -4
- data/lib/shrine/storage/memory.rb +5 -3
- data/lib/shrine/storage/s3.rb +117 -38
- data/lib/shrine/version.rb +1 -1
- data/shrine.gemspec +8 -8
- metadata +42 -34
data/doc/refile.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
---
|
2
|
-
title:
|
2
|
+
title: Upgrading from Refile
|
3
3
|
---
|
4
4
|
|
5
5
|
This guide is aimed at helping Refile users transition to Shrine, and it consists
|
@@ -110,13 +110,6 @@ into separate columns.
|
|
110
110
|
Shrine provides on-the-fly processing via the
|
111
111
|
[`derivation_endpoint`][derivation_endpoint] plugin:
|
112
112
|
|
113
|
-
```rb
|
114
|
-
# config/routes.rb (Rails)
|
115
|
-
Rails.application.routes.draw do
|
116
|
-
# ...
|
117
|
-
mount ImageUploader.derivation_endpoint => "/derivations/image"
|
118
|
-
end
|
119
|
-
```
|
120
113
|
```rb
|
121
114
|
require "image_processing/mini_magick"
|
122
115
|
|
@@ -132,8 +125,15 @@ class ImageUploader < Shrine
|
|
132
125
|
end
|
133
126
|
end
|
134
127
|
```
|
128
|
+
```rb
|
129
|
+
# config/routes.rb (Rails)
|
130
|
+
Rails.application.routes.draw do
|
131
|
+
# ...
|
132
|
+
mount ImageUploader.derivation_endpoint => "/derivations/image"
|
133
|
+
end
|
134
|
+
```
|
135
135
|
|
136
|
-
Shrine also support processing
|
136
|
+
Shrine also support eager processing using the [`derivatives`][derivatives]
|
137
137
|
plugin.
|
138
138
|
|
139
139
|
### Validation
|
data/doc/release_notes/2.8.0.md
CHANGED
data/doc/release_notes/3.0.0.md
CHANGED
@@ -366,7 +366,7 @@ end
|
|
366
366
|
```
|
367
367
|
```rb
|
368
368
|
attacher = photo.image_attacher
|
369
|
-
attacher.form_assign("image" => file, "title" => "...", "description" => "...")
|
369
|
+
attacher.form_assign({ "image" => file, "title" => "...", "description" => "..." })
|
370
370
|
attacher.file #=> #<Shrine::UploadedFile id="..." storage=:cache ...>
|
371
371
|
```
|
372
372
|
|
@@ -0,0 +1,96 @@
|
|
1
|
+
---
|
2
|
+
title: Shrine 3.2.0
|
3
|
+
---
|
4
|
+
|
5
|
+
## New features
|
6
|
+
|
7
|
+
* The `type_predicates` plugin has been added, which adds convenient predicate
|
8
|
+
methods to `Shrine::UploadedFile` based on the MIME type.
|
9
|
+
|
10
|
+
```rb
|
11
|
+
# Gemfile
|
12
|
+
gem "mini_mime" # default dependency of type_predicates
|
13
|
+
```
|
14
|
+
```rb
|
15
|
+
Shrine.plugin :type_predicates
|
16
|
+
```
|
17
|
+
|
18
|
+
The plugin adds four predicate methods based on the general type of the file:
|
19
|
+
|
20
|
+
```rb
|
21
|
+
file.image? # returns true for any "image/*" MIME type
|
22
|
+
file.video? # returns true for any "video/*" MIME type
|
23
|
+
file.audio? # returns true for any "audio/*" MIME type
|
24
|
+
file.text? # returns true for any "text/*" MIME type
|
25
|
+
```
|
26
|
+
|
27
|
+
You can also check for specific MIME type using the extension name:
|
28
|
+
|
29
|
+
```rb
|
30
|
+
file.type?(:jpg) # returns true if MIME type is "image/jpeg"
|
31
|
+
file.type?(:svg) # returns true if MIME type is "image/svg+xml"
|
32
|
+
file.type?(:mov) # returns true if MIME type is "video/quicktime"
|
33
|
+
file.type?(:ppt) # returns true if MIME type is "application/vnd.ms-powerpoint"
|
34
|
+
...
|
35
|
+
```
|
36
|
+
|
37
|
+
For convenience, you can create predicate methods for specific file types:
|
38
|
+
|
39
|
+
```rb
|
40
|
+
Shrine.plugin :type_predicates, methods: %i[jpg svg mov ppt]
|
41
|
+
```
|
42
|
+
```rb
|
43
|
+
file.jpg? # returns true if MIME type is "image/jpeg"
|
44
|
+
file.svg? # returns true if MIME type is "image/svg+xml"
|
45
|
+
file.mov? # returns true if MIME type is "video/quicktime"
|
46
|
+
file.ppt? # returns true if MIME type is "application/vnd.ms-powerpoint"
|
47
|
+
```
|
48
|
+
|
49
|
+
* The `#add_metadata` method has been added to the `add_metadata` plugin for
|
50
|
+
adding new metadata to an existing file/attachment.
|
51
|
+
|
52
|
+
```rb
|
53
|
+
attacher.file.metadata #=> { ... }
|
54
|
+
attacher.add_metadata("foo" => "bar")
|
55
|
+
attacher.file.metadata #=> { ..., "foo" => "bar" }
|
56
|
+
```
|
57
|
+
|
58
|
+
## Other improvements
|
59
|
+
|
60
|
+
* The `remove_invalid` plugin now works correctly with `derivatives` plugin.
|
61
|
+
|
62
|
+
* The `remove_invalid` plugin is now also activated when `Attacher#validate`
|
63
|
+
is called manually.
|
64
|
+
|
65
|
+
* The current attached file data can now be assigned back to the attachment
|
66
|
+
attribute, and this operation will be a no-op.
|
67
|
+
|
68
|
+
```rb
|
69
|
+
photo.image #=> #<Shrine::UploadedFile id="foo" storage=:store metadata={...}>
|
70
|
+
photo.image = { "id" => "foo", "storage" => "store", "metadata" => { ... } } # no-op
|
71
|
+
```
|
72
|
+
|
73
|
+
This allows treating the attachment attribute as a persistent attribute,
|
74
|
+
where the current value can be assigned back on record updates.
|
75
|
+
|
76
|
+
* When promoting derivatives, the `:derivative` parameter value was being
|
77
|
+
passed to the uploader as an array. This has been fixed, and the value is now
|
78
|
+
the same as when uploading derivatives directly to permanent storage.
|
79
|
+
|
80
|
+
* The `derivatives` plugin now includes additional `:io` and `:attacher` values
|
81
|
+
in the instrumentation event payload.
|
82
|
+
|
83
|
+
## Backwards compatibility
|
84
|
+
|
85
|
+
* The `validation` plugin now runs validations on `Attacher#attach` and
|
86
|
+
`Attacher#attach_cached`. If you were using `Attacher#change` directly and
|
87
|
+
expecting the validations to be run automatically, you will need to update
|
88
|
+
your code.
|
89
|
+
|
90
|
+
* If you were updating the cached file metadata via file data assignment, this
|
91
|
+
will no longer work.
|
92
|
+
|
93
|
+
```rb
|
94
|
+
photo.image #=> #<Shrine::UploadedFile id="foo" storage=:cache metadata={...}>
|
95
|
+
photo.image = { "id" => "foo", "storage" => "cache", "metadata" => { ... } } # no-op
|
96
|
+
```
|
@@ -0,0 +1,31 @@
|
|
1
|
+
---
|
2
|
+
title: Shrine 3.2.1
|
3
|
+
---
|
4
|
+
|
5
|
+
## Ruby 2.7 compatibility
|
6
|
+
|
7
|
+
* Shrine doesn't trigger [Ruby 2.7 warnings for separation of positional and
|
8
|
+
keyword arguments][kwargs] anymore.
|
9
|
+
|
10
|
+
* Down 5.1.0 has been released, which resolves warnings and a `FrozenError`
|
11
|
+
exception on Ruby 2.7. Shrine now requires at least this version of Down.
|
12
|
+
|
13
|
+
If you're using `Down::Http`, make sure you're using http.rb 4.3.0 or newer.
|
14
|
+
|
15
|
+
* ImageProcessing 1.10.3 gem has been released which resolves Ruby 2.7 warnings
|
16
|
+
as well. If you're using it for image processing, make sure to upgrade to
|
17
|
+
this version:
|
18
|
+
|
19
|
+
```rb
|
20
|
+
gem "image_processing", ">= 1.10.3", "< 2"
|
21
|
+
```
|
22
|
+
|
23
|
+
## Rack 2.1 compatibility
|
24
|
+
|
25
|
+
* The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1 or newer.
|
26
|
+
|
27
|
+
## Other improvements
|
28
|
+
|
29
|
+
* The `S3#open` method now handles empty S3 objects.
|
30
|
+
|
31
|
+
[kwargs]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/
|
@@ -0,0 +1,14 @@
|
|
1
|
+
---
|
2
|
+
title: Shrine 3.2.2
|
3
|
+
---
|
4
|
+
|
5
|
+
## Bug fixes
|
6
|
+
|
7
|
+
* aws-sdk-core 3.104.0 introduced a backwards incompatible changes that caused
|
8
|
+
`Shrine::Storage::S3#open` to start raising an exception.
|
9
|
+
|
10
|
+
```
|
11
|
+
NoMethodError: undefined method `bytesize' for #<Array:0x000000000a721be0>
|
12
|
+
```
|
13
|
+
|
14
|
+
This has now been fixed.
|
@@ -0,0 +1,105 @@
|
|
1
|
+
---
|
2
|
+
title: Shrine 3.3.0
|
3
|
+
---
|
4
|
+
|
5
|
+
## New features
|
6
|
+
|
7
|
+
* The `:create_on_promote` option has been added to the `derivatives` plugin
|
8
|
+
for automatically creating derivatives after the attached cached file is
|
9
|
+
promoted to permanent storage.
|
10
|
+
|
11
|
+
```rb
|
12
|
+
Shrine.plugin :derivatives, create_on_promote: true
|
13
|
+
```
|
14
|
+
|
15
|
+
* The `:auto_extraction` option has been added to the `store_dimensions` plugin
|
16
|
+
for skipping automatically extracting dimensions on upload.
|
17
|
+
|
18
|
+
```rb
|
19
|
+
Shrine.plugin :store_dimensions, auto_extraction: false
|
20
|
+
```
|
21
|
+
|
22
|
+
* The `:skip_nil` option has been added to the `add_metadata` plugin for
|
23
|
+
excluding metadata keys whose values are nil.
|
24
|
+
|
25
|
+
```rb
|
26
|
+
class PdfUploader < Shrine
|
27
|
+
add_metadata :pages, skip_nil: true do |io|
|
28
|
+
if is_pdf?(io)
|
29
|
+
reader = PDF::Reader.new(io)
|
30
|
+
reader.page_count
|
31
|
+
else
|
32
|
+
# If this is not a PDF, then the pages metadata will not be stored
|
33
|
+
nil
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
37
|
+
```
|
38
|
+
|
39
|
+
* The `:download` option has been added to derivatives processors in
|
40
|
+
`derivatives` plugin for skipping converting the source IO object into a
|
41
|
+
file. This can be used to avoid a potentially expensive download/copy when
|
42
|
+
the derivatives processor doesn't need the file.
|
43
|
+
|
44
|
+
```rb
|
45
|
+
Attacher.derivatives :my_processor, download: false do |source|
|
46
|
+
source #=> Could be File, Shrine::UploadedFile, or other IO-like object
|
47
|
+
shrine_class.with_file(source) do |file|
|
48
|
+
# can force download/copy if necessary with `with_file`,
|
49
|
+
end
|
50
|
+
end
|
51
|
+
```
|
52
|
+
|
53
|
+
## Bug fixes
|
54
|
+
|
55
|
+
* The `upload_endpoint` now handles calling `Shrine.upload_response` method
|
56
|
+
from a Rails controller.
|
57
|
+
|
58
|
+
* The `derivation_endpoint` plugin now applies the `version` query parameter
|
59
|
+
to the derivation when creating the response.
|
60
|
+
|
61
|
+
## Other improvements
|
62
|
+
|
63
|
+
* The new `Aws:S3::EncryptionV2::Client` is now supported by the S3 storage for
|
64
|
+
client-side encryption.
|
65
|
+
|
66
|
+
* The `derivation_endpoint` now reduces the possibility of timing attacks by
|
67
|
+
comparing URL signatures in constant time using `Rack::Utils.secure_compare`.
|
68
|
+
|
69
|
+
* The `derivatives` plugin now copies non-file source IO objects to disk before
|
70
|
+
passing them to the processor. This is consistent with how a
|
71
|
+
`Shrine::UploadedFile` object is downloaded to disk.
|
72
|
+
|
73
|
+
* The `sequel` and `activerecord` plugins now call `Attacher#reload` when
|
74
|
+
reloading the model, which reloads the attached files but keeps other
|
75
|
+
attacher state.
|
76
|
+
|
77
|
+
* The `derivatives` plugin doesn't download the attached file anymore if
|
78
|
+
attempting to process derivatives when no derivatives processor was defined.
|
79
|
+
|
80
|
+
* The `mirroring` plugin now forwards attacher options when uploading to mirror
|
81
|
+
storages.
|
82
|
+
|
83
|
+
* The `presign_endpoint` plugin now handles the `OPTIONS` HTTP verb, which
|
84
|
+
newer versions of Uppy are requesting.
|
85
|
+
|
86
|
+
* `Shrine::Storage::Memory#open` now always returns a `StringIO` in the file
|
87
|
+
content's original encoding, instead of the encoding set by
|
88
|
+
`Encoding.default_internal`. This works around a [bug][ruby-lang #16497]
|
89
|
+
in `StringIO` introduced in Ruby 2.7.0.
|
90
|
+
|
91
|
+
* The `remove_attachment` plugin now deletes the removed file if a new file was
|
92
|
+
attached right after removal.
|
93
|
+
|
94
|
+
## Backwards compatibility
|
95
|
+
|
96
|
+
* If you were passing a non-file IO object to the derivatives processor, Shrine
|
97
|
+
will now convert it into a file beforehand. If you're currently doing this
|
98
|
+
and are converting the IO object into a file inside the processor, you can
|
99
|
+
now remove the conversion code to avoid doubling the amount of disk writes.
|
100
|
+
|
101
|
+
* When reloading a Sequel/ActiveRecord model, any attacher state other than
|
102
|
+
uploaded files will now be retained after the reload. If you were relying on
|
103
|
+
all the attacher state being re-initialized, you'll need to update your code.
|
104
|
+
|
105
|
+
[ruby-lang #16497]: https://bugs.ruby-lang.org/issues/16497
|
@@ -0,0 +1,35 @@
|
|
1
|
+
---
|
2
|
+
title: Shrine 3.4.0
|
3
|
+
---
|
4
|
+
|
5
|
+
* Passing attacher options to `Shrine.Attachment` method now works on Ruby 3.0.
|
6
|
+
|
7
|
+
* Defining validation errors as an array of I18n key and options in
|
8
|
+
`activerecord` plugin now works on Ruby 3.0.
|
9
|
+
|
10
|
+
* The `:fastimage` MIME type analyzer now correctly detects SVGs as
|
11
|
+
`image/svg+html` in the `determine_mime_type` plugin.
|
12
|
+
|
13
|
+
* The `Shrine::Attacher#read` method provided by the `entity` plugin is now
|
14
|
+
public. This is consistent with `Shrine::Attacher#write` from `model` plugin
|
15
|
+
being public as well.
|
16
|
+
|
17
|
+
* The `Shrine::Attacher#reload` method now resets attachment's dirty state.
|
18
|
+
This means that for a model whose `Attacher#changed?` returns `true`, calling
|
19
|
+
`#reload` on the model will make `Attacher#changed?` return `false`. This was
|
20
|
+
the behaviour before Shrine 3.3.0.
|
21
|
+
|
22
|
+
```rb
|
23
|
+
# before
|
24
|
+
model.file_attacher.changed? #=> true
|
25
|
+
model.reload
|
26
|
+
model.file_attacher.changed? #=> true
|
27
|
+
|
28
|
+
# after
|
29
|
+
model.file_attacher.changed? #=> true
|
30
|
+
model.reload
|
31
|
+
model.file_attacher.changed? #=> false
|
32
|
+
```
|
33
|
+
|
34
|
+
* Calling `#reload` on the model will not initialize a `Shrine::Attacher`
|
35
|
+
instance anymore if one hasn't previously been initialized.
|
data/doc/securing_uploads.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
---
|
2
2
|
id: securing-uploads
|
3
|
-
title: Securing
|
3
|
+
title: Securing Uploads
|
4
4
|
---
|
5
5
|
|
6
6
|
Shrine does a lot to make your file uploads secure, but there are still a lot
|
@@ -63,7 +63,7 @@ in the `:max_size` option to reject files that are larger than the specified
|
|
63
63
|
limit:
|
64
64
|
|
65
65
|
```rb
|
66
|
-
plugin :upload_endpoint, max_size: 100*1024*1024 #
|
66
|
+
plugin :upload_endpoint, max_size: 100*1024*1024 # 100 MB
|
67
67
|
```
|
68
68
|
|
69
69
|
If you're doing direct uploads to Amazon S3 using the `presign_endpoint`
|
@@ -0,0 +1,19 @@
|
|
1
|
+
---
|
2
|
+
title: Memory
|
3
|
+
---
|
4
|
+
|
5
|
+
The Memory storage stores uploaded files in memory, which is suitable for
|
6
|
+
testing.
|
7
|
+
|
8
|
+
```rb
|
9
|
+
Shrine.storages[:store] = Shrine::Storage::Memory.new
|
10
|
+
```
|
11
|
+
|
12
|
+
By default, each storage instance uses a new Hash object for storing files,
|
13
|
+
but you can pass your own:
|
14
|
+
|
15
|
+
```rb
|
16
|
+
my_store = Hash.new
|
17
|
+
|
18
|
+
Shrine.storages[:store] = Shrine::Storage::Memory.new(my_store)
|
19
|
+
```
|
data/doc/storage/s3.md
CHANGED
@@ -2,30 +2,40 @@
|
|
2
2
|
title: AWS S3
|
3
3
|
---
|
4
4
|
|
5
|
-
The S3 storage handles uploads to
|
5
|
+
The S3 storage handles uploads to [AWS S3] service (or any s3-compatible
|
6
|
+
service such as [DigitalOcean Spaces] or [MinIO]). It requires the [aws-sdk-s3]
|
6
7
|
gem:
|
7
8
|
|
8
9
|
```rb
|
10
|
+
# Gemfile
|
9
11
|
gem "aws-sdk-s3", "~> 1.14"
|
10
12
|
```
|
11
13
|
|
12
|
-
|
14
|
+
## Initialization
|
15
|
+
|
16
|
+
The storage is initialized by providing your bucket name, region and
|
17
|
+
credentials:
|
13
18
|
|
14
19
|
```rb
|
15
20
|
require "shrine/storage/s3"
|
16
21
|
|
17
22
|
s3 = Shrine::Storage::S3.new(
|
18
23
|
bucket: "my-app", # required
|
24
|
+
region: "eu-west-1", # required
|
19
25
|
access_key_id: "abc",
|
20
26
|
secret_access_key: "xyz",
|
21
|
-
region: "eu-west-1",
|
22
27
|
)
|
23
28
|
```
|
24
29
|
|
25
|
-
The
|
26
|
-
|
27
|
-
|
28
|
-
|
30
|
+
> The storage requires the following AWS S3 permissions:
|
31
|
+
>
|
32
|
+
> * `s3:ListBucket` for the bucket resource
|
33
|
+
> * `s3:GetObject`, `s3:PutObject`, `s3:PutObjectAcl`, `s3:DeleteObject`,
|
34
|
+
> `s3:ListMultipartUploadParts` and `s3:AbortMultipartUpload` for the object
|
35
|
+
> resources
|
36
|
+
|
37
|
+
> The `:access_key_id` and `:secret_access_key` options is just one form of
|
38
|
+
> authentication, see the [AWS SDK docs][credentials] for more options.
|
29
39
|
|
30
40
|
The storage exposes the underlying Aws objects:
|
31
41
|
|
@@ -45,14 +55,29 @@ s3.object("key") #=> #<Aws::S3::Object>
|
|
45
55
|
|
46
56
|
By default, uploaded S3 objects will have private visibility, meaning they can
|
47
57
|
only be accessed via signed expiring URLs generated using your private S3
|
48
|
-
credentials.
|
49
|
-
to make uploads public:
|
58
|
+
credentials.
|
50
59
|
|
51
60
|
```rb
|
52
|
-
s3 = Shrine::Storage::S3.new(
|
61
|
+
s3 = Shrine::Storage::S3.new(**s3_options)
|
62
|
+
s3.upload(io, "key") # uploads with default "private" ACL
|
63
|
+
s3.url("key") # https://my-bucket.s3.amazonaws.com/key?X-Amz-Expires=900&X-Amz-Signature=b22d37c37d...
|
64
|
+
```
|
53
65
|
|
66
|
+
If you would like to generate public URLs, you can tell S3 storage to make
|
67
|
+
uploads public:
|
68
|
+
|
69
|
+
```rb
|
70
|
+
s3 = Shrine::Storage::S3.new(public: true, **s3_options)
|
54
71
|
s3.upload(io, "key") # uploads with "public-read" ACL
|
55
|
-
s3.url("key") #
|
72
|
+
s3.url("key") # https://my-bucket.s3.amazonaws.com/key
|
73
|
+
```
|
74
|
+
|
75
|
+
If you want to make only *some* uploads public, you can conditionally apply the
|
76
|
+
`:acl` upload option and `:public` URL option:
|
77
|
+
|
78
|
+
```rb
|
79
|
+
Shrine.plugin :upload_options, store: -> (io, **) { { acl: "public-read" } }
|
80
|
+
Shrine.plugin :url_options, store: -> (io, **) { { public: true } }
|
56
81
|
```
|
57
82
|
|
58
83
|
## Prefix
|
@@ -80,13 +105,11 @@ You can also generate upload options per upload with the `upload_options`
|
|
80
105
|
plugin
|
81
106
|
|
82
107
|
```rb
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
|
88
|
-
{ acl: "private" }
|
89
|
-
end
|
108
|
+
Shrine.plugin :upload_options, store: -> (io, derivative: nil, **) do
|
109
|
+
if derivative == :thumb
|
110
|
+
{ acl: "public-read" }
|
111
|
+
else
|
112
|
+
{ acl: "private" }
|
90
113
|
end
|
91
114
|
end
|
92
115
|
```
|
@@ -97,23 +120,9 @@ or when using the uploader directly
|
|
97
120
|
uploader.upload(file, upload_options: { acl: "private" })
|
98
121
|
```
|
99
122
|
|
100
|
-
|
101
|
-
the uploader level won't be forwarded for generating presigns, since presigns
|
102
|
-
are generated using the storage directly.
|
103
|
-
|
104
|
-
## URL options
|
105
|
-
|
106
|
-
Other than [`:host`](#url-host) and [`:public`](#public-uploads) URL options,
|
107
|
-
all additional options are forwarded to [`Aws::S3::Object#presigned_url`].
|
108
|
-
|
109
|
-
```rb
|
110
|
-
s3.url(
|
111
|
-
expires_in: 15,
|
112
|
-
response_content_disposition: ContentDisposition.attachment("my-filename"),
|
113
|
-
response_content_type: "foo/bar",
|
114
|
-
# ...
|
115
|
-
)
|
116
|
-
```
|
123
|
+
> Unlike the `:upload_options` storage option, upload options given on
|
124
|
+
the uploader level won't be forwarded for generating presigns, since presigns
|
125
|
+
are generated using the storage directly.
|
117
126
|
|
118
127
|
## URL Host
|
119
128
|
|
@@ -133,12 +142,14 @@ s3.url("image.jpg", host: "https://your-s3-host.com/prefix/") # needs to end wit
|
|
133
142
|
```
|
134
143
|
|
135
144
|
To have the `:host` option passed automatically for every URL, use the
|
136
|
-
`url_options` plugin
|
145
|
+
`url_options` plugin:
|
137
146
|
|
138
147
|
```rb
|
139
148
|
plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
|
140
149
|
```
|
141
150
|
|
151
|
+
### Signer
|
152
|
+
|
142
153
|
If you would like to [serve private content via CloudFront], you need to sign
|
143
154
|
the object URLs with a special signer, such as [`Aws::CloudFront::UrlSigner`]
|
144
155
|
provided by the `aws-sdk-cloudfront` gem. The S3 storage initializer accepts a
|
@@ -157,13 +168,27 @@ Shrine::Storage::S3.new(signer: signer.method(:signed_url))
|
|
157
168
|
Shrine::Storage::S3.new(signer: -> (url, **options) { signer.signed_url(url, **options) })
|
158
169
|
```
|
159
170
|
|
171
|
+
## URL options
|
172
|
+
|
173
|
+
Other than `:host` and `:public` URL options, all additional `S3#url` options
|
174
|
+
are forwarded to [`Aws::S3::Object#presigned_url`].
|
175
|
+
|
176
|
+
```rb
|
177
|
+
s3.url(
|
178
|
+
expires_in: 15,
|
179
|
+
response_content_disposition: ContentDisposition.attachment("my-filename"),
|
180
|
+
response_content_type: "foo/bar",
|
181
|
+
# ...
|
182
|
+
)
|
183
|
+
```
|
184
|
+
|
160
185
|
## Presigns
|
161
186
|
|
162
|
-
The
|
163
|
-
to
|
187
|
+
The `S3#presign` method can be used for generating parameters for direct upload
|
188
|
+
to S3:
|
164
189
|
|
165
190
|
```rb
|
166
|
-
s3.presign("
|
191
|
+
s3.presign("key") #=>
|
167
192
|
# {
|
168
193
|
# url: "https://my-bucket.s3.amazonaws.com/...",
|
169
194
|
# fields: { ... }, # blank for PUT presigns
|
@@ -172,11 +197,25 @@ s3.presign("/path/to/file") #=>
|
|
172
197
|
# }
|
173
198
|
```
|
174
199
|
|
175
|
-
|
200
|
+
By default, parameters for a POST upload is generated, but you can also
|
201
|
+
generate PUT upload parameters:
|
202
|
+
|
203
|
+
```rb
|
204
|
+
s3.presign("key", method: :put)
|
205
|
+
```
|
176
206
|
|
177
|
-
|
178
|
-
|
179
|
-
|
207
|
+
Any additional options are forwarded to [`Aws::S3::Object#presigned_post`]
|
208
|
+
(for POST uploads) and [`Aws::S3::Object#presigned_url`] (for PUT uploads).
|
209
|
+
|
210
|
+
```rb
|
211
|
+
s3.presign("key", method: :put, content_disposition: "attachment; filename=my-file.txt") #=>
|
212
|
+
# {
|
213
|
+
# url: "https://my-bucket.s3.amazonaws.com/...",
|
214
|
+
# fields: {},
|
215
|
+
# headers: { "Content-Disposition" => "attachment; filename=my-file.txt" },
|
216
|
+
# method :put,
|
217
|
+
# }
|
218
|
+
```
|
180
219
|
|
181
220
|
## Large files
|
182
221
|
|
@@ -184,33 +223,24 @@ The aws-sdk-s3 gem has the ability to automatically use multipart upload/copy
|
|
184
223
|
for larger files, splitting the file into multiple chunks and uploading/copying
|
185
224
|
them in parallel.
|
186
225
|
|
187
|
-
By default
|
188
|
-
|
189
|
-
|
190
|
-
`:multipart_threshold`.
|
191
|
-
|
192
|
-
```rb
|
193
|
-
thresholds = { upload: 30*1024*1024, copy: 200*1024*1024 }
|
194
|
-
Shrine::Storage::S3.new(multipart_threshold: thresholds, **s3_options)
|
195
|
-
```
|
196
|
-
|
197
|
-
If you want to change how many threads aws-sdk-s3 will use for multipart
|
198
|
-
upload/copy, you can use the `upload_options` plugin to specify
|
199
|
-
`:thread_count`.
|
226
|
+
By default, multipart upload will be used for files larger than 15MB, and
|
227
|
+
multipart copy for files larger than 100MB, but you can change the thresholds
|
228
|
+
via `:multipart_threshold`:
|
200
229
|
|
201
230
|
```rb
|
202
|
-
|
203
|
-
{
|
204
|
-
|
231
|
+
Shrine::Storage::S3.new(
|
232
|
+
multipart_threshold: { upload: 30*1024*1024, copy: 200*1024*1024 },
|
233
|
+
**s3_options,
|
234
|
+
)
|
205
235
|
```
|
206
236
|
|
207
237
|
## Encryption
|
208
238
|
|
209
|
-
The easiest way to use server-side encryption for uploaded S3 objects is to
|
239
|
+
The easiest way to use **server-side** encryption for uploaded S3 objects is to
|
210
240
|
configure default encryption for your S3 bucket. Alternatively, you can pass
|
211
241
|
server-side encryption parameters to the API calls.
|
212
242
|
|
213
|
-
The
|
243
|
+
The `S3#upload` method accepts `:sse_*` options:
|
214
244
|
|
215
245
|
```rb
|
216
246
|
s3.upload(io, "key", sse_customer_algorithm: "AES256",
|
@@ -219,7 +249,7 @@ s3.upload(io, "key", sse_customer_algorithm: "AES256",
|
|
219
249
|
ssekms_key_id: "key_id")
|
220
250
|
```
|
221
251
|
|
222
|
-
The
|
252
|
+
The `S3#presign` method accepts `:server_side_encryption_*` options for POST
|
223
253
|
presigns, and the same `:sse_*` options as above for PUT presigns.
|
224
254
|
|
225
255
|
```rb
|
@@ -237,15 +267,14 @@ s3.open("key", sse_customer_algorithm: "AES256",
|
|
237
267
|
sse_customer_key_md5: "secret_key_md5")
|
238
268
|
```
|
239
269
|
|
240
|
-
|
241
|
-
storage with an `Aws::S3::Encryption::Client` instance.
|
270
|
+
**Client-side** encryption is supported as well:
|
242
271
|
|
243
272
|
```rb
|
244
|
-
|
245
|
-
|
246
|
-
)
|
273
|
+
encryption_client = Aws::S3::EncryptionV2::Client.new(...)
|
274
|
+
s3 = Shrine::Storage::S3.new(client: encryption_client, **other_options)
|
247
275
|
|
248
|
-
|
276
|
+
s3.upload(io, "key") # encrypts on upload
|
277
|
+
s3.open("key") # decrypts on download
|
249
278
|
```
|
250
279
|
|
251
280
|
## Accelerate endpoint
|
@@ -279,20 +308,18 @@ Alternatively you can periodically call the `#clear!` method:
|
|
279
308
|
s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
|
280
309
|
```
|
281
310
|
|
282
|
-
|
283
|
-
|
284
|
-
|
285
|
-
|
286
|
-
per second per prefix in a bucket (a prefix is a top-level "directory" in the
|
287
|
-
bucket). If your app needs to support higher request rates to S3 than that, you
|
288
|
-
can scale exponentially by using more prefixes.
|
289
|
-
|
311
|
+
[AWS S3]: https://aws.amazon.com/s3/
|
312
|
+
[MinIO]: https://min.io/
|
313
|
+
[DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
|
314
|
+
[aws-sdk-s3]: https://rubygems.org/gems/aws-sdk-s3
|
290
315
|
[uploading]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#put-instance_method
|
291
316
|
[copying]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#copy_from-instance_method
|
292
317
|
[presigning]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
|
293
318
|
[`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
|
294
|
-
[aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
|
295
319
|
[Transfer Acceleration]: http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html
|
296
320
|
[object lifecycle]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
|
297
321
|
[serve private content via CloudFront]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html
|
298
322
|
[`Aws::CloudFront::UrlSigner`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/CloudFront/UrlSigner.html
|
323
|
+
[credentials]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
|
324
|
+
[`Aws::S3::Object#presigned_post`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
|
325
|
+
[`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
|