shrine 3.1.0 → 3.4.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (93) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +82 -0
  3. data/README.md +11 -4
  4. data/doc/advantages.md +4 -4
  5. data/doc/attacher.md +2 -2
  6. data/doc/carrierwave.md +24 -12
  7. data/doc/changing_derivatives.md +1 -1
  8. data/doc/changing_location.md +6 -5
  9. data/doc/design.md +134 -85
  10. data/doc/direct_s3.md +26 -0
  11. data/doc/external/articles.md +57 -45
  12. data/doc/external/extensions.md +41 -35
  13. data/doc/external/misc.md +23 -8
  14. data/doc/getting_started.md +156 -85
  15. data/doc/metadata.md +80 -44
  16. data/doc/multiple_files.md +1 -1
  17. data/doc/paperclip.md +28 -9
  18. data/doc/plugins/add_metadata.md +112 -35
  19. data/doc/plugins/atomic_helpers.md +41 -3
  20. data/doc/plugins/backgrounding.md +12 -2
  21. data/doc/plugins/column.md +36 -7
  22. data/doc/plugins/default_url.md +6 -3
  23. data/doc/plugins/derivatives.md +83 -44
  24. data/doc/plugins/download_endpoint.md +5 -5
  25. data/doc/plugins/dynamic_storage.md +1 -1
  26. data/doc/plugins/entity.md +12 -4
  27. data/doc/plugins/form_assign.md +5 -5
  28. data/doc/plugins/included.md +25 -5
  29. data/doc/plugins/infer_extension.md +9 -0
  30. data/doc/plugins/instrumentation.md +1 -1
  31. data/doc/plugins/metadata_attributes.md +1 -0
  32. data/doc/plugins/mirroring.md +1 -1
  33. data/doc/plugins/model.md +8 -3
  34. data/doc/plugins/persistence.md +10 -1
  35. data/doc/plugins/remote_url.md +6 -1
  36. data/doc/plugins/remove_invalid.md +9 -1
  37. data/doc/plugins/sequel.md +1 -1
  38. data/doc/plugins/store_dimensions.md +10 -0
  39. data/doc/plugins/type_predicates.md +96 -0
  40. data/doc/plugins/upload_endpoint.md +1 -1
  41. data/doc/plugins/upload_options.md +1 -1
  42. data/doc/plugins/url_options.md +4 -4
  43. data/doc/plugins/validation.md +14 -4
  44. data/doc/plugins/versions.md +7 -7
  45. data/doc/processing.md +287 -123
  46. data/doc/refile.md +9 -9
  47. data/doc/release_notes/2.8.0.md +1 -1
  48. data/doc/release_notes/3.0.0.md +1 -1
  49. data/doc/release_notes/3.2.0.md +96 -0
  50. data/doc/release_notes/3.2.1.md +31 -0
  51. data/doc/release_notes/3.2.2.md +14 -0
  52. data/doc/release_notes/3.3.0.md +105 -0
  53. data/doc/release_notes/3.4.0.md +35 -0
  54. data/doc/securing_uploads.md +2 -2
  55. data/doc/storage/memory.md +19 -0
  56. data/doc/storage/s3.md +104 -77
  57. data/doc/testing.md +12 -2
  58. data/doc/upgrading_to_3.md +99 -53
  59. data/lib/shrine.rb +9 -8
  60. data/lib/shrine/attacher.rb +20 -10
  61. data/lib/shrine/attachment.rb +2 -2
  62. data/lib/shrine/plugins.rb +22 -0
  63. data/lib/shrine/plugins/activerecord.rb +3 -3
  64. data/lib/shrine/plugins/add_metadata.rb +20 -5
  65. data/lib/shrine/plugins/backgrounding.rb +2 -2
  66. data/lib/shrine/plugins/default_url.rb +1 -1
  67. data/lib/shrine/plugins/derivation_endpoint.rb +13 -8
  68. data/lib/shrine/plugins/derivatives.rb +59 -30
  69. data/lib/shrine/plugins/determine_mime_type.rb +5 -3
  70. data/lib/shrine/plugins/entity.rb +12 -11
  71. data/lib/shrine/plugins/instrumentation.rb +12 -18
  72. data/lib/shrine/plugins/mirroring.rb +8 -8
  73. data/lib/shrine/plugins/model.rb +3 -3
  74. data/lib/shrine/plugins/presign_endpoint.rb +16 -4
  75. data/lib/shrine/plugins/pretty_location.rb +1 -1
  76. data/lib/shrine/plugins/processing.rb +1 -1
  77. data/lib/shrine/plugins/refresh_metadata.rb +2 -2
  78. data/lib/shrine/plugins/remote_url.rb +3 -3
  79. data/lib/shrine/plugins/remove_attachment.rb +5 -0
  80. data/lib/shrine/plugins/remove_invalid.rb +10 -5
  81. data/lib/shrine/plugins/sequel.rb +1 -1
  82. data/lib/shrine/plugins/store_dimensions.rb +4 -2
  83. data/lib/shrine/plugins/type_predicates.rb +113 -0
  84. data/lib/shrine/plugins/upload_endpoint.rb +10 -5
  85. data/lib/shrine/plugins/upload_options.rb +2 -2
  86. data/lib/shrine/plugins/url_options.rb +2 -2
  87. data/lib/shrine/plugins/validation.rb +9 -7
  88. data/lib/shrine/storage/linter.rb +4 -4
  89. data/lib/shrine/storage/memory.rb +5 -3
  90. data/lib/shrine/storage/s3.rb +117 -38
  91. data/lib/shrine/version.rb +1 -1
  92. data/shrine.gemspec +8 -8
  93. metadata +42 -34
data/doc/refile.md CHANGED
@@ -1,5 +1,5 @@
1
1
  ---
2
- title: Shrine for Refile Users
2
+ title: Upgrading from Refile
3
3
  ---
4
4
 
5
5
  This guide is aimed at helping Refile users transition to Shrine, and it consists
@@ -110,13 +110,6 @@ into separate columns.
110
110
  Shrine provides on-the-fly processing via the
111
111
  [`derivation_endpoint`][derivation_endpoint] plugin:
112
112
 
113
- ```rb
114
- # config/routes.rb (Rails)
115
- Rails.application.routes.draw do
116
- # ...
117
- mount ImageUploader.derivation_endpoint => "/derivations/image"
118
- end
119
- ```
120
113
  ```rb
121
114
  require "image_processing/mini_magick"
122
115
 
@@ -132,8 +125,15 @@ class ImageUploader < Shrine
132
125
  end
133
126
  end
134
127
  ```
128
+ ```rb
129
+ # config/routes.rb (Rails)
130
+ Rails.application.routes.draw do
131
+ # ...
132
+ mount ImageUploader.derivation_endpoint => "/derivations/image"
133
+ end
134
+ ```
135
135
 
136
- Shrine also support processing up front using the [`derivatives`][derivatives]
136
+ Shrine also support eager processing using the [`derivatives`][derivatives]
137
137
  plugin.
138
138
 
139
139
  ### Validation
@@ -1,5 +1,5 @@
1
1
  ---
2
- title: Shrine 2.9.0
2
+ title: Shrine 2.8.0
3
3
  ---
4
4
 
5
5
  ## New Features
@@ -366,7 +366,7 @@ end
366
366
  ```
367
367
  ```rb
368
368
  attacher = photo.image_attacher
369
- attacher.form_assign("image" => file, "title" => "...", "description" => "...")
369
+ attacher.form_assign({ "image" => file, "title" => "...", "description" => "..." })
370
370
  attacher.file #=> #<Shrine::UploadedFile id="..." storage=:cache ...>
371
371
  ```
372
372
 
@@ -0,0 +1,96 @@
1
+ ---
2
+ title: Shrine 3.2.0
3
+ ---
4
+
5
+ ## New features
6
+
7
+ * The `type_predicates` plugin has been added, which adds convenient predicate
8
+ methods to `Shrine::UploadedFile` based on the MIME type.
9
+
10
+ ```rb
11
+ # Gemfile
12
+ gem "mini_mime" # default dependency of type_predicates
13
+ ```
14
+ ```rb
15
+ Shrine.plugin :type_predicates
16
+ ```
17
+
18
+ The plugin adds four predicate methods based on the general type of the file:
19
+
20
+ ```rb
21
+ file.image? # returns true for any "image/*" MIME type
22
+ file.video? # returns true for any "video/*" MIME type
23
+ file.audio? # returns true for any "audio/*" MIME type
24
+ file.text? # returns true for any "text/*" MIME type
25
+ ```
26
+
27
+ You can also check for specific MIME type using the extension name:
28
+
29
+ ```rb
30
+ file.type?(:jpg) # returns true if MIME type is "image/jpeg"
31
+ file.type?(:svg) # returns true if MIME type is "image/svg+xml"
32
+ file.type?(:mov) # returns true if MIME type is "video/quicktime"
33
+ file.type?(:ppt) # returns true if MIME type is "application/vnd.ms-powerpoint"
34
+ ...
35
+ ```
36
+
37
+ For convenience, you can create predicate methods for specific file types:
38
+
39
+ ```rb
40
+ Shrine.plugin :type_predicates, methods: %i[jpg svg mov ppt]
41
+ ```
42
+ ```rb
43
+ file.jpg? # returns true if MIME type is "image/jpeg"
44
+ file.svg? # returns true if MIME type is "image/svg+xml"
45
+ file.mov? # returns true if MIME type is "video/quicktime"
46
+ file.ppt? # returns true if MIME type is "application/vnd.ms-powerpoint"
47
+ ```
48
+
49
+ * The `#add_metadata` method has been added to the `add_metadata` plugin for
50
+ adding new metadata to an existing file/attachment.
51
+
52
+ ```rb
53
+ attacher.file.metadata #=> { ... }
54
+ attacher.add_metadata("foo" => "bar")
55
+ attacher.file.metadata #=> { ..., "foo" => "bar" }
56
+ ```
57
+
58
+ ## Other improvements
59
+
60
+ * The `remove_invalid` plugin now works correctly with `derivatives` plugin.
61
+
62
+ * The `remove_invalid` plugin is now also activated when `Attacher#validate`
63
+ is called manually.
64
+
65
+ * The current attached file data can now be assigned back to the attachment
66
+ attribute, and this operation will be a no-op.
67
+
68
+ ```rb
69
+ photo.image #=> #<Shrine::UploadedFile id="foo" storage=:store metadata={...}>
70
+ photo.image = { "id" => "foo", "storage" => "store", "metadata" => { ... } } # no-op
71
+ ```
72
+
73
+ This allows treating the attachment attribute as a persistent attribute,
74
+ where the current value can be assigned back on record updates.
75
+
76
+ * When promoting derivatives, the `:derivative` parameter value was being
77
+ passed to the uploader as an array. This has been fixed, and the value is now
78
+ the same as when uploading derivatives directly to permanent storage.
79
+
80
+ * The `derivatives` plugin now includes additional `:io` and `:attacher` values
81
+ in the instrumentation event payload.
82
+
83
+ ## Backwards compatibility
84
+
85
+ * The `validation` plugin now runs validations on `Attacher#attach` and
86
+ `Attacher#attach_cached`. If you were using `Attacher#change` directly and
87
+ expecting the validations to be run automatically, you will need to update
88
+ your code.
89
+
90
+ * If you were updating the cached file metadata via file data assignment, this
91
+ will no longer work.
92
+
93
+ ```rb
94
+ photo.image #=> #<Shrine::UploadedFile id="foo" storage=:cache metadata={...}>
95
+ photo.image = { "id" => "foo", "storage" => "cache", "metadata" => { ... } } # no-op
96
+ ```
@@ -0,0 +1,31 @@
1
+ ---
2
+ title: Shrine 3.2.1
3
+ ---
4
+
5
+ ## Ruby 2.7 compatibility
6
+
7
+ * Shrine doesn't trigger [Ruby 2.7 warnings for separation of positional and
8
+ keyword arguments][kwargs] anymore.
9
+
10
+ * Down 5.1.0 has been released, which resolves warnings and a `FrozenError`
11
+ exception on Ruby 2.7. Shrine now requires at least this version of Down.
12
+
13
+ If you're using `Down::Http`, make sure you're using http.rb 4.3.0 or newer.
14
+
15
+ * ImageProcessing 1.10.3 gem has been released which resolves Ruby 2.7 warnings
16
+ as well. If you're using it for image processing, make sure to upgrade to
17
+ this version:
18
+
19
+ ```rb
20
+ gem "image_processing", ">= 1.10.3", "< 2"
21
+ ```
22
+
23
+ ## Rack 2.1 compatibility
24
+
25
+ * The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1 or newer.
26
+
27
+ ## Other improvements
28
+
29
+ * The `S3#open` method now handles empty S3 objects.
30
+
31
+ [kwargs]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/
@@ -0,0 +1,14 @@
1
+ ---
2
+ title: Shrine 3.2.2
3
+ ---
4
+
5
+ ## Bug fixes
6
+
7
+ * aws-sdk-core 3.104.0 introduced a backwards incompatible changes that caused
8
+ `Shrine::Storage::S3#open` to start raising an exception.
9
+
10
+ ```
11
+ NoMethodError: undefined method `bytesize' for #<Array:0x000000000a721be0>
12
+ ```
13
+
14
+ This has now been fixed.
@@ -0,0 +1,105 @@
1
+ ---
2
+ title: Shrine 3.3.0
3
+ ---
4
+
5
+ ## New features
6
+
7
+ * The `:create_on_promote` option has been added to the `derivatives` plugin
8
+ for automatically creating derivatives after the attached cached file is
9
+ promoted to permanent storage.
10
+
11
+ ```rb
12
+ Shrine.plugin :derivatives, create_on_promote: true
13
+ ```
14
+
15
+ * The `:auto_extraction` option has been added to the `store_dimensions` plugin
16
+ for skipping automatically extracting dimensions on upload.
17
+
18
+ ```rb
19
+ Shrine.plugin :store_dimensions, auto_extraction: false
20
+ ```
21
+
22
+ * The `:skip_nil` option has been added to the `add_metadata` plugin for
23
+ excluding metadata keys whose values are nil.
24
+
25
+ ```rb
26
+ class PdfUploader < Shrine
27
+ add_metadata :pages, skip_nil: true do |io|
28
+ if is_pdf?(io)
29
+ reader = PDF::Reader.new(io)
30
+ reader.page_count
31
+ else
32
+ # If this is not a PDF, then the pages metadata will not be stored
33
+ nil
34
+ end
35
+ end
36
+ end
37
+ ```
38
+
39
+ * The `:download` option has been added to derivatives processors in
40
+ `derivatives` plugin for skipping converting the source IO object into a
41
+ file. This can be used to avoid a potentially expensive download/copy when
42
+ the derivatives processor doesn't need the file.
43
+
44
+ ```rb
45
+ Attacher.derivatives :my_processor, download: false do |source|
46
+ source #=> Could be File, Shrine::UploadedFile, or other IO-like object
47
+ shrine_class.with_file(source) do |file|
48
+ # can force download/copy if necessary with `with_file`,
49
+ end
50
+ end
51
+ ```
52
+
53
+ ## Bug fixes
54
+
55
+ * The `upload_endpoint` now handles calling `Shrine.upload_response` method
56
+ from a Rails controller.
57
+
58
+ * The `derivation_endpoint` plugin now applies the `version` query parameter
59
+ to the derivation when creating the response.
60
+
61
+ ## Other improvements
62
+
63
+ * The new `Aws:S3::EncryptionV2::Client` is now supported by the S3 storage for
64
+ client-side encryption.
65
+
66
+ * The `derivation_endpoint` now reduces the possibility of timing attacks by
67
+ comparing URL signatures in constant time using `Rack::Utils.secure_compare`.
68
+
69
+ * The `derivatives` plugin now copies non-file source IO objects to disk before
70
+ passing them to the processor. This is consistent with how a
71
+ `Shrine::UploadedFile` object is downloaded to disk.
72
+
73
+ * The `sequel` and `activerecord` plugins now call `Attacher#reload` when
74
+ reloading the model, which reloads the attached files but keeps other
75
+ attacher state.
76
+
77
+ * The `derivatives` plugin doesn't download the attached file anymore if
78
+ attempting to process derivatives when no derivatives processor was defined.
79
+
80
+ * The `mirroring` plugin now forwards attacher options when uploading to mirror
81
+ storages.
82
+
83
+ * The `presign_endpoint` plugin now handles the `OPTIONS` HTTP verb, which
84
+ newer versions of Uppy are requesting.
85
+
86
+ * `Shrine::Storage::Memory#open` now always returns a `StringIO` in the file
87
+ content's original encoding, instead of the encoding set by
88
+ `Encoding.default_internal`. This works around a [bug][ruby-lang #16497]
89
+ in `StringIO` introduced in Ruby 2.7.0.
90
+
91
+ * The `remove_attachment` plugin now deletes the removed file if a new file was
92
+ attached right after removal.
93
+
94
+ ## Backwards compatibility
95
+
96
+ * If you were passing a non-file IO object to the derivatives processor, Shrine
97
+ will now convert it into a file beforehand. If you're currently doing this
98
+ and are converting the IO object into a file inside the processor, you can
99
+ now remove the conversion code to avoid doubling the amount of disk writes.
100
+
101
+ * When reloading a Sequel/ActiveRecord model, any attacher state other than
102
+ uploaded files will now be retained after the reload. If you were relying on
103
+ all the attacher state being re-initialized, you'll need to update your code.
104
+
105
+ [ruby-lang #16497]: https://bugs.ruby-lang.org/issues/16497
@@ -0,0 +1,35 @@
1
+ ---
2
+ title: Shrine 3.4.0
3
+ ---
4
+
5
+ * Passing attacher options to `Shrine.Attachment` method now works on Ruby 3.0.
6
+
7
+ * Defining validation errors as an array of I18n key and options in
8
+ `activerecord` plugin now works on Ruby 3.0.
9
+
10
+ * The `:fastimage` MIME type analyzer now correctly detects SVGs as
11
+ `image/svg+html` in the `determine_mime_type` plugin.
12
+
13
+ * The `Shrine::Attacher#read` method provided by the `entity` plugin is now
14
+ public. This is consistent with `Shrine::Attacher#write` from `model` plugin
15
+ being public as well.
16
+
17
+ * The `Shrine::Attacher#reload` method now resets attachment's dirty state.
18
+ This means that for a model whose `Attacher#changed?` returns `true`, calling
19
+ `#reload` on the model will make `Attacher#changed?` return `false`. This was
20
+ the behaviour before Shrine 3.3.0.
21
+
22
+ ```rb
23
+ # before
24
+ model.file_attacher.changed? #=> true
25
+ model.reload
26
+ model.file_attacher.changed? #=> true
27
+
28
+ # after
29
+ model.file_attacher.changed? #=> true
30
+ model.reload
31
+ model.file_attacher.changed? #=> false
32
+ ```
33
+
34
+ * Calling `#reload` on the model will not initialize a `Shrine::Attacher`
35
+ instance anymore if one hasn't previously been initialized.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  id: securing-uploads
3
- title: Securing uploads
3
+ title: Securing Uploads
4
4
  ---
5
5
 
6
6
  Shrine does a lot to make your file uploads secure, but there are still a lot
@@ -63,7 +63,7 @@ in the `:max_size` option to reject files that are larger than the specified
63
63
  limit:
64
64
 
65
65
  ```rb
66
- plugin :upload_endpoint, max_size: 100*1024*1024 # 20 MB
66
+ plugin :upload_endpoint, max_size: 100*1024*1024 # 100 MB
67
67
  ```
68
68
 
69
69
  If you're doing direct uploads to Amazon S3 using the `presign_endpoint`
@@ -0,0 +1,19 @@
1
+ ---
2
+ title: Memory
3
+ ---
4
+
5
+ The Memory storage stores uploaded files in memory, which is suitable for
6
+ testing.
7
+
8
+ ```rb
9
+ Shrine.storages[:store] = Shrine::Storage::Memory.new
10
+ ```
11
+
12
+ By default, each storage instance uses a new Hash object for storing files,
13
+ but you can pass your own:
14
+
15
+ ```rb
16
+ my_store = Hash.new
17
+
18
+ Shrine.storages[:store] = Shrine::Storage::Memory.new(my_store)
19
+ ```
data/doc/storage/s3.md CHANGED
@@ -2,30 +2,40 @@
2
2
  title: AWS S3
3
3
  ---
4
4
 
5
- The S3 storage handles uploads to Amazon S3 service, using the [aws-sdk-s3]
5
+ The S3 storage handles uploads to [AWS S3] service (or any s3-compatible
6
+ service such as [DigitalOcean Spaces] or [MinIO]). It requires the [aws-sdk-s3]
6
7
  gem:
7
8
 
8
9
  ```rb
10
+ # Gemfile
9
11
  gem "aws-sdk-s3", "~> 1.14"
10
12
  ```
11
13
 
12
- It can be initialized by providing the bucket name and credentials:
14
+ ## Initialization
15
+
16
+ The storage is initialized by providing your bucket name, region and
17
+ credentials:
13
18
 
14
19
  ```rb
15
20
  require "shrine/storage/s3"
16
21
 
17
22
  s3 = Shrine::Storage::S3.new(
18
23
  bucket: "my-app", # required
24
+ region: "eu-west-1", # required
19
25
  access_key_id: "abc",
20
26
  secret_access_key: "xyz",
21
- region: "eu-west-1",
22
27
  )
23
28
  ```
24
29
 
25
- The core features of this storage require the following AWS permissions:
26
- `s3:ListBucket`, `s3:PutObject`, `s3:GetObject`, and `s3:DeleteObject`. If you
27
- have additional upload options configured such as setting object ACLs, then
28
- additional permissions may be required.
30
+ > The storage requires the following AWS S3 permissions:
31
+ >
32
+ > * `s3:ListBucket` for the bucket resource
33
+ > * `s3:GetObject`, `s3:PutObject`, `s3:PutObjectAcl`, `s3:DeleteObject`,
34
+ > `s3:ListMultipartUploadParts` and `s3:AbortMultipartUpload` for the object
35
+ > resources
36
+
37
+ > The `:access_key_id` and `:secret_access_key` options is just one form of
38
+ > authentication, see the [AWS SDK docs][credentials] for more options.
29
39
 
30
40
  The storage exposes the underlying Aws objects:
31
41
 
@@ -45,14 +55,29 @@ s3.object("key") #=> #<Aws::S3::Object>
45
55
 
46
56
  By default, uploaded S3 objects will have private visibility, meaning they can
47
57
  only be accessed via signed expiring URLs generated using your private S3
48
- credentials. If you would like to generate public URLs, you can tell S3 storage
49
- to make uploads public:
58
+ credentials.
50
59
 
51
60
  ```rb
52
- s3 = Shrine::Storage::S3.new(public: true, **s3_options)
61
+ s3 = Shrine::Storage::S3.new(**s3_options)
62
+ s3.upload(io, "key") # uploads with default "private" ACL
63
+ s3.url("key") # https://my-bucket.s3.amazonaws.com/key?X-Amz-Expires=900&X-Amz-Signature=b22d37c37d...
64
+ ```
53
65
 
66
+ If you would like to generate public URLs, you can tell S3 storage to make
67
+ uploads public:
68
+
69
+ ```rb
70
+ s3 = Shrine::Storage::S3.new(public: true, **s3_options)
54
71
  s3.upload(io, "key") # uploads with "public-read" ACL
55
- s3.url("key") # returns public (unsigned) object URL
72
+ s3.url("key") # https://my-bucket.s3.amazonaws.com/key
73
+ ```
74
+
75
+ If you want to make only *some* uploads public, you can conditionally apply the
76
+ `:acl` upload option and `:public` URL option:
77
+
78
+ ```rb
79
+ Shrine.plugin :upload_options, store: -> (io, **) { { acl: "public-read" } }
80
+ Shrine.plugin :url_options, store: -> (io, **) { { public: true } }
56
81
  ```
57
82
 
58
83
  ## Prefix
@@ -80,13 +105,11 @@ You can also generate upload options per upload with the `upload_options`
80
105
  plugin
81
106
 
82
107
  ```rb
83
- class MyUploader < Shrine
84
- plugin :upload_options, store: -> (io, derivative: nil, **) do
85
- if derivative == :thumb
86
- { acl: "public-read" }
87
- else
88
- { acl: "private" }
89
- end
108
+ Shrine.plugin :upload_options, store: -> (io, derivative: nil, **) do
109
+ if derivative == :thumb
110
+ { acl: "public-read" }
111
+ else
112
+ { acl: "private" }
90
113
  end
91
114
  end
92
115
  ```
@@ -97,23 +120,9 @@ or when using the uploader directly
97
120
  uploader.upload(file, upload_options: { acl: "private" })
98
121
  ```
99
122
 
100
- Note that, unlike the `:upload_options` storage option, upload options given on
101
- the uploader level won't be forwarded for generating presigns, since presigns
102
- are generated using the storage directly.
103
-
104
- ## URL options
105
-
106
- Other than [`:host`](#url-host) and [`:public`](#public-uploads) URL options,
107
- all additional options are forwarded to [`Aws::S3::Object#presigned_url`].
108
-
109
- ```rb
110
- s3.url(
111
- expires_in: 15,
112
- response_content_disposition: ContentDisposition.attachment("my-filename"),
113
- response_content_type: "foo/bar",
114
- # ...
115
- )
116
- ```
123
+ > Unlike the `:upload_options` storage option, upload options given on
124
+ the uploader level won't be forwarded for generating presigns, since presigns
125
+ are generated using the storage directly.
117
126
 
118
127
  ## URL Host
119
128
 
@@ -133,12 +142,14 @@ s3.url("image.jpg", host: "https://your-s3-host.com/prefix/") # needs to end wit
133
142
  ```
134
143
 
135
144
  To have the `:host` option passed automatically for every URL, use the
136
- `url_options` plugin.
145
+ `url_options` plugin:
137
146
 
138
147
  ```rb
139
148
  plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
140
149
  ```
141
150
 
151
+ ### Signer
152
+
142
153
  If you would like to [serve private content via CloudFront], you need to sign
143
154
  the object URLs with a special signer, such as [`Aws::CloudFront::UrlSigner`]
144
155
  provided by the `aws-sdk-cloudfront` gem. The S3 storage initializer accepts a
@@ -157,13 +168,27 @@ Shrine::Storage::S3.new(signer: signer.method(:signed_url))
157
168
  Shrine::Storage::S3.new(signer: -> (url, **options) { signer.signed_url(url, **options) })
158
169
  ```
159
170
 
171
+ ## URL options
172
+
173
+ Other than `:host` and `:public` URL options, all additional `S3#url` options
174
+ are forwarded to [`Aws::S3::Object#presigned_url`].
175
+
176
+ ```rb
177
+ s3.url(
178
+ expires_in: 15,
179
+ response_content_disposition: ContentDisposition.attachment("my-filename"),
180
+ response_content_type: "foo/bar",
181
+ # ...
182
+ )
183
+ ```
184
+
160
185
  ## Presigns
161
186
 
162
- The `#presign` method can be used for generating paramters for direct uploads
163
- to Amazon S3:
187
+ The `S3#presign` method can be used for generating parameters for direct upload
188
+ to S3:
164
189
 
165
190
  ```rb
166
- s3.presign("/path/to/file") #=>
191
+ s3.presign("key") #=>
167
192
  # {
168
193
  # url: "https://my-bucket.s3.amazonaws.com/...",
169
194
  # fields: { ... }, # blank for PUT presigns
@@ -172,11 +197,25 @@ s3.presign("/path/to/file") #=>
172
197
  # }
173
198
  ```
174
199
 
175
- Additional presign options can be given in three places:
200
+ By default, parameters for a POST upload is generated, but you can also
201
+ generate PUT upload parameters:
202
+
203
+ ```rb
204
+ s3.presign("key", method: :put)
205
+ ```
176
206
 
177
- * in `Storage::S3#presign` by forwarding options
178
- * in `:upload_options` option on this storage
179
- * in `presign_endpoint` plugin through `:presign_options`
207
+ Any additional options are forwarded to [`Aws::S3::Object#presigned_post`]
208
+ (for POST uploads) and [`Aws::S3::Object#presigned_url`] (for PUT uploads).
209
+
210
+ ```rb
211
+ s3.presign("key", method: :put, content_disposition: "attachment; filename=my-file.txt") #=>
212
+ # {
213
+ # url: "https://my-bucket.s3.amazonaws.com/...",
214
+ # fields: {},
215
+ # headers: { "Content-Disposition" => "attachment; filename=my-file.txt" },
216
+ # method :put,
217
+ # }
218
+ ```
180
219
 
181
220
  ## Large files
182
221
 
@@ -184,33 +223,24 @@ The aws-sdk-s3 gem has the ability to automatically use multipart upload/copy
184
223
  for larger files, splitting the file into multiple chunks and uploading/copying
185
224
  them in parallel.
186
225
 
187
- By default any files that are uploaded will use the multipart upload if they're
188
- larger than 15MB, and any files that are copied will use the multipart copy if
189
- they're larger than 150MB, but you can change the thresholds via
190
- `:multipart_threshold`.
191
-
192
- ```rb
193
- thresholds = { upload: 30*1024*1024, copy: 200*1024*1024 }
194
- Shrine::Storage::S3.new(multipart_threshold: thresholds, **s3_options)
195
- ```
196
-
197
- If you want to change how many threads aws-sdk-s3 will use for multipart
198
- upload/copy, you can use the `upload_options` plugin to specify
199
- `:thread_count`.
226
+ By default, multipart upload will be used for files larger than 15MB, and
227
+ multipart copy for files larger than 100MB, but you can change the thresholds
228
+ via `:multipart_threshold`:
200
229
 
201
230
  ```rb
202
- plugin :upload_options, store: -> (io, context) do
203
- { thread_count: 5 }
204
- end
231
+ Shrine::Storage::S3.new(
232
+ multipart_threshold: { upload: 30*1024*1024, copy: 200*1024*1024 },
233
+ **s3_options,
234
+ )
205
235
  ```
206
236
 
207
237
  ## Encryption
208
238
 
209
- The easiest way to use server-side encryption for uploaded S3 objects is to
239
+ The easiest way to use **server-side** encryption for uploaded S3 objects is to
210
240
  configure default encryption for your S3 bucket. Alternatively, you can pass
211
241
  server-side encryption parameters to the API calls.
212
242
 
213
- The `#upload` method accepts `:sse_*` options:
243
+ The `S3#upload` method accepts `:sse_*` options:
214
244
 
215
245
  ```rb
216
246
  s3.upload(io, "key", sse_customer_algorithm: "AES256",
@@ -219,7 +249,7 @@ s3.upload(io, "key", sse_customer_algorithm: "AES256",
219
249
  ssekms_key_id: "key_id")
220
250
  ```
221
251
 
222
- The `#presign` method accepts `:server_side_encryption_*` options for POST
252
+ The `S3#presign` method accepts `:server_side_encryption_*` options for POST
223
253
  presigns, and the same `:sse_*` options as above for PUT presigns.
224
254
 
225
255
  ```rb
@@ -237,15 +267,14 @@ s3.open("key", sse_customer_algorithm: "AES256",
237
267
  sse_customer_key_md5: "secret_key_md5")
238
268
  ```
239
269
 
240
- If you want to use client-side encryption instead, you can instantiate the
241
- storage with an `Aws::S3::Encryption::Client` instance.
270
+ **Client-side** encryption is supported as well:
242
271
 
243
272
  ```rb
244
- client = Aws::S3::Encryption::Client.new(
245
- kms_key_id: "alias/my-key"
246
- )
273
+ encryption_client = Aws::S3::EncryptionV2::Client.new(...)
274
+ s3 = Shrine::Storage::S3.new(client: encryption_client, **other_options)
247
275
 
248
- Shrine::Storage::S3(client: client, bucket: "my-bucket")
276
+ s3.upload(io, "key") # encrypts on upload
277
+ s3.open("key") # decrypts on download
249
278
  ```
250
279
 
251
280
  ## Accelerate endpoint
@@ -279,20 +308,18 @@ Alternatively you can periodically call the `#clear!` method:
279
308
  s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
280
309
  ```
281
310
 
282
- ## Request Rate and Performance Guidelines
283
-
284
- Amazon S3 automatically scales to high request rates. For example, your
285
- application can achieve at least 3,500 PUT/POST/DELETE and 5,500 GET requests
286
- per second per prefix in a bucket (a prefix is a top-level "directory" in the
287
- bucket). If your app needs to support higher request rates to S3 than that, you
288
- can scale exponentially by using more prefixes.
289
-
311
+ [AWS S3]: https://aws.amazon.com/s3/
312
+ [MinIO]: https://min.io/
313
+ [DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
314
+ [aws-sdk-s3]: https://rubygems.org/gems/aws-sdk-s3
290
315
  [uploading]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#put-instance_method
291
316
  [copying]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#copy_from-instance_method
292
317
  [presigning]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
293
318
  [`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
294
- [aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
295
319
  [Transfer Acceleration]: http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html
296
320
  [object lifecycle]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
297
321
  [serve private content via CloudFront]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html
298
322
  [`Aws::CloudFront::UrlSigner`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/CloudFront/UrlSigner.html
323
+ [credentials]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
324
+ [`Aws::S3::Object#presigned_post`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
325
+ [`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method