shrine 2.18.1 → 2.19.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +53 -1
  3. data/README.md +96 -137
  4. data/doc/advantages.md +4 -4
  5. data/doc/attacher.md +1 -2
  6. data/doc/carrierwave.md +3 -2
  7. data/doc/creating_storages.md +0 -20
  8. data/doc/design.md +1 -1
  9. data/doc/metadata.md +62 -36
  10. data/doc/paperclip.md +7 -6
  11. data/doc/plugins/data_uri.md +50 -4
  12. data/doc/plugins/derivation_endpoint.md +24 -0
  13. data/doc/plugins/determine_mime_type.md +47 -5
  14. data/doc/plugins/infer_extension.md +45 -9
  15. data/doc/plugins/instrumentation.md +170 -0
  16. data/doc/plugins/presign_endpoint.md +1 -1
  17. data/doc/plugins/pretty_location.md +23 -0
  18. data/doc/plugins/remote_url.md +59 -8
  19. data/doc/plugins/signature.md +54 -7
  20. data/doc/plugins/store_dimensions.md +69 -4
  21. data/doc/plugins/upload_endpoint.md +2 -2
  22. data/doc/plugins/validation_helpers.md +71 -29
  23. data/doc/refile.md +1 -1
  24. data/doc/release_notes/2.18.0.md +2 -2
  25. data/doc/release_notes/2.19.0.md +263 -0
  26. data/doc/storage/file_system.md +26 -8
  27. data/doc/testing.md +10 -10
  28. data/lib/shrine.rb +32 -16
  29. data/lib/shrine/attacher.rb +3 -0
  30. data/lib/shrine/attachment.rb +3 -0
  31. data/lib/shrine/plugins/add_metadata.rb +12 -16
  32. data/lib/shrine/plugins/backup.rb +2 -0
  33. data/lib/shrine/plugins/copy.rb +2 -0
  34. data/lib/shrine/plugins/data_uri.rb +56 -28
  35. data/lib/shrine/plugins/derivation_endpoint.rb +61 -27
  36. data/lib/shrine/plugins/determine_mime_type.rb +27 -5
  37. data/lib/shrine/plugins/infer_extension.rb +26 -5
  38. data/lib/shrine/plugins/instrumentation.rb +300 -0
  39. data/lib/shrine/plugins/logging.rb +2 -0
  40. data/lib/shrine/plugins/moving.rb +2 -0
  41. data/lib/shrine/plugins/pretty_location.rb +21 -12
  42. data/lib/shrine/plugins/rack_file.rb +23 -18
  43. data/lib/shrine/plugins/refresh_metadata.rb +4 -4
  44. data/lib/shrine/plugins/remote_url.rb +42 -23
  45. data/lib/shrine/plugins/signature.rb +32 -1
  46. data/lib/shrine/plugins/store_dimensions.rb +54 -9
  47. data/lib/shrine/plugins/validation_helpers.rb +148 -47
  48. data/lib/shrine/storage/file_system.rb +32 -15
  49. data/lib/shrine/storage/linter.rb +0 -13
  50. data/lib/shrine/storage/s3.rb +2 -5
  51. data/lib/shrine/uploaded_file.rb +8 -0
  52. data/lib/shrine/version.rb +2 -2
  53. data/shrine.gemspec +18 -3
  54. metadata +58 -27
@@ -66,7 +66,7 @@ class UploadsController < ApplicationController
66
66
  def image
67
67
  # ... we can perform authentication here ...
68
68
 
69
- set_rack_response ImageUploader.upload_response(:cache, env)
69
+ set_rack_response ImageUploader.upload_response(:cache, request.env)
70
70
  end
71
71
 
72
72
  private
@@ -161,7 +161,7 @@ You can override any of the options above when creating the endpoint/response:
161
161
  ```rb
162
162
  Shrine.upload_endpoint(:cache, max_size: 20*1024*1024)
163
163
  # or
164
- Shrine.upload_response(:cache, env, max_size: 20*1024*1024)
164
+ Shrine.upload_response(:cache, request.env, max_size: 20*1024*1024)
165
165
  ```
166
166
 
167
167
  ## Checksum
@@ -17,84 +17,116 @@ end
17
17
  ### File size
18
18
 
19
19
  The `#validate_max_size`/`#validate_min_size` methods accept a number of bytes,
20
- and validate that the `size` metadata value is not larger or smaller than the
20
+ and validate that the `size` metadata value is not greater/less than the
21
21
  specified size.
22
22
 
23
23
  ```rb
24
- validate_max_size 5*1024*1024 # file must be smaller than 5 MB
25
- validate_min_size 1024 # file must be larger than 1 KB
24
+ validate_max_size 5*1024*1024 # file size must not be greater than 5 MB
25
+ validate_min_size 1024 # file size must not be less than 1 KB
26
+ ```
27
+
28
+ You can also use the `#validate_size` method, which combines these two:
29
+
30
+ ```rb
31
+ validate_size 1024..5*1024*1024 # file size must not be greater than 5 MB nor less than 1 KB
26
32
  ```
27
33
 
28
34
  ### MIME type
29
35
 
30
36
  The `#validate_mime_type_inclusion`/`#validate_mime_type_exclusion` methods
31
37
  accept a list of MIME types, and validate that the `mime_type` metadata value
32
- is (not) a member of that list.
38
+ is/is not a member of that list.
33
39
 
34
40
  ```rb
35
41
  validate_mime_type_inclusion %w[image/jpeg image/png image/gif] # file must be a JPEG, PNG or a GIF image
36
42
  validate_mime_type_exclusion %w[application/x-php] # file must not be a PHP script
37
43
  ```
38
44
 
45
+ Instead of `#validate_mime_type_inclusion` you can also use just
46
+ `#validate_mime_type`.
47
+
39
48
  ### File extension
40
49
 
41
50
  The `#validate_extension_inclusion`/`#validation_extension_exclusion` methods
42
51
  accept a list of file extensions, and validate that the `filename` metadata
43
- value extension is (not) a member of that list.
52
+ value extension is/is not a member of that list.
44
53
 
45
54
  ```rb
46
55
  validate_extension_inclusion %w[jpg jpeg png gif] # file must have .jpg, .jpeg, .png, or .gif extension
47
56
  validate_extension_exclusion %w[php] # file must not have a .php extension
48
57
  ```
49
58
 
59
+ Instead of `#validate_extension_inclusion` you can also use just
60
+ `#validate_extension`.
61
+
50
62
  Since file extension doesn't have to match the type of the file, it's good
51
63
  practice to validate both the file extension and the MIME type.
52
64
 
53
65
  ### Image Dimensions
54
66
 
67
+ These validations validate `width` and `height` metadata values, which are
68
+ extracted by the `store_dimensions` plugin.
69
+
70
+ ```rb
71
+ plugin :store_dimensions
72
+ ```
73
+
74
+ It's good practice to validate dimensions in addition to filesize, as a guard
75
+ against decompression attacks.
76
+
77
+ #### Width
78
+
55
79
  The `#validate_max_width`/`#validate_min_width` methods accept a width in
56
- pixels, and validates that the `width` metadata value is not larger or smaller
80
+ pixels, and validates that the `width` metadata value is not greater/less
57
81
  than the specified number:
58
82
 
59
83
  ```rb
60
- validate_max_width 5000 # image width must be smaller than 5000px
61
- validate_min_width 100 # image width must be larger than 100px
84
+ validate_max_width 5000 # image width must not be greater than 5000px
85
+ validate_min_width 100 # image width must not be less than 100px
62
86
  ```
63
87
 
88
+ You can also use the `#validate_width` method, which combines these two:
89
+
90
+ ```rb
91
+ validate_width 100..5000 # image width must not be greater than 5000px nor less than 100px
92
+ ```
93
+
94
+ #### Height
95
+
64
96
  The `#validate_max_height`/`#validate_min_height` methods accept a height in
65
- pixels, and validates that the `height` metadata value is not larger or smaller
97
+ pixels, and validates that the `height` metadata value is not greater/less
66
98
  than the specified number:
67
99
 
68
100
  ```rb
69
- validate_max_height 5000 # image height must be smaller than 5000px
70
- validate_min_height 100 # image height must be larger than 100px
101
+ validate_max_height 5000 # image height must not be greater than 5000px
102
+ validate_min_height 100 # image height must not be less than 100px
71
103
  ```
72
104
 
73
- It's good practice to validate dimensions in addition to filesize, as a guard
74
- against decompression attacks. Note that these validations only make sense if
75
- the `store_dimensions` plugin is loaded, so that image dimensions are extracted.
105
+ You can also use the `#validate_height` method, which combines these two:
76
106
 
77
107
  ```rb
78
- plugin :store_dimensions
108
+ validate_height 100..5000 # image height must not be greater than 5000px nor less than 100px
79
109
  ```
80
110
 
81
- ## Dynamic evaluation
111
+ #### Width & Height
82
112
 
83
- The validation block is evaluated dynamically in the context of a
84
- `Shrine::Attacher` instance, so you can access the attachment name, record and
85
- context:
113
+ The `#validate_max_dimensions`/`#validate_min_dimensions` methods accept an
114
+ array of width and height in pixels, and validates that the `width` and
115
+ `height` metadata values are not greater/less than the specified numbers:
86
116
 
87
117
  ```rb
88
- Attacher.validate do
89
- self #=> #<Shrine::Attacher>
90
- name #=> :image
91
- record #=> #<Photo>
92
- context #=> { ... }
118
+ validate_max_dimensions [5000, 5000] # image dimensions must not be greater than 5000x5000
119
+ validate_min_dimensions [100, 100] # image dimensions must not be less than 100x100
120
+ ```
93
121
 
94
- # ...
95
- end
122
+ You can also use the `#validate_dimensions` methods, which combines these two:
123
+
124
+ ```rb
125
+ validate_dimensions [100..5000, 100..5000] # image dimensions must not be greater than 5000x5000 nor less than 100x100
96
126
  ```
97
127
 
128
+ ## Dynamic evaluation
129
+
98
130
  The validation methods return whether the validation succeeded, allowing you to
99
131
  easily do conditional validation:
100
132
 
@@ -114,8 +146,18 @@ the `:default_messages` option to the plugin:
114
146
 
115
147
  ```rb
116
148
  plugin :validation_helpers, default_messages: {
117
- max_size: ->(max) { I18n.t("errors.file.max_size", max: max) },
118
- mime_type_inclusion: ->(whitelist) { I18n.t("errors.file.mime_type_inclusion", whitelist: whitelist) },
149
+ max_size: -> (max) { I18n.t("errors.file.max_size", max: max) },
150
+ min_size: -> (max) { I18n.t("errors.file.min_size", min: min) },
151
+ max_width: -> (max) { I18n.t("errors.file.max_width", max: max) },
152
+ min_width: -> (max) { I18n.t("errors.file.min_width", min: min) },
153
+ max_height: -> (max) { I18n.t("errors.file.max_height", max: max) },
154
+ min_height: -> (max) { I18n.t("errors.file.min_height", min: min) },
155
+ max_dimensions: -> (dims) { I18n.t("errors.file.max_dimensions", dims: dims) },
156
+ min_dimensions: -> (dims) { I18n.t("errors.file.min_dimensions", dims: dims) },
157
+ mime_type_inclusion: -> (list) { I18n.t("errors.file.mime_type_inclusion", list: list) },
158
+ mime_type_exclusion: -> (list) { I18n.t("errors.file.mime_type_exclusion", list: list) },
159
+ extension_inclusion: -> (list) { I18n.t("errors.file.extension_inclusion", list: list) },
160
+ extension_exclusion: -> (list) { I18n.t("errors.file.extension_exclusion", list: list) },
119
161
  }
120
162
  ```
121
163
 
@@ -124,7 +166,7 @@ If you would like to change the error message inline, you can pass the
124
166
 
125
167
  ```rb
126
168
  Attacher.validate do
127
- validate_mime_type_inclusion %w[image/jpeg image/png image/gif], message: "must be JPEG, PNG or GIF"
169
+ validate_mime_type %w[image/jpeg image/png image/gif], message: "must be JPEG, PNG or GIF"
128
170
  end
129
171
  ```
130
172
 
data/doc/refile.md CHANGED
@@ -312,7 +312,7 @@ specify the storage that will be used.
312
312
  #### `.logger`
313
313
 
314
314
  ```rb
315
- Shrine.plugin :logging
315
+ Shrine.logger
316
316
  ```
317
317
 
318
318
  #### `.processors`, `.processor`
@@ -17,7 +17,7 @@
17
17
  def image
18
18
  authenticate_user!
19
19
 
20
- set_rack_response ImageUploader.upload_response(:cache, env)
20
+ set_rack_response ImageUploader.upload_response(:cache, request.env)
21
21
  end
22
22
 
23
23
  private
@@ -47,7 +47,7 @@
47
47
  def image
48
48
  authenticate_user!
49
49
 
50
- set_rack_response ImageUploader.presign_response(:cache, env)
50
+ set_rack_response ImageUploader.presign_response(:cache, request.env)
51
51
  end
52
52
 
53
53
  private
@@ -0,0 +1,263 @@
1
+ ## New features
2
+
3
+ * A new `instrumentation` plugin has been added. It sends and logs events for
4
+ various Shrine operations, and provides an API for other plugins to send and
5
+ log their own events.
6
+
7
+ ```rb
8
+ Shrine.plugin :instrumentation
9
+
10
+ uploaded_file = Shrine.upload(io, :store)
11
+ uploaded_file.exists?
12
+ uploaded_file.download
13
+ uploaded_file.delete
14
+ ```
15
+ ```
16
+ Metadata (32ms) – {:storage=>:store, :io=>StringIO, :uploader=>Shrine}
17
+ Upload (1523ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :io=>StringIO, :upload_options=>{}, :uploader=>Shrine}
18
+ Exists (755ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
19
+ Download (1002ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :download_options=>{}, :uploader=>Shrine}
20
+ Delete (700ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
21
+ ```
22
+
23
+ It supports [ActiveSupport::Notifications] (default) and [dry-monitor]
24
+ notification backends out of the box.
25
+
26
+ ```rb
27
+ require "dry-monitor"
28
+
29
+ Shrine.plugin :instrumentation, notifications: Dry::Monitor::Notifications.new(:test)
30
+ ```
31
+
32
+ * Metadata extraction can now be skipped during the upload by passing
33
+ `metadata: false` to the uploader. This is useful in tests, where you might
34
+ want to skip any potentially expensive metadata extraction while creating
35
+ test data.
36
+
37
+ ```rb
38
+ uploaded_file = uploader.upload(io, metadata: false) # skips metadata extraction
39
+ uploaded_file.metadata #=> {}
40
+ ```
41
+
42
+ * Metadata extraction can now be forced during the upload of a
43
+ `Shrine::UploadedFile` object by passing `metadata: true` to the uploader.
44
+
45
+ ```rb
46
+ uploaded_file = uploader.upload(another_uploaded_file, metadata: true) # forces metadata extraction
47
+ uploaded_file.metadata #=> re-extracted metadata
48
+ ```
49
+
50
+ * The `pretty_location` plugin now supports specifying a different identifier.
51
+
52
+ ```rb
53
+ plugin :pretty_location, identifier: "uuid"
54
+ # "user/aa357797-5845-451b-8662-08eecdc9f762/profile_picture/493g82jf23.jpg"
55
+
56
+ plugin :pretty_location, identifier: :email
57
+ # "user/foo@bar.com/profile_picture/493g82jf23.jpg"
58
+ ```
59
+
60
+ * The `store_dimensions` plugin has gained an `:on_error` option for specifying
61
+ how to react when an error occurrs while extracting dimensions.
62
+
63
+ ```rb
64
+ plugin :store_dimensions, on_error: :warn # prints a warning message (default)
65
+ plugin :store_dimensions, on_error: :fail # raises the exception
66
+ plugin :store_dimensions, on_error: :ignore # ignores the exception
67
+ plugin :store_dimensions, on_error: -> (error) { # custom handler
68
+ # report the exception to your exception handler
69
+ }
70
+ ```
71
+
72
+ * The `FileSystem#upload` method now accepts a `:move` option for moving the
73
+ file instead of copying it. This is now preferred over the `moving` plugin.
74
+
75
+ ```rb
76
+ uploader.upload(file, move: true)
77
+ File.exist?(file.path) #=> false
78
+ ```
79
+
80
+ * The `FileSystem#clear!` method can now take a block for more control over
81
+ which files should be deleted.
82
+
83
+ ```rb
84
+ file_system.clear! { |path| path.mtime < Time.now - 7*24*60*60 } # delete files older than 1 week
85
+ ```
86
+
87
+ ## Other improvements
88
+
89
+ * Registering storage objects under string keys now works again (this got
90
+ broken in version 2.18.0).
91
+
92
+ * Several plugins now add their own instrumentation when the `instrumentation`
93
+ plugin has been loaded:
94
+
95
+ | Plugin | Instrumentation |
96
+ | :----- | :-------------- |
97
+ | `derivation_endpoint` | instruments file processing |
98
+ | `determine_mime_type` | instruments analyzing MIME type |
99
+ | `store_dimensions` | instruments extracting image dimensions |
100
+ | `signature` | instruments calculating signature |
101
+ | `infer_extension` | instruments inferring extension |
102
+ | `remote_url` | instruments remote URL downloading |
103
+ | `data_uri` | instruments data URI parsing |
104
+
105
+ * `Shrine.logger` has been added, and any warnings or other messages (such as
106
+ from the `instrumentation` plugin) now go through it. This way you can change
107
+ the logging destination:
108
+
109
+ ```rb
110
+ # log messages into the Rails logger
111
+ Shrine.logger = Rails.logger
112
+ ```
113
+
114
+ * The `store_dimensions` plugin now prints warnings by default when extracting
115
+ dimensions failed.
116
+
117
+ * The `pretty_location` plugin now comes with a `#pretty_location` method which
118
+ you can call for customization.
119
+
120
+ ```rb
121
+ def generate_location(io, record: nil, **context)
122
+ identifier = record.email if record.is_a?(User)
123
+ pretty_location(io, record: record, identifier: identifier, **context)
124
+ end
125
+ ```
126
+
127
+ * The `Shrine::UploadedFile#[]` operator has been added for easier metadata
128
+ retrieving.
129
+
130
+ ```rb
131
+ uploaded_file.metadata["duration"]
132
+ # can now be just
133
+ uploaded_file["duration"]
134
+ ```
135
+
136
+ * The `Shrine.mime_type`, `Shrine.dimensions`, and `Shrine.signature` aliases
137
+ have been added to `determine_mime_type`, `store_dimensions`, and `signature`
138
+ plugins.
139
+
140
+ ```rb
141
+ Shrine.determine_mime_type(io)
142
+ # can now be just
143
+ Shrine.mime_type(io)
144
+ ```
145
+ ```rb
146
+ Shrine.extract_dimensions(io)
147
+ # can now be just
148
+ Shrine.dimensions(io)
149
+ ```
150
+ ```rb
151
+ Shrine.calculate_signature(io)
152
+ # can now be just
153
+ Shrine.signature(io)
154
+ ```
155
+
156
+ * The `#validate_{max,min}_dimensions` validators have been added to the
157
+ `validation_helpers` plugin.
158
+
159
+ ```rb
160
+ validate_min_width 10
161
+ validate_min_height 10
162
+ validate_max_width 5000
163
+ validate_max_height 5000
164
+
165
+ # can now be written as
166
+
167
+ validate_min_dimensions [10, 10]
168
+ validate_max_dimensions [5000, 5000]
169
+
170
+ # which can be additionally shortened to
171
+
172
+ validate_dimensions [10..5000, 10..5000]
173
+ ```
174
+
175
+ * The `#validate_size`, `#validate_width`, and `#validate_height` shorthands
176
+ have been added to the `validation_helpers` plugin.
177
+
178
+ ```rb
179
+ validate_min_size 1024
180
+ validate_max_size 10*1024*1024
181
+
182
+ # can now be shortned to
183
+
184
+ validate_size 1024..10*1024*1024
185
+ ```
186
+ ```rb
187
+ validate_min_width 10
188
+ validate_max_width 5000
189
+
190
+ # can now be shortened to
191
+
192
+ validate_width 10..5000
193
+ ```
194
+ ```rb
195
+ validate_min_height 10
196
+ validate_max_height 5000
197
+
198
+ # can now be shortened to
199
+
200
+ validate_height 10..5000
201
+ ```
202
+
203
+ * The `#validate_mime_type` and `#validate_extension` shorthands have been
204
+ added to the `validation_helpers` plugin.
205
+
206
+ ```rb
207
+ validate_mime_type_inclusion %w[image/jpeg image/png image/webp]
208
+ # can now be just
209
+ validate_mime_type %w[image/jpeg image/png image/webp]
210
+ ```
211
+ ```rb
212
+ validate_extension_inclusion %w[jpeg png webp]
213
+ # can now be just
214
+ validate_extension %w[jpeg png webp]
215
+ ```
216
+
217
+ * Default error messages in `validation_helpers` plugin have been simplified.
218
+
219
+ * You can now call `super` when overriding `Shrine::UploadedFile` methods
220
+ defined by the `add_metadata` plugin.
221
+
222
+ ## Backwards compatibility
223
+
224
+ * The `logging` plugin has been deprecated in favour of the `instrumentation`
225
+ plugin.
226
+
227
+ * The `moving` pluing has been deprecated in favour of the `:move` option to
228
+ `FileSystem#upload`. This means that the `#move` && `#movable?` methods are
229
+ not part of the storage abstraction anymore.
230
+
231
+ * The `backup` plugin has been deprecated over [mirroring uploads] via the
232
+ `instrumentation` plugin.
233
+
234
+ * The `copy` plugin has been deprecated.
235
+
236
+ * The `Shrine::Plugins::DataUri::DataFile` constant from the `data_uri` plugin
237
+ has been renamed to `Shrine::DataFile`.
238
+
239
+ * The `Shrine::Plugins::RackFile::UploadedFile` constant from the `data_uri` plugin
240
+ has been renamed to `Shrine::RackFile`.
241
+
242
+ * The `:older_than` option for `FileSystem#clear!` has been deprecated in
243
+ favour of passing a block.
244
+
245
+ ```rb
246
+ file_system.clear!(older_than: Time.now - 7*24*60*60)
247
+ # should now be replaced with
248
+ file_system.clear! { |path| path.mtime < Time.now - 7*24*60*60 }
249
+ ```
250
+
251
+ * The `FileSystem#upload` method deprecates ignoring unrecognized upload
252
+ options.
253
+
254
+ * The `FileSystem#upload` method doesn't backfill `size` metadata anymore if an
255
+ IO with unknown size is being uploaded via `Shrine#upload`.
256
+
257
+ * Several plugins have changed how they store configuration options internally.
258
+ If you were accessing these options directly, you will need to update your
259
+ code.
260
+
261
+ [ActiveSupport::Notifications]: https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html
262
+ [dry-monitor]: https://github.com/dry-rb/dry-monitor
263
+ [mirroring uploads]: https://github.com/shrinerb/shrine/wiki/Mirroring-Uploads