shrine 3.0.0.beta2 → 3.0.0.beta3

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

Files changed (61) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +45 -1
  3. data/README.md +100 -106
  4. data/doc/advantages.md +90 -88
  5. data/doc/attacher.md +322 -152
  6. data/doc/carrierwave.md +105 -113
  7. data/doc/changing_derivatives.md +308 -0
  8. data/doc/changing_location.md +92 -21
  9. data/doc/changing_storage.md +107 -0
  10. data/doc/creating_plugins.md +1 -1
  11. data/doc/design.md +8 -9
  12. data/doc/direct_s3.md +3 -2
  13. data/doc/metadata.md +97 -78
  14. data/doc/multiple_files.md +3 -3
  15. data/doc/paperclip.md +89 -88
  16. data/doc/plugins/activerecord.md +3 -12
  17. data/doc/plugins/backgrounding.md +126 -100
  18. data/doc/plugins/derivation_endpoint.md +4 -5
  19. data/doc/plugins/derivatives.md +63 -32
  20. data/doc/plugins/download_endpoint.md +54 -1
  21. data/doc/plugins/entity.md +1 -0
  22. data/doc/plugins/form_assign.md +53 -0
  23. data/doc/plugins/mirroring.md +37 -16
  24. data/doc/plugins/multi_cache.md +22 -0
  25. data/doc/plugins/presign_endpoint.md +1 -1
  26. data/doc/plugins/remote_url.md +19 -4
  27. data/doc/plugins/validation.md +83 -0
  28. data/doc/processing.md +149 -133
  29. data/doc/refile.md +68 -63
  30. data/doc/release_notes/3.0.0.md +835 -0
  31. data/doc/securing_uploads.md +56 -36
  32. data/doc/storage/s3.md +2 -2
  33. data/doc/testing.md +104 -120
  34. data/doc/upgrading_to_3.md +538 -0
  35. data/doc/validation.md +48 -87
  36. data/lib/shrine.rb +7 -4
  37. data/lib/shrine/attacher.rb +16 -6
  38. data/lib/shrine/plugins/activerecord.rb +33 -14
  39. data/lib/shrine/plugins/atomic_helpers.rb +1 -1
  40. data/lib/shrine/plugins/backgrounding.rb +23 -89
  41. data/lib/shrine/plugins/data_uri.rb +13 -2
  42. data/lib/shrine/plugins/derivation_endpoint.rb +7 -11
  43. data/lib/shrine/plugins/derivatives.rb +44 -20
  44. data/lib/shrine/plugins/download_endpoint.rb +26 -0
  45. data/lib/shrine/plugins/form_assign.rb +6 -3
  46. data/lib/shrine/plugins/keep_files.rb +2 -2
  47. data/lib/shrine/plugins/mirroring.rb +62 -22
  48. data/lib/shrine/plugins/model.rb +2 -2
  49. data/lib/shrine/plugins/multi_cache.rb +27 -0
  50. data/lib/shrine/plugins/remote_url.rb +25 -10
  51. data/lib/shrine/plugins/remove_invalid.rb +1 -1
  52. data/lib/shrine/plugins/sequel.rb +39 -20
  53. data/lib/shrine/plugins/validation.rb +3 -0
  54. data/lib/shrine/storage/s3.rb +16 -1
  55. data/lib/shrine/uploaded_file.rb +1 -0
  56. data/lib/shrine/version.rb +1 -1
  57. data/shrine.gemspec +1 -1
  58. metadata +12 -7
  59. data/doc/migrating_storage.md +0 -76
  60. data/doc/regenerating_versions.md +0 -143
  61. data/lib/shrine/plugins/attacher_options.rb +0 -55
@@ -1,20 +1,23 @@
1
1
  # File Validation
2
2
 
3
- Shrine allows validating assigned files based on their metadata. Validation
4
- code is defined inside a `Shrine::Attacher.validate` block:
3
+ Shrine allows validating assigned files using the [`validation`][validation]
4
+ plugin. Validation code is defined inside an `Attacher.validate` block:
5
5
 
6
+ ```rb
7
+ Shrine.plugin :validation
8
+ ```
6
9
  ```rb
7
10
  class ImageUploader < Shrine
8
11
  Attacher.validate do
9
- # validations
12
+ # ... perform validation ...
10
13
  end
11
14
  end
12
15
  ```
13
16
 
14
- The validation block is run when a file is assigned to an attachment attribute,
15
- afterwards the validation errors are stored in `Shrine::Attacher#errors`. ORM
16
- plugins like `sequel` and `activerecord` will automatically merge these
17
- validation errors into the `#errors` hash on the model instance.
17
+ The validation block is run when a new file is assigned, and any validation
18
+ errors are stored in `Shrine::Attacher#errors`. ORM plugins like `sequel` and
19
+ `activerecord` will automatically merge these validation errors into the
20
+ `#errors` hash on the model instance.
18
21
 
19
22
  ```rb
20
23
  photo = Photo.new
@@ -23,57 +26,20 @@ photo.valid? #=> false
23
26
  photo.errors[:image] #=> [...]
24
27
  ```
25
28
 
26
- By default the invalid file will remain assigned to the attachment attribute,
27
- but you can have it automatically removed and deleted by loading the
28
- `remove_invalid` plugin.
29
-
30
- ```rb
31
- Shrine.plugin :remove_invalid # remove and delete files that failed validation
32
- ```
33
-
34
- The validation block is evaluated in the context of a `Shrine::Attacher`
35
- instance, so you have access to the original file and the record:
36
-
37
- ```rb
38
- class ImageUploader < Shrine
39
- Attacher.validate do
40
- self #=> #<Shrine::Attacher>
41
-
42
- get #=> #<Shrine::UploadedFile>
43
- record #=> #<Photo>
44
- name #=> :image
45
- end
46
- end
47
- ```
29
+ ## Validation helpers
48
30
 
49
- You can use the attacher context to pass additional parameters you want to use
50
- for validation:
31
+ The [`validation_helpers`][validation_helpers] plugin provides convenient
32
+ validators for built-in metadata:
51
33
 
52
34
  ```rb
53
- photo.image_attacher.context[:foo] = "bar"
35
+ Shrine.plugin :validation_helpers
54
36
  ```
55
37
  ```rb
56
38
  class ImageUploader < Shrine
57
39
  Attacher.validate do
58
- context[:foo] #=> "bar"
59
- end
60
- end
61
- ```
62
-
63
- ## Validation helpers
64
-
65
- The `validation_helpers` plugin provides helper methods for validating common
66
- metadata values:
67
-
68
- ```rb
69
- class ImageUploader < Shrine
70
- plugin :validation_helpers
71
-
72
- Attacher.validate do
73
- validate_min_size 1, message: "must not be empty"
74
- validate_max_size 5*1024*1024, message: "is too large (max is 5 MB)"
75
- validate_mime_type_inclusion %w[image/jpeg image/png image/tiff]
76
- validate_extension_inclusion %w[jpg jpeg png tiff tif]
40
+ validate_size 1..5*1024*1024
41
+ validate_mime_type %w[image/jpeg image/png image/webp image/tiff]
42
+ validate_extension %w[jpg jpeg png webp tiff tif]
77
43
  end
78
44
  end
79
45
  ```
@@ -81,47 +47,31 @@ end
81
47
  Note that for secure MIME type validation it's recommended to also load
82
48
  `determine_mime_type` and `restore_cached_data` plugins.
83
49
 
84
- It's also easy to do conditional validations with these helper methods:
85
-
86
- ```rb
87
- class ImageUploader < Shrine
88
- plugin :validation_helpers
89
-
90
- Attacher.validate do
91
- # validate dimensions only of the attached file is an image
92
- if validate_extension_inclusion %w[jpg jpeg png tiff tif]
93
- validate_max_width 5000
94
- validate_max_height 5000
95
- end
96
- end
97
- end
98
- ```
99
-
100
- See the `validation_helpers` plugin documentation for more details.
50
+ See the [`validation_helpers`][validation_helpers] plugin documentation for
51
+ more details.
101
52
 
102
53
  ## Custom validations
103
54
 
104
- You might sometimes want to validate custom metadata, or in general do custom
105
- validation that the `validation_helpers` plugin does not provide. The
106
- `Shrine::Attacher.validate` block is evaluated at instance level, so you're
107
- free to write there any code you like and add validation errors onto the
108
- `Shrine::Attacher#errors` array.
109
-
110
- For example, if you're uploading images, you might want to validate that the
111
- image is processable using the [ImageProcessing] gem:
55
+ You can also do your own custom validations:
112
56
 
113
57
  ```rb
114
- require "image_processing/mini_magick"
58
+ # Gemfile
59
+ gem "streamio-ffmpeg"
60
+ ```
61
+ ```rb
62
+ require "streamio-ffmpeg"
115
63
 
116
- class ImageUploader < Shrine
117
- plugin :validation_helpers
64
+ class VideoUploader < Shrine
65
+ plugin :add_metadata
66
+
67
+ add_metadata :duration do |io|
68
+ movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
69
+ movie.duration
70
+ end
118
71
 
119
72
  Attacher.validate do
120
- # validate dimensions only of the attached file is an image
121
- if validate_mime_type_inclusion %w[image/jpeg image/png image/tiff]
122
- get.download do |tempfile|
123
- errors << "is corrupted or invalid" unless ImageProcessing::MiniMagick.valid_image?(tempfile)
124
- end
73
+ if file.duration > 5*60*60
74
+ errors << "duration must not be longer than 5 hours"
125
75
  end
126
76
  end
127
77
  end
@@ -134,15 +84,26 @@ when defining more validations:
134
84
 
135
85
  ```rb
136
86
  class ApplicationUploader < Shrine
137
- Attacher.validate { validate_max_size 5.megabytes }
87
+ Attacher.validate { validate_max_size 5*1024*1024 }
138
88
  end
139
89
 
140
90
  class ImageUploader < ApplicationUploader
141
91
  Attacher.validate do
142
92
  super() # empty braces are required
143
- validate_mime_type_inclusion %w[image/jpeg image/jpg image/png]
93
+ validate_mime_type %w[image/jpeg image/png image/webp]
144
94
  end
145
95
  end
146
96
  ```
147
97
 
148
- [ImageProcessing]: https://github.com/janko/image_processing
98
+ ## Removing invalid files
99
+
100
+ By default, an invalid file will remain assigned after validation failed, but
101
+ you can have it automatically removed and deleted by loading the
102
+ `remove_invalid` plugin.
103
+
104
+ ```rb
105
+ Shrine.plugin :remove_invalid # remove and delete files that failed validation
106
+ ```
107
+
108
+ [validation]: /doc/plugins/validation.md#readme
109
+ [validation_helpers]: /doc/plugins/validation_helpers.md#readme
@@ -178,15 +178,18 @@ class Shrine
178
178
  # The symbol identifier for the storage used by the uploader.
179
179
  attr_reader :storage_key
180
180
 
181
- # The storage object used by the uploader.
182
- attr_reader :storage
183
-
184
181
  # Accepts a storage symbol registered in `Shrine.storages`.
185
182
  #
186
183
  # Shrine.new(:store)
187
184
  def initialize(storage_key)
188
- @storage = self.class.find_storage(storage_key)
189
185
  @storage_key = storage_key.to_sym
186
+
187
+ storage # ensure storage is registered
188
+ end
189
+
190
+ # Returns the storage object referenced by the identifier.
191
+ def storage
192
+ self.class.find_storage(storage_key)
190
193
  end
191
194
 
192
195
  # The main method for uploading files. Takes an IO-like object and an
@@ -154,7 +154,7 @@ class Shrine
154
154
  # attacher.promote_cached
155
155
  # attacher.stored? #=> true
156
156
  def promote_cached(**options)
157
- promote(action: :store, **options) if changed? && cached?
157
+ promote(action: :store, **options) if promote?
158
158
  end
159
159
 
160
160
  # Uploads current file to permanent storage and sets the stored file.
@@ -183,8 +183,8 @@ class Shrine
183
183
  # attacher.attach(file)
184
184
  # attacher.destroy_previous
185
185
  # previous_file.exists? #=> false
186
- def destroy_previous(**options)
187
- @previous.destroy_attached(**options) if changed?
186
+ def destroy_previous
187
+ @previous.destroy_attached if changed?
188
188
  end
189
189
 
190
190
  # Destroys the attached file if it exists and is uploaded to permanent
@@ -193,8 +193,8 @@ class Shrine
193
193
  # attacher.file.exists? #=> true
194
194
  # attacher.destroy_attached
195
195
  # attacher.file.exists? #=> false
196
- def destroy_attached(**options)
197
- destroy(**options) if attached? && !cached?
196
+ def destroy_attached
197
+ destroy if destroy?
198
198
  end
199
199
 
200
200
  # Destroys the attachment.
@@ -202,7 +202,7 @@ class Shrine
202
202
  # attacher.file.exists? #=> true
203
203
  # attacher.destroy
204
204
  # attacher.file.exists? #=> false
205
- def destroy(**options)
205
+ def destroy
206
206
  file&.delete
207
207
  end
208
208
 
@@ -358,6 +358,16 @@ class Shrine
358
358
  uploaded_file
359
359
  end
360
360
 
361
+ # Whether attached file should be uploaded to permanent storage.
362
+ def promote?
363
+ changed? && cached?
364
+ end
365
+
366
+ # Whether attached file should be deleted.
367
+ def destroy?
368
+ attached? && !cached?
369
+ end
370
+
361
371
  # Returns whether the file is uploaded to specified storage.
362
372
  def uploaded?(file, storage_key)
363
373
  file&.storage_key == storage_key
@@ -27,34 +27,24 @@ class Shrine
27
27
  name = @name
28
28
 
29
29
  if shrine_class.opts[:activerecord][:validations]
30
- # add validation plugin integration
31
30
  model.validate do
32
- next unless send(:"#{name}_attacher").respond_to?(:errors)
33
-
34
- send(:"#{name}_attacher").errors.each do |message|
35
- errors.add(name, *message)
36
- end
31
+ send(:"#{name}_attacher").send(:activerecord_validate)
37
32
  end
38
33
  end
39
34
 
40
35
  if shrine_class.opts[:activerecord][:callbacks]
41
36
  model.before_save do
42
- if send(:"#{name}_attacher").changed?
43
- send(:"#{name}_attacher").save
44
- end
37
+ send(:"#{name}_attacher").send(:activerecord_before_save)
45
38
  end
46
39
 
47
40
  [:create, :update].each do |action|
48
41
  model.after_commit on: action do
49
- if send(:"#{name}_attacher").changed?
50
- send(:"#{name}_attacher").finalize
51
- send(:"#{name}_attacher").persist
52
- end
42
+ send(:"#{name}_attacher").send(:activerecord_after_save)
53
43
  end
54
44
  end
55
45
 
56
46
  model.after_commit on: :destroy do
57
- send(:"#{name}_attacher").destroy_attached
47
+ send(:"#{name}_attacher").send(:activerecord_after_destroy)
58
48
  end
59
49
  end
60
50
 
@@ -77,6 +67,35 @@ class Shrine
77
67
  module AttacherMethods
78
68
  private
79
69
 
70
+ # Adds file validation errors to the model. Called on model validation.
71
+ def activerecord_validate
72
+ return unless respond_to?(:errors)
73
+
74
+ errors.each do |message|
75
+ record.errors.add(name, *message)
76
+ end
77
+ end
78
+
79
+ # Calls Attacher#save. Called before model save.
80
+ def activerecord_before_save
81
+ return unless changed?
82
+
83
+ save
84
+ end
85
+
86
+ # Finalizes attachment and persists changes. Called after model save.
87
+ def activerecord_after_save
88
+ return unless changed?
89
+
90
+ finalize
91
+ persist
92
+ end
93
+
94
+ # Deletes attached files. Called after model destroy.
95
+ def activerecord_after_destroy
96
+ destroy_attached
97
+ end
98
+
80
99
  # Saves changes to the model instance, skipping validations. Used by
81
100
  # the _persistence plugin.
82
101
  def activerecord_persist
@@ -54,7 +54,7 @@ class Shrine
54
54
  abstract_atomic_persist(original_file, reload: reload, persist: persist, &block)
55
55
  result
56
56
  rescue Shrine::AttachmentChanged
57
- destroy(background: true)
57
+ destroy_attached
58
58
  raise
59
59
  end
60
60
  end
@@ -2,75 +2,9 @@
2
2
 
3
3
  class Shrine
4
4
  module Plugins
5
- # The backgrounding plugin allows delaying promotion and deletion into a
6
- # background job.
5
+ # Documentation lives in [doc/plugins/backgrounding.md] on GitHub.
7
6
  #
8
- # You can register promotion and deletion blocks on an instance of the
9
- # attacher, and they will be called as needed.
10
- #
11
- # ## Promotion
12
- #
13
- # attacher.promote_block do
14
- # Attachment::PromoteJob.perform_async(record, name, file_data)
15
- # end
16
- #
17
- # attacher.assign(io)
18
- # attacher.finalize # promote block called
19
- #
20
- # attacher.file # cached file
21
- # # ... background job finishes ...
22
- # attacher.file # stored file
23
- #
24
- # The promote worker can be implemented like this:
25
- #
26
- # class Attachment::PromoteJob
27
- # def perform(record, name, file_data)
28
- # attacher = Shrine::Attacher.retrieve(model: record, name: name, file: file_data)
29
- # attacher.atomic_promote
30
- # end
31
- # end
32
- #
33
- # ## Deletion
34
- #
35
- # attacher.destroy_block do
36
- # Attachment::DestroyJob.perform_async(data)
37
- # end
38
- #
39
- # previous_file = attacher.file
40
- #
41
- # attacher.attach(io)
42
- # attacher.finalize # delete hook called
43
- #
44
- # previous_file.exists? #=> true
45
- # # ... background job finishes ...
46
- # previous_file.exists? #=> false
47
- #
48
- # attacher.destroy_attached
49
- #
50
- # attacher.file.exists? #=> true
51
- # # ... background job finishes ...
52
- # attacher.file.exists? #=> false
53
- #
54
- # The delete worker can be implemented like this:
55
- #
56
- # class Attachment::DestroyJob
57
- # def perform(data)
58
- # attacher = Shrine::Attacher.from_data(data)
59
- # attacher.destroy
60
- # end
61
- # end
62
- #
63
- # ## Global hooks
64
- #
65
- # You can also register promotion and deletion hooks globally:
66
- #
67
- # Shrine::Attacher.promote_block do
68
- # Attachment::PromoteJob.perform_async(record, name, file_data)
69
- # end
70
- #
71
- # Shrine::Attacher.destroy_block do
72
- # Attachment::DestroyJob.perform_async(data)
73
- # end
7
+ # [doc/plugins/backgrounding.md]: https://github.com/shrinerb/shrine/blob/master/doc/plugins/backgrounding.md
74
8
  module Backgrounding
75
9
  def self.configure(uploader)
76
10
  uploader.opts[:backgrounding] ||= {}
@@ -134,38 +68,38 @@ class Shrine
134
68
  @destroy_block
135
69
  end
136
70
 
137
- # Signals the #promote method that it should use backgrounding if
138
- # registered.
71
+ # Does a background promote if promote block was registered.
139
72
  def promote_cached(**options)
140
- super(background: true, **options)
141
- end
142
-
143
- # Calls the promotion hook if registered and called via #promote_cached,
144
- # otherwise promotes synchronously.
145
- def promote(background: false, **options)
146
- if promote_block && background
147
- background_block(promote_block, **options)
73
+ if promote? && promote_block
74
+ promote_background(**options)
148
75
  else
149
- super(**options)
76
+ super
150
77
  end
151
78
  end
152
79
 
153
- # Signals the #destroy method that it should use backgrounding if
154
- # registered.
155
- def destroy_attached(**options)
156
- super(background: true, **options)
80
+ # Calls the registered promote block.
81
+ def promote_background(**options)
82
+ fail Error, "promote block is not registered" unless promote_block
83
+
84
+ background_block(promote_block, **options)
157
85
  end
158
86
 
159
- # Calls the destroy hook if registered and called via #destroy_attached,
160
- # otherwise destroys synchronously.
161
- def destroy(background: false, **options)
162
- if destroy_block && background
163
- background_block(destroy_block, **options)
87
+ # Does a background destroy if destroy block was registered.
88
+ def destroy_attached
89
+ if destroy? && destroy_block
90
+ destroy_background
164
91
  else
165
- super(**options)
92
+ super
166
93
  end
167
94
  end
168
95
 
96
+ # Calls the registered destroy block.
97
+ def destroy_background
98
+ fail Error, "destroy block is not registered" unless destroy_block
99
+
100
+ background_block(destroy_block)
101
+ end
102
+
169
103
  private
170
104
 
171
105
  def background_block(block, **options)