shrine 3.0.0.beta2 → 3.0.0.beta3
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of shrine might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +45 -1
- data/README.md +100 -106
- data/doc/advantages.md +90 -88
- data/doc/attacher.md +322 -152
- data/doc/carrierwave.md +105 -113
- data/doc/changing_derivatives.md +308 -0
- data/doc/changing_location.md +92 -21
- data/doc/changing_storage.md +107 -0
- data/doc/creating_plugins.md +1 -1
- data/doc/design.md +8 -9
- data/doc/direct_s3.md +3 -2
- data/doc/metadata.md +97 -78
- data/doc/multiple_files.md +3 -3
- data/doc/paperclip.md +89 -88
- data/doc/plugins/activerecord.md +3 -12
- data/doc/plugins/backgrounding.md +126 -100
- data/doc/plugins/derivation_endpoint.md +4 -5
- data/doc/plugins/derivatives.md +63 -32
- data/doc/plugins/download_endpoint.md +54 -1
- data/doc/plugins/entity.md +1 -0
- data/doc/plugins/form_assign.md +53 -0
- data/doc/plugins/mirroring.md +37 -16
- data/doc/plugins/multi_cache.md +22 -0
- data/doc/plugins/presign_endpoint.md +1 -1
- data/doc/plugins/remote_url.md +19 -4
- data/doc/plugins/validation.md +83 -0
- data/doc/processing.md +149 -133
- data/doc/refile.md +68 -63
- data/doc/release_notes/3.0.0.md +835 -0
- data/doc/securing_uploads.md +56 -36
- data/doc/storage/s3.md +2 -2
- data/doc/testing.md +104 -120
- data/doc/upgrading_to_3.md +538 -0
- data/doc/validation.md +48 -87
- data/lib/shrine.rb +7 -4
- data/lib/shrine/attacher.rb +16 -6
- data/lib/shrine/plugins/activerecord.rb +33 -14
- data/lib/shrine/plugins/atomic_helpers.rb +1 -1
- data/lib/shrine/plugins/backgrounding.rb +23 -89
- data/lib/shrine/plugins/data_uri.rb +13 -2
- data/lib/shrine/plugins/derivation_endpoint.rb +7 -11
- data/lib/shrine/plugins/derivatives.rb +44 -20
- data/lib/shrine/plugins/download_endpoint.rb +26 -0
- data/lib/shrine/plugins/form_assign.rb +6 -3
- data/lib/shrine/plugins/keep_files.rb +2 -2
- data/lib/shrine/plugins/mirroring.rb +62 -22
- data/lib/shrine/plugins/model.rb +2 -2
- data/lib/shrine/plugins/multi_cache.rb +27 -0
- data/lib/shrine/plugins/remote_url.rb +25 -10
- data/lib/shrine/plugins/remove_invalid.rb +1 -1
- data/lib/shrine/plugins/sequel.rb +39 -20
- data/lib/shrine/plugins/validation.rb +3 -0
- data/lib/shrine/storage/s3.rb +16 -1
- data/lib/shrine/uploaded_file.rb +1 -0
- data/lib/shrine/version.rb +1 -1
- data/shrine.gemspec +1 -1
- metadata +12 -7
- data/doc/migrating_storage.md +0 -76
- data/doc/regenerating_versions.md +0 -143
- data/lib/shrine/plugins/attacher_options.rb +0 -55
data/doc/validation.md
CHANGED
@@ -1,20 +1,23 @@
|
|
1
1
|
# File Validation
|
2
2
|
|
3
|
-
Shrine allows validating assigned files
|
4
|
-
code is defined inside
|
3
|
+
Shrine allows validating assigned files using the [`validation`][validation]
|
4
|
+
plugin. Validation code is defined inside an `Attacher.validate` block:
|
5
5
|
|
6
|
+
```rb
|
7
|
+
Shrine.plugin :validation
|
8
|
+
```
|
6
9
|
```rb
|
7
10
|
class ImageUploader < Shrine
|
8
11
|
Attacher.validate do
|
9
|
-
#
|
12
|
+
# ... perform validation ...
|
10
13
|
end
|
11
14
|
end
|
12
15
|
```
|
13
16
|
|
14
|
-
The validation block is run when a file is assigned
|
15
|
-
|
16
|
-
|
17
|
-
|
17
|
+
The validation block is run when a new file is assigned, and any validation
|
18
|
+
errors are stored in `Shrine::Attacher#errors`. ORM plugins like `sequel` and
|
19
|
+
`activerecord` will automatically merge these validation errors into the
|
20
|
+
`#errors` hash on the model instance.
|
18
21
|
|
19
22
|
```rb
|
20
23
|
photo = Photo.new
|
@@ -23,57 +26,20 @@ photo.valid? #=> false
|
|
23
26
|
photo.errors[:image] #=> [...]
|
24
27
|
```
|
25
28
|
|
26
|
-
|
27
|
-
but you can have it automatically removed and deleted by loading the
|
28
|
-
`remove_invalid` plugin.
|
29
|
-
|
30
|
-
```rb
|
31
|
-
Shrine.plugin :remove_invalid # remove and delete files that failed validation
|
32
|
-
```
|
33
|
-
|
34
|
-
The validation block is evaluated in the context of a `Shrine::Attacher`
|
35
|
-
instance, so you have access to the original file and the record:
|
36
|
-
|
37
|
-
```rb
|
38
|
-
class ImageUploader < Shrine
|
39
|
-
Attacher.validate do
|
40
|
-
self #=> #<Shrine::Attacher>
|
41
|
-
|
42
|
-
get #=> #<Shrine::UploadedFile>
|
43
|
-
record #=> #<Photo>
|
44
|
-
name #=> :image
|
45
|
-
end
|
46
|
-
end
|
47
|
-
```
|
29
|
+
## Validation helpers
|
48
30
|
|
49
|
-
|
50
|
-
for
|
31
|
+
The [`validation_helpers`][validation_helpers] plugin provides convenient
|
32
|
+
validators for built-in metadata:
|
51
33
|
|
52
34
|
```rb
|
53
|
-
|
35
|
+
Shrine.plugin :validation_helpers
|
54
36
|
```
|
55
37
|
```rb
|
56
38
|
class ImageUploader < Shrine
|
57
39
|
Attacher.validate do
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
```
|
62
|
-
|
63
|
-
## Validation helpers
|
64
|
-
|
65
|
-
The `validation_helpers` plugin provides helper methods for validating common
|
66
|
-
metadata values:
|
67
|
-
|
68
|
-
```rb
|
69
|
-
class ImageUploader < Shrine
|
70
|
-
plugin :validation_helpers
|
71
|
-
|
72
|
-
Attacher.validate do
|
73
|
-
validate_min_size 1, message: "must not be empty"
|
74
|
-
validate_max_size 5*1024*1024, message: "is too large (max is 5 MB)"
|
75
|
-
validate_mime_type_inclusion %w[image/jpeg image/png image/tiff]
|
76
|
-
validate_extension_inclusion %w[jpg jpeg png tiff tif]
|
40
|
+
validate_size 1..5*1024*1024
|
41
|
+
validate_mime_type %w[image/jpeg image/png image/webp image/tiff]
|
42
|
+
validate_extension %w[jpg jpeg png webp tiff tif]
|
77
43
|
end
|
78
44
|
end
|
79
45
|
```
|
@@ -81,47 +47,31 @@ end
|
|
81
47
|
Note that for secure MIME type validation it's recommended to also load
|
82
48
|
`determine_mime_type` and `restore_cached_data` plugins.
|
83
49
|
|
84
|
-
|
85
|
-
|
86
|
-
```rb
|
87
|
-
class ImageUploader < Shrine
|
88
|
-
plugin :validation_helpers
|
89
|
-
|
90
|
-
Attacher.validate do
|
91
|
-
# validate dimensions only of the attached file is an image
|
92
|
-
if validate_extension_inclusion %w[jpg jpeg png tiff tif]
|
93
|
-
validate_max_width 5000
|
94
|
-
validate_max_height 5000
|
95
|
-
end
|
96
|
-
end
|
97
|
-
end
|
98
|
-
```
|
99
|
-
|
100
|
-
See the `validation_helpers` plugin documentation for more details.
|
50
|
+
See the [`validation_helpers`][validation_helpers] plugin documentation for
|
51
|
+
more details.
|
101
52
|
|
102
53
|
## Custom validations
|
103
54
|
|
104
|
-
You
|
105
|
-
validation that the `validation_helpers` plugin does not provide. The
|
106
|
-
`Shrine::Attacher.validate` block is evaluated at instance level, so you're
|
107
|
-
free to write there any code you like and add validation errors onto the
|
108
|
-
`Shrine::Attacher#errors` array.
|
109
|
-
|
110
|
-
For example, if you're uploading images, you might want to validate that the
|
111
|
-
image is processable using the [ImageProcessing] gem:
|
55
|
+
You can also do your own custom validations:
|
112
56
|
|
113
57
|
```rb
|
114
|
-
|
58
|
+
# Gemfile
|
59
|
+
gem "streamio-ffmpeg"
|
60
|
+
```
|
61
|
+
```rb
|
62
|
+
require "streamio-ffmpeg"
|
115
63
|
|
116
|
-
class
|
117
|
-
plugin :
|
64
|
+
class VideoUploader < Shrine
|
65
|
+
plugin :add_metadata
|
66
|
+
|
67
|
+
add_metadata :duration do |io|
|
68
|
+
movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
|
69
|
+
movie.duration
|
70
|
+
end
|
118
71
|
|
119
72
|
Attacher.validate do
|
120
|
-
|
121
|
-
|
122
|
-
get.download do |tempfile|
|
123
|
-
errors << "is corrupted or invalid" unless ImageProcessing::MiniMagick.valid_image?(tempfile)
|
124
|
-
end
|
73
|
+
if file.duration > 5*60*60
|
74
|
+
errors << "duration must not be longer than 5 hours"
|
125
75
|
end
|
126
76
|
end
|
127
77
|
end
|
@@ -134,15 +84,26 @@ when defining more validations:
|
|
134
84
|
|
135
85
|
```rb
|
136
86
|
class ApplicationUploader < Shrine
|
137
|
-
Attacher.validate { validate_max_size 5
|
87
|
+
Attacher.validate { validate_max_size 5*1024*1024 }
|
138
88
|
end
|
139
89
|
|
140
90
|
class ImageUploader < ApplicationUploader
|
141
91
|
Attacher.validate do
|
142
92
|
super() # empty braces are required
|
143
|
-
|
93
|
+
validate_mime_type %w[image/jpeg image/png image/webp]
|
144
94
|
end
|
145
95
|
end
|
146
96
|
```
|
147
97
|
|
148
|
-
|
98
|
+
## Removing invalid files
|
99
|
+
|
100
|
+
By default, an invalid file will remain assigned after validation failed, but
|
101
|
+
you can have it automatically removed and deleted by loading the
|
102
|
+
`remove_invalid` plugin.
|
103
|
+
|
104
|
+
```rb
|
105
|
+
Shrine.plugin :remove_invalid # remove and delete files that failed validation
|
106
|
+
```
|
107
|
+
|
108
|
+
[validation]: /doc/plugins/validation.md#readme
|
109
|
+
[validation_helpers]: /doc/plugins/validation_helpers.md#readme
|
data/lib/shrine.rb
CHANGED
@@ -178,15 +178,18 @@ class Shrine
|
|
178
178
|
# The symbol identifier for the storage used by the uploader.
|
179
179
|
attr_reader :storage_key
|
180
180
|
|
181
|
-
# The storage object used by the uploader.
|
182
|
-
attr_reader :storage
|
183
|
-
|
184
181
|
# Accepts a storage symbol registered in `Shrine.storages`.
|
185
182
|
#
|
186
183
|
# Shrine.new(:store)
|
187
184
|
def initialize(storage_key)
|
188
|
-
@storage = self.class.find_storage(storage_key)
|
189
185
|
@storage_key = storage_key.to_sym
|
186
|
+
|
187
|
+
storage # ensure storage is registered
|
188
|
+
end
|
189
|
+
|
190
|
+
# Returns the storage object referenced by the identifier.
|
191
|
+
def storage
|
192
|
+
self.class.find_storage(storage_key)
|
190
193
|
end
|
191
194
|
|
192
195
|
# The main method for uploading files. Takes an IO-like object and an
|
data/lib/shrine/attacher.rb
CHANGED
@@ -154,7 +154,7 @@ class Shrine
|
|
154
154
|
# attacher.promote_cached
|
155
155
|
# attacher.stored? #=> true
|
156
156
|
def promote_cached(**options)
|
157
|
-
promote(action: :store, **options) if
|
157
|
+
promote(action: :store, **options) if promote?
|
158
158
|
end
|
159
159
|
|
160
160
|
# Uploads current file to permanent storage and sets the stored file.
|
@@ -183,8 +183,8 @@ class Shrine
|
|
183
183
|
# attacher.attach(file)
|
184
184
|
# attacher.destroy_previous
|
185
185
|
# previous_file.exists? #=> false
|
186
|
-
def destroy_previous
|
187
|
-
@previous.destroy_attached
|
186
|
+
def destroy_previous
|
187
|
+
@previous.destroy_attached if changed?
|
188
188
|
end
|
189
189
|
|
190
190
|
# Destroys the attached file if it exists and is uploaded to permanent
|
@@ -193,8 +193,8 @@ class Shrine
|
|
193
193
|
# attacher.file.exists? #=> true
|
194
194
|
# attacher.destroy_attached
|
195
195
|
# attacher.file.exists? #=> false
|
196
|
-
def destroy_attached
|
197
|
-
destroy
|
196
|
+
def destroy_attached
|
197
|
+
destroy if destroy?
|
198
198
|
end
|
199
199
|
|
200
200
|
# Destroys the attachment.
|
@@ -202,7 +202,7 @@ class Shrine
|
|
202
202
|
# attacher.file.exists? #=> true
|
203
203
|
# attacher.destroy
|
204
204
|
# attacher.file.exists? #=> false
|
205
|
-
def destroy
|
205
|
+
def destroy
|
206
206
|
file&.delete
|
207
207
|
end
|
208
208
|
|
@@ -358,6 +358,16 @@ class Shrine
|
|
358
358
|
uploaded_file
|
359
359
|
end
|
360
360
|
|
361
|
+
# Whether attached file should be uploaded to permanent storage.
|
362
|
+
def promote?
|
363
|
+
changed? && cached?
|
364
|
+
end
|
365
|
+
|
366
|
+
# Whether attached file should be deleted.
|
367
|
+
def destroy?
|
368
|
+
attached? && !cached?
|
369
|
+
end
|
370
|
+
|
361
371
|
# Returns whether the file is uploaded to specified storage.
|
362
372
|
def uploaded?(file, storage_key)
|
363
373
|
file&.storage_key == storage_key
|
@@ -27,34 +27,24 @@ class Shrine
|
|
27
27
|
name = @name
|
28
28
|
|
29
29
|
if shrine_class.opts[:activerecord][:validations]
|
30
|
-
# add validation plugin integration
|
31
30
|
model.validate do
|
32
|
-
|
33
|
-
|
34
|
-
send(:"#{name}_attacher").errors.each do |message|
|
35
|
-
errors.add(name, *message)
|
36
|
-
end
|
31
|
+
send(:"#{name}_attacher").send(:activerecord_validate)
|
37
32
|
end
|
38
33
|
end
|
39
34
|
|
40
35
|
if shrine_class.opts[:activerecord][:callbacks]
|
41
36
|
model.before_save do
|
42
|
-
|
43
|
-
send(:"#{name}_attacher").save
|
44
|
-
end
|
37
|
+
send(:"#{name}_attacher").send(:activerecord_before_save)
|
45
38
|
end
|
46
39
|
|
47
40
|
[:create, :update].each do |action|
|
48
41
|
model.after_commit on: action do
|
49
|
-
|
50
|
-
send(:"#{name}_attacher").finalize
|
51
|
-
send(:"#{name}_attacher").persist
|
52
|
-
end
|
42
|
+
send(:"#{name}_attacher").send(:activerecord_after_save)
|
53
43
|
end
|
54
44
|
end
|
55
45
|
|
56
46
|
model.after_commit on: :destroy do
|
57
|
-
send(:"#{name}_attacher").
|
47
|
+
send(:"#{name}_attacher").send(:activerecord_after_destroy)
|
58
48
|
end
|
59
49
|
end
|
60
50
|
|
@@ -77,6 +67,35 @@ class Shrine
|
|
77
67
|
module AttacherMethods
|
78
68
|
private
|
79
69
|
|
70
|
+
# Adds file validation errors to the model. Called on model validation.
|
71
|
+
def activerecord_validate
|
72
|
+
return unless respond_to?(:errors)
|
73
|
+
|
74
|
+
errors.each do |message|
|
75
|
+
record.errors.add(name, *message)
|
76
|
+
end
|
77
|
+
end
|
78
|
+
|
79
|
+
# Calls Attacher#save. Called before model save.
|
80
|
+
def activerecord_before_save
|
81
|
+
return unless changed?
|
82
|
+
|
83
|
+
save
|
84
|
+
end
|
85
|
+
|
86
|
+
# Finalizes attachment and persists changes. Called after model save.
|
87
|
+
def activerecord_after_save
|
88
|
+
return unless changed?
|
89
|
+
|
90
|
+
finalize
|
91
|
+
persist
|
92
|
+
end
|
93
|
+
|
94
|
+
# Deletes attached files. Called after model destroy.
|
95
|
+
def activerecord_after_destroy
|
96
|
+
destroy_attached
|
97
|
+
end
|
98
|
+
|
80
99
|
# Saves changes to the model instance, skipping validations. Used by
|
81
100
|
# the _persistence plugin.
|
82
101
|
def activerecord_persist
|
@@ -2,75 +2,9 @@
|
|
2
2
|
|
3
3
|
class Shrine
|
4
4
|
module Plugins
|
5
|
-
#
|
6
|
-
# background job.
|
5
|
+
# Documentation lives in [doc/plugins/backgrounding.md] on GitHub.
|
7
6
|
#
|
8
|
-
#
|
9
|
-
# attacher, and they will be called as needed.
|
10
|
-
#
|
11
|
-
# ## Promotion
|
12
|
-
#
|
13
|
-
# attacher.promote_block do
|
14
|
-
# Attachment::PromoteJob.perform_async(record, name, file_data)
|
15
|
-
# end
|
16
|
-
#
|
17
|
-
# attacher.assign(io)
|
18
|
-
# attacher.finalize # promote block called
|
19
|
-
#
|
20
|
-
# attacher.file # cached file
|
21
|
-
# # ... background job finishes ...
|
22
|
-
# attacher.file # stored file
|
23
|
-
#
|
24
|
-
# The promote worker can be implemented like this:
|
25
|
-
#
|
26
|
-
# class Attachment::PromoteJob
|
27
|
-
# def perform(record, name, file_data)
|
28
|
-
# attacher = Shrine::Attacher.retrieve(model: record, name: name, file: file_data)
|
29
|
-
# attacher.atomic_promote
|
30
|
-
# end
|
31
|
-
# end
|
32
|
-
#
|
33
|
-
# ## Deletion
|
34
|
-
#
|
35
|
-
# attacher.destroy_block do
|
36
|
-
# Attachment::DestroyJob.perform_async(data)
|
37
|
-
# end
|
38
|
-
#
|
39
|
-
# previous_file = attacher.file
|
40
|
-
#
|
41
|
-
# attacher.attach(io)
|
42
|
-
# attacher.finalize # delete hook called
|
43
|
-
#
|
44
|
-
# previous_file.exists? #=> true
|
45
|
-
# # ... background job finishes ...
|
46
|
-
# previous_file.exists? #=> false
|
47
|
-
#
|
48
|
-
# attacher.destroy_attached
|
49
|
-
#
|
50
|
-
# attacher.file.exists? #=> true
|
51
|
-
# # ... background job finishes ...
|
52
|
-
# attacher.file.exists? #=> false
|
53
|
-
#
|
54
|
-
# The delete worker can be implemented like this:
|
55
|
-
#
|
56
|
-
# class Attachment::DestroyJob
|
57
|
-
# def perform(data)
|
58
|
-
# attacher = Shrine::Attacher.from_data(data)
|
59
|
-
# attacher.destroy
|
60
|
-
# end
|
61
|
-
# end
|
62
|
-
#
|
63
|
-
# ## Global hooks
|
64
|
-
#
|
65
|
-
# You can also register promotion and deletion hooks globally:
|
66
|
-
#
|
67
|
-
# Shrine::Attacher.promote_block do
|
68
|
-
# Attachment::PromoteJob.perform_async(record, name, file_data)
|
69
|
-
# end
|
70
|
-
#
|
71
|
-
# Shrine::Attacher.destroy_block do
|
72
|
-
# Attachment::DestroyJob.perform_async(data)
|
73
|
-
# end
|
7
|
+
# [doc/plugins/backgrounding.md]: https://github.com/shrinerb/shrine/blob/master/doc/plugins/backgrounding.md
|
74
8
|
module Backgrounding
|
75
9
|
def self.configure(uploader)
|
76
10
|
uploader.opts[:backgrounding] ||= {}
|
@@ -134,38 +68,38 @@ class Shrine
|
|
134
68
|
@destroy_block
|
135
69
|
end
|
136
70
|
|
137
|
-
#
|
138
|
-
# registered.
|
71
|
+
# Does a background promote if promote block was registered.
|
139
72
|
def promote_cached(**options)
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
# Calls the promotion hook if registered and called via #promote_cached,
|
144
|
-
# otherwise promotes synchronously.
|
145
|
-
def promote(background: false, **options)
|
146
|
-
if promote_block && background
|
147
|
-
background_block(promote_block, **options)
|
73
|
+
if promote? && promote_block
|
74
|
+
promote_background(**options)
|
148
75
|
else
|
149
|
-
super
|
76
|
+
super
|
150
77
|
end
|
151
78
|
end
|
152
79
|
|
153
|
-
#
|
154
|
-
|
155
|
-
|
156
|
-
|
80
|
+
# Calls the registered promote block.
|
81
|
+
def promote_background(**options)
|
82
|
+
fail Error, "promote block is not registered" unless promote_block
|
83
|
+
|
84
|
+
background_block(promote_block, **options)
|
157
85
|
end
|
158
86
|
|
159
|
-
#
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
background_block(destroy_block, **options)
|
87
|
+
# Does a background destroy if destroy block was registered.
|
88
|
+
def destroy_attached
|
89
|
+
if destroy? && destroy_block
|
90
|
+
destroy_background
|
164
91
|
else
|
165
|
-
super
|
92
|
+
super
|
166
93
|
end
|
167
94
|
end
|
168
95
|
|
96
|
+
# Calls the registered destroy block.
|
97
|
+
def destroy_background
|
98
|
+
fail Error, "destroy block is not registered" unless destroy_block
|
99
|
+
|
100
|
+
background_block(destroy_block)
|
101
|
+
end
|
102
|
+
|
169
103
|
private
|
170
104
|
|
171
105
|
def background_block(block, **options)
|