shrine 3.0.0.beta2 → 3.0.0.beta3
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of shrine might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +45 -1
- data/README.md +100 -106
- data/doc/advantages.md +90 -88
- data/doc/attacher.md +322 -152
- data/doc/carrierwave.md +105 -113
- data/doc/changing_derivatives.md +308 -0
- data/doc/changing_location.md +92 -21
- data/doc/changing_storage.md +107 -0
- data/doc/creating_plugins.md +1 -1
- data/doc/design.md +8 -9
- data/doc/direct_s3.md +3 -2
- data/doc/metadata.md +97 -78
- data/doc/multiple_files.md +3 -3
- data/doc/paperclip.md +89 -88
- data/doc/plugins/activerecord.md +3 -12
- data/doc/plugins/backgrounding.md +126 -100
- data/doc/plugins/derivation_endpoint.md +4 -5
- data/doc/plugins/derivatives.md +63 -32
- data/doc/plugins/download_endpoint.md +54 -1
- data/doc/plugins/entity.md +1 -0
- data/doc/plugins/form_assign.md +53 -0
- data/doc/plugins/mirroring.md +37 -16
- data/doc/plugins/multi_cache.md +22 -0
- data/doc/plugins/presign_endpoint.md +1 -1
- data/doc/plugins/remote_url.md +19 -4
- data/doc/plugins/validation.md +83 -0
- data/doc/processing.md +149 -133
- data/doc/refile.md +68 -63
- data/doc/release_notes/3.0.0.md +835 -0
- data/doc/securing_uploads.md +56 -36
- data/doc/storage/s3.md +2 -2
- data/doc/testing.md +104 -120
- data/doc/upgrading_to_3.md +538 -0
- data/doc/validation.md +48 -87
- data/lib/shrine.rb +7 -4
- data/lib/shrine/attacher.rb +16 -6
- data/lib/shrine/plugins/activerecord.rb +33 -14
- data/lib/shrine/plugins/atomic_helpers.rb +1 -1
- data/lib/shrine/plugins/backgrounding.rb +23 -89
- data/lib/shrine/plugins/data_uri.rb +13 -2
- data/lib/shrine/plugins/derivation_endpoint.rb +7 -11
- data/lib/shrine/plugins/derivatives.rb +44 -20
- data/lib/shrine/plugins/download_endpoint.rb +26 -0
- data/lib/shrine/plugins/form_assign.rb +6 -3
- data/lib/shrine/plugins/keep_files.rb +2 -2
- data/lib/shrine/plugins/mirroring.rb +62 -22
- data/lib/shrine/plugins/model.rb +2 -2
- data/lib/shrine/plugins/multi_cache.rb +27 -0
- data/lib/shrine/plugins/remote_url.rb +25 -10
- data/lib/shrine/plugins/remove_invalid.rb +1 -1
- data/lib/shrine/plugins/sequel.rb +39 -20
- data/lib/shrine/plugins/validation.rb +3 -0
- data/lib/shrine/storage/s3.rb +16 -1
- data/lib/shrine/uploaded_file.rb +1 -0
- data/lib/shrine/version.rb +1 -1
- data/shrine.gemspec +1 -1
- metadata +12 -7
- data/doc/migrating_storage.md +0 -76
- data/doc/regenerating_versions.md +0 -143
- data/lib/shrine/plugins/attacher_options.rb +0 -55
data/doc/carrierwave.md
CHANGED
@@ -65,26 +65,27 @@ deleting files, they also represent the uploaded file. Shrine has a separate
|
|
65
65
|
uploaded_file = ImageUploader.upload(file, :store)
|
66
66
|
uploaded_file #=> #<Shrine::UploadedFile>
|
67
67
|
uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/store/kfds0lg9rer.jpg"
|
68
|
-
uploaded_file.download #=> #<
|
68
|
+
uploaded_file.download #=> #<File:/tmp/path/to/file>
|
69
69
|
```
|
70
70
|
|
71
|
-
|
71
|
+
## Processing
|
72
72
|
|
73
73
|
In contrast to CarrierWave's class-level DSL, in Shrine processing is defined
|
74
|
-
and performed on the instance-level.
|
75
|
-
file or a hash of versions:
|
74
|
+
and performed on the instance-level.
|
76
75
|
|
77
76
|
```rb
|
78
77
|
class ImageUploader < CarrierWave::Uploader::Base
|
79
78
|
include CarrierWave::MiniMagick
|
80
79
|
|
81
|
-
|
80
|
+
version :large do
|
81
|
+
process resize_to_limit: [800, 800]
|
82
|
+
end
|
82
83
|
|
83
84
|
version :medium do
|
84
85
|
process resize_to_limit: [500, 500]
|
85
86
|
end
|
86
87
|
|
87
|
-
version :small
|
88
|
+
version :small do
|
88
89
|
process resize_to_limit: [300, 300]
|
89
90
|
end
|
90
91
|
end
|
@@ -94,39 +95,27 @@ end
|
|
94
95
|
require "image_processing/mini_magick"
|
95
96
|
|
96
97
|
class ImageUploader < Shrine
|
97
|
-
plugin :
|
98
|
-
plugin :versions
|
99
|
-
|
100
|
-
process(:store) do |io, context|
|
101
|
-
versions = {}
|
98
|
+
plugin :derivatives
|
102
99
|
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
versions[:original] = pipeline.resize_to_limit!(800, 800)
|
107
|
-
versions[:medium] = pipeline.resize_to_limit!(500, 500)
|
108
|
-
versions[:small] = pipeline.resize_to_limit!(300, 300)
|
109
|
-
end
|
100
|
+
Attacher.derivatives_processor do |original|
|
101
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
110
102
|
|
111
|
-
|
103
|
+
{
|
104
|
+
large: magick.resize_to_limit!(800, 800),
|
105
|
+
medium: magick.resize_to_limit!(500, 500),
|
106
|
+
small: magick.resize_to_limit!(300, 300),
|
107
|
+
}
|
112
108
|
end
|
113
109
|
end
|
114
110
|
```
|
115
111
|
|
116
|
-
This allows you to fully optimize processing, because you can easily specify
|
117
|
-
which files are processed from which, and even add parallelization.
|
118
|
-
|
119
112
|
CarrierWave performs processing before validations, which is a huge security
|
120
113
|
issue, as it allows users to give arbitrary files to your processing tool, even
|
121
|
-
if you have validations. Shrine
|
114
|
+
if you have validations. With Shrine you can perform processing after
|
115
|
+
validations.
|
122
116
|
|
123
|
-
|
124
|
-
|
125
|
-
Shrine doesn't have a built-in way of regenerating versions, because that has
|
126
|
-
to be written and optimized differently depending on whether you're adding or
|
127
|
-
removing a version, what ORM are you using, how many records there are in the
|
128
|
-
database etc. The [Reprocessing versions] guide provides some useful tips on
|
129
|
-
this task.
|
117
|
+
Shrine doesn't have a built-in way of regenerating versions, but there is an
|
118
|
+
extensive [Managing Derivatives] guide.
|
130
119
|
|
131
120
|
### Validations
|
132
121
|
|
@@ -154,9 +143,9 @@ class ImageUploader < Shrine
|
|
154
143
|
plugin :validation_helpers
|
155
144
|
|
156
145
|
Attacher.validate do
|
157
|
-
|
158
|
-
|
159
|
-
validate_max_size 10*1024*1024
|
146
|
+
validate_extension %w[jpg jpeg gif png]
|
147
|
+
validate_mime_type %w[image/jpeg image/gif image/png]
|
148
|
+
validate_max_size 10*1024*1024
|
160
149
|
end
|
161
150
|
end
|
162
151
|
```
|
@@ -184,7 +173,7 @@ end
|
|
184
173
|
```
|
185
174
|
```rb
|
186
175
|
class Photo < ActiveRecord::Base
|
187
|
-
include ImageUploader::Attachment
|
176
|
+
include ImageUploader::Attachment(:image)
|
188
177
|
end
|
189
178
|
```
|
190
179
|
|
@@ -196,12 +185,12 @@ uses to save storage, location, and metadata of the uploaded file.
|
|
196
185
|
```rb
|
197
186
|
photo.image_data #=>
|
198
187
|
# {
|
199
|
-
# "storage"
|
200
|
-
# "id"
|
201
|
-
# "metadata"
|
202
|
-
# "filename"
|
203
|
-
# "size"
|
204
|
-
# "mime_type"
|
188
|
+
# "storage": "store",
|
189
|
+
# "id": "photo/1/image/0d9o8dk42.png",
|
190
|
+
# "metadata": {
|
191
|
+
# "filename": "nature.png",
|
192
|
+
# "size": 49349138,
|
193
|
+
# "mime_type": "image/png"
|
205
194
|
# }
|
206
195
|
# }
|
207
196
|
|
@@ -218,11 +207,11 @@ Unlike CarrierWave, Shrine will store this information for each processed
|
|
218
207
|
version, making them first-class citizens:
|
219
208
|
|
220
209
|
```rb
|
221
|
-
photo.image
|
222
|
-
photo.image
|
210
|
+
photo.image #=> #<Shrine::UploadedFile>
|
211
|
+
photo.image.width #=> 800
|
223
212
|
|
224
|
-
photo.image
|
225
|
-
photo.image
|
213
|
+
photo.image(:thumb) #=> #<Shrine::UploadedFile>
|
214
|
+
photo.image(:thumb).width #=> 300
|
226
215
|
```
|
227
216
|
|
228
217
|
Also, since CarrierWave stores only the filename, it has to recalculate the
|
@@ -254,6 +243,17 @@ Afterwards we need to make new uploads write to the `image_data` column. This
|
|
254
243
|
can be done by including the below module to all models that have CarrierWave
|
255
244
|
attachments:
|
256
245
|
|
246
|
+
```rb
|
247
|
+
require "shrine"
|
248
|
+
|
249
|
+
Shrine.storages = {
|
250
|
+
cache: ...,
|
251
|
+
store: ...,
|
252
|
+
}
|
253
|
+
|
254
|
+
Shrine.plugin :model
|
255
|
+
Shrine.plugin :derivatives
|
256
|
+
```
|
257
257
|
```rb
|
258
258
|
module CarrierwaveShrineSynchronization
|
259
259
|
def self.included(model)
|
@@ -266,22 +266,16 @@ module CarrierwaveShrineSynchronization
|
|
266
266
|
|
267
267
|
def write_shrine_data(name)
|
268
268
|
uploader = send(name)
|
269
|
+
attacher = Shrine::Attacher.form_model(self, name)
|
269
270
|
|
270
271
|
if read_attribute(name).present?
|
271
|
-
|
272
|
+
attacher.set shrine_file(uploader)
|
272
273
|
|
273
|
-
|
274
|
-
|
275
|
-
uploader.versions.each do |name, version|
|
276
|
-
data[name] = uploader_to_shrine_data(version)
|
277
|
-
end
|
274
|
+
uploader.versions.each do |name, version|
|
275
|
+
attacher.merge_derivatives(name => shrine_file(version))
|
278
276
|
end
|
279
|
-
|
280
|
-
# Remove the `.to_json` if you're using a JSON column, otherwise the JSON
|
281
|
-
# object will be saved as an escaped string.
|
282
|
-
write_attribute(:"#{name}_data", data.to_json)
|
283
277
|
else
|
284
|
-
|
278
|
+
attacher.set nil
|
285
279
|
end
|
286
280
|
end
|
287
281
|
|
@@ -289,15 +283,16 @@ module CarrierwaveShrineSynchronization
|
|
289
283
|
|
290
284
|
# If you'll be using `:prefix` on your Shrine storage, make sure to
|
291
285
|
# subtract it from the path assigned as `:id`.
|
292
|
-
def
|
293
|
-
|
286
|
+
def shrine_file(uploader)
|
287
|
+
name = uploader.mounted_as
|
288
|
+
filename = read_attribute(name)
|
294
289
|
path = uploader.store_path(filename)
|
295
290
|
|
296
|
-
|
297
|
-
storage:
|
298
|
-
id:
|
299
|
-
metadata: { filename
|
300
|
-
|
291
|
+
Shrine.uploaded_file(
|
292
|
+
storage: :store,
|
293
|
+
id: path,
|
294
|
+
metadata: { "filename" => filename },
|
295
|
+
)
|
301
296
|
end
|
302
297
|
end
|
303
298
|
```
|
@@ -332,8 +327,8 @@ your Shrine uploader:
|
|
332
327
|
Shrine.plugin :refresh_metadata
|
333
328
|
|
334
329
|
Photo.find_each do |photo|
|
335
|
-
|
336
|
-
photo.
|
330
|
+
photo.image_attacher.refresh_metadata!
|
331
|
+
photo.save
|
337
332
|
end
|
338
333
|
```
|
339
334
|
|
@@ -358,26 +353,28 @@ end
|
|
358
353
|
|
359
354
|
#### `.process`, `.version`
|
360
355
|
|
361
|
-
|
362
|
-
`Shrine#process` method.
|
363
|
-
|
364
|
-
#### `.before`, `.after`
|
365
|
-
|
366
|
-
In Shrine you can get callbacks by loading the `hooks` plugin. Unlike
|
367
|
-
CarrierWave, and much like Sequel, Shrine implements callbacks by overriding
|
368
|
-
instance methods:
|
356
|
+
Processing is defined by using the `derivatives` plugin:
|
369
357
|
|
370
358
|
```rb
|
371
359
|
class ImageUploader < Shrine
|
372
|
-
plugin :
|
360
|
+
plugin :derivatives
|
361
|
+
|
362
|
+
Attacher.derivatives_processor do |original|
|
363
|
+
magick = ImageProcessing::MiniMagick.source(image)
|
373
364
|
|
374
|
-
|
375
|
-
|
376
|
-
|
365
|
+
{
|
366
|
+
large: magick.resize_to_limit!(800, 800),
|
367
|
+
medium: magick.resize_to_limit!(500, 500),
|
368
|
+
small: magick.resize_to_limit!(300, 300),
|
369
|
+
}
|
377
370
|
end
|
378
371
|
end
|
379
372
|
```
|
380
373
|
|
374
|
+
#### `.before`, `.after`
|
375
|
+
|
376
|
+
There is no Shrine equivalent for CarrierWave's callbacks.
|
377
|
+
|
381
378
|
#### `#store!`, `#cache!`
|
382
379
|
|
383
380
|
In Shrine you store and cache files by passing the corresponding storage to
|
@@ -406,8 +403,9 @@ uploaded_file.download #=> #<Tempfile:/path/to/file>
|
|
406
403
|
In Shrine you call `#url` on uploaded files:
|
407
404
|
|
408
405
|
```rb
|
409
|
-
|
410
|
-
|
406
|
+
photo.image #=> #<Shrine::UploadedFile>
|
407
|
+
photo.image.url #=> "/uploads/398454ujedfggf.jpg"
|
408
|
+
photo.image_url #=> "/uploads/398454ujedfggf.jpg" (shorthand)
|
411
409
|
```
|
412
410
|
|
413
411
|
#### `#identifier`
|
@@ -415,8 +413,8 @@ user.avatar.url #=> "/uploads/398454ujedfggf.jpg"
|
|
415
413
|
This method corresponds to `#original_filename` on the uploaded file:
|
416
414
|
|
417
415
|
```rb
|
418
|
-
|
419
|
-
|
416
|
+
photo.image #=> #<Shrine::UploadedFile>
|
417
|
+
photo.image.original_filename #=> "avatar.jpg"
|
420
418
|
```
|
421
419
|
|
422
420
|
#### `#store_dir`, `#cache_dir`
|
@@ -426,15 +424,14 @@ storages:
|
|
426
424
|
|
427
425
|
```rb
|
428
426
|
class ImageUploader < Shrine
|
429
|
-
def generate_location(io,
|
430
|
-
"#{
|
427
|
+
def generate_location(io, record: nil, **)
|
428
|
+
"#{record.class}/#{record.id}/#{io.original_filename}"
|
431
429
|
end
|
432
430
|
end
|
433
431
|
```
|
434
432
|
|
435
|
-
|
436
|
-
|
437
|
-
for automatically generating an organized folder structure.
|
433
|
+
You might also want to use the `pretty_location` plugin for automatically
|
434
|
+
generating an organized folder structure.
|
438
435
|
|
439
436
|
#### `#default_url`
|
440
437
|
|
@@ -450,12 +447,9 @@ class ImageUploader < Shrine
|
|
450
447
|
end
|
451
448
|
```
|
452
449
|
|
453
|
-
The `context` variable holds the name of the attachment, record instance and
|
454
|
-
in some cases the `:version`.
|
455
|
-
|
456
450
|
#### `#extension_white_list`, `#extension_black_list`
|
457
451
|
|
458
|
-
In Shrine extension whitelisting/blacklisting is a part of validations, and is
|
452
|
+
In Shrine, extension whitelisting/blacklisting is a part of validations, and is
|
459
453
|
provided by the `validation_helpers` plugin:
|
460
454
|
|
461
455
|
```rb
|
@@ -471,7 +465,7 @@ end
|
|
471
465
|
|
472
466
|
#### `#blacklist_mime_type_pattern`, `#whitelist_mime_type_pattern`, `#content_type_whitelist`, `#content_type_blacklist`
|
473
467
|
|
474
|
-
In Shrine MIME type whitelisting/blacklisting is part of validations, and is
|
468
|
+
In Shrine, MIME type whitelisting/blacklisting is part of validations, and is
|
475
469
|
provided by the `validation_helpers` plugin, though it doesn't support regexes:
|
476
470
|
|
477
471
|
```rb
|
@@ -490,14 +484,12 @@ end
|
|
490
484
|
In Shrine file size validations are typically done using the
|
491
485
|
`validation_helpers` plugin:
|
492
486
|
|
493
|
-
|
494
487
|
```rb
|
495
488
|
class ImageUploader < Shrine
|
496
489
|
plugin :validation_helpers
|
497
490
|
|
498
491
|
Attacher.validate do
|
499
|
-
|
500
|
-
validate_max_size 5*1024*1024 # 5 MB
|
492
|
+
validate_size 0..5*1024*1024 # 5 MB
|
501
493
|
end
|
502
494
|
end
|
503
495
|
```
|
@@ -506,13 +498,13 @@ end
|
|
506
498
|
|
507
499
|
Shrine doesn't have a built-in way of regenerating versions, because that's
|
508
500
|
very individual and depends on what versions you want regenerated, what ORM are
|
509
|
-
you using, how many records there are in your database etc. The [
|
510
|
-
|
501
|
+
you using, how many records there are in your database etc. The [Managing
|
502
|
+
Derivatives] guide provides some useful tips on this task.
|
511
503
|
|
512
504
|
### Models
|
513
505
|
|
514
506
|
The only thing that Shrine requires from your models is a `<attachment>_data`
|
515
|
-
column (e.g. if your attachment is "
|
507
|
+
column (e.g. if your attachment is "image", you need the `image_data` column).
|
516
508
|
|
517
509
|
#### `.mount_uploader`
|
518
510
|
|
@@ -523,7 +515,7 @@ Shrine.plugin :sequel
|
|
523
515
|
```
|
524
516
|
```rb
|
525
517
|
class User < Sequel::Model
|
526
|
-
include ImageUploader::Attachment
|
518
|
+
include ImageUploader::Attachment(:avatar)
|
527
519
|
end
|
528
520
|
```
|
529
521
|
|
@@ -532,7 +524,7 @@ end
|
|
532
524
|
The attachment module adds an attachment setter:
|
533
525
|
|
534
526
|
```rb
|
535
|
-
|
527
|
+
photo.image = File.open("avatar.jpg", "rb")
|
536
528
|
```
|
537
529
|
|
538
530
|
Note that unlike CarrierWave, you cannot pass in file paths, the input needs to
|
@@ -544,8 +536,8 @@ CarrierWave returns the uploader, but Shrine returns a `Shrine::UploadedFile`,
|
|
544
536
|
a representation of the file uploaded to the storage:
|
545
537
|
|
546
538
|
```rb
|
547
|
-
|
548
|
-
|
539
|
+
photo.image #=> #<Shrine::UploadedFile>
|
540
|
+
photo.image.methods #=> [:url, :download, :read, :exists?, :delete, ...]
|
549
541
|
```
|
550
542
|
|
551
543
|
If attachment is missing, nil is returned.
|
@@ -556,13 +548,13 @@ This method is simply a shorthand for "if attachment is present, call `#url`
|
|
556
548
|
on it, otherwise return nil":
|
557
549
|
|
558
550
|
```rb
|
559
|
-
|
560
|
-
|
561
|
-
|
551
|
+
photo.image_url #=> nil
|
552
|
+
photo.image = File.open("avatar.jpg", "rb")
|
553
|
+
photo.image_url #=> "/uploads/ksdf934rt.jpg"
|
562
554
|
```
|
563
555
|
|
564
|
-
The `
|
565
|
-
argument (`
|
556
|
+
The `derivatives` plugin extends this method to also accept a version name as
|
557
|
+
the argument (`photo.image_url(:thumb)`).
|
566
558
|
|
567
559
|
#### `#<attachment>_cache`
|
568
560
|
|
@@ -573,9 +565,9 @@ that you can use for retaining the cached file:
|
|
573
565
|
Shrine.plugin :cached_attachment_data
|
574
566
|
```
|
575
567
|
```rb
|
576
|
-
form_for @
|
577
|
-
f.hidden_field :
|
578
|
-
f.file_field :
|
568
|
+
form_for @photo do |f|
|
569
|
+
f.hidden_field :image, value: @photo.cached_image_data
|
570
|
+
f.file_field :image
|
579
571
|
end
|
580
572
|
```
|
581
573
|
|
@@ -608,7 +600,7 @@ By default Shrine deletes cached and replaced files, but you can choose to keep
|
|
608
600
|
those files by loading the `keep_files` plugin:
|
609
601
|
|
610
602
|
```rb
|
611
|
-
Shrine.plugin :keep_files
|
603
|
+
Shrine.plugin :keep_files
|
612
604
|
```
|
613
605
|
|
614
606
|
#### `move_to_cache`, `move_to_store`
|
@@ -632,8 +624,8 @@ class ImageUploader < Shrine
|
|
632
624
|
Attacher.validate do
|
633
625
|
# Evaluated inside an instance of Shrine::Attacher.
|
634
626
|
if record.guest?
|
635
|
-
validate_max_size 2*1024*1024, message: "
|
636
|
-
|
627
|
+
validate_max_size 2*1024*1024, message: "must not be larger than 2 MB"
|
628
|
+
validate_mime_type %w[image/jpg image/png image/webp]
|
637
629
|
end
|
638
630
|
end
|
639
631
|
end
|
@@ -709,12 +701,12 @@ plugin :url_options, store: { expires_in: 600 }
|
|
709
701
|
Shrine allows you to override the S3 endpoint:
|
710
702
|
|
711
703
|
```rb
|
712
|
-
Shrine::Storage::S3.new(
|
704
|
+
Shrine::Storage::S3.new(use_accelerate_endpoint: true, **options)
|
713
705
|
```
|
714
706
|
|
715
707
|
[image_processing]: https://github.com/janko/image_processing
|
716
708
|
[demo app]: https://github.com/shrinerb/shrine/tree/master/demo
|
717
|
-
[
|
709
|
+
[Managing Derivatives]: /doc/changing_derivatives.md#readme
|
718
710
|
[shrine-fog]: https://github.com/shrinerb/shrine-fog
|
719
711
|
[direct uploads]: /doc/direct_s3.md#readme
|
720
712
|
[`Shrine::Storage::S3`]: /doc/storage/s3.md#readme
|
@@ -0,0 +1,308 @@
|
|
1
|
+
# Managing Derivatives
|
2
|
+
|
3
|
+
This guide shows how to add, create, update, and remove [derivatives] for an
|
4
|
+
app in production already handling file attachments, with zero downtime.
|
5
|
+
|
6
|
+
Let's assume we have a `Photo` model with an `image` file attachment. The
|
7
|
+
examples will be showing image thumbnails, but the advice applies to any kind
|
8
|
+
of derivatives.
|
9
|
+
|
10
|
+
```rb
|
11
|
+
Shrine.plugin :derivatives
|
12
|
+
Shrine.plugin :activerecord
|
13
|
+
```
|
14
|
+
```rb
|
15
|
+
class ImageUploader < Shrine
|
16
|
+
# ...
|
17
|
+
end
|
18
|
+
```
|
19
|
+
```rb
|
20
|
+
class Photo < ActiveRecord::Base
|
21
|
+
include ImageUploader::Attachment(:image)
|
22
|
+
end
|
23
|
+
```
|
24
|
+
|
25
|
+
## Contents
|
26
|
+
|
27
|
+
* [Adding derivatives](#adding-derivatives)
|
28
|
+
* [Reprocessing all derivatives](#reprocessing-all-derivatives)
|
29
|
+
* [Reprocessing certain derivatives](#reprocessing-certain-derivatives)
|
30
|
+
* [Adding new derivatives](#adding-new-derivatives)
|
31
|
+
* [Removing derivatives](#removing-derivatives)
|
32
|
+
* [Backgrounding](#backgrounding)
|
33
|
+
|
34
|
+
## Adding derivatives
|
35
|
+
|
36
|
+
*Scenario: Your app is currently working only with original files, and you want
|
37
|
+
to introduce derivatives.*
|
38
|
+
|
39
|
+
You'll first want to start creating the derivatives in production, without yet
|
40
|
+
generating URLs for them (because existing attachments won't yet have
|
41
|
+
derivatives generated). Let's assume you're generating image thumbnails:
|
42
|
+
|
43
|
+
```rb
|
44
|
+
# Gemfile
|
45
|
+
gem "image_processing", "~> 1.8"
|
46
|
+
```
|
47
|
+
```rb
|
48
|
+
require "image_processing/mini_magick"
|
49
|
+
|
50
|
+
class ImageUploader < Shrine
|
51
|
+
Attacher.derivatives_processor do |original|
|
52
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
53
|
+
|
54
|
+
# generate the thumbnails you want here
|
55
|
+
{
|
56
|
+
small: magick.resize_to_limit!(300, 300),
|
57
|
+
medium: magick.resize_to_limit!(500, 500),
|
58
|
+
large: magick.resize_to_limit!(800, 800),
|
59
|
+
}
|
60
|
+
end
|
61
|
+
end
|
62
|
+
```
|
63
|
+
```rb
|
64
|
+
photo = Photo.new(photo_params)
|
65
|
+
photo.image_derivatives! # generate derivatives
|
66
|
+
photo.save
|
67
|
+
```
|
68
|
+
|
69
|
+
Once we've deployed this to production, we can run the following script to
|
70
|
+
generate derivatives for all existing attachments in production. It fetches the
|
71
|
+
records in batches, downloads attachments on permanent storage, creates
|
72
|
+
derivatives, and persists the changes.
|
73
|
+
|
74
|
+
```rb
|
75
|
+
Photo.find_each do |photo|
|
76
|
+
attacher = photo.image_attacher
|
77
|
+
|
78
|
+
next unless attacher.stored?
|
79
|
+
|
80
|
+
attacher.create_derivatives
|
81
|
+
|
82
|
+
begin
|
83
|
+
attacher.atomic_persist # persist changes if attachment has not changed in the meantime
|
84
|
+
rescue Shrine::AttachmentChanged, # attachment has changed
|
85
|
+
ActiveRecord::RecordNotFound # record has been deleted
|
86
|
+
attacher.delete_derivatives # delete now orphaned derivatives
|
87
|
+
end
|
88
|
+
end
|
89
|
+
```
|
90
|
+
|
91
|
+
Now all attachments should have correctly generated derivatives. You can update
|
92
|
+
the attachment URLs to use derivatives as needed.
|
93
|
+
|
94
|
+
## Reprocessing all derivatives
|
95
|
+
|
96
|
+
*Scenario: The processing logic has changed for all or most derivatives, and
|
97
|
+
now you want to reprocess them for existing attachments.*
|
98
|
+
|
99
|
+
Let's assume we've made the following change and have deployed it to production:
|
100
|
+
|
101
|
+
```diff
|
102
|
+
Attacher.derivatives_processor do |original|
|
103
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
104
|
+
+ .saver(quality: 85)
|
105
|
+
|
106
|
+
{
|
107
|
+
small: magick.resize_to_limit!(300, 300),
|
108
|
+
medium: magick.resize_to_limit!(500, 500),
|
109
|
+
large: magick.resize_to_limit!(800, 800),
|
110
|
+
}
|
111
|
+
end
|
112
|
+
```
|
113
|
+
|
114
|
+
We can now run the following script to reprocess derivatives for all existing
|
115
|
+
records. It fetches the records in batches, downloads attachments on permanent
|
116
|
+
storage, reprocesses new derivatives, persists the changes, and deletes old
|
117
|
+
derivatives.
|
118
|
+
|
119
|
+
```rb
|
120
|
+
Photo.find_each do |photo|
|
121
|
+
attacher = photo.image_attacher
|
122
|
+
|
123
|
+
next unless attacher.stored?
|
124
|
+
|
125
|
+
old_derivatives = attacher.derivatives
|
126
|
+
|
127
|
+
attacher.set_derivatives({}) # clear derivatives
|
128
|
+
attacher.create_derivatives # reprocess derivatives
|
129
|
+
|
130
|
+
begin
|
131
|
+
attacher.atomic_persist # persist changes if attachment has not changed in the meantime
|
132
|
+
attacher.delete_derivatives(old_derivatives) # delete old derivatives
|
133
|
+
rescue Shrine::AttachmentChanged, # attachment has changed
|
134
|
+
ActiveRecord::RecordNotFound # record has been deleted
|
135
|
+
attacher.delete_derivatives # delete now orphaned derivatives
|
136
|
+
end
|
137
|
+
end
|
138
|
+
```
|
139
|
+
|
140
|
+
## Reprocessing certain derivatives
|
141
|
+
|
142
|
+
*Scenario: The processing logic has changed for specific derivatives, and now
|
143
|
+
you want to reprocess them for existing attachments.*
|
144
|
+
|
145
|
+
Let's assume we've made a following change and have deployed it to production:
|
146
|
+
|
147
|
+
```diff
|
148
|
+
Attacher.derivatives_processor do |original|
|
149
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
150
|
+
|
151
|
+
{
|
152
|
+
small: magick.resize_to_limit!(300, 300),
|
153
|
+
- medium: magick.resize_to_limit!(500, 500),
|
154
|
+
+ medium: magick.resize_to_limit!(600, 600),
|
155
|
+
large: magick.resize_to_limit!(800, 800),
|
156
|
+
}
|
157
|
+
end
|
158
|
+
```
|
159
|
+
|
160
|
+
We can now run the following script to reprocess the derivative for all
|
161
|
+
existing records. It fetches the records in batches, downloads attachments with
|
162
|
+
derivatives, reprocesses the specific derivative, persists the change, and
|
163
|
+
deletes old derivative.
|
164
|
+
|
165
|
+
```rb
|
166
|
+
Photo.find_each do |photo|
|
167
|
+
attacher = photo.image_attacher
|
168
|
+
|
169
|
+
next unless attacher.derivatives.key?(:medium)
|
170
|
+
|
171
|
+
old_medium = attacher.derivatives[:medium]
|
172
|
+
new_medium = attacher.file.download do |original|
|
173
|
+
ImageProcessing::MiniMagick
|
174
|
+
.source(original)
|
175
|
+
.resize_to_limit!(600, 600)
|
176
|
+
end
|
177
|
+
|
178
|
+
attacher.add_derivative(:medium, new_medium)
|
179
|
+
|
180
|
+
begin
|
181
|
+
attacher.atomic_persist # persist changes if attachment has not changed in the meantime
|
182
|
+
old_medium.delete # delete old derivative
|
183
|
+
rescue Shrine::AttachmentChanged, # attachment has changed
|
184
|
+
ActiveRecord::RecordNotFound # record has been deleted
|
185
|
+
attacher.derivatives[:medium].delete # delete now orphaned derivative
|
186
|
+
end
|
187
|
+
end
|
188
|
+
```
|
189
|
+
|
190
|
+
## Adding new derivatives
|
191
|
+
|
192
|
+
*Scenario: A new derivative has been added to the processor, and now
|
193
|
+
you want to add it to existing attachments.*
|
194
|
+
|
195
|
+
Let's assume we've made a following change and have deployed it to production:
|
196
|
+
|
197
|
+
```diff
|
198
|
+
Attacher.derivatives_processor do |original|
|
199
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
200
|
+
|
201
|
+
{
|
202
|
+
+ square: magick.resize_to_fill!(150, 150),
|
203
|
+
small: magick.resize_to_limit!(300, 300),
|
204
|
+
medium: magick.resize_to_limit!(600, 600),
|
205
|
+
large: magick.resize_to_limit!(800, 800),
|
206
|
+
}
|
207
|
+
end
|
208
|
+
```
|
209
|
+
|
210
|
+
We can now run following script to add the new derivative for all existing
|
211
|
+
records. It fetches the records in batches, downloads attachments on permanent
|
212
|
+
storage, creates the new derivative, and persists the changes.
|
213
|
+
|
214
|
+
```rb
|
215
|
+
Photo.find_each do |photo|
|
216
|
+
attacher = photo.image_attacher
|
217
|
+
|
218
|
+
next unless attacher.stored?
|
219
|
+
|
220
|
+
square = attacher.file.download do |original|
|
221
|
+
ImageProcessor::MiniMagick
|
222
|
+
.source(original)
|
223
|
+
.resize_to_fill!(150, 150)
|
224
|
+
end
|
225
|
+
|
226
|
+
attacher.add_derivative(:square, square)
|
227
|
+
|
228
|
+
begin
|
229
|
+
attacher.atomic_persist # persist changes if attachment has not changed in the meantime
|
230
|
+
rescue Shrine::AttachmentChanged, # attachment has changed
|
231
|
+
ActiveRecord::RecordNotFound # record has been deleted
|
232
|
+
attacher.derivatives[:square].delete # delete now orphaned derivative
|
233
|
+
end
|
234
|
+
end
|
235
|
+
```
|
236
|
+
|
237
|
+
Now all attachments should have the new derivative and you can start generating
|
238
|
+
URLs for it.
|
239
|
+
|
240
|
+
## Removing derivatives
|
241
|
+
|
242
|
+
*Scenario: A derivative isn't being used anymore, so we want to delete it for
|
243
|
+
existing attachments.*
|
244
|
+
|
245
|
+
Let's assume we've made the following change and have deployed it to production:
|
246
|
+
|
247
|
+
```diff
|
248
|
+
Attacher.derivatives_processor do |original|
|
249
|
+
magick = ImageProcessing::MiniMagick.source(original)
|
250
|
+
|
251
|
+
{
|
252
|
+
- square: magick.resize_to_fill!(150, 150),
|
253
|
+
small: magick.resize_to_limit!(300, 300),
|
254
|
+
medium: magick.resize_to_limit!(600, 600),
|
255
|
+
large: magick.resize_to_limit!(800, 800),
|
256
|
+
}
|
257
|
+
end
|
258
|
+
```
|
259
|
+
|
260
|
+
We can now run following script to remove the unused derivative for all
|
261
|
+
existing record. It fetches the records in batches, removes and deletes the
|
262
|
+
unused derivative, and persists the changes.
|
263
|
+
|
264
|
+
```rb
|
265
|
+
Photo.find_each do |photo|
|
266
|
+
attacher = photo.image_attacher
|
267
|
+
|
268
|
+
next unless attacher.derivatives.key?(:square)
|
269
|
+
|
270
|
+
attacher.remove_derivative(:square, delete: true)
|
271
|
+
|
272
|
+
begin
|
273
|
+
attacher.atomic_persist # persist changes if attachment has not changed in the meantime
|
274
|
+
rescue Shrine::AttachmentChanged, # attachment has changed
|
275
|
+
ActiveRecord::RecordNotFound # record has been deleted
|
276
|
+
end
|
277
|
+
end
|
278
|
+
```
|
279
|
+
|
280
|
+
## Backgrounding
|
281
|
+
|
282
|
+
For faster migration, we can also delay any of the operations above into a
|
283
|
+
background job:
|
284
|
+
|
285
|
+
```rb
|
286
|
+
Photo.find_each do |photo|
|
287
|
+
attacher = photo.image_attacher
|
288
|
+
|
289
|
+
next unless attacher.stored?
|
290
|
+
|
291
|
+
MakeChangeJob.perform_later(
|
292
|
+
attacher.class,
|
293
|
+
attacher.record,
|
294
|
+
attacher.name,
|
295
|
+
attacher.file_data,
|
296
|
+
)
|
297
|
+
end
|
298
|
+
```
|
299
|
+
```rb
|
300
|
+
class MakeChangeJob < ActiveJob::Base
|
301
|
+
def perform(attacher_class, record, name, file_data)
|
302
|
+
attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
|
303
|
+
# ... make our change ...
|
304
|
+
end
|
305
|
+
end
|
306
|
+
```
|
307
|
+
|
308
|
+
[derivatives]: /doc/plugins/derivatives.md#readme
|