shrine 3.2.2 → 3.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +36 -0
  3. data/README.md +5 -0
  4. data/doc/advantages.md +1 -1
  5. data/doc/carrierwave.md +7 -4
  6. data/doc/direct_s3.md +1 -0
  7. data/doc/external/articles.md +6 -8
  8. data/doc/external/extensions.md +3 -0
  9. data/doc/external/misc.md +23 -8
  10. data/doc/getting_started.md +4 -8
  11. data/doc/multiple_files.md +1 -1
  12. data/doc/paperclip.md +14 -5
  13. data/doc/plugins/add_metadata.md +20 -0
  14. data/doc/plugins/atomic_helpers.md +41 -3
  15. data/doc/plugins/derivatives.md +43 -12
  16. data/doc/plugins/download_endpoint.md +5 -5
  17. data/doc/plugins/dynamic_storage.md +1 -1
  18. data/doc/plugins/infer_extension.md +9 -0
  19. data/doc/plugins/metadata_attributes.md +1 -0
  20. data/doc/plugins/mirroring.md +1 -1
  21. data/doc/plugins/persistence.md +10 -1
  22. data/doc/plugins/store_dimensions.md +10 -0
  23. data/doc/plugins/url_options.md +2 -2
  24. data/doc/processing.md +7 -8
  25. data/doc/release_notes/2.8.0.md +1 -1
  26. data/doc/release_notes/3.2.1.md +2 -3
  27. data/doc/release_notes/3.3.0.md +105 -0
  28. data/doc/storage/s3.md +9 -5
  29. data/doc/upgrading_to_3.md +11 -27
  30. data/lib/shrine/attacher.rb +6 -1
  31. data/lib/shrine/plugins/activerecord.rb +1 -1
  32. data/lib/shrine/plugins/add_metadata.rb +6 -4
  33. data/lib/shrine/plugins/backgrounding.rb +2 -2
  34. data/lib/shrine/plugins/derivation_endpoint.rb +2 -1
  35. data/lib/shrine/plugins/derivatives.rb +45 -15
  36. data/lib/shrine/plugins/mirroring.rb +8 -8
  37. data/lib/shrine/plugins/presign_endpoint.rb +14 -2
  38. data/lib/shrine/plugins/remove_attachment.rb +5 -0
  39. data/lib/shrine/plugins/sequel.rb +1 -1
  40. data/lib/shrine/plugins/store_dimensions.rb +4 -2
  41. data/lib/shrine/plugins/upload_endpoint.rb +7 -2
  42. data/lib/shrine/storage/memory.rb +5 -3
  43. data/lib/shrine/storage/s3.rb +61 -6
  44. data/lib/shrine/version.rb +2 -2
  45. data/shrine.gemspec +1 -1
  46. metadata +6 -5
@@ -87,6 +87,10 @@ class DownloadsController < ApplicationController
87
87
  end
88
88
  ```
89
89
 
90
+ If you want to create an endpoint with a custom path, you can use the
91
+ [`rack_response`][rack_response] plugin directly, which this plugin uses
92
+ internally.
93
+
90
94
  ## Host
91
95
 
92
96
  You can specify download URL host via the `:host` plugin option:
@@ -155,11 +159,6 @@ You can override any of the options above when creating the endpoint:
155
159
  Shrine.download_endpoint(disposition: "attachment")
156
160
  ```
157
161
 
158
- ## Custom endpoint
159
-
160
- If you want to have more control on download requests, you can use the
161
- `rack_response` plugin which this plugin uses internally.
162
-
163
162
  ## Plugin options
164
163
 
165
164
  | Name | Description | Default |
@@ -171,3 +170,4 @@ If you want to have more control on download requests, you can use the
171
170
  | `:redirect` | Whether to redirect to uploaded files on the storage | `false` |
172
171
 
173
172
  [download_endpoint]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/download_endpoint.rb
173
+ [rack_response]: https://shrinerb.com/docs/plugins/rack_response
@@ -12,7 +12,7 @@ Example:
12
12
  plugin :dynamic_storage
13
13
 
14
14
  storage /store_(\w+)/ do |match|
15
- Shrine::Storages::S3.new(bucket: match[1])
15
+ Shrine::Storage::S3.new(bucket: match[1])
16
16
  end
17
17
  ```
18
18
 
@@ -11,6 +11,15 @@ extension might not be known.
11
11
  plugin :infer_extension
12
12
  ```
13
13
 
14
+ By default an extension will only be inferred if needed to supply an otherwise
15
+ missing extension. But option `force: true` will normalize even an already
16
+ present extension to the extension inferred from MIME type. This could be used
17
+ to fix incorrect or malicious extensions on user-submitted files.
18
+
19
+ ```rb
20
+ plugin :infer_extension, force: true
21
+ ```
22
+
14
23
  ## Inferrers
15
24
 
16
25
  By default, the [mini_mime] gem will be used for inferring the extension, but
@@ -72,3 +72,4 @@ photo.original_filename #=> "nature.jpg"
72
72
 
73
73
  [metadata_attributes]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/metadata_attributes.rb
74
74
  [entity]: https://shrinerb.com/docs/plugins/entity
75
+ [model]: https://shrinerb.com/docs/plugins/model
@@ -50,7 +50,7 @@ Shrine.plugin :mirroring, mirror: { ... }, delete: false
50
50
  You can have mirroring performed in a background job:
51
51
 
52
52
  ```rb
53
- Shrine.mirror_upload_block do |file|
53
+ Shrine.mirror_upload_block do |file, **options|
54
54
  MirrorUploadJob.perform_async(file.shrine_class.name, file.data)
55
55
  end
56
56
 
@@ -6,6 +6,10 @@ This is an internal plugin that provides uniform persistence interface across
6
6
  different persistence plugins (e.g. [`activerecord`][activerecord],
7
7
  [`sequel`][sequel]).
8
8
 
9
+ For these activerecord and sequel, atomic persistence is implemented in terms
10
+ of database locks, eg "SELECT... FOR UPDATE". For more discussion of concurrency
11
+ challenges, see the [atomic_helpers] documentation.
12
+
9
13
  ## Atomic promotion
10
14
 
11
15
  If you're promoting cached file to permanent storage
@@ -65,11 +69,15 @@ changed, and if it hasn't the attachment is persisted. If the attachment has
65
69
  changed, `Shrine::AttachmentChanged` exception is raised.
66
70
 
67
71
  If you want to execute code after the attachment change check but before
68
- persistence, you can pass a block:
72
+ persistence, you can pass a block. For instance, one way to allow concurrent
73
+ changes to metadata, perhaps in different background workers, without
74
+ overwriting each other might be:
69
75
 
70
76
  ```rb
71
77
  attacher.atomic_persist do |reloaded_attacher|
72
78
  # run code after attachment change check but before persistence
79
+ attacher.file.metadata.merge!(reloaded_attacher.file.metadata)
80
+ attacher.file.metadata["some_key"] = "changed_value"
73
81
  end
74
82
  ```
75
83
 
@@ -89,4 +97,5 @@ attacher.persist # saves the underlying record
89
97
 
90
98
  [activerecord]: https://shrinerb.com/docs/plugins/activerecord
91
99
  [sequel]: https://shrinerb.com/docs/plugins/sequel
100
+ [atomic_helpers]: https://shrinerb.com/docs/plugins/atomic_helpers
92
101
  [backgrounding]: https://shrinerb.com/docs/plugins/backgrounding
@@ -72,6 +72,16 @@ Shrine.dimensions(io) #=> [300, 400] (calls the defined analyzer)
72
72
  Shrine.dimensions_analyzers[:fastimage].call(io) #=> [300, 400] (calls a built-in analyzer)
73
73
  ```
74
74
 
75
+ ### Disabling auto-extraction
76
+
77
+ If you want to use the dimensions extraction methods but not automatically
78
+ extract dimensions on upload, you can setup this plugin with the
79
+ `auto_extraction: false` option.
80
+
81
+ ```rb
82
+ plugin :store_dimensions, auto_extraction: false
83
+ ```
84
+
75
85
  ## Errors
76
86
 
77
87
  By default, any exceptions that the analyzer raises while extracting dimensions
@@ -4,10 +4,10 @@ title: URL Options
4
4
 
5
5
  The [`url_options`][url_options] plugin allows you to specify
6
6
  URL options that will be applied by default for uploaded files of specified
7
- storages.
7
+ storages. `url_options` are parameters specific to the storage service.
8
8
 
9
9
  ```rb
10
- plugin :url_options, store: { expires_in: 24*60*60 }
10
+ plugin :url_options, store: { expires_in: 24*60*60 } # `expires_in` is a URL option for AWS S3
11
11
  ```
12
12
 
13
13
  You can also generate the default URL options dynamically by using a block,
@@ -60,8 +60,11 @@ end
60
60
  ```
61
61
  ```rb
62
62
  photo = Photo.new(image: file)
63
- photo.image_derivatives! # calls derivatives processor
64
- photo.save
63
+
64
+ if photo.valid?
65
+ photo.image_derivatives! if photo.image_changed? # creates derivatives
66
+ photo.save
67
+ end
65
68
  ```
66
69
 
67
70
  After the processed files are uploaded, their data is saved into the
@@ -468,15 +471,11 @@ end
468
471
  ### Automatic derivatives
469
472
 
470
473
  If you would like derivatives to be automatically created with promotion, you
471
- can override `Attacher#promote` for call `Attacher#create_derivatives` before
472
- promotion:
474
+ can use the `create_on_promote` option built-in to the derivatives plugin.
473
475
 
474
476
  ```rb
475
477
  class Shrine::Attacher
476
- def promote(*)
477
- create_derivatives
478
- super
479
- end
478
+ plugin :derivatives, create_on_promote: true
480
479
  end
481
480
  ```
482
481
 
@@ -1,5 +1,5 @@
1
1
  ---
2
- title: Shrine 2.9.0
2
+ title: Shrine 2.8.0
3
3
  ---
4
4
 
5
5
  ## New Features
@@ -20,10 +20,9 @@ title: Shrine 3.2.1
20
20
  gem "image_processing", ">= 1.10.3", "< 2"
21
21
  ```
22
22
 
23
- ## Rack 2.1.0 compatibility
23
+ ## Rack 2.1 compatibility
24
24
 
25
- * The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1.0 or
26
- newer.
25
+ * The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1 or newer.
27
26
 
28
27
  ## Other improvements
29
28
 
@@ -0,0 +1,105 @@
1
+ ---
2
+ title: Shrine 3.3.0
3
+ ---
4
+
5
+ ## New features
6
+
7
+ * The `:create_on_promote` option has been added to the `derivatives` plugin
8
+ for automatically creating derivatives after the attached cached file is
9
+ promoted to permanent storage.
10
+
11
+ ```rb
12
+ Shrine.plugin :derivatives, create_on_promote: true
13
+ ```
14
+
15
+ * The `:auto_extraction` option has been added to the `store_dimensions` plugin
16
+ for skipping automatically extracting dimensions on upload.
17
+
18
+ ```rb
19
+ Shrine.plugin :store_dimensions, auto_extraction: false
20
+ ```
21
+
22
+ * The `:skip_nil` option has been added to the `add_metadata` plugin for
23
+ excluding metadata keys whose values are nil.
24
+
25
+ ```rb
26
+ class PdfUploader < Shrine
27
+ add_metadata :pages, skip_nil: true do |io|
28
+ if is_pdf?(io)
29
+ reader = PDF::Reader.new(io)
30
+ reader.page_count
31
+ else
32
+ # If this is not a PDF, then the pages metadata will not be stored
33
+ nil
34
+ end
35
+ end
36
+ end
37
+ ```
38
+
39
+ * The `:download` option has been added to derivatives processors in
40
+ `derivatives` plugin for skipping converting the source IO object into a
41
+ file. This can be used to avoid a potentially expensive download/copy when
42
+ the derivatives processor doesn't need the file.
43
+
44
+ ```rb
45
+ Attacher.derivatives :my_processor, download: false do |source|
46
+ source #=> Could be File, Shrine::UploadedFile, or other IO-like object
47
+ shrine_class.with_file(source) do |file|
48
+ # can force download/copy if necessary with `with_file`,
49
+ end
50
+ end
51
+ ```
52
+
53
+ ## Bug fixes
54
+
55
+ * The `upload_endpoint` now handles calling `Shrine.upload_response` method
56
+ from a Rails controller.
57
+
58
+ * The `derivation_endpoint` plugin now applies the `version` query parameter
59
+ to the derivation when creating the response.
60
+
61
+ ## Other improvements
62
+
63
+ * The new `Aws:S3::EncryptionV2::Client` is now supported by the S3 storage for
64
+ client-side encryption.
65
+
66
+ * The `derivation_endpoint` now reduces the possibility of timing attacks by
67
+ comparing URL signatures in constant time using `Rack::Utils.secure_compare`.
68
+
69
+ * The `derivatives` plugin now copies non-file source IO objects to disk before
70
+ passing them to the processor. This is consistent with how a
71
+ `Shrine::UploadedFile` object is downloaded to disk.
72
+
73
+ * The `sequel` and `activerecord` plugins now call `Attacher#reload` when
74
+ reloading the model, which reloads the attached files but keeps other
75
+ attacher state.
76
+
77
+ * The `derivatives` plugin doesn't download the attached file anymore if
78
+ attempting to process derivatives when no derivatives processor was defined.
79
+
80
+ * The `mirroring` plugin now forwards attacher options when uploading to mirror
81
+ storages.
82
+
83
+ * The `presign_endpoint` plugin now handles the `OPTIONS` HTTP verb, which
84
+ newer versions of Uppy are requesting.
85
+
86
+ * `Shrine::Storage::Memory#open` now always returns a `StringIO` in the file
87
+ content's original encoding, instead of the encoding set by
88
+ `Encoding.default_internal`. This works around a [bug][ruby-lang #16497]
89
+ in `StringIO` introduced in Ruby 2.7.0.
90
+
91
+ * The `remove_attachment` plugin now deletes the removed file if a new file was
92
+ attached right after removal.
93
+
94
+ ## Backwards compatibility
95
+
96
+ * If you were passing a non-file IO object to the derivatives processor, Shrine
97
+ will now convert it into a file beforehand. If you're currently doing this
98
+ and are converting the IO object into a file inside the processor, you can
99
+ now remove the conversion code to avoid doubling the amount of disk writes.
100
+
101
+ * When reloading a Sequel/ActiveRecord model, any attacher state other than
102
+ uploaded files will now be retained after the reload. If you were relying on
103
+ all the attacher state being re-initialized, you'll need to update your code.
104
+
105
+ [ruby-lang #16497]: https://bugs.ruby-lang.org/issues/16497
@@ -267,9 +267,15 @@ s3.open("key", sse_customer_algorithm: "AES256",
267
267
  sse_customer_key_md5: "secret_key_md5")
268
268
  ```
269
269
 
270
- If you want to use **client-side** encryption instead, note that it's still a
271
- work in progress, see issue [#348] for some discussion and
272
- [workarounds][client-side encryption workaround].
270
+ **Client-side** encryption is supported as well:
271
+
272
+ ```rb
273
+ encryption_client = Aws::S3::EncryptionV2::Client.new(...)
274
+ s3 = Shrine::Storage::S3.new(client: encryption_client, **other_options)
275
+
276
+ s3.upload(io, "key") # encrypts on upload
277
+ s3.open("key") # decrypts on download
278
+ ```
273
279
 
274
280
  ## Accelerate endpoint
275
281
 
@@ -317,5 +323,3 @@ s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
317
323
  [credentials]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
318
324
  [`Aws::S3::Object#presigned_post`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
319
325
  [`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
320
- [#348]: https://github.com/shrinerb/shrine/issues/348
321
- [client-side encryption workaround]: https://github.com/shrinerb/shrine/issues/348#issuecomment-486445382
@@ -323,15 +323,17 @@ else
323
323
  end
324
324
  ```
325
325
 
326
- With `derivatives`, the original file is automatically downloaded and retained,
327
- so the processing code is much simpler:
326
+ With `derivatives`, the original file is automatically downloaded and retained
327
+ during processing, so the setup is simpler:
328
328
 
329
329
  ```rb
330
- Shrine.plugin :derivatives, versions_compatibility: true # handle versions column format
330
+ Shrine.plugin :derivatives,
331
+ create_on_promote: true, # automatically create derivatives on promotion
332
+ versions_compatibility: true # handle versions column format
331
333
  ```
332
334
  ```rb
333
335
  class ImageUploader < Shrine
334
- Attacher.derivatives_processor do |original|
336
+ Attacher.derivatives do |original|
335
337
  magick = ImageProcessing::MiniMagick.source(original)
336
338
 
337
339
  # the :original file should NOT be included anymore
@@ -343,35 +345,17 @@ class ImageUploader < Shrine
343
345
  end
344
346
  end
345
347
  ```
346
-
347
- However, you now need to trigger processing manually during attachment:
348
-
349
348
  ```rb
350
349
  photo = Photo.new(photo_params)
351
350
 
352
351
  if photo.valid?
353
- photo.image_derivatives! if photo.image_changed? # create derivatives
354
- photo.save
352
+ photo.save # creates derivatives on promotion
355
353
  # ...
356
354
  else
357
355
  # ...
358
356
  end
359
357
  ```
360
358
 
361
- ### Automatic processing
362
-
363
- If you prefer processing to happen automatically with promotion (like it did
364
- with the `versions` plugin), you can put the following in your initializer:
365
-
366
- ```rb
367
- class Shrine::Attacher
368
- def promote(*)
369
- create_derivatives
370
- super
371
- end
372
- end
373
- ```
374
-
375
359
  ### Accessing derivatives
376
360
 
377
361
  The derivative URLs are accessed in the same way as versions:
@@ -475,11 +459,11 @@ creating another derivatives processor that you will trigger in the controller:
475
459
 
476
460
  ```rb
477
461
  class ImageUploader < Shrine
478
- Attacher.derivatives_processor do |original|
462
+ Attacher.derivatives do |original|
479
463
  # this will be triggered in the background job
480
464
  end
481
465
 
482
- Attacher.derivatives_processor :foreground do |original|
466
+ Attacher.derivatives :foreground do |original|
483
467
  # this will be triggered in the controller
484
468
  end
485
469
  end
@@ -586,7 +570,7 @@ you should now add the processed file as a derivative:
586
570
  class MyUploader < Shrine
587
571
  plugin :derivatives
588
572
 
589
- Attacher.derivatives_processor do |original|
573
+ Attacher.derivatives do |original|
590
574
  magick = ImageProcessing::MiniMagick.source(original)
591
575
 
592
576
  { normalized: magick.resize_to_limit!(1600, 1600) }
@@ -665,7 +649,7 @@ attacher.copy(other_attacher)
665
649
  with
666
650
 
667
651
  ```rb
668
- attacher.attach other_attacher.file
652
+ attacher.set attacher.upload(other_attacher.file)
669
653
  attacher.add_derivatives other_attacher.derivatives # if using derivatives
670
654
  ```
671
655
 
@@ -217,7 +217,7 @@ class Shrine
217
217
  # attacher.file #=> #<Shrine::UploadedFile>
218
218
  # attacher.changed? #=> true
219
219
  def change(file)
220
- @previous = dup unless @file == file
220
+ @previous = dup if change?(file)
221
221
  set(file)
222
222
  end
223
223
 
@@ -373,6 +373,11 @@ class Shrine
373
373
  attached? && !cached?
374
374
  end
375
375
 
376
+ # Whether assigning the given file is considered a change.
377
+ def change?(file)
378
+ @file != file
379
+ end
380
+
376
381
  # Returns whether the file is uploaded to specified storage.
377
382
  def uploaded?(file, storage_key)
378
383
  file&.storage_key == storage_key
@@ -55,7 +55,7 @@ class Shrine
55
55
  # reload the attacher on record reload
56
56
  define_method :reload do |*args|
57
57
  result = super(*args)
58
- instance_variable_set(:"@#{name}_attacher", nil)
58
+ send(:"#{name}_attacher").reload
59
59
  result
60
60
  end
61
61
  end
@@ -9,8 +9,8 @@ class Shrine
9
9
  end
10
10
 
11
11
  module ClassMethods
12
- def add_metadata(name = nil, &block)
13
- opts[:add_metadata][:definitions] << [name, block]
12
+ def add_metadata(name = nil, **options, &block)
13
+ opts[:add_metadata][:definitions] << [name, options, block]
14
14
 
15
15
  metadata_method(name) if name
16
16
  end
@@ -40,10 +40,12 @@ class Shrine
40
40
  private
41
41
 
42
42
  def extract_custom_metadata(io, **options)
43
- opts[:add_metadata][:definitions].each do |name, block|
43
+ opts[:add_metadata][:definitions].each do |name, definition_options, block|
44
44
  result = instance_exec(io, **options, &block)
45
45
 
46
- if name
46
+ if result.nil? && definition_options[:skip_nil]
47
+ # Do not store this metadata
48
+ elsif name
47
49
  options[:metadata].merge! name.to_s => result
48
50
  else
49
51
  options[:metadata].merge! result.transform_keys(&:to_s) if result