shrine 3.0.0 → 3.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

Files changed (93) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +87 -33
  3. data/LICENSE.txt +1 -1
  4. data/README.md +94 -4
  5. data/doc/advantages.md +35 -18
  6. data/doc/attacher.md +16 -17
  7. data/doc/carrierwave.md +75 -34
  8. data/doc/changing_derivatives.md +39 -39
  9. data/doc/design.md +134 -85
  10. data/doc/external/articles.md +56 -41
  11. data/doc/external/extensions.md +38 -34
  12. data/doc/getting_started.md +182 -112
  13. data/doc/metadata.md +79 -43
  14. data/doc/multiple_files.md +5 -3
  15. data/doc/paperclip.md +110 -42
  16. data/doc/plugins/activerecord.md +5 -5
  17. data/doc/plugins/add_metadata.md +92 -35
  18. data/doc/plugins/backgrounding.md +12 -2
  19. data/doc/plugins/column.md +36 -7
  20. data/doc/plugins/data_uri.md +2 -2
  21. data/doc/plugins/default_url.md +6 -3
  22. data/doc/plugins/derivation_endpoint.md +26 -28
  23. data/doc/plugins/derivatives.md +205 -169
  24. data/doc/plugins/determine_mime_type.md +2 -2
  25. data/doc/plugins/entity.md +3 -3
  26. data/doc/plugins/form_assign.md +5 -5
  27. data/doc/plugins/included.md +25 -5
  28. data/doc/plugins/infer_extension.md +2 -2
  29. data/doc/plugins/instrumentation.md +1 -1
  30. data/doc/plugins/metadata_attributes.md +21 -10
  31. data/doc/plugins/model.md +4 -4
  32. data/doc/plugins/persistence.md +1 -0
  33. data/doc/plugins/refresh_metadata.md +5 -4
  34. data/doc/plugins/remote_url.md +8 -3
  35. data/doc/plugins/remove_invalid.md +9 -1
  36. data/doc/plugins/sequel.md +4 -4
  37. data/doc/plugins/signature.md +11 -2
  38. data/doc/plugins/store_dimensions.md +2 -2
  39. data/doc/plugins/type_predicates.md +96 -0
  40. data/doc/plugins/upload_endpoint.md +7 -11
  41. data/doc/plugins/upload_options.md +1 -1
  42. data/doc/plugins/url_options.md +2 -2
  43. data/doc/plugins/validation.md +14 -4
  44. data/doc/plugins/validation_helpers.md +3 -3
  45. data/doc/plugins/versions.md +11 -11
  46. data/doc/processing.md +289 -125
  47. data/doc/refile.md +39 -18
  48. data/doc/release_notes/2.19.0.md +1 -1
  49. data/doc/release_notes/3.0.0.md +275 -258
  50. data/doc/release_notes/3.0.1.md +22 -0
  51. data/doc/release_notes/3.1.0.md +73 -0
  52. data/doc/release_notes/3.2.0.md +96 -0
  53. data/doc/release_notes/3.2.1.md +32 -0
  54. data/doc/release_notes/3.2.2.md +14 -0
  55. data/doc/securing_uploads.md +3 -3
  56. data/doc/storage/file_system.md +1 -1
  57. data/doc/storage/memory.md +19 -0
  58. data/doc/storage/s3.md +105 -86
  59. data/doc/testing.md +2 -2
  60. data/doc/upgrading_to_3.md +115 -33
  61. data/doc/validation.md +3 -2
  62. data/lib/shrine.rb +8 -8
  63. data/lib/shrine/attacher.rb +19 -14
  64. data/lib/shrine/attachment.rb +5 -5
  65. data/lib/shrine/plugins.rb +22 -0
  66. data/lib/shrine/plugins/add_metadata.rb +12 -3
  67. data/lib/shrine/plugins/default_storage.rb +6 -6
  68. data/lib/shrine/plugins/default_url.rb +1 -1
  69. data/lib/shrine/plugins/derivation_endpoint.rb +10 -6
  70. data/lib/shrine/plugins/derivatives.rb +19 -17
  71. data/lib/shrine/plugins/determine_mime_type.rb +3 -3
  72. data/lib/shrine/plugins/entity.rb +6 -6
  73. data/lib/shrine/plugins/metadata_attributes.rb +1 -1
  74. data/lib/shrine/plugins/model.rb +3 -3
  75. data/lib/shrine/plugins/presign_endpoint.rb +2 -2
  76. data/lib/shrine/plugins/pretty_location.rb +1 -1
  77. data/lib/shrine/plugins/processing.rb +1 -1
  78. data/lib/shrine/plugins/refresh_metadata.rb +2 -2
  79. data/lib/shrine/plugins/remote_url.rb +3 -3
  80. data/lib/shrine/plugins/remove_invalid.rb +10 -5
  81. data/lib/shrine/plugins/signature.rb +7 -6
  82. data/lib/shrine/plugins/store_dimensions.rb +18 -9
  83. data/lib/shrine/plugins/type_predicates.rb +113 -0
  84. data/lib/shrine/plugins/upload_endpoint.rb +3 -3
  85. data/lib/shrine/plugins/upload_options.rb +2 -2
  86. data/lib/shrine/plugins/url_options.rb +2 -2
  87. data/lib/shrine/plugins/validation.rb +9 -7
  88. data/lib/shrine/storage/linter.rb +4 -4
  89. data/lib/shrine/storage/s3.rb +62 -38
  90. data/lib/shrine/uploaded_file.rb +5 -1
  91. data/lib/shrine/version.rb +2 -2
  92. data/shrine.gemspec +6 -7
  93. metadata +23 -29
@@ -28,12 +28,12 @@ photo = Photo.new
28
28
 
29
29
  photo.image = file # cache attachment
30
30
 
31
- photo.image #=> #<Shrine::UploadedFile @id="bc2e13.jpg" @storage_key=:cache ...>
31
+ photo.image #=> #<Shrine::UploadedFile id="bc2e13.jpg" storage=:cache ...>
32
32
  photo.image_data #=> '{"id":"bc2e13.jpg","storage":"cache","metadata":{...}}'
33
33
 
34
34
  photo.save # persist, promote attachment, then persist again
35
35
 
36
- photo.image #=> #<Shrine::UploadedFile @id="397eca.jpg" @storage_key=:store ...>
36
+ photo.image #=> #<Shrine::UploadedFile id="397eca.jpg" storage=:store ...>
37
37
  photo.image_data #=> '{"id":"397eca.jpg","storage":"store","metadata":{...}}'
38
38
 
39
39
  photo.destroy # delete attachment
@@ -158,7 +158,7 @@ end
158
158
  ```
159
159
  ```yml
160
160
  en:
161
- activerecord
161
+ activerecord:
162
162
  errors:
163
163
  models:
164
164
  photo:
@@ -192,14 +192,14 @@ attacher = ImageUploader::Attacher.from_model(photo, :image)
192
192
 
193
193
  attacher.assign(file) # cache
194
194
 
195
- attacher.file #=> #<Shrine::UploadedFile @id="bc2e13.jpg" @storage_key=:cache ...>
195
+ attacher.file #=> #<Shrine::UploadedFile id="bc2e13.jpg" storage=:cache ...>
196
196
  photo.image_data #=> '{"id":"bc2e13.jpg","storage":"cache","metadata":{...}}'
197
197
 
198
198
  photo.save # persist
199
199
  attacher.finalize # promote
200
200
  photo.save # persist
201
201
 
202
- attacher.file #=> #<Shrine::UploadedFile @id="397eca.jpg" @storage_key=:store ...>
202
+ attacher.file #=> #<Shrine::UploadedFile id="397eca.jpg" storage=:store ...>
203
203
  photo.image_data #=> '{"id":"397eca.jpg","storage":"store","metadata":{...}}'
204
204
  ```
205
205
 
@@ -2,76 +2,110 @@
2
2
  title: Add Metadata
3
3
  ---
4
4
 
5
- The [`add_metadata`][add_metadata] plugin provides a convenient method for
6
- extracting and adding custom metadata values.
5
+ The [`add_metadata`][add_metadata] plugin allows adding custom metadata to
6
+ uploaded files.
7
7
 
8
8
  ```rb
9
- plugin :add_metadata
9
+ Shrine.plugin :add_metadata
10
+ ```
11
+
12
+ ## Metadata block
10
13
 
11
- add_metadata :exif do |io|
12
- begin
13
- Exif::Data.new(io).to_h
14
- rescue Exif::NotReadable # not a valid image
15
- {}
14
+ The `Shrine.add_metadata` method allows you to register a block that will get
15
+ executed on upload, where you can return custom metadata:
16
+
17
+ ```rb
18
+ require "pdf-reader" # https://github.com/yob/pdf-reader
19
+
20
+ class PdfUploader < Shrine
21
+ add_metadata :page_count do |io|
22
+ reader = PDF::Reader.new(io)
23
+ reader.page_count
16
24
  end
17
25
  end
18
26
  ```
19
27
 
20
- The above will add "exif" to the metadata hash, and also create the `#exif`
21
- reader method on `Shrine::UploadedFile`.
28
+ The above will add `page_count` key to the metadata hash, and also create the
29
+ `#page_count` reader method on the `Shrine::UploadedFile`.
22
30
 
23
31
  ```rb
24
- image.metadata["exif"]
32
+ uploaded_file.metadata["page_count"] #=> 30
25
33
  # or
26
- image.exif
34
+ uploaded_file.page_count #=> 30
27
35
  ```
28
36
 
29
- ## Multiple values
37
+ ### Multiple values
30
38
 
31
39
  You can also extract multiple metadata values at once, by using `add_metadata`
32
40
  without an argument and returning a hash of metadata.
33
41
 
34
42
  ```rb
35
- add_metadata do |io|
36
- begin
37
- data = Exif::Data.new(io)
38
- rescue Exif::NotReadable # not a valid image
39
- next {}
43
+ require "exif" # https://github.com/tonytonyjan/exif
44
+
45
+ class ImageUploader < Shrine
46
+ add_metadata do |io|
47
+ begin
48
+ data = Exif::Data.new(io)
49
+ rescue Exif::NotReadable # not a valid image
50
+ next {}
51
+ end
52
+
53
+ { "date_time" => data.date_time,
54
+ "flash" => data.flash,
55
+ "focal_length" => data.focal_length,
56
+ "exposure_time" => data.exposure_time }
40
57
  end
41
-
42
- { date_time: data.date_time,
43
- flash: data.flash,
44
- focal_length: data.focal_length,
45
- exposure_time: data.exposure_time }
46
58
  end
47
59
  ```
60
+ ```rb
61
+ uploaded_file.metadata #=>
62
+ # {
63
+ # ...
64
+ # "date_time" => "2019:07:20 16:16:08",
65
+ # "flash" => 16,
66
+ # "focal_length" => 26/1,
67
+ # "exposure_time" => 1/500,
68
+ # }
69
+ ```
48
70
 
49
71
  In this case Shrine won't automatically create reader methods for the extracted
50
- metadata on Shrine::UploadedFile, but you can create them via
51
- `#metadata_method`.
72
+ metadata, but you can create them via `Shrine.metadata_method`:
52
73
 
53
74
  ```rb
54
- metadata_method :date_time, :flash
75
+ class ImageUploader < Shrine
76
+ # ...
77
+ metadata_method :date_time, :flash
78
+ end
79
+ ```
80
+ ```rb
81
+ uploaded_file.date_time #=> "2019:07:20 16:16:08"
82
+ uploaded_file.flash #=> 16
55
83
  ```
56
84
 
57
- ## Ensuring file
85
+ ### Ensuring file
58
86
 
59
87
  The `io` might not always be a file object, so if you're using an analyzer
60
88
  which requires the source file to be on disk, you can use `Shrine.with_file` to
61
89
  ensure you have a file object.
62
90
 
63
91
  ```rb
64
- add_metadata do |io|
65
- movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
66
-
67
- { "duration" => movie.duration,
68
- "bitrate" => movie.bitrate,
69
- "resolution" => movie.resolution,
70
- "frame_rate" => movie.frame_rate }
92
+ require "streamio-ffmpeg" # https://github.com/streamio/streamio-ffmpeg
93
+
94
+ class VideoUploader < Shrine
95
+ add_metadata do |io|
96
+ movie = Shrine.with_file(io) do |file|
97
+ FFMPEG::Movie.new(file.path)
98
+ end
99
+
100
+ { "duration" => movie.duration,
101
+ "bitrate" => movie.bitrate,
102
+ "resolution" => movie.resolution,
103
+ "frame_rate" => movie.frame_rate }
104
+ end
71
105
  end
72
106
  ```
73
107
 
74
- ## Uploader options
108
+ ### Uploader options
75
109
 
76
110
  Uploader options are also yielded to the block, you can access them for more
77
111
  context:
@@ -89,6 +123,8 @@ add_metadata do |io, **options|
89
123
  end
90
124
  ```
91
125
 
126
+ #### Metadata
127
+
92
128
  The `:metadata` option holds metadata that was extracted so far:
93
129
 
94
130
  ```rb
@@ -116,4 +152,25 @@ add_metadata :bar do |io, metadata:, **|
116
152
  end
117
153
  ```
118
154
 
155
+ ## Updating metadata
156
+
157
+ If you just wish to add some custom metadata to existing uploads, you can do it
158
+ with `UploadedFile#add_metadata` (and write the changes back to the model):
159
+
160
+ ```rb
161
+ attacher.file.add_metadata("foo" => "bar")
162
+ attacher.write # write changes to the model attribute
163
+
164
+ attacher.file.metadata #=> { ..., "foo" => "bar" }
165
+ ```
166
+
167
+ You can also use the `Attacher#add_metadata` shorthand, which also takes care
168
+ of syncing the model:
169
+
170
+ ```rb
171
+ attacher.add_metadata("foo" => "bar")
172
+
173
+ attacher.file.metadata #=> { ..., "foo" => "bar" }
174
+ ```
175
+
119
176
  [add_metadata]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/add_metadata.rb
@@ -187,12 +187,22 @@ and make the execution synchronous, you can override them on the attacher level
187
187
  and call the default behaviour:
188
188
 
189
189
  ```rb
190
- photo.image_attacher.promote_block(&:promote)
191
- photo.image_attacher.destroy_block(&:destroy)
190
+ photo.image_attacher.promote_block { promote } # promote synchronously
191
+ photo.image_attacher.destroy_block { destroy } # destroy synchronously
192
192
 
193
193
  # ... now promotion and deletion will be synchronous ...
194
194
  ```
195
195
 
196
+ You can also do this on the class level if you want to disable backgrounding
197
+ that was set up by a superclass:
198
+
199
+ ```rb
200
+ class MyUploader < Shrine
201
+ Attacher.promote_block { promote } # promote synchronously
202
+ Attacher.destroy_block { destroy } # destroy synchronously
203
+ end
204
+ ```
205
+
196
206
  [backgrounding]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/backgrounding.rb
197
207
  [derivatives]: https://shrinerb.com/docs/plugins/derivatives
198
208
  [atomic_helpers]: https://shrinerb.com/docs/plugins/atomic_helpers
@@ -66,23 +66,48 @@ If you want to load attachment from a Hash, use `Attacher.from_data` or
66
66
 
67
67
  ## Serializer
68
68
 
69
- By default the `JSON` standard library is used as the serializer, but you can
70
- use your own serializer. The serializer object needs to implement `#dump` and
71
- `#load` methods.
69
+ By default, the `JSON` standard library is used for serializing hash data. With
70
+ the [`model`][model] and [`entity`][entity] plugin, the data is serialized
71
+ before writing to and deserialized after reading from the data attribute.
72
+
73
+ You can also use your own serializer via the `:serializer` option. The
74
+ serializer object needs to implement `#dump` and `#load` methods:
75
+
76
+ ```rb
77
+ class MyDataSerializer
78
+ def self.dump(data)
79
+ data #=> { "id" => "...", "storage" => "...", "metadata" => { ... } }
80
+
81
+ JSON.generate(data) # serialize data, e.g. into JSON
82
+ end
83
+
84
+ def self.load(data)
85
+ data #=> '{"id":"...", "storage":"...", "metadata": {...}}'
86
+
87
+ JSON.parse(data) # deserialize data, e.g. from JSON
88
+ end
89
+ end
90
+
91
+ plugin :column, serializer: MyDataSerializer
92
+ ```
93
+
94
+ Some serialization libraries such as [Oj] and [MessagePack] already implement
95
+ this interface, which simplifies the configuration:
72
96
 
73
97
  ```rb
74
- require "oj"
98
+ require "oj" # https://github.com/ohler55/oj
75
99
 
76
- plugin :column, serializer: Oj # use custom serializer
100
+ plugin :column, serializer: Oj
77
101
  ```
78
102
 
79
- If you want to disable serialization, you can set serializer to `nil`.
103
+ If you want to disable serialization and work with hashes directly, you can set
104
+ `:serializer` to `nil`:
80
105
 
81
106
  ```rb
82
107
  plugin :column, serializer: nil # disable serialization
83
108
  ```
84
109
 
85
- You can also change the serializer on the attacher level:
110
+ The serializer can also be changed for a particular attacher instance:
86
111
 
87
112
  ```rb
88
113
  Shrine::Attacher.new(column_serializer: Oj) # use custom serializer
@@ -90,3 +115,7 @@ Shrine::Attacher.new(column_serializer: nil) # disable serialization
90
115
  ```
91
116
 
92
117
  [column]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/column.rb
118
+ [model]: https://shrinerb.com/docs/plugins/model
119
+ [entity]: https://shrinerb.com/docs/plugins/entity
120
+ [Oj]: https://github.com/ohler55/oj
121
+ [MessagePack]: https://github.com/msgpack/msgpack-ruby
@@ -123,7 +123,7 @@ payload:
123
123
 
124
124
  A default log subscriber is added as well which logs these events:
125
125
 
126
- ```plaintext
126
+ ```
127
127
  Data URI (5ms) – {:uploader=>Shrine}
128
128
  ```
129
129
 
@@ -134,7 +134,7 @@ plugin :data_uri, log_subscriber: -> (event) {
134
134
  Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, uploader: event[:uploader])
135
135
  }
136
136
  ```
137
- ```plaintext
137
+ ```
138
138
  {"name":"data_uri","duration":5,"uploader":"Shrine"}
139
139
  ```
140
140
 
@@ -8,7 +8,7 @@ returned when there is no attached file.
8
8
  ```rb
9
9
  plugin :default_url
10
10
 
11
- Attacher.default_url do |options|
11
+ Attacher.default_url do |**options|
12
12
  "/#{name}/missing.jpg"
13
13
  end
14
14
  ```
@@ -28,7 +28,7 @@ Any URL options passed will be available in the default URL block:
28
28
  attacher.url(foo: "bar")
29
29
  ```
30
30
  ```rb
31
- Attacher.default_url do |options|
31
+ Attacher.default_url do |**options|
32
32
  options #=> { foo: "bar" }
33
33
  end
34
34
  ```
@@ -37,12 +37,15 @@ The default URL block is evaluated in the context of an instance of
37
37
  `Shrine::Attacher`.
38
38
 
39
39
  ```rb
40
- Attacher.default_url do |options|
40
+ Attacher.default_url do |**options|
41
41
  self #=> #<Shrine::Attacher>
42
42
 
43
+ file #=> #<Shrine::UploadedFile>
43
44
  name #=> :avatar
44
45
  record #=> #<User>
45
46
  context #=> { ... }
47
+
48
+ # ...
46
49
  end
47
50
  ```
48
51
 
@@ -68,7 +68,7 @@ generates an URL consisting of the configured [path prefix](#prefix),
68
68
  derivation name and arguments, serialized uploaded file, and an URL signature
69
69
  generated using the configured secret key:
70
70
 
71
- ```plaintext
71
+ ```
72
72
  / derivations/image / thumbnail / 600/400 / eyJmZvbyIb3JhZ2UiOiJzdG9yZSJ9 ? signature=...
73
73
  └──── prefix ─────┘ └── name ──┘ └─ args ─┘ └─── serialized source file ───┘
74
74
  ```
@@ -197,7 +197,8 @@ Rails.application.routes.draw do
197
197
  end
198
198
  end
199
199
  end
200
-
200
+ ```
201
+ ```rb
201
202
  # app/controllers/photos_controller.rb
202
203
  class PhotosController < ApplicationController
203
204
  def thumbnail
@@ -483,32 +484,27 @@ such as AWS S3 or Google Cloud Storage.
483
484
  ### Deleting derivatives
484
485
 
485
486
  When the original attachment is deleted, its uploaded derivatives will not be
486
- automatically deleted, you will need to do the deletion manually. To ensure
487
- this gets called both on destroying and replacing, you can add that code to
488
- `Attacher#destroy`.
487
+ automatically deleted, you will need to do the deletion manually. If you're
488
+ using [backgrounding], you can do this in your `DestroyJob`.
489
489
 
490
- The easiest way is to delete the directory containing your derivatives:
490
+ If your storage implements `#delete_prefixed`, and you're using the default
491
+ [`:upload_location`](#upload-location), you can delete the directory containing
492
+ derivatives:
491
493
 
492
494
  ```rb
493
- class ImageUploader < Shrine
494
- class Attacher
495
- def destroy(*args)
496
- super
495
+ class DestroyJob < ActiveJob::Base
496
+ def perform(attacher_class, data)
497
+ # ... destroy attached file ...
497
498
 
498
- derivatives_directory = file.id
499
- storage = store.storage
499
+ derivatives_directory = attacher.file.id + "/"
500
+ storage = attacher.store.storage
500
501
 
501
- storage.delete_prefixed(derivatives_directory)
502
- end
502
+ storage.delete_prefixed(derivatives_directory)
503
503
  end
504
504
  end
505
505
  ```
506
506
 
507
- This is under the assumption that your storage implements `#delete_prefixed`
508
- and that you're using default [`:upload_location`](#upload-location).
509
-
510
- Alternatively, you can delete each derivative individually using
511
- `Derivation#delete`:
507
+ Alternatively, you can delete each derivative individually:
512
508
 
513
509
  ```rb
514
510
  class ImageUploader < Shrine
@@ -518,14 +514,15 @@ class ImageUploader < Shrine
518
514
  [:thumbnail, 400, 300],
519
515
  ...
520
516
  ]
517
+ end
518
+ ```
519
+ ```rb
520
+ class DestroyJob < ActiveJob::Base
521
+ def perform(attacher_class, data)
522
+ # ... destroy attached file ...
521
523
 
522
- class Attacher
523
- def destroy(*args)
524
- super
525
-
526
- DERIVATIONS.each do |args|
527
- file.derivation(*args).delete
528
- end
524
+ attacher.shrine_class::DERIVATIONS.each do |args|
525
+ attacher.file.derivation(*args).delete
529
526
  end
530
527
  end
531
528
  end
@@ -830,7 +827,7 @@ following payload:
830
827
 
831
828
  A default log subscriber is added as well which logs these events:
832
829
 
833
- ```plaintext
830
+ ```
834
831
  Derivation (492ms) – {:name=>:thumbnail, :args=>[600, 600], :uploader=>Shrine}
835
832
  ```
836
833
 
@@ -846,7 +843,7 @@ plugin :derivation_endpoint, log_subscriber: -> (event) {
846
843
  )
847
844
  }
848
845
  ```
849
- ```plaintext
846
+ ```
850
847
  {"name":"derivation","duration":492,"name":"thumbnail","args":[600,600],"uploader":"Shrine"}
851
848
  ```
852
849
 
@@ -861,3 +858,4 @@ plugin :derivation_endpoint, log_subscriber: nil
861
858
  [`Content-Type`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type
862
859
  [`Content-Disposition`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition
863
860
  [`Cache-Control`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
861
+ [backgrounding]: https://shrinerb.com/docs/plugins/backgrounding