shrine 2.19.3 → 3.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (211) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +523 -41
  3. data/LICENSE.txt +1 -1
  4. data/README.md +83 -979
  5. data/doc/advantages.md +231 -204
  6. data/doc/attacher.md +304 -153
  7. data/doc/carrierwave.md +297 -226
  8. data/doc/changing_derivatives.md +308 -0
  9. data/doc/changing_location.md +103 -21
  10. data/doc/changing_storage.md +110 -0
  11. data/doc/creating_persistence_plugins.md +132 -0
  12. data/doc/creating_plugins.md +43 -23
  13. data/doc/creating_storages.md +19 -5
  14. data/doc/design.md +147 -97
  15. data/doc/direct_s3.md +38 -28
  16. data/doc/external/articles.md +63 -0
  17. data/doc/external/extensions.md +53 -0
  18. data/doc/external/misc.md +32 -0
  19. data/doc/getting_started.md +1156 -0
  20. data/doc/metadata.md +190 -109
  21. data/doc/multiple_files.md +93 -30
  22. data/doc/paperclip.md +384 -262
  23. data/doc/plugins/activerecord.md +177 -46
  24. data/doc/plugins/add_metadata.md +139 -38
  25. data/doc/plugins/atomic_helpers.md +217 -0
  26. data/doc/plugins/backgrounding.md +156 -98
  27. data/doc/plugins/cached_attachment_data.md +7 -5
  28. data/doc/plugins/column.md +121 -0
  29. data/doc/plugins/data_uri.md +23 -22
  30. data/doc/plugins/default_storage.md +36 -10
  31. data/doc/plugins/default_url.md +30 -13
  32. data/doc/plugins/delete_raw.md +4 -2
  33. data/doc/plugins/derivation_endpoint.md +186 -101
  34. data/doc/plugins/derivatives.md +839 -0
  35. data/doc/plugins/determine_mime_type.md +4 -2
  36. data/doc/plugins/download_endpoint.md +64 -8
  37. data/doc/plugins/dynamic_storage.md +5 -3
  38. data/doc/plugins/entity.md +263 -0
  39. data/doc/plugins/form_assign.md +55 -0
  40. data/doc/plugins/included.md +31 -8
  41. data/doc/plugins/infer_extension.md +21 -10
  42. data/doc/plugins/instrumentation.md +38 -16
  43. data/doc/plugins/keep_files.md +16 -17
  44. data/doc/plugins/metadata_attributes.md +42 -13
  45. data/doc/plugins/mirroring.md +118 -0
  46. data/doc/plugins/model.md +210 -0
  47. data/doc/plugins/module_include.md +4 -2
  48. data/doc/plugins/multi_cache.md +24 -0
  49. data/doc/plugins/persistence.md +101 -0
  50. data/doc/plugins/presign_endpoint.md +9 -4
  51. data/doc/plugins/pretty_location.md +16 -3
  52. data/doc/plugins/processing.md +4 -2
  53. data/doc/plugins/rack_file.md +8 -2
  54. data/doc/plugins/rack_response.md +6 -2
  55. data/doc/plugins/recache.md +4 -2
  56. data/doc/plugins/refresh_metadata.md +49 -9
  57. data/doc/plugins/remote_url.md +84 -47
  58. data/doc/plugins/remove_attachment.md +27 -6
  59. data/doc/plugins/remove_invalid.md +21 -6
  60. data/doc/plugins/restore_cached_data.md +11 -3
  61. data/doc/plugins/sequel.md +159 -35
  62. data/doc/plugins/signature.md +16 -5
  63. data/doc/plugins/store_dimensions.md +14 -2
  64. data/doc/plugins/tempfile.md +4 -2
  65. data/doc/plugins/type_predicates.md +96 -0
  66. data/doc/plugins/upload_endpoint.md +13 -13
  67. data/doc/plugins/upload_options.md +6 -4
  68. data/doc/plugins/{default_url_options.md → url_options.md} +9 -7
  69. data/doc/plugins/validation.md +97 -0
  70. data/doc/plugins/validation_helpers.md +16 -13
  71. data/doc/plugins/versions.md +15 -19
  72. data/doc/processing.md +438 -221
  73. data/doc/refile.md +188 -170
  74. data/doc/release_notes/1.0.0.md +4 -0
  75. data/doc/release_notes/1.1.0.md +6 -2
  76. data/doc/release_notes/1.2.0.md +4 -0
  77. data/doc/release_notes/1.3.0.md +4 -0
  78. data/doc/release_notes/1.4.0.md +4 -0
  79. data/doc/release_notes/1.4.1.md +4 -0
  80. data/doc/release_notes/1.4.2.md +4 -0
  81. data/doc/release_notes/2.0.0.md +4 -0
  82. data/doc/release_notes/2.0.1.md +4 -0
  83. data/doc/release_notes/2.1.0.md +5 -1
  84. data/doc/release_notes/2.1.1.md +4 -0
  85. data/doc/release_notes/2.10.0.md +4 -0
  86. data/doc/release_notes/2.10.1.md +4 -0
  87. data/doc/release_notes/2.11.0.md +4 -0
  88. data/doc/release_notes/2.12.0.md +4 -0
  89. data/doc/release_notes/2.13.0.md +4 -0
  90. data/doc/release_notes/2.14.0.md +5 -1
  91. data/doc/release_notes/2.15.0.md +11 -7
  92. data/doc/release_notes/2.16.0.md +4 -0
  93. data/doc/release_notes/2.17.0.md +4 -0
  94. data/doc/release_notes/2.18.0.md +4 -0
  95. data/doc/release_notes/2.19.0.md +6 -3
  96. data/doc/release_notes/2.2.0.md +4 -0
  97. data/doc/release_notes/2.3.0.md +4 -0
  98. data/doc/release_notes/2.3.1.md +4 -0
  99. data/doc/release_notes/2.4.0.md +4 -0
  100. data/doc/release_notes/2.4.1.md +4 -0
  101. data/doc/release_notes/2.5.0.md +4 -0
  102. data/doc/release_notes/2.6.0.md +4 -0
  103. data/doc/release_notes/2.6.1.md +4 -0
  104. data/doc/release_notes/2.7.0.md +4 -0
  105. data/doc/release_notes/2.8.0.md +4 -0
  106. data/doc/release_notes/2.9.0.md +4 -0
  107. data/doc/release_notes/3.0.0.md +981 -0
  108. data/doc/release_notes/3.0.1.md +22 -0
  109. data/doc/release_notes/3.1.0.md +73 -0
  110. data/doc/release_notes/3.2.0.md +96 -0
  111. data/doc/release_notes/3.2.1.md +31 -0
  112. data/doc/release_notes/3.2.2.md +14 -0
  113. data/doc/release_notes/3.3.0.md +105 -0
  114. data/doc/release_notes/3.4.0.md +35 -0
  115. data/doc/release_notes/3.5.0.md +63 -0
  116. data/doc/release_notes/3.6.0.md +23 -0
  117. data/doc/retrieving_uploads.md +5 -2
  118. data/doc/securing_uploads.md +60 -37
  119. data/doc/storage/file_system.md +20 -3
  120. data/doc/storage/memory.md +19 -0
  121. data/doc/storage/s3.md +122 -78
  122. data/doc/testing.md +141 -133
  123. data/doc/upgrading_to_3.md +708 -0
  124. data/doc/validation.md +54 -90
  125. data/lib/shrine/attacher.rb +292 -169
  126. data/lib/shrine/attachment.rb +13 -46
  127. data/lib/shrine/plugins/_persistence.rb +93 -0
  128. data/lib/shrine/plugins/activerecord.rb +77 -34
  129. data/lib/shrine/plugins/add_metadata.rb +25 -17
  130. data/lib/shrine/plugins/atomic_helpers.rb +119 -0
  131. data/lib/shrine/plugins/backgrounding.rb +77 -113
  132. data/lib/shrine/plugins/cached_attachment_data.rb +6 -15
  133. data/lib/shrine/plugins/column.rb +102 -0
  134. data/lib/shrine/plugins/data_uri.rb +38 -36
  135. data/lib/shrine/plugins/default_storage.rb +45 -15
  136. data/lib/shrine/plugins/default_url.rb +12 -24
  137. data/lib/shrine/plugins/default_url_options.rb +3 -30
  138. data/lib/shrine/plugins/delete_raw.rb +10 -16
  139. data/lib/shrine/plugins/derivation_endpoint.rb +130 -171
  140. data/lib/shrine/plugins/derivatives.rb +645 -0
  141. data/lib/shrine/plugins/determine_mime_type.rb +9 -21
  142. data/lib/shrine/plugins/download_endpoint.rb +118 -133
  143. data/lib/shrine/plugins/dynamic_storage.rb +5 -11
  144. data/lib/shrine/plugins/entity.rb +158 -0
  145. data/lib/shrine/plugins/form_assign.rb +108 -0
  146. data/lib/shrine/plugins/included.rb +6 -6
  147. data/lib/shrine/plugins/infer_extension.rb +17 -20
  148. data/lib/shrine/plugins/instrumentation.rb +59 -43
  149. data/lib/shrine/plugins/keep_files.rb +3 -15
  150. data/lib/shrine/plugins/metadata_attributes.rb +28 -19
  151. data/lib/shrine/plugins/mirroring.rb +142 -0
  152. data/lib/shrine/plugins/model.rb +160 -0
  153. data/lib/shrine/plugins/module_include.rb +3 -3
  154. data/lib/shrine/plugins/multi_cache.rb +27 -0
  155. data/lib/shrine/plugins/presign_endpoint.rb +27 -28
  156. data/lib/shrine/plugins/pretty_location.rb +15 -9
  157. data/lib/shrine/plugins/processing.rb +22 -9
  158. data/lib/shrine/plugins/rack_file.rb +2 -42
  159. data/lib/shrine/plugins/rack_response.rb +21 -10
  160. data/lib/shrine/plugins/recache.rb +6 -5
  161. data/lib/shrine/plugins/refresh_metadata.rb +13 -11
  162. data/lib/shrine/plugins/remote_url.rb +49 -49
  163. data/lib/shrine/plugins/remove_attachment.rb +12 -6
  164. data/lib/shrine/plugins/remove_invalid.rb +19 -8
  165. data/lib/shrine/plugins/restore_cached_data.rb +13 -7
  166. data/lib/shrine/plugins/sequel.rb +86 -36
  167. data/lib/shrine/plugins/signature.rb +10 -16
  168. data/lib/shrine/plugins/store_dimensions.rb +35 -40
  169. data/lib/shrine/plugins/tempfile.rb +1 -3
  170. data/lib/shrine/plugins/type_predicates.rb +113 -0
  171. data/lib/shrine/plugins/upload_endpoint.rb +28 -24
  172. data/lib/shrine/plugins/upload_options.rb +14 -15
  173. data/lib/shrine/plugins/url_options.rb +31 -0
  174. data/lib/shrine/plugins/validation.rb +80 -0
  175. data/lib/shrine/plugins/validation_helpers.rb +35 -58
  176. data/lib/shrine/plugins/versions.rb +107 -87
  177. data/lib/shrine/plugins.rb +22 -0
  178. data/lib/shrine/storage/file_system.rb +46 -64
  179. data/lib/shrine/storage/linter.rb +42 -7
  180. data/lib/shrine/storage/memory.rb +49 -0
  181. data/lib/shrine/storage/s3.rb +173 -160
  182. data/lib/shrine/uploaded_file.rb +32 -32
  183. data/lib/shrine/version.rb +3 -3
  184. data/lib/shrine.rb +87 -150
  185. data/shrine.gemspec +11 -12
  186. metadata +92 -82
  187. data/doc/migrating_storage.md +0 -76
  188. data/doc/plugins/backup.md +0 -31
  189. data/doc/plugins/copy.md +0 -24
  190. data/doc/plugins/delete_promoted.md +0 -12
  191. data/doc/plugins/direct_upload.md +0 -172
  192. data/doc/plugins/hooks.md +0 -58
  193. data/doc/plugins/logging.md +0 -42
  194. data/doc/plugins/migration_helpers.md +0 -60
  195. data/doc/plugins/moving.md +0 -19
  196. data/doc/plugins/multi_delete.md +0 -20
  197. data/doc/plugins/parallelize.md +0 -16
  198. data/doc/plugins/parsed_json.md +0 -23
  199. data/doc/regenerating_versions.md +0 -143
  200. data/lib/shrine/plugins/background_helpers.rb +0 -5
  201. data/lib/shrine/plugins/backup.rb +0 -90
  202. data/lib/shrine/plugins/copy.rb +0 -50
  203. data/lib/shrine/plugins/delete_promoted.rb +0 -20
  204. data/lib/shrine/plugins/direct_upload.rb +0 -217
  205. data/lib/shrine/plugins/hooks.rb +0 -90
  206. data/lib/shrine/plugins/logging.rb +0 -142
  207. data/lib/shrine/plugins/migration_helpers.rb +0 -70
  208. data/lib/shrine/plugins/moving.rb +0 -57
  209. data/lib/shrine/plugins/multi_delete.rb +0 -32
  210. data/lib/shrine/plugins/parallelize.rb +0 -78
  211. data/lib/shrine/plugins/parsed_json.rb +0 -29
data/doc/processing.md CHANGED
@@ -1,215 +1,335 @@
1
- # File Processing
1
+ ---
2
+ title: File Processing
3
+ ---
2
4
 
3
- Shrine allows you to process files in two ways. One is processing "[on
4
- upload](#processing-on-upload)", where the processing gets triggered when the file is
5
- attached to a record. The other is "[on-the-fly](#on-the-fly-processing)"
6
- processing, where the processing is performed lazily at the moment the file is
7
- requested.
8
-
9
- With both ways you need to define some kind of processing block, which accepts
10
- a source file and is expected to return the processed result file.
11
-
12
- ```rb
13
- some_process_block do |source_file|
14
- # process source file and return the result
15
- end
16
- ```
5
+ Shrine allows you to process attached files eagerly or on-the-fly. For
6
+ example, if your app is accepting image uploads, you can generate a predefined
7
+ set of of thumbnails when the image is attached to a record, or you can have
8
+ thumbnails generated dynamically as they're needed.
17
9
 
18
10
  How you're going to implement processing is entirely up to you. For images it's
19
11
  recommended to use the **[ImageProcessing]** gem, which provides wrappers for
20
- processing with [ImageMagick]/[GraphicsMagick] (using the [MiniMagick] gem) or
21
- [libvips] (using the [ruby-vips] gem; see the [libvips section](#libvips)).
22
- Here is an example of generating a thumbnail with ImageProcessing:
12
+ processing with [MiniMagick][ImageProcessing::MiniMagick] and
13
+ [libvips][ImageProcessing::Vips]. Here is an example of generating a thumbnail
14
+ with ImageProcessing:
23
15
 
24
- ```sh
16
+ ```
25
17
  $ brew install imagemagick
26
18
  ```
27
-
28
19
  ```rb
29
20
  # Gemfile
30
- gem "image_processing", "~> 1.0"
21
+ gem "image_processing", "~> 1.8"
31
22
  ```
32
-
33
23
  ```rb
34
24
  require "image_processing/mini_magick"
35
25
 
36
26
  thumbnail = ImageProcessing::MiniMagick
37
- .source(image)
38
- .resize_to_limit!(600, 400)
27
+ .source(image) # input file
28
+ .resize_to_limit(600, 400) # resize macro
29
+ .colorspace("grayscale") # custom operation
30
+ .convert("jpeg") # output type
31
+ .saver(quality: 90) # output options
32
+ .call # run the pipeline
39
33
 
40
34
  thumbnail #=> #<Tempfile:...> (a 600x400 thumbnail of the source image)
41
35
  ```
42
36
 
43
- ## Processing on upload
37
+ ## Eager processing
44
38
 
45
- Shrine allows you to process files before they're uploaded to a storage. It's
46
- generally best to process cached files when they're being promoted to permanent
47
- storage, because (a) at that point the file has already been successfully
48
- [validated][validation], (b) the parent record has been saved and the database
49
- transaction has been committed, and (c) this can be delayed into a [background
50
- job][backgrounding].
51
-
52
- You can define processing using the `processing` plugin, which we'll use to
53
- hook into the `:store` phase (when cached file is uploaded to permanent
54
- storage).
39
+ Let's say we're handling images, and want to generate a predefined set of
40
+ thumbnails with various dimensions. We can use the
41
+ **[`derivatives`][derivatives]** plugin to upload and save the processed files:
55
42
 
56
43
  ```rb
57
- class ImageUploader < Shrine
58
- plugin :processing
59
-
60
- process(:store) do |io, context|
61
- io #=> #<Shrine::UploadedFile ...>
62
- context #=> {:record=>#<Photo...>,:name=>:image,...}
44
+ Shrine.plugin :derivatives
45
+ ```
46
+ ```rb
47
+ require "image_processing/mini_magick"
63
48
 
64
- # ...
49
+ class ImageUploader < Shrine
50
+ Attacher.derivatives do |original|
51
+ magick = ImageProcessing::MiniMagick.source(original)
52
+
53
+ {
54
+ large: magick.resize_to_limit!(800, 800),
55
+ medium: magick.resize_to_limit!(500, 500),
56
+ small: magick.resize_to_limit!(300, 300),
57
+ }
65
58
  end
66
59
  end
67
60
  ```
61
+ ```rb
62
+ photo = Photo.new(image: file)
63
+
64
+ if photo.valid?
65
+ photo.image_derivatives! if photo.image_changed? # creates derivatives
66
+ photo.save
67
+ end
68
+ ```
68
69
 
69
- The processing block yields two arguments: a [`Shrine::UploadedFile`] object
70
- representing the file uploaded to temporary storage, and a Hash containing
71
- additional data such as the model instance and attachment name. The block
72
- result should be file(s) that will be uploaded to permanent storage.
70
+ After the processed files are uploaded, their data is saved into the
71
+ `<attachment>_data` column. You can then retrieve the derivatives as
72
+ [`Shrine::UploadedFile`] objects:
73
+
74
+ ```rb
75
+ photo.image(:large) #=> #<Shrine::UploadedFile ...>
76
+ photo.image(:large).url #=> "/uploads/store/lg043.jpg"
77
+ photo.image(:large).size #=> 5825949
78
+ photo.image(:large).mime_type #=> "image/jpeg"
79
+ ```
73
80
 
74
- ### Versions
81
+ ### Conditional derivatives
75
82
 
76
- Let's say we're handling images, and want to generate thumbnails of various
77
- dimensions. In this case we can use the ImageProcessing gem to generate the
78
- thumbnails, and return a hash of processed files at the end of the block. We'll
79
- need to load the `versions` plugin which extends Shrine with the ability to
80
- handle collections of files inside the same attachment.
83
+ The `Attacher.derivatives` block is evaluated in context of a
84
+ `Shrine::Attacher` instance:
81
85
 
82
86
  ```rb
83
- require "image_processing/mini_magick"
87
+ Attacher.derivatives do |original|
88
+ self #=> #<Shrine::Attacher>
84
89
 
85
- class ImageUploader < Shrine
86
- plugin :processing # allows hooking into promoting
87
- plugin :versions # enable Shrine to handle a hash of files
88
- plugin :delete_raw # delete processed files after uploading
90
+ file #=> #<Shrine::UploadedFile>
91
+ record #=> #<Photo>
92
+ name #=> :image
93
+ context #=> { ... }
89
94
 
90
- process(:store) do |io, context|
91
- versions = { original: io } # retain original
95
+ # ...
96
+ end
97
+ ```
92
98
 
93
- # download the uploaded file from the temporary storage
94
- io.download do |original|
95
- pipeline = ImageProcessing::MiniMagick.source(original)
99
+ This gives you the ability to branch the processing logic based on the
100
+ attachment information:
96
101
 
97
- versions[:large] = pipeline.resize_to_limit!(800, 800)
98
- versions[:medium] = pipeline.resize_to_limit!(500, 500)
99
- versions[:small] = pipeline.resize_to_limit!(300, 300)
100
- end
102
+ ```rb
103
+ Attacher.derivatives do |original|
104
+ magick = ImageProcessing::MiniMagick.source(original)
105
+ result = {}
101
106
 
102
- versions # return the hash of processed files
107
+ if record.is_a?(Photo)
108
+ result[:jpg] = magick.convert!("jpeg")
109
+ result[:gray] = magick.colorspace!("grayscale")
103
110
  end
111
+
112
+ if file.mime_type == "image/svg+xml"
113
+ result[:png] = magick.loader(transparent: "white").convert!("png")
114
+ end
115
+
116
+ result
104
117
  end
105
118
  ```
106
119
 
107
- **NOTE: It's recommended to always keep the original file, just in case you'll
108
- ever need to reprocess it.**
120
+ The [`type_predicates`][type_predicates] plugin provides convenient predicate
121
+ methods for branching based on the file type.
122
+
123
+ ### Backgrounding
109
124
 
110
- ### Conditional processing
125
+ Since file processing can be time consuming, it's recommended to move it into a
126
+ background job.
111
127
 
112
- The process block yields the attached file uploaded to temporary storage, so we
113
- have information like file extension and MIME type available. Together with
114
- ImageProcessing's chainable API, it's easy to do conditional proccessing.
128
+ #### A) Creating derivatives with promotion
115
129
 
116
- For example, let's say we want our thumbnails to be either JPEGs or PNGs, and
117
- we also want to save JPEGs as progressive (interlaced). Here's how the code for
118
- this might look like:
130
+ The simplest way is to use the [`backgrounding`][backgrounding] plugin to move
131
+ promotion into a background job, and then create derivatives as part of
132
+ promotion:
119
133
 
120
134
  ```rb
121
- process(:store) do |io, context|
122
- versions = { original: io }
135
+ Shrine.plugin :backgrounding
136
+ Shrine::Attacher.promote_block do
137
+ PromoteJob.perform_async(self.class.name, record.class.name, record.id, name, file_data)
138
+ end
139
+ ```
140
+ ```rb
141
+ class PromoteJob
142
+ include Sidekiq::Worker
143
+
144
+ def perform(attacher_class, record_class, record_id, name, file_data)
145
+ attacher_class = Object.const_get(attacher_class)
146
+ record = Object.const_get(record_class).find(record_id) # if using Active Record
147
+
148
+ attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
149
+ attacher.create_derivatives # calls derivatives processor
150
+ attacher.atomic_promote
151
+ rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
152
+ # attachment has changed or the record has been deleted, nothing to do
153
+ end
154
+ end
155
+ ```
123
156
 
124
- io.download do |original|
125
- pipeline = ImageProcessing::Vips.source(original)
157
+ #### B) Creating derivatives separately from promotion
126
158
 
127
- # Shrine::UploadedFile object contains information about the MIME type
128
- unless io.mime_type == "image/png"
129
- pipeline = pipeline
130
- .convert("jpeg")
131
- .saver(interlace: true)
132
- end
159
+ Derivatives don't need to be created as part of the attachment flow, you can
160
+ create them at any point after promotion:
133
161
 
134
- versions[:large] = pipeline.resize_to_limit!(800, 800)
135
- versions[:medium] = pipeline.resize_to_limit!(500, 500)
136
- versions[:small] = pipeline.resize_to_limit!(300, 300)
162
+ ```rb
163
+ DerivativesJob.perform_async(
164
+ attacher.class.name,
165
+ attacher.record.class.name,
166
+ attacher.record.id,
167
+ attacher.name,
168
+ attacher.file_data,
169
+ )
170
+ ```
171
+ ```rb
172
+ class DerivativesJob
173
+ include Sidekiq::Worker
174
+
175
+ def perform(attacher_class, record_class, record_id, name, file_data)
176
+ attacher_class = Object.const_get(attacher_class)
177
+ record = Object.const_get(record_class).find(record_id) # if using Active Record
178
+
179
+ attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
180
+ attacher.create_derivatives # calls derivatives processor
181
+ attacher.atomic_persist
182
+ rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
183
+ attacher&.destroy_attached # delete now orphaned derivatives
137
184
  end
185
+ end
186
+ ```
187
+
188
+ #### C) Creating derivatives concurrently
189
+
190
+ You can also generate derivatives concurrently:
138
191
 
139
- versions
192
+ ```rb
193
+ class ImageUploader < Shrine
194
+ THUMBNAILS = {
195
+ large: [800, 800],
196
+ medium: [500, 500],
197
+ small: [300, 300],
198
+ }
199
+
200
+ Attacher.derivatives do |original, name:|
201
+ thumbnail = ImageProcessing::MiniMagick
202
+ .source(original)
203
+ .resize_to_limit!(*THUMBNAILS.fetch(name))
204
+
205
+ { name => thumbnail }
206
+ end
207
+ end
208
+ ```
209
+ ```rb
210
+ ImageUploader::THUMBNAILS.each_key do |derivative_name|
211
+ DerivativeJob.perform_async(
212
+ attacher.class.name,
213
+ attacher.record.class.name,
214
+ attacher.record.id,
215
+ attacher.name,
216
+ attacher.file_data,
217
+ derivative_name,
218
+ )
219
+ end
220
+ ```
221
+ ```rb
222
+ class DerivativeJob
223
+ include Sidekiq::Worker
224
+
225
+ def perform(attacher_class, record_class, record_id, name, file_data, derivative_name)
226
+ attacher_class = Object.const_get(attacher_class)
227
+ record = Object.const_get(record_class).find(record_id) # if using Active Record
228
+
229
+ attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
230
+ attacher.create_derivatives(name: derivative_name)
231
+ attacher.atomic_persist do |reloaded_attacher|
232
+ # make sure we don't override derivatives created in other jobs
233
+ attacher.merge_derivatives(reloaded_attacher.derivatives)
234
+ end
235
+ rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
236
+ attacher.derivatives[derivative_name].delete # delete now orphaned derivative
237
+ end
140
238
  end
141
239
  ```
142
240
 
143
- ### Processing other file types
241
+ ### URL fallbacks
144
242
 
145
- So far we've only been talking about processing images. However, there is
146
- nothing image-specific in Shrine's processing API, you can just as well process
147
- any other types of files. The processing tool doesn't need to have any special
148
- Shrine integration, the ImageProcessing gem that we saw earlier is a completely
149
- generic gem.
243
+ If you're creating derivatives in a background job, you'll likely want to use
244
+ some fallbacks for derivative URLs while the background job is still
245
+ processing. You can do that with the [`default_url`][default_url] plugin.
150
246
 
151
- To demonstrate, here is an example of transcoding videos using
152
- [streamio-ffmpeg]:
247
+ ```rb
248
+ Shrine.plugin :default_url
249
+ ```
250
+
251
+ #### A) Fallback to original
252
+
253
+ You can fall back to the original file URL when the derivative is missing:
153
254
 
154
255
  ```rb
155
- require "streamio-ffmpeg"
156
- require "tempfile"
256
+ Attacher.default_url do |derivative: nil, **|
257
+ file&.url if derivative
258
+ end
259
+ ```
260
+ ```rb
261
+ photo.image_url(:large) #=> "https://example.com/path/to/original.jpg"
262
+ # ... background job finishes ...
263
+ photo.image_url(:large) #=> "https://example.com/path/to/large.jpg"
264
+ ```
157
265
 
158
- class VideoUploader < Shrine
159
- plugin :processing
160
- plugin :versions
161
- plugin :delete_raw
266
+ #### B) Fallback to derivative
162
267
 
163
- process(:store) do |io, context|
164
- versions = { original: io }
268
+ You can fall back to another derivative URL when the derivative is missing:
165
269
 
166
- io.download do |original|
167
- transcoded = Tempfile.new(["transcoded", ".mp4"], binmode: true)
168
- screenshot = Tempfile.new(["screenshot", ".jpg"], binmode: true)
270
+ ```rb
271
+ Attacher.default_url do |derivative: nil, **|
272
+ derivatives[:optimized]&.url if derivative
273
+ end
274
+ ```
275
+ ```rb
276
+ photo.image_url(:large) #=> "https://example.com/path/to/optimized.jpg"
277
+ # ... background job finishes ...
278
+ photo.image_url(:large) #=> "https://example.com/path/to/large.jpg"
279
+ ```
169
280
 
170
- movie = FFMPEG::Movie.new(original.path)
171
- movie.transcode(transcoded.path)
172
- movie.screenshot(screenshot.path)
281
+ #### C) Fallback to on-the-fly
173
282
 
174
- [transcoded, screenshot].each(&:open) # refresh file descriptors
283
+ You can also fall back to [on-the-fly processing](#on-the-fly-processing),
284
+ which should generally provide the best user experience.
175
285
 
176
- versions.merge!(transcoded: transcoded, screenshot: screenshot)
177
- end
286
+ ```rb
287
+ THUMBNAILS = {
288
+ small: [300, 300],
289
+ medium: [500, 500],
290
+ large: [800, 800],
291
+ }
178
292
 
179
- versions
180
- end
293
+ Attacher.default_url do |derivative: nil, **|
294
+ file&.derivation_url(:thumbnail, *THUMBNAILS.fetch(derivative)) if derivative
181
295
  end
182
296
  ```
297
+ ```rb
298
+ photo.image_url(:large) #=> "../derivations/thumbnail/800/800/..."
299
+ # ... background job finishes ...
300
+ photo.image_url(:large) #=> "https://example.com/path/to/large.jpg"
301
+ ```
183
302
 
184
303
  ## On-the-fly processing
185
304
 
186
- Generating image thumbnails on upload can be a pain to maintain, because
305
+ Having eagerly created image thumbnails can be a pain to maintain, because
187
306
  whenever you need to add a new version or change an existing one, you need to
188
- retroactively apply it to all existing uploads (see the [Reprocessing Versions]
189
- guide for more details).
307
+ retroactively apply it to all existing attachments (see the [Managing
308
+ Derivatives] guide for more details).
190
309
 
191
- As an alternative, it's very common to instead generate thumbnails dynamically
192
- as they're requested, and then cache them for future requests. This strategy is
193
- known as "on-the-fly processing", and it's suitable for generating thumbnails
194
- or document previews.
310
+ Sometimes it makes more sense to generate thumbnails dynamically as they're
311
+ requested, and then cache them for future requests. This strategy is known as
312
+ processing "**on-the-fly**" or "**on-demand**", and it's suitable for
313
+ short-running processing such as creating image thumbnails or document
314
+ previews.
195
315
 
196
316
  Shrine provides on-the-fly processing functionality via the
197
- [`derivation_endpoint`][derivation_endpoint] plugin. The basic setup is the
198
- following:
199
-
200
- 1. load the plugin with a secret key and a path prefix for the endpoint
201
- 2. mount the endpoint into your main app's router
202
- 3. define a processing block for the type files you want to generate
203
-
204
- Together it might look something like this:
317
+ **[`derivation_endpoint`][derivation_endpoint]** plugin. You set it up by
318
+ loading the plugin with a secret key (you generate this yourself, maybe via
319
+ something like `SecureRandom.hex`) and a path prefix, mount its Rack app in
320
+ your routes on the configured path prefix, and define processing you want to
321
+ perform:
205
322
 
323
+ ```rb
324
+ # config/initializers/shrine.rb (Rails)
325
+ # ...
326
+ Shrine.plugin :derivation_endpoint, secret_key: "<SHRINE_SECRET_KEY>"
327
+ ```
206
328
  ```rb
207
329
  require "image_processing/mini_magick"
208
330
 
209
331
  class ImageUploader < Shrine
210
- plugin :derivation_endpoint,
211
- secret_key: "<YOUR SECRET KEY>",
212
- prefix: "derivations/image"
332
+ plugin :derivation_endpoint, prefix: "derivations/image" # matches mount point
213
333
 
214
334
  derivation :thumbnail do |file, width, height|
215
335
  ImageProcessing::MiniMagick
@@ -218,11 +338,11 @@ class ImageUploader < Shrine
218
338
  end
219
339
  end
220
340
  ```
221
-
222
341
  ```rb
223
342
  # config/routes.rb (Rails)
224
343
  Rails.application.routes.draw do
225
- mount ImageUploader.derivation_endpoint => "derivations/image"
344
+ # ...
345
+ mount ImageUploader.derivation_endpoint => "/derivations/image"
226
346
  end
227
347
  ```
228
348
 
@@ -230,7 +350,7 @@ Now you can generate thumbnail URLs from attached files, and the actual
230
350
  thumbnail will be generated when the URL is requested:
231
351
 
232
352
  ```rb
233
- photo.image.derivation_url(:thumbnail, "600", "400")
353
+ photo.image.derivation_url(:thumbnail, 600, 400)
234
354
  #=> "/derivations/image/thumbnail/600/400/eyJpZCI6ImZvbyIsInN0b3JhZ2UiOiJzdG9yZSJ9?signature=..."
235
355
  ```
236
356
 
@@ -238,8 +358,133 @@ The plugin is highly customizable, be sure to check out the
238
358
  [documentation][derivation_endpoint], especially the [performance
239
359
  section][derivation_endpoint performance].
240
360
 
361
+ ### Dynamic derivation
362
+
363
+ If you have multiple types of transformations and don't want to have a
364
+ derivation for each one, you can set up a single derivation that applies any
365
+ series of transformations:
366
+
367
+ ```rb
368
+ class ImageUploader < Shrine
369
+ derivation :transform do |original, transformations|
370
+ transformations = Shrine.urlsafe_deserialize(transformations)
371
+
372
+ vips = ImageProcessing::Vips.source(original)
373
+ vips.apply!(transformations)
374
+ end
375
+ end
376
+ ```
377
+ ```rb
378
+ photo.image.derivation_url :transform, Shrine.urlsafe_serialize(
379
+ crop: [10, 10, 500, 500],
380
+ resize_to_fit: [300, 300],
381
+ gaussblur: 1,
382
+ )
383
+ ```
384
+
385
+ You can create a helper method for convenience:
386
+
387
+ ```rb
388
+ def derivation_url(file, transformations)
389
+ file.derivation_url(:transform, Shrine.urlsafe_serialize(transformations))
390
+ end
391
+ ```
392
+ ```rb
393
+ derivation_url photo.image,
394
+ crop: [10, 10, 500, 500],
395
+ resize_to_fit: [300, 300],
396
+ gaussblur: 1
397
+ ```
398
+
399
+ ## Processing other filetypes
400
+
401
+ So far we've only been talking about processing images. However, there is
402
+ nothing image-specific in Shrine's processing API, you can just as well process
403
+ any other types of files. The processing tool doesn't need to have any special
404
+ Shrine integration, the ImageProcessing gem that we saw earlier is a completely
405
+ generic gem.
406
+
407
+ To demonstrate, here is an example of transcoding videos using
408
+ [streamio-ffmpeg]:
409
+
410
+ ```rb
411
+ # Gemfile
412
+ gem "streamio-ffmpeg"
413
+ ```
414
+ ```rb
415
+ require "streamio-ffmpeg"
416
+
417
+ class VideoUploader < Shrine
418
+ Attacher.derivatives do |original|
419
+ transcoded = Tempfile.new ["transcoded", ".mp4"]
420
+ screenshot = Tempfile.new ["screenshot", ".jpg"]
421
+
422
+ movie = FFMPEG::Movie.new(original.path)
423
+ movie.transcode(transcoded.path)
424
+ movie.screenshot(screenshot.path)
425
+
426
+ { transcoded: transcoded, screenshot: screenshot }
427
+ end
428
+ end
429
+ ```
430
+
431
+ ### Polymorphic uploader
432
+
433
+ Sometimes you might want an attachment attribute to accept multiple types of
434
+ files, and apply different processing depending on the type. Since Shrine's
435
+ processing blocks are evaluated dynamically, you can use conditional logic:
436
+
437
+ ```rb
438
+ class PolymorphicUploader < Shrine
439
+ IMAGE_TYPES = %w[image/jpeg image/png image/webp]
440
+ VIDEO_TYPES = %w[video/mp4 video/quicktime]
441
+ PDF_TYPES = %w[application/pdf]
442
+
443
+ Attacher.validate do
444
+ validate_mime_type IMAGE_TYPES + VIDEO_TYPES + PDF_TYPES
445
+ # ...
446
+ end
447
+
448
+ Attacher.derivatives do |original|
449
+ case file.mime_type
450
+ when *IMAGE_TYPES then process_derivatives(:image, original)
451
+ when *VIDEO_TYPES then process_derivatives(:video, original)
452
+ when *PDF_TYPES then process_derivatives(:pdf, original)
453
+ end
454
+ end
455
+
456
+ Attacher.derivatives :image do |original|
457
+ # ...
458
+ end
459
+
460
+ Attacher.derivatives :video do |original|
461
+ # ...
462
+ end
463
+
464
+ Attacher.derivatives :pdf do |original|
465
+ # ...
466
+ end
467
+ end
468
+ ```
469
+
241
470
  ## Extras
242
471
 
472
+ ### Automatic derivatives
473
+
474
+ If you would like derivatives to be automatically created with promotion, you
475
+ can use the `create_on_promote` option built-in to the derivatives plugin.
476
+
477
+ ```rb
478
+ class Shrine::Attacher
479
+ plugin :derivatives, create_on_promote: true
480
+ end
481
+ ```
482
+
483
+ This shouldn't be needed if you're processing in the
484
+ [background](#backgrounding), as in that case you have a background worker that
485
+ will be called for each attachment, so you can call
486
+ `Attacher#create_derivatives` there.
487
+
243
488
  ### libvips
244
489
 
245
490
  As mentioned, ImageProcessing gem also has an alternative backend for
@@ -249,15 +494,15 @@ characteristics – it's often **multiple times faster** than ImageMagick and ha
249
494
  low memory usage (see [Why is libvips quick]).
250
495
 
251
496
  Using libvips is as easy as installing it and switching to the
252
- `ImageProcessing::Vips` backend:
497
+ [`ImageProcessing::Vips`][ImageProcessing::Vips] backend:
253
498
 
254
- ```sh
499
+ ```
255
500
  $ brew install vips
256
501
  ```
257
502
 
258
503
  ```rb
259
504
  # Gemfile
260
- gem "image_processing", "~> 1.0"
505
+ gem "image_processing", "~> 1.8"
261
506
  ```
262
507
 
263
508
  ```rb
@@ -271,113 +516,85 @@ thumbnail = ImageProcessing::Vips
271
516
  thumbnail #=> #<Tempfile:...> (a 600x400 thumbnail of the source image)
272
517
  ```
273
518
 
274
- ### Optimizing thumbnails
519
+ ### Parallelize uploads
275
520
 
276
- If you're generating image thumbnails, you can additionally use the
277
- [image_optim] gem to further reduce their filesize:
521
+ If you're generating derivatives, you can parallelize the uploads using the
522
+ [concurrent-ruby] gem:
278
523
 
279
524
  ```rb
280
525
  # Gemfile
281
- gem "image_processing", "~> 1.0"
282
- gem "image_optim"
283
- gem "image_optim_pack" # precompiled binaries
526
+ gem "concurrent-ruby"
284
527
  ```
285
-
286
528
  ```rb
287
- require "image_processing/mini_magick"
529
+ require "concurrent"
288
530
 
289
- thumbnail = ImageProcessing::MiniMagick
290
- .source(image)
291
- .resize_to_limit!(600, 400)
531
+ derivatives = attacher.process_derivatives
292
532
 
293
- image_optim = ImageOptim.new
294
- image_optim.optimize_image!(thumbnail.path)
533
+ tasks = derivatives.map do |name, file|
534
+ Concurrent::Promises.future(name, file) do |name, file|
535
+ attacher.add_derivative(name, file)
536
+ end
537
+ end
295
538
 
296
- thumbnail.open # refresh file descriptor
297
- thumbnail
539
+ Concurrent::Promises.zip(*tasks).wait!
298
540
  ```
299
541
 
300
542
  ### External processing
301
543
 
302
- Since processing is so dynamic, you're not limited to using the ImageProcessing
303
- gem, you can also use a 3rd-party service to generate thumbnails for you. Here
304
- is an example of generating thumbnails on-the-fly using [ImageOptim.com] (not
305
- to be confused with the [image_optim] gem):
544
+ You can also integrate Shrine with 3rd-party processing services such as
545
+ [Cloudinary] and [Imgix]. In the most common case, you'd serve images directly
546
+ from these services, see the corresponding plugin docs for more details
547
+ ([shrine-cloudinary], [shrine-imgix] and [others][external storages])
548
+
549
+ You can also choose to use these services as an implementation detail of your
550
+ application, by downloading the processed images and saving them to your
551
+ storage. Here is how you might store files processed by Imgix as derivatives:
306
552
 
307
553
  ```rb
308
554
  # Gemfile
309
- gem "down", "~> 4.4"
555
+ gem "down", "~> 5.0"
310
556
  gem "http", "~> 4.0"
557
+ gem "shrine-imgix", "~> 0.5"
558
+ ```
559
+ ```rb
560
+ Shrine.plugin :derivatives
561
+ Shrine.plugin :imgix, client: { host: "my-app.imgix.net", secure_url_token: "secret" }
311
562
  ```
312
-
313
563
  ```rb
314
564
  require "down/http"
315
565
 
316
566
  class ImageUploader < Shrine
317
- plugin :derivation_endpoint,
318
- secret_key: "secret",
319
- prefix: "derivations/image",
320
- download: false
321
-
322
- derivation :thumbnail do |source, width, height|
323
- # generate thumbnails using ImageOptim.com
324
- down = Down::Http.new(method: :post)
325
- down.download("https://im2.io/<USERNAME>/#{width}x#{height}/#{source.url}")
567
+ IMGIX_THUMBNAIL = -> (file, width, height) do
568
+ Down::Http.download(file.imgix_url(w: width, h: height))
326
569
  end
327
- end
328
- ```
329
-
330
- ### Cloudinary
331
-
332
- [Cloudinary] is a popular commercial service for on-the-fly image processing,
333
- so it's a good alternative to the `derivation_endpoint` plugin. The
334
- [shrine-cloudinary] gem provides a Shrine storage that we can set for our
335
- temporary and permanent storage:
336
-
337
- ```rb
338
- # Gemfile
339
- gem "shrine-cloudinary"
340
- ```
341
-
342
- ```rb
343
- require "cloudinary"
344
- require "shrine/storage/cloudinary"
345
-
346
- Cloudinary.config(
347
- cloud_name: "<YOUR_CLOUD_NAME>",
348
- api_key: "<YOUR_API_KEY>",
349
- api_secret: "<YOUR_API_SECRET>",
350
- )
351
-
352
- Shrine.storages = {
353
- cache: Shrine::Storage::Cloudinary.new(prefix: "cache"),
354
- store: Shrine::Storage::Cloudinary.new,
355
- }
356
- ```
357
570
 
358
- Now when we upload our images to Cloudinary, we can generate URLs with various
359
- processing parameters:
360
-
361
- ```rb
362
- photo.image.url(width: 100, height: 100, crop: :fit)
363
- #=> "http://res.cloudinary.com/myapp/image/upload/w_100,h_100,c_fit/nature.jpg"
571
+ Attacher.derivatives do
572
+ {
573
+ large: IMGIX_THUMBNAIL[file, 800, 800],
574
+ medium: IMGIX_THUMBNAIL[file, 500, 500],
575
+ small: IMGIX_THUMBNAIL[file, 300, 300],
576
+ }
577
+ end
578
+ end
364
579
  ```
365
580
 
366
581
  [`Shrine::UploadedFile`]: http://shrinerb.com/rdoc/classes/Shrine/UploadedFile/InstanceMethods.html
367
582
  [ImageProcessing]: https://github.com/janko/image_processing
368
- [ImageMagick]: https://www.imagemagick.org
369
- [GraphicsMagick]: http://www.graphicsmagick.org
370
- [libvips]: http://libvips.github.io/libvips/
371
- [Why is libvips quick]: https://github.com/libvips/libvips/wiki/Why-is-libvips-quick
372
- [image_optim]: https://github.com/toy/image_optim
373
- [ImageOptim.com]: https://imageoptim.com/api
583
+ [ImageProcessing::MiniMagick]: https://github.com/janko/image_processing/blob/master/doc/minimagick.md#readme
584
+ [ImageProcessing::Vips]: https://github.com/janko/image_processing/blob/master/doc/vips.md#readme
374
585
  [streamio-ffmpeg]: https://github.com/streamio/streamio-ffmpeg
375
- [Reprocessing Versions]: /doc/regenerating_versions.md#readme
376
- [Cloudinary]: https://cloudinary.com
586
+ [Managing Derivatives]: https://shrinerb.com/docs/changing-derivatives
587
+ [Cloudinary]: https://cloudinary.com/
588
+ [Imgix]: https://www.imgix.com/
377
589
  [shrine-cloudinary]: https://github.com/shrinerb/shrine-cloudinary
378
- [backgrounding]: /doc/plugins/backgrounding.md#readme
379
- [validation]: /doc/validation.md#readme
380
- [ruby-vips]: https://github.com/libvips/ruby-vips
381
- [MiniMagick]: https://github.com/minimagick/minimagick
382
- [derivation_endpoint]: /doc/plugins/derivation_endpoint.md#readme
383
- [derivation_endpoint performance]: /doc/plugins/derivation_endpoint.md#performance
590
+ [shrine-imgix]: https://github.com/shrinerb/shrine-imgix
591
+ [backgrounding]: https://shrinerb.com/docs/plugins/backgrounding
592
+ [derivation_endpoint]: https://shrinerb.com/docs/plugins/derivation_endpoint
593
+ [derivation_endpoint performance]: https://shrinerb.com/docs/plugins/derivation_endpoint#performance
594
+ [derivatives]: https://shrinerb.com/docs/plugins/derivatives
595
+ [concurrent-ruby]: https://github.com/ruby-concurrency/concurrent-ruby
596
+ [default_url]: https://shrinerb.com/docs/plugins/default_url
597
+ [external storages]: https://shrinerb.com/docs/external/extensions#storages
598
+ [libvips]: https://libvips.github.io/libvips/
599
+ [Why is libvips quick]: https://github.com/libvips/libvips/wiki/Why-is-libvips-quick
600
+ [type_predicates]: https://shrinerb.com/docs/plugins/type_predicates