shrine 2.19.4 → 3.4.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (209) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +485 -43
  3. data/LICENSE.txt +1 -1
  4. data/README.md +81 -977
  5. data/doc/advantages.md +231 -204
  6. data/doc/attacher.md +304 -153
  7. data/doc/carrierwave.md +297 -226
  8. data/doc/changing_derivatives.md +308 -0
  9. data/doc/changing_location.md +102 -21
  10. data/doc/changing_storage.md +110 -0
  11. data/doc/creating_persistence_plugins.md +132 -0
  12. data/doc/creating_plugins.md +43 -23
  13. data/doc/creating_storages.md +19 -5
  14. data/doc/design.md +147 -97
  15. data/doc/direct_s3.md +38 -28
  16. data/doc/external/articles.md +63 -0
  17. data/doc/external/extensions.md +53 -0
  18. data/doc/external/misc.md +32 -0
  19. data/doc/getting_started.md +1115 -0
  20. data/doc/metadata.md +190 -109
  21. data/doc/multiple_files.md +62 -34
  22. data/doc/paperclip.md +384 -262
  23. data/doc/plugins/activerecord.md +177 -46
  24. data/doc/plugins/add_metadata.md +139 -38
  25. data/doc/plugins/atomic_helpers.md +217 -0
  26. data/doc/plugins/backgrounding.md +156 -98
  27. data/doc/plugins/cached_attachment_data.md +7 -5
  28. data/doc/plugins/column.md +121 -0
  29. data/doc/plugins/data_uri.md +23 -22
  30. data/doc/plugins/default_storage.md +36 -10
  31. data/doc/plugins/default_url.md +30 -13
  32. data/doc/plugins/delete_raw.md +4 -2
  33. data/doc/plugins/derivation_endpoint.md +162 -101
  34. data/doc/plugins/derivatives.md +829 -0
  35. data/doc/plugins/determine_mime_type.md +4 -2
  36. data/doc/plugins/download_endpoint.md +64 -8
  37. data/doc/plugins/dynamic_storage.md +5 -3
  38. data/doc/plugins/entity.md +263 -0
  39. data/doc/plugins/form_assign.md +55 -0
  40. data/doc/plugins/included.md +31 -8
  41. data/doc/plugins/infer_extension.md +21 -10
  42. data/doc/plugins/instrumentation.md +38 -16
  43. data/doc/plugins/keep_files.md +14 -17
  44. data/doc/plugins/metadata_attributes.md +42 -13
  45. data/doc/plugins/mirroring.md +118 -0
  46. data/doc/plugins/model.md +210 -0
  47. data/doc/plugins/module_include.md +4 -2
  48. data/doc/plugins/multi_cache.md +24 -0
  49. data/doc/plugins/persistence.md +101 -0
  50. data/doc/plugins/presign_endpoint.md +9 -4
  51. data/doc/plugins/pretty_location.md +16 -3
  52. data/doc/plugins/processing.md +4 -2
  53. data/doc/plugins/rack_file.md +8 -2
  54. data/doc/plugins/rack_response.md +6 -2
  55. data/doc/plugins/recache.md +4 -2
  56. data/doc/plugins/refresh_metadata.md +49 -9
  57. data/doc/plugins/remote_url.md +84 -47
  58. data/doc/plugins/remove_attachment.md +27 -6
  59. data/doc/plugins/remove_invalid.md +21 -6
  60. data/doc/plugins/restore_cached_data.md +11 -3
  61. data/doc/plugins/sequel.md +159 -35
  62. data/doc/plugins/signature.md +16 -5
  63. data/doc/plugins/store_dimensions.md +14 -2
  64. data/doc/plugins/tempfile.md +4 -2
  65. data/doc/plugins/type_predicates.md +96 -0
  66. data/doc/plugins/upload_endpoint.md +13 -13
  67. data/doc/plugins/upload_options.md +6 -4
  68. data/doc/plugins/{default_url_options.md → url_options.md} +9 -7
  69. data/doc/plugins/validation.md +97 -0
  70. data/doc/plugins/validation_helpers.md +16 -13
  71. data/doc/plugins/versions.md +15 -19
  72. data/doc/processing.md +438 -221
  73. data/doc/refile.md +185 -167
  74. data/doc/release_notes/1.0.0.md +4 -0
  75. data/doc/release_notes/1.1.0.md +6 -2
  76. data/doc/release_notes/1.2.0.md +4 -0
  77. data/doc/release_notes/1.3.0.md +4 -0
  78. data/doc/release_notes/1.4.0.md +4 -0
  79. data/doc/release_notes/1.4.1.md +4 -0
  80. data/doc/release_notes/1.4.2.md +4 -0
  81. data/doc/release_notes/2.0.0.md +4 -0
  82. data/doc/release_notes/2.0.1.md +4 -0
  83. data/doc/release_notes/2.1.0.md +4 -0
  84. data/doc/release_notes/2.1.1.md +4 -0
  85. data/doc/release_notes/2.10.0.md +4 -0
  86. data/doc/release_notes/2.10.1.md +4 -0
  87. data/doc/release_notes/2.11.0.md +4 -0
  88. data/doc/release_notes/2.12.0.md +4 -0
  89. data/doc/release_notes/2.13.0.md +4 -0
  90. data/doc/release_notes/2.14.0.md +5 -1
  91. data/doc/release_notes/2.15.0.md +11 -7
  92. data/doc/release_notes/2.16.0.md +4 -0
  93. data/doc/release_notes/2.17.0.md +4 -0
  94. data/doc/release_notes/2.18.0.md +4 -0
  95. data/doc/release_notes/2.19.0.md +6 -3
  96. data/doc/release_notes/2.2.0.md +4 -0
  97. data/doc/release_notes/2.3.0.md +4 -0
  98. data/doc/release_notes/2.3.1.md +4 -0
  99. data/doc/release_notes/2.4.0.md +4 -0
  100. data/doc/release_notes/2.4.1.md +4 -0
  101. data/doc/release_notes/2.5.0.md +4 -0
  102. data/doc/release_notes/2.6.0.md +4 -0
  103. data/doc/release_notes/2.6.1.md +4 -0
  104. data/doc/release_notes/2.7.0.md +4 -0
  105. data/doc/release_notes/2.8.0.md +4 -0
  106. data/doc/release_notes/2.9.0.md +4 -0
  107. data/doc/release_notes/3.0.0.md +981 -0
  108. data/doc/release_notes/3.0.1.md +22 -0
  109. data/doc/release_notes/3.1.0.md +73 -0
  110. data/doc/release_notes/3.2.0.md +96 -0
  111. data/doc/release_notes/3.2.1.md +31 -0
  112. data/doc/release_notes/3.2.2.md +14 -0
  113. data/doc/release_notes/3.3.0.md +105 -0
  114. data/doc/release_notes/3.4.0.md +35 -0
  115. data/doc/retrieving_uploads.md +4 -1
  116. data/doc/securing_uploads.md +60 -37
  117. data/doc/storage/file_system.md +20 -3
  118. data/doc/storage/memory.md +19 -0
  119. data/doc/storage/s3.md +117 -83
  120. data/doc/testing.md +124 -144
  121. data/doc/upgrading_to_3.md +710 -0
  122. data/doc/validation.md +54 -90
  123. data/lib/shrine/attacher.rb +287 -171
  124. data/lib/shrine/attachment.rb +13 -46
  125. data/lib/shrine/plugins/_persistence.rb +93 -0
  126. data/lib/shrine/plugins/activerecord.rb +77 -34
  127. data/lib/shrine/plugins/add_metadata.rb +25 -17
  128. data/lib/shrine/plugins/atomic_helpers.rb +119 -0
  129. data/lib/shrine/plugins/backgrounding.rb +77 -113
  130. data/lib/shrine/plugins/cached_attachment_data.rb +6 -15
  131. data/lib/shrine/plugins/column.rb +102 -0
  132. data/lib/shrine/plugins/data_uri.rb +38 -36
  133. data/lib/shrine/plugins/default_storage.rb +45 -15
  134. data/lib/shrine/plugins/default_url.rb +12 -24
  135. data/lib/shrine/plugins/default_url_options.rb +3 -30
  136. data/lib/shrine/plugins/delete_raw.rb +10 -16
  137. data/lib/shrine/plugins/derivation_endpoint.rb +89 -134
  138. data/lib/shrine/plugins/derivatives.rb +637 -0
  139. data/lib/shrine/plugins/determine_mime_type.rb +9 -21
  140. data/lib/shrine/plugins/download_endpoint.rb +109 -133
  141. data/lib/shrine/plugins/dynamic_storage.rb +5 -11
  142. data/lib/shrine/plugins/entity.rb +152 -0
  143. data/lib/shrine/plugins/form_assign.rb +108 -0
  144. data/lib/shrine/plugins/included.rb +6 -6
  145. data/lib/shrine/plugins/infer_extension.rb +13 -20
  146. data/lib/shrine/plugins/instrumentation.rb +54 -42
  147. data/lib/shrine/plugins/keep_files.rb +3 -15
  148. data/lib/shrine/plugins/metadata_attributes.rb +28 -19
  149. data/lib/shrine/plugins/mirroring.rb +142 -0
  150. data/lib/shrine/plugins/model.rb +158 -0
  151. data/lib/shrine/plugins/module_include.rb +3 -3
  152. data/lib/shrine/plugins/multi_cache.rb +27 -0
  153. data/lib/shrine/plugins/presign_endpoint.rb +18 -22
  154. data/lib/shrine/plugins/pretty_location.rb +15 -9
  155. data/lib/shrine/plugins/processing.rb +22 -9
  156. data/lib/shrine/plugins/rack_file.rb +2 -42
  157. data/lib/shrine/plugins/rack_response.rb +15 -10
  158. data/lib/shrine/plugins/recache.rb +6 -5
  159. data/lib/shrine/plugins/refresh_metadata.rb +13 -11
  160. data/lib/shrine/plugins/remote_url.rb +49 -49
  161. data/lib/shrine/plugins/remove_attachment.rb +10 -6
  162. data/lib/shrine/plugins/remove_invalid.rb +19 -8
  163. data/lib/shrine/plugins/restore_cached_data.rb +13 -7
  164. data/lib/shrine/plugins/sequel.rb +86 -36
  165. data/lib/shrine/plugins/signature.rb +10 -16
  166. data/lib/shrine/plugins/store_dimensions.rb +35 -40
  167. data/lib/shrine/plugins/tempfile.rb +1 -3
  168. data/lib/shrine/plugins/type_predicates.rb +113 -0
  169. data/lib/shrine/plugins/upload_endpoint.rb +25 -23
  170. data/lib/shrine/plugins/upload_options.rb +14 -15
  171. data/lib/shrine/plugins/url_options.rb +31 -0
  172. data/lib/shrine/plugins/validation.rb +80 -0
  173. data/lib/shrine/plugins/validation_helpers.rb +34 -57
  174. data/lib/shrine/plugins/versions.rb +107 -87
  175. data/lib/shrine/plugins.rb +22 -0
  176. data/lib/shrine/storage/file_system.rb +46 -64
  177. data/lib/shrine/storage/linter.rb +42 -7
  178. data/lib/shrine/storage/memory.rb +49 -0
  179. data/lib/shrine/storage/s3.rb +154 -158
  180. data/lib/shrine/uploaded_file.rb +28 -30
  181. data/lib/shrine/version.rb +3 -3
  182. data/lib/shrine.rb +86 -149
  183. data/shrine.gemspec +9 -10
  184. metadata +79 -83
  185. data/doc/migrating_storage.md +0 -76
  186. data/doc/plugins/backup.md +0 -31
  187. data/doc/plugins/copy.md +0 -24
  188. data/doc/plugins/delete_promoted.md +0 -12
  189. data/doc/plugins/direct_upload.md +0 -172
  190. data/doc/plugins/hooks.md +0 -58
  191. data/doc/plugins/logging.md +0 -42
  192. data/doc/plugins/migration_helpers.md +0 -60
  193. data/doc/plugins/moving.md +0 -19
  194. data/doc/plugins/multi_delete.md +0 -20
  195. data/doc/plugins/parallelize.md +0 -16
  196. data/doc/plugins/parsed_json.md +0 -23
  197. data/doc/regenerating_versions.md +0 -143
  198. data/lib/shrine/plugins/background_helpers.rb +0 -5
  199. data/lib/shrine/plugins/backup.rb +0 -90
  200. data/lib/shrine/plugins/copy.rb +0 -50
  201. data/lib/shrine/plugins/delete_promoted.rb +0 -20
  202. data/lib/shrine/plugins/direct_upload.rb +0 -217
  203. data/lib/shrine/plugins/hooks.rb +0 -90
  204. data/lib/shrine/plugins/logging.rb +0 -142
  205. data/lib/shrine/plugins/migration_helpers.rb +0 -70
  206. data/lib/shrine/plugins/moving.rb +0 -57
  207. data/lib/shrine/plugins/multi_delete.rb +0 -32
  208. data/lib/shrine/plugins/parallelize.rb +0 -78
  209. data/lib/shrine/plugins/parsed_json.rb +0 -29
data/doc/paperclip.md CHANGED
@@ -1,15 +1,60 @@
1
- # Shrine for Paperclip Users
1
+ ---
2
+ title: Upgrading from Paperclip
3
+ ---
2
4
 
3
5
  This guide is aimed at helping Paperclip users transition to Shrine, and it
4
6
  consists of three parts:
5
7
 
6
8
  1. Explanation of the key differences in design between Paperclip and Shrine
7
- 2. Instructions how to migrate and existing app that uses Paperclip to Shrine
9
+ 2. Instructions how to migrate an existing app that uses Paperclip to Shrine
8
10
  3. Extensive reference of Paperclip's interface with Shrine equivalents
9
11
 
10
- ## Storages
12
+ ## Overview
11
13
 
12
- In Paperclip the storage is configure inside the global options:
14
+ ### Uploader
15
+
16
+ In Paperclip, the attachment logic is configured directly inside Active Record
17
+ models:
18
+
19
+ ```rb
20
+ class Photo < ActiveRecord::Base
21
+ has_attached_file :image,
22
+ preserve_files: true,
23
+ default_url: "/images/:style/missing.png"
24
+
25
+ validated_attachment_content_type :image, content_type: "image/jpeg"
26
+ end
27
+ ```
28
+
29
+ Shrine takes a more object-oriented approach, by encapsulating attachment logic
30
+ in "uploader" classes:
31
+
32
+ ```rb
33
+ class ImageUploader < Shrine
34
+ plugin :keep_files
35
+ plugin :default_url
36
+ plugin :validation_helpers
37
+
38
+ Attacher.default_url do |derivative: nil, **|
39
+ "/images/#{derivative}/missing.png" if derivative
40
+ end
41
+
42
+ Attacher.validate do
43
+ validate_mime_type %w[image/jpeg]
44
+ end
45
+ end
46
+ ```
47
+ ```rb
48
+ class Photo < ActiveRecord::Base
49
+ include ImageUploader::Attachment(:image)
50
+ end
51
+ ```
52
+
53
+ ### Storage
54
+
55
+ Paperclip storage is configured together with other attachment options. Also,
56
+ the storage implementations themselves are mixed into the attachment class,
57
+ which couples them to the attachment flow.
13
58
 
14
59
  ```rb
15
60
  class Photo < ActiveRecord::Base
@@ -24,7 +69,8 @@ class Photo < ActiveRecord::Base
24
69
  end
25
70
  ```
26
71
 
27
- In contrast, a Shrine storage is just a class which you configure individually:
72
+ Shrine storage objects are configured separately and are decoupled from
73
+ attachment:
28
74
 
29
75
  ```rb
30
76
  Shrine.storages[:store] = Shrine::Storage::S3.new(
@@ -35,10 +81,8 @@ Shrine.storages[:store] = Shrine::Storage::S3.new(
35
81
  )
36
82
  ```
37
83
 
38
- Paperclip doesn't have a concept of "temporary" storage, so it cannot retain
39
- uploaded files in case of validation errors, and [direct S3 uploads] cannot be
40
- implemented in a safe way. Shrine uses separate "temporary" and "permanent"
41
- storage for attaching files:
84
+ Shrine also has a concept of "temporary" storage, which enables retaining
85
+ uploaded files in case of validation errors and [direct uploads].
42
86
 
43
87
  ```rb
44
88
  Shrine.storages = {
@@ -47,42 +91,79 @@ Shrine.storages = {
47
91
  }
48
92
  ```
49
93
 
50
- ## Uploaders
94
+ ### Persistence
95
+
96
+ When using Paperclip, the attached file data will be persisted into several
97
+ columns:
98
+
99
+ * `<name>_file_name`
100
+ * `<name>_content_type`
101
+ * `<name>_file_size`
102
+ * `<name>_updated_at`
103
+ * `<name>_created_at` (optional)
104
+ * `<name>_fingerprint` (optional)
51
105
 
52
- While in Paperclip you define all your uploading logic inside your models,
53
- Shrine takes a more object-oriented approach and lets you define uploading logic
54
- inside "uploader" classes:
106
+ In contrast, Shrine uses a single `<name>_data` column to store data in JSON
107
+ format:
55
108
 
56
109
  ```rb
57
- class Photo < ActiveRecord::Base
58
- has_attached_file :image
59
- end
110
+ {
111
+ "id": "path/to/image.jpg",
112
+ "storage": "store",
113
+ "metadata": {
114
+ "filename": "nature.jpg",
115
+ "size": 4739472,
116
+ "mime_type": "image/jpeg"
117
+ }
118
+ }
60
119
  ```
61
-
62
120
  ```rb
63
- class ImageUploader < Shrine
64
- # ...
65
- end
121
+ photo.image.id #=> "path/to/image.jpg"
122
+ photo.image.storage_key #=> :store
123
+ photo.image.metadata #=> { "filename" => "...", "size" => ..., "mime_type" => "..." }
66
124
 
67
- class Photo < ActiveRecord::Base
68
- include ImageUploader::Attachment.new(:image)
69
- end
125
+ photo.image.original_filename #=> "nature.jpg"
126
+ photo.image.size #=> 4739472
127
+ photo.image.mime_type #=> "image/jpeg"
70
128
  ```
71
129
 
72
- Among other things, this allows you to use uploader classes standalone, which
73
- gives you more power:
130
+ This column can be queried if it's made a JSON column. Alternatively, you can
131
+ use the [`metadata_attributes`][metadata_attributes] plugin to save metadata
132
+ into separate columns.
133
+
134
+ #### ORM
135
+
136
+ While Paperclip works only with Active Record, Shrine is designed to integrate
137
+ with any persistence library (there are integrations for [Active
138
+ Record][activerecord], [Sequel][sequel], [ROM][rom], [Hanami][hanami] and
139
+ [Mongoid][mongoid]), and can also be used standalone:
74
140
 
75
141
  ```rb
76
- uploaded_file = ImageUploader.upload(File.open("nature.jpg"), :store)
77
- uploaded_file #=> #<Shrine::UploadedFile>
78
- uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/store/kfds0lg9rer.jpg"
142
+ attacher = ImageUploader::Attacher.new
143
+ attacher.attach File.open("nature.jpg")
144
+ attacher.file #=> #<Shrine::UploadedFile id="f4ba5bdbf366ef0b.jpg" ...>
145
+ attacher.url #=> "https://my-bucket.s3.amazonaws.com/f4ba5bdbf366ef0b.jpg"
146
+ attacher.data #=> { "id" => "f4ba5bdbf366ef0b.jpg", "storage" => "store", "metadata" => { ... } }
79
147
  ```
80
148
 
149
+ #### Location
150
+
151
+ Paperclip persists only the filename of the uploaded file, and recalculates the
152
+ full location dynamically based on location configuration. This can be
153
+ dangerous, because if some component of the location happens to change, all
154
+ existing links might become invalid.
155
+
156
+ To avoid this, Shrine persists the full location on attachment, and uses it
157
+ when generating file URL. So, even if you change how file locations are
158
+ generated, existing files that are on old locations will still remain
159
+ accessible.
160
+
81
161
  ### Processing
82
162
 
83
- In contrast to Paperclip's static options, in Shrine you define and perform
84
- processing on instance-level. The result of processing can be a single file
85
- or a hash of versions:
163
+ In Shrine, processing is defined and performed on the instance level, which
164
+ gives you more control. You're also not coupled to ImageMagick, e.g. you can
165
+ use [libvips] instead (both integrations are provided by the [image_processing]
166
+ gem).
86
167
 
87
168
  ```rb
88
169
  class Photo < ActiveRecord::Base
@@ -99,47 +180,65 @@ end
99
180
  require "image_processing/mini_magick"
100
181
 
101
182
  class ImageUploader < Shrine
102
- plugin :processing
103
- plugin :versions
104
-
105
- process(:store) do |io, context|
106
- versions = { original: io } # retain original
183
+ plugin :derivatives
107
184
 
108
- io.download do |original|
109
- pipeline = ImageProcessing::MiniMagick.source(original)
185
+ Attacher.derivatives do |original|
186
+ magick = ImageProcessing::MiniMagick.source(original)
110
187
 
111
- versions[:large] = pipeline.resize_to_limit!(800, 800)
112
- versions[:medium] = pipeline.resize_to_limit!(500, 500)
113
- versions[:small] = pipeline.resize_to_limit!(300, 300)
114
- end
115
-
116
- versions # return the hash of processed files
188
+ {
189
+ large: magick.resize_to_limit!(800, 800),
190
+ medium: magick.resize_to_limit!(500, 500),
191
+ small: magick.resize_to_limit!(300, 300),
192
+ }
117
193
  end
118
194
  end
119
195
  ```
120
196
 
121
- This allows you to fully optimize processing, because you can easily specify
122
- which files are processed from which, and even add parallelization.
197
+ Shrine is agnostic as to how you're performing your processing, so you can
198
+ easily use any other processing tools. You can also combine different
199
+ processors for different versions.
200
+
201
+ #### Retrieving versions
202
+
203
+ When retrieving versions, Paperclip returns a list of declared styles which
204
+ may or may not have been generated. In contrast, Shrine persists data of
205
+ uploaded processed files into the database (including any extracted metadata),
206
+ which then becomes the source of truth on which versions have been generated.
207
+
208
+ ```rb
209
+ photo.image #=> #<Shrine::UploadedFile id="original.jpg" ...>
210
+ photo.image_derivatives #=> {}
211
+
212
+ photo.image_derivatives! # triggers processing
213
+ photo.image_derivatives #=>
214
+ # {
215
+ # large: #<Shrine::UploadedFile id="large.jpg" metadata={"size"=>873232, ...} ...>,
216
+ # medium: #<Shrine::UploadedFile id="medium.jpg" metadata={"size"=>94823, ...} ...>,
217
+ # small: #<Shrine::UploadedFile id="small.jpg" metadata={"size"=>37322, ...} ...>,
218
+ # }
219
+ ```
123
220
 
124
221
  #### Reprocessing versions
125
222
 
126
223
  Shrine doesn't have a built-in way of regenerating versions, because that has
127
- to be written and optimized differently depending on whether you're adding or
128
- removing a version, what ORM are you using, how many records there are in the
129
- database etc. The [Reprocessing versions] guide provides some useful tips on
130
- this task.
224
+ to be written and optimized differently depending on what versions have changed
225
+ which persistence library you're using, how many records there are in the table
226
+ etc.
227
+
228
+ However, there is an extensive guide for [Managing Derivatives], which provides
229
+ instructions on how to make these changes safely and with zero downtime.
131
230
 
132
- ### Validations
231
+ ### Validation
133
232
 
134
- Validations are also defined inside the uploader on the instance-level, which
135
- allows you to do conditional validations:
233
+ File validation in Shrine is also instance-level, which allows using
234
+ conditionals:
136
235
 
137
236
  ```rb
138
237
  class Photo < ActiveRecord::Base
139
238
  has_attached_file :image
140
239
  validates_attachment :image,
141
- content_type: {content_type: %w[image/jpeg image/png image/gif]},
142
- size: {in: 0..10.megabytes}
240
+ size: { in: 0..10.megabytes },
241
+ content_type: { content_type: %w[image/jpeg image/png image/webp] }
143
242
  end
144
243
  ```
145
244
 
@@ -148,151 +247,83 @@ class ImageUploader < Shrine
148
247
  plugin :validation_helpers
149
248
 
150
249
  Attacher.validate do
151
- validate_mime_type_inclusion %w[image/jpeg image/gif image/png]
152
- validate_max_size 10*1024*1024 unless record.admin?
250
+ validate_max_size 10*1024*1024
251
+
252
+ if validate_mime_type %w[image/jpeg image/png image/webp]
253
+ validate_max_dimensions [5000, 5000]
254
+ end
153
255
  end
154
256
  end
155
257
  ```
156
258
 
157
- #### MIME type spoofing
158
-
159
- Paperclip detects MIME type spoofing, in the way that it extracts the MIME type
160
- from file contents using the `file` command and MimeMagic, compares it to the
161
- value that the `mime-types` gem determined from file extension, and raises a
162
- validation error if these two values mismatch.
163
-
164
- However, this turned out to be very problematic, leading to a lot of valid
165
- files being classified as "spoofed", because of the differences of MIME
166
- type databases between the `mime-types` gem, `file` command, and MimeMagic.
259
+ #### Custom metadata
167
260
 
168
- Shrine takes a different approach here. By default it will extract MIME
169
- type from file extension, but it has a plugin for determining MIME type from
170
- file contents, which by default uses the `file` command:
261
+ With Shrine you can also extract and validate any custom metadata:
171
262
 
172
263
  ```rb
173
- Shrine.plugin :determine_mime_type
174
- ```
264
+ class VideoUploader < Shrine
265
+ plugin :add_metadata
266
+ plugin :validation
175
267
 
176
- However, it doesn't try to compare this value with the one from file extension,
177
- it just means that now this value will be used for your MIME type validations.
178
- With this approach you can still prevent malicious files from being attached,
179
- but without the possibility of false negatives.
180
-
181
- ### Logging
182
-
183
- In Paperclip you enable logging by setting `Paperclip.options[:log] = true`,
184
- however, this only logs ImageMagick commands. Shrine has full logging support,
185
- which measures processing, uploading and deleting individually:
268
+ add_metadata :duration do |io|
269
+ FFMPEG::Movie.new(io.path).duration
270
+ end
186
271
 
187
- ```rb
188
- Shrine.plugin :instrumentation
189
- ```
190
- ```
191
- Metadata (32ms) – {:storage=>:store, :io=>StringIO, :uploader=>Shrine}
192
- Upload (1523ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :io=>StringIO, :upload_options=>{}, :uploader=>Shrine}
193
- Exists (755ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
194
- Download (1002ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :download_options=>{}, :uploader=>Shrine}
195
- Delete (700ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
272
+ Attacher.validate do
273
+ if file.duration > 5*60*60
274
+ errors << "must not be longer than 5 hours"
275
+ end
276
+ end
277
+ end
196
278
  ```
197
279
 
198
- ## Attachments
199
-
200
- While Paperclip is designed to only integrate with ActiveRecord, Shrine is
201
- designed to be completely generic and integrate with any ORM. It ships with
202
- plugins for ActiveRecord and Sequel:
280
+ #### MIME type spoofing
203
281
 
204
- ```rb
205
- Shrine.plugin :activerecord # if you're using ActiveRecord
206
- Shrine.plugin :sequel # if you're using Sequel
207
- ```
282
+ Paperclip attempts to detect MIME type spoofing, which turned out to be
283
+ unreliable due to differences in MIME type databases between different ruby
284
+ libraries.
208
285
 
209
- Instead of giving you class methods for defining attachments, in Shrine you
210
- generate attachment modules which you simply include in your models, which
211
- gives your models similar set of methods that Paperclip gives:
286
+ Shrine on the other hand simply allows you to determine MIME type from file
287
+ content, which you can then validate.
212
288
 
213
289
  ```rb
214
- class Photo < Sequel::Model
215
- include ImageUploader::Attachment.new(:image)
216
- end
290
+ Shrine.plugin :determine_mime_type, analyzer: :marcel
217
291
  ```
218
-
219
- ### Attachment column
220
-
221
- Unlike in Paperclip which requires you to have 4 `<attachment>_*` columns, in
222
- Shrine you only need to have a single `<attachment>_data` text column (in the
223
- above case `image_data`), and all information will be stored there.
224
-
225
292
  ```rb
226
- photo.image_data #=>
227
- # {
228
- # "storage" => "store",
229
- # "id" => "photo/1/image/0d9o8dk42.png",
230
- # "metadata" => {
231
- # "filename" => "nature.png",
232
- # "size" => 49349138,
233
- # "mime_type" => "image/png"
234
- # }
235
- # }
236
-
237
- photo.image.original_filename #=> "nature.png"
238
- photo.image.size #=> 49349138
239
- photo.image.mime_type #=> "image/png"
293
+ file = uploader.upload StringIO.new("<?php ... ?>")
294
+ file.mime_type #=> "application/x-php"
240
295
  ```
241
296
 
242
- Unlike Paperclip, Shrine will store this information for each processed
243
- version, making them first-class citizens:
244
-
245
- ```rb
246
- photo.image[:original] #=> #<Shrine::UploadedFile>
247
- photo.image[:original].width #=> 800
248
-
249
- photo.image[:thumb] #=> #<Shrine::UploadedFile>
250
- photo.image[:thumb].width #=> 300
251
- ```
297
+ ## Migrating from Paperclip
252
298
 
253
- Also, since Paperclip stores only the filename, it has to recalculate the full
254
- location each time it wants to generate the URL. That makes it really difficult
255
- to move files to a new location, because changing how the location is generated
256
- will now cause incorrect URLs to be generated for all existing files. Shrine
257
- calculates the whole location only once and saves it to the column.
299
+ You have an existing app using Paperclip and you want to transfer it to Shrine.
300
+ Let's assume we have a `Photo` model with the "image" attachment.
258
301
 
259
- ### Hooks/Callbacks
302
+ ### 1. Add Shrine column
260
303
 
261
- Shrine's `hooks` plugin provides callbacks for Shrine, so to get Paperclip's
262
- `(before|after)_post_process`, you can override `#before_process` and
263
- `#after_process` methods:
304
+ First we need to create the `image_data` column for Shrine:
264
305
 
265
306
  ```rb
266
- class ImageUploader < Shrine
267
- plugin :hooks
268
-
269
- def before_process(io, context)
270
- # ...
271
- super
272
- end
273
-
274
- def after_process(io, context)
275
- super
276
- # ...
277
- end
278
- end
307
+ add_column :photos, :image_data, :text
279
308
  ```
280
309
 
281
- ## Migrating from Paperclip
310
+ ### 2. Dual write
282
311
 
283
- You have an existing app using Paperclip and you want to transfer it to Shrine.
284
- First we need to make new uploads write to the `<attachment>_data` column.
285
- Let's assume we have a `Photo` model with the "image" attachment:
312
+ Next, we need to make new Paperclip attachments write to the `image_data`
313
+ column. This can be done by including the below module to all models that have
314
+ Paperclip attachments:
286
315
 
287
316
  ```rb
288
- add_column :photos, :image_data, :text
289
- ```
317
+ require "shrine"
290
318
 
291
- Afterwards we need to make new uploads write to the `image_data` column. This
292
- can be done by including the below module to all models that have Paperclip
293
- attachments:
319
+ Shrine.storages = {
320
+ cache: ...,
321
+ store: ...,
322
+ }
323
+
324
+ Shrine.plugin :model
325
+ Shrine.plugin :derivatives
294
326
 
295
- ```rb
296
327
  module PaperclipShrineSynchronization
297
328
  def self.included(model)
298
329
  model.before_save do
@@ -304,49 +335,62 @@ module PaperclipShrineSynchronization
304
335
 
305
336
  def write_shrine_data(name)
306
337
  attachment = send(name)
338
+ attacher = Shrine::Attacher.from_model(self, name)
307
339
 
308
340
  if attachment.size.present?
309
- data = attachment_to_shrine_data(attachment)
341
+ attacher.set shrine_file(attachment)
310
342
 
311
- if attachment.styles.any?
312
- data = {original: data}
313
- attachment.styles.each do |name, style|
314
- data[name] = style_to_shrine_data(style)
315
- end
343
+ attachment.styles.each do |style_name, style|
344
+ attacher.merge_derivatives(style_name => shrine_file(style))
316
345
  end
317
-
318
- write_attribute(:"#{name}_data", data.to_json)
319
346
  else
320
- write_attribute(:"#{name}_data", nil)
347
+ attacher.set nil
321
348
  end
322
349
  end
323
350
 
324
351
  private
325
352
 
326
- # If you'll be using a `:prefix` on your Shrine storage, or you're storing
327
- # files on the filesystem, make sure to subtract the appropriate part
328
- # from the path assigned to `:id`.
329
- def attachment_to_shrine_data(attachment)
330
- {
331
- storage: :store,
332
- id: attachment.path,
353
+ def shrine_file(object)
354
+ if object.is_a?(Paperclip::Attachment)
355
+ shrine_attachment_file(object)
356
+ else
357
+ shrine_style_file(object)
358
+ end
359
+ end
360
+
361
+ def shrine_attachment_file(attachment)
362
+ location = attachment.path
363
+ # if you're storing files on disk, make sure to subtract the absolute path
364
+ location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
365
+
366
+ Shrine.uploaded_file(
367
+ storage: :store,
368
+ id: location,
333
369
  metadata: {
334
- size: attachment.size,
335
- filename: attachment.original_filename,
336
- mime_type: attachment.content_type,
370
+ "size" => attachment.size,
371
+ "filename" => attachment.original_filename,
372
+ "mime_type" => attachment.content_type,
337
373
  }
338
- }
374
+ )
339
375
  end
340
376
 
341
377
  # If you'll be using a `:prefix` on your Shrine storage, or you're storing
342
378
  # files on the filesystem, make sure to subtract the appropriate part
343
379
  # from the path assigned to `:id`.
344
- def style_to_shrine_data(style)
345
- {
346
- storage: :store,
347
- id: style.attachment.path(style.name),
348
- metadata: {}
349
- }
380
+ def shrine_style_file(style)
381
+ location = style.attachment.path(style.name)
382
+ # if you're storing files on disk, make sure to subtract the absolute path
383
+ location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
384
+
385
+ Shrine.uploaded_file(
386
+ storage: :store,
387
+ id: location,
388
+ metadata: {},
389
+ )
390
+ end
391
+
392
+ def storage
393
+ Shrine.storages[:store]
350
394
  end
351
395
  end
352
396
  ```
@@ -358,34 +402,35 @@ end
358
402
  ```
359
403
 
360
404
  After you deploy this code, the `image_data` column should now be successfully
361
- synchronized with new attachments. Next step is to run a script which writes
362
- all existing Paperclip attachments to `image_data`:
405
+ synchronized with new attachments.
406
+
407
+ ### 3. Data migration
408
+
409
+ Next step is to run a script which writes all existing Paperclip attachments to
410
+ `image_data`:
363
411
 
364
412
  ```rb
365
413
  Photo.find_each do |photo|
366
- Paperclip::AttachmentRegistry.each_definition do |klass, name, options|
367
- photo.write_shrine_data(name) if klass == Photo
368
- end
414
+ photo.write_shrine_data(:image)
369
415
  photo.save!
370
416
  end
371
417
  ```
372
418
 
419
+ ### 4. Rewrite code
420
+
373
421
  Now you should be able to rewrite your application so that it uses Shrine
374
- instead of Paperclip, using equivalent Shrine storages. For help with
375
- translating the code from Paperclip to Shrine, you can consult the reference
376
- below.
422
+ instead of Paperclip (you can consult the reference in the next section). You
423
+ can remove the `PaperclipShrineSynchronization` module as well.
377
424
 
378
- You'll notice that Shrine metadata will be absent from the migrated files' data
379
- (specifically versions). You can run a script that will fill in any missing
380
- metadata defined in your Shrine uploader:
425
+ ### 5. Remove Paperclip columns
381
426
 
382
- ```rb
383
- Shrine.plugin :refresh_metadata
427
+ If everything is looking good, we can remove Paperclip columns:
384
428
 
385
- Photo.find_each do |photo|
386
- attachment = ImageUploader.uploaded_file(photo.image, &:refresh_metadata!)
387
- photo.update(image_data: attachment.to_json)
388
- end
429
+ ```rb
430
+ remove_column :photos, :image_file_name
431
+ remove_column :photos, :image_file_size
432
+ remove_column :photos, :image_content_type
433
+ remove_column :photos, :image_updated_at
389
434
  ```
390
435
 
391
436
  ## Paperclip to Shrine direct mapping
@@ -396,8 +441,8 @@ As mentioned above, Shrine's equivalent of `has_attached_file` is including
396
441
  an attachment module:
397
442
 
398
443
  ```rb
399
- class User < Sequel::Model
400
- include ImageUploader::Attachment.new(:avatar) # adds `avatar`, `avatar=` and `avatar_url` methods
444
+ class Photo < Sequel::Model
445
+ include ImageUploader::Attachment(:image) # adds `image`, `image=` and `image_url` methods
401
446
  end
402
447
  ```
403
448
 
@@ -420,8 +465,23 @@ You can change that for a specific uploader with the `default_storage` plugin.
420
465
 
421
466
  #### `:styles`, `:processors`, `:convert_options`
422
467
 
423
- As explained in the "Processing" section, processing is done by overriding the
424
- `Shrine#process` method.
468
+ Processing is defined by using the `derivatives` plugin:
469
+
470
+ ```rb
471
+ class ImageUploader < Shrine
472
+ plugin :derivatives
473
+
474
+ Attacher.derivatives do |original|
475
+ magick = ImageProcessing::MiniMagick.source(original)
476
+
477
+ {
478
+ large: magick.resize_to_limit!(800, 800),
479
+ medium: magick.resize_to_limit!(500, 500),
480
+ small: magick.resize_to_limit!(300, 300),
481
+ }
482
+ end
483
+ end
484
+ ```
425
485
 
426
486
  #### `:default_url`
427
487
 
@@ -431,8 +491,8 @@ For default URLs you can use the `default_url` plugin:
431
491
  class ImageUploader < Shrine
432
492
  plugin :default_url
433
493
 
434
- Attacher.default_url do |options|
435
- "/attachments/#{name}/default.jpg"
494
+ Attacher.default_url do |derivative: nil, **|
495
+ "/images/placeholders/#{derivative || "original"}.jpg"
436
496
  end
437
497
  end
438
498
  ```
@@ -443,7 +503,7 @@ Shrine provides a `keep_files` plugin which allows you to keep files that would
443
503
  otherwise be deleted:
444
504
 
445
505
  ```rb
446
- Shrine.plugin :keep_files, destroyed: true
506
+ Shrine.plugin :keep_files
447
507
  ```
448
508
 
449
509
  #### `:path`, `:url`, `:interpolator`, `:url_generator`
@@ -460,38 +520,108 @@ Alternatively, if you want to generate locations yourself you can override the
460
520
 
461
521
  ```rb
462
522
  class ImageUploader < Shrine
463
- def generate_location(io, context)
464
- # ...
523
+ def generate_location(io, record: nil, name: nil, **)
524
+ [ storage_key,
525
+ record && record.class.name.underscore,
526
+ record && record.id,
527
+ super,
528
+ io.original_filename ].compact.join("/")
465
529
  end
466
530
  end
467
531
  ```
532
+ ```
533
+ cache/user/123/2feff8c724e7ce17/nature.jpg
534
+ store/user/456/7f99669fde1e01fc/kitten.jpg
535
+ ...
536
+ ```
468
537
 
469
538
  #### `:validate_media_type`
470
539
 
471
540
  Shrine has this functionality in the `determine_mime_type` plugin.
472
541
 
542
+ ### `validates_attachment`
543
+
544
+ #### `:presence`
545
+
546
+ For presence validation you can use your ORM's presence validator:
547
+
548
+ ```rb
549
+ class Photo < ActiveRecord::Base
550
+ include ImageUploader::Attachment(:image)
551
+ validates_presence_of :image
552
+ end
553
+ ```
554
+
555
+ #### `:content_type`
556
+
557
+ You can do MIME type validation with Shrine's `validation_helpers` plugin:
558
+
559
+ ```rb
560
+ class ImageUploader < Shrine
561
+ plugin :validation_helpers
562
+
563
+ Attacher.validate do
564
+ validate_mime_type %w[image/jpeg image/png image/webp]
565
+ end
566
+ end
567
+ ```
568
+
569
+ Make sure to also load the `determine_mime_type` plugin to detect MIME type
570
+ from file content.
571
+
572
+ ```rb
573
+ # Gemfile
574
+ gem "mimemagic"
575
+ ```
576
+ ```rb
577
+ Shrine.plugin :determine_mime_type, analyzer: -> (io, analyzers) do
578
+ analyzers[:mimemagic].call(io) || analyzers[:file].call(io)
579
+ end
580
+ ```
581
+
582
+ #### `:size`
583
+
584
+ You can do filesize validation with Shrine's `validation_helpers` plugin:
585
+
586
+ ```rb
587
+ class ImageUploader < Shrine
588
+ plugin :validation_helpers
589
+
590
+ Attacher.validate do
591
+ validate_max_size 10*1024*1024
592
+ end
593
+ end
594
+ ```
595
+
473
596
  ### `Paperclip::Attachment`
474
597
 
475
598
  This section explains the equivalent of Paperclip attachment's methods, in
476
599
  Shrine this is an instance of `Shrine::UploadedFile`.
477
600
 
478
- #### `#url`, `#styles`
601
+ #### `#url`
602
+
603
+ In Shrine you can generate URLs with `#<name>_url`:
604
+
605
+ ```rb
606
+ photo.image_url #=> "https://example.com/path/to/original.jpg"
607
+ photo.image_url(:large) #=> "https://example.com/path/to/large.jpg"
608
+ ```
609
+
610
+ #### `#styles`
479
611
 
480
- If you're generating versions in Shrine, the attachment will be a hash of
481
- uploaded files:
612
+ In Shrine you can use `#<name>_derivatives` to retrieve a list of versions:
482
613
 
483
614
  ```rb
484
- user.avatar.class #=> Hash
485
- user.avatar #=>
615
+ photo.image_derivatives #=>
486
616
  # {
487
617
  # small: #<Shrine::UploadedFile>,
488
618
  # medium: #<Shrine::UploadedFile>,
489
619
  # large: #<Shrine::UploadedFile>,
490
620
  # }
491
621
 
492
- user.avatar[:small].url #=> "..."
622
+ photo.image_derivatives[:small] #=> #<Shrine::UploadedFile>
493
623
  # or
494
- user.avatar_url(:small) #=> "..."
624
+ photo.image(:small) #=> #<Shrine::UploadedFile>
495
625
  ```
496
626
 
497
627
  #### `#path`
@@ -500,17 +630,17 @@ Shrine doesn't have this because storages are abstract and this would be
500
630
  specific to the filesystem, but the closest is probably `#id`:
501
631
 
502
632
  ```rb
503
- user.avatar.id #=> "users/342/avatar/398543qjfdsf.jpg"
633
+ photo.image.id #=> "photo/342/image/398543qjfdsf.jpg"
504
634
  ```
505
635
 
506
636
  #### `#reprocess!`
507
637
 
508
- Shrine doesn't have an equivalent to this, but the [Reprocessing versions]
638
+ Shrine doesn't have an equivalent to this, but the [Managing Derivatives]
509
639
  guide provides some useful tips on how to do this.
510
640
 
511
641
  ### `Paperclip::Storage::S3`
512
642
 
513
- The built-in [`Shrine::Storage::S3`] storage is a direct replacement for
643
+ The built-in [`Shrine::Storage::S3`][S3] storage is a direct replacement for
514
644
  `Paperclip::Storage::S3`.
515
645
 
516
646
  #### `:s3_credentials`, `:s3_region`, `:bucket`
@@ -527,43 +657,28 @@ Shrine::Storage::S3.new(
527
657
  )
528
658
  ```
529
659
 
530
- #### `:s3_headers`
531
-
532
- The object data can be configured via the `:upload_options` hash:
533
-
534
- ```rb
535
- Shrine::Storage::S3.new(upload_options: {content_disposition: "attachment"}, **options)
536
- ```
537
-
538
- You can use the `upload_options` plugin to set upload options dynamically.
539
-
540
- #### `:s3_permissions`
541
-
542
- The object permissions can be configured with the `:acl` upload option:
543
-
544
- ```rb
545
- Shrine::Storage::S3.new(upload_options: {acl: "private"}, **options)
546
- ```
547
-
548
- You can use the `upload_options` plugin to set upload options dynamically.
549
-
550
- #### `:s3_metadata`
660
+ #### `:s3_headers`, `:s3_permissions`, `:s3_metadata`
551
661
 
552
- The object metadata can be configured with the `:metadata` upload option:
662
+ These can be configured via the `:upload_options` option:
553
663
 
554
664
  ```rb
555
- Shrine::Storage::S3.new(upload_options: {metadata: {"key" => "value"}}, **options)
665
+ Shrine::Storage::S3.new(
666
+ upload_options: {
667
+ content_disposition: "attachment", # headers
668
+ acl: "private", # permissions
669
+ metadata: { "key" => "value" }, # metadata
670
+ },
671
+ **options
672
+ )
556
673
  ```
557
674
 
558
- You can use the `upload_options` plugin to set upload options dynamically.
559
-
560
675
  #### `:s3_protocol`, `:s3_host_alias`, `:s3_host_name`
561
676
 
562
677
  The `#url` method accepts a `:host` option for specifying a CDN host. You can
563
- use the `default_url_options` plugin to set it by default:
678
+ use the `url_options` plugin to set it by default:
564
679
 
565
680
  ```rb
566
- Shrine.plugin :default_url_options, store: {host: "http://abc123.cloudfront.net"}
681
+ Shrine.plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
567
682
  ```
568
683
 
569
684
  #### `:path`
@@ -580,7 +695,14 @@ s3.upload(io, "object/destination/path")
580
695
  The Shrine storage has no replacement for the `:url` Paperclip option, and it
581
696
  isn't needed.
582
697
 
583
- [file]: http://linux.die.net/man/1/file
584
- [Reprocessing versions]: /doc/regenerating_versions.md#readme
585
- [direct S3 uploads]: /doc/direct_s3.md#readme
586
- [`Shrine::Storage::S3`]: /doc/storage/s3.md#readme
698
+ [metadata_attributes]: https://shrinerb.com/docs/plugins/metadata_attributes
699
+ [Managing Derivatives]: https://shrinerb.com/docs/changing-derivatives
700
+ [direct uploads]: https://shrinerb.com/docs/getting-started#direct-uploads
701
+ [S3]: https://shrinerb.com/docs/storage/s3
702
+ [image_processing]: https://github.com/janko/image_processing
703
+ [libvips]: http://libvips.github.io/libvips/
704
+ [activerecord]: https://shrinerb.com/docs/plugins/activerecord
705
+ [sequel]: https://shrinerb.com/docs/plugins/sequel
706
+ [rom]: https://github.com/shrinerb/shrine-rom
707
+ [hanami]: https://github.com/katafrakt/hanami-shrine
708
+ [mongoid]: https://github.com/shrinerb/shrine-mongoid