shrine 3.0.0.beta2 → 3.0.0.beta3

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

Files changed (61) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +45 -1
  3. data/README.md +100 -106
  4. data/doc/advantages.md +90 -88
  5. data/doc/attacher.md +322 -152
  6. data/doc/carrierwave.md +105 -113
  7. data/doc/changing_derivatives.md +308 -0
  8. data/doc/changing_location.md +92 -21
  9. data/doc/changing_storage.md +107 -0
  10. data/doc/creating_plugins.md +1 -1
  11. data/doc/design.md +8 -9
  12. data/doc/direct_s3.md +3 -2
  13. data/doc/metadata.md +97 -78
  14. data/doc/multiple_files.md +3 -3
  15. data/doc/paperclip.md +89 -88
  16. data/doc/plugins/activerecord.md +3 -12
  17. data/doc/plugins/backgrounding.md +126 -100
  18. data/doc/plugins/derivation_endpoint.md +4 -5
  19. data/doc/plugins/derivatives.md +63 -32
  20. data/doc/plugins/download_endpoint.md +54 -1
  21. data/doc/plugins/entity.md +1 -0
  22. data/doc/plugins/form_assign.md +53 -0
  23. data/doc/plugins/mirroring.md +37 -16
  24. data/doc/plugins/multi_cache.md +22 -0
  25. data/doc/plugins/presign_endpoint.md +1 -1
  26. data/doc/plugins/remote_url.md +19 -4
  27. data/doc/plugins/validation.md +83 -0
  28. data/doc/processing.md +149 -133
  29. data/doc/refile.md +68 -63
  30. data/doc/release_notes/3.0.0.md +835 -0
  31. data/doc/securing_uploads.md +56 -36
  32. data/doc/storage/s3.md +2 -2
  33. data/doc/testing.md +104 -120
  34. data/doc/upgrading_to_3.md +538 -0
  35. data/doc/validation.md +48 -87
  36. data/lib/shrine.rb +7 -4
  37. data/lib/shrine/attacher.rb +16 -6
  38. data/lib/shrine/plugins/activerecord.rb +33 -14
  39. data/lib/shrine/plugins/atomic_helpers.rb +1 -1
  40. data/lib/shrine/plugins/backgrounding.rb +23 -89
  41. data/lib/shrine/plugins/data_uri.rb +13 -2
  42. data/lib/shrine/plugins/derivation_endpoint.rb +7 -11
  43. data/lib/shrine/plugins/derivatives.rb +44 -20
  44. data/lib/shrine/plugins/download_endpoint.rb +26 -0
  45. data/lib/shrine/plugins/form_assign.rb +6 -3
  46. data/lib/shrine/plugins/keep_files.rb +2 -2
  47. data/lib/shrine/plugins/mirroring.rb +62 -22
  48. data/lib/shrine/plugins/model.rb +2 -2
  49. data/lib/shrine/plugins/multi_cache.rb +27 -0
  50. data/lib/shrine/plugins/remote_url.rb +25 -10
  51. data/lib/shrine/plugins/remove_invalid.rb +1 -1
  52. data/lib/shrine/plugins/sequel.rb +39 -20
  53. data/lib/shrine/plugins/validation.rb +3 -0
  54. data/lib/shrine/storage/s3.rb +16 -1
  55. data/lib/shrine/uploaded_file.rb +1 -0
  56. data/lib/shrine/version.rb +1 -1
  57. data/shrine.gemspec +1 -1
  58. metadata +12 -7
  59. data/doc/migrating_storage.md +0 -76
  60. data/doc/regenerating_versions.md +0 -143
  61. data/lib/shrine/plugins/attacher_options.rb +0 -55
@@ -58,7 +58,7 @@ apply to an attached file. For example, we can generate image thumbnails using
58
58
  the [ImageProcessing] gem:
59
59
 
60
60
  ```rb
61
- gem "image_processing", "~> 1.2"
61
+ gem "image_processing", "~> 1.8"
62
62
  ```
63
63
  ```rb
64
64
  require "image_processing/mini_magick"
@@ -117,13 +117,12 @@ derivation :thumbnail do |file, arg1, arg2, ...|
117
117
 
118
118
  # ... do processing ...
119
119
 
120
- # return result as a File/Tempfile object or String/Pathname path
120
+ # return result as a File or Tempfile object
121
121
  end
122
122
  ```
123
123
 
124
- The derivation block is expected to return the processed file is a
125
- `File`/`Tempfile` object or a `String`/`Pathname` path. The resulting file is
126
- then rendered in the HTTP response.
124
+ The derivation block is expected to return the processed file as a `File` or
125
+ `Tempfile` object. The resulting file is then rendered in the HTTP response.
127
126
 
128
127
  ### Performance
129
128
 
@@ -5,13 +5,14 @@ the main attached file. The processed file data will be saved together with the
5
5
  main attachment data in the same record attribute.
6
6
 
7
7
  ```rb
8
- plugin :derivatives
8
+ Shrine.plugin :derivatives
9
9
  ```
10
10
 
11
11
  ## Contents
12
12
 
13
13
  * [API overview](#api-overview)
14
14
  * [Creating derivatives](#creating-derivatives)
15
+ - [Naming processors](#naming-processors)
15
16
  - [Derivatives storage](#derivatives-storage)
16
17
  - [Nesting derivatives](#nesting-derivatives)
17
18
  * [Retrieving derivatives](#retrieving-derivatives)
@@ -57,21 +58,19 @@ Here is an example of generating image thumbnails:
57
58
 
58
59
  ```rb
59
60
  # Gemfile
60
- gem "image_processing", "~> 1.2"
61
+ gem "image_processing", "~> 1.8"
61
62
  ```
62
63
  ```rb
63
64
  require "image_processing/mini_magick"
64
65
 
65
66
  class ImageUploader < Shrine
66
- plugin :derivatives
67
-
68
- Attacher.derivatives_processor :thumbnails do |original|
69
- processor = ImageProcessing::MiniMagick.source(original)
67
+ Attacher.derivatives_processor do |original|
68
+ magick = ImageProcessing::MiniMagick.source(original)
70
69
 
71
70
  {
72
- small: processor.resize_to_limit!(300, 300),
73
- medium: processor.resize_to_limit!(500, 500),
74
- large: processor.resize_to_limit!(800, 800),
71
+ small: magick.resize_to_limit!(300, 300),
72
+ medium: magick.resize_to_limit!(500, 500),
73
+ large: magick.resize_to_limit!(800, 800),
75
74
  }
76
75
  end
77
76
  end
@@ -82,20 +81,20 @@ class Photo < Model(:image_data)
82
81
  end
83
82
  ```
84
83
  ```rb
85
- photo.image #=> #<Shrine::UploadedFile @id="original.jpg" @storage_key=:store ...>
86
- photo.image_derivatives #=> {}
84
+ photo = Photo.new(image: file)
85
+ photo.image_derivatives! # calls derivatives processor and uploads results
86
+ photo.save
87
+ ```
87
88
 
88
- photo.image_derivatives!(:thumbnails) # calls registered processor and uploads results
89
- photo.image_derivatives #=>
90
- # {
91
- # small: #<Shrine::UploadedFile @id="small.jpg" @storage_key=:store ...>,
92
- # medium: #<Shrine::UploadedFile @id="medium.jpg" @storage_key=:store ...>,
93
- # large: #<Shrine::UploadedFile @id="large.jpg" @storage_key=:store ...>,
94
- # }
89
+ If you're allowing the attached file to be updated later on, in your update
90
+ route make sure to create derivatives for new attachments:
91
+
92
+ ```rb
93
+ photo.image_derivatives! if photo.image_changed?
95
94
  ```
96
95
 
97
- The derivatives data is stored in the `#<name>_data` record attribute alongside
98
- the main file data:
96
+ Once derivatives have been created, their data is stored in the `#<name>_data`
97
+ record attribute alongside the main file data:
99
98
 
100
99
  ```rb
101
100
  photo.image_data #=>
@@ -111,16 +110,23 @@ photo.image_data #=>
111
110
  # }
112
111
  ```
113
112
 
113
+ You can then retrieve derivatives as follows:
114
+
115
+ ```rb
116
+ photo.image(:large) #=> #<Shrine::UploadedFile>
117
+ photo.image(:large).url #=> "https://s3.amazonaws.com/path/to/large.jpg"
118
+ photo.image(:large).size #=> 43843
119
+ photo.image(:large).mime_type #=> "image/jpeg"
120
+ ```
121
+
114
122
  The `#<name>_derivatives!` model method delegates to
115
123
  `Attacher#create_derivatives`, which you can use if you're using
116
124
  `Shrine::Attacher` directly:
117
125
 
118
126
  ```rb
119
- attacher.file #=> #<Shrine::UploadedFile @id="original.jpg" @storage_key=:store ...>
120
- attacher.derivatives #=> {}
121
-
122
- attacher.create_derivatives(:thumbnails) # calls registered processor and uploads results
123
- attacher.derivatives #=>
127
+ attacher.file #=> #<Shrine::UploadedFile @id="original.jpg" @storage_key=:store ...>
128
+ attacher.create_derivatives # calls registered processor and uploads results
129
+ attacher.derivatives #=>
124
130
  # {
125
131
  # small: #<Shrine::UploadedFile @id="small.jpg" @storage_key=:store ...>,
126
132
  # medium: #<Shrine::UploadedFile @id="medium.jpg" @storage_key=:store ...>,
@@ -130,14 +136,39 @@ attacher.derivatives #=>
130
136
 
131
137
  By default, the `Attacher#create_derivatives` method downloads the attached
132
138
  file, calls the processor, uploads results to attacher's permanent storage, and
133
- saves uploaded files on the attacher.
134
-
135
- Any additional arguments are forwarded to
139
+ saves uploaded files on the attacher. Any additional arguments are forwarded to
136
140
  [`Attacher#process_derivatives`](#processing-derivatives):
137
141
 
138
142
  ```rb
139
- attacher.create_derivatives(:thumbnails, different_source) # pass a different source file
140
- attacher.create_derivatives(:thumbnails, foo: "bar") # pass custom options to the processor
143
+ attacher.create_derivatives(different_source) # pass a different source file
144
+ attacher.create_derivatives(foo: "bar") # pass custom options to the processor
145
+ ```
146
+
147
+ ### Naming processors
148
+
149
+ If you want to have multiple processors for an uploader, you can assign each
150
+ processor a name. Then when creating derivatives you can specify the name of
151
+ the desired processor.
152
+
153
+ ```rb
154
+ class ImageUploader < Shrine
155
+ Attacher.derivatives_processor :thumbnails do |original|
156
+ # ...
157
+ end
158
+
159
+ Attacher.derivatives_processor :crop do |original|
160
+ # ...
161
+ end
162
+
163
+ # ...
164
+ end
165
+ ```
166
+ ```rb
167
+ # ...
168
+ photo.image_derivatives!(:thumbnails)
169
+ # or
170
+ attacher.create_derivatives(:thumbnails)
171
+ # ...
141
172
  ```
142
173
 
143
174
  ### Derivatives storage
@@ -244,7 +275,7 @@ photo.image_derivatives #=> { thumbnail: { small: ..., medium: ..., large: ... }
244
275
 
245
276
  photo.image_derivatives.dig(:thumbnail, :small) #=> #<Shrine::UploadedFile>
246
277
  photo.image_derivatives(:thumbnail, :small) #=> #<Shrine::UploadedFile>
247
- photo.image(:thumbnails :small) #=> #<Shrine::UploadedFile>
278
+ photo.image(:thumbnails, :small) #=> #<Shrine::UploadedFile>
248
279
  ```
249
280
 
250
281
  When using `Shrine::Attacher` directly, you can retrieve derivatives using
@@ -288,7 +319,7 @@ You can use the [`default_url`][default_url] plugin to set up URL fallbacks:
288
319
 
289
320
  ```rb
290
321
  Attacher.default_url do |derivative: nil, **|
291
- "https://fallbacks.com/#{derivative}.jpg" if derivative
322
+ "https://my-app.com/fallbacks/#{derivative}.jpg" if derivative
292
323
  end
293
324
  ```
294
325
  ```rb
@@ -5,11 +5,13 @@ downloading uploaded files from specified storages. This can be useful when
5
5
  files from your storage isn't accessible over URL (e.g. database storages) or
6
6
  if you want to authenticate your downloads.
7
7
 
8
+ ## Global Endpoint
9
+
8
10
  You can configure the plugin with the path prefix which the endpoint will be
9
11
  mounted on.
10
12
 
11
13
  ```rb
12
- plugin :download_endpoint, prefix: "attachments"
14
+ Shrine.plugin :download_endpoint, prefix: "attachments"
13
15
  ```
14
16
 
15
17
  The plugin adds a `Shrine.download_endpoint` method which returns a Rack
@@ -30,6 +32,57 @@ Links to the download endpoint are generated by calling
30
32
  ```rb
31
33
  uploaded_file.download_url #=> "/attachments/eyJpZCI6ImFkdzlyeTM..."
32
34
  ```
35
+ ## Endpoint via Uploader
36
+
37
+ You can also configure the plugin in the uploader directly - just make sure to mount it via your Uploader-class.
38
+
39
+ ```rb
40
+ class ImageUploader < Shrine
41
+ plugin :download_endpoint, prefix: "images"
42
+ end
43
+ ```
44
+
45
+ ```rb
46
+ # config/routes.rb (Rails)
47
+ Rails.application.routes.draw do
48
+ # ...
49
+ mount ImageUploader.download_endpoint => "/images"
50
+ end
51
+ ```
52
+
53
+ *Hint: For shrine versions 2.x -> ensure that you don't include the plugin twice (globally and in your uploader class - see #408)*
54
+
55
+ ## Calling from a controller
56
+
57
+ If you want to run additional code around the download (such as authentication),
58
+ mounting the download endpoint in your router might be limiting. You can instead
59
+ create a custom controller action and handle download requests there using
60
+ `Shrine.download_response`:
61
+
62
+ ```rb
63
+ # config/routes.rb (Rails)
64
+ Rails.application.routes.draw do
65
+ # ...
66
+ get "/attachments/*rest", to: "downloads#image"
67
+ end
68
+ ```
69
+ ```rb
70
+ # app/controllers/downloads_controller.rb (Rails)
71
+ class DownloadsController < ApplicationController
72
+ def image
73
+ # ... we can perform authentication here ...
74
+ set_rack_response ImageUploader.download_response(request.env)
75
+ end
76
+
77
+ private
78
+
79
+ def set_rack_response((status, headers, body))
80
+ self.status = status
81
+ self.headers.merge!(headers)
82
+ self.response_body = body
83
+ end
84
+ end
85
+ ```
33
86
 
34
87
  ## Host
35
88
 
@@ -248,5 +248,6 @@ By default, attachment data is serialized into JSON using the `JSON` standard
248
248
  library. If you want to change how data is serialized, see the
249
249
  [`column`][column serializer] plugin docs.
250
250
 
251
+ [column]: /doc/plugins/column.md#readme
251
252
  [entity]: /lib/shrine/plugins/entity.rb
252
253
  [column serializer]: /doc/plugins/column.md#serializer
@@ -0,0 +1,53 @@
1
+ # Form Assign
2
+
3
+ The [`form_assign`][form_assign] plugin allows attaching file from form params
4
+ without a form object.
5
+
6
+ ```rb
7
+ plugin :form_assign
8
+ ```
9
+
10
+ The `Attacher#form_assign` method will detect the file param and assign it to
11
+ the attacher:
12
+
13
+ ```rb
14
+ attacher = photo.image_attacher
15
+ attacher.form_assign("image" => file, "title" => "...", "description" => "...")
16
+ attacher.file #=> #<Shrine::UploadedFile>
17
+ ```
18
+
19
+ It works with `remote_url`, `data_uri`, and `remove_attachment` plugins:
20
+
21
+ ```rb
22
+ # remote_url plugin
23
+ attacher.form_assign("image_remote_url" => "https://example.com/...")
24
+ attacher.file #=> #<Shrine::UploadedFile>
25
+ ```
26
+ ```rb
27
+ # data_uri plugin
28
+ attacher.form_assign("image_data_uri" => "data:image/jpeg;base64,...")
29
+ attacher.file #=> #<Shrine::UploadedFile>
30
+ ```
31
+ ```rb
32
+ # remove_attachment plugin
33
+ attacher.form_assign("remove_image" => "1")
34
+ attacher.file #=> nil
35
+ ```
36
+
37
+ The return value is a hash with form params, with file param replaced with
38
+ cached file data, which can later be assigned again to the record.
39
+
40
+ ```rb
41
+ attacher.form_assign("image" => file, "title" => "...", "description" => "...")
42
+ #=> { :image => '{"id":"...","storage":"...","metadata":"..."}', "title" => "...", "description" => "..." }
43
+ ```
44
+
45
+ You can also have attached file data returned as the `<name>_data` attribute,
46
+ suitable for persisting.
47
+
48
+ ```rb
49
+ attacher.form_assign({ "image" => image, ... }, result: :attributes)
50
+ #=> { :image_data => '{"id":"...","storage":"...","metadata":"..."}', "title" => "...", "description" => "..." }
51
+ ```
52
+
53
+ [form_assign]: /lib/shrine/plugins/form_assign.rb
@@ -12,8 +12,16 @@ With the above setup, any upload and delete to `:store` will be replicated to
12
12
  `:other_store`.
13
13
 
14
14
  ```rb
15
- uploaded_file = Shrine.upload(io, :store) # uploads to :store and :other_store
16
- uploaded_file.delete # deletes from :store and :other_store
15
+ file = Shrine.upload(io, :store) # uploads to :store and :other_store
16
+ file.delete # deletes from :store and :other_store
17
+ ```
18
+
19
+ You can skip mirroring for a specific upload/delete call by passing `mirror:
20
+ false`:
21
+
22
+ ```rb
23
+ file = Shrine.upload(io, :store, mirror: false) # skips mirroring
24
+ file.delete(mirror: false) # skips mirroring
17
25
  ```
18
26
 
19
27
  ## Multiple storages
@@ -40,42 +48,43 @@ Shrine.plugin :mirroring, mirror: { ... }, delete: false
40
48
  You can have mirroring performed in a background job:
41
49
 
42
50
  ```rb
43
- Shrine.mirror_upload do |uploaded_file|
44
- MirrorUploadJob.perform_async(uploaded_file.shrine_class, uploaded_file.data)
51
+ Shrine.mirror_upload_block do |file|
52
+ MirrorUploadJob.perform_later(file.shrine_class, file.data)
45
53
  end
46
54
 
47
- Shrine.mirror_delete do |uploaded_file|
48
- MirrorDeleteJob.perform_async(uploaded_file.shrine_class, uploaded_file.data)
55
+ Shrine.mirror_delete_block do |file|
56
+ MirrorDeleteJob.perform_later(file.shrine_class, file.data)
49
57
  end
50
58
  ```
51
59
  ```rb
52
- class MirrorUploadJob
53
- include Sidekiq::Worker
60
+ class MirrorUploadJob < ActiveJob::Base
54
61
  def perform(shrine_class, file_data)
55
- uploaded_file = Object.const_get(shrine_class).uploaded_file(file_data)
56
- uploaded_file.mirror_upload
62
+ file = shrine_class.uploaded_file(file_data)
63
+ file.mirror_upload
57
64
  end
58
65
  end
59
66
  ```
60
67
  ```rb
61
- class MirrorDeleteJob
62
- include Sidekiq::Worker
68
+ class MirrorDeleteJob < ActiveJob::Base
63
69
  def perform(shrine_class, file_data)
64
- uploaded_file = Object.const_get(shrine_class).uploaded_file(file_data)
65
- uploaded_file.mirror_delete
70
+ file = shrine_class.uploaded_file(file_data)
71
+ file.mirror_delete
66
72
  end
67
73
  end
68
74
  ```
69
75
 
70
76
  ## API
71
77
 
72
- You can mirror manually via `UploadedFile#mirror_upload` and
73
- `UploadedFile#mirror_delete`:
78
+ You can disable automatic mirroring and perform mirroring manually:
74
79
 
75
80
  ```rb
76
81
  # disable automatic mirroring of uploads and deletes
77
82
  Shrine.plugin :mirroring, mirror: { ... }, upload: false, delete: false
78
83
  ```
84
+
85
+ To perform mirroring, you can call `UploadedFile#mirror_upload` and
86
+ `UploadedFile#mirror_delete`:
87
+
79
88
  ```rb
80
89
  file = Shrine.upload(io, :store) # upload to :store
81
90
  file.mirror_upload # upload to :other_store
@@ -84,4 +93,16 @@ file.delete # delete from :store
84
93
  file.mirror_delete # delete from :other_store
85
94
  ```
86
95
 
96
+ If you've set up backgrounding, you can use
97
+ `UploadedFile#mirror_upload_background` and
98
+ `UploadedFile#mirror_delete_background` to call the background block instead:
99
+
100
+ ```rb
101
+ file = Shrine.upload(io, :store) # upload to :store
102
+ file.mirror_upload_background # spawn mirror upload background job
103
+
104
+ file.delete # delete from :store
105
+ file.mirror_delete_background # spawn mirror delete background job
106
+ ```
107
+
87
108
  [mirroring]: /lib/shrine/plugins/mirroring.rb
@@ -0,0 +1,22 @@
1
+ # Multi Cache
2
+
3
+ The [`multi_cache`][multi_cache] plugin allows an attacher to accept files from
4
+ additional temporary storages.
5
+
6
+ ```rb
7
+ Shrine.storages = { cache: ..., cache_one: ..., cache_two: ..., store: ... }
8
+
9
+ Shrine.plugin :multi_cache, additional_cache: [:cache_one, :cache_two]
10
+ ```
11
+ ```rb
12
+ photo.image = { "id" => "...", "storage" => "cache", "metadata" => { ... } }
13
+ photo.image.storage_key #=> :cache
14
+ # or
15
+ photo.image = { "id" => "...", "storage" => "cache_one", "metadata" => { ... } }
16
+ photo.image.storage_key #=> :cache_one
17
+ # or
18
+ photo.image = { "id" => "...", "storage" => "cache_two", "metadata" => { ... } }
19
+ photo.image.storage_key #=> :cache_two
20
+ ```
21
+
22
+ [multi_cache]: /lib/shrine/plugins/multi_cache.rb
@@ -60,7 +60,7 @@ create a custom controller action and handle presign requests there using
60
60
  # config/routes.rb (Rails)
61
61
  Rails.application.routes.draw do
62
62
  # ...
63
- post "/images/presign", to: "presigns#image"
63
+ get "/images/presign", to: "presigns#image"
64
64
  end
65
65
  ```
66
66
  ```rb