shrine 3.0.1 → 3.3.0

data/doc/plugins/derivatives.md

@@ -2,7 +2,7 @@
 title: Derivatives
 ---
 
-The derivatives plugin allows storing processed files ("derivatives") alongside
+The [`derivatives`][derivatives] plugin allows storing processed files ("derivatives") alongside
 the main attached file. The processed file data will be saved together with the
 main attachment data in the same record attribute.
 
@@ -10,12 +10,11 @@ main attachment data in the same record attribute.
 Shrine.plugin :derivatives
 ```
 
-## Creating derivatives
+## Quick start
 
-When you have a file attached, you can generate derivatives from it and save
-them alongside the attached file. The simplest way to do this is to define a
-processor which returns the processed files, and then trigger it when you want
-to create derivatives.
+You'll usually want to create derivatives from an attached file. The simplest
+way to do this is to define a processor which returns the processed files, and
+then trigger it when you want to create derivatives.
 
 Here is an example of generating image thumbnails:
 
@@ -27,7 +26,7 @@ gem "image_processing", "~> 1.8"
 require "image_processing/mini_magick"
 
 class ImageUploader < Shrine
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(original)
 
     {
@@ -39,56 +38,149 @@ class ImageUploader < Shrine
 end
 ```
 ```rb
-class Photo < Model(:image_data)
-  include ImageUploader::Attachment(:image)
-end
-```
-```rb
 photo = Photo.new(image: file)
-photo.image_derivatives! # calls derivatives processor and uploads results
+photo.image_derivatives! # creates derivatives
 photo.save
 ```
 
-If you're allowing the attached file to be updated later on, in your update
-route make sure to create derivatives for new attachments:
+You can then retrieve the URL of a processed derivative:
 
 ```rb
-photo.image_derivatives! if photo.image_changed?
+photo.image_url(:large) #=> "https://s3.amazonaws.com/path/to/large.jpg"
 ```
 
-Once derivatives have been created, their data is stored in the `#<name>_data`
-record attribute alongside the main file data:
+The derivatives data is stored in the `<attachment>_data` column alongside the
+main file:
 
 ```rb
 photo.image_data #=>
 # {
-#   "id": "original.jpg",
+#   "id": "path/to/original.jpg",
 #   "store": "store",
 #   "metadata": { ... },
 #   "derivatives": {
-#     "small": { "id": "small.jpg", "storage": "store", "metadata": { ... } },
-#     "medium": { "id": "medium.jpg", "storage": "store", "metadata": { ... } },
-#     "large": { "id": "large.jpg", "storage": "store", "metadata": { ... } },
+#     "small": { "id": "path/to/small.jpg", "storage": "store", "metadata": { ... } },
+#     "medium": { "id": "path/to/medium.jpg", "storage": "store", "metadata": { ... } },
+#     "large": { "id": "path/to/large.jpg", "storage": "store", "metadata": { ... } },
 #   }
 # }
 ```
 
-You can then retrieve derivatives as follows:
+And they can be retrieved as `Shrine::UploadedFile` objects:
 
 ```rb
-photo.image(:large)           #=> #<Shrine::UploadedFile>
+photo.image(:large)           #=> #<Shrine::UploadedFile id="path/to/large.jpg" storage=:store metadata={...}>
 photo.image(:large).url       #=> "https://s3.amazonaws.com/path/to/large.jpg"
-photo.image(:large).size      #=> 43843
+photo.image(:large).size      #=> 5825949
 photo.image(:large).mime_type #=> "image/jpeg"
 ```
 
-The `#<name>_derivatives!` model method delegates to
-`Attacher#create_derivatives`, which you can use if you're using
-`Shrine::Attacher` directly:
+## Retrieving derivatives
+
+The list of stored derivatives can be retrieved with `#<name>_derivatives`:
+
+```rb
+photo.image_derivatives #=>
+# {
+#   small:  #<Shrine::UploadedFile ...>,
+#   medium: #<Shrine::UploadedFile ...>,
+#   large:  #<Shrine::UploadedFile ...>,
+# }
+```
+
+A specific derivative can be retrieved in any of the following ways:
+
+```rb
+photo.image_derivatives[:small] #=> #<Shrine::UploadedFile ...>
+photo.image_derivatives(:small) #=> #<Shrine::UploadedFile ...>
+photo.image(:small)             #=> #<Shrine::UploadedFile ...>
+```
+
+Or with nested derivatives:
+
+```rb
+photo.image_derivatives #=> { thumbnail: { small: ..., medium: ..., large: ... } }
+
+photo.image_derivatives.dig(:thumbnail, :small) #=> #<Shrine::UploadedFile ...>
+photo.image_derivatives(:thumbnail, :small)     #=> #<Shrine::UploadedFile ...>
+photo.image(:thumbnails, :small)                #=> #<Shrine::UploadedFile ...>
+```
+
+### Derivative URL
+
+You can retrieve the URL of a derivative URL with `#<name>_url`:
+
+```rb
+photo.image_url(:small)  #=> "https://example.com/small.jpg"
+photo.image_url(:medium) #=> "https://example.com/medium.jpg"
+photo.image_url(:large)  #=> "https://example.com/large.jpg"
+```
+
+For nested derivatives you can pass multiple keys:
+
+```rb
+photo.image_derivatives #=> { thumbnail: { small: ..., medium: ..., large: ... } }
+
+photo.image_url(:thumbnail, :medium) #=> "https://example.com/medium.jpg"
+```
+
+By default, `#<name>_url` method will return `nil` if derivative is not found.
+You can use the [`default_url`][default_url] plugin to set up URL fallbacks:
+
+```rb
+Attacher.default_url do |derivative: nil, **|
+  "/fallbacks/#{derivative}.jpg" if derivative
+end
+```
+```rb
+photo.image_url(:medium) #=> "https://example.com/fallbacks.com/medium.jpg"
+```
+
+Any additional URL options passed to `#<name>_url` will be forwarded to the
+storage:
+
+```rb
+photo.image_url(:small, response_content_disposition: "attachment")
+```
+
+You can also retrieve the derivative URL via `UploadedFile#url`:
+
+```rb
+photo.image_derivatives[:large].url
+```
+
+## Attacher API
+
+The derivatives API is primarily defined on the `Shrine::Attacher` class, with
+some important methods also being exposed through the `Shrine::Attachment`
+module.
+
+Here is a model example with equivalent attacher code:
+
+```rb
+photo.image_derivatives!(:thumbnails)
+photo.image_derivatives #=> { ... }
+
+photo.image_url(:large) #=> "https://..."
+photo.image(:large)     #=> #<Shrine::UploadedFile ...>
+```
+```rb
+attacher.create_derivatives(:thumbnails)
+attacher.get_derivatives #=> { ... }
+
+attacher.url(:large) #=> "https://..."
+attacher.get(:large) #=> "#<Shrine::UploadedFile>"
+```
+
+## Creating derivatives
+
+By default, the `Attacher#create_derivatives` method downloads the attached
+file, calls the processor, uploads results to attacher's permanent storage, and
+saves uploaded files on the attacher.
 
 ```rb
 attacher.file               #=> #<Shrine::UploadedFile id="original.jpg" storage=:store ...>
-attacher.create_derivatives # calls registered processor and uploads results
+attacher.create_derivatives # calls default processor and uploads results
 attacher.derivatives        #=>
 # {
 #   small:  #<Shrine::UploadedFile id="small.jpg" storage=:store ...>,
@@ -97,9 +189,7 @@ attacher.derivatives        #=>
 # }
 ```
 
-By default, the `Attacher#create_derivatives` method downloads the attached
-file, calls the processor, uploads results to attacher's permanent storage, and
-saves uploaded files on the attacher. Any additional arguments are forwarded to
+Any additional arguments are forwarded to
 [`Attacher#process_derivatives`](#processing-derivatives):
 
 ```rb
@@ -107,35 +197,59 @@ attacher.create_derivatives(different_source) # pass a different source file
 attacher.create_derivatives(foo: "bar")       # pass custom options to the processor
 ```
 
+### Create on promote
+
+You can also have derivatives created automatically on promotion:
+
+```rb
+Shrine.plugin :derivatives, create_on_promote: true
+```
+```rb
+attacher.assign(file)
+attacher.finalize # creates derivatives on promotion
+attacher.derivatives #=> { small: ..., medium: ..., large: ... }
+```
+
 ### Naming processors
 
 If you want to have multiple processors for an uploader, you can assign each
-processor a name. Then when creating derivatives you can specify the name of
-the desired processor.
+processor a name:
 
 ```rb
 class ImageUploader < Shrine
-  Attacher.derivatives_processor :thumbnails do |original|
-    # ...
+  Attacher.derivatives :thumbnails do |original|
+    { large: ..., medium: ..., small: ... }
   end
 
-  Attacher.derivatives_processor :crop do |original|
-    # ...
+  Attacher.derivatives :crop do |original|
+    { cropped: ... }
   end
-
-  # ...
 end
 ```
+
+Then when creating derivatives you can specify the name of the desired
+processor. New derivatives will be merged with any existing ones.
+
 ```rb
-photo.image_derivatives!(:thumbnails)
-# or
 attacher.create_derivatives(:thumbnails)
+attacher.derivatives #=> { large: ..., medium: ..., small: ... }
+
+attacher.create_derivatives(:crop)
+attacher.derivatives #=> { large: ..., medium: ..., small: ..., cropped: ... }
 ```
 
 ### Derivatives storage
 
 By default, derivatives are uploaded to the permanent storage of the attacher.
-You can change the default destination storage with the `:storage` plugin
+You can change the destination storage by passing `:storage` to the creation
+call:
+
+```rb
+attacher.create_derivatives(storage: :cache) # will be promoted together with main file
+attacher.create_derivatives(storage: :other_store)
+```
+
+You can also change the default destination storage with the `:storage` plugin
 option:
 
 ```rb
@@ -172,11 +286,14 @@ The storage block is evaluated in the context of a `Shrine::Attacher` instance:
 
 ```rb
 Attacher.derivatives_storage do |derivative|
-  self   #=> #<Shrine::Attacher>
+  self    #=> #<Shrine::Attacher>
 
+  file    #=> #<Shrine::UploadedFile>
   record  #=> #<Photo>
   name    #=> :image
   context #=> { ... }
+
+  # ...
 end
 ```
 
@@ -186,7 +303,7 @@ Derivatives can be nested to any level, using both hashes and arrays, but the
 top-level object must be a hash.
 
 ```rb
-Attacher.derivatives_processor :tiff do |original|
+Attacher.derivatives :tiff do |original|
   {
     thumbnail: {
       small:  small,
@@ -201,114 +318,29 @@ Attacher.derivatives_processor :tiff do |original|
   }
 end
 ```
-
-## Retrieving derivatives
-
-If you're using the `Shrine::Attachment` module, you can retrieve stored
-derivatives by calling `#<name>_derivatives` on your model/entity.
-
-```rb
-class Photo < Model(:image_data)
-  include ImageUploader::Attachment(:image)
-end
-```
-```rb
-photo.image_derivatives #=>
-# {
-#   small:  #<Shrine::UploadedFile>,
-#   medium: #<Shrine::UploadedFile>,
-#   large:  #<Shrine::UploadedFile>,
-# }
-```
-
-A specific derivative can be retrieved in any of the following ways:
-
-```rb
-photo.image_derivatives[:small] #=> #<Shrine::UploadedFile>
-photo.image_derivatives(:small) #=> #<Shrine::UploadedFile>
-photo.image(:small)             #=> #<Shrine::UploadedFile>
-```
-
-And with nested derivatives:
-
-```rb
-photo.image_derivatives #=> { thumbnail: { small: ..., medium: ..., large: ... } }
-
-photo.image_derivatives.dig(:thumbnail, :small) #=> #<Shrine::UploadedFile>
-photo.image_derivatives(:thumbnail, :small)     #=> #<Shrine::UploadedFile>
-photo.image(:thumbnails, :small)                #=> #<Shrine::UploadedFile>
-```
-
-When using `Shrine::Attacher` directly, you can retrieve derivatives using
-`Attacher#derivatives`:
-
 ```rb
 attacher.derivatives #=>
 # {
-#   small:  #<Shrine::UploadedFile>,
-#   medium: #<Shrine::UploadedFile>,
-#   large:  #<Shrine::UploadedFile>,
+#   thumbnail: {
+#     small:  #<Shrine::UploadedFile ...>,
+#     medium: #<Shrine::UploadedFile ...>,
+#     large:  #<Shrine::UploadedFile ...>,
+#   },
+#   layers: [
+#     #<Shrine::UploadedFile ...>,
+#     #<Shrine::UploadedFile ...>,
+#     # ...
+#   ]
 # }
 ```
 
-## Derivative URL
-
-If you're using the `Shrine::Attachment` module, you can use the `#<name>_url`
-method to retrieve the URL of a derivative.
-
-```rb
-class Photo < Model(:image_data)
-  include ImageUploader::Attachment(:image)
-end
-```
-```rb
-photo.image_url(:small)  #=> "https://example.com/small.jpg"
-photo.image_url(:medium) #=> "https://example.com/medium.jpg"
-photo.image_url(:large)  #=> "https://example.com/large.jpg"
-```
-
-For nested derivatives you can pass multiple keys:
-
-```rb
-photo.image_derivatives #=> { thumbnail: { small: ..., medium: ..., large: ... } }
-
-photo.image_url(:thumbnail, :medium) #=> "https://example.com/medium.jpg"
-```
-
-By default, `#<name>_url` method will return `nil` if derivative is not found.
-You can use the [`default_url`][default_url] plugin to set up URL fallbacks:
-
-```rb
-Attacher.default_url do |derivative: nil, **|
-  "https://my-app.com/fallbacks/#{derivative}.jpg" if derivative
-end
-```
-```rb
-photo.image_url(:medium) #=> "https://example.com/fallbacks.com/medium.jpg"
-```
-
-Any additional URL options passed to `#<name>_url` will be forwarded to the
-storage:
-
-```rb
-photo.image_url(:small, response_content_disposition: "attachment")
-```
-
-You can also retrieve the derivative URL via `UploadedFile#url`:
-
-```rb
-photo.image_derivatives[:large].url
-# or
-attacher.derivatives[:large].url
-```
-
 ## Processing derivatives
 
 A derivatives processor block takes the original file, and is expected to
 return a hash of processed files (it can be [nested](#nesting-derivatives)).
 
 ```rb
-Attacher.derivatives_processor :my_processor do |original|
+Attacher.derivatives :my_processor do |original|
   # return a hash of processed files
 end
 ```
@@ -327,9 +359,10 @@ The processor block is evaluated in context of the `Shrine::Attacher` instance,
 which allows you to change your processing logic based on the record data.
 
 ```rb
-Attacher.derivatives_processor :my_processor do |original|
+Attacher.derivatives :my_processor do |original|
   self    #=> #<Shrine::Attacher>
 
+  file    #=> #<Shrine::UploadedFile>
   record  #=> #<Photo>
   name    #=> :image
   context #=> { ... }
@@ -345,7 +378,7 @@ forwarded to the processor:
 attacher.process_derivatives(:my_processor, foo: "bar")
 ```
 ```rb
-Attacher.derivatives_processor :my_processor do |original, **options|
+Attacher.derivatives :my_processor do |original, **options|
   options #=> { :foo => "bar" }
   # ...
 end
@@ -357,7 +390,7 @@ By default, the `Attacher#process_derivatives` method will download the
 attached file and pass it to the processor:
 
 ```rb
-Attacher.derivatives_processor :my_processor do |original|
+Attacher.derivatives :my_processor do |original|
   original #=> #<File:...>
   # ...
 end
@@ -366,29 +399,54 @@ end
 attacher.process_derivatives(:my_processor) # downloads attached file and passes it to the processor
 ```
 
-If you want to use a different source file, or if you're calling multiple
-processors in a row and want to avoid re-downloading the same source file each
-time, you can pass the source file as the second argument:
+If you want to use a different source file, you can pass it in to the process
+call. Typically you'd pass a local file on disk. If you pass a
+`Shrine::UploadedFile` object or another IO-like object, it will be
+automatically downloaded/copied to a local TempFile on disk.
+
+```rb
+# named processor:
+attacher.process_derivatives(:my_processor, source_file)
+
+# default processor:
+attacher.process_derivatives(source_file)
+```
+
+If you want to call multiple processors in a row with the same source file, you
+can use this to avoid re-downloading the same source file each time:
 
 ```rb
-# this way the source file is downloaded only once
 attacher.file.download do |original|
   attacher.process_derivatives(:thumbnails, original)
   attacher.process_derivatives(:colors,     original)
 end
 ```
 
+If a processor might not always need a local source file, you avoid a
+potentially expensive download/copy by registering the processor with
+`download: false`, in which case the source file will be passed to the
+processor as is.
+
+```rb
+Attacher.derivatives :my_processor, download: false do |source|
+  source #=> Could be File, Shrine::UploadedFile, or other IO-like object
+  shrine_class.with_file(source) do |file|
+    # can force download/copy if necessary with `with_file`,
+  end
+end
+```
+
 ## Adding derivatives
 
 If you already have processed files that you want to save, you can do that with
 `Attacher#add_derivatives`:
 
 ```rb
-attacher.add_derivatives(
+attacher.add_derivatives({
   one: file_1,
   two: file_2,
   # ...
-)
+})
 
 attacher.derivatives #=>
 # {
@@ -402,7 +460,7 @@ New derivatives will be merged with existing ones:
 
 ```rb
 attacher.derivatives #=> { one: #<Shrine::UploadedFile> }
-attacher.add_derivatives(two: two_file)
+attacher.add_derivatives({ two: two_file })
 attacher.derivatives #=> { one: #<Shrine::UploadedFile>, two: #<Shrine::UploadedFile> }
 ```
 
@@ -410,7 +468,7 @@ The merging is deep, so the following will work as well:
 
 ```rb
 attacher.derivatives #=> { nested: { one: #<Shrine::UploadedFile> } }
-attacher.add_derivatives(nested: { two: two_file })
+attacher.add_derivatives({ nested: { two: two_file } })
 attacher.derivatives #=> { nested: { one: #<Shrine::UploadedFile>, two: #<Shrine::UploadedFile> } }
 ```
 
@@ -421,6 +479,11 @@ For adding a single derivative, you can also use the singular
 attacher.add_derivative(:thumb, thumbnail_file)
 ```
 
+> Note that new derivatives will replace any existing derivatives living under
+the same key, but won't delete them. If this is your case, make sure to save a
+reference to the old derivatives before assigning new ones, and then delete
+them after persisting the change.
+
 Any options passed to `Attacher#add_derivative(s)` will be forwarded to
 [`Attacher#upload_derivatives`](#uploading-derivatives).
 
@@ -437,11 +500,11 @@ If you want to upload processed files without setting them, you can use
 `Attacher#upload_derivatives`:
 
 ```rb
-derivatives = attacher.upload_derivatives(
+derivatives = attacher.upload_derivatives({
   one: file_1,
   two: file_2,
   # ...
-)
+})
 
 derivatives #=>
 # {
@@ -479,9 +542,7 @@ attacher.upload_derivative :thumb, thumbnail_file,
   location: "path/to/derivative"
 ```
 
-A `:derivative` option is automatically passed to the uploader and holds the
-name of the derivative, which you can use when extracting metadata, generating
-location or generating upload options:
+The `:derivative` name is automatically passed to the uploader:
 
 ```rb
 class MyUploader < Shrine
@@ -510,8 +571,6 @@ If you want to disable this behaviour, pass `delete: false`:
 
 ```rb
 attacher.upload_derivative(:thumb, thumbnail_file, delete: false)
-
-File.exist?(thumbnail_file.path) #=> true
 ```
 
 ## Merging derivatives
@@ -521,7 +580,7 @@ If you want to save already uploaded derivatives, you can use
 
 ```rb
 attacher.derivatives #=> { one: #<Shrine::UploadedFile> }
-attacher.merge_derivatives attacher.upload_derivatives(two: two_file)
+attacher.merge_derivatives attacher.upload_derivatives({ two: two_file })
 attacher.derivatives #=> { one: #<Shrine::UploadedFile>, two: #<Shrine::UploadedFile> }
 ```
 
@@ -529,10 +588,15 @@ This does a deep merge, so the following will work as well:
 
 ```rb
 attacher.derivatives #=> { nested: { one: #<Shrine::UploadedFile> } }
-attacher.merge_derivatives attacher.upload_derivatives(nested: { two: two_file })
+attacher.merge_derivatives attacher.upload_derivatives({ nested: { two: two_file } })
 attacher.derivatives #=> { nested: { one: #<Shrine::UploadedFile>, two: #<Shrine::UploadedFile> } }
 ```
 
+> Note that new derivatives will replace any existing derivatives living under
+the same key, but won't delete them. If this is your case, make sure to save a
+reference to the old derivatives before assigning new ones, and then delete
+them after persisting the change.
+
 The `Attacher#merge_derivatives` method is thread-safe.
 
 ### Setting derivatives
@@ -542,7 +606,7 @@ If instead of adding you want to *override* existing derivatives, you can use
 
 ```rb
 attacher.derivatives #=> { one: #<Shrine::UploadedFile> }
-attacher.set_derivatives attacher.upload_derivatives(two: two_file)
+attacher.set_derivatives attacher.upload_derivatives({ two: two_file })
 attacher.derivatives #=> { two: #<Shrine::UploadedFile> }
 ```
 
@@ -650,7 +714,7 @@ You can store derivatives even if there is no main attached file:
 
 ```rb
 attacher.file #=> nil
-attacher.add_derivatives(one: one_file, two: two_file)
+attacher.add_derivatives({ one: one_file, two: two_file })
 attacher.data #=>
 # {
 #   "derivatives" => {
@@ -728,15 +792,17 @@ plugin :derivatives
 Processing derivatives will trigger a `derivatives.shrine` event with the
 following payload:
 
-| Key                  | Description                            |
-| :--                  | :----                                  |
-| `:processor`         | Name of the derivatives processor      |
-| `:processor_options` | Any options passed to the processor    |
-| `:uploader`          | The uploader class that sent the event |
+| Key                  | Description                                |
+| :--                  | :----                                      |
+| `:processor`         | Name of the derivatives processor          |
+| `:processor_options` | Any options passed to the processor        |
+| `:io`                | The source file passed to the processor    |
+| `:attacher`          | The attacher instance doing the processing |
+| `:uploader`          | The uploader class that sent the event     |
 
 A default log subscriber is added as well which logs these events:
 
-```plaintext
+```
 Derivatives (2133ms) – {:processor=>:thumbnails, :processor_options=>{}, :uploader=>ImageUploader}
 ```
 
@@ -747,8 +813,8 @@ plugin :derivatives, log_subscriber: -> (event) {
   Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, **event.payload)
 }
 ```
-```plaintext
-{"name":"derivatives","duration":2133,"processor":"thumbnails","processor_options":{},"uploader":"ImageUploader"}
+```
+{"name":"derivatives","duration":2133,"processor":"thumbnails","processor_options":{},"io":"#<File:...>","uploader":"ImageUploader"}
 ```
 
 Or disable logging altogether:
@@ -757,6 +823,7 @@ Or disable logging altogether:
 plugin :derivatives, log_subscriber: nil
 ```
 
+[derivatives]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/derivatives.rb
 [default_url]: https://shrinerb.com/docs/plugins/default_url
 [entity]: https://shrinerb.com/docs/plugins/entity
 [model]: https://shrinerb.com/docs/plugins/model