shrine 3.0.0.beta2 → 3.0.0.beta3

data/doc/multiple_files.md

@@ -104,12 +104,12 @@ In the Photo model, create a Shrine attachment attribute named `image`
 ```rb
 # with Sequel:
 class Photo < Sequel::Model
-  include ImageUploader::Attachment.new(:image)
+  include ImageUploader::Attachment(:image)
 end
 
 # with Active Record:
 class Photo < ActiveRecord::Base
-  include ImageUploader::Attachment.new(:image)
+  include ImageUploader::Attachment(:image)
 end
 ```
 
@@ -257,7 +257,7 @@ class ImageUploader < Shrine
 
   Attacher.validate do
     validate_max_size 10*1024*1024
-    validate_mime_type_inclusion %w[image/jpeg image/png]
+    validate_mime_type %w[image/jpeg image/png image/webp]
   end
 end
 ```

data/doc/paperclip.md

@@ -65,7 +65,7 @@ class ImageUploader < Shrine
 end
 
 class Photo < ActiveRecord::Base
-  include ImageUploader::Attachment.new(:image)
+  include ImageUploader::Attachment(:image)
 end
 ```
 
@@ -81,8 +81,9 @@ uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/store/kfds0lg9rer.jpg"
 ### Processing
 
 In contrast to Paperclip's static options, in Shrine you define and perform
-processing on instance-level. The result of processing can be a single file
-or a hash of versions:
+processing on instance-level. You also have the descriptive method names
+provided by the [image_processing] gem.
+
 
 ```rb
 class Photo < ActiveRecord::Base
@@ -99,34 +100,26 @@ end
 require "image_processing/mini_magick"
 
 class ImageUploader < Shrine
-  plugin :processing
-  plugin :versions
-
-  process(:store) do |io, context|
-    versions = { original: io } # retain original
-
-    io.download do |original|
-      pipeline = ImageProcessing::MiniMagick.source(original)
+  plugin :derivatives
 
-      versions[:large]  = pipeline.resize_to_limit!(800, 800)
-      versions[:medium] = pipeline.resize_to_limit!(500, 500)
-      versions[:small]  = pipeline.resize_to_limit!(300, 300)
-    end
+  Attacher.derivatives_processor do |original|
+    magick = ImageProcessing::MiniMagick.source(original)
 
-    versions # return the hash of processed files
+    {
+      large:  magick.resize_to_limit!(800, 800),
+      medium: magick.resize_to_limit!(500, 500),
+      small:  magick.resize_to_limit!(300, 300),
+    }
   end
 end
 ```
 
-This allows you to fully optimize processing, because you can easily specify
-which files are processed from which, and even add parallelization.
-
 #### Reprocessing versions
 
 Shrine doesn't have a built-in way of regenerating versions, because that has
 to be written and optimized differently depending on whether you're adding or
 removing a version, what ORM are you using, how many records there are in the
-database etc. The [Reprocessing versions] guide provides some useful tips on
+database etc. The [Managing Derivatives] guide provides some useful tips on
 this task.
 
 ### Validations
@@ -148,8 +141,8 @@ class ImageUploader < Shrine
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_mime_type_inclusion %w[image/jpeg image/gif image/png]
-    validate_max_size 10*1024*1024 unless record.admin?
+    validate_mime_type %w[image/jpeg image/gif image/png]
+    validate_max_size 10*1024*1024
   end
 end
 ```
@@ -212,7 +205,7 @@ gives your models similar set of methods that Paperclip gives:
 
 ```rb
 class Photo < Sequel::Model
-  include ImageUploader::Attachment.new(:image)
+  include ImageUploader::Attachment(:image)
 end
 ```
 
@@ -243,11 +236,11 @@ Unlike Paperclip, Shrine will store this information for each processed
 version, making them first-class citizens:
 
 ```rb
-photo.image[:original]       #=> #<Shrine::UploadedFile>
-photo.image[:original].width #=> 800
+photo.image               #=> #<Shrine::UploadedFile>
+photo.image.width         #=> 800
 
-photo.image[:thumb]          #=> #<Shrine::UploadedFile>
-photo.image[:thumb].width    #=> 300
+photo.image(:thumb)       #=> #<Shrine::UploadedFile>
+photo.image(:thumb).width #=> 300
 ```
 
 Also, since Paperclip stores only the filename, it has to recalculate the full
@@ -256,28 +249,6 @@ to move files to a new location, because changing how the location is generated
 will now cause incorrect URLs to be generated for all existing files. Shrine
 calculates the whole location only once and saves it to the column.
 
-### Hooks/Callbacks
-
-Shrine's `hooks` plugin provides callbacks for Shrine, so to get Paperclip's
-`(before|after)_post_process`, you can override `#before_process` and
-`#after_process` methods:
-
-```rb
-class ImageUploader < Shrine
-  plugin :hooks
-
-  def before_process(io, context)
-    # ...
-    super
-  end
-
-  def after_process(io, context)
-    super
-    # ...
-  end
-end
-```
-
 ## Migrating from Paperclip
 
 You have an existing app using Paperclip and you want to transfer it to Shrine.
@@ -292,6 +263,17 @@ Afterwards we need to make new uploads write to the `image_data` column. This
 can be done by including the below module to all models that have Paperclip
 attachments:
 
+```rb
+require "shrine"
+
+Shrine.storages = {
+  cache: ...,
+  store: ...,
+}
+
+Shrine.plugin :model
+Shrine.plugin :derivatives
+```
 ```rb
 module PaperclipShrineSynchronization
   def self.included(model)
@@ -304,49 +286,53 @@ module PaperclipShrineSynchronization
 
   def write_shrine_data(name)
     attachment = send(name)
+    attacher   = Shrine::Attacher.from_model(self, name)
 
     if attachment.size.present?
-      data = attachment_to_shrine_data(attachment)
+      attacher.set shrine_file(attachment)
 
-      if attachment.styles.any?
-        data = {original: data}
-        attachment.styles.each do |name, style|
-          data[name] = style_to_shrine_data(style)
-        end
+      attachment.styles.each do |name, style|
+        attacher.merge_derivatives(name => shrine_file(style))
       end
-
-      write_attribute(:"#{name}_data", data.to_json)
     else
-      write_attribute(:"#{name}_data", nil)
+      attacher.set nil
     end
   end
 
   private
 
+  def shrine_file(object)
+    if object.is_a?(Paperclip::Attachment)
+      shrine_attachment_file(object)
+    else
+      shrine_style_file(object)
+    end
+  end
+
   # If you'll be using a `:prefix` on your Shrine storage, or you're storing
   # files on the filesystem, make sure to subtract the appropriate part
   # from the path assigned to `:id`.
-  def attachment_to_shrine_data(attachment)
-    {
-      storage: :store,
-      id: attachment.path,
+  def shrine_attachment_file(attachment)
+    Shrine.uploaded_file(
+      storage:  :store,
+      id:       attachment.path,
       metadata: {
-        size: attachment.size,
-        filename: attachment.original_filename,
-        mime_type: attachment.content_type,
+        "size"      => attachment.size,
+        "filename"  => attachment.original_filename,
+        "mime_type" => attachment.content_type,
       }
-    }
+    )
   end
 
   # If you'll be using a `:prefix` on your Shrine storage, or you're storing
   # files on the filesystem, make sure to subtract the appropriate part
   # from the path assigned to `:id`.
   def style_to_shrine_data(style)
-    {
-      storage: :store,
-      id: style.attachment.path(style.name),
-      metadata: {}
-    }
+    Shrine.uploaded_file(
+      storage:  :store,
+      id:       style.attachment.path(style.name),
+      metadata: {},
+    )
   end
 end
 ```
@@ -383,8 +369,8 @@ metadata defined in your Shrine uploader:
 Shrine.plugin :refresh_metadata
 
 Photo.find_each do |photo|
-  attachment = ImageUploader.uploaded_file(photo.image, &:refresh_metadata!)
-  photo.update(image_data: attachment.to_json)
+  photo.image_attacher.refresh_metadata!
+  photo.save
 end
 ```
 
@@ -396,8 +382,8 @@ As mentioned above, Shrine's equivalent of `has_attached_file` is including
 an attachment module:
 
 ```rb
-class User < Sequel::Model
-  include ImageUploader::Attachment.new(:avatar) # adds `avatar`, `avatar=` and `avatar_url` methods
+class Photo < Sequel::Model
+  include ImageUploader::Attachment(:image) # adds `image`, `image=` and `image_url` methods
 end
 ```
 
@@ -420,8 +406,23 @@ You can change that for a specific uploader with the `default_storage` plugin.
 
 #### `:styles`, `:processors`, `:convert_options`
 
-As explained in the "Processing" section, processing is done by overriding the
-`Shrine#process` method.
+Processing is defined by using the `derivatives` plugin:
+
+```rb
+class ImageUploader < Shrine
+  plugin :derivatives
+
+  Attacher.derivatives_processor do |original|
+    magick = ImageProcessing::MiniMagick.source(image)
+
+    {
+      large:  magick.resize_to_limit!(800, 800),
+      medium: magick.resize_to_limit!(500, 500),
+      small:  magick.resize_to_limit!(300, 300),
+    }
+  end
+end
+```
 
 #### `:default_url`
 
@@ -443,7 +444,7 @@ Shrine provides a `keep_files` plugin which allows you to keep files that would
 otherwise be deleted:
 
 ```rb
-Shrine.plugin :keep_files, destroyed: true
+Shrine.plugin :keep_files
 ```
 
 #### `:path`, `:url`, `:interpolator`, `:url_generator`
@@ -460,7 +461,7 @@ Alternatively, if you want to generate locations yourself you can override the
 
 ```rb
 class ImageUploader < Shrine
-  def generate_location(io, context)
+  def generate_location(io, **options)
     # ...
   end
 end
@@ -481,17 +482,16 @@ If you're generating versions in Shrine, the attachment will be a hash of
 uploaded files:
 
 ```rb
-user.avatar.class #=> Hash
-user.avatar #=>
+photo.image_derivatives #=>
 # {
 #   small:  #<Shrine::UploadedFile>,
 #   medium: #<Shrine::UploadedFile>,
 #   large:  #<Shrine::UploadedFile>,
 # }
 
-user.avatar[:small].url #=> "..."
+photo.image(:small).url #=> "..."
 # or
-user.avatar_url(:small) #=> "..."
+photo.image_url(:small) #=> "..."
 ```
 
 #### `#path`
@@ -500,12 +500,12 @@ Shrine doesn't have this because storages are abstract and this would be
 specific to the filesystem, but the closest is probably `#id`:
 
 ```rb
-user.avatar.id #=> "users/342/avatar/398543qjfdsf.jpg"
+photo.image.id #=> "photo/342/image/398543qjfdsf.jpg"
 ```
 
 #### `#reprocess!`
 
-Shrine doesn't have an equivalent to this, but the [Reprocessing versions]
+Shrine doesn't have an equivalent to this, but the [Managing Derivatives]
 guide provides some useful tips on how to do this.
 
 ### `Paperclip::Storage::S3`
@@ -532,7 +532,7 @@ Shrine::Storage::S3.new(
 The object data can be configured via the `:upload_options` hash:
 
 ```rb
-Shrine::Storage::S3.new(upload_options: {content_disposition: "attachment"}, **options)
+Shrine::Storage::S3.new(upload_options: { content_disposition: "attachment" }, **options)
 ```
 
 You can use the `upload_options` plugin to set upload options dynamically.
@@ -542,7 +542,7 @@ You can use the `upload_options` plugin to set upload options dynamically.
 The object permissions can be configured with the `:acl` upload option:
 
 ```rb
-Shrine::Storage::S3.new(upload_options: {acl: "private"}, **options)
+Shrine::Storage::S3.new(upload_options: { acl: "private" }, **options)
 ```
 
 You can use the `upload_options` plugin to set upload options dynamically.
@@ -581,6 +581,7 @@ The Shrine storage has no replacement for the `:url` Paperclip option, and it
 isn't needed.
 
 [file]: http://linux.die.net/man/1/file
-[Reprocessing versions]: /doc/regenerating_versions.md#readme
+[Managing Derivatives]: /doc/changing_derivatives.md#readme
 [direct S3 uploads]: /doc/direct_s3.md#readme
 [`Shrine::Storage::S3`]: /doc/storage/s3.md#readme
+[image_processing]: https://github.com/janko/image_processing

data/doc/plugins/activerecord.md

@@ -74,18 +74,9 @@ photo.image.exists? #=> false
 
 #### Caveats
 
-Active Record versions prior to 5.x silence errors that occur in callbacks,
-which can make debugging more difficult, so it's recommended that you disable
-this behaviour:
-
-```rb
-# This is the default in ActiveRecord 5
-ActiveRecord::Base.raise_in_transactional_callbacks = true
-```
-
-Active Record also currently has a [bug with transaction callbacks], so if
-you have any "after commit" callbacks, make sure to include Shrine's attachment
-module *after* they have all been defined.
+Active Record currently has a [bug with transaction callbacks], so if you have
+any "after commit" callbacks, make sure to include Shrine's attachment module
+*after* they have all been defined.
 
 #### Skipping Callbacks
 

data/doc/plugins/backgrounding.md

@@ -1,150 +1,176 @@
 # Backgrounding
 
 The [`backgrounding`][backgrounding] plugin enables you to move promoting and
-deleting of files from record's lifecycle into background jobs. This is
-especially useful if you're doing processing and/or you're storing files on an
-external storage service.
-
-The plugin provides `Attacher.promote` and `Attacher.delete` methods, which
-allow you to hook up to promoting and deleting and spawn background jobs, by
-passing a block.
+deleting of files into background jobs. This is especially useful if you're
+processing [derivatives] and storing files to a remote storage service.
 
 ```rb
-Shrine.plugin :backgrounding
-
-# makes all uploaders use background jobs
-Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
-Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
+Shrine.plugin :backgrounding # load the plugin globally
 ```
 
-If you don't want to apply backgrounding for all uploaders, you can declare the
-hooks only for specific uploaders (in this case it's still recommended to keep
-the plugin loaded globally).
+The plugin provides `Attacher.promote_block` and `Attacher.destroy_block`
+methods, which allow you to register blocks that will get executed in place of
+synchronous promotion and deletion. Inside them you can spawn your background
+jobs:
 
 ```rb
-class MyUploader < Shrine
-  # makes this uploader use background jobs
-  Attacher.promote { |data| PromoteJob.perform_async(data) }
-  Attacher.delete { |data| DeleteJob.perform_async(data) }
-end
+# register backgrounding blocks for all uploaders
+Shrine::Attacher.promote_block { PromoteJob.perform_later(self.class, record, name, file_data) }
+Shrine::Attacher.destroy_block { DestroyJob.perform_later(self.class, data) }
 ```
-
-The yielded `data` variable is a serializable hash containing all context
-needed for promotion/deletion. Now you just need to declare the job classes,
-and inside them call `Attacher.promote` or `Attacher.delete`, this time with
-the received data.
-
 ```rb
-class PromoteJob
-  include Sidekiq::Worker
-  def perform(data)
-    Shrine::Attacher.promote(data)
+class PromoteJob < ActiveJob::Base
+  def perform(attacher_class, record, name, file_data)
+    attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
+    attacher.atomic_promote
+  rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
+    # attachment has changed or record has been deleted, nothing to do
   end
 end
-
-class DeleteJob
-  include Sidekiq::Worker
-  def perform(data)
-    Shrine::Attacher.delete(data)
+```
+```rb
+class DestroyJob < ActiveJob::Base
+  def perform(attacher_class, data)
+    attacher = attacher_class.from_data(data)
+    attacher.destroy
   end
 end
 ```
 
-This example used Sidekiq, but obviously you could just as well use any other
-backgrounding library. This setup will be applied globally for all uploaders.
+If you don't want to apply backgrounding for all uploaders, you can register
+backgrounding blocks only for a specific uploader:
 
-If you're generating versions, and you want to process some versions in the
-foreground before kicking off a background job, you can use the `recache`
-plugin.
+```rb
+class MyUploader < Shrine
+  # register backgrounding blocks only for this uploader
+  Attacher.promote_block { PromoteJob.perform_later(self.class, record, name, file_data) }
+  Attacher.destroy_block { DestroyJob.perform_later(self.class, data) }
+end
+```
 
-In your application you can use `Attacher#cached?` and `Attacher#stored?`
-to differentiate between your background job being in progress and
-having completed.
+Backgrounding will automatically get triggered as part of your attachment flow
+if you're using `Shrine::Attachment` with a persistence plugin such as
+`activerecord` or `sequel`:
 
 ```rb
-if user.avatar_attacher.cached? # background job is still in progress
-  # ...
-elsif user.avatar_attacher.stored? # background job has finished
-  # ...
-end
+photo = Photo.new
+photo.image = file
+photo.save    # spawns promote job
+photo.destroy # spawns destroy job
 ```
 
-## `Attacher.promote` and `Attacher.delete`
+In terms of `Shrine::Attacher`, the background jobs are spawned on
+`Attacher#promote_cached` (called on `Attacher#finalize`) and
+`Attacher#destroy_attached`:
 
-In background jobs, `Attacher.promote` and `Attacher.delete` will resolve all
-necessary objects, and do the promotion/deletion. If `Attacher.find_record` is
-defined (which comes with ORM plugins), model instances will be treated as
-database records, with the `#id` attribute assumed to represent the primary
-key. Then promotion will have the following behaviour:
+```rb
+attacher.assign(file)
+attacher.finalize         # spawns promote job
+attacher.destroy_attached # spawns destroy job
+```
 
-1. retrieves the database record
-  * if record is not found, it finishes
-  * if record is found but attachment has changed, it finishes
-2. uploads cached file to permanent storage
-3. reloads the database record
-  * if record is not found, it deletes the promoted files and finishes
-  * if record is found but attachment has changed, it deletes the promoted files and finishes
-4. updates the record with the promoted files
+## Promotion
 
-Both `Attacher.promote` and `Attacher.delete` return a `Shrine::Attacher`
-instance (if the action hasn't aborted), so you can use it to perform
-additional tasks:
+While background deletion acts only on file data, background promotion is more
+complex as it deals with persistence and concurrency safety:
 
 ```rb
-def perform(data)
-  attacher = Shrine::Attacher.promote(data)
-  attacher.record.update(published: true) if attacher && attacher.record.is_a?(Post)
-end
+attacher = Shrine::Attacher.retrieve(model: record, name: name, file: file_data)
+attacher.atomic_promote
 ```
 
-### Plain models
+The `Attacher.retrieve` and `Attacher#atomic_promote` methods are provided by
+the [`atomic_helpers`][atomic_helpers] plugin, which is automatically loaded by
+your persistence plugin (`activerecord`, `sequel`). They add concurrency safety
+by verifying that the attachment hasn't changed on the outside before or after
+promotion.
 
-You can also do backgrounding with plain models which don't represent database
-records; the plugin will use that mode if `Attacher.find_record` is not
-defined. In that case promotion will have the following behaviour:
+When we remove the concurrency safety, promotion would look like this:
 
-1. instantiates the model
-2. uploads cached file to permanent storage
-3. writes promoted files to the model instance
+```rb
+attacher = record.send(:"#{name}_attacher")
+attacher.promote
+attacher.persist
+```
+
+## Backgrounding blocks
 
-You can then retrieve the promoted files via the attacher object that
-`Attacher.promote` returns, and do any additional tasks if you need to.
+The blocks registered by `Attacher.promote_block` and `Attacher#destroy_block`
+are by default evaluated in context of a `Shrine::Attacher` instance. You can
+also use the explicit version by declaring an attacher argument:
+
+```rb
+Shrine::Attacher.promote_block do |attacher|
+  PromoteJob.perform_later(
+    attacher.class,
+    attacher.record,
+    attacher.name,
+    attacher.file_data,
+  )
+end
 
-## `Attacher#_promote` and `Attacher#_delete`
+Shrine::Attacher.destroy_block do |attacher|
+  DestroyJob.perform_later(
+    attacher.class,
+    attacher.data,
+  )
+end
+```
 
-The plugin modifies `Attacher#_promote` and `Attacher#_delete` to call the
-registered blocks with serializable attacher data, and these methods are
-internally called by the attacher. `Attacher#promote` and `Attacher#delete!`
-remain synchronous.
+You can also register backgrounding blocks on attacher *instances* for more
+flexibility:
 
 ```rb
-# asynchronous (spawn background jobs)
-attacher._promote
-attacher._delete(attachment)
+photo.image_attacher.promote_block do |attacher|
+  PromoteJob.perform_later(
+    attacher.class,
+    attacher.record,
+    attacher.name,
+    attacher.file_data,
+    current_user.id, # pass arguments known at the controller level
+  )
+end
 
-# synchronous
-attacher.promote
-attacher.delete!(attachment)
+photo.image = file
+photo.save # executes the promote block above
 ```
 
-## `Attacher.dump` and `Attacher.load`
+## Other backgrounding libraries
 
-The plugin adds `Attacher.dump` and `Attacher.load` methods for serializing
-attacher object and loading it back up. You can use them to spawn background
-jobs for custom tasks.
+If you're not using Active Job for backgrounding, you might need to retrieve
+records and resolve constants from job arguments manually:
 
 ```rb
-data = Shrine::Attacher.dump(attacher)
-SomethingJob.perform_async(data)
+# register backgrounding blocks for all uploaders
+Shrine::Attacher.promote_block { PromoteJob.perform_async(self.class, record.class, record.id, name, file_data) }
+Shrine::Attacher.destroy_block { DestroyJob.perform_async(self.class, data) }
+```
+```rb
+class PromoteJob
+  include Sidekiq::Worker
+
+  def perform(attacher_class, record_class, record_id, name, file_data)
+    attacher_class = Object.const_get(attacher_class)
+    record         = Object.const_get(record_class).find(record_id) # if using Active Record
+
+    attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
+    attacher.atomic_promote
+  rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
+  end
+end
+
+class DestroyJob
+  include Sidekiq::Worker
 
-# ...
+  def perform(attacher_class, data)
+    attacher_class = Object.const_get(attacher_class)
 
-class SomethingJob
-  def perform(data)
-    attacher = Shrine::Attacher.load(data)
-    # ...
+    attacher = attacher_class.from_data(data)
+    attacher.destroy
   end
 end
 ```
 
 [backgrounding]: /lib/shrine/plugins/backgrounding.rb
+[derivatives]: /doc/plugins/derivatives.md#readme
+[atomic_helpers]: /doc/plugins/atomic_helpers.md#readme