shrine 2.19.4 → 3.0.0.alpha

data/doc/plugins/presign_endpoint.md

@@ -130,7 +130,8 @@ option:
 
 ```rb
 plugin :presign_endpoint, presign: -> (id, options, request) do
-  # return a Hash with :url, :fields, and :headers keys
+  # return a Hash with :method, :url, :fields, and :headers keys
+  Shrine.storages[:cache].presign(id, options)
 end
 ```
 

data/doc/plugins/refresh_metadata.md

@@ -7,27 +7,64 @@ metadata from an uploaded file.
 plugin :refresh_metadata
 ```
 
-It provides `UploadedFile#refresh_metadata!` method, which triggers metadata
-extraction (calls `Shrine#extract_metadata`) with the uploaded file opened for
-reading, and updates the existing metadata hash with the results.
+It provides `#refresh_metadata!` method, which triggers metadata extraction
+(calls `Shrine#extract_metadata`) with the uploaded file opened for reading,
+and updates the existing metadata hash with the results. This can be done
+on the attacher or the uploaded file level.
+
+## Attacher
+
+Calling `#refresh_metadata!` on a `Shrine::Attacher` object will re-extract
+metadata of the attached file. When used with a [model], it will write new file
+data back into the attachment attribute.
+
+```rb
+attacher.refresh_metadata!
+attacher.file.metadata # re-extracted metadata
+```
+
+The `Attacher#context` hash will be forwarded to metadata extraction, as well
+as any options that you pass in.
+
+```rb
+# via context
+attacher.context[:foo] = "bar"
+attacher.refresh_metadata! # passes `{ foo: "bar" }` options to metadata extraction
+
+# via arguments
+attacher.refresh_metadata!(foo: "bar") # passes `{ foo: "bar" }` options to metadata extraction
+```
+
+## Uploaded File
+
+The `#refresh_metadata!` method can be called on a `Shrine::UploadedFile` object
+as well.
 
 ```rb
 uploaded_file.refresh_metadata!
 uploaded_file.metadata # re-extracted metadata
 ```
 
-For remote storages this will make an HTTP request to open the file for
-reading, but only the portion of the file needed for extracting each metadata
-value will be downloaded.
+If the uploaded file is not open, it is opened before and closed after metadata
+extraction. For remote storage services this will make an HTTP request.
+However, only the portion of the file needed for extracting metadata will be
+downloaded.
 
 If the uploaded file is already open, it is passed to metadata extraction as
 is.
 
 ```rb
 uploaded_file.open do
-  uploaded_file.refresh_metadata!
+  uploaded_file.refresh_metadata! # uses the already opened file
   # ...
 end
 ```
 
+Any options passed in will be forwarded to metadata extraction:
+
+```rb
+uploaded_file.refresh_metadata!(foo: "bar") # passes `{ foo: "bar" }` options to metadata extraction
+```
+
 [refresh_metadata]: /lib/shrine/plugins/refresh_metadata.rb
+[model]: /doc/plugins/model.md#readme

data/doc/plugins/sequel.md

@@ -1,67 +1,224 @@
 # Sequel
 
-The [`sequel`][sequel] plugin extends the "attachment" interface with support
-for Sequel.
+The [`sequel`][sequel] plugin adds [Sequel] integration to the attachment
+interface. It is built on top of the [`model`][model] plugin.
 
 ```rb
 plugin :sequel
 ```
 
-## Callbacks
+## Attachment
 
-Now the attachment module will add additional callbacks to the model:
+When `Shrine::Attachment` module is included into a `Sequel::Model` subclass,
+additional [hooks] are added to tie the attachment process to the record
+lifecycle.
 
-* "before save" – Used by the `recache` plugin.
-* "after commit" (save) – Promotes the attachment, deletes replaced ones.
-* "after commit" (destroy) – Deletes the attachment.
+```rb
+class Photo < Sequel::Model
+  include ImageUploader::Attachment(:image) # adds callbacks & validations
+end
+```
 
-If you want to put promoting/deleting into a background job, see the
-`backgrounding` plugin.
+### Callbacks
 
-Since attaching first saves the record with a cached attachment, then saves
-again with a stored attachment, you can detect this in callbacks:
+#### Save
+
+After a record is saved and the transaction is committed, `Attacher#finalize`
+is called, which promotes cached file to permanent storage and deletes previous
+file if any.
 
 ```rb
-class User < Sequel::Model
-  include ImageUploader::Attachment.new(:avatar)
+photo = Photo.new
 
-  def before_save
-    super
+photo.image = file
+photo.image.storage_key #=> :cache
 
-    if changed_columns.include?(:avatar) && avatar_attacher.cached?
-      # cached
-    elsif changed_columns.include?(:avatar) && avatar_attacher.stored?
-      # promoted
-    end
-  end
-end
+photo.save
+photo.image.storage_key #=> :store
 ```
 
-If you don't want the attachment module to add any callbacks to the model, and
-would instead prefer to call these actions manually, you can disable callbacks:
+#### Destroy
+
+After a record is destroyed and the transaction is committed,
+`Attacher#destroy_attached` method is called, which deletes stored attached
+file if any.
+
+```rb
+photo = Photo.find(photo_id)
+photo.image #=> #<Shrine::UploadedFile>
+photo.image.exists? #=> true
+
+photo.destroy
+photo.image.exists? #=> false
+```
+
+#### Skipping
+
+If you don't want the attachment module to add any callbacks to your Sequel
+model, you can set `:callbacks` to `false`:
 
 ```rb
 plugin :sequel, callbacks: false
 ```
 
-## Validations
+### Validations
+
+If you're using the [`validation`][validation] plugin, the attachment module
+will automatically merge attacher errors with model errors.
+
+```rb
+class ImageUploader < Shrine
+  plugin :validation_helpers
+
+  Attacher.validate do
+    validate_max_size 10 * 1024 * 1024
+  end
+end
+```
+```rb
+photo = Photo.new
+photo.image = file
+photo.valid?
+photo.errors #=> { image: ["size must not be greater than 10.0 MB"] }
+```
+
+#### Presence
 
-Additionally, any Shrine validation errors will added to Sequel's errors upon
-validation. Note that if you want to validate presence of the attachment, you
-can do it directly on the model.
+If you want to validate presence of the attachment, you can use Sequel's
+presence validator:
 
 ```rb
-class User < Sequel::Model
-  include ImageUploader::Attachment.new(:avatar)
-  validates_presence_of :avatar
+class Photo < Sequel::Model
+  include ImageUploader::Attachment.new(:image)
+
+  plugin :validation_helpers
+
+  def validate
+    super
+    validates_presence :image
+  end
 end
 ```
 
-If don't want the attachment module to merge file validations errors into model
-errors, you can disable it:
+#### Skipping
+
+If don't want the attachment module to merge file validations errors into
+model errors, you can set `:validations` to `false`:
 
 ```rb
 plugin :sequel, validations: false
 ```
 
+## Attacher
+
+This section will cover methods added to the `Shrine::Attacher` instance. If
+you're not familar with how to obtain it, see the [`model`][model] plugin docs.
+
+### Atomic promotion
+
+If you're promoting cached file to permanent storage
+[asynchronously][backgrounding], you might want to handle the possibility of
+the attachment changing during promotion. You can do that with
+`Attacher#atomic_promote`:
+
+```rb
+# in your controller
+attacher.attach_cached(io)
+attacher.cached? #=> true
+```
+```rb
+# in a background job
+attacher.atomic_promote # promotes cached file and persists
+attacher.stored? #=> true
+```
+
+After cached file is uploaded to permanent storage, the record is reloaded in
+order to check whether the attachment hasn't changed, and if it hasn't the
+attachment is persisted. If the attachment has changed,
+`Shrine::AttachmentChanged` exception is raised.
+
+Additional options are passed to `Attacher#promote`.
+
+#### Reloader & persister
+
+You can change how the record is reloaded or persisted during atomic promotion:
+
+```rb
+# reloader
+attacher.atomic_promote(reload: :lock)    # uses database locking (default)
+attacher.atomic_promote(reload: :fetch)   # reloads with no locking
+attacher.atomic_promote(reload: ->(&b){}) # custom reloader (see atomic_helpers plugin docs)
+attacher.atomic_promote(reload: false)    # skips reloading
+
+# persister
+attacher.atomic_promote(persist: :save) # persists stored file (default)
+attacher.atomic_promote(persist: ->{})  # custom persister (see atomic_helpers plugin docs)
+attacher.atomic_promote(persist: false) # skips persistence
+```
+
+For more details, see the [`atomic_helpers`][atomic_helpers] plugin docs.
+
+### Atomic persistence
+
+If you're updating something based on the attached file
+[asynchronously][backgrounding], you might want to handle the possibility of
+the attachment changing in the meanwhile. You can do that with
+`Attacher#atomic_persist`:
+
+```rb
+# in a background job
+attacher.refresh_metadata! # refresh_metadata plugin
+attacher.atomic_persist # persists attachment data
+```
+
+The record is first reloaded in order to check whether the attachment hasn't
+changed, and if it hasn't the attachment is persisted. If the attachment has
+changed, `Shrine::AttachmentChanged` exception is raised.
+
+#### Reloader & persister
+
+You can change how the record is reloaded or persisted during atomic
+persistence:
+
+```rb
+# reloader
+attacher.atomic_persist(reload: :lock)       # uses database locking (default)
+attacher.atomic_persist(reload: :fetch)      # reloads with no locking
+attacher.atomic_persist(reload: ->(&b){...}) # custom reloader (see atomic_helpers plugin docs)
+attacher.atomic_persist(reload: false)       # skips reloading
+
+# persister
+attacher.atomic_persist(persist: :save)   # persists stored file (default)
+attacher.atomic_persist(persist: ->{...}) # custom persister (see atomic_helpers plugin docs)
+attacher.atomic_persist(persist: false)   # skips persistence
+```
+
+For more details, see the [`atomic_helpers`][atomic_helpers] plugin docs.
+
+### Persistence
+
+You can call `Attacher#persist` to save any changes to the underlying record:
+
+```rb
+attacher.attach(io)
+attacher.persist # saves the underlying record
+```
+
+### With other database plugins
+
+If you have another database plugin loaded together with the `sequel` plugin,
+you can prefix any method above with `sequel_*` to avoid naming clashes:
+
+```rb
+attacher.sequel_atomic_promote
+attacher.sequel_atomic_persist
+attacher.sequel_persist
+```
+
 [sequel]: /lib/shrine/plugins/sequel.rb
+[Sequel]: https://sequel.jeremyevans.net/
+[model]: /doc/plugins/model.md#readme
+[hooks]: http://sequel.jeremyevans.net/rdoc/files/doc/model_hooks_rdoc.html
+[validation]: /doc/plugins/validation.md#readme
+[atomic_helpers]: /doc/plugins/atomic_helpers.md#readme
+[backgrounding]: /doc/plugins/backgrounding.md#readme

data/doc/plugins/{default_url_options.md → url_options.md}

@@ -1,11 +1,11 @@
-# Default URL Options
+# URL Options
 
-The [`default_url_options`][default_url_options] plugin allows you to specify
+The [`url_options`][url_options] plugin allows you to specify
 URL options that will be applied by default for uploaded files of specified
 storages.
 
 ```rb
-plugin :default_url_options, store: { expires_in: 24*60*60 }
+plugin :url_options, store: { expires_in: 24*60*60 }
 ```
 
 You can also generate the default URL options dynamically by using a block,
@@ -13,7 +13,7 @@ which will receive the UploadedFile object along with any options that were
 passed to `UploadedFile#url`.
 
 ```rb
-plugin :default_url_options, store: -> (io, options) do
+plugin :url_options, store: -> (io, options) do
   { response_content_disposition: ContentDisposition.attachment(io.original_filename) }
 end
 ```
@@ -22,4 +22,4 @@ In both cases the default options are merged with options passed to
 `UploadedFile#url`, and the latter will always have precedence over default
 options.
 
-[default_url_options]: /lib/shrine/plugins/default_url_options.rb
+[url_options]: /lib/shrine/plugins/url_options.rb

data/doc/processing.md

@@ -230,7 +230,7 @@ Now you can generate thumbnail URLs from attached files, and the actual
 thumbnail will be generated when the URL is requested:
 
 ```rb
-photo.image.derivation_url(:thumbnail, "600", "400")
+photo.image.derivation_url(:thumbnail, 600, 400)
 #=> "/derivations/image/thumbnail/600/400/eyJpZCI6ImZvbyIsInN0b3JhZ2UiOiJzdG9yZSJ9?signature=..."
 ```
 

data/doc/release_notes/1.1.0.md

@@ -39,11 +39,11 @@ plugin :keep_location, :cache => :store
 ```rb
 user = User.new
 user.avatar = image
-user.avatar.storage_key #=> "cache"
+user.avatar.storage_key #=> :cache
 user.avatar.id #=> "abc123.jpg"
 
 user.save
-user.avatar.storage_key #=> "store"
+user.avatar.storage_key #=> :store
 user.avatar.id #=> "abc123.jpg"
 ```
 

data/doc/release_notes/2.15.0.md

@@ -10,7 +10,7 @@
   "derivation" blocks, passing any arguments you need for the processing.
 
   ```rb
-  photo.image.derivation_url(:thumbnail, "600", "400")
+  photo.image.derivation_url(:thumbnail, 600, 400)
   #=> "derivations/image/thumbnail/600/400/eyJpZCI6ImZvbyIsInN0b3JhZ2UiOiJzdG9yZSJ9?signature=..."
   ```
 

data/doc/storage/s3.md

@@ -131,10 +131,10 @@ s3.url("image.jpg", host: "https://your-s3-host.com/prefix/") # needs to end wit
 ```
 
 To have the `:host` option passed automatically for every URL, use the
-`default_url_options` plugin.
+`url_options` plugin.
 
 ```rb
-plugin :default_url_options, store: { host: "http://abc123.cloudfront.net" }
+plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
 ```
 
 If you would like to [serve private content via CloudFront], you need to sign

data/doc/testing.md

@@ -158,7 +158,7 @@ Regular routing tests in Rails use [Rack::Test], in which case you can create
 `Rack::Test::UploadedFile` objects and pass them as form parameters:
 
 ```rb
-post "/photos", photo: {image: Rack::Test::UploadedFile.new("test/files/image.jpg", "image/jpeg")}
+post "/photos", photo: { image: Rack::Test::UploadedFile.new("test/files/image.jpg", "image/jpeg") }
 ```
 
 ### Rack::TestApp

data/lib/shrine.rb

@@ -20,21 +20,13 @@ class Shrine
   # Raised when a file is not a valid IO.
   class InvalidFile < Error
     def initialize(io, missing_methods)
-      super "#{io.inspect} is not a valid IO object (it doesn't respond to \
-        #{missing_methods.map{|m, _|"##{m}"}.join(", ")})"
+      super "#{io.inspect} is not a valid IO object (it doesn't respond to #{missing_methods.map{|m, _|"##{m}"}.join(", ")})"
     end
   end
 
-  # Methods which an object has to respond to in order to be considered
-  # an IO object, along with their arguments.
-  IO_METHODS = {
-    read:   [:length, :outbuf],
-    eof?:   [],
-    rewind: [],
-    size:   [],
-    close:  [],
-  }
-  deprecate_constant(:IO_METHODS)
+  # Raised by the storage in the #open method.
+  class FileNotFound < Error
+  end
 
   @opts = {}
   @storages = {}
@@ -54,12 +46,7 @@ class Shrine
     # When inheriting Shrine, copy the instance variables into the subclass,
     # and create subclasses of core classes.
     def inherited(subclass)
-      subclass.instance_variable_set(:@opts, opts.dup)
-      subclass.opts.each do |key, value|
-        if value.is_a?(Enumerable) && !value.frozen?
-          subclass.opts[key] = value.dup
-        end
-      end
+      subclass.instance_variable_set(:@opts, deep_dup(opts))
       subclass.instance_variable_set(:@storages, storages.dup)
 
       file_class = Class.new(self::UploadedFile)
@@ -117,25 +104,26 @@ class Shrine
     # Uploads the file to the specified storage. It delegates to `Shrine#upload`.
     #
     #     Shrine.upload(io, :store) #=> #<Shrine::UploadedFile>
-    def upload(io, storage, context = {})
-      new(storage).upload(io, context)
+    def upload(io, storage, **options)
+      new(storage).upload(io, **options)
     end
 
     # Instantiates a Shrine::UploadedFile from a hash, and optionally
     # yields the returned object.
     #
-    #     data = {"storage" => "cache", "id" => "abc123.jpg", "metadata" => {}}
+    #     data = { "storage" => "cache", "id" => "abc123.jpg", "metadata" => {} }
     #     Shrine.uploaded_file(data) #=> #<Shrine::UploadedFile>
-    def uploaded_file(object, &block)
+    def uploaded_file(object)
       case object
       when String
-        uploaded_file(JSON.parse(object), &block)
+        uploaded_file(JSON.parse(object))
       when Hash
-        uploaded_file(self::UploadedFile.new(object), &block)
+        object = JSON.parse(object.to_json) if object.keys.grep(Symbol).any? # deep stringify keys
+        self::UploadedFile.new(object)
       when self::UploadedFile
-        object.tap { |f| yield(f) if block_given? }
+        object
       else
-        raise Error, "cannot convert #{object.inspect} to a #{self}::UploadedFile"
+        fail ArgumentError, "cannot convert #{object.inspect} to a #{self}::UploadedFile"
       end
     end
 
@@ -169,6 +157,21 @@ class Shrine
     def deprecation(message)
       Shrine.logger.warn "SHRINE DEPRECATION WARNING: #{message}"
     end
+
+    private
+
+    # Deep duplicates a nested hash of options.
+    def deep_dup(collection)
+      duplicate_collection = collection.dup
+
+      if duplicate_collection.is_a?(Hash)
+        duplicate_collection.each do |key, value|
+          duplicate_collection[key] = deep_dup(value) if value.is_a?(Enumerable)
+        end
+      end
+
+      duplicate_collection
+    end
   end
 
   module InstanceMethods
@@ -182,16 +185,10 @@ class Shrine
     #
     #     Shrine.new(:store)
     def initialize(storage_key)
-      @storage = self.class.find_storage(storage_key)
+      @storage     = self.class.find_storage(storage_key)
       @storage_key = storage_key.to_sym
     end
 
-    # The class-level options hash. This should probably not be modified at
-    # the instance level.
-    def opts
-      self.class.opts
-    end
-
     # The main method for uploading files. Takes an IO-like object and an
     # optional context hash (used internally by Shrine::Attacher). It calls
     # user-defined #process, and afterwards it calls #store. The `io` is
@@ -201,55 +198,31 @@ class Shrine
     #   uploader.upload(io, metadata: { "foo" => "bar" })           # add metadata
     #   uploader.upload(io, location: "path/to/file")               # specify location
     #   uploader.upload(io, upload_options: { acl: "public-read" }) # add upload options
-    def upload(io, context = {})
-      io = processed(io, context) || io
-      store(io, context)
-    end
-
-    # User is expected to perform processing inside this method, and
-    # return the processed files. Returning nil signals that no proccessing
-    # has been done and that the original file should be used.
-    #
-    #     class ImageUploader < Shrine
-    #       def process(io, context)
-    #         # do processing and return processed files
-    #       end
-    #     end
-    def process(io, context = {})
-    end
+    def upload(io, **options)
+      _enforce_io(io)
 
-    # Uploads the file and returns an instance of Shrine::UploadedFile. By
-    # default the location of the file is automatically generated by
-    # \#generate_location, but you can pass in `:location` to upload to
-    # a specific location.
-    #
-    #     uploader.store(io)
-    def store(io, context = {})
-      _store(io, context)
-    end
+      metadata = get_metadata(io, **options)
+      location = get_location(io, **options, metadata: metadata)
 
-    # Returns true if the storage of the given uploaded file matches the
-    # storage of this uploader.
-    def uploaded?(uploaded_file)
-      uploaded_file.storage_key == storage_key.to_s
-    end
+      _upload(io, **options, location: location, metadata: metadata)
 
-    # Deletes the given uploaded file and returns it.
-    def delete(uploaded_file, context = {})
-      _delete(uploaded_file, context)
-      uploaded_file
+      self.class::UploadedFile.new(
+        id:       location,
+        storage:  storage_key,
+        metadata: metadata,
+      )
     end
 
     # Generates a unique location for the uploaded file, preserving the
     # file extension. Can be overriden in uploaders for generating custom
     # location.
-    def generate_location(io, context = {})
-      basic_location(io, metadata: context[:metadata] || {})
+    def generate_location(io, metadata: {}, **options)
+      basic_location(io, metadata: metadata)
     end
 
     # Extracts filename, size and MIME type from the file, which is later
     # accessible through UploadedFile#metadata.
-    def extract_metadata(io, context = {})
+    def extract_metadata(io, **options)
       {
         "filename"  => extract_filename(io),
         "size"      => extract_size(io),
@@ -257,8 +230,21 @@ class Shrine
       }
     end
 
+    # The class-level options hash. This should probably not be modified at
+    # the instance level.
+    def opts
+      self.class.opts
+    end
+
     private
 
+    def _upload(io, location:, metadata:, upload_options: {}, close: true, delete: false, **)
+      storage.upload(io, location, shrine_metadata: metadata, **upload_options)
+    ensure
+      io.close             if close
+      File.unlink(io.path) if delete && io.respond_to?(:path) && File.exist?(io.path)
+    end
+
     # Attempts to extract the appropriate filename from the IO object.
     def extract_filename(io)
       if io.respond_to?(:original_filename)
@@ -281,61 +267,6 @@ class Shrine
       io.size if io.respond_to?(:size)
     end
 
-    # It first asserts that `io` is a valid IO object. It then extracts
-    # metadata and generates the location, before calling the storage to
-    # upload the IO object, passing the extracted metadata and location.
-    # Finally it returns a Shrine::UploadedFile object which represents the
-    # file that was uploaded.
-    def _store(io, context)
-      _enforce_io(io)
-
-      metadata = get_metadata(io, context)
-      metadata = metadata.merge(context[:metadata]) if context[:metadata].is_a?(Hash)
-
-      location = get_location(io, context.merge(metadata: metadata))
-
-      put(io, context.merge(location: location, metadata: metadata))
-
-      self.class.uploaded_file(
-        "id"       => location,
-        "storage"  => storage_key.to_s,
-        "metadata" => metadata,
-      )
-    end
-
-    # Delegates to #remove.
-    def _delete(uploaded_file, context)
-      remove(uploaded_file, context)
-    end
-
-    # Delegates to #copy.
-    def put(io, context)
-      copy(io, context)
-    end
-
-    # Calls `#upload` on the storage, passing to it the location, metadata
-    # and any upload options. The storage might modify the location or
-    # metadata that were passed in. The uploaded IO is then closed.
-    def copy(io, context)
-      location       = context[:location]
-      metadata       = context[:metadata]
-      upload_options = context[:upload_options] || {}
-
-      storage.upload(io, location, shrine_metadata: metadata, **upload_options)
-    ensure
-      io.close rescue nil
-    end
-
-    # Delegates to `UploadedFile#delete`.
-    def remove(uploaded_file, context)
-      uploaded_file.delete
-    end
-
-    # Delegates to #process.
-    def processed(io, context)
-      process(io, context)
-    end
-
     # Generates a basic location for an uploaded file
     def basic_location(io, metadata:)
       extension   = ".#{io.extension}" if io.is_a?(UploadedFile) && io.extension
@@ -345,23 +276,26 @@ class Shrine
       basename + extension
     end
 
-    # Retrieves the location for the given IO and context. First it looks
-    # for the `:location` option, otherwise it calls #generate_location.
-    def get_location(io, context)
-      location = context[:location] || generate_location(io, context)
-      location or raise Error, "location generated for #{io.inspect} was nil (context = #{context})"
-    end
-
     # If the IO object is a Shrine::UploadedFile, it simply copies over its
     # metadata, otherwise it calls #extract_metadata.
-    def get_metadata(io, context)
-      if io.is_a?(UploadedFile) && context[:metadata] != true
-        io.metadata.dup
-      elsif context[:metadata] != false
-        extract_metadata(io, context)
+    def get_metadata(io, metadata: nil, **options)
+      if io.is_a?(UploadedFile) && metadata != true
+        result = io.metadata.dup
+      elsif metadata != false
+        result = extract_metadata(io, **options)
       else
-        {}
+        result = {}
       end
+
+      result = result.merge(metadata) if metadata.is_a?(Hash)
+      result
+    end
+
+    # Retrieves the location for the given IO and context. First it looks
+    # for the `:location` option, otherwise it calls #generate_location.
+    def get_location(io, location: nil, **options)
+      location ||= generate_location(io, options)
+      location or fail Error, "location generated for #{io.inspect} was nil"
     end
 
     # Asserts that the object is a valid IO object, specifically that it