shrine 1.4.2 → 2.0.0

data/doc/changing_location.md

@@ -1,4 +1,4 @@
-# Changing Location of Files
+# Migrating to Different Location
 
 You have a production app with already uploaded attachments. However, you've
 realized that the existing store folder structure for attachments isn't working
@@ -19,11 +19,13 @@ live production).
 Shrine.plugin :delete_promoted
 
 User.paged_each do |user|
-  user.promote(user.avatar, phase: :change_location)
+  attacher = user.avatar_attacher
+  attacher.promote(phase: :migrate) if attacher.stored?
+  # use `attacher._promote(phase: :migrate)` if you want promoting to be backgrounded
 end
 ```
 
-Note that the phase has to be overriden, otherwise it defaults to `:store`
-which would trigger processing if you have it set up.
+The `:phase` is not mandatory, it's just for better introspection when
+monitoring background jobs and logs.
 
 Now all your existing attachments should be happily living on new locations.

data/doc/creating_storages.md

@@ -13,7 +13,7 @@ class Shrine
         # initializing logic
       end
 
-      def upload(io, id, metadata = {})
+      def upload(io, id, shrine_metadata: {}, **upload_options)
         # uploads `io` to the location `id`
       end
 
@@ -53,7 +53,7 @@ If your storage doesn't control which id the uploaded file will have, you
 can modify the `id` variable:
 
 ```rb
-def upload(io, id, metadata = {})
+def upload(io, id, shrine_metadata: {}, **upload_options)
   actual_id = do_upload(io, id, metadata)
   id.replace(actual_id)
 end
@@ -63,7 +63,7 @@ Likewise, if you need to save some information into the metadata after upload,
 you can modify the metadata hash:
 
 ```rb
-def upload(io, id, metadata = {})
+def upload(io, id, shrine_metadata: {}, **upload_options)
   additional_metadata = do_upload(io, id, metadata)
   metadata.merge!(additional_metadata)
 end
@@ -125,7 +125,7 @@ class Shrine
     class MyStorage
       # ...
 
-      def move(io, id, metadata = {})
+      def move(io, id, shrine_metadata: {}, **upload_options)
         # does the moving of the `io` to the location `id`
       end
 

data/doc/design.md

@@ -0,0 +1,223 @@
+# The Design of Shrine
+
+There are five main types of objects that you deal with in Shrine:
+
+* Storage
+* `Shrine`
+* `Shrine::UploadedFile`
+* `Shrine::Attacher`
+* `Shrine::Attachment`
+
+## Storage
+
+On the lowest level we have a storage. A storage class encapsulates file
+management logic on a particular service. It is what actually performs uploads,
+generation of URLs, deletions and similar. By convention it is namespaced under
+`Shrine::Storage`.
+
+```rb
+filesystem = Shrine::Storage::FileSystem.new("uploads")
+filesystem.upload(file, "foo")
+filesystem.url("foo") #=> "uploads/foo"
+filesystem.delete("foo")
+```
+
+A storage is a PORO which responds to certain methods:
+
+```rb
+class Shrine
+  module Storage
+    class MyStorage
+      def initialize(*args)
+        # initializing logic
+      end
+
+      def upload(io, id, shrine_metadata: {}, **upload_options)
+        # uploads `io` to the location `id`
+      end
+
+      def download(id)
+        # downloads the file from the storage
+      end
+
+      def open(id)
+        # returns the remote file as an IO-like object
+      end
+
+      def read(id)
+        # returns the file contents as a string
+      end
+
+      def exists?(id)
+        # checks if the file exists on the storage
+      end
+
+      def delete(id)
+        # deletes the file from the storage
+      end
+
+      def url(id, options = {})
+        # URL to the remote file, accepts options for customizing the URL
+      end
+
+      def clear!(confirm = nil)
+        # deletes all the files in the storage
+      end
+
+      # ...
+    end
+  end
+end
+```
+
+Storages are typically not used directly, but through `Shrine`.
+
+## `Shrine`
+
+A `Shrine` object (also called an "uploader") acts as a wrapper around a
+storage. First the storage needs to be registered under a name:
+
+```rb
+Shrine.storages[:file_system] = Shrine::Storage::FileSystem.new("uploads")
+```
+
+Now we can instantiate an uploader with this identifier, and upload files:
+
+```rb
+uploader = Shrine.new(:file_system)
+uploaded_file = uploader.upload(file)
+uploaded_file #=> #<Shrine::UploadedFile>
+```
+
+The argument to `Shrine#upload` must be an IO-like object. The method does the
+following:
+
+* generates a unique location
+* extracts metadata
+* uploads the file
+* closes the file
+* creates a `Shrine::UploadedFile` from the data
+
+In applications it's common to create subclasses of `Shrine`, in order to allow
+having different uploading logic for different types of files.
+
+## `Shrine::UploadedFile`
+
+`Shrine::UploadedFile` represents a file that was uploaded to a storage, and is
+the result of `Shrine#upload`. It is essentially a wrapper around a data hash
+containing information about the uploaded file.
+
+```rb
+uploaded_file      #=> #<Shrine::UploadedFile>
+uploaded_file.data #=>
+# {
+#   "storage"  => "file_system",
+#   "id"       => "9260ea09d8effd.pdf",
+#   "metadata" => {
+#     "filename"  => "resume.pdf",
+#     "mime_type" => "application/pdf",
+#     "size"      => 983294,
+#   },
+# }
+```
+
+The data hash contains the storage the file was uploaded to, the location, and
+some metadata: original filename, MIME type and filesize. The
+`Shrine::UploadedFile` object has handy methods which use this data:
+
+```rb
+# metadata methods
+uploaded_file.original_filename
+uploaded_file.mime_type
+uploaded_file.size
+# ...
+
+# storage methods
+uploaded_file.url
+uploaded_file.exists?
+uploaded_file.download
+uploaded_file.delete
+# ...
+```
+
+A `Shrine::UploadedFile` is itself an IO-like object (representing the
+remote file), so it can be passed to `Shrine#upload` as well.
+
+## `Shrine::Attacher`
+
+We usually want to treat uploaded files as *attachments* to records, saving
+their data into a database column. This is the responsibility of
+`Shrine::Attacher`. A `Shrine::Attacher` uses `Shrine` uploaders and
+`Shrine::UploadedFile` objects internally.
+
+The attaching process requires a temporary and a permanent storage to be
+registered (by default that's `:cache` and `:store`):
+
+```rb
+Shrine.storages = {
+  cache: Shrine::Storage::FileSystem.new("uploads/cache"),
+  store: Shrine::Storage::FileSystem.new("uploads/store"),
+}
+```
+
+A `Shrine::Attacher` is instantiated with a model instance and an attachment
+name (an "image" attachment will be saved to `image_data` field):
+
+```rb
+attacher = Shrine::Attacher.new(photo, :image)
+
+attacher.assign(file)
+attacher.get #=> #<Shrine::UploadedFile>
+attacher.record.image_data #=> "{\"storage\":\"cache\",\"id\":\"9260ea09d8effd.jpg\",\"metadata\":{...}}"
+
+attacher._promote
+attacher.get #=> #<Shrine::UploadedFile>
+attacher.record.image_data #=> "{\"storage\":\"store\",\"id\":\"ksdf02lr9sf3la.jpg\",\"metadata\":{...}}"
+```
+
+Above a file is assigned by the attacher, which "caches" (uploads) the file to
+the temporary storage. The cached file is then "promoted" (uploaded) to
+permanent storage. Behind the scenes a cached `Shrine::UploadedFile` is given
+to `Shrine#upload`, which works because `Shrine::UploadedFile` is an IO-like
+object. After both caching and promoting the data hash of the uploaded file is
+assigned to the record's column as JSON.
+
+## `Shrine::Attachment`
+
+`Shrine::Attachment` is the highest level of abstraction. A
+`Shrine::Attachment` module exposes the `Shrine::Attacher` object through the
+model instance. The `Shrine::Attachment` class is a sublcass of `Module`, which
+means that an instance of `Shrine::Attachment` is a module:
+
+```rb
+Shrine::Attachment.new(:image).is_a?(Module) #=> true
+Shrine::Attachment.new(:image).instance_methods #=> [:image=, :image, :image_url, :image_attacher]
+
+# equivalents
+Shrine::Attachment.new(:image)
+Shrine.attachment(:image)
+Shrine[:image]
+```
+
+We can include this module to a model:
+
+```rb
+class Photo
+  include Shrine[:image]
+end
+```
+```rb
+photo.image = file # shorthand for `photo.image_attacher.assign(file)`
+photo.image        # shorthand for `photo.image_attacher.get`
+photo.image_url    # shorthand for `photo.image_attacher.url`
+
+photo.image_attacher #=> #<Shrine::Attacher>
+```
+
+When an ORM plugin is loaded, the `Shrine::Attachment` module also
+automatically:
+
+* syncs Shrine's validation errors with the record
+* triggers promoting after record is saved
+* deletes the uploaded file if attachment was replaced, removed or the record
+  destroyed

data/doc/migrating_storage.md

@@ -1,4 +1,4 @@
-# Migrating to Another Storage
+# Migrating to Different Storage
 
 While your application is live in production and performing uploads, it may
 happen that you decide you want to change your storage (the `:store`). Shrine
@@ -29,13 +29,11 @@ After you've deployed the previous change, it's time to copy all the existing
 files to the new storage, and update the records. This is how you can do it
 if you're using Sequel:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
 ```rb
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    user.avatar_store.upload(avatar, {record: user, name: :avatar})
+  if (attacher = user.avatar_attacher).stored?
+    uploaded_file = attacher.store!(user.avatar)
+    attacher.swap(uploaded_file)
   end
 end
 
@@ -64,13 +62,10 @@ Shrine.storages[:new_store] = Shrine.storages[:store]
 **Second**, you should rename the storage names on existing records. With
 Sequel it would be something like:
 
-```rb
-Shrine.plugin :migration_helper # before the model is loaded
-```
 ```rb
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    avatar.to_json.gsub('"new_store"', '"store"')
+  if user.avatar_attacher.stored?
+    user.update(avatar_data: user.avatar_data.gsub('"new_store"', '"store"'))
   end
 end
 

data/doc/regenerating_versions.md

@@ -20,7 +20,7 @@ class ImageUploader < Shrine
   def process(io, context)
     case context[:phase]
     when :store
-      thumb = process_thumb(io.download) # replace with actual processing method
+      thumb = process_thumb(io.download)
       {original: io, thumb: thumb}
     end
   end
@@ -40,17 +40,13 @@ unprocessed file URL.
 Afterwards you should run a script which reprocesses the versions for existing
 files:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
 ```rb
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    unless avatar.is_a?(Hash)
-      file = some_processing(avatar.download)
-      thumb = user.avatar_store.upload(file, {record: user, name: :avatar, version: :thumb})
-      {original: avatar, thumb: thumb}
-    end
+  attacher, attachment = user.avatar_attacher, user.avatar
+  if attacher.stored? && !attachment.is_a?(Hash)
+    file = some_processing(attachment.download)
+    thumb = attacher.store!(file, version: :thumb)
+    attacher.swap({original: avatar, thumb: thumb})
   end
 end
 ```
@@ -61,15 +57,13 @@ The simplest scenario is where you need to regenerate an existing version.
 First you need to change and deploy your updated processing code, and
 afterwards you can run a script like this on your production database:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
-
 ```rb
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    thumb = some_processing(avatar[:original].download)
-    avatar.merge(thumb: avatar[:thumb].replace(thumb))
+  attacher, attachment = user.avatar_attacher, user.avatar
+  if attacher.stored?
+    file = some_processing(attachment[:original].download)
+    thumb = attachment[:thumb].replace(thumb)
+    attacher.swap(attachment.merge(thumb: thumb))
   end
 end
 ```
@@ -97,18 +91,13 @@ end
 After you've deployed this change, you should run a script that will generate
 the new version for all existing records:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
-
 ```rb
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    unless avatar[:new]
-      file = some_processing(avatar[:original].download, *args)
-      new = user.avatar_store.upload(file, {record: user, name: :avatar, version: :new})
-      avatar.merge(new: new)
-    end
+  attacher, attachment = user.avatar_attacher, user.avatar
+  if attacher.stored? && !attachment[:new]
+    file = some_processing(attachment[:original].download, *args)
+    new = attacher.store!(file, version: :new)
+    attacher.swap(attachment.merge(new: new))
   end
 end
 ```
@@ -124,18 +113,14 @@ generate it (but keep the version name in the list), as well as update your app
 not to use the new version, and deploy that code. After you've done that, you
 can run a script which removes that version:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
-
 ```rb
 old_versions = []
 
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    old_version = avatar.delete(:old_version)
-    old_versions << old_version if old_version
-    avatar
+  attacher, attachment = user.avatar_attacher, user.avatar
+  if attacher.stored? && attachment[:old_version]
+    old_versions << attachment.delete(:old_version)
+    attacher.swap(attachment)
   end
 end
 
@@ -154,13 +139,10 @@ If you made a lot of changes to versions, it might make sense to simply
 regenerate all versions. After you've deployed the change in processing, you
 can run a script which updates existing records:
 
-```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-```
-
 ```rb
 User.paged_each do |user|
-  if user.avatar && user.avatar_store.uploaded?(user.avatar)
+  if user.avatar_attacher.stored?
+    # assuming your largest version is named ":original"
     user.update(avatar: user.avatar[:original])
   end
 end

data/lib/shrine.rb

@@ -23,13 +23,6 @@ class Shrine
     end
   end
 
-  # Raised by storages in method `#clear!` when confirmation wasn't passed in.
-  class Confirm < Error
-    def message
-      "Are you sure you want to delete all files from the storage? (confirm with `clear!(:confirm)`)"
-    end
-  end
-
   # Methods which an object has to respond to in order to be considered
   # an IO object.  Keys are method names, and values are arguments.
   IO_METHODS = {
@@ -276,11 +269,6 @@ class Shrine
           }
         end
 
-        # User-defined default URL for when a file is missing (called by
-        # `Attacher#url`).
-        def default_url(context)
-        end
-
         private
 
         # Extracts the filename from the IO using some basic heuristics.
@@ -294,9 +282,8 @@ class Shrine
 
         # Extracts the MIME type from the IO using some basic heuristics.
         def extract_mime_type(io)
-          if io.respond_to?(:mime_type)
-            io.mime_type
-          elsif io.respond_to?(:content_type)
+          if io.respond_to?(:content_type)
+            warn "The \"mime_type\" Shrine metadata field will be set from the \"Content-Type\" request header, which might not hold the actual MIME type of the file. It is recommended to load the determine_mime_type plugin which determines MIME type from file content." unless opts.key?(:mime_type_analyzer)
             io.content_type
           end
         end
@@ -335,7 +322,8 @@ class Shrine
 
         # Does the actual uploading, calling `#upload` on the storage.
         def copy(io, context)
-          storage.upload(io, context[:location], context[:metadata])
+          context[:upload_options] = (context[:upload_options] || {}).merge(shrine_metadata: context[:metadata])
+          storage.upload(io, context[:location], context[:upload_options])
         ensure
           io.close rescue nil
         end
@@ -379,7 +367,7 @@ class Shrine
 
         # Generates a UID to use in location for uploaded files.
         def generate_uid(io)
-          SecureRandom.hex(30)
+          SecureRandom.hex
         end
       end
 
@@ -481,7 +469,7 @@ class Shrine
         # Otherwise it assumes that it's an IO object and caches it.
         def assign(value)
           if value.is_a?(String)
-            assign_cached(value) unless value == ""
+            assign_cached(value) unless value == "" || value == read
           else
             uploaded_file = cache!(value, phase: :cache) if value
             set(uploaded_file)
@@ -491,11 +479,9 @@ class Shrine
         # Assigns a Shrine::UploadedFile, runs validation and schedules the
         # old file for deletion.
         def set(uploaded_file)
-          @old = get unless get == uploaded_file
+          @old = get
           _set(uploaded_file)
           validate
-
-          get
         end
 
         # Retrieves the uploaded file from the record column.
@@ -505,7 +491,7 @@ class Shrine
 
         # Returns true if a new file has been attached.
         def attached?
-          instance_variable_defined?("@old")
+          instance_variable_defined?(:@old)
         end
 
         # Plugins can override this if they want something to be done on save.
@@ -516,45 +502,61 @@ class Shrine
         # be called after saving.
         def finalize
           replace
-          remove_instance_variable("@old")
-          _promote
+          remove_instance_variable(:@old)
+          _promote(phase: :store) if cached?
         end
 
-        # Calls #promote if attached file is cached.
-        def _promote
-          promote(get) if promote?(get)
+        # Promotes the file.
+        def _promote(uploaded_file = get, phase: nil)
+          promote(uploaded_file, phase: phase)
         end
 
         # Uploads the cached file to store, and updates the record with the
         # stored file.
-        def promote(cached_file, phase: :store)
-          stored_file = store!(cached_file, phase: phase)
-          result = swap(stored_file) or delete!(stored_file, phase: :stored)
+        def promote(uploaded_file = get, **options)
+          stored_file = store!(uploaded_file, **options)
+          result = swap(stored_file) or _delete(stored_file, phase: :abort)
           result
         end
 
+        # Calls #update, overriden in ORM plugins.
+        def swap(uploaded_file)
+          update(uploaded_file)
+          uploaded_file if uploaded_file == get
+        end
+
         # Deletes the attachment that was replaced, and is called after saving
         # by ORM integrations. If also removes `@old` so that #save and #finalize
         # don't get called for the current attachment anymore.
         def replace
-          delete!(@old, phase: :replaced) if @old && !cache.uploaded?(@old)
+          _delete(@old, phase: :replace) if @old && !cache.uploaded?(@old)
         end
 
         # Deletes the attachment. Typically this should be called after
         # destroying a record.
         def destroy
-          delete!(get, phase: :destroyed) if get && !cache.uploaded?(get)
+          _delete(get, phase: :destroy) if get && !cache.uploaded?(get)
+        end
+
+        # Deletes the uploaded file.
+        def _delete(uploaded_file, phase: nil)
+          delete!(uploaded_file, phase: phase)
         end
 
         # Returns the URL to the attached file (internally calls `#url` on the
-        # storage).  If the attachment is missing, it calls
-        # `Shrine#default_url`.  Forwards any URL options to the storage.
+        # storage), forwarding any URL options to the storage.
         def url(**options)
-          if uploaded_file = get
-            uploaded_file.url(**options)
-          else
-            default_url(**options)
-          end
+          get.url(**options) if read
+        end
+
+        # Returns true if attachment is present and cached.
+        def cached?
+          get && cache.uploaded?(get)
+        end
+
+        # Returns true if attachment is present and stored.
+        def stored?
+          get && store.uploaded?(get)
         end
 
         # Runs the validations defined by `Attacher.validate`.
@@ -568,6 +570,21 @@ class Shrine
           shrine_class.uploaded_file(*args, &block)
         end
 
+        # Uploads the file to cache passing context.
+        def cache!(io, **options)
+          cache.upload(io, context.merge(options))
+        end
+
+        # Uploads the file to store passing context.
+        def store!(io, **options)
+          store.upload(io, context.merge(options))
+        end
+
+        # Deletes the file passing context.
+        def delete!(uploaded_file, **options)
+          store.delete(uploaded_file, context.merge(options))
+        end
+
         # Returns the Shrine class related to this attacher.
         def shrine_class
           self.class.shrine_class
@@ -577,20 +594,8 @@ class Shrine
 
         # Assigns a cached file (refuses if the file is stored).
         def assign_cached(value)
-          uploaded_file = uploaded_file(value)
-          set(uploaded_file) if cache.uploaded?(uploaded_file)
-        end
-
-        # Returns true if uploaded_file exists and is cached.  If it's true,
-        # \#promote will be called.
-        def promote?(uploaded_file)
-          uploaded_file && cache.uploaded?(uploaded_file)
-        end
-
-        # Calls #update, overriden in ORM plugins.
-        def swap(uploaded_file)
-          update(uploaded_file)
-          uploaded_file if get == uploaded_file
+          cached_file = uploaded_file(value)
+          set(cached_file) if cache.uploaded?(cached_file)
         end
 
         # Sets and saves the uploaded file.
@@ -598,28 +603,6 @@ class Shrine
           _set(uploaded_file)
         end
 
-        # Uploads the file to cache (calls `Shrine#upload`).
-        def cache!(io, phase:)
-          cache.upload(io, context.merge(phase: phase))
-        end
-
-        # Uploads the file to store (calls `Shrine#upload`).
-        def store!(io, phase:)
-          store.upload(io, context.merge(phase: phase))
-        end
-
-        # Deletes the file (calls `Shrine#delete`).
-        def delete!(uploaded_file, phase:)
-          store.delete(uploaded_file, context.merge(phase: phase))
-        end
-
-        # Delegates to `Shrine#default_url`.
-        def default_url(**options)
-          url = store.default_url(options.merge(context))
-          warn "Overriding Shrine#default_url is deprecated and will be removed in Shrine 2. You should use the default_url plugin." if url
-          url
-        end
-
         # The validation block provided by `Shrine.validate`.
         def validate_block
           shrine_class.opts[:validate]
@@ -632,12 +615,12 @@ class Shrine
 
         # It writes to record's `<attachment>_data` column.
         def write(value)
-          record.send("#{name}_data=", value)
+          record.send(:"#{name}_data=", value)
         end
 
         # It reads from the record's `<attachment>_data` column.
         def read
-          value = record.send("#{name}_data")
+          value = record.send(:"#{name}_data")
           value unless value.nil? || value.empty?
         end