shrine 1.3.0 → 1.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d59731ea1e8a0376a6f8ad86e8d627250dec2f44
-  data.tar.gz: 14bd2696930a028e157cc1535995a5342958170b
+  metadata.gz: cd828e546e0d244ed304996054f0bd01b790800d
+  data.tar.gz: 46f44feb2075dd6d8001e990199acd99deae4223
 SHA512:
-  metadata.gz: a0ca4170b242be2e01fa6abdeb9ff0e3380e1271c0783099e738182e4d6f995ac7b8042884c52147052a751b6a7143eb16766879165c87f221ed6e84b7deb53a
-  data.tar.gz: 6aee5bb9a3a44f162c9d2da791c498c2d592a05aa3efc7bf058d38def80040af8856db96431eb558ed93538cc327f718874f59a376e3b55e11778189c87401be
+  metadata.gz: a2f0a33b73fb8a012b0e8036c6e1c62b2ad661ec38af4f16b72e80f3f7a39d9f42273232f5fd718f20330848c4db09b65f4f5fa28ca754bdaa31a56b32801fde
+  data.tar.gz: c3e3791da4e71e600b0c973edc27a6f40145dffedbc8e0b2c835415d25990caed5d5f651f77091a4f12a7fdef26452a2ad53f20158486f0f486a1743470e5943

data/README.md

@@ -7,10 +7,10 @@ explains the motivation behind Shrine.
 
 ## Resources
 
-* Documentation: [shrinerb.com](http://shrinerb.com)
-* Source: [github.com/janko-m/shrine](https://github.com/janko-m/shrine)
-* Bugs: [github.com/janko-m/shrine/issues](https://github.com/janko-m/shrine/issues)
-* Help & Dicussion: [groups.google.com/group/ruby-shrine](https://groups.google.com/forum/#!forum/ruby-shrine)
+- Documentation: [shrinerb.com](http://shrinerb.com)
+- Source: [github.com/janko-m/shrine](https://github.com/janko-m/shrine)
+- Bugs: [github.com/janko-m/shrine/issues](https://github.com/janko-m/shrine/issues)
+- Help & Dicussion: [groups.google.com/group/ruby-shrine](https://groups.google.com/forum/#!forum/ruby-shrine)
 
 ## Installation
 
@@ -64,10 +64,10 @@ that was uploaded, and we can do a lot with it:
 
 ```rb
 uploaded_file.url      #=> "/uploads/938kjsdf932.mp4"
+uploaded_file.metadata #=> {...}
 uploaded_file.read     #=> "..."
 uploaded_file.exists?  #=> true
 uploaded_file.download #=> #<Tempfile:/var/folders/k7/6zx6dx6x7ys3rv3srh0nyfj00000gn/T/20151004-74201-1t2jacf.mp4>
-uploaded_file.metadata #=> {...}
 uploaded_file.delete
 # ...
 ```
@@ -84,6 +84,7 @@ Shrine we do this by generating and including "attachment" modules.
 Firstly we need to assign the special `:cache` and `:store` storages:
 
 ```rb
+require "shrine"
 require "shrine/storage/file_system"
 
 Shrine.storages = {
@@ -212,7 +213,7 @@ end
 ```
 ```js
 $('[type="file"]').fileupload({
-  url:       '/attachments/images/cache/avatar',
+  url:       '/attachments/images/cache/upload',
   paramName: 'file',
   add:       function(e, data) { /* Disable the submit button */ },
   progress:  function(e, data) { /* Add a nice progress bar */ },
@@ -452,6 +453,9 @@ class ImageUploader < Shrine
 end
 ```
 
+Note that if you're extracting custom metadata from Ruby, you should always
+rewind the file afterwards.
+
 ## Locations
 
 By default files will all be put in the same folder. If you want that each
@@ -478,7 +482,7 @@ end
 
 Note that there should always be a random component in the location, otherwise
 dirty tracking won't be detected properly (you can use `Shrine#generate_uid`).
-Also note that you can access the extracted metadata here through
+Inside this method you can access the extracted metadata through
 `context[:metadata]`.
 
 When using the uploader directly, it's possible to bypass `#generate_location`
@@ -542,11 +546,11 @@ makes it really easy to plug in your backgrounding library:
 
 ```rb
 Shrine.plugin :backgrounding
-Shrine::Attacher.promote { |data| UploadJob.perform_async(data) }
+Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
 Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
 ```
 ```rb
-class UploadJob
+class PromoteJob
   include Sidekiq::Worker
   def perform(data)
     Shrine::Attacher.promote(data)
@@ -595,7 +599,7 @@ be applied to which uploaders:
 Shrine.plugin :logging # enables logging for all uploaders
 
 class ImageUploader < Shrine
-  plugin :store_dimensions # stores dimensions only for this uploader
+  plugin :store_dimensions # stores dimensions only for this uploader and its descendants
 end
 ```
 

data/doc/carrierwave.md

@@ -451,7 +451,7 @@ you can change this with the `default_storage` plugin.
 
 #### `fog_*`
 
-These options will be set on the soon-to-be-released Fog storage for Shrine.
+These options are set on the [shrine-fog] storage.
 
 #### `delete_tmp_file_after_storage`, `remove_previously_stored_file_after_update`
 
@@ -522,3 +522,4 @@ multipart or not.
 [image_processing]: https://github.com/janko-m/image_processing
 [example app]: https://github.com/janko-m/shrine-example
 [Regenerating versions]: http://shrinerb.com/rdoc/files/doc/regenerating_versions_md.html
+[shrine-fog]: https://github.com/janko-m/shrine-fog

data/doc/changing_location.md

@@ -4,33 +4,26 @@ You have a production app with already uploaded attachments. However, you've
 realized that the existing store folder structure for attachments isn't working
 for you.
 
-The first step is to change the location (by overriding `#generate_location` or
-with the pretty_location plugin), and deploy that change. Attachments on old
-locations will still continue to work properly.
+The first step is to change the location, by overriding `#generate_location` or
+with the pretty_location plugin, and deploy that change. This will make any new
+files upload to the desired location, attachments on old locations will still
+continue to work normally.
 
-The next step is to run a script that will move those to new locations. The
-easiest way to do that is to reupload them, and afterwards delete them:
+The next step is to run a script that will move old files to new locations. The
+easiest way to do that is to reupload them and delete them. Shrine has a method
+exactly for that, `Attacher#promote`, which also handles the situation when
+someone attaches a new file during "moving" (since we're running this script on
+live production).
 
 ```rb
-Shrine.plugin :migration_helpers # before the model is loaded
-Shrine.plugin :multi_delete # for deleting multiple files at once
-```
-```rb
-old_avatars = []
+Shrine.plugin :delete_promoted
 
 User.paged_each do |user|
-  user.update_avatar do |avatar|
-    old_avatars << avatar
-    user.avatar_store.upload(avatar)
-  end
-end
-
-if old_avatars.any?
-  # you'll have to change this code slightly if you're using versions
-  uploader = old_avatars.first.uploader
-  uploader.delete(old_avatars)
+  user.promote(user.avatar, phase: :change_location)
 end
 ```
 
-And now all your existing attachments should be happily living on new
-locations.
+Note that the phase has to be overriden, otherwise it defaults to `:store`
+which would trigger processing if you have it set up.
+
+Now all your existing attachments should be happily living on new locations.

data/doc/creating_storages.md

@@ -171,8 +171,8 @@ linter = Shrine::Storage::Linter.new(storage)
 linter.call
 ```
 
-The linter will test your methods with simple IO objects, and raise an error
-with an appropriate message if a part of the specification isn't satisfied.
+The linter will test your methods with fake IO objects, and raise a
+`Shrine::LintError` if any part of the contract isn't satisfied.
 
 If you want to specify the IO object to use for testing (e.g. you need the IO
 to be an actual image), you can pass in a lambda which returns the IO when
@@ -188,3 +188,7 @@ pass `action: :warn` when initializing
 ```rb
 linter = Shrine::Storage::Linter.new(storage, action: :warn)
 ```
+
+Note that using the linter doesn't mean that you shouldn't write any manual
+tests for your storage. There will likely be some edge cases that won't be
+tested by the linter.

data/doc/direct_s3.md

@@ -46,9 +46,10 @@ Shrine's JSON representation of an uploaded file looks like this:
 }
 ```
 
-The `id`, `storage` and `metadata.size` fields are required, and the rest of
-the metadata is optional. After uploading the file to S3, you need to construct
-this JSON and assign it to the hidden attachment field in the form.
+The `id`, `storage` fields are optional, while the `metadata` values are
+optional (`metadata.size` is only required to later upload that file to a
+non-S3 storage). After uploading the file to S3, you need to construct this
+JSON and assign it to the hidden attachment field in the form.
 
 ## Strategy A (dynamic)
 
@@ -95,7 +96,7 @@ usually write it to the hidden attachment field in the form:
 
 ```js
 var image = {
-  id: /cache\/(.+)/.exec(key)[1], # we have to remove the prefix part
+  id: key.match(/cache\/(.+)/)[1], # we have to remove the prefix part
   storage: 'cache',
   metadata: {
     size:      data.files[0].size,
@@ -156,6 +157,19 @@ Notice that we needed to fetch and assign the size of the uploaded file. This
 is because this hash is later transformed into an IO which requires `#size`
 to be non-nil (and it is read from the metadata field).
 
+## Metadata
+
+With direct uploads any metadata has to be extracted on the client, since
+caching the file doesn't touch your application. When the cached file is stored,
+Shrine's default behaviour is to simply copy over cached file's metadata.
+
+If you want to extract metadata on the server before storing, you can just
+load the restore_cached_data plugin.
+
+```rb
+plugin :restore_cached_data
+```
+
 ## Eventual consistency
 
 When uploading objects to Amazon S3, sometimes they may not be available
@@ -180,7 +194,7 @@ backgrounding library to perform the job with a delay:
 ```rb
 Shrine.plugin :backgrounding
 Shrine::Attacher.promote do |data|
-  UploadJob.perform_in(60, data) # tells a Sidekiq worker to perform in 1 minute
+  PromoteJob.perform_in(60, data) # tells a Sidekiq worker to perform in 1 minute
 end
 ```
 

data/doc/refile.md

@@ -158,7 +158,7 @@ Rails.application.routes.draw do
 end
 ```
 ```rb
-# POST /attachments/images/cache/avatar
+# POST /attachments/images/cache/upload
 {
   "id": "43kewit94.jpg",
   "storage": "cache",

data/lib/shrine/plugins/activerecord.rb

@@ -56,7 +56,7 @@ class Shrine
               #{@name}_attacher.finalize if #{@name}_attacher.attached?
             end
 
-            after_commit on: :destroy do
+            after_commit on: [:destroy] do
               #{@name}_attacher.destroy
             end
           RUBY
@@ -75,18 +75,12 @@ class Shrine
 
         # Updates the current attachment with the new one, unless the current
         # attachment has changed.
-        def swap(uploaded_file)
-          record.class.transaction do
-            break if record.send("#{name}_data") != record.reload.send("#{name}_data")
-            super
-          end
-        rescue ::ActiveRecord::RecordNotFound
-        end
-
-        # We save the record after updating, raising any validation errors.
         def update(uploaded_file)
-          super
-          record.save!
+          record.class.where(record.class.primary_key => record.id)
+            .where(:"#{name}_data" => record.send(:"#{name}_data"))
+            .update_all(:"#{name}_data" => uploaded_file.to_json)
+          record.reload
+        rescue ::ActiveRecord::RecordNotFound
         end
       end
     end

data/lib/shrine/plugins/backgrounding.rb

@@ -6,15 +6,15 @@ class Shrine
     # something other than Storage::FileSystem.
     #
     #     Shrine.plugin :backgrounding
-    #     Shrine::Attacher.promote { |data| UploadJob.perform_async(data) }
+    #     Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
     #     Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
     #
     # The `data` variable is a serializable hash containing all context needed
-    # for promotion/deletion. You then just need to declare `UploadJob` and
+    # for promotion/deletion. You then just need to declare `PromoteJob` and
     # `DeleteJob`, and call `Shrine::Attacher.promote`/`Shrine::Attacher.delete`
     # with the data hash:
     #
-    #     class UploadJob
+    #     class PromoteJob
     #       include Sidekiq::Worker
     #
     #       def perform(data)
@@ -47,6 +47,26 @@ class Shrine
     #       record.update(published: true) if record.is_a?(Post)
     #     end
     #
+    # You can also write custom background jobs with `Attacher.dump` and
+    # `Attacher.load`:
+    #
+    #     class User < Sequel::Model
+    #       def after_commit
+    #         if some_condition
+    #           data = Shrine::Attacher.dump(avatar_attacher)
+    #           SomethingJob.perform_async(data)
+    #         end
+    #       end
+    #     end
+    #
+    #     class SomethingJob
+    #       include Sidekiq::Worker
+    #       def perform(data)
+    #         attacher = Shrine::Attacher.load(data)
+    #         # ...
+    #       end
+    #     end
+    #
     # If you're generating versions, and you want to process some versions in
     # the foreground before kicking off a background job, you can use the
     # `recache` plugin.
@@ -58,18 +78,13 @@ class Shrine
           if block
             shrine_class.opts[:backgrounding_promote] = block
           else
-            record_class, record_id = data["record"]
-            record_class = Object.const_get(record_class)
-            record = find_record(record_class, record_id) or return
-
-            name = data["attachment"]
-            attacher = record.send("#{name}_attacher")
+            attacher = load(data)
             cached_file = attacher.uploaded_file(data["uploaded_file"])
-            return if cached_file != record.send(name)
+            phase = data["phase"].to_sym
 
-            attacher.promote(cached_file) or return
+            attacher.promote(cached_file, phase: phase) or return
 
-            record
+            attacher.record
           end
         end
 
@@ -79,20 +94,39 @@ class Shrine
           if block
             shrine_class.opts[:backgrounding_delete] = block
           else
-            record_class, record_id = data["record"]
-            record = Object.const_get(record_class).new
-            record.id = record_id
-
-            name, phase = data["attachment"], data["phase"]
-            attacher = record.send("#{name}_attacher")
+            attacher = load(data)
             uploaded_file = attacher.uploaded_file(data["uploaded_file"])
-            context = {name: name.to_sym, record: record, phase: phase.to_sym}
+            context = {name: attacher.name, record: attacher.record, phase: data["phase"].to_sym}
 
             attacher.store.delete(uploaded_file, context)
 
-            record
+            attacher.record
           end
         end
+
+        # Dumps all the information about the attacher in a serializable hash
+        # suitable for passing as an argument to background jobs.
+        def dump(attacher)
+          {
+            "uploaded_file" => attacher.get && attacher.get.to_json,
+            "record"        => [attacher.record.class.to_s, attacher.record.id],
+            "attachment"    => attacher.name.to_s,
+          }
+        end
+
+        # Loads the data created by #dump, resolving the record and returning
+        # the attacher.
+        def load(data)
+          record_class, record_id = data["record"]
+          record_class = Object.const_get(record_class)
+          record = find_record(record_class, record_id) ||
+            record_class.new.tap { |object| object.id = record_id }
+
+          name = data["attachment"]
+          attacher = record.send("#{name}_attacher")
+
+          attacher
+        end
       end
 
       module AttacherMethods
@@ -100,32 +134,32 @@ class Shrine
         # hash.
         def _promote
           if background_promote = shrine_class.opts[:backgrounding_promote]
-            data = {
-              "uploaded_file" => get.to_json,
-              "record"        => [record.class.to_s, record.id],
-              "attachment"    => name.to_s,
-            }
-
+            data = self.class.dump(self).merge("phase" => "store")
             instance_exec(data, &background_promote) if promote?(get)
           else
             super
           end
         end
 
+        # Returns early if attachments don't match.
+        def promote(cached_file, *)
+          return if cached_file != get
+          super
+        end
+
         private
 
         # Calls the deleting block (if registered) with a serializable data
         # hash.
         def delete!(uploaded_file, phase:)
           if background_delete = shrine_class.opts[:backgrounding_delete]
-            data = {
+            data = self.class.dump(self).merge(
               "uploaded_file" => uploaded_file.to_json,
-              "record"        => [record.class.to_s, record.id],
-              "attachment"    => name.to_s,
               "phase"         => phase.to_s,
-            }
-
+            )
             instance_exec(data, &background_delete)
+
+            uploaded_file
           else
             super(uploaded_file, phase: phase)
           end

data/lib/shrine/plugins/backup.rb

@@ -1,13 +1,13 @@
 class Shrine
   module Plugins
-    # The backup plugin allows you to automatically backup up stored files to
+    # The backup plugin allows you to automatically back up stored files to
     # an additional storage.
     #
     #     storages[:backup_store] = Shrine::Storage::S3.new(options)
     #     plugin :backup, storage: :backup_store
     #
-    # After the cached file is promoted to store, it will be reuploaded from
-    # store to the provided "backup" storage.
+    # After a file is stored, it will be reuploaded from store to the provided
+    # backup storage.
     #
     #     user.update(avatar: file) # uploaded both to :store and :backup_store
     #
@@ -29,6 +29,29 @@ class Shrine
       end
 
       module AttacherMethods
+        # Backs up the stored file after promoting.
+        def promote(*)
+          result = super
+          store_backup!(result) if result
+          result
+        end
+
+        # Deletes the backup file in addition to the stored file.
+        def replace
+          result = super
+          delete_backup!(@old) if result && delete_backup?
+          result
+        end
+
+        # Deletes the backup file in addition to the stored file.
+        def destroy
+          result = super
+          delete_backup!(get) if result && delete_backup?
+          result
+        end
+
+        # Returns a copy of the given uploaded file with storage changed to
+        # backup storage.
         def backup_file(uploaded_file)
           uploaded_file(uploaded_file.to_json) do |file|
             file.data["storage"] = backup_storage.to_s
@@ -37,20 +60,6 @@ class Shrine
 
         private
 
-        # Back up the stored file and return it.
-        def store!(io, phase:)
-          stored_file = super
-          store_backup!(stored_file)
-          stored_file
-        end
-
-        # Delete the backed up file unless `:delete` was set to false.
-        def delete!(uploaded_file, phase:)
-          deleted_file = super
-          delete_backup!(deleted_file) if backup_delete?
-          deleted_file
-        end
-
         # Upload the stored file to the backup storage.
         def store_backup!(stored_file)
           backup_store.upload(stored_file, context.merge(phase: :backup))
@@ -58,7 +67,7 @@ class Shrine
 
         # Deleted the stored file from the backup storage.
         def delete_backup!(deleted_file)
-          backup_store.delete(backup_file(deleted_file), context.merge(phase: :backup))
+          delete!(backup_file(deleted_file), phase: :backup)
         end
 
         def backup_store
@@ -69,7 +78,7 @@ class Shrine
           shrine_class.opts[:backup_storage]
         end
 
-        def backup_delete?
+        def delete_backup?
           shrine_class.opts[:backup_delete]
         end
       end

data/lib/shrine/plugins/data_uri.rb

@@ -8,7 +8,7 @@ class Shrine
     #
     #     plugin :data_uri
     #
-    # If for example your attachment is called "avatar", this plugin will add
+    # If your attachment is called "avatar", this plugin will add
     # `#avatar_data_uri` and `#avatar_data_uri=` methods to your model.
     #
     #     user.avatar #=> nil
@@ -17,7 +17,14 @@ class Shrine
     #
     #     user.avatar.mime_type         #=> "image/png"
     #     user.avatar.size              #=> 43423
-    #     user.avatar.original_filename #=> nil
+    #
+    # If you want the uploaded file to have an extension, you can generate a
+    # filename based on the content type of the data URI:
+    #
+    #     plugin :data_uri, filename: ->(content_type) do
+    #       extension = MIME::Types[content_type].first.preferred_extension
+    #       "data_uri.#{extension}"
+    #     end
     #
     # If the data URI wasn't correctly parsed, an error message will be added to
     # the attachment column. You can change the default error message:
@@ -38,7 +45,8 @@ class Shrine
       DEFAULT_CONTENT_TYPE = "text/plain"
       DATA_URI_REGEXP = /\Adata:([-\w.+]+\/[-\w.+]+)?(;base64)?,(.*)\z/m
 
-      def self.configure(uploader, error_message: DEFAULT_ERROR_MESSAGE)
+      def self.configure(uploader, filename: nil, error_message: DEFAULT_ERROR_MESSAGE)
+        uploader.opts[:data_uri_filename] = filename
         uploader.opts[:data_uri_error_message] = error_message
       end
 
@@ -69,8 +77,10 @@ class Shrine
           if match = uri.match(DATA_URI_REGEXP)
             content_type = match[1] || DEFAULT_CONTENT_TYPE
             content      = match[2] ? Base64.decode64(match[3]) : match[3]
+            filename     = shrine_class.opts[:data_uri_filename]
+            filename     = filename.call(content_type) if filename
 
-            assign DataFile.new(content, content_type: content_type)
+            assign DataFile.new(content, content_type: content_type, filename: filename)
           else
             message = shrine_class.opts[:data_uri_error_message]
             message = message.call(uri) if message.respond_to?(:call)
@@ -94,16 +104,16 @@ class Shrine
         # Returns contents of the file base64-encoded.
         def base64
           content = storage.read(id)
-          base64 = Base64.encode64(content)
-          base64.chomp
+          Base64.encode64(content).chomp
         end
       end
 
       class DataFile < StringIO
-        attr_reader :content_type
+        attr_reader :content_type, :original_filename
 
-        def initialize(content, content_type: nil)
+        def initialize(content, content_type: nil, filename: nil)
           @content_type = content_type
+          @original_filename = filename
           super(content)
         end
       end

data/lib/shrine/plugins/delete_promoted.rb

@@ -0,0 +1,21 @@
+class Shrine
+  module Plugins
+    # The delete_promoted plugin deletes files that have been promoted, after
+    # the record is saved. This means that cached files handled by the attacher
+    # will automatically get deleted once they're uploaded to store. This also
+    # applies to any other uploaded file passed to `Attacher#promote`.
+    #
+    #     plugin :delete_promoted
+    module DeletePromoted
+      module AttacherMethods
+        def promote(uploaded_file, *)
+          result = super
+          delete!(uploaded_file, phase: :promote)
+          result
+        end
+      end
+    end
+
+    register_plugin(:delete_promoted, DeletePromoted)
+  end
+end

data/lib/shrine/plugins/delete_raw.rb

@@ -0,0 +1,38 @@
+class Shrine
+  module Plugins
+    # The delete_raw plugin will automatically delete raw files that have been
+    # uploaded. This is especially useful when doing processing, to ensure that
+    # temporary files have been deleted after upload.
+    #
+    #     plugin :delete_raw
+    #
+    # By default any raw file that was uploaded will be deleted, but you can
+    # limit this only to files uploaded to certain storages:
+    #
+    #     plugin :delete_raw, storages: [:store]
+    module DeleteRaw
+      def self.configure(uploader, storages: nil)
+        uploader.opts[:delete_uploaded_storages] = storages
+      end
+
+      module InstanceMethods
+        private
+
+        # Deletes the file that was uploaded, unless it's an UploadedFile.
+        def copy(io, context)
+          super
+          if io.respond_to?(:delete) && !io.is_a?(UploadedFile)
+            io.delete if delete_uploaded?(io)
+          end
+        end
+
+        def delete_uploaded?(io)
+          opts[:delete_uploaded_storages].nil? ||
+          opts[:delete_uploaded_storages].include?(storage_key)
+        end
+      end
+    end
+
+    register_plugin(:delete_raw, DeleteRaw)
+  end
+end