shrine 2.4.1 → 2.5.0

data/doc/carrierwave.md

@@ -430,8 +430,10 @@ For default URLs you can use the `default_url` plugin:
 
 ```rb
 class ImageUploader < Shrine
-  plugin :default_url do |context|
-    "/attachments/#{context[:name]}/default.jpg"
+  plugin :default_url
+
+  Attacher.default_url do |options|
+    "/attachments/#{name}/default.jpg"
   end
 end
 ```

data/doc/direct_s3.md

@@ -23,6 +23,7 @@ You can start by setting both temporary and permanent storage to S3 with
 different prefixes (or even buckets):
 
 ```rb
+# Gemfile
 gem "aws-sdk", "~> 2.1"
 ```
 ```rb
@@ -86,6 +87,7 @@ server, and use this information to start uploading the file to S3. The
 in our application:
 
 ```rb
+# Gemfile
 gem "roda"
 ```
 ```rb
@@ -185,13 +187,30 @@ With direct uploads any metadata has to be extracted on the client, since
 caching the file doesn't touch your application. When the cached file is stored,
 Shrine's default behaviour is to simply copy over cached file's metadata.
 
-If you want to extract metadata on the server before storing, you can just
-load the `restore_cached_data` plugin.
+If you want to re-extract metadata on the server before file validation, you
+can load the `restore_cached_data`. That will make Shrine open the S3 file for
+reading, give it for metadata extraction, and then override the metadata
+received from the client with one extracted by Shrine.
 
 ```rb
 plugin :restore_cached_data
 ```
 
+Note that if you don't need this metadata before file validation, and you would
+like to have it extracted in a background job, you can do the following trick:
+
+```rb
+class MyUploader < Shrine
+  plugin :processing
+
+  process(:store) do |io, context|
+    real_metadata = io.open { |opened_io| extract_metadata(opened_io, context) }
+    io.metadata.update(real_metadata)
+    io # return the same cached IO
+  end
+end
+```
+
 ## Clearing cache
 
 Since directly uploaded files will stay in your temporary storage, you will

data/doc/multiple_files.md

@@ -0,0 +1,118 @@
+# Multiple Files
+
+There are often times when you want to allow users to attach multiple files to
+a single resource. Some file attachment libraries provide a special interface
+for multiple attachments, but Shrine doesn't come with one, because it's much
+more robust and flexible to implement this using your ORM directly.
+
+The idea is to create a new table, and attach each uploaded file to a separate
+record on that table, while having a "many-to-one" relationship with the main
+table. That way a database record from the main table can implicitly have
+multiple attachments through the associated records.
+
+```
+album
+  photo1
+    - attachment1
+  photo2
+    - attachment2
+  photo3
+    - attachment3
+```
+
+This design gives you great flexibility, allowing you to support:
+
+* adding new attachments
+* updating existing attachments
+* removing existing attachments
+* sorting attachments
+* having additional fields on attachments (captions, votes, number of downloads etc.)
+* ...
+
+If you're using Sequel or ActiveRecord, the easiest way to implement this is
+via nested attributes, which you would in general use for any dynamic
+"one-to-many" association. The examples will be using Sequel, but with
+ActiveRecord it's very similar, here are the docs:
+
+* [`Sequel::Model.nested_attributes`]
+* [`ActiveRecord::Base.accepts_nested_attributes_for`]
+
+For simplicity, for the rest of this guide we will assume that we have "albums"
+that can have multiple "photos".
+
+## 1. Attachments table
+
+Let's create a table for our attachments, and add a foreign key for the main table:
+
+```rb
+Sequel.migration do
+  change do
+    create_table :photos do
+      primary_key :id
+      foreign_key :album_id, :albums
+      column      :image_data, :text
+    end
+  end
+end
+```
+
+In our new model we can create a Shrine attachment attribute:
+
+```rb
+class Photo < Sequel::Model
+  include ImageUploader[:image]
+end
+```
+
+## 2. Nested attributes
+
+In our main model we can now declare the association to the new table, and
+allow it to directly accept attributes for the associated records:
+
+```rb
+class Album < Sequel::Model
+  one_to_many :photos
+
+  plugin :nested_attributes # load the plugin
+  nested_attributes :photos
+end
+```
+
+## 3. View
+
+In order to allow the user to select multiple files in the form, we just need
+to add the `multiple` attribute to the file field.
+
+```html
+<input type="file" multiple name="file">
+```
+
+You can then use a generic JavaScript file upload library like
+[jQuery-File-Upload], [Dropzone] or [FineUploader] to asynchronously upload
+each the selected files to your app or an external service. See the
+`direct_upload` plugin, and [Direct Uploads to S3] guide for more details.
+
+After each upload finishes, you can generate a nested hash for the new
+associated record, and write the uploaded file JSON to the attachment field:
+
+```rb
+album[photos_attributes][0][image] = '{"storage":"cache","id":"38k25.jpg","metadata":{...}}'
+album[photos_attributes][1][image] = '{"storage":"cache","id":"sg0fg.jpg","metadata":{...}}'
+album[photos_attributes][2][image] = '{"storage":"cache","id":"041jd.jpg","metadata":{...}}'
+```
+
+Once you submit this to the app, the ORM's nested attributes behaviour will
+create the associated records, and assign the Shrine attachments.
+
+Now you can manage adding new, or updating and deleting existing attachments,
+just by using your ORM's nested attributes behaviour, the same way that you
+would do with any other dynamic one-to-many association. The callbacks that
+are added by including the Shrine module to the associated model will
+automatically take care of the attachment management.
+
+[`Sequel::Model.nested_attributes`]: http://sequel.jeremyevans.net/rdoc-plugins/classes/Sequel/Plugins/NestedAttributes.html
+[`ActiveRecord::Base.accepts_nested_attributes_for`]: http://api.rubyonrails.org/classes/ActiveRecord/NestedAttributes/ClassMethods.html
+[jQuery-File-Upload]: https://github.com/blueimp/jQuery-File-Upload
+[Dropzone]: https://github.com/enyo/dropzone
+[FineUploader]: https://github.com/FineUploader/fine-uploader
+[Direct Uploads to S3]: http://shrinerb.com/rdoc/files/doc/direct_s3_md.html

data/doc/paperclip.md

@@ -425,8 +425,10 @@ For default URLs you can use the `default_url` plugin:
 
 ```rb
 class ImageUploader < Shrine
-  plugin :default_url do |context|
-    "/attachments/#{context[:name]}/default.jpg"
+  plugin :default_url
+
+  Attacher.default_url do |options|
+    "/attachments/#{name}/default.jpg"
   end
 end
 ```

data/doc/testing.md

@@ -38,11 +38,12 @@ end
 
 If you're using an external storage in development, it is common in tests to
 switch to a filesystem storage. However, that means that you'll also have to
-clean up the test directory between tests, and writing to filesystem a lot can
-affect the performance of your tests.
+clean up the test directory between tests, and writing to filesystem can affect
+the performance of your tests.
 
-Instead of filesystem you can use [memory storage][shrine-memory], which is
-both faster and doesn't require you to clean up anything between tests.
+If your tests are run in a single process, instead of filesystem you can use
+[memory storage][shrine-memory], which is both faster and doesn't require you
+to clean up anything between tests.
 
 ```rb
 gem "shrine-memory"
@@ -57,6 +58,17 @@ Shrine.storages = {
 }
 ```
 
+Alternatively, if you're using Amazon S3 storage, in tests (and development)
+you can swap it out for [FakeS3]. You just need tell aws-sdk that instead of
+`s3.amazonaws.com` it should use the host of your FakeS3 server when generating
+URLs.
+
+```rb
+Shrine::Storage::S3.new(endpoint: "http://localhost:10000")
+```
+
+Note that for using FakeS3 you need aws-sdk version 2.2.25 or higher.
+
 ## Test data
 
 If you're creating test data dynamically using libraries like [factory_girl],
@@ -219,7 +231,7 @@ end
 However, it's even better to design your processing code in such a way that
 it's easier to swap out in tests. In your *application* code you could extract
 processing into a single `#call`-able object, and register it inside uploader
-generic `#opts` hash.
+generic `opts` hash.
 
 ```rb
 class ImageUploader < Shrine
@@ -264,3 +276,4 @@ provided by the `direct_upload` app mounted in your routes.
 [`#attach_file`]: http://www.rubydoc.info/github/jnicklas/capybara/master/Capybara/Node/Actions#attach_file-instance_method
 [Rack::Test]: https://github.com/brynary/rack-test
 [Rack::TestApp]: https://github.com/kwatch/rack-test_app
+[FakeS3]: https://github.com/jubos/fake-s3

data/lib/shrine.rb

@@ -710,9 +710,11 @@ class Shrine
 
         # Initializes the uploaded file with the given data hash.
         def initialize(data)
+          raise Error, "#{data.inspect} isn't valid uploaded file data" unless data["id"] && data["storage"]
+
           @data = data
           @data["metadata"] ||= {}
-          storage # ensure storage exists
+          storage # ensure storage is registered
         end
 
         # The location where the file was uploaded to the storage.

data/lib/shrine/plugins/add_metadata.rb

@@ -5,39 +5,77 @@ class Shrine
     #
     #     plugin :add_metadata
     #
-    #     add_metadata :exif do |io, context|
-    #       MiniMagick::Image.new(io.path).exif
+    #     add_metadata :pages do |io, context|
+    #       PDF::Reader.new(io.path).page_count
     #     end
     #
-    # The above will add "exif" to the metadata hash, and also add the `#exif`
-    # method to the `UploadedFile`:
+    # The above will add "pages" to the metadata hash, and also create the
+    # `#pages` reader method on Shrine::UploadedFile.
     #
-    #     uploaded_file.metadata["exif"]
+    #     document.metadata["pages"]
     #     # or
-    #     uploaded_file.exif
+    #     document.pages
+    #
+    # You can also extract multiple metadata values at once, by using
+    # `add_metadata` without an argument.
+    #
+    #     add_metadata do |io, context|
+    #       movie = FFMPEG::Movie.new(io.path)
+    #
+    #       { "duration"   => movie.duration,
+    #         "bitrate"    => movie.bitrate,
+    #         "resolution" => movie.resolution,
+    #         "frame_rate" => movie.frame_rate }
+    #     end
+    #
+    # In this case Shrine won't automatically create reader methods for the
+    # extracted metadata on Shrine::UploadedFile, but you can create them via
+    # `metadata_method`.
+    #
+    #     metadata_method :duration, :bitrate, :resolution, :frame_rate
     module AddMetadata
       def self.configure(uploader)
-        uploader.opts[:metadata] = {}
+        uploader.opts[:metadata] = []
       end
 
       module ClassMethods
-        def add_metadata(name, &block)
-          opts[:metadata][name] = block
+        def add_metadata(name = nil, &block)
+          if name
+            opts[:metadata] << _metadata_proc(name, &block)
+            metadata_method(name)
+          else
+            opts[:metadata] << block
+          end
+        end
 
+        def metadata_method(*names)
+          names.each { |name| _metadata_method(name) }
+        end
+
+        private
+
+        def _metadata_method(name)
           self::UploadedFile.send(:define_method, name) do
             metadata[name.to_s]
           end
         end
+
+        def _metadata_proc(name, &block)
+          proc do |io, context|
+            value = instance_exec(io, context, &block)
+            {name.to_s => value} unless value.nil?
+          end
+        end
       end
 
       module InstanceMethods
         def extract_metadata(io, context)
           metadata = super
 
-          opts[:metadata].each do |name, block|
-            value = instance_exec(io, context, &block)
-            metadata[name.to_s] = value unless value.nil?
+          opts[:metadata].each do |metadata_block|
+            custom_metadata = instance_exec(io, context, &metadata_block)
             io.rewind
+            metadata.merge!(custom_metadata) unless custom_metadata.nil?
           end
 
           metadata

data/lib/shrine/plugins/backgrounding.rb

@@ -1,18 +1,25 @@
 class Shrine
   module Plugins
-    # The `backgrounding` plugin enables you to move processing, storing and
-    # deleting of files from record's lifecycle into background jobs. This is
-    # generally useful if you're doing processing and/or your store is
-    # something other than Storage::FileSystem.
+    # The `backgrounding` plugin enables you to move promoting and deleting of
+    # files from record's lifecycle into background jobs. This is especially
+    # useful if you're doing processing and/or you're storing files on an
+    # external storage service.
+    #
+    #     plugin :backgrounding
+    #
+    # ## Usage
+    #
+    # The plugin provides `Attacher.promote` and `Attacher.delete` methods,
+    # which allow you to hook up to promoting and deleting and spawn background
+    # jobs, by passing a block.
     #
-    #     Shrine.plugin :backgrounding
     #     Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
     #     Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
     #
-    # The `data` variable is a serializable hash containing all context needed
-    # for promotion/deletion. You then just need to declare `PromoteJob` and
-    # `DeleteJob`, and call `Shrine::Attacher.promote`/`Shrine::Attacher.delete`
-    # with the data hash:
+    # The yielded `data` variable is a serializable hash containing all context
+    # needed for promotion/deletion. Now you just need to declare the job
+    # classes, and inside them call `Attacher.promote` or `Attacher.delete`,
+    # this time with the received data.
     #
     #     class PromoteJob
     #       include Sidekiq::Worker
@@ -30,17 +37,60 @@ class Shrine
     #       end
     #     end
     #
-    # Internally these methods will resolve all necessary objects, do the
-    # promotion/deletion, and in case of promotion update the record with the
-    # stored attachment.
+    # This example used Sidekiq, but obviously you could just as well use
+    # any other backgrounding library. This setup will be applied globally for
+    # all uploaders.
+    #
+    # If you're generating versions, and you want to process some versions in
+    # the foreground before kicking off a background job, you can use the
+    # `recache` plugin.
+    #
+    # ## `Attacher.promote` and `Attacher.delete`
+    #
+    # In background jobs, `Attacher.promote` and `Attacher.delete` will resolve
+    # all necessary objects, and do the promotion/deletion. If
+    # `Attacher.find_record` is defined (which comes with ORM plugins), model
+    # instances will be treated as database records, with the `#id` attribute
+    # assumed to represent the primary key. Then promotion will have the
+    # following behaviour:
     #
-    # The examples above used Sidekiq, but obviously you can just as well use
-    # any other backgrounding library. This setup will work globally for all
-    # uploaders.
+    # 1. retrieves the database record
+    #     * if record is not found, it finishes
+    #     * if record is found but attachment has changed, it finishes
+    # 2. uploads cached file to permanent storage
+    # 3. reloads the database record
+    #     * if record is not found, it deletes the promoted files and finishes
+    #     * if record is found but attachment has changed, it deletes the promoted files and finishes
+    # 4. updates the record with the promoted files
     #
-    # The `backgrounding` plugin affects the `Shrine::Attacher` in a way that
-    # `#_promote` and `#_delete` spawn background jobs, while `#promote` and
-    # `#delete!` are always synchronous:
+    # Both `Attacher.promote` and `Attacher.delete` return a `Shrine::Attacher`
+    # instance (if the action hasn't aborted), so you can use it to perform
+    # additional tasks:
+    #
+    #     def perform(data)
+    #       attacher = Shrine::Attacher.promote(data)
+    #       attacher.record.update(published: true) if attacher && attacher.record.is_a?(Post)
+    #     end
+    #
+    # ### Plain models
+    #
+    # You can also do backgrounding with plain models which don't represent
+    # database records; the plugin will use that mode if `Attacher.find_record`
+    # is not defined. In that case promotion will have the following behaviour:
+    #
+    # 1. instantiates the model
+    # 2. uploads cached file to permanent storage
+    # 3. writes promoted files to the model instance
+    #
+    # You can then retrieve the promoted files via the attacher object that
+    # `Attacher.promote` returns, and do any additional tasks if you need to.
+    #
+    # ## `Attacher#_promote` and `Attacher#_delete`
+    #
+    # The plugin modifies `Attacher#_promote` and `Attacher#_delete` to call
+    # the registered blocks with serializable attacher data, and these methods
+    # are internally called by the attacher. `Attacher#promote` and
+    # `Attacher#delete!` remain synchronous.
     #
     #     # asynchronous (spawn background jobs)
     #     attacher._promote
@@ -50,38 +100,23 @@ class Shrine
     #     attacher.promote
     #     attacher.delete!(attachment)
     #
-    # Both methods return the `Shrine::Attacher` instance (if the action didn't
-    # abort), so you can use it to do additional actions:
+    # ## `Attacher.dump` and `Attacher.load`
     #
-    #     def perform(data)
-    #       attacher = Shrine::Attacher.promote(data)
-    #       attacher.record.update(published: true) if attacher && attacher.record.is_a?(Post)
-    #     end
+    # The plugin adds `Attacher.dump` and `Attacher.load` methods for
+    # serializing attacher object and loading it back up. You can use them to
+    # spawn background jobs for custom tasks.
     #
-    # You can also write custom background jobs with `Attacher.dump` and
-    # `Attacher.load`:
+    #     data = Shrine::Attacher.dump(attacher)
+    #     SomethingJob.perform_async(data)
     #
-    #     class User < Sequel::Model
-    #       def after_commit
-    #         super
-    #         if some_condition
-    #           data = Shrine::Attacher.dump(avatar_attacher)
-    #           SomethingJob.perform_async(data)
-    #         end
-    #       end
-    #     end
+    #     # ...
     #
     #     class SomethingJob
-    #       include Sidekiq::Worker
     #       def perform(data)
     #         attacher = Shrine::Attacher.load(data)
     #         # ...
     #       end
     #     end
-    #
-    # If you're generating versions, and you want to process some versions in
-    # the foreground before kicking off a background job, you can use the
-    # `recache` plugin.
     module Backgrounding
       module AttacherClassMethods
         # If block is passed in, stores it to be called on promotion. Otherwise
@@ -125,15 +160,7 @@ class Shrine
         # Loads the data created by #dump, resolving the record and returning
         # the attacher.
         def load(data)
-          record_class, record_id = data["record"]
-          record_class = Object.const_get(record_class)
-
-          record   = find_record(record_class, record_id)
-          record ||= record_class.new.tap do |instance|
-            # so that the id is always included in file deletion logs
-            instance.singleton_class.send(:define_method, :id) { record_id }
-          end
-
+          record = load_record(data)
           name = data["name"].to_sym
 
           if data["shrine_class"]
@@ -146,6 +173,29 @@ class Shrine
 
           attacher
         end
+
+        # Resolves the record from backgrounding data. If the record was found,
+        # returns it. If the record wasn't found, returns an instance of the
+        # model with ID assigned for logging. If `find_record` isn't defined,
+        # then it is a PORO model and should be instantiated with the cached
+        # attachment.
+        def load_record(data)
+          record_class, record_id = data["record"]
+          record_class = Object.const_get(record_class)
+
+          if respond_to?(:find_record)
+            record   = find_record(record_class, record_id)
+            record ||= record_class.new.tap do |instance|
+              # so that the id is always included in file deletion logs
+              instance.singleton_class.send(:define_method, :id) { record_id }
+            end
+          else
+            record = record_class.new
+            record.send(:"#{data["name"]}_data=", data["attachment"])
+          end
+
+          record
+        end
       end
 
       module AttacherMethods
@@ -193,8 +243,10 @@ class Shrine
 
         # Updates with the new file only if the attachment hasn't changed.
         def swap(new_file)
-          reloaded = self.class.find_record(record.class, record.id)
-          return if reloaded.nil? || self.class.new(reloaded, name).read != read
+          if self.class.respond_to?(:find_record)
+            reloaded = self.class.find_record(record.class, record.id)
+            return if reloaded.nil? || self.class.new(reloaded, name).read != read
+          end
           super
         end
       end