shrine 2.12.0 → 2.13.0

data/doc/metadata.md

@@ -14,8 +14,8 @@ uploaded_file.metadata #=>
 # }
 ```
 
-You can also use `Shrine#extract_metadata` directly to extract metadata from
-any IO object.
+Under the hood `Shrine#extract_metadata` is called, which you can also use
+directly to extract metadata from any IO object.
 
 ```rb
 uploader.extract_metadata(io) #=>
@@ -26,6 +26,15 @@ uploader.extract_metadata(io) #=>
 # }
 ```
 
+Note that you can also manually add or override metadata on upload by passing
+the `:metadata` option to `Shrine#upload`:
+
+```rb
+uploaded_file = uploader.upload(file, metadata: { "filename" => "Matrix[1999].mp4", "foo" => "bar" })
+uploaded_file.original_filename #=> "Matrix[1999].mp4"
+uploaded_file.metadata["foo"]   #=> "bar"
+```
+
 ## MIME type
 
 By default, the `mime_type` metadata will be copied over from the
@@ -108,7 +117,7 @@ uploaded_file.metadata["exif"] #=> {...}
 uploaded_file.exif             #=> {...}
 ```
 
-Of, if you're uploading videos, you might want to extract some video-specific
+Or, if you're uploading videos, you might want to extract some video-specific
 meatadata:
 
 ```rb
@@ -162,17 +171,31 @@ photo = Photo.new(image: file)
 photo.image_type #=> "image/jpeg"
 ```
 
-## Refreshing metadata
-
-When uploading directly to the cloud, the metadata of the original file by
-default won't get extracted on the server side, because your application never
-received the file content.
-
-To have Shrine extra metadata when a cached file is assigned to the attachment
-attribute, it's recommended to load the `restore_cached_data` plugin.
+## Direct uploads
+
+When attaching files that were uploaded directly to the cloud or a [tus
+server], Shrine won't automatically extract metadata from them, instead it will
+copy any existing metadata that was set on the client side. The reason why this
+is the default behaviour is because extracting the metadata would require (at
+least partially) retrieving file content from the storage, which could
+potentially be expensive depending on the storage and the type of metadata
+being extracted.
+
+If you're just attaching files uploaded directly to local disk, there wouldn't
+be any additional performance penalty for extracting metadata. However, if
+you're attaching files uploaded directly to a cloud service like S3, retrieving
+file content for metadata extraction would require an HTTP download. If you're
+using just the `determine_mime_type` plugin, only a small portion of the file
+will be downloaded, so the performance impact might not be so big. But in other
+cases you might have to download the whole file.
+
+There are two ways of extracting metadata from directly uploaded files. If you
+want metadata to be automatically extracted on assignment (which is useful if
+you want to validate the extracted metadata or have it immediately available),
+you can load the `restore_cached_data` plugin:
 
 ```rb
-Shrine.plugin :restore_cached_data # extract metadata from cached files on assingment
+Shrine.plugin :restore_cached_data # automatically extract metadata from cached files on assingment
 ```
 ```rb
 photo.image = '{"id":"ks9elsd.jpg","storage":"cache","metadata":{}}' # metadata is extracted
@@ -184,25 +207,56 @@ photo.image.metadata #=>
 # }
 ```
 
-Extracting metadata from a cached file requires retrieving file content from
-the storage, which might not be desirable depending on your case, that's why
-`restore_cached_data` plugin is not loaded by default. However, Shrine will not
-download the whole file from the storage, instead, it will open a connection to
-the storage, and the metadata analyzers will download how much of the file they
-need. Most MIME type analyzers and the FastImage dimensions analyzer need only
-the first few kilobytes.
-
-You can also extract metadata from an uploaded file explicitly using the
-`refresh_metadata` plugin (which the `restore_cached_data` plugin uses
-internally).
+On the other hand, if you're using backgrounding, you can extract metadata
+during background promotion using the `refresh_metadata` plugin (which the
+`restore_cached_data` plugin uses internally):
 
 ```rb
 Shrine.plugin :refresh_metadata
 ```
 ```rb
-uploaded_file.metadata #=> {}
-uploaded_file.refresh_metadata!
-uploaded_file.metadata #=> {"filename"=>"nature.jpg","size"=>532894,"mime_type"=>"image/jpeg"}
+class MyUploader < Shrine
+  plugin :processing
+
+  # this will be called in the background if using backgrounding plugin
+  process(:store) do |io, context|
+    io.refresh_metadata! # extracts metadata and updates `io.metadata`
+    io
+  end
+end
+```
+
+If you have metadata that is cheap to extract in the foreground, but also have
+additional metadata that can be extracted asynchronously, you can combine the
+two approaches. For example, if you're attaching video files, you might want to
+extract MIME type upfront and video-specific metadata in a background job, which
+can be done as follows (provided that `backgrounding` plugin is used):
+
+```rb
+Shrine.plugin :restore_cached_data
+```
+```rb
+class MyUploader < Shrine
+  plugin :determine_mime_type # this will be called in the foreground
+  plugin :processing
+
+  # this will be called in the background if using backgrounding plugin
+  process(:store) do |io, context|
+    additional_metadata = io.download do |file|
+      # example of metadata extraction
+      movie = FFMPEG::Movie.new(file.path) # uses the streamio-ffmpeg gem
+
+      { "duration"   => movie.duration,
+        "bitrate"    => movie.bitrate,
+        "resolution" => movie.resolution,
+        "frame_rate" => movie.frame_rate }
+    end
+
+    io.metadata.merge!(additional_metadata)
+
+    io
+  end
+end
 ```
 
 [`file`]: http://linux.die.net/man/1/file
@@ -210,4 +264,5 @@ uploaded_file.metadata #=> {"filename"=>"nature.jpg","size"=>532894,"mime_type"=
 [Marcel]: https://github.com/basecamp/marcel
 [FastImage]: https://github.com/sdsykes/fastimage
 [MiniMagick]: https://github.com/minimagick/minimagick
-[ruby-vips]: https://github.com/jcupitt/ruby-vips
+[ruby-vips]: https://github.com/libvips/ruby-vips
+[tus server]: https://github.com/janko-m/tus-ruby-server

data/doc/multiple_files.md

@@ -65,7 +65,7 @@ Let's create a table for the main resource and attachments, and add a foreign
 key in the attachment table for the main table:
 
 ```rb
-# Sequel
+# with Sequel:
 Sequel.migration do
   change do
     create_table :albums do
@@ -81,7 +81,7 @@ Sequel.migration do
   end
 end
 
-# Active Record
+# with Active Record:
 class CreateAlbumsAndPhotos < ActiveRecord::Migration
   def change
     create_table :albums do |t|
@@ -102,12 +102,12 @@ In the Photo model, create a Shrine attachment attribute named `image`
 (`:image` matches the `_data` column prefix above):
 
 ```rb
-# Sequel
+# with Sequel:
 class Photo < Sequel::Model
   include ImageUploader::Attachment.new(:image)
 end
 
-# Active Record
+# with Active Record:
 class Photo < ActiveRecord::Base
   include ImageUploader::Attachment.new(:image)
 end
@@ -121,7 +121,7 @@ relationship to the photos table, and allow it to directly accept attributes
 for the associated photo records by enabling nested attributes:
 
 ```rb
-# Sequel
+# with Sequel:
 class Album < Sequel::Model
   one_to_many :photos
   plugin :association_dependencies, photos: :destroy # destroy photos when album is destroyed
@@ -130,7 +130,7 @@ class Album < Sequel::Model
   nested_attributes :photos, destroy: true
 end
 
-# Active Record
+# with Active Record:
 class Album < ActiveRecord::Base
   has_many :photos, dependent: :destroy
   accepts_nested_attributes_for :photos, allow_destroy: true
@@ -141,6 +141,7 @@ Documentation on nested attributes:
 
 * [`Sequel::Model.nested_attributes`]
 * [`ActiveRecord::Base.accepts_nested_attributes_for`]
+* [`Mongoid::Document.accepts_nested_attributes_for`]
 
 ### 3. Create the View
 
@@ -165,7 +166,7 @@ form @album, action: "/photos", enctype: "multipart/form-data" do |f|
 end
 
 # with Rails form builder:
-form_for @album do |f|
+form_for @album, html: { enctype: "multipart/form-data" } do |f|
   f.text_field :title
   f.fields_for :photos do |p| # adds new `album[photos_attributes]` parameter
     p.hidden_field :image, value: p.object.cached_image_data
@@ -179,37 +180,69 @@ end
 
 In your controller you should still be able to assign all the attributes to the
 album, just remember to whitelist the new parameter for the nested attributes,
-in this case `photos_attributes`.
+in this case `album[photos_attributes]`.
 
-### 4. Direct Upload
+### 4a. Form upload
 
-On the client side, you can asynchronously upload each of the files to a direct
-upload endpoint as soon as they're selected. There are two methods of
-implementing direct uploads: upload to your app using the [`upload_endpoint`]
-plugin or upload to directly to storage like S3 using the [`presign_endpoint`]
-plugin. It's recommended to use [Uppy] to handle the uploads on the client side.
+If you would like to avoid writing JavaScript, you can have selected files
+uploaded via the form, and then in your controller you can assign them in the
+format of nested attributes. You'll want to merge these newly uploaded photos
+with the existing photos params that will come from the nested form fields.
+This could look something like this:
+
+```rb
+# In your controller:
+
+# transform the list of uploaded files into a photos attributes hash
+new_photos_attributes = params[:files].inject({}) do |hash, file|
+  hash.merge!(SecureRandom.hex => { image: file })
+end
+
+# merge new photos attributes with existing (`album_params` is whitelisted `params[:album]`)
+photos_attributes = album_params[:photos_attributes].to_h.merge(new_photos_attributes)
+album_attributes  = album_params.merge(photos_attributes: photos_attributes)
+
+# create the album with photos
+Album.create(album_params)
+```
+
+In this case you need to make sure your form tag has
+`enctype="multipart/form-data"` attributes specified, otherwise form upload
+won't succeed.
+
+### 4b. Direct upload
+
+Alternatively, you can have selected files immediately uploaded to a direct
+upload endpoint, as soon as they're selected. It's recommended to use [Uppy]
+for handling client side file uploads. There are two methods of implementing
+direct uploads: to your app using the [`upload_endpoint`] plugin, or directly
+to a storage like S3 using the [`presign_endpoint`] plugin. See the
+walkthroughs for setting up simple [direct app uploads] and [direct S3
+uploads], as well as the [Roda][roda demo] or [Rails][rails demo] demo app
+which implement multiple file uploads.
 
 Once files are uploaded asynchronously, you can dynamically insert photo
-attachment fields for the `image` attribute to the form filled with uploaded
-file data, so that the corresponding photo records are automatically created
-after form submission. The hidden attachment fields should contain uploaded
-file data in JSON format, just like when doing single direct uploads. The
-attachment field names should be namespaced according to the convention that
-the nested attributes feature expects. In this case it should be
-`album[photos_attributes][<idx>]`, where `<idx>` is any incrementing integer
-(e.g. you can use the current UNIX timestamp).
+attachment fields for the `image` attachment attribute into the form, where the
+hidden field is filled with uploaded file data in JSON format, just like when
+doing single direct uploads. The attachment field names should be namespaced
+according to the convention that the nested attributes feature expects. In this
+case it should be `album[photos_attributes][<uid>]`, where `<uid>` is any
+unique string.
 
 ```rb
 # naming format in which photos fields should be generated and submitted
 album[photos_attributes][11111][image] = '{"id":"38k25.jpg","storage":"cache","metadata":{...}}'
 album[photos_attributes][29323][image] = '{"id":"sg0fg.jpg","storage":"cache","metadata":{...}}'
 album[photos_attributes][34820][image] = '{"id":"041jd.jpg","storage":"cache","metadata":{...}}'
-# ...
 ```
 
-See the walkthroughs for setting up simple [direct app uploads] and
-[direct S3 uploads], and the [Roda][roda demo] or [Rails][rails demo] demo app
-which implements multiple file uploads.
+In your controller you don't need to make any changes to handle the nested
+photos attributes (other than allowing them), your ORM will take care of
+creating, updating, or removing the associated photos.
+
+```rb
+Album.create(album_params) # or `params[:album]`
+```
 
 ### 5. Adding Validations
 
@@ -229,28 +262,38 @@ class ImageUploader < Shrine
 end
 ```
 ```rb
-# Sequel
+# with Sequel:
 class Album < Sequel::Model
   # ... (nested_attributes already enables validating associated photos) ...
 end
 
-# ActiveRecord
+# with ActiveRecord:
 class Album < ActiveRecord::Base
   # ...
   validates_associated :photos
 end
 ```
 
-### 6. Conclusion
+Note that by default only metadata set on the client side will be available for
+validations. Shrine will not automatically run metadata extraction for directly
+uploaded files, as this would require (at least partially) downloading each
+file from storage, which could be an unwanted performance penalty. If you want
+metadata to be extracted on the server side, you can load the
+`restore_cached_data` plugin which will make metadata extraction run on
+assignment, or you can extract the metadata manually (e.g. in a background job)
+using the `refresh_metadata` plugin.
+
+### Conclusion
 
 Now we have a simple interface for accepting multiple attachments, which
-internally uses nested attributes to create multiple associated records, each
-with a single attachment. After creation you can also add new attachments, or
-update and delete existing ones, which the nested attributes feature gives you
-for free.
+internally uses the nested attributes feature of your ORM to create multiple
+associated records, each with a single attachment. After creation you can also
+add new attachments, or update and delete existing ones, which the nested
+attributes feature gives you for free.
 
 [`Sequel::Model.nested_attributes`]: http://sequel.jeremyevans.net/rdoc-plugins/classes/Sequel/Plugins/NestedAttributes.html
 [`ActiveRecord::Base.accepts_nested_attributes_for`]: http://api.rubyonrails.org/classes/ActiveRecord/NestedAttributes/ClassMethods.html
+[`Mongoid::Document.accepts_nested_attributes_for`]: https://docs.mongodb.com/mongoid/master/tutorials/mongoid-nested-attributes/
 [`upload_endpoint`]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/UploadEndpoint.html
 [`presign_endpoint`]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/PresignEndpoint.html
 [Uppy]: https://uppy.io

data/doc/processing.md

@@ -253,15 +253,6 @@ generic gem.
 To demonstrate, here is an example of transcoding videos using
 [streamio-ffmpeg]:
 
-```sh
-$ brew install ffmpeg
-```
-
-```rb
-# Gemfile
-gem "streamio-ffmpeg"
-```
-
 ```rb
 require "streamio-ffmpeg"
 
@@ -404,8 +395,8 @@ photo.image.url(width: 100, height: 100, crop: :fit)
 [`ImageProcessing::MiniMagick`]: https://github.com/janko-m/image_processing/blob/master/doc/minimagick.md
 [ImageMagick]: https://www.imagemagick.org
 [GraphicsMagick]: http://www.graphicsmagick.org
-[libvips]: http://jcupitt.github.io/libvips/
-[Why is libvips quick]: https://github.com/jcupitt/libvips/wiki/Why-is-libvips-quick
+[libvips]: http://libvips.github.io/libvips/
+[Why is libvips quick]: https://github.com/libvips/libvips/wiki/Why-is-libvips-quick
 [ImageOptim.com]: https://imageoptim.com/api
 [Down]: https://github.com/janko-m/down
 [HTTP.rb]: https://github.com/httprb/http

data/doc/testing.md

@@ -54,8 +54,8 @@ Shrine.storages = {
 }
 ```
 
-If you're using AWS S3 storage, in you can switch to using [Minio] (explained
-below), both in test and development environment. Alternatively, you can [stub
+If you're using AWS S3 storage, you can use [Minio] (explained below) instead
+of S3, both in test and development environment. Alternatively, you can [stub
 aws-sdk-s3 requests][aws-sdk-ruby stubs] in tests.
 
 ### Minio

data/lib/shrine.rb

@@ -13,7 +13,8 @@ class Shrine
   # Raised when a file is not a valid IO.
   class InvalidFile < Error
     def initialize(io, missing_methods)
-      super "#{io.inspect} is not a valid IO object (it doesn't respond to #{missing_methods.map{|m, args|"##{m}"}.join(", ")})"
+      super "#{io.inspect} is not a valid IO object (it doesn't respond to \
+        #{missing_methods.map{|m, _|"##{m}"}.join(", ")})"
     end
   end
 
@@ -218,7 +219,7 @@ class Shrine
 
         # The main method for uploading files. Takes an IO-like object and an
         # optional context hash (used internally by Shrine::Attacher). It calls
-        # user-defined #process, and aferwards it calls #store. The `io` is
+        # user-defined #process, and afterwards it calls #store. The `io` is
         # closed after upload.
         def upload(io, context = {})
           io = processed(io, context) || io
@@ -311,7 +312,10 @@ class Shrine
         # file that was uploaded.
         def _store(io, context)
           _enforce_io(io)
+
           metadata = get_metadata(io, context)
+          metadata = metadata.merge(context[:metadata]) if context[:metadata]
+
           location = get_location(io, context.merge(metadata: metadata))
 
           put(io, context.merge(location: location, metadata: metadata))
@@ -527,21 +531,22 @@ class Shrine
         # already cached file as a JSON string, otherwise it assumes that it's
         # an IO object and uploads it to the temporary storage. The cached file
         # is then written to the attachment attribute in the JSON format.
-        def assign(value)
+        def assign(value, **options)
           if value.is_a?(String)
             return if value == "" || !cache.uploaded?(uploaded_file(value))
             assign_cached(uploaded_file(value))
           else
-            uploaded_file = cache!(value, action: :cache) if value
+            uploaded_file = cache!(value, action: :cache, **options) if value
             set(uploaded_file)
           end
         end
 
-        # Accepts a Shrine::UploadedFile object and writes is to the attachment
+        # Accepts a Shrine::UploadedFile object and writes it to the attachment
         # attribute. It then runs file validations, and records that the
         # attachment has changed.
         def set(uploaded_file)
-          @old = get unless uploaded_file == get
+          file = get
+          @old = file unless uploaded_file == file
           _set(uploaded_file)
           validate
         end
@@ -601,7 +606,8 @@ class Shrine
         # Deletes the current attachment, typically called after destroying the
         # record.
         def destroy
-          _delete(get, action: :destroy) if get && !cache.uploaded?(get)
+          file = get
+          _delete(file, action: :destroy) if file && !cache.uploaded?(file)
         end
 
         # Delegates to #delete!, overriden for backgrounding.
@@ -617,12 +623,14 @@ class Shrine
 
         # Returns true if attachment is present and cached.
         def cached?
-          get && cache.uploaded?(get)
+          file = get
+          file && cache.uploaded?(file)
         end
 
         # Returns true if attachment is present and stored.
         def stored?
-          get && store.uploaded?(get)
+          file = get
+          file && store.uploaded?(file)
         end
 
         # Returns a Shrine::UploadedFile instantiated from the data written to
@@ -681,7 +689,7 @@ class Shrine
           set(cached_file)
         end
 
-        # Writes the uploaded file the attachment attribute. Overriden in ORM
+        # Writes the uploaded file to the attachment attribute. Overriden in ORM
         # plugins to additionally save the model instance.
         def update(uploaded_file)
           _set(uploaded_file)