shrine 3.0.1 → 3.3.0

data/doc/metadata.md

@@ -15,6 +15,18 @@ uploaded_file.metadata #=>
 # }
 ```
 
+Under the hood, `Shrine#upload` calls `Shrine#extract_metadata`, which you can
+also use directly to extract metadata from any IO object:
+
+```rb
+uploader.extract_metadata(io) #=>
+# {
+#   "size" => 345993,
+#   "filename" => "matrix.mp4",
+#   "mime_type" => "video/mp4",
+# }
+```
+
 The following metadata is extracted by default:
 
 | Key         | Default source                                     |
@@ -23,7 +35,9 @@ The following metadata is extracted by default:
 | `mime_type` | extracted from `io.content_type`                   |
 | `size`      | extracted from `io.size`                           |
 
-You can access extracted metadata in three ways:
+## Accessing metadata
+
+You can access the stored metadata in three ways:
 
 ```rb
 # via methods (if they're defined)
@@ -42,17 +56,7 @@ uploaded_file["filename"]
 uploaded_file["mime_type"]
 ```
 
-Under the hood, `Shrine#upload` calls `Shrine#extract_metadata`, which you can
-also use directly to extract metadata from any IO object:
-
-```rb
-uploader.extract_metadata(io) #=>
-# {
-#   "size" => 345993,
-#   "filename" => "matrix.mp4",
-#   "mime_type" => "video/mp4",
-# }
-```
+## Controlling extraction
 
 `Shrine#upload` accepts a `:metadata` option which accepts the following values:
 
@@ -85,11 +89,11 @@ By default, the `mime_type` metadata will be copied over from the
 `#content_type` attribute of the input file (if present). However, since
 `#content_type` value comes from the `Content-Type` header of the upload
 request, it's *not guaranteed* to hold the actual MIME type of the file (browser
-determines this header based on file extension). Moreover, only
-`ActionDispatch::Http::UploadedFile`, `Shrine::RackFile`, and
-`Shrine::DataFile` objects have `#content_type` defined, so, when uploading
-simple file objects, `mime_type` will be nil. That makes relying on
-`#content_type` both a security risk and limiting.
+determines this header based on file extension).
+
+Moreover, only `ActionDispatch::Http::UploadedFile`, `Shrine::RackFile`, and
+`Shrine::DataFile` objects have `#content_type` defined, so when uploading
+objects such as `File`, the `mime_type` value will be nil by default.
 
 To remedy that, Shrine comes with a
 [`determine_mime_type`][determine_mime_type] plugin which is able to extract
@@ -112,15 +116,14 @@ You can choose different analyzers, and even mix-and-match them. See the
 
 ## Image Dimensions
 
-Shrine comes with a `store_dimensions` plugin for extracting image dimensions.
-It adds `width` and `height` metadata values, and also adds `#width`,
-`#height`, and `#dimensions` methods to the `Shrine::UploadedFile` object. By
-default, the plugin uses [FastImage] to analyze dimensions, but you can also
-have it use [MiniMagick] or [ruby-vips]:
+Shrine comes with a [`store_dimensions`][store_dimensions] plugin for
+extracting image dimensions. It adds `width` and `height` metadata values, and
+also adds `#width`, `#height`, and `#dimensions` methods to the
+`Shrine::UploadedFile` object.
 
 ```rb
 # Gemfile
-gem "fastimage"
+gem "fastimage" # default analyzer
 ```
 ```rb
 Shrine.plugin :store_dimensions
@@ -136,12 +139,17 @@ uploaded_file.height     #=> 900
 uploaded_file.dimensions #=> [1600, 900]
 ```
 
+By default, the plugin uses [FastImage] to analyze dimensions, but you can also
+have it use [MiniMagick] or [ruby-vips]. See the
+[`store_dimensions`][store_dimensions] plugin docs for more details.
+
 ## Custom metadata
 
 In addition to the built-in metadata, Shrine allows you to extract and store
-any custom metadata, using the `add_metadata` plugin (which extends
-`Shrine#extract_metadata`). For example, you might want to extract EXIF data
-from images:
+any custom metadata, using the [`add_metadata`][add_metadata] plugin (which
+internally extends `Shrine#extract_metadata`).
+
+For example, you might want to extract EXIF data from images:
 
 ```rb
 # Gemfile
@@ -203,9 +211,20 @@ uploaded_file.metadata #=>
 
 The yielded `io` object will not always be an object that responds to `#path`.
 For example, with the `data_uri` plugin the `io` can be a `StringIO` wrapper,
-with `restore_cached_data` or `refresh_metadata` plugins the `io` might be a
-`Shrine::UploadedFile` object. So we're using `Shrine.with_file` to ensure we
-have a file object.
+while with `restore_cached_data` or `refresh_metadata` plugins the `io` might
+be a `Shrine::UploadedFile` object. So, we're using `Shrine.with_file` to
+ensure we have a file object.
+
+### Adding metadata
+
+If you wish to add metadata to an already attached file, you can do it as
+follows:
+
+```rb
+photo.image_attacher.add_metadata("foo" => "bar")
+photo.image.metadata #=> { ..., "foo" => "bar" }
+photo.save # persist changes
+```
 
 ## Metadata columns
 
@@ -225,15 +244,20 @@ photo.image_type #=> "image/jpeg"
 When attaching files that were uploaded directly to the cloud or a [tus
 server], Shrine won't automatically extract metadata from them, instead it will
 copy any existing metadata that was set on the client side. The reason why this
-is the default behaviour is because extracting the metadata would require (at
-least partially) retrieving file content from the storage, which could
-potentially be expensive depending on the storage and the type of metadata
-being extracted.
+is the default behaviour is because metadata extraction requires (at least
+partially) retrieving file content from the storage, which could potentially be
+expensive depending on the storage and the type of metadata being extracted.
+
+```rb
+# no additional metadata will be extracted in this assignment by default
+photo.image = '{"id":"9e6581a4ea1.jpg","storage":"cache","metadata":{...}}'
+```
 
-There are two ways of extracting metadata from directly uploaded files. If you
-want metadata to be automatically extracted on assignment (which is useful if
-you want to validate the extracted metadata or have it immediately available
-for any other reason), you can load the `restore_cached_data` plugin:
+### Extracting on attachment
+
+If you want metadata to be automatically extracted on assignment (which is
+useful if you want to validate the extracted metadata or have it immediately
+available for any other reason), you can load the `restore_cached_data` plugin:
 
 ```rb
 Shrine.plugin :restore_cached_data # automatically extract metadata from cached files on assignment
@@ -248,7 +272,9 @@ photo.image.metadata #=>
 # }
 ```
 
-### Backgrounding
+### Extracting in the background
+
+#### A) Extracting with promotion
 
 If you're using [backgrounding], you can extract metadata during background
 promotion using the `refresh_metadata` plugin (which the `restore_cached_data`
@@ -271,12 +297,14 @@ class PromoteJob
     record         = Object.const_get(record_class).find(record_id) # if using Active Record
 
     attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
-    attacher.refresh_metadata!
+    attacher.refresh_metadata! # extract metadata
     attacher.atomic_promote
   end
 end
 ```
 
+#### B) Extracting separately from promotion
+
 You can also extract metadata in the background separately from promotion:
 
 ```rb
@@ -303,11 +331,13 @@ class MetadataJob
 end
 ```
 
+### Combining foreground and background
+
 If you have some metadata that you want to extract in the foreground and some
 that you want to extract in the background, you can use the uploader context:
 
 ```rb
-class MyUploader < Shrine
+class VideoUploader < Shrine
   plugin :add_metadata
 
   add_metadata do |io, **options|
@@ -324,7 +354,7 @@ class MyUploader < Shrine
 end
 ```
 ```rb
-class MetadataJob
+class PromoteJob
   include Sidekiq::Worker
 
   def perform(attacher_class, record_class, record_id, name, file_data)
@@ -332,12 +362,16 @@ class MetadataJob
     record         = Object.const_get(record_class).find(record_id) # if using Active Record
 
     attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
-    attacher.refresh_metadata!(background: true)
-    attacher.atomic_persist
+    attacher.refresh_metadata!(background: true) # specify the flag
+    attacher.atomic_promote
   end
 end
 ```
 
+Now triggering metadata extraction in the controller on attachment (using
+`restore_cached_data` or `refresh_metadata` plugin) will skip the video
+metadata block, which will be triggered later in the background job.
+
 ### Optimizations
 
 If you want to do both metadata extraction and file processing during
@@ -389,4 +423,6 @@ end
 [ruby-vips]: https://github.com/libvips/ruby-vips
 [tus server]: https://github.com/janko/tus-ruby-server
 [determine_mime_type]: https://shrinerb.com/docs/plugins/determine_mime_type
+[store_dimensions]: https://shrinerb.com/docs/plugins/store_dimensions
+[add_metadata]: https://shrinerb.com/docs/plugins/add_metadata
 [backgrounding]: https://shrinerb.com/docs/plugins/backgrounding

data/doc/multiple_files.md

@@ -14,7 +14,7 @@ relationship with the main table, and files will be attached on the records in
 the new table. That way each record from the main table can implicitly have
 multiple attachments through the associated records.
 
-```plaintext
+```
 album1
   photo1
     - attachment1
@@ -74,13 +74,15 @@ Sequel.migration do
   change do
     create_table :albums do
       primary_key :id
-      column      :title, :text
+
+      String :title
     end
 
     create_table :photos do
       primary_key :id
       foreign_key :album_id, :albums
-      column      :image_data, :text
+
+      String :image_data
     end
   end
 end
@@ -226,7 +228,7 @@ photos_attributes = album_params[:photos_attributes].to_h.merge(new_photos_attri
 album_attributes  = album_params.merge(photos_attributes: photos_attributes)
 
 # create the album with photos
-Album.create(album_params)
+Album.create(album_attributes)
 ```
 
 In this case you need to make sure your form tag has

data/doc/paperclip.md

@@ -1,12 +1,12 @@
 ---
-title: Shrine for Paperclip Users
+title: Upgrading from Paperclip
 ---
 
 This guide is aimed at helping Paperclip users transition to Shrine, and it
 consists of three parts:
 
 1. Explanation of the key differences in design between Paperclip and Shrine
-2. Instructions how to migrate and existing app that uses Paperclip to Shrine
+2. Instructions how to migrate an existing app that uses Paperclip to Shrine
 3. Extensive reference of Paperclip's interface with Shrine equivalents
 
 ## Overview
@@ -136,7 +136,7 @@ into separate columns.
 While Paperclip works only with Active Record, Shrine is designed to integrate
 with any persistence library (there are integrations for [Active
 Record][activerecord], [Sequel][sequel], [ROM][rom], [Hanami][hanami] and
-[Hanami][mongoid]), and can also be used standalone:
+[Mongoid][mongoid]), and can also be used standalone:
 
 ```rb
 attacher = ImageUploader::Attacher.new
@@ -182,7 +182,7 @@ require "image_processing/mini_magick"
 class ImageUploader < Shrine
   plugin :derivatives
 
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(original)
 
     {
@@ -297,16 +297,21 @@ file.mime_type #=> "application/x-php"
 ## Migrating from Paperclip
 
 You have an existing app using Paperclip and you want to transfer it to Shrine.
-First we need to make new uploads write to the `<attachment>_data` column.
-Let's assume we have a `Photo` model with the "image" attachment:
+Let's assume we have a `Photo` model with the "image" attachment.
+
+### 1. Add Shrine column
+
+First we need to create the `image_data` column for Shrine:
 
 ```rb
 add_column :photos, :image_data, :text
 ```
 
-Afterwards we need to make new uploads write to the `image_data` column. This
-can be done by including the below module to all models that have Paperclip
-attachments:
+### 2. Dual write
+
+Next, we need to make new Paperclip attachments write to the `image_data`
+column. This can be done by including the below module to all models that have
+Paperclip attachments:
 
 ```rb
 require "shrine"
@@ -318,8 +323,7 @@ Shrine.storages = {
 
 Shrine.plugin :model
 Shrine.plugin :derivatives
-```
-```rb
+
 module PaperclipShrineSynchronization
   def self.included(model)
     model.before_save do
@@ -336,8 +340,8 @@ module PaperclipShrineSynchronization
     if attachment.size.present?
       attacher.set shrine_file(attachment)
 
-      attachment.styles.each do |name, style|
-        attacher.merge_derivatives(name => shrine_file(style))
+      attachment.styles.each do |style_name, style|
+        attacher.merge_derivatives(style_name => shrine_file(style))
       end
     else
       attacher.set nil
@@ -354,13 +358,14 @@ module PaperclipShrineSynchronization
     end
   end
 
-  # If you'll be using a `:prefix` on your Shrine storage, or you're storing
-  # files on the filesystem, make sure to subtract the appropriate part
-  # from the path assigned to `:id`.
   def shrine_attachment_file(attachment)
+    location = attachment.path
+    # if you're storing files on disk, make sure to subtract the absolute path
+    location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
+
     Shrine.uploaded_file(
       storage:  :store,
-      id:       attachment.path,
+      id:       location,
       metadata: {
         "size"      => attachment.size,
         "filename"  => attachment.original_filename,
@@ -372,13 +377,21 @@ module PaperclipShrineSynchronization
   # If you'll be using a `:prefix` on your Shrine storage, or you're storing
   # files on the filesystem, make sure to subtract the appropriate part
   # from the path assigned to `:id`.
-  def style_to_shrine_data(style)
+  def shrine_style_file(style)
+    location = style.attachment.path(style.name)
+    # if you're storing files on disk, make sure to subtract the absolute path
+    location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
+
     Shrine.uploaded_file(
       storage:  :store,
-      id:       style.attachment.path(style.name),
+      id:       location,
       metadata: {},
     )
   end
+
+  def storage
+    Shrine.storages[:store]
+  end
 end
 ```
 ```rb
@@ -389,34 +402,35 @@ end
 ```
 
 After you deploy this code, the `image_data` column should now be successfully
-synchronized with new attachments.  Next step is to run a script which writes
-all existing Paperclip attachments to `image_data`:
+synchronized with new attachments.
+
+### 3. Data migration
+
+Next step is to run a script which writes all existing Paperclip attachments to
+`image_data`:
 
 ```rb
 Photo.find_each do |photo|
-  Paperclip::AttachmentRegistry.each_definition do |klass, name, options|
-    photo.write_shrine_data(name) if klass == Photo
-  end
+  photo.write_shrine_data(:image)
   photo.save!
 end
 ```
 
+### 4. Rewrite code
+
 Now you should be able to rewrite your application so that it uses Shrine
-instead of Paperclip, using equivalent Shrine storages. For help with
-translating the code from Paperclip to Shrine, you can consult the reference
-below.
+instead of Paperclip (you can consult the reference in the next section). You
+can remove the `PaperclipShrineSynchronization` module as well.
 
-You'll notice that Shrine metadata will be absent from the migrated files' data
-(specifically versions). You can run a script that will fill in any missing
-metadata defined in your Shrine uploader:
+### 5. Remove Paperclip columns
 
-```rb
-Shrine.plugin :refresh_metadata
+If everything is looking good, we can remove Paperclip columns:
 
-Photo.find_each do |photo|
-  photo.image_attacher.refresh_metadata!
-  photo.save
-end
+```rb
+remove_column :photos, :image_file_name
+remove_column :photos, :image_file_size
+remove_column :photos, :image_content_type
+remove_column :photos, :image_updated_at
 ```
 
 ## Paperclip to Shrine direct mapping
@@ -457,8 +471,8 @@ Processing is defined by using the `derivatives` plugin:
 class ImageUploader < Shrine
   plugin :derivatives
 
-  Attacher.derivatives_processor do |original|
-    magick = ImageProcessing::MiniMagick.source(image)
+  Attacher.derivatives do |original|
+    magick = ImageProcessing::MiniMagick.source(original)
 
     {
       large:  magick.resize_to_limit!(800, 800),
@@ -477,8 +491,8 @@ For default URLs you can use the `default_url` plugin:
 class ImageUploader < Shrine
   plugin :default_url
 
-  Attacher.default_url do |options|
-    "/attachments/#{name}/default.jpg"
+  Attacher.default_url do |derivative: nil, **|
+    "/images/placeholders/#{derivative || "original"}.jpg"
   end
 end
 ```
@@ -506,16 +520,79 @@ Alternatively, if you want to generate locations yourself you can override the
 
 ```rb
 class ImageUploader < Shrine
-  def generate_location(io, **options)
-    # ...
+  def generate_location(io, record: nil, name: nil, **)
+    [ storage_key,
+      record && record.class.name.underscore,
+      record && record.id,
+      super,
+      io.original_filename ].compact.join("/")
   end
 end
 ```
+```
+cache/user/123/2feff8c724e7ce17/nature.jpg
+store/user/456/7f99669fde1e01fc/kitten.jpg
+...
+```
 
 #### `:validate_media_type`
 
 Shrine has this functionality in the `determine_mime_type` plugin.
 
+### `validates_attachment`
+
+#### `:presence`
+
+For presence validation you can use your ORM's presence validator:
+
+```rb
+class Photo < ActiveRecord::Base
+  include ImageUploader::Attachment(:image)
+  validates_presence_of :image
+end
+```
+
+#### `:content_type`
+
+You can do MIME type validation with Shrine's `validation_helpers` plugin:
+
+```rb
+class ImageUploader < Shrine
+  plugin :validation_helpers
+
+  Attacher.validate do
+    validate_mime_type %w[image/jpeg image/png image/webp]
+  end
+end
+```
+
+Make sure to also load the `determine_mime_type` plugin to detect MIME type
+from file content.
+
+```rb
+# Gemfile
+gem "mimemagic"
+```
+```rb
+Shrine.plugin :determine_mime_type, analyzer: -> (io, analyzers) do
+  analyzers[:mimemagic].call(io) || analyzers[:file].call(io)
+end
+```
+
+#### `:size`
+
+You can do filesize validation with Shrine's `validation_helpers` plugin:
+
+```rb
+class ImageUploader < Shrine
+  plugin :validation_helpers
+
+  Attacher.validate do
+    validate_max_size 10*1024*1024
+  end
+end
+```
+
 ### `Paperclip::Attachment`
 
 This section explains the equivalent of Paperclip attachment's methods, in