shrine 3.1.0 → 3.4.0

data/doc/metadata.md

@@ -15,6 +15,18 @@ uploaded_file.metadata #=>
 # }
 ```
 
+Under the hood, `Shrine#upload` calls `Shrine#extract_metadata`, which you can
+also use directly to extract metadata from any IO object:
+
+```rb
+uploader.extract_metadata(io) #=>
+# {
+#   "size" => 345993,
+#   "filename" => "matrix.mp4",
+#   "mime_type" => "video/mp4",
+# }
+```
+
 The following metadata is extracted by default:
 
 | Key         | Default source                                     |
@@ -23,7 +35,9 @@ The following metadata is extracted by default:
 | `mime_type` | extracted from `io.content_type`                   |
 | `size`      | extracted from `io.size`                           |
 
-You can access extracted metadata in three ways:
+## Accessing metadata
+
+You can access the stored metadata in three ways:
 
 ```rb
 # via methods (if they're defined)
@@ -42,17 +56,7 @@ uploaded_file["filename"]
 uploaded_file["mime_type"]
 ```
 
-Under the hood, `Shrine#upload` calls `Shrine#extract_metadata`, which you can
-also use directly to extract metadata from any IO object:
-
-```rb
-uploader.extract_metadata(io) #=>
-# {
-#   "size" => 345993,
-#   "filename" => "matrix.mp4",
-#   "mime_type" => "video/mp4",
-# }
-```
+## Controlling extraction
 
 `Shrine#upload` accepts a `:metadata` option which accepts the following values:
 
@@ -85,11 +89,11 @@ By default, the `mime_type` metadata will be copied over from the
 `#content_type` attribute of the input file (if present). However, since
 `#content_type` value comes from the `Content-Type` header of the upload
 request, it's *not guaranteed* to hold the actual MIME type of the file (browser
-determines this header based on file extension). Moreover, only
-`ActionDispatch::Http::UploadedFile`, `Shrine::RackFile`, and
-`Shrine::DataFile` objects have `#content_type` defined, so, when uploading
-simple file objects, `mime_type` will be nil. That makes relying on
-`#content_type` both a security risk and limiting.
+determines this header based on file extension).
+
+Moreover, only `ActionDispatch::Http::UploadedFile`, `Shrine::RackFile`, and
+`Shrine::DataFile` objects have `#content_type` defined, so when uploading
+objects such as `File`, the `mime_type` value will be nil by default.
 
 To remedy that, Shrine comes with a
 [`determine_mime_type`][determine_mime_type] plugin which is able to extract
@@ -112,15 +116,14 @@ You can choose different analyzers, and even mix-and-match them. See the
 
 ## Image Dimensions
 
-Shrine comes with a `store_dimensions` plugin for extracting image dimensions.
-It adds `width` and `height` metadata values, and also adds `#width`,
-`#height`, and `#dimensions` methods to the `Shrine::UploadedFile` object. By
-default, the plugin uses [FastImage] to analyze dimensions, but you can also
-have it use [MiniMagick] or [ruby-vips]:
+Shrine comes with a [`store_dimensions`][store_dimensions] plugin for
+extracting image dimensions. It adds `width` and `height` metadata values, and
+also adds `#width`, `#height`, and `#dimensions` methods to the
+`Shrine::UploadedFile` object.
 
 ```rb
 # Gemfile
-gem "fastimage"
+gem "fastimage" # default analyzer
 ```
 ```rb
 Shrine.plugin :store_dimensions
@@ -136,12 +139,17 @@ uploaded_file.height     #=> 900
 uploaded_file.dimensions #=> [1600, 900]
 ```
 
+By default, the plugin uses [FastImage] to analyze dimensions, but you can also
+have it use [MiniMagick] or [ruby-vips]. See the
+[`store_dimensions`][store_dimensions] plugin docs for more details.
+
 ## Custom metadata
 
 In addition to the built-in metadata, Shrine allows you to extract and store
-any custom metadata, using the `add_metadata` plugin (which extends
-`Shrine#extract_metadata`). For example, you might want to extract EXIF data
-from images:
+any custom metadata, using the [`add_metadata`][add_metadata] plugin (which
+internally extends `Shrine#extract_metadata`).
+
+For example, you might want to extract EXIF data from images:
 
 ```rb
 # Gemfile
@@ -167,7 +175,7 @@ uploaded_file.exif             #=> {...}
 ```
 
 Or, if you're uploading videos, you might want to extract some video-specific
-meatadata:
+metadata:
 
 ```rb
 # Gemfile
@@ -203,9 +211,20 @@ uploaded_file.metadata #=>
 
 The yielded `io` object will not always be an object that responds to `#path`.
 For example, with the `data_uri` plugin the `io` can be a `StringIO` wrapper,
-with `restore_cached_data` or `refresh_metadata` plugins the `io` might be a
-`Shrine::UploadedFile` object. So we're using `Shrine.with_file` to ensure we
-have a file object.
+while with `restore_cached_data` or `refresh_metadata` plugins the `io` might
+be a `Shrine::UploadedFile` object. So, we're using `Shrine.with_file` to
+ensure we have a file object.
+
+### Adding metadata
+
+If you wish to add metadata to an already attached file, you can do it as
+follows:
+
+```rb
+photo.image_attacher.add_metadata("foo" => "bar")
+photo.image.metadata #=> { ..., "foo" => "bar" }
+photo.save # persist changes
+```
 
 ## Metadata columns
 
@@ -225,15 +244,20 @@ photo.image_type #=> "image/jpeg"
 When attaching files that were uploaded directly to the cloud or a [tus
 server], Shrine won't automatically extract metadata from them, instead it will
 copy any existing metadata that was set on the client side. The reason why this
-is the default behaviour is because extracting the metadata would require (at
-least partially) retrieving file content from the storage, which could
-potentially be expensive depending on the storage and the type of metadata
-being extracted.
+is the default behaviour is because metadata extraction requires (at least
+partially) retrieving file content from the storage, which could potentially be
+expensive depending on the storage and the type of metadata being extracted.
+
+```rb
+# no additional metadata will be extracted in this assignment by default
+photo.image = '{"id":"9e6581a4ea1.jpg","storage":"cache","metadata":{...}}'
+```
 
-There are two ways of extracting metadata from directly uploaded files. If you
-want metadata to be automatically extracted on assignment (which is useful if
-you want to validate the extracted metadata or have it immediately available
-for any other reason), you can load the `restore_cached_data` plugin:
+### Extracting on attachment
+
+If you want metadata to be automatically extracted on assignment (which is
+useful if you want to validate the extracted metadata or have it immediately
+available for any other reason), you can load the `restore_cached_data` plugin:
 
 ```rb
 Shrine.plugin :restore_cached_data # automatically extract metadata from cached files on assignment
@@ -248,7 +272,9 @@ photo.image.metadata #=>
 # }
 ```
 
-### Backgrounding
+### Extracting in the background
+
+#### A) Extracting with promotion
 
 If you're using [backgrounding], you can extract metadata during background
 promotion using the `refresh_metadata` plugin (which the `restore_cached_data`
@@ -271,12 +297,14 @@ class PromoteJob
     record         = Object.const_get(record_class).find(record_id) # if using Active Record
 
     attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
-    attacher.refresh_metadata!
+    attacher.refresh_metadata! # extract metadata
     attacher.atomic_promote
   end
 end
 ```
 
+#### B) Extracting separately from promotion
+
 You can also extract metadata in the background separately from promotion:
 
 ```rb
@@ -303,11 +331,13 @@ class MetadataJob
 end
 ```
 
+### Combining foreground and background
+
 If you have some metadata that you want to extract in the foreground and some
 that you want to extract in the background, you can use the uploader context:
 
 ```rb
-class MyUploader < Shrine
+class VideoUploader < Shrine
   plugin :add_metadata
 
   add_metadata do |io, **options|
@@ -324,7 +354,7 @@ class MyUploader < Shrine
 end
 ```
 ```rb
-class MetadataJob
+class PromoteJob
   include Sidekiq::Worker
 
   def perform(attacher_class, record_class, record_id, name, file_data)
@@ -332,12 +362,16 @@ class MetadataJob
     record         = Object.const_get(record_class).find(record_id) # if using Active Record
 
     attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
-    attacher.refresh_metadata!(background: true)
-    attacher.atomic_persist
+    attacher.refresh_metadata!(background: true) # specify the flag
+    attacher.atomic_promote
   end
 end
 ```
 
+Now triggering metadata extraction in the controller on attachment (using
+`restore_cached_data` or `refresh_metadata` plugin) will skip the video
+metadata block, which will be triggered later in the background job.
+
 ### Optimizations
 
 If you want to do both metadata extraction and file processing during
@@ -389,4 +423,6 @@ end
 [ruby-vips]: https://github.com/libvips/ruby-vips
 [tus server]: https://github.com/janko/tus-ruby-server
 [determine_mime_type]: https://shrinerb.com/docs/plugins/determine_mime_type
+[store_dimensions]: https://shrinerb.com/docs/plugins/store_dimensions
+[add_metadata]: https://shrinerb.com/docs/plugins/add_metadata
 [backgrounding]: https://shrinerb.com/docs/plugins/backgrounding

data/doc/multiple_files.md

@@ -228,7 +228,7 @@ photos_attributes = album_params[:photos_attributes].to_h.merge(new_photos_attri
 album_attributes  = album_params.merge(photos_attributes: photos_attributes)
 
 # create the album with photos
-Album.create(album_params)
+Album.create(album_attributes)
 ```
 
 In this case you need to make sure your form tag has

data/doc/paperclip.md

@@ -1,5 +1,5 @@
 ---
-title: Shrine for Paperclip Users
+title: Upgrading from Paperclip
 ---
 
 This guide is aimed at helping Paperclip users transition to Shrine, and it
@@ -136,7 +136,7 @@ into separate columns.
 While Paperclip works only with Active Record, Shrine is designed to integrate
 with any persistence library (there are integrations for [Active
 Record][activerecord], [Sequel][sequel], [ROM][rom], [Hanami][hanami] and
-[Hanami][mongoid]), and can also be used standalone:
+[Mongoid][mongoid]), and can also be used standalone:
 
 ```rb
 attacher = ImageUploader::Attacher.new
@@ -358,13 +358,14 @@ module PaperclipShrineSynchronization
     end
   end
 
-  # If you'll be using a `:prefix` on your Shrine storage, or you're storing
-  # files on the filesystem, make sure to subtract the appropriate part
-  # from the path assigned to `:id`.
   def shrine_attachment_file(attachment)
+    location = attachment.path
+    # if you're storing files on disk, make sure to subtract the absolute path
+    location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
+
     Shrine.uploaded_file(
       storage:  :store,
-      id:       attachment.path,
+      id:       location,
       metadata: {
         "size"      => attachment.size,
         "filename"  => attachment.original_filename,
@@ -377,12 +378,20 @@ module PaperclipShrineSynchronization
   # files on the filesystem, make sure to subtract the appropriate part
   # from the path assigned to `:id`.
   def shrine_style_file(style)
+    location = style.attachment.path(style.name)
+    # if you're storing files on disk, make sure to subtract the absolute path
+    location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
+
     Shrine.uploaded_file(
       storage:  :store,
-      id:       style.attachment.path(style.name),
+      id:       location,
       metadata: {},
     )
   end
+
+  def storage
+    Shrine.storages[:store]
+  end
 end
 ```
 ```rb
@@ -511,11 +520,20 @@ Alternatively, if you want to generate locations yourself you can override the
 
 ```rb
 class ImageUploader < Shrine
-  def generate_location(io, **options)
-    # ...
+  def generate_location(io, record: nil, name: nil, **)
+    [ storage_key,
+      record && record.class.name.underscore,
+      record && record.id,
+      super,
+      io.original_filename ].compact.join("/")
   end
 end
 ```
+```
+cache/user/123/2feff8c724e7ce17/nature.jpg
+store/user/456/7f99669fde1e01fc/kitten.jpg
+...
+```
 
 #### `:validate_media_type`
 
@@ -677,6 +695,7 @@ s3.upload(io, "object/destination/path")
 The Shrine storage has no replacement for the `:url` Paperclip option, and it
 isn't needed.
 
+[metadata_attributes]: https://shrinerb.com/docs/plugins/metadata_attributes
 [Managing Derivatives]: https://shrinerb.com/docs/changing-derivatives
 [direct uploads]: https://shrinerb.com/docs/getting-started#direct-uploads
 [S3]: https://shrinerb.com/docs/storage/s3

data/doc/plugins/add_metadata.md

@@ -2,76 +2,130 @@
 title: Add Metadata
 ---
 
-The [`add_metadata`][add_metadata] plugin provides a convenient method for
-extracting and adding custom metadata values.
+The [`add_metadata`][add_metadata] plugin allows adding custom metadata to
+uploaded files.
 
 ```rb
-plugin :add_metadata
+Shrine.plugin :add_metadata
+```
+
+## Metadata block
+
+The `Shrine.add_metadata` method allows you to register a block that will get
+executed on upload, where you can return custom metadata:
+
+```rb
+require "pdf-reader" # https://github.com/yob/pdf-reader
 
-add_metadata :exif do |io|
-  begin
-    Exif::Data.new(io).to_h
-  rescue Exif::NotReadable # not a valid image
-    {}
+class PdfUploader < Shrine
+  add_metadata :page_count do |io|
+    reader = PDF::Reader.new(io)
+    reader.page_count
   end
 end
 ```
 
-The above will add "exif" to the metadata hash, and also create the `#exif`
-reader method on `Shrine::UploadedFile`.
+The above will add `page_count` key to the metadata hash, and also create the
+`#page_count` reader method on the `Shrine::UploadedFile`.
 
 ```rb
-image.metadata["exif"]
+uploaded_file.metadata["page_count"] #=> 30
 # or
-image.exif
+uploaded_file.page_count #=> 30
+```
+
+### Skipping nil values
+
+By default, if your block returns `nil` then the `nil` value will be stored into
+metadata. If you do not want to store anything when your block returns nil, you
+can use the `skip_nil: true` option:
+
+```rb
+class PdfUploader < Shrine
+  add_metadata :pages, skip_nil: true do |io|
+    if is_pdf?(io)
+      reader = PDF::Reader.new(io)
+      reader.page_count
+    else
+      # If this is not a PDF, then the pages metadata will not be stored
+      nil
+    end
+  end
+end
 ```
 
-## Multiple values
+### Multiple values
 
 You can also extract multiple metadata values at once, by using `add_metadata`
 without an argument and returning a hash of metadata.
 
 ```rb
-add_metadata do |io|
-  begin
-    data = Exif::Data.new(io)
-  rescue Exif::NotReadable # not a valid image
-    next {}
+require "exif" # https://github.com/tonytonyjan/exif
+
+class ImageUploader < Shrine
+  add_metadata do |io|
+    begin
+      data = Exif::Data.new(io)
+    rescue Exif::NotReadable # not a valid image
+      next {}
+    end
+
+    { "date_time"     => data.date_time,
+      "flash"         => data.flash,
+      "focal_length"  => data.focal_length,
+      "exposure_time" => data.exposure_time }
   end
-
-  { date_time:     data.date_time,
-    flash:         data.flash,
-    focal_length:  data.focal_length,
-    exposure_time: data.exposure_time }
 end
 ```
+```rb
+uploaded_file.metadata #=>
+# {
+#   ...
+#   "date_time" => "2019:07:20 16:16:08",
+#   "flash" => 16,
+#   "focal_length" => 26/1,
+#   "exposure_time" => 1/500,
+# }
+```
 
 In this case Shrine won't automatically create reader methods for the extracted
-metadata on Shrine::UploadedFile, but you can create them via
-`#metadata_method`.
+metadata, but you can create them via `Shrine.metadata_method`:
 
 ```rb
-metadata_method :date_time, :flash
+class ImageUploader < Shrine
+  # ...
+  metadata_method :date_time, :flash
+end
+```
+```rb
+uploaded_file.date_time #=> "2019:07:20 16:16:08"
+uploaded_file.flash     #=> 16
 ```
 
-## Ensuring file
+### Ensuring file
 
 The `io` might not always be a file object, so if you're using an analyzer
 which requires the source file to be on disk, you can use `Shrine.with_file` to
 ensure you have a file object.
 
 ```rb
-add_metadata do |io|
-  movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
-
-  { "duration"   => movie.duration,
-    "bitrate"    => movie.bitrate,
-    "resolution" => movie.resolution,
-    "frame_rate" => movie.frame_rate }
+require "streamio-ffmpeg" # https://github.com/streamio/streamio-ffmpeg
+
+class VideoUploader < Shrine
+  add_metadata do |io|
+    movie = Shrine.with_file(io) do |file|
+      FFMPEG::Movie.new(file.path)
+    end
+
+    { "duration"   => movie.duration,
+      "bitrate"    => movie.bitrate,
+      "resolution" => movie.resolution,
+      "frame_rate" => movie.frame_rate }
+  end
 end
 ```
 
-## Uploader options
+### Uploader options
 
 Uploader options are also yielded to the block, you can access them for more
 context:
@@ -89,6 +143,8 @@ add_metadata do |io, **options|
 end
 ```
 
+#### Metadata
+
 The `:metadata` option holds metadata that was extracted so far:
 
 ```rb
@@ -116,4 +172,25 @@ add_metadata :bar do |io, metadata:, **|
 end
 ```
 
+## Updating metadata
+
+If you just wish to add some custom metadata to existing uploads, you can do it
+with `UploadedFile#add_metadata` (and write the changes back to the model):
+
+```rb
+attacher.file.add_metadata("foo" => "bar")
+attacher.write # write changes to the model attribute
+
+attacher.file.metadata #=> { ..., "foo" => "bar" }
+```
+
+You can also use the `Attacher#add_metadata` shorthand, which also takes care
+of syncing the model:
+
+```rb
+attacher.add_metadata("foo" => "bar")
+
+attacher.file.metadata #=> { ..., "foo" => "bar" }
+```
+
 [add_metadata]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/add_metadata.rb