shrine 3.0.1 → 3.3.0

data/doc/plugins/activerecord.md

@@ -158,7 +158,7 @@ end
 ```
 ```yml
 en:
-  activerecord
+  activerecord:
     errors:
       models:
         photo:

data/doc/plugins/add_metadata.md

@@ -2,76 +2,130 @@
 title: Add Metadata
 ---
 
-The [`add_metadata`][add_metadata] plugin provides a convenient method for
-extracting and adding custom metadata values.
+The [`add_metadata`][add_metadata] plugin allows adding custom metadata to
+uploaded files.
 
 ```rb
-plugin :add_metadata
+Shrine.plugin :add_metadata
+```
+
+## Metadata block
+
+The `Shrine.add_metadata` method allows you to register a block that will get
+executed on upload, where you can return custom metadata:
+
+```rb
+require "pdf-reader" # https://github.com/yob/pdf-reader
 
-add_metadata :exif do |io|
-  begin
-    Exif::Data.new(io).to_h
-  rescue Exif::NotReadable # not a valid image
-    {}
+class PdfUploader < Shrine
+  add_metadata :page_count do |io|
+    reader = PDF::Reader.new(io)
+    reader.page_count
   end
 end
 ```
 
-The above will add "exif" to the metadata hash, and also create the `#exif`
-reader method on `Shrine::UploadedFile`.
+The above will add `page_count` key to the metadata hash, and also create the
+`#page_count` reader method on the `Shrine::UploadedFile`.
 
 ```rb
-image.metadata["exif"]
+uploaded_file.metadata["page_count"] #=> 30
 # or
-image.exif
+uploaded_file.page_count #=> 30
+```
+
+### Skipping nil values
+
+By default, if your block returns `nil` then the `nil` value will be stored into
+metadata. If you do not want to store anything when your block returns nil, you
+can use the `skip_nil: true` option:
+
+```rb
+class PdfUploader < Shrine
+  add_metadata :pages, skip_nil: true do |io|
+    if is_pdf?(io)
+      reader = PDF::Reader.new(io)
+      reader.page_count
+    else
+      # If this is not a PDF, then the pages metadata will not be stored
+      nil
+    end
+  end
+end
 ```
 
-## Multiple values
+### Multiple values
 
 You can also extract multiple metadata values at once, by using `add_metadata`
 without an argument and returning a hash of metadata.
 
 ```rb
-add_metadata do |io|
-  begin
-    data = Exif::Data.new(io)
-  rescue Exif::NotReadable # not a valid image
-    next {}
+require "exif" # https://github.com/tonytonyjan/exif
+
+class ImageUploader < Shrine
+  add_metadata do |io|
+    begin
+      data = Exif::Data.new(io)
+    rescue Exif::NotReadable # not a valid image
+      next {}
+    end
+
+    { "date_time"     => data.date_time,
+      "flash"         => data.flash,
+      "focal_length"  => data.focal_length,
+      "exposure_time" => data.exposure_time }
   end
-
-  { date_time:     data.date_time,
-    flash:         data.flash,
-    focal_length:  data.focal_length,
-    exposure_time: data.exposure_time }
 end
 ```
+```rb
+uploaded_file.metadata #=>
+# {
+#   ...
+#   "date_time" => "2019:07:20 16:16:08",
+#   "flash" => 16,
+#   "focal_length" => 26/1,
+#   "exposure_time" => 1/500,
+# }
+```
 
 In this case Shrine won't automatically create reader methods for the extracted
-metadata on Shrine::UploadedFile, but you can create them via
-`#metadata_method`.
+metadata, but you can create them via `Shrine.metadata_method`:
 
 ```rb
-metadata_method :date_time, :flash
+class ImageUploader < Shrine
+  # ...
+  metadata_method :date_time, :flash
+end
+```
+```rb
+uploaded_file.date_time #=> "2019:07:20 16:16:08"
+uploaded_file.flash     #=> 16
 ```
 
-## Ensuring file
+### Ensuring file
 
 The `io` might not always be a file object, so if you're using an analyzer
 which requires the source file to be on disk, you can use `Shrine.with_file` to
 ensure you have a file object.
 
 ```rb
-add_metadata do |io|
-  movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
-
-  { "duration"   => movie.duration,
-    "bitrate"    => movie.bitrate,
-    "resolution" => movie.resolution,
-    "frame_rate" => movie.frame_rate }
+require "streamio-ffmpeg" # https://github.com/streamio/streamio-ffmpeg
+
+class VideoUploader < Shrine
+  add_metadata do |io|
+    movie = Shrine.with_file(io) do |file|
+      FFMPEG::Movie.new(file.path)
+    end
+
+    { "duration"   => movie.duration,
+      "bitrate"    => movie.bitrate,
+      "resolution" => movie.resolution,
+      "frame_rate" => movie.frame_rate }
+  end
 end
 ```
 
-## Uploader options
+### Uploader options
 
 Uploader options are also yielded to the block, you can access them for more
 context:
@@ -89,6 +143,8 @@ add_metadata do |io, **options|
 end
 ```
 
+#### Metadata
+
 The `:metadata` option holds metadata that was extracted so far:
 
 ```rb
@@ -116,4 +172,25 @@ add_metadata :bar do |io, metadata:, **|
 end
 ```
 
+## Updating metadata
+
+If you just wish to add some custom metadata to existing uploads, you can do it
+with `UploadedFile#add_metadata` (and write the changes back to the model):
+
+```rb
+attacher.file.add_metadata("foo" => "bar")
+attacher.write # write changes to the model attribute
+
+attacher.file.metadata #=> { ..., "foo" => "bar" }
+```
+
+You can also use the `Attacher#add_metadata` shorthand, which also takes care
+of syncing the model:
+
+```rb
+attacher.add_metadata("foo" => "bar")
+
+attacher.file.metadata #=> { ..., "foo" => "bar" }
+```
+
 [add_metadata]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/add_metadata.rb

data/doc/plugins/atomic_helpers.md

@@ -3,14 +3,48 @@ title: Atomic Helpers
 ---
 
 The [`atomic_helpers`][atomic_helpers] plugin provides API for retrieving and
-persisting attachments in a concurrency-safe way, which is useful when using
-the `backgrounding` plugin. The database plugins (`activerecord` and `sequel`)
-implement atomic promotion and atomic persistence on top of this plugin.
+persisting attachments in a concurrency-safe way, which is especially useful
+when using the `backgrounding` plugin. The database plugins (`activerecord`
+and `sequel`) implement atomic promotion and atomic persistence on top of this
+plugin.
 
 ```rb
 plugin :atomic_helpers
 ```
 
+## Problem Statement
+
+What happens if two different processors (web workers, background jobs,
+command-line executions, whatever) try to edit a shrine attachment
+concurrently? The kinds of edits typically made include: "promoting a file",
+moving it to a different storage and persisting that change in the model;
+adding or changing a derivative; adding or changing a metadata element.
+
+There are two main categories of "race condition":
+
+1. The file could be switched out from under you. If you were promoting a file,
+but some other process has *changed* the attachment, you don't want to
+overwrite it with the promomoted version of the *prior* attacchment. Likewise,
+if you were adding metadata or a derivative, they would be corresponding to a
+certain attachment, and you don't want to accidentally add them to a now changed
+attacchment for which they are inappropriate.
+
+2. Overwriting each other's edits. Since all shrine (meta)data is stored in a
+single JSON hash, standard implementations will write the entire JSON hash at
+once to a rdbms column or other store. If two processes both read in the hash,
+make a change to different keys in it, and then write it back out, the second
+process to write will 'win' and overwrite changes made by the first.
+
+The atomic helpers give you tools to avoid both of these sorts of race
+conditions, under conditions of concurrent editing.
+
+## High-level ORM helpers
+
+If you are using the `sequel` or `activerecord` plugins, they give you two
+higher-level helpers: `atomic_persist` and `atomic_promote`. See the
+[persistence]  documentation for more.
+
+
 ## Retrieving
 
 The `Attacher.retrieve` method provided by the plugin instantiates an attacher
@@ -177,3 +211,7 @@ end
 ```
 
 [atomic_helpers]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/atomic_helpers.rb
+
+[persistence]: https://shrinerb.com/docs/plugins/persistence
+
+[backgrounding]: https://shrinerb.com/docs/plugins/backgrounding

data/doc/plugins/backgrounding.md

@@ -187,12 +187,22 @@ and make the execution synchronous, you can override them on the attacher level
 and call the default behaviour:
 
 ```rb
-photo.image_attacher.promote_block(&:promote)
-photo.image_attacher.destroy_block(&:destroy)
+photo.image_attacher.promote_block { promote } # promote synchronously
+photo.image_attacher.destroy_block { destroy } # destroy synchronously
 
 # ... now promotion and deletion will be synchronous ...
 ```
 
+You can also do this on the class level if you want to disable backgrounding
+that was set up by a superclass:
+
+```rb
+class MyUploader < Shrine
+  Attacher.promote_block { promote } # promote synchronously
+  Attacher.destroy_block { destroy } # destroy synchronously
+end
+```
+
 [backgrounding]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/backgrounding.rb
 [derivatives]: https://shrinerb.com/docs/plugins/derivatives
 [atomic_helpers]: https://shrinerb.com/docs/plugins/atomic_helpers

data/doc/plugins/column.md

@@ -66,23 +66,48 @@ If you want to load attachment from a Hash, use `Attacher.from_data` or
 
 ## Serializer
 
-By default the `JSON` standard library is used as the serializer, but you can
-use your own serializer. The serializer object needs to implement `#dump` and
-`#load` methods.
+By default, the `JSON` standard library is used for serializing hash data. With
+the [`model`][model] and [`entity`][entity] plugin, the data is serialized
+before writing to and deserialized after reading from the data attribute.
+
+You can also use your own serializer via the `:serializer` option. The
+serializer object needs to implement `#dump` and `#load` methods:
+
+```rb
+class MyDataSerializer
+  def self.dump(data)
+    data #=> { "id" => "...", "storage" => "...", "metadata" => { ... } }
+
+    JSON.generate(data) # serialize data, e.g. into JSON
+  end
+
+  def self.load(data)
+    data #=> '{"id":"...", "storage":"...", "metadata": {...}}'
+
+    JSON.parse(data) # deserialize data, e.g. from JSON
+  end
+end
+
+plugin :column, serializer: MyDataSerializer
+```
+
+Some serialization libraries such as [Oj] and [MessagePack] already implement
+this interface, which simplifies the configuration:
 
 ```rb
-require "oj"
+require "oj" # https://github.com/ohler55/oj
 
-plugin :column, serializer: Oj # use custom serializer
+plugin :column, serializer: Oj
 ```
 
-If you want to disable serialization, you can set serializer to `nil`.
+If you want to disable serialization and work with hashes directly, you can set
+`:serializer` to `nil`:
 
 ```rb
 plugin :column, serializer: nil # disable serialization
 ```
 
-You can also change the serializer on the attacher level:
+The serializer can also be changed for a particular attacher instance:
 
 ```rb
 Shrine::Attacher.new(column_serializer: Oj)  # use custom serializer
@@ -90,3 +115,7 @@ Shrine::Attacher.new(column_serializer: nil) # disable serialization
 ```
 
 [column]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/column.rb
+[model]: https://shrinerb.com/docs/plugins/model
+[entity]: https://shrinerb.com/docs/plugins/entity
+[Oj]: https://github.com/ohler55/oj
+[MessagePack]: https://github.com/msgpack/msgpack-ruby

data/doc/plugins/data_uri.md

@@ -123,7 +123,7 @@ payload:
 
 A default log subscriber is added as well which logs these events:
 
-```plaintext
+```
 Data URI (5ms) – {:uploader=>Shrine}
 ```
 
@@ -134,7 +134,7 @@ plugin :data_uri, log_subscriber: -> (event) {
   Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, uploader: event[:uploader])
 }
 ```
-```plaintext
+```
 {"name":"data_uri","duration":5,"uploader":"Shrine"}
 ```
 

data/doc/plugins/default_url.md

@@ -8,7 +8,7 @@ returned when there is no attached file.
 ```rb
 plugin :default_url
 
-Attacher.default_url do |options|
+Attacher.default_url do |**options|
   "/#{name}/missing.jpg"
 end
 ```
@@ -28,7 +28,7 @@ Any URL options passed will be available in the default URL block:
 attacher.url(foo: "bar")
 ```
 ```rb
-Attacher.default_url do |options|
+Attacher.default_url do |**options|
   options #=> { foo: "bar" }
 end
 ```
@@ -37,12 +37,15 @@ The default URL block is evaluated in the context of an instance of
 `Shrine::Attacher`.
 
 ```rb
-Attacher.default_url do |options|
+Attacher.default_url do |**options|
   self    #=> #<Shrine::Attacher>
 
+  file    #=> #<Shrine::UploadedFile>
   name    #=> :avatar
   record  #=> #<User>
   context #=> { ... }
+
+  # ...
 end
 ```
 

data/doc/plugins/derivation_endpoint.md

@@ -68,7 +68,7 @@ generates an URL consisting of the configured [path prefix](#prefix),
 derivation name and arguments, serialized uploaded file, and an URL signature
 generated using the configured secret key:
 
-```plaintext
+```
 /  derivations/image  /  thumbnail  /  600/400  /  eyJmZvbyIb3JhZ2UiOiJzdG9yZSJ9  ?  signature=...
   └──── prefix ─────┘  └── name ──┘  └─ args ─┘  └─── serialized source file ───┘
 ```
@@ -197,7 +197,8 @@ Rails.application.routes.draw do
     end
   end
 end
-
+```
+```rb
 # app/controllers/photos_controller.rb
 class PhotosController < ApplicationController
   def thumbnail
@@ -483,32 +484,27 @@ such as AWS S3 or Google Cloud Storage.
 ### Deleting derivatives
 
 When the original attachment is deleted, its uploaded derivatives will not be
-automatically deleted, you will need to do the deletion manually. To ensure
-this gets called both on destroying and replacing, you can add that code to
-`Attacher#destroy`.
+automatically deleted, you will need to do the deletion manually. If you're
+using [backgrounding], you can do this in your `DestroyJob`.
 
-The easiest way is to delete the directory containing your derivatives:
+If your storage implements `#delete_prefixed`, and you're using the default
+[`:upload_location`](#upload-location), you can delete the directory containing
+derivatives:
 
 ```rb
-class ImageUploader < Shrine
-  class Attacher
-    def destroy(*args)
-      super
+class DestroyJob < ActiveJob::Base
+  def perform(attacher_class, data)
+    # ... destroy attached file ...
 
-      derivatives_directory = file.id
-      storage               = store.storage
+    derivatives_directory = attacher.file.id + "/"
+    storage               = attacher.store.storage
 
-      storage.delete_prefixed(derivatives_directory)
-    end
+    storage.delete_prefixed(derivatives_directory)
   end
 end
 ```
 
-This is under the assumption that your storage implements `#delete_prefixed`
-and that you're using default [`:upload_location`](#upload-location).
-
-Alternatively, you can delete each derivative individually using
-`Derivation#delete`:
+Alternatively, you can delete each derivative individually:
 
 ```rb
 class ImageUploader < Shrine
@@ -518,14 +514,15 @@ class ImageUploader < Shrine
     [:thumbnail, 400, 300],
     ...
   ]
+end
+```
+```rb
+class DestroyJob < ActiveJob::Base
+  def perform(attacher_class, data)
+    # ... destroy attached file ...
 
-  class Attacher
-    def destroy(*args)
-      super
-
-      DERIVATIONS.each do |args|
-        file.derivation(*args).delete
-      end
+    attacher.shrine_class::DERIVATIONS.each do |args|
+      attacher.file.derivation(*args).delete
     end
   end
 end
@@ -830,7 +827,7 @@ following payload:
 
 A default log subscriber is added as well which logs these events:
 
-```plaintext
+```
 Derivation (492ms) – {:name=>:thumbnail, :args=>[600, 600], :uploader=>Shrine}
 ```
 
@@ -846,7 +843,7 @@ plugin :derivation_endpoint, log_subscriber: -> (event) {
   )
 }
 ```
-```plaintext
+```
 {"name":"derivation","duration":492,"name":"thumbnail","args":[600,600],"uploader":"Shrine"}
 ```
 
@@ -861,3 +858,4 @@ plugin :derivation_endpoint, log_subscriber: nil
 [`Content-Type`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type
 [`Content-Disposition`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition
 [`Cache-Control`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
+[backgrounding]: https://shrinerb.com/docs/plugins/backgrounding