shrine 3.0.0.beta2 → 3.0.0.beta3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e35bbe08c0a54aca90746357823eea8bd7f65f7ee64a988fe6bc9c3931f70a91
-  data.tar.gz: 1c70a7212c59f13fa435b866d0dad8c81989d7ee41722247d5107edec461047b
+  metadata.gz: ad93860597c6a720f0159063d8cbb95a6aa0e93ea89947803342500f1193eb64
+  data.tar.gz: 79801e74ecd993986099f0f4679fcb37c2b778157c4df5bf6bb80064f2338a44
 SHA512:
-  metadata.gz: 784c4f79052ef3132ffe26544a79de6f27e9209563cd9c08d475a3e52e0b7b201041f516caebc00315f802c1c9d18e7bd74b0e8e5977576727dccfefe47d0c3b
-  data.tar.gz: 6788e1946a1062a3f184b87cf016b7104d6c524621853fb38a8c092d245fa244625a16866dde958bf53caa4bbb113b6d0e01fb09c7340b39633360e40b5d4ee4
+  metadata.gz: 12337c687f57272f9af85b8b8831de448fea258e25e6b7191b95131a60e705374807a87efb7c5e4e7c3db0c6f11a8c3dc87dfe21255d24785e908a93cc94be7e
+  data.tar.gz: 04561d887c0612ed193c31c470fcdd9ca5ed466794923ceecf979f78049ec3c9c46cf0161776121c72419bcc5f77babfc565ceaf6892a06a547bd93f9c1cccce

data/CHANGELOG.md

@@ -1,3 +1,45 @@
+## 3.0.0.beta3 (2019-09-25)
+
+* `multi_cache` – Add new plugin for whitelisting additional temporary storages (@janko, @jrochkind)
+
+* `s3` – Allow uploading files larger than 50 GB (@janko)
+
+* `sequel` – Extract callback code into attacher methods that can be overridden (@janko)
+
+* `activerecord` – Extract callback code into attacher methods that can be overridden (@janko)
+
+* `derivatives` – Ensure binary mode for `File` and `Tempfile` objects (@janko)
+
+* `derivatives` – Ensure refreshed file descriptor for `Tempfile` objects (@janko)
+
+* `derivation_endpoint` – Stop re-opening `File` objects returned in derivation result (@janko)
+
+* `derivation_endpoint` – Allow only `File` or `Tempfile` object as derivation result (@janko)
+
+* `derivatives` – Add support for default processors (@janko)
+
+* `remote_url` – Convert only `Down::Error` and `DownloadError` exceptions into validation errors (@janko)
+
+* `remote_url` – Bring back `Attacher#remote_url=` and `Attacher#remote_url` (@janko)
+
+* `data_uri` – Bring back `Attacher#data_uri=` and `Attacher#data_uri` (@janko)
+
+* `mirroring` – Allow skipping mirroring by passing `mirror: false` to `#upload` and `#delete` (@janko)
+
+* `mirroring` – Rename `Shrine.mirror_(upload|delete)` to `Shrine.mirror_(upload|delete)_block` (@janko)
+
+* `mirroring` – Add `UploadedFile#mirror_(upload|delete)_background` (@janko)
+
+* `attacher_options` – Remove plugin (@janko)
+
+* `backgrounding` – Move `Attacher#destroy(background: true)` to `Attacher#destroy_background` (@janko)
+
+* `backgrounding` – Move `Attacher#promote(background: true)` to `Attacher#promote_background` (@janko)
+
+* `download_endpoint` – Add `Shrine.download_response` for calling in controller (@janko)
+
+* `core` – Fetch storage object lazily in `Shrine` instance (@janko)
+
 ## 3.0.0.beta2 (2019-09-11)
 
 * `column` – Allow `Attacher#load_column` to receive a hash (@janko)
@@ -154,7 +196,7 @@
 
 * `upload_endpoint` – Stop passing `Rack::Request` object to the uploader (@janko)
 
-* `remote_url` – Require custom downlodaers to raise `Shrine::Plugins::RemoteUrl::DownloadError` for conversion into a validation error (@janko)
+* `remote_url` – Require custom downloaders to raise `Shrine::Plugins::RemoteUrl::DownloadError` for conversion into a validation error (@janko)
 
 * `remote_url` – Remove `Attacher#remote_url=` and `Attacher#remote_url` (@janko)
 
@@ -262,6 +304,8 @@
 
 * `hooks` – Remove plugin (@janko)
 
+* `core` – Remove deprecated `Shrine::IO_METHODS` constant (@janko)
+
 * `s3` – Replace source object metadata when copying a file from S3 (@janko)
 
 * `core` – Change `UploadedFile#storage_key` to return a Symbol instead of a String (@janko)

data/README.md

@@ -5,8 +5,8 @@ Shrine is a toolkit for file attachments in Ruby applications. Some highlights:
 * **Modular design** – the [plugin system] allows you to load only the functionality you need
 * **Memory friendly** – streaming uploads and [downloads][Retrieving Uploads] make it work great with large files
 * **Cloud storage** – store files on [disk][FileSystem], [AWS S3][S3], [Google Cloud][GCS], [Cloudinary] and [others][external]
-* **ORM integrations** – works with [Sequel][sequel plugin], [ActiveRecord][activerecord plugin], [Hanami::Model][hanami plugin] and [Mongoid][mongoid plugin]
-* **Flexible processing** – generate thumbnails [on upload] or [on-the-fly] using [ImageMagick][ImageProcessing::MiniMagick] or [libvips][ImageProcessing::Vips]
+* **Persistence integrations** – works with [Sequel][sequel plugin], [ActiveRecord][activerecord plugin], [ROM][rom plugin], [Hanami::Model][hanami plugin] and [Mongoid][mongoid plugin]
+* **Flexible processing** – generate thumbnails [up front] or [on-the-fly] using [ImageMagick][ImageProcessing::MiniMagick] or [libvips][ImageProcessing::Vips]
 * **Metadata validation** – [validate files][validation] based on [extracted metadata][metadata]
 * **Direct uploads** – upload asynchronously [to your app][simple upload] or [to the cloud][presigned upload] using [Uppy]
 * **Resumable uploads** – make large file uploads [resumable][resumable upload] on [S3][uppy-s3_multipart] or [tus][tus-ruby-server]
@@ -16,14 +16,14 @@ If you're curious how it compares to other file attachment libraries, see the [A
 
 ## Resources
 
-| Resource          | URL                                                                                        |
-| :---------------- | :----------------------------------------------------------------------------------------- |
-| Website           | [shrinerb.com](https://shrinerb.com)                                                       |
-| Demo code         | [Roda][roda demo] / [Rails][rails demo]                                                    |
-| Source            | [github.com/shrinerb/shrine](https://github.com/shrinerb/shrine)                           |
-| Wiki              | [github.com/shrinerb/shrine/wiki](https://github.com/shrinerb/shrine/wiki)                 |
-| Bugs              | [github.com/shrinerb/shrine/issues](https://github.com/shrinerb/shrine/issues)             |
-| Help & Discussion | [groups.google.com/group/ruby-shrine](https://groups.google.com/forum/#!forum/ruby-shrine) |
+| Resource          | URL                                                                            |
+| :---------------- | :----------------------------------------------------------------------------- |
+| Website           | [shrinerb.com](https://shrinerb.com)                                           |
+| Demo code         | [Roda][roda demo] / [Rails][rails demo]                                        |
+| Source            | [github.com/shrinerb/shrine](https://github.com/shrinerb/shrine)               |
+| Wiki              | [github.com/shrinerb/shrine/wiki](https://github.com/shrinerb/shrine/wiki)     |
+| Bugs              | [github.com/shrinerb/shrine/issues](https://github.com/shrinerb/shrine/issues) |
+| Help & Discussion | [discourse.shrinerb.com](https://discourse.shrinerb.com)                       |
 
 ## Contents
 
@@ -40,7 +40,7 @@ If you're curious how it compares to other file attachment libraries, see the [A
   * [MIME type](#mime-type)
   * [Other metadata](#other-metadata)
 * [Processing](#processing)
-  * [Processing on upload](#processing-on-upload)
+  * [Processing up front](#processing-up-front)
   * [Processing on-the-fly](#processing-on-the-fly)
 * [Validation](#validation)
 * [Location](#location)
@@ -84,7 +84,7 @@ will use to store all information about the attachment:
 ```rb
 Sequel.migration do
   change do
-    add_column :photos, :image_data, :text
+    add_column :photos, :image_data, :text # or :jsonb
   end
 end
 ```
@@ -97,7 +97,7 @@ $ rails generate migration add_image_data_to_photos image_data:text
 ```rb
 class AddImageDataToPhotos < ActiveRecord::Migration
   def change
-    add_column :photos, :image_data, :text
+    add_column :photos, :image_data, :text # or :jsonb
   end
 end
 ```
@@ -115,14 +115,14 @@ end
 
 ```rb
 class Photo < Sequel::Model # ActiveRecord::Base
-  include ImageUploader::Attachment.new(:image) # adds an `image` virtual attribute
+  include ImageUploader::Attachment(:image) # adds an `image` virtual attribute
 end
 ```
 
-Let's now add the form fields which will use this virtual attribute (which is
-`image`, NOT `image_data`). We need (1) a file field for choosing files, and
-(2) a hidden field for retaining the uploaded file in case of validation errors
-and for potential [direct uploads].
+Let's now add the form fields which will use this virtual attribute (NOT the
+`<attachment>_data` column attribute). We need (1) a file field for choosing
+files, and (2) a hidden field for retaining the uploaded file in case of
+validation errors and for potential [direct uploads].
 
 ```rb
 # with Rails form builder:
@@ -251,7 +251,7 @@ organize them in any way you like.
 
 ### Uploading
 
-The main method of the uploader is `#upload`, which takes an [IO-like
+The main method of the uploader is `Shrine.upload`, which takes an [IO-like
 object][io abstraction] and a storage identifier on the input, and returns a
 representation of the [uploaded file] on the output.
 
@@ -259,8 +259,8 @@ representation of the [uploaded file] on the output.
 MyUploader.upload(file, :store) #=> #<Shrine::UploadedFile>
 ```
 
-Internally this instantiates the uploader with the storage and calls `#upload`
-on it:
+Internally this instantiates the uploader with the storage and calls
+`Shrine#upload`:
 
 ```rb
 uploader = MyUploader.new(:store)
@@ -269,13 +269,12 @@ uploader.upload(file) #=> #<Shrine::UploadedFile>
 
 Some of the tasks performed by `#upload` include:
 
-* any defined [file processing][on upload]
 * extracting [metadata]
 * generating [location]
 * uploading (this is where the [storage] is called)
 * closing the uploaded file
 
-The second argument is a `context` hash which is forwarded to places like
+The second argument is a "context" hash which is forwarded to places like
 metadata extraction and location generation, but it has a few special options:
 
 ```rb
@@ -434,10 +433,10 @@ photo.image_attacher #=> #<Shrine::Attacher>
 The `Shrine::Attacher` object can be instantiated and used directly:
 
 ```rb
-attacher = ImageUploader::Attacher.new(photo, :image)
+attacher = ImageUploader::Attacher.from_model(photo, :image)
 
 attacher.assign(file) # equivalent to `photo.image = file`
-attacher.get          # equivalent to `photo.image`
+attacher.file         # equivalent to `photo.image`
 attacher.url          # equivalent to `photo.image_url`
 ```
 
@@ -521,79 +520,72 @@ the [Extracting Metadata] guide for more details.
 
 ## Processing
 
-Shrine allows you to process attached files either "on upload" or
-"on-the-fly". For example, if your app is accepting image uploads, you can
-generate a pre-defined set of of thumbnails as soon as the image is attached to
-a record ("on upload"), or you can generate necessary thumbnails dynamically as
-they're needed ("on-the-fly").
+Shrine allows you to process attached files up front or on-the-fly. For
+example, if your app is accepting image uploads, you can generate a predefined
+set of of thumbnails when the image is attached to a record, or you can have
+thumbnails generated dynamically as they're needed.
 
-For image processing it's recommended to use the **[ImageProcessing]** gem,
+For image processing, it's recommended to use the **[ImageProcessing]** gem,
 which is a high-level wrapper for processing with [ImageMagick] (via
 [MiniMagick]) or [libvips] (via [ruby-vips]).
 
-### Processing on upload
-
-For processing "on upload", you can intercept when a cached file is being
-uploaded to permanent storage, and perform any file processing you might want.
-The [`processing`][processing plugin] plugin provides the promotion hook, while
-the [`versions`][versions plugin] plugin enables handling a hash of versions.
-
 ```sh
-$ brew install imagemagick
+$ brew install imagemagick vips
 ```
+
+### Processing up front
+
+You can use the [`derivatives`][derivatives plugin] plugin to generate a set of
+pre-defined processed files:
+
 ```rb
 # Gemfile
-gem "image_processing", "~> 1.2"
+gem "image_processing", "~> 1.8"
+```
+```rb
+Shrine.plugin :derivatives
 ```
 ```rb
 require "image_processing/mini_magick"
 
 class ImageUploader < Shrine
-  plugin :processing # allows hooking into promoting
-  plugin :versions   # enable Shrine to handle a hash of files
-  plugin :delete_raw # delete processed files after uploading
-
-  process(:store) do |io, context|
-    versions = { original: io } # retain original
-
-    io.download do |original|
-      pipeline = ImageProcessing::MiniMagick.source(original)
-
-      versions[:large]  = pipeline.resize_to_limit!(800, 800)
-      versions[:medium] = pipeline.resize_to_limit!(500, 500)
-      versions[:small]  = pipeline.resize_to_limit!(300, 300)
-    end
-
-    versions # return the hash of processed files
+  Attacher.derivatives_processor do |original|
+    magick = ImageProcessing::MiniMagick.source(original)
+
+    {
+      large:  magick.resize_to_limit!(800, 800),
+      medium: magick.resize_to_limit!(500, 500),
+      small:  magick.resize_to_limit!(300, 300),
+    }
   end
 end
 ```
+```rb
+photo = Photo.new(image: file)
+photo.image_derivatives! # calls derivatives processor and uploads results
+photo.save
+```
 
-After the files are uploaded, their data is saved into the `<attachment>_data`
-column, and the attachment getter will read them as a Hash of
-[`Shrine::UploadedFile`][uploaded file] objects.
+If you're allowing the attached file to be updated later on, in your update
+route make sure to create derivatives for new attachments:
 
 ```rb
-photo = Photo.create(image: file) # processing is triggered
-photo.image #=>
-# {
-#   :original => #<Shrine::UploadedFile @data={"id"=>"9sd84.jpg", ...}>,
-#   :large    => #<Shrine::UploadedFile @data={"id"=>"lg043.jpg", ...}>,
-#   :medium   => #<Shrine::UploadedFile @data={"id"=>"kd9fk.jpg", ...}>,
-#   :small    => #<Shrine::UploadedFile @data={"id"=>"932fl.jpg", ...}>,
-# }
+photo.image_derivatives! if photo.image_changed?
+```
 
-photo.image[:medium]           #=> #<Shrine::UploadedFile>
-photo.image[:medium].url       #=> "/uploads/store/lg043.jpg"
-photo.image[:medium].size      #=> 5825949
-photo.image[:medium].mime_type #=> "image/jpeg"
+After the processed files are uploaded, their data is saved into the
+`<attachment>_data` column. You can then retrieve the derivatives as
+[`Shrine::UploadedFile`][uploaded file] objects:
 
-photo.image_url(:large) # returns version URL with fallbacks in case version is missing
+```rb
+photo.image(:large)            #=> #<Shrine::UploadedFile ...>
+photo.image(:large).url        #=> "/uploads/store/lg043.jpg"
+photo.image(:large).size       #=> 5825949
+photo.image(:large).mime_type  #=> "image/jpeg"
 ```
 
-By default processing is executed synchronously, but you can choose to delay it
-into a [background job][backgrounding]. You can also do any other type of file
-processing you want, see the [File Processing] guide for more details.
+For more details, see the [`derivatives`][derivatives plugin] plugin
+documentation and the [File Processing] guide.
 
 ### Processing on-the-fly
 
@@ -606,12 +598,9 @@ To set it up, we mount the Rack app in our router on a chosen path prefix,
 configure the plugin with a secret key and that path prefix, and define
 processing we want to perform:
 
-```sh
-$ brew install imagemagick
-```
 ```rb
 # Gemfile
-gem "image_processing", "~> 1.2"
+gem "image_processing", "~> 1.8"
 ```
 ```rb
 # config/routes.rb (Rails)
@@ -650,17 +639,19 @@ more details.
 
 ## Validation
 
-Shrine can perform file validations for files assigned to the model, with
-[`validation_helpers`][validation_helpers plugin] plugin providing some common
-validation methods:
+The [`validation`][validation plugin] plugin allows performing validation for
+attached files. For common validations, the
+[`validation_helpers`][validation_helpers plugin] plugin provides useful
+validators for built in metadata:
 
+```rb
+Shrine.plugin :validation_helpers
+```
 ```rb
 class DocumentUploader < Shrine
-  plugin :validation_helpers
-
   Attacher.validate do
     validate_max_size 5*1024*1024, message: "is too large (max is 5 MB)"
-    validate_mime_type_inclusion %w[application/pdf]
+    validate_mime_type %w[application/pdf]
   end
 end
 ```
@@ -685,9 +676,9 @@ plugin provides a good default hierarchy, but you can also override
 
 ```rb
 class ImageUploader < Shrine
-  def generate_location(io, context)
-    type  = context[:record].class.name.downcase if context[:record]
-    style = context[:version] == :original ? "originals" : "thumbs" if context[:version]
+  def generate_location(io, record: nil, derivative: nil, **)
+    type  = record.class.name.downcase if record
+    style = derivative ? "thumbs" : "originals"
     name  = super # the default unique identifier
 
     [type, style, name].compact.join("/")
@@ -706,7 +697,7 @@ uploads/
 
 Note that there should always be a random component in the location, so that
 the ORM dirty tracking is detected properly. Inside `#generate_location` you
-can also access the extracted metadata through `context[:metadata]`.
+can also access the extracted metadata through the `:metadata` option.
 
 ## Direct uploads
 
@@ -851,29 +842,30 @@ resumable uploads from scratch, it includes a complete JavaScript example
 
 ## Backgrounding
 
-Shrine allows you to put file deletion and promotion of cached files to
-permanent storage into a background job. The [`backgrounding`][backgrounding
-plugin] plugin provides hooks for plugging in your [favourite backgrounding
-library][Backgrounding Libraries].
+The [`backgrounding`][backgrounding plugin] allows you to move file promotion
+and deletion into a background job, using the backgrounding library [of your
+choice][Backgrounding Libraries]:
 
 ```rb
 Shrine.plugin :backgrounding
-Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
-Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
+Shrine::Attacher.promote_block { PromoteJob.perform_later(self.class, record, name, file_data) }
+Shrine::Attacher.destroy_block { DestroyJob.perform_later(self.class, data) }
 ```
 ```rb
-class PromoteJob
-  include Sidekiq::Worker
-  def perform(data)
-    Shrine::Attacher.promote(data)
+class PromoteJob < ActiveJob::Base
+  def perform(attacher_class, record, name, file_data)
+    attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
+    attacher.atomic_promote
+  rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
+    # attachment has changed or the record has been deleted, nothing to do
   end
 end
 ```
 ```rb
-class DeleteJob
-  include Sidekiq::Worker
-  def perform(data)
-    Shrine::Attacher.delete(data)
+class DestroyJob < ActiveJob::Base
+  def perform(attacher_class, data)
+    attacher = attacher_class.from_data(data)
+    attacher.destroy
   end
 end
 ```
@@ -926,6 +918,7 @@ Some plugins add their own instrumentation as well when they detect that the
 | Plugin                | Instrumentation                         |
 | :-----                | :--------------                         |
 | `derivation_endpoint` | instruments file processing             |
+| `derivatives`         | instruments file processing             |
 | `determine_mime_type` | instruments analyzing MIME type         |
 | `store_dimensions`    | instruments extracting image dimensions |
 | `signature`           | instruments calculating signature       |
@@ -993,7 +986,7 @@ The gem is available as open source under the terms of the [MIT License].
 [io abstraction]: #io-abstraction
 [location]: #location
 [metadata]: #metadata
-[on upload]: #processing-on-upload
+[up front]: #processing-up-front
 [on-the-fly]: #processing-on-the-fly
 [plugin system]: #plugin-system
 [simple upload]: #simple-direct-upload
@@ -1045,20 +1038,21 @@ The gem is available as open source under the terms of the [MIT License].
 [add_metadata plugin]: /doc/plugins/add_metadata.md#readme
 [backgrounding plugin]: /doc/plugins/backgrounding.md#readme
 [derivation_endpoint plugin]: /doc/plugins/derivation_endpoint.md#readme
+[derivatives plugin]: /doc/plugins/derivatives.md#readme
 [determine_mime_type plugin]: /doc/plugins/determine_mime_type.md#readme
 [instrumentation plugin]: /doc/plugins/instrumentation.md#readme
 [hanami plugin]: https://github.com/katafrakt/hanami-shrine
 [mongoid plugin]: https://github.com/shrinerb/shrine-mongoid
 [presign_endpoint plugin]: /doc/plugins/presign_endpoint.md#readme
 [pretty_location plugin]: /doc/plugins/pretty_location.md#readme
-[processing plugin]: /doc/plugins/processing.md#readme
 [rack_file plugin]: /doc/plugins/rack_file.md#readme
+[rom plugin]: https://github.com/shrinerb/shrine-rom
 [sequel plugin]: /doc/plugins/sequel.md#readme
 [signature plugin]: /doc/plugins/signature.md#readme
 [store_dimensions plugin]: /doc/plugins/store_dimensions.md#readme
 [upload_endpoint plugin]: /doc/plugins/upload_endpoint.md#readme
 [validation_helpers plugin]: /doc/plugins/validation_helpers.md#readme
-[versions plugin]: /doc/plugins/versions.md#readme
+[validation plugin]: /doc/plugins/validation.md#readme
 
 <!-- Demos -->
 [rails demo]: https://github.com/erikdahlstrand/shrine-rails-example