shrine 3.0.1 → 3.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0b327565465d4140206d1f88433647e7959b6c56ecbdea7d433f1aa5eb01825d
-  data.tar.gz: c19fb7f15299b1d061325f8f83e790f44481456fbc9730d4a909923193dde1a3
+  metadata.gz: 8cfa59ed1f6143ee9298ede17eeda07dba85f0f9c3309a5b7a9e7f6c41399f29
+  data.tar.gz: 67a64a3a4f40c34797ca04279af73829acf4071eb118f78addefdbaa15e048da
 SHA512:
-  metadata.gz: 897bbe8787b3a1f7e59db3e6beebbc91b32ec5e4c19cd832029e05205425c12d837bf8475f21949c23a6c805504da45a0937de0d3dc9f50d3d35921b5f6a7901
-  data.tar.gz: 995b0ada0e5bdb2c4e198304d913f6cfe22d8814328d58bf559490ea572ac3d5555f3eb15fce898ae423ff54f76ba3bcbbeb7f6e05470ddca8acee37ee675455
+  metadata.gz: e87a3dbb80a304f8af559b9c3d4e8badf8e47e4377afd74fe08d570042d65358d44878c4a0366fa3cded98fe166083275e86c61f3b51d28d35e070e0233dea35
+  data.tar.gz: 83c93e76bf5513ab317ccabbdda86ce90f760e7b898dc9f40036af3e7ec915f48e55bd6f8efcf8661dacea672ce6b24b450db217815b062881169c8c90fd0cd3

data/CHANGELOG.md

@@ -1,3 +1,85 @@
+## 3.3.0 (2020-10-04)
+
+* `s3` - Support new `Aws::S3::EncryptionV2::Client` for client-side encryption (@janko)
+
+* `derivation_endpoint` – Reduce possibility of timing attacks when comparing signatures (@esparta)
+
+* `derivatives` – Avoid downloading the attached file when calling default no-op processor (@janko)
+
+* `derivatives` – Add `:download` processor setting for skipping downloading source file (@jrochkind, @janko)
+
+* `derivatives` – Copy non-file source IO objects into local file before passing them to the processor (@jrochkind)
+
+* `sequel` – Call `Attacher#reload` in `Sequel::Model#reload`, which keeps rest of attacher state (@janko, @jrochkind)
+
+* `activerecord` – Call `Attacher#reload` in `ActiveRecord::Base#reload`, which keeps rest of attacher state (@janko, @jrochkind)
+
+* `add_metadata` – Add `:skip_nil` option for excluding metadata keys whose values are nil (@renchap)
+
+* `store_dimensions` – Add `:auto_extraction` option for disabling automatically extracting dimensions on upload (@renchap)
+
+* `mirroring` – Forward original upload options when mirroring upload (@corneverbruggen)
+
+* `derivation_endpoint` – Apply `version` URL option in derivation endpoint (@janko)
+
+* `remove_attachment` – Delete removed file if a new file was attached right after removal (@janko)
+
+* `upload_endpoint` – Fix `Shrine.upload_response` not working in a Rails controller (@pldavid2)
+
+* `presign_endpoint` – Add `OPTIONS` route that newer versions of Uppy check (@janko)
+
+* `derivatives` – Add `:create_on_promote` option for auto-creating derivatives on promotion (@janko)
+
+* `s3` – Add back support for client-side encryption (@janko)
+
+* `memory` – Ensure `Memory#open` returns content in original encoding (@jrochkind)
+
+## 3.2.2 (2020-08-05)
+
+* `s3` – Fix `S3#open` not working on aws-sdk-core 3.104 and above (@janko)
+
+## 3.2.1 (2020-01-12)
+
+* `derivation_endpoint` – Use `Rack::Files` constant on Rack >= 2.1 (@janko)
+
+* Fix Ruby 2.7 warnings regarding separation of positional and keyword arguments (@janko)
+
+* `s3` – Make `S3#open` handle empty S3 objects (@janko)
+
+## 3.2.0 (2019-12-17) [[release notes]](https://shrinerb.com/docs/release_notes/3.2.0)
+
+* `validation` – Run validation on `Attacher#attach` & `Attacher#attach_cached` instead of `Attacher#change` (@janko)
+
+* `remove_invalid` – Activate also when `Attacher#validate` is run manually (@janko)
+
+* `remove_invalid` – Fix incompatibility with `derivatives` plugin (@janko)
+
+* `type_predicates` – Add new plugin with convenient `UploadedFile` predicate methods based on MIME type (@janko)
+
+* `core` – Allow assigning back current attached file data (@janko)
+
+* `derivatives` – Fix `:derivative` value inconsistency when derivatives are being promoted (@janko)
+
+* `add_metadata` – Add `#add_metadata` method for adding metadata to uploaded files (@janko)
+
+* `derivatives` – Add `:io` and `:attacher` values to instrumentation event payload (@janko)
+
+## 3.1.0 (2019-11-15) [[release notes]](https://shrinerb.com/docs/release_notes/3.1.0)
+
+* `default_storage` – Coerce storage key to symbol in `Attacher#cache_key` & `Attacher#store_key` (@janko)
+
+* `core` – Coerce storage key to symbol in `Attacher#cache_key` & `Attacher#store_key` (@janko)
+
+* `add_metadata` – Define metadata methods only for the target uploader class (@janko)
+
+* `derivatives` – Add `:storage` option to `Attacher#create_derivatives` (@janko)
+
+* `store_dimensions` – Propagate exceptions on loading `ruby-vips` in `:vips` analyzer (@janko)
+
+* `signature` – Allow skipping rewinding by passing `rewind: false` to `Shrine.signature` (@janko)
+
+* `derivatives` – Add `Attacher.derivatives` alias for `Attacher.derivatives_processor` (@janko)
+
 ## 3.0.1 (2019-10-17) [[release notes]](https://shrinerb.com/docs/release_notes/3.0.1)
 
 * `metadata_attributes` – Fix exception being raised when there is no attached file (@janko)

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015-2017 Janko Marohnić
+Copyright (c) 2015-2019 Janko Marohnić
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,18 +1,22 @@
 # [Shrine]
 
+<img src="https://shrinerb.com/img/logo.png" width="100" alt="Shrine logo: a red paperclip" align="right" />
+
 Shrine is a toolkit for handling file attachments in Ruby applications. Some highlights:
 
 * **Modular design** – the [plugin system] allows you to load only the functionality you need
 * **Memory friendly** – streaming uploads and [downloads][Retrieving Uploads] make it work great with large files
 * **Cloud storage** – store files on [disk][FileSystem], [AWS S3][S3], [Google Cloud][GCS], [Cloudinary] and others
 * **Persistence integrations** – works with [Sequel], [ActiveRecord], [ROM], [Hanami] and [Mongoid] and others
-* **Flexible processing** – generate thumbnails [up front] or [on-the-fly] using [ImageMagick] or [libvips]
+* **Flexible processing** – generate thumbnails [eagerly] or [on-the-fly] using [ImageMagick] or [libvips]
 * **Metadata validation** – [validate files][validation] based on [extracted metadata][metadata]
 * **Direct uploads** – upload asynchronously [to your app][simple upload] or [to the cloud][presigned upload] using [Uppy]
 * **Resumable uploads** – make large file uploads [resumable][resumable upload] on [S3][uppy-s3_multipart] or [tus][tus-ruby-server]
 * **Background jobs** – built-in support for [background processing][backgrounding] that supports [any backgrounding library][Backgrounding Libraries]
 
-Please follow along with the **[Getting Started guide]**.
+If you're curious how it compares to other file attachment libraries, see the
+[Advantages of Shrine]. Otherwise, follow along with the **[Getting Started
+guide]**.
 
 ## Links
 
@@ -52,7 +56,7 @@ Shrine.plugin :restore_cached_data    # extracts metadata for assigned cached fi
 Next, add the `<name>_data` column to the table you want to attach files to. For
 an "image" attachment on a `photos` table this would be an `image_data` column:
 
-```sh
+```
 $ rails generate migration add_image_data_to_photos image_data:text
 ```
 
@@ -123,6 +127,10 @@ system.
 * Refile
 * Active Storage
 
+## Contributing
+
+Please refer to the [contributing page][Contributing].
+
 ## Code of Conduct
 
 Everyone interacting in the Shrine project’s codebases, issue trackers, and
@@ -133,6 +141,7 @@ mailing lists is expected to follow the [Shrine code of conduct][CoC].
 The gem is available as open source under the terms of the [MIT License].
 
 [Shrine]: https://shrinerb.com
+[Advantages of Shrine]: https://shrinerb.com/docs/advantages
 [plugin system]: https://shrinerb.com/docs/getting-started#plugin-system
 [Retrieving Uploads]: https://shrinerb.com/docs/retrieving-uploads
 [FileSystem]: https://shrinerb.com/docs/storage/file-system
@@ -144,8 +153,8 @@ The gem is available as open source under the terms of the [MIT License].
 [ROM]: https://github.com/shrinerb/shrine-rom
 [Hanami]: https://github.com/katafrakt/hanami-shrine
 [Mongoid]: https://github.com/shrinerb/shrine-mongoid
-[up front]: https://shrinerb.com/docs/getting-started#processing-up-front
-[on-the-fly]: https://shrinerb.com/docs/getting-started#processing-on-the-fly
+[eagerly]: https://shrinerb.com/docs/getting-started#eager-processing
+[on-the-fly]: https://shrinerb.com/docs/getting-started#on-the-fly-processing
 [ImageMagick]: https://github.com/janko/image_processing/blob/master/doc/minimagick.md#readme
 [libvips]: https://github.com/janko/image_processing/blob/master/doc/vips.md#readme
 [validation]: https://shrinerb.com/docs/validation
@@ -165,3 +174,4 @@ The gem is available as open source under the terms of the [MIT License].
 [Roda]: https://github.com/jeremyevans/roda
 [CoC]: /CODE_OF_CONDUCT.md
 [MIT License]: /LICENSE.txt
+[Contributing]: https://github.com/shrinerb/shrine/blob/master/CONTRIBUTING.md

data/doc/advantages.md

@@ -93,7 +93,7 @@ low-level abstractions that give you the flexibility to build your own flow.
 ```rb
 uploaded_file = ImageUploader.upload(image, :store) # metadata extraction, upload location generation
 uploaded_file.id       #=> "44ccafc10ce6a4ff22829e8f579ee6b9.jpg"
-uplaoded_file.metadata #=> { ... extracted metadata ... }
+uploaded_file.metadata #=> { ... extracted metadata ... }
 
 data = uploaded_file.to_json # serialization
 # ...
@@ -157,21 +157,21 @@ end
 
 ## Processing
 
-Most file attachment libraries provide either processing files up front
-(Paperclip, CarrierWave) or on-the-fly (Dragonfly, Refile, Active Storage).
+Most file attachment libraries allow you to process files either "eagerly"
+(Paperclip, CarrierWave) or "on-the-fly" (Dragonfly, Refile, Active Storage).
 However, each approach is suitable for different requirements. For instance,
 while on-the-fly processing is suitable for fast processing (image thumbnails,
 document previews), longer running processing (video transcoding, raw images)
 should be moved into a background job.
 
-That's why Shrine supports both [up front][derivatives] and
+That's why Shrine supports both [eager][derivatives] and
 [on-the-fly][derivation_endpoint] processing. For example, if you're handling
 image uploads, you can choose to either generate a set of pre-defined
 thumbnails during attachment:
 
 ```rb
 class ImageUploader < Shrine
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(original)
 
     {
@@ -261,7 +261,7 @@ gem "streamio-ffmpeg"
 ```
 ```rb
 class VideoUploader < Shrine
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     transcoded = Tempfile.new ["transcoded", ".mp4"]
     screenshot = Tempfile.new ["screenshot", ".jpg"]
 
@@ -281,7 +281,7 @@ movie.video(:transcoded) #=> #<Shrine::UploadedFile id="7481d6.mp4" ...>
 movie.video(:screenshot) #=> #<Shrine::UploadedFile id="8f3136.jpg" ...>
 ```
 
-## Metadata & Validation
+## Metadata
 
 Shrine automatically [extracts metadata][metadata] from each uploaded file,
 including derivatives like image thumbnails, and saves them into the database
@@ -289,6 +289,17 @@ column. In addition to filename, filesize, and MIME type that are extracted by
 default, you can also extract [image dimensions][store_dimensions], or your own
 [custom metadata][add_metadata].
 
+```rb
+class ImageUploader < Shrine
+  plugin :determine_mime_type # mime_type
+  plugin :store_dimensions    # width & height
+
+  add_metadata :resolution do |io|
+    image = MiniMagick::Image.new(io.path)
+    image.resolution
+  end
+end
+```
 ```rb
 photo.image.metadata #=>
 # {
@@ -297,23 +308,30 @@ photo.image.metadata #=>
 #   "mime_type" => "image/jpeg",
 #   "width" => 600,
 #   "height" => 400,
+#   "resolution" => [72, 72],
 #   ...
 # }
 ```
 
-For common metadata you can use the built-in [validators][validation_helpers],
-but you can also [validate any custom metadata][custom validations].
+## Validation
+
+For file validations there are [built-in validators][validation_helpers], but
+you can also just use plain Ruby code:
 
 ```rb
-class DocumentUploader < Shrine
+class ImageUploader < Shrine
+  plugin :validation_helpers
+
   Attacher.validate do
-    # validation macros
     validate_max_size 10*1024*1024
-    validate_mime_type %W[application/pdf]
+    validate_extension %w[jpg jpeg png webp]
+
+    if validate_mime_type %W[image/jpeg image/png image/webp]
+      validate_max_dimensions [5000, 5000]
 
-    # custom validations
-    if file["page_count"] > 30
-      errors << "must not have more than 30 pages"
+      unless ImageProcessing::MiniMagick.valid_image?(file.download.path)
+        error << "seems to be corrupted"
+      end
     end
   end
 end
@@ -432,7 +450,6 @@ on top of Rack, so that they can be used with any Ruby web framework.
 [backgrounding libraries]: https://github.com/shrinerb/shrine/wiki/Backgrounding-Libraries
 [Down streaming]: https://github.com/janko/down#streaming
 [validation_helpers]: https://shrinerb.com/docs/plugins/validation_helpers
-[custom validations]: https://shrinerb.com/docs/validation#custom-validations
 [derivatives]: https://shrinerb.com/docs/plugins/derivatives
 [derivation_endpoint]: https://shrinerb.com/docs/plugins/derivation_endpoint
 [libvips performance]: https://github.com/libvips/libvips/wiki/Speed-and-memory-use#results

data/doc/attacher.md

@@ -130,8 +130,8 @@ attacher.attach_cached(file)
 
 # sets cached file
 attacher.attach_cached('{"id":"asdf.jpg","storage":"cache","metadata":{...}}')
-attacher.attach_cached("id" => "asdf.jpg", "storage" => "cache", "metadata" => { ... })
-attacher.attach_cached(id: "asdf.jpg", storage: "cache", metadata: { ... })
+attacher.attach_cached({ "id" => "asdf.jpg", "storage" => "cache", "metadata" => { ... } })
+attacher.attach_cached({ id: "asdf.jpg", storage: "cache", metadata: { ... } })
 
 # unsets attached file
 attacher.attach_cached(nil)

data/doc/carrierwave.md

@@ -1,12 +1,12 @@
 ---
-title: Shrine for CarrierWave Users
+title: Upgrading from CarrierWave
 ---
 
 This guide is aimed at helping CarrierWave users transition to Shrine, and it
 consists of three parts:
 
 1. Explanation of the key differences in design between CarrierWave and Shrine
-2. Instructions how to migrate and existing app that uses CarrierWave to Shrine
+2. Instructions how to migrate an existing app that uses CarrierWave to Shrine
 3. Extensive reference of CarrierWave's interface with Shrine equivalents
 
 ## Overview
@@ -159,7 +159,7 @@ require "image_processing/mini_magick"
 class ImageUploader < Shrine
   plugin :derivatives
 
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(original)
 
     {
@@ -268,18 +268,24 @@ Files] guide explains this setup in more detail.
 ## Migrating from CarrierWave
 
 You have an existing app using CarrierWave and you want to transfer it to
-Shrine. Let's assume we have a `Photo` model with the "image" attachment. First
-we need to create the `image_data` column for Shrine:
+Shrine. Let's assume we have a `Photo` model with the "image" attachment.
+
+### 1. Add Shrine column
+
+First we need to create the `image_data` column for Shrine:
 
 ```rb
 add_column :photos, :image_data, :text # or :json or :jsonb if supported
 ```
 
-Afterwards we need to make new uploads write to the `image_data` column. This
-can be done by including the below module to all models that have CarrierWave
-attachments:
+### 2. Dual write
+
+Next, we need to make new CarrierWave attachments write to the
+`image_data` column. This can be done by including the below module to all
+models that have CarrierWave attachments:
 
 ```rb
+# config/initializers/shrine.rb (Rails)
 require "shrine"
 
 Shrine.storages = {
@@ -289,8 +295,7 @@ Shrine.storages = {
 
 Shrine.plugin :model
 Shrine.plugin :derivatives
-```
-```rb
+
 module CarrierwaveShrineSynchronization
   def self.included(model)
     model.before_save do
@@ -302,13 +307,13 @@ module CarrierwaveShrineSynchronization
 
   def write_shrine_data(name)
     uploader = send(name)
-    attacher = Shrine::Attacher.form_model(self, name)
+    attacher = Shrine::Attacher.from_model(self, name)
 
     if read_attribute(name).present?
       attacher.set shrine_file(uploader)
 
-      uploader.versions.each do |name, version|
-        attacher.merge_derivatives(name => shrine_file(version))
+      uploader.versions.each do |version_name, version|
+        attacher.merge_derivatives(version_name => shrine_file(version))
       end
     else
       attacher.set nil
@@ -317,19 +322,22 @@ module CarrierwaveShrineSynchronization
 
   private
 
-  # If you'll be using `:prefix` on your Shrine storage, make sure to
-  # subtract it from the path assigned as `:id`.
   def shrine_file(uploader)
     name     = uploader.mounted_as
     filename = read_attribute(name)
-    path     = uploader.store_path(filename)
+    location = uploader.store_path(filename)
+    location = location.sub(%r{^#{storage.prefix}/}, "") if storage.prefix
 
     Shrine.uploaded_file(
       storage:  :store,
-      id:       path,
+      id:       location,
       metadata: { "filename" => filename },
     )
   end
+
+  def storage
+    Shrine.storages[:store]
+  end
 end
 ```
 ```rb
@@ -340,20 +348,27 @@ end
 ```
 
 After you deploy this code, the `image_data` column should now be successfully
-synchronized with new attachments.  Next step is to run a script which writes
-all existing CarrierWave attachments to `image_data`:
+synchronized with new attachments.
+
+### 3. Data migration
+
+Next step is to run a script which writes all existing CarrierWave attachments
+to `image_data`:
 
 ```rb
 Photo.find_each do |photo|
-  Photo.uploaders.each_key { |name| photo.write_shrine_data(name) }
+  photo.write_shrine_data(:image)
   photo.save!
 end
 ```
 
+### 4. Rewrite code
+
 Now you should be able to rewrite your application so that it uses Shrine
-instead of CarrierWave, using equivalent Shrine storages. For help with
-translating the code from CarrierWave to Shrine, you can consult the reference
-below.
+instead of CarrierWave (you can consult the reference in the next section). You
+can remove the `CarrierwaveShrineSynchronization` module as well.
+
+### 5. Backill metadata
 
 You'll notice that Shrine metadata will be absent from the migrated files'
 data. You can run a script that will fill in any missing metadata defined in
@@ -363,11 +378,20 @@ your Shrine uploader:
 Shrine.plugin :refresh_metadata
 
 Photo.find_each do |photo|
-  photo.image_attacher.refresh_metadata!
-  photo.save
+  attacher = photo.image_attacher
+  attacher.refresh_metadata!
+  attacher.atomic_persist
 end
 ```
 
+### 6. Remove CarrierWave column
+
+If everything is looking good, we can remove the CarrierWave column:
+
+```rb
+remove_column :photos, :image
+```
+
 ## CarrierWave to Shrine direct mapping
 
 ### `CarrierWave::Uploader::Base`
@@ -395,7 +419,7 @@ Processing is defined by using the `derivatives` plugin:
 class ImageUploader < Shrine
   plugin :derivatives
 
-  Attacher.derivatives_processor do |original|
+  Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(image)
 
     {
@@ -455,16 +479,25 @@ photo.image.original_filename #=> "avatar.jpg"
 
 #### `#store_dir`, `#cache_dir`
 
-Shrine here provides a `#generate_location` method, which is triggered for all
-storages:
+Shrine here provides a single `#generate_location` method that's triggered for
+all storages:
 
 ```rb
 class ImageUploader < Shrine
-  def generate_location(io, record: nil, **)
-    "#{record.class}/#{record.id}/#{io.original_filename}"
+  def generate_location(io, record: nil, name: nil, **)
+    [ storage_key,
+      record && record.class.name.underscore,
+      record && record.id,
+      super,
+      io.original_filename ].compact.join("/")
   end
 end
 ```
+```
+cache/user/123/2feff8c724e7ce17/nature.jpg
+store/user/456/7f99669fde1e01fc/kitten.jpg
+...
+```
 
 You might also want to use the `pretty_location` plugin for automatically
 generating an organized folder structure.
@@ -477,8 +510,8 @@ For default URLs you can use the `default_url` plugin:
 class ImageUploader < Shrine
   plugin :default_url
 
-  Attacher.default_url do |options|
-    "/attachments/#{name}/default.jpg"
+  Attacher.default_url do |derivative: nil, **|
+    "/fallbacks/#{derivative || "original"}.jpg"
   end
 end
 ```
@@ -499,7 +532,7 @@ class ImageUploader < Shrine
 end
 ```
 
-#### `#blacklist_mime_type_pattern`, `#whitelist_mime_type_pattern`, `#content_type_whitelist`, `#content_type_blacklist`
+####  `#content_type_whitelist`, `#content_type_blacklist`
 
 In Shrine, MIME type whitelisting/blacklisting is part of validations, and is
 provided by the `validation_helpers` plugin, though it doesn't support regexes:
@@ -515,6 +548,17 @@ class ImageUploader < Shrine
 end
 ```
 
+Make sure to also load the `determine_mime_type` plugin to detect MIME type
+from file content.
+
+```rb
+# Gemfile
+gem "mimemagic"
+```
+```rb
+Shrine.plugin :determine_mime_type, analyzer: :mimemagic
+```
+
 #### `#size_range`
 
 In Shrine file size validations are typically done using the
@@ -622,7 +666,7 @@ shows what are Shrine's equivalents.
 
 #### `root`, `base_path`, `permissions`, `directory_permissions`
 
-In Shrine these are configured on the FileSystem storage directly.
+In Shrine these are configured on the `FileSystem` storage directly.
 
 #### `storage`, `storage_engines`