shrine 2.19.3 → 3.6.0

data/doc/release_notes/3.0.1.md

@@ -0,0 +1,22 @@
+---
+title: Shrine 3.0.1
+---
+
+## Regressions
+
+* Fixed `metadata_attributes` plugin raising an exception when the attachment
+  is removed (assigned to `nil`).
+
+## Improvements
+
+* Simplified `UploadedFile#inspect` output:
+
+  ```rb
+  # before
+  uploaded_file.inspect
+  #=> #<Shrine::UploadedFile:0x00007fa4e4140d30 @id="cda84bbdab12b2bd41ea34590060a807", @storage_key=:memory, @metadata={"filename"=>nil, "size"=>0, "mime_type"=>nil}>
+
+  # after
+  uploaded_file.inspect
+  #=> #<Shrine::UploadedFile id="488b887dd792b4c82d3fc445140c5a38" storage=:memory metadata={"filename"=>nil, "size"=>0, "mime_type"=>nil}>
+  ```

data/doc/release_notes/3.1.0.md

@@ -0,0 +1,73 @@
+---
+title: Shrine 3.1.0
+---
+
+## New features
+
+* The `Attacher#create_derivatives` method now accepts a `:storage` option for
+  specifying the storage to which derivatives should be uploaded.
+
+  ```rb
+  # with attachment module
+  photo.image_derivatives!(storage: :other_store)
+
+  # with attacher
+  attacher.create_derivatives(storage: :other_store)
+  ```
+
+* The `Shrine.calculate_signature` now accepts a `:rewind` boolean option for
+  choosing whether the IO object should be rewinded after reading. This is
+  useful if you want to calculate signature from non-rewindable IO objects,
+  such as `IO.pipe`, `Socket`, non-rewindable `Down::ChunkedIO` etc.
+
+  ```rb
+  Shrine.signature(io, rewind: false)
+  ```
+
+## Improvements
+
+* The derivatives processor can now be registered with `Attacher.derivatives`,
+  which is just an alias for `Attacher.derivatives_processor`.
+
+  ```rb
+  class ImageUploader < Shrine
+    Attacher.derivatives_processor do |original|
+      # ...
+    end
+  end
+
+  # can now be written as
+
+  class ImageUploader < Shrine
+    Attacher.derivatives do |original|
+      # ...
+    end
+  end
+  ```
+
+* The `Attacher#cached?` and `Attacher#stored?` methods now work correctly if
+  temporary/permanent storage identifiers were specified as strings.
+
+* The `store_dimensions` plugin now properly propagates exceptions when loading
+  the `ruby-vips` gem in `:vips` analyzer.
+
+* The `add_metadata` plugin now respects inheritance again when defining
+  metadata methods on the `Shrine::UploadedFile` class. In 2.19.0, the
+  `add_metadata` plugin was changed to define metadata methods on the internal
+  `FileMethods` plugin module, which is shared across all uploaders. This
+  change has now been reverted.
+
+## Backwards compatibility
+
+* The `Attacher#cache_key` and `Attacher#store_key` methods now always return
+  symbol keys, even if the storage key that was specified was a string key.
+
+  ```rb
+  attacher = Shrine::Attacher.new(cache: "cache", store: "store")
+  attacher.cache_key #=> :cache (previously "cache")
+  attacher.store_key #=> :store (previously "store")
+  ```
+
+* The `add_metadata` plugin now defines metadata methods directly on the
+  `UploadedFile` class, which means that if you happen to have been overriding
+  these metadata methods and calling `super`, this won't work anymore.

data/doc/release_notes/3.2.0.md

@@ -0,0 +1,96 @@
+---
+title: Shrine 3.2.0
+---
+
+## New features
+
+* The `type_predicates` plugin has been added, which adds convenient predicate
+  methods to `Shrine::UploadedFile` based on the MIME type.
+
+  ```rb
+  # Gemfile
+  gem "mini_mime" # default dependency of type_predicates
+  ```
+  ```rb
+  Shrine.plugin :type_predicates
+  ```
+
+  The plugin adds four predicate methods based on the general type of the file:
+
+  ```rb
+  file.image? # returns true for any "image/*" MIME type
+  file.video? # returns true for any "video/*" MIME type
+  file.audio? # returns true for any "audio/*" MIME type
+  file.text?  # returns true for any "text/*" MIME type
+  ```
+
+  You can also check for specific MIME type using the extension name:
+
+  ```rb
+  file.type?(:jpg) # returns true if MIME type is "image/jpeg"
+  file.type?(:svg) # returns true if MIME type is "image/svg+xml"
+  file.type?(:mov) # returns true if MIME type is "video/quicktime"
+  file.type?(:ppt) # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ...
+  ```
+
+  For convenience, you can create predicate methods for specific file types:
+
+  ```rb
+  Shrine.plugin :type_predicates, methods: %i[jpg svg mov ppt]
+  ```
+  ```rb
+  file.jpg? # returns true if MIME type is "image/jpeg"
+  file.svg? # returns true if MIME type is "image/svg+xml"
+  file.mov? # returns true if MIME type is "video/quicktime"
+  file.ppt? # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ```
+
+* The `#add_metadata` method has been added to the `add_metadata` plugin for
+  adding new metadata to an existing file/attachment.
+
+  ```rb
+  attacher.file.metadata #=> { ... }
+  attacher.add_metadata("foo" => "bar")
+  attacher.file.metadata #=> { ..., "foo" => "bar" }
+  ```
+
+## Other improvements
+
+* The `remove_invalid` plugin now works correctly with `derivatives` plugin.
+
+* The `remove_invalid` plugin is now also activated when `Attacher#validate`
+  is called manually.
+
+* The current attached file data can now be assigned back to the attachment
+  attribute, and this operation will be a no-op.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:store metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "store", "metadata" => { ... } } # no-op
+  ```
+
+  This allows treating the attachment attribute as a persistent attribute,
+  where the current value can be assigned back on record updates.
+
+* When promoting derivatives, the `:derivative` parameter value was being
+  passed to the uploader as an array. This has been fixed, and the value is now
+  the same as when uploading derivatives directly to permanent storage.
+
+* The `derivatives` plugin now includes additional `:io` and `:attacher` values
+  in the instrumentation event payload.
+
+## Backwards compatibility
+
+* The `validation` plugin now runs validations on `Attacher#attach` and
+  `Attacher#attach_cached`. If you were using `Attacher#change` directly and
+  expecting the validations to be run automatically, you will need to update
+  your code.
+
+* If you were updating the cached file metadata via file data assignment, this
+  will no longer work.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:cache metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "cache", "metadata" => { ... } } # no-op
+  ```

data/doc/release_notes/3.2.1.md

@@ -0,0 +1,31 @@
+---
+title: Shrine 3.2.1
+---
+
+## Ruby 2.7 compatibility
+
+* Shrine doesn't trigger [Ruby 2.7 warnings for separation of positional and
+  keyword arguments][kwargs] anymore.
+
+* Down 5.1.0 has been released, which resolves warnings and a `FrozenError`
+  exception on Ruby 2.7. Shrine now requires at least this version of Down.
+
+  If you're using `Down::Http`, make sure you're using http.rb 4.3.0 or newer.
+
+* ImageProcessing 1.10.3 gem has been released which resolves Ruby 2.7 warnings
+  as well. If you're using it for image processing, make sure to upgrade to
+  this version:
+
+  ```rb
+  gem "image_processing", ">= 1.10.3", "< 2"
+  ```
+
+## Rack 2.1 compatibility
+
+* The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1 or newer.
+
+## Other improvements 
+
+* The `S3#open` method now handles empty S3 objects.
+
+[kwargs]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/

data/doc/release_notes/3.2.2.md

@@ -0,0 +1,14 @@
+---
+title: Shrine 3.2.2
+---
+
+## Bug fixes
+
+* aws-sdk-core 3.104.0 introduced a backwards incompatible changes that caused
+  `Shrine::Storage::S3#open` to start raising an exception.
+
+  ```
+  NoMethodError: undefined method `bytesize' for #<Array:0x000000000a721be0>
+  ```
+
+  This has now been fixed.

data/doc/release_notes/3.3.0.md

@@ -0,0 +1,105 @@
+---
+title: Shrine 3.3.0
+---
+
+## New features
+
+* The `:create_on_promote` option has been added to the `derivatives` plugin
+  for automatically creating derivatives after the attached cached file is
+  promoted to permanent storage.
+
+  ```rb
+  Shrine.plugin :derivatives, create_on_promote: true
+  ```
+
+* The `:auto_extraction` option has been added to the `store_dimensions` plugin
+  for skipping automatically extracting dimensions on upload.
+
+  ```rb
+  Shrine.plugin :store_dimensions, auto_extraction: false
+  ```
+
+* The `:skip_nil` option has been added to the `add_metadata` plugin for
+  excluding metadata keys whose values are nil.
+
+  ```rb
+  class PdfUploader < Shrine
+    add_metadata :pages, skip_nil: true do |io|
+      if is_pdf?(io)
+        reader = PDF::Reader.new(io)
+        reader.page_count
+      else
+        # If this is not a PDF, then the pages metadata will not be stored
+        nil
+      end
+    end
+  end
+  ```
+
+* The `:download` option has been added to derivatives processors in
+  `derivatives` plugin for skipping converting the source IO object into a
+  file. This can be used to avoid a potentially expensive download/copy when
+  the derivatives processor doesn't need the file.
+
+  ```rb
+  Attacher.derivatives :my_processor, download: false do |source|
+    source #=> Could be File, Shrine::UploadedFile, or other IO-like object
+    shrine_class.with_file(source) do |file|
+      # can force download/copy if necessary with `with_file`,
+    end
+  end
+  ```
+
+## Bug fixes
+
+* The `upload_endpoint` now handles calling `Shrine.upload_response` method
+  from a Rails controller.
+
+* The `derivation_endpoint` plugin now applies the `version` query parameter
+  to the derivation when creating the response.
+
+## Other improvements
+
+* The new `Aws:S3::EncryptionV2::Client` is now supported by the S3 storage for
+  client-side encryption.
+
+* The `derivation_endpoint` now reduces the possibility of timing attacks by
+  comparing URL signatures in constant time using `Rack::Utils.secure_compare`.
+
+* The `derivatives` plugin now copies non-file source IO objects to disk before
+  passing them to the processor. This is consistent with how a
+  `Shrine::UploadedFile` object is downloaded to disk.
+
+* The `sequel` and `activerecord` plugins now call `Attacher#reload` when
+  reloading the model, which reloads the attached files but keeps other
+  attacher state.
+
+* The `derivatives` plugin doesn't download the attached file anymore if
+  attempting to process derivatives when no derivatives processor was defined.
+
+* The `mirroring` plugin now forwards attacher options when uploading to mirror
+  storages.
+
+* The `presign_endpoint` plugin now handles the `OPTIONS` HTTP verb, which
+  newer versions of Uppy are requesting.
+
+* `Shrine::Storage::Memory#open` now always returns a `StringIO` in the file
+  content's original encoding, instead of the encoding set by
+  `Encoding.default_internal`. This works around a [bug][ruby-lang #16497]
+  in `StringIO` introduced in Ruby 2.7.0.
+
+* The `remove_attachment` plugin now deletes the removed file if a new file was
+  attached right after removal.
+
+## Backwards compatibility
+
+* If you were passing a non-file IO object to the derivatives processor, Shrine
+  will now convert it into a file beforehand. If you're currently doing this
+  and are converting the IO object into a file inside the processor, you can
+  now remove the conversion code to avoid doubling the amount of disk writes.
+
+* When reloading a Sequel/ActiveRecord model, any attacher state other than
+  uploaded files will now be retained after the reload. If you were relying on
+  all the attacher state being re-initialized, you'll need to update your code.
+
+[ruby-lang #16497]: https://bugs.ruby-lang.org/issues/16497

data/doc/release_notes/3.4.0.md

@@ -0,0 +1,35 @@
+---
+title: Shrine 3.4.0
+---
+
+* Passing attacher options to `Shrine.Attachment` method now works on Ruby 3.0.
+
+* Defining validation errors as an array of I18n key and options in
+  `activerecord` plugin now works on Ruby 3.0.
+
+* The `:fastimage` MIME type analyzer now correctly detects SVGs as
+  `image/svg+html` in the `determine_mime_type` plugin.
+
+* The `Shrine::Attacher#read` method provided by the `entity` plugin is now
+  public. This is consistent with `Shrine::Attacher#write` from `model` plugin
+  being public as well.
+
+* The `Shrine::Attacher#reload` method now resets attachment's dirty state.
+  This means that for a model whose `Attacher#changed?` returns `true`, calling
+  `#reload` on the model will make `Attacher#changed?` return `false`. This was
+  the behaviour before Shrine 3.3.0.
+
+  ```rb
+  # before
+  model.file_attacher.changed? #=> true
+  model.reload
+  model.file_attacher.changed? #=> true
+
+  # after
+  model.file_attacher.changed? #=> true
+  model.reload
+  model.file_attacher.changed? #=> false
+  ```
+
+* Calling `#reload` on the model will not initialize a `Shrine::Attacher`
+  instance anymore if one hasn't previously been initialized.

data/doc/release_notes/3.5.0.md

@@ -0,0 +1,63 @@
+---
+title: Shrine 3.5.0
+---
+
+## New features
+
+* The website has been migrated to Docusaurus v2. :sparkles:
+
+* The `:signer` option has been added to the `derivation_endpoint` plugin, for when you want to use custom URL signing. This is useful when using `:expires_in`, and wanting to have expiring URLs work with CDN caching.
+
+  ```rb
+  require "aws-sdk-cloudfront"
+  signer = Aws::CloudFront::UrlSigner.new(key_pair_id: "...", private_key: "...")
+
+  plugin :derivation_endpoint,
+    expires_in: 90,
+    signer: -> (url, expires_in:) do
+      signer.signed_url(url, expires: Time.now.to_i + expires_in)
+    end
+  ```
+
+* The S3 storage now supports `:max_multipart_parts` option for specifying the maximum number of concurrent parts in which a large file will get uploaded. This number defaults to `10_000`.
+
+  ```rb
+  Shrine::Storage::S3.new(max_multipart_parts: 1000, ...)
+  ```
+
+* The `:encoding` option can now be passed to `S3#open`, which is applied to downloaded chunks.
+
+  ```rb
+  io = uploaded_file.open(encoding: Encoding::UTF_8)
+  csv = CSV.new(io)
+  # ...
+  ```
+
+## Other improvements
+
+* Passing a boolean value to the `#remove_attachment=` setter now works on Ruby 3.2. Previously this would raise an error, because Shrine would try to call `=~` on it, but `Object#=~` method has been removed in Ruby 3.2.
+
+* When duplicating a model instance, the duplicated attacher now references the duplicated model instance instead of the original one.
+
+* The download endpoint now returns a `400 Bad Request` response when the serialized file component is invalid.
+
+* The `derivatives` plugin now supports passing `mutex: false` option to disable usage of a mutex. This makes the `Shrine::Attacher` object marshallable, which should enable using `Marshal.dump` and `Marshal.load` on model instances with attachments. This should be safe unless you're adding derivatives on the same attacher object concurrently.
+
+* When loading the `derivatives` plugin with `versions_compatibility: true`, this setting doesn't leak to other uploaders anymore. Previously if other uploaders would load `derivatives` plugin without this option, versions compatibility would still get enabled for them. This change also fixes behavior on JRuby.
+
+* When S3 storage copies files, the AWS tag are not inherited anymore. This allows passing the `:tagging` upload option when promoting from temporary to permanent storage, and have it take effect.
+
+* The `UploadedFile#url` method doesn't call the obsolete `URI.regexp` method anymore, which should avoid warnings.
+
+* The `infer_extension` plugin now defines `infer_extension` instance method (in addition to class method) on the uploader for convenience, so that it can be easily called at the uploader instance level.
+
+  ```rb
+  class MyUploader < Shrine
+    plugin :infer_extension
+
+    def generate_location(io, metadata:, **)
+      extension = infer_extension(metadata["mime_type"])
+      # ...
+    end
+  end
+  ```

data/doc/release_notes/3.6.0.md

@@ -0,0 +1,23 @@
+---
+title: Shrine 3.6.0
+---
+
+## New features
+
+* The S3 storage now accepts `:copy_options` when initializing. This can be used for supporting Cloudflare R2 by removing `:tagging_directive` when copying file from temporary to permanent storage.
+
+  ```rb
+  Shrine::Storage::S3.new(bucket: BUCKET, copy_options: {}, **s3_options)
+  ```
+
+## Other improvements
+
+* Rack 3 is now supported.
+
+* When duplicating the attacher, the `Attacher#context` hash is now copied as well, instead of being kept the same between the two attachers.
+
+* After `UploadedFile#close` was called, `UploadedFile#opened?` will return `false` and the uploaded file can be implicitly re-opened again.
+
+## Backwards compatibility
+
+* Shrine API that is returning a rack response triple will now return headers as an instance of `Rack::Headers` on Rack 3, which is a subclass of `Hash`. This should keep user code that references header names in mixed case working (in addition to lowercase), but could theoretically cause issues for code explicitly requiring headers to be an instance of `Hash`.

data/doc/retrieving_uploads.md

@@ -1,4 +1,7 @@
-# Retrieving Uploads
+---
+id: retrieving-uploads
+title: Retrieving Uploads
+---
 
 Uploaded file content is typically retrieved from the storage using a
 `Shrine::UploadedFile` object. This guide explains the various methods of
@@ -121,7 +124,7 @@ end # underlying IO object is closed
 ```
 
 `Shrine::UploadedFile#open` will return the result of a given block.
-block. We can use that to safely retrieve the whole content of a file, without
+We can use that to safely retrieve the whole content of a file, without
 leaving any temporary files lying around.
 
 ```rb

data/doc/securing_uploads.md

@@ -1,9 +1,12 @@
-# Securing uploads
+---
+id: securing-uploads
+title: Securing Uploads
+---
 
 Shrine does a lot to make your file uploads secure, but there are still a lot
 of security measures that could be added by the user on the application's side.
-This guide will try to cover all the well-known security issues, ranging from
-the obvious ones to not-so-obvious ones, and try to provide solutions.
+This guide will try to cover some well-known security issues, ranging from the
+obvious ones to not-so-obvious ones, and try to provide solutions.
 
 ## Validate file type
 
@@ -12,17 +15,21 @@ idea to create a whitelist (or a blacklist) of extensions and MIME types.
 
 By default Shrine stores the MIME type derived from the extension, which means
 it's not guaranteed to hold the actual MIME type of the the file. However, you
-can load the `determine_mime_type` plugin which by default uses the [file]
-utility to determine the MIME type from magic file headers.
+can load the `determine_mime_type` plugin to determine MIME type from magic
+file headers.
 
+```rb
+# Gemfile
+gem "marcel", "~> 0.3"
+```
 ```rb
 class MyUploader < Shrine
+  plugin :determine_mime_type, analyzer: :marcel
   plugin :validation_helpers
-  plugin :determine_mime_type
 
   Attacher.validate do
-    validate_extension_inclusion %w[jpg jpeg png gif]
-    validate_mime_type_inclusion %w[image/jpeg image/png image/gif]
+    validate_extension %w[jpg jpeg png webp]
+    validate_mime_type %w[image/jpeg image/png image/webp]
   end
 end
 ```
@@ -31,22 +38,23 @@ end
 
 It's a good idea to generally limit the filesize of uploaded files, so that
 attackers cannot easily flood your storage. There are various layers at which
-you can apply filesize limits, depending on how you're accepting uploads.
-Firstly, you should probably add a filesize validation to prevent large files
-from being uploaded to `:store`:
+you can apply filesize limits, depending on how you're accepting uploads. For
+starters you can add a filesize validation to prevent large files from being
+uploaded to `:store`:
 
 ```rb
 class MyUploader < Shrine
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_max_size 20*1024*1024 # 20 MB
+    validate_max_size 100*1024*1024 # 100 MB
   end
 end
 ```
 
-In the following sections we talk about various strategies to prevent files from
-being uploaded to cache and the temporary directory.
+In the following sections we talk about various strategies to prevent files
+from being uploaded to Shrine's temporary storage and the system's temporary
+directory.
 
 ### Limiting filesize in direct uploads
 
@@ -55,7 +63,7 @@ in the `:max_size` option to reject files that are larger than the specified
 limit:
 
 ```rb
-plugin :upload_endpoint, max_size: 20*1024*1024 # 20 MB
+plugin :upload_endpoint, max_size: 100*1024*1024 # 100 MB
 ```
 
 If you're doing direct uploads to Amazon S3 using the `presign_endpoint`
@@ -63,7 +71,7 @@ plugin, you can pass in the `:content_length_range` presign option:
 
 ```rb
 plugin :presign_endpoint, presign_options: -> (request) do
-  { content_length_range: 0..20*1024*1024 }
+  { content_length_range: 0..100*1024*1024 }
 end
 ```
 
@@ -92,20 +100,17 @@ loading the `remove_invalid` plugin.
 plugin :remove_invalid
 ```
 
-### Paranoid filesize limiting
+### Failsafe filesize limiting
 
-If you want to make sure that no large files ever get to your storages, and
-you don't really care about the error message, you can use the `hooks` plugin
-and raise an error:
+If you want to make sure that no large files ever get to your storages, and you
+don't really care about the error message, you can override `Shrine#upload`:
 
 ```rb
-class MyUploader
-  plugin :hooks
+class MyUploader < Shrine
+  def upload(io, **options)
+    fail FileTooLarge if io.size >= 100*1024*1024
 
-  def before_upload(io, context)
-    if io.respond_to?(:read)
-      raise FileTooLarge if io.size >= 20*1024*1024
-    end
+    super
   end
 end
 ```
@@ -118,24 +123,43 @@ image processing, since processing them can take a lot of time and memory. This
 makes it trivial to DoS the application which doesn't have any protection
 against them.
 
-Shrine uses the [fastimage] gem for determining image dimensions which has
-built-in protection against image bombs (ImageMagick for example doesn't), but
-you still need to prevent those files from being attached and processed:
+So, in addition to validating filesize, we should also validate image
+dimensions:
 
 ```rb
-class MyUploader < Shrine
+# Gemfile
+gem "fastimage"
+```
+```rb
+class ImageUploader < Shrine
   plugin :store_dimensions
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_max_width  2500
-    validate_max_height 2500
+    validate_max_size 100*1024*1024
+
+    if validate_mime_type %w[image/jpeg image/png image/webp]
+      validate_max_dimensions [5000, 5000]
+    end
   end
 end
 ```
 
-If you're doing processing on caching, you can use the fastimage gem directly
-in a conditional.
+If you want to be extra safe, you can add a failsafe before performing
+processing:
+
+```rb
+class ImageUploader < Shrine
+  # ...
+  Attacher.derivatives do |original|
+    width, height = Shrine.dimensions(original)
+
+    fail ImageBombError if width > 5000 || height > 5000
+
+    # ...
+  end
+end
+```
 
 ## Prevent metadata tampering
 
@@ -153,7 +177,7 @@ app. To guard yourself from such attacks, you can load the
 cached files on assignment and override the received metadata.
 
 ```rb
-plugin :restore_cached_data
+Shrine.plugin :restore_cached_data
 ```
 
 ## Limit number of files
@@ -172,6 +196,7 @@ class MyUploader < Shrine
 
   Attacher.validate do
     validate_min_size 10*1024 # 10 KB
+    # ...
   end
 end
 ```
@@ -183,6 +208,4 @@ end
 * [AppSec: 8 Basic Rules to Implement Secure File Uploads](https://software-security.sans.org/blog/2009/12/28/8-basic-rules-to-implement-secure-file-uploads/)
 
 [image bombs]: https://www.bamsoftware.com/hacks/deflate.html
-[fastimage]: https://github.com/sdsykes/fastimage
-[file]: http://linux.die.net/man/1/file
 [rack-attack]: https://github.com/kickstarter/rack-attack