shrine 2.19.4 → 3.4.0

data/doc/release_notes/3.0.1.md

@@ -0,0 +1,22 @@
+---
+title: Shrine 3.0.1
+---
+
+## Regressions
+
+* Fixed `metadata_attributes` plugin raising an exception when the attachment
+  is removed (assigned to `nil`).
+
+## Improvements
+
+* Simplified `UploadedFile#inspect` output:
+
+  ```rb
+  # before
+  uploaded_file.inspect
+  #=> #<Shrine::UploadedFile:0x00007fa4e4140d30 @id="cda84bbdab12b2bd41ea34590060a807", @storage_key=:memory, @metadata={"filename"=>nil, "size"=>0, "mime_type"=>nil}>
+
+  # after
+  uploaded_file.inspect
+  #=> #<Shrine::UploadedFile id="488b887dd792b4c82d3fc445140c5a38" storage=:memory metadata={"filename"=>nil, "size"=>0, "mime_type"=>nil}>
+  ```

data/doc/release_notes/3.1.0.md

@@ -0,0 +1,73 @@
+---
+title: Shrine 3.1.0
+---
+
+## New features
+
+* The `Attacher#create_derivatives` method now accepts a `:storage` option for
+  specifying the storage to which derivatives should be uploaded.
+
+  ```rb
+  # with attachment module
+  photo.image_derivatives!(storage: :other_store)
+
+  # with attacher
+  attacher.create_derivatives(storage: :other_store)
+  ```
+
+* The `Shrine.calculate_signature` now accepts a `:rewind` boolean option for
+  choosing whether the IO object should be rewinded after reading. This is
+  useful if you want to calculate signature from non-rewindable IO objects,
+  such as `IO.pipe`, `Socket`, non-rewindable `Down::ChunkedIO` etc.
+
+  ```rb
+  Shrine.signature(io, rewind: false)
+  ```
+
+## Improvements
+
+* The derivatives processor can now be registered with `Attacher.derivatives`,
+  which is just an alias for `Attacher.derivatives_processor`.
+
+  ```rb
+  class ImageUploader < Shrine
+    Attacher.derivatives_processor do |original|
+      # ...
+    end
+  end
+
+  # can now be written as
+
+  class ImageUploader < Shrine
+    Attacher.derivatives do |original|
+      # ...
+    end
+  end
+  ```
+
+* The `Attacher#cached?` and `Attacher#stored?` methods now work correctly if
+  temporary/permanent storage identifiers were specified as strings.
+
+* The `store_dimensions` plugin now properly propagates exceptions when loading
+  the `ruby-vips` gem in `:vips` analyzer.
+
+* The `add_metadata` plugin now respects inheritance again when defining
+  metadata methods on the `Shrine::UploadedFile` class. In 2.19.0, the
+  `add_metadata` plugin was changed to define metadata methods on the internal
+  `FileMethods` plugin module, which is shared across all uploaders. This
+  change has now been reverted.
+
+## Backwards compatibility
+
+* The `Attacher#cache_key` and `Attacher#store_key` methods now always return
+  symbol keys, even if the storage key that was specified was a string key.
+
+  ```rb
+  attacher = Shrine::Attacher.new(cache: "cache", store: "store")
+  attacher.cache_key #=> :cache (previously "cache")
+  attacher.store_key #=> :store (previously "store")
+  ```
+
+* The `add_metadata` plugin now defines metadata methods directly on the
+  `UploadedFile` class, which means that if you happen to have been overriding
+  these metadata methods and calling `super`, this won't work anymore.

data/doc/release_notes/3.2.0.md

@@ -0,0 +1,96 @@
+---
+title: Shrine 3.2.0
+---
+
+## New features
+
+* The `type_predicates` plugin has been added, which adds convenient predicate
+  methods to `Shrine::UploadedFile` based on the MIME type.
+
+  ```rb
+  # Gemfile
+  gem "mini_mime" # default dependency of type_predicates
+  ```
+  ```rb
+  Shrine.plugin :type_predicates
+  ```
+
+  The plugin adds four predicate methods based on the general type of the file:
+
+  ```rb
+  file.image? # returns true for any "image/*" MIME type
+  file.video? # returns true for any "video/*" MIME type
+  file.audio? # returns true for any "audio/*" MIME type
+  file.text?  # returns true for any "text/*" MIME type
+  ```
+
+  You can also check for specific MIME type using the extension name:
+
+  ```rb
+  file.type?(:jpg) # returns true if MIME type is "image/jpeg"
+  file.type?(:svg) # returns true if MIME type is "image/svg+xml"
+  file.type?(:mov) # returns true if MIME type is "video/quicktime"
+  file.type?(:ppt) # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ...
+  ```
+
+  For convenience, you can create predicate methods for specific file types:
+
+  ```rb
+  Shrine.plugin :type_predicates, methods: %i[jpg svg mov ppt]
+  ```
+  ```rb
+  file.jpg? # returns true if MIME type is "image/jpeg"
+  file.svg? # returns true if MIME type is "image/svg+xml"
+  file.mov? # returns true if MIME type is "video/quicktime"
+  file.ppt? # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ```
+
+* The `#add_metadata` method has been added to the `add_metadata` plugin for
+  adding new metadata to an existing file/attachment.
+
+  ```rb
+  attacher.file.metadata #=> { ... }
+  attacher.add_metadata("foo" => "bar")
+  attacher.file.metadata #=> { ..., "foo" => "bar" }
+  ```
+
+## Other improvements
+
+* The `remove_invalid` plugin now works correctly with `derivatives` plugin.
+
+* The `remove_invalid` plugin is now also activated when `Attacher#validate`
+  is called manually.
+
+* The current attached file data can now be assigned back to the attachment
+  attribute, and this operation will be a no-op.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:store metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "store", "metadata" => { ... } } # no-op
+  ```
+
+  This allows treating the attachment attribute as a persistent attribute,
+  where the current value can be assigned back on record updates.
+
+* When promoting derivatives, the `:derivative` parameter value was being
+  passed to the uploader as an array. This has been fixed, and the value is now
+  the same as when uploading derivatives directly to permanent storage.
+
+* The `derivatives` plugin now includes additional `:io` and `:attacher` values
+  in the instrumentation event payload.
+
+## Backwards compatibility
+
+* The `validation` plugin now runs validations on `Attacher#attach` and
+  `Attacher#attach_cached`. If you were using `Attacher#change` directly and
+  expecting the validations to be run automatically, you will need to update
+  your code.
+
+* If you were updating the cached file metadata via file data assignment, this
+  will no longer work.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:cache metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "cache", "metadata" => { ... } } # no-op
+  ```

data/doc/release_notes/3.2.1.md

@@ -0,0 +1,31 @@
+---
+title: Shrine 3.2.1
+---
+
+## Ruby 2.7 compatibility
+
+* Shrine doesn't trigger [Ruby 2.7 warnings for separation of positional and
+  keyword arguments][kwargs] anymore.
+
+* Down 5.1.0 has been released, which resolves warnings and a `FrozenError`
+  exception on Ruby 2.7. Shrine now requires at least this version of Down.
+
+  If you're using `Down::Http`, make sure you're using http.rb 4.3.0 or newer.
+
+* ImageProcessing 1.10.3 gem has been released which resolves Ruby 2.7 warnings
+  as well. If you're using it for image processing, make sure to upgrade to
+  this version:
+
+  ```rb
+  gem "image_processing", ">= 1.10.3", "< 2"
+  ```
+
+## Rack 2.1 compatibility
+
+* The `derivation_endpoint` plugin now uses `Rack::Files` on Rack 2.1 or newer.
+
+## Other improvements 
+
+* The `S3#open` method now handles empty S3 objects.
+
+[kwargs]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/

data/doc/release_notes/3.2.2.md

@@ -0,0 +1,14 @@
+---
+title: Shrine 3.2.2
+---
+
+## Bug fixes
+
+* aws-sdk-core 3.104.0 introduced a backwards incompatible changes that caused
+  `Shrine::Storage::S3#open` to start raising an exception.
+
+  ```
+  NoMethodError: undefined method `bytesize' for #<Array:0x000000000a721be0>
+  ```
+
+  This has now been fixed.

data/doc/release_notes/3.3.0.md

@@ -0,0 +1,105 @@
+---
+title: Shrine 3.3.0
+---
+
+## New features
+
+* The `:create_on_promote` option has been added to the `derivatives` plugin
+  for automatically creating derivatives after the attached cached file is
+  promoted to permanent storage.
+
+  ```rb
+  Shrine.plugin :derivatives, create_on_promote: true
+  ```
+
+* The `:auto_extraction` option has been added to the `store_dimensions` plugin
+  for skipping automatically extracting dimensions on upload.
+
+  ```rb
+  Shrine.plugin :store_dimensions, auto_extraction: false
+  ```
+
+* The `:skip_nil` option has been added to the `add_metadata` plugin for
+  excluding metadata keys whose values are nil.
+
+  ```rb
+  class PdfUploader < Shrine
+    add_metadata :pages, skip_nil: true do |io|
+      if is_pdf?(io)
+        reader = PDF::Reader.new(io)
+        reader.page_count
+      else
+        # If this is not a PDF, then the pages metadata will not be stored
+        nil
+      end
+    end
+  end
+  ```
+
+* The `:download` option has been added to derivatives processors in
+  `derivatives` plugin for skipping converting the source IO object into a
+  file. This can be used to avoid a potentially expensive download/copy when
+  the derivatives processor doesn't need the file.
+
+  ```rb
+  Attacher.derivatives :my_processor, download: false do |source|
+    source #=> Could be File, Shrine::UploadedFile, or other IO-like object
+    shrine_class.with_file(source) do |file|
+      # can force download/copy if necessary with `with_file`,
+    end
+  end
+  ```
+
+## Bug fixes
+
+* The `upload_endpoint` now handles calling `Shrine.upload_response` method
+  from a Rails controller.
+
+* The `derivation_endpoint` plugin now applies the `version` query parameter
+  to the derivation when creating the response.
+
+## Other improvements
+
+* The new `Aws:S3::EncryptionV2::Client` is now supported by the S3 storage for
+  client-side encryption.
+
+* The `derivation_endpoint` now reduces the possibility of timing attacks by
+  comparing URL signatures in constant time using `Rack::Utils.secure_compare`.
+
+* The `derivatives` plugin now copies non-file source IO objects to disk before
+  passing them to the processor. This is consistent with how a
+  `Shrine::UploadedFile` object is downloaded to disk.
+
+* The `sequel` and `activerecord` plugins now call `Attacher#reload` when
+  reloading the model, which reloads the attached files but keeps other
+  attacher state.
+
+* The `derivatives` plugin doesn't download the attached file anymore if
+  attempting to process derivatives when no derivatives processor was defined.
+
+* The `mirroring` plugin now forwards attacher options when uploading to mirror
+  storages.
+
+* The `presign_endpoint` plugin now handles the `OPTIONS` HTTP verb, which
+  newer versions of Uppy are requesting.
+
+* `Shrine::Storage::Memory#open` now always returns a `StringIO` in the file
+  content's original encoding, instead of the encoding set by
+  `Encoding.default_internal`. This works around a [bug][ruby-lang #16497]
+  in `StringIO` introduced in Ruby 2.7.0.
+
+* The `remove_attachment` plugin now deletes the removed file if a new file was
+  attached right after removal.
+
+## Backwards compatibility
+
+* If you were passing a non-file IO object to the derivatives processor, Shrine
+  will now convert it into a file beforehand. If you're currently doing this
+  and are converting the IO object into a file inside the processor, you can
+  now remove the conversion code to avoid doubling the amount of disk writes.
+
+* When reloading a Sequel/ActiveRecord model, any attacher state other than
+  uploaded files will now be retained after the reload. If you were relying on
+  all the attacher state being re-initialized, you'll need to update your code.
+
+[ruby-lang #16497]: https://bugs.ruby-lang.org/issues/16497

data/doc/release_notes/3.4.0.md

@@ -0,0 +1,35 @@
+---
+title: Shrine 3.4.0
+---
+
+* Passing attacher options to `Shrine.Attachment` method now works on Ruby 3.0.
+
+* Defining validation errors as an array of I18n key and options in
+  `activerecord` plugin now works on Ruby 3.0.
+
+* The `:fastimage` MIME type analyzer now correctly detects SVGs as
+  `image/svg+html` in the `determine_mime_type` plugin.
+
+* The `Shrine::Attacher#read` method provided by the `entity` plugin is now
+  public. This is consistent with `Shrine::Attacher#write` from `model` plugin
+  being public as well.
+
+* The `Shrine::Attacher#reload` method now resets attachment's dirty state.
+  This means that for a model whose `Attacher#changed?` returns `true`, calling
+  `#reload` on the model will make `Attacher#changed?` return `false`. This was
+  the behaviour before Shrine 3.3.0.
+
+  ```rb
+  # before
+  model.file_attacher.changed? #=> true
+  model.reload
+  model.file_attacher.changed? #=> true
+
+  # after
+  model.file_attacher.changed? #=> true
+  model.reload
+  model.file_attacher.changed? #=> false
+  ```
+
+* Calling `#reload` on the model will not initialize a `Shrine::Attacher`
+  instance anymore if one hasn't previously been initialized.

data/doc/retrieving_uploads.md

@@ -1,4 +1,7 @@
-# Retrieving Uploads
+---
+id: retrieving-uploads
+title: Retrieving Uploads
+---
 
 Uploaded file content is typically retrieved from the storage using a
 `Shrine::UploadedFile` object. This guide explains the various methods of

data/doc/securing_uploads.md

@@ -1,9 +1,12 @@
-# Securing uploads
+---
+id: securing-uploads
+title: Securing Uploads
+---
 
 Shrine does a lot to make your file uploads secure, but there are still a lot
 of security measures that could be added by the user on the application's side.
-This guide will try to cover all the well-known security issues, ranging from
-the obvious ones to not-so-obvious ones, and try to provide solutions.
+This guide will try to cover some well-known security issues, ranging from the
+obvious ones to not-so-obvious ones, and try to provide solutions.
 
 ## Validate file type
 
@@ -12,17 +15,21 @@ idea to create a whitelist (or a blacklist) of extensions and MIME types.
 
 By default Shrine stores the MIME type derived from the extension, which means
 it's not guaranteed to hold the actual MIME type of the the file. However, you
-can load the `determine_mime_type` plugin which by default uses the [file]
-utility to determine the MIME type from magic file headers.
+can load the `determine_mime_type` plugin to determine MIME type from magic
+file headers.
 
+```rb
+# Gemfile
+gem "marcel", "~> 0.3"
+```
 ```rb
 class MyUploader < Shrine
+  plugin :determine_mime_type, analyzer: :marcel
   plugin :validation_helpers
-  plugin :determine_mime_type
 
   Attacher.validate do
-    validate_extension_inclusion %w[jpg jpeg png gif]
-    validate_mime_type_inclusion %w[image/jpeg image/png image/gif]
+    validate_extension %w[jpg jpeg png webp]
+    validate_mime_type %w[image/jpeg image/png image/webp]
   end
 end
 ```
@@ -31,22 +38,23 @@ end
 
 It's a good idea to generally limit the filesize of uploaded files, so that
 attackers cannot easily flood your storage. There are various layers at which
-you can apply filesize limits, depending on how you're accepting uploads.
-Firstly, you should probably add a filesize validation to prevent large files
-from being uploaded to `:store`:
+you can apply filesize limits, depending on how you're accepting uploads. For
+starters you can add a filesize validation to prevent large files from being
+uploaded to `:store`:
 
 ```rb
 class MyUploader < Shrine
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_max_size 20*1024*1024 # 20 MB
+    validate_max_size 100*1024*1024 # 100 MB
   end
 end
 ```
 
-In the following sections we talk about various strategies to prevent files from
-being uploaded to cache and the temporary directory.
+In the following sections we talk about various strategies to prevent files
+from being uploaded to Shrine's temporary storage and the system's temporary
+directory.
 
 ### Limiting filesize in direct uploads
 
@@ -55,7 +63,7 @@ in the `:max_size` option to reject files that are larger than the specified
 limit:
 
 ```rb
-plugin :upload_endpoint, max_size: 20*1024*1024 # 20 MB
+plugin :upload_endpoint, max_size: 100*1024*1024 # 100 MB
 ```
 
 If you're doing direct uploads to Amazon S3 using the `presign_endpoint`
@@ -63,7 +71,7 @@ plugin, you can pass in the `:content_length_range` presign option:
 
 ```rb
 plugin :presign_endpoint, presign_options: -> (request) do
-  { content_length_range: 0..20*1024*1024 }
+  { content_length_range: 0..100*1024*1024 }
 end
 ```
 
@@ -92,20 +100,17 @@ loading the `remove_invalid` plugin.
 plugin :remove_invalid
 ```
 
-### Paranoid filesize limiting
+### Failsafe filesize limiting
 
-If you want to make sure that no large files ever get to your storages, and
-you don't really care about the error message, you can use the `hooks` plugin
-and raise an error:
+If you want to make sure that no large files ever get to your storages, and you
+don't really care about the error message, you can override `Shrine#upload`:
 
 ```rb
-class MyUploader
-  plugin :hooks
+class MyUploader < Shrine
+  def upload(io, **options)
+    fail FileTooLarge if io.size >= 100*1024*1024
 
-  def before_upload(io, context)
-    if io.respond_to?(:read)
-      raise FileTooLarge if io.size >= 20*1024*1024
-    end
+    super
   end
 end
 ```
@@ -118,24 +123,43 @@ image processing, since processing them can take a lot of time and memory. This
 makes it trivial to DoS the application which doesn't have any protection
 against them.
 
-Shrine uses the [fastimage] gem for determining image dimensions which has
-built-in protection against image bombs (ImageMagick for example doesn't), but
-you still need to prevent those files from being attached and processed:
+So, in addition to validating filesize, we should also validate image
+dimensions:
 
 ```rb
-class MyUploader < Shrine
+# Gemfile
+gem "fastimage"
+```
+```rb
+class ImageUploader < Shrine
   plugin :store_dimensions
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_max_width  2500
-    validate_max_height 2500
+    validate_max_size 100*1024*1024
+
+    if validate_mime_type %w[image/jpeg image/png image/webp]
+      validate_max_dimensions [5000, 5000]
+    end
   end
 end
 ```
 
-If you're doing processing on caching, you can use the fastimage gem directly
-in a conditional.
+If you want to be extra safe, you can add a failsafe before performing
+processing:
+
+```rb
+class ImageUploader < Shrine
+  # ...
+  Attacher.derivatives do |original|
+    width, height = Shrine.dimensions(original)
+
+    fail ImageBombError if width > 5000 || height > 5000
+
+    # ...
+  end
+end
+```
 
 ## Prevent metadata tampering
 
@@ -153,7 +177,7 @@ app. To guard yourself from such attacks, you can load the
 cached files on assignment and override the received metadata.
 
 ```rb
-plugin :restore_cached_data
+Shrine.plugin :restore_cached_data
 ```
 
 ## Limit number of files
@@ -172,6 +196,7 @@ class MyUploader < Shrine
 
   Attacher.validate do
     validate_min_size 10*1024 # 10 KB
+    # ...
   end
 end
 ```
@@ -183,6 +208,4 @@ end
 * [AppSec: 8 Basic Rules to Implement Secure File Uploads](https://software-security.sans.org/blog/2009/12/28/8-basic-rules-to-implement-secure-file-uploads/)
 
 [image bombs]: https://www.bamsoftware.com/hacks/deflate.html
-[fastimage]: https://github.com/sdsykes/fastimage
-[file]: http://linux.die.net/man/1/file
 [rack-attack]: https://github.com/kickstarter/rack-attack

data/doc/storage/file_system.md

@@ -1,4 +1,7 @@
-# Shrine::Storage::FileSystem
+---
+id: file-system
+title: File System
+---
 
 The FileSystem storage handles uploads to the filesystem, and it is most
 commonly initialized with a "base" folder and a "prefix":
@@ -64,7 +67,12 @@ File.exist?(file.path) #=> false
 ```
 
 If you want to make this option default, you can use the
-[`upload_options`][upload_options] plugin.
+[`upload_options`][upload_options] plugin, provided that both `:cache` and
+`:store` storages are `FileSystem`):
+
+```rb
+plugin :upload_options, cache: { move: true }, store: { move: true }
+```
 
 ## Path
 
@@ -74,6 +82,15 @@ You can retrieve path to the file using `#path`:
 storage.path("image.jpg") #=> #<Pathname:public/image.jpg>
 ```
 
+## Deleting prefixed
+
+If you want to delete all files in some directory, you can use
+`FileSystem#delete_prefixed`:
+
+```rb
+storage.delete_prefixed("some_directory/") # deletes all files in "some_directory/"
+```
+
 ## Clearing cache
 
 If you're using FileSystem as cache, you will probably want to periodically
@@ -111,4 +128,4 @@ also means that deploying the app can cancel someone's uploading if you're
 using backgrounding. Also, by default you cannot generate URLs to files in the
 "tmp" directory, but you can with the `download_endpoint` plugin.
 
-[upload_options]: /doc/plugins/upload_options.md#readme
+[upload_options]: https://shrinerb.com/docs/plugins/upload_options

data/doc/storage/memory.md

@@ -0,0 +1,19 @@
+---
+title: Memory
+---
+
+The Memory storage stores uploaded files in memory, which is suitable for
+testing.
+
+```rb
+Shrine.storages[:store] = Shrine::Storage::Memory.new
+```
+
+By default, each storage instance uses a new Hash object for storing files,
+but you can pass your own:
+
+```rb
+my_store = Hash.new
+
+Shrine.storages[:store] = Shrine::Storage::Memory.new(my_store)
+```