shrine 2.14.0 → 2.15.0

data/doc/release_notes/2.10.0.md

@@ -0,0 +1,52 @@
+## New features
+
+* The `:mini_magick` analyzer has been added to the `store_dimensions` plugin,
+  which uses the [MiniMagick] gem to extract image dimensions.
+
+* The `:ruby_vips` analyzer has been added to the `store_dimensions` plugin,
+  which uses the [ruby-vips] gem to extract image dimensions.
+
+* The `:fastimage` analyzer has been added to the `determine_mime_type` plugin,
+  which uses the [FastImage] gem to determine the MIME type of the file.
+
+* `Shrine::UploadedFile#download` now accepts a block for downloading an
+  uploaded file temporarily. This is useful when wanting to validate whether an
+  uploaded image is valid or corrupted, or when generating thumbnails.
+
+```rb
+uploaded_file.download do |tempfile|
+  # ...
+end # tempfile is deleted
+```
+
+## Other improvements
+
+* It's not required that IO objects respond to `#size` anymore. This is
+  useful when uploading streams of data where the size is not known.
+
+* The S3 storage now supports IO objects with unknown size. Under the hood it
+  will use multipart upload.
+
+* The logger is now properly inherited and shared between `Shrine` subclasses.
+
+* The attachment URL generated by `download_endpoint` now stays the same
+  regardless of the order of elements in the metadata hash. Previously it could
+  change after the uploaded file data is loaded from the database, because the
+  order of the metadata hash elements would change, which is not desirable for
+  caching.
+
+## Backwards compatibility
+
+* The `:rack_mime` extension inferrer has been removed from the
+  `infer_extension` plugin, due to not having acceptable behaviour. The new
+  default extension inferrer is `:mime_types`.
+
+* The `:heroku` formatter in the `logging` plugin has been soft-renamed to
+  `:logfmt`. The `:heroku` alias will stop being supported in Shrine 3.
+
+* The `Shrine::IO_METHODS` constant has been depreacted, and will become
+  private in Shrine 3.
+
+[MiniMagick]: https://github.com/minimagick/minimagick
+[ruby-vips]: https://github.com/libvips/ruby-vips
+[FastImage]: https://github.com/sdsykes/fastimage

data/doc/release_notes/2.10.1.md

@@ -0,0 +1,6 @@
+## Regressions
+
+* Determining MIME type from file extension (using `:mime_types` or
+  `:mini_mime` analyzers) when file is empty now works again. This was the
+  behaviour prior to version 2.7.0, but 2.7.0 introduced a regression where
+  `nil` was returned for empty files, even if they had an extension.

data/doc/release_notes/2.11.0.md

@@ -0,0 +1,69 @@
+## New features
+
+* `Shrine::UploadedFile#stream` has been added for streaming the uploaded
+  content to a writable destination.
+
+```rb
+uploaded_file.stream(StringIO.new)
+# or
+uploaded_file.stream("/path/to/destination")
+```
+
+* `Shrine.with_file` has been added for temporarily converting an IO-like
+  object into a file. This is useful when wanting to extract metadata using an
+  analyzer which requires the source file to be on disk.
+
+```rb
+add_metadata do |io, context|
+  movie = Shrine.with_file(io) { |file| FFMPEG::Movie.new(file.path) }
+
+  { "duration"   => movie.duration,
+    "bitrate"    => movie.bitrate,
+    "resolution" => movie.resolution,
+    "frame_rate" => movie.frame_rate }
+end
+```
+
+* The `upload_endpoint` plugin now accepts the `Content-MD5` request header,
+  in which case it will verify the provided checksum.
+
+* `Shrine::Storage::S3#presign` now accepts `method: :put` for changing from a
+  POST to a PUT presigned URL. PUT presigned URLs are generally preferred as
+  they support more parameters, such as `:content_md5` for specifying the
+  checksum.
+
+## Other improvements
+
+* `Shrine::UploadedFile#download` will now reuse an already opened file, and
+  in this case will simply rewind it after it's finished.
+
+* The `:mini_magick` and `:ruby_vips` dimensions analyzers now silently fail
+  on processing errors, to allow validations to be reached when invalid file
+  is attached.
+
+* The `#presign` storage method can now return a Hash. This means it's not
+  required for result to be wrapped in a `Struct` or `OpenStruct` anymore.
+
+* The `Shrine::Storage::S3#presign` now also returns a `:method` value
+  indicating the HTTP verb that needs to be used for the direct upload.
+
+* The bucket name is not removed from S3 URL path anymore when both `:host`
+  and `:force_path_style` are set in `Shrine::Storage::S3#url`.
+
+## Regressions
+
+* The MIME type is now correctly determined on empty files for `:mime_types`
+  and `:mini_mime` analyzers. This regression was introduced in Shrine 2.7.0.
+
+## Backwards compatibility
+
+* The `direct_upload` plugin has been deprecated in favour of `upload_endpoint`
+  and `presign_endpoint` plugins. The `direct_upload` plugin will be removed in
+  Shrine 3.
+
+* `Storage#presign` returning an object that doesn't respond to `#to_h` is now
+  deprecated, the support for these objects will be removed in Shrine 3.
+
+* `Shrine::Storage::S3#presign` now returns a `Struct` instead of an
+  `Aws::S3::PresignedPost` object. Any applications relying on any methods
+  other than `#url` and `#fields` will have to update their code.

data/doc/release_notes/2.12.0.md

@@ -0,0 +1,65 @@
+## New features
+
+* The `Shrine::Attacher#assign_remote_url` method has been added, which acts the
+  same as `#remote_url=` but allows you to also dynamically provide options to
+  the underlying downloader (`Down.download` by default).
+
+```rb
+attacher.assign_remote_url("https://example.com/image.jpg", downloader: { 'Cookie' => 'abc123' })
+```
+
+* The `Shrine::UploadedFile#download_url` method has been added to the
+  `download_endpoint` plugin for explicitly generating the file URL through
+  the download endpoint.
+
+* The `:redirect` option has been added to `download_endpoint` plugin, which
+  makes download URLs redirect directly to the uploaded file on its storage.
+  This allows you to avoding streaming the file through your app, and thus not
+  having it impact your request throughput.
+
+* The `Shrine::Plugins::ValidationHelpers::PRETTY_FILESIZE` callbable has been
+  added, which takes the filesize in bytes and returns the representation in
+  correct unit. This can be used when providing custom validation error messages.
+
+```rb
+pretty_filesize = Shrine::Plugins::ValidationHelpers::PRETTY_FILESIZE
+pretty_filesize.call(1*1024)           #=> "1.0 KB"
+pretty_filesize.call(1.5*1024*1024)    #=> "1.5 MB"
+pretty_filesize.call(2*1024*1024*1024) #=> "2.0 GB"
+```
+
+## Other improvements
+
+* The `Shrine::Storage::FileSystem#open` now forwards any additional options to
+  `File.open`.
+
+* The `Shrine::Storage::S3#open` method now accepts the `:rewindable` option to
+  disable caching read content to disk.
+
+* In validation error messages in filesize validators in the
+  `validation_helpers` plugin show specified limits in the appropriate unit,
+  instead of always using megabytes.
+
+* `Shrine::UploadedFile#open` will now always open a new IO object, and close
+  the previous one. This way the uploaded file will be reopened if it's closed.
+
+* Fixed possible encoding issues in `Shrine::Storage::S3#upload` when filesize
+  is unknown.
+
+* The registered storage resolvers in the `dynamic_storage` plugin are now
+  correctly inherited upon subclassing.
+
+* The `:file` MIME type analyzer from `determine_mime_type` plugin will now
+  raise an explicit `Shrine::Error` when the subprocess from the shell command
+  failed to be spawned.
+
+* `Attacher#data_uri=` and `Attacher#remote_url=` in addition to `""` now also
+  ignore `nil` values, instead of raising an exception.
+
+## Backwards compatibility
+
+* The `:storages` option has been deprecated in the `download_endpoint` plugin
+  in favour of the new `Shrine::UploadedFile#download_url` method.
+
+* The deprecation of assigning cached versions has been undone, as it can be a
+  useful feature.

data/doc/release_notes/2.13.0.md

@@ -0,0 +1,146 @@
+## New features
+
+* The S3 object URLs can now be signed with a custom signer. This enables
+  serving private objects via AWS CloudFront by signing the URLs with the
+  signer from the `aws-sdk-cloudfront` gem.
+
+```rb
+require "aws-sdk-cloudfront"
+
+signer = Aws::CloudFront::UrlSigner.new(
+  key_pair_id:      "cf-keypair-id",
+  private_key_path: "./cf_private_key.pem",
+)
+
+Shrine::Storage::S3.new(signer: signer.method(:signed_url))
+```
+
+* To make your S3 storage public, previously you needed to do two things: set
+  the `acl: "public-read"` upload option and set pass `public: true` when
+  generating a URL.
+
+```rb
+Shrine.storages = {
+  cache: Shrine::Storage::S3.new(upload_options: { acl: "public-read" }, **options),
+  store: Shrine::Storage::S3.new(upload_options: { acl: "public-read" }, **options),
+}
+
+Shrine.plugin :default_url_options,
+  cache: { public: true },
+  store: { public: true }
+```
+
+  Now you can achieve the same thing just by setting `public: true` when
+  initializing the S3 storage.
+
+```rb
+Shrine.storages = {
+  cache: Shrine::Storage::S3.new(public: true, **options),
+  store: Shrine::Storage::S3.new(public: true, **options),
+}
+```
+
+* The `:force` option has been added to the `infer_extension` plugin, which
+  makes the extension always determined from the MIME type, regardless of
+  whether it exists or not.
+
+```rb
+Shrine.plugin :infer_extension, force: true
+```
+
+  This is useful for when you want to normalize the file extensions of uploaded
+  files.
+
+```rb
+uploader.upload(File.open("image.jpg"))  #
+uploader.upload(File.open("image.jpeg")) # all these will be uploaded with a .jpeg extension
+uploader.upload(File.open("image.JPG"))  #
+```
+
+* `Shrine#upload` now accepts a `:metadata` option for manually overrding the
+  extracted metadata.
+
+```rb
+uploaded_file = uploader.upload(file, metadata: { "filename" => "my-file.txt" })
+uploaded_file.original_filename    #=> "my-file.txt"
+```
+
+  Furthermore, `Shrine::Attacher#assign` now forwards any additional options to
+  `Shrine#upload`, so you can also override metadata when attaching files.
+
+```rb
+photo.image_attacher.assign(file, metadata: { "mime_type" => "text/plain" })
+```
+
+## Other improvements
+
+* It's now possible to use an S3 endpoint which requires bucket name to be in
+  the URI path (e.g. Minio) with CDNs where bucket name shouldn't be in the URI
+  path (e.g. CloudFront). Since version 2.11.0, when initializing
+  `Shrine::Storage::S3` with `:endpoint` with `:force_path_style`, generating
+  an URL with a `:host` returned an URI with bucket name in the path. This
+  introduced a regression for anyone relying on previous behaviour, so that
+  change has been reverted, and this is the current behaviour:
+
+```rb
+s3 = Shrine::Storage::S3.new(endpoint: "https://minio.example.com")
+s3.url("foo")                                     #=> "https://my-s3-endpoint.com/foo"
+s3.url("foo", host: "https://123.cloudfront.net") #=> "https://123.cloudfront.net/foo"
+
+s3 = Shrine::Storage::S3.new(endpoint: "https://my-s3-endpoint.com", force_path_style: true, **options)
+s3.url("foo")                                     #=> "https://my-s3-endpoint.com/my-bucket/foo"
+s3.url("foo", host: "https://123.cloudfront.net") #=> "https://123.cloudfront.net/foo"
+```
+
+* The `:host` option to `Shrine::Storage::S3#url` now handles URLs with
+  path prefixes, provided that the URL ends with a slash.
+
+```rb
+# old behaviour
+s3.url("foo", host: "https://example.com/prefix/") #=> "https://example.com/foo"
+# new behaviour
+s3.url("foo", host: "https://example.com/prefix/") #=> "https://example.com/prefix/foo"
+```
+
+* Fixed error that would happen when uploading a file with a filename that had
+  certain combination of UTF-8 characters to `upload_endpoint`.
+
+* The `Content-Type` header in `upload_endpoint` and `presign_endpoint`
+  responses now specifies `charset=utf-8`.
+
+* `Shrine::Storage::S3` now uses `Aws::S3::Object#upload_stream` if available
+  when uploading large IO streams which are not file objects, which uses
+  parallelized multipart upload. This can make such uploads finish up to 2x
+  faster.
+
+* `Shrine::Storage::S3` now uses `Aws::S3::Object#upload_stream` if available
+  when uploading files of unknown size.
+
+* The `file` command could sometimes exit successfully, but return a `cannot
+  open: No such file or directory` on stdout. This is now detected and a
+  `Shrine::Error` is raised.
+
+* The `upload_endpoint` now returns `Upload Not Valid` error message when file
+  parameter was present but not in correct format (previously `Upload Not
+  Found` was returned, which was a bit misleading).
+
+## Backwards compatibility
+
+* When `Shrine::Storage::S3` is initialized with `:endpoint` with
+  `:force_path_style`, a file URL generated with a `:host` will not include the
+  bucket name in the URL path anymore. Users relying on this behaviour should
+  update their code to include the bucket name in the `:host` URL path.
+
+```rb
+s3.url("foo", host: "https://example.com/my-bucket/")
+```
+
+* Using aws-sdk-s3 older than 1.14 with `Shrine::Storage::S3` when uploading
+  files with unknown size is now deprecated and won't be supported in Shrine 3.
+
+  Also, in this case `Shrine::Storage::S3` will now first copy the whole file
+  onto disk before uploading it (previously only a chunk of the input file was
+  copied to disk at a time).
+
+  If you're uploading files of unknown size (ones where `#size` is not defined
+  or returns `nil`), you should upgrade to aws-sdk-s3 1.14 or higher.

data/doc/release_notes/2.14.0.md

@@ -0,0 +1,278 @@
+## New features
+
+* `Shrine::Storage::S3` now accepts a `:client` option for specifying an AWS
+  SDK client object. This allows the user to configure client-side encryption by
+  passing a `Aws::S3::Encryption::Client` object:
+
+  ```rb
+  client = Aws::S3::Encryption::Client.new(
+    kms_key_id: "alias/my-key",
+    **options
+  )
+
+  Shrine::Storage::S3.new(client: client, bucket: "my-bucket")
+  ```
+
+* The metadata blocks in the `add_metadata` plugin now have previous metadata
+  available to them in `context[:metadata]` (thanks to @jrochkind).
+
+  ```rb
+  add_metadata :foo do |io, context|
+    context[:metadata] #=>
+    # {
+    #   "filename"  => "nature.jpg",
+    #   "size"      => 123,
+    #   "mime_type" => "image/jpeg"
+    # }
+
+    "foo"
+  end
+
+  add_metadata :bar do |io, context|
+    context[:metadata] #=>
+    # {
+    #   "filename"  => "nature.jpg",
+    #   "size"      => 123,
+    #   "mime_type" => "image/jpeg",
+    #   "foo"       => "foo"
+    # }
+
+    "bar"
+  end
+  ```
+
+* `UploadedFile#to_rack_response` from `rack_response` plugin now accepts
+  `:type` and `:filename` options for overriding the values in metadata that
+  would otherwise be used for the `Content-Type` and `Content-Disposition`
+  response headers.
+
+  ```rb
+  status, headers, body = uploaded_file.to_rack_response(
+    type:     "text/csv; charset=utf-8",
+    filename: "export.csv",
+  )
+
+  headers["Content-Type"]        #=> "text/csv; charset=utf-8"
+  headers["Content-Disposition"] #=> "inline; filename=\"export.csv\""
+  ```
+
+* `Shrine.data_uri` from `data_uri` plugin now accepts a `:filename` string,
+  which will be returned in
+  `Shrine::Plugins::DataUri::DataFile#original_filename`.
+
+  ```rb
+  io = Shrine.data_uri("data:,content", filename: "foo.txt")
+  io.original_filename #=> "foo.txt"
+  ```
+
+* `UploadedFile#download_url` from `download_endpoint` plugin now accepts a
+  `:host` option. Previously it was only possible to configure the URL host on
+  the plugin level.
+
+  ```rb
+  uploaded_file.download_url(host: "https://example.com")
+  ```
+
+* `Attacher#cached?` and `Attacher#stored?` now accept an optional uploaded
+  file parameter, and return whether the given uploaded file is uploaded to the
+  temporary or permanent storage of the attacher (thanks to @jrochkind).
+
+## Performance improvements
+
+* `UploadedFile#download` doesn't call `Storage#download` anymore (only
+  `Storage#open`). This improves performance when the uploaded file is
+  already opened, e.g. when using `refresh_metadata` plugin and
+  `Shrine.with_file` during metadata extraction. Instead of performing
+  a new download request, `UploadedFile#download` will in this case reuse the
+  already opened IO object.
+
+* Added `tempfile` plugin that makes it easier to avoid copying the
+  `UploadedFile` to disk multiple times during promotion. For example, in the
+  following code the uploaded file will be downloaded to disk only once
+  instead of three times.
+
+  ```rb
+  Shrine.plugin :tempfile
+  ```
+  ```rb
+  class MyUploader < Shrine
+    plugin :processing
+    plugin :add_metadata
+    plugin :refresh_metadata
+
+    process(:store) do |io, context|
+      io.open do
+        io.refresh_metadata!
+
+        processor = ImageProcessing::Vips.source(io.tempfile)
+        # ... processing ...
+      end
+    end
+
+    add_metadata :foo do |io, context|
+      Shrine.with_file(io) { "..." } unless context[:action] == :store
+    end
+
+    add_metadata :bar do |io, context|
+      Shrine.with_file(io) { "..." } unless context[:action] == :store
+    end
+  end
+  ```
+
+* `UploadedFile#refresh_metadata!` from `refresh_metadata` plugin now detects
+  when the uploaded file is already opened and in that case doesn't re-open the
+  file. This makes code like this faster:
+
+  ```rb
+  uploaded_file.open do
+    uploaded_file.refresh_metadata! # doesn't re-open the file anymore
+  end
+  ```
+
+* The `rack_response` plugin now integrates with `Rack::Sendfile` middleware
+  when `Shrine::Storage::FileSystem` is used. It does so by making the response
+  body respond to `#to_path`.
+
+* `#<name>_attacher` doesn't look up the attachment class every time it's
+  called with a new model instance anymore, making it slightly faster (thanks
+  to @printercu).
+
+## Other improvements
+
+* The `lib/shrine.rb` file has been split into `lib/shrine.rb`,
+  `lib/shrine/uploaded_file.rb`, `lib/shrine/attacher.rb`,
+  `lib/shrine/attachment.rb`, and `lib/shrine/plugins.rb` (thanks to
+  @printercu).
+
+* `Shrine::Storage::S3` and `rack_response` plugin now use the
+  [content_disposition] gem for generating correctly formatted
+  `Content-Disposition` header values.
+
+* The `S3#download` and `S3#open` methods now work correctly with server-side
+  encryption (as long as the necessary `:sse_*` parameters are passed in).
+
+* Fixed `FileSystem#clear!` not working with symlinks (at least on MRI).
+
+* Fixed `add_metadata` plugin overriding previously defined metadata
+  blocks when loaded again.
+
+* Fixed `processing` plugin overriding previously defined processing
+  blocks when loaded again.
+
+* Fixed `backgrounding` plugin erroring when temorary/permanent storage is
+  declared in `Shrine::Attachment.new` and there are no `:cache` or `:store`
+  storages defined.
+
+* Fixed `UploadedFile#extension` (and thus the tempfile in
+  `UploadedFile#download`) including URL query parameters in the file extension
+  when `Shrine::Storage::Url` is used (`shrine-url` gem) with URLs that have
+  query parameters (thanks to @jrochkind).
+
+* Fixed `backgrounding` plugin aborting promotion when cached file metadata was
+  refreshed in the processing block and Active Record JSON column was used.
+    - `refresh_metadata` plugin now avoids mutating the uploaded file data hash
+    - `backgrounding` plugin now continues promotion if cached file metadata has changed
+
+* The `Shrine.data_uri` method from the `data_uri` plugin now picks up media
+  type parameters from the data URI and includes them in
+  `Shrine::Plugins::DataUri::DataFile#content_type`.
+
+  ```rb
+  io = Shrine.data_uri("data:text/plain;charset=utf-8,content")
+  io.content_type #=>
+  # BEFORE: "text/plain"
+  # AFTER:  "text/plain;charset=utf-8"
+  ```
+
+* When fetching `mime_type` metadata value from `io.content_type`, any media
+  type parameters are now stripped. This means that if the user uploads a text
+  file with a `Content-Type` of `text/plain; charset=utf-8`, the `mime_type`
+  will now correctly be stored as `text/plain`. This fixes any MIME type
+  validations that might not have been passing due to additional media type
+  parameters.
+
+* When `determine_mime_type` plugin is loaded with the `:default` analyzer,
+  a warning is not printed anymore.
+
+* When `Shrine::Storage::S3` is initialized with `bucket: nil`, an appropriate
+  error message is now raised.
+
+* The `:content_type` MIME type analyzer has been added to the
+  `Shrine.mime_type_analyzers` hash in `determine_mime_type` plugin (previously
+  known as `:default`).
+
+## Documentation
+
+* The [Retrieving Uploads] guide has been added.
+
+## Backwards compatibility
+
+* Support for MRI 2.1 and MRI 2.2 has been dropped.
+
+* `Shrine::Plugins::Base` module has been removed and contained modules have
+  been moved to `InstanceMethods` and `ClassMethods` modules inside the
+  corresponding core classes. This should not break anything unless you were
+  referencing those modules directly.
+
+  ```rb
+  Shrine::Plugins::Base::InstanceMethods        => Shrine::InstanceMethods
+  Shrine::Plugins::Base::ClassMethods           => Shrine::ClassMethods
+  Shrine::Plugins::Base::FileMethods            => Shrine::UploadedFile::InstanceMethods
+  Shrine::Plugins::Base::FileClassMethods       => Shrine::UploadedFile::ClassMethods
+  Shrine::Plugins::Base::AttacherMethods        => Shrine::Attacher::InstanceMethods
+  Shrine::Plugins::Base::AttacherClassMethods   => Shrine::Attacher::ClassMethods
+  Shrine::Plugins::Base::AttachmentMethods      => Shrine::Attachment::InstanceMethods
+  Shrine::Plugins::Base::AttachmentClassMethods => Shrine::Attachment::ClassMethods
+  ```
+
+* `UploadedFile#download` doesn't call `Storage#download` anymore, it now
+  always calls `Storage#open`. Since `Storage#download` is not used anywhere
+  else in Shrine, storages can remove the `#download` method.
+
+* `Shrine::Storage::S3#download` has been deprecated and will be removed in
+  Shrine 3.
+
+* In `Shrine::Storage::S3#upload`, `#open`, and `#presign` it's now deprecated
+  to pass the `Content-Disposition` header value with non-ASCII characters.
+  Starting with Shrine 3 these characters won't be escaped anymore. You should
+  now use the [content_disposition] gem to properly format the
+  `Content-Disposition` header value.
+
+  ```rb
+  # BAD:
+  plugin :default_url_options, store: -> (io, **options) do
+    { response_content_disposition: "attachment; filename=\"#{io.original_filename}\"" }
+  end
+
+  # GOOD:
+  plugin :default_url_options, store: -> (io, **options) do
+    { response_content_disposition: ContentDisposition.attachment(io.original_filename) }
+  end
+  ```
+
+* The `mime_type` metadata value will now strip any additional media type
+  parameters such as `charset` from `io.content_type`. If you were relying on
+  this behaviour, you will need to update your code.
+
+* In `determine_mime_type` plugin, the `:default` MIME type analyzer has been
+  renamed to `:content_type`. The `:default` alias will stop being supported in
+  Shrine 3.
+
+  ```rb
+  plugin :determine_mime_type, analyzer: :default
+  # should be changed to
+  plugin :determine_mime_type, analyzer: :content_type
+  ```
+
+* The `UploadedFile` private methods that were added by the `rack_response`
+  plugin have now been moved to an internal class (leaving only
+  `#to_rack_response`). If you've are currently overriding any of these private
+  methods, you'll need to update your code.
+
+* The `UploadedFile` private methods that were added by the `download_endpoint`
+  plugin have now been moved to an internal class (leaving only
+  `#download_url`). If you've are currently overriding any of these private
+  methods, you'll need to update your code.
+
+[Retrieving Uploads]: /doc/retrieving_uploads.md#readme
+[content_disposition]: https://github.com/shrinerb/content_disposition