shrine 3.1.0 → 3.2.0

data/doc/refile.md

@@ -1,5 +1,5 @@
 ---
-title: Shrine for Refile Users
+title: Upgrading from Refile
 ---
 
 This guide is aimed at helping Refile users transition to Shrine, and it consists
@@ -110,13 +110,6 @@ into separate columns.
 Shrine provides on-the-fly processing via the
 [`derivation_endpoint`][derivation_endpoint] plugin:
 
-```rb
-# config/routes.rb (Rails)
-Rails.application.routes.draw do
-  # ...
-  mount ImageUploader.derivation_endpoint => "/derivations/image"
-end
-```
 ```rb
 require "image_processing/mini_magick"
 
@@ -132,8 +125,15 @@ class ImageUploader < Shrine
   end
 end
 ```
+```rb
+# config/routes.rb (Rails)
+Rails.application.routes.draw do
+  # ...
+  mount ImageUploader.derivation_endpoint => "/derivations/image"
+end
+```
 
-Shrine also support processing up front using the [`derivatives`][derivatives]
+Shrine also support eager processing using the [`derivatives`][derivatives]
 plugin.
 
 ### Validation

data/doc/release_notes/3.2.0.md

@@ -0,0 +1,96 @@
+---
+title: Shrine 3.2.0
+---
+
+## New features
+
+* The `type_predicates` plugin has been added, which adds convenient predicate
+  methods to `Shrine::UploadedFile` based on the MIME type.
+
+  ```rb
+  # Gemfile
+  gem "mini_mime" # default dependency of type_predicates
+  ```
+  ```rb
+  Shrine.plugin :type_predicates
+  ```
+
+  The plugin adds four predicate methods based on the general type of the file:
+
+  ```rb
+  file.image? # returns true for any "image/*" MIME type
+  file.video? # returns true for any "video/*" MIME type
+  file.audio? # returns true for any "audio/*" MIME type
+  file.text?  # returns true for any "text/*" MIME type
+  ```
+
+  You can also check for specific MIME type using the extension name:
+
+  ```rb
+  file.type?(:jpg) # returns true if MIME type is "image/jpeg"
+  file.type?(:svg) # returns true if MIME type is "image/svg+xml"
+  file.type?(:mov) # returns true if MIME type is "video/quicktime"
+  file.type?(:ppt) # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ...
+  ```
+
+  For convenience, you can create predicate methods for specific file types:
+
+  ```rb
+  Shrine.plugin :type_predicates, methods: %i[jpg svg mov ppt]
+  ```
+  ```rb
+  file.jpg? # returns true if MIME type is "image/jpeg"
+  file.svg? # returns true if MIME type is "image/svg+xml"
+  file.mov? # returns true if MIME type is "video/quicktime"
+  file.ppt? # returns true if MIME type is "application/vnd.ms-powerpoint"
+  ```
+
+* The `#add_metadata` method has been added to the `add_metadata` plugin for
+  adding new metadata to an existing file/attachment.
+
+  ```rb
+  attacher.file.metadata #=> { ... }
+  attacher.add_metadata("foo" => "bar")
+  attacher.file.metadata #=> { ..., "foo" => "bar" }
+  ```
+
+## Other improvements
+
+* The `remove_invalid` plugin now works correctly with `derivatives` plugin.
+
+* The `remove_invalid` plugin is now also activated when `Attacher#validate`
+  is called manually.
+
+* The current attached file data can now be assigned back to the attachment
+  attribute, and this operation will be a no-op.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:store metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "store", "metadata" => { ... } } # no-op
+  ```
+
+  This allows treating the attachment attribute as a persistent attribute,
+  where the current value can be assigned back on record updates.
+
+* When promoting derivatives, the `:derivative` parameter value was being
+  passed to the uploader as an array. This has been fixed, and the value is now
+  the same as when uploading derivatives directly to permanent storage.
+
+* The `derivatives` plugin now includes additional `:io` and `:attacher` values
+  in the instrumentation event payload.
+
+## Backwards compatibility
+
+* The `validation` plugin now runs validations on `Attacher#attach` and
+  `Attacher#attach_cached`. If you were using `Attacher#change` directly and
+  expecting the validations to be run automatically, you will need to update
+  your code.
+
+* If you were updating the cached file metadata via file data assignment, this
+  will no longer work.
+
+  ```rb
+  photo.image #=> #<Shrine::UploadedFile id="foo" storage=:cache metadata={...}>
+  photo.image = { "id" => "foo", "storage" => "cache", "metadata" => { ... } } # no-op
+  ```

data/doc/securing_uploads.md

@@ -1,6 +1,6 @@
 ---
 id: securing-uploads
-title: Securing uploads
+title: Securing Uploads
 ---
 
 Shrine does a lot to make your file uploads secure, but there are still a lot

data/doc/storage/memory.md

@@ -0,0 +1,19 @@
+---
+title: Memory
+---
+
+The Memory storage stores uploaded files in memory, which is suitable for
+testing.
+
+```rb
+Shrine.storages[:store] = Shrine::Storage::Memory.new
+```
+
+By default, each storage instance uses a new Hash object for storing files,
+but you can pass your own:
+
+```rb
+my_store = Hash.new
+
+Shrine.storages[:store] = Shrine::Storage::Memory.new(my_store)
+```

data/doc/storage/s3.md

@@ -2,30 +2,38 @@
 title: AWS S3
 ---
 
-The S3 storage handles uploads to Amazon S3 service, using the [aws-sdk-s3]
+The S3 storage handles uploads to [AWS S3] service (or any s3-compatible
+service such as [DigitalOcean Spaces] or [MinIO]). It requires the [aws-sdk-s3]
 gem:
 
 ```rb
+# Gemfile
 gem "aws-sdk-s3", "~> 1.14"
 ```
 
-It can be initialized by providing the bucket name and credentials:
+## Initialization
+
+The storage is initialized by providing your bucket name, region and
+credentials:
 
 ```rb
 require "shrine/storage/s3"
 
 s3 = Shrine::Storage::S3.new(
   bucket: "my-app", # required
+  region: "eu-west-1", # required
   access_key_id: "abc",
   secret_access_key: "xyz",
-  region: "eu-west-1",
 )
 ```
 
-The core features of this storage require the following AWS permissions:
-`s3:ListBucket`, `s3:PutObject`, `s3:GetObject`, and `s3:DeleteObject`. If you
-have additional upload options configured such as setting object ACLs, then
-additional permissions may be required.
+The `:access_key_id` and `:secret_access_key` options are just one form of
+authentication, see [`Aws::S3::Client#initialize`] docs for more details.
+
+> The core features of this storage require the following AWS permissions:
+  `s3:ListBucket`, `s3:PutObject`, `s3:GetObject`, and `s3:DeleteObject`. If you
+  have additional upload options configured such as setting object ACLs, then
+  additional permissions may be required.
 
 The storage exposes the underlying Aws objects:
 
@@ -45,14 +53,29 @@ s3.object("key") #=> #<Aws::S3::Object>
 
 By default, uploaded S3 objects will have private visibility, meaning they can
 only be accessed via signed expiring URLs generated using your private S3
-credentials. If you would like to generate public URLs, you can tell S3 storage
-to make uploads public:
+credentials.
 
 ```rb
-s3 = Shrine::Storage::S3.new(public: true, **s3_options)
+s3 = Shrine::Storage::S3.new(**s3_options)
+s3.upload(io, "key") # uploads with default "private" ACL
+s3.url("key")        # https://my-bucket.s3.amazonaws.com/key?X-Amz-Expires=900&X-Amz-Signature=b22d37c37d...
+```
 
+If you would like to generate public URLs, you can tell S3 storage to make
+uploads public:
+
+```rb
+s3 = Shrine::Storage::S3.new(public: true, **s3_options)
 s3.upload(io, "key") # uploads with "public-read" ACL
-s3.url("key")        # returns public (unsigned) object URL
+s3.url("key")        # https://my-bucket.s3.amazonaws.com/key
+```
+
+If you want to make only *some* uploads public, you can conditionally apply the
+`:acl` upload option and `:public` URL option:
+
+```rb
+Shrine.plugin :upload_options, store: -> (io, **) { { acl: "public-read" } }
+Shrine.plugin :url_options,    store: -> (io, **) { { public: true } }
 ```
 
 ## Prefix
@@ -80,13 +103,11 @@ You can also generate upload options per upload with the `upload_options`
 plugin
 
 ```rb
-class MyUploader < Shrine
-  plugin :upload_options, store: -> (io, derivative: nil, **) do
-    if derivative == :thumb
-      { acl: "public-read" }
-    else
-      { acl: "private" }
-    end
+Shrine.plugin :upload_options, store: -> (io, derivative: nil, **) do
+  if derivative == :thumb
+    { acl: "public-read" }
+  else
+    { acl: "private" }
   end
 end
 ```
@@ -97,23 +118,9 @@ or when using the uploader directly
 uploader.upload(file, upload_options: { acl: "private" })
 ```
 
-Note that, unlike the `:upload_options` storage option, upload options given on
-the uploader level won't be forwarded for generating presigns, since presigns
-are generated using the storage directly.
-
-## URL options
-
-Other than [`:host`](#url-host) and [`:public`](#public-uploads) URL options,
-all additional options are forwarded to [`Aws::S3::Object#presigned_url`].
-
-```rb
-s3.url(
-  expires_in: 15,
-  response_content_disposition: ContentDisposition.attachment("my-filename"),
-  response_content_type: "foo/bar",
-  # ...
-)
-```
+> Unlike the `:upload_options` storage option, upload options given on
+  the uploader level won't be forwarded for generating presigns, since presigns
+  are generated using the storage directly.
 
 ## URL Host
 
@@ -133,12 +140,14 @@ s3.url("image.jpg", host: "https://your-s3-host.com/prefix/") # needs to end wit
 ```
 
 To have the `:host` option passed automatically for every URL, use the
-`url_options` plugin.
+`url_options` plugin:
 
 ```rb
 plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
 ```
 
+### Signer
+
 If you would like to [serve private content via CloudFront], you need to sign
 the object URLs with a special signer, such as [`Aws::CloudFront::UrlSigner`]
 provided by the `aws-sdk-cloudfront` gem. The S3 storage initializer accepts a
@@ -157,13 +166,27 @@ Shrine::Storage::S3.new(signer: signer.method(:signed_url))
 Shrine::Storage::S3.new(signer: -> (url, **options) { signer.signed_url(url, **options) })
 ```
 
+## URL options
+
+Other than `:host` and `:public` URL options, all additional `S3#url` options
+are forwarded to [`Aws::S3::Object#presigned_url`].
+
+```rb
+s3.url(
+  expires_in: 15,
+  response_content_disposition: ContentDisposition.attachment("my-filename"),
+  response_content_type: "foo/bar",
+  # ...
+)
+```
+
 ## Presigns
 
-The `#presign` method can be used for generating paramters for direct uploads
-to Amazon S3:
+The `S3#presign` method can be used for generating parameters for direct upload
+to S3:
 
 ```rb
-s3.presign("/path/to/file") #=>
+s3.presign("key") #=>
 # {
 #   url: "https://my-bucket.s3.amazonaws.com/...",
 #   fields: { ... },  # blank for PUT presigns
@@ -172,11 +195,25 @@ s3.presign("/path/to/file") #=>
 # }
 ```
 
-Additional presign options can be given in three places:
+By default, parameters for a POST upload is generated, but you can also
+generate PUT upload parameters:
 
-* in `Storage::S3#presign` by forwarding options
-* in `:upload_options` option on this storage
-* in `presign_endpoint` plugin through `:presign_options`
+```rb
+s3.presign("key", method: :put)
+```
+
+Any additional options are forwarded to [`Aws::S3::Object#presigned_post`]
+(for POST uploads) and [`Aws::S3::Object#presigned_url`] (for PUT uploads).
+
+```rb
+s3.presign("key", method: :put, content_disposition: "attachment; filename=my-file.txt") #=>
+# {
+#   url: "https://my-bucket.s3.amazonaws.com/...",
+#   fields: {},
+#   headers: { "Content-Disposition" => "attachment; filename=my-file.txt" },
+#   method :put,
+# }
+```
 
 ## Large files
 
@@ -184,33 +221,24 @@ The aws-sdk-s3 gem has the ability to automatically use multipart upload/copy
 for larger files, splitting the file into multiple chunks and uploading/copying
 them in parallel.
 
-By default any files that are uploaded will use the multipart upload if they're
-larger than 15MB, and any files that are copied will use the multipart copy if
-they're larger than 150MB, but you can change the thresholds via
-`:multipart_threshold`.
+By default, multipart upload will be used for files larger than 15MB, and
+multipart copy for files larger than 100MB, but you can change the thresholds
+via `:multipart_threshold`:
 
 ```rb
-thresholds = { upload: 30*1024*1024, copy: 200*1024*1024 }
-Shrine::Storage::S3.new(multipart_threshold: thresholds, **s3_options)
-```
-
-If you want to change how many threads aws-sdk-s3 will use for multipart
-upload/copy, you can use the `upload_options` plugin to specify
-`:thread_count`.
-
-```rb
-plugin :upload_options, store: -> (io, context) do
-  { thread_count: 5 }
-end
+Shrine::Storage::S3.new(
+  multipart_threshold: { upload: 30*1024*1024, copy: 200*1024*1024 },
+  **s3_options,
+)
 ```
 
 ## Encryption
 
-The easiest way to use server-side encryption for uploaded S3 objects is to
+The easiest way to use **server-side** encryption for uploaded S3 objects is to
 configure default encryption for your S3 bucket. Alternatively, you can pass
 server-side encryption parameters to the API calls.
 
-The `#upload` method accepts `:sse_*` options:
+The `S3#upload` method accepts `:sse_*` options:
 
 ```rb
 s3.upload(io, "key", sse_customer_algorithm: "AES256",
@@ -219,7 +247,7 @@ s3.upload(io, "key", sse_customer_algorithm: "AES256",
                      ssekms_key_id:          "key_id")
 ```
 
-The `#presign` method accepts `:server_side_encryption_*` options for POST
+The `S3#presign` method accepts `:server_side_encryption_*` options for POST
 presigns, and the same `:sse_*` options as above for PUT presigns.
 
 ```rb
@@ -237,16 +265,9 @@ s3.open("key", sse_customer_algorithm: "AES256",
                sse_customer_key_md5:   "secret_key_md5")
 ```
 
-If you want to use client-side encryption instead, you can instantiate the
-storage with an `Aws::S3::Encryption::Client` instance.
-
-```rb
-client = Aws::S3::Encryption::Client.new(
-  kms_key_id: "alias/my-key"
-)
-
-Shrine::Storage::S3(client: client, bucket: "my-bucket")
-```
+If you want to use **client-side** encryption instead, note that it's still a
+work in progress, see issue [#348] for some discussion and
+[workarounds][client-side encryption workaround].
 
 ## Accelerate endpoint
 
@@ -279,20 +300,20 @@ Alternatively you can periodically call the `#clear!` method:
 s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
 ```
 
-## Request Rate and Performance Guidelines
-
-Amazon S3 automatically scales to high request rates. For example, your
-application can achieve at least 3,500 PUT/POST/DELETE and 5,500 GET requests
-per second per prefix in a bucket (a prefix is a top-level "directory" in the
-bucket). If your app needs to support higher request rates to S3 than that, you
-can scale exponentially by using more prefixes.
-
+[AWS S3]: https://aws.amazon.com/s3/
+[MinIO]: https://min.io/
+[DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
+[aws-sdk-s3]: https://rubygems.org/gems/aws-sdk-s3
 [uploading]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#put-instance_method
 [copying]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#copy_from-instance_method
 [presigning]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
 [`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
-[aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
 [Transfer Acceleration]: http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html
 [object lifecycle]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
 [serve private content via CloudFront]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html
 [`Aws::CloudFront::UrlSigner`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/CloudFront/UrlSigner.html
+[`Aws::S3::Client#initialize`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
+[`Aws::S3::Object#presigned_post`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
+[`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
+[#348]: https://github.com/shrinerb/shrine/issues/348
+[client-side encryption workaround]: https://github.com/shrinerb/shrine/issues/348#issuecomment-486445382