shrine 2.19.3 → 3.6.0

data/doc/storage/file_system.md

@@ -1,4 +1,7 @@
-# Shrine::Storage::FileSystem
+---
+id: file-system
+title: File System
+---
 
 The FileSystem storage handles uploads to the filesystem, and it is most
 commonly initialized with a "base" folder and a "prefix":
@@ -64,7 +67,12 @@ File.exist?(file.path) #=> false
 ```
 
 If you want to make this option default, you can use the
-[`upload_options`][upload_options] plugin.
+[`upload_options`][upload_options] plugin, provided that both `:cache` and
+`:store` storages are `FileSystem`):
+
+```rb
+plugin :upload_options, cache: { move: true }, store: { move: true }
+```
 
 ## Path
 
@@ -74,6 +82,15 @@ You can retrieve path to the file using `#path`:
 storage.path("image.jpg") #=> #<Pathname:public/image.jpg>
 ```
 
+## Deleting prefixed
+
+If you want to delete all files in some directory, you can use
+`FileSystem#delete_prefixed`:
+
+```rb
+storage.delete_prefixed("some_directory/") # deletes all files in "some_directory/"
+```
+
 ## Clearing cache
 
 If you're using FileSystem as cache, you will probably want to periodically
@@ -111,4 +128,4 @@ also means that deploying the app can cancel someone's uploading if you're
 using backgrounding. Also, by default you cannot generate URLs to files in the
 "tmp" directory, but you can with the `download_endpoint` plugin.
 
-[upload_options]: /doc/plugins/upload_options.md#readme
+[upload_options]: https://shrinerb.com/docs/plugins/upload_options

data/doc/storage/memory.md

@@ -0,0 +1,19 @@
+---
+title: Memory
+---
+
+The Memory storage stores uploaded files in memory, which is suitable for
+testing.
+
+```rb
+Shrine.storages[:store] = Shrine::Storage::Memory.new
+```
+
+By default, each storage instance uses a new Hash object for storing files,
+but you can pass your own:
+
+```rb
+my_store = Hash.new
+
+Shrine.storages[:store] = Shrine::Storage::Memory.new(my_store)
+```

data/doc/storage/s3.md

@@ -1,29 +1,41 @@
-# Shrine::Storage::S3
+---
+title: AWS S3
+---
 
-The S3 storage handles uploads to Amazon S3 service, using the [aws-sdk-s3]
+The S3 storage handles uploads to [AWS S3] service (or any s3-compatible
+service such as [DigitalOcean Spaces] or [MinIO]). It requires the [aws-sdk-s3]
 gem:
 
 ```rb
+# Gemfile
 gem "aws-sdk-s3", "~> 1.14"
 ```
 
-It can be initialized by providing the bucket name and credentials:
+## Initialization
+
+The storage is initialized by providing your bucket name, region and
+credentials:
 
 ```rb
 require "shrine/storage/s3"
 
 s3 = Shrine::Storage::S3.new(
   bucket: "my-app", # required
+  region: "eu-west-1", # required
   access_key_id: "abc",
   secret_access_key: "xyz",
-  region: "eu-west-1",
 )
 ```
 
-The core features of this storage require the following AWS permissions:
-`s3:ListBucket`, `s3:PutObject`, `s3:GetObject`, and `s3:DeleteObject`. If you
-have additional upload options configured such as setting object ACLs, then
-additional permissions may be required.
+> The storage requires the following AWS S3 permissions:
+>
+>  * `s3:ListBucket` for the bucket resource
+>  * `s3:GetObject`, `s3:PutObject`, `s3:PutObjectAcl`, `s3:DeleteObject`,
+>    `s3:ListMultipartUploadParts` and `s3:AbortMultipartUpload` for the object
+>    resources
+
+> The `:access_key_id` and `:secret_access_key` options is just one form of
+> authentication, see the [AWS SDK docs][credentials] for more options.
 
 The storage exposes the underlying Aws objects:
 
@@ -43,14 +55,29 @@ s3.object("key") #=> #<Aws::S3::Object>
 
 By default, uploaded S3 objects will have private visibility, meaning they can
 only be accessed via signed expiring URLs generated using your private S3
-credentials. If you would like to generate public URLs, you can tell S3 storage
-to make uploads public:
+credentials.
 
 ```rb
-s3 = Shrine::Storage::S3.new(public: true, **s3_options)
+s3 = Shrine::Storage::S3.new(**s3_options)
+s3.upload(io, "key") # uploads with default "private" ACL
+s3.url("key")        # https://my-bucket.s3.amazonaws.com/key?X-Amz-Expires=900&X-Amz-Signature=b22d37c37d...
+```
 
+If you would like to generate public URLs, you can tell S3 storage to make
+uploads public:
+
+```rb
+s3 = Shrine::Storage::S3.new(public: true, **s3_options)
 s3.upload(io, "key") # uploads with "public-read" ACL
-s3.url("key")        # returns public (unsigned) object URL
+s3.url("key")        # https://my-bucket.s3.amazonaws.com/key
+```
+
+If you want to make only *some* uploads public, you can conditionally apply the
+`:acl` upload option and `:public` URL option:
+
+```rb
+Shrine.plugin :upload_options, store: -> (io, **) { { acl: "public-read" } }
+Shrine.plugin :url_options,    store: -> (io, **) { { public: true } }
 ```
 
 ## Prefix
@@ -78,13 +105,11 @@ You can also generate upload options per upload with the `upload_options`
 plugin
 
 ```rb
-class MyUploader < Shrine
-  plugin :upload_options, store: -> (io, context) do
-    if context[:version] == :thumb
-      { acl: "public-read" }
-    else
-      { acl: "private" }
-    end
+Shrine.plugin :upload_options, store: -> (io, derivative: nil, **) do
+  if derivative == :thumb
+    { acl: "public-read" }
+  else
+    { acl: "private" }
   end
 end
 ```
@@ -95,22 +120,18 @@ or when using the uploader directly
 uploader.upload(file, upload_options: { acl: "private" })
 ```
 
-Note that, unlike the `:upload_options` storage option, upload options given on
-the uploader level won't be forwarded for generating presigns, since presigns
-are generated using the storage directly.
+> Unlike the `:upload_options` storage option, upload options given on
+  the uploader level won't be forwarded for generating presigns, since presigns
+  are generated using the storage directly.
 
-## URL options
+## Copy options
 
-Other than [`:host`](#url-host) and [`:public`](#public-uploads) URL options,
-all additional options are forwarded to [`Aws::S3::Object#presigned_url`].
+If you wish to override options that are passed when copying objects from
+temporary to permanent storage, you can pass `:copy_options`:
 
 ```rb
-s3.url(
-  expires_in: 15,
-  response_content_disposition: ContentDisposition.attachment("my-filename"),
-  response_content_type: "foo/bar",
-  # ...
-)
+# Removes default :tagging_directive, which isn't supported by Cloudflare R2
+Shrine::Storage::S3.new(copy_options: {}, **s3_options)
 ```
 
 ## URL Host
@@ -131,12 +152,14 @@ s3.url("image.jpg", host: "https://your-s3-host.com/prefix/") # needs to end wit
 ```
 
 To have the `:host` option passed automatically for every URL, use the
-`default_url_options` plugin.
+`url_options` plugin:
 
 ```rb
-plugin :default_url_options, store: { host: "http://abc123.cloudfront.net" }
+plugin :url_options, store: { host: "http://abc123.cloudfront.net" }
 ```
 
+### Signer
+
 If you would like to [serve private content via CloudFront], you need to sign
 the object URLs with a special signer, such as [`Aws::CloudFront::UrlSigner`]
 provided by the `aws-sdk-cloudfront` gem. The S3 storage initializer accepts a
@@ -155,13 +178,27 @@ Shrine::Storage::S3.new(signer: signer.method(:signed_url))
 Shrine::Storage::S3.new(signer: -> (url, **options) { signer.signed_url(url, **options) })
 ```
 
+## URL options
+
+Other than `:host` and `:public` URL options, all additional `S3#url` options
+are forwarded to [`Aws::S3::Object#presigned_url`].
+
+```rb
+s3.url(
+  expires_in: 15,
+  response_content_disposition: ContentDisposition.attachment("my-filename"),
+  response_content_type: "foo/bar",
+  # ...
+)
+```
+
 ## Presigns
 
-The `#presign` method can be used for generating paramters for direct uploads
-to Amazon S3:
+The `S3#presign` method can be used for generating parameters for direct upload
+to S3:
 
 ```rb
-s3.presign("/path/to/file") #=>
+s3.presign("key") #=>
 # {
 #   url: "https://my-bucket.s3.amazonaws.com/...",
 #   fields: { ... },  # blank for PUT presigns
@@ -170,11 +207,25 @@ s3.presign("/path/to/file") #=>
 # }
 ```
 
-Additional presign options can be given in three places:
+By default, parameters for a POST upload is generated, but you can also
+generate PUT upload parameters:
+
+```rb
+s3.presign("key", method: :put)
+```
+
+Any additional options are forwarded to [`Aws::S3::Object#presigned_post`]
+(for POST uploads) and [`Aws::S3::Object#presigned_url`] (for PUT uploads).
 
-* in `Storage::S3#presign` by forwarding options
-* in `:upload_options` option on this storage
-* in `presign_endpoint` plugin through `:presign_options`
+```rb
+s3.presign("key", method: :put, content_disposition: "attachment; filename=my-file.txt") #=>
+# {
+#   url: "https://my-bucket.s3.amazonaws.com/...",
+#   fields: {},
+#   headers: { "Content-Disposition" => "attachment; filename=my-file.txt" },
+#   method :put,
+# }
+```
 
 ## Large files
 
@@ -182,33 +233,24 @@ The aws-sdk-s3 gem has the ability to automatically use multipart upload/copy
 for larger files, splitting the file into multiple chunks and uploading/copying
 them in parallel.
 
-By default any files that are uploaded will use the multipart upload if they're
-larger than 15MB, and any files that are copied will use the multipart copy if
-they're larger than 150MB, but you can change the thresholds via
-`:multipart_threshold`.
-
-```rb
-thresholds = { upload: 30*1024*1024, copy: 200*1024*1024 }
-Shrine::Storage::S3.new(multipart_threshold: thresholds, **s3_options)
-```
-
-If you want to change how many threads aws-sdk-s3 will use for multipart
-upload/copy, you can use the `upload_options` plugin to specify
-`:thread_count`.
+By default, multipart upload will be used for files larger than 15MB, and
+multipart copy for files larger than 100MB, but you can change the thresholds
+via `:multipart_threshold`:
 
 ```rb
-plugin :upload_options, store: -> (io, context) do
-  { thread_count: 5 }
-end
+Shrine::Storage::S3.new(
+  multipart_threshold: { upload: 30*1024*1024, copy: 200*1024*1024 },
+  **s3_options,
+)
 ```
 
 ## Encryption
 
-The easiest way to use server-side encryption for uploaded S3 objects is to
+The easiest way to use **server-side** encryption for uploaded S3 objects is to
 configure default encryption for your S3 bucket. Alternatively, you can pass
 server-side encryption parameters to the API calls.
 
-The `#upload` method accepts `:sse_*` options:
+The `S3#upload` method accepts `:sse_*` options:
 
 ```rb
 s3.upload(io, "key", sse_customer_algorithm: "AES256",
@@ -217,7 +259,7 @@ s3.upload(io, "key", sse_customer_algorithm: "AES256",
                      ssekms_key_id:          "key_id")
 ```
 
-The `#presign` method accepts `:server_side_encryption_*` options for POST
+The `S3#presign` method accepts `:server_side_encryption_*` options for POST
 presigns, and the same `:sse_*` options as above for PUT presigns.
 
 ```rb
@@ -230,24 +272,19 @@ When downloading encrypted S3 objects, the same server-side encryption
 parameters need to be passed in.
 
 ```rb
-s3.download("key", sse_customer_algorithm: "AES256",
-                   sse_customer_key:       "secret_key",
-                   sse_customer_key_md5:   "secret_key_md5")
-
 s3.open("key", sse_customer_algorithm: "AES256",
                sse_customer_key:       "secret_key",
                sse_customer_key_md5:   "secret_key_md5")
 ```
 
-If you want to use client-side encryption instead, you can instantiate the
-storage with an `Aws::S3::Encryption::Client` instance.
+**Client-side** encryption is supported as well:
 
 ```rb
-client = Aws::S3::Encryption::Client.new(
-  kms_key_id: "alias/my-key"
-)
+encryption_client = Aws::S3::EncryptionV2::Client.new(...)
+s3 = Shrine::Storage::S3.new(client: encryption_client, **other_options)
 
-Shrine::Storage::S3(client: client, bucket: "my-bucket")
+s3.upload(io, "key") # encrypts on upload
+s3.open("key")       # decrypts on download
 ```
 
 ## Accelerate endpoint
@@ -259,6 +296,15 @@ To use Amazon S3's [Transfer Acceleration] feature, set
 Shrine::Storage::S3.new(use_accelerate_endpoint: true, **other_options)
 ```
 
+## Deleting prefixed
+
+If you want to delete all objects in some prefix, you can use
+`S3#delete_prefixed`:
+
+```rb
+s3.delete_prefixed("some_prefix/") # deletes all objects in "some_prefix/"
+```
+
 ## Clearing cache
 
 If you're using S3 as a cache, you will probably want to periodically delete
@@ -272,20 +318,18 @@ Alternatively you can periodically call the `#clear!` method:
 s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
 ```
 
-## Request Rate and Performance Guidelines
-
-Amazon S3 automatically scales to high request rates. For example, your
-application can achieve at least 3,500 PUT/POST/DELETE and 5,500 GET requests
-per second per prefix in a bucket (a prefix is a top-level "directory" in the
-bucket). If your app needs to support higher request rates to S3 than that, you
-can scale exponentially by using more prefixes.
-
+[AWS S3]: https://aws.amazon.com/s3/
+[MinIO]: https://min.io/
+[DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
+[aws-sdk-s3]: https://rubygems.org/gems/aws-sdk-s3
 [uploading]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#put-instance_method
 [copying]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#copy_from-instance_method
 [presigning]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
 [`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method
-[aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
 [Transfer Acceleration]: http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html
 [object lifecycle]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
 [serve private content via CloudFront]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html
 [`Aws::CloudFront::UrlSigner`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/CloudFront/UrlSigner.html
+[credentials]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
+[`Aws::S3::Object#presigned_post`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
+[`Aws::S3::Object#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_url-instance_method