shrine 2.19.4 → 3.4.0

data/doc/creating_plugins.md

@@ -1,6 +1,13 @@
-# Creating a New Plugin
+---
+id: creating-plugins
+title: Writing a Plugin
+---
+
+Shrine has a lot of plugins built-in, but you can use Shrine's plugin system to
+create your own.
+
+## Definition
 
-Shrine has a lot of plugins built-in, but you can also easily create your own.
 Simply put, a plugin is a module:
 
 ```rb
@@ -11,9 +18,9 @@ end
 Shrine.plugin MyPlugin
 ```
 
-If you would like to load plugins with a symbol, like you already load plugins
-that ship with Shrine, you need to put the plugin in
-`shrine/plugins/my_plugin.rb` in the load path, and register it:
+If you would like to load plugins with a symbol (like you already do with
+plugins that ship with Shrine), you need to put the plugin in
+`shrine/plugins/my_plugin.rb` in your load path and register it:
 
 ```rb
 # shrine/plugins/my_plugin.rb
@@ -31,6 +38,8 @@ end
 Shrine.plugin :my_plugin
 ```
 
+## Methods
+
 The way to make plugins actually extend Shrine's core classes is by defining
 special modules inside the plugin. Here's a list of all "special" modules:
 
@@ -51,7 +60,7 @@ uploading:
 ```rb
 module MyPlugin
   module InstanceMethods
-    def upload(io, context)
+    def upload(io, **options)
       time = Time.now
       result = super
       duration = Time.now - time
@@ -61,40 +70,51 @@ module MyPlugin
 end
 ```
 
-Notice that we can call `super` to get the original behaviour. In addition to
-these modules, you can also make your plugin configurable:
+Notice that we can call `super` to get the original behaviour.
 
-```rb
-Shrine.plugin :my_plugin, foo: "bar"
-```
+## Configuration
 
-You can do this by adding a `.configure` method to your plugin, which will be
-given any passed in arguments or blocks. Typically you'll want to save these
-options into Shrine's `opts`, so that you can access them inside of Shrine's
-methods.
+You'll likely want to make your plugin configurable. You can do that by
+overriding the `.configure` class method and storing received options into
+`Shrine.opts`:
 
 ```rb
 module MyPlugin
-  def self.configure(uploader, options = {})
-    uploader # The uploader class which called `.plugin`
-    uploader.opts[:my_plugin_options] = options
+  def self.configure(uploader, **opts)
+    uploader.opts[:my_plugin] ||= {}
+    uploader.opts[:my_plugin].merge!(opts)
   end
 
   module InstanceMethods
-    def foo
-      opts[:my_plugin_options] #=> {foo: "bar"}
+    def upload(io, **options)
+      opts[:my_plugin] #=> { ... }
+      # ...
     end
   end
 end
 ```
 
+Users can now pass these configuration options when loading your plugin:
+
+```rb
+Shrine.plugin :my_plugin, foo: "bar"
+```
+
+## Dependencies
+
 If your plugin depends on other plugins, you can load them inside of
-`.load_dependencies` (which is given the same arguments as `.configure`):
+`.load_dependencies`:
 
 ```rb
 module MyPlugin
-  def self.load_dependencies(uploader, *)
-    uploader.plugin :versions # depends on the versions plugin
+  def self.load_dependencies(uploader, **opts)
+    uploader.plugin :derivatives # depends on the derivatives plugin
   end
 end
 ```
+
+The dependencies will get loaded before your plugin, allowing you to override
+methods of your dependencies in your method modules.
+
+The same configuration options passed to `.configure` are passed to
+`.load_dependencies` as well.

data/doc/creating_storages.md

@@ -1,4 +1,7 @@
-# Creating a New Storage
+---
+id: creating-storages
+title: Writing a Storage
+---
 
 Shrine ships with the FileSystem and S3 storages, but it's also easy to create
 your own. A storage is a class which needs to implement `#upload`, `#url`,
@@ -122,6 +125,8 @@ The storage can support additional options to customize how the file will be
 opened, `Shrine::UploadedFile#open` and `Shrine::UploadedFile#download` will
 forward any given options to `#open`.
 
+When file is not found, `Shrine::FileNotFound` exception should be raised.
+
 ## Url
 
 The `#url` storage method is called by `Shrine::UploadedFile#url`, it accepts a
@@ -204,15 +209,24 @@ The storage can support additional options to customize how the presign will be
 generated, those can be forwarded via the `:presign_options` option on the
 `presign_endpoint` plugin.
 
-## Clear
+## Delete Prefixed and Clear
+
+There are two methods that are not currently used by shrine, but which it's good
+for storages to provide to allow client code to delete files from storage. If
+storages provide these conventional methods, then clients can delete files using
+consistent API for any storage.
 
-While this method is not used by Shrine, it is good to give users the
-possibility to delete all files in a storage, and the conventional name for
-this method is `#clear!`.
+`#clear!` deletes all files from storage, and `#delete_prefixed` will delete all
+files in a given directory/prefix/path. While not strictly required for shrine storage
+service functionality, storages should usually implement if possible.
 
 ```rb
 class MyStorage
   # ...
+  def delete_prefixed(prefix_path)
+    # deletes all files under the supplied argument prefix
+  end
+
   def clear!
     # deletes all files in the storage
   end

data/doc/design.md

@@ -1,21 +1,26 @@
-# The Design of Shrine
+---
+title: The Design of Shrine
+---
 
-*If you want an in-depth walkthrough through the Shrine codebase, see [Notes on study of shrine implementation] article by Jonathan Rochkind.*
+*If you want an in-depth walkthrough through the Shrine codebase, see [Notes on
+study of shrine implementation] article by Jonathan Rochkind.*
 
-There are five main types of objects that you deal with in Shrine:
+There are five main types of classes that you deal with in Shrine:
 
-* Storage
-* `Shrine`
-* `Shrine::UploadedFile`
-* `Shrine::Attacher`
-* `Shrine::Attachment`
+| Class | Description |
+| :---- | :---------- |
+| [`Shrine::Storage::*`](#Storage) | Manages files on a particular storage service |
+| [`Shrine`](#Shrine) | Wraps uploads and handles loading plugins |
+| [`Shrine::UploadedFile`](#shrineuploadedfile) | Represents a file uploaded to a storage |
+| [`Shrine::Attacher`](#shrineattacher) | Handles file attachment logic |
+| [`Shrine::Attachment`](#shrineattachment) | Provides convenience model attachment interface |
 
 ## Storage
 
 On the lowest level we have a storage. A storage class encapsulates file
 management logic on a particular service. It is what actually performs uploads,
 generation of URLs, deletions and similar. By convention it is namespaced under
-`Shrine::Storage`.
+`Shrine::Storage::*`.
 
 ```rb
 filesystem = Shrine::Storage::FileSystem.new("uploads")
@@ -24,7 +29,7 @@ filesystem.url("foo") #=> "uploads/foo"
 filesystem.delete("foo")
 ```
 
-A storage is a PORO which responds to certain methods:
+A storage is a PORO which implements the following interface:
 
 ```rb
 class Shrine
@@ -34,7 +39,7 @@ class Shrine
         # uploads `io` to the location `id`
       end
 
-      def open(id)
+      def open(id, **options)
         # returns the remote file as an IO-like object
       end
 
@@ -46,7 +51,7 @@ class Shrine
         # deletes the file from the storage
       end
 
-      def url(id, options = {})
+      def url(id, **options)
         # URL to the remote file, accepts options for customizing the URL
       end
     end
@@ -54,24 +59,24 @@ class Shrine
 end
 ```
 
-Storages are typically not used directly, but through `Shrine`.
+Storages are typically not used directly, but through [`Shrine`](#shrine) and
+[`Shrine::UploadedFile`](#shrine-uploadedfile) classes.
 
 ## `Shrine`
 
-A `Shrine` object (also called an "uploader") is essentially a wrapper around
-the `#upload` storage method. First the storage needs to be registered under a
-name:
+The `Shrine` class (also called an "uploader") primarily provides a wrapper
+method around `Storage#upload`. First, the storage needs to be registered under
+a name:
 
 ```rb
-Shrine.storages[:file_system] = Shrine::Storage::FileSystem.new("uploads")
+Shrine.storages[:disk] = Shrine::Storage::FileSystem.new("uploads")
 ```
 
-Now we can instantiate an uploader with this identifier and upload files:
+Now we can upload files to the registered storage:
 
 ```rb
-uploader = Shrine.new(:file_system)
-uploaded_file = uploader.upload(file)
-uploaded_file #=> #<Shrine::UploadedFile>
+uploaded_file = Shrine.upload(file, :disk)
+uploaded_file #=> #<Shrine::UploadedFile storage=:disk id="6a9fb596cc554efb" ...>
 ```
 
 The argument to `Shrine#upload` must be an IO-like object. The method does the
@@ -83,63 +88,97 @@ following:
 * closes the file
 * creates a `Shrine::UploadedFile` from the data
 
-`Shrine` class and subclasses are also used for loading plugins that extend all
-core classes. Each `Shrine` subclass has its own subclass of each of the core
-classes (`Shrine::UploadedFile`, `Shrine::Attacher`, and `Shrine::Attachment`),
-which makes it possible to have different `Shrine` subclasses with differently
-customized attachment logic. See [Creating a New Plugin] guide and the [Plugin
-system of Sequel and Roda] article for more details on the design of Shrine's
-plugin system.
+### Plugins
+
+The `Shrine` class is also used for loading plugins, which provide additional
+functionality by extending core classes.
+
+```rb
+Shrine.plugin :derivatives
+
+Shrine::UploadedFile.ancestors #=> [..., Shrine::Plugins::Derivatives::FileMethods, Shrine::UploadedFile::InstanceMethods, ...]
+Shrine::Attacher.ancestors     #=> [..., Shrine::Plugins::Derivatives::AttacherMethods, Shrine::Attacher::InstanceMethods,  ...]
+Shrine::Attachment.ancestors   #=> [..., Shrine::Plugins::Derivatives::AttachmentMethods, Shrine::Attachment::InstanceMethods, ...]
+```
+
+The plugins store their configuration in `Shrine.opts`:
+
+```rb
+Shrine.plugin :derivation_endpoint, secret_key: "foo"
+Shrine.plugin :default_storage, store: :other_store
+Shrine.plugin :activerecord
+
+Shrine.opts #=>
+# { derivation_endpoint: { options: { secret_key: "foo" }, derivations: {} },
+#   default_storage: { store: :other_store },
+#   column: { serializer: Shrine::Plugins::Column::JsonSerializer },
+#   model: { cache: true },
+#   activerecord: { callbacks: true, validations: true } }
+```
+
+Each `Shrine` subclass has its own copy of the core classes, storages and
+options, which makes it possible to customize attachment logic per uploader.
+
+```rb
+MyUploader = Class.new(Shrine)
+MyUploader::UploadedFile.superclass #=> Shrine::UploadedFile
+MyUploader::Attacher.superclass     #=> Shrine::Attacher
+MyUploader::Attachment.superclass   #=> Shrine::Attachment
+```
+
+See [Creating a New Plugin] guide and the [Plugin system of Sequel and Roda]
+article for more details on the design of Shrine's plugin system.
 
 ## `Shrine::UploadedFile`
 
-`Shrine::UploadedFile` represents a file that was uploaded to a storage, and is
-the result of `Shrine#upload`. It is essentially a wrapper around a data hash
-containing information about the uploaded file.
+A `Shrine::UploadedFile` object represents a file that was uploaded to a
+storage, containing upload location, storage, and any metadata extracted during
+the upload.
 
 ```rb
-uploaded_file      #=> #<Shrine::UploadedFile>
-uploaded_file.data #=>
+uploaded_file #=> #<Shrine::UploadedFile id="949sdjg834.jpg" storage=:store metadata={...}>
+
+uploaded_file.id          #=> "949sdjg834.jpg"
+uploaded_file.storage_key #=> :store
+uploaded_file.storage     #=> #<Shrine::Storage::S3>
+uploaded_file.metadata    #=> {...}
+```
+
+It has convenience methods for accessing metadata:
+
+```rb
+uploaded_file.metadata #=>
 # {
-#   "storage"  => "file_system",
-#   "id"       => "9260ea09d8effd.pdf",
-#   "metadata" => {
-#     "filename"  => "resume.pdf",
-#     "mime_type" => "application/pdf",
-#     "size"      => 983294,
-#   },
+#   "filename" => "matrix.mp4",
+#   "mime_type" => "video/mp4",
+#   "size" => 345993,
 # }
+
+uploaded_file.original_filename #=> "matrix.mp4"
+uploaded_file.extension         #=> "mp4"
+uploaded_file.mime_type         #=> "video/mp4"
+uploaded_file.size              #=> 345993
 ```
 
-The data hash contains the storage the file was uploaded to, the location, and
-some metadata: original filename, MIME type and filesize. The
-`Shrine::UploadedFile` object has handy methods which use this data:
+It also has methods that delegate to the storage:
 
 ```rb
-# metadata methods
-uploaded_file.original_filename
-uploaded_file.mime_type
-uploaded_file.size
-# ...
-
-# storage methods
-uploaded_file.url
-uploaded_file.exists?
-uploaded_file.open
-uploaded_file.download
-uploaded_file.delete
-# ...
+uploaded_file.url                     #=> "https://my-bucket.s3.amazonaws.com/949sdjg834.jpg"
+uploaded_file.open { |io| ... }       # opens the uploaded file stream
+uploaded_file.download { |file| ... } # downloads the uploaded file to disk
+uploaded_file.stream(destination)     # streams uploaded content into a writable destination
+uploaded_file.exists?                 #=> true
+uploaded_file.delete                  # deletes the uploaded file from the storage
 ```
 
-A `Shrine::UploadedFile` is itself an IO-like object (representing the
-remote file), so it can be passed to `Shrine#upload` as well.
+A `Shrine::UploadedFile` is itself an IO-like object (built on top of
+`Storage#open`), so it can be passed to `Shrine#upload` as well.
 
 ## `Shrine::Attacher`
 
 We usually want to treat uploaded files as *attachments* to records, saving
-their data into a database column. This is the responsibility of
-`Shrine::Attacher`. A `Shrine::Attacher` uses `Shrine` uploaders and
-`Shrine::UploadedFile` objects internally.
+their data into a database column. This is done by `Shrine::Attacher`, which
+internally uses `Shrine` and `Shrine::UploadedFile` classes.
 
 The attaching process requires a temporary and a permanent storage to be
 registered (by default that's `:cache` and `:store`):
@@ -151,71 +190,82 @@ Shrine.storages = {
 }
 ```
 
-A `Shrine::Attacher` is instantiated with a model instance and an attachment
-name (an "image" attachment will be saved to `image_data` field):
+A `Shrine::Attacher` can be initialized standalone and handle the common
+attachment flow, which includes dirty tracking (promoting cached file to
+permanent storage, deleting previously attached file), validation, processing,
+serialization etc.
 
 ```rb
-attacher = Shrine::Attacher.new(photo, :image)
+attacher = Shrine::Attacher.new
 
-attacher.assign(file)
-attacher.get #=> #<Shrine::UploadedFile>
-attacher.record.image_data #=> "{\"storage\":\"cache\",\"id\":\"9260ea09d8effd.jpg\",\"metadata\":{...}}"
+# ... user uploads a file ...
+
+attacher.assign(io) # uploads to temporary storage
+attacher.file       #=> #<Shrine::UploadedFile storage=:cache ...>
+
+# ... handle file validations ...
 
-attacher._promote
-attacher.get #=> #<Shrine::UploadedFile>
-attacher.record.image_data #=> "{\"storage\":\"store\",\"id\":\"ksdf02lr9sf3la.jpg\",\"metadata\":{...}}"
+attacher.finalize   # uploads to permanent storage
+attacher.file       #=> #<Shrine::UploadedFile storage=:store ...>
 ```
 
-Above a file is assigned by the attacher, which "caches" (uploads) the file to
-the temporary storage. The cached file is then "promoted" (uploaded) to
-permanent storage. Behind the scenes a cached `Shrine::UploadedFile` is given
-to `Shrine#upload`, which works because `Shrine::UploadedFile` is an IO-like
-object. After both caching and promoting the data hash of the uploaded file is
-assigned to the record's column as JSON.
+It can also be initialized with a model instance to handle serialization into a
+model attribute:
 
-For more details see [Using Attacher].
+```rb
+attacher = Shrine::Attacher.from_model(photo, :image)
+
+attacher.assign(file)
+photo.image_data #=> "{\"storage\":\"cache\",\"id\":\"9260ea09d8effd.jpg\",\"metadata\":{...}}"
+
+attacher.finalize
+photo.image_data #=> "{\"storage\":\"store\",\"id\":\"ksdf02lr9sf3la.jpg\",\"metadata\":{...}}"
+```
+
+For more details, see the [Using Attacher] guide and
+[`entity`][entity]/[`model`][model] plugins.
 
 ## `Shrine::Attachment`
 
-`Shrine::Attachment` is the highest level of abstraction. A
-`Shrine::Attachment` module exposes the `Shrine::Attacher` object through the
-model instance. The `Shrine::Attachment` class is a sublcass of `Module`, which
-means that an instance of `Shrine::Attachment` is a module:
+A `Shrine::Attachment` module provides a convenience model interface around the
+`Shrine::Attacher` object. The `Shrine::Attachment` class is a subclass of
+`Module`, which means that an instance of `Shrine::Attachment` is a module:
 
 ```rb
 Shrine::Attachment.new(:image).is_a?(Module) #=> true
-Shrine::Attachment.new(:image).instance_methods #=> [:image=, :image, :image_url, :image_attacher]
+Shrine::Attachment.new(:image).instance_methods #=> [:image=, :image, :image_url, :image_attacher, ...]
 
 # equivalents
 Shrine::Attachment.new(:image)
+Shrine::Attachment[:image]
 Shrine::Attachment(:image)
-Shrine[:image]
 ```
 
-We can include this module to a model:
+We can include this module into a model:
 
 ```rb
-class Photo
-  include Shrine::Attachment.new(:image)
-end
+Photo.include Shrine::Attachment(:image)
 ```
 ```rb
-photo.image = file # shorthand for `photo.image_attacher.assign(file)`
-photo.image        # shorthand for `photo.image_attacher.get`
-photo.image_url    # shorthand for `photo.image_attacher.url`
+photo.image = file   # shorthand for `photo.image_attacher.assign(file)`
+photo.image          # shorthand for `photo.image_attacher.get`
+photo.image_url      # shorthand for `photo.image_attacher.url`
 
-photo.image_attacher #=> #<Shrine::Attacher>
+photo.image_attacher #=> #<Shrine::Attacher @cache_key=:cache @store_key=:store ...>
 ```
 
-When an ORM plugin is loaded, the `Shrine::Attachment` module also
-automatically:
+When a persistence plugin is loaded ([`activerecord`][activerecord],
+[`sequel`][sequel]), the `Shrine::Attachment` module also automatically:
 
 * syncs Shrine's validation errors with the record
 * triggers promoting after record is saved
-* deletes the uploaded file if attachment was replaced/removed or the record
-  destroyed
+* deletes the uploaded file if attachment was replaced or the record destroyed
 
-[Using Attacher]: /doc/attacher.md#readme
+[Using Attacher]: https://shrinerb.com/docs/attacher
 [Notes on study of shrine implementation]: https://bibwild.wordpress.com/2018/09/12/notes-on-study-of-shrine-implementation/
-[Creating a New Plugin]: /doc/creating_plugins.md#readme
+[Creating a New Plugin]: https://shrinerb.com/docs/creating-plugins
 [Plugin system of Sequel and Roda]: https://twin.github.io/the-plugin-system-of-sequel-and-roda/
+[entity]: https://shrinerb.com/docs/plugins/entity
+[model]: https://shrinerb.com/docs/plugins/model
+[activerecord]: https://shrinerb.com/docs/plugins/activerecord
+[sequel]: https://shrinerb.com/docs/plugins/sequel

data/doc/direct_s3.md

@@ -1,4 +1,7 @@
-# Direct Uploads to S3
+---
+id: direct-s3
+title: Direct Uploads to S3
+---
 
 Shrine gives you the ability to upload files directly to Amazon S3 (or any
 other storage service that accepts direct uploads). Uploading directly to a
@@ -23,12 +26,14 @@ storage service is beneficial for several reasons:
   request-response lifecycle might not be able to finish before the request
   times out.
 
+## Storage
+
 To start, let's set both temporary and permanent storage to S3, with the
 temporary storage uploading to the `cache/` prefix:
 
 ```rb
 # Gemfile
-gem "shrine", "~> 2.11"
+gem "shrine", "~> 3.0"
 gem "aws-sdk-s3", "~> 1.14"
 ```
 ```rb
@@ -71,6 +76,7 @@ If you're using [Uppy], this is the recommended CORS configuration for the
     <AllowedHeader>x-amz-content-sha256</AllowedHeader>
     <AllowedHeader>content-type</AllowedHeader>
     <AllowedHeader>content-disposition</AllowedHeader>
+    <ExposeHeader>ETag</ExposeHeader>
   </CORSRule>
   <CORSRule>
     <AllowedOrigin>*</AllowedOrigin>
@@ -80,6 +86,31 @@ If you're using [Uppy], this is the recommended CORS configuration for the
 </CORSConfiguration>
 ```
 
+Or in JSON format:
+
+```json
+[
+    {
+        "AllowedOrigins": [ "https://my-app.com" ],
+        "AllowedMethods": [ "GET", "POST", "PUT" ],
+        "MaxAgeSeconds": 3000,
+        "AllowedHeaders": [
+            "Authorization",
+            "x-amz-date",
+            "x-amz-content-sha256",
+            "Content-Type",
+            "Content-Disposition"
+        ],
+        "ExposeHeaders": [ "ETag" ]
+    },
+    {
+        "AllowedOrigins": [ "*" ],
+        "AllowedMethods": [ "GET" ],
+        "MaxAgeSeconds": 3000
+    }
+]
+```
+
 Replace `https://my-app.com` with the URL to your app (in development you can
 set this to `*`). Once you've hit "Save", it may take some time for the
 new CORS settings to be applied.
@@ -247,28 +278,6 @@ additional HTTP requests.
 See [this section][metadata direct uploads] or the rationale and instructions
 on how to opt in.
 
-## Object data
-
-When the cached S3 object is copied to permanent storage, the destination S3
-object will by default inherit any object data that was assigned to the cached
-object via presign parameters. However, S3 will by default also ignore any new
-object parameters that are given to the copy request.
-
-Whether object data will be copied or replaced depends on the value of the
-`:metadata_directive` parameter:
-
-* `"COPY"` - destination object will inherit source object data and any new data will be ignored (default)
-* `"REPLACE"` - destination object will not inherit any of the source object data and will accept new data
-
-You can use the `upload_options` plugin to change the `:metadata_directive`
-option when S3 objects are copied:
-
-```rb
-plugin :upload_options, store: -> (io, context) do
-  { metadata_directive: "REPLACE" } if io.is_a?(Shrine::UploadedFile)
-end
-```
-
 ## Clearing cache
 
 Directly uploaded files won't automatically be deleted from your temporary
@@ -330,8 +339,9 @@ backgrounding library to perform the job with a delay:
 ```rb
 Shrine.plugin :backgrounding
 
-Shrine::Attacher.promote do |data|
-  PromoteJob.perform_in(3, data) # tells a Sidekiq worker to perform in 3 seconds
+Shrine::Attacher.promote_block do
+  # tells a Sidekiq worker to perform in 3 seconds
+  PromoteJob.perform_in(3, self.class.name, record.class.name, record.id, name, file_data)
 end
 ```
 
@@ -396,6 +406,6 @@ setup] guide.
 [lifecycle Console]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
 [lifecycle API]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#put_bucket_lifecycle_configuration-instance_method
 [Minio]: https://minio.io
-[minio setup]: /doc/testing.md#minio
-[metadata direct uploads]: /doc/metadata.md#direct-uploads
+[minio setup]: https://shrinerb.com/docs/testing#minio
+[metadata direct uploads]: https://shrinerb.com/docs/metadata#direct-uploads
 [content_disposition]: https://github.com/shrinerb/content_disposition