shrine 3.0.0.rc → 3.0.0

data/doc/carrierwave.md

@@ -1,4 +1,6 @@
-# Shrine for CarrierWave Users
+---
+title: Shrine for CarrierWave Users
+---
 
 This guide is aimed at helping CarrierWave users transition to Shrine, and it
 consists of three parts:
@@ -7,10 +9,44 @@ consists of three parts:
 2. Instructions how to migrate and existing app that uses CarrierWave to Shrine
 3. Extensive reference of CarrierWave's interface with Shrine equivalents
 
-## Storage
+## Overview
 
-While in CarrierWave you configure storage in global configuration, in Shrine
-storage is a class which you can pass options to during initialization:
+### Uploader
+
+Shrine shares CarrierWave's concept of **uploaders**, classes which encapsulate
+file attachment logic for different file types:
+
+```rb
+class ImageUploader < Shrine
+  # attachment logic
+end
+```
+
+However, while CarrierWave uploaders are responsible for most of the
+attachment logic (uploading to temporary/permanent storage, retrieving the
+uploaded file, file validation, processing versions), Shrine distributes
+these responsibilities across several core classes:
+
+| Class                  | Description                                                        |
+| :----                  | :-----------                                                       |
+| `Shrine`               | handles uploads, metadata extraction, location generation          |
+| `Shrine::UploadedFile` | exposes metadata, implements downloading, URL generation, deletion |
+| `Shrine::Attacher`     | handles caching & storing, dirty tracking, persistence, versions   |
+
+Shrine uploaders themselves are functional: they receive a file on the input
+and return the uploaded file on the output. There are no state changes.
+
+```rb
+uploader      = ImageUploader.new(:store)
+uploaded_file = uploader.upload(file, :store)
+uploaded_file          #=> #<Shrine::UploadedFile>
+uploaded_file.url      #=> "https://my-bucket.s3.amazonaws.com/store/kfds0lg9rer.jpg"
+uploaded_file.download #=> #<File:/tmp/path/to/file>
+```
+
+### Storage
+
+In CarrierWave, you configure storage in global configuration:
 
 ```rb
 CarrierWave.configure do |config|
@@ -24,6 +60,9 @@ CarrierWave.configure do |config|
   config.fog_directory = "my-bucket"
 end
 ```
+
+In Shrine, the configuration options are passed directly to the storage class:
+
 ```rb
 Shrine.storages[:store] = Shrine::Storage::S3.new(
   bucket:            "my-bucket",
@@ -33,11 +72,11 @@ Shrine.storages[:store] = Shrine::Storage::S3.new(
 )
 ```
 
-In CarrierWave temporary storage cannot be configured; it saves and retrieves
-files from the filesystem, you can only set the directory. With Shrine both
-temporary (`:cache`) and permanent (`:store`) storage are first-class citizens
-and fully configurable, so you can also have files *cached* on S3 (preferrably
-via [direct uploads]):
+#### Temporary storage
+
+Where CarrierWave's temporary storage is hardcoded to disk, Shrine can use any
+storage for temporary storage. So, if you have multiple servers or want to do
+[direct uploads], you can use AWS S3 as temporary storage:
 
 ```rb
 Shrine.storages = {
@@ -46,32 +85,52 @@ Shrine.storages = {
 }
 ```
 
-## Uploader
+### Persistence
 
-Shrine shares CarrierWave's concept of *uploaders*, classes which encapsulate
-file attachment logic for different file types:
+While CarrierWave persists only the filename of the original uploaded file,
+Shrine persists storage and metadata information as well:
 
 ```rb
-class ImageUploader < Shrine
-  # attachment logic
-end
+{
+  "id": "path/to/image.jpg",
+  "storage": "store",
+  "metadata": {
+    "filename": "nature.jpg",
+    "size": 4739472,
+    "mime_type": "image/jpeg"
+  }
+}
 ```
 
-However, uploaders in CarrierWave are very broad; in addition to uploading and
-deleting files, they also represent the uploaded file. Shrine has a separate
-`Shrine::UploadedFile` class which represents the uploaded file.
+This way we have all information about uploaded files, without having to
+retrieve the file from the storage.
 
 ```rb
-uploaded_file = ImageUploader.upload(file, :store)
-uploaded_file          #=> #<Shrine::UploadedFile>
-uploaded_file.url      #=> "https://my-bucket.s3.amazonaws.com/store/kfds0lg9rer.jpg"
-uploaded_file.download #=> #<File:/tmp/path/to/file>
+photo.image.id          #=> "path/to/image.jpg"
+photo.image.storage_key #=> :store
+photo.image.metadata    #=> { "filename" => "...", "size" => ..., "mime_type" => "..." }
+
+photo.image.original_filename #=> "nature.jpg"
+photo.image.size              #=> 4739472
+photo.image.mime_type         #=> "image/jpeg"
 ```
 
-## Processing
+#### Location
+
+CarrierWave persists only the filename of the uploaded file, and recalculates
+the full location dynamically based on location configuration. This can be
+dangerous, because if some component of the location happens to change, all
+existing links might become invalid.
 
-In contrast to CarrierWave's class-level DSL, in Shrine processing is defined
-and performed on the instance-level.
+To avoid this, Shrine persists the full location on attachment, and uses it
+when generating file URL. So, even if you change how file locations are
+generated, existing files that are on old locations will still remain
+accessible.
+
+### Processing
+
+CarrierWave uses a class-level DSL for generating versions, which internally
+uses uploader subclassing and does in-place processing.
 
 ```rb
 class ImageUploader < CarrierWave::Uploader::Base
@@ -91,6 +150,9 @@ class ImageUploader < CarrierWave::Uploader::Base
 end
 ```
 
+In contrast, in Shrine you perform processing on the instance level as a
+functional transformation, which is a lot simpler and more flexible:
+
 ```rb
 require "image_processing/mini_magick"
 
@@ -109,23 +171,45 @@ class ImageUploader < Shrine
 end
 ```
 
-CarrierWave performs processing before validations, which is a huge security
-issue, as it allows users to give arbitrary files to your processing tool, even
-if you have validations. With Shrine you can perform processing after
-validations.
+#### Retrieving versions
 
-Shrine doesn't have a built-in way of regenerating versions, but there is an
-extensive [Managing Derivatives] guide.
+When retrieving versions, CarrierWave returns a list of declared versions which
+may or may not have been generated. In contrast, Shrine persists data of
+uploaded processed files into the database (including any extracted metadata),
+which then becomes the source of truth on which versions have been generated.
 
-### Validations
+```rb
+photo.image              #=> #<Shrine::UploadedFile @id="original.jpg" ...>
+photo.image_derivatives  #=> {}
 
-Like with processing, validations in Shrine are also defined and performed on
-instance-level:
+photo.image_derivatives! # triggers processing
+photo.image_derivatives  #=>
+# {
+#   large: #<Shrine::UploadedFile @id="large.jpg" @metadata={"size"=>873232, ...} ...>,
+#   medium: #<Shrine::UploadedFile @id="medium.jpg" @metadata={"size"=>94823, ...} ...>,
+#   small: #<Shrine::UploadedFile @id="small.jpg" @metadata={"size"=>37322, ...} ...>,
+# }
+```
+
+#### Reprocessing versions
+
+Shrine doesn't have a built-in way of regenerating versions, because that has
+to be written and optimized differently depending on what versions have changed
+which persistence library you're using, how many records there are in the table
+etc.
+
+However, there is an extensive guide for [Managing Derivatives], which provides
+instructions on how to make these changes safely and with zero downtime.
+
+### Validation
+
+File validation in Shrine is also instance-level, which allows using
+conditionals:
 
 ```rb
 class ImageUploader < CarrierWave::Uploader::Base
   def extension_whitelist
-    %w[jpg jpeg gif png]
+    %w[jpg jpeg png webp]
   end
 
   def content_type_whitelist
@@ -143,91 +227,43 @@ class ImageUploader < Shrine
   plugin :validation_helpers
 
   Attacher.validate do
-    validate_extension %w[jpg jpeg gif png]
-    validate_mime_type %w[image/jpeg image/gif image/png]
     validate_max_size 10*1024*1024
+    validate_extension %w[jpg jpeg png webp]
+
+    if validate_mime_type %w[image/jpeg image/png image/webp]
+      validate_max_dimensions [5000, 5000]
+    end
   end
 end
 ```
 
-## Attachments
+#### Custom metadata
 
-Like CarrierWave, Shrine also provides integrations with ORMs. It ships with
-plugins for both Sequel and ActiveRecord, but can also be used with just PORO
-models.
+With Shrine you can also extract and validate any custom metadata:
 
 ```rb
-Shrine.plugin :sequel       # if you're using Sequel
-Shrine.plugin :activerecord # if you're using ActiveRecord
-```
+class VideoUploader < Shrine
+  plugin :add_metadata
+  plugin :validation
 
-Instead of giving you class methods for "mounting" uploaders, in Shrine you
-generate attachment modules which you simply include in your models, which
-gives your models similar set of methods that CarrierWave gives:
+  add_metadata :duration do |io|
+    FFMPEG::Movie.new(io.path).duration
+  end
 
-```rb
-class Photo < ActiveRecord::Base
-  extend CarrierWave::ActiveRecord # done automatically by CarrierWave
-  mount_uploader :image, ImageUploader
-end
-```
-```rb
-class Photo < ActiveRecord::Base
-  include ImageUploader::Attachment(:image)
+  Attacher.validate do
+    if file.duration > 5*60*60
+      errors << "must not be longer than 5 hours"
+    end
+  end
 end
 ```
 
-### Attachment column
-
-You models are required to have the `<attachment>_data` column, which Shrine
-uses to save storage, location, and metadata of the uploaded file.
-
-```rb
-photo.image_data #=>
-# {
-#   "storage": "store",
-#   "id": "photo/1/image/0d9o8dk42.png",
-#   "metadata": {
-#     "filename":  "nature.png",
-#     "size":      49349138,
-#     "mime_type": "image/png"
-#   }
-# }
-
-photo.image.original_filename #=> "nature.png"
-photo.image.size              #=> 49349138
-photo.image.mime_type         #=> "image/png"
-```
-
-This is much more powerful than storing only the filename like CarrierWave
-does, as it allows you to also store any additional metadata that you might
-want to extract.
-
-Unlike CarrierWave, Shrine will store this information for each processed
-version, making them first-class citizens:
-
-```rb
-photo.image               #=> #<Shrine::UploadedFile>
-photo.image.width         #=> 800
-
-photo.image(:thumb)       #=> #<Shrine::UploadedFile>
-photo.image(:thumb).width #=> 300
-```
-
-Also, since CarrierWave stores only the filename, it has to recalculate the
-full location each time it wants to generate the URL. That makes it really
-difficult to move files to a new location, because changing how the location is
-generated will now cause incorrect URLs to be generated for all existing files.
-Shrine calculates the whole location only once and saves it to the column.
-
 ### Multiple uploads
 
-Shrine doesn't have support for multiple uploads like CarrierWave does, instead
-it expects that you will attach each file to a separate database record. This
-is a good thing, because the implementation is specific to the ORM you're
-using, and it's analogous to how you would implement any nested one-to-many
-associations. Take a look at the [demo app] which shows how easy it is to
-implement multiple uploads.
+Shrine doesn't have support for multiple uploads out-of-the-box like
+CarrierWave does. Instead, you can implement them using a separate table with a
+one-to-many relationship to which the files will be attached. The [Multiple
+Files] guide explains this setup in more detail.
 
 ## Migrating from CarrierWave
 
@@ -653,9 +689,10 @@ multipart or not.
 
 ### `CarrierWave::Storage::Fog`
 
-You can use [`Shrine::Storage::S3`] \(built-in\),
-[`Shrine::Storage::GoogleCloudStorage`], or generic [`Shrine::Storage::Fog`]
-storage. The reference will assume you're using S3 storage.
+You can use [`Shrine::Storage::S3`][S3] (built-in),
+[`Shrine::Storage::GoogleCloudStorage`][shrine-gcs], or generic
+[`Shrine::Storage::Fog`][shrine-fog] storage. The reference will assume you're
+using S3 storage.
 
 #### `:fog_credentials`, `:fog_directory`
 
@@ -704,11 +741,9 @@ Shrine allows you to override the S3 endpoint:
 Shrine::Storage::S3.new(use_accelerate_endpoint: true, **options)
 ```
 
-[image_processing]: https://github.com/janko/image_processing
-[demo app]: https://github.com/shrinerb/shrine/tree/master/demo
-[Managing Derivatives]: /doc/changing_derivatives.md#readme
+[Managing Derivatives]: https://shrinerb.com/docs/changing-derivatives
+[direct uploads]: https://shrinerb.com/docs/getting-started#direct-uploads
+[S3]: https://shrinerb.com/docs/storage/s3
+[shrine-gcs]: https://github.com/renchap/shrine-google_cloud_storage
 [shrine-fog]: https://github.com/shrinerb/shrine-fog
-[direct uploads]: /doc/direct_s3.md#readme
-[`Shrine::Storage::S3`]: /doc/storage/s3.md#readme
-[`Shrine::Storage::GoogleCloudStorage`]: https://github.com/renchap/shrine-google_cloud_storage
-[`Shrine::Storage::Fog`]: https://github.com/shrinerb/shrine-fog
+[Multiple Files]: https://shrinerb.com/docs/multiple-files

data/doc/changing_derivatives.md

@@ -1,4 +1,7 @@
-# Managing Derivatives
+---
+id: changing-derivatives
+title: Managing Derivatives
+---
 
 This guide shows how to add, create, update, and remove [derivatives] for an 
 app in production already handling file attachments, with zero downtime.
@@ -22,15 +25,6 @@ class Photo < ActiveRecord::Base
 end
 ```
 
-## Contents
-
-* [Adding derivatives](#adding-derivatives)
-* [Reprocessing all derivatives](#reprocessing-all-derivatives)
-* [Reprocessing certain derivatives](#reprocessing-certain-derivatives)
-* [Adding new derivatives](#adding-new-derivatives)
-* [Removing derivatives](#removing-derivatives)
-* [Backgrounding](#backgrounding)
-
 ## Adding derivatives
 
 *Scenario: Your app is currently working only with original files, and you want
@@ -311,4 +305,4 @@ class MakeChangeJob
 end
 ```
 
-[derivatives]: /doc/plugins/derivatives.md#readme
+[derivatives]: https://shrinerb.com/docs/plugins/derivatives

data/doc/changing_location.md

@@ -1,4 +1,7 @@
-# Migrating File Locations
+---
+id: changing-location
+title: Migrating File Locations
+---
 
 This guide shows how to migrate the location of uploaded files on the same 
 storage in production, with zero downtime.
@@ -56,10 +59,10 @@ Photo.find_each do |photo|
 
   begin
     attacher.atomic_persist           # persist changes if attachment has not changed in the meantime
-    old_attacher.destroy              # delete files on old location
+    old_attacher.destroy_attached     # delete files on old location
   rescue Shrine::AttachmentChanged,   # attachment has changed during reuploading
          ActiveRecord::RecordNotFound # record has been deleted during reuploading
-    attacher.destroy                  # delete now orphaned files
+    attacher.destroy_attached         # delete now orphaned files
   end
 end
 ```
@@ -100,9 +103,9 @@ class MoveFilesJob
     attacher.set_derivatives attacher.upload_derivatives(attacher.derivatives)
 
     attacher.atomic_persist
-    old_attacher.destroy
+    old_attacher.destroy_attached
   rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
-    attacher&.destroy
+    attacher&.destroy_attached
   end
 end
 ```

data/doc/changing_storage.md

@@ -1,4 +1,7 @@
-# Migrating File Storage
+---
+id: changing-storage
+title: Migrating File Storage
+---
 
 This guides shows how to move file attachments to a different storage in 
 production, with zero downtime.
@@ -104,4 +107,4 @@ Shrine.storages = {
 - Shrine.plugin :mirroring, mirror: { store: :s3 } # mirror to :s3 storage
 ```
 
-[mirroring backgrounding]: /doc/plugins/mirroring.md#backgrounding
+[mirroring backgrounding]: https://shrinerb.com/docs/plugins/mirroring#backgrounding

data/doc/creating_persistence_plugins.md

@@ -1,4 +1,7 @@
-# Writing an Persistence Plugin
+---
+id: creating-persistence-plugins
+title: Writing a Persistence Plugin
+---
 
 This guide explains some conventions for writing Shrine plugins that integrate
 with persistence libraries such as Active Record, Sequel, ROM and Mongoid. It
@@ -118,12 +121,12 @@ module Shrine::Plugins::Raptor
 end
 ```
 
-[Writing a Plugin]: /doc/creating_plugins.md#readme
+[Writing a Plugin]: https://shrinerb.com/docs/creating-plugins
 [Active Record pattern]: https://www.martinfowler.com/eaaCatalog/activeRecord.html
-[model]: /doc/plugins/model.md#readme
-[entity]: /doc/plugins/entity.md#readme
+[model]: https://shrinerb.com/docs/plugins/model
+[entity]: https://shrinerb.com/docs/plugins/entity
 [Repository pattern]: https://martinfowler.com/eaaCatalog/repository.html
-[backgrounding]: /doc/plugins/backgrounding.md#readme
-[atomic_helpers]: /doc/plugins/atomic_helpers.md#readme
+[backgrounding]: https://shrinerb.com/docs/plugins/backgrounding
+[atomic_helpers]: https://shrinerb.com/docs/plugins/atomic_helpers
 [activerecord]: /lib/shrine/plugins/activerecord.rb
 [sequel]: /lib/shrine/plugins/sequel.rb