shrine 2.3.1 → 2.4.0

data/doc/attacher.md

@@ -0,0 +1,227 @@
+# Using Attacher
+
+The most convenient way to use Shrine is through the model, using the interface
+provided by Shrine's attachment module. This way you can interact with the
+attachment just like with any other column attribute, and adding attachment
+fields to the form just works.
+
+```rb
+class Photo < Sequel::Model
+  include ImageUploader[:image]
+end
+```
+
+However, you don't want to add additional methods on the model and prefer
+explicitness, or you need more control, you can achieve the same behaviour
+using the `Shrine::Attacher` object, which is what the attachment interface
+uses under the hood.
+
+```rb
+attacher = ImageUploader::Attacher.new(photo, :image) # equivalent to `photo.image_attacher`
+attacher.assign(file)                                 # equivalent to `photo.image = file`
+attacher.get                                          # equivalent to `photo.image`
+```
+
+## Attributes
+
+The attacher object exposes the objects it uses:
+
+```rb
+attacher.record #=> #<Photo>
+attacher.name   #=> :image
+attacher.cache  #=> #<ImageUploader @storage_key=:cache>
+attacher.store  #=> #<ImageUploader @storage_key=:store>
+```
+
+The attacher will automatically use `:cache` and `:store` storages, but you can
+also tell it to use different temporary and permanent storage:
+
+```rb
+ImageUploader::Attacher.new(photo, :image, cache: :other_cache, store: :other_store)
+```
+
+The attacher will use the `<attachment>_data` attribute for storing information
+about the attachment.
+
+```rb
+attacher.data_attribute #=> :image_data
+```
+
+## Assignment
+
+The `#assign` method accepts either an IO object to be cached, or an already
+cached file in form of a JSON string, and assigns the cached result to record's
+`<attachment>_data` attribute.
+
+```rb
+# uploads the `io` object to temporary storage, and writes to the data column
+attacher.assign(io)
+
+# writes the given cached file to the data column
+attacher.assign '{
+  "storage": "cache",
+  "id": "9260ea09d8effd.jpg",
+  "metadata": { ... }
+}'
+```
+
+For security reasons `#assign` doesn't accept files uploaded to permanent
+storage, but you can also use `#set` to attach any `Shrine::UploadedFile`
+object.
+
+```rb
+uploaded_file #=> #<Shrine::UploadedFile>
+attacher.set(uploaded_file)
+```
+
+## Retrieval
+
+The `#get` method reads record's `<attachment>_data` attribute, and constructs
+a `Shrine::UploadedFile` object from it.
+
+```rb
+attacher.get #=> #<Shrine::UploadedFile>
+```
+
+The `#read` method will just return the value of the underlying
+`<attachment>_data` attribute.
+
+```rb
+attacher.read #=> '{"storage":"cache","id":"dsg024lfs.jpg",...}'
+```
+
+In general you can use `#uploaded_file` to contruct a `Shrine::UploadedFile`
+from a JSON string.
+
+```rb
+attacher.uploaded_file('{"storage":"cache","id":"dsg024lfs.jpg",...}') #=> #<Shrine::UploadedFile>
+```
+
+## URL
+
+The `#url` method returns the URL to the attached file, and returns `nil` if
+no file is attached.
+
+```rb
+attacher.url # calls `attacher.get.url`
+```
+
+## State
+
+You can ask the attacher whether the currently attached file is cached or
+stored.
+
+```rb
+attacher.cached?
+attacher.stored?
+```
+
+## Validations
+
+Whenever a file is assigned via `#assign` or `#set`, the file validations are
+automatically run, and you can access the validation errors through `#errors`:
+
+```rb
+attacher.assign(large_file)
+attacher.errors #=> ["is larger than 10 MB"]
+```
+
+## Promoting
+
+After the attachment is assigned and you run validations, it should be promoted
+to permanent storage after the record is saved. You can use `#finalize` for
+that, since that will also automatically delete any previously attached files.
+
+```rb
+# We run the finalization only if a new file was attached
+attacher.finalize if attacher.attached?
+```
+
+This is normally automatically added to a callback by the ORM plugin when going
+through the model. Internally this calls `#promote`, which uploads a given
+`Shrine::UploadedFile` to permanent storage, and swaps it with the current
+attachment, unless a new file was attached in the meanwhile.
+
+```rb
+# uploads cached file to permanent storage and replaces the current one
+attacher.promote(cached_file, action: :custom_name)
+```
+
+The `:action` parameter is optional; it can be used for triggering a certain
+processing block, and it is also automatically printed by the `logging` plugin
+to aid in debugging.
+
+Internally this calls `#swap`, which will update the record with any uploaded
+file, but will reload the record to check if the current attachment hasn't
+changed (if the `backgrounding` plugin is loaded).
+
+```rb
+attacher.swap(uploaded_file)
+```
+
+Both `#promote` and `#swap` are useful for [file migrations].
+
+## Backgrounding
+
+When the `backgrounding` plugin is loaded, it allows you to promote and delete
+files in the background, and the corresponding methods are prefixed with `_`:
+
+```rb
+Shrine.plugin :backgrounding
+Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
+Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
+```
+```rb
+attacher._promote(cached_file)  # calls the registered `Attacher.promote` block
+attacher._delete(uploaded_file) # calls the registered `Attacher.delete` block
+```
+
+These are automatically used when using Shrine through models.
+
+## Context
+
+The attacher sends `#context` to each upload/delete call to the uploader. By
+default it will hold `:record` and `:name`:
+
+```rb
+attacher.context #=>
+# {
+#   record: #<Photo...>,
+#   name:   :image,
+# }
+```
+
+However, you can change/add additional context to be sent when calling the
+uploaders:
+
+```rb
+attacher.context[:foo] = "bar"
+```
+
+This is useful for example if you have immutable model instances, and you want
+to assign a new updated instance. For example both foreground and background
+`#promote` requires that the record is persisted (and its `#id` is present).
+
+## Uploading and deleting
+
+Normally you can upload and delete directly by using the uploader.
+
+```rb
+uploader = ImageUploader.new(:store)
+uploaded_file = uploader.upload(image) # uploads the file to `:store` storage
+uploader.delete(uploaded_file)         # deletes the file uploaded to `:store`
+```
+
+The attacher has methods for "caching", "storing" and "deleting" files, which
+delegate to these uploader methods, but also pass in the `#context`:
+
+```rb
+cached_file = attacher.cache!(image) # delegates to `Shrine#upload`
+stored_file = attacher.store!(image) # delegates to `Shrine#upload`
+attacher.delete!(stored_file)        # delegates to `Shrine#delete`
+```
+
+The `#cache!` and `#store!` only upload the file to the storage, they don't
+write to record's data column.
+
+[file migrations]: http://shrinerb.com/rdoc/files/doc/migrating_storage_md.html

data/doc/design.md

@@ -28,26 +28,14 @@ A storage is a PORO which responds to certain methods:
 class Shrine
   module Storage
     class MyStorage
-      def initialize(*args)
-        # initializing logic
-      end
-
       def upload(io, id, shrine_metadata: {}, **upload_options)
         # uploads `io` to the location `id`
       end
 
-      def download(id)
-        # downloads the file from the storage
-      end
-
       def open(id)
         # returns the remote file as an IO-like object
       end
 
-      def read(id)
-        # returns the file contents as a string
-      end
-
       def exists?(id)
         # checks if the file exists on the storage
       end
@@ -59,12 +47,6 @@ class Shrine
       def url(id, options = {})
         # URL to the remote file, accepts options for customizing the URL
       end
-
-      def clear!(confirm = nil)
-        # deletes all the files in the storage
-      end
-
-      # ...
     end
   end
 end
@@ -182,6 +164,8 @@ to `Shrine#upload`, which works because `Shrine::UploadedFile` is an IO-like
 object. After both caching and promoting the data hash of the uploaded file is
 assigned to the record's column as JSON.
 
+For more details see [Using Attacher].
+
 ## `Shrine::Attachment`
 
 `Shrine::Attachment` is the highest level of abstraction. A
@@ -219,5 +203,7 @@ automatically:
 
 * syncs Shrine's validation errors with the record
 * triggers promoting after record is saved
-* deletes the uploaded file if attachment was replaced, removed or the record
+* deletes the uploaded file if attachment was replaced/removed or the record
   destroyed
+
+[Using Attacher]: http://shrinerb.com/rdoc/files/doc/attacher_md.html

data/doc/paperclip.md

@@ -29,8 +29,9 @@ Shrine.storages[:store] = Shrine::Storage::S3.new(
   bucket:            "my-bucket",
   access_key_id:     "abc",
   secret_access_key: "xyz",
-  host:              "http://abc123.cloudfront.net",
 )
+
+Shrine.plugin :default_url_options, store: {host: "http://abc123.cloudfront.net"}
 ```
 
 Paperclip doesn't have a concept of "temporary" storage, so it cannot retain

data/doc/testing.md

@@ -0,0 +1,266 @@
+# Testing with Shrine
+
+The goal of this guide is to provide some useful tips for testing file
+attachments implemented with Shrine in your application.
+
+## Callbacks
+
+When you first try to test file attachments, you might experience that files
+are simply not being promoted (uploaded from temporary to permanent storage).
+This is because your tests are likely setup to be wrapped inside database
+transactions, and that doesn't work with Shrine callbacks.
+
+Specifically, Shrine uses "after commit" callbacks for promoting and deleting
+attached files. This means that if your tests are wrapped inside transactions,
+those Shrine actions will happen only after those transactions commit, which
+happens only after the test has already finished.
+
+```rb
+# Promoting will happen only after the test transaction commits
+it "can attach images" do
+  photo = Photo.create(image: image_file)
+  photo.image.storage_key #=> :cache (we expected it to be promoted to permanent storage)
+end
+```
+
+For file attachments to properly work, you'll need to disable transactions for
+those tests. For Rails apps you can tell Rails not to use transactions, and
+instead use libraries like [DatabaseCleaner] which allow you to use table
+truncation or deletion strategies instead of transactions.
+
+```rb
+RSpec.configure do |config|
+  config.use_transactional_fixtures = false
+end
+```
+
+## Storage
+
+If you're using an external storage in development, it is common in tests to
+switch to a filesystem storage. However, that means that you'll also have to
+clean up the test directory between tests, and writing to filesystem a lot can
+affect the performance of your tests.
+
+Instead of filesystem you can use [memory storage][shrine-memory], which is
+both faster and doesn't require you to clean up anything between tests.
+
+```rb
+gem "shrine-memory"
+```
+```rb
+# test/test_helper.rb
+require "shrine/storage/memory"
+
+Shrine.storages = {
+  cache: Shrine::Storage::Memory.new,
+  store: Shrine::Storage::Memory.new,
+}
+```
+
+## Test data
+
+If you're creating test data dynamically using libraries like [factory_girl],
+you can have the test file assigned dynamically when the record is created:
+
+```rb
+factory :photo do
+  image File.open("test/files/image.jpg")
+end
+```
+
+On the other hand, if you're setting up test data using YAML fixtures, you
+aren't that flexible, because you can only use primitive data types that are
+part of the YAML language. In that case you can load the `data_uri` Shrine
+plugin, and assign files in form of data URI strings through the
+`<attachment>_data_uri` accessor provided by the plugin.
+
+```rb
+Shrine.plugin :data_uri
+```
+```yml
+# test/fixtures/photos.yml
+photo:
+  image_data_uri: "data:image/png,<%= File.read("test/files/image.png") %>"
+```
+
+## Background jobs
+
+If you're using background jobs with Shrine, you probably want to make them
+synchronous in tests. Your favourite backgrounding library should already
+support this, examples:
+
+```rb
+# Sidekiq
+require "sidekiq/testing"
+Sidekiq::Testing.inline!
+```
+
+```rb
+# SuckerPunch
+require "sucker_punch/testing/inline"
+```
+
+```rb
+# ActiveJob
+ActiveJob::Base.queue_adapter = :inline
+```
+
+## Acceptance tests
+
+In acceptance tests you're testing your app end-to-end, and you likely want to
+also test file attachments here. There are a variety of libraries that you
+might be using for your acceptance tests.
+
+### Capybara
+
+If you're testing with the [Capybara] acceptance test framework, you can use
+[`#attach_file`] to select a file from your filesystem in the form:
+
+```rb
+attach_file("#image-field", "test/files/image.jpg")
+```
+
+### Rack::Test
+
+Regular routing tests in Rails use [Rack::Test], in which case you can create
+`Rack::Test::UploadedFile` objects and pass them as form parameters:
+
+```rb
+post "/photos", photo: {image: Rack::Test::UploadedFile.new("test/files/image.jpg", "image/jpeg")}
+```
+
+### Rack::TestApp
+
+With [Rack::TestApp] you can create multipart file upload requests by using the
+`:multipart` option and passing a `File` object:
+
+```rb
+http.post "/photos", multipart: {"photo[image]" => File.open("test/files/image.jpg")}
+```
+
+## Attachment
+
+Even though all the file attachment logic is usually encapsulated in your
+uploader classes, in general it's still best to test this logic through models.
+
+In your controller the attachment attribute using the uploaded file from the
+controller, in Rails case it's an `ActionDispatch::Http::UploadedFile`.
+However, you can also assign plain `File` objects, or any other kind of IO-like
+objects.
+
+```rb
+describe ImageUploader do
+  it "generates image thumbnails" do
+    photo = Photo.create(image: File.open("test/files/image.png"))
+    assert_equal [:small, :medium, :large], photo.image.keys
+  end
+end
+```
+
+If you want test with an IO object that closely resembles the kind of IO that
+is assigned by your web framework, you can use this:
+
+```rb
+require "forwardable"
+require "stringio"
+
+class FakeIO
+  attr_reader :original_filename, :content_type
+
+  def initialize(content, filename: nil, content_type: nil)
+    @io = StringIO.new(content)
+    @original_filename = filename
+    @content_type = content_type
+  end
+
+  extend Forwardable
+  delegate Shrine::IO_METHODS.keys => :@io
+end
+```
+
+```rb
+describe ImageUploader do
+  it "generates image thumbnails" do
+    photo = Photo.create(image: FakeIO.new(File.read("test/files/image.png")))
+    assert_equal [:small, :medium, :large], photo.image.keys
+  end
+end
+```
+
+## Processing
+
+In tests you usually don't want to perform processing, or at least don't want
+it to be performed by default (only when you're actually testing it).
+
+If you're processing only single files, you can override the `Shrine#process`
+method in tests to return nil:
+
+```rb
+class ImageUploader
+  def process(io, context)
+    # don't do any processing
+  end
+end
+```
+
+If you're processing versions, you can override `Shrine#process` to simply
+return a hash of unprocessed original files:
+
+```rb
+class ImageUploader
+  def process(io, context)
+    if context[:action] == :store
+      {small: io, medium: io, large: io}
+    end
+  end
+end
+```
+
+However, it's even better to design your processing code in such a way that
+it's easier to swap out in tests. In your *application* code you could extract
+processing into a single `#call`-able object, and register it inside uploader
+generic `#opts` hash.
+
+```rb
+class ImageUploader < Shrine
+  opts[:processor] = ImageThumbnailsGenerator
+
+  process(:store) do |io, context|
+    opts[:processor].call(io, context)
+  end
+end
+```
+
+Now in your tests you can easily swap out `ImageThumbnailsGenerator` with
+"fake" processing, which just returns the result in correct format (single file
+or hash of versions). Since the only requirement of the processor is that it
+responds to `#call`, we can just swap it out for a proc or a lambda:
+
+```rb
+ImageUploader.opts[:processor] = proc do |io, context|
+  # return unprocessed file(s)
+end
+```
+
+This also has the benefit of allowing you to test `ImageThumbnailsGenerator` in
+isolation.
+
+## Direct upload
+
+In case you're doing direct uploads to S3 on production and staging
+environments, in development and test you might want to just store files on
+the filesystem for speed.
+
+In that case you can swap out S3 for FileSystem, and the `direct_upload` app
+should still continue to work without any changes. This is because Shrine
+detects that you're using a storage which isn't an external service, and in
+that case the presign endpoint returns an URL to the upload route that's also
+provided by the `direct_upload` app mounted in your routes.
+
+[DatabaseCleaner]: https://github.com/DatabaseCleaner/database_cleaner
+[shrine-memory]: https://github.com/janko-m/shrine-memory
+[factory_girl]: https://github.com/thoughtbot/factory_girl
+[Capybara]: https://github.com/jnicklas/capybara
+[`#attach_file`]: http://www.rubydoc.info/github/jnicklas/capybara/master/Capybara/Node/Actions#attach_file-instance_method
+[Rack::Test]: https://github.com/brynary/rack-test
+[Rack::TestApp]: https://github.com/kwatch/rack-test_app