shrine 2.17.1 → 2.18.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7732350278c5d815ab0b52cbedf453707d77fae3dd4bb65b809d804dc79264bd
4
- data.tar.gz: 1c7d2767978fdc0a97be12fb3732e8cf3ff8fa440aa3fc88118fba4538b858b2
3
+ metadata.gz: e805d8626c94174d171e7bbddbcad967caf1d3278065d2177fa513c840bb2728
4
+ data.tar.gz: e00dca26185799041130da5d51cca0188eac2737ad0d9c8ccc23176e589c6d09
5
5
  SHA512:
6
- metadata.gz: 916e5df12daae5c50b1493a92593d6e030fbdabde239726edafb11eb7a8992a0d03479cd59ddab5ea720152281c4a042fd69812f8fb8b487a98943d0841cbd00
7
- data.tar.gz: 8f1b8eb9abf796b86696c27f1a2614ae98724b1b0a5a4a00506b35cacac75dcd9ae9d193d2f5c5d8aaeab6f3cbd2a2934aeb88ab51de368b14738e18e3e33735
6
+ metadata.gz: f504716d480883fe8b4d4b11c4c98dd68de9b0f0da6cb43be6f8b2ff46489b9f61b4598fc3f41ae50096fe822c94e74362a455e08270b54ce1f340a150f21f35
7
+ data.tar.gz: 9f25fd5c79c08badbdd5d9ba1c76a3e503fe8fcecd93a51626d12f2841cdeffc7aa8bcb4bf02f944c1dec89ecaf3e2981c18342c312ceaa330e54fb1bb8aa1fb
data/CHANGELOG.md CHANGED
@@ -1,4 +1,22 @@
1
- ## 2.17.1 (2019-05-15)
1
+ ## 2.18.0 (2019-06-24)
2
+
3
+ * `core` – Add `Shrine.upload` method as a shorthand for `Shrine.new(...).upload(...)` (@janko)
4
+
5
+ * `upload_endpoint` – Accept file uploads from Uppy's default `files[]` array (@janko)
6
+
7
+ * `core` – Add `Shrine::Attachment()` shorthand for `Shrine::Attachment.new` (@janko)
8
+
9
+ * `upload_endpoint` – Add `:url` option for adding uploaded file URL to response body (@janko)
10
+
11
+ * `s3` – Deprecate `:download` URL option over `:response_content_disposition` (@janko)
12
+
13
+ * `s3` – Remove backfilling `size` metadata when uploading IO objects of unknown size (@janko)
14
+
15
+ * `s3` – Deprecate `aws-sdk-s3` version less than 1.14.0 (@janko)
16
+
17
+ * `presign_endpoint` – Add `Shrine.presign_response` for handling presigns inside a custom controller (@janko)
18
+
19
+ * `upload_endpoint` – Add `Shrine.upload_response` for handling uploads inside a custom controller (@janko)
2
20
 
3
21
  * `rack_file` – Fix overriden `Attacher#assign` not accepting second argument (@janko)
4
22
 
data/README.md CHANGED
@@ -2,22 +2,22 @@
2
2
 
3
3
  Shrine is a toolkit for file attachments in Ruby applications. Some highlights:
4
4
 
5
- * **Modular design** – the [plugin system][plugin system] allows you to load only the functionality you need
6
- * **Memory friendly** – streaming uploads and downloads make it work great with large files
7
- * **Cloud storage** – store files on [disk][FileSystem], [AWS S3][S3], [Google Cloud][GCS], [Cloudinary] and others
5
+ * **Modular design** – the [plugin system] allows you to load only the functionality you need
6
+ * **Memory friendly** – streaming uploads and [downloads][Retrieving Uploads] make it work great with large files
7
+ * **Cloud storage** – store files on [disk][FileSystem], [AWS S3][S3], [Google Cloud][GCS], [Cloudinary] and [others][external]
8
8
  * **ORM integrations** – works with [Sequel][sequel plugin], [ActiveRecord][activerecord plugin], [Hanami::Model][hanami plugin] and [Mongoid][mongoid plugin]
9
9
  * **Flexible processing** – generate thumbnails [on upload] or [on-the-fly] using [ImageMagick][ImageProcessing::MiniMagick] or [libvips][ImageProcessing::Vips]
10
- * **Metadata validation** – [validate files][validation_helpers plugin] based on [extracted metadata][Extracting Metadata]
11
- * **Direct uploads** – upload asynchronously [to your app][upload_endpoint plugin] or [to the cloud][presign_endpoint plugin] using [Uppy]
12
- * **Resumable uploads** – make large file uploads [resumable][tus] by pointing [Uppy][uppy tus] to a [resumable endpoint][tus-ruby-server]
13
- * **Background jobs** – built-in support for [background processing][backgrounding plugin] that supports [any backgrounding library][backgrounding libraries]
10
+ * **Metadata validation** – [validate files][validation] based on [extracted metadata][metadata]
11
+ * **Direct uploads** – upload asynchronously [to your app][simple upload] or [to the cloud][presigned upload] using [Uppy]
12
+ * **Resumable uploads** – make large file uploads [resumable][resumable upload] on [S3][uppy-s3_multipart] or [tus][tus-ruby-server]
13
+ * **Background jobs** – built-in support for [background processing][backgrounding] that supports [any backgrounding library][Backgrounding Libraries]
14
14
 
15
15
  If you're curious how it compares to other file attachment libraries, see the [Advantages of Shrine].
16
16
 
17
17
  ## Resources
18
18
 
19
19
  | Resource | URL |
20
- | :------- | :--- |
20
+ | :---------------- | :----------------------------------------------------------------------------------------- |
21
21
  | Website | [shrinerb.com](https://shrinerb.com) |
22
22
  | Demo code | [Roda][roda demo] / [Rails][rails demo] |
23
23
  | Source | [github.com/shrinerb/shrine](https://github.com/shrinerb/shrine) |
@@ -25,6 +25,33 @@ If you're curious how it compares to other file attachment libraries, see the [A
25
25
  | Bugs | [github.com/shrinerb/shrine/issues](https://github.com/shrinerb/shrine/issues) |
26
26
  | Help & Discussion | [groups.google.com/group/ruby-shrine](https://groups.google.com/forum/#!forum/ruby-shrine) |
27
27
 
28
+ ## Contents
29
+
30
+ * [Quick start](#quick-start)
31
+ * [Storage](#storage)
32
+ * [Uploader](#uploader)
33
+ - [Uploading](#uploading)
34
+ - [IO abstraction](#io-abstraction)
35
+ * [Uploaded file](#uploaded-file)
36
+ * [Attachment](#attachment)
37
+ * [Attacher](#attacher)
38
+ * [Plugin system](#plugin-system)
39
+ * [Metadata](#metadata)
40
+ * [MIME type](#mime-type)
41
+ * [Other metadata](#other-metadata)
42
+ * [Processing](#processing)
43
+ * [Processing on upload](#processing-on-upload)
44
+ * [Processing on-the-fly](#processing-on-the-fly)
45
+ * [Validation](#validation)
46
+ * [Location](#location)
47
+ * [Direct uploads](#direct-uploads)
48
+ - [Simple direct upload](#simple-direct-upload)
49
+ - [Presigned direct upload](#presigned-direct-upload)
50
+ - [Resumable direct upload](#resumable-direct-upload)
51
+ * [Backgrounding](#backgrounding)
52
+ * [Clearing cache](#clearing-cache)
53
+ * [Settings](#settings)
54
+
28
55
  ## Quick start
29
56
 
30
57
  Add Shrine to the Gemfile and write an initializer which sets up the storage and
@@ -55,16 +82,30 @@ migration that adds an `<attachment>_data` text or JSON column, which Shrine
55
82
  will use to store all information about the attachment:
56
83
 
57
84
  ```rb
58
- Sequel.migration do # class AddImageDataToPhotos < ActiveRecord::Migration
59
- change do # def change
60
- add_column :photos, :image_data, :text # add_column :photos, :image_data, :text
61
- end # end
62
- end # end
85
+ Sequel.migration do
86
+ change do
87
+ add_column :photos, :image_data, :text
88
+ end
89
+ end
90
+ ```
91
+
92
+ In Rails with Active Record the migration would look similar:
93
+
94
+ ```sh
95
+ $ rails generate migration add_image_data_to_photos image_data:text
96
+ ```
97
+ ```rb
98
+ class AddImageDataToPhotos < ActiveRecord::Migration
99
+ def change
100
+ add_column :photos, :image_data, :text
101
+ end
102
+ end
63
103
  ```
64
104
 
65
105
  Now you can create an uploader class for the type of files you want to upload,
66
106
  and add a virtual attribute for handling attachments using this uploader to
67
- your model:
107
+ your model. If you do not care about adding plugins or additional processing,
108
+ you can use `Shrine::Attachment`.
68
109
 
69
110
  ```rb
70
111
  class ImageUploader < Shrine
@@ -80,24 +121,17 @@ end
80
121
 
81
122
  Let's now add the form fields which will use this virtual attribute. We need
82
123
  (1) a file field for choosing files, and (2) a hidden field for retaining the
83
- uploaded file in case of validation errors and for potential [direct
84
- uploads][direct S3 uploads guide].
124
+ uploaded file in case of validation errors and for potential [direct uploads].
85
125
 
86
126
  ```rb
87
- # with Forme:
88
- form @photo, action: "/photos", enctype: "multipart/form-data" do |f|
89
- f.input :image, type: :hidden, value: @photo.cached_image_data
90
- f.input :image, type: :file
91
- f.button "Create"
92
- end
93
-
94
127
  # with Rails form builder:
95
128
  form_for @photo do |f|
96
129
  f.hidden_field :image, value: @photo.cached_image_data
97
130
  f.file_field :image
98
131
  f.submit
99
132
  end
100
-
133
+ ```
134
+ ```rb
101
135
  # with Simple Form:
102
136
  simple_form_for @photo do |f|
103
137
  f.input :image, as: :hidden, input_html: { value: @photo.cached_image_data }
@@ -105,17 +139,41 @@ simple_form_for @photo do |f|
105
139
  f.button :submit
106
140
  end
107
141
  ```
142
+ ```rb
143
+ # with Forme:
144
+ form @photo, action: "/photos", enctype: "multipart/form-data" do |f|
145
+ f.input :image, type: :hidden, value: @photo.cached_image_data
146
+ f.input :image, type: :file
147
+ f.button "Create"
148
+ end
149
+ ```
108
150
 
109
151
  Note that the file field needs to go *after* the hidden field, so that
110
152
  selecting a new file can always override the cached file in the hidden field.
111
153
  Also notice the `enctype="multipart/form-data"` HTML attribute, which is
112
- required for submitting files through the form; the Rails form builder
113
- will automatically generate this for you.
154
+ required for submitting files through the form (the Rails form builder
155
+ will automatically generate this for you).
156
+
157
+ When the form is submitted, in your router/controller you can assign the file
158
+ from request params to the attachment attribute on the model.
159
+
160
+ ```rb
161
+ # In Rails:
162
+ class PhotosController < ApplicationController
163
+ def create
164
+ Photo.create(photo_params)
165
+ # ...
166
+ end
114
167
 
115
- Now in your router/controller the attachment request parameter can be assigned
116
- to the model like any other attribute:
168
+ private
117
169
 
170
+ def photo_params
171
+ params.require(:photo).permit(:image)
172
+ end
173
+ end
174
+ ```
118
175
  ```rb
176
+ # In Sinatra:
119
177
  post "/photos" do
120
178
  Photo.create(params[:photo])
121
179
  # ...
@@ -125,29 +183,38 @@ end
125
183
  Once a file is uploaded and attached to the record, you can retrieve a URL to
126
184
  the uploaded file with `#<attachment>_url` and display it on the page:
127
185
 
128
- ```rb
129
- image_tag @photo.image_url
186
+ ```erb
187
+ <!-- In Rails: -->
188
+ <%= image_tag @photo.image_url %>
189
+ ```
190
+ ```erb
191
+ <!-- In HTML: -->
192
+ <img src="<%= @photo.image_url %>" />
130
193
  ```
131
194
 
132
195
  ## Storage
133
196
 
134
- A "storage" in Shrine is an object responsible for managing files on a specific
135
- storage service (disk, AWS S3, Google Cloud etc), which implements a generic
136
- method interface. Storages are configured directly and registered under a name
137
- in `Shrine.storages`, so that they can later be used by uploaders.
197
+ A "storage" in Shrine is an object that encapsulates communication with a
198
+ specific storage service, by implementing a common public interface. Storage
199
+ instances are registered under an identifier in `Shrine.storages`, so that they
200
+ can later by used by [uploaders][uploader].
201
+
202
+ Previously we've shown the [FileSystem] storage which saves files to disk, but
203
+ Shrine also ships with [S3] storage which stores files on [AWS S3] (or any
204
+ S3-compatible service such as [DigitalOcean Spaces] or [MinIO]).
138
205
 
139
206
  ```rb
140
207
  # Gemfile
141
- gem "aws-sdk-s3", "~> 1.2" # for AWS S3 storage
208
+ gem "aws-sdk-s3", "~> 1.14" # for AWS S3 storage
142
209
  ```
143
210
  ```rb
144
211
  require "shrine/storage/s3"
145
212
 
146
213
  s3_options = {
147
- bucket: "my-bucket", # required
148
- access_key_id: "abc",
149
- secret_access_key: "xyz",
150
- region: "my-region",
214
+ bucket: "<YOUR BUCKET>", # required
215
+ access_key_id: "<YOUR ACCESS KEY ID>",
216
+ secret_access_key: "<YOUR SECRET ACCESS KEY>",
217
+ region: "<YOUR REGION>",
151
218
  }
152
219
 
153
220
  Shrine.storages = {
@@ -156,129 +223,125 @@ Shrine.storages = {
156
223
  }
157
224
  ```
158
225
 
159
- The above example sets up AWS S3 storage both for temporary and permanent
160
- storage, which is suitable for [direct uploads][direct S3 uploads guide]. The
161
- `:cache` and `:store` names are special only in terms that the attacher will
162
- automatically pick them up, but you can also register more storages under
163
- different names.
226
+ The above example sets up S3 for both temporary and permanent storage, which is
227
+ suitable for [direct uploads][Direct Uploads to S3]. The `:cache` and
228
+ `:store` names are special only in terms that the [attacher] will automatically
229
+ pick them up, you can also register more storage objects under different names.
164
230
 
165
- Shrine ships with [FileSystem] and [S3] storage, take a look at their
166
- documentation for more details on various features they support. There are
167
- [many more Shrine storages][external storages] provided by external gems, and
168
- you can also [create your own storage][creating storage].
231
+ See the [FileSystem] and [S3] storage docs for more details. There are [many
232
+ more Shrine storages][external] provided by external gems, and you can also
233
+ [create your own storage][Creating Storages].
169
234
 
170
235
  ## Uploader
171
236
 
172
- Uploaders are subclasses of `Shrine`, and are essentially wrappers around
173
- storages. They perform common tasks around upload that aren't related to a
237
+ Uploaders are subclasses of `Shrine`, and they wrap the actual upload to the
238
+ storage. They perform common tasks around upload that aren't related to a
174
239
  particular storage.
175
240
 
176
241
  ```rb
177
- class ImageUploader < Shrine
242
+ class MyUploader < Shrine
178
243
  # image attachent logic
179
244
  end
180
245
  ```
181
- ```rb
182
- uploader = ImageUploader.new(:store)
183
- uploader #=> uploader for storage registered under `:store`
184
- ```
185
246
 
186
247
  It's common to create an uploader for each type of file that you want to handle
187
- (image, video, audio, document etc), but really you can organize them in any way
188
- you like.
248
+ (`ImageUploader`, `VideoUploader`, `AudioUploader` etc), but really you can
249
+ organize them in any way you like.
189
250
 
190
251
  ### Uploading
191
252
 
192
- The main method of the uploader is `#upload`, which takes an IO-like object on
193
- the input, and returns a representation of the uploaded file on the output.
253
+ The main method of the uploader is `#upload`, which takes an [IO-like
254
+ object][io abstraction] and a storage identifier on the input, and returns a
255
+ representation of the [uploaded file] on the output.
194
256
 
195
257
  ```rb
196
- uploaded_file = uploader.upload(file)
197
- uploaded_file #=> #<Shrine::UploadedFile>
258
+ MyUploader.upload(file, :store) #=> #<Shrine::UploadedFile>
259
+ ```
260
+
261
+ Internally this instantiates the uploader with the storage and calls `#upload`
262
+ on it:
263
+
264
+ ```rb
265
+ uploader = MyUploader.new(:store)
266
+ uploader.upload(file) #=> #<Shrine::UploadedFile>
198
267
  ```
199
268
 
200
269
  Some of the tasks performed by `#upload` include:
201
270
 
202
- * file processing (if defined)
203
- * extracting metadata
204
- * generating location
205
- * uploading (this is where the storage is called)
271
+ * any defined [file processing][on upload]
272
+ * extracting [metadata]
273
+ * generating [location]
274
+ * uploading (this is where the [storage] is called)
206
275
  * closing the uploaded file
207
276
 
208
- Additional upload options can be passed via `:upload_options`, and they will be
209
- forwarded directly to `Storage#upload` (see the documentation of your storage
210
- for the list of available options):
277
+ The second argument is a `context` hash which is forwarded to places like
278
+ metadata extraction and location generation, but it has a few special options:
211
279
 
212
280
  ```rb
213
- uploader.upload(file, upload_options: { acl: "public-read" })
281
+ uploader.upload(io, metadata: { "foo" => "bar" }) # add metadata
282
+ uploader.upload(io, location: "path/to/file") # specify custom location
283
+ uploader.upload(io, upload_options: { acl: "public-read" }) # add options to Storage#upload
214
284
  ```
215
285
 
216
286
  ### IO abstraction
217
287
 
218
- Shrine is able to upload any IO-like object that responds to `#read`,
219
- `#rewind`, `#eof?` and `#close`. This includes built-in IO and IO-like objects
220
- like File, Tempfile and StringIO.
288
+ Shrine is able to upload any IO-like object that implement methods [`#read`],
289
+ [`#rewind`], [`#eof?`] and [`#close`] whose behaviour matches the [`IO`] class.
290
+ This includes built-in IO and IO-like objects like File, Tempfile and StringIO.
221
291
 
222
- When a file is uploaded to a Rails app, it will be represented by an
223
- ActionDispatch::Http::UploadedFile object in the params. This is also an
292
+ When a file is uploaded to a Rails app, in request params it will be
293
+ represented by an `ActionDispatch::Http::UploadedFile` object, which is also an
224
294
  IO-like object accepted by Shrine. In other Rack applications the uploaded file
225
- will be represented as a Hash, but it can still be attached when `rack_file`
226
- plugin is loaded.
295
+ will be represented as a Hash, but it can be converted into an IO-like object
296
+ with the [`rack_file`][rack_file plugin] plugin.
227
297
 
228
- Here are some examples of IO objects that can be uploaded:
298
+ Here are some examples of various IO-like objects that can be uploaded:
229
299
 
230
300
  ```rb
231
- uploader.upload File.open("/path/to/file", binmode: true) # upload from disk
232
- uploader.upload StringIO.new("file content") # upload from memory
233
- uploader.upload ActionDispatch::Http::UploadedFile.new # upload from Rails controller
234
- uploader.upload Shrine.rack_file({ tempfile: tempfile }) # upload from Rack controller
235
- uploader.upload Rack::Test::UploadedFile.new # upload from rack-test
236
- uploader.upload Down.open("https://example.org/file") # upload from internet
301
+ uploader.upload File.open("/path/to/file", binmode: true) # upload from disk
302
+ uploader.upload StringIO.new("file content") # upload from memory
303
+ uploader.upload ActionDispatch::Http::UploadedFile.new(...) # upload from Rails controller
304
+ uploader.upload Shrine.rack_file({ tempfile: tempfile }) # upload from Rack controller
305
+ uploader.upload Rack::Test::UploadedFile.new(...) # upload from rack-test
306
+ uploader.upload Down.open("https://example.org/file") # upload from internet
307
+ uploader.upload Shrine::UploadedFile.new(...) # upload from Shrine storage
237
308
  ```
238
309
 
239
- `Shrine::UploadedFile`, the object returned after upload, is itself an IO-like
240
- object as well. This makes it trivial to reupload a file from one storage to
241
- another, and this is used by the attacher to reupload a file stored on
242
- temporary storage to permanent storage.
243
-
244
310
  ## Uploaded file
245
311
 
246
- The `Shrine::UploadedFile` object represents the file that was uploaded to the
312
+ The `Shrine::UploadedFile` object represents the file that was uploaded to a
247
313
  storage, and it's what's returned from `Shrine#upload` or when retrieving a
248
- record attachment. It contains the following information:
314
+ record [attachment]. It contains the following information:
249
315
 
250
- * `storage` – identifier of the storage the file was uploaded to
251
- * `id` – location of the file on the storage
252
- * `metadata` file metadata that was extracted before upload
316
+ | Key | Description |
317
+ | :------- | :---------- |
318
+ | `id` | location of the file on the storage |
319
+ | `storage` | identifier of the storage the file was uploaded to |
320
+ | `metadata` | file [metadata] that was extracted before upload |
253
321
 
254
322
  ```rb
255
323
  uploaded_file = uploader.upload(file)
324
+ uploaded_file.data #=> {"id"=>"949sdjg834.jpg","storage"=>"store","metadata"=>{...}}
256
325
 
257
326
  uploaded_file.id #=> "949sdjg834.jpg"
258
327
  uploaded_file.storage #=> #<Shrine::Storage::FileSystem>
259
328
  uploaded_file.metadata #=> {...}
260
-
261
- # It can be serialized into JSON and saved to a database column
262
- uploaded_file.to_json #=> '{"id":"949sdjg834.jpg","storage":"store","metadata":{...}}'
263
329
  ```
264
330
 
265
331
  It comes with many convenient methods that delegate to the storage:
266
332
 
267
333
  ```rb
268
- uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/949sdjg834.jpg"
269
- uploaded_file.open # opens the uploaded file
270
- uploaded_file.download #=> #<File:/var/folders/.../20180302-33119-1h1vjbq.jpg>
271
- uploaded_file.stream(destination) # streams uploaded content into a writable destination
272
- uploaded_file.exists? #=> true
273
- uploaded_file.delete # deletes the file from the storage
274
-
275
- # open/download the uploaded file for the duration of the block
276
- uploaded_file.open { |io| io.read }
277
- uploaded_file.download { |tempfile| tempfile.read }
334
+ uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/949sdjg834.jpg"
335
+ uploaded_file.open { |io| ... } # opens the uploaded file stream
336
+ uploaded_file.download { |file| ... } # downloads the uploaded file to disk
337
+ uploaded_file.stream(destination) # streams uploaded content into a writable destination
338
+ uploaded_file.exists? #=> true
339
+ uploaded_file.delete # deletes the uploaded file from the storage
278
340
  ```
279
341
 
280
- It also implements the IO-like interface that conforms to Shrine's IO
281
- abstraction, which allows it to be uploaded again to other storages.
342
+ It also implements the IO-like interface that conforms to Shrine's [IO
343
+ abstraction][io abstraction], which allows it to be uploaded again to other
344
+ storages.
282
345
 
283
346
  ```rb
284
347
  uploaded_file.read # returns content of the uploaded file
@@ -287,19 +350,20 @@ uploaded_file.rewind # rewinds the IO
287
350
  uploaded_file.close # closes the IO
288
351
  ```
289
352
 
290
- For more details on these `Shrine::UploadedFile` methods, see the [Retrieving
291
- Uploads] guide.
353
+ For more details, see the [Retrieving Uploads] guide and
354
+ [`Shrine::UploadedFile`] API docs.
292
355
 
293
356
  ## Attachment
294
357
 
295
- Storages, uploaders, and uploaded file objects are the main components for
296
- managing files. Since most often you also want to *attach* the uploaded files
297
- to database records, Shrine comes with a high-level attachment interface, which
298
- uses these components internally.
358
+ Storages, uploaders, and uploaded file objects are Shrine's foundational
359
+ components. To help you actually attach uploaded files to database records in
360
+ your application, Shrine comes with a high-level attachment interface built on
361
+ top of these components.
299
362
 
300
- Usually you're using an ORM for saving database records, in which case you can
301
- load an additional plugin to automatically tie the attached files to record
302
- lifecycle. But you can also use Shrine just with plain models.
363
+ There are plugins for hooking into most database libraries, and in case of
364
+ ActiveRecord and Sequel the plugin will automatically tie the attached files to
365
+ records' lifecycles. But you can also use Shrine just with plain old Ruby
366
+ objects.
303
367
 
304
368
  ```rb
305
369
  Shrine.plugin :sequel # :activerecord
@@ -308,23 +372,25 @@ Shrine.plugin :sequel # :activerecord
308
372
  ```rb
309
373
  class Photo < Sequel::Model # ActiveRecord::Base
310
374
  include ImageUploader::Attachment.new(:image) #
311
- include ImageUploader.attachment(:image) # these are all equivalent
375
+ include ImageUploader::Attachment(:image) # these are all equivalent
312
376
  include ImageUploader[:image] #
313
377
  end
314
378
  ```
315
379
 
316
- You can choose whichever of these three syntaxes you prefer. Either of these
380
+ You can choose whichever of these syntaxes you prefer. Either of these
317
381
  will create a `Shrine::Attachment` module with attachment methods for the
318
382
  specified attribute, which then get added to your model when you include it:
319
383
 
320
- * `#image=` – uploads the file to temporary storage and serializes the result into `image_data`
321
- * `#image` – returns `Shrine::UploadedFile` instantiated from `image_data`
322
- * `#image_url` calls `url` on the attachment if it's present, otherwise returns nil
323
- * `#image_attacher` returns instance of `Shrine::Attacher` which handles the attaching
384
+ | Method | Description |
385
+ | :----- | :---------- |
386
+ | `#image=` | uploads the file to temporary storage and serializes the result into `image_data` |
387
+ | `#image` | returns [`Shrine::UploadedFile`][uploaded file] instantiated from `image_data` |
388
+ | `#image_url` | calls `url` on the attachment if it's present, otherwise returns nil |
389
+ | `#image_attacher` | returns instance of [`Shrine::Attacher`][attacher] which handles the attaching |
324
390
 
325
- The ORM plugin that we loaded adds appropriate callbacks, so when record is
326
- saved the attachment is uploaded to permanent storage, and when record is
327
- deleted the attachment is deleted as well.
391
+ The ORM plugin that we loaded adds appropriate callbacks. For example, saving
392
+ the record uploads the attachment to permanent storage, while deleting the
393
+ record deletes the attachment.
328
394
 
329
395
  ```rb
330
396
  # no file is attached
@@ -352,23 +418,13 @@ attachment will get deleted when the record gets saved.
352
418
 
353
419
  ```rb
354
420
  photo.update(image: new_file) # changes the attachment and deletes previous
355
- # or
356
421
  photo.update(image: nil) # removes the attachment and deletes previous
357
422
  ```
358
423
 
359
- In addition to assigning raw files, you can also assign a JSON representation
360
- of files that are already uploaded to the temporary storage. This allows Shrine
361
- to retain cached files in case of validation errors and handle [direct uploads]
362
- via the hidden form field.
363
-
364
- ```rb
365
- photo.image = '{"id":"9260ea09d8effd.jpg","storage":"cache","metadata":{...}}'
366
- ```
367
-
368
424
  ## Attacher
369
425
 
370
- The model attachment attributes and callbacks just delegate the behaviour
371
- to their underlying `Shrine::Attacher` object.
426
+ The model attachment attributes and callbacks added by `Shrine::Attachment`
427
+ just delegate the behaviour to their underlying `Shrine::Attacher` object.
372
428
 
373
429
  ```rb
374
430
  photo.image_attacher #=> #<Shrine::Attacher>
@@ -384,40 +440,13 @@ attacher.get # equivalent to `photo.image`
384
440
  attacher.url # equivalent to `photo.image_url`
385
441
  ```
386
442
 
387
- The attacher is what drives attaching files to model instances, and it functions
388
- independently from models' attachment interface. This means that you can use it
389
- as an alternative, in case you prefer not to add additional attributes to the
390
- model, or prefer explicitness over callbacks. It's also useful when you need
391
- something more advanced which isn't available through the attachment
392
- attributes.
393
-
394
- The `Shrine::Attacher` by default uses `:cache` for temporary and `:store` for
395
- permanent storage, but you can specify a different storage:
396
-
397
- ```rb
398
- ImageUploader::Attacher.new(photo, :image, cache: :other_cache, store: :other_store)
399
-
400
- # OR
401
-
402
- photo.image_attacher(cache: :other_cache, store: :other_store)
403
- photo.image = file # uploads to :other_cache storage
404
- photo.save # promotes to :other_store storage
405
- ```
406
-
407
- You can also skip the temporary storage altogether and upload files directly to
408
- the primary storage:
409
-
410
- ```rb
411
- uploaded_file = attacher.store!(file) # upload file directly to permanent storage
412
- attacher.set(uploaded_file) # attach the uploaded file
413
- ```
414
-
415
- Whenever the attacher uploads or deletes files, it sends a `context` hash
416
- which includes `:record`, `:name`, and `:action` keys, so that you can perform
417
- processing or generate location differently depending on this information. See
418
- "Context" section for more details.
443
+ The attacher is what drives attaching files to model instances; you can use it
444
+ as a more explicit alternative to models' attachment interface, or simply when
445
+ you need something that's not available through the attachment methods.
419
446
 
420
- For more information about `Shrine::Attacher`, see the [Using Attacher] guide.
447
+ You can do things such as change the temporary and permanent storage the
448
+ attacher uses, or upload files directly to permanent storage. See the [Using
449
+ Attacher] guide for more details.
421
450
 
422
451
  ## Plugin system
423
452
 
@@ -441,7 +470,7 @@ end
441
470
  ```
442
471
 
443
472
  If you want to extend Shrine functionality with custom behaviour, you can also
444
- [create your own plugin][creating plugin].
473
+ [create your own plugin][Creating Plugins].
445
474
 
446
475
  ## Metadata
447
476
 
@@ -463,23 +492,12 @@ uploaded_file.mime_type #=> "video/mp4"
463
492
  uploaded_file.size #=> 345993
464
493
  ```
465
494
 
466
- By default these values are determined from the following attributes on the IO
467
- object:
468
-
469
- * `filename` – `io.original_filename` or `io.path`
470
- * `mime_type` – `io.content_type`
471
- * `size` – `io.size`
472
-
473
495
  ### MIME type
474
496
 
475
- By default `mime_type` will be inherited from `#content_type` attribute of the
476
- uploaded file, which is set from the `Content-Type` request header. However,
477
- this header is determined by the browser solely based on the file extension.
478
- This means that by default Shrine's `mime_type` is *not guaranteed* to hold
479
- the actual MIME type of the file.
480
-
481
- To remedy that, you can load the `determine_mime_type` plugin, which will make
482
- Shrine extract the MIME type from *file content*.
497
+ By default, `mime_type` metadata will be inherited from the `#content_type`
498
+ attribute of the uploaded file, which is generally not secure and will trigger
499
+ a warning. You can load the [`determine_mime_type`][determine_mime_type plugin]
500
+ plugin to have MIME type extracted from file *content* instead.
483
501
 
484
502
  ```rb
485
503
  Shrine.plugin :determine_mime_type
@@ -490,66 +508,42 @@ photo.image.mime_type #=> "text/x-php"
490
508
  ```
491
509
 
492
510
  By the default the UNIX [`file`] utility is used to determine the MIME type,
493
- but you can also choose a different analyzer see the plugin documentation for
494
- more details.
511
+ but you can also choose a different analyzer, see the
512
+ [`determine_mime_type`][determine_mime_type plugin] docs for more details.
495
513
 
496
514
  ### Other metadata
497
515
 
498
- In addition to `size`, `filename`, and `mime_type`, you can also extract image
499
- dimensions using the `store_dimensions` plugin, as well as any custom metadata
500
- using the `add_metadata` plugin. Check out the [Extracting Metadata] guide for
501
- more details.
502
-
503
- Note that you can also manually override extracted metadata by passing the
504
- `:metadata` option to `Shrine#upload`:
505
-
506
- ```rb
507
- uploaded_file = uploader.upload(file, metadata: { "filename" => "Matrix[1999].mp4", "foo" => "bar" })
508
- uploaded_file.original_filename #=> "Matrix[1999].mp4"
509
- uploaded_file.metadata["foo"] #=> "bar"
510
- ```
516
+ In addition to basic metadata, you can also extract [image
517
+ dimensions][store_dimensions plugin], calculate [signatures][signature plugin],
518
+ and in general extract any [custom metadata][add_metadata plugin]. Check out
519
+ the [Extracting Metadata] guide for more details.
511
520
 
512
521
  ## Processing
513
522
 
514
- Shrine allows you to processing attached files either "on upload" or
523
+ Shrine allows you to process attached files either "on upload" or
515
524
  "on-the-fly". For example, if your app is accepting image uploads, you can
516
525
  generate a pre-defined set of of thumbnails as soon as the image is attached to
517
526
  a record ("on upload"), or you can generate necessary thumbnails dynamically as
518
527
  they're needed ("on-the-fly").
519
528
 
520
- In both cases, for image processing you can use the **[ImageProcessing]** gem.
521
- It provides a convenient unified API for processing with [ImageMagick] or
522
- [libvips]. Here is an example of generating a thumbnail with ImageProcessing:
529
+ For image processing it's recommended to use the **[ImageProcessing]** gem,
530
+ which is a high-level wrapper for processing with [ImageMagick] (via
531
+ [MiniMagick]) or [libvips] (via [ruby-vips]).
532
+
533
+ ### Processing on upload
534
+
535
+ For processing "on upload", you can intercept when a cached file is being
536
+ uploaded to permanent storage, and perform any file processing you might want.
537
+ The [`processing`][processing plugin] plugin provides the promotion hook, while
538
+ the [`versions`][versions plugin] plugin enables handling a hash of versions.
523
539
 
524
540
  ```sh
525
541
  $ brew install imagemagick
526
542
  ```
527
543
  ```rb
528
544
  # Gemfile
529
- gem "image_processing", "~> 1.0"
545
+ gem "image_processing", "~> 1.2"
530
546
  ```
531
- ```rb
532
- require "image/mini_magick"
533
-
534
- thumbnail = ImageProcessing::MiniMagick
535
- .source(original_image)
536
- .resize_to_limit!(600, 400)
537
-
538
- thumbnail #=> #<Tempfile:...> (thumbnail limited to 600x400)
539
- ```
540
-
541
- ### On upload
542
-
543
- Shrine allows you intercept when a cached file is being uploaded to permanent
544
- storage, and perform any file processing you might want. The result of
545
- processing can also be multiple files, such as thumbnails of various
546
- dimensions. This processing can additionaly be delayed into a [background
547
- job](#backgrounding).
548
-
549
- The promotion hook is provided by the `processing` plugin, while the ability
550
- to save multiple files is provided by the `versions` plugin. Let's set up our
551
- uploader to generate some thumbnails from the attached image:
552
-
553
547
  ```rb
554
548
  require "image_processing/mini_magick"
555
549
 
@@ -574,21 +568,12 @@ class ImageUploader < Shrine
574
568
  end
575
569
  ```
576
570
 
577
- After these files have been uploaded, their data will all be saved to the
578
- `<attachment>_data` column. The attachment getter will then read them as a Hash
579
- of `Shrine::UploadedFile` objects.
571
+ After the files are uploaded, their data is saved into the `<attachment>_data`
572
+ column, and the attachment getter will read them as a Hash of
573
+ [`Shrine::UploadedFile`][uploaded file] objects.
580
574
 
581
575
  ```rb
582
- photo = Photo.create(image: image)
583
-
584
- photo.image_data #=>
585
- # '{
586
- # "original": {"id":"9sd84.jpg", "storage":"store", "metadata":{...}},
587
- # "large": {"id":"lg043.jpg", "storage":"store", "metadata":{...}},
588
- # "medium": {"id":"kd9fk.jpg", "storage":"store", "metadata":{...}},
589
- # "small": {"id":"932fl.jpg", "storage":"store", "metadata":{...}}
590
- # }'
591
-
576
+ photo = Photo.create(image: file) # processing is triggered
592
577
  photo.image #=>
593
578
  # {
594
579
  # :original => #<Shrine::UploadedFile @data={"id"=>"9sd84.jpg", ...}>,
@@ -601,32 +586,46 @@ photo.image[:medium] #=> #<Shrine::UploadedFile>
601
586
  photo.image[:medium].url #=> "/uploads/store/lg043.jpg"
602
587
  photo.image[:medium].size #=> 5825949
603
588
  photo.image[:medium].mime_type #=> "image/jpeg"
604
- ```
605
-
606
- The `versions` plugin also expands `#<attachment>_url` to accept version names:
607
589
 
608
- ```rb
609
- photo.image_url(:large) #=> "https://..."
590
+ photo.image_url(:large) # returns version URL with fallbacks in case version is missing
610
591
  ```
611
592
 
612
- For more details see the [File Processing] guide.
593
+ By default processing is executed synchronously, but you can choose to delay it
594
+ into a [background job][backgrounding]. You can also do any other type of file
595
+ processing you want, see the [File Processing] guide for more details.
613
596
 
614
- ### On-the-fly
597
+ ### Processing on-the-fly
615
598
 
616
599
  On-the-fly processing is provided by the
617
- [`derivation_endpoint`][derivation_endpoint plugin] plugin. It provides a
618
- mountable Rack app, which on request will call the processing we defined.
600
+ [`derivation_endpoint`][derivation_endpoint plugin] plugin. It comes with a
601
+ [mountable][Mounting Endpoints] Rack app which applies processing on request
602
+ and returns processed files.
619
603
 
620
- We start by loading the plugin with a secret key and a path prefix to where
621
- we'll mount the Rack app, and defining a "derivation" we want the app to call:
604
+ To set it up, we mount the Rack app in our router on a chosen path prefix,
605
+ configure the plugin with a secret key and that path prefix, and define
606
+ processing we want to perform:
622
607
 
608
+ ```sh
609
+ $ brew install imagemagick
610
+ ```
611
+ ```rb
612
+ # Gemfile
613
+ gem "image_processing", "~> 1.2"
614
+ ```
615
+ ```rb
616
+ # config/routes.rb (Rails)
617
+ Rails.application.routes.draw do
618
+ # ...
619
+ mount ImageUploader.derivation_endpoint => "/derivations/image"
620
+ end
621
+ ```
623
622
  ```rb
624
623
  require "image_processing/mini_magick"
625
624
 
626
625
  class ImageUploader < Shrine
627
626
  plugin :derivation_endpoint,
628
627
  secret_key: "<YOUR SECRET KEY>",
629
- prefix: "derivations/image"
628
+ prefix: "derivations/image" # needs to match the mount point in routes
630
629
 
631
630
  derivation :thumbnail do |file, width, height|
632
631
  ImageProcessing::MiniMagick
@@ -636,62 +635,23 @@ class ImageUploader < Shrine
636
635
  end
637
636
  ```
638
637
 
639
- Then we can mount the Rack app into our router (for web frameworks other than
640
- Rails see [Mounting Endpoints] wiki):
641
-
642
- ```rb
643
- # config/routes.rb (Rails)
644
- Rails.application.routes.draw do
645
- mount ImageUploader.derivation_endpoint => "derivations/image"
646
- end
647
- ```
648
-
649
- Now we can generate URLs from attached files that on request will call the
650
- processing we defined:
638
+ Now we can generate URLs from attached files that will perform the desired
639
+ processing:
651
640
 
652
641
  ```rb
653
642
  photo.image.derivation_url(:thumbnail, "600", "400")
654
643
  #=> "/derivations/image/thumbnail/600/400/eyJpZCI6ImZvbyIsInN0b3JhZ2UiOiJzdG9yZSJ9?signature=..."
655
644
  ```
656
645
 
657
- The `derivation_endpoint` plugin is highly customizable, for more details see
658
- its [documentation][derivation_endpoint plugin].
659
-
660
- ## Context
661
-
662
- The `#upload` (and `#delete`) methods accept a hash of options as the second
663
- argument, which is forwarded down the chain and be available for processing,
664
- extracting metadata and generating location.
665
-
666
- ```rb
667
- uploader.upload(file, { foo: "bar" }) # context hash is forwarded to all tasks around upload
668
- ```
669
-
670
- Some options are actually recognized by Shrine (such as `:location`,
671
- `:upload_options`, and `:metadata`), some are added by plugins, and the rest are
672
- there just to provide additional context, for more flexibility in performing
673
- tasks and more descriptive logging.
674
-
675
- The attacher automatically includes additional `context` information for each
676
- upload and delete operation:
677
-
678
- * `context[:record]` – model instance where the file is attached
679
- * `context[:name]` – name of the attachment attribute on the model
680
- * `context[:action]` – identifier for the action being performed (`:cache`, `:store`, `:recache`, `:backup`, ...)
681
-
682
- ```rb
683
- class VideoUploader < Shrine
684
- process(:store) do |io, context|
685
- trim_video(io, 300) if context[:record].user.free_plan?
686
- end
687
- end
688
- ```
646
+ The on-the-fly processing feature is highly customizable, see the
647
+ [`derivation_endpoint`][derivation_endpoint plugin] plugin documentation for
648
+ more details.
689
649
 
690
650
  ## Validation
691
651
 
692
- Shrine can perform file validations for files assigned to the model. The
693
- validations are defined inside the `Attacher.validate` block, and you can load
694
- the `validation_helpers` plugin to get convenient file validation methods:
652
+ Shrine can perform file validations for files assigned to the model, with
653
+ [`validation_helpers`][validation_helpers plugin] plugin providing some common
654
+ validation methods:
695
655
 
696
656
  ```rb
697
657
  class DocumentUploader < Shrine
@@ -706,20 +666,21 @@ end
706
666
 
707
667
  ```rb
708
668
  user = User.new
709
- user.cv = File.open("cv.pdf")
669
+ user.cv = File.open("cv.pdf", "rb")
710
670
  user.valid? #=> false
711
671
  user.errors.to_hash #=> {:cv=>["is too large (max is 5 MB)"]}
712
672
  ```
713
673
 
714
- See the [File Validation] guide and [`validation_helpers`][validation_helpers
715
- plugin] plugin documentation for more details.
674
+ For more details, see the [File Validation] guide and
675
+ [`validation_helpers`][validation_helpers plugin] plugin docs.
716
676
 
717
677
  ## Location
718
678
 
719
- Before Shrine uploads a file, it generates a random location for it. By default
720
- the hierarchy is flat; all files are stored in the root directory of the
721
- storage. The `pretty_location` plugin provides a nice default hierarchy, but
722
- you can also override `#generate_location` with a custom implementation:
679
+ Shrine automatically generated random locations before uploading files. By
680
+ default the hierarchy is flat, meaning all files are stored in the root
681
+ directory of the storage. The [`pretty_location`][pretty_location plugin]
682
+ plugin provides a good default hierarchy, but you can also override
683
+ `#generate_location` with a custom implementation:
723
684
 
724
685
  ```rb
725
686
  class ImageUploader < Shrine
@@ -746,46 +707,23 @@ Note that there should always be a random component in the location, so that
746
707
  the ORM dirty tracking is detected properly. Inside `#generate_location` you
747
708
  can also access the extracted metadata through `context[:metadata]`.
748
709
 
749
- When uploading single files, it's possible to bypass `#generate_location` via
750
- the uploader, by specifying `:location`:
751
-
752
- ```rb
753
- uploader.upload(file, location: "some/specific/location.mp4")
754
- ```
755
-
756
710
  ## Direct uploads
757
711
 
758
- To really improve the user experience, it's recommended to start uploading the
759
- files asynchronously as soon they're selected. This way the UI is still
760
- responsive during upload, so the user can fill in other fields while the files
761
- are being uploaded, and if you display a progress bar they can see when the
762
- upload will finish.
763
-
764
- These asynchronous uploads will have to go to an endpoint separate from the one
765
- where the form is submitted. This can be an endpoint in your app, or an
766
- endpoint of a cloud service. In either case, the uploads should go to
767
- *temporary* storage (`:cache`), to ensure there won't be any orphan files in
768
- the primary storage (`:store`).
769
-
770
- Once files are uploaded on the client side, their data can be submitted to the
771
- server and attached to a record, just like with raw files. The only difference
772
- is that they won't be additionally uploaded to temporary storage on assignment,
773
- as they were already uploaded on the client side. Note that by default **Shrine
774
- won't extract metadata from directly uploaded files**, instead it will just copy
775
- metadata that was extacted on the client side; see [this section][metadata
776
- direct uploads] for the rationale and instructions on how to opt in.
777
-
778
- For handling client side uploads it's recommended to use **[Uppy]**. Uppy is a
779
- very flexible modern JavaScript file upload library, which happens to integrate
780
- nicely with Shrine.
712
+ To improve the user experience, it's recommended to upload files asynchronously
713
+ as soon as the user selects them. The direct uploads would go to temporary
714
+ storage, just like in the synchronous flow.
715
+
716
+ On the client side it's highly recommended to use **[Uppy]**, a very flexible
717
+ modern JavaScript file upload library that happens to integrate nicely with
718
+ Shrine.
781
719
 
782
720
  ### Simple direct upload
783
721
 
784
722
  The simplest approach is creating an upload endpoint in your app that will
785
- receive uploads and forward them to the specified storage. You can use the
786
- `upload_endpoint` Shrine plugin to create a Rack app that handles uploads,
787
- and mount it inside your application (for web frameworks other than Rails
788
- see [Mounting Endpoints] wiki).
723
+ forward uploads to the specified storage. The
724
+ [`upload_endpoint`][upload_endpoint plugin] Shrine plugin provides a
725
+ [mountable][Mounting Endpoints] Rack application that does that, and you can
726
+ combine it with Uppy's [XHR Upload][uppy xhr-upload] plugin:
789
727
 
790
728
  ```rb
791
729
  Shrine.plugin :upload_endpoint
@@ -793,45 +731,47 @@ Shrine.plugin :upload_endpoint
793
731
  ```rb
794
732
  # config/routes.rb (Rails)
795
733
  Rails.application.routes.draw do
796
- mount ImageUploader.upload_endpoint(:cache) => "images/upload"
734
+ # ...
735
+ mount ImageUploader.upload_endpoint(:cache) => "/images/upload" # POST /images/upload
797
736
  end
798
737
  ```
738
+ ```js
739
+ // upload.js
740
+ // ...
741
+ uppy.use(Uppy.XHRUpload, {
742
+ endpoint: '/images/upload'
743
+ })
799
744
 
800
- The above will add a `POST /images/upload` route to your app. You can now use
801
- Uppy's [XHR Upload][uppy xhr upload] plugin to upload selected files to this
802
- endpoint, and have the uploaded file data submitted to your app. The client
803
- side code for this will depend on your application, see [this
804
- walkthrough][direct uploads walkthrough] for an example of adding simple direct
805
- uploads from scratch.
806
-
807
- If you wanted to implement this enpdoint yourself, this is how it could roughly
808
- look like in Sinatra:
809
-
810
- ```rb
811
- Shrine.plugin :rack_file # only if not using Rails
745
+ uppy.on('upload-success', (file, response) => {
746
+ const uploadedFileData = JSON.stringify(response.body)
747
+ // ... add this data to your form or submit it to your app ...
748
+ })
812
749
  ```
813
- ```rb
814
- post "/images/upload" do
815
- uploader = ImageUploader.new(:cache)
816
- file = Shrine.rack_file(params["file"]) # only `params[:file]` in Rails
817
-
818
- uploaded_file = uploader.upload(file)
819
750
 
820
- json uploaded_file.data
821
- end
822
- ```
751
+ For adding simple direct uploads from scratch, see [this walkthrough][Adding
752
+ Direct App Uploads] (there is also the [Roda][roda demo] / [Rails][rails demo]
753
+ demo app).
823
754
 
824
755
  ### Presigned direct upload
825
756
 
826
- If you want to free your app from receiving file uploads, you can also upload
827
- files directly to the cloud (AWS S3, Google Cloud etc). In this flow the client
828
- is required to first fetch upload parameters from the server, and then use these
829
- parameters to make the upload.
757
+ For better performance, you can also upload files directly to your cloud
758
+ storage service (AWS S3, Google Cloud Storage etc). For this, your temporary
759
+ storage needs to be your cloud service:
830
760
 
831
- You can use the `presign_endpoint` Shrine plugin to create a Rack app that
832
- generates these upload parameters (provided that the underlying storage
833
- implements `#presign`), and mount it inside your application (for web
834
- frameworks other than Rails see [Mounting Endpoints] wiki):
761
+ ```rb
762
+ require "shrine/storage/s3"
763
+
764
+ Shrine.storages = {
765
+ cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
766
+ store: Shrine::Storage::S3.new(**s3_options)
767
+ }
768
+ ```
769
+
770
+ In this flow, the client needs to first fetch upload parameters from the
771
+ server, and then use these parameters for the upload to the cloud service. The
772
+ [`presign_endpoint`][presign_endpoint plugin] Shrine plugin provides a
773
+ [mountable][Mounting Endpoints] Rack application that generates upload
774
+ parameters, and you can combine it with Uppy's [AWS S3][uppy aws-s3] plugin:
835
775
 
836
776
  ```rb
837
777
  Shrine.plugin :presign_endpoint
@@ -839,62 +779,144 @@ Shrine.plugin :presign_endpoint
839
779
  ```rb
840
780
  # config/routes.rb (Rails)
841
781
  Rails.application.routes.draw do
842
- mount Shrine.presign_endpoint(:cache) => "s3/params"
782
+ # ...
783
+ mount Shrine.presign_endpoint(:cache) => "/s3/params" # GET /s3/params
843
784
  end
844
785
  ```
786
+ ```js
787
+ // upload.js
788
+ // ...
789
+ uppy.use(Uppy.AwsS3, {
790
+ companionUrl: '/' // uses '/s3/params'
791
+ })
792
+
793
+ uppy.on('upload-success', (file, response) => {
794
+ const uploadedFileData = JSON.stringify({
795
+ id: file.meta['key'].match(/^cache\/(.+)/)[1], // object key without prefix
796
+ storage: 'cache',
797
+ metadata: {
798
+ size: file.size,
799
+ filename: file.name,
800
+ mime_type: file.type,
801
+ }
802
+ })
803
+ // ... add this data to your form or submit it to your app ...
804
+ })
805
+ ```
806
+
807
+ For adding direct S3 uploads from scratch, see [this walkthrough][Adding Direct
808
+ S3 Uploads] (there is also the [Roda][roda demo] / [Rails][rails demo] demo).
809
+ See also the [Direct Uploads to S3] guide for more details.
845
810
 
846
- The above will add a `GET /s3/params` route to your app. You can now hook Uppy's
847
- [AWS S3][uppy aws s3] plugin to this endpoint and have it upload directly to
848
- S3. See [this walkthrough][direct S3 uploads walkthrough] that shows adding
849
- direct S3 uploads from scratch, as well as the [Direct Uploads to S3][direct S3
850
- uploads guide] guide that provides some useful tips. Also check out the
851
- [Roda][roda demo] / [Rails][rails demo] demo app which implements multiple
852
- uploads directly to S3.
811
+ ### Resumable direct upload
853
812
 
854
- If you wanted to implement this enpdoint yourself, this is how it could roughly
855
- look like for S3 storage in Sinatra:
813
+ If your app is accepting large uploads, you can make the uploads **resumable**.
814
+ This can significantly improve experience for users on slow and flaky internet
815
+ connections.
856
816
 
857
- ```rb
858
- get "/s3/params" do
859
- storage = Shrine.storages[:cache]
860
- location = SecureRandom.hex + File.extname(params["filename"].to_s)
817
+ #### Uppy S3 Multipart
861
818
 
862
- presign_data = storage.presign(location, content_type: params["type"])
819
+ You can achieve resumable uploads directly to S3 with the [AWS S3
820
+ Multipart][uppy aws-s3-multipart] Uppy plugin, accompanied with the Shrine
821
+ plugin provided by the [uppy-s3_multipart] gem (assuming your temporary storage
822
+ is `Shrine::Storage::S3`):
863
823
 
864
- json presign_data
824
+ ```rb
825
+ # Gemfile
826
+ gem "uppy-s3_multipart", "~> 0.3"
827
+ ```
828
+ ```rb
829
+ Shrine.plugin :uppy_s3_multipart
830
+ ```
831
+ ```rb
832
+ # config/routes.rb (Rails)
833
+ Rails.application.routes.draw do
834
+ # ...
835
+ mount Shrine.uppy_s3_multipart(:cache) => "/s3/multipart"
865
836
  end
866
837
  ```
838
+ ```js
839
+ // upload.js
840
+ // ...
841
+ uppy.use(Uppy.AwsS3Multipart, {
842
+ companionUrl: '/' // uses '/s3/multipart/*' routes
843
+ })
867
844
 
868
- ### Resumable direct upload
845
+ uppy.on('upload-success', (file, response) => {
846
+ const uploadedFileData = JSON.stringify({
847
+ id: response.uploadURL.match(/\/cache\/([^\?]+)/)[1], // object key without prefix
848
+ storage: 'cache',
849
+ metadata: {
850
+ size: file.size,
851
+ filename: file.name,
852
+ mime_type: file.type,
853
+ }
854
+ })
855
+ // ... add this data to your form or submit it to your app ...
856
+ })
857
+ ```
869
858
 
870
- If your app is dealing with large uploads (e.g. videos), keep in mind that it
871
- can be challening for your users to upload these large files to your app. Many
872
- users might not have a great internet connection, and if it happens to break at
873
- any point during uploading, they need to retry the upload from the beginning.
859
+ See the [uppy-s3_multipart] docs for more details.
874
860
 
875
- This problem has been solved by **[tus]**. tus is an open protocol for
876
- resumable file uploads, which enables the client and the server to achieve
877
- reliable file uploads even on unstable connections, by enabling the upload to
878
- be resumed in case of interruptions, even after the browser was closed or the
879
- device was shut down.
861
+ #### Tus protocol
880
862
 
881
- [tus-ruby-server] provides a Ruby server implemenation of the tus protocol.
882
- Uppy's [Tus][uppy tus] plugin can then be configured to do resumable uploads to
883
- a tus-ruby-server instance, and then the uploaded files can be attached to the
884
- record with the help of [shrine-tus]. See [this walkthrough][resumable uploads
885
- walkthrough] that adds resumable uploads from scratch, as well as the
886
- [demo][resumable demo] for a complete example.
863
+ If you want a more generic approach, you can build your resumable uploads on
864
+ **[tus]**, an open resumable upload protocol. On the server side you can use
865
+ the [tus-ruby-server] gem, on the client side Uppy's [Tus][uppy tus] plugin,
866
+ and the [shrine-tus] gem for the glue:
887
867
 
888
- Alternatively, you can have resumable uploads directly to S3 using Uppy's [AWS
889
- S3 Multipart][uppy aws s3 multipart] plugin, accompanied with the
890
- [uppy-s3_multipart] gem.
868
+ ```rb
869
+ # Gemfile
870
+ gem "tus-server", "~> 2.0"
871
+ gem "shrine-tus", "~> 1.2"
872
+ ```
873
+ ```rb
874
+ require "shrine/storage/tus"
875
+
876
+ Shrine.storages = {
877
+ cache: Shrine::Storage::Tus.new, # tus server acts as temporary storage
878
+ store: ..., # your permanent storage
879
+ }
880
+ ```
881
+ ```rb
882
+ # config/routes.rb (Rails)
883
+ Rails.application.routes.draw do
884
+ # ...
885
+ mount Tus::Server => "/files"
886
+ end
887
+ ```
888
+ ```js
889
+ // upload.js
890
+ // ...
891
+ uppy.use(Uppy.Tus, {
892
+ endpoint: '/files',
893
+ chunkSize: 5*1024*1024,
894
+ })
895
+
896
+ uppy.on('upload-success', (file, response) => {
897
+ const uploadedFileData = JSON.stringify({
898
+ id: response.uploadURL,
899
+ storage: "cache",
900
+ metadata: {
901
+ filename: file.name,
902
+ size: file.size,
903
+ mime_type: file.type,
904
+ }
905
+ })
906
+ // ... add this data to your form or submit it to your app ...
907
+ })
908
+ ```
909
+
910
+ For adding tus-powered resumable uploads from scratch, see [this
911
+ walkthrough][Adding Resumable Uploads] (there is also a [demo app][resumable
912
+ demo]). See also [shrine-tus] and [tus-ruby-server] docs for more details.
891
913
 
892
914
  ## Backgrounding
893
915
 
894
- Shrine is the first file attachment library designed for backgrounding support.
895
- Moving phases of managing file attachments to background jobs is essential for
896
- scaling and good user experience, and Shrine provides a `backgrounding` plugin
897
- which makes it easy to plug in your favourite backgrounding library:
916
+ Shrine allows you to put file deletion and promotion of cached files to
917
+ permanent storage into a background job. The [`backgrounding`][backgrounding
918
+ plugin] plugin provides hooks for plugging in your [favourite backgrounding
919
+ library][Backgrounding Libraries].
898
920
 
899
921
  ```rb
900
922
  Shrine.plugin :backgrounding
@@ -918,11 +940,6 @@ class DeleteJob
918
940
  end
919
941
  ```
920
942
 
921
- The above puts promoting (uploading cached file to permanent storage) and
922
- deleting of files for all uploaders into background jobs using Sidekiq.
923
- Obviously instead of Sidekiq you can use [any other backgrounding
924
- library][backgrounding libraries].
925
-
926
943
  ## Clearing cache
927
944
 
928
945
  Shrine doesn't automatically delete files uploaded to temporary storage, instead
@@ -943,47 +960,6 @@ s3 = Shrine.storages[:cache]
943
960
  s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 } # delete files older than 1 week
944
961
  ```
945
962
 
946
- Note that for AWS S3 you can also configure bucket lifecycle rules to do this
947
- for you. This can be done either from the [AWS Console][S3 lifecycle console]
948
- or via an [API call][S3 lifecycle API]:
949
-
950
- ```rb
951
- require "aws-sdk-s3"
952
-
953
- client = Aws::S3::Client.new(
954
- access_key_id: "<YOUR KEY>",
955
- secret_access_key: "<YOUR SECRET>",
956
- region: "<REGION>",
957
- )
958
-
959
- client.put_bucket_lifecycle_configuration(
960
- bucket: "<YOUR BUCKET>",
961
- lifecycle_configuration: {
962
- rules: [{
963
- expiration: { days: 7 },
964
- filter: { prefix: "cache/" },
965
- id: "cache-clear",
966
- status: "Enabled"
967
- }]
968
- }
969
- )
970
- ```
971
-
972
- ## Logging
973
-
974
- Shrine ships with the `logging` which automatically logs processing, uploading,
975
- and deleting of files. This can be very helpful for debugging and performance
976
- monitoring.
977
-
978
- ```rb
979
- Shrine.plugin :logging
980
- ```
981
- ```
982
- 2015-10-09T20:06:06.676Z #25602: STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s)
983
- 2015-10-09T20:06:06.854Z #25602: PROCESS[store]: ImageUploader[:avatar] User[29543] 1-3 files (0.22s)
984
- 2015-10-09T20:06:07.133Z #25602: DELETE[destroyed]: ImageUploader[:avatar] User[29543] 3 files (0.07s)
985
- ```
986
-
987
963
  ## Settings
988
964
 
989
965
  Each uploader can store generic settings in the `opts` hash, which can be
@@ -1030,62 +1006,111 @@ mailing lists is expected to follow the [Shrine code of conduct][CoC].
1030
1006
 
1031
1007
  The gem is available as open source under the terms of the [MIT License].
1032
1008
 
1033
- [Shrine]: https://shrinerb.com
1034
- [plugin system]: https://twin.github.io/the-plugin-system-of-sequel-and-roda/
1009
+ <!-- Guides & RDocs -->
1010
+ [Advantages of Shrine]: /doc/advantages.md#readme
1011
+ [Creating Plugins]: /doc/creating_plugins.md#readme
1012
+ [Creating Storages]: /doc/creating_storages.md#readme
1013
+ [Direct Uploads to S3]: /doc/direct_s3.md#readme
1014
+ [Extracting Metadata]: /doc/metadata.md#readme
1015
+ [File Processing]: /doc/processing.md#readme
1016
+ [File Validation]: /doc/validation.md#readme
1017
+ [Retrieving Uploads]: /doc/retrieving_uploads.md#readme
1018
+ [Using Attacher]: /doc/attacher.md#readme
1035
1019
  [FileSystem]: /doc/storage/file_system.md#readme
1036
1020
  [S3]: /doc/storage/s3.md#readme
1037
- [GCS]: https://github.com/renchap/shrine-google_cloud_storage
1021
+ [`Shrine::UploadedFile`]: https://shrinerb.com/rdoc/classes/Shrine/UploadedFile/InstanceMethods.html
1022
+
1023
+ <!-- Sections -->
1024
+ [attacher]: #attacher
1025
+ [attachment]: #attachment
1026
+ [backgrounding]: #backgrounding
1027
+ [direct uploads]: #direct-uploads
1028
+ [io abstraction]: #io-abstraction
1029
+ [location]: #location
1030
+ [metadata]: #metadata
1031
+ [on upload]: #processing-on-upload
1032
+ [on-the-fly]: #processing-on-the-fly
1033
+ [plugin system]: #plugin-system
1034
+ [simple upload]: #simple-direct-upload
1035
+ [presigned upload]: #presigned-direct-upload
1036
+ [resumable upload]: #resumable-direct-upload
1037
+ [storage]: #storage
1038
+ [uploaded file]: #uploaded-file
1039
+ [uploading]: #uploading
1040
+ [uploader]: #uploader
1041
+ [validation]: #validation
1042
+
1043
+ <!-- Wikis -->
1044
+ [Adding Direct App Uploads]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-App-Uploads
1045
+ [Adding Resumable Uploads]: https://github.com/shrinerb/shrine/wiki/Adding-Resumable-Uploads
1046
+ [Adding Direct S3 Uploads]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-S3-Uploads
1047
+ [Backgrounding Libraries]: https://github.com/shrinerb/shrine/wiki/Backgrounding-Libraries
1048
+ [Mounting Endpoints]: https://github.com/shrinerb/shrine/wiki/Mounting-Endpoints
1049
+
1050
+ <!-- Storage & Destinations -->
1051
+ [AWS S3]: https://aws.amazon.com/s3/
1052
+ [MinIO]: https://min.io/
1053
+ [DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
1038
1054
  [Cloudinary]: https://github.com/shrinerb/shrine-cloudinary
1039
- [Transloadit]: https://github.com/shrinerb/shrine-transloadit
1040
- [activerecord plugin]: /doc/plugins/activerecord.md#readme
1041
- [sequel plugin]: /doc/plugins/sequel.md#readme
1042
- [hanami plugin]: https://github.com/katafrakt/hanami-shrine
1043
- [mongoid plugin]: https://github.com/shrinerb/shrine-mongoid
1055
+ [GCS]: https://github.com/renchap/shrine-google_cloud_storage
1056
+ [uppy-s3_multipart]: https://github.com/janko/uppy-s3_multipart
1057
+ [tus-ruby-server]: https://github.com/janko/tus-ruby-server
1058
+
1059
+ <!-- Direct Uploads -->
1060
+ [Uppy]: https://uppy.io
1061
+ [shrine-tus]: https://github.com/shrinerb/shrine-tus
1062
+ [tus]: https://tus.io
1063
+ [uppy aws-s3-multipart]: https://uppy.io/docs/aws-s3-multipart/
1064
+ [uppy aws-s3]: https://uppy.io/docs/aws-s3/
1065
+ [uppy tus]: https://uppy.io/docs/tus/
1066
+ [uppy xhr-upload]: https://uppy.io/docs/xhr-upload/
1067
+
1068
+ <!-- Processing -->
1069
+ [ImageMagick]: https://imagemagick.org/
1070
+ [MiniMagick]: https://github.com/minimagick/minimagick
1071
+ [ruby-vips]: https://github.com/libvips/ruby-vips
1044
1072
  [ImageProcessing]: https://github.com/janko/image_processing
1045
- [on upload]: #on-upload
1046
- [on-the-fly]: #on-the-fly
1047
1073
  [ImageProcessing::MiniMagick]: https://github.com/janko/image_processing/blob/master/doc/minimagick.md#readme
1048
1074
  [ImageProcessing::Vips]: https://github.com/janko/image_processing/blob/master/doc/vips.md#readme
1049
- [ImageMagick]: https://imagemagick.org/
1050
1075
  [libvips]: http://libvips.github.io/libvips/
1076
+ [`file`]: http://linux.die.net/man/1/file
1077
+
1078
+ <!-- Plugins -->
1079
+ [activerecord plugin]: /doc/plugins/activerecord.md#readme
1080
+ [add_metadata plugin]: /doc/plugins/add_metadata.md#readme
1081
+ [backgrounding plugin]: /doc/plugins/backgrounding.md#readme
1051
1082
  [derivation_endpoint plugin]: /doc/plugins/derivation_endpoint.md#readme
1052
- [validation_helpers plugin]: /doc/plugins/validation_helpers.md#readme
1053
- [upload_endpoint plugin]: /doc/plugins/upload_endpoint.md#readme
1083
+ [determine_mime_type plugin]: /doc/plugins/determine_mime_type.md#readme
1084
+ [hanami plugin]: https://github.com/katafrakt/hanami-shrine
1085
+ [mongoid plugin]: https://github.com/shrinerb/shrine-mongoid
1054
1086
  [presign_endpoint plugin]: /doc/plugins/presign_endpoint.md#readme
1055
- [Uppy]: https://uppy.io
1056
- [tus]: https://tus.io
1057
- [uppy tus]: https://uppy.io/docs/tus/
1058
- [tus-ruby-server]: https://github.com/janko/tus-ruby-server
1059
- [backgrounding plugin]: /doc/plugins/backgrounding.md#readme
1060
- [Advantages of Shrine]: /doc/advantages.md#readme
1061
- [external storages]: https://shrinerb.com/#external
1062
- [creating storage]: /doc/creating_storages.md#readme
1063
- [creating plugin]: /doc/creating_plugins.md#readme
1064
- [Retrieving Uploads]: /doc/retrieving_uploads.md#readme
1065
- [Using Attacher]: /doc/attacher.md#readme
1066
- [plugins]: https://shrinerb.com/#plugins
1067
- [`file`]: http://linux.die.net/man/1/file
1068
- [Extracting Metadata]: /doc/metadata.md#readme
1069
- [File Processing]: /doc/processing.md#readme
1070
- [File Validation]: /doc/validation.md#readme
1071
- [metadata direct uploads]: /doc/metadata.md#direct-uploads
1072
- [uppy xhr upload]: https://uppy.io/docs/xhr-upload/
1073
- [direct uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-App-Uploads
1074
- [uppy aws s3]: https://uppy.io/docs/aws-s3/
1075
- [direct S3 uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-S3-Uploads<Paste>
1076
- [direct S3 uploads guide]: /doc/direct_s3.md#readme
1077
- [roda demo]: https://github.com/shrinerb/shrine/tree/master/demo
1087
+ [pretty_location plugin]: /doc/plugins/pretty_location.md#readme
1088
+ [processing plugin]: /doc/plugins/processing.md#readme
1089
+ [rack_file plugin]: /doc/plugins/rack_file.md#readme
1090
+ [sequel plugin]: /doc/plugins/sequel.md#readme
1091
+ [signature plugin]: /doc/plugins/signature.md#readme
1092
+ [store_dimensions plugin]: /doc/plugins/store_dimensions.md#readme
1093
+ [upload_endpoint plugin]: /doc/plugins/upload_endpoint.md#readme
1094
+ [validation_helpers plugin]: /doc/plugins/validation_helpers.md#readme
1095
+ [versions plugin]: /doc/plugins/versions.md#readme
1096
+
1097
+ <!-- Demos -->
1078
1098
  [rails demo]: https://github.com/erikdahlstrand/shrine-rails-example
1079
- [shrine-tus]: https://github.com/shrinerb/shrine-tus
1080
- [uppy aws s3 multipart]: https://uppy.io/docs/aws-s3-multipart/
1081
- [uppy-s3_multipart]: https://github.com/janko/uppy-s3_multipart
1082
- [resumable uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Resumable-Uploads
1099
+ [roda demo]: https://github.com/shrinerb/shrine/tree/master/demo
1083
1100
  [resumable demo]: https://github.com/shrinerb/shrine-tus-demo
1084
- [backgrounding libraries]: https://github.com/shrinerb/shrine/wiki/Backgrounding-Libraries
1085
- [Mounting Endpoints]: https://github.com/shrinerb/shrine/wiki/Mounting-Endpoints
1086
- [S3 lifecycle Console]: http://docs.aws.amazon.com/AmazonS3/latest/UG/lifecycle-configuration-bucket-no-versioning.html
1087
- [S3 lifecycle API]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#put_bucket_lifecycle_configuration-instance_method
1088
- [Roda]: https://github.com/jeremyevans/roda
1101
+
1102
+ <!-- Misc -->
1103
+ [`#read`]: https://ruby-doc.org/core/IO.html#method-i-read
1104
+ [`#eof?`]: https://ruby-doc.org/core/IO.html#method-i-eof
1105
+ [`#rewind`]: https://ruby-doc.org/core/IO.html#method-i-rewind
1106
+ [`#close`]: https://ruby-doc.org/core/IO.html#method-i-close
1107
+ [`IO`]: https://ruby-doc.org/core/IO.html
1089
1108
  [Refile]: https://github.com/refile/refile
1109
+ [Roda]: https://github.com/jeremyevans/roda
1110
+
1111
+ <!-- Project -->
1112
+ [Shrine]: https://shrinerb.com
1113
+ [external]: https://shrinerb.com/#external
1114
+ [plugins]: https://shrinerb.com/#plugins
1090
1115
  [CoC]: CODE_OF_CONDUCT.md
1091
1116
  [MIT License]: http://opensource.org/licenses/MIT