shrine 2.12.0 → 2.13.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of shrine might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +30 -0
- data/README.md +153 -41
- data/doc/advantages.md +96 -106
- data/doc/attacher.md +55 -18
- data/doc/design.md +16 -5
- data/doc/direct_s3.md +132 -113
- data/doc/metadata.md +82 -27
- data/doc/multiple_files.md +76 -33
- data/doc/processing.md +2 -11
- data/doc/testing.md +2 -2
- data/lib/shrine.rb +18 -10
- data/lib/shrine/plugins/determine_mime_type.rb +6 -1
- data/lib/shrine/plugins/download_endpoint.rb +3 -0
- data/lib/shrine/plugins/infer_extension.rb +25 -10
- data/lib/shrine/plugins/module_include.rb +1 -1
- data/lib/shrine/plugins/presign_endpoint.rb +10 -8
- data/lib/shrine/plugins/rack_file.rb +8 -0
- data/lib/shrine/plugins/store_dimensions.rb +1 -1
- data/lib/shrine/plugins/upload_endpoint.rb +8 -4
- data/lib/shrine/plugins/versions.rb +1 -2
- data/lib/shrine/storage/file_system.rb +6 -4
- data/lib/shrine/storage/s3.rb +89 -82
- data/lib/shrine/version.rb +1 -1
- data/shrine.gemspec +3 -2
- metadata +22 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9f51ec7d13c7f830e1cebb7d12c6444aad0b0c2df6c7d57e2cec61da7e8bc8b1
|
4
|
+
data.tar.gz: 9c75a7232d0e10910582a418646fa850305ef2cc164a6a8bc6e021ec52ac5de7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 03c0e1e29b8ba35b64fc6e33ab1a297a7f843726e3e7d21bca57449d1b36ee14c2781c3a2ddc6aa1f5bf36aeff9c7f8e77ee1eec8658cc3177bf701b9e942362
|
7
|
+
data.tar.gz: 3d2e11839c487fbd1782bc41751718a6d8e601070aa4c2a03f469f79a9b6fd176d2f5da9db37f2d1236ec1c64bb480c580217061f1d67c87eec0a2f86ae46d74
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,33 @@
|
|
1
|
+
## 2.13.0 (2018-11-04)
|
2
|
+
|
3
|
+
* Specify UTF-8 charset in `Content-Type` response header in `presign_endpoint` plugin (@janko-m)
|
4
|
+
|
5
|
+
* Specify UTF-8 charset in `Content-Type` response header in `upload_endpoint` plugin (@janko-m)
|
6
|
+
|
7
|
+
* Force UTF-8 encoding on filenames coming from Rack's multipart request params in `rack_file` plugin (@janko-m)
|
8
|
+
|
9
|
+
* Allow `:host` in `S3#url` to specify a host URL with an additional path prefix (@janko-m)
|
10
|
+
|
11
|
+
* Revert adding bucket name to URL path in `S3#url` when `:host` is used with `:force_path_style` (@janko-m)
|
12
|
+
|
13
|
+
* In `upload_endpoint` error with "Upload Not Valid" when `file` parameter is present but not a file (@janko-m)
|
14
|
+
|
15
|
+
* Allow `Attacher#assign` to accept options for `Shrine#upload` (@janko-m)
|
16
|
+
|
17
|
+
* Add `:metadata` option to `Shrine#upload` for manually overriding extracted metadata (@janko-m)
|
18
|
+
|
19
|
+
* Add `:force` option to `infer_extension` plugin for always replacing the current extension (@jrochkind)
|
20
|
+
|
21
|
+
* Add `:public` option to `S3#initialize` for enabling public uploads (@janko-m)
|
22
|
+
|
23
|
+
* Add ability to specify a custom `:signer` for `Shrine::Storage::S3#url` (@janko-m)
|
24
|
+
|
25
|
+
* In `S3#upload` do multipart upload for large non-file IO objects (@janko-m)
|
26
|
+
|
27
|
+
* In `S3#upload` switch to `Aws::S3::Object#upload_stream` for multipart uploads of IO objects of unknown size (@janko-m)
|
28
|
+
|
29
|
+
* In `S3#upload` deprecate using aws-sdk-s3 lower than 1.14 when uploading IO objects of unknown size (@janko-m)
|
30
|
+
|
1
31
|
## 2.12.0 (2018-08-22)
|
2
32
|
|
3
33
|
* Ignore nil values when assigning files from a remote URL (@janko-m)
|
data/README.md
CHANGED
@@ -19,6 +19,7 @@ If you're curious how it compares to other file attachment libraries, see the [A
|
|
19
19
|
- Documentation: [shrinerb.com](https://shrinerb.com)
|
20
20
|
- Demo code: [Roda][roda demo] / [Rails][rails demo]
|
21
21
|
- Source: [github.com/shrinerb/shrine](https://github.com/shrinerb/shrine)
|
22
|
+
- Wiki: [github.com/shrinerb/shrine/wiki](https://github.com/shrinerb/shrine/wiki)
|
22
23
|
- Bugs: [github.com/shrinerb/shrine/issues](https://github.com/shrinerb/shrine/issues)
|
23
24
|
- Help & Discussion: [groups.google.com/group/ruby-shrine](https://groups.google.com/forum/#!forum/ruby-shrine)
|
24
25
|
|
@@ -202,6 +203,14 @@ Some of the tasks performed by `#upload` include:
|
|
202
203
|
* uploading (this is where the storage is called)
|
203
204
|
* closing the uploaded file
|
204
205
|
|
206
|
+
Additional upload options can be passed via `:upload_options`, and they will be
|
207
|
+
forwarded directly to `Storage#upload` (see the documentation of your storage
|
208
|
+
for the list of available options):
|
209
|
+
|
210
|
+
```rb
|
211
|
+
uploader.upload(file, upload_options: { acl: "public-read" })
|
212
|
+
```
|
213
|
+
|
205
214
|
### IO abstraction
|
206
215
|
|
207
216
|
Shrine is able to upload any IO-like object that responds to `#read`,
|
@@ -351,8 +360,8 @@ photo.update(image: nil) # removes the attachment and deletes previous
|
|
351
360
|
|
352
361
|
In addition to assigning raw files, you can also assign a JSON representation
|
353
362
|
of files that are already uploaded to the temporary storage. This allows Shrine
|
354
|
-
to retain cached files in case of validation errors and handle [direct
|
355
|
-
|
363
|
+
to retain cached files in case of validation errors and handle [direct uploads]
|
364
|
+
via the hidden form field.
|
356
365
|
|
357
366
|
```rb
|
358
367
|
photo.image = '{"id":"9260ea09d8effd.jpg","storage":"cache","metadata":{...}}'
|
@@ -361,7 +370,7 @@ photo.image = '{"id":"9260ea09d8effd.jpg","storage":"cache","metadata":{...}}'
|
|
361
370
|
## Attacher
|
362
371
|
|
363
372
|
The model attachment attributes and callbacks just delegate the behaviour
|
364
|
-
to
|
373
|
+
to their underlying `Shrine::Attacher` object.
|
365
374
|
|
366
375
|
```rb
|
367
376
|
photo.image_attacher #=> #<Shrine::Attacher>
|
@@ -397,6 +406,14 @@ photo.image = file # uploads to :other_cache storage
|
|
397
406
|
photo.save # promotes to :other_store storage
|
398
407
|
```
|
399
408
|
|
409
|
+
You can also skip the temporary storage altogether and upload files directly to
|
410
|
+
the primary storage:
|
411
|
+
|
412
|
+
```rb
|
413
|
+
uploaded_file = attacher.store!(file) # upload file directly to permanent storage
|
414
|
+
attacher.set(uploaded_file) # attach the uploaded file
|
415
|
+
```
|
416
|
+
|
400
417
|
Whenever the attacher uploads or deletes files, it sends a `context` hash
|
401
418
|
which includes `:record`, `:name`, and `:action` keys, so that you can perform
|
402
419
|
processing or generate location differently depending on this information. See
|
@@ -471,13 +488,22 @@ By the default the UNIX [`file`] utility is used to determine the MIME type,
|
|
471
488
|
but you can also choose a different analyzer – see the plugin documentation for
|
472
489
|
more details.
|
473
490
|
|
474
|
-
###
|
491
|
+
### Other metadata
|
475
492
|
|
476
493
|
In addition to `size`, `filename`, and `mime_type`, you can also extract image
|
477
494
|
dimensions using the `store_dimensions` plugin, as well as any custom metadata
|
478
495
|
using the `add_metadata` plugin. Check out the [Extracting Metadata] guide for
|
479
496
|
more details.
|
480
497
|
|
498
|
+
Note that you can also manually override extracted metadata by passing the
|
499
|
+
`:metadata` option to `Shrine#upload`:
|
500
|
+
|
501
|
+
```rb
|
502
|
+
uploaded_file = uploader.upload(file, metadata: { "filename" => "Matrix[1999].mp4", "foo" => "bar" })
|
503
|
+
uploaded_file.original_filename #=> "Matrix[1999].mp4"
|
504
|
+
uploaded_file.metadata["foo"] #=> "bar"
|
505
|
+
```
|
506
|
+
|
481
507
|
## Processing
|
482
508
|
|
483
509
|
Shrine's `processing` plugin allows you to intercept when the cached file is
|
@@ -552,23 +578,23 @@ The `versions` plugin also expands `#<attachment>_url` to accept version names:
|
|
552
578
|
photo.image_url(:large) #=> "https://..."
|
553
579
|
```
|
554
580
|
|
555
|
-
For more details, including examples of how to do custom
|
556
|
-
[File Processing] guide.
|
581
|
+
For more details, including examples of how to do custom and on-the-fly
|
582
|
+
processing, see the [File Processing] guide.
|
557
583
|
|
558
584
|
## Context
|
559
585
|
|
560
586
|
The `#upload` (and `#delete`) methods accept a hash of options as the second
|
561
|
-
argument, which is forwarded
|
562
|
-
metadata and generating location.
|
587
|
+
argument, which is forwarded down the chain and be available for processing,
|
588
|
+
extracting metadata and generating location.
|
563
589
|
|
564
590
|
```rb
|
565
591
|
uploader.upload(file, { foo: "bar" }) # context hash is forwarded to all tasks around upload
|
566
592
|
```
|
567
593
|
|
568
|
-
Some options are actually recognized by Shrine
|
569
|
-
`:upload_options`, some are added by plugins, and the rest are
|
570
|
-
provide additional context, for more flexibility in performing
|
571
|
-
descriptive logging.
|
594
|
+
Some options are actually recognized by Shrine (such as `:location`,
|
595
|
+
`:upload_options`, and `:metadata`), some are added by plugins, and the rest are
|
596
|
+
there just to provide additional context, for more flexibility in performing
|
597
|
+
tasks and more descriptive logging.
|
572
598
|
|
573
599
|
The attacher automatically includes additional `context` information for each
|
574
600
|
upload and delete operation:
|
@@ -659,11 +685,30 @@ responsive during upload, so the user can fill in other fields while the files
|
|
659
685
|
are being uploaded, and if you display a progress bar they can see when the
|
660
686
|
upload will finish.
|
661
687
|
|
662
|
-
|
663
|
-
where the form is submitted.
|
664
|
-
|
665
|
-
storage
|
666
|
-
|
688
|
+
These asynchronous uploads will have to go to an endpoint separate from the one
|
689
|
+
where the form is submitted. This can be an endpoint in your app, or an
|
690
|
+
endpoint of a cloud service. In either case, the uploads should go to
|
691
|
+
*temporary* storage (`:cache`), to ensure there won't be any orphan files in
|
692
|
+
the primary storage (`:store`).
|
693
|
+
|
694
|
+
Once files are uploaded on the client side, their data can be submitted to the
|
695
|
+
server and attached to a record, just like with raw files. The only difference
|
696
|
+
is that they won't be additionally uploaded to temporary storage on assignment,
|
697
|
+
as they were already uploaded on the client side. Note that by default **Shrine
|
698
|
+
won't extract metadata from directly uploaded files**, instead it will just copy
|
699
|
+
metadata that was extacted on the client side; see [this section][metadata direct uploads]
|
700
|
+
for the rationale and instructions on how to opt in.
|
701
|
+
|
702
|
+
For handling client side uploads it's recommended to use **[Uppy]**. Uppy is a
|
703
|
+
very flexible modern JavaScript file upload library, which happens to integrate
|
704
|
+
nicely with Shrine.
|
705
|
+
|
706
|
+
### Simple direct upload
|
707
|
+
|
708
|
+
The simplest approach is creating an upload endpoint in your app that will
|
709
|
+
receive uploads and forward them to the specified storage. You can use the
|
710
|
+
`upload_endpoint` Shrine plugin to create a Rack app that handles uploads,
|
711
|
+
and mount it inside your application.
|
667
712
|
|
668
713
|
```rb
|
669
714
|
Shrine.plugin :upload_endpoint
|
@@ -682,38 +727,101 @@ Rails.application.routes.draw do
|
|
682
727
|
end
|
683
728
|
```
|
684
729
|
|
685
|
-
The above will add a `POST /images/upload` route to your app. You can now
|
686
|
-
|
687
|
-
|
688
|
-
for this will depend on your application, see [this
|
689
|
-
walkthrough]
|
730
|
+
The above will add a `POST /images/upload` route to your app. You can now use
|
731
|
+
Uppy's [XHR Upload][uppy xhr upload] plugin to upload selected files to this
|
732
|
+
endpoint, and have the uploaded file data submitted to your app. The client
|
733
|
+
side code for this will depend on your application, see [this
|
734
|
+
walkthrough][direct uploads walkthrough] for an example of adding simple direct
|
735
|
+
uploads from scratch.
|
736
|
+
|
737
|
+
If you wanted to implement this enpdoint yourself, this is how it could roughly
|
738
|
+
look like in Sinatra:
|
739
|
+
|
740
|
+
```rb
|
741
|
+
Shrine.plugin :rack_file # only if not using Rails
|
742
|
+
```
|
743
|
+
```rb
|
744
|
+
post "/images/upload" do
|
745
|
+
uploader = ImageUploader.new(:cache)
|
746
|
+
file = Shrine.rack_file(params["file"]) # only `params[:file]` in Rails
|
747
|
+
|
748
|
+
uploaded_file = uploader.upload(file)
|
749
|
+
|
750
|
+
json uploaded_file.data
|
751
|
+
end
|
752
|
+
```
|
753
|
+
|
754
|
+
### Presigned direct upload
|
755
|
+
|
756
|
+
If you want to free your app from receiving file uploads, you can also upload
|
757
|
+
files directly to the cloud (AWS S3, Google Cloud etc). In this flow the client
|
758
|
+
is required to first fetch upload parameters from the server, and then use these
|
759
|
+
parameters to make the upload. The `presign_endpoint` Shrine plugin can be used
|
760
|
+
to create a Rack app that generates these upload parameters (provided that the
|
761
|
+
underlying storage implements `#presign`):
|
762
|
+
|
763
|
+
```rb
|
764
|
+
Shrine.plugin :presign_endpoint
|
765
|
+
```
|
766
|
+
```rb
|
767
|
+
# config.ru (Rack)
|
768
|
+
map "/presign" do
|
769
|
+
run Shrine.presign_endpoint(:cache)
|
770
|
+
end
|
771
|
+
|
772
|
+
# OR
|
773
|
+
|
774
|
+
# config/routes.rb (Rails)
|
775
|
+
Rails.application.routes.draw do
|
776
|
+
mount Shrine.presign_endpoint(:cache) => "/presign"
|
777
|
+
end
|
778
|
+
```
|
779
|
+
|
780
|
+
The above will add a `GET /presign` route to your app. You can now hook Uppy's
|
781
|
+
[AWS S3][uppy aws s3] plugin to this endpoint and have it upload directly to
|
782
|
+
S3. See [this walkthrough][direct S3 uploads walkthrough] that shows adding
|
783
|
+
direct S3 uploads from scratch, as well as the [Direct Uploads to S3][direct S3
|
784
|
+
uploads guide] guide that provides some useful tips. Also check out the
|
785
|
+
[Roda][roda demo] / [Rails][rails demo] demo app which implements multiple
|
786
|
+
uploads directly to S3.
|
787
|
+
|
788
|
+
If you wanted to implement this enpdoint yourself, this is how it could roughly
|
789
|
+
look like for S3 storage in Sinatra:
|
690
790
|
|
691
|
-
|
692
|
-
|
693
|
-
|
694
|
-
|
695
|
-
|
696
|
-
|
791
|
+
```rb
|
792
|
+
get "/presign" do
|
793
|
+
storage = Shrine.storages[:cache]
|
794
|
+
location = SecureRandom.hex + File.extname(params["filename"].to_s)
|
795
|
+
|
796
|
+
presign_data = storage.presign(location, content_type: params["type"])
|
797
|
+
|
798
|
+
json presign_data
|
799
|
+
end
|
800
|
+
```
|
697
801
|
|
698
|
-
### Resumable
|
802
|
+
### Resumable direct upload
|
699
803
|
|
700
|
-
|
804
|
+
If your app is dealing with large uploads (e.g. videos), keep in mind that it
|
701
805
|
can be challening for your users to upload these large files to your app. Many
|
702
806
|
users might not have a great internet connection, and if it happens to break at
|
703
|
-
any point during uploading, they
|
704
|
-
beginning.
|
807
|
+
any point during uploading, they need to retry the upload from the beginning.
|
705
808
|
|
706
|
-
|
707
|
-
|
809
|
+
This problem has been solved by **[tus]**. tus is an open protocol for
|
810
|
+
resumable file uploads, which enables the client and the server to achieve
|
708
811
|
reliable file uploads even on unstable connections, by enabling the upload to
|
709
812
|
be resumed in case of interruptions, even after the browser was closed or the
|
710
813
|
device was shut down.
|
711
814
|
|
712
|
-
|
713
|
-
|
714
|
-
the
|
715
|
-
|
716
|
-
|
815
|
+
[tus-ruby-server] provides a Ruby server implemenation of the tus protocol.
|
816
|
+
Uppy's [Tus][uppy tus] plugin can then be configured to do resumable uploads to
|
817
|
+
a tus-ruby-server instance, and then the uploaded files can be attached to the
|
818
|
+
record with the help of [shrine-tus]. See [this walkthrough][resumable uploads
|
819
|
+
walkthrough] that adds resumable uploads from scratch, as well as the
|
820
|
+
[demo][resumable demo] for a complete example.
|
821
|
+
|
822
|
+
Alternatively, you can have resumable uploads directly to S3 using Uppy's [AWS
|
823
|
+
S3 Multipart][uppy aws s3 multipart] plugin, accompanied with the
|
824
|
+
[uppy-s3_multipart] gem.
|
717
825
|
|
718
826
|
## Backgrounding
|
719
827
|
|
@@ -869,7 +977,7 @@ The gem is available as open source under the terms of the [MIT License].
|
|
869
977
|
[mongoid plugin]: https://github.com/shrinerb/shrine-mongoid
|
870
978
|
[image_processing]: https://github.com/janko-m/image_processing
|
871
979
|
[ImageMagick]: https://www.imagemagick.org/script/index.php
|
872
|
-
[libvips]: http://
|
980
|
+
[libvips]: http://libvips.github.io/libvips/
|
873
981
|
[validation_helpers plugin]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/ValidationHelpers.html
|
874
982
|
[upload_endpoint plugin]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/UploadEndpoint.html
|
875
983
|
[presign_endpoint plugin]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/PresignEndpoint.html
|
@@ -888,13 +996,17 @@ The gem is available as open source under the terms of the [MIT License].
|
|
888
996
|
[Extracting Metadata]: https://shrinerb.com/rdoc/files/doc/metadata_md.html
|
889
997
|
[File Processing]: https://shrinerb.com/rdoc/files/doc/processing_md.html
|
890
998
|
[File Validation]: https://shrinerb.com/rdoc/files/doc/validation_md.html
|
999
|
+
[metadata direct uploads]: https://github.com/shrinerb/shrine/blob/master/doc/metadata.md#direct-uploads
|
1000
|
+
[uppy xhr upload]: https://uppy.io/docs/xhr-upload/
|
891
1001
|
[direct uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-App-Uploads
|
1002
|
+
[uppy aws s3]: https://uppy.io/docs/aws-s3/
|
892
1003
|
[direct S3 uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-S3-Uploads<Paste>
|
893
1004
|
[direct S3 uploads guide]: https://shrinerb.com/rdoc/files/doc/direct_s3_md.html
|
894
1005
|
[roda demo]: https://github.com/shrinerb/shrine/tree/master/demo
|
895
1006
|
[rails demo]: https://github.com/erikdahlstrand/shrine-rails-example
|
896
|
-
[tus-js-client]: https://github.com/tus/tus-js-client
|
897
1007
|
[shrine-tus]: https://github.com/shrinerb/shrine-tus
|
1008
|
+
[uppy aws s3 multipart]: https://uppy.io/docs/aws-s3-multipart/
|
1009
|
+
[uppy-s3_multipart]: https://github.com/janko-m/uppy-s3_multipart
|
898
1010
|
[resumable uploads walkthrough]: https://github.com/shrinerb/shrine/wiki/Adding-Resumable-Uploads
|
899
1011
|
[resumable demo]: https://github.com/shrinerb/shrine-tus-demo
|
900
1012
|
[backgrounding libraries]: https://github.com/shrinerb/shrine/wiki/Backgrounding-libraries
|
data/doc/advantages.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Advantages of Shrine
|
2
2
|
|
3
|
-
There are many
|
3
|
+
There are many existing file upload solutions for Ruby out there – [Paperclip],
|
4
4
|
[CarrierWave], [Dragonfly], [Refile], and [Active Storage], to name the most
|
5
5
|
popular ones. This guide will attempt to cover some of the main advantages that
|
6
6
|
Shrine offers compared to these alternatives.
|
@@ -8,15 +8,15 @@ Shrine offers compared to these alternatives.
|
|
8
8
|
## Generality
|
9
9
|
|
10
10
|
Many alternative file upload solutions are coupled to either Rails (Active
|
11
|
-
Storage) or Active Record itself (Paperclip,
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
11
|
+
Storage) or Active Record itself (Paperclip, Dragonfly). This is not ideal, as
|
12
|
+
Rails-specific solutions fragment the Ruby community between developers that
|
13
|
+
use Rails and developers that don't. There are many great web frameworks
|
14
|
+
([Sinatra], [Roda], [Cuba], [Hanami], [Grape]) and database libraries
|
15
|
+
([Sequel], [ROM], [Hanami::Model]) out there that people use instead of
|
16
|
+
Rails and Active Record.
|
17
17
|
|
18
18
|
Shrine, on the other hand, doesn't make any assumptions about which web
|
19
|
-
framework or ORM you're using. Any
|
19
|
+
framework or ORM you're using. Any web-specific functionality is implemented
|
20
20
|
on top of [Rack], the Ruby web server interface that powers all the popular
|
21
21
|
Ruby web frameworks (including Rails). The integrations for specific ORMs are
|
22
22
|
provided as plugins.
|
@@ -36,6 +36,31 @@ Shrine.plugin :mongoid # https://github.com/shrinerb/shrine-mongoid
|
|
36
36
|
Shrine.plugin :hanami # https://github.com/katafrakt/hanami-shrine
|
37
37
|
```
|
38
38
|
|
39
|
+
## Simplicity
|
40
|
+
|
41
|
+
Shrine was designed with simplicity in mind. Where other solutions favour
|
42
|
+
complex class-level DSLs, Shrine chooses simple instance-level interfaces where
|
43
|
+
you can write regular Ruby code.
|
44
|
+
|
45
|
+
There are no `CarrierWave::Uploader::Base` and `Paperclip::Attachment` [God
|
46
|
+
objects], Shrine has several core classes each with clear responsibilities:
|
47
|
+
|
48
|
+
* Storage classes encapsulate file operations for the underlying service
|
49
|
+
* `Shrine` handles uploads and manages plugins
|
50
|
+
* `Shrine::UploadedFile` repesents a file that was uploaded to a storage
|
51
|
+
* `Shrine::Attacher` handles attaching files to records
|
52
|
+
* `Shrine::Attachment` adds convenience methods to model instances
|
53
|
+
|
54
|
+
```rb
|
55
|
+
photo.image #=> #<Shrine::UploadedFile>
|
56
|
+
photo.image.storage #=> #<Shrine::Storage::S3>
|
57
|
+
photo.image.uploader #=> #<Shrine>
|
58
|
+
photo.image_attacher #=> #<Shrine::Attacher>
|
59
|
+
```
|
60
|
+
|
61
|
+
Special care was taken to make integrating new storages and ORMs possible with
|
62
|
+
minimal amount of code.
|
63
|
+
|
39
64
|
## Modularity
|
40
65
|
|
41
66
|
Shrine uses a [plugin system] that allows you to pick and choose the features
|
@@ -47,53 +72,32 @@ very fast.
|
|
47
72
|
Shrine.plugin :logging # loads the logging feature
|
48
73
|
```
|
49
74
|
|
75
|
+
Shrine comes with a complete attachment functionality, but it also exposes many
|
76
|
+
low level APIs that can be used for building your own customized attachment
|
77
|
+
flow.
|
78
|
+
|
50
79
|
### Dependencies
|
51
80
|
|
52
81
|
Shrine is very diligent when it comes to dependencies. It has only one
|
53
82
|
mandatory dependency - [Down], a gem for streaming downloads from a URL. Some
|
54
|
-
Shrine plugins require
|
55
|
-
you're using those plugins.
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
[
|
61
|
-
|
62
|
-
[
|
63
|
-
can choose between [FastImage], [MiniMagick] or [ruby-vips] gems for extracting
|
64
|
-
image dimensions.
|
83
|
+
Shrine plugins require additional dependencies, but you only need to load them
|
84
|
+
if you're using those plugins.
|
85
|
+
|
86
|
+
Moreover, Shrine often let you choose between multiple alternative dependencies
|
87
|
+
for doing the same task. For example, the `determine_mime_type` plugin allows
|
88
|
+
you to choose between the [`file`] command, [FileMagic], [FastImage],
|
89
|
+
[MimeMagic], or [Marcel] gem for determining the MIME type, while the
|
90
|
+
`store_dimensions` plugin can extract dimensions using [FastImage],
|
91
|
+
[MiniMagick], or [ruby-vips] gem.
|
65
92
|
|
66
93
|
```rb
|
67
94
|
Shrine.plugin :determine_mime_type, analyzer: :marcel
|
68
95
|
Shrine.plugin :store_dimensions, analyzer: :mini_magick
|
69
96
|
```
|
70
97
|
|
71
|
-
|
98
|
+
This approach gives you control over your dependencies by allowing you to
|
72
99
|
choose the combination that best suit your needs.
|
73
100
|
|
74
|
-
## Simplicity
|
75
|
-
|
76
|
-
Shrine was designed with simplicity in mind. Where other solutions favour
|
77
|
-
complex class-level DSLs, Shrine chooses simple instance-level interfaces where
|
78
|
-
you can write regular Ruby code.
|
79
|
-
|
80
|
-
There are also no `CarrierWave::Uploader::Base` and `Paperclip::Attachment` [God
|
81
|
-
objects]. The `Shrine` class is responsible for uploading files to the storage
|
82
|
-
(which are simple Ruby classes), `Shrine::UploadedFile` exposes extracted
|
83
|
-
metadata and can retrieve the file from the storage, and `Shrine::Attacher`
|
84
|
-
wraps those two classes to provide an interface for attaching files to database
|
85
|
-
records.
|
86
|
-
|
87
|
-
```rb
|
88
|
-
photo.image #=> #<Shrine::UploadedFile>
|
89
|
-
photo.image.storage #=> #<Shrine::Storage::S3>
|
90
|
-
photo.image.uploader #=> #<Shrine>
|
91
|
-
photo.image_attacher #=> #<Shrine::Attacher>
|
92
|
-
```
|
93
|
-
|
94
|
-
Special care was taken to make integrating new ORMs and storages possible with
|
95
|
-
minimal amount of code.
|
96
|
-
|
97
101
|
## Inheritance
|
98
102
|
|
99
103
|
Shrine is designed to handle any types of files. If you're accepting uploads of
|
@@ -125,11 +129,12 @@ end
|
|
125
129
|
|
126
130
|
## Processing
|
127
131
|
|
128
|
-
Instead of
|
129
|
-
thumbnails,
|
130
|
-
but it can
|
131
|
-
|
132
|
-
|
132
|
+
Instead of having yet another vendored solution for generating image
|
133
|
+
thumbnails, Shrine chose to adopt a generic **[ImageProcessing]** gem. The
|
134
|
+
ImageProcessing gem was created for Shrine, but it can be used in any other
|
135
|
+
file upload library. It has a very flexible API and takes care of many details
|
136
|
+
for you, such as [auto orienting] the input image and [sharpening] the
|
137
|
+
thumbnails after they are resized.
|
133
138
|
|
134
139
|
```rb
|
135
140
|
require "image_processing"
|
@@ -145,10 +150,11 @@ thumbnail #=> #<Tempfile:/var/folders/.../image_processing20180316-18446-1j247h6
|
|
145
150
|
### libvips
|
146
151
|
|
147
152
|
Probably the biggest ImageProcessing feature is the support for **[libvips]**.
|
148
|
-
|
149
|
-
|
150
|
-
libvips quick]). The `ImageProcessing::Vips` backend
|
151
|
-
`ImageProcessing::MiniMagick`, so you can easily
|
153
|
+
libvips is also a full-featured image processing library, which can process
|
154
|
+
images very rapidly – often multiple times faster than ImageMagick – with low
|
155
|
+
memory usage (see [Why is libvips quick]). The `ImageProcessing::Vips` backend
|
156
|
+
implements the same API as `ImageProcessing::MiniMagick`, so you can easily
|
157
|
+
swap one for the other.
|
152
158
|
|
153
159
|
```rb
|
154
160
|
require "image_processing/mini_magick"
|
@@ -166,11 +172,9 @@ ImageProcessing::Vips.resize_to_fit(800, 800).call(original)
|
|
166
172
|
|
167
173
|
### Other processors
|
168
174
|
|
169
|
-
|
170
|
-
|
171
|
-
|
172
|
-
transformation: you get the original file on the input, and you're expected to
|
173
|
-
return processed file(s) on the output.
|
175
|
+
Shrine's processing block simply executes the Ruby code inside it, so you can
|
176
|
+
call there any other processor your want. The only thing that Shrine requires
|
177
|
+
is that processed files are returned as the block result.
|
174
178
|
|
175
179
|
```rb
|
176
180
|
class VideoUploader < Shrine
|
@@ -182,28 +186,6 @@ class VideoUploader < Shrine
|
|
182
186
|
end
|
183
187
|
```
|
184
188
|
|
185
|
-
### On-the-fly processing
|
186
|
-
|
187
|
-
Shrine is primarily designed for processing files on upload, since that's
|
188
|
-
applicable to all types of files, though on-the-fly processing that [Dragonfly],
|
189
|
-
[Refile], and [Active Storage] can make managing image thumbnails a lot easier.
|
190
|
-
|
191
|
-
However, there are many specialized solutions that provide on-the-fly
|
192
|
-
processing functionality, both open source and commercial, and it's fairly easy
|
193
|
-
to apply them to files uploaded by Shrine.
|
194
|
-
|
195
|
-
```rb
|
196
|
-
def thumbnail_url(uploaded_file, dimensions)
|
197
|
-
Dragonfly.app
|
198
|
-
.fetch(uploaded_file.url)
|
199
|
-
.thumb(dimensions)
|
200
|
-
.url
|
201
|
-
end
|
202
|
-
```
|
203
|
-
```rb
|
204
|
-
thumbnail_url(photo.image, "500x400") #=> "/attachments/W1siZnUiLCJodHRwOi8vd3d3LnB1YmxpY2RvbWFpbn..."
|
205
|
-
```
|
206
|
-
|
207
189
|
## Metadata
|
208
190
|
|
209
191
|
Shrine automatically [extracts metadata][metadata] from each uploaded file,
|
@@ -227,35 +209,38 @@ photo.image.metadata #=>
|
|
227
209
|
|
228
210
|
## Direct Uploads
|
229
211
|
|
230
|
-
Instead of submitting selected files synchronously via the form, it's
|
231
|
-
start uploading files asynchronously as soon as they're selected.
|
232
|
-
streamlines this workflow, allowing you to upload directly [to your
|
212
|
+
Instead of submitting selected files synchronously via the form, it's generally
|
213
|
+
better to start uploading files asynchronously as soon as they're selected.
|
214
|
+
Shrine streamlines this workflow, allowing you to upload directly [to your
|
233
215
|
app][upload_endpoint] or [to the cloud][presign_endpoint].
|
234
216
|
|
235
217
|
[Refile] and [Active Storage] provide this functionality as well, and they also
|
236
218
|
ship with a custom plug-and-play JavaScript solution for integrating these
|
237
|
-
endpoints. In contrast, Shrine doesn't ship with any custom JavaScript but
|
219
|
+
endpoints. In contrast, Shrine doesn't ship with any custom JavaScript, but
|
238
220
|
instead recommends using **[Uppy]**. Uppy is a flexible JavaScript file upload
|
239
221
|
library that allows uploading to a [custom endpoint][XHRUpload], to [AWS
|
240
|
-
S3][AwsS3], or even to a [resumable endpoint][Tus]
|
241
|
-
|
222
|
+
S3][AwsS3], or even to a [resumable endpoint][Tus]. It comes with a set of UI
|
223
|
+
components, ranging from a simple [status bar][StatusBar] to a full-featured
|
242
224
|
[dashboard][Dashboard]. Since Uppy is maintained by the whole JavaScript
|
243
225
|
community, it will generally be better than any homegrown solution.
|
244
226
|
|
245
227
|
## Backgrounding
|
246
228
|
|
247
|
-
|
248
|
-
|
249
|
-
is supported via the `backgrounding` plugin
|
250
|
-
|
229
|
+
In most file upload solutions background processing was an afterthought, which
|
230
|
+
resulted in complex implementations. Shrine was designed with backgrounding
|
231
|
+
feature in mind from day one. It is supported via the `backgrounding` plugin
|
232
|
+
and can be used with [any backgrounding library][backgrounding libraries].
|
251
233
|
|
252
234
|
## Large Files
|
253
235
|
|
254
|
-
|
255
|
-
|
256
|
-
|
257
|
-
|
258
|
-
|
236
|
+
If your application needs to handle large files (such as videos), Shrine will
|
237
|
+
go out of the way to make this as resilient and performant as possible.
|
238
|
+
|
239
|
+
### Streaming
|
240
|
+
|
241
|
+
Shrine uses and encourages streaming uploads and downloads, where only a small
|
242
|
+
part of the file is loaded into memory at any given time. This means that
|
243
|
+
Shrine will use very little memory regardless of the size of the files.
|
259
244
|
|
260
245
|
Shrine storages also automatically support [partial downloads][Down streaming]
|
261
246
|
(provided by the [Down] gem), which allows you to read only a portion of the
|
@@ -265,22 +250,25 @@ file, so it's enough to download just the first few kilobytes of the file.
|
|
265
250
|
|
266
251
|
### Resumable uploads
|
267
252
|
|
268
|
-
Another challenge with large files is that it can be difficult for users
|
269
|
-
to upload them to your app, especially on flaky internet connections.
|
270
|
-
|
271
|
-
the
|
253
|
+
Another challenge with large files is that it can be difficult for your users
|
254
|
+
to upload them to your app, especially on flaky internet connections. Since by
|
255
|
+
default an upload is made in a single long HTTP request, any connection
|
256
|
+
failures will cause the upload to fail and have to be restarted from the
|
257
|
+
beginning.
|
272
258
|
|
273
259
|
To fix this problem, [Transloadit] company has created an open HTTP-based
|
274
260
|
protocol for resumable uploads – **[tus]**. To use it, you can choose from
|
275
261
|
numerous client and server [implementations][tus implementations] of the
|
276
|
-
protocol. In
|
277
|
-
|
278
|
-
|
279
|
-
|
262
|
+
protocol. In a typical app you would have a [JavaScript client][tus-js-client]
|
263
|
+
(via [Uppy][uppy tus]) upload to a [Ruby server][tus-ruby-server], and then
|
264
|
+
attach uploaded files using the handy [Shrine integration][shrine-tus].
|
265
|
+
|
266
|
+
Alternatively, you can have [resumable multipart uploads directly to
|
267
|
+
S3][uppy-s3_multipart].
|
280
268
|
|
281
269
|
## Security
|
282
270
|
|
283
|
-
It's important to care about security when handling file uploads, and
|
271
|
+
It's [important][OWASP] to care about security when handling file uploads, and
|
284
272
|
Shrine bakes in many good practices. For starters, it uses a separate
|
285
273
|
"temporary" storage for direct uploads, making it easy to periodically clear
|
286
274
|
uploads that didn't end up being attached and difficult for the attacker to
|
@@ -324,15 +312,15 @@ limit.
|
|
324
312
|
[mime-types]: https://github.com/mime-types/ruby-mime-types
|
325
313
|
[mini_mime]: https://github.com/discourse/mini_mime
|
326
314
|
[MiniMagick]: https://github.com/minimagick/minimagick
|
327
|
-
[ruby-vips]: https://github.com/
|
315
|
+
[ruby-vips]: https://github.com/libvips/ruby-vips
|
328
316
|
[God objects]: https://en.wikipedia.org/wiki/God_object
|
329
317
|
[ImageMagick]: https://www.imagemagick.org
|
330
318
|
[refile-mini_magick]: https://github.com/refile/refile-mini_magick
|
331
319
|
[ImageProcessing]: https://github.com/janko-m/image_processing
|
332
320
|
[auto orienting]: https://www.imagemagick.org/script/command-line-options.php#auto-orient
|
333
321
|
[sharpening]: https://photography.tutsplus.com/tutorials/what-is-image-sharpening--cms-26627
|
334
|
-
[libvips]: http://
|
335
|
-
[Why is libvips quick]: https://github.com/
|
322
|
+
[libvips]: http://libvips.github.io/libvips/
|
323
|
+
[Why is libvips quick]: https://github.com/libvips/libvips/wiki/Why-is-libvips-quick
|
336
324
|
[metadata]: https://shrinerb.com/rdoc/files/doc/metadata_md.html
|
337
325
|
[store_dimensions]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/StoreDimensions.html
|
338
326
|
[add_metadata]: https://shrinerb.com/rdoc/classes/Shrine/Plugins/AddMetadata.html
|
@@ -356,3 +344,5 @@ limit.
|
|
356
344
|
[tus-ruby-server]: https://github.com/janko-m/tus-ruby-server
|
357
345
|
[shrine-tus]: https://github.com/shrinerb/shrine-tus
|
358
346
|
[ImageTragick]: https://imagetragick.com
|
347
|
+
[uppy-s3_multipart]: https://github.com/janko-m/uppy-s3_multipart
|
348
|
+
[OWASP]: https://www.owasp.org/index.php/Unrestricted_File_Upload
|