shrine 2.18.1 → 2.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of shrine might be problematic. Click here for more details.

Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +53 -1
  3. data/README.md +96 -137
  4. data/doc/advantages.md +4 -4
  5. data/doc/attacher.md +1 -2
  6. data/doc/carrierwave.md +3 -2
  7. data/doc/creating_storages.md +0 -20
  8. data/doc/design.md +1 -1
  9. data/doc/metadata.md +62 -36
  10. data/doc/paperclip.md +7 -6
  11. data/doc/plugins/data_uri.md +50 -4
  12. data/doc/plugins/derivation_endpoint.md +24 -0
  13. data/doc/plugins/determine_mime_type.md +47 -5
  14. data/doc/plugins/infer_extension.md +45 -9
  15. data/doc/plugins/instrumentation.md +170 -0
  16. data/doc/plugins/presign_endpoint.md +1 -1
  17. data/doc/plugins/pretty_location.md +23 -0
  18. data/doc/plugins/remote_url.md +59 -8
  19. data/doc/plugins/signature.md +54 -7
  20. data/doc/plugins/store_dimensions.md +69 -4
  21. data/doc/plugins/upload_endpoint.md +2 -2
  22. data/doc/plugins/validation_helpers.md +71 -29
  23. data/doc/refile.md +1 -1
  24. data/doc/release_notes/2.18.0.md +2 -2
  25. data/doc/release_notes/2.19.0.md +263 -0
  26. data/doc/storage/file_system.md +26 -8
  27. data/doc/testing.md +10 -10
  28. data/lib/shrine.rb +32 -16
  29. data/lib/shrine/attacher.rb +3 -0
  30. data/lib/shrine/attachment.rb +3 -0
  31. data/lib/shrine/plugins/add_metadata.rb +12 -16
  32. data/lib/shrine/plugins/backup.rb +2 -0
  33. data/lib/shrine/plugins/copy.rb +2 -0
  34. data/lib/shrine/plugins/data_uri.rb +56 -28
  35. data/lib/shrine/plugins/derivation_endpoint.rb +61 -27
  36. data/lib/shrine/plugins/determine_mime_type.rb +27 -5
  37. data/lib/shrine/plugins/infer_extension.rb +26 -5
  38. data/lib/shrine/plugins/instrumentation.rb +300 -0
  39. data/lib/shrine/plugins/logging.rb +2 -0
  40. data/lib/shrine/plugins/moving.rb +2 -0
  41. data/lib/shrine/plugins/pretty_location.rb +21 -12
  42. data/lib/shrine/plugins/rack_file.rb +23 -18
  43. data/lib/shrine/plugins/refresh_metadata.rb +4 -4
  44. data/lib/shrine/plugins/remote_url.rb +42 -23
  45. data/lib/shrine/plugins/signature.rb +32 -1
  46. data/lib/shrine/plugins/store_dimensions.rb +54 -9
  47. data/lib/shrine/plugins/validation_helpers.rb +148 -47
  48. data/lib/shrine/storage/file_system.rb +32 -15
  49. data/lib/shrine/storage/linter.rb +0 -13
  50. data/lib/shrine/storage/s3.rb +2 -5
  51. data/lib/shrine/uploaded_file.rb +8 -0
  52. data/lib/shrine/version.rb +2 -2
  53. data/shrine.gemspec +18 -3
  54. metadata +58 -27
@@ -0,0 +1,170 @@
1
+ # Instrumentation
2
+
3
+ The [`instrumentation`][instrumentation] plugin sends events for various
4
+ operations to a centralized notification component. In addition to that it
5
+ provides default logging for these events.
6
+
7
+ ```rb
8
+ Shrine.plugin :instrumentation
9
+ ```
10
+
11
+ By default, the notification component is assumed to be
12
+ [ActiveSupport::Notifications], but [dry-monitor] is supported as well:
13
+
14
+ ```rb
15
+ # Gemfile
16
+ gem "dry-monitor"
17
+ ```
18
+ ```rb
19
+ require "dry-monitor"
20
+
21
+ Shrine.plugin :instrumentation, notifications: Dry::Monitor::Notifications.new(:test)
22
+ ```
23
+
24
+ ## Logging
25
+
26
+ By default, the `instrumentation` plugin adds logging to the instrumented
27
+ events:
28
+
29
+ ```rb
30
+ uploaded_file = Shrine.upload(StringIO.new("file"), :store)
31
+ uploaded_file.exists?
32
+ uploaded_file.download
33
+ uploaded_file.delete
34
+ ```
35
+ ```
36
+ Metadata (32ms) – {:storage=>:store, :io=>StringIO, :uploader=>Shrine}
37
+ Upload (1523ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :io=>StringIO, :upload_options=>{}, :uploader=>Shrine}
38
+ Exists (755ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
39
+ Download (1002ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :download_options=>{}, :uploader=>Shrine}
40
+ Delete (700ms) – {:storage=>:store, :location=>"ed0e30ddec8b97813f2c1f4cfd1700b4", :uploader=>Shrine}
41
+ ```
42
+
43
+ You can choose to log only certain events, e.g. we can exclude metadata
44
+ extraction:
45
+
46
+ ```rb
47
+ Shrine.plugin :instrumentation, log_events: [
48
+ :upload,
49
+ :exists,
50
+ :download,
51
+ :delete,
52
+ ]
53
+ ```
54
+
55
+ You can also use your own log subscriber:
56
+
57
+ ```rb
58
+ Shrine.plugin :instrumentation, log_subscriber: -> (event) {
59
+ Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, **event.payload)
60
+ }
61
+ ```
62
+ ```
63
+ {"name":"metadata","duration":0,"storage":"store","io":"#<StringIO:0x00007fd1d4a1b9d8>","options":{},"uploader":"Shrine"}
64
+ {"name":"upload","duration":0,"storage":"store","location":"dbeb3c3ed664059eb41a608e54a29f54","io":"#<StringIO:0x00007fd1d4a1b9d8>","upload_options":{},"options":{"location":"dbeb3c3ed664059eb41a608e54a29f54","metadata":{"filename":null,"size":4,"mime_type":null}},"uploader":"Shrine"}
65
+ {"name":"exists","duration":0,"storage":"store","location":"dbeb3c3ed664059eb41a608e54a29f54","uploader":"Shrine"}
66
+ {"name":"download","duration":0,"storage":"store","location":"dbeb3c3ed664059eb41a608e54a29f54","download_options":{},"uploader":"Shrine"}
67
+ {"name":"delete","duration":0,"storage":"store","location":"dbeb3c3ed664059eb41a608e54a29f54","uploader":"Shrine"}
68
+ ```
69
+
70
+ Or disable logging altogether:
71
+
72
+ ```rb
73
+ Shrine.plugin :instrumentation, log_subscriber: nil
74
+ ```
75
+
76
+ ## Events
77
+
78
+ The following events are instrumented by the `instrumentation` plugin:
79
+
80
+ * [`upload.shrine`](#uploadshrine)
81
+ * [`download.shrine`](#downloadshrine)
82
+ * [`exists.shrine`](#existsshrine)
83
+ * [`delete.shrine`](#deleteshrine)
84
+ * [`metadata.shrine`](#metadatashrine)
85
+
86
+ ### upload.shrine
87
+
88
+ The `upload.shrine` event is logged on `Shrine#upload`, and contains the
89
+ following payload:
90
+
91
+ | Key | Description |
92
+ | :-- | :---- |
93
+ | `:storage` | The storage identifier |
94
+ | `:location` | The location of the uploaded file |
95
+ | `:io` | The uploaded IO object |
96
+ | `:upload_options` | Any upload options that were specified |
97
+ | `:options` | Some additional context information |
98
+ | `:uploader` | The uploader class that sent the event |
99
+
100
+ ### download.shrine
101
+
102
+ The `download.shrine` event is logged on `UploadedFile#open` (which includes
103
+ `UploadedFile#download` and `UploadedFile#stream` methods as well), and
104
+ contains the following payload:
105
+
106
+ | Key | Description |
107
+ | :-- | :---- |
108
+ | `:storage` | The storage identifier |
109
+ | `:location` | The location of the uploaded file |
110
+ | `:download_options` | Any upload options that were specified |
111
+ | `:uploader` | The uploader class that sent the event |
112
+
113
+ ### exists.shrine
114
+
115
+ The `exists.shrine` event is logged on `UploadedFile#exists?`, and contains the
116
+ following payload:
117
+
118
+ | Key | Description |
119
+ | :-- | :---- |
120
+ | `:storage` | The storage identifier |
121
+ | `:location` | The location of the uploaded file |
122
+ | `:uploader` | The uploader class that sent the event |
123
+
124
+ ### delete.shrine
125
+
126
+ The `delete.shrine` event is logged on `UploadedFile#delete`, and contains the
127
+ following payload:
128
+
129
+ | Key | Description |
130
+ | :-- | :---- |
131
+ | `:storage` | The storage identifier |
132
+ | `:location` | The location of the uploaded file |
133
+ | `:uploader` | The uploader class that sent the event |
134
+
135
+ ### metadata.shrine
136
+
137
+ The `metadata.shrine` event is logged on `Shrine#upload`, and contains the
138
+ following payload:
139
+
140
+ | Key | Description |
141
+ | :-- | :---- |
142
+ | `:storage` | The storage identifier |
143
+ | `:io` | The uploaded IO object |
144
+ | `:options` | Some additional context information |
145
+ | `:uploader` | The uploader class that sent the event |
146
+
147
+ ## API
148
+
149
+ The `instrumentation` plugin adds `Shrine.instrument` and `Shrine.subscribe`
150
+ methods:
151
+
152
+ ```rb
153
+ # sends a `my_event.shrine` event to the notifications component
154
+ Shrine.instrument(:my_event, foo: "bar") do
155
+ # do work
156
+ end
157
+ ```
158
+ ```rb
159
+ # subscribes to `my_event.shrine` events on the notifications component
160
+ Shrine.subscribe(:my_event) do |event|
161
+ event.name #=> :my_event
162
+ event.payload #=> { foo: "bar", uploader: Shrine }
163
+ event[:foo] #=> "bar"
164
+ event.duration #=> 15 (in milliseconds)
165
+ end
166
+ ```
167
+
168
+ [instrumentation]: /lib/shrine/plugins/instrumentation.rb
169
+ [ActiveSupport::Notifications]: https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html
170
+ [dry-monitor]: https://github.com/dry-rb/dry-monitor
@@ -69,7 +69,7 @@ class PresignsController < ApplicationController
69
69
  def image
70
70
  # ... we can perform authentication here ...
71
71
 
72
- set_rack_response ImageUploader.presign_response(:cache, env)
72
+ set_rack_response ImageUploader.presign_response(:cache, request.env)
73
73
  end
74
74
 
75
75
  private
@@ -28,4 +28,27 @@ plugin :pretty_location, namespace: "/"
28
28
  # "blog/user/.../493g82jf23.jpg"
29
29
  ```
30
30
 
31
+ By default, if there is a record present, the record `id` will is used in the location.
32
+ If you want to use a different identifier for the record, you can pass in
33
+ the `:identifier` option with the desired method/attribute name as the value:
34
+
35
+ ```rb
36
+ plugin :pretty_location, identifier: "uuid"
37
+ # "user/aa357797-5845-451b-8662-08eecdc9f762/profile_picture/493g82jf23.jpg"
38
+
39
+ plugin :pretty_location, identifier: :email
40
+ # "user/foo@bar.com/profile_picture/493g82jf23.jpg"
41
+ ```
42
+
43
+ For a more custom identifier logic, you can overwrite the method
44
+ `#generate_location` and call `#pretty_location` with the identifier you have
45
+ calculated.
46
+
47
+ ```rb
48
+ def generate_location(io, record: nil, **context)
49
+ identifier = record.email if record.is_a?(User)
50
+ pretty_location(io, record: record, identifier: identifier, **context)
51
+ end
52
+ ```
53
+
31
54
  [pretty_location]: /lib/shrine/plugins/pretty_location.rb
@@ -27,9 +27,9 @@ You can also use `#remote_url=` and `#remote_url` methods directly on the
27
27
  attacher.remote_url = "http://example.com/cool-image.png"
28
28
  ```
29
29
 
30
- The file will by default be downloaded using [Down], which is a wrapper around
31
- the `open-uri` standard library. Note that Down expects the given URL to be
32
- URI-encoded.
30
+ By default, the file will be downloaded using `Down.download` from the [Down]
31
+ gem. This will use the `Down::NetHttp` backend by default, which is a wrapper
32
+ around [open-uri].
33
33
 
34
34
  ## Dynamic options
35
35
 
@@ -67,16 +67,22 @@ plugin :remote_url, max_size: nil
67
67
 
68
68
  If you want to customize how the file is downloaded, you can override the
69
69
  `:downloader` parameter and provide your own implementation. For example, you
70
- can use the HTTP.rb Down backend for downloading:
70
+ can use the [http.rb] Down backend for downloading:
71
71
 
72
+ ```rb
73
+ # Gemfile
74
+ gem "http"
75
+ ```
72
76
  ```rb
73
77
  require "down/http"
74
78
 
75
- plugin :remote_url, max_size: 20*1024*1024, downloader: -> (url, max_size:, **options) do
76
- Down::Http.download(url, max_size: max_size, **options) do |http|
77
- http.follow(max_hops: 2).timeout(connect: 2, read: 2)
78
- end
79
+ down = Down::Http.new do |client|
80
+ client
81
+ .follow(max_hops: 2)
82
+ .timeout(connect: 2, read: 2)
79
83
  end
84
+
85
+ plugin :remote_url, max_size: 20*1024*1024, downloader: down.method(:download)
80
86
  ```
81
87
 
82
88
  ## Errors
@@ -107,6 +113,51 @@ load the `infer_extension` plugin to infer it from the MIME type.
107
113
  plugin :infer_extension
108
114
  ```
109
115
 
116
+ ## Instrumentation
117
+
118
+ If the `instrumentation` plugin has been loaded, the `remote_url` plugin adds
119
+ instrumentation around remote URL downloading.
120
+
121
+ ```rb
122
+ # instrumentation plugin needs to be loaded *before* remote_url
123
+ plugin :instrumentation
124
+ plugin :remote_url
125
+ ```
126
+
127
+ Downloading remote URLs will trigger a `remote_url.shrine` event with the
128
+ following payload:
129
+
130
+ | Key | Description |
131
+ | :-- | :---- |
132
+ | `:remote_url` | The remote URL string |
133
+ | `:download_options` | Any download options passed in |
134
+ | `:uploader` | The uploader class that sent the event |
135
+
136
+ A default log subscriber is added as well which logs these events:
137
+
138
+ ```
139
+ Remote URL (1550ms) – {:remote_url=>"https://example.com/image.jpg",:download_options=>{},:uploader=>Shrine}
140
+ ```
141
+
142
+ You can also use your own log subscriber:
143
+
144
+ ```rb
145
+ plugin :remote_url, log_subscriber: -> (event) {
146
+ Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, **event.payload)
147
+ }
148
+ ```
149
+ ```
150
+ {"name":"remote_url","duration":5,"remote_url":"https://example.com/image.jpg","download_options":{},"uploader":"Shrine"}
151
+ ```
152
+
153
+ Or disable logging altogether:
154
+
155
+ ```rb
156
+ plugin :remote_url, log_subscriber: nil
157
+ ```
158
+
110
159
  [remote_url]: /lib/shrine/plugins/remote_url.rb
111
160
  [Down]: https://github.com/janko/down
161
+ [open-uri]: https://ruby-doc.org/stdlib/libdoc/open-uri/rdoc/OpenURI.html
162
+ [http.rb]: https://github.com/httprb/http
112
163
  [shrine-url]: https://github.com/shrinerb/shrine-url
@@ -8,15 +8,31 @@ signature for the uploaded file.
8
8
  Shrine.plugin :signature
9
9
  ```
10
10
 
11
+ ## API
12
+
11
13
  The plugin adds a `#calculate_signature` instance and class method to the
12
14
  uploader. The method accepts an IO object and a hashing algorithm, and returns
13
15
  the calculated hash.
14
16
 
15
17
  ```rb
16
- Shrine.calculate_signature(io, :md5)
17
- #=> "9a0364b9e99bb480dd25e1f0284c8555"
18
+ Shrine.calculate_signature(io, :md5) #=> "9a0364b9e99bb480dd25e1f0284c8555"
19
+ # or just
20
+ Shrine.signature(io, :md5) #=> "9a0364b9e99bb480dd25e1f0284c8555"
18
21
  ```
19
22
 
23
+ The following hashing algorithms are supported: SHA1, SHA256, SHA384, SHA512,
24
+ MD5, and CRC32.
25
+
26
+ You can also choose which format will the calculated hash be encoded in:
27
+
28
+ ```rb
29
+ Shrine.calculate_signature(io, :sha256, format: :base64)
30
+ ```
31
+
32
+ The supported encoding formats are `hex` (default), `base64`, and `none`.
33
+
34
+ ## Adding metadata
35
+
20
36
  You can then use the `add_metadata` plugin to add a new metadata field with the
21
37
  calculated hash.
22
38
 
@@ -37,15 +53,46 @@ add_metadata :md5 do |io, context|
37
53
  end
38
54
  ```
39
55
 
40
- The following hashing algorithms are supported: SHA1, SHA256, SHA384, SHA512,
41
- MD5, and CRC32.
56
+ ## Instrumentation
42
57
 
43
- You can also choose which format will the calculated hash be encoded in:
58
+ If the `instrumentation` plugin has been loaded, the `signature` plugin adds
59
+ instrumentation around signature calculation.
44
60
 
45
61
  ```rb
46
- Shrine.calculate_signature(io, :sha256, format: :base64)
62
+ # instrumentation plugin needs to be loaded *before* signature
63
+ plugin :instrumentation
64
+ plugin :signature
47
65
  ```
48
66
 
49
- The supported encoding formats are `hex` (default), `base64`, and `none`.
67
+ Calculating signature will trigger a `signature.shrine` event with the
68
+ following payload:
69
+
70
+ | Key | Description |
71
+ | :-- | :---- |
72
+ | `:io` | The IO object |
73
+ | `:uploader` | The uploader class that sent the event |
74
+
75
+ A default log subscriber is added as well which logs these events:
76
+
77
+ ```
78
+ MIME Type (33ms) – {:io=>StringIO, :uploader=>Shrine}
79
+ ```
80
+
81
+ You can also use your own log subscriber:
82
+
83
+ ```rb
84
+ plugin :signature, log_subscriber: -> (event) {
85
+ Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, **event.payload)
86
+ }
87
+ ```
88
+ ```
89
+ {"name":"signature","duration":24,"io":"#<StringIO:0x00007fb7c5b08b80>","uploader":"Shrine"}
90
+ ```
91
+
92
+ Or disable logging altogether:
93
+
94
+ ```rb
95
+ plugin :signature, log_subscriber: nil
96
+ ```
50
97
 
51
98
  [signature]: /lib/shrine/plugins/signature.rb
@@ -8,6 +8,8 @@ uploaded images and stores them into the metadata hash (by default it uses the
8
8
  plugin :store_dimensions
9
9
  ```
10
10
 
11
+ ## Metadata
12
+
11
13
  The dimensions are stored as "width" and "height" metadata values on the
12
14
  Shrine::UploadedFile object. For convenience the plugin also adds `#width`,
13
15
  `#height` and `#dimensions` reader methods.
@@ -24,6 +26,8 @@ image.height #=> 500
24
26
  image.dimensions #=> [300, 500]
25
27
  ```
26
28
 
29
+ ## Analyzers
30
+
27
31
  By default the [fastimage] gem is used to extract dimensions. You can choose a
28
32
  different built-in analyzer via the `:analyzer` option:
29
33
 
@@ -52,16 +56,77 @@ plugin :store_dimensions, analyzer: -> (io, analyzers) do
52
56
  end
53
57
  ```
54
58
 
59
+ ## API
60
+
55
61
  You can use methods for extracting the dimensions directly:
56
62
 
57
63
  ```rb
58
64
  # or YourUploader.extract_dimensions(io)
59
- Shrine.extract_dimensions(io) # calls the defined analyzer
60
- #=> [300, 400]
65
+ Shrine.extract_dimensions(io) #=> [300, 400] (calls the defined analyzer)
66
+ # or just
67
+ Shrine.dimensions(io) #=> [300, 400] (calls the defined analyzer)
61
68
 
62
69
  # or YourUploader.dimensions_analyzers
63
- Shrine.dimensions_analyzers[:fastimage].call(io) # calls a built-in analyzer
64
- #=> [300, 400]
70
+ Shrine.dimensions_analyzers[:fastimage].call(io) #=> [300, 400] (calls a built-in analyzer)
71
+ ```
72
+
73
+ ## Errors
74
+
75
+ By default, any exceptions that the analyzer raises while extracting dimensions
76
+ will be caught and a warning will be printed out. This allows you to have the
77
+ plugin loaded even for files that are not images.
78
+
79
+ However, you can choose different strategies for handling these exceptions:
80
+
81
+ ```rb
82
+ plugin :store_dimensions, on_error: :warn # prints a warning (default)
83
+ plugin :store_dimensions, on_error: :fail # raises the exception
84
+ plugin :store_dimensions, on_error: :ignore # ignores exceptions
85
+ plugin :store_dimensions, on_error: -> (error) { # custom handler
86
+ # report the exception to your exception handler
87
+ }
88
+ ```
89
+
90
+ ## Instrumentation
91
+
92
+ If the `instrumentation` plugin has been loaded, the `store_dimensions` plugin
93
+ adds instrumentation around dimensions extraction.
94
+
95
+ ```rb
96
+ # instrumentation plugin needs to be loaded *before* store_dimensions
97
+ plugin :instrumentation
98
+ plugin :store_dimensions
99
+ ```
100
+
101
+ Extracting metadata will send a `image_dimensions.shrine` event with the
102
+ following payload:
103
+
104
+ | Key | Description |
105
+ | :-- | :---- |
106
+ | `:io` | The IO object |
107
+ | `:uploader` | The uploader class that sent the event |
108
+
109
+ A default log subscriber is added as well which logs these events:
110
+
111
+ ```
112
+ Image Dimensions (108ms) – {:io=>File, :uploader=>Shrine}
113
+ ```
114
+
115
+ You can also use your own log subscriber:
116
+
117
+ ```rb
118
+ plugin :store_dimensions, log_subscriber: -> (event) {
119
+ Shrine.logger.info JSON.generate(name: event.name, duration: event.duration, **event.payload)
120
+ }
121
+ ```
122
+ ```
123
+ {"name":"image_dimensions","duration":114,"io":"#<File:0x00007fc445371d90>","uploader":"Shrine"}
124
+ ```
125
+
126
+ Or disable logging altogether:
127
+
128
+ ```rb
129
+ plugin :store_dimensions, log_subscriber: nil
65
130
  ```
66
131
 
67
132
  [store_dimensions]: /lib/shrine/plugins/store_dimensions.rb