uppy-s3_multipart 0.3.2 → 0.3.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +88 -98
- data/lib/uppy/s3_multipart/app.rb +8 -9
- data/lib/uppy/s3_multipart/client.rb +1 -7
- data/uppy-s3_multipart.gemspec +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dc97a26ba80d456081aa3972306acf87ab3b16a2689da06abf0351ea328c4ca6
|
4
|
+
data.tar.gz: 20f0a190ff1a875858cd69dd06d3505292c918b3af2056838d28cad554828c72
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eed1595f31dc9ac39e9ecf4f52f38608c3c1c9ed04902618d05782d63092f1af6933a700ecf24b1fca8217af327367ff93b4510ef16167684662f354d0ab3821
|
7
|
+
data.tar.gz: dbfe414174143758540e1077b65d611e5bd846132c0d4b609959d93d3d7fe73693bd2e5dfa995ddc0f3c84170018b8fb44ed1b4594c26696ae6438aa77b4c7e3
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Uppy::S3Multipart
|
2
2
|
|
3
|
-
Provides a Rack application that implements endpoints for the [
|
3
|
+
Provides a Rack application that implements endpoints for the [aws-s3-multipart]
|
4
4
|
Uppy plugin. This enables multipart uploads directly to S3, which is
|
5
5
|
recommended when dealing with large files, as it allows resuming interrupted
|
6
6
|
uploads.
|
@@ -10,12 +10,12 @@ uploads.
|
|
10
10
|
Add the gem to your Gemfile:
|
11
11
|
|
12
12
|
```rb
|
13
|
-
gem "uppy-s3_multipart", "~> 0.
|
13
|
+
gem "uppy-s3_multipart", "~> 0.3"
|
14
14
|
```
|
15
15
|
|
16
16
|
## Setup
|
17
17
|
|
18
|
-
In order to allow direct multipart uploads to your S3 bucket,
|
18
|
+
In order to allow direct multipart uploads to your S3 bucket, you need to update
|
19
19
|
the bucket's CORS configuration. In the AWS S3 Console go to your bucket, click
|
20
20
|
on "Permissions" tab and then on "CORS configuration". There paste in the
|
21
21
|
following:
|
@@ -51,11 +51,63 @@ CORS settings to be applied.
|
|
51
51
|
|
52
52
|
This gem provides a Rack application that you can mount inside your main
|
53
53
|
application. If you're using [Shrine], you can initialize the Rack application
|
54
|
-
via the
|
55
|
-
|
54
|
+
via the [Shrine plugin](#shrine).
|
55
|
+
|
56
|
+
### App
|
57
|
+
|
58
|
+
At its core, you initialize an `Uppy::S3Multipart::App` with an
|
59
|
+
`Aws::S3::Bucket` object:
|
60
|
+
|
61
|
+
```rb
|
62
|
+
require "uppy/s3_multipart"
|
63
|
+
|
64
|
+
bucket = Aws::S3::Bucket.new(
|
65
|
+
name: "my-bucket",
|
66
|
+
access_key_id: "...",
|
67
|
+
secret_access_key: "...",
|
68
|
+
region: "...",
|
69
|
+
)
|
70
|
+
|
71
|
+
UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
|
72
|
+
```
|
73
|
+
|
74
|
+
The instance of `Uppy::S3Multipart::App` is a Rack application that can be
|
75
|
+
mounted in your router (`config/routes.rb` in Rails). It should be
|
76
|
+
mounted at `/s3/multipart`:
|
77
|
+
|
78
|
+
```rb
|
79
|
+
# config/routes.rb (Rails)
|
80
|
+
Rails.application.routes.draw do
|
81
|
+
# ...
|
82
|
+
mount UPPY_S3_MULTIPART_APP => "/s3/multipart"
|
83
|
+
end
|
84
|
+
```
|
85
|
+
|
86
|
+
This will add the routes that the `aws-s3-multipart` Uppy plugin expects:
|
87
|
+
|
88
|
+
```
|
89
|
+
POST /s3/multipart
|
90
|
+
GET /s3/multipart/:uploadId
|
91
|
+
GET /s3/multipart/:uploadId/:partNumber
|
92
|
+
POST /s3/multipart/:uploadId/complete
|
93
|
+
DELETE /s3/multipart/:uploadId
|
94
|
+
```
|
95
|
+
|
96
|
+
Since your app will now play the role of Uppy Companion, in your Uppy
|
97
|
+
configuration you can point `companionUrl` to your app's URL:
|
98
|
+
|
99
|
+
```js
|
100
|
+
// ...
|
101
|
+
uppy.use(Uppy.AwsS3Multipart, {
|
102
|
+
companionUrl: '/',
|
103
|
+
})
|
104
|
+
```
|
56
105
|
|
57
106
|
### Shrine
|
58
107
|
|
108
|
+
If you're using Shrine, you can use the `uppy_s3_multipart` Shrine plugin that
|
109
|
+
ships with this gem to simplify the setup.
|
110
|
+
|
59
111
|
In your Shrine initializer load the `uppy_s3_multipart` plugin:
|
60
112
|
|
61
113
|
```rb
|
@@ -71,34 +123,29 @@ Shrine.storages = {
|
|
71
123
|
Shrine.plugin :uppy_s3_multipart # load the plugin
|
72
124
|
```
|
73
125
|
|
74
|
-
The plugin will provide a `Shrine.uppy_s3_multipart` method that creates
|
75
|
-
`Uppy::S3Multipart::App` instance, which
|
76
|
-
inside your main application:
|
126
|
+
The plugin will provide a `Shrine.uppy_s3_multipart` method that creates the
|
127
|
+
`Uppy::S3Multipart::App` instance, which you can then mount inside your router:
|
77
128
|
|
78
129
|
```rb
|
79
|
-
#
|
130
|
+
# config/routes.rb (Rails)
|
80
131
|
Rails.application.routes.draw do
|
132
|
+
# ...
|
81
133
|
mount Shrine.uppy_s3_multipart(:cache) => "/s3/multipart"
|
82
134
|
end
|
83
|
-
|
84
|
-
# Rack (config.ru)
|
85
|
-
map "/s3/multipart" do
|
86
|
-
run Shrine.uppy_s3_multipart(:cache)
|
87
|
-
end
|
88
135
|
```
|
89
136
|
|
90
|
-
Now in your Uppy configuration point `
|
137
|
+
Now in your Uppy configuration point `companionUrl` to your app's URL:
|
91
138
|
|
92
139
|
```js
|
93
140
|
// ...
|
94
141
|
uppy.use(Uppy.AwsS3Multipart, {
|
95
|
-
|
142
|
+
companionUrl: '/',
|
96
143
|
})
|
97
144
|
```
|
98
145
|
|
99
|
-
In the `upload-success` Uppy callback you can
|
100
|
-
|
101
|
-
|
146
|
+
In the `upload-success` Uppy callback, you can construct the Shrine uploaded
|
147
|
+
file data (this example assumes your temporary Shrine S3 storage has `prefix:
|
148
|
+
"cache"` set):
|
102
149
|
|
103
150
|
```js
|
104
151
|
uppy.on('upload-success', function (file, response) {
|
@@ -115,108 +162,57 @@ uppy.on('upload-success', function (file, response) {
|
|
115
162
|
})
|
116
163
|
```
|
117
164
|
|
118
|
-
|
119
|
-
Shrine. From there you can swap the `presign_endpoint` + `
|
120
|
-
`uppy_s3_multipart` + `
|
165
|
+
See [Adding Direct S3 Uploads] for an example of a complete Uppy setup with
|
166
|
+
Shrine. From there you can swap the `presign_endpoint` + `aws-s3` code with the
|
167
|
+
`uppy_s3_multipart` + `aws-s3-multipart` setup.
|
121
168
|
|
122
169
|
Note that **Shrine won't extract metadata from directly upload files on
|
123
170
|
assignment** by default. Instead, it will just copy metadata that was extracted
|
124
171
|
on the client side. See [this section][metadata direct uploads] for the
|
125
172
|
rationale and instructions on how to opt in.
|
126
173
|
|
127
|
-
|
128
|
-
|
129
|
-
You can also use `uppy-s3_multipart` without Shrine, by initializing the
|
130
|
-
`Uppy::S3Multipart::App` directly:
|
131
|
-
|
132
|
-
```rb
|
133
|
-
require "uppy/s3_multipart"
|
134
|
-
|
135
|
-
resource = Aws::S3::Resource.new(
|
136
|
-
access_key_id: "...",
|
137
|
-
secret_access_key: "...",
|
138
|
-
region: "...",
|
139
|
-
)
|
140
|
-
|
141
|
-
bucket = resource.bucket("my-bucket")
|
142
|
-
|
143
|
-
UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
|
144
|
-
```
|
145
|
-
|
146
|
-
You can mount it inside your main app in the same way:
|
147
|
-
|
148
|
-
```rb
|
149
|
-
# Rails (config/routes.rb)
|
150
|
-
Rails.application.routes.draw do
|
151
|
-
mount UPPY_S3_MULTIPART_APP => "/s3/multipart"
|
152
|
-
end
|
153
|
-
|
154
|
-
# Rack (config.ru)
|
155
|
-
map "/s3/multipart" do
|
156
|
-
run UPPY_S3_MULTIPART_APP
|
157
|
-
end
|
158
|
-
```
|
159
|
-
|
160
|
-
This will add the routes that the `AwsS3Multipart` Uppy plugin expects:
|
161
|
-
|
162
|
-
```
|
163
|
-
POST /s3/multipart
|
164
|
-
GET /s3/multipart/:uploadId
|
165
|
-
GET /s3/multipart/:uploadId/:partNumber
|
166
|
-
POST /s3/multipart/:uploadId/complete
|
167
|
-
DELETE /s3/multipart/:uploadId
|
168
|
-
```
|
169
|
-
|
170
|
-
Now in your Uppy configuration point `serverUrl` to your app's URL:
|
171
|
-
|
172
|
-
```js
|
173
|
-
// ...
|
174
|
-
uppy.use(Uppy.AwsS3Multipart, {
|
175
|
-
serverUrl: '/',
|
176
|
-
})
|
177
|
-
```
|
178
|
-
|
179
|
-
### Configuration
|
174
|
+
## Configuration
|
180
175
|
|
181
176
|
This section describe various configuration options that you can pass to
|
182
177
|
`Uppy::S3Multipart::App`.
|
183
178
|
|
184
179
|
#### `:bucket`
|
185
180
|
|
186
|
-
The `:bucket` option is mandatory and accepts an instance of `Aws::S3::Bucket
|
187
|
-
It's easiest to create an `Aws::S3::Resource`, and call `#bucket` on it.
|
181
|
+
The `:bucket` option is mandatory and accepts an instance of `Aws::S3::Bucket`:
|
188
182
|
|
189
183
|
```rb
|
190
184
|
require "uppy/s3_multipart"
|
191
185
|
|
192
|
-
|
186
|
+
bucket = Aws::S3::Bucket.new(
|
187
|
+
name: "<BUCKET>",
|
193
188
|
access_key_id: "<ACCESS_KEY_ID>",
|
194
189
|
secret_access_key: "<SECRET_ACCESS_KEY>",
|
195
190
|
region: "<REGION>",
|
196
191
|
)
|
197
192
|
|
198
|
-
bucket
|
199
|
-
|
200
|
-
Uppy::S3MUltipart::App.new(bucket: bucket)
|
193
|
+
Uppy::S3Multipart::App.new(bucket: bucket)
|
201
194
|
```
|
202
195
|
|
203
|
-
If you want to use [Minio], you can easily configure
|
204
|
-
|
196
|
+
If you want to use [Minio], you can easily configure the `Aws::S3::Bucket` to
|
197
|
+
use your Minio server:
|
205
198
|
|
206
199
|
```rb
|
207
|
-
|
200
|
+
bucket = Aws::S3::Bucket.new(
|
201
|
+
name: "<MINIO_BUCKET>",
|
208
202
|
access_key_id: "<MINIO_ACCESS_KEY>", # "AccessKey" value
|
209
203
|
secret_access_key: "<MINIO_SECRET_KEY>", # "SecretKey" value
|
210
204
|
endpoint: "<MINIO_ENDPOINT>", # "Endpoint" value
|
211
205
|
region: "us-east-1",
|
212
|
-
force_path_style: true,
|
213
206
|
)
|
214
207
|
|
215
|
-
bucket
|
208
|
+
Uppy::S3Multipart::App.new(bucket: bucket)
|
216
209
|
```
|
217
210
|
|
218
|
-
|
219
|
-
|
211
|
+
Except for `:name`, all options passed to [`Aws::S3::Bucket#initialize`] are
|
212
|
+
forwarded to [`Aws::S3::Client#initialize`], see its documentation for
|
213
|
+
additional options.
|
214
|
+
|
215
|
+
In the Shrine plugin this configuration is inferred from the S3 storage.
|
220
216
|
|
221
217
|
#### `:prefix`
|
222
218
|
|
@@ -227,14 +223,7 @@ to be uploaded to.
|
|
227
223
|
Uppy::S3Multipart::App.new(bucket: bucket, prefix: "cache")
|
228
224
|
```
|
229
225
|
|
230
|
-
In the Shrine plugin this option is inferred from the S3 storage
|
231
|
-
|
232
|
-
```rb
|
233
|
-
Shrine.storages = {
|
234
|
-
cache: Shrine::Storage::S3.new(prefix: "cache", **options),
|
235
|
-
store: Shrine::Storage::S3.new(**options),
|
236
|
-
}
|
237
|
-
```
|
226
|
+
In the Shrine plugin this option is inferred from the S3 storage.
|
238
227
|
|
239
228
|
#### `:options`
|
240
229
|
|
@@ -326,7 +315,7 @@ Shrine.storages = {
|
|
326
315
|
}
|
327
316
|
```
|
328
317
|
|
329
|
-
|
318
|
+
## Client
|
330
319
|
|
331
320
|
If you would rather implement the endpoints yourself, you can utilize the
|
332
321
|
`Uppy::S3Multipart::Client` to make S3 requests.
|
@@ -475,13 +464,14 @@ Covenant](http://contributor-covenant.org) code of conduct.
|
|
475
464
|
The gem is available as open source under the terms of the [MIT
|
476
465
|
License](https://opensource.org/licenses/MIT).
|
477
466
|
|
478
|
-
[
|
467
|
+
[aws-s3-multipart]: https://uppy.io/docs/aws-s3-multipart/
|
479
468
|
[Shrine]: https://shrinerb.com
|
480
469
|
[Adding Direct S3 Uploads]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-S3-Uploads
|
481
470
|
[Minio]: https://minio.io/
|
482
471
|
[Client]: #client
|
483
472
|
[content_disposition]: https://github.com/shrinerb/content_disposition
|
484
473
|
[`Rack::Request`]: https://www.rubydoc.info/github/rack/rack/master/Rack/Request
|
474
|
+
[`Aws::S3::Bucket#initialize`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Bucket.html#initialize-instance_method
|
485
475
|
[`Aws::S3::Client#initialize`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
|
486
476
|
[`Aws::S3::Client#create_multipart_upload`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#create_multipart_upload-instance_method
|
487
477
|
[`Aws::S3::Client#list_parts`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_parts-instance_method
|
@@ -38,17 +38,16 @@ module Uppy
|
|
38
38
|
route do |r|
|
39
39
|
# POST /s3/multipart
|
40
40
|
r.post ["", true] do
|
41
|
-
|
42
|
-
filename
|
41
|
+
type = r.params["type"]
|
42
|
+
filename = r.params["filename"]
|
43
43
|
|
44
|
-
|
45
|
-
key
|
46
|
-
key = "#{opts[:prefix]}/#{key}" if opts[:prefix]
|
44
|
+
key = SecureRandom.hex + File.extname(filename.to_s)
|
45
|
+
key = [*opts[:prefix], key].join("/")
|
47
46
|
|
48
|
-
|
49
|
-
|
50
|
-
options =
|
51
|
-
options[:acl]
|
47
|
+
options = {}
|
48
|
+
options[:content_type] = type if type
|
49
|
+
options[:content_disposition] = ContentDisposition.inline(filename) if filename
|
50
|
+
options[:acl] = "public-read" if opts[:public]
|
52
51
|
|
53
52
|
result = client_call(:create_multipart_upload, key: key, **options)
|
54
53
|
|
@@ -56,13 +56,7 @@ module Uppy
|
|
56
56
|
|
57
57
|
def abort_multipart_upload(upload_id:, key:, **options)
|
58
58
|
multipart_upload = multipart_upload(upload_id, key)
|
59
|
-
|
60
|
-
# aws-sdk-s3 docs recommend retrying the abort in case the multipart
|
61
|
-
# upload still has parts
|
62
|
-
loop do
|
63
|
-
multipart_upload.abort(**options)
|
64
|
-
break unless multipart_upload.parts.any?
|
65
|
-
end
|
59
|
+
multipart_upload.abort(**options)
|
66
60
|
|
67
61
|
{}
|
68
62
|
end
|
data/uppy-s3_multipart.gemspec
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: uppy-s3_multipart
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Janko Marohnić
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-01-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: roda
|
@@ -176,7 +176,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
176
176
|
- !ruby/object:Gem::Version
|
177
177
|
version: '0'
|
178
178
|
requirements: []
|
179
|
-
rubygems_version: 3.
|
179
|
+
rubygems_version: 3.1.1
|
180
180
|
signing_key:
|
181
181
|
specification_version: 4
|
182
182
|
summary: Provides a Rack application that implements endpoints for the AwsS3Multipart
|