uppy-s3_multipart 0.3.2 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +88 -98
- data/lib/uppy/s3_multipart/app.rb +8 -9
- data/lib/uppy/s3_multipart/client.rb +1 -7
- data/uppy-s3_multipart.gemspec +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dc97a26ba80d456081aa3972306acf87ab3b16a2689da06abf0351ea328c4ca6
|
4
|
+
data.tar.gz: 20f0a190ff1a875858cd69dd06d3505292c918b3af2056838d28cad554828c72
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eed1595f31dc9ac39e9ecf4f52f38608c3c1c9ed04902618d05782d63092f1af6933a700ecf24b1fca8217af327367ff93b4510ef16167684662f354d0ab3821
|
7
|
+
data.tar.gz: dbfe414174143758540e1077b65d611e5bd846132c0d4b609959d93d3d7fe73693bd2e5dfa995ddc0f3c84170018b8fb44ed1b4594c26696ae6438aa77b4c7e3
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Uppy::S3Multipart
|
2
2
|
|
3
|
-
Provides a Rack application that implements endpoints for the [
|
3
|
+
Provides a Rack application that implements endpoints for the [aws-s3-multipart]
|
4
4
|
Uppy plugin. This enables multipart uploads directly to S3, which is
|
5
5
|
recommended when dealing with large files, as it allows resuming interrupted
|
6
6
|
uploads.
|
@@ -10,12 +10,12 @@ uploads.
|
|
10
10
|
Add the gem to your Gemfile:
|
11
11
|
|
12
12
|
```rb
|
13
|
-
gem "uppy-s3_multipart", "~> 0.
|
13
|
+
gem "uppy-s3_multipart", "~> 0.3"
|
14
14
|
```
|
15
15
|
|
16
16
|
## Setup
|
17
17
|
|
18
|
-
In order to allow direct multipart uploads to your S3 bucket,
|
18
|
+
In order to allow direct multipart uploads to your S3 bucket, you need to update
|
19
19
|
the bucket's CORS configuration. In the AWS S3 Console go to your bucket, click
|
20
20
|
on "Permissions" tab and then on "CORS configuration". There paste in the
|
21
21
|
following:
|
@@ -51,11 +51,63 @@ CORS settings to be applied.
|
|
51
51
|
|
52
52
|
This gem provides a Rack application that you can mount inside your main
|
53
53
|
application. If you're using [Shrine], you can initialize the Rack application
|
54
|
-
via the
|
55
|
-
|
54
|
+
via the [Shrine plugin](#shrine).
|
55
|
+
|
56
|
+
### App
|
57
|
+
|
58
|
+
At its core, you initialize an `Uppy::S3Multipart::App` with an
|
59
|
+
`Aws::S3::Bucket` object:
|
60
|
+
|
61
|
+
```rb
|
62
|
+
require "uppy/s3_multipart"
|
63
|
+
|
64
|
+
bucket = Aws::S3::Bucket.new(
|
65
|
+
name: "my-bucket",
|
66
|
+
access_key_id: "...",
|
67
|
+
secret_access_key: "...",
|
68
|
+
region: "...",
|
69
|
+
)
|
70
|
+
|
71
|
+
UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
|
72
|
+
```
|
73
|
+
|
74
|
+
The instance of `Uppy::S3Multipart::App` is a Rack application that can be
|
75
|
+
mounted in your router (`config/routes.rb` in Rails). It should be
|
76
|
+
mounted at `/s3/multipart`:
|
77
|
+
|
78
|
+
```rb
|
79
|
+
# config/routes.rb (Rails)
|
80
|
+
Rails.application.routes.draw do
|
81
|
+
# ...
|
82
|
+
mount UPPY_S3_MULTIPART_APP => "/s3/multipart"
|
83
|
+
end
|
84
|
+
```
|
85
|
+
|
86
|
+
This will add the routes that the `aws-s3-multipart` Uppy plugin expects:
|
87
|
+
|
88
|
+
```
|
89
|
+
POST /s3/multipart
|
90
|
+
GET /s3/multipart/:uploadId
|
91
|
+
GET /s3/multipart/:uploadId/:partNumber
|
92
|
+
POST /s3/multipart/:uploadId/complete
|
93
|
+
DELETE /s3/multipart/:uploadId
|
94
|
+
```
|
95
|
+
|
96
|
+
Since your app will now play the role of Uppy Companion, in your Uppy
|
97
|
+
configuration you can point `companionUrl` to your app's URL:
|
98
|
+
|
99
|
+
```js
|
100
|
+
// ...
|
101
|
+
uppy.use(Uppy.AwsS3Multipart, {
|
102
|
+
companionUrl: '/',
|
103
|
+
})
|
104
|
+
```
|
56
105
|
|
57
106
|
### Shrine
|
58
107
|
|
108
|
+
If you're using Shrine, you can use the `uppy_s3_multipart` Shrine plugin that
|
109
|
+
ships with this gem to simplify the setup.
|
110
|
+
|
59
111
|
In your Shrine initializer load the `uppy_s3_multipart` plugin:
|
60
112
|
|
61
113
|
```rb
|
@@ -71,34 +123,29 @@ Shrine.storages = {
|
|
71
123
|
Shrine.plugin :uppy_s3_multipart # load the plugin
|
72
124
|
```
|
73
125
|
|
74
|
-
The plugin will provide a `Shrine.uppy_s3_multipart` method that creates
|
75
|
-
`Uppy::S3Multipart::App` instance, which
|
76
|
-
inside your main application:
|
126
|
+
The plugin will provide a `Shrine.uppy_s3_multipart` method that creates the
|
127
|
+
`Uppy::S3Multipart::App` instance, which you can then mount inside your router:
|
77
128
|
|
78
129
|
```rb
|
79
|
-
#
|
130
|
+
# config/routes.rb (Rails)
|
80
131
|
Rails.application.routes.draw do
|
132
|
+
# ...
|
81
133
|
mount Shrine.uppy_s3_multipart(:cache) => "/s3/multipart"
|
82
134
|
end
|
83
|
-
|
84
|
-
# Rack (config.ru)
|
85
|
-
map "/s3/multipart" do
|
86
|
-
run Shrine.uppy_s3_multipart(:cache)
|
87
|
-
end
|
88
135
|
```
|
89
136
|
|
90
|
-
Now in your Uppy configuration point `
|
137
|
+
Now in your Uppy configuration point `companionUrl` to your app's URL:
|
91
138
|
|
92
139
|
```js
|
93
140
|
// ...
|
94
141
|
uppy.use(Uppy.AwsS3Multipart, {
|
95
|
-
|
142
|
+
companionUrl: '/',
|
96
143
|
})
|
97
144
|
```
|
98
145
|
|
99
|
-
In the `upload-success` Uppy callback you can
|
100
|
-
|
101
|
-
|
146
|
+
In the `upload-success` Uppy callback, you can construct the Shrine uploaded
|
147
|
+
file data (this example assumes your temporary Shrine S3 storage has `prefix:
|
148
|
+
"cache"` set):
|
102
149
|
|
103
150
|
```js
|
104
151
|
uppy.on('upload-success', function (file, response) {
|
@@ -115,108 +162,57 @@ uppy.on('upload-success', function (file, response) {
|
|
115
162
|
})
|
116
163
|
```
|
117
164
|
|
118
|
-
|
119
|
-
Shrine. From there you can swap the `presign_endpoint` + `
|
120
|
-
`uppy_s3_multipart` + `
|
165
|
+
See [Adding Direct S3 Uploads] for an example of a complete Uppy setup with
|
166
|
+
Shrine. From there you can swap the `presign_endpoint` + `aws-s3` code with the
|
167
|
+
`uppy_s3_multipart` + `aws-s3-multipart` setup.
|
121
168
|
|
122
169
|
Note that **Shrine won't extract metadata from directly upload files on
|
123
170
|
assignment** by default. Instead, it will just copy metadata that was extracted
|
124
171
|
on the client side. See [this section][metadata direct uploads] for the
|
125
172
|
rationale and instructions on how to opt in.
|
126
173
|
|
127
|
-
|
128
|
-
|
129
|
-
You can also use `uppy-s3_multipart` without Shrine, by initializing the
|
130
|
-
`Uppy::S3Multipart::App` directly:
|
131
|
-
|
132
|
-
```rb
|
133
|
-
require "uppy/s3_multipart"
|
134
|
-
|
135
|
-
resource = Aws::S3::Resource.new(
|
136
|
-
access_key_id: "...",
|
137
|
-
secret_access_key: "...",
|
138
|
-
region: "...",
|
139
|
-
)
|
140
|
-
|
141
|
-
bucket = resource.bucket("my-bucket")
|
142
|
-
|
143
|
-
UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
|
144
|
-
```
|
145
|
-
|
146
|
-
You can mount it inside your main app in the same way:
|
147
|
-
|
148
|
-
```rb
|
149
|
-
# Rails (config/routes.rb)
|
150
|
-
Rails.application.routes.draw do
|
151
|
-
mount UPPY_S3_MULTIPART_APP => "/s3/multipart"
|
152
|
-
end
|
153
|
-
|
154
|
-
# Rack (config.ru)
|
155
|
-
map "/s3/multipart" do
|
156
|
-
run UPPY_S3_MULTIPART_APP
|
157
|
-
end
|
158
|
-
```
|
159
|
-
|
160
|
-
This will add the routes that the `AwsS3Multipart` Uppy plugin expects:
|
161
|
-
|
162
|
-
```
|
163
|
-
POST /s3/multipart
|
164
|
-
GET /s3/multipart/:uploadId
|
165
|
-
GET /s3/multipart/:uploadId/:partNumber
|
166
|
-
POST /s3/multipart/:uploadId/complete
|
167
|
-
DELETE /s3/multipart/:uploadId
|
168
|
-
```
|
169
|
-
|
170
|
-
Now in your Uppy configuration point `serverUrl` to your app's URL:
|
171
|
-
|
172
|
-
```js
|
173
|
-
// ...
|
174
|
-
uppy.use(Uppy.AwsS3Multipart, {
|
175
|
-
serverUrl: '/',
|
176
|
-
})
|
177
|
-
```
|
178
|
-
|
179
|
-
### Configuration
|
174
|
+
## Configuration
|
180
175
|
|
181
176
|
This section describe various configuration options that you can pass to
|
182
177
|
`Uppy::S3Multipart::App`.
|
183
178
|
|
184
179
|
#### `:bucket`
|
185
180
|
|
186
|
-
The `:bucket` option is mandatory and accepts an instance of `Aws::S3::Bucket
|
187
|
-
It's easiest to create an `Aws::S3::Resource`, and call `#bucket` on it.
|
181
|
+
The `:bucket` option is mandatory and accepts an instance of `Aws::S3::Bucket`:
|
188
182
|
|
189
183
|
```rb
|
190
184
|
require "uppy/s3_multipart"
|
191
185
|
|
192
|
-
|
186
|
+
bucket = Aws::S3::Bucket.new(
|
187
|
+
name: "<BUCKET>",
|
193
188
|
access_key_id: "<ACCESS_KEY_ID>",
|
194
189
|
secret_access_key: "<SECRET_ACCESS_KEY>",
|
195
190
|
region: "<REGION>",
|
196
191
|
)
|
197
192
|
|
198
|
-
bucket
|
199
|
-
|
200
|
-
Uppy::S3MUltipart::App.new(bucket: bucket)
|
193
|
+
Uppy::S3Multipart::App.new(bucket: bucket)
|
201
194
|
```
|
202
195
|
|
203
|
-
If you want to use [Minio], you can easily configure
|
204
|
-
|
196
|
+
If you want to use [Minio], you can easily configure the `Aws::S3::Bucket` to
|
197
|
+
use your Minio server:
|
205
198
|
|
206
199
|
```rb
|
207
|
-
|
200
|
+
bucket = Aws::S3::Bucket.new(
|
201
|
+
name: "<MINIO_BUCKET>",
|
208
202
|
access_key_id: "<MINIO_ACCESS_KEY>", # "AccessKey" value
|
209
203
|
secret_access_key: "<MINIO_SECRET_KEY>", # "SecretKey" value
|
210
204
|
endpoint: "<MINIO_ENDPOINT>", # "Endpoint" value
|
211
205
|
region: "us-east-1",
|
212
|
-
force_path_style: true,
|
213
206
|
)
|
214
207
|
|
215
|
-
bucket
|
208
|
+
Uppy::S3Multipart::App.new(bucket: bucket)
|
216
209
|
```
|
217
210
|
|
218
|
-
|
219
|
-
|
211
|
+
Except for `:name`, all options passed to [`Aws::S3::Bucket#initialize`] are
|
212
|
+
forwarded to [`Aws::S3::Client#initialize`], see its documentation for
|
213
|
+
additional options.
|
214
|
+
|
215
|
+
In the Shrine plugin this configuration is inferred from the S3 storage.
|
220
216
|
|
221
217
|
#### `:prefix`
|
222
218
|
|
@@ -227,14 +223,7 @@ to be uploaded to.
|
|
227
223
|
Uppy::S3Multipart::App.new(bucket: bucket, prefix: "cache")
|
228
224
|
```
|
229
225
|
|
230
|
-
In the Shrine plugin this option is inferred from the S3 storage
|
231
|
-
|
232
|
-
```rb
|
233
|
-
Shrine.storages = {
|
234
|
-
cache: Shrine::Storage::S3.new(prefix: "cache", **options),
|
235
|
-
store: Shrine::Storage::S3.new(**options),
|
236
|
-
}
|
237
|
-
```
|
226
|
+
In the Shrine plugin this option is inferred from the S3 storage.
|
238
227
|
|
239
228
|
#### `:options`
|
240
229
|
|
@@ -326,7 +315,7 @@ Shrine.storages = {
|
|
326
315
|
}
|
327
316
|
```
|
328
317
|
|
329
|
-
|
318
|
+
## Client
|
330
319
|
|
331
320
|
If you would rather implement the endpoints yourself, you can utilize the
|
332
321
|
`Uppy::S3Multipart::Client` to make S3 requests.
|
@@ -475,13 +464,14 @@ Covenant](http://contributor-covenant.org) code of conduct.
|
|
475
464
|
The gem is available as open source under the terms of the [MIT
|
476
465
|
License](https://opensource.org/licenses/MIT).
|
477
466
|
|
478
|
-
[
|
467
|
+
[aws-s3-multipart]: https://uppy.io/docs/aws-s3-multipart/
|
479
468
|
[Shrine]: https://shrinerb.com
|
480
469
|
[Adding Direct S3 Uploads]: https://github.com/shrinerb/shrine/wiki/Adding-Direct-S3-Uploads
|
481
470
|
[Minio]: https://minio.io/
|
482
471
|
[Client]: #client
|
483
472
|
[content_disposition]: https://github.com/shrinerb/content_disposition
|
484
473
|
[`Rack::Request`]: https://www.rubydoc.info/github/rack/rack/master/Rack/Request
|
474
|
+
[`Aws::S3::Bucket#initialize`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Bucket.html#initialize-instance_method
|
485
475
|
[`Aws::S3::Client#initialize`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
|
486
476
|
[`Aws::S3::Client#create_multipart_upload`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#create_multipart_upload-instance_method
|
487
477
|
[`Aws::S3::Client#list_parts`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_parts-instance_method
|
@@ -38,17 +38,16 @@ module Uppy
|
|
38
38
|
route do |r|
|
39
39
|
# POST /s3/multipart
|
40
40
|
r.post ["", true] do
|
41
|
-
|
42
|
-
filename
|
41
|
+
type = r.params["type"]
|
42
|
+
filename = r.params["filename"]
|
43
43
|
|
44
|
-
|
45
|
-
key
|
46
|
-
key = "#{opts[:prefix]}/#{key}" if opts[:prefix]
|
44
|
+
key = SecureRandom.hex + File.extname(filename.to_s)
|
45
|
+
key = [*opts[:prefix], key].join("/")
|
47
46
|
|
48
|
-
|
49
|
-
|
50
|
-
options =
|
51
|
-
options[:acl]
|
47
|
+
options = {}
|
48
|
+
options[:content_type] = type if type
|
49
|
+
options[:content_disposition] = ContentDisposition.inline(filename) if filename
|
50
|
+
options[:acl] = "public-read" if opts[:public]
|
52
51
|
|
53
52
|
result = client_call(:create_multipart_upload, key: key, **options)
|
54
53
|
|
@@ -56,13 +56,7 @@ module Uppy
|
|
56
56
|
|
57
57
|
def abort_multipart_upload(upload_id:, key:, **options)
|
58
58
|
multipart_upload = multipart_upload(upload_id, key)
|
59
|
-
|
60
|
-
# aws-sdk-s3 docs recommend retrying the abort in case the multipart
|
61
|
-
# upload still has parts
|
62
|
-
loop do
|
63
|
-
multipart_upload.abort(**options)
|
64
|
-
break unless multipart_upload.parts.any?
|
65
|
-
end
|
59
|
+
multipart_upload.abort(**options)
|
66
60
|
|
67
61
|
{}
|
68
62
|
end
|
data/uppy-s3_multipart.gemspec
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: uppy-s3_multipart
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Janko Marohnić
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-01-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: roda
|
@@ -176,7 +176,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
176
176
|
- !ruby/object:Gem::Version
|
177
177
|
version: '0'
|
178
178
|
requirements: []
|
179
|
-
rubygems_version: 3.
|
179
|
+
rubygems_version: 3.1.1
|
180
180
|
signing_key:
|
181
181
|
specification_version: 4
|
182
182
|
summary: Provides a Rack application that implements endpoints for the AwsS3Multipart
|